2020-12-05T16:42:43.671 INFO:root:teuthology version: 1.0.0-e6b33b7b 2020-12-05T16:42:43.672 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342 --verbose --description fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} --name teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi --block --lock -- /tmp/teuthology-worker.7eriqjxk.tmp 2020-12-05T16:42:43.722 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-12-05T16:42:43.789 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342 branch: octopus description: fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} email: ceph-qa@ceph.io first_in_suite: false job_id: '5667342' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 30 - machine: disk: 200 os_type: ubuntu os_version: '18.04' overrides: admin_socket: branch: octopus ceph: cephfs_ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 15m rados osd op timeout: 15m mds: debug mds: 20 debug ms: 1 mds bal frag: true mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 15m rados osd op timeout: 15m mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - evicting unresponsive client - POOL_APP_NOT_ENABLED - has not responded to cap revoke by MDS for over - MDS_CLIENT_LATE_RELEASE - responding to mclientcaps sha1: 3b1861acb8137d863184795ccadbd5d0cb026674 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: sha1: 3b1861acb8137d863184795ccadbd5d0cb026674 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: octopus sha1: 3b1861acb8137d863184795ccadbd5d0cb026674 owner: scheduled_teuthology@teuthology priority: 1000 repo: git://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 - - client.0 - - client.1 - - client.2 sha1: 3b1861acb8137d863184795ccadbd5d0cb026674 sleep_before_teardown: 0 suite: fs suite_branch: octopus suite_path: /home/teuthworker/src/git.ceph.com_ceph_octopus/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph.git suite_sha1: 3b1861acb8137d863184795ccadbd5d0cb026674 tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell rpm: - python3-cephfs extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_misc teuthology_branch: master timestamp: 2020-11-30_04:10:01 tube: smithi user: teuthology verbose: true worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.19794 2020-12-05T16:42:43.789 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_octopus/qa; will attempt to use it 2020-12-05T16:42:43.790 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks 2020-12-05T16:42:43.790 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-12-05T16:42:43.792 INFO:teuthology.task.internal:Checking packages... 2020-12-05T16:42:43.815 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'basic' and ceph hash '3b1861acb8137d863184795ccadbd5d0cb026674' 2020-12-05T16:42:43.816 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-12-05T16:42:43.817 INFO:teuthology.packaging:ref: None 2020-12-05T16:42:43.817 INFO:teuthology.packaging:tag: None 2020-12-05T16:42:43.817 INFO:teuthology.packaging:branch: octopus 2020-12-05T16:42:43.818 INFO:teuthology.packaging:sha1: 3b1861acb8137d863184795ccadbd5d0cb026674 2020-12-05T16:42:43.818 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=octopus 2020-12-05T16:42:44.098 INFO:teuthology.task.internal:Found packages for ceph version 15.2.7-615-g00b4001e-1bionic 2020-12-05T16:42:44.098 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-12-05T16:42:44.112 INFO:teuthology.task.internal:no buildpackages task found 2020-12-05T16:42:44.113 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-12-05T16:42:44.145 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-12-05T16:42:44.146 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-12-05T16:42:44.200 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:42:54.217 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:43:04.235 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:43:14.251 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:43:24.263 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:43:34.279 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:43:44.293 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:43:54.311 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:44:04.326 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:44:14.342 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:44:24.359 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:44:34.374 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:44:44.390 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:44:54.407 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:45:04.423 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:45:14.438 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 3)... 2020-12-05T16:45:24.453 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:45:34.473 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:45:44.492 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:45:54.509 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2020-12-05T16:46:04.526 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:46:14.542 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:46:24.560 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:46:34.576 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:46:44.595 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:46:54.611 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:47:04.623 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:47:14.641 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:47:24.658 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:47:34.674 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:47:44.690 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:47:54.706 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:48:04.716 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:48:14.734 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:48:24.748 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:48:34.766 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:48:44.781 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:48:54.800 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2020-12-05T16:49:04.813 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:49:14.825 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:49:24.843 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:49:34.858 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:49:44.875 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 3)... 2020-12-05T16:49:54.890 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:50:04.901 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:50:14.920 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 3)... 2020-12-05T16:50:24.936 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:50:34.955 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:50:44.970 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:50:54.987 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 3)... 2020-12-05T16:51:05.003 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:51:15.019 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:51:25.039 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:51:35.055 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:51:45.073 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:51:55.090 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:52:05.109 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2020-12-05T16:52:15.130 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:52:25.147 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:52:35.164 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:52:45.179 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:52:55.197 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:53:05.215 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:53:15.232 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:53:25.244 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:53:35.260 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:53:45.277 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:53:55.294 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:54:05.311 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:54:15.328 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:54:25.345 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:54:35.364 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:54:45.379 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:54:55.394 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:55:05.410 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:55:15.427 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:55:25.444 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T16:55:35.458 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:55:45.476 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:55:55.490 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:56:05.507 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:56:15.522 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:56:25.539 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:56:35.555 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:56:45.573 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 3)... 2020-12-05T16:56:55.591 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:57:05.607 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:57:15.623 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:57:25.638 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:57:35.656 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:57:45.670 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:57:55.686 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:58:05.704 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:58:15.723 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:58:25.746 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:58:35.765 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:58:45.782 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:58:55.803 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 3)... 2020-12-05T16:59:05.820 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:59:15.843 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:59:25.858 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:59:35.875 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:59:45.891 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T16:59:55.908 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T17:00:05.921 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T17:00:15.939 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T17:00:25.955 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2020-12-05T17:00:35.972 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2020-12-05T17:00:45.987 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2020-12-05T17:00:56.003 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:01:06.020 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:01:16.039 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:01:26.057 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:01:36.078 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:01:46.095 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:01:56.123 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:02:06.138 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:02:16.154 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:02:26.170 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2020-12-05T17:02:36.189 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2020-12-05T17:02:46.206 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:02:56.221 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:03:06.231 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2020-12-05T17:03:16.249 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2020-12-05T17:03:26.267 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:03:36.284 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:03:46.303 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:03:56.319 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2020-12-05T17:04:06.337 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2020-12-05T17:04:16.354 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2020-12-05T17:04:26.370 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T17:04:36.389 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T17:04:46.406 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T17:04:56.426 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T17:05:06.445 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T17:05:16.461 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T17:05:26.478 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T17:05:36.495 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T17:05:46.510 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T17:05:56.525 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:06:06.545 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2020-12-05T17:06:16.562 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:06:26.579 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:06:36.597 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:06:46.614 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:06:56.631 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:07:06.643 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:07:16.661 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:07:26.678 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 3)... 2020-12-05T17:07:36.695 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T17:07:46.706 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T17:07:56.723 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T17:08:06.741 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T17:08:16.759 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2020-12-05T17:08:26.776 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 4)... 2020-12-05T17:08:36.790 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:08:46.807 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:08:56.822 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:09:06.840 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:09:16.860 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2020-12-05T17:09:26.877 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:09:36.895 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2020-12-05T17:09:46.911 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:09:56.927 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:10:06.942 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:10:16.959 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 4)... 2020-12-05T17:10:26.975 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:10:36.988 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:10:47.003 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:10:57.018 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:11:07.053 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:11:17.071 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2020-12-05T17:11:27.087 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:11:37.102 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:11:47.118 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:11:57.134 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:12:07.152 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:12:17.166 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:12:27.197 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:12:37.215 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:12:47.231 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 3)... 2020-12-05T17:12:57.246 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2020-12-05T17:13:07.264 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:13:17.284 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2020-12-05T17:13:27.303 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2020-12-05T17:13:37.319 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 4)... 2020-12-05T17:13:47.338 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 4)... 2020-12-05T17:13:57.357 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 4)... 2020-12-05T17:14:07.378 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_teuthology@teuthology', 'count': 5, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342'} 2020-12-05T17:14:07.406 DEBUG:teuthology.lock.ops:locked ubuntu@smithi032.front.sepia.ceph.com, ubuntu@smithi061.front.sepia.ceph.com, ubuntu@smithi036.front.sepia.ceph.com, ubuntu@smithi002.front.sepia.ceph.com, ubuntu@smithi053.front.sepia.ceph.com 2020-12-05T17:14:07.546 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f99f7cc79d8>, signals=[15]) 2020-12-05T17:14:07.768 INFO:teuthology.provision.fog.smithi032:Scheduling deploy of ubuntu 18.04 2020-12-05T17:14:07.771 INFO:teuthology.provision.fog.smithi061:Scheduling deploy of ubuntu 18.04 2020-12-05T17:14:07.774 INFO:teuthology.provision.fog.smithi036:Scheduling deploy of ubuntu 18.04 2020-12-05T17:14:07.781 INFO:teuthology.provision.fog.smithi053:Scheduling deploy of ubuntu 18.04 2020-12-05T17:14:07.789 INFO:teuthology.provision.fog.smithi002:Scheduling deploy of ubuntu 18.04 2020-12-05T17:14:08.336 INFO:teuthology.orchestra.console:Power off smithi036 2020-12-05T17:14:08.337 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi036.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-12-05T17:14:08.363 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi036.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:08.378 INFO:teuthology.orchestra.console:Power off smithi032 2020-12-05T17:14:08.378 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-12-05T17:14:08.392 INFO:teuthology.orchestra.console:Power off smithi061 2020-12-05T17:14:08.393 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-12-05T17:14:08.408 INFO:teuthology.orchestra.console:Power off smithi002 2020-12-05T17:14:08.409 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-12-05T17:14:08.418 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:08.427 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:08.435 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:08.486 INFO:teuthology.orchestra.console:Power off smithi053 2020-12-05T17:14:08.487 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-12-05T17:14:08.512 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:12.422 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi036.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:12.451 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:12.462 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:12.474 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:12.537 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:16.547 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi036.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:16.564 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:16.576 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:16.586 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:16.648 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:16.761 INFO:teuthology.orchestra.console:Power off for smithi036 completed 2020-12-05T17:14:16.775 INFO:teuthology.orchestra.console:Power off for smithi032 completed 2020-12-05T17:14:16.796 INFO:teuthology.orchestra.console:Power off for smithi002 completed 2020-12-05T17:14:16.862 INFO:teuthology.orchestra.console:Power on smithi036 2020-12-05T17:14:16.863 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi036.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-12-05T17:14:16.876 INFO:teuthology.orchestra.console:Power on smithi032 2020-12-05T17:14:16.876 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-12-05T17:14:16.885 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi036.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:16.897 INFO:teuthology.orchestra.console:Power on smithi002 2020-12-05T17:14:16.898 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-12-05T17:14:16.908 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:16.920 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:20.690 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:20.760 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:20.902 INFO:teuthology.orchestra.console:Power off for smithi061 completed 2020-12-05T17:14:20.918 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi036.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:20.932 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:20.942 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:20.970 INFO:teuthology.orchestra.console:Power off for smithi053 completed 2020-12-05T17:14:21.004 INFO:teuthology.orchestra.console:Power on smithi061 2020-12-05T17:14:21.004 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-12-05T17:14:21.024 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:21.072 INFO:teuthology.orchestra.console:Power on smithi053 2020-12-05T17:14:21.073 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-12-05T17:14:21.093 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:25.039 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi036.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:25.048 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:25.054 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:25.063 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:25.115 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-12-05T17:14:25.257 INFO:teuthology.orchestra.console:Power on for smithi032 completed 2020-12-05T17:14:25.265 INFO:teuthology.orchestra.console:Power on for smithi061 completed 2020-12-05T17:14:25.273 INFO:teuthology.orchestra.console:Power on for smithi002 completed 2020-12-05T17:14:25.327 INFO:teuthology.orchestra.console:Power on for smithi053 completed 2020-12-05T17:14:25.337 INFO:teuthology.orchestra.console:Power on for smithi036 completed 2020-12-05T17:14:25.359 INFO:teuthology.provision.fog.smithi032:Waiting for deploy to finish 2020-12-05T17:14:25.366 INFO:teuthology.provision.fog.smithi061:Waiting for deploy to finish 2020-12-05T17:14:25.375 INFO:teuthology.provision.fog.smithi002:Waiting for deploy to finish 2020-12-05T17:14:25.428 INFO:teuthology.provision.fog.smithi053:Waiting for deploy to finish 2020-12-05T17:14:25.438 INFO:teuthology.provision.fog.smithi036:Waiting for deploy to finish 2020-12-05T17:16:57.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:16:57.021 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:16:57.121 ERROR:teuthology.orchestra.connection:Error connecting to smithi002.front.sepia.ceph.com Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/connection.py", line 105, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/client.py", line 446, in connect passphrase, File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/client.py", line 764, in _auth raise saved_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/client.py", line 740, in _auth self._transport.auth_publickey(username, key) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 1580, in auth_publickey return self.auth_handler.wait_for_response(my_event) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/auth_handler.py", line 250, in wait_for_response raise e paramiko.ssh_exception.AuthenticationException: Authentication failed. 2020-12-05T17:17:06.096 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:17:12.169 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:17:12.207 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:17:15.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:17:21.231 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:17:21.263 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:17:24.207 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:17:27.251 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:17:30.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:17:30.322 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:17:33.267 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:17:36.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:17:39.344 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:17:39.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:17:42.323 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:17:45.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:17:48.399 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:17:48.430 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:17:51.379 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:17:54.451 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:17:57.456 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:18:03.503 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:18:04.127 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:18:12.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:18:13.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:18:21.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:18:22.255 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:18:30.671 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:18:31.315 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:18:40.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:18:40.379 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-12-05T17:18:46.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:18:46.491 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-12-05T17:18:46.493 INFO:teuthology.orchestra.run.smithi002:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-12-05T17:18:47.415 INFO:teuthology.provision.fog.smithi002:Node is ready 2020-12-05T17:18:47.416 INFO:teuthology.orchestra.run.smithi002:> hostname 2020-12-05T17:18:47.463 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002.front.sepia.ceph.com 2020-12-05T17:18:47.464 INFO:teuthology.orchestra.run.smithi002:> grep smithi002.front.sepia.ceph.com /etc/hosts 2020-12-05T17:18:47.512 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2 smithi002.front.sepia.ceph.com smithi002 2020-12-05T17:18:47.512 INFO:teuthology.orchestra.run.smithi002:> sudo hostname smithi002 2020-12-05T17:18:47.580 INFO:teuthology.orchestra.run.smithi002:> sudo sed -i -e 's/smithi002.front.sepia.ceph.com/smithi002/g' /etc/hosts 2020-12-05T17:18:47.638 INFO:teuthology.orchestra.run.smithi002:> sudo sed -i -e 's/172.21.15.2/172.21.15.2/g' /etc/hosts 2020-12-05T17:18:47.696 INFO:teuthology.orchestra.run.smithi002:> sudo sed -i -e 's/smithi002.front.sepia.ceph.com/smithi002/g' /etc/hostname 2020-12-05T17:18:47.752 INFO:teuthology.orchestra.run.smithi002:> sudo hostname smithi002 2020-12-05T17:18:47.807 INFO:teuthology.provision.fog.smithi002:Deploy complete! 2020-12-05T17:18:47.808 DEBUG:teuthology.parallel:result is None 2020-12-05T17:18:54.434 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:18:54.534 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-12-05T17:18:54.534 INFO:teuthology.orchestra.run.smithi053:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-12-05T17:18:54.927 INFO:teuthology.provision.fog.smithi053:Node is ready 2020-12-05T17:18:54.927 INFO:teuthology.orchestra.run.smithi053:> hostname 2020-12-05T17:18:54.973 INFO:teuthology.orchestra.run.smithi053.stdout:smithi053.front.sepia.ceph.com 2020-12-05T17:18:54.974 INFO:teuthology.orchestra.run.smithi053:> grep smithi053.front.sepia.ceph.com /etc/hosts 2020-12-05T17:18:55.022 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53 smithi053.front.sepia.ceph.com smithi053 2020-12-05T17:18:55.023 INFO:teuthology.orchestra.run.smithi053:> sudo hostname smithi053 2020-12-05T17:18:55.118 INFO:teuthology.orchestra.run.smithi053:> sudo sed -i -e 's/smithi053.front.sepia.ceph.com/smithi053/g' /etc/hosts 2020-12-05T17:18:55.135 INFO:teuthology.orchestra.run.smithi053:> sudo sed -i -e 's/172.21.15.53/172.21.15.53/g' /etc/hosts 2020-12-05T17:18:55.192 INFO:teuthology.orchestra.run.smithi053:> sudo sed -i -e 's/smithi053.front.sepia.ceph.com/smithi053/g' /etc/hostname 2020-12-05T17:18:55.245 INFO:teuthology.orchestra.run.smithi053:> sudo hostname smithi053 2020-12-05T17:18:55.302 INFO:teuthology.provision.fog.smithi053:Deploy complete! 2020-12-05T17:18:55.303 DEBUG:teuthology.parallel:result is None 2020-12-05T17:18:57.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:18:57.499 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-12-05T17:18:57.500 INFO:teuthology.orchestra.run.smithi036:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-12-05T17:18:57.906 INFO:teuthology.provision.fog.smithi036:Node is ready 2020-12-05T17:18:57.907 INFO:teuthology.orchestra.run.smithi036:> hostname 2020-12-05T17:18:57.959 INFO:teuthology.orchestra.run.smithi036.stdout:smithi036.front.sepia.ceph.com 2020-12-05T17:18:57.959 INFO:teuthology.orchestra.run.smithi036:> grep smithi036.front.sepia.ceph.com /etc/hosts 2020-12-05T17:18:58.008 INFO:teuthology.orchestra.run.smithi036.stdout:172.21.15.36 smithi036.front.sepia.ceph.com smithi036 2020-12-05T17:18:58.009 INFO:teuthology.orchestra.run.smithi036:> sudo hostname smithi036 2020-12-05T17:18:58.091 INFO:teuthology.orchestra.run.smithi036:> sudo sed -i -e 's/smithi036.front.sepia.ceph.com/smithi036/g' /etc/hosts 2020-12-05T17:18:58.150 INFO:teuthology.orchestra.run.smithi036:> sudo sed -i -e 's/172.21.15.36/172.21.15.36/g' /etc/hosts 2020-12-05T17:18:58.204 INFO:teuthology.orchestra.run.smithi036:> sudo sed -i -e 's/smithi036.front.sepia.ceph.com/smithi036/g' /etc/hostname 2020-12-05T17:18:58.262 INFO:teuthology.orchestra.run.smithi036:> sudo hostname smithi036 2020-12-05T17:18:58.314 INFO:teuthology.provision.fog.smithi036:Deploy complete! 2020-12-05T17:18:58.315 DEBUG:teuthology.parallel:result is None 2020-12-05T17:19:02.236 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-12-05T17:19:03.458 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:19:03.575 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-12-05T17:19:03.576 INFO:teuthology.orchestra.run.smithi032:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-12-05T17:19:03.995 INFO:teuthology.provision.fog.smithi032:Node is ready 2020-12-05T17:19:03.996 INFO:teuthology.orchestra.run.smithi032:> hostname 2020-12-05T17:19:04.043 INFO:teuthology.orchestra.run.smithi032.stdout:smithi032.front.sepia.ceph.com 2020-12-05T17:19:04.044 INFO:teuthology.orchestra.run.smithi032:> grep smithi032.front.sepia.ceph.com /etc/hosts 2020-12-05T17:19:04.093 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32 smithi032.front.sepia.ceph.com smithi032 2020-12-05T17:19:04.094 INFO:teuthology.orchestra.run.smithi032:> sudo hostname smithi032 2020-12-05T17:19:04.194 INFO:teuthology.orchestra.run.smithi032:> sudo sed -i -e 's/smithi032.front.sepia.ceph.com/smithi032/g' /etc/hosts 2020-12-05T17:19:04.208 INFO:teuthology.orchestra.run.smithi032:> sudo sed -i -e 's/172.21.15.32/172.21.15.32/g' /etc/hosts 2020-12-05T17:19:04.266 INFO:teuthology.orchestra.run.smithi032:> sudo sed -i -e 's/smithi032.front.sepia.ceph.com/smithi032/g' /etc/hostname 2020-12-05T17:19:04.321 INFO:teuthology.orchestra.run.smithi032:> sudo hostname smithi032 2020-12-05T17:19:04.377 INFO:teuthology.provision.fog.smithi032:Deploy complete! 2020-12-05T17:19:04.378 DEBUG:teuthology.parallel:result is None 2020-12-05T17:19:08.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:19:08.249 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-12-05T17:19:14.251 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:19:14.261 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-12-05T17:19:20.263 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:19:20.274 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-12-05T17:19:26.276 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:19:26.287 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-12-05T17:19:32.289 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:19:32.403 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-12-05T17:19:32.404 INFO:teuthology.orchestra.run.smithi061:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-12-05T17:19:32.859 INFO:teuthology.provision.fog.smithi061:Node is ready 2020-12-05T17:19:32.859 INFO:teuthology.orchestra.run.smithi061:> hostname 2020-12-05T17:19:32.907 INFO:teuthology.orchestra.run.smithi061.stdout:smithi061.front.sepia.ceph.com 2020-12-05T17:19:32.908 INFO:teuthology.orchestra.run.smithi061:> grep smithi061.front.sepia.ceph.com /etc/hosts 2020-12-05T17:19:32.956 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61 smithi061.front.sepia.ceph.com smithi061 2020-12-05T17:19:32.957 INFO:teuthology.orchestra.run.smithi061:> sudo hostname smithi061 2020-12-05T17:19:33.114 INFO:teuthology.orchestra.run.smithi061:> sudo sed -i -e 's/smithi061.front.sepia.ceph.com/smithi061/g' /etc/hosts 2020-12-05T17:19:33.125 INFO:teuthology.orchestra.run.smithi061:> sudo sed -i -e 's/172.21.15.61/172.21.15.61/g' /etc/hosts 2020-12-05T17:19:33.176 INFO:teuthology.orchestra.run.smithi061:> sudo sed -i -e 's/smithi061.front.sepia.ceph.com/smithi061/g' /etc/hostname 2020-12-05T17:19:33.231 INFO:teuthology.orchestra.run.smithi061:> sudo hostname smithi061 2020-12-05T17:19:33.287 INFO:teuthology.provision.fog.smithi061:Deploy complete! 2020-12-05T17:19:33.288 DEBUG:teuthology.parallel:result is None 2020-12-05T17:19:33.526 INFO:teuthology.lock.ops:Checking smithi032.front.sepia.ceph.com 2020-12-05T17:19:33.527 INFO:teuthology.lock.ops:New key found. Updating... 2020-12-05T17:19:33.551 INFO:teuthology.lock.ops:Checking smithi061.front.sepia.ceph.com 2020-12-05T17:19:33.552 INFO:teuthology.lock.ops:New key found. Updating... 2020-12-05T17:19:33.565 INFO:teuthology.lock.ops:Checking smithi036.front.sepia.ceph.com 2020-12-05T17:19:33.566 INFO:teuthology.lock.ops:New key found. Updating... 2020-12-05T17:19:33.578 INFO:teuthology.lock.ops:Checking smithi002.front.sepia.ceph.com 2020-12-05T17:19:33.578 INFO:teuthology.lock.ops:New key found. Updating... 2020-12-05T17:19:33.590 INFO:teuthology.lock.ops:Checking smithi053.front.sepia.ceph.com 2020-12-05T17:19:33.591 INFO:teuthology.lock.ops:New key found. Updating... 2020-12-05T17:19:33.605 INFO:teuthology.lock.ops:Updating [smithi032.front.sepia.ceph.com]: set os type and version on server 2020-12-05T17:19:33.606 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-12-05T17:19:33.607 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:19:33.682 INFO:teuthology.orchestra.run.smithi032:> true 2020-12-05T17:19:33.931 INFO:teuthology.orchestra.run.smithi032:> uname -m 2020-12-05T17:19:33.980 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2020-12-05T17:19:33.980 INFO:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2020-12-05T17:19:34.029 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="Ubuntu" 2020-12-05T17:19:34.029 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-12-05T17:19:34.030 INFO:teuthology.orchestra.run.smithi032.stdout:ID=ubuntu 2020-12-05T17:19:34.030 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE=debian 2020-12-05T17:19:34.031 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-12-05T17:19:34.031 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="18.04" 2020-12-05T17:19:34.031 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://www.ubuntu.com/" 2020-12-05T17:19:34.032 INFO:teuthology.orchestra.run.smithi032.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-12-05T17:19:34.032 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-12-05T17:19:34.032 INFO:teuthology.orchestra.run.smithi032.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-12-05T17:19:34.033 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_CODENAME=bionic 2020-12-05T17:19:34.033 INFO:teuthology.orchestra.run.smithi032.stdout:UBUNTU_CODENAME=bionic 2020-12-05T17:19:34.034 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2020-12-05T17:19:34.051 INFO:teuthology.lock.ops:Updating [smithi061.front.sepia.ceph.com]: set os type and version on server 2020-12-05T17:19:34.051 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-12-05T17:19:34.052 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:19:34.119 INFO:teuthology.orchestra.run.smithi061:> true 2020-12-05T17:19:34.331 INFO:teuthology.orchestra.run.smithi061:> uname -m 2020-12-05T17:19:34.380 INFO:teuthology.orchestra.run.smithi061.stdout:x86_64 2020-12-05T17:19:34.380 INFO:teuthology.orchestra.run.smithi061:> cat /etc/os-release 2020-12-05T17:19:34.428 INFO:teuthology.orchestra.run.smithi061.stdout:NAME="Ubuntu" 2020-12-05T17:19:34.429 INFO:teuthology.orchestra.run.smithi061.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-12-05T17:19:34.429 INFO:teuthology.orchestra.run.smithi061.stdout:ID=ubuntu 2020-12-05T17:19:34.429 INFO:teuthology.orchestra.run.smithi061.stdout:ID_LIKE=debian 2020-12-05T17:19:34.429 INFO:teuthology.orchestra.run.smithi061.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-12-05T17:19:34.430 INFO:teuthology.orchestra.run.smithi061.stdout:VERSION_ID="18.04" 2020-12-05T17:19:34.430 INFO:teuthology.orchestra.run.smithi061.stdout:HOME_URL="https://www.ubuntu.com/" 2020-12-05T17:19:34.430 INFO:teuthology.orchestra.run.smithi061.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-12-05T17:19:34.430 INFO:teuthology.orchestra.run.smithi061.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-12-05T17:19:34.430 INFO:teuthology.orchestra.run.smithi061.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-12-05T17:19:34.431 INFO:teuthology.orchestra.run.smithi061.stdout:VERSION_CODENAME=bionic 2020-12-05T17:19:34.431 INFO:teuthology.orchestra.run.smithi061.stdout:UBUNTU_CODENAME=bionic 2020-12-05T17:19:34.432 INFO:teuthology.lock.ops:Updating smithi061.front.sepia.ceph.com on lock server 2020-12-05T17:19:34.449 INFO:teuthology.lock.ops:Updating [smithi036.front.sepia.ceph.com]: set os type and version on server 2020-12-05T17:19:34.449 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-12-05T17:19:34.450 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:19:34.526 INFO:teuthology.orchestra.run.smithi036:> true 2020-12-05T17:19:34.779 INFO:teuthology.orchestra.run.smithi036:> uname -m 2020-12-05T17:19:34.827 INFO:teuthology.orchestra.run.smithi036.stdout:x86_64 2020-12-05T17:19:34.828 INFO:teuthology.orchestra.run.smithi036:> cat /etc/os-release 2020-12-05T17:19:34.875 INFO:teuthology.orchestra.run.smithi036.stdout:NAME="Ubuntu" 2020-12-05T17:19:34.876 INFO:teuthology.orchestra.run.smithi036.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-12-05T17:19:34.876 INFO:teuthology.orchestra.run.smithi036.stdout:ID=ubuntu 2020-12-05T17:19:34.876 INFO:teuthology.orchestra.run.smithi036.stdout:ID_LIKE=debian 2020-12-05T17:19:34.877 INFO:teuthology.orchestra.run.smithi036.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-12-05T17:19:34.877 INFO:teuthology.orchestra.run.smithi036.stdout:VERSION_ID="18.04" 2020-12-05T17:19:34.877 INFO:teuthology.orchestra.run.smithi036.stdout:HOME_URL="https://www.ubuntu.com/" 2020-12-05T17:19:34.877 INFO:teuthology.orchestra.run.smithi036.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-12-05T17:19:34.878 INFO:teuthology.orchestra.run.smithi036.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-12-05T17:19:34.878 INFO:teuthology.orchestra.run.smithi036.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-12-05T17:19:34.878 INFO:teuthology.orchestra.run.smithi036.stdout:VERSION_CODENAME=bionic 2020-12-05T17:19:34.878 INFO:teuthology.orchestra.run.smithi036.stdout:UBUNTU_CODENAME=bionic 2020-12-05T17:19:34.879 INFO:teuthology.lock.ops:Updating smithi036.front.sepia.ceph.com on lock server 2020-12-05T17:19:34.894 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: set os type and version on server 2020-12-05T17:19:34.895 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-12-05T17:19:34.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:19:34.971 INFO:teuthology.orchestra.run.smithi002:> true 2020-12-05T17:19:35.223 INFO:teuthology.orchestra.run.smithi002:> uname -m 2020-12-05T17:19:35.274 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2020-12-05T17:19:35.275 INFO:teuthology.orchestra.run.smithi002:> cat /etc/os-release 2020-12-05T17:19:35.322 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="Ubuntu" 2020-12-05T17:19:35.323 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-12-05T17:19:35.323 INFO:teuthology.orchestra.run.smithi002.stdout:ID=ubuntu 2020-12-05T17:19:35.323 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE=debian 2020-12-05T17:19:35.323 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-12-05T17:19:35.324 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="18.04" 2020-12-05T17:19:35.324 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://www.ubuntu.com/" 2020-12-05T17:19:35.324 INFO:teuthology.orchestra.run.smithi002.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-12-05T17:19:35.324 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-12-05T17:19:35.324 INFO:teuthology.orchestra.run.smithi002.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-12-05T17:19:35.325 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_CODENAME=bionic 2020-12-05T17:19:35.325 INFO:teuthology.orchestra.run.smithi002.stdout:UBUNTU_CODENAME=bionic 2020-12-05T17:19:35.325 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2020-12-05T17:19:35.340 INFO:teuthology.lock.ops:Updating [smithi053.front.sepia.ceph.com]: set os type and version on server 2020-12-05T17:19:35.341 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-12-05T17:19:35.341 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:19:35.416 INFO:teuthology.orchestra.run.smithi053:> true 2020-12-05T17:19:35.662 INFO:teuthology.orchestra.run.smithi053:> uname -m 2020-12-05T17:19:35.708 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2020-12-05T17:19:35.709 INFO:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2020-12-05T17:19:35.757 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="Ubuntu" 2020-12-05T17:19:35.757 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-12-05T17:19:35.757 INFO:teuthology.orchestra.run.smithi053.stdout:ID=ubuntu 2020-12-05T17:19:35.758 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE=debian 2020-12-05T17:19:35.758 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-12-05T17:19:35.758 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="18.04" 2020-12-05T17:19:35.759 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://www.ubuntu.com/" 2020-12-05T17:19:35.759 INFO:teuthology.orchestra.run.smithi053.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-12-05T17:19:35.759 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-12-05T17:19:35.759 INFO:teuthology.orchestra.run.smithi053.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-12-05T17:19:35.759 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_CODENAME=bionic 2020-12-05T17:19:35.760 INFO:teuthology.orchestra.run.smithi053.stdout:UBUNTU_CODENAME=bionic 2020-12-05T17:19:35.761 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2020-12-05T17:19:35.778 INFO:teuthology.task.internal.lock_machines:5 smithi machines locked this try, 5/5 locked so far 2020-12-05T17:19:35.862 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi002.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkDTvkviRzNiFZGfh3gCKeTidXH4uumXTktmGPZgASqWZhBZQ9nkgtkXAEXFQdtPaQTjNr6/xbTq4qzVD1q4P8zzbLeI878CQl6xN2aI7IERGAk2YzbXiLNvAR57PogcBm+KIsGSK4r0omvsmthQje29maOgwaMc3X6eFVhyHJ/X6c6njN0gvcDlmSiBBJRoc/E9HedOxgkVkFzCI29jrovWlvqqKL9BHjLrLgeaRMkVZcac1hCPcWRY9TwLH6L7q5imeuqexSamXVIdzVNXBO0P+V2BW9UZGAuytYCtyQZdI/yvLf9k+1uMo+NvcmxfXicq7b/9PHxZMVa/vXvzqd smithi032.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/FLkMrt6+QSoSo0F9Fz2GyQK8DYsvAWqmMIMvMjiRqbincENM1U3gYC5iSjlRU5eJ4SLT7kvRi4YpEVjV553onM7n0ZiqF6XhcYXKBUONp/Imo9hB7sL9kB7HofZFInvoyQxNrSYQa3tAeueMgnFVpnj5CsjzWIaut8NgC+RHb81+V5YwNKqQsQ2d9d+nhyf/qp4Gw3Si6I2pdS1+ThOtijDUX9qm7TDk0hV23olr//Fo1sMuf9GpEKCealUhT8CUo0yNUz7sqXI6PvKESqvVVt+2p8dLVDITT+clwf1baLktyjTa5lMczBcTvW8w0uu5d4dYXjC4NpNhawFK0UOX smithi036.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUNu7cVfo9oLQpXclxNwdTCuj3jPUUJK5MeZB2q1hsCZqa3401oAVtHy+TLkz/1zChuCqGapIGnvS/Yy757gznNrxRiDC3b9qUz0HDSf4fVxulWlb5xBWbCwZkzSxWRWwyIq7qUj5ssC49HvxEFGmxIa7KfCeYxOIL7te5GvUtyTeMytrN+lDcPmFByuUu8yseqqq5r8s8hGMy8ETmYkBPEJMgQoComuqI0H5OHWvXOWQd44MBd82LsfLNYpLrkRglHpSmzFT3IW9klCPP83Th4NkPgyejmsaUHOeQvVy4g4Zc0CNLilO1vrcwmN0+FRMGbBFvQq5kgfveNjmz3KYP smithi053.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCd0U0suNtO6c2szbrg5E2OvQVna9Pj09OJubTsedEBCHaOAiZKzmF3y4129SI4cFuYPBiDF/pLks25JVh6VHdcs2QBZd1ALGza+HiOfTBc0KHeI+ieOMWKvbm9F6fI+oA/xbSWPILmDMY12SH7zvGQR4TgeFg0gnlJxlI9EJbBY7KKNKiqUuKnq1333FfNCPLFAoC3wPuWPBBmGS2bztSfTwEXCh6BvfUnpEBwRHS4ZfgCUKjldohPjN5EPN3ch+M7JQasfhYsb7nPsLGIAoGLsrFdYi2FiAK6n2ayZDutP3pAfZZJz1+R+Gp46GtJEEB35Iu+Kb3pqvxzDoWPLRtn smithi061.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCuh59zncHb/+PmIORHkjdSiLduZqiHuvYzmmcgknMzwtYc46PloMcS4PiX0BjinbWMGOX4EVnlkBKS3UVqt7/EfkJZF9lQgeQw6vgXfs+1l73QQvHdXuwGxKW015qJWSjC0adW7CG/imjRgT1pTqeHSZlD6YpjaRIR2VeXXa3BtabmJuboojdO7SCv8EkqbsxZIvM+/KGn4b9VDmPmCcJNML/nkbj+C8zes4e7h7DV6Jc3M/PA5M4b1qdr23BgVfM55rkRieoTCtP02nf/CLM+H55PXYgkSmKx1LHSlPUTnsLK3L9GLqNoLJT7i2hObOblmOR5QFdhfZWZaXK4ySlh 2020-12-05T17:19:35.862 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-12-05T17:19:35.931 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-12-05T17:19:35.937 INFO:teuthology.task.internal:Saving configuration 2020-12-05T17:19:35.957 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-12-05T17:19:35.968 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-12-05T17:19:35.985 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-12-05 17:14:07.385667', 'description': '/home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIQIKbWcbNgH4eceNAI1XDwcaO/uAVa3gh4kdmp8jhEp', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi032.front.sepia.ceph.com'} 2020-12-05T17:19:36.019 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-12-05 17:14:07.385689', 'description': '/home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIG4awJ9GMbzCVRB2M3KRKCEaAeiePKypTAp2lIN9yiGe', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi061.front.sepia.ceph.com'} 2020-12-05T17:19:36.034 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-12-05 17:14:07.385706', 'description': '/home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINsgJTj3/2oTcT4UvFPYghnzu7ce5vLM/SaLRqHCkxfq', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi036.front.sepia.ceph.com'} 2020-12-05T17:19:36.049 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-12-05 17:14:07.385721', 'description': '/home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJNNusJohh9DYd0jJHBtOolfXG9Neftx4ZBK63cClRFb', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi002.front.sepia.ceph.com'} 2020-12-05T17:19:36.065 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-12-05 17:14:07.385736', 'description': '/home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIB531F8q+4nW//Z/tbYnolApvmCc91TOQdTQh1pkML6J', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi053.front.sepia.ceph.com'} 2020-12-05T17:19:36.065 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-12-05T17:19:36.076 INFO:teuthology.task.internal:roles: ubuntu@smithi032.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2020-12-05T17:19:36.076 INFO:teuthology.task.internal:roles: ubuntu@smithi061.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2020-12-05T17:19:36.077 INFO:teuthology.task.internal:roles: ubuntu@smithi036.front.sepia.ceph.com - ['client.0'] 2020-12-05T17:19:36.077 INFO:teuthology.task.internal:roles: ubuntu@smithi002.front.sepia.ceph.com - ['client.1'] 2020-12-05T17:19:36.077 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['client.2'] 2020-12-05T17:19:36.078 INFO:teuthology.run_tasks:Running task console_log... 2020-12-05T17:19:36.214 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f99f7d06b70>, signals=[15]) 2020-12-05T17:19:36.215 INFO:teuthology.run_tasks:Running task internal.connect... 2020-12-05T17:19:36.226 INFO:teuthology.task.internal:Opening connections... 2020-12-05T17:19:36.226 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi032.front.sepia.ceph.com 2020-12-05T17:19:36.228 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:19:36.305 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi061.front.sepia.ceph.com 2020-12-05T17:19:36.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:19:36.381 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi036.front.sepia.ceph.com 2020-12-05T17:19:36.383 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:19:36.456 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi002.front.sepia.ceph.com 2020-12-05T17:19:36.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:19:36.534 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi053.front.sepia.ceph.com 2020-12-05T17:19:36.536 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:19:36.612 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-12-05T17:19:36.623 INFO:teuthology.orchestra.run.smithi032:> uname -m 2020-12-05T17:19:36.630 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2020-12-05T17:19:36.631 INFO:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2020-12-05T17:19:36.679 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="Ubuntu" 2020-12-05T17:19:36.680 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-12-05T17:19:36.680 INFO:teuthology.orchestra.run.smithi032.stdout:ID=ubuntu 2020-12-05T17:19:36.681 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE=debian 2020-12-05T17:19:36.681 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-12-05T17:19:36.681 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="18.04" 2020-12-05T17:19:36.681 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://www.ubuntu.com/" 2020-12-05T17:19:36.681 INFO:teuthology.orchestra.run.smithi032.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-12-05T17:19:36.682 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-12-05T17:19:36.682 INFO:teuthology.orchestra.run.smithi032.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-12-05T17:19:36.682 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_CODENAME=bionic 2020-12-05T17:19:36.682 INFO:teuthology.orchestra.run.smithi032.stdout:UBUNTU_CODENAME=bionic 2020-12-05T17:19:36.683 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2020-12-05T17:19:36.698 INFO:teuthology.orchestra.run.smithi061:> uname -m 2020-12-05T17:19:36.704 INFO:teuthology.orchestra.run.smithi061.stdout:x86_64 2020-12-05T17:19:36.704 INFO:teuthology.orchestra.run.smithi061:> cat /etc/os-release 2020-12-05T17:19:36.753 INFO:teuthology.orchestra.run.smithi061.stdout:NAME="Ubuntu" 2020-12-05T17:19:36.753 INFO:teuthology.orchestra.run.smithi061.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-12-05T17:19:36.753 INFO:teuthology.orchestra.run.smithi061.stdout:ID=ubuntu 2020-12-05T17:19:36.754 INFO:teuthology.orchestra.run.smithi061.stdout:ID_LIKE=debian 2020-12-05T17:19:36.754 INFO:teuthology.orchestra.run.smithi061.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-12-05T17:19:36.754 INFO:teuthology.orchestra.run.smithi061.stdout:VERSION_ID="18.04" 2020-12-05T17:19:36.754 INFO:teuthology.orchestra.run.smithi061.stdout:HOME_URL="https://www.ubuntu.com/" 2020-12-05T17:19:36.755 INFO:teuthology.orchestra.run.smithi061.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-12-05T17:19:36.755 INFO:teuthology.orchestra.run.smithi061.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-12-05T17:19:36.755 INFO:teuthology.orchestra.run.smithi061.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-12-05T17:19:36.755 INFO:teuthology.orchestra.run.smithi061.stdout:VERSION_CODENAME=bionic 2020-12-05T17:19:36.755 INFO:teuthology.orchestra.run.smithi061.stdout:UBUNTU_CODENAME=bionic 2020-12-05T17:19:36.756 INFO:teuthology.lock.ops:Updating smithi061.front.sepia.ceph.com on lock server 2020-12-05T17:19:36.771 INFO:teuthology.orchestra.run.smithi036:> uname -m 2020-12-05T17:19:36.777 INFO:teuthology.orchestra.run.smithi036.stdout:x86_64 2020-12-05T17:19:36.778 INFO:teuthology.orchestra.run.smithi036:> cat /etc/os-release 2020-12-05T17:19:36.825 INFO:teuthology.orchestra.run.smithi036.stdout:NAME="Ubuntu" 2020-12-05T17:19:36.825 INFO:teuthology.orchestra.run.smithi036.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-12-05T17:19:36.825 INFO:teuthology.orchestra.run.smithi036.stdout:ID=ubuntu 2020-12-05T17:19:36.825 INFO:teuthology.orchestra.run.smithi036.stdout:ID_LIKE=debian 2020-12-05T17:19:36.826 INFO:teuthology.orchestra.run.smithi036.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-12-05T17:19:36.826 INFO:teuthology.orchestra.run.smithi036.stdout:VERSION_ID="18.04" 2020-12-05T17:19:36.826 INFO:teuthology.orchestra.run.smithi036.stdout:HOME_URL="https://www.ubuntu.com/" 2020-12-05T17:19:36.826 INFO:teuthology.orchestra.run.smithi036.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-12-05T17:19:36.827 INFO:teuthology.orchestra.run.smithi036.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-12-05T17:19:36.827 INFO:teuthology.orchestra.run.smithi036.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-12-05T17:19:36.827 INFO:teuthology.orchestra.run.smithi036.stdout:VERSION_CODENAME=bionic 2020-12-05T17:19:36.827 INFO:teuthology.orchestra.run.smithi036.stdout:UBUNTU_CODENAME=bionic 2020-12-05T17:19:36.828 INFO:teuthology.lock.ops:Updating smithi036.front.sepia.ceph.com on lock server 2020-12-05T17:19:36.841 INFO:teuthology.orchestra.run.smithi002:> uname -m 2020-12-05T17:19:36.847 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2020-12-05T17:19:36.848 INFO:teuthology.orchestra.run.smithi002:> cat /etc/os-release 2020-12-05T17:19:36.894 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="Ubuntu" 2020-12-05T17:19:36.895 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-12-05T17:19:36.895 INFO:teuthology.orchestra.run.smithi002.stdout:ID=ubuntu 2020-12-05T17:19:36.895 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE=debian 2020-12-05T17:19:36.895 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-12-05T17:19:36.895 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="18.04" 2020-12-05T17:19:36.896 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://www.ubuntu.com/" 2020-12-05T17:19:36.896 INFO:teuthology.orchestra.run.smithi002.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-12-05T17:19:36.896 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-12-05T17:19:36.896 INFO:teuthology.orchestra.run.smithi002.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-12-05T17:19:36.896 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_CODENAME=bionic 2020-12-05T17:19:36.897 INFO:teuthology.orchestra.run.smithi002.stdout:UBUNTU_CODENAME=bionic 2020-12-05T17:19:36.897 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2020-12-05T17:19:36.911 INFO:teuthology.orchestra.run.smithi053:> uname -m 2020-12-05T17:19:36.917 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2020-12-05T17:19:36.918 INFO:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2020-12-05T17:19:36.965 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="Ubuntu" 2020-12-05T17:19:36.965 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-12-05T17:19:36.965 INFO:teuthology.orchestra.run.smithi053.stdout:ID=ubuntu 2020-12-05T17:19:36.965 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE=debian 2020-12-05T17:19:36.966 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-12-05T17:19:36.966 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="18.04" 2020-12-05T17:19:36.966 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://www.ubuntu.com/" 2020-12-05T17:19:36.966 INFO:teuthology.orchestra.run.smithi053.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-12-05T17:19:36.967 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-12-05T17:19:36.967 INFO:teuthology.orchestra.run.smithi053.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-12-05T17:19:36.967 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_CODENAME=bionic 2020-12-05T17:19:36.967 INFO:teuthology.orchestra.run.smithi053.stdout:UBUNTU_CODENAME=bionic 2020-12-05T17:19:36.968 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2020-12-05T17:19:36.982 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-12-05T17:19:37.022 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-12-05T17:19:37.032 INFO:teuthology.task.internal:Checking for old test directory... 2020-12-05T17:19:37.032 INFO:teuthology.orchestra.run.smithi002:> test '!' -e /home/ubuntu/cephtest 2020-12-05T17:19:37.035 INFO:teuthology.orchestra.run.smithi032:> test '!' -e /home/ubuntu/cephtest 2020-12-05T17:19:37.038 INFO:teuthology.orchestra.run.smithi036:> test '!' -e /home/ubuntu/cephtest 2020-12-05T17:19:37.041 INFO:teuthology.orchestra.run.smithi053:> test '!' -e /home/ubuntu/cephtest 2020-12-05T17:19:37.044 INFO:teuthology.orchestra.run.smithi061:> test '!' -e /home/ubuntu/cephtest 2020-12-05T17:19:37.051 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-12-05T17:19:37.059 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-12-05T17:19:37.060 INFO:teuthology.orchestra.run.smithi002:> test -z $(ls -A /var/lib/ceph) 2020-12-05T17:19:37.082 INFO:teuthology.orchestra.run.smithi032:> test -z $(ls -A /var/lib/ceph) 2020-12-05T17:19:37.085 INFO:teuthology.orchestra.run.smithi036:> test -z $(ls -A /var/lib/ceph) 2020-12-05T17:19:37.087 INFO:teuthology.orchestra.run.smithi053:> test -z $(ls -A /var/lib/ceph) 2020-12-05T17:19:37.093 INFO:teuthology.orchestra.run.smithi061:> test -z $(ls -A /var/lib/ceph) 2020-12-05T17:19:37.114 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-12-05T17:19:37.204 INFO:teuthology.run_tasks:Running task kernel... 2020-12-05T17:19:37.225 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.y': {'kdb': True, 'sha1': 'distro'}, 'mds.a': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}, 'osd.3': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}, 'mds.b': {'kdb': True, 'sha1': 'distro'}, 'osd.4': {'kdb': True, 'sha1': 'distro'}, 'osd.5': {'kdb': True, 'sha1': 'distro'}, 'osd.6': {'kdb': True, 'sha1': 'distro'}, 'osd.7': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}, 'client.2': {'kdb': True, 'sha1': 'distro'}} 2020-12-05T17:19:37.226 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}, 'client.2': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2020-12-05T17:19:37.226 INFO:teuthology.orchestra.run.smithi032:> uname -r 2020-12-05T17:19:37.232 INFO:teuthology.orchestra.run.smithi032.stdout:4.15.0-124-generic 2020-12-05T17:19:37.233 INFO:teuthology.task.kernel:Running kernel on smithi032: 4.15.0-124-generic 2020-12-05T17:19:37.233 INFO:teuthology.orchestra.run.smithi032:> sudo apt-get clean 2020-12-05T17:19:37.349 INFO:teuthology.orchestra.run.smithi032:> sudo apt-get update 2020-12-05T17:19:37.593 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2020-12-05T17:19:37.610 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-12-05T17:19:37.675 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2020-12-05T17:19:37.928 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2020-12-05T17:19:38.014 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [870 kB] 2020-12-05T17:19:38.155 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,789 kB] 2020-12-05T17:19:38.403 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,453 kB] 2020-12-05T17:19:38.520 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [284 kB] 2020-12-05T17:19:38.539 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [184 kB] 2020-12-05T17:19:38.542 INFO:teuthology.orchestra.run.smithi032.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,172 kB] 2020-12-05T17:19:38.549 INFO:teuthology.orchestra.run.smithi032.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [24.4 kB] 2020-12-05T17:19:38.550 INFO:teuthology.orchestra.run.smithi032.stdout:Get:12 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [968 kB] 2020-12-05T17:19:38.578 INFO:teuthology.orchestra.run.smithi032.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [376 kB] 2020-12-05T17:19:38.584 INFO:teuthology.orchestra.run.smithi032.stdout:Get:14 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,092 kB] 2020-12-05T17:19:38.589 INFO:teuthology.orchestra.run.smithi032.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [206 kB] 2020-12-05T17:19:38.595 INFO:teuthology.orchestra.run.smithi032.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [28.0 kB] 2020-12-05T17:19:38.596 INFO:teuthology.orchestra.run.smithi032.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,548 kB] 2020-12-05T17:19:38.619 INFO:teuthology.orchestra.run.smithi032.stdout:Get:18 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [244 kB] 2020-12-05T17:19:38.666 INFO:teuthology.orchestra.run.smithi032.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,695 kB] 2020-12-05T17:19:38.715 INFO:teuthology.orchestra.run.smithi032.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [357 kB] 2020-12-05T17:19:40.118 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 12.5 MB in 3s (4,881 kB/s) 2020-12-05T17:19:41.814 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2020-12-05T17:19:41.861 INFO:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-12-05T17:19:41.945 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2020-12-05T17:19:42.205 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2020-12-05T17:19:42.206 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2020-12-05T17:19:42.290 INFO:teuthology.orchestra.run.smithi032.stderr:E: Unable to locate package linux-image-current-generic 2020-12-05T17:19:42.291 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-12-05T17:19:42.292 INFO:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-12-05T17:19:42.373 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2020-12-05T17:19:42.611 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2020-12-05T17:19:42.612 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2020-12-05T17:19:42.794 INFO:teuthology.orchestra.run.smithi032.stdout:The following additional packages will be installed: 2020-12-05T17:19:42.794 INFO:teuthology.orchestra.run.smithi032.stdout: linux-generic linux-headers-4.15.0-126 linux-headers-4.15.0-126-generic 2020-12-05T17:19:42.795 INFO:teuthology.orchestra.run.smithi032.stdout: linux-headers-generic linux-image-4.15.0-126-generic 2020-12-05T17:19:42.795 INFO:teuthology.orchestra.run.smithi032.stdout: linux-modules-4.15.0-126-generic linux-modules-extra-4.15.0-126-generic 2020-12-05T17:19:42.796 INFO:teuthology.orchestra.run.smithi032.stdout:Suggested packages: 2020-12-05T17:19:42.796 INFO:teuthology.orchestra.run.smithi032.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2020-12-05T17:19:42.797 INFO:teuthology.orchestra.run.smithi032.stdout:Recommended packages: 2020-12-05T17:19:42.797 INFO:teuthology.orchestra.run.smithi032.stdout: thermald 2020-12-05T17:19:42.831 INFO:teuthology.orchestra.run.smithi032.stdout:The following NEW packages will be installed: 2020-12-05T17:19:42.832 INFO:teuthology.orchestra.run.smithi032.stdout: linux-headers-4.15.0-126 linux-headers-4.15.0-126-generic 2020-12-05T17:19:42.832 INFO:teuthology.orchestra.run.smithi032.stdout: linux-image-4.15.0-126-generic linux-modules-4.15.0-126-generic 2020-12-05T17:19:42.832 INFO:teuthology.orchestra.run.smithi032.stdout: linux-modules-extra-4.15.0-126-generic 2020-12-05T17:19:42.833 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages will be upgraded: 2020-12-05T17:19:42.834 INFO:teuthology.orchestra.run.smithi032.stdout: linux-generic linux-headers-generic linux-image-generic 2020-12-05T17:19:43.015 INFO:teuthology.orchestra.run.smithi032.stdout:3 upgraded, 5 newly installed, 0 to remove and 7 not upgraded. 2020-12-05T17:19:43.015 INFO:teuthology.orchestra.run.smithi032.stdout:Need to get 65.9 MB of archives. 2020-12-05T17:19:43.015 INFO:teuthology.orchestra.run.smithi032.stdout:After this operation, 333 MB of additional disk space will be used. 2020-12-05T17:19:43.016 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-126-generic amd64 4.15.0-126.129 [13.1 MB] 2020-12-05T17:19:44.018 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-126-generic amd64 4.15.0-126.129 [8,013 kB] 2020-12-05T17:19:44.262 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-126-generic amd64 4.15.0-126.129 [32.8 MB] 2020-12-05T17:19:45.197 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.126.113 [1,872 B] 2020-12-05T17:19:45.198 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.126.113 [2,620 B] 2020-12-05T17:19:45.198 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-126 all 4.15.0-126.129 [10.9 MB] 2020-12-05T17:19:45.508 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-126-generic amd64 4.15.0-126.129 [1,096 kB] 2020-12-05T17:19:45.538 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.126.113 [2,524 B] 2020-12-05T17:19:46.042 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 65.9 MB in 3s (24.5 MB/s) 2020-12-05T17:19:46.216 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-modules-4.15.0-126-generic. 2020-12-05T17:19:46.255 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102077 files and directories currently installed.) 2020-12-05T17:19:46.258 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../0-linux-modules-4.15.0-126-generic_4.15.0-126.129_amd64.deb ... 2020-12-05T17:19:46.297 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-modules-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:19:47.947 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-image-4.15.0-126-generic. 2020-12-05T17:19:47.959 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../1-linux-image-4.15.0-126-generic_4.15.0-126.129_amd64.deb ... 2020-12-05T17:19:48.029 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-image-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:19:48.558 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-126-generic. 2020-12-05T17:19:48.570 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-126-generic_4.15.0-126.129_amd64.deb ... 2020-12-05T17:19:48.608 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-modules-extra-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:19:52.474 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../3-linux-generic_4.15.0.126.113_amd64.deb ... 2020-12-05T17:19:52.602 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-generic (4.15.0.126.113) over (4.15.0.124.111) ... 2020-12-05T17:19:52.902 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.126.113_amd64.deb ... 2020-12-05T17:19:53.014 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-image-generic (4.15.0.126.113) over (4.15.0.124.111) ... 2020-12-05T17:19:53.317 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-headers-4.15.0-126. 2020-12-05T17:19:53.329 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../5-linux-headers-4.15.0-126_4.15.0-126.129_all.deb ... 2020-12-05T17:19:53.367 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-headers-4.15.0-126 (4.15.0-126.129) ... 2020-12-05T17:19:58.517 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-headers-4.15.0-126-generic. 2020-12-05T17:19:58.536 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../6-linux-headers-4.15.0-126-generic_4.15.0-126.129_amd64.deb ... 2020-12-05T17:19:58.575 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-headers-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:20:01.081 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.126.113_amd64.deb ... 2020-12-05T17:20:01.180 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-headers-generic (4.15.0.126.113) over (4.15.0.124.111) ... 2020-12-05T17:20:01.482 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-headers-4.15.0-126 (4.15.0-126.129) ... 2020-12-05T17:20:01.601 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-modules-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:20:02.161 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-headers-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:20:02.279 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-image-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:20:02.732 INFO:teuthology.orchestra.run.smithi032.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-126-generic 2020-12-05T17:20:02.732 INFO:teuthology.orchestra.run.smithi032.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-126-generic 2020-12-05T17:20:02.848 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-headers-generic (4.15.0.126.113) ... 2020-12-05T17:20:02.949 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-modules-extra-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:20:03.457 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-image-generic (4.15.0.126.113) ... 2020-12-05T17:20:03.559 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-generic (4.15.0.126.113) ... 2020-12-05T17:20:03.678 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for linux-image-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:20:03.883 INFO:teuthology.orchestra.run.smithi032.stdout:/etc/kernel/postinst.d/initramfs-tools: 2020-12-05T17:20:03.884 INFO:teuthology.orchestra.run.smithi032.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-126-generic 2020-12-05T17:20:19.486 INFO:teuthology.orchestra.run.smithi032.stdout:/etc/kernel/postinst.d/zz-update-grub: 2020-12-05T17:20:19.487 INFO:teuthology.orchestra.run.smithi032.stdout:Sourcing file `/etc/default/grub' 2020-12-05T17:20:19.515 INFO:teuthology.orchestra.run.smithi032.stdout:Generating grub configuration file ... 2020-12-05T17:20:19.839 INFO:teuthology.orchestra.run.smithi032.stdout:Found linux image: /boot/vmlinuz-4.15.0-126-generic 2020-12-05T17:20:19.877 INFO:teuthology.orchestra.run.smithi032.stdout:Found initrd image: /boot/initrd.img-4.15.0-126-generic 2020-12-05T17:20:20.186 INFO:teuthology.orchestra.run.smithi032.stdout:Found linux image: /boot/vmlinuz-4.15.0-124-generic 2020-12-05T17:20:20.216 INFO:teuthology.orchestra.run.smithi032.stdout:Found initrd image: /boot/initrd.img-4.15.0-124-generic 2020-12-05T17:20:21.587 INFO:teuthology.orchestra.run.smithi032.stdout:done 2020-12-05T17:20:22.044 INFO:teuthology.orchestra.run.smithi032:> dpkg -s linux-image-generic 2020-12-05T17:20:22.064 INFO:teuthology.orchestra.run.smithi032.stdout:Package: linux-image-generic 2020-12-05T17:20:22.064 INFO:teuthology.orchestra.run.smithi032.stdout:Status: install ok installed 2020-12-05T17:20:22.065 INFO:teuthology.orchestra.run.smithi032.stdout:Priority: optional 2020-12-05T17:20:22.065 INFO:teuthology.orchestra.run.smithi032.stdout:Section: kernel 2020-12-05T17:20:22.065 INFO:teuthology.orchestra.run.smithi032.stdout:Installed-Size: 16 2020-12-05T17:20:22.065 INFO:teuthology.orchestra.run.smithi032.stdout:Maintainer: Ubuntu Kernel Team 2020-12-05T17:20:22.065 INFO:teuthology.orchestra.run.smithi032.stdout:Architecture: amd64 2020-12-05T17:20:22.066 INFO:teuthology.orchestra.run.smithi032.stdout:Source: linux-meta 2020-12-05T17:20:22.066 INFO:teuthology.orchestra.run.smithi032.stdout:Version: 4.15.0.126.113 2020-12-05T17:20:22.066 INFO:teuthology.orchestra.run.smithi032.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2020-12-05T17:20:22.066 INFO:teuthology.orchestra.run.smithi032.stdout:Depends: linux-image-4.15.0-126-generic, linux-modules-extra-4.15.0-126-generic, linux-firmware, intel-microcode, amd64-microcode 2020-12-05T17:20:22.067 INFO:teuthology.orchestra.run.smithi032.stdout:Recommends: thermald 2020-12-05T17:20:22.067 INFO:teuthology.orchestra.run.smithi032.stdout:Description: Generic Linux kernel image 2020-12-05T17:20:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: This package will always depend on the latest generic kernel image 2020-12-05T17:20:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: available. 2020-12-05T17:20:22.067 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-124-generic Expected: 4.15.0-126-generic 2020-12-05T17:20:22.068 INFO:teuthology.orchestra.run.smithi061:> uname -r 2020-12-05T17:20:22.074 INFO:teuthology.orchestra.run.smithi061.stdout:4.15.0-124-generic 2020-12-05T17:20:22.075 INFO:teuthology.task.kernel:Running kernel on smithi061: 4.15.0-124-generic 2020-12-05T17:20:22.075 INFO:teuthology.orchestra.run.smithi061:> sudo apt-get clean 2020-12-05T17:20:22.201 INFO:teuthology.orchestra.run.smithi061:> sudo apt-get update 2020-12-05T17:20:22.343 INFO:teuthology.orchestra.run.smithi061.stdout:Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2020-12-05T17:20:22.457 INFO:teuthology.orchestra.run.smithi061.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-12-05T17:20:22.535 INFO:teuthology.orchestra.run.smithi061.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2020-12-05T17:20:22.619 INFO:teuthology.orchestra.run.smithi061.stdout:Get:4 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [870 kB] 2020-12-05T17:20:22.727 INFO:teuthology.orchestra.run.smithi061.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,453 kB] 2020-12-05T17:20:22.760 INFO:teuthology.orchestra.run.smithi061.stdout:Get:6 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [284 kB] 2020-12-05T17:20:22.763 INFO:teuthology.orchestra.run.smithi061.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [184 kB] 2020-12-05T17:20:22.765 INFO:teuthology.orchestra.run.smithi061.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [24.4 kB] 2020-12-05T17:20:22.765 INFO:teuthology.orchestra.run.smithi061.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [968 kB] 2020-12-05T17:20:22.780 INFO:teuthology.orchestra.run.smithi061.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,092 kB] 2020-12-05T17:20:22.794 INFO:teuthology.orchestra.run.smithi061.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [244 kB] 2020-12-05T17:20:22.798 INFO:teuthology.orchestra.run.smithi061.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2020-12-05T17:20:22.909 INFO:teuthology.orchestra.run.smithi061.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,789 kB] 2020-12-05T17:20:23.290 INFO:teuthology.orchestra.run.smithi061.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,172 kB] 2020-12-05T17:20:23.309 INFO:teuthology.orchestra.run.smithi061.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [376 kB] 2020-12-05T17:20:23.314 INFO:teuthology.orchestra.run.smithi061.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [206 kB] 2020-12-05T17:20:23.317 INFO:teuthology.orchestra.run.smithi061.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [28.0 kB] 2020-12-05T17:20:23.318 INFO:teuthology.orchestra.run.smithi061.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,695 kB] 2020-12-05T17:20:23.391 INFO:teuthology.orchestra.run.smithi061.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,548 kB] 2020-12-05T17:20:23.452 INFO:teuthology.orchestra.run.smithi061.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [357 kB] 2020-12-05T17:20:24.564 INFO:teuthology.orchestra.run.smithi061.stdout:Fetched 12.5 MB in 2s (5,792 kB/s) 2020-12-05T17:20:26.370 INFO:teuthology.orchestra.run.smithi061.stdout:Reading package lists... 2020-12-05T17:20:26.418 INFO:teuthology.orchestra.run.smithi061:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-12-05T17:20:26.501 INFO:teuthology.orchestra.run.smithi061.stdout:Reading package lists... 2020-12-05T17:20:26.750 INFO:teuthology.orchestra.run.smithi061.stdout:Building dependency tree... 2020-12-05T17:20:26.751 INFO:teuthology.orchestra.run.smithi061.stdout:Reading state information... 2020-12-05T17:20:26.841 INFO:teuthology.orchestra.run.smithi061.stderr:E: Unable to locate package linux-image-current-generic 2020-12-05T17:20:26.842 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-12-05T17:20:26.843 INFO:teuthology.orchestra.run.smithi061:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-12-05T17:20:26.939 INFO:teuthology.orchestra.run.smithi061.stdout:Reading package lists... 2020-12-05T17:20:27.226 INFO:teuthology.orchestra.run.smithi061.stdout:Building dependency tree... 2020-12-05T17:20:27.227 INFO:teuthology.orchestra.run.smithi061.stdout:Reading state information... 2020-12-05T17:20:27.413 INFO:teuthology.orchestra.run.smithi061.stdout:The following additional packages will be installed: 2020-12-05T17:20:27.414 INFO:teuthology.orchestra.run.smithi061.stdout: linux-generic linux-headers-4.15.0-126 linux-headers-4.15.0-126-generic 2020-12-05T17:20:27.414 INFO:teuthology.orchestra.run.smithi061.stdout: linux-headers-generic linux-image-4.15.0-126-generic 2020-12-05T17:20:27.414 INFO:teuthology.orchestra.run.smithi061.stdout: linux-modules-4.15.0-126-generic linux-modules-extra-4.15.0-126-generic 2020-12-05T17:20:27.416 INFO:teuthology.orchestra.run.smithi061.stdout:Suggested packages: 2020-12-05T17:20:27.416 INFO:teuthology.orchestra.run.smithi061.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2020-12-05T17:20:27.416 INFO:teuthology.orchestra.run.smithi061.stdout:Recommended packages: 2020-12-05T17:20:27.416 INFO:teuthology.orchestra.run.smithi061.stdout: thermald 2020-12-05T17:20:27.450 INFO:teuthology.orchestra.run.smithi061.stdout:The following NEW packages will be installed: 2020-12-05T17:20:27.450 INFO:teuthology.orchestra.run.smithi061.stdout: linux-headers-4.15.0-126 linux-headers-4.15.0-126-generic 2020-12-05T17:20:27.450 INFO:teuthology.orchestra.run.smithi061.stdout: linux-image-4.15.0-126-generic linux-modules-4.15.0-126-generic 2020-12-05T17:20:27.451 INFO:teuthology.orchestra.run.smithi061.stdout: linux-modules-extra-4.15.0-126-generic 2020-12-05T17:20:27.452 INFO:teuthology.orchestra.run.smithi061.stdout:The following packages will be upgraded: 2020-12-05T17:20:27.452 INFO:teuthology.orchestra.run.smithi061.stdout: linux-generic linux-headers-generic linux-image-generic 2020-12-05T17:20:27.633 INFO:teuthology.orchestra.run.smithi061.stdout:3 upgraded, 5 newly installed, 0 to remove and 7 not upgraded. 2020-12-05T17:20:27.634 INFO:teuthology.orchestra.run.smithi061.stdout:Need to get 65.9 MB of archives. 2020-12-05T17:20:27.634 INFO:teuthology.orchestra.run.smithi061.stdout:After this operation, 333 MB of additional disk space will be used. 2020-12-05T17:20:27.634 INFO:teuthology.orchestra.run.smithi061.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-126-generic amd64 4.15.0-126.129 [13.1 MB] 2020-12-05T17:20:28.632 INFO:teuthology.orchestra.run.smithi061.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-126-generic amd64 4.15.0-126.129 [8,013 kB] 2020-12-05T17:20:28.881 INFO:teuthology.orchestra.run.smithi061.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-126-generic amd64 4.15.0-126.129 [32.8 MB] 2020-12-05T17:20:29.831 INFO:teuthology.orchestra.run.smithi061.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.126.113 [1,872 B] 2020-12-05T17:20:29.832 INFO:teuthology.orchestra.run.smithi061.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.126.113 [2,620 B] 2020-12-05T17:20:29.832 INFO:teuthology.orchestra.run.smithi061.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-126 all 4.15.0-126.129 [10.9 MB] 2020-12-05T17:20:30.143 INFO:teuthology.orchestra.run.smithi061.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-126-generic amd64 4.15.0-126.129 [1,096 kB] 2020-12-05T17:20:30.160 INFO:teuthology.orchestra.run.smithi061.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.126.113 [2,524 B] 2020-12-05T17:20:30.694 INFO:teuthology.orchestra.run.smithi061.stdout:Fetched 65.9 MB in 3s (24.5 MB/s) 2020-12-05T17:20:30.867 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package linux-modules-4.15.0-126-generic. 2020-12-05T17:20:30.904 INFO:teuthology.orchestra.run.smithi061.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102077 files and directories currently installed.) 2020-12-05T17:20:30.906 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../0-linux-modules-4.15.0-126-generic_4.15.0-126.129_amd64.deb ... 2020-12-05T17:20:30.964 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking linux-modules-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:20:32.655 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package linux-image-4.15.0-126-generic. 2020-12-05T17:20:32.668 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../1-linux-image-4.15.0-126-generic_4.15.0-126.129_amd64.deb ... 2020-12-05T17:20:32.757 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking linux-image-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:20:33.258 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-126-generic. 2020-12-05T17:20:33.270 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-126-generic_4.15.0-126.129_amd64.deb ... 2020-12-05T17:20:33.308 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking linux-modules-extra-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:20:37.174 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../3-linux-generic_4.15.0.126.113_amd64.deb ... 2020-12-05T17:20:37.300 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking linux-generic (4.15.0.126.113) over (4.15.0.124.111) ... 2020-12-05T17:20:37.699 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.126.113_amd64.deb ... 2020-12-05T17:20:37.820 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking linux-image-generic (4.15.0.126.113) over (4.15.0.124.111) ... 2020-12-05T17:20:38.174 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package linux-headers-4.15.0-126. 2020-12-05T17:20:38.184 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../5-linux-headers-4.15.0-126_4.15.0-126.129_all.deb ... 2020-12-05T17:20:38.223 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking linux-headers-4.15.0-126 (4.15.0-126.129) ... 2020-12-05T17:20:43.254 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package linux-headers-4.15.0-126-generic. 2020-12-05T17:20:43.275 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../6-linux-headers-4.15.0-126-generic_4.15.0-126.129_amd64.deb ... 2020-12-05T17:20:43.312 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking linux-headers-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:20:46.093 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.126.113_amd64.deb ... 2020-12-05T17:20:46.214 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking linux-headers-generic (4.15.0.126.113) over (4.15.0.124.111) ... 2020-12-05T17:20:46.591 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up linux-headers-4.15.0-126 (4.15.0-126.129) ... 2020-12-05T17:20:46.752 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up linux-modules-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:20:47.329 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up linux-headers-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:20:47.455 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up linux-image-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:20:47.984 INFO:teuthology.orchestra.run.smithi061.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-126-generic 2020-12-05T17:20:47.984 INFO:teuthology.orchestra.run.smithi061.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-126-generic 2020-12-05T17:20:48.132 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up linux-headers-generic (4.15.0.126.113) ... 2020-12-05T17:20:48.259 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up linux-modules-extra-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:20:48.793 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up linux-image-generic (4.15.0.126.113) ... 2020-12-05T17:20:48.919 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up linux-generic (4.15.0.126.113) ... 2020-12-05T17:20:49.070 INFO:teuthology.orchestra.run.smithi061.stdout:Processing triggers for linux-image-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:20:49.251 INFO:teuthology.orchestra.run.smithi061.stdout:/etc/kernel/postinst.d/initramfs-tools: 2020-12-05T17:20:49.252 INFO:teuthology.orchestra.run.smithi061.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-126-generic 2020-12-05T17:21:05.012 INFO:teuthology.orchestra.run.smithi061.stdout:/etc/kernel/postinst.d/zz-update-grub: 2020-12-05T17:21:05.013 INFO:teuthology.orchestra.run.smithi061.stdout:Sourcing file `/etc/default/grub' 2020-12-05T17:21:05.041 INFO:teuthology.orchestra.run.smithi061.stdout:Generating grub configuration file ... 2020-12-05T17:21:05.421 INFO:teuthology.orchestra.run.smithi061.stdout:Found linux image: /boot/vmlinuz-4.15.0-126-generic 2020-12-05T17:21:05.470 INFO:teuthology.orchestra.run.smithi061.stdout:Found initrd image: /boot/initrd.img-4.15.0-126-generic 2020-12-05T17:21:05.920 INFO:teuthology.orchestra.run.smithi061.stdout:Found linux image: /boot/vmlinuz-4.15.0-124-generic 2020-12-05T17:21:05.952 INFO:teuthology.orchestra.run.smithi061.stdout:Found initrd image: /boot/initrd.img-4.15.0-124-generic 2020-12-05T17:21:07.498 INFO:teuthology.orchestra.run.smithi061.stdout:done 2020-12-05T17:21:07.978 INFO:teuthology.orchestra.run.smithi061:> dpkg -s linux-image-generic 2020-12-05T17:21:07.998 INFO:teuthology.orchestra.run.smithi061.stdout:Package: linux-image-generic 2020-12-05T17:21:07.998 INFO:teuthology.orchestra.run.smithi061.stdout:Status: install ok installed 2020-12-05T17:21:07.999 INFO:teuthology.orchestra.run.smithi061.stdout:Priority: optional 2020-12-05T17:21:07.999 INFO:teuthology.orchestra.run.smithi061.stdout:Section: kernel 2020-12-05T17:21:07.999 INFO:teuthology.orchestra.run.smithi061.stdout:Installed-Size: 16 2020-12-05T17:21:07.999 INFO:teuthology.orchestra.run.smithi061.stdout:Maintainer: Ubuntu Kernel Team 2020-12-05T17:21:07.999 INFO:teuthology.orchestra.run.smithi061.stdout:Architecture: amd64 2020-12-05T17:21:08.000 INFO:teuthology.orchestra.run.smithi061.stdout:Source: linux-meta 2020-12-05T17:21:08.000 INFO:teuthology.orchestra.run.smithi061.stdout:Version: 4.15.0.126.113 2020-12-05T17:21:08.000 INFO:teuthology.orchestra.run.smithi061.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2020-12-05T17:21:08.000 INFO:teuthology.orchestra.run.smithi061.stdout:Depends: linux-image-4.15.0-126-generic, linux-modules-extra-4.15.0-126-generic, linux-firmware, intel-microcode, amd64-microcode 2020-12-05T17:21:08.000 INFO:teuthology.orchestra.run.smithi061.stdout:Recommends: thermald 2020-12-05T17:21:08.001 INFO:teuthology.orchestra.run.smithi061.stdout:Description: Generic Linux kernel image 2020-12-05T17:21:08.001 INFO:teuthology.orchestra.run.smithi061.stdout: This package will always depend on the latest generic kernel image 2020-12-05T17:21:08.001 INFO:teuthology.orchestra.run.smithi061.stdout: available. 2020-12-05T17:21:08.001 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-124-generic Expected: 4.15.0-126-generic 2020-12-05T17:21:08.002 INFO:teuthology.orchestra.run.smithi036:> uname -r 2020-12-05T17:21:08.008 INFO:teuthology.orchestra.run.smithi036.stdout:4.15.0-124-generic 2020-12-05T17:21:08.009 INFO:teuthology.task.kernel:Running kernel on smithi036: 4.15.0-124-generic 2020-12-05T17:21:08.009 INFO:teuthology.orchestra.run.smithi036:> sudo apt-get clean 2020-12-05T17:21:08.140 INFO:teuthology.orchestra.run.smithi036:> sudo apt-get update 2020-12-05T17:21:08.278 INFO:teuthology.orchestra.run.smithi036.stdout:Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2020-12-05T17:21:08.396 INFO:teuthology.orchestra.run.smithi036.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-12-05T17:21:08.477 INFO:teuthology.orchestra.run.smithi036.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2020-12-05T17:21:08.570 INFO:teuthology.orchestra.run.smithi036.stdout:Get:4 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,453 kB] 2020-12-05T17:21:08.700 INFO:teuthology.orchestra.run.smithi036.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [870 kB] 2020-12-05T17:21:08.716 INFO:teuthology.orchestra.run.smithi036.stdout:Get:6 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [284 kB] 2020-12-05T17:21:08.718 INFO:teuthology.orchestra.run.smithi036.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [184 kB] 2020-12-05T17:21:08.721 INFO:teuthology.orchestra.run.smithi036.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [24.4 kB] 2020-12-05T17:21:08.722 INFO:teuthology.orchestra.run.smithi036.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,092 kB] 2020-12-05T17:21:08.734 INFO:teuthology.orchestra.run.smithi036.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [968 kB] 2020-12-05T17:21:08.744 INFO:teuthology.orchestra.run.smithi036.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2020-12-05T17:21:08.744 INFO:teuthology.orchestra.run.smithi036.stdout:Get:12 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [244 kB] 2020-12-05T17:21:08.885 INFO:teuthology.orchestra.run.smithi036.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,172 kB] 2020-12-05T17:21:09.259 INFO:teuthology.orchestra.run.smithi036.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,789 kB] 2020-12-05T17:21:09.363 INFO:teuthology.orchestra.run.smithi036.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [376 kB] 2020-12-05T17:21:09.380 INFO:teuthology.orchestra.run.smithi036.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [206 kB] 2020-12-05T17:21:09.389 INFO:teuthology.orchestra.run.smithi036.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [28.0 kB] 2020-12-05T17:21:09.390 INFO:teuthology.orchestra.run.smithi036.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,548 kB] 2020-12-05T17:21:09.434 INFO:teuthology.orchestra.run.smithi036.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,695 kB] 2020-12-05T17:21:09.491 INFO:teuthology.orchestra.run.smithi036.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [357 kB] 2020-12-05T17:21:10.443 INFO:teuthology.orchestra.run.smithi036.stdout:Fetched 12.5 MB in 2s (6,001 kB/s) 2020-12-05T17:21:12.250 INFO:teuthology.orchestra.run.smithi036.stdout:Reading package lists... 2020-12-05T17:21:12.298 INFO:teuthology.orchestra.run.smithi036:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-12-05T17:21:12.382 INFO:teuthology.orchestra.run.smithi036.stdout:Reading package lists... 2020-12-05T17:21:12.647 INFO:teuthology.orchestra.run.smithi036.stdout:Building dependency tree... 2020-12-05T17:21:12.648 INFO:teuthology.orchestra.run.smithi036.stdout:Reading state information... 2020-12-05T17:21:12.737 INFO:teuthology.orchestra.run.smithi036.stderr:E: Unable to locate package linux-image-current-generic 2020-12-05T17:21:12.739 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-12-05T17:21:12.739 INFO:teuthology.orchestra.run.smithi036:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-12-05T17:21:12.821 INFO:teuthology.orchestra.run.smithi036.stdout:Reading package lists... 2020-12-05T17:21:13.068 INFO:teuthology.orchestra.run.smithi036.stdout:Building dependency tree... 2020-12-05T17:21:13.069 INFO:teuthology.orchestra.run.smithi036.stdout:Reading state information... 2020-12-05T17:21:13.254 INFO:teuthology.orchestra.run.smithi036.stdout:The following additional packages will be installed: 2020-12-05T17:21:13.254 INFO:teuthology.orchestra.run.smithi036.stdout: linux-generic linux-headers-4.15.0-126 linux-headers-4.15.0-126-generic 2020-12-05T17:21:13.254 INFO:teuthology.orchestra.run.smithi036.stdout: linux-headers-generic linux-image-4.15.0-126-generic 2020-12-05T17:21:13.255 INFO:teuthology.orchestra.run.smithi036.stdout: linux-modules-4.15.0-126-generic linux-modules-extra-4.15.0-126-generic 2020-12-05T17:21:13.256 INFO:teuthology.orchestra.run.smithi036.stdout:Suggested packages: 2020-12-05T17:21:13.256 INFO:teuthology.orchestra.run.smithi036.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2020-12-05T17:21:13.256 INFO:teuthology.orchestra.run.smithi036.stdout:Recommended packages: 2020-12-05T17:21:13.257 INFO:teuthology.orchestra.run.smithi036.stdout: thermald 2020-12-05T17:21:13.289 INFO:teuthology.orchestra.run.smithi036.stdout:The following NEW packages will be installed: 2020-12-05T17:21:13.290 INFO:teuthology.orchestra.run.smithi036.stdout: linux-headers-4.15.0-126 linux-headers-4.15.0-126-generic 2020-12-05T17:21:13.290 INFO:teuthology.orchestra.run.smithi036.stdout: linux-image-4.15.0-126-generic linux-modules-4.15.0-126-generic 2020-12-05T17:21:13.290 INFO:teuthology.orchestra.run.smithi036.stdout: linux-modules-extra-4.15.0-126-generic 2020-12-05T17:21:13.291 INFO:teuthology.orchestra.run.smithi036.stdout:The following packages will be upgraded: 2020-12-05T17:21:13.292 INFO:teuthology.orchestra.run.smithi036.stdout: linux-generic linux-headers-generic linux-image-generic 2020-12-05T17:21:13.470 INFO:teuthology.orchestra.run.smithi036.stdout:3 upgraded, 5 newly installed, 0 to remove and 7 not upgraded. 2020-12-05T17:21:13.470 INFO:teuthology.orchestra.run.smithi036.stdout:Need to get 65.9 MB of archives. 2020-12-05T17:21:13.471 INFO:teuthology.orchestra.run.smithi036.stdout:After this operation, 333 MB of additional disk space will be used. 2020-12-05T17:21:13.471 INFO:teuthology.orchestra.run.smithi036.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-126-generic amd64 4.15.0-126.129 [13.1 MB] 2020-12-05T17:21:14.483 INFO:teuthology.orchestra.run.smithi036.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-126-generic amd64 4.15.0-126.129 [8,013 kB] 2020-12-05T17:21:14.686 INFO:teuthology.orchestra.run.smithi036.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-126-generic amd64 4.15.0-126.129 [32.8 MB] 2020-12-05T17:21:15.688 INFO:teuthology.orchestra.run.smithi036.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.126.113 [1,872 B] 2020-12-05T17:21:15.688 INFO:teuthology.orchestra.run.smithi036.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.126.113 [2,620 B] 2020-12-05T17:21:15.688 INFO:teuthology.orchestra.run.smithi036.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-126 all 4.15.0-126.129 [10.9 MB] 2020-12-05T17:21:16.008 INFO:teuthology.orchestra.run.smithi036.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-126-generic amd64 4.15.0-126.129 [1,096 kB] 2020-12-05T17:21:16.045 INFO:teuthology.orchestra.run.smithi036.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.126.113 [2,524 B] 2020-12-05T17:21:16.556 INFO:teuthology.orchestra.run.smithi036.stdout:Fetched 65.9 MB in 3s (24.1 MB/s) 2020-12-05T17:21:16.726 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package linux-modules-4.15.0-126-generic. 2020-12-05T17:21:16.765 INFO:teuthology.orchestra.run.smithi036.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102077 files and directories currently installed.) 2020-12-05T17:21:16.768 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../0-linux-modules-4.15.0-126-generic_4.15.0-126.129_amd64.deb ... 2020-12-05T17:21:16.814 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking linux-modules-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:21:18.430 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package linux-image-4.15.0-126-generic. 2020-12-05T17:21:18.443 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../1-linux-image-4.15.0-126-generic_4.15.0-126.129_amd64.deb ... 2020-12-05T17:21:18.534 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking linux-image-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:21:19.067 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-126-generic. 2020-12-05T17:21:19.079 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-126-generic_4.15.0-126.129_amd64.deb ... 2020-12-05T17:21:19.117 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking linux-modules-extra-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:21:22.830 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../3-linux-generic_4.15.0.126.113_amd64.deb ... 2020-12-05T17:21:22.951 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking linux-generic (4.15.0.126.113) over (4.15.0.124.111) ... 2020-12-05T17:21:23.257 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.126.113_amd64.deb ... 2020-12-05T17:21:23.396 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking linux-image-generic (4.15.0.126.113) over (4.15.0.124.111) ... 2020-12-05T17:21:23.748 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package linux-headers-4.15.0-126. 2020-12-05T17:21:23.762 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../5-linux-headers-4.15.0-126_4.15.0-126.129_all.deb ... 2020-12-05T17:21:23.798 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking linux-headers-4.15.0-126 (4.15.0-126.129) ... 2020-12-05T17:21:28.740 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package linux-headers-4.15.0-126-generic. 2020-12-05T17:21:28.761 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../6-linux-headers-4.15.0-126-generic_4.15.0-126.129_amd64.deb ... 2020-12-05T17:21:28.807 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking linux-headers-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:21:31.717 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.126.113_amd64.deb ... 2020-12-05T17:21:31.822 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking linux-headers-generic (4.15.0.126.113) over (4.15.0.124.111) ... 2020-12-05T17:21:32.151 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up linux-headers-4.15.0-126 (4.15.0-126.129) ... 2020-12-05T17:21:32.285 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up linux-modules-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:21:32.853 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up linux-headers-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:21:32.979 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up linux-image-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:21:33.460 INFO:teuthology.orchestra.run.smithi036.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-126-generic 2020-12-05T17:21:33.460 INFO:teuthology.orchestra.run.smithi036.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-126-generic 2020-12-05T17:21:33.572 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up linux-headers-generic (4.15.0.126.113) ... 2020-12-05T17:21:33.690 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up linux-modules-extra-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:21:34.200 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up linux-image-generic (4.15.0.126.113) ... 2020-12-05T17:21:34.326 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up linux-generic (4.15.0.126.113) ... 2020-12-05T17:21:34.452 INFO:teuthology.orchestra.run.smithi036.stdout:Processing triggers for linux-image-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:21:34.637 INFO:teuthology.orchestra.run.smithi036.stdout:/etc/kernel/postinst.d/initramfs-tools: 2020-12-05T17:21:34.637 INFO:teuthology.orchestra.run.smithi036.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-126-generic 2020-12-05T17:21:49.944 INFO:teuthology.orchestra.run.smithi036.stdout:/etc/kernel/postinst.d/zz-update-grub: 2020-12-05T17:21:49.944 INFO:teuthology.orchestra.run.smithi036.stdout:Sourcing file `/etc/default/grub' 2020-12-05T17:21:49.982 INFO:teuthology.orchestra.run.smithi036.stdout:Generating grub configuration file ... 2020-12-05T17:21:50.279 INFO:teuthology.orchestra.run.smithi036.stdout:Found linux image: /boot/vmlinuz-4.15.0-126-generic 2020-12-05T17:21:50.333 INFO:teuthology.orchestra.run.smithi036.stdout:Found initrd image: /boot/initrd.img-4.15.0-126-generic 2020-12-05T17:21:50.690 INFO:teuthology.orchestra.run.smithi036.stdout:Found linux image: /boot/vmlinuz-4.15.0-124-generic 2020-12-05T17:21:50.718 INFO:teuthology.orchestra.run.smithi036.stdout:Found initrd image: /boot/initrd.img-4.15.0-124-generic 2020-12-05T17:21:52.095 INFO:teuthology.orchestra.run.smithi036.stdout:done 2020-12-05T17:21:52.519 INFO:teuthology.orchestra.run.smithi036:> dpkg -s linux-image-generic 2020-12-05T17:21:52.540 INFO:teuthology.orchestra.run.smithi036.stdout:Package: linux-image-generic 2020-12-05T17:21:52.540 INFO:teuthology.orchestra.run.smithi036.stdout:Status: install ok installed 2020-12-05T17:21:52.541 INFO:teuthology.orchestra.run.smithi036.stdout:Priority: optional 2020-12-05T17:21:52.541 INFO:teuthology.orchestra.run.smithi036.stdout:Section: kernel 2020-12-05T17:21:52.541 INFO:teuthology.orchestra.run.smithi036.stdout:Installed-Size: 16 2020-12-05T17:21:52.541 INFO:teuthology.orchestra.run.smithi036.stdout:Maintainer: Ubuntu Kernel Team 2020-12-05T17:21:52.542 INFO:teuthology.orchestra.run.smithi036.stdout:Architecture: amd64 2020-12-05T17:21:52.542 INFO:teuthology.orchestra.run.smithi036.stdout:Source: linux-meta 2020-12-05T17:21:52.542 INFO:teuthology.orchestra.run.smithi036.stdout:Version: 4.15.0.126.113 2020-12-05T17:21:52.542 INFO:teuthology.orchestra.run.smithi036.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2020-12-05T17:21:52.543 INFO:teuthology.orchestra.run.smithi036.stdout:Depends: linux-image-4.15.0-126-generic, linux-modules-extra-4.15.0-126-generic, linux-firmware, intel-microcode, amd64-microcode 2020-12-05T17:21:52.543 INFO:teuthology.orchestra.run.smithi036.stdout:Recommends: thermald 2020-12-05T17:21:52.543 INFO:teuthology.orchestra.run.smithi036.stdout:Description: Generic Linux kernel image 2020-12-05T17:21:52.543 INFO:teuthology.orchestra.run.smithi036.stdout: This package will always depend on the latest generic kernel image 2020-12-05T17:21:52.543 INFO:teuthology.orchestra.run.smithi036.stdout: available. 2020-12-05T17:21:52.544 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-124-generic Expected: 4.15.0-126-generic 2020-12-05T17:21:52.544 INFO:teuthology.orchestra.run.smithi002:> uname -r 2020-12-05T17:21:52.551 INFO:teuthology.orchestra.run.smithi002.stdout:4.15.0-124-generic 2020-12-05T17:21:52.551 INFO:teuthology.task.kernel:Running kernel on smithi002: 4.15.0-124-generic 2020-12-05T17:21:52.551 INFO:teuthology.orchestra.run.smithi002:> sudo apt-get clean 2020-12-05T17:21:52.677 INFO:teuthology.orchestra.run.smithi002:> sudo apt-get update 2020-12-05T17:21:52.928 INFO:teuthology.orchestra.run.smithi002.stdout:Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2020-12-05T17:21:52.961 INFO:teuthology.orchestra.run.smithi002.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-12-05T17:21:53.025 INFO:teuthology.orchestra.run.smithi002.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2020-12-05T17:21:53.305 INFO:teuthology.orchestra.run.smithi002.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2020-12-05T17:21:53.384 INFO:teuthology.orchestra.run.smithi002.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,453 kB] 2020-12-05T17:21:53.520 INFO:teuthology.orchestra.run.smithi002.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,789 kB] 2020-12-05T17:21:53.818 INFO:teuthology.orchestra.run.smithi002.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [870 kB] 2020-12-05T17:21:53.853 INFO:teuthology.orchestra.run.smithi002.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [284 kB] 2020-12-05T17:21:53.864 INFO:teuthology.orchestra.run.smithi002.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [184 kB] 2020-12-05T17:21:53.871 INFO:teuthology.orchestra.run.smithi002.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [24.4 kB] 2020-12-05T17:21:53.872 INFO:teuthology.orchestra.run.smithi002.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [968 kB] 2020-12-05T17:21:53.910 INFO:teuthology.orchestra.run.smithi002.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,172 kB] 2020-12-05T17:21:53.929 INFO:teuthology.orchestra.run.smithi002.stdout:Get:13 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,092 kB] 2020-12-05T17:21:53.946 INFO:teuthology.orchestra.run.smithi002.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [376 kB] 2020-12-05T17:21:53.957 INFO:teuthology.orchestra.run.smithi002.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [206 kB] 2020-12-05T17:21:53.963 INFO:teuthology.orchestra.run.smithi002.stdout:Get:16 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [244 kB] 2020-12-05T17:21:53.964 INFO:teuthology.orchestra.run.smithi002.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [28.0 kB] 2020-12-05T17:21:53.964 INFO:teuthology.orchestra.run.smithi002.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,548 kB] 2020-12-05T17:21:54.032 INFO:teuthology.orchestra.run.smithi002.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,695 kB] 2020-12-05T17:21:54.080 INFO:teuthology.orchestra.run.smithi002.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [357 kB] 2020-12-05T17:21:55.500 INFO:teuthology.orchestra.run.smithi002.stdout:Fetched 12.5 MB in 2s (6,368 kB/s) 2020-12-05T17:21:57.216 INFO:teuthology.orchestra.run.smithi002.stdout:Reading package lists... 2020-12-05T17:21:57.264 INFO:teuthology.orchestra.run.smithi002:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-12-05T17:21:57.346 INFO:teuthology.orchestra.run.smithi002.stdout:Reading package lists... 2020-12-05T17:21:57.611 INFO:teuthology.orchestra.run.smithi002.stdout:Building dependency tree... 2020-12-05T17:21:57.612 INFO:teuthology.orchestra.run.smithi002.stdout:Reading state information... 2020-12-05T17:21:57.704 INFO:teuthology.orchestra.run.smithi002.stderr:E: Unable to locate package linux-image-current-generic 2020-12-05T17:21:57.706 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-12-05T17:21:57.707 INFO:teuthology.orchestra.run.smithi002:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-12-05T17:21:57.785 INFO:teuthology.orchestra.run.smithi002.stdout:Reading package lists... 2020-12-05T17:21:58.048 INFO:teuthology.orchestra.run.smithi002.stdout:Building dependency tree... 2020-12-05T17:21:58.049 INFO:teuthology.orchestra.run.smithi002.stdout:Reading state information... 2020-12-05T17:21:58.234 INFO:teuthology.orchestra.run.smithi002.stdout:The following additional packages will be installed: 2020-12-05T17:21:58.234 INFO:teuthology.orchestra.run.smithi002.stdout: linux-generic linux-headers-4.15.0-126 linux-headers-4.15.0-126-generic 2020-12-05T17:21:58.235 INFO:teuthology.orchestra.run.smithi002.stdout: linux-headers-generic linux-image-4.15.0-126-generic 2020-12-05T17:21:58.235 INFO:teuthology.orchestra.run.smithi002.stdout: linux-modules-4.15.0-126-generic linux-modules-extra-4.15.0-126-generic 2020-12-05T17:21:58.236 INFO:teuthology.orchestra.run.smithi002.stdout:Suggested packages: 2020-12-05T17:21:58.236 INFO:teuthology.orchestra.run.smithi002.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2020-12-05T17:21:58.237 INFO:teuthology.orchestra.run.smithi002.stdout:Recommended packages: 2020-12-05T17:21:58.237 INFO:teuthology.orchestra.run.smithi002.stdout: thermald 2020-12-05T17:21:58.272 INFO:teuthology.orchestra.run.smithi002.stdout:The following NEW packages will be installed: 2020-12-05T17:21:58.272 INFO:teuthology.orchestra.run.smithi002.stdout: linux-headers-4.15.0-126 linux-headers-4.15.0-126-generic 2020-12-05T17:21:58.272 INFO:teuthology.orchestra.run.smithi002.stdout: linux-image-4.15.0-126-generic linux-modules-4.15.0-126-generic 2020-12-05T17:21:58.273 INFO:teuthology.orchestra.run.smithi002.stdout: linux-modules-extra-4.15.0-126-generic 2020-12-05T17:21:58.274 INFO:teuthology.orchestra.run.smithi002.stdout:The following packages will be upgraded: 2020-12-05T17:21:58.275 INFO:teuthology.orchestra.run.smithi002.stdout: linux-generic linux-headers-generic linux-image-generic 2020-12-05T17:21:58.452 INFO:teuthology.orchestra.run.smithi002.stdout:3 upgraded, 5 newly installed, 0 to remove and 7 not upgraded. 2020-12-05T17:21:58.453 INFO:teuthology.orchestra.run.smithi002.stdout:Need to get 65.9 MB of archives. 2020-12-05T17:21:58.453 INFO:teuthology.orchestra.run.smithi002.stdout:After this operation, 333 MB of additional disk space will be used. 2020-12-05T17:21:58.453 INFO:teuthology.orchestra.run.smithi002.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-126-generic amd64 4.15.0-126.129 [13.1 MB] 2020-12-05T17:21:59.468 INFO:teuthology.orchestra.run.smithi002.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-126-generic amd64 4.15.0-126.129 [8,013 kB] 2020-12-05T17:21:59.670 INFO:teuthology.orchestra.run.smithi002.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-126-generic amd64 4.15.0-126.129 [32.8 MB] 2020-12-05T17:22:00.669 INFO:teuthology.orchestra.run.smithi002.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.126.113 [1,872 B] 2020-12-05T17:22:00.670 INFO:teuthology.orchestra.run.smithi002.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.126.113 [2,620 B] 2020-12-05T17:22:00.670 INFO:teuthology.orchestra.run.smithi002.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-126 all 4.15.0-126.129 [10.9 MB] 2020-12-05T17:22:00.939 INFO:teuthology.orchestra.run.smithi002.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-126-generic amd64 4.15.0-126.129 [1,096 kB] 2020-12-05T17:22:00.997 INFO:teuthology.orchestra.run.smithi002.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.126.113 [2,524 B] 2020-12-05T17:22:01.469 INFO:teuthology.orchestra.run.smithi002.stdout:Fetched 65.9 MB in 3s (24.4 MB/s) 2020-12-05T17:22:01.641 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package linux-modules-4.15.0-126-generic. 2020-12-05T17:22:01.682 INFO:teuthology.orchestra.run.smithi002.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102077 files and directories currently installed.) 2020-12-05T17:22:01.684 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../0-linux-modules-4.15.0-126-generic_4.15.0-126.129_amd64.deb ... 2020-12-05T17:22:01.738 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking linux-modules-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:22:03.438 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package linux-image-4.15.0-126-generic. 2020-12-05T17:22:03.450 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../1-linux-image-4.15.0-126-generic_4.15.0-126.129_amd64.deb ... 2020-12-05T17:22:03.547 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking linux-image-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:22:04.058 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-126-generic. 2020-12-05T17:22:04.071 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-126-generic_4.15.0-126.129_amd64.deb ... 2020-12-05T17:22:04.108 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking linux-modules-extra-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:22:07.906 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../3-linux-generic_4.15.0.126.113_amd64.deb ... 2020-12-05T17:22:08.027 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking linux-generic (4.15.0.126.113) over (4.15.0.124.111) ... 2020-12-05T17:22:08.350 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.126.113_amd64.deb ... 2020-12-05T17:22:08.471 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking linux-image-generic (4.15.0.126.113) over (4.15.0.124.111) ... 2020-12-05T17:22:08.783 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package linux-headers-4.15.0-126. 2020-12-05T17:22:08.796 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../5-linux-headers-4.15.0-126_4.15.0-126.129_all.deb ... 2020-12-05T17:22:08.841 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking linux-headers-4.15.0-126 (4.15.0-126.129) ... 2020-12-05T17:22:13.614 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package linux-headers-4.15.0-126-generic. 2020-12-05T17:22:13.636 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../6-linux-headers-4.15.0-126-generic_4.15.0-126.129_amd64.deb ... 2020-12-05T17:22:13.664 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking linux-headers-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:22:16.108 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.126.113_amd64.deb ... 2020-12-05T17:22:16.238 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking linux-headers-generic (4.15.0.126.113) over (4.15.0.124.111) ... 2020-12-05T17:22:16.565 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up linux-headers-4.15.0-126 (4.15.0-126.129) ... 2020-12-05T17:22:16.701 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up linux-modules-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:22:17.253 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up linux-headers-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:22:17.379 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up linux-image-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:22:17.875 INFO:teuthology.orchestra.run.smithi002.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-126-generic 2020-12-05T17:22:17.875 INFO:teuthology.orchestra.run.smithi002.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-126-generic 2020-12-05T17:22:18.014 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up linux-headers-generic (4.15.0.126.113) ... 2020-12-05T17:22:18.156 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up linux-modules-extra-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:22:18.658 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up linux-image-generic (4.15.0.126.113) ... 2020-12-05T17:22:18.759 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up linux-generic (4.15.0.126.113) ... 2020-12-05T17:22:18.887 INFO:teuthology.orchestra.run.smithi002.stdout:Processing triggers for linux-image-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:22:19.062 INFO:teuthology.orchestra.run.smithi002.stdout:/etc/kernel/postinst.d/initramfs-tools: 2020-12-05T17:22:19.062 INFO:teuthology.orchestra.run.smithi002.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-126-generic 2020-12-05T17:22:34.644 INFO:teuthology.orchestra.run.smithi002.stdout:/etc/kernel/postinst.d/zz-update-grub: 2020-12-05T17:22:34.645 INFO:teuthology.orchestra.run.smithi002.stdout:Sourcing file `/etc/default/grub' 2020-12-05T17:22:34.678 INFO:teuthology.orchestra.run.smithi002.stdout:Generating grub configuration file ... 2020-12-05T17:22:35.241 INFO:teuthology.orchestra.run.smithi002.stdout:Found linux image: /boot/vmlinuz-4.15.0-126-generic 2020-12-05T17:22:35.282 INFO:teuthology.orchestra.run.smithi002.stdout:Found initrd image: /boot/initrd.img-4.15.0-126-generic 2020-12-05T17:22:35.613 INFO:teuthology.orchestra.run.smithi002.stdout:Found linux image: /boot/vmlinuz-4.15.0-124-generic 2020-12-05T17:22:35.645 INFO:teuthology.orchestra.run.smithi002.stdout:Found initrd image: /boot/initrd.img-4.15.0-124-generic 2020-12-05T17:22:37.054 INFO:teuthology.orchestra.run.smithi002.stdout:done 2020-12-05T17:22:37.512 INFO:teuthology.orchestra.run.smithi002:> dpkg -s linux-image-generic 2020-12-05T17:22:37.534 INFO:teuthology.orchestra.run.smithi002.stdout:Package: linux-image-generic 2020-12-05T17:22:37.535 INFO:teuthology.orchestra.run.smithi002.stdout:Status: install ok installed 2020-12-05T17:22:37.535 INFO:teuthology.orchestra.run.smithi002.stdout:Priority: optional 2020-12-05T17:22:37.535 INFO:teuthology.orchestra.run.smithi002.stdout:Section: kernel 2020-12-05T17:22:37.535 INFO:teuthology.orchestra.run.smithi002.stdout:Installed-Size: 16 2020-12-05T17:22:37.536 INFO:teuthology.orchestra.run.smithi002.stdout:Maintainer: Ubuntu Kernel Team 2020-12-05T17:22:37.536 INFO:teuthology.orchestra.run.smithi002.stdout:Architecture: amd64 2020-12-05T17:22:37.536 INFO:teuthology.orchestra.run.smithi002.stdout:Source: linux-meta 2020-12-05T17:22:37.536 INFO:teuthology.orchestra.run.smithi002.stdout:Version: 4.15.0.126.113 2020-12-05T17:22:37.537 INFO:teuthology.orchestra.run.smithi002.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2020-12-05T17:22:37.537 INFO:teuthology.orchestra.run.smithi002.stdout:Depends: linux-image-4.15.0-126-generic, linux-modules-extra-4.15.0-126-generic, linux-firmware, intel-microcode, amd64-microcode 2020-12-05T17:22:37.537 INFO:teuthology.orchestra.run.smithi002.stdout:Recommends: thermald 2020-12-05T17:22:37.537 INFO:teuthology.orchestra.run.smithi002.stdout:Description: Generic Linux kernel image 2020-12-05T17:22:37.537 INFO:teuthology.orchestra.run.smithi002.stdout: This package will always depend on the latest generic kernel image 2020-12-05T17:22:37.538 INFO:teuthology.orchestra.run.smithi002.stdout: available. 2020-12-05T17:22:37.539 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-124-generic Expected: 4.15.0-126-generic 2020-12-05T17:22:37.540 INFO:teuthology.orchestra.run.smithi053:> uname -r 2020-12-05T17:22:37.546 INFO:teuthology.orchestra.run.smithi053.stdout:4.15.0-124-generic 2020-12-05T17:22:37.547 INFO:teuthology.task.kernel:Running kernel on smithi053: 4.15.0-124-generic 2020-12-05T17:22:37.547 INFO:teuthology.orchestra.run.smithi053:> sudo apt-get clean 2020-12-05T17:22:37.688 INFO:teuthology.orchestra.run.smithi053:> sudo apt-get update 2020-12-05T17:22:37.821 INFO:teuthology.orchestra.run.smithi053.stdout:Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2020-12-05T17:22:37.935 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-12-05T17:22:38.013 INFO:teuthology.orchestra.run.smithi053.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2020-12-05T17:22:38.083 INFO:teuthology.orchestra.run.smithi053.stdout:Get:4 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,453 kB] 2020-12-05T17:22:38.208 INFO:teuthology.orchestra.run.smithi053.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [870 kB] 2020-12-05T17:22:38.221 INFO:teuthology.orchestra.run.smithi053.stdout:Get:6 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [284 kB] 2020-12-05T17:22:38.225 INFO:teuthology.orchestra.run.smithi053.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [184 kB] 2020-12-05T17:22:38.227 INFO:teuthology.orchestra.run.smithi053.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [24.4 kB] 2020-12-05T17:22:38.227 INFO:teuthology.orchestra.run.smithi053.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,092 kB] 2020-12-05T17:22:38.239 INFO:teuthology.orchestra.run.smithi053.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [968 kB] 2020-12-05T17:22:38.257 INFO:teuthology.orchestra.run.smithi053.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [244 kB] 2020-12-05T17:22:38.271 INFO:teuthology.orchestra.run.smithi053.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2020-12-05T17:22:38.388 INFO:teuthology.orchestra.run.smithi053.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,172 kB] 2020-12-05T17:22:38.717 INFO:teuthology.orchestra.run.smithi053.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,789 kB] 2020-12-05T17:22:38.817 INFO:teuthology.orchestra.run.smithi053.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [376 kB] 2020-12-05T17:22:38.833 INFO:teuthology.orchestra.run.smithi053.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [206 kB] 2020-12-05T17:22:38.842 INFO:teuthology.orchestra.run.smithi053.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [28.0 kB] 2020-12-05T17:22:38.843 INFO:teuthology.orchestra.run.smithi053.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,548 kB] 2020-12-05T17:22:38.886 INFO:teuthology.orchestra.run.smithi053.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,695 kB] 2020-12-05T17:22:38.936 INFO:teuthology.orchestra.run.smithi053.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [357 kB] 2020-12-05T17:22:39.991 INFO:teuthology.orchestra.run.smithi053.stdout:Fetched 12.5 MB in 2s (5,999 kB/s) 2020-12-05T17:22:41.686 INFO:teuthology.orchestra.run.smithi053.stdout:Reading package lists... 2020-12-05T17:22:41.732 INFO:teuthology.orchestra.run.smithi053:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-12-05T17:22:41.809 INFO:teuthology.orchestra.run.smithi053.stdout:Reading package lists... 2020-12-05T17:22:42.073 INFO:teuthology.orchestra.run.smithi053.stdout:Building dependency tree... 2020-12-05T17:22:42.073 INFO:teuthology.orchestra.run.smithi053.stdout:Reading state information... 2020-12-05T17:22:42.160 INFO:teuthology.orchestra.run.smithi053.stderr:E: Unable to locate package linux-image-current-generic 2020-12-05T17:22:42.161 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-12-05T17:22:42.162 INFO:teuthology.orchestra.run.smithi053:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-12-05T17:22:42.244 INFO:teuthology.orchestra.run.smithi053.stdout:Reading package lists... 2020-12-05T17:22:42.504 INFO:teuthology.orchestra.run.smithi053.stdout:Building dependency tree... 2020-12-05T17:22:42.504 INFO:teuthology.orchestra.run.smithi053.stdout:Reading state information... 2020-12-05T17:22:42.690 INFO:teuthology.orchestra.run.smithi053.stdout:The following additional packages will be installed: 2020-12-05T17:22:42.691 INFO:teuthology.orchestra.run.smithi053.stdout: linux-generic linux-headers-4.15.0-126 linux-headers-4.15.0-126-generic 2020-12-05T17:22:42.691 INFO:teuthology.orchestra.run.smithi053.stdout: linux-headers-generic linux-image-4.15.0-126-generic 2020-12-05T17:22:42.691 INFO:teuthology.orchestra.run.smithi053.stdout: linux-modules-4.15.0-126-generic linux-modules-extra-4.15.0-126-generic 2020-12-05T17:22:42.693 INFO:teuthology.orchestra.run.smithi053.stdout:Suggested packages: 2020-12-05T17:22:42.693 INFO:teuthology.orchestra.run.smithi053.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2020-12-05T17:22:42.693 INFO:teuthology.orchestra.run.smithi053.stdout:Recommended packages: 2020-12-05T17:22:42.694 INFO:teuthology.orchestra.run.smithi053.stdout: thermald 2020-12-05T17:22:42.729 INFO:teuthology.orchestra.run.smithi053.stdout:The following NEW packages will be installed: 2020-12-05T17:22:42.729 INFO:teuthology.orchestra.run.smithi053.stdout: linux-headers-4.15.0-126 linux-headers-4.15.0-126-generic 2020-12-05T17:22:42.729 INFO:teuthology.orchestra.run.smithi053.stdout: linux-image-4.15.0-126-generic linux-modules-4.15.0-126-generic 2020-12-05T17:22:42.730 INFO:teuthology.orchestra.run.smithi053.stdout: linux-modules-extra-4.15.0-126-generic 2020-12-05T17:22:42.731 INFO:teuthology.orchestra.run.smithi053.stdout:The following packages will be upgraded: 2020-12-05T17:22:42.731 INFO:teuthology.orchestra.run.smithi053.stdout: linux-generic linux-headers-generic linux-image-generic 2020-12-05T17:22:42.911 INFO:teuthology.orchestra.run.smithi053.stdout:3 upgraded, 5 newly installed, 0 to remove and 7 not upgraded. 2020-12-05T17:22:42.912 INFO:teuthology.orchestra.run.smithi053.stdout:Need to get 65.9 MB of archives. 2020-12-05T17:22:42.912 INFO:teuthology.orchestra.run.smithi053.stdout:After this operation, 333 MB of additional disk space will be used. 2020-12-05T17:22:42.912 INFO:teuthology.orchestra.run.smithi053.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-126-generic amd64 4.15.0-126.129 [13.1 MB] 2020-12-05T17:22:43.914 INFO:teuthology.orchestra.run.smithi053.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-126-generic amd64 4.15.0-126.129 [8,013 kB] 2020-12-05T17:22:44.166 INFO:teuthology.orchestra.run.smithi053.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-126-generic amd64 4.15.0-126.129 [32.8 MB] 2020-12-05T17:22:45.129 INFO:teuthology.orchestra.run.smithi053.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.126.113 [1,872 B] 2020-12-05T17:22:45.129 INFO:teuthology.orchestra.run.smithi053.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.126.113 [2,620 B] 2020-12-05T17:22:45.130 INFO:teuthology.orchestra.run.smithi053.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-126 all 4.15.0-126.129 [10.9 MB] 2020-12-05T17:22:45.442 INFO:teuthology.orchestra.run.smithi053.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-126-generic amd64 4.15.0-126.129 [1,096 kB] 2020-12-05T17:22:45.466 INFO:teuthology.orchestra.run.smithi053.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.126.113 [2,524 B] 2020-12-05T17:22:45.963 INFO:teuthology.orchestra.run.smithi053.stdout:Fetched 65.9 MB in 3s (24.3 MB/s) 2020-12-05T17:22:46.149 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package linux-modules-4.15.0-126-generic. 2020-12-05T17:22:46.186 INFO:teuthology.orchestra.run.smithi053.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102077 files and directories currently installed.) 2020-12-05T17:22:46.189 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../0-linux-modules-4.15.0-126-generic_4.15.0-126.129_amd64.deb ... 2020-12-05T17:22:46.237 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-modules-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:22:48.020 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package linux-image-4.15.0-126-generic. 2020-12-05T17:22:48.034 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../1-linux-image-4.15.0-126-generic_4.15.0-126.129_amd64.deb ... 2020-12-05T17:22:48.118 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-image-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:22:48.631 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-126-generic. 2020-12-05T17:22:48.640 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-126-generic_4.15.0-126.129_amd64.deb ... 2020-12-05T17:22:48.682 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-modules-extra-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:22:52.396 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../3-linux-generic_4.15.0.126.113_amd64.deb ... 2020-12-05T17:22:52.515 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-generic (4.15.0.126.113) over (4.15.0.124.111) ... 2020-12-05T17:22:52.839 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.126.113_amd64.deb ... 2020-12-05T17:22:52.960 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-image-generic (4.15.0.126.113) over (4.15.0.124.111) ... 2020-12-05T17:22:53.279 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package linux-headers-4.15.0-126. 2020-12-05T17:22:53.293 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../5-linux-headers-4.15.0-126_4.15.0-126.129_all.deb ... 2020-12-05T17:22:53.329 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-headers-4.15.0-126 (4.15.0-126.129) ... 2020-12-05T17:22:58.053 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package linux-headers-4.15.0-126-generic. 2020-12-05T17:22:58.071 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../6-linux-headers-4.15.0-126-generic_4.15.0-126.129_amd64.deb ... 2020-12-05T17:22:58.111 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-headers-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:23:01.442 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.126.113_amd64.deb ... 2020-12-05T17:23:01.579 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-headers-generic (4.15.0.126.113) over (4.15.0.124.111) ... 2020-12-05T17:23:01.907 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-headers-4.15.0-126 (4.15.0-126.129) ... 2020-12-05T17:23:02.076 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-modules-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:23:02.636 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-headers-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:23:02.771 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-image-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:23:03.231 INFO:teuthology.orchestra.run.smithi053.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-126-generic 2020-12-05T17:23:03.231 INFO:teuthology.orchestra.run.smithi053.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-126-generic 2020-12-05T17:23:03.373 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-headers-generic (4.15.0.126.113) ... 2020-12-05T17:23:03.499 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-modules-extra-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:23:03.983 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-image-generic (4.15.0.126.113) ... 2020-12-05T17:23:04.135 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-generic (4.15.0.126.113) ... 2020-12-05T17:23:04.261 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for linux-image-4.15.0-126-generic (4.15.0-126.129) ... 2020-12-05T17:23:04.455 INFO:teuthology.orchestra.run.smithi053.stdout:/etc/kernel/postinst.d/initramfs-tools: 2020-12-05T17:23:04.455 INFO:teuthology.orchestra.run.smithi053.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-126-generic 2020-12-05T17:23:19.898 INFO:teuthology.orchestra.run.smithi053.stdout:/etc/kernel/postinst.d/zz-update-grub: 2020-12-05T17:23:19.899 INFO:teuthology.orchestra.run.smithi053.stdout:Sourcing file `/etc/default/grub' 2020-12-05T17:23:19.920 INFO:teuthology.orchestra.run.smithi053.stdout:Generating grub configuration file ... 2020-12-05T17:23:20.219 INFO:teuthology.orchestra.run.smithi053.stdout:Found linux image: /boot/vmlinuz-4.15.0-126-generic 2020-12-05T17:23:20.282 INFO:teuthology.orchestra.run.smithi053.stdout:Found initrd image: /boot/initrd.img-4.15.0-126-generic 2020-12-05T17:23:20.585 INFO:teuthology.orchestra.run.smithi053.stdout:Found linux image: /boot/vmlinuz-4.15.0-124-generic 2020-12-05T17:23:20.615 INFO:teuthology.orchestra.run.smithi053.stdout:Found initrd image: /boot/initrd.img-4.15.0-124-generic 2020-12-05T17:23:21.963 INFO:teuthology.orchestra.run.smithi053.stdout:done 2020-12-05T17:23:22.384 INFO:teuthology.orchestra.run.smithi053:> dpkg -s linux-image-generic 2020-12-05T17:23:22.408 INFO:teuthology.orchestra.run.smithi053.stdout:Package: linux-image-generic 2020-12-05T17:23:22.408 INFO:teuthology.orchestra.run.smithi053.stdout:Status: install ok installed 2020-12-05T17:23:22.408 INFO:teuthology.orchestra.run.smithi053.stdout:Priority: optional 2020-12-05T17:23:22.409 INFO:teuthology.orchestra.run.smithi053.stdout:Section: kernel 2020-12-05T17:23:22.409 INFO:teuthology.orchestra.run.smithi053.stdout:Installed-Size: 16 2020-12-05T17:23:22.409 INFO:teuthology.orchestra.run.smithi053.stdout:Maintainer: Ubuntu Kernel Team 2020-12-05T17:23:22.409 INFO:teuthology.orchestra.run.smithi053.stdout:Architecture: amd64 2020-12-05T17:23:22.409 INFO:teuthology.orchestra.run.smithi053.stdout:Source: linux-meta 2020-12-05T17:23:22.410 INFO:teuthology.orchestra.run.smithi053.stdout:Version: 4.15.0.126.113 2020-12-05T17:23:22.410 INFO:teuthology.orchestra.run.smithi053.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2020-12-05T17:23:22.410 INFO:teuthology.orchestra.run.smithi053.stdout:Depends: linux-image-4.15.0-126-generic, linux-modules-extra-4.15.0-126-generic, linux-firmware, intel-microcode, amd64-microcode 2020-12-05T17:23:22.410 INFO:teuthology.orchestra.run.smithi053.stdout:Recommends: thermald 2020-12-05T17:23:22.411 INFO:teuthology.orchestra.run.smithi053.stdout:Description: Generic Linux kernel image 2020-12-05T17:23:22.411 INFO:teuthology.orchestra.run.smithi053.stdout: This package will always depend on the latest generic kernel image 2020-12-05T17:23:22.411 INFO:teuthology.orchestra.run.smithi053.stdout: available. 2020-12-05T17:23:22.411 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-124-generic Expected: 4.15.0-126-generic 2020-12-05T17:23:22.412 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2020-12-05T17:23:22.412 DEBUG:teuthology.task.kernel:src is distro, skipping download 2020-12-05T17:23:22.412 DEBUG:teuthology.task.kernel:src is distro, skipping download 2020-12-05T17:23:22.412 DEBUG:teuthology.task.kernel:src is distro, skipping download 2020-12-05T17:23:22.413 DEBUG:teuthology.task.kernel:src is distro, skipping download 2020-12-05T17:23:22.413 DEBUG:teuthology.task.kernel:src is distro, skipping download 2020-12-05T17:23:22.413 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2020-12-05T17:23:22.413 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi032.front.sepia.ceph.com, path=None, version=distro) 2020-12-05T17:23:22.414 INFO:teuthology.orchestra.run.smithi032:> sudo apt-get clean 2020-12-05T17:23:22.535 INFO:teuthology.orchestra.run.smithi032:> sudo apt-get update 2020-12-05T17:23:22.682 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-12-05T17:23:22.789 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-12-05T17:23:22.872 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-12-05T17:23:22.960 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-12-05T17:23:24.601 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2020-12-05T17:23:24.652 INFO:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-12-05T17:23:24.730 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2020-12-05T17:23:24.974 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2020-12-05T17:23:24.974 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2020-12-05T17:23:25.059 INFO:teuthology.orchestra.run.smithi032.stderr:E: Unable to locate package linux-image-current-generic 2020-12-05T17:23:25.060 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-12-05T17:23:25.061 INFO:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-12-05T17:23:25.140 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2020-12-05T17:23:25.376 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2020-12-05T17:23:25.377 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2020-12-05T17:23:25.586 INFO:teuthology.orchestra.run.smithi032.stdout:linux-image-generic is already the newest version (4.15.0.126.113). 2020-12-05T17:23:25.586 INFO:teuthology.orchestra.run.smithi032.stdout:0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded. 2020-12-05T17:23:25.588 INFO:teuthology.orchestra.run.smithi032:> dpkg -s linux-image-generic 2020-12-05T17:23:25.610 INFO:teuthology.orchestra.run.smithi032.stdout:Package: linux-image-generic 2020-12-05T17:23:25.611 INFO:teuthology.orchestra.run.smithi032.stdout:Status: install ok installed 2020-12-05T17:23:25.611 INFO:teuthology.orchestra.run.smithi032.stdout:Priority: optional 2020-12-05T17:23:25.611 INFO:teuthology.orchestra.run.smithi032.stdout:Section: kernel 2020-12-05T17:23:25.611 INFO:teuthology.orchestra.run.smithi032.stdout:Installed-Size: 16 2020-12-05T17:23:25.612 INFO:teuthology.orchestra.run.smithi032.stdout:Maintainer: Ubuntu Kernel Team 2020-12-05T17:23:25.612 INFO:teuthology.orchestra.run.smithi032.stdout:Architecture: amd64 2020-12-05T17:23:25.612 INFO:teuthology.orchestra.run.smithi032.stdout:Source: linux-meta 2020-12-05T17:23:25.612 INFO:teuthology.orchestra.run.smithi032.stdout:Version: 4.15.0.126.113 2020-12-05T17:23:25.613 INFO:teuthology.orchestra.run.smithi032.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2020-12-05T17:23:25.613 INFO:teuthology.orchestra.run.smithi032.stdout:Depends: linux-image-4.15.0-126-generic, linux-modules-extra-4.15.0-126-generic, linux-firmware, intel-microcode, amd64-microcode 2020-12-05T17:23:25.613 INFO:teuthology.orchestra.run.smithi032.stdout:Recommends: thermald 2020-12-05T17:23:25.613 INFO:teuthology.orchestra.run.smithi032.stdout:Description: Generic Linux kernel image 2020-12-05T17:23:25.614 INFO:teuthology.orchestra.run.smithi032.stdout: This package will always depend on the latest generic kernel image 2020-12-05T17:23:25.614 INFO:teuthology.orchestra.run.smithi032.stdout: available. 2020-12-05T17:23:25.614 INFO:teuthology.orchestra.run.smithi032:> mktemp 2020-12-05T17:23:25.658 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.YGKjpqSen1 2020-12-05T17:23:25.659 INFO:teuthology.orchestra.run.smithi032:> sudo cp /boot/grub/grub.cfg /tmp/tmp.YGKjpqSen1 2020-12-05T17:23:25.725 INFO:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.YGKjpqSen1 2020-12-05T17:23:25.853 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.YGKjpqSen1 is 8KB 2020-12-05T17:23:25.865 INFO:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.YGKjpqSen1 2020-12-05T17:23:25.871 INFO:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2020-12-05T17:23:25.927 INFO:teuthology.orchestra.run.smithi032:> set -ex 2020-12-05T17:23:25.929 INFO:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/grub.d/01_ceph_kernel 2020-12-05T17:23:25.929 INFO:teuthology.orchestra.run.smithi032:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-12-05T17:23:25.973 INFO:teuthology.orchestra.run.smithi032.stderr:+ sudo dd of=/etc/grub.d/01_ceph_kernel 2020-12-05T17:23:25.997 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2020-12-05T17:23:25.997 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2020-12-05T17:23:25.998 INFO:teuthology.orchestra.run.smithi032.stderr:93 bytes copied, 3.9092e-05 s, 2.4 MB/s 2020-12-05T17:23:25.998 INFO:teuthology.orchestra.run.smithi032.stderr:+ sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-12-05T17:23:26.007 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-126-generic 2020-12-05T17:23:26.008 INFO:teuthology.orchestra.run.smithi032:> sudo update-grub 2020-12-05T17:23:27.226 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub' 2020-12-05T17:23:27.257 INFO:teuthology.orchestra.run.smithi032.stderr:Generating grub configuration file ... 2020-12-05T17:23:27.494 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-4.15.0-126-generic 2020-12-05T17:23:27.507 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-4.15.0-126-generic 2020-12-05T17:23:27.814 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-4.15.0-124-generic 2020-12-05T17:23:27.827 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-4.15.0-124-generic 2020-12-05T17:23:29.009 INFO:teuthology.orchestra.run.smithi032.stderr:done 2020-12-05T17:23:29.011 INFO:teuthology.orchestra.run.smithi032:> sudo shutdown -r now 2020-12-05T17:23:29.013 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2020-12-05T17:23:29.014 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi061.front.sepia.ceph.com, path=None, version=distro) 2020-12-05T17:23:29.014 INFO:teuthology.orchestra.run.smithi061:> sudo apt-get clean 2020-12-05T17:23:29.097 INFO:teuthology.orchestra.run.smithi061:> sudo apt-get update 2020-12-05T17:23:29.350 INFO:teuthology.orchestra.run.smithi061.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-12-05T17:23:29.350 INFO:teuthology.orchestra.run.smithi061.stdout:Get:2 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2020-12-05T17:23:29.417 INFO:teuthology.orchestra.run.smithi061.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-12-05T17:23:29.497 INFO:teuthology.orchestra.run.smithi061.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-12-05T17:23:29.995 INFO:teuthology.orchestra.run.smithi061.stdout:Fetched 88.7 kB in 1s (121 kB/s) 2020-12-05T17:23:31.282 INFO:teuthology.orchestra.run.smithi061.stdout:Reading package lists... 2020-12-05T17:23:31.331 INFO:teuthology.orchestra.run.smithi061:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-12-05T17:23:31.409 INFO:teuthology.orchestra.run.smithi061.stdout:Reading package lists... 2020-12-05T17:23:31.664 INFO:teuthology.orchestra.run.smithi061.stdout:Building dependency tree... 2020-12-05T17:23:31.665 INFO:teuthology.orchestra.run.smithi061.stdout:Reading state information... 2020-12-05T17:23:31.750 INFO:teuthology.orchestra.run.smithi061.stderr:E: Unable to locate package linux-image-current-generic 2020-12-05T17:23:31.752 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-12-05T17:23:31.752 INFO:teuthology.orchestra.run.smithi061:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-12-05T17:23:31.828 INFO:teuthology.orchestra.run.smithi061.stdout:Reading package lists... 2020-12-05T17:23:32.095 INFO:teuthology.orchestra.run.smithi061.stdout:Building dependency tree... 2020-12-05T17:23:32.096 INFO:teuthology.orchestra.run.smithi061.stdout:Reading state information... 2020-12-05T17:23:32.311 INFO:teuthology.orchestra.run.smithi061.stdout:linux-image-generic is already the newest version (4.15.0.126.113). 2020-12-05T17:23:32.311 INFO:teuthology.orchestra.run.smithi061.stdout:0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded. 2020-12-05T17:23:32.313 INFO:teuthology.orchestra.run.smithi061:> dpkg -s linux-image-generic 2020-12-05T17:23:32.336 INFO:teuthology.orchestra.run.smithi061.stdout:Package: linux-image-generic 2020-12-05T17:23:32.336 INFO:teuthology.orchestra.run.smithi061.stdout:Status: install ok installed 2020-12-05T17:23:32.336 INFO:teuthology.orchestra.run.smithi061.stdout:Priority: optional 2020-12-05T17:23:32.337 INFO:teuthology.orchestra.run.smithi061.stdout:Section: kernel 2020-12-05T17:23:32.337 INFO:teuthology.orchestra.run.smithi061.stdout:Installed-Size: 16 2020-12-05T17:23:32.337 INFO:teuthology.orchestra.run.smithi061.stdout:Maintainer: Ubuntu Kernel Team 2020-12-05T17:23:32.337 INFO:teuthology.orchestra.run.smithi061.stdout:Architecture: amd64 2020-12-05T17:23:32.337 INFO:teuthology.orchestra.run.smithi061.stdout:Source: linux-meta 2020-12-05T17:23:32.338 INFO:teuthology.orchestra.run.smithi061.stdout:Version: 4.15.0.126.113 2020-12-05T17:23:32.338 INFO:teuthology.orchestra.run.smithi061.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2020-12-05T17:23:32.338 INFO:teuthology.orchestra.run.smithi061.stdout:Depends: linux-image-4.15.0-126-generic, linux-modules-extra-4.15.0-126-generic, linux-firmware, intel-microcode, amd64-microcode 2020-12-05T17:23:32.338 INFO:teuthology.orchestra.run.smithi061.stdout:Recommends: thermald 2020-12-05T17:23:32.338 INFO:teuthology.orchestra.run.smithi061.stdout:Description: Generic Linux kernel image 2020-12-05T17:23:32.339 INFO:teuthology.orchestra.run.smithi061.stdout: This package will always depend on the latest generic kernel image 2020-12-05T17:23:32.339 INFO:teuthology.orchestra.run.smithi061.stdout: available. 2020-12-05T17:23:32.339 INFO:teuthology.orchestra.run.smithi061:> mktemp 2020-12-05T17:23:32.385 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/tmp.SbP36wDkAh 2020-12-05T17:23:32.385 INFO:teuthology.orchestra.run.smithi061:> sudo cp /boot/grub/grub.cfg /tmp/tmp.SbP36wDkAh 2020-12-05T17:23:32.442 INFO:teuthology.orchestra.run.smithi061:> sudo chmod 0666 /tmp/tmp.SbP36wDkAh 2020-12-05T17:23:32.558 DEBUG:teuthology.orchestra.remote:smithi061:/tmp/tmp.SbP36wDkAh is 8KB 2020-12-05T17:23:32.569 INFO:teuthology.orchestra.run.smithi061:> rm -fr /tmp/tmp.SbP36wDkAh 2020-12-05T17:23:32.575 INFO:teuthology.orchestra.run.smithi061:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2020-12-05T17:23:32.630 INFO:teuthology.orchestra.run.smithi061:> set -ex 2020-12-05T17:23:32.631 INFO:teuthology.orchestra.run.smithi061:> sudo dd of=/etc/grub.d/01_ceph_kernel 2020-12-05T17:23:32.631 INFO:teuthology.orchestra.run.smithi061:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-12-05T17:23:32.676 INFO:teuthology.orchestra.run.smithi061.stderr:+ sudo dd of=/etc/grub.d/01_ceph_kernel 2020-12-05T17:23:32.690 INFO:teuthology.orchestra.run.smithi061.stderr:0+1 records in 2020-12-05T17:23:32.690 INFO:teuthology.orchestra.run.smithi061.stderr:0+1 records out 2020-12-05T17:23:32.690 INFO:teuthology.orchestra.run.smithi061.stderr:93 bytes copied, 3.65e-05 s, 2.5 MB/s 2020-12-05T17:23:32.691 INFO:teuthology.orchestra.run.smithi061.stderr:+ sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-12-05T17:23:32.700 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-126-generic 2020-12-05T17:23:32.701 INFO:teuthology.orchestra.run.smithi061:> sudo update-grub 2020-12-05T17:23:34.070 INFO:teuthology.orchestra.run.smithi061.stderr:Sourcing file `/etc/default/grub' 2020-12-05T17:23:34.099 INFO:teuthology.orchestra.run.smithi061.stderr:Generating grub configuration file ... 2020-12-05T17:23:34.325 INFO:teuthology.orchestra.run.smithi061.stderr:Found linux image: /boot/vmlinuz-4.15.0-126-generic 2020-12-05T17:23:34.336 INFO:teuthology.orchestra.run.smithi061.stderr:Found initrd image: /boot/initrd.img-4.15.0-126-generic 2020-12-05T17:23:34.635 INFO:teuthology.orchestra.run.smithi061.stderr:Found linux image: /boot/vmlinuz-4.15.0-124-generic 2020-12-05T17:23:34.649 INFO:teuthology.orchestra.run.smithi061.stderr:Found initrd image: /boot/initrd.img-4.15.0-124-generic 2020-12-05T17:23:35.866 INFO:teuthology.orchestra.run.smithi061.stderr:done 2020-12-05T17:23:35.867 INFO:teuthology.orchestra.run.smithi061:> sudo shutdown -r now 2020-12-05T17:23:35.913 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2020-12-05T17:23:35.913 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi036.front.sepia.ceph.com, path=None, version=distro) 2020-12-05T17:23:35.914 INFO:teuthology.orchestra.run.smithi036:> sudo apt-get clean 2020-12-05T17:23:36.031 INFO:teuthology.orchestra.run.smithi036:> sudo apt-get update 2020-12-05T17:23:36.291 INFO:teuthology.orchestra.run.smithi036.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-12-05T17:23:36.292 INFO:teuthology.orchestra.run.smithi036.stdout:Get:2 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2020-12-05T17:23:36.358 INFO:teuthology.orchestra.run.smithi036.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-12-05T17:23:36.436 INFO:teuthology.orchestra.run.smithi036.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-12-05T17:23:36.923 INFO:teuthology.orchestra.run.smithi036.stdout:Fetched 88.7 kB in 1s (124 kB/s) 2020-12-05T17:23:38.209 INFO:teuthology.orchestra.run.smithi036.stdout:Reading package lists... 2020-12-05T17:23:38.258 INFO:teuthology.orchestra.run.smithi036:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-12-05T17:23:38.338 INFO:teuthology.orchestra.run.smithi036.stdout:Reading package lists... 2020-12-05T17:23:38.590 INFO:teuthology.orchestra.run.smithi036.stdout:Building dependency tree... 2020-12-05T17:23:38.591 INFO:teuthology.orchestra.run.smithi036.stdout:Reading state information... 2020-12-05T17:23:38.682 INFO:teuthology.orchestra.run.smithi036.stderr:E: Unable to locate package linux-image-current-generic 2020-12-05T17:23:38.684 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-12-05T17:23:38.685 INFO:teuthology.orchestra.run.smithi036:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-12-05T17:23:38.762 INFO:teuthology.orchestra.run.smithi036.stdout:Reading package lists... 2020-12-05T17:23:39.022 INFO:teuthology.orchestra.run.smithi036.stdout:Building dependency tree... 2020-12-05T17:23:39.023 INFO:teuthology.orchestra.run.smithi036.stdout:Reading state information... 2020-12-05T17:23:39.228 INFO:teuthology.orchestra.run.smithi036.stdout:linux-image-generic is already the newest version (4.15.0.126.113). 2020-12-05T17:23:39.229 INFO:teuthology.orchestra.run.smithi036.stdout:0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded. 2020-12-05T17:23:39.230 INFO:teuthology.orchestra.run.smithi036:> dpkg -s linux-image-generic 2020-12-05T17:23:39.246 INFO:teuthology.orchestra.run.smithi036.stdout:Package: linux-image-generic 2020-12-05T17:23:39.246 INFO:teuthology.orchestra.run.smithi036.stdout:Status: install ok installed 2020-12-05T17:23:39.247 INFO:teuthology.orchestra.run.smithi036.stdout:Priority: optional 2020-12-05T17:23:39.247 INFO:teuthology.orchestra.run.smithi036.stdout:Section: kernel 2020-12-05T17:23:39.247 INFO:teuthology.orchestra.run.smithi036.stdout:Installed-Size: 16 2020-12-05T17:23:39.247 INFO:teuthology.orchestra.run.smithi036.stdout:Maintainer: Ubuntu Kernel Team 2020-12-05T17:23:39.247 INFO:teuthology.orchestra.run.smithi036.stdout:Architecture: amd64 2020-12-05T17:23:39.248 INFO:teuthology.orchestra.run.smithi036.stdout:Source: linux-meta 2020-12-05T17:23:39.248 INFO:teuthology.orchestra.run.smithi036.stdout:Version: 4.15.0.126.113 2020-12-05T17:23:39.248 INFO:teuthology.orchestra.run.smithi036.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2020-12-05T17:23:39.248 INFO:teuthology.orchestra.run.smithi036.stdout:Depends: linux-image-4.15.0-126-generic, linux-modules-extra-4.15.0-126-generic, linux-firmware, intel-microcode, amd64-microcode 2020-12-05T17:23:39.248 INFO:teuthology.orchestra.run.smithi036.stdout:Recommends: thermald 2020-12-05T17:23:39.249 INFO:teuthology.orchestra.run.smithi036.stdout:Description: Generic Linux kernel image 2020-12-05T17:23:39.249 INFO:teuthology.orchestra.run.smithi036.stdout: This package will always depend on the latest generic kernel image 2020-12-05T17:23:39.249 INFO:teuthology.orchestra.run.smithi036.stdout: available. 2020-12-05T17:23:39.250 INFO:teuthology.orchestra.run.smithi036:> mktemp 2020-12-05T17:23:39.293 INFO:teuthology.orchestra.run.smithi036.stdout:/tmp/tmp.GQrta8lC82 2020-12-05T17:23:39.293 INFO:teuthology.orchestra.run.smithi036:> sudo cp /boot/grub/grub.cfg /tmp/tmp.GQrta8lC82 2020-12-05T17:23:39.348 INFO:teuthology.orchestra.run.smithi036:> sudo chmod 0666 /tmp/tmp.GQrta8lC82 2020-12-05T17:23:39.465 DEBUG:teuthology.orchestra.remote:smithi036:/tmp/tmp.GQrta8lC82 is 8KB 2020-12-05T17:23:39.476 INFO:teuthology.orchestra.run.smithi036:> rm -fr /tmp/tmp.GQrta8lC82 2020-12-05T17:23:39.483 INFO:teuthology.orchestra.run.smithi036:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2020-12-05T17:23:39.537 INFO:teuthology.orchestra.run.smithi036:> set -ex 2020-12-05T17:23:39.538 INFO:teuthology.orchestra.run.smithi036:> sudo dd of=/etc/grub.d/01_ceph_kernel 2020-12-05T17:23:39.538 INFO:teuthology.orchestra.run.smithi036:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-12-05T17:23:39.583 INFO:teuthology.orchestra.run.smithi036.stderr:+ sudo dd of=/etc/grub.d/01_ceph_kernel 2020-12-05T17:23:39.602 INFO:teuthology.orchestra.run.smithi036.stderr:0+1 records in 2020-12-05T17:23:39.602 INFO:teuthology.orchestra.run.smithi036.stderr:0+1 records out 2020-12-05T17:23:39.603 INFO:teuthology.orchestra.run.smithi036.stderr:93 bytes copied, 3.2382e-05 s, 2.9 MB/s 2020-12-05T17:23:39.603 INFO:teuthology.orchestra.run.smithi036.stderr:+ sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-12-05T17:23:39.611 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-126-generic 2020-12-05T17:23:39.612 INFO:teuthology.orchestra.run.smithi036:> sudo update-grub 2020-12-05T17:23:40.644 INFO:teuthology.orchestra.run.smithi036.stderr:Sourcing file `/etc/default/grub' 2020-12-05T17:23:40.685 INFO:teuthology.orchestra.run.smithi036.stderr:Generating grub configuration file ... 2020-12-05T17:23:40.916 INFO:teuthology.orchestra.run.smithi036.stderr:Found linux image: /boot/vmlinuz-4.15.0-126-generic 2020-12-05T17:23:40.927 INFO:teuthology.orchestra.run.smithi036.stderr:Found initrd image: /boot/initrd.img-4.15.0-126-generic 2020-12-05T17:23:41.262 INFO:teuthology.orchestra.run.smithi036.stderr:Found linux image: /boot/vmlinuz-4.15.0-124-generic 2020-12-05T17:23:41.276 INFO:teuthology.orchestra.run.smithi036.stderr:Found initrd image: /boot/initrd.img-4.15.0-124-generic 2020-12-05T17:23:42.409 INFO:teuthology.orchestra.run.smithi036.stderr:done 2020-12-05T17:23:42.411 INFO:teuthology.orchestra.run.smithi036:> sudo shutdown -r now 2020-12-05T17:23:42.413 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2020-12-05T17:23:42.413 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi002.front.sepia.ceph.com, path=None, version=distro) 2020-12-05T17:23:42.414 INFO:teuthology.orchestra.run.smithi002:> sudo apt-get clean 2020-12-05T17:23:42.506 INFO:teuthology.orchestra.run.smithi002:> sudo apt-get update 2020-12-05T17:23:42.761 INFO:teuthology.orchestra.run.smithi002.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-12-05T17:23:42.761 INFO:teuthology.orchestra.run.smithi002.stdout:Hit:2 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-12-05T17:23:42.828 INFO:teuthology.orchestra.run.smithi002.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-12-05T17:23:42.907 INFO:teuthology.orchestra.run.smithi002.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-12-05T17:23:44.680 INFO:teuthology.orchestra.run.smithi002.stdout:Reading package lists... 2020-12-05T17:23:44.730 INFO:teuthology.orchestra.run.smithi002:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-12-05T17:23:44.809 INFO:teuthology.orchestra.run.smithi002.stdout:Reading package lists... 2020-12-05T17:23:45.047 INFO:teuthology.orchestra.run.smithi002.stdout:Building dependency tree... 2020-12-05T17:23:45.047 INFO:teuthology.orchestra.run.smithi002.stdout:Reading state information... 2020-12-05T17:23:45.132 INFO:teuthology.orchestra.run.smithi002.stderr:E: Unable to locate package linux-image-current-generic 2020-12-05T17:23:45.133 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-12-05T17:23:45.134 INFO:teuthology.orchestra.run.smithi002:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-12-05T17:23:45.215 INFO:teuthology.orchestra.run.smithi002.stdout:Reading package lists... 2020-12-05T17:23:45.477 INFO:teuthology.orchestra.run.smithi002.stdout:Building dependency tree... 2020-12-05T17:23:45.478 INFO:teuthology.orchestra.run.smithi002.stdout:Reading state information... 2020-12-05T17:23:45.689 INFO:teuthology.orchestra.run.smithi002.stdout:linux-image-generic is already the newest version (4.15.0.126.113). 2020-12-05T17:23:45.689 INFO:teuthology.orchestra.run.smithi002.stdout:0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded. 2020-12-05T17:23:45.690 INFO:teuthology.orchestra.run.smithi002:> dpkg -s linux-image-generic 2020-12-05T17:23:45.713 INFO:teuthology.orchestra.run.smithi002.stdout:Package: linux-image-generic 2020-12-05T17:23:45.713 INFO:teuthology.orchestra.run.smithi002.stdout:Status: install ok installed 2020-12-05T17:23:45.714 INFO:teuthology.orchestra.run.smithi002.stdout:Priority: optional 2020-12-05T17:23:45.714 INFO:teuthology.orchestra.run.smithi002.stdout:Section: kernel 2020-12-05T17:23:45.714 INFO:teuthology.orchestra.run.smithi002.stdout:Installed-Size: 16 2020-12-05T17:23:45.715 INFO:teuthology.orchestra.run.smithi002.stdout:Maintainer: Ubuntu Kernel Team 2020-12-05T17:23:45.715 INFO:teuthology.orchestra.run.smithi002.stdout:Architecture: amd64 2020-12-05T17:23:45.715 INFO:teuthology.orchestra.run.smithi002.stdout:Source: linux-meta 2020-12-05T17:23:45.715 INFO:teuthology.orchestra.run.smithi002.stdout:Version: 4.15.0.126.113 2020-12-05T17:23:45.716 INFO:teuthology.orchestra.run.smithi002.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2020-12-05T17:23:45.716 INFO:teuthology.orchestra.run.smithi002.stdout:Depends: linux-image-4.15.0-126-generic, linux-modules-extra-4.15.0-126-generic, linux-firmware, intel-microcode, amd64-microcode 2020-12-05T17:23:45.716 INFO:teuthology.orchestra.run.smithi002.stdout:Recommends: thermald 2020-12-05T17:23:45.717 INFO:teuthology.orchestra.run.smithi002.stdout:Description: Generic Linux kernel image 2020-12-05T17:23:45.717 INFO:teuthology.orchestra.run.smithi002.stdout: This package will always depend on the latest generic kernel image 2020-12-05T17:23:45.718 INFO:teuthology.orchestra.run.smithi002.stdout: available. 2020-12-05T17:23:45.718 INFO:teuthology.orchestra.run.smithi002:> mktemp 2020-12-05T17:23:45.760 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/tmp.Bgdb40q1Q7 2020-12-05T17:23:45.761 INFO:teuthology.orchestra.run.smithi002:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Bgdb40q1Q7 2020-12-05T17:23:45.816 INFO:teuthology.orchestra.run.smithi002:> sudo chmod 0666 /tmp/tmp.Bgdb40q1Q7 2020-12-05T17:23:45.931 DEBUG:teuthology.orchestra.remote:smithi002:/tmp/tmp.Bgdb40q1Q7 is 8KB 2020-12-05T17:23:45.941 INFO:teuthology.orchestra.run.smithi002:> rm -fr /tmp/tmp.Bgdb40q1Q7 2020-12-05T17:23:45.947 INFO:teuthology.orchestra.run.smithi002:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2020-12-05T17:23:46.004 INFO:teuthology.orchestra.run.smithi002:> set -ex 2020-12-05T17:23:46.005 INFO:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/grub.d/01_ceph_kernel 2020-12-05T17:23:46.005 INFO:teuthology.orchestra.run.smithi002:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-12-05T17:23:46.051 INFO:teuthology.orchestra.run.smithi002.stderr:+ sudo dd of=/etc/grub.d/01_ceph_kernel 2020-12-05T17:23:46.065 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records in 2020-12-05T17:23:46.065 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records out 2020-12-05T17:23:46.066 INFO:teuthology.orchestra.run.smithi002.stderr:93 bytes copied, 3.2696e-05 s, 2.8 MB/s 2020-12-05T17:23:46.066 INFO:teuthology.orchestra.run.smithi002.stderr:+ sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-12-05T17:23:46.076 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-126-generic 2020-12-05T17:23:46.076 INFO:teuthology.orchestra.run.smithi002:> sudo update-grub 2020-12-05T17:23:47.316 INFO:teuthology.orchestra.run.smithi002.stderr:Sourcing file `/etc/default/grub' 2020-12-05T17:23:47.326 INFO:teuthology.orchestra.run.smithi002.stderr:Generating grub configuration file ... 2020-12-05T17:23:47.540 INFO:teuthology.orchestra.run.smithi002.stderr:Found linux image: /boot/vmlinuz-4.15.0-126-generic 2020-12-05T17:23:47.552 INFO:teuthology.orchestra.run.smithi002.stderr:Found initrd image: /boot/initrd.img-4.15.0-126-generic 2020-12-05T17:23:47.842 INFO:teuthology.orchestra.run.smithi002.stderr:Found linux image: /boot/vmlinuz-4.15.0-124-generic 2020-12-05T17:23:47.855 INFO:teuthology.orchestra.run.smithi002.stderr:Found initrd image: /boot/initrd.img-4.15.0-124-generic 2020-12-05T17:23:49.002 INFO:teuthology.orchestra.run.smithi002.stderr:done 2020-12-05T17:23:49.004 INFO:teuthology.orchestra.run.smithi002:> sudo shutdown -r now 2020-12-05T17:23:49.006 INFO:teuthology.task.kernel:Installing distro kernel on client.2... 2020-12-05T17:23:49.007 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi053.front.sepia.ceph.com, path=None, version=distro) 2020-12-05T17:23:49.007 INFO:teuthology.orchestra.run.smithi053:> sudo apt-get clean 2020-12-05T17:23:49.068 INFO:teuthology.orchestra.run.smithi053:> sudo apt-get update 2020-12-05T17:23:49.197 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-12-05T17:23:49.306 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-12-05T17:23:49.384 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-12-05T17:23:49.463 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-12-05T17:23:51.146 INFO:teuthology.orchestra.run.smithi053.stdout:Reading package lists... 2020-12-05T17:23:51.192 INFO:teuthology.orchestra.run.smithi053:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-12-05T17:23:51.316 INFO:teuthology.orchestra.run.smithi053.stdout:Reading package lists... 2020-12-05T17:23:51.561 INFO:teuthology.orchestra.run.smithi053.stdout:Building dependency tree... 2020-12-05T17:23:51.561 INFO:teuthology.orchestra.run.smithi053.stdout:Reading state information... 2020-12-05T17:23:51.647 INFO:teuthology.orchestra.run.smithi053.stderr:E: Unable to locate package linux-image-current-generic 2020-12-05T17:23:51.649 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-12-05T17:23:51.649 INFO:teuthology.orchestra.run.smithi053:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-12-05T17:23:51.728 INFO:teuthology.orchestra.run.smithi053.stdout:Reading package lists... 2020-12-05T17:23:51.965 INFO:teuthology.orchestra.run.smithi053.stdout:Building dependency tree... 2020-12-05T17:23:51.966 INFO:teuthology.orchestra.run.smithi053.stdout:Reading state information... 2020-12-05T17:23:52.173 INFO:teuthology.orchestra.run.smithi053.stdout:linux-image-generic is already the newest version (4.15.0.126.113). 2020-12-05T17:23:52.173 INFO:teuthology.orchestra.run.smithi053.stdout:0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded. 2020-12-05T17:23:52.174 INFO:teuthology.orchestra.run.smithi053:> dpkg -s linux-image-generic 2020-12-05T17:23:52.198 INFO:teuthology.orchestra.run.smithi053.stdout:Package: linux-image-generic 2020-12-05T17:23:52.198 INFO:teuthology.orchestra.run.smithi053.stdout:Status: install ok installed 2020-12-05T17:23:52.199 INFO:teuthology.orchestra.run.smithi053.stdout:Priority: optional 2020-12-05T17:23:52.199 INFO:teuthology.orchestra.run.smithi053.stdout:Section: kernel 2020-12-05T17:23:52.199 INFO:teuthology.orchestra.run.smithi053.stdout:Installed-Size: 16 2020-12-05T17:23:52.199 INFO:teuthology.orchestra.run.smithi053.stdout:Maintainer: Ubuntu Kernel Team 2020-12-05T17:23:52.199 INFO:teuthology.orchestra.run.smithi053.stdout:Architecture: amd64 2020-12-05T17:23:52.200 INFO:teuthology.orchestra.run.smithi053.stdout:Source: linux-meta 2020-12-05T17:23:52.200 INFO:teuthology.orchestra.run.smithi053.stdout:Version: 4.15.0.126.113 2020-12-05T17:23:52.200 INFO:teuthology.orchestra.run.smithi053.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2020-12-05T17:23:52.200 INFO:teuthology.orchestra.run.smithi053.stdout:Depends: linux-image-4.15.0-126-generic, linux-modules-extra-4.15.0-126-generic, linux-firmware, intel-microcode, amd64-microcode 2020-12-05T17:23:52.201 INFO:teuthology.orchestra.run.smithi053.stdout:Recommends: thermald 2020-12-05T17:23:52.201 INFO:teuthology.orchestra.run.smithi053.stdout:Description: Generic Linux kernel image 2020-12-05T17:23:52.201 INFO:teuthology.orchestra.run.smithi053.stdout: This package will always depend on the latest generic kernel image 2020-12-05T17:23:52.201 INFO:teuthology.orchestra.run.smithi053.stdout: available. 2020-12-05T17:23:52.202 INFO:teuthology.orchestra.run.smithi053:> mktemp 2020-12-05T17:23:52.246 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.KDGQaxRwye 2020-12-05T17:23:52.246 INFO:teuthology.orchestra.run.smithi053:> sudo cp /boot/grub/grub.cfg /tmp/tmp.KDGQaxRwye 2020-12-05T17:23:52.304 INFO:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.KDGQaxRwye 2020-12-05T17:23:52.419 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.KDGQaxRwye is 8KB 2020-12-05T17:23:52.429 INFO:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.KDGQaxRwye 2020-12-05T17:23:52.435 INFO:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2020-12-05T17:23:52.492 INFO:teuthology.orchestra.run.smithi053:> set -ex 2020-12-05T17:23:52.492 INFO:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/grub.d/01_ceph_kernel 2020-12-05T17:23:52.493 INFO:teuthology.orchestra.run.smithi053:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-12-05T17:23:52.538 INFO:teuthology.orchestra.run.smithi053.stderr:+ sudo dd of=/etc/grub.d/01_ceph_kernel 2020-12-05T17:23:52.554 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records in 2020-12-05T17:23:52.555 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records out 2020-12-05T17:23:52.555 INFO:teuthology.orchestra.run.smithi053.stderr:93 bytes copied, 3.5968e-05 s, 2.6 MB/s 2020-12-05T17:23:52.556 INFO:teuthology.orchestra.run.smithi053.stderr:+ sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-12-05T17:23:52.565 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-126-generic 2020-12-05T17:23:52.566 INFO:teuthology.orchestra.run.smithi053:> sudo update-grub 2020-12-05T17:23:53.939 INFO:teuthology.orchestra.run.smithi053.stderr:Sourcing file `/etc/default/grub' 2020-12-05T17:23:53.949 INFO:teuthology.orchestra.run.smithi053.stderr:Generating grub configuration file ... 2020-12-05T17:23:54.176 INFO:teuthology.orchestra.run.smithi053.stderr:Found linux image: /boot/vmlinuz-4.15.0-126-generic 2020-12-05T17:23:54.188 INFO:teuthology.orchestra.run.smithi053.stderr:Found initrd image: /boot/initrd.img-4.15.0-126-generic 2020-12-05T17:23:54.476 INFO:teuthology.orchestra.run.smithi053.stderr:Found linux image: /boot/vmlinuz-4.15.0-124-generic 2020-12-05T17:23:54.490 INFO:teuthology.orchestra.run.smithi053.stderr:Found initrd image: /boot/initrd.img-4.15.0-124-generic 2020-12-05T17:23:55.608 INFO:teuthology.orchestra.run.smithi053.stderr:done 2020-12-05T17:23:55.610 INFO:teuthology.orchestra.run.smithi053:> sudo shutdown -r now 2020-12-05T17:24:25.612 INFO:teuthology.misc:Re-opening connections... 2020-12-05T17:24:25.613 INFO:teuthology.misc:trying to connect to ubuntu@smithi032.front.sepia.ceph.com 2020-12-05T17:24:25.613 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-12-05T17:24:25.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:25:25.617 DEBUG:teuthology.orchestra.remote:timed out 2020-12-05T17:25:25.618 INFO:teuthology.misc:trying to connect to ubuntu@smithi061.front.sepia.ceph.com 2020-12-05T17:25:25.619 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-12-05T17:25:25.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:25:25.763 INFO:teuthology.orchestra.run.smithi061:> true 2020-12-05T17:25:26.242 INFO:teuthology.misc:trying to connect to ubuntu@smithi002.front.sepia.ceph.com 2020-12-05T17:25:26.243 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-12-05T17:25:26.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:25:26.379 INFO:teuthology.orchestra.run.smithi002:> true 2020-12-05T17:25:26.827 DEBUG:teuthology.misc:waited 61.213786125183105 2020-12-05T17:25:27.828 INFO:teuthology.misc:trying to connect to ubuntu@smithi032.front.sepia.ceph.com 2020-12-05T17:25:27.828 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-12-05T17:25:27.829 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:25:27.958 INFO:teuthology.orchestra.run.smithi032:> true 2020-12-05T17:25:28.389 INFO:teuthology.misc:trying to connect to ubuntu@smithi053.front.sepia.ceph.com 2020-12-05T17:25:28.389 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-12-05T17:25:28.390 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:25:28.514 INFO:teuthology.orchestra.run.smithi053:> true 2020-12-05T17:25:28.999 DEBUG:teuthology.misc:waited 63.38587260246277 2020-12-05T17:25:29.999 INFO:teuthology.misc:trying to connect to ubuntu@smithi036.front.sepia.ceph.com 2020-12-05T17:25:30.000 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-12-05T17:25:30.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:25:30.122 INFO:teuthology.orchestra.run.smithi036:> true 2020-12-05T17:25:30.579 DEBUG:teuthology.misc:waited 64.96601152420044 2020-12-05T17:25:31.580 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2020-12-05T17:25:31.581 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.15.0-126-generic"... 2020-12-05T17:25:31.582 INFO:teuthology.orchestra.run.smithi032:> uname -r 2020-12-05T17:25:31.589 INFO:teuthology.orchestra.run.smithi032.stdout:4.15.0-126-generic 2020-12-05T17:25:31.590 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-126-generic vs 4.15.0-126-generic 2020-12-05T17:25:31.590 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2020-12-05T17:25:31.591 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2020-12-05T17:25:31.591 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2020-12-05T17:25:31.592 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.15.0-126-generic"... 2020-12-05T17:25:31.592 INFO:teuthology.orchestra.run.smithi061:> uname -r 2020-12-05T17:25:31.599 INFO:teuthology.orchestra.run.smithi061.stdout:4.15.0-126-generic 2020-12-05T17:25:31.600 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-126-generic vs 4.15.0-126-generic 2020-12-05T17:25:31.600 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2020-12-05T17:25:31.601 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2020-12-05T17:25:31.601 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2020-12-05T17:25:31.602 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "4.15.0-126-generic"... 2020-12-05T17:25:31.602 INFO:teuthology.orchestra.run.smithi036:> uname -r 2020-12-05T17:25:31.609 INFO:teuthology.orchestra.run.smithi036.stdout:4.15.0-126-generic 2020-12-05T17:25:31.609 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-126-generic vs 4.15.0-126-generic 2020-12-05T17:25:31.610 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2020-12-05T17:25:31.610 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2020-12-05T17:25:31.610 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2020-12-05T17:25:31.611 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "4.15.0-126-generic"... 2020-12-05T17:25:31.611 INFO:teuthology.orchestra.run.smithi002:> uname -r 2020-12-05T17:25:31.618 INFO:teuthology.orchestra.run.smithi002.stdout:4.15.0-126-generic 2020-12-05T17:25:31.618 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-126-generic vs 4.15.0-126-generic 2020-12-05T17:25:31.618 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2020-12-05T17:25:31.619 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2020-12-05T17:25:31.619 INFO:teuthology.task.kernel:Checking client client.2 for new kernel version... 2020-12-05T17:25:31.620 INFO:teuthology.task.kernel:Checking kernel version of client.2, want "4.15.0-126-generic"... 2020-12-05T17:25:31.620 INFO:teuthology.orchestra.run.smithi053:> uname -r 2020-12-05T17:25:31.627 INFO:teuthology.orchestra.run.smithi053.stdout:4.15.0-126-generic 2020-12-05T17:25:31.627 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-126-generic vs 4.15.0-126-generic 2020-12-05T17:25:31.627 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2020-12-05T17:25:31.628 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2020-12-05T17:25:32.629 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2020-12-05T17:25:32.630 INFO:teuthology.orchestra.run.smithi032:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-12-05T17:25:32.664 INFO:teuthology.orchestra.run.smithi032.stdout:ttyS1 2020-12-05T17:25:32.686 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2020-12-05T17:25:32.686 INFO:teuthology.orchestra.run.smithi061:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-12-05T17:25:32.727 INFO:teuthology.orchestra.run.smithi061.stdout:ttyS1 2020-12-05T17:25:32.754 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2020-12-05T17:25:32.754 INFO:teuthology.orchestra.run.smithi036:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-12-05T17:25:32.781 INFO:teuthology.orchestra.run.smithi036.stdout:ttyS1 2020-12-05T17:25:32.801 INFO:teuthology.task.kernel:Enabling kdb on client.1... 2020-12-05T17:25:32.801 INFO:teuthology.orchestra.run.smithi002:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-12-05T17:25:32.833 INFO:teuthology.orchestra.run.smithi002.stdout:ttyS1 2020-12-05T17:25:32.857 INFO:teuthology.task.kernel:Enabling kdb on client.2... 2020-12-05T17:25:32.857 INFO:teuthology.orchestra.run.smithi053:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-12-05T17:25:32.891 INFO:teuthology.orchestra.run.smithi053.stdout:ttyS1 2020-12-05T17:25:32.911 INFO:teuthology.run_tasks:Running task internal.base... 2020-12-05T17:25:32.920 INFO:teuthology.task.internal:Creating test directory... 2020-12-05T17:25:32.920 INFO:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-12-05T17:25:32.923 INFO:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-12-05T17:25:32.925 INFO:teuthology.orchestra.run.smithi036:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-12-05T17:25:32.928 INFO:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-12-05T17:25:32.953 INFO:teuthology.orchestra.run.smithi061:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-12-05T17:25:32.961 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2020-12-05T17:25:32.970 INFO:teuthology.run_tasks:Running task internal.archive... 2020-12-05T17:25:32.984 INFO:teuthology.task.internal:Creating archive directory... 2020-12-05T17:25:32.985 INFO:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-12-05T17:25:32.987 INFO:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-12-05T17:25:32.990 INFO:teuthology.orchestra.run.smithi036:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-12-05T17:25:32.994 INFO:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-12-05T17:25:33.001 INFO:teuthology.orchestra.run.smithi061:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-12-05T17:25:33.018 INFO:teuthology.run_tasks:Running task internal.coredump... 2020-12-05T17:25:33.027 INFO:teuthology.task.internal:Enabling coredump saving... 2020-12-05T17:25:33.027 INFO:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-12-05T17:25:33.044 INFO:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-12-05T17:25:33.061 INFO:teuthology.orchestra.run.smithi036:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-12-05T17:25:33.064 INFO:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-12-05T17:25:33.066 INFO:teuthology.orchestra.run.smithi061:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-12-05T17:25:33.069 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-12-05T17:25:33.076 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-12-05T17:25:33.080 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-12-05T17:25:33.087 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-12-05T17:25:33.087 INFO:teuthology.orchestra.run.smithi061.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-12-05T17:25:33.090 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-12-05T17:25:33.097 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-12-05T17:25:33.097 INFO:teuthology.orchestra.run.smithi061.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-12-05T17:25:33.098 INFO:teuthology.orchestra.run.smithi036.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-12-05T17:25:33.105 INFO:teuthology.orchestra.run.smithi036.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-12-05T17:25:33.106 INFO:teuthology.run_tasks:Running task internal.sudo... 2020-12-05T17:25:33.117 INFO:teuthology.task.internal:Configuring sudo... 2020-12-05T17:25:33.117 INFO:teuthology.orchestra.run.smithi002:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-12-05T17:25:33.128 INFO:teuthology.orchestra.run.smithi032:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-12-05T17:25:33.138 INFO:teuthology.orchestra.run.smithi036:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-12-05T17:25:33.152 INFO:teuthology.orchestra.run.smithi053:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-12-05T17:25:33.155 INFO:teuthology.orchestra.run.smithi061:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-12-05T17:25:33.172 INFO:teuthology.run_tasks:Running task internal.syslog... 2020-12-05T17:25:33.235 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2020-12-05T17:25:33.236 INFO:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-12-05T17:25:33.239 INFO:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-12-05T17:25:33.242 INFO:teuthology.orchestra.run.smithi036:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-12-05T17:25:33.246 INFO:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-12-05T17:25:33.248 INFO:teuthology.orchestra.run.smithi061:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-12-05T17:25:33.258 INFO:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-12-05T17:25:33.295 INFO:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-12-05T17:25:33.342 INFO:teuthology.orchestra.run.smithi032:> set -ex 2020-12-05T17:25:33.343 INFO:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-12-05T17:25:33.388 INFO:teuthology.orchestra.run.smithi032.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-12-05T17:25:33.405 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2020-12-05T17:25:33.406 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2020-12-05T17:25:33.406 INFO:teuthology.orchestra.run.smithi032.stderr:152 bytes copied, 3.2551e-05 s, 4.7 MB/s 2020-12-05T17:25:33.408 INFO:teuthology.orchestra.run.smithi061:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-12-05T17:25:33.415 INFO:teuthology.orchestra.run.smithi061:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-12-05T17:25:33.462 INFO:teuthology.orchestra.run.smithi061:> set -ex 2020-12-05T17:25:33.462 INFO:teuthology.orchestra.run.smithi061:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-12-05T17:25:33.508 INFO:teuthology.orchestra.run.smithi061.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-12-05T17:25:33.524 INFO:teuthology.orchestra.run.smithi061.stderr:0+1 records in 2020-12-05T17:25:33.524 INFO:teuthology.orchestra.run.smithi061.stderr:0+1 records out 2020-12-05T17:25:33.525 INFO:teuthology.orchestra.run.smithi061.stderr:152 bytes copied, 3.4749e-05 s, 4.4 MB/s 2020-12-05T17:25:33.526 INFO:teuthology.orchestra.run.smithi036:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-12-05T17:25:33.533 INFO:teuthology.orchestra.run.smithi036:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-12-05T17:25:33.581 INFO:teuthology.orchestra.run.smithi036:> set -ex 2020-12-05T17:25:33.581 INFO:teuthology.orchestra.run.smithi036:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-12-05T17:25:33.627 INFO:teuthology.orchestra.run.smithi036.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-12-05T17:25:33.645 INFO:teuthology.orchestra.run.smithi036.stderr:0+1 records in 2020-12-05T17:25:33.645 INFO:teuthology.orchestra.run.smithi036.stderr:0+1 records out 2020-12-05T17:25:33.645 INFO:teuthology.orchestra.run.smithi036.stderr:152 bytes copied, 3.318e-05 s, 4.6 MB/s 2020-12-05T17:25:33.647 INFO:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-12-05T17:25:33.654 INFO:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-12-05T17:25:33.704 INFO:teuthology.orchestra.run.smithi002:> set -ex 2020-12-05T17:25:33.705 INFO:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-12-05T17:25:33.750 INFO:teuthology.orchestra.run.smithi002.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-12-05T17:25:33.765 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records in 2020-12-05T17:25:33.765 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records out 2020-12-05T17:25:33.765 INFO:teuthology.orchestra.run.smithi002.stderr:152 bytes copied, 3.3519e-05 s, 4.5 MB/s 2020-12-05T17:25:33.766 INFO:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-12-05T17:25:33.773 INFO:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-12-05T17:25:33.823 INFO:teuthology.orchestra.run.smithi053:> set -ex 2020-12-05T17:25:33.824 INFO:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-12-05T17:25:33.868 INFO:teuthology.orchestra.run.smithi053.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-12-05T17:25:33.891 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records in 2020-12-05T17:25:33.891 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records out 2020-12-05T17:25:33.892 INFO:teuthology.orchestra.run.smithi053.stderr:152 bytes copied, 3.2526e-05 s, 4.7 MB/s 2020-12-05T17:25:33.893 INFO:teuthology.orchestra.run.smithi002:> sudo service rsyslog restart 2020-12-05T17:25:33.896 INFO:teuthology.orchestra.run.smithi032:> sudo service rsyslog restart 2020-12-05T17:25:33.899 INFO:teuthology.orchestra.run.smithi036:> sudo service rsyslog restart 2020-12-05T17:25:33.902 INFO:teuthology.orchestra.run.smithi053:> sudo service rsyslog restart 2020-12-05T17:25:33.938 INFO:teuthology.orchestra.run.smithi061:> sudo service rsyslog restart 2020-12-05T17:25:33.988 INFO:teuthology.run_tasks:Running task internal.timer... 2020-12-05T17:25:33.998 INFO:teuthology.task.internal:Starting timer... 2020-12-05T17:25:33.999 INFO:teuthology.run_tasks:Running task pcp... 2020-12-05T17:25:34.032 INFO:teuthology.run_tasks:Running task selinux... 2020-12-05T17:25:34.047 DEBUG:teuthology.task.selinux:Excluding smithi032: OS 'ubuntu' does not support SELinux 2020-12-05T17:25:34.048 DEBUG:teuthology.task.selinux:Excluding smithi061: OS 'ubuntu' does not support SELinux 2020-12-05T17:25:34.049 DEBUG:teuthology.task.selinux:Excluding smithi036: OS 'ubuntu' does not support SELinux 2020-12-05T17:25:34.049 DEBUG:teuthology.task.selinux:Excluding smithi002: OS 'ubuntu' does not support SELinux 2020-12-05T17:25:34.051 DEBUG:teuthology.task.selinux:Excluding smithi053: OS 'ubuntu' does not support SELinux 2020-12-05T17:25:34.051 DEBUG:teuthology.task.selinux:Getting current SELinux state 2020-12-05T17:25:34.052 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2020-12-05T17:25:34.052 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2020-12-05T17:25:34.052 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2020-12-05T17:25:34.064 DEBUG:teuthology.repo_utils:Setting repo remote to git://git.ceph.com/git/ceph-cm-ansible.git 2020-12-05T17:25:34.071 INFO:teuthology.repo_utils:Fetching master from origin 2020-12-05T17:25:34.109 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2020-12-05T17:25:34.121 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2020-12-05T17:25:34.123 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi032.front.sepia.ceph.com,smithi061.front.sepia.ceph.com,smithi036.front.sepia.ceph.com,smithi002.front.sepia.ceph.com,smithi053.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2020-12-05T17:25:34.520 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2020-12-05T17:25:35.471 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2020-12-05T17:25:38.176 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2020-12-05T17:25:38.199 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:38 +0000 (0:00:00.080) 0:00:00.080 ***** 2020-12-05T17:25:38.213 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:38 +0000 (0:00:00.014) 0:00:00.095 ***** 2020-12-05T17:25:38.229 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:38 +0000 (0:00:00.015) 0:00:00.111 ***** 2020-12-05T17:25:38.243 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:38 +0000 (0:00:00.013) 0:00:00.124 ***** 2020-12-05T17:25:38.259 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:38 +0000 (0:00:00.016) 0:00:00.141 ***** 2020-12-05T17:25:39.803 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi061.front.sepia.ceph.com] 2020-12-05T17:25:39.863 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] 2020-12-05T17:25:39.954 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:39 +0000 (0:00:01.694) 0:00:01.835 ***** 2020-12-05T17:25:39.968 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] 2020-12-05T17:25:39.983 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] 2020-12-05T17:25:39.995 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] 2020-12-05T17:25:40.094 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:40 +0000 (0:00:00.135) 0:00:01.971 ***** 2020-12-05T17:25:40.157 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:40 +0000 (0:00:00.067) 0:00:02.039 ***** 2020-12-05T17:25:40.189 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:40 +0000 (0:00:00.031) 0:00:02.071 ***** 2020-12-05T17:25:40.222 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:40 +0000 (0:00:00.032) 0:00:02.104 ***** 2020-12-05T17:25:40.228 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2020-12-05T17:25:40.229 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-12-05T17:25:40.266 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:40 +0000 (0:00:00.044) 0:00:02.148 ***** 2020-12-05T17:25:40.304 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:40 +0000 (0:00:00.037) 0:00:02.185 ***** 2020-12-05T17:25:40.310 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-12-05T17:25:40.312 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-12-05T17:25:40.351 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:40 +0000 (0:00:00.047) 0:00:02.232 ***** 2020-12-05T17:25:40.390 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:40 +0000 (0:00:00.039) 0:00:02.272 ***** 2020-12-05T17:25:40.400 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-12-05T17:25:40.440 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:40 +0000 (0:00:00.048) 0:00:02.321 ***** 2020-12-05T17:25:40.701 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** changed: [smithi053.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-12-05T17:25:40.702 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-12-05T17:25:40.738 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:40 +0000 (0:00:00.298) 0:00:02.619 ***** 2020-12-05T17:25:40.806 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:40 +0000 (0:00:00.067) 0:00:02.687 ***** 2020-12-05T17:25:40.814 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-12-05T17:25:40.816 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-12-05T17:25:40.818 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-12-05T17:25:40.855 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:40 +0000 (0:00:00.049) 0:00:02.737 ***** 2020-12-05T17:25:40.891 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:40 +0000 (0:00:00.036) 0:00:02.773 ***** 2020-12-05T17:25:40.929 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:40 +0000 (0:00:00.036) 0:00:02.810 ***** 2020-12-05T17:25:41.041 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2020-12-05T17:25:41.043 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.056445", "end": "2020-12-05 17:25:41.258786", "rc": 0, "start": "2020-12-05 17:25:41.202341", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2020-12-05T17:25:41.077 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:41 +0000 (0:00:00.148) 0:00:02.959 ***** 2020-12-05T17:25:41.085 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.056617", "end": "2020-12-05 17:25:41.106734", "rc": 0, "start": "2020-12-05 17:25:41.050117", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2020-12-05T17:25:41.123 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:41 +0000 (0:00:00.045) 0:00:03.004 ***** 2020-12-05T17:25:41.129 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.076085", "end": "2020-12-05 17:25:41.439548", "rc": 0, "start": "2020-12-05 17:25:41.363463", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2020-12-05T17:25:41.130 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.071602", "end": "2020-12-05 17:25:41.300202", "rc": 0, "start": "2020-12-05 17:25:41.228600", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2020-12-05T17:25:41.168 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:41 +0000 (0:00:00.045) 0:00:03.050 ***** 2020-12-05T17:25:41.204 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:41 +0000 (0:00:00.035) 0:00:03.086 ***** 2020-12-05T17:25:41.211 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.200510", "end": "2020-12-05 17:25:41.168249", "rc": 0, "start": "2020-12-05 17:25:40.967739", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2020-12-05T17:25:41.253 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:41 +0000 (0:00:00.049) 0:00:03.135 ***** 2020-12-05T17:25:41.312 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2020-12-05T17:25:41.314 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-12-05T17:25:41.355 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:41 +0000 (0:00:00.101) 0:00:03.236 ***** 2020-12-05T17:25:41.362 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-12-05T17:25:41.363 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-12-05T17:25:41.364 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-12-05T17:25:41.403 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:41 +0000 (0:00:00.047) 0:00:03.284 ***** 2020-12-05T17:25:41.438 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:41 +0000 (0:00:00.035) 0:00:03.320 ***** 2020-12-05T17:25:41.476 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:41 +0000 (0:00:00.037) 0:00:03.358 ***** 2020-12-05T17:25:41.483 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-12-05T17:25:41.525 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:41 +0000 (0:00:00.048) 0:00:03.407 ***** 2020-12-05T17:25:41.968 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2020-12-05T17:25:41.970 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1607189141.4165413-70807106189087/source", "state": "file", "uid": 0} changed: [smithi061.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1607189141.4904888-91645865294739/source", "state": "file", "uid": 0} 2020-12-05T17:25:41.970 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1607189141.3675952-7240267398433/source", "state": "file", "uid": 0} 2020-12-05T17:25:42.007 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:42 +0000 (0:00:00.481) 0:00:03.889 ***** 2020-12-05T17:25:42.039 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:42 +0000 (0:00:00.031) 0:00:03.921 ***** 2020-12-05T17:25:42.072 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:42 +0000 (0:00:00.032) 0:00:03.953 ***** 2020-12-05T17:25:42.078 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1607189141.538578-193072489910758/source", "state": "file", "uid": 0} 2020-12-05T17:25:42.079 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1607189141.4520361-224657666117612/source", "state": "file", "uid": 0} 2020-12-05T17:25:42.116 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:42 +0000 (0:00:00.043) 0:00:03.997 ***** 2020-12-05T17:25:42.148 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:42 +0000 (0:00:00.032) 0:00:04.030 ***** 2020-12-05T17:25:42.301 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2020-12-05T17:25:42.302 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-12-05T17:25:42.304 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail 2020-12-05T17:25:42.305 INFO:teuthology.task.ansible.out:.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@ 2020-12-05T17:25:42.307 INFO:teuthology.task.ansible.out:aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-12-05T17:25:42.362 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-12-05T17:25:42.364 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail 2020-12-05T17:25:42.365 INFO:teuthology.task.ansible.out:.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@ 2020-12-05T17:25:42.367 INFO:teuthology.task.ansible.out:aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEz 2020-12-05T17:25:42.369 INFO:teuthology.task.ansible.out:IKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-12-05T17:25:42.599 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2020-12-05T17:25:42.607 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2020-12-05T17:25:42.774 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:42 +0000 (0:00:00.626) 0:00:04.656 ***** 2020-12-05T17:25:42.809 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:42 +0000 (0:00:00.034) 0:00:04.690 ***** 2020-12-05T17:25:42.843 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:42 +0000 (0:00:00.034) 0:00:04.725 ***** 2020-12-05T17:25:42.880 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:42 +0000 (0:00:00.036) 0:00:04.762 ***** 2020-12-05T17:25:42.918 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:42 +0000 (0:00:00.037) 0:00:04.799 ***** 2020-12-05T17:25:43.518 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2020-12-05T17:25:43.520 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] 2020-12-05T17:25:43.570 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] 2020-12-05T17:25:43.580 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] 2020-12-05T17:25:43.590 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] 2020-12-05T17:25:43.637 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2020-12-05T17:25:43.638 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:43 +0000 (0:00:00.719) 0:00:05.519 ***** 2020-12-05T17:25:43.735 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] 2020-12-05T17:25:43.782 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:43 +0000 (0:00:00.144) 0:00:05.663 ***** 2020-12-05T17:25:43.821 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:43 +0000 (0:00:00.039) 0:00:05.703 ***** 2020-12-05T17:25:43.860 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:43 +0000 (0:00:00.039) 0:00:05.742 ***** 2020-12-05T17:25:43.932 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:43 +0000 (0:00:00.071) 0:00:05.813 ***** 2020-12-05T17:25:43.991 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2020-12-05T17:25:43.997 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.166724", "end": "2020-12-05 17:25:43.967358", "rc": 0, "start": "2020-12-05 17:25:43.800634", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-12-05T17:25:44.032 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:44 +0000 (0:00:00.100) 0:00:05.914 ***** 2020-12-05T17:25:44.138 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.171171", "end": "2020-12-05 17:25:44.114989", "rc": 0, "start": "2020-12-05 17:25:43.943818", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-12-05T17:25:44.187 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:44 +0000 (0:00:00.154) 0:00:06.069 ***** 2020-12-05T17:25:44.193 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2020-12-05T17:25:44.195 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.011915", "end": "2020-12-05 17:25:44.153406", "failed_when_result": false, "rc": 0, "start": "2020-12-05 17:25:44.141491", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} TASK [users : Check keys_repo HEAD sha1] *************************************** changed: [smithi036.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.174441", "end": "2020-12-05 17:25:44.162386", "rc": 0, "start": "2020-12-05 17:25:43.987945", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-12-05T17:25:44.196 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.146030", "end": "2020-12-05 17:25:44.164749", "rc": 0, "start": "2020-12-05 17:25:44.018719", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-12-05T17:25:44.241 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:44 +0000 (0:00:00.053) 0:00:06.123 ***** 2020-12-05T17:25:44.277 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:44 +0000 (0:00:00.035) 0:00:06.158 ***** 2020-12-05T17:25:44.316 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:44 +0000 (0:00:00.039) 0:00:06.198 ***** 2020-12-05T17:25:44.323 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-12-05T17:25:44.325 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} TASK [users : Check keys_repo HEAD sha1] *************************************** 2020-12-05T17:25:44.325 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.159930", "end": "2020-12-05 17:25:44.260175", "rc": 0, "start": "2020-12-05 17:25:44.100245", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-12-05T17:25:44.326 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2020-12-05T17:25:44.327 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.012051", "end": "2020-12-05 17:25:44.537842", "failed_when_result": false, "rc": 0, "start": "2020-12-05 17:25:44.525791", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-12-05T17:25:44.364 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:44 +0000 (0:00:00.047) 0:00:06.246 ***** 2020-12-05T17:25:44.398 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:44 +0000 (0:00:00.033) 0:00:06.279 ***** 2020-12-05T17:25:44.431 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:44 +0000 (0:00:00.032) 0:00:06.312 ***** 2020-12-05T17:25:44.437 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-12-05T17:25:44.439 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** changed: [smithi036.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.010666", "end": "2020-12-05 17:25:44.748126", "failed_when_result": false, "rc": 0, "start": "2020-12-05 17:25:44.737460", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} TASK [users : Determine if we can skip users and pubkeys updates] ************** ok: [smithi032.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-12-05T17:25:44.492 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:44 +0000 (0:00:00.061) 0:00:06.374 ***** 2020-12-05T17:25:44.527 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:44 +0000 (0:00:00.034) 0:00:06.409 ***** 2020-12-05T17:25:44.562 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:44 +0000 (0:00:00.035) 0:00:06.444 ***** 2020-12-05T17:25:44.568 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2020-12-05T17:25:44.569 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.012144", "end": "2020-12-05 17:25:44.506915", "failed_when_result": false, "rc": 0, "start": "2020-12-05 17:25:44.494771", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} TASK [users : Remove managed_admin_users from managed_users] ******************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Merge extra_admin_users into managed_admin_users] **************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:44.570 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** changed: [smithi061.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.011470", "end": "2020-12-05 17:25:44.743847", "failed_when_result": false, "rc": 0, "start": "2020-12-05 17:25:44.732377", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-12-05T17:25:44.608 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:44 +0000 (0:00:00.045) 0:00:06.489 ***** 2020-12-05T17:25:44.642 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:44 +0000 (0:00:00.034) 0:00:06.524 ***** 2020-12-05T17:25:44.677 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:44 +0000 (0:00:00.034) 0:00:06.558 ***** 2020-12-05T17:25:44.711 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:44 +0000 (0:00:00.034) 0:00:06.592 ***** 2020-12-05T17:25:44.716 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-12-05T17:25:44.717 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-12-05T17:25:44.719 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove managed_admin_users from managed_users] ******************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-12-05T17:25:44.720 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-12-05T17:25:44.758 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:44 +0000 (0:00:00.047) 0:00:06.640 ***** 2020-12-05T17:25:44.799 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:44 +0000 (0:00:00.040) 0:00:06.680 ***** 2020-12-05T17:25:44.832 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:44 +0000 (0:00:00.033) 0:00:06.714 ***** 2020-12-05T17:25:44.866 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:44 +0000 (0:00:00.033) 0:00:06.747 ***** 2020-12-05T17:25:44.872 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-12-05T17:25:44.874 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Filter the managed_users list] *********************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Merge extra_admin_users into managed_admin_users] **************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:44.911 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:44 +0000 (0:00:00.045) 0:00:06.793 ***** 2020-12-05T17:25:44.957 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:44 +0000 (0:00:00.044) 0:00:06.838 ***** 2020-12-05T17:25:44.996 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:44 +0000 (0:00:00.039) 0:00:06.878 ***** 2020-12-05T17:25:45.032 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:45 +0000 (0:00:00.036) 0:00:06.914 ***** 2020-12-05T17:25:45.040 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.041 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.041 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.076 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:45 +0000 (0:00:00.043) 0:00:06.957 ***** 2020-12-05T17:25:45.109 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:45 +0000 (0:00:00.033) 0:00:06.990 ***** 2020-12-05T17:25:45.151 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:45 +0000 (0:00:00.042) 0:00:07.033 ***** 2020-12-05T17:25:45.192 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:45 +0000 (0:00:00.040) 0:00:07.074 ***** 2020-12-05T17:25:45.200 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-12-05T17:25:45.202 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.203 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.203 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.204 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [sm 2020-12-05T17:25:45.206 INFO:teuthology.task.ansible.out:ithi002.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f79891 2020-12-05T17:25:45.208 INFO:teuthology.task.ansible.out:3da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "it 2020-12-05T17:25:45.210 INFO:teuthology.task.ansible.out:em", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} TASK [users : Filter the managed_users list] *********************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c555 2020-12-05T17:25:45.212 INFO:teuthology.task.ansible.out:4e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30 2020-12-05T17:25:45.214 INFO:teuthology.task.ansible.out:g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia. 2020-12-05T17:25:45.216 INFO:teuthology.task.ansible.out:ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"n 2020-12-05T17:25:45.217 INFO:teuthology.task.ansible.out:ame": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} TASK [users : Remove managed_admin_users from managed_users] ******************* skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.259 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:45 +0000 (0:00:00.067) 0:00:07.141 ***** 2020-12-05T17:25:45.299 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:45 +0000 (0:00:00.039) 0:00:07.181 ***** 2020-12-05T17:25:45.340 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:45 +0000 (0:00:00.040) 0:00:07.222 ***** 2020-12-05T17:25:45.384 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:45 +0000 (0:00:00.043) 0:00:07.265 ***** 2020-12-05T17:25:45.390 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-12-05T17:25:45.392 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.394 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.395 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef 2020-12-05T17:25:45.397 INFO:teuthology.task.ansible.out:168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front. 2020-12-05T17:25:45.398 INFO:teuthology.task.ansible.out:sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw e 2020-12-05T17:25:45.400 INFO:teuthology.task.ansible.out:e37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.401 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item 2020-12-05T17:25:45.404 INFO:teuthology.task.ansible.out:": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uV 2020-12-05T17:25:45.405 INFO:teuthology.task.ansible.out:Ext16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.406 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.407 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e2 2020-12-05T17:25:45.408 INFO:teuthology.task.ansible.out:9b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab 2020-12-05T17:25:45.410 INFO:teuthology.task.ansible.out:96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7P 2020-12-05T17:25:45.412 INFO:teuthology.task.ansible.out:HdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae 2020-12-05T17:25:45.415 INFO:teuthology.task.ansible.out:6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/Z 2020-12-05T17:25:45.417 INFO:teuthology.task.ansible.out:Y33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb75 2020-12-05T17:25:45.419 INFO:teuthology.task.ansible.out:02a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} TASK [users : Filter the managed_users list] *********************************** skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pd 2020-12-05T17:25:45.421 INFO:teuthology.task.ansible.out:onnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273 2020-12-05T17:25:45.423 INFO:teuthology.task.ansible.out:c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi0 2020-12-05T17:25:45.425 INFO:teuthology.task.ansible.out:32.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef84 2020-12-05T17:25:45.427 INFO:teuthology.task.ansible.out:54742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d2 2020-12-05T17:25:45.429 INFO:teuthology.task.ansible.out:36fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.469 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:45 +0000 (0:00:00.085) 0:00:07.351 ***** 2020-12-05T17:25:45.506 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:45 +0000 (0:00:00.037) 0:00:07.388 ***** 2020-12-05T17:25:45.513 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2020-12-05T17:25:45.515 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.516 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.517 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@t 2020-12-05T17:25:45.519 INFO:teuthology.task.ansible.out:hinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuri 2020-12-05T17:25:45.521 INFO:teuthology.task.ansible.out:w", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ans 2020-12-05T17:25:45.522 INFO:teuthology.task.ansible.out:ible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.523 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.se 2020-12-05T17:25:45.525 INFO:teuthology.task.ansible.out:pia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1 2020-12-05T17:25:45.527 INFO:teuthology.task.ansible.out:WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df 2020-12-05T17:25:45.529 INFO:teuthology.task.ansible.out:86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "i 2020-12-05T17:25:45.531 INFO:teuthology.task.ansible.out:tem", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5 2020-12-05T17:25:45.533 INFO:teuthology.task.ansible.out:f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} TASK [users : Set UID_MIN to 1001] ********************************************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874 2020-12-05T17:25:45.535 INFO:teuthology.task.ansible.out:cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansi 2020-12-05T17:25:45.537 INFO:teuthology.task.ansible.out:ble_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.585 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:45 +0000 (0:00:00.078) 0:00:07.467 ***** 2020-12-05T17:25:45.623 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:45 +0000 (0:00:00.038) 0:00:07.505 ***** 2020-12-05T17:25:45.630 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.631 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.632 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.634 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_re 2020-12-05T17:25:45.636 INFO:teuthology.task.ansible.out:ason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.co 2020-12-05T17:25:45.638 INFO:teuthology.task.ansible.out:m] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.638 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, " 2020-12-05T17:25:45.640 INFO:teuthology.task.ansible.out:item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.641 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.643 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.643 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.644 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'nam 2020-12-05T17:25:45.646 INFO:teuthology.task.ansible.out:e': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c 2020-12-05T17:25:45.648 INFO:teuthology.task.ansible.out:3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Co 2020-12-05T17:25:45.650 INFO:teuthology.task.ansible.out:nditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": " 2020-12-05T17:25:45.652 INFO:teuthology.task.ansible.out:hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@z 2020-12-05T17:25:45.653 INFO:teuthology.task.ansible.out:te Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip 2020-12-05T17:25:45.655 INFO:teuthology.task.ansible.out:_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} TASK [users : Filter the managed_users list] *********************************** skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) 2020-12-05T17:25:45.658 INFO:teuthology.task.ansible.out: => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.714 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:45 +0000 (0:00:00.090) 0:00:07.595 ***** 2020-12-05T17:25:45.778 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:45 +0000 (0:00:00.063) 0:00:07.659 ***** 2020-12-05T17:25:45.789 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.792 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.793 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.795 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.799 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.803 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5 2020-12-05T17:25:45.807 INFO:teuthology.task.ansible.out:XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf 2020-12-05T17:25:45.813 INFO:teuthology.task.ansible.out:0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.815 INFO:teuthology.task.ansible.out: skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false 2020-12-05T17:25:45.817 INFO:teuthology.task.ansible.out:, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} TASK [users : Set UID_MIN to 1001] ********************************************* skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was Fals 2020-12-05T17:25:45.819 INFO:teuthology.task.ansible.out:e"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var 2020-12-05T17:25:45.820 INFO:teuthology.task.ansible.out:": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.864 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:45 +0000 (0:00:00.085) 0:00:07.745 ***** 2020-12-05T17:25:45.873 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.878 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.879 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2020-12-05T17:25:45.881 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.883 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.883 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.884 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": 2020-12-05T17:25:45.885 INFO:teuthology.task.ansible.out:{"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.886 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:45.887 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1y 2020-12-05T17:25:45.888 INFO:teuthology.task.ansible.out:c2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 6 2020-12-05T17:25:45.890 INFO:teuthology.task.ansible.out:2fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d 2020-12-05T17:25:45.892 INFO:teuthology.task.ansible.out:677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.036 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:46 +0000 (0:00:00.172) 0:00:07.917 ***** 2020-12-05T17:25:46.047 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.048 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.051 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.058 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.062 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.067 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.069 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.073 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.076 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.078 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.085 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.087 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.087 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.088 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.088 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe63747 2020-12-05T17:25:46.090 INFO:teuthology.task.ansible.out:91f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.097 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.100 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.102 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.106 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc 2020-12-05T17:25:46.108 INFO:teuthology.task.ansible.out:979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@my 2020-12-05T17:25:46.110 INFO:teuthology.task.ansible.out:host IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144 2020-12-05T17:25:46.113 INFO:teuthology.task.ansible.out:dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-deskt 2020-12-05T17:25:46.115 INFO:teuthology.task.ansible.out:op OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd5 2020-12-05T17:25:46.117 INFO:teuthology.task.ansible.out:99bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional r 2020-12-05T17:25:46.120 INFO:teuthology.task.ansible.out:esult was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaox 2020-12-05T17:25:46.122 INFO:teuthology.task.ansible.out:ichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e04619 2020-12-05T17:25:46.124 INFO:teuthology.task.ansible.out:9ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.135 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.137 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} TASK [users : Set UID_MIN to 1001] ********************************************* skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.138 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.140 INFO:teuthology.task.ansible.out: skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name 2020-12-05T17:25:46.143 INFO:teuthology.task.ansible.out:": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d01 2020-12-05T17:25:46.145 INFO:teuthology.task.ansible.out:9f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Condit 2020-12-05T17:25:46.148 INFO:teuthology.task.ansible.out:ional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.180 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:46 +0000 (0:00:00.144) 0:00:08.062 ***** 2020-12-05T17:25:46.224 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:46 +0000 (0:00:00.043) 0:00:08.105 ***** 2020-12-05T17:25:46.230 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.232 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.233 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc7 2020-12-05T17:25:46.235 INFO:teuthology.task.ansible.out:9b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@la 2020-12-05T17:25:46.236 INFO:teuthology.task.ansible.out:ptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.237 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.237 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.se 2020-12-05T17:25:46.239 INFO:teuthology.task.ansible.out:pia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.239 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCos 2020-12-05T17:25:46.241 INFO:teuthology.task.ansible.out:ntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b 2020-12-05T17:25:46.244 INFO:teuthology.task.ansible.out:6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansi 2020-12-05T17:25:46.249 INFO:teuthology.task.ansible.out:ble_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPL 2020-12-05T17:25:46.251 INFO:teuthology.task.ansible.out:U++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} TASK [users : Create all users without sudo access.] *************************** TASK [users : Create all admin users with sudo access.] 2020-12-05T17:25:46.253 INFO:teuthology.task.ansible.out:************************ skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84c 2020-12-05T17:25:46.254 INFO:teuthology.task.ansible.out:a78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed 2020-12-05T17:25:46.256 INFO:teuthology.task.ansible.out:": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.288 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:46 +0000 (0:00:00.064) 0:00:08.170 ***** 2020-12-05T17:25:46.296 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.297 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.298 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.300 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.302 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.303 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.305 INFO:teuthology.task.ansible.out: skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.306 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.307 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.307 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge6 2020-12-05T17:25:46.309 INFO:teuthology.task.ansible.out:09mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9 2020-12-05T17:25:46.313 INFO:teuthology.task.ansible.out:e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284 2020-12-05T17:25:46.315 INFO:teuthology.task.ansible.out:fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307 2020-12-05T17:25:46.317 INFO:teuthology.task.ansible.out:bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsal 2020-12-05T17:25:46.319 INFO:teuthology.task.ansible.out:omon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684 2020-12-05T17:25:46.321 INFO:teuthology.task.ansible.out:072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} skipping 2020-12-05T17:25:46.322 INFO:teuthology.task.ansible.out:: [smithi053.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": " 2020-12-05T17:25:46.324 INFO:teuthology.task.ansible.out:item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_ 2020-12-05T17:25:46.326 INFO:teuthology.task.ansible.out:var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.342 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.343 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.345 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.390 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:46 +0000 (0:00:00.102) 0:00:08.272 ***** 2020-12-05T17:25:46.429 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:46 +0000 (0:00:00.038) 0:00:08.311 ***** 2020-12-05T17:25:46.440 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.442 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.443 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.444 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@ 2020-12-05T17:25:46.446 INFO:teuthology.task.ansible.out:tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansi 2020-12-05T17:25:46.447 INFO:teuthology.task.ansible.out:ble_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss 2020-12-05T17:25:46.449 INFO:teuthology.task.ansible.out:8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, 2020-12-05T17:25:46.451 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-m 2020-12-05T17:25:46.452 INFO:teuthology.task.ansible.out:bp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a92116 2020-12-05T17:25:46.454 INFO:teuthology.task.ansible.out:9f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4 2020-12-05T17:25:46.456 INFO:teuthology.task.ansible.out:d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", 2020-12-05T17:25:46.457 INFO:teuthology.task.ansible.out: "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621 2020-12-05T17:25:46.459 INFO:teuthology.task.ansible.out:e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, " 2020-12-05T17:25:46.461 INFO:teuthology.task.ansible.out:skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc 2020-12-05T17:25:46.462 INFO:teuthology.task.ansible.out:29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (ite 2020-12-05T17:25:46.464 INFO:teuthology.task.ansible.out:m={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} TASK [users : Clone the keys repo] ********************************************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c 2020-12-05T17:25:46.465 INFO:teuthology.task.ansible.out:9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchan 2020-12-05T17:25:46.468 INFO:teuthology.task.ansible.out:gir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab 2020-12-05T17:25:46.469 INFO:teuthology.task.ansible.out:.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.523 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:46 +0000 (0:00:00.093) 0:00:08.404 ***** 2020-12-05T17:25:46.530 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.532 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88 2020-12-05T17:25:46.534 INFO:teuthology.task.ansible.out:f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.535 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092d 2020-12-05T17:25:46.536 INFO:teuthology.task.ansible.out:c522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} TASK [users : Create all users without sudo access.] *************************** TASK [users : Create all admin users with sudo access.] ************************ 2020-12-05T17:25:46.537 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was Fa 2020-12-05T17:25:46.539 INFO:teuthology.task.ansible.out:lse"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAa 2020-12-05T17:25:46.542 INFO:teuthology.task.ansible.out:v6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea5 2020-12-05T17:25:46.543 INFO:teuthology.task.ansible.out:8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": fals 2020-12-05T17:25:46.545 INFO:teuthology.task.ansible.out:e, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksiriva 2020-12-05T17:25:46.547 INFO:teuthology.task.ansible.out:d@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd 2020-12-05T17:25:46.548 INFO:teuthology.task.ansible.out:390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.550 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.587 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:46 +0000 (0:00:00.064) 0:00:08.469 ***** 2020-12-05T17:25:46.596 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.598 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.599 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@womba 2020-12-05T17:25:46.601 INFO:teuthology.task.ansible.out:t Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "sk 2020-12-05T17:25:46.603 INFO:teuthology.task.ansible.out:ip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "ite 2020-12-05T17:25:46.605 INFO:teuthology.task.ansible.out:m": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-12-05T17:25:46.606 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": " 2020-12-05T17:25:46.608 INFO:teuthology.task.ansible.out:item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'nam 2020-12-05T17:25:46.610 INFO:teuthology.task.ansible.out:e': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirw 2020-12-05T17:25:46.612 INFO:teuthology.task.ansible.out:xGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.613 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.614 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b 2020-12-05T17:25:46.615 INFO:teuthology.task.ansible.out:431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae627 2020-12-05T17:25:46.618 INFO:teuthology.task.ansible.out:59c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (it 2020-12-05T17:25:46.620 INFO:teuthology.task.ansible.out:em={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db 2020-12-05T17:25:46.621 INFO:teuthology.task.ansible.out:9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} TASK [users : Merge managed_users and managed_admin users] ********************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.664 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:46 +0000 (0:00:00.076) 0:00:08.545 ***** 2020-12-05T17:25:46.705 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:46 +0000 (0:00:00.041) 0:00:08.587 ***** 2020-12-05T17:25:46.711 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.713 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.714 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.714 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawr 2020-12-05T17:25:46.716 INFO:teuthology.task.ansible.out:acay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.716 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoI 2020-12-05T17:25:46.718 INFO:teuthology.task.ansible.out:wg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.718 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.719 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd0 2020-12-05T17:25:46.721 INFO:teuthology.task.ansible.out:9606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.721 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "it 2020-12-05T17:25:46.723 INFO:teuthology.task.ansible.out:em": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.723 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed 2020-12-05T17:25:46.725 INFO:teuthology.task.ansible.out:": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.726 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={' 2020-12-05T17:25:46.728 INFO:teuthology.task.ansible.out:name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8y 2020-12-05T17:25:46.729 INFO:teuthology.task.ansible.out:KG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76 2020-12-05T17:25:46.731 INFO:teuthology.task.ansible.out:c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37eb 2020-12-05T17:25:46.733 INFO:teuthology.task.ansible.out:c2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc03 2020-12-05T17:25:46.734 INFO:teuthology.task.ansible.out:6a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.735 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.735 INFO:teuthology.task.ansible.out:skipping: [smithi0 2020-12-05T17:25:46.737 INFO:teuthology.task.ansible.out:53.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false 2020-12-05T17:25:46.739 INFO:teuthology.task.ansible.out:, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@ma 2020-12-05T17:25:46.740 INFO:teuthology.task.ansible.out:cbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.774 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:46 +0000 (0:00:00.068) 0:00:08.656 ***** 2020-12-05T17:25:46.781 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2020-12-05T17:25:46.782 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.783 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.783 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.784 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.784 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (it 2020-12-05T17:25:46.785 INFO:teuthology.task.ansible.out:em={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.786 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0Fm 2020-12-05T17:25:46.788 INFO:teuthology.task.ansible.out:BQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.803 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.804 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.805 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.805 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.805 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph. 2020-12-05T17:25:46.807 INFO:teuthology.task.ansible.out:com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.807 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": 2020-12-05T17:25:46.809 INFO:teuthology.task.ansible.out: {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false 2020-12-05T17:25:46.811 INFO:teuthology.task.ansible.out:, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.847 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:46 +0000 (0:00:00.072) 0:00:08.728 ***** 2020-12-05T17:25:46.891 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:46 +0000 (0:00:00.044) 0:00:08.773 ***** 2020-12-05T17:25:46.900 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.901 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.903 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.903 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.904 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.905 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "j 2020-12-05T17:25:46.906 INFO:teuthology.task.ansible.out:schmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.908 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.910 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.911 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.913 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.915 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.916 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.918 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sh 2020-12-05T17:25:46.919 INFO:teuthology.task.ansible.out:yukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.920 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.921 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.922 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+E 2020-12-05T17:25:46.923 INFO:teuthology.task.ansible.out:EmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.925 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Condit 2020-12-05T17:25:46.927 INFO:teuthology.task.ansible.out:ional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.928 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d 2020-12-05T17:25:46.930 INFO:teuthology.task.ansible.out:9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.930 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.931 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com 2020-12-05T17:25:46.932 INFO:teuthology.task.ansible.out:] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:46.974 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:46 +0000 (0:00:00.083) 0:00:08.856 ***** 2020-12-05T17:25:47.009 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:47 +0000 (0:00:00.034) 0:00:08.891 ***** 2020-12-05T17:25:47.016 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.017 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.019 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} TASK [users : Create all users without sudo access.] *************************** 2020-12-05T17:25:47.020 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.021 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.021 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.023 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.023 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.024 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": 2020-12-05T17:25:47.025 INFO:teuthology.task.ansible.out:"item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.027 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.027 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.056 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.058 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.059 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item= 2020-12-05T17:25:47.060 INFO:teuthology.task.ansible.out:{'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** 2020-12-05T17:25:47.061 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was 2020-12-05T17:25:47.062 INFO:teuthology.task.ansible.out:False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi002.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** 2020-12-05T17:25:47.063 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh- 2020-12-05T17:25:47.065 INFO:teuthology.task.ansible.out:laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi002.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi002.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-12-05T17:25:47.065 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi002.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.066 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys 2020-12-05T17:25:47.067 INFO:teuthology.task.ansible.out:for each user with literal keys] ********** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi002.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.104 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:47 +0000 (0:00:00.095) 0:00:08.986 ***** 2020-12-05T17:25:47.137 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:47 +0000 (0:00:00.033) 0:00:09.019 ***** 2020-12-05T17:25:47.170 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:47 +0000 (0:00:00.032) 0:00:09.052 ***** 2020-12-05T17:25:47.176 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-12-05T17:25:47.177 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.195 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-12-05T17:25:47.197 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.197 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.197 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.198 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.198 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.199 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.199 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.199 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.200 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepi 2020-12-05T17:25:47.201 INFO:teuthology.task.ansible.out:a.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.202 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.202 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.202 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.203 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi002.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.204 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Condit 2020-12-05T17:25:47.205 INFO:teuthology.task.ansible.out:ional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} TASK [users : Merge managed_users and managed_admin users] ********************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi002.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.206 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.206 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.207 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.207 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.207 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.208 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=dorinda) => { 2020-12-05T17:25:47.209 INFO:teuthology.task.ansible.out:"ansible_loop_var": "item", "changed": false, "item": "dorinda", "skip_reason": "Conditional result was False"} TASK [users : Create all users without sudo access.] *************************** TASK [users : Remove revoked users] ******************************************** skipping: [smithi002.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.210 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.248 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:47 +0000 (0:00:00.077) 0:00:09.129 ***** 2020-12-05T17:25:47.285 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:47 +0000 (0:00:00.037) 0:00:09.167 ***** 2020-12-05T17:25:47.319 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:47 +0000 (0:00:00.033) 0:00:09.201 ***** 2020-12-05T17:25:47.352 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:47 +0000 (0:00:00.033) 0:00:09.234 ***** 2020-12-05T17:25:47.359 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.361 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.361 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.362 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.366 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.368 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.368 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.370 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi032.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.371 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.371 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.371 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.372 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi032.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.372 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.372 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.373 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.373 INFO:teuthology.task.ansible.out:skipping 2020-12-05T17:25:47.375 INFO:teuthology.task.ansible.out:: [smithi032.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.411 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:47 +0000 (0:00:00.058) 0:00:09.293 ***** 2020-12-05T17:25:47.446 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:47 +0000 (0:00:00.034) 0:00:09.327 ***** 2020-12-05T17:25:47.485 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:47 +0000 (0:00:00.039) 0:00:09.367 ***** 2020-12-05T17:25:47.491 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-12-05T17:25:47.491 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-12-05T17:25:47.492 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.492 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.493 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.493 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.493 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.495 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.495 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.496 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.496 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.497 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.497 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.497 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.498 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.498 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.499 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.c 2020-12-05T17:25:47.500 INFO:teuthology.task.ansible.out:om] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.501 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi032.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.501 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.501 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.502 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.538 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:47 +0000 (0:00:00.052) 0:00:09.420 ***** 2020-12-05T17:25:47.571 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:47 +0000 (0:00:00.033) 0:00:09.453 ***** 2020-12-05T17:25:47.577 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.579 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.581 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi032.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.581 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.586 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-12-05T17:25:47.588 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.621 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:47 +0000 (0:00:00.049) 0:00:09.503 ***** 2020-12-05T17:25:47.653 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:47 +0000 (0:00:00.031) 0:00:09.535 ***** 2020-12-05T17:25:47.686 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:47 +0000 (0:00:00.032) 0:00:09.568 ***** 2020-12-05T17:25:47.692 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2020-12-05T17:25:47.693 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.734 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:47 +0000 (0:00:00.048) 0:00:09.616 ***** 2020-12-05T17:25:47.773 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:47 +0000 (0:00:00.038) 0:00:09.655 ***** 2020-12-05T17:25:47.780 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-12-05T17:25:47.781 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi036.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.782 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.782 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.784 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.820 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:47 +0000 (0:00:00.046) 0:00:09.701 ***** 2020-12-05T17:25:47.826 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.828 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} TASK [users : Clone the keys repo] ********************************************* skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.828 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi036.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.829 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.829 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.830 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.891 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:47 +0000 (0:00:00.071) 0:00:09.773 ***** 2020-12-05T17:25:47.901 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.903 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.905 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "size": 40, "state": "file", "uid": 0} 2020-12-05T17:25:47.905 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi036.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.906 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.907 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi036.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.907 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.908 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.908 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.909 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] 2020-12-05T17:25:47.911 INFO:teuthology.task.ansible.out:=> (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.911 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.913 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.951 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:47 +0000 (0:00:00.059) 0:00:09.832 ***** 2020-12-05T17:25:47.986 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:47 +0000 (0:00:00.034) 0:00:09.867 ***** 2020-12-05T17:25:47.992 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-12-05T17:25:47.993 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi036.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-12-05T17:25:47.994 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "size": 40, "state": "file", "uid": 0} 2020-12-05T17:25:47.995 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi036.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.997 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.997 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.998 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.998 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.999 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.999 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:47.999 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** ok: [smithi002.front.sepia.ceph.com] => { "msg": "Host smithi002.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2020-12-05T17:25:48.000 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-12-05T17:25:48.000 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.000 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, 2020-12-05T17:25:48.002 INFO:teuthology.task.ansible.out:"item": "rturk", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.040 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.054) 0:00:09.922 ***** 2020-12-05T17:25:48.076 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.036) 0:00:09.958 ***** 2020-12-05T17:25:48.113 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.036) 0:00:09.994 ***** 2020-12-05T17:25:48.119 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.121 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.125 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2020-12-05T17:25:48.127 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Including major version specific variables.] ******************** ok: [smithi002.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2020-12-05T17:25:48.127 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** ok: [smithi032.front.sepia.ceph.com] => { "msg": "Host smithi032.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2020-12-05T17:25:48.166 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.053) 0:00:10.048 ***** 2020-12-05T17:25:48.202 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.035) 0:00:10.084 ***** 2020-12-05T17:25:48.238 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.035) 0:00:10.119 ***** 2020-12-05T17:25:48.274 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.035) 0:00:10.155 ***** 2020-12-05T17:25:48.289 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-12-05T17:25:48.291 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Including major version specific variables.] ******************** ok: [smithi032.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2020-12-05T17:25:48.331 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.058) 0:00:10.213 ***** 2020-12-05T17:25:48.369 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.038) 0:00:10.251 ***** 2020-12-05T17:25:48.407 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.037) 0:00:10.288 ***** 2020-12-05T17:25:48.414 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-12-05T17:25:48.416 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.416 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.416 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.417 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.417 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.419 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.419 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.421 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi053.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.421 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.421 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.422 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.423 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.424 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.424 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.425 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi053.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.425 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.426 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.427 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.427 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.428 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.467 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.060) 0:00:10.349 ***** 2020-12-05T17:25:48.505 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.037) 0:00:10.387 ***** 2020-12-05T17:25:48.511 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2020-12-05T17:25:48.513 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi053.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.513 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.514 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.516 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.517 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.518 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.518 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.518 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.519 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.519 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.520 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.520 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi053.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.521 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.521 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", 2020-12-05T17:25:48.523 INFO:teuthology.task.ansible.out:"changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.561 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.056) 0:00:10.443 ***** 2020-12-05T17:25:48.594 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.033) 0:00:10.476 ***** 2020-12-05T17:25:48.601 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.601 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.602 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.604 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi053.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.604 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.608 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-12-05T17:25:48.610 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "size": 40, "state": "file", "uid": 0} TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.649 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.054) 0:00:10.531 ***** 2020-12-05T17:25:48.683 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.033) 0:00:10.565 ***** 2020-12-05T17:25:48.715 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.031) 0:00:10.596 ***** 2020-12-05T17:25:48.748 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.033) 0:00:10.629 ***** 2020-12-05T17:25:48.753 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-12-05T17:25:48.755 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.756 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.757 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2020-12-05T17:25:48.758 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** 2020-12-05T17:25:48.758 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.759 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.759 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.760 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.760 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi061.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.762 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.762 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.763 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.763 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-12-05T17:25:48.764 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => { "msg": "Host smithi036.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } TASK [users : Remove revoked users] ******************************************** skipping: [smithi061.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.764 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.765 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.765 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.765 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false 2020-12-05T17:25:48.767 INFO:teuthology.task.ansible.out:, "item": "erwan", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.807 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.058) 0:00:10.688 ***** 2020-12-05T17:25:48.840 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.033) 0:00:10.722 ***** 2020-12-05T17:25:48.873 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.032) 0:00:10.755 ***** 2020-12-05T17:25:48.880 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.882 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.882 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.883 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.883 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.885 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.886 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.887 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi061.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.887 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.888 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.890 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.890 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.891 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi061.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.891 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sep 2020-12-05T17:25:48.893 INFO:teuthology.task.ansible.out:ia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:48.934 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.060) 0:00:10.815 ***** 2020-12-05T17:25:48.969 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.035) 0:00:10.851 ***** 2020-12-05T17:25:49.006 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:48 +0000 (0:00:00.036) 0:00:10.887 ***** 2020-12-05T17:25:49.011 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-12-05T17:25:49.013 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-12-05T17:25:49.013 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:49.014 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:49.053 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.047) 0:00:10.935 ***** 2020-12-05T17:25:49.089 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.035) 0:00:10.971 ***** 2020-12-05T17:25:49.125 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.035) 0:00:11.006 ***** 2020-12-05T17:25:49.132 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-12-05T17:25:49.134 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "size": 40, "state": "file", "uid": 0} TASK [common : Set entitlements_path] ****************************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:49.134 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:49.172 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.047) 0:00:11.054 ***** 2020-12-05T17:25:49.208 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.036) 0:00:11.090 ***** 2020-12-05T17:25:49.244 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.035) 0:00:11.125 ***** 2020-12-05T17:25:49.250 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-12-05T17:25:49.251 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [smithi002.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi002.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:49.289 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.045) 0:00:11.171 ***** 2020-12-05T17:25:49.325 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.035) 0:00:11.206 ***** 2020-12-05T17:25:49.363 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.038) 0:00:11.245 ***** 2020-12-05T17:25:49.369 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-12-05T17:25:49.371 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => { "msg": "Host smithi053.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } TASK [common : Set have_entitlements] ****************************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:49.371 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "size": 40, "state": "file", "uid": 0} 2020-12-05T17:25:49.373 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:49.412 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.049) 0:00:11.294 ***** 2020-12-05T17:25:49.449 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.036) 0:00:11.331 ***** 2020-12-05T17:25:49.485 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.035) 0:00:11.367 ***** 2020-12-05T17:25:49.521 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.035) 0:00:11.402 ***** 2020-12-05T17:25:49.527 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-12-05T17:25:49.528 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [smithi032.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:25:49.529 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:25:49.529 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** ok: [smithi053.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2020-12-05T17:25:49.565 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.044) 0:00:11.447 ***** 2020-12-05T17:25:49.598 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.031) 0:00:11.479 ***** 2020-12-05T17:25:49.631 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.032) 0:00:11.512 ***** 2020-12-05T17:25:49.668 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.037) 0:00:11.550 ***** 2020-12-05T17:25:49.674 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-12-05T17:25:49.676 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => { "msg": "Host smithi061.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi002.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:25:49.676 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:49.677 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:49.735 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.067) 0:00:11.617 ***** 2020-12-05T17:25:49.780 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.044) 0:00:11.662 ***** 2020-12-05T17:25:49.813 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.032) 0:00:11.694 ***** 2020-12-05T17:25:49.844 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.032) 0:00:11.726 ***** 2020-12-05T17:25:49.850 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-12-05T17:25:49.851 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:49.892 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.048) 0:00:11.774 ***** 2020-12-05T17:25:49.928 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.035) 0:00:11.810 ***** 2020-12-05T17:25:49.975 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:49 +0000 (0:00:00.046) 0:00:11.856 ***** 2020-12-05T17:25:50.020 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.044) 0:00:11.901 ***** 2020-12-05T17:25:50.026 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-12-05T17:25:50.028 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} TASK [common : Register with subscription-manager.] **************************** skipping: [smithi002.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi032.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2020-12-05T17:25:50.029 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:50.065 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.045) 0:00:11.947 ***** 2020-12-05T17:25:50.115 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.049) 0:00:11.997 ***** 2020-12-05T17:25:50.151 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.036) 0:00:12.033 ***** 2020-12-05T17:25:50.183 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.031) 0:00:12.065 ***** 2020-12-05T17:25:50.189 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-12-05T17:25:50.191 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:50.226 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.042) 0:00:12.107 ***** 2020-12-05T17:25:50.258 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.032) 0:00:12.140 ***** 2020-12-05T17:25:50.291 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.032) 0:00:12.172 ***** 2020-12-05T17:25:50.323 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.032) 0:00:12.205 ***** 2020-12-05T17:25:50.329 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-12-05T17:25:50.330 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : List CDN releases available to system] ************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Register with subscription-manager.] **************************** skipping: [smithi032.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:25:50.331 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:50.366 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.043) 0:00:12.248 ***** 2020-12-05T17:25:50.398 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.032) 0:00:12.280 ***** 2020-12-05T17:25:50.430 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.031) 0:00:12.312 ***** 2020-12-05T17:25:50.462 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.032) 0:00:12.344 ***** 2020-12-05T17:25:50.468 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-12-05T17:25:50.470 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set entitlements_path] ****************************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:50.506 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.043) 0:00:12.387 ***** 2020-12-05T17:25:50.538 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.032) 0:00:12.420 ***** 2020-12-05T17:25:50.578 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.039) 0:00:12.459 ***** 2020-12-05T17:25:50.611 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.033) 0:00:12.493 ***** 2020-12-05T17:25:50.617 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-12-05T17:25:50.619 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : List CDN releases available to system] ************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Include Red Hat encrypted variables.] *************************** 2020-12-05T17:25:50.619 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi036.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:25:50.658 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.046) 0:00:12.540 ***** 2020-12-05T17:25:50.694 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.035) 0:00:12.576 ***** 2020-12-05T17:25:50.733 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.038) 0:00:12.614 ***** 2020-12-05T17:25:50.767 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.034) 0:00:12.648 ***** 2020-12-05T17:25:50.772 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-12-05T17:25:50.774 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi002.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:50.774 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:50.810 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.042) 0:00:12.691 ***** 2020-12-05T17:25:50.843 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.033) 0:00:12.724 ***** 2020-12-05T17:25:50.875 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.031) 0:00:12.756 ***** 2020-12-05T17:25:50.907 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.032) 0:00:12.789 ***** 2020-12-05T17:25:50.918 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-12-05T17:25:50.920 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Clean yum cache] ************************************************ skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:50.952 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.043) 0:00:12.832 ***** 2020-12-05T17:25:50.984 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:50 +0000 (0:00:00.032) 0:00:12.865 ***** 2020-12-05T17:25:51.016 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:51 +0000 (0:00:00.032) 0:00:12.897 ***** 2020-12-05T17:25:51.048 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:51 +0000 (0:00:00.031) 0:00:12.929 ***** 2020-12-05T17:25:51.054 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2020-12-05T17:25:51.055 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:51.056 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi032.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:25:51.056 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi036.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:25:51.119 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:51 +0000 (0:00:00.070) 0:00:13.000 ***** 2020-12-05T17:25:51.283 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:51 +0000 (0:00:00.058) 0:00:13.058 ***** Saturday 05 December 2020 17:25:51 +0000 (0:00:00.047) 0:00:13.106 ***** Saturday 05 December 2020 17:25:51 +0000 (0:00:00.047) 0:00:13.154 ***** TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Store list of enabled repos] ************************************ skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Clean yum cache] ************************************************ skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:51.322 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:51 +0000 (0:00:00.050) 0:00:13.204 ***** 2020-12-05T17:25:51.357 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:51 +0000 (0:00:00.034) 0:00:13.239 ***** 2020-12-05T17:25:51.399 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:51 +0000 (0:00:00.042) 0:00:13.281 ***** 2020-12-05T17:25:51.444 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:51 +0000 (0:00:00.044) 0:00:13.326 ***** 2020-12-05T17:25:51.451 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-12-05T17:25:51.453 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:51.453 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:51.454 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** skipping: [smithi036.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:25:51.501 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:51 +0000 (0:00:00.056) 0:00:13.382 ***** 2020-12-05T17:25:51.549 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:51 +0000 (0:00:00.048) 0:00:13.431 ***** 2020-12-05T17:25:51.587 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:51 +0000 (0:00:00.037) 0:00:13.469 ***** 2020-12-05T17:25:51.621 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:51 +0000 (0:00:00.033) 0:00:13.502 ***** 2020-12-05T17:25:51.628 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2020-12-05T17:25:51.630 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Store list of enabled repos] ************************************ skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered if we just registered] ********************** 2020-12-05T17:25:51.630 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:51.673 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:51 +0000 (0:00:00.052) 0:00:13.555 ***** 2020-12-05T17:25:51.707 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:51 +0000 (0:00:00.033) 0:00:13.589 ***** 2020-12-05T17:25:51.740 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:51 +0000 (0:00:00.033) 0:00:13.622 ***** 2020-12-05T17:25:51.774 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:51 +0000 (0:00:00.033) 0:00:13.656 ***** 2020-12-05T17:25:51.780 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-12-05T17:25:51.782 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : List CDN releases available to system] ************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:51.833 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:51 +0000 (0:00:00.058) 0:00:13.714 ***** 2020-12-05T17:25:51.875 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:51 +0000 (0:00:00.042) 0:00:13.757 ***** 2020-12-05T17:25:51.925 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:51 +0000 (0:00:00.049) 0:00:13.806 ***** 2020-12-05T17:25:51.972 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:51 +0000 (0:00:00.046) 0:00:13.853 ***** 2020-12-05T17:25:51.980 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-12-05T17:25:51.981 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Disable all rhsm repos] ***************************************** skipping: [smithi002.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:25:51.982 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:51.982 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:52.034 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.061) 0:00:13.915 ***** 2020-12-05T17:25:52.068 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.034) 0:00:13.950 ***** 2020-12-05T17:25:52.104 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.035) 0:00:13.986 ***** 2020-12-05T17:25:52.138 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.033) 0:00:14.019 ***** 2020-12-05T17:25:52.144 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-12-05T17:25:52.146 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Enable necessary rhsm repos] ************************************ skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:52.182 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.044) 0:00:14.063 ***** 2020-12-05T17:25:52.216 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.033) 0:00:14.097 ***** 2020-12-05T17:25:52.248 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.032) 0:00:14.130 ***** 2020-12-05T17:25:52.282 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.033) 0:00:14.163 ***** 2020-12-05T17:25:52.287 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-12-05T17:25:52.289 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Remove old apt-mirror repository definition.] ******************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Disable all rhsm repos] ***************************************** skipping: [smithi032.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi036.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:25:52.338 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.057) 0:00:14.220 ***** 2020-12-05T17:25:52.377 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.038) 0:00:14.258 ***** 2020-12-05T17:25:52.410 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.032) 0:00:14.291 ***** 2020-12-05T17:25:52.444 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.033) 0:00:14.325 ***** 2020-12-05T17:25:52.449 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-12-05T17:25:52.451 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi053.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Increase the yum timeout.] ************************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Enable necessary rhsm repos] ************************************ skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:52.452 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:52.495 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.051) 0:00:14.377 ***** 2020-12-05T17:25:52.547 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.051) 0:00:14.428 ***** 2020-12-05T17:25:52.580 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.033) 0:00:14.462 ***** 2020-12-05T17:25:52.616 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.035) 0:00:14.497 ***** 2020-12-05T17:25:52.622 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-12-05T17:25:52.624 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** skipping: [smithi002.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:52.625 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:52.626 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get list of enabled RHSM repos] ********************************* 2020-12-05T17:25:52.627 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:52.663 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.047) 0:00:14.545 ***** 2020-12-05T17:25:52.704 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.036) 0:00:14.582 ***** 2020-12-05T17:25:52.747 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.047) 0:00:14.629 ***** 2020-12-05T17:25:52.809 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.061) 0:00:14.691 ***** 2020-12-05T17:25:52.819 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:52.821 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* skipping: [smithi002.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} TASK [common : Increase the yum timeout.] ************************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:52.822 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:52.875 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.065) 0:00:14.757 ***** 2020-12-05T17:25:52.923 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.048) 0:00:14.805 ***** 2020-12-05T17:25:52.967 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.043) 0:00:14.849 ***** 2020-12-05T17:25:53.004 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:52 +0000 (0:00:00.036) 0:00:14.886 ***** 2020-12-05T17:25:53.010 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-12-05T17:25:53.015 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi061.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:25:53.016 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** skipping: [smithi032.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:53.055 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.051) 0:00:14.937 ***** 2020-12-05T17:25:53.094 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.038) 0:00:14.975 ***** 2020-12-05T17:25:53.127 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.032) 0:00:15.008 ***** 2020-12-05T17:25:53.171 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.043) 0:00:15.052 ***** 2020-12-05T17:25:53.177 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2020-12-05T17:25:53.179 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure local epel mirrorlists] ******************************* skipping: [smithi032.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:53.180 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:53.229 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.058) 0:00:15.111 ***** 2020-12-05T17:25:53.262 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.033) 0:00:15.144 ***** 2020-12-05T17:25:53.295 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.032) 0:00:15.177 ***** 2020-12-05T17:25:53.301 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2020-12-05T17:25:53.302 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get the current timezone] *************************************** ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.010204", "end": "2020-12-05 17:25:53.182447", "rc": 0, "start": "2020-12-05 17:25:53.172243", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-12-05T17:25:53.303 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:53.303 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2020-12-05T17:25:53.304 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:53.340 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.044) 0:00:15.221 ***** 2020-12-05T17:25:53.373 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.033) 0:00:15.255 ***** 2020-12-05T17:25:53.406 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.032) 0:00:15.288 ***** 2020-12-05T17:25:53.441 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.034) 0:00:15.323 ***** 2020-12-05T17:25:53.447 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2020-12-05T17:25:53.448 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Disable all rhsm repos] ***************************************** skipping: [smithi036.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:25:53.486 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.044) 0:00:15.367 ***** 2020-12-05T17:25:53.519 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.032) 0:00:15.400 ***** 2020-12-05T17:25:53.552 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.033) 0:00:15.434 ***** 2020-12-05T17:25:53.558 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2020-12-05T17:25:53.559 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:53.561 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.022042", "end": "2020-12-05 17:25:53.727118", "rc": 0, "start": "2020-12-05 17:25:53.705076", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} TASK [common : Enable necessary rhsm repos] ************************************ skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:53.596 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.044) 0:00:15.478 ***** 2020-12-05T17:25:53.631 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.034) 0:00:15.512 ***** 2020-12-05T17:25:53.667 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.036) 0:00:15.549 ***** 2020-12-05T17:25:53.708 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.040) 0:00:15.590 ***** 2020-12-05T17:25:53.715 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:53.716 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-12-05T17:25:53.717 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:53.753 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.044) 0:00:15.635 ***** 2020-12-05T17:25:53.786 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.033) 0:00:15.668 ***** 2020-12-05T17:25:53.819 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.032) 0:00:15.700 ***** 2020-12-05T17:25:53.851 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.032) 0:00:15.732 ***** 2020-12-05T17:25:53.856 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2020-12-05T17:25:53.858 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Set the timezone] *********************************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:53.858 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:53.892 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.041) 0:00:15.774 ***** 2020-12-05T17:25:53.925 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.032) 0:00:15.807 ***** 2020-12-05T17:25:53.958 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.032) 0:00:15.840 ***** 2020-12-05T17:25:53.992 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:53 +0000 (0:00:00.034) 0:00:15.874 ***** 2020-12-05T17:25:53.999 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2020-12-05T17:25:54.001 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:25:54.002 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:54.002 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:54.004 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** skipping: [smithi036.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:54.004 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:54.061 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:54 +0000 (0:00:00.068) 0:00:15.943 ***** 2020-12-05T17:25:54.098 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:54 +0000 (0:00:00.036) 0:00:15.980 ***** 2020-12-05T17:25:54.135 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:54 +0000 (0:00:00.036) 0:00:16.016 ***** 2020-12-05T17:25:54.144 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-12-05T17:25:54.146 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the timezone] *********************************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:54.188 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:54 +0000 (0:00:00.053) 0:00:16.070 ***** 2020-12-05T17:25:54.239 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:54 +0000 (0:00:00.050) 0:00:16.121 ***** 2020-12-05T17:25:54.273 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:54 +0000 (0:00:00.033) 0:00:16.154 ***** 2020-12-05T17:25:54.309 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:54 +0000 (0:00:00.033) 0:00:16.188 ***** 2020-12-05T17:25:54.312 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2020-12-05T17:25:54.314 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Refresh repos] ************************************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:54.356 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:54 +0000 (0:00:00.049) 0:00:16.237 ***** 2020-12-05T17:25:54.390 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:54 +0000 (0:00:00.034) 0:00:16.272 ***** 2020-12-05T17:25:54.432 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:54 +0000 (0:00:00.042) 0:00:16.314 ***** 2020-12-05T17:25:54.440 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-12-05T17:25:54.440 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:54.443 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} TASK [common : Register with subscription-manager.] **************************** skipping: [smithi061.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:25:54.444 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:54.481 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:54 +0000 (0:00:00.049) 0:00:16.363 ***** 2020-12-05T17:25:54.517 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:54 +0000 (0:00:00.035) 0:00:16.398 ***** 2020-12-05T17:25:54.552 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:54 +0000 (0:00:00.035) 0:00:16.433 ***** 2020-12-05T17:25:54.558 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-12-05T17:25:54.560 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Refresh repos] ************************************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the hardware clock] ***************************************** changed: [smithi002.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.045380", "end": "2020-12-05 17:25:54.509279", "rc": 0, "start": "2020-12-05 17:25:54.463899", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:25:54.560 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:54.598 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:54 +0000 (0:00:00.046) 0:00:16.480 ***** 2020-12-05T17:25:54.630 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:54 +0000 (0:00:00.032) 0:00:16.512 ***** 2020-12-05T17:25:54.665 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:54 +0000 (0:00:00.034) 0:00:16.546 ***** 2020-12-05T17:25:54.701 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:54 +0000 (0:00:00.035) 0:00:16.582 ***** 2020-12-05T17:25:54.707 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2020-12-05T17:25:54.708 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install Kerberos Packages (RedHat)] ***************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:54.758 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:54 +0000 (0:00:00.057) 0:00:16.640 ***** 2020-12-05T17:25:54.807 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:54 +0000 (0:00:00.048) 0:00:16.689 ***** 2020-12-05T17:25:54.814 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2020-12-05T17:25:54.816 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:54.852 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:54 +0000 (0:00:00.044) 0:00:16.733 ***** 2020-12-05T17:25:54.858 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-12-05T17:25:54.860 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.016428", "end": "2020-12-05 17:25:55.161293", "rc": 0, "start": "2020-12-05 17:25:55.144865", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} TASK [common : List CDN releases available to system] ************************** skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:54.896 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:54 +0000 (0:00:00.044) 0:00:16.778 ***** 2020-12-05T17:25:54.931 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:54 +0000 (0:00:00.034) 0:00:16.813 ***** 2020-12-05T17:25:54.937 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-12-05T17:25:54.939 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:54.974 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:54 +0000 (0:00:00.042) 0:00:16.856 ***** 2020-12-05T17:25:55.007 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:55 +0000 (0:00:00.033) 0:00:16.889 ***** 2020-12-05T17:25:55.021 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-12-05T17:25:55.023 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the apt module rather than running 'apt-get'. If you need to use command because apt is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi002.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.057938", "end": "2020-12-05 17:25:54.926338", "rc": 0, "start": "2020-12-05 17:25:54.868400", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:55.059 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:55 +0000 (0:00:00.052) 0:00:16.941 ***** 2020-12-05T17:25:55.198 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:55 +0000 (0:00:00.138) 0:00:17.080 ***** 2020-12-05T17:25:55.233 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:55 +0000 (0:00:00.034) 0:00:17.114 ***** 2020-12-05T17:25:55.240 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi053.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Get the current timezone] *************************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:55.276 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:55 +0000 (0:00:00.044) 0:00:17.158 ***** 2020-12-05T17:25:55.313 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:55 +0000 (0:00:00.036) 0:00:17.195 ***** 2020-12-05T17:25:55.319 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-12-05T17:25:55.321 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the hardware clock] ***************************************** changed: [smithi032.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.533237", "end": "2020-12-05 17:25:55.500705", "rc": 0, "start": "2020-12-05 17:25:54.967468", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:25:55.322 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:55.360 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:55 +0000 (0:00:00.046) 0:00:17.241 ***** 2020-12-05T17:25:55.394 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:55 +0000 (0:00:00.034) 0:00:17.276 ***** 2020-12-05T17:25:55.429 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:55 +0000 (0:00:00.035) 0:00:17.311 ***** 2020-12-05T17:25:55.435 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-12-05T17:25:55.437 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install Kerberos Packages (RedHat)] ***************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:55.472 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:55 +0000 (0:00:00.043) 0:00:17.354 ***** 2020-12-05T17:25:55.505 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:55 +0000 (0:00:00.032) 0:00:17.386 ***** 2020-12-05T17:25:55.541 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:55 +0000 (0:00:00.036) 0:00:17.423 ***** 2020-12-05T17:25:55.549 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi061.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:25:55.551 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:55.586 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:55 +0000 (0:00:00.044) 0:00:17.468 ***** 2020-12-05T17:25:55.620 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:55 +0000 (0:00:00.033) 0:00:17.502 ***** 2020-12-05T17:25:55.626 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2020-12-05T17:25:55.628 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Refresh repos] ************************************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:55.665 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:55 +0000 (0:00:00.044) 0:00:17.546 ***** 2020-12-05T17:25:55.697 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:55 +0000 (0:00:00.031) 0:00:17.578 ***** 2020-12-05T17:25:55.704 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Clean apt cache] ************************************************ 2020-12-05T17:25:55.706 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.055495", "end": "2020-12-05 17:25:55.863993", "rc": 0, "start": "2020-12-05 17:25:55.808498", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:25:55.743 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:55 +0000 (0:00:00.046) 0:00:17.625 ***** 2020-12-05T17:25:55.782 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:55 +0000 (0:00:00.037) 0:00:17.662 ***** 2020-12-05T17:25:55.796 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2020-12-05T17:25:55.798 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:55.831 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:55 +0000 (0:00:00.050) 0:00:17.713 ***** 2020-12-05T17:25:55.838 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:55.875 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:55 +0000 (0:00:00.043) 0:00:17.756 ***** 2020-12-05T17:25:55.880 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2020-12-05T17:25:55.882 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:55.917 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:55 +0000 (0:00:00.042) 0:00:17.799 ***** 2020-12-05T17:25:55.923 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:55.959 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:55 +0000 (0:00:00.042) 0:00:17.841 ***** 2020-12-05T17:25:55.965 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2020-12-05T17:25:55.967 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:56.003 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:55 +0000 (0:00:00.043) 0:00:17.884 ***** 2020-12-05T17:25:56.009 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2020-12-05T17:25:56.010 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:56.046 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:56 +0000 (0:00:00.043) 0:00:17.928 ***** 2020-12-05T17:25:56.053 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2020-12-05T17:25:56.055 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:56.090 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:56 +0000 (0:00:00.042) 0:00:17.971 ***** 2020-12-05T17:25:56.097 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2020-12-05T17:25:56.098 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:56.132 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:56 +0000 (0:00:00.043) 0:00:18.014 ***** 2020-12-05T17:25:56.139 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-12-05T17:25:56.141 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:25:56.177 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:56 +0000 (0:00:00.044) 0:00:18.059 ***** 2020-12-05T17:25:56.184 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2020-12-05T17:25:56.186 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the hardware clock] ***************************************** changed: [smithi036.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.379075", "end": "2020-12-05 17:25:56.500799", "rc": 0, "start": "2020-12-05 17:25:56.121724", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:25:56.221 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:56 +0000 (0:00:00.043) 0:00:18.103 ***** 2020-12-05T17:25:56.254 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:56 +0000 (0:00:00.033) 0:00:18.136 ***** 2020-12-05T17:25:56.260 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2020-12-05T17:25:56.262 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install Kerberos Packages (RedHat)] ***************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:56.297 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:56 +0000 (0:00:00.042) 0:00:18.179 ***** 2020-12-05T17:25:56.330 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:56 +0000 (0:00:00.033) 0:00:18.212 ***** 2020-12-05T17:25:56.337 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-12-05T17:25:56.339 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:25:56.375 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:56 +0000 (0:00:00.044) 0:00:18.257 ***** 2020-12-05T17:25:56.380 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-12-05T17:25:56.382 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:56.422 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:56 +0000 (0:00:00.047) 0:00:18.304 ***** 2020-12-05T17:25:56.429 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2020-12-05T17:25:56.431 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:56.466 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:56 +0000 (0:00:00.043) 0:00:18.348 ***** 2020-12-05T17:25:56.472 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2020-12-05T17:25:56.474 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:56.508 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:56 +0000 (0:00:00.042) 0:00:18.390 ***** 2020-12-05T17:25:56.515 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-12-05T17:25:56.517 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Clean apt cache] ************************************************ changed: [smithi036.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.060206", "end": "2020-12-05 17:25:56.832135", "rc": 0, "start": "2020-12-05 17:25:56.771929", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:25:56.555 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:56 +0000 (0:00:00.046) 0:00:18.436 ***** 2020-12-05T17:25:56.587 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:56 +0000 (0:00:00.032) 0:00:18.469 ***** 2020-12-05T17:25:56.593 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-12-05T17:25:56.595 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:56.632 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:56 +0000 (0:00:00.044) 0:00:18.513 ***** 2020-12-05T17:25:56.638 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2020-12-05T17:25:56.639 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:56.677 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:56 +0000 (0:00:00.045) 0:00:18.559 ***** 2020-12-05T17:25:56.684 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-12-05T17:25:56.685 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:56.729 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:56 +0000 (0:00:00.051) 0:00:18.611 ***** 2020-12-05T17:25:56.735 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-12-05T17:25:56.738 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:56.778 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:56 +0000 (0:00:00.048) 0:00:18.659 ***** 2020-12-05T17:25:56.785 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-12-05T17:25:56.786 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:56.832 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:56 +0000 (0:00:00.053) 0:00:18.713 ***** 2020-12-05T17:25:56.838 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-12-05T17:25:56.840 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2020-12-05T17:25:56.879 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:56 +0000 (0:00:00.047) 0:00:18.760 ***** 2020-12-05T17:25:56.906 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-12-05T17:25:56.908 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:56.946 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:56 +0000 (0:00:00.066) 0:00:18.827 ***** 2020-12-05T17:25:56.974 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-12-05T17:25:56.976 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.011932", "end": "2020-12-05 17:25:57.015636", "rc": 0, "start": "2020-12-05 17:25:57.003704", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-12-05T17:25:57.011 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:57 +0000 (0:00:00.065) 0:00:18.893 ***** 2020-12-05T17:25:57.050 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-12-05T17:25:57.052 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:57.106 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:57 +0000 (0:00:00.094) 0:00:18.987 ***** 2020-12-05T17:25:57.115 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-12-05T17:25:57.118 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.011106", "end": "2020-12-05 17:25:57.269879", "rc": 0, "start": "2020-12-05 17:25:57.258773", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-12-05T17:25:57.179 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:57 +0000 (0:00:00.072) 0:00:19.060 ***** 2020-12-05T17:25:57.189 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:57.231 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:57 +0000 (0:00:00.052) 0:00:19.113 ***** 2020-12-05T17:25:57.239 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-12-05T17:25:57.241 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:57.283 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:57 +0000 (0:00:00.051) 0:00:19.165 ***** 2020-12-05T17:25:57.289 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-12-05T17:25:57.291 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:57.326 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:57 +0000 (0:00:00.043) 0:00:19.208 ***** 2020-12-05T17:25:57.333 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:57.369 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:57 +0000 (0:00:00.043) 0:00:19.251 ***** 2020-12-05T17:25:57.376 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:57.424 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:57 +0000 (0:00:00.054) 0:00:19.305 ***** 2020-12-05T17:25:57.430 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-12-05T17:25:57.432 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:57.466 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:57 +0000 (0:00:00.042) 0:00:19.348 ***** 2020-12-05T17:25:57.473 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-12-05T17:25:57.475 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:57.510 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:57 +0000 (0:00:00.043) 0:00:19.392 ***** 2020-12-05T17:25:57.517 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:57.562 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:57 +0000 (0:00:00.050) 0:00:19.443 ***** 2020-12-05T17:25:57.569 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2020-12-05T17:25:57.571 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:57.631 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:57 +0000 (0:00:00.070) 0:00:19.513 ***** 2020-12-05T17:25:57.639 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:57.685 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:57 +0000 (0:00:00.053) 0:00:19.567 ***** 2020-12-05T17:25:57.692 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:57.729 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:57 +0000 (0:00:00.043) 0:00:19.611 ***** 2020-12-05T17:25:57.736 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:57.774 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:57 +0000 (0:00:00.044) 0:00:19.656 ***** 2020-12-05T17:25:57.802 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-12-05T17:25:57.804 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:57.841 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:57 +0000 (0:00:00.064) 0:00:19.720 ***** 2020-12-05T17:25:58.327 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-12-05T17:25:58.329 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.364798", "end": "2020-12-05 17:25:58.500499", "rc": 0, "start": "2020-12-05 17:25:58.135701", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:25:58.361 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:58 +0000 (0:00:00.522) 0:00:20.243 ***** 2020-12-05T17:25:58.387 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:58.420 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:58 +0000 (0:00:00.059) 0:00:20.302 ***** 2020-12-05T17:25:58.457 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-12-05T17:25:58.459 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.594379", "end": "2020-12-05 17:25:58.500717", "rc": 0, "start": "2020-12-05 17:25:57.906338", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:25:58.493 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:58 +0000 (0:00:00.072) 0:00:20.375 ***** 2020-12-05T17:25:58.520 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-12-05T17:25:58.521 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:25:58.554 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:58 +0000 (0:00:00.061) 0:00:20.436 ***** 2020-12-05T17:25:58.612 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-12-05T17:25:58.614 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.054020", "end": "2020-12-05 17:25:58.783142", "rc": 0, "start": "2020-12-05 17:25:58.729122", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:25:58.648 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:58 +0000 (0:00:00.093) 0:00:20.529 ***** 2020-12-05T17:25:58.738 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.054832", "end": "2020-12-05 17:25:58.777760", "rc": 0, "start": "2020-12-05 17:25:58.722928", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:25:58.782 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:58 +0000 (0:00:00.134) 0:00:20.664 ***** 2020-12-05T17:25:59.627 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-12-05T17:25:59.630 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1607188915, "cache_updated": false, "changed": false} 2020-12-05T17:25:59.664 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:59 +0000 (0:00:00.882) 0:00:21.546 ***** 2020-12-05T17:25:59.671 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1607188779, "cache_updated": false, "changed": false} 2020-12-05T17:25:59.708 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:25:59 +0000 (0:00:00.043) 0:00:21.589 ***** 2020-12-05T17:26:00.503 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-12-05T17:26:00.505 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"cache_update_time": 1607188915, "cache_updated": false, "changed": false} 2020-12-05T17:26:00.545 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:00 +0000 (0:00:00.836) 0:00:22.426 ***** 2020-12-05T17:26:00.573 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-12-05T17:26:00.575 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:00.576 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** ok: [smithi036.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1607189016, "cache_updated": false, "changed": false} 2020-12-05T17:26:00.610 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:00 +0000 (0:00:00.065) 0:00:22.492 ***** 2020-12-05T17:26:00.652 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:00 +0000 (0:00:00.042) 0:00:22.534 ***** 2020-12-05T17:26:00.662 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-12-05T17:26:00.663 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"cache_update_time": 1607188779, "cache_updated": false, "changed": false} 2020-12-05T17:26:00.708 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:00 +0000 (0:00:00.055) 0:00:22.590 ***** 2020-12-05T17:26:00.739 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-12-05T17:26:00.741 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:00.776 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:00 +0000 (0:00:00.068) 0:00:22.658 ***** 2020-12-05T17:26:00.913 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-12-05T17:26:00.915 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-12-05T17:26:00.955 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:00 +0000 (0:00:00.179) 0:00:22.837 ***** 2020-12-05T17:26:01.108 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-12-05T17:26:01.144 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:01 +0000 (0:00:00.188) 0:00:23.026 ***** 2020-12-05T17:26:01.610 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-12-05T17:26:01.612 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"cache_update_time": 1607189016, "cache_updated": false, "changed": false} 2020-12-05T17:26:01.646 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:01 +0000 (0:00:00.502) 0:00:23.528 ***** 2020-12-05T17:26:01.673 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-12-05T17:26:01.674 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:01.710 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:01 +0000 (0:00:00.063) 0:00:23.592 ***** 2020-12-05T17:26:01.875 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-12-05T17:26:01.877 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"cache_update_time": 1607188915, "cache_updated": false, "changed": false} 2020-12-05T17:26:01.915 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:01 +0000 (0:00:00.205) 0:00:23.797 ***** 2020-12-05T17:26:02.016 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-12-05T17:26:02.018 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-12-05T17:26:02.057 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:02 +0000 (0:00:00.141) 0:00:23.938 ***** 2020-12-05T17:26:02.064 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-12-05T17:26:02.066 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"cache_update_time": 1607188779, "cache_updated": false, "changed": false} 2020-12-05T17:26:02.106 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:02 +0000 (0:00:00.049) 0:00:23.988 ***** 2020-12-05T17:26:02.239 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-12-05T17:26:02.241 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-12-05T17:26:02.504 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-12-05T17:26:02.558 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-12-05T17:26:02.560 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1607188959, "cache_updated": false, "changed": false} 2020-12-05T17:26:02.596 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:02 +0000 (0:00:00.489) 0:00:24.477 ***** 2020-12-05T17:26:02.603 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1607189009, "cache_updated": false, "changed": false} 2020-12-05T17:26:02.639 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:02 +0000 (0:00:00.043) 0:00:24.521 ***** 2020-12-05T17:26:02.761 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-12-05T17:26:02.763 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-12-05T17:26:02.801 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:02 +0000 (0:00:00.161) 0:00:24.682 ***** 2020-12-05T17:26:02.924 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-12-05T17:26:02.926 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-12-05T17:26:02.961 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:02 +0000 (0:00:00.160) 0:00:24.843 ***** 2020-12-05T17:26:03.031 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-12-05T17:26:03.032 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"cache_update_time": 1607189016, "cache_updated": false, "changed": false} 2020-12-05T17:26:03.070 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:03 +0000 (0:00:00.108) 0:00:24.951 ***** 2020-12-05T17:26:03.268 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-12-05T17:26:03.271 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2020-12-05T17:26:03.397 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-12-05T17:26:03.399 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-12-05T17:26:03.441 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-12-05T17:26:03.527 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-12-05T17:26:03.528 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-12-05T17:26:03.560 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-12-05T17:26:03.562 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"cache_update_time": 1607189009, "cache_updated": false, "changed": false} 2020-12-05T17:26:03.598 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:03 +0000 (0:00:00.527) 0:00:25.479 ***** 2020-12-05T17:26:03.604 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"cache_update_time": 1607188959, "cache_updated": false, "changed": false} 2020-12-05T17:26:03.641 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:03 +0000 (0:00:00.043) 0:00:25.522 ***** 2020-12-05T17:26:03.648 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-12-05T17:26:03.650 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:03.687 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:03 +0000 (0:00:00.046) 0:00:25.569 ***** 2020-12-05T17:26:03.693 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:03.695 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi036.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-12-05T17:26:03.733 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:03 +0000 (0:00:00.045) 0:00:25.615 ***** 2020-12-05T17:26:03.739 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-12-05T17:26:03.796 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-12-05T17:26:03.798 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-12-05T17:26:03.835 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:03 +0000 (0:00:00.101) 0:00:25.716 ***** 2020-12-05T17:26:03.863 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-12-05T17:26:03.865 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2020-12-05T17:26:03.899 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:03 +0000 (0:00:00.064) 0:00:25.781 ***** 2020-12-05T17:26:03.927 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-12-05T17:26:03.929 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:03.963 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:03 +0000 (0:00:00.063) 0:00:25.844 ***** 2020-12-05T17:26:03.969 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-12-05T17:26:03.971 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-12-05T17:26:04.015 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:04 +0000 (0:00:00.052) 0:00:25.897 ***** 2020-12-05T17:26:04.021 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-12-05T17:26:04.022 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi032.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-12-05T17:26:04.025 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-12-05T17:26:04.026 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-12-05T17:26:04.069 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:04 +0000 (0:00:00.053) 0:00:25.951 ***** 2020-12-05T17:26:04.104 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:04 +0000 (0:00:00.034) 0:00:25.985 ***** 2020-12-05T17:26:04.144 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:04 +0000 (0:00:00.039) 0:00:26.025 ***** 2020-12-05T17:26:04.150 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-12-05T17:26:04.151 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-12-05T17:26:04.151 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:04.152 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:04.152 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:04.153 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:04.154 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-12-05T17:26:04.191 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:04 +0000 (0:00:00.047) 0:00:26.073 ***** 2020-12-05T17:26:04.224 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:04 +0000 (0:00:00.032) 0:00:26.105 ***** 2020-12-05T17:26:04.257 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:04 +0000 (0:00:00.033) 0:00:26.139 ***** 2020-12-05T17:26:04.263 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-12-05T17:26:04.265 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Create /usr/libexec.] ******************************************* ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-12-05T17:26:04.301 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:04 +0000 (0:00:00.043) 0:00:26.182 ***** 2020-12-05T17:26:04.334 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:04 +0000 (0:00:00.033) 0:00:26.216 ***** 2020-12-05T17:26:04.529 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-12-05T17:26:04.531 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2020-12-05T17:26:04.629 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2020-12-05T17:26:04.789 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-12-05T17:26:04.895 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-12-05T17:26:05.034 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-12-05T17:26:05.035 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"cache_update_time": 1607189009, "cache_updated": false, "changed": false} 2020-12-05T17:26:05.075 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:05 +0000 (0:00:00.740) 0:00:26.956 ***** 2020-12-05T17:26:05.081 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-12-05T17:26:05.083 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-12-05T17:26:05.125 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:05 +0000 (0:00:00.050) 0:00:27.006 ***** 2020-12-05T17:26:05.155 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-12-05T17:26:05.157 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [smithi032.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-12-05T17:26:05.204 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:05 +0000 (0:00:00.079) 0:00:27.085 ***** 2020-12-05T17:26:05.237 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:05 +0000 (0:00:00.032) 0:00:27.118 ***** 2020-12-05T17:26:05.243 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-12-05T17:26:05.244 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"cache_update_time": 1607188915, "cache_updated": false, "changed": false} TASK [common : Include apt_system vars] **************************************** ok: [smithi032.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} TASK [common : Make sure smartmontools is installed] *************************** 2020-12-05T17:26:05.245 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"cache_update_time": 1607188959, "cache_updated": false, "changed": false} 2020-12-05T17:26:05.287 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:05 +0000 (0:00:00.050) 0:00:27.169 ***** 2020-12-05T17:26:05.330 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:05 +0000 (0:00:00.042) 0:00:27.212 ***** 2020-12-05T17:26:05.367 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:05 +0000 (0:00:00.037) 0:00:27.249 ***** 2020-12-05T17:26:05.374 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-12-05T17:26:05.376 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:05.413 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:05 +0000 (0:00:00.046) 0:00:27.295 ***** 2020-12-05T17:26:05.450 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:05 +0000 (0:00:00.036) 0:00:27.332 ***** 2020-12-05T17:26:05.457 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-12-05T17:26:05.459 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} TASK [common : Check for epel] ************************************************* skipping: [smithi032.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:26:05.499 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:05 +0000 (0:00:00.048) 0:00:27.380 ***** 2020-12-05T17:26:05.506 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:26:05.545 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:05 +0000 (0:00:00.046) 0:00:27.427 ***** 2020-12-05T17:26:05.555 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-12-05T17:26:05.557 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:05.571 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:05.571 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:05.622 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:05 +0000 (0:00:00.077) 0:00:27.504 ***** 2020-12-05T17:26:05.632 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-12-05T17:26:05.634 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-12-05T17:26:05.636 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:05.691 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:05 +0000 (0:00:00.068) 0:00:27.572 ***** 2020-12-05T17:26:05.728 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:05 +0000 (0:00:00.036) 0:00:27.609 ***** 2020-12-05T17:26:05.736 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-12-05T17:26:05.737 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi061.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-12-05T17:26:05.739 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-12-05T17:26:05.781 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:05 +0000 (0:00:00.053) 0:00:27.662 ***** 2020-12-05T17:26:05.787 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-12-05T17:26:05.789 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:05.840 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:05 +0000 (0:00:00.059) 0:00:27.722 ***** 2020-12-05T17:26:05.945 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi061.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-12-05T17:26:05.983 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:05 +0000 (0:00:00.142) 0:00:27.865 ***** 2020-12-05T17:26:06.001 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-12-05T17:26:06.012 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-12-05T17:26:06.014 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-12-05T17:26:06.051 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:06 +0000 (0:00:00.068) 0:00:27.933 ***** 2020-12-05T17:26:06.102 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-12-05T17:26:06.136 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:06 +0000 (0:00:00.084) 0:00:28.017 ***** 2020-12-05T17:26:06.196 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-12-05T17:26:06.198 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2020-12-05T17:26:06.283 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-12-05T17:26:06.285 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-12-05T17:26:06.331 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:06 +0000 (0:00:00.195) 0:00:28.213 ***** 2020-12-05T17:26:06.340 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-12-05T17:26:06.341 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-12-05T17:26:06.376 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:06 +0000 (0:00:00.045) 0:00:28.258 ***** 2020-12-05T17:26:06.402 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-12-05T17:26:06.404 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:06.435 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:06 +0000 (0:00:00.058) 0:00:28.317 ***** 2020-12-05T17:26:06.451 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-12-05T17:26:06.453 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2020-12-05T17:26:06.463 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** skipping: [smithi002.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:26:06.465 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-12-05T17:26:06.496 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:06 +0000 (0:00:00.060) 0:00:28.378 ***** 2020-12-05T17:26:06.528 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:06 +0000 (0:00:00.031) 0:00:28.410 ***** 2020-12-05T17:26:06.535 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-12-05T17:26:06.537 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:06.569 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:06 +0000 (0:00:00.041) 0:00:28.451 ***** 2020-12-05T17:26:06.712 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-12-05T17:26:06.714 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-12-05T17:26:06.735 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-12-05T17:26:06.737 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"cache_update_time": 1607188779, "cache_updated": false, "changed": false} 2020-12-05T17:26:06.774 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:06 +0000 (0:00:00.204) 0:00:28.655 ***** 2020-12-05T17:26:06.780 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"cache_update_time": 1607189016, "cache_updated": false, "changed": false} 2020-12-05T17:26:06.826 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:06 +0000 (0:00:00.052) 0:00:28.708 ***** 2020-12-05T17:26:06.846 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-12-05T17:26:06.848 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2020-12-05T17:26:06.986 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-12-05T17:26:07.028 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:07 +0000 (0:00:00.201) 0:00:28.910 ***** 2020-12-05T17:26:07.044 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-12-05T17:26:07.047 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2020-12-05T17:26:07.084 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:07 +0000 (0:00:00.055) 0:00:28.965 ***** 2020-12-05T17:26:07.091 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-12-05T17:26:07.093 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-12-05T17:26:07.093 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-12-05T17:26:07.146 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:07 +0000 (0:00:00.062) 0:00:29.027 ***** 2020-12-05T17:26:07.155 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-12-05T17:26:07.157 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-12-05T17:26:07.198 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:07 +0000 (0:00:00.051) 0:00:29.079 ***** 2020-12-05T17:26:07.207 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-12-05T17:26:07.209 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-12-05T17:26:07.209 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:07.259 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:07 +0000 (0:00:00.061) 0:00:29.141 ***** 2020-12-05T17:26:07.293 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:07 +0000 (0:00:00.033) 0:00:29.174 ***** 2020-12-05T17:26:07.335 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-12-05T17:26:07.336 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:26:07.384 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:07 +0000 (0:00:00.090) 0:00:29.265 ***** 2020-12-05T17:26:07.390 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-12-05T17:26:07.391 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-12-05T17:26:07.439 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:07 +0000 (0:00:00.056) 0:00:29.321 ***** 2020-12-05T17:26:07.446 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-12-05T17:26:07.447 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:07.448 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:07.486 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:07 +0000 (0:00:00.046) 0:00:29.368 ***** 2020-12-05T17:26:07.493 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-12-05T17:26:07.495 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2020-12-05T17:26:07.530 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:07 +0000 (0:00:00.043) 0:00:29.411 ***** 2020-12-05T17:26:07.536 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-12-05T17:26:07.538 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Upload nagios check_mem script] ********************************* ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-12-05T17:26:07.574 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:07 +0000 (0:00:00.043) 0:00:29.455 ***** 2020-12-05T17:26:07.608 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:07 +0000 (0:00:00.033) 0:00:29.489 ***** 2020-12-05T17:26:07.613 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-12-05T17:26:07.616 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Upload nagios check_mem script] ********************************* ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-12-05T17:26:07.663 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:07 +0000 (0:00:00.055) 0:00:29.545 ***** 2020-12-05T17:26:07.702 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:07 +0000 (0:00:00.038) 0:00:29.584 ***** 2020-12-05T17:26:07.712 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-12-05T17:26:07.714 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-12-05 17:25:19 UTC", "ActiveEnterTimestampMonotonic": "19901565", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "nss-lookup.target time-sync.target -.mount sysinit.target system.slice remote-fs.target systemd-tmpfiles-setup.service basic.target local-fs.target network.target var-run.mount systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-12-05 17:25:19 UTC", "AssertTimestampMonotonic": "19900547", "Before": "shutdown.target multi-user.target xdm.service plymouth-quit.service getty@tty1.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-12-05 17:25:19 UTC", "ConditionTimestampMonotonic": "19900546", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-ser 2020-12-05T17:26:07.716 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1329", "ExecMainStartTimestamp": "Sat 2020-12-05 17:25:19 UTC", "ExecMainStartTimestampMonotonic": "19901534", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sat 2020-12-05 17:25:19 UTC] ; stop_time=[n/a] ; pid=1329 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-12-05 17:25:19 UTC", "InactiveExitTimestampMonotonic": "19901565", "InvocationID": "595e7b84da9c43de9b508f61d6398127", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "i 2020-12-05T17:26:07.717 INFO:teuthology.task.ansible.out:nfinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128053", "LimitNPROCSoft": "128053", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128053", "LimitSIGPENDINGSoft": "128053", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1329", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", 2020-12-05T17:26:07.719 INFO:teuthology.task.ansible.out: "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-12-05 17:25:19 UTC", "StateChangeTimestampMonotonic": "19901565", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-12-05 17:25:19 UTC", "WatchdogTimestampMonotonic": "19901564", "WatchdogUSec": "0"}} TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ ok: [smithi032.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2020-12-05T17:26:07.752 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:07 +0000 (0:00:00.049) 0:00:29.633 ***** 2020-12-05T17:26:07.759 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-12-05T17:26:07.760 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:26:07.797 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:07 +0000 (0:00:00.045) 0:00:29.679 ***** 2020-12-05T17:26:07.808 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-12-05T17:26:07.809 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ ok: [smithi036.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2020-12-05T17:26:07.845 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:07 +0000 (0:00:00.047) 0:00:29.727 ***** 2020-12-05T17:26:07.853 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-12-05T17:26:07.855 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-12-05T17:26:07.857 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:07.857 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:07.893 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:07 +0000 (0:00:00.048) 0:00:29.775 ***** 2020-12-05T17:26:07.900 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:07.902 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:07.938 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:07 +0000 (0:00:00.044) 0:00:29.820 ***** 2020-12-05T17:26:07.972 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:07 +0000 (0:00:00.034) 0:00:29.854 ***** 2020-12-05T17:26:07.979 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-12-05T17:26:07.981 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-12-05T17:26:07.982 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-12-05T17:26:07.983 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:08.031 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:08 +0000 (0:00:00.058) 0:00:29.913 ***** 2020-12-05T17:26:08.066 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:08 +0000 (0:00:00.034) 0:00:29.947 ***** 2020-12-05T17:26:08.100 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:08 +0000 (0:00:00.033) 0:00:29.981 ***** 2020-12-05T17:26:08.105 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-12-05T17:26:08.107 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:08.108 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** skipping: [smithi032.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:26:08.163 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:08 +0000 (0:00:00.063) 0:00:30.045 ***** 2020-12-05T17:26:08.198 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:08 +0000 (0:00:00.034) 0:00:30.079 ***** 2020-12-05T17:26:08.231 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:08 +0000 (0:00:00.033) 0:00:30.113 ***** 2020-12-05T17:26:08.237 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-12-05T17:26:08.240 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Remove SELinux policy package] *************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Open nrpe port if firewalld enabled] **************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:08.280 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:08 +0000 (0:00:00.048) 0:00:30.162 ***** 2020-12-05T17:26:08.314 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:08 +0000 (0:00:00.033) 0:00:30.196 ***** 2020-12-05T17:26:08.349 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:08 +0000 (0:00:00.034) 0:00:30.231 ***** 2020-12-05T17:26:08.356 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-12-05T17:26:08.358 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:08.394 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:08 +0000 (0:00:00.044) 0:00:30.276 ***** 2020-12-05T17:26:08.401 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-12-05T17:26:08.403 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:26:08.438 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:08 +0000 (0:00:00.044) 0:00:30.320 ***** 2020-12-05T17:26:08.445 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-12-05T17:26:08.447 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:08.483 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:08 +0000 (0:00:00.044) 0:00:30.365 ***** 2020-12-05T17:26:08.491 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-12-05T17:26:08.493 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:08.543 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:08 +0000 (0:00:00.060) 0:00:30.425 ***** 2020-12-05T17:26:08.550 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-12-05T17:26:08.552 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** ok: [smithi061.front.sepia.ceph.com] => {"cache_update_time": 1607189009, "cache_updated": false, "changed": false} 2020-12-05T17:26:08.589 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:08 +0000 (0:00:00.046) 0:00:30.471 ***** 2020-12-05T17:26:08.634 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:08 +0000 (0:00:00.044) 0:00:30.515 ***** 2020-12-05T17:26:08.639 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-12-05T17:26:08.641 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:08.688 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:08 +0000 (0:00:00.054) 0:00:30.570 ***** 2020-12-05T17:26:08.694 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-12-05T17:26:08.696 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2020-12-05T17:26:08.734 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:08 +0000 (0:00:00.045) 0:00:30.615 ***** 2020-12-05T17:26:08.740 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-12-05T17:26:08.742 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:08.780 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:08 +0000 (0:00:00.046) 0:00:30.662 ***** 2020-12-05T17:26:08.819 INFO:teuthology.task.ansible.out: 2020-12-05T17:26:08.821 INFO:teuthology.task.ansible.out:TASK [testnode : Include package type specific vars.] ************************** ok: [smithi002.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2020-12-05T17:26:08.857 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:08 +0000 (0:00:00.076) 0:00:30.738 ***** 2020-12-05T17:26:08.858 INFO:teuthology.task.ansible.out: 2020-12-05T17:26:08.863 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-12-05T17:26:08.865 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2020-12-05T17:26:08.901 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:08 +0000 (0:00:00.044) 0:00:30.783 ***** 2020-12-05T17:26:08.908 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-12-05T17:26:08.909 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-12-05T17:26:08.957 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:08 +0000 (0:00:00.056) 0:00:30.839 ***** 2020-12-05T17:26:08.967 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-12-05T17:26:08.970 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-12-05 17:25:03 UTC", "ActiveEnterTimestampMonotonic": "20976876", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "var-run.mount systemd-journald.socket systemd-tmpfiles-setup.service system.slice network.target sysinit.target local-fs.target nss-lookup.target -.mount basic.target remote-fs.target time-sync.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-12-05 17:25:03 UTC", "AssertTimestampMonotonic": "20975084", "Before": "shutdown.target xdm.service multi-user.target plymouth-quit.service getty@tty1.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-12-05 17:25:03 UTC", "ConditionTimestampMonotonic": "20975083", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-ser 2020-12-05T17:26:08.972 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1488", "ExecMainStartTimestamp": "Sat 2020-12-05 17:25:03 UTC", "ExecMainStartTimestampMonotonic": "20976790", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sat 2020-12-05 17:25:03 UTC] ; stop_time=[n/a] ; pid=1488 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-12-05 17:25:03 UTC", "InactiveExitTimestampMonotonic": "20976876", "InvocationID": "510899d387284c65b5391475838aea71", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "i 2020-12-05T17:26:08.973 INFO:teuthology.task.ansible.out:nfinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128053", "LimitNPROCSoft": "128053", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128053", "LimitSIGPENDINGSoft": "128053", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1488", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", 2020-12-05T17:26:08.975 INFO:teuthology.task.ansible.out: "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-12-05 17:25:03 UTC", "StateChangeTimestampMonotonic": "20976876", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-12-05 17:25:03 UTC", "WatchdogTimestampMonotonic": "20976874", "WatchdogUSec": "0"}} TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-12-05T17:26:09.013 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.055) 0:00:30.895 ***** 2020-12-05T17:26:09.048 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.034) 0:00:30.929 ***** 2020-12-05T17:26:09.054 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-12-05T17:26:09.056 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2020-12-05T17:26:09.056 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:09.092 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.044) 0:00:30.973 ***** 2020-12-05T17:26:09.124 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.032) 0:00:31.006 ***** 2020-12-05T17:26:09.133 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-12-05T17:26:09.135 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-12-05T17:26:09.137 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-12-05 17:25:12 UTC", "ActiveEnterTimestampMonotonic": "19994253", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "-.mount time-sync.target sysinit.target var-run.mount network.target remote-fs.target systemd-tmpfiles-setup.service systemd-journald.socket local-fs.target nss-lookup.target basic.target system.slice", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-12-05 17:25:12 UTC", "AssertTimestampMonotonic": "19993520", "Before": "plymouth-quit.service xdm.service getty@tty1.service multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-12-05 17:25:12 UTC", "ConditionTimestampMonotonic": "19993520", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nagios-nrpe-ser 2020-12-05T17:26:09.139 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1324", "ExecMainStartTimestamp": "Sat 2020-12-05 17:25:12 UTC", "ExecMainStartTimestampMonotonic": "19994228", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sat 2020-12-05 17:25:12 UTC] ; stop_time=[n/a] ; pid=1324 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-12-05 17:25:12 UTC", "InactiveExitTimestampMonotonic": "19994253", "InvocationID": "982b76d3646a4df3ad859a8fa5d80f1f", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "i 2020-12-05T17:26:09.141 INFO:teuthology.task.ansible.out:nfinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128053", "LimitNPROCSoft": "128053", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128053", "LimitSIGPENDINGSoft": "128053", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1324", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", 2020-12-05T17:26:09.142 INFO:teuthology.task.ansible.out: "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-12-05 17:25:12 UTC", "StateChangeTimestampMonotonic": "19994253", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-12-05 17:25:12 UTC", "WatchdogTimestampMonotonic": "19994253", "WatchdogUSec": "0"}} 2020-12-05T17:26:09.173 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.049) 0:00:31.055 ***** 2020-12-05T17:26:09.206 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.032) 0:00:31.088 ***** 2020-12-05T17:26:09.214 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-12-05T17:26:09.215 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** ok: [smithi053.front.sepia.ceph.com] => {"cache_update_time": 1607188959, "cache_updated": false, "changed": false} 2020-12-05T17:26:09.250 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.043) 0:00:31.131 ***** 2020-12-05T17:26:09.282 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.032) 0:00:31.164 ***** 2020-12-05T17:26:09.289 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-12-05T17:26:09.290 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:09.325 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.042) 0:00:31.207 ***** 2020-12-05T17:26:09.358 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.032) 0:00:31.239 ***** 2020-12-05T17:26:09.364 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-12-05T17:26:09.366 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} TASK [common : Upload nagios check_mem script] ********************************* ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-12-05T17:26:09.367 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:09.401 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.043) 0:00:31.283 ***** 2020-12-05T17:26:09.434 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.032) 0:00:31.316 ***** 2020-12-05T17:26:09.469 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.034) 0:00:31.351 ***** 2020-12-05T17:26:09.475 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-12-05T17:26:09.477 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Remove SELinux policy package] *************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:09.517 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.048) 0:00:31.399 ***** 2020-12-05T17:26:09.551 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.033) 0:00:31.433 ***** 2020-12-05T17:26:09.558 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-12-05T17:26:09.560 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "name": "ubuntu", "state": "present", "system": false} TASK [common : nrpe - Copy SELinux type enforcement file] ********************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:09.595 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.043) 0:00:31.476 ***** 2020-12-05T17:26:09.628 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.032) 0:00:31.509 ***** 2020-12-05T17:26:09.633 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-12-05T17:26:09.635 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-12-05T17:26:09.636 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-12-05T17:26:09.636 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2020-12-05T17:26:09.638 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:09.673 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.045) 0:00:31.554 ***** 2020-12-05T17:26:09.705 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.031) 0:00:31.586 ***** 2020-12-05T17:26:09.737 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.032) 0:00:31.619 ***** 2020-12-05T17:26:09.742 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-12-05T17:26:09.744 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Delete the teuthology users password.] ************************ skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:09.744 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-12-05T17:26:09.745 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-12-05T17:26:09.783 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.045) 0:00:31.665 ***** 2020-12-05T17:26:09.815 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.031) 0:00:31.696 ***** 2020-12-05T17:26:09.848 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.032) 0:00:31.729 ***** 2020-12-05T17:26:09.884 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.036) 0:00:31.766 ***** 2020-12-05T17:26:09.892 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-12-05T17:26:09.894 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Remove SELinux policy package] *************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:09.942 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.058) 0:00:31.824 ***** 2020-12-05T17:26:09.977 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:09 +0000 (0:00:00.034) 0:00:31.859 ***** 2020-12-05T17:26:09.985 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-12-05T17:26:09.987 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add a user for xfstests to test user quotas.] ***************** ok: [smithi002.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2020-12-05T17:26:09.987 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:10.036 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.058) 0:00:31.918 ***** 2020-12-05T17:26:10.071 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.034) 0:00:31.952 ***** 2020-12-05T17:26:10.112 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.041) 0:00:31.994 ***** 2020-12-05T17:26:10.117 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-12-05T17:26:10.119 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Upload nagios check_mem script] ********************************* ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-12-05T17:26:10.119 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:10.157 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.045) 0:00:32.039 ***** 2020-12-05T17:26:10.190 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.032) 0:00:32.072 ***** 2020-12-05T17:26:10.223 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.032) 0:00:32.104 ***** 2020-12-05T17:26:10.230 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** skipping: [smithi061.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:26:10.232 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** ok: [smithi032.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} TASK [common : nrpe - Compile SELinux module file] ***************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:10.268 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.045) 0:00:32.149 ***** 2020-12-05T17:26:10.301 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.032) 0:00:32.182 ***** 2020-12-05T17:26:10.334 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.033) 0:00:32.216 ***** 2020-12-05T17:26:10.340 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-12-05T17:26:10.342 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-12-05T17:26:10.343 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:10.380 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.045) 0:00:32.261 ***** 2020-12-05T17:26:10.412 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.032) 0:00:32.294 ***** 2020-12-05T17:26:10.418 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-12-05T17:26:10.420 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ ok: [smithi053.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2020-12-05T17:26:10.421 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-12-05T17:26:10.421 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2020-12-05T17:26:10.422 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-12-05T17:26:10.422 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2020-12-05T17:26:10.460 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.047) 0:00:32.342 ***** 2020-12-05T17:26:10.496 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.035) 0:00:32.377 ***** 2020-12-05T17:26:10.530 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.034) 0:00:32.411 ***** 2020-12-05T17:26:10.535 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-12-05T17:26:10.537 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:10.538 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ ok: [smithi053.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-12-05T17:26:10.540 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-12-05T17:26:10.540 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi002"}, "changed": false} 2020-12-05T17:26:10.541 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-12-05T17:26:10.541 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-12-05T17:26:10.580 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.049) 0:00:32.461 ***** 2020-12-05T17:26:10.621 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.041) 0:00:32.503 ***** 2020-12-05T17:26:10.657 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.035) 0:00:32.538 ***** 2020-12-05T17:26:10.694 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.037) 0:00:32.576 ***** 2020-12-05T17:26:10.701 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-12-05T17:26:10.702 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:10.741 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.046) 0:00:32.622 ***** 2020-12-05T17:26:10.747 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-12-05T17:26:10.749 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure the sudo group exists.] ******************************** ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-12-05T17:26:10.785 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.044) 0:00:32.667 ***** 2020-12-05T17:26:10.820 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.033) 0:00:32.700 ***** 2020-12-05T17:26:10.825 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-12-05T17:26:10.828 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2020-12-05T17:26:10.873 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.053) 0:00:32.754 ***** 2020-12-05T17:26:10.880 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2020-12-05T17:26:10.882 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** skipping: [smithi053.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-12-05T17:26:10.926 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.053) 0:00:32.808 ***** 2020-12-05T17:26:10.959 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.032) 0:00:32.840 ***** 2020-12-05T17:26:10.964 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-12-05T17:26:10.966 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-12-05T17:26:10.968 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "name": "ubuntu", "state": "present", "system": false} TASK [common : Open nrpe port if firewalld enabled] **************************** skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:11.005 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:10 +0000 (0:00:00.046) 0:00:32.887 ***** 2020-12-05T17:26:11.038 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:11 +0000 (0:00:00.033) 0:00:32.920 ***** 2020-12-05T17:26:11.072 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:11 +0000 (0:00:00.033) 0:00:32.953 ***** 2020-12-05T17:26:11.078 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-12-05T17:26:11.080 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Including major version specific variables.] ****************** ok: [smithi036.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2020-12-05T17:26:11.117 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:11 +0000 (0:00:00.045) 0:00:32.998 ***** 2020-12-05T17:26:11.150 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:11 +0000 (0:00:00.032) 0:00:33.031 ***** 2020-12-05T17:26:11.156 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-12-05T17:26:11.158 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:11.198 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:11 +0000 (0:00:00.048) 0:00:33.079 ***** 2020-12-05T17:26:11.204 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-12-05T17:26:11.206 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-12-05 17:25:19 UTC", "ActiveEnterTimestampMonotonic": "18529304", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "sysinit.target nss-lookup.target -.mount systemd-tmpfiles-setup.service systemd-journald.socket basic.target remote-fs.target network.target time-sync.target system.slice var-run.mount local-fs.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-12-05 17:25:19 UTC", "AssertTimestampMonotonic": "18528455", "Before": "multi-user.target getty@tty1.service xdm.service shutdown.target plymouth-quit.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-12-05 17:25:19 UTC", "ConditionTimestampMonotonic": "18528455", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-ser 2020-12-05T17:26:11.209 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1315", "ExecMainStartTimestamp": "Sat 2020-12-05 17:25:19 UTC", "ExecMainStartTimestampMonotonic": "18529271", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sat 2020-12-05 17:25:19 UTC] ; stop_time=[n/a] ; pid=1315 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-12-05 17:25:19 UTC", "InactiveExitTimestampMonotonic": "18529304", "InvocationID": "428f79f75c7e455e91728097bac66961", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "i 2020-12-05T17:26:11.210 INFO:teuthology.task.ansible.out:nfinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1315", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", 2020-12-05T17:26:11.212 INFO:teuthology.task.ansible.out: "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-12-05 17:25:19 UTC", "StateChangeTimestampMonotonic": "18529304", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-12-05 17:25:19 UTC", "WatchdogTimestampMonotonic": "18529302", "WatchdogUSec": "0"}} TASK [testnode : Including version specific variables.] ************************ ok: [smithi036.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/ 2020-12-05T17:26:11.214 INFO:teuthology.task.ansible.out:git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-12-05T17:26:11.246 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:11 +0000 (0:00:00.048) 0:00:33.128 ***** 2020-12-05T17:26:11.281 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:11 +0000 (0:00:00.034) 0:00:33.162 ***** 2020-12-05T17:26:11.308 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-12-05T17:26:11.309 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:11.351 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:11 +0000 (0:00:00.070) 0:00:33.233 ***** 2020-12-05T17:26:11.358 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-12-05T17:26:11.360 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2020-12-05T17:26:11.397 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:11 +0000 (0:00:00.045) 0:00:33.278 ***** 2020-12-05T17:26:11.403 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-12-05T17:26:11.405 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} TASK [common : nrpe - Install SELinux dependencies] **************************** skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:11.406 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2020-12-05T17:26:11.444 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:11 +0000 (0:00:00.046) 0:00:33.325 ***** 2020-12-05T17:26:11.478 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:11 +0000 (0:00:00.034) 0:00:33.359 ***** 2020-12-05T17:26:11.512 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:11 +0000 (0:00:00.034) 0:00:33.394 ***** 2020-12-05T17:26:11.518 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-12-05T17:26:11.520 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi002.front.sepia.ceph.com", "ansible_hostname": "smithi002", "ansible_nodename": "smithi002"}, "changed": true, "name": "smithi002"} 2020-12-05T17:26:11.558 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:11 +0000 (0:00:00.046) 0:00:33.440 ***** 2020-12-05T17:26:11.565 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:11.603 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:11 +0000 (0:00:00.044) 0:00:33.485 ***** 2020-12-05T17:26:11.610 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-12-05T17:26:11.612 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "name": "ubuntu", "state": "present", "system": false} 2020-12-05T17:26:11.648 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:11 +0000 (0:00:00.044) 0:00:33.530 ***** 2020-12-05T17:26:11.656 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-12-05T17:26:11.658 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:11.695 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:11 +0000 (0:00:00.046) 0:00:33.576 ***** 2020-12-05T17:26:11.701 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-12-05T17:26:11.703 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:11.703 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-12-05T17:26:11.705 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-12-05 17:25:24 UTC", "ActiveEnterTimestampMonotonic": "20349889", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "system.slice systemd-journald.socket network.target systemd-tmpfiles-setup.service sysinit.target nss-lookup.target local-fs.target remote-fs.target var-run.mount -.mount basic.target time-sync.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-12-05 17:25:24 UTC", "AssertTimestampMonotonic": "20349255", "Before": "shutdown.target plymouth-quit.service getty@tty1.service multi-user.target xdm.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-12-05 17:25:24 UTC", "ConditionTimestampMonotonic": "20349255", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-ser 2020-12-05T17:26:11.706 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1292", "ExecMainStartTimestamp": "Sat 2020-12-05 17:25:24 UTC", "ExecMainStartTimestampMonotonic": "20349865", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sat 2020-12-05 17:25:24 UTC] ; stop_time=[n/a] ; pid=1292 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-12-05 17:25:24 UTC", "InactiveExitTimestampMonotonic": "20349889", "InvocationID": "a50c7916bb88470298c19d4c256deabc", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "i 2020-12-05T17:26:11.708 INFO:teuthology.task.ansible.out:nfinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128053", "LimitNPROCSoft": "128053", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128053", "LimitSIGPENDINGSoft": "128053", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1292", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", 2020-12-05T17:26:11.709 INFO:teuthology.task.ansible.out: "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-12-05 17:25:24 UTC", "StateChangeTimestampMonotonic": "20349889", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-12-05 17:25:24 UTC", "WatchdogTimestampMonotonic": "20349889", "WatchdogUSec": "0"}} 2020-12-05T17:26:11.742 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:11 +0000 (0:00:00.046) 0:00:33.623 ***** 2020-12-05T17:26:11.780 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:11 +0000 (0:00:00.038) 0:00:33.662 ***** 2020-12-05T17:26:11.787 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-12-05T17:26:11.788 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Delete the teuthology users password.] ************************ skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:11.825 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:11 +0000 (0:00:00.044) 0:00:33.706 ***** 2020-12-05T17:26:11.860 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:11 +0000 (0:00:00.034) 0:00:33.741 ***** 2020-12-05T17:26:11.867 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-12-05T17:26:11.868 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set a high max open files limit for the teuthology user.] ***** ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2020-12-05T17:26:11.907 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:11 +0000 (0:00:00.046) 0:00:33.788 ***** 2020-12-05T17:26:11.941 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:11 +0000 (0:00:00.034) 0:00:33.823 ***** 2020-12-05T17:26:11.949 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-12-05T17:26:11.951 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} TASK [common : nrpe - Copy SELinux type enforcement file] ********************** skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set hostname var] ********************************************* 2020-12-05T17:26:11.951 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi032"}, "changed": false} 2020-12-05T17:26:11.987 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:11 +0000 (0:00:00.046) 0:00:33.869 ***** 2020-12-05T17:26:12.023 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:12 +0000 (0:00:00.035) 0:00:33.905 ***** 2020-12-05T17:26:12.059 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:12 +0000 (0:00:00.035) 0:00:33.941 ***** 2020-12-05T17:26:12.065 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-12-05T17:26:12.067 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add a user for xfstests to test user quotas.] ***************** ok: [smithi036.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2020-12-05T17:26:12.106 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:12 +0000 (0:00:00.047) 0:00:33.988 ***** 2020-12-05T17:26:12.157 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:12 +0000 (0:00:00.050) 0:00:34.038 ***** 2020-12-05T17:26:12.165 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-12-05T17:26:12.167 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:12.207 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:12 +0000 (0:00:00.050) 0:00:34.089 ***** 2020-12-05T17:26:12.213 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-12-05T17:26:12.215 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:12.251 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:12 +0000 (0:00:00.043) 0:00:34.133 ***** 2020-12-05T17:26:12.259 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-12-05T17:26:12.261 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:12.296 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:12 +0000 (0:00:00.045) 0:00:34.178 ***** 2020-12-05T17:26:12.302 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-12-05T17:26:12.304 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:12.340 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:12 +0000 (0:00:00.043) 0:00:34.222 ***** 2020-12-05T17:26:12.346 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-12-05T17:26:12.348 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:12.385 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:12 +0000 (0:00:00.044) 0:00:34.267 ***** 2020-12-05T17:26:12.399 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-12-05T17:26:12.401 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload ssh_config] ******************************************** ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2020-12-05T17:26:12.435 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:12 +0000 (0:00:00.050) 0:00:34.317 ***** 2020-12-05T17:26:12.577 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:12 +0000 (0:00:00.141) 0:00:34.458 ***** 2020-12-05T17:26:12.583 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-12-05T17:26:12.585 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set the system's hostname] ************************************ changed: [smithi032.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi032.front.sepia.ceph.com", "ansible_hostname": "smithi032", "ansible_nodename": "smithi032"}, "changed": true, "name": "smithi032"} 2020-12-05T17:26:12.621 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:12 +0000 (0:00:00.043) 0:00:34.502 ***** 2020-12-05T17:26:12.655 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:12 +0000 (0:00:00.033) 0:00:34.536 ***** 2020-12-05T17:26:12.661 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-12-05T17:26:12.663 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set a high max open files limit for the teuthology user.] ***** ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2020-12-05T17:26:12.701 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:12 +0000 (0:00:00.046) 0:00:34.583 ***** 2020-12-05T17:26:12.737 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:12 +0000 (0:00:00.035) 0:00:34.618 ***** 2020-12-05T17:26:12.745 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-12-05T17:26:12.746 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} TASK [testnode : Set hostname var] ********************************************* ok: [smithi036.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi036"}, "changed": false} 2020-12-05T17:26:12.782 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:12 +0000 (0:00:00.044) 0:00:34.663 ***** 2020-12-05T17:26:12.817 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:12 +0000 (0:00:00.035) 0:00:34.698 ***** 2020-12-05T17:26:12.825 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-12-05T17:26:12.827 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:12.872 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:12 +0000 (0:00:00.054) 0:00:34.753 ***** 2020-12-05T17:26:12.878 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-12-05T17:26:12.880 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-12-05T17:26:12.917 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:12 +0000 (0:00:00.045) 0:00:34.799 ***** 2020-12-05T17:26:12.923 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-12-05T17:26:12.925 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload distro major version specific sshd_config] ************* ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2020-12-05T17:26:12.962 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:12 +0000 (0:00:00.045) 0:00:34.844 ***** 2020-12-05T17:26:12.995 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:12 +0000 (0:00:00.032) 0:00:34.876 ***** 2020-12-05T17:26:13.002 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-12-05T17:26:13.004 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2020-12-05T17:26:13.040 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.044) 0:00:34.921 ***** 2020-12-05T17:26:13.047 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-12-05T17:26:13.049 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:13.084 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.043) 0:00:34.965 ***** 2020-12-05T17:26:13.090 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-12-05T17:26:13.092 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} TASK [testnode : Including version specific variables.] ************************ ok: [smithi061.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-12-05T17:26:13.131 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.046) 0:00:35.012 ***** 2020-12-05T17:26:13.164 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.033) 0:00:35.046 ***** 2020-12-05T17:26:13.169 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-12-05T17:26:13.171 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create remote.conf] ******************************************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:13.212 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.047) 0:00:35.093 ***** 2020-12-05T17:26:13.246 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.033) 0:00:35.127 ***** 2020-12-05T17:26:13.252 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:13.253 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-12-05T17:26:13.253 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi036.front.sepia.ceph.com", "ansible_hostname": "smithi036", "ansible_nodename": "smithi036"}, "changed": true, "name": "smithi036"} 2020-12-05T17:26:13.290 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.044) 0:00:35.172 ***** 2020-12-05T17:26:13.323 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.033) 0:00:35.205 ***** 2020-12-05T17:26:13.329 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-12-05T17:26:13.331 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} TASK [testnode : Ensure the sudo group exists.] ******************************** ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-12-05T17:26:13.332 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:13.388 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.065) 0:00:35.270 ***** 2020-12-05T17:26:13.423 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.034) 0:00:35.304 ***** 2020-12-05T17:26:13.456 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.033) 0:00:35.338 ***** 2020-12-05T17:26:13.463 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-12-05T17:26:13.465 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Including distro specific variables.] ************************* ok: [smithi053.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-12-05T17:26:13.505 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.048) 0:00:35.386 ***** 2020-12-05T17:26:13.546 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.041) 0:00:35.428 ***** 2020-12-05T17:26:13.582 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.035) 0:00:35.463 ***** 2020-12-05T17:26:13.588 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-12-05T17:26:13.590 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure the teuthology_user group exists.] ********************* ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "name": "ubuntu", "state": "present", "system": false} 2020-12-05T17:26:13.627 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.045) 0:00:35.508 ***** 2020-12-05T17:26:13.659 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.032) 0:00:35.540 ***** 2020-12-05T17:26:13.664 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-12-05T17:26:13.666 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2020-12-05T17:26:13.667 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-12-05T17:26:13.667 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2020-12-05T17:26:13.668 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:13.704 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.045) 0:00:35.586 ***** 2020-12-05T17:26:13.744 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.039) 0:00:35.626 ***** 2020-12-05T17:26:13.785 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.040) 0:00:35.666 ***** 2020-12-05T17:26:13.791 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-12-05T17:26:13.793 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:13.830 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.044) 0:00:35.711 ***** 2020-12-05T17:26:13.865 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.035) 0:00:35.747 ***** 2020-12-05T17:26:13.870 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-12-05T17:26:13.872 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} TASK [testnode : Configure local mirrorlists] ********************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:13.908 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.043) 0:00:35.790 ***** 2020-12-05T17:26:13.945 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.036) 0:00:35.826 ***** 2020-12-05T17:26:13.951 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-12-05T17:26:13.953 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-12-05T17:26:13.999 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:13 +0000 (0:00:00.054) 0:00:35.881 ***** 2020-12-05T17:26:14.034 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.034) 0:00:35.916 ***** 2020-12-05T17:26:14.042 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-12-05T17:26:14.080 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.045) 0:00:35.961 ***** 2020-12-05T17:26:14.086 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-12-05T17:26:14.088 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} TASK [testnode : Upload ssh_config] ******************************************** ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2020-12-05T17:26:14.088 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-12-05T17:26:14.126 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.046) 0:00:36.007 ***** 2020-12-05T17:26:14.160 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.034) 0:00:36.042 ***** 2020-12-05T17:26:14.195 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.035) 0:00:36.077 ***** 2020-12-05T17:26:14.202 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-12-05T17:26:14.203 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-12-05T17:26:14.204 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2020-12-05T17:26:14.243 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.048) 0:00:36.125 ***** 2020-12-05T17:26:14.279 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.035) 0:00:36.160 ***** 2020-12-05T17:26:14.312 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.033) 0:00:36.193 ***** 2020-12-05T17:26:14.317 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-12-05T17:26:14.319 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:14.358 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.046) 0:00:36.240 ***** 2020-12-05T17:26:14.397 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.038) 0:00:36.279 ***** 2020-12-05T17:26:14.403 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-12-05T17:26:14.404 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "name": "ubuntu", "state": "present", "system": false} TASK [testnode : Install GPG keys] ********************************************* skipping: [smithi002.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:14.405 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:14.441 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.044) 0:00:36.323 ***** 2020-12-05T17:26:14.475 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.033) 0:00:36.356 ***** 2020-12-05T17:26:14.480 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-12-05T17:26:14.482 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Clean yum cache] ********************************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:14.517 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.042) 0:00:36.398 ***** 2020-12-05T17:26:14.551 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.033) 0:00:36.432 ***** 2020-12-05T17:26:14.558 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-12-05T17:26:14.560 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:14.594 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.043) 0:00:36.476 ***** 2020-12-05T17:26:14.627 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.032) 0:00:36.509 ***** 2020-12-05T17:26:14.633 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-12-05T17:26:14.635 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure yum_utils is present.] ********************************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:14.670 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.043) 0:00:36.552 ***** 2020-12-05T17:26:14.702 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.032) 0:00:36.584 ***** 2020-12-05T17:26:14.708 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-12-05T17:26:14.710 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} TASK [testnode : Delete the teuthology users password.] ************************ skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Removing saved yum transactions] ****************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:14.749 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.046) 0:00:36.630 ***** 2020-12-05T17:26:14.781 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.032) 0:00:36.663 ***** 2020-12-05T17:26:14.815 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.033) 0:00:36.696 ***** 2020-12-05T17:26:14.821 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-12-05T17:26:14.822 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2020-12-05T17:26:14.824 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check if ceph-debuginfo is installed] ************************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:14.825 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:14.863 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.048) 0:00:36.745 ***** 2020-12-05T17:26:14.898 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.034) 0:00:36.780 ***** 2020-12-05T17:26:14.932 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.033) 0:00:36.813 ***** 2020-12-05T17:26:14.968 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:14 +0000 (0:00:00.035) 0:00:36.849 ***** 2020-12-05T17:26:14.982 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-12-05T17:26:14.984 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:14.985 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** ok: [smithi053.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2020-12-05T17:26:15.020 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.053) 0:00:36.902 ***** 2020-12-05T17:26:15.054 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.034) 0:00:36.936 ***** 2020-12-05T17:26:15.088 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.033) 0:00:36.970 ***** 2020-12-05T17:26:15.124 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.036) 0:00:37.006 ***** 2020-12-05T17:26:15.131 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-12-05T17:26:15.133 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi061"}, "changed": false} TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:15.133 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:15.172 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.047) 0:00:37.054 ***** 2020-12-05T17:26:15.208 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.035) 0:00:37.090 ***** 2020-12-05T17:26:15.263 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.054) 0:00:37.145 ***** 2020-12-05T17:26:15.301 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.038) 0:00:37.183 ***** 2020-12-05T17:26:15.307 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-12-05T17:26:15.309 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure local mirrorlists] ********************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:15.345 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.043) 0:00:37.227 ***** 2020-12-05T17:26:15.377 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.032) 0:00:37.259 ***** 2020-12-05T17:26:15.410 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.032) 0:00:37.292 ***** 2020-12-05T17:26:15.416 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-12-05T17:26:15.418 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-12-05T17:26:15.453 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.042) 0:00:37.335 ***** 2020-12-05T17:26:15.487 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.033) 0:00:37.369 ***** 2020-12-05T17:26:15.493 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-12-05T17:26:15.496 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install epel packages] **************************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:15.531 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.044) 0:00:37.413 ***** 2020-12-05T17:26:15.565 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.033) 0:00:37.447 ***** 2020-12-05T17:26:15.571 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-12-05T17:26:15.573 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-12-05T17:26:15.573 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:15.611 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.046) 0:00:37.493 ***** 2020-12-05T17:26:15.654 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.043) 0:00:37.536 ***** 2020-12-05T17:26:15.689 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.034) 0:00:37.570 ***** 2020-12-05T17:26:15.695 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-12-05T17:26:15.697 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable copr repos] ******************************************** 2020-12-05T17:26:15.733 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.044) 0:00:37.615 ***** 2020-12-05T17:26:15.768 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.034) 0:00:37.650 ***** 2020-12-05T17:26:15.803 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.034) 0:00:37.684 ***** 2020-12-05T17:26:15.809 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-12-05T17:26:15.811 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi053"}, "changed": false} TASK [testnode : Set the system's hostname] ************************************ changed: [smithi061.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi061.front.sepia.ceph.com", "ansible_hostname": "smithi061", "ansible_nodename": "smithi061"}, "changed": true, "name": "smithi061"} 2020-12-05T17:26:15.811 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:15.812 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:15.848 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.044) 0:00:37.729 ***** 2020-12-05T17:26:15.882 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.034) 0:00:37.764 ***** 2020-12-05T17:26:15.917 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.035) 0:00:37.799 ***** 2020-12-05T17:26:15.953 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.035) 0:00:37.834 ***** 2020-12-05T17:26:15.959 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-12-05T17:26:15.960 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Stop and disable iptables] ************************************ skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install GPG keys] ********************************************* skipping: [smithi032.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:15.961 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:16.002 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:15 +0000 (0:00:00.049) 0:00:37.884 ***** 2020-12-05T17:26:16.038 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:16 +0000 (0:00:00.036) 0:00:37.920 ***** 2020-12-05T17:26:16.087 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:16 +0000 (0:00:00.049) 0:00:37.969 ***** 2020-12-05T17:26:16.097 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-12-05T17:26:16.100 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Clean yum cache] ********************************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:16.137 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:16 +0000 (0:00:00.049) 0:00:38.018 ***** 2020-12-05T17:26:16.172 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:16 +0000 (0:00:00.035) 0:00:38.054 ***** 2020-12-05T17:26:16.180 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-12-05T17:26:16.182 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install abrt] ************************************************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:16.238 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:16 +0000 (0:00:00.065) 0:00:38.119 ***** 2020-12-05T17:26:16.282 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:16 +0000 (0:00:00.044) 0:00:38.164 ***** 2020-12-05T17:26:16.299 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:26:16.301 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload distro major version specific sshd_config] ************* ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2020-12-05T17:26:16.302 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:16.337 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:16 +0000 (0:00:00.055) 0:00:38.219 ***** 2020-12-05T17:26:16.372 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:16 +0000 (0:00:00.034) 0:00:38.254 ***** 2020-12-05T17:26:16.406 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:16 +0000 (0:00:00.034) 0:00:38.288 ***** 2020-12-05T17:26:16.413 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-12-05T17:26:16.414 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:16.414 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-12-05T17:26:16.415 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi053.front.sepia.ceph.com", "ansible_hostname": "smithi053", "ansible_nodename": "smithi053"}, "changed": true, "name": "smithi053"} 2020-12-05T17:26:16.415 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:16.457 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:16 +0000 (0:00:00.050) 0:00:38.339 ***** 2020-12-05T17:26:16.498 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:16 +0000 (0:00:00.040) 0:00:38.380 ***** 2020-12-05T17:26:16.536 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:16 +0000 (0:00:00.037) 0:00:38.417 ***** 2020-12-05T17:26:16.570 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:16 +0000 (0:00:00.033) 0:00:38.451 ***** 2020-12-05T17:26:16.576 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2020-12-05T17:26:16.578 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Removing saved yum transactions] ****************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-12-05T17:26:16.616 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:16 +0000 (0:00:00.045) 0:00:38.497 ***** 2020-12-05T17:26:16.651 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:16 +0000 (0:00:00.035) 0:00:38.532 ***** 2020-12-05T17:26:16.687 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:16 +0000 (0:00:00.036) 0:00:38.569 ***** 2020-12-05T17:26:16.695 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2020-12-05T17:26:16.697 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check if ceph-debuginfo is installed] ************************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:16.733 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:16 +0000 (0:00:00.046) 0:00:38.615 ***** 2020-12-05T17:26:16.769 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:16 +0000 (0:00:00.035) 0:00:38.651 ***** 2020-12-05T17:26:16.776 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-12-05T17:26:16.777 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} TASK [testnode : Enable copr repos] ******************************************** 2020-12-05T17:26:16.819 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:16 +0000 (0:00:00.050) 0:00:38.701 ***** 2020-12-05T17:26:16.855 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:16 +0000 (0:00:00.035) 0:00:38.736 ***** 2020-12-05T17:26:16.861 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-12-05T17:26:16.865 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install GPG] ************************************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:16.866 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.004179", "end": "2020-12-05 17:26:16.844022", "rc": 0, "start": "2020-12-05 17:26:16.839843", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:26:16.910 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:16 +0000 (0:00:00.055) 0:00:38.792 ***** 2020-12-05T17:26:16.943 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:16 +0000 (0:00:00.032) 0:00:38.825 ***** 2020-12-05T17:26:16.976 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:16 +0000 (0:00:00.032) 0:00:38.858 ***** 2020-12-05T17:26:16.983 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-12-05T17:26:16.986 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} TASK [testnode : Remove custom repos] ****************************************** TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:17.024 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.047) 0:00:38.905 ***** 2020-12-05T17:26:17.058 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.033) 0:00:38.939 ***** 2020-12-05T17:26:17.093 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.034) 0:00:38.974 ***** 2020-12-05T17:26:17.099 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-12-05T17:26:17.101 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-12-05T17:26:17.102 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:17.140 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.047) 0:00:39.021 ***** 2020-12-05T17:26:17.176 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.035) 0:00:39.057 ***** 2020-12-05T17:26:17.183 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-12-05T17:26:17.185 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:17.236 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.060) 0:00:39.118 ***** 2020-12-05T17:26:17.245 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-12-05T17:26:17.246 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:17.283 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.047) 0:00:39.165 ***** 2020-12-05T17:26:17.289 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-12-05T17:26:17.291 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:17.328 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.044) 0:00:39.209 ***** 2020-12-05T17:26:17.336 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:26:17.338 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:17.376 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.048) 0:00:39.258 ***** 2020-12-05T17:26:17.384 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-12-05T17:26:17.386 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} TASK [testnode : Remove packages] ********************************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add ssh pubkeys] ********************************************** 2020-12-05T17:26:17.386 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-12-05T17:26:17.428 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.051) 0:00:39.310 ***** 2020-12-05T17:26:17.465 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.036) 0:00:39.346 ***** 2020-12-05T17:26:17.506 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.041) 0:00:39.387 ***** 2020-12-05T17:26:17.511 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-12-05T17:26:17.513 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload ssh_config] ******************************************** ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2020-12-05T17:26:17.514 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:17.515 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:17.550 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.044) 0:00:39.432 ***** 2020-12-05T17:26:17.581 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.031) 0:00:39.463 ***** 2020-12-05T17:26:17.613 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.031) 0:00:39.494 ***** 2020-12-05T17:26:17.644 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.030) 0:00:39.525 ***** 2020-12-05T17:26:17.649 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:17.651 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Removing saved yum transactions] ****************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:17.685 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.042) 0:00:39.567 ***** 2020-12-05T17:26:17.718 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.032) 0:00:39.600 ***** 2020-12-05T17:26:17.751 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.032) 0:00:39.632 ***** 2020-12-05T17:26:17.757 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-12-05T17:26:17.759 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check if ceph-debuginfo is installed] ************************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:17.797 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.046) 0:00:39.679 ***** 2020-12-05T17:26:17.830 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.033) 0:00:39.712 ***** 2020-12-05T17:26:17.839 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-12-05T17:26:17.841 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable SELinux] *********************************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:17.889 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.058) 0:00:39.771 ***** 2020-12-05T17:26:17.934 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.044) 0:00:39.816 ***** 2020-12-05T17:26:17.943 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-12-05T17:26:17.945 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install abrt] ************************************************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:17.982 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:17 +0000 (0:00:00.048) 0:00:39.864 ***** 2020-12-05T17:26:18.016 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.033) 0:00:39.897 ***** 2020-12-05T17:26:18.023 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable abrt-auto-reporting] *********************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:18.062 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.045) 0:00:39.943 ***** 2020-12-05T17:26:18.099 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.037) 0:00:39.981 ***** 2020-12-05T17:26:18.108 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-12-05T17:26:18.109 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:18.109 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2020-12-05T17:26:18.109 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:18.159 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.059) 0:00:40.040 ***** 2020-12-05T17:26:18.193 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.034) 0:00:40.075 ***** 2020-12-05T17:26:18.201 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-12-05T17:26:18.203 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:18.242 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.048) 0:00:40.123 ***** 2020-12-05T17:26:18.290 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.048) 0:00:40.172 ***** 2020-12-05T17:26:18.326 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.035) 0:00:40.207 ***** 2020-12-05T17:26:18.332 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-12-05T17:26:18.334 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Restart abrtd] ************************************************ skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create remote.conf] ******************************************* skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:18.370 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.044) 0:00:40.252 ***** 2020-12-05T17:26:18.404 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.033) 0:00:40.285 ***** 2020-12-05T17:26:18.438 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.033) 0:00:40.319 ***** 2020-12-05T17:26:18.444 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-12-05T17:26:18.446 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install packages] ********************************************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:18.483 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.045) 0:00:40.365 ***** 2020-12-05T17:26:18.518 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.034) 0:00:40.399 ***** 2020-12-05T17:26:18.523 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-12-05T17:26:18.525 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for custom repos] *************************************** ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005699", "end": "2020-12-05 17:26:18.705838", "rc": 0, "start": "2020-12-05 17:26:18.700139", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:26:18.525 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:18.562 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.043) 0:00:40.443 ***** 2020-12-05T17:26:18.594 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.032) 0:00:40.476 ***** 2020-12-05T17:26:18.626 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.032) 0:00:40.508 ***** 2020-12-05T17:26:18.633 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-12-05T17:26:18.635 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove custom repos] ****************************************** TASK [testnode : Remove packages] ********************************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:18.669 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.043) 0:00:40.551 ***** 2020-12-05T17:26:18.703 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.032) 0:00:40.584 ***** 2020-12-05T17:26:18.742 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.039) 0:00:40.623 ***** 2020-12-05T17:26:18.747 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-12-05T17:26:18.749 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:18.785 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.042) 0:00:40.666 ***** 2020-12-05T17:26:18.818 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.033) 0:00:40.699 ***** 2020-12-05T17:26:18.825 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-12-05T17:26:18.827 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Stop and disable firewalld] *********************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:18.862 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.044) 0:00:40.744 ***** 2020-12-05T17:26:18.897 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.034) 0:00:40.778 ***** 2020-12-05T17:26:18.904 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-12-05T17:26:18.906 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Stop and disable iptables] ************************************ skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:18.941 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.044) 0:00:40.823 ***** 2020-12-05T17:26:18.973 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:18 +0000 (0:00:00.031) 0:00:40.854 ***** 2020-12-05T17:26:18.978 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-12-05T17:26:18.981 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install apt keys] ********************************************* ok: [smithi002.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} TASK [testnode : Enable SELinux] *********************************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:19.016 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.042) 0:00:40.897 ***** 2020-12-05T17:26:19.047 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.031) 0:00:40.928 ***** 2020-12-05T17:26:19.053 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-12-05T17:26:19.054 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set apt preferences] ****************************************** 2020-12-05T17:26:19.055 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-12-05T17:26:19.055 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:19.092 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.045) 0:00:40.974 ***** 2020-12-05T17:26:19.126 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.033) 0:00:41.008 ***** 2020-12-05T17:26:19.159 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.032) 0:00:41.040 ***** 2020-12-05T17:26:19.166 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-12-05T17:26:19.168 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable abrt-auto-reporting] *********************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:19.202 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.043) 0:00:41.084 ***** 2020-12-05T17:26:19.236 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.033) 0:00:41.117 ***** 2020-12-05T17:26:19.274 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.037) 0:00:41.155 ***** 2020-12-05T17:26:19.280 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-12-05T17:26:19.282 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:19.317 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.042) 0:00:41.198 ***** 2020-12-05T17:26:19.350 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.033) 0:00:41.232 ***** 2020-12-05T17:26:19.356 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-12-05T17:26:19.358 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:19.392 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.042) 0:00:41.274 ***** 2020-12-05T17:26:19.425 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.032) 0:00:41.307 ***** 2020-12-05T17:26:19.433 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-12-05T17:26:19.434 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Restart abrtd] ************************************************ skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:19.469 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.043) 0:00:41.350 ***** 2020-12-05T17:26:19.503 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.034) 0:00:41.385 ***** 2020-12-05T17:26:19.510 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-12-05T17:26:19.552 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.048) 0:00:41.433 ***** 2020-12-05T17:26:19.558 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-12-05T17:26:19.560 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:19.599 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.046) 0:00:41.479 ***** 2020-12-05T17:26:19.605 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-12-05T17:26:19.606 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:19.642 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.043) 0:00:41.523 ***** 2020-12-05T17:26:19.648 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-12-05T17:26:19.651 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005174", "end": "2020-12-05 17:26:19.929720", "rc": 0, "start": "2020-12-05 17:26:19.924546", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-12-05T17:26:19.687 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.045) 0:00:41.568 ***** 2020-12-05T17:26:19.720 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.033) 0:00:41.602 ***** 2020-12-05T17:26:19.727 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* skipping: [smithi061.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:19.728 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} TASK [testnode : Remove custom repos] ****************************************** 2020-12-05T17:26:19.764 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.043) 0:00:41.645 ***** 2020-12-05T17:26:19.798 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.033) 0:00:41.679 ***** 2020-12-05T17:26:19.804 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-12-05T17:26:19.854 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.056) 0:00:41.736 ***** 2020-12-05T17:26:19.860 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-12-05T17:26:19.862 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:19.907 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.053) 0:00:41.789 ***** 2020-12-05T17:26:19.913 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-12-05T17:26:19.915 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* ok: [smithi002.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-12-05T17:26:19.952 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.044) 0:00:41.833 ***** 2020-12-05T17:26:19.986 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:19 +0000 (0:00:00.033) 0:00:41.867 ***** 2020-12-05T17:26:19.993 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:20.030 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:20 +0000 (0:00:00.044) 0:00:41.912 ***** 2020-12-05T17:26:20.036 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-12-05T17:26:20.037 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:20.080 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:20 +0000 (0:00:00.049) 0:00:41.961 ***** 2020-12-05T17:26:20.088 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-12-05T17:26:20.090 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set apt preferences] ****************************************** ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-12-05T17:26:20.139 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:20 +0000 (0:00:00.059) 0:00:42.020 ***** 2020-12-05T17:26:20.175 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:20 +0000 (0:00:00.035) 0:00:42.056 ***** 2020-12-05T17:26:20.181 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-12-05T17:26:20.183 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:20.184 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:20.221 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:20 +0000 (0:00:00.045) 0:00:42.102 ***** 2020-12-05T17:26:20.256 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:20 +0000 (0:00:00.035) 0:00:42.138 ***** 2020-12-05T17:26:20.264 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-12-05T17:26:20.266 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install apt keys] ********************************************* ok: [smithi032.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2020-12-05T17:26:20.303 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:20 +0000 (0:00:00.046) 0:00:42.185 ***** 2020-12-05T17:26:20.310 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-12-05T17:26:20.311 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:20.364 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:20 +0000 (0:00:00.060) 0:00:42.245 ***** 2020-12-05T17:26:20.373 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-12-05T17:26:20.374 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:20.418 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:20 +0000 (0:00:00.054) 0:00:42.300 ***** 2020-12-05T17:26:20.424 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:26:20.425 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:20.461 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:20 +0000 (0:00:00.043) 0:00:42.343 ***** 2020-12-05T17:26:20.467 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-12-05T17:26:20.469 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:20.508 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:20 +0000 (0:00:00.047) 0:00:42.390 ***** 2020-12-05T17:26:20.514 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-12-05T17:26:20.516 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:20.552 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:20 +0000 (0:00:00.043) 0:00:42.434 ***** 2020-12-05T17:26:20.559 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-12-05T17:26:20.561 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:20.601 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:20 +0000 (0:00:00.049) 0:00:42.483 ***** 2020-12-05T17:26:20.617 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-12-05T17:26:20.619 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:20.652 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:20 +0000 (0:00:00.050) 0:00:42.534 ***** 2020-12-05T17:26:20.659 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-12-05T17:26:20.660 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:20.698 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:20 +0000 (0:00:00.045) 0:00:42.580 ***** 2020-12-05T17:26:20.704 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-12-05T17:26:20.706 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:20.744 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:20 +0000 (0:00:00.045) 0:00:42.626 ***** 2020-12-05T17:26:20.750 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-12-05T17:26:20.752 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:20.788 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:20 +0000 (0:00:00.044) 0:00:42.670 ***** 2020-12-05T17:26:20.795 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-12-05T17:26:20.796 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:20.832 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:20 +0000 (0:00:00.043) 0:00:42.714 ***** 2020-12-05T17:26:20.838 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-12-05T17:26:20.840 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:20.875 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:20 +0000 (0:00:00.043) 0:00:42.757 ***** 2020-12-05T17:26:20.881 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-12-05T17:26:20.883 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install python-apt] ******************************************* ok: [smithi002.front.sepia.ceph.com] => {"cache_update_time": 1607188915, "cache_updated": false, "changed": false} 2020-12-05T17:26:20.922 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:20 +0000 (0:00:00.046) 0:00:42.803 ***** 2020-12-05T17:26:20.957 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:20 +0000 (0:00:00.035) 0:00:42.838 ***** 2020-12-05T17:26:20.965 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:20.967 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-12-05T17:26:21.001 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:20 +0000 (0:00:00.044) 0:00:42.883 ***** 2020-12-05T17:26:21.035 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:21 +0000 (0:00:00.033) 0:00:42.916 ***** 2020-12-05T17:26:21.041 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-12-05T17:26:21.043 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:21.080 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:21 +0000 (0:00:00.045) 0:00:42.961 ***** 2020-12-05T17:26:21.086 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-12-05T17:26:21.088 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:21.126 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:21 +0000 (0:00:00.046) 0:00:43.008 ***** 2020-12-05T17:26:21.133 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-12-05T17:26:21.134 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:21.171 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:21 +0000 (0:00:00.045) 0:00:43.053 ***** 2020-12-05T17:26:21.178 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-12-05T17:26:21.179 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Blast the apt cache.] ***************************************** [WARNING]: Consider using 'become', 'become_method', and 'become_user' rather than running sudo changed: [smithi002.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.033782", "end": "2020-12-05 17:26:21.144968", "rc": 0, "start": "2020-12-05 17:26:21.111186", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:26:21.181 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* ok: [smithi032.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-12-05T17:26:21.225 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:21 +0000 (0:00:00.053) 0:00:43.107 ***** 2020-12-05T17:26:21.259 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:21 +0000 (0:00:00.034) 0:00:43.141 ***** 2020-12-05T17:26:21.296 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:21 +0000 (0:00:00.036) 0:00:43.177 ***** 2020-12-05T17:26:21.302 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-12-05T17:26:21.304 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install apt keys] ********************************************* ok: [smithi036.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2020-12-05T17:26:21.341 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:21 +0000 (0:00:00.045) 0:00:43.223 ***** 2020-12-05T17:26:21.348 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-12-05T17:26:21.349 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:21.385 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:21 +0000 (0:00:00.044) 0:00:43.267 ***** 2020-12-05T17:26:21.392 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-12-05T17:26:21.394 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:21.430 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:21 +0000 (0:00:00.044) 0:00:43.312 ***** 2020-12-05T17:26:21.436 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-12-05T17:26:21.437 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:21.474 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:21 +0000 (0:00:00.043) 0:00:43.355 ***** 2020-12-05T17:26:21.480 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-12-05T17:26:21.481 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:21.518 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:21 +0000 (0:00:00.044) 0:00:43.400 ***** 2020-12-05T17:26:21.525 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2020-12-05T17:26:21.526 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:21.563 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:21 +0000 (0:00:00.044) 0:00:43.444 ***** 2020-12-05T17:26:21.570 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-12-05T17:26:21.572 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:21.606 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:21 +0000 (0:00:00.043) 0:00:43.488 ***** 2020-12-05T17:26:21.612 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2020-12-05T17:26:21.614 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:21.651 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:21 +0000 (0:00:00.044) 0:00:43.532 ***** 2020-12-05T17:26:21.657 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-12-05T17:26:21.658 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:21.695 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:21 +0000 (0:00:00.044) 0:00:43.576 ***** 2020-12-05T17:26:21.701 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2020-12-05T17:26:21.703 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:21.739 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:21 +0000 (0:00:00.044) 0:00:43.621 ***** 2020-12-05T17:26:21.745 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:21.784 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:21 +0000 (0:00:00.044) 0:00:43.665 ***** 2020-12-05T17:26:21.790 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2020-12-05T17:26:21.792 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:21.827 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:21 +0000 (0:00:00.043) 0:00:43.709 ***** 2020-12-05T17:26:21.833 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2020-12-05T17:26:21.835 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:21.869 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:21 +0000 (0:00:00.041) 0:00:43.751 ***** 2020-12-05T17:26:21.884 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2020-12-05T17:26:21.886 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:21.919 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:21 +0000 (0:00:00.049) 0:00:43.800 ***** 2020-12-05T17:26:21.925 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2020-12-05T17:26:21.926 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:21.962 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:21 +0000 (0:00:00.043) 0:00:43.844 ***** 2020-12-05T17:26:21.989 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2020-12-05T17:26:21.991 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:22.025 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:22 +0000 (0:00:00.062) 0:00:43.907 ***** 2020-12-05T17:26:22.052 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2020-12-05T17:26:22.054 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for custom repos] *************************************** ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.003624", "end": "2020-12-05 17:26:22.212406", "rc": 0, "start": "2020-12-05 17:26:22.208782", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:26:22.089 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:22 +0000 (0:00:00.063) 0:00:43.970 ***** 2020-12-05T17:26:22.122 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:22 +0000 (0:00:00.033) 0:00:44.003 ***** 2020-12-05T17:26:22.128 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2020-12-05T17:26:22.129 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:22.165 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:22 +0000 (0:00:00.043) 0:00:44.047 ***** 2020-12-05T17:26:22.174 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** TASK [testnode : Install apt keys] ********************************************* ok: [smithi036.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-12-05T17:26:22.210 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:22 +0000 (0:00:00.045) 0:00:44.092 ***** 2020-12-05T17:26:22.245 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:22 +0000 (0:00:00.034) 0:00:44.127 ***** 2020-12-05T17:26:22.253 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-12-05T17:26:22.255 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"cache_update_time": 1607188779, "cache_updated": false, "changed": false} 2020-12-05T17:26:22.292 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:22 +0000 (0:00:00.045) 0:00:44.173 ***** 2020-12-05T17:26:22.411 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005263", "end": "2020-12-05 17:26:22.332387", "rc": 0, "start": "2020-12-05 17:26:22.327124", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} Saturday 05 December 2020 17:26:22 +0000 (0:00:00.066) 0:00:44.240 ***** TASK [testnode : Add local apt repos.] ***************************************** 2020-12-05T17:26:22.418 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:22 +0000 (0:00:00.059) 0:00:44.299 ***** 2020-12-05T17:26:22.427 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-12-05T17:26:22.487 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:22 +0000 (0:00:00.069) 0:00:44.369 ***** 2020-12-05T17:26:22.590 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-12-05T17:26:22.592 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-12-05T17:26:22.640 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:22 +0000 (0:00:00.153) 0:00:44.522 ***** 2020-12-05T17:26:22.649 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-12-05T17:26:22.650 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.033503", "end": "2020-12-05 17:26:22.797763", "rc": 0, "start": "2020-12-05 17:26:22.764260", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:26:22.705 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:22 +0000 (0:00:00.064) 0:00:44.586 ***** 2020-12-05T17:26:22.712 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-12-05T17:26:22.714 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:22.756 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:22 +0000 (0:00:00.050) 0:00:44.637 ***** 2020-12-05T17:26:22.860 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-12-05T17:26:22.862 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-12-05T17:26:22.901 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:22 +0000 (0:00:00.145) 0:00:44.783 ***** 2020-12-05T17:26:22.930 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-12-05T17:26:22.932 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:22.967 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:22 +0000 (0:00:00.065) 0:00:44.848 ***** 2020-12-05T17:26:23.155 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-12-05T17:26:23.157 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"cache_update_time": 1607189016, "cache_updated": false, "changed": false} 2020-12-05T17:26:23.214 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:23 +0000 (0:00:00.246) 0:00:45.095 ***** 2020-12-05T17:26:23.242 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-12-05T17:26:23.277 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:23 +0000 (0:00:00.063) 0:00:45.159 ***** 2020-12-05T17:26:23.445 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-12-05T17:26:23.447 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.037911", "end": "2020-12-05 17:26:23.770699", "rc": 0, "start": "2020-12-05 17:26:23.732788", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:26:23.481 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:23 +0000 (0:00:00.203) 0:00:45.363 ***** 2020-12-05T17:26:23.764 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* ok: [smithi061.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2020-12-05T17:26:23.990 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2020-12-05T17:26:24.119 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-12-05T17:26:24.121 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"cache_update_time": 1607188915, "cache_updated": false, "changed": false} 2020-12-05T17:26:24.157 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:24 +0000 (0:00:00.676) 0:00:46.039 ***** 2020-12-05T17:26:24.734 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-12-05T17:26:24.736 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-12-05T17:26:24.772 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:24 +0000 (0:00:00.614) 0:00:46.653 ***** 2020-12-05T17:26:24.942 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-12-05T17:26:24.978 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:24 +0000 (0:00:00.206) 0:00:46.860 ***** 2020-12-05T17:26:25.697 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* ok: [smithi061.front.sepia.ceph.com] => {"cache_update_time": 1607189009, "cache_updated": false, "changed": false} 2020-12-05T17:26:25.734 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:25 +0000 (0:00:00.755) 0:00:47.615 ***** 2020-12-05T17:26:25.761 INFO:teuthology.task.ansible.out: 2020-12-05T17:26:25.763 INFO:teuthology.task.ansible.out:TASK [testnode : Add local apt repos.] ***************************************** 2020-12-05T17:26:25.799 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:25 +0000 (0:00:00.064) 0:00:47.680 ***** 2020-12-05T17:26:25.826 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-12-05T17:26:25.828 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"cache_update_time": 1607188779, "cache_updated": false, "changed": false} 2020-12-05T17:26:25.869 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:25 +0000 (0:00:00.070) 0:00:47.751 ***** 2020-12-05T17:26:25.910 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-12-05T17:26:25.912 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"cache_update_time": 1607188959, "cache_updated": false, "changed": false} 2020-12-05T17:26:25.948 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:25 +0000 (0:00:00.078) 0:00:47.830 ***** 2020-12-05T17:26:25.975 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-12-05T17:26:25.977 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.035653", "end": "2020-12-05 17:26:26.129917", "rc": 0, "start": "2020-12-05 17:26:26.094264", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [testnode : Add local apt repos.] ***************************************** 2020-12-05T17:26:26.011 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:26 +0000 (0:00:00.063) 0:00:47.893 ***** 2020-12-05T17:26:26.045 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:26 +0000 (0:00:00.033) 0:00:47.927 ***** 2020-12-05T17:26:26.179 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-12-05T17:26:26.181 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.038436", "end": "2020-12-05 17:26:26.209691", "rc": 0, "start": "2020-12-05 17:26:26.171255", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:26:26.218 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:26 +0000 (0:00:00.173) 0:00:48.100 ***** 2020-12-05T17:26:26.489 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-12-05T17:26:26.491 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"cache_update_time": 1607189016, "cache_updated": false, "changed": false} 2020-12-05T17:26:26.534 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:26 +0000 (0:00:00.315) 0:00:48.416 ***** 2020-12-05T17:26:28.950 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"cache_update_time": 1607189009, "cache_updated": false, "changed": false} 2020-12-05T17:26:28.985 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:28 +0000 (0:00:02.450) 0:00:50.867 ***** 2020-12-05T17:26:29.306 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"cache_update_time": 1607188959, "cache_updated": false, "changed": false} 2020-12-05T17:26:29.343 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:29 +0000 (0:00:00.357) 0:00:51.224 ***** 2020-12-05T17:26:33.756 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-12-05T17:26:33.758 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 3 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137608 files and directories 2020-12-05T17:26:33.759 INFO:teuthology.task.ansible.out: currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 3 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Readin 2020-12-05T17:26:33.761 INFO:teuthology.task.ansible.out:g database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137608 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-12-05T17:26:33.792 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:33 +0000 (0:00:04.449) 0:00:55.674 ***** 2020-12-05T17:26:34.539 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 3 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137608 files and directories 2020-12-05T17:26:34.541 INFO:teuthology.task.ansible.out: currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 3 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Readin 2020-12-05T17:26:34.542 INFO:teuthology.task.ansible.out:g database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137608 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-12-05T17:26:34.579 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:34 +0000 (0:00:00.785) 0:00:56.460 ***** 2020-12-05T17:26:34.701 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-12-05T17:26:34.703 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"cache_update_time": 1607188915, "cache_updated": false, "changed": false} 2020-12-05T17:26:34.739 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:34 +0000 (0:00:00.160) 0:00:56.620 ***** 2020-12-05T17:26:34.767 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-12-05T17:26:34.768 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:34.814 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:34 +0000 (0:00:00.075) 0:00:56.696 ***** 2020-12-05T17:26:35.326 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-12-05T17:26:35.328 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 3 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137608 files and directories 2020-12-05T17:26:35.329 INFO:teuthology.task.ansible.out: currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 3 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Readin 2020-12-05T17:26:35.331 INFO:teuthology.task.ansible.out:g database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137608 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-12-05T17:26:35.363 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:35 +0000 (0:00:00.549) 0:00:57.245 ***** 2020-12-05T17:26:35.485 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-12-05T17:26:35.487 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"cache_update_time": 1607188779, "cache_updated": false, "changed": false} 2020-12-05T17:26:35.521 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:35 +0000 (0:00:00.158) 0:00:57.403 ***** 2020-12-05T17:26:35.549 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-12-05T17:26:35.551 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:35.584 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:35 +0000 (0:00:00.062) 0:00:57.466 ***** 2020-12-05T17:26:36.289 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-12-05T17:26:36.291 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"cache_update_time": 1607189016, "cache_updated": false, "changed": false} 2020-12-05T17:26:36.326 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:36 +0000 (0:00:00.741) 0:00:58.207 ***** 2020-12-05T17:26:36.353 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-12-05T17:26:36.354 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:36.388 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:36 +0000 (0:00:00.061) 0:00:58.269 ***** 2020-12-05T17:26:37.422 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-12-05T17:26:37.424 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 3 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137608 files and directories 2020-12-05T17:26:37.426 INFO:teuthology.task.ansible.out: currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 3 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Readin 2020-12-05T17:26:37.427 INFO:teuthology.task.ansible.out:g database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137608 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-12-05T17:26:37.463 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:37 +0000 (0:00:01.075) 0:00:59.345 ***** 2020-12-05T17:26:37.851 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 3 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137608 files and directories 2020-12-05T17:26:37.852 INFO:teuthology.task.ansible.out: currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 3 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Readin 2020-12-05T17:26:37.854 INFO:teuthology.task.ansible.out:g database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137608 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-12-05T17:26:37.887 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:37 +0000 (0:00:00.424) 0:00:59.769 ***** 2020-12-05T17:26:38.345 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-12-05T17:26:38.347 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"cache_update_time": 1607189009, "cache_updated": false, "changed": false} 2020-12-05T17:26:38.383 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:38 +0000 (0:00:00.495) 0:01:00.265 ***** 2020-12-05T17:26:38.411 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-12-05T17:26:38.413 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:38.448 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:38 +0000 (0:00:00.064) 0:01:00.329 ***** 2020-12-05T17:26:38.879 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-12-05T17:26:38.881 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"cache_update_time": 1607188959, "cache_updated": false, "changed": false} 2020-12-05T17:26:38.917 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:38 +0000 (0:00:00.469) 0:01:00.799 ***** 2020-12-05T17:26:38.944 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-12-05T17:26:38.946 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:38.982 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:38 +0000 (0:00:00.064) 0:01:00.864 ***** 2020-12-05T17:26:44.266 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-12-05T17:26:44.267 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => {"cache_update_time": 1607188779, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 3 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.34 [40.2 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.34 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.34 [5188 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.34 [871 kB]\nFetched 10.4 MB in 1s (9832 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 2020-12-05T17:26:44.268 INFO:teuthology.task.ansible.out: 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137466 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu 2020-12-05T17:26:44.270 INFO:teuthology.task.ansible.out:1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 3 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.34 [40.2 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.34 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.34 [5188 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.34 [871 kB]", "Fetched 10.4 MB in 1s (9832 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Rea 2020-12-05T17:26:44.271 INFO:teuthology.task.ansible.out:ding database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137466 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-12-05T17:26:44.321 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:44 +0000 (0:00:05.339) 0:01:06.203 ***** 2020-12-05T17:26:44.352 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => {"cache_update_time": 1607188915, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 3 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.34 [40.2 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.34 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.34 [5188 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.34 [871 kB]\nFetched 10.4 MB in 1s (9900 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 2020-12-05T17:26:44.354 INFO:teuthology.task.ansible.out: 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137466 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu 2020-12-05T17:26:44.356 INFO:teuthology.task.ansible.out:1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 3 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.34 [40.2 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.34 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.34 [5188 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.34 [871 kB]", "Fetched 10.4 MB in 1s (9900 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Rea 2020-12-05T17:26:44.358 INFO:teuthology.task.ansible.out:ding database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137466 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-12-05T17:26:44.391 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:44 +0000 (0:00:00.069) 0:01:06.272 ***** 2020-12-05T17:26:45.104 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => {"cache_update_time": 1607189016, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 3 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.34 [40.2 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.34 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.34 [5188 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.34 [871 kB]\nFetched 10.4 MB in 1s (9542 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 2020-12-05T17:26:45.106 INFO:teuthology.task.ansible.out: 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137466 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu 2020-12-05T17:26:45.108 INFO:teuthology.task.ansible.out:1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 3 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.34 [40.2 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.34 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.34 [5188 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.34 [871 kB]", "Fetched 10.4 MB in 1s (9542 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Rea 2020-12-05T17:26:45.110 INFO:teuthology.task.ansible.out:ding database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137466 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-12-05T17:26:45.145 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:45 +0000 (0:00:00.754) 0:01:07.027 ***** 2020-12-05T17:26:45.300 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-12-05T17:26:45.302 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"cache_update_time": 1607188779, "cache_updated": false, "changed": false} 2020-12-05T17:26:45.338 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:45 +0000 (0:00:00.192) 0:01:07.219 ***** 2020-12-05T17:26:45.345 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"cache_update_time": 1607188915, "cache_updated": false, "changed": false} 2020-12-05T17:26:45.381 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:45 +0000 (0:00:00.043) 0:01:07.263 ***** 2020-12-05T17:26:45.463 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-12-05T17:26:45.465 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2020-12-05T17:26:45.500 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:45 +0000 (0:00:00.118) 0:01:07.382 ***** 2020-12-05T17:26:45.507 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2020-12-05T17:26:45.543 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:45 +0000 (0:00:00.043) 0:01:07.425 ***** 2020-12-05T17:26:45.551 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-12-05T17:26:45.552 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:45.587 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:45 +0000 (0:00:00.043) 0:01:07.469 ***** 2020-12-05T17:26:45.594 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:45.630 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:45 +0000 (0:00:00.043) 0:01:07.512 ***** 2020-12-05T17:26:45.732 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-12-05T17:26:45.734 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2020-12-05T17:26:45.769 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:45 +0000 (0:00:00.139) 0:01:07.651 ***** 2020-12-05T17:26:45.776 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2020-12-05T17:26:45.814 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:45 +0000 (0:00:00.044) 0:01:07.695 ***** 2020-12-05T17:26:46.073 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-12-05T17:26:46.075 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"cache_update_time": 1607189016, "cache_updated": false, "changed": false} 2020-12-05T17:26:46.115 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:46 +0000 (0:00:00.300) 0:01:07.996 ***** 2020-12-05T17:26:46.121 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-12-05T17:26:46.126 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-12-05T17:26:46.160 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:46 +0000 (0:00:00.044) 0:01:08.041 ***** 2020-12-05T17:26:46.167 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-12-05T17:26:46.217 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:46 +0000 (0:00:00.058) 0:01:08.099 ***** 2020-12-05T17:26:46.246 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-12-05T17:26:46.248 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2020-12-05T17:26:46.290 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:46 +0000 (0:00:00.072) 0:01:08.172 ***** 2020-12-05T17:26:46.330 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-12-05T17:26:46.332 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ ok: [smithi032.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1235, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-12-05T17:26:46.377 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:46 +0000 (0:00:00.086) 0:01:08.258 ***** 2020-12-05T17:26:46.412 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:46 +0000 (0:00:00.035) 0:01:08.293 ***** 2020-12-05T17:26:46.422 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1235, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-12-05T17:26:46.460 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:46 +0000 (0:00:00.048) 0:01:08.342 ***** 2020-12-05T17:26:46.533 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-12-05T17:26:46.534 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-12-05T17:26:46.574 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:46 +0000 (0:00:00.113) 0:01:08.455 ***** 2020-12-05T17:26:46.581 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-12-05T17:26:46.582 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2020-12-05T17:26:46.618 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:46 +0000 (0:00:00.044) 0:01:08.500 ***** 2020-12-05T17:26:46.625 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-12-05T17:26:46.627 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-12-05T17:26:46.671 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:46 +0000 (0:00:00.052) 0:01:08.553 ***** 2020-12-05T17:26:46.870 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-12-05T17:26:46.871 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sat 2020-12-05 17:25:04 UTC", "ActiveEnterTimestampMonotonic": "21409340", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "nss-lookup.target systemd-journald.socket systemd-tmpfiles-setup.service network.target system.slice basic.target remote-fs.target -.mount sysinit.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-12-05 17:25:03 UTC", "AssertTimestampMonotonic": "20977232", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-12-05 17:25:03 UTC", "ConditionTimestampMonotonic": "20977231", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2020-12-05T17:26:46.873 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1565", "ExecMainStartTimestamp": "Sat 2020-12-05 17:25:04 UTC", "ExecMainStartTimestampMonotonic": "21409310", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-12-05 17:25:03 UTC", "InactiveExitTimestampMonotonic": "20979718", "InvocationID": "a30ece0b8faa49a2a6f03bfbf70e0308", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2020-12-05T17:26:46.875 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128053", "LimitNPROCSoft": "128053", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128053", "LimitSIGPENDINGSoft": "128053", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1565", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2020-12-05T17:26:46.876 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-12-05 17:25:04 UTC", "StateChangeTimestampMonotonic": "21409340", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-12-05 17:25:04 UTC", "WatchdogTimestampMonotonic": "21409338", "WatchdogUSec": "0"}} 2020-12-05T17:26:46.909 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:46 +0000 (0:00:00.237) 0:01:08.790 ***** 2020-12-05T17:26:46.936 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-12-05T17:26:46.938 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload /etc/fuse.conf.] *************************************** ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-12-05T17:26:46.975 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:46 +0000 (0:00:00.066) 0:01:08.857 ***** 2020-12-05T17:26:47.011 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.035) 0:01:08.893 ***** 2020-12-05T17:26:47.018 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-12-05T17:26:47.020 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sat 2020-12-05 17:25:19 UTC", "ActiveEnterTimestampMonotonic": "20187984", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target nss-lookup.target network.target system.slice systemd-journald.socket remote-fs.target systemd-tmpfiles-setup.service -.mount sysinit.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-12-05 17:25:19 UTC", "AssertTimestampMonotonic": "19899139", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-12-05 17:25:19 UTC", "ConditionTimestampMonotonic": "19899139", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2020-12-05T17:26:47.021 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1441", "ExecMainStartTimestamp": "Sat 2020-12-05 17:25:19 UTC", "ExecMainStartTimestampMonotonic": "20187973", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-12-05 17:25:19 UTC", "InactiveExitTimestampMonotonic": "19899819", "InvocationID": "4311d541beb946ac9a83a90ff239a9eb", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2020-12-05T17:26:47.023 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128053", "LimitNPROCSoft": "128053", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128053", "LimitSIGPENDINGSoft": "128053", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1441", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2020-12-05T17:26:47.026 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-12-05 17:25:19 UTC", "StateChangeTimestampMonotonic": "20187984", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-12-05 17:25:19 UTC", "WatchdogTimestampMonotonic": "20187983", "WatchdogUSec": "0"}} TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:47.059 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.047) 0:01:08.940 ***** 2020-12-05T17:26:47.094 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.034) 0:01:08.975 ***** 2020-12-05T17:26:47.100 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-12-05T17:26:47.102 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:47.140 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.046) 0:01:09.021 ***** 2020-12-05T17:26:47.148 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-12-05T17:26:47.150 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:47.188 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.048) 0:01:09.070 ***** 2020-12-05T17:26:47.199 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-12-05T17:26:47.201 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ ok: [smithi036.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1235, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-12-05T17:26:47.239 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.050) 0:01:09.121 ***** 2020-12-05T17:26:47.272 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.033) 0:01:09.154 ***** 2020-12-05T17:26:47.278 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-12-05T17:26:47.280 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:47.317 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.044) 0:01:09.199 ***** 2020-12-05T17:26:47.350 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.033) 0:01:09.232 ***** 2020-12-05T17:26:47.358 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-12-05T17:26:47.359 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => {"cache_update_time": 1607189009, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 3 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.34 [40.2 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.34 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.34 [5188 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.34 [871 kB]\nFetched 10.4 MB in 1s (9924 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 2020-12-05T17:26:47.361 INFO:teuthology.task.ansible.out: 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137466 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu 2020-12-05T17:26:47.362 INFO:teuthology.task.ansible.out:1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 3 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.34 [40.2 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.34 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.34 [5188 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.34 [871 kB]", "Fetched 10.4 MB in 1s (9924 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Rea 2020-12-05T17:26:47.364 INFO:teuthology.task.ansible.out:ding database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137466 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:47.397 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.047) 0:01:09.279 ***** 2020-12-05T17:26:47.430 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.032) 0:01:09.312 ***** 2020-12-05T17:26:47.437 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-12-05T17:26:47.439 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set up static IP in /etc/hosts.] ****************************** ok: [smithi036.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:47.475 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.044) 0:01:09.357 ***** 2020-12-05T17:26:47.509 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.033) 0:01:09.390 ***** 2020-12-05T17:26:47.541 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.032) 0:01:09.423 ***** 2020-12-05T17:26:47.548 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-12-05T17:26:47.550 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:47.586 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.044) 0:01:09.468 ***** 2020-12-05T17:26:47.621 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.034) 0:01:09.502 ***** 2020-12-05T17:26:47.627 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-12-05T17:26:47.629 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:47.668 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.047) 0:01:09.549 ***** 2020-12-05T17:26:47.674 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:47.711 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.043) 0:01:09.593 ***** 2020-12-05T17:26:47.718 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-12-05T17:26:47.720 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:47.754 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.042) 0:01:09.636 ***** 2020-12-05T17:26:47.762 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-12-05T17:26:47.764 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => {"cache_update_time": 1607188959, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 3 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.34 [40.2 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.34 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.34 [5188 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.34 [871 kB]\nFetched 10.4 MB in 1s (9607 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 2020-12-05T17:26:47.766 INFO:teuthology.task.ansible.out: 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137466 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu 2020-12-05T17:26:47.767 INFO:teuthology.task.ansible.out:1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 3 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.34 [40.2 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.34 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.34 [5188 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.34 [871 kB]", "Fetched 10.4 MB in 1s (9607 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Rea 2020-12-05T17:26:47.769 INFO:teuthology.task.ansible.out:ding database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137466 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:47.801 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.046) 0:01:09.682 ***** 2020-12-05T17:26:47.835 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.033) 0:01:09.716 ***** 2020-12-05T17:26:47.841 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:26:47.842 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:47.842 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-12-05T17:26:47.844 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sat 2020-12-05 17:25:13 UTC", "ActiveEnterTimestampMonotonic": "20408989", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-tmpfiles-setup.service sysinit.target -.mount basic.target systemd-journald.socket remote-fs.target system.slice network.target nss-lookup.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-12-05 17:25:12 UTC", "AssertTimestampMonotonic": "19994999", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-12-05 17:25:12 UTC", "ConditionTimestampMonotonic": "19994999", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2020-12-05T17:26:47.846 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1450", "ExecMainStartTimestamp": "Sat 2020-12-05 17:25:13 UTC", "ExecMainStartTimestampMonotonic": "20408961", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-12-05 17:25:12 UTC", "InactiveExitTimestampMonotonic": "19995620", "InvocationID": "27a4ea4307b347c2b971bd728f6af9f3", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2020-12-05T17:26:47.847 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128053", "LimitNPROCSoft": "128053", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128053", "LimitSIGPENDINGSoft": "128053", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1450", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2020-12-05T17:26:47.849 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-12-05 17:25:13 UTC", "StateChangeTimestampMonotonic": "20408989", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-12-05 17:25:13 UTC", "WatchdogTimestampMonotonic": "20408988", "WatchdogUSec": "0"}} TASK [testnode : set_fact] ***************************************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:47.882 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.047) 0:01:09.764 ***** 2020-12-05T17:26:47.916 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.033) 0:01:09.798 ***** 2020-12-05T17:26:47.950 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.033) 0:01:09.831 ***** 2020-12-05T17:26:47.956 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-12-05T17:26:47.958 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:47.993 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:47 +0000 (0:00:00.043) 0:01:09.875 ***** 2020-12-05T17:26:48.028 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.032) 0:01:09.908 ***** 2020-12-05T17:26:48.034 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-12-05T17:26:48.036 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install packages] ********************************************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:48.071 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.044) 0:01:09.952 ***** 2020-12-05T17:26:48.104 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.032) 0:01:09.985 ***** 2020-12-05T17:26:48.111 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:48.112 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:48.148 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.044) 0:01:10.029 ***** 2020-12-05T17:26:48.182 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.033) 0:01:10.063 ***** 2020-12-05T17:26:48.189 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-12-05T17:26:48.190 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:48.230 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.048) 0:01:10.112 ***** 2020-12-05T17:26:48.263 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.033) 0:01:10.145 ***** 2020-12-05T17:26:48.271 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-12-05T17:26:48.272 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:48.308 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.044) 0:01:10.190 ***** 2020-12-05T17:26:48.344 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.035) 0:01:10.225 ***** 2020-12-05T17:26:48.350 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-12-05T17:26:48.352 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:48.387 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.043) 0:01:10.269 ***** 2020-12-05T17:26:48.420 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.033) 0:01:10.302 ***** 2020-12-05T17:26:48.427 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-12-05T17:26:48.428 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:48.465 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.044) 0:01:10.346 ***** 2020-12-05T17:26:48.499 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.034) 0:01:10.381 ***** 2020-12-05T17:26:48.506 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-12-05T17:26:48.507 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"cache_update_time": 1607189009, "cache_updated": false, "changed": false} TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:48.508 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:48.544 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.044) 0:01:10.425 ***** 2020-12-05T17:26:48.577 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.033) 0:01:10.459 ***** 2020-12-05T17:26:48.611 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.033) 0:01:10.492 ***** 2020-12-05T17:26:48.617 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-12-05T17:26:48.619 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:48.656 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.045) 0:01:10.538 ***** 2020-12-05T17:26:48.690 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.033) 0:01:10.571 ***** 2020-12-05T17:26:48.724 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.034) 0:01:10.606 ***** 2020-12-05T17:26:48.731 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-12-05T17:26:48.732 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:48.733 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-12-05T17:26:48.733 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2020-12-05T17:26:48.769 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.044) 0:01:10.650 ***** 2020-12-05T17:26:48.805 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.036) 0:01:10.687 ***** 2020-12-05T17:26:48.839 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.033) 0:01:10.721 ***** 2020-12-05T17:26:48.845 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-12-05T17:26:48.847 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install non aarch64 packages] ********************************* 2020-12-05T17:26:48.847 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"cache_update_time": 1607188959, "cache_updated": false, "changed": false} 2020-12-05T17:26:48.848 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-12-05T17:26:48.848 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:48.884 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.044) 0:01:10.765 ***** 2020-12-05T17:26:48.918 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.034) 0:01:10.800 ***** 2020-12-05T17:26:48.953 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.035) 0:01:10.835 ***** 2020-12-05T17:26:48.989 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:48 +0000 (0:00:00.035) 0:01:10.871 ***** 2020-12-05T17:26:48.995 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-12-05T17:26:48.997 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:48.997 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:48.998 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:49.052 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:49 +0000 (0:00:00.062) 0:01:10.934 ***** 2020-12-05T17:26:49.092 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:49 +0000 (0:00:00.040) 0:01:10.974 ***** 2020-12-05T17:26:49.146 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:49 +0000 (0:00:00.053) 0:01:11.028 ***** 2020-12-05T17:26:49.179 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:49 +0000 (0:00:00.033) 0:01:11.061 ***** 2020-12-05T17:26:49.186 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-12-05T17:26:49.188 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2020-12-05T17:26:49.188 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:49.189 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:49.248 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:49 +0000 (0:00:00.068) 0:01:11.130 ***** 2020-12-05T17:26:49.287 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:49 +0000 (0:00:00.039) 0:01:11.169 ***** 2020-12-05T17:26:49.321 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:49 +0000 (0:00:00.033) 0:01:11.202 ***** 2020-12-05T17:26:49.353 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:49 +0000 (0:00:00.032) 0:01:11.235 ***** 2020-12-05T17:26:49.360 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-12-05T17:26:49.362 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure fuse group exists.] ************************************ ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} TASK [testnode : set_fact] ***************************************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:49.411 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:49 +0000 (0:00:00.057) 0:01:11.293 ***** 2020-12-05T17:26:49.460 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:49 +0000 (0:00:00.048) 0:01:11.341 ***** 2020-12-05T17:26:49.496 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:49 +0000 (0:00:00.036) 0:01:11.378 ***** 2020-12-05T17:26:49.534 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:49 +0000 (0:00:00.037) 0:01:11.416 ***** 2020-12-05T17:26:49.541 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-12-05T17:26:49.543 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:49.544 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:49.586 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:49 +0000 (0:00:00.052) 0:01:11.468 ***** 2020-12-05T17:26:49.624 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:49 +0000 (0:00:00.037) 0:01:11.506 ***** 2020-12-05T17:26:49.660 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:49 +0000 (0:00:00.036) 0:01:11.542 ***** 2020-12-05T17:26:49.698 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:49 +0000 (0:00:00.037) 0:01:11.580 ***** 2020-12-05T17:26:49.705 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-12-05T17:26:49.707 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install packages] ********************************************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:49.744 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:49 +0000 (0:00:00.046) 0:01:11.626 ***** 2020-12-05T17:26:49.782 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:49 +0000 (0:00:00.037) 0:01:11.664 ***** 2020-12-05T17:26:49.819 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:49 +0000 (0:00:00.036) 0:01:11.701 ***** 2020-12-05T17:26:49.825 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-12-05T17:26:49.827 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:49.862 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:49 +0000 (0:00:00.043) 0:01:11.744 ***** 2020-12-05T17:26:49.896 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:49 +0000 (0:00:00.032) 0:01:11.776 ***** 2020-12-05T17:26:49.901 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-12-05T17:26:49.902 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} TASK [testnode : Upload /etc/fuse.conf.] *************************************** ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-12-05T17:26:49.903 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:49.940 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:49 +0000 (0:00:00.045) 0:01:11.821 ***** 2020-12-05T17:26:49.973 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:49 +0000 (0:00:00.033) 0:01:11.854 ***** 2020-12-05T17:26:50.006 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:50 +0000 (0:00:00.033) 0:01:11.888 ***** 2020-12-05T17:26:50.012 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-12-05T17:26:50.013 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:50.014 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2020-12-05T17:26:50.052 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:50 +0000 (0:00:00.045) 0:01:11.934 ***** 2020-12-05T17:26:50.097 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:50 +0000 (0:00:00.044) 0:01:11.978 ***** 2020-12-05T17:26:50.136 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:50 +0000 (0:00:00.038) 0:01:12.017 ***** 2020-12-05T17:26:50.143 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-12-05T17:26:50.145 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:50.188 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:50 +0000 (0:00:00.052) 0:01:12.070 ***** 2020-12-05T17:26:50.196 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-12-05T17:26:50.198 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ ok: [smithi061.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1235, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-12-05T17:26:50.235 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:50 +0000 (0:00:00.046) 0:01:12.116 ***** 2020-12-05T17:26:50.269 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:50 +0000 (0:00:00.033) 0:01:12.150 ***** 2020-12-05T17:26:50.275 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:50.277 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:50.315 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:50 +0000 (0:00:00.046) 0:01:12.197 ***** 2020-12-05T17:26:50.353 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:50 +0000 (0:00:00.037) 0:01:12.235 ***** 2020-12-05T17:26:50.361 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-12-05T17:26:50.363 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} TASK [testnode : Set grub config.] ********************************************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:50.400 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:50 +0000 (0:00:00.047) 0:01:12.282 ***** 2020-12-05T17:26:50.435 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:50 +0000 (0:00:00.034) 0:01:12.317 ***** 2020-12-05T17:26:50.442 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-12-05T17:26:50.444 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:50.444 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** ok: [smithi061.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-12-05T17:26:50.483 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:50 +0000 (0:00:00.048) 0:01:12.365 ***** 2020-12-05T17:26:50.518 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:50 +0000 (0:00:00.035) 0:01:12.400 ***** 2020-12-05T17:26:50.554 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:50 +0000 (0:00:00.035) 0:01:12.435 ***** 2020-12-05T17:26:50.560 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-12-05T17:26:50.562 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove /etc/ceph] ********************************************* ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:50.600 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:50 +0000 (0:00:00.046) 0:01:12.482 ***** 2020-12-05T17:26:50.635 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:50 +0000 (0:00:00.034) 0:01:12.516 ***** 2020-12-05T17:26:50.676 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:50 +0000 (0:00:00.040) 0:01:12.557 ***** 2020-12-05T17:26:50.683 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-12-05T17:26:50.685 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1235, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-12-05T17:26:50.727 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:50 +0000 (0:00:00.051) 0:01:12.609 ***** 2020-12-05T17:26:50.735 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-12-05T17:26:50.736 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:50.778 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:50 +0000 (0:00:00.051) 0:01:12.660 ***** 2020-12-05T17:26:50.807 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-12-05T17:26:50.809 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:50.846 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:50 +0000 (0:00:00.067) 0:01:12.728 ***** 2020-12-05T17:26:50.854 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-12-05T17:26:50.856 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sat 2020-12-05 17:25:19 UTC", "ActiveEnterTimestampMonotonic": "18840721", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target system.slice systemd-tmpfiles-setup.service systemd-journald.socket sysinit.target remote-fs.target basic.target nss-lookup.target -.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-12-05 17:25:19 UTC", "AssertTimestampMonotonic": "18527741", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-12-05 17:25:19 UTC", "ConditionTimestampMonotonic": "18527741", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2020-12-05T17:26:50.858 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1375", "ExecMainStartTimestamp": "Sat 2020-12-05 17:25:19 UTC", "ExecMainStartTimestampMonotonic": "18840709", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-12-05 17:25:19 UTC", "InactiveExitTimestampMonotonic": "18528403", "InvocationID": "f634e399a4f749918e80195e20fc8034", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2020-12-05T17:26:50.859 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1375", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2020-12-05T17:26:50.861 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-12-05 17:25:19 UTC", "StateChangeTimestampMonotonic": "18840721", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-12-05 17:25:19 UTC", "WatchdogTimestampMonotonic": "18840719", "WatchdogUSec": "0"}} 2020-12-05T17:26:50.895 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:50 +0000 (0:00:00.048) 0:01:12.776 ***** 2020-12-05T17:26:50.900 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-12-05T17:26:50.902 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:50.943 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:50 +0000 (0:00:00.048) 0:01:12.825 ***** 2020-12-05T17:26:50.983 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:50 +0000 (0:00:00.040) 0:01:12.865 ***** 2020-12-05T17:26:50.990 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-12-05T17:26:50.992 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload weekly kernel-clean crontab.] ************************** ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-12-05T17:26:50.998 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:51.031 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.047) 0:01:12.912 ***** 2020-12-05T17:26:51.065 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.034) 0:01:12.947 ***** 2020-12-05T17:26:51.100 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.034) 0:01:12.981 ***** 2020-12-05T17:26:51.134 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.033) 0:01:13.015 ***** 2020-12-05T17:26:51.142 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-12-05T17:26:51.144 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:51.183 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.049) 0:01:13.065 ***** 2020-12-05T17:26:51.218 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.034) 0:01:13.099 ***** 2020-12-05T17:26:51.226 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:51.274 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.056) 0:01:13.156 ***** 2020-12-05T17:26:51.282 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-12-05T17:26:51.284 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:51.285 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* changed: [smithi053.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sat 2020-12-05 17:25:24 UTC", "ActiveEnterTimestampMonotonic": "20715182", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target systemd-journald.socket basic.target system.slice sysinit.target nss-lookup.target systemd-tmpfiles-setup.service remote-fs.target -.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-12-05 17:25:24 UTC", "AssertTimestampMonotonic": "20350556", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-12-05 17:25:24 UTC", "ConditionTimestampMonotonic": "20350556", "ConfigurationDirectoryMode": "0755", "Conflicts": "shu 2020-12-05T17:26:51.287 INFO:teuthology.task.ansible.out:tdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", "DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1372", "ExecMainStartTimestamp": "Sat 2020-12-05 17:25:24 UTC", "ExecMainStartTimestampMonotonic": "20715171", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-12-05 17:25:24 UTC", "InactiveExitTimestampMonotonic": "20351386", "InvocationID": "ea5366ea281940db8718aaa82cd40001", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinit 2020-12-05T17:26:51.288 INFO:teuthology.task.ansible.out:y", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128053", "LimitNPROCSoft": "128053", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128053", "LimitSIGPENDINGSoft": "128053", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1372", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Resu 2020-12-05T17:26:51.290 INFO:teuthology.task.ansible.out:lt": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-12-05 17:25:24 UTC", "StateChangeTimestampMonotonic": "20715182", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-12-05 17:25:24 UTC", "WatchdogTimestampMonotonic": "20715181", "WatchdogUSec": "0"}} 2020-12-05T17:26:51.326 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.052) 0:01:13.208 ***** 2020-12-05T17:26:51.366 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.039) 0:01:13.248 ***** 2020-12-05T17:26:51.372 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-12-05T17:26:51.374 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-12-05T17:26:51.374 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-12-05T17:26:51.411 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.044) 0:01:13.292 ***** 2020-12-05T17:26:51.445 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.033) 0:01:13.326 ***** 2020-12-05T17:26:51.478 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.033) 0:01:13.360 ***** 2020-12-05T17:26:51.485 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-12-05T17:26:51.488 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:51.525 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.046) 0:01:13.407 ***** 2020-12-05T17:26:51.561 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.035) 0:01:13.442 ***** 2020-12-05T17:26:51.569 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-12-05T17:26:51.571 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload weekly kernel-clean crontab.] ************************** ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-12-05T17:26:51.571 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:51.607 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.046) 0:01:13.489 ***** 2020-12-05T17:26:51.641 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.033) 0:01:13.522 ***** 2020-12-05T17:26:51.674 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.033) 0:01:13.556 ***** 2020-12-05T17:26:51.681 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-12-05T17:26:51.683 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:51.718 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.044) 0:01:13.600 ***** 2020-12-05T17:26:51.726 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-12-05T17:26:51.728 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:51.763 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.044) 0:01:13.644 ***** 2020-12-05T17:26:51.769 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-12-05T17:26:51.771 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:51.807 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.043) 0:01:13.688 ***** 2020-12-05T17:26:51.840 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.032) 0:01:13.721 ***** 2020-12-05T17:26:51.847 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-12-05T17:26:51.848 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:51.885 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.045) 0:01:13.767 ***** 2020-12-05T17:26:51.919 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.034) 0:01:13.801 ***** 2020-12-05T17:26:51.926 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-12-05T17:26:51.927 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:51.929 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-12-05T17:26:51.974 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:51 +0000 (0:00:00.054) 0:01:13.856 ***** 2020-12-05T17:26:52.021 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.046) 0:01:13.902 ***** 2020-12-05T17:26:52.029 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-12-05T17:26:52.031 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-12-05T17:26:52.033 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-12-05T17:26:52.033 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:52.071 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.050) 0:01:13.953 ***** 2020-12-05T17:26:52.107 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.035) 0:01:13.989 ***** 2020-12-05T17:26:52.112 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-12-05T17:26:52.114 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:52.150 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.043) 0:01:14.032 ***** 2020-12-05T17:26:52.184 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.033) 0:01:14.065 ***** 2020-12-05T17:26:52.190 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-12-05T17:26:52.192 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Stop collectl] ************************************************ skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:52.226 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.042) 0:01:14.108 ***** 2020-12-05T17:26:52.258 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.032) 0:01:14.140 ***** 2020-12-05T17:26:52.264 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-12-05T17:26:52.266 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2020-12-05T17:26:52.268 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable kernel modules to load at boot time.] ****************** ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-12-05T17:26:52.303 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.044) 0:01:14.185 ***** 2020-12-05T17:26:52.335 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.032) 0:01:14.217 ***** 2020-12-05T17:26:52.367 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.031) 0:01:14.249 ***** 2020-12-05T17:26:52.400 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.032) 0:01:14.281 ***** 2020-12-05T17:26:52.406 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:26:52.408 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:52.443 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.043) 0:01:14.325 ***** 2020-12-05T17:26:52.476 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.032) 0:01:14.357 ***** 2020-12-05T17:26:52.508 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.032) 0:01:14.390 ***** 2020-12-05T17:26:52.515 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-12-05T17:26:52.516 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** ok: [smithi032.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-12-05T17:26:52.517 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:52.558 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.049) 0:01:14.439 ***** 2020-12-05T17:26:52.591 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.033) 0:01:14.473 ***** 2020-12-05T17:26:52.625 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.033) 0:01:14.506 ***** 2020-12-05T17:26:52.632 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-12-05T17:26:52.633 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** ok: [smithi002.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:52.668 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.043) 0:01:14.550 ***** 2020-12-05T17:26:52.701 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.033) 0:01:14.583 ***** 2020-12-05T17:26:52.736 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.034) 0:01:14.617 ***** 2020-12-05T17:26:52.742 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-12-05T17:26:52.743 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi002.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:52.743 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:52.744 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:52.745 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi002.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:52.746 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:52.746 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:52.783 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.047) 0:01:14.664 ***** 2020-12-05T17:26:52.818 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.035) 0:01:14.700 ***** 2020-12-05T17:26:52.825 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-12-05T17:26:52.825 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:52.827 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi002.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:52.829 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:52.829 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-12-05T17:26:52.829 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-12-05T17:26:52.831 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi002.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:52.872 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.053) 0:01:14.753 ***** 2020-12-05T17:26:52.904 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.032) 0:01:14.786 ***** 2020-12-05T17:26:52.938 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.033) 0:01:14.819 ***** 2020-12-05T17:26:52.971 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:52 +0000 (0:00:00.033) 0:01:14.853 ***** 2020-12-05T17:26:52.977 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:26:52.979 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:53.019 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.048) 0:01:14.901 ***** 2020-12-05T17:26:53.054 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.034) 0:01:14.936 ***** 2020-12-05T17:26:53.060 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-12-05T17:26:53.062 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:53.062 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi002.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003991", "end": "2020-12-05 17:26:53.036933", "rc": 0, "start": "2020-12-05 17:26:53.032942", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-12-05T17:26:53.108 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.053) 0:01:14.990 ***** 2020-12-05T17:26:53.142 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.034) 0:01:15.024 ***** 2020-12-05T17:26:53.177 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.034) 0:01:15.058 ***** 2020-12-05T17:26:53.183 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-12-05T17:26:53.185 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** ok: [smithi032.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-12-05T17:26:53.224 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.046) 0:01:15.105 ***** 2020-12-05T17:26:53.257 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.032) 0:01:15.138 ***** 2020-12-05T17:26:53.289 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.032) 0:01:15.170 ***** 2020-12-05T17:26:53.295 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-12-05T17:26:53.297 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-12-05T17:26:53.298 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-12-05T17:26:53.334 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.045) 0:01:15.216 ***** 2020-12-05T17:26:53.368 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.033) 0:01:15.250 ***** 2020-12-05T17:26:53.402 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.033) 0:01:15.284 ***** 2020-12-05T17:26:53.408 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-12-05T17:26:53.410 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} TASK [testnode : Install packages] ********************************************* 2020-12-05T17:26:53.411 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi032.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:53.411 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:53.412 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} ok: [smithi032.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-12-05T17:26:53.413 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:53.413 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional res 2020-12-05T17:26:53.415 INFO:teuthology.task.ansible.out:ult was False"} TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi032.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:53.416 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:53.416 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:53.416 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:53.453 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.050) 0:01:15.335 ***** 2020-12-05T17:26:53.489 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.035) 0:01:15.371 ***** 2020-12-05T17:26:53.522 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.032) 0:01:15.404 ***** 2020-12-05T17:26:53.529 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-12-05T17:26:53.530 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:53.565 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.042) 0:01:15.446 ***** 2020-12-05T17:26:53.570 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-12-05T17:26:53.572 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:53.607 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.042) 0:01:15.489 ***** 2020-12-05T17:26:53.613 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-12-05T17:26:53.615 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:53.648 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.041) 0:01:15.530 ***** 2020-12-05T17:26:53.662 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-12-05T17:26:53.664 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi032.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.002902", "end": "2020-12-05 17:26:53.643343", "rc": 0, "start": "2020-12-05 17:26:53.640441", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-12-05T17:26:53.703 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.054) 0:01:15.585 ***** 2020-12-05T17:26:53.736 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.033) 0:01:15.618 ***** 2020-12-05T17:26:53.742 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-12-05T17:26:53.744 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable kernel modules to load at boot time.] ****************** ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-12-05T17:26:53.745 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:53.785 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.048) 0:01:15.667 ***** 2020-12-05T17:26:53.818 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.032) 0:01:15.700 ***** 2020-12-05T17:26:53.852 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.034) 0:01:15.734 ***** 2020-12-05T17:26:53.858 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-12-05T17:26:53.860 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:53.896 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.043) 0:01:15.778 ***** 2020-12-05T17:26:53.929 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.033) 0:01:15.811 ***** 2020-12-05T17:26:53.936 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-12-05T17:26:53.937 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:53.972 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.043) 0:01:15.854 ***** 2020-12-05T17:26:54.005 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:53 +0000 (0:00:00.032) 0:01:15.886 ***** 2020-12-05T17:26:54.010 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-12-05T17:26:54.012 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:54.013 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** ok: [smithi036.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-12-05T17:26:54.049 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:54 +0000 (0:00:00.044) 0:01:15.931 ***** 2020-12-05T17:26:54.083 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:54 +0000 (0:00:00.033) 0:01:15.965 ***** 2020-12-05T17:26:54.089 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-12-05T17:26:54.090 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:54.126 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:54 +0000 (0:00:00.043) 0:01:16.008 ***** 2020-12-05T17:26:54.165 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:54 +0000 (0:00:00.038) 0:01:16.047 ***** 2020-12-05T17:26:54.171 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-12-05T17:26:54.172 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Stop collectl] ************************************************ skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:54.209 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:54 +0000 (0:00:00.044) 0:01:16.091 ***** 2020-12-05T17:26:54.248 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:54 +0000 (0:00:00.039) 0:01:16.130 ***** 2020-12-05T17:26:54.255 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-12-05T17:26:54.256 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:54.290 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:54 +0000 (0:00:00.041) 0:01:16.172 ***** 2020-12-05T17:26:54.322 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:54 +0000 (0:00:00.032) 0:01:16.204 ***** 2020-12-05T17:26:54.329 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-12-05T17:26:54.331 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:54.365 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:54 +0000 (0:00:00.042) 0:01:16.247 ***** 2020-12-05T17:26:54.397 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:54 +0000 (0:00:00.031) 0:01:16.279 ***** 2020-12-05T17:26:54.404 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-12-05T17:26:54.406 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make sure apt dependencies are installed] ********************* ok: [smithi002.front.sepia.ceph.com] => {"cache_update_time": 1607188915, "cache_updated": false, "changed": false} TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:54.442 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:54 +0000 (0:00:00.044) 0:01:16.323 ***** 2020-12-05T17:26:54.474 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:54 +0000 (0:00:00.032) 0:01:16.356 ***** 2020-12-05T17:26:54.505 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:54 +0000 (0:00:00.031) 0:01:16.387 ***** 2020-12-05T17:26:54.510 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-12-05T17:26:54.512 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make sure rpm dependencies are installed] ********************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** ok: [smithi036.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-12-05T17:26:54.545 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:54 +0000 (0:00:00.040) 0:01:16.427 ***** 2020-12-05T17:26:54.579 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:54 +0000 (0:00:00.033) 0:01:16.460 ***** 2020-12-05T17:26:54.612 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:54 +0000 (0:00:00.032) 0:01:16.493 ***** 2020-12-05T17:26:54.619 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-12-05T17:26:54.621 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set root disk] ************************************************ ok: [smithi002.front.sepia.ceph.com] => (item={'block_used': 13755773, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226339335, 'size_available': 927085916160, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149528, 'block_size': 4096, 'inode_available': 60905448}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226339335, "block_size": 4096, "block_total": 240095108, "block_used": 13755773, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905448, "inode_total": 61054976, "inode_used": 149528, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927085916160, "size_total": 983429562368, "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}} 2020-12-05T17:26:54.622 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi036.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:54.657 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:54 +0000 (0:00:00.044) 0:01:16.538 ***** 2020-12-05T17:26:54.692 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:54 +0000 (0:00:00.034) 0:01:16.573 ***** 2020-12-05T17:26:54.698 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:54.699 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-12-05T17:26:54.699 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:54.700 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi036.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:54.737 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:54 +0000 (0:00:00.044) 0:01:16.618 ***** 2020-12-05T17:26:54.743 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-12-05T17:26:54.744 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:54.785 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:54 +0000 (0:00:00.048) 0:01:16.667 ***** 2020-12-05T17:26:54.791 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-12-05T17:26:54.792 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:54.793 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi036.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:54.793 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:54.794 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-12-05T17:26:54.794 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:54.796 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** changed: [smithi002.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.009822", "end": "2020-12-05 17:26:54.775798", "rc": 0, "start": "2020-12-05 17:26:54.765976", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi036.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:54.796 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:54.934 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:54 +0000 (0:00:00.149) 0:01:16.816 ***** 2020-12-05T17:26:54.970 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:54 +0000 (0:00:00.035) 0:01:16.852 ***** 2020-12-05T17:26:54.977 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-12-05T17:26:54.978 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi036.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:54.979 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:54.980 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:54.982 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-12-05T17:26:54.984 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"cache_update_time": 1607188779, "cache_updated": false, "changed": false} TASK [testnode : Unmount any non-root mountpoints] ***************************** skipping: [smithi002.front.sepia.ceph.com] => (item={'block_used': 13755773, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226339335, 'size_available': 927085916160, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149528, 'block_size': 4096, 'inode_available': 60905448}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226339335, "block_size": 4096, "block_total": 240095108, "block_used": 13755773, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905448, "inode_total": 61054976, "inode_used": 149528, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927085916160, "size_total": 983429562368, "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:55.021 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.051) 0:01:16.903 ***** 2020-12-05T17:26:55.057 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.035) 0:01:16.938 ***** 2020-12-05T17:26:55.096 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.038) 0:01:16.977 ***** 2020-12-05T17:26:55.131 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.035) 0:01:17.013 ***** 2020-12-05T17:26:55.137 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-12-05T17:26:55.139 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Wipe filesystems on non-root partitions] ********************** TASK [testnode : Make sure rpm dependencies are installed] ********************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:55.176 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.044) 0:01:17.058 ***** 2020-12-05T17:26:55.213 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.036) 0:01:17.094 ***** 2020-12-05T17:26:55.249 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.036) 0:01:17.131 ***** 2020-12-05T17:26:55.255 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-12-05T17:26:55.257 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Zap all non-root disks] *************************************** skipping: [smithi002.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:55.258 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ ok: [smithi032.front.sepia.ceph.com] => (item={'block_used': 13755772, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226339336, 'size_available': 927085920256, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149528, 'block_size': 4096, 'inode_available': 60905448}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226339336, "block_size": 4096, "block_total": 240095108, "block_used": 13755772, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905448, "inode_total": 61054976, "inode_used": 149528, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927085920256, "size_total": 983429562368, "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}} 2020-12-05T17:26:55.295 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.046) 0:01:17.177 ***** 2020-12-05T17:26:55.331 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.035) 0:01:17.212 ***** 2020-12-05T17:26:55.337 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-12-05T17:26:55.338 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi036.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003070", "end": "2020-12-05 17:26:55.254124", "rc": 0, "start": "2020-12-05 17:26:55.251054", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-12-05T17:26:55.379 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.048) 0:01:17.261 ***** 2020-12-05T17:26:55.414 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.034) 0:01:17.296 ***** 2020-12-05T17:26:55.420 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-12-05T17:26:55.422 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:55.460 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.045) 0:01:17.342 ***** 2020-12-05T17:26:55.495 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.035) 0:01:17.377 ***** 2020-12-05T17:26:55.501 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-12-05T17:26:55.502 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.008288", "end": "2020-12-05 17:26:55.614264", "rc": 0, "start": "2020-12-05 17:26:55.605976", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:55.538 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.042) 0:01:17.419 ***** 2020-12-05T17:26:55.570 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.032) 0:01:17.452 ***** 2020-12-05T17:26:55.603 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.032) 0:01:17.484 ***** 2020-12-05T17:26:55.608 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-12-05T17:26:55.610 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Unmount any non-root mountpoints] ***************************** skipping: [smithi032.front.sepia.ceph.com] => (item={'block_used': 13755772, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226339336, 'size_available': 927085920256, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149528, 'block_size': 4096, 'inode_available': 60905448}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226339336, "block_size": 4096, "block_total": 240095108, "block_used": 13755772, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905448, "inode_total": 61054976, "inode_used": 149528, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927085920256, "size_total": 983429562368, "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:55.611 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:55.646 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.043) 0:01:17.528 ***** 2020-12-05T17:26:55.679 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.032) 0:01:17.560 ***** 2020-12-05T17:26:55.711 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.032) 0:01:17.593 ***** 2020-12-05T17:26:55.716 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-12-05T17:26:55.718 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-12-05T17:26:55.754 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.043) 0:01:17.636 ***** 2020-12-05T17:26:55.788 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.034) 0:01:17.670 ***** 2020-12-05T17:26:55.806 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-12-05T17:26:55.807 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:55.818 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:55.853 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.065) 0:01:17.735 ***** 2020-12-05T17:26:55.859 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-12-05T17:26:55.861 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2020-12-05T17:26:55.896 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.042) 0:01:17.778 ***** 2020-12-05T17:26:55.904 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-12-05T17:26:55.905 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:55.939 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:55 +0000 (0:00:00.042) 0:01:17.821 ***** 2020-12-05T17:26:56.070 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-12-05T17:26:56.072 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2020-12-05T17:26:56.105 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:56 +0000 (0:00:00.166) 0:01:17.987 ***** 2020-12-05T17:26:56.207 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-12-05T17:26:56.209 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2020-12-05T17:26:56.243 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:56 +0000 (0:00:00.138) 0:01:18.125 ***** 2020-12-05T17:26:56.270 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-12-05T17:26:56.272 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:56.305 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:56 +0000 (0:00:00.061) 0:01:18.187 ***** 2020-12-05T17:26:56.418 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-12-05T17:26:56.420 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2020-12-05T17:26:56.479 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:56 +0000 (0:00:00.173) 0:01:18.361 ***** 2020-12-05T17:26:56.508 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-12-05T17:26:56.510 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:56.550 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:56 +0000 (0:00:00.071) 0:01:18.432 ***** 2020-12-05T17:26:56.590 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-12-05T17:26:56.592 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"cache_update_time": 1607189016, "cache_updated": false, "changed": false} 2020-12-05T17:26:56.628 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:56 +0000 (0:00:00.077) 0:01:18.509 ***** 2020-12-05T17:26:56.634 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-12-05T17:26:56.635 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-12-05T17:26:56.671 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:56 +0000 (0:00:00.043) 0:01:18.553 ***** 2020-12-05T17:26:56.678 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-12-05T17:26:56.679 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:56.714 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:56 +0000 (0:00:00.043) 0:01:18.596 ***** 2020-12-05T17:26:56.752 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-12-05T17:26:56.754 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => (item={'block_used': 13755773, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226339335, 'size_available': 927085916160, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149528, 'block_size': 4096, 'inode_available': 60905448}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226339335, "block_size": 4096, "block_total": 240095108, "block_used": 13755773, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905448, "inode_total": 61054976, "inode_used": 149528, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927085916160, "size_total": 983429562368, "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}} 2020-12-05T17:26:56.788 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:56 +0000 (0:00:00.073) 0:01:18.670 ***** 2020-12-05T17:26:56.836 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-12-05T17:26:56.838 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-12-05T17:26:56.873 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:56 +0000 (0:00:00.085) 0:01:18.755 ***** 2020-12-05T17:26:56.932 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-12-05T17:26:56.934 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.011165", "end": "2020-12-05 17:26:57.240780", "rc": 0, "start": "2020-12-05 17:26:57.229615", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:26:56.970 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:56 +0000 (0:00:00.096) 0:01:18.852 ***** 2020-12-05T17:26:56.996 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-12-05T17:26:56.998 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-12-05T17:26:57.033 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:57 +0000 (0:00:00.062) 0:01:18.915 ***** 2020-12-05T17:26:57.039 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-12-05T17:26:57.040 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-Z7OWrnoNVLFO33ZqvS1UNeFEKSkYadZ2uEXYeJclX20MT3GcOv0gU4WzQCq9Mn3J'], 'uuids': ['b069103d-6552-435e-a95a-94a6a28285d8']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.663784", "end": "2020-12-05 17:26:56.982872", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-Z7OWrnoNVLFO33ZqvS1UNeFEKSkYadZ2uEXYeJclX20MT3GcOv0gU4WzQCq9Mn3J"], "labels": [], "masters": [], "uuids": ["b069103d-6552-435e-a95a-94a6a28285d8"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:26:55.319088", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:26:57.042 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** skipping: [smithi036.front.sepia.ceph.com] => (item={'block_used': 13755773, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226339335, 'size_available': 927085916160, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149528, 'block_size': 4096, 'inode_available': 60905448}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226339335, "block_size": 4096, "block_total": 240095108, "block_used": 13755773, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905448, "inode_total": 61054976, "inode_used": 149528, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927085916160, "size_total": 983429562368, "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:57.043 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** skipping: [smithi002.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:57.045 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:57.081 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:57 +0000 (0:00:00.047) 0:01:18.962 ***** 2020-12-05T17:26:57.107 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-12-05T17:26:57.142 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:57 +0000 (0:00:00.061) 0:01:19.024 ***** 2020-12-05T17:26:57.179 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-12-05T17:26:57.181 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-12-05T17:26:57.216 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:57 +0000 (0:00:00.074) 0:01:19.098 ***** 2020-12-05T17:26:57.223 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-12-05T17:26:57.225 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:57.330 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-12-05T17:26:57.331 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-12-05T17:26:57.370 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:57 +0000 (0:00:00.154) 0:01:19.252 ***** 2020-12-05T17:26:57.471 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-12-05T17:26:57.473 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-iuDKVQlK8CuPQcSO8QqzZYqsR03q8qQf4rRVa18u6jjT3xFF9hwvR8HolmZCTziT'], 'uuids': ['01cc38fa-b575-4e99-b094-a907bbb92eab']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.555040", "end": "2020-12-05 17:26:57.645291", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-iuDKVQlK8CuPQcSO8QqzZYqsR03q8qQf4rRVa18u6jjT3xFF9hwvR8HolmZCTziT"], "labels": [], "masters": [], "uuids": ["01cc38fa-b575-4e99-b094-a907bbb92eab"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:26:56.090251", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:26:57.483 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:57.494 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:57.495 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** ok: [smithi061.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-12-05T17:26:57.531 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:57 +0000 (0:00:00.160) 0:01:19.412 ***** 2020-12-05T17:26:57.536 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-12-05T17:26:57.538 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-12-05T17:26:57.573 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:57 +0000 (0:00:00.042) 0:01:19.455 ***** 2020-12-05T17:26:57.579 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-12-05T17:26:57.581 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:57.617 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:57 +0000 (0:00:00.043) 0:01:19.499 ***** 2020-12-05T17:26:57.643 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-12-05T17:26:57.645 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:57.679 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:57 +0000 (0:00:00.061) 0:01:19.560 ***** 2020-12-05T17:26:57.705 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-12-05T17:26:57.707 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:57.743 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:57 +0000 (0:00:00.064) 0:01:19.625 ***** 2020-12-05T17:26:57.777 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:57 +0000 (0:00:00.033) 0:01:19.658 ***** 2020-12-05T17:26:57.782 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-12-05T17:26:57.784 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:57.818 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:57 +0000 (0:00:00.041) 0:01:19.700 ***** 2020-12-05T17:26:57.824 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-12-05T17:26:57.825 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:57.860 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:57 +0000 (0:00:00.041) 0:01:19.742 ***** 2020-12-05T17:26:57.866 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-12-05T17:26:57.867 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:57.902 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:57 +0000 (0:00:00.042) 0:01:19.784 ***** 2020-12-05T17:26:57.908 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-12-05T17:26:57.910 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:57.946 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:57 +0000 (0:00:00.043) 0:01:19.827 ***** 2020-12-05T17:26:57.951 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-12-05T17:26:57.953 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:57.988 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:57 +0000 (0:00:00.041) 0:01:19.869 ***** 2020-12-05T17:26:57.993 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:26:57.995 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-12-05T17:26:58.030 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:58 +0000 (0:00:00.042) 0:01:19.912 ***** 2020-12-05T17:26:58.036 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-12-05T17:26:58.038 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.073 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:58 +0000 (0:00:00.042) 0:01:19.955 ***** 2020-12-05T17:26:58.088 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-12-05T17:26:58.090 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.100 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.102 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.139 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:58 +0000 (0:00:00.065) 0:01:20.021 ***** 2020-12-05T17:26:58.147 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-12-05T17:26:58.148 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.150 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} skipping: [smithi061.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.151 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.164 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.165 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.175 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:26:58.177 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi061.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.212 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:58 +0000 (0:00:00.072) 0:01:20.093 ***** 2020-12-05T17:26:58.218 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.285 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:58 +0000 (0:00:00.073) 0:01:20.166 ***** 2020-12-05T17:26:58.291 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.293 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.303 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.305 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi002.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-Z7OWrnoNVLFO33ZqvS1UNeFEKSkYadZ2XNyIfYpMSoNUDqzjd77tgI2g655W0upY'], 'uuids': ['6a8bb42b-313b-495e-a575-967568e2240f']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.151675", "end": "2020-12-05 17:26:58.283199", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-Z7OWrnoNVLFO33ZqvS1UNeFEKSkYadZ2XNyIfYpMSoNUDqzjd77tgI2g655W0upY"], "labels": [], "masters": [], "uuids": ["6a8bb42b-313b-495e-a575-967568e2240f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:26:57.131524", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old par 2020-12-05T17:26:58.307 INFO:teuthology.task.ansible.out:tition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:26:58.318 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HPVS', 'wwn-0x5000c500871a96d1'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HPVS-part1', 'wwn-0x5000c500871a96d1-part1'], 'uuids': ['d8c38c2f-ff33-4bf4-853e-25622c47cb18']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500871a96d1', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W4HPVS', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HPVS", "wwn-0x5000c500871a96d1"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HPVS-part1", "wwn-0x5000c500871a96d1-part1"], "labels": [], "masters": [], "uuids": ["d8c38c2f-ff33-4bf4-853e-25622c47cb18"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4HPVS", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871a96d1"}}, "skip_reason": 2020-12-05T17:26:58.320 INFO:teuthology.task.ansible.out:"Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** 2020-12-05T17:26:58.321 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.321 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.333 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-12-05T17:26:58.344 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.355 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.367 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.378 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.389 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.400 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.411 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.422 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.465 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:58 +0000 (0:00:00.180) 0:01:20.347 ***** 2020-12-05T17:26:58.472 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-12-05T17:26:58.474 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003330", "end": "2020-12-05 17:26:58.431855", "rc": 0, "start": "2020-12-05 17:26:58.428525", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-12-05T17:26:58.514 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:58 +0000 (0:00:00.049) 0:01:20.396 ***** 2020-12-05T17:26:58.541 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-12-05T17:26:58.543 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.582 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:58 +0000 (0:00:00.067) 0:01:20.463 ***** 2020-12-05T17:26:58.608 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-12-05T17:26:58.609 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.644 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:58 +0000 (0:00:00.062) 0:01:20.525 ***** 2020-12-05T17:26:58.670 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-12-05T17:26:58.672 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.706 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:58 +0000 (0:00:00.062) 0:01:20.587 ***** 2020-12-05T17:26:58.712 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-12-05T17:26:58.714 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003540", "end": "2020-12-05 17:26:58.661314", "rc": 0, "start": "2020-12-05 17:26:58.657774", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-12-05T17:26:58.752 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:58 +0000 (0:00:00.046) 0:01:20.634 ***** 2020-12-05T17:26:58.758 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-12-05T17:26:58.759 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-iuDKVQlK8CuPQcSO8QqzZYqsR03q8qQf6fPwBXsI5P8eIQGdCVg0WFwakLI6v5aj'], 'uuids': ['1cbc9f37-a96b-48fc-96bd-b1f5c7e2cdcc']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.116511", "end": "2020-12-05 17:26:58.892757", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-iuDKVQlK8CuPQcSO8QqzZYqsR03q8qQf6fPwBXsI5P8eIQGdCVg0WFwakLI6v5aj"], "labels": [], "masters": [], "uuids": ["1cbc9f37-a96b-48fc-96bd-b1f5c7e2cdcc"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:26:57.776246", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:26:58.761 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4J2LL', 'wwn-0x5000c500871b79da'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4J2LL-part1', 'wwn-0x5000c500871b79da-part1'], 'uuids': ['d8c38c2f-ff33-4bf4-853e-25622c47cb18']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500871b79da', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W4J2LL', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2LL", "wwn-0x5000c500871b79da"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2LL-part1", "wwn-0x5000c500871b79da-part1"], "labels": [], "masters": [], "uuids": ["d8c38c2f-ff33-4bf4-853e-25622c47cb18"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4J2LL", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871b79da"}}, "skip_reason": 2020-12-05T17:26:58.762 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-12-05T17:26:58.783 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-12-05T17:26:58.784 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.827 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:58 +0000 (0:00:00.074) 0:01:20.708 ***** 2020-12-05T17:26:58.854 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-12-05T17:26:58.855 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.890 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:58 +0000 (0:00:00.063) 0:01:20.772 ***** 2020-12-05T17:26:58.916 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-12-05T17:26:58.918 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.966 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:58 +0000 (0:00:00.076) 0:01:20.848 ***** 2020-12-05T17:26:58.976 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-12-05T17:26:58.977 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-mafM79ffexrgbdCaP1FsWthRDmjRXJ0Fi2El0HNcFH3ltF7mu0TWQc2CJtpAf6xu'], 'uuids': ['93dd1baf-fdac-44fd-b9c8-f681dc07569f']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.629965", "end": "2020-12-05 17:26:59.247883", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-mafM79ffexrgbdCaP1FsWthRDmjRXJ0Fi2El0HNcFH3ltF7mu0TWQc2CJtpAf6xu"], "labels": [], "masters": [], "uuids": ["93dd1baf-fdac-44fd-b9c8-f681dc07569f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:26:57.617918", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:26:58.980 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:58.981 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:59.591 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '937703088', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-BxaNel-Q3hm-hCX4-Hrxd-HdN9-TkgC-5ui8Jy', 'nvme-INTEL_SSDPED1D480GA_PHMB751300EV480DGN', 'nvme-nvme.8086-50484d42373531333030455634383044474e-494e54454c2053534450454431443438304741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPED1D480GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '447.13 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.157909", "end": "2020-12-05 17:26:59.572318", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series", "links": {"ids": ["lvm-pv-uuid-BxaNel-Q3hm-hCX4-Hrxd-HdN9-TkgC-5ui8Jy", "nvme-INTEL_SSDPED1D480GA_PHMB751300EV480DGN", "nvme-nvme.8086-50484d42373531333030455634383044474e-494e54454c2053534450454431443438304741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPED1D480GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "937703088", "sectorsize": "512", "size": "447.13 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:26:58.414409", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now pa 2020-12-05T17:26:59.593 INFO:teuthology.task.ansible.out:rtition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:26:59.603 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:59.614 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-12-05T17:26:59.616 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"cache_update_time": 1607189009, "cache_updated": false, "changed": false} TASK [testnode : Zap all non-root disks] *************************************** skipping: [smithi002.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:59.662 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:59 +0000 (0:00:00.695) 0:01:21.544 ***** 2020-12-05T17:26:59.690 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-12-05T17:26:59.692 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:59.726 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:59 +0000 (0:00:00.063) 0:01:21.607 ***** 2020-12-05T17:26:59.762 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-12-05T17:26:59.764 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => (item={'block_used': 13755770, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226339338, 'size_available': 927085928448, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149528, 'block_size': 4096, 'inode_available': 60905448}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226339338, "block_size": 4096, "block_total": 240095108, "block_used": 13755770, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905448, "inode_total": 61054976, "inode_used": 149528, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927085928448, "size_total": 983429562368, "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}} 2020-12-05T17:26:59.801 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:59 +0000 (0:00:00.075) 0:01:21.683 ***** 2020-12-05T17:26:59.860 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-12-05T17:26:59.862 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"cache_update_time": 1607188959, "cache_updated": false, "changed": false} 2020-12-05T17:26:59.897 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:59 +0000 (0:00:00.095) 0:01:21.778 ***** 2020-12-05T17:26:59.923 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-12-05T17:26:59.925 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:26:59.959 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:26:59 +0000 (0:00:00.062) 0:01:21.841 ***** 2020-12-05T17:26:59.965 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-12-05T17:26:59.967 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.009312", "end": "2020-12-05 17:27:00.089898", "rc": 0, "start": "2020-12-05 17:27:00.080586", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:00.002 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:00 +0000 (0:00:00.043) 0:01:21.884 ***** 2020-12-05T17:27:00.008 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-12-05T17:27:00.009 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => (item={'block_used': 13755779, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226339329, 'size_available': 927085891584, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149528, 'block_size': 4096, 'inode_available': 60905448}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226339329, "block_size": 4096, "block_total": 240095108, "block_used": 13755779, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905448, "inode_total": 61054976, "inode_used": 149528, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927085891584, "size_total": 983429562368, "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}} 2020-12-05T17:27:00.046 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:00 +0000 (0:00:00.043) 0:01:21.928 ***** 2020-12-05T17:27:00.053 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-12-05T17:27:00.055 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '937703088', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-DQt78P-pY0e-D4x0-pZKr-gB7Z-kpjL-1E1MDV', 'nvme-INTEL_SSDPED1D480GA_PHMB747402DN480DGN', 'nvme-nvme.8086-50484d42373437343032444e34383044474e-494e54454c2053534450454431443438304741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPED1D480GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '447.13 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.171262", "end": "2020-12-05 17:27:00.188332", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series", "links": {"ids": ["lvm-pv-uuid-DQt78P-pY0e-D4x0-pZKr-gB7Z-kpjL-1E1MDV", "nvme-INTEL_SSDPED1D480GA_PHMB747402DN480DGN", "nvme-nvme.8086-50484d42373437343032444e34383044474e-494e54454c2053534450454431443438304741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPED1D480GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "937703088", "sectorsize": "512", "size": "447.13 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:26:59.017070", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now pa 2020-12-05T17:27:00.056 INFO:teuthology.task.ansible.out:rtition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [smithi032.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Condit 2020-12-05T17:27:00.058 INFO:teuthology.task.ansible.out:ional result was False"} TASK [testnode : Unmount any non-root mountpoints] ***************************** skipping: [smithi061.front.sepia.ceph.com] => (item={'block_used': 13755770, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226339338, 'size_available': 927085928448, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149528, 'block_size': 4096, 'inode_available': 60905448}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226339338, "block_size": 4096, "block_total": 240095108, "block_used": 13755770, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905448, "inode_total": 61054976, "inode_used": 149528, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927085928448, "size_total": 983429562368, "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:00.098 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:00 +0000 (0:00:00.051) 0:01:21.980 ***** 2020-12-05T17:27:00.126 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-12-05T17:27:00.162 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:00 +0000 (0:00:00.063) 0:01:22.044 ***** 2020-12-05T17:27:00.192 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-12-05T17:27:00.194 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.009538", "end": "2020-12-05 17:27:00.203934", "rc": 0, "start": "2020-12-05 17:27:00.194396", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [testnode : Zap all non-root disks] *************************************** 2020-12-05T17:27:00.196 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-mafM79ffexrgbdCaP1FsWthRDmjRXJ0F5hQOytNS5EU7UBd9BWiSe98dhZUUPaOs'], 'uuids': ['a5be18d8-dccc-443e-b3c7-b306789fd074']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.115703", "end": "2020-12-05 17:27:00.502590", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-mafM79ffexrgbdCaP1FsWthRDmjRXJ0F5hQOytNS5EU7UBd9BWiSe98dhZUUPaOs"], "labels": [], "masters": [], "uuids": ["a5be18d8-dccc-443e-b3c7-b306789fd074"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:26:59.386887", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:27:00.233 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:00 +0000 (0:00:00.070) 0:01:22.115 ***** 2020-12-05T17:27:00.240 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4J2QS', 'wwn-0x5000c500871b68bf'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4J2QS-part1', 'wwn-0x5000c500871b68bf-part1'], 'uuids': ['d8c38c2f-ff33-4bf4-853e-25622c47cb18']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500871b68bf', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W4J2QS', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2QS", "wwn-0x5000c500871b68bf"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2QS-part1", "wwn-0x5000c500871b68bf-part1"], "labels": [], "masters": [], "uuids": ["d8c38c2f-ff33-4bf4-853e-25622c47cb18"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4J2QS", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871b68bf"}}, "skip_reason": 2020-12-05T17:27:00.242 INFO:teuthology.task.ansible.out:"Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:00.265 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-12-05T17:27:00.266 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'block_used': 13755779, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226339329, 'size_available': 927085891584, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149528, 'block_size': 4096, 'inode_available': 60905448}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226339329, "block_size": 4096, "block_total": 240095108, "block_used": 13755779, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905448, "inode_total": 61054976, "inode_used": 149528, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927085891584, "size_total": 983429562368, "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:00.302 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:00 +0000 (0:00:00.069) 0:01:22.184 ***** 2020-12-05T17:27:00.329 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-12-05T17:27:00.367 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:00 +0000 (0:00:00.064) 0:01:22.248 ***** 2020-12-05T17:27:00.416 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-12-05T17:27:00.418 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:00.914 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-Z7OWrnoNVLFO33ZqvS1UNeFEKSkYadZ2JM00qunYnSyvva2vpeQwo8DfbUrmfJ56'], 'uuids': ['c898f6a0-0422-4c55-a507-fe888fa61353']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.179266", "end": "2020-12-05 17:27:00.888259", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-Z7OWrnoNVLFO33ZqvS1UNeFEKSkYadZ2JM00qunYnSyvva2vpeQwo8DfbUrmfJ56"], "labels": [], "masters": [], "uuids": ["c898f6a0-0422-4c55-a507-fe888fa61353"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:26:59.708993", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:27:00.926 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:00.927 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:01.296 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-iuDKVQlK8CuPQcSO8QqzZYqsR03q8qQfN2ekduGV8ASPBVl1HWytHfRmX5VeIDKM'], 'uuids': ['f430c299-86e4-480d-a61a-3bd13788768a']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.136126", "end": "2020-12-05 17:27:01.467833", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-iuDKVQlK8CuPQcSO8QqzZYqsR03q8qQfN2ekduGV8ASPBVl1HWytHfRmX5VeIDKM"], "labels": [], "masters": [], "uuids": ["f430c299-86e4-480d-a61a-3bd13788768a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:00.331707", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:27:01.308 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:01.319 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:01.487 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-1qYrAu-HJYg-dQCZ-v1Km-efx4-lmmj-jGLo30', 'nvme-INTEL_SSDPEDMD400G4_CVFT53300040400BGN', 'nvme-nvme.8086-43564654353333303030343034303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.161276", "end": "2020-12-05 17:27:01.784288", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-1qYrAu-HJYg-dQCZ-v1Km-efx4-lmmj-jGLo30", "nvme-INTEL_SSDPEDMD400G4_CVFT53300040400BGN", "nvme-nvme.8086-43564654353333303030343034303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:00.623012", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2020-12-05T17:27:01.489 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:27:01.499 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:01.510 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:01.989 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-gkfB7RSHLTfof900eU10N7HgmzPrcbkaa8tlRZx3z5xZYNan2YNrws0JNG5T7fyn'], 'uuids': ['7029c252-e032-4cfb-82d2-74d8d9d72254']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.645600", "end": "2020-12-05 17:27:02.119265", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-gkfB7RSHLTfof900eU10N7HgmzPrcbkaa8tlRZx3z5xZYNan2YNrws0JNG5T7fyn"], "labels": [], "masters": [], "uuids": ["7029c252-e032-4cfb-82d2-74d8d9d72254"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:00.473665", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:27:02.001 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:02.002 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:02.193 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-Z7OWrnoNVLFO33ZqvS1UNeFEKSkYadZ2iwvGwX9Y9oGkJavZ07c7TB6NtQJqma8a'], 'uuids': ['6658eb06-4854-469f-b44c-d952cdf8acb5']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.142392", "end": "2020-12-05 17:27:02.169936", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-Z7OWrnoNVLFO33ZqvS1UNeFEKSkYadZ2iwvGwX9Y9oGkJavZ07c7TB6NtQJqma8a"], "labels": [], "masters": [], "uuids": ["6658eb06-4854-469f-b44c-d952cdf8acb5"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:01.027544", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:27:02.194 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:02.405 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-oEDae8BcSUQDZ9eYC9mvZ0vFo80IWggbwfOHqNyQTiDCNoe3C5mRImTqHwVSS1Mu'], 'uuids': ['b78ca16a-2ec3-4a02-881f-ef4737303b81']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.872145", "end": "2020-12-05 17:27:02.410663", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-oEDae8BcSUQDZ9eYC9mvZ0vFo80IWggbwfOHqNyQTiDCNoe3C5mRImTqHwVSS1Mu"], "labels": [], "masters": [], "uuids": ["b78ca16a-2ec3-4a02-881f-ef4737303b81"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:00.538518", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:27:02.416 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:02.428 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:02.575 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-iuDKVQlK8CuPQcSO8QqzZYqsR03q8qQfsTLdxp0UNf1zt4Gwx504Hau4CT3LW65e'], 'uuids': ['51c541ab-4ca4-4134-b2e0-6fa95bc2b5f9']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.133693", "end": "2020-12-05 17:27:02.742561", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-iuDKVQlK8CuPQcSO8QqzZYqsR03q8qQfsTLdxp0UNf1zt4Gwx504Hau4CT3LW65e"], "labels": [], "masters": [], "uuids": ["51c541ab-4ca4-4134-b2e0-6fa95bc2b5f9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:01.608868", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:27:02.586 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:02.786 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-mafM79ffexrgbdCaP1FsWthRDmjRXJ0FR9mUfuDQSms934AGXKkuOYwBG4lkxFqm'], 'uuids': ['926a5ddd-6f8a-4069-99de-f817a5d365fc']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.163044", "end": "2020-12-05 17:27:03.084090", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-mafM79ffexrgbdCaP1FsWthRDmjRXJ0FR9mUfuDQSms934AGXKkuOYwBG4lkxFqm"], "labels": [], "masters": [], "uuids": ["926a5ddd-6f8a-4069-99de-f817a5d365fc"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:01.921046", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:27:02.787 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:02.798 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:03.312 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-gkfB7RSHLTfof900eU10N7HgmzPrcbkaD2cbFMQBpzmdVcVfcRVrq5brPh33985k'], 'uuids': ['4e9bc91e-6ca3-47d2-a295-fd605bb913e0']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.181414", "end": "2020-12-05 17:27:03.444774", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-gkfB7RSHLTfof900eU10N7HgmzPrcbkaD2cbFMQBpzmdVcVfcRVrq5brPh33985k"], "labels": [], "masters": [], "uuids": ["4e9bc91e-6ca3-47d2-a295-fd605bb913e0"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:02.263360", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:27:03.313 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4W2W8', 'wwn-0x5000c50087b8cd81'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4W2W8-part1', 'wwn-0x5000c50087b8cd81-part1'], 'uuids': ['d8c38c2f-ff33-4bf4-853e-25622c47cb18']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50087b8cd81', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W4W2W8', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4W2W8", "wwn-0x5000c50087b8cd81"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4W2W8-part1", "wwn-0x5000c50087b8cd81-part1"], "labels": [], "masters": [], "uuids": ["d8c38c2f-ff33-4bf4-853e-25622c47cb18"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4W2W8", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50087b8cd81"}}, "skip_reason": 2020-12-05T17:27:03.314 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-12-05T17:27:03.491 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-Z7OWrnoNVLFO33ZqvS1UNeFEKSkYadZ21OGe0EYBUp0CBRripnf6Iu9OTRte5eRf'], 'uuids': ['3d62348b-4cb1-4f4a-992d-8d9e3abe3fe2']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.172294", "end": "2020-12-05 17:27:03.466183", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-Z7OWrnoNVLFO33ZqvS1UNeFEKSkYadZ21OGe0EYBUp0CBRripnf6Iu9OTRte5eRf"], "labels": [], "masters": [], "uuids": ["3d62348b-4cb1-4f4a-992d-8d9e3abe3fe2"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:02.293889", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:27:03.538 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:03 +0000 (0:00:03.171) 0:01:25.420 ***** 2020-12-05T17:27:03.576 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:03.578 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:03.652 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi053.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-oEDae8BcSUQDZ9eYC9mvZ0vFo80IWggbr9RfW4m9OPGgfl1eE7HsAsQLtPZnudpn'], 'uuids': ['90bcf67e-7a6e-4318-b687-85298f16ea80']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.126260", "end": "2020-12-05 17:27:03.658894", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-oEDae8BcSUQDZ9eYC9mvZ0vFo80IWggbr9RfW4m9OPGgfl1eE7HsAsQLtPZnudpn"], "labels": [], "masters": [], "uuids": ["90bcf67e-7a6e-4318-b687-85298f16ea80"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:02.532634", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk u 2020-12-05T17:27:03.653 INFO:teuthology.task.ansible.out:sing fdisk or", "other utilities."]} 2020-12-05T17:27:03.664 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4J2LS', 'wwn-0x5000c500871b78e7'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4J2LS-part1', 'wwn-0x5000c500871b78e7-part1'], 'uuids': ['d8c38c2f-ff33-4bf4-853e-25622c47cb18']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500871b78e7', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W4J2LS', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2LS", "wwn-0x5000c500871b78e7"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2LS-part1", "wwn-0x5000c500871b78e7-part1"], "labels": [], "masters": [], "uuids": ["d8c38c2f-ff33-4bf4-853e-25622c47cb18"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4J2LS", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871b78e7"}}, "skip_reason": 2020-12-05T17:27:03.665 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-12-05T17:27:03.813 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:03.815 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-Z7OWrnoNVLFO33ZqvS1UNeFEKSkYadZ2uEXYeJclX20MT3GcOv0gU4WzQCq9Mn3J'], 'uuids': ['b069103d-6552-435e-a95a-94a6a28285d8']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.118174", "end": "2020-12-05 17:27:03.795017", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-Z7OWrnoNVLFO33ZqvS1UNeFEKSkYadZ2uEXYeJclX20MT3GcOv0gU4WzQCq9Mn3J"], "labels": [], "masters": [], "uuids": ["b069103d-6552-435e-a95a-94a6a28285d8"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:03.676843", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.113485 s, 1.0 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.113485 s, 1.0 GB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:03.826 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:03.827 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-12-05T17:27:03.827 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-iuDKVQlK8CuPQcSO8QqzZYqsR03q8qQfi0nzyA9qhcBQ3MPEkaaWZy4jce7oJcAm'], 'uuids': ['acc39d6d-1e1c-4b08-9572-51aa27d0a459']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.137210", "end": "2020-12-05 17:27:04.000674", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-iuDKVQlK8CuPQcSO8QqzZYqsR03q8qQfi0nzyA9qhcBQ3MPEkaaWZy4jce7oJcAm"], "labels": [], "masters": [], "uuids": ["acc39d6d-1e1c-4b08-9572-51aa27d0a459"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:02.863464", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk u 2020-12-05T17:27:03.829 INFO:teuthology.task.ansible.out:sing fdisk or", "other utilities."]} 2020-12-05T17:27:03.840 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:03.842 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:03.895 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:03 +0000 (0:00:00.356) 0:01:25.777 ***** 2020-12-05T17:27:03.939 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:04.024 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-12-05T17:27:04.026 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-mafM79ffexrgbdCaP1FsWthRDmjRXJ0FdCRlPz8BnAi4RefAuVD96imm6ejgGMI7'], 'uuids': ['4363a1e3-38da-4042-ae0f-5019fa501772']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.098338", "end": "2020-12-05 17:27:04.317452", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-mafM79ffexrgbdCaP1FsWthRDmjRXJ0FdCRlPz8BnAi4RefAuVD96imm6ejgGMI7"], "labels": [], "masters": [], "uuids": ["4363a1e3-38da-4042-ae0f-5019fa501772"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:03.219114", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:27:04.036 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:04.058 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:04.060 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-Z7OWrnoNVLFO33ZqvS1UNeFEKSkYadZ2XNyIfYpMSoNUDqzjd77tgI2g655W0upY'], 'uuids': ['6a8bb42b-313b-495e-a575-967568e2240f']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.111579", "end": "2020-12-05 17:27:04.037215", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-Z7OWrnoNVLFO33ZqvS1UNeFEKSkYadZ2XNyIfYpMSoNUDqzjd77tgI2g655W0upY"], "labels": [], "masters": [], "uuids": ["6a8bb42b-313b-495e-a575-967568e2240f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:03.925636", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.106864 s, 1.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.106864 s, 1.1 GB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:04.070 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HPVS', 'wwn-0x5000c500871a96d1'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HPVS-part1', 'wwn-0x5000c500871a96d1-part1'], 'uuids': ['d8c38c2f-ff33-4bf4-853e-25622c47cb18']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500871a96d1', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W4HPVS', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HPVS", "wwn-0x5000c500871a96d1"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HPVS-part1", "wwn-0x5000c500871a96d1-part1"], "labels": [], "masters": [], "uuids": ["d8c38c2f-ff33-4bf4-853e-25622c47cb18"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4HPVS", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871a96d1"}}, "skip_reason": 2020-12-05T17:27:04.071 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-12-05T17:27:04.165 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-iuDKVQlK8CuPQcSO8QqzZYqsR03q8qQf4rRVa18u6jjT3xFF9hwvR8HolmZCTziT'], 'uuids': ['01cc38fa-b575-4e99-b094-a907bbb92eab']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.106674", "end": "2020-12-05 17:27:04.336656", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-iuDKVQlK8CuPQcSO8QqzZYqsR03q8qQf4rRVa18u6jjT3xFF9hwvR8HolmZCTziT"], "labels": [], "masters": [], "uuids": ["01cc38fa-b575-4e99-b094-a907bbb92eab"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:04.229982", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.102421 s, 1.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.102421 s, 1.1 GB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:04.177 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:04.188 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:04.231 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '937703088', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-BxaNel-Q3hm-hCX4-Hrxd-HdN9-TkgC-5ui8Jy', 'nvme-INTEL_SSDPED1D480GA_PHMB751300EV480DGN', 'nvme-nvme.8086-50484d42373531333030455634383044474e-494e54454c2053534450454431443438304741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPED1D480GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '447.13 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.049259", "end": "2020-12-05 17:27:04.205158", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series", "links": {"ids": ["lvm-pv-uuid-BxaNel-Q3hm-hCX4-Hrxd-HdN9-TkgC-5ui8Jy", "nvme-INTEL_SSDPED1D480GA_PHMB751300EV480DGN", "nvme-nvme.8086-50484d42373531333030455634383044474e-494e54454c2053534450454431443438304741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPED1D480GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "937703088", "sectorsize": "512", "size": "447.13 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:04.155899", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0454057 s, 2.5 GB/s", "stderr_lines": [" 2020-12-05T17:27:04.233 INFO:teuthology.task.ansible.out:110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0454057 s, 2.5 GB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:04.244 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:04.255 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:04.423 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-iuDKVQlK8CuPQcSO8QqzZYqsR03q8qQf6fPwBXsI5P8eIQGdCVg0WFwakLI6v5aj'], 'uuids': ['1cbc9f37-a96b-48fc-96bd-b1f5c7e2cdcc']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.105161", "end": "2020-12-05 17:27:04.587965", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-iuDKVQlK8CuPQcSO8QqzZYqsR03q8qQf6fPwBXsI5P8eIQGdCVg0WFwakLI6v5aj"], "labels": [], "masters": [], "uuids": ["1cbc9f37-a96b-48fc-96bd-b1f5c7e2cdcc"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:04.482804", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.100559 s, 1.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.100559 s, 1.1 GB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:04.424 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4J2LL', 'wwn-0x5000c500871b79da'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4J2LL-part1', 'wwn-0x5000c500871b79da-part1'], 'uuids': ['d8c38c2f-ff33-4bf4-853e-25622c47cb18']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500871b79da', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W4J2LL', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2LL", "wwn-0x5000c500871b79da"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2LL-part1", "wwn-0x5000c500871b79da-part1"], "labels": [], "masters": [], "uuids": ["d8c38c2f-ff33-4bf4-853e-25622c47cb18"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4J2LL", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871b79da"}}, "skip_reason": 2020-12-05T17:27:04.426 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-12-05T17:27:04.478 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-Z7OWrnoNVLFO33ZqvS1UNeFEKSkYadZ2JM00qunYnSyvva2vpeQwo8DfbUrmfJ56'], 'uuids': ['c898f6a0-0422-4c55-a507-fe888fa61353']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.109017", "end": "2020-12-05 17:27:04.458327", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-Z7OWrnoNVLFO33ZqvS1UNeFEKSkYadZ2JM00qunYnSyvva2vpeQwo8DfbUrmfJ56"], "labels": [], "masters": [], "uuids": ["c898f6a0-0422-4c55-a507-fe888fa61353"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:04.349310", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.104826 s, 1.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.104826 s, 1.1 GB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:04.489 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:04.501 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:04.597 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-12-05T17:27:04.599 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-EnaFJs-wJ1a-Viuj-eDek-8iup-i26S-Ycck0V', 'nvme-INTEL_SSDPEDMD400G4_CVFT534300NV400BGN', 'nvme-nvme.8086-435646543533343330304e5634303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.157883", "end": "2020-12-05 17:27:04.718854", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-EnaFJs-wJ1a-Viuj-eDek-8iup-i26S-Ycck0V", "nvme-INTEL_SSDPEDMD400G4_CVFT534300NV400BGN", "nvme-nvme.8086-435646543533343330304e5634303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:03.560971", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2020-12-05T17:27:04.600 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi032.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '937703088', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-DQt78P-pY0e-D4x0-pZKr-gB7Z-kpjL-1E1MDV', 'nvme-INTEL_SSDPED1D480GA_PHMB747402DN480DGN', 'nvme-nvme.8086-50484d42373437343032444e34383044474e-494e54454c2053534450454431443438304741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPED1D480GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '447.13 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.037143", "end": "2020-12-05 17:27:04.761239", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series", "links": {"ids": ["lvm-pv-uuid-DQt78P-pY0e-D4x0-pZKr-gB7Z-kpjL-1E1MDV", "nvme-INTEL_SSDPED1D480GA_PHMB747402DN480DGN", "nvme-nvme.8086-50484d42373437343032444e34383044474e-494e54454c2053534450454431443438304741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPED1D480GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "secto 2020-12-05T17:27:04.602 INFO:teuthology.task.ansible.out:rs": "937703088", "sectorsize": "512", "size": "447.13 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:04.724096", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0331997 s, 3.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0331997 s, 3.5 GB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:04.610 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-12-05T17:27:04.612 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [smithi032.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:04.612 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** skippi 2020-12-05T17:27:04.614 INFO:teuthology.task.ansible.out:ng: [smithi061.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [smithi032.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:04.731 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-Z7OWrnoNVLFO33ZqvS1UNeFEKSkYadZ2iwvGwX9Y9oGkJavZ07c7TB6NtQJqma8a'], 'uuids': ['6658eb06-4854-469f-b44c-d952cdf8acb5']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.111282", "end": "2020-12-05 17:27:04.702927", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-Z7OWrnoNVLFO33ZqvS1UNeFEKSkYadZ2iwvGwX9Y9oGkJavZ07c7TB6NtQJqma8a"], "labels": [], "masters": [], "uuids": ["6658eb06-4854-469f-b44c-d952cdf8acb5"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:04.591645", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.106704 s, 1.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.106704 s, 1.1 GB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:04.733 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:04.827 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-iuDKVQlK8CuPQcSO8QqzZYqsR03q8qQfN2ekduGV8ASPBVl1HWytHfRmX5VeIDKM'], 'uuids': ['f430c299-86e4-480d-a61a-3bd13788768a']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.101669", "end": "2020-12-05 17:27:04.999667", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-iuDKVQlK8CuPQcSO8QqzZYqsR03q8qQfN2ekduGV8ASPBVl1HWytHfRmX5VeIDKM"], "labels": [], "masters": [], "uuids": ["f430c299-86e4-480d-a61a-3bd13788768a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:04.897998", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0990998 s, 1.2 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0990998 s, 1.2 GB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:04.839 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:04.850 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:04.968 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-12-05T17:27:04.969 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '937703088', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-69nwQE-o6IF-MAFw-QnL7-JdAn-Mcug-kNzECc', 'nvme-INTEL_SSDPED1D480GA_PHMB751300G8480DGN', 'nvme-nvme.8086-50484d42373531333030473834383044474e-494e54454c2053534450454431443438304741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPED1D480GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '447.13 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.185999", "end": "2020-12-05 17:27:04.962904", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series", "links": {"ids": ["lvm-pv-uuid-69nwQE-o6IF-MAFw-QnL7-JdAn-Mcug-kNzECc", "nvme-INTEL_SSDPED1D480GA_PHMB751300G8480DGN", "nvme-nvme.8086-50484d42373531333030473834383044474e-494e54454c2053534450454431443438304741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPED1D480GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "937703088", "sectorsize": "512", "size": "447.13 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:03.776905", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now pa 2020-12-05T17:27:04.971 INFO:teuthology.task.ansible.out:rtition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:27:04.971 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:04.972 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-Z7OWrnoNVLFO33ZqvS1UNeFEKSkYadZ21OGe0EYBUp0CBRripnf6Iu9OTRte5eRf'], 'uuids': ['3d62348b-4cb1-4f4a-992d-8d9e3abe3fe2']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.109719", "end": "2020-12-05 17:27:04.941587", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-Z7OWrnoNVLFO33ZqvS1UNeFEKSkYadZ21OGe0EYBUp0CBRripnf6Iu9OTRte5eRf"], "labels": [], "masters": [], "uuids": ["3d62348b-4cb1-4f4a-992d-8d9e3abe3fe2"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:04.831868", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.105148 s, 1.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.105148 s, 1.1 GB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:04.984 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-12-05T17:27:04.986 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:05.019 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:05 +0000 (0:00:01.123) 0:01:26.901 ***** 2020-12-05T17:27:05.026 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:05.070 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:05.072 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-iuDKVQlK8CuPQcSO8QqzZYqsR03q8qQfsTLdxp0UNf1zt4Gwx504Hau4CT3LW65e'], 'uuids': ['51c541ab-4ca4-4134-b2e0-6fa95bc2b5f9']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.108006", "end": "2020-12-05 17:27:05.242341", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-iuDKVQlK8CuPQcSO8QqzZYqsR03q8qQfsTLdxp0UNf1zt4Gwx504Hau4CT3LW65e"], "labels": [], "masters": [], "uuids": ["51c541ab-4ca4-4134-b2e0-6fa95bc2b5f9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:05.134335", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.103831 s, 1.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.103831 s, 1.1 GB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:05.082 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:05.288 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-12-05T17:27:05.289 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-mafM79ffexrgbdCaP1FsWthRDmjRXJ0FPadlzmMo3lOroRqfLEAztEzctEoqqzax'], 'uuids': ['769bdba5-e7ee-448c-8c3c-422e6f4915f7']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.136169", "end": "2020-12-05 17:27:05.583487", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-mafM79ffexrgbdCaP1FsWthRDmjRXJ0FPadlzmMo3lOroRqfLEAztEzctEoqqzax"], "labels": [], "masters": [], "uuids": ["769bdba5-e7ee-448c-8c3c-422e6f4915f7"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:04.447318", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:27:05.337 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:05 +0000 (0:00:00.317) 0:01:27.218 ***** 2020-12-05T17:27:05.343 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:05.344 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-iuDKVQlK8CuPQcSO8QqzZYqsR03q8qQfi0nzyA9qhcBQ3MPEkaaWZy4jce7oJcAm'], 'uuids': ['acc39d6d-1e1c-4b08-9572-51aa27d0a459']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.108698", "end": "2020-12-05 17:27:05.485340", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-iuDKVQlK8CuPQcSO8QqzZYqsR03q8qQfi0nzyA9qhcBQ3MPEkaaWZy4jce7oJcAm"], "labels": [], "masters": [], "uuids": ["acc39d6d-1e1c-4b08-9572-51aa27d0a459"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:05.376642", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.104011 s, 1.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.104011 s, 1.1 GB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:05.355 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-12-05T17:27:05.357 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.184316", "end": "2020-12-05 17:27:05.312185", "rc": 0, "start": "2020-12-05 17:27:05.127869", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:05.392 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:05 +0000 (0:00:00.055) 0:01:27.273 ***** 2020-12-05T17:27:05.425 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:05 +0000 (0:00:00.033) 0:01:27.307 ***** 2020-12-05T17:27:05.432 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:05.434 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:05.570 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-12-05T17:27:05.571 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.047380", "end": "2020-12-05 17:27:05.552126", "rc": 0, "start": "2020-12-05 17:27:05.504746", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:05.606 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:05 +0000 (0:00:00.180) 0:01:27.488 ***** 2020-12-05T17:27:05.633 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-12-05T17:27:05.671 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:05 +0000 (0:00:00.065) 0:01:27.553 ***** 2020-12-05T17:27:05.677 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:05.679 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-mafM79ffexrgbdCaP1FsWthRDmjRXJ0Fi2El0HNcFH3ltF7mu0TWQc2CJtpAf6xu'], 'uuids': ['93dd1baf-fdac-44fd-b9c8-f681dc07569f']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.150528", "end": "2020-12-05 17:27:05.951741", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-mafM79ffexrgbdCaP1FsWthRDmjRXJ0Fi2El0HNcFH3ltF7mu0TWQc2CJtpAf6xu"], "labels": [], "masters": [], "uuids": ["93dd1baf-fdac-44fd-b9c8-f681dc07569f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:05.801213", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.14624 s, 789 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.14624 s, 789 MB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:05.679 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': ' 2020-12-05T17:27:05.681 INFO:teuthology.task.ansible.out:0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:05.691 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-12-05T17:27:05.693 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.132606", "end": "2020-12-05 17:27:05.855986", "rc": 0, "start": "2020-12-05 17:27:05.723380", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:05.728 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:05 +0000 (0:00:00.057) 0:01:27.610 ***** 2020-12-05T17:27:05.760 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:05 +0000 (0:00:00.032) 0:01:27.642 ***** 2020-12-05T17:27:05.766 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-12-05T17:27:05.768 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:05.802 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:05 +0000 (0:00:00.041) 0:01:27.684 ***** 2020-12-05T17:27:05.828 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-12-05T17:27:05.830 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:05.866 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:05 +0000 (0:00:00.064) 0:01:27.748 ***** 2020-12-05T17:27:05.895 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-12-05T17:27:05.897 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:05.943 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:05 +0000 (0:00:00.076) 0:01:27.825 ***** 2020-12-05T17:27:05.952 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:05.953 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-mafM79ffexrgbdCaP1FsWthRDmjRXJ0F5hQOytNS5EU7UBd9BWiSe98dhZUUPaOs'], 'uuids': ['a5be18d8-dccc-443e-b3c7-b306789fd074']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.152934", "end": "2020-12-05 17:27:06.233300", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-mafM79ffexrgbdCaP1FsWthRDmjRXJ0F5hQOytNS5EU7UBd9BWiSe98dhZUUPaOs"], "labels": [], "masters": [], "uuids": ["a5be18d8-dccc-443e-b3c7-b306789fd074"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:06.080366", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.148145 s, 779 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.148145 s, 779 MB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:05.954 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4J2QS', 'wwn-0x5000c500871b68bf'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4J2QS-part1', 'wwn-0x5000c500871b68bf-part1'], 'uuids': ['d8c38c2f-ff33-4bf4-853e-25622c47cb18']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500871b68bf', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W4J2QS', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2QS", "wwn-0x5000c500871b68bf"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2QS-part1", "wwn-0x5000c500871b68bf-part1"], "labels": [], "masters": [], "uuids": ["d8c38c2f-ff33-4bf4-853e-25622c47cb18"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4J2QS", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871b68bf"}}, "skip_reason": 2020-12-05T17:27:05.956 INFO:teuthology.task.ansible.out:"Conditional result was False"} TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi061.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-gkfB7RSHLTfof900eU10N7HgmzPrcbkafDRUG1bsqsPljpUt2IrEbI3TqnsRTCwA'], 'uuids': ['0c026eb7-f0af-4f63-b7b4-f87d0bbc209b']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.201141", "end": "2020-12-05 17:27:06.071275", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-gkfB7RSHLTfof900eU10N7HgmzPrcbkafDRUG1bsqsPljpUt2IrEbI3TqnsRTCwA"], "labels": [], "masters": [], "uuids": ["0c026eb7-f0af-4f63-b7b4-f87d0bbc209b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:04.870134", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! Y 2020-12-05T17:27:05.958 INFO:teuthology.task.ansible.out:ou may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:27:05.968 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:05.970 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-12-05T17:27:05.970 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.069449", "end": "2020-12-05 17:27:06.129914", "rc": 0, "start": "2020-12-05 17:27:06.060465", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:05.972 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-12-05T17:27:05.972 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:06.020 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:06 +0000 (0:00:00.076) 0:01:27.901 ***** 2020-12-05T17:27:06.047 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-12-05T17:27:06.084 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:06 +0000 (0:00:00.064) 0:01:27.966 ***** 2020-12-05T17:27:06.090 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-12-05T17:27:06.092 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-12-05T17:27:06.150 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:06 +0000 (0:00:00.064) 0:01:28.031 ***** 2020-12-05T17:27:06.156 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:06.157 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-1qYrAu-HJYg-dQCZ-v1Km-efx4-lmmj-jGLo30', 'nvme-INTEL_SSDPEDMD400G4_CVFT53300040400BGN', 'nvme-nvme.8086-43564654353333303030343034303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.048048", "end": "2020-12-05 17:27:06.412879", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-1qYrAu-HJYg-dQCZ-v1Km-efx4-lmmj-jGLo30", "nvme-INTEL_SSDPEDMD400G4_CVFT53300040400BGN", "nvme-nvme.8086-43564654353333303030343034303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:06.364831", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0438368 s, 2.6 GB/s", "std 2020-12-05T17:27:06.159 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0438368 s, 2.6 GB/s"], "stdout": "", "stdout_lines": []} TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:06.159 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:06.160 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:06.161 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:06.200 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:06 +0000 (0:00:00.050) 0:01:28.082 ***** 2020-12-05T17:27:06.207 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-12-05T17:27:06.208 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'block_used': 13755773, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226339335, 'size_available': 927085916160, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149528, 'block_size': 4096, 'inode_available': 60905448}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226339335, "block_size": 4096, "block_total": 240095108, "block_used": 13755773, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905448, "inode_total": 61054976, "inode_used": 149528, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927085916160, "size_total": 983429562368, "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:06.245 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:06 +0000 (0:00:00.045) 0:01:28.127 ***** 2020-12-05T17:27:06.252 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-12-05T17:27:06.254 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:06.290 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:06 +0000 (0:00:00.044) 0:01:28.172 ***** 2020-12-05T17:27:06.297 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-12-05T17:27:06.299 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-oEDae8BcSUQDZ9eYC9mvZ0vFo80IWggbrteC0NziBQlvfrT0ff8rL1UOhUsjlAAl'], 'uuids': ['f28c8f7e-d904-476e-8178-4c8d79f6217f']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.154770", "end": "2020-12-05 17:27:06.266569", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-oEDae8BcSUQDZ9eYC9mvZ0vFo80IWggbrteC0NziBQlvfrT0ff8rL1UOhUsjlAAl"], "labels": [], "masters": [], "uuids": ["f28c8f7e-d904-476e-8178-4c8d79f6217f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:05.111799", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} TASK [testnode : Combine list of non-root disk 2020-12-05T17:27:06.301 INFO:teuthology.task.ansible.out:s] ******************************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Zap all non-root disks] *************************************** skipping: [smithi053.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virt 2020-12-05T17:27:06.303 INFO:teuthology.task.ansible.out:ual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:06.337 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:06 +0000 (0:00:00.047) 0:01:28.219 ***** 2020-12-05T17:27:06.344 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-12-05T17:27:06.346 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:06.384 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:06 +0000 (0:00:00.046) 0:01:28.266 ***** 2020-12-05T17:27:06.391 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:27:06.392 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:06.428 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:06 +0000 (0:00:00.043) 0:01:28.310 ***** 2020-12-05T17:27:06.435 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-12-05T17:27:06.436 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:06.438 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi036.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-mafM79ffexrgbdCaP1FsWthRDmjRXJ0FR9mUfuDQSms934AGXKkuOYwBG4lkxFqm'], 'uuids': ['926a5ddd-6f8a-4069-99de-f817a5d365fc']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.150520", "end": "2020-12-05 17:27:06.716944", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-mafM79ffexrgbdCaP1FsWthRDmjRXJ0FR9mUfuDQSms934AGXKkuOYwBG4lkxFqm"], "labels": [], "masters": [], "uuids": ["926a5ddd-6f8a-4069-99de-f817a5d365fc"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:06.566424", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.147522 s, 782 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.147522 s, 782 MB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:06.439 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:06.441 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:06.476 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:06 +0000 (0:00:00.048) 0:01:28.358 ***** 2020-12-05T17:27:06.483 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:27:06.484 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:06.523 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:06 +0000 (0:00:00.046) 0:01:28.405 ***** 2020-12-05T17:27:06.550 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-12-05T17:27:06.552 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:06.586 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:06 +0000 (0:00:00.063) 0:01:28.468 ***** 2020-12-05T17:27:06.604 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-12-05T17:27:06.606 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-12-05T17:27:06.642 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:06 +0000 (0:00:00.055) 0:01:28.524 ***** 2020-12-05T17:27:06.649 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-12-05T17:27:06.650 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:06.686 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:06 +0000 (0:00:00.043) 0:01:28.567 ***** 2020-12-05T17:27:06.691 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-12-05T17:27:06.693 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'block_used': 13755772, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226339336, 'size_available': 927085920256, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149528, 'block_size': 4096, 'inode_available': 60905448}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226339336, "block_size": 4096, "block_total": 240095108, "block_used": 13755772, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905448, "inode_total": 61054976, "inode_used": 149528, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927085920256, "size_total": 983429562368, "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:06.728 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:06 +0000 (0:00:00.042) 0:01:28.610 ***** 2020-12-05T17:27:06.734 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:06.736 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-mafM79ffexrgbdCaP1FsWthRDmjRXJ0FdCRlPz8BnAi4RefAuVD96imm6ejgGMI7'], 'uuids': ['4363a1e3-38da-4042-ae0f-5019fa501772']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.151353", "end": "2020-12-05 17:27:06.993648", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-mafM79ffexrgbdCaP1FsWthRDmjRXJ0FdCRlPz8BnAi4RefAuVD96imm6ejgGMI7"], "labels": [], "masters": [], "uuids": ["4363a1e3-38da-4042-ae0f-5019fa501772"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:06.842295", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.146978 s, 785 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.146978 s, 785 MB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:06.737 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': 2020-12-05T17:27:06.738 INFO:teuthology.task.ansible.out: '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : Create logical_volumes dict] ********************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:06.772 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:06 +0000 (0:00:00.043) 0:01:28.654 ***** 2020-12-05T17:27:06.780 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-12-05T17:27:06.782 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:06.816 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:06 +0000 (0:00:00.043) 0:01:28.698 ***** 2020-12-05T17:27:06.844 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:06.888 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:06 +0000 (0:00:00.071) 0:01:28.770 ***** 2020-12-05T17:27:06.915 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:27:06.917 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:06.955 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:06 +0000 (0:00:00.066) 0:01:28.837 ***** 2020-12-05T17:27:06.983 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:06.984 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-mafM79ffexrgbdCaP1FsWthRDmjRXJ0FPadlzmMo3lOroRqfLEAztEzctEoqqzax'], 'uuids': ['769bdba5-e7ee-448c-8c3c-422e6f4915f7']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.151695", "end": "2020-12-05 17:27:07.275104", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-mafM79ffexrgbdCaP1FsWthRDmjRXJ0FPadlzmMo3lOroRqfLEAztEzctEoqqzax"], "labels": [], "masters": [], "uuids": ["769bdba5-e7ee-448c-8c3c-422e6f4915f7"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:07.123409", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.148652 s, 776 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.148652 s, 776 MB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:06.995 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:07.031 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:07 +0000 (0:00:00.075) 0:01:28.913 ***** 2020-12-05T17:27:07.066 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:07 +0000 (0:00:00.034) 0:01:28.947 ***** 2020-12-05T17:27:07.074 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-12-05T17:27:07.075 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:07.111 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:07 +0000 (0:00:00.044) 0:01:28.992 ***** 2020-12-05T17:27:07.138 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-12-05T17:27:07.140 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:07.176 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:07 +0000 (0:00:00.065) 0:01:29.057 ***** 2020-12-05T17:27:07.205 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-12-05T17:27:07.207 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-gkfB7RSHLTfof900eU10N7HgmzPrcbkalCEbrBis34GNFvaSy9XN2L2fZ0U83Izx'], 'uuids': ['5c2df557-8327-4a6b-a29d-e3831ae4d1e1']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.135878", "end": "2020-12-05 17:27:07.330349", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-gkfB7RSHLTfof900eU10N7HgmzPrcbkalCEbrBis34GNFvaSy9XN2L2fZ0U83Izx"], "labels": [], "masters": [], "uuids": ["5c2df557-8327-4a6b-a29d-e3831ae4d1e1"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:06.194471", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:27:07.208 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:07.336 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-12-05T17:27:07.338 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-12-05T17:27:07.374 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:07 +0000 (0:00:00.198) 0:01:29.256 ***** 2020-12-05T17:27:07.381 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-12-05T17:27:07.383 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.166499", "end": "2020-12-05 17:27:07.672247", "rc": 0, "start": "2020-12-05 17:27:07.505748", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:07.419 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:07 +0000 (0:00:00.044) 0:01:29.300 ***** 2020-12-05T17:27:07.485 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-12-05T17:27:07.487 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-12-05T17:27:07.526 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:07 +0000 (0:00:00.106) 0:01:29.407 ***** 2020-12-05T17:27:07.536 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-12-05T17:27:07.537 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-oEDae8BcSUQDZ9eYC9mvZ0vFo80IWggbVe8OXvfphSS98wDLD3TWVlnDqX3AOLE5'], 'uuids': ['b4c5d8f1-619e-41b3-bc23-66ae4d162600']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.126021", "end": "2020-12-05 17:27:07.532375", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-oEDae8BcSUQDZ9eYC9mvZ0vFo80IWggbVe8OXvfphSS98wDLD3TWVlnDqX3AOLE5"], "labels": [], "masters": [], "uuids": ["b4c5d8f1-619e-41b3-bc23-66ae4d162600"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:06.406354", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:27:07.540 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:07.636 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-12-05T17:27:07.638 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.068327", "end": "2020-12-05 17:27:07.938828", "rc": 0, "start": "2020-12-05 17:27:07.870501", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:07.676 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:07 +0000 (0:00:00.150) 0:01:29.558 ***** 2020-12-05T17:27:07.704 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-12-05T17:27:07.741 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:07 +0000 (0:00:00.065) 0:01:29.623 ***** 2020-12-05T17:27:07.768 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-12-05T17:27:07.770 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:07.807 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:07 +0000 (0:00:00.065) 0:01:29.688 ***** 2020-12-05T17:27:07.835 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-12-05T17:27:07.837 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:07.873 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:07 +0000 (0:00:00.065) 0:01:29.754 ***** 2020-12-05T17:27:07.890 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-12-05T17:27:07.891 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:07.901 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-12-05T17:27:07.903 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi002.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:07.940 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:07 +0000 (0:00:00.066) 0:01:29.821 ***** 2020-12-05T17:27:07.967 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-12-05T17:27:07.969 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:08.002 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:08 +0000 (0:00:00.062) 0:01:29.884 ***** 2020-12-05T17:27:08.103 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-12-05T17:27:08.104 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:08.125 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-12-05T17:27:08.127 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-12-05T17:27:08.163 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:08 +0000 (0:00:00.160) 0:01:30.044 ***** 2020-12-05T17:27:08.169 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-12-05T17:27:08.171 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:08.203 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-12-05T17:27:08.204 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'block_used': 13755773, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226339335, 'size_available': 927085916160, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149528, 'block_size': 4096, 'inode_available': 60905448}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226339335, "block_size": 4096, "block_total": 240095108, "block_used": 13755773, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905448, "inode_total": 61054976, "inode_used": 149528, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927085916160, "size_total": 983429562368, "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:08.241 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:08 +0000 (0:00:00.078) 0:01:30.123 ***** 2020-12-05T17:27:08.270 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-12-05T17:27:08.272 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:08.307 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:08 +0000 (0:00:00.065) 0:01:30.189 ***** 2020-12-05T17:27:08.315 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-12-05T17:27:08.317 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:08.337 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:27:08.339 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:08.387 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:08 +0000 (0:00:00.079) 0:01:30.268 ***** 2020-12-05T17:27:08.398 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi002.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:08.421 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:27:08.423 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:08.465 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:08 +0000 (0:00:00.078) 0:01:30.347 ***** 2020-12-05T17:27:08.483 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-12-05T17:27:08.484 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-gkfB7RSHLTfof900eU10N7HgmzPrcbkaEzzYQciHGsI19OvlmjKnvYWvH15sTfiP'], 'uuids': ['3a9ffa4b-7273-4407-8780-3143c1fdc484']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.153224", "end": "2020-12-05 17:27:08.604926", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-gkfB7RSHLTfof900eU10N7HgmzPrcbkaEzzYQciHGsI19OvlmjKnvYWvH15sTfiP"], "labels": [], "masters": [], "uuids": ["3a9ffa4b-7273-4407-8780-3143c1fdc484"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:07.451702", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-12-05T17:27:08.536 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:08 +0000 (0:00:00.071) 0:01:30.418 ***** 2020-12-05T17:27:08.547 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-12-05T17:27:08.549 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi032.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:08.589 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:08 +0000 (0:00:00.052) 0:01:30.470 ***** 2020-12-05T17:27:08.597 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:08.599 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:08.620 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:08.669 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:08 +0000 (0:00:00.080) 0:01:30.551 ***** 2020-12-05T17:27:08.675 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-12-05T17:27:08.677 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:08.697 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:08.733 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:08 +0000 (0:00:00.063) 0:01:30.614 ***** 2020-12-05T17:27:08.763 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-12-05T17:27:08.765 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:08.767 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi053.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-oEDae8BcSUQDZ9eYC9mvZ0vFo80IWggbJeKUoszRrHHxHfzMO2AeIc8cVpMf0LoJ'], 'uuids': ['3b56198c-769d-4b7d-ac95-823ead07050c']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.111491", "end": "2020-12-05 17:27:08.764307", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-oEDae8BcSUQDZ9eYC9mvZ0vFo80IWggbJeKUoszRrHHxHfzMO2AeIc8cVpMf0LoJ"], "labels": [], "masters": [], "uuids": ["3b56198c-769d-4b7d-ac95-823ead07050c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:07.652816", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk u 2020-12-05T17:27:08.768 INFO:teuthology.task.ansible.out:sing fdisk or", "other utilities."]} 2020-12-05T17:27:08.816 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:08 +0000 (0:00:00.083) 0:01:30.698 ***** 2020-12-05T17:27:08.850 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:08 +0000 (0:00:00.033) 0:01:30.731 ***** 2020-12-05T17:27:08.868 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:08.870 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-gkfB7RSHLTfof900eU10N7HgmzPrcbkaa8tlRZx3z5xZYNan2YNrws0JNG5T7fyn'], 'uuids': ['7029c252-e032-4cfb-82d2-74d8d9d72254']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.147119", "end": "2020-12-05 17:27:08.991729", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-gkfB7RSHLTfof900eU10N7HgmzPrcbkaa8tlRZx3z5xZYNan2YNrws0JNG5T7fyn"], "labels": [], "masters": [], "uuids": ["7029c252-e032-4cfb-82d2-74d8d9d72254"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:08.844610", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.144381 s, 799 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.144381 s, 799 MB/s"], "stdout": "", "stdout_lines": []} TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi002.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:08.909 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:08 +0000 (0:00:00.059) 0:01:30.790 ***** 2020-12-05T17:27:08.917 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:08.919 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:08.919 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:08.921 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:08.953 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-12-05T17:27:08.955 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-12-05T17:27:08.993 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:08 +0000 (0:00:00.084) 0:01:30.875 ***** 2020-12-05T17:27:09.044 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-12-05T17:27:09.046 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** changed: [smithi036.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-12-05T17:27:09.084 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:09 +0000 (0:00:00.090) 0:01:30.965 ***** 2020-12-05T17:27:09.117 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:09 +0000 (0:00:00.033) 0:01:30.999 ***** 2020-12-05T17:27:09.135 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:09.137 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-oEDae8BcSUQDZ9eYC9mvZ0vFo80IWggbwfOHqNyQTiDCNoe3C5mRImTqHwVSS1Mu'], 'uuids': ['b78ca16a-2ec3-4a02-881f-ef4737303b81']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.114199", "end": "2020-12-05 17:27:09.131538", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-oEDae8BcSUQDZ9eYC9mvZ0vFo80IWggbwfOHqNyQTiDCNoe3C5mRImTqHwVSS1Mu"], "labels": [], "masters": [], "uuids": ["b78ca16a-2ec3-4a02-881f-ef4737303b81"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:09.017339", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.109382 s, 1.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.109382 s, 1.1 GB/s"], "stdout": "", "stdout_lines": []} skipping: [smithi053.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': 2020-12-05T17:27:09.139 INFO:teuthology.task.ansible.out: '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:09.149 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:09.151 INFO:teuthology.task.ansible.out: 2020-12-05T17:27:09.161 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-12-05T17:27:09.163 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-12-05T17:27:09.173 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:09.175 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-gkfB7RSHLTfof900eU10N7HgmzPrcbkaD2cbFMQBpzmdVcVfcRVrq5brPh33985k'], 'uuids': ['4e9bc91e-6ca3-47d2-a295-fd605bb913e0']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.148255", "end": "2020-12-05 17:27:09.293635", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-gkfB7RSHLTfof900eU10N7HgmzPrcbkaD2cbFMQBpzmdVcVfcRVrq5brPh33985k"], "labels": [], "masters": [], "uuids": ["4e9bc91e-6ca3-47d2-a295-fd605bb913e0"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:09.145380", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.144264 s, 800 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.144264 s, 800 MB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:09.186 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4W2W8', 'wwn-0x5000c50087b8cd81'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4W2W8-part1', 'wwn-0x5000c50087b8cd81-part1'], 'uuids': ['d8c38c2f-ff33-4bf4-853e-25622c47cb18']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50087b8cd81', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W4W2W8', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4W2W8", "wwn-0x5000c50087b8cd81"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4W2W8-part1", "wwn-0x5000c50087b8cd81-part1"], "labels": [], "masters": [], "uuids": ["d8c38c2f-ff33-4bf4-853e-25622c47cb18"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4W2W8", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50087b8cd81"}}, "skip_reason": 2020-12-05T17:27:09.188 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-12-05T17:27:09.229 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-12-05T17:27:09.231 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-12-05T17:27:09.264 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-12-05T17:27:09.340 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-12-05T17:27:09.353 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:09.354 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-EnaFJs-wJ1a-Viuj-eDek-8iup-i26S-Ycck0V', 'nvme-INTEL_SSDPEDMD400G4_CVFT534300NV400BGN', 'nvme-nvme.8086-435646543533343330304e5634303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.047609", "end": "2020-12-05 17:27:09.481704", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-EnaFJs-wJ1a-Viuj-eDek-8iup-i26S-Ycck0V", "nvme-INTEL_SSDPEDMD400G4_CVFT534300NV400BGN", "nvme-nvme.8086-435646543533343330304e5634303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:09.434095", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0431179 s, 2.7 GB/s", "std 2020-12-05T17:27:09.356 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0431179 s, 2.7 GB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:09.356 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:09.368 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:09.379 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-oEDae8BcSUQDZ9eYC9mvZ0vFo80IWggbr9RfW4m9OPGgfl1eE7HsAsQLtPZnudpn'], 'uuids': ['90bcf67e-7a6e-4318-b687-85298f16ea80']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.106355", "end": "2020-12-05 17:27:09.376898", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-oEDae8BcSUQDZ9eYC9mvZ0vFo80IWggbr9RfW4m9OPGgfl1eE7HsAsQLtPZnudpn"], "labels": [], "masters": [], "uuids": ["90bcf67e-7a6e-4318-b687-85298f16ea80"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:09.270543", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.102098 s, 1.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.102098 s, 1.1 GB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:09.382 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi032.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-12-05T17:27:09.392 INFO:teuthology.task.ansible.out: 2020-12-05T17:27:09.394 INFO:teuthology.task.ansible.out:TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [smithi053.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4J2LS', 'wwn-0x5000c500871b78e7'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4J2LS-part1', 'wwn-0x5000c500871b78e7-part1'], 'uuids': ['d8c38c2f-ff33-4bf4-853e-25622c47cb18']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500871b78e7', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W4J2LS', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2LS", "wwn-0x5000c500871b78e7"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2LS-part1", "wwn-0x5000c500871b78e7-part1"], "labels": [], "masters": [], "uuids": ["d8c38c2f-ff33-4bf4-853e-25622c47cb18"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4J2LS", "size": "931.51 GB", "support_discard": 2020-12-05T17:27:09.395 INFO:teuthology.task.ansible.out:"0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871b78e7"}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:09.405 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-12-05T17:27:09.406 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:09.450 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-12-05T17:27:09.452 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-12-05T17:27:09.483 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-12-05T17:27:09.495 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:09.535 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:09 +0000 (0:00:00.417) 0:01:31.416 ***** 2020-12-05T17:27:09.554 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:09.555 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '937703088', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-69nwQE-o6IF-MAFw-QnL7-JdAn-Mcug-kNzECc', 'nvme-INTEL_SSDPED1D480GA_PHMB751300G8480DGN', 'nvme-nvme.8086-50484d42373531333030473834383044474e-494e54454c2053534450454431443438304741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPED1D480GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '447.13 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.048644", "end": "2020-12-05 17:27:09.559890", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series", "links": {"ids": ["lvm-pv-uuid-69nwQE-o6IF-MAFw-QnL7-JdAn-Mcug-kNzECc", "nvme-INTEL_SSDPED1D480GA_PHMB751300G8480DGN", "nvme-nvme.8086-50484d42373531333030473834383044474e-494e54454c2053534450454431443438304741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPED1D480GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "937703088", "sectorsize": "512", "size": "447.13 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:09.511246", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0439171 s, 2.6 GB/s", "stderr_lines": [" 2020-12-05T17:27:09.557 INFO:teuthology.task.ansible.out:110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0439171 s, 2.6 GB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:09.567 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2020-12-05T17:27:09.569 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi002.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-12-05T17:27:09.570 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [smithi053.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:09.610 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:09 +0000 (0:00:00.075) 0:01:31.492 ***** 2020-12-05T17:27:09.620 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:09.622 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** skipping: [smithi002.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:09.625 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:09.627 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-gkfB7RSHLTfof900eU10N7HgmzPrcbkafDRUG1bsqsPljpUt2IrEbI3TqnsRTCwA'], 'uuids': ['0c026eb7-f0af-4f63-b7b4-f87d0bbc209b']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.138966", "end": "2020-12-05 17:27:09.743313", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-gkfB7RSHLTfof900eU10N7HgmzPrcbkafDRUG1bsqsPljpUt2IrEbI3TqnsRTCwA"], "labels": [], "masters": [], "uuids": ["0c026eb7-f0af-4f63-b7b4-f87d0bbc209b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:09.604347", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.136571 s, 845 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.136571 s, 845 MB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:09.667 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:09 +0000 (0:00:00.056) 0:01:31.549 ***** 2020-12-05T17:27:09.675 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:09.676 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-12-05T17:27:09.676 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:09.676 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2020-12-05T17:27:09.677 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:09.677 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:09.678 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:09.714 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:09 +0000 (0:00:00.046) 0:01:31.595 ***** 2020-12-05T17:27:09.720 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2020-12-05T17:27:09.722 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:09.756 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:09 +0000 (0:00:00.042) 0:01:31.638 ***** 2020-12-05T17:27:09.783 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2020-12-05T17:27:09.785 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:09.820 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:09 +0000 (0:00:00.063) 0:01:31.702 ***** 2020-12-05T17:27:09.827 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:09.827 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-oEDae8BcSUQDZ9eYC9mvZ0vFo80IWggbrteC0NziBQlvfrT0ff8rL1UOhUsjlAAl'], 'uuids': ['f28c8f7e-d904-476e-8178-4c8d79f6217f']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.105359", "end": "2020-12-05 17:27:09.803283", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-oEDae8BcSUQDZ9eYC9mvZ0vFo80IWggbrteC0NziBQlvfrT0ff8rL1UOhUsjlAAl"], "labels": [], "masters": [], "uuids": ["f28c8f7e-d904-476e-8178-4c8d79f6217f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:09.697924", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.102414 s, 1.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.102414 s, 1.1 GB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:09.829 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:09.831 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:09.852 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-12-05T17:27:09.854 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-12-05T17:27:09.894 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:09 +0000 (0:00:00.073) 0:01:31.775 ***** 2020-12-05T17:27:09.900 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-12-05T17:27:09.902 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:09.912 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:09.914 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-gkfB7RSHLTfof900eU10N7HgmzPrcbkalCEbrBis34GNFvaSy9XN2L2fZ0U83Izx'], 'uuids': ['5c2df557-8327-4a6b-a29d-e3831ae4d1e1']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.149573", "end": "2020-12-05 17:27:10.037863", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-gkfB7RSHLTfof900eU10N7HgmzPrcbkalCEbrBis34GNFvaSy9XN2L2fZ0U83Izx"], "labels": [], "masters": [], "uuids": ["5c2df557-8327-4a6b-a29d-e3831ae4d1e1"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:09.888290", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.14567 s, 792 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.14567 s, 792 MB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:09.925 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:09.926 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** ok: [smithi032.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-12-05T17:27:09.965 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:09 +0000 (0:00:00.071) 0:01:31.847 ***** 2020-12-05T17:27:09.972 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-12-05T17:27:09.975 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-12-05T17:27:10.011 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:10 +0000 (0:00:00.046) 0:01:31.893 ***** 2020-12-05T17:27:10.041 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:10.043 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-oEDae8BcSUQDZ9eYC9mvZ0vFo80IWggbVe8OXvfphSS98wDLD3TWVlnDqX3AOLE5'], 'uuids': ['b4c5d8f1-619e-41b3-bc23-66ae4d162600']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.101330", "end": "2020-12-05 17:27:10.044417", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-oEDae8BcSUQDZ9eYC9mvZ0vFo80IWggbVe8OXvfphSS98wDLD3TWVlnDqX3AOLE5"], "labels": [], "masters": [], "uuids": ["b4c5d8f1-619e-41b3-bc23-66ae4d162600"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:09.943087", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0972229 s, 1.2 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0972229 s, 1.2 GB/s"], "stdout": "", "stdout_lines": []} TASK [testnode : Set xfs_opts on newer OSes] *********************************** ok: [smithi002.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-12-05T17:27:10.045 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [smithi053.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:10.083 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:10 +0000 (0:00:00.071) 0:01:31.965 ***** 2020-12-05T17:27:10.101 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-12-05T17:27:10.103 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:10.187 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-12-05T17:27:10.189 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-gkfB7RSHLTfof900eU10N7HgmzPrcbkaEzzYQciHGsI19OvlmjKnvYWvH15sTfiP'], 'uuids': ['3a9ffa4b-7273-4407-8780-3143c1fdc484']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.153647", "end": "2020-12-05 17:27:10.321039", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-gkfB7RSHLTfof900eU10N7HgmzPrcbkaEzzYQciHGsI19OvlmjKnvYWvH15sTfiP"], "labels": [], "masters": [], "uuids": ["3a9ffa4b-7273-4407-8780-3143c1fdc484"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:10.167392", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.149172 s, 773 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.149172 s, 773 MB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:10.251 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:10 +0000 (0:00:00.167) 0:01:32.132 ***** 2020-12-05T17:27:10.281 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-oEDae8BcSUQDZ9eYC9mvZ0vFo80IWggbJeKUoszRrHHxHfzMO2AeIc8cVpMf0LoJ'], 'uuids': ['3b56198c-769d-4b7d-ac95-823ead07050c']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.107671", "end": "2020-12-05 17:27:10.280036", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-oEDae8BcSUQDZ9eYC9mvZ0vFo80IWggbJeKUoszRrHHxHfzMO2AeIc8cVpMf0LoJ"], "labels": [], "masters": [], "uuids": ["3b56198c-769d-4b7d-ac95-823ead07050c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-12-05 17:27:10.172365", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.102915 s, 1.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.102915 s, 1.1 GB/s"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:10.330 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:10 +0000 (0:00:00.079) 0:01:32.212 ***** 2020-12-05T17:27:10.337 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-12-05T17:27:10.339 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:10.379 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:10 +0000 (0:00:00.048) 0:01:32.260 ***** 2020-12-05T17:27:10.397 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-12-05T17:27:10.400 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => {"changed": true} 2020-12-05T17:27:10.439 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:10 +0000 (0:00:00.059) 0:01:32.320 ***** 2020-12-05T17:27:10.509 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-12-05T17:27:10.511 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-12-05T17:27:10.549 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:10 +0000 (0:00:00.110) 0:01:32.431 ***** 2020-12-05T17:27:10.558 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** changed: [smithi061.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.141351", "end": "2020-12-05 17:27:10.646702", "rc": 0, "start": "2020-12-05 17:27:10.505351", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:10.600 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:10 +0000 (0:00:00.051) 0:01:32.482 ***** 2020-12-05T17:27:10.617 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.144853", "end": "2020-12-05 17:27:10.606887", "rc": 0, "start": "2020-12-05 17:27:10.462034", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:10.654 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:10 +0000 (0:00:00.053) 0:01:32.535 ***** 2020-12-05T17:27:10.703 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-12-05T17:27:10.705 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-12-05T17:27:10.781 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-12-05T17:27:10.783 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.045134", "end": "2020-12-05 17:27:10.914149", "rc": 0, "start": "2020-12-05 17:27:10.869015", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:10.818 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:10 +0000 (0:00:00.164) 0:01:32.700 ***** 2020-12-05T17:27:10.825 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.042550", "end": "2020-12-05 17:27:10.829558", "rc": 0, "start": "2020-12-05 17:27:10.787008", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:10.826 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-12-05T17:27:10.826 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-12-05T17:27:10.863 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:10 +0000 (0:00:00.044) 0:01:32.745 ***** 2020-12-05T17:27:10.869 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-12-05T17:27:10.870 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-12-05T17:27:10.870 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-12-05T17:27:10.906 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:10 +0000 (0:00:00.042) 0:01:32.788 ***** 2020-12-05T17:27:10.942 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:10 +0000 (0:00:00.035) 0:01:32.824 ***** 2020-12-05T17:27:10.948 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-12-05T17:27:10.950 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Write /scratch_devs] ****************************************** 2020-12-05T17:27:10.950 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-12-05T17:27:10.987 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:10 +0000 (0:00:00.044) 0:01:32.868 ***** 2020-12-05T17:27:11.026 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.039) 0:01:32.908 ***** 2020-12-05T17:27:11.032 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-12-05T17:27:11.034 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install cachefilesd conf file] ******************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:11.071 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.044) 0:01:32.952 ***** 2020-12-05T17:27:11.107 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.036) 0:01:32.989 ***** 2020-12-05T17:27:11.113 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-12-05T17:27:11.115 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi036.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-12-05T17:27:11.116 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:11.119 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:11.155 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.047) 0:01:33.037 ***** 2020-12-05T17:27:11.188 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.033) 0:01:33.070 ***** 2020-12-05T17:27:11.224 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.035) 0:01:33.105 ***** 2020-12-05T17:27:11.231 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-12-05T17:27:11.232 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Restart cachefilesd] ****************************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create filesystems] ******************************************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:11.268 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.044) 0:01:33.150 ***** 2020-12-05T17:27:11.302 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.033) 0:01:33.183 ***** 2020-12-05T17:27:11.335 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.033) 0:01:33.217 ***** 2020-12-05T17:27:11.341 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-12-05T17:27:11.342 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Mount filesystems] ******************************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:11.378 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.043) 0:01:33.260 ***** 2020-12-05T17:27:11.412 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.033) 0:01:33.294 ***** 2020-12-05T17:27:11.418 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-12-05T17:27:11.419 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [smithi032.front.sepia.ceph.com] => {"changed": true} TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-12-05T17:27:11.420 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-12-05T17:27:11.457 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.045) 0:01:33.339 ***** 2020-12-05T17:27:11.491 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.033) 0:01:33.372 ***** 2020-12-05T17:27:11.534 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.042) 0:01:33.415 ***** 2020-12-05T17:27:11.540 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-12-05T17:27:11.541 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create /var/lib/ceph] ***************************************** ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-12-05T17:27:11.579 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.045) 0:01:33.461 ***** 2020-12-05T17:27:11.616 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.037) 0:01:33.498 ***** 2020-12-05T17:27:11.623 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-12-05T17:27:11.625 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add CPAN config directory for the root user.] ***************** ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-12-05T17:27:11.625 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** ok: [smithi036.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-12-05T17:27:11.662 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.045) 0:01:33.544 ***** 2020-12-05T17:27:11.696 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.034) 0:01:33.578 ***** 2020-12-05T17:27:11.730 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.033) 0:01:33.611 ***** 2020-12-05T17:27:11.735 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-12-05T17:27:11.737 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-12-05T17:27:11.738 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-12-05T17:27:11.776 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.045) 0:01:33.657 ***** 2020-12-05T17:27:11.809 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.033) 0:01:33.691 ***** 2020-12-05T17:27:11.815 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** 2020-12-05T17:27:11.817 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:11.854 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.044) 0:01:33.736 ***** 2020-12-05T17:27:11.871 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-12-05T17:27:11.873 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-12-05T17:27:11.910 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.055) 0:01:33.792 ***** 2020-12-05T17:27:11.916 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2020-12-05T17:27:11.918 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:11.955 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.044) 0:01:33.837 ***** 2020-12-05T17:27:11.961 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-12-05T17:27:11.963 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-12-05T17:27:11.963 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-12-05T17:27:11.963 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'block_used': 13755770, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226339338, 'size_available': 927085928448, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149528, 'block_size': 4096, 'inode_available': 60905448}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226339338, "block_size": 4096, "block_total": 240095108, "block_used": 13755770, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905448, "inode_total": 61054976, "inode_used": 149528, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927085928448, "size_total": 983429562368, "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:12.000 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:11 +0000 (0:00:00.045) 0:01:33.882 ***** 2020-12-05T17:27:12.034 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:12 +0000 (0:00:00.032) 0:01:33.915 ***** 2020-12-05T17:27:12.039 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2020-12-05T17:27:12.041 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload CPAN config for the teuthology user.] ****************** ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1235, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2020-12-05T17:27:12.041 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-12-05T17:27:12.041 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'block_used': 13755779, 'uuid': 'd8c38c2f-ff33-4bf4-853e-25622c47cb18', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226339329, 'size_available': 927085891584, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149528, 'block_size': 4096, 'inode_available': 60905448}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226339329, "block_size": 4096, "block_total": 240095108, "block_used": 13755779, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905448, "inode_total": 61054976, "inode_used": 149528, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927085891584, "size_total": 983429562368, "uuid": "d8c38c2f-ff33-4bf4-853e-25622c47cb18"}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:12.078 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:12 +0000 (0:00:00.044) 0:01:33.960 ***** 2020-12-05T17:27:12.118 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:12 +0000 (0:00:00.039) 0:01:33.999 ***** 2020-12-05T17:27:12.153 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:12 +0000 (0:00:00.035) 0:01:34.035 ***** 2020-12-05T17:27:12.159 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-12-05T17:27:12.161 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Restart cachefilesd] ****************************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:12.197 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:12 +0000 (0:00:00.043) 0:01:34.078 ***** 2020-12-05T17:27:12.231 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:12 +0000 (0:00:00.034) 0:01:34.113 ***** 2020-12-05T17:27:12.237 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-12-05T17:27:12.239 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:12.275 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:12 +0000 (0:00:00.043) 0:01:34.157 ***** 2020-12-05T17:27:12.281 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:27:12.283 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:12.319 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:12 +0000 (0:00:00.043) 0:01:34.201 ***** 2020-12-05T17:27:12.326 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:12.363 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:12 +0000 (0:00:00.043) 0:01:34.244 ***** 2020-12-05T17:27:12.369 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-12-05T17:27:12.371 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} TASK [testnode : set_fact] ***************************************************** skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:12.409 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:12 +0000 (0:00:00.046) 0:01:34.291 ***** 2020-12-05T17:27:12.444 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:12 +0000 (0:00:00.034) 0:01:34.326 ***** 2020-12-05T17:27:12.452 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload CPAN config for root.] ********************************* ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-12-05T17:27:12.487 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:12 +0000 (0:00:00.043) 0:01:34.369 ***** 2020-12-05T17:27:12.532 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:12 +0000 (0:00:00.044) 0:01:34.413 ***** 2020-12-05T17:27:12.538 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-12-05T17:27:12.540 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:12.540 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-12-05T17:27:12.540 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-12-05T17:27:12.581 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:12 +0000 (0:00:00.049) 0:01:34.463 ***** 2020-12-05T17:27:12.615 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:12 +0000 (0:00:00.033) 0:01:34.497 ***** 2020-12-05T17:27:12.622 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:12.662 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:12 +0000 (0:00:00.046) 0:01:34.544 ***** 2020-12-05T17:27:12.670 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-12-05T17:27:12.672 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:12.713 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:12 +0000 (0:00:00.050) 0:01:34.595 ***** 2020-12-05T17:27:12.719 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:12.756 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:12 +0000 (0:00:00.042) 0:01:34.637 ***** 2020-12-05T17:27:12.763 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-12-05T17:27:12.765 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:12.801 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:12 +0000 (0:00:00.045) 0:01:34.683 ***** 2020-12-05T17:27:12.807 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:12.851 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:12 +0000 (0:00:00.050) 0:01:34.733 ***** 2020-12-05T17:27:12.913 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1235, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2020-12-05T17:27:12.960 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:12 +0000 (0:00:00.108) 0:01:34.842 ***** 2020-12-05T17:27:12.968 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-12-05T17:27:12.969 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => {"changed": true} 2020-12-05T17:27:13.005 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:13 +0000 (0:00:00.044) 0:01:34.887 ***** 2020-12-05T17:27:13.097 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-12-05T17:27:13.099 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-12-05T17:27:13.137 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:13 +0000 (0:00:00.131) 0:01:35.019 ***** 2020-12-05T17:27:13.143 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-12-05T17:27:13.198 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:13 +0000 (0:00:00.061) 0:01:35.080 ***** 2020-12-05T17:27:13.217 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-12-05T17:27:13.219 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-12-05T17:27:13.258 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:13 +0000 (0:00:00.059) 0:01:35.139 ***** 2020-12-05T17:27:13.277 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-12-05T17:27:13.279 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-12-05T17:27:13.279 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:13.319 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:13 +0000 (0:00:00.061) 0:01:35.201 ***** 2020-12-05T17:27:13.365 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:13 +0000 (0:00:00.045) 0:01:35.247 ***** 2020-12-05T17:27:13.393 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2020-12-05T17:27:13.395 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:13.432 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:13 +0000 (0:00:00.066) 0:01:35.314 ***** 2020-12-05T17:27:13.440 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-12-05T17:27:13.441 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"cache_update_time": 1607188915, "cache_updated": false, "changed": false} 2020-12-05T17:27:13.443 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi061.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:13.483 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:13 +0000 (0:00:00.050) 0:01:35.364 ***** 2020-12-05T17:27:13.491 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2020-12-05T17:27:13.493 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:13.537 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:13 +0000 (0:00:00.052) 0:01:35.417 ***** 2020-12-05T17:27:13.544 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-12-05T17:27:13.546 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:13.568 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2020-12-05T17:27:13.570 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:13.615 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:13 +0000 (0:00:00.078) 0:01:35.496 ***** 2020-12-05T17:27:13.655 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-12-05T17:27:13.664 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:13.697 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-12-05T17:27:13.700 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.047574", "end": "2020-12-05 17:27:13.672713", "rc": 0, "start": "2020-12-05 17:27:13.625139", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2020-12-05T17:27:13.733 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:13 +0000 (0:00:00.118) 0:01:35.615 ***** 2020-12-05T17:27:13.740 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-12-05T17:27:13.742 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:13.762 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:13.797 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:13 +0000 (0:00:00.064) 0:01:35.679 ***** 2020-12-05T17:27:13.804 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-12-05T17:27:13.806 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-12-05T17:27:13.839 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:13 +0000 (0:00:00.041) 0:01:35.721 ***** 2020-12-05T17:27:13.847 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-12-05T17:27:13.848 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:13.882 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:13 +0000 (0:00:00.043) 0:01:35.764 ***** 2020-12-05T17:27:13.902 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-12-05T17:27:13.903 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:13.956 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:13.969 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-12-05T17:27:13.970 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-12-05T17:27:14.009 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:14 +0000 (0:00:00.126) 0:01:35.891 ***** 2020-12-05T17:27:14.152 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-12-05T17:27:14.154 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} changed: [smithi053.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:14.249 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-12-05T17:27:14.251 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"cache_update_time": 1607188779, "cache_updated": false, "changed": false} 2020-12-05T17:27:14.290 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:14 +0000 (0:00:00.280) 0:01:36.171 ***** 2020-12-05T17:27:14.350 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-12-05T17:27:14.351 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1235, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2020-12-05T17:27:14.387 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:14 +0000 (0:00:00.097) 0:01:36.268 ***** 2020-12-05T17:27:14.393 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-12-05T17:27:14.395 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:14.396 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-12-05T17:27:14.436 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:14 +0000 (0:00:00.049) 0:01:36.318 ***** 2020-12-05T17:27:14.475 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:14 +0000 (0:00:00.038) 0:01:36.356 ***** 2020-12-05T17:27:14.482 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-12-05T17:27:14.483 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.045790", "end": "2020-12-05 17:27:14.643829", "rc": 0, "start": "2020-12-05 17:27:14.598039", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2020-12-05T17:27:14.520 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:14 +0000 (0:00:00.045) 0:01:36.402 ***** 2020-12-05T17:27:14.549 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-12-05T17:27:14.550 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} TASK [testnode : Install Amazon::S3.] ****************************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:14.587 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:14 +0000 (0:00:00.065) 0:01:36.468 ***** 2020-12-05T17:27:14.621 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:14 +0000 (0:00:00.034) 0:01:36.503 ***** 2020-12-05T17:27:14.628 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-12-05T17:27:14.629 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} TASK [testnode : Install ntp package on rpm based systems.] ******************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:14.666 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:14 +0000 (0:00:00.044) 0:01:36.548 ***** 2020-12-05T17:27:14.707 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:14 +0000 (0:00:00.040) 0:01:36.588 ***** 2020-12-05T17:27:14.713 INFO:teuthology.task.ansible.out: 2020-12-05T17:27:14.714 INFO:teuthology.task.ansible.out:TASK [testnode : Upload CPAN config for root.] ********************************* ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-12-05T17:27:14.752 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:14 +0000 (0:00:00.045) 0:01:36.634 ***** 2020-12-05T17:27:14.760 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-12-05T17:27:14.761 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"cache_update_time": 1607188915, "cache_updated": false, "changed": false} 2020-12-05T17:27:14.799 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:14 +0000 (0:00:00.046) 0:01:36.681 ***** 2020-12-05T17:27:14.806 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-12-05T17:27:14.808 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-12-05T17:27:14.829 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-12-05T17:27:14.831 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:14.866 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:14 +0000 (0:00:00.066) 0:01:36.747 ***** 2020-12-05T17:27:14.873 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-12-05T17:27:14.875 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-12-05T17:27:14.907 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-12-05T17:27:14.981 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-12-05T17:27:15.024 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-12-05T17:27:15.100 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-12-05T17:27:15.144 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-12-05T17:27:15.157 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:15.203 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:15 +0000 (0:00:00.337) 0:01:37.085 ***** 2020-12-05T17:27:15.211 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-12-05T17:27:15.213 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} TASK [testnode : Write /scratch_devs] ****************************************** 2020-12-05T17:27:15.213 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-12-05T17:27:15.253 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:15 +0000 (0:00:00.049) 0:01:37.135 ***** 2020-12-05T17:27:15.260 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:15.264 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2020-12-05T17:27:15.265 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:15.299 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:15 +0000 (0:00:00.045) 0:01:37.180 ***** 2020-12-05T17:27:15.333 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:15 +0000 (0:00:00.034) 0:01:37.215 ***** 2020-12-05T17:27:15.339 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-12-05T17:27:15.341 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Mount filesystems] ******************************************** skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:15.376 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:15 +0000 (0:00:00.043) 0:01:37.258 ***** 2020-12-05T17:27:15.410 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:15 +0000 (0:00:00.033) 0:01:37.291 ***** 2020-12-05T17:27:15.416 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2020-12-05T17:27:15.417 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:15.453 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:15 +0000 (0:00:00.043) 0:01:37.335 ***** 2020-12-05T17:27:15.480 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2020-12-05T17:27:15.482 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:15.514 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:15 +0000 (0:00:00.060) 0:01:37.396 ***** 2020-12-05T17:27:15.541 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-12-05T17:27:15.543 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-12-05T17:27:15.576 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:15 +0000 (0:00:00.062) 0:01:37.458 ***** 2020-12-05T17:27:15.590 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-12-05T17:27:15.592 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-12-05 17:25:25 UTC", "ActiveEnterTimestampMonotonic": "26154954", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target systemd-tmpfiles-setup.service sysinit.target network.target system.slice systemd-journald.socket -.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-12-05 17:25:19 UTC", "AssertTimestampMonotonic": "19552293", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-12-05 17:25:19 UTC", "ConditionTimestampMonotonic": "19552293", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2020-12-05T17:27:15.594 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1820", "ExecMainStartTimestamp": "Sat 2020-12-05 17:25:25 UTC", "ExecMainStartTimestampMonotonic": "26154941", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-12-05 17:25:19 UTC", "InactiveExitTimestampMonotonic": "19552955", "InvocationID": "41a87891521d4fb0b811c6aba0e08057", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128053", "LimitNPROCSoft": "128053", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2020-12-05T17:27:15.595 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128053", "LimitSIGPENDINGSoft": "128053", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1820", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-12-05 17:25:25 UTC", "S 2020-12-05T17:27:15.597 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "26154954", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-12-05 17:25:25 UTC", "WatchdogTimestampMonotonic": "26154953", "WatchdogUSec": "0"}} TASK [testnode : Install ntp package on apt based systems.] ******************** ok: [smithi032.front.sepia.ceph.com] => {"cache_update_time": 1607188779, "cache_updated": false, "changed": false} 2020-12-05T17:27:15.630 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:15 +0000 (0:00:00.053) 0:01:37.511 ***** 2020-12-05T17:27:15.663 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:15 +0000 (0:00:00.033) 0:01:37.545 ***** 2020-12-05T17:27:15.669 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-12-05T17:27:15.671 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} TASK [testnode : Create /var/lib/ceph] ***************************************** ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-12-05T17:27:15.671 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** ok: [smithi036.front.sepia.ceph.com] => {"cache_update_time": 1607189016, "cache_updated": false, "changed": false} 2020-12-05T17:27:15.673 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi002.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-12-05T17:27:15.707 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:15 +0000 (0:00:00.044) 0:01:37.589 ***** 2020-12-05T17:27:15.740 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:15 +0000 (0:00:00.032) 0:01:37.621 ***** 2020-12-05T17:27:15.771 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:15 +0000 (0:00:00.030) 0:01:37.652 ***** 2020-12-05T17:27:15.803 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:15 +0000 (0:00:00.032) 0:01:37.685 ***** 2020-12-05T17:27:15.809 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-12-05T17:27:15.811 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:15.846 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:15 +0000 (0:00:00.042) 0:01:37.728 ***** 2020-12-05T17:27:15.881 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:15 +0000 (0:00:00.034) 0:01:37.762 ***** 2020-12-05T17:27:15.887 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-12-05T17:27:15.888 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} TASK [testnode : Install python-pip on rpm based systems.] ********************* skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:15.925 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:15 +0000 (0:00:00.044) 0:01:37.807 ***** 2020-12-05T17:27:15.960 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:15 +0000 (0:00:00.033) 0:01:37.841 ***** 2020-12-05T17:27:15.966 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-12-05T17:27:15.968 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.047057", "end": "2020-12-05 17:27:16.211407", "rc": 0, "start": "2020-12-05 17:27:16.164350", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [smithi053.front.sepia.ceph.com] => {"changed": true} 2020-12-05T17:27:16.007 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:16 +0000 (0:00:00.046) 0:01:37.888 ***** 2020-12-05T17:27:16.043 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:16 +0000 (0:00:00.036) 0:01:37.925 ***** 2020-12-05T17:27:16.051 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-12-05T17:27:16.053 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:16.090 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:16 +0000 (0:00:00.045) 0:01:37.971 ***** 2020-12-05T17:27:16.118 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-12-05T17:27:16.120 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:16.155 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:16 +0000 (0:00:00.065) 0:01:38.037 ***** 2020-12-05T17:27:16.183 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-12-05T17:27:16.184 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [smithi061.front.sepia.ceph.com] => {"changed": true} 2020-12-05T17:27:16.222 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:16 +0000 (0:00:00.066) 0:01:38.103 ***** 2020-12-05T17:27:16.263 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:16 +0000 (0:00:00.041) 0:01:38.145 ***** 2020-12-05T17:27:16.270 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-12-05T17:27:16.271 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} TASK [testnode : Create the chrony.conf file] ********************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:16.309 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:16 +0000 (0:00:00.045) 0:01:38.191 ***** 2020-12-05T17:27:16.455 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:16 +0000 (0:00:00.145) 0:01:38.337 ***** 2020-12-05T17:27:16.483 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** 2020-12-05T17:27:16.484 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:16.519 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:16 +0000 (0:00:00.063) 0:01:38.401 ***** 2020-12-05T17:27:16.546 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2020-12-05T17:27:16.548 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:16.582 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:16 +0000 (0:00:00.062) 0:01:38.464 ***** 2020-12-05T17:27:16.609 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2020-12-05T17:27:16.611 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:16.645 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:16 +0000 (0:00:00.062) 0:01:38.526 ***** 2020-12-05T17:27:16.651 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-12-05T17:27:16.654 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-12-05 17:25:09 UTC", "ActiveEnterTimestampMonotonic": "27268816", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target systemd-tmpfiles-setup.service sysinit.target systemd-journald.socket -.mount network.target system.slice", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-12-05 17:25:03 UTC", "AssertTimestampMonotonic": "20680217", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-12-05 17:25:03 UTC", "ConditionTimestampMonotonic": "20680217", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2020-12-05T17:27:16.656 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1994", "ExecMainStartTimestamp": "Sat 2020-12-05 17:25:09 UTC", "ExecMainStartTimestampMonotonic": "27268802", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-12-05 17:25:03 UTC", "InactiveExitTimestampMonotonic": "20680982", "InvocationID": "020cd98c14d24c5ca95cbd6141d8b773", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128053", "LimitNPROCSoft": "128053", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2020-12-05T17:27:16.657 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128053", "LimitSIGPENDINGSoft": "128053", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1994", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount system.slice sysinit.target", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-12-05 17:25:09 UTC", "S 2020-12-05T17:27:16.659 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "27268816", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-12-05 17:25:09 UTC", "WatchdogTimestampMonotonic": "27268814", "WatchdogUSec": "0"}} 2020-12-05T17:27:16.691 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:16 +0000 (0:00:00.046) 0:01:38.573 ***** 2020-12-05T17:27:16.697 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2020-12-05T17:27:16.699 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:16.741 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:16 +0000 (0:00:00.050) 0:01:38.623 ***** 2020-12-05T17:27:16.749 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:27:16.751 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-12-05T17:27:16.787 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:16 +0000 (0:00:00.045) 0:01:38.669 ***** 2020-12-05T17:27:16.814 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:27:16.816 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:16.850 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:16 +0000 (0:00:00.062) 0:01:38.732 ***** 2020-12-05T17:27:16.856 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-12-05T17:27:16.858 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"cache_update_time": 1607188915, "cache_updated": false, "changed": false} 2020-12-05T17:27:16.894 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:16 +0000 (0:00:00.043) 0:01:38.775 ***** 2020-12-05T17:27:16.900 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-12-05T17:27:16.901 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add CPAN config directory for the teuthology user.] *********** ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-12-05T17:27:16.936 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:16 +0000 (0:00:00.042) 0:01:38.818 ***** 2020-12-05T17:27:16.970 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:16 +0000 (0:00:00.032) 0:01:38.851 ***** 2020-12-05T17:27:16.976 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-12-05T17:27:16.977 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:17.013 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:17 +0000 (0:00:00.043) 0:01:38.894 ***** 2020-12-05T17:27:17.031 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-12-05T17:27:17.033 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"cache_update_time": 1607189016, "cache_updated": false, "changed": false} 2020-12-05T17:27:17.070 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:17 +0000 (0:00:00.056) 0:01:38.951 ***** 2020-12-05T17:27:17.108 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-12-05T17:27:17.109 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add CPAN config directory for the root user.] ***************** ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-12-05T17:27:17.153 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:17 +0000 (0:00:00.084) 0:01:39.035 ***** 2020-12-05T17:27:17.187 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:17 +0000 (0:00:00.034) 0:01:39.069 ***** 2020-12-05T17:27:17.194 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-12-05T17:27:17.195 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-12-05T17:27:17.233 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:17 +0000 (0:00:00.045) 0:01:39.114 ***** 2020-12-05T17:27:17.452 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-12-05T17:27:17.454 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-12-05T17:27:17.494 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:17 +0000 (0:00:00.261) 0:01:39.376 ***** 2020-12-05T17:27:17.501 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-12-05T17:27:17.502 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-12-05T17:27:17.538 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:17 +0000 (0:00:00.043) 0:01:39.419 ***** 2020-12-05T17:27:17.543 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-12-05T17:27:17.544 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1235, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} TASK [testnode : Create the chrony.conf file] ********************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:17.545 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1235, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2020-12-05T17:27:17.581 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:17 +0000 (0:00:00.043) 0:01:39.463 ***** 2020-12-05T17:27:17.615 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:17 +0000 (0:00:00.033) 0:01:39.497 ***** 2020-12-05T17:27:17.650 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:17 +0000 (0:00:00.034) 0:01:39.531 ***** 2020-12-05T17:27:17.656 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** 2020-12-05T17:27:17.658 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:17.694 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:17 +0000 (0:00:00.044) 0:01:39.576 ***** 2020-12-05T17:27:17.722 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2020-12-05T17:27:17.724 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:17.758 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:17 +0000 (0:00:00.063) 0:01:39.640 ***** 2020-12-05T17:27:17.785 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:17.820 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:17 +0000 (0:00:00.061) 0:01:39.701 ***** 2020-12-05T17:27:17.837 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-12-05T17:27:17.838 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-12-05 17:25:18 UTC", "ActiveEnterTimestampMonotonic": "26252914", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-tmpfiles-setup.service network.target systemd-journald.socket basic.target sysinit.target -.mount system.slice", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-12-05 17:25:12 UTC", "AssertTimestampMonotonic": "19716072", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-12-05 17:25:12 UTC", "ConditionTimestampMonotonic": "19716071", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2020-12-05T17:27:17.840 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1827", "ExecMainStartTimestamp": "Sat 2020-12-05 17:25:18 UTC", "ExecMainStartTimestampMonotonic": "26252899", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-12-05 17:25:12 UTC", "InactiveExitTimestampMonotonic": "19716768", "InvocationID": "28b1b14380ff49dd89b265ad61415cbc", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128053", "LimitNPROCSoft": "128053", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2020-12-05T17:27:17.841 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128053", "LimitSIGPENDINGSoft": "128053", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1827", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-12-05 17:25:18 UTC", "S 2020-12-05T17:27:17.843 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "26252914", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-12-05 17:25:18 UTC", "WatchdogTimestampMonotonic": "26252912", "WatchdogUSec": "0"}} TASK [testnode : Install python-pip on apt based systems.] ********************* ok: [smithi032.front.sepia.ceph.com] => {"cache_update_time": 1607188779, "cache_updated": false, "changed": false} TASK [testnode : Restart cachefilesd] ****************************************** skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:17.876 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:17 +0000 (0:00:00.055) 0:01:39.757 ***** 2020-12-05T17:27:17.909 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:17 +0000 (0:00:00.032) 0:01:39.790 ***** 2020-12-05T17:27:17.942 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:17 +0000 (0:00:00.033) 0:01:39.823 ***** 2020-12-05T17:27:17.948 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-12-05T17:27:17.950 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** ok: [smithi036.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-12-05T17:27:17.987 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:17 +0000 (0:00:00.044) 0:01:39.868 ***** 2020-12-05T17:27:18.026 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:18 +0000 (0:00:00.039) 0:01:39.908 ***** 2020-12-05T17:27:18.032 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-12-05T17:27:18.034 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-12-05T17:27:18.071 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:18 +0000 (0:00:00.044) 0:01:39.952 ***** 2020-12-05T17:27:18.077 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:27:18.079 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:18.115 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:18 +0000 (0:00:00.043) 0:01:39.996 ***** 2020-12-05T17:27:18.122 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-12-05T17:27:18.123 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-12-05T17:27:18.161 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:18 +0000 (0:00:00.046) 0:01:40.043 ***** 2020-12-05T17:27:18.167 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-12-05T17:27:18.168 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} TASK [testnode : Install python-pip on rpm based systems.] ********************* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:18.205 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:18 +0000 (0:00:00.044) 0:01:40.087 ***** 2020-12-05T17:27:18.239 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:18 +0000 (0:00:00.033) 0:01:40.120 ***** 2020-12-05T17:27:18.298 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-12-05T17:27:18.299 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-12-05T17:27:18.336 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:18 +0000 (0:00:00.097) 0:01:40.217 ***** 2020-12-05T17:27:18.490 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-12-05T17:27:18.493 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-12-05T17:27:18.553 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:18 +0000 (0:00:00.216) 0:01:40.434 ***** 2020-12-05T17:27:18.559 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-12-05T17:27:18.561 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1235, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2020-12-05T17:27:18.597 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:18 +0000 (0:00:00.044) 0:01:40.479 ***** 2020-12-05T17:27:18.680 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-12-05T17:27:18.682 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1235, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2020-12-05T17:27:18.725 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:18 +0000 (0:00:00.127) 0:01:40.607 ***** 2020-12-05T17:27:19.024 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-12-05T17:27:19.026 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"cache_update_time": 1607188959, "cache_updated": false, "changed": false} 2020-12-05T17:27:19.060 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:19 +0000 (0:00:00.335) 0:01:40.942 ***** 2020-12-05T17:27:19.067 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-12-05T17:27:19.068 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false} TASK [testnode : Upload CPAN config for root.] ********************************* ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-12-05T17:27:19.105 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:19 +0000 (0:00:00.044) 0:01:40.987 ***** 2020-12-05T17:27:19.145 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:19 +0000 (0:00:00.039) 0:01:41.027 ***** 2020-12-05T17:27:19.161 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-12-05T17:27:19.162 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"ansible_facts": {"interface": "ens1f0"}, "changed": false} 2020-12-05T17:27:19.201 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:19 +0000 (0:00:00.056) 0:01:41.083 ***** 2020-12-05T17:27:19.241 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-12-05T17:27:19.244 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"cache_update_time": 1607189016, "cache_updated": false, "changed": false} 2020-12-05T17:27:19.278 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:19 +0000 (0:00:00.076) 0:01:41.160 ***** 2020-12-05T17:27:19.286 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.046806", "end": "2020-12-05 17:27:19.249206", "rc": 0, "start": "2020-12-05 17:27:19.202400", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2020-12-05T17:27:19.326 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:19 +0000 (0:00:00.047) 0:01:41.207 ***** 2020-12-05T17:27:19.334 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-12-05T17:27:19.335 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:19.373 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:19 +0000 (0:00:00.047) 0:01:41.254 ***** 2020-12-05T17:27:19.380 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-12-05T17:27:19.382 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1607188719.3081641, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1607188719.3041642, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 113087, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1607188719.3041642, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 67, "uid": 0, "version": "3531524185", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} TASK [testnode : Install Amazon::S3.] ****************************************** skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:19.419 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:19 +0000 (0:00:00.045) 0:01:41.300 ***** 2020-12-05T17:27:19.453 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:19 +0000 (0:00:00.035) 0:01:41.335 ***** 2020-12-05T17:27:19.461 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-12-05T17:27:19.463 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-12-05T17:27:19.501 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:19 +0000 (0:00:00.047) 0:01:41.383 ***** 2020-12-05T17:27:19.508 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-12-05T17:27:19.510 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create the .pip directory for the teuthology user.] *********** ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-12-05T17:27:19.551 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:19 +0000 (0:00:00.050) 0:01:41.433 ***** 2020-12-05T17:27:19.597 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:19 +0000 (0:00:00.045) 0:01:41.479 ***** 2020-12-05T17:27:19.669 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-12-05T17:27:19.671 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2020-12-05T17:27:19.708 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:19 +0000 (0:00:00.110) 0:01:41.590 ***** 2020-12-05T17:27:19.736 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-12-05T17:27:19.737 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-12-05T17:27:19.774 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:19 +0000 (0:00:00.066) 0:01:41.656 ***** 2020-12-05T17:27:19.813 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-12-05T17:27:19.814 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:19.849 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:19 +0000 (0:00:00.075) 0:01:41.731 ***** 2020-12-05T17:27:19.866 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-12-05T17:27:19.867 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1235, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2020-12-05T17:27:19.909 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:19 +0000 (0:00:00.059) 0:01:41.791 ***** 2020-12-05T17:27:19.979 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-12-05T17:27:19.981 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-12-05T17:27:20.017 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:20 +0000 (0:00:00.108) 0:01:41.899 ***** 2020-12-05T17:27:20.068 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-12-05T17:27:20.070 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false} 2020-12-05T17:27:20.119 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:20 +0000 (0:00:00.101) 0:01:42.001 ***** 2020-12-05T17:27:20.126 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** ok: [smithi061.front.sepia.ceph.com] => {"cache_update_time": 1607189009, "cache_updated": false, "changed": false} 2020-12-05T17:27:20.165 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:20 +0000 (0:00:00.046) 0:01:42.047 ***** 2020-12-05T17:27:20.172 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-12-05T17:27:20.173 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"ansible_facts": {"interface": "ens1f0"}, "changed": false} TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* changed: [smithi002.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-12-05T17:27:20.219 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:20 +0000 (0:00:00.053) 0:01:42.100 ***** 2020-12-05T17:27:20.255 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:20 +0000 (0:00:00.036) 0:01:42.137 ***** 2020-12-05T17:27:20.261 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-12-05T17:27:20.263 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:20.299 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:20 +0000 (0:00:00.043) 0:01:42.181 ***** 2020-12-05T17:27:20.337 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-12-05T17:27:20.338 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=ens1f1) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-12-05T17:27:20.349 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-12-05T17:27:20.351 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item=ens1f0) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False"} TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-12-05T17:27:20.351 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.047208", "end": "2020-12-05 17:27:20.475537", "rc": 0, "start": "2020-12-05 17:27:20.428329", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2020-12-05T17:27:20.386 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:20 +0000 (0:00:00.087) 0:01:42.268 ***** 2020-12-05T17:27:20.423 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:20 +0000 (0:00:00.036) 0:01:42.305 ***** 2020-12-05T17:27:20.430 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-12-05T17:27:20.431 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1607188726.1123097, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1607188726.1083097, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 113087, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1607188726.1083097, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 67, "uid": 0, "version": "3531524185", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} TASK [testnode : Define net_to_configure var] ********************************** skipping: [smithi002.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:20.433 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi002.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:20.469 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:20 +0000 (0:00:00.046) 0:01:42.351 ***** 2020-12-05T17:27:20.476 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:20.477 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:20.479 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-12-05T17:27:20.481 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install ntp package on apt based systems.] ******************** ok: [smithi053.front.sepia.ceph.com] => {"cache_update_time": 1607188959, "cache_updated": false, "changed": false} 2020-12-05T17:27:20.518 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:20 +0000 (0:00:00.048) 0:01:42.400 ***** 2020-12-05T17:27:20.553 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:20 +0000 (0:00:00.034) 0:01:42.435 ***** 2020-12-05T17:27:20.587 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:20 +0000 (0:00:00.034) 0:01:42.469 ***** 2020-12-05T17:27:20.594 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:27:20.595 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:20.635 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:20 +0000 (0:00:00.047) 0:01:42.517 ***** 2020-12-05T17:27:20.669 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:20 +0000 (0:00:00.033) 0:01:42.550 ***** 2020-12-05T17:27:20.675 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-12-05T17:27:20.676 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "msg": ""} TASK [testnode : Install ntp package on rpm based systems.] ******************** skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:20.677 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:20.719 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:20 +0000 (0:00:00.050) 0:01:42.601 ***** 2020-12-05T17:27:20.752 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:20 +0000 (0:00:00.033) 0:01:42.634 ***** 2020-12-05T17:27:20.785 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:20 +0000 (0:00:00.032) 0:01:42.667 ***** 2020-12-05T17:27:20.791 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-12-05T17:27:20.793 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:20.793 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** ok: [smithi032.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-12-05T17:27:20.833 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:20 +0000 (0:00:00.047) 0:01:42.714 ***** 2020-12-05T17:27:20.867 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:20 +0000 (0:00:00.034) 0:01:42.749 ***** 2020-12-05T17:27:20.900 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:20 +0000 (0:00:00.033) 0:01:42.782 ***** 2020-12-05T17:27:20.907 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-12-05T17:27:20.910 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : ifdown and ifup] ********************************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:20.945 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:20 +0000 (0:00:00.044) 0:01:42.827 ***** 2020-12-05T17:27:20.979 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:20 +0000 (0:00:00.034) 0:01:42.861 ***** 2020-12-05T17:27:20.986 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-12-05T17:27:20.988 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:21.024 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:21 +0000 (0:00:00.045) 0:01:42.906 ***** 2020-12-05T17:27:21.032 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-12-05T17:27:21.032 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-12-05T17:27:21.069 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:21 +0000 (0:00:00.044) 0:01:42.950 ***** 2020-12-05T17:27:21.075 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-12-05T17:27:21.077 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:21.114 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:21 +0000 (0:00:00.045) 0:01:42.996 ***** 2020-12-05T17:27:21.120 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-12-05T17:27:21.122 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** changed: [smithi032.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-12-05T17:27:21.157 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:21 +0000 (0:00:00.043) 0:01:43.039 ***** 2020-12-05T17:27:21.190 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:21 +0000 (0:00:00.032) 0:01:43.072 ***** 2020-12-05T17:27:21.196 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-12-05T17:27:21.198 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:21.235 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:21 +0000 (0:00:00.044) 0:01:43.117 ***** 2020-12-05T17:27:21.262 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-12-05T17:27:21.264 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:21.302 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:21 +0000 (0:00:00.067) 0:01:43.184 ***** 2020-12-05T17:27:21.309 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-12-05T17:27:21.310 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-12-05T17:27:21.346 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:21 +0000 (0:00:00.043) 0:01:43.228 ***** 2020-12-05T17:27:21.373 INFO:teuthology.task.ansible.out: 2020-12-05T17:27:21.378 INFO:teuthology.task.ansible.out:TASK [testnode : Make sure ethtool is installed] ******************************* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:21.413 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:21 +0000 (0:00:00.066) 0:01:43.295 ***** 2020-12-05T17:27:21.421 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-12-05T17:27:21.423 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-12-05 17:25:30 UTC", "ActiveEnterTimestampMonotonic": "26653999", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-tmpfiles-setup.service basic.target system.slice sysinit.target network.target -.mount systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-12-05 17:25:23 UTC", "AssertTimestampMonotonic": "20068127", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-12-05 17:25:23 UTC", "ConditionTimestampMonotonic": "20068126", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2020-12-05T17:27:21.425 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1851", "ExecMainStartTimestamp": "Sat 2020-12-05 17:25:30 UTC", "ExecMainStartTimestampMonotonic": "26653986", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-12-05 17:25:23 UTC", "InactiveExitTimestampMonotonic": "20068790", "InvocationID": "f5842c0d5ff74887909b460b81b152dd", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128053", "LimitNPROCSoft": "128053", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2020-12-05T17:27:21.426 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128053", "LimitSIGPENDINGSoft": "128053", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1851", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-12-05 17:25:30 UTC", "S 2020-12-05T17:27:21.428 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "26653999", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-12-05 17:25:30 UTC", "WatchdogTimestampMonotonic": "26653997", "WatchdogUSec": "0"}} 2020-12-05T17:27:21.469 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:21 +0000 (0:00:00.055) 0:01:43.350 ***** 2020-12-05T17:27:21.474 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-12-05T17:27:21.476 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=ens1f1) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False"} skipping: [smithi032.front.sepia.ceph.com] => (item=ens1f0) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False"} TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi002.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.011080", "end": "2020-12-05 17:27:21.422597", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-12-05 17:27:21.411517", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:21.477 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* skipping: [smithi032.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-12-05T17:27:21.477 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-12-05T17:27:21.478 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-12-05T17:27:21.514 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:21 +0000 (0:00:00.045) 0:01:43.395 ***** 2020-12-05T17:27:21.520 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:27:21.522 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-12-05T17:27:21.559 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:21 +0000 (0:00:00.045) 0:01:43.441 ***** 2020-12-05T17:27:21.566 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-12-05T17:27:21.567 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:21.569 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* ok: [smithi036.front.sepia.ceph.com] => {"changed": false} 2020-12-05T17:27:21.569 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-12-05T17:27:21.570 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:21.570 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:21.570 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:21.571 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:21.573 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-12-05T17:27:21.574 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.005030", "end": "2020-12-05 17:27:21.547354", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-12-05 17:27:21.542324", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:21.608 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:21 +0000 (0:00:00.048) 0:01:43.490 ***** 2020-12-05T17:27:21.641 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:21 +0000 (0:00:00.033) 0:01:43.523 ***** 2020-12-05T17:27:21.647 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:27:21.649 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:21.687 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:21 +0000 (0:00:00.046) 0:01:43.569 ***** 2020-12-05T17:27:21.724 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:21 +0000 (0:00:00.036) 0:01:43.606 ***** 2020-12-05T17:27:21.730 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-12-05T17:27:21.732 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"ansible_facts": {"interface": "ens1f0"}, "changed": false} TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi002.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005421", "end": "2020-12-05 17:27:21.661338", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-12-05 17:27:21.655917", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:21.732 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** ok: [smithi061.front.sepia.ceph.com] => {"cache_update_time": 1607189009, "cache_updated": false, "changed": false} 2020-12-05T17:27:21.734 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:21.769 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:21 +0000 (0:00:00.044) 0:01:43.651 ***** 2020-12-05T17:27:21.805 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:21 +0000 (0:00:00.035) 0:01:43.687 ***** 2020-12-05T17:27:21.839 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:21 +0000 (0:00:00.033) 0:01:43.721 ***** 2020-12-05T17:27:21.845 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-12-05T17:27:21.847 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi002.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.030130", "end": "2020-12-05 17:27:21.798447", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-12-05 17:27:21.768317", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:21.885 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:21 +0000 (0:00:00.046) 0:01:43.767 ***** 2020-12-05T17:27:21.925 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:21 +0000 (0:00:00.039) 0:01:43.807 ***** 2020-12-05T17:27:21.959 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:21 +0000 (0:00:00.033) 0:01:43.841 ***** 2020-12-05T17:27:21.965 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-12-05T17:27:21.966 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-12-05T17:27:21.967 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:21.968 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1607188718.976, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1607188718.9720001, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 113087, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1607188718.9720001, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 67, "uid": 0, "version": "3531524185", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2020-12-05T17:27:22.005 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:22 +0000 (0:00:00.045) 0:01:43.886 ***** 2020-12-05T17:27:22.041 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:22 +0000 (0:00:00.035) 0:01:43.922 ***** 2020-12-05T17:27:22.076 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:22 +0000 (0:00:00.035) 0:01:43.958 ***** 2020-12-05T17:27:22.111 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:22 +0000 (0:00:00.034) 0:01:43.993 ***** 2020-12-05T17:27:22.118 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-12-05T17:27:22.124 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:22.159 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:22 +0000 (0:00:00.047) 0:01:44.041 ***** 2020-12-05T17:27:22.165 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-12-05T17:27:22.167 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-12-05T17:27:22.207 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:22.268 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:22 +0000 (0:00:00.109) 0:01:44.150 ***** 2020-12-05T17:27:22.276 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-12-05T17:27:22.278 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2020-12-05T17:27:22.316 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:22 +0000 (0:00:00.046) 0:01:44.196 ***** 2020-12-05T17:27:22.323 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:22.368 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:22 +0000 (0:00:00.053) 0:01:44.250 ***** 2020-12-05T17:27:22.376 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** ok: [smithi036.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-12-05T17:27:22.419 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:22 +0000 (0:00:00.050) 0:01:44.301 ***** 2020-12-05T17:27:22.426 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-12-05T17:27:22.428 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:22.465 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:22 +0000 (0:00:00.044) 0:01:44.345 ***** 2020-12-05T17:27:22.471 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-12-05T17:27:22.473 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create the ntp.conf file.] ************************************ ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-12-05T17:27:22.514 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:22 +0000 (0:00:00.050) 0:01:44.395 ***** 2020-12-05T17:27:22.559 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:22 +0000 (0:00:00.045) 0:01:44.441 ***** 2020-12-05T17:27:22.588 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-12-05T17:27:22.590 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:22.632 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:22 +0000 (0:00:00.072) 0:01:44.513 ***** 2020-12-05T17:27:22.640 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-12-05T17:27:22.642 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.011007", "end": "2020-12-05 17:27:22.779936", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-12-05 17:27:22.768929", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:22.664 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-12-05T17:27:22.666 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-12-05T17:27:22.701 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:22 +0000 (0:00:00.069) 0:01:44.583 ***** 2020-12-05T17:27:22.764 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-12-05T17:27:22.766 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006000", "end": "2020-12-05 17:27:22.915605", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-12-05 17:27:22.909605", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:22.840 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-12-05T17:27:22.842 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-12-05T17:27:22.899 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:22 +0000 (0:00:00.197) 0:01:44.780 ***** 2020-12-05T17:27:22.906 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-12-05T17:27:22.907 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-12-05 17:25:25 UTC", "ActiveEnterTimestampMonotonic": "24762180", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target basic.target systemd-journald.socket systemd-tmpfiles-setup.service system.slice sysinit.target -.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-12-05 17:25:18 UTC", "AssertTimestampMonotonic": "18228067", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-12-05 17:25:18 UTC", "ConditionTimestampMonotonic": "18228067", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2020-12-05T17:27:22.909 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1906", "ExecMainStartTimestamp": "Sat 2020-12-05 17:25:25 UTC", "ExecMainStartTimestampMonotonic": "24762167", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-12-05 17:25:18 UTC", "InactiveExitTimestampMonotonic": "18228836", "InvocationID": "7e71ae80368943ada5c19a98ac7ae297", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2020-12-05T17:27:22.912 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1906", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount system.slice sysinit.target", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-12-05 17:25:25 UTC", "S 2020-12-05T17:27:22.914 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "24762180", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-12-05 17:25:25 UTC", "WatchdogTimestampMonotonic": "24762179", "WatchdogUSec": "0"}} TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi032.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005535", "end": "2020-12-05 17:27:23.043830", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-12-05 17:27:23.038295", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} TASK [testnode : Install python-pip on apt based systems.] ********************* ok: [smithi053.front.sepia.ceph.com] => {"cache_update_time": 1607188959, "cache_updated": false, "changed": false} 2020-12-05T17:27:22.951 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:22 +0000 (0:00:00.052) 0:01:44.833 ***** 2020-12-05T17:27:22.988 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:22 +0000 (0:00:00.036) 0:01:44.870 ***** 2020-12-05T17:27:22.995 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-12-05T17:27:22.997 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install python-pip on zypper based systems.] ****************** skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:23.036 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:23 +0000 (0:00:00.047) 0:01:44.917 ***** 2020-12-05T17:27:23.074 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:23 +0000 (0:00:00.038) 0:01:44.955 ***** 2020-12-05T17:27:23.081 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:27:23.083 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi032.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.024865", "end": "2020-12-05 17:27:23.206798", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-12-05 17:27:23.181933", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:23.085 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-12-05T17:27:23.088 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=ens1f1) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-12-05T17:27:23.125 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:23 +0000 (0:00:00.051) 0:01:45.007 ***** 2020-12-05T17:27:23.162 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:23 +0000 (0:00:00.036) 0:01:45.043 ***** 2020-12-05T17:27:23.169 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-12-05T17:27:23.171 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item=ens1f0) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-12-05T17:27:23.172 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-12-05T17:27:23.219 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:23 +0000 (0:00:00.057) 0:01:45.101 ***** 2020-12-05T17:27:23.255 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:23 +0000 (0:00:00.035) 0:01:45.136 ***** 2020-12-05T17:27:23.262 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:27:23.264 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:23.265 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-12-05T17:27:23.266 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-12-05T17:27:23.305 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:23 +0000 (0:00:00.050) 0:01:45.187 ***** 2020-12-05T17:27:23.353 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:23 +0000 (0:00:00.047) 0:01:45.234 ***** 2020-12-05T17:27:23.360 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-12-05T17:27:23.362 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi036.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:23.363 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result 2020-12-05T17:27:23.365 INFO:teuthology.task.ansible.out: was False", "skipped": true}, "skip_reason": "Conditional result was False"} TASK [testnode : Touch /ceph-qa-ready] ***************************************** changed: [smithi032.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-12-05T17:27:23.432 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:23 +0000 (0:00:00.078) 0:01:45.313 ***** 2020-12-05T17:27:23.439 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-12-05T17:27:23.441 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:23.501 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:23 +0000 (0:00:00.069) 0:01:45.383 ***** 2020-12-05T17:27:23.507 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:27:23.509 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:23.553 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:23 +0000 (0:00:00.051) 0:01:45.434 ***** 2020-12-05T17:27:23.581 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-12-05T17:27:23.582 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:23.629 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:23 +0000 (0:00:00.076) 0:01:45.511 ***** 2020-12-05T17:27:23.647 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-12-05T17:27:23.649 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1235, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2020-12-05T17:27:23.684 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:23 +0000 (0:00:00.054) 0:01:45.566 ***** 2020-12-05T17:27:23.690 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-12-05T17:27:23.691 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:23.731 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:23 +0000 (0:00:00.046) 0:01:45.612 ***** 2020-12-05T17:27:24.277 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Saturday 05 December 2020 17:27:23 +0000 (0:00:00.076) 0:01:45.689 ***** TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Saturday 05 December 2020 17:27:23 +0000 (0:00:00.079) 0:01:45.769 ***** TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Saturday 05 December 2020 17:27:23 +0000 (0:00:00.076) 0:01:45.845 ***** TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Saturday 05 December 2020 17:27:24 +0000 (0:00:00.113) 0:01:45.959 ***** TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Saturday 05 December 2020 17:27:24 +0000 (0:00:00.072) 0:01:46.031 ***** 2020-12-05T17:27:24.305 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-12-05T17:27:24.307 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.010986", "end": "2020-12-05 17:27:24.591285", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-12-05 17:27:24.580299", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:24.434 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.004396", "end": "2020-12-05 17:27:24.714581", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-12-05 17:27:24.710185", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:24.446 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-12-05T17:27:24.448 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-12-05T17:27:24.483 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:24 +0000 (0:00:00.333) 0:01:46.365 ***** 2020-12-05T17:27:24.502 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-12-05T17:27:24.504 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"cache_update_time": 1607189009, "cache_updated": false, "changed": false} 2020-12-05T17:27:24.538 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:24 +0000 (0:00:00.054) 0:01:46.420 ***** 2020-12-05T17:27:24.555 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-12-05T17:27:24.556 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005360", "end": "2020-12-05 17:27:24.834286", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-12-05 17:27:24.828926", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:24.566 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-12-05T17:27:24.568 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:24.605 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:24 +0000 (0:00:00.066) 0:01:46.486 ***** 2020-12-05T17:27:24.695 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-12-05T17:27:24.697 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.026420", "end": "2020-12-05 17:27:24.980621", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-12-05 17:27:24.954201", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:24.735 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:24 +0000 (0:00:00.130) 0:01:46.616 ***** 2020-12-05T17:27:24.741 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-12-05T17:27:24.743 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-12-05T17:27:24.778 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:24 +0000 (0:00:00.043) 0:01:46.660 ***** 2020-12-05T17:27:24.784 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-12-05T17:27:24.820 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:24 +0000 (0:00:00.042) 0:01:46.702 ***** 2020-12-05T17:27:24.943 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-12-05T17:27:24.944 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-12-05T17:27:25.027 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-12-05T17:27:25.029 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false} 2020-12-05T17:27:25.066 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:25 +0000 (0:00:00.245) 0:01:46.948 ***** 2020-12-05T17:27:25.093 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-12-05T17:27:25.094 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"ansible_facts": {"interface": "ens1f0"}, "changed": false} 2020-12-05T17:27:25.128 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:25 +0000 (0:00:00.062) 0:01:47.010 ***** 2020-12-05T17:27:25.135 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-12-05T17:27:25.136 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1235, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2020-12-05T17:27:25.180 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:25 +0000 (0:00:00.051) 0:01:47.061 ***** 2020-12-05T17:27:25.280 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-12-05T17:27:25.282 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1607188721.6343222, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1607188721.630322, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 113087, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1607188721.630322, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 67, "uid": 0, "version": "3531524185", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2020-12-05T17:27:25.318 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:25 +0000 (0:00:00.138) 0:01:47.200 ***** 2020-12-05T17:27:25.451 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2020-12-05T17:27:25.492 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:25 +0000 (0:00:00.174) 0:01:47.374 ***** 2020-12-05T17:27:25.519 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-12-05T17:27:25.521 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-12-05T17:27:25.554 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:25 +0000 (0:00:00.061) 0:01:47.436 ***** 2020-12-05T17:27:25.581 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-12-05T17:27:25.583 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:25.617 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:25 +0000 (0:00:00.062) 0:01:47.499 ***** 2020-12-05T17:27:25.738 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-12-05T17:27:25.740 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-12-05T17:27:25.774 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:25 +0000 (0:00:00.157) 0:01:47.656 ***** 2020-12-05T17:27:25.904 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-12-05T17:27:25.906 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-12-05T17:27:25.940 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:25 +0000 (0:00:00.165) 0:01:47.821 ***** 2020-12-05T17:27:25.967 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-12-05T17:27:25.968 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} TASK [testnode : Make sure ethtool is installed] ******************************* skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:26.002 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:26 +0000 (0:00:00.062) 0:01:47.884 ***** 2020-12-05T17:27:26.036 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:26 +0000 (0:00:00.033) 0:01:47.918 ***** 2020-12-05T17:27:26.042 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-12-05T17:27:26.044 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=ens1f1) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False"} 2020-12-05T17:27:26.054 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-12-05T17:27:26.056 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item=ens1f0) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False"} skipping: [smithi053.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-12-05T17:27:26.092 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:26 +0000 (0:00:00.055) 0:01:47.974 ***** 2020-12-05T17:27:26.130 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** skipping: [smithi053.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:26.141 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:26.143 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:26.152 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:26.154 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:26.188 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:26 +0000 (0:00:00.096) 0:01:48.070 ***** 2020-12-05T17:27:26.214 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-12-05T17:27:26.216 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:26.250 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:26 +0000 (0:00:00.061) 0:01:48.132 ***** 2020-12-05T17:27:26.276 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-12-05T17:27:26.278 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:26.311 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:26 +0000 (0:00:00.061) 0:01:48.193 ***** 2020-12-05T17:27:26.338 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-12-05T17:27:26.340 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:26.376 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:26 +0000 (0:00:00.064) 0:01:48.258 ***** 2020-12-05T17:27:26.403 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-12-05T17:27:26.405 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:26.439 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:26 +0000 (0:00:00.062) 0:01:48.321 ***** 2020-12-05T17:27:26.466 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-12-05T17:27:26.468 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:26.508 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:26 +0000 (0:00:00.068) 0:01:48.389 ***** 2020-12-05T17:27:26.534 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-12-05T17:27:26.536 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:26.575 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:26 +0000 (0:00:00.066) 0:01:48.456 ***** 2020-12-05T17:27:26.582 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-12-05T17:27:26.585 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false} 2020-12-05T17:27:26.625 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:26 +0000 (0:00:00.050) 0:01:48.506 ***** 2020-12-05T17:27:26.632 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-12-05T17:27:26.634 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:26.671 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:26 +0000 (0:00:00.046) 0:01:48.553 ***** 2020-12-05T17:27:26.678 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-12-05T17:27:26.680 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-12-05T17:27:26.716 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:26 +0000 (0:00:00.044) 0:01:48.597 ***** 2020-12-05T17:27:26.722 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-12-05T17:27:26.724 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:26.765 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:26 +0000 (0:00:00.048) 0:01:48.647 ***** 2020-12-05T17:27:26.867 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-12-05T17:27:26.869 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1607188761.989063, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1607188760.553781, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 113087, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1607188760.553781, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "3531524185", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2020-12-05T17:27:26.905 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:26 +0000 (0:00:00.140) 0:01:48.787 ***** 2020-12-05T17:27:26.932 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-12-05T17:27:26.934 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.009629", "end": "2020-12-05 17:27:26.912982", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-12-05 17:27:26.903353", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:27.027 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-12-05T17:27:27.029 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2020-12-05T17:27:27.063 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:27 +0000 (0:00:00.157) 0:01:48.945 ***** 2020-12-05T17:27:27.069 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-12-05T17:27:27.071 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006079", "end": "2020-12-05 17:27:27.042863", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-12-05 17:27:27.036784", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:27.092 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** ok: [smithi061.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-12-05T17:27:27.128 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:27 +0000 (0:00:00.064) 0:01:49.009 ***** 2020-12-05T17:27:27.155 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-12-05T17:27:27.157 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:27.190 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:27 +0000 (0:00:00.062) 0:01:49.072 ***** 2020-12-05T17:27:27.196 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-12-05T17:27:27.198 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005580", "end": "2020-12-05 17:27:27.156555", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-12-05 17:27:27.150975", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:27.313 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.033160", "end": "2020-12-05 17:27:27.300738", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-12-05 17:27:27.267578", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:27.351 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:27 +0000 (0:00:00.160) 0:01:49.232 ***** 2020-12-05T17:27:27.357 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-12-05T17:27:27.358 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-12-05T17:27:27.393 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:27 +0000 (0:00:00.042) 0:01:49.275 ***** 2020-12-05T17:27:27.399 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-12-05T17:27:27.447 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:27 +0000 (0:00:00.053) 0:01:49.328 ***** 2020-12-05T17:27:27.516 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-12-05T17:27:27.518 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-12-05T17:27:27.555 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:27 +0000 (0:00:00.108) 0:01:49.436 ***** 2020-12-05T17:27:27.572 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-12-05T17:27:27.574 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-12-05T17:27:27.608 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:27.663 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:27 +0000 (0:00:00.108) 0:01:49.545 ***** 2020-12-05T17:27:27.700 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-12-05T17:27:27.702 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-12-05T17:27:27.713 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-12-05T17:27:27.715 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-12-05T17:27:27.749 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:27 +0000 (0:00:00.085) 0:01:49.631 ***** 2020-12-05T17:27:27.786 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-12-05T17:27:27.788 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi061.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:27.798 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:27.799 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:27.811 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:27.848 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:27 +0000 (0:00:00.099) 0:01:49.730 ***** 2020-12-05T17:27:27.875 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:27.913 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:27 +0000 (0:00:00.064) 0:01:49.794 ***** 2020-12-05T17:27:27.939 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-12-05T17:27:27.941 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:27.976 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:27 +0000 (0:00:00.063) 0:01:49.858 ***** 2020-12-05T17:27:28.002 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-12-05T17:27:28.005 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:28.041 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:28 +0000 (0:00:00.065) 0:01:49.923 ***** 2020-12-05T17:27:28.068 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-12-05T17:27:28.070 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:28.105 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:28 +0000 (0:00:00.064) 0:01:49.987 ***** 2020-12-05T17:27:28.132 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-12-05T17:27:28.133 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:28.170 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:28 +0000 (0:00:00.064) 0:01:50.052 ***** 2020-12-05T17:27:28.196 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-12-05T17:27:28.198 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:28.238 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:28 +0000 (0:00:00.068) 0:01:50.120 ***** 2020-12-05T17:27:28.265 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-12-05T17:27:28.267 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:28.305 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:28 +0000 (0:00:00.066) 0:01:50.186 ***** 2020-12-05T17:27:28.331 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-12-05T17:27:28.333 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:28.369 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:28 +0000 (0:00:00.064) 0:01:50.250 ***** 2020-12-05T17:27:28.532 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-12-05T17:27:28.534 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.010740", "end": "2020-12-05 17:27:28.640676", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-12-05 17:27:28.629936", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:28.660 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.003970", "end": "2020-12-05 17:27:28.775810", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-12-05 17:27:28.771840", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:28.787 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005360", "end": "2020-12-05 17:27:28.906005", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-12-05 17:27:28.900645", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:28.936 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.028294", "end": "2020-12-05 17:27:29.051152", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-12-05 17:27:29.022858", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:28.979 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:28 +0000 (0:00:00.610) 0:01:50.861 ***** 2020-12-05T17:27:29.007 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-12-05T17:27:29.043 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:29 +0000 (0:00:00.063) 0:01:50.924 ***** 2020-12-05T17:27:29.176 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-12-05T17:27:29.178 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-12-05T17:27:29.299 INFO:teuthology.task.ansible.out: PLAY [testnodes,senta,vossi] *************************************************** 2020-12-05T17:27:29.339 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:29 +0000 (0:00:00.295) 0:01:51.220 ***** 2020-12-05T17:27:29.376 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:29 +0000 (0:00:00.037) 0:01:51.257 ***** 2020-12-05T17:27:29.413 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:29 +0000 (0:00:00.037) 0:01:51.295 ***** 2020-12-05T17:27:29.448 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:29 +0000 (0:00:00.034) 0:01:51.330 ***** 2020-12-05T17:27:29.483 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:29 +0000 (0:00:00.034) 0:01:51.365 ***** 2020-12-05T17:27:30.060 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2020-12-05T17:27:30.062 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] 2020-12-05T17:27:30.110 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] 2020-12-05T17:27:30.154 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:30 +0000 (0:00:00.671) 0:01:52.036 ***** 2020-12-05T17:27:30.278 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] 2020-12-05T17:27:30.287 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] 2020-12-05T17:27:30.294 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2020-12-05T17:27:30.295 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2020-12-05T17:27:30.298 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi061.front.sepia.ceph.com] 2020-12-05T17:27:30.362 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:30 +0000 (0:00:00.208) 0:01:52.244 ***** 2020-12-05T17:27:30.396 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:30 +0000 (0:00:00.033) 0:01:52.277 ***** 2020-12-05T17:27:30.454 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2020-12-05T17:27:30.455 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [container-host : set_fact] *********************************************** ok: [smithi036.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2020-12-05T17:27:30.489 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:30 +0000 (0:00:00.093) 0:01:52.371 ***** 2020-12-05T17:27:30.523 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:30 +0000 (0:00:00.034) 0:01:52.405 ***** 2020-12-05T17:27:30.559 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:30 +0000 (0:00:00.035) 0:01:52.440 ***** 2020-12-05T17:27:30.591 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:30 +0000 (0:00:00.032) 0:01:52.473 ***** 2020-12-05T17:27:30.624 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:30 +0000 (0:00:00.033) 0:01:52.506 ***** 2020-12-05T17:27:30.631 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2020-12-05T17:27:30.633 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** ok: [smithi032.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} TASK [container-host : set_fact] *********************************************** 2020-12-05T17:27:30.633 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [container-host : set_fact] *********************************************** 2020-12-05T17:27:30.634 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2020-12-05T17:27:30.670 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:30 +0000 (0:00:00.045) 0:01:52.552 ***** 2020-12-05T17:27:30.703 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:30 +0000 (0:00:00.032) 0:01:52.585 ***** 2020-12-05T17:27:30.737 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:30 +0000 (0:00:00.033) 0:01:52.619 ***** 2020-12-05T17:27:30.775 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:30 +0000 (0:00:00.037) 0:01:52.657 ***** 2020-12-05T17:27:30.781 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2020-12-05T17:27:30.783 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [container-host : Including distro specific variables] ******************** ok: [smithi036.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2020-12-05T17:27:30.819 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:30 +0000 (0:00:00.043) 0:01:52.701 ***** 2020-12-05T17:27:30.853 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:30 +0000 (0:00:00.034) 0:01:52.735 ***** 2020-12-05T17:27:30.887 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:30 +0000 (0:00:00.033) 0:01:52.769 ***** 2020-12-05T17:27:30.892 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2020-12-05T17:27:30.894 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [container-host : Including distro specific variables] ******************** ok: [smithi002.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2020-12-05T17:27:30.930 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:30 +0000 (0:00:00.043) 0:01:52.812 ***** 2020-12-05T17:27:30.964 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:30 +0000 (0:00:00.033) 0:01:52.846 ***** 2020-12-05T17:27:30.970 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2020-12-05T17:27:30.972 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:31.009 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:31 +0000 (0:00:00.044) 0:01:52.890 ***** 2020-12-05T17:27:31.014 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** 2020-12-05T17:27:31.016 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2020-12-05T17:27:31.053 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:31 +0000 (0:00:00.044) 0:01:52.935 ***** 2020-12-05T17:27:31.059 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2020-12-05T17:27:31.097 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:31 +0000 (0:00:00.043) 0:01:52.978 ***** 2020-12-05T17:27:31.647 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2020-12-05T17:27:31.649 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"cache_update_time": 1607188779, "cache_updated": false, "changed": false} 2020-12-05T17:27:31.685 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:31 +0000 (0:00:00.587) 0:01:53.566 ***** 2020-12-05T17:27:31.711 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2020-12-05T17:27:31.713 INFO:teuthology.task.ansible.out:skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:31.749 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:31 +0000 (0:00:00.064) 0:01:53.631 ***** 2020-12-05T17:27:31.786 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2020-12-05T17:27:31.788 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"ansible_facts": {"container_mirror_cert_path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2020-12-05T17:27:31.826 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:31 +0000 (0:00:00.076) 0:01:53.708 ***** 2020-12-05T17:27:31.832 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2020-12-05T17:27:31.834 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"cache_update_time": 1607189016, "cache_updated": false, "changed": false} 2020-12-05T17:27:31.872 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:31 +0000 (0:00:00.046) 0:01:53.754 ***** 2020-12-05T17:27:31.879 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"cache_update_time": 1607188915, "cache_updated": false, "changed": false} 2020-12-05T17:27:31.923 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:31 +0000 (0:00:00.050) 0:01:53.805 ***** 2020-12-05T17:27:31.929 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2020-12-05T17:27:31.931 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:31.967 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:31 +0000 (0:00:00.043) 0:01:53.849 ***** 2020-12-05T17:27:31.974 INFO:teuthology.task.ansible.out:skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:31.975 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000] *** changed: [smithi032.front.sepia.ceph.com] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2020-12-05T17:27:32.013 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:32 +0000 (0:00:00.045) 0:01:53.895 ***** 2020-12-05T17:27:32.050 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:32 +0000 (0:00:00.036) 0:01:53.932 ***** 2020-12-05T17:27:32.056 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2020-12-05T17:27:32.058 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"ansible_facts": {"container_mirror_cert_path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "container_service_conf": "/etc/docker/daemon.json"}, "changed": false} TASK [container-host : Install container packages] ***************************** ok: [smithi053.front.sepia.ceph.com] => {"cache_update_time": 1607188959, "cache_updated": false, "changed": false} TASK [container-host : set_fact] *********************************************** 2020-12-05T17:27:32.058 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"ansible_facts": {"container_mirror_cert_path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2020-12-05T17:27:32.098 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:32 +0000 (0:00:00.047) 0:01:53.979 ***** 2020-12-05T17:27:32.133 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:32 +0000 (0:00:00.036) 0:01:54.015 ***** 2020-12-05T17:27:32.169 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:32 +0000 (0:00:00.035) 0:01:54.051 ***** 2020-12-05T17:27:32.176 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2020-12-05T17:27:32.177 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"cache_update_time": 1607189009, "cache_updated": false, "changed": false} 2020-12-05T17:27:32.215 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:32 +0000 (0:00:00.045) 0:01:54.097 ***** 2020-12-05T17:27:32.222 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2020-12-05T17:27:32.225 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:32.260 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:32 +0000 (0:00:00.044) 0:01:54.141 ***** 2020-12-05T17:27:32.266 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000] *** 2020-12-05T17:27:32.267 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} TASK [container-host : set_fact] *********************************************** skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:32.310 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:32 +0000 (0:00:00.048) 0:01:54.190 ***** 2020-12-05T17:27:32.347 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:32 +0000 (0:00:00.039) 0:01:54.229 ***** 2020-12-05T17:27:32.355 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000] *** 2020-12-05T17:27:32.358 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} TASK [container-host : set_fact] *********************************************** ok: [smithi053.front.sepia.ceph.com] => {"ansible_facts": {"container_mirror_cert_path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2020-12-05T17:27:32.398 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:32 +0000 (0:00:00.050) 0:01:54.279 ***** 2020-12-05T17:27:32.438 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:32 +0000 (0:00:00.039) 0:01:54.319 ***** 2020-12-05T17:27:32.446 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2020-12-05T17:27:32.447 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => {"changed": true, "checksum": "d386a5d594d8bb6c408cf08541a4eb609577a03d", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "md5sum": "e2cf8b6a8ad65ba91ea9fc53d49f1bf4", "mode": "0644", "owner": "root", "size": 2236, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1607189252.0752716-137920447616793/source", "state": "file", "uid": 0} TASK [container-host : set_fact] *********************************************** ok: [smithi061.front.sepia.ceph.com] => {"ansible_facts": {"container_mirror_cert_path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2020-12-05T17:27:32.502 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:32 +0000 (0:00:00.064) 0:01:54.383 ***** 2020-12-05T17:27:32.548 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:32 +0000 (0:00:00.045) 0:01:54.429 ***** 2020-12-05T17:27:32.565 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000] *** 2020-12-05T17:27:32.567 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2020-12-05T17:27:32.604 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:32 +0000 (0:00:00.055) 0:01:54.485 ***** 2020-12-05T17:27:32.620 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2020-12-05T17:27:32.622 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => {"changed": true, "checksum": "d386a5d594d8bb6c408cf08541a4eb609577a03d", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "md5sum": "e2cf8b6a8ad65ba91ea9fc53d49f1bf4", "mode": "0644", "owner": "root", "size": 2236, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1607189252.3335376-225461620246338/source", "state": "file", "uid": 0} 2020-12-05T17:27:32.659 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:32 +0000 (0:00:00.055) 0:01:54.541 ***** 2020-12-05T17:27:32.687 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000] *** 2020-12-05T17:27:32.689 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2020-12-05T17:27:32.726 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:32 +0000 (0:00:00.066) 0:01:54.608 ***** 2020-12-05T17:27:32.743 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2020-12-05T17:27:32.745 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => {"changed": true, "checksum": "d386a5d594d8bb6c408cf08541a4eb609577a03d", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "md5sum": "e2cf8b6a8ad65ba91ea9fc53d49f1bf4", "mode": "0644", "owner": "root", "size": 2236, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1607189252.4229114-132149876980278/source", "state": "file", "uid": 0} 2020-12-05T17:27:32.782 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:32 +0000 (0:00:00.056) 0:01:54.664 ***** 2020-12-05T17:27:32.936 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => {"changed": true, "checksum": "d386a5d594d8bb6c408cf08541a4eb609577a03d", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "md5sum": "e2cf8b6a8ad65ba91ea9fc53d49f1bf4", "mode": "0644", "owner": "root", "size": 2236, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1607189252.635449-280059545021811/source", "state": "file", "uid": 0} 2020-12-05T17:27:32.974 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:32 +0000 (0:00:00.191) 0:01:54.856 ***** 2020-12-05T17:27:33.053 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => {"changed": true, "checksum": "d386a5d594d8bb6c408cf08541a4eb609577a03d", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "md5sum": "e2cf8b6a8ad65ba91ea9fc53d49f1bf4", "mode": "0644", "owner": "root", "size": 2236, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1607189252.7541528-154973358953151/source", "state": "file", "uid": 0} 2020-12-05T17:27:33.091 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:33 +0000 (0:00:00.116) 0:01:54.973 ***** 2020-12-05T17:27:36.053 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** changed: [smithi032.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "", "stderr_lines": [], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-ZS1YZK-build\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nBuilding wheels for collected packages: docopt\n Running setup.py bdist_wheel for docopt: started\n Running setup.py bdist_wheel for docopt: finished with status 'done'\n Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\nSuccessfully built docopt\nInstalling collected packages: docopt, toml, typing, registries-conf-ctl\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-ZS1YZK-build", "Collecting docopt (f 2020-12-05T17:27:36.054 INFO:teuthology.task.ansible.out:rom registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Building wheels for collected packages: docopt", " Running setup.py bdist_wheel for docopt: started", " Running setup.py bdist_wheel for docopt: finished with status 'done'", " Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e", "Successfully built docopt", "Installing collected packages: docopt, toml, typing, registries-conf-ctl", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} 2020-12-05T17:27:36.090 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:36 +0000 (0:00:02.998) 0:01:57.971 ***** 2020-12-05T17:27:36.221 INFO:teuthology.task.ansible.out: TASK [container-host : Find registries-conf-ctl] ******************************* 2020-12-05T17:27:36.223 INFO:teuthology.task.ansible.out:ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-12-05T17:27:36.258 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:36 +0000 (0:00:00.168) 0:01:58.140 ***** 2020-12-05T17:27:36.317 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2020-12-05T17:27:36.319 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "", "stderr_lines": [], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-x7mI2H-build\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nBuilding wheels for collected packages: docopt\n Running setup.py bdist_wheel for docopt: started\n Running setup.py bdist_wheel for docopt: finished with status 'done'\n Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\nSuccessfully built docopt\nInstalling collected packages: docopt, toml, typing, registries-conf-ctl\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-x7mI2H-build", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packag 2020-12-05T17:27:36.320 INFO:teuthology.task.ansible.out:es/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Building wheels for collected packages: docopt", " Running setup.py bdist_wheel for docopt: started", " Running setup.py bdist_wheel for docopt: finished with status 'done'", " Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e", "Successfully built docopt", "Installing collected packages: docopt, toml, typing, registries-conf-ctl", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} 2020-12-05T17:27:36.355 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:36 +0000 (0:00:00.096) 0:01:58.237 ***** 2020-12-05T17:27:36.393 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "", "stderr_lines": [], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-bNM44W-build\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nBuilding wheels for collected packages: docopt\n Running setup.py bdist_wheel for docopt: started\n Running setup.py bdist_wheel for docopt: finished with status 'done'\n Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\nSuccessfully built docopt\nInstalling collected packages: docopt, toml, typing, registries-conf-ctl\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-bNM44W-build", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packag 2020-12-05T17:27:36.395 INFO:teuthology.task.ansible.out:es/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Building wheels for collected packages: docopt", " Running setup.py bdist_wheel for docopt: started", " Running setup.py bdist_wheel for docopt: finished with status 'done'", " Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e", "Successfully built docopt", "Installing collected packages: docopt, toml, typing, registries-conf-ctl", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} 2020-12-05T17:27:36.431 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:36 +0000 (0:00:00.076) 0:01:58.313 ***** 2020-12-05T17:27:36.439 INFO:teuthology.task.ansible.out: TASK [container-host : Find registries-conf-ctl again] ************************* ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1607189255.5986593, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "checksum": "7c9342d5a9fce575259687fd2510a929c13d7ab7", "ctime": 1607189255.5986593, "dev": 2049, "device_type": 0, "executable": true, "exists": true, "gid": 0, "gr_name": "root", "inode": 66903, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/x-python", "mode": "0755", "mtime": 1607189255.5986593, "nlink": 1, "path": "/usr/local/bin/registries-conf-ctl", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 435, "uid": 0, "version": "4293794299", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": true, "xoth": true, "xusr": true}} 2020-12-05T17:27:36.478 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:36 +0000 (0:00:00.046) 0:01:58.360 ***** 2020-12-05T17:27:36.486 INFO:teuthology.task.ansible.out: TASK [container-host : Find registries-conf-ctl] ******************************* 2020-12-05T17:27:36.487 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-12-05T17:27:36.524 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:36 +0000 (0:00:00.046) 0:01:58.406 ***** 2020-12-05T17:27:36.530 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2020-12-05T17:27:36.532 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "", "stderr_lines": [], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-NHZAXO-build\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nBuilding wheels for collected packages: docopt\n Running setup.py bdist_wheel for docopt: started\n Running setup.py bdist_wheel for docopt: finished with status 'done'\n Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\nSuccessfully built docopt\nInstalling collected packages: docopt, toml, typing, registries-conf-ctl\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-NHZAXO-build", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packag 2020-12-05T17:27:36.533 INFO:teuthology.task.ansible.out:es/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Building wheels for collected packages: docopt", " Running setup.py bdist_wheel for docopt: started", " Running setup.py bdist_wheel for docopt: finished with status 'done'", " Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e", "Successfully built docopt", "Installing collected packages: docopt, toml, typing, registries-conf-ctl", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} TASK [container-host : set_fact] *********************************************** skipping: [smithi032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:36.570 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:36 +0000 (0:00:00.045) 0:01:58.451 ***** 2020-12-05T17:27:36.604 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:36 +0000 (0:00:00.033) 0:01:58.485 ***** 2020-12-05T17:27:36.610 INFO:teuthology.task.ansible.out: TASK [container-host : Find registries-conf-ctl] ******************************* 2020-12-05T17:27:36.611 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} TASK [container-host : set_fact] *********************************************** ok: [smithi032.front.sepia.ceph.com] => {"ansible_facts": {"rcc_path": "/usr/local/bin/registries-conf-ctl"}, "changed": false} 2020-12-05T17:27:36.649 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:36 +0000 (0:00:00.044) 0:01:58.530 ***** 2020-12-05T17:27:36.687 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:36 +0000 (0:00:00.038) 0:01:58.569 ***** 2020-12-05T17:27:36.693 INFO:teuthology.task.ansible.out: TASK [container-host : Find registries-conf-ctl again] ************************* 2020-12-05T17:27:36.695 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1607189255.65558, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "checksum": "7c9342d5a9fce575259687fd2510a929c13d7ab7", "ctime": 1607189255.65558, "dev": 2049, "device_type": 0, "executable": true, "exists": true, "gid": 0, "gr_name": "root", "inode": 66903, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/x-python", "mode": "0755", "mtime": 1607189255.65558, "nlink": 1, "path": "/usr/local/bin/registries-conf-ctl", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 435, "uid": 0, "version": "1395870839", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": true, "xoth": true, "xusr": true}} TASK [container-host : Install registries-conf-ctl] **************************** 2020-12-05T17:27:36.695 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "", "stderr_lines": [], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-XrktcB-build\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799ea 2020-12-05T17:27:36.697 INFO:teuthology.task.ansible.out:c02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nBuilding wheels for collected packages: docopt\n Running setup.py bdist_wheel for docopt: started\n Running setup.py bdist_wheel for docopt: finished with status 'done'\n Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\nSuccessfully built docopt\nInstalling collected packages: docopt, toml, typing, registries-conf-ctl\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-XrktcB-build", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Building wheels for collected packages: docopt", " Running setup.py bdist_wheel for docopt: started", " Running setup.py bdist_wheel for docopt: finished with status 'done'", " Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e", "Successfully built docopt", "Installing collected packages: docopt, toml, typing, registries-conf-ctl", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf- 2020-12-05T17:27:36.700 INFO:teuthology.task.ansible.out:ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} 2020-12-05T17:27:36.734 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:36 +0000 (0:00:00.046) 0:01:58.615 ***** 2020-12-05T17:27:36.768 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:36 +0000 (0:00:00.034) 0:01:58.650 ***** 2020-12-05T17:27:36.775 INFO:teuthology.task.ansible.out: TASK [container-host : Find registries-conf-ctl] ******************************* 2020-12-05T17:27:36.777 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} TASK [container-host : set_fact] *********************************************** skipping: [smithi002.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [container-host : Check for docker's daemon.json] ************************* ok: [smithi032.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-12-05T17:27:36.815 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:36 +0000 (0:00:00.046) 0:01:58.697 ***** 2020-12-05T17:27:36.852 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:36 +0000 (0:00:00.037) 0:01:58.734 ***** 2020-12-05T17:27:36.885 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:36 +0000 (0:00:00.032) 0:01:58.767 ***** 2020-12-05T17:27:36.892 INFO:teuthology.task.ansible.out: TASK [container-host : Find registries-conf-ctl again] ************************* 2020-12-05T17:27:36.893 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1607189255.9643805, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "checksum": "7c9342d5a9fce575259687fd2510a929c13d7ab7", "ctime": 1607189255.9643805, "dev": 2049, "device_type": 0, "executable": true, "exists": true, "gid": 0, "gr_name": "root", "inode": 66903, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/x-python", "mode": "0755", "mtime": 1607189255.9643805, "nlink": 1, "path": "/usr/local/bin/registries-conf-ctl", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 435, "uid": 0, "version": "924406076", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": true, "xoth": true, "xusr": true}} TASK [container-host : set_fact] *********************************************** ok: [smithi002.front.sepia.ceph.com] => {"ansible_facts": {"rcc_path": "/usr/local/bin/registries-conf-ctl"}, "changed": false} 2020-12-05T17:27:36.932 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:36 +0000 (0:00:00.046) 0:01:58.814 ***** 2020-12-05T17:27:36.967 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:36 +0000 (0:00:00.034) 0:01:58.848 ***** 2020-12-05T17:27:36.973 INFO:teuthology.task.ansible.out: TASK [container-host : Find registries-conf-ctl] ******************************* 2020-12-05T17:27:36.975 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-12-05T17:27:37.012 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:37 +0000 (0:00:00.045) 0:01:58.893 ***** 2020-12-05T17:27:37.018 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2020-12-05T17:27:37.020 INFO:teuthology.task.ansible.out:skipping: [smithi036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:37.057 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:37 +0000 (0:00:00.045) 0:01:58.939 ***** 2020-12-05T17:27:37.064 INFO:teuthology.task.ansible.out: TASK [container-host : Find registries-conf-ctl again] ************************* 2020-12-05T17:27:37.065 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1607189255.8594189, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "checksum": "7c9342d5a9fce575259687fd2510a929c13d7ab7", "ctime": 1607189255.8594189, "dev": 2049, "device_type": 0, "executable": true, "exists": true, "gid": 0, "gr_name": "root", "inode": 66903, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/x-python", "mode": "0755", "mtime": 1607189255.8594189, "nlink": 1, "path": "/usr/local/bin/registries-conf-ctl", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 435, "uid": 0, "version": "1813400389", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": true, "xoth": true, "xusr": true}} 2020-12-05T17:27:37.106 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:37 +0000 (0:00:00.048) 0:01:58.988 ***** 2020-12-05T17:27:37.112 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2020-12-05T17:27:37.114 INFO:teuthology.task.ansible.out:ok: [smithi002.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} TASK [container-host : set_fact] *********************************************** ok: [smithi036.front.sepia.ceph.com] => {"ansible_facts": {"rcc_path": "/usr/local/bin/registries-conf-ctl"}, "changed": false} 2020-12-05T17:27:37.156 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:37 +0000 (0:00:00.050) 0:01:59.038 ***** 2020-12-05T17:27:37.201 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:37 +0000 (0:00:00.044) 0:01:59.082 ***** 2020-12-05T17:27:37.206 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2020-12-05T17:27:37.208 INFO:teuthology.task.ansible.out:skipping: [smithi053.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [container-host : Find registries-conf-ctl again] ************************* 2020-12-05T17:27:37.208 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1607189256.1448383, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "checksum": "7c9342d5a9fce575259687fd2510a929c13d7ab7", "ctime": 1607189256.1448383, "dev": 2049, "device_type": 0, "executable": true, "exists": true, "gid": 0, "gr_name": "root", "inode": 66903, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/x-python", "mode": "0755", "mtime": 1607189256.1448383, "nlink": 1, "path": "/usr/local/bin/registries-conf-ctl", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 435, "uid": 0, "version": "3731766962", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": true, "xoth": true, "xusr": true}} 2020-12-05T17:27:37.210 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ changed: [smithi032.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1607189256.883485-170857260415520/source", "state": "file", "uid": 0} 2020-12-05T17:27:37.253 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:37 +0000 (0:00:00.052) 0:01:59.135 ***** 2020-12-05T17:27:37.302 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:37 +0000 (0:00:00.048) 0:01:59.184 ***** 2020-12-05T17:27:37.339 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:37 +0000 (0:00:00.036) 0:01:59.221 ***** 2020-12-05T17:27:37.345 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2020-12-05T17:27:37.347 INFO:teuthology.task.ansible.out:ok: [smithi036.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} TASK [container-host : set_fact] *********************************************** ok: [smithi053.front.sepia.ceph.com] => {"ansible_facts": {"rcc_path": "/usr/local/bin/registries-conf-ctl"}, "changed": false} 2020-12-05T17:27:37.385 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:37 +0000 (0:00:00.045) 0:01:59.266 ***** 2020-12-05T17:27:37.420 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:37 +0000 (0:00:00.034) 0:01:59.301 ***** 2020-12-05T17:27:37.426 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2020-12-05T17:27:37.428 INFO:teuthology.task.ansible.out:skipping: [smithi061.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-12-05T17:27:37.466 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:37 +0000 (0:00:00.045) 0:01:59.347 ***** 2020-12-05T17:27:37.483 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ 2020-12-05T17:27:37.485 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1607189257.194328-50188593799117/source", "state": "file", "uid": 0} 2020-12-05T17:27:37.523 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:37 +0000 (0:00:00.057) 0:01:59.405 ***** 2020-12-05T17:27:37.530 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2020-12-05T17:27:37.532 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"ansible_facts": {"rcc_path": "/usr/local/bin/registries-conf-ctl"}, "changed": false} 2020-12-05T17:27:37.575 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:37 +0000 (0:00:00.051) 0:01:59.457 ***** 2020-12-05T17:27:37.582 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2020-12-05T17:27:37.584 INFO:teuthology.task.ansible.out:ok: [smithi053.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-12-05T17:27:37.633 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:37 +0000 (0:00:00.058) 0:01:59.515 ***** 2020-12-05T17:27:37.661 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2020-12-05T17:27:37.663 INFO:teuthology.task.ansible.out:changed: [smithi032.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/local/bin/registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.269480", "end": "2020-12-05 17:27:37.819114", "rc": 0, "start": "2020-12-05 17:27:37.549634", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:37.741 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2020-12-05T17:27:37.742 INFO:teuthology.task.ansible.out:ok: [smithi061.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ changed: [smithi036.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1607189257.4185493-188069375563761/source", "state": "file", "uid": 0} 2020-12-05T17:27:37.777 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:37 +0000 (0:00:00.144) 0:01:59.659 ***** 2020-12-05T17:27:37.813 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:37 +0000 (0:00:00.035) 0:01:59.694 ***** 2020-12-05T17:27:37.819 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2020-12-05T17:27:37.821 INFO:teuthology.task.ansible.out:changed: [smithi002.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/local/bin/registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.127620", "end": "2020-12-05 17:27:37.771040", "rc": 0, "start": "2020-12-05 17:27:37.643420", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:37.962 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ 2020-12-05T17:27:37.964 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1607189257.6677349-279510259774956/source", "state": "file", "uid": 0} 2020-12-05T17:27:37.997 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:37 +0000 (0:00:00.184) 0:01:59.879 ***** 2020-12-05T17:27:38.035 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2020-12-05T17:27:38.036 INFO:teuthology.task.ansible.out:changed: [smithi036.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/local/bin/registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.111252", "end": "2020-12-05 17:27:38.320192", "rc": 0, "start": "2020-12-05 17:27:38.208940", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:38.157 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ 2020-12-05T17:27:38.159 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1607189257.8482537-175719627079108/source", "state": "file", "uid": 0} 2020-12-05T17:27:38.193 INFO:teuthology.task.ansible.out:Saturday 05 December 2020 17:27:38 +0000 (0:00:00.195) 0:02:00.074 ***** 2020-12-05T17:27:38.273 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2020-12-05T17:27:38.275 INFO:teuthology.task.ansible.out:changed: [smithi053.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/local/bin/registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.143936", "end": "2020-12-05 17:27:38.271024", "rc": 0, "start": "2020-12-05 17:27:38.127088", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:38.466 INFO:teuthology.task.ansible.out:changed: [smithi061.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/local/bin/registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.145438", "end": "2020-12-05 17:27:38.589873", "rc": 0, "start": "2020-12-05 17:27:38.444435", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-12-05T17:27:38.560 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2020-12-05T17:27:38.564 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2020-12-05T17:27:38.568 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched PLAY RECAP ********************************************************************* 2020-12-05T17:27:38.569 INFO:teuthology.task.ansible.out:smithi002.front.sepia.ceph.com : ok=122 changed=32 unreachable=0 failed=0 skipped=190 rescued=0 ignored=0 smithi032.front.sepia.ceph.com : ok=122 changed=32 unreachable=0 failed=0 skipped=190 rescued=0 ignored=0 smithi036.front.sepia.ceph.com : ok=122 changed=32 unreachable=0 failed=0 skipped=190 rescued=0 ignored=0 smithi053.front.sepia.ceph.com : ok=122 changed=32 unreachable=0 failed=0 skipped=190 rescued=0 ignored=0 smithi061.front.sepia.ceph.com : ok=122 changed=32 unreachable=0 failed=0 skipped=190 rescued=0 ignored=0 Saturday 05 December 2020 17:27:38 +0000 (0:00:00.375) 0:02:00.450 ***** =============================================================================== testnode : Install packages --------------------------------------------- 5.34s testnode : Ensure packages are not present. ----------------------------- 4.45s testnode : Zap all non-root disks --------------------------------------- 3.17s container-host : Install registries-conf-ctl ---------------------------- 3.00s Gathering Facts --------------------------------------------------------- 1.69s common : Update apt cache ----------------------------------------------- 0.88s testnode : Install python-apt ------------------------------------------- 0.76s Gathering Facts --------------------------------------------------------- 0.72s testnode : Make sure apt dependencies are installed --------------------- 0.70s Gathering Facts --------------------------------------------------------- 0.67s ansible-managed : Add authorized keys for the ansible user. ------------- 0.63s testnode : List any leftover Ceph artifacts from previous jobs ---------- 0.61s container-host : Install container packages ----------------------------- 0.59s common : Set the hardware clock ----------------------------------------- 0.52s testnode : Upgrade packages --------------------------------------------- 0.50s ansi 2020-12-05T17:27:38.571 INFO:teuthology.task.ansible.out:ble-managed : Create the cephlab_sudo sudoers.d file. --------------- 0.48s container-host : Add local docker.io registry mirror -------------------- 0.38s testnode : Upload CPAN config for root. --------------------------------- 0.34s testnode : Update apt cache. -------------------------------------------- 0.32s ansible-managed : Create the ansible user. ------------------------------ 0.30s 2020-12-05T17:27:38.841 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi032.front.sepia.ceph.com'), Remote(name='ubuntu@smithi061.front.sepia.ceph.com'), Remote(name='ubuntu@smithi036.front.sepia.ceph.com'), Remote(name='ubuntu@smithi002.front.sepia.ceph.com'), Remote(name='ubuntu@smithi053.front.sepia.ceph.com')] 2020-12-05T17:27:38.842 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-12-05T17:27:38.843 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:27:38.911 INFO:teuthology.orchestra.run.smithi032:> true 2020-12-05T17:27:39.106 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-12-05T17:27:39.107 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:27:39.242 INFO:teuthology.orchestra.run.smithi061:> true 2020-12-05T17:27:39.438 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-12-05T17:27:39.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:27:39.518 INFO:teuthology.orchestra.run.smithi036:> true 2020-12-05T17:27:39.715 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-12-05T17:27:39.715 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:27:39.792 INFO:teuthology.orchestra.run.smithi002:> true 2020-12-05T17:27:39.999 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-12-05T17:27:39.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-12-05T17:27:40.073 INFO:teuthology.orchestra.run.smithi053:> true 2020-12-05T17:27:40.267 INFO:teuthology.run_tasks:Running task clock... 2020-12-05T17:27:40.276 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2020-12-05T17:27:40.276 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-12-05T17:27:40.277 INFO:teuthology.orchestra.run.smithi032:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-12-05T17:27:40.327 INFO:teuthology.orchestra.run.smithi032.stdout: 5 Dec 17:27:40 ntpd[6297]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-12-05T17:27:40.327 INFO:teuthology.orchestra.run.smithi032.stdout: 5 Dec 17:27:40 ntpd[6297]: Command line: ntpd -gq 2020-12-05T17:27:40.329 INFO:teuthology.orchestra.run.smithi032.stdout: 5 Dec 17:27:40 ntpd[6297]: proto: precision = 0.112 usec (-23) 2020-12-05T17:27:40.329 INFO:teuthology.orchestra.run.smithi032.stdout: 5 Dec 17:27:40 ntpd[6297]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-12-05T17:27:40.329 INFO:teuthology.orchestra.run.smithi032.stdout: 5 Dec 17:27:40 ntpd[6297]: restrict ::: KOD does nothing without LIMITED. 2020-12-05T17:27:40.330 INFO:teuthology.orchestra.run.smithi032.stdout: 5 Dec 17:27:40 ntpd[6297]: Listen and drop on 0 v6wildcard [::]:123 2020-12-05T17:27:40.330 INFO:teuthology.orchestra.run.smithi032.stdout: 5 Dec 17:27:40 ntpd[6297]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-12-05T17:27:40.330 INFO:teuthology.orchestra.run.smithi032.stdout: 5 Dec 17:27:40 ntpd[6297]: Listen normally on 2 lo 127.0.0.1:123 2020-12-05T17:27:40.331 INFO:teuthology.orchestra.run.smithi032.stdout: 5 Dec 17:27:40 ntpd[6297]: Listen normally on 3 ens1f0 172.21.15.32:123 2020-12-05T17:27:40.331 INFO:teuthology.orchestra.run.smithi032.stdout: 5 Dec 17:27:40 ntpd[6297]: Listen normally on 4 lo [::1]:123 2020-12-05T17:27:40.331 INFO:teuthology.orchestra.run.smithi032.stdout: 5 Dec 17:27:40 ntpd[6297]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febc:154c%3]:123 2020-12-05T17:27:40.331 INFO:teuthology.orchestra.run.smithi032.stdout: 5 Dec 17:27:40 ntpd[6297]: Listening on routing socket on fd #22 for interface updates 2020-12-05T17:27:40.331 INFO:teuthology.orchestra.run.smithi032.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-12-05T17:27:40.332 INFO:teuthology.orchestra.run.smithi032.stderr:restrict ::: KOD does nothing without LIMITED. 2020-12-05T17:27:41.328 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:41 ntpd[6297]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:41.328 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:41 ntpd[6297]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:41.329 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:41 ntpd[6297]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:41.329 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:41 ntpd[6297]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:42.328 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:42 ntpd[6297]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:42.328 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:42 ntpd[6297]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:42.329 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:42 ntpd[6297]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:42.329 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:42 ntpd[6297]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:43.329 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:43 ntpd[6297]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:43.329 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:43 ntpd[6297]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:43.330 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:43 ntpd[6297]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:43.330 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:43 ntpd[6297]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:43.330 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:43 ntpd[6297]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:43.330 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:43 ntpd[6297]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:43.331 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:43 ntpd[6297]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:43.331 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:43 ntpd[6297]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:44.331 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:44 ntpd[6297]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:44.332 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:44 ntpd[6297]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:44.332 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:44 ntpd[6297]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:44.333 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:44 ntpd[6297]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:44.333 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:44 ntpd[6297]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:44.334 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:44 ntpd[6297]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:44.334 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:44 ntpd[6297]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:44.335 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:44 ntpd[6297]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:45.330 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:45 ntpd[6297]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:45.331 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:45 ntpd[6297]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:45.331 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:45 ntpd[6297]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:45.331 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:45 ntpd[6297]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:45.331 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:45 ntpd[6297]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:45.331 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:45 ntpd[6297]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:45.332 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:45 ntpd[6297]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:45.332 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:45 ntpd[6297]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:46.330 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:46 ntpd[6297]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:46.331 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:46 ntpd[6297]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:46.331 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:46 ntpd[6297]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:46.331 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:46 ntpd[6297]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:46.331 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:46 ntpd[6297]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:46.332 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:46 ntpd[6297]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:46.332 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:46 ntpd[6297]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:46.332 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:46 ntpd[6297]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:47.331 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:47 ntpd[6297]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:47.331 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:47 ntpd[6297]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:47.332 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:47 ntpd[6297]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:47.332 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:47 ntpd[6297]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:47.332 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:47 ntpd[6297]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-12-05T17:27:47.332 INFO:teuthology.orchestra.run.smithi032.stderr: 5 Dec 17:27:47 ntpd[6297]: can't open /var/log/ntpstats/loopstats.20201205: Permission denied 2020-12-05T17:27:47.333 INFO:teuthology.orchestra.run.smithi032.stdout: 5 Dec 17:27:47 ntpd[6297]: ntpd: time set -0.173066 s 2020-12-05T17:27:47.333 INFO:teuthology.orchestra.run.smithi032.stdout:ntpd: time set -0.173066s 2020-12-05T17:27:47.388 INFO:teuthology.orchestra.run.smithi032.stdout: remote refid st t when poll reach delay offset jitter 2020-12-05T17:27:47.389 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================== 2020-12-05T17:27:47.389 INFO:teuthology.orchestra.run.smithi032.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-12-05T17:27:47.389 INFO:teuthology.orchestra.run.smithi032.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-12-05T17:27:47.389 INFO:teuthology.orchestra.run.smithi032.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-12-05T17:27:47.389 INFO:teuthology.orchestra.run.smithi032.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-12-05T17:27:47.391 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-12-05T17:27:47.391 INFO:teuthology.orchestra.run.smithi061:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-12-05T17:27:47.442 INFO:teuthology.orchestra.run.smithi061.stdout: 5 Dec 17:27:47 ntpd[6136]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-12-05T17:27:47.443 INFO:teuthology.orchestra.run.smithi061.stdout: 5 Dec 17:27:47 ntpd[6136]: Command line: ntpd -gq 2020-12-05T17:27:47.443 INFO:teuthology.orchestra.run.smithi061.stdout: 5 Dec 17:27:47 ntpd[6136]: proto: precision = 0.126 usec (-23) 2020-12-05T17:27:47.444 INFO:teuthology.orchestra.run.smithi061.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-12-05T17:27:47.444 INFO:teuthology.orchestra.run.smithi061.stderr:restrict ::: KOD does nothing without LIMITED. 2020-12-05T17:27:47.445 INFO:teuthology.orchestra.run.smithi061.stdout: 5 Dec 17:27:47 ntpd[6136]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-12-05T17:27:47.445 INFO:teuthology.orchestra.run.smithi061.stdout: 5 Dec 17:27:47 ntpd[6136]: restrict ::: KOD does nothing without LIMITED. 2020-12-05T17:27:47.445 INFO:teuthology.orchestra.run.smithi061.stdout: 5 Dec 17:27:47 ntpd[6136]: Listen and drop on 0 v6wildcard [::]:123 2020-12-05T17:27:47.445 INFO:teuthology.orchestra.run.smithi061.stdout: 5 Dec 17:27:47 ntpd[6136]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-12-05T17:27:47.446 INFO:teuthology.orchestra.run.smithi061.stdout: 5 Dec 17:27:47 ntpd[6136]: Listen normally on 2 lo 127.0.0.1:123 2020-12-05T17:27:47.446 INFO:teuthology.orchestra.run.smithi061.stdout: 5 Dec 17:27:47 ntpd[6136]: Listen normally on 3 enp3s0f1 172.21.15.61:123 2020-12-05T17:27:47.447 INFO:teuthology.orchestra.run.smithi061.stdout: 5 Dec 17:27:47 ntpd[6136]: Listen normally on 4 lo [::1]:123 2020-12-05T17:27:47.447 INFO:teuthology.orchestra.run.smithi061.stdout: 5 Dec 17:27:47 ntpd[6136]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:feb6:f8b9%5]:123 2020-12-05T17:27:47.447 INFO:teuthology.orchestra.run.smithi061.stdout: 5 Dec 17:27:47 ntpd[6136]: Listening on routing socket on fd #22 for interface updates 2020-12-05T17:27:48.444 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:48 ntpd[6136]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:48.444 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:48 ntpd[6136]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:48.444 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:48 ntpd[6136]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:48.445 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:48 ntpd[6136]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:48.445 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:48 ntpd[6136]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:48.445 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:48 ntpd[6136]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:48.445 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:48 ntpd[6136]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:48.445 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:48 ntpd[6136]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:49.444 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:49 ntpd[6136]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:49.444 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:49 ntpd[6136]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:49.445 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:49 ntpd[6136]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:49.445 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:49 ntpd[6136]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:50.445 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:50 ntpd[6136]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:50.445 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:50 ntpd[6136]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:50.446 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:50 ntpd[6136]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:50.446 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:50 ntpd[6136]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:50.446 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:50 ntpd[6136]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:50.446 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:50 ntpd[6136]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:50.447 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:50 ntpd[6136]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:50.447 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:50 ntpd[6136]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:50.447 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:50 ntpd[6136]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:50.447 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:50 ntpd[6136]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:50.448 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:50 ntpd[6136]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:50.448 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:50 ntpd[6136]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:51.445 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:51 ntpd[6136]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:51.445 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:51 ntpd[6136]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:51.446 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:51 ntpd[6136]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:51.446 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:51 ntpd[6136]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:52.446 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:52 ntpd[6136]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:52.447 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:52 ntpd[6136]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:52.447 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:52 ntpd[6136]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:52.447 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:52 ntpd[6136]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:52.447 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:52 ntpd[6136]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:52.447 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:52 ntpd[6136]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:52.448 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:52 ntpd[6136]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:52.448 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:52 ntpd[6136]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:52.448 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:52 ntpd[6136]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:52.448 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:52 ntpd[6136]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:52.449 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:52 ntpd[6136]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:52.449 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:52 ntpd[6136]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:53.446 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:53 ntpd[6136]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:53.446 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:53 ntpd[6136]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:53.447 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:53 ntpd[6136]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:53.447 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:53 ntpd[6136]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:54.448 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:54 ntpd[6136]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:54.448 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:54 ntpd[6136]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:54.448 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:54 ntpd[6136]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:54.448 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:54 ntpd[6136]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:54.449 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:54 ntpd[6136]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:54.449 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:54 ntpd[6136]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:54.449 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:54 ntpd[6136]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:54.449 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:54 ntpd[6136]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:54.449 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:54 ntpd[6136]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-12-05T17:27:54.450 INFO:teuthology.orchestra.run.smithi061.stderr: 5 Dec 17:27:54 ntpd[6136]: can't open /var/log/ntpstats/loopstats.20201205: Permission denied 2020-12-05T17:27:54.450 INFO:teuthology.orchestra.run.smithi061.stdout: 5 Dec 17:27:54 ntpd[6136]: ntpd: time set -0.133371 s 2020-12-05T17:27:54.450 INFO:teuthology.orchestra.run.smithi061.stdout:ntpd: time set -0.133371s 2020-12-05T17:27:54.505 INFO:teuthology.orchestra.run.smithi061.stdout: remote refid st t when poll reach delay offset jitter 2020-12-05T17:27:54.506 INFO:teuthology.orchestra.run.smithi061.stdout:============================================================================== 2020-12-05T17:27:54.506 INFO:teuthology.orchestra.run.smithi061.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-12-05T17:27:54.506 INFO:teuthology.orchestra.run.smithi061.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-12-05T17:27:54.506 INFO:teuthology.orchestra.run.smithi061.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-12-05T17:27:54.507 INFO:teuthology.orchestra.run.smithi061.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-12-05T17:27:54.507 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-12-05T17:27:54.508 INFO:teuthology.orchestra.run.smithi036:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-12-05T17:27:54.566 INFO:teuthology.orchestra.run.smithi036.stdout: 5 Dec 17:27:54 ntpd[6038]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-12-05T17:27:54.567 INFO:teuthology.orchestra.run.smithi036.stdout: 5 Dec 17:27:54 ntpd[6038]: Command line: ntpd -gq 2020-12-05T17:27:54.568 INFO:teuthology.orchestra.run.smithi036.stdout: 5 Dec 17:27:54 ntpd[6038]: proto: precision = 0.127 usec (-23) 2020-12-05T17:27:54.569 INFO:teuthology.orchestra.run.smithi036.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-12-05T17:27:54.569 INFO:teuthology.orchestra.run.smithi036.stderr:restrict ::: KOD does nothing without LIMITED. 2020-12-05T17:27:54.569 INFO:teuthology.orchestra.run.smithi036.stdout: 5 Dec 17:27:54 ntpd[6038]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-12-05T17:27:54.570 INFO:teuthology.orchestra.run.smithi036.stdout: 5 Dec 17:27:54 ntpd[6038]: restrict ::: KOD does nothing without LIMITED. 2020-12-05T17:27:54.570 INFO:teuthology.orchestra.run.smithi036.stdout: 5 Dec 17:27:54 ntpd[6038]: Listen and drop on 0 v6wildcard [::]:123 2020-12-05T17:27:54.570 INFO:teuthology.orchestra.run.smithi036.stdout: 5 Dec 17:27:54 ntpd[6038]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-12-05T17:27:54.570 INFO:teuthology.orchestra.run.smithi036.stdout: 5 Dec 17:27:54 ntpd[6038]: Listen normally on 2 lo 127.0.0.1:123 2020-12-05T17:27:54.570 INFO:teuthology.orchestra.run.smithi036.stdout: 5 Dec 17:27:54 ntpd[6038]: Listen normally on 3 ens1f0 172.21.15.36:123 2020-12-05T17:27:54.571 INFO:teuthology.orchestra.run.smithi036.stdout: 5 Dec 17:27:54 ntpd[6038]: Listen normally on 4 lo [::1]:123 2020-12-05T17:27:54.571 INFO:teuthology.orchestra.run.smithi036.stdout: 5 Dec 17:27:54 ntpd[6038]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:2542%3]:123 2020-12-05T17:27:54.572 INFO:teuthology.orchestra.run.smithi036.stdout: 5 Dec 17:27:54 ntpd[6038]: Listening on routing socket on fd #22 for interface updates 2020-12-05T17:27:55.568 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:55 ntpd[6038]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:55.568 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:55 ntpd[6038]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:55.569 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:55 ntpd[6038]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:55.569 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:55 ntpd[6038]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:55.569 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:55 ntpd[6038]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:55.569 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:55 ntpd[6038]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:55.570 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:55 ntpd[6038]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:55.570 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:55 ntpd[6038]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:56.568 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:56 ntpd[6038]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:56.568 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:56 ntpd[6038]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:56.569 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:56 ntpd[6038]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:56.569 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:56 ntpd[6038]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:57.570 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:57 ntpd[6038]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:57.570 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:57 ntpd[6038]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:57.570 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:57 ntpd[6038]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:57.571 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:57 ntpd[6038]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:57.571 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:57 ntpd[6038]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:57.571 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:57 ntpd[6038]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:57.571 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:57 ntpd[6038]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:57.571 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:57 ntpd[6038]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:58.570 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:58 ntpd[6038]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:58.570 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:58 ntpd[6038]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:58.570 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:58 ntpd[6038]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:58.570 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:58 ntpd[6038]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:58.571 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:58 ntpd[6038]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:58.571 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:58 ntpd[6038]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:58.571 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:58 ntpd[6038]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:58.571 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:58 ntpd[6038]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:59.571 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:59 ntpd[6038]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:59.571 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:59 ntpd[6038]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:59.571 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:59 ntpd[6038]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:59.572 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:59 ntpd[6038]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:27:59.572 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:59 ntpd[6038]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:27:59.572 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:59 ntpd[6038]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:27:59.572 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:59 ntpd[6038]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:27:59.572 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:27:59 ntpd[6038]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:00.570 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:28:00 ntpd[6038]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:00.571 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:28:00 ntpd[6038]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:00.571 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:28:00 ntpd[6038]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:00.571 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:28:00 ntpd[6038]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:00.572 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:28:00 ntpd[6038]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:00.572 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:28:00 ntpd[6038]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:00.573 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:28:00 ntpd[6038]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:00.573 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:28:00 ntpd[6038]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:01.571 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:28:01 ntpd[6038]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:01.571 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:28:01 ntpd[6038]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:01.572 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:28:01 ntpd[6038]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:01.572 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:28:01 ntpd[6038]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:01.572 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:28:01 ntpd[6038]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-12-05T17:28:01.572 INFO:teuthology.orchestra.run.smithi036.stderr: 5 Dec 17:28:01 ntpd[6038]: can't open /var/log/ntpstats/loopstats.20201205: Permission denied 2020-12-05T17:28:01.573 INFO:teuthology.orchestra.run.smithi036.stdout: 5 Dec 17:28:01 ntpd[6038]: ntpd: time set -0.295007 s 2020-12-05T17:28:01.573 INFO:teuthology.orchestra.run.smithi036.stdout:ntpd: time set -0.295007s 2020-12-05T17:28:01.638 INFO:teuthology.orchestra.run.smithi036.stdout: remote refid st t when poll reach delay offset jitter 2020-12-05T17:28:01.638 INFO:teuthology.orchestra.run.smithi036.stdout:============================================================================== 2020-12-05T17:28:01.638 INFO:teuthology.orchestra.run.smithi036.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-12-05T17:28:01.639 INFO:teuthology.orchestra.run.smithi036.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-12-05T17:28:01.639 INFO:teuthology.orchestra.run.smithi036.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-12-05T17:28:01.639 INFO:teuthology.orchestra.run.smithi036.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-12-05T17:28:01.639 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-12-05T17:28:01.640 INFO:teuthology.orchestra.run.smithi002:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-12-05T17:28:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: 5 Dec 17:28:01 ntpd[6134]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-12-05T17:28:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: 5 Dec 17:28:01 ntpd[6134]: Command line: ntpd -gq 2020-12-05T17:28:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: 5 Dec 17:28:01 ntpd[6134]: proto: precision = 0.094 usec (-23) 2020-12-05T17:28:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: 5 Dec 17:28:01 ntpd[6134]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-12-05T17:28:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: 5 Dec 17:28:01 ntpd[6134]: restrict ::: KOD does nothing without LIMITED. 2020-12-05T17:28:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: 5 Dec 17:28:01 ntpd[6134]: Listen and drop on 0 v6wildcard [::]:123 2020-12-05T17:28:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: 5 Dec 17:28:01 ntpd[6134]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-12-05T17:28:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: 5 Dec 17:28:01 ntpd[6134]: Listen normally on 2 lo 127.0.0.1:123 2020-12-05T17:28:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: 5 Dec 17:28:01 ntpd[6134]: Listen normally on 3 ens1f0 172.21.15.2:123 2020-12-05T17:28:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: 5 Dec 17:28:01 ntpd[6134]: Listen normally on 4 lo [::1]:123 2020-12-05T17:28:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: 5 Dec 17:28:01 ntpd[6134]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1920%3]:123 2020-12-05T17:28:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: 5 Dec 17:28:01 ntpd[6134]: Listening on routing socket on fd #22 for interface updates 2020-12-05T17:28:01.701 INFO:teuthology.orchestra.run.smithi002.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-12-05T17:28:01.701 INFO:teuthology.orchestra.run.smithi002.stderr:restrict ::: KOD does nothing without LIMITED. 2020-12-05T17:28:03.698 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:03 ntpd[6134]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:03.698 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:03 ntpd[6134]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:03.699 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:03 ntpd[6134]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:03.699 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:03 ntpd[6134]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:04.698 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:04 ntpd[6134]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:04.698 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:04 ntpd[6134]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:04.699 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:04 ntpd[6134]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:04.699 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:04 ntpd[6134]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:05.698 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:05 ntpd[6134]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:05.698 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:05 ntpd[6134]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:05.699 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:05 ntpd[6134]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:05.699 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:05 ntpd[6134]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:06.698 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:06 ntpd[6134]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:06.699 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:06 ntpd[6134]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:06.699 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:06 ntpd[6134]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:06.699 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:06 ntpd[6134]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:06.699 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:06 ntpd[6134]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:06.700 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:06 ntpd[6134]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:06.700 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:06 ntpd[6134]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:06.700 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:06 ntpd[6134]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:06.700 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:06 ntpd[6134]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:06.700 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:06 ntpd[6134]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:06.701 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:06 ntpd[6134]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:06.701 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:06 ntpd[6134]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:07.698 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:07 ntpd[6134]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:07.698 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:07 ntpd[6134]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:07.699 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:07 ntpd[6134]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:07.699 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:07 ntpd[6134]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:08.698 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:08 ntpd[6134]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:08.699 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:08 ntpd[6134]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:08.699 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:08 ntpd[6134]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:08.699 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:08 ntpd[6134]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:08.699 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:08 ntpd[6134]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:08.700 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:08 ntpd[6134]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:08.700 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:08 ntpd[6134]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:08.700 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:08 ntpd[6134]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:08.700 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:08 ntpd[6134]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:08.700 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:08 ntpd[6134]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:08.701 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:08 ntpd[6134]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:08.701 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:08 ntpd[6134]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:09.698 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:09 ntpd[6134]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:09.698 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:09 ntpd[6134]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:09.699 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:09 ntpd[6134]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:09.699 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:09 ntpd[6134]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:09.699 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:09 ntpd[6134]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-12-05T17:28:09.699 INFO:teuthology.orchestra.run.smithi002.stderr: 5 Dec 17:28:09 ntpd[6134]: can't open /var/log/ntpstats/loopstats.20201205: Permission denied 2020-12-05T17:28:09.699 INFO:teuthology.orchestra.run.smithi002.stdout: 5 Dec 17:28:09 ntpd[6134]: ntpd: time slew -0.005435 s 2020-12-05T17:28:09.700 INFO:teuthology.orchestra.run.smithi002.stdout:ntpd: time slew -0.005435s 2020-12-05T17:28:09.750 INFO:teuthology.orchestra.run.smithi002.stdout: remote refid st t when poll reach delay offset jitter 2020-12-05T17:28:09.750 INFO:teuthology.orchestra.run.smithi002.stdout:============================================================================== 2020-12-05T17:28:09.750 INFO:teuthology.orchestra.run.smithi002.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-12-05T17:28:09.751 INFO:teuthology.orchestra.run.smithi002.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-12-05T17:28:09.751 INFO:teuthology.orchestra.run.smithi002.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-12-05T17:28:09.751 INFO:teuthology.orchestra.run.smithi002.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-12-05T17:28:09.752 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-12-05T17:28:09.753 INFO:teuthology.orchestra.run.smithi053:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-12-05T17:28:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: 5 Dec 17:28:09 ntpd[6112]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-12-05T17:28:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: 5 Dec 17:28:09 ntpd[6112]: Command line: ntpd -gq 2020-12-05T17:28:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: 5 Dec 17:28:09 ntpd[6112]: proto: precision = 0.071 usec (-24) 2020-12-05T17:28:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: 5 Dec 17:28:09 ntpd[6112]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-12-05T17:28:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: 5 Dec 17:28:09 ntpd[6112]: restrict ::: KOD does nothing without LIMITED. 2020-12-05T17:28:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: 5 Dec 17:28:09 ntpd[6112]: Listen and drop on 0 v6wildcard [::]:123 2020-12-05T17:28:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: 5 Dec 17:28:09 ntpd[6112]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-12-05T17:28:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: 5 Dec 17:28:09 ntpd[6112]: Listen normally on 2 lo 127.0.0.1:123 2020-12-05T17:28:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: 5 Dec 17:28:09 ntpd[6112]: Listen normally on 3 ens1f0 172.21.15.53:123 2020-12-05T17:28:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: 5 Dec 17:28:09 ntpd[6112]: Listen normally on 4 lo [::1]:123 2020-12-05T17:28:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: 5 Dec 17:28:09 ntpd[6112]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:3718%3]:123 2020-12-05T17:28:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: 5 Dec 17:28:09 ntpd[6112]: Listening on routing socket on fd #22 for interface updates 2020-12-05T17:28:09.813 INFO:teuthology.orchestra.run.smithi053.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-12-05T17:28:09.813 INFO:teuthology.orchestra.run.smithi053.stderr:restrict ::: KOD does nothing without LIMITED. 2020-12-05T17:28:11.810 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:11 ntpd[6112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:11.810 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:11 ntpd[6112]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:11.810 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:11 ntpd[6112]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:11.811 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:11 ntpd[6112]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:12.810 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:12 ntpd[6112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:12.811 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:12 ntpd[6112]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:12.811 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:12 ntpd[6112]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:12.811 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:12 ntpd[6112]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:13.812 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:13 ntpd[6112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:13.812 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:13 ntpd[6112]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:13.813 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:13 ntpd[6112]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:13.813 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:13 ntpd[6112]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:13.813 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:13 ntpd[6112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:13.814 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:13 ntpd[6112]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:13.814 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:13 ntpd[6112]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:13.815 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:13 ntpd[6112]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:14.811 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:14 ntpd[6112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:14.812 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:14 ntpd[6112]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:14.812 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:14 ntpd[6112]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:14.812 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:14 ntpd[6112]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:14.812 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:14 ntpd[6112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:14.813 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:14 ntpd[6112]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:14.813 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:14 ntpd[6112]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:14.813 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:14 ntpd[6112]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:15.813 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:15 ntpd[6112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:15.813 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:15 ntpd[6112]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:15.814 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:15 ntpd[6112]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:15.815 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:15 ntpd[6112]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:15.815 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:15 ntpd[6112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:15.816 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:15 ntpd[6112]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:15.817 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:15 ntpd[6112]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:15.818 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:15 ntpd[6112]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:16.812 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:16 ntpd[6112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:16.813 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:16 ntpd[6112]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:16.813 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:16 ntpd[6112]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:16.813 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:16 ntpd[6112]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:16.813 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:16 ntpd[6112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:16.813 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:16 ntpd[6112]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:16.814 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:16 ntpd[6112]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:16.814 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:16 ntpd[6112]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:17.813 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:17 ntpd[6112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:17.814 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:17 ntpd[6112]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:17.814 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:17 ntpd[6112]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:17.814 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:17 ntpd[6112]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:17.814 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:17 ntpd[6112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-12-05T17:28:17.815 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:17 ntpd[6112]: can't open /var/log/ntpstats/rawstats.20201205: Permission denied 2020-12-05T17:28:17.815 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:17 ntpd[6112]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-12-05T17:28:17.815 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:17 ntpd[6112]: can't open /var/log/ntpstats/peerstats.20201205: Permission denied 2020-12-05T17:28:17.815 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:17 ntpd[6112]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-12-05T17:28:17.815 INFO:teuthology.orchestra.run.smithi053.stderr: 5 Dec 17:28:17 ntpd[6112]: can't open /var/log/ntpstats/loopstats.20201205: Permission denied 2020-12-05T17:28:17.816 INFO:teuthology.orchestra.run.smithi053.stdout: 5 Dec 17:28:17 ntpd[6112]: ntpd: time slew +0.006096 s 2020-12-05T17:28:17.816 INFO:teuthology.orchestra.run.smithi053.stdout:ntpd: time slew +0.006096s 2020-12-05T17:28:17.859 INFO:teuthology.orchestra.run.smithi053.stdout: remote refid st t when poll reach delay offset jitter 2020-12-05T17:28:17.859 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================== 2020-12-05T17:28:17.859 INFO:teuthology.orchestra.run.smithi053.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-12-05T17:28:17.859 INFO:teuthology.orchestra.run.smithi053.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-12-05T17:28:17.860 INFO:teuthology.orchestra.run.smithi053.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-12-05T17:28:17.860 INFO:teuthology.orchestra.run.smithi053.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-12-05T17:28:17.860 INFO:teuthology.run_tasks:Running task install... 2020-12-05T17:28:17.876 DEBUG:teuthology.task.install:project ceph 2020-12-05T17:28:17.877 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': '3b1861acb8137d863184795ccadbd5d0cb026674'}} 2020-12-05T17:28:17.877 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': '3b1861acb8137d863184795ccadbd5d0cb026674'} 2020-12-05T17:28:17.877 INFO:teuthology.task.install:Using flavor: basic 2020-12-05T17:28:17.886 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2020-12-05T17:28:17.887 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']} 2020-12-05T17:28:17.887 INFO:teuthology.orchestra.run.smithi032:> sudo lsb_release -is 2020-12-05T17:28:17.942 INFO:teuthology.orchestra.run.smithi032.stdout:Ubuntu 2020-12-05T17:28:17.948 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-12-05T17:28:17.948 INFO:teuthology.orchestra.run.smithi061:> sudo lsb_release -is 2020-12-05T17:28:17.949 INFO:teuthology.orchestra.run.smithi032:> sudo apt-key list | grep Ceph 2020-12-05T17:28:18.002 INFO:teuthology.orchestra.run.smithi061.stdout:Ubuntu 2020-12-05T17:28:18.007 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-12-05T17:28:18.008 INFO:teuthology.orchestra.run.smithi036:> sudo lsb_release -is 2020-12-05T17:28:18.008 INFO:teuthology.orchestra.run.smithi061:> sudo apt-key list | grep Ceph 2020-12-05T17:28:18.042 INFO:teuthology.orchestra.run.smithi032.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-12-05T17:28:18.062 INFO:teuthology.orchestra.run.smithi036.stdout:Ubuntu 2020-12-05T17:28:18.068 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-12-05T17:28:18.068 INFO:teuthology.orchestra.run.smithi002:> sudo lsb_release -is 2020-12-05T17:28:18.069 INFO:teuthology.orchestra.run.smithi036:> sudo apt-key list | grep Ceph 2020-12-05T17:28:18.100 INFO:teuthology.orchestra.run.smithi032.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-12-05T17:28:18.100 INFO:teuthology.orchestra.run.smithi032.stdout:uid [ unknown] Ceph.com (release key) 2020-12-05T17:28:18.102 INFO:teuthology.orchestra.run.smithi032:> sudo lsb_release -is 2020-12-05T17:28:18.103 INFO:teuthology.orchestra.run.smithi061.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-12-05T17:28:18.121 INFO:teuthology.orchestra.run.smithi002.stdout:Ubuntu 2020-12-05T17:28:18.126 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-12-05T17:28:18.126 INFO:teuthology.orchestra.run.smithi053:> sudo lsb_release -is 2020-12-05T17:28:18.127 INFO:teuthology.orchestra.run.smithi002:> sudo apt-key list | grep Ceph 2020-12-05T17:28:18.155 INFO:teuthology.orchestra.run.smithi032.stdout:Ubuntu 2020-12-05T17:28:18.160 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-12-05T17:28:18.160 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2020-12-05T17:28:18.160 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2020-12-05T17:28:18.161 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=3b1861acb8137d863184795ccadbd5d0cb026674 2020-12-05T17:28:18.164 INFO:teuthology.orchestra.run.smithi036.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-12-05T17:28:18.165 INFO:teuthology.orchestra.run.smithi061.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-12-05T17:28:18.165 INFO:teuthology.orchestra.run.smithi061.stdout:uid [ unknown] Ceph.com (release key) 2020-12-05T17:28:18.166 INFO:teuthology.orchestra.run.smithi061:> sudo lsb_release -is 2020-12-05T17:28:18.179 INFO:teuthology.orchestra.run.smithi053.stdout:Ubuntu 2020-12-05T17:28:18.184 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-12-05T17:28:18.185 INFO:teuthology.orchestra.run.smithi053:> sudo apt-key list | grep Ceph 2020-12-05T17:28:18.215 INFO:teuthology.orchestra.run.smithi061.stdout:Ubuntu 2020-12-05T17:28:18.216 INFO:teuthology.orchestra.run.smithi002.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-12-05T17:28:18.220 INFO:teuthology.orchestra.run.smithi036.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-12-05T17:28:18.220 INFO:teuthology.orchestra.run.smithi036.stdout:uid [ unknown] Ceph.com (release key) 2020-12-05T17:28:18.221 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-12-05T17:28:18.222 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2020-12-05T17:28:18.222 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2020-12-05T17:28:18.222 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=3b1861acb8137d863184795ccadbd5d0cb026674 2020-12-05T17:28:18.225 INFO:teuthology.orchestra.run.smithi036:> sudo lsb_release -is 2020-12-05T17:28:18.280 INFO:teuthology.orchestra.run.smithi036.stdout:Ubuntu 2020-12-05T17:28:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-12-05T17:28:18.281 INFO:teuthology.orchestra.run.smithi002.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-12-05T17:28:18.281 INFO:teuthology.orchestra.run.smithi002.stdout:uid [ unknown] Ceph.com (release key) 2020-12-05T17:28:18.281 INFO:teuthology.orchestra.run.smithi002:> sudo lsb_release -is 2020-12-05T17:28:18.283 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-12-05T17:28:18.283 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2020-12-05T17:28:18.283 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2020-12-05T17:28:18.284 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=3b1861acb8137d863184795ccadbd5d0cb026674 2020-12-05T17:28:18.331 INFO:teuthology.orchestra.run.smithi053.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-12-05T17:28:18.332 INFO:teuthology.orchestra.run.smithi053.stdout:uid [ unknown] Ceph.com (release key) 2020-12-05T17:28:18.333 INFO:teuthology.orchestra.run.smithi053:> sudo lsb_release -is 2020-12-05T17:28:18.338 INFO:teuthology.orchestra.run.smithi002.stdout:Ubuntu 2020-12-05T17:28:18.343 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-12-05T17:28:18.343 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2020-12-05T17:28:18.344 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2020-12-05T17:28:18.344 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=3b1861acb8137d863184795ccadbd5d0cb026674 2020-12-05T17:28:18.357 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default/ 2020-12-05T17:28:18.357 INFO:teuthology.task.install.deb:Package version is 15.2.6-607-g3b1861ac-1bionic 2020-12-05T17:28:18.394 INFO:teuthology.orchestra.run.smithi053.stdout:Ubuntu 2020-12-05T17:28:18.394 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-12-05T17:28:18.394 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2020-12-05T17:28:18.395 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2020-12-05T17:28:18.395 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=3b1861acb8137d863184795ccadbd5d0cb026674 2020-12-05T17:28:18.409 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default/ 2020-12-05T17:28:18.409 INFO:teuthology.task.install.deb:Package version is 15.2.6-607-g3b1861ac-1bionic 2020-12-05T17:28:18.448 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default/ 2020-12-05T17:28:18.449 INFO:teuthology.task.install.deb:Package version is 15.2.6-607-g3b1861ac-1bionic 2020-12-05T17:28:18.492 INFO:teuthology.orchestra.run.smithi032:> set -ex 2020-12-05T17:28:18.492 INFO:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-12-05T17:28:18.498 INFO:teuthology.orchestra.run.smithi032.stderr:+ sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-12-05T17:28:18.507 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2020-12-05T17:28:18.507 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2020-12-05T17:28:18.507 INFO:teuthology.orchestra.run.smithi032.stderr:143 bytes copied, 7.3147e-05 s, 2.0 MB/s 2020-12-05T17:28:18.508 INFO:teuthology.orchestra.run.smithi032:> sudo apt-get update 2020-12-05T17:28:18.521 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default/ 2020-12-05T17:28:18.521 INFO:teuthology.task.install.deb:Package version is 15.2.6-607-g3b1861ac-1bionic 2020-12-05T17:28:18.536 INFO:teuthology.orchestra.run.smithi061:> set -ex 2020-12-05T17:28:18.536 INFO:teuthology.orchestra.run.smithi061:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-12-05T17:28:18.541 INFO:teuthology.orchestra.run.smithi061.stderr:+ sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-12-05T17:28:18.555 INFO:teuthology.orchestra.run.smithi061.stderr:0+1 records in 2020-12-05T17:28:18.555 INFO:teuthology.orchestra.run.smithi061.stderr:0+1 records out 2020-12-05T17:28:18.556 INFO:teuthology.orchestra.run.smithi061.stderr:143 bytes copied, 7.3704e-05 s, 1.9 MB/s 2020-12-05T17:28:18.556 INFO:teuthology.orchestra.run.smithi061:> sudo apt-get update 2020-12-05T17:28:18.565 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default/ 2020-12-05T17:28:18.565 INFO:teuthology.task.install.deb:Package version is 15.2.6-607-g3b1861ac-1bionic 2020-12-05T17:28:18.569 INFO:teuthology.orchestra.run.smithi036:> set -ex 2020-12-05T17:28:18.570 INFO:teuthology.orchestra.run.smithi036:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-12-05T17:28:18.575 INFO:teuthology.orchestra.run.smithi036.stderr:+ sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-12-05T17:28:18.584 INFO:teuthology.orchestra.run.smithi036.stderr:0+1 records in 2020-12-05T17:28:18.584 INFO:teuthology.orchestra.run.smithi036.stderr:0+1 records out 2020-12-05T17:28:18.585 INFO:teuthology.orchestra.run.smithi036.stderr:143 bytes copied, 6.9876e-05 s, 2.0 MB/s 2020-12-05T17:28:18.586 INFO:teuthology.orchestra.run.smithi036:> sudo apt-get update 2020-12-05T17:28:18.643 INFO:teuthology.orchestra.run.smithi002:> set -ex 2020-12-05T17:28:18.644 INFO:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-12-05T17:28:18.650 INFO:teuthology.orchestra.run.smithi002.stderr:+ sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-12-05T17:28:18.659 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records in 2020-12-05T17:28:18.659 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records out 2020-12-05T17:28:18.660 INFO:teuthology.orchestra.run.smithi002.stderr:143 bytes copied, 9.376e-05 s, 1.5 MB/s 2020-12-05T17:28:18.661 INFO:teuthology.orchestra.run.smithi002:> sudo apt-get update 2020-12-05T17:28:18.690 INFO:teuthology.orchestra.run.smithi053:> set -ex 2020-12-05T17:28:18.691 INFO:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-12-05T17:28:18.696 INFO:teuthology.orchestra.run.smithi053.stderr:+ sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-12-05T17:28:18.704 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-12-05T17:28:18.705 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records in 2020-12-05T17:28:18.706 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records out 2020-12-05T17:28:18.706 INFO:teuthology.orchestra.run.smithi053.stderr:143 bytes copied, 7.5218e-05 s, 1.9 MB/s 2020-12-05T17:28:18.707 INFO:teuthology.orchestra.run.smithi053:> sudo apt-get update 2020-12-05T17:28:18.777 INFO:teuthology.orchestra.run.smithi032.stdout:Ign:2 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic InRelease 2020-12-05T17:28:18.779 INFO:teuthology.orchestra.run.smithi036.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-12-05T17:28:18.800 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2020-12-05T17:28:18.812 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-12-05T17:28:18.813 INFO:teuthology.orchestra.run.smithi061.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic InRelease 2020-12-05T17:28:18.825 INFO:teuthology.orchestra.run.smithi032.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic Release.gpg 2020-12-05T17:28:18.836 INFO:teuthology.orchestra.run.smithi036.stdout:Ign:2 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic InRelease 2020-12-05T17:28:18.837 INFO:teuthology.orchestra.run.smithi061.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2020-12-05T17:28:18.849 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 Packages [15.8 kB] 2020-12-05T17:28:18.859 INFO:teuthology.orchestra.run.smithi036.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2020-12-05T17:28:18.863 INFO:teuthology.orchestra.run.smithi061.stdout:Hit:3 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-12-05T17:28:18.863 INFO:teuthology.orchestra.run.smithi061.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-12-05T17:28:18.864 INFO:teuthology.orchestra.run.smithi061.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic Release.gpg 2020-12-05T17:28:18.881 INFO:teuthology.orchestra.run.smithi036.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-12-05T17:28:18.884 INFO:teuthology.orchestra.run.smithi036.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic Release.gpg 2020-12-05T17:28:18.888 INFO:teuthology.orchestra.run.smithi061.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 Packages [15.8 kB] 2020-12-05T17:28:18.892 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,081 B] 2020-12-05T17:28:18.893 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-12-05T17:28:18.894 INFO:teuthology.orchestra.run.smithi002.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic InRelease 2020-12-05T17:28:18.908 INFO:teuthology.orchestra.run.smithi036.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 Packages [15.8 kB] 2020-12-05T17:28:18.917 INFO:teuthology.orchestra.run.smithi002.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2020-12-05T17:28:18.931 INFO:teuthology.orchestra.run.smithi061.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,081 B] 2020-12-05T17:28:18.934 INFO:teuthology.orchestra.run.smithi061.stdout:Hit:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-12-05T17:28:18.942 INFO:teuthology.orchestra.run.smithi002.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic Release.gpg 2020-12-05T17:28:18.951 INFO:teuthology.orchestra.run.smithi036.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,081 B] 2020-12-05T17:28:18.955 INFO:teuthology.orchestra.run.smithi002.stdout:Hit:4 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-12-05T17:28:18.955 INFO:teuthology.orchestra.run.smithi002.stdout:Hit:5 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-12-05T17:28:18.959 INFO:teuthology.orchestra.run.smithi053.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic InRelease 2020-12-05T17:28:18.959 INFO:teuthology.orchestra.run.smithi036.stdout:Hit:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-12-05T17:28:18.965 INFO:teuthology.orchestra.run.smithi002.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,081 B] 2020-12-05T17:28:18.981 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-12-05T17:28:18.982 INFO:teuthology.orchestra.run.smithi053.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2020-12-05T17:28:19.002 INFO:teuthology.orchestra.run.smithi002.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 Packages [15.8 kB] 2020-12-05T17:28:19.014 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-12-05T17:28:19.014 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:4 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-12-05T17:28:19.015 INFO:teuthology.orchestra.run.smithi061.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-12-05T17:28:19.015 INFO:teuthology.orchestra.run.smithi053.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic Release.gpg 2020-12-05T17:28:19.026 INFO:teuthology.orchestra.run.smithi002.stdout:Hit:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-12-05T17:28:19.038 INFO:teuthology.orchestra.run.smithi036.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-12-05T17:28:19.039 INFO:teuthology.orchestra.run.smithi053.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 Packages [15.8 kB] 2020-12-05T17:28:19.077 INFO:teuthology.orchestra.run.smithi053.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,081 B] 2020-12-05T17:28:19.083 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-12-05T17:28:19.106 INFO:teuthology.orchestra.run.smithi002.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-12-05T17:28:19.167 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-12-05T17:28:19.317 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 26.6 kB in 1s (44.3 kB/s) 2020-12-05T17:28:19.410 INFO:teuthology.orchestra.run.smithi036.stdout:Fetched 26.6 kB in 1s (42.9 kB/s) 2020-12-05T17:28:19.482 INFO:teuthology.orchestra.run.smithi061.stdout:Fetched 26.6 kB in 1s (36.7 kB/s) 2020-12-05T17:28:19.589 INFO:teuthology.orchestra.run.smithi002.stdout:Fetched 26.6 kB in 1s (36.8 kB/s) 2020-12-05T17:28:19.645 INFO:teuthology.orchestra.run.smithi053.stdout:Fetched 26.6 kB in 1s (36.6 kB/s) 2020-12-05T17:28:20.587 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2020-12-05T17:28:20.637 INFO:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.6-607-g3b1861ac-1bionic cephadm=15.2.6-607-g3b1861ac-1bionic ceph-mds=15.2.6-607-g3b1861ac-1bionic ceph-mgr=15.2.6-607-g3b1861ac-1bionic ceph-common=15.2.6-607-g3b1861ac-1bionic ceph-fuse=15.2.6-607-g3b1861ac-1bionic ceph-test=15.2.6-607-g3b1861ac-1bionic radosgw=15.2.6-607-g3b1861ac-1bionic python3-rados=15.2.6-607-g3b1861ac-1bionic python3-rgw=15.2.6-607-g3b1861ac-1bionic python3-cephfs=15.2.6-607-g3b1861ac-1bionic python3-rbd=15.2.6-607-g3b1861ac-1bionic libcephfs2=15.2.6-607-g3b1861ac-1bionic libcephfs-dev=15.2.6-607-g3b1861ac-1bionic librados2=15.2.6-607-g3b1861ac-1bionic librbd1=15.2.6-607-g3b1861ac-1bionic rbd-fuse=15.2.6-607-g3b1861ac-1bionic python3-cephfs=15.2.6-607-g3b1861ac-1bionic cephfs-shell=15.2.6-607-g3b1861ac-1bionic 2020-12-05T17:28:20.685 INFO:teuthology.orchestra.run.smithi036.stdout:Reading package lists... 2020-12-05T17:28:20.734 INFO:teuthology.orchestra.run.smithi036:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.6-607-g3b1861ac-1bionic cephadm=15.2.6-607-g3b1861ac-1bionic ceph-mds=15.2.6-607-g3b1861ac-1bionic ceph-mgr=15.2.6-607-g3b1861ac-1bionic ceph-common=15.2.6-607-g3b1861ac-1bionic ceph-fuse=15.2.6-607-g3b1861ac-1bionic ceph-test=15.2.6-607-g3b1861ac-1bionic radosgw=15.2.6-607-g3b1861ac-1bionic python3-rados=15.2.6-607-g3b1861ac-1bionic python3-rgw=15.2.6-607-g3b1861ac-1bionic python3-cephfs=15.2.6-607-g3b1861ac-1bionic python3-rbd=15.2.6-607-g3b1861ac-1bionic libcephfs2=15.2.6-607-g3b1861ac-1bionic libcephfs-dev=15.2.6-607-g3b1861ac-1bionic librados2=15.2.6-607-g3b1861ac-1bionic librbd1=15.2.6-607-g3b1861ac-1bionic rbd-fuse=15.2.6-607-g3b1861ac-1bionic python3-cephfs=15.2.6-607-g3b1861ac-1bionic cephfs-shell=15.2.6-607-g3b1861ac-1bionic 2020-12-05T17:28:20.735 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2020-12-05T17:28:20.762 INFO:teuthology.orchestra.run.smithi061.stdout:Reading package lists... 2020-12-05T17:28:20.809 INFO:teuthology.orchestra.run.smithi061:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.6-607-g3b1861ac-1bionic cephadm=15.2.6-607-g3b1861ac-1bionic ceph-mds=15.2.6-607-g3b1861ac-1bionic ceph-mgr=15.2.6-607-g3b1861ac-1bionic ceph-common=15.2.6-607-g3b1861ac-1bionic ceph-fuse=15.2.6-607-g3b1861ac-1bionic ceph-test=15.2.6-607-g3b1861ac-1bionic radosgw=15.2.6-607-g3b1861ac-1bionic python3-rados=15.2.6-607-g3b1861ac-1bionic python3-rgw=15.2.6-607-g3b1861ac-1bionic python3-cephfs=15.2.6-607-g3b1861ac-1bionic python3-rbd=15.2.6-607-g3b1861ac-1bionic libcephfs2=15.2.6-607-g3b1861ac-1bionic libcephfs-dev=15.2.6-607-g3b1861ac-1bionic librados2=15.2.6-607-g3b1861ac-1bionic librbd1=15.2.6-607-g3b1861ac-1bionic rbd-fuse=15.2.6-607-g3b1861ac-1bionic python3-cephfs=15.2.6-607-g3b1861ac-1bionic cephfs-shell=15.2.6-607-g3b1861ac-1bionic 2020-12-05T17:28:20.835 INFO:teuthology.orchestra.run.smithi036.stdout:Reading package lists... 2020-12-05T17:28:20.908 INFO:teuthology.orchestra.run.smithi061.stdout:Reading package lists... 2020-12-05T17:28:20.929 INFO:teuthology.orchestra.run.smithi002.stdout:Reading package lists... 2020-12-05T17:28:20.931 INFO:teuthology.orchestra.run.smithi053.stdout:Reading package lists... 2020-12-05T17:28:20.971 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2020-12-05T17:28:20.972 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2020-12-05T17:28:20.977 INFO:teuthology.orchestra.run.smithi053:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.6-607-g3b1861ac-1bionic cephadm=15.2.6-607-g3b1861ac-1bionic ceph-mds=15.2.6-607-g3b1861ac-1bionic ceph-mgr=15.2.6-607-g3b1861ac-1bionic ceph-common=15.2.6-607-g3b1861ac-1bionic ceph-fuse=15.2.6-607-g3b1861ac-1bionic ceph-test=15.2.6-607-g3b1861ac-1bionic radosgw=15.2.6-607-g3b1861ac-1bionic python3-rados=15.2.6-607-g3b1861ac-1bionic python3-rgw=15.2.6-607-g3b1861ac-1bionic python3-cephfs=15.2.6-607-g3b1861ac-1bionic python3-rbd=15.2.6-607-g3b1861ac-1bionic libcephfs2=15.2.6-607-g3b1861ac-1bionic libcephfs-dev=15.2.6-607-g3b1861ac-1bionic librados2=15.2.6-607-g3b1861ac-1bionic librbd1=15.2.6-607-g3b1861ac-1bionic rbd-fuse=15.2.6-607-g3b1861ac-1bionic python3-cephfs=15.2.6-607-g3b1861ac-1bionic cephfs-shell=15.2.6-607-g3b1861ac-1bionic 2020-12-05T17:28:20.978 INFO:teuthology.orchestra.run.smithi002:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.6-607-g3b1861ac-1bionic cephadm=15.2.6-607-g3b1861ac-1bionic ceph-mds=15.2.6-607-g3b1861ac-1bionic ceph-mgr=15.2.6-607-g3b1861ac-1bionic ceph-common=15.2.6-607-g3b1861ac-1bionic ceph-fuse=15.2.6-607-g3b1861ac-1bionic ceph-test=15.2.6-607-g3b1861ac-1bionic radosgw=15.2.6-607-g3b1861ac-1bionic python3-rados=15.2.6-607-g3b1861ac-1bionic python3-rgw=15.2.6-607-g3b1861ac-1bionic python3-cephfs=15.2.6-607-g3b1861ac-1bionic python3-rbd=15.2.6-607-g3b1861ac-1bionic libcephfs2=15.2.6-607-g3b1861ac-1bionic libcephfs-dev=15.2.6-607-g3b1861ac-1bionic librados2=15.2.6-607-g3b1861ac-1bionic librbd1=15.2.6-607-g3b1861ac-1bionic rbd-fuse=15.2.6-607-g3b1861ac-1bionic python3-cephfs=15.2.6-607-g3b1861ac-1bionic cephfs-shell=15.2.6-607-g3b1861ac-1bionic 2020-12-05T17:28:21.063 INFO:teuthology.orchestra.run.smithi002.stdout:Reading package lists... 2020-12-05T17:28:21.064 INFO:teuthology.orchestra.run.smithi053.stdout:Reading package lists... 2020-12-05T17:28:21.096 INFO:teuthology.orchestra.run.smithi036.stdout:Building dependency tree... 2020-12-05T17:28:21.097 INFO:teuthology.orchestra.run.smithi036.stdout:Reading state information... 2020-12-05T17:28:21.146 INFO:teuthology.orchestra.run.smithi032.stdout:The following additional packages will be installed: 2020-12-05T17:28:21.147 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-12-05T17:28:21.147 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-12-05T17:28:21.149 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook ceph-mon ceph-osd jq libjq1 libleveldb1v5 liblttng-ust0 2020-12-05T17:28:21.149 INFO:teuthology.orchestra.run.smithi032.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2020-12-05T17:28:21.149 INFO:teuthology.orchestra.run.smithi032.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2020-12-05T17:28:21.149 INFO:teuthology.orchestra.run.smithi032.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2020-12-05T17:28:21.150 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-12-05T17:28:21.150 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2020-12-05T17:28:21.150 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2020-12-05T17:28:21.150 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2020-12-05T17:28:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2020-12-05T17:28:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2020-12-05T17:28:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2020-12-05T17:28:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2020-12-05T17:28:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2020-12-05T17:28:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2020-12-05T17:28:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2020-12-05T17:28:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2020-12-05T17:28:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2020-12-05T17:28:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2020-12-05T17:28:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2020-12-05T17:28:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2020-12-05T17:28:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2020-12-05T17:28:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2020-12-05T17:28:21.154 INFO:teuthology.orchestra.run.smithi032.stdout:Suggested packages: 2020-12-05T17:28:21.154 INFO:teuthology.orchestra.run.smithi032.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2020-12-05T17:28:21.154 INFO:teuthology.orchestra.run.smithi032.stdout: python-pytest-xdist python-mock python-cryptography-doc 2020-12-05T17:28:21.154 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2020-12-05T17:28:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2020-12-05T17:28:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2020-12-05T17:28:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2020-12-05T17:28:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2020-12-05T17:28:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2020-12-05T17:28:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2020-12-05T17:28:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2020-12-05T17:28:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2020-12-05T17:28:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2020-12-05T17:28:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2020-12-05T17:28:21.157 INFO:teuthology.orchestra.run.smithi032.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2020-12-05T17:28:21.157 INFO:teuthology.orchestra.run.smithi032.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2020-12-05T17:28:21.173 INFO:teuthology.orchestra.run.smithi061.stdout:Building dependency tree... 2020-12-05T17:28:21.174 INFO:teuthology.orchestra.run.smithi061.stdout:Reading state information... 2020-12-05T17:28:21.259 INFO:teuthology.orchestra.run.smithi032.stdout:The following NEW packages will be installed: 2020-12-05T17:28:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2020-12-05T17:28:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-12-05T17:28:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-12-05T17:28:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq 2020-12-05T17:28:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 liboath0 2020-12-05T17:28:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2020-12-05T17:28:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: libxslt1.1 nvme-cli python-attr python-funcsigs python-pastedeploy-tpl 2020-12-05T17:28:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: python-pluggy python-py python-pytest python3-asn1crypto python3-attr 2020-12-05T17:28:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: python3-bcrypt python3-bs4 python3-ceph-argparse python3-ceph-common 2020-12-05T17:28:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs python3-cffi-backend python3-cherrypy3 python3-cmd2 2020-12-05T17:28:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: python3-colorama python3-cryptography python3-cycler python3-dateutil 2020-12-05T17:28:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2020-12-05T17:28:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2020-12-05T17:28:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2020-12-05T17:28:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2020-12-05T17:28:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2020-12-05T17:28:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2020-12-05T17:28:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2020-12-05T17:28:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2020-12-05T17:28:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2020-12-05T17:28:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2020-12-05T17:28:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2020-12-05T17:28:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2020-12-05T17:28:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2020-12-05T17:28:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2020-12-05T17:28:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2020-12-05T17:28:21.269 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages will be upgraded: 2020-12-05T17:28:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: librados2 librbd1 2020-12-05T17:28:21.276 INFO:teuthology.orchestra.run.smithi036.stdout:The following additional packages will be installed: 2020-12-05T17:28:21.276 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-12-05T17:28:21.277 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-12-05T17:28:21.279 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mgr-rook ceph-mon ceph-osd jq libjq1 libleveldb1v5 liblttng-ust0 2020-12-05T17:28:21.280 INFO:teuthology.orchestra.run.smithi036.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2020-12-05T17:28:21.280 INFO:teuthology.orchestra.run.smithi036.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2020-12-05T17:28:21.280 INFO:teuthology.orchestra.run.smithi036.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2020-12-05T17:28:21.280 INFO:teuthology.orchestra.run.smithi036.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-12-05T17:28:21.281 INFO:teuthology.orchestra.run.smithi036.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2020-12-05T17:28:21.281 INFO:teuthology.orchestra.run.smithi036.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2020-12-05T17:28:21.281 INFO:teuthology.orchestra.run.smithi036.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2020-12-05T17:28:21.281 INFO:teuthology.orchestra.run.smithi036.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2020-12-05T17:28:21.282 INFO:teuthology.orchestra.run.smithi036.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2020-12-05T17:28:21.282 INFO:teuthology.orchestra.run.smithi036.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2020-12-05T17:28:21.282 INFO:teuthology.orchestra.run.smithi036.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2020-12-05T17:28:21.282 INFO:teuthology.orchestra.run.smithi036.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2020-12-05T17:28:21.282 INFO:teuthology.orchestra.run.smithi036.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2020-12-05T17:28:21.283 INFO:teuthology.orchestra.run.smithi036.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2020-12-05T17:28:21.283 INFO:teuthology.orchestra.run.smithi036.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2020-12-05T17:28:21.283 INFO:teuthology.orchestra.run.smithi036.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2020-12-05T17:28:21.283 INFO:teuthology.orchestra.run.smithi036.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2020-12-05T17:28:21.283 INFO:teuthology.orchestra.run.smithi036.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2020-12-05T17:28:21.284 INFO:teuthology.orchestra.run.smithi036.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2020-12-05T17:28:21.284 INFO:teuthology.orchestra.run.smithi036.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2020-12-05T17:28:21.284 INFO:teuthology.orchestra.run.smithi036.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2020-12-05T17:28:21.284 INFO:teuthology.orchestra.run.smithi036.stdout:Suggested packages: 2020-12-05T17:28:21.284 INFO:teuthology.orchestra.run.smithi036.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2020-12-05T17:28:21.285 INFO:teuthology.orchestra.run.smithi036.stdout: python-pytest-xdist python-mock python-cryptography-doc 2020-12-05T17:28:21.285 INFO:teuthology.orchestra.run.smithi036.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2020-12-05T17:28:21.285 INFO:teuthology.orchestra.run.smithi036.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2020-12-05T17:28:21.285 INFO:teuthology.orchestra.run.smithi036.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2020-12-05T17:28:21.285 INFO:teuthology.orchestra.run.smithi036.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2020-12-05T17:28:21.286 INFO:teuthology.orchestra.run.smithi036.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2020-12-05T17:28:21.286 INFO:teuthology.orchestra.run.smithi036.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2020-12-05T17:28:21.286 INFO:teuthology.orchestra.run.smithi036.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2020-12-05T17:28:21.286 INFO:teuthology.orchestra.run.smithi036.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2020-12-05T17:28:21.286 INFO:teuthology.orchestra.run.smithi036.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2020-12-05T17:28:21.287 INFO:teuthology.orchestra.run.smithi036.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2020-12-05T17:28:21.287 INFO:teuthology.orchestra.run.smithi036.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2020-12-05T17:28:21.287 INFO:teuthology.orchestra.run.smithi036.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2020-12-05T17:28:21.287 INFO:teuthology.orchestra.run.smithi036.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2020-12-05T17:28:21.312 INFO:teuthology.orchestra.run.smithi002.stdout:Building dependency tree... 2020-12-05T17:28:21.313 INFO:teuthology.orchestra.run.smithi002.stdout:Reading state information... 2020-12-05T17:28:21.332 INFO:teuthology.orchestra.run.smithi053.stdout:Building dependency tree... 2020-12-05T17:28:21.332 INFO:teuthology.orchestra.run.smithi053.stdout:Reading state information... 2020-12-05T17:28:21.356 INFO:teuthology.orchestra.run.smithi061.stdout:The following additional packages will be installed: 2020-12-05T17:28:21.356 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-12-05T17:28:21.356 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-12-05T17:28:21.358 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-rook ceph-mon ceph-osd jq libjq1 libleveldb1v5 liblttng-ust0 2020-12-05T17:28:21.358 INFO:teuthology.orchestra.run.smithi061.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2020-12-05T17:28:21.359 INFO:teuthology.orchestra.run.smithi061.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2020-12-05T17:28:21.359 INFO:teuthology.orchestra.run.smithi061.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2020-12-05T17:28:21.359 INFO:teuthology.orchestra.run.smithi061.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-12-05T17:28:21.360 INFO:teuthology.orchestra.run.smithi061.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2020-12-05T17:28:21.360 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2020-12-05T17:28:21.360 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2020-12-05T17:28:21.360 INFO:teuthology.orchestra.run.smithi061.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2020-12-05T17:28:21.362 INFO:teuthology.orchestra.run.smithi061.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2020-12-05T17:28:21.362 INFO:teuthology.orchestra.run.smithi061.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2020-12-05T17:28:21.362 INFO:teuthology.orchestra.run.smithi061.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2020-12-05T17:28:21.362 INFO:teuthology.orchestra.run.smithi061.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2020-12-05T17:28:21.363 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2020-12-05T17:28:21.363 INFO:teuthology.orchestra.run.smithi061.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2020-12-05T17:28:21.363 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2020-12-05T17:28:21.363 INFO:teuthology.orchestra.run.smithi061.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2020-12-05T17:28:21.363 INFO:teuthology.orchestra.run.smithi061.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2020-12-05T17:28:21.364 INFO:teuthology.orchestra.run.smithi061.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2020-12-05T17:28:21.364 INFO:teuthology.orchestra.run.smithi061.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2020-12-05T17:28:21.364 INFO:teuthology.orchestra.run.smithi061.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2020-12-05T17:28:21.364 INFO:teuthology.orchestra.run.smithi061.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2020-12-05T17:28:21.365 INFO:teuthology.orchestra.run.smithi061.stdout:Suggested packages: 2020-12-05T17:28:21.365 INFO:teuthology.orchestra.run.smithi061.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2020-12-05T17:28:21.365 INFO:teuthology.orchestra.run.smithi061.stdout: python-pytest-xdist python-mock python-cryptography-doc 2020-12-05T17:28:21.365 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2020-12-05T17:28:21.366 INFO:teuthology.orchestra.run.smithi061.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2020-12-05T17:28:21.366 INFO:teuthology.orchestra.run.smithi061.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2020-12-05T17:28:21.366 INFO:teuthology.orchestra.run.smithi061.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2020-12-05T17:28:21.366 INFO:teuthology.orchestra.run.smithi061.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2020-12-05T17:28:21.367 INFO:teuthology.orchestra.run.smithi061.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2020-12-05T17:28:21.367 INFO:teuthology.orchestra.run.smithi061.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2020-12-05T17:28:21.367 INFO:teuthology.orchestra.run.smithi061.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2020-12-05T17:28:21.367 INFO:teuthology.orchestra.run.smithi061.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2020-12-05T17:28:21.367 INFO:teuthology.orchestra.run.smithi061.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2020-12-05T17:28:21.368 INFO:teuthology.orchestra.run.smithi061.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2020-12-05T17:28:21.368 INFO:teuthology.orchestra.run.smithi061.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2020-12-05T17:28:21.368 INFO:teuthology.orchestra.run.smithi061.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2020-12-05T17:28:21.387 INFO:teuthology.orchestra.run.smithi036.stdout:The following NEW packages will be installed: 2020-12-05T17:28:21.387 INFO:teuthology.orchestra.run.smithi036.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2020-12-05T17:28:21.387 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-12-05T17:28:21.387 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-12-05T17:28:21.388 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq 2020-12-05T17:28:21.389 INFO:teuthology.orchestra.run.smithi036.stdout: libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 liboath0 2020-12-05T17:28:21.389 INFO:teuthology.orchestra.run.smithi036.stdout: libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2020-12-05T17:28:21.389 INFO:teuthology.orchestra.run.smithi036.stdout: libxslt1.1 nvme-cli python-attr python-funcsigs python-pastedeploy-tpl 2020-12-05T17:28:21.389 INFO:teuthology.orchestra.run.smithi036.stdout: python-pluggy python-py python-pytest python3-asn1crypto python3-attr 2020-12-05T17:28:21.389 INFO:teuthology.orchestra.run.smithi036.stdout: python3-bcrypt python3-bs4 python3-ceph-argparse python3-ceph-common 2020-12-05T17:28:21.390 INFO:teuthology.orchestra.run.smithi036.stdout: python3-cephfs python3-cffi-backend python3-cherrypy3 python3-cmd2 2020-12-05T17:28:21.390 INFO:teuthology.orchestra.run.smithi036.stdout: python3-colorama python3-cryptography python3-cycler python3-dateutil 2020-12-05T17:28:21.390 INFO:teuthology.orchestra.run.smithi036.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2020-12-05T17:28:21.390 INFO:teuthology.orchestra.run.smithi036.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2020-12-05T17:28:21.391 INFO:teuthology.orchestra.run.smithi036.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2020-12-05T17:28:21.391 INFO:teuthology.orchestra.run.smithi036.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2020-12-05T17:28:21.391 INFO:teuthology.orchestra.run.smithi036.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2020-12-05T17:28:21.391 INFO:teuthology.orchestra.run.smithi036.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2020-12-05T17:28:21.391 INFO:teuthology.orchestra.run.smithi036.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2020-12-05T17:28:21.392 INFO:teuthology.orchestra.run.smithi036.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2020-12-05T17:28:21.392 INFO:teuthology.orchestra.run.smithi036.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2020-12-05T17:28:21.392 INFO:teuthology.orchestra.run.smithi036.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2020-12-05T17:28:21.392 INFO:teuthology.orchestra.run.smithi036.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2020-12-05T17:28:21.393 INFO:teuthology.orchestra.run.smithi036.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2020-12-05T17:28:21.393 INFO:teuthology.orchestra.run.smithi036.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2020-12-05T17:28:21.393 INFO:teuthology.orchestra.run.smithi036.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2020-12-05T17:28:21.393 INFO:teuthology.orchestra.run.smithi036.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2020-12-05T17:28:21.394 INFO:teuthology.orchestra.run.smithi036.stdout:The following packages will be upgraded: 2020-12-05T17:28:21.394 INFO:teuthology.orchestra.run.smithi036.stdout: librados2 librbd1 2020-12-05T17:28:21.425 INFO:teuthology.orchestra.run.smithi032.stdout:2 upgraded, 115 newly installed, 0 to remove and 3 not upgraded. 2020-12-05T17:28:21.425 INFO:teuthology.orchestra.run.smithi032.stdout:Need to get 156 MB of archives. 2020-12-05T17:28:21.426 INFO:teuthology.orchestra.run.smithi032.stdout:After this operation, 724 MB of additional disk space will be used. 2020-12-05T17:28:21.426 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 15.2.6-607-g3b1861ac-1bionic [1,634 kB] 2020-12-05T17:28:21.453 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2020-12-05T17:28:21.467 INFO:teuthology.orchestra.run.smithi061.stdout:The following NEW packages will be installed: 2020-12-05T17:28:21.467 INFO:teuthology.orchestra.run.smithi061.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2020-12-05T17:28:21.467 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-12-05T17:28:21.468 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-12-05T17:28:21.469 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq 2020-12-05T17:28:21.469 INFO:teuthology.orchestra.run.smithi061.stdout: libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 liboath0 2020-12-05T17:28:21.469 INFO:teuthology.orchestra.run.smithi061.stdout: libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2020-12-05T17:28:21.470 INFO:teuthology.orchestra.run.smithi061.stdout: libxslt1.1 nvme-cli python-attr python-funcsigs python-pastedeploy-tpl 2020-12-05T17:28:21.470 INFO:teuthology.orchestra.run.smithi061.stdout: python-pluggy python-py python-pytest python3-asn1crypto python3-attr 2020-12-05T17:28:21.470 INFO:teuthology.orchestra.run.smithi061.stdout: python3-bcrypt python3-bs4 python3-ceph-argparse python3-ceph-common 2020-12-05T17:28:21.470 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cephfs python3-cffi-backend python3-cherrypy3 python3-cmd2 2020-12-05T17:28:21.471 INFO:teuthology.orchestra.run.smithi061.stdout: python3-colorama python3-cryptography python3-cycler python3-dateutil 2020-12-05T17:28:21.471 INFO:teuthology.orchestra.run.smithi061.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2020-12-05T17:28:21.471 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2020-12-05T17:28:21.471 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2020-12-05T17:28:21.472 INFO:teuthology.orchestra.run.smithi061.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2020-12-05T17:28:21.472 INFO:teuthology.orchestra.run.smithi061.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2020-12-05T17:28:21.472 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2020-12-05T17:28:21.472 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2020-12-05T17:28:21.472 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2020-12-05T17:28:21.473 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2020-12-05T17:28:21.473 INFO:teuthology.orchestra.run.smithi061.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2020-12-05T17:28:21.473 INFO:teuthology.orchestra.run.smithi061.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2020-12-05T17:28:21.473 INFO:teuthology.orchestra.run.smithi061.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2020-12-05T17:28:21.473 INFO:teuthology.orchestra.run.smithi061.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2020-12-05T17:28:21.474 INFO:teuthology.orchestra.run.smithi061.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2020-12-05T17:28:21.474 INFO:teuthology.orchestra.run.smithi061.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2020-12-05T17:28:21.474 INFO:teuthology.orchestra.run.smithi061.stdout:The following packages will be upgraded: 2020-12-05T17:28:21.475 INFO:teuthology.orchestra.run.smithi061.stdout: librados2 librbd1 2020-12-05T17:28:21.491 INFO:teuthology.orchestra.run.smithi002.stdout:The following additional packages will be installed: 2020-12-05T17:28:21.491 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-12-05T17:28:21.491 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-12-05T17:28:21.492 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook ceph-mon ceph-osd jq libjq1 libleveldb1v5 liblttng-ust0 2020-12-05T17:28:21.493 INFO:teuthology.orchestra.run.smithi002.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2020-12-05T17:28:21.493 INFO:teuthology.orchestra.run.smithi002.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2020-12-05T17:28:21.493 INFO:teuthology.orchestra.run.smithi002.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2020-12-05T17:28:21.493 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-12-05T17:28:21.494 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2020-12-05T17:28:21.494 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2020-12-05T17:28:21.494 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2020-12-05T17:28:21.494 INFO:teuthology.orchestra.run.smithi002.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2020-12-05T17:28:21.495 INFO:teuthology.orchestra.run.smithi002.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2020-12-05T17:28:21.495 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2020-12-05T17:28:21.495 INFO:teuthology.orchestra.run.smithi002.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2020-12-05T17:28:21.495 INFO:teuthology.orchestra.run.smithi002.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2020-12-05T17:28:21.496 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2020-12-05T17:28:21.496 INFO:teuthology.orchestra.run.smithi002.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2020-12-05T17:28:21.496 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2020-12-05T17:28:21.496 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2020-12-05T17:28:21.496 INFO:teuthology.orchestra.run.smithi002.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2020-12-05T17:28:21.497 INFO:teuthology.orchestra.run.smithi002.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2020-12-05T17:28:21.497 INFO:teuthology.orchestra.run.smithi002.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2020-12-05T17:28:21.497 INFO:teuthology.orchestra.run.smithi002.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2020-12-05T17:28:21.497 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2020-12-05T17:28:21.498 INFO:teuthology.orchestra.run.smithi002.stdout:Suggested packages: 2020-12-05T17:28:21.498 INFO:teuthology.orchestra.run.smithi002.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2020-12-05T17:28:21.498 INFO:teuthology.orchestra.run.smithi002.stdout: python-pytest-xdist python-mock python-cryptography-doc 2020-12-05T17:28:21.499 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2020-12-05T17:28:21.499 INFO:teuthology.orchestra.run.smithi002.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2020-12-05T17:28:21.499 INFO:teuthology.orchestra.run.smithi002.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2020-12-05T17:28:21.499 INFO:teuthology.orchestra.run.smithi002.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2020-12-05T17:28:21.499 INFO:teuthology.orchestra.run.smithi002.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2020-12-05T17:28:21.500 INFO:teuthology.orchestra.run.smithi002.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2020-12-05T17:28:21.500 INFO:teuthology.orchestra.run.smithi002.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2020-12-05T17:28:21.500 INFO:teuthology.orchestra.run.smithi002.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2020-12-05T17:28:21.500 INFO:teuthology.orchestra.run.smithi002.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2020-12-05T17:28:21.500 INFO:teuthology.orchestra.run.smithi002.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2020-12-05T17:28:21.501 INFO:teuthology.orchestra.run.smithi002.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2020-12-05T17:28:21.501 INFO:teuthology.orchestra.run.smithi002.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2020-12-05T17:28:21.501 INFO:teuthology.orchestra.run.smithi002.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2020-12-05T17:28:21.513 INFO:teuthology.orchestra.run.smithi053.stdout:The following additional packages will be installed: 2020-12-05T17:28:21.513 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-12-05T17:28:21.513 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-12-05T17:28:21.514 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook ceph-mon ceph-osd jq libjq1 libleveldb1v5 liblttng-ust0 2020-12-05T17:28:21.515 INFO:teuthology.orchestra.run.smithi053.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2020-12-05T17:28:21.515 INFO:teuthology.orchestra.run.smithi053.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2020-12-05T17:28:21.515 INFO:teuthology.orchestra.run.smithi053.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2020-12-05T17:28:21.515 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-12-05T17:28:21.516 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2020-12-05T17:28:21.516 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2020-12-05T17:28:21.516 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2020-12-05T17:28:21.516 INFO:teuthology.orchestra.run.smithi053.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2020-12-05T17:28:21.516 INFO:teuthology.orchestra.run.smithi053.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2020-12-05T17:28:21.517 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2020-12-05T17:28:21.517 INFO:teuthology.orchestra.run.smithi053.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2020-12-05T17:28:21.517 INFO:teuthology.orchestra.run.smithi053.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2020-12-05T17:28:21.517 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2020-12-05T17:28:21.518 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2020-12-05T17:28:21.518 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2020-12-05T17:28:21.518 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2020-12-05T17:28:21.518 INFO:teuthology.orchestra.run.smithi053.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2020-12-05T17:28:21.519 INFO:teuthology.orchestra.run.smithi053.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2020-12-05T17:28:21.519 INFO:teuthology.orchestra.run.smithi053.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2020-12-05T17:28:21.519 INFO:teuthology.orchestra.run.smithi053.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2020-12-05T17:28:21.519 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2020-12-05T17:28:21.520 INFO:teuthology.orchestra.run.smithi053.stdout:Suggested packages: 2020-12-05T17:28:21.520 INFO:teuthology.orchestra.run.smithi053.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2020-12-05T17:28:21.521 INFO:teuthology.orchestra.run.smithi053.stdout: python-pytest-xdist python-mock python-cryptography-doc 2020-12-05T17:28:21.521 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2020-12-05T17:28:21.521 INFO:teuthology.orchestra.run.smithi053.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2020-12-05T17:28:21.521 INFO:teuthology.orchestra.run.smithi053.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2020-12-05T17:28:21.521 INFO:teuthology.orchestra.run.smithi053.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2020-12-05T17:28:21.522 INFO:teuthology.orchestra.run.smithi053.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2020-12-05T17:28:21.522 INFO:teuthology.orchestra.run.smithi053.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2020-12-05T17:28:21.522 INFO:teuthology.orchestra.run.smithi053.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2020-12-05T17:28:21.522 INFO:teuthology.orchestra.run.smithi053.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2020-12-05T17:28:21.523 INFO:teuthology.orchestra.run.smithi053.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2020-12-05T17:28:21.523 INFO:teuthology.orchestra.run.smithi053.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2020-12-05T17:28:21.523 INFO:teuthology.orchestra.run.smithi053.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2020-12-05T17:28:21.523 INFO:teuthology.orchestra.run.smithi053.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2020-12-05T17:28:21.524 INFO:teuthology.orchestra.run.smithi053.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2020-12-05T17:28:21.553 INFO:teuthology.orchestra.run.smithi036.stdout:2 upgraded, 115 newly installed, 0 to remove and 3 not upgraded. 2020-12-05T17:28:21.554 INFO:teuthology.orchestra.run.smithi036.stdout:Need to get 156 MB of archives. 2020-12-05T17:28:21.554 INFO:teuthology.orchestra.run.smithi036.stdout:After this operation, 724 MB of additional disk space will be used. 2020-12-05T17:28:21.554 INFO:teuthology.orchestra.run.smithi036.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 15.2.6-607-g3b1861ac-1bionic [1,634 kB] 2020-12-05T17:28:21.579 INFO:teuthology.orchestra.run.smithi036.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2020-12-05T17:28:21.585 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 15.2.6-607-g3b1861ac-1bionic [3,059 kB] 2020-12-05T17:28:21.600 INFO:teuthology.orchestra.run.smithi002.stdout:The following NEW packages will be installed: 2020-12-05T17:28:21.600 INFO:teuthology.orchestra.run.smithi002.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2020-12-05T17:28:21.600 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-12-05T17:28:21.601 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-12-05T17:28:21.602 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq 2020-12-05T17:28:21.602 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 liboath0 2020-12-05T17:28:21.602 INFO:teuthology.orchestra.run.smithi002.stdout: libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2020-12-05T17:28:21.603 INFO:teuthology.orchestra.run.smithi002.stdout: libxslt1.1 nvme-cli python-attr python-funcsigs python-pastedeploy-tpl 2020-12-05T17:28:21.603 INFO:teuthology.orchestra.run.smithi002.stdout: python-pluggy python-py python-pytest python3-asn1crypto python3-attr 2020-12-05T17:28:21.603 INFO:teuthology.orchestra.run.smithi002.stdout: python3-bcrypt python3-bs4 python3-ceph-argparse python3-ceph-common 2020-12-05T17:28:21.603 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cephfs python3-cffi-backend python3-cherrypy3 python3-cmd2 2020-12-05T17:28:21.604 INFO:teuthology.orchestra.run.smithi002.stdout: python3-colorama python3-cryptography python3-cycler python3-dateutil 2020-12-05T17:28:21.604 INFO:teuthology.orchestra.run.smithi002.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2020-12-05T17:28:21.604 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2020-12-05T17:28:21.604 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2020-12-05T17:28:21.605 INFO:teuthology.orchestra.run.smithi002.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2020-12-05T17:28:21.605 INFO:teuthology.orchestra.run.smithi002.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2020-12-05T17:28:21.605 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2020-12-05T17:28:21.605 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2020-12-05T17:28:21.606 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2020-12-05T17:28:21.606 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2020-12-05T17:28:21.606 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2020-12-05T17:28:21.606 INFO:teuthology.orchestra.run.smithi002.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2020-12-05T17:28:21.607 INFO:teuthology.orchestra.run.smithi002.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2020-12-05T17:28:21.607 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2020-12-05T17:28:21.607 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2020-12-05T17:28:21.607 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2020-12-05T17:28:21.608 INFO:teuthology.orchestra.run.smithi002.stdout:The following packages will be upgraded: 2020-12-05T17:28:21.608 INFO:teuthology.orchestra.run.smithi002.stdout: librados2 librbd1 2020-12-05T17:28:21.628 INFO:teuthology.orchestra.run.smithi053.stdout:The following NEW packages will be installed: 2020-12-05T17:28:21.629 INFO:teuthology.orchestra.run.smithi053.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2020-12-05T17:28:21.629 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-12-05T17:28:21.629 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-12-05T17:28:21.631 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq 2020-12-05T17:28:21.632 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 liboath0 2020-12-05T17:28:21.632 INFO:teuthology.orchestra.run.smithi053.stdout: libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2020-12-05T17:28:21.632 INFO:teuthology.orchestra.run.smithi053.stdout: libxslt1.1 nvme-cli python-attr python-funcsigs python-pastedeploy-tpl 2020-12-05T17:28:21.633 INFO:teuthology.orchestra.run.smithi053.stdout: python-pluggy python-py python-pytest python3-asn1crypto python3-attr 2020-12-05T17:28:21.633 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt python3-bs4 python3-ceph-argparse python3-ceph-common 2020-12-05T17:28:21.633 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs python3-cffi-backend python3-cherrypy3 python3-cmd2 2020-12-05T17:28:21.633 INFO:teuthology.orchestra.run.smithi053.stdout: python3-colorama python3-cryptography python3-cycler python3-dateutil 2020-12-05T17:28:21.634 INFO:teuthology.orchestra.run.smithi053.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2020-12-05T17:28:21.634 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2020-12-05T17:28:21.634 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2020-12-05T17:28:21.635 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2020-12-05T17:28:21.635 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2020-12-05T17:28:21.635 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2020-12-05T17:28:21.635 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2020-12-05T17:28:21.635 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2020-12-05T17:28:21.636 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2020-12-05T17:28:21.636 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2020-12-05T17:28:21.636 INFO:teuthology.orchestra.run.smithi053.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2020-12-05T17:28:21.637 INFO:teuthology.orchestra.run.smithi053.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2020-12-05T17:28:21.637 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2020-12-05T17:28:21.637 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2020-12-05T17:28:21.637 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2020-12-05T17:28:21.638 INFO:teuthology.orchestra.run.smithi053.stdout:The following packages will be upgraded: 2020-12-05T17:28:21.638 INFO:teuthology.orchestra.run.smithi053.stdout: librados2 librbd1 2020-12-05T17:28:21.663 INFO:teuthology.orchestra.run.smithi061.stdout:2 upgraded, 115 newly installed, 0 to remove and 3 not upgraded. 2020-12-05T17:28:21.663 INFO:teuthology.orchestra.run.smithi061.stdout:Need to get 156 MB of archives. 2020-12-05T17:28:21.663 INFO:teuthology.orchestra.run.smithi061.stdout:After this operation, 724 MB of additional disk space will be used. 2020-12-05T17:28:21.664 INFO:teuthology.orchestra.run.smithi061.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2020-12-05T17:28:21.673 INFO:teuthology.orchestra.run.smithi061.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 15.2.6-607-g3b1861ac-1bionic [1,634 kB] 2020-12-05T17:28:21.690 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 15.2.6-607-g3b1861ac-1bionic [435 kB] 2020-12-05T17:28:21.704 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2020-12-05T17:28:21.704 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 15.2.6-607-g3b1861ac-1bionic [332 kB] 2020-12-05T17:28:21.713 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2020-12-05T17:28:21.722 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 15.2.6-607-g3b1861ac-1bionic [20.7 kB] 2020-12-05T17:28:21.723 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 15.2.6-607-g3b1861ac-1bionic [133 kB] 2020-12-05T17:28:21.723 INFO:teuthology.orchestra.run.smithi032.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2020-12-05T17:28:21.724 INFO:teuthology.orchestra.run.smithi032.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-12-05T17:28:21.727 INFO:teuthology.orchestra.run.smithi032.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 15.2.6-607-g3b1861ac-1bionic [27.8 kB] 2020-12-05T17:28:21.728 INFO:teuthology.orchestra.run.smithi032.stdout:Get:13 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 15.2.6-607-g3b1861ac-1bionic [312 kB] 2020-12-05T17:28:21.738 INFO:teuthology.orchestra.run.smithi032.stdout:Get:14 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 15.2.6-607-g3b1861ac-1bionic [2,889 kB] 2020-12-05T17:28:21.794 INFO:teuthology.orchestra.run.smithi002.stdout:2 upgraded, 115 newly installed, 0 to remove and 3 not upgraded. 2020-12-05T17:28:21.795 INFO:teuthology.orchestra.run.smithi002.stdout:Need to get 156 MB of archives. 2020-12-05T17:28:21.795 INFO:teuthology.orchestra.run.smithi002.stdout:After this operation, 724 MB of additional disk space will be used. 2020-12-05T17:28:21.795 INFO:teuthology.orchestra.run.smithi002.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2020-12-05T17:28:21.819 INFO:teuthology.orchestra.run.smithi053.stdout:2 upgraded, 115 newly installed, 0 to remove and 3 not upgraded. 2020-12-05T17:28:21.819 INFO:teuthology.orchestra.run.smithi053.stdout:Need to get 156 MB of archives. 2020-12-05T17:28:21.819 INFO:teuthology.orchestra.run.smithi053.stdout:After this operation, 724 MB of additional disk space will be used. 2020-12-05T17:28:21.820 INFO:teuthology.orchestra.run.smithi053.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2020-12-05T17:28:21.835 INFO:teuthology.orchestra.run.smithi036.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2020-12-05T17:28:21.836 INFO:teuthology.orchestra.run.smithi036.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2020-12-05T17:28:21.839 INFO:teuthology.orchestra.run.smithi036.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2020-12-05T17:28:21.845 INFO:teuthology.orchestra.run.smithi036.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-12-05T17:28:21.862 INFO:teuthology.orchestra.run.smithi032.stdout:Get:15 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 15.2.6-607-g3b1861ac-1bionic [99.3 kB] 2020-12-05T17:28:21.863 INFO:teuthology.orchestra.run.smithi002.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 15.2.6-607-g3b1861ac-1bionic [1,634 kB] 2020-12-05T17:28:21.865 INFO:teuthology.orchestra.run.smithi032.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 15.2.6-607-g3b1861ac-1bionic [337 kB] 2020-12-05T17:28:21.876 INFO:teuthology.orchestra.run.smithi032.stdout:Get:17 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 15.2.6-607-g3b1861ac-1bionic [18.8 MB] 2020-12-05T17:28:21.887 INFO:teuthology.orchestra.run.smithi032.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2020-12-05T17:28:21.892 INFO:teuthology.orchestra.run.smithi032.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2020-12-05T17:28:21.899 INFO:teuthology.orchestra.run.smithi032.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2020-12-05T17:28:21.908 INFO:teuthology.orchestra.run.smithi053.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 15.2.6-607-g3b1861ac-1bionic [1,634 kB] 2020-12-05T17:28:21.914 INFO:teuthology.orchestra.run.smithi061.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2020-12-05T17:28:21.923 INFO:teuthology.orchestra.run.smithi061.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2020-12-05T17:28:21.948 INFO:teuthology.orchestra.run.smithi061.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2020-12-05T17:28:21.949 INFO:teuthology.orchestra.run.smithi061.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-12-05T17:28:21.984 INFO:teuthology.orchestra.run.smithi032.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-12-05T17:28:21.986 INFO:teuthology.orchestra.run.smithi032.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-12-05T17:28:22.003 INFO:teuthology.orchestra.run.smithi036.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2020-12-05T17:28:22.006 INFO:teuthology.orchestra.run.smithi036.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2020-12-05T17:28:22.015 INFO:teuthology.orchestra.run.smithi036.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2020-12-05T17:28:22.024 INFO:teuthology.orchestra.run.smithi032.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-12-05T17:28:22.031 INFO:teuthology.orchestra.run.smithi032.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2020-12-05T17:28:22.041 INFO:teuthology.orchestra.run.smithi032.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2020-12-05T17:28:22.048 INFO:teuthology.orchestra.run.smithi002.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2020-12-05T17:28:22.049 INFO:teuthology.orchestra.run.smithi032.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2020-12-05T17:28:22.054 INFO:teuthology.orchestra.run.smithi053.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2020-12-05T17:28:22.055 INFO:teuthology.orchestra.run.smithi032.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2020-12-05T17:28:22.057 INFO:teuthology.orchestra.run.smithi002.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2020-12-05T17:28:22.064 INFO:teuthology.orchestra.run.smithi053.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2020-12-05T17:28:22.068 INFO:teuthology.orchestra.run.smithi002.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2020-12-05T17:28:22.069 INFO:teuthology.orchestra.run.smithi002.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-12-05T17:28:22.069 INFO:teuthology.orchestra.run.smithi036.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 15.2.6-607-g3b1861ac-1bionic [3,059 kB] 2020-12-05T17:28:22.073 INFO:teuthology.orchestra.run.smithi053.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2020-12-05T17:28:22.074 INFO:teuthology.orchestra.run.smithi053.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-12-05T17:28:22.074 INFO:teuthology.orchestra.run.smithi032.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2020-12-05T17:28:22.078 INFO:teuthology.orchestra.run.smithi032.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2020-12-05T17:28:22.081 INFO:teuthology.orchestra.run.smithi061.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2020-12-05T17:28:22.092 INFO:teuthology.orchestra.run.smithi061.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2020-12-05T17:28:22.100 INFO:teuthology.orchestra.run.smithi036.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-12-05T17:28:22.101 INFO:teuthology.orchestra.run.smithi036.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-12-05T17:28:22.103 INFO:teuthology.orchestra.run.smithi061.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2020-12-05T17:28:22.120 INFO:teuthology.orchestra.run.smithi036.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-12-05T17:28:22.126 INFO:teuthology.orchestra.run.smithi036.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2020-12-05T17:28:22.128 INFO:teuthology.orchestra.run.smithi032.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2020-12-05T17:28:22.132 INFO:teuthology.orchestra.run.smithi036.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2020-12-05T17:28:22.136 INFO:teuthology.orchestra.run.smithi032.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2020-12-05T17:28:22.137 INFO:teuthology.orchestra.run.smithi032.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2020-12-05T17:28:22.138 INFO:teuthology.orchestra.run.smithi032.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2020-12-05T17:28:22.139 INFO:teuthology.orchestra.run.smithi032.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2020-12-05T17:28:22.140 INFO:teuthology.orchestra.run.smithi036.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2020-12-05T17:28:22.158 INFO:teuthology.orchestra.run.smithi036.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2020-12-05T17:28:22.180 INFO:teuthology.orchestra.run.smithi036.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2020-12-05T17:28:22.184 INFO:teuthology.orchestra.run.smithi036.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2020-12-05T17:28:22.185 INFO:teuthology.orchestra.run.smithi061.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-12-05T17:28:22.188 INFO:teuthology.orchestra.run.smithi061.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-12-05T17:28:22.207 INFO:teuthology.orchestra.run.smithi032.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2020-12-05T17:28:22.208 INFO:teuthology.orchestra.run.smithi032.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2020-12-05T17:28:22.209 INFO:teuthology.orchestra.run.smithi032.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-12-05T17:28:22.211 INFO:teuthology.orchestra.run.smithi032.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2020-12-05T17:28:22.212 INFO:teuthology.orchestra.run.smithi061.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-12-05T17:28:22.215 INFO:teuthology.orchestra.run.smithi032.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2020-12-05T17:28:22.215 INFO:teuthology.orchestra.run.smithi002.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2020-12-05T17:28:22.216 INFO:teuthology.orchestra.run.smithi053.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2020-12-05T17:28:22.220 INFO:teuthology.orchestra.run.smithi061.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2020-12-05T17:28:22.222 INFO:teuthology.orchestra.run.smithi053.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2020-12-05T17:28:22.225 INFO:teuthology.orchestra.run.smithi002.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2020-12-05T17:28:22.228 INFO:teuthology.orchestra.run.smithi061.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2020-12-05T17:28:22.236 INFO:teuthology.orchestra.run.smithi053.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2020-12-05T17:28:22.237 INFO:teuthology.orchestra.run.smithi002.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2020-12-05T17:28:22.239 INFO:teuthology.orchestra.run.smithi061.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2020-12-05T17:28:22.245 INFO:teuthology.orchestra.run.smithi061.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2020-12-05T17:28:22.246 INFO:teuthology.orchestra.run.smithi036.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2020-12-05T17:28:22.253 INFO:teuthology.orchestra.run.smithi036.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2020-12-05T17:28:22.254 INFO:teuthology.orchestra.run.smithi036.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2020-12-05T17:28:22.255 INFO:teuthology.orchestra.run.smithi036.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2020-12-05T17:28:22.255 INFO:teuthology.orchestra.run.smithi036.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2020-12-05T17:28:22.258 INFO:teuthology.orchestra.run.smithi036.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2020-12-05T17:28:22.258 INFO:teuthology.orchestra.run.smithi036.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2020-12-05T17:28:22.267 INFO:teuthology.orchestra.run.smithi061.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2020-12-05T17:28:22.270 INFO:teuthology.orchestra.run.smithi061.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2020-12-05T17:28:22.287 INFO:teuthology.orchestra.run.smithi032.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2020-12-05T17:28:22.288 INFO:teuthology.orchestra.run.smithi032.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-12-05T17:28:22.304 INFO:teuthology.orchestra.run.smithi032.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-12-05T17:28:22.305 INFO:teuthology.orchestra.run.smithi032.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2020-12-05T17:28:22.306 INFO:teuthology.orchestra.run.smithi032.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2020-12-05T17:28:22.325 INFO:teuthology.orchestra.run.smithi002.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-12-05T17:28:22.326 INFO:teuthology.orchestra.run.smithi036.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-12-05T17:28:22.327 INFO:teuthology.orchestra.run.smithi061.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2020-12-05T17:28:22.328 INFO:teuthology.orchestra.run.smithi002.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-12-05T17:28:22.328 INFO:teuthology.orchestra.run.smithi036.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2020-12-05T17:28:22.332 INFO:teuthology.orchestra.run.smithi036.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2020-12-05T17:28:22.336 INFO:teuthology.orchestra.run.smithi061.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2020-12-05T17:28:22.336 INFO:teuthology.orchestra.run.smithi061.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2020-12-05T17:28:22.338 INFO:teuthology.orchestra.run.smithi061.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2020-12-05T17:28:22.338 INFO:teuthology.orchestra.run.smithi061.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2020-12-05T17:28:22.340 INFO:teuthology.orchestra.run.smithi061.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2020-12-05T17:28:22.341 INFO:teuthology.orchestra.run.smithi061.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2020-12-05T17:28:22.341 INFO:teuthology.orchestra.run.smithi053.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-12-05T17:28:22.342 INFO:teuthology.orchestra.run.smithi053.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-12-05T17:28:22.354 INFO:teuthology.orchestra.run.smithi002.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-12-05T17:28:22.362 INFO:teuthology.orchestra.run.smithi002.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2020-12-05T17:28:22.364 INFO:teuthology.orchestra.run.smithi053.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-12-05T17:28:22.366 INFO:teuthology.orchestra.run.smithi032.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2020-12-05T17:28:22.370 INFO:teuthology.orchestra.run.smithi002.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2020-12-05T17:28:22.370 INFO:teuthology.orchestra.run.smithi053.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2020-12-05T17:28:22.371 INFO:teuthology.orchestra.run.smithi032.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.1 [174 kB] 2020-12-05T17:28:22.375 INFO:teuthology.orchestra.run.smithi053.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2020-12-05T17:28:22.377 INFO:teuthology.orchestra.run.smithi032.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2020-12-05T17:28:22.378 INFO:teuthology.orchestra.run.smithi002.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2020-12-05T17:28:22.380 INFO:teuthology.orchestra.run.smithi032.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2020-12-05T17:28:22.381 INFO:teuthology.orchestra.run.smithi032.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2020-12-05T17:28:22.383 INFO:teuthology.orchestra.run.smithi002.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2020-12-05T17:28:22.384 INFO:teuthology.orchestra.run.smithi053.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2020-12-05T17:28:22.394 INFO:teuthology.orchestra.run.smithi053.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2020-12-05T17:28:22.407 INFO:teuthology.orchestra.run.smithi036.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2020-12-05T17:28:22.408 INFO:teuthology.orchestra.run.smithi036.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-12-05T17:28:22.408 INFO:teuthology.orchestra.run.smithi061.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-12-05T17:28:22.409 INFO:teuthology.orchestra.run.smithi002.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2020-12-05T17:28:22.410 INFO:teuthology.orchestra.run.smithi061.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2020-12-05T17:28:22.413 INFO:teuthology.orchestra.run.smithi002.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2020-12-05T17:28:22.414 INFO:teuthology.orchestra.run.smithi061.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2020-12-05T17:28:22.422 INFO:teuthology.orchestra.run.smithi053.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2020-12-05T17:28:22.426 INFO:teuthology.orchestra.run.smithi053.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2020-12-05T17:28:22.428 INFO:teuthology.orchestra.run.smithi036.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-12-05T17:28:22.428 INFO:teuthology.orchestra.run.smithi036.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2020-12-05T17:28:22.428 INFO:teuthology.orchestra.run.smithi036.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2020-12-05T17:28:22.429 INFO:teuthology.orchestra.run.smithi036.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2020-12-05T17:28:22.433 INFO:teuthology.orchestra.run.smithi036.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.1 [174 kB] 2020-12-05T17:28:22.445 INFO:teuthology.orchestra.run.smithi032.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2020-12-05T17:28:22.469 INFO:teuthology.orchestra.run.smithi002.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2020-12-05T17:28:22.472 INFO:teuthology.orchestra.run.smithi053.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2020-12-05T17:28:22.477 INFO:teuthology.orchestra.run.smithi002.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2020-12-05T17:28:22.479 INFO:teuthology.orchestra.run.smithi002.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2020-12-05T17:28:22.479 INFO:teuthology.orchestra.run.smithi002.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2020-12-05T17:28:22.480 INFO:teuthology.orchestra.run.smithi002.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2020-12-05T17:28:22.481 INFO:teuthology.orchestra.run.smithi053.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2020-12-05T17:28:22.481 INFO:teuthology.orchestra.run.smithi053.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2020-12-05T17:28:22.482 INFO:teuthology.orchestra.run.smithi002.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2020-12-05T17:28:22.482 INFO:teuthology.orchestra.run.smithi053.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2020-12-05T17:28:22.483 INFO:teuthology.orchestra.run.smithi053.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2020-12-05T17:28:22.484 INFO:teuthology.orchestra.run.smithi002.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2020-12-05T17:28:22.485 INFO:teuthology.orchestra.run.smithi053.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2020-12-05T17:28:22.487 INFO:teuthology.orchestra.run.smithi036.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2020-12-05T17:28:22.489 INFO:teuthology.orchestra.run.smithi061.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2020-12-05T17:28:22.489 INFO:teuthology.orchestra.run.smithi061.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-12-05T17:28:22.491 INFO:teuthology.orchestra.run.smithi036.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2020-12-05T17:28:22.492 INFO:teuthology.orchestra.run.smithi036.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2020-12-05T17:28:22.509 INFO:teuthology.orchestra.run.smithi061.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-12-05T17:28:22.510 INFO:teuthology.orchestra.run.smithi061.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2020-12-05T17:28:22.510 INFO:teuthology.orchestra.run.smithi061.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2020-12-05T17:28:22.511 INFO:teuthology.orchestra.run.smithi061.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2020-12-05T17:28:22.515 INFO:teuthology.orchestra.run.smithi061.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.1 [174 kB] 2020-12-05T17:28:22.537 INFO:teuthology.orchestra.run.smithi032.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2020-12-05T17:28:22.537 INFO:teuthology.orchestra.run.smithi032.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2020-12-05T17:28:22.549 INFO:teuthology.orchestra.run.smithi002.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-12-05T17:28:22.550 INFO:teuthology.orchestra.run.smithi053.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2020-12-05T17:28:22.551 INFO:teuthology.orchestra.run.smithi053.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-12-05T17:28:22.551 INFO:teuthology.orchestra.run.smithi002.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2020-12-05T17:28:22.553 INFO:teuthology.orchestra.run.smithi053.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2020-12-05T17:28:22.554 INFO:teuthology.orchestra.run.smithi002.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2020-12-05T17:28:22.556 INFO:teuthology.orchestra.run.smithi053.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2020-12-05T17:28:22.568 INFO:teuthology.orchestra.run.smithi036.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2020-12-05T17:28:22.569 INFO:teuthology.orchestra.run.smithi061.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2020-12-05T17:28:22.573 INFO:teuthology.orchestra.run.smithi061.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2020-12-05T17:28:22.576 INFO:teuthology.orchestra.run.smithi061.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2020-12-05T17:28:22.584 INFO:teuthology.orchestra.run.smithi061.stdout:Get:39 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 15.2.6-607-g3b1861ac-1bionic [3,059 kB] 2020-12-05T17:28:22.604 INFO:teuthology.orchestra.run.smithi036.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2020-12-05T17:28:22.604 INFO:teuthology.orchestra.run.smithi036.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2020-12-05T17:28:22.629 INFO:teuthology.orchestra.run.smithi002.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2020-12-05T17:28:22.629 INFO:teuthology.orchestra.run.smithi053.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2020-12-05T17:28:22.630 INFO:teuthology.orchestra.run.smithi002.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-12-05T17:28:22.630 INFO:teuthology.orchestra.run.smithi053.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-12-05T17:28:22.649 INFO:teuthology.orchestra.run.smithi053.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-12-05T17:28:22.649 INFO:teuthology.orchestra.run.smithi053.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2020-12-05T17:28:22.651 INFO:teuthology.orchestra.run.smithi061.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2020-12-05T17:28:22.651 INFO:teuthology.orchestra.run.smithi002.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-12-05T17:28:22.651 INFO:teuthology.orchestra.run.smithi002.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2020-12-05T17:28:22.652 INFO:teuthology.orchestra.run.smithi053.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2020-12-05T17:28:22.652 INFO:teuthology.orchestra.run.smithi053.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2020-12-05T17:28:22.652 INFO:teuthology.orchestra.run.smithi002.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2020-12-05T17:28:22.653 INFO:teuthology.orchestra.run.smithi002.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2020-12-05T17:28:22.656 INFO:teuthology.orchestra.run.smithi002.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.1 [174 kB] 2020-12-05T17:28:22.707 INFO:teuthology.orchestra.run.smithi053.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.1 [174 kB] 2020-12-05T17:28:22.708 INFO:teuthology.orchestra.run.smithi002.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2020-12-05T17:28:22.713 INFO:teuthology.orchestra.run.smithi061.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2020-12-05T17:28:22.713 INFO:teuthology.orchestra.run.smithi002.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2020-12-05T17:28:22.714 INFO:teuthology.orchestra.run.smithi061.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2020-12-05T17:28:22.714 INFO:teuthology.orchestra.run.smithi002.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2020-12-05T17:28:22.715 INFO:teuthology.orchestra.run.smithi053.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2020-12-05T17:28:22.718 INFO:teuthology.orchestra.run.smithi053.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2020-12-05T17:28:22.719 INFO:teuthology.orchestra.run.smithi053.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2020-12-05T17:28:22.723 INFO:teuthology.orchestra.run.smithi032.stdout:Get:53 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 15.2.6-607-g3b1861ac-1bionic [5,170 kB] 2020-12-05T17:28:22.786 INFO:teuthology.orchestra.run.smithi053.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2020-12-05T17:28:22.788 INFO:teuthology.orchestra.run.smithi002.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2020-12-05T17:28:22.804 INFO:teuthology.orchestra.run.smithi032.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2020-12-05T17:28:22.855 INFO:teuthology.orchestra.run.smithi002.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2020-12-05T17:28:22.856 INFO:teuthology.orchestra.run.smithi002.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2020-12-05T17:28:22.866 INFO:teuthology.orchestra.run.smithi053.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2020-12-05T17:28:22.867 INFO:teuthology.orchestra.run.smithi053.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2020-12-05T17:28:22.868 INFO:teuthology.orchestra.run.smithi032.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2020-12-05T17:28:22.870 INFO:teuthology.orchestra.run.smithi032.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2020-12-05T17:28:22.883 INFO:teuthology.orchestra.run.smithi032.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2020-12-05T17:28:22.883 INFO:teuthology.orchestra.run.smithi032.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2020-12-05T17:28:22.884 INFO:teuthology.orchestra.run.smithi032.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2020-12-05T17:28:22.884 INFO:teuthology.orchestra.run.smithi032.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2020-12-05T17:28:22.885 INFO:teuthology.orchestra.run.smithi032.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2020-12-05T17:28:22.885 INFO:teuthology.orchestra.run.smithi032.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2020-12-05T17:28:22.886 INFO:teuthology.orchestra.run.smithi032.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2020-12-05T17:28:22.911 INFO:teuthology.orchestra.run.smithi036.stdout:Get:43 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 15.2.6-607-g3b1861ac-1bionic [435 kB] 2020-12-05T17:28:22.911 INFO:teuthology.orchestra.run.smithi036.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2020-12-05T17:28:22.917 INFO:teuthology.orchestra.run.smithi032.stdout:Get:64 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 15.2.6-607-g3b1861ac-1bionic [148 kB] 2020-12-05T17:28:22.921 INFO:teuthology.orchestra.run.smithi032.stdout:Get:65 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 15.2.6-607-g3b1861ac-1bionic [1,282 kB] 2020-12-05T17:28:22.933 INFO:teuthology.orchestra.run.smithi036.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2020-12-05T17:28:22.934 INFO:teuthology.orchestra.run.smithi036.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2020-12-05T17:28:22.963 INFO:teuthology.orchestra.run.smithi032.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2020-12-05T17:28:22.964 INFO:teuthology.orchestra.run.smithi032.stdout:Get:67 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 15.2.6-607-g3b1861ac-1bionic [5,942 kB] 2020-12-05T17:28:22.965 INFO:teuthology.orchestra.run.smithi032.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2020-12-05T17:28:22.970 INFO:teuthology.orchestra.run.smithi032.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2020-12-05T17:28:22.991 INFO:teuthology.orchestra.run.smithi036.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2020-12-05T17:28:22.991 INFO:teuthology.orchestra.run.smithi036.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2020-12-05T17:28:22.992 INFO:teuthology.orchestra.run.smithi036.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2020-12-05T17:28:22.992 INFO:teuthology.orchestra.run.smithi036.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2020-12-05T17:28:22.993 INFO:teuthology.orchestra.run.smithi036.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2020-12-05T17:28:22.994 INFO:teuthology.orchestra.run.smithi036.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2020-12-05T17:28:22.995 INFO:teuthology.orchestra.run.smithi036.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2020-12-05T17:28:23.010 INFO:teuthology.orchestra.run.smithi061.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2020-12-05T17:28:23.040 INFO:teuthology.orchestra.run.smithi036.stdout:Get:54 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 15.2.6-607-g3b1861ac-1bionic [332 kB] 2020-12-05T17:28:23.043 INFO:teuthology.orchestra.run.smithi032.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2020-12-05T17:28:23.044 INFO:teuthology.orchestra.run.smithi061.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2020-12-05T17:28:23.045 INFO:teuthology.orchestra.run.smithi002.stdout:Get:42 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 15.2.6-607-g3b1861ac-1bionic [3,059 kB] 2020-12-05T17:28:23.046 INFO:teuthology.orchestra.run.smithi032.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2020-12-05T17:28:23.047 INFO:teuthology.orchestra.run.smithi061.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2020-12-05T17:28:23.047 INFO:teuthology.orchestra.run.smithi032.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2020-12-05T17:28:23.048 INFO:teuthology.orchestra.run.smithi032.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-12-05T17:28:23.050 INFO:teuthology.orchestra.run.smithi032.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-12-05T17:28:23.052 INFO:teuthology.orchestra.run.smithi032.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-12-05T17:28:23.053 INFO:teuthology.orchestra.run.smithi032.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-12-05T17:28:23.075 INFO:teuthology.orchestra.run.smithi036.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2020-12-05T17:28:23.077 INFO:teuthology.orchestra.run.smithi036.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2020-12-05T17:28:23.084 INFO:teuthology.orchestra.run.smithi036.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2020-12-05T17:28:23.085 INFO:teuthology.orchestra.run.smithi036.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2020-12-05T17:28:23.088 INFO:teuthology.orchestra.run.smithi036.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2020-12-05T17:28:23.092 INFO:teuthology.orchestra.run.smithi061.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2020-12-05T17:28:23.093 INFO:teuthology.orchestra.run.smithi061.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2020-12-05T17:28:23.093 INFO:teuthology.orchestra.run.smithi061.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2020-12-05T17:28:23.095 INFO:teuthology.orchestra.run.smithi061.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2020-12-05T17:28:23.096 INFO:teuthology.orchestra.run.smithi061.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2020-12-05T17:28:23.097 INFO:teuthology.orchestra.run.smithi061.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2020-12-05T17:28:23.098 INFO:teuthology.orchestra.run.smithi061.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2020-12-05T17:28:23.122 INFO:teuthology.orchestra.run.smithi032.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-12-05T17:28:23.127 INFO:teuthology.orchestra.run.smithi032.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-12-05T17:28:23.136 INFO:teuthology.orchestra.run.smithi032.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2020-12-05T17:28:23.136 INFO:teuthology.orchestra.run.smithi002.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2020-12-05T17:28:23.141 INFO:teuthology.orchestra.run.smithi053.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2020-12-05T17:28:23.155 INFO:teuthology.orchestra.run.smithi036.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2020-12-05T17:28:23.156 INFO:teuthology.orchestra.run.smithi036.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-12-05T17:28:23.160 INFO:teuthology.orchestra.run.smithi036.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-12-05T17:28:23.163 INFO:teuthology.orchestra.run.smithi036.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-12-05T17:28:23.165 INFO:teuthology.orchestra.run.smithi036.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-12-05T17:28:23.172 INFO:teuthology.orchestra.run.smithi036.stdout:Get:65 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 15.2.6-607-g3b1861ac-1bionic [20.7 kB] 2020-12-05T17:28:23.172 INFO:teuthology.orchestra.run.smithi061.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2020-12-05T17:28:23.173 INFO:teuthology.orchestra.run.smithi036.stdout:Get:66 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 15.2.6-607-g3b1861ac-1bionic [133 kB] 2020-12-05T17:28:23.173 INFO:teuthology.orchestra.run.smithi002.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2020-12-05T17:28:23.174 INFO:teuthology.orchestra.run.smithi061.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2020-12-05T17:28:23.176 INFO:teuthology.orchestra.run.smithi002.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2020-12-05T17:28:23.177 INFO:teuthology.orchestra.run.smithi036.stdout:Get:67 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 15.2.6-607-g3b1861ac-1bionic [27.8 kB] 2020-12-05T17:28:23.185 INFO:teuthology.orchestra.run.smithi053.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2020-12-05T17:28:23.187 INFO:teuthology.orchestra.run.smithi053.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2020-12-05T17:28:23.202 INFO:teuthology.orchestra.run.smithi032.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2020-12-05T17:28:23.208 INFO:teuthology.orchestra.run.smithi032.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2020-12-05T17:28:23.209 INFO:teuthology.orchestra.run.smithi032.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2020-12-05T17:28:23.210 INFO:teuthology.orchestra.run.smithi032.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2020-12-05T17:28:23.210 INFO:teuthology.orchestra.run.smithi032.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2020-12-05T17:28:23.212 INFO:teuthology.orchestra.run.smithi032.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2020-12-05T17:28:23.213 INFO:teuthology.orchestra.run.smithi002.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2020-12-05T17:28:23.213 INFO:teuthology.orchestra.run.smithi002.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2020-12-05T17:28:23.214 INFO:teuthology.orchestra.run.smithi032.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2020-12-05T17:28:23.214 INFO:teuthology.orchestra.run.smithi002.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2020-12-05T17:28:23.215 INFO:teuthology.orchestra.run.smithi002.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2020-12-05T17:28:23.216 INFO:teuthology.orchestra.run.smithi002.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2020-12-05T17:28:23.217 INFO:teuthology.orchestra.run.smithi053.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2020-12-05T17:28:23.217 INFO:teuthology.orchestra.run.smithi053.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2020-12-05T17:28:23.218 INFO:teuthology.orchestra.run.smithi002.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2020-12-05T17:28:23.218 INFO:teuthology.orchestra.run.smithi053.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2020-12-05T17:28:23.219 INFO:teuthology.orchestra.run.smithi053.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2020-12-05T17:28:23.219 INFO:teuthology.orchestra.run.smithi002.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2020-12-05T17:28:23.220 INFO:teuthology.orchestra.run.smithi053.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2020-12-05T17:28:23.220 INFO:teuthology.orchestra.run.smithi053.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2020-12-05T17:28:23.221 INFO:teuthology.orchestra.run.smithi053.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2020-12-05T17:28:23.231 INFO:teuthology.orchestra.run.smithi032.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 15.2.6-607-g3b1861ac-1bionic [22.7 MB] 2020-12-05T17:28:23.236 INFO:teuthology.orchestra.run.smithi036.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-12-05T17:28:23.238 INFO:teuthology.orchestra.run.smithi036.stdout:Get:69 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 15.2.6-607-g3b1861ac-1bionic [312 kB] 2020-12-05T17:28:23.242 INFO:teuthology.orchestra.run.smithi036.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-12-05T17:28:23.249 INFO:teuthology.orchestra.run.smithi036.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2020-12-05T17:28:23.250 INFO:teuthology.orchestra.run.smithi036.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2020-12-05T17:28:23.252 INFO:teuthology.orchestra.run.smithi036.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2020-12-05T17:28:23.253 INFO:teuthology.orchestra.run.smithi061.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2020-12-05T17:28:23.255 INFO:teuthology.orchestra.run.smithi061.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2020-12-05T17:28:23.258 INFO:teuthology.orchestra.run.smithi061.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2020-12-05T17:28:23.259 INFO:teuthology.orchestra.run.smithi061.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2020-12-05T17:28:23.260 INFO:teuthology.orchestra.run.smithi061.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-12-05T17:28:23.262 INFO:teuthology.orchestra.run.smithi061.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-12-05T17:28:23.264 INFO:teuthology.orchestra.run.smithi061.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-12-05T17:28:23.265 INFO:teuthology.orchestra.run.smithi061.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-12-05T17:28:23.281 INFO:teuthology.orchestra.run.smithi032.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2020-12-05T17:28:23.282 INFO:teuthology.orchestra.run.smithi032.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2020-12-05T17:28:23.282 INFO:teuthology.orchestra.run.smithi032.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2020-12-05T17:28:23.297 INFO:teuthology.orchestra.run.smithi002.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2020-12-05T17:28:23.299 INFO:teuthology.orchestra.run.smithi053.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2020-12-05T17:28:23.299 INFO:teuthology.orchestra.run.smithi002.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2020-12-05T17:28:23.304 INFO:teuthology.orchestra.run.smithi002.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2020-12-05T17:28:23.305 INFO:teuthology.orchestra.run.smithi002.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2020-12-05T17:28:23.309 INFO:teuthology.orchestra.run.smithi002.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2020-12-05T17:28:23.309 INFO:teuthology.orchestra.run.smithi002.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2020-12-05T17:28:23.310 INFO:teuthology.orchestra.run.smithi053.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2020-12-05T17:28:23.311 INFO:teuthology.orchestra.run.smithi036.stdout:Get:74 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 15.2.6-607-g3b1861ac-1bionic [2,889 kB] 2020-12-05T17:28:23.313 INFO:teuthology.orchestra.run.smithi053.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2020-12-05T17:28:23.313 INFO:teuthology.orchestra.run.smithi053.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2020-12-05T17:28:23.315 INFO:teuthology.orchestra.run.smithi053.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2020-12-05T17:28:23.316 INFO:teuthology.orchestra.run.smithi053.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2020-12-05T17:28:23.316 INFO:teuthology.orchestra.run.smithi036.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2020-12-05T17:28:23.317 INFO:teuthology.orchestra.run.smithi036.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2020-12-05T17:28:23.318 INFO:teuthology.orchestra.run.smithi036.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2020-12-05T17:28:23.320 INFO:teuthology.orchestra.run.smithi036.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2020-12-05T17:28:23.324 INFO:teuthology.orchestra.run.smithi036.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2020-12-05T17:28:23.333 INFO:teuthology.orchestra.run.smithi061.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-12-05T17:28:23.339 INFO:teuthology.orchestra.run.smithi061.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-12-05T17:28:23.360 INFO:teuthology.orchestra.run.smithi032.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.1 [1,091 kB] 2020-12-05T17:28:23.378 INFO:teuthology.orchestra.run.smithi053.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-12-05T17:28:23.379 INFO:teuthology.orchestra.run.smithi002.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-12-05T17:28:23.380 INFO:teuthology.orchestra.run.smithi032.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2020-12-05T17:28:23.381 INFO:teuthology.orchestra.run.smithi002.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-12-05T17:28:23.381 INFO:teuthology.orchestra.run.smithi032.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2020-12-05T17:28:23.382 INFO:teuthology.orchestra.run.smithi053.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-12-05T17:28:23.385 INFO:teuthology.orchestra.run.smithi053.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-12-05T17:28:23.385 INFO:teuthology.orchestra.run.smithi002.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-12-05T17:28:23.386 INFO:teuthology.orchestra.run.smithi053.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-12-05T17:28:23.386 INFO:teuthology.orchestra.run.smithi002.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-12-05T17:28:23.387 INFO:teuthology.orchestra.run.smithi053.stdout:Get:62 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 15.2.6-607-g3b1861ac-1bionic [3,059 kB] 2020-12-05T17:28:23.397 INFO:teuthology.orchestra.run.smithi036.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2020-12-05T17:28:23.397 INFO:teuthology.orchestra.run.smithi036.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2020-12-05T17:28:23.398 INFO:teuthology.orchestra.run.smithi036.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2020-12-05T17:28:23.401 INFO:teuthology.orchestra.run.smithi036.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.1 [1,091 kB] 2020-12-05T17:28:23.415 INFO:teuthology.orchestra.run.smithi061.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2020-12-05T17:28:23.415 INFO:teuthology.orchestra.run.smithi061.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2020-12-05T17:28:23.419 INFO:teuthology.orchestra.run.smithi036.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2020-12-05T17:28:23.423 INFO:teuthology.orchestra.run.smithi061.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2020-12-05T17:28:23.424 INFO:teuthology.orchestra.run.smithi061.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2020-12-05T17:28:23.424 INFO:teuthology.orchestra.run.smithi061.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2020-12-05T17:28:23.425 INFO:teuthology.orchestra.run.smithi061.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2020-12-05T17:28:23.427 INFO:teuthology.orchestra.run.smithi061.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2020-12-05T17:28:23.429 INFO:teuthology.orchestra.run.smithi061.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2020-12-05T17:28:23.455 INFO:teuthology.orchestra.run.smithi053.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-12-05T17:28:23.457 INFO:teuthology.orchestra.run.smithi002.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-12-05T17:28:23.461 INFO:teuthology.orchestra.run.smithi053.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-12-05T17:28:23.463 INFO:teuthology.orchestra.run.smithi002.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-12-05T17:28:23.471 INFO:teuthology.orchestra.run.smithi002.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2020-12-05T17:28:23.471 INFO:teuthology.orchestra.run.smithi002.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2020-12-05T17:28:23.472 INFO:teuthology.orchestra.run.smithi053.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2020-12-05T17:28:23.472 INFO:teuthology.orchestra.run.smithi053.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2020-12-05T17:28:23.472 INFO:teuthology.orchestra.run.smithi032.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2020-12-05T17:28:23.473 INFO:teuthology.orchestra.run.smithi032.stdout:Get:95 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2020-12-05T17:28:23.474 INFO:teuthology.orchestra.run.smithi002.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2020-12-05T17:28:23.474 INFO:teuthology.orchestra.run.smithi002.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2020-12-05T17:28:23.474 INFO:teuthology.orchestra.run.smithi053.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2020-12-05T17:28:23.474 INFO:teuthology.orchestra.run.smithi053.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2020-12-05T17:28:23.475 INFO:teuthology.orchestra.run.smithi032.stdout:Get:96 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.3 [330 kB] 2020-12-05T17:28:23.477 INFO:teuthology.orchestra.run.smithi036.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2020-12-05T17:28:23.494 INFO:teuthology.orchestra.run.smithi061.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2020-12-05T17:28:23.495 INFO:teuthology.orchestra.run.smithi061.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2020-12-05T17:28:23.527 INFO:teuthology.orchestra.run.smithi032.stdout:Get:97 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2020-12-05T17:28:23.528 INFO:teuthology.orchestra.run.smithi032.stdout:Get:98 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2020-12-05T17:28:23.530 INFO:teuthology.orchestra.run.smithi032.stdout:Get:99 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2020-12-05T17:28:23.530 INFO:teuthology.orchestra.run.smithi032.stdout:Get:100 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2020-12-05T17:28:23.533 INFO:teuthology.orchestra.run.smithi032.stdout:Get:101 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2020-12-05T17:28:23.533 INFO:teuthology.orchestra.run.smithi032.stdout:Get:102 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2020-12-05T17:28:23.534 INFO:teuthology.orchestra.run.smithi053.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2020-12-05T17:28:23.534 INFO:teuthology.orchestra.run.smithi053.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2020-12-05T17:28:23.536 INFO:teuthology.orchestra.run.smithi053.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2020-12-05T17:28:23.537 INFO:teuthology.orchestra.run.smithi002.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2020-12-05T17:28:23.537 INFO:teuthology.orchestra.run.smithi002.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2020-12-05T17:28:23.540 INFO:teuthology.orchestra.run.smithi002.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2020-12-05T17:28:23.541 INFO:teuthology.orchestra.run.smithi053.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2020-12-05T17:28:23.544 INFO:teuthology.orchestra.run.smithi002.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2020-12-05T17:28:23.575 INFO:teuthology.orchestra.run.smithi061.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2020-12-05T17:28:23.578 INFO:teuthology.orchestra.run.smithi061.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.1 [1,091 kB] 2020-12-05T17:28:23.581 INFO:teuthology.orchestra.run.smithi036.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2020-12-05T17:28:23.582 INFO:teuthology.orchestra.run.smithi036.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2020-12-05T17:28:23.582 INFO:teuthology.orchestra.run.smithi036.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.3 [330 kB] 2020-12-05T17:28:23.587 INFO:teuthology.orchestra.run.smithi036.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2020-12-05T17:28:23.587 INFO:teuthology.orchestra.run.smithi036.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2020-12-05T17:28:23.587 INFO:teuthology.orchestra.run.smithi036.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2020-12-05T17:28:23.588 INFO:teuthology.orchestra.run.smithi036.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2020-12-05T17:28:23.589 INFO:teuthology.orchestra.run.smithi036.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2020-12-05T17:28:23.600 INFO:teuthology.orchestra.run.smithi061.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2020-12-05T17:28:23.600 INFO:teuthology.orchestra.run.smithi061.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2020-12-05T17:28:23.612 INFO:teuthology.orchestra.run.smithi053.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2020-12-05T17:28:23.612 INFO:teuthology.orchestra.run.smithi053.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2020-12-05T17:28:23.613 INFO:teuthology.orchestra.run.smithi053.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2020-12-05T17:28:23.616 INFO:teuthology.orchestra.run.smithi053.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.1 [1,091 kB] 2020-12-05T17:28:23.617 INFO:teuthology.orchestra.run.smithi002.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2020-12-05T17:28:23.619 INFO:teuthology.orchestra.run.smithi002.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2020-12-05T17:28:23.619 INFO:teuthology.orchestra.run.smithi002.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2020-12-05T17:28:23.621 INFO:teuthology.orchestra.run.smithi002.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.1 [1,091 kB] 2020-12-05T17:28:23.636 INFO:teuthology.orchestra.run.smithi053.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2020-12-05T17:28:23.636 INFO:teuthology.orchestra.run.smithi053.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2020-12-05T17:28:23.638 INFO:teuthology.orchestra.run.smithi002.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2020-12-05T17:28:23.639 INFO:teuthology.orchestra.run.smithi036.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2020-12-05T17:28:23.639 INFO:teuthology.orchestra.run.smithi002.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2020-12-05T17:28:23.739 INFO:teuthology.orchestra.run.smithi002.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2020-12-05T17:28:23.740 INFO:teuthology.orchestra.run.smithi002.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2020-12-05T17:28:23.741 INFO:teuthology.orchestra.run.smithi061.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2020-12-05T17:28:23.742 INFO:teuthology.orchestra.run.smithi061.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2020-12-05T17:28:23.743 INFO:teuthology.orchestra.run.smithi061.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.3 [330 kB] 2020-12-05T17:28:23.753 INFO:teuthology.orchestra.run.smithi061.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2020-12-05T17:28:23.753 INFO:teuthology.orchestra.run.smithi061.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2020-12-05T17:28:23.754 INFO:teuthology.orchestra.run.smithi061.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2020-12-05T17:28:23.755 INFO:teuthology.orchestra.run.smithi061.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2020-12-05T17:28:23.756 INFO:teuthology.orchestra.run.smithi061.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2020-12-05T17:28:23.756 INFO:teuthology.orchestra.run.smithi061.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2020-12-05T17:28:23.777 INFO:teuthology.orchestra.run.smithi053.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2020-12-05T17:28:23.778 INFO:teuthology.orchestra.run.smithi053.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2020-12-05T17:28:23.779 INFO:teuthology.orchestra.run.smithi053.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.3 [330 kB] 2020-12-05T17:28:23.781 INFO:teuthology.orchestra.run.smithi002.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.3 [330 kB] 2020-12-05T17:28:23.785 INFO:teuthology.orchestra.run.smithi053.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2020-12-05T17:28:23.785 INFO:teuthology.orchestra.run.smithi053.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2020-12-05T17:28:23.786 INFO:teuthology.orchestra.run.smithi053.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2020-12-05T17:28:23.786 INFO:teuthology.orchestra.run.smithi053.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2020-12-05T17:28:23.787 INFO:teuthology.orchestra.run.smithi053.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2020-12-05T17:28:23.788 INFO:teuthology.orchestra.run.smithi053.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2020-12-05T17:28:23.792 INFO:teuthology.orchestra.run.smithi002.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2020-12-05T17:28:23.793 INFO:teuthology.orchestra.run.smithi002.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2020-12-05T17:28:23.793 INFO:teuthology.orchestra.run.smithi002.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2020-12-05T17:28:23.796 INFO:teuthology.orchestra.run.smithi002.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2020-12-05T17:28:23.796 INFO:teuthology.orchestra.run.smithi002.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2020-12-05T17:28:23.797 INFO:teuthology.orchestra.run.smithi002.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2020-12-05T17:28:24.130 INFO:teuthology.orchestra.run.smithi036.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 15.2.6-607-g3b1861ac-1bionic [99.3 kB] 2020-12-05T17:28:24.202 INFO:teuthology.orchestra.run.smithi036.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 15.2.6-607-g3b1861ac-1bionic [337 kB] 2020-12-05T17:28:24.284 INFO:teuthology.orchestra.run.smithi036.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 15.2.6-607-g3b1861ac-1bionic [18.8 MB] 2020-12-05T17:28:24.310 INFO:teuthology.orchestra.run.smithi032.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 15.2.6-607-g3b1861ac-1bionic [3,892 B] 2020-12-05T17:28:24.310 INFO:teuthology.orchestra.run.smithi032.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 15.2.6-607-g3b1861ac-1bionic [515 kB] 2020-12-05T17:28:24.322 INFO:teuthology.orchestra.run.smithi032.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 15.2.6-607-g3b1861ac-1bionic [1,850 kB] 2020-12-05T17:28:24.358 INFO:teuthology.orchestra.run.smithi061.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 15.2.6-607-g3b1861ac-1bionic [435 kB] 2020-12-05T17:28:24.408 INFO:teuthology.orchestra.run.smithi032.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 15.2.6-607-g3b1861ac-1bionic [51.4 kB] 2020-12-05T17:28:24.410 INFO:teuthology.orchestra.run.smithi032.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 15.2.6-607-g3b1861ac-1bionic [55.3 kB] 2020-12-05T17:28:24.411 INFO:teuthology.orchestra.run.smithi032.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 15.2.6-607-g3b1861ac-1bionic [3,177 kB] 2020-12-05T17:28:24.567 INFO:teuthology.orchestra.run.smithi032.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-cloud all 15.2.6-607-g3b1861ac-1bionic [34.3 kB] 2020-12-05T17:28:24.568 INFO:teuthology.orchestra.run.smithi032.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 15.2.6-607-g3b1861ac-1bionic [7,889 kB] 2020-12-05T17:28:24.602 INFO:teuthology.orchestra.run.smithi061.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 15.2.6-607-g3b1861ac-1bionic [332 kB] 2020-12-05T17:28:24.708 INFO:teuthology.orchestra.run.smithi053.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 15.2.6-607-g3b1861ac-1bionic [435 kB] 2020-12-05T17:28:24.755 INFO:teuthology.orchestra.run.smithi061.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 15.2.6-607-g3b1861ac-1bionic [20.7 kB] 2020-12-05T17:28:24.756 INFO:teuthology.orchestra.run.smithi061.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 15.2.6-607-g3b1861ac-1bionic [133 kB] 2020-12-05T17:28:24.833 INFO:teuthology.orchestra.run.smithi061.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 15.2.6-607-g3b1861ac-1bionic [27.8 kB] 2020-12-05T17:28:24.834 INFO:teuthology.orchestra.run.smithi061.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 15.2.6-607-g3b1861ac-1bionic [312 kB] 2020-12-05T17:28:24.910 INFO:teuthology.orchestra.run.smithi032.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 15.2.6-607-g3b1861ac-1bionic [4,012 B] 2020-12-05T17:28:24.911 INFO:teuthology.orchestra.run.smithi032.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 15.2.6-607-g3b1861ac-1bionic [22.4 kB] 2020-12-05T17:28:24.911 INFO:teuthology.orchestra.run.smithi032.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 15.2.6-607-g3b1861ac-1bionic [43.9 MB] 2020-12-05T17:28:24.927 INFO:teuthology.orchestra.run.smithi053.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 15.2.6-607-g3b1861ac-1bionic [332 kB] 2020-12-05T17:28:24.991 INFO:teuthology.orchestra.run.smithi061.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 15.2.6-607-g3b1861ac-1bionic [2,889 kB] 2020-12-05T17:28:25.085 INFO:teuthology.orchestra.run.smithi053.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 15.2.6-607-g3b1861ac-1bionic [20.7 kB] 2020-12-05T17:28:25.086 INFO:teuthology.orchestra.run.smithi053.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 15.2.6-607-g3b1861ac-1bionic [133 kB] 2020-12-05T17:28:25.164 INFO:teuthology.orchestra.run.smithi053.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 15.2.6-607-g3b1861ac-1bionic [27.8 kB] 2020-12-05T17:28:25.165 INFO:teuthology.orchestra.run.smithi053.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 15.2.6-607-g3b1861ac-1bionic [312 kB] 2020-12-05T17:28:25.170 INFO:teuthology.orchestra.run.smithi002.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 15.2.6-607-g3b1861ac-1bionic [435 kB] 2020-12-05T17:28:25.252 INFO:teuthology.orchestra.run.smithi053.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 15.2.6-607-g3b1861ac-1bionic [2,889 kB] 2020-12-05T17:28:25.545 INFO:teuthology.orchestra.run.smithi002.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 15.2.6-607-g3b1861ac-1bionic [332 kB] 2020-12-05T17:28:25.785 INFO:teuthology.orchestra.run.smithi002.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 15.2.6-607-g3b1861ac-1bionic [20.7 kB] 2020-12-05T17:28:25.786 INFO:teuthology.orchestra.run.smithi002.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 15.2.6-607-g3b1861ac-1bionic [133 kB] 2020-12-05T17:28:25.875 INFO:teuthology.orchestra.run.smithi002.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 15.2.6-607-g3b1861ac-1bionic [27.8 kB] 2020-12-05T17:28:25.901 INFO:teuthology.orchestra.run.smithi002.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 15.2.6-607-g3b1861ac-1bionic [312 kB] 2020-12-05T17:28:26.083 INFO:teuthology.orchestra.run.smithi002.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 15.2.6-607-g3b1861ac-1bionic [2,889 kB] 2020-12-05T17:28:26.379 INFO:teuthology.orchestra.run.smithi061.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 15.2.6-607-g3b1861ac-1bionic [99.3 kB] 2020-12-05T17:28:26.395 INFO:teuthology.orchestra.run.smithi053.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 15.2.6-607-g3b1861ac-1bionic [99.3 kB] 2020-12-05T17:28:26.438 INFO:teuthology.orchestra.run.smithi061.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 15.2.6-607-g3b1861ac-1bionic [337 kB] 2020-12-05T17:28:26.455 INFO:teuthology.orchestra.run.smithi053.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 15.2.6-607-g3b1861ac-1bionic [337 kB] 2020-12-05T17:28:26.570 INFO:teuthology.orchestra.run.smithi061.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 15.2.6-607-g3b1861ac-1bionic [18.8 MB] 2020-12-05T17:28:26.581 INFO:teuthology.orchestra.run.smithi053.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 15.2.6-607-g3b1861ac-1bionic [18.8 MB] 2020-12-05T17:28:26.941 INFO:teuthology.orchestra.run.smithi032.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 15.2.6-607-g3b1861ac-1bionic [15.7 kB] 2020-12-05T17:28:26.942 INFO:teuthology.orchestra.run.smithi032.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 15.2.6-607-g3b1861ac-1bionic [17.9 kB] 2020-12-05T17:28:26.942 INFO:teuthology.orchestra.run.smithi032.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 15.2.6-607-g3b1861ac-1bionic [8,845 kB] 2020-12-05T17:28:27.346 INFO:teuthology.orchestra.run.smithi032.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 15.2.6-607-g3b1861ac-1bionic [62.5 kB] 2020-12-05T17:28:27.488 INFO:teuthology.orchestra.run.smithi002.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 15.2.6-607-g3b1861ac-1bionic [99.3 kB] 2020-12-05T17:28:27.498 INFO:teuthology.orchestra.run.smithi002.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 15.2.6-607-g3b1861ac-1bionic [337 kB] 2020-12-05T17:28:27.565 INFO:teuthology.orchestra.run.smithi002.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 15.2.6-607-g3b1861ac-1bionic [18.8 MB] 2020-12-05T17:28:28.062 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 156 MB in 6s (25.7 MB/s) 2020-12-05T17:28:28.181 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyparsing. 2020-12-05T17:28:28.228 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137609 files and directories currently installed.) 2020-12-05T17:28:28.232 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2020-12-05T17:28:28.272 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2020-12-05T17:28:28.564 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyperclip. 2020-12-05T17:28:28.579 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2020-12-05T17:28:28.616 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2020-12-05T17:28:28.638 INFO:teuthology.orchestra.run.smithi036.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 15.2.6-607-g3b1861ac-1bionic [5,170 kB] 2020-12-05T17:28:28.852 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cmd2. 2020-12-05T17:28:28.866 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2020-12-05T17:28:28.902 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2020-12-05T17:28:29.213 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-uritemplate. 2020-12-05T17:28:29.227 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2020-12-05T17:28:29.243 INFO:teuthology.orchestra.run.smithi036.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 15.2.6-607-g3b1861ac-1bionic [148 kB] 2020-12-05T17:28:29.254 INFO:teuthology.orchestra.run.smithi036.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 15.2.6-607-g3b1861ac-1bionic [1,282 kB] 2020-12-05T17:28:29.263 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2020-12-05T17:28:29.400 INFO:teuthology.orchestra.run.smithi053.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 15.2.6-607-g3b1861ac-1bionic [5,170 kB] 2020-12-05T17:28:29.409 INFO:teuthology.orchestra.run.smithi036.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 15.2.6-607-g3b1861ac-1bionic [5,942 kB] 2020-12-05T17:28:29.683 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-12-05T17:28:29.696 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../004-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-12-05T17:28:29.766 INFO:teuthology.orchestra.run.smithi061.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 15.2.6-607-g3b1861ac-1bionic [5,170 kB] 2020-12-05T17:28:29.767 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-12-05T17:28:30.020 INFO:teuthology.orchestra.run.smithi053.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 15.2.6-607-g3b1861ac-1bionic [148 kB] 2020-12-05T17:28:30.035 INFO:teuthology.orchestra.run.smithi053.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 15.2.6-607-g3b1861ac-1bionic [1,282 kB] 2020-12-05T17:28:30.043 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../005-librbd1_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:30.113 INFO:teuthology.orchestra.run.smithi036.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 15.2.6-607-g3b1861ac-1bionic [22.7 MB] 2020-12-05T17:28:30.145 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librbd1 (15.2.6-607-g3b1861ac-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2020-12-05T17:28:30.190 INFO:teuthology.orchestra.run.smithi053.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 15.2.6-607-g3b1861ac-1bionic [5,942 kB] 2020-12-05T17:28:30.465 INFO:teuthology.orchestra.run.smithi061.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 15.2.6-607-g3b1861ac-1bionic [148 kB] 2020-12-05T17:28:30.487 INFO:teuthology.orchestra.run.smithi061.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 15.2.6-607-g3b1861ac-1bionic [1,282 kB] 2020-12-05T17:28:30.634 INFO:teuthology.orchestra.run.smithi002.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 15.2.6-607-g3b1861ac-1bionic [5,170 kB] 2020-12-05T17:28:30.655 INFO:teuthology.orchestra.run.smithi061.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 15.2.6-607-g3b1861ac-1bionic [5,942 kB] 2020-12-05T17:28:30.698 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../006-librados2_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:30.824 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librados2 (15.2.6-607-g3b1861ac-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2020-12-05T17:28:30.911 INFO:teuthology.orchestra.run.smithi053.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 15.2.6-607-g3b1861ac-1bionic [22.7 MB] 2020-12-05T17:28:31.411 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libcephfs2. 2020-12-05T17:28:31.416 INFO:teuthology.orchestra.run.smithi002.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 15.2.6-607-g3b1861ac-1bionic [148 kB] 2020-12-05T17:28:31.425 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../007-libcephfs2_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:31.440 INFO:teuthology.orchestra.run.smithi002.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 15.2.6-607-g3b1861ac-1bionic [1,282 kB] 2020-12-05T17:28:31.458 INFO:teuthology.orchestra.run.smithi061.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 15.2.6-607-g3b1861ac-1bionic [22.7 MB] 2020-12-05T17:28:31.461 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libcephfs2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:31.639 INFO:teuthology.orchestra.run.smithi002.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 15.2.6-607-g3b1861ac-1bionic [5,942 kB] 2020-12-05T17:28:31.764 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rados. 2020-12-05T17:28:31.779 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../008-python3-rados_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:31.822 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rados (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:32.175 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-ceph-argparse. 2020-12-05T17:28:32.188 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../009-python3-ceph-argparse_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:28:32.234 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-ceph-argparse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:32.486 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cephfs. 2020-12-05T17:28:32.501 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../010-python3-cephfs_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:32.536 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cephfs (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:32.546 INFO:teuthology.orchestra.run.smithi002.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 15.2.6-607-g3b1861ac-1bionic [22.7 MB] 2020-12-05T17:28:32.832 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-ceph-common. 2020-12-05T17:28:32.844 INFO:teuthology.orchestra.run.smithi036.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 15.2.6-607-g3b1861ac-1bionic [3,892 B] 2020-12-05T17:28:32.844 INFO:teuthology.orchestra.run.smithi036.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 15.2.6-607-g3b1861ac-1bionic [515 kB] 2020-12-05T17:28:32.847 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../011-python3-ceph-common_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:28:32.891 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-ceph-common (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:32.907 INFO:teuthology.orchestra.run.smithi036.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 15.2.6-607-g3b1861ac-1bionic [1,850 kB] 2020-12-05T17:28:33.131 INFO:teuthology.orchestra.run.smithi036.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 15.2.6-607-g3b1861ac-1bionic [51.4 kB] 2020-12-05T17:28:33.137 INFO:teuthology.orchestra.run.smithi036.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 15.2.6-607-g3b1861ac-1bionic [55.3 kB] 2020-12-05T17:28:33.144 INFO:teuthology.orchestra.run.smithi036.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 15.2.6-607-g3b1861ac-1bionic [3,177 kB] 2020-12-05T17:28:33.185 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-prettytable. 2020-12-05T17:28:33.201 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-3_all.deb ... 2020-12-05T17:28:33.243 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2020-12-05T17:28:33.463 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rbd. 2020-12-05T17:28:33.476 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../013-python3-rbd_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:33.512 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rbd (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:33.534 INFO:teuthology.orchestra.run.smithi036.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-cloud all 15.2.6-607-g3b1861ac-1bionic [34.3 kB] 2020-12-05T17:28:33.537 INFO:teuthology.orchestra.run.smithi036.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 15.2.6-607-g3b1861ac-1bionic [7,889 kB] 2020-12-05T17:28:33.692 INFO:teuthology.orchestra.run.smithi053.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 15.2.6-607-g3b1861ac-1bionic [3,892 B] 2020-12-05T17:28:33.693 INFO:teuthology.orchestra.run.smithi053.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 15.2.6-607-g3b1861ac-1bionic [515 kB] 2020-12-05T17:28:33.751 INFO:teuthology.orchestra.run.smithi053.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 15.2.6-607-g3b1861ac-1bionic [1,850 kB] 2020-12-05T17:28:33.832 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librabbitmq4:amd64. 2020-12-05T17:28:33.847 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../014-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2020-12-05T17:28:33.890 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-12-05T17:28:33.979 INFO:teuthology.orchestra.run.smithi053.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 15.2.6-607-g3b1861ac-1bionic [51.4 kB] 2020-12-05T17:28:33.985 INFO:teuthology.orchestra.run.smithi053.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 15.2.6-607-g3b1861ac-1bionic [55.3 kB] 2020-12-05T17:28:33.991 INFO:teuthology.orchestra.run.smithi053.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 15.2.6-607-g3b1861ac-1bionic [3,177 kB] 2020-12-05T17:28:34.210 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librdkafka1:amd64. 2020-12-05T17:28:34.224 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../015-librdkafka1_0.11.3-1build1_amd64.deb ... 2020-12-05T17:28:34.268 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2020-12-05T17:28:34.383 INFO:teuthology.orchestra.run.smithi053.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-cloud all 15.2.6-607-g3b1861ac-1bionic [34.3 kB] 2020-12-05T17:28:34.386 INFO:teuthology.orchestra.run.smithi053.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 15.2.6-607-g3b1861ac-1bionic [7,889 kB] 2020-12-05T17:28:34.501 INFO:teuthology.orchestra.run.smithi036.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 15.2.6-607-g3b1861ac-1bionic [4,012 B] 2020-12-05T17:28:34.501 INFO:teuthology.orchestra.run.smithi036.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 15.2.6-607-g3b1861ac-1bionic [22.4 kB] 2020-12-05T17:28:34.507 INFO:teuthology.orchestra.run.smithi036.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 15.2.6-607-g3b1861ac-1bionic [43.9 MB] 2020-12-05T17:28:34.522 INFO:teuthology.orchestra.run.smithi061.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 15.2.6-607-g3b1861ac-1bionic [3,892 B] 2020-12-05T17:28:34.523 INFO:teuthology.orchestra.run.smithi061.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 15.2.6-607-g3b1861ac-1bionic [515 kB] 2020-12-05T17:28:34.579 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librgw2. 2020-12-05T17:28:34.591 INFO:teuthology.orchestra.run.smithi061.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 15.2.6-607-g3b1861ac-1bionic [1,850 kB] 2020-12-05T17:28:34.595 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../016-librgw2_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:34.629 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librgw2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:34.839 INFO:teuthology.orchestra.run.smithi061.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 15.2.6-607-g3b1861ac-1bionic [51.4 kB] 2020-12-05T17:28:34.843 INFO:teuthology.orchestra.run.smithi061.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 15.2.6-607-g3b1861ac-1bionic [55.3 kB] 2020-12-05T17:28:34.853 INFO:teuthology.orchestra.run.smithi061.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 15.2.6-607-g3b1861ac-1bionic [3,177 kB] 2020-12-05T17:28:35.164 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rgw. 2020-12-05T17:28:35.181 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../017-python3-rgw_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:35.278 INFO:teuthology.orchestra.run.smithi061.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-cloud all 15.2.6-607-g3b1861ac-1bionic [34.3 kB] 2020-12-05T17:28:35.279 INFO:teuthology.orchestra.run.smithi061.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 15.2.6-607-g3b1861ac-1bionic [7,889 kB] 2020-12-05T17:28:35.361 INFO:teuthology.orchestra.run.smithi053.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 15.2.6-607-g3b1861ac-1bionic [4,012 B] 2020-12-05T17:28:35.361 INFO:teuthology.orchestra.run.smithi053.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 15.2.6-607-g3b1861ac-1bionic [22.4 kB] 2020-12-05T17:28:35.362 INFO:teuthology.orchestra.run.smithi053.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 15.2.6-607-g3b1861ac-1bionic [43.9 MB] 2020-12-05T17:28:35.423 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rgw (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:35.767 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-12-05T17:28:35.784 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../018-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-12-05T17:28:35.825 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-12-05T17:28:35.881 INFO:teuthology.orchestra.run.smithi002.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 15.2.6-607-g3b1861ac-1bionic [3,892 B] 2020-12-05T17:28:35.881 INFO:teuthology.orchestra.run.smithi002.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 15.2.6-607-g3b1861ac-1bionic [515 kB] 2020-12-05T17:28:35.952 INFO:teuthology.orchestra.run.smithi002.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 15.2.6-607-g3b1861ac-1bionic [1,850 kB] 2020-12-05T17:28:36.128 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-12-05T17:28:36.144 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../019-libleveldb1v5_1.20-2_amd64.deb ... 2020-12-05T17:28:36.186 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-12-05T17:28:36.189 INFO:teuthology.orchestra.run.smithi002.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 15.2.6-607-g3b1861ac-1bionic [51.4 kB] 2020-12-05T17:28:36.192 INFO:teuthology.orchestra.run.smithi002.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 15.2.6-607-g3b1861ac-1bionic [55.3 kB] 2020-12-05T17:28:36.200 INFO:teuthology.orchestra.run.smithi002.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 15.2.6-607-g3b1861ac-1bionic [3,177 kB] 2020-12-05T17:28:36.372 INFO:teuthology.orchestra.run.smithi061.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 15.2.6-607-g3b1861ac-1bionic [4,012 B] 2020-12-05T17:28:36.373 INFO:teuthology.orchestra.run.smithi061.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 15.2.6-607-g3b1861ac-1bionic [22.4 kB] 2020-12-05T17:28:36.373 INFO:teuthology.orchestra.run.smithi061.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 15.2.6-607-g3b1861ac-1bionic [43.9 MB] 2020-12-05T17:28:36.506 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liboath0. 2020-12-05T17:28:36.522 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../020-liboath0_2.6.1-1_amd64.deb ... 2020-12-05T17:28:36.556 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-12-05T17:28:36.594 INFO:teuthology.orchestra.run.smithi002.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-cloud all 15.2.6-607-g3b1861ac-1bionic [34.3 kB] 2020-12-05T17:28:36.595 INFO:teuthology.orchestra.run.smithi002.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 15.2.6-607-g3b1861ac-1bionic [7,889 kB] 2020-12-05T17:28:36.833 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libradosstriper1. 2020-12-05T17:28:36.846 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../021-libradosstriper1_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:36.875 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libradosstriper1 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:37.320 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-common. 2020-12-05T17:28:37.333 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../022-ceph-common_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:37.378 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-common (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:37.549 INFO:teuthology.orchestra.run.smithi002.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 15.2.6-607-g3b1861ac-1bionic [4,012 B] 2020-12-05T17:28:37.550 INFO:teuthology.orchestra.run.smithi002.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 15.2.6-607-g3b1861ac-1bionic [22.4 kB] 2020-12-05T17:28:37.551 INFO:teuthology.orchestra.run.smithi002.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 15.2.6-607-g3b1861ac-1bionic [43.9 MB] 2020-12-05T17:28:39.083 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-base. 2020-12-05T17:28:39.099 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../023-ceph-base_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:39.143 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-base (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:40.004 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-dateutil. 2020-12-05T17:28:40.018 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../024-python3-dateutil_2.6.1-1_all.deb ... 2020-12-05T17:28:40.054 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2020-12-05T17:28:40.285 INFO:teuthology.orchestra.run.smithi036.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 15.2.6-607-g3b1861ac-1bionic [15.7 kB] 2020-12-05T17:28:40.286 INFO:teuthology.orchestra.run.smithi036.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 15.2.6-607-g3b1861ac-1bionic [17.9 kB] 2020-12-05T17:28:40.286 INFO:teuthology.orchestra.run.smithi036.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 15.2.6-607-g3b1861ac-1bionic [8,845 kB] 2020-12-05T17:28:40.398 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-asn1crypto. 2020-12-05T17:28:40.415 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../025-python3-asn1crypto_0.24.0-1_all.deb ... 2020-12-05T17:28:40.457 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2020-12-05T17:28:40.726 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cffi-backend. 2020-12-05T17:28:40.741 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../026-python3-cffi-backend_1.11.5-1_amd64.deb ... 2020-12-05T17:28:40.776 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2020-12-05T17:28:41.062 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cryptography. 2020-12-05T17:28:41.078 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../027-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2020-12-05T17:28:41.112 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-12-05T17:28:41.251 INFO:teuthology.orchestra.run.smithi053.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 15.2.6-607-g3b1861ac-1bionic [15.7 kB] 2020-12-05T17:28:41.251 INFO:teuthology.orchestra.run.smithi053.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 15.2.6-607-g3b1861ac-1bionic [17.9 kB] 2020-12-05T17:28:41.252 INFO:teuthology.orchestra.run.smithi053.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 15.2.6-607-g3b1861ac-1bionic [8,845 kB] 2020-12-05T17:28:41.495 INFO:teuthology.orchestra.run.smithi036.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 15.2.6-607-g3b1861ac-1bionic [62.5 kB] 2020-12-05T17:28:41.549 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-openssl. 2020-12-05T17:28:41.563 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../028-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2020-12-05T17:28:41.616 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2020-12-05T17:28:41.910 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2020-12-05T17:28:41.924 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../029-ceph-mgr-modules-core_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:28:41.968 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-modules-core (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:42.137 INFO:teuthology.orchestra.run.smithi061.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 15.2.6-607-g3b1861ac-1bionic [15.7 kB] 2020-12-05T17:28:42.138 INFO:teuthology.orchestra.run.smithi061.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 15.2.6-607-g3b1861ac-1bionic [17.9 kB] 2020-12-05T17:28:42.139 INFO:teuthology.orchestra.run.smithi061.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 15.2.6-607-g3b1861ac-1bionic [8,845 kB] 2020-12-05T17:28:42.218 INFO:teuthology.orchestra.run.smithi053.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 15.2.6-607-g3b1861ac-1bionic [62.5 kB] 2020-12-05T17:28:42.230 INFO:teuthology.orchestra.run.smithi036.stdout:Fetched 156 MB in 20s (7,762 kB/s) 2020-12-05T17:28:42.355 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-bcrypt. 2020-12-05T17:28:42.369 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../030-python3-bcrypt_3.1.4-2_amd64.deb ... 2020-12-05T17:28:42.384 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-pyparsing. 2020-12-05T17:28:42.414 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2020-12-05T17:28:42.416 INFO:teuthology.orchestra.run.smithi002.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 15.2.6-607-g3b1861ac-1bionic [15.7 kB] 2020-12-05T17:28:42.417 INFO:teuthology.orchestra.run.smithi002.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 15.2.6-607-g3b1861ac-1bionic [17.9 kB] 2020-12-05T17:28:42.417 INFO:teuthology.orchestra.run.smithi002.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 15.2.6-607-g3b1861ac-1bionic [8,845 kB] 2020-12-05T17:28:42.435 INFO:teuthology.orchestra.run.smithi036.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137609 files and directories currently installed.) 2020-12-05T17:28:42.440 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2020-12-05T17:28:42.474 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2020-12-05T17:28:42.717 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cherrypy3. 2020-12-05T17:28:42.732 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../031-python3-cherrypy3_8.9.1-2_all.deb ... 2020-12-05T17:28:42.775 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2020-12-05T17:28:42.776 INFO:teuthology.orchestra.run.smithi061.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 15.2.6-607-g3b1861ac-1bionic [62.5 kB] 2020-12-05T17:28:42.785 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-pyperclip. 2020-12-05T17:28:42.801 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2020-12-05T17:28:42.843 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2020-12-05T17:28:42.882 INFO:teuthology.orchestra.run.smithi002.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/octopus/3b1861acb8137d863184795ccadbd5d0cb026674/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 15.2.6-607-g3b1861ac-1bionic [62.5 kB] 2020-12-05T17:28:42.954 INFO:teuthology.orchestra.run.smithi053.stdout:Fetched 156 MB in 21s (7,582 kB/s) 2020-12-05T17:28:43.079 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-cmd2. 2020-12-05T17:28:43.096 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2020-12-05T17:28:43.103 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pyparsing. 2020-12-05T17:28:43.129 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2020-12-05T17:28:43.153 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jwt. 2020-12-05T17:28:43.153 INFO:teuthology.orchestra.run.smithi053.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137609 files and directories currently installed.) 2020-12-05T17:28:43.157 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2020-12-05T17:28:43.169 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../032-python3-jwt_1.5.3+ds1-1_all.deb ... 2020-12-05T17:28:43.193 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2020-12-05T17:28:43.209 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2020-12-05T17:28:43.373 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-uritemplate. 2020-12-05T17:28:43.389 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2020-12-05T17:28:43.423 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2020-12-05T17:28:43.496 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pyperclip. 2020-12-05T17:28:43.507 INFO:teuthology.orchestra.run.smithi061.stdout:Fetched 156 MB in 21s (7,327 kB/s) 2020-12-05T17:28:43.511 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2020-12-05T17:28:43.528 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-logutils. 2020-12-05T17:28:43.542 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-5_all.deb ... 2020-12-05T17:28:43.555 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2020-12-05T17:28:43.579 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-logutils (0.3.3-5) ... 2020-12-05T17:28:43.618 INFO:teuthology.orchestra.run.smithi002.stdout:Fetched 156 MB in 21s (7,338 kB/s) 2020-12-05T17:28:43.631 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-pyparsing. 2020-12-05T17:28:43.681 INFO:teuthology.orchestra.run.smithi061.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137609 files and directories currently installed.) 2020-12-05T17:28:43.685 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2020-12-05T17:28:43.713 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2020-12-05T17:28:43.726 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-12-05T17:28:43.742 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../004-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-12-05T17:28:43.768 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-pyparsing. 2020-12-05T17:28:43.815 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-cmd2. 2020-12-05T17:28:43.819 INFO:teuthology.orchestra.run.smithi002.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137609 files and directories currently installed.) 2020-12-05T17:28:43.823 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2020-12-05T17:28:43.829 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2020-12-05T17:28:43.852 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-12-05T17:28:43.858 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2020-12-05T17:28:43.865 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2020-12-05T17:28:43.873 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-markupsafe. 2020-12-05T17:28:43.889 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../034-python3-markupsafe_1.0-1build1_amd64.deb ... 2020-12-05T17:28:43.956 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2020-12-05T17:28:44.091 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-pyperclip. 2020-12-05T17:28:44.101 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-uritemplate. 2020-12-05T17:28:44.108 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2020-12-05T17:28:44.116 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2020-12-05T17:28:44.141 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2020-12-05T17:28:44.151 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2020-12-05T17:28:44.193 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../005-librbd1_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:44.228 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-pyperclip. 2020-12-05T17:28:44.244 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2020-12-05T17:28:44.250 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-mako. 2020-12-05T17:28:44.264 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../035-python3-mako_1.0.7+ds1-1_all.deb ... 2020-12-05T17:28:44.286 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2020-12-05T17:28:44.305 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking librbd1 (15.2.6-607-g3b1861ac-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2020-12-05T17:28:44.309 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2020-12-05T17:28:44.444 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-cmd2. 2020-12-05T17:28:44.455 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-12-05T17:28:44.460 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2020-12-05T17:28:44.467 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../004-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-12-05T17:28:44.502 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2020-12-05T17:28:44.546 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-12-05T17:28:44.580 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-cmd2. 2020-12-05T17:28:44.595 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2020-12-05T17:28:44.630 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2020-12-05T17:28:44.645 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-simplegeneric. 2020-12-05T17:28:44.660 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-1_all.deb ... 2020-12-05T17:28:44.703 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2020-12-05T17:28:44.782 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../006-librados2_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:44.884 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking librados2 (15.2.6-607-g3b1861ac-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2020-12-05T17:28:44.898 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../005-librbd1_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:44.899 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-uritemplate. 2020-12-05T17:28:44.913 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2020-12-05T17:28:44.955 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-uritemplate. 2020-12-05T17:28:44.966 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2020-12-05T17:28:44.971 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2020-12-05T17:28:45.004 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2020-12-05T17:28:45.016 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking librbd1 (15.2.6-607-g3b1861ac-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2020-12-05T17:28:45.040 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-singledispatch. 2020-12-05T17:28:45.053 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2_all.deb ... 2020-12-05T17:28:45.090 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2020-12-05T17:28:45.261 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-12-05T17:28:45.277 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../004-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-12-05T17:28:45.307 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-12-05T17:28:45.321 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../004-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-12-05T17:28:45.386 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-12-05T17:28:45.392 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-webob. 2020-12-05T17:28:45.408 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../038-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-12-05T17:28:45.409 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-12-05T17:28:45.444 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2020-12-05T17:28:45.529 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libcephfs2. 2020-12-05T17:28:45.544 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../007-libcephfs2_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:45.588 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libcephfs2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:45.734 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../006-librados2_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:45.741 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../005-librbd1_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:45.761 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../005-librbd1_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:45.764 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-bs4. 2020-12-05T17:28:45.779 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../039-python3-bs4_4.6.0-1_all.deb ... 2020-12-05T17:28:45.822 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2020-12-05T17:28:45.837 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking librados2 (15.2.6-607-g3b1861ac-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2020-12-05T17:28:45.849 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-rados. 2020-12-05T17:28:45.862 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../008-python3-rados_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:45.869 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking librbd1 (15.2.6-607-g3b1861ac-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2020-12-05T17:28:45.890 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-rados (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:45.898 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking librbd1 (15.2.6-607-g3b1861ac-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2020-12-05T17:28:46.176 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-ceph-argparse. 2020-12-05T17:28:46.191 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../009-python3-ceph-argparse_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:28:46.208 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-waitress. 2020-12-05T17:28:46.223 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../040-python3-waitress_1.0.1-1_all.deb ... 2020-12-05T17:28:46.226 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-ceph-argparse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:46.275 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-waitress (1.0.1-1) ... 2020-12-05T17:28:46.390 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../006-librados2_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:46.412 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../006-librados2_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:46.474 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libcephfs2. 2020-12-05T17:28:46.487 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../007-libcephfs2_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:46.504 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-cephfs. 2020-12-05T17:28:46.520 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking librados2 (15.2.6-607-g3b1861ac-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2020-12-05T17:28:46.521 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../010-python3-cephfs_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:46.524 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libcephfs2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:46.532 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking librados2 (15.2.6-607-g3b1861ac-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2020-12-05T17:28:46.563 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-cephfs (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:46.637 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-tempita. 2020-12-05T17:28:46.650 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2020-12-05T17:28:46.686 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-tempita (0.5.2-2) ... 2020-12-05T17:28:46.810 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-rados. 2020-12-05T17:28:46.824 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../008-python3-rados_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:46.860 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-rados (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:46.890 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-ceph-common. 2020-12-05T17:28:46.907 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../011-python3-ceph-common_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:28:46.948 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-ceph-common (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:47.014 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-paste. 2020-12-05T17:28:47.027 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-12-05T17:28:47.072 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-12-05T17:28:47.154 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-ceph-argparse. 2020-12-05T17:28:47.161 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package libcephfs2. 2020-12-05T17:28:47.167 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../009-python3-ceph-argparse_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:28:47.178 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../007-libcephfs2_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:47.190 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libcephfs2. 2020-12-05T17:28:47.196 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-ceph-argparse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:47.204 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../007-libcephfs2_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:47.212 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-prettytable. 2020-12-05T17:28:47.219 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking libcephfs2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:47.225 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-3_all.deb ... 2020-12-05T17:28:47.248 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libcephfs2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:47.260 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2020-12-05T17:28:47.367 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-12-05T17:28:47.382 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-12-05T17:28:47.425 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-12-05T17:28:47.453 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-rbd. 2020-12-05T17:28:47.469 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../013-python3-rbd_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:47.481 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-cephfs. 2020-12-05T17:28:47.498 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../010-python3-cephfs_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:47.503 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-rbd (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:47.538 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-rados. 2020-12-05T17:28:47.540 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-cephfs (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:47.542 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-rados. 2020-12-05T17:28:47.554 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../008-python3-rados_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:47.557 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../008-python3-rados_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:47.592 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-rados (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:47.604 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-rados (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:47.727 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pastedeploy. 2020-12-05T17:28:47.741 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2020-12-05T17:28:47.777 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2020-12-05T17:28:47.867 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-ceph-common. 2020-12-05T17:28:47.873 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package librabbitmq4:amd64. 2020-12-05T17:28:47.881 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../011-python3-ceph-common_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:28:47.887 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../014-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2020-12-05T17:28:47.926 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-ceph-common (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:47.931 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-12-05T17:28:47.937 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-ceph-argparse. 2020-12-05T17:28:47.940 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-ceph-argparse. 2020-12-05T17:28:47.951 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../009-python3-ceph-argparse_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:28:47.957 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../009-python3-ceph-argparse_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:28:47.987 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-ceph-argparse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:47.990 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-ceph-argparse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:48.105 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-webtest. 2020-12-05T17:28:48.120 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2020-12-05T17:28:48.163 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2020-12-05T17:28:48.187 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-prettytable. 2020-12-05T17:28:48.201 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-3_all.deb ... 2020-12-05T17:28:48.237 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2020-12-05T17:28:48.259 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package librdkafka1:amd64. 2020-12-05T17:28:48.268 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-cephfs. 2020-12-05T17:28:48.272 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../015-librdkafka1_0.11.3-1build1_amd64.deb ... 2020-12-05T17:28:48.284 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../010-python3-cephfs_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:48.289 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-cephfs. 2020-12-05T17:28:48.301 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2020-12-05T17:28:48.302 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../010-python3-cephfs_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:48.327 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-cephfs (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:48.356 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-cephfs (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:48.439 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-rbd. 2020-12-05T17:28:48.453 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../013-python3-rbd_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:48.481 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-rbd (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:48.541 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pecan. 2020-12-05T17:28:48.556 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2020-12-05T17:28:48.587 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package librgw2. 2020-12-05T17:28:48.602 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../016-librgw2_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:48.608 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pecan (1.2.1-2) ... 2020-12-05T17:28:48.637 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking librgw2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:48.658 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-ceph-common. 2020-12-05T17:28:48.662 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-ceph-common. 2020-12-05T17:28:48.674 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../011-python3-ceph-common_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:28:48.676 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../011-python3-ceph-common_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:28:48.709 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-ceph-common (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:48.721 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-ceph-common (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:48.850 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package librabbitmq4:amd64. 2020-12-05T17:28:48.865 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../014-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2020-12-05T17:28:48.909 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-12-05T17:28:49.012 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-werkzeug. 2020-12-05T17:28:49.023 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-prettytable. 2020-12-05T17:28:49.027 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.1_all.deb ... 2020-12-05T17:28:49.028 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-prettytable. 2020-12-05T17:28:49.037 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-3_all.deb ... 2020-12-05T17:28:49.044 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-3_all.deb ... 2020-12-05T17:28:49.078 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2020-12-05T17:28:49.082 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2020-12-05T17:28:49.087 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2020-12-05T17:28:49.131 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-rgw. 2020-12-05T17:28:49.146 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../017-python3-rgw_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:49.181 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-rgw (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:49.245 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package librdkafka1:amd64. 2020-12-05T17:28:49.260 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../015-librdkafka1_0.11.3-1build1_amd64.deb ... 2020-12-05T17:28:49.295 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2020-12-05T17:28:49.306 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-rbd. 2020-12-05T17:28:49.318 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-rbd. 2020-12-05T17:28:49.319 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../013-python3-rbd_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:49.332 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../013-python3-rbd_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:49.348 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-rbd (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:49.368 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-rbd (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:49.490 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr. 2020-12-05T17:28:49.504 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../048-ceph-mgr_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:49.516 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-12-05T17:28:49.530 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../018-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-12-05T17:28:49.557 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:49.564 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package librgw2. 2020-12-05T17:28:49.566 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-12-05T17:28:49.579 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../016-librgw2_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:49.614 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking librgw2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:49.709 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package librabbitmq4:amd64. 2020-12-05T17:28:49.712 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package librabbitmq4:amd64. 2020-12-05T17:28:49.723 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../014-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2020-12-05T17:28:49.727 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../014-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2020-12-05T17:28:49.759 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-12-05T17:28:49.770 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-12-05T17:28:49.811 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-12-05T17:28:49.827 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../019-libleveldb1v5_1.20-2_amd64.deb ... 2020-12-05T17:28:49.861 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-12-05T17:28:50.133 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-rgw. 2020-12-05T17:28:50.140 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package librdkafka1:amd64. 2020-12-05T17:28:50.146 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../017-python3-rgw_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:50.151 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mon. 2020-12-05T17:28:50.153 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package librdkafka1:amd64. 2020-12-05T17:28:50.155 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../015-librdkafka1_0.11.3-1build1_amd64.deb ... 2020-12-05T17:28:50.163 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package liboath0. 2020-12-05T17:28:50.165 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../049-ceph-mon_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:50.166 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../015-librdkafka1_0.11.3-1build1_amd64.deb ... 2020-12-05T17:28:50.180 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../020-liboath0_2.6.1-1_amd64.deb ... 2020-12-05T17:28:50.183 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-rgw (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:50.198 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2020-12-05T17:28:50.210 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mon (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:50.213 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-12-05T17:28:50.237 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2020-12-05T17:28:50.508 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libradosstriper1. 2020-12-05T17:28:50.510 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-12-05T17:28:50.522 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../021-libradosstriper1_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:50.526 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../018-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-12-05T17:28:50.534 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package librgw2. 2020-12-05T17:28:50.548 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../016-librgw2_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:50.557 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libradosstriper1 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:50.569 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-12-05T17:28:50.589 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package librgw2. 2020-12-05T17:28:50.593 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking librgw2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:50.602 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../016-librgw2_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:50.664 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking librgw2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:50.796 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-12-05T17:28:50.812 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../019-libleveldb1v5_1.20-2_amd64.deb ... 2020-12-05T17:28:50.846 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-12-05T17:28:50.969 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-common. 2020-12-05T17:28:50.984 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../022-ceph-common_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:51.019 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-common (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:51.103 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-rgw. 2020-12-05T17:28:51.119 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../017-python3-rgw_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:51.162 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-rgw (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:51.166 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package liboath0. 2020-12-05T17:28:51.181 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../020-liboath0_2.6.1-1_amd64.deb ... 2020-12-05T17:28:51.216 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-12-05T17:28:51.217 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-rgw. 2020-12-05T17:28:51.230 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../017-python3-rgw_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:51.292 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-rgw (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:51.506 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-12-05T17:28:51.518 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libradosstriper1. 2020-12-05T17:28:51.520 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../018-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-12-05T17:28:51.534 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../021-libradosstriper1_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:51.564 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-12-05T17:28:51.577 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libradosstriper1 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:51.619 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-12-05T17:28:51.633 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../018-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-12-05T17:28:51.703 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-12-05T17:28:51.755 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-osd. 2020-12-05T17:28:51.770 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../050-ceph-osd_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:51.813 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-osd (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:51.858 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-12-05T17:28:51.874 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../019-libleveldb1v5_1.20-2_amd64.deb ... 2020-12-05T17:28:51.917 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-12-05T17:28:51.963 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-common. 2020-12-05T17:28:51.976 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../022-ceph-common_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:52.013 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-common (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:52.099 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-12-05T17:28:52.114 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../019-libleveldb1v5_1.20-2_amd64.deb ... 2020-12-05T17:28:52.164 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-12-05T17:28:52.236 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package liboath0. 2020-12-05T17:28:52.250 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../020-liboath0_2.6.1-1_amd64.deb ... 2020-12-05T17:28:52.278 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-12-05T17:28:52.575 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package liboath0. 2020-12-05T17:28:52.588 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../020-liboath0_2.6.1-1_amd64.deb ... 2020-12-05T17:28:52.614 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package libradosstriper1. 2020-12-05T17:28:52.625 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-12-05T17:28:52.628 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../021-libradosstriper1_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:52.672 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking libradosstriper1 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:52.919 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libradosstriper1. 2020-12-05T17:28:52.933 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../021-libradosstriper1_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:52.986 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libradosstriper1 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:53.100 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package ceph-common. 2020-12-05T17:28:53.117 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../022-ceph-common_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:53.159 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking ceph-common (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:53.266 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-base. 2020-12-05T17:28:53.280 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../023-ceph-base_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:53.333 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-base (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:53.481 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-common. 2020-12-05T17:28:53.495 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../022-ceph-common_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:53.539 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-common (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:53.685 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-base. 2020-12-05T17:28:53.700 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../023-ceph-base_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:53.743 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph. 2020-12-05T17:28:53.752 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../051-ceph_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:53.752 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-base (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:53.793 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:54.113 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-fuse. 2020-12-05T17:28:54.127 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../052-ceph-fuse_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:54.171 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-fuse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:54.195 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-dateutil. 2020-12-05T17:28:54.211 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../024-python3-dateutil_2.6.1-1_all.deb ... 2020-12-05T17:28:54.254 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2020-12-05T17:28:54.566 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mds. 2020-12-05T17:28:54.581 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../053-ceph-mds_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:54.582 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-asn1crypto. 2020-12-05T17:28:54.596 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../025-python3-asn1crypto_0.24.0-1_all.deb ... 2020-12-05T17:28:54.632 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mds (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:54.640 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2020-12-05T17:28:54.756 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-dateutil. 2020-12-05T17:28:54.771 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../024-python3-dateutil_2.6.1-1_all.deb ... 2020-12-05T17:28:54.814 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2020-12-05T17:28:54.914 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package ceph-base. 2020-12-05T17:28:54.919 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-cffi-backend. 2020-12-05T17:28:54.928 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../023-ceph-base_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:54.934 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../026-python3-cffi-backend_1.11.5-1_amd64.deb ... 2020-12-05T17:28:54.978 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2020-12-05T17:28:54.981 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking ceph-base (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:55.118 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package cephadm. 2020-12-05T17:28:55.134 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../054-cephadm_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:55.167 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-asn1crypto. 2020-12-05T17:28:55.176 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking cephadm (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:55.181 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../025-python3-asn1crypto_0.24.0-1_all.deb ... 2020-12-05T17:28:55.225 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2020-12-05T17:28:55.297 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-cryptography. 2020-12-05T17:28:55.313 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../027-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2020-12-05T17:28:55.345 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-base. 2020-12-05T17:28:55.359 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../023-ceph-base_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:28:55.375 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-12-05T17:28:55.404 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-base (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:55.504 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jinja2. 2020-12-05T17:28:55.521 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../055-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2020-12-05T17:28:55.563 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-12-05T17:28:55.751 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-openssl. 2020-12-05T17:28:55.765 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../028-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2020-12-05T17:28:55.809 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2020-12-05T17:28:55.865 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2020-12-05T17:28:55.881 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../056-ceph-mgr-cephadm_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:28:55.923 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-cephadm (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:56.111 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2020-12-05T17:28:56.125 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../029-ceph-mgr-modules-core_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:28:56.162 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-mgr-modules-core (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:56.178 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-cffi-backend. 2020-12-05T17:28:56.192 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../026-python3-cffi-backend_1.11.5-1_amd64.deb ... 2020-12-05T17:28:56.227 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-repoze.lru. 2020-12-05T17:28:56.236 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2020-12-05T17:28:56.243 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../057-python3-repoze.lru_0.7-1_all.deb ... 2020-12-05T17:28:56.285 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2020-12-05T17:28:56.383 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-dateutil. 2020-12-05T17:28:56.399 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../024-python3-dateutil_2.6.1-1_all.deb ... 2020-12-05T17:28:56.441 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2020-12-05T17:28:56.522 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-cryptography. 2020-12-05T17:28:56.536 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-dateutil. 2020-12-05T17:28:56.538 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../027-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2020-12-05T17:28:56.549 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-bcrypt. 2020-12-05T17:28:56.551 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../024-python3-dateutil_2.6.1-1_all.deb ... 2020-12-05T17:28:56.563 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../030-python3-bcrypt_3.1.4-2_amd64.deb ... 2020-12-05T17:28:56.580 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-12-05T17:28:56.585 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2020-12-05T17:28:56.596 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-routes. 2020-12-05T17:28:56.613 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../058-python3-routes_2.4.1-1_all.deb ... 2020-12-05T17:28:56.615 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2020-12-05T17:28:56.654 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-routes (2.4.1-1) ... 2020-12-05T17:28:56.827 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-asn1crypto. 2020-12-05T17:28:56.841 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../025-python3-asn1crypto_0.24.0-1_all.deb ... 2020-12-05T17:28:56.863 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-asn1crypto. 2020-12-05T17:28:56.876 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../025-python3-asn1crypto_0.24.0-1_all.deb ... 2020-12-05T17:28:56.877 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2020-12-05T17:28:56.905 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2020-12-05T17:28:56.923 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2020-12-05T17:28:56.930 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../059-ceph-mgr-dashboard_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:28:56.959 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-openssl. 2020-12-05T17:28:56.965 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-dashboard (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:56.972 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../028-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2020-12-05T17:28:56.985 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-cherrypy3. 2020-12-05T17:28:57.000 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../031-python3-cherrypy3_8.9.1-2_all.deb ... 2020-12-05T17:28:57.009 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2020-12-05T17:28:57.051 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2020-12-05T17:28:57.132 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-cffi-backend. 2020-12-05T17:28:57.145 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../026-python3-cffi-backend_1.11.5-1_amd64.deb ... 2020-12-05T17:28:57.173 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2020-12-05T17:28:57.278 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2020-12-05T17:28:57.291 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../029-ceph-mgr-modules-core_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:28:57.330 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mgr-modules-core (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:57.413 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-jwt. 2020-12-05T17:28:57.428 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../032-python3-jwt_1.5.3+ds1-1_all.deb ... 2020-12-05T17:28:57.447 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-cffi-backend. 2020-12-05T17:28:57.461 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../026-python3-cffi-backend_1.11.5-1_amd64.deb ... 2020-12-05T17:28:57.480 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2020-12-05T17:28:57.497 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2020-12-05T17:28:57.518 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-cryptography. 2020-12-05T17:28:57.532 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../027-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2020-12-05T17:28:57.576 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-12-05T17:28:57.676 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-bcrypt. 2020-12-05T17:28:57.691 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../030-python3-bcrypt_3.1.4-2_amd64.deb ... 2020-12-05T17:28:57.734 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2020-12-05T17:28:57.808 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-logutils. 2020-12-05T17:28:57.821 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-5_all.deb ... 2020-12-05T17:28:57.858 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-cryptography. 2020-12-05T17:28:57.866 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-logutils (0.3.3-5) ... 2020-12-05T17:28:57.872 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../027-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2020-12-05T17:28:57.905 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2020-12-05T17:28:57.916 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-12-05T17:28:57.921 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-openssl. 2020-12-05T17:28:57.922 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../060-ceph-mgr-diskprediction-cloud_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:28:57.937 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../028-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2020-12-05T17:28:57.963 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:57.980 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2020-12-05T17:28:58.061 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-cherrypy3. 2020-12-05T17:28:58.076 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../031-python3-cherrypy3_8.9.1-2_all.deb ... 2020-12-05T17:28:58.120 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2020-12-05T17:28:58.232 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2020-12-05T17:28:58.248 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../029-ceph-mgr-modules-core_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:28:58.282 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking ceph-mgr-modules-core (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:58.291 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-numpy. 2020-12-05T17:28:58.305 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../061-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2020-12-05T17:28:58.342 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-markupsafe. 2020-12-05T17:28:58.349 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2020-12-05T17:28:58.355 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-openssl. 2020-12-05T17:28:58.356 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../034-python3-markupsafe_1.0-1build1_amd64.deb ... 2020-12-05T17:28:58.369 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../028-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2020-12-05T17:28:58.400 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2020-12-05T17:28:58.413 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2020-12-05T17:28:58.473 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-jwt. 2020-12-05T17:28:58.488 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../032-python3-jwt_1.5.3+ds1-1_all.deb ... 2020-12-05T17:28:58.531 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2020-12-05T17:28:58.628 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-bcrypt. 2020-12-05T17:28:58.644 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../030-python3-bcrypt_3.1.4-2_amd64.deb ... 2020-12-05T17:28:58.686 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2020-12-05T17:28:58.702 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-mako. 2020-12-05T17:28:58.710 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2020-12-05T17:28:58.716 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../035-python3-mako_1.0.7+ds1-1_all.deb ... 2020-12-05T17:28:58.724 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../029-ceph-mgr-modules-core_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:28:58.761 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2020-12-05T17:28:58.766 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-mgr-modules-core (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:28:58.849 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-logutils. 2020-12-05T17:28:58.864 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-5_all.deb ... 2020-12-05T17:28:58.907 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-logutils (0.3.3-5) ... 2020-12-05T17:28:58.980 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-cherrypy3. 2020-12-05T17:28:58.994 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../031-python3-cherrypy3_8.9.1-2_all.deb ... 2020-12-05T17:28:58.999 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-decorator. 2020-12-05T17:28:59.016 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../062-python3-decorator_4.1.2-1_all.deb ... 2020-12-05T17:28:59.030 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2020-12-05T17:28:59.057 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-decorator (4.1.2-1) ... 2020-12-05T17:28:59.081 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-simplegeneric. 2020-12-05T17:28:59.094 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-1_all.deb ... 2020-12-05T17:28:59.130 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2020-12-05T17:28:59.170 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-bcrypt. 2020-12-05T17:28:59.184 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../030-python3-bcrypt_3.1.4-2_amd64.deb ... 2020-12-05T17:28:59.218 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-markupsafe. 2020-12-05T17:28:59.228 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2020-12-05T17:28:59.232 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../034-python3-markupsafe_1.0-1build1_amd64.deb ... 2020-12-05T17:28:59.276 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2020-12-05T17:28:59.385 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-scipy. 2020-12-05T17:28:59.391 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-singledispatch. 2020-12-05T17:28:59.400 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../063-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2020-12-05T17:28:59.407 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2_all.deb ... 2020-12-05T17:28:59.415 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-jwt. 2020-12-05T17:28:59.429 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../032-python3-jwt_1.5.3+ds1-1_all.deb ... 2020-12-05T17:28:59.441 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2020-12-05T17:28:59.451 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2020-12-05T17:28:59.473 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2020-12-05T17:28:59.573 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-cherrypy3. 2020-12-05T17:28:59.587 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../031-python3-cherrypy3_8.9.1-2_all.deb ... 2020-12-05T17:28:59.604 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-mako. 2020-12-05T17:28:59.618 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../035-python3-mako_1.0.7+ds1-1_all.deb ... 2020-12-05T17:28:59.631 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2020-12-05T17:28:59.661 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-webob. 2020-12-05T17:28:59.662 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2020-12-05T17:28:59.675 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../038-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-12-05T17:28:59.728 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2020-12-05T17:28:59.784 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-logutils. 2020-12-05T17:28:59.801 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-5_all.deb ... 2020-12-05T17:28:59.843 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-logutils (0.3.3-5) ... 2020-12-05T17:28:59.964 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-bs4. 2020-12-05T17:28:59.979 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../039-python3-bs4_4.6.0-1_all.deb ... 2020-12-05T17:28:59.982 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-simplegeneric. 2020-12-05T17:28:59.997 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-1_all.deb ... 2020-12-05T17:29:00.014 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2020-12-05T17:29:00.040 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2020-12-05T17:29:00.041 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-jwt. 2020-12-05T17:29:00.056 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../032-python3-jwt_1.5.3+ds1-1_all.deb ... 2020-12-05T17:29:00.100 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2020-12-05T17:29:00.120 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-markupsafe. 2020-12-05T17:29:00.137 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../034-python3-markupsafe_1.0-1build1_amd64.deb ... 2020-12-05T17:29:00.179 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2020-12-05T17:29:00.310 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-waitress. 2020-12-05T17:29:00.324 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../040-python3-waitress_1.0.1-1_all.deb ... 2020-12-05T17:29:00.368 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-singledispatch. 2020-12-05T17:29:00.369 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-waitress (1.0.1-1) ... 2020-12-05T17:29:00.382 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2_all.deb ... 2020-12-05T17:29:00.426 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2020-12-05T17:29:00.461 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-logutils. 2020-12-05T17:29:00.464 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-mako. 2020-12-05T17:29:00.475 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-5_all.deb ... 2020-12-05T17:29:00.481 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../035-python3-mako_1.0.7+ds1-1_all.deb ... 2020-12-05T17:29:00.519 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-logutils (0.3.3-5) ... 2020-12-05T17:29:00.523 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2020-12-05T17:29:00.663 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-tempita. 2020-12-05T17:29:00.677 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2020-12-05T17:29:00.693 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-sklearn-lib. 2020-12-05T17:29:00.707 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2020-12-05T17:29:00.713 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-tempita (0.5.2-2) ... 2020-12-05T17:29:00.737 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-webob. 2020-12-05T17:29:00.750 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../038-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-12-05T17:29:00.751 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2020-12-05T17:29:00.787 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2020-12-05T17:29:00.864 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-markupsafe. 2020-12-05T17:29:00.876 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-simplegeneric. 2020-12-05T17:29:00.881 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../034-python3-markupsafe_1.0-1build1_amd64.deb ... 2020-12-05T17:29:00.891 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-1_all.deb ... 2020-12-05T17:29:00.922 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2020-12-05T17:29:00.934 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2020-12-05T17:29:00.949 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-paste. 2020-12-05T17:29:00.965 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-12-05T17:29:00.999 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-12-05T17:29:01.100 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-bs4. 2020-12-05T17:29:01.114 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../039-python3-bs4_4.6.0-1_all.deb ... 2020-12-05T17:29:01.159 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2020-12-05T17:29:01.229 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-singledispatch. 2020-12-05T17:29:01.245 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2_all.deb ... 2020-12-05T17:29:01.246 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-joblib. 2020-12-05T17:29:01.261 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../065-python3-joblib_0.11-1_all.deb ... 2020-12-05T17:29:01.267 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-mako. 2020-12-05T17:29:01.282 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../035-python3-mako_1.0.7+ds1-1_all.deb ... 2020-12-05T17:29:01.287 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2020-12-05T17:29:01.305 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-joblib (0.11-1) ... 2020-12-05T17:29:01.325 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2020-12-05T17:29:01.369 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-12-05T17:29:01.382 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-12-05T17:29:01.418 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-12-05T17:29:01.520 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-waitress. 2020-12-05T17:29:01.535 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../040-python3-waitress_1.0.1-1_all.deb ... 2020-12-05T17:29:01.564 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-webob. 2020-12-05T17:29:01.578 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../038-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-12-05T17:29:01.586 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-waitress (1.0.1-1) ... 2020-12-05T17:29:01.625 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2020-12-05T17:29:01.681 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-sklearn. 2020-12-05T17:29:01.694 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../066-python3-sklearn_0.19.1-3_all.deb ... 2020-12-05T17:29:01.695 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-simplegeneric. 2020-12-05T17:29:01.710 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-1_all.deb ... 2020-12-05T17:29:01.713 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-pastedeploy. 2020-12-05T17:29:01.727 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2020-12-05T17:29:01.739 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2020-12-05T17:29:01.753 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2020-12-05T17:29:01.771 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2020-12-05T17:29:01.923 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-tempita. 2020-12-05T17:29:01.936 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-bs4. 2020-12-05T17:29:01.937 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2020-12-05T17:29:01.951 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../039-python3-bs4_4.6.0-1_all.deb ... 2020-12-05T17:29:01.998 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-tempita (0.5.2-2) ... 2020-12-05T17:29:02.019 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2020-12-05T17:29:02.066 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-webtest. 2020-12-05T17:29:02.080 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2020-12-05T17:29:02.091 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-singledispatch. 2020-12-05T17:29:02.108 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2_all.deb ... 2020-12-05T17:29:02.132 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2020-12-05T17:29:02.156 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2020-12-05T17:29:02.247 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2020-12-05T17:29:02.263 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:02.300 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-paste. 2020-12-05T17:29:02.306 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:02.314 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-12-05T17:29:02.347 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-waitress. 2020-12-05T17:29:02.359 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-12-05T17:29:02.366 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../040-python3-waitress_1.0.1-1_all.deb ... 2020-12-05T17:29:02.405 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-waitress (1.0.1-1) ... 2020-12-05T17:29:02.477 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-pecan. 2020-12-05T17:29:02.492 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2020-12-05T17:29:02.528 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-webob. 2020-12-05T17:29:02.545 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../038-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-12-05T17:29:02.560 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-pecan (1.2.1-2) ... 2020-12-05T17:29:02.594 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2020-12-05T17:29:02.678 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-12-05T17:29:02.692 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-12-05T17:29:02.716 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-tempita. 2020-12-05T17:29:02.728 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-12-05T17:29:02.733 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2020-12-05T17:29:02.775 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-tempita (0.5.2-2) ... 2020-12-05T17:29:02.940 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-werkzeug. 2020-12-05T17:29:02.955 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.1_all.deb ... 2020-12-05T17:29:02.972 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-bs4. 2020-12-05T17:29:02.986 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../039-python3-bs4_4.6.0-1_all.deb ... 2020-12-05T17:29:03.014 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2020-12-05T17:29:03.022 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2020-12-05T17:29:03.071 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-paste. 2020-12-05T17:29:03.072 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pastedeploy. 2020-12-05T17:29:03.085 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-12-05T17:29:03.088 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2020-12-05T17:29:03.129 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-12-05T17:29:03.139 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2020-12-05T17:29:03.350 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-waitress. 2020-12-05T17:29:03.366 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../040-python3-waitress_1.0.1-1_all.deb ... 2020-12-05T17:29:03.401 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-mgr. 2020-12-05T17:29:03.415 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../048-ceph-mgr_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:03.417 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-waitress (1.0.1-1) ... 2020-12-05T17:29:03.442 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-webtest. 2020-12-05T17:29:03.451 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-mgr (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:03.456 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2020-12-05T17:29:03.467 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-monotonic. 2020-12-05T17:29:03.482 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-12-05T17:29:03.484 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../068-python3-monotonic_1.1-2_all.deb ... 2020-12-05T17:29:03.496 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-12-05T17:29:03.500 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2020-12-05T17:29:03.525 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-monotonic (1.1-2) ... 2020-12-05T17:29:03.549 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-12-05T17:29:03.753 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-tempita. 2020-12-05T17:29:03.768 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2020-12-05T17:29:03.836 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-tempita (0.5.2-2) ... 2020-12-05T17:29:03.843 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-pastedeploy. 2020-12-05T17:29:03.856 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2020-12-05T17:29:03.869 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pecan. 2020-12-05T17:29:03.885 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2020-12-05T17:29:03.893 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2020-12-05T17:29:03.893 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-fasteners. 2020-12-05T17:29:03.909 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../069-python3-fasteners_0.12.0-3_all.deb ... 2020-12-05T17:29:03.928 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pecan (1.2.1-2) ... 2020-12-05T17:29:03.954 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-mon. 2020-12-05T17:29:03.960 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2020-12-05T17:29:03.970 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../049-ceph-mon_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:04.012 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-mon (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:04.156 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-paste. 2020-12-05T17:29:04.169 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-12-05T17:29:04.188 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-webtest. 2020-12-05T17:29:04.203 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2020-12-05T17:29:04.246 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2020-12-05T17:29:04.271 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-httplib2. 2020-12-05T17:29:04.282 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-werkzeug. 2020-12-05T17:29:04.285 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../070-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2020-12-05T17:29:04.297 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.1_all.deb ... 2020-12-05T17:29:04.329 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-12-05T17:29:04.341 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2020-12-05T17:29:04.347 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-12-05T17:29:04.607 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-pecan. 2020-12-05T17:29:04.624 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2020-12-05T17:29:04.657 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyasn1. 2020-12-05T17:29:04.670 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3_all.deb ... 2020-12-05T17:29:04.673 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-pecan (1.2.1-2) ... 2020-12-05T17:29:04.700 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-12-05T17:29:04.706 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2020-12-05T17:29:04.710 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mgr. 2020-12-05T17:29:04.715 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-12-05T17:29:04.725 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../048-ceph-mgr_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:04.759 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-12-05T17:29:04.768 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mgr (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:04.924 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-osd. 2020-12-05T17:29:04.937 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../050-ceph-osd_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:04.974 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-osd (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:05.020 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-werkzeug. 2020-12-05T17:29:05.035 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.1_all.deb ... 2020-12-05T17:29:05.043 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyasn1-modules. 2020-12-05T17:29:05.057 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2020-12-05T17:29:05.078 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2020-12-05T17:29:05.102 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2020-12-05T17:29:05.128 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-pastedeploy. 2020-12-05T17:29:05.143 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2020-12-05T17:29:05.186 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2020-12-05T17:29:05.222 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mon. 2020-12-05T17:29:05.236 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../049-ceph-mon_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:05.280 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mon (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:05.421 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rsa. 2020-12-05T17:29:05.423 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package ceph-mgr. 2020-12-05T17:29:05.436 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../073-python3-rsa_3.4.2-1_all.deb ... 2020-12-05T17:29:05.439 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../048-ceph-mgr_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:05.479 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rsa (3.4.2-1) ... 2020-12-05T17:29:05.481 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking ceph-mgr (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:05.540 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-webtest. 2020-12-05T17:29:05.553 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2020-12-05T17:29:05.589 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2020-12-05T17:29:05.807 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-sqlalchemy. 2020-12-05T17:29:05.821 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../074-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2020-12-05T17:29:05.865 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-12-05T17:29:05.967 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package ceph-mon. 2020-12-05T17:29:05.981 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../049-ceph-mon_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:05.992 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-pecan. 2020-12-05T17:29:06.007 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2020-12-05T17:29:06.033 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking ceph-mon (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:06.050 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-pecan (1.2.1-2) ... 2020-12-05T17:29:06.208 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-osd. 2020-12-05T17:29:06.222 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../050-ceph-osd_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:06.266 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-osd (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:06.447 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-werkzeug. 2020-12-05T17:29:06.460 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.1_all.deb ... 2020-12-05T17:29:06.505 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2020-12-05T17:29:06.736 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-oauth2client. 2020-12-05T17:29:06.752 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../075-python3-oauth2client_4.1.2-3_all.deb ... 2020-12-05T17:29:06.795 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2020-12-05T17:29:06.904 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package ceph-osd. 2020-12-05T17:29:06.919 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../050-ceph-osd_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:06.933 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-mgr. 2020-12-05T17:29:06.948 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../048-ceph-mgr_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:06.962 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking ceph-osd (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:07.014 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-mgr (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:07.029 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph. 2020-12-05T17:29:07.044 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../051-ceph_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:07.096 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:07.148 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-setuptools. 2020-12-05T17:29:07.163 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../076-python3-setuptools_39.0.1-2_all.deb ... 2020-12-05T17:29:07.206 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2020-12-05T17:29:07.415 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-fuse. 2020-12-05T17:29:07.431 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../052-ceph-fuse_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:07.473 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-fuse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:07.520 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-mon. 2020-12-05T17:29:07.535 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../049-ceph-mon_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:07.578 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-mon (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:07.610 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-websocket. 2020-12-05T17:29:07.624 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../077-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2020-12-05T17:29:07.668 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2020-12-05T17:29:07.860 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-mds. 2020-12-05T17:29:07.875 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../053-ceph-mds_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:07.918 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-mds (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:07.988 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-kubernetes. 2020-12-05T17:29:08.004 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../078-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2020-12-05T17:29:08.046 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2020-12-05T17:29:08.188 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph. 2020-12-05T17:29:08.202 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../051-ceph_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:08.238 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:08.454 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package cephadm. 2020-12-05T17:29:08.466 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2020-12-05T17:29:08.468 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../054-cephadm_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:08.480 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../079-ceph-mgr-k8sevents_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:08.513 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking cephadm (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:08.533 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-k8sevents (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:08.557 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-osd. 2020-12-05T17:29:08.573 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../050-ceph-osd_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:08.583 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-fuse. 2020-12-05T17:29:08.597 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../052-ceph-fuse_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:08.615 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-osd (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:08.766 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-fuse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:08.840 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-jinja2. 2020-12-05T17:29:08.855 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../055-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2020-12-05T17:29:08.884 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package ceph. 2020-12-05T17:29:08.885 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-json-pointer. 2020-12-05T17:29:08.897 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../051-ceph_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:08.898 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-12-05T17:29:08.902 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../080-python3-json-pointer_1.10-1_all.deb ... 2020-12-05T17:29:08.942 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking ceph (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:09.021 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-json-pointer (1.10-1) ... 2020-12-05T17:29:09.144 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mds. 2020-12-05T17:29:09.158 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../053-ceph-mds_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:09.202 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mds (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:09.218 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2020-12-05T17:29:09.232 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../056-ceph-mgr-cephadm_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:09.262 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package ceph-fuse. 2020-12-05T17:29:09.276 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../052-ceph-fuse_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:09.285 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-mgr-cephadm (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:09.320 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking ceph-fuse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:09.403 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jsonpatch. 2020-12-05T17:29:09.418 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../081-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2020-12-05T17:29:09.465 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-12-05T17:29:09.649 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-repoze.lru. 2020-12-05T17:29:09.665 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../057-python3-repoze.lru_0.7-1_all.deb ... 2020-12-05T17:29:09.680 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package cephadm. 2020-12-05T17:29:09.693 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../054-cephadm_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:09.698 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package ceph-mds. 2020-12-05T17:29:09.712 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../053-ceph-mds_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:09.713 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2020-12-05T17:29:09.730 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking cephadm (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:09.756 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking ceph-mds (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:09.818 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-rook. 2020-12-05T17:29:09.834 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../082-ceph-mgr-rook_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:09.876 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-rook (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:10.007 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-routes. 2020-12-05T17:29:10.023 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../058-python3-routes_2.4.1-1_all.deb ... 2020-12-05T17:29:10.049 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-jinja2. 2020-12-05T17:29:10.066 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../055-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2020-12-05T17:29:10.074 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-routes (2.4.1-1) ... 2020-12-05T17:29:10.108 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-12-05T17:29:10.213 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libonig4:amd64. 2020-12-05T17:29:10.227 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../083-libonig4_6.7.0-1_amd64.deb ... 2020-12-05T17:29:10.242 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package cephadm. 2020-12-05T17:29:10.256 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../054-cephadm_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:10.271 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-12-05T17:29:10.292 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking cephadm (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:10.401 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2020-12-05T17:29:10.416 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../059-ceph-mgr-dashboard_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:10.452 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2020-12-05T17:29:10.460 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-mgr-dashboard (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:10.466 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../056-ceph-mgr-cephadm_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:10.519 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mgr-cephadm (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:10.632 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libjq1:amd64. 2020-12-05T17:29:10.646 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../084-libjq1_1.5+dfsg-2_amd64.deb ... 2020-12-05T17:29:10.690 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-12-05T17:29:10.703 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-jinja2. 2020-12-05T17:29:10.717 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../055-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2020-12-05T17:29:10.751 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph. 2020-12-05T17:29:10.753 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-12-05T17:29:10.768 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../051-ceph_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:10.812 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:10.863 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-repoze.lru. 2020-12-05T17:29:10.877 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../057-python3-repoze.lru_0.7-1_all.deb ... 2020-12-05T17:29:10.921 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2020-12-05T17:29:11.010 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package jq. 2020-12-05T17:29:11.024 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../085-jq_1.5+dfsg-2_amd64.deb ... 2020-12-05T17:29:11.056 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2020-12-05T17:29:11.073 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../056-ceph-mgr-cephadm_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:11.101 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-12-05T17:29:11.114 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking ceph-mgr-cephadm (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:11.190 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-fuse. 2020-12-05T17:29:11.206 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../052-ceph-fuse_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:11.224 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-routes. 2020-12-05T17:29:11.240 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../058-python3-routes_2.4.1-1_all.deb ... 2020-12-05T17:29:11.248 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-fuse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:11.283 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-routes (2.4.1-1) ... 2020-12-05T17:29:11.387 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package socat. 2020-12-05T17:29:11.404 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../086-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-12-05T17:29:11.408 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2020-12-05T17:29:11.426 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../060-ceph-mgr-diskprediction-cloud_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:11.445 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-12-05T17:29:11.467 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:11.467 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-repoze.lru. 2020-12-05T17:29:11.481 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../057-python3-repoze.lru_0.7-1_all.deb ... 2020-12-05T17:29:11.517 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2020-12-05T17:29:11.594 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2020-12-05T17:29:11.609 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../059-ceph-mgr-dashboard_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:11.652 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mgr-dashboard (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:11.693 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-mds. 2020-12-05T17:29:11.707 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../053-ceph-mds_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:11.751 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-mds (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:11.786 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-numpy. 2020-12-05T17:29:11.800 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../061-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2020-12-05T17:29:11.812 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-routes. 2020-12-05T17:29:11.828 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../058-python3-routes_2.4.1-1_all.deb ... 2020-12-05T17:29:11.844 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2020-12-05T17:29:11.865 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-12-05T17:29:11.870 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-routes (2.4.1-1) ... 2020-12-05T17:29:11.882 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../087-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-12-05T17:29:11.924 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-12-05T17:29:12.156 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2020-12-05T17:29:12.172 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../059-ceph-mgr-dashboard_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:12.201 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package xmlstarlet. 2020-12-05T17:29:12.214 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking ceph-mgr-dashboard (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:12.218 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2_amd64.deb ... 2020-12-05T17:29:12.260 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-12-05T17:29:12.338 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package cephadm. 2020-12-05T17:29:12.351 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../054-cephadm_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:12.396 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking cephadm (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:12.567 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2020-12-05T17:29:12.581 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../060-ceph-mgr-diskprediction-cloud_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:12.601 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-decorator. 2020-12-05T17:29:12.605 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-test. 2020-12-05T17:29:12.616 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../062-python3-decorator_4.1.2-1_all.deb ... 2020-12-05T17:29:12.619 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../089-ceph-test_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:12.625 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:12.664 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-test (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:12.668 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-decorator (4.1.2-1) ... 2020-12-05T17:29:12.765 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-jinja2. 2020-12-05T17:29:12.780 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../055-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2020-12-05T17:29:12.824 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-12-05T17:29:12.953 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-numpy. 2020-12-05T17:29:12.970 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../061-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2020-12-05T17:29:12.987 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-scipy. 2020-12-05T17:29:13.003 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../063-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2020-12-05T17:29:13.011 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2020-12-05T17:29:13.046 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2020-12-05T17:29:13.129 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2020-12-05T17:29:13.144 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../060-ceph-mgr-diskprediction-cloud_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:13.185 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2020-12-05T17:29:13.187 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:13.201 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../056-ceph-mgr-cephadm_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:13.243 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-mgr-cephadm (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:13.524 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-numpy. 2020-12-05T17:29:13.539 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../061-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2020-12-05T17:29:13.590 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2020-12-05T17:29:13.617 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-decorator. 2020-12-05T17:29:13.629 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-repoze.lru. 2020-12-05T17:29:13.632 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../062-python3-decorator_4.1.2-1_all.deb ... 2020-12-05T17:29:13.646 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../057-python3-repoze.lru_0.7-1_all.deb ... 2020-12-05T17:29:13.684 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-decorator (4.1.2-1) ... 2020-12-05T17:29:13.688 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2020-12-05T17:29:13.995 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-scipy. 2020-12-05T17:29:14.009 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../063-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2020-12-05T17:29:14.041 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-routes. 2020-12-05T17:29:14.053 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2020-12-05T17:29:14.055 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../058-python3-routes_2.4.1-1_all.deb ... 2020-12-05T17:29:14.099 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-routes (2.4.1-1) ... 2020-12-05T17:29:14.246 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-decorator. 2020-12-05T17:29:14.260 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../062-python3-decorator_4.1.2-1_all.deb ... 2020-12-05T17:29:14.304 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-decorator (4.1.2-1) ... 2020-12-05T17:29:14.354 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-sklearn-lib. 2020-12-05T17:29:14.369 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2020-12-05T17:29:14.412 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2020-12-05T17:29:14.468 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2020-12-05T17:29:14.483 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../059-ceph-mgr-dashboard_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:14.527 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-mgr-dashboard (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:14.624 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-scipy. 2020-12-05T17:29:14.639 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../063-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2020-12-05T17:29:14.691 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2020-12-05T17:29:14.866 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-joblib. 2020-12-05T17:29:14.881 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../065-python3-joblib_0.11-1_all.deb ... 2020-12-05T17:29:14.924 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-joblib (0.11-1) ... 2020-12-05T17:29:15.286 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-sklearn. 2020-12-05T17:29:15.300 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../066-python3-sklearn_0.19.1-3_all.deb ... 2020-12-05T17:29:15.320 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-sklearn-lib. 2020-12-05T17:29:15.335 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2020-12-05T17:29:15.344 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2020-12-05T17:29:15.378 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2020-12-05T17:29:15.534 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2020-12-05T17:29:15.549 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../060-ceph-mgr-diskprediction-cloud_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:15.592 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:15.859 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-joblib. 2020-12-05T17:29:15.868 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../065-python3-joblib_0.11-1_all.deb ... 2020-12-05T17:29:15.900 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2020-12-05T17:29:15.914 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:15.928 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-numpy. 2020-12-05T17:29:15.945 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../061-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2020-12-05T17:29:15.958 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:15.982 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2020-12-05T17:29:16.050 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-colorama. 2020-12-05T17:29:16.064 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../090-python3-colorama_0.3.7-1_all.deb ... 2020-12-05T17:29:16.117 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-colorama (0.3.7-1) ... 2020-12-05T17:29:16.273 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-joblib (0.11-1) ... 2020-12-05T17:29:16.341 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-sklearn-lib. 2020-12-05T17:29:16.355 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2020-12-05T17:29:16.399 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2020-12-05T17:29:16.528 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package cephfs-shell. 2020-12-05T17:29:16.545 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../091-cephfs-shell_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:16.586 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking cephfs-shell (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:16.593 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-sklearn. 2020-12-05T17:29:16.609 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../066-python3-sklearn_0.19.1-3_all.deb ... 2020-12-05T17:29:16.651 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2020-12-05T17:29:16.688 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-decorator. 2020-12-05T17:29:16.704 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../062-python3-decorator_4.1.2-1_all.deb ... 2020-12-05T17:29:16.780 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-decorator (4.1.2-1) ... 2020-12-05T17:29:16.822 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libcephfs-dev. 2020-12-05T17:29:16.836 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../092-libcephfs-dev_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:16.880 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libcephfs-dev (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:16.886 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-joblib. 2020-12-05T17:29:16.900 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../065-python3-joblib_0.11-1_all.deb ... 2020-12-05T17:29:16.952 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-joblib (0.11-1) ... 2020-12-05T17:29:17.105 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-monotonic. 2020-12-05T17:29:17.120 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../068-python3-monotonic_1.1-2_all.deb ... 2020-12-05T17:29:17.166 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-scipy. 2020-12-05T17:29:17.172 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-monotonic (1.1-2) ... 2020-12-05T17:29:17.182 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../063-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2020-12-05T17:29:17.191 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package nvme-cli. 2020-12-05T17:29:17.207 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../093-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2020-12-05T17:29:17.224 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2020-12-05T17:29:17.232 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2020-12-05T17:29:17.249 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2020-12-05T17:29:17.250 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:17.282 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:17.303 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-sklearn. 2020-12-05T17:29:17.317 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../066-python3-sklearn_0.19.1-3_all.deb ... 2020-12-05T17:29:17.361 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2020-12-05T17:29:17.637 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-attr. 2020-12-05T17:29:17.654 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../094-python-attr_17.4.0-2_all.deb ... 2020-12-05T17:29:17.704 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-attr (17.4.0-2) ... 2020-12-05T17:29:17.928 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2020-12-05T17:29:17.945 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:17.949 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-fasteners. 2020-12-05T17:29:17.964 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../069-python3-fasteners_0.12.0-3_all.deb ... 2020-12-05T17:29:17.995 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:18.008 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2020-12-05T17:29:18.034 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-funcsigs. 2020-12-05T17:29:18.050 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../095-python-funcsigs_1.0.2-4_all.deb ... 2020-12-05T17:29:18.090 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2020-12-05T17:29:18.376 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-pluggy. 2020-12-05T17:29:18.377 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-httplib2. 2020-12-05T17:29:18.390 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../096-python-pluggy_0.6.0-1_all.deb ... 2020-12-05T17:29:18.392 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../070-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2020-12-05T17:29:18.421 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-monotonic. 2020-12-05T17:29:18.434 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-pluggy (0.6.0-1) ... 2020-12-05T17:29:18.435 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../068-python3-monotonic_1.1-2_all.deb ... 2020-12-05T17:29:18.443 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-12-05T17:29:18.475 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-sklearn-lib. 2020-12-05T17:29:18.479 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-monotonic (1.1-2) ... 2020-12-05T17:29:18.491 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2020-12-05T17:29:18.550 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2020-12-05T17:29:18.765 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-fasteners. 2020-12-05T17:29:18.772 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-py. 2020-12-05T17:29:18.779 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-pyasn1. 2020-12-05T17:29:18.782 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../069-python3-fasteners_0.12.0-3_all.deb ... 2020-12-05T17:29:18.789 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../097-python-py_1.5.2-1_all.deb ... 2020-12-05T17:29:18.794 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3_all.deb ... 2020-12-05T17:29:18.823 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2020-12-05T17:29:18.839 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-py (1.5.2-1) ... 2020-12-05T17:29:18.846 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2020-12-05T17:29:19.080 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-joblib. 2020-12-05T17:29:19.095 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../065-python3-joblib_0.11-1_all.deb ... 2020-12-05T17:29:19.123 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-monotonic. 2020-12-05T17:29:19.137 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../068-python3-monotonic_1.1-2_all.deb ... 2020-12-05T17:29:19.153 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-httplib2. 2020-12-05T17:29:19.164 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-joblib (0.11-1) ... 2020-12-05T17:29:19.167 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../070-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2020-12-05T17:29:19.191 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-pyasn1-modules. 2020-12-05T17:29:19.205 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2020-12-05T17:29:19.206 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-monotonic (1.1-2) ... 2020-12-05T17:29:19.219 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-12-05T17:29:19.249 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2020-12-05T17:29:19.250 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-pytest. 2020-12-05T17:29:19.265 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../098-python-pytest_3.3.2-2_all.deb ... 2020-12-05T17:29:19.309 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-pytest (3.3.2-2) ... 2020-12-05T17:29:19.522 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pyasn1. 2020-12-05T17:29:19.532 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-sklearn. 2020-12-05T17:29:19.537 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-fasteners. 2020-12-05T17:29:19.539 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3_all.deb ... 2020-12-05T17:29:19.543 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-rsa. 2020-12-05T17:29:19.547 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../066-python3-sklearn_0.19.1-3_all.deb ... 2020-12-05T17:29:19.551 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../069-python3-fasteners_0.12.0-3_all.deb ... 2020-12-05T17:29:19.559 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../073-python3-rsa_3.4.2-1_all.deb ... 2020-12-05T17:29:19.581 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2020-12-05T17:29:19.595 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2020-12-05T17:29:19.602 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-rsa (3.4.2-1) ... 2020-12-05T17:29:19.615 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2020-12-05T17:29:19.663 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-attr. 2020-12-05T17:29:19.678 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../099-python3-attr_17.4.0-2_all.deb ... 2020-12-05T17:29:19.728 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-attr (17.4.0-2) ... 2020-12-05T17:29:19.909 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pyasn1-modules. 2020-12-05T17:29:19.923 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-httplib2. 2020-12-05T17:29:19.923 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2020-12-05T17:29:19.938 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../070-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2020-12-05T17:29:19.968 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2020-12-05T17:29:19.971 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-sqlalchemy. 2020-12-05T17:29:19.981 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-12-05T17:29:19.989 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../074-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2020-12-05T17:29:20.030 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-12-05T17:29:20.040 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cycler. 2020-12-05T17:29:20.054 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../100-python3-cycler_0.10.0-1_all.deb ... 2020-12-05T17:29:20.123 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cycler (0.10.0-1) ... 2020-12-05T17:29:20.267 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-pyasn1. 2020-12-05T17:29:20.283 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3_all.deb ... 2020-12-05T17:29:20.315 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2020-12-05T17:29:20.325 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2020-12-05T17:29:20.329 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:20.329 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-rsa. 2020-12-05T17:29:20.343 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../073-python3-rsa_3.4.2-1_all.deb ... 2020-12-05T17:29:20.373 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:20.395 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-rsa (3.4.2-1) ... 2020-12-05T17:29:20.434 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-webencodings. 2020-12-05T17:29:20.450 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../101-python3-webencodings_0.5-2_all.deb ... 2020-12-05T17:29:20.484 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-oauth2client. 2020-12-05T17:29:20.492 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-webencodings (0.5-2) ... 2020-12-05T17:29:20.499 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../075-python3-oauth2client_4.1.2-3_all.deb ... 2020-12-05T17:29:20.542 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2020-12-05T17:29:20.696 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-pyasn1-modules. 2020-12-05T17:29:20.698 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-sqlalchemy. 2020-12-05T17:29:20.710 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2020-12-05T17:29:20.713 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../074-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2020-12-05T17:29:20.754 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2020-12-05T17:29:20.757 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-12-05T17:29:20.820 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-html5lib. 2020-12-05T17:29:20.836 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../102-python3-html5lib_0.999999999-1_all.deb ... 2020-12-05T17:29:20.878 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2020-12-05T17:29:20.887 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-setuptools. 2020-12-05T17:29:20.901 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../076-python3-setuptools_39.0.1-2_all.deb ... 2020-12-05T17:29:20.945 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2020-12-05T17:29:21.048 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-rsa. 2020-12-05T17:29:21.065 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../073-python3-rsa_3.4.2-1_all.deb ... 2020-12-05T17:29:21.107 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-rsa (3.4.2-1) ... 2020-12-05T17:29:21.228 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-oauth2client. 2020-12-05T17:29:21.240 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-lxml:amd64. 2020-12-05T17:29:21.242 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../075-python3-oauth2client_4.1.2-3_all.deb ... 2020-12-05T17:29:21.257 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../103-python3-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-12-05T17:29:21.286 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2020-12-05T17:29:21.298 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-12-05T17:29:21.357 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-websocket. 2020-12-05T17:29:21.372 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../077-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2020-12-05T17:29:21.415 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2020-12-05T17:29:21.418 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-sqlalchemy. 2020-12-05T17:29:21.433 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../074-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2020-12-05T17:29:21.476 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-12-05T17:29:21.518 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-monotonic. 2020-12-05T17:29:21.532 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../068-python3-monotonic_1.1-2_all.deb ... 2020-12-05T17:29:21.604 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-monotonic (1.1-2) ... 2020-12-05T17:29:21.640 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-setuptools. 2020-12-05T17:29:21.656 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../076-python3-setuptools_39.0.1-2_all.deb ... 2020-12-05T17:29:21.698 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2020-12-05T17:29:21.743 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-tz. 2020-12-05T17:29:21.752 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-kubernetes. 2020-12-05T17:29:21.759 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../104-python3-tz_2018.3-2_all.deb ... 2020-12-05T17:29:21.769 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../078-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2020-12-05T17:29:21.801 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-tz (2018.3-2) ... 2020-12-05T17:29:21.810 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2020-12-05T17:29:21.923 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-fasteners. 2020-12-05T17:29:21.937 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../069-python3-fasteners_0.12.0-3_all.deb ... 2020-12-05T17:29:21.964 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-oauth2client. 2020-12-05T17:29:21.981 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2020-12-05T17:29:21.981 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../075-python3-oauth2client_4.1.2-3_all.deb ... 2020-12-05T17:29:22.022 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2020-12-05T17:29:22.093 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-websocket. 2020-12-05T17:29:22.095 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-matplotlib. 2020-12-05T17:29:22.109 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../077-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2020-12-05T17:29:22.113 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../105-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2020-12-05T17:29:22.152 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2020-12-05T17:29:22.171 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2020-12-05T17:29:22.248 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2020-12-05T17:29:22.265 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../079-ceph-mgr-k8sevents_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:22.306 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-mgr-k8sevents (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:22.367 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-httplib2. 2020-12-05T17:29:22.383 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../070-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2020-12-05T17:29:22.400 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-setuptools. 2020-12-05T17:29:22.416 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../076-python3-setuptools_39.0.1-2_all.deb ... 2020-12-05T17:29:22.450 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-12-05T17:29:22.458 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2020-12-05T17:29:22.506 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-kubernetes. 2020-12-05T17:29:22.521 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../078-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2020-12-05T17:29:22.564 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2020-12-05T17:29:22.642 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-json-pointer. 2020-12-05T17:29:22.656 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../080-python3-json-pointer_1.10-1_all.deb ... 2020-12-05T17:29:22.786 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-json-pointer (1.10-1) ... 2020-12-05T17:29:22.786 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-pyasn1. 2020-12-05T17:29:22.800 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3_all.deb ... 2020-12-05T17:29:22.870 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-olefile. 2020-12-05T17:29:22.871 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2020-12-05T17:29:22.871 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-websocket. 2020-12-05T17:29:22.883 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../106-python3-olefile_0.45.1-1_all.deb ... 2020-12-05T17:29:22.885 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../077-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2020-12-05T17:29:22.927 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-olefile (0.45.1-1) ... 2020-12-05T17:29:22.929 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2020-12-05T17:29:22.994 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2020-12-05T17:29:23.009 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../079-ceph-mgr-k8sevents_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:23.060 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mgr-k8sevents (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:23.198 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-pyasn1-modules. 2020-12-05T17:29:23.205 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-jsonpatch. 2020-12-05T17:29:23.212 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2020-12-05T17:29:23.222 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../081-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2020-12-05T17:29:23.257 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-kubernetes. 2020-12-05T17:29:23.267 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-12-05T17:29:23.272 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../078-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2020-12-05T17:29:23.280 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pastescript. 2020-12-05T17:29:23.282 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2020-12-05T17:29:23.296 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-2_all.deb ... 2020-12-05T17:29:23.315 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2020-12-05T17:29:23.338 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2020-12-05T17:29:23.379 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-json-pointer. 2020-12-05T17:29:23.396 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../080-python3-json-pointer_1.10-1_all.deb ... 2020-12-05T17:29:23.540 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-json-pointer (1.10-1) ... 2020-12-05T17:29:23.579 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-mgr-rook. 2020-12-05T17:29:23.593 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-rsa. 2020-12-05T17:29:23.593 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../082-ceph-mgr-rook_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:23.609 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../073-python3-rsa_3.4.2-1_all.deb ... 2020-12-05T17:29:23.631 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-mgr-rook (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:23.651 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-rsa (3.4.2-1) ... 2020-12-05T17:29:23.675 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pil:amd64. 2020-12-05T17:29:23.692 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../108-python3-pil_5.1.0-1ubuntu0.3_amd64.deb ... 2020-12-05T17:29:23.711 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2020-12-05T17:29:23.727 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../079-ceph-mgr-k8sevents_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:23.733 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-12-05T17:29:23.769 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking ceph-mgr-k8sevents (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:23.902 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-jsonpatch. 2020-12-05T17:29:23.917 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../081-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2020-12-05T17:29:23.963 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-12-05T17:29:24.012 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libonig4:amd64. 2020-12-05T17:29:24.029 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-sqlalchemy. 2020-12-05T17:29:24.030 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../083-libonig4_6.7.0-1_amd64.deb ... 2020-12-05T17:29:24.042 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../074-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2020-12-05T17:29:24.077 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-12-05T17:29:24.079 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-12-05T17:29:24.089 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-json-pointer. 2020-12-05T17:29:24.104 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pluggy. 2020-12-05T17:29:24.104 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../080-python3-json-pointer_1.10-1_all.deb ... 2020-12-05T17:29:24.121 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../109-python3-pluggy_0.6.0-1_all.deb ... 2020-12-05T17:29:24.181 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2020-12-05T17:29:24.233 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-json-pointer (1.10-1) ... 2020-12-05T17:29:24.265 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mgr-rook. 2020-12-05T17:29:24.277 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../082-ceph-mgr-rook_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:24.322 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mgr-rook (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:24.438 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libjq1:amd64. 2020-12-05T17:29:24.462 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../084-libjq1_1.5+dfsg-2_amd64.deb ... 2020-12-05T17:29:24.497 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-12-05T17:29:24.540 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-py. 2020-12-05T17:29:24.556 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../110-python3-py_1.5.2-1_all.deb ... 2020-12-05T17:29:24.606 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-py (1.5.2-1) ... 2020-12-05T17:29:24.607 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-jsonpatch. 2020-12-05T17:29:24.617 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-oauth2client. 2020-12-05T17:29:24.620 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../081-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2020-12-05T17:29:24.634 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../075-python3-oauth2client_4.1.2-3_all.deb ... 2020-12-05T17:29:24.658 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libonig4:amd64. 2020-12-05T17:29:24.668 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-12-05T17:29:24.673 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../083-libonig4_6.7.0-1_amd64.deb ... 2020-12-05T17:29:24.675 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2020-12-05T17:29:24.725 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-12-05T17:29:24.766 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package jq. 2020-12-05T17:29:24.782 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../085-jq_1.5+dfsg-2_amd64.deb ... 2020-12-05T17:29:24.824 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-12-05T17:29:24.939 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package ceph-mgr-rook. 2020-12-05T17:29:24.951 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyinotify. 2020-12-05T17:29:24.952 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../082-ceph-mgr-rook_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:24.968 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1_all.deb ... 2020-12-05T17:29:24.989 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking ceph-mgr-rook (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:25.009 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2020-12-05T17:29:25.078 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-setuptools. 2020-12-05T17:29:25.086 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libjq1:amd64. 2020-12-05T17:29:25.092 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../076-python3-setuptools_39.0.1-2_all.deb ... 2020-12-05T17:29:25.094 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package socat. 2020-12-05T17:29:25.101 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../084-libjq1_1.5+dfsg-2_amd64.deb ... 2020-12-05T17:29:25.108 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../086-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-12-05T17:29:25.137 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2020-12-05T17:29:25.144 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-12-05T17:29:25.152 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-12-05T17:29:25.316 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package libonig4:amd64. 2020-12-05T17:29:25.332 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../083-libonig4_6.7.0-1_amd64.deb ... 2020-12-05T17:29:25.354 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pytest. 2020-12-05T17:29:25.368 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../112-python3-pytest_3.3.2-2_all.deb ... 2020-12-05T17:29:25.374 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-12-05T17:29:25.412 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pytest (3.3.2-2) ... 2020-12-05T17:29:25.439 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package jq. 2020-12-05T17:29:25.455 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../085-jq_1.5+dfsg-2_amd64.deb ... 2020-12-05T17:29:25.497 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-12-05T17:29:25.582 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-websocket. 2020-12-05T17:29:25.596 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../077-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2020-12-05T17:29:25.614 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-12-05T17:29:25.628 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../087-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-12-05T17:29:25.659 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2020-12-05T17:29:25.672 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-12-05T17:29:25.750 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-simplejson. 2020-12-05T17:29:25.753 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package libjq1:amd64. 2020-12-05T17:29:25.764 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../113-python3-simplejson_3.13.2-1_amd64.deb ... 2020-12-05T17:29:25.769 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../084-libjq1_1.5+dfsg-2_amd64.deb ... 2020-12-05T17:29:25.791 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package socat. 2020-12-05T17:29:25.806 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../086-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-12-05T17:29:25.807 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2020-12-05T17:29:25.811 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-12-05T17:29:25.859 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-12-05T17:29:25.941 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package xmlstarlet. 2020-12-05T17:29:25.956 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2_amd64.deb ... 2020-12-05T17:29:25.985 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-kubernetes. 2020-12-05T17:29:26.000 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-12-05T17:29:26.002 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../078-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2020-12-05T17:29:26.043 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2020-12-05T17:29:26.080 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package jq. 2020-12-05T17:29:26.095 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../085-jq_1.5+dfsg-2_amd64.deb ... 2020-12-05T17:29:26.139 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-12-05T17:29:26.152 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-tk:amd64. 2020-12-05T17:29:26.166 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../114-python3-tk_3.6.9-1~18.04_amd64.deb ... 2020-12-05T17:29:26.226 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2020-12-05T17:29:26.246 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-12-05T17:29:26.262 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../087-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-12-05T17:29:26.304 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-12-05T17:29:26.354 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-test. 2020-12-05T17:29:26.371 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../089-ceph-test_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:26.374 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package socat. 2020-12-05T17:29:26.390 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../086-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-12-05T17:29:26.412 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-test (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:26.433 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-12-05T17:29:26.539 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2020-12-05T17:29:26.554 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../079-ceph-mgr-k8sevents_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:26.582 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package xmlstarlet. 2020-12-05T17:29:26.597 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-mgr-k8sevents (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:26.598 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2_amd64.deb ... 2020-12-05T17:29:26.640 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-12-05T17:29:26.689 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package radosgw. 2020-12-05T17:29:26.705 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../115-radosgw_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:26.755 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking radosgw (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:26.820 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-12-05T17:29:26.835 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../087-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-12-05T17:29:26.878 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-12-05T17:29:26.975 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-json-pointer. 2020-12-05T17:29:26.989 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../080-python3-json-pointer_1.10-1_all.deb ... 2020-12-05T17:29:27.003 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-test. 2020-12-05T17:29:27.017 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../089-ceph-test_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:27.061 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-test (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:27.164 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-json-pointer (1.10-1) ... 2020-12-05T17:29:27.181 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package xmlstarlet. 2020-12-05T17:29:27.195 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2_amd64.deb ... 2020-12-05T17:29:27.240 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-12-05T17:29:27.548 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-jsonpatch. 2020-12-05T17:29:27.563 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../081-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2020-12-05T17:29:27.585 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package ceph-test. 2020-12-05T17:29:27.599 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../089-ceph-test_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:27.643 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking ceph-test (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:27.750 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package rbd-fuse. 2020-12-05T17:29:27.764 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../116-rbd-fuse_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:27.808 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking rbd-fuse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:28.034 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-12-05T17:29:28.104 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-logutils (0.3.3-5) ... 2020-12-05T17:29:28.367 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-mgr-rook. 2020-12-05T17:29:28.381 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../082-ceph-mgr-rook_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:28.430 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2020-12-05T17:29:28.449 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-mgr-rook (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:28.806 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2020-12-05T17:29:28.810 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libonig4:amd64. 2020-12-05T17:29:28.828 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../083-libonig4_6.7.0-1_amd64.deb ... 2020-12-05T17:29:28.886 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-12-05T17:29:28.932 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-12-05T17:29:29.305 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libjq1:amd64. 2020-12-05T17:29:29.320 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../084-libjq1_1.5+dfsg-2_amd64.deb ... 2020-12-05T17:29:29.363 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-12-05T17:29:29.576 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-12-05T17:29:29.658 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package jq. 2020-12-05T17:29:29.673 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../085-jq_1.5+dfsg-2_amd64.deb ... 2020-12-05T17:29:29.741 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-12-05T17:29:29.757 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-colorama. 2020-12-05T17:29:29.772 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../090-python3-colorama_0.3.7-1_all.deb ... 2020-12-05T17:29:29.815 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-colorama (0.3.7-1) ... 2020-12-05T17:29:29.977 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package socat. 2020-12-05T17:29:29.986 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2020-12-05T17:29:29.994 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../086-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-12-05T17:29:30.027 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-12-05T17:29:30.068 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package cephfs-shell. 2020-12-05T17:29:30.082 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../091-cephfs-shell_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:30.126 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking cephfs-shell (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:30.287 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-webencodings (0.5-2) ... 2020-12-05T17:29:30.422 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-12-05T17:29:30.437 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../087-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-12-05T17:29:30.437 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libcephfs-dev. 2020-12-05T17:29:30.449 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-colorama. 2020-12-05T17:29:30.453 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../092-libcephfs-dev_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:30.464 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../090-python3-colorama_0.3.7-1_all.deb ... 2020-12-05T17:29:30.480 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-12-05T17:29:30.495 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libcephfs-dev (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:30.498 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-colorama (0.3.7-1) ... 2020-12-05T17:29:30.614 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-attr (17.4.0-2) ... 2020-12-05T17:29:30.726 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package cephfs-shell. 2020-12-05T17:29:30.739 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../091-cephfs-shell_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:30.773 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package nvme-cli. 2020-12-05T17:29:30.784 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking cephfs-shell (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:30.788 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../093-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2020-12-05T17:29:30.815 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2020-12-05T17:29:30.817 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package xmlstarlet. 2020-12-05T17:29:30.831 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2020-12-05T17:29:30.832 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2_amd64.deb ... 2020-12-05T17:29:30.875 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-12-05T17:29:30.941 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-py (1.5.2-1) ... 2020-12-05T17:29:31.079 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libcephfs-dev. 2020-12-05T17:29:31.095 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../092-libcephfs-dev_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:31.133 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-colorama. 2020-12-05T17:29:31.137 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libcephfs-dev (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:31.148 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../090-python3-colorama_0.3.7-1_all.deb ... 2020-12-05T17:29:31.178 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python-attr. 2020-12-05T17:29:31.188 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-colorama (0.3.7-1) ... 2020-12-05T17:29:31.194 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../094-python-attr_17.4.0-2_all.deb ... 2020-12-05T17:29:31.227 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python-attr (17.4.0-2) ... 2020-12-05T17:29:31.263 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-test. 2020-12-05T17:29:31.279 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../089-ceph-test_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:31.284 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-waitress (1.0.1-1) ... 2020-12-05T17:29:31.321 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-test (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:31.398 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package nvme-cli. 2020-12-05T17:29:31.411 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../093-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2020-12-05T17:29:31.441 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package cephfs-shell. 2020-12-05T17:29:31.447 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2020-12-05T17:29:31.455 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../091-cephfs-shell_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:31.466 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-12-05T17:29:31.499 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking cephfs-shell (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:31.505 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python-funcsigs. 2020-12-05T17:29:31.520 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../095-python-funcsigs_1.0.2-4_all.deb ... 2020-12-05T17:29:31.563 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2020-12-05T17:29:31.719 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package libcephfs-dev. 2020-12-05T17:29:31.736 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../092-libcephfs-dev_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:31.761 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python-attr. 2020-12-05T17:29:31.762 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-olefile (0.45.1-1) ... 2020-12-05T17:29:31.768 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking libcephfs-dev (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:31.775 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../094-python-attr_17.4.0-2_all.deb ... 2020-12-05T17:29:31.820 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python-attr (17.4.0-2) ... 2020-12-05T17:29:31.899 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python-pluggy. 2020-12-05T17:29:31.915 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../096-python-pluggy_0.6.0-1_all.deb ... 2020-12-05T17:29:31.958 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python-pluggy (0.6.0-1) ... 2020-12-05T17:29:32.013 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package nvme-cli. 2020-12-05T17:29:32.029 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../093-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2020-12-05T17:29:32.063 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2020-12-05T17:29:32.097 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-ceph-argparse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:32.139 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python-funcsigs. 2020-12-05T17:29:32.156 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../095-python-funcsigs_1.0.2-4_all.deb ... 2020-12-05T17:29:32.197 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2020-12-05T17:29:32.218 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python-py. 2020-12-05T17:29:32.235 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../097-python-py_1.5.2-1_all.deb ... 2020-12-05T17:29:32.268 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python-py (1.5.2-1) ... 2020-12-05T17:29:32.406 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-tempita (0.5.2-2) ... 2020-12-05T17:29:32.443 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python-attr. 2020-12-05T17:29:32.460 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../094-python-attr_17.4.0-2_all.deb ... 2020-12-05T17:29:32.501 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python-attr (17.4.0-2) ... 2020-12-05T17:29:32.517 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python-pluggy. 2020-12-05T17:29:32.534 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../096-python-pluggy_0.6.0-1_all.deb ... 2020-12-05T17:29:32.563 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python-pytest. 2020-12-05T17:29:32.576 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python-pluggy (0.6.0-1) ... 2020-12-05T17:29:32.577 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../098-python-pytest_3.3.2-2_all.deb ... 2020-12-05T17:29:32.613 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python-pytest (3.3.2-2) ... 2020-12-05T17:29:32.716 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-12-05T17:29:32.779 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python-funcsigs. 2020-12-05T17:29:32.796 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../095-python-funcsigs_1.0.2-4_all.deb ... 2020-12-05T17:29:32.828 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2020-12-05T17:29:32.842 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2020-12-05T17:29:32.895 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python-py. 2020-12-05T17:29:32.910 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../097-python-py_1.5.2-1_all.deb ... 2020-12-05T17:29:32.950 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-attr. 2020-12-05T17:29:32.953 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python-py (1.5.2-1) ... 2020-12-05T17:29:32.964 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../099-python3-attr_17.4.0-2_all.deb ... 2020-12-05T17:29:33.000 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-attr (17.4.0-2) ... 2020-12-05T17:29:33.073 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python-pluggy. 2020-12-05T17:29:33.088 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../096-python-pluggy_0.6.0-1_all.deb ... 2020-12-05T17:29:33.123 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python-pluggy (0.6.0-1) ... 2020-12-05T17:29:33.244 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-cycler. 2020-12-05T17:29:33.263 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../100-python3-cycler_0.10.0-1_all.deb ... 2020-12-05T17:29:33.265 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python-pytest. 2020-12-05T17:29:33.282 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../098-python-pytest_3.3.2-2_all.deb ... 2020-12-05T17:29:33.294 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-cycler (0.10.0-1) ... 2020-12-05T17:29:33.323 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python-pytest (3.3.2-2) ... 2020-12-05T17:29:33.392 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python-py. 2020-12-05T17:29:33.408 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../097-python-py_1.5.2-1_all.deb ... 2020-12-05T17:29:33.451 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python-py (1.5.2-1) ... 2020-12-05T17:29:33.580 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-webencodings. 2020-12-05T17:29:33.597 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../101-python3-webencodings_0.5-2_all.deb ... 2020-12-05T17:29:33.638 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-webencodings (0.5-2) ... 2020-12-05T17:29:33.677 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-attr. 2020-12-05T17:29:33.694 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../099-python3-attr_17.4.0-2_all.deb ... 2020-12-05T17:29:33.735 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-attr (17.4.0-2) ... 2020-12-05T17:29:33.796 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python-pytest. 2020-12-05T17:29:33.811 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../098-python-pytest_3.3.2-2_all.deb ... 2020-12-05T17:29:33.820 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2020-12-05T17:29:33.854 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python-pytest (3.3.2-2) ... 2020-12-05T17:29:33.950 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-html5lib. 2020-12-05T17:29:33.964 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../102-python3-html5lib_0.999999999-1_all.deb ... 2020-12-05T17:29:34.024 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2020-12-05T17:29:34.038 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-cycler. 2020-12-05T17:29:34.052 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../100-python3-cycler_0.10.0-1_all.deb ... 2020-12-05T17:29:34.096 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-cycler (0.10.0-1) ... 2020-12-05T17:29:34.163 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2020-12-05T17:29:34.215 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-attr. 2020-12-05T17:29:34.231 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../099-python3-attr_17.4.0-2_all.deb ... 2020-12-05T17:29:34.274 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-attr (17.4.0-2) ... 2020-12-05T17:29:34.395 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-lxml:amd64. 2020-12-05T17:29:34.411 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../103-python3-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-12-05T17:29:34.441 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-webencodings. 2020-12-05T17:29:34.453 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-12-05T17:29:34.455 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-py (1.5.2-1) ... 2020-12-05T17:29:34.458 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../101-python3-webencodings_0.5-2_all.deb ... 2020-12-05T17:29:34.508 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-webencodings (0.5-2) ... 2020-12-05T17:29:34.560 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-cycler. 2020-12-05T17:29:34.577 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../100-python3-cycler_0.10.0-1_all.deb ... 2020-12-05T17:29:34.618 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-cycler (0.10.0-1) ... 2020-12-05T17:29:34.682 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-simplejson (3.13.2-1) ... 2020-12-05T17:29:34.708 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-colorama. 2020-12-05T17:29:34.723 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../090-python3-colorama_0.3.7-1_all.deb ... 2020-12-05T17:29:34.757 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-colorama (0.3.7-1) ... 2020-12-05T17:29:34.820 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-html5lib. 2020-12-05T17:29:34.838 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../102-python3-html5lib_0.999999999-1_all.deb ... 2020-12-05T17:29:34.878 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2020-12-05T17:29:34.882 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-tz. 2020-12-05T17:29:34.897 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../104-python3-tz_2018.3-2_all.deb ... 2020-12-05T17:29:34.948 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-tz (2018.3-2) ... 2020-12-05T17:29:34.954 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-webencodings. 2020-12-05T17:29:34.969 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../101-python3-webencodings_0.5-2_all.deb ... 2020-12-05T17:29:34.993 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package cephfs-shell. 2020-12-05T17:29:35.008 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../091-cephfs-shell_15.2.6-607-g3b1861ac-1bionic_all.deb ... 2020-12-05T17:29:35.013 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-webencodings (0.5-2) ... 2020-12-05T17:29:35.025 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-json-pointer (1.10-1) ... 2020-12-05T17:29:35.043 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking cephfs-shell (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:35.114 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2020-12-05T17:29:35.232 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-lxml:amd64. 2020-12-05T17:29:35.250 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../103-python3-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-12-05T17:29:35.260 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-matplotlib. 2020-12-05T17:29:35.277 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../105-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2020-12-05T17:29:35.290 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-12-05T17:29:35.318 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2020-12-05T17:29:35.321 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libcephfs-dev. 2020-12-05T17:29:35.335 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../092-libcephfs-dev_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:35.349 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-html5lib. 2020-12-05T17:29:35.360 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-12-05T17:29:35.364 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../102-python3-html5lib_0.999999999-1_all.deb ... 2020-12-05T17:29:35.379 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libcephfs-dev (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:35.415 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2020-12-05T17:29:35.511 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-pluggy (0.6.0-1) ... 2020-12-05T17:29:35.681 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package nvme-cli. 2020-12-05T17:29:35.696 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../093-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2020-12-05T17:29:35.703 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-colorama (0.3.7-1) ... 2020-12-05T17:29:35.756 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2020-12-05T17:29:35.769 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-lxml:amd64. 2020-12-05T17:29:35.785 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../103-python3-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-12-05T17:29:35.802 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-tz. 2020-12-05T17:29:35.819 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../104-python3-tz_2018.3-2_all.deb ... 2020-12-05T17:29:35.826 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-12-05T17:29:35.860 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-tz (2018.3-2) ... 2020-12-05T17:29:35.982 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-olefile. 2020-12-05T17:29:35.997 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../106-python3-olefile_0.45.1-1_all.deb ... 2020-12-05T17:29:36.021 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2020-12-05T17:29:36.040 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-olefile (0.45.1-1) ... 2020-12-05T17:29:36.153 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python-attr. 2020-12-05T17:29:36.170 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../094-python-attr_17.4.0-2_all.deb ... 2020-12-05T17:29:36.181 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-matplotlib. 2020-12-05T17:29:36.196 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../105-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2020-12-05T17:29:36.212 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python-attr (17.4.0-2) ... 2020-12-05T17:29:36.239 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2020-12-05T17:29:36.322 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2020-12-05T17:29:36.338 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-tz. 2020-12-05T17:29:36.353 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../104-python3-tz_2018.3-2_all.deb ... 2020-12-05T17:29:36.376 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-pastescript. 2020-12-05T17:29:36.392 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-2_all.deb ... 2020-12-05T17:29:36.396 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-tz (2018.3-2) ... 2020-12-05T17:29:36.434 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2020-12-05T17:29:36.572 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python-funcsigs. 2020-12-05T17:29:36.587 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../095-python-funcsigs_1.0.2-4_all.deb ... 2020-12-05T17:29:36.631 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2020-12-05T17:29:36.699 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-matplotlib. 2020-12-05T17:29:36.714 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../105-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2020-12-05T17:29:36.758 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2020-12-05T17:29:36.804 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-pil:amd64. 2020-12-05T17:29:36.816 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cycler (0.10.0-1) ... 2020-12-05T17:29:36.821 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../108-python3-pil_5.1.0-1ubuntu0.3_amd64.deb ... 2020-12-05T17:29:36.887 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-12-05T17:29:36.935 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python-pluggy. 2020-12-05T17:29:36.948 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../096-python-pluggy_0.6.0-1_all.deb ... 2020-12-05T17:29:36.953 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-olefile. 2020-12-05T17:29:36.970 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../106-python3-olefile_0.45.1-1_all.deb ... 2020-12-05T17:29:37.009 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python-pluggy (0.6.0-1) ... 2020-12-05T17:29:37.012 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-olefile (0.45.1-1) ... 2020-12-05T17:29:37.268 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-pluggy. 2020-12-05T17:29:37.278 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python-py. 2020-12-05T17:29:37.284 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../109-python3-pluggy_0.6.0-1_all.deb ... 2020-12-05T17:29:37.293 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../097-python-py_1.5.2-1_all.deb ... 2020-12-05T17:29:37.314 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pastescript. 2020-12-05T17:29:37.326 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2020-12-05T17:29:37.328 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-2_all.deb ... 2020-12-05T17:29:37.328 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python-py (1.5.2-1) ... 2020-12-05T17:29:37.358 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2020-12-05T17:29:37.380 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2020-12-05T17:29:37.439 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-olefile. 2020-12-05T17:29:37.455 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../106-python3-olefile_0.45.1-1_all.deb ... 2020-12-05T17:29:37.498 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-olefile (0.45.1-1) ... 2020-12-05T17:29:37.639 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python-pytest. 2020-12-05T17:29:37.654 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-py. 2020-12-05T17:29:37.655 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../098-python-pytest_3.3.2-2_all.deb ... 2020-12-05T17:29:37.669 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../110-python3-py_1.5.2-1_all.deb ... 2020-12-05T17:29:37.689 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python-pytest (3.3.2-2) ... 2020-12-05T17:29:37.721 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-py (1.5.2-1) ... 2020-12-05T17:29:37.726 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-bs4 (4.6.0-1) ... 2020-12-05T17:29:37.751 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pil:amd64. 2020-12-05T17:29:37.768 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../108-python3-pil_5.1.0-1ubuntu0.3_amd64.deb ... 2020-12-05T17:29:37.826 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-12-05T17:29:37.850 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-pastescript. 2020-12-05T17:29:37.865 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-2_all.deb ... 2020-12-05T17:29:37.917 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2020-12-05T17:29:38.009 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-attr. 2020-12-05T17:29:38.026 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../099-python3-attr_17.4.0-2_all.deb ... 2020-12-05T17:29:38.052 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pluggy (0.6.0-1) ... 2020-12-05T17:29:38.076 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-attr (17.4.0-2) ... 2020-12-05T17:29:38.099 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-pyinotify. 2020-12-05T17:29:38.115 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1_all.deb ... 2020-12-05T17:29:38.157 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2020-12-05T17:29:38.196 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pluggy. 2020-12-05T17:29:38.212 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../109-python3-pluggy_0.6.0-1_all.deb ... 2020-12-05T17:29:38.254 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2020-12-05T17:29:38.262 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-pil:amd64. 2020-12-05T17:29:38.277 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../108-python3-pil_5.1.0-1ubuntu0.3_amd64.deb ... 2020-12-05T17:29:38.321 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-12-05T17:29:38.354 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up cephadm (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:38.396 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-cycler. 2020-12-05T17:29:38.413 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../100-python3-cycler_0.10.0-1_all.deb ... 2020-12-05T17:29:38.487 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-cycler (0.10.0-1) ... 2020-12-05T17:29:38.494 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-pytest. 2020-12-05T17:29:38.509 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../112-python3-pytest_3.3.2-2_all.deb ... 2020-12-05T17:29:38.552 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-pytest (3.3.2-2) ... 2020-12-05T17:29:38.583 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-py. 2020-12-05T17:29:38.600 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../110-python3-py_1.5.2-1_all.deb ... 2020-12-05T17:29:38.640 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-py (1.5.2-1) ... 2020-12-05T17:29:38.733 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-pluggy. 2020-12-05T17:29:38.748 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../109-python3-pluggy_0.6.0-1_all.deb ... 2020-12-05T17:29:38.791 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2020-12-05T17:29:38.831 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-webencodings. 2020-12-05T17:29:38.847 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../101-python3-webencodings_0.5-2_all.deb ... 2020-12-05T17:29:38.906 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-simplejson. 2020-12-05T17:29:38.920 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../113-python3-simplejson_3.13.2-1_amd64.deb ... 2020-12-05T17:29:38.931 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-webencodings (0.5-2) ... 2020-12-05T17:29:38.964 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2020-12-05T17:29:39.038 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pyinotify. 2020-12-05T17:29:39.055 INFO:teuthology.orchestra.run.smithi032.stdout:Adding system user cephadm....done 2020-12-05T17:29:39.055 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1_all.deb ... 2020-12-05T17:29:39.104 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2020-12-05T17:29:39.127 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-py. 2020-12-05T17:29:39.141 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../110-python3-py_1.5.2-1_all.deb ... 2020-12-05T17:29:39.209 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-html5lib. 2020-12-05T17:29:39.210 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-py (1.5.2-1) ... 2020-12-05T17:29:39.219 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2020-12-05T17:29:39.224 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../102-python3-html5lib_0.999999999-1_all.deb ... 2020-12-05T17:29:39.259 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2020-12-05T17:29:39.317 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-tk:amd64. 2020-12-05T17:29:39.332 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../114-python3-tk_3.6.9-1~18.04_amd64.deb ... 2020-12-05T17:29:39.375 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2020-12-05T17:29:39.415 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pytest. 2020-12-05T17:29:39.434 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../112-python3-pytest_3.3.2-2_all.deb ... 2020-12-05T17:29:39.474 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pytest (3.3.2-2) ... 2020-12-05T17:29:39.542 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-12-05T17:29:39.563 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-pyinotify. 2020-12-05T17:29:39.579 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1_all.deb ... 2020-12-05T17:29:39.622 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2020-12-05T17:29:39.637 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-lxml:amd64. 2020-12-05T17:29:39.654 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../103-python3-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-12-05T17:29:39.671 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2020-12-05T17:29:39.695 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-12-05T17:29:39.806 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2020-12-05T17:29:39.836 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-simplejson. 2020-12-05T17:29:39.850 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../113-python3-simplejson_3.13.2-1_amd64.deb ... 2020-12-05T17:29:39.893 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2020-12-05T17:29:39.929 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package radosgw. 2020-12-05T17:29:39.943 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-pytest. 2020-12-05T17:29:39.944 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../115-radosgw_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:39.956 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../112-python3-pytest_3.3.2-2_all.deb ... 2020-12-05T17:29:39.996 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking radosgw (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:39.999 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-pytest (3.3.2-2) ... 2020-12-05T17:29:40.157 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-prettytable (0.7.2-3) ... 2020-12-05T17:29:40.192 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-tz. 2020-12-05T17:29:40.207 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../104-python3-tz_2018.3-2_all.deb ... 2020-12-05T17:29:40.238 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-tk:amd64. 2020-12-05T17:29:40.249 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-tz (2018.3-2) ... 2020-12-05T17:29:40.255 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../114-python3-tk_3.6.9-1~18.04_amd64.deb ... 2020-12-05T17:29:40.296 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2020-12-05T17:29:40.361 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-simplejson. 2020-12-05T17:29:40.377 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../113-python3-simplejson_3.13.2-1_amd64.deb ... 2020-12-05T17:29:40.419 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2020-12-05T17:29:40.483 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-12-05T17:29:40.596 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-matplotlib. 2020-12-05T17:29:40.613 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../105-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2020-12-05T17:29:40.634 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2020-12-05T17:29:40.660 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2020-12-05T17:29:40.747 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package python3-tk:amd64. 2020-12-05T17:29:40.765 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../114-python3-tk_3.6.9-1~18.04_amd64.deb ... 2020-12-05T17:29:40.767 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package radosgw. 2020-12-05T17:29:40.783 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../115-radosgw_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:40.805 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2020-12-05T17:29:40.825 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking radosgw (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:40.974 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package rbd-fuse. 2020-12-05T17:29:40.977 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-ceph-common (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:40.989 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../116-rbd-fuse_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:41.041 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking rbd-fuse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:41.267 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package radosgw. 2020-12-05T17:29:41.283 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../115-radosgw_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:41.296 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-monotonic (1.1-2) ... 2020-12-05T17:29:41.326 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking radosgw (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:41.355 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-logutils (0.3.3-5) ... 2020-12-05T17:29:41.418 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-olefile. 2020-12-05T17:29:41.433 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../106-python3-olefile_0.45.1-1_all.deb ... 2020-12-05T17:29:41.502 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-olefile (0.45.1-1) ... 2020-12-05T17:29:41.622 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-12-05T17:29:41.654 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2020-12-05T17:29:41.747 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-12-05T17:29:41.846 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-pastescript. 2020-12-05T17:29:41.861 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-2_all.deb ... 2020-12-05T17:29:41.938 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2020-12-05T17:29:41.962 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package rbd-fuse. 2020-12-05T17:29:41.972 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../116-rbd-fuse_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:42.011 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking rbd-fuse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:42.031 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2020-12-05T17:29:42.107 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-html5lib (0.999999999-1) ... 2020-12-05T17:29:42.158 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-12-05T17:29:42.254 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package rbd-fuse. 2020-12-05T17:29:42.270 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../116-rbd-fuse_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:42.312 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking rbd-fuse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:42.314 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-logutils (0.3.3-5) ... 2020-12-05T17:29:42.367 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-pil:amd64. 2020-12-05T17:29:42.383 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../108-python3-pil_5.1.0-1ubuntu0.3_amd64.deb ... 2020-12-05T17:29:42.425 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-12-05T17:29:42.467 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-12-05T17:29:42.548 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2020-12-05T17:29:42.580 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2020-12-05T17:29:42.616 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2020-12-05T17:29:42.623 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-logutils (0.3.3-5) ... 2020-12-05T17:29:42.785 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-12-05T17:29:42.852 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-repoze.lru (0.7-1) ... 2020-12-05T17:29:42.853 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-pluggy. 2020-12-05T17:29:42.884 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../109-python3-pluggy_0.6.0-1_all.deb ... 2020-12-05T17:29:42.911 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2020-12-05T17:29:42.926 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2020-12-05T17:29:43.009 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2020-12-05T17:29:43.128 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-12-05T17:29:43.152 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-12-05T17:29:43.195 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2020-12-05T17:29:43.282 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-py. 2020-12-05T17:29:43.296 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../110-python3-py_1.5.2-1_all.deb ... 2020-12-05T17:29:43.302 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2020-12-05T17:29:43.339 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-py (1.5.2-1) ... 2020-12-05T17:29:43.421 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2020-12-05T17:29:43.444 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-12-05T17:29:43.480 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-webencodings (0.5-2) ... 2020-12-05T17:29:43.701 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-pyinotify. 2020-12-05T17:29:43.719 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1_all.deb ... 2020-12-05T17:29:43.759 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2020-12-05T17:29:43.764 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-attr (17.4.0-2) ... 2020-12-05T17:29:43.771 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-12-05T17:29:43.772 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python-attr (17.4.0-2) ... 2020-12-05T17:29:43.965 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2020-12-05T17:29:44.056 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-funcsigs (1.0.2-4) ... 2020-12-05T17:29:44.064 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-12-05T17:29:44.091 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-py (1.5.2-1) ... 2020-12-05T17:29:44.120 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-pytest. 2020-12-05T17:29:44.135 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../112-python3-pytest_3.3.2-2_all.deb ... 2020-12-05T17:29:44.148 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2020-12-05T17:29:44.178 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-pytest (3.3.2-2) ... 2020-12-05T17:29:44.249 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-12-05T17:29:44.435 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-waitress (1.0.1-1) ... 2020-12-05T17:29:44.449 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-webencodings (0.5-2) ... 2020-12-05T17:29:44.482 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2020-12-05T17:29:44.556 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-simplejson. 2020-12-05T17:29:44.574 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../113-python3-simplejson_3.13.2-1_amd64.deb ... 2020-12-05T17:29:44.605 INFO:teuthology.orchestra.run.smithi036.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-12-05T17:29:44.615 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2020-12-05T17:29:44.684 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2020-12-05T17:29:44.750 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python-attr (17.4.0-2) ... 2020-12-05T17:29:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2020-12-05T17:29:44.768 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-webencodings (0.5-2) ... 2020-12-05T17:29:44.879 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-olefile (0.45.1-1) ... 2020-12-05T17:29:44.960 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2020-12-05T17:29:44.976 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-tk:amd64. 2020-12-05T17:29:44.993 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../114-python3-tk_3.6.9-1~18.04_amd64.deb ... 2020-12-05T17:29:45.011 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liboath0 (2.6.1-1) ... 2020-12-05T17:29:45.034 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2020-12-05T17:29:45.068 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python-attr (17.4.0-2) ... 2020-12-05T17:29:45.112 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-py (1.5.2-1) ... 2020-12-05T17:29:45.154 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-12-05T17:29:45.181 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-ceph-argparse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:45.269 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2020-12-05T17:29:45.272 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2020-12-05T17:29:45.436 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-py (1.5.2-1) ... 2020-12-05T17:29:45.455 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-waitress (1.0.1-1) ... 2020-12-05T17:29:45.499 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-tempita (0.5.2-2) ... 2020-12-05T17:29:45.527 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-12-05T17:29:45.617 INFO:teuthology.orchestra.run.smithi053.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-12-05T17:29:45.621 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package radosgw. 2020-12-05T17:29:45.636 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../115-radosgw_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:45.649 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-setuptools (39.0.1-2) ... 2020-12-05T17:29:45.680 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking radosgw (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:45.788 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-waitress (1.0.1-1) ... 2020-12-05T17:29:45.825 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-12-05T17:29:45.924 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-olefile (0.45.1-1) ... 2020-12-05T17:29:45.947 INFO:teuthology.orchestra.run.smithi061.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-12-05T17:29:45.951 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2020-12-05T17:29:46.075 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-12-05T17:29:46.185 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-decorator (4.1.2-1) ... 2020-12-05T17:29:46.225 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-ceph-argparse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:46.258 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-olefile (0.45.1-1) ... 2020-12-05T17:29:46.461 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-tz (2018.3-2) ... 2020-12-05T17:29:46.633 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-tempita (0.5.2-2) ... 2020-12-05T17:29:46.745 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2020-12-05T17:29:46.758 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package rbd-fuse. 2020-12-05T17:29:46.775 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../116-rbd-fuse_15.2.6-607-g3b1861ac-1bionic_amd64.deb ... 2020-12-05T17:29:46.820 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking rbd-fuse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:46.896 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2020-12-05T17:29:46.934 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-ceph-argparse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:47.030 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2020-12-05T17:29:47.177 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-logutils (0.3.3-5) ... 2020-12-05T17:29:47.222 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2020-12-05T17:29:47.244 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-12-05T17:29:47.260 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-tempita (0.5.2-2) ... 2020-12-05T17:29:47.339 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-joblib (0.11-1) ... 2020-12-05T17:29:47.404 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2020-12-05T17:29:47.488 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2020-12-05T17:29:47.498 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python-py (1.5.2-1) ... 2020-12-05T17:29:47.570 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-12-05T17:29:47.696 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2020-12-05T17:29:47.724 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-routes (2.4.1-1) ... 2020-12-05T17:29:47.733 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-simplejson (3.13.2-1) ... 2020-12-05T17:29:48.025 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2020-12-05T17:29:48.048 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2020-12-05T17:29:48.182 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-12-05T17:29:48.334 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-dateutil (2.6.1-1) ... 2020-12-05T17:29:48.365 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2020-12-05T17:29:48.376 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-json-pointer (1.10-1) ... 2020-12-05T17:29:48.635 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2020-12-05T17:29:48.648 INFO:teuthology.orchestra.run.smithi036.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2020-12-05T17:29:48.674 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2020-12-05T17:29:48.683 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2020-12-05T17:29:48.864 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-12-05T17:29:48.895 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-12-05T17:29:48.912 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-12-05T17:29:48.992 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python-py (1.5.2-1) ... 2020-12-05T17:29:49.000 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2020-12-05T17:29:49.021 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python-pluggy (0.6.0-1) ... 2020-12-05T17:29:49.226 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-simplejson (3.13.2-1) ... 2020-12-05T17:29:49.238 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-colorama (0.3.7-1) ... 2020-12-05T17:29:49.264 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-fasteners (0.12.0-3) ... 2020-12-05T17:29:49.284 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python-py (1.5.2-1) ... 2020-12-05T17:29:49.311 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2020-12-05T17:29:49.531 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-simplejson (3.13.2-1) ... 2020-12-05T17:29:49.540 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2020-12-05T17:29:49.578 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-json-pointer (1.10-1) ... 2020-12-05T17:29:49.581 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up jq (1.5+dfsg-2) ... 2020-12-05T17:29:49.613 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-webencodings (0.5-2) ... 2020-12-05T17:29:49.685 INFO:teuthology.orchestra.run.smithi053.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2020-12-05T17:29:49.708 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2020-12-05T17:29:49.854 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-json-pointer (1.10-1) ... 2020-12-05T17:29:49.866 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2020-12-05T17:29:49.930 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-12-05T17:29:49.943 INFO:teuthology.orchestra.run.smithi061.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2020-12-05T17:29:49.955 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python-attr (17.4.0-2) ... 2020-12-05T17:29:50.056 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python-pluggy (0.6.0-1) ... 2020-12-05T17:29:50.165 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2020-12-05T17:29:50.184 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-cycler (0.10.0-1) ... 2020-12-05T17:29:50.197 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-12-05T17:29:50.257 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-colorama (0.3.7-1) ... 2020-12-05T17:29:50.290 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-py (1.5.2-1) ... 2020-12-05T17:29:50.323 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python-pluggy (0.6.0-1) ... 2020-12-05T17:29:50.502 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2020-12-05T17:29:50.508 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-colorama (0.3.7-1) ... 2020-12-05T17:29:50.542 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2020-12-05T17:29:50.659 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-waitress (1.0.1-1) ... 2020-12-05T17:29:50.801 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2020-12-05T17:29:50.840 INFO:teuthology.orchestra.run.smithi002.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-12-05T17:29:50.845 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2020-12-05T17:29:50.887 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-bs4 (4.6.0-1) ... 2020-12-05T17:29:50.962 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-12-05T17:29:51.101 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2020-12-05T17:29:51.130 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-olefile (0.45.1-1) ... 2020-12-05T17:29:51.168 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-cycler (0.10.0-1) ... 2020-12-05T17:29:51.246 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-pluggy (0.6.0-1) ... 2020-12-05T17:29:51.272 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2020-12-05T17:29:51.411 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-cycler (0.10.0-1) ... 2020-12-05T17:29:51.455 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-ceph-argparse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:51.495 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2020-12-05T17:29:51.556 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up cephadm (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:51.607 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2020-12-05T17:29:51.695 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2020-12-05T17:29:51.826 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-tempita (0.5.2-2) ... 2020-12-05T17:29:51.827 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pytest (3.3.2-2) ... 2020-12-05T17:29:51.871 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-bs4 (4.6.0-1) ... 2020-12-05T17:29:52.066 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-bs4 (4.6.0-1) ... 2020-12-05T17:29:52.107 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-12-05T17:29:52.169 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2020-12-05T17:29:52.215 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pluggy (0.6.0-1) ... 2020-12-05T17:29:52.232 INFO:teuthology.orchestra.run.smithi036.stdout:Adding system user cephadm....done 2020-12-05T17:29:52.259 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2020-12-05T17:29:52.405 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2020-12-05T17:29:52.409 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-pluggy (0.6.0-1) ... 2020-12-05T17:29:52.452 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-pytest (3.3.2-2) ... 2020-12-05T17:29:52.523 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up cephadm (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:52.685 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up cephadm (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:52.701 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-12-05T17:29:52.741 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-12-05T17:29:52.800 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-12-05T17:29:52.865 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2020-12-05T17:29:52.939 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librados2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:53.026 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2020-12-05T17:29:53.065 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rsa (3.4.2-1) ... 2020-12-05T17:29:53.200 INFO:teuthology.orchestra.run.smithi053.stdout:Adding system user cephadm....done 2020-12-05T17:29:53.236 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2020-12-05T17:29:53.301 INFO:teuthology.orchestra.run.smithi061.stdout:Adding system user cephadm....done 2020-12-05T17:29:53.360 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-prettytable (0.7.2-3) ... 2020-12-05T17:29:53.364 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2020-12-05T17:29:53.383 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libcephfs2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:53.499 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2020-12-05T17:29:53.509 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libcephfs-dev (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:53.596 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2020-12-05T17:29:53.618 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rados (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:53.665 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-12-05T17:29:53.686 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-12-05T17:29:53.719 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2020-12-05T17:29:53.791 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2020-12-05T17:29:53.792 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-12-05T17:29:53.812 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2020-12-05T17:29:53.880 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python-py (1.5.2-1) ... 2020-12-05T17:29:53.893 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2020-12-05T17:29:53.959 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2020-12-05T17:29:54.011 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2020-12-05T17:29:54.114 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2020-12-05T17:29:54.172 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-ceph-common (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:54.173 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-simplejson (3.13.2-1) ... 2020-12-05T17:29:54.335 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-prettytable (0.7.2-3) ... 2020-12-05T17:29:54.345 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-prettytable (0.7.2-3) ... 2020-12-05T17:29:54.407 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2020-12-05T17:29:54.498 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-monotonic (1.1-2) ... 2020-12-05T17:29:54.516 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-json-pointer (1.10-1) ... 2020-12-05T17:29:54.631 INFO:teuthology.orchestra.run.smithi002.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2020-12-05T17:29:54.644 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-12-05T17:29:54.663 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-12-05T17:29:54.716 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2020-12-05T17:29:54.796 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2020-12-05T17:29:54.799 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-12-05T17:29:54.814 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2020-12-05T17:29:54.901 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-12-05T17:29:54.934 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-12-05T17:29:55.052 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python-pluggy (0.6.0-1) ... 2020-12-05T17:29:55.139 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-ceph-common (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:55.141 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-ceph-common (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:55.262 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-colorama (0.3.7-1) ... 2020-12-05T17:29:55.286 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-html5lib (0.999999999-1) ... 2020-12-05T17:29:55.450 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-monotonic (1.1-2) ... 2020-12-05T17:29:55.465 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-monotonic (1.1-2) ... 2020-12-05T17:29:55.571 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2020-12-05T17:29:55.586 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-sklearn (0.19.1-3) ... 2020-12-05T17:29:55.662 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-12-05T17:29:55.718 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-12-05T17:29:55.760 INFO:teuthology.orchestra.run.smithi036.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2020-12-05T17:29:55.766 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-12-05T17:29:55.793 INFO:teuthology.orchestra.run.smithi036.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2020-12-05T17:29:55.819 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-12-05T17:29:55.892 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-12-05T17:29:55.897 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2020-12-05T17:29:56.031 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-repoze.lru (0.7-1) ... 2020-12-05T17:29:56.162 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-html5lib (0.999999999-1) ... 2020-12-05T17:29:56.215 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-cycler (0.10.0-1) ... 2020-12-05T17:29:56.252 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-html5lib (0.999999999-1) ... 2020-12-05T17:29:56.332 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-12-05T17:29:56.440 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2020-12-05T17:29:56.505 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-12-05T17:29:56.558 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2020-12-05T17:29:56.595 INFO:teuthology.orchestra.run.smithi061.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2020-12-05T17:29:56.618 INFO:teuthology.orchestra.run.smithi061.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2020-12-05T17:29:56.636 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-12-05T17:29:56.650 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2020-12-05T17:29:56.727 INFO:teuthology.orchestra.run.smithi053.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2020-12-05T17:29:56.742 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libradosstriper1 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:56.759 INFO:teuthology.orchestra.run.smithi053.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2020-12-05T17:29:56.865 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-repoze.lru (0.7-1) ... 2020-12-05T17:29:56.867 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cephfs (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:56.926 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-bs4 (4.6.0-1) ... 2020-12-05T17:29:56.997 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-repoze.lru (0.7-1) ... 2020-12-05T17:29:57.010 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-attr (17.4.0-2) ... 2020-12-05T17:29:57.167 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-12-05T17:29:57.185 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librgw2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:57.269 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-pluggy (0.6.0-1) ... 2020-12-05T17:29:57.286 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-fuse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:57.306 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-12-05T17:29:57.336 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python-funcsigs (1.0.2-4) ... 2020-12-05T17:29:57.443 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2020-12-05T17:29:57.487 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-12-05T17:29:57.487 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-12-05T17:29:57.529 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-12-05T17:29:57.595 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up cephadm (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:57.616 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2020-12-05T17:29:57.769 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-attr (17.4.0-2) ... 2020-12-05T17:29:57.772 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librbd1 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:57.889 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up cephfs-shell (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:57.947 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2020-12-05T17:29:57.959 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-attr (17.4.0-2) ... 2020-12-05T17:29:58.045 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python-funcsigs (1.0.2-4) ... 2020-12-05T17:29:58.048 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pecan (1.2.1-2) ... 2020-12-05T17:29:58.054 INFO:teuthology.orchestra.run.smithi036.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2020-12-05T17:29:58.138 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-12-05T17:29:58.171 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-12-05T17:29:58.229 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-12-05T17:29:58.268 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python-funcsigs (1.0.2-4) ... 2020-12-05T17:29:58.307 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up liboath0 (2.6.1-1) ... 2020-12-05T17:29:58.469 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-12-05T17:29:58.477 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2020-12-05T17:29:58.484 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-12-05T17:29:58.522 INFO:teuthology.orchestra.run.smithi002.stdout:Adding system user cephadm....done 2020-12-05T17:29:58.610 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2020-12-05T17:29:58.639 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2020-12-05T17:29:58.757 INFO:teuthology.orchestra.run.smithi061.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2020-12-05T17:29:58.853 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-12-05T17:29:58.905 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2020-12-05T17:29:58.969 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2020-12-05T17:29:58.994 INFO:teuthology.orchestra.run.smithi053.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2020-12-05T17:29:58.996 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-setuptools (39.0.1-2) ... 2020-12-05T17:29:59.008 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up liboath0 (2.6.1-1) ... 2020-12-05T17:29:59.146 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-modules-core (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:59.175 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-12-05T17:29:59.255 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pastescript (2.0.2-2) ... 2020-12-05T17:29:59.256 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up liboath0 (2.6.1-1) ... 2020-12-05T17:29:59.276 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2020-12-05T17:29:59.295 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-12-05T17:29:59.422 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-12-05T17:29:59.429 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2020-12-05T17:29:59.432 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-12-05T17:29:59.503 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-12-05T17:29:59.548 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-decorator (4.1.2-1) ... 2020-12-05T17:29:59.558 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2020-12-05T17:29:59.590 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rgw (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:59.604 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-setuptools (39.0.1-2) ... 2020-12-05T17:29:59.605 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2020-12-05T17:29:59.716 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up rbd-fuse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:59.802 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-12-05T17:29:59.842 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rbd (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:59.875 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-tz (2018.3-2) ... 2020-12-05T17:29:59.928 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-setuptools (39.0.1-2) ... 2020-12-05T17:29:59.968 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-common (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:29:59.980 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-12-05T17:29:59.990 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-prettytable (0.7.2-3) ... 2020-12-05T17:30:00.131 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-decorator (4.1.2-1) ... 2020-12-05T17:30:00.184 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2020-12-05T17:30:00.329 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-12-05T17:30:00.341 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-12-05T17:30:00.436 INFO:teuthology.orchestra.run.smithi032.stdout:Adding group ceph....done 2020-12-05T17:30:00.441 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-tz (2018.3-2) ... 2020-12-05T17:30:00.455 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-decorator (4.1.2-1) ... 2020-12-05T17:30:00.467 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2020-12-05T17:30:00.485 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2020-12-05T17:30:00.748 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-tz (2018.3-2) ... 2020-12-05T17:30:00.750 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2020-12-05T17:30:00.789 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-joblib (0.11-1) ... 2020-12-05T17:30:00.822 INFO:teuthology.orchestra.run.smithi032.stdout:Adding system user ceph....done 2020-12-05T17:30:00.835 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-ceph-common (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:00.980 INFO:teuthology.orchestra.run.smithi032.stdout:Setting system user ceph properties....done 2020-12-05T17:30:01.018 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2020-12-05T17:30:01.049 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2020-12-05T17:30:01.141 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-12-05T17:30:01.163 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-routes (2.4.1-1) ... 2020-12-05T17:30:01.187 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-monotonic (1.1-2) ... 2020-12-05T17:30:01.311 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-joblib (0.11-1) ... 2020-12-05T17:30:01.358 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2020-12-05T17:30:01.372 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-12-05T17:30:01.488 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-12-05T17:30:01.498 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2020-12-05T17:30:01.655 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-12-05T17:30:01.671 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-routes (2.4.1-1) ... 2020-12-05T17:30:01.676 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-joblib (0.11-1) ... 2020-12-05T17:30:01.679 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up radosgw (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:01.808 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-dateutil (2.6.1-1) ... 2020-12-05T17:30:01.935 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-12-05T17:30:01.936 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-12-05T17:30:01.997 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2020-12-05T17:30:02.040 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-html5lib (0.999999999-1) ... 2020-12-05T17:30:02.078 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-routes (2.4.1-1) ... 2020-12-05T17:30:02.184 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2020-12-05T17:30:02.298 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-dateutil (2.6.1-1) ... 2020-12-05T17:30:02.381 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-test (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:02.396 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2020-12-05T17:30:02.400 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-12-05T17:30:02.495 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-12-05T17:30:02.508 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-base (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:02.523 INFO:teuthology.orchestra.run.smithi002.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2020-12-05T17:30:02.555 INFO:teuthology.orchestra.run.smithi002.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2020-12-05T17:30:02.654 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2020-12-05T17:30:02.705 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-dateutil (2.6.1-1) ... 2020-12-05T17:30:02.802 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-repoze.lru (0.7-1) ... 2020-12-05T17:30:02.846 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-12-05T17:30:02.854 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-fasteners (0.12.0-3) ... 2020-12-05T17:30:02.959 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-12-05T17:30:03.031 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2020-12-05T17:30:03.128 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-12-05T17:30:03.135 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:03.156 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up jq (1.5+dfsg-2) ... 2020-12-05T17:30:03.298 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2020-12-05T17:30:03.302 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-fasteners (0.12.0-3) ... 2020-12-05T17:30:03.333 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-12-05T17:30:03.360 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-12-05T17:30:03.360 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-12-05T17:30:03.438 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2020-12-05T17:30:03.587 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up jq (1.5+dfsg-2) ... 2020-12-05T17:30:03.644 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-diskprediction-local (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:03.688 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2020-12-05T17:30:03.693 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-fasteners (0.12.0-3) ... 2020-12-05T17:30:03.770 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-osd (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:03.825 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-attr (17.4.0-2) ... 2020-12-05T17:30:04.002 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up jq (1.5+dfsg-2) ... 2020-12-05T17:30:04.145 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2020-12-05T17:30:04.165 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python-funcsigs (1.0.2-4) ... 2020-12-05T17:30:04.213 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-12-05T17:30:04.213 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-12-05T17:30:04.374 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-12-05T17:30:04.490 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-cephadm (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:04.521 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-12-05T17:30:04.632 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:04.775 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-rook (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:04.802 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2020-12-05T17:30:04.848 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2020-12-05T17:30:04.876 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-12-05T17:30:04.891 INFO:teuthology.orchestra.run.smithi002.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2020-12-05T17:30:04.918 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mds (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:05.139 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-12-05T17:30:05.140 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-12-05T17:30:05.145 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up liboath0 (2.6.1-1) ... 2020-12-05T17:30:05.178 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2020-12-05T17:30:05.207 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2020-12-05T17:30:05.354 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-12-05T17:30:05.359 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-12-05T17:30:05.366 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-pytest (3.3.2-2) ... 2020-12-05T17:30:05.419 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mon (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:05.470 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2020-12-05T17:30:05.513 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2020-12-05T17:30:05.588 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-pytest (3.3.2-2) ... 2020-12-05T17:30:05.623 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-12-05T17:30:05.624 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-12-05T17:30:05.693 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2020-12-05T17:30:05.759 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2020-12-05T17:30:05.824 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-12-05T17:30:05.895 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-k8sevents (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:05.949 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-setuptools (39.0.1-2) ... 2020-12-05T17:30:05.964 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2020-12-05T17:30:05.996 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-dashboard (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:06.030 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2020-12-05T17:30:06.060 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python-pytest (3.3.2-2) ... 2020-12-05T17:30:06.122 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:06.171 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pytest (3.3.2-2) ... 2020-12-05T17:30:06.266 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for systemd (237-3ubuntu10.43) ... 2020-12-05T17:30:06.274 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python-pytest (3.3.2-2) ... 2020-12-05T17:30:06.320 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-12-05T17:30:06.393 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-12-05T17:30:06.448 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-12-05T17:30:06.466 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-12-05T17:30:06.534 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-12-05T17:30:06.543 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-decorator (4.1.2-1) ... 2020-12-05T17:30:06.547 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2020-12-05T17:30:06.572 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up librados2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:06.668 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-12-05T17:30:06.731 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-rsa (3.4.2-1) ... 2020-12-05T17:30:06.769 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up librados2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:06.845 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-tz (2018.3-2) ... 2020-12-05T17:30:06.857 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python-pytest (3.3.2-2) ... 2020-12-05T17:30:06.869 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-rsa (3.4.2-1) ... 2020-12-05T17:30:07.059 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libcephfs2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:07.108 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-12-05T17:30:07.171 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up libcephfs2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:07.185 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libcephfs-dev (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:07.187 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2020-12-05T17:30:07.234 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-12-05T17:30:07.297 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up libcephfs-dev (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:07.313 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-rados (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:07.377 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up librados2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:07.424 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-rados (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:07.437 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2020-12-05T17:30:07.520 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-rsa (3.4.2-1) ... 2020-12-05T17:30:07.550 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2020-12-05T17:30:07.564 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2020-12-05T17:30:07.831 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2020-12-05T17:30:07.838 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libcephfs2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:07.865 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-joblib (0.11-1) ... 2020-12-05T17:30:07.936 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2020-12-05T17:30:07.972 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libcephfs-dev (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:08.098 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-rados (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:08.133 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2020-12-05T17:30:08.220 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2020-12-05T17:30:08.224 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2020-12-05T17:30:08.266 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-routes (2.4.1-1) ... 2020-12-05T17:30:08.493 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2020-12-05T17:30:08.513 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2020-12-05T17:30:08.610 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2020-12-05T17:30:08.618 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2020-12-05T17:30:08.729 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-12-05T17:30:08.763 INFO:teuthology.orchestra.run.smithi032.stdout:ureadahead will be reprofiled on next reboot 2020-12-05T17:30:08.812 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-12-05T17:30:08.919 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-dateutil (2.6.1-1) ... 2020-12-05T17:30:08.920 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2020-12-05T17:30:09.236 INFO:teuthology.orchestra.run.smithi032.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-12-05T17:30:09.238 INFO:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2020-12-05T17:30:09.263 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2020-12-05T17:30:09.270 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2020-12-05T17:30:09.318 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2020-12-05T17:30:09.354 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-sklearn (0.19.1-3) ... 2020-12-05T17:30:09.391 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-sklearn (0.19.1-3) ... 2020-12-05T17:30:09.571 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-12-05T17:30:09.583 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2020-12-05T17:30:09.584 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2020-12-05T17:30:09.763 INFO:teuthology.orchestra.run.smithi032.stdout:libtool-bin is already the newest version (2.4.6-2). 2020-12-05T17:30:09.763 INFO:teuthology.orchestra.run.smithi032.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2020-12-05T17:30:09.764 INFO:teuthology.orchestra.run.smithi032.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2020-12-05T17:30:09.764 INFO:teuthology.orchestra.run.smithi032.stdout:The following additional packages will be installed: 2020-12-05T17:30:09.765 INFO:teuthology.orchestra.run.smithi032.stdout: libbison-dev libfl-dev libfl2 zlib1g-dev 2020-12-05T17:30:09.766 INFO:teuthology.orchestra.run.smithi032.stdout:Suggested packages: 2020-12-05T17:30:09.766 INFO:teuthology.orchestra.run.smithi032.stdout: bison-doc flex-doc indent-doc libssl-doc 2020-12-05T17:30:09.794 INFO:teuthology.orchestra.run.smithi032.stdout:The following NEW packages will be installed: 2020-12-05T17:30:09.795 INFO:teuthology.orchestra.run.smithi032.stdout: bison dump flex indent libaio-dev libbison-dev libelf-dev libfl-dev libfl2 2020-12-05T17:30:09.796 INFO:teuthology.orchestra.run.smithi032.stdout: libssl-dev zlib1g-dev 2020-12-05T17:30:09.965 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-fasteners (0.12.0-3) ... 2020-12-05T17:30:09.980 INFO:teuthology.orchestra.run.smithi032.stdout:0 upgraded, 11 newly installed, 0 to remove and 3 not upgraded. 2020-12-05T17:30:09.980 INFO:teuthology.orchestra.run.smithi032.stdout:Need to get 2,960 kB of archives. 2020-12-05T17:30:09.980 INFO:teuthology.orchestra.run.smithi032.stdout:After this operation, 12.2 MB of additional disk space will be used. 2020-12-05T17:30:09.981 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2020-12-05T17:30:10.174 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-sklearn (0.19.1-3) ... 2020-12-05T17:30:10.193 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2020-12-05T17:30:10.230 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2020-12-05T17:30:10.308 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up jq (1.5+dfsg-2) ... 2020-12-05T17:30:10.425 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2020-12-05T17:30:10.434 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2020-12-05T17:30:10.502 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libradosstriper1 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:10.506 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2020-12-05T17:30:10.548 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up libradosstriper1 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:10.556 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2020-12-05T17:30:10.577 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2020-12-05T17:30:10.586 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2020-12-05T17:30:10.588 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2020-12-05T17:30:10.603 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2020-12-05T17:30:10.607 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2020-12-05T17:30:10.608 INFO:teuthology.orchestra.run.smithi032.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2020-12-05T17:30:10.609 INFO:teuthology.orchestra.run.smithi032.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.6 [1,566 kB] 2020-12-05T17:30:10.628 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-cephfs (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:10.682 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-cephfs (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:10.946 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up librgw2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:10.992 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up librgw2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:11.012 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2020-12-05T17:30:11.097 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 2,960 kB in 1s (3,332 kB/s) 2020-12-05T17:30:11.098 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-fuse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:11.118 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up ceph-fuse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:11.227 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package flex. 2020-12-05T17:30:11.280 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 145453 files and directories currently installed.) 2020-12-05T17:30:11.284 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2020-12-05T17:30:11.321 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-12-05T17:30:11.322 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-12-05T17:30:11.322 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libradosstriper1 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:11.325 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking flex (2.6.4-6) ... 2020-12-05T17:30:11.337 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-12-05T17:30:11.338 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-12-05T17:30:11.473 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-cephfs (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:11.594 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up librbd1 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:11.624 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up librbd1 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:11.672 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-12-05T17:30:11.695 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up cephfs-shell (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:11.712 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libbison-dev:amd64. 2020-12-05T17:30:11.728 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../01-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-12-05T17:30:11.759 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up cephfs-shell (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:11.762 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-12-05T17:30:11.782 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up librgw2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:11.796 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-pecan (1.2.1-2) ... 2020-12-05T17:30:11.885 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-pecan (1.2.1-2) ... 2020-12-05T17:30:11.908 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-fuse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:11.911 INFO:teuthology.orchestra.run.smithi061.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-12-05T17:30:11.944 INFO:teuthology.orchestra.run.smithi061.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-12-05T17:30:11.975 INFO:teuthology.orchestra.run.smithi036.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-12-05T17:30:12.015 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2020-12-05T17:30:12.016 INFO:teuthology.orchestra.run.smithi036.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-12-05T17:30:12.041 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package bison. 2020-12-05T17:30:12.056 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../02-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-12-05T17:30:12.095 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2020-12-05T17:30:12.128 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-12-05T17:30:12.129 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-12-05T17:30:12.267 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2020-12-05T17:30:12.338 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2020-12-05T17:30:12.373 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2020-12-05T17:30:12.393 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up librbd1 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:12.501 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-pytest (3.3.2-2) ... 2020-12-05T17:30:12.527 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up cephfs-shell (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:12.587 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package dump. 2020-12-05T17:30:12.603 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../03-dump_0.4b46-3_amd64.deb ... 2020-12-05T17:30:12.636 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking dump (0.4b46-3) ... 2020-12-05T17:30:12.779 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pecan (1.2.1-2) ... 2020-12-05T17:30:12.885 INFO:teuthology.orchestra.run.smithi053.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-12-05T17:30:12.911 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2020-12-05T17:30:12.912 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package indent. 2020-12-05T17:30:12.917 INFO:teuthology.orchestra.run.smithi053.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-12-05T17:30:12.928 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../04-indent_2.2.11-5_amd64.deb ... 2020-12-05T17:30:12.935 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up ceph-mgr-modules-core (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:12.971 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking indent (2.2.11-5) ... 2020-12-05T17:30:13.036 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-pastescript (2.0.2-2) ... 2020-12-05T17:30:13.040 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-mgr-modules-core (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:13.167 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-pastescript (2.0.2-2) ... 2020-12-05T17:30:13.187 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python-pytest (3.3.2-2) ... 2020-12-05T17:30:13.240 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2020-12-05T17:30:13.273 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libaio-dev:amd64. 2020-12-05T17:30:13.288 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../05-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2020-12-05T17:30:13.321 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-rgw (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:13.357 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-12-05T17:30:13.430 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up rbd-fuse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:13.463 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-12-05T17:30:13.502 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-rgw (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:13.557 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up python3-rbd (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:13.560 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2020-12-05T17:30:13.564 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-12-05T17:30:13.574 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../06-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2020-12-05T17:30:13.610 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-12-05T17:30:13.629 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up rbd-fuse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:13.683 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up ceph-common (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:13.715 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up librados2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:13.754 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-rbd (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:13.860 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-rsa (3.4.2-1) ... 2020-12-05T17:30:13.905 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libelf-dev:amd64. 2020-12-05T17:30:13.905 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-common (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:13.921 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../07-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2020-12-05T17:30:13.926 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mgr-modules-core (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:13.963 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-12-05T17:30:14.052 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pastescript (2.0.2-2) ... 2020-12-05T17:30:14.151 INFO:teuthology.orchestra.run.smithi061.stdout:Adding group ceph....done 2020-12-05T17:30:14.209 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libcephfs2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:14.360 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libcephfs-dev (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:14.366 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libfl2:amd64. 2020-12-05T17:30:14.378 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-rgw (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:14.383 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../08-libfl2_2.6.4-6_amd64.deb ... 2020-12-05T17:30:14.467 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2020-12-05T17:30:14.478 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-rados (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:14.513 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up rbd-fuse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:14.529 INFO:teuthology.orchestra.run.smithi061.stdout:Adding system user ceph....done 2020-12-05T17:30:14.591 INFO:teuthology.orchestra.run.smithi036.stdout:Adding group ceph....done 2020-12-05T17:30:14.604 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2020-12-05T17:30:14.639 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-rbd (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:14.711 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libfl-dev:amd64. 2020-12-05T17:30:14.721 INFO:teuthology.orchestra.run.smithi061.stdout:Setting system user ceph properties....done 2020-12-05T17:30:14.728 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../09-libfl-dev_2.6.4-6_amd64.deb ... 2020-12-05T17:30:14.765 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-common (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:14.769 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2020-12-05T17:30:14.881 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-12-05T17:30:15.003 INFO:teuthology.orchestra.run.smithi036.stdout:Adding system user ceph....done 2020-12-05T17:30:15.015 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2020-12-05T17:30:15.039 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libssl-dev:amd64. 2020-12-05T17:30:15.054 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../10-libssl-dev_1.1.1-1ubuntu2.1~18.04.6_amd64.deb ... 2020-12-05T17:30:15.114 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-12-05T17:30:15.115 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-12-05T17:30:15.195 INFO:teuthology.orchestra.run.smithi036.stdout:Setting system user ceph properties....done 2020-12-05T17:30:15.325 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2020-12-05T17:30:15.325 INFO:teuthology.orchestra.run.smithi053.stdout:Adding group ceph....done 2020-12-05T17:30:15.359 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-12-05T17:30:15.411 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up radosgw (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:15.588 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-12-05T17:30:15.607 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up flex (2.6.4-6) ... 2020-12-05T17:30:15.655 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-12-05T17:30:15.655 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-12-05T17:30:15.684 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2020-12-05T17:30:15.728 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up dump (0.4b46-3) ... 2020-12-05T17:30:15.778 INFO:teuthology.orchestra.run.smithi053.stdout:Adding system user ceph....done 2020-12-05T17:30:15.853 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2020-12-05T17:30:15.902 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up radosgw (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:15.921 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-12-05T17:30:15.987 INFO:teuthology.orchestra.run.smithi053.stdout:Setting system user ceph properties....done 2020-12-05T17:30:16.046 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up ceph-test (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:16.047 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up indent (2.2.11-5) ... 2020-12-05T17:30:16.145 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-12-05T17:30:16.146 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up ceph-base (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:16.171 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-12-05T17:30:16.171 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-12-05T17:30:16.173 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-12-05T17:30:16.299 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2020-12-05T17:30:16.364 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-12-05T17:30:16.451 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2020-12-05T17:30:16.487 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-12-05T17:30:16.540 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2020-12-05T17:30:16.607 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-sklearn (0.19.1-3) ... 2020-12-05T17:30:16.609 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up radosgw (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:16.612 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-test (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:16.619 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-12-05T17:30:16.753 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-12-05T17:30:16.755 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-base (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:16.774 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up ceph-mgr (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:16.870 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-12-05T17:30:16.870 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-12-05T17:30:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-12-05T17:30:16.999 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-12-05T17:30:16.999 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-12-05T17:30:17.005 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2020-12-05T17:30:17.109 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-12-05T17:30:17.132 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-12-05T17:30:17.262 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-test (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:17.399 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-mgr (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:17.475 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2020-12-05T17:30:17.613 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-12-05T17:30:17.644 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-12-05T17:30:17.685 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up ceph-mgr-diskprediction-local (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:17.785 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libradosstriper1 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:17.811 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up ceph-osd (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:17.897 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-base (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:17.903 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-mgr-diskprediction-local (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:17.936 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-cephfs (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:18.052 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-osd (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:18.195 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up librgw2 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:18.244 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-12-05T17:30:18.245 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-12-05T17:30:18.245 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-12-05T17:30:18.329 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-fuse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:18.504 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up ceph-mgr-cephadm (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:18.532 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mgr (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:18.535 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-12-05T17:30:18.535 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-12-05T17:30:18.570 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-12-05T17:30:18.570 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-12-05T17:30:18.622 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:18.757 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up ceph-mgr-rook (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:18.757 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-12-05T17:30:18.758 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-12-05T17:30:18.772 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up librbd1 (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:18.870 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-mgr-cephadm (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:18.890 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up cephfs-shell (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:18.908 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up ceph-mds (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:18.969 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2020-12-05T17:30:19.014 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:19.041 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-pecan (1.2.1-2) ... 2020-12-05T17:30:19.050 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mgr-diskprediction-local (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:19.136 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-12-05T17:30:19.136 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-12-05T17:30:19.193 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-osd (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:19.273 INFO:teuthology.orchestra.run.smithi002.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-12-05T17:30:19.314 INFO:teuthology.orchestra.run.smithi002.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-12-05T17:30:19.320 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-12-05T17:30:19.409 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up ceph-mon (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:19.612 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2020-12-05T17:30:19.628 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-12-05T17:30:19.628 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-12-05T17:30:19.646 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-12-05T17:30:19.646 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-12-05T17:30:19.724 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-mgr-rook (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:19.814 INFO:teuthology.orchestra.run.smithi032.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-12-05T17:30:19.817 DEBUG:teuthology.parallel:result is None 2020-12-05T17:30:19.894 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up ceph-mgr-k8sevents (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:19.917 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-mds (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:19.954 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mgr-cephadm (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:19.995 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up ceph-mgr-dashboard (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:20.113 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up ceph (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:20.122 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:20.142 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-12-05T17:30:20.143 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-12-05T17:30:20.214 INFO:teuthology.orchestra.run.smithi061.stdout:Processing triggers for systemd (237-3ubuntu10.43) ... 2020-12-05T17:30:20.265 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mgr-rook (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:20.281 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-mgr-modules-core (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:20.390 INFO:teuthology.orchestra.run.smithi061.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-12-05T17:30:20.407 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mds (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:20.427 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-mon (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:20.440 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-pastescript (2.0.2-2) ... 2020-12-05T17:30:20.631 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-12-05T17:30:20.631 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-12-05T17:30:20.645 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-12-05T17:30:20.645 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-12-05T17:30:20.775 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-rgw (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:20.892 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mon (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:20.935 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up rbd-fuse (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:20.945 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-mgr-k8sevents (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:21.086 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-rbd (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:21.096 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-mgr-dashboard (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:21.111 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-12-05T17:30:21.112 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-12-05T17:30:21.195 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-common (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:21.247 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:21.390 INFO:teuthology.orchestra.run.smithi036.stdout:Processing triggers for systemd (237-3ubuntu10.43) ... 2020-12-05T17:30:21.393 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mgr-k8sevents (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:21.519 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mgr-dashboard (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:21.600 INFO:teuthology.orchestra.run.smithi036.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-12-05T17:30:21.679 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:21.755 INFO:teuthology.orchestra.run.smithi002.stdout:Adding group ceph....done 2020-12-05T17:30:21.822 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for systemd (237-3ubuntu10.43) ... 2020-12-05T17:30:22.015 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-12-05T17:30:22.183 INFO:teuthology.orchestra.run.smithi002.stdout:Adding system user ceph....done 2020-12-05T17:30:22.384 INFO:teuthology.orchestra.run.smithi002.stdout:Setting system user ceph properties....done 2020-12-05T17:30:22.535 INFO:teuthology.orchestra.run.smithi061.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-12-05T17:30:22.547 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-12-05T17:30:22.578 INFO:teuthology.orchestra.run.smithi061.stdout:ureadahead will be reprofiled on next reboot 2020-12-05T17:30:22.628 INFO:teuthology.orchestra.run.smithi061.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-12-05T17:30:22.774 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-12-05T17:30:23.040 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up radosgw (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:23.067 INFO:teuthology.orchestra.run.smithi061.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-12-05T17:30:23.069 INFO:teuthology.orchestra.run.smithi061:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2020-12-05T17:30:23.163 INFO:teuthology.orchestra.run.smithi061.stdout:Reading package lists... 2020-12-05T17:30:23.273 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-12-05T17:30:23.275 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-12-05T17:30:23.429 INFO:teuthology.orchestra.run.smithi061.stdout:Building dependency tree... 2020-12-05T17:30:23.430 INFO:teuthology.orchestra.run.smithi061.stdout:Reading state information... 2020-12-05T17:30:23.614 INFO:teuthology.orchestra.run.smithi061.stdout:libtool-bin is already the newest version (2.4.6-2). 2020-12-05T17:30:23.614 INFO:teuthology.orchestra.run.smithi061.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2020-12-05T17:30:23.615 INFO:teuthology.orchestra.run.smithi061.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2020-12-05T17:30:23.615 INFO:teuthology.orchestra.run.smithi061.stdout:The following additional packages will be installed: 2020-12-05T17:30:23.616 INFO:teuthology.orchestra.run.smithi061.stdout: libbison-dev libfl-dev libfl2 zlib1g-dev 2020-12-05T17:30:23.617 INFO:teuthology.orchestra.run.smithi061.stdout:Suggested packages: 2020-12-05T17:30:23.617 INFO:teuthology.orchestra.run.smithi061.stdout: bison-doc flex-doc indent-doc libssl-doc 2020-12-05T17:30:23.644 INFO:teuthology.orchestra.run.smithi061.stdout:The following NEW packages will be installed: 2020-12-05T17:30:23.645 INFO:teuthology.orchestra.run.smithi061.stdout: bison dump flex indent libaio-dev libbison-dev libelf-dev libfl-dev libfl2 2020-12-05T17:30:23.646 INFO:teuthology.orchestra.run.smithi061.stdout: libssl-dev zlib1g-dev 2020-12-05T17:30:23.658 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-test (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:23.728 INFO:teuthology.orchestra.run.smithi036.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-12-05T17:30:23.771 INFO:teuthology.orchestra.run.smithi036.stdout:ureadahead will be reprofiled on next reboot 2020-12-05T17:30:23.818 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-base (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:23.829 INFO:teuthology.orchestra.run.smithi036.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-12-05T17:30:23.831 INFO:teuthology.orchestra.run.smithi061.stdout:0 upgraded, 11 newly installed, 0 to remove and 3 not upgraded. 2020-12-05T17:30:23.832 INFO:teuthology.orchestra.run.smithi061.stdout:Need to get 2,960 kB of archives. 2020-12-05T17:30:23.832 INFO:teuthology.orchestra.run.smithi061.stdout:After this operation, 12.2 MB of additional disk space will be used. 2020-12-05T17:30:23.833 INFO:teuthology.orchestra.run.smithi061.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2020-12-05T17:30:24.144 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-12-05T17:30:24.157 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-12-05T17:30:24.186 INFO:teuthology.orchestra.run.smithi053.stdout:ureadahead will be reprofiled on next reboot 2020-12-05T17:30:24.244 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-12-05T17:30:24.266 INFO:teuthology.orchestra.run.smithi061.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2020-12-05T17:30:24.299 INFO:teuthology.orchestra.run.smithi036.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-12-05T17:30:24.301 INFO:teuthology.orchestra.run.smithi036:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2020-12-05T17:30:24.345 INFO:teuthology.orchestra.run.smithi061.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2020-12-05T17:30:24.396 INFO:teuthology.orchestra.run.smithi061.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2020-12-05T17:30:24.413 INFO:teuthology.orchestra.run.smithi061.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2020-12-05T17:30:24.425 INFO:teuthology.orchestra.run.smithi061.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2020-12-05T17:30:24.426 INFO:teuthology.orchestra.run.smithi061.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2020-12-05T17:30:24.440 INFO:teuthology.orchestra.run.smithi036.stdout:Reading package lists... 2020-12-05T17:30:24.441 INFO:teuthology.orchestra.run.smithi061.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2020-12-05T17:30:24.446 INFO:teuthology.orchestra.run.smithi061.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2020-12-05T17:30:24.447 INFO:teuthology.orchestra.run.smithi061.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2020-12-05T17:30:24.448 INFO:teuthology.orchestra.run.smithi061.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.6 [1,566 kB] 2020-12-05T17:30:24.462 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-mgr (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:24.674 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-12-05T17:30:24.674 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-12-05T17:30:24.703 INFO:teuthology.orchestra.run.smithi036.stdout:Building dependency tree... 2020-12-05T17:30:24.704 INFO:teuthology.orchestra.run.smithi036.stdout:Reading state information... 2020-12-05T17:30:24.708 INFO:teuthology.orchestra.run.smithi053.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-12-05T17:30:24.712 INFO:teuthology.orchestra.run.smithi053:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2020-12-05T17:30:24.795 INFO:teuthology.orchestra.run.smithi053.stdout:Reading package lists... 2020-12-05T17:30:24.880 INFO:teuthology.orchestra.run.smithi036.stdout:libtool-bin is already the newest version (2.4.6-2). 2020-12-05T17:30:24.881 INFO:teuthology.orchestra.run.smithi036.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2020-12-05T17:30:24.881 INFO:teuthology.orchestra.run.smithi036.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2020-12-05T17:30:24.881 INFO:teuthology.orchestra.run.smithi036.stdout:The following additional packages will be installed: 2020-12-05T17:30:24.882 INFO:teuthology.orchestra.run.smithi036.stdout: libbison-dev libfl-dev libfl2 zlib1g-dev 2020-12-05T17:30:24.883 INFO:teuthology.orchestra.run.smithi036.stdout:Suggested packages: 2020-12-05T17:30:24.883 INFO:teuthology.orchestra.run.smithi036.stdout: bison-doc flex-doc indent-doc libssl-doc 2020-12-05T17:30:24.909 INFO:teuthology.orchestra.run.smithi036.stdout:The following NEW packages will be installed: 2020-12-05T17:30:24.910 INFO:teuthology.orchestra.run.smithi036.stdout: bison dump flex indent libaio-dev libbison-dev libelf-dev libfl-dev libfl2 2020-12-05T17:30:24.912 INFO:teuthology.orchestra.run.smithi036.stdout: libssl-dev zlib1g-dev 2020-12-05T17:30:24.938 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-mgr-diskprediction-local (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:24.945 INFO:teuthology.orchestra.run.smithi061.stdout:Fetched 2,960 kB in 1s (3,371 kB/s) 2020-12-05T17:30:25.062 INFO:teuthology.orchestra.run.smithi053.stdout:Building dependency tree... 2020-12-05T17:30:25.062 INFO:teuthology.orchestra.run.smithi053.stdout:Reading state information... 2020-12-05T17:30:25.084 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package flex. 2020-12-05T17:30:25.089 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-osd (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:25.092 INFO:teuthology.orchestra.run.smithi036.stdout:0 upgraded, 11 newly installed, 0 to remove and 3 not upgraded. 2020-12-05T17:30:25.093 INFO:teuthology.orchestra.run.smithi036.stdout:Need to get 2,960 kB of archives. 2020-12-05T17:30:25.093 INFO:teuthology.orchestra.run.smithi036.stdout:After this operation, 12.2 MB of additional disk space will be used. 2020-12-05T17:30:25.093 INFO:teuthology.orchestra.run.smithi036.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2020-12-05T17:30:25.135 INFO:teuthology.orchestra.run.smithi061.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 145453 files and directories currently installed.) 2020-12-05T17:30:25.140 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2020-12-05T17:30:25.182 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking flex (2.6.4-6) ... 2020-12-05T17:30:25.252 INFO:teuthology.orchestra.run.smithi053.stdout:libtool-bin is already the newest version (2.4.6-2). 2020-12-05T17:30:25.252 INFO:teuthology.orchestra.run.smithi053.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2020-12-05T17:30:25.253 INFO:teuthology.orchestra.run.smithi053.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2020-12-05T17:30:25.253 INFO:teuthology.orchestra.run.smithi053.stdout:The following additional packages will be installed: 2020-12-05T17:30:25.253 INFO:teuthology.orchestra.run.smithi053.stdout: libbison-dev libfl-dev libfl2 zlib1g-dev 2020-12-05T17:30:25.255 INFO:teuthology.orchestra.run.smithi053.stdout:Suggested packages: 2020-12-05T17:30:25.255 INFO:teuthology.orchestra.run.smithi053.stdout: bison-doc flex-doc indent-doc libssl-doc 2020-12-05T17:30:25.281 INFO:teuthology.orchestra.run.smithi053.stdout:The following NEW packages will be installed: 2020-12-05T17:30:25.282 INFO:teuthology.orchestra.run.smithi053.stdout: bison dump flex indent libaio-dev libbison-dev libelf-dev libfl-dev libfl2 2020-12-05T17:30:25.283 INFO:teuthology.orchestra.run.smithi053.stdout: libssl-dev zlib1g-dev 2020-12-05T17:30:25.465 INFO:teuthology.orchestra.run.smithi053.stdout:0 upgraded, 11 newly installed, 0 to remove and 3 not upgraded. 2020-12-05T17:30:25.465 INFO:teuthology.orchestra.run.smithi053.stdout:Need to get 2,960 kB of archives. 2020-12-05T17:30:25.465 INFO:teuthology.orchestra.run.smithi053.stdout:After this operation, 12.2 MB of additional disk space will be used. 2020-12-05T17:30:25.465 INFO:teuthology.orchestra.run.smithi053.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2020-12-05T17:30:25.525 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-12-05T17:30:25.526 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-12-05T17:30:25.543 INFO:teuthology.orchestra.run.smithi036.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2020-12-05T17:30:25.570 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package libbison-dev:amd64. 2020-12-05T17:30:25.584 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../01-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-12-05T17:30:25.625 INFO:teuthology.orchestra.run.smithi036.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2020-12-05T17:30:25.628 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-12-05T17:30:25.680 INFO:teuthology.orchestra.run.smithi036.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2020-12-05T17:30:25.714 INFO:teuthology.orchestra.run.smithi036.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2020-12-05T17:30:25.720 INFO:teuthology.orchestra.run.smithi036.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2020-12-05T17:30:25.721 INFO:teuthology.orchestra.run.smithi036.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2020-12-05T17:30:25.736 INFO:teuthology.orchestra.run.smithi036.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2020-12-05T17:30:25.740 INFO:teuthology.orchestra.run.smithi036.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2020-12-05T17:30:25.741 INFO:teuthology.orchestra.run.smithi036.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2020-12-05T17:30:25.742 INFO:teuthology.orchestra.run.smithi036.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.6 [1,566 kB] 2020-12-05T17:30:25.775 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-mgr-cephadm (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:25.909 INFO:teuthology.orchestra.run.smithi053.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2020-12-05T17:30:25.923 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package bison. 2020-12-05T17:30:25.934 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:25.938 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../02-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-12-05T17:30:25.977 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2020-12-05T17:30:25.989 INFO:teuthology.orchestra.run.smithi053.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2020-12-05T17:30:26.040 INFO:teuthology.orchestra.run.smithi053.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2020-12-05T17:30:26.057 INFO:teuthology.orchestra.run.smithi053.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2020-12-05T17:30:26.069 INFO:teuthology.orchestra.run.smithi053.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2020-12-05T17:30:26.070 INFO:teuthology.orchestra.run.smithi053.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2020-12-05T17:30:26.077 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-mgr-rook (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:26.085 INFO:teuthology.orchestra.run.smithi053.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2020-12-05T17:30:26.090 INFO:teuthology.orchestra.run.smithi053.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2020-12-05T17:30:26.091 INFO:teuthology.orchestra.run.smithi053.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2020-12-05T17:30:26.092 INFO:teuthology.orchestra.run.smithi053.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.6 [1,566 kB] 2020-12-05T17:30:26.240 INFO:teuthology.orchestra.run.smithi036.stdout:Fetched 2,960 kB in 1s (3,251 kB/s) 2020-12-05T17:30:26.244 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-mds (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:26.377 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package dump. 2020-12-05T17:30:26.391 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../03-dump_0.4b46-3_amd64.deb ... 2020-12-05T17:30:26.402 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package flex. 2020-12-05T17:30:26.435 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking dump (0.4b46-3) ... 2020-12-05T17:30:26.455 INFO:teuthology.orchestra.run.smithi036.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 145453 files and directories currently installed.) 2020-12-05T17:30:26.460 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2020-12-05T17:30:26.480 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-12-05T17:30:26.480 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-12-05T17:30:26.501 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking flex (2.6.4-6) ... 2020-12-05T17:30:26.593 INFO:teuthology.orchestra.run.smithi053.stdout:Fetched 2,960 kB in 1s (3,343 kB/s) 2020-12-05T17:30:26.717 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package indent. 2020-12-05T17:30:26.732 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../04-indent_2.2.11-5_amd64.deb ... 2020-12-05T17:30:26.742 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package flex. 2020-12-05T17:30:26.775 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking indent (2.2.11-5) ... 2020-12-05T17:30:26.779 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-mon (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:26.794 INFO:teuthology.orchestra.run.smithi053.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 145453 files and directories currently installed.) 2020-12-05T17:30:26.798 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2020-12-05T17:30:26.840 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking flex (2.6.4-6) ... 2020-12-05T17:30:26.956 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libbison-dev:amd64. 2020-12-05T17:30:26.971 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../01-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-12-05T17:30:26.981 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-12-05T17:30:26.982 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-12-05T17:30:27.014 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-12-05T17:30:27.053 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package libaio-dev:amd64. 2020-12-05T17:30:27.067 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../05-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2020-12-05T17:30:27.111 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-12-05T17:30:27.247 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-mgr-k8sevents (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:27.253 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libbison-dev:amd64. 2020-12-05T17:30:27.268 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../01-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-12-05T17:30:27.320 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-12-05T17:30:27.364 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2020-12-05T17:30:27.367 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package bison. 2020-12-05T17:30:27.367 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-mgr-dashboard (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:27.378 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../06-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2020-12-05T17:30:27.381 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../02-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-12-05T17:30:27.422 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-12-05T17:30:27.429 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2020-12-05T17:30:27.499 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph (15.2.6-607-g3b1861ac-1bionic) ... 2020-12-05T17:30:27.667 INFO:teuthology.orchestra.run.smithi002.stdout:Processing triggers for systemd (237-3ubuntu10.43) ... 2020-12-05T17:30:27.681 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package bison. 2020-12-05T17:30:27.695 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../02-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-12-05T17:30:27.744 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2020-12-05T17:30:27.826 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package libelf-dev:amd64. 2020-12-05T17:30:27.832 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package dump. 2020-12-05T17:30:27.840 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../07-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2020-12-05T17:30:27.847 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../03-dump_0.4b46-3_amd64.deb ... 2020-12-05T17:30:27.868 INFO:teuthology.orchestra.run.smithi002.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-12-05T17:30:27.884 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-12-05T17:30:27.891 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking dump (0.4b46-3) ... 2020-12-05T17:30:28.204 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package indent. 2020-12-05T17:30:28.221 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../04-indent_2.2.11-5_amd64.deb ... 2020-12-05T17:30:28.262 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package libfl2:amd64. 2020-12-05T17:30:28.270 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking indent (2.2.11-5) ... 2020-12-05T17:30:28.278 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../08-libfl2_2.6.4-6_amd64.deb ... 2020-12-05T17:30:28.363 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2020-12-05T17:30:28.389 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package dump. 2020-12-05T17:30:28.402 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../03-dump_0.4b46-3_amd64.deb ... 2020-12-05T17:30:28.447 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking dump (0.4b46-3) ... 2020-12-05T17:30:28.572 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libaio-dev:amd64. 2020-12-05T17:30:28.589 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../05-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2020-12-05T17:30:28.615 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package libfl-dev:amd64. 2020-12-05T17:30:28.630 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../09-libfl-dev_2.6.4-6_amd64.deb ... 2020-12-05T17:30:28.631 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-12-05T17:30:28.690 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2020-12-05T17:30:28.734 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package indent. 2020-12-05T17:30:28.750 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../04-indent_2.2.11-5_amd64.deb ... 2020-12-05T17:30:28.792 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking indent (2.2.11-5) ... 2020-12-05T17:30:28.884 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2020-12-05T17:30:28.901 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../06-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2020-12-05T17:30:28.951 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-12-05T17:30:28.959 INFO:teuthology.orchestra.run.smithi061.stdout:Selecting previously unselected package libssl-dev:amd64. 2020-12-05T17:30:28.976 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing to unpack .../10-libssl-dev_1.1.1-1ubuntu2.1~18.04.6_amd64.deb ... 2020-12-05T17:30:29.018 INFO:teuthology.orchestra.run.smithi061.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-12-05T17:30:29.061 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libaio-dev:amd64. 2020-12-05T17:30:29.075 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../05-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2020-12-05T17:30:29.128 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-12-05T17:30:29.237 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libelf-dev:amd64. 2020-12-05T17:30:29.252 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../07-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2020-12-05T17:30:29.295 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-12-05T17:30:29.398 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2020-12-05T17:30:29.414 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../06-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2020-12-05T17:30:29.464 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-12-05T17:30:29.479 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up flex (2.6.4-6) ... 2020-12-05T17:30:29.606 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up dump (0.4b46-3) ... 2020-12-05T17:30:29.706 INFO:teuthology.orchestra.run.smithi061.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2020-12-05T17:30:29.749 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libfl2:amd64. 2020-12-05T17:30:29.759 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libelf-dev:amd64. 2020-12-05T17:30:29.765 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../08-libfl2_2.6.4-6_amd64.deb ... 2020-12-05T17:30:29.773 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../07-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2020-12-05T17:30:29.783 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-12-05T17:30:29.817 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-12-05T17:30:29.849 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2020-12-05T17:30:29.917 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up indent (2.2.11-5) ... 2020-12-05T17:30:29.996 INFO:teuthology.orchestra.run.smithi002.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-12-05T17:30:30.031 INFO:teuthology.orchestra.run.smithi002.stdout:ureadahead will be reprofiled on next reboot 2020-12-05T17:30:30.060 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-12-05T17:30:30.080 INFO:teuthology.orchestra.run.smithi002.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-12-05T17:30:30.104 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libfl-dev:amd64. 2020-12-05T17:30:30.119 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../09-libfl-dev_2.6.4-6_amd64.deb ... 2020-12-05T17:30:30.162 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2020-12-05T17:30:30.179 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libfl2:amd64. 2020-12-05T17:30:30.193 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../08-libfl2_2.6.4-6_amd64.deb ... 2020-12-05T17:30:30.219 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2020-12-05T17:30:30.279 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2020-12-05T17:30:30.345 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2020-12-05T17:30:30.432 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libssl-dev:amd64. 2020-12-05T17:30:30.435 INFO:teuthology.orchestra.run.smithi061.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2020-12-05T17:30:30.447 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../10-libssl-dev_1.1.1-1ubuntu2.1~18.04.6_amd64.deb ... 2020-12-05T17:30:30.499 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-12-05T17:30:30.505 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-12-05T17:30:30.547 INFO:teuthology.orchestra.run.smithi002.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-12-05T17:30:30.548 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libfl-dev:amd64. 2020-12-05T17:30:30.549 INFO:teuthology.orchestra.run.smithi002:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2020-12-05T17:30:30.563 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../09-libfl-dev_2.6.4-6_amd64.deb ... 2020-12-05T17:30:30.631 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-12-05T17:30:30.632 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2020-12-05T17:30:30.637 INFO:teuthology.orchestra.run.smithi002.stdout:Reading package lists... 2020-12-05T17:30:30.765 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-12-05T17:30:30.892 INFO:teuthology.orchestra.run.smithi061.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2020-12-05T17:30:30.892 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libssl-dev:amd64. 2020-12-05T17:30:30.900 INFO:teuthology.orchestra.run.smithi002.stdout:Building dependency tree... 2020-12-05T17:30:30.901 INFO:teuthology.orchestra.run.smithi002.stdout:Reading state information... 2020-12-05T17:30:30.906 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../10-libssl-dev_1.1.1-1ubuntu2.1~18.04.6_amd64.deb ... 2020-12-05T17:30:30.951 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-12-05T17:30:30.967 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up flex (2.6.4-6) ... 2020-12-05T17:30:31.035 INFO:teuthology.orchestra.run.smithi061.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-12-05T17:30:31.077 INFO:teuthology.orchestra.run.smithi002.stdout:libtool-bin is already the newest version (2.4.6-2). 2020-12-05T17:30:31.078 INFO:teuthology.orchestra.run.smithi002.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2020-12-05T17:30:31.078 INFO:teuthology.orchestra.run.smithi002.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2020-12-05T17:30:31.079 INFO:teuthology.orchestra.run.smithi002.stdout:The following additional packages will be installed: 2020-12-05T17:30:31.079 INFO:teuthology.orchestra.run.smithi002.stdout: libbison-dev libfl-dev libfl2 zlib1g-dev 2020-12-05T17:30:31.080 INFO:teuthology.orchestra.run.smithi002.stdout:Suggested packages: 2020-12-05T17:30:31.081 INFO:teuthology.orchestra.run.smithi002.stdout: bison-doc flex-doc indent-doc libssl-doc 2020-12-05T17:30:31.108 INFO:teuthology.orchestra.run.smithi002.stdout:The following NEW packages will be installed: 2020-12-05T17:30:31.109 INFO:teuthology.orchestra.run.smithi002.stdout: bison dump flex indent libaio-dev libbison-dev libelf-dev libfl-dev libfl2 2020-12-05T17:30:31.110 INFO:teuthology.orchestra.run.smithi002.stdout: libssl-dev zlib1g-dev 2020-12-05T17:30:31.120 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up dump (0.4b46-3) ... 2020-12-05T17:30:31.234 INFO:teuthology.orchestra.run.smithi036.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2020-12-05T17:30:31.290 INFO:teuthology.orchestra.run.smithi002.stdout:0 upgraded, 11 newly installed, 0 to remove and 3 not upgraded. 2020-12-05T17:30:31.291 INFO:teuthology.orchestra.run.smithi002.stdout:Need to get 2,960 kB of archives. 2020-12-05T17:30:31.291 INFO:teuthology.orchestra.run.smithi002.stdout:After this operation, 12.2 MB of additional disk space will be used. 2020-12-05T17:30:31.291 INFO:teuthology.orchestra.run.smithi002.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2020-12-05T17:30:31.305 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-12-05T17:30:31.432 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up indent (2.2.11-5) ... 2020-12-05T17:30:31.473 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up flex (2.6.4-6) ... 2020-12-05T17:30:31.558 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-12-05T17:30:31.599 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up dump (0.4b46-3) ... 2020-12-05T17:30:31.684 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2020-12-05T17:30:31.702 INFO:teuthology.orchestra.run.smithi053.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2020-12-05T17:30:31.749 INFO:teuthology.orchestra.run.smithi002.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2020-12-05T17:30:31.768 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-12-05T17:30:31.801 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2020-12-05T17:30:31.831 INFO:teuthology.orchestra.run.smithi002.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2020-12-05T17:30:31.875 INFO:teuthology.orchestra.run.smithi036.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2020-12-05T17:30:31.880 INFO:teuthology.orchestra.run.smithi002.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2020-12-05T17:30:31.894 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up indent (2.2.11-5) ... 2020-12-05T17:30:31.896 INFO:teuthology.orchestra.run.smithi002.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2020-12-05T17:30:31.908 INFO:teuthology.orchestra.run.smithi002.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2020-12-05T17:30:31.909 INFO:teuthology.orchestra.run.smithi002.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2020-12-05T17:30:31.926 INFO:teuthology.orchestra.run.smithi002.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2020-12-05T17:30:31.931 INFO:teuthology.orchestra.run.smithi002.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2020-12-05T17:30:31.932 INFO:teuthology.orchestra.run.smithi002.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2020-12-05T17:30:31.932 INFO:teuthology.orchestra.run.smithi002.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.6 [1,566 kB] 2020-12-05T17:30:31.937 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-12-05T17:30:32.019 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-12-05T17:30:32.037 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-12-05T17:30:32.139 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-12-05T17:30:32.146 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2020-12-05T17:30:32.255 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2020-12-05T17:30:32.256 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2020-12-05T17:30:32.336 INFO:teuthology.orchestra.run.smithi053.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2020-12-05T17:30:32.383 INFO:teuthology.orchestra.run.smithi036.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-12-05T17:30:32.398 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-12-05T17:30:32.421 INFO:teuthology.orchestra.run.smithi002.stdout:Fetched 2,960 kB in 1s (3,292 kB/s) 2020-12-05T17:30:32.499 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-12-05T17:30:32.595 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package flex. 2020-12-05T17:30:32.600 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-12-05T17:30:32.650 INFO:teuthology.orchestra.run.smithi002.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 145453 files and directories currently installed.) 2020-12-05T17:30:32.654 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2020-12-05T17:30:32.719 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking flex (2.6.4-6) ... 2020-12-05T17:30:32.726 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2020-12-05T17:30:32.852 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-12-05T17:30:32.880 INFO:teuthology.orchestra.run.smithi061.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2020-12-05T17:30:33.143 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libbison-dev:amd64. 2020-12-05T17:30:33.160 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../01-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-12-05T17:30:33.201 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-12-05T17:30:33.223 INFO:teuthology.orchestra.run.smithi061.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-12-05T17:30:33.604 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package bison. 2020-12-05T17:30:33.619 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../02-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-12-05T17:30:33.667 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2020-12-05T17:30:33.686 INFO:teuthology.orchestra.run.smithi061.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-12-05T17:30:33.688 DEBUG:teuthology.parallel:result is None 2020-12-05T17:30:34.074 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package dump. 2020-12-05T17:30:34.090 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../03-dump_0.4b46-3_amd64.deb ... 2020-12-05T17:30:34.124 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking dump (0.4b46-3) ... 2020-12-05T17:30:34.236 INFO:teuthology.orchestra.run.smithi036.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2020-12-05T17:30:34.436 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package indent. 2020-12-05T17:30:34.452 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../04-indent_2.2.11-5_amd64.deb ... 2020-12-05T17:30:34.494 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking indent (2.2.11-5) ... 2020-12-05T17:30:34.563 INFO:teuthology.orchestra.run.smithi036.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-12-05T17:30:34.698 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2020-12-05T17:30:34.864 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libaio-dev:amd64. 2020-12-05T17:30:34.881 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../05-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2020-12-05T17:30:34.922 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-12-05T17:30:35.016 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-12-05T17:30:35.133 INFO:teuthology.orchestra.run.smithi036.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-12-05T17:30:35.135 DEBUG:teuthology.parallel:result is None 2020-12-05T17:30:35.169 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2020-12-05T17:30:35.183 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../06-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2020-12-05T17:30:35.219 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-12-05T17:30:35.482 INFO:teuthology.orchestra.run.smithi053.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-12-05T17:30:35.484 DEBUG:teuthology.parallel:result is None 2020-12-05T17:30:35.572 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libelf-dev:amd64. 2020-12-05T17:30:35.588 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../07-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2020-12-05T17:30:35.630 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-12-05T17:30:36.000 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libfl2:amd64. 2020-12-05T17:30:36.014 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../08-libfl2_2.6.4-6_amd64.deb ... 2020-12-05T17:30:36.084 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2020-12-05T17:30:36.370 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libfl-dev:amd64. 2020-12-05T17:30:36.386 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../09-libfl-dev_2.6.4-6_amd64.deb ... 2020-12-05T17:30:36.419 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2020-12-05T17:30:36.722 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libssl-dev:amd64. 2020-12-05T17:30:36.739 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../10-libssl-dev_1.1.1-1ubuntu2.1~18.04.6_amd64.deb ... 2020-12-05T17:30:36.805 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-12-05T17:30:37.305 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up flex (2.6.4-6) ... 2020-12-05T17:30:37.452 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up dump (0.4b46-3) ... 2020-12-05T17:30:37.563 INFO:teuthology.orchestra.run.smithi002.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2020-12-05T17:30:37.654 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-12-05T17:30:37.780 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up indent (2.2.11-5) ... 2020-12-05T17:30:37.931 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-12-05T17:30:38.099 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2020-12-05T17:30:38.258 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2020-12-05T17:30:38.356 INFO:teuthology.orchestra.run.smithi002.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2020-12-05T17:30:38.426 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-12-05T17:30:38.587 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-12-05T17:30:38.746 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-12-05T17:30:38.905 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2020-12-05T17:30:39.032 INFO:teuthology.orchestra.run.smithi002.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-12-05T17:30:40.876 INFO:teuthology.orchestra.run.smithi002.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2020-12-05T17:30:41.303 INFO:teuthology.orchestra.run.smithi002.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-12-05T17:30:41.770 INFO:teuthology.orchestra.run.smithi002.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-12-05T17:30:41.772 DEBUG:teuthology.parallel:result is None 2020-12-05T17:30:41.772 INFO:teuthology.orchestra.run.smithi032:> sudo lsb_release -is 2020-12-05T17:30:41.829 INFO:teuthology.orchestra.run.smithi032.stdout:Ubuntu 2020-12-05T17:30:41.835 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-12-05T17:30:41.835 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=3b1861acb8137d863184795ccadbd5d0cb026674 2020-12-05T17:30:42.005 INFO:teuthology.orchestra.run.smithi032:> dpkg-query -W -f '${Version}' ceph 2020-12-05T17:30:42.030 INFO:teuthology.orchestra.run.smithi032.stdout:15.2.6-607-g3b1861ac-1bionic 2020-12-05T17:30:42.031 INFO:teuthology.packaging:The installed version of ceph is 15.2.6-607-g3b1861ac-1bionic 2020-12-05T17:30:42.031 INFO:teuthology.task.install:The correct ceph version 15.2.6-607-g3b1861ac-1bionic is installed. 2020-12-05T17:30:42.032 INFO:teuthology.orchestra.run.smithi061:> sudo lsb_release -is 2020-12-05T17:30:42.086 INFO:teuthology.orchestra.run.smithi061.stdout:Ubuntu 2020-12-05T17:30:42.091 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-12-05T17:30:42.092 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=3b1861acb8137d863184795ccadbd5d0cb026674 2020-12-05T17:30:42.257 INFO:teuthology.orchestra.run.smithi061:> dpkg-query -W -f '${Version}' ceph 2020-12-05T17:30:42.282 INFO:teuthology.orchestra.run.smithi061.stdout:15.2.6-607-g3b1861ac-1bionic 2020-12-05T17:30:42.283 INFO:teuthology.packaging:The installed version of ceph is 15.2.6-607-g3b1861ac-1bionic 2020-12-05T17:30:42.283 INFO:teuthology.task.install:The correct ceph version 15.2.6-607-g3b1861ac-1bionic is installed. 2020-12-05T17:30:42.284 INFO:teuthology.orchestra.run.smithi036:> sudo lsb_release -is 2020-12-05T17:30:42.343 INFO:teuthology.orchestra.run.smithi036.stdout:Ubuntu 2020-12-05T17:30:42.348 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-12-05T17:30:42.349 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=3b1861acb8137d863184795ccadbd5d0cb026674 2020-12-05T17:30:42.514 INFO:teuthology.orchestra.run.smithi036:> dpkg-query -W -f '${Version}' ceph 2020-12-05T17:30:42.539 INFO:teuthology.orchestra.run.smithi036.stdout:15.2.6-607-g3b1861ac-1bionic 2020-12-05T17:30:42.540 INFO:teuthology.packaging:The installed version of ceph is 15.2.6-607-g3b1861ac-1bionic 2020-12-05T17:30:42.540 INFO:teuthology.task.install:The correct ceph version 15.2.6-607-g3b1861ac-1bionic is installed. 2020-12-05T17:30:42.540 INFO:teuthology.orchestra.run.smithi002:> sudo lsb_release -is 2020-12-05T17:30:42.598 INFO:teuthology.orchestra.run.smithi002.stdout:Ubuntu 2020-12-05T17:30:42.603 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-12-05T17:30:42.604 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=3b1861acb8137d863184795ccadbd5d0cb026674 2020-12-05T17:30:42.783 INFO:teuthology.orchestra.run.smithi002:> dpkg-query -W -f '${Version}' ceph 2020-12-05T17:30:42.807 INFO:teuthology.orchestra.run.smithi002.stdout:15.2.6-607-g3b1861ac-1bionic 2020-12-05T17:30:42.808 INFO:teuthology.packaging:The installed version of ceph is 15.2.6-607-g3b1861ac-1bionic 2020-12-05T17:30:42.809 INFO:teuthology.task.install:The correct ceph version 15.2.6-607-g3b1861ac-1bionic is installed. 2020-12-05T17:30:42.809 INFO:teuthology.orchestra.run.smithi053:> sudo lsb_release -is 2020-12-05T17:30:42.867 INFO:teuthology.orchestra.run.smithi053.stdout:Ubuntu 2020-12-05T17:30:42.872 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-12-05T17:30:42.873 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=3b1861acb8137d863184795ccadbd5d0cb026674 2020-12-05T17:30:43.057 INFO:teuthology.orchestra.run.smithi053:> dpkg-query -W -f '${Version}' ceph 2020-12-05T17:30:43.081 INFO:teuthology.orchestra.run.smithi053.stdout:15.2.6-607-g3b1861ac-1bionic 2020-12-05T17:30:43.081 INFO:teuthology.packaging:The installed version of ceph is 15.2.6-607-g3b1861ac-1bionic 2020-12-05T17:30:43.081 INFO:teuthology.task.install:The correct ceph version 15.2.6-607-g3b1861ac-1bionic is installed. 2020-12-05T17:30:43.082 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2020-12-05T17:30:43.082 INFO:teuthology.orchestra.run.smithi032:> set -ex 2020-12-05T17:30:43.083 INFO:teuthology.orchestra.run.smithi032:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-12-05T17:30:43.087 INFO:teuthology.orchestra.run.smithi032.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-12-05T17:30:43.097 INFO:teuthology.orchestra.run.smithi032.stderr:24+1 records in 2020-12-05T17:30:43.098 INFO:teuthology.orchestra.run.smithi032.stderr:24+1 records out 2020-12-05T17:30:43.098 INFO:teuthology.orchestra.run.smithi032.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.000219397 s, 57.7 MB/s 2020-12-05T17:30:43.099 INFO:teuthology.orchestra.run.smithi061:> set -ex 2020-12-05T17:30:43.099 INFO:teuthology.orchestra.run.smithi061:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-12-05T17:30:43.106 INFO:teuthology.orchestra.run.smithi061.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-12-05T17:30:43.114 INFO:teuthology.orchestra.run.smithi061.stderr:24+1 records in 2020-12-05T17:30:43.115 INFO:teuthology.orchestra.run.smithi061.stderr:24+1 records out 2020-12-05T17:30:43.115 INFO:teuthology.orchestra.run.smithi061.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.000227847 s, 55.6 MB/s 2020-12-05T17:30:43.116 INFO:teuthology.orchestra.run.smithi036:> set -ex 2020-12-05T17:30:43.117 INFO:teuthology.orchestra.run.smithi036:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-12-05T17:30:43.123 INFO:teuthology.orchestra.run.smithi036.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-12-05T17:30:43.132 INFO:teuthology.orchestra.run.smithi036.stderr:24+1 records in 2020-12-05T17:30:43.133 INFO:teuthology.orchestra.run.smithi036.stderr:24+1 records out 2020-12-05T17:30:43.133 INFO:teuthology.orchestra.run.smithi036.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.000220193 s, 57.5 MB/s 2020-12-05T17:30:43.135 INFO:teuthology.orchestra.run.smithi002:> set -ex 2020-12-05T17:30:43.135 INFO:teuthology.orchestra.run.smithi002:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-12-05T17:30:43.141 INFO:teuthology.orchestra.run.smithi002.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-12-05T17:30:43.151 INFO:teuthology.orchestra.run.smithi002.stderr:24+1 records in 2020-12-05T17:30:43.151 INFO:teuthology.orchestra.run.smithi002.stderr:24+1 records out 2020-12-05T17:30:43.151 INFO:teuthology.orchestra.run.smithi002.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.000217134 s, 58.3 MB/s 2020-12-05T17:30:43.152 INFO:teuthology.orchestra.run.smithi053:> set -ex 2020-12-05T17:30:43.152 INFO:teuthology.orchestra.run.smithi053:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-12-05T17:30:43.157 INFO:teuthology.orchestra.run.smithi053.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-12-05T17:30:43.167 INFO:teuthology.orchestra.run.smithi053.stderr:24+1 records in 2020-12-05T17:30:43.167 INFO:teuthology.orchestra.run.smithi053.stderr:24+1 records out 2020-12-05T17:30:43.167 INFO:teuthology.orchestra.run.smithi053.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.000203215 s, 62.3 MB/s 2020-12-05T17:30:43.168 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2020-12-05T17:30:43.169 INFO:teuthology.orchestra.run.smithi032:> set -ex 2020-12-05T17:30:43.169 INFO:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/daemon-helper 2020-12-05T17:30:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-12-05T17:30:43.183 INFO:teuthology.orchestra.run.smithi032.stderr:5+1 records in 2020-12-05T17:30:43.184 INFO:teuthology.orchestra.run.smithi032.stderr:5+1 records out 2020-12-05T17:30:43.184 INFO:teuthology.orchestra.run.smithi032.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 0.00010503 s, 29.1 MB/s 2020-12-05T17:30:43.185 INFO:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-12-05T17:30:43.244 INFO:teuthology.orchestra.run.smithi061:> set -ex 2020-12-05T17:30:43.244 INFO:teuthology.orchestra.run.smithi061:> sudo dd of=/usr/bin/daemon-helper 2020-12-05T17:30:43.249 INFO:teuthology.orchestra.run.smithi061.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-12-05T17:30:43.258 INFO:teuthology.orchestra.run.smithi061.stderr:5+1 records in 2020-12-05T17:30:43.258 INFO:teuthology.orchestra.run.smithi061.stderr:5+1 records out 2020-12-05T17:30:43.259 INFO:teuthology.orchestra.run.smithi061.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 9.2843e-05 s, 32.9 MB/s 2020-12-05T17:30:43.261 INFO:teuthology.orchestra.run.smithi061:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-12-05T17:30:43.318 INFO:teuthology.orchestra.run.smithi036:> set -ex 2020-12-05T17:30:43.319 INFO:teuthology.orchestra.run.smithi036:> sudo dd of=/usr/bin/daemon-helper 2020-12-05T17:30:43.324 INFO:teuthology.orchestra.run.smithi036.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-12-05T17:30:43.333 INFO:teuthology.orchestra.run.smithi036.stderr:5+1 records in 2020-12-05T17:30:43.333 INFO:teuthology.orchestra.run.smithi036.stderr:5+1 records out 2020-12-05T17:30:43.334 INFO:teuthology.orchestra.run.smithi036.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 9.6593e-05 s, 31.7 MB/s 2020-12-05T17:30:43.335 INFO:teuthology.orchestra.run.smithi036:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-12-05T17:30:43.391 INFO:teuthology.orchestra.run.smithi002:> set -ex 2020-12-05T17:30:43.392 INFO:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/daemon-helper 2020-12-05T17:30:43.396 INFO:teuthology.orchestra.run.smithi002.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-12-05T17:30:43.405 INFO:teuthology.orchestra.run.smithi002.stderr:5+1 records in 2020-12-05T17:30:43.406 INFO:teuthology.orchestra.run.smithi002.stderr:5+1 records out 2020-12-05T17:30:43.406 INFO:teuthology.orchestra.run.smithi002.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 9.8474e-05 s, 31.1 MB/s 2020-12-05T17:30:43.407 INFO:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-12-05T17:30:43.462 INFO:teuthology.orchestra.run.smithi053:> set -ex 2020-12-05T17:30:43.463 INFO:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/daemon-helper 2020-12-05T17:30:43.468 INFO:teuthology.orchestra.run.smithi053.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-12-05T17:30:43.477 INFO:teuthology.orchestra.run.smithi053.stderr:5+1 records in 2020-12-05T17:30:43.477 INFO:teuthology.orchestra.run.smithi053.stderr:5+1 records out 2020-12-05T17:30:43.477 INFO:teuthology.orchestra.run.smithi053.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 7.71e-05 s, 39.7 MB/s 2020-12-05T17:30:43.478 INFO:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-12-05T17:30:43.536 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2020-12-05T17:30:43.537 INFO:teuthology.orchestra.run.smithi032:> set -ex 2020-12-05T17:30:43.538 INFO:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/adjust-ulimits 2020-12-05T17:30:43.543 INFO:teuthology.orchestra.run.smithi032.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-12-05T17:30:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2020-12-05T17:30:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2020-12-05T17:30:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:299 bytes copied, 6.7553e-05 s, 4.4 MB/s 2020-12-05T17:30:43.552 INFO:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-12-05T17:30:43.611 INFO:teuthology.orchestra.run.smithi061:> set -ex 2020-12-05T17:30:43.611 INFO:teuthology.orchestra.run.smithi061:> sudo dd of=/usr/bin/adjust-ulimits 2020-12-05T17:30:43.616 INFO:teuthology.orchestra.run.smithi061.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-12-05T17:30:43.623 INFO:teuthology.orchestra.run.smithi061.stderr:0+1 records in 2020-12-05T17:30:43.623 INFO:teuthology.orchestra.run.smithi061.stderr:0+1 records out 2020-12-05T17:30:43.624 INFO:teuthology.orchestra.run.smithi061.stderr:299 bytes copied, 6.8073e-05 s, 4.4 MB/s 2020-12-05T17:30:43.625 INFO:teuthology.orchestra.run.smithi061:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-12-05T17:30:43.683 INFO:teuthology.orchestra.run.smithi036:> set -ex 2020-12-05T17:30:43.684 INFO:teuthology.orchestra.run.smithi036:> sudo dd of=/usr/bin/adjust-ulimits 2020-12-05T17:30:43.689 INFO:teuthology.orchestra.run.smithi036.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-12-05T17:30:43.698 INFO:teuthology.orchestra.run.smithi036.stderr:0+1 records in 2020-12-05T17:30:43.699 INFO:teuthology.orchestra.run.smithi036.stderr:0+1 records out 2020-12-05T17:30:43.699 INFO:teuthology.orchestra.run.smithi036.stderr:299 bytes copied, 7.2847e-05 s, 4.1 MB/s 2020-12-05T17:30:43.700 INFO:teuthology.orchestra.run.smithi036:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-12-05T17:30:43.755 INFO:teuthology.orchestra.run.smithi002:> set -ex 2020-12-05T17:30:43.756 INFO:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/adjust-ulimits 2020-12-05T17:30:43.761 INFO:teuthology.orchestra.run.smithi002.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-12-05T17:30:43.770 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records in 2020-12-05T17:30:43.771 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records out 2020-12-05T17:30:43.771 INFO:teuthology.orchestra.run.smithi002.stderr:299 bytes copied, 7.4747e-05 s, 4.0 MB/s 2020-12-05T17:30:43.772 INFO:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-12-05T17:30:43.829 INFO:teuthology.orchestra.run.smithi053:> set -ex 2020-12-05T17:30:43.829 INFO:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/adjust-ulimits 2020-12-05T17:30:43.834 INFO:teuthology.orchestra.run.smithi053.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-12-05T17:30:43.843 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records in 2020-12-05T17:30:43.844 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records out 2020-12-05T17:30:43.844 INFO:teuthology.orchestra.run.smithi053.stderr:299 bytes copied, 6.8543e-05 s, 4.4 MB/s 2020-12-05T17:30:43.846 INFO:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-12-05T17:30:43.900 INFO:teuthology.run_tasks:Running task ceph... 2020-12-05T17:30:43.961 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2020-12-05T17:30:43.962 INFO:teuthology.orchestra.run.smithi002:> sudo chmod 777 /var/log/ceph 2020-12-05T17:30:43.966 INFO:teuthology.orchestra.run.smithi032:> sudo chmod 777 /var/log/ceph 2020-12-05T17:30:43.968 INFO:teuthology.orchestra.run.smithi036:> sudo chmod 777 /var/log/ceph 2020-12-05T17:30:43.971 INFO:teuthology.orchestra.run.smithi053:> sudo chmod 777 /var/log/ceph 2020-12-05T17:30:43.973 INFO:teuthology.orchestra.run.smithi061:> sudo chmod 777 /var/log/ceph 2020-12-05T17:30:43.988 INFO:tasks.ceph:Disabling ceph logrotate... 2020-12-05T17:30:43.989 INFO:teuthology.orchestra.run.smithi002:> sudo rm -f -- /etc/logrotate.d/ceph 2020-12-05T17:30:44.024 INFO:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/logrotate.d/ceph 2020-12-05T17:30:44.027 INFO:teuthology.orchestra.run.smithi036:> sudo rm -f -- /etc/logrotate.d/ceph 2020-12-05T17:30:44.029 INFO:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/logrotate.d/ceph 2020-12-05T17:30:44.031 INFO:teuthology.orchestra.run.smithi061:> sudo rm -f -- /etc/logrotate.d/ceph 2020-12-05T17:30:44.046 INFO:tasks.ceph:Creating extra log directories... 2020-12-05T17:30:44.047 INFO:teuthology.orchestra.run.smithi002:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-12-05T17:30:44.082 INFO:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-12-05T17:30:44.086 INFO:teuthology.orchestra.run.smithi036:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-12-05T17:30:44.088 INFO:teuthology.orchestra.run.smithi053:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-12-05T17:30:44.090 INFO:teuthology.orchestra.run.smithi061:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-12-05T17:30:44.106 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2020-12-05T17:30:44.107 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2020-12-05T17:30:44.107 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2020-12-05T17:30:44.107 INFO:teuthology.orchestra.run.smithi032:> set -ex 2020-12-05T17:30:44.108 INFO:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-12-05T17:30:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:+ dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-12-05T17:30:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2020-12-05T17:30:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2020-12-05T17:30:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:408 bytes copied, 7.5203e-05 s, 5.4 MB/s 2020-12-05T17:30:44.151 INFO:teuthology.orchestra.run.smithi032:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-12-05T17:30:44.230 INFO:teuthology.orchestra.run.smithi061:> set -ex 2020-12-05T17:30:44.230 INFO:teuthology.orchestra.run.smithi061:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-12-05T17:30:44.235 INFO:teuthology.orchestra.run.smithi061.stderr:+ dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-12-05T17:30:44.237 INFO:teuthology.orchestra.run.smithi061.stderr:0+1 records in 2020-12-05T17:30:44.237 INFO:teuthology.orchestra.run.smithi061.stderr:0+1 records out 2020-12-05T17:30:44.237 INFO:teuthology.orchestra.run.smithi061.stderr:408 bytes copied, 8.093e-05 s, 5.0 MB/s 2020-12-05T17:30:44.238 INFO:teuthology.orchestra.run.smithi061:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-12-05T17:30:44.318 INFO:teuthology.orchestra.run.smithi036:> set -ex 2020-12-05T17:30:44.319 INFO:teuthology.orchestra.run.smithi036:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-12-05T17:30:44.324 INFO:teuthology.orchestra.run.smithi036.stderr:+ dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-12-05T17:30:44.326 INFO:teuthology.orchestra.run.smithi036.stderr:0+1 records in 2020-12-05T17:30:44.326 INFO:teuthology.orchestra.run.smithi036.stderr:0+1 records out 2020-12-05T17:30:44.327 INFO:teuthology.orchestra.run.smithi036.stderr:408 bytes copied, 7.4376e-05 s, 5.5 MB/s 2020-12-05T17:30:44.327 INFO:teuthology.orchestra.run.smithi036:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-12-05T17:30:44.405 INFO:teuthology.orchestra.run.smithi002:> set -ex 2020-12-05T17:30:44.405 INFO:teuthology.orchestra.run.smithi002:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-12-05T17:30:44.410 INFO:teuthology.orchestra.run.smithi002.stderr:+ dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-12-05T17:30:44.411 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records in 2020-12-05T17:30:44.412 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records out 2020-12-05T17:30:44.412 INFO:teuthology.orchestra.run.smithi002.stderr:408 bytes copied, 7.4326e-05 s, 5.5 MB/s 2020-12-05T17:30:44.412 INFO:teuthology.orchestra.run.smithi002:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-12-05T17:30:44.489 INFO:teuthology.orchestra.run.smithi053:> set -ex 2020-12-05T17:30:44.490 INFO:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-12-05T17:30:44.496 INFO:teuthology.orchestra.run.smithi053.stderr:+ dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-12-05T17:30:44.497 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records in 2020-12-05T17:30:44.498 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records out 2020-12-05T17:30:44.498 INFO:teuthology.orchestra.run.smithi053.stderr:408 bytes copied, 7.5467e-05 s, 5.4 MB/s 2020-12-05T17:30:44.498 INFO:teuthology.orchestra.run.smithi053:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-12-05T17:30:44.575 INFO:tasks.ceph:Creating ceph cluster ceph... 2020-12-05T17:30:44.576 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'POOL_APP_NOT_ENABLED', 'has not responded to cap revoke by MDS for over', 'MDS_CLIENT_LATE_RELEASE', 'responding to mclientcaps'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2020-12-05T17:30:44.576 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342', 'branch': 'octopus', 'description': 'fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests}', 'email': 'ceph-qa@ceph.io', 'first_in_suite': False, 'job_id': '5667342', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 30}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '18.04', 'overrides': {'admin_socket': {'branch': 'octopus'}, 'ceph': {'cephfs_ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'POOL_APP_NOT_ENABLED', 'has not responded to cap revoke by MDS for over', 'MDS_CLIENT_LATE_RELEASE', 'responding to mclientcaps'], 'sha1': '3b1861acb8137d863184795ccadbd5d0cb026674'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'sha1': '3b1861acb8137d863184795ccadbd5d0cb026674'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'octopus', 'sha1': '3b1861acb8137d863184795ccadbd5d0cb026674'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 1000, 'repo': 'git://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'], ['client.0'], ['client.1'], ['client.2']], 'sha1': '3b1861acb8137d863184795ccadbd5d0cb026674', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'octopus', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_octopus/qa', 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph.git', 'suite_sha1': '3b1861acb8137d863184795ccadbd5d0cb026674', 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (5, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': '3b1861acb8137d863184795ccadbd5d0cb026674'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_misc']}}], 'teuthology_branch': 'master', 'timestamp': '2020-11-30_04:10:01', 'tube': 'smithi', 'user': 'teuthology', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.19794', 'targets': {'smithi032.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/FLkMrt6+QSoSo0F9Fz2GyQK8DYsvAWqmMIMvMjiRqbincENM1U3gYC5iSjlRU5eJ4SLT7kvRi4YpEVjV553onM7n0ZiqF6XhcYXKBUONp/Imo9hB7sL9kB7HofZFInvoyQxNrSYQa3tAeueMgnFVpnj5CsjzWIaut8NgC+RHb81+V5YwNKqQsQ2d9d+nhyf/qp4Gw3Si6I2pdS1+ThOtijDUX9qm7TDk0hV23olr//Fo1sMuf9GpEKCealUhT8CUo0yNUz7sqXI6PvKESqvVVt+2p8dLVDITT+clwf1baLktyjTa5lMczBcTvW8w0uu5d4dYXjC4NpNhawFK0UOX', 'smithi061.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCuh59zncHb/+PmIORHkjdSiLduZqiHuvYzmmcgknMzwtYc46PloMcS4PiX0BjinbWMGOX4EVnlkBKS3UVqt7/EfkJZF9lQgeQw6vgXfs+1l73QQvHdXuwGxKW015qJWSjC0adW7CG/imjRgT1pTqeHSZlD6YpjaRIR2VeXXa3BtabmJuboojdO7SCv8EkqbsxZIvM+/KGn4b9VDmPmCcJNML/nkbj+C8zes4e7h7DV6Jc3M/PA5M4b1qdr23BgVfM55rkRieoTCtP02nf/CLM+H55PXYgkSmKx1LHSlPUTnsLK3L9GLqNoLJT7i2hObOblmOR5QFdhfZWZaXK4ySlh', 'smithi036.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUNu7cVfo9oLQpXclxNwdTCuj3jPUUJK5MeZB2q1hsCZqa3401oAVtHy+TLkz/1zChuCqGapIGnvS/Yy757gznNrxRiDC3b9qUz0HDSf4fVxulWlb5xBWbCwZkzSxWRWwyIq7qUj5ssC49HvxEFGmxIa7KfCeYxOIL7te5GvUtyTeMytrN+lDcPmFByuUu8yseqqq5r8s8hGMy8ETmYkBPEJMgQoComuqI0H5OHWvXOWQd44MBd82LsfLNYpLrkRglHpSmzFT3IW9klCPP83Th4NkPgyejmsaUHOeQvVy4g4Zc0CNLilO1vrcwmN0+FRMGbBFvQq5kgfveNjmz3KYP', 'smithi002.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkDTvkviRzNiFZGfh3gCKeTidXH4uumXTktmGPZgASqWZhBZQ9nkgtkXAEXFQdtPaQTjNr6/xbTq4qzVD1q4P8zzbLeI878CQl6xN2aI7IERGAk2YzbXiLNvAR57PogcBm+KIsGSK4r0omvsmthQje29maOgwaMc3X6eFVhyHJ/X6c6njN0gvcDlmSiBBJRoc/E9HedOxgkVkFzCI29jrovWlvqqKL9BHjLrLgeaRMkVZcac1hCPcWRY9TwLH6L7q5imeuqexSamXVIdzVNXBO0P+V2BW9UZGAuytYCtyQZdI/yvLf9k+1uMo+NvcmxfXicq7b/9PHxZMVa/vXvzqd', 'smithi053.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCd0U0suNtO6c2szbrg5E2OvQVna9Pj09OJubTsedEBCHaOAiZKzmF3y4129SI4cFuYPBiDF/pLks25JVh6VHdcs2QBZd1ALGza+HiOfTBc0KHeI+ieOMWKvbm9F6fI+oA/xbSWPILmDMY12SH7zvGQR4TgeFg0gnlJxlI9EJbBY7KKNKiqUuKnq1333FfNCPLFAoC3wPuWPBBmGS2bztSfTwEXCh6BvfUnpEBwRHS4ZfgCUKjldohPjN5EPN3ch+M7JQasfhYsb7nPsLGIAoGLsrFdYi2FiAK6n2ayZDutP3pAfZZJz1+R+Gp46GtJEEB35Iu+Kb3pqvxzDoWPLRtn'}} 2020-12-05T17:30:44.578 INFO:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-12-05T17:30:44.581 INFO:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-12-05T17:30:44.584 INFO:teuthology.orchestra.run.smithi036:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-12-05T17:30:44.588 INFO:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-12-05T17:30:44.619 INFO:teuthology.orchestra.run.smithi061:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-12-05T17:30:44.627 INFO:teuthology.orchestra.run.smithi002:> sudo install -d -m0777 -- /var/run/ceph 2020-12-05T17:30:44.630 INFO:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/run/ceph 2020-12-05T17:30:44.634 INFO:teuthology.orchestra.run.smithi036:> sudo install -d -m0777 -- /var/run/ceph 2020-12-05T17:30:44.636 INFO:teuthology.orchestra.run.smithi053:> sudo install -d -m0777 -- /var/run/ceph 2020-12-05T17:30:44.667 INFO:teuthology.orchestra.run.smithi061:> sudo install -d -m0777 -- /var/run/ceph 2020-12-05T17:30:44.683 INFO:teuthology.orchestra.run.smithi032:> set -ex 2020-12-05T17:30:44.684 INFO:teuthology.orchestra.run.smithi032:> dd if=/scratch_devs of=/dev/stdout 2020-12-05T17:30:44.692 INFO:teuthology.orchestra.run.smithi032.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-12-05T17:30:44.694 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/vg_nvme/lv_1 2020-12-05T17:30:44.694 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/vg_nvme/lv_2 2020-12-05T17:30:44.694 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/vg_nvme/lv_3 2020-12-05T17:30:44.694 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/vg_nvme/lv_4 2020-12-05T17:30:44.695 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2020-12-05T17:30:44.695 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2020-12-05T17:30:44.695 INFO:teuthology.orchestra.run.smithi032.stderr:72 bytes copied, 0.00017861 s, 403 kB/s 2020-12-05T17:30:44.695 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-12-05T17:30:44.696 INFO:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-12-05T17:30:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-12-05T17:30:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-12-05T17:30:44.744 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 340 Links: 1 2020-12-05T17:30:44.744 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-12-05T17:30:44.745 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2020-12-05 17:27:08.176377538 +0000 2020-12-05T17:30:44.745 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2020-12-05 17:27:08.072429538 +0000 2020-12-05T17:30:44.745 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2020-12-05 17:27:08.072429538 +0000 2020-12-05T17:30:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2020-12-05T17:30:44.753 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2020-12-05T17:30:44.754 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2020-12-05T17:30:44.754 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000307907 s, 1.7 MB/s 2020-12-05T17:30:44.758 INFO:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-12-05T17:30:44.806 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-12-05T17:30:44.807 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-12-05T17:30:44.807 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 347 Links: 1 2020-12-05T17:30:44.807 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-12-05T17:30:44.808 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2020-12-05 17:27:08.388271535 +0000 2020-12-05T17:30:44.808 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2020-12-05 17:27:08.296317536 +0000 2020-12-05T17:30:44.808 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2020-12-05 17:27:08.296317536 +0000 2020-12-05T17:30:44.808 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2020-12-05T17:30:44.816 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2020-12-05T17:30:44.816 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2020-12-05T17:30:44.817 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000285438 s, 1.8 MB/s 2020-12-05T17:30:44.821 INFO:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-12-05T17:30:44.871 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-12-05T17:30:44.871 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-12-05T17:30:44.872 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 353 Links: 1 2020-12-05T17:30:44.872 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-12-05T17:30:44.873 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2020-12-05 17:27:08.612159532 +0000 2020-12-05T17:30:44.873 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2020-12-05 17:27:08.496217534 +0000 2020-12-05T17:30:44.873 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2020-12-05 17:27:08.496217534 +0000 2020-12-05T17:30:44.874 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2020-12-05T17:30:44.880 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2020-12-05T17:30:44.881 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2020-12-05T17:30:44.881 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000280314 s, 1.8 MB/s 2020-12-05T17:30:44.885 INFO:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-12-05T17:30:44.935 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-12-05T17:30:44.935 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-12-05T17:30:44.936 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 359 Links: 1 2020-12-05T17:30:44.936 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-12-05T17:30:44.936 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2020-12-05 17:27:08.836047529 +0000 2020-12-05T17:30:44.937 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2020-12-05 17:27:08.740095530 +0000 2020-12-05T17:30:44.937 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2020-12-05 17:27:08.740095530 +0000 2020-12-05T17:30:44.937 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2020-12-05T17:30:44.945 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2020-12-05T17:30:44.945 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2020-12-05T17:30:44.945 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000249451 s, 2.1 MB/s 2020-12-05T17:30:44.950 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-12-05T17:30:44.951 INFO:teuthology.orchestra.run.smithi061:> set -ex 2020-12-05T17:30:44.951 INFO:teuthology.orchestra.run.smithi061:> dd if=/scratch_devs of=/dev/stdout 2020-12-05T17:30:44.956 INFO:teuthology.orchestra.run.smithi061.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-12-05T17:30:44.958 INFO:teuthology.orchestra.run.smithi061.stdout:/dev/vg_nvme/lv_1 2020-12-05T17:30:44.958 INFO:teuthology.orchestra.run.smithi061.stdout:/dev/vg_nvme/lv_2 2020-12-05T17:30:44.959 INFO:teuthology.orchestra.run.smithi061.stdout:/dev/vg_nvme/lv_3 2020-12-05T17:30:44.959 INFO:teuthology.orchestra.run.smithi061.stdout:/dev/vg_nvme/lv_4 2020-12-05T17:30:44.959 INFO:teuthology.orchestra.run.smithi061.stderr:0+1 records in 2020-12-05T17:30:44.959 INFO:teuthology.orchestra.run.smithi061.stderr:0+1 records out 2020-12-05T17:30:44.960 INFO:teuthology.orchestra.run.smithi061.stderr:72 bytes copied, 0.000172433 s, 418 kB/s 2020-12-05T17:30:44.960 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-12-05T17:30:44.960 INFO:teuthology.orchestra.run.smithi061:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-12-05T17:30:45.007 INFO:teuthology.orchestra.run.smithi061.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-12-05T17:30:45.008 INFO:teuthology.orchestra.run.smithi061.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-12-05T17:30:45.008 INFO:teuthology.orchestra.run.smithi061.stdout:Device: 6h/6d Inode: 341 Links: 1 2020-12-05T17:30:45.008 INFO:teuthology.orchestra.run.smithi061.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-12-05T17:30:45.008 INFO:teuthology.orchestra.run.smithi061.stdout:Access: 2020-12-05 17:27:13.584124649 +0000 2020-12-05T17:30:45.008 INFO:teuthology.orchestra.run.smithi061.stdout:Modify: 2020-12-05 17:27:13.580126648 +0000 2020-12-05T17:30:45.009 INFO:teuthology.orchestra.run.smithi061.stdout:Change: 2020-12-05 17:27:13.580126648 +0000 2020-12-05T17:30:45.009 INFO:teuthology.orchestra.run.smithi061.stdout: Birth: - 2020-12-05T17:30:45.017 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records in 2020-12-05T17:30:45.017 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records out 2020-12-05T17:30:45.017 INFO:teuthology.orchestra.run.smithi061.stderr:512 bytes copied, 0.000410887 s, 1.2 MB/s 2020-12-05T17:30:45.021 INFO:teuthology.orchestra.run.smithi061:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-12-05T17:30:45.071 INFO:teuthology.orchestra.run.smithi061.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-12-05T17:30:45.071 INFO:teuthology.orchestra.run.smithi061.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-12-05T17:30:45.071 INFO:teuthology.orchestra.run.smithi061.stdout:Device: 6h/6d Inode: 348 Links: 1 2020-12-05T17:30:45.072 INFO:teuthology.orchestra.run.smithi061.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-12-05T17:30:45.072 INFO:teuthology.orchestra.run.smithi061.stdout:Access: 2020-12-05 17:27:13.927952644 +0000 2020-12-05T17:30:45.072 INFO:teuthology.orchestra.run.smithi061.stdout:Modify: 2020-12-05 17:27:13.835998645 +0000 2020-12-05T17:30:45.072 INFO:teuthology.orchestra.run.smithi061.stdout:Change: 2020-12-05 17:27:13.835998645 +0000 2020-12-05T17:30:45.072 INFO:teuthology.orchestra.run.smithi061.stdout: Birth: - 2020-12-05T17:30:45.080 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records in 2020-12-05T17:30:45.081 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records out 2020-12-05T17:30:45.081 INFO:teuthology.orchestra.run.smithi061.stderr:512 bytes copied, 0.000350024 s, 1.5 MB/s 2020-12-05T17:30:45.085 INFO:teuthology.orchestra.run.smithi061:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-12-05T17:30:45.135 INFO:teuthology.orchestra.run.smithi061.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-12-05T17:30:45.135 INFO:teuthology.orchestra.run.smithi061.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-12-05T17:30:45.136 INFO:teuthology.orchestra.run.smithi061.stdout:Device: 6h/6d Inode: 354 Links: 1 2020-12-05T17:30:45.136 INFO:teuthology.orchestra.run.smithi061.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-12-05T17:30:45.136 INFO:teuthology.orchestra.run.smithi061.stdout:Access: 2020-12-05 17:27:14.055888643 +0000 2020-12-05T17:30:45.136 INFO:teuthology.orchestra.run.smithi061.stdout:Modify: 2020-12-05 17:27:14.051890643 +0000 2020-12-05T17:30:45.137 INFO:teuthology.orchestra.run.smithi061.stdout:Change: 2020-12-05 17:27:14.051890643 +0000 2020-12-05T17:30:45.137 INFO:teuthology.orchestra.run.smithi061.stdout: Birth: - 2020-12-05T17:30:45.145 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records in 2020-12-05T17:30:45.145 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records out 2020-12-05T17:30:45.145 INFO:teuthology.orchestra.run.smithi061.stderr:512 bytes copied, 0.00030235 s, 1.7 MB/s 2020-12-05T17:30:45.149 INFO:teuthology.orchestra.run.smithi061:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-12-05T17:30:45.199 INFO:teuthology.orchestra.run.smithi061.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-12-05T17:30:45.199 INFO:teuthology.orchestra.run.smithi061.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-12-05T17:30:45.199 INFO:teuthology.orchestra.run.smithi061.stdout:Device: 6h/6d Inode: 360 Links: 1 2020-12-05T17:30:45.200 INFO:teuthology.orchestra.run.smithi061.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-12-05T17:30:45.200 INFO:teuthology.orchestra.run.smithi061.stdout:Access: 2020-12-05 17:27:14.395718640 +0000 2020-12-05T17:30:45.200 INFO:teuthology.orchestra.run.smithi061.stdout:Modify: 2020-12-05 17:27:14.303764640 +0000 2020-12-05T17:30:45.200 INFO:teuthology.orchestra.run.smithi061.stdout:Change: 2020-12-05 17:27:14.303764640 +0000 2020-12-05T17:30:45.201 INFO:teuthology.orchestra.run.smithi061.stdout: Birth: - 2020-12-05T17:30:45.209 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records in 2020-12-05T17:30:45.209 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records out 2020-12-05T17:30:45.209 INFO:teuthology.orchestra.run.smithi061.stderr:512 bytes copied, 0.000423842 s, 1.2 MB/s 2020-12-05T17:30:45.214 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-12-05T17:30:45.214 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi061.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-12-05T17:30:45.215 INFO:tasks.ceph:Generating config... 2020-12-05T17:30:45.216 INFO:tasks.ceph:[client] client mount timeout = 600 2020-12-05T17:30:45.216 INFO:tasks.ceph:[client] debug client = 20 2020-12-05T17:30:45.216 INFO:tasks.ceph:[client] debug ms = 1 2020-12-05T17:30:45.217 INFO:tasks.ceph:[client] rados mon op timeout = 15m 2020-12-05T17:30:45.217 INFO:tasks.ceph:[client] rados osd op timeout = 15m 2020-12-05T17:30:45.217 INFO:tasks.ceph:[mds] debug mds = 20 2020-12-05T17:30:45.218 INFO:tasks.ceph:[mds] debug ms = 1 2020-12-05T17:30:45.218 INFO:tasks.ceph:[mds] mds bal frag = True 2020-12-05T17:30:45.218 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2020-12-05T17:30:45.219 INFO:tasks.ceph:[mds] mds bal merge size = 5 2020-12-05T17:30:45.219 INFO:tasks.ceph:[mds] mds bal split bits = 3 2020-12-05T17:30:45.219 INFO:tasks.ceph:[mds] mds bal split size = 100 2020-12-05T17:30:45.219 INFO:tasks.ceph:[mds] mds debug frag = True 2020-12-05T17:30:45.220 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2020-12-05T17:30:45.220 INFO:tasks.ceph:[mds] mds op complaint time = 180 2020-12-05T17:30:45.220 INFO:tasks.ceph:[mds] mds verify scatter = True 2020-12-05T17:30:45.220 INFO:tasks.ceph:[mds] osd op complaint time = 180 2020-12-05T17:30:45.221 INFO:tasks.ceph:[mds] rados mon op timeout = 15m 2020-12-05T17:30:45.221 INFO:tasks.ceph:[mds] rados osd op timeout = 15m 2020-12-05T17:30:45.221 INFO:tasks.ceph:[mgr] debug mgr = 20 2020-12-05T17:30:45.224 INFO:tasks.ceph:[mgr] debug ms = 1 2020-12-05T17:30:45.225 INFO:tasks.ceph:[mon] debug mon = 20 2020-12-05T17:30:45.225 INFO:tasks.ceph:[mon] debug ms = 1 2020-12-05T17:30:45.225 INFO:tasks.ceph:[mon] debug paxos = 20 2020-12-05T17:30:45.226 INFO:tasks.ceph:[mon] mon op complaint time = 120 2020-12-05T17:30:45.226 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2020-12-05T17:30:45.226 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2020-12-05T17:30:45.226 INFO:tasks.ceph:[osd] debug bluefs = 20 2020-12-05T17:30:45.227 INFO:tasks.ceph:[osd] debug bluestore = 20 2020-12-05T17:30:45.227 INFO:tasks.ceph:[osd] debug filestore = 20 2020-12-05T17:30:45.227 INFO:tasks.ceph:[osd] debug journal = 20 2020-12-05T17:30:45.228 INFO:tasks.ceph:[osd] debug ms = 20 2020-12-05T17:30:45.228 INFO:tasks.ceph:[osd] debug osd = 25 2020-12-05T17:30:45.228 INFO:tasks.ceph:[osd] debug rocksdb = 10 2020-12-05T17:30:45.228 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2020-12-05T17:30:45.229 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2020-12-05T17:30:45.230 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2020-12-05T17:30:45.230 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2020-12-05T17:30:45.230 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2020-12-05T17:30:45.231 INFO:tasks.ceph:[osd] osd op complaint time = 180 2020-12-05T17:30:45.231 INFO:tasks.ceph:Setting up mon.a... 2020-12-05T17:30:45.231 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2020-12-05T17:30:45.262 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.keyring 2020-12-05T17:30:45.264 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2020-12-05T17:30:45.343 INFO:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-12-05T17:30:45.399 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.32'), ('mon.b', '172.21.15.61'), ('mon.c', '[v2:172.21.15.61:3301,v1:172.21.15.61:6790]')] 2020-12-05T17:30:45.399 INFO:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.32 --add b 172.21.15.61 --addv c '[v2:172.21.15.61:3301,v1:172.21.15.61:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2020-12-05T17:30:45.475 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2020-12-05T17:30:45.475 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: generated fsid efd7e861-a502-461d-b20d-3b2c0a25fbd9 2020-12-05T17:30:45.476 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 0 2020-12-05T17:30:45.476 INFO:teuthology.orchestra.run.smithi032.stdout:fsid efd7e861-a502-461d-b20d-3b2c0a25fbd9 2020-12-05T17:30:45.476 INFO:teuthology.orchestra.run.smithi032.stdout:last_changed 2020-12-05T17:30:45.480244+0000 2020-12-05T17:30:45.477 INFO:teuthology.orchestra.run.smithi032.stdout:created 2020-12-05T17:30:45.480244+0000 2020-12-05T17:30:45.477 INFO:teuthology.orchestra.run.smithi032.stdout:min_mon_release 0 (unknown) 2020-12-05T17:30:45.478 INFO:teuthology.orchestra.run.smithi032.stdout:0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2020-12-05T17:30:45.478 INFO:teuthology.orchestra.run.smithi032.stdout:1: [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] mon.b 2020-12-05T17:30:45.478 INFO:teuthology.orchestra.run.smithi032.stdout:2: [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] mon.c 2020-12-05T17:30:45.479 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2020-12-05T17:30:45.481 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID efd7e861-a502-461d-b20d-3b2c0a25fbd9... 2020-12-05T17:30:45.483 INFO:teuthology.orchestra.run.smithi002:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-12-05T17:30:45.487 INFO:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-12-05T17:30:45.526 INFO:teuthology.orchestra.run.smithi036:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-12-05T17:30:45.529 INFO:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-12-05T17:30:45.531 INFO:teuthology.orchestra.run.smithi061:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-12-05T17:30:45.535 INFO:teuthology.orchestra.run.smithi002.stdout:[global] 2020-12-05T17:30:45.536 INFO:teuthology.orchestra.run.smithi002.stdout: chdir = "" 2020-12-05T17:30:45.536 INFO:teuthology.orchestra.run.smithi002.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-12-05T17:30:45.536 INFO:teuthology.orchestra.run.smithi002.stdout: auth supported = cephx 2020-12-05T17:30:45.537 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.537 INFO:teuthology.orchestra.run.smithi002.stdout: filestore xattr use omap = true 2020-12-05T17:30:45.537 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.537 INFO:teuthology.orchestra.run.smithi002.stdout: mon clock drift allowed = 1.000 2020-12-05T17:30:45.537 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.538 INFO:teuthology.orchestra.run.smithi002.stdout: osd crush chooseleaf type = 0 2020-12-05T17:30:45.538 INFO:teuthology.orchestra.run.smithi002.stdout: auth debug = true 2020-12-05T17:30:45.538 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.538 INFO:teuthology.orchestra.run.smithi002.stdout: ms die on old message = true 2020-12-05T17:30:45.538 INFO:teuthology.orchestra.run.smithi002.stdout: ms die on bug = true 2020-12-05T17:30:45.539 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.539 INFO:teuthology.orchestra.run.smithi002.stdout: mon max pg per osd = 10000 # >= luminous 2020-12-05T17:30:45.539 INFO:teuthology.orchestra.run.smithi002.stdout: mon pg warn max object skew = 0 2020-12-05T17:30:45.539 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.539 INFO:teuthology.orchestra.run.smithi002.stdout: # disable pg_autoscaler by default for new pools 2020-12-05T17:30:45.540 INFO:teuthology.orchestra.run.smithi002.stdout: osd_pool_default_pg_autoscale_mode = off 2020-12-05T17:30:45.540 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.540 INFO:teuthology.orchestra.run.smithi002.stdout: osd pool default size = 2 2020-12-05T17:30:45.540 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.541 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd allow primary affinity = true 2020-12-05T17:30:45.541 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd allow pg remap = true 2020-12-05T17:30:45.541 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on legacy crush tunables = false 2020-12-05T17:30:45.541 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on crush straw calc version zero = false 2020-12-05T17:30:45.541 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on no sortbitwise = false 2020-12-05T17:30:45.542 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on osd down out interval zero = false 2020-12-05T17:30:45.542 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on too few osds = false 2020-12-05T17:30:45.542 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-12-05T17:30:45.542 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_pool_no_redundancy = false 2020-12-05T17:30:45.543 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.543 INFO:teuthology.orchestra.run.smithi002.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-12-05T17:30:45.543 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.543 INFO:teuthology.orchestra.run.smithi002.stdout: osd default data pool replay window = 5 2020-12-05T17:30:45.543 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.544 INFO:teuthology.orchestra.run.smithi002.stdout: mon allow pool delete = true 2020-12-05T17:30:45.544 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.544 INFO:teuthology.orchestra.run.smithi002.stdout: mon cluster log file level = debug 2020-12-05T17:30:45.544 INFO:teuthology.orchestra.run.smithi002.stdout: debug asserts on shutdown = true 2020-12-05T17:30:45.544 INFO:teuthology.orchestra.run.smithi002.stdout: mon host = "172.21.15.32,172.21.15.61,[v2:172.21.15.61:3301,v1:172.21.15.61:6790]" 2020-12-05T17:30:45.545 INFO:teuthology.orchestra.run.smithi002.stdout: fsid = efd7e861-a502-461d-b20d-3b2c0a25fbd9 2020-12-05T17:30:45.545 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.545 INFO:teuthology.orchestra.run.smithi002.stdout:[osd] 2020-12-05T17:30:45.545 INFO:teuthology.orchestra.run.smithi002.stdout: osd journal size = 100 2020-12-05T17:30:45.545 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.546 INFO:teuthology.orchestra.run.smithi002.stdout: osd scrub load threshold = 5.0 2020-12-05T17:30:45.546 INFO:teuthology.orchestra.run.smithi002.stdout: osd scrub max interval = 600 2020-12-05T17:30:45.546 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.546 INFO:teuthology.orchestra.run.smithi002.stdout: osd recover clone overlap = true 2020-12-05T17:30:45.546 INFO:teuthology.orchestra.run.smithi002.stdout: osd recovery max chunk = 1048576 2020-12-05T17:30:45.547 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.547 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug shutdown = true 2020-12-05T17:30:45.547 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug op order = true 2020-12-05T17:30:45.547 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug verify stray on activate = true 2020-12-05T17:30:45.547 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.548 INFO:teuthology.orchestra.run.smithi002.stdout: osd open classes on start = true 2020-12-05T17:30:45.548 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug pg log writeout = true 2020-12-05T17:30:45.548 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.548 INFO:teuthology.orchestra.run.smithi002.stdout: osd deep scrub update digest min age = 30 2020-12-05T17:30:45.548 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.549 INFO:teuthology.orchestra.run.smithi002.stdout: osd map max advance = 10 2020-12-05T17:30:45.549 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.549 INFO:teuthology.orchestra.run.smithi002.stdout: journal zero on create = true 2020-12-05T17:30:45.549 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.549 INFO:teuthology.orchestra.run.smithi002.stdout: filestore ondisk finisher threads = 3 2020-12-05T17:30:45.550 INFO:teuthology.orchestra.run.smithi002.stdout: filestore apply finisher threads = 3 2020-12-05T17:30:45.550 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.550 INFO:teuthology.orchestra.run.smithi002.stdout: bdev debug aio = true 2020-12-05T17:30:45.550 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug misdirected ops = true 2020-12-05T17:30:45.551 INFO:teuthology.orchestra.run.smithi002.stdout: bluestore block size = 96636764160 2020-12-05T17:30:45.551 INFO:teuthology.orchestra.run.smithi002.stdout: bluestore fsck on mount = True 2020-12-05T17:30:45.551 INFO:teuthology.orchestra.run.smithi002.stdout: debug bluefs = 20 2020-12-05T17:30:45.551 INFO:teuthology.orchestra.run.smithi002.stdout: debug bluestore = 20 2020-12-05T17:30:45.551 INFO:teuthology.orchestra.run.smithi002.stdout: debug filestore = 20 2020-12-05T17:30:45.552 INFO:teuthology.orchestra.run.smithi002.stdout: debug journal = 20 2020-12-05T17:30:45.552 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 20 2020-12-05T17:30:45.552 INFO:teuthology.orchestra.run.smithi002.stdout: debug osd = 25 2020-12-05T17:30:45.552 INFO:teuthology.orchestra.run.smithi002.stdout: debug rocksdb = 10 2020-12-05T17:30:45.552 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd backfillfull_ratio = 0.85 2020-12-05T17:30:45.553 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd full ratio = 0.9 2020-12-05T17:30:45.553 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd nearfull ratio = 0.8 2020-12-05T17:30:45.553 INFO:teuthology.orchestra.run.smithi002.stdout: osd failsafe full ratio = 0.95 2020-12-05T17:30:45.553 INFO:teuthology.orchestra.run.smithi002.stdout: osd objectstore = bluestore 2020-12-05T17:30:45.554 INFO:teuthology.orchestra.run.smithi002.stdout: osd op complaint time = 180 2020-12-05T17:30:45.554 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.555 INFO:teuthology.orchestra.run.smithi002.stdout:[mgr] 2020-12-05T17:30:45.555 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2020-12-05T17:30:45.555 INFO:teuthology.orchestra.run.smithi002.stdout: debug mgr = 20 2020-12-05T17:30:45.555 INFO:teuthology.orchestra.run.smithi002.stdout: debug mon = 20 2020-12-05T17:30:45.555 INFO:teuthology.orchestra.run.smithi002.stdout: debug auth = 20 2020-12-05T17:30:45.556 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:30:45.556 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:30:45.556 INFO:teuthology.orchestra.run.smithi002.stdout: mgr/telemetry/nag = false 2020-12-05T17:30:45.556 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.556 INFO:teuthology.orchestra.run.smithi002.stdout:[mon] 2020-12-05T17:30:45.557 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2020-12-05T17:30:45.557 INFO:teuthology.orchestra.run.smithi002.stdout: debug mon = 20 2020-12-05T17:30:45.557 INFO:teuthology.orchestra.run.smithi002.stdout: debug paxos = 20 2020-12-05T17:30:45.557 INFO:teuthology.orchestra.run.smithi002.stdout: debug auth = 20 2020-12-05T17:30:45.558 INFO:teuthology.orchestra.run.smithi002.stdout: mon data avail warn = 5 2020-12-05T17:30:45.558 INFO:teuthology.orchestra.run.smithi002.stdout: mon mgr mkfs grace = 240 2020-12-05T17:30:45.558 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:30:45.558 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd reporter subtree level = osd 2020-12-05T17:30:45.558 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd prime pg temp = true 2020-12-05T17:30:45.559 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:30:45.559 INFO:teuthology.orchestra.run.smithi002.stdout: mon op complaint time = 120 2020-12-05T17:30:45.559 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:30:45.559 INFO:teuthology.orchestra.run.smithi002.stdout:[client] 2020-12-05T17:30:45.559 INFO:teuthology.orchestra.run.smithi002.stdout: rgw cache enabled = true 2020-12-05T17:30:45.560 INFO:teuthology.orchestra.run.smithi002.stdout: rgw enable ops log = true 2020-12-05T17:30:45.560 INFO:teuthology.orchestra.run.smithi002.stdout: rgw enable usage log = true 2020-12-05T17:30:45.560 INFO:teuthology.orchestra.run.smithi002.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-12-05T17:30:45.560 INFO:teuthology.orchestra.run.smithi002.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-12-05T17:30:45.561 INFO:teuthology.orchestra.run.smithi002.stdout: client mount timeout = 600 2020-12-05T17:30:45.561 INFO:teuthology.orchestra.run.smithi002.stdout: debug client = 20 2020-12-05T17:30:45.561 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2020-12-05T17:30:45.561 INFO:teuthology.orchestra.run.smithi002.stdout: rados mon op timeout = 15m 2020-12-05T17:30:45.561 INFO:teuthology.orchestra.run.smithi002.stdout: rados osd op timeout = 15m 2020-12-05T17:30:45.562 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.a] 2020-12-05T17:30:45.562 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.b] 2020-12-05T17:30:45.562 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.c] 2020-12-05T17:30:45.562 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.a] 2020-12-05T17:30:45.562 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.b] 2020-12-05T17:30:45.563 INFO:teuthology.orchestra.run.smithi002.stdout:[mds] 2020-12-05T17:30:45.563 INFO:teuthology.orchestra.run.smithi002.stdout: debug mds = 20 2020-12-05T17:30:45.563 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2020-12-05T17:30:45.563 INFO:teuthology.orchestra.run.smithi002.stdout: mds bal frag = True 2020-12-05T17:30:45.563 INFO:teuthology.orchestra.run.smithi002.stdout: mds bal fragment size max = 10000 2020-12-05T17:30:45.564 INFO:teuthology.orchestra.run.smithi002.stdout: mds bal merge size = 5 2020-12-05T17:30:45.564 INFO:teuthology.orchestra.run.smithi002.stdout: mds bal split bits = 3 2020-12-05T17:30:45.564 INFO:teuthology.orchestra.run.smithi002.stdout: mds bal split size = 100 2020-12-05T17:30:45.564 INFO:teuthology.orchestra.run.smithi002.stdout: mds debug frag = True 2020-12-05T17:30:45.564 INFO:teuthology.orchestra.run.smithi002.stdout: mds debug scatterstat = True 2020-12-05T17:30:45.565 INFO:teuthology.orchestra.run.smithi002.stdout: mds op complaint time = 180 2020-12-05T17:30:45.565 INFO:teuthology.orchestra.run.smithi002.stdout: mds verify scatter = True 2020-12-05T17:30:45.565 INFO:teuthology.orchestra.run.smithi002.stdout: osd op complaint time = 180 2020-12-05T17:30:45.565 INFO:teuthology.orchestra.run.smithi002.stdout: rados mon op timeout = 15m 2020-12-05T17:30:45.566 INFO:teuthology.orchestra.run.smithi002.stdout: rados osd op timeout = 15m 2020-12-05T17:30:45.568 INFO:teuthology.orchestra.run.smithi061.stdout:[global] 2020-12-05T17:30:45.568 INFO:teuthology.orchestra.run.smithi061.stdout: chdir = "" 2020-12-05T17:30:45.568 INFO:teuthology.orchestra.run.smithi061.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-12-05T17:30:45.568 INFO:teuthology.orchestra.run.smithi061.stdout: auth supported = cephx 2020-12-05T17:30:45.569 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.569 INFO:teuthology.orchestra.run.smithi061.stdout: filestore xattr use omap = true 2020-12-05T17:30:45.569 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.569 INFO:teuthology.orchestra.run.smithi061.stdout: mon clock drift allowed = 1.000 2020-12-05T17:30:45.569 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.570 INFO:teuthology.orchestra.run.smithi061.stdout: osd crush chooseleaf type = 0 2020-12-05T17:30:45.570 INFO:teuthology.orchestra.run.smithi061.stdout: auth debug = true 2020-12-05T17:30:45.570 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.570 INFO:teuthology.orchestra.run.smithi061.stdout: ms die on old message = true 2020-12-05T17:30:45.570 INFO:teuthology.orchestra.run.smithi061.stdout: ms die on bug = true 2020-12-05T17:30:45.571 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.571 INFO:teuthology.orchestra.run.smithi061.stdout: mon max pg per osd = 10000 # >= luminous 2020-12-05T17:30:45.571 INFO:teuthology.orchestra.run.smithi061.stdout: mon pg warn max object skew = 0 2020-12-05T17:30:45.571 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.571 INFO:teuthology.orchestra.run.smithi061.stdout: # disable pg_autoscaler by default for new pools 2020-12-05T17:30:45.572 INFO:teuthology.orchestra.run.smithi061.stdout: osd_pool_default_pg_autoscale_mode = off 2020-12-05T17:30:45.572 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.572 INFO:teuthology.orchestra.run.smithi061.stdout: osd pool default size = 2 2020-12-05T17:30:45.572 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.572 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd allow primary affinity = true 2020-12-05T17:30:45.573 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd allow pg remap = true 2020-12-05T17:30:45.573 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on legacy crush tunables = false 2020-12-05T17:30:45.573 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on crush straw calc version zero = false 2020-12-05T17:30:45.573 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on no sortbitwise = false 2020-12-05T17:30:45.574 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on osd down out interval zero = false 2020-12-05T17:30:45.574 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on too few osds = false 2020-12-05T17:30:45.574 INFO:teuthology.orchestra.run.smithi061.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-12-05T17:30:45.574 INFO:teuthology.orchestra.run.smithi061.stdout: mon_warn_on_pool_no_redundancy = false 2020-12-05T17:30:45.574 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.575 INFO:teuthology.orchestra.run.smithi061.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-12-05T17:30:45.575 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.575 INFO:teuthology.orchestra.run.smithi061.stdout: osd default data pool replay window = 5 2020-12-05T17:30:45.575 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.575 INFO:teuthology.orchestra.run.smithi061.stdout: mon allow pool delete = true 2020-12-05T17:30:45.576 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.576 INFO:teuthology.orchestra.run.smithi061.stdout: mon cluster log file level = debug 2020-12-05T17:30:45.576 INFO:teuthology.orchestra.run.smithi061.stdout: debug asserts on shutdown = true 2020-12-05T17:30:45.576 INFO:teuthology.orchestra.run.smithi061.stdout: mon host = "172.21.15.32,172.21.15.61,[v2:172.21.15.61:3301,v1:172.21.15.61:6790]" 2020-12-05T17:30:45.577 INFO:teuthology.orchestra.run.smithi061.stdout: fsid = efd7e861-a502-461d-b20d-3b2c0a25fbd9 2020-12-05T17:30:45.577 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.577 INFO:teuthology.orchestra.run.smithi061.stdout:[osd] 2020-12-05T17:30:45.577 INFO:teuthology.orchestra.run.smithi061.stdout: osd journal size = 100 2020-12-05T17:30:45.577 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.578 INFO:teuthology.orchestra.run.smithi061.stdout: osd scrub load threshold = 5.0 2020-12-05T17:30:45.578 INFO:teuthology.orchestra.run.smithi061.stdout: osd scrub max interval = 600 2020-12-05T17:30:45.578 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.578 INFO:teuthology.orchestra.run.smithi061.stdout: osd recover clone overlap = true 2020-12-05T17:30:45.578 INFO:teuthology.orchestra.run.smithi061.stdout: osd recovery max chunk = 1048576 2020-12-05T17:30:45.579 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.579 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug shutdown = true 2020-12-05T17:30:45.579 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug op order = true 2020-12-05T17:30:45.579 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug verify stray on activate = true 2020-12-05T17:30:45.579 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.580 INFO:teuthology.orchestra.run.smithi061.stdout: osd open classes on start = true 2020-12-05T17:30:45.580 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug pg log writeout = true 2020-12-05T17:30:45.580 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.580 INFO:teuthology.orchestra.run.smithi061.stdout: osd deep scrub update digest min age = 30 2020-12-05T17:30:45.580 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.581 INFO:teuthology.orchestra.run.smithi061.stdout: osd map max advance = 10 2020-12-05T17:30:45.581 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.581 INFO:teuthology.orchestra.run.smithi061.stdout: journal zero on create = true 2020-12-05T17:30:45.581 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.581 INFO:teuthology.orchestra.run.smithi061.stdout: filestore ondisk finisher threads = 3 2020-12-05T17:30:45.582 INFO:teuthology.orchestra.run.smithi061.stdout: filestore apply finisher threads = 3 2020-12-05T17:30:45.582 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.582 INFO:teuthology.orchestra.run.smithi061.stdout: bdev debug aio = true 2020-12-05T17:30:45.582 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug misdirected ops = true 2020-12-05T17:30:45.582 INFO:teuthology.orchestra.run.smithi061.stdout: bluestore block size = 96636764160 2020-12-05T17:30:45.583 INFO:teuthology.orchestra.run.smithi061.stdout: bluestore fsck on mount = True 2020-12-05T17:30:45.583 INFO:teuthology.orchestra.run.smithi061.stdout: debug bluefs = 20 2020-12-05T17:30:45.583 INFO:teuthology.orchestra.run.smithi061.stdout: debug bluestore = 20 2020-12-05T17:30:45.583 INFO:teuthology.orchestra.run.smithi061.stdout: debug filestore = 20 2020-12-05T17:30:45.583 INFO:teuthology.orchestra.run.smithi061.stdout: debug journal = 20 2020-12-05T17:30:45.584 INFO:teuthology.orchestra.run.smithi061.stdout: debug ms = 20 2020-12-05T17:30:45.584 INFO:teuthology.orchestra.run.smithi061.stdout: debug osd = 25 2020-12-05T17:30:45.584 INFO:teuthology.orchestra.run.smithi061.stdout: debug rocksdb = 10 2020-12-05T17:30:45.584 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd backfillfull_ratio = 0.85 2020-12-05T17:30:45.585 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd full ratio = 0.9 2020-12-05T17:30:45.585 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd nearfull ratio = 0.8 2020-12-05T17:30:45.585 INFO:teuthology.orchestra.run.smithi061.stdout: osd failsafe full ratio = 0.95 2020-12-05T17:30:45.585 INFO:teuthology.orchestra.run.smithi061.stdout: osd objectstore = bluestore 2020-12-05T17:30:45.585 INFO:teuthology.orchestra.run.smithi061.stdout: osd op complaint time = 180 2020-12-05T17:30:45.586 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.586 INFO:teuthology.orchestra.run.smithi061.stdout:[mgr] 2020-12-05T17:30:45.586 INFO:teuthology.orchestra.run.smithi061.stdout: debug ms = 1 2020-12-05T17:30:45.586 INFO:teuthology.orchestra.run.smithi061.stdout: debug mgr = 20 2020-12-05T17:30:45.587 INFO:teuthology.orchestra.run.smithi061.stdout: debug mon = 20 2020-12-05T17:30:45.587 INFO:teuthology.orchestra.run.smithi061.stdout: debug auth = 20 2020-12-05T17:30:45.587 INFO:teuthology.orchestra.run.smithi061.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:30:45.587 INFO:teuthology.orchestra.run.smithi061.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:30:45.587 INFO:teuthology.orchestra.run.smithi061.stdout: mgr/telemetry/nag = false 2020-12-05T17:30:45.588 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.588 INFO:teuthology.orchestra.run.smithi061.stdout:[mon] 2020-12-05T17:30:45.588 INFO:teuthology.orchestra.run.smithi061.stdout: debug ms = 1 2020-12-05T17:30:45.588 INFO:teuthology.orchestra.run.smithi061.stdout: debug mon = 20 2020-12-05T17:30:45.588 INFO:teuthology.orchestra.run.smithi061.stdout: debug paxos = 20 2020-12-05T17:30:45.589 INFO:teuthology.orchestra.run.smithi061.stdout: debug auth = 20 2020-12-05T17:30:45.589 INFO:teuthology.orchestra.run.smithi061.stdout: mon data avail warn = 5 2020-12-05T17:30:45.589 INFO:teuthology.orchestra.run.smithi061.stdout: mon mgr mkfs grace = 240 2020-12-05T17:30:45.589 INFO:teuthology.orchestra.run.smithi061.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:30:45.589 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd reporter subtree level = osd 2020-12-05T17:30:45.590 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd prime pg temp = true 2020-12-05T17:30:45.590 INFO:teuthology.orchestra.run.smithi061.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:30:45.590 INFO:teuthology.orchestra.run.smithi061.stdout: mon op complaint time = 120 2020-12-05T17:30:45.590 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:30:45.591 INFO:teuthology.orchestra.run.smithi061.stdout:[client] 2020-12-05T17:30:45.591 INFO:teuthology.orchestra.run.smithi061.stdout: rgw cache enabled = true 2020-12-05T17:30:45.591 INFO:teuthology.orchestra.run.smithi061.stdout: rgw enable ops log = true 2020-12-05T17:30:45.591 INFO:teuthology.orchestra.run.smithi061.stdout: rgw enable usage log = true 2020-12-05T17:30:45.592 INFO:teuthology.orchestra.run.smithi061.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-12-05T17:30:45.592 INFO:teuthology.orchestra.run.smithi061.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-12-05T17:30:45.592 INFO:teuthology.orchestra.run.smithi061.stdout: client mount timeout = 600 2020-12-05T17:30:45.592 INFO:teuthology.orchestra.run.smithi061.stdout: debug client = 20 2020-12-05T17:30:45.593 INFO:teuthology.orchestra.run.smithi061.stdout: debug ms = 1 2020-12-05T17:30:45.593 INFO:teuthology.orchestra.run.smithi061.stdout: rados mon op timeout = 15m 2020-12-05T17:30:45.593 INFO:teuthology.orchestra.run.smithi061.stdout: rados osd op timeout = 15m 2020-12-05T17:30:45.593 INFO:teuthology.orchestra.run.smithi061.stdout:[mon.a] 2020-12-05T17:30:45.593 INFO:teuthology.orchestra.run.smithi061.stdout:[mon.b] 2020-12-05T17:30:45.594 INFO:teuthology.orchestra.run.smithi061.stdout:[mon.c] 2020-12-05T17:30:45.594 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.a] 2020-12-05T17:30:45.594 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.b] 2020-12-05T17:30:45.594 INFO:teuthology.orchestra.run.smithi061.stdout:[mds] 2020-12-05T17:30:45.594 INFO:teuthology.orchestra.run.smithi061.stdout: debug mds = 20 2020-12-05T17:30:45.595 INFO:teuthology.orchestra.run.smithi061.stdout: debug ms = 1 2020-12-05T17:30:45.595 INFO:teuthology.orchestra.run.smithi061.stdout: mds bal frag = True 2020-12-05T17:30:45.595 INFO:teuthology.orchestra.run.smithi061.stdout: mds bal fragment size max = 10000 2020-12-05T17:30:45.595 INFO:teuthology.orchestra.run.smithi061.stdout: mds bal merge size = 5 2020-12-05T17:30:45.595 INFO:teuthology.orchestra.run.smithi061.stdout: mds bal split bits = 3 2020-12-05T17:30:45.596 INFO:teuthology.orchestra.run.smithi061.stdout: mds bal split size = 100 2020-12-05T17:30:45.596 INFO:teuthology.orchestra.run.smithi061.stdout: mds debug frag = True 2020-12-05T17:30:45.596 INFO:teuthology.orchestra.run.smithi061.stdout: mds debug scatterstat = True 2020-12-05T17:30:45.596 INFO:teuthology.orchestra.run.smithi061.stdout: mds op complaint time = 180 2020-12-05T17:30:45.596 INFO:teuthology.orchestra.run.smithi061.stdout: mds verify scatter = True 2020-12-05T17:30:45.597 INFO:teuthology.orchestra.run.smithi061.stdout: osd op complaint time = 180 2020-12-05T17:30:45.597 INFO:teuthology.orchestra.run.smithi061.stdout: rados mon op timeout = 15m 2020-12-05T17:30:45.597 INFO:teuthology.orchestra.run.smithi061.stdout: rados osd op timeout = 15m 2020-12-05T17:30:45.597 INFO:teuthology.orchestra.run.smithi053.stdout:[global] 2020-12-05T17:30:45.598 INFO:teuthology.orchestra.run.smithi053.stdout: chdir = "" 2020-12-05T17:30:45.598 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-12-05T17:30:45.598 INFO:teuthology.orchestra.run.smithi053.stdout: auth supported = cephx 2020-12-05T17:30:45.598 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.599 INFO:teuthology.orchestra.run.smithi053.stdout: filestore xattr use omap = true 2020-12-05T17:30:45.599 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.599 INFO:teuthology.orchestra.run.smithi053.stdout: mon clock drift allowed = 1.000 2020-12-05T17:30:45.599 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.599 INFO:teuthology.orchestra.run.smithi053.stdout: osd crush chooseleaf type = 0 2020-12-05T17:30:45.600 INFO:teuthology.orchestra.run.smithi053.stdout: auth debug = true 2020-12-05T17:30:45.600 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.600 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on old message = true 2020-12-05T17:30:45.600 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on bug = true 2020-12-05T17:30:45.600 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.601 INFO:teuthology.orchestra.run.smithi053.stdout: mon max pg per osd = 10000 # >= luminous 2020-12-05T17:30:45.601 INFO:teuthology.orchestra.run.smithi053.stdout: mon pg warn max object skew = 0 2020-12-05T17:30:45.601 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.601 INFO:teuthology.orchestra.run.smithi053.stdout: # disable pg_autoscaler by default for new pools 2020-12-05T17:30:45.601 INFO:teuthology.orchestra.run.smithi053.stdout: osd_pool_default_pg_autoscale_mode = off 2020-12-05T17:30:45.602 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.602 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default size = 2 2020-12-05T17:30:45.602 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.602 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow primary affinity = true 2020-12-05T17:30:45.603 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow pg remap = true 2020-12-05T17:30:45.603 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on legacy crush tunables = false 2020-12-05T17:30:45.603 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on crush straw calc version zero = false 2020-12-05T17:30:45.603 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on no sortbitwise = false 2020-12-05T17:30:45.604 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on osd down out interval zero = false 2020-12-05T17:30:45.604 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on too few osds = false 2020-12-05T17:30:45.604 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-12-05T17:30:45.604 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_no_redundancy = false 2020-12-05T17:30:45.605 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.605 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-12-05T17:30:45.605 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.605 INFO:teuthology.orchestra.run.smithi053.stdout: osd default data pool replay window = 5 2020-12-05T17:30:45.605 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.606 INFO:teuthology.orchestra.run.smithi053.stdout: mon allow pool delete = true 2020-12-05T17:30:45.606 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.606 INFO:teuthology.orchestra.run.smithi053.stdout: mon cluster log file level = debug 2020-12-05T17:30:45.606 INFO:teuthology.orchestra.run.smithi053.stdout: debug asserts on shutdown = true 2020-12-05T17:30:45.606 INFO:teuthology.orchestra.run.smithi053.stdout: mon host = "172.21.15.32,172.21.15.61,[v2:172.21.15.61:3301,v1:172.21.15.61:6790]" 2020-12-05T17:30:45.607 INFO:teuthology.orchestra.run.smithi053.stdout: fsid = efd7e861-a502-461d-b20d-3b2c0a25fbd9 2020-12-05T17:30:45.607 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.607 INFO:teuthology.orchestra.run.smithi053.stdout:[osd] 2020-12-05T17:30:45.607 INFO:teuthology.orchestra.run.smithi053.stdout: osd journal size = 100 2020-12-05T17:30:45.607 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.608 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub load threshold = 5.0 2020-12-05T17:30:45.608 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub max interval = 600 2020-12-05T17:30:45.608 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.608 INFO:teuthology.orchestra.run.smithi053.stdout: osd recover clone overlap = true 2020-12-05T17:30:45.609 INFO:teuthology.orchestra.run.smithi053.stdout: osd recovery max chunk = 1048576 2020-12-05T17:30:45.609 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.609 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug shutdown = true 2020-12-05T17:30:45.609 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug op order = true 2020-12-05T17:30:45.609 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug verify stray on activate = true 2020-12-05T17:30:45.610 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.610 INFO:teuthology.orchestra.run.smithi053.stdout: osd open classes on start = true 2020-12-05T17:30:45.610 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug pg log writeout = true 2020-12-05T17:30:45.610 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.610 INFO:teuthology.orchestra.run.smithi053.stdout: osd deep scrub update digest min age = 30 2020-12-05T17:30:45.611 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.611 INFO:teuthology.orchestra.run.smithi053.stdout: osd map max advance = 10 2020-12-05T17:30:45.611 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.611 INFO:teuthology.orchestra.run.smithi053.stdout: journal zero on create = true 2020-12-05T17:30:45.611 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.612 INFO:teuthology.orchestra.run.smithi053.stdout: filestore ondisk finisher threads = 3 2020-12-05T17:30:45.612 INFO:teuthology.orchestra.run.smithi053.stdout: filestore apply finisher threads = 3 2020-12-05T17:30:45.612 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.612 INFO:teuthology.orchestra.run.smithi053.stdout: bdev debug aio = true 2020-12-05T17:30:45.612 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug misdirected ops = true 2020-12-05T17:30:45.613 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore block size = 96636764160 2020-12-05T17:30:45.613 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore fsck on mount = True 2020-12-05T17:30:45.613 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluefs = 20 2020-12-05T17:30:45.613 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluestore = 20 2020-12-05T17:30:45.613 INFO:teuthology.orchestra.run.smithi053.stdout: debug filestore = 20 2020-12-05T17:30:45.614 INFO:teuthology.orchestra.run.smithi053.stdout: debug journal = 20 2020-12-05T17:30:45.614 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 20 2020-12-05T17:30:45.614 INFO:teuthology.orchestra.run.smithi053.stdout: debug osd = 25 2020-12-05T17:30:45.614 INFO:teuthology.orchestra.run.smithi053.stdout: debug rocksdb = 10 2020-12-05T17:30:45.614 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd backfillfull_ratio = 0.85 2020-12-05T17:30:45.615 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd full ratio = 0.9 2020-12-05T17:30:45.615 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd nearfull ratio = 0.8 2020-12-05T17:30:45.615 INFO:teuthology.orchestra.run.smithi053.stdout: osd failsafe full ratio = 0.95 2020-12-05T17:30:45.615 INFO:teuthology.orchestra.run.smithi053.stdout: osd objectstore = bluestore 2020-12-05T17:30:45.615 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2020-12-05T17:30:45.616 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.616 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr] 2020-12-05T17:30:45.616 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2020-12-05T17:30:45.616 INFO:teuthology.orchestra.run.smithi053.stdout: debug mgr = 20 2020-12-05T17:30:45.617 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2020-12-05T17:30:45.617 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2020-12-05T17:30:45.617 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:30:45.617 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:30:45.617 INFO:teuthology.orchestra.run.smithi053.stdout: mgr/telemetry/nag = false 2020-12-05T17:30:45.618 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.618 INFO:teuthology.orchestra.run.smithi053.stdout:[mon] 2020-12-05T17:30:45.618 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2020-12-05T17:30:45.618 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2020-12-05T17:30:45.618 INFO:teuthology.orchestra.run.smithi053.stdout: debug paxos = 20 2020-12-05T17:30:45.619 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2020-12-05T17:30:45.619 INFO:teuthology.orchestra.run.smithi053.stdout: mon data avail warn = 5 2020-12-05T17:30:45.619 INFO:teuthology.orchestra.run.smithi053.stdout: mon mgr mkfs grace = 240 2020-12-05T17:30:45.619 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:30:45.619 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd reporter subtree level = osd 2020-12-05T17:30:45.620 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd prime pg temp = true 2020-12-05T17:30:45.620 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:30:45.620 INFO:teuthology.orchestra.run.smithi053.stdout: mon op complaint time = 120 2020-12-05T17:30:45.620 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:30:45.620 INFO:teuthology.orchestra.run.smithi053.stdout:[client] 2020-12-05T17:30:45.621 INFO:teuthology.orchestra.run.smithi053.stdout: rgw cache enabled = true 2020-12-05T17:30:45.621 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable ops log = true 2020-12-05T17:30:45.621 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable usage log = true 2020-12-05T17:30:45.621 INFO:teuthology.orchestra.run.smithi053.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-12-05T17:30:45.621 INFO:teuthology.orchestra.run.smithi053.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-12-05T17:30:45.622 INFO:teuthology.orchestra.run.smithi053.stdout: client mount timeout = 600 2020-12-05T17:30:45.622 INFO:teuthology.orchestra.run.smithi053.stdout: debug client = 20 2020-12-05T17:30:45.622 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2020-12-05T17:30:45.622 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 15m 2020-12-05T17:30:45.622 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 15m 2020-12-05T17:30:45.623 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.a] 2020-12-05T17:30:45.623 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.b] 2020-12-05T17:30:45.623 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.c] 2020-12-05T17:30:45.623 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.a] 2020-12-05T17:30:45.623 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.b] 2020-12-05T17:30:45.624 INFO:teuthology.orchestra.run.smithi053.stdout:[mds] 2020-12-05T17:30:45.624 INFO:teuthology.orchestra.run.smithi053.stdout: debug mds = 20 2020-12-05T17:30:45.624 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2020-12-05T17:30:45.624 INFO:teuthology.orchestra.run.smithi053.stdout: mds bal frag = True 2020-12-05T17:30:45.624 INFO:teuthology.orchestra.run.smithi053.stdout: mds bal fragment size max = 10000 2020-12-05T17:30:45.625 INFO:teuthology.orchestra.run.smithi053.stdout: mds bal merge size = 5 2020-12-05T17:30:45.625 INFO:teuthology.orchestra.run.smithi053.stdout: mds bal split bits = 3 2020-12-05T17:30:45.625 INFO:teuthology.orchestra.run.smithi053.stdout: mds bal split size = 100 2020-12-05T17:30:45.625 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug frag = True 2020-12-05T17:30:45.625 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug scatterstat = True 2020-12-05T17:30:45.626 INFO:teuthology.orchestra.run.smithi053.stdout: mds op complaint time = 180 2020-12-05T17:30:45.626 INFO:teuthology.orchestra.run.smithi053.stdout: mds verify scatter = True 2020-12-05T17:30:45.626 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2020-12-05T17:30:45.626 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 15m 2020-12-05T17:30:45.627 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 15m 2020-12-05T17:30:45.627 INFO:teuthology.orchestra.run.smithi036.stdout:[global] 2020-12-05T17:30:45.627 INFO:teuthology.orchestra.run.smithi036.stdout: chdir = "" 2020-12-05T17:30:45.627 INFO:teuthology.orchestra.run.smithi036.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-12-05T17:30:45.628 INFO:teuthology.orchestra.run.smithi036.stdout: auth supported = cephx 2020-12-05T17:30:45.628 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.628 INFO:teuthology.orchestra.run.smithi036.stdout: filestore xattr use omap = true 2020-12-05T17:30:45.628 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.629 INFO:teuthology.orchestra.run.smithi036.stdout: mon clock drift allowed = 1.000 2020-12-05T17:30:45.629 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.629 INFO:teuthology.orchestra.run.smithi036.stdout: osd crush chooseleaf type = 0 2020-12-05T17:30:45.629 INFO:teuthology.orchestra.run.smithi036.stdout: auth debug = true 2020-12-05T17:30:45.629 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.630 INFO:teuthology.orchestra.run.smithi036.stdout: ms die on old message = true 2020-12-05T17:30:45.630 INFO:teuthology.orchestra.run.smithi036.stdout: ms die on bug = true 2020-12-05T17:30:45.630 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.630 INFO:teuthology.orchestra.run.smithi036.stdout: mon max pg per osd = 10000 # >= luminous 2020-12-05T17:30:45.630 INFO:teuthology.orchestra.run.smithi036.stdout: mon pg warn max object skew = 0 2020-12-05T17:30:45.631 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.631 INFO:teuthology.orchestra.run.smithi036.stdout: # disable pg_autoscaler by default for new pools 2020-12-05T17:30:45.631 INFO:teuthology.orchestra.run.smithi036.stdout: osd_pool_default_pg_autoscale_mode = off 2020-12-05T17:30:45.631 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.631 INFO:teuthology.orchestra.run.smithi036.stdout: osd pool default size = 2 2020-12-05T17:30:45.632 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.632 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd allow primary affinity = true 2020-12-05T17:30:45.632 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd allow pg remap = true 2020-12-05T17:30:45.632 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on legacy crush tunables = false 2020-12-05T17:30:45.632 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on crush straw calc version zero = false 2020-12-05T17:30:45.633 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on no sortbitwise = false 2020-12-05T17:30:45.633 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on osd down out interval zero = false 2020-12-05T17:30:45.633 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on too few osds = false 2020-12-05T17:30:45.633 INFO:teuthology.orchestra.run.smithi036.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-12-05T17:30:45.633 INFO:teuthology.orchestra.run.smithi036.stdout: mon_warn_on_pool_no_redundancy = false 2020-12-05T17:30:45.634 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.634 INFO:teuthology.orchestra.run.smithi036.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-12-05T17:30:45.634 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.634 INFO:teuthology.orchestra.run.smithi036.stdout: osd default data pool replay window = 5 2020-12-05T17:30:45.634 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.635 INFO:teuthology.orchestra.run.smithi036.stdout: mon allow pool delete = true 2020-12-05T17:30:45.635 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.635 INFO:teuthology.orchestra.run.smithi036.stdout: mon cluster log file level = debug 2020-12-05T17:30:45.635 INFO:teuthology.orchestra.run.smithi036.stdout: debug asserts on shutdown = true 2020-12-05T17:30:45.635 INFO:teuthology.orchestra.run.smithi036.stdout: mon host = "172.21.15.32,172.21.15.61,[v2:172.21.15.61:3301,v1:172.21.15.61:6790]" 2020-12-05T17:30:45.636 INFO:teuthology.orchestra.run.smithi036.stdout: fsid = efd7e861-a502-461d-b20d-3b2c0a25fbd9 2020-12-05T17:30:45.636 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.636 INFO:teuthology.orchestra.run.smithi036.stdout:[osd] 2020-12-05T17:30:45.636 INFO:teuthology.orchestra.run.smithi036.stdout: osd journal size = 100 2020-12-05T17:30:45.636 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.637 INFO:teuthology.orchestra.run.smithi036.stdout: osd scrub load threshold = 5.0 2020-12-05T17:30:45.637 INFO:teuthology.orchestra.run.smithi036.stdout: osd scrub max interval = 600 2020-12-05T17:30:45.637 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.637 INFO:teuthology.orchestra.run.smithi036.stdout: osd recover clone overlap = true 2020-12-05T17:30:45.637 INFO:teuthology.orchestra.run.smithi036.stdout: osd recovery max chunk = 1048576 2020-12-05T17:30:45.638 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.638 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug shutdown = true 2020-12-05T17:30:45.638 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug op order = true 2020-12-05T17:30:45.638 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug verify stray on activate = true 2020-12-05T17:30:45.638 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.639 INFO:teuthology.orchestra.run.smithi036.stdout: osd open classes on start = true 2020-12-05T17:30:45.639 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug pg log writeout = true 2020-12-05T17:30:45.639 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.639 INFO:teuthology.orchestra.run.smithi036.stdout: osd deep scrub update digest min age = 30 2020-12-05T17:30:45.639 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.640 INFO:teuthology.orchestra.run.smithi036.stdout: osd map max advance = 10 2020-12-05T17:30:45.640 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.640 INFO:teuthology.orchestra.run.smithi036.stdout: journal zero on create = true 2020-12-05T17:30:45.640 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.641 INFO:teuthology.orchestra.run.smithi036.stdout: filestore ondisk finisher threads = 3 2020-12-05T17:30:45.641 INFO:teuthology.orchestra.run.smithi036.stdout: filestore apply finisher threads = 3 2020-12-05T17:30:45.641 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.641 INFO:teuthology.orchestra.run.smithi036.stdout: bdev debug aio = true 2020-12-05T17:30:45.641 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug misdirected ops = true 2020-12-05T17:30:45.642 INFO:teuthology.orchestra.run.smithi036.stdout: bluestore block size = 96636764160 2020-12-05T17:30:45.642 INFO:teuthology.orchestra.run.smithi036.stdout: bluestore fsck on mount = True 2020-12-05T17:30:45.642 INFO:teuthology.orchestra.run.smithi036.stdout: debug bluefs = 20 2020-12-05T17:30:45.642 INFO:teuthology.orchestra.run.smithi036.stdout: debug bluestore = 20 2020-12-05T17:30:45.643 INFO:teuthology.orchestra.run.smithi036.stdout: debug filestore = 20 2020-12-05T17:30:45.643 INFO:teuthology.orchestra.run.smithi036.stdout: debug journal = 20 2020-12-05T17:30:45.643 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 20 2020-12-05T17:30:45.643 INFO:teuthology.orchestra.run.smithi036.stdout: debug osd = 25 2020-12-05T17:30:45.644 INFO:teuthology.orchestra.run.smithi036.stdout: debug rocksdb = 10 2020-12-05T17:30:45.644 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd backfillfull_ratio = 0.85 2020-12-05T17:30:45.644 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd full ratio = 0.9 2020-12-05T17:30:45.644 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd nearfull ratio = 0.8 2020-12-05T17:30:45.644 INFO:teuthology.orchestra.run.smithi036.stdout: osd failsafe full ratio = 0.95 2020-12-05T17:30:45.645 INFO:teuthology.orchestra.run.smithi036.stdout: osd objectstore = bluestore 2020-12-05T17:30:45.645 INFO:teuthology.orchestra.run.smithi036.stdout: osd op complaint time = 180 2020-12-05T17:30:45.645 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.645 INFO:teuthology.orchestra.run.smithi036.stdout:[mgr] 2020-12-05T17:30:45.645 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2020-12-05T17:30:45.646 INFO:teuthology.orchestra.run.smithi036.stdout: debug mgr = 20 2020-12-05T17:30:45.646 INFO:teuthology.orchestra.run.smithi036.stdout: debug mon = 20 2020-12-05T17:30:45.646 INFO:teuthology.orchestra.run.smithi036.stdout: debug auth = 20 2020-12-05T17:30:45.646 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:30:45.646 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:30:45.647 INFO:teuthology.orchestra.run.smithi036.stdout: mgr/telemetry/nag = false 2020-12-05T17:30:45.647 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.647 INFO:teuthology.orchestra.run.smithi036.stdout:[mon] 2020-12-05T17:30:45.647 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2020-12-05T17:30:45.647 INFO:teuthology.orchestra.run.smithi036.stdout: debug mon = 20 2020-12-05T17:30:45.648 INFO:teuthology.orchestra.run.smithi036.stdout: debug paxos = 20 2020-12-05T17:30:45.648 INFO:teuthology.orchestra.run.smithi036.stdout: debug auth = 20 2020-12-05T17:30:45.648 INFO:teuthology.orchestra.run.smithi036.stdout: mon data avail warn = 5 2020-12-05T17:30:45.648 INFO:teuthology.orchestra.run.smithi036.stdout: mon mgr mkfs grace = 240 2020-12-05T17:30:45.648 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:30:45.649 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd reporter subtree level = osd 2020-12-05T17:30:45.649 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd prime pg temp = true 2020-12-05T17:30:45.649 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:30:45.649 INFO:teuthology.orchestra.run.smithi036.stdout: mon op complaint time = 120 2020-12-05T17:30:45.649 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:30:45.650 INFO:teuthology.orchestra.run.smithi036.stdout:[client] 2020-12-05T17:30:45.650 INFO:teuthology.orchestra.run.smithi036.stdout: rgw cache enabled = true 2020-12-05T17:30:45.650 INFO:teuthology.orchestra.run.smithi036.stdout: rgw enable ops log = true 2020-12-05T17:30:45.650 INFO:teuthology.orchestra.run.smithi036.stdout: rgw enable usage log = true 2020-12-05T17:30:45.650 INFO:teuthology.orchestra.run.smithi036.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-12-05T17:30:45.651 INFO:teuthology.orchestra.run.smithi036.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-12-05T17:30:45.651 INFO:teuthology.orchestra.run.smithi036.stdout: client mount timeout = 600 2020-12-05T17:30:45.651 INFO:teuthology.orchestra.run.smithi036.stdout: debug client = 20 2020-12-05T17:30:45.651 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2020-12-05T17:30:45.651 INFO:teuthology.orchestra.run.smithi036.stdout: rados mon op timeout = 15m 2020-12-05T17:30:45.652 INFO:teuthology.orchestra.run.smithi036.stdout: rados osd op timeout = 15m 2020-12-05T17:30:45.652 INFO:teuthology.orchestra.run.smithi036.stdout:[mon.a] 2020-12-05T17:30:45.652 INFO:teuthology.orchestra.run.smithi036.stdout:[mon.b] 2020-12-05T17:30:45.652 INFO:teuthology.orchestra.run.smithi036.stdout:[mon.c] 2020-12-05T17:30:45.652 INFO:teuthology.orchestra.run.smithi036.stdout:[mds.a] 2020-12-05T17:30:45.653 INFO:teuthology.orchestra.run.smithi036.stdout:[mds.b] 2020-12-05T17:30:45.653 INFO:teuthology.orchestra.run.smithi036.stdout:[mds] 2020-12-05T17:30:45.653 INFO:teuthology.orchestra.run.smithi036.stdout: debug mds = 20 2020-12-05T17:30:45.653 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2020-12-05T17:30:45.653 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal frag = True 2020-12-05T17:30:45.654 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal fragment size max = 10000 2020-12-05T17:30:45.654 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal merge size = 5 2020-12-05T17:30:45.654 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal split bits = 3 2020-12-05T17:30:45.654 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal split size = 100 2020-12-05T17:30:45.654 INFO:teuthology.orchestra.run.smithi036.stdout: mds debug frag = True 2020-12-05T17:30:45.655 INFO:teuthology.orchestra.run.smithi036.stdout: mds debug scatterstat = True 2020-12-05T17:30:45.655 INFO:teuthology.orchestra.run.smithi036.stdout: mds op complaint time = 180 2020-12-05T17:30:45.655 INFO:teuthology.orchestra.run.smithi036.stdout: mds verify scatter = True 2020-12-05T17:30:45.655 INFO:teuthology.orchestra.run.smithi036.stdout: osd op complaint time = 180 2020-12-05T17:30:45.655 INFO:teuthology.orchestra.run.smithi036.stdout: rados mon op timeout = 15m 2020-12-05T17:30:45.656 INFO:teuthology.orchestra.run.smithi036.stdout: rados osd op timeout = 15m 2020-12-05T17:30:45.656 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2020-12-05T17:30:45.656 INFO:teuthology.orchestra.run.smithi032.stdout: chdir = "" 2020-12-05T17:30:45.656 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-12-05T17:30:45.657 INFO:teuthology.orchestra.run.smithi032.stdout: auth supported = cephx 2020-12-05T17:30:45.657 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.657 INFO:teuthology.orchestra.run.smithi032.stdout: filestore xattr use omap = true 2020-12-05T17:30:45.657 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.657 INFO:teuthology.orchestra.run.smithi032.stdout: mon clock drift allowed = 1.000 2020-12-05T17:30:45.658 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.658 INFO:teuthology.orchestra.run.smithi032.stdout: osd crush chooseleaf type = 0 2020-12-05T17:30:45.658 INFO:teuthology.orchestra.run.smithi032.stdout: auth debug = true 2020-12-05T17:30:45.658 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.658 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on old message = true 2020-12-05T17:30:45.659 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on bug = true 2020-12-05T17:30:45.659 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.659 INFO:teuthology.orchestra.run.smithi032.stdout: mon max pg per osd = 10000 # >= luminous 2020-12-05T17:30:45.659 INFO:teuthology.orchestra.run.smithi032.stdout: mon pg warn max object skew = 0 2020-12-05T17:30:45.659 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.660 INFO:teuthology.orchestra.run.smithi032.stdout: # disable pg_autoscaler by default for new pools 2020-12-05T17:30:45.660 INFO:teuthology.orchestra.run.smithi032.stdout: osd_pool_default_pg_autoscale_mode = off 2020-12-05T17:30:45.660 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.660 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default size = 2 2020-12-05T17:30:45.660 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.661 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow primary affinity = true 2020-12-05T17:30:45.661 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow pg remap = true 2020-12-05T17:30:45.661 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on legacy crush tunables = false 2020-12-05T17:30:45.661 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on crush straw calc version zero = false 2020-12-05T17:30:45.661 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on no sortbitwise = false 2020-12-05T17:30:45.662 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on osd down out interval zero = false 2020-12-05T17:30:45.662 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on too few osds = false 2020-12-05T17:30:45.662 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-12-05T17:30:45.662 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_no_redundancy = false 2020-12-05T17:30:45.662 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.663 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-12-05T17:30:45.663 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.663 INFO:teuthology.orchestra.run.smithi032.stdout: osd default data pool replay window = 5 2020-12-05T17:30:45.663 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.663 INFO:teuthology.orchestra.run.smithi032.stdout: mon allow pool delete = true 2020-12-05T17:30:45.664 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.664 INFO:teuthology.orchestra.run.smithi032.stdout: mon cluster log file level = debug 2020-12-05T17:30:45.664 INFO:teuthology.orchestra.run.smithi032.stdout: debug asserts on shutdown = true 2020-12-05T17:30:45.664 INFO:teuthology.orchestra.run.smithi032.stdout: mon host = "172.21.15.32,172.21.15.61,[v2:172.21.15.61:3301,v1:172.21.15.61:6790]" 2020-12-05T17:30:45.664 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = efd7e861-a502-461d-b20d-3b2c0a25fbd9 2020-12-05T17:30:45.665 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.665 INFO:teuthology.orchestra.run.smithi032.stdout:[osd] 2020-12-05T17:30:45.665 INFO:teuthology.orchestra.run.smithi032.stdout: osd journal size = 100 2020-12-05T17:30:45.665 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.665 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub load threshold = 5.0 2020-12-05T17:30:45.666 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub max interval = 600 2020-12-05T17:30:45.666 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.666 INFO:teuthology.orchestra.run.smithi032.stdout: osd recover clone overlap = true 2020-12-05T17:30:45.666 INFO:teuthology.orchestra.run.smithi032.stdout: osd recovery max chunk = 1048576 2020-12-05T17:30:45.666 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.667 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug shutdown = true 2020-12-05T17:30:45.667 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug op order = true 2020-12-05T17:30:45.667 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug verify stray on activate = true 2020-12-05T17:30:45.667 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.667 INFO:teuthology.orchestra.run.smithi032.stdout: osd open classes on start = true 2020-12-05T17:30:45.668 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug pg log writeout = true 2020-12-05T17:30:45.668 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.668 INFO:teuthology.orchestra.run.smithi032.stdout: osd deep scrub update digest min age = 30 2020-12-05T17:30:45.668 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.668 INFO:teuthology.orchestra.run.smithi032.stdout: osd map max advance = 10 2020-12-05T17:30:45.669 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.669 INFO:teuthology.orchestra.run.smithi032.stdout: journal zero on create = true 2020-12-05T17:30:45.669 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.669 INFO:teuthology.orchestra.run.smithi032.stdout: filestore ondisk finisher threads = 3 2020-12-05T17:30:45.669 INFO:teuthology.orchestra.run.smithi032.stdout: filestore apply finisher threads = 3 2020-12-05T17:30:45.670 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.670 INFO:teuthology.orchestra.run.smithi032.stdout: bdev debug aio = true 2020-12-05T17:30:45.670 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug misdirected ops = true 2020-12-05T17:30:45.670 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore block size = 96636764160 2020-12-05T17:30:45.671 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore fsck on mount = True 2020-12-05T17:30:45.671 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluefs = 20 2020-12-05T17:30:45.671 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluestore = 20 2020-12-05T17:30:45.671 INFO:teuthology.orchestra.run.smithi032.stdout: debug filestore = 20 2020-12-05T17:30:45.671 INFO:teuthology.orchestra.run.smithi032.stdout: debug journal = 20 2020-12-05T17:30:45.672 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 20 2020-12-05T17:30:45.672 INFO:teuthology.orchestra.run.smithi032.stdout: debug osd = 25 2020-12-05T17:30:45.672 INFO:teuthology.orchestra.run.smithi032.stdout: debug rocksdb = 10 2020-12-05T17:30:45.672 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd backfillfull_ratio = 0.85 2020-12-05T17:30:45.672 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd full ratio = 0.9 2020-12-05T17:30:45.673 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd nearfull ratio = 0.8 2020-12-05T17:30:45.673 INFO:teuthology.orchestra.run.smithi032.stdout: osd failsafe full ratio = 0.95 2020-12-05T17:30:45.673 INFO:teuthology.orchestra.run.smithi032.stdout: osd objectstore = bluestore 2020-12-05T17:30:45.673 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2020-12-05T17:30:45.673 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.674 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr] 2020-12-05T17:30:45.674 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2020-12-05T17:30:45.674 INFO:teuthology.orchestra.run.smithi032.stdout: debug mgr = 20 2020-12-05T17:30:45.674 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2020-12-05T17:30:45.674 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2020-12-05T17:30:45.675 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:30:45.675 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:30:45.675 INFO:teuthology.orchestra.run.smithi032.stdout: mgr/telemetry/nag = false 2020-12-05T17:30:45.675 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.675 INFO:teuthology.orchestra.run.smithi032.stdout:[mon] 2020-12-05T17:30:45.676 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2020-12-05T17:30:45.676 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2020-12-05T17:30:45.676 INFO:teuthology.orchestra.run.smithi032.stdout: debug paxos = 20 2020-12-05T17:30:45.676 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2020-12-05T17:30:45.676 INFO:teuthology.orchestra.run.smithi032.stdout: mon data avail warn = 5 2020-12-05T17:30:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: mon mgr mkfs grace = 240 2020-12-05T17:30:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:30:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd reporter subtree level = osd 2020-12-05T17:30:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd prime pg temp = true 2020-12-05T17:30:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:30:45.678 INFO:teuthology.orchestra.run.smithi032.stdout: mon op complaint time = 120 2020-12-05T17:30:45.678 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:30:45.678 INFO:teuthology.orchestra.run.smithi032.stdout:[client] 2020-12-05T17:30:45.678 INFO:teuthology.orchestra.run.smithi032.stdout: rgw cache enabled = true 2020-12-05T17:30:45.679 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable ops log = true 2020-12-05T17:30:45.679 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable usage log = true 2020-12-05T17:30:45.679 INFO:teuthology.orchestra.run.smithi032.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-12-05T17:30:45.679 INFO:teuthology.orchestra.run.smithi032.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-12-05T17:30:45.679 INFO:teuthology.orchestra.run.smithi032.stdout: client mount timeout = 600 2020-12-05T17:30:45.680 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2020-12-05T17:30:45.680 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2020-12-05T17:30:45.680 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 15m 2020-12-05T17:30:45.680 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 15m 2020-12-05T17:30:45.680 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.a] 2020-12-05T17:30:45.681 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.b] 2020-12-05T17:30:45.681 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.c] 2020-12-05T17:30:45.681 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2020-12-05T17:30:45.681 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2020-12-05T17:30:45.681 INFO:teuthology.orchestra.run.smithi032.stdout:[mds] 2020-12-05T17:30:45.682 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds = 20 2020-12-05T17:30:45.682 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2020-12-05T17:30:45.682 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal frag = True 2020-12-05T17:30:45.682 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal fragment size max = 10000 2020-12-05T17:30:45.682 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal merge size = 5 2020-12-05T17:30:45.683 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal split bits = 3 2020-12-05T17:30:45.683 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal split size = 100 2020-12-05T17:30:45.683 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug frag = True 2020-12-05T17:30:45.683 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug scatterstat = True 2020-12-05T17:30:45.683 INFO:teuthology.orchestra.run.smithi032.stdout: mds op complaint time = 180 2020-12-05T17:30:45.684 INFO:teuthology.orchestra.run.smithi032.stdout: mds verify scatter = True 2020-12-05T17:30:45.684 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2020-12-05T17:30:45.684 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 15m 2020-12-05T17:30:45.684 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 15m 2020-12-05T17:30:45.686 INFO:tasks.ceph:Creating admin key on mon.a... 2020-12-05T17:30:45.687 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2020-12-05T17:30:45.736 INFO:tasks.ceph:Copying monmap to all nodes... 2020-12-05T17:30:45.743 DEBUG:teuthology.orchestra.remote:smithi032:/etc/ceph/ceph.keyring is 206B 2020-12-05T17:30:45.801 DEBUG:teuthology.orchestra.remote:smithi032:/home/ubuntu/cephtest/ceph.monmap is 379B 2020-12-05T17:30:45.812 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi032.front.sepia.ceph.com 2020-12-05T17:30:45.812 INFO:teuthology.orchestra.run.smithi032:> set -ex 2020-12-05T17:30:45.813 INFO:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.keyring 2020-12-05T17:30:45.813 INFO:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-12-05T17:30:45.818 INFO:teuthology.orchestra.run.smithi032.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-12-05T17:30:45.827 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2020-12-05T17:30:45.827 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2020-12-05T17:30:45.827 INFO:teuthology.orchestra.run.smithi032.stderr:206 bytes copied, 0.00013769 s, 1.5 MB/s 2020-12-05T17:30:45.828 INFO:teuthology.orchestra.run.smithi032.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-12-05T17:30:45.837 INFO:teuthology.orchestra.run.smithi032:> set -ex 2020-12-05T17:30:45.837 INFO:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-12-05T17:30:45.884 INFO:teuthology.orchestra.run.smithi032.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-12-05T17:30:45.886 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2020-12-05T17:30:45.886 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2020-12-05T17:30:45.886 INFO:teuthology.orchestra.run.smithi032.stderr:379 bytes copied, 0.000130367 s, 2.9 MB/s 2020-12-05T17:30:45.887 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi061.front.sepia.ceph.com 2020-12-05T17:30:45.887 INFO:teuthology.orchestra.run.smithi061:> set -ex 2020-12-05T17:30:45.887 INFO:teuthology.orchestra.run.smithi061:> sudo dd of=/etc/ceph/ceph.keyring 2020-12-05T17:30:45.888 INFO:teuthology.orchestra.run.smithi061:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-12-05T17:30:45.893 INFO:teuthology.orchestra.run.smithi061.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-12-05T17:30:45.902 INFO:teuthology.orchestra.run.smithi061.stderr:0+1 records in 2020-12-05T17:30:45.902 INFO:teuthology.orchestra.run.smithi061.stderr:0+1 records out 2020-12-05T17:30:45.902 INFO:teuthology.orchestra.run.smithi061.stderr:206 bytes copied, 7.0523e-05 s, 2.9 MB/s 2020-12-05T17:30:45.903 INFO:teuthology.orchestra.run.smithi061.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-12-05T17:30:45.913 INFO:teuthology.orchestra.run.smithi061:> set -ex 2020-12-05T17:30:45.914 INFO:teuthology.orchestra.run.smithi061:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-12-05T17:30:45.965 INFO:teuthology.orchestra.run.smithi061.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-12-05T17:30:45.967 INFO:teuthology.orchestra.run.smithi061.stderr:0+1 records in 2020-12-05T17:30:45.967 INFO:teuthology.orchestra.run.smithi061.stderr:0+1 records out 2020-12-05T17:30:45.967 INFO:teuthology.orchestra.run.smithi061.stderr:379 bytes copied, 7.5703e-05 s, 5.0 MB/s 2020-12-05T17:30:45.968 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi036.front.sepia.ceph.com 2020-12-05T17:30:45.969 INFO:teuthology.orchestra.run.smithi036:> set -ex 2020-12-05T17:30:45.969 INFO:teuthology.orchestra.run.smithi036:> sudo dd of=/etc/ceph/ceph.keyring 2020-12-05T17:30:45.970 INFO:teuthology.orchestra.run.smithi036:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-12-05T17:30:45.975 INFO:teuthology.orchestra.run.smithi036.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-12-05T17:30:45.985 INFO:teuthology.orchestra.run.smithi036.stderr:0+1 records in 2020-12-05T17:30:45.985 INFO:teuthology.orchestra.run.smithi036.stderr:0+1 records out 2020-12-05T17:30:45.985 INFO:teuthology.orchestra.run.smithi036.stderr:206 bytes copied, 7.4434e-05 s, 2.8 MB/s 2020-12-05T17:30:45.986 INFO:teuthology.orchestra.run.smithi036.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-12-05T17:30:45.996 INFO:teuthology.orchestra.run.smithi036:> set -ex 2020-12-05T17:30:45.996 INFO:teuthology.orchestra.run.smithi036:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-12-05T17:30:46.042 INFO:teuthology.orchestra.run.smithi036.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-12-05T17:30:46.043 INFO:teuthology.orchestra.run.smithi036.stderr:0+1 records in 2020-12-05T17:30:46.043 INFO:teuthology.orchestra.run.smithi036.stderr:0+1 records out 2020-12-05T17:30:46.043 INFO:teuthology.orchestra.run.smithi036.stderr:379 bytes copied, 7.9643e-05 s, 4.8 MB/s 2020-12-05T17:30:46.044 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi002.front.sepia.ceph.com 2020-12-05T17:30:46.044 INFO:teuthology.orchestra.run.smithi002:> set -ex 2020-12-05T17:30:46.045 INFO:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.keyring 2020-12-05T17:30:46.045 INFO:teuthology.orchestra.run.smithi002:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-12-05T17:30:46.050 INFO:teuthology.orchestra.run.smithi002.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-12-05T17:30:46.058 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records in 2020-12-05T17:30:46.058 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records out 2020-12-05T17:30:46.059 INFO:teuthology.orchestra.run.smithi002.stderr:206 bytes copied, 7.8503e-05 s, 2.6 MB/s 2020-12-05T17:30:46.059 INFO:teuthology.orchestra.run.smithi002.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-12-05T17:30:46.069 INFO:teuthology.orchestra.run.smithi002:> set -ex 2020-12-05T17:30:46.069 INFO:teuthology.orchestra.run.smithi002:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-12-05T17:30:46.116 INFO:teuthology.orchestra.run.smithi002.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-12-05T17:30:46.118 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records in 2020-12-05T17:30:46.120 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records out 2020-12-05T17:30:46.120 INFO:teuthology.orchestra.run.smithi002.stderr:379 bytes copied, 7.376e-05 s, 5.1 MB/s 2020-12-05T17:30:46.121 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi053.front.sepia.ceph.com 2020-12-05T17:30:46.121 INFO:teuthology.orchestra.run.smithi053:> set -ex 2020-12-05T17:30:46.122 INFO:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.keyring 2020-12-05T17:30:46.122 INFO:teuthology.orchestra.run.smithi053:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-12-05T17:30:46.128 INFO:teuthology.orchestra.run.smithi053.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-12-05T17:30:46.138 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records in 2020-12-05T17:30:46.138 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records out 2020-12-05T17:30:46.138 INFO:teuthology.orchestra.run.smithi053.stderr:206 bytes copied, 7.377e-05 s, 2.8 MB/s 2020-12-05T17:30:46.139 INFO:teuthology.orchestra.run.smithi053.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-12-05T17:30:46.148 INFO:teuthology.orchestra.run.smithi053:> set -ex 2020-12-05T17:30:46.148 INFO:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-12-05T17:30:46.194 INFO:teuthology.orchestra.run.smithi053.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-12-05T17:30:46.195 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records in 2020-12-05T17:30:46.195 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records out 2020-12-05T17:30:46.196 INFO:teuthology.orchestra.run.smithi053.stderr:379 bytes copied, 6.4477e-05 s, 5.9 MB/s 2020-12-05T17:30:46.196 INFO:tasks.ceph:Setting up mon nodes... 2020-12-05T17:30:46.196 INFO:tasks.ceph:Setting up mgr nodes... 2020-12-05T17:30:46.197 INFO:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2020-12-05T17:30:46.239 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2020-12-05T17:30:46.242 INFO:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2020-12-05T17:30:46.284 INFO:teuthology.orchestra.run.smithi061.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2020-12-05T17:30:46.287 INFO:tasks.ceph:Setting up mds nodes... 2020-12-05T17:30:46.287 INFO:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2020-12-05T17:30:46.327 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2020-12-05T17:30:46.330 INFO:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2020-12-05T17:30:46.388 INFO:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2020-12-05T17:30:46.430 INFO:teuthology.orchestra.run.smithi061.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2020-12-05T17:30:46.433 INFO:teuthology.orchestra.run.smithi061:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2020-12-05T17:30:46.491 INFO:tasks.ceph_client:Setting up client nodes... 2020-12-05T17:30:46.491 INFO:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2020-12-05T17:30:46.525 INFO:teuthology.orchestra.run.smithi036.stdout:creating /etc/ceph/ceph.client.0.keyring 2020-12-05T17:30:46.537 INFO:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2020-12-05T17:30:46.566 INFO:teuthology.orchestra.run.smithi002.stdout:creating /etc/ceph/ceph.client.1.keyring 2020-12-05T17:30:46.578 INFO:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2020-12-05T17:30:46.611 INFO:teuthology.orchestra.run.smithi053.stdout:creating /etc/ceph/ceph.client.2.keyring 2020-12-05T17:30:46.621 INFO:tasks.ceph:Running mkfs on osd nodes... 2020-12-05T17:30:46.622 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi061.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-12-05T17:30:46.622 INFO:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2020-12-05T17:30:46.638 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-12-05T17:30:46.639 INFO:tasks.ceph:role: osd.0 2020-12-05T17:30:46.639 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com 2020-12-05T17:30:46.639 INFO:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-12-05T17:30:46.720 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-12-05T17:30:46.721 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2020-12-05T17:30:46.721 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-12-05T17:30:46.721 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-12-05T17:30:46.722 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2020-12-05T17:30:46.722 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-12-05T17:30:46.722 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-12-05T17:30:46.722 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-12-05T17:30:46.722 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-12-05T17:30:46.723 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2020-12-05T17:30:46.724 INFO:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2020-12-05T17:30:46.783 INFO:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2020-12-05T17:30:46.835 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2020-12-05T17:30:46.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:30:46.837 INFO:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2020-12-05T17:30:46.895 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-12-05T17:30:46.895 INFO:tasks.ceph:role: osd.1 2020-12-05T17:30:46.895 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com 2020-12-05T17:30:46.896 INFO:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-12-05T17:30:46.977 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-12-05T17:30:46.978 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2020-12-05T17:30:46.978 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-12-05T17:30:46.979 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-12-05T17:30:46.979 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2020-12-05T17:30:46.979 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-12-05T17:30:46.980 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-12-05T17:30:46.980 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-12-05T17:30:46.980 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-12-05T17:30:46.981 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2020-12-05T17:30:46.981 INFO:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2020-12-05T17:30:47.045 INFO:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2020-12-05T17:30:47.099 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2020-12-05T17:30:47.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:30:47.102 INFO:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2020-12-05T17:30:47.159 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-12-05T17:30:47.160 INFO:tasks.ceph:role: osd.2 2020-12-05T17:30:47.161 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com 2020-12-05T17:30:47.162 INFO:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-12-05T17:30:47.240 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-12-05T17:30:47.240 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2020-12-05T17:30:47.241 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-12-05T17:30:47.241 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-12-05T17:30:47.241 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2020-12-05T17:30:47.241 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-12-05T17:30:47.241 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-12-05T17:30:47.242 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-12-05T17:30:47.242 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-12-05T17:30:47.243 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2020-12-05T17:30:47.243 INFO:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2020-12-05T17:30:47.305 INFO:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2020-12-05T17:30:47.359 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2020-12-05T17:30:47.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:30:47.361 INFO:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2020-12-05T17:30:47.419 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-12-05T17:30:47.419 INFO:tasks.ceph:role: osd.3 2020-12-05T17:30:47.419 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com 2020-12-05T17:30:47.420 INFO:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-12-05T17:30:47.500 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-12-05T17:30:47.501 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2020-12-05T17:30:47.501 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-12-05T17:30:47.501 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-12-05T17:30:47.501 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2020-12-05T17:30:47.502 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-12-05T17:30:47.502 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-12-05T17:30:47.502 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-12-05T17:30:47.502 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-12-05T17:30:47.503 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2020-12-05T17:30:47.504 INFO:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2020-12-05T17:30:47.565 INFO:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2020-12-05T17:30:47.618 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2020-12-05T17:30:47.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:30:47.620 INFO:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-12-05T17:30:47.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:47.704+0000 7f72b13f4d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2020-12-05T17:30:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:47.704+0000 7f72b13f4d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2020-12-05T17:30:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:47.704+0000 7f72b13f4d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-12-05T17:30:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:47.704+0000 7f72b13f4d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-12-05T17:30:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:47.704+0000 7f72b13f4d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-12-05T17:30:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:47.708+0000 7f72b13f4d80 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2020-12-05T17:30:47.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:47.720+0000 7f72b13f4d80 -1 freelist read_size_meta_from_db missing size meta in DB 2020-12-05T17:30:50.147 INFO:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2020-12-05T17:30:50.204 INFO:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-12-05T17:30:50.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:50.284+0000 7f24fa634d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2020-12-05T17:30:50.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:50.284+0000 7f24fa634d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2020-12-05T17:30:50.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:50.284+0000 7f24fa634d80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-12-05T17:30:50.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:50.284+0000 7f24fa634d80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-12-05T17:30:50.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:50.284+0000 7f24fa634d80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-12-05T17:30:50.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:50.284+0000 7f24fa634d80 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2020-12-05T17:30:50.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:50.296+0000 7f24fa634d80 -1 freelist read_size_meta_from_db missing size meta in DB 2020-12-05T17:30:52.723 INFO:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2020-12-05T17:30:52.781 INFO:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-12-05T17:30:52.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:52.864+0000 7fc1ae940d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2020-12-05T17:30:52.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:52.864+0000 7fc1ae940d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2020-12-05T17:30:52.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:52.864+0000 7fc1ae940d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-12-05T17:30:52.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:52.864+0000 7fc1ae940d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-12-05T17:30:52.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:52.864+0000 7fc1ae940d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-12-05T17:30:52.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:52.864+0000 7fc1ae940d80 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2020-12-05T17:30:52.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:52.880+0000 7fc1ae940d80 -1 freelist read_size_meta_from_db missing size meta in DB 2020-12-05T17:30:55.322 INFO:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2020-12-05T17:30:55.384 INFO:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-12-05T17:30:55.462 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:55.464+0000 7fe0f32e3d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2020-12-05T17:30:55.463 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:55.464+0000 7fe0f32e3d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2020-12-05T17:30:55.463 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:55.464+0000 7fe0f32e3d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-12-05T17:30:55.463 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:55.464+0000 7fe0f32e3d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-12-05T17:30:55.463 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:55.464+0000 7fe0f32e3d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-12-05T17:30:55.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:55.464+0000 7fe0f32e3d80 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2020-12-05T17:30:55.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:30:55.476+0000 7fe0f32e3d80 -1 freelist read_size_meta_from_db missing size meta in DB 2020-12-05T17:30:57.906 INFO:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2020-12-05T17:30:57.963 INFO:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2020-12-05T17:30:57.979 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-12-05T17:30:57.979 INFO:tasks.ceph:role: osd.4 2020-12-05T17:30:57.979 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi061.front.sepia.ceph.com 2020-12-05T17:30:57.980 INFO:teuthology.orchestra.run.smithi061:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-12-05T17:30:58.508 INFO:teuthology.orchestra.run.smithi061.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-12-05T17:30:58.508 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 attr=2, projid32bit=1 2020-12-05T17:30:58.509 INFO:teuthology.orchestra.run.smithi061.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-12-05T17:30:58.509 INFO:teuthology.orchestra.run.smithi061.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-12-05T17:30:58.509 INFO:teuthology.orchestra.run.smithi061.stdout: = sunit=0 swidth=0 blks 2020-12-05T17:30:58.509 INFO:teuthology.orchestra.run.smithi061.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-12-05T17:30:58.509 INFO:teuthology.orchestra.run.smithi061.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-12-05T17:30:58.510 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-12-05T17:30:58.510 INFO:teuthology.orchestra.run.smithi061.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-12-05T17:30:58.511 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi061.front.sepia.ceph.com -o noatime 2020-12-05T17:30:58.511 INFO:teuthology.orchestra.run.smithi061:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2020-12-05T17:30:58.598 INFO:teuthology.orchestra.run.smithi061:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2020-12-05T17:30:58.611 INFO:teuthology.orchestra.run.smithi061.stderr:sudo: /sbin/restorecon: command not found 2020-12-05T17:30:58.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:30:58.612 INFO:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2020-12-05T17:30:58.666 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-12-05T17:30:58.666 INFO:tasks.ceph:role: osd.5 2020-12-05T17:30:58.667 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi061.front.sepia.ceph.com 2020-12-05T17:30:58.668 INFO:teuthology.orchestra.run.smithi061:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-12-05T17:30:59.181 INFO:teuthology.orchestra.run.smithi061.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-12-05T17:30:59.181 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 attr=2, projid32bit=1 2020-12-05T17:30:59.181 INFO:teuthology.orchestra.run.smithi061.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-12-05T17:30:59.182 INFO:teuthology.orchestra.run.smithi061.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-12-05T17:30:59.182 INFO:teuthology.orchestra.run.smithi061.stdout: = sunit=0 swidth=0 blks 2020-12-05T17:30:59.182 INFO:teuthology.orchestra.run.smithi061.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-12-05T17:30:59.182 INFO:teuthology.orchestra.run.smithi061.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-12-05T17:30:59.183 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-12-05T17:30:59.183 INFO:teuthology.orchestra.run.smithi061.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-12-05T17:30:59.184 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi061.front.sepia.ceph.com -o noatime 2020-12-05T17:30:59.184 INFO:teuthology.orchestra.run.smithi061:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2020-12-05T17:30:59.235 INFO:teuthology.orchestra.run.smithi061:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2020-12-05T17:30:59.246 INFO:teuthology.orchestra.run.smithi061.stderr:sudo: /sbin/restorecon: command not found 2020-12-05T17:30:59.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:30:59.248 INFO:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2020-12-05T17:30:59.301 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-12-05T17:30:59.302 INFO:tasks.ceph:role: osd.6 2020-12-05T17:30:59.302 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi061.front.sepia.ceph.com 2020-12-05T17:30:59.303 INFO:teuthology.orchestra.run.smithi061:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-12-05T17:30:59.795 INFO:teuthology.orchestra.run.smithi061.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-12-05T17:30:59.796 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 attr=2, projid32bit=1 2020-12-05T17:30:59.796 INFO:teuthology.orchestra.run.smithi061.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-12-05T17:30:59.796 INFO:teuthology.orchestra.run.smithi061.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-12-05T17:30:59.796 INFO:teuthology.orchestra.run.smithi061.stdout: = sunit=0 swidth=0 blks 2020-12-05T17:30:59.796 INFO:teuthology.orchestra.run.smithi061.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-12-05T17:30:59.797 INFO:teuthology.orchestra.run.smithi061.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-12-05T17:30:59.797 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-12-05T17:30:59.797 INFO:teuthology.orchestra.run.smithi061.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-12-05T17:30:59.798 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi061.front.sepia.ceph.com -o noatime 2020-12-05T17:30:59.799 INFO:teuthology.orchestra.run.smithi061:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2020-12-05T17:30:59.838 INFO:teuthology.orchestra.run.smithi061:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2020-12-05T17:30:59.891 INFO:teuthology.orchestra.run.smithi061.stderr:sudo: /sbin/restorecon: command not found 2020-12-05T17:30:59.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:30:59.893 INFO:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2020-12-05T17:30:59.950 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-12-05T17:30:59.950 INFO:tasks.ceph:role: osd.7 2020-12-05T17:30:59.951 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi061.front.sepia.ceph.com 2020-12-05T17:30:59.951 INFO:teuthology.orchestra.run.smithi061:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-12-05T17:31:00.502 INFO:teuthology.orchestra.run.smithi061.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-12-05T17:31:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 attr=2, projid32bit=1 2020-12-05T17:31:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-12-05T17:31:00.503 INFO:teuthology.orchestra.run.smithi061.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-12-05T17:31:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: = sunit=0 swidth=0 blks 2020-12-05T17:31:00.504 INFO:teuthology.orchestra.run.smithi061.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-12-05T17:31:00.504 INFO:teuthology.orchestra.run.smithi061.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-12-05T17:31:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-12-05T17:31:00.504 INFO:teuthology.orchestra.run.smithi061.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-12-05T17:31:00.505 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi061.front.sepia.ceph.com -o noatime 2020-12-05T17:31:00.506 INFO:teuthology.orchestra.run.smithi061:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2020-12-05T17:31:00.553 INFO:teuthology.orchestra.run.smithi061:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2020-12-05T17:31:00.563 INFO:teuthology.orchestra.run.smithi061.stderr:sudo: /sbin/restorecon: command not found 2020-12-05T17:31:00.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:31:00.565 INFO:teuthology.orchestra.run.smithi061:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-12-05T17:31:00.651 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:00.651+0000 7f5a841abd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2020-12-05T17:31:00.653 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:00.655+0000 7f5a841abd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2020-12-05T17:31:00.653 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:00.655+0000 7f5a841abd80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-12-05T17:31:00.654 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:00.655+0000 7f5a841abd80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-12-05T17:31:00.654 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:00.655+0000 7f5a841abd80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-12-05T17:31:00.665 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:00.667+0000 7f5a841abd80 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2020-12-05T17:31:00.704 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:00.703+0000 7f5a841abd80 -1 freelist read_size_meta_from_db missing size meta in DB 2020-12-05T17:31:03.115 INFO:teuthology.orchestra.run.smithi061:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2020-12-05T17:31:03.172 INFO:teuthology.orchestra.run.smithi061:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-12-05T17:31:03.254 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:03.255+0000 7f643f4ead80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2020-12-05T17:31:03.255 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:03.255+0000 7f643f4ead80 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2020-12-05T17:31:03.255 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:03.255+0000 7f643f4ead80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-12-05T17:31:03.255 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:03.255+0000 7f643f4ead80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-12-05T17:31:03.255 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:03.255+0000 7f643f4ead80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-12-05T17:31:03.256 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:03.255+0000 7f643f4ead80 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2020-12-05T17:31:03.269 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:03.271+0000 7f643f4ead80 -1 freelist read_size_meta_from_db missing size meta in DB 2020-12-05T17:31:05.707 INFO:teuthology.orchestra.run.smithi061:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2020-12-05T17:31:05.763 INFO:teuthology.orchestra.run.smithi061:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-12-05T17:31:05.843 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:05.843+0000 7fd99f7acd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2020-12-05T17:31:05.843 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:05.843+0000 7fd99f7acd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2020-12-05T17:31:05.843 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:05.843+0000 7fd99f7acd80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-12-05T17:31:05.844 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:05.843+0000 7fd99f7acd80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-12-05T17:31:05.844 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:05.843+0000 7fd99f7acd80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-12-05T17:31:05.844 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:05.843+0000 7fd99f7acd80 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2020-12-05T17:31:05.861 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:05.863+0000 7fd99f7acd80 -1 freelist read_size_meta_from_db missing size meta in DB 2020-12-05T17:31:08.310 INFO:teuthology.orchestra.run.smithi061:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2020-12-05T17:31:08.367 INFO:teuthology.orchestra.run.smithi061:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-12-05T17:31:08.446 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:08.447+0000 7fbca22a7d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2020-12-05T17:31:08.447 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:08.447+0000 7fbca22a7d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2020-12-05T17:31:08.447 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:08.447+0000 7fbca22a7d80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-12-05T17:31:08.447 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:08.447+0000 7fbca22a7d80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-12-05T17:31:08.448 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:08.447+0000 7fbca22a7d80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-12-05T17:31:08.448 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:08.447+0000 7fbca22a7d80 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2020-12-05T17:31:08.467 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:08.467+0000 7fbca22a7d80 -1 freelist read_size_meta_from_db missing size meta in DB 2020-12-05T17:31:10.910 INFO:teuthology.orchestra.run.smithi061:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2020-12-05T17:31:10.967 INFO:tasks.ceph:Reading keys from all nodes... 2020-12-05T17:31:10.968 INFO:teuthology.orchestra.run.smithi032:> mktemp 2020-12-05T17:31:10.975 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.i4CVgtIKDA 2020-12-05T17:31:10.975 INFO:teuthology.orchestra.run.smithi032:> sudo cp /var/lib/ceph/mgr/ceph-y/keyring /tmp/tmp.i4CVgtIKDA 2020-12-05T17:31:11.033 INFO:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.i4CVgtIKDA 2020-12-05T17:31:11.139 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.i4CVgtIKDA is 56B 2020-12-05T17:31:11.151 INFO:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.i4CVgtIKDA 2020-12-05T17:31:11.159 INFO:teuthology.orchestra.run.smithi032:> mktemp 2020-12-05T17:31:11.204 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.nJT3VT54rk 2020-12-05T17:31:11.205 INFO:teuthology.orchestra.run.smithi032:> sudo cp /var/lib/ceph/mds/ceph-a/keyring /tmp/tmp.nJT3VT54rk 2020-12-05T17:31:11.263 INFO:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.nJT3VT54rk 2020-12-05T17:31:11.371 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.nJT3VT54rk is 56B 2020-12-05T17:31:11.383 INFO:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.nJT3VT54rk 2020-12-05T17:31:11.389 INFO:teuthology.orchestra.run.smithi032:> mktemp 2020-12-05T17:31:11.437 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.JXqyiqqJNV 2020-12-05T17:31:11.438 INFO:teuthology.orchestra.run.smithi032:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.JXqyiqqJNV 2020-12-05T17:31:11.495 INFO:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.JXqyiqqJNV 2020-12-05T17:31:11.598 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.JXqyiqqJNV is 56B 2020-12-05T17:31:11.609 INFO:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.JXqyiqqJNV 2020-12-05T17:31:11.615 INFO:teuthology.orchestra.run.smithi032:> mktemp 2020-12-05T17:31:11.661 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.nzIEkeEoa5 2020-12-05T17:31:11.661 INFO:teuthology.orchestra.run.smithi032:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.nzIEkeEoa5 2020-12-05T17:31:11.718 INFO:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.nzIEkeEoa5 2020-12-05T17:31:11.823 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.nzIEkeEoa5 is 56B 2020-12-05T17:31:11.834 INFO:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.nzIEkeEoa5 2020-12-05T17:31:11.839 INFO:teuthology.orchestra.run.smithi032:> mktemp 2020-12-05T17:31:11.885 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.UsTKC1bkUI 2020-12-05T17:31:11.885 INFO:teuthology.orchestra.run.smithi032:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.UsTKC1bkUI 2020-12-05T17:31:11.942 INFO:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.UsTKC1bkUI 2020-12-05T17:31:12.051 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.UsTKC1bkUI is 56B 2020-12-05T17:31:12.062 INFO:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.UsTKC1bkUI 2020-12-05T17:31:12.068 INFO:teuthology.orchestra.run.smithi032:> mktemp 2020-12-05T17:31:12.118 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.vSGE4tOTrL 2020-12-05T17:31:12.118 INFO:teuthology.orchestra.run.smithi032:> sudo cp /var/lib/ceph/osd/ceph-3/keyring /tmp/tmp.vSGE4tOTrL 2020-12-05T17:31:12.175 INFO:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.vSGE4tOTrL 2020-12-05T17:31:12.283 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.vSGE4tOTrL is 56B 2020-12-05T17:31:12.294 INFO:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.vSGE4tOTrL 2020-12-05T17:31:12.299 INFO:teuthology.orchestra.run.smithi061:> mktemp 2020-12-05T17:31:12.305 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/tmp.VuCvBaQ1ib 2020-12-05T17:31:12.306 INFO:teuthology.orchestra.run.smithi061:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.VuCvBaQ1ib 2020-12-05T17:31:12.361 INFO:teuthology.orchestra.run.smithi061:> sudo chmod 0666 /tmp/tmp.VuCvBaQ1ib 2020-12-05T17:31:12.468 DEBUG:teuthology.orchestra.remote:smithi061:/tmp/tmp.VuCvBaQ1ib is 56B 2020-12-05T17:31:12.479 INFO:teuthology.orchestra.run.smithi061:> rm -fr /tmp/tmp.VuCvBaQ1ib 2020-12-05T17:31:12.485 INFO:teuthology.orchestra.run.smithi061:> mktemp 2020-12-05T17:31:12.537 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/tmp.0bhEeeu23B 2020-12-05T17:31:12.538 INFO:teuthology.orchestra.run.smithi061:> sudo cp /var/lib/ceph/mds/ceph-b/keyring /tmp/tmp.0bhEeeu23B 2020-12-05T17:31:12.594 INFO:teuthology.orchestra.run.smithi061:> sudo chmod 0666 /tmp/tmp.0bhEeeu23B 2020-12-05T17:31:12.700 DEBUG:teuthology.orchestra.remote:smithi061:/tmp/tmp.0bhEeeu23B is 56B 2020-12-05T17:31:12.710 INFO:teuthology.orchestra.run.smithi061:> rm -fr /tmp/tmp.0bhEeeu23B 2020-12-05T17:31:12.716 INFO:teuthology.orchestra.run.smithi061:> mktemp 2020-12-05T17:31:12.762 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/tmp.irs2q9WJr4 2020-12-05T17:31:12.762 INFO:teuthology.orchestra.run.smithi061:> sudo cp /var/lib/ceph/osd/ceph-4/keyring /tmp/tmp.irs2q9WJr4 2020-12-05T17:31:12.819 INFO:teuthology.orchestra.run.smithi061:> sudo chmod 0666 /tmp/tmp.irs2q9WJr4 2020-12-05T17:31:12.924 DEBUG:teuthology.orchestra.remote:smithi061:/tmp/tmp.irs2q9WJr4 is 56B 2020-12-05T17:31:12.935 INFO:teuthology.orchestra.run.smithi061:> rm -fr /tmp/tmp.irs2q9WJr4 2020-12-05T17:31:12.941 INFO:teuthology.orchestra.run.smithi061:> mktemp 2020-12-05T17:31:12.990 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/tmp.6VidYjUl8v 2020-12-05T17:31:12.991 INFO:teuthology.orchestra.run.smithi061:> sudo cp /var/lib/ceph/osd/ceph-5/keyring /tmp/tmp.6VidYjUl8v 2020-12-05T17:31:13.047 INFO:teuthology.orchestra.run.smithi061:> sudo chmod 0666 /tmp/tmp.6VidYjUl8v 2020-12-05T17:31:13.152 DEBUG:teuthology.orchestra.remote:smithi061:/tmp/tmp.6VidYjUl8v is 56B 2020-12-05T17:31:13.163 INFO:teuthology.orchestra.run.smithi061:> rm -fr /tmp/tmp.6VidYjUl8v 2020-12-05T17:31:13.169 INFO:teuthology.orchestra.run.smithi061:> mktemp 2020-12-05T17:31:13.218 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/tmp.1MXUWMYcUC 2020-12-05T17:31:13.218 INFO:teuthology.orchestra.run.smithi061:> sudo cp /var/lib/ceph/osd/ceph-6/keyring /tmp/tmp.1MXUWMYcUC 2020-12-05T17:31:13.275 INFO:teuthology.orchestra.run.smithi061:> sudo chmod 0666 /tmp/tmp.1MXUWMYcUC 2020-12-05T17:31:13.380 DEBUG:teuthology.orchestra.remote:smithi061:/tmp/tmp.1MXUWMYcUC is 56B 2020-12-05T17:31:13.430 INFO:teuthology.orchestra.run.smithi061:> rm -fr /tmp/tmp.1MXUWMYcUC 2020-12-05T17:31:13.436 INFO:teuthology.orchestra.run.smithi061:> mktemp 2020-12-05T17:31:13.482 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/tmp.sUlMnWXBPJ 2020-12-05T17:31:13.483 INFO:teuthology.orchestra.run.smithi061:> sudo cp /var/lib/ceph/osd/ceph-7/keyring /tmp/tmp.sUlMnWXBPJ 2020-12-05T17:31:13.540 INFO:teuthology.orchestra.run.smithi061:> sudo chmod 0666 /tmp/tmp.sUlMnWXBPJ 2020-12-05T17:31:13.644 DEBUG:teuthology.orchestra.remote:smithi061:/tmp/tmp.sUlMnWXBPJ is 56B 2020-12-05T17:31:13.655 INFO:teuthology.orchestra.run.smithi061:> rm -fr /tmp/tmp.sUlMnWXBPJ 2020-12-05T17:31:13.670 DEBUG:teuthology.orchestra.remote:smithi036:/etc/ceph/ceph.client.0.keyring is 59B 2020-12-05T17:31:13.690 DEBUG:teuthology.orchestra.remote:smithi002:/etc/ceph/ceph.client.1.keyring is 59B 2020-12-05T17:31:13.709 DEBUG:teuthology.orchestra.remote:smithi053:/etc/ceph/ceph.client.2.keyring is 59B 2020-12-05T17:31:13.719 INFO:tasks.ceph:Adding keys to all mons... 2020-12-05T17:31:13.720 INFO:teuthology.orchestra.run.smithi032:> sudo tee -a /etc/ceph/ceph.keyring 2020-12-05T17:31:13.722 INFO:teuthology.orchestra.run.smithi061:> sudo tee -a /etc/ceph/ceph.keyring 2020-12-05T17:31:13.734 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.y] 2020-12-05T17:31:13.734 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDGw8tf3HmcDhAATNKHhH9zgasedaNEzgG0OQ== 2020-12-05T17:31:13.735 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2020-12-05T17:31:13.735 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDGw8tfpb/aExAAdMKW4yIgC2zhkzyW6l8xug== 2020-12-05T17:31:13.735 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.0] 2020-12-05T17:31:13.735 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDHw8tflo5EKhAAwV9D64fAoY3uvXCm+Aksag== 2020-12-05T17:31:13.735 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.1] 2020-12-05T17:31:13.736 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDKw8tfTjEiERAAkW6mkgApoXwyF49kPjZBGg== 2020-12-05T17:31:13.736 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.2] 2020-12-05T17:31:13.736 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDMw8tfLEzBMxAAiJ2NifsOLEkt1e3vmhwSTA== 2020-12-05T17:31:13.736 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.3] 2020-12-05T17:31:13.737 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDPw8tfV+vnGxAAQHvXJRW5X9Vhe/eYJBJ/uQ== 2020-12-05T17:31:13.737 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.x] 2020-12-05T17:31:13.737 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDGw8tfDr8mERAAS7bpAQXgW78W8GDq1v1zaA== 2020-12-05T17:31:13.737 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2020-12-05T17:31:13.737 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDGw8tfBKTYGRAAaPkezwmTiAvkUfTuDGDNZA== 2020-12-05T17:31:13.738 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.4] 2020-12-05T17:31:13.738 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDUw8tfN8wMJxAApc5LY9HGDeIV/o0eil/g4A== 2020-12-05T17:31:13.738 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.5] 2020-12-05T17:31:13.738 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDXw8tfvZ5dDxAAtAfftExkljlBvn7ZSoD/Og== 2020-12-05T17:31:13.738 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.6] 2020-12-05T17:31:13.739 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDZw8tfhXFzMhAAZXNwNGfKv360Tq4N/DX5Mw== 2020-12-05T17:31:13.739 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.7] 2020-12-05T17:31:13.739 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDcw8tfROfSGhAA/Y2Y1b4k5zQju0MM7LASeQ== 2020-12-05T17:31:13.739 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2020-12-05T17:31:13.740 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDGw8tfWDp/HxAAmvLPj7C3WDgj0OC2o7lBJQ== 2020-12-05T17:31:13.740 INFO:teuthology.orchestra.run.smithi032.stdout:[client.1] 2020-12-05T17:31:13.740 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDGw8tfugnmIRAAW6+/QGMLkHw+Z97RHpV4kw== 2020-12-05T17:31:13.740 INFO:teuthology.orchestra.run.smithi032.stdout:[client.2] 2020-12-05T17:31:13.740 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDGw8tfojyqJBAAyta+dSy2yGnC1kKe3YOAYw== 2020-12-05T17:31:13.742 INFO:teuthology.orchestra.run.smithi061.stdout:[mgr.y] 2020-12-05T17:31:13.742 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQDGw8tf3HmcDhAATNKHhH9zgasedaNEzgG0OQ== 2020-12-05T17:31:13.742 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.a] 2020-12-05T17:31:13.742 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQDGw8tfpb/aExAAdMKW4yIgC2zhkzyW6l8xug== 2020-12-05T17:31:13.743 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.0] 2020-12-05T17:31:13.743 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQDHw8tflo5EKhAAwV9D64fAoY3uvXCm+Aksag== 2020-12-05T17:31:13.743 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.1] 2020-12-05T17:31:13.743 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQDKw8tfTjEiERAAkW6mkgApoXwyF49kPjZBGg== 2020-12-05T17:31:13.743 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.2] 2020-12-05T17:31:13.744 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQDMw8tfLEzBMxAAiJ2NifsOLEkt1e3vmhwSTA== 2020-12-05T17:31:13.744 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.3] 2020-12-05T17:31:13.744 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQDPw8tfV+vnGxAAQHvXJRW5X9Vhe/eYJBJ/uQ== 2020-12-05T17:31:13.744 INFO:teuthology.orchestra.run.smithi061.stdout:[mgr.x] 2020-12-05T17:31:13.744 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQDGw8tfDr8mERAAS7bpAQXgW78W8GDq1v1zaA== 2020-12-05T17:31:13.745 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.b] 2020-12-05T17:31:13.745 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQDGw8tfBKTYGRAAaPkezwmTiAvkUfTuDGDNZA== 2020-12-05T17:31:13.745 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.4] 2020-12-05T17:31:13.745 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQDUw8tfN8wMJxAApc5LY9HGDeIV/o0eil/g4A== 2020-12-05T17:31:13.745 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.5] 2020-12-05T17:31:13.746 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQDXw8tfvZ5dDxAAtAfftExkljlBvn7ZSoD/Og== 2020-12-05T17:31:13.746 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.6] 2020-12-05T17:31:13.746 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQDZw8tfhXFzMhAAZXNwNGfKv360Tq4N/DX5Mw== 2020-12-05T17:31:13.746 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.7] 2020-12-05T17:31:13.747 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQDcw8tfROfSGhAA/Y2Y1b4k5zQju0MM7LASeQ== 2020-12-05T17:31:13.747 INFO:teuthology.orchestra.run.smithi061.stdout:[client.0] 2020-12-05T17:31:13.747 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQDGw8tfWDp/HxAAmvLPj7C3WDgj0OC2o7lBJQ== 2020-12-05T17:31:13.747 INFO:teuthology.orchestra.run.smithi061.stdout:[client.1] 2020-12-05T17:31:13.747 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQDGw8tfugnmIRAAW6+/QGMLkHw+Z97RHpV4kw== 2020-12-05T17:31:13.748 INFO:teuthology.orchestra.run.smithi061.stdout:[client.2] 2020-12-05T17:31:13.748 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQDGw8tfojyqJBAAyta+dSy2yGnC1kKe3YOAYw== 2020-12-05T17:31:13.748 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-12-05T17:31:13.786 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-12-05T17:31:13.826 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-12-05T17:31:13.870 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-12-05T17:31:13.916 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-12-05T17:31:13.918 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-12-05T17:31:13.999 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-12-05T17:31:14.002 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-12-05T17:31:14.083 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-12-05T17:31:14.086 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-12-05T17:31:14.167 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-12-05T17:31:14.169 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-12-05T17:31:14.251 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-12-05T17:31:14.253 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-12-05T17:31:14.335 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-12-05T17:31:14.338 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-12-05T17:31:14.428 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-12-05T17:31:14.434 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-12-05T17:31:14.521 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-12-05T17:31:14.530 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-12-05T17:31:14.580 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:31:14.586 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:31:14.605 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-12-05T17:31:14.617 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:31:14.619 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-12-05T17:31:14.635 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:31:14.642 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:31:14.691 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-12-05T17:31:14.724 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-12-05T17:31:14.801 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-12-05T17:31:14.810 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-12-05T17:31:14.893 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-12-05T17:31:14.897 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-12-05T17:31:14.982 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-12-05T17:31:14.985 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-12-05T17:31:15.070 INFO:tasks.ceph:Running mkfs on mon nodes... 2020-12-05T17:31:15.071 INFO:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2020-12-05T17:31:15.086 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-12-05T17:31:15.203 INFO:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2020-12-05T17:31:15.219 INFO:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2020-12-05T17:31:15.235 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-12-05T17:31:15.350 INFO:teuthology.orchestra.run.smithi061:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2020-12-05T17:31:15.366 INFO:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2020-12-05T17:31:15.424 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-12-05T17:31:15.540 INFO:teuthology.orchestra.run.smithi061:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2020-12-05T17:31:15.555 INFO:teuthology.orchestra.run.smithi032:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-12-05T17:31:15.558 INFO:teuthology.orchestra.run.smithi061:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-12-05T17:31:15.603 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2020-12-05T17:31:15.604 INFO:tasks.ceph.mon.a:Restarting daemon 2020-12-05T17:31:15.604 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2020-12-05T17:31:15.606 INFO:tasks.ceph.mon.a:Started 2020-12-05T17:31:15.607 INFO:tasks.ceph.mon.b:Restarting daemon 2020-12-05T17:31:15.607 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2020-12-05T17:31:15.647 INFO:tasks.ceph.mon.b:Started 2020-12-05T17:31:15.648 INFO:tasks.ceph.mon.c:Restarting daemon 2020-12-05T17:31:15.648 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2020-12-05T17:31:15.650 INFO:tasks.ceph.mon.c:Started 2020-12-05T17:31:15.651 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2020-12-05T17:31:15.651 INFO:tasks.ceph.mgr.y:Restarting daemon 2020-12-05T17:31:15.652 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2020-12-05T17:31:15.654 INFO:tasks.ceph.mgr.y:Started 2020-12-05T17:31:15.654 INFO:tasks.ceph.mgr.x:Restarting daemon 2020-12-05T17:31:15.655 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2020-12-05T17:31:15.657 INFO:tasks.ceph.mgr.x:Started 2020-12-05T17:31:15.657 INFO:tasks.ceph:Setting crush tunables to default 2020-12-05T17:31:15.657 INFO:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd crush tunables default 2020-12-05T17:31:21.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:15.883+0000 7fecb5a39700 1 Processor -- start 2020-12-05T17:31:21.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:15.883+0000 7fecb5a39700 1 -- start start 2020-12-05T17:31:21.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:15.883+0000 7fecb5a39700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fecb005f6f0 0x7fecb0061860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:21.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:15.883+0000 7fecb5a39700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fecb0064910 0x7fecb0066740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:21.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:15.883+0000 7fecb5a39700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb0063d20 con 0x7fecb0063250 2020-12-05T17:31:21.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:15.883+0000 7fecb5a39700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb00602e0 con 0x7fecb0064910 2020-12-05T17:31:21.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:15.883+0000 7fecb5a39700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb005fad0 con 0x7fecb005f6f0 2020-12-05T17:31:21.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:15.883+0000 7fecaeffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fecb0064910 0x7fecb0066740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:15.883+0000 7fecaffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fecb0063250 0x7fecb007ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34434/0 (socket says 172.21.15.32:34434) 2020-12-05T17:31:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:15.883+0000 7fecaffff700 1 -- 172.21.15.32:0/3675111440 learned_addr learned my addr 172.21.15.32:0/3675111440 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:15.883+0000 7fecaf7fe700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.61:3300/0 conn(0x7fecb005f6f0 0x7fecb0061860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:15.883+0000 7fecaeffd700 1 -- 172.21.15.32:0/3675111440 >> v2:172.21.15.32:3300/0 conn(0x7fecb0064910 msgr2=0x7fecb0066740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-12-05T17:31:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:15.883+0000 7fecaeffd700 1 -- 172.21.15.32:0/3675111440 >> v2:172.21.15.32:3300/0 conn(0x7fecb0064910 msgr2=0x7fecb0066740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-12-05T17:31:21.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:15.883+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.32:3300/0 conn(0x7fecb0064910 0x7fecb0066740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-12-05T17:31:21.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:15.883+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.32:3300/0 conn(0x7fecb0064910 0x7fecb0066740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-12-05T17:31:21.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:15.883+0000 7fecaf7fe700 1 -- 172.21.15.32:0/3675111440 >> v2:172.21.15.61:3300/0 conn(0x7fecb005f6f0 msgr2=0x7fecb0061860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2020-12-05T17:31:21.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:15.883+0000 7fecaf7fe700 1 -- 172.21.15.32:0/3675111440 >> v2:172.21.15.61:3300/0 conn(0x7fecb005f6f0 msgr2=0x7fecb0061860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-12-05T17:31:21.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:15.883+0000 7fecaf7fe700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.61:3300/0 conn(0x7fecb005f6f0 0x7fecb0061860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-12-05T17:31:21.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:15.883+0000 7fecaf7fe700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.61:3300/0 conn(0x7fecb005f6f0 0x7fecb0061860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-12-05T17:31:21.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:16.087+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.32:3300/0 conn(0x7fecb0064910 0x7fecb0066740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:21.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:16.087+0000 7fecaf7fe700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.61:3300/0 conn(0x7fecb005f6f0 0x7fecb0061860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:21.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:16.087+0000 7fecaeffd700 1 -- 172.21.15.32:0/3675111440 >> v2:172.21.15.32:3300/0 conn(0x7fecb0064910 msgr2=0x7fecb0066740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2020-12-05T17:31:21.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:16.087+0000 7fecaeffd700 1 -- 172.21.15.32:0/3675111440 >> v2:172.21.15.32:3300/0 conn(0x7fecb0064910 msgr2=0x7fecb0066740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-12-05T17:31:21.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:16.087+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.32:3300/0 conn(0x7fecb0064910 0x7fecb0066740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-12-05T17:31:21.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:16.087+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.32:3300/0 conn(0x7fecb0064910 0x7fecb0066740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-12-05T17:31:21.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:16.087+0000 7fecaf7fe700 1 -- 172.21.15.32:0/3675111440 >> v2:172.21.15.61:3300/0 conn(0x7fecb005f6f0 msgr2=0x7fecb0061860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-12-05T17:31:21.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:16.087+0000 7fecaf7fe700 1 -- 172.21.15.32:0/3675111440 >> v2:172.21.15.61:3300/0 conn(0x7fecb005f6f0 msgr2=0x7fecb0061860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-12-05T17:31:21.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:16.087+0000 7fecaf7fe700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.61:3300/0 conn(0x7fecb005f6f0 0x7fecb0061860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-12-05T17:31:21.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:16.087+0000 7fecaf7fe700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.61:3300/0 conn(0x7fecb005f6f0 0x7fecb0061860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-12-05T17:31:21.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:16.491+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.32:3300/0 conn(0x7fecb0064910 0x7fecb0066740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:16.491+0000 7fecaf7fe700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.61:3300/0 conn(0x7fecb005f6f0 0x7fecb0061860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:16.491+0000 7fecaeffd700 1 -- 172.21.15.32:0/3675111440 >> v2:172.21.15.32:3300/0 conn(0x7fecb0064910 msgr2=0x7fecb0066740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2020-12-05T17:31:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:16.491+0000 7fecaeffd700 1 -- 172.21.15.32:0/3675111440 >> v2:172.21.15.32:3300/0 conn(0x7fecb0064910 msgr2=0x7fecb0066740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-12-05T17:31:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:16.491+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.32:3300/0 conn(0x7fecb0064910 0x7fecb0066740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-12-05T17:31:21.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:16.491+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.32:3300/0 conn(0x7fecb0064910 0x7fecb0066740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-12-05T17:31:21.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:16.491+0000 7fecaf7fe700 1 -- 172.21.15.32:0/3675111440 >> v2:172.21.15.61:3300/0 conn(0x7fecb005f6f0 msgr2=0x7fecb0061860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-12-05T17:31:21.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:16.491+0000 7fecaf7fe700 1 -- 172.21.15.32:0/3675111440 >> v2:172.21.15.61:3300/0 conn(0x7fecb005f6f0 msgr2=0x7fecb0061860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-12-05T17:31:21.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:16.491+0000 7fecaf7fe700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.61:3300/0 conn(0x7fecb005f6f0 0x7fecb0061860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-12-05T17:31:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:16.491+0000 7fecaf7fe700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.61:3300/0 conn(0x7fecb005f6f0 0x7fecb0061860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-12-05T17:31:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:17.295+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.32:3300/0 conn(0x7fecb0064910 0x7fecb0066740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:17.295+0000 7fecaf7fe700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.61:3300/0 conn(0x7fecb005f6f0 0x7fecb0061860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:17.295+0000 7fecaeffd700 1 -- 172.21.15.32:0/3675111440 >> v2:172.21.15.32:3300/0 conn(0x7fecb0064910 msgr2=0x7fecb0066740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2020-12-05T17:31:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:17.295+0000 7fecaeffd700 1 -- 172.21.15.32:0/3675111440 >> v2:172.21.15.32:3300/0 conn(0x7fecb0064910 msgr2=0x7fecb0066740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-12-05T17:31:21.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:17.295+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.32:3300/0 conn(0x7fecb0064910 0x7fecb0066740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-12-05T17:31:21.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:17.295+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.32:3300/0 conn(0x7fecb0064910 0x7fecb0066740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-12-05T17:31:21.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:17.295+0000 7fecaf7fe700 1 -- 172.21.15.32:0/3675111440 >> v2:172.21.15.61:3300/0 conn(0x7fecb005f6f0 msgr2=0x7fecb0061860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-12-05T17:31:21.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:17.295+0000 7fecaf7fe700 1 -- 172.21.15.32:0/3675111440 >> v2:172.21.15.61:3300/0 conn(0x7fecb005f6f0 msgr2=0x7fecb0061860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-12-05T17:31:21.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:17.295+0000 7fecaf7fe700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.61:3300/0 conn(0x7fecb005f6f0 0x7fecb0061860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-12-05T17:31:21.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:17.295+0000 7fecaf7fe700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.61:3300/0 conn(0x7fecb005f6f0 0x7fecb0061860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-12-05T17:31:21.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:18.883+0000 7fecad7fa700 1 -- 172.21.15.32:0/3675111440 >> v2:172.21.15.61:3300/0 conn(0x7fecb005f6f0 msgr2=0x7fecb0061860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-12-05T17:31:21.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:18.883+0000 7fecad7fa700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.61:3300/0 conn(0x7fecb005f6f0 0x7fecb0061860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:21.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:18.883+0000 7fecad7fa700 1 -- 172.21.15.32:0/3675111440 >> v2:172.21.15.32:3300/0 conn(0x7fecb0064910 msgr2=0x7fecb0066740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-12-05T17:31:21.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:18.883+0000 7fecad7fa700 1 --2- 172.21.15.32:0/3675111440 >> v2:172.21.15.32:3300/0 conn(0x7fecb0064910 0x7fecb0066740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:21.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:18.883+0000 7fecad7fa700 1 -- 172.21.15.32:0/3675111440 >> v1:172.21.15.32:6789/0 conn(0x7fecb0063250 legacy=0x7fecb007ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:21.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:18.883+0000 7fecad7fa700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec90006fa0 0x7fec90009440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:21.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:18.883+0000 7fecad7fa700 1 -- 172.21.15.32:0/3675111440 --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb014f0c0 con 0x7fec90003390 2020-12-05T17:31:21.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:18.883+0000 7fecad7fa700 1 -- 172.21.15.32:0/3675111440 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb00602e0 con 0x7fec90006fa0 2020-12-05T17:31:21.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:18.883+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec90006fa0 0x7fec90009440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:21.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:18.883+0000 7fecaeffd700 1 -- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec90006fa0 msgr2=0x7fec90009440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-12-05T17:31:21.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:18.883+0000 7fecaeffd700 1 -- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec90006fa0 msgr2=0x7fec90009440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-12-05T17:31:21.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:18.883+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec90006fa0 0x7fec90009440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-12-05T17:31:21.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:18.883+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec90006fa0 0x7fec90009440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-12-05T17:31:21.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:19.087+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec90006fa0 0x7fec90009440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:21.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:19.087+0000 7fecaeffd700 1 -- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec90006fa0 msgr2=0x7fec90009440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-12-05T17:31:21.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:19.087+0000 7fecaeffd700 1 -- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec90006fa0 msgr2=0x7fec90009440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-12-05T17:31:21.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:19.087+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec90006fa0 0x7fec90009440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-12-05T17:31:21.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:19.087+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec90006fa0 0x7fec90009440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-12-05T17:31:21.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:19.491+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec90006fa0 0x7fec90009440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:21.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:19.491+0000 7fecaeffd700 1 -- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec90006fa0 msgr2=0x7fec90009440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-12-05T17:31:21.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:19.491+0000 7fecaeffd700 1 -- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec90006fa0 msgr2=0x7fec90009440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-12-05T17:31:21.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:19.491+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec90006fa0 0x7fec90009440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-12-05T17:31:21.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:19.491+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec90006fa0 0x7fec90009440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-12-05T17:31:21.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:20.295+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec90006fa0 0x7fec90009440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:21.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:20.295+0000 7fecaeffd700 1 -- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec90006fa0 msgr2=0x7fec90009440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-12-05T17:31:21.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:20.295+0000 7fecaeffd700 1 -- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec90006fa0 msgr2=0x7fec90009440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-12-05T17:31:21.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:20.295+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec90006fa0 0x7fec90009440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-12-05T17:31:21.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:20.295+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec90006fa0 0x7fec90009440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-12-05T17:31:21.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecae7fc700 1 -- 172.21.15.32:0/3675111440 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2385043831 0 0) 0x7fecb014f0c0 con 0x7fec90003390 2020-12-05T17:31:21.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecae7fc700 1 -- 172.21.15.32:0/3675111440 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec9c003230 con 0x7fec90003390 2020-12-05T17:31:21.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecae7fc700 1 -- 172.21.15.32:0/3675111440 <== mon.1 v1:172.21.15.61:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1198030473 0 0) 0x7fec9c003230 con 0x7fec90003390 2020-12-05T17:31:21.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecae7fc700 1 -- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec90006fa0 msgr2=0x7fec90009440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-12-05T17:31:21.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecae7fc700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec90006fa0 0x7fec90009440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:21.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecae7fc700 1 -- 172.21.15.32:0/3675111440 --> v1:172.21.15.61:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9000abb0 con 0x7fec90003390 2020-12-05T17:31:21.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecae7fc700 1 -- 172.21.15.32:0/3675111440 <== mon.1 v1:172.21.15.61:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7feca000d3b0 con 0x7fec90003390 2020-12-05T17:31:21.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecae7fc700 1 -- 172.21.15.32:0/3675111440 >> v1:172.21.15.61:6789/0 conn(0x7fec90003390 legacy=0x7fec900057d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:21.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecaffff700 1 -- 172.21.15.32:0/3675111440 reap_dead start 2020-12-05T17:31:21.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecae7fc700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec9c003720 0x7fec9c003b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:21.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecae7fc700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9c0044d0 0x7fec9c004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:21.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecae7fc700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec9c004e40 0x7fec9c00b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:21.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecae7fc700 1 -- 172.21.15.32:0/3675111440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb00602e0 con 0x7fec9c0044d0 2020-12-05T17:31:21.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecae7fc700 1 -- 172.21.15.32:0/3675111440 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c004230 con 0x7fec9c004e40 2020-12-05T17:31:21.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecae7fc700 1 -- 172.21.15.32:0/3675111440 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c00bc10 con 0x7fec9c003720 2020-12-05T17:31:21.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9c0044d0 0x7fec9c004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:21.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecaf7fe700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec9c003720 0x7fec9c003b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:21.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecaffff700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec9c004e40 0x7fec9c00b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:21.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecaeffd700 1 -- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec9c003720 msgr2=0x7fec9c003b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:21.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec9c003720 0x7fec9c003b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:21.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecaeffd700 1 -- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec9c004e40 msgr2=0x7fec9c00b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:21.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec9c004e40 0x7fec9c00b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:21.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecaeffd700 1 -- 172.21.15.32:0/3675111440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c003230 con 0x7fec9c0044d0 2020-12-05T17:31:21.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.003+0000 7fecaffff700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec9c004e40 0x7fec9c00b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:21.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.007+0000 7fecaeffd700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9c0044d0 0x7fec9c004950 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fec9c004440 tx=0x7fec98008040).ready entity=mon.0 client_cookie=fca193b3354d61a9 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:21.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.007+0000 7fecae7fc700 1 -- 172.21.15.32:0/3675111440 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec98059aa0 con 0x7fec9c0044d0 2020-12-05T17:31:21.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.007+0000 7fecae7fc700 1 -- 172.21.15.32:0/3675111440 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec98059c00 con 0x7fec9c0044d0 2020-12-05T17:31:21.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.007+0000 7fecae7fc700 1 -- 172.21.15.32:0/3675111440 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec98012460 con 0x7fec9c0044d0 2020-12-05T17:31:21.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.011+0000 7fecb5a39700 1 -- 172.21.15.32:0/3675111440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9c0044d0 msgr2=0x7fec9c004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:21.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.011+0000 7fecb5a39700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9c0044d0 0x7fec9c004950 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fec9c004440 tx=0x7fec98008040).stop 2020-12-05T17:31:21.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.011+0000 7fecb5a39700 1 -- 172.21.15.32:0/3675111440 shutdown_connections 2020-12-05T17:31:21.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.011+0000 7fecb5a39700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec9c004e40 0x7fec9c00b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:21.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.011+0000 7fecb5a39700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9c0044d0 0x7fec9c004950 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.011+0000 7fecb5a39700 1 --2- 172.21.15.32:0/3675111440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec9c003720 0x7fec9c003b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.011+0000 7fecb5a39700 1 -- 172.21.15.32:0/3675111440 >> 172.21.15.32:0/3675111440 conn(0x7fecb00b4b10 msgr2=0x7fecb00b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.011+0000 7fecb5a39700 1 -- 172.21.15.32:0/3675111440 shutdown_connections 2020-12-05T17:31:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.011+0000 7fecb5a39700 1 -- 172.21.15.32:0/3675111440 wait complete. 2020-12-05T17:31:21.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.015+0000 7fecb5a39700 1 Processor -- start 2020-12-05T17:31:21.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.015+0000 7fecb5a39700 1 -- start start 2020-12-05T17:31:21.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.015+0000 7fecb5a39700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec9c003720 0x7fecb013e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:21.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.015+0000 7fecb5a39700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec9c0044d0 0x7fecb0110ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:21.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.015+0000 7fecb5a39700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9c004e40 0x7fecb014de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:21.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.015+0000 7fecb5a39700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb0110710 con 0x7fec9c004e40 2020-12-05T17:31:21.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.015+0000 7fecb5a39700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb0131870 con 0x7fec9c003720 2020-12-05T17:31:21.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.015+0000 7fecb5a39700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb0131aa0 con 0x7fec9c0044d0 2020-12-05T17:31:21.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.015+0000 7fecaffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9c004e40 0x7fecb014de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:21.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.015+0000 7fecaffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9c004e40 0x7fecb014de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60092/0 (socket says 172.21.15.32:60092) 2020-12-05T17:31:21.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.015+0000 7fecaf7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec9c003720 0x7fecb013e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:21.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.015+0000 7fecaffff700 1 -- 172.21.15.32:0/2555979266 learned_addr learned my addr 172.21.15.32:0/2555979266 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:21.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.015+0000 7fecaeffd700 1 --2- 172.21.15.32:0/2555979266 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec9c0044d0 0x7fecb0110ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:21.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.019+0000 7fecaffff700 1 -- 172.21.15.32:0/2555979266 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec9c0044d0 msgr2=0x7fecb0110ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:21.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.019+0000 7fecaffff700 1 --2- 172.21.15.32:0/2555979266 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec9c0044d0 0x7fecb0110ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:21.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.019+0000 7fecaffff700 1 -- 172.21.15.32:0/2555979266 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec9c003720 msgr2=0x7fecb013e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:21.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.019+0000 7fecaffff700 1 --2- 172.21.15.32:0/2555979266 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec9c003720 0x7fecb013e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:21.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.019+0000 7fecaffff700 1 -- 172.21.15.32:0/2555979266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb0131d40 con 0x7fec9c004e40 2020-12-05T17:31:21.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.019+0000 7fecaffff700 1 --2- 172.21.15.32:0/2555979266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9c004e40 0x7fecb014de30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fecb00acbf0 tx=0x7feca8005880).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:21.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.019+0000 7feca7fff700 1 -- 172.21.15.32:0/2555979266 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feca8010710 con 0x7fec9c004e40 2020-12-05T17:31:21.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.019+0000 7feca7fff700 1 -- 172.21.15.32:0/2555979266 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca8010870 con 0x7fec9c004e40 2020-12-05T17:31:21.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.019+0000 7fecb5a39700 1 -- 172.21.15.32:0/2555979266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb0113b20 con 0x7fec9c004e40 2020-12-05T17:31:21.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.019+0000 7feca7fff700 1 -- 172.21.15.32:0/2555979266 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feca8010bd0 con 0x7fec9c004e40 2020-12-05T17:31:21.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.019+0000 7fecb5a39700 1 -- 172.21.15.32:0/2555979266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb0112410 con 0x7fec9c004e40 2020-12-05T17:31:21.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.019+0000 7feca7fff700 1 -- 172.21.15.32:0/2555979266 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7feca801e070 con 0x7fec9c004e40 2020-12-05T17:31:21.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.019+0000 7feca7fff700 1 -- 172.21.15.32:0/2555979266 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7feca8014a40 con 0x7fec9c004e40 2020-12-05T17:31:21.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.019+0000 7fecb5a39700 1 -- 172.21.15.32:0/2555979266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb0113cb0 con 0x7fec9c004e40 2020-12-05T17:31:21.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.023+0000 7feca7fff700 1 -- 172.21.15.32:0/2555979266 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7feca801c060 con 0x7fec9c004e40 2020-12-05T17:31:21.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.099+0000 7fecb5a39700 1 -- 172.21.15.32:0/2555979266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fecb0057050 con 0x7fec9c004e40 2020-12-05T17:31:21.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.103+0000 7feca7fff700 1 -- 172.21.15.32:0/2555979266 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7feca8014d40 con 0x7fec9c004e40 2020-12-05T17:31:21.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.103+0000 7fecb5a39700 1 -- 172.21.15.32:0/2555979266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9c004e40 msgr2=0x7fecb014de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:21.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.103+0000 7fecb5a39700 1 --2- 172.21.15.32:0/2555979266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9c004e40 0x7fecb014de30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fecb00acbf0 tx=0x7feca8005880).stop 2020-12-05T17:31:21.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.103+0000 7fecb5a39700 1 -- 172.21.15.32:0/2555979266 shutdown_connections 2020-12-05T17:31:21.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.103+0000 7fecb5a39700 1 --2- 172.21.15.32:0/2555979266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9c004e40 0x7fecb014de30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:21.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.103+0000 7fecb5a39700 1 --2- 172.21.15.32:0/2555979266 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec9c003720 0x7fecb013e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:21.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.103+0000 7fecb5a39700 1 --2- 172.21.15.32:0/2555979266 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec9c0044d0 0x7fecb0110ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:21.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.103+0000 7fecb5a39700 1 -- 172.21.15.32:0/2555979266 >> 172.21.15.32:0/2555979266 conn(0x7fecb00b4b10 msgr2=0x7fecb014f900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:21.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.103+0000 7fecb5a39700 1 -- 172.21.15.32:0/2555979266 shutdown_connections 2020-12-05T17:31:21.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:21.103+0000 7fecb5a39700 1 -- 172.21.15.32:0/2555979266 wait complete. 2020-12-05T17:31:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:adjusted tunables profile to default 2020-12-05T17:31:21.123 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2020-12-05T17:31:21.123 INFO:teuthology.orchestra.run.smithi032:> mktemp 2020-12-05T17:31:21.127 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.3eHRQ4mu04 2020-12-05T17:31:21.128 INFO:teuthology.orchestra.run.smithi032:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.3eHRQ4mu04 2020-12-05T17:31:21.184 INFO:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.3eHRQ4mu04 2020-12-05T17:31:21.288 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.3eHRQ4mu04 is 37B 2020-12-05T17:31:21.298 INFO:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.3eHRQ4mu04 2020-12-05T17:31:21.304 INFO:teuthology.orchestra.run.smithi032:> mktemp 2020-12-05T17:31:21.350 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.bddmxqB3Gq 2020-12-05T17:31:21.351 INFO:teuthology.orchestra.run.smithi032:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.bddmxqB3Gq 2020-12-05T17:31:21.406 INFO:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.bddmxqB3Gq 2020-12-05T17:31:21.513 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.bddmxqB3Gq is 37B 2020-12-05T17:31:21.562 INFO:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.bddmxqB3Gq 2020-12-05T17:31:21.568 INFO:teuthology.orchestra.run.smithi032:> mktemp 2020-12-05T17:31:21.614 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.2qsb1LheOL 2020-12-05T17:31:21.615 INFO:teuthology.orchestra.run.smithi032:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.2qsb1LheOL 2020-12-05T17:31:21.672 INFO:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.2qsb1LheOL 2020-12-05T17:31:21.776 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.2qsb1LheOL is 37B 2020-12-05T17:31:21.787 INFO:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.2qsb1LheOL 2020-12-05T17:31:21.793 INFO:teuthology.orchestra.run.smithi032:> mktemp 2020-12-05T17:31:21.838 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.LrF0NH52RE 2020-12-05T17:31:21.839 INFO:teuthology.orchestra.run.smithi032:> sudo cp /var/lib/ceph/osd/ceph-3/fsid /tmp/tmp.LrF0NH52RE 2020-12-05T17:31:21.894 INFO:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.LrF0NH52RE 2020-12-05T17:31:22.000 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.LrF0NH52RE is 37B 2020-12-05T17:31:22.010 INFO:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.LrF0NH52RE 2020-12-05T17:31:22.015 INFO:teuthology.orchestra.run.smithi061:> mktemp 2020-12-05T17:31:22.022 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/tmp.TOSvh9E4aD 2020-12-05T17:31:22.022 INFO:teuthology.orchestra.run.smithi061:> sudo cp /var/lib/ceph/osd/ceph-4/fsid /tmp/tmp.TOSvh9E4aD 2020-12-05T17:31:22.081 INFO:teuthology.orchestra.run.smithi061:> sudo chmod 0666 /tmp/tmp.TOSvh9E4aD 2020-12-05T17:31:22.185 DEBUG:teuthology.orchestra.remote:smithi061:/tmp/tmp.TOSvh9E4aD is 37B 2020-12-05T17:31:22.195 INFO:teuthology.orchestra.run.smithi061:> rm -fr /tmp/tmp.TOSvh9E4aD 2020-12-05T17:31:22.200 INFO:teuthology.orchestra.run.smithi061:> mktemp 2020-12-05T17:31:22.247 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/tmp.42t1UHfGDN 2020-12-05T17:31:22.248 INFO:teuthology.orchestra.run.smithi061:> sudo cp /var/lib/ceph/osd/ceph-5/fsid /tmp/tmp.42t1UHfGDN 2020-12-05T17:31:22.305 INFO:teuthology.orchestra.run.smithi061:> sudo chmod 0666 /tmp/tmp.42t1UHfGDN 2020-12-05T17:31:22.409 DEBUG:teuthology.orchestra.remote:smithi061:/tmp/tmp.42t1UHfGDN is 37B 2020-12-05T17:31:22.458 INFO:teuthology.orchestra.run.smithi061:> rm -fr /tmp/tmp.42t1UHfGDN 2020-12-05T17:31:22.464 INFO:teuthology.orchestra.run.smithi061:> mktemp 2020-12-05T17:31:22.511 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/tmp.SIbxuVcSyr 2020-12-05T17:31:22.511 INFO:teuthology.orchestra.run.smithi061:> sudo cp /var/lib/ceph/osd/ceph-6/fsid /tmp/tmp.SIbxuVcSyr 2020-12-05T17:31:22.568 INFO:teuthology.orchestra.run.smithi061:> sudo chmod 0666 /tmp/tmp.SIbxuVcSyr 2020-12-05T17:31:22.673 DEBUG:teuthology.orchestra.remote:smithi061:/tmp/tmp.SIbxuVcSyr is 37B 2020-12-05T17:31:22.682 INFO:teuthology.orchestra.run.smithi061:> rm -fr /tmp/tmp.SIbxuVcSyr 2020-12-05T17:31:22.688 INFO:teuthology.orchestra.run.smithi061:> mktemp 2020-12-05T17:31:22.735 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/tmp.MLAszHMcx1 2020-12-05T17:31:22.735 INFO:teuthology.orchestra.run.smithi061:> sudo cp /var/lib/ceph/osd/ceph-7/fsid /tmp/tmp.MLAszHMcx1 2020-12-05T17:31:22.792 INFO:teuthology.orchestra.run.smithi061:> sudo chmod 0666 /tmp/tmp.MLAszHMcx1 2020-12-05T17:31:22.898 DEBUG:teuthology.orchestra.remote:smithi061:/tmp/tmp.MLAszHMcx1 is 37B 2020-12-05T17:31:22.948 INFO:teuthology.orchestra.run.smithi061:> rm -fr /tmp/tmp.MLAszHMcx1 2020-12-05T17:31:22.954 INFO:teuthology.orchestra.run.smithi061:> sudo ceph --cluster ceph osd new b941127b-0a49-4e06-8d11-17794ff8b013 0 2020-12-05T17:31:23.161 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.153+0000 7f7264f3c700 1 Processor -- start 2020-12-05T17:31:23.161 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.153+0000 7f7264f3c700 1 -- start start 2020-12-05T17:31:23.161 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.153+0000 7f7264f3c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72600b1450 0x7f72601530d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:23.162 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.153+0000 7f7264f3c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72600b3670 0x7f72600af020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:23.162 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.153+0000 7f7264f3c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72600ae360 con 0x7f72600af510 2020-12-05T17:31:23.162 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.153+0000 7f7264f3c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7260152990 con 0x7f72600b3670 2020-12-05T17:31:23.162 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.153+0000 7f7264f3c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7260152c30 con 0x7f72600b1450 2020-12-05T17:31:23.162 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.153+0000 7f725ffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72600b1450 0x7f72601530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:23.163 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.153+0000 7f725ffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72600b1450 0x7f72601530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.61:54176/0 (socket says 172.21.15.61:54176) 2020-12-05T17:31:23.163 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.157+0000 7f725ffff700 1 -- 172.21.15.61:0/1463414432 learned_addr learned my addr 172.21.15.61:0/1463414432 (peer_addr_for_me v2:172.21.15.61:0/0) 2020-12-05T17:31:23.163 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.157+0000 7f725f7fe700 1 --2- 172.21.15.61:0/1463414432 >> v2:172.21.15.32:3300/0 conn(0x7f72600b3670 0x7f72600af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:23.163 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.157+0000 7f725ffff700 1 -- 172.21.15.61:0/1463414432 >> v2:172.21.15.32:3300/0 conn(0x7f72600b3670 msgr2=0x7f72600af020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:23.163 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.157+0000 7f725ffff700 1 --2- 172.21.15.61:0/1463414432 >> v2:172.21.15.32:3300/0 conn(0x7f72600b3670 0x7f72600af020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.164 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.157+0000 7f725ffff700 1 -- 172.21.15.61:0/1463414432 >> v1:172.21.15.32:6789/0 conn(0x7f72600af510 legacy=0x7f726007ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:23.164 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.157+0000 7f725ffff700 1 -- 172.21.15.61:0/1463414432 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72600ae050 con 0x7f72600b1450 2020-12-05T17:31:23.164 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.157+0000 7f725ffff700 1 --2- 172.21.15.61:0/1463414432 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72600b1450 0x7f72601530d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f7250002f30 tx=0x7f7250006e80).ready entity=mon.2 client_cookie=589d3dc714121204 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:23.164 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.157+0000 7f725effd700 1 -- 172.21.15.61:0/1463414432 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72500097f0 con 0x7f72600b1450 2020-12-05T17:31:23.164 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.157+0000 7f725effd700 1 -- 172.21.15.61:0/1463414432 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7250009950 con 0x7f72600b1450 2020-12-05T17:31:23.165 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.157+0000 7f725effd700 1 -- 172.21.15.61:0/1463414432 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7250009cb0 con 0x7f72600b1450 2020-12-05T17:31:23.165 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.157+0000 7f7264f3c700 1 -- 172.21.15.61:0/1463414432 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72600b1450 msgr2=0x7f72601530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:23.165 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.157+0000 7f7264f3c700 1 --2- 172.21.15.61:0/1463414432 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72600b1450 0x7f72601530d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f7250002f30 tx=0x7f7250006e80).stop 2020-12-05T17:31:23.165 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.157+0000 7f7264f3c700 1 -- 172.21.15.61:0/1463414432 shutdown_connections 2020-12-05T17:31:23.166 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.157+0000 7f7264f3c700 1 --2- 172.21.15.61:0/1463414432 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72600b1450 0x7f72601530d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.166 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.157+0000 7f7264f3c700 1 --2- 172.21.15.61:0/1463414432 >> v2:172.21.15.32:3300/0 conn(0x7f72600b3670 0x7f72600af020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.166 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.157+0000 7f7264f3c700 1 -- 172.21.15.61:0/1463414432 >> 172.21.15.61:0/1463414432 conn(0x7f7260061010 msgr2=0x7f7260061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:23.166 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.157+0000 7f7264f3c700 1 -- 172.21.15.61:0/1463414432 shutdown_connections 2020-12-05T17:31:23.166 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.157+0000 7f7264f3c700 1 -- 172.21.15.61:0/1463414432 wait complete. 2020-12-05T17:31:23.169 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.157+0000 7f7264f3c700 1 Processor -- start 2020-12-05T17:31:23.169 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.157+0000 7f7264f3c700 1 -- start start 2020-12-05T17:31:23.169 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f7264f3c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72600af510 0x7f72600258d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:23.169 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f7264f3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72600b1450 0x7f7260139530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:23.170 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f7264f3c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72600b3670 0x7f7260139a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:23.170 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f7264f3c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7260158120 con 0x7f72600b3670 2020-12-05T17:31:23.170 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f7264f3c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726007fb60 con 0x7f72600b1450 2020-12-05T17:31:23.170 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f7264f3c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7260158400 con 0x7f72600af510 2020-12-05T17:31:23.170 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f725ffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72600af510 0x7f72600258d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:23.171 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f725ffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72600af510 0x7f72600258d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.61:54184/0 (socket says 172.21.15.61:54184) 2020-12-05T17:31:23.171 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f725ffff700 1 -- 172.21.15.61:0/1375712861 learned_addr learned my addr 172.21.15.61:0/1375712861 (peer_addr_for_me v2:172.21.15.61:0/0) 2020-12-05T17:31:23.171 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f725f7fe700 1 --2- 172.21.15.61:0/1375712861 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72600b1450 0x7f7260139530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:23.171 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f726619e700 1 --2- 172.21.15.61:0/1375712861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72600b3670 0x7f7260139a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:23.171 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f725f7fe700 1 -- 172.21.15.61:0/1375712861 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72600af510 msgr2=0x7f72600258d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:23.172 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f725f7fe700 1 --2- 172.21.15.61:0/1375712861 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72600af510 0x7f72600258d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.172 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f725f7fe700 1 -- 172.21.15.61:0/1375712861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72600b3670 msgr2=0x7f7260139a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:23.172 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f725f7fe700 1 --2- 172.21.15.61:0/1375712861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72600b3670 0x7f7260139a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.172 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f725f7fe700 1 -- 172.21.15.61:0/1375712861 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72601586a0 con 0x7f72600b1450 2020-12-05T17:31:23.173 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f725ffff700 1 --2- 172.21.15.61:0/1375712861 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72600af510 0x7f72600258d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:31:23.173 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f725f7fe700 1 --2- 172.21.15.61:0/1375712861 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72600b1450 0x7f7260139530 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f724800c000 tx=0x7f7248008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:23.173 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f725cff9700 1 -- 172.21.15.61:0/1375712861 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f724800c610 con 0x7f72600b1450 2020-12-05T17:31:23.173 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f725cff9700 1 -- 172.21.15.61:0/1375712861 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724800c770 con 0x7f72600b1450 2020-12-05T17:31:23.173 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f7264f3c700 1 -- 172.21.15.61:0/1375712861 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f726007f850 con 0x7f72600b1450 2020-12-05T17:31:23.174 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f725cff9700 1 -- 172.21.15.61:0/1375712861 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f724800cad0 con 0x7f72600b1450 2020-12-05T17:31:23.174 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f7264f3c700 1 -- 172.21.15.61:0/1375712861 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72600645c0 con 0x7f72600b1450 2020-12-05T17:31:23.174 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f725cff9700 1 -- 172.21.15.61:0/1375712861 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f7248022070 con 0x7f72600b1450 2020-12-05T17:31:23.174 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f725cff9700 1 -- 172.21.15.61:0/1375712861 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7f724801b9d0 con 0x7f72600b1450 2020-12-05T17:31:23.174 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.161+0000 7f7264f3c700 1 -- 172.21.15.61:0/1375712861 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7260158830 con 0x7f72600b1450 2020-12-05T17:31:23.175 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.165+0000 7f725cff9700 1 -- 172.21.15.61:0/1375712861 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f7248039870 con 0x7f72600b1450 2020-12-05T17:31:23.258 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.257+0000 7f7264f3c700 1 -- 172.21.15.61:0/1375712861 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b941127b-0a49-4e06-8d11-17794ff8b013", "id": 0} v 0) v1 -- 0x7f7260057050 con 0x7f72600b1450 2020-12-05T17:31:23.265 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.261+0000 7f725cff9700 1 -- 172.21.15.61:0/1375712861 <== mon.1 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b941127b-0a49-4e06-8d11-17794ff8b013", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7248029030 con 0x7f72600b1450 2020-12-05T17:31:23.265 INFO:teuthology.orchestra.run.smithi061.stdout:0 2020-12-05T17:31:23.267 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.265+0000 7f7264f3c700 1 -- 172.21.15.61:0/1375712861 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72600b1450 msgr2=0x7f7260139530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:23.267 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.265+0000 7f7264f3c700 1 --2- 172.21.15.61:0/1375712861 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72600b1450 0x7f7260139530 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f724800c000 tx=0x7f7248008040).stop 2020-12-05T17:31:23.267 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.265+0000 7f7264f3c700 1 -- 172.21.15.61:0/1375712861 shutdown_connections 2020-12-05T17:31:23.267 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.265+0000 7f7264f3c700 1 --2- 172.21.15.61:0/1375712861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72600b3670 0x7f7260139a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.268 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.265+0000 7f7264f3c700 1 --2- 172.21.15.61:0/1375712861 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72600af510 0x7f72600258d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.268 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.265+0000 7f7264f3c700 1 --2- 172.21.15.61:0/1375712861 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72600b1450 0x7f7260139530 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.268 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.265+0000 7f7264f3c700 1 -- 172.21.15.61:0/1375712861 >> 172.21.15.61:0/1375712861 conn(0x7f7260061010 msgr2=0x7f72600b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:23.268 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.265+0000 7f7264f3c700 1 -- 172.21.15.61:0/1375712861 shutdown_connections 2020-12-05T17:31:23.268 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.265+0000 7f7264f3c700 1 -- 172.21.15.61:0/1375712861 wait complete. 2020-12-05T17:31:23.279 INFO:teuthology.orchestra.run.smithi061:> sudo ceph --cluster ceph osd new 01bd7ff5-a86b-4e11-ba1e-484c309c0fba 1 2020-12-05T17:31:23.394 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.385+0000 7f8fc12c9700 1 Processor -- start 2020-12-05T17:31:23.395 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.385+0000 7f8fc12c9700 1 -- start start 2020-12-05T17:31:23.395 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fc12c9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8fbc0b1450 0x7f8fbc1530d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:23.395 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fc12c9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8fbc0b3670 0x7f8fbc0af020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:23.396 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fc12c9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fbc0ae360 con 0x7f8fbc0af510 2020-12-05T17:31:23.396 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fc12c9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fbc152990 con 0x7f8fbc0b1450 2020-12-05T17:31:23.396 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fc12c9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc152c30 con 0x7f8fbc0b3670 2020-12-05T17:31:23.397 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fbbfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8fbc0b3670 0x7f8fbc0af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:23.397 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fbbfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8fbc0b3670 0x7f8fbc0af020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.61:54188/0 (socket says 172.21.15.61:54188) 2020-12-05T17:31:23.398 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fbbfff700 1 -- 172.21.15.61:0/1634908048 learned_addr learned my addr 172.21.15.61:0/1634908048 (peer_addr_for_me v2:172.21.15.61:0/0) 2020-12-05T17:31:23.398 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fc0ac8700 1 --2- 172.21.15.61:0/1634908048 >> v2:172.21.15.32:3300/0 conn(0x7f8fbc0b1450 0x7f8fbc1530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:23.398 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fc0ac8700 1 -- 172.21.15.61:0/1634908048 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8fbc0b3670 msgr2=0x7f8fbc0af020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:23.398 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fc0ac8700 1 --2- 172.21.15.61:0/1634908048 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8fbc0b3670 0x7f8fbc0af020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.399 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fc0ac8700 1 -- 172.21.15.61:0/1634908048 >> v1:172.21.15.32:6789/0 conn(0x7f8fbc0af510 legacy=0x7f8fbc07ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:23.399 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fc0ac8700 1 -- 172.21.15.61:0/1634908048 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fbc0ae050 con 0x7f8fbc0b1450 2020-12-05T17:31:23.399 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fc0ac8700 1 --2- 172.21.15.61:0/1634908048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fbc0b1450 0x7f8fbc1530d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f8fb0002f30 tx=0x7f8fb0006dc0).ready entity=mon.0 client_cookie=5578bf9f7ef760e0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:23.399 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fbb7fe700 1 -- 172.21.15.61:0/1634908048 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fb0009760 con 0x7f8fbc0b1450 2020-12-05T17:31:23.399 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fbb7fe700 1 -- 172.21.15.61:0/1634908048 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb00098c0 con 0x7f8fbc0b1450 2020-12-05T17:31:23.400 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fbb7fe700 1 -- 172.21.15.61:0/1634908048 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fb0009c20 con 0x7f8fbc0b1450 2020-12-05T17:31:23.400 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fc12c9700 1 -- 172.21.15.61:0/1634908048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fbc0b1450 msgr2=0x7f8fbc1530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:23.400 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fc12c9700 1 --2- 172.21.15.61:0/1634908048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fbc0b1450 0x7f8fbc1530d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f8fb0002f30 tx=0x7f8fb0006dc0).stop 2020-12-05T17:31:23.400 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fc12c9700 1 -- 172.21.15.61:0/1634908048 shutdown_connections 2020-12-05T17:31:23.401 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fc12c9700 1 --2- 172.21.15.61:0/1634908048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fbc0b1450 0x7f8fbc1530d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.401 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fc12c9700 1 --2- 172.21.15.61:0/1634908048 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8fbc0b3670 0x7f8fbc0af020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.401 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fc12c9700 1 -- 172.21.15.61:0/1634908048 >> 172.21.15.61:0/1634908048 conn(0x7f8fbc061010 msgr2=0x7f8fbc061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:23.401 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fc12c9700 1 -- 172.21.15.61:0/1634908048 shutdown_connections 2020-12-05T17:31:23.401 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.389+0000 7f8fc12c9700 1 -- 172.21.15.61:0/1634908048 wait complete. 2020-12-05T17:31:23.404 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fc12c9700 1 Processor -- start 2020-12-05T17:31:23.404 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fc12c9700 1 -- start start 2020-12-05T17:31:23.404 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fc12c9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8fbc0af510 0x7f8fbc071140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:23.404 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fc12c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fbc0b1450 0x7f8fbc151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:23.405 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fc12c9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8fbc0b3670 0x7f8fbc11b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:23.405 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fc12c9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc064dd0 con 0x7f8fbc0b1450 2020-12-05T17:31:23.405 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fc12c9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc0657b0 con 0x7f8fbc0b3670 2020-12-05T17:31:23.405 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fc12c9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc064370 con 0x7f8fbc0af510 2020-12-05T17:31:23.405 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fc0ac8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8fbc0af510 0x7f8fbc071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:23.406 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fc0ac8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8fbc0af510 0x7f8fbc071140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.61:54194/0 (socket says 172.21.15.61:54194) 2020-12-05T17:31:23.406 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fc0ac8700 1 -- 172.21.15.61:0/2220955549 learned_addr learned my addr 172.21.15.61:0/2220955549 (peer_addr_for_me v2:172.21.15.61:0/0) 2020-12-05T17:31:23.406 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fc252b700 1 --2- 172.21.15.61:0/2220955549 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8fbc0b3670 0x7f8fbc11b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:23.406 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fbbfff700 1 --2- 172.21.15.61:0/2220955549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fbc0b1450 0x7f8fbc151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:23.407 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fc0ac8700 1 -- 172.21.15.61:0/2220955549 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8fbc0b3670 msgr2=0x7f8fbc11b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:23.407 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fc0ac8700 1 --2- 172.21.15.61:0/2220955549 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8fbc0b3670 0x7f8fbc11b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.407 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fc0ac8700 1 -- 172.21.15.61:0/2220955549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fbc0b1450 msgr2=0x7f8fbc151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:23.407 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fc0ac8700 1 --2- 172.21.15.61:0/2220955549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fbc0b1450 0x7f8fbc151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.407 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fc0ac8700 1 -- 172.21.15.61:0/2220955549 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fbc064610 con 0x7f8fbc0af510 2020-12-05T17:31:23.408 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fc0ac8700 1 --2- 172.21.15.61:0/2220955549 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8fbc0af510 0x7f8fbc071140 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f8fb0002f30 tx=0x7f8fb0015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:23.408 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fb97fa700 1 -- 172.21.15.61:0/2220955549 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fb00119e0 con 0x7f8fbc0af510 2020-12-05T17:31:23.408 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fb97fa700 1 -- 172.21.15.61:0/2220955549 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb0011b40 con 0x7f8fbc0af510 2020-12-05T17:31:23.408 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fc12c9700 1 -- 172.21.15.61:0/2220955549 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fbc0648b0 con 0x7f8fbc0af510 2020-12-05T17:31:23.409 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fb97fa700 1 -- 172.21.15.61:0/2220955549 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fb0011ea0 con 0x7f8fbc0af510 2020-12-05T17:31:23.409 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fc12c9700 1 -- 172.21.15.61:0/2220955549 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fbc065400 con 0x7f8fbc0af510 2020-12-05T17:31:23.409 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fb97fa700 1 -- 172.21.15.61:0/2220955549 <== mon.2 v2:172.21.15.61:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f8fb0024030 con 0x7f8fbc0af510 2020-12-05T17:31:23.409 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.393+0000 7f8fb97fa700 1 -- 172.21.15.61:0/2220955549 <== mon.2 v2:172.21.15.61:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7f8fb0027b40 con 0x7f8fbc0af510 2020-12-05T17:31:23.409 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.397+0000 7f8fc12c9700 1 -- 172.21.15.61:0/2220955549 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fbc057050 con 0x7f8fbc0af510 2020-12-05T17:31:23.410 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.401+0000 7f8fb97fa700 1 -- 172.21.15.61:0/2220955549 <== mon.2 v2:172.21.15.61:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f8fb0022070 con 0x7f8fbc0af510 2020-12-05T17:31:23.481 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.477+0000 7f8fc12c9700 1 -- 172.21.15.61:0/2220955549 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "osd new", "uuid": "01bd7ff5-a86b-4e11-ba1e-484c309c0fba", "id": 1} v 0) v1 -- 0x7f8fbc0ad2d0 con 0x7f8fbc0af510 2020-12-05T17:31:23.487 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.485+0000 7f8fb97fa700 1 -- 172.21.15.61:0/2220955549 <== mon.2 v2:172.21.15.61:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "01bd7ff5-a86b-4e11-ba1e-484c309c0fba", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8fb0006880 con 0x7f8fbc0af510 2020-12-05T17:31:23.487 INFO:teuthology.orchestra.run.smithi061.stdout:1 2020-12-05T17:31:23.489 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.485+0000 7f8fc12c9700 1 -- 172.21.15.61:0/2220955549 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8fbc0af510 msgr2=0x7f8fbc071140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:23.489 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.485+0000 7f8fc12c9700 1 --2- 172.21.15.61:0/2220955549 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8fbc0af510 0x7f8fbc071140 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f8fb0002f30 tx=0x7f8fb0015000).stop 2020-12-05T17:31:23.489 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.485+0000 7f8fc12c9700 1 -- 172.21.15.61:0/2220955549 shutdown_connections 2020-12-05T17:31:23.489 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.485+0000 7f8fc12c9700 1 --2- 172.21.15.61:0/2220955549 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8fbc0b3670 0x7f8fbc11b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.490 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.485+0000 7f8fc12c9700 1 --2- 172.21.15.61:0/2220955549 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8fbc0af510 0x7f8fbc071140 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.490 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.485+0000 7f8fc12c9700 1 --2- 172.21.15.61:0/2220955549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fbc0b1450 0x7f8fbc151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.490 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.485+0000 7f8fc12c9700 1 -- 172.21.15.61:0/2220955549 >> 172.21.15.61:0/2220955549 conn(0x7f8fbc061010 msgr2=0x7f8fbc0b53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:23.490 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.485+0000 7f8fc12c9700 1 -- 172.21.15.61:0/2220955549 shutdown_connections 2020-12-05T17:31:23.491 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.485+0000 7f8fc12c9700 1 -- 172.21.15.61:0/2220955549 wait complete. 2020-12-05T17:31:23.499 INFO:teuthology.orchestra.run.smithi061:> sudo ceph --cluster ceph osd new 94736ef5-64c2-4a97-b963-bd38cddd9c52 2 2020-12-05T17:31:23.614 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.605+0000 7fd6d54c8700 1 Processor -- start 2020-12-05T17:31:23.615 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.605+0000 7fd6d54c8700 1 -- start start 2020-12-05T17:31:23.615 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6d54c8700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd6d00b1450 0x7fd6d01530d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:23.615 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6d54c8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6d00b3670 0x7fd6d00af020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:23.615 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6d54c8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d00ae360 con 0x7fd6d00af510 2020-12-05T17:31:23.616 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6d54c8700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d0152990 con 0x7fd6d00b1450 2020-12-05T17:31:23.616 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6d54c8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d0152c30 con 0x7fd6d00b3670 2020-12-05T17:31:23.616 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6cffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6d00b3670 0x7fd6d00af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:23.616 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6d4cc7700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd6d00b1450 0x7fd6d01530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:23.617 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6cffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6d00b3670 0x7fd6d00af020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.61:54202/0 (socket says 172.21.15.61:54202) 2020-12-05T17:31:23.617 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6d4cc7700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd6d00b1450 0x7fd6d01530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:41878/0 (socket says 172.21.15.61:41878) 2020-12-05T17:31:23.617 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6cffff700 1 -- 172.21.15.61:0/126893277 learned_addr learned my addr 172.21.15.61:0/126893277 (peer_addr_for_me v2:172.21.15.61:0/0) 2020-12-05T17:31:23.617 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6cffff700 1 -- 172.21.15.61:0/126893277 >> v2:172.21.15.61:3300/0 conn(0x7fd6d00b1450 msgr2=0x7fd6d01530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:23.617 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6cffff700 1 --2- 172.21.15.61:0/126893277 >> v2:172.21.15.61:3300/0 conn(0x7fd6d00b1450 0x7fd6d01530d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.618 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6cffff700 1 -- 172.21.15.61:0/126893277 >> v1:172.21.15.32:6789/0 conn(0x7fd6d00af510 legacy=0x7fd6d007ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:23.618 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6cffff700 1 -- 172.21.15.61:0/126893277 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d00ae050 con 0x7fd6d00b3670 2020-12-05T17:31:23.618 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6cffff700 1 --2- 172.21.15.61:0/126893277 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6d00b3670 0x7fd6d00af020 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fd6c0002f30 tx=0x7fd6c0006e80).ready entity=mon.2 client_cookie=f020d36335ac8c10 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:23.618 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6cf7fe700 1 -- 172.21.15.61:0/126893277 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6c00097f0 con 0x7fd6d00b3670 2020-12-05T17:31:23.619 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6cf7fe700 1 -- 172.21.15.61:0/126893277 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c0009950 con 0x7fd6d00b3670 2020-12-05T17:31:23.619 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6cf7fe700 1 -- 172.21.15.61:0/126893277 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6c0009cb0 con 0x7fd6d00b3670 2020-12-05T17:31:23.619 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6d54c8700 1 -- 172.21.15.61:0/126893277 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6d00b3670 msgr2=0x7fd6d00af020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:23.619 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6d54c8700 1 --2- 172.21.15.61:0/126893277 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6d00b3670 0x7fd6d00af020 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fd6c0002f30 tx=0x7fd6c0006e80).stop 2020-12-05T17:31:23.620 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6d54c8700 1 -- 172.21.15.61:0/126893277 shutdown_connections 2020-12-05T17:31:23.620 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6d54c8700 1 --2- 172.21.15.61:0/126893277 >> v2:172.21.15.61:3300/0 conn(0x7fd6d00b1450 0x7fd6d01530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.620 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6d54c8700 1 --2- 172.21.15.61:0/126893277 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6d00b3670 0x7fd6d00af020 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.620 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6d54c8700 1 -- 172.21.15.61:0/126893277 >> 172.21.15.61:0/126893277 conn(0x7fd6d0061010 msgr2=0x7fd6d0061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:23.621 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6d54c8700 1 -- 172.21.15.61:0/126893277 shutdown_connections 2020-12-05T17:31:23.621 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.609+0000 7fd6d54c8700 1 -- 172.21.15.61:0/126893277 wait complete. 2020-12-05T17:31:23.623 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.613+0000 7fd6d54c8700 1 Processor -- start 2020-12-05T17:31:23.624 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.613+0000 7fd6d54c8700 1 -- start start 2020-12-05T17:31:23.624 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.613+0000 7fd6d54c8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6d00af510 0x7fd6d00252c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:23.624 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.613+0000 7fd6d54c8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6d00b1450 0x7fd6d0137300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:23.624 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.613+0000 7fd6d54c8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6d00b3670 0x7fd6d014e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:23.625 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.613+0000 7fd6d54c8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d01580b0 con 0x7fd6d00b1450 2020-12-05T17:31:23.625 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.613+0000 7fd6d54c8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d0156a90 con 0x7fd6d00af510 2020-12-05T17:31:23.625 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.613+0000 7fd6d54c8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d01377f0 con 0x7fd6d00b3670 2020-12-05T17:31:23.625 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.613+0000 7fd6d4cc7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6d00af510 0x7fd6d00252c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:23.625 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.613+0000 7fd6d4cc7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6d00af510 0x7fd6d00252c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:41884/0 (socket says 172.21.15.61:41884) 2020-12-05T17:31:23.625 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.613+0000 7fd6d4cc7700 1 -- 172.21.15.61:0/4119596623 learned_addr learned my addr 172.21.15.61:0/4119596623 (peer_addr_for_me v2:172.21.15.61:0/0) 2020-12-05T17:31:23.626 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.613+0000 7fd6d672a700 1 --2- 172.21.15.61:0/4119596623 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6d00b3670 0x7fd6d014e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:23.626 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.613+0000 7fd6cffff700 1 --2- 172.21.15.61:0/4119596623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6d00b1450 0x7fd6d0137300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:23.626 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.613+0000 7fd6d672a700 1 -- 172.21.15.61:0/4119596623 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6d00af510 msgr2=0x7fd6d00252c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:23.626 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.613+0000 7fd6d672a700 1 --2- 172.21.15.61:0/4119596623 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6d00af510 0x7fd6d00252c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.627 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.613+0000 7fd6d672a700 1 -- 172.21.15.61:0/4119596623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6d00b1450 msgr2=0x7fd6d0137300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:23.627 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.613+0000 7fd6d672a700 1 --2- 172.21.15.61:0/4119596623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6d00b1450 0x7fd6d0137300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.627 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.613+0000 7fd6d672a700 1 -- 172.21.15.61:0/4119596623 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d0155310 con 0x7fd6d00b3670 2020-12-05T17:31:23.627 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.613+0000 7fd6d672a700 1 --2- 172.21.15.61:0/4119596623 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6d00b3670 0x7fd6d014e9e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fd6d0062f20 tx=0x7fd6c8007e80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:23.628 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.613+0000 7fd6cd7fa700 1 -- 172.21.15.61:0/4119596623 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6c800a5a0 con 0x7fd6d00b3670 2020-12-05T17:31:23.628 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.617+0000 7fd6cd7fa700 1 -- 172.21.15.61:0/4119596623 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c800a700 con 0x7fd6d00b3670 2020-12-05T17:31:23.628 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.617+0000 7fd6d54c8700 1 -- 172.21.15.61:0/4119596623 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d01555b0 con 0x7fd6d00b3670 2020-12-05T17:31:23.628 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.617+0000 7fd6cd7fa700 1 -- 172.21.15.61:0/4119596623 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6c800aa60 con 0x7fd6d00b3670 2020-12-05T17:31:23.628 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.617+0000 7fd6d54c8700 1 -- 172.21.15.61:0/4119596623 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d0158cf0 con 0x7fd6d00b3670 2020-12-05T17:31:23.629 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.617+0000 7fd6cd7fa700 1 -- 172.21.15.61:0/4119596623 <== mon.2 v2:172.21.15.61:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fd6c8019030 con 0x7fd6d00b3670 2020-12-05T17:31:23.629 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.617+0000 7fd6cd7fa700 1 -- 172.21.15.61:0/4119596623 <== mon.2 v2:172.21.15.61:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7fd6c801b800 con 0x7fd6d00b3670 2020-12-05T17:31:23.629 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.617+0000 7fd6d54c8700 1 -- 172.21.15.61:0/4119596623 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d01568b0 con 0x7fd6d00b3670 2020-12-05T17:31:23.629 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.621+0000 7fd6cd7fa700 1 -- 172.21.15.61:0/4119596623 <== mon.2 v2:172.21.15.61:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fd6c8017070 con 0x7fd6d00b3670 2020-12-05T17:31:23.701 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.697+0000 7fd6d54c8700 1 -- 172.21.15.61:0/4119596623 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "osd new", "uuid": "94736ef5-64c2-4a97-b963-bd38cddd9c52", "id": 2} v 0) v1 -- 0x7fd6d0155740 con 0x7fd6d00b3670 2020-12-05T17:31:23.708 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.705+0000 7fd6cd7fa700 1 -- 172.21.15.61:0/4119596623 <== mon.2 v2:172.21.15.61:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "94736ef5-64c2-4a97-b963-bd38cddd9c52", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd6c801b3b0 con 0x7fd6d00b3670 2020-12-05T17:31:23.708 INFO:teuthology.orchestra.run.smithi061.stdout:2 2020-12-05T17:31:23.710 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.705+0000 7fd6d54c8700 1 -- 172.21.15.61:0/4119596623 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6d00b3670 msgr2=0x7fd6d014e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:23.710 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.705+0000 7fd6d54c8700 1 --2- 172.21.15.61:0/4119596623 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6d00b3670 0x7fd6d014e9e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fd6d0062f20 tx=0x7fd6c8007e80).stop 2020-12-05T17:31:23.711 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.705+0000 7fd6d54c8700 1 -- 172.21.15.61:0/4119596623 shutdown_connections 2020-12-05T17:31:23.711 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.705+0000 7fd6d54c8700 1 --2- 172.21.15.61:0/4119596623 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6d00b3670 0x7fd6d014e9e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.711 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.705+0000 7fd6d54c8700 1 --2- 172.21.15.61:0/4119596623 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6d00af510 0x7fd6d00252c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.712 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.705+0000 7fd6d54c8700 1 --2- 172.21.15.61:0/4119596623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6d00b1450 0x7fd6d0137300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.712 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.705+0000 7fd6d54c8700 1 -- 172.21.15.61:0/4119596623 >> 172.21.15.61:0/4119596623 conn(0x7fd6d0061010 msgr2=0x7fd6d00b5430 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:23.713 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.705+0000 7fd6d54c8700 1 -- 172.21.15.61:0/4119596623 shutdown_connections 2020-12-05T17:31:23.713 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.705+0000 7fd6d54c8700 1 -- 172.21.15.61:0/4119596623 wait complete. 2020-12-05T17:31:23.720 INFO:teuthology.orchestra.run.smithi061:> sudo ceph --cluster ceph osd new 69a1f3ff-fd25-43df-b60c-e32a89c35234 3 2020-12-05T17:31:23.832 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.825+0000 7f6236d6f700 1 Processor -- start 2020-12-05T17:31:23.833 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.825+0000 7f6236d6f700 1 -- start start 2020-12-05T17:31:23.833 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.825+0000 7f6236d6f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6230064910 0x7f6230066740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:23.833 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.825+0000 7f6236d6f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f6230063250 0x7f623007ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:23.833 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.825+0000 7f6236d6f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6230063d20 con 0x7f623005f6f0 2020-12-05T17:31:23.834 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.825+0000 7f6236d6f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6230061f70 con 0x7f6230063250 2020-12-05T17:31:23.834 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.825+0000 7f6236d6f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62300602e0 con 0x7f6230064910 2020-12-05T17:31:23.834 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.825+0000 7f6235b0d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f6230063250 0x7f623007ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:23.834 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.825+0000 7f6234b0b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6230064910 0x7f6230066740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:23.834 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.825+0000 7f6235b0d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f6230063250 0x7f623007ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:41894/0 (socket says 172.21.15.61:41894) 2020-12-05T17:31:23.835 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.825+0000 7f6235b0d700 1 -- 172.21.15.61:0/2046575118 learned_addr learned my addr 172.21.15.61:0/2046575118 (peer_addr_for_me v2:172.21.15.61:0/0) 2020-12-05T17:31:23.835 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.825+0000 7f623530c700 1 --1- 172.21.15.61:0/2046575118 >> v1:172.21.15.61:6789/0 conn(0x7f623005f6f0 0x7f6230061860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.61:33100/0 (socket says 172.21.15.61:33100) 2020-12-05T17:31:23.835 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.829+0000 7f6235b0d700 1 -- 172.21.15.61:0/2046575118 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6230064910 msgr2=0x7f6230066740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:23.835 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.829+0000 7f6235b0d700 1 --2- 172.21.15.61:0/2046575118 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6230064910 0x7f6230066740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.835 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.829+0000 7f6235b0d700 1 -- 172.21.15.61:0/2046575118 >> v1:172.21.15.61:6789/0 conn(0x7f623005f6f0 legacy=0x7f6230061860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:23.836 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.829+0000 7f6235b0d700 1 -- 172.21.15.61:0/2046575118 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623005fad0 con 0x7f6230063250 2020-12-05T17:31:23.836 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.829+0000 7f6235b0d700 1 --2- 172.21.15.61:0/2046575118 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6230063250 0x7f623007ff60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f62300639a0 tx=0x7f6228009270).ready entity=mon.1 client_cookie=9754279ece9c37bb server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:23.836 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.829+0000 7f6227fff700 1 -- 172.21.15.61:0/2046575118 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6228009e90 con 0x7f6230063250 2020-12-05T17:31:23.836 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.829+0000 7f6227fff700 1 -- 172.21.15.61:0/2046575118 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622800bbc0 con 0x7f6230063250 2020-12-05T17:31:23.836 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.829+0000 7f6227fff700 1 -- 172.21.15.61:0/2046575118 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f622800bd30 con 0x7f6230063250 2020-12-05T17:31:23.837 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.829+0000 7f6236d6f700 1 -- 172.21.15.61:0/2046575118 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6230063250 msgr2=0x7f623007ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:23.837 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.829+0000 7f6236d6f700 1 --2- 172.21.15.61:0/2046575118 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6230063250 0x7f623007ff60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f62300639a0 tx=0x7f6228009270).stop 2020-12-05T17:31:23.837 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.829+0000 7f6236d6f700 1 -- 172.21.15.61:0/2046575118 shutdown_connections 2020-12-05T17:31:23.837 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.829+0000 7f6236d6f700 1 --2- 172.21.15.61:0/2046575118 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6230063250 0x7f623007ff60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.838 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.829+0000 7f6236d6f700 1 --2- 172.21.15.61:0/2046575118 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6230064910 0x7f6230066740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.838 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.829+0000 7f6236d6f700 1 -- 172.21.15.61:0/2046575118 >> 172.21.15.61:0/2046575118 conn(0x7f62300b4b10 msgr2=0x7f62300b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:23.838 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.829+0000 7f6236d6f700 1 -- 172.21.15.61:0/2046575118 shutdown_connections 2020-12-05T17:31:23.838 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.829+0000 7f6236d6f700 1 -- 172.21.15.61:0/2046575118 wait complete. 2020-12-05T17:31:23.840 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.829+0000 7f6236d6f700 1 Processor -- start 2020-12-05T17:31:23.841 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.829+0000 7f6236d6f700 1 -- start start 2020-12-05T17:31:23.841 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f6236d6f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f623005f6f0 0x7f62301506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:23.841 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f6236d6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6230063250 0x7f62300b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:23.841 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f6236d6f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6230064910 0x7f6230125860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:23.842 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f6236d6f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6230152510 con 0x7f6230063250 2020-12-05T17:31:23.842 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f6236d6f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6230150cd0 con 0x7f623005f6f0 2020-12-05T17:31:23.842 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f6236d6f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6230150f70 con 0x7f6230064910 2020-12-05T17:31:23.842 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f623530c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f623005f6f0 0x7f62301506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:23.842 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f623530c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f623005f6f0 0x7f62301506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:41896/0 (socket says 172.21.15.61:41896) 2020-12-05T17:31:23.843 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f623530c700 1 -- 172.21.15.61:0/211845653 learned_addr learned my addr 172.21.15.61:0/211845653 (peer_addr_for_me v2:172.21.15.61:0/0) 2020-12-05T17:31:23.843 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f6235b0d700 1 --2- 172.21.15.61:0/211845653 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6230064910 0x7f6230125860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:23.843 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f6234b0b700 1 --2- 172.21.15.61:0/211845653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6230063250 0x7f62300b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:23.843 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f623530c700 1 -- 172.21.15.61:0/211845653 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6230064910 msgr2=0x7f6230125860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:23.843 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f623530c700 1 --2- 172.21.15.61:0/211845653 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6230064910 0x7f6230125860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.844 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f623530c700 1 -- 172.21.15.61:0/211845653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6230063250 msgr2=0x7f62300b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:23.844 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f623530c700 1 --2- 172.21.15.61:0/211845653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6230063250 0x7f62300b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.844 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f623530c700 1 -- 172.21.15.61:0/211845653 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6230125d50 con 0x7f623005f6f0 2020-12-05T17:31:23.844 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f623530c700 1 --2- 172.21.15.61:0/211845653 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f623005f6f0 0x7f62301506a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f6230151240 tx=0x7f622c007c10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:23.844 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f6225ffb700 1 -- 172.21.15.61:0/211845653 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f622c0072e0 con 0x7f623005f6f0 2020-12-05T17:31:23.845 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f6225ffb700 1 -- 172.21.15.61:0/211845653 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622c005520 con 0x7f623005f6f0 2020-12-05T17:31:23.845 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f6225ffb700 1 -- 172.21.15.61:0/211845653 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f622c005880 con 0x7f623005f6f0 2020-12-05T17:31:23.845 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f6236d6f700 1 -- 172.21.15.61:0/211845653 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6230152940 con 0x7f623005f6f0 2020-12-05T17:31:23.845 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f6236d6f700 1 -- 172.21.15.61:0/211845653 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6230152e90 con 0x7f623005f6f0 2020-12-05T17:31:23.845 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f6225ffb700 1 -- 172.21.15.61:0/211845653 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f622c010070 con 0x7f623005f6f0 2020-12-05T17:31:23.846 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f6225ffb700 1 -- 172.21.15.61:0/211845653 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7f622c005bb0 con 0x7f623005f6f0 2020-12-05T17:31:23.846 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.833+0000 7f6236d6f700 1 -- 172.21.15.61:0/211845653 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6230057050 con 0x7f623005f6f0 2020-12-05T17:31:23.846 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.837+0000 7f6225ffb700 1 -- 172.21.15.61:0/211845653 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f622c018020 con 0x7f623005f6f0 2020-12-05T17:31:23.918 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.917+0000 7f6236d6f700 1 -- 172.21.15.61:0/211845653 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd new", "uuid": "69a1f3ff-fd25-43df-b60c-e32a89c35234", "id": 3} v 0) v1 -- 0x7f623005aab0 con 0x7f623005f6f0 2020-12-05T17:31:23.924 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.921+0000 7f6225ffb700 1 -- 172.21.15.61:0/211845653 <== mon.1 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "69a1f3ff-fd25-43df-b60c-e32a89c35234", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f622c013450 con 0x7f623005f6f0 2020-12-05T17:31:23.925 INFO:teuthology.orchestra.run.smithi061.stdout:3 2020-12-05T17:31:23.927 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.921+0000 7f6236d6f700 1 -- 172.21.15.61:0/211845653 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f623005f6f0 msgr2=0x7f62301506a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:23.928 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.921+0000 7f6236d6f700 1 --2- 172.21.15.61:0/211845653 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f623005f6f0 0x7f62301506a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f6230151240 tx=0x7f622c007c10).stop 2020-12-05T17:31:23.928 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.921+0000 7f6236d6f700 1 -- 172.21.15.61:0/211845653 shutdown_connections 2020-12-05T17:31:23.929 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.921+0000 7f6236d6f700 1 --2- 172.21.15.61:0/211845653 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6230064910 0x7f6230125860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.929 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.921+0000 7f6236d6f700 1 --2- 172.21.15.61:0/211845653 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f623005f6f0 0x7f62301506a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.929 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.921+0000 7f6236d6f700 1 --2- 172.21.15.61:0/211845653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6230063250 0x7f62300b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:23.930 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.921+0000 7f6236d6f700 1 -- 172.21.15.61:0/211845653 >> 172.21.15.61:0/211845653 conn(0x7f62300b4b10 msgr2=0x7f62300b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:23.930 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.921+0000 7f6236d6f700 1 -- 172.21.15.61:0/211845653 shutdown_connections 2020-12-05T17:31:23.931 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:23.921+0000 7f6236d6f700 1 -- 172.21.15.61:0/211845653 wait complete. 2020-12-05T17:31:23.936 INFO:teuthology.orchestra.run.smithi061:> sudo ceph --cluster ceph osd new c301fe4e-5eab-402c-ae6f-db3e6a7181af 4 2020-12-05T17:31:24.048 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.041+0000 7f9adccad700 1 Processor -- start 2020-12-05T17:31:24.048 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.041+0000 7f9adccad700 1 -- start start 2020-12-05T17:31:24.048 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.041+0000 7f9adccad700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ad805f6f0 0x7f9ad8061860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:24.048 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.041+0000 7f9adccad700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad8063d20 con 0x7f9ad8063250 2020-12-05T17:31:24.049 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.041+0000 7f9adccad700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad8066e50 con 0x7f9ad8064910 2020-12-05T17:31:24.049 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.041+0000 7f9adccad700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad80602e0 con 0x7f9ad805f6f0 2020-12-05T17:31:24.049 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.041+0000 7f9ad6d9d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ad805f6f0 0x7f9ad8061860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:24.049 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.041+0000 7f9ad659c700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f9ad8064910 0x7f9ad8066740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.61:33112/0 (socket says 172.21.15.61:33112) 2020-12-05T17:31:24.050 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.041+0000 7f9ad659c700 1 -- 172.21.15.61:0/1244172391 learned_addr learned my addr 172.21.15.61:0/1244172391 (peer_addr_for_me v1:172.21.15.61:0/0) 2020-12-05T17:31:24.050 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.041+0000 7f9ad5d9b700 1 -- 172.21.15.61:0/1244172391 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 377819212 0 0) 0x7f9ad8066e50 con 0x7f9ad8064910 2020-12-05T17:31:24.050 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.041+0000 7f9ad5d9b700 1 -- 172.21.15.61:0/1244172391 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ac4003230 con 0x7f9ad8064910 2020-12-05T17:31:24.050 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.041+0000 7f9ad6d9d700 1 -- 172.21.15.61:0/1244172391 >> v1:172.21.15.61:6789/0 conn(0x7f9ad8064910 legacy=0x7f9ad8066740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:24.050 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.041+0000 7f9ad5d9b700 1 -- 172.21.15.61:0/1244172391 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 850766559 0 0) 0x7f9ad8063d20 con 0x7f9ad8063250 2020-12-05T17:31:24.051 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.041+0000 7f9ad6d9d700 1 -- 172.21.15.61:0/1244172391 >> v1:172.21.15.32:6789/0 conn(0x7f9ad8063250 legacy=0x7f9ad807ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:24.051 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.041+0000 7f9ad6d9d700 1 -- 172.21.15.61:0/1244172391 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad805fad0 con 0x7f9ad805f6f0 2020-12-05T17:31:24.051 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.041+0000 7f9ad6d9d700 1 --2- 172.21.15.61:0/1244172391 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ad805f6f0 0x7f9ad8061860 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f9ad80639a0 tx=0x7f9ac0006dc0).ready entity=mon.2 client_cookie=1410563428d5dd43 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:24.051 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9ad5d9b700 1 -- 172.21.15.61:0/1244172391 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ac00097b0 con 0x7f9ad805f6f0 2020-12-05T17:31:24.052 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9ad5d9b700 1 -- 172.21.15.61:0/1244172391 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac0009910 con 0x7f9ad805f6f0 2020-12-05T17:31:24.052 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9ad5d9b700 1 -- 172.21.15.61:0/1244172391 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ac0009c70 con 0x7f9ad805f6f0 2020-12-05T17:31:24.052 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9adccad700 1 -- 172.21.15.61:0/1244172391 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ad805f6f0 msgr2=0x7f9ad8061860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:24.052 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9adccad700 1 --2- 172.21.15.61:0/1244172391 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ad805f6f0 0x7f9ad8061860 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f9ad80639a0 tx=0x7f9ac0006dc0).stop 2020-12-05T17:31:24.053 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9adccad700 1 -- 172.21.15.61:0/1244172391 shutdown_connections 2020-12-05T17:31:24.053 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9adccad700 1 --2- 172.21.15.61:0/1244172391 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ad805f6f0 0x7f9ad8061860 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.053 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9adccad700 1 -- 172.21.15.61:0/1244172391 >> 172.21.15.61:0/1244172391 conn(0x7f9ad80b4b10 msgr2=0x7f9ad80b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:24.053 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9adccad700 1 -- 172.21.15.61:0/1244172391 shutdown_connections 2020-12-05T17:31:24.053 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9adccad700 1 -- 172.21.15.61:0/1244172391 wait complete. 2020-12-05T17:31:24.056 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9adccad700 1 Processor -- start 2020-12-05T17:31:24.056 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9adccad700 1 -- start start 2020-12-05T17:31:24.056 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9adccad700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ad805f6f0 0x7f9ad8125510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:24.056 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9adccad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad8063250 0x7f9ad8138650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:24.057 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9adccad700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ad8064910 0x7f9ad8134770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:24.057 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9adccad700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad8137f10 con 0x7f9ad8063250 2020-12-05T17:31:24.057 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9adccad700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad81381b0 con 0x7f9ad8064910 2020-12-05T17:31:24.057 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9adccad700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad80ae370 con 0x7f9ad805f6f0 2020-12-05T17:31:24.057 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9ad6d9d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ad805f6f0 0x7f9ad8125510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:24.058 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9ad759e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ad8064910 0x7f9ad8134770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:24.058 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9ad759e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ad8064910 0x7f9ad8134770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:41912/0 (socket says 172.21.15.61:41912) 2020-12-05T17:31:24.058 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9ad759e700 1 -- 172.21.15.61:0/2316023764 learned_addr learned my addr 172.21.15.61:0/2316023764 (peer_addr_for_me v2:172.21.15.61:0/0) 2020-12-05T17:31:24.058 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9ad6d9d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ad805f6f0 0x7f9ad8125510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.61:54232/0 (socket says 172.21.15.61:54232) 2020-12-05T17:31:24.058 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9ad659c700 1 --2- 172.21.15.61:0/2316023764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad8063250 0x7f9ad8138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:24.059 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9ad759e700 1 -- 172.21.15.61:0/2316023764 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ad805f6f0 msgr2=0x7f9ad8125510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:24.059 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9ad759e700 1 --2- 172.21.15.61:0/2316023764 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ad805f6f0 0x7f9ad8125510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.059 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9ad759e700 1 -- 172.21.15.61:0/2316023764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad8063250 msgr2=0x7f9ad8138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:24.059 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9ad759e700 1 --2- 172.21.15.61:0/2316023764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad8063250 0x7f9ad8138650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.060 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9ad759e700 1 -- 172.21.15.61:0/2316023764 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad8125a00 con 0x7f9ad8064910 2020-12-05T17:31:24.060 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9ad6d9d700 1 --2- 172.21.15.61:0/2316023764 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ad805f6f0 0x7f9ad8125510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:31:24.060 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.045+0000 7f9ad759e700 1 --2- 172.21.15.61:0/2316023764 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ad8064910 0x7f9ad8134770 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f9ad0000f00 tx=0x7f9ad0007da0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:24.060 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.049+0000 7f9acf7fe700 1 -- 172.21.15.61:0/2316023764 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ad000a7b0 con 0x7f9ad8064910 2020-12-05T17:31:24.060 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.049+0000 7f9acf7fe700 1 -- 172.21.15.61:0/2316023764 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad000a910 con 0x7f9ad8064910 2020-12-05T17:31:24.061 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.049+0000 7f9adccad700 1 -- 172.21.15.61:0/2316023764 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad8138b40 con 0x7f9ad8064910 2020-12-05T17:31:24.061 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.049+0000 7f9acf7fe700 1 -- 172.21.15.61:0/2316023764 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ad000ac70 con 0x7f9ad8064910 2020-12-05T17:31:24.061 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.049+0000 7f9adccad700 1 -- 172.21.15.61:0/2316023764 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad8152560 con 0x7f9ad8064910 2020-12-05T17:31:24.061 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.049+0000 7f9acf7fe700 1 -- 172.21.15.61:0/2316023764 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f9ad0019030 con 0x7f9ad8064910 2020-12-05T17:31:24.061 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.049+0000 7f9acf7fe700 1 -- 172.21.15.61:0/2316023764 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7f9ad0012930 con 0x7f9ad8064910 2020-12-05T17:31:24.062 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.049+0000 7f9adccad700 1 -- 172.21.15.61:0/2316023764 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad8134fa0 con 0x7f9ad8064910 2020-12-05T17:31:24.062 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.053+0000 7f9acf7fe700 1 -- 172.21.15.61:0/2316023764 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f9ad8134fa0 con 0x7f9ad8064910 2020-12-05T17:31:24.135 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.133+0000 7f9adccad700 1 -- 172.21.15.61:0/2316023764 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c301fe4e-5eab-402c-ae6f-db3e6a7181af", "id": 4} v 0) v1 -- 0x7f9ad81526f0 con 0x7f9ad8064910 2020-12-05T17:31:24.142 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.141+0000 7f9acf7fe700 1 -- 172.21.15.61:0/2316023764 <== mon.1 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c301fe4e-5eab-402c-ae6f-db3e6a7181af", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9ad0017070 con 0x7f9ad8064910 2020-12-05T17:31:24.143 INFO:teuthology.orchestra.run.smithi061.stdout:4 2020-12-05T17:31:24.144 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.141+0000 7f9adccad700 1 -- 172.21.15.61:0/2316023764 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ad8064910 msgr2=0x7f9ad8134770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:24.144 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.141+0000 7f9adccad700 1 --2- 172.21.15.61:0/2316023764 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ad8064910 0x7f9ad8134770 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f9ad0000f00 tx=0x7f9ad0007da0).stop 2020-12-05T17:31:24.144 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.141+0000 7f9adccad700 1 -- 172.21.15.61:0/2316023764 shutdown_connections 2020-12-05T17:31:24.145 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.141+0000 7f9adccad700 1 --2- 172.21.15.61:0/2316023764 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ad8064910 0x7f9ad8134770 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.145 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.141+0000 7f9adccad700 1 --2- 172.21.15.61:0/2316023764 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ad805f6f0 0x7f9ad8125510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.145 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.141+0000 7f9adccad700 1 --2- 172.21.15.61:0/2316023764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad8063250 0x7f9ad8138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.145 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.141+0000 7f9adccad700 1 -- 172.21.15.61:0/2316023764 >> 172.21.15.61:0/2316023764 conn(0x7f9ad80b4b10 msgr2=0x7f9ad80b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:24.146 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.141+0000 7f9adccad700 1 -- 172.21.15.61:0/2316023764 shutdown_connections 2020-12-05T17:31:24.146 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.141+0000 7f9adccad700 1 -- 172.21.15.61:0/2316023764 wait complete. 2020-12-05T17:31:24.155 INFO:teuthology.orchestra.run.smithi061:> sudo ceph --cluster ceph osd new 9476eff4-345b-46c5-a473-1d9a89a6d327 5 2020-12-05T17:31:24.263 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.257+0000 7fa4eab08700 1 Processor -- start 2020-12-05T17:31:24.264 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.257+0000 7fa4eab08700 1 -- start start 2020-12-05T17:31:24.264 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.257+0000 7fa4eab08700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa4e40b1450 0x7fa4e41530d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:24.264 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.257+0000 7fa4eab08700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa4e40b0960 0x7fa4e407ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:24.264 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.257+0000 7fa4eab08700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4e40af730 con 0x7fa4e40b3670 2020-12-05T17:31:24.265 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.257+0000 7fa4eab08700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4e40ae360 con 0x7fa4e40b1450 2020-12-05T17:31:24.265 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.257+0000 7fa4eab08700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e4152990 con 0x7fa4e40b0960 2020-12-05T17:31:24.265 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.257+0000 7fa4ea307700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa4e40b1450 0x7fa4e41530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:24.265 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.257+0000 7fa4ea307700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa4e40b1450 0x7fa4e41530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:41914/0 (socket says 172.21.15.61:41914) 2020-12-05T17:31:24.266 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.257+0000 7fa4ea307700 1 -- 172.21.15.61:0/1782154847 learned_addr learned my addr 172.21.15.61:0/1782154847 (peer_addr_for_me v2:172.21.15.61:0/0) 2020-12-05T17:31:24.266 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.257+0000 7fa4e9b06700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fa4e40b3670 0x7fa4e40af020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.61:33124/0 (socket says 172.21.15.61:33124) 2020-12-05T17:31:24.266 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.257+0000 7fa4ebd6a700 1 --2- 172.21.15.61:0/1782154847 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa4e40b0960 0x7fa4e407ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:24.266 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.257+0000 7fa4ea307700 1 -- 172.21.15.61:0/1782154847 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa4e40b0960 msgr2=0x7fa4e407ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:24.266 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.257+0000 7fa4ea307700 1 --2- 172.21.15.61:0/1782154847 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa4e40b0960 0x7fa4e407ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.267 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.257+0000 7fa4ea307700 1 -- 172.21.15.61:0/1782154847 >> v1:172.21.15.61:6789/0 conn(0x7fa4e40b3670 legacy=0x7fa4e40af020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:24.267 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.257+0000 7fa4ea307700 1 -- 172.21.15.61:0/1782154847 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e4152c30 con 0x7fa4e40b1450 2020-12-05T17:31:24.267 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.257+0000 7fa4ea307700 1 --2- 172.21.15.61:0/1782154847 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4e40b1450 0x7fa4e41530d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fa4d40032b0 tx=0x7fa4d4009270).ready entity=mon.1 client_cookie=d7502bafc38b1e39 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:24.267 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.257+0000 7fa4e9305700 1 -- 172.21.15.61:0/1782154847 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4d4009e60 con 0x7fa4e40b1450 2020-12-05T17:31:24.267 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.257+0000 7fa4e9305700 1 -- 172.21.15.61:0/1782154847 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d400bc00 con 0x7fa4e40b1450 2020-12-05T17:31:24.268 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.257+0000 7fa4e9305700 1 -- 172.21.15.61:0/1782154847 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4d400bd70 con 0x7fa4e40b1450 2020-12-05T17:31:24.268 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.257+0000 7fa4eab08700 1 -- 172.21.15.61:0/1782154847 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4e40b1450 msgr2=0x7fa4e41530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:24.268 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.257+0000 7fa4eab08700 1 --2- 172.21.15.61:0/1782154847 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4e40b1450 0x7fa4e41530d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fa4d40032b0 tx=0x7fa4d4009270).stop 2020-12-05T17:31:24.268 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4eab08700 1 -- 172.21.15.61:0/1782154847 shutdown_connections 2020-12-05T17:31:24.269 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4eab08700 1 --2- 172.21.15.61:0/1782154847 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa4e40b0960 0x7fa4e407ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.269 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4eab08700 1 --2- 172.21.15.61:0/1782154847 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4e40b1450 0x7fa4e41530d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.269 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4eab08700 1 -- 172.21.15.61:0/1782154847 >> 172.21.15.61:0/1782154847 conn(0x7fa4e4061010 msgr2=0x7fa4e4061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:24.269 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4eab08700 1 -- 172.21.15.61:0/1782154847 shutdown_connections 2020-12-05T17:31:24.269 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4eab08700 1 -- 172.21.15.61:0/1782154847 wait complete. 2020-12-05T17:31:24.272 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4eab08700 1 Processor -- start 2020-12-05T17:31:24.272 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4eab08700 1 -- start start 2020-12-05T17:31:24.272 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4eab08700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4e40b0960 0x7fa4e4074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:24.272 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4eab08700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e40b1450 0x7fa4e4064460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:24.273 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4eab08700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa4e40b3670 0x7fa4e4025120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:24.273 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4eab08700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e4065e60 con 0x7fa4e40b1450 2020-12-05T17:31:24.273 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4eab08700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e40631c0 con 0x7fa4e40b0960 2020-12-05T17:31:24.273 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4eab08700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e4063460 con 0x7fa4e40b3670 2020-12-05T17:31:24.273 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4ebd6a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa4e40b3670 0x7fa4e4025120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:24.274 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4ea307700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4e40b0960 0x7fa4e4074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:24.274 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4ebd6a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa4e40b3670 0x7fa4e4025120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.61:54244/0 (socket says 172.21.15.61:54244) 2020-12-05T17:31:24.274 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4ebd6a700 1 -- 172.21.15.61:0/2366654079 learned_addr learned my addr 172.21.15.61:0/2366654079 (peer_addr_for_me v2:172.21.15.61:0/0) 2020-12-05T17:31:24.274 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4e9b06700 1 --2- 172.21.15.61:0/2366654079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e40b1450 0x7fa4e4064460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:24.275 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4ea307700 1 -- 172.21.15.61:0/2366654079 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa4e40b3670 msgr2=0x7fa4e4025120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:24.275 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4ea307700 1 --2- 172.21.15.61:0/2366654079 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa4e40b3670 0x7fa4e4025120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.275 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4ea307700 1 -- 172.21.15.61:0/2366654079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e40b1450 msgr2=0x7fa4e4064460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:24.275 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4ea307700 1 --2- 172.21.15.61:0/2366654079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e40b1450 0x7fa4e4064460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.275 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4ea307700 1 -- 172.21.15.61:0/2366654079 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e411f2a0 con 0x7fa4e40b0960 2020-12-05T17:31:24.276 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4ea307700 1 --2- 172.21.15.61:0/2366654079 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4e40b0960 0x7fa4e4074120 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fa4d40032b0 tx=0x7fa4d4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:24.276 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4daffd700 1 -- 172.21.15.61:0/2366654079 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4d40107a0 con 0x7fa4e40b0960 2020-12-05T17:31:24.276 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4daffd700 1 -- 172.21.15.61:0/2366654079 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d4010900 con 0x7fa4e40b0960 2020-12-05T17:31:24.276 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4eab08700 1 -- 172.21.15.61:0/2366654079 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e411f740 con 0x7fa4e40b0960 2020-12-05T17:31:24.276 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4daffd700 1 -- 172.21.15.61:0/2366654079 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4d4010c60 con 0x7fa4e40b0960 2020-12-05T17:31:24.277 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.261+0000 7fa4eab08700 1 -- 172.21.15.61:0/2366654079 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e4063d20 con 0x7fa4e40b0960 2020-12-05T17:31:24.277 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.265+0000 7fa4daffd700 1 -- 172.21.15.61:0/2366654079 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fa4d4024020 con 0x7fa4e40b0960 2020-12-05T17:31:24.277 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.265+0000 7fa4daffd700 1 -- 172.21.15.61:0/2366654079 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1329+0+0 (secure 0 0 0) 0x7fa4d40279e0 con 0x7fa4e40b0960 2020-12-05T17:31:24.277 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.265+0000 7fa4eab08700 1 -- 172.21.15.61:0/2366654079 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e4063020 con 0x7fa4e40b0960 2020-12-05T17:31:24.277 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.269+0000 7fa4daffd700 1 -- 172.21.15.61:0/2366654079 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fa4e4063020 con 0x7fa4e40b0960 2020-12-05T17:31:24.349 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.345+0000 7fa4eab08700 1 -- 172.21.15.61:0/2366654079 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9476eff4-345b-46c5-a473-1d9a89a6d327", "id": 5} v 0) v1 -- 0x7fa4e4057050 con 0x7fa4e40b0960 2020-12-05T17:31:24.355 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.353+0000 7fa4daffd700 1 -- 172.21.15.61:0/2366654079 <== mon.1 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9476eff4-345b-46c5-a473-1d9a89a6d327", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa4d402c0b0 con 0x7fa4e40b0960 2020-12-05T17:31:24.356 INFO:teuthology.orchestra.run.smithi061.stdout:5 2020-12-05T17:31:24.357 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.353+0000 7fa4eab08700 1 -- 172.21.15.61:0/2366654079 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4e40b0960 msgr2=0x7fa4e4074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:24.357 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.353+0000 7fa4eab08700 1 --2- 172.21.15.61:0/2366654079 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4e40b0960 0x7fa4e4074120 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fa4d40032b0 tx=0x7fa4d4016000).stop 2020-12-05T17:31:24.358 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.353+0000 7fa4eab08700 1 -- 172.21.15.61:0/2366654079 shutdown_connections 2020-12-05T17:31:24.358 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.353+0000 7fa4eab08700 1 --2- 172.21.15.61:0/2366654079 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa4e40b3670 0x7fa4e4025120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.358 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.353+0000 7fa4eab08700 1 --2- 172.21.15.61:0/2366654079 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4e40b0960 0x7fa4e4074120 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.358 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.353+0000 7fa4eab08700 1 --2- 172.21.15.61:0/2366654079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e40b1450 0x7fa4e4064460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.359 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.353+0000 7fa4eab08700 1 -- 172.21.15.61:0/2366654079 >> 172.21.15.61:0/2366654079 conn(0x7fa4e4061010 msgr2=0x7fa4e40b5430 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:24.359 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.353+0000 7fa4eab08700 1 -- 172.21.15.61:0/2366654079 shutdown_connections 2020-12-05T17:31:24.359 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.353+0000 7fa4eab08700 1 -- 172.21.15.61:0/2366654079 wait complete. 2020-12-05T17:31:24.369 INFO:teuthology.orchestra.run.smithi061:> sudo ceph --cluster ceph osd new 8dfd8567-9b1c-4d13-9c96-071183e469c2 6 2020-12-05T17:31:24.486 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d9010700 1 Processor -- start 2020-12-05T17:31:24.486 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d9010700 1 -- start start 2020-12-05T17:31:24.487 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d9010700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5d405f6f0 0x7fe5d4061860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:24.487 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d9010700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5d4063250 0x7fe5d407ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:24.487 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d9010700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d4063d20 con 0x7fe5d4064910 2020-12-05T17:31:24.487 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d9010700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d4066e50 con 0x7fe5d4063250 2020-12-05T17:31:24.488 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d9010700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d40602e0 con 0x7fe5d405f6f0 2020-12-05T17:31:24.488 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d359e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5d405f6f0 0x7fe5d4061860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:24.488 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d359e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5d405f6f0 0x7fe5d4061860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.61:54252/0 (socket says 172.21.15.61:54252) 2020-12-05T17:31:24.488 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5cbfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe5d4064910 0x7fe5d4066740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.61:50494/0 (socket says 172.21.15.61:50494) 2020-12-05T17:31:24.488 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d359e700 1 -- 172.21.15.61:0/545256093 learned_addr learned my addr 172.21.15.61:0/545256093 (peer_addr_for_me v2:172.21.15.61:0/0) 2020-12-05T17:31:24.489 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d880f700 1 --2- 172.21.15.61:0/545256093 >> v2:172.21.15.32:3300/0 conn(0x7fe5d4063250 0x7fe5d407ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:24.489 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d359e700 1 -- 172.21.15.61:0/545256093 >> v2:172.21.15.32:3300/0 conn(0x7fe5d4063250 msgr2=0x7fe5d407ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:24.489 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d359e700 1 --2- 172.21.15.61:0/545256093 >> v2:172.21.15.32:3300/0 conn(0x7fe5d4063250 0x7fe5d407ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.489 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d359e700 1 -- 172.21.15.61:0/545256093 >> v1:172.21.15.32:6789/0 conn(0x7fe5d4064910 legacy=0x7fe5d4066740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:24.490 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d359e700 1 -- 172.21.15.61:0/545256093 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d405fad0 con 0x7fe5d405f6f0 2020-12-05T17:31:24.490 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d880f700 1 --2- 172.21.15.61:0/545256093 >> v2:172.21.15.32:3300/0 conn(0x7fe5d4063250 0x7fe5d407ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:31:24.490 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d359e700 1 --2- 172.21.15.61:0/545256093 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5d405f6f0 0x7fe5d4061860 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fe5d40639a0 tx=0x7fe5bc006e80).ready entity=mon.2 client_cookie=7dc7b31c0c632501 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:24.490 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d2d9d700 1 -- 172.21.15.61:0/545256093 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5bc0097b0 con 0x7fe5d405f6f0 2020-12-05T17:31:24.490 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d2d9d700 1 -- 172.21.15.61:0/545256093 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc009910 con 0x7fe5d405f6f0 2020-12-05T17:31:24.491 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d2d9d700 1 -- 172.21.15.61:0/545256093 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5bc009c70 con 0x7fe5d405f6f0 2020-12-05T17:31:24.491 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d9010700 1 -- 172.21.15.61:0/545256093 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5d405f6f0 msgr2=0x7fe5d4061860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:24.491 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d9010700 1 --2- 172.21.15.61:0/545256093 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5d405f6f0 0x7fe5d4061860 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fe5d40639a0 tx=0x7fe5bc006e80).stop 2020-12-05T17:31:24.491 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d9010700 1 -- 172.21.15.61:0/545256093 shutdown_connections 2020-12-05T17:31:24.492 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d9010700 1 --2- 172.21.15.61:0/545256093 >> v2:172.21.15.32:3300/0 conn(0x7fe5d4063250 0x7fe5d407ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.492 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d9010700 1 --2- 172.21.15.61:0/545256093 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5d405f6f0 0x7fe5d4061860 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.492 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d9010700 1 -- 172.21.15.61:0/545256093 >> 172.21.15.61:0/545256093 conn(0x7fe5d40b4b10 msgr2=0x7fe5d40b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:24.493 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d9010700 1 -- 172.21.15.61:0/545256093 shutdown_connections 2020-12-05T17:31:24.493 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.481+0000 7fe5d9010700 1 -- 172.21.15.61:0/545256093 wait complete. 2020-12-05T17:31:24.499 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5d9010700 1 Processor -- start 2020-12-05T17:31:24.499 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5d9010700 1 -- start start 2020-12-05T17:31:24.499 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5d9010700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d405f6f0 0x7fe5d40af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:24.500 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5d9010700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5d4063250 0x7fe5d406fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:24.500 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5d9010700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe5d4064910 0x7fe5d407f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:24.501 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5d9010700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d4158410 con 0x7fe5d405f6f0 2020-12-05T17:31:24.501 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5d9010700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d41580e0 con 0x7fe5d4064910 2020-12-05T17:31:24.501 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5d9010700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d4157ba0 con 0x7fe5d4063250 2020-12-05T17:31:24.502 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5d880f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe5d4064910 0x7fe5d407f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:24.502 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5cbfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5d4063250 0x7fe5d406fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:24.502 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5cbfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5d4063250 0x7fe5d406fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.61:54256/0 (socket says 172.21.15.61:54256) 2020-12-05T17:31:24.502 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5cbfff700 1 -- 172.21.15.61:0/3057563029 learned_addr learned my addr 172.21.15.61:0/3057563029 (peer_addr_for_me v2:172.21.15.61:0/0) 2020-12-05T17:31:24.503 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5d359e700 1 --2- 172.21.15.61:0/3057563029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d405f6f0 0x7fe5d40af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:24.503 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5d880f700 1 -- 172.21.15.61:0/3057563029 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5d4063250 msgr2=0x7fe5d406fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:24.503 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5d880f700 1 --2- 172.21.15.61:0/3057563029 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5d4063250 0x7fe5d406fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.503 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5d880f700 1 -- 172.21.15.61:0/3057563029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d405f6f0 msgr2=0x7fe5d40af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:24.503 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5d880f700 1 --2- 172.21.15.61:0/3057563029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d405f6f0 0x7fe5d40af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.504 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5d880f700 1 -- 172.21.15.61:0/3057563029 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d4157e40 con 0x7fe5d4064910 2020-12-05T17:31:24.504 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5d880f700 1 --2- 172.21.15.61:0/3057563029 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe5d4064910 0x7fe5d407f980 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fe5cc00c000 tx=0x7fe5cc0091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:24.504 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5d0d99700 1 -- 172.21.15.61:0/3057563029 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5cc00cbd0 con 0x7fe5d4064910 2020-12-05T17:31:24.504 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5d0d99700 1 -- 172.21.15.61:0/3057563029 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5cc00cd30 con 0x7fe5d4064910 2020-12-05T17:31:24.505 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5d9010700 1 -- 172.21.15.61:0/3057563029 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d4157880 con 0x7fe5d4064910 2020-12-05T17:31:24.505 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5d0d99700 1 -- 172.21.15.61:0/3057563029 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5cc00cea0 con 0x7fe5d4064910 2020-12-05T17:31:24.505 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5d9010700 1 -- 172.21.15.61:0/3057563029 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d4135d40 con 0x7fe5d4064910 2020-12-05T17:31:24.505 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5d0d99700 1 -- 172.21.15.61:0/3057563029 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fe5cc022070 con 0x7fe5d4064910 2020-12-05T17:31:24.506 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.485+0000 7fe5d0d99700 1 -- 172.21.15.61:0/3057563029 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1444+0+0 (secure 0 0 0) 0x7fe5cc0127b0 con 0x7fe5d4064910 2020-12-05T17:31:24.506 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.489+0000 7fe5d9010700 1 -- 172.21.15.61:0/3057563029 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d4057050 con 0x7fe5d4064910 2020-12-05T17:31:24.506 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.493+0000 7fe5d0d99700 1 -- 172.21.15.61:0/3057563029 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fe5cc039870 con 0x7fe5d4064910 2020-12-05T17:31:24.574 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.569+0000 7fe5d9010700 1 -- 172.21.15.61:0/3057563029 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8dfd8567-9b1c-4d13-9c96-071183e469c2", "id": 6} v 0) v1 -- 0x7fe5d4080a20 con 0x7fe5d4064910 2020-12-05T17:31:24.580 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.577+0000 7fe5d0d99700 1 -- 172.21.15.61:0/3057563029 <== mon.1 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8dfd8567-9b1c-4d13-9c96-071183e469c2", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe5cc029030 con 0x7fe5d4064910 2020-12-05T17:31:24.580 INFO:teuthology.orchestra.run.smithi061.stdout:6 2020-12-05T17:31:24.582 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.577+0000 7fe5d9010700 1 -- 172.21.15.61:0/3057563029 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe5d4064910 msgr2=0x7fe5d407f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:24.582 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.577+0000 7fe5d9010700 1 --2- 172.21.15.61:0/3057563029 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe5d4064910 0x7fe5d407f980 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fe5cc00c000 tx=0x7fe5cc0091a0).stop 2020-12-05T17:31:24.582 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.577+0000 7fe5d9010700 1 -- 172.21.15.61:0/3057563029 shutdown_connections 2020-12-05T17:31:24.582 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.577+0000 7fe5d9010700 1 --2- 172.21.15.61:0/3057563029 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe5d4064910 0x7fe5d407f980 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.583 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.577+0000 7fe5d9010700 1 --2- 172.21.15.61:0/3057563029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d405f6f0 0x7fe5d40af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.583 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.577+0000 7fe5d9010700 1 --2- 172.21.15.61:0/3057563029 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5d4063250 0x7fe5d406fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.583 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.577+0000 7fe5d9010700 1 -- 172.21.15.61:0/3057563029 >> 172.21.15.61:0/3057563029 conn(0x7fe5d40b4b10 msgr2=0x7fe5d4061860 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:24.583 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.577+0000 7fe5d9010700 1 -- 172.21.15.61:0/3057563029 shutdown_connections 2020-12-05T17:31:24.583 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.577+0000 7fe5d9010700 1 -- 172.21.15.61:0/3057563029 wait complete. 2020-12-05T17:31:24.593 INFO:teuthology.orchestra.run.smithi061:> sudo ceph --cluster ceph osd new 24d466b3-01dc-4ba2-8b81-85250d163bd2 7 2020-12-05T17:31:24.703 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.693+0000 7fe2df59e700 1 Processor -- start 2020-12-05T17:31:24.703 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.697+0000 7fe2df59e700 1 -- start start 2020-12-05T17:31:24.704 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.697+0000 7fe2df59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2e00b0960 0x7fe2e007ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:24.704 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.697+0000 7fe2df59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e01537e0 con 0x7fe2e00b3670 2020-12-05T17:31:24.704 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.697+0000 7fe2df59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e00af730 con 0x7fe2e00b1450 2020-12-05T17:31:24.704 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.697+0000 7fe2df59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e00ae360 con 0x7fe2e00b0960 2020-12-05T17:31:24.705 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.697+0000 7fe2ded9d700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fe2e00b1450 0x7fe2e01530d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.61:33148/0 (socket says 172.21.15.61:33148) 2020-12-05T17:31:24.705 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.697+0000 7fe2ded9d700 1 -- 172.21.15.61:0/3889911206 learned_addr learned my addr 172.21.15.61:0/3889911206 (peer_addr_for_me v1:172.21.15.61:0/0) 2020-12-05T17:31:24.705 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.697+0000 7fe2e4e06700 1 --2- 172.21.15.61:0/3889911206 >> v2:172.21.15.32:3300/0 conn(0x7fe2e00b0960 0x7fe2e007ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:24.705 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.697+0000 7fe2ddd9b700 1 -- 172.21.15.61:0/3889911206 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4107958525 0 0) 0x7fe2e00af730 con 0x7fe2e00b1450 2020-12-05T17:31:24.706 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.697+0000 7fe2ddd9b700 1 -- 172.21.15.61:0/3889911206 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2cc003230 con 0x7fe2e00b1450 2020-12-05T17:31:24.706 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.697+0000 7fe2ddd9b700 1 -- 172.21.15.61:0/3889911206 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2740513125 0 0) 0x7fe2e01537e0 con 0x7fe2e00b3670 2020-12-05T17:31:24.706 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.697+0000 7fe2ddd9b700 1 -- 172.21.15.61:0/3889911206 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2e00af730 con 0x7fe2e00b3670 2020-12-05T17:31:24.706 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.697+0000 7fe2e4e06700 1 -- 172.21.15.61:0/3889911206 >> v1:172.21.15.61:6789/0 conn(0x7fe2e00b1450 legacy=0x7fe2e01530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:24.707 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.697+0000 7fe2ddd9b700 1 -- 172.21.15.61:0/3889911206 <== mon.1 v1:172.21.15.61:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4219871791 0 0) 0x7fe2cc003230 con 0x7fe2e00b1450 2020-12-05T17:31:24.707 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.697+0000 7fe2e4e06700 1 -- 172.21.15.61:0/3889911206 >> v1:172.21.15.32:6789/0 conn(0x7fe2e00b3670 legacy=0x7fe2e00af020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:24.707 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.697+0000 7fe2e4e06700 1 -- 172.21.15.61:0/3889911206 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e0152990 con 0x7fe2e00b0960 2020-12-05T17:31:24.707 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.697+0000 7fe2e4e06700 1 --2- 172.21.15.61:0/3889911206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e00b0960 0x7fe2e007ff60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fe2c80010d0 tx=0x7fe2d8006dc0).ready entity=mon.0 client_cookie=38f2b06ea6708acd server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:24.707 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.697+0000 7fe2ddd9b700 1 -- 172.21.15.61:0/3889911206 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2c80016d0 con 0x7fe2e00b0960 2020-12-05T17:31:24.708 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.697+0000 7fe2ddd9b700 1 -- 172.21.15.61:0/3889911206 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d800d5c0 con 0x7fe2e00b0960 2020-12-05T17:31:24.708 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.697+0000 7fe2ddd9b700 1 -- 172.21.15.61:0/3889911206 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2d0001a50 con 0x7fe2e00b0960 2020-12-05T17:31:24.708 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.697+0000 7fe2df59e700 1 -- 172.21.15.61:0/3889911206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e00b0960 msgr2=0x7fe2e007ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:24.708 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.697+0000 7fe2df59e700 1 --2- 172.21.15.61:0/3889911206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e00b0960 0x7fe2e007ff60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fe2c80010d0 tx=0x7fe2d8006dc0).stop 2020-12-05T17:31:24.709 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.701+0000 7fe2df59e700 1 -- 172.21.15.61:0/3889911206 shutdown_connections 2020-12-05T17:31:24.709 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.701+0000 7fe2df59e700 1 --2- 172.21.15.61:0/3889911206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e00b0960 0x7fe2e007ff60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.709 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.701+0000 7fe2df59e700 1 -- 172.21.15.61:0/3889911206 >> 172.21.15.61:0/3889911206 conn(0x7fe2e0061010 msgr2=0x7fe2e0061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:24.709 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.701+0000 7fe2df59e700 1 -- 172.21.15.61:0/3889911206 shutdown_connections 2020-12-05T17:31:24.709 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.701+0000 7fe2df59e700 1 -- 172.21.15.61:0/3889911206 wait complete. 2020-12-05T17:31:24.712 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.701+0000 7fe2df59e700 1 Processor -- start 2020-12-05T17:31:24.712 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.701+0000 7fe2df59e700 1 -- start start 2020-12-05T17:31:24.712 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.701+0000 7fe2df59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2e00b0960 0x7fe2e0025340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:24.712 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.701+0000 7fe2df59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2e00b1450 0x7fe2e012ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:24.713 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.701+0000 7fe2df59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e00b3670 0x7fe2e012d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:24.713 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.701+0000 7fe2df59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e0066290 con 0x7fe2e00b3670 2020-12-05T17:31:24.713 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.701+0000 7fe2df59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e0066c10 con 0x7fe2e00b0960 2020-12-05T17:31:24.713 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.701+0000 7fe2df59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e00666d0 con 0x7fe2e00b1450 2020-12-05T17:31:24.713 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.701+0000 7fe2ded9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2e00b0960 0x7fe2e0025340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:24.714 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.701+0000 7fe2de59c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2e00b1450 0x7fe2e012ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:24.714 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.705+0000 7fe2ded9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2e00b0960 0x7fe2e0025340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:41948/0 (socket says 172.21.15.61:41948) 2020-12-05T17:31:24.714 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.705+0000 7fe2de59c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2e00b1450 0x7fe2e012ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.61:54266/0 (socket says 172.21.15.61:54266) 2020-12-05T17:31:24.714 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.705+0000 7fe2ded9d700 1 -- 172.21.15.61:0/3979038744 learned_addr learned my addr 172.21.15.61:0/3979038744 (peer_addr_for_me v2:172.21.15.61:0/0) 2020-12-05T17:31:24.715 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.705+0000 7fe2e4e06700 1 --2- 172.21.15.61:0/3979038744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e00b3670 0x7fe2e012d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:24.715 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.705+0000 7fe2de59c700 1 -- 172.21.15.61:0/3979038744 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2e00b0960 msgr2=0x7fe2e0025340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:24.715 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.705+0000 7fe2de59c700 1 --2- 172.21.15.61:0/3979038744 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2e00b0960 0x7fe2e0025340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.715 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.705+0000 7fe2de59c700 1 -- 172.21.15.61:0/3979038744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e00b3670 msgr2=0x7fe2e012d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:24.716 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.705+0000 7fe2de59c700 1 --2- 172.21.15.61:0/3979038744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e00b3670 0x7fe2e012d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.716 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.705+0000 7fe2de59c700 1 -- 172.21.15.61:0/3979038744 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e0066970 con 0x7fe2e00b1450 2020-12-05T17:31:24.716 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.705+0000 7fe2ded9d700 1 --2- 172.21.15.61:0/3979038744 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2e00b0960 0x7fe2e0025340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:31:24.716 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.705+0000 7fe2de59c700 1 --2- 172.21.15.61:0/3979038744 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2e00b1450 0x7fe2e012ce30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fe2e0119060 tx=0x7fe2c80090f0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:24.716 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.705+0000 7fe2d77fe700 1 -- 172.21.15.61:0/3979038744 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2c8001690 con 0x7fe2e00b1450 2020-12-05T17:31:24.717 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.705+0000 7fe2df59e700 1 -- 172.21.15.61:0/3979038744 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e0066e10 con 0x7fe2e00b1450 2020-12-05T17:31:24.717 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.705+0000 7fe2df59e700 1 -- 172.21.15.61:0/3979038744 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e00653f0 con 0x7fe2e00b1450 2020-12-05T17:31:24.717 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.705+0000 7fe2d77fe700 1 -- 172.21.15.61:0/3979038744 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c8010070 con 0x7fe2e00b1450 2020-12-05T17:31:24.717 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.705+0000 7fe2d77fe700 1 -- 172.21.15.61:0/3979038744 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2c80019e0 con 0x7fe2e00b1450 2020-12-05T17:31:24.718 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.705+0000 7fe2d77fe700 1 -- 172.21.15.61:0/3979038744 <== mon.2 v2:172.21.15.61:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fe2c801c070 con 0x7fe2e00b1450 2020-12-05T17:31:24.718 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.705+0000 7fe2d77fe700 1 -- 172.21.15.61:0/3979038744 <== mon.2 v2:172.21.15.61:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1559+0+0 (secure 0 0 0) 0x7fe2c800cb00 con 0x7fe2e00b1450 2020-12-05T17:31:24.718 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.705+0000 7fe2df59e700 1 -- 172.21.15.61:0/3979038744 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e00716a0 con 0x7fe2e00b1450 2020-12-05T17:31:24.718 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.709+0000 7fe2d77fe700 1 -- 172.21.15.61:0/3979038744 <== mon.2 v2:172.21.15.61:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fe2c801a060 con 0x7fe2e00b1450 2020-12-05T17:31:24.787 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.785+0000 7fe2d77fe700 1 -- 172.21.15.61:0/3979038744 <== mon.2 v2:172.21.15.61:3301/0 7 ==== mgrmap(e 2) v1 ==== 33316+0+0 (secure 0 0 0) 0x7fe2c800cd10 con 0x7fe2e00b1450 2020-12-05T17:31:24.801 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.797+0000 7fe2df59e700 1 -- 172.21.15.61:0/3979038744 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "osd new", "uuid": "24d466b3-01dc-4ba2-8b81-85250d163bd2", "id": 7} v 0) v1 -- 0x7fe2e0057050 con 0x7fe2e00b1450 2020-12-05T17:31:24.808 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.805+0000 7fe2d77fe700 1 -- 172.21.15.61:0/3979038744 <== mon.2 v2:172.21.15.61:3301/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "24d466b3-01dc-4ba2-8b81-85250d163bd2", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe2c801f480 con 0x7fe2e00b1450 2020-12-05T17:31:24.808 INFO:teuthology.orchestra.run.smithi061.stdout:7 2020-12-05T17:31:24.809 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.805+0000 7fe2df59e700 1 -- 172.21.15.61:0/3979038744 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2e00b1450 msgr2=0x7fe2e012ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:24.810 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.805+0000 7fe2df59e700 1 --2- 172.21.15.61:0/3979038744 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2e00b1450 0x7fe2e012ce30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fe2e0119060 tx=0x7fe2c80090f0).stop 2020-12-05T17:31:24.810 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.805+0000 7fe2df59e700 1 -- 172.21.15.61:0/3979038744 shutdown_connections 2020-12-05T17:31:24.810 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.805+0000 7fe2df59e700 1 --2- 172.21.15.61:0/3979038744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e00b3670 0x7fe2e012d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.810 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.805+0000 7fe2df59e700 1 --2- 172.21.15.61:0/3979038744 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2e00b0960 0x7fe2e0025340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.811 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.805+0000 7fe2df59e700 1 --2- 172.21.15.61:0/3979038744 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2e00b1450 0x7fe2e012ce30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:24.811 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.805+0000 7fe2df59e700 1 -- 172.21.15.61:0/3979038744 >> 172.21.15.61:0/3979038744 conn(0x7fe2e0061010 msgr2=0x7fe2e00b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:24.811 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.805+0000 7fe2df59e700 1 -- 172.21.15.61:0/3979038744 shutdown_connections 2020-12-05T17:31:24.811 INFO:teuthology.orchestra.run.smithi061.stderr:2020-12-05T17:31:24.805+0000 7fe2df59e700 1 -- 172.21.15.61:0/3979038744 wait complete. 2020-12-05T17:31:24.821 INFO:tasks.ceph.osd.0:Restarting daemon 2020-12-05T17:31:24.821 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2020-12-05T17:31:24.824 INFO:tasks.ceph.osd.0:Started 2020-12-05T17:31:24.824 INFO:tasks.ceph.osd.1:Restarting daemon 2020-12-05T17:31:24.824 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2020-12-05T17:31:24.827 INFO:tasks.ceph.osd.1:Started 2020-12-05T17:31:24.827 INFO:tasks.ceph.osd.2:Restarting daemon 2020-12-05T17:31:24.827 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2020-12-05T17:31:24.829 INFO:tasks.ceph.osd.2:Started 2020-12-05T17:31:24.830 INFO:tasks.ceph.osd.3:Restarting daemon 2020-12-05T17:31:24.830 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2020-12-05T17:31:24.832 INFO:tasks.ceph.osd.3:Started 2020-12-05T17:31:24.833 INFO:tasks.ceph.osd.4:Restarting daemon 2020-12-05T17:31:24.833 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2020-12-05T17:31:24.835 INFO:tasks.ceph.osd.4:Started 2020-12-05T17:31:24.836 INFO:tasks.ceph.osd.5:Restarting daemon 2020-12-05T17:31:24.836 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2020-12-05T17:31:24.838 INFO:tasks.ceph.osd.5:Started 2020-12-05T17:31:24.838 INFO:tasks.ceph.osd.6:Restarting daemon 2020-12-05T17:31:24.839 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2020-12-05T17:31:24.841 INFO:tasks.ceph.osd.6:Started 2020-12-05T17:31:24.842 INFO:tasks.ceph.osd.7:Restarting daemon 2020-12-05T17:31:24.842 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2020-12-05T17:31:24.845 INFO:tasks.ceph.osd.7:Started 2020-12-05T17:31:24.845 INFO:tasks.ceph:Waiting for OSDs to come up 2020-12-05T17:31:25.223 INFO:tasks.ceph.osd.0.smithi032.stderr:2020-12-05T17:31:25.219+0000 7febdd388d80 -1 Falling back to public interface 2020-12-05T17:31:25.435 INFO:tasks.ceph.osd.2.smithi032.stderr:2020-12-05T17:31:25.431+0000 7fbef9cb6d80 -1 Falling back to public interface 2020-12-05T17:31:25.455 INFO:tasks.ceph.osd.5.smithi061.stderr:2020-12-05T17:31:25.453+0000 7f93b5f7ed80 -1 Falling back to public interface 2020-12-05T17:31:25.466 INFO:tasks.ceph.osd.4.smithi061.stderr:2020-12-05T17:31:25.465+0000 7fe377cf2d80 -1 Falling back to public interface 2020-12-05T17:31:25.467 INFO:tasks.ceph.osd.1.smithi032.stderr:2020-12-05T17:31:25.463+0000 7fe74f78ad80 -1 Falling back to public interface 2020-12-05T17:31:25.468 INFO:tasks.ceph.osd.7.smithi061.stderr:2020-12-05T17:31:25.465+0000 7f478b4d3d80 -1 Falling back to public interface 2020-12-05T17:31:25.470 INFO:tasks.ceph.osd.6.smithi061.stderr:2020-12-05T17:31:25.469+0000 7f4e77154d80 -1 Falling back to public interface 2020-12-05T17:31:25.475 INFO:tasks.ceph.osd.3.smithi032.stderr:2020-12-05T17:31:25.471+0000 7fbc37c29d80 -1 Falling back to public interface 2020-12-05T17:31:25.653 INFO:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-12-05T17:31:25.762 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8ab59e700 1 Processor -- start 2020-12-05T17:31:25.763 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8ab59e700 1 -- start start 2020-12-05T17:31:25.763 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8ab59e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fe8ac0b1450 0x7fe8ac1530d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:25.763 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8ab59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ac0af730 con 0x7fe8ac0b3670 2020-12-05T17:31:25.764 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8ab59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ac0ae360 con 0x7fe8ac0b0960 2020-12-05T17:31:25.764 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8ab59e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ac152990 con 0x7fe8ac0b1450 2020-12-05T17:31:25.764 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8aa59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe8ac0b3670 0x7fe8ac0af020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34564/0 (socket says 172.21.15.32:34564) 2020-12-05T17:31:25.765 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8aa59c700 1 -- 172.21.15.32:0/1507091934 learned_addr learned my addr 172.21.15.32:0/1507091934 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:25.765 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8aad9d700 1 --2- 172.21.15.32:0/1507091934 >> v2:172.21.15.61:3300/0 conn(0x7fe8ac0b1450 0x7fe8ac1530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:25.765 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8a9d9b700 1 -- 172.21.15.32:0/1507091934 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1111516027 0 0) 0x7fe8ac0af730 con 0x7fe8ac0b3670 2020-12-05T17:31:25.765 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8a9d9b700 1 -- 172.21.15.32:0/1507091934 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe898003230 con 0x7fe8ac0b3670 2020-12-05T17:31:25.765 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8a9d9b700 1 -- 172.21.15.32:0/1507091934 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3367255763 0 0) 0x7fe898003230 con 0x7fe8ac0b3670 2020-12-05T17:31:25.766 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8a9d9b700 1 -- 172.21.15.32:0/1507091934 >> v2:172.21.15.61:3300/0 conn(0x7fe8ac0b1450 msgr2=0x7fe8ac1530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:25.766 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8a9d9b700 1 --2- 172.21.15.32:0/1507091934 >> v2:172.21.15.61:3300/0 conn(0x7fe8ac0b1450 0x7fe8ac1530d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:25.766 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8a9d9b700 1 -- 172.21.15.32:0/1507091934 >> v1:172.21.15.61:6789/0 conn(0x7fe8ac0b0960 legacy=0x7fe8ac07ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:25.766 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8a9d9b700 1 -- 172.21.15.32:0/1507091934 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac152c30 con 0x7fe8ac0b3670 2020-12-05T17:31:25.766 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8a9d9b700 1 -- 172.21.15.32:0/1507091934 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fe8940016d0 con 0x7fe8ac0b3670 2020-12-05T17:31:25.767 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8a9d9b700 1 -- 172.21.15.32:0/1507091934 >> v1:172.21.15.32:6789/0 conn(0x7fe8ac0b3670 legacy=0x7fe8ac0af020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:25.767 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8a9d9b700 1 --2- 172.21.15.32:0/1507091934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe898003720 0x7fe898003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:25.767 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8a9d9b700 1 --2- 172.21.15.32:0/1507091934 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8980044d0 0x7fe898004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:25.767 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8a9d9b700 1 --2- 172.21.15.32:0/1507091934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe898004e40 0x7fe89800b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:25.768 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8a9d9b700 1 -- 172.21.15.32:0/1507091934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac152990 con 0x7fe898004e40 2020-12-05T17:31:25.768 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8a9d9b700 1 -- 172.21.15.32:0/1507091934 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe898004250 con 0x7fe898003720 2020-12-05T17:31:25.768 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8a9d9b700 1 -- 172.21.15.32:0/1507091934 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89804fc20 con 0x7fe8980044d0 2020-12-05T17:31:25.768 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8b0c8c700 1 --2- 172.21.15.32:0/1507091934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe898004e40 0x7fe89800b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:25.768 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8aad9d700 1 --2- 172.21.15.32:0/1507091934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe898003720 0x7fe898003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:25.769 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8aa59c700 1 --2- 172.21.15.32:0/1507091934 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8980044d0 0x7fe898004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:25.769 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8aad9d700 1 -- 172.21.15.32:0/1507091934 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8980044d0 msgr2=0x7fe898004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:25.769 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8aad9d700 1 --2- 172.21.15.32:0/1507091934 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8980044d0 0x7fe898004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:25.769 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8aad9d700 1 -- 172.21.15.32:0/1507091934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe898004e40 msgr2=0x7fe89800b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:25.769 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8aad9d700 1 --2- 172.21.15.32:0/1507091934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe898004e40 0x7fe89800b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:25.770 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8aad9d700 1 -- 172.21.15.32:0/1507091934 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe898003230 con 0x7fe898003720 2020-12-05T17:31:25.770 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8aa59c700 1 --2- 172.21.15.32:0/1507091934 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8980044d0 0x7fe898004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:31:25.770 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8b0c8c700 1 --2- 172.21.15.32:0/1507091934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe898004e40 0x7fe89800b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:31:25.770 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8b0c8c700 1 -- 172.21.15.32:0/1507091934 reap_dead start 2020-12-05T17:31:25.770 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.751+0000 7fe8aad9d700 1 --2- 172.21.15.32:0/1507091934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe898003720 0x7fe898003b40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fe89c00c000 tx=0x7fe89c0091a0).ready entity=mon.1 client_cookie=1076d80c92e92ba7 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:25.771 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.755+0000 7fe8a9d9b700 1 -- 172.21.15.32:0/1507091934 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe89c00eb80 con 0x7fe898003720 2020-12-05T17:31:25.771 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.755+0000 7fe8a9d9b700 1 -- 172.21.15.32:0/1507091934 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89c00ece0 con 0x7fe898003720 2020-12-05T17:31:25.771 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.755+0000 7fe8a9d9b700 1 -- 172.21.15.32:0/1507091934 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe89c0125e0 con 0x7fe898003720 2020-12-05T17:31:25.771 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.755+0000 7fe8ab59e700 1 -- 172.21.15.32:0/1507091934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe898003720 msgr2=0x7fe898003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:25.772 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.755+0000 7fe8ab59e700 1 --2- 172.21.15.32:0/1507091934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe898003720 0x7fe898003b40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fe89c00c000 tx=0x7fe89c0091a0).stop 2020-12-05T17:31:25.772 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.755+0000 7fe8ab59e700 1 -- 172.21.15.32:0/1507091934 shutdown_connections 2020-12-05T17:31:25.772 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.755+0000 7fe8ab59e700 1 --2- 172.21.15.32:0/1507091934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe898003720 0x7fe898003b40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:25.772 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.755+0000 7fe8ab59e700 1 -- 172.21.15.32:0/1507091934 >> 172.21.15.32:0/1507091934 conn(0x7fe8ac061010 msgr2=0x7fe8ac061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:25.772 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.755+0000 7fe8ab59e700 1 -- 172.21.15.32:0/1507091934 shutdown_connections 2020-12-05T17:31:25.773 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.755+0000 7fe8ab59e700 1 -- 172.21.15.32:0/1507091934 wait complete. 2020-12-05T17:31:25.773 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.755+0000 7fe8ab59e700 1 Processor -- start 2020-12-05T17:31:25.773 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.755+0000 7fe8ab59e700 1 -- start start 2020-12-05T17:31:25.773 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.759+0000 7fe8ab59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe898003720 0x7fe8ac12f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:25.773 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.759+0000 7fe8ab59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8ac074040 0x7fe8ac064a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:25.774 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.759+0000 7fe8ab59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8ac151810 0x7fe8ac151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:25.774 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.759+0000 7fe8ab59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac150dd0 con 0x7fe8ac074040 2020-12-05T17:31:25.774 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.759+0000 7fe8ab59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac150890 con 0x7fe898003720 2020-12-05T17:31:25.774 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.759+0000 7fe8ab59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac150b30 con 0x7fe8ac151810 2020-12-05T17:31:25.774 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.759+0000 7fe8aa59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8ac074040 0x7fe8ac064a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:25.775 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.759+0000 7fe8aa59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8ac074040 0x7fe8ac064a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60160/0 (socket says 172.21.15.32:60160) 2020-12-05T17:31:25.775 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.759+0000 7fe8aa59c700 1 -- 172.21.15.32:0/3890279444 learned_addr learned my addr 172.21.15.32:0/3890279444 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:25.775 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.759+0000 7fe8aad9d700 1 --2- 172.21.15.32:0/3890279444 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe898003720 0x7fe8ac12f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:25.775 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.759+0000 7fe8b0c8c700 1 --2- 172.21.15.32:0/3890279444 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8ac151810 0x7fe8ac151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:25.777 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.759+0000 7fe8aa59c700 1 -- 172.21.15.32:0/3890279444 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8ac151810 msgr2=0x7fe8ac151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:25.777 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.759+0000 7fe8aa59c700 1 --2- 172.21.15.32:0/3890279444 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8ac151810 0x7fe8ac151cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:25.777 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.759+0000 7fe8aa59c700 1 -- 172.21.15.32:0/3890279444 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe898003720 msgr2=0x7fe8ac12f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:25.777 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.759+0000 7fe8aa59c700 1 --2- 172.21.15.32:0/3890279444 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe898003720 0x7fe8ac12f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:25.778 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.759+0000 7fe8aa59c700 1 -- 172.21.15.32:0/3890279444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac12f6b0 con 0x7fe8ac074040 2020-12-05T17:31:25.778 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.759+0000 7fe8aa59c700 1 --2- 172.21.15.32:0/3890279444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8ac074040 0x7fe8ac064a60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fe89400d000 tx=0x7fe894007770).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:25.778 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.759+0000 7fe8a37fe700 1 -- 172.21.15.32:0/3890279444 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe89400fb60 con 0x7fe8ac074040 2020-12-05T17:31:25.778 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.759+0000 7fe8a37fe700 1 -- 172.21.15.32:0/3890279444 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89400fcc0 con 0x7fe8ac074040 2020-12-05T17:31:25.778 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.759+0000 7fe8ab59e700 1 -- 172.21.15.32:0/3890279444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ac131b70 con 0x7fe8ac074040 2020-12-05T17:31:25.779 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.759+0000 7fe8a37fe700 1 -- 172.21.15.32:0/3890279444 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe894012540 con 0x7fe8ac074040 2020-12-05T17:31:25.779 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.759+0000 7fe8ab59e700 1 -- 172.21.15.32:0/3890279444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ac07fa80 con 0x7fe8ac074040 2020-12-05T17:31:25.779 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.759+0000 7fe8ab59e700 1 -- 172.21.15.32:0/3890279444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ac064750 con 0x7fe8ac074040 2020-12-05T17:31:25.779 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.763+0000 7fe8a37fe700 1 -- 172.21.15.32:0/3890279444 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 2) v1 ==== 33316+0+0 (secure 0 0 0) 0x7fe89401b030 con 0x7fe8ac074040 2020-12-05T17:31:25.779 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.763+0000 7fe8a37fe700 1 -- 172.21.15.32:0/3890279444 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1674+0+0 (secure 0 0 0) 0x7fe89403b2b0 con 0x7fe8ac074040 2020-12-05T17:31:25.780 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.767+0000 7fe8a37fe700 1 -- 172.21.15.32:0/3890279444 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fe8940126a0 con 0x7fe8ac074040 2020-12-05T17:31:25.810 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.807+0000 7fe8a37fe700 1 -- 172.21.15.32:0/3890279444 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 3) v1 ==== 66760+0+0 (secure 0 0 0) 0x7fe89401dd70 con 0x7fe8ac074040 2020-12-05T17:31:25.811 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.807+0000 7fe8a37fe700 1 --2- 172.21.15.32:0/3890279444 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe890055a60 0x7fe890057e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:25.813 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.811+0000 7fe8aad9d700 1 --2- 172.21.15.32:0/3890279444 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe890055a60 0x7fe890057e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:25.814 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.811+0000 7fe8aad9d700 1 --2- 172.21.15.32:0/3890279444 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe890055a60 0x7fe890057e70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7fe89c01d000 tx=0x7fe89c01a000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:25.877 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.875+0000 7fe8ab59e700 1 -- 172.21.15.32:0/3890279444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8ac057050 con 0x7fe8ac074040 2020-12-05T17:31:25.880 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.875+0000 7fe8a37fe700 1 -- 172.21.15.32:0/3890279444 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+8254 (secure 0 0 0) 0x7fe894019030 con 0x7fe8ac074040 2020-12-05T17:31:25.881 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.875+0000 7fe8ab59e700 1 -- 172.21.15.32:0/3890279444 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe890055a60 msgr2=0x7fe890057e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:25.881 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.875+0000 7fe8ab59e700 1 --2- 172.21.15.32:0/3890279444 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe890055a60 0x7fe890057e70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7fe89c01d000 tx=0x7fe89c01a000).stop 2020-12-05T17:31:25.882 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.875+0000 7fe8ab59e700 1 -- 172.21.15.32:0/3890279444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8ac074040 msgr2=0x7fe8ac064a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:25.882 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.875+0000 7fe8ab59e700 1 --2- 172.21.15.32:0/3890279444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8ac074040 0x7fe8ac064a60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fe89400d000 tx=0x7fe894007770).stop 2020-12-05T17:31:25.883 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.875+0000 7fe8ab59e700 1 -- 172.21.15.32:0/3890279444 shutdown_connections 2020-12-05T17:31:25.883 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.875+0000 7fe8ab59e700 1 --2- 172.21.15.32:0/3890279444 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe890055a60 0x7fe890057e70 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:25.883 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.875+0000 7fe8ab59e700 1 --2- 172.21.15.32:0/3890279444 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8ac151810 0x7fe8ac151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:25.884 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.875+0000 7fe8ab59e700 1 --2- 172.21.15.32:0/3890279444 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe898003720 0x7fe8ac12f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:25.884 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.875+0000 7fe8ab59e700 1 --2- 172.21.15.32:0/3890279444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8ac074040 0x7fe8ac064a60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:25.885 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.875+0000 7fe8ab59e700 1 -- 172.21.15.32:0/3890279444 >> 172.21.15.32:0/3890279444 conn(0x7fe8ac061010 msgr2=0x7fe8ac14f500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:25.885 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.875+0000 7fe8ab59e700 1 -- 172.21.15.32:0/3890279444 shutdown_connections 2020-12-05T17:31:25.885 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:25.875+0000 7fe8ab59e700 1 -- 172.21.15.32:0/3890279444 wait complete. 2020-12-05T17:31:25.886 INFO:teuthology.misc.health.smithi032.stdout: 2020-12-05T17:31:25.886 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":11,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:31:25.806577+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:31:25.889 DEBUG:teuthology.misc:0 of 8 OSDs are up 2020-12-05T17:31:26.401 INFO:tasks.ceph.osd.0.smithi032.stderr:2020-12-05T17:31:26.399+0000 7febdd388d80 -1 osd.0 0 log_to_monitors {default=true} 2020-12-05T17:31:26.600 INFO:tasks.ceph.osd.2.smithi032.stderr:2020-12-05T17:31:26.595+0000 7fbef9cb6d80 -1 osd.2 0 log_to_monitors {default=true} 2020-12-05T17:31:26.607 INFO:tasks.ceph.osd.5.smithi061.stderr:2020-12-05T17:31:26.605+0000 7f93b5f7ed80 -1 osd.5 0 log_to_monitors {default=true} 2020-12-05T17:31:26.637 INFO:tasks.ceph.osd.1.smithi032.stderr:2020-12-05T17:31:26.635+0000 7fe74f78ad80 -1 osd.1 0 log_to_monitors {default=true} 2020-12-05T17:31:26.640 INFO:tasks.ceph.osd.4.smithi061.stderr:2020-12-05T17:31:26.637+0000 7fe377cf2d80 -1 osd.4 0 log_to_monitors {default=true} 2020-12-05T17:31:26.641 INFO:tasks.ceph.osd.6.smithi061.stderr:2020-12-05T17:31:26.637+0000 7f4e77154d80 -1 osd.6 0 log_to_monitors {default=true} 2020-12-05T17:31:26.664 INFO:tasks.ceph.osd.7.smithi061.stderr:2020-12-05T17:31:26.661+0000 7f478b4d3d80 -1 osd.7 0 log_to_monitors {default=true} 2020-12-05T17:31:26.675 INFO:tasks.ceph.osd.3.smithi032.stderr:2020-12-05T17:31:26.671+0000 7fbc37c29d80 -1 osd.3 0 log_to_monitors {default=true} 2020-12-05T17:31:27.841 INFO:tasks.ceph.osd.7.smithi061.stderr:2020-12-05T17:31:27.837+0000 7f477471e700 -1 osd.7 0 waiting for initial osdmap 2020-12-05T17:31:27.842 INFO:tasks.ceph.osd.2.smithi032.stderr:2020-12-05T17:31:27.839+0000 7fbee2f01700 -1 osd.2 0 waiting for initial osdmap 2020-12-05T17:31:27.842 INFO:tasks.ceph.osd.4.smithi061.stderr:2020-12-05T17:31:27.837+0000 7fe360f3d700 -1 osd.4 0 waiting for initial osdmap 2020-12-05T17:31:27.842 INFO:tasks.ceph.osd.6.smithi061.stderr:2020-12-05T17:31:27.841+0000 7f4e6039f700 -1 osd.6 0 waiting for initial osdmap 2020-12-05T17:31:27.845 INFO:tasks.ceph.osd.0.smithi032.stderr:2020-12-05T17:31:27.839+0000 7febc65d3700 -1 osd.0 0 waiting for initial osdmap 2020-12-05T17:31:27.845 INFO:tasks.ceph.osd.5.smithi061.stderr:2020-12-05T17:31:27.841+0000 7f939f1c9700 -1 osd.5 0 waiting for initial osdmap 2020-12-05T17:31:27.847 INFO:tasks.ceph.osd.1.smithi032.stderr:2020-12-05T17:31:27.843+0000 7fe7389d5700 -1 osd.1 0 waiting for initial osdmap 2020-12-05T17:31:27.847 INFO:tasks.ceph.osd.3.smithi032.stderr:2020-12-05T17:31:27.843+0000 7fbc20e74700 -1 osd.3 0 waiting for initial osdmap 2020-12-05T17:31:32.696 INFO:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-12-05T17:31:32.814 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.803+0000 7f78e7fff700 1 Processor -- start 2020-12-05T17:31:32.814 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.803+0000 7f78e7fff700 1 -- start start 2020-12-05T17:31:32.814 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.807+0000 7f78e7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f78e80b1450 0x7f78e81530d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:32.815 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.807+0000 7f78e7fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f78e80b0960 0x7f78e807ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:32.815 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.807+0000 7f78e7fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e80af730 con 0x7f78e80b3670 2020-12-05T17:31:32.815 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.807+0000 7f78e7fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e80ae360 con 0x7f78e80b1450 2020-12-05T17:31:32.815 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.807+0000 7f78e7fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e8152990 con 0x7f78e80b0960 2020-12-05T17:31:32.816 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.807+0000 7f78e77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f78e80b1450 0x7f78e81530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:32.816 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.807+0000 7f78e6ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f78e80b3670 0x7f78e80af020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34824/0 (socket says 172.21.15.32:34824) 2020-12-05T17:31:32.816 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.807+0000 7f78e6ffd700 1 -- 172.21.15.32:0/1691843598 learned_addr learned my addr 172.21.15.32:0/1691843598 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:32.817 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.807+0000 7f78ed7f3700 1 --2- 172.21.15.32:0/1691843598 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f78e80b0960 0x7f78e807ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:32.817 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.807+0000 7f78e77fe700 1 -- 172.21.15.32:0/1691843598 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f78e80b0960 msgr2=0x7f78e807ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:32.817 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.807+0000 7f78e77fe700 1 --2- 172.21.15.32:0/1691843598 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f78e80b0960 0x7f78e807ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:32.817 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.807+0000 7f78e77fe700 1 -- 172.21.15.32:0/1691843598 >> v1:172.21.15.32:6789/0 conn(0x7f78e80b3670 legacy=0x7f78e80af020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:32.818 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.807+0000 7f78e77fe700 1 -- 172.21.15.32:0/1691843598 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e8152c30 con 0x7f78e80b1450 2020-12-05T17:31:32.818 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.807+0000 7f78e77fe700 1 --2- 172.21.15.32:0/1691843598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e80b1450 0x7f78e81530d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f78d8002f30 tx=0x7f78d8006dc0).ready entity=mon.0 client_cookie=9b3934582454d5d9 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:32.818 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.807+0000 7f78e67fc700 1 -- 172.21.15.32:0/1691843598 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78d8009760 con 0x7f78e80b1450 2020-12-05T17:31:32.819 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.807+0000 7f78e67fc700 1 -- 172.21.15.32:0/1691843598 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d80098c0 con 0x7f78e80b1450 2020-12-05T17:31:32.819 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.807+0000 7f78e67fc700 1 -- 172.21.15.32:0/1691843598 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78d8009c20 con 0x7f78e80b1450 2020-12-05T17:31:32.819 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.807+0000 7f78e7fff700 1 -- 172.21.15.32:0/1691843598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e80b1450 msgr2=0x7f78e81530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:32.819 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.807+0000 7f78e7fff700 1 --2- 172.21.15.32:0/1691843598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e80b1450 0x7f78e81530d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f78d8002f30 tx=0x7f78d8006dc0).stop 2020-12-05T17:31:32.820 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.807+0000 7f78e7fff700 1 -- 172.21.15.32:0/1691843598 shutdown_connections 2020-12-05T17:31:32.820 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.811+0000 7f78e7fff700 1 --2- 172.21.15.32:0/1691843598 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f78e80b0960 0x7f78e807ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:32.820 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.811+0000 7f78e7fff700 1 --2- 172.21.15.32:0/1691843598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e80b1450 0x7f78e81530d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:32.820 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.811+0000 7f78e7fff700 1 -- 172.21.15.32:0/1691843598 >> 172.21.15.32:0/1691843598 conn(0x7f78e8061010 msgr2=0x7f78e8061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:32.820 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.811+0000 7f78e7fff700 1 -- 172.21.15.32:0/1691843598 shutdown_connections 2020-12-05T17:31:32.821 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.811+0000 7f78e7fff700 1 -- 172.21.15.32:0/1691843598 wait complete. 2020-12-05T17:31:32.825 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.811+0000 7f78e7fff700 1 Processor -- start 2020-12-05T17:31:32.825 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.811+0000 7f78e7fff700 1 -- start start 2020-12-05T17:31:32.825 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78e7fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f78e80b0960 0x7f78e8064170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:32.825 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78e7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e80b1450 0x7f78e80252c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:32.826 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78e7fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f78e80b3670 0x7f78e8063a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:32.826 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78e7fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e8065920 con 0x7f78e80b1450 2020-12-05T17:31:32.826 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78e7fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e8074150 con 0x7f78e80b3670 2020-12-05T17:31:32.827 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78e7fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e80743f0 con 0x7f78e80b0960 2020-12-05T17:31:32.827 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78e6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e80b1450 0x7f78e80252c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:32.827 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78e6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e80b1450 0x7f78e80252c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60412/0 (socket says 172.21.15.32:60412) 2020-12-05T17:31:32.827 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78e6ffd700 1 -- 172.21.15.32:0/2512440718 learned_addr learned my addr 172.21.15.32:0/2512440718 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:32.827 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78e77fe700 1 --2- 172.21.15.32:0/2512440718 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f78e80b0960 0x7f78e8064170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:32.828 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78ed7f3700 1 --2- 172.21.15.32:0/2512440718 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f78e80b3670 0x7f78e8063a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:32.828 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78e6ffd700 1 -- 172.21.15.32:0/2512440718 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f78e80b0960 msgr2=0x7f78e8064170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:32.828 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78e6ffd700 1 --2- 172.21.15.32:0/2512440718 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f78e80b0960 0x7f78e8064170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:32.828 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78e6ffd700 1 -- 172.21.15.32:0/2512440718 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f78e80b3670 msgr2=0x7f78e8063a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:32.829 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78e6ffd700 1 --2- 172.21.15.32:0/2512440718 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f78e80b3670 0x7f78e8063a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:32.829 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78e6ffd700 1 -- 172.21.15.32:0/2512440718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e81515a0 con 0x7f78e80b1450 2020-12-05T17:31:32.829 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78e6ffd700 1 --2- 172.21.15.32:0/2512440718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e80b1450 0x7f78e80252c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f78e8062a20 tx=0x7f78dc007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:32.829 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78cffff700 1 -- 172.21.15.32:0/2512440718 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78dc0072e0 con 0x7f78e80b1450 2020-12-05T17:31:32.830 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78cffff700 1 -- 172.21.15.32:0/2512440718 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78dc007440 con 0x7f78e80b1450 2020-12-05T17:31:32.830 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78e7fff700 1 -- 172.21.15.32:0/2512440718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e8151a70 con 0x7f78e80b1450 2020-12-05T17:31:32.830 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78cffff700 1 -- 172.21.15.32:0/2512440718 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78dc0057d0 con 0x7f78e80b1450 2020-12-05T17:31:32.831 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78e7fff700 1 -- 172.21.15.32:0/2512440718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e80631c0 con 0x7f78e80b1450 2020-12-05T17:31:32.831 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78cffff700 1 -- 172.21.15.32:0/2512440718 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f78dc010070 con 0x7f78e80b1450 2020-12-05T17:31:32.831 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.815+0000 7f78e7fff700 1 -- 172.21.15.32:0/2512440718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e8057050 con 0x7f78e80b1450 2020-12-05T17:31:32.831 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.819+0000 7f78cffff700 1 --2- 172.21.15.32:0/2512440718 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f78c80553b0 0x7f78c8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:32.832 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.819+0000 7f78cffff700 1 -- 172.21.15.32:0/2512440718 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4900+0+0 (secure 0 0 0) 0x7f78dc06cfb0 con 0x7f78e80b1450 2020-12-05T17:31:32.832 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.819+0000 7f78e77fe700 1 --2- 172.21.15.32:0/2512440718 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f78c80553b0 0x7f78c8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:32.832 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.819+0000 7f78e77fe700 1 --2- 172.21.15.32:0/2512440718 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f78c80553b0 0x7f78c8057870 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f78d801a000 tx=0x7f78d8015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:32.833 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.823+0000 7f78cffff700 1 -- 172.21.15.32:0/2512440718 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f78dc043740 con 0x7f78e80b1450 2020-12-05T17:31:32.971 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.967+0000 7f78e7fff700 1 -- 172.21.15.32:0/2512440718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78e805d810 con 0x7f78e80b1450 2020-12-05T17:31:32.973 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.971+0000 7f78cffff700 1 -- 172.21.15.32:0/2512440718 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+11805 (secure 0 0 0) 0x7f78dc046d60 con 0x7f78e80b1450 2020-12-05T17:31:32.973 INFO:teuthology.misc.health.smithi032.stdout: 2020-12-05T17:31:32.974 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":16,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:31:30.866212+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:31:32.975 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.971+0000 7f78e7fff700 1 -- 172.21.15.32:0/2512440718 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f78c80553b0 msgr2=0x7f78c8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:32.975 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.971+0000 7f78e7fff700 1 --2- 172.21.15.32:0/2512440718 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f78c80553b0 0x7f78c8057870 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f78d801a000 tx=0x7f78d8015000).stop 2020-12-05T17:31:32.976 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.971+0000 7f78e7fff700 1 -- 172.21.15.32:0/2512440718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e80b1450 msgr2=0x7f78e80252c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:32.976 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.971+0000 7f78e7fff700 1 --2- 172.21.15.32:0/2512440718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e80b1450 0x7f78e80252c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f78e8062a20 tx=0x7f78dc007c40).stop 2020-12-05T17:31:32.976 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.971+0000 7f78e7fff700 1 -- 172.21.15.32:0/2512440718 shutdown_connections 2020-12-05T17:31:32.976 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.971+0000 7f78e7fff700 1 --2- 172.21.15.32:0/2512440718 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f78e80b3670 0x7f78e8063a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:32.976 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.971+0000 7f78e7fff700 1 --2- 172.21.15.32:0/2512440718 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f78c80553b0 0x7f78c8057870 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:32.977 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.971+0000 7f78e7fff700 1 --2- 172.21.15.32:0/2512440718 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f78e80b0960 0x7f78e8064170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:32.977 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.971+0000 7f78e7fff700 1 --2- 172.21.15.32:0/2512440718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e80b1450 0x7f78e80252c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:32.977 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.971+0000 7f78e7fff700 1 -- 172.21.15.32:0/2512440718 >> 172.21.15.32:0/2512440718 conn(0x7f78e8061010 msgr2=0x7f78e80b1830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:32.977 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.971+0000 7f78e7fff700 1 -- 172.21.15.32:0/2512440718 shutdown_connections 2020-12-05T17:31:32.978 INFO:teuthology.misc.health.smithi032.stderr:2020-12-05T17:31:32.971+0000 7f78e7fff700 1 -- 172.21.15.32:0/2512440718 wait complete. 2020-12-05T17:31:32.985 DEBUG:teuthology.misc:8 of 8 OSDs are up 2020-12-05T17:31:32.986 INFO:tasks.ceph:Creating RBD pool 2020-12-05T17:31:32.986 INFO:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd pool create rbd 8 2020-12-05T17:31:33.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.095+0000 7f4d23fff700 1 Processor -- start 2020-12-05T17:31:33.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.095+0000 7f4d23fff700 1 -- start start 2020-12-05T17:31:33.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d23fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4d24061a60 0x7f4d24025a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:33.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d23fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d240802a0 0x7f4d24153480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:33.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d23fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d24025800 con 0x7f4d2407fe40 2020-12-05T17:31:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d23fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d24066610 con 0x7f4d24061a60 2020-12-05T17:31:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d23fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d24061e40 con 0x7f4d240802a0 2020-12-05T17:31:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d237fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4d24061a60 0x7f4d24025a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d1affd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4d2407fe40 0x7f4d24152d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34836/0 (socket says 172.21.15.32:34836) 2020-12-05T17:31:33.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d1affd700 1 -- 172.21.15.32:0/1911076501 learned_addr learned my addr 172.21.15.32:0/1911076501 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:33.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d2972e700 1 --2- 172.21.15.32:0/1911076501 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d240802a0 0x7f4d24153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:33.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d237fe700 1 -- 172.21.15.32:0/1911076501 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d240802a0 msgr2=0x7f4d24153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:33.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d237fe700 1 --2- 172.21.15.32:0/1911076501 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d240802a0 0x7f4d24153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:33.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d237fe700 1 -- 172.21.15.32:0/1911076501 >> v1:172.21.15.32:6789/0 conn(0x7f4d2407fe40 legacy=0x7f4d24152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:33.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d22ffd700 1 -- 172.21.15.32:0/1911076501 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2560226429 0 0) 0x7f4d24025800 con 0x7f4d2407fe40 2020-12-05T17:31:33.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d237fe700 1 -- 172.21.15.32:0/1911076501 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d24066870 con 0x7f4d24061a60 2020-12-05T17:31:33.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d237fe700 1 --2- 172.21.15.32:0/1911076501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d24061a60 0x7f4d24025a00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f4d14002f30 tx=0x7f4d14006dc0).ready entity=mon.0 client_cookie=8850efc25217eb72 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:33.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d22ffd700 1 -- 172.21.15.32:0/1911076501 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d140097f0 con 0x7f4d24061a60 2020-12-05T17:31:33.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d22ffd700 1 -- 172.21.15.32:0/1911076501 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d14009950 con 0x7f4d24061a60 2020-12-05T17:31:33.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d23fff700 1 -- 172.21.15.32:0/1911076501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d24061a60 msgr2=0x7f4d24025a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:33.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d23fff700 1 --2- 172.21.15.32:0/1911076501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d24061a60 0x7f4d24025a00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f4d14002f30 tx=0x7f4d14006dc0).stop 2020-12-05T17:31:33.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d22ffd700 1 -- 172.21.15.32:0/1911076501 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d14009cb0 con 0x7f4d24061a60 2020-12-05T17:31:33.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d23fff700 1 -- 172.21.15.32:0/1911076501 shutdown_connections 2020-12-05T17:31:33.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d23fff700 1 --2- 172.21.15.32:0/1911076501 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d240802a0 0x7f4d24153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:33.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d23fff700 1 --2- 172.21.15.32:0/1911076501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d24061a60 0x7f4d24025a00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:33.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d23fff700 1 -- 172.21.15.32:0/1911076501 >> 172.21.15.32:0/1911076501 conn(0x7f4d240b6ab0 msgr2=0x7f4d24061010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:33.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d23fff700 1 -- 172.21.15.32:0/1911076501 shutdown_connections 2020-12-05T17:31:33.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.099+0000 7f4d23fff700 1 -- 172.21.15.32:0/1911076501 wait complete. 2020-12-05T17:31:33.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d23fff700 1 Processor -- start 2020-12-05T17:31:33.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d23fff700 1 -- start start 2020-12-05T17:31:33.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d23fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d24061a60 0x7f4d24065e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:33.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d23fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d2407fe40 0x7f4d240a4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:33.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d23fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d240802a0 0x7f4d2413d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:33.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d23fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d240a3040 con 0x7f4d24061a60 2020-12-05T17:31:33.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d23fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d24140ba0 con 0x7f4d240802a0 2020-12-05T17:31:33.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d23fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d240a05f0 con 0x7f4d2407fe40 2020-12-05T17:31:33.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d237fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d24061a60 0x7f4d24065e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:33.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d237fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d24061a60 0x7f4d24065e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60424/0 (socket says 172.21.15.32:60424) 2020-12-05T17:31:33.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d237fe700 1 -- 172.21.15.32:0/3648547297 learned_addr learned my addr 172.21.15.32:0/3648547297 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:33.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d1affd700 1 --2- 172.21.15.32:0/3648547297 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d2407fe40 0x7f4d240a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d2972e700 1 --2- 172.21.15.32:0/3648547297 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d240802a0 0x7f4d2413d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d237fe700 1 -- 172.21.15.32:0/3648547297 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d2407fe40 msgr2=0x7f4d240a4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d237fe700 1 --2- 172.21.15.32:0/3648547297 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d2407fe40 0x7f4d240a4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d237fe700 1 -- 172.21.15.32:0/3648547297 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d240802a0 msgr2=0x7f4d2413d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:33.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d237fe700 1 --2- 172.21.15.32:0/3648547297 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d240802a0 0x7f4d2413d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:33.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d237fe700 1 -- 172.21.15.32:0/3648547297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d24140dd0 con 0x7f4d24061a60 2020-12-05T17:31:33.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d237fe700 1 --2- 172.21.15.32:0/3648547297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d24061a60 0x7f4d24065e60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f4d14002f30 tx=0x7f4d14015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:33.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d20ff9700 1 -- 172.21.15.32:0/3648547297 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d14011c30 con 0x7f4d24061a60 2020-12-05T17:31:33.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d20ff9700 1 -- 172.21.15.32:0/3648547297 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d14011d90 con 0x7f4d24061a60 2020-12-05T17:31:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d20ff9700 1 -- 172.21.15.32:0/3648547297 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d1401d640 con 0x7f4d24061a60 2020-12-05T17:31:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d23fff700 1 -- 172.21.15.32:0/3648547297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d2413da50 con 0x7f4d24061a60 2020-12-05T17:31:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d23fff700 1 -- 172.21.15.32:0/3648547297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d240a0210 con 0x7f4d24061a60 2020-12-05T17:31:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.103+0000 7f4d23fff700 1 -- 172.21.15.32:0/3648547297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d240a0430 con 0x7f4d24061a60 2020-12-05T17:31:33.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.107+0000 7f4d20ff9700 1 -- 172.21.15.32:0/3648547297 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4d14024030 con 0x7f4d24061a60 2020-12-05T17:31:33.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.107+0000 7f4d20ff9700 1 --2- 172.21.15.32:0/3648547297 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4d08055470 0x7f4d08057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:33.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.107+0000 7f4d20ff9700 1 -- 172.21.15.32:0/3648547297 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4900+0+0 (secure 0 0 0) 0x7f4d1406ede0 con 0x7f4d24061a60 2020-12-05T17:31:33.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.107+0000 7f4d1affd700 1 --2- 172.21.15.32:0/3648547297 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4d08055470 0x7f4d08057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:33.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.107+0000 7f4d1affd700 1 --2- 172.21.15.32:0/3648547297 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4d08055470 0x7f4d08057930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f4d0c00d000 tx=0x7f4d0c007bf0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.111+0000 7f4d20ff9700 1 -- 172.21.15.32:0/3648547297 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4d140454d0 con 0x7f4d24061a60 2020-12-05T17:31:33.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:33.251+0000 7f4d23fff700 1 -- 172.21.15.32:0/3648547297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f4d24140f60 con 0x7f4d24061a60 2020-12-05T17:31:34.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.095+0000 7f4d20ff9700 1 -- 172.21.15.32:0/3648547297 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7f4d14022070 con 0x7f4d24061a60 2020-12-05T17:31:34.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.099+0000 7f4d23fff700 1 -- 172.21.15.32:0/3648547297 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4d08055470 msgr2=0x7f4d08057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:34.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.099+0000 7f4d23fff700 1 --2- 172.21.15.32:0/3648547297 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4d08055470 0x7f4d08057930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f4d0c00d000 tx=0x7f4d0c007bf0).stop 2020-12-05T17:31:34.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.099+0000 7f4d23fff700 1 -- 172.21.15.32:0/3648547297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d24061a60 msgr2=0x7f4d24065e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:34.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.099+0000 7f4d23fff700 1 --2- 172.21.15.32:0/3648547297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d24061a60 0x7f4d24065e60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f4d14002f30 tx=0x7f4d14015000).stop 2020-12-05T17:31:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.099+0000 7f4d23fff700 1 -- 172.21.15.32:0/3648547297 shutdown_connections 2020-12-05T17:31:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.099+0000 7f4d23fff700 1 --2- 172.21.15.32:0/3648547297 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d240802a0 0x7f4d2413d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.099+0000 7f4d23fff700 1 --2- 172.21.15.32:0/3648547297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d24061a60 0x7f4d24065e60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.099+0000 7f4d23fff700 1 --2- 172.21.15.32:0/3648547297 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4d08055470 0x7f4d08057930 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.099+0000 7f4d23fff700 1 --2- 172.21.15.32:0/3648547297 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d2407fe40 0x7f4d240a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.099+0000 7f4d23fff700 1 -- 172.21.15.32:0/3648547297 >> 172.21.15.32:0/3648547297 conn(0x7f4d240b6ab0 msgr2=0x7f4d240b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.099+0000 7f4d23fff700 1 -- 172.21.15.32:0/3648547297 shutdown_connections 2020-12-05T17:31:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.099+0000 7f4d23fff700 1 -- 172.21.15.32:0/3648547297 wait complete. 2020-12-05T17:31:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'rbd' created 2020-12-05T17:31:34.120 INFO:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2020-12-05T17:31:34.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.231+0000 7f8b45c6d700 1 Processor -- start 2020-12-05T17:31:34.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.231+0000 7f8b45c6d700 1 -- start start 2020-12-05T17:31:34.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.235+0000 7f8b45c6d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8b40153240 0x7f8b400ae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:34.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.235+0000 7f8b45c6d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b40080a90 con 0x7f8b400b0240 2020-12-05T17:31:34.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.235+0000 7f8b45c6d700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b40152860 con 0x7f8b400611b0 2020-12-05T17:31:34.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.235+0000 7f8b45c6d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b40152b00 con 0x7f8b40153240 2020-12-05T17:31:34.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.235+0000 7f8b46ecf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8b40153240 0x7f8b400ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:34.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.235+0000 7f8b44c6b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8b400b0240 0x7f8b4014f0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34850/0 (socket says 172.21.15.32:34850) 2020-12-05T17:31:34.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.235+0000 7f8b44c6b700 1 -- 172.21.15.32:0/1296158962 learned_addr learned my addr 172.21.15.32:0/1296158962 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.235+0000 7f8b46ecf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8b40153240 0x7f8b400ae860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60434/0 (socket says 172.21.15.32:60434) 2020-12-05T17:31:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.235+0000 7f8b46ecf700 1 -- 172.21.15.32:0/1296158962 >> v1:172.21.15.61:6789/0 conn(0x7f8b400611b0 legacy=0x7f8b4014c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.235+0000 7f8b46ecf700 1 -- 172.21.15.32:0/1296158962 >> v1:172.21.15.32:6789/0 conn(0x7f8b400b0240 legacy=0x7f8b4014f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.235+0000 7f8b37fff700 1 -- 172.21.15.32:0/1296158962 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 971130578 0 0) 0x7f8b40080a90 con 0x7f8b400b0240 2020-12-05T17:31:34.244 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.235+0000 7f8b46ecf700 1 -- 172.21.15.32:0/1296158962 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b40152da0 con 0x7f8b40153240 2020-12-05T17:31:34.244 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.235+0000 7f8b46ecf700 1 --2- 172.21.15.32:0/1296158962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b40153240 0x7f8b400ae860 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f8b380032b0 tx=0x7f8b380091a0).ready entity=mon.0 client_cookie=57496f354f23e619 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:34.244 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.235+0000 7f8b37fff700 1 -- 172.21.15.32:0/1296158962 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b3800ba50 con 0x7f8b40153240 2020-12-05T17:31:34.244 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.235+0000 7f8b37fff700 1 -- 172.21.15.32:0/1296158962 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3800bbb0 con 0x7f8b40153240 2020-12-05T17:31:34.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.235+0000 7f8b37fff700 1 -- 172.21.15.32:0/1296158962 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b3800bd20 con 0x7f8b40153240 2020-12-05T17:31:34.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.235+0000 7f8b45c6d700 1 -- 172.21.15.32:0/1296158962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b40153240 msgr2=0x7f8b400ae860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:34.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.235+0000 7f8b45c6d700 1 --2- 172.21.15.32:0/1296158962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b40153240 0x7f8b400ae860 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f8b380032b0 tx=0x7f8b380091a0).stop 2020-12-05T17:31:34.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.235+0000 7f8b45c6d700 1 -- 172.21.15.32:0/1296158962 shutdown_connections 2020-12-05T17:31:34.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.235+0000 7f8b45c6d700 1 --2- 172.21.15.32:0/1296158962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b40153240 0x7f8b400ae860 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:34.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.235+0000 7f8b45c6d700 1 -- 172.21.15.32:0/1296158962 >> 172.21.15.32:0/1296158962 conn(0x7f8b400b5f90 msgr2=0x7f8b400b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:34.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.235+0000 7f8b45c6d700 1 -- 172.21.15.32:0/1296158962 shutdown_connections 2020-12-05T17:31:34.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.235+0000 7f8b45c6d700 1 -- 172.21.15.32:0/1296158962 wait complete. 2020-12-05T17:31:34.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b45c6d700 1 Processor -- start 2020-12-05T17:31:34.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b45c6d700 1 -- start start 2020-12-05T17:31:34.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b45c6d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b400611b0 0x7f8b400b5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:34.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b45c6d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b400b0240 0x7f8b4012c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:34.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b45c6d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b40153240 0x7f8b4013e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:34.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b45c6d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b400b3960 con 0x7f8b400b0240 2020-12-05T17:31:34.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b45c6d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b400b36c0 con 0x7f8b400611b0 2020-12-05T17:31:34.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b45c6d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b40066b40 con 0x7f8b40153240 2020-12-05T17:31:34.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b44c6b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b400b0240 0x7f8b4012c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:34.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b44c6b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b400b0240 0x7f8b4012c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60438/0 (socket says 172.21.15.32:60438) 2020-12-05T17:31:34.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b44c6b700 1 -- 172.21.15.32:0/1115650665 learned_addr learned my addr 172.21.15.32:0/1115650665 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:34.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b46ecf700 1 --2- 172.21.15.32:0/1115650665 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b40153240 0x7f8b4013e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:34.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b4546c700 1 --2- 172.21.15.32:0/1115650665 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b400611b0 0x7f8b400b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:34.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b44c6b700 1 -- 172.21.15.32:0/1115650665 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b40153240 msgr2=0x7f8b4013e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:34.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b44c6b700 1 --2- 172.21.15.32:0/1115650665 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b40153240 0x7f8b4013e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:34.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b44c6b700 1 -- 172.21.15.32:0/1115650665 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b400611b0 msgr2=0x7f8b400b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:34.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b44c6b700 1 --2- 172.21.15.32:0/1115650665 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b400611b0 0x7f8b400b5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:34.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b44c6b700 1 -- 172.21.15.32:0/1115650665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b40063160 con 0x7f8b400b0240 2020-12-05T17:31:34.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b44c6b700 1 --2- 172.21.15.32:0/1115650665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b400b0240 0x7f8b4012c010 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8b30000b80 tx=0x7f8b30007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:34.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b35ffb700 1 -- 172.21.15.32:0/1115650665 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b300072e0 con 0x7f8b400b0240 2020-12-05T17:31:34.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b35ffb700 1 -- 172.21.15.32:0/1115650665 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b30007440 con 0x7f8b400b0240 2020-12-05T17:31:34.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b35ffb700 1 -- 172.21.15.32:0/1115650665 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b300057d0 con 0x7f8b400b0240 2020-12-05T17:31:34.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b45c6d700 1 -- 172.21.15.32:0/1115650665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4012c500 con 0x7f8b400b0240 2020-12-05T17:31:34.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.239+0000 7f8b45c6d700 1 -- 172.21.15.32:0/1115650665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4012aa60 con 0x7f8b400b0240 2020-12-05T17:31:34.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.243+0000 7f8b45c6d700 1 -- 172.21.15.32:0/1115650665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b40066d00 con 0x7f8b400b0240 2020-12-05T17:31:34.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.243+0000 7f8b35ffb700 1 -- 172.21.15.32:0/1115650665 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8b30010070 con 0x7f8b400b0240 2020-12-05T17:31:34.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.243+0000 7f8b35ffb700 1 --2- 172.21.15.32:0/1115650665 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8b20055320 0x7f8b200577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:34.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.243+0000 7f8b35ffb700 1 -- 172.21.15.32:0/1115650665 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5242+0+0 (secure 0 0 0) 0x7f8b3006d250 con 0x7f8b400b0240 2020-12-05T17:31:34.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.243+0000 7f8b4546c700 1 --2- 172.21.15.32:0/1115650665 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8b20055320 0x7f8b200577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:34.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.243+0000 7f8b4546c700 1 --2- 172.21.15.32:0/1115650665 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8b20055320 0x7f8b200577e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f8b3c00d000 tx=0x7f8b3c000b80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:34.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.247+0000 7f8b35ffb700 1 -- 172.21.15.32:0/1115650665 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8b30043890 con 0x7f8b400b0240 2020-12-05T17:31:34.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:34.391+0000 7f8b45c6d700 1 -- 172.21.15.32:0/1115650665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8b40057050 con 0x7f8b400b0240 2020-12-05T17:31:35.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.107+0000 7f8b35ffb700 1 -- 172.21.15.32:0/1115650665 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7f8b30046eb0 con 0x7f8b400b0240 2020-12-05T17:31:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.111+0000 7f8b45c6d700 1 -- 172.21.15.32:0/1115650665 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8b20055320 msgr2=0x7f8b200577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.111+0000 7f8b45c6d700 1 --2- 172.21.15.32:0/1115650665 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8b20055320 0x7f8b200577e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f8b3c00d000 tx=0x7f8b3c000b80).stop 2020-12-05T17:31:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.111+0000 7f8b45c6d700 1 -- 172.21.15.32:0/1115650665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b400b0240 msgr2=0x7f8b4012c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.111+0000 7f8b45c6d700 1 --2- 172.21.15.32:0/1115650665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b400b0240 0x7f8b4012c010 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8b30000b80 tx=0x7f8b30007ac0).stop 2020-12-05T17:31:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.111+0000 7f8b45c6d700 1 -- 172.21.15.32:0/1115650665 shutdown_connections 2020-12-05T17:31:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.111+0000 7f8b45c6d700 1 --2- 172.21.15.32:0/1115650665 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8b20055320 0x7f8b200577e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.111+0000 7f8b45c6d700 1 --2- 172.21.15.32:0/1115650665 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b40153240 0x7f8b4013e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.111+0000 7f8b45c6d700 1 --2- 172.21.15.32:0/1115650665 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b400611b0 0x7f8b400b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.111+0000 7f8b45c6d700 1 --2- 172.21.15.32:0/1115650665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b400b0240 0x7f8b4012c010 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.111+0000 7f8b45c6d700 1 -- 172.21.15.32:0/1115650665 >> 172.21.15.32:0/1115650665 conn(0x7f8b400b5f90 msgr2=0x7f8b4012cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.111+0000 7f8b45c6d700 1 -- 172.21.15.32:0/1115650665 shutdown_connections 2020-12-05T17:31:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.111+0000 7f8b45c6d700 1 -- 172.21.15.32:0/1115650665 wait complete. 2020-12-05T17:31:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:enabled application 'rbd' on pool 'rbd' 2020-12-05T17:31:35.130 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2020-12-05T17:31:35.130 INFO:tasks.ceph.mds.a:Restarting daemon 2020-12-05T17:31:35.131 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-12-05T17:31:35.134 INFO:tasks.ceph.mds.a:Started 2020-12-05T17:31:35.135 INFO:tasks.ceph.mds.b:Restarting daemon 2020-12-05T17:31:35.135 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-12-05T17:31:35.142 INFO:tasks.ceph.mds.b:Started 2020-12-05T17:31:35.143 INFO:tasks.ceph:Setting up CephFS filesystem... 2020-12-05T17:31:35.144 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:31:35.205 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:31:35.218 INFO:tasks.ceph.mds.b.smithi061.stdout:starting mds.b at 2020-12-05T17:31:35.324 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.267+0000 7f620f59e700 1 Processor -- start 2020-12-05T17:31:35.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.267+0000 7f620f59e700 1 -- start start 2020-12-05T17:31:35.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.267+0000 7f620f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f621005f3b0 0x7f621005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:35.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.267+0000 7f620f59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f621014f190 con 0x7f621005feb0 2020-12-05T17:31:35.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.267+0000 7f620f59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62101528c0 con 0x7f6210061e90 2020-12-05T17:31:35.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.267+0000 7f620f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6210152b60 con 0x7f621005f3b0 2020-12-05T17:31:35.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.267+0000 7f620dd9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f621005feb0 0x7f6210061780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34876/0 (socket says 172.21.15.32:34876) 2020-12-05T17:31:35.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.267+0000 7f620dd9b700 1 -- 172.21.15.32:0/684444052 learned_addr learned my addr 172.21.15.32:0/684444052 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:35.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.267+0000 7f620e59c700 1 --2- 172.21.15.32:0/684444052 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f621005f3b0 0x7f621005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:35.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.267+0000 7f620d59a700 1 -- 172.21.15.32:0/684444052 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1947194322 0 0) 0x7f621014f190 con 0x7f621005feb0 2020-12-05T17:31:35.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.267+0000 7f620d59a700 1 -- 172.21.15.32:0/684444052 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61f8003230 con 0x7f621005feb0 2020-12-05T17:31:35.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.271+0000 7f620d59a700 1 -- 172.21.15.32:0/684444052 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1814981367 0 0) 0x7f61f8003230 con 0x7f621005feb0 2020-12-05T17:31:35.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620d59a700 1 -- 172.21.15.32:0/684444052 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f621005f3b0 msgr2=0x7f621005f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:35.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620d59a700 1 --2- 172.21.15.32:0/684444052 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f621005f3b0 0x7f621005f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620d59a700 1 -- 172.21.15.32:0/684444052 >> v1:172.21.15.61:6789/0 conn(0x7f6210061e90 legacy=0x7f621007fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:35.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620d59a700 1 -- 172.21.15.32:0/684444052 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6210152e00 con 0x7f621005feb0 2020-12-05T17:31:35.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620d59a700 1 -- 172.21.15.32:0/684444052 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f62040016d0 con 0x7f621005feb0 2020-12-05T17:31:35.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620d59a700 1 -- 172.21.15.32:0/684444052 >> v1:172.21.15.32:6789/0 conn(0x7f621005feb0 legacy=0x7f6210061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:35.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620d59a700 1 --2- 172.21.15.32:0/684444052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61f8003750 0x7f61f8003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:35.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620d59a700 1 --2- 172.21.15.32:0/684444052 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f621005f3b0 0x7f61f80044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:35.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620d59a700 1 --2- 172.21.15.32:0/684444052 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f61f8004020 0x7f61f80049c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:35.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620d59a700 1 -- 172.21.15.32:0/684444052 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6210152b60 con 0x7f61f8003750 2020-12-05T17:31:35.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620d59a700 1 -- 172.21.15.32:0/684444052 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f8004f00 con 0x7f61f8004020 2020-12-05T17:31:35.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620d59a700 1 -- 172.21.15.32:0/684444052 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f80050c0 con 0x7f621005f3b0 2020-12-05T17:31:35.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620e59c700 1 --2- 172.21.15.32:0/684444052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61f8003750 0x7f61f8003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:35.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620dd9b700 1 --2- 172.21.15.32:0/684444052 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f621005f3b0 0x7f61f80044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:35.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620ed9d700 1 --2- 172.21.15.32:0/684444052 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f61f8004020 0x7f61f80049c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:35.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620e59c700 1 -- 172.21.15.32:0/684444052 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f621005f3b0 msgr2=0x7f61f80044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:35.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620e59c700 1 --2- 172.21.15.32:0/684444052 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f621005f3b0 0x7f61f80044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620e59c700 1 -- 172.21.15.32:0/684444052 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f61f8004020 msgr2=0x7f61f80049c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:35.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620e59c700 1 --2- 172.21.15.32:0/684444052 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f61f8004020 0x7f61f80049c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620e59c700 1 -- 172.21.15.32:0/684444052 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f8003230 con 0x7f61f8003750 2020-12-05T17:31:35.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620e59c700 1 --2- 172.21.15.32:0/684444052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61f8003750 0x7f61f8003b30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f620000d000 tx=0x7f6200006dc0).ready entity=mon.0 client_cookie=25e23263b32c256b server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:35.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620d59a700 1 -- 172.21.15.32:0/684444052 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f620000e940 con 0x7f61f8003750 2020-12-05T17:31:35.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620d59a700 1 -- 172.21.15.32:0/684444052 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620000c030 con 0x7f61f8003750 2020-12-05T17:31:35.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620d59a700 1 -- 172.21.15.32:0/684444052 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f620000ec90 con 0x7f61f8003750 2020-12-05T17:31:35.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620f59e700 1 -- 172.21.15.32:0/684444052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61f8003750 msgr2=0x7f61f8003b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:35.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620f59e700 1 --2- 172.21.15.32:0/684444052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61f8003750 0x7f61f8003b30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f620000d000 tx=0x7f6200006dc0).stop 2020-12-05T17:31:35.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.319+0000 7f620ed9d700 1 -- 172.21.15.32:0/684444052 reap_dead start 2020-12-05T17:31:35.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.323+0000 7f620f59e700 1 -- 172.21.15.32:0/684444052 shutdown_connections 2020-12-05T17:31:35.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.323+0000 7f620f59e700 1 -- 172.21.15.32:0/684444052 >> 172.21.15.32:0/684444052 conn(0x7f62100b4ad0 msgr2=0x7f62100b4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:35.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.323+0000 7f620f59e700 1 -- 172.21.15.32:0/684444052 shutdown_connections 2020-12-05T17:31:35.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.323+0000 7f620f59e700 1 -- 172.21.15.32:0/684444052 wait complete. 2020-12-05T17:31:35.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.323+0000 7f620f59e700 1 Processor -- start 2020-12-05T17:31:35.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.323+0000 7f620f59e700 1 -- start start 2020-12-05T17:31:35.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.323+0000 7f620f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62100afce0 0x7f6210064bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:35.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.323+0000 7f620f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f62100af1f0 0x7f62100664b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:35.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.323+0000 7f620f59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f62100669a0 0x7f621015de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:35.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.323+0000 7f620f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62100ae300 con 0x7f62100afce0 2020-12-05T17:31:35.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.323+0000 7f620f59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62100b0530 con 0x7f62100669a0 2020-12-05T17:31:35.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.323+0000 7f620f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6210066e20 con 0x7f62100af1f0 2020-12-05T17:31:35.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.327+0000 7f620e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62100afce0 0x7f6210064bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:35.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.327+0000 7f620dd9b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f62100af1f0 0x7f62100664b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:35.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.327+0000 7f620dd9b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f62100af1f0 0x7f62100664b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:57264/0 (socket says 172.21.15.32:57264) 2020-12-05T17:31:35.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.327+0000 7f620e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62100afce0 0x7f6210064bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60472/0 (socket says 172.21.15.32:60472) 2020-12-05T17:31:35.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.327+0000 7f620dd9b700 1 -- 172.21.15.32:0/1364696187 learned_addr learned my addr 172.21.15.32:0/1364696187 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:35.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.327+0000 7f620ed9d700 1 --2- 172.21.15.32:0/1364696187 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f62100669a0 0x7f621015de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:35.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.327+0000 7f620dd9b700 1 -- 172.21.15.32:0/1364696187 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f62100669a0 msgr2=0x7f621015de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:35.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.327+0000 7f620dd9b700 1 --2- 172.21.15.32:0/1364696187 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f62100669a0 0x7f621015de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.327+0000 7f620dd9b700 1 -- 172.21.15.32:0/1364696187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62100afce0 msgr2=0x7f6210064bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:35.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.327+0000 7f620dd9b700 1 --2- 172.21.15.32:0/1364696187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62100afce0 0x7f6210064bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.327+0000 7f620dd9b700 1 -- 172.21.15.32:0/1364696187 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62100ae850 con 0x7f62100af1f0 2020-12-05T17:31:35.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.327+0000 7f620e59c700 1 --2- 172.21.15.32:0/1364696187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62100afce0 0x7f6210064bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:31:35.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.327+0000 7f620dd9b700 1 --2- 172.21.15.32:0/1364696187 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f62100af1f0 0x7f62100664b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f620400d000 tx=0x7f6210061120).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:35.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.327+0000 7f61ff7fe700 1 -- 172.21.15.32:0/1364696187 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f620400fa90 con 0x7f62100af1f0 2020-12-05T17:31:35.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.327+0000 7f61ff7fe700 1 -- 172.21.15.32:0/1364696187 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620400fbf0 con 0x7f62100af1f0 2020-12-05T17:31:35.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.327+0000 7f61ff7fe700 1 -- 172.21.15.32:0/1364696187 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6204012350 con 0x7f62100af1f0 2020-12-05T17:31:35.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.327+0000 7f620f59e700 1 -- 172.21.15.32:0/1364696187 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62100b00c0 con 0x7f62100af1f0 2020-12-05T17:31:35.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.327+0000 7f620f59e700 1 -- 172.21.15.32:0/1364696187 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621015e4c0 con 0x7f62100af1f0 2020-12-05T17:31:35.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.327+0000 7f620f59e700 1 -- 172.21.15.32:0/1364696187 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f621015eab0 con 0x7f62100af1f0 2020-12-05T17:31:35.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.331+0000 7f61ff7fe700 1 -- 172.21.15.32:0/1364696187 <== mon.2 v2:172.21.15.61:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f62040306a0 con 0x7f62100af1f0 2020-12-05T17:31:35.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.331+0000 7f61ff7fe700 1 --2- 172.21.15.32:0/1364696187 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f61f40553b0 0x7f61f4057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:35.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.331+0000 7f61ff7fe700 1 -- 172.21.15.32:0/1364696187 <== mon.2 v2:172.21.15.61:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f620406d500 con 0x7f62100af1f0 2020-12-05T17:31:35.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.331+0000 7f620e59c700 1 --2- 172.21.15.32:0/1364696187 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f61f40553b0 0x7f61f4057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:35.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.331+0000 7f620e59c700 1 --2- 172.21.15.32:0/1364696187 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f61f40553b0 0x7f61f4057870 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f620001c000 tx=0x7f6200019000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:35.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.339+0000 7f61ff7fe700 1 -- 172.21.15.32:0/1364696187 <== mon.2 v2:172.21.15.61:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6204043b30 con 0x7f62100af1f0 2020-12-05T17:31:35.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.487+0000 7f620f59e700 1 -- 172.21.15.32:0/1364696187 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62100b06f0 con 0x7f62100af1f0 2020-12-05T17:31:35.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.491+0000 7f61ff7fe700 1 -- 172.21.15.32:0/1364696187 <== mon.2 v2:172.21.15.61:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13191 (secure 0 0 0) 0x7f6204019070 con 0x7f62100af1f0 2020-12-05T17:31:35.493 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:31:35.494 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:31:35.095963+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:31:35.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.491+0000 7f620f59e700 1 -- 172.21.15.32:0/1364696187 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f61f40553b0 msgr2=0x7f61f4057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:35.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.491+0000 7f620f59e700 1 --2- 172.21.15.32:0/1364696187 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f61f40553b0 0x7f61f4057870 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f620001c000 tx=0x7f6200019000).stop 2020-12-05T17:31:35.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.491+0000 7f620f59e700 1 -- 172.21.15.32:0/1364696187 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f62100af1f0 msgr2=0x7f62100664b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:35.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.491+0000 7f620f59e700 1 --2- 172.21.15.32:0/1364696187 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f62100af1f0 0x7f62100664b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f620400d000 tx=0x7f6210061120).stop 2020-12-05T17:31:35.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.491+0000 7f620f59e700 1 -- 172.21.15.32:0/1364696187 shutdown_connections 2020-12-05T17:31:35.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.491+0000 7f620f59e700 1 --2- 172.21.15.32:0/1364696187 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f62100669a0 0x7f621015de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.491+0000 7f620f59e700 1 --2- 172.21.15.32:0/1364696187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62100afce0 0x7f6210064bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.491+0000 7f620f59e700 1 --2- 172.21.15.32:0/1364696187 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f61f40553b0 0x7f61f4057870 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.491+0000 7f620f59e700 1 --2- 172.21.15.32:0/1364696187 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f62100af1f0 0x7f62100664b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.491+0000 7f620f59e700 1 -- 172.21.15.32:0/1364696187 >> 172.21.15.32:0/1364696187 conn(0x7f62100b4ad0 msgr2=0x7f62100b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:35.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.491+0000 7f620f59e700 1 -- 172.21.15.32:0/1364696187 shutdown_connections 2020-12-05T17:31:35.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.491+0000 7f620f59e700 1 -- 172.21.15.32:0/1364696187 wait complete. 2020-12-05T17:31:35.507 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-12-05T17:31:35.508 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:31:35.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.623+0000 7efec648e700 1 Processor -- start 2020-12-05T17:31:35.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.623+0000 7efec648e700 1 -- start start 2020-12-05T17:31:35.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.623+0000 7efec648e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efec00b6690 0x7efec00b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:35.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.623+0000 7efec648e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec00b56f0 con 0x7efec00b5b20 2020-12-05T17:31:35.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.623+0000 7efec648e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec00b1920 con 0x7efec00b5240 2020-12-05T17:31:35.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.623+0000 7efec648e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec00b1bc0 con 0x7efec00b6690 2020-12-05T17:31:35.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.623+0000 7efebffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efec00b5b20 0x7efec00b2060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34896/0 (socket says 172.21.15.32:34896) 2020-12-05T17:31:35.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.623+0000 7efebffff700 1 -- 172.21.15.32:0/3424056478 learned_addr learned my addr 172.21.15.32:0/3424056478 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:35.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.623+0000 7efebf7fe700 1 --2- 172.21.15.32:0/3424056478 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efec00b6690 0x7efec00b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:35.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.623+0000 7efebeffd700 1 -- 172.21.15.32:0/3424056478 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862172621 0 0) 0x7efec00b56f0 con 0x7efec00b5b20 2020-12-05T17:31:35.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.623+0000 7efebeffd700 1 -- 172.21.15.32:0/3424056478 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efea0003230 con 0x7efec00b5b20 2020-12-05T17:31:35.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.623+0000 7efebeffd700 1 -- 172.21.15.32:0/3424056478 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 849623645 0 0) 0x7efea0003230 con 0x7efec00b5b20 2020-12-05T17:31:35.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.623+0000 7efebeffd700 1 -- 172.21.15.32:0/3424056478 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efec00b6690 msgr2=0x7efec00b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:35.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.623+0000 7efebeffd700 1 --2- 172.21.15.32:0/3424056478 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efec00b6690 0x7efec00b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.623+0000 7efebeffd700 1 -- 172.21.15.32:0/3424056478 >> v1:172.21.15.61:6789/0 conn(0x7efec00b5240 legacy=0x7efec0145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:35.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.623+0000 7efebeffd700 1 -- 172.21.15.32:0/3424056478 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec0146620 con 0x7efec00b5b20 2020-12-05T17:31:35.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.623+0000 7efebeffd700 1 -- 172.21.15.32:0/3424056478 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7efeb00016d0 con 0x7efec00b5b20 2020-12-05T17:31:35.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efebeffd700 1 -- 172.21.15.32:0/3424056478 >> v1:172.21.15.32:6789/0 conn(0x7efec00b5b20 legacy=0x7efec00b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:35.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efebeffd700 1 --2- 172.21.15.32:0/3424056478 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efec00b6690 0x7efea0003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:35.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efebeffd700 1 --2- 172.21.15.32:0/3424056478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efea0003c40 0x7efea00044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:35.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efebeffd700 1 --2- 172.21.15.32:0/3424056478 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efea00049c0 0x7efea0004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:35.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efebeffd700 1 -- 172.21.15.32:0/3424056478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec00b1bc0 con 0x7efea0003c40 2020-12-05T17:31:35.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efebeffd700 1 -- 172.21.15.32:0/3424056478 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea0004160 con 0x7efea00049c0 2020-12-05T17:31:35.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efebeffd700 1 -- 172.21.15.32:0/3424056478 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea0004320 con 0x7efec00b6690 2020-12-05T17:31:35.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efebffff700 1 --2- 172.21.15.32:0/3424056478 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efec00b6690 0x7efea0003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:35.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efebf7fe700 1 --2- 172.21.15.32:0/3424056478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efea0003c40 0x7efea00044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:35.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efec4a2b700 1 --2- 172.21.15.32:0/3424056478 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efea00049c0 0x7efea0004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:35.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efebf7fe700 1 -- 172.21.15.32:0/3424056478 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efec00b6690 msgr2=0x7efea0003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:35.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efebf7fe700 1 --2- 172.21.15.32:0/3424056478 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efec00b6690 0x7efea0003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efebf7fe700 1 -- 172.21.15.32:0/3424056478 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efea00049c0 msgr2=0x7efea0004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:35.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efebf7fe700 1 --2- 172.21.15.32:0/3424056478 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efea00049c0 0x7efea0004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efebf7fe700 1 -- 172.21.15.32:0/3424056478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea0003230 con 0x7efea0003c40 2020-12-05T17:31:35.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efec4a2b700 1 --2- 172.21.15.32:0/3424056478 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efea00049c0 0x7efea0004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:31:35.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efebf7fe700 1 --2- 172.21.15.32:0/3424056478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efea0003c40 0x7efea00044d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7efea800c000 tx=0x7efea80091a0).ready entity=mon.0 client_cookie=92e0323cae85dd30 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:35.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efebeffd700 1 -- 172.21.15.32:0/3424056478 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efea800ebd0 con 0x7efea0003c40 2020-12-05T17:31:35.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efebeffd700 1 -- 172.21.15.32:0/3424056478 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea800ed30 con 0x7efea0003c40 2020-12-05T17:31:35.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efebeffd700 1 -- 172.21.15.32:0/3424056478 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efea800eea0 con 0x7efea0003c40 2020-12-05T17:31:35.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efec648e700 1 -- 172.21.15.32:0/3424056478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efea0003c40 msgr2=0x7efea00044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:35.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efec648e700 1 --2- 172.21.15.32:0/3424056478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efea0003c40 0x7efea00044d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7efea800c000 tx=0x7efea80091a0).stop 2020-12-05T17:31:35.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efec4a2b700 1 -- 172.21.15.32:0/3424056478 reap_dead start 2020-12-05T17:31:35.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efec648e700 1 -- 172.21.15.32:0/3424056478 shutdown_connections 2020-12-05T17:31:35.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efec648e700 1 -- 172.21.15.32:0/3424056478 >> 172.21.15.32:0/3424056478 conn(0x7efec005f3b0 msgr2=0x7efec005f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:35.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efec648e700 1 -- 172.21.15.32:0/3424056478 shutdown_connections 2020-12-05T17:31:35.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efec648e700 1 -- 172.21.15.32:0/3424056478 wait complete. 2020-12-05T17:31:35.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efec648e700 1 Processor -- start 2020-12-05T17:31:35.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.627+0000 7efec648e700 1 -- start start 2020-12-05T17:31:35.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efec648e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec00a3380 0x7efec0065710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:35.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efec648e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efec00a3760 0x7efec0065c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:35.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efec648e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efec00a28d0 0x7efec00644e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:35.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efec648e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec00a4220 con 0x7efec00a3380 2020-12-05T17:31:35.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efec648e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec0141100 con 0x7efec00a28d0 2020-12-05T17:31:35.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efec648e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec00663a0 con 0x7efec00a3760 2020-12-05T17:31:35.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efebffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec00a3380 0x7efec0065710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:35.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efebffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec00a3380 0x7efec0065710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60492/0 (socket says 172.21.15.32:60492) 2020-12-05T17:31:35.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efebffff700 1 -- 172.21.15.32:0/2016703772 learned_addr learned my addr 172.21.15.32:0/2016703772 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:35.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efebf7fe700 1 --2- 172.21.15.32:0/2016703772 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efec00a3760 0x7efec0065c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:35.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efec4a2b700 1 --2- 172.21.15.32:0/2016703772 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efec00a28d0 0x7efec00644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:35.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efebffff700 1 -- 172.21.15.32:0/2016703772 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efec00a3760 msgr2=0x7efec0065c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:35.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efebffff700 1 --2- 172.21.15.32:0/2016703772 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efec00a3760 0x7efec0065c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efebffff700 1 -- 172.21.15.32:0/2016703772 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efec00a28d0 msgr2=0x7efec00644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:35.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efebffff700 1 --2- 172.21.15.32:0/2016703772 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efec00a28d0 0x7efec00644e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efebffff700 1 -- 172.21.15.32:0/2016703772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec01522c0 con 0x7efec00a3380 2020-12-05T17:31:35.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efebf7fe700 1 --2- 172.21.15.32:0/2016703772 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efec00a3760 0x7efec0065c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:35.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efebffff700 1 --2- 172.21.15.32:0/2016703772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec00a3380 0x7efec0065710 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7efec00a50b0 tx=0x7efeb0007770).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:35.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efebd7fa700 1 -- 172.21.15.32:0/2016703772 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efeb000fc10 con 0x7efec00a3380 2020-12-05T17:31:35.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efebd7fa700 1 -- 172.21.15.32:0/2016703772 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb000fd70 con 0x7efec00a3380 2020-12-05T17:31:35.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efec648e700 1 -- 172.21.15.32:0/2016703772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec01527f0 con 0x7efec00a3380 2020-12-05T17:31:35.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efebd7fa700 1 -- 172.21.15.32:0/2016703772 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efeb000fee0 con 0x7efec00a3380 2020-12-05T17:31:35.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efec648e700 1 -- 172.21.15.32:0/2016703772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec00667d0 con 0x7efec00a3380 2020-12-05T17:31:35.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efec648e700 1 -- 172.21.15.32:0/2016703772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec0152100 con 0x7efec00a3380 2020-12-05T17:31:35.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efebd7fa700 1 -- 172.21.15.32:0/2016703772 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7efeb001b030 con 0x7efec00a3380 2020-12-05T17:31:35.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efebd7fa700 1 --2- 172.21.15.32:0/2016703772 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efea4055330 0x7efea40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:35.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efebf7fe700 1 --2- 172.21.15.32:0/2016703772 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efea4055330 0x7efea40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:35.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.631+0000 7efebd7fa700 1 -- 172.21.15.32:0/2016703772 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7efeb006dd40 con 0x7efec00a3380 2020-12-05T17:31:35.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.635+0000 7efebf7fe700 1 --2- 172.21.15.32:0/2016703772 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efea4055330 0x7efea40577f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7efea801b000 tx=0x7efea801a000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:35.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.639+0000 7efebd7fa700 1 -- 172.21.15.32:0/2016703772 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7efeb0044370 con 0x7efec00a3380 2020-12-05T17:31:35.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.783+0000 7efec648e700 1 -- 172.21.15.32:0/2016703772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7efec00664e0 con 0x7efec00a3380 2020-12-05T17:31:35.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.783+0000 7efebd7fa700 1 -- 172.21.15.32:0/2016703772 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v18) v1 ==== 110+0+10 (secure 0 0 0) 0x7efeb0019030 con 0x7efec00a3380 2020-12-05T17:31:35.787 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:31:35.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.783+0000 7efec648e700 1 -- 172.21.15.32:0/2016703772 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efea4055330 msgr2=0x7efea40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:35.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.783+0000 7efec648e700 1 --2- 172.21.15.32:0/2016703772 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efea4055330 0x7efea40577f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7efea801b000 tx=0x7efea801a000).stop 2020-12-05T17:31:35.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.783+0000 7efec648e700 1 -- 172.21.15.32:0/2016703772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec00a3380 msgr2=0x7efec0065710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:35.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.783+0000 7efec648e700 1 --2- 172.21.15.32:0/2016703772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec00a3380 0x7efec0065710 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7efec00a50b0 tx=0x7efeb0007770).stop 2020-12-05T17:31:35.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.787+0000 7efec648e700 1 -- 172.21.15.32:0/2016703772 shutdown_connections 2020-12-05T17:31:35.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.787+0000 7efec648e700 1 --2- 172.21.15.32:0/2016703772 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efec00a28d0 0x7efec00644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.787+0000 7efec648e700 1 --2- 172.21.15.32:0/2016703772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec00a3380 0x7efec0065710 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.787+0000 7efec648e700 1 --2- 172.21.15.32:0/2016703772 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efea4055330 0x7efea40577f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.787+0000 7efec648e700 1 --2- 172.21.15.32:0/2016703772 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efec00a3760 0x7efec0065c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.787+0000 7efec648e700 1 -- 172.21.15.32:0/2016703772 >> 172.21.15.32:0/2016703772 conn(0x7efec005f3b0 msgr2=0x7efec00b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:35.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.787+0000 7efec648e700 1 -- 172.21.15.32:0/2016703772 shutdown_connections 2020-12-05T17:31:35.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.787+0000 7efec648e700 1 -- 172.21.15.32:0/2016703772 wait complete. 2020-12-05T17:31:35.800 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:31:35.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.915+0000 7f9769a30700 1 Processor -- start 2020-12-05T17:31:35.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.915+0000 7f9769a30700 1 -- start start 2020-12-05T17:31:35.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.915+0000 7f9769a30700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f97640b3910 0x7f97640b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:35.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.915+0000 7f9769a30700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97641468d0 con 0x7f97640b42e0 2020-12-05T17:31:35.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.915+0000 7f9769a30700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9764146da0 con 0x7f97640b6970 2020-12-05T17:31:35.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.915+0000 7f9769a30700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97640b5e90 con 0x7f97640b3910 2020-12-05T17:31:35.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.915+0000 7f97627fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f97640b42e0 0x7f97640b2b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34920/0 (socket says 172.21.15.32:34920) 2020-12-05T17:31:35.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.915+0000 7f97627fc700 1 -- 172.21.15.32:0/3216419494 learned_addr learned my addr 172.21.15.32:0/3216419494 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:35.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.915+0000 7f9762ffd700 1 --2- 172.21.15.32:0/3216419494 >> v2:172.21.15.61:3300/0 conn(0x7f97640b3910 0x7f97640b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:35.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.915+0000 7f9761ffb700 1 -- 172.21.15.32:0/3216419494 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3122187711 0 0) 0x7f97641468d0 con 0x7f97640b42e0 2020-12-05T17:31:35.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.915+0000 7f9761ffb700 1 -- 172.21.15.32:0/3216419494 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9748003230 con 0x7f97640b42e0 2020-12-05T17:31:35.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.915+0000 7f9762ffd700 1 -- 172.21.15.32:0/3216419494 >> v1:172.21.15.61:6789/0 conn(0x7f97640b6970 legacy=0x7f97641428c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:35.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.915+0000 7f9761ffb700 1 -- 172.21.15.32:0/3216419494 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3996870670 0 0) 0x7f9748003230 con 0x7f97640b42e0 2020-12-05T17:31:35.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.915+0000 7f9762ffd700 1 -- 172.21.15.32:0/3216419494 >> v1:172.21.15.32:6789/0 conn(0x7f97640b42e0 legacy=0x7f97640b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:35.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.915+0000 7f9762ffd700 1 -- 172.21.15.32:0/3216419494 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9764146b00 con 0x7f97640b3910 2020-12-05T17:31:35.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.915+0000 7f9762ffd700 1 --2- 172.21.15.32:0/3216419494 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97640b3910 0x7f97640b3cf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f9758001870 tx=0x7f9754006dc0).ready entity=mon.1 client_cookie=eb4e1476f90e92e6 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:35.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.919+0000 7f9761ffb700 1 -- 172.21.15.32:0/3216419494 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97580016d0 con 0x7f97640b3910 2020-12-05T17:31:35.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.919+0000 7f9761ffb700 1 -- 172.21.15.32:0/3216419494 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9754009670 con 0x7f97640b3910 2020-12-05T17:31:35.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.919+0000 7f9761ffb700 1 -- 172.21.15.32:0/3216419494 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97540097e0 con 0x7f97640b3910 2020-12-05T17:31:35.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.919+0000 7f9769a30700 1 -- 172.21.15.32:0/3216419494 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97640b3910 msgr2=0x7f97640b3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:35.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.919+0000 7f9769a30700 1 --2- 172.21.15.32:0/3216419494 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97640b3910 0x7f97640b3cf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f9758001870 tx=0x7f9754006dc0).stop 2020-12-05T17:31:35.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.919+0000 7f9769a30700 1 -- 172.21.15.32:0/3216419494 shutdown_connections 2020-12-05T17:31:35.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.919+0000 7f9769a30700 1 --2- 172.21.15.32:0/3216419494 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97640b3910 0x7f97640b3cf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.919+0000 7f9769a30700 1 -- 172.21.15.32:0/3216419494 >> 172.21.15.32:0/3216419494 conn(0x7f976405f3b0 msgr2=0x7f976405f790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:35.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.919+0000 7f9769a30700 1 -- 172.21.15.32:0/3216419494 shutdown_connections 2020-12-05T17:31:35.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.919+0000 7f9769a30700 1 -- 172.21.15.32:0/3216419494 wait complete. 2020-12-05T17:31:35.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.919+0000 7f9769a30700 1 Processor -- start 2020-12-05T17:31:35.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.919+0000 7f9769a30700 1 -- start start 2020-12-05T17:31:35.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f9769a30700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97640b3910 0x7f9764138050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:35.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f9769a30700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97640b42e0 0x7f97640b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:35.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f9769a30700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97640b6970 0x7f97640b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:35.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f9769a30700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9764138da0 con 0x7f97640b3910 2020-12-05T17:31:35.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f9769a30700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9764139840 con 0x7f97640b6970 2020-12-05T17:31:35.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f9769a30700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9764145f10 con 0x7f97640b42e0 2020-12-05T17:31:35.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f9762ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97640b3910 0x7f9764138050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:35.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f9762ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97640b3910 0x7f9764138050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60506/0 (socket says 172.21.15.32:60506) 2020-12-05T17:31:35.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f9762ffd700 1 -- 172.21.15.32:0/3304736425 learned_addr learned my addr 172.21.15.32:0/3304736425 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:35.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f97637fe700 1 --2- 172.21.15.32:0/3304736425 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97640b6970 0x7f97640b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:35.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f97627fc700 1 --2- 172.21.15.32:0/3304736425 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97640b42e0 0x7f97640b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:35.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f9762ffd700 1 -- 172.21.15.32:0/3304736425 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97640b42e0 msgr2=0x7f97640b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:35.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f9762ffd700 1 --2- 172.21.15.32:0/3304736425 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97640b42e0 0x7f97640b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f9762ffd700 1 -- 172.21.15.32:0/3304736425 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97640b6970 msgr2=0x7f97640b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f9762ffd700 1 --2- 172.21.15.32:0/3304736425 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97640b6970 0x7f97640b0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f9762ffd700 1 -- 172.21.15.32:0/3304736425 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97640b1010 con 0x7f97640b3910 2020-12-05T17:31:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f9762ffd700 1 --2- 172.21.15.32:0/3304736425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97640b3910 0x7f9764138050 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f9758001870 tx=0x7f9754015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f9753fff700 1 -- 172.21.15.32:0/3304736425 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9754011b40 con 0x7f97640b3910 2020-12-05T17:31:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f9753fff700 1 -- 172.21.15.32:0/3304736425 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9754011ca0 con 0x7f97640b3910 2020-12-05T17:31:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f9769a30700 1 -- 172.21.15.32:0/3304736425 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9764145390 con 0x7f97640b3910 2020-12-05T17:31:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f9753fff700 1 -- 172.21.15.32:0/3304736425 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f975401d450 con 0x7f97640b3910 2020-12-05T17:31:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f9769a30700 1 -- 172.21.15.32:0/3304736425 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97640aed70 con 0x7f97640b3910 2020-12-05T17:31:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f9769a30700 1 -- 172.21.15.32:0/3304736425 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97641399d0 con 0x7f97640b3910 2020-12-05T17:31:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.923+0000 7f9753fff700 1 -- 172.21.15.32:0/3304736425 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9754024030 con 0x7f97640b3910 2020-12-05T17:31:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.927+0000 7f9753fff700 1 --2- 172.21.15.32:0/3304736425 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9740055330 0x7f97400577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.927+0000 7f9753fff700 1 -- 172.21.15.32:0/3304736425 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f975406ed70 con 0x7f97640b3910 2020-12-05T17:31:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.927+0000 7f97627fc700 1 --2- 172.21.15.32:0/3304736425 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9740055330 0x7f97400577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.927+0000 7f97627fc700 1 --2- 172.21.15.32:0/3304736425 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9740055330 0x7f97400577f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f975800d000 tx=0x7f9758000b80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:35.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:35.931+0000 7f9753fff700 1 -- 172.21.15.32:0/3304736425 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f97540453a0 con 0x7f97640b3910 2020-12-05T17:31:36.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.071+0000 7f9769a30700 1 -- 172.21.15.32:0/3304736425 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9764057050 con 0x7f97640b3910 2020-12-05T17:31:36.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.075+0000 7f9753fff700 1 -- 172.21.15.32:0/3304736425 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9754022070 con 0x7f97640b3910 2020-12-05T17:31:36.077 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:31:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.075+0000 7f9769a30700 1 -- 172.21.15.32:0/3304736425 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9740055330 msgr2=0x7f97400577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.075+0000 7f9769a30700 1 --2- 172.21.15.32:0/3304736425 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9740055330 0x7f97400577f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f975800d000 tx=0x7f9758000b80).stop 2020-12-05T17:31:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.075+0000 7f9769a30700 1 -- 172.21.15.32:0/3304736425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97640b3910 msgr2=0x7f9764138050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.075+0000 7f9769a30700 1 --2- 172.21.15.32:0/3304736425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97640b3910 0x7f9764138050 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f9758001870 tx=0x7f9754015000).stop 2020-12-05T17:31:36.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.075+0000 7f9769a30700 1 -- 172.21.15.32:0/3304736425 shutdown_connections 2020-12-05T17:31:36.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.075+0000 7f9769a30700 1 --2- 172.21.15.32:0/3304736425 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97640b6970 0x7f97640b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:36.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.075+0000 7f9769a30700 1 --2- 172.21.15.32:0/3304736425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97640b3910 0x7f9764138050 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:36.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.075+0000 7f9769a30700 1 --2- 172.21.15.32:0/3304736425 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9740055330 0x7f97400577f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:36.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.075+0000 7f9769a30700 1 --2- 172.21.15.32:0/3304736425 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97640b42e0 0x7f97640b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:36.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.075+0000 7f9769a30700 1 -- 172.21.15.32:0/3304736425 >> 172.21.15.32:0/3304736425 conn(0x7f976405f3b0 msgr2=0x7f976413b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:36.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.075+0000 7f9769a30700 1 -- 172.21.15.32:0/3304736425 shutdown_connections 2020-12-05T17:31:36.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.075+0000 7f9769a30700 1 -- 172.21.15.32:0/3304736425 wait complete. 2020-12-05T17:31:36.091 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-12-05T17:31:36.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.203+0000 7f79df306700 1 Processor -- start 2020-12-05T17:31:36.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.203+0000 7f79df306700 1 -- start start 2020-12-05T17:31:36.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.203+0000 7f79df306700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f79d80b11a0 0x7f79d81421e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:36.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.203+0000 7f79df306700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d80adfe0 con 0x7f79d80afd20 2020-12-05T17:31:36.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.203+0000 7f79df306700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d80b15e0 con 0x7f79d80b54b0 2020-12-05T17:31:36.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.203+0000 7f79df306700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d80b0630 con 0x7f79d80b11a0 2020-12-05T17:31:36.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.203+0000 7f79deb05700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f79d80b11a0 0x7f79d81421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:36.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.203+0000 7f79ddb03700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f79d80afd20 0x7f79d8141ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34932/0 (socket says 172.21.15.32:34932) 2020-12-05T17:31:36.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.203+0000 7f79ddb03700 1 -- 172.21.15.32:0/474532735 learned_addr learned my addr 172.21.15.32:0/474532735 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:36.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.203+0000 7f79dd302700 1 -- 172.21.15.32:0/474532735 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1771206412 0 0) 0x7f79d80adfe0 con 0x7f79d80afd20 2020-12-05T17:31:36.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.203+0000 7f79dd302700 1 -- 172.21.15.32:0/474532735 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79c0003230 con 0x7f79d80afd20 2020-12-05T17:31:36.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.203+0000 7f79deb05700 1 -- 172.21.15.32:0/474532735 >> v1:172.21.15.61:6789/0 conn(0x7f79d80b54b0 legacy=0x7f79d80b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:36.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.203+0000 7f79deb05700 1 -- 172.21.15.32:0/474532735 >> v1:172.21.15.32:6789/0 conn(0x7f79d80afd20 legacy=0x7f79d8141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:36.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.203+0000 7f79deb05700 1 -- 172.21.15.32:0/474532735 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d80ae210 con 0x7f79d80b11a0 2020-12-05T17:31:36.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.203+0000 7f79deb05700 1 --2- 172.21.15.32:0/474532735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d80b11a0 0x7f79d81421e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f79d0002f30 tx=0x7f79d0006dc0).ready entity=mon.0 client_cookie=505e9ed626a2c4e5 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:36.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.203+0000 7f79dd302700 1 -- 172.21.15.32:0/474532735 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79d0009800 con 0x7f79d80b11a0 2020-12-05T17:31:36.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.203+0000 7f79dd302700 1 -- 172.21.15.32:0/474532735 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d0009960 con 0x7f79d80b11a0 2020-12-05T17:31:36.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.203+0000 7f79dd302700 1 -- 172.21.15.32:0/474532735 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79d0009cc0 con 0x7f79d80b11a0 2020-12-05T17:31:36.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.203+0000 7f79df306700 1 -- 172.21.15.32:0/474532735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d80b11a0 msgr2=0x7f79d81421e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:36.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.203+0000 7f79df306700 1 --2- 172.21.15.32:0/474532735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d80b11a0 0x7f79d81421e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f79d0002f30 tx=0x7f79d0006dc0).stop 2020-12-05T17:31:36.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79df306700 1 -- 172.21.15.32:0/474532735 shutdown_connections 2020-12-05T17:31:36.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79df306700 1 --2- 172.21.15.32:0/474532735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d80b11a0 0x7f79d81421e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:36.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79df306700 1 -- 172.21.15.32:0/474532735 >> 172.21.15.32:0/474532735 conn(0x7f79d805bbb0 msgr2=0x7f79d8062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:36.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79df306700 1 -- 172.21.15.32:0/474532735 shutdown_connections 2020-12-05T17:31:36.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79df306700 1 -- 172.21.15.32:0/474532735 wait complete. 2020-12-05T17:31:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79df306700 1 Processor -- start 2020-12-05T17:31:36.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79df306700 1 -- start start 2020-12-05T17:31:36.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79df306700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f79d80afd20 0x7f79d8140a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:36.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79df306700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d80b11a0 0x7f79d810e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:36.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79df306700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f79d80b54b0 0x7f79d8155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:36.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79df306700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d8154a80 con 0x7f79d80b11a0 2020-12-05T17:31:36.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79df306700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d8111700 con 0x7f79d80afd20 2020-12-05T17:31:36.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79df306700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d8154cb0 con 0x7f79d80b54b0 2020-12-05T17:31:36.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79ddb03700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d80b11a0 0x7f79d810e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:36.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79ddb03700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d80b11a0 0x7f79d810e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60522/0 (socket says 172.21.15.32:60522) 2020-12-05T17:31:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79ddb03700 1 -- 172.21.15.32:0/2789358214 learned_addr learned my addr 172.21.15.32:0/2789358214 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79de304700 1 --2- 172.21.15.32:0/2789358214 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f79d80afd20 0x7f79d8140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79deb05700 1 --2- 172.21.15.32:0/2789358214 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f79d80b54b0 0x7f79d8155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79ddb03700 1 -- 172.21.15.32:0/2789358214 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f79d80b54b0 msgr2=0x7f79d8155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79ddb03700 1 --2- 172.21.15.32:0/2789358214 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f79d80b54b0 0x7f79d8155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:36.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79ddb03700 1 -- 172.21.15.32:0/2789358214 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f79d80afd20 msgr2=0x7f79d8140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:36.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79ddb03700 1 --2- 172.21.15.32:0/2789358214 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f79d80afd20 0x7f79d8140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:36.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79ddb03700 1 -- 172.21.15.32:0/2789358214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d8111400 con 0x7f79d80b11a0 2020-12-05T17:31:36.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79deb05700 1 --2- 172.21.15.32:0/2789358214 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f79d80b54b0 0x7f79d8155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:31:36.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79ddb03700 1 --2- 172.21.15.32:0/2789358214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d80b11a0 0x7f79d810e570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f79c80010d0 tx=0x7f79c8007af0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79c77fe700 1 -- 172.21.15.32:0/2789358214 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79c8007320 con 0x7f79d80b11a0 2020-12-05T17:31:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79c77fe700 1 -- 172.21.15.32:0/2789358214 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c8007480 con 0x7f79d80b11a0 2020-12-05T17:31:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79df306700 1 -- 172.21.15.32:0/2789358214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d8140f50 con 0x7f79d80b11a0 2020-12-05T17:31:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79c77fe700 1 -- 172.21.15.32:0/2789358214 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79c80057d0 con 0x7f79d80b11a0 2020-12-05T17:31:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.207+0000 7f79df306700 1 -- 172.21.15.32:0/2789358214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d81460d0 con 0x7f79d80b11a0 2020-12-05T17:31:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.211+0000 7f79df306700 1 -- 172.21.15.32:0/2789358214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d8155980 con 0x7f79d80b11a0 2020-12-05T17:31:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.211+0000 7f79c77fe700 1 -- 172.21.15.32:0/2789358214 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f79c8010030 con 0x7f79d80b11a0 2020-12-05T17:31:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.211+0000 7f79c77fe700 1 --2- 172.21.15.32:0/2789358214 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79b8055440 0x7f79b8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.211+0000 7f79c77fe700 1 -- 172.21.15.32:0/2789358214 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f79c806dac0 con 0x7f79d80b11a0 2020-12-05T17:31:36.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.211+0000 7f79de304700 1 --2- 172.21.15.32:0/2789358214 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79b8055440 0x7f79b8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:36.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.211+0000 7f79de304700 1 --2- 172.21.15.32:0/2789358214 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79b8055440 0x7f79b8057900 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f79d400b000 tx=0x7f79d4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:36.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.219+0000 7f79c77fe700 1 -- 172.21.15.32:0/2789358214 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f79c8043fe0 con 0x7f79d80b11a0 2020-12-05T17:31:36.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.359+0000 7f79df306700 1 -- 172.21.15.32:0/2789358214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f79d8154e70 con 0x7f79d80b11a0 2020-12-05T17:31:36.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.363+0000 7f79c77fe700 1 -- 172.21.15.32:0/2789358214 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+143 (secure 0 0 0) 0x7f79c8047600 con 0x7f79d80b11a0 2020-12-05T17:31:36.366 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:31:36.366 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2020-12-05T17:31:36.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:31:36.366 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2020-12-05T17:31:36.366 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2020-12-05T17:31:36.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:31:36.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:31:36.367 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2020-12-05T17:31:36.367 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2020-12-05T17:31:36.368 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:31:36.368 INFO:teuthology.orchestra.run.smithi032.stdout:] 2020-12-05T17:31:36.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.363+0000 7f79df306700 1 -- 172.21.15.32:0/2789358214 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79b8055440 msgr2=0x7f79b8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:36.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.363+0000 7f79df306700 1 --2- 172.21.15.32:0/2789358214 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79b8055440 0x7f79b8057900 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f79d400b000 tx=0x7f79d4009040).stop 2020-12-05T17:31:36.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.363+0000 7f79df306700 1 -- 172.21.15.32:0/2789358214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d80b11a0 msgr2=0x7f79d810e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:36.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.363+0000 7f79df306700 1 --2- 172.21.15.32:0/2789358214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d80b11a0 0x7f79d810e570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f79c80010d0 tx=0x7f79c8007af0).stop 2020-12-05T17:31:36.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.363+0000 7f79df306700 1 -- 172.21.15.32:0/2789358214 shutdown_connections 2020-12-05T17:31:36.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.363+0000 7f79df306700 1 --2- 172.21.15.32:0/2789358214 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79b8055440 0x7f79b8057900 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:36.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.363+0000 7f79df306700 1 --2- 172.21.15.32:0/2789358214 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f79d80b54b0 0x7f79d8155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:36.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.363+0000 7f79df306700 1 --2- 172.21.15.32:0/2789358214 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f79d80afd20 0x7f79d8140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:36.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.363+0000 7f79df306700 1 --2- 172.21.15.32:0/2789358214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d80b11a0 0x7f79d810e570 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:36.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.363+0000 7f79df306700 1 -- 172.21.15.32:0/2789358214 >> 172.21.15.32:0/2789358214 conn(0x7f79d805bbb0 msgr2=0x7f79d81421d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:36.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.363+0000 7f79df306700 1 -- 172.21.15.32:0/2789358214 shutdown_connections 2020-12-05T17:31:36.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.363+0000 7f79df306700 1 -- 172.21.15.32:0/2789358214 wait complete. 2020-12-05T17:31:36.379 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-12-05T17:31:36.379 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-12-05T17:31:36.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.491+0000 7fdacf0bf700 1 Processor -- start 2020-12-05T17:31:36.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.491+0000 7fdacf0bf700 1 -- start start 2020-12-05T17:31:36.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.491+0000 7fdacf0bf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdac81433d0 0x7fdac805f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.491+0000 7fdacf0bf700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac80b2df0 con 0x7fdac805f3b0 2020-12-05T17:31:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.491+0000 7fdacf0bf700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac80b3c90 con 0x7fdac805feb0 2020-12-05T17:31:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.491+0000 7fdacf0bf700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdac80b2b50 con 0x7fdac81433d0 2020-12-05T17:31:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.491+0000 7fdace8be700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdac81433d0 0x7fdac805f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.491+0000 7fdace8be700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdac81433d0 0x7fdac805f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60532/0 (socket says 172.21.15.32:60532) 2020-12-05T17:31:36.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.491+0000 7fdace0bd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdac805f3b0 0x7fdac813c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34944/0 (socket says 172.21.15.32:34944) 2020-12-05T17:31:36.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.491+0000 7fdace8be700 1 -- 172.21.15.32:0/3172072140 learned_addr learned my addr 172.21.15.32:0/3172072140 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:36.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdace8be700 1 -- 172.21.15.32:0/3172072140 >> v1:172.21.15.61:6789/0 conn(0x7fdac805feb0 legacy=0x7fdac8142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:36.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdace8be700 1 -- 172.21.15.32:0/3172072140 >> v1:172.21.15.32:6789/0 conn(0x7fdac805f3b0 legacy=0x7fdac813c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:36.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdace8be700 1 -- 172.21.15.32:0/3172072140 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac8142820 con 0x7fdac81433d0 2020-12-05T17:31:36.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdace8be700 1 --2- 172.21.15.32:0/3172072140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac81433d0 0x7fdac805f790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fdac00032b0 tx=0x7fdac0009270).ready entity=mon.0 client_cookie=58cb350ea6e56504 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:36.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdacd0bb700 1 -- 172.21.15.32:0/3172072140 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdac000bc00 con 0x7fdac81433d0 2020-12-05T17:31:36.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdacd0bb700 1 -- 172.21.15.32:0/3172072140 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac000bd60 con 0x7fdac81433d0 2020-12-05T17:31:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdacd0bb700 1 -- 172.21.15.32:0/3172072140 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdac0011400 con 0x7fdac81433d0 2020-12-05T17:31:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdacf0bf700 1 -- 172.21.15.32:0/3172072140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac81433d0 msgr2=0x7fdac805f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdacf0bf700 1 --2- 172.21.15.32:0/3172072140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac81433d0 0x7fdac805f790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fdac00032b0 tx=0x7fdac0009270).stop 2020-12-05T17:31:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdacf0bf700 1 -- 172.21.15.32:0/3172072140 shutdown_connections 2020-12-05T17:31:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdacf0bf700 1 --2- 172.21.15.32:0/3172072140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac81433d0 0x7fdac805f790 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:36.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdacf0bf700 1 -- 172.21.15.32:0/3172072140 >> 172.21.15.32:0/3172072140 conn(0x7fdac80b8f20 msgr2=0x7fdac80b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:36.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdacf0bf700 1 -- 172.21.15.32:0/3172072140 shutdown_connections 2020-12-05T17:31:36.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdacf0bf700 1 -- 172.21.15.32:0/3172072140 wait complete. 2020-12-05T17:31:36.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdacf0bf700 1 Processor -- start 2020-12-05T17:31:36.503 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdacf0bf700 1 -- start start 2020-12-05T17:31:36.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdacf0bf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac805f3b0 0x7fdac8145fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:36.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdacf0bf700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdac805feb0 0x7fdac8063a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:36.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdacf0bf700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdac81433d0 0x7fdac813b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:36.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdacf0bf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac8063f40 con 0x7fdac805f3b0 2020-12-05T17:31:36.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdacf0bf700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac8064730 con 0x7fdac805feb0 2020-12-05T17:31:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdacf0bf700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac8064c00 con 0x7fdac81433d0 2020-12-05T17:31:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdace0bd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac805f3b0 0x7fdac8145fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdace0bd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac805f3b0 0x7fdac8145fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60534/0 (socket says 172.21.15.32:60534) 2020-12-05T17:31:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdace0bd700 1 -- 172.21.15.32:0/2225885527 learned_addr learned my addr 172.21.15.32:0/2225885527 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdacd8bc700 1 --2- 172.21.15.32:0/2225885527 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdac805feb0 0x7fdac8063a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:36.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdace8be700 1 --2- 172.21.15.32:0/2225885527 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdac81433d0 0x7fdac813b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:36.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdace0bd700 1 -- 172.21.15.32:0/2225885527 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdac81433d0 msgr2=0x7fdac813b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:36.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdace0bd700 1 --2- 172.21.15.32:0/2225885527 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdac81433d0 0x7fdac813b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:36.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdace0bd700 1 -- 172.21.15.32:0/2225885527 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdac805feb0 msgr2=0x7fdac8063a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:36.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdace0bd700 1 --2- 172.21.15.32:0/2225885527 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdac805feb0 0x7fdac8063a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:36.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.495+0000 7fdace0bd700 1 -- 172.21.15.32:0/2225885527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac8064960 con 0x7fdac805f3b0 2020-12-05T17:31:36.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.499+0000 7fdace0bd700 1 --2- 172.21.15.32:0/2225885527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac805f3b0 0x7fdac8145fc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fdac8064080 tx=0x7fdabc007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:36.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.499+0000 7fdabb7fe700 1 -- 172.21.15.32:0/2225885527 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdabc0072e0 con 0x7fdac805f3b0 2020-12-05T17:31:36.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.499+0000 7fdabb7fe700 1 -- 172.21.15.32:0/2225885527 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdabc005520 con 0x7fdac805f3b0 2020-12-05T17:31:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.499+0000 7fdacf0bf700 1 -- 172.21.15.32:0/2225885527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac81516d0 con 0x7fdac805f3b0 2020-12-05T17:31:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.499+0000 7fdabb7fe700 1 -- 172.21.15.32:0/2225885527 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdabc005880 con 0x7fdac805f3b0 2020-12-05T17:31:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.499+0000 7fdacf0bf700 1 -- 172.21.15.32:0/2225885527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac8065140 con 0x7fdac805f3b0 2020-12-05T17:31:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.499+0000 7fdacf0bf700 1 -- 172.21.15.32:0/2225885527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac8057050 con 0x7fdac805f3b0 2020-12-05T17:31:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.499+0000 7fdabb7fe700 1 -- 172.21.15.32:0/2225885527 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fdabc01a060 con 0x7fdac805f3b0 2020-12-05T17:31:36.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.499+0000 7fdabb7fe700 1 --2- 172.21.15.32:0/2225885527 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdaa80553b0 0x7fdaa8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:36.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.499+0000 7fdabb7fe700 1 -- 172.21.15.32:0/2225885527 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fdabc06cfe0 con 0x7fdac805f3b0 2020-12-05T17:31:36.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.499+0000 7fdacd8bc700 1 --2- 172.21.15.32:0/2225885527 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdaa80553b0 0x7fdaa8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:36.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.499+0000 7fdacd8bc700 1 --2- 172.21.15.32:0/2225885527 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdaa80553b0 0x7fdaa8057870 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fdac400b000 tx=0x7fdac4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:36.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.507+0000 7fdabb7fe700 1 -- 172.21.15.32:0/2225885527 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fdabc043610 con 0x7fdac805f3b0 2020-12-05T17:31:36.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:36.651+0000 7fdacf0bf700 1 -- 172.21.15.32:0/2225885527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fdac813cb70 con 0x7fdac805f3b0 2020-12-05T17:31:37.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.239+0000 7fdabb7fe700 1 -- 172.21.15.32:0/2225885527 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v20) v1 ==== 133+0+0 (secure 0 0 0) 0x7fdabc018070 con 0x7fdac805f3b0 2020-12-05T17:31:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.243+0000 7fdacf0bf700 1 -- 172.21.15.32:0/2225885527 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdaa80553b0 msgr2=0x7fdaa8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.243+0000 7fdacf0bf700 1 --2- 172.21.15.32:0/2225885527 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdaa80553b0 0x7fdaa8057870 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fdac400b000 tx=0x7fdac4009040).stop 2020-12-05T17:31:37.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.243+0000 7fdacf0bf700 1 -- 172.21.15.32:0/2225885527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac805f3b0 msgr2=0x7fdac8145fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:37.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.243+0000 7fdacf0bf700 1 --2- 172.21.15.32:0/2225885527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac805f3b0 0x7fdac8145fc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fdac8064080 tx=0x7fdabc007bd0).stop 2020-12-05T17:31:37.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.247+0000 7fdacf0bf700 1 -- 172.21.15.32:0/2225885527 shutdown_connections 2020-12-05T17:31:37.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.247+0000 7fdacf0bf700 1 --2- 172.21.15.32:0/2225885527 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdaa80553b0 0x7fdaa8057870 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:37.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.247+0000 7fdacf0bf700 1 --2- 172.21.15.32:0/2225885527 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdac81433d0 0x7fdac813b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:37.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.247+0000 7fdacf0bf700 1 --2- 172.21.15.32:0/2225885527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac805f3b0 0x7fdac8145fc0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:37.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.247+0000 7fdacf0bf700 1 --2- 172.21.15.32:0/2225885527 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdac805feb0 0x7fdac8063a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:37.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.247+0000 7fdacf0bf700 1 -- 172.21.15.32:0/2225885527 >> 172.21.15.32:0/2225885527 conn(0x7fdac80b8f20 msgr2=0x7fdac80b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:37.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.247+0000 7fdacf0bf700 1 -- 172.21.15.32:0/2225885527 shutdown_connections 2020-12-05T17:31:37.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.247+0000 7fdacf0bf700 1 -- 172.21.15.32:0/2225885527 wait complete. 2020-12-05T17:31:37.252 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2020-12-05T17:31:37.264 INFO:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2020-12-05T17:31:37.265 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd erasure-code-profile set cephfs_data m=2 k=2 crush-failure-domain=osd 2020-12-05T17:31:37.385 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a759e700 1 Processor -- start 2020-12-05T17:31:37.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a759e700 1 -- start start 2020-12-05T17:31:37.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a759e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f04a805f3b0 0x7f04a805f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:37.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a759e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04a80b2fb0 0x7f04a8061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:37.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04a80b2df0 con 0x7f04a805feb0 2020-12-05T17:31:37.388 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a759e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a80b3c90 con 0x7f04a805f3b0 2020-12-05T17:31:37.388 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a759e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a80b2b50 con 0x7f04a80b2fb0 2020-12-05T17:31:37.388 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a5d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f04a805feb0 0x7f04a813c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34962/0 (socket says 172.21.15.32:34962) 2020-12-05T17:31:37.388 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a5d9b700 1 -- 172.21.15.32:0/858462485 learned_addr learned my addr 172.21.15.32:0/858462485 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:37.389 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a659c700 1 --2- 172.21.15.32:0/858462485 >> v2:172.21.15.61:3300/0 conn(0x7f04a805f3b0 0x7f04a805f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:37.389 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a6d9d700 1 --2- 172.21.15.32:0/858462485 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04a80b2fb0 0x7f04a8061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:37.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a559a700 1 -- 172.21.15.32:0/858462485 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1949773200 0 0) 0x7f04a80b2df0 con 0x7f04a805feb0 2020-12-05T17:31:37.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a559a700 1 -- 172.21.15.32:0/858462485 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0488003230 con 0x7f04a805feb0 2020-12-05T17:31:37.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a6d9d700 1 -- 172.21.15.32:0/858462485 >> v2:172.21.15.61:3300/0 conn(0x7f04a805f3b0 msgr2=0x7f04a805f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:37.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a6d9d700 1 --2- 172.21.15.32:0/858462485 >> v2:172.21.15.61:3300/0 conn(0x7f04a805f3b0 0x7f04a805f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:37.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a6d9d700 1 -- 172.21.15.32:0/858462485 >> v1:172.21.15.32:6789/0 conn(0x7f04a805feb0 legacy=0x7f04a813c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:37.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a6d9d700 1 -- 172.21.15.32:0/858462485 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a8142820 con 0x7f04a80b2fb0 2020-12-05T17:31:37.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a659c700 1 --2- 172.21.15.32:0/858462485 >> v2:172.21.15.61:3300/0 conn(0x7f04a805f3b0 0x7f04a805f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:31:37.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a6d9d700 1 --2- 172.21.15.32:0/858462485 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04a80b2fb0 0x7f04a8061780 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f04a0002f30 tx=0x7f04a0006dc0).ready entity=mon.2 client_cookie=74dd71589904cd33 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:37.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a559a700 1 -- 172.21.15.32:0/858462485 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04a00096a0 con 0x7f04a80b2fb0 2020-12-05T17:31:37.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a559a700 1 -- 172.21.15.32:0/858462485 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a0009800 con 0x7f04a80b2fb0 2020-12-05T17:31:37.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a559a700 1 -- 172.21.15.32:0/858462485 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04a0009b60 con 0x7f04a80b2fb0 2020-12-05T17:31:37.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a759e700 1 -- 172.21.15.32:0/858462485 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04a80b2fb0 msgr2=0x7f04a8061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:37.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.383+0000 7f04a759e700 1 --2- 172.21.15.32:0/858462485 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04a80b2fb0 0x7f04a8061780 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f04a0002f30 tx=0x7f04a0006dc0).stop 2020-12-05T17:31:37.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.387+0000 7f04a759e700 1 -- 172.21.15.32:0/858462485 shutdown_connections 2020-12-05T17:31:37.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.387+0000 7f04a759e700 1 --2- 172.21.15.32:0/858462485 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04a80b2fb0 0x7f04a8061780 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:37.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.387+0000 7f04a759e700 1 --2- 172.21.15.32:0/858462485 >> v2:172.21.15.61:3300/0 conn(0x7f04a805f3b0 0x7f04a805f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:37.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.387+0000 7f04a759e700 1 -- 172.21.15.32:0/858462485 >> 172.21.15.32:0/858462485 conn(0x7f04a80b8f20 msgr2=0x7f04a80b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:37.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.387+0000 7f04a759e700 1 -- 172.21.15.32:0/858462485 shutdown_connections 2020-12-05T17:31:37.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.387+0000 7f04a759e700 1 -- 172.21.15.32:0/858462485 wait complete. 2020-12-05T17:31:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.387+0000 7f04a759e700 1 Processor -- start 2020-12-05T17:31:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.387+0000 7f04a759e700 1 -- start start 2020-12-05T17:31:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.387+0000 7f04a759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a805f3b0 0x7f04a8140a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.387+0000 7f04a759e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04a805feb0 0x7f04a813b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.387+0000 7f04a759e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04a80b2fb0 0x7f04a813be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.387+0000 7f04a759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a81152f0 con 0x7f04a805f3b0 2020-12-05T17:31:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.387+0000 7f04a759e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a8112700 con 0x7f04a80b2fb0 2020-12-05T17:31:37.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.387+0000 7f04a759e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a81129a0 con 0x7f04a805feb0 2020-12-05T17:31:37.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.387+0000 7f04a659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a805f3b0 0x7f04a8140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:37.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.387+0000 7f04a659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a805f3b0 0x7f04a8140a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60548/0 (socket says 172.21.15.32:60548) 2020-12-05T17:31:37.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.387+0000 7f04a659c700 1 -- 172.21.15.32:0/2655780251 learned_addr learned my addr 172.21.15.32:0/2655780251 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:37.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.387+0000 7f04a6d9d700 1 --2- 172.21.15.32:0/2655780251 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04a80b2fb0 0x7f04a813be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:37.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.387+0000 7f04a5d9b700 1 --2- 172.21.15.32:0/2655780251 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04a805feb0 0x7f04a813b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:37.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.391+0000 7f04a659c700 1 -- 172.21.15.32:0/2655780251 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04a805feb0 msgr2=0x7f04a813b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:37.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.391+0000 7f04a659c700 1 --2- 172.21.15.32:0/2655780251 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04a805feb0 0x7f04a813b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:37.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.391+0000 7f04a659c700 1 -- 172.21.15.32:0/2655780251 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04a80b2fb0 msgr2=0x7f04a813be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.391+0000 7f04a659c700 1 --2- 172.21.15.32:0/2655780251 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04a80b2fb0 0x7f04a813be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.391+0000 7f04a659c700 1 -- 172.21.15.32:0/2655780251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a81122a0 con 0x7f04a805f3b0 2020-12-05T17:31:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.391+0000 7f04a659c700 1 --2- 172.21.15.32:0/2655780251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a805f3b0 0x7f04a8140a40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f049800c000 tx=0x7f04980091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.391+0000 7f049f7fe700 1 -- 172.21.15.32:0/2655780251 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f049800cbb0 con 0x7f04a805f3b0 2020-12-05T17:31:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.391+0000 7f049f7fe700 1 -- 172.21.15.32:0/2655780251 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049800cd10 con 0x7f04a805f3b0 2020-12-05T17:31:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.391+0000 7f049f7fe700 1 -- 172.21.15.32:0/2655780251 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04980125e0 con 0x7f04a805f3b0 2020-12-05T17:31:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.391+0000 7f04a759e700 1 -- 172.21.15.32:0/2655780251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a813c330 con 0x7f04a805f3b0 2020-12-05T17:31:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.391+0000 7f04a759e700 1 -- 172.21.15.32:0/2655780251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a8113060 con 0x7f04a805f3b0 2020-12-05T17:31:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.391+0000 7f04a759e700 1 -- 172.21.15.32:0/2655780251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a8057050 con 0x7f04a805f3b0 2020-12-05T17:31:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.391+0000 7f049f7fe700 1 -- 172.21.15.32:0/2655780251 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0498022070 con 0x7f04a805f3b0 2020-12-05T17:31:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.391+0000 7f049f7fe700 1 --2- 172.21.15.32:0/2655780251 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f048c055500 0x7f048c0579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.391+0000 7f04a5d9b700 1 --2- 172.21.15.32:0/2655780251 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f048c055500 0x7f048c0579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:37.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.391+0000 7f049f7fe700 1 -- 172.21.15.32:0/2655780251 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f049806e560 con 0x7f04a805f3b0 2020-12-05T17:31:37.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.395+0000 7f04a5d9b700 1 --2- 172.21.15.32:0/2655780251 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f048c055500 0x7f048c0579c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f049000d000 tx=0x7f0490009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:37.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.403+0000 7f049f7fe700 1 -- 172.21.15.32:0/2655780251 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0498044a30 con 0x7f04a805f3b0 2020-12-05T17:31:37.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:37.563+0000 7f04a759e700 1 -- 172.21.15.32:0/2655780251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f04a805aab0 con 0x7f04a805f3b0 2020-12-05T17:31:38.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.247+0000 7f049f7fe700 1 -- 172.21.15.32:0/2655780251 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 154+0+0 (secure 0 0 0) 0x7f0498048050 con 0x7f04a805f3b0 2020-12-05T17:31:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.255+0000 7f04a759e700 1 -- 172.21.15.32:0/2655780251 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f048c055500 msgr2=0x7f048c0579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.255+0000 7f04a759e700 1 --2- 172.21.15.32:0/2655780251 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f048c055500 0x7f048c0579c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f049000d000 tx=0x7f0490009040).stop 2020-12-05T17:31:38.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.255+0000 7f04a759e700 1 -- 172.21.15.32:0/2655780251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a805f3b0 msgr2=0x7f04a8140a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:38.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.255+0000 7f04a759e700 1 --2- 172.21.15.32:0/2655780251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a805f3b0 0x7f04a8140a40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f049800c000 tx=0x7f04980091a0).stop 2020-12-05T17:31:38.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.255+0000 7f04a759e700 1 -- 172.21.15.32:0/2655780251 shutdown_connections 2020-12-05T17:31:38.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.255+0000 7f04a759e700 1 --2- 172.21.15.32:0/2655780251 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04a80b2fb0 0x7f04a813be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:38.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.255+0000 7f04a759e700 1 --2- 172.21.15.32:0/2655780251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a805f3b0 0x7f04a8140a40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:38.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.255+0000 7f04a759e700 1 --2- 172.21.15.32:0/2655780251 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f048c055500 0x7f048c0579c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:38.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.255+0000 7f04a759e700 1 --2- 172.21.15.32:0/2655780251 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04a805feb0 0x7f04a813b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:38.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.255+0000 7f04a759e700 1 -- 172.21.15.32:0/2655780251 >> 172.21.15.32:0/2655780251 conn(0x7f04a80b8f20 msgr2=0x7f04a8142010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:38.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.255+0000 7f04a759e700 1 -- 172.21.15.32:0/2655780251 shutdown_connections 2020-12-05T17:31:38.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.255+0000 7f04a759e700 1 -- 172.21.15.32:0/2655780251 wait complete. 2020-12-05T17:31:38.276 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 erasure cephfs_data 2020-12-05T17:31:38.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.391+0000 7f4014478700 1 Processor -- start 2020-12-05T17:31:38.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.391+0000 7f4014478700 1 -- start start 2020-12-05T17:31:38.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.391+0000 7f4014478700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f400c0b5b20 0x7f400c0b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:38.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.391+0000 7f4014478700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f400c0b6690 0x7f400c0b5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:38.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.391+0000 7f4014478700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f400c0b2060 0x7f400c145f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:38.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4014478700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f400c0b5730 con 0x7f400c0b2060 2020-12-05T17:31:38.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4014478700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f400c0b1920 con 0x7f400c0b5b20 2020-12-05T17:31:38.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4014478700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400c0b1bc0 con 0x7f400c0b6690 2020-12-05T17:31:38.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4012a15700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f400c0b2060 0x7f400c145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:38.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4012a15700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f400c0b2060 0x7f400c145f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60560/0 (socket says 172.21.15.32:60560) 2020-12-05T17:31:38.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4012a15700 1 -- 172.21.15.32:0/3157609561 learned_addr learned my addr 172.21.15.32:0/3157609561 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:38.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4012214700 1 --2- 172.21.15.32:0/3157609561 >> v2:172.21.15.61:3300/0 conn(0x7f400c0b5b20 0x7f400c0b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:38.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4011a13700 1 --2- 172.21.15.32:0/3157609561 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f400c0b6690 0x7f400c0b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:38.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4012a15700 1 -- 172.21.15.32:0/3157609561 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f400c0b6690 msgr2=0x7f400c0b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:38.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4012a15700 1 --2- 172.21.15.32:0/3157609561 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f400c0b6690 0x7f400c0b5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:38.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4012a15700 1 -- 172.21.15.32:0/3157609561 >> v2:172.21.15.61:3300/0 conn(0x7f400c0b5b20 msgr2=0x7f400c0b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:38.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4012a15700 1 --2- 172.21.15.32:0/3157609561 >> v2:172.21.15.61:3300/0 conn(0x7f400c0b5b20 0x7f400c0b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:38.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4012a15700 1 -- 172.21.15.32:0/3157609561 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400c0b24c0 con 0x7f400c0b2060 2020-12-05T17:31:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4012a15700 1 --2- 172.21.15.32:0/3157609561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f400c0b2060 0x7f400c145f10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f40040032b0 tx=0x7f4004009270).ready entity=mon.0 client_cookie=32d3c5ae8f602760 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4011212700 1 -- 172.21.15.32:0/3157609561 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4004009e60 con 0x7f400c0b2060 2020-12-05T17:31:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4011212700 1 -- 172.21.15.32:0/3157609561 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400400bb50 con 0x7f400c0b2060 2020-12-05T17:31:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4011212700 1 -- 172.21.15.32:0/3157609561 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f400400bcc0 con 0x7f400c0b2060 2020-12-05T17:31:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4014478700 1 -- 172.21.15.32:0/3157609561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f400c0b2060 msgr2=0x7f400c145f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:38.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4014478700 1 --2- 172.21.15.32:0/3157609561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f400c0b2060 0x7f400c145f10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f40040032b0 tx=0x7f4004009270).stop 2020-12-05T17:31:38.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4014478700 1 -- 172.21.15.32:0/3157609561 shutdown_connections 2020-12-05T17:31:38.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4014478700 1 --2- 172.21.15.32:0/3157609561 >> v2:172.21.15.61:3300/0 conn(0x7f400c0b5b20 0x7f400c0b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:38.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4014478700 1 --2- 172.21.15.32:0/3157609561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f400c0b2060 0x7f400c145f10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:38.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4014478700 1 --2- 172.21.15.32:0/3157609561 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f400c0b6690 0x7f400c0b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:38.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4014478700 1 -- 172.21.15.32:0/3157609561 >> 172.21.15.32:0/3157609561 conn(0x7f400c05f3b0 msgr2=0x7f400c05f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:38.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4014478700 1 -- 172.21.15.32:0/3157609561 shutdown_connections 2020-12-05T17:31:38.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4014478700 1 -- 172.21.15.32:0/3157609561 wait complete. 2020-12-05T17:31:38.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4014478700 1 Processor -- start 2020-12-05T17:31:38.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.395+0000 7f4014478700 1 -- start start 2020-12-05T17:31:38.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f4014478700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f400c0b2060 0x7f400c0665d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:38.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f4014478700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f400c0b5b20 0x7f400c148550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:38.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f4014478700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f400c0b6690 0x7f400c13ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:38.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f4014478700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c1450d0 con 0x7f400c0b2060 2020-12-05T17:31:38.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f4014478700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c1444d0 con 0x7f400c0b6690 2020-12-05T17:31:38.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f4014478700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400c144770 con 0x7f400c0b5b20 2020-12-05T17:31:38.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f4012214700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f400c0b2060 0x7f400c0665d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:38.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f4012214700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f400c0b2060 0x7f400c0665d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60562/0 (socket says 172.21.15.32:60562) 2020-12-05T17:31:38.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f4012214700 1 -- 172.21.15.32:0/272833323 learned_addr learned my addr 172.21.15.32:0/272833323 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:38.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f4011a13700 1 --2- 172.21.15.32:0/272833323 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f400c0b5b20 0x7f400c148550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:38.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f4012a15700 1 --2- 172.21.15.32:0/272833323 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f400c0b6690 0x7f400c13ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:38.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f4012214700 1 -- 172.21.15.32:0/272833323 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f400c0b5b20 msgr2=0x7f400c148550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:38.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f4012214700 1 --2- 172.21.15.32:0/272833323 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f400c0b5b20 0x7f400c148550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:38.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f4012214700 1 -- 172.21.15.32:0/272833323 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f400c0b6690 msgr2=0x7f400c13ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:38.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f4012214700 1 --2- 172.21.15.32:0/272833323 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f400c0b6690 0x7f400c13ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:38.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f4012214700 1 -- 172.21.15.32:0/272833323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400c145300 con 0x7f400c0b2060 2020-12-05T17:31:38.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f4012a15700 1 --2- 172.21.15.32:0/272833323 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f400c0b6690 0x7f400c13ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:38.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f4012214700 1 --2- 172.21.15.32:0/272833323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f400c0b2060 0x7f400c0665d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f400800c000 tx=0x7f4008006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:38.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f3ffb7fe700 1 -- 172.21.15.32:0/272833323 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f400800ca80 con 0x7f400c0b2060 2020-12-05T17:31:38.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f3ffb7fe700 1 -- 172.21.15.32:0/272833323 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400800cbe0 con 0x7f400c0b2060 2020-12-05T17:31:38.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f4014478700 1 -- 172.21.15.32:0/272833323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f400c066110 con 0x7f400c0b2060 2020-12-05T17:31:38.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f3ffb7fe700 1 -- 172.21.15.32:0/272833323 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f400800b370 con 0x7f400c0b2060 2020-12-05T17:31:38.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f4014478700 1 -- 172.21.15.32:0/272833323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400c144d80 con 0x7f400c0b2060 2020-12-05T17:31:38.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.399+0000 7f3ffb7fe700 1 -- 172.21.15.32:0/272833323 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f400801c460 con 0x7f400c0b2060 2020-12-05T17:31:38.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.403+0000 7f4014478700 1 -- 172.21.15.32:0/272833323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f400c066440 con 0x7f400c0b2060 2020-12-05T17:31:38.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.403+0000 7f3ffb7fe700 1 --2- 172.21.15.32:0/272833323 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3fec055470 0x7f3fec057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:38.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.403+0000 7f3ffb7fe700 1 -- 172.21.15.32:0/272833323 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5863+0+0 (secure 0 0 0) 0x7f400806e350 con 0x7f400c0b2060 2020-12-05T17:31:38.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.403+0000 7f4011a13700 1 --2- 172.21.15.32:0/272833323 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3fec055470 0x7f3fec057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:38.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.403+0000 7f4011a13700 1 --2- 172.21.15.32:0/272833323 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3fec055470 0x7f3fec057930 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f3ffc00b000 tx=0x7f3ffc006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:38.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.411+0000 7f3ffb7fe700 1 -- 172.21.15.32:0/272833323 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f40080446c0 con 0x7f400c0b2060 2020-12-05T17:31:38.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:38.551+0000 7f4014478700 1 -- 172.21.15.32:0/272833323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8, "pool_type": "erasure", "erasure_code_profile": "cephfs_data"} v 0) v1 -- 0x7f400c057050 con 0x7f400c0b2060 2020-12-05T17:31:40.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.263+0000 7f3ffb7fe700 1 -- 172.21.15.32:0/272833323 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8, "pool_type": "erasure", "erasure_code_profile": "cephfs_data"}]=0 pool 'cephfs_data' created v23) v1 ==== 188+0+0 (secure 0 0 0) 0x7f4008047ce0 con 0x7f400c0b2060 2020-12-05T17:31:40.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.267+0000 7f4014478700 1 -- 172.21.15.32:0/272833323 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3fec055470 msgr2=0x7f3fec057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:40.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.267+0000 7f4014478700 1 --2- 172.21.15.32:0/272833323 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3fec055470 0x7f3fec057930 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f3ffc00b000 tx=0x7f3ffc006dc0).stop 2020-12-05T17:31:40.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.267+0000 7f4014478700 1 -- 172.21.15.32:0/272833323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f400c0b2060 msgr2=0x7f400c0665d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:40.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.267+0000 7f4014478700 1 --2- 172.21.15.32:0/272833323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f400c0b2060 0x7f400c0665d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f400800c000 tx=0x7f4008006dc0).stop 2020-12-05T17:31:40.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.271+0000 7f4014478700 1 -- 172.21.15.32:0/272833323 shutdown_connections 2020-12-05T17:31:40.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.271+0000 7f4014478700 1 --2- 172.21.15.32:0/272833323 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f400c0b6690 0x7f400c13ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:40.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.271+0000 7f4014478700 1 --2- 172.21.15.32:0/272833323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f400c0b2060 0x7f400c0665d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:40.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.271+0000 7f4014478700 1 --2- 172.21.15.32:0/272833323 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3fec055470 0x7f3fec057930 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:40.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.271+0000 7f4014478700 1 --2- 172.21.15.32:0/272833323 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f400c0b5b20 0x7f400c148550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:40.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.271+0000 7f4014478700 1 -- 172.21.15.32:0/272833323 >> 172.21.15.32:0/272833323 conn(0x7f400c05f3b0 msgr2=0x7f400c05f790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:40.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.271+0000 7f4014478700 1 -- 172.21.15.32:0/272833323 shutdown_connections 2020-12-05T17:31:40.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.271+0000 7f4014478700 1 -- 172.21.15.32:0/272833323 wait complete. 2020-12-05T17:31:40.281 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2020-12-05T17:31:40.290 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool set cephfs_data allow_ec_overwrites true 2020-12-05T17:31:40.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.423+0000 7f3124aec700 1 Processor -- start 2020-12-05T17:31:40.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.423+0000 7f3124aec700 1 -- start start 2020-12-05T17:31:40.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f3124aec700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f312005feb0 0x7f312005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:40.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f3124aec700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31200b2fb0 0x7f3120061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:40.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f3124aec700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31200b2df0 con 0x7f312005f3b0 2020-12-05T17:31:40.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f3124aec700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31200b3c90 con 0x7f312005feb0 2020-12-05T17:31:40.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f3124aec700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31200b2b50 con 0x7f31200b2fb0 2020-12-05T17:31:40.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f311f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f312005f3b0 0x7f312013c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34986/0 (socket says 172.21.15.32:34986) 2020-12-05T17:31:40.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f311f7fe700 1 -- 172.21.15.32:0/3352103289 learned_addr learned my addr 172.21.15.32:0/3352103289 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f311effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f312005feb0 0x7f312005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f311ffff700 1 --2- 172.21.15.32:0/3352103289 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31200b2fb0 0x7f3120061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f311e7fc700 1 -- 172.21.15.32:0/3352103289 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3544248143 0 0) 0x7f31200b2df0 con 0x7f312005f3b0 2020-12-05T17:31:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f311e7fc700 1 -- 172.21.15.32:0/3352103289 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3100003230 con 0x7f312005f3b0 2020-12-05T17:31:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f311effd700 1 -- 172.21.15.32:0/3352103289 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31200b2fb0 msgr2=0x7f3120061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f311effd700 1 --2- 172.21.15.32:0/3352103289 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31200b2fb0 0x7f3120061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f311effd700 1 -- 172.21.15.32:0/3352103289 >> v1:172.21.15.32:6789/0 conn(0x7f312005f3b0 legacy=0x7f312013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f311effd700 1 -- 172.21.15.32:0/3352103289 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3120142820 con 0x7f312005feb0 2020-12-05T17:31:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f311effd700 1 --2- 172.21.15.32:0/3352103289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f312005feb0 0x7f312005f790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f3108002f30 tx=0x7f3108006dc0).ready entity=mon.0 client_cookie=12dfa24864b05778 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f311e7fc700 1 -- 172.21.15.32:0/3352103289 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31080096a0 con 0x7f312005feb0 2020-12-05T17:31:40.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f311e7fc700 1 -- 172.21.15.32:0/3352103289 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3108009800 con 0x7f312005feb0 2020-12-05T17:31:40.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f311e7fc700 1 -- 172.21.15.32:0/3352103289 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3108009b60 con 0x7f312005feb0 2020-12-05T17:31:40.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f3124aec700 1 -- 172.21.15.32:0/3352103289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f312005feb0 msgr2=0x7f312005f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:40.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f3124aec700 1 --2- 172.21.15.32:0/3352103289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f312005feb0 0x7f312005f790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f3108002f30 tx=0x7f3108006dc0).stop 2020-12-05T17:31:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f3124aec700 1 -- 172.21.15.32:0/3352103289 shutdown_connections 2020-12-05T17:31:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f3124aec700 1 --2- 172.21.15.32:0/3352103289 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31200b2fb0 0x7f3120061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f3124aec700 1 --2- 172.21.15.32:0/3352103289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f312005feb0 0x7f312005f790 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f3124aec700 1 -- 172.21.15.32:0/3352103289 >> 172.21.15.32:0/3352103289 conn(0x7f31200b8f20 msgr2=0x7f31200b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f3124aec700 1 -- 172.21.15.32:0/3352103289 shutdown_connections 2020-12-05T17:31:40.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.427+0000 7f3124aec700 1 -- 172.21.15.32:0/3352103289 wait complete. 2020-12-05T17:31:40.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f3124aec700 1 Processor -- start 2020-12-05T17:31:40.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f3124aec700 1 -- start start 2020-12-05T17:31:40.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f3124aec700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f312005f3b0 0x7f312013e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:40.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f3124aec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f312005feb0 0x7f312013bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:40.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f3124aec700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31200b2fb0 0x7f312013bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:40.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f3124aec700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3120112110 con 0x7f312005feb0 2020-12-05T17:31:40.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f3124aec700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3120141870 con 0x7f312005f3b0 2020-12-05T17:31:40.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f3124aec700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3120112340 con 0x7f31200b2fb0 2020-12-05T17:31:40.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f311effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f312005feb0 0x7f312013bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:40.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f311effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f312005feb0 0x7f312013bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60578/0 (socket says 172.21.15.32:60578) 2020-12-05T17:31:40.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f311effd700 1 -- 172.21.15.32:0/2300826087 learned_addr learned my addr 172.21.15.32:0/2300826087 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:40.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f311ffff700 1 --2- 172.21.15.32:0/2300826087 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31200b2fb0 0x7f312013bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:40.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f311f7fe700 1 --2- 172.21.15.32:0/2300826087 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f312005f3b0 0x7f312013e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:40.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f311effd700 1 -- 172.21.15.32:0/2300826087 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31200b2fb0 msgr2=0x7f312013bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:40.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f311effd700 1 --2- 172.21.15.32:0/2300826087 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31200b2fb0 0x7f312013bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:40.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f311effd700 1 -- 172.21.15.32:0/2300826087 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f312005f3b0 msgr2=0x7f312013e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:40.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f311effd700 1 --2- 172.21.15.32:0/2300826087 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f312005f3b0 0x7f312013e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:40.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f311effd700 1 -- 172.21.15.32:0/2300826087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3120141fd0 con 0x7f312005feb0 2020-12-05T17:31:40.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f311f7fe700 1 --2- 172.21.15.32:0/2300826087 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f312005f3b0 0x7f312013e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:40.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f311effd700 1 --2- 172.21.15.32:0/2300826087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f312005feb0 0x7f312013bb00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f3108002f30 tx=0x7f3108015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:40.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f311cff9700 1 -- 172.21.15.32:0/2300826087 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3108011a90 con 0x7f312005feb0 2020-12-05T17:31:40.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f311cff9700 1 -- 172.21.15.32:0/2300826087 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3108011bf0 con 0x7f312005feb0 2020-12-05T17:31:40.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f3124aec700 1 -- 172.21.15.32:0/2300826087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f312013c4e0 con 0x7f312005feb0 2020-12-05T17:31:40.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f311cff9700 1 -- 172.21.15.32:0/2300826087 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f310801d450 con 0x7f312005feb0 2020-12-05T17:31:40.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.431+0000 7f3124aec700 1 -- 172.21.15.32:0/2300826087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312013f9e0 con 0x7f312005feb0 2020-12-05T17:31:40.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.435+0000 7f3124aec700 1 -- 172.21.15.32:0/2300826087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3120057050 con 0x7f312005feb0 2020-12-05T17:31:40.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.435+0000 7f311cff9700 1 -- 172.21.15.32:0/2300826087 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3108024030 con 0x7f312005feb0 2020-12-05T17:31:40.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.435+0000 7f311cff9700 1 --2- 172.21.15.32:0/2300826087 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f31040553b0 0x7f3104057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.435+0000 7f311cff9700 1 -- 172.21.15.32:0/2300826087 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6315+0+0 (secure 0 0 0) 0x7f310806f680 con 0x7f312005feb0 2020-12-05T17:31:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.435+0000 7f311f7fe700 1 --2- 172.21.15.32:0/2300826087 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f31040553b0 0x7f3104057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.435+0000 7f311f7fe700 1 --2- 172.21.15.32:0/2300826087 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f31040553b0 0x7f3104057870 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f311000b000 tx=0x7f3110009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.443+0000 7f311cff9700 1 -- 172.21.15.32:0/2300826087 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3108045780 con 0x7f312005feb0 2020-12-05T17:31:40.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:40.587+0000 7f3124aec700 1 -- 172.21.15.32:0/2300826087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f312005aab0 con 0x7f312005feb0 2020-12-05T17:31:41.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.263+0000 7f311cff9700 1 -- 172.21.15.32:0/2300826087 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) v1 ==== 166+0+0 (secure 0 0 0) 0x7f3108022070 con 0x7f312005feb0 2020-12-05T17:31:41.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.271+0000 7f3124aec700 1 -- 172.21.15.32:0/2300826087 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f31040553b0 msgr2=0x7f3104057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:41.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.271+0000 7f3124aec700 1 --2- 172.21.15.32:0/2300826087 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f31040553b0 0x7f3104057870 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f311000b000 tx=0x7f3110009040).stop 2020-12-05T17:31:41.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.271+0000 7f3124aec700 1 -- 172.21.15.32:0/2300826087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f312005feb0 msgr2=0x7f312013bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:41.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.271+0000 7f3124aec700 1 --2- 172.21.15.32:0/2300826087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f312005feb0 0x7f312013bb00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f3108002f30 tx=0x7f3108015000).stop 2020-12-05T17:31:41.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.275+0000 7f3124aec700 1 -- 172.21.15.32:0/2300826087 shutdown_connections 2020-12-05T17:31:41.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.275+0000 7f3124aec700 1 --2- 172.21.15.32:0/2300826087 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f31040553b0 0x7f3104057870 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.275+0000 7f3124aec700 1 --2- 172.21.15.32:0/2300826087 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31200b2fb0 0x7f312013bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.275+0000 7f3124aec700 1 --2- 172.21.15.32:0/2300826087 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f312005f3b0 0x7f312013e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.275+0000 7f3124aec700 1 --2- 172.21.15.32:0/2300826087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f312005feb0 0x7f312013bb00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.275+0000 7f3124aec700 1 -- 172.21.15.32:0/2300826087 >> 172.21.15.32:0/2300826087 conn(0x7f31200b8f20 msgr2=0x7f31200b78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:41.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.279+0000 7f3124aec700 1 -- 172.21.15.32:0/2300826087 shutdown_connections 2020-12-05T17:31:41.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.279+0000 7f3124aec700 1 -- 172.21.15.32:0/2300826087 wait complete. 2020-12-05T17:31:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:set pool 4 allow_ec_overwrites to true 2020-12-05T17:31:41.299 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-12-05T17:31:41.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.407+0000 7f2f8b384700 1 Processor -- start 2020-12-05T17:31:41.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.407+0000 7f2f8b384700 1 -- start start 2020-12-05T17:31:41.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f8b384700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f84061c50 0x7f2f840ac660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:41.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f8b384700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f2f84062030 0x7f2f840ab6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:41.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f8b384700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f8413fa30 con 0x7f2f840abbb0 2020-12-05T17:31:41.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f8b384700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f840acb50 con 0x7f2f84062030 2020-12-05T17:31:41.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f8b384700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f840aeea0 con 0x7f2f84061c50 2020-12-05T17:31:41.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f89921700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2f840abbb0 0x7f2f84142430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35004/0 (socket says 172.21.15.32:35004) 2020-12-05T17:31:41.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f89921700 1 -- 172.21.15.32:0/2403095336 learned_addr learned my addr 172.21.15.32:0/2403095336 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:41.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f8891f700 1 --2- 172.21.15.32:0/2403095336 >> v2:172.21.15.61:3300/0 conn(0x7f2f84062030 0x7f2f840ab6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:41.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f89120700 1 --2- 172.21.15.32:0/2403095336 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f84061c50 0x7f2f840ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:41.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f77fff700 1 -- 172.21.15.32:0/2403095336 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3720376392 0 0) 0x7f2f8413fa30 con 0x7f2f840abbb0 2020-12-05T17:31:41.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f77fff700 1 -- 172.21.15.32:0/2403095336 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f68003230 con 0x7f2f840abbb0 2020-12-05T17:31:41.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f77fff700 1 -- 172.21.15.32:0/2403095336 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2269872825 0 0) 0x7f2f68003230 con 0x7f2f840abbb0 2020-12-05T17:31:41.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f77fff700 1 -- 172.21.15.32:0/2403095336 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f84061c50 msgr2=0x7f2f840ac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:41.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f77fff700 1 --2- 172.21.15.32:0/2403095336 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f84061c50 0x7f2f840ac660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f77fff700 1 -- 172.21.15.32:0/2403095336 >> v2:172.21.15.61:3300/0 conn(0x7f2f84062030 msgr2=0x7f2f840ab6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:41.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f77fff700 1 --2- 172.21.15.32:0/2403095336 >> v2:172.21.15.61:3300/0 conn(0x7f2f84062030 0x7f2f840ab6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f77fff700 1 -- 172.21.15.32:0/2403095336 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8413fcd0 con 0x7f2f840abbb0 2020-12-05T17:31:41.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f77fff700 1 -- 172.21.15.32:0/2403095336 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f2f7c001a50 con 0x7f2f840abbb0 2020-12-05T17:31:41.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f89120700 1 --2- 172.21.15.32:0/2403095336 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f84061c50 0x7f2f840ac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:31:41.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f77fff700 1 -- 172.21.15.32:0/2403095336 >> v1:172.21.15.32:6789/0 conn(0x7f2f840abbb0 legacy=0x7f2f84142430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:41.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f77fff700 1 --2- 172.21.15.32:0/2403095336 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2f68003720 0x7f2f68003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:41.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f77fff700 1 --2- 172.21.15.32:0/2403095336 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f84061c50 0x7f2f680044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:41.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f77fff700 1 --2- 172.21.15.32:0/2403095336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f680049c0 0x7f2f68004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:41.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f77fff700 1 -- 172.21.15.32:0/2403095336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f840aeea0 con 0x7f2f680049c0 2020-12-05T17:31:41.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f77fff700 1 -- 172.21.15.32:0/2403095336 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f680041a0 con 0x7f2f68003720 2020-12-05T17:31:41.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f77fff700 1 -- 172.21.15.32:0/2403095336 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6800b720 con 0x7f2f84061c50 2020-12-05T17:31:41.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f89921700 1 --2- 172.21.15.32:0/2403095336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f680049c0 0x7f2f68004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:41.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f8891f700 1 --2- 172.21.15.32:0/2403095336 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f84061c50 0x7f2f680044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:41.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f89120700 1 --2- 172.21.15.32:0/2403095336 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2f68003720 0x7f2f68003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:41.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f89921700 1 -- 172.21.15.32:0/2403095336 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f84061c50 msgr2=0x7f2f680044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:41.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f89921700 1 --2- 172.21.15.32:0/2403095336 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f84061c50 0x7f2f680044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f89921700 1 -- 172.21.15.32:0/2403095336 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2f68003720 msgr2=0x7f2f68003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f89921700 1 --2- 172.21.15.32:0/2403095336 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2f68003720 0x7f2f68003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f89921700 1 -- 172.21.15.32:0/2403095336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f68003230 con 0x7f2f680049c0 2020-12-05T17:31:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f89921700 1 --2- 172.21.15.32:0/2403095336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f680049c0 0x7f2f68004e40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f2f7c001760 tx=0x7f2f7c007b40).ready entity=mon.0 client_cookie=c30db5b2e5f435a9 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f77fff700 1 -- 172.21.15.32:0/2403095336 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f7c005b70 con 0x7f2f680049c0 2020-12-05T17:31:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f77fff700 1 -- 172.21.15.32:0/2403095336 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7c005cd0 con 0x7f2f680049c0 2020-12-05T17:31:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f77fff700 1 -- 172.21.15.32:0/2403095336 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f7c0072c0 con 0x7f2f680049c0 2020-12-05T17:31:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f8b384700 1 -- 172.21.15.32:0/2403095336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f680049c0 msgr2=0x7f2f68004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f8b384700 1 --2- 172.21.15.32:0/2403095336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f680049c0 0x7f2f68004e40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f2f7c001760 tx=0x7f2f7c007b40).stop 2020-12-05T17:31:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f89921700 1 -- 172.21.15.32:0/2403095336 reap_dead start 2020-12-05T17:31:41.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f8b384700 1 -- 172.21.15.32:0/2403095336 shutdown_connections 2020-12-05T17:31:41.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f8b384700 1 -- 172.21.15.32:0/2403095336 >> 172.21.15.32:0/2403095336 conn(0x7f2f8405fe80 msgr2=0x7f2f840b4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:41.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f8b384700 1 -- 172.21.15.32:0/2403095336 shutdown_connections 2020-12-05T17:31:41.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.411+0000 7f2f8b384700 1 -- 172.21.15.32:0/2403095336 wait complete. 2020-12-05T17:31:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f8b384700 1 Processor -- start 2020-12-05T17:31:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f8b384700 1 -- start start 2020-12-05T17:31:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f8b384700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f84142be0 0x7f2f84151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:41.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f8b384700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f84151950 0x7f2f84140b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:41.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f8b384700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2f84141080 0x7f2f84154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:41.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f8b384700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8413f330 con 0x7f2f84151950 2020-12-05T17:31:41.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f8b384700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8413e340 con 0x7f2f84141080 2020-12-05T17:31:41.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f8b384700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f84142fc0 con 0x7f2f84142be0 2020-12-05T17:31:41.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f8891f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f84151950 0x7f2f84140b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:41.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f8891f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f84151950 0x7f2f84140b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60596/0 (socket says 172.21.15.32:60596) 2020-12-05T17:31:41.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f8891f700 1 -- 172.21.15.32:0/241486655 learned_addr learned my addr 172.21.15.32:0/241486655 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:41.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f89120700 1 --2- 172.21.15.32:0/241486655 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f84142be0 0x7f2f84151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:41.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f89921700 1 --2- 172.21.15.32:0/241486655 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2f84141080 0x7f2f84154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:41.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f8891f700 1 -- 172.21.15.32:0/241486655 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f84142be0 msgr2=0x7f2f84151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:41.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f8891f700 1 --2- 172.21.15.32:0/241486655 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f84142be0 0x7f2f84151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f8891f700 1 -- 172.21.15.32:0/241486655 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2f84141080 msgr2=0x7f2f84154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:41.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f8891f700 1 --2- 172.21.15.32:0/241486655 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2f84141080 0x7f2f84154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f8891f700 1 -- 172.21.15.32:0/241486655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8413f080 con 0x7f2f84151950 2020-12-05T17:31:41.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f89120700 1 --2- 172.21.15.32:0/241486655 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f84142be0 0x7f2f84151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:31:41.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f8891f700 1 --2- 172.21.15.32:0/241486655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f84151950 0x7f2f84140b90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f2f84143150 tx=0x7f2f78008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:41.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f767fc700 1 -- 172.21.15.32:0/241486655 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f7800fce0 con 0x7f2f84151950 2020-12-05T17:31:41.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f767fc700 1 -- 172.21.15.32:0/241486655 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7800fe40 con 0x7f2f84151950 2020-12-05T17:31:41.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f8b384700 1 -- 172.21.15.32:0/241486655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8413e5d0 con 0x7f2f84151950 2020-12-05T17:31:41.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f767fc700 1 -- 172.21.15.32:0/241486655 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f78013630 con 0x7f2f84151950 2020-12-05T17:31:41.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f8b384700 1 -- 172.21.15.32:0/241486655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f8413e850 con 0x7f2f84151950 2020-12-05T17:31:41.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.415+0000 7f2f8b384700 1 -- 172.21.15.32:0/241486655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f84057050 con 0x7f2f84151950 2020-12-05T17:31:41.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.419+0000 7f2f767fc700 1 -- 172.21.15.32:0/241486655 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2f7800e040 con 0x7f2f84151950 2020-12-05T17:31:41.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.419+0000 7f2f767fc700 1 --2- 172.21.15.32:0/241486655 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2f6c055330 0x7f2f6c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:41.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.419+0000 7f2f767fc700 1 -- 172.21.15.32:0/241486655 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6315+0+0 (secure 0 0 0) 0x7f2f7806ea50 con 0x7f2f84151950 2020-12-05T17:31:41.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.419+0000 7f2f89120700 1 --2- 172.21.15.32:0/241486655 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2f6c055330 0x7f2f6c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:41.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.419+0000 7f2f89120700 1 --2- 172.21.15.32:0/241486655 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2f6c055330 0x7f2f6c0577f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f2f80008000 tx=0x7f2f80008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:41.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.423+0000 7f2f767fc700 1 -- 172.21.15.32:0/241486655 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2f78044ad0 con 0x7f2f84151950 2020-12-05T17:31:41.609 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.607+0000 7f2f8b384700 1 -- 172.21.15.32:0/241486655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f2f8405bae0 con 0x7f2f84151950 2020-12-05T17:31:41.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.615+0000 7f2f767fc700 1 -- 172.21.15.32:0/241486655 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 187+0+0 (secure 0 0 0) 0x7f2f780480f0 con 0x7f2f84151950 2020-12-05T17:31:41.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.619+0000 7f2f8b384700 1 -- 172.21.15.32:0/241486655 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2f6c055330 msgr2=0x7f2f6c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:41.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.619+0000 7f2f8b384700 1 --2- 172.21.15.32:0/241486655 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2f6c055330 0x7f2f6c0577f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f2f80008000 tx=0x7f2f80008040).stop 2020-12-05T17:31:41.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.619+0000 7f2f8b384700 1 -- 172.21.15.32:0/241486655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f84151950 msgr2=0x7f2f84140b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:41.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.619+0000 7f2f8b384700 1 --2- 172.21.15.32:0/241486655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f84151950 0x7f2f84140b90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f2f84143150 tx=0x7f2f78008040).stop 2020-12-05T17:31:41.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.619+0000 7f2f8b384700 1 -- 172.21.15.32:0/241486655 shutdown_connections 2020-12-05T17:31:41.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.619+0000 7f2f8b384700 1 --2- 172.21.15.32:0/241486655 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2f84141080 0x7f2f84154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.619+0000 7f2f8b384700 1 --2- 172.21.15.32:0/241486655 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2f6c055330 0x7f2f6c0577f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.619+0000 7f2f8b384700 1 --2- 172.21.15.32:0/241486655 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f84142be0 0x7f2f84151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.619+0000 7f2f8b384700 1 --2- 172.21.15.32:0/241486655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f84151950 0x7f2f84140b90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.619+0000 7f2f8b384700 1 -- 172.21.15.32:0/241486655 >> 172.21.15.32:0/241486655 conn(0x7f2f8405fe80 msgr2=0x7f2f8413bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:41.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.619+0000 7f2f8b384700 1 -- 172.21.15.32:0/241486655 shutdown_connections 2020-12-05T17:31:41.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.623+0000 7f2f8b384700 1 -- 172.21.15.32:0/241486655 wait complete. 2020-12-05T17:31:41.629 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 3 and data pool 4 2020-12-05T17:31:41.645 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:31:41.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f7fff700 1 Processor -- start 2020-12-05T17:31:41.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f7fff700 1 -- start start 2020-12-05T17:31:41.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f7fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5f80669b0 0x7fb5f80b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:41.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5f80ae3e0 0x7fb5f81490e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:41.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f7fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f80b3020 con 0x7fb5f805f6f0 2020-12-05T17:31:41.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f7fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f8147ec0 con 0x7fb5f80ae3e0 2020-12-05T17:31:41.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f7fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f80b1f50 con 0x7fb5f80669b0 2020-12-05T17:31:41.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f6ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb5f805f6f0 0x7fb5f80662a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35030/0 (socket says 172.21.15.32:35030) 2020-12-05T17:31:41.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f6ffd700 1 -- 172.21.15.32:0/2725017479 learned_addr learned my addr 172.21.15.32:0/2725017479 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:41.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f77fe700 1 --2- 172.21.15.32:0/2725017479 >> v2:172.21.15.32:3300/0 conn(0x7fb5f80ae3e0 0x7fb5f81490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:41.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f67fc700 1 --2- 172.21.15.32:0/2725017479 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5f80669b0 0x7fb5f80b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:41.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f5ffb700 1 -- 172.21.15.32:0/2725017479 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 770581926 0 0) 0x7fb5f80b3020 con 0x7fb5f805f6f0 2020-12-05T17:31:41.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f5ffb700 1 -- 172.21.15.32:0/2725017479 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5dc003230 con 0x7fb5f805f6f0 2020-12-05T17:31:41.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f5ffb700 1 -- 172.21.15.32:0/2725017479 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2346699848 0 0) 0x7fb5dc003230 con 0x7fb5f805f6f0 2020-12-05T17:31:41.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f5ffb700 1 -- 172.21.15.32:0/2725017479 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5f80669b0 msgr2=0x7fb5f80b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:41.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f5ffb700 1 --2- 172.21.15.32:0/2725017479 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5f80669b0 0x7fb5f80b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f5ffb700 1 -- 172.21.15.32:0/2725017479 >> v2:172.21.15.32:3300/0 conn(0x7fb5f80ae3e0 msgr2=0x7fb5f81490e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:41.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f5ffb700 1 --2- 172.21.15.32:0/2725017479 >> v2:172.21.15.32:3300/0 conn(0x7fb5f80ae3e0 0x7fb5f81490e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f5ffb700 1 -- 172.21.15.32:0/2725017479 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f805fad0 con 0x7fb5f805f6f0 2020-12-05T17:31:41.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f5ffb700 1 -- 172.21.15.32:0/2725017479 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fb5e80016d0 con 0x7fb5f805f6f0 2020-12-05T17:31:41.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f77fe700 1 --2- 172.21.15.32:0/2725017479 >> v2:172.21.15.32:3300/0 conn(0x7fb5f80ae3e0 0x7fb5f81490e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:31:41.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f5ffb700 1 -- 172.21.15.32:0/2725017479 >> v1:172.21.15.32:6789/0 conn(0x7fb5f805f6f0 legacy=0x7fb5f80662a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:41.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f5ffb700 1 --2- 172.21.15.32:0/2725017479 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5f80669b0 0x7fb5dc003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:41.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f5ffb700 1 --2- 172.21.15.32:0/2725017479 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5dc003c60 0x7fb5dc0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:41.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f5ffb700 1 --2- 172.21.15.32:0/2725017479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5dc0049c0 0x7fb5dc004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:41.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f5ffb700 1 -- 172.21.15.32:0/2725017479 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f8147ec0 con 0x7fb5dc0049c0 2020-12-05T17:31:41.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f5ffb700 1 -- 172.21.15.32:0/2725017479 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f80b1f50 con 0x7fb5dc003c60 2020-12-05T17:31:41.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f5ffb700 1 -- 172.21.15.32:0/2725017479 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc004220 con 0x7fb5f80669b0 2020-12-05T17:31:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f77fe700 1 --2- 172.21.15.32:0/2725017479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5dc0049c0 0x7fb5dc004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f67fc700 1 --2- 172.21.15.32:0/2725017479 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5dc003c60 0x7fb5dc0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f6ffd700 1 --2- 172.21.15.32:0/2725017479 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5f80669b0 0x7fb5dc003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f77fe700 1 -- 172.21.15.32:0/2725017479 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5f80669b0 msgr2=0x7fb5dc003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f77fe700 1 --2- 172.21.15.32:0/2725017479 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5f80669b0 0x7fb5dc003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f77fe700 1 -- 172.21.15.32:0/2725017479 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5dc003c60 msgr2=0x7fb5dc0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:41.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f77fe700 1 --2- 172.21.15.32:0/2725017479 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5dc003c60 0x7fb5dc0044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f77fe700 1 -- 172.21.15.32:0/2725017479 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5dc003230 con 0x7fb5dc0049c0 2020-12-05T17:31:41.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.743+0000 7fb5f77fe700 1 --2- 172.21.15.32:0/2725017479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5dc0049c0 0x7fb5dc004e40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fb5f000c000 tx=0x7fb5f00091a0).ready entity=mon.0 client_cookie=9777959c7a22bba6 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:41.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.747+0000 7fb5f5ffb700 1 -- 172.21.15.32:0/2725017479 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5f000ee60 con 0x7fb5dc0049c0 2020-12-05T17:31:41.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.747+0000 7fb5f5ffb700 1 -- 172.21.15.32:0/2725017479 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f00123f0 con 0x7fb5dc0049c0 2020-12-05T17:31:41.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.747+0000 7fb5f5ffb700 1 -- 172.21.15.32:0/2725017479 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5f0012750 con 0x7fb5dc0049c0 2020-12-05T17:31:41.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.747+0000 7fb5f7fff700 1 -- 172.21.15.32:0/2725017479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5dc0049c0 msgr2=0x7fb5dc004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:41.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.747+0000 7fb5f7fff700 1 --2- 172.21.15.32:0/2725017479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5dc0049c0 0x7fb5dc004e40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fb5f000c000 tx=0x7fb5f00091a0).stop 2020-12-05T17:31:41.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.747+0000 7fb5f77fe700 1 -- 172.21.15.32:0/2725017479 reap_dead start 2020-12-05T17:31:41.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.747+0000 7fb5f7fff700 1 -- 172.21.15.32:0/2725017479 shutdown_connections 2020-12-05T17:31:41.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.747+0000 7fb5f7fff700 1 -- 172.21.15.32:0/2725017479 >> 172.21.15.32:0/2725017479 conn(0x7fb5f80604c0 msgr2=0x7fb5f805c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:41.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.747+0000 7fb5f7fff700 1 -- 172.21.15.32:0/2725017479 shutdown_connections 2020-12-05T17:31:41.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.747+0000 7fb5f7fff700 1 -- 172.21.15.32:0/2725017479 wait complete. 2020-12-05T17:31:41.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.747+0000 7fb5f7fff700 1 Processor -- start 2020-12-05T17:31:41.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.747+0000 7fb5f7fff700 1 -- start start 2020-12-05T17:31:41.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.747+0000 7fb5f7fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5f810f950 0x7fb5f8110ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:41.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.747+0000 7fb5f7fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5f8149060 0x7fb5f814c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:41.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.747+0000 7fb5f7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f814ca30 0x7fb5f8155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:41.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.747+0000 7fb5f7fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f81101f0 con 0x7fb5f814ca30 2020-12-05T17:31:41.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.747+0000 7fb5f7fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f8148680 con 0x7fb5f810f950 2020-12-05T17:31:41.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.747+0000 7fb5f7fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f8148920 con 0x7fb5f8149060 2020-12-05T17:31:41.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.751+0000 7fb5f77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f814ca30 0x7fb5f8155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:41.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.751+0000 7fb5f77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f814ca30 0x7fb5f8155a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60630/0 (socket says 172.21.15.32:60630) 2020-12-05T17:31:41.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.751+0000 7fb5f6ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5f810f950 0x7fb5f8110ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:41.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.751+0000 7fb5f77fe700 1 -- 172.21.15.32:0/251700195 learned_addr learned my addr 172.21.15.32:0/251700195 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:41.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.751+0000 7fb5f67fc700 1 --2- 172.21.15.32:0/251700195 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5f8149060 0x7fb5f814c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:41.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.751+0000 7fb5f77fe700 1 -- 172.21.15.32:0/251700195 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5f8149060 msgr2=0x7fb5f814c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:41.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.751+0000 7fb5f77fe700 1 --2- 172.21.15.32:0/251700195 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5f8149060 0x7fb5f814c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.751+0000 7fb5f77fe700 1 -- 172.21.15.32:0/251700195 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5f810f950 msgr2=0x7fb5f8110ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:41.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.751+0000 7fb5f77fe700 1 --2- 172.21.15.32:0/251700195 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5f810f950 0x7fb5f8110ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.751+0000 7fb5f77fe700 1 -- 172.21.15.32:0/251700195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f8148bc0 con 0x7fb5f814ca30 2020-12-05T17:31:41.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.751+0000 7fb5f77fe700 1 --2- 172.21.15.32:0/251700195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f814ca30 0x7fb5f8155a40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fb5f8062020 tx=0x7fb5f001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:41.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.751+0000 7fb5e7fff700 1 -- 172.21.15.32:0/251700195 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5f00127f0 con 0x7fb5f814ca30 2020-12-05T17:31:41.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.751+0000 7fb5e7fff700 1 -- 172.21.15.32:0/251700195 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f0012950 con 0x7fb5f814ca30 2020-12-05T17:31:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.751+0000 7fb5e7fff700 1 -- 172.21.15.32:0/251700195 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5f0012cb0 con 0x7fb5f814ca30 2020-12-05T17:31:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.751+0000 7fb5f7fff700 1 -- 172.21.15.32:0/251700195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f814ceb0 con 0x7fb5f814ca30 2020-12-05T17:31:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.751+0000 7fb5f7fff700 1 -- 172.21.15.32:0/251700195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f810fd30 con 0x7fb5f814ca30 2020-12-05T17:31:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.751+0000 7fb5f7fff700 1 -- 172.21.15.32:0/251700195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f81484d0 con 0x7fb5f814ca30 2020-12-05T17:31:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.751+0000 7fb5e7fff700 1 -- 172.21.15.32:0/251700195 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb5f0030480 con 0x7fb5f814ca30 2020-12-05T17:31:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.755+0000 7fb5e7fff700 1 --2- 172.21.15.32:0/251700195 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb5d4055440 0x7fb5d4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.755+0000 7fb5e7fff700 1 -- 172.21.15.32:0/251700195 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb5f0002020 con 0x7fb5f814ca30 2020-12-05T17:31:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.755+0000 7fb5f6ffd700 1 --2- 172.21.15.32:0/251700195 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb5d4055440 0x7fb5d4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.755+0000 7fb5f6ffd700 1 --2- 172.21.15.32:0/251700195 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb5d4055440 0x7fb5d4057900 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fb5e800d000 tx=0x7fb5e8007040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:41.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.759+0000 7fb5e7fff700 1 -- 172.21.15.32:0/251700195 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb5f00465d0 con 0x7fb5f814ca30 2020-12-05T17:31:41.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.903+0000 7fb5f7fff700 1 -- 172.21.15.32:0/251700195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5f8057050 con 0x7fb5f814ca30 2020-12-05T17:31:41.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.907+0000 7fb5e7fff700 1 -- 172.21.15.32:0/251700195 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7fb5f0016020 con 0x7fb5f814ca30 2020-12-05T17:31:41.909 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:31:41.909 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:31:41.610914+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:31:36.706361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-12-05T17:31:39.298269+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:31:41.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.907+0000 7fb5f7fff700 1 -- 172.21.15.32:0/251700195 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb5d4055440 msgr2=0x7fb5d4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:41.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.907+0000 7fb5f7fff700 1 --2- 172.21.15.32:0/251700195 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb5d4055440 0x7fb5d4057900 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fb5e800d000 tx=0x7fb5e8007040).stop 2020-12-05T17:31:41.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.907+0000 7fb5f7fff700 1 -- 172.21.15.32:0/251700195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f814ca30 msgr2=0x7fb5f8155a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:41.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.907+0000 7fb5f7fff700 1 --2- 172.21.15.32:0/251700195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f814ca30 0x7fb5f8155a40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fb5f8062020 tx=0x7fb5f001a000).stop 2020-12-05T17:31:41.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.907+0000 7fb5f7fff700 1 -- 172.21.15.32:0/251700195 shutdown_connections 2020-12-05T17:31:41.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.907+0000 7fb5f7fff700 1 --2- 172.21.15.32:0/251700195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f814ca30 0x7fb5f8155a40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.907+0000 7fb5f7fff700 1 --2- 172.21.15.32:0/251700195 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5f810f950 0x7fb5f8110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.907+0000 7fb5f7fff700 1 --2- 172.21.15.32:0/251700195 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb5d4055440 0x7fb5d4057900 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.907+0000 7fb5f7fff700 1 --2- 172.21.15.32:0/251700195 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5f8149060 0x7fb5f814c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:41.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.907+0000 7fb5f7fff700 1 -- 172.21.15.32:0/251700195 >> 172.21.15.32:0/251700195 conn(0x7fb5f80604c0 msgr2=0x7fb5f805fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:41.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.907+0000 7fb5f7fff700 1 -- 172.21.15.32:0/251700195 shutdown_connections 2020-12-05T17:31:41.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:41.907+0000 7fb5f7fff700 1 -- 172.21.15.32:0/251700195 wait complete. 2020-12-05T17:31:41.923 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:31:42.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec18e05700 1 Processor -- start 2020-12-05T17:31:42.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec18e05700 1 -- start start 2020-12-05T17:31:42.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec18e05700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fec140b00a0 0x7fec14061910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:42.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec18e05700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec140acca0 0x7fec1414a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:42.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec18e05700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec140601e0 con 0x7fec1405fe00 2020-12-05T17:31:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec18e05700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec140ac1a0 con 0x7fec140b00a0 2020-12-05T17:31:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec18e05700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec14061fa0 con 0x7fec140acca0 2020-12-05T17:31:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec1259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fec1405fe00 0x7fec140ac590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35050/0 (socket says 172.21.15.32:35050) 2020-12-05T17:31:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec1259c700 1 -- 172.21.15.32:0/3167893464 learned_addr learned my addr 172.21.15.32:0/3167893464 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec11d9b700 1 --2- 172.21.15.32:0/3167893464 >> v2:172.21.15.32:3300/0 conn(0x7fec140b00a0 0x7fec14061910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec1159a700 1 -- 172.21.15.32:0/3167893464 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2927634631 0 0) 0x7fec140601e0 con 0x7fec1405fe00 2020-12-05T17:31:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec1159a700 1 -- 172.21.15.32:0/3167893464 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febf4003230 con 0x7fec1405fe00 2020-12-05T17:31:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec12d9d700 1 --2- 172.21.15.32:0/3167893464 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec140acca0 0x7fec1414a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec1159a700 1 -- 172.21.15.32:0/3167893464 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3011757581 0 0) 0x7febf4003230 con 0x7fec1405fe00 2020-12-05T17:31:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec11d9b700 1 -- 172.21.15.32:0/3167893464 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec140acca0 msgr2=0x7fec1414a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec11d9b700 1 --2- 172.21.15.32:0/3167893464 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec140acca0 0x7fec1414a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec11d9b700 1 -- 172.21.15.32:0/3167893464 >> v1:172.21.15.32:6789/0 conn(0x7fec1405fe00 legacy=0x7fec140ac590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec11d9b700 1 -- 172.21.15.32:0/3167893464 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec14062240 con 0x7fec140b00a0 2020-12-05T17:31:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec11d9b700 1 --2- 172.21.15.32:0/3167893464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec140b00a0 0x7fec14061910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fec04001870 tx=0x7febfc006dc0).ready entity=mon.0 client_cookie=fe82c45f7c4d794e server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec1159a700 1 -- 172.21.15.32:0/3167893464 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec040016d0 con 0x7fec140b00a0 2020-12-05T17:31:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec1159a700 1 -- 172.21.15.32:0/3167893464 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febfc00d570 con 0x7fec140b00a0 2020-12-05T17:31:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec1159a700 1 -- 172.21.15.32:0/3167893464 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febfc00d6e0 con 0x7fec140b00a0 2020-12-05T17:31:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec18e05700 1 -- 172.21.15.32:0/3167893464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec140b00a0 msgr2=0x7fec14061910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec18e05700 1 --2- 172.21.15.32:0/3167893464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec140b00a0 0x7fec14061910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fec04001870 tx=0x7febfc006dc0).stop 2020-12-05T17:31:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec18e05700 1 -- 172.21.15.32:0/3167893464 shutdown_connections 2020-12-05T17:31:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec18e05700 1 --2- 172.21.15.32:0/3167893464 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec140acca0 0x7fec1414a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec18e05700 1 --2- 172.21.15.32:0/3167893464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec140b00a0 0x7fec14061910 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec18e05700 1 -- 172.21.15.32:0/3167893464 >> 172.21.15.32:0/3167893464 conn(0x7fec140b57c0 msgr2=0x7fec1405f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec18e05700 1 -- 172.21.15.32:0/3167893464 shutdown_connections 2020-12-05T17:31:42.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.035+0000 7fec18e05700 1 -- 172.21.15.32:0/3167893464 wait complete. 2020-12-05T17:31:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec18e05700 1 Processor -- start 2020-12-05T17:31:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec18e05700 1 -- start start 2020-12-05T17:31:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec18e05700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec1405fe00 0x7fec140665e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec18e05700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec140acca0 0x7fec14066ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec18e05700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec140b00a0 0x7fec14141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec18e05700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec14064b90 con 0x7fec1405fe00 2020-12-05T17:31:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec18e05700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec14062c80 con 0x7fec140b00a0 2020-12-05T17:31:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec18e05700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec14062f20 con 0x7fec140acca0 2020-12-05T17:31:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec1259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec1405fe00 0x7fec140665e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec1259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec1405fe00 0x7fec140665e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60640/0 (socket says 172.21.15.32:60640) 2020-12-05T17:31:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec1259c700 1 -- 172.21.15.32:0/1077986301 learned_addr learned my addr 172.21.15.32:0/1077986301 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec11d9b700 1 --2- 172.21.15.32:0/1077986301 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec140acca0 0x7fec14066ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec12d9d700 1 --2- 172.21.15.32:0/1077986301 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec140b00a0 0x7fec14141ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:42.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec1259c700 1 -- 172.21.15.32:0/1077986301 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec140acca0 msgr2=0x7fec14066ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:42.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec1259c700 1 --2- 172.21.15.32:0/1077986301 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec140acca0 0x7fec14066ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec1259c700 1 -- 172.21.15.32:0/1077986301 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec140b00a0 msgr2=0x7fec14141ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:42.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec1259c700 1 --2- 172.21.15.32:0/1077986301 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec140b00a0 0x7fec14141ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec1259c700 1 -- 172.21.15.32:0/1077986301 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1414ad10 con 0x7fec1405fe00 2020-12-05T17:31:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec1259c700 1 --2- 172.21.15.32:0/1077986301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec1405fe00 0x7fec140665e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fec040018b0 tx=0x7fec04001990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec0b7fe700 1 -- 172.21.15.32:0/1077986301 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec04007170 con 0x7fec1405fe00 2020-12-05T17:31:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec0b7fe700 1 -- 172.21.15.32:0/1077986301 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec040072d0 con 0x7fec1405fe00 2020-12-05T17:31:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec18e05700 1 -- 172.21.15.32:0/1077986301 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec14063bc0 con 0x7fec1405fe00 2020-12-05T17:31:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec0b7fe700 1 -- 172.21.15.32:0/1077986301 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec040048f0 con 0x7fec1405fe00 2020-12-05T17:31:42.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.039+0000 7fec18e05700 1 -- 172.21.15.32:0/1077986301 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec14142580 con 0x7fec1405fe00 2020-12-05T17:31:42.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.043+0000 7fec0b7fe700 1 -- 172.21.15.32:0/1077986301 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fec0401b060 con 0x7fec1405fe00 2020-12-05T17:31:42.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.043+0000 7fec18e05700 1 -- 172.21.15.32:0/1077986301 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec14057050 con 0x7fec1405fe00 2020-12-05T17:31:42.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.043+0000 7fec0b7fe700 1 --2- 172.21.15.32:0/1077986301 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7febf8055330 0x7febf80577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:42.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.043+0000 7fec11d9b700 1 --2- 172.21.15.32:0/1077986301 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7febf8055330 0x7febf80577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:42.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.043+0000 7fec0b7fe700 1 -- 172.21.15.32:0/1077986301 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fec0406dcf0 con 0x7fec1405fe00 2020-12-05T17:31:42.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.043+0000 7fec11d9b700 1 --2- 172.21.15.32:0/1077986301 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7febf8055330 0x7febf80577f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7febfc018000 tx=0x7febfc015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:42.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.051+0000 7fec0b7fe700 1 -- 172.21.15.32:0/1077986301 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fec04043d80 con 0x7fec1405fe00 2020-12-05T17:31:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.191+0000 7fec18e05700 1 -- 172.21.15.32:0/1077986301 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec14069690 con 0x7fec1405fe00 2020-12-05T17:31:42.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.195+0000 7fec0b7fe700 1 -- 172.21.15.32:0/1077986301 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7fec04019030 con 0x7fec1405fe00 2020-12-05T17:31:42.199 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:31:42.199 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:31:41.610914+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:31:36.706361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-12-05T17:31:39.298269+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:31:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.199+0000 7fec18e05700 1 -- 172.21.15.32:0/1077986301 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7febf8055330 msgr2=0x7febf80577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.199+0000 7fec18e05700 1 --2- 172.21.15.32:0/1077986301 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7febf8055330 0x7febf80577f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7febfc018000 tx=0x7febfc015000).stop 2020-12-05T17:31:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.199+0000 7fec18e05700 1 -- 172.21.15.32:0/1077986301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec1405fe00 msgr2=0x7fec140665e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.199+0000 7fec18e05700 1 --2- 172.21.15.32:0/1077986301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec1405fe00 0x7fec140665e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fec040018b0 tx=0x7fec04001990).stop 2020-12-05T17:31:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.199+0000 7fec18e05700 1 -- 172.21.15.32:0/1077986301 shutdown_connections 2020-12-05T17:31:42.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.199+0000 7fec18e05700 1 --2- 172.21.15.32:0/1077986301 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec140b00a0 0x7fec14141ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.199+0000 7fec18e05700 1 --2- 172.21.15.32:0/1077986301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec1405fe00 0x7fec140665e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.199+0000 7fec18e05700 1 --2- 172.21.15.32:0/1077986301 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7febf8055330 0x7febf80577f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.199+0000 7fec18e05700 1 --2- 172.21.15.32:0/1077986301 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec140acca0 0x7fec14066ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.199+0000 7fec18e05700 1 -- 172.21.15.32:0/1077986301 >> 172.21.15.32:0/1077986301 conn(0x7fec140b57c0 msgr2=0x7fec140af690 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:42.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.199+0000 7fec18e05700 1 -- 172.21.15.32:0/1077986301 shutdown_connections 2020-12-05T17:31:42.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.199+0000 7fec18e05700 1 -- 172.21.15.32:0/1077986301 wait complete. 2020-12-05T17:31:42.217 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-12-05T17:31:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.339+0000 7f4555afc700 1 Processor -- start 2020-12-05T17:31:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.339+0000 7f4555afc700 1 -- start start 2020-12-05T17:31:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.339+0000 7f4555afc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45500b5b20 0x7f45500b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:42.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.339+0000 7f4555afc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45500b6690 0x7f45500b5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:42.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.339+0000 7f4555afc700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45500b1920 con 0x7f45500b2060 2020-12-05T17:31:42.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.339+0000 7f4555afc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45500b5730 con 0x7f45500b5b20 2020-12-05T17:31:42.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.339+0000 7f4555afc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45500b1bc0 con 0x7f45500b6690 2020-12-05T17:31:42.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.339+0000 7f454f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45500b5b20 0x7f45500b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.339+0000 7f454f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45500b5b20 0x7f45500b5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60648/0 (socket says 172.21.15.32:60648) 2020-12-05T17:31:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.339+0000 7f454f7fe700 1 -- 172.21.15.32:0/2501624116 learned_addr learned my addr 172.21.15.32:0/2501624116 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.339+0000 7f454effd700 1 --2- 172.21.15.32:0/2501624116 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45500b6690 0x7f45500b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.339+0000 7f454f7fe700 1 -- 172.21.15.32:0/2501624116 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45500b6690 msgr2=0x7f45500b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.339+0000 7f454f7fe700 1 --2- 172.21.15.32:0/2501624116 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45500b6690 0x7f45500b5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.339+0000 7f454f7fe700 1 -- 172.21.15.32:0/2501624116 >> v1:172.21.15.61:6789/0 conn(0x7f45500b2060 legacy=0x7f4550145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.339+0000 7f454f7fe700 1 -- 172.21.15.32:0/2501624116 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45500b24c0 con 0x7f45500b5b20 2020-12-05T17:31:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.339+0000 7f454f7fe700 1 --2- 172.21.15.32:0/2501624116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45500b5b20 0x7f45500b5f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f45400032b0 tx=0x7f4540009270).ready entity=mon.0 client_cookie=eba27e5343a0c0d4 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.339+0000 7f454e7fc700 1 -- 172.21.15.32:0/2501624116 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f454000bc00 con 0x7f45500b5b20 2020-12-05T17:31:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.339+0000 7f454e7fc700 1 -- 172.21.15.32:0/2501624116 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454000bd60 con 0x7f45500b5b20 2020-12-05T17:31:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.339+0000 7f454e7fc700 1 -- 172.21.15.32:0/2501624116 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4540011400 con 0x7f45500b5b20 2020-12-05T17:31:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.339+0000 7f4555afc700 1 -- 172.21.15.32:0/2501624116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45500b5b20 msgr2=0x7f45500b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:42.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.339+0000 7f4555afc700 1 --2- 172.21.15.32:0/2501624116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45500b5b20 0x7f45500b5f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f45400032b0 tx=0x7f4540009270).stop 2020-12-05T17:31:42.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.343+0000 7f4555afc700 1 -- 172.21.15.32:0/2501624116 shutdown_connections 2020-12-05T17:31:42.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.343+0000 7f4555afc700 1 --2- 172.21.15.32:0/2501624116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45500b5b20 0x7f45500b5f00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.343+0000 7f4555afc700 1 --2- 172.21.15.32:0/2501624116 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45500b6690 0x7f45500b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.343+0000 7f4555afc700 1 -- 172.21.15.32:0/2501624116 >> 172.21.15.32:0/2501624116 conn(0x7f455005f3b0 msgr2=0x7f455005f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:42.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.343+0000 7f4555afc700 1 -- 172.21.15.32:0/2501624116 shutdown_connections 2020-12-05T17:31:42.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.343+0000 7f4555afc700 1 -- 172.21.15.32:0/2501624116 wait complete. 2020-12-05T17:31:42.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.343+0000 7f4555afc700 1 Processor -- start 2020-12-05T17:31:42.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.343+0000 7f4555afc700 1 -- start start 2020-12-05T17:31:42.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.343+0000 7f4555afc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45500b2060 0x7f45500653f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:42.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.343+0000 7f4555afc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45500b5b20 0x7f4550143e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:42.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.343+0000 7f4555afc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45500b6690 0x7f45501484c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:42.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.343+0000 7f4555afc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4550144300 con 0x7f45500b5b20 2020-12-05T17:31:42.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.343+0000 7f4555afc700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45500a26b0 con 0x7f45500b6690 2020-12-05T17:31:42.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.343+0000 7f4555afc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4550152530 con 0x7f45500b2060 2020-12-05T17:31:42.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.347+0000 7f454effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45500b5b20 0x7f4550143e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:42.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.347+0000 7f454f7fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45500b2060 0x7f45500653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:42.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.347+0000 7f454ffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45500b6690 0x7f45501484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:42.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.347+0000 7f454effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45500b5b20 0x7f4550143e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60656/0 (socket says 172.21.15.32:60656) 2020-12-05T17:31:42.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.347+0000 7f454effd700 1 -- 172.21.15.32:0/1342160005 learned_addr learned my addr 172.21.15.32:0/1342160005 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:42.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.347+0000 7f454effd700 1 -- 172.21.15.32:0/1342160005 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45500b2060 msgr2=0x7f45500653f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:42.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.347+0000 7f454effd700 1 --2- 172.21.15.32:0/1342160005 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45500b2060 0x7f45500653f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.347+0000 7f454effd700 1 -- 172.21.15.32:0/1342160005 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45500b6690 msgr2=0x7f45501484c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:42.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.347+0000 7f454effd700 1 --2- 172.21.15.32:0/1342160005 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45500b6690 0x7f45501484c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.347+0000 7f454effd700 1 -- 172.21.15.32:0/1342160005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4550145330 con 0x7f45500b5b20 2020-12-05T17:31:42.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.347+0000 7f454effd700 1 --2- 172.21.15.32:0/1342160005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45500b5b20 0x7f4550143e10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f4544003070 tx=0x7f4544008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:42.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.347+0000 7f454cff9700 1 -- 172.21.15.32:0/1342160005 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45440127e0 con 0x7f45500b5b20 2020-12-05T17:31:42.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.347+0000 7f454cff9700 1 -- 172.21.15.32:0/1342160005 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4544012940 con 0x7f45500b5b20 2020-12-05T17:31:42.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.347+0000 7f454cff9700 1 -- 172.21.15.32:0/1342160005 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4544012ca0 con 0x7f45500b5b20 2020-12-05T17:31:42.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.347+0000 7f4555afc700 1 -- 172.21.15.32:0/1342160005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4550066c60 con 0x7f45500b5b20 2020-12-05T17:31:42.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.347+0000 7f4555afc700 1 -- 172.21.15.32:0/1342160005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45500af400 con 0x7f45500b5b20 2020-12-05T17:31:42.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.347+0000 7f4555afc700 1 -- 172.21.15.32:0/1342160005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4550152370 con 0x7f45500b5b20 2020-12-05T17:31:42.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.347+0000 7f454cff9700 1 -- 172.21.15.32:0/1342160005 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4544013480 con 0x7f45500b5b20 2020-12-05T17:31:42.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.347+0000 7f454cff9700 1 --2- 172.21.15.32:0/1342160005 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4530055440 0x7f4530057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:42.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.351+0000 7f454cff9700 1 -- 172.21.15.32:0/1342160005 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f454406e6c0 con 0x7f45500b5b20 2020-12-05T17:31:42.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.351+0000 7f454f7fe700 1 --2- 172.21.15.32:0/1342160005 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4530055440 0x7f4530057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:42.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.351+0000 7f454f7fe700 1 --2- 172.21.15.32:0/1342160005 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4530055440 0x7f4530057900 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f454001a000 tx=0x7f4540016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:42.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.355+0000 7f454cff9700 1 -- 172.21.15.32:0/1342160005 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4544044760 con 0x7f45500b5b20 2020-12-05T17:31:42.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.543+0000 7f4555afc700 1 -- 172.21.15.32:0/1342160005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f45501526f0 con 0x7f45500b5b20 2020-12-05T17:31:42.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.615+0000 7f454cff9700 1 -- 172.21.15.32:0/1342160005 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4544047d80 con 0x7f45500b5b20 2020-12-05T17:31:42.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.619+0000 7f4555afc700 1 -- 172.21.15.32:0/1342160005 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4530055440 msgr2=0x7f4530057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:42.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.619+0000 7f4555afc700 1 --2- 172.21.15.32:0/1342160005 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4530055440 0x7f4530057900 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f454001a000 tx=0x7f4540016000).stop 2020-12-05T17:31:42.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.619+0000 7f4555afc700 1 -- 172.21.15.32:0/1342160005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45500b5b20 msgr2=0x7f4550143e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:42.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.619+0000 7f4555afc700 1 --2- 172.21.15.32:0/1342160005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45500b5b20 0x7f4550143e10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f4544003070 tx=0x7f4544008040).stop 2020-12-05T17:31:42.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.619+0000 7f4555afc700 1 -- 172.21.15.32:0/1342160005 shutdown_connections 2020-12-05T17:31:42.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.619+0000 7f4555afc700 1 --2- 172.21.15.32:0/1342160005 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45500b6690 0x7f45501484c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.619+0000 7f4555afc700 1 --2- 172.21.15.32:0/1342160005 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4530055440 0x7f4530057900 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.619+0000 7f4555afc700 1 --2- 172.21.15.32:0/1342160005 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45500b2060 0x7f45500653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.619+0000 7f4555afc700 1 --2- 172.21.15.32:0/1342160005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45500b5b20 0x7f4550143e10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.619+0000 7f4555afc700 1 -- 172.21.15.32:0/1342160005 >> 172.21.15.32:0/1342160005 conn(0x7f455005f3b0 msgr2=0x7f45500b82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:42.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.619+0000 7f4555afc700 1 -- 172.21.15.32:0/1342160005 shutdown_connections 2020-12-05T17:31:42.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.619+0000 7f4555afc700 1 -- 172.21.15.32:0/1342160005 wait complete. 2020-12-05T17:31:42.640 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:31:42.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.755+0000 7fdafb852700 1 Processor -- start 2020-12-05T17:31:42.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.755+0000 7fdafb852700 1 -- start start 2020-12-05T17:31:42.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdafb852700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fdaf405f6f0 0x7fdaf4143d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:42.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdafb852700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdaf413a380 0x7fdaf413a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:42.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdafb852700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdaf4065680 0x7fdaf4065af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:42.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdafb852700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf413ad20 con 0x7fdaf413a380 2020-12-05T17:31:42.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdafb852700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf40b2860 con 0x7fdaf405f6f0 2020-12-05T17:31:42.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdafb852700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf40acb80 con 0x7fdaf4065680 2020-12-05T17:31:42.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdaf8ded700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdaf413a380 0x7fdaf413a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:42.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdaf8ded700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdaf413a380 0x7fdaf413a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60662/0 (socket says 172.21.15.32:60662) 2020-12-05T17:31:42.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdaf8ded700 1 -- 172.21.15.32:0/4063451267 learned_addr learned my addr 172.21.15.32:0/4063451267 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:42.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdaf9def700 1 --2- 172.21.15.32:0/4063451267 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdaf4065680 0x7fdaf4065af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:42.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdaf95ee700 1 --2- 172.21.15.32:0/4063451267 >> v2:172.21.15.61:3300/0 conn(0x7fdaf405f6f0 0x7fdaf4143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:42.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdaf8ded700 1 -- 172.21.15.32:0/4063451267 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdaf4065680 msgr2=0x7fdaf4065af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:42.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdaf8ded700 1 --2- 172.21.15.32:0/4063451267 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdaf4065680 0x7fdaf4065af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdaf8ded700 1 -- 172.21.15.32:0/4063451267 >> v2:172.21.15.61:3300/0 conn(0x7fdaf405f6f0 msgr2=0x7fdaf4143d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:42.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdaf8ded700 1 --2- 172.21.15.32:0/4063451267 >> v2:172.21.15.61:3300/0 conn(0x7fdaf405f6f0 0x7fdaf4143d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdaf8ded700 1 -- 172.21.15.32:0/4063451267 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf4065250 con 0x7fdaf413a380 2020-12-05T17:31:42.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdaf95ee700 1 --2- 172.21.15.32:0/4063451267 >> v2:172.21.15.61:3300/0 conn(0x7fdaf405f6f0 0x7fdaf4143d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:31:42.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdaf8ded700 1 --2- 172.21.15.32:0/4063451267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf413a380 0x7fdaf413a830 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fdaf0002f30 tx=0x7fdaf0006e80).ready entity=mon.0 client_cookie=26864d68dc6f8485 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:42.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdaebfff700 1 -- 172.21.15.32:0/4063451267 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdaf00097f0 con 0x7fdaf413a380 2020-12-05T17:31:42.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdaebfff700 1 -- 172.21.15.32:0/4063451267 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf0009950 con 0x7fdaf413a380 2020-12-05T17:31:42.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdaebfff700 1 -- 172.21.15.32:0/4063451267 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdaf0009cb0 con 0x7fdaf413a380 2020-12-05T17:31:42.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdafb852700 1 -- 172.21.15.32:0/4063451267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf413a380 msgr2=0x7fdaf413a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:42.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdafb852700 1 --2- 172.21.15.32:0/4063451267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf413a380 0x7fdaf413a830 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fdaf0002f30 tx=0x7fdaf0006e80).stop 2020-12-05T17:31:42.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdafb852700 1 -- 172.21.15.32:0/4063451267 shutdown_connections 2020-12-05T17:31:42.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdafb852700 1 --2- 172.21.15.32:0/4063451267 >> v2:172.21.15.61:3300/0 conn(0x7fdaf405f6f0 0x7fdaf4143d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdafb852700 1 --2- 172.21.15.32:0/4063451267 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdaf4065680 0x7fdaf4065af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdafb852700 1 --2- 172.21.15.32:0/4063451267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf413a380 0x7fdaf413a830 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdafb852700 1 -- 172.21.15.32:0/4063451267 >> 172.21.15.32:0/4063451267 conn(0x7fdaf40b6ef0 msgr2=0x7fdaf40b72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:42.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdafb852700 1 -- 172.21.15.32:0/4063451267 shutdown_connections 2020-12-05T17:31:42.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.759+0000 7fdafb852700 1 -- 172.21.15.32:0/4063451267 wait complete. 2020-12-05T17:31:42.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.763+0000 7fdafb852700 1 Processor -- start 2020-12-05T17:31:42.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.763+0000 7fdafb852700 1 -- start start 2020-12-05T17:31:42.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.763+0000 7fdafb852700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf405f6f0 0x7fdaf413f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:42.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.763+0000 7fdafb852700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdaf4065680 0x7fdaf413d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:42.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.763+0000 7fdafb852700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdaf413a380 0x7fdaf413dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:42.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.763+0000 7fdafb852700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf4152d00 con 0x7fdaf405f6f0 2020-12-05T17:31:42.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.763+0000 7fdafb852700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf40b15b0 con 0x7fdaf413a380 2020-12-05T17:31:42.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.763+0000 7fdafb852700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf4143050 con 0x7fdaf4065680 2020-12-05T17:31:42.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.763+0000 7fdaf95ee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf405f6f0 0x7fdaf413f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:42.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.763+0000 7fdaf95ee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf405f6f0 0x7fdaf413f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60668/0 (socket says 172.21.15.32:60668) 2020-12-05T17:31:42.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.763+0000 7fdaf95ee700 1 -- 172.21.15.32:0/1579830360 learned_addr learned my addr 172.21.15.32:0/1579830360 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:42.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.763+0000 7fdaf9def700 1 --2- 172.21.15.32:0/1579830360 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdaf413a380 0x7fdaf413dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:42.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.763+0000 7fdaf8ded700 1 --2- 172.21.15.32:0/1579830360 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdaf4065680 0x7fdaf413d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:42.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.763+0000 7fdaf95ee700 1 -- 172.21.15.32:0/1579830360 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdaf4065680 msgr2=0x7fdaf413d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:42.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.763+0000 7fdaf95ee700 1 --2- 172.21.15.32:0/1579830360 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdaf4065680 0x7fdaf413d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.763+0000 7fdaf95ee700 1 -- 172.21.15.32:0/1579830360 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdaf413a380 msgr2=0x7fdaf413dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:42.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.763+0000 7fdaf95ee700 1 --2- 172.21.15.32:0/1579830360 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdaf413a380 0x7fdaf413dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.763+0000 7fdaf95ee700 1 -- 172.21.15.32:0/1579830360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf41432f0 con 0x7fdaf405f6f0 2020-12-05T17:31:42.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.767+0000 7fdaf95ee700 1 --2- 172.21.15.32:0/1579830360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf405f6f0 0x7fdaf413f670 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fdaf40b1740 tx=0x7fdae4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:42.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.767+0000 7fdaea7fc700 1 -- 172.21.15.32:0/1579830360 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdae4010580 con 0x7fdaf405f6f0 2020-12-05T17:31:42.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.767+0000 7fdaea7fc700 1 -- 172.21.15.32:0/1579830360 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae40106e0 con 0x7fdaf405f6f0 2020-12-05T17:31:42.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.767+0000 7fdafb852700 1 -- 172.21.15.32:0/1579830360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf4143590 con 0x7fdaf405f6f0 2020-12-05T17:31:42.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.767+0000 7fdaea7fc700 1 -- 172.21.15.32:0/1579830360 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdae4010a40 con 0x7fdaf405f6f0 2020-12-05T17:31:42.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.767+0000 7fdafb852700 1 -- 172.21.15.32:0/1579830360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf413fd90 con 0x7fdaf405f6f0 2020-12-05T17:31:42.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.767+0000 7fdaea7fc700 1 -- 172.21.15.32:0/1579830360 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fdae4010ba0 con 0x7fdaf405f6f0 2020-12-05T17:31:42.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.767+0000 7fdafb852700 1 -- 172.21.15.32:0/1579830360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf4141f80 con 0x7fdaf405f6f0 2020-12-05T17:31:42.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.767+0000 7fdaea7fc700 1 --2- 172.21.15.32:0/1579830360 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdad4055440 0x7fdad4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:42.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.767+0000 7fdaea7fc700 1 -- 172.21.15.32:0/1579830360 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fdae406e1c0 con 0x7fdaf405f6f0 2020-12-05T17:31:42.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.767+0000 7fdaf8ded700 1 --2- 172.21.15.32:0/1579830360 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdad4055440 0x7fdad4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:42.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.771+0000 7fdaf8ded700 1 --2- 172.21.15.32:0/1579830360 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdad4055440 0x7fdad4057900 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fdaf001c000 tx=0x7fdaf0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:42.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.775+0000 7fdaea7fc700 1 -- 172.21.15.32:0/1579830360 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fdae40442c0 con 0x7fdaf405f6f0 2020-12-05T17:31:42.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.963+0000 7fdafb852700 1 -- 172.21.15.32:0/1579830360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdaf4152e50 con 0x7fdaf405f6f0 2020-12-05T17:31:42.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.967+0000 7fdaea7fc700 1 -- 172.21.15.32:0/1579830360 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2134 (secure 0 0 0) 0x7fdaf4152e50 con 0x7fdaf405f6f0 2020-12-05T17:31:42.969 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:31:42.970 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4023797731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4023797731},{"type":"v1","addr":"172.21.15.61:6833","nonce":4023797731}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:31:41.611214+0000","modified":"2020-12-05T17:31:42.614053+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3503848021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3503848021},{"type":"v1","addr":"172.21.15.32:6835","nonce":3503848021}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-12-05T17:31:42.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.967+0000 7fdafb852700 1 -- 172.21.15.32:0/1579830360 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdad4055440 msgr2=0x7fdad4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:42.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.967+0000 7fdafb852700 1 --2- 172.21.15.32:0/1579830360 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdad4055440 0x7fdad4057900 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fdaf001c000 tx=0x7fdaf0015000).stop 2020-12-05T17:31:42.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.967+0000 7fdafb852700 1 -- 172.21.15.32:0/1579830360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf405f6f0 msgr2=0x7fdaf413f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.967+0000 7fdafb852700 1 --2- 172.21.15.32:0/1579830360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf405f6f0 0x7fdaf413f670 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fdaf40b1740 tx=0x7fdae4008040).stop 2020-12-05T17:31:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.967+0000 7fdafb852700 1 -- 172.21.15.32:0/1579830360 shutdown_connections 2020-12-05T17:31:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.967+0000 7fdafb852700 1 --2- 172.21.15.32:0/1579830360 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdaf413a380 0x7fdaf413dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.967+0000 7fdafb852700 1 --2- 172.21.15.32:0/1579830360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf405f6f0 0x7fdaf413f670 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.967+0000 7fdafb852700 1 --2- 172.21.15.32:0/1579830360 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdad4055440 0x7fdad4057900 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.967+0000 7fdafb852700 1 --2- 172.21.15.32:0/1579830360 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdaf4065680 0x7fdaf413d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.967+0000 7fdafb852700 1 -- 172.21.15.32:0/1579830360 >> 172.21.15.32:0/1579830360 conn(0x7fdaf40b6ef0 msgr2=0x7fdaf4065af0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.967+0000 7fdafb852700 1 -- 172.21.15.32:0/1579830360 shutdown_connections 2020-12-05T17:31:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:42.967+0000 7fdafb852700 1 -- 172.21.15.32:0/1579830360 wait complete. 2020-12-05T17:31:42.974 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 5 2020-12-05T17:31:42.982 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:31:43.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c4672b700 1 Processor -- start 2020-12-05T17:31:43.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c4672b700 1 -- start start 2020-12-05T17:31:43.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c4672b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f3c400b00a0 0x7f3c40061910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c4672b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3c400acca0 0x7f3c4014a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c4672b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c400601e0 con 0x7f3c4005fe00 2020-12-05T17:31:43.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c4672b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c400ac1a0 con 0x7f3c400acca0 2020-12-05T17:31:43.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c4672b700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c40061fa0 con 0x7f3c400b00a0 2020-12-05T17:31:43.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3c4005fe00 0x7f3c400ac590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35092/0 (socket says 172.21.15.32:35092) 2020-12-05T17:31:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3ffff700 1 -- 172.21.15.32:0/2081274514 learned_addr learned my addr 172.21.15.32:0/2081274514 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c44cc8700 1 --2- 172.21.15.32:0/2081274514 >> v2:172.21.15.32:3300/0 conn(0x7f3c400acca0 0x7f3c4014a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3f7fe700 1 --2- 172.21.15.32:0/2081274514 >> v2:172.21.15.61:3300/0 conn(0x7f3c400b00a0 0x7f3c40061910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3effd700 1 -- 172.21.15.32:0/2081274514 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 681075080 0 0) 0x7f3c400601e0 con 0x7f3c4005fe00 2020-12-05T17:31:43.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3effd700 1 -- 172.21.15.32:0/2081274514 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c20003230 con 0x7f3c4005fe00 2020-12-05T17:31:43.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3effd700 1 -- 172.21.15.32:0/2081274514 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2206345312 0 0) 0x7f3c20003230 con 0x7f3c4005fe00 2020-12-05T17:31:43.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3effd700 1 -- 172.21.15.32:0/2081274514 >> v2:172.21.15.61:3300/0 conn(0x7f3c400b00a0 msgr2=0x7f3c40061910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3effd700 1 --2- 172.21.15.32:0/2081274514 >> v2:172.21.15.61:3300/0 conn(0x7f3c400b00a0 0x7f3c40061910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3effd700 1 -- 172.21.15.32:0/2081274514 >> v2:172.21.15.32:3300/0 conn(0x7f3c400acca0 msgr2=0x7f3c4014a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3effd700 1 --2- 172.21.15.32:0/2081274514 >> v2:172.21.15.32:3300/0 conn(0x7f3c400acca0 0x7f3c4014a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3effd700 1 -- 172.21.15.32:0/2081274514 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c40062240 con 0x7f3c4005fe00 2020-12-05T17:31:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3effd700 1 -- 172.21.15.32:0/2081274514 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f3c30001a50 con 0x7f3c4005fe00 2020-12-05T17:31:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3effd700 1 -- 172.21.15.32:0/2081274514 >> v1:172.21.15.32:6789/0 conn(0x7f3c4005fe00 legacy=0x7f3c400ac590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3effd700 1 --2- 172.21.15.32:0/2081274514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c20003720 0x7f3c20003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3effd700 1 --2- 172.21.15.32:0/2081274514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c200044d0 0x7f3c20004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3effd700 1 --2- 172.21.15.32:0/2081274514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3c20004e40 0x7f3c2000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3effd700 1 -- 172.21.15.32:0/2081274514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c40061fa0 con 0x7f3c20003720 2020-12-05T17:31:43.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3effd700 1 -- 172.21.15.32:0/2081274514 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c20004250 con 0x7f3c200044d0 2020-12-05T17:31:43.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3effd700 1 -- 172.21.15.32:0/2081274514 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2000bc10 con 0x7f3c20004e40 2020-12-05T17:31:43.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3ffff700 1 --2- 172.21.15.32:0/2081274514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c20003720 0x7f3c20003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3f7fe700 1 --2- 172.21.15.32:0/2081274514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c200044d0 0x7f3c20004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c44cc8700 1 --2- 172.21.15.32:0/2081274514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3c20004e40 0x7f3c2000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3ffff700 1 -- 172.21.15.32:0/2081274514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3c20004e40 msgr2=0x7f3c2000b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3ffff700 1 --2- 172.21.15.32:0/2081274514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3c20004e40 0x7f3c2000b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3ffff700 1 -- 172.21.15.32:0/2081274514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c200044d0 msgr2=0x7f3c20004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3ffff700 1 --2- 172.21.15.32:0/2081274514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c200044d0 0x7f3c20004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3ffff700 1 -- 172.21.15.32:0/2081274514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c20003230 con 0x7f3c20003720 2020-12-05T17:31:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c44cc8700 1 -- 172.21.15.32:0/2081274514 reap_dead start 2020-12-05T17:31:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3ffff700 1 --2- 172.21.15.32:0/2081274514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c20003720 0x7f3c20003b40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f3c30001760 tx=0x7f3c400b15a0).ready entity=mon.0 client_cookie=b894d03301911244 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3effd700 1 -- 172.21.15.32:0/2081274514 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c30005b70 con 0x7f3c20003720 2020-12-05T17:31:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3effd700 1 -- 172.21.15.32:0/2081274514 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c30005cd0 con 0x7f3c20003720 2020-12-05T17:31:43.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c3effd700 1 -- 172.21.15.32:0/2081274514 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c30007300 con 0x7f3c20003720 2020-12-05T17:31:43.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c4672b700 1 -- 172.21.15.32:0/2081274514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c20003720 msgr2=0x7f3c20003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c4672b700 1 --2- 172.21.15.32:0/2081274514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c20003720 0x7f3c20003b40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f3c30001760 tx=0x7f3c400b15a0).stop 2020-12-05T17:31:43.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c4672b700 1 -- 172.21.15.32:0/2081274514 shutdown_connections 2020-12-05T17:31:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c4672b700 1 --2- 172.21.15.32:0/2081274514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c20003720 0x7f3c20003b40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c4672b700 1 -- 172.21.15.32:0/2081274514 >> 172.21.15.32:0/2081274514 conn(0x7f3c400b57c0 msgr2=0x7f3c4005f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c4672b700 1 -- 172.21.15.32:0/2081274514 shutdown_connections 2020-12-05T17:31:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.095+0000 7f3c4672b700 1 -- 172.21.15.32:0/2081274514 wait complete. 2020-12-05T17:31:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c4672b700 1 Processor -- start 2020-12-05T17:31:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c4672b700 1 -- start start 2020-12-05T17:31:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c4672b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3c20003720 0x7f3c40158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c4672b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c400a19c0 0x7f3c40146120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c4672b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c40146610 0x7f3c40155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c4672b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400664a0 con 0x7f3c40146610 2020-12-05T17:31:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c4672b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400666b0 con 0x7f3c400a19c0 2020-12-05T17:31:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c4672b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c401598c0 con 0x7f3c20003720 2020-12-05T17:31:43.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c44cc8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c40146610 0x7f3c40155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c44cc8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c40146610 0x7f3c40155740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60692/0 (socket says 172.21.15.32:60692) 2020-12-05T17:31:43.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c44cc8700 1 -- 172.21.15.32:0/3040026492 learned_addr learned my addr 172.21.15.32:0/3040026492 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:43.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c3ffff700 1 --2- 172.21.15.32:0/3040026492 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3c20003720 0x7f3c40158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c3f7fe700 1 --2- 172.21.15.32:0/3040026492 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c400a19c0 0x7f3c40146120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c44cc8700 1 -- 172.21.15.32:0/3040026492 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3c20003720 msgr2=0x7f3c40158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c44cc8700 1 --2- 172.21.15.32:0/3040026492 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3c20003720 0x7f3c40158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c44cc8700 1 -- 172.21.15.32:0/3040026492 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c400a19c0 msgr2=0x7f3c40146120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c44cc8700 1 --2- 172.21.15.32:0/3040026492 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c400a19c0 0x7f3c40146120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c44cc8700 1 -- 172.21.15.32:0/3040026492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c40146a90 con 0x7f3c40146610 2020-12-05T17:31:43.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c3f7fe700 1 --2- 172.21.15.32:0/3040026492 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c400a19c0 0x7f3c40146120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:43.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c44cc8700 1 --2- 172.21.15.32:0/3040026492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c40146610 0x7f3c40155740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f3c38008000 tx=0x7f3c38008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:43.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c3d7fa700 1 -- 172.21.15.32:0/3040026492 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c38012610 con 0x7f3c40146610 2020-12-05T17:31:43.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c3d7fa700 1 -- 172.21.15.32:0/3040026492 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c38012770 con 0x7f3c40146610 2020-12-05T17:31:43.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c4672b700 1 -- 172.21.15.32:0/3040026492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c40159620 con 0x7f3c40146610 2020-12-05T17:31:43.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.099+0000 7f3c3d7fa700 1 -- 172.21.15.32:0/3040026492 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c380128e0 con 0x7f3c40146610 2020-12-05T17:31:43.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.103+0000 7f3c4672b700 1 -- 172.21.15.32:0/3040026492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c400a0ae0 con 0x7f3c40146610 2020-12-05T17:31:43.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.103+0000 7f3c4672b700 1 -- 172.21.15.32:0/3040026492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c40066cb0 con 0x7f3c40146610 2020-12-05T17:31:43.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.103+0000 7f3c3d7fa700 1 -- 172.21.15.32:0/3040026492 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3c38012a40 con 0x7f3c40146610 2020-12-05T17:31:43.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.103+0000 7f3c3d7fa700 1 --2- 172.21.15.32:0/3040026492 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3c24055330 0x7f3c240577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.103+0000 7f3c3d7fa700 1 -- 172.21.15.32:0/3040026492 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f3c3806ed30 con 0x7f3c40146610 2020-12-05T17:31:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.103+0000 7f3c3ffff700 1 --2- 172.21.15.32:0/3040026492 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3c24055330 0x7f3c240577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.103+0000 7f3c3ffff700 1 --2- 172.21.15.32:0/3040026492 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3c24055330 0x7f3c240577f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f3c30019000 tx=0x7f3c30014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.111+0000 7f3c3d7fa700 1 -- 172.21.15.32:0/3040026492 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3c38044ed0 con 0x7f3c40146610 2020-12-05T17:31:43.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.251+0000 7f3c4672b700 1 -- 172.21.15.32:0/3040026492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c40057050 con 0x7f3c40146610 2020-12-05T17:31:43.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.255+0000 7f3c3d7fa700 1 -- 172.21.15.32:0/3040026492 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f3c380484f0 con 0x7f3c40146610 2020-12-05T17:31:43.257 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:31:43.257 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:31:41.610914+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:31:36.706361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-12-05T17:31:39.298269+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:31:43.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.255+0000 7f3c4672b700 1 -- 172.21.15.32:0/3040026492 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3c24055330 msgr2=0x7f3c240577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.255+0000 7f3c4672b700 1 --2- 172.21.15.32:0/3040026492 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3c24055330 0x7f3c240577f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f3c30019000 tx=0x7f3c30014000).stop 2020-12-05T17:31:43.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.255+0000 7f3c4672b700 1 -- 172.21.15.32:0/3040026492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c40146610 msgr2=0x7f3c40155740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.255+0000 7f3c4672b700 1 --2- 172.21.15.32:0/3040026492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c40146610 0x7f3c40155740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f3c38008000 tx=0x7f3c38008040).stop 2020-12-05T17:31:43.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.255+0000 7f3c4672b700 1 -- 172.21.15.32:0/3040026492 shutdown_connections 2020-12-05T17:31:43.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.255+0000 7f3c4672b700 1 --2- 172.21.15.32:0/3040026492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c40146610 0x7f3c40155740 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.255+0000 7f3c4672b700 1 --2- 172.21.15.32:0/3040026492 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3c24055330 0x7f3c240577f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.255+0000 7f3c4672b700 1 --2- 172.21.15.32:0/3040026492 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3c20003720 0x7f3c40158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.255+0000 7f3c4672b700 1 --2- 172.21.15.32:0/3040026492 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c400a19c0 0x7f3c40146120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.255+0000 7f3c4672b700 1 -- 172.21.15.32:0/3040026492 >> 172.21.15.32:0/3040026492 conn(0x7f3c400b57c0 msgr2=0x7f3c400b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:43.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.255+0000 7f3c4672b700 1 -- 172.21.15.32:0/3040026492 shutdown_connections 2020-12-05T17:31:43.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.255+0000 7f3c4672b700 1 -- 172.21.15.32:0/3040026492 wait complete. 2020-12-05T17:31:43.278 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:31:43.279 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2020-12-05T17:31:43.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549359e700 1 Processor -- start 2020-12-05T17:31:43.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549359e700 1 -- start start 2020-12-05T17:31:43.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549359e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f549405feb0 0x7f549405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5494142450 con 0x7f549405f3b0 2020-12-05T17:31:43.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549359e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5494061780 con 0x7f5494062400 2020-12-05T17:31:43.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549359e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5494061a20 con 0x7f549405feb0 2020-12-05T17:31:43.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f549405f3b0 0x7f54940b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35112/0 (socket says 172.21.15.32:35112) 2020-12-05T17:31:43.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549259c700 1 -- 172.21.15.32:0/2972547819 learned_addr learned my addr 172.21.15.32:0/2972547819 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:43.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f5492d9d700 1 --2- 172.21.15.32:0/2972547819 >> v2:172.21.15.61:3300/0 conn(0x7f549405feb0 0x7f549405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549159a700 1 -- 172.21.15.32:0/2972547819 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625604436 0 0) 0x7f5494142450 con 0x7f549405f3b0 2020-12-05T17:31:43.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549159a700 1 -- 172.21.15.32:0/2972547819 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5478003230 con 0x7f549405f3b0 2020-12-05T17:31:43.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549159a700 1 -- 172.21.15.32:0/2972547819 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1280567448 0 0) 0x7f5478003230 con 0x7f549405f3b0 2020-12-05T17:31:43.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549159a700 1 -- 172.21.15.32:0/2972547819 >> v2:172.21.15.61:3300/0 conn(0x7f549405feb0 msgr2=0x7f549405f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:43.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549159a700 1 --2- 172.21.15.32:0/2972547819 >> v2:172.21.15.61:3300/0 conn(0x7f549405feb0 0x7f549405f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549159a700 1 -- 172.21.15.32:0/2972547819 >> v1:172.21.15.61:6789/0 conn(0x7f5494062400 legacy=0x7f54940b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549159a700 1 -- 172.21.15.32:0/2972547819 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5494061cc0 con 0x7f549405f3b0 2020-12-05T17:31:43.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549159a700 1 -- 172.21.15.32:0/2972547819 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f548c001a50 con 0x7f549405f3b0 2020-12-05T17:31:43.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549159a700 1 -- 172.21.15.32:0/2972547819 >> v1:172.21.15.32:6789/0 conn(0x7f549405f3b0 legacy=0x7f54940b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549159a700 1 --2- 172.21.15.32:0/2972547819 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5478003720 0x7f5478003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549159a700 1 --2- 172.21.15.32:0/2972547819 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f54780044d0 0x7f5478004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549159a700 1 --2- 172.21.15.32:0/2972547819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5478004e40 0x7f547800b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549159a700 1 -- 172.21.15.32:0/2972547819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5494061a20 con 0x7f5478004e40 2020-12-05T17:31:43.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549159a700 1 -- 172.21.15.32:0/2972547819 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5478004250 con 0x7f54780044d0 2020-12-05T17:31:43.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549159a700 1 -- 172.21.15.32:0/2972547819 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547800bc10 con 0x7f5478003720 2020-12-05T17:31:43.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f5492d9d700 1 --2- 172.21.15.32:0/2972547819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5478004e40 0x7f547800b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f5491d9b700 1 --2- 172.21.15.32:0/2972547819 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f54780044d0 0x7f5478004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549259c700 1 --2- 172.21.15.32:0/2972547819 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5478003720 0x7f5478003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f5492d9d700 1 -- 172.21.15.32:0/2972547819 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5478003720 msgr2=0x7f5478003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:43.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f5492d9d700 1 --2- 172.21.15.32:0/2972547819 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5478003720 0x7f5478003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f5492d9d700 1 -- 172.21.15.32:0/2972547819 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f54780044d0 msgr2=0x7f5478004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:43.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f5492d9d700 1 --2- 172.21.15.32:0/2972547819 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f54780044d0 0x7f5478004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f5492d9d700 1 -- 172.21.15.32:0/2972547819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5478003230 con 0x7f5478004e40 2020-12-05T17:31:43.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f5491d9b700 1 --2- 172.21.15.32:0/2972547819 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f54780044d0 0x7f5478004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:43.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549259c700 1 --2- 172.21.15.32:0/2972547819 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5478003720 0x7f5478003b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:31:43.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f5492d9d700 1 -- 172.21.15.32:0/2972547819 reap_dead start 2020-12-05T17:31:43.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f5492d9d700 1 --2- 172.21.15.32:0/2972547819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5478004e40 0x7f547800b720 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f548400c000 tx=0x7f5484008040).ready entity=mon.0 client_cookie=abfff25db107faa server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:43.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549159a700 1 -- 172.21.15.32:0/2972547819 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f548400e9d0 con 0x7f5478004e40 2020-12-05T17:31:43.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549159a700 1 -- 172.21.15.32:0/2972547819 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548400eb30 con 0x7f5478004e40 2020-12-05T17:31:43.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549359e700 1 -- 172.21.15.32:0/2972547819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5478004e40 msgr2=0x7f547800b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549159a700 1 -- 172.21.15.32:0/2972547819 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f548400ee90 con 0x7f5478004e40 2020-12-05T17:31:43.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.387+0000 7f549359e700 1 --2- 172.21.15.32:0/2972547819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5478004e40 0x7f547800b720 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f548400c000 tx=0x7f5484008040).stop 2020-12-05T17:31:43.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f549359e700 1 -- 172.21.15.32:0/2972547819 shutdown_connections 2020-12-05T17:31:43.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f549359e700 1 --2- 172.21.15.32:0/2972547819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5478004e40 0x7f547800b720 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f549359e700 1 -- 172.21.15.32:0/2972547819 >> 172.21.15.32:0/2972547819 conn(0x7f54940b8ee0 msgr2=0x7f54940b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:43.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f549359e700 1 -- 172.21.15.32:0/2972547819 shutdown_connections 2020-12-05T17:31:43.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f549359e700 1 -- 172.21.15.32:0/2972547819 wait complete. 2020-12-05T17:31:43.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f549359e700 1 Processor -- start 2020-12-05T17:31:43.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f549359e700 1 -- start start 2020-12-05T17:31:43.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f549359e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5478004e40 0x7f5494140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f549359e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5494063000 0x7f54941320a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f549359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5494132590 0x7f5494064810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f549359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5494144a70 con 0x7f5494132590 2020-12-05T17:31:43.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f549359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5494146590 con 0x7f5478004e40 2020-12-05T17:31:43.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f549359e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54940634d0 con 0x7f5494063000 2020-12-05T17:31:43.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f5492d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5494132590 0x7f5494064810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f5492d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5494132590 0x7f5494064810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60712/0 (socket says 172.21.15.32:60712) 2020-12-05T17:31:43.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f5492d9d700 1 -- 172.21.15.32:0/9508350 learned_addr learned my addr 172.21.15.32:0/9508350 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:43.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f549259c700 1 --2- 172.21.15.32:0/9508350 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5478004e40 0x7f5494140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f5491d9b700 1 --2- 172.21.15.32:0/9508350 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5494063000 0x7f54941320a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f5492d9d700 1 -- 172.21.15.32:0/9508350 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5494063000 msgr2=0x7f54941320a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f5492d9d700 1 --2- 172.21.15.32:0/9508350 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5494063000 0x7f54941320a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f5492d9d700 1 -- 172.21.15.32:0/9508350 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5478004e40 msgr2=0x7f5494140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f5492d9d700 1 --2- 172.21.15.32:0/9508350 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5478004e40 0x7f5494140ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f5492d9d700 1 -- 172.21.15.32:0/9508350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5494132a10 con 0x7f5494132590 2020-12-05T17:31:43.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f5492d9d700 1 --2- 172.21.15.32:0/9508350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5494132590 0x7f5494064810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f548400c000 tx=0x7f5484019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:43.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f54837fe700 1 -- 172.21.15.32:0/9508350 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5484020070 con 0x7f5494132590 2020-12-05T17:31:43.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f54837fe700 1 -- 172.21.15.32:0/9508350 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548401c440 con 0x7f5494132590 2020-12-05T17:31:43.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f549359e700 1 -- 172.21.15.32:0/9508350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54940af880 con 0x7f5494132590 2020-12-05T17:31:43.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f54837fe700 1 -- 172.21.15.32:0/9508350 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f548401c7a0 con 0x7f5494132590 2020-12-05T17:31:43.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.391+0000 7f549359e700 1 -- 172.21.15.32:0/9508350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5494064ea0 con 0x7f5494132590 2020-12-05T17:31:43.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.395+0000 7f549359e700 1 -- 172.21.15.32:0/9508350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5494132ba0 con 0x7f5494132590 2020-12-05T17:31:43.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.395+0000 7f54837fe700 1 -- 172.21.15.32:0/9508350 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5484018020 con 0x7f5494132590 2020-12-05T17:31:43.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.395+0000 7f54837fe700 1 --2- 172.21.15.32:0/9508350 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5470055500 0x7f54700579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.395+0000 7f549259c700 1 --2- 172.21.15.32:0/9508350 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5470055500 0x7f54700579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.395+0000 7f54837fe700 1 -- 172.21.15.32:0/9508350 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f548406ef10 con 0x7f5494132590 2020-12-05T17:31:43.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.395+0000 7f549259c700 1 --2- 172.21.15.32:0/9508350 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5470055500 0x7f54700579c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f548c00c000 tx=0x7f548c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:43.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.403+0000 7f54837fe700 1 -- 172.21.15.32:0/9508350 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5484044f20 con 0x7f5494132590 2020-12-05T17:31:43.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.543+0000 7f549359e700 1 -- 172.21.15.32:0/9508350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5494057050 con 0x7f5494132590 2020-12-05T17:31:43.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.547+0000 7f54837fe700 1 -- 172.21.15.32:0/9508350 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f5484048540 con 0x7f5494132590 2020-12-05T17:31:43.549 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:31:43.550 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:31:41.610914+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:31:36.706361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-12-05T17:31:39.298269+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:31:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.547+0000 7f549359e700 1 -- 172.21.15.32:0/9508350 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5470055500 msgr2=0x7f54700579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.547+0000 7f549359e700 1 --2- 172.21.15.32:0/9508350 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5470055500 0x7f54700579c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f548c00c000 tx=0x7f548c009040).stop 2020-12-05T17:31:43.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.547+0000 7f549359e700 1 -- 172.21.15.32:0/9508350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5494132590 msgr2=0x7f5494064810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.547+0000 7f549359e700 1 --2- 172.21.15.32:0/9508350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5494132590 0x7f5494064810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f548400c000 tx=0x7f5484019000).stop 2020-12-05T17:31:43.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.547+0000 7f549359e700 1 -- 172.21.15.32:0/9508350 shutdown_connections 2020-12-05T17:31:43.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.547+0000 7f549359e700 1 --2- 172.21.15.32:0/9508350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5494132590 0x7f5494064810 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.547+0000 7f549359e700 1 --2- 172.21.15.32:0/9508350 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5478004e40 0x7f5494140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.547+0000 7f549359e700 1 --2- 172.21.15.32:0/9508350 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5470055500 0x7f54700579c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.547+0000 7f549359e700 1 --2- 172.21.15.32:0/9508350 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5494063000 0x7f54941320a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.547+0000 7f549359e700 1 -- 172.21.15.32:0/9508350 >> 172.21.15.32:0/9508350 conn(0x7f54940b8ee0 msgr2=0x7f54940b76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:43.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.547+0000 7f549359e700 1 -- 172.21.15.32:0/9508350 shutdown_connections 2020-12-05T17:31:43.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.547+0000 7f549359e700 1 -- 172.21.15.32:0/9508350 wait complete. 2020-12-05T17:31:43.563 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:31:36.706361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:31:39.298269+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:31:43.564 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:31:43.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.679+0000 7f01c2bd0700 1 Processor -- start 2020-12-05T17:31:43.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.679+0000 7f01c2bd0700 1 -- start start 2020-12-05T17:31:43.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c2bd0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01bc0af0d0 0x7f01bc13daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c2bd0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01bc05f5c0 con 0x7f01bc0b0d70 2020-12-05T17:31:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c2bd0700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01bc0af4b0 con 0x7f01bc13df90 2020-12-05T17:31:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c2bd0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc05f7f0 con 0x7f01bc0af0d0 2020-12-05T17:31:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c23cf700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f01bc0b0d70 0x7f01bc063f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35136/0 (socket says 172.21.15.32:35136) 2020-12-05T17:31:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c23cf700 1 -- 172.21.15.32:0/2773517062 learned_addr learned my addr 172.21.15.32:0/2773517062 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c1bce700 1 --2- 172.21.15.32:0/2773517062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01bc0af0d0 0x7f01bc13daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c0bcc700 1 -- 172.21.15.32:0/2773517062 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3826975072 0 0) 0x7f01bc05f5c0 con 0x7f01bc0b0d70 2020-12-05T17:31:43.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c0bcc700 1 -- 172.21.15.32:0/2773517062 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01a4003230 con 0x7f01bc0b0d70 2020-12-05T17:31:43.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c0bcc700 1 -- 172.21.15.32:0/2773517062 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 681653092 0 0) 0x7f01bc0af4b0 con 0x7f01bc13df90 2020-12-05T17:31:43.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c0bcc700 1 -- 172.21.15.32:0/2773517062 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01bc05f5c0 con 0x7f01bc13df90 2020-12-05T17:31:43.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c0bcc700 1 -- 172.21.15.32:0/2773517062 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 987127745 0 0) 0x7f01a4003230 con 0x7f01bc0b0d70 2020-12-05T17:31:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c0bcc700 1 -- 172.21.15.32:0/2773517062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01bc0af0d0 msgr2=0x7f01bc13daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c0bcc700 1 --2- 172.21.15.32:0/2773517062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01bc0af0d0 0x7f01bc13daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c0bcc700 1 -- 172.21.15.32:0/2773517062 >> v1:172.21.15.61:6789/0 conn(0x7f01bc13df90 legacy=0x7f01bc0637f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c0bcc700 1 -- 172.21.15.32:0/2773517062 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01bc066ae0 con 0x7f01bc0b0d70 2020-12-05T17:31:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c0bcc700 1 -- 172.21.15.32:0/2773517062 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f01b4001a50 con 0x7f01bc0b0d70 2020-12-05T17:31:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c0bcc700 1 -- 172.21.15.32:0/2773517062 >> v1:172.21.15.32:6789/0 conn(0x7f01bc0b0d70 legacy=0x7f01bc063f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c0bcc700 1 --2- 172.21.15.32:0/2773517062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01bc0af0d0 0x7f01a40038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c0bcc700 1 --2- 172.21.15.32:0/2773517062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a4003dd0 0x7f01a40044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c0bcc700 1 --2- 172.21.15.32:0/2773517062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01a40049c0 0x7f01a4004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c0bcc700 1 -- 172.21.15.32:0/2773517062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc05f7f0 con 0x7f01a4003dd0 2020-12-05T17:31:43.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c0bcc700 1 -- 172.21.15.32:0/2773517062 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a4005360 con 0x7f01a40049c0 2020-12-05T17:31:43.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c0bcc700 1 -- 172.21.15.32:0/2773517062 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a400fb80 con 0x7f01bc0af0d0 2020-12-05T17:31:43.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c13cd700 1 --2- 172.21.15.32:0/2773517062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a4003dd0 0x7f01a40044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c1bce700 1 --2- 172.21.15.32:0/2773517062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01bc0af0d0 0x7f01a40038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c13cd700 1 -- 172.21.15.32:0/2773517062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01bc0af0d0 msgr2=0x7f01a40038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:43.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c13cd700 1 --2- 172.21.15.32:0/2773517062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01bc0af0d0 0x7f01a40038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c13cd700 1 -- 172.21.15.32:0/2773517062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01a40049c0 msgr2=0x7f01a4004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:43.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c23cf700 1 --2- 172.21.15.32:0/2773517062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01a40049c0 0x7f01a4004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c13cd700 1 --2- 172.21.15.32:0/2773517062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01a40049c0 0x7f01a4004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c13cd700 1 -- 172.21.15.32:0/2773517062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a4003230 con 0x7f01a4003dd0 2020-12-05T17:31:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c23cf700 1 --2- 172.21.15.32:0/2773517062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01a40049c0 0x7f01a4004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:31:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c13cd700 1 --2- 172.21.15.32:0/2773517062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a4003dd0 0x7f01a40044d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f01a40054a0 tx=0x7f01ac007b60).ready entity=mon.0 client_cookie=5739d4ed4eae7539 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c0bcc700 1 -- 172.21.15.32:0/2773517062 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01ac00c7d0 con 0x7f01a4003dd0 2020-12-05T17:31:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c0bcc700 1 -- 172.21.15.32:0/2773517062 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ac00c930 con 0x7f01a4003dd0 2020-12-05T17:31:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c0bcc700 1 -- 172.21.15.32:0/2773517062 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01ac00cc90 con 0x7f01a4003dd0 2020-12-05T17:31:43.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c2bd0700 1 -- 172.21.15.32:0/2773517062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a4003dd0 msgr2=0x7f01a40044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c2bd0700 1 --2- 172.21.15.32:0/2773517062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a4003dd0 0x7f01a40044d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f01a40054a0 tx=0x7f01ac007b60).stop 2020-12-05T17:31:43.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c23cf700 1 -- 172.21.15.32:0/2773517062 reap_dead start 2020-12-05T17:31:43.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.683+0000 7f01c2bd0700 1 -- 172.21.15.32:0/2773517062 shutdown_connections 2020-12-05T17:31:43.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.687+0000 7f01c2bd0700 1 -- 172.21.15.32:0/2773517062 >> 172.21.15.32:0/2773517062 conn(0x7f01bc0620b0 msgr2=0x7f01bc062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:43.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.687+0000 7f01c2bd0700 1 -- 172.21.15.32:0/2773517062 shutdown_connections 2020-12-05T17:31:43.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.687+0000 7f01c2bd0700 1 -- 172.21.15.32:0/2773517062 wait complete. 2020-12-05T17:31:43.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.687+0000 7f01c2bd0700 1 Processor -- start 2020-12-05T17:31:43.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.687+0000 7f01c2bd0700 1 -- start start 2020-12-05T17:31:43.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.691+0000 7f01c2bd0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01bc13e510 0x7f01bc13f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.691+0000 7f01c2bd0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc1405e0 0x7f01bc150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.691+0000 7f01c2bd0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01bc0b57c0 0x7f01bc0b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.691+0000 7f01c2bd0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc140e30 con 0x7f01bc1405e0 2020-12-05T17:31:43.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.691+0000 7f01c2bd0700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc0b6160 con 0x7f01bc13e510 2020-12-05T17:31:43.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.691+0000 7f01c2bd0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc139ef0 con 0x7f01bc0b57c0 2020-12-05T17:31:43.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.691+0000 7f01c13cd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc1405e0 0x7f01bc150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.691+0000 7f01c13cd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc1405e0 0x7f01bc150970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60730/0 (socket says 172.21.15.32:60730) 2020-12-05T17:31:43.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.691+0000 7f01c13cd700 1 -- 172.21.15.32:0/2245725659 learned_addr learned my addr 172.21.15.32:0/2245725659 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:43.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.691+0000 7f01c23cf700 1 --2- 172.21.15.32:0/2245725659 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01bc0b57c0 0x7f01bc0b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.691+0000 7f01c1bce700 1 --2- 172.21.15.32:0/2245725659 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01bc13e510 0x7f01bc13f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.691+0000 7f01c13cd700 1 -- 172.21.15.32:0/2245725659 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01bc0b57c0 msgr2=0x7f01bc0b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.691+0000 7f01c13cd700 1 --2- 172.21.15.32:0/2245725659 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01bc0b57c0 0x7f01bc0b5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.691+0000 7f01c13cd700 1 -- 172.21.15.32:0/2245725659 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01bc13e510 msgr2=0x7f01bc13f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.691+0000 7f01c13cd700 1 --2- 172.21.15.32:0/2245725659 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01bc13e510 0x7f01bc13f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.691+0000 7f01c13cd700 1 -- 172.21.15.32:0/2245725659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01bc13a190 con 0x7f01bc1405e0 2020-12-05T17:31:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.691+0000 7f01c23cf700 1 --2- 172.21.15.32:0/2245725659 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01bc0b57c0 0x7f01bc0b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:31:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.691+0000 7f01c13cd700 1 --2- 172.21.15.32:0/2245725659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc1405e0 0x7f01bc150970 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f01a40054a0 tx=0x7f01ac017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.691+0000 7f01aaffd700 1 -- 172.21.15.32:0/2245725659 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01ac001ea0 con 0x7f01bc1405e0 2020-12-05T17:31:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.691+0000 7f01aaffd700 1 -- 172.21.15.32:0/2245725659 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ac012970 con 0x7f01bc1405e0 2020-12-05T17:31:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.691+0000 7f01c2bd0700 1 -- 172.21.15.32:0/2245725659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01bc139b70 con 0x7f01bc1405e0 2020-12-05T17:31:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.691+0000 7f01aaffd700 1 -- 172.21.15.32:0/2245725659 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01ac00cee0 con 0x7f01bc1405e0 2020-12-05T17:31:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.691+0000 7f01c2bd0700 1 -- 172.21.15.32:0/2245725659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01bc0b38f0 con 0x7f01bc1405e0 2020-12-05T17:31:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.695+0000 7f01c2bd0700 1 -- 172.21.15.32:0/2245725659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01bc057050 con 0x7f01bc1405e0 2020-12-05T17:31:43.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.695+0000 7f01aaffd700 1 -- 172.21.15.32:0/2245725659 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f01ac010070 con 0x7f01bc1405e0 2020-12-05T17:31:43.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.695+0000 7f01aaffd700 1 --2- 172.21.15.32:0/2245725659 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f019c0553b0 0x7f019c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.695+0000 7f01aaffd700 1 -- 172.21.15.32:0/2245725659 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f01ac06e890 con 0x7f01bc1405e0 2020-12-05T17:31:43.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.695+0000 7f01c1bce700 1 --2- 172.21.15.32:0/2245725659 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f019c0553b0 0x7f019c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.695+0000 7f01c1bce700 1 --2- 172.21.15.32:0/2245725659 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f019c0553b0 0x7f019c057870 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f01b8008000 tx=0x7f01b8008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:43.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.699+0000 7f01aaffd700 1 -- 172.21.15.32:0/2245725659 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f01ac044a30 con 0x7f01bc1405e0 2020-12-05T17:31:43.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.839+0000 7f01c2bd0700 1 -- 172.21.15.32:0/2245725659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f01bc05fee0 con 0x7f01bc1405e0 2020-12-05T17:31:43.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.839+0000 7f01aaffd700 1 -- 172.21.15.32:0/2245725659 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7f01ac048050 con 0x7f01bc1405e0 2020-12-05T17:31:43.844 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:31:43.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.843+0000 7f01c2bd0700 1 -- 172.21.15.32:0/2245725659 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f019c0553b0 msgr2=0x7f019c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.843+0000 7f01c2bd0700 1 --2- 172.21.15.32:0/2245725659 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f019c0553b0 0x7f019c057870 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f01b8008000 tx=0x7f01b8008040).stop 2020-12-05T17:31:43.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.843+0000 7f01c2bd0700 1 -- 172.21.15.32:0/2245725659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc1405e0 msgr2=0x7f01bc150970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.843+0000 7f01c2bd0700 1 --2- 172.21.15.32:0/2245725659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc1405e0 0x7f01bc150970 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f01a40054a0 tx=0x7f01ac017000).stop 2020-12-05T17:31:43.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.843+0000 7f01c2bd0700 1 -- 172.21.15.32:0/2245725659 shutdown_connections 2020-12-05T17:31:43.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.843+0000 7f01c2bd0700 1 --2- 172.21.15.32:0/2245725659 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f019c0553b0 0x7f019c057870 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.843+0000 7f01c2bd0700 1 --2- 172.21.15.32:0/2245725659 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01bc0b57c0 0x7f01bc0b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.843+0000 7f01c2bd0700 1 --2- 172.21.15.32:0/2245725659 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01bc13e510 0x7f01bc13f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.843+0000 7f01c2bd0700 1 --2- 172.21.15.32:0/2245725659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc1405e0 0x7f01bc150970 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.843+0000 7f01c2bd0700 1 -- 172.21.15.32:0/2245725659 >> 172.21.15.32:0/2245725659 conn(0x7f01bc0620b0 msgr2=0x7f01bc0adde0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:43.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.843+0000 7f01c2bd0700 1 -- 172.21.15.32:0/2245725659 shutdown_connections 2020-12-05T17:31:43.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.843+0000 7f01c2bd0700 1 -- 172.21.15.32:0/2245725659 wait complete. 2020-12-05T17:31:43.858 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:31:43.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.975+0000 7fc81f234700 1 Processor -- start 2020-12-05T17:31:43.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.975+0000 7fc81f234700 1 -- start start 2020-12-05T17:31:43.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81f234700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8180af9a0 0x7fc818144610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81f234700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc81813f5b0 0x7fc81813f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81f234700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc818144b00 con 0x7fc8180af5c0 2020-12-05T17:31:43.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81f234700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8181405f0 con 0x7fc8180af9a0 2020-12-05T17:31:43.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81f234700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8181407e0 con 0x7fc81813f5b0 2020-12-05T17:31:43.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81da31700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8180af9a0 0x7fc818144610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81da31700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8180af9a0 0x7fc818144610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60738/0 (socket says 172.21.15.32:60738) 2020-12-05T17:31:43.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81da31700 1 -- 172.21.15.32:0/3931215468 learned_addr learned my addr 172.21.15.32:0/3931215468 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:43.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81ea33700 1 --2- 172.21.15.32:0/3931215468 >> v2:172.21.15.61:3300/0 conn(0x7fc81813f5b0 0x7fc81813f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81da31700 1 -- 172.21.15.32:0/3931215468 >> v2:172.21.15.61:3300/0 conn(0x7fc81813f5b0 msgr2=0x7fc81813f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:43.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81da31700 1 --2- 172.21.15.32:0/3931215468 >> v2:172.21.15.61:3300/0 conn(0x7fc81813f5b0 0x7fc81813f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81da31700 1 -- 172.21.15.32:0/3931215468 >> v1:172.21.15.61:6789/0 conn(0x7fc8180af5c0 legacy=0x7fc818143f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81da31700 1 -- 172.21.15.32:0/3931215468 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818061b30 con 0x7fc8180af9a0 2020-12-05T17:31:43.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81da31700 1 --2- 172.21.15.32:0/3931215468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8180af9a0 0x7fc818144610 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fc808002f30 tx=0x7fc808006e80).ready entity=mon.0 client_cookie=6329392fc22f8430 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:43.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81d230700 1 -- 172.21.15.32:0/3931215468 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8080097f0 con 0x7fc8180af9a0 2020-12-05T17:31:43.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81d230700 1 -- 172.21.15.32:0/3931215468 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc808009950 con 0x7fc8180af9a0 2020-12-05T17:31:43.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81d230700 1 -- 172.21.15.32:0/3931215468 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc808009cb0 con 0x7fc8180af9a0 2020-12-05T17:31:43.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81f234700 1 -- 172.21.15.32:0/3931215468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8180af9a0 msgr2=0x7fc818144610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81f234700 1 --2- 172.21.15.32:0/3931215468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8180af9a0 0x7fc818144610 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fc808002f30 tx=0x7fc808006e80).stop 2020-12-05T17:31:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81f234700 1 -- 172.21.15.32:0/3931215468 shutdown_connections 2020-12-05T17:31:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81f234700 1 --2- 172.21.15.32:0/3931215468 >> v2:172.21.15.61:3300/0 conn(0x7fc81813f5b0 0x7fc81813f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81f234700 1 --2- 172.21.15.32:0/3931215468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8180af9a0 0x7fc818144610 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81f234700 1 -- 172.21.15.32:0/3931215468 >> 172.21.15.32:0/3931215468 conn(0x7fc818061010 msgr2=0x7fc818061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81f234700 1 -- 172.21.15.32:0/3931215468 shutdown_connections 2020-12-05T17:31:43.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.979+0000 7fc81f234700 1 -- 172.21.15.32:0/3931215468 wait complete. 2020-12-05T17:31:43.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.983+0000 7fc81f234700 1 Processor -- start 2020-12-05T17:31:43.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.983+0000 7fc81f234700 1 -- start start 2020-12-05T17:31:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.983+0000 7fc81f234700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8180af5c0 0x7fc81810f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.983+0000 7fc81f234700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8180af9a0 0x7fc818141dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.983+0000 7fc81f234700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc81813f5b0 0x7fc818155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.983+0000 7fc81f234700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81810f660 con 0x7fc8180af9a0 2020-12-05T17:31:43.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.983+0000 7fc81f234700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc818110b30 con 0x7fc8180af5c0 2020-12-05T17:31:43.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.983+0000 7fc81f234700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180b6a20 con 0x7fc81813f5b0 2020-12-05T17:31:43.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.983+0000 7fc81e232700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8180af5c0 0x7fc81810f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.983+0000 7fc81e232700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8180af5c0 0x7fc81810f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:52402/0 (socket says 172.21.15.32:52402) 2020-12-05T17:31:43.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.983+0000 7fc81e232700 1 -- 172.21.15.32:0/2778450957 learned_addr learned my addr 172.21.15.32:0/2778450957 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:43.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.983+0000 7fc81ea33700 1 --2- 172.21.15.32:0/2778450957 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc81813f5b0 0x7fc818155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.983+0000 7fc81da31700 1 --2- 172.21.15.32:0/2778450957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8180af9a0 0x7fc818141dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.983+0000 7fc81e232700 1 -- 172.21.15.32:0/2778450957 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc81813f5b0 msgr2=0x7fc818155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.983+0000 7fc81e232700 1 --2- 172.21.15.32:0/2778450957 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc81813f5b0 0x7fc818155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.983+0000 7fc81e232700 1 -- 172.21.15.32:0/2778450957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8180af9a0 msgr2=0x7fc818141dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:43.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.983+0000 7fc81e232700 1 --2- 172.21.15.32:0/2778450957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8180af9a0 0x7fc818141dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:43.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.983+0000 7fc81e232700 1 -- 172.21.15.32:0/2778450957 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818155f10 con 0x7fc8180af5c0 2020-12-05T17:31:43.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.983+0000 7fc81ea33700 1 --2- 172.21.15.32:0/2778450957 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc81813f5b0 0x7fc818155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:31:43.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.983+0000 7fc81e232700 1 --2- 172.21.15.32:0/2778450957 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8180af5c0 0x7fc81810f030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fc81805c5c0 tx=0x7fc814007c10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:43.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.987+0000 7fc8077fe700 1 -- 172.21.15.32:0/2778450957 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc814001ea0 con 0x7fc8180af5c0 2020-12-05T17:31:43.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.987+0000 7fc8077fe700 1 -- 172.21.15.32:0/2778450957 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc814007020 con 0x7fc8180af5c0 2020-12-05T17:31:43.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.987+0000 7fc81f234700 1 -- 172.21.15.32:0/2778450957 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8180b51c0 con 0x7fc8180af5c0 2020-12-05T17:31:43.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.987+0000 7fc8077fe700 1 -- 172.21.15.32:0/2778450957 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc814007380 con 0x7fc8180af5c0 2020-12-05T17:31:43.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.987+0000 7fc81f234700 1 -- 172.21.15.32:0/2778450957 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc818111170 con 0x7fc8180af5c0 2020-12-05T17:31:43.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.987+0000 7fc81f234700 1 -- 172.21.15.32:0/2778450957 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc818110980 con 0x7fc8180af5c0 2020-12-05T17:31:43.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.987+0000 7fc8077fe700 1 -- 172.21.15.32:0/2778450957 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc814010030 con 0x7fc8180af5c0 2020-12-05T17:31:43.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.987+0000 7fc8077fe700 1 --2- 172.21.15.32:0/2778450957 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc7f80554e0 0x7fc7f80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:43.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.987+0000 7fc8077fe700 1 -- 172.21.15.32:0/2778450957 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc81406d7c0 con 0x7fc8180af5c0 2020-12-05T17:31:43.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.987+0000 7fc81da31700 1 --2- 172.21.15.32:0/2778450957 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc7f80554e0 0x7fc7f80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:43.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.991+0000 7fc81da31700 1 --2- 172.21.15.32:0/2778450957 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc7f80554e0 0x7fc7f80579a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fc80801c000 tx=0x7fc808015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:44.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:43.995+0000 7fc8077fe700 1 -- 172.21.15.32:0/2778450957 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc814043960 con 0x7fc8180af5c0 2020-12-05T17:31:44.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.139+0000 7fc81f234700 1 -- 172.21.15.32:0/2778450957 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc818057050 con 0x7fc8180af5c0 2020-12-05T17:31:44.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.139+0000 7fc8077fe700 1 -- 172.21.15.32:0/2778450957 <== mon.1 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc814018070 con 0x7fc8180af5c0 2020-12-05T17:31:44.145 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:31:44.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.143+0000 7fc81f234700 1 -- 172.21.15.32:0/2778450957 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc7f80554e0 msgr2=0x7fc7f80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.143+0000 7fc81f234700 1 --2- 172.21.15.32:0/2778450957 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc7f80554e0 0x7fc7f80579a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fc80801c000 tx=0x7fc808015000).stop 2020-12-05T17:31:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.143+0000 7fc81f234700 1 -- 172.21.15.32:0/2778450957 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8180af5c0 msgr2=0x7fc81810f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.143+0000 7fc81f234700 1 --2- 172.21.15.32:0/2778450957 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8180af5c0 0x7fc81810f030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fc81805c5c0 tx=0x7fc814007c10).stop 2020-12-05T17:31:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.143+0000 7fc81f234700 1 -- 172.21.15.32:0/2778450957 shutdown_connections 2020-12-05T17:31:44.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.143+0000 7fc81f234700 1 --2- 172.21.15.32:0/2778450957 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc7f80554e0 0x7fc7f80579a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.143+0000 7fc81f234700 1 --2- 172.21.15.32:0/2778450957 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc81813f5b0 0x7fc818155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.143+0000 7fc81f234700 1 --2- 172.21.15.32:0/2778450957 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8180af5c0 0x7fc81810f030 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.143+0000 7fc81f234700 1 --2- 172.21.15.32:0/2778450957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8180af9a0 0x7fc818141dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.143+0000 7fc81f234700 1 -- 172.21.15.32:0/2778450957 >> 172.21.15.32:0/2778450957 conn(0x7fc818061010 msgr2=0x7fc818143ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.143+0000 7fc81f234700 1 -- 172.21.15.32:0/2778450957 shutdown_connections 2020-12-05T17:31:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.143+0000 7fc81f234700 1 -- 172.21.15.32:0/2778450957 wait complete. 2020-12-05T17:31:44.159 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:31:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.271+0000 7f71cb59e700 1 Processor -- start 2020-12-05T17:31:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.271+0000 7f71cb59e700 1 -- start start 2020-12-05T17:31:44.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71cb59e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f71cc061920 0x7f71cc064a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:44.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71cb59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71cc0b16b0 0x7f71cc146860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71cb59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71cc146d50 con 0x7f71cc1428b0 2020-12-05T17:31:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71cb59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71cc145e80 con 0x7f71cc0b16b0 2020-12-05T17:31:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71cb59e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71cc146120 con 0x7f71cc061920 2020-12-05T17:31:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71c9d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71cc0b16b0 0x7f71cc146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71c9d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71cc0b16b0 0x7f71cc146860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60750/0 (socket says 172.21.15.32:60750) 2020-12-05T17:31:44.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71c9d9b700 1 -- 172.21.15.32:0/1170765331 learned_addr learned my addr 172.21.15.32:0/1170765331 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:44.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71ca59c700 1 --2- 172.21.15.32:0/1170765331 >> v2:172.21.15.61:3300/0 conn(0x7f71cc061920 0x7f71cc064a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:44.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71c9d9b700 1 -- 172.21.15.32:0/1170765331 >> v2:172.21.15.61:3300/0 conn(0x7f71cc061920 msgr2=0x7f71cc064a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:44.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71c9d9b700 1 --2- 172.21.15.32:0/1170765331 >> v2:172.21.15.61:3300/0 conn(0x7f71cc061920 0x7f71cc064a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71c9d9b700 1 -- 172.21.15.32:0/1170765331 >> v1:172.21.15.61:6789/0 conn(0x7f71cc1428b0 legacy=0x7f71cc142cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71c9d9b700 1 -- 172.21.15.32:0/1170765331 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc1463c0 con 0x7f71cc0b16b0 2020-12-05T17:31:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71c9d9b700 1 --2- 172.21.15.32:0/1170765331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71cc0b16b0 0x7f71cc146860 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f71cc05f230 tx=0x7f71c0006e80).ready entity=mon.0 client_cookie=2383a644a223c507 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71c959a700 1 -- 172.21.15.32:0/1170765331 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71c00097b0 con 0x7f71cc0b16b0 2020-12-05T17:31:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71c959a700 1 -- 172.21.15.32:0/1170765331 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c0009910 con 0x7f71cc0b16b0 2020-12-05T17:31:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71c959a700 1 -- 172.21.15.32:0/1170765331 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71c0009a80 con 0x7f71cc0b16b0 2020-12-05T17:31:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71cb59e700 1 -- 172.21.15.32:0/1170765331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71cc0b16b0 msgr2=0x7f71cc146860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71cb59e700 1 --2- 172.21.15.32:0/1170765331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71cc0b16b0 0x7f71cc146860 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f71cc05f230 tx=0x7f71c0006e80).stop 2020-12-05T17:31:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71cb59e700 1 -- 172.21.15.32:0/1170765331 shutdown_connections 2020-12-05T17:31:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71cb59e700 1 --2- 172.21.15.32:0/1170765331 >> v2:172.21.15.61:3300/0 conn(0x7f71cc061920 0x7f71cc064a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71cb59e700 1 --2- 172.21.15.32:0/1170765331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71cc0b16b0 0x7f71cc146860 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71cb59e700 1 -- 172.21.15.32:0/1170765331 >> 172.21.15.32:0/1170765331 conn(0x7f71cc0b6d10 msgr2=0x7f71cc0b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71cb59e700 1 -- 172.21.15.32:0/1170765331 shutdown_connections 2020-12-05T17:31:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.275+0000 7f71cb59e700 1 -- 172.21.15.32:0/1170765331 wait complete. 2020-12-05T17:31:44.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71cb59e700 1 Processor -- start 2020-12-05T17:31:44.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71cb59e700 1 -- start start 2020-12-05T17:31:44.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71cb59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71cc061920 0x7f71cc1497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:44.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71cb59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71cc0b16b0 0x7f71cc154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:44.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71cb59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71cc1428b0 0x7f71cc1553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:44.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71cb59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc13a240 con 0x7f71cc0b16b0 2020-12-05T17:31:44.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71cb59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc1489d0 con 0x7f71cc061920 2020-12-05T17:31:44.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71cb59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc0660c0 con 0x7f71cc1428b0 2020-12-05T17:31:44.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71c9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71cc0b16b0 0x7f71cc154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:44.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71c9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71cc0b16b0 0x7f71cc154f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60758/0 (socket says 172.21.15.32:60758) 2020-12-05T17:31:44.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71c9d9b700 1 -- 172.21.15.32:0/1312966764 learned_addr learned my addr 172.21.15.32:0/1312966764 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:44.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71ca59c700 1 --2- 172.21.15.32:0/1312966764 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71cc061920 0x7f71cc1497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:44.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71cad9d700 1 --2- 172.21.15.32:0/1312966764 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71cc1428b0 0x7f71cc1553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:44.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71c9d9b700 1 -- 172.21.15.32:0/1312966764 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71cc1428b0 msgr2=0x7f71cc1553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:44.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71c9d9b700 1 --2- 172.21.15.32:0/1312966764 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71cc1428b0 0x7f71cc1553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71c9d9b700 1 -- 172.21.15.32:0/1312966764 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71cc061920 msgr2=0x7f71cc1497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:44.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71c9d9b700 1 --2- 172.21.15.32:0/1312966764 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71cc061920 0x7f71cc1497c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71c9d9b700 1 -- 172.21.15.32:0/1312966764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc066de0 con 0x7f71cc0b16b0 2020-12-05T17:31:44.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71cad9d700 1 --2- 172.21.15.32:0/1312966764 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71cc1428b0 0x7f71cc1553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71c9d9b700 1 --2- 172.21.15.32:0/1312966764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71cc0b16b0 0x7f71cc154f00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f71cc149d30 tx=0x7f71c0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71bb7fe700 1 -- 172.21.15.32:0/1312966764 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71c0011de0 con 0x7f71cc0b16b0 2020-12-05T17:31:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71bb7fe700 1 -- 172.21.15.32:0/1312966764 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c0020400 con 0x7f71cc0b16b0 2020-12-05T17:31:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71cb59e700 1 -- 172.21.15.32:0/1312966764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71cc0662f0 con 0x7f71cc0b16b0 2020-12-05T17:31:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71bb7fe700 1 -- 172.21.15.32:0/1312966764 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71c0020760 con 0x7f71cc0b16b0 2020-12-05T17:31:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.279+0000 7f71cb59e700 1 -- 172.21.15.32:0/1312966764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71cc065e60 con 0x7f71cc0b16b0 2020-12-05T17:31:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.283+0000 7f71cb59e700 1 -- 172.21.15.32:0/1312966764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71cc057050 con 0x7f71cc0b16b0 2020-12-05T17:31:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.283+0000 7f71bb7fe700 1 -- 172.21.15.32:0/1312966764 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f71c0027030 con 0x7f71cc0b16b0 2020-12-05T17:31:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.283+0000 7f71bb7fe700 1 --2- 172.21.15.32:0/1312966764 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71a8055500 0x7f71a80579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.283+0000 7f71ca59c700 1 --2- 172.21.15.32:0/1312966764 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71a8055500 0x7f71a80579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.283+0000 7f71bb7fe700 1 -- 172.21.15.32:0/1312966764 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f71c0048b30 con 0x7f71cc0b16b0 2020-12-05T17:31:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.283+0000 7f71ca59c700 1 --2- 172.21.15.32:0/1312966764 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71a8055500 0x7f71a80579c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f71bc00c000 tx=0x7f71bc008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.291+0000 7f71bb7fe700 1 -- 172.21.15.32:0/1312966764 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f71c0044ed0 con 0x7f71cc0b16b0 2020-12-05T17:31:44.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.427+0000 7f71cb59e700 1 -- 172.21.15.32:0/1312966764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f71cc0b7a10 con 0x7f71cc0b16b0 2020-12-05T17:31:44.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.427+0000 7f71bb7fe700 1 -- 172.21.15.32:0/1312966764 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7f71c00250a0 con 0x7f71cc0b16b0 2020-12-05T17:31:44.432 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:31:44.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.431+0000 7f71cb59e700 1 -- 172.21.15.32:0/1312966764 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71a8055500 msgr2=0x7f71a80579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.431+0000 7f71cb59e700 1 --2- 172.21.15.32:0/1312966764 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71a8055500 0x7f71a80579c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f71bc00c000 tx=0x7f71bc008040).stop 2020-12-05T17:31:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.431+0000 7f71cb59e700 1 -- 172.21.15.32:0/1312966764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71cc0b16b0 msgr2=0x7f71cc154f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.431+0000 7f71cb59e700 1 --2- 172.21.15.32:0/1312966764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71cc0b16b0 0x7f71cc154f00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f71cc149d30 tx=0x7f71c0016000).stop 2020-12-05T17:31:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.431+0000 7f71cb59e700 1 -- 172.21.15.32:0/1312966764 shutdown_connections 2020-12-05T17:31:44.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.431+0000 7f71cb59e700 1 --2- 172.21.15.32:0/1312966764 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71a8055500 0x7f71a80579c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.431+0000 7f71cb59e700 1 --2- 172.21.15.32:0/1312966764 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71cc1428b0 0x7f71cc1553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.431+0000 7f71cb59e700 1 --2- 172.21.15.32:0/1312966764 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71cc061920 0x7f71cc1497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.431+0000 7f71cb59e700 1 --2- 172.21.15.32:0/1312966764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71cc0b16b0 0x7f71cc154f00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.431+0000 7f71cb59e700 1 -- 172.21.15.32:0/1312966764 >> 172.21.15.32:0/1312966764 conn(0x7f71cc0b6d10 msgr2=0x7f71cc13a620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:44.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.431+0000 7f71cb59e700 1 -- 172.21.15.32:0/1312966764 shutdown_connections 2020-12-05T17:31:44.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.431+0000 7f71cb59e700 1 -- 172.21.15.32:0/1312966764 wait complete. 2020-12-05T17:31:44.445 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:31:44.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.563+0000 7fea70951700 1 Processor -- start 2020-12-05T17:31:44.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.563+0000 7fea70951700 1 -- start start 2020-12-05T17:31:44.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea70951700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea6c0af9a0 0x7fea6c144610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:44.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea70951700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fea6c13f5b0 0x7fea6c13fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:44.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea70951700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea6c144b00 con 0x7fea6c0af5c0 2020-12-05T17:31:44.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea70951700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea6c1405f0 con 0x7fea6c13f5b0 2020-12-05T17:31:44.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea70951700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c0afe20 con 0x7fea6c0af9a0 2020-12-05T17:31:44.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea6b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fea6c0af5c0 0x7fea6c143f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35180/0 (socket says 172.21.15.32:35180) 2020-12-05T17:31:44.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea6b7fe700 1 -- 172.21.15.32:0/678125807 learned_addr learned my addr 172.21.15.32:0/678125807 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:44.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea6bfff700 1 --2- 172.21.15.32:0/678125807 >> v2:172.21.15.32:3300/0 conn(0x7fea6c13f5b0 0x7fea6c13fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:44.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea6affd700 1 --2- 172.21.15.32:0/678125807 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea6c0af9a0 0x7fea6c144610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:44.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea6bfff700 1 -- 172.21.15.32:0/678125807 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea6c0af9a0 msgr2=0x7fea6c144610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:44.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea6bfff700 1 --2- 172.21.15.32:0/678125807 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea6c0af9a0 0x7fea6c144610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea6bfff700 1 -- 172.21.15.32:0/678125807 >> v1:172.21.15.32:6789/0 conn(0x7fea6c0af5c0 legacy=0x7fea6c143f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:44.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea6bfff700 1 -- 172.21.15.32:0/678125807 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6c1407e0 con 0x7fea6c13f5b0 2020-12-05T17:31:44.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea6bfff700 1 --2- 172.21.15.32:0/678125807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea6c13f5b0 0x7fea6c13fa40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fea640032b0 tx=0x7fea640091a0).ready entity=mon.0 client_cookie=5a7f2ad3bbae40b5 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:44.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea6a7fc700 1 -- 172.21.15.32:0/678125807 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea6400b9a0 con 0x7fea6c13f5b0 2020-12-05T17:31:44.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea6a7fc700 1 -- 172.21.15.32:0/678125807 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6400bb00 con 0x7fea6c13f5b0 2020-12-05T17:31:44.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea70951700 1 -- 172.21.15.32:0/678125807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea6c13f5b0 msgr2=0x7fea6c13fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:44.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea6a7fc700 1 -- 172.21.15.32:0/678125807 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea6400bc70 con 0x7fea6c13f5b0 2020-12-05T17:31:44.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea70951700 1 --2- 172.21.15.32:0/678125807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea6c13f5b0 0x7fea6c13fa40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fea640032b0 tx=0x7fea640091a0).stop 2020-12-05T17:31:44.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea70951700 1 -- 172.21.15.32:0/678125807 shutdown_connections 2020-12-05T17:31:44.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea70951700 1 --2- 172.21.15.32:0/678125807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea6c13f5b0 0x7fea6c13fa40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea70951700 1 --2- 172.21.15.32:0/678125807 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea6c0af9a0 0x7fea6c144610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea70951700 1 -- 172.21.15.32:0/678125807 >> 172.21.15.32:0/678125807 conn(0x7fea6c061010 msgr2=0x7fea6c061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:44.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea70951700 1 -- 172.21.15.32:0/678125807 shutdown_connections 2020-12-05T17:31:44.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.567+0000 7fea70951700 1 -- 172.21.15.32:0/678125807 wait complete. 2020-12-05T17:31:44.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea70951700 1 Processor -- start 2020-12-05T17:31:44.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea70951700 1 -- start start 2020-12-05T17:31:44.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea70951700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea6c0af5c0 0x7fea6c111740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:44.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea70951700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea6c0af9a0 0x7fea6c1475f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:44.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea70951700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea6c13f5b0 0x7fea6c1528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:44.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea70951700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c065b30 con 0x7fea6c0af5c0 2020-12-05T17:31:44.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea70951700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c10ee20 con 0x7fea6c13f5b0 2020-12-05T17:31:44.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea70951700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c10f0c0 con 0x7fea6c0af9a0 2020-12-05T17:31:44.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea6b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea6c0af5c0 0x7fea6c111740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:44.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea6b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea6c0af5c0 0x7fea6c111740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60770/0 (socket says 172.21.15.32:60770) 2020-12-05T17:31:44.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea6b7fe700 1 -- 172.21.15.32:0/710706916 learned_addr learned my addr 172.21.15.32:0/710706916 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:44.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea6affd700 1 --2- 172.21.15.32:0/710706916 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea6c0af9a0 0x7fea6c1475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:44.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea6bfff700 1 --2- 172.21.15.32:0/710706916 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea6c13f5b0 0x7fea6c1528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:44.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea6b7fe700 1 -- 172.21.15.32:0/710706916 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea6c0af9a0 msgr2=0x7fea6c1475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:44.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea6b7fe700 1 --2- 172.21.15.32:0/710706916 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea6c0af9a0 0x7fea6c1475f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea6b7fe700 1 -- 172.21.15.32:0/710706916 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea6c13f5b0 msgr2=0x7fea6c1528d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:44.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea6b7fe700 1 --2- 172.21.15.32:0/710706916 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea6c13f5b0 0x7fea6c1528d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea6b7fe700 1 -- 172.21.15.32:0/710706916 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6c10f820 con 0x7fea6c0af5c0 2020-12-05T17:31:44.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea6b7fe700 1 --2- 172.21.15.32:0/710706916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea6c0af5c0 0x7fea6c111740 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fea6c062450 tx=0x7fea5c007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:44.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea68ff9700 1 -- 172.21.15.32:0/710706916 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea5c0072e0 con 0x7fea6c0af5c0 2020-12-05T17:31:44.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea68ff9700 1 -- 172.21.15.32:0/710706916 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5c007440 con 0x7fea6c0af5c0 2020-12-05T17:31:44.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea70951700 1 -- 172.21.15.32:0/710706916 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea6c066450 con 0x7fea6c0af5c0 2020-12-05T17:31:44.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea68ff9700 1 -- 172.21.15.32:0/710706916 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea5c0057d0 con 0x7fea6c0af5c0 2020-12-05T17:31:44.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.571+0000 7fea70951700 1 -- 172.21.15.32:0/710706916 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea6c064f70 con 0x7fea6c0af5c0 2020-12-05T17:31:44.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.575+0000 7fea68ff9700 1 -- 172.21.15.32:0/710706916 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fea5c005930 con 0x7fea6c0af5c0 2020-12-05T17:31:44.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.575+0000 7fea70951700 1 -- 172.21.15.32:0/710706916 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea6c057050 con 0x7fea6c0af5c0 2020-12-05T17:31:44.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.575+0000 7fea68ff9700 1 --2- 172.21.15.32:0/710706916 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fea4c055440 0x7fea4c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:44.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.575+0000 7fea6affd700 1 --2- 172.21.15.32:0/710706916 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fea4c055440 0x7fea4c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:44.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.575+0000 7fea68ff9700 1 -- 172.21.15.32:0/710706916 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fea5c06db60 con 0x7fea6c0af5c0 2020-12-05T17:31:44.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.575+0000 7fea6affd700 1 --2- 172.21.15.32:0/710706916 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fea4c055440 0x7fea4c057900 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fea60008000 tx=0x7fea60008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:44.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.583+0000 7fea68ff9700 1 -- 172.21.15.32:0/710706916 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fea5c043bf0 con 0x7fea6c0af5c0 2020-12-05T17:31:44.712 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:31:44.714 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:31:44.717 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:31:44.719 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:31:44.721 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:31:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.727+0000 7fea70951700 1 -- 172.21.15.32:0/710706916 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fea6c069690 con 0x7fea6c0af5c0 2020-12-05T17:31:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.727+0000 7fea68ff9700 1 -- 172.21.15.32:0/710706916 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7fea5c047210 con 0x7fea6c0af5c0 2020-12-05T17:31:44.730 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:31:44.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.727+0000 7fea70951700 1 -- 172.21.15.32:0/710706916 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fea4c055440 msgr2=0x7fea4c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:44.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.727+0000 7fea70951700 1 --2- 172.21.15.32:0/710706916 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fea4c055440 0x7fea4c057900 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fea60008000 tx=0x7fea60008040).stop 2020-12-05T17:31:44.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.727+0000 7fea70951700 1 -- 172.21.15.32:0/710706916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea6c0af5c0 msgr2=0x7fea6c111740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:44.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.727+0000 7fea70951700 1 --2- 172.21.15.32:0/710706916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea6c0af5c0 0x7fea6c111740 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fea6c062450 tx=0x7fea5c007bb0).stop 2020-12-05T17:31:44.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.727+0000 7fea70951700 1 -- 172.21.15.32:0/710706916 shutdown_connections 2020-12-05T17:31:44.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.727+0000 7fea70951700 1 --2- 172.21.15.32:0/710706916 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea6c13f5b0 0x7fea6c1528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.727+0000 7fea70951700 1 --2- 172.21.15.32:0/710706916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea6c0af5c0 0x7fea6c111740 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.727+0000 7fea70951700 1 --2- 172.21.15.32:0/710706916 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fea4c055440 0x7fea4c057900 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.727+0000 7fea70951700 1 --2- 172.21.15.32:0/710706916 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea6c0af9a0 0x7fea6c1475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.727+0000 7fea70951700 1 -- 172.21.15.32:0/710706916 >> 172.21.15.32:0/710706916 conn(0x7fea6c061010 msgr2=0x7fea6c140b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.727+0000 7fea70951700 1 -- 172.21.15.32:0/710706916 shutdown_connections 2020-12-05T17:31:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.727+0000 7fea70951700 1 -- 172.21.15.32:0/710706916 wait complete. 2020-12-05T17:31:44.748 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2020-12-05T17:31:44.748 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2020-12-05T17:31:44.748 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early mgr dump --format=json 2020-12-05T17:31:44.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb09f0f7700 1 Processor -- start 2020-12-05T17:31:44.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb09f0f7700 1 -- start start 2020-12-05T17:31:44.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb09f0f7700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb0980afe50 0x7fb098146a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:44.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb09f0f7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0980b3aa0 0x7fb0981429e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:44.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb09f0f7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb0980b1880 0x7fb098142ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:44.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb09f0f7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0980b0230 con 0x7fb0980b3aa0 2020-12-05T17:31:44.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb09f0f7700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0980b6a50 con 0x7fb0980afe50 2020-12-05T17:31:44.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb09f0f7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb098146080 con 0x7fb0980b1880 2020-12-05T17:31:44.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb097fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0980b3aa0 0x7fb0981429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:44.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb097fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0980b3aa0 0x7fb0981429e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60778/0 (socket says 172.21.15.32:60778) 2020-12-05T17:31:44.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb097fff700 1 -- 172.21.15.32:0/3751028486 learned_addr learned my addr 172.21.15.32:0/3751028486 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:44.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb09ce93700 1 --2- 172.21.15.32:0/3751028486 >> v2:172.21.15.61:3300/0 conn(0x7fb0980afe50 0x7fb098146a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:44.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb09d694700 1 --2- 172.21.15.32:0/3751028486 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb0980b1880 0x7fb098142ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:44.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb097fff700 1 -- 172.21.15.32:0/3751028486 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb0980b1880 msgr2=0x7fb098142ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:44.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb097fff700 1 --2- 172.21.15.32:0/3751028486 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb0980b1880 0x7fb098142ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb097fff700 1 -- 172.21.15.32:0/3751028486 >> v2:172.21.15.61:3300/0 conn(0x7fb0980afe50 msgr2=0x7fb098146a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:44.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb097fff700 1 --2- 172.21.15.32:0/3751028486 >> v2:172.21.15.61:3300/0 conn(0x7fb0980afe50 0x7fb098146a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb097fff700 1 -- 172.21.15.32:0/3751028486 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb098146320 con 0x7fb0980b3aa0 2020-12-05T17:31:44.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb097fff700 1 --2- 172.21.15.32:0/3751028486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0980b3aa0 0x7fb0981429e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fb088002f30 tx=0x7fb088006e80).ready entity=mon.0 client_cookie=45b0179a40fbee98 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:44.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb0977fe700 1 -- 172.21.15.32:0/3751028486 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0880097f0 con 0x7fb0980b3aa0 2020-12-05T17:31:44.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb0977fe700 1 -- 172.21.15.32:0/3751028486 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb088009950 con 0x7fb0980b3aa0 2020-12-05T17:31:44.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb0977fe700 1 -- 172.21.15.32:0/3751028486 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb088009cb0 con 0x7fb0980b3aa0 2020-12-05T17:31:44.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb09f0f7700 1 -- 172.21.15.32:0/3751028486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0980b3aa0 msgr2=0x7fb0981429e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:44.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb09f0f7700 1 --2- 172.21.15.32:0/3751028486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0980b3aa0 0x7fb0981429e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fb088002f30 tx=0x7fb088006e80).stop 2020-12-05T17:31:44.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb09f0f7700 1 -- 172.21.15.32:0/3751028486 shutdown_connections 2020-12-05T17:31:44.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb09f0f7700 1 --2- 172.21.15.32:0/3751028486 >> v2:172.21.15.61:3300/0 conn(0x7fb0980afe50 0x7fb098146a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb09f0f7700 1 --2- 172.21.15.32:0/3751028486 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb0980b1880 0x7fb098142ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb09f0f7700 1 --2- 172.21.15.32:0/3751028486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0980b3aa0 0x7fb0981429e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb09f0f7700 1 -- 172.21.15.32:0/3751028486 >> 172.21.15.32:0/3751028486 conn(0x7fb0980b9100 msgr2=0x7fb0980b9510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:44.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb09f0f7700 1 -- 172.21.15.32:0/3751028486 shutdown_connections 2020-12-05T17:31:44.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.907+0000 7fb09f0f7700 1 -- 172.21.15.32:0/3751028486 wait complete. 2020-12-05T17:31:44.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb09f0f7700 1 Processor -- start 2020-12-05T17:31:44.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb09f0f7700 1 -- start start 2020-12-05T17:31:44.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb09f0f7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb0980afe50 0x7fb098142890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:44.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb09f0f7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0980b1880 0x7fb098062610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:44.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb09f0f7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb0980b3aa0 0x7fb098062b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:44.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb09f0f7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb098061ef0 con 0x7fb0980b1880 2020-12-05T17:31:44.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb09f0f7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0980619b0 con 0x7fb0980b3aa0 2020-12-05T17:31:44.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb09f0f7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb098061c50 con 0x7fb0980afe50 2020-12-05T17:31:44.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb097fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0980b1880 0x7fb098062610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:44.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb097fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0980b1880 0x7fb098062610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60786/0 (socket says 172.21.15.32:60786) 2020-12-05T17:31:44.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb097fff700 1 -- 172.21.15.32:0/939610918 learned_addr learned my addr 172.21.15.32:0/939610918 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:44.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb09ce93700 1 --2- 172.21.15.32:0/939610918 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb0980afe50 0x7fb098142890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:44.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb09d694700 1 --2- 172.21.15.32:0/939610918 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb0980b3aa0 0x7fb098062b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:44.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb097fff700 1 -- 172.21.15.32:0/939610918 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb0980afe50 msgr2=0x7fb098142890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:44.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb097fff700 1 --2- 172.21.15.32:0/939610918 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb0980afe50 0x7fb098142890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb097fff700 1 -- 172.21.15.32:0/939610918 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb0980b3aa0 msgr2=0x7fb098062b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:44.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb097fff700 1 --2- 172.21.15.32:0/939610918 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb0980b3aa0 0x7fb098062b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:44.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb097fff700 1 -- 172.21.15.32:0/939610918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb098065420 con 0x7fb0980b1880 2020-12-05T17:31:44.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb097fff700 1 --2- 172.21.15.32:0/939610918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0980b1880 0x7fb098062610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fb088002f30 tx=0x7fb088015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:44.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb095ffb700 1 -- 172.21.15.32:0/939610918 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb088011c10 con 0x7fb0980b1880 2020-12-05T17:31:44.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb095ffb700 1 -- 172.21.15.32:0/939610918 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb088011d70 con 0x7fb0980b1880 2020-12-05T17:31:44.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb09f0f7700 1 -- 172.21.15.32:0/939610918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb098065610 con 0x7fb0980b1880 2020-12-05T17:31:44.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb095ffb700 1 -- 172.21.15.32:0/939610918 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb08801e650 con 0x7fb0980b1880 2020-12-05T17:31:44.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.911+0000 7fb09f0f7700 1 -- 172.21.15.32:0/939610918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb098065ae0 con 0x7fb0980b1880 2020-12-05T17:31:44.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.915+0000 7fb09f0f7700 1 -- 172.21.15.32:0/939610918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0981425d0 con 0x7fb0980b1880 2020-12-05T17:31:44.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.915+0000 7fb095ffb700 1 -- 172.21.15.32:0/939610918 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb088025030 con 0x7fb0980b1880 2020-12-05T17:31:44.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.915+0000 7fb095ffb700 1 --2- 172.21.15.32:0/939610918 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb070055330 0x7fb0700577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:44.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.915+0000 7fb095ffb700 1 -- 172.21.15.32:0/939610918 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb08806f370 con 0x7fb0980b1880 2020-12-05T17:31:44.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.915+0000 7fb09ce93700 1 --2- 172.21.15.32:0/939610918 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb070055330 0x7fb0700577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:44.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.915+0000 7fb09ce93700 1 --2- 172.21.15.32:0/939610918 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb070055330 0x7fb0700577f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fb08000c000 tx=0x7fb080008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:44.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:44.923+0000 7fb095ffb700 1 -- 172.21.15.32:0/939610918 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb0981425d0 con 0x7fb0980b1880 2020-12-05T17:31:45.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.099+0000 7fb09f0f7700 1 -- 172.21.15.32:0/939610918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fb0981425d0 con 0x7fb0980b1880 2020-12-05T17:31:45.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.103+0000 7fb095ffb700 1 -- 172.21.15.32:0/939610918 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+133618 (secure 0 0 0) 0x7fb0981425d0 con 0x7fb0980b1880 2020-12-05T17:31:45.109 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:31:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.107+0000 7fb09f0f7700 1 -- 172.21.15.32:0/939610918 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb070055330 msgr2=0x7fb0700577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.107+0000 7fb09f0f7700 1 --2- 172.21.15.32:0/939610918 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb070055330 0x7fb0700577f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fb08000c000 tx=0x7fb080008040).stop 2020-12-05T17:31:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.107+0000 7fb09f0f7700 1 -- 172.21.15.32:0/939610918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0980b1880 msgr2=0x7fb098062610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.107+0000 7fb09f0f7700 1 --2- 172.21.15.32:0/939610918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0980b1880 0x7fb098062610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fb088002f30 tx=0x7fb088015000).stop 2020-12-05T17:31:45.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.107+0000 7fb09f0f7700 1 -- 172.21.15.32:0/939610918 shutdown_connections 2020-12-05T17:31:45.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.107+0000 7fb09f0f7700 1 --2- 172.21.15.32:0/939610918 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb0980b3aa0 0x7fb098062b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.107+0000 7fb09f0f7700 1 --2- 172.21.15.32:0/939610918 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb070055330 0x7fb0700577f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.107+0000 7fb09f0f7700 1 --2- 172.21.15.32:0/939610918 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb0980afe50 0x7fb098142890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.107+0000 7fb09f0f7700 1 --2- 172.21.15.32:0/939610918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0980b1880 0x7fb098062610 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.107+0000 7fb09f0f7700 1 -- 172.21.15.32:0/939610918 >> 172.21.15.32:0/939610918 conn(0x7fb0980b9100 msgr2=0x7fb0981490b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:45.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.107+0000 7fb09f0f7700 1 -- 172.21.15.32:0/939610918 shutdown_connections 2020-12-05T17:31:45.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.107+0000 7fb09f0f7700 1 -- 172.21.15.32:0/939610918 wait complete. 2020-12-05T17:31:45.123 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":11355},{"type":"v1","addr":"172.21.15.32:6801","nonce":11355}]},"active_addr":"172.21.15.32:6801/11355","active_change":"2020-12-05T17:31:24.780000+0000","active_mgr_features":4540138292840890367,"available":true,"standbys":[{"gid":4114,"name":"x","mgr_features":4540138292840890367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":3616324273}]},{"addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2990619998}]},{"addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":3697360454}]}]}} 2020-12-05T17:31:45.128 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2020-12-05T17:31:45.128 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2020-12-05T17:31:45.129 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:31:45.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02f5133700 1 Processor -- start 2020-12-05T17:31:45.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02f5133700 1 -- start start 2020-12-05T17:31:45.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02f5133700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f02f0144540 0x7f02f0061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02f5133700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02f0143a00 con 0x7f02f01403f0 2020-12-05T17:31:45.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02f5133700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02f0140830 con 0x7f02f00b7720 2020-12-05T17:31:45.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02f5133700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02f00b7b00 con 0x7f02f0144540 2020-12-05T17:31:45.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02f4932700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f02f01403f0 0x7f02f013d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35212/0 (socket says 172.21.15.32:35212) 2020-12-05T17:31:45.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02f4932700 1 -- 172.21.15.32:0/3462826948 learned_addr learned my addr 172.21.15.32:0/3462826948 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:45.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02ef7fe700 1 --2- 172.21.15.32:0/3462826948 >> v2:172.21.15.61:3300/0 conn(0x7f02f0144540 0x7f02f0061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02eeffd700 1 -- 172.21.15.32:0/3462826948 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1561917473 0 0) 0x7f02f0143a00 con 0x7f02f01403f0 2020-12-05T17:31:45.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02eeffd700 1 -- 172.21.15.32:0/3462826948 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02d4003230 con 0x7f02f01403f0 2020-12-05T17:31:45.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02eeffd700 1 -- 172.21.15.32:0/3462826948 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1028802354 0 0) 0x7f02d4003230 con 0x7f02f01403f0 2020-12-05T17:31:45.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02eeffd700 1 -- 172.21.15.32:0/3462826948 >> v2:172.21.15.61:3300/0 conn(0x7f02f0144540 msgr2=0x7f02f0061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:45.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02eeffd700 1 --2- 172.21.15.32:0/3462826948 >> v2:172.21.15.61:3300/0 conn(0x7f02f0144540 0x7f02f0061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02eeffd700 1 -- 172.21.15.32:0/3462826948 >> v1:172.21.15.61:6789/0 conn(0x7f02f00b7720 legacy=0x7f02f0143e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02eeffd700 1 -- 172.21.15.32:0/3462826948 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f005fe60 con 0x7f02f01403f0 2020-12-05T17:31:45.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02eeffd700 1 -- 172.21.15.32:0/3462826948 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f02e80016d0 con 0x7f02f01403f0 2020-12-05T17:31:45.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02ef7fe700 1 --2- 172.21.15.32:0/3462826948 >> v2:172.21.15.61:3300/0 conn(0x7f02f0144540 0x7f02f0061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:31:45.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02eeffd700 1 -- 172.21.15.32:0/3462826948 >> v1:172.21.15.32:6789/0 conn(0x7f02f01403f0 legacy=0x7f02f013d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02eeffd700 1 --2- 172.21.15.32:0/3462826948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d4003770 0x7f02d4003b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02eeffd700 1 --2- 172.21.15.32:0/3462826948 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f02d40044d0 0x7f02d4004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02eeffd700 1 --2- 172.21.15.32:0/3462826948 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02d4004e40 0x7f02d400b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02eeffd700 1 -- 172.21.15.32:0/3462826948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f00b7b00 con 0x7f02d4003770 2020-12-05T17:31:45.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02eeffd700 1 -- 172.21.15.32:0/3462826948 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d40052f0 con 0x7f02d4004e40 2020-12-05T17:31:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02eeffd700 1 -- 172.21.15.32:0/3462826948 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d400bc10 con 0x7f02d40044d0 2020-12-05T17:31:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02effff700 1 --2- 172.21.15.32:0/3462826948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d4003770 0x7f02d4003b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02f4932700 1 --2- 172.21.15.32:0/3462826948 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02d4004e40 0x7f02d400b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02ef7fe700 1 --2- 172.21.15.32:0/3462826948 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f02d40044d0 0x7f02d4004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02effff700 1 -- 172.21.15.32:0/3462826948 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f02d40044d0 msgr2=0x7f02d4004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:45.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02effff700 1 --2- 172.21.15.32:0/3462826948 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f02d40044d0 0x7f02d4004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02effff700 1 -- 172.21.15.32:0/3462826948 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02d4004e40 msgr2=0x7f02d400b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:45.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02effff700 1 --2- 172.21.15.32:0/3462826948 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02d4004e40 0x7f02d400b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02effff700 1 -- 172.21.15.32:0/3462826948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d4003230 con 0x7f02d4003770 2020-12-05T17:31:45.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.239+0000 7f02f4932700 1 -- 172.21.15.32:0/3462826948 reap_dead start 2020-12-05T17:31:45.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02effff700 1 --2- 172.21.15.32:0/3462826948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d4003770 0x7f02d4003b90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f02e0001000 tx=0x7f02e0007b60).ready entity=mon.0 client_cookie=66c48fe9ca77c7a7 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:45.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02eeffd700 1 -- 172.21.15.32:0/3462826948 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02e000c660 con 0x7f02d4003770 2020-12-05T17:31:45.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02eeffd700 1 -- 172.21.15.32:0/3462826948 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e000c7c0 con 0x7f02d4003770 2020-12-05T17:31:45.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02eeffd700 1 -- 172.21.15.32:0/3462826948 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02e000cb20 con 0x7f02d4003770 2020-12-05T17:31:45.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02f5133700 1 -- 172.21.15.32:0/3462826948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d4003770 msgr2=0x7f02d4003b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02f5133700 1 --2- 172.21.15.32:0/3462826948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d4003770 0x7f02d4003b90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f02e0001000 tx=0x7f02e0007b60).stop 2020-12-05T17:31:45.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02f5133700 1 -- 172.21.15.32:0/3462826948 shutdown_connections 2020-12-05T17:31:45.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02f5133700 1 --2- 172.21.15.32:0/3462826948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d4003770 0x7f02d4003b90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02f5133700 1 -- 172.21.15.32:0/3462826948 >> 172.21.15.32:0/3462826948 conn(0x7f02f005bc20 msgr2=0x7f02f00b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:45.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02f5133700 1 -- 172.21.15.32:0/3462826948 shutdown_connections 2020-12-05T17:31:45.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02f5133700 1 -- 172.21.15.32:0/3462826948 wait complete. 2020-12-05T17:31:45.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02f5133700 1 Processor -- start 2020-12-05T17:31:45.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02f5133700 1 -- start start 2020-12-05T17:31:45.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02f5133700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f02d4003770 0x7f02f00b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02f5133700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f00b0580 0x7f02f013c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02f5133700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02f00af500 0x7f02f00b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02f5133700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f00b17b0 con 0x7f02f00b0580 2020-12-05T17:31:45.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02f5133700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f00b1510 con 0x7f02f00af500 2020-12-05T17:31:45.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02f5133700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f00b3ce0 con 0x7f02d4003770 2020-12-05T17:31:45.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02ef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f00b0580 0x7f02f013c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02effff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f02d4003770 0x7f02f00b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02ef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f00b0580 0x7f02f013c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60808/0 (socket says 172.21.15.32:60808) 2020-12-05T17:31:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02ef7fe700 1 -- 172.21.15.32:0/891428504 learned_addr learned my addr 172.21.15.32:0/891428504 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.243+0000 7f02f4932700 1 --2- 172.21.15.32:0/891428504 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02f00af500 0x7f02f00b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.247+0000 7f02ef7fe700 1 -- 172.21.15.32:0/891428504 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f02d4003770 msgr2=0x7f02f00b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.247+0000 7f02ef7fe700 1 --2- 172.21.15.32:0/891428504 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f02d4003770 0x7f02f00b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.247+0000 7f02ef7fe700 1 -- 172.21.15.32:0/891428504 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02f00af500 msgr2=0x7f02f00b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.247+0000 7f02ef7fe700 1 --2- 172.21.15.32:0/891428504 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02f00af500 0x7f02f00b2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.247+0000 7f02ef7fe700 1 -- 172.21.15.32:0/891428504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f00af2b0 con 0x7f02f00b0580 2020-12-05T17:31:45.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.247+0000 7f02f4932700 1 --2- 172.21.15.32:0/891428504 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02f00af500 0x7f02f00b2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:45.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.247+0000 7f02ef7fe700 1 --2- 172.21.15.32:0/891428504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f00b0580 0x7f02f013c670 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f02d800b000 tx=0x7f02d8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:45.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.247+0000 7f02ed7fa700 1 -- 172.21.15.32:0/891428504 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02d800ee20 con 0x7f02f00b0580 2020-12-05T17:31:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.247+0000 7f02ed7fa700 1 -- 172.21.15.32:0/891428504 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d8014430 con 0x7f02f00b0580 2020-12-05T17:31:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.247+0000 7f02f5133700 1 -- 172.21.15.32:0/891428504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f00b30b0 con 0x7f02f00b0580 2020-12-05T17:31:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.247+0000 7f02ed7fa700 1 -- 172.21.15.32:0/891428504 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02d8014790 con 0x7f02f00b0580 2020-12-05T17:31:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.247+0000 7f02f5133700 1 -- 172.21.15.32:0/891428504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f013b5b0 con 0x7f02f00b0580 2020-12-05T17:31:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.247+0000 7f02ed7fa700 1 -- 172.21.15.32:0/891428504 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f02d8024070 con 0x7f02f00b0580 2020-12-05T17:31:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.247+0000 7f02f5133700 1 -- 172.21.15.32:0/891428504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f00b3ea0 con 0x7f02f00b0580 2020-12-05T17:31:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.251+0000 7f02ed7fa700 1 --2- 172.21.15.32:0/891428504 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f02c8055330 0x7f02c80577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.251+0000 7f02ed7fa700 1 -- 172.21.15.32:0/891428504 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f02d806edc0 con 0x7f02f00b0580 2020-12-05T17:31:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.251+0000 7f02effff700 1 --2- 172.21.15.32:0/891428504 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f02c8055330 0x7f02c80577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.251+0000 7f02effff700 1 --2- 172.21.15.32:0/891428504 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f02c8055330 0x7f02c80577f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f02e001a000 tx=0x7f02e0016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:45.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.255+0000 7f02ed7fa700 1 -- 172.21.15.32:0/891428504 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f02d8044e50 con 0x7f02f00b0580 2020-12-05T17:31:45.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.395+0000 7f02f5133700 1 -- 172.21.15.32:0/891428504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02f00b3240 con 0x7f02f00b0580 2020-12-05T17:31:45.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.399+0000 7f02ed7fa700 1 -- 172.21.15.32:0/891428504 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f02f00b3240 con 0x7f02f00b0580 2020-12-05T17:31:45.402 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:31:45.402 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:31:41.610914+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:31:36.706361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-12-05T17:31:39.298269+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:31:45.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.399+0000 7f02f5133700 1 -- 172.21.15.32:0/891428504 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f02c8055330 msgr2=0x7f02c80577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.399+0000 7f02f5133700 1 --2- 172.21.15.32:0/891428504 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f02c8055330 0x7f02c80577f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f02e001a000 tx=0x7f02e0016000).stop 2020-12-05T17:31:45.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.399+0000 7f02f5133700 1 -- 172.21.15.32:0/891428504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f00b0580 msgr2=0x7f02f013c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.399+0000 7f02f5133700 1 --2- 172.21.15.32:0/891428504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f00b0580 0x7f02f013c670 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f02d800b000 tx=0x7f02d8008040).stop 2020-12-05T17:31:45.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.399+0000 7f02f5133700 1 -- 172.21.15.32:0/891428504 shutdown_connections 2020-12-05T17:31:45.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.399+0000 7f02f5133700 1 --2- 172.21.15.32:0/891428504 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02f00af500 0x7f02f00b2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.399+0000 7f02f5133700 1 --2- 172.21.15.32:0/891428504 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f02c8055330 0x7f02c80577f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.399+0000 7f02f5133700 1 --2- 172.21.15.32:0/891428504 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f02d4003770 0x7f02f00b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.399+0000 7f02f5133700 1 --2- 172.21.15.32:0/891428504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f00b0580 0x7f02f013c670 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.399+0000 7f02f5133700 1 -- 172.21.15.32:0/891428504 >> 172.21.15.32:0/891428504 conn(0x7f02f005bc20 msgr2=0x7f02f00b5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:45.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.399+0000 7f02f5133700 1 -- 172.21.15.32:0/891428504 shutdown_connections 2020-12-05T17:31:45.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.399+0000 7f02f5133700 1 -- 172.21.15.32:0/891428504 wait complete. 2020-12-05T17:31:45.415 INFO:tasks.ceph.ceph_manager.ceph:all up! 2020-12-05T17:31:45.416 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:31:45.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6dd736700 1 Processor -- start 2020-12-05T17:31:45.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6dd736700 1 -- start start 2020-12-05T17:31:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6dd736700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6d8062400 0x7fa6d805f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6dd736700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6d8142450 con 0x7fa6d805f3b0 2020-12-05T17:31:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6dd736700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6d8061780 con 0x7fa6d805feb0 2020-12-05T17:31:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6dd736700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d8061a20 con 0x7fa6d8062400 2020-12-05T17:31:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa6d805f3b0 0x7fa6d80b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35228/0 (socket says 172.21.15.32:35228) 2020-12-05T17:31:45.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d7fff700 1 -- 172.21.15.32:0/1489446487 learned_addr learned my addr 172.21.15.32:0/1489446487 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:45.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d77fe700 1 --2- 172.21.15.32:0/1489446487 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6d8062400 0x7fa6d805f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d6ffd700 1 -- 172.21.15.32:0/1489446487 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2278080164 0 0) 0x7fa6d8142450 con 0x7fa6d805f3b0 2020-12-05T17:31:45.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d6ffd700 1 -- 172.21.15.32:0/1489446487 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6b8003230 con 0x7fa6d805f3b0 2020-12-05T17:31:45.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d6ffd700 1 -- 172.21.15.32:0/1489446487 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1284274622 0 0) 0x7fa6d8061780 con 0x7fa6d805feb0 2020-12-05T17:31:45.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d6ffd700 1 -- 172.21.15.32:0/1489446487 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6d8142450 con 0x7fa6d805feb0 2020-12-05T17:31:45.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d6ffd700 1 -- 172.21.15.32:0/1489446487 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2846920512 0 0) 0x7fa6b8003230 con 0x7fa6d805f3b0 2020-12-05T17:31:45.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d6ffd700 1 -- 172.21.15.32:0/1489446487 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6d8062400 msgr2=0x7fa6d805f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:45.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d6ffd700 1 --2- 172.21.15.32:0/1489446487 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6d8062400 0x7fa6d805f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d6ffd700 1 -- 172.21.15.32:0/1489446487 >> v1:172.21.15.61:6789/0 conn(0x7fa6d805feb0 legacy=0x7fa6d80b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d6ffd700 1 -- 172.21.15.32:0/1489446487 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d8061cc0 con 0x7fa6d805f3b0 2020-12-05T17:31:45.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d6ffd700 1 -- 172.21.15.32:0/1489446487 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fa6c8001a50 con 0x7fa6d805f3b0 2020-12-05T17:31:45.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d77fe700 1 --2- 172.21.15.32:0/1489446487 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6d8062400 0x7fa6d805f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:31:45.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d6ffd700 1 -- 172.21.15.32:0/1489446487 >> v1:172.21.15.32:6789/0 conn(0x7fa6d805f3b0 legacy=0x7fa6d80b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d6ffd700 1 --2- 172.21.15.32:0/1489446487 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa6b80038e0 0x7fa6b8003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d6ffd700 1 --2- 172.21.15.32:0/1489446487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6b80044d0 0x7fa6b8004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d6ffd700 1 --2- 172.21.15.32:0/1489446487 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6d8062400 0x7fa6b8004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d6ffd700 1 -- 172.21.15.32:0/1489446487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d8061a20 con 0x7fa6b80044d0 2020-12-05T17:31:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d6ffd700 1 -- 172.21.15.32:0/1489446487 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b8005330 con 0x7fa6b80038e0 2020-12-05T17:31:45.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d6ffd700 1 -- 172.21.15.32:0/1489446487 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b800b720 con 0x7fa6d8062400 2020-12-05T17:31:45.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d77fe700 1 --2- 172.21.15.32:0/1489446487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6b80044d0 0x7fa6b8004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6dcf35700 1 --2- 172.21.15.32:0/1489446487 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6d8062400 0x7fa6b8004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d7fff700 1 --2- 172.21.15.32:0/1489446487 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa6b80038e0 0x7fa6b8003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d77fe700 1 -- 172.21.15.32:0/1489446487 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6d8062400 msgr2=0x7fa6b8004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:45.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d77fe700 1 --2- 172.21.15.32:0/1489446487 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6d8062400 0x7fa6b8004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d77fe700 1 -- 172.21.15.32:0/1489446487 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa6b80038e0 msgr2=0x7fa6b8003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:45.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d77fe700 1 --2- 172.21.15.32:0/1489446487 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa6b80038e0 0x7fa6b8003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d77fe700 1 -- 172.21.15.32:0/1489446487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b8003230 con 0x7fa6b80044d0 2020-12-05T17:31:45.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d7fff700 1 --2- 172.21.15.32:0/1489446487 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa6b80038e0 0x7fa6b8003ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:45.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d77fe700 1 --2- 172.21.15.32:0/1489446487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6b80044d0 0x7fa6b8004950 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fa6b80054a0 tx=0x7fa6c0008040).ready entity=mon.0 client_cookie=1b51d05cc3462f8f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:45.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d6ffd700 1 -- 172.21.15.32:0/1489446487 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6c000b4d0 con 0x7fa6b80044d0 2020-12-05T17:31:45.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d6ffd700 1 -- 172.21.15.32:0/1489446487 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c000ec30 con 0x7fa6b80044d0 2020-12-05T17:31:45.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6d6ffd700 1 -- 172.21.15.32:0/1489446487 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6c00123f0 con 0x7fa6b80044d0 2020-12-05T17:31:45.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6dd736700 1 -- 172.21.15.32:0/1489446487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6b80044d0 msgr2=0x7fa6b8004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6dd736700 1 --2- 172.21.15.32:0/1489446487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6b80044d0 0x7fa6b8004950 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fa6b80054a0 tx=0x7fa6c0008040).stop 2020-12-05T17:31:45.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.527+0000 7fa6dcf35700 1 -- 172.21.15.32:0/1489446487 reap_dead start 2020-12-05T17:31:45.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6dd736700 1 -- 172.21.15.32:0/1489446487 shutdown_connections 2020-12-05T17:31:45.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6dd736700 1 -- 172.21.15.32:0/1489446487 >> 172.21.15.32:0/1489446487 conn(0x7fa6d80b8ee0 msgr2=0x7fa6d80b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:45.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6dd736700 1 -- 172.21.15.32:0/1489446487 shutdown_connections 2020-12-05T17:31:45.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6dd736700 1 -- 172.21.15.32:0/1489446487 wait complete. 2020-12-05T17:31:45.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6dd736700 1 Processor -- start 2020-12-05T17:31:45.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6dd736700 1 -- start start 2020-12-05T17:31:45.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6dd736700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6d8148f90 0x7fa6d8154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6dd736700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d8064f60 0x7fa6d80b2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6dd736700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa6d8065b50 0x7fa6d80b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6dd736700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d8064ae0 con 0x7fa6d8064f60 2020-12-05T17:31:45.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6dd736700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d8066dc0 con 0x7fa6d8065b50 2020-12-05T17:31:45.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6dd736700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d80b1de0 con 0x7fa6d8148f90 2020-12-05T17:31:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6d77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d8064f60 0x7fa6d80b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6d77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d8064f60 0x7fa6d80b2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60826/0 (socket says 172.21.15.32:60826) 2020-12-05T17:31:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6d77fe700 1 -- 172.21.15.32:0/3081809551 learned_addr learned my addr 172.21.15.32:0/3081809551 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6d7fff700 1 --2- 172.21.15.32:0/3081809551 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6d8148f90 0x7fa6d8154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6dcf35700 1 --2- 172.21.15.32:0/3081809551 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa6d8065b50 0x7fa6d80b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6d77fe700 1 -- 172.21.15.32:0/3081809551 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6d8148f90 msgr2=0x7fa6d8154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6d77fe700 1 --2- 172.21.15.32:0/3081809551 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6d8148f90 0x7fa6d8154810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6d77fe700 1 -- 172.21.15.32:0/3081809551 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa6d8065b50 msgr2=0x7fa6d80b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6d77fe700 1 --2- 172.21.15.32:0/3081809551 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa6d8065b50 0x7fa6d80b2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6d77fe700 1 -- 172.21.15.32:0/3081809551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d8149370 con 0x7fa6d8064f60 2020-12-05T17:31:45.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6d77fe700 1 --2- 172.21.15.32:0/3081809551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d8064f60 0x7fa6d80b2240 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fa6b80054a0 tx=0x7fa6c001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:45.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6d57fa700 1 -- 172.21.15.32:0/3081809551 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6c0020070 con 0x7fa6d8064f60 2020-12-05T17:31:45.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6d57fa700 1 -- 172.21.15.32:0/3081809551 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c001bdd0 con 0x7fa6d8064f60 2020-12-05T17:31:45.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6dd736700 1 -- 172.21.15.32:0/3081809551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d8064840 con 0x7fa6d8064f60 2020-12-05T17:31:45.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6d57fa700 1 -- 172.21.15.32:0/3081809551 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6c0024600 con 0x7fa6d8064f60 2020-12-05T17:31:45.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.531+0000 7fa6dd736700 1 -- 172.21.15.32:0/3081809551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d80b3f80 con 0x7fa6d8064f60 2020-12-05T17:31:45.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.535+0000 7fa6dd736700 1 -- 172.21.15.32:0/3081809551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d80b1c30 con 0x7fa6d8064f60 2020-12-05T17:31:45.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.535+0000 7fa6d57fa700 1 -- 172.21.15.32:0/3081809551 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa6c0024760 con 0x7fa6d8064f60 2020-12-05T17:31:45.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.535+0000 7fa6d57fa700 1 --2- 172.21.15.32:0/3081809551 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa6bc0553b0 0x7fa6bc057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.535+0000 7fa6d57fa700 1 -- 172.21.15.32:0/3081809551 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa6c006f5f0 con 0x7fa6d8064f60 2020-12-05T17:31:45.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.535+0000 7fa6d7fff700 1 --2- 172.21.15.32:0/3081809551 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa6bc0553b0 0x7fa6bc057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.535+0000 7fa6d7fff700 1 --2- 172.21.15.32:0/3081809551 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa6bc0553b0 0x7fa6bc057870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fa6c800d000 tx=0x7fa6d8061120).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:45.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.539+0000 7fa6d57fa700 1 -- 172.21.15.32:0/3081809551 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa6c00456d0 con 0x7fa6d8064f60 2020-12-05T17:31:45.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.679+0000 7fa6dd736700 1 -- 172.21.15.32:0/3081809551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6d8057050 con 0x7fa6d8064f60 2020-12-05T17:31:45.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.683+0000 7fa6d57fa700 1 -- 172.21.15.32:0/3081809551 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7fa6c0048cf0 con 0x7fa6d8064f60 2020-12-05T17:31:45.687 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:31:45.687 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:31:41.610914+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:31:36.706361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-12-05T17:31:39.298269+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:31:45.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.683+0000 7fa6dd736700 1 -- 172.21.15.32:0/3081809551 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa6bc0553b0 msgr2=0x7fa6bc057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.683+0000 7fa6dd736700 1 --2- 172.21.15.32:0/3081809551 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa6bc0553b0 0x7fa6bc057870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fa6c800d000 tx=0x7fa6d8061120).stop 2020-12-05T17:31:45.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.683+0000 7fa6dd736700 1 -- 172.21.15.32:0/3081809551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d8064f60 msgr2=0x7fa6d80b2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.683+0000 7fa6dd736700 1 --2- 172.21.15.32:0/3081809551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d8064f60 0x7fa6d80b2240 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fa6b80054a0 tx=0x7fa6c001a000).stop 2020-12-05T17:31:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.683+0000 7fa6dd736700 1 -- 172.21.15.32:0/3081809551 shutdown_connections 2020-12-05T17:31:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.683+0000 7fa6dd736700 1 --2- 172.21.15.32:0/3081809551 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa6d8065b50 0x7fa6d80b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.683+0000 7fa6dd736700 1 --2- 172.21.15.32:0/3081809551 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa6bc0553b0 0x7fa6bc057870 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.683+0000 7fa6dd736700 1 --2- 172.21.15.32:0/3081809551 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6d8148f90 0x7fa6d8154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.683+0000 7fa6dd736700 1 --2- 172.21.15.32:0/3081809551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d8064f60 0x7fa6d80b2240 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.683+0000 7fa6dd736700 1 -- 172.21.15.32:0/3081809551 >> 172.21.15.32:0/3081809551 conn(0x7fa6d80b8ee0 msgr2=0x7fa6d8145f00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:45.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.683+0000 7fa6dd736700 1 -- 172.21.15.32:0/3081809551 shutdown_connections 2020-12-05T17:31:45.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.683+0000 7fa6dd736700 1 -- 172.21.15.32:0/3081809551 wait complete. 2020-12-05T17:31:45.700 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.0 flush_pg_stats 2020-12-05T17:31:45.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.811+0000 7f18f3c7d700 1 Processor -- start 2020-12-05T17:31:45.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.811+0000 7f18f3c7d700 1 -- start start 2020-12-05T17:31:45.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.811+0000 7f18f3c7d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18ec0af860 0x7f18ec066ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.811+0000 7f18f3c7d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f18ec0ad290 0x7f18ec13fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.811+0000 7f18f3c7d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18ec0adf20 con 0x7f18ec067480 2020-12-05T17:31:45.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.811+0000 7f18f3c7d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18ec1436f0 con 0x7f18ec0ad290 2020-12-05T17:31:45.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.811+0000 7f18f3c7d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec1438f0 con 0x7f18ec0af860 2020-12-05T17:31:45.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.811+0000 7f18f221a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f18ec0ad290 0x7f18ec13fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.811+0000 7f18f1218700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f18ec067480 0x7f18ec0acb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35248/0 (socket says 172.21.15.32:35248) 2020-12-05T17:31:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.815+0000 7f18f1218700 1 -- 172.21.15.32:0/2531120451 learned_addr learned my addr 172.21.15.32:0/2531120451 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.815+0000 7f18f1a19700 1 --2- 172.21.15.32:0/2531120451 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18ec0af860 0x7f18ec066ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.815+0000 7f18f221a700 1 -- 172.21.15.32:0/2531120451 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18ec0af860 msgr2=0x7f18ec066ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:45.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.815+0000 7f18f221a700 1 --2- 172.21.15.32:0/2531120451 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18ec0af860 0x7f18ec066ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.815+0000 7f18f221a700 1 -- 172.21.15.32:0/2531120451 >> v1:172.21.15.32:6789/0 conn(0x7f18ec067480 legacy=0x7f18ec0acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.815+0000 7f18f221a700 1 -- 172.21.15.32:0/2531120451 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec0afc40 con 0x7f18ec0ad290 2020-12-05T17:31:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.815+0000 7f18f221a700 1 --2- 172.21.15.32:0/2531120451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ec0ad290 0x7f18ec13fc90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f18e40032b0 tx=0x7f18e40091a0).ready entity=mon.0 client_cookie=2c72509558487c9b server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:45.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.815+0000 7f18f0a17700 1 -- 172.21.15.32:0/2531120451 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18e400b9a0 con 0x7f18ec0ad290 2020-12-05T17:31:45.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.815+0000 7f18f0a17700 1 -- 172.21.15.32:0/2531120451 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e400bb00 con 0x7f18ec0ad290 2020-12-05T17:31:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.815+0000 7f18f0a17700 1 -- 172.21.15.32:0/2531120451 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18e400bc70 con 0x7f18ec0ad290 2020-12-05T17:31:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.815+0000 7f18f3c7d700 1 -- 172.21.15.32:0/2531120451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ec0ad290 msgr2=0x7f18ec13fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.815+0000 7f18f3c7d700 1 --2- 172.21.15.32:0/2531120451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ec0ad290 0x7f18ec13fc90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f18e40032b0 tx=0x7f18e40091a0).stop 2020-12-05T17:31:45.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.815+0000 7f18f3c7d700 1 -- 172.21.15.32:0/2531120451 shutdown_connections 2020-12-05T17:31:45.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.815+0000 7f18f3c7d700 1 --2- 172.21.15.32:0/2531120451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ec0ad290 0x7f18ec13fc90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.815+0000 7f18f3c7d700 1 --2- 172.21.15.32:0/2531120451 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18ec0af860 0x7f18ec066ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.815+0000 7f18f3c7d700 1 -- 172.21.15.32:0/2531120451 >> 172.21.15.32:0/2531120451 conn(0x7f18ec0b5580 msgr2=0x7f18ec0b5990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:45.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.815+0000 7f18f3c7d700 1 -- 172.21.15.32:0/2531120451 shutdown_connections 2020-12-05T17:31:45.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.815+0000 7f18f3c7d700 1 -- 172.21.15.32:0/2531120451 wait complete. 2020-12-05T17:31:45.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.815+0000 7f18f3c7d700 1 Processor -- start 2020-12-05T17:31:45.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.815+0000 7f18f3c7d700 1 -- start start 2020-12-05T17:31:45.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18f3c7d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ec067480 0x7f18ec140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18f3c7d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18ec0ad290 0x7f18ec1470b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18f3c7d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18ec0af860 0x7f18ec1475a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18f3c7d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec1459e0 con 0x7f18ec067480 2020-12-05T17:31:45.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18f3c7d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec145c10 con 0x7f18ec0af860 2020-12-05T17:31:45.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18f3c7d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec145eb0 con 0x7f18ec0ad290 2020-12-05T17:31:45.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18f1a19700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ec067480 0x7f18ec140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18f1a19700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ec067480 0x7f18ec140380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60838/0 (socket says 172.21.15.32:60838) 2020-12-05T17:31:45.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18f1a19700 1 -- 172.21.15.32:0/2955526385 learned_addr learned my addr 172.21.15.32:0/2955526385 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:45.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18f1218700 1 --2- 172.21.15.32:0/2955526385 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18ec0ad290 0x7f18ec1470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18f221a700 1 --2- 172.21.15.32:0/2955526385 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18ec0af860 0x7f18ec1475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18f1a19700 1 -- 172.21.15.32:0/2955526385 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18ec0ad290 msgr2=0x7f18ec1470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18f1a19700 1 --2- 172.21.15.32:0/2955526385 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18ec0ad290 0x7f18ec1470b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18f1a19700 1 -- 172.21.15.32:0/2955526385 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18ec0af860 msgr2=0x7f18ec1475a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18f1a19700 1 --2- 172.21.15.32:0/2955526385 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18ec0af860 0x7f18ec1475a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18f1a19700 1 -- 172.21.15.32:0/2955526385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec147a90 con 0x7f18ec067480 2020-12-05T17:31:45.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18f221a700 1 --2- 172.21.15.32:0/2955526385 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18ec0af860 0x7f18ec1475a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:45.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18f1a19700 1 --2- 172.21.15.32:0/2955526385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ec067480 0x7f18ec140380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f18dc00c000 tx=0x7f18dc006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:45.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18e2ffd700 1 -- 172.21.15.32:0/2955526385 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18dc00ca10 con 0x7f18ec067480 2020-12-05T17:31:45.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18e2ffd700 1 -- 172.21.15.32:0/2955526385 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18dc00cb70 con 0x7f18ec067480 2020-12-05T17:31:45.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18e2ffd700 1 -- 172.21.15.32:0/2955526385 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18dc0123f0 con 0x7f18ec067480 2020-12-05T17:31:45.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18f3c7d700 1 -- 172.21.15.32:0/2955526385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ec1455e0 con 0x7f18ec067480 2020-12-05T17:31:45.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18f3c7d700 1 -- 172.21.15.32:0/2955526385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ec146b20 con 0x7f18ec067480 2020-12-05T17:31:45.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.819+0000 7f18f3c7d700 1 -- 172.21.15.32:0/2955526385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f18ec057050 con 0x7f18ec067480 2020-12-05T17:31:45.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.823+0000 7f18e2ffd700 1 -- 172.21.15.32:0/2955526385 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f18dc012550 con 0x7f18ec067480 2020-12-05T17:31:45.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.823+0000 7f18e2ffd700 1 --2- 172.21.15.32:0/2955526385 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f18cc055440 0x7f18cc057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.823+0000 7f18e2ffd700 1 -- 172.21.15.32:0/2955526385 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f18dc06f260 con 0x7f18ec067480 2020-12-05T17:31:45.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.823+0000 7f18e2ffd700 1 --2- 172.21.15.32:0/2955526385 >> [v2:172.21.15.32:6802/11526,v1:172.21.15.32:6803/11526] conn(0x7f18cc05cec0 0x7f18cc05f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.823+0000 7f18f1218700 1 --2- 172.21.15.32:0/2955526385 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f18cc055440 0x7f18cc057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.823+0000 7f18e2ffd700 1 -- 172.21.15.32:0/2955526385 --> [v2:172.21.15.32:6802/11526,v1:172.21.15.32:6803/11526] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f18cc05f8a0 con 0x7f18cc05cec0 2020-12-05T17:31:45.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.823+0000 7f18e2ffd700 1 -- 172.21.15.32:0/2955526385 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18dc06f530 con 0x7f18ec067480 2020-12-05T17:31:45.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.823+0000 7f18f221a700 1 --2- 172.21.15.32:0/2955526385 >> [v2:172.21.15.32:6802/11526,v1:172.21.15.32:6803/11526] conn(0x7f18cc05cec0 0x7f18cc05f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.823+0000 7f18f1218700 1 --2- 172.21.15.32:0/2955526385 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f18cc055440 0x7f18cc057900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f18e800d000 tx=0x7f18e8009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:45.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.823+0000 7f18f221a700 1 --2- 172.21.15.32:0/2955526385 >> [v2:172.21.15.32:6802/11526,v1:172.21.15.32:6803/11526] conn(0x7f18cc05cec0 0x7f18cc05f2e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:45.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.827+0000 7f18e2ffd700 1 -- 172.21.15.32:0/2955526385 <== osd.0 v2:172.21.15.32:6802/11526 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f18cc05f8a0 con 0x7f18cc05cec0 2020-12-05T17:31:45.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.847+0000 7f18f3c7d700 1 -- 172.21.15.32:0/2955526385 --> [v2:172.21.15.32:6802/11526,v1:172.21.15.32:6803/11526] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f18ec1492b0 con 0x7f18cc05cec0 2020-12-05T17:31:45.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.847+0000 7f18e2ffd700 1 -- 172.21.15.32:0/2955526385 <== osd.0 v2:172.21.15.32:6802/11526 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f18ec1492b0 con 0x7f18cc05cec0 2020-12-05T17:31:45.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.847+0000 7f18f3c7d700 1 -- 172.21.15.32:0/2955526385 >> [v2:172.21.15.32:6802/11526,v1:172.21.15.32:6803/11526] conn(0x7f18cc05cec0 msgr2=0x7f18cc05f2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.847+0000 7f18f3c7d700 1 --2- 172.21.15.32:0/2955526385 >> [v2:172.21.15.32:6802/11526,v1:172.21.15.32:6803/11526] conn(0x7f18cc05cec0 0x7f18cc05f2e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.847+0000 7f18f3c7d700 1 -- 172.21.15.32:0/2955526385 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f18cc055440 msgr2=0x7f18cc057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.847+0000 7f18f3c7d700 1 --2- 172.21.15.32:0/2955526385 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f18cc055440 0x7f18cc057900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f18e800d000 tx=0x7f18e8009040).stop 2020-12-05T17:31:45.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.847+0000 7f18f3c7d700 1 -- 172.21.15.32:0/2955526385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ec067480 msgr2=0x7f18ec140380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.847+0000 7f18f3c7d700 1 --2- 172.21.15.32:0/2955526385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ec067480 0x7f18ec140380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f18dc00c000 tx=0x7f18dc006dc0).stop 2020-12-05T17:31:45.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.847+0000 7f18f221a700 1 -- 172.21.15.32:0/2955526385 reap_dead start 2020-12-05T17:31:45.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.847+0000 7f18f3c7d700 1 -- 172.21.15.32:0/2955526385 shutdown_connections 2020-12-05T17:31:45.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.847+0000 7f18f3c7d700 1 -- 172.21.15.32:0/2955526385 >> 172.21.15.32:0/2955526385 conn(0x7f18ec0b5580 msgr2=0x7f18ec0b4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:45.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.847+0000 7f18f3c7d700 1 -- 172.21.15.32:0/2955526385 shutdown_connections 2020-12-05T17:31:45.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.847+0000 7f18f3c7d700 1 -- 172.21.15.32:0/2955526385 wait complete. 2020-12-05T17:31:45.863 INFO:teuthology.orchestra.run.smithi032.stdout:60129542150 2020-12-05T17:31:45.864 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.1 flush_pg_stats 2020-12-05T17:31:45.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.975+0000 7f047b59e700 1 Processor -- start 2020-12-05T17:31:45.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.975+0000 7f047b59e700 1 -- start start 2020-12-05T17:31:45.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f047b59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f047c1421e0 0x7f047c143140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f047b59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f047c0af300 con 0x7f047c0afd20 2020-12-05T17:31:45.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f047b59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f047c0b5a30 con 0x7f047c0b5650 2020-12-05T17:31:45.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f047b59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047c142650 con 0x7f047c1421e0 2020-12-05T17:31:45.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f0471d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f047c0afd20 0x7f047c142a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35264/0 (socket says 172.21.15.32:35264) 2020-12-05T17:31:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f0471d9b700 1 -- 172.21.15.32:0/226205307 learned_addr learned my addr 172.21.15.32:0/226205307 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f0479d9b700 1 -- 172.21.15.32:0/226205307 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1993425498 0 0) 0x7f047c0af300 con 0x7f047c0afd20 2020-12-05T17:31:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f047ad9d700 1 --2- 172.21.15.32:0/226205307 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f047c1421e0 0x7f047c143140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:45.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f0479d9b700 1 -- 172.21.15.32:0/226205307 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f045c003230 con 0x7f047c0afd20 2020-12-05T17:31:45.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f0479d9b700 1 -- 172.21.15.32:0/226205307 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 346872945 0 0) 0x7f045c003230 con 0x7f047c0afd20 2020-12-05T17:31:45.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f0479d9b700 1 -- 172.21.15.32:0/226205307 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f047c1421e0 msgr2=0x7f047c143140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:45.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f0479d9b700 1 --2- 172.21.15.32:0/226205307 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f047c1421e0 0x7f047c143140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:45.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f0479d9b700 1 -- 172.21.15.32:0/226205307 >> v1:172.21.15.61:6789/0 conn(0x7f047c0b5650 legacy=0x7f047c141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f0479d9b700 1 -- 172.21.15.32:0/226205307 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c0aea90 con 0x7f047c0afd20 2020-12-05T17:31:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f0479d9b700 1 -- 172.21.15.32:0/226205307 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f04640016d0 con 0x7f047c0afd20 2020-12-05T17:31:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f0479d9b700 1 -- 172.21.15.32:0/226205307 >> v1:172.21.15.32:6789/0 conn(0x7f047c0afd20 legacy=0x7f047c142a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f0479d9b700 1 --2- 172.21.15.32:0/226205307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f045c003750 0x7f045c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f0479d9b700 1 --2- 172.21.15.32:0/226205307 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f047c1421e0 0x7f045c0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f0479d9b700 1 --2- 172.21.15.32:0/226205307 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f045c0049c0 0x7f045c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f0479d9b700 1 -- 172.21.15.32:0/226205307 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c142650 con 0x7f045c003750 2020-12-05T17:31:46.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f047a59c700 1 --2- 172.21.15.32:0/226205307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f045c003750 0x7f045c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f0479d9b700 1 -- 172.21.15.32:0/226205307 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045c0041f0 con 0x7f045c0049c0 2020-12-05T17:31:46.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f0479d9b700 1 -- 172.21.15.32:0/226205307 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f045c00b720 con 0x7f047c1421e0 2020-12-05T17:31:46.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f047ad9d700 1 --2- 172.21.15.32:0/226205307 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f045c0049c0 0x7f045c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f0471d9b700 1 --2- 172.21.15.32:0/226205307 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f047c1421e0 0x7f045c0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f047a59c700 1 -- 172.21.15.32:0/226205307 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f047c1421e0 msgr2=0x7f045c0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:46.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f047a59c700 1 --2- 172.21.15.32:0/226205307 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f047c1421e0 0x7f045c0044d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f047a59c700 1 -- 172.21.15.32:0/226205307 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f045c0049c0 msgr2=0x7f045c004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:46.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f047a59c700 1 --2- 172.21.15.32:0/226205307 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f045c0049c0 0x7f045c004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f047a59c700 1 -- 172.21.15.32:0/226205307 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045c003230 con 0x7f045c003750 2020-12-05T17:31:46.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f047ad9d700 1 --2- 172.21.15.32:0/226205307 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f045c0049c0 0x7f045c004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:31:46.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f047a59c700 1 --2- 172.21.15.32:0/226205307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f045c003750 0x7f045c003b70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f047c05cd40 tx=0x7f046c007b60).ready entity=mon.0 client_cookie=62671ee3b69f0571 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f0479d9b700 1 -- 172.21.15.32:0/226205307 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f046c00c660 con 0x7f045c003750 2020-12-05T17:31:46.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f0479d9b700 1 -- 172.21.15.32:0/226205307 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046c00c7c0 con 0x7f045c003750 2020-12-05T17:31:46.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f0479d9b700 1 -- 172.21.15.32:0/226205307 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f046c00cb20 con 0x7f045c003750 2020-12-05T17:31:46.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f047b59e700 1 -- 172.21.15.32:0/226205307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f045c003750 msgr2=0x7f045c003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f047b59e700 1 --2- 172.21.15.32:0/226205307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f045c003750 0x7f045c003b70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f047c05cd40 tx=0x7f046c007b60).stop 2020-12-05T17:31:46.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f047ad9d700 1 -- 172.21.15.32:0/226205307 reap_dead start 2020-12-05T17:31:46.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f047b59e700 1 -- 172.21.15.32:0/226205307 shutdown_connections 2020-12-05T17:31:46.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f047b59e700 1 -- 172.21.15.32:0/226205307 >> 172.21.15.32:0/226205307 conn(0x7f047c05bbb0 msgr2=0x7f047c062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:46.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f047b59e700 1 -- 172.21.15.32:0/226205307 shutdown_connections 2020-12-05T17:31:46.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.979+0000 7f047b59e700 1 -- 172.21.15.32:0/226205307 wait complete. 2020-12-05T17:31:46.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f047b59e700 1 Processor -- start 2020-12-05T17:31:46.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f047b59e700 1 -- start start 2020-12-05T17:31:46.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f047b59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f047c147ed0 0x7f047c0b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f047b59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f047c0b1cf0 0x7f047c151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f047b59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f047c112450 0x7f047c151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f047b59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c152490 con 0x7f047c147ed0 2020-12-05T17:31:46.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f047b59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c10ecb0 con 0x7f047c112450 2020-12-05T17:31:46.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f047b59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047c1482b0 con 0x7f047c0b1cf0 2020-12-05T17:31:46.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f047a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f047c147ed0 0x7f047c0b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f047a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f047c147ed0 0x7f047c0b08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60860/0 (socket says 172.21.15.32:60860) 2020-12-05T17:31:46.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f047a59c700 1 -- 172.21.15.32:0/262757701 learned_addr learned my addr 172.21.15.32:0/262757701 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:46.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f0471d9b700 1 --2- 172.21.15.32:0/262757701 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f047c0b1cf0 0x7f047c151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f047ad9d700 1 --2- 172.21.15.32:0/262757701 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f047c112450 0x7f047c151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f047a59c700 1 -- 172.21.15.32:0/262757701 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f047c0b1cf0 msgr2=0x7f047c151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f047a59c700 1 --2- 172.21.15.32:0/262757701 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f047c0b1cf0 0x7f047c151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f047a59c700 1 -- 172.21.15.32:0/262757701 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f047c112450 msgr2=0x7f047c151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f047a59c700 1 --2- 172.21.15.32:0/262757701 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f047c112450 0x7f047c151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f047a59c700 1 -- 172.21.15.32:0/262757701 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c10ea10 con 0x7f047c147ed0 2020-12-05T17:31:46.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f0471d9b700 1 --2- 172.21.15.32:0/262757701 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f047c0b1cf0 0x7f047c151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:31:46.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f047ad9d700 1 --2- 172.21.15.32:0/262757701 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f047c112450 0x7f047c151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:31:46.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f047a59c700 1 --2- 172.21.15.32:0/262757701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f047c147ed0 0x7f047c0b08e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f047c0b1ab0 tx=0x7f046c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f0473fff700 1 -- 172.21.15.32:0/262757701 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f046c001ea0 con 0x7f047c147ed0 2020-12-05T17:31:46.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f047b59e700 1 -- 172.21.15.32:0/262757701 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f047c151610 con 0x7f047c147ed0 2020-12-05T17:31:46.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f0473fff700 1 -- 172.21.15.32:0/262757701 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046c012760 con 0x7f047c147ed0 2020-12-05T17:31:46.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f0473fff700 1 -- 172.21.15.32:0/262757701 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f046c012ac0 con 0x7f047c147ed0 2020-12-05T17:31:46.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f047b59e700 1 -- 172.21.15.32:0/262757701 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f047c13de90 con 0x7f047c147ed0 2020-12-05T17:31:46.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.983+0000 7f047b59e700 1 -- 172.21.15.32:0/262757701 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f047c151460 con 0x7f047c147ed0 2020-12-05T17:31:46.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.987+0000 7f0473fff700 1 -- 172.21.15.32:0/262757701 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f046c010070 con 0x7f047c147ed0 2020-12-05T17:31:46.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.987+0000 7f0473fff700 1 --2- 172.21.15.32:0/262757701 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0460055330 0x7f04600577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.987+0000 7f0473fff700 1 -- 172.21.15.32:0/262757701 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f046c06ead0 con 0x7f047c147ed0 2020-12-05T17:31:46.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.987+0000 7f0471d9b700 1 --2- 172.21.15.32:0/262757701 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0460055330 0x7f04600577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.987+0000 7f0473fff700 1 --2- 172.21.15.32:0/262757701 >> [v2:172.21.15.32:6818/11538,v1:172.21.15.32:6819/11538] conn(0x7f046005cdd0 0x7f046005f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.987+0000 7f0473fff700 1 -- 172.21.15.32:0/262757701 --> [v2:172.21.15.32:6818/11538,v1:172.21.15.32:6819/11538] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f046005f850 con 0x7f046005cdd0 2020-12-05T17:31:46.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.987+0000 7f0473fff700 1 -- 172.21.15.32:0/262757701 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f046c073220 con 0x7f047c147ed0 2020-12-05T17:31:46.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.987+0000 7f047ad9d700 1 --2- 172.21.15.32:0/262757701 >> [v2:172.21.15.32:6818/11538,v1:172.21.15.32:6819/11538] conn(0x7f046005cdd0 0x7f046005f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.987+0000 7f0471d9b700 1 --2- 172.21.15.32:0/262757701 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0460055330 0x7f04600577f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f0464009000 tx=0x7f0464009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.987+0000 7f047ad9d700 1 --2- 172.21.15.32:0/262757701 >> [v2:172.21.15.32:6818/11538,v1:172.21.15.32:6819/11538] conn(0x7f046005cdd0 0x7f046005f1f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:45.987+0000 7f0473fff700 1 -- 172.21.15.32:0/262757701 <== osd.1 v2:172.21.15.32:6818/11538 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f046005f850 con 0x7f046005cdd0 2020-12-05T17:31:46.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.011+0000 7f047b59e700 1 -- 172.21.15.32:0/262757701 --> [v2:172.21.15.32:6818/11538,v1:172.21.15.32:6819/11538] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f047c13e7a0 con 0x7f046005cdd0 2020-12-05T17:31:46.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.011+0000 7f0473fff700 1 -- 172.21.15.32:0/262757701 <== osd.1 v2:172.21.15.32:6818/11538 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f047c13e7a0 con 0x7f046005cdd0 2020-12-05T17:31:46.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.011+0000 7f047b59e700 1 -- 172.21.15.32:0/262757701 >> [v2:172.21.15.32:6818/11538,v1:172.21.15.32:6819/11538] conn(0x7f046005cdd0 msgr2=0x7f046005f1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.011+0000 7f047b59e700 1 --2- 172.21.15.32:0/262757701 >> [v2:172.21.15.32:6818/11538,v1:172.21.15.32:6819/11538] conn(0x7f046005cdd0 0x7f046005f1f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.011+0000 7f047b59e700 1 -- 172.21.15.32:0/262757701 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0460055330 msgr2=0x7f04600577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.011+0000 7f047b59e700 1 --2- 172.21.15.32:0/262757701 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0460055330 0x7f04600577f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f0464009000 tx=0x7f0464009040).stop 2020-12-05T17:31:46.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.011+0000 7f047b59e700 1 -- 172.21.15.32:0/262757701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f047c147ed0 msgr2=0x7f047c0b08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.011+0000 7f047b59e700 1 --2- 172.21.15.32:0/262757701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f047c147ed0 0x7f047c0b08e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f047c0b1ab0 tx=0x7f046c017000).stop 2020-12-05T17:31:46.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.011+0000 7f047ad9d700 1 -- 172.21.15.32:0/262757701 reap_dead start 2020-12-05T17:31:46.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.011+0000 7f047b59e700 1 -- 172.21.15.32:0/262757701 shutdown_connections 2020-12-05T17:31:46.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.011+0000 7f047b59e700 1 -- 172.21.15.32:0/262757701 >> 172.21.15.32:0/262757701 conn(0x7f047c05bbb0 msgr2=0x7f047c0b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:46.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.011+0000 7f047b59e700 1 -- 172.21.15.32:0/262757701 shutdown_connections 2020-12-05T17:31:46.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.011+0000 7f047b59e700 1 -- 172.21.15.32:0/262757701 wait complete. 2020-12-05T17:31:46.032 INFO:teuthology.orchestra.run.smithi032.stdout:60129542149 2020-12-05T17:31:46.032 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.2 flush_pg_stats 2020-12-05T17:31:46.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.147+0000 7ff6959a0700 1 Processor -- start 2020-12-05T17:31:46.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.147+0000 7ff6959a0700 1 -- start start 2020-12-05T17:31:46.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff6959a0700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff6900b1e20 0x7ff690143b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff6959a0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6900aec10 0x7ff690144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff6959a0700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6900ab4a0 con 0x7ff6900abb70 2020-12-05T17:31:46.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff6959a0700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6900ad800 con 0x7ff6900b1e20 2020-12-05T17:31:46.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff6959a0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6900ad170 con 0x7ff6900aec10 2020-12-05T17:31:46.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff68ffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6900aec10 0x7ff690144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff68ffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6900aec10 0x7ff690144080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:57660/0 (socket says 172.21.15.32:57660) 2020-12-05T17:31:46.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff69499e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff6900b1e20 0x7ff690143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff68ffff700 1 -- 172.21.15.32:0/1564733494 learned_addr learned my addr 172.21.15.32:0/1564733494 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:46.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff68f7fe700 1 -- 172.21.15.32:0/1564733494 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2052703828 0 0) 0x7ff6900ab4a0 con 0x7ff6900abb70 2020-12-05T17:31:46.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff68ffff700 1 -- 172.21.15.32:0/1564733494 >> v2:172.21.15.61:3300/0 conn(0x7ff6900b1e20 msgr2=0x7ff690143b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:46.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff68ffff700 1 --2- 172.21.15.32:0/1564733494 >> v2:172.21.15.61:3300/0 conn(0x7ff6900b1e20 0x7ff690143b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff68ffff700 1 -- 172.21.15.32:0/1564733494 >> v1:172.21.15.61:6789/0 conn(0x7ff6900abb70 legacy=0x7ff69013f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff68ffff700 1 -- 172.21.15.32:0/1564733494 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6900aca70 con 0x7ff6900aec10 2020-12-05T17:31:46.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff68ffff700 1 --2- 172.21.15.32:0/1564733494 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6900aec10 0x7ff690144080 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7ff684002f30 tx=0x7ff684006e80).ready entity=mon.2 client_cookie=4a9885951c732f2f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff68f7fe700 1 -- 172.21.15.32:0/1564733494 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6840097f0 con 0x7ff6900aec10 2020-12-05T17:31:46.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff68f7fe700 1 -- 172.21.15.32:0/1564733494 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff684009950 con 0x7ff6900aec10 2020-12-05T17:31:46.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff6959a0700 1 -- 172.21.15.32:0/1564733494 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6900aec10 msgr2=0x7ff690144080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff68f7fe700 1 -- 172.21.15.32:0/1564733494 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff684009cb0 con 0x7ff6900aec10 2020-12-05T17:31:46.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff6959a0700 1 --2- 172.21.15.32:0/1564733494 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6900aec10 0x7ff690144080 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7ff684002f30 tx=0x7ff684006e80).stop 2020-12-05T17:31:46.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff6959a0700 1 -- 172.21.15.32:0/1564733494 shutdown_connections 2020-12-05T17:31:46.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff6959a0700 1 --2- 172.21.15.32:0/1564733494 >> v2:172.21.15.61:3300/0 conn(0x7ff6900b1e20 0x7ff690143b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff6959a0700 1 --2- 172.21.15.32:0/1564733494 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6900aec10 0x7ff690144080 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff6959a0700 1 -- 172.21.15.32:0/1564733494 >> 172.21.15.32:0/1564733494 conn(0x7ff6900b5530 msgr2=0x7ff6900b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:46.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff6959a0700 1 -- 172.21.15.32:0/1564733494 shutdown_connections 2020-12-05T17:31:46.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.151+0000 7ff6959a0700 1 -- 172.21.15.32:0/1564733494 wait complete. 2020-12-05T17:31:46.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff6959a0700 1 Processor -- start 2020-12-05T17:31:46.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff6959a0700 1 -- start start 2020-12-05T17:31:46.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff6959a0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6900abb70 0x7ff690144300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff6959a0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6900aec10 0x7ff6900b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff6959a0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6900b1e20 0x7ff6901527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff6959a0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff690145ab0 con 0x7ff6900abb70 2020-12-05T17:31:46.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff6959a0700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff690143330 con 0x7ff6900aec10 2020-12-05T17:31:46.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff6959a0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6901435d0 con 0x7ff6900b1e20 2020-12-05T17:31:46.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff69499e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6900abb70 0x7ff690144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff69499e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6900abb70 0x7ff690144300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60876/0 (socket says 172.21.15.32:60876) 2020-12-05T17:31:46.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff69499e700 1 -- 172.21.15.32:0/2898239989 learned_addr learned my addr 172.21.15.32:0/2898239989 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:46.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff68ffff700 1 --2- 172.21.15.32:0/2898239989 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6900aec10 0x7ff6900b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff69519f700 1 --2- 172.21.15.32:0/2898239989 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6900b1e20 0x7ff6901527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff69499e700 1 -- 172.21.15.32:0/2898239989 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6900b1e20 msgr2=0x7ff6901527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff69499e700 1 --2- 172.21.15.32:0/2898239989 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6900b1e20 0x7ff6901527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff69499e700 1 -- 172.21.15.32:0/2898239989 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6900aec10 msgr2=0x7ff6900b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff69499e700 1 --2- 172.21.15.32:0/2898239989 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6900aec10 0x7ff6900b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff69499e700 1 -- 172.21.15.32:0/2898239989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff690142020 con 0x7ff6900abb70 2020-12-05T17:31:46.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff69499e700 1 --2- 172.21.15.32:0/2898239989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6900abb70 0x7ff690144300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7ff68000c000 tx=0x7ff680006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff68dffb700 1 -- 172.21.15.32:0/2898239989 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff68000ca10 con 0x7ff6900abb70 2020-12-05T17:31:46.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff68dffb700 1 -- 172.21.15.32:0/2898239989 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68000cb70 con 0x7ff6900abb70 2020-12-05T17:31:46.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff6959a0700 1 -- 172.21.15.32:0/2898239989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6901423b0 con 0x7ff6900abb70 2020-12-05T17:31:46.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff68dffb700 1 -- 172.21.15.32:0/2898239989 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6800123f0 con 0x7ff6900abb70 2020-12-05T17:31:46.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.155+0000 7ff6959a0700 1 -- 172.21.15.32:0/2898239989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff690146100 con 0x7ff6900abb70 2020-12-05T17:31:46.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.159+0000 7ff6959a0700 1 -- 172.21.15.32:0/2898239989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff690145c20 con 0x7ff6900abb70 2020-12-05T17:31:46.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.159+0000 7ff68dffb700 1 -- 172.21.15.32:0/2898239989 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff680012550 con 0x7ff6900abb70 2020-12-05T17:31:46.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.159+0000 7ff68dffb700 1 --2- 172.21.15.32:0/2898239989 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff6700554e0 0x7ff6700579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.159+0000 7ff68dffb700 1 -- 172.21.15.32:0/2898239989 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff68006e4a0 con 0x7ff6900abb70 2020-12-05T17:31:46.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.159+0000 7ff68ffff700 1 --2- 172.21.15.32:0/2898239989 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff6700554e0 0x7ff6700579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.159+0000 7ff68dffb700 1 --2- 172.21.15.32:0/2898239989 >> [v2:172.21.15.32:6810/11521,v1:172.21.15.32:6811/11521] conn(0x7ff67005cf60 0x7ff67005f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.159+0000 7ff68dffb700 1 -- 172.21.15.32:0/2898239989 --> [v2:172.21.15.32:6810/11521,v1:172.21.15.32:6811/11521] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff67005f940 con 0x7ff67005cf60 2020-12-05T17:31:46.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.159+0000 7ff68dffb700 1 -- 172.21.15.32:0/2898239989 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff680070a20 con 0x7ff6900abb70 2020-12-05T17:31:46.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.159+0000 7ff69519f700 1 --2- 172.21.15.32:0/2898239989 >> [v2:172.21.15.32:6810/11521,v1:172.21.15.32:6811/11521] conn(0x7ff67005cf60 0x7ff67005f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.159+0000 7ff68ffff700 1 --2- 172.21.15.32:0/2898239989 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff6700554e0 0x7ff6700579a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7ff68401c000 tx=0x7ff684015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.159+0000 7ff69519f700 1 --2- 172.21.15.32:0/2898239989 >> [v2:172.21.15.32:6810/11521,v1:172.21.15.32:6811/11521] conn(0x7ff67005cf60 0x7ff67005f380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.163+0000 7ff68dffb700 1 -- 172.21.15.32:0/2898239989 <== osd.2 v2:172.21.15.32:6810/11521 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7ff67005f940 con 0x7ff67005cf60 2020-12-05T17:31:46.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.183+0000 7ff6959a0700 1 -- 172.21.15.32:0/2898239989 --> [v2:172.21.15.32:6810/11521,v1:172.21.15.32:6811/11521] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff690146950 con 0x7ff67005cf60 2020-12-05T17:31:46.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.183+0000 7ff68dffb700 1 -- 172.21.15.32:0/2898239989 <== osd.2 v2:172.21.15.32:6810/11521 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff690146950 con 0x7ff67005cf60 2020-12-05T17:31:46.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.183+0000 7ff6959a0700 1 -- 172.21.15.32:0/2898239989 >> [v2:172.21.15.32:6810/11521,v1:172.21.15.32:6811/11521] conn(0x7ff67005cf60 msgr2=0x7ff67005f380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.183+0000 7ff6959a0700 1 --2- 172.21.15.32:0/2898239989 >> [v2:172.21.15.32:6810/11521,v1:172.21.15.32:6811/11521] conn(0x7ff67005cf60 0x7ff67005f380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.183+0000 7ff6959a0700 1 -- 172.21.15.32:0/2898239989 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff6700554e0 msgr2=0x7ff6700579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.183+0000 7ff6959a0700 1 --2- 172.21.15.32:0/2898239989 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff6700554e0 0x7ff6700579a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7ff68401c000 tx=0x7ff684015000).stop 2020-12-05T17:31:46.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.183+0000 7ff6959a0700 1 -- 172.21.15.32:0/2898239989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6900abb70 msgr2=0x7ff690144300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.183+0000 7ff6959a0700 1 --2- 172.21.15.32:0/2898239989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6900abb70 0x7ff690144300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7ff68000c000 tx=0x7ff680006dc0).stop 2020-12-05T17:31:46.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.183+0000 7ff69519f700 1 -- 172.21.15.32:0/2898239989 reap_dead start 2020-12-05T17:31:46.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.183+0000 7ff6959a0700 1 -- 172.21.15.32:0/2898239989 shutdown_connections 2020-12-05T17:31:46.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.183+0000 7ff6959a0700 1 -- 172.21.15.32:0/2898239989 >> 172.21.15.32:0/2898239989 conn(0x7ff6900b5530 msgr2=0x7ff6900625c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:46.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.183+0000 7ff6959a0700 1 -- 172.21.15.32:0/2898239989 shutdown_connections 2020-12-05T17:31:46.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.183+0000 7ff6959a0700 1 -- 172.21.15.32:0/2898239989 wait complete. 2020-12-05T17:31:46.200 INFO:teuthology.orchestra.run.smithi032.stdout:60129542149 2020-12-05T17:31:46.201 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.3 flush_pg_stats 2020-12-05T17:31:46.324 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.319+0000 7f4bb514f700 1 Processor -- start 2020-12-05T17:31:46.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.319+0000 7f4bb514f700 1 -- start start 2020-12-05T17:31:46.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.323+0000 7f4bb514f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4bb0062400 0x7f4bb005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.323+0000 7f4bb514f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb0142450 con 0x7f4bb005feb0 2020-12-05T17:31:46.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.323+0000 7f4bb514f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb0061780 con 0x7f4bb005f3b0 2020-12-05T17:31:46.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.323+0000 7f4bb514f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bb0061a20 con 0x7f4bb0062400 2020-12-05T17:31:46.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.323+0000 7f4baf7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4bb0062400 0x7f4bb005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.323+0000 7f4bb494e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4bb005feb0 0x7f4bb00b0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35306/0 (socket says 172.21.15.32:35306) 2020-12-05T17:31:46.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.323+0000 7f4bb494e700 1 -- 172.21.15.32:0/1619969659 learned_addr learned my addr 172.21.15.32:0/1619969659 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:46.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.323+0000 7f4baf7fe700 1 -- 172.21.15.32:0/1619969659 >> v1:172.21.15.61:6789/0 conn(0x7f4bb005f3b0 legacy=0x7f4bb00b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.323+0000 7f4baf7fe700 1 -- 172.21.15.32:0/1619969659 >> v1:172.21.15.32:6789/0 conn(0x7f4bb005feb0 legacy=0x7f4bb00b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.323+0000 7f4baeffd700 1 -- 172.21.15.32:0/1619969659 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 927770566 0 0) 0x7f4bb0061780 con 0x7f4bb005f3b0 2020-12-05T17:31:46.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.323+0000 7f4baf7fe700 1 -- 172.21.15.32:0/1619969659 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb0061cc0 con 0x7f4bb0062400 2020-12-05T17:31:46.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.323+0000 7f4baf7fe700 1 --2- 172.21.15.32:0/1619969659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bb0062400 0x7f4bb005f790 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f4ba80010d0 tx=0x7f4ba00091a0).ready entity=mon.0 client_cookie=9853043083266f4f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.323+0000 7f4baeffd700 1 -- 172.21.15.32:0/1619969659 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ba000b910 con 0x7f4bb0062400 2020-12-05T17:31:46.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.323+0000 7f4baeffd700 1 -- 172.21.15.32:0/1619969659 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba000ba70 con 0x7f4bb0062400 2020-12-05T17:31:46.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.323+0000 7f4baeffd700 1 -- 172.21.15.32:0/1619969659 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ba000bbe0 con 0x7f4bb0062400 2020-12-05T17:31:46.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.323+0000 7f4bb514f700 1 -- 172.21.15.32:0/1619969659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bb0062400 msgr2=0x7f4bb005f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.323+0000 7f4bb514f700 1 --2- 172.21.15.32:0/1619969659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bb0062400 0x7f4bb005f790 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f4ba80010d0 tx=0x7f4ba00091a0).stop 2020-12-05T17:31:46.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.323+0000 7f4bb514f700 1 -- 172.21.15.32:0/1619969659 shutdown_connections 2020-12-05T17:31:46.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.323+0000 7f4bb514f700 1 --2- 172.21.15.32:0/1619969659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bb0062400 0x7f4bb005f790 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.323+0000 7f4bb514f700 1 -- 172.21.15.32:0/1619969659 >> 172.21.15.32:0/1619969659 conn(0x7f4bb00b8ee0 msgr2=0x7f4bb00b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:46.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.323+0000 7f4bb514f700 1 -- 172.21.15.32:0/1619969659 shutdown_connections 2020-12-05T17:31:46.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.323+0000 7f4bb514f700 1 -- 172.21.15.32:0/1619969659 wait complete. 2020-12-05T17:31:46.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.327+0000 7f4bb514f700 1 Processor -- start 2020-12-05T17:31:46.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.327+0000 7f4bb514f700 1 -- start start 2020-12-05T17:31:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.327+0000 7f4bb514f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bb005f3b0 0x7f4bb0063f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.327+0000 7f4bb514f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4bb005feb0 0x7f4bb013e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.327+0000 7f4bb514f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4bb0062400 0x7f4bb0155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.327+0000 7f4bb514f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb0154cb0 con 0x7f4bb005f3b0 2020-12-05T17:31:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.327+0000 7f4bb514f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb0154f50 con 0x7f4bb005feb0 2020-12-05T17:31:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.327+0000 7f4bb514f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb01551f0 con 0x7f4bb0062400 2020-12-05T17:31:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.327+0000 7f4baffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bb005f3b0 0x7f4bb0063f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.331+0000 7f4baf7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4bb005feb0 0x7f4bb013e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.331+0000 7f4baf7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4bb005feb0 0x7f4bb013e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:52554/0 (socket says 172.21.15.32:52554) 2020-12-05T17:31:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.331+0000 7f4baf7fe700 1 -- 172.21.15.32:0/1546962168 learned_addr learned my addr 172.21.15.32:0/1546962168 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.331+0000 7f4baffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bb005f3b0 0x7f4bb0063f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60892/0 (socket says 172.21.15.32:60892) 2020-12-05T17:31:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.331+0000 7f4bb494e700 1 --2- 172.21.15.32:0/1546962168 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4bb0062400 0x7f4bb0155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.331+0000 7f4baffff700 1 -- 172.21.15.32:0/1546962168 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4bb0062400 msgr2=0x7f4bb0155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.331+0000 7f4baffff700 1 --2- 172.21.15.32:0/1546962168 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4bb0062400 0x7f4bb0155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.331+0000 7f4baffff700 1 -- 172.21.15.32:0/1546962168 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4bb005feb0 msgr2=0x7f4bb013e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.331+0000 7f4baffff700 1 --2- 172.21.15.32:0/1546962168 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4bb005feb0 0x7f4bb013e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.331+0000 7f4baffff700 1 -- 172.21.15.32:0/1546962168 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb00669d0 con 0x7f4bb005f3b0 2020-12-05T17:31:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.331+0000 7f4baf7fe700 1 --2- 172.21.15.32:0/1546962168 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4bb005feb0 0x7f4bb013e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:31:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.331+0000 7f4baffff700 1 --2- 172.21.15.32:0/1546962168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bb005f3b0 0x7f4bb0063f90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f4b98000b80 tx=0x7f4b98007980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.331+0000 7f4bad7fa700 1 -- 172.21.15.32:0/1546962168 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b9800f470 con 0x7f4bb005f3b0 2020-12-05T17:31:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.331+0000 7f4bad7fa700 1 -- 172.21.15.32:0/1546962168 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9800f5d0 con 0x7f4bb005f3b0 2020-12-05T17:31:46.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.331+0000 7f4bb514f700 1 -- 172.21.15.32:0/1546962168 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb0066c80 con 0x7f4bb005f3b0 2020-12-05T17:31:46.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.331+0000 7f4bad7fa700 1 -- 172.21.15.32:0/1546962168 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b9800f930 con 0x7f4bb005f3b0 2020-12-05T17:31:46.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.331+0000 7f4bb514f700 1 -- 172.21.15.32:0/1546962168 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb0151860 con 0x7f4bb005f3b0 2020-12-05T17:31:46.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.331+0000 7f4bb514f700 1 -- 172.21.15.32:0/1546962168 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4bb01553b0 con 0x7f4bb005f3b0 2020-12-05T17:31:46.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.331+0000 7f4bad7fa700 1 -- 172.21.15.32:0/1546962168 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4b98019030 con 0x7f4bb005f3b0 2020-12-05T17:31:46.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.335+0000 7f4bad7fa700 1 --2- 172.21.15.32:0/1546962168 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4b940554e0 0x7f4b940579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.335+0000 7f4bad7fa700 1 -- 172.21.15.32:0/1546962168 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f4b9806d1e0 con 0x7f4bb005f3b0 2020-12-05T17:31:46.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.335+0000 7f4baf7fe700 1 --2- 172.21.15.32:0/1546962168 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4b940554e0 0x7f4b940579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.335+0000 7f4bad7fa700 1 --2- 172.21.15.32:0/1546962168 >> [v2:172.21.15.32:6826/11535,v1:172.21.15.32:6827/11535] conn(0x7f4b9405cf60 0x7f4b9405f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.335+0000 7f4bad7fa700 1 -- 172.21.15.32:0/1546962168 --> [v2:172.21.15.32:6826/11535,v1:172.21.15.32:6827/11535] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b9405f940 con 0x7f4b9405cf60 2020-12-05T17:31:46.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.335+0000 7f4bad7fa700 1 -- 172.21.15.32:0/1546962168 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b9806f870 con 0x7f4bb005f3b0 2020-12-05T17:31:46.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.335+0000 7f4bb494e700 1 --2- 172.21.15.32:0/1546962168 >> [v2:172.21.15.32:6826/11535,v1:172.21.15.32:6827/11535] conn(0x7f4b9405cf60 0x7f4b9405f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.335+0000 7f4baf7fe700 1 --2- 172.21.15.32:0/1546962168 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4b940554e0 0x7f4b940579a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f4ba001c000 tx=0x7f4ba0016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.335+0000 7f4bb494e700 1 --2- 172.21.15.32:0/1546962168 >> [v2:172.21.15.32:6826/11535,v1:172.21.15.32:6827/11535] conn(0x7f4b9405cf60 0x7f4b9405f380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.335+0000 7f4bad7fa700 1 -- 172.21.15.32:0/1546962168 <== osd.3 v2:172.21.15.32:6826/11535 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f4b9405f940 con 0x7f4b9405cf60 2020-12-05T17:31:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.359+0000 7f4bb514f700 1 -- 172.21.15.32:0/1546962168 --> [v2:172.21.15.32:6826/11535,v1:172.21.15.32:6827/11535] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4bb0057050 con 0x7f4b9405cf60 2020-12-05T17:31:46.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.359+0000 7f4bad7fa700 1 -- 172.21.15.32:0/1546962168 <== osd.3 v2:172.21.15.32:6826/11535 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4bb0057050 con 0x7f4b9405cf60 2020-12-05T17:31:46.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.359+0000 7f4bb514f700 1 -- 172.21.15.32:0/1546962168 >> [v2:172.21.15.32:6826/11535,v1:172.21.15.32:6827/11535] conn(0x7f4b9405cf60 msgr2=0x7f4b9405f380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.359+0000 7f4bb514f700 1 --2- 172.21.15.32:0/1546962168 >> [v2:172.21.15.32:6826/11535,v1:172.21.15.32:6827/11535] conn(0x7f4b9405cf60 0x7f4b9405f380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.359+0000 7f4bb514f700 1 -- 172.21.15.32:0/1546962168 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4b940554e0 msgr2=0x7f4b940579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.359+0000 7f4bb514f700 1 --2- 172.21.15.32:0/1546962168 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4b940554e0 0x7f4b940579a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f4ba001c000 tx=0x7f4ba0016000).stop 2020-12-05T17:31:46.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.359+0000 7f4bb514f700 1 -- 172.21.15.32:0/1546962168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bb005f3b0 msgr2=0x7f4bb0063f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.359+0000 7f4bb514f700 1 --2- 172.21.15.32:0/1546962168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bb005f3b0 0x7f4bb0063f90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f4b98000b80 tx=0x7f4b98007980).stop 2020-12-05T17:31:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.359+0000 7f4bb494e700 1 -- 172.21.15.32:0/1546962168 reap_dead start 2020-12-05T17:31:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.359+0000 7f4bb514f700 1 -- 172.21.15.32:0/1546962168 shutdown_connections 2020-12-05T17:31:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.359+0000 7f4bb514f700 1 -- 172.21.15.32:0/1546962168 >> 172.21.15.32:0/1546962168 conn(0x7f4bb00b8ee0 msgr2=0x7f4bb013f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.359+0000 7f4bb514f700 1 -- 172.21.15.32:0/1546962168 shutdown_connections 2020-12-05T17:31:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.359+0000 7f4bb514f700 1 -- 172.21.15.32:0/1546962168 wait complete. 2020-12-05T17:31:46.377 INFO:teuthology.orchestra.run.smithi032.stdout:60129542149 2020-12-05T17:31:46.377 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.4 flush_pg_stats 2020-12-05T17:31:46.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.495+0000 7f04613e2700 1 Processor -- start 2020-12-05T17:31:46.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.495+0000 7f04613e2700 1 -- start start 2020-12-05T17:31:46.503 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f04613e2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f045c0b6080 0x7f045c1400b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.503 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f04613e2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f045c0ad250 0x7f045c1421c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.503 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f04613e2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f045c0b6460 con 0x7f045c1426b0 2020-12-05T17:31:46.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f04613e2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f045c1405a0 con 0x7f045c0ad250 2020-12-05T17:31:46.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f04613e2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f045c1407b0 con 0x7f045c0b6080 2020-12-05T17:31:46.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f045b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f045c1426b0 0x7f045c0af0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35320/0 (socket says 172.21.15.32:35320) 2020-12-05T17:31:46.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f045b7fe700 1 -- 172.21.15.32:0/2044549744 learned_addr learned my addr 172.21.15.32:0/2044549744 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:46.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f04527fc700 1 --2- 172.21.15.32:0/2044549744 >> v2:172.21.15.32:3300/0 conn(0x7f045c0ad250 0x7f045c1421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f045affd700 1 --2- 172.21.15.32:0/2044549744 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f045c0b6080 0x7f045c1400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f045a7fc700 1 -- 172.21.15.32:0/2044549744 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74592806 0 0) 0x7f045c0b6460 con 0x7f045c1426b0 2020-12-05T17:31:46.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f045a7fc700 1 -- 172.21.15.32:0/2044549744 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f043c003230 con 0x7f045c1426b0 2020-12-05T17:31:46.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f045affd700 1 -- 172.21.15.32:0/2044549744 >> v2:172.21.15.32:3300/0 conn(0x7f045c0ad250 msgr2=0x7f045c1421c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:46.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f045affd700 1 --2- 172.21.15.32:0/2044549744 >> v2:172.21.15.32:3300/0 conn(0x7f045c0ad250 0x7f045c1421c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f045affd700 1 -- 172.21.15.32:0/2044549744 >> v1:172.21.15.32:6789/0 conn(0x7f045c1426b0 legacy=0x7f045c0af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f045affd700 1 -- 172.21.15.32:0/2044549744 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045c0ae9b0 con 0x7f045c0b6080 2020-12-05T17:31:46.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f04527fc700 1 --2- 172.21.15.32:0/2044549744 >> v2:172.21.15.32:3300/0 conn(0x7f045c0ad250 0x7f045c1421c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:31:46.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f045affd700 1 --2- 172.21.15.32:0/2044549744 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f045c0b6080 0x7f045c1400b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f044c002f30 tx=0x7f044c006dc0).ready entity=mon.2 client_cookie=4df14907801f4937 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f045a7fc700 1 -- 172.21.15.32:0/2044549744 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f044c0096a0 con 0x7f045c0b6080 2020-12-05T17:31:46.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f045a7fc700 1 -- 172.21.15.32:0/2044549744 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044c009800 con 0x7f045c0b6080 2020-12-05T17:31:46.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f045a7fc700 1 -- 172.21.15.32:0/2044549744 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f044c009b60 con 0x7f045c0b6080 2020-12-05T17:31:46.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f04613e2700 1 -- 172.21.15.32:0/2044549744 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f045c0b6080 msgr2=0x7f045c1400b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f04613e2700 1 --2- 172.21.15.32:0/2044549744 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f045c0b6080 0x7f045c1400b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f044c002f30 tx=0x7f044c006dc0).stop 2020-12-05T17:31:46.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f04613e2700 1 -- 172.21.15.32:0/2044549744 shutdown_connections 2020-12-05T17:31:46.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f04613e2700 1 --2- 172.21.15.32:0/2044549744 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f045c0b6080 0x7f045c1400b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f04613e2700 1 --2- 172.21.15.32:0/2044549744 >> v2:172.21.15.32:3300/0 conn(0x7f045c0ad250 0x7f045c1421c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.499+0000 7f04613e2700 1 -- 172.21.15.32:0/2044549744 >> 172.21.15.32:0/2044549744 conn(0x7f045c069e50 msgr2=0x7f045c0604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:46.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.503+0000 7f04613e2700 1 -- 172.21.15.32:0/2044549744 shutdown_connections 2020-12-05T17:31:46.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.503+0000 7f04613e2700 1 -- 172.21.15.32:0/2044549744 wait complete. 2020-12-05T17:31:46.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.503+0000 7f04613e2700 1 Processor -- start 2020-12-05T17:31:46.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.503+0000 7f04613e2700 1 -- start start 2020-12-05T17:31:46.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.503+0000 7f04613e2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f045c0ad250 0x7f045c063b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.503+0000 7f04613e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f045c0b6080 0x7f045c1451c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.503+0000 7f04613e2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f045c1426b0 0x7f045c065610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.503+0000 7f04613e2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045c0641f0 con 0x7f045c0b6080 2020-12-05T17:31:46.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.503+0000 7f04613e2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045c0646c0 con 0x7f045c0ad250 2020-12-05T17:31:46.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.503+0000 7f04613e2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f045c064420 con 0x7f045c1426b0 2020-12-05T17:31:46.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.503+0000 7f04527fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f045c0b6080 0x7f045c1451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.503+0000 7f04527fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f045c0b6080 0x7f045c1451c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60910/0 (socket says 172.21.15.32:60910) 2020-12-05T17:31:46.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.503+0000 7f04527fc700 1 -- 172.21.15.32:0/3752156790 learned_addr learned my addr 172.21.15.32:0/3752156790 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:46.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.503+0000 7f045affd700 1 --2- 172.21.15.32:0/3752156790 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f045c0ad250 0x7f045c063b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.503+0000 7f045b7fe700 1 --2- 172.21.15.32:0/3752156790 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f045c1426b0 0x7f045c065610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.507+0000 7f04527fc700 1 -- 172.21.15.32:0/3752156790 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f045c1426b0 msgr2=0x7f045c065610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.507+0000 7f04527fc700 1 --2- 172.21.15.32:0/3752156790 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f045c1426b0 0x7f045c065610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.507+0000 7f04527fc700 1 -- 172.21.15.32:0/3752156790 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f045c0ad250 msgr2=0x7f045c063b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.507+0000 7f04527fc700 1 --2- 172.21.15.32:0/3752156790 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f045c0ad250 0x7f045c063b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.507+0000 7f04527fc700 1 -- 172.21.15.32:0/3752156790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045c065170 con 0x7f045c0b6080 2020-12-05T17:31:46.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.507+0000 7f04527fc700 1 --2- 172.21.15.32:0/3752156790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f045c0b6080 0x7f045c1451c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f044400c000 tx=0x7f0444006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.507+0000 7f0458ff9700 1 -- 172.21.15.32:0/3752156790 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f044400cb40 con 0x7f045c0b6080 2020-12-05T17:31:46.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.507+0000 7f0458ff9700 1 -- 172.21.15.32:0/3752156790 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044400cca0 con 0x7f045c0b6080 2020-12-05T17:31:46.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.507+0000 7f04613e2700 1 -- 172.21.15.32:0/3752156790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f045c155920 con 0x7f045c0b6080 2020-12-05T17:31:46.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.507+0000 7f0458ff9700 1 -- 172.21.15.32:0/3752156790 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f044400b570 con 0x7f045c0b6080 2020-12-05T17:31:46.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.507+0000 7f04613e2700 1 -- 172.21.15.32:0/3752156790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f045c151990 con 0x7f045c0b6080 2020-12-05T17:31:46.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.507+0000 7f04613e2700 1 -- 172.21.15.32:0/3752156790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f045c155c80 con 0x7f045c0b6080 2020-12-05T17:31:46.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.507+0000 7f0458ff9700 1 -- 172.21.15.32:0/3752156790 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f044401c460 con 0x7f045c0b6080 2020-12-05T17:31:46.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.507+0000 7f0458ff9700 1 --2- 172.21.15.32:0/3752156790 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0440055440 0x7f0440057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.507+0000 7f0458ff9700 1 -- 172.21.15.32:0/3752156790 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f044406e740 con 0x7f045c0b6080 2020-12-05T17:31:46.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.507+0000 7f045affd700 1 --2- 172.21.15.32:0/3752156790 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0440055440 0x7f0440057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.507+0000 7f0458ff9700 1 --2- 172.21.15.32:0/3752156790 >> [v2:172.21.15.61:6808/11541,v1:172.21.15.61:6809/11541] conn(0x7f044005cec0 0x7f044005f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.507+0000 7f0458ff9700 1 -- 172.21.15.32:0/3752156790 --> [v2:172.21.15.61:6808/11541,v1:172.21.15.61:6809/11541] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f044005f8a0 con 0x7f044005cec0 2020-12-05T17:31:46.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.507+0000 7f0458ff9700 1 -- 172.21.15.32:0/3752156790 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0444070cc0 con 0x7f045c0b6080 2020-12-05T17:31:46.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.511+0000 7f045b7fe700 1 --2- 172.21.15.32:0/3752156790 >> [v2:172.21.15.61:6808/11541,v1:172.21.15.61:6809/11541] conn(0x7f044005cec0 0x7f044005f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.511+0000 7f045affd700 1 --2- 172.21.15.32:0/3752156790 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0440055440 0x7f0440057900 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f044c01b000 tx=0x7f044c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.511+0000 7f045b7fe700 1 --2- 172.21.15.32:0/3752156790 >> [v2:172.21.15.61:6808/11541,v1:172.21.15.61:6809/11541] conn(0x7f044005cec0 0x7f044005f2e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.511+0000 7f0458ff9700 1 -- 172.21.15.32:0/3752156790 <== osd.4 v2:172.21.15.61:6808/11541 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f044005f8a0 con 0x7f044005cec0 2020-12-05T17:31:46.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.535+0000 7f04613e2700 1 -- 172.21.15.32:0/3752156790 --> [v2:172.21.15.61:6808/11541,v1:172.21.15.61:6809/11541] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f045c066160 con 0x7f044005cec0 2020-12-05T17:31:46.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.535+0000 7f0458ff9700 1 -- 172.21.15.32:0/3752156790 <== osd.4 v2:172.21.15.61:6808/11541 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f045c066160 con 0x7f044005cec0 2020-12-05T17:31:46.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.535+0000 7f04613e2700 1 -- 172.21.15.32:0/3752156790 >> [v2:172.21.15.61:6808/11541,v1:172.21.15.61:6809/11541] conn(0x7f044005cec0 msgr2=0x7f044005f2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.535+0000 7f04613e2700 1 --2- 172.21.15.32:0/3752156790 >> [v2:172.21.15.61:6808/11541,v1:172.21.15.61:6809/11541] conn(0x7f044005cec0 0x7f044005f2e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.535+0000 7f04613e2700 1 -- 172.21.15.32:0/3752156790 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0440055440 msgr2=0x7f0440057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.535+0000 7f04613e2700 1 --2- 172.21.15.32:0/3752156790 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0440055440 0x7f0440057900 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f044c01b000 tx=0x7f044c015000).stop 2020-12-05T17:31:46.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.535+0000 7f04613e2700 1 -- 172.21.15.32:0/3752156790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f045c0b6080 msgr2=0x7f045c1451c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.535+0000 7f04613e2700 1 --2- 172.21.15.32:0/3752156790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f045c0b6080 0x7f045c1451c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f044400c000 tx=0x7f0444006dc0).stop 2020-12-05T17:31:46.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.535+0000 7f045b7fe700 1 -- 172.21.15.32:0/3752156790 reap_dead start 2020-12-05T17:31:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.535+0000 7f04613e2700 1 -- 172.21.15.32:0/3752156790 shutdown_connections 2020-12-05T17:31:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.535+0000 7f04613e2700 1 -- 172.21.15.32:0/3752156790 >> 172.21.15.32:0/3752156790 conn(0x7f045c069e50 msgr2=0x7f045c0b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.535+0000 7f04613e2700 1 -- 172.21.15.32:0/3752156790 shutdown_connections 2020-12-05T17:31:46.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.535+0000 7f04613e2700 1 -- 172.21.15.32:0/3752156790 wait complete. 2020-12-05T17:31:46.554 INFO:teuthology.orchestra.run.smithi032.stdout:60129542150 2020-12-05T17:31:46.555 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.5 flush_pg_stats 2020-12-05T17:31:46.676 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.671+0000 7f0cea5d9700 1 Processor -- start 2020-12-05T17:31:46.676 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.671+0000 7f0cea5d9700 1 -- start start 2020-12-05T17:31:46.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.671+0000 7f0cea5d9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ce40abeb0 0x7f0ce40ab610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.671+0000 7f0cea5d9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ce4062660 0x7f0ce40ad190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.671+0000 7f0cea5d9700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0ce40ae6e0 0x7f0ce413dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.671+0000 7f0cea5d9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce40ac290 con 0x7f0ce40abeb0 2020-12-05T17:31:46.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.671+0000 7f0cea5d9700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce40b0930 con 0x7f0ce40ae6e0 2020-12-05T17:31:46.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.671+0000 7f0cea5d9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce40b0bd0 con 0x7f0ce4062660 2020-12-05T17:31:46.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.675+0000 7f0ce95d7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ce40abeb0 0x7f0ce40ab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.675+0000 7f0ce95d7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ce40abeb0 0x7f0ce40ab610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60918/0 (socket says 172.21.15.32:60918) 2020-12-05T17:31:46.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.675+0000 7f0ce95d7700 1 -- 172.21.15.32:0/3046597378 learned_addr learned my addr 172.21.15.32:0/3046597378 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:46.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.675+0000 7f0ce9dd8700 1 --2- 172.21.15.32:0/3046597378 >> v2:172.21.15.61:3300/0 conn(0x7f0ce40ae6e0 0x7f0ce413dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.675+0000 7f0ce8dd6700 1 --2- 172.21.15.32:0/3046597378 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ce4062660 0x7f0ce40ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.675+0000 7f0ce95d7700 1 -- 172.21.15.32:0/3046597378 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ce4062660 msgr2=0x7f0ce40ad190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:46.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.675+0000 7f0ce95d7700 1 --2- 172.21.15.32:0/3046597378 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ce4062660 0x7f0ce40ad190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.675+0000 7f0ce95d7700 1 -- 172.21.15.32:0/3046597378 >> v2:172.21.15.61:3300/0 conn(0x7f0ce40ae6e0 msgr2=0x7f0ce413dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:46.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.675+0000 7f0ce95d7700 1 --2- 172.21.15.32:0/3046597378 >> v2:172.21.15.61:3300/0 conn(0x7f0ce40ae6e0 0x7f0ce413dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.675+0000 7f0ce95d7700 1 -- 172.21.15.32:0/3046597378 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce40ae240 con 0x7f0ce40abeb0 2020-12-05T17:31:46.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.675+0000 7f0ce95d7700 1 --2- 172.21.15.32:0/3046597378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce40abeb0 0x7f0ce40ab610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f0ce4061230 tx=0x7f0cd4006e80).ready entity=mon.0 client_cookie=2bd0e71290a1f317 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.675+0000 7f0cd3fff700 1 -- 172.21.15.32:0/3046597378 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cd400be60 con 0x7f0ce40abeb0 2020-12-05T17:31:46.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.675+0000 7f0cd3fff700 1 -- 172.21.15.32:0/3046597378 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd40097b0 con 0x7f0ce40abeb0 2020-12-05T17:31:46.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.675+0000 7f0cd3fff700 1 -- 172.21.15.32:0/3046597378 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cd4009b10 con 0x7f0ce40abeb0 2020-12-05T17:31:46.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.675+0000 7f0cea5d9700 1 -- 172.21.15.32:0/3046597378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce40abeb0 msgr2=0x7f0ce40ab610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.675+0000 7f0cea5d9700 1 --2- 172.21.15.32:0/3046597378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce40abeb0 0x7f0ce40ab610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f0ce4061230 tx=0x7f0cd4006e80).stop 2020-12-05T17:31:46.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.675+0000 7f0cea5d9700 1 -- 172.21.15.32:0/3046597378 shutdown_connections 2020-12-05T17:31:46.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.675+0000 7f0cea5d9700 1 --2- 172.21.15.32:0/3046597378 >> v2:172.21.15.61:3300/0 conn(0x7f0ce40ae6e0 0x7f0ce413dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.675+0000 7f0cea5d9700 1 --2- 172.21.15.32:0/3046597378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce40abeb0 0x7f0ce40ab610 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.675+0000 7f0cea5d9700 1 --2- 172.21.15.32:0/3046597378 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ce4062660 0x7f0ce40ad190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.675+0000 7f0cea5d9700 1 -- 172.21.15.32:0/3046597378 >> 172.21.15.32:0/3046597378 conn(0x7f0ce40b5530 msgr2=0x7f0ce40b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:46.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.675+0000 7f0cea5d9700 1 -- 172.21.15.32:0/3046597378 shutdown_connections 2020-12-05T17:31:46.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.675+0000 7f0cea5d9700 1 -- 172.21.15.32:0/3046597378 wait complete. 2020-12-05T17:31:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.679+0000 7f0cea5d9700 1 Processor -- start 2020-12-05T17:31:46.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.679+0000 7f0cea5d9700 1 -- start start 2020-12-05T17:31:46.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.679+0000 7f0cea5d9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ce4062660 0x7f0ce413c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.679+0000 7f0cea5d9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ce40abeb0 0x7f0ce4063dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.679+0000 7f0cea5d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce40ae6e0 0x7f0ce40659c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.679+0000 7f0cea5d9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce40642c0 con 0x7f0ce40ae6e0 2020-12-05T17:31:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.679+0000 7f0cea5d9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce41515d0 con 0x7f0ce40abeb0 2020-12-05T17:31:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.679+0000 7f0cea5d9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce4151870 con 0x7f0ce4062660 2020-12-05T17:31:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.683+0000 7f0ce9dd8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce40ae6e0 0x7f0ce40659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.683+0000 7f0ce95d7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ce4062660 0x7f0ce413c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.683+0000 7f0ce9dd8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce40ae6e0 0x7f0ce40659c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60928/0 (socket says 172.21.15.32:60928) 2020-12-05T17:31:46.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.683+0000 7f0ce9dd8700 1 -- 172.21.15.32:0/269459667 learned_addr learned my addr 172.21.15.32:0/269459667 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:46.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.683+0000 7f0ce8dd6700 1 --2- 172.21.15.32:0/269459667 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ce40abeb0 0x7f0ce4063dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.683+0000 7f0ce9dd8700 1 -- 172.21.15.32:0/269459667 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ce4062660 msgr2=0x7f0ce413c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.683+0000 7f0ce9dd8700 1 --2- 172.21.15.32:0/269459667 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ce4062660 0x7f0ce413c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.683+0000 7f0ce9dd8700 1 -- 172.21.15.32:0/269459667 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ce40abeb0 msgr2=0x7f0ce4063dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.683+0000 7f0ce9dd8700 1 --2- 172.21.15.32:0/269459667 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ce40abeb0 0x7f0ce4063dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.683+0000 7f0ce9dd8700 1 -- 172.21.15.32:0/269459667 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce4151b10 con 0x7f0ce40ae6e0 2020-12-05T17:31:46.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.683+0000 7f0ce9dd8700 1 --2- 172.21.15.32:0/269459667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce40ae6e0 0x7f0ce40659c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f0cdc006dd0 tx=0x7f0cdc0091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.707 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.683+0000 7f0cd27fc700 1 -- 172.21.15.32:0/269459667 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cdc00c950 con 0x7f0ce40ae6e0 2020-12-05T17:31:46.707 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.683+0000 7f0cd27fc700 1 -- 172.21.15.32:0/269459667 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc00cab0 con 0x7f0ce40ae6e0 2020-12-05T17:31:46.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.683+0000 7f0cea5d9700 1 -- 172.21.15.32:0/269459667 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce4065eb0 con 0x7f0ce40ae6e0 2020-12-05T17:31:46.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.683+0000 7f0cd27fc700 1 -- 172.21.15.32:0/269459667 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cdc00ce10 con 0x7f0ce40ae6e0 2020-12-05T17:31:46.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.683+0000 7f0cea5d9700 1 -- 172.21.15.32:0/269459667 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce4064e40 con 0x7f0ce40ae6e0 2020-12-05T17:31:46.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.683+0000 7f0cea5d9700 1 -- 172.21.15.32:0/269459667 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0ce4151ca0 con 0x7f0ce40ae6e0 2020-12-05T17:31:46.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.683+0000 7f0cd27fc700 1 -- 172.21.15.32:0/269459667 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0cdc022070 con 0x7f0ce40ae6e0 2020-12-05T17:31:46.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.683+0000 7f0cd27fc700 1 --2- 172.21.15.32:0/269459667 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0cc40554e0 0x7f0cc40579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.683+0000 7f0ce95d7700 1 --2- 172.21.15.32:0/269459667 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0cc40554e0 0x7f0cc40579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.683+0000 7f0cd27fc700 1 -- 172.21.15.32:0/269459667 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0cdc06e6a0 con 0x7f0ce40ae6e0 2020-12-05T17:31:46.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.687+0000 7f0cd27fc700 1 --2- 172.21.15.32:0/269459667 >> [v2:172.21.15.61:6800/11539,v1:172.21.15.61:6801/11539] conn(0x7f0cc405cf60 0x7f0cc405f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.687+0000 7f0cd27fc700 1 -- 172.21.15.32:0/269459667 --> [v2:172.21.15.61:6800/11539,v1:172.21.15.61:6801/11539] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0cc405f940 con 0x7f0cc405cf60 2020-12-05T17:31:46.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.687+0000 7f0cd27fc700 1 -- 172.21.15.32:0/269459667 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cdc070d30 con 0x7f0ce40ae6e0 2020-12-05T17:31:46.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.687+0000 7f0ce95d7700 1 --2- 172.21.15.32:0/269459667 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0cc40554e0 0x7f0cc40579a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f0cd401c000 tx=0x7f0cd4015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.687+0000 7f0ce8dd6700 1 --2- 172.21.15.32:0/269459667 >> [v2:172.21.15.61:6800/11539,v1:172.21.15.61:6801/11539] conn(0x7f0cc405cf60 0x7f0cc405f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.687+0000 7f0ce8dd6700 1 --2- 172.21.15.32:0/269459667 >> [v2:172.21.15.61:6800/11539,v1:172.21.15.61:6801/11539] conn(0x7f0cc405cf60 0x7f0cc405f380 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.687+0000 7f0cd27fc700 1 -- 172.21.15.32:0/269459667 <== osd.5 v2:172.21.15.61:6800/11539 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f0cc405f940 con 0x7f0cc405cf60 2020-12-05T17:31:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.711+0000 7f0cea5d9700 1 -- 172.21.15.32:0/269459667 --> [v2:172.21.15.61:6800/11539,v1:172.21.15.61:6801/11539] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0ce4152530 con 0x7f0cc405cf60 2020-12-05T17:31:46.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.711+0000 7f0cd27fc700 1 -- 172.21.15.32:0/269459667 <== osd.5 v2:172.21.15.61:6800/11539 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0ce4152530 con 0x7f0cc405cf60 2020-12-05T17:31:46.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.711+0000 7f0cea5d9700 1 -- 172.21.15.32:0/269459667 >> [v2:172.21.15.61:6800/11539,v1:172.21.15.61:6801/11539] conn(0x7f0cc405cf60 msgr2=0x7f0cc405f380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.711+0000 7f0cea5d9700 1 --2- 172.21.15.32:0/269459667 >> [v2:172.21.15.61:6800/11539,v1:172.21.15.61:6801/11539] conn(0x7f0cc405cf60 0x7f0cc405f380 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.711+0000 7f0cea5d9700 1 -- 172.21.15.32:0/269459667 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0cc40554e0 msgr2=0x7f0cc40579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.711+0000 7f0cea5d9700 1 --2- 172.21.15.32:0/269459667 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0cc40554e0 0x7f0cc40579a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f0cd401c000 tx=0x7f0cd4015000).stop 2020-12-05T17:31:46.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.711+0000 7f0cea5d9700 1 -- 172.21.15.32:0/269459667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce40ae6e0 msgr2=0x7f0ce40659c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.711+0000 7f0cea5d9700 1 --2- 172.21.15.32:0/269459667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce40ae6e0 0x7f0ce40659c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f0cdc006dd0 tx=0x7f0cdc0091a0).stop 2020-12-05T17:31:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.711+0000 7f0ce9dd8700 1 -- 172.21.15.32:0/269459667 reap_dead start 2020-12-05T17:31:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.711+0000 7f0cea5d9700 1 -- 172.21.15.32:0/269459667 shutdown_connections 2020-12-05T17:31:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.711+0000 7f0cea5d9700 1 -- 172.21.15.32:0/269459667 >> 172.21.15.32:0/269459667 conn(0x7f0ce40b5530 msgr2=0x7f0ce40b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:46.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.711+0000 7f0cea5d9700 1 -- 172.21.15.32:0/269459667 shutdown_connections 2020-12-05T17:31:46.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.711+0000 7f0cea5d9700 1 -- 172.21.15.32:0/269459667 wait complete. 2020-12-05T17:31:46.730 INFO:teuthology.orchestra.run.smithi032.stdout:60129542149 2020-12-05T17:31:46.730 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.6 flush_pg_stats 2020-12-05T17:31:46.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.839+0000 7f1b9dbe6700 1 Processor -- start 2020-12-05T17:31:46.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.839+0000 7f1b9dbe6700 1 -- start start 2020-12-05T17:31:46.845 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b9dbe6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b9805f3b0 0x7f1b9805f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b9dbe6700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1b98062400 0x7f1b980b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b9dbe6700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b98142450 con 0x7f1b9805feb0 2020-12-05T17:31:46.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b9dbe6700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b980b42f0 con 0x7f1b98062400 2020-12-05T17:31:46.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b9dbe6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b98061780 con 0x7f1b9805f3b0 2020-12-05T17:31:46.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b97fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1b98062400 0x7f1b980b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b9cbe4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b9805f3b0 0x7f1b9805f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b97fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1b98062400 0x7f1b980b36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:52596/0 (socket says 172.21.15.32:52596) 2020-12-05T17:31:46.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b9d3e5700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f1b9805feb0 0x7f1b980b3be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:47994/0 (socket says 172.21.15.32:47994) 2020-12-05T17:31:46.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b97fff700 1 -- 172.21.15.32:0/3296474236 learned_addr learned my addr 172.21.15.32:0/3296474236 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:46.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b97fff700 1 -- 172.21.15.32:0/3296474236 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b9805f3b0 msgr2=0x7f1b9805f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:46.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b97fff700 1 --2- 172.21.15.32:0/3296474236 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b9805f3b0 0x7f1b9805f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b97fff700 1 -- 172.21.15.32:0/3296474236 >> v1:172.21.15.61:6789/0 conn(0x7f1b9805feb0 legacy=0x7f1b980b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b97fff700 1 -- 172.21.15.32:0/3296474236 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b98061a20 con 0x7f1b98062400 2020-12-05T17:31:46.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b97fff700 1 --2- 172.21.15.32:0/3296474236 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b98062400 0x7f1b980b36f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f1b80002f30 tx=0x7f1b80006e80).ready entity=mon.1 client_cookie=18bcbcaa3b1a23d0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b977fe700 1 -- 172.21.15.32:0/3296474236 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b800097f0 con 0x7f1b98062400 2020-12-05T17:31:46.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b977fe700 1 -- 172.21.15.32:0/3296474236 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b80009950 con 0x7f1b98062400 2020-12-05T17:31:46.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b977fe700 1 -- 172.21.15.32:0/3296474236 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b80009cb0 con 0x7f1b98062400 2020-12-05T17:31:46.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b9dbe6700 1 -- 172.21.15.32:0/3296474236 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b98062400 msgr2=0x7f1b980b36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b9dbe6700 1 --2- 172.21.15.32:0/3296474236 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b98062400 0x7f1b980b36f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f1b80002f30 tx=0x7f1b80006e80).stop 2020-12-05T17:31:46.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b9dbe6700 1 -- 172.21.15.32:0/3296474236 shutdown_connections 2020-12-05T17:31:46.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b9dbe6700 1 --2- 172.21.15.32:0/3296474236 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b9805f3b0 0x7f1b9805f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b9dbe6700 1 --2- 172.21.15.32:0/3296474236 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b98062400 0x7f1b980b36f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b9dbe6700 1 -- 172.21.15.32:0/3296474236 >> 172.21.15.32:0/3296474236 conn(0x7f1b980b8ee0 msgr2=0x7f1b980b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:46.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b9dbe6700 1 -- 172.21.15.32:0/3296474236 shutdown_connections 2020-12-05T17:31:46.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.843+0000 7f1b9dbe6700 1 -- 172.21.15.32:0/3296474236 wait complete. 2020-12-05T17:31:46.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b9dbe6700 1 Processor -- start 2020-12-05T17:31:46.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b9dbe6700 1 -- start start 2020-12-05T17:31:46.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b9dbe6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b9805f3b0 0x7f1b98063380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b9dbe6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b9805feb0 0x7f1b981459c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b9dbe6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b98062400 0x7f1b98145eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b9dbe6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b98144fd0 con 0x7f1b98062400 2020-12-05T17:31:46.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b9dbe6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b981463a0 con 0x7f1b9805feb0 2020-12-05T17:31:46.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b9dbe6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b98065680 con 0x7f1b9805f3b0 2020-12-05T17:31:46.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b9d3e5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b98062400 0x7f1b98145eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b9d3e5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b98062400 0x7f1b98145eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60944/0 (socket says 172.21.15.32:60944) 2020-12-05T17:31:46.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b9d3e5700 1 -- 172.21.15.32:0/1956719986 learned_addr learned my addr 172.21.15.32:0/1956719986 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:46.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b97fff700 1 --2- 172.21.15.32:0/1956719986 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b9805feb0 0x7f1b981459c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b9cbe4700 1 --2- 172.21.15.32:0/1956719986 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b9805f3b0 0x7f1b98063380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b9d3e5700 1 -- 172.21.15.32:0/1956719986 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b9805f3b0 msgr2=0x7f1b98063380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b9d3e5700 1 --2- 172.21.15.32:0/1956719986 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b9805f3b0 0x7f1b98063380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b9d3e5700 1 -- 172.21.15.32:0/1956719986 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b9805feb0 msgr2=0x7f1b981459c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b9d3e5700 1 --2- 172.21.15.32:0/1956719986 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b9805feb0 0x7f1b981459c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b9d3e5700 1 -- 172.21.15.32:0/1956719986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b980653e0 con 0x7f1b98062400 2020-12-05T17:31:46.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b9d3e5700 1 --2- 172.21.15.32:0/1956719986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b98062400 0x7f1b98145eb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f1b9805c5c0 tx=0x7f1b90007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b95ffb700 1 -- 172.21.15.32:0/1956719986 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b90005ab0 con 0x7f1b98062400 2020-12-05T17:31:46.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b9dbe6700 1 -- 172.21.15.32:0/1956719986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b98145520 con 0x7f1b98062400 2020-12-05T17:31:46.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b9dbe6700 1 -- 172.21.15.32:0/1956719986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b98144b10 con 0x7f1b98062400 2020-12-05T17:31:46.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b95ffb700 1 -- 172.21.15.32:0/1956719986 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b90005c10 con 0x7f1b98062400 2020-12-05T17:31:46.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.847+0000 7f1b95ffb700 1 -- 172.21.15.32:0/1956719986 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b90007110 con 0x7f1b98062400 2020-12-05T17:31:46.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.851+0000 7f1b9dbe6700 1 -- 172.21.15.32:0/1956719986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1b98057050 con 0x7f1b98062400 2020-12-05T17:31:46.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.851+0000 7f1b95ffb700 1 -- 172.21.15.32:0/1956719986 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1b90010030 con 0x7f1b98062400 2020-12-05T17:31:46.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.851+0000 7f1b95ffb700 1 --2- 172.21.15.32:0/1956719986 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1b7c0554e0 0x7f1b7c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.851+0000 7f1b95ffb700 1 -- 172.21.15.32:0/1956719986 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f1b9006e0a0 con 0x7f1b98062400 2020-12-05T17:31:46.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.851+0000 7f1b9cbe4700 1 --2- 172.21.15.32:0/1956719986 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1b7c0554e0 0x7f1b7c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.851+0000 7f1b95ffb700 1 --2- 172.21.15.32:0/1956719986 >> [v2:172.21.15.61:6820/11542,v1:172.21.15.61:6822/11542] conn(0x7f1b7c05cf60 0x7f1b7c05f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:46.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.851+0000 7f1b95ffb700 1 -- 172.21.15.32:0/1956719986 --> [v2:172.21.15.61:6820/11542,v1:172.21.15.61:6822/11542] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b7c05f940 con 0x7f1b7c05cf60 2020-12-05T17:31:46.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.851+0000 7f1b95ffb700 1 -- 172.21.15.32:0/1956719986 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b90070730 con 0x7f1b98062400 2020-12-05T17:31:46.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.851+0000 7f1b97fff700 1 --2- 172.21.15.32:0/1956719986 >> [v2:172.21.15.61:6820/11542,v1:172.21.15.61:6822/11542] conn(0x7f1b7c05cf60 0x7f1b7c05f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:46.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.851+0000 7f1b9cbe4700 1 --2- 172.21.15.32:0/1956719986 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1b7c0554e0 0x7f1b7c0579a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f1b8800b000 tx=0x7f1b88008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.855+0000 7f1b97fff700 1 --2- 172.21.15.32:0/1956719986 >> [v2:172.21.15.61:6820/11542,v1:172.21.15.61:6822/11542] conn(0x7f1b7c05cf60 0x7f1b7c05f380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:46.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.855+0000 7f1b95ffb700 1 -- 172.21.15.32:0/1956719986 <== osd.6 v2:172.21.15.61:6820/11542 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f1b7c05f940 con 0x7f1b7c05cf60 2020-12-05T17:31:46.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.875+0000 7f1b9dbe6700 1 -- 172.21.15.32:0/1956719986 --> [v2:172.21.15.61:6820/11542,v1:172.21.15.61:6822/11542] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1b980b34c0 con 0x7f1b7c05cf60 2020-12-05T17:31:46.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.879+0000 7f1b95ffb700 1 -- 172.21.15.32:0/1956719986 <== osd.6 v2:172.21.15.61:6820/11542 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1b980b34c0 con 0x7f1b7c05cf60 2020-12-05T17:31:46.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.879+0000 7f1b9dbe6700 1 -- 172.21.15.32:0/1956719986 >> [v2:172.21.15.61:6820/11542,v1:172.21.15.61:6822/11542] conn(0x7f1b7c05cf60 msgr2=0x7f1b7c05f380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.879+0000 7f1b9dbe6700 1 --2- 172.21.15.32:0/1956719986 >> [v2:172.21.15.61:6820/11542,v1:172.21.15.61:6822/11542] conn(0x7f1b7c05cf60 0x7f1b7c05f380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:46.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.879+0000 7f1b9dbe6700 1 -- 172.21.15.32:0/1956719986 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1b7c0554e0 msgr2=0x7f1b7c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.879+0000 7f1b9dbe6700 1 --2- 172.21.15.32:0/1956719986 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1b7c0554e0 0x7f1b7c0579a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f1b8800b000 tx=0x7f1b88008040).stop 2020-12-05T17:31:46.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.879+0000 7f1b9dbe6700 1 -- 172.21.15.32:0/1956719986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b98062400 msgr2=0x7f1b98145eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:46.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.879+0000 7f1b9dbe6700 1 --2- 172.21.15.32:0/1956719986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b98062400 0x7f1b98145eb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f1b9805c5c0 tx=0x7f1b90007f10).stop 2020-12-05T17:31:46.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.879+0000 7f1b9d3e5700 1 -- 172.21.15.32:0/1956719986 reap_dead start 2020-12-05T17:31:46.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.879+0000 7f1b9dbe6700 1 -- 172.21.15.32:0/1956719986 shutdown_connections 2020-12-05T17:31:46.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.879+0000 7f1b9dbe6700 1 -- 172.21.15.32:0/1956719986 >> 172.21.15.32:0/1956719986 conn(0x7f1b980b8ee0 msgr2=0x7f1b98142620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:46.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.879+0000 7f1b9dbe6700 1 -- 172.21.15.32:0/1956719986 shutdown_connections 2020-12-05T17:31:46.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:46.879+0000 7f1b9dbe6700 1 -- 172.21.15.32:0/1956719986 wait complete. 2020-12-05T17:31:46.895 INFO:teuthology.orchestra.run.smithi032.stdout:60129542150 2020-12-05T17:31:46.895 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.7 flush_pg_stats 2020-12-05T17:31:47.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.007+0000 7f17d57f9700 1 Processor -- start 2020-12-05T17:31:47.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.007+0000 7f17d57f9700 1 -- start start 2020-12-05T17:31:47.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17d57f9700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f17d00b0940 0x7f17d0069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17d57f9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17d00b25a0 con 0x7f17d00b21c0 2020-12-05T17:31:47.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17d57f9700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17d00b0e50 con 0x7f17d013f2c0 2020-12-05T17:31:47.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17d57f9700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17d00acce0 con 0x7f17d00b0940 2020-12-05T17:31:47.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17ceffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f17d00b21c0 0x7f17d013ebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35368/0 (socket says 172.21.15.32:35368) 2020-12-05T17:31:47.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17ceffd700 1 -- 172.21.15.32:0/1711336582 learned_addr learned my addr 172.21.15.32:0/1711336582 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:47.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17c67fc700 1 --2- 172.21.15.32:0/1711336582 >> v2:172.21.15.61:3300/0 conn(0x7f17d00b0940 0x7f17d0069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17ce7fc700 1 -- 172.21.15.32:0/1711336582 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 982922781 0 0) 0x7f17d00b25a0 con 0x7f17d00b21c0 2020-12-05T17:31:47.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17ce7fc700 1 -- 172.21.15.32:0/1711336582 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17b0003230 con 0x7f17d00b21c0 2020-12-05T17:31:47.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17c67fc700 1 -- 172.21.15.32:0/1711336582 >> v1:172.21.15.61:6789/0 conn(0x7f17d013f2c0 legacy=0x7f17d0142cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17ce7fc700 1 -- 172.21.15.32:0/1711336582 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3937347973 0 0) 0x7f17d00b0e50 con 0x7f17d013f2c0 2020-12-05T17:31:47.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17c67fc700 1 -- 172.21.15.32:0/1711336582 >> v1:172.21.15.32:6789/0 conn(0x7f17d00b21c0 legacy=0x7f17d013ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17c67fc700 1 -- 172.21.15.32:0/1711336582 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d00ab510 con 0x7f17d00b0940 2020-12-05T17:31:47.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17c67fc700 1 --2- 172.21.15.32:0/1711336582 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17d00b0940 0x7f17d0069cb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f17b8002f30 tx=0x7f17b8006dc0).ready entity=mon.1 client_cookie=3f7214d037a17a26 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:47.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17ce7fc700 1 -- 172.21.15.32:0/1711336582 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17b8009800 con 0x7f17d00b0940 2020-12-05T17:31:47.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17ce7fc700 1 -- 172.21.15.32:0/1711336582 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b8009960 con 0x7f17d00b0940 2020-12-05T17:31:47.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17ce7fc700 1 -- 172.21.15.32:0/1711336582 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17b8009cc0 con 0x7f17d00b0940 2020-12-05T17:31:47.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17d57f9700 1 -- 172.21.15.32:0/1711336582 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17d00b0940 msgr2=0x7f17d0069cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17d57f9700 1 --2- 172.21.15.32:0/1711336582 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17d00b0940 0x7f17d0069cb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f17b8002f30 tx=0x7f17b8006dc0).stop 2020-12-05T17:31:47.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17d57f9700 1 -- 172.21.15.32:0/1711336582 shutdown_connections 2020-12-05T17:31:47.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17d57f9700 1 --2- 172.21.15.32:0/1711336582 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17d00b0940 0x7f17d0069cb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17d57f9700 1 -- 172.21.15.32:0/1711336582 >> 172.21.15.32:0/1711336582 conn(0x7f17d00b57a0 msgr2=0x7f17d00b1770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:47.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17d57f9700 1 -- 172.21.15.32:0/1711336582 shutdown_connections 2020-12-05T17:31:47.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.011+0000 7f17d57f9700 1 -- 172.21.15.32:0/1711336582 wait complete. 2020-12-05T17:31:47.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.015+0000 7f17d57f9700 1 Processor -- start 2020-12-05T17:31:47.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.015+0000 7f17d57f9700 1 -- start start 2020-12-05T17:31:47.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.015+0000 7f17d57f9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17d00b0940 0x7f17d00b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.015+0000 7f17d57f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17d00b21c0 0x7f17d0140bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.015+0000 7f17d57f9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f17d013f2c0 0x7f17d01410c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.015+0000 7f17d57f9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d013da90 con 0x7f17d00b21c0 2020-12-05T17:31:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.015+0000 7f17d57f9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d013d0b0 con 0x7f17d00b0940 2020-12-05T17:31:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.015+0000 7f17d57f9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17d013d350 con 0x7f17d013f2c0 2020-12-05T17:31:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.015+0000 7f17c67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17d00b21c0 0x7f17d0140bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.015+0000 7f17c67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17d00b21c0 0x7f17d0140bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60956/0 (socket says 172.21.15.32:60956) 2020-12-05T17:31:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.015+0000 7f17c67fc700 1 -- 172.21.15.32:0/3968082468 learned_addr learned my addr 172.21.15.32:0/3968082468 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:47.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.015+0000 7f17ceffd700 1 --2- 172.21.15.32:0/3968082468 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17d00b0940 0x7f17d00b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.015+0000 7f17cf7fe700 1 --2- 172.21.15.32:0/3968082468 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f17d013f2c0 0x7f17d01410c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.015+0000 7f17c67fc700 1 -- 172.21.15.32:0/3968082468 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f17d013f2c0 msgr2=0x7f17d01410c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.015+0000 7f17c67fc700 1 --2- 172.21.15.32:0/3968082468 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f17d013f2c0 0x7f17d01410c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.015+0000 7f17c67fc700 1 -- 172.21.15.32:0/3968082468 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17d00b0940 msgr2=0x7f17d00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.015+0000 7f17c67fc700 1 --2- 172.21.15.32:0/3968082468 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17d00b0940 0x7f17d00b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.015+0000 7f17c67fc700 1 -- 172.21.15.32:0/3968082468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d013d5f0 con 0x7f17d00b21c0 2020-12-05T17:31:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.015+0000 7f17c67fc700 1 --2- 172.21.15.32:0/3968082468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17d00b21c0 0x7f17d0140bd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f17b8002f30 tx=0x7f17b8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.015+0000 7f17ccff9700 1 -- 172.21.15.32:0/3968082468 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17b8009c60 con 0x7f17d00b21c0 2020-12-05T17:31:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.019+0000 7f17ccff9700 1 -- 172.21.15.32:0/3968082468 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b8011b80 con 0x7f17d00b21c0 2020-12-05T17:31:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.019+0000 7f17d57f9700 1 -- 172.21.15.32:0/3968082468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d013b720 con 0x7f17d00b21c0 2020-12-05T17:31:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.019+0000 7f17ccff9700 1 -- 172.21.15.32:0/3968082468 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17b801d440 con 0x7f17d00b21c0 2020-12-05T17:31:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.019+0000 7f17d57f9700 1 -- 172.21.15.32:0/3968082468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d013ce30 con 0x7f17d00b21c0 2020-12-05T17:31:47.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.019+0000 7f17d57f9700 1 -- 172.21.15.32:0/3968082468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f17d013d780 con 0x7f17d00b21c0 2020-12-05T17:31:47.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.019+0000 7f17ccff9700 1 -- 172.21.15.32:0/3968082468 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f17b8024030 con 0x7f17d00b21c0 2020-12-05T17:31:47.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.019+0000 7f17ccff9700 1 --2- 172.21.15.32:0/3968082468 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f17b40553b0 0x7f17b4057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.019+0000 7f17ceffd700 1 --2- 172.21.15.32:0/3968082468 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f17b40553b0 0x7f17b4057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.019+0000 7f17ccff9700 1 -- 172.21.15.32:0/3968082468 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f17b806ed40 con 0x7f17d00b21c0 2020-12-05T17:31:47.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.023+0000 7f17ccff9700 1 --2- 172.21.15.32:0/3968082468 >> [v2:172.21.15.61:6813/11540,v1:172.21.15.61:6815/11540] conn(0x7f17b405ce30 0x7f17b405f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.023+0000 7f17ccff9700 1 -- 172.21.15.32:0/3968082468 --> [v2:172.21.15.61:6813/11540,v1:172.21.15.61:6815/11540] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f17b405f8b0 con 0x7f17b405ce30 2020-12-05T17:31:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.023+0000 7f17ccff9700 1 -- 172.21.15.32:0/3968082468 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17b80713d0 con 0x7f17d00b21c0 2020-12-05T17:31:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.023+0000 7f17ceffd700 1 --2- 172.21.15.32:0/3968082468 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f17b40553b0 0x7f17b4057870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f17c000d000 tx=0x7f17c0009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.023+0000 7f17cf7fe700 1 --2- 172.21.15.32:0/3968082468 >> [v2:172.21.15.61:6813/11540,v1:172.21.15.61:6815/11540] conn(0x7f17b405ce30 0x7f17b405f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.023+0000 7f17cf7fe700 1 --2- 172.21.15.32:0/3968082468 >> [v2:172.21.15.61:6813/11540,v1:172.21.15.61:6815/11540] conn(0x7f17b405ce30 0x7f17b405f250 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.023+0000 7f17ccff9700 1 -- 172.21.15.32:0/3968082468 <== osd.7 v2:172.21.15.61:6813/11540 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f17b405f8b0 con 0x7f17b405ce30 2020-12-05T17:31:47.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.047+0000 7f17d57f9700 1 -- 172.21.15.32:0/3968082468 --> [v2:172.21.15.61:6813/11540,v1:172.21.15.61:6815/11540] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f17d01418d0 con 0x7f17b405ce30 2020-12-05T17:31:47.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.047+0000 7f17ccff9700 1 -- 172.21.15.32:0/3968082468 <== osd.7 v2:172.21.15.61:6813/11540 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f17d01418d0 con 0x7f17b405ce30 2020-12-05T17:31:47.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.047+0000 7f17d57f9700 1 -- 172.21.15.32:0/3968082468 >> [v2:172.21.15.61:6813/11540,v1:172.21.15.61:6815/11540] conn(0x7f17b405ce30 msgr2=0x7f17b405f250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.047+0000 7f17d57f9700 1 --2- 172.21.15.32:0/3968082468 >> [v2:172.21.15.61:6813/11540,v1:172.21.15.61:6815/11540] conn(0x7f17b405ce30 0x7f17b405f250 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.047+0000 7f17d57f9700 1 -- 172.21.15.32:0/3968082468 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f17b40553b0 msgr2=0x7f17b4057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.047+0000 7f17d57f9700 1 --2- 172.21.15.32:0/3968082468 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f17b40553b0 0x7f17b4057870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f17c000d000 tx=0x7f17c0009040).stop 2020-12-05T17:31:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.047+0000 7f17d57f9700 1 -- 172.21.15.32:0/3968082468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17d00b21c0 msgr2=0x7f17d0140bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.047+0000 7f17d57f9700 1 --2- 172.21.15.32:0/3968082468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17d00b21c0 0x7f17d0140bd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f17b8002f30 tx=0x7f17b8015000).stop 2020-12-05T17:31:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.047+0000 7f17cf7fe700 1 -- 172.21.15.32:0/3968082468 reap_dead start 2020-12-05T17:31:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.047+0000 7f17d57f9700 1 -- 172.21.15.32:0/3968082468 shutdown_connections 2020-12-05T17:31:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.047+0000 7f17d57f9700 1 -- 172.21.15.32:0/3968082468 >> 172.21.15.32:0/3968082468 conn(0x7f17d00b57a0 msgr2=0x7f17d00ac110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.047+0000 7f17d57f9700 1 -- 172.21.15.32:0/3968082468 shutdown_connections 2020-12-05T17:31:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.047+0000 7f17d57f9700 1 -- 172.21.15.32:0/3968082468 wait complete. 2020-12-05T17:31:47.064 INFO:teuthology.orchestra.run.smithi032.stdout:60129542149 2020-12-05T17:31:47.065 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.0 2020-12-05T17:31:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.183+0000 7f907628b700 1 Processor -- start 2020-12-05T17:31:47.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.183+0000 7f907628b700 1 -- start start 2020-12-05T17:31:47.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.183+0000 7f907628b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f907005f6f0 0x7f90700b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.183+0000 7f907628b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f90700af920 0x7f9070063040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.183+0000 7f907628b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90700afea0 0x7f907013f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.183+0000 7f907628b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90700b2cd0 con 0x7f907005f6f0 2020-12-05T17:31:47.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.183+0000 7f907628b700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90700b3560 con 0x7f90700af920 2020-12-05T17:31:47.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.183+0000 7f907628b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907005fad0 con 0x7f90700afea0 2020-12-05T17:31:47.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.183+0000 7f9075289700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f907005f6f0 0x7f90700b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.183+0000 7f9075289700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f907005f6f0 0x7f90700b1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60966/0 (socket says 172.21.15.32:60966) 2020-12-05T17:31:47.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.183+0000 7f9075289700 1 -- 172.21.15.32:0/2545082905 learned_addr learned my addr 172.21.15.32:0/2545082905 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:47.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.183+0000 7f9075a8a700 1 --2- 172.21.15.32:0/2545082905 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90700afea0 0x7f907013f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.183+0000 7f9074a88700 1 --2- 172.21.15.32:0/2545082905 >> v2:172.21.15.61:3300/0 conn(0x7f90700af920 0x7f9070063040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.187+0000 7f9075289700 1 -- 172.21.15.32:0/2545082905 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90700afea0 msgr2=0x7f907013f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:47.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.187+0000 7f9075289700 1 --2- 172.21.15.32:0/2545082905 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90700afea0 0x7f907013f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.187+0000 7f9075289700 1 -- 172.21.15.32:0/2545082905 >> v2:172.21.15.61:3300/0 conn(0x7f90700af920 msgr2=0x7f9070063040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:47.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.187+0000 7f9075289700 1 --2- 172.21.15.32:0/2545082905 >> v2:172.21.15.61:3300/0 conn(0x7f90700af920 0x7f9070063040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.187+0000 7f9075289700 1 -- 172.21.15.32:0/2545082905 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9070144650 con 0x7f907005f6f0 2020-12-05T17:31:47.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.187+0000 7f9075289700 1 --2- 172.21.15.32:0/2545082905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f907005f6f0 0x7f90700b1d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f906c002f30 tx=0x7f906c006e80).ready entity=mon.0 client_cookie=d4b65b2acd60c61c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:47.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.187+0000 7f9067fff700 1 -- 172.21.15.32:0/2545082905 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f906c009640 con 0x7f907005f6f0 2020-12-05T17:31:47.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.187+0000 7f9067fff700 1 -- 172.21.15.32:0/2545082905 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906c0097a0 con 0x7f907005f6f0 2020-12-05T17:31:47.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.187+0000 7f9067fff700 1 -- 172.21.15.32:0/2545082905 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f906c009b00 con 0x7f907005f6f0 2020-12-05T17:31:47.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.187+0000 7f907628b700 1 -- 172.21.15.32:0/2545082905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f907005f6f0 msgr2=0x7f90700b1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.187+0000 7f907628b700 1 --2- 172.21.15.32:0/2545082905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f907005f6f0 0x7f90700b1d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f906c002f30 tx=0x7f906c006e80).stop 2020-12-05T17:31:47.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.187+0000 7f907628b700 1 -- 172.21.15.32:0/2545082905 shutdown_connections 2020-12-05T17:31:47.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.187+0000 7f907628b700 1 --2- 172.21.15.32:0/2545082905 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90700afea0 0x7f907013f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.187+0000 7f907628b700 1 --2- 172.21.15.32:0/2545082905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f907005f6f0 0x7f90700b1d90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.187+0000 7f907628b700 1 --2- 172.21.15.32:0/2545082905 >> v2:172.21.15.61:3300/0 conn(0x7f90700af920 0x7f9070063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.187+0000 7f907628b700 1 -- 172.21.15.32:0/2545082905 >> 172.21.15.32:0/2545082905 conn(0x7f90700604c0 msgr2=0x7f907005c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:47.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.187+0000 7f907628b700 1 -- 172.21.15.32:0/2545082905 shutdown_connections 2020-12-05T17:31:47.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.187+0000 7f907628b700 1 -- 172.21.15.32:0/2545082905 wait complete. 2020-12-05T17:31:47.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.187+0000 7f907628b700 1 Processor -- start 2020-12-05T17:31:47.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.187+0000 7f907628b700 1 -- start start 2020-12-05T17:31:47.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f907628b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f907005f6f0 0x7f9070147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f907628b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f90700af920 0x7f90701528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f907628b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90700afea0 0x7f90701110f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f907628b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9070110110 con 0x7f90700afea0 2020-12-05T17:31:47.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f907628b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9070065580 con 0x7f90700af920 2020-12-05T17:31:47.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f907628b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9070065820 con 0x7f907005f6f0 2020-12-05T17:31:47.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f9075a8a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90700afea0 0x7f90701110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f9075a8a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90700afea0 0x7f90701110f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60976/0 (socket says 172.21.15.32:60976) 2020-12-05T17:31:47.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f9075a8a700 1 -- 172.21.15.32:0/4057762534 learned_addr learned my addr 172.21.15.32:0/4057762534 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:47.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f9074a88700 1 --2- 172.21.15.32:0/4057762534 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f90700af920 0x7f90701528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f9075289700 1 --2- 172.21.15.32:0/4057762534 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f907005f6f0 0x7f9070147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f9075a8a700 1 -- 172.21.15.32:0/4057762534 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f907005f6f0 msgr2=0x7f9070147430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f9075a8a700 1 --2- 172.21.15.32:0/4057762534 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f907005f6f0 0x7f9070147430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f9075a8a700 1 -- 172.21.15.32:0/4057762534 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f90700af920 msgr2=0x7f90701528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f9075a8a700 1 --2- 172.21.15.32:0/4057762534 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f90700af920 0x7f90701528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f9075a8a700 1 -- 172.21.15.32:0/4057762534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9070110c50 con 0x7f90700afea0 2020-12-05T17:31:47.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f9074a88700 1 --2- 172.21.15.32:0/4057762534 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f90700af920 0x7f90701528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:47.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f9075a8a700 1 --2- 172.21.15.32:0/4057762534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90700afea0 0x7f90701110f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f9068006dd0 tx=0x7f90680091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:47.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f90667fc700 1 -- 172.21.15.32:0/4057762534 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f906800ca00 con 0x7f90700afea0 2020-12-05T17:31:47.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f90667fc700 1 -- 172.21.15.32:0/4057762534 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906800cb60 con 0x7f90700afea0 2020-12-05T17:31:47.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f907628b700 1 -- 172.21.15.32:0/4057762534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90700665b0 con 0x7f90700afea0 2020-12-05T17:31:47.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f90667fc700 1 -- 172.21.15.32:0/4057762534 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90680123f0 con 0x7f90700afea0 2020-12-05T17:31:47.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f907628b700 1 -- 172.21.15.32:0/4057762534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90700652f0 con 0x7f90700afea0 2020-12-05T17:31:47.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.191+0000 7f907628b700 1 -- 172.21.15.32:0/4057762534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9070057050 con 0x7f90700afea0 2020-12-05T17:31:47.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.195+0000 7f90667fc700 1 -- 172.21.15.32:0/4057762534 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9068022070 con 0x7f90700afea0 2020-12-05T17:31:47.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.195+0000 7f90667fc700 1 --2- 172.21.15.32:0/4057762534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f90580554e0 0x7f90580579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.195+0000 7f90667fc700 1 -- 172.21.15.32:0/4057762534 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f906806e790 con 0x7f90700afea0 2020-12-05T17:31:47.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.195+0000 7f9075289700 1 --2- 172.21.15.32:0/4057762534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f90580554e0 0x7f90580579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.195+0000 7f9075289700 1 --2- 172.21.15.32:0/4057762534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f90580554e0 0x7f90580579a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f906c019000 tx=0x7f906c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:47.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.203+0000 7f90667fc700 1 -- 172.21.15.32:0/4057762534 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9068044930 con 0x7f90700afea0 2020-12-05T17:31:47.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.347+0000 7f907628b700 1 -- 172.21.15.32:0/4057762534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9070110de0 con 0x7f90700afea0 2020-12-05T17:31:47.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.347+0000 7f90667fc700 1 -- 172.21.15.32:0/4057762534 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9068016070 con 0x7f90700afea0 2020-12-05T17:31:47.352 INFO:teuthology.orchestra.run.smithi032.stdout:60129542150 2020-12-05T17:31:47.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.351+0000 7f907628b700 1 -- 172.21.15.32:0/4057762534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f90580554e0 msgr2=0x7f90580579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.351+0000 7f907628b700 1 --2- 172.21.15.32:0/4057762534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f90580554e0 0x7f90580579a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f906c019000 tx=0x7f906c015000).stop 2020-12-05T17:31:47.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.351+0000 7f907628b700 1 -- 172.21.15.32:0/4057762534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90700afea0 msgr2=0x7f90701110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.351+0000 7f907628b700 1 --2- 172.21.15.32:0/4057762534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90700afea0 0x7f90701110f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f9068006dd0 tx=0x7f90680091a0).stop 2020-12-05T17:31:47.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.351+0000 7f907628b700 1 -- 172.21.15.32:0/4057762534 shutdown_connections 2020-12-05T17:31:47.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.351+0000 7f907628b700 1 --2- 172.21.15.32:0/4057762534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90700afea0 0x7f90701110f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.351+0000 7f907628b700 1 --2- 172.21.15.32:0/4057762534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f90580554e0 0x7f90580579a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.351+0000 7f907628b700 1 --2- 172.21.15.32:0/4057762534 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f907005f6f0 0x7f9070147430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.351+0000 7f907628b700 1 --2- 172.21.15.32:0/4057762534 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f90700af920 0x7f90701528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.351+0000 7f907628b700 1 -- 172.21.15.32:0/4057762534 >> 172.21.15.32:0/4057762534 conn(0x7f90700604c0 msgr2=0x7f90700b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:47.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.351+0000 7f907628b700 1 -- 172.21.15.32:0/4057762534 shutdown_connections 2020-12-05T17:31:47.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.351+0000 7f907628b700 1 -- 172.21.15.32:0/4057762534 wait complete. 2020-12-05T17:31:47.371 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.0 2020-12-05T17:31:47.371 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.1 2020-12-05T17:31:47.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.475+0000 7f9a3f59e700 1 Processor -- start 2020-12-05T17:31:47.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.475+0000 7f9a3f59e700 1 -- start start 2020-12-05T17:31:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.475+0000 7f9a3f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a4005f3b0 0x7f9a4005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.475+0000 7f9a3f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a40062400 0x7f9a400b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.475+0000 7f9a3f59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a40142450 con 0x7f9a4005feb0 2020-12-05T17:31:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.475+0000 7f9a3f59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a400b42f0 con 0x7f9a4005f3b0 2020-12-05T17:31:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.475+0000 7f9a3f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a40061780 con 0x7f9a40062400 2020-12-05T17:31:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.475+0000 7f9a3e59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a4005f3b0 0x7f9a4005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.475+0000 7f9a3e59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a4005f3b0 0x7f9a4005f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60982/0 (socket says 172.21.15.32:60982) 2020-12-05T17:31:47.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.475+0000 7f9a3dd9b700 1 --2- 172.21.15.32:0/1059176419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a40062400 0x7f9a400b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.475+0000 7f9a3e59c700 1 -- 172.21.15.32:0/1059176419 learned_addr learned my addr 172.21.15.32:0/1059176419 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.479+0000 7f9a3e59c700 1 -- 172.21.15.32:0/1059176419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a40062400 msgr2=0x7f9a400b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.479+0000 7f9a3e59c700 1 --2- 172.21.15.32:0/1059176419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a40062400 0x7f9a400b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.479+0000 7f9a3e59c700 1 -- 172.21.15.32:0/1059176419 >> v1:172.21.15.61:6789/0 conn(0x7f9a4005feb0 legacy=0x7f9a400b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.479+0000 7f9a3e59c700 1 -- 172.21.15.32:0/1059176419 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a40061a20 con 0x7f9a4005f3b0 2020-12-05T17:31:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.479+0000 7f9a3e59c700 1 --2- 172.21.15.32:0/1059176419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a4005f3b0 0x7f9a4005f790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f9a34002f30 tx=0x7f9a34006e80).ready entity=mon.0 client_cookie=e4ec655724348d1a server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.479+0000 7f9a3d59a700 1 -- 172.21.15.32:0/1059176419 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a340097f0 con 0x7f9a4005f3b0 2020-12-05T17:31:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.479+0000 7f9a3d59a700 1 -- 172.21.15.32:0/1059176419 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a34009950 con 0x7f9a4005f3b0 2020-12-05T17:31:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.479+0000 7f9a3d59a700 1 -- 172.21.15.32:0/1059176419 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a34009cb0 con 0x7f9a4005f3b0 2020-12-05T17:31:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.479+0000 7f9a3f59e700 1 -- 172.21.15.32:0/1059176419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a4005f3b0 msgr2=0x7f9a4005f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.479+0000 7f9a3f59e700 1 --2- 172.21.15.32:0/1059176419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a4005f3b0 0x7f9a4005f790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f9a34002f30 tx=0x7f9a34006e80).stop 2020-12-05T17:31:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.479+0000 7f9a3f59e700 1 -- 172.21.15.32:0/1059176419 shutdown_connections 2020-12-05T17:31:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.479+0000 7f9a3f59e700 1 --2- 172.21.15.32:0/1059176419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a4005f3b0 0x7f9a4005f790 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.479+0000 7f9a3f59e700 1 --2- 172.21.15.32:0/1059176419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a40062400 0x7f9a400b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.479+0000 7f9a3f59e700 1 -- 172.21.15.32:0/1059176419 >> 172.21.15.32:0/1059176419 conn(0x7f9a400b8ee0 msgr2=0x7f9a400b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.479+0000 7f9a3f59e700 1 -- 172.21.15.32:0/1059176419 shutdown_connections 2020-12-05T17:31:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.479+0000 7f9a3f59e700 1 -- 172.21.15.32:0/1059176419 wait complete. 2020-12-05T17:31:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.479+0000 7f9a3f59e700 1 Processor -- start 2020-12-05T17:31:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.479+0000 7f9a3f59e700 1 -- start start 2020-12-05T17:31:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.483+0000 7f9a3f59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a4005f3b0 0x7f9a4013ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.483+0000 7f9a3f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a4005feb0 0x7f9a400b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.483+0000 7f9a3f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a40062400 0x7f9a40146320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.483+0000 7f9a3f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a40145ed0 con 0x7f9a40062400 2020-12-05T17:31:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.483+0000 7f9a3f59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a40145b90 con 0x7f9a4005f3b0 2020-12-05T17:31:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.483+0000 7f9a3f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a401457e0 con 0x7f9a4005feb0 2020-12-05T17:31:47.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.483+0000 7f9a3ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a40062400 0x7f9a40146320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.483+0000 7f9a3e59c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a4005f3b0 0x7f9a4013ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.483+0000 7f9a3e59c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a4005f3b0 0x7f9a4013ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:52646/0 (socket says 172.21.15.32:52646) 2020-12-05T17:31:47.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.483+0000 7f9a3e59c700 1 -- 172.21.15.32:0/1779546417 learned_addr learned my addr 172.21.15.32:0/1779546417 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:47.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.483+0000 7f9a3dd9b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a4005feb0 0x7f9a400b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.483+0000 7f9a3ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a40062400 0x7f9a40146320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60990/0 (socket says 172.21.15.32:60990) 2020-12-05T17:31:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.483+0000 7f9a3ed9d700 1 -- 172.21.15.32:0/1779546417 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a4005feb0 msgr2=0x7f9a400b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.483+0000 7f9a3ed9d700 1 --2- 172.21.15.32:0/1779546417 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a4005feb0 0x7f9a400b1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.483+0000 7f9a3ed9d700 1 -- 172.21.15.32:0/1779546417 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a4005f3b0 msgr2=0x7f9a4013ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.483+0000 7f9a3ed9d700 1 --2- 172.21.15.32:0/1779546417 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a4005f3b0 0x7f9a4013ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.483+0000 7f9a3ed9d700 1 -- 172.21.15.32:0/1779546417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a400b3000 con 0x7f9a40062400 2020-12-05T17:31:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.483+0000 7f9a3ed9d700 1 --2- 172.21.15.32:0/1779546417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a40062400 0x7f9a40146320 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f9a4005c5c0 tx=0x7f9a38007b00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.483+0000 7f9a2f7fe700 1 -- 172.21.15.32:0/1779546417 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a38013410 con 0x7f9a40062400 2020-12-05T17:31:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.483+0000 7f9a2f7fe700 1 -- 172.21.15.32:0/1779546417 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a38013570 con 0x7f9a40062400 2020-12-05T17:31:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.483+0000 7f9a2f7fe700 1 -- 172.21.15.32:0/1779546417 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a380138d0 con 0x7f9a40062400 2020-12-05T17:31:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.483+0000 7f9a3f59e700 1 -- 172.21.15.32:0/1779546417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a400b1180 con 0x7f9a40062400 2020-12-05T17:31:47.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.483+0000 7f9a3f59e700 1 -- 172.21.15.32:0/1779546417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a400b22c0 con 0x7f9a40062400 2020-12-05T17:31:47.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.487+0000 7f9a3f59e700 1 -- 172.21.15.32:0/1779546417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a400b31f0 con 0x7f9a40062400 2020-12-05T17:31:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.487+0000 7f9a2f7fe700 1 -- 172.21.15.32:0/1779546417 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9a38010030 con 0x7f9a40062400 2020-12-05T17:31:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.487+0000 7f9a2f7fe700 1 --2- 172.21.15.32:0/1779546417 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9a1c0553b0 0x7f9a1c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.487+0000 7f9a2f7fe700 1 -- 172.21.15.32:0/1779546417 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f9a3806de40 con 0x7f9a40062400 2020-12-05T17:31:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.487+0000 7f9a3e59c700 1 --2- 172.21.15.32:0/1779546417 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9a1c0553b0 0x7f9a1c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.487+0000 7f9a3e59c700 1 --2- 172.21.15.32:0/1779546417 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9a1c0553b0 0x7f9a1c057870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f9a3401c000 tx=0x7f9a34015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.491+0000 7f9a2f7fe700 1 -- 172.21.15.32:0/1779546417 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9a400b31f0 con 0x7f9a40062400 2020-12-05T17:31:47.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.631+0000 7f9a3f59e700 1 -- 172.21.15.32:0/1779546417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9a400b31f0 con 0x7f9a40062400 2020-12-05T17:31:47.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.635+0000 7f9a2f7fe700 1 -- 172.21.15.32:0/1779546417 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9a400b31f0 con 0x7f9a40062400 2020-12-05T17:31:47.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.635+0000 7f9a3f59e700 1 -- 172.21.15.32:0/1779546417 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9a1c0553b0 msgr2=0x7f9a1c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.635+0000 7f9a3f59e700 1 --2- 172.21.15.32:0/1779546417 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9a1c0553b0 0x7f9a1c057870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f9a3401c000 tx=0x7f9a34015000).stop 2020-12-05T17:31:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.635+0000 7f9a3f59e700 1 -- 172.21.15.32:0/1779546417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a40062400 msgr2=0x7f9a40146320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.635+0000 7f9a3f59e700 1 --2- 172.21.15.32:0/1779546417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a40062400 0x7f9a40146320 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f9a4005c5c0 tx=0x7f9a38007b00).stop 2020-12-05T17:31:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.635+0000 7f9a3f59e700 1 -- 172.21.15.32:0/1779546417 shutdown_connections 2020-12-05T17:31:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.635+0000 7f9a3f59e700 1 --2- 172.21.15.32:0/1779546417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a40062400 0x7f9a40146320 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.635+0000 7f9a3f59e700 1 --2- 172.21.15.32:0/1779546417 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a4005f3b0 0x7f9a4013ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.635+0000 7f9a3f59e700 1 --2- 172.21.15.32:0/1779546417 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9a1c0553b0 0x7f9a1c057870 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.635+0000 7f9a3f59e700 1 --2- 172.21.15.32:0/1779546417 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a4005feb0 0x7f9a400b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.635+0000 7f9a3f59e700 1 -- 172.21.15.32:0/1779546417 >> 172.21.15.32:0/1779546417 conn(0x7f9a400b8ee0 msgr2=0x7f9a40140f90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:47.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.635+0000 7f9a3f59e700 1 -- 172.21.15.32:0/1779546417 shutdown_connections 2020-12-05T17:31:47.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.635+0000 7f9a3f59e700 1 -- 172.21.15.32:0/1779546417 wait complete. 2020-12-05T17:31:47.642 INFO:teuthology.orchestra.run.smithi032.stdout:60129542149 2020-12-05T17:31:47.651 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.1 2020-12-05T17:31:47.651 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.2 2020-12-05T17:31:47.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.759+0000 7f936bfff700 1 Processor -- start 2020-12-05T17:31:47.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.759+0000 7f936bfff700 1 -- start start 2020-12-05T17:31:47.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f936bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f936c0b14e0 0x7f936c0ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f936bfff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f936c0af480 0x7f936c142ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f936bfff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936c142ab0 con 0x7f936c0b5f00 2020-12-05T17:31:47.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f936bfff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f936c0b62e0 con 0x7f936c0b14e0 2020-12-05T17:31:47.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f936bfff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f936c0af880 con 0x7f936c0af480 2020-12-05T17:31:47.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f936affd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f936c0b5f00 0x7f936c066370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35410/0 (socket says 172.21.15.32:35410) 2020-12-05T17:31:47.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f936affd700 1 -- 172.21.15.32:0/2262513067 learned_addr learned my addr 172.21.15.32:0/2262513067 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:47.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f936a7fc700 1 --2- 172.21.15.32:0/2262513067 >> v2:172.21.15.32:3300/0 conn(0x7f936c0b14e0 0x7f936c0ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f936b7fe700 1 --2- 172.21.15.32:0/2262513067 >> v2:172.21.15.61:3300/0 conn(0x7f936c0af480 0x7f936c142ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f9369ffb700 1 -- 172.21.15.32:0/2262513067 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 69575278 0 0) 0x7f936c142ab0 con 0x7f936c0b5f00 2020-12-05T17:31:47.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f9369ffb700 1 -- 172.21.15.32:0/2262513067 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f934c003230 con 0x7f936c0b5f00 2020-12-05T17:31:47.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f936a7fc700 1 -- 172.21.15.32:0/2262513067 >> v2:172.21.15.61:3300/0 conn(0x7f936c0af480 msgr2=0x7f936c142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:47.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f936a7fc700 1 --2- 172.21.15.32:0/2262513067 >> v2:172.21.15.61:3300/0 conn(0x7f936c0af480 0x7f936c142ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f936a7fc700 1 -- 172.21.15.32:0/2262513067 >> v1:172.21.15.32:6789/0 conn(0x7f936c0b5f00 legacy=0x7f936c066370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f936a7fc700 1 -- 172.21.15.32:0/2262513067 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c0afbf0 con 0x7f936c0b14e0 2020-12-05T17:31:47.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f936a7fc700 1 --2- 172.21.15.32:0/2262513067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f936c0b14e0 0x7f936c0ad510 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f9354002f30 tx=0x7f9354006dc0).ready entity=mon.0 client_cookie=6502e3ba2d5fb19f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:47.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f9369ffb700 1 -- 172.21.15.32:0/2262513067 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93540096f0 con 0x7f936c0b14e0 2020-12-05T17:31:47.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f9369ffb700 1 -- 172.21.15.32:0/2262513067 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9354009850 con 0x7f936c0b14e0 2020-12-05T17:31:47.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f9369ffb700 1 -- 172.21.15.32:0/2262513067 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9354009bb0 con 0x7f936c0b14e0 2020-12-05T17:31:47.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f936bfff700 1 -- 172.21.15.32:0/2262513067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f936c0b14e0 msgr2=0x7f936c0ad510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f936bfff700 1 --2- 172.21.15.32:0/2262513067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f936c0b14e0 0x7f936c0ad510 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f9354002f30 tx=0x7f9354006dc0).stop 2020-12-05T17:31:47.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f936bfff700 1 -- 172.21.15.32:0/2262513067 shutdown_connections 2020-12-05T17:31:47.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f936bfff700 1 --2- 172.21.15.32:0/2262513067 >> v2:172.21.15.61:3300/0 conn(0x7f936c0af480 0x7f936c142ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f936bfff700 1 --2- 172.21.15.32:0/2262513067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f936c0b14e0 0x7f936c0ad510 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f936bfff700 1 -- 172.21.15.32:0/2262513067 >> 172.21.15.32:0/2262513067 conn(0x7f936c0604c0 msgr2=0x7f936c05c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:47.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f936bfff700 1 -- 172.21.15.32:0/2262513067 shutdown_connections 2020-12-05T17:31:47.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.763+0000 7f936bfff700 1 -- 172.21.15.32:0/2262513067 wait complete. 2020-12-05T17:31:47.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f936bfff700 1 Processor -- start 2020-12-05T17:31:47.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f936bfff700 1 -- start start 2020-12-05T17:31:47.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f936bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f936c0af480 0x7f936c064130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f936bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f936c0b14e0 0x7f936c063120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f936bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f936c0b5f00 0x7f936c065810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f936bfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c142540 con 0x7f936c0b5f00 2020-12-05T17:31:47.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f936bfff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c063610 con 0x7f936c0af480 2020-12-05T17:31:47.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f936bfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936c133100 con 0x7f936c0b14e0 2020-12-05T17:31:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f936b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f936c0b5f00 0x7f936c065810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f936b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f936c0b5f00 0x7f936c065810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32772/0 (socket says 172.21.15.32:32772) 2020-12-05T17:31:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f936b7fe700 1 -- 172.21.15.32:0/965932044 learned_addr learned my addr 172.21.15.32:0/965932044 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f936affd700 1 --2- 172.21.15.32:0/965932044 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f936c0af480 0x7f936c064130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f936a7fc700 1 --2- 172.21.15.32:0/965932044 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f936c0b14e0 0x7f936c063120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f936b7fe700 1 -- 172.21.15.32:0/965932044 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f936c0b14e0 msgr2=0x7f936c063120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f936b7fe700 1 --2- 172.21.15.32:0/965932044 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f936c0b14e0 0x7f936c063120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f936b7fe700 1 -- 172.21.15.32:0/965932044 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f936c0af480 msgr2=0x7f936c064130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f936b7fe700 1 --2- 172.21.15.32:0/965932044 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f936c0af480 0x7f936c064130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f936b7fe700 1 -- 172.21.15.32:0/965932044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c142730 con 0x7f936c0b5f00 2020-12-05T17:31:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f936b7fe700 1 --2- 172.21.15.32:0/965932044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f936c0b5f00 0x7f936c065810 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f9364006dd0 tx=0x7f93640091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f9363fff700 1 -- 172.21.15.32:0/965932044 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f936400eee0 con 0x7f936c0b5f00 2020-12-05T17:31:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f9363fff700 1 -- 172.21.15.32:0/965932044 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936400cbb0 con 0x7f936c0b5f00 2020-12-05T17:31:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f9363fff700 1 -- 172.21.15.32:0/965932044 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93640123f0 con 0x7f936c0b5f00 2020-12-05T17:31:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f936bfff700 1 -- 172.21.15.32:0/965932044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f936c133330 con 0x7f936c0b5f00 2020-12-05T17:31:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.767+0000 7f936bfff700 1 -- 172.21.15.32:0/965932044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936c065d00 con 0x7f936c0b5f00 2020-12-05T17:31:47.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.771+0000 7f9363fff700 1 -- 172.21.15.32:0/965932044 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9364022070 con 0x7f936c0b5f00 2020-12-05T17:31:47.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.771+0000 7f936bfff700 1 -- 172.21.15.32:0/965932044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f936c141f30 con 0x7f936c0b5f00 2020-12-05T17:31:47.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.771+0000 7f9363fff700 1 --2- 172.21.15.32:0/965932044 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9350055470 0x7f9350057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:47.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.771+0000 7f936affd700 1 --2- 172.21.15.32:0/965932044 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9350055470 0x7f9350057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:47.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.771+0000 7f9363fff700 1 -- 172.21.15.32:0/965932044 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f936406e8f0 con 0x7f936c0b5f00 2020-12-05T17:31:47.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.771+0000 7f936affd700 1 --2- 172.21.15.32:0/965932044 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9350055470 0x7f9350057930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f935c00b000 tx=0x7f935c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:47.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.775+0000 7f9363fff700 1 -- 172.21.15.32:0/965932044 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9364044980 con 0x7f936c0b5f00 2020-12-05T17:31:47.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.915+0000 7f936bfff700 1 -- 172.21.15.32:0/965932044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f936c1334c0 con 0x7f936c0b5f00 2020-12-05T17:31:47.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.915+0000 7f9363fff700 1 -- 172.21.15.32:0/965932044 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9364016070 con 0x7f936c0b5f00 2020-12-05T17:31:47.921 INFO:teuthology.orchestra.run.smithi032.stdout:60129542149 2020-12-05T17:31:47.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.919+0000 7f936bfff700 1 -- 172.21.15.32:0/965932044 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9350055470 msgr2=0x7f9350057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.919+0000 7f936bfff700 1 --2- 172.21.15.32:0/965932044 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9350055470 0x7f9350057930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f935c00b000 tx=0x7f935c009040).stop 2020-12-05T17:31:47.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.919+0000 7f936bfff700 1 -- 172.21.15.32:0/965932044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f936c0b5f00 msgr2=0x7f936c065810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:47.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.919+0000 7f936bfff700 1 --2- 172.21.15.32:0/965932044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f936c0b5f00 0x7f936c065810 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f9364006dd0 tx=0x7f93640091a0).stop 2020-12-05T17:31:47.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.919+0000 7f936bfff700 1 -- 172.21.15.32:0/965932044 shutdown_connections 2020-12-05T17:31:47.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.919+0000 7f936bfff700 1 --2- 172.21.15.32:0/965932044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f936c0b5f00 0x7f936c065810 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.919+0000 7f936bfff700 1 --2- 172.21.15.32:0/965932044 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f936c0af480 0x7f936c064130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.919+0000 7f936bfff700 1 --2- 172.21.15.32:0/965932044 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9350055470 0x7f9350057930 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.919+0000 7f936bfff700 1 --2- 172.21.15.32:0/965932044 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f936c0b14e0 0x7f936c063120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:47.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.919+0000 7f936bfff700 1 -- 172.21.15.32:0/965932044 >> 172.21.15.32:0/965932044 conn(0x7f936c0604c0 msgr2=0x7f936c0ae9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:47.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.919+0000 7f936bfff700 1 -- 172.21.15.32:0/965932044 shutdown_connections 2020-12-05T17:31:47.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:47.919+0000 7f936bfff700 1 -- 172.21.15.32:0/965932044 wait complete. 2020-12-05T17:31:47.934 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.2 2020-12-05T17:31:47.935 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.3 2020-12-05T17:31:48.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.047+0000 7f5fcb20a700 1 Processor -- start 2020-12-05T17:31:48.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.047+0000 7f5fcb20a700 1 -- start start 2020-12-05T17:31:48.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.047+0000 7f5fcb20a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5fc40b14e0 0x7f5fc40ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.047+0000 7f5fcb20a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc4142ab0 con 0x7f5fc40af480 2020-12-05T17:31:48.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.047+0000 7f5fcb20a700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc40b62e0 con 0x7f5fc40b5f00 2020-12-05T17:31:48.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.047+0000 7f5fcb20a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc40af880 con 0x7f5fc40b14e0 2020-12-05T17:31:48.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.047+0000 7f5fc9a07700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5fc40b14e0 0x7f5fc40ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.047+0000 7f5fcaa09700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5fc40af480 0x7f5fc4142ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35428/0 (socket says 172.21.15.32:35428) 2020-12-05T17:31:48.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.047+0000 7f5fcaa09700 1 -- 172.21.15.32:0/1570398870 learned_addr learned my addr 172.21.15.32:0/1570398870 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:48.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.047+0000 7f5fc9a07700 1 -- 172.21.15.32:0/1570398870 >> v1:172.21.15.61:6789/0 conn(0x7f5fc40b5f00 legacy=0x7f5fc4066370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:48.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.047+0000 7f5fc9a07700 1 -- 172.21.15.32:0/1570398870 >> v1:172.21.15.32:6789/0 conn(0x7f5fc40af480 legacy=0x7f5fc4142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:48.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.047+0000 7f5fc9206700 1 -- 172.21.15.32:0/1570398870 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 885155239 0 0) 0x7f5fc40b62e0 con 0x7f5fc40b5f00 2020-12-05T17:31:48.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.047+0000 7f5fc9a07700 1 -- 172.21.15.32:0/1570398870 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc40afbf0 con 0x7f5fc40b14e0 2020-12-05T17:31:48.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.047+0000 7f5fc9a07700 1 --2- 172.21.15.32:0/1570398870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fc40b14e0 0x7f5fc40ad510 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f5fbc001450 tx=0x7f5fac006dc0).ready entity=mon.0 client_cookie=11dea0e696b0ca1c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:48.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.047+0000 7f5fc9206700 1 -- 172.21.15.32:0/1570398870 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fac0096b0 con 0x7f5fc40b14e0 2020-12-05T17:31:48.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.047+0000 7f5fc9206700 1 -- 172.21.15.32:0/1570398870 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac009810 con 0x7f5fc40b14e0 2020-12-05T17:31:48.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.047+0000 7f5fc9206700 1 -- 172.21.15.32:0/1570398870 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fac009b70 con 0x7f5fc40b14e0 2020-12-05T17:31:48.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.047+0000 7f5fcb20a700 1 -- 172.21.15.32:0/1570398870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fc40b14e0 msgr2=0x7f5fc40ad510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:48.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.047+0000 7f5fcb20a700 1 --2- 172.21.15.32:0/1570398870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fc40b14e0 0x7f5fc40ad510 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f5fbc001450 tx=0x7f5fac006dc0).stop 2020-12-05T17:31:48.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.051+0000 7f5fcb20a700 1 -- 172.21.15.32:0/1570398870 shutdown_connections 2020-12-05T17:31:48.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.051+0000 7f5fcb20a700 1 --2- 172.21.15.32:0/1570398870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fc40b14e0 0x7f5fc40ad510 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.051+0000 7f5fcb20a700 1 -- 172.21.15.32:0/1570398870 >> 172.21.15.32:0/1570398870 conn(0x7f5fc40604c0 msgr2=0x7f5fc405c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:48.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.051+0000 7f5fcb20a700 1 -- 172.21.15.32:0/1570398870 shutdown_connections 2020-12-05T17:31:48.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.051+0000 7f5fcb20a700 1 -- 172.21.15.32:0/1570398870 wait complete. 2020-12-05T17:31:48.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.051+0000 7f5fcb20a700 1 Processor -- start 2020-12-05T17:31:48.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.051+0000 7f5fcb20a700 1 -- start start 2020-12-05T17:31:48.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.051+0000 7f5fcb20a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fc40af480 0x7f5fc4065590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.051+0000 7f5fcb20a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fc40b14e0 0x7f5fc413d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.051+0000 7f5fcb20a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5fc40b5f00 0x7f5fc413d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.051+0000 7f5fcb20a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc41453a0 con 0x7f5fc40af480 2020-12-05T17:31:48.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.051+0000 7f5fcb20a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc4145640 con 0x7f5fc40b14e0 2020-12-05T17:31:48.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.051+0000 7f5fcb20a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc41458e0 con 0x7f5fc40b5f00 2020-12-05T17:31:48.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.051+0000 7f5fca208700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fc40af480 0x7f5fc4065590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.051+0000 7f5fca208700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fc40af480 0x7f5fc4065590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32782/0 (socket says 172.21.15.32:32782) 2020-12-05T17:31:48.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.051+0000 7f5fca208700 1 -- 172.21.15.32:0/2743238341 learned_addr learned my addr 172.21.15.32:0/2743238341 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:48.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.051+0000 7f5fcaa09700 1 --2- 172.21.15.32:0/2743238341 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5fc40b5f00 0x7f5fc413d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.051+0000 7f5fc9a07700 1 --2- 172.21.15.32:0/2743238341 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fc40b14e0 0x7f5fc413d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.055+0000 7f5fca208700 1 -- 172.21.15.32:0/2743238341 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5fc40b5f00 msgr2=0x7f5fc413d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:48.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.055+0000 7f5fca208700 1 --2- 172.21.15.32:0/2743238341 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5fc40b5f00 0x7f5fc413d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.055+0000 7f5fca208700 1 -- 172.21.15.32:0/2743238341 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fc40b14e0 msgr2=0x7f5fc413d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:48.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.055+0000 7f5fca208700 1 --2- 172.21.15.32:0/2743238341 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fc40b14e0 0x7f5fc413d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.055+0000 7f5fca208700 1 -- 172.21.15.32:0/2743238341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc413da90 con 0x7f5fc40af480 2020-12-05T17:31:48.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.055+0000 7f5fc9a07700 1 --2- 172.21.15.32:0/2743238341 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fc40b14e0 0x7f5fc413d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:48.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.055+0000 7f5fcaa09700 1 --2- 172.21.15.32:0/2743238341 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5fc40b5f00 0x7f5fc413d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:48.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.055+0000 7f5fca208700 1 --2- 172.21.15.32:0/2743238341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fc40af480 0x7f5fc4065590 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f5fb4000e90 tx=0x7f5fb4007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:48.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.055+0000 7f5fc37fe700 1 -- 172.21.15.32:0/2743238341 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fb400e4a0 con 0x7f5fc40af480 2020-12-05T17:31:48.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.055+0000 7f5fc37fe700 1 -- 172.21.15.32:0/2743238341 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb400e600 con 0x7f5fc40af480 2020-12-05T17:31:48.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.055+0000 7f5fcb20a700 1 -- 172.21.15.32:0/2743238341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc4145020 con 0x7f5fc40af480 2020-12-05T17:31:48.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.055+0000 7f5fc37fe700 1 -- 172.21.15.32:0/2743238341 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fb400e960 con 0x7f5fc40af480 2020-12-05T17:31:48.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.055+0000 7f5fcb20a700 1 -- 172.21.15.32:0/2743238341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc414f4c0 con 0x7f5fc40af480 2020-12-05T17:31:48.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.055+0000 7f5fcb20a700 1 -- 172.21.15.32:0/2743238341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc41451b0 con 0x7f5fc40af480 2020-12-05T17:31:48.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.055+0000 7f5fc37fe700 1 -- 172.21.15.32:0/2743238341 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5fb4010030 con 0x7f5fc40af480 2020-12-05T17:31:48.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.059+0000 7f5fc37fe700 1 --2- 172.21.15.32:0/2743238341 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5fa80554e0 0x7f5fa80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.059+0000 7f5fc9a07700 1 --2- 172.21.15.32:0/2743238341 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5fa80554e0 0x7f5fa80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.059+0000 7f5fc37fe700 1 -- 172.21.15.32:0/2743238341 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5fb406d950 con 0x7f5fc40af480 2020-12-05T17:31:48.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.059+0000 7f5fc9a07700 1 --2- 172.21.15.32:0/2743238341 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5fa80554e0 0x7f5fa80579a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f5fac019000 tx=0x7f5fac015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:48.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.063+0000 7f5fc37fe700 1 -- 172.21.15.32:0/2743238341 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5fb4043af0 con 0x7f5fc40af480 2020-12-05T17:31:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.203+0000 7f5fcb20a700 1 -- 172.21.15.32:0/2743238341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5fc4057050 con 0x7f5fc40af480 2020-12-05T17:31:48.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.207+0000 7f5fc37fe700 1 -- 172.21.15.32:0/2743238341 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5fb4001e60 con 0x7f5fc40af480 2020-12-05T17:31:48.209 INFO:teuthology.orchestra.run.smithi032.stdout:60129542149 2020-12-05T17:31:48.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.207+0000 7f5fcb20a700 1 -- 172.21.15.32:0/2743238341 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5fa80554e0 msgr2=0x7f5fa80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:48.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.207+0000 7f5fcb20a700 1 --2- 172.21.15.32:0/2743238341 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5fa80554e0 0x7f5fa80579a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f5fac019000 tx=0x7f5fac015000).stop 2020-12-05T17:31:48.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.207+0000 7f5fcb20a700 1 -- 172.21.15.32:0/2743238341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fc40af480 msgr2=0x7f5fc4065590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:48.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.207+0000 7f5fcb20a700 1 --2- 172.21.15.32:0/2743238341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fc40af480 0x7f5fc4065590 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f5fb4000e90 tx=0x7f5fb4007c20).stop 2020-12-05T17:31:48.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.207+0000 7f5fcb20a700 1 -- 172.21.15.32:0/2743238341 shutdown_connections 2020-12-05T17:31:48.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.207+0000 7f5fcb20a700 1 --2- 172.21.15.32:0/2743238341 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5fa80554e0 0x7f5fa80579a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.207+0000 7f5fcb20a700 1 --2- 172.21.15.32:0/2743238341 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5fc40b5f00 0x7f5fc413d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.207+0000 7f5fcb20a700 1 --2- 172.21.15.32:0/2743238341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fc40af480 0x7f5fc4065590 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.207+0000 7f5fcb20a700 1 --2- 172.21.15.32:0/2743238341 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fc40b14e0 0x7f5fc413d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.207+0000 7f5fcb20a700 1 -- 172.21.15.32:0/2743238341 >> 172.21.15.32:0/2743238341 conn(0x7f5fc40604c0 msgr2=0x7f5fc40ae9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:48.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.207+0000 7f5fcb20a700 1 -- 172.21.15.32:0/2743238341 shutdown_connections 2020-12-05T17:31:48.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.207+0000 7f5fcb20a700 1 -- 172.21.15.32:0/2743238341 wait complete. 2020-12-05T17:31:48.223 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.3 2020-12-05T17:31:48.223 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.4 2020-12-05T17:31:48.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.335+0000 7f6a9759e700 1 Processor -- start 2020-12-05T17:31:48.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.335+0000 7f6a9759e700 1 -- start start 2020-12-05T17:31:48.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.335+0000 7f6a9759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6a980b2070 0x7f6a98138e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.335+0000 7f6a9759e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f6a98139320 0x7f6a98143090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.335+0000 7f6a9759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a980b2450 con 0x7f6a980b2ca0 2020-12-05T17:31:48.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.335+0000 7f6a9759e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a980acb80 con 0x7f6a980b2070 2020-12-05T17:31:48.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.335+0000 7f6a9759e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a98066d40 con 0x7f6a98139320 2020-12-05T17:31:48.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.335+0000 7f6a9659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6a980b2070 0x7f6a98138e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.335+0000 7f6a95d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6a980b2ca0 0x7f6a98142980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35440/0 (socket says 172.21.15.32:35440) 2020-12-05T17:31:48.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.335+0000 7f6a95d9b700 1 -- 172.21.15.32:0/30835565 learned_addr learned my addr 172.21.15.32:0/30835565 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:48.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.339+0000 7f6a96d9d700 1 --2- 172.21.15.32:0/30835565 >> v2:172.21.15.61:3300/0 conn(0x7f6a98139320 0x7f6a98143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.339+0000 7f6a9559a700 1 -- 172.21.15.32:0/30835565 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2721456489 0 0) 0x7f6a980b2450 con 0x7f6a980b2ca0 2020-12-05T17:31:48.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.339+0000 7f6a9559a700 1 -- 172.21.15.32:0/30835565 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a7c003230 con 0x7f6a980b2ca0 2020-12-05T17:31:48.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.339+0000 7f6a9559a700 1 -- 172.21.15.32:0/30835565 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3751729501 0 0) 0x7f6a7c003230 con 0x7f6a980b2ca0 2020-12-05T17:31:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.339+0000 7f6a9659c700 1 -- 172.21.15.32:0/30835565 >> v2:172.21.15.61:3300/0 conn(0x7f6a98139320 msgr2=0x7f6a98143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.339+0000 7f6a9659c700 1 --2- 172.21.15.32:0/30835565 >> v2:172.21.15.61:3300/0 conn(0x7f6a98139320 0x7f6a98143090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.339+0000 7f6a9659c700 1 -- 172.21.15.32:0/30835565 >> v1:172.21.15.32:6789/0 conn(0x7f6a980b2ca0 legacy=0x7f6a98142980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.339+0000 7f6a9659c700 1 -- 172.21.15.32:0/30835565 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a98066fe0 con 0x7f6a980b2070 2020-12-05T17:31:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.339+0000 7f6a9659c700 1 --2- 172.21.15.32:0/30835565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a980b2070 0x7f6a98138e30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f6a98067360 tx=0x7f6a88006dc0).ready entity=mon.0 client_cookie=f78187f5d407ff20 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:48.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.339+0000 7f6a9559a700 1 -- 172.21.15.32:0/30835565 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a8c0016d0 con 0x7f6a980b2070 2020-12-05T17:31:48.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.339+0000 7f6a9559a700 1 -- 172.21.15.32:0/30835565 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8800d5c0 con 0x7f6a980b2070 2020-12-05T17:31:48.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.339+0000 7f6a9559a700 1 -- 172.21.15.32:0/30835565 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a8800d730 con 0x7f6a980b2070 2020-12-05T17:31:48.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.339+0000 7f6a9759e700 1 -- 172.21.15.32:0/30835565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a980b2070 msgr2=0x7f6a98138e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:48.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.339+0000 7f6a9759e700 1 --2- 172.21.15.32:0/30835565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a980b2070 0x7f6a98138e30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f6a98067360 tx=0x7f6a88006dc0).stop 2020-12-05T17:31:48.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.339+0000 7f6a9759e700 1 -- 172.21.15.32:0/30835565 shutdown_connections 2020-12-05T17:31:48.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.339+0000 7f6a9759e700 1 --2- 172.21.15.32:0/30835565 >> v2:172.21.15.61:3300/0 conn(0x7f6a98139320 0x7f6a98143090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.339+0000 7f6a9759e700 1 --2- 172.21.15.32:0/30835565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a980b2070 0x7f6a98138e30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.339+0000 7f6a9759e700 1 -- 172.21.15.32:0/30835565 >> 172.21.15.32:0/30835565 conn(0x7f6a9805bbb0 msgr2=0x7f6a98062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.339+0000 7f6a9759e700 1 -- 172.21.15.32:0/30835565 shutdown_connections 2020-12-05T17:31:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.339+0000 7f6a9759e700 1 -- 172.21.15.32:0/30835565 wait complete. 2020-12-05T17:31:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a9759e700 1 Processor -- start 2020-12-05T17:31:48.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a9759e700 1 -- start start 2020-12-05T17:31:48.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a9759e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a980b2070 0x7f6a9813e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a9759e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a980b2ca0 0x7f6a98139e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a9759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a98139320 0x7f6a9813abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a9759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a981421f0 con 0x7f6a98139320 2020-12-05T17:31:48.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a9759e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9813e810 con 0x7f6a980b2070 2020-12-05T17:31:48.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a9759e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9813a300 con 0x7f6a980b2ca0 2020-12-05T17:31:48.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a96d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a98139320 0x7f6a9813abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a96d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a98139320 0x7f6a9813abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32796/0 (socket says 172.21.15.32:32796) 2020-12-05T17:31:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a96d9d700 1 -- 172.21.15.32:0/3254898548 learned_addr learned my addr 172.21.15.32:0/3254898548 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a9659c700 1 --2- 172.21.15.32:0/3254898548 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a980b2070 0x7f6a9813e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a95d9b700 1 --2- 172.21.15.32:0/3254898548 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a980b2ca0 0x7f6a98139e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a96d9d700 1 -- 172.21.15.32:0/3254898548 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a980b2ca0 msgr2=0x7f6a98139e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a96d9d700 1 --2- 172.21.15.32:0/3254898548 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a980b2ca0 0x7f6a98139e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a96d9d700 1 -- 172.21.15.32:0/3254898548 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a980b2070 msgr2=0x7f6a9813e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:48.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a96d9d700 1 --2- 172.21.15.32:0/3254898548 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a980b2070 0x7f6a9813e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a96d9d700 1 -- 172.21.15.32:0/3254898548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9813eef0 con 0x7f6a98139320 2020-12-05T17:31:48.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a9659c700 1 --2- 172.21.15.32:0/3254898548 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a980b2070 0x7f6a9813e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:31:48.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a95d9b700 1 --2- 172.21.15.32:0/3254898548 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a980b2ca0 0x7f6a98139e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:31:48.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a96d9d700 1 --2- 172.21.15.32:0/3254898548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a98139320 0x7f6a9813abc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f6a98143930 tx=0x7f6a900091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:48.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a877fe700 1 -- 172.21.15.32:0/3254898548 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a9000cd40 con 0x7f6a98139320 2020-12-05T17:31:48.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a877fe700 1 -- 172.21.15.32:0/3254898548 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9000cea0 con 0x7f6a98139320 2020-12-05T17:31:48.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a9759e700 1 -- 172.21.15.32:0/3254898548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a9813f170 con 0x7f6a98139320 2020-12-05T17:31:48.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a877fe700 1 -- 172.21.15.32:0/3254898548 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a9001a660 con 0x7f6a98139320 2020-12-05T17:31:48.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a9759e700 1 -- 172.21.15.32:0/3254898548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a98136f00 con 0x7f6a98139320 2020-12-05T17:31:48.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a9759e700 1 -- 172.21.15.32:0/3254898548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a9813ec50 con 0x7f6a98139320 2020-12-05T17:31:48.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.343+0000 7f6a877fe700 1 -- 172.21.15.32:0/3254898548 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6a9001a7c0 con 0x7f6a98139320 2020-12-05T17:31:48.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.347+0000 7f6a877fe700 1 --2- 172.21.15.32:0/3254898548 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6a74055440 0x7f6a74057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.347+0000 7f6a877fe700 1 -- 172.21.15.32:0/3254898548 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f6a90070770 con 0x7f6a98139320 2020-12-05T17:31:48.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.347+0000 7f6a9659c700 1 --2- 172.21.15.32:0/3254898548 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6a74055440 0x7f6a74057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.347+0000 7f6a9659c700 1 --2- 172.21.15.32:0/3254898548 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6a74055440 0x7f6a74057900 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f6a88016000 tx=0x7f6a88015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:48.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.351+0000 7f6a877fe700 1 -- 172.21.15.32:0/3254898548 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6a9813ec50 con 0x7f6a98139320 2020-12-05T17:31:48.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.487+0000 7f6a9759e700 1 -- 172.21.15.32:0/3254898548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6a9813ec50 con 0x7f6a98139320 2020-12-05T17:31:48.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.487+0000 7f6a877fe700 1 -- 172.21.15.32:0/3254898548 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6a9813ec50 con 0x7f6a98139320 2020-12-05T17:31:48.493 INFO:teuthology.orchestra.run.smithi032.stdout:60129542150 2020-12-05T17:31:48.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.491+0000 7f6a9759e700 1 -- 172.21.15.32:0/3254898548 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6a74055440 msgr2=0x7f6a74057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:48.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.491+0000 7f6a9759e700 1 --2- 172.21.15.32:0/3254898548 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6a74055440 0x7f6a74057900 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f6a88016000 tx=0x7f6a88015000).stop 2020-12-05T17:31:48.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.491+0000 7f6a9759e700 1 -- 172.21.15.32:0/3254898548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a98139320 msgr2=0x7f6a9813abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:48.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.491+0000 7f6a9759e700 1 --2- 172.21.15.32:0/3254898548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a98139320 0x7f6a9813abc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f6a98143930 tx=0x7f6a900091a0).stop 2020-12-05T17:31:48.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.491+0000 7f6a9759e700 1 -- 172.21.15.32:0/3254898548 shutdown_connections 2020-12-05T17:31:48.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.491+0000 7f6a9759e700 1 --2- 172.21.15.32:0/3254898548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a98139320 0x7f6a9813abc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.491+0000 7f6a9759e700 1 --2- 172.21.15.32:0/3254898548 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a980b2070 0x7f6a9813e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.491+0000 7f6a9759e700 1 --2- 172.21.15.32:0/3254898548 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6a74055440 0x7f6a74057900 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.491+0000 7f6a9759e700 1 --2- 172.21.15.32:0/3254898548 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a980b2ca0 0x7f6a98139e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.491+0000 7f6a9759e700 1 -- 172.21.15.32:0/3254898548 >> 172.21.15.32:0/3254898548 conn(0x7f6a9805bbb0 msgr2=0x7f6a980aded0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:48.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.491+0000 7f6a9759e700 1 -- 172.21.15.32:0/3254898548 shutdown_connections 2020-12-05T17:31:48.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.491+0000 7f6a9759e700 1 -- 172.21.15.32:0/3254898548 wait complete. 2020-12-05T17:31:48.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.4 2020-12-05T17:31:48.510 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.5 2020-12-05T17:31:48.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.619+0000 7fd73d71e700 1 Processor -- start 2020-12-05T17:31:48.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.619+0000 7fd73d71e700 1 -- start start 2020-12-05T17:31:48.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.619+0000 7fd73d71e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd7380b5f00 0x7fd7380b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.619+0000 7fd73d71e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7380ad510 0x7fd7380af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.619+0000 7fd73d71e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd738142ab0 con 0x7fd738066370 2020-12-05T17:31:48.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.619+0000 7fd73d71e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7380b62e0 con 0x7fd7380b5f00 2020-12-05T17:31:48.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd73d71e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7380afbf0 con 0x7fd7380ad510 2020-12-05T17:31:48.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd7377fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7380ad510 0x7fd7380af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd737fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd7380b5f00 0x7fd7380b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd73cf1d700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fd738066370 0x7fd738142ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:48096/0 (socket says 172.21.15.32:48096) 2020-12-05T17:31:48.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd737fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd7380b5f00 0x7fd7380b14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:52696/0 (socket says 172.21.15.32:52696) 2020-12-05T17:31:48.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd73cf1d700 1 -- 172.21.15.32:0/1595016785 learned_addr learned my addr 172.21.15.32:0/1595016785 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:48.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd7377fe700 1 --2- 172.21.15.32:0/1595016785 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7380ad510 0x7fd7380af480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:57828/0 (socket says 172.21.15.32:57828) 2020-12-05T17:31:48.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd7377fe700 1 -- 172.21.15.32:0/1595016785 >> v2:172.21.15.61:3300/0 conn(0x7fd7380b5f00 msgr2=0x7fd7380b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:48.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd7377fe700 1 --2- 172.21.15.32:0/1595016785 >> v2:172.21.15.61:3300/0 conn(0x7fd7380b5f00 0x7fd7380b14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd7377fe700 1 -- 172.21.15.32:0/1595016785 >> v1:172.21.15.61:6789/0 conn(0x7fd738066370 legacy=0x7fd738142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:48.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd7377fe700 1 -- 172.21.15.32:0/1595016785 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7380afe90 con 0x7fd7380ad510 2020-12-05T17:31:48.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd737fff700 1 --2- 172.21.15.32:0/1595016785 >> v2:172.21.15.61:3300/0 conn(0x7fd7380b5f00 0x7fd7380b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:31:48.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd7377fe700 1 --2- 172.21.15.32:0/1595016785 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7380ad510 0x7fd7380af480 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fd72c002f30 tx=0x7fd72c006dc0).ready entity=mon.2 client_cookie=56ad2abaaaed6a90 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:48.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd736ffd700 1 -- 172.21.15.32:0/1595016785 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd72c009760 con 0x7fd7380ad510 2020-12-05T17:31:48.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd736ffd700 1 -- 172.21.15.32:0/1595016785 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72c0098c0 con 0x7fd7380ad510 2020-12-05T17:31:48.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd736ffd700 1 -- 172.21.15.32:0/1595016785 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd72c009c20 con 0x7fd7380ad510 2020-12-05T17:31:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd73d71e700 1 -- 172.21.15.32:0/1595016785 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7380ad510 msgr2=0x7fd7380af480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd73d71e700 1 --2- 172.21.15.32:0/1595016785 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7380ad510 0x7fd7380af480 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fd72c002f30 tx=0x7fd72c006dc0).stop 2020-12-05T17:31:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd73d71e700 1 -- 172.21.15.32:0/1595016785 shutdown_connections 2020-12-05T17:31:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd73d71e700 1 --2- 172.21.15.32:0/1595016785 >> v2:172.21.15.61:3300/0 conn(0x7fd7380b5f00 0x7fd7380b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd73d71e700 1 --2- 172.21.15.32:0/1595016785 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7380ad510 0x7fd7380af480 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd73d71e700 1 -- 172.21.15.32:0/1595016785 >> 172.21.15.32:0/1595016785 conn(0x7fd7380604c0 msgr2=0x7fd73805c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd73d71e700 1 -- 172.21.15.32:0/1595016785 shutdown_connections 2020-12-05T17:31:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.623+0000 7fd73d71e700 1 -- 172.21.15.32:0/1595016785 wait complete. 2020-12-05T17:31:48.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd73d71e700 1 Processor -- start 2020-12-05T17:31:48.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd73d71e700 1 -- start start 2020-12-05T17:31:48.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd73d71e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd738066370 0x7fd73813d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd73d71e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7380ad510 0x7fd738144ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd73d71e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd7380b5f00 0x7fd738151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd73d71e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd738063040 con 0x7fd738066370 2020-12-05T17:31:48.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd73d71e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7380632e0 con 0x7fd7380b5f00 2020-12-05T17:31:48.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd73d71e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd738063580 con 0x7fd7380ad510 2020-12-05T17:31:48.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd737fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd738066370 0x7fd73813d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd737fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd738066370 0x7fd73813d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32810/0 (socket says 172.21.15.32:32810) 2020-12-05T17:31:48.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd737fff700 1 -- 172.21.15.32:0/387315480 learned_addr learned my addr 172.21.15.32:0/387315480 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:48.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd7377fe700 1 --2- 172.21.15.32:0/387315480 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7380ad510 0x7fd738144ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd73cf1d700 1 --2- 172.21.15.32:0/387315480 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd7380b5f00 0x7fd738151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd737fff700 1 -- 172.21.15.32:0/387315480 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7380ad510 msgr2=0x7fd738144ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:48.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd737fff700 1 --2- 172.21.15.32:0/387315480 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7380ad510 0x7fd738144ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd737fff700 1 -- 172.21.15.32:0/387315480 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd7380b5f00 msgr2=0x7fd738151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:48.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd737fff700 1 --2- 172.21.15.32:0/387315480 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd7380b5f00 0x7fd738151a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd737fff700 1 -- 172.21.15.32:0/387315480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd738063820 con 0x7fd738066370 2020-12-05T17:31:48.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd73cf1d700 1 --2- 172.21.15.32:0/387315480 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd7380b5f00 0x7fd738151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:48.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd737fff700 1 --2- 172.21.15.32:0/387315480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd738066370 0x7fd73813d270 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fd72800c000 tx=0x7fd7280091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:48.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd7357fa700 1 -- 172.21.15.32:0/387315480 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd72800cc30 con 0x7fd738066370 2020-12-05T17:31:48.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd7357fa700 1 -- 172.21.15.32:0/387315480 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72800cd90 con 0x7fd738066370 2020-12-05T17:31:48.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd7357fa700 1 -- 172.21.15.32:0/387315480 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7280125e0 con 0x7fd738066370 2020-12-05T17:31:48.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd73d71e700 1 -- 172.21.15.32:0/387315480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd738148850 con 0x7fd738066370 2020-12-05T17:31:48.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.627+0000 7fd73d71e700 1 -- 172.21.15.32:0/387315480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7381520a0 con 0x7fd738066370 2020-12-05T17:31:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.631+0000 7fd73d71e700 1 -- 172.21.15.32:0/387315480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd738151770 con 0x7fd738066370 2020-12-05T17:31:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.631+0000 7fd7357fa700 1 -- 172.21.15.32:0/387315480 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd728022070 con 0x7fd738066370 2020-12-05T17:31:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.631+0000 7fd7357fa700 1 --2- 172.21.15.32:0/387315480 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd718055440 0x7fd718057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.631+0000 7fd7357fa700 1 -- 172.21.15.32:0/387315480 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd72806eee0 con 0x7fd738066370 2020-12-05T17:31:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.631+0000 7fd7377fe700 1 --2- 172.21.15.32:0/387315480 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd718055440 0x7fd718057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.631+0000 7fd7377fe700 1 --2- 172.21.15.32:0/387315480 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd718055440 0x7fd718057900 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fd72c01a000 tx=0x7fd72c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.635+0000 7fd7357fa700 1 -- 172.21.15.32:0/387315480 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd728045080 con 0x7fd738066370 2020-12-05T17:31:48.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.775+0000 7fd73d71e700 1 -- 172.21.15.32:0/387315480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fd738151770 con 0x7fd738066370 2020-12-05T17:31:48.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.775+0000 7fd7357fa700 1 -- 172.21.15.32:0/387315480 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd738151770 con 0x7fd738066370 2020-12-05T17:31:48.779 INFO:teuthology.orchestra.run.smithi032.stdout:60129542149 2020-12-05T17:31:48.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.775+0000 7fd73d71e700 1 -- 172.21.15.32:0/387315480 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd718055440 msgr2=0x7fd718057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:48.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.775+0000 7fd73d71e700 1 --2- 172.21.15.32:0/387315480 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd718055440 0x7fd718057900 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fd72c01a000 tx=0x7fd72c015000).stop 2020-12-05T17:31:48.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.775+0000 7fd73d71e700 1 -- 172.21.15.32:0/387315480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd738066370 msgr2=0x7fd73813d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:48.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.775+0000 7fd73d71e700 1 --2- 172.21.15.32:0/387315480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd738066370 0x7fd73813d270 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fd72800c000 tx=0x7fd7280091a0).stop 2020-12-05T17:31:48.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.775+0000 7fd73d71e700 1 -- 172.21.15.32:0/387315480 shutdown_connections 2020-12-05T17:31:48.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.779+0000 7fd73d71e700 1 --2- 172.21.15.32:0/387315480 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd7380b5f00 0x7fd738151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.779+0000 7fd73d71e700 1 --2- 172.21.15.32:0/387315480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd738066370 0x7fd73813d270 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.779+0000 7fd73d71e700 1 --2- 172.21.15.32:0/387315480 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd718055440 0x7fd718057900 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.779+0000 7fd73d71e700 1 --2- 172.21.15.32:0/387315480 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7380ad510 0x7fd738144ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.779+0000 7fd73d71e700 1 -- 172.21.15.32:0/387315480 >> 172.21.15.32:0/387315480 conn(0x7fd7380604c0 msgr2=0x7fd73805f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:48.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.779+0000 7fd73d71e700 1 -- 172.21.15.32:0/387315480 shutdown_connections 2020-12-05T17:31:48.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.779+0000 7fd73d71e700 1 -- 172.21.15.32:0/387315480 wait complete. 2020-12-05T17:31:48.792 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.5 2020-12-05T17:31:48.793 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.6 2020-12-05T17:31:48.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.903+0000 7f9b20c1f700 1 Processor -- start 2020-12-05T17:31:48.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.903+0000 7f9b20c1f700 1 -- start start 2020-12-05T17:31:48.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.907+0000 7f9b20c1f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b1c05f3b0 0x7f9b1c05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.907+0000 7f9b20c1f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b1c062400 0x7f9b1c0b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.907+0000 7f9b20c1f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9b1c05feb0 0x7f9b1c0b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.907+0000 7f9b20c1f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b1c142450 con 0x7f9b1c062400 2020-12-05T17:31:48.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.907+0000 7f9b20c1f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b1c061780 con 0x7f9b1c05feb0 2020-12-05T17:31:48.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.907+0000 7f9b20c1f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c061a20 con 0x7f9b1c05f3b0 2020-12-05T17:31:48.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.907+0000 7f9b1affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b1c062400 0x7f9b1c0b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.907+0000 7f9b1affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b1c062400 0x7f9b1c0b36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32818/0 (socket says 172.21.15.32:32818) 2020-12-05T17:31:48.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.907+0000 7f9b1affd700 1 -- 172.21.15.32:0/414956772 learned_addr learned my addr 172.21.15.32:0/414956772 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:48.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.907+0000 7f9b1bfff700 1 --2- 172.21.15.32:0/414956772 >> v2:172.21.15.61:3300/0 conn(0x7f9b1c05feb0 0x7f9b1c0b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.907+0000 7f9b1b7fe700 1 --2- 172.21.15.32:0/414956772 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b1c05f3b0 0x7f9b1c05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.907+0000 7f9b1affd700 1 -- 172.21.15.32:0/414956772 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b1c05f3b0 msgr2=0x7f9b1c05f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:48.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.907+0000 7f9b1affd700 1 --2- 172.21.15.32:0/414956772 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b1c05f3b0 0x7f9b1c05f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.907+0000 7f9b1affd700 1 -- 172.21.15.32:0/414956772 >> v2:172.21.15.61:3300/0 conn(0x7f9b1c05feb0 msgr2=0x7f9b1c0b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:48.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.907+0000 7f9b1affd700 1 --2- 172.21.15.32:0/414956772 >> v2:172.21.15.61:3300/0 conn(0x7f9b1c05feb0 0x7f9b1c0b3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.907+0000 7f9b1affd700 1 -- 172.21.15.32:0/414956772 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c061cc0 con 0x7f9b1c062400 2020-12-05T17:31:48.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.907+0000 7f9b1affd700 1 --2- 172.21.15.32:0/414956772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b1c062400 0x7f9b1c0b36f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f9b10002f30 tx=0x7f9b10006e80).ready entity=mon.0 client_cookie=c5ef5377aee7a290 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:48.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.907+0000 7f9b1a7fc700 1 -- 172.21.15.32:0/414956772 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b10009640 con 0x7f9b1c062400 2020-12-05T17:31:48.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.907+0000 7f9b1a7fc700 1 -- 172.21.15.32:0/414956772 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b100097a0 con 0x7f9b1c062400 2020-12-05T17:31:48.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.907+0000 7f9b1a7fc700 1 -- 172.21.15.32:0/414956772 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b10009b00 con 0x7f9b1c062400 2020-12-05T17:31:48.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.907+0000 7f9b20c1f700 1 -- 172.21.15.32:0/414956772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b1c062400 msgr2=0x7f9b1c0b36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:48.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.907+0000 7f9b20c1f700 1 --2- 172.21.15.32:0/414956772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b1c062400 0x7f9b1c0b36f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f9b10002f30 tx=0x7f9b10006e80).stop 2020-12-05T17:31:48.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b20c1f700 1 -- 172.21.15.32:0/414956772 shutdown_connections 2020-12-05T17:31:48.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b20c1f700 1 --2- 172.21.15.32:0/414956772 >> v2:172.21.15.61:3300/0 conn(0x7f9b1c05feb0 0x7f9b1c0b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b20c1f700 1 --2- 172.21.15.32:0/414956772 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b1c05f3b0 0x7f9b1c05f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b20c1f700 1 --2- 172.21.15.32:0/414956772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b1c062400 0x7f9b1c0b36f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b20c1f700 1 -- 172.21.15.32:0/414956772 >> 172.21.15.32:0/414956772 conn(0x7f9b1c0b8ee0 msgr2=0x7f9b1c0b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:48.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b20c1f700 1 -- 172.21.15.32:0/414956772 shutdown_connections 2020-12-05T17:31:48.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b20c1f700 1 -- 172.21.15.32:0/414956772 wait complete. 2020-12-05T17:31:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b20c1f700 1 Processor -- start 2020-12-05T17:31:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b20c1f700 1 -- start start 2020-12-05T17:31:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b20c1f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b1c05f3b0 0x7f9b1c146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b20c1f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b1c05feb0 0x7f9b1c1482e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b20c1f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b1c062400 0x7f9b1c065fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b20c1f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c1451a0 con 0x7f9b1c062400 2020-12-05T17:31:48.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b20c1f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c1459b0 con 0x7f9b1c05feb0 2020-12-05T17:31:48.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b20c1f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c145c50 con 0x7f9b1c05f3b0 2020-12-05T17:31:48.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b1bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b1c062400 0x7f9b1c065fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b1bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b1c062400 0x7f9b1c065fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32828/0 (socket says 172.21.15.32:32828) 2020-12-05T17:31:48.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b1bfff700 1 -- 172.21.15.32:0/3968587838 learned_addr learned my addr 172.21.15.32:0/3968587838 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:48.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b1b7fe700 1 --2- 172.21.15.32:0/3968587838 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b1c05f3b0 0x7f9b1c146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b1affd700 1 --2- 172.21.15.32:0/3968587838 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b1c05feb0 0x7f9b1c1482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b1bfff700 1 -- 172.21.15.32:0/3968587838 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b1c05f3b0 msgr2=0x7f9b1c146550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:48.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b1bfff700 1 --2- 172.21.15.32:0/3968587838 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b1c05f3b0 0x7f9b1c146550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b1bfff700 1 -- 172.21.15.32:0/3968587838 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b1c05feb0 msgr2=0x7f9b1c1482e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:48.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b1bfff700 1 --2- 172.21.15.32:0/3968587838 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b1c05feb0 0x7f9b1c1482e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:48.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b1bfff700 1 -- 172.21.15.32:0/3968587838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c145ef0 con 0x7f9b1c062400 2020-12-05T17:31:48.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b1affd700 1 --2- 172.21.15.32:0/3968587838 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b1c05feb0 0x7f9b1c1482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:48.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b1bfff700 1 --2- 172.21.15.32:0/3968587838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b1c062400 0x7f9b1c065fb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f9b14006dd0 tx=0x7f9b140091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:48.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b18ff9700 1 -- 172.21.15.32:0/3968587838 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b140095f0 con 0x7f9b1c062400 2020-12-05T17:31:48.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b18ff9700 1 -- 172.21.15.32:0/3968587838 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1400cbb0 con 0x7f9b1c062400 2020-12-05T17:31:48.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b20c1f700 1 -- 172.21.15.32:0/3968587838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b1c0659b0 con 0x7f9b1c062400 2020-12-05T17:31:48.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b18ff9700 1 -- 172.21.15.32:0/3968587838 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b140123f0 con 0x7f9b1c062400 2020-12-05T17:31:48.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.911+0000 7f9b20c1f700 1 -- 172.21.15.32:0/3968587838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b1c0afb90 con 0x7f9b1c062400 2020-12-05T17:31:48.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.915+0000 7f9b20c1f700 1 -- 172.21.15.32:0/3968587838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b1c057050 con 0x7f9b1c062400 2020-12-05T17:31:48.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.915+0000 7f9b18ff9700 1 -- 172.21.15.32:0/3968587838 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9b14022070 con 0x7f9b1c062400 2020-12-05T17:31:48.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.915+0000 7f9b18ff9700 1 --2- 172.21.15.32:0/3968587838 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9afc055330 0x7f9afc0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:48.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.915+0000 7f9b1b7fe700 1 --2- 172.21.15.32:0/3968587838 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9afc055330 0x7f9afc0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:48.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.915+0000 7f9b18ff9700 1 -- 172.21.15.32:0/3968587838 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f9b1406e910 con 0x7f9b1c062400 2020-12-05T17:31:48.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.915+0000 7f9b1b7fe700 1 --2- 172.21.15.32:0/3968587838 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9afc055330 0x7f9afc0577f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f9b0c00b000 tx=0x7f9b0c006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:48.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:48.919+0000 7f9b18ff9700 1 -- 172.21.15.32:0/3968587838 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9b140449a0 con 0x7f9b1c062400 2020-12-05T17:31:49.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:49.063+0000 7f9b20c1f700 1 -- 172.21.15.32:0/3968587838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f9b1c0b30f0 con 0x7f9b1c062400 2020-12-05T17:31:49.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:49.063+0000 7f9b18ff9700 1 -- 172.21.15.32:0/3968587838 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9b14016020 con 0x7f9b1c062400 2020-12-05T17:31:49.067 INFO:teuthology.orchestra.run.smithi032.stdout:60129542149 2020-12-05T17:31:49.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:49.063+0000 7f9b20c1f700 1 -- 172.21.15.32:0/3968587838 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9afc055330 msgr2=0x7f9afc0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:49.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:49.063+0000 7f9b20c1f700 1 --2- 172.21.15.32:0/3968587838 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9afc055330 0x7f9afc0577f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f9b0c00b000 tx=0x7f9b0c006dc0).stop 2020-12-05T17:31:49.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:49.063+0000 7f9b20c1f700 1 -- 172.21.15.32:0/3968587838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b1c062400 msgr2=0x7f9b1c065fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:49.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:49.063+0000 7f9b20c1f700 1 --2- 172.21.15.32:0/3968587838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b1c062400 0x7f9b1c065fb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f9b14006dd0 tx=0x7f9b140091a0).stop 2020-12-05T17:31:49.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:49.063+0000 7f9b20c1f700 1 -- 172.21.15.32:0/3968587838 shutdown_connections 2020-12-05T17:31:49.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:49.063+0000 7f9b20c1f700 1 --2- 172.21.15.32:0/3968587838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b1c062400 0x7f9b1c065fb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:49.063+0000 7f9b20c1f700 1 --2- 172.21.15.32:0/3968587838 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9afc055330 0x7f9afc0577f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:49.063+0000 7f9b20c1f700 1 --2- 172.21.15.32:0/3968587838 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b1c05f3b0 0x7f9b1c146550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:49.063+0000 7f9b20c1f700 1 --2- 172.21.15.32:0/3968587838 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b1c05feb0 0x7f9b1c1482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:49.063+0000 7f9b20c1f700 1 -- 172.21.15.32:0/3968587838 >> 172.21.15.32:0/3968587838 conn(0x7f9b1c0b8ee0 msgr2=0x7f9b1c13f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:49.063+0000 7f9b20c1f700 1 -- 172.21.15.32:0/3968587838 shutdown_connections 2020-12-05T17:31:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:49.063+0000 7f9b20c1f700 1 -- 172.21.15.32:0/3968587838 wait complete. 2020-12-05T17:31:49.080 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.6 2020-12-05T17:31:50.081 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.6 2020-12-05T17:31:50.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f013905c700 1 Processor -- start 2020-12-05T17:31:50.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f013905c700 1 -- start start 2020-12-05T17:31:50.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f013905c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f013405f3b0 0x7f013405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:50.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f013905c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0134062400 0x7f01340b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:50.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f013905c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0134142450 con 0x7f013405feb0 2020-12-05T17:31:50.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f013905c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01340b42f0 con 0x7f013405f3b0 2020-12-05T17:31:50.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f013905c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0134061780 con 0x7f0134062400 2020-12-05T17:31:50.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f0133fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f013405f3b0 0x7f013405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:50.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f013885b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f013405feb0 0x7f01340b3be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35482/0 (socket says 172.21.15.32:35482) 2020-12-05T17:31:50.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f013885b700 1 -- 172.21.15.32:0/1080575308 learned_addr learned my addr 172.21.15.32:0/1080575308 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:50.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f01337fe700 1 --2- 172.21.15.32:0/1080575308 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0134062400 0x7f01340b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:50.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f0133fff700 1 -- 172.21.15.32:0/1080575308 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0134062400 msgr2=0x7f01340b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:50.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f0133fff700 1 --2- 172.21.15.32:0/1080575308 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0134062400 0x7f01340b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f0133fff700 1 -- 172.21.15.32:0/1080575308 >> v1:172.21.15.32:6789/0 conn(0x7f013405feb0 legacy=0x7f01340b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:50.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f0132ffd700 1 -- 172.21.15.32:0/1080575308 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 156303327 0 0) 0x7f0134142450 con 0x7f013405feb0 2020-12-05T17:31:50.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f0133fff700 1 -- 172.21.15.32:0/1080575308 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0134061a20 con 0x7f013405f3b0 2020-12-05T17:31:50.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f0133fff700 1 --2- 172.21.15.32:0/1080575308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013405f3b0 0x7f013405f790 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f01240032b0 tx=0x7f01240091a0).ready entity=mon.0 client_cookie=e12dc926052e5019 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f0132ffd700 1 -- 172.21.15.32:0/1080575308 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f012400ba50 con 0x7f013405f3b0 2020-12-05T17:31:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f0132ffd700 1 -- 172.21.15.32:0/1080575308 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012400bbb0 con 0x7f013405f3b0 2020-12-05T17:31:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f0132ffd700 1 -- 172.21.15.32:0/1080575308 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f012400bd20 con 0x7f013405f3b0 2020-12-05T17:31:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f013905c700 1 -- 172.21.15.32:0/1080575308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013405f3b0 msgr2=0x7f013405f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f013905c700 1 --2- 172.21.15.32:0/1080575308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013405f3b0 0x7f013405f790 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f01240032b0 tx=0x7f01240091a0).stop 2020-12-05T17:31:50.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f013905c700 1 -- 172.21.15.32:0/1080575308 shutdown_connections 2020-12-05T17:31:50.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f013905c700 1 --2- 172.21.15.32:0/1080575308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013405f3b0 0x7f013405f790 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f013905c700 1 --2- 172.21.15.32:0/1080575308 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0134062400 0x7f01340b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f013905c700 1 -- 172.21.15.32:0/1080575308 >> 172.21.15.32:0/1080575308 conn(0x7f01340b8ee0 msgr2=0x7f01340b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:50.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f013905c700 1 -- 172.21.15.32:0/1080575308 shutdown_connections 2020-12-05T17:31:50.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.195+0000 7f013905c700 1 -- 172.21.15.32:0/1080575308 wait complete. 2020-12-05T17:31:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f013905c700 1 Processor -- start 2020-12-05T17:31:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f013905c700 1 -- start start 2020-12-05T17:31:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f013905c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013405f3b0 0x7f0134151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f013905c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f013405feb0 0x7f0134065310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f013905c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0134062400 0x7f0134140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:50.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f013905c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013413eeb0 con 0x7f013405f3b0 2020-12-05T17:31:50.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f013905c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0134063550 con 0x7f013405feb0 2020-12-05T17:31:50.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f013905c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01340637f0 con 0x7f0134062400 2020-12-05T17:31:50.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f0133fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013405f3b0 0x7f0134151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:50.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f0133fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013405f3b0 0x7f0134151350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32838/0 (socket says 172.21.15.32:32838) 2020-12-05T17:31:50.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f0133fff700 1 -- 172.21.15.32:0/3152720776 learned_addr learned my addr 172.21.15.32:0/3152720776 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:50.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f01337fe700 1 --2- 172.21.15.32:0/3152720776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f013405feb0 0x7f0134065310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:50.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f013885b700 1 --2- 172.21.15.32:0/3152720776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0134062400 0x7f0134140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:50.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f0133fff700 1 -- 172.21.15.32:0/3152720776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0134062400 msgr2=0x7f0134140ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:50.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f0133fff700 1 --2- 172.21.15.32:0/3152720776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0134062400 0x7f0134140ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f0133fff700 1 -- 172.21.15.32:0/3152720776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f013405feb0 msgr2=0x7f0134065310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f0133fff700 1 --2- 172.21.15.32:0/3152720776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f013405feb0 0x7f0134065310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f0133fff700 1 -- 172.21.15.32:0/3152720776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0134063040 con 0x7f013405f3b0 2020-12-05T17:31:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f0133fff700 1 --2- 172.21.15.32:0/3152720776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013405f3b0 0x7f0134151350 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f01240032b0 tx=0x7f0124016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f01317fa700 1 -- 172.21.15.32:0/3152720776 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f012401f5e0 con 0x7f013405f3b0 2020-12-05T17:31:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f01317fa700 1 -- 172.21.15.32:0/3152720776 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012401f740 con 0x7f013405f3b0 2020-12-05T17:31:50.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f013905c700 1 -- 172.21.15.32:0/3152720776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01340632f0 con 0x7f013405f3b0 2020-12-05T17:31:50.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f01317fa700 1 -- 172.21.15.32:0/3152720776 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f012401faa0 con 0x7f013405f3b0 2020-12-05T17:31:50.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.199+0000 7f013905c700 1 -- 172.21.15.32:0/3152720776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0134151d60 con 0x7f013405f3b0 2020-12-05T17:31:50.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.203+0000 7f013905c700 1 -- 172.21.15.32:0/3152720776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01340639b0 con 0x7f013405f3b0 2020-12-05T17:31:50.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.203+0000 7f01317fa700 1 -- 172.21.15.32:0/3152720776 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f012400f070 con 0x7f013405f3b0 2020-12-05T17:31:50.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.203+0000 7f01317fa700 1 --2- 172.21.15.32:0/3152720776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f010c055440 0x7f010c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:50.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.203+0000 7f01317fa700 1 -- 172.21.15.32:0/3152720776 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f012406f010 con 0x7f013405f3b0 2020-12-05T17:31:50.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.203+0000 7f01337fe700 1 --2- 172.21.15.32:0/3152720776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f010c055440 0x7f010c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:50.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.203+0000 7f01337fe700 1 --2- 172.21.15.32:0/3152720776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f010c055440 0x7f010c057900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f011c00c000 tx=0x7f011c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:50.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.211+0000 7f01317fa700 1 -- 172.21.15.32:0/3152720776 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f01240451b0 con 0x7f013405f3b0 2020-12-05T17:31:50.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.351+0000 7f013905c700 1 -- 172.21.15.32:0/3152720776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0134057050 con 0x7f013405f3b0 2020-12-05T17:31:50.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.355+0000 7f01317fa700 1 -- 172.21.15.32:0/3152720776 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0124015090 con 0x7f013405f3b0 2020-12-05T17:31:50.358 INFO:teuthology.orchestra.run.smithi032.stdout:60129542150 2020-12-05T17:31:50.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.355+0000 7f013905c700 1 -- 172.21.15.32:0/3152720776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f010c055440 msgr2=0x7f010c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:50.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.355+0000 7f013905c700 1 --2- 172.21.15.32:0/3152720776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f010c055440 0x7f010c057900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f011c00c000 tx=0x7f011c008040).stop 2020-12-05T17:31:50.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.355+0000 7f013905c700 1 -- 172.21.15.32:0/3152720776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013405f3b0 msgr2=0x7f0134151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:50.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.355+0000 7f013905c700 1 --2- 172.21.15.32:0/3152720776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013405f3b0 0x7f0134151350 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f01240032b0 tx=0x7f0124016000).stop 2020-12-05T17:31:50.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.355+0000 7f013905c700 1 -- 172.21.15.32:0/3152720776 shutdown_connections 2020-12-05T17:31:50.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.355+0000 7f013905c700 1 --2- 172.21.15.32:0/3152720776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f010c055440 0x7f010c057900 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.355+0000 7f013905c700 1 --2- 172.21.15.32:0/3152720776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0134062400 0x7f0134140ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.355+0000 7f013905c700 1 --2- 172.21.15.32:0/3152720776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013405f3b0 0x7f0134151350 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.355+0000 7f013905c700 1 --2- 172.21.15.32:0/3152720776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f013405feb0 0x7f0134065310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.355+0000 7f013905c700 1 -- 172.21.15.32:0/3152720776 >> 172.21.15.32:0/3152720776 conn(0x7f01340b8ee0 msgr2=0x7f01340b7580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:50.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.355+0000 7f013905c700 1 -- 172.21.15.32:0/3152720776 shutdown_connections 2020-12-05T17:31:50.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.355+0000 7f013905c700 1 -- 172.21.15.32:0/3152720776 wait complete. 2020-12-05T17:31:50.371 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.6 2020-12-05T17:31:50.372 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.7 2020-12-05T17:31:50.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb887fff700 1 Processor -- start 2020-12-05T17:31:50.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb887fff700 1 -- start start 2020-12-05T17:31:50.503 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb887fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8880b1fc0 0x7fb8880af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:50.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb887fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8880ab710 0x7fb888061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:50.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb887fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb888147400 0x7fb888145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:50.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb887fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8880b0110 con 0x7fb8880b1fc0 2020-12-05T17:31:50.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb887fff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb888146a90 con 0x7fb888147400 2020-12-05T17:31:50.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb887fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880b23a0 con 0x7fb8880ab710 2020-12-05T17:31:50.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb886ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8880b1fc0 0x7fb8880af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:50.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb8867fc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8880ab710 0x7fb888061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:50.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb8867fc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8880ab710 0x7fb888061780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:57868/0 (socket says 172.21.15.32:57868) 2020-12-05T17:31:50.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb8867fc700 1 -- 172.21.15.32:0/2507224016 learned_addr learned my addr 172.21.15.32:0/2507224016 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:50.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb8877fe700 1 --2- 172.21.15.32:0/2507224016 >> v2:172.21.15.61:3300/0 conn(0x7fb888147400 0x7fb888145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:50.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb8867fc700 1 -- 172.21.15.32:0/2507224016 >> v2:172.21.15.61:3300/0 conn(0x7fb888147400 msgr2=0x7fb888145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:50.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb8867fc700 1 --2- 172.21.15.32:0/2507224016 >> v2:172.21.15.61:3300/0 conn(0x7fb888147400 0x7fb888145d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb8867fc700 1 -- 172.21.15.32:0/2507224016 >> v2:172.21.15.32:3300/0 conn(0x7fb8880b1fc0 msgr2=0x7fb8880af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:50.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb8867fc700 1 --2- 172.21.15.32:0/2507224016 >> v2:172.21.15.32:3300/0 conn(0x7fb8880b1fc0 0x7fb8880af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb8867fc700 1 -- 172.21.15.32:0/2507224016 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb888146cc0 con 0x7fb8880ab710 2020-12-05T17:31:50.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb8867fc700 1 --2- 172.21.15.32:0/2507224016 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8880ab710 0x7fb888061780 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fb8780032b0 tx=0x7fb878009270).ready entity=mon.2 client_cookie=7f7c6168ffceae5d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:50.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb885ffb700 1 -- 172.21.15.32:0/2507224016 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb878009e60 con 0x7fb8880ab710 2020-12-05T17:31:50.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb885ffb700 1 -- 172.21.15.32:0/2507224016 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87800bb50 con 0x7fb8880ab710 2020-12-05T17:31:50.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb885ffb700 1 -- 172.21.15.32:0/2507224016 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb87800bcc0 con 0x7fb8880ab710 2020-12-05T17:31:50.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb887fff700 1 -- 172.21.15.32:0/2507224016 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8880ab710 msgr2=0x7fb888061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:50.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb887fff700 1 --2- 172.21.15.32:0/2507224016 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8880ab710 0x7fb888061780 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fb8780032b0 tx=0x7fb878009270).stop 2020-12-05T17:31:50.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb887fff700 1 -- 172.21.15.32:0/2507224016 shutdown_connections 2020-12-05T17:31:50.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb887fff700 1 --2- 172.21.15.32:0/2507224016 >> v2:172.21.15.61:3300/0 conn(0x7fb888147400 0x7fb888145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb887fff700 1 --2- 172.21.15.32:0/2507224016 >> v2:172.21.15.32:3300/0 conn(0x7fb8880b1fc0 0x7fb8880af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb887fff700 1 --2- 172.21.15.32:0/2507224016 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8880ab710 0x7fb888061780 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb887fff700 1 -- 172.21.15.32:0/2507224016 >> 172.21.15.32:0/2507224016 conn(0x7fb8880b5530 msgr2=0x7fb8880b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:50.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb887fff700 1 -- 172.21.15.32:0/2507224016 shutdown_connections 2020-12-05T17:31:50.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.491+0000 7fb887fff700 1 -- 172.21.15.32:0/2507224016 wait complete. 2020-12-05T17:31:50.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb887fff700 1 Processor -- start 2020-12-05T17:31:50.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb887fff700 1 -- start start 2020-12-05T17:31:50.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb887fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8880ab710 0x7fb88814dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:50.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb887fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8880b1fc0 0x7fb88814b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:50.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb887fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb888147400 0x7fb8880b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:50.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb887fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880ad230 con 0x7fb8880b1fc0 2020-12-05T17:31:50.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb887fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880ac8a0 con 0x7fb888147400 2020-12-05T17:31:50.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb887fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb88814ce00 con 0x7fb8880ab710 2020-12-05T17:31:50.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb8867fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8880b1fc0 0x7fb88814b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:50.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb8867fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8880b1fc0 0x7fb88814b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32856/0 (socket says 172.21.15.32:32856) 2020-12-05T17:31:50.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb8867fc700 1 -- 172.21.15.32:0/1817481196 learned_addr learned my addr 172.21.15.32:0/1817481196 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:50.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb8877fe700 1 --2- 172.21.15.32:0/1817481196 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb888147400 0x7fb8880b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:50.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb886ffd700 1 --2- 172.21.15.32:0/1817481196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8880ab710 0x7fb88814dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:50.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb8867fc700 1 -- 172.21.15.32:0/1817481196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8880ab710 msgr2=0x7fb88814dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:50.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb8867fc700 1 --2- 172.21.15.32:0/1817481196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8880ab710 0x7fb88814dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb8867fc700 1 -- 172.21.15.32:0/1817481196 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb888147400 msgr2=0x7fb8880b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:50.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb8867fc700 1 --2- 172.21.15.32:0/1817481196 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb888147400 0x7fb8880b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb8867fc700 1 -- 172.21.15.32:0/1817481196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88814d0a0 con 0x7fb8880b1fc0 2020-12-05T17:31:50.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb8867fc700 1 --2- 172.21.15.32:0/1817481196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8880b1fc0 0x7fb88814b0b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fb8780032b0 tx=0x7fb878016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:50.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb87ffff700 1 -- 172.21.15.32:0/1817481196 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8780107a0 con 0x7fb8880b1fc0 2020-12-05T17:31:50.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb87ffff700 1 -- 172.21.15.32:0/1817481196 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb878010900 con 0x7fb8880b1fc0 2020-12-05T17:31:50.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb887fff700 1 -- 172.21.15.32:0/1817481196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88814c8c0 con 0x7fb8880b1fc0 2020-12-05T17:31:50.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb87ffff700 1 -- 172.21.15.32:0/1817481196 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb878010c60 con 0x7fb8880b1fc0 2020-12-05T17:31:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.495+0000 7fb887fff700 1 -- 172.21.15.32:0/1817481196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88814cbd0 con 0x7fb8880b1fc0 2020-12-05T17:31:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.499+0000 7fb887fff700 1 -- 172.21.15.32:0/1817481196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb888057050 con 0x7fb8880b1fc0 2020-12-05T17:31:50.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.499+0000 7fb87ffff700 1 -- 172.21.15.32:0/1817481196 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb878026030 con 0x7fb8880b1fc0 2020-12-05T17:31:50.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.499+0000 7fb87ffff700 1 --2- 172.21.15.32:0/1817481196 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb86c055330 0x7fb86c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:50.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.499+0000 7fb886ffd700 1 --2- 172.21.15.32:0/1817481196 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb86c055330 0x7fb86c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:50.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.499+0000 7fb87ffff700 1 -- 172.21.15.32:0/1817481196 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb87806f180 con 0x7fb8880b1fc0 2020-12-05T17:31:50.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.499+0000 7fb886ffd700 1 --2- 172.21.15.32:0/1817481196 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb86c055330 0x7fb86c0577f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fb87000b000 tx=0x7fb870006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:50.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.503+0000 7fb87ffff700 1 -- 172.21.15.32:0/1817481196 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb878045320 con 0x7fb8880b1fc0 2020-12-05T17:31:50.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.647+0000 7fb887fff700 1 -- 172.21.15.32:0/1817481196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb88814d230 con 0x7fb8880b1fc0 2020-12-05T17:31:50.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.647+0000 7fb87ffff700 1 -- 172.21.15.32:0/1817481196 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb878024020 con 0x7fb8880b1fc0 2020-12-05T17:31:50.652 INFO:teuthology.orchestra.run.smithi032.stdout:60129542149 2020-12-05T17:31:50.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.651+0000 7fb887fff700 1 -- 172.21.15.32:0/1817481196 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb86c055330 msgr2=0x7fb86c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:50.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.651+0000 7fb887fff700 1 --2- 172.21.15.32:0/1817481196 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb86c055330 0x7fb86c0577f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fb87000b000 tx=0x7fb870006dc0).stop 2020-12-05T17:31:50.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.651+0000 7fb887fff700 1 -- 172.21.15.32:0/1817481196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8880b1fc0 msgr2=0x7fb88814b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:50.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.651+0000 7fb887fff700 1 --2- 172.21.15.32:0/1817481196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8880b1fc0 0x7fb88814b0b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fb8780032b0 tx=0x7fb878016000).stop 2020-12-05T17:31:50.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.651+0000 7fb887fff700 1 -- 172.21.15.32:0/1817481196 shutdown_connections 2020-12-05T17:31:50.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.651+0000 7fb887fff700 1 --2- 172.21.15.32:0/1817481196 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb888147400 0x7fb8880b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.651+0000 7fb887fff700 1 --2- 172.21.15.32:0/1817481196 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb86c055330 0x7fb86c0577f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.651+0000 7fb887fff700 1 --2- 172.21.15.32:0/1817481196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8880ab710 0x7fb88814dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.651+0000 7fb887fff700 1 --2- 172.21.15.32:0/1817481196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8880b1fc0 0x7fb88814b0b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.651+0000 7fb887fff700 1 -- 172.21.15.32:0/1817481196 >> 172.21.15.32:0/1817481196 conn(0x7fb8880b5530 msgr2=0x7fb888144a70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:50.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.651+0000 7fb887fff700 1 -- 172.21.15.32:0/1817481196 shutdown_connections 2020-12-05T17:31:50.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.651+0000 7fb887fff700 1 -- 172.21.15.32:0/1817481196 wait complete. 2020-12-05T17:31:50.670 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.7 2020-12-05T17:31:50.671 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2020-12-05T17:31:50.671 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-12-05T17:31:50.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.783+0000 7f214d423700 1 Processor -- start 2020-12-05T17:31:50.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.783+0000 7f214d423700 1 -- start start 2020-12-05T17:31:50.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.783+0000 7f214d423700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f21480b6080 0x7f2148142c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:50.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.783+0000 7f214d423700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f21480ad1b0 0x7f2148143130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:50.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.783+0000 7f214d423700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21480ae8b0 con 0x7f2148143620 2020-12-05T17:31:50.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.783+0000 7f214d423700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21480afcf0 con 0x7f21480ad1b0 2020-12-05T17:31:50.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.783+0000 7f214d423700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21480b0da0 con 0x7f21480b6080 2020-12-05T17:31:50.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.783+0000 7f21477fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2148143620 0x7f214813e740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35510/0 (socket says 172.21.15.32:35510) 2020-12-05T17:31:50.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.783+0000 7f21467fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f21480ad1b0 0x7f2148143130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:50.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.783+0000 7f21477fe700 1 -- 172.21.15.32:0/2480701972 learned_addr learned my addr 172.21.15.32:0/2480701972 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:50.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.783+0000 7f2146ffd700 1 --2- 172.21.15.32:0/2480701972 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f21480b6080 0x7f2148142c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:50.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.783+0000 7f2145ffb700 1 -- 172.21.15.32:0/2480701972 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3773750658 0 0) 0x7f21480ae8b0 con 0x7f2148143620 2020-12-05T17:31:50.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.783+0000 7f2145ffb700 1 -- 172.21.15.32:0/2480701972 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f212c003230 con 0x7f2148143620 2020-12-05T17:31:50.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.783+0000 7f21467fc700 1 -- 172.21.15.32:0/2480701972 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f21480b6080 msgr2=0x7f2148142c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:50.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.783+0000 7f21467fc700 1 --2- 172.21.15.32:0/2480701972 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f21480b6080 0x7f2148142c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.783+0000 7f21467fc700 1 -- 172.21.15.32:0/2480701972 >> v1:172.21.15.32:6789/0 conn(0x7f2148143620 legacy=0x7f214813e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:50.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.783+0000 7f21467fc700 1 -- 172.21.15.32:0/2480701972 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21480b6460 con 0x7f21480ad1b0 2020-12-05T17:31:50.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.783+0000 7f21467fc700 1 --2- 172.21.15.32:0/2480701972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21480ad1b0 0x7f2148143130 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f213c002f30 tx=0x7f213c006dc0).ready entity=mon.0 client_cookie=7d8db2b2cd6909c6 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:50.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.787+0000 7f2145ffb700 1 -- 172.21.15.32:0/2480701972 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f213c0096a0 con 0x7f21480ad1b0 2020-12-05T17:31:50.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.787+0000 7f2145ffb700 1 -- 172.21.15.32:0/2480701972 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c009800 con 0x7f21480ad1b0 2020-12-05T17:31:50.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.787+0000 7f2145ffb700 1 -- 172.21.15.32:0/2480701972 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f213c009b60 con 0x7f21480ad1b0 2020-12-05T17:31:50.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.787+0000 7f214d423700 1 -- 172.21.15.32:0/2480701972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21480ad1b0 msgr2=0x7f2148143130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:50.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.787+0000 7f214d423700 1 --2- 172.21.15.32:0/2480701972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21480ad1b0 0x7f2148143130 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f213c002f30 tx=0x7f213c006dc0).stop 2020-12-05T17:31:50.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.787+0000 7f214d423700 1 -- 172.21.15.32:0/2480701972 shutdown_connections 2020-12-05T17:31:50.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.787+0000 7f214d423700 1 --2- 172.21.15.32:0/2480701972 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f21480b6080 0x7f2148142c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.787+0000 7f214d423700 1 --2- 172.21.15.32:0/2480701972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21480ad1b0 0x7f2148143130 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.787+0000 7f214d423700 1 -- 172.21.15.32:0/2480701972 >> 172.21.15.32:0/2480701972 conn(0x7f2148069cb0 msgr2=0x7f21480604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:50.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.787+0000 7f214d423700 1 -- 172.21.15.32:0/2480701972 shutdown_connections 2020-12-05T17:31:50.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.787+0000 7f214d423700 1 -- 172.21.15.32:0/2480701972 wait complete. 2020-12-05T17:31:50.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.787+0000 7f214d423700 1 Processor -- start 2020-12-05T17:31:50.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.787+0000 7f214d423700 1 -- start start 2020-12-05T17:31:50.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.787+0000 7f214d423700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21480ad1b0 0x7f2148150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:50.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.787+0000 7f214d423700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f21480b6080 0x7f214805f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:50.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.787+0000 7f214d423700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2148143620 0x7f2148144b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:50.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.787+0000 7f214d423700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2148153590 con 0x7f21480ad1b0 2020-12-05T17:31:50.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.787+0000 7f214d423700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2148144570 con 0x7f2148143620 2020-12-05T17:31:50.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.787+0000 7f214d423700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21481537c0 con 0x7f21480b6080 2020-12-05T17:31:50.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.787+0000 7f2146ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21480ad1b0 0x7f2148150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:50.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.787+0000 7f2146ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21480ad1b0 0x7f2148150a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32866/0 (socket says 172.21.15.32:32866) 2020-12-05T17:31:50.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.787+0000 7f2146ffd700 1 -- 172.21.15.32:0/524930442 learned_addr learned my addr 172.21.15.32:0/524930442 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:50.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.791+0000 7f21477fe700 1 --2- 172.21.15.32:0/524930442 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2148143620 0x7f2148144b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:50.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.791+0000 7f21467fc700 1 --2- 172.21.15.32:0/524930442 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f21480b6080 0x7f214805f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:50.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.791+0000 7f2146ffd700 1 -- 172.21.15.32:0/524930442 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f21480b6080 msgr2=0x7f214805f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:50.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.791+0000 7f2146ffd700 1 --2- 172.21.15.32:0/524930442 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f21480b6080 0x7f214805f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.791+0000 7f2146ffd700 1 -- 172.21.15.32:0/524930442 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2148143620 msgr2=0x7f2148144b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:50.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.791+0000 7f2146ffd700 1 --2- 172.21.15.32:0/524930442 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2148143620 0x7f2148144b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.791+0000 7f2146ffd700 1 -- 172.21.15.32:0/524930442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21480654f0 con 0x7f21480ad1b0 2020-12-05T17:31:50.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.791+0000 7f21467fc700 1 --2- 172.21.15.32:0/524930442 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f21480b6080 0x7f214805f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:50.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.791+0000 7f21477fe700 1 --2- 172.21.15.32:0/524930442 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2148143620 0x7f2148144b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:50.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.791+0000 7f2146ffd700 1 --2- 172.21.15.32:0/524930442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21480ad1b0 0x7f2148150a80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f213800c000 tx=0x7f2138006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:50.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.791+0000 7f2137fff700 1 -- 172.21.15.32:0/524930442 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f213800ca80 con 0x7f21480ad1b0 2020-12-05T17:31:50.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.791+0000 7f214d423700 1 -- 172.21.15.32:0/524930442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2148065fd0 con 0x7f21480ad1b0 2020-12-05T17:31:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.791+0000 7f2137fff700 1 -- 172.21.15.32:0/524930442 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213800cbe0 con 0x7f21480ad1b0 2020-12-05T17:31:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.791+0000 7f2137fff700 1 -- 172.21.15.32:0/524930442 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21380123f0 con 0x7f21480ad1b0 2020-12-05T17:31:50.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.791+0000 7f214d423700 1 -- 172.21.15.32:0/524930442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2148066570 con 0x7f21480ad1b0 2020-12-05T17:31:50.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.791+0000 7f214d423700 1 -- 172.21.15.32:0/524930442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2148057050 con 0x7f21480ad1b0 2020-12-05T17:31:50.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.795+0000 7f2137fff700 1 -- 172.21.15.32:0/524930442 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2138012550 con 0x7f21480ad1b0 2020-12-05T17:31:50.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.795+0000 7f2137fff700 1 --2- 172.21.15.32:0/524930442 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2124055500 0x7f21240579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:50.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.795+0000 7f2137fff700 1 -- 172.21.15.32:0/524930442 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f213806f430 con 0x7f21480ad1b0 2020-12-05T17:31:50.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.795+0000 7f21467fc700 1 --2- 172.21.15.32:0/524930442 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2124055500 0x7f21240579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:50.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.795+0000 7f21467fc700 1 --2- 172.21.15.32:0/524930442 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2124055500 0x7f21240579c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f213c019000 tx=0x7f213c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:50.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.799+0000 7f2137fff700 1 -- 172.21.15.32:0/524930442 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2138045460 con 0x7f21480ad1b0 2020-12-05T17:31:50.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.935+0000 7f214d423700 1 -- 172.21.15.32:0/524930442 --> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f21481443f0 con 0x7f2124055500 2020-12-05T17:31:50.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.943+0000 7f2137fff700 1 -- 172.21.15.32:0/524930442 <== mgr.4101 v2:172.21.15.32:6800/11355 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82393 (secure 0 0 0) 0x7f213c009e60 con 0x7f2124055500 2020-12-05T17:31:50.949 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:31:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.947+0000 7f214d423700 1 -- 172.21.15.32:0/524930442 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2124055500 msgr2=0x7f21240579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.947+0000 7f214d423700 1 --2- 172.21.15.32:0/524930442 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2124055500 0x7f21240579c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f213c019000 tx=0x7f213c015000).stop 2020-12-05T17:31:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.947+0000 7f214d423700 1 -- 172.21.15.32:0/524930442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21480ad1b0 msgr2=0x7f2148150a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:50.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.947+0000 7f214d423700 1 --2- 172.21.15.32:0/524930442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21480ad1b0 0x7f2148150a80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f213800c000 tx=0x7f2138006dc0).stop 2020-12-05T17:31:50.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.947+0000 7f214d423700 1 -- 172.21.15.32:0/524930442 shutdown_connections 2020-12-05T17:31:50.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.947+0000 7f214d423700 1 --2- 172.21.15.32:0/524930442 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2148143620 0x7f2148144b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.947+0000 7f214d423700 1 --2- 172.21.15.32:0/524930442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21480ad1b0 0x7f2148150a80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.947+0000 7f214d423700 1 --2- 172.21.15.32:0/524930442 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2124055500 0x7f21240579c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.947+0000 7f214d423700 1 --2- 172.21.15.32:0/524930442 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f21480b6080 0x7f214805f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:50.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.947+0000 7f214d423700 1 -- 172.21.15.32:0/524930442 >> 172.21.15.32:0/524930442 conn(0x7f2148069cb0 msgr2=0x7f214805ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:50.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.947+0000 7f214d423700 1 -- 172.21.15.32:0/524930442 shutdown_connections 2020-12-05T17:31:50.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:50.947+0000 7f214d423700 1 -- 172.21.15.32:0/524930442 wait complete. 2020-12-05T17:31:50.954 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2020-12-05T17:31:50.963 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2020-12-05T17:31:50.801342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390784,"kb_used_data":2112,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746583936,"statfs":{"total":773094113280,"available":764501950464,"internally_reserved":8589934592,"allocated":2162688,"data_stored":828856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.536458"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.295094+0000","last_change":"2020-12-05T17:31:41.293987+0000","last_active":"2020-12-05T17:31:41.295094+0000","last_peered":"2020-12-05T17:31:41.295094+0000","last_clean":"2020-12-05T17:31:41.295094+0000","last_became_active":"2020-12-05T17:31:41.293037+0000","last_became_peered":"2020-12-05T17:31:41.293037+0000","last_unstale":"2020-12-05T17:31:41.295094+0000","last_undegraded":"2020-12-05T17:31:41.295094+0000","last_fullsized":"2020-12-05T17:31:41.295094+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_clean_scrub_stamp":"2020-12-05T17:31:40.257305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"14","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.269142+0000","last_change":"2020-12-05T17:31:35.374114+0000","last_active":"2020-12-05T17:31:41.269142+0000","last_peered":"2020-12-05T17:31:41.269142+0000","last_clean":"2020-12-05T17:31:41.269142+0000","last_became_active":"2020-12-05T17:31:35.373231+0000","last_became_peered":"2020-12-05T17:31:35.373231+0000","last_unstale":"2020-12-05T17:31:41.269142+0000","last_undegraded":"2020-12-05T17:31:41.269142+0000","last_fullsized":"2020-12-05T17:31:41.269142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"25'6","reported_seq":"18","reported_epoch":"25","state":"active+clean","last_fresh":"2020-12-05T17:31:41.641581+0000","last_change":"2020-12-05T17:31:38.264052+0000","last_active":"2020-12-05T17:31:41.641581+0000","last_peered":"2020-12-05T17:31:41.641581+0000","last_clean":"2020-12-05T17:31:41.641581+0000","last_became_active":"2020-12-05T17:31:38.263431+0000","last_became_peered":"2020-12-05T17:31:38.263431+0000","last_unstale":"2020-12-05T17:31:41.641581+0000","last_undegraded":"2020-12-05T17:31:41.641581+0000","last_fullsized":"2020-12-05T17:31:41.641581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_clean_scrub_stamp":"2020-12-05T17:31:37.227982+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.289851+0000","last_change":"2020-12-05T17:31:41.288122+0000","last_active":"2020-12-05T17:31:41.289851+0000","last_peered":"2020-12-05T17:31:41.289851+0000","last_clean":"2020-12-05T17:31:41.289851+0000","last_became_active":"2020-12-05T17:31:41.287342+0000","last_became_peered":"2020-12-05T17:31:41.287342+0000","last_unstale":"2020-12-05T17:31:41.289851+0000","last_undegraded":"2020-12-05T17:31:41.289851+0000","last_fullsized":"2020-12-05T17:31:41.289851+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_clean_scrub_stamp":"2020-12-05T17:31:40.257305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.291220+0000","last_change":"2020-12-05T17:31:35.123847+0000","last_active":"2020-12-05T17:31:41.291220+0000","last_peered":"2020-12-05T17:31:41.291220+0000","last_clean":"2020-12-05T17:31:41.291220+0000","last_became_active":"2020-12-05T17:31:35.123169+0000","last_became_peered":"2020-12-05T17:31:35.123169+0000","last_unstale":"2020-12-05T17:31:41.291220+0000","last_undegraded":"2020-12-05T17:31:41.291220+0000","last_fullsized":"2020-12-05T17:31:41.291220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"25'5","reported_seq":"17","reported_epoch":"25","state":"active+clean","last_fresh":"2020-12-05T17:31:41.638742+0000","last_change":"2020-12-05T17:31:38.263937+0000","last_active":"2020-12-05T17:31:41.638742+0000","last_peered":"2020-12-05T17:31:41.638742+0000","last_clean":"2020-12-05T17:31:41.638742+0000","last_became_active":"2020-12-05T17:31:38.263349+0000","last_became_peered":"2020-12-05T17:31:38.263349+0000","last_unstale":"2020-12-05T17:31:41.638742+0000","last_undegraded":"2020-12-05T17:31:41.638742+0000","last_fullsized":"2020-12-05T17:31:41.638742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_clean_scrub_stamp":"2020-12-05T17:31:37.227982+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.295098+0000","last_change":"2020-12-05T17:31:41.293648+0000","last_active":"2020-12-05T17:31:41.295098+0000","last_peered":"2020-12-05T17:31:41.295098+0000","last_clean":"2020-12-05T17:31:41.295098+0000","last_became_active":"2020-12-05T17:31:41.292941+0000","last_became_peered":"2020-12-05T17:31:41.292941+0000","last_unstale":"2020-12-05T17:31:41.295098+0000","last_undegraded":"2020-12-05T17:31:41.295098+0000","last_fullsized":"2020-12-05T17:31:41.295098+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_clean_scrub_stamp":"2020-12-05T17:31:40.257305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.285861+0000","last_change":"2020-12-05T17:31:35.617802+0000","last_active":"2020-12-05T17:31:41.285861+0000","last_peered":"2020-12-05T17:31:41.285861+0000","last_clean":"2020-12-05T17:31:41.285861+0000","last_became_active":"2020-12-05T17:31:35.617427+0000","last_became_peered":"2020-12-05T17:31:35.617427+0000","last_unstale":"2020-12-05T17:31:41.285861+0000","last_undegraded":"2020-12-05T17:31:41.285861+0000","last_fullsized":"2020-12-05T17:31:41.285861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"25'8","reported_seq":"20","reported_epoch":"25","state":"active+clean","last_fresh":"2020-12-05T17:31:41.641591+0000","last_change":"2020-12-05T17:31:38.263384+0000","last_active":"2020-12-05T17:31:41.641591+0000","last_peered":"2020-12-05T17:31:41.641591+0000","last_clean":"2020-12-05T17:31:41.641591+0000","last_became_active":"2020-12-05T17:31:38.262951+0000","last_became_peered":"2020-12-05T17:31:38.262951+0000","last_unstale":"2020-12-05T17:31:41.641591+0000","last_undegraded":"2020-12-05T17:31:41.641591+0000","last_fullsized":"2020-12-05T17:31:41.641591+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_clean_scrub_stamp":"2020-12-05T17:31:37.227982+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.287696+0000","last_change":"2020-12-05T17:31:41.283820+0000","last_active":"2020-12-05T17:31:41.287696+0000","last_peered":"2020-12-05T17:31:41.287696+0000","last_clean":"2020-12-05T17:31:41.287696+0000","last_became_active":"2020-12-05T17:31:41.283290+0000","last_became_peered":"2020-12-05T17:31:41.283290+0000","last_unstale":"2020-12-05T17:31:41.287696+0000","last_undegraded":"2020-12-05T17:31:41.287696+0000","last_fullsized":"2020-12-05T17:31:41.287696+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_clean_scrub_stamp":"2020-12-05T17:31:40.257305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.291418+0000","last_change":"2020-12-05T17:31:35.124252+0000","last_active":"2020-12-05T17:31:41.291418+0000","last_peered":"2020-12-05T17:31:41.291418+0000","last_clean":"2020-12-05T17:31:41.291418+0000","last_became_active":"2020-12-05T17:31:35.123671+0000","last_became_peered":"2020-12-05T17:31:35.123671+0000","last_unstale":"2020-12-05T17:31:41.291418+0000","last_undegraded":"2020-12-05T17:31:41.291418+0000","last_fullsized":"2020-12-05T17:31:41.291418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"19","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.271067+0000","last_change":"2020-12-05T17:31:30.406003+0000","last_active":"2020-12-05T17:31:41.271067+0000","last_peered":"2020-12-05T17:31:41.271067+0000","last_clean":"2020-12-05T17:31:41.271067+0000","last_became_active":"2020-12-05T17:31:30.405245+0000","last_became_peered":"2020-12-05T17:31:30.405245+0000","last_unstale":"2020-12-05T17:31:41.271067+0000","last_undegraded":"2020-12-05T17:31:41.271067+0000","last_fullsized":"2020-12-05T17:31:41.271067+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:25.806577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:25.806577+0000","last_clean_scrub_stamp":"2020-12-05T17:31:25.806577+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"25'1","reported_seq":"13","reported_epoch":"25","state":"active+clean","last_fresh":"2020-12-05T17:31:41.618588+0000","last_change":"2020-12-05T17:31:38.260319+0000","last_active":"2020-12-05T17:31:41.618588+0000","last_peered":"2020-12-05T17:31:41.618588+0000","last_clean":"2020-12-05T17:31:41.618588+0000","last_became_active":"2020-12-05T17:31:38.259830+0000","last_became_peered":"2020-12-05T17:31:38.259830+0000","last_unstale":"2020-12-05T17:31:41.618588+0000","last_undegraded":"2020-12-05T17:31:41.618588+0000","last_fullsized":"2020-12-05T17:31:41.618588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_clean_scrub_stamp":"2020-12-05T17:31:37.227982+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.291959+0000","last_change":"2020-12-05T17:31:41.290757+0000","last_active":"2020-12-05T17:31:41.291959+0000","last_peered":"2020-12-05T17:31:41.291959+0000","last_clean":"2020-12-05T17:31:41.291959+0000","last_became_active":"2020-12-05T17:31:41.290144+0000","last_became_peered":"2020-12-05T17:31:41.290144+0000","last_unstale":"2020-12-05T17:31:41.291959+0000","last_undegraded":"2020-12-05T17:31:41.291959+0000","last_fullsized":"2020-12-05T17:31:41.291959+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_clean_scrub_stamp":"2020-12-05T17:31:40.257305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"14","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.271329+0000","last_change":"2020-12-05T17:31:35.118985+0000","last_active":"2020-12-05T17:31:41.271329+0000","last_peered":"2020-12-05T17:31:41.271329+0000","last_clean":"2020-12-05T17:31:41.271329+0000","last_became_active":"2020-12-05T17:31:35.118441+0000","last_became_peered":"2020-12-05T17:31:35.118441+0000","last_unstale":"2020-12-05T17:31:41.271329+0000","last_undegraded":"2020-12-05T17:31:41.271329+0000","last_fullsized":"2020-12-05T17:31:41.271329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.275676+0000","last_change":"2020-12-05T17:31:38.265560+0000","last_active":"2020-12-05T17:31:41.275676+0000","last_peered":"2020-12-05T17:31:41.275676+0000","last_clean":"2020-12-05T17:31:41.275676+0000","last_became_active":"2020-12-05T17:31:38.265128+0000","last_became_peered":"2020-12-05T17:31:38.265128+0000","last_unstale":"2020-12-05T17:31:41.275676+0000","last_undegraded":"2020-12-05T17:31:41.275676+0000","last_fullsized":"2020-12-05T17:31:41.275676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_clean_scrub_stamp":"2020-12-05T17:31:37.227982+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.294841+0000","last_change":"2020-12-05T17:31:41.293913+0000","last_active":"2020-12-05T17:31:41.294841+0000","last_peered":"2020-12-05T17:31:41.294841+0000","last_clean":"2020-12-05T17:31:41.294841+0000","last_became_active":"2020-12-05T17:31:41.293297+0000","last_became_peered":"2020-12-05T17:31:41.293297+0000","last_unstale":"2020-12-05T17:31:41.294841+0000","last_undegraded":"2020-12-05T17:31:41.294841+0000","last_fullsized":"2020-12-05T17:31:41.294841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_clean_scrub_stamp":"2020-12-05T17:31:40.257305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"14","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.271221+0000","last_change":"2020-12-05T17:31:35.375163+0000","last_active":"2020-12-05T17:31:41.271221+0000","last_peered":"2020-12-05T17:31:41.271221+0000","last_clean":"2020-12-05T17:31:41.271221+0000","last_became_active":"2020-12-05T17:31:35.374461+0000","last_became_peered":"2020-12-05T17:31:35.374461+0000","last_unstale":"2020-12-05T17:31:41.271221+0000","last_undegraded":"2020-12-05T17:31:41.271221+0000","last_fullsized":"2020-12-05T17:31:41.271221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"25'8","reported_seq":"20","reported_epoch":"25","state":"active+clean","last_fresh":"2020-12-05T17:31:41.640996+0000","last_change":"2020-12-05T17:31:38.262338+0000","last_active":"2020-12-05T17:31:41.640996+0000","last_peered":"2020-12-05T17:31:41.640996+0000","last_clean":"2020-12-05T17:31:41.640996+0000","last_became_active":"2020-12-05T17:31:38.261896+0000","last_became_peered":"2020-12-05T17:31:38.261896+0000","last_unstale":"2020-12-05T17:31:41.640996+0000","last_undegraded":"2020-12-05T17:31:41.640996+0000","last_fullsized":"2020-12-05T17:31:41.640996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_clean_scrub_stamp":"2020-12-05T17:31:37.227982+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.288807+0000","last_change":"2020-12-05T17:31:41.287845+0000","last_active":"2020-12-05T17:31:41.288807+0000","last_peered":"2020-12-05T17:31:41.288807+0000","last_clean":"2020-12-05T17:31:41.288807+0000","last_became_active":"2020-12-05T17:31:41.287109+0000","last_became_peered":"2020-12-05T17:31:41.287109+0000","last_unstale":"2020-12-05T17:31:41.288807+0000","last_undegraded":"2020-12-05T17:31:41.288807+0000","last_fullsized":"2020-12-05T17:31:41.288807+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_clean_scrub_stamp":"2020-12-05T17:31:40.257305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"25'7","reported_seq":"19","reported_epoch":"25","state":"active+clean","last_fresh":"2020-12-05T17:31:41.635837+0000","last_change":"2020-12-05T17:31:38.672245+0000","last_active":"2020-12-05T17:31:41.635837+0000","last_peered":"2020-12-05T17:31:41.635837+0000","last_clean":"2020-12-05T17:31:41.635837+0000","last_became_active":"2020-12-05T17:31:38.671182+0000","last_became_peered":"2020-12-05T17:31:38.671182+0000","last_unstale":"2020-12-05T17:31:41.635837+0000","last_undegraded":"2020-12-05T17:31:41.635837+0000","last_fullsized":"2020-12-05T17:31:41.635837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_clean_scrub_stamp":"2020-12-05T17:31:37.227982+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"14","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.269072+0000","last_change":"2020-12-05T17:31:35.123497+0000","last_active":"2020-12-05T17:31:41.269072+0000","last_peered":"2020-12-05T17:31:41.269072+0000","last_clean":"2020-12-05T17:31:41.269072+0000","last_became_active":"2020-12-05T17:31:35.122972+0000","last_became_peered":"2020-12-05T17:31:35.122972+0000","last_unstale":"2020-12-05T17:31:41.269072+0000","last_undegraded":"2020-12-05T17:31:41.269072+0000","last_fullsized":"2020-12-05T17:31:41.269072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.296339+0000","last_change":"2020-12-05T17:31:41.295397+0000","last_active":"2020-12-05T17:31:41.296339+0000","last_peered":"2020-12-05T17:31:41.296339+0000","last_clean":"2020-12-05T17:31:41.296339+0000","last_became_active":"2020-12-05T17:31:41.294256+0000","last_became_peered":"2020-12-05T17:31:41.294256+0000","last_unstale":"2020-12-05T17:31:41.296339+0000","last_undegraded":"2020-12-05T17:31:41.296339+0000","last_fullsized":"2020-12-05T17:31:41.296339+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_clean_scrub_stamp":"2020-12-05T17:31:40.257305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"14","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.271162+0000","last_change":"2020-12-05T17:31:35.127416+0000","last_active":"2020-12-05T17:31:41.271162+0000","last_peered":"2020-12-05T17:31:41.271162+0000","last_clean":"2020-12-05T17:31:41.271162+0000","last_became_active":"2020-12-05T17:31:35.126829+0000","last_became_peered":"2020-12-05T17:31:35.126829+0000","last_unstale":"2020-12-05T17:31:41.271162+0000","last_undegraded":"2020-12-05T17:31:41.271162+0000","last_fullsized":"2020-12-05T17:31:41.271162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"25'4","reported_seq":"16","reported_epoch":"25","state":"active+clean","last_fresh":"2020-12-05T17:31:41.641987+0000","last_change":"2020-12-05T17:31:38.262588+0000","last_active":"2020-12-05T17:31:41.641987+0000","last_peered":"2020-12-05T17:31:41.641987+0000","last_clean":"2020-12-05T17:31:41.641987+0000","last_became_active":"2020-12-05T17:31:38.262071+0000","last_became_peered":"2020-12-05T17:31:38.262071+0000","last_unstale":"2020-12-05T17:31:41.641987+0000","last_undegraded":"2020-12-05T17:31:41.641987+0000","last_fullsized":"2020-12-05T17:31:41.641987+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_clean_scrub_stamp":"2020-12-05T17:31:37.227982+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048848,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322992,"statfs":{"total":96636764160,"available":95562743808,"internally_reserved":1073741824,"allocated":270336,"data_stored":103606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1880000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1269999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1829999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7350000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2730000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.657}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7010000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048840,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323000,"statfs":{"total":96636764160,"available":95562752000,"internally_reserved":1073741824,"allocated":262144,"data_stored":103031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8109999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4830000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0379999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.125}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.847}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.7589999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048852,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322988,"statfs":{"total":96636764160,"available":95562739712,"internally_reserved":1073741824,"allocated":274432,"data_stored":104566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6619999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5669999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.27}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7149999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6720000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.786}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1880000000000002}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":103065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9730000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.677}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7889999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.008}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7330000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9220000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2269999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048852,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322988,"statfs":{"total":96636764160,"available":95562739712,"internally_reserved":1073741824,"allocated":274432,"data_stored":104566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2789999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9419999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.228}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1360000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.637}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6120000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8260000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048856,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322984,"statfs":{"total":96636764160,"available":95562735616,"internally_reserved":1073741824,"allocated":278528,"data_stored":103766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.333}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8680000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2240000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.964}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5390000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0169999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":103065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7709999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7250000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6909999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6379999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5960000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5030000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048848,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322992,"statfs":{"total":96636764160,"available":95562743808,"internally_reserved":1073741824,"allocated":270336,"data_stored":103191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.397}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2080000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4580000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2869999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6019999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0510000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7719999999999998}]}]}]}} 2020-12-05T17:31:50.966 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-12-05T17:31:51.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.075+0000 7f5b5aff9700 1 Processor -- start 2020-12-05T17:31:51.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.075+0000 7f5b5aff9700 1 -- start start 2020-12-05T17:31:51.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.075+0000 7f5b5aff9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b540afa90 0x7f5b540b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:51.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.075+0000 7f5b5aff9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b54062940 0x7f5b540ad9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:51.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.075+0000 7f5b5aff9700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b540b14f0 con 0x7f5b540b1de0 2020-12-05T17:31:51.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.075+0000 7f5b5aff9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b541467b0 con 0x7f5b540afa90 2020-12-05T17:31:51.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.075+0000 7f5b5aff9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b540ad310 con 0x7f5b54062940 2020-12-05T17:31:51.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.075+0000 7f5b53fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b540afa90 0x7f5b540b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:51.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.075+0000 7f5b53fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b540afa90 0x7f5b540b21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32876/0 (socket says 172.21.15.32:32876) 2020-12-05T17:31:51.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.075+0000 7f5b53fff700 1 -- 172.21.15.32:0/544249607 learned_addr learned my addr 172.21.15.32:0/544249607 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:51.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.075+0000 7f5b59596700 1 --2- 172.21.15.32:0/544249607 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b54062940 0x7f5b540ad9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:51.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.075+0000 7f5b53fff700 1 -- 172.21.15.32:0/544249607 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b54062940 msgr2=0x7f5b540ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:51.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.075+0000 7f5b53fff700 1 --2- 172.21.15.32:0/544249607 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b54062940 0x7f5b540ad9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.075+0000 7f5b53fff700 1 -- 172.21.15.32:0/544249607 >> v1:172.21.15.61:6789/0 conn(0x7f5b540b1de0 legacy=0x7f5b541460a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:51.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.075+0000 7f5b53fff700 1 -- 172.21.15.32:0/544249607 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b540b1750 con 0x7f5b540afa90 2020-12-05T17:31:51.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.075+0000 7f5b53fff700 1 --2- 172.21.15.32:0/544249607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b540afa90 0x7f5b540b21c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f5b440032b0 tx=0x7f5b44009270).ready entity=mon.0 client_cookie=ecad59cd1f13a12a server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:51.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.075+0000 7f5b537fe700 1 -- 172.21.15.32:0/544249607 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b4400bc00 con 0x7f5b540afa90 2020-12-05T17:31:51.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.075+0000 7f5b537fe700 1 -- 172.21.15.32:0/544249607 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4400bd60 con 0x7f5b540afa90 2020-12-05T17:31:51.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.075+0000 7f5b537fe700 1 -- 172.21.15.32:0/544249607 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b44011400 con 0x7f5b540afa90 2020-12-05T17:31:51.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.075+0000 7f5b5aff9700 1 -- 172.21.15.32:0/544249607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b540afa90 msgr2=0x7f5b540b21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:51.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.075+0000 7f5b5aff9700 1 --2- 172.21.15.32:0/544249607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b540afa90 0x7f5b540b21c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f5b440032b0 tx=0x7f5b44009270).stop 2020-12-05T17:31:51.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.079+0000 7f5b5aff9700 1 -- 172.21.15.32:0/544249607 shutdown_connections 2020-12-05T17:31:51.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.079+0000 7f5b5aff9700 1 --2- 172.21.15.32:0/544249607 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b54062940 0x7f5b540ad9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.079+0000 7f5b5aff9700 1 --2- 172.21.15.32:0/544249607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b540afa90 0x7f5b540b21c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.079+0000 7f5b5aff9700 1 -- 172.21.15.32:0/544249607 >> 172.21.15.32:0/544249607 conn(0x7f5b5405c8d0 msgr2=0x7f5b5405ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:51.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.079+0000 7f5b5aff9700 1 -- 172.21.15.32:0/544249607 shutdown_connections 2020-12-05T17:31:51.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.079+0000 7f5b5aff9700 1 -- 172.21.15.32:0/544249607 wait complete. 2020-12-05T17:31:51.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.079+0000 7f5b5aff9700 1 Processor -- start 2020-12-05T17:31:51.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.079+0000 7f5b5aff9700 1 -- start start 2020-12-05T17:31:51.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.083+0000 7f5b5aff9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b54062940 0x7f5b54065fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:51.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.083+0000 7f5b5aff9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5b540afa90 0x7f5b54145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:51.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.083+0000 7f5b5aff9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b540b1de0 0x7f5b541556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:51.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.083+0000 7f5b5aff9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b540a2d70 con 0x7f5b540b1de0 2020-12-05T17:31:51.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.083+0000 7f5b5aff9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b540650f0 con 0x7f5b540afa90 2020-12-05T17:31:51.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.083+0000 7f5b5aff9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b54066570 con 0x7f5b54062940 2020-12-05T17:31:51.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.083+0000 7f5b59596700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b540b1de0 0x7f5b541556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:51.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.083+0000 7f5b59596700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b540b1de0 0x7f5b541556c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32884/0 (socket says 172.21.15.32:32884) 2020-12-05T17:31:51.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.083+0000 7f5b59596700 1 -- 172.21.15.32:0/2449300808 learned_addr learned my addr 172.21.15.32:0/2449300808 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:51.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.083+0000 7f5b58d95700 1 --2- 172.21.15.32:0/2449300808 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b54062940 0x7f5b54065fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:51.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.083+0000 7f5b53fff700 1 --2- 172.21.15.32:0/2449300808 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5b540afa90 0x7f5b54145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:51.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.083+0000 7f5b59596700 1 -- 172.21.15.32:0/2449300808 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b54062940 msgr2=0x7f5b54065fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:51.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.083+0000 7f5b59596700 1 --2- 172.21.15.32:0/2449300808 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b54062940 0x7f5b54065fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.083+0000 7f5b59596700 1 -- 172.21.15.32:0/2449300808 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5b540afa90 msgr2=0x7f5b54145800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:51.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.083+0000 7f5b59596700 1 --2- 172.21.15.32:0/2449300808 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5b540afa90 0x7f5b54145800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.083+0000 7f5b59596700 1 -- 172.21.15.32:0/2449300808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b54066810 con 0x7f5b540b1de0 2020-12-05T17:31:51.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.083+0000 7f5b59596700 1 --2- 172.21.15.32:0/2449300808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b540b1de0 0x7f5b541556c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f5b4c00a000 tx=0x7f5b4c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:51.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.083+0000 7f5b51ffb700 1 -- 172.21.15.32:0/2449300808 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b4c0127e0 con 0x7f5b540b1de0 2020-12-05T17:31:51.093 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.083+0000 7f5b51ffb700 1 -- 172.21.15.32:0/2449300808 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4c012940 con 0x7f5b540b1de0 2020-12-05T17:31:51.093 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.083+0000 7f5b5aff9700 1 -- 172.21.15.32:0/2449300808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b54066ab0 con 0x7f5b540b1de0 2020-12-05T17:31:51.093 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.083+0000 7f5b51ffb700 1 -- 172.21.15.32:0/2449300808 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b4c012ca0 con 0x7f5b540b1de0 2020-12-05T17:31:51.093 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.083+0000 7f5b5aff9700 1 -- 172.21.15.32:0/2449300808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b540659f0 con 0x7f5b540b1de0 2020-12-05T17:31:51.093 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.083+0000 7f5b5aff9700 1 -- 172.21.15.32:0/2449300808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b54066c40 con 0x7f5b540b1de0 2020-12-05T17:31:51.094 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.087+0000 7f5b51ffb700 1 -- 172.21.15.32:0/2449300808 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5b4c013480 con 0x7f5b540b1de0 2020-12-05T17:31:51.094 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.087+0000 7f5b51ffb700 1 --2- 172.21.15.32:0/2449300808 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b380553b0 0x7f5b38057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:51.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.087+0000 7f5b58d95700 1 --2- 172.21.15.32:0/2449300808 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b380553b0 0x7f5b38057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:51.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.087+0000 7f5b51ffb700 1 -- 172.21.15.32:0/2449300808 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5b4c06e7a0 con 0x7f5b540b1de0 2020-12-05T17:31:51.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.087+0000 7f5b58d95700 1 --2- 172.21.15.32:0/2449300808 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b380553b0 0x7f5b38057870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f5b3c00d000 tx=0x7f5b3c007b40).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:51.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.091+0000 7f5b51ffb700 1 -- 172.21.15.32:0/2449300808 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5b4c0447d0 con 0x7f5b540b1de0 2020-12-05T17:31:51.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.223+0000 7f5b5aff9700 1 -- 172.21.15.32:0/2449300808 --> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5b54057050 con 0x7f5b380553b0 2020-12-05T17:31:51.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.231+0000 7f5b51ffb700 1 -- 172.21.15.32:0/2449300808 <== mgr.4101 v2:172.21.15.32:6800/11355 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82393 (secure 0 0 0) 0x7f5b54057050 con 0x7f5b380553b0 2020-12-05T17:31:51.237 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:31:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.235+0000 7f5b5aff9700 1 -- 172.21.15.32:0/2449300808 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b380553b0 msgr2=0x7f5b38057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.235+0000 7f5b5aff9700 1 --2- 172.21.15.32:0/2449300808 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b380553b0 0x7f5b38057870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f5b3c00d000 tx=0x7f5b3c007b40).stop 2020-12-05T17:31:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.235+0000 7f5b5aff9700 1 -- 172.21.15.32:0/2449300808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b540b1de0 msgr2=0x7f5b541556c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.235+0000 7f5b5aff9700 1 --2- 172.21.15.32:0/2449300808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b540b1de0 0x7f5b541556c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f5b4c00a000 tx=0x7f5b4c008040).stop 2020-12-05T17:31:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.235+0000 7f5b5aff9700 1 -- 172.21.15.32:0/2449300808 shutdown_connections 2020-12-05T17:31:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.235+0000 7f5b5aff9700 1 --2- 172.21.15.32:0/2449300808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b540b1de0 0x7f5b541556c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.235+0000 7f5b5aff9700 1 --2- 172.21.15.32:0/2449300808 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b380553b0 0x7f5b38057870 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.235+0000 7f5b5aff9700 1 --2- 172.21.15.32:0/2449300808 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b54062940 0x7f5b54065fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.235+0000 7f5b5aff9700 1 --2- 172.21.15.32:0/2449300808 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5b540afa90 0x7f5b54145800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.235+0000 7f5b5aff9700 1 -- 172.21.15.32:0/2449300808 >> 172.21.15.32:0/2449300808 conn(0x7f5b5405c8d0 msgr2=0x7f5b540abad0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.235+0000 7f5b5aff9700 1 -- 172.21.15.32:0/2449300808 shutdown_connections 2020-12-05T17:31:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.235+0000 7f5b5aff9700 1 -- 172.21.15.32:0/2449300808 wait complete. 2020-12-05T17:31:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2020-12-05T17:31:51.252 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2020-12-05T17:31:50.801342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390784,"kb_used_data":2112,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746583936,"statfs":{"total":773094113280,"available":764501950464,"internally_reserved":8589934592,"allocated":2162688,"data_stored":828856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.536458"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.295094+0000","last_change":"2020-12-05T17:31:41.293987+0000","last_active":"2020-12-05T17:31:41.295094+0000","last_peered":"2020-12-05T17:31:41.295094+0000","last_clean":"2020-12-05T17:31:41.295094+0000","last_became_active":"2020-12-05T17:31:41.293037+0000","last_became_peered":"2020-12-05T17:31:41.293037+0000","last_unstale":"2020-12-05T17:31:41.295094+0000","last_undegraded":"2020-12-05T17:31:41.295094+0000","last_fullsized":"2020-12-05T17:31:41.295094+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_clean_scrub_stamp":"2020-12-05T17:31:40.257305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"14","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.269142+0000","last_change":"2020-12-05T17:31:35.374114+0000","last_active":"2020-12-05T17:31:41.269142+0000","last_peered":"2020-12-05T17:31:41.269142+0000","last_clean":"2020-12-05T17:31:41.269142+0000","last_became_active":"2020-12-05T17:31:35.373231+0000","last_became_peered":"2020-12-05T17:31:35.373231+0000","last_unstale":"2020-12-05T17:31:41.269142+0000","last_undegraded":"2020-12-05T17:31:41.269142+0000","last_fullsized":"2020-12-05T17:31:41.269142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"25'6","reported_seq":"18","reported_epoch":"25","state":"active+clean","last_fresh":"2020-12-05T17:31:41.641581+0000","last_change":"2020-12-05T17:31:38.264052+0000","last_active":"2020-12-05T17:31:41.641581+0000","last_peered":"2020-12-05T17:31:41.641581+0000","last_clean":"2020-12-05T17:31:41.641581+0000","last_became_active":"2020-12-05T17:31:38.263431+0000","last_became_peered":"2020-12-05T17:31:38.263431+0000","last_unstale":"2020-12-05T17:31:41.641581+0000","last_undegraded":"2020-12-05T17:31:41.641581+0000","last_fullsized":"2020-12-05T17:31:41.641581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_clean_scrub_stamp":"2020-12-05T17:31:37.227982+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.289851+0000","last_change":"2020-12-05T17:31:41.288122+0000","last_active":"2020-12-05T17:31:41.289851+0000","last_peered":"2020-12-05T17:31:41.289851+0000","last_clean":"2020-12-05T17:31:41.289851+0000","last_became_active":"2020-12-05T17:31:41.287342+0000","last_became_peered":"2020-12-05T17:31:41.287342+0000","last_unstale":"2020-12-05T17:31:41.289851+0000","last_undegraded":"2020-12-05T17:31:41.289851+0000","last_fullsized":"2020-12-05T17:31:41.289851+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_clean_scrub_stamp":"2020-12-05T17:31:40.257305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.291220+0000","last_change":"2020-12-05T17:31:35.123847+0000","last_active":"2020-12-05T17:31:41.291220+0000","last_peered":"2020-12-05T17:31:41.291220+0000","last_clean":"2020-12-05T17:31:41.291220+0000","last_became_active":"2020-12-05T17:31:35.123169+0000","last_became_peered":"2020-12-05T17:31:35.123169+0000","last_unstale":"2020-12-05T17:31:41.291220+0000","last_undegraded":"2020-12-05T17:31:41.291220+0000","last_fullsized":"2020-12-05T17:31:41.291220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"25'5","reported_seq":"17","reported_epoch":"25","state":"active+clean","last_fresh":"2020-12-05T17:31:41.638742+0000","last_change":"2020-12-05T17:31:38.263937+0000","last_active":"2020-12-05T17:31:41.638742+0000","last_peered":"2020-12-05T17:31:41.638742+0000","last_clean":"2020-12-05T17:31:41.638742+0000","last_became_active":"2020-12-05T17:31:38.263349+0000","last_became_peered":"2020-12-05T17:31:38.263349+0000","last_unstale":"2020-12-05T17:31:41.638742+0000","last_undegraded":"2020-12-05T17:31:41.638742+0000","last_fullsized":"2020-12-05T17:31:41.638742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_clean_scrub_stamp":"2020-12-05T17:31:37.227982+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.295098+0000","last_change":"2020-12-05T17:31:41.293648+0000","last_active":"2020-12-05T17:31:41.295098+0000","last_peered":"2020-12-05T17:31:41.295098+0000","last_clean":"2020-12-05T17:31:41.295098+0000","last_became_active":"2020-12-05T17:31:41.292941+0000","last_became_peered":"2020-12-05T17:31:41.292941+0000","last_unstale":"2020-12-05T17:31:41.295098+0000","last_undegraded":"2020-12-05T17:31:41.295098+0000","last_fullsized":"2020-12-05T17:31:41.295098+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_clean_scrub_stamp":"2020-12-05T17:31:40.257305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.285861+0000","last_change":"2020-12-05T17:31:35.617802+0000","last_active":"2020-12-05T17:31:41.285861+0000","last_peered":"2020-12-05T17:31:41.285861+0000","last_clean":"2020-12-05T17:31:41.285861+0000","last_became_active":"2020-12-05T17:31:35.617427+0000","last_became_peered":"2020-12-05T17:31:35.617427+0000","last_unstale":"2020-12-05T17:31:41.285861+0000","last_undegraded":"2020-12-05T17:31:41.285861+0000","last_fullsized":"2020-12-05T17:31:41.285861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"25'8","reported_seq":"20","reported_epoch":"25","state":"active+clean","last_fresh":"2020-12-05T17:31:41.641591+0000","last_change":"2020-12-05T17:31:38.263384+0000","last_active":"2020-12-05T17:31:41.641591+0000","last_peered":"2020-12-05T17:31:41.641591+0000","last_clean":"2020-12-05T17:31:41.641591+0000","last_became_active":"2020-12-05T17:31:38.262951+0000","last_became_peered":"2020-12-05T17:31:38.262951+0000","last_unstale":"2020-12-05T17:31:41.641591+0000","last_undegraded":"2020-12-05T17:31:41.641591+0000","last_fullsized":"2020-12-05T17:31:41.641591+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_clean_scrub_stamp":"2020-12-05T17:31:37.227982+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.287696+0000","last_change":"2020-12-05T17:31:41.283820+0000","last_active":"2020-12-05T17:31:41.287696+0000","last_peered":"2020-12-05T17:31:41.287696+0000","last_clean":"2020-12-05T17:31:41.287696+0000","last_became_active":"2020-12-05T17:31:41.283290+0000","last_became_peered":"2020-12-05T17:31:41.283290+0000","last_unstale":"2020-12-05T17:31:41.287696+0000","last_undegraded":"2020-12-05T17:31:41.287696+0000","last_fullsized":"2020-12-05T17:31:41.287696+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_clean_scrub_stamp":"2020-12-05T17:31:40.257305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.291418+0000","last_change":"2020-12-05T17:31:35.124252+0000","last_active":"2020-12-05T17:31:41.291418+0000","last_peered":"2020-12-05T17:31:41.291418+0000","last_clean":"2020-12-05T17:31:41.291418+0000","last_became_active":"2020-12-05T17:31:35.123671+0000","last_became_peered":"2020-12-05T17:31:35.123671+0000","last_unstale":"2020-12-05T17:31:41.291418+0000","last_undegraded":"2020-12-05T17:31:41.291418+0000","last_fullsized":"2020-12-05T17:31:41.291418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"19","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.271067+0000","last_change":"2020-12-05T17:31:30.406003+0000","last_active":"2020-12-05T17:31:41.271067+0000","last_peered":"2020-12-05T17:31:41.271067+0000","last_clean":"2020-12-05T17:31:41.271067+0000","last_became_active":"2020-12-05T17:31:30.405245+0000","last_became_peered":"2020-12-05T17:31:30.405245+0000","last_unstale":"2020-12-05T17:31:41.271067+0000","last_undegraded":"2020-12-05T17:31:41.271067+0000","last_fullsized":"2020-12-05T17:31:41.271067+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:25.806577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:25.806577+0000","last_clean_scrub_stamp":"2020-12-05T17:31:25.806577+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"25'1","reported_seq":"13","reported_epoch":"25","state":"active+clean","last_fresh":"2020-12-05T17:31:41.618588+0000","last_change":"2020-12-05T17:31:38.260319+0000","last_active":"2020-12-05T17:31:41.618588+0000","last_peered":"2020-12-05T17:31:41.618588+0000","last_clean":"2020-12-05T17:31:41.618588+0000","last_became_active":"2020-12-05T17:31:38.259830+0000","last_became_peered":"2020-12-05T17:31:38.259830+0000","last_unstale":"2020-12-05T17:31:41.618588+0000","last_undegraded":"2020-12-05T17:31:41.618588+0000","last_fullsized":"2020-12-05T17:31:41.618588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_clean_scrub_stamp":"2020-12-05T17:31:37.227982+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.291959+0000","last_change":"2020-12-05T17:31:41.290757+0000","last_active":"2020-12-05T17:31:41.291959+0000","last_peered":"2020-12-05T17:31:41.291959+0000","last_clean":"2020-12-05T17:31:41.291959+0000","last_became_active":"2020-12-05T17:31:41.290144+0000","last_became_peered":"2020-12-05T17:31:41.290144+0000","last_unstale":"2020-12-05T17:31:41.291959+0000","last_undegraded":"2020-12-05T17:31:41.291959+0000","last_fullsized":"2020-12-05T17:31:41.291959+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_clean_scrub_stamp":"2020-12-05T17:31:40.257305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"14","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.271329+0000","last_change":"2020-12-05T17:31:35.118985+0000","last_active":"2020-12-05T17:31:41.271329+0000","last_peered":"2020-12-05T17:31:41.271329+0000","last_clean":"2020-12-05T17:31:41.271329+0000","last_became_active":"2020-12-05T17:31:35.118441+0000","last_became_peered":"2020-12-05T17:31:35.118441+0000","last_unstale":"2020-12-05T17:31:41.271329+0000","last_undegraded":"2020-12-05T17:31:41.271329+0000","last_fullsized":"2020-12-05T17:31:41.271329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.275676+0000","last_change":"2020-12-05T17:31:38.265560+0000","last_active":"2020-12-05T17:31:41.275676+0000","last_peered":"2020-12-05T17:31:41.275676+0000","last_clean":"2020-12-05T17:31:41.275676+0000","last_became_active":"2020-12-05T17:31:38.265128+0000","last_became_peered":"2020-12-05T17:31:38.265128+0000","last_unstale":"2020-12-05T17:31:41.275676+0000","last_undegraded":"2020-12-05T17:31:41.275676+0000","last_fullsized":"2020-12-05T17:31:41.275676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_clean_scrub_stamp":"2020-12-05T17:31:37.227982+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.294841+0000","last_change":"2020-12-05T17:31:41.293913+0000","last_active":"2020-12-05T17:31:41.294841+0000","last_peered":"2020-12-05T17:31:41.294841+0000","last_clean":"2020-12-05T17:31:41.294841+0000","last_became_active":"2020-12-05T17:31:41.293297+0000","last_became_peered":"2020-12-05T17:31:41.293297+0000","last_unstale":"2020-12-05T17:31:41.294841+0000","last_undegraded":"2020-12-05T17:31:41.294841+0000","last_fullsized":"2020-12-05T17:31:41.294841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_clean_scrub_stamp":"2020-12-05T17:31:40.257305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"14","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.271221+0000","last_change":"2020-12-05T17:31:35.375163+0000","last_active":"2020-12-05T17:31:41.271221+0000","last_peered":"2020-12-05T17:31:41.271221+0000","last_clean":"2020-12-05T17:31:41.271221+0000","last_became_active":"2020-12-05T17:31:35.374461+0000","last_became_peered":"2020-12-05T17:31:35.374461+0000","last_unstale":"2020-12-05T17:31:41.271221+0000","last_undegraded":"2020-12-05T17:31:41.271221+0000","last_fullsized":"2020-12-05T17:31:41.271221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"25'8","reported_seq":"20","reported_epoch":"25","state":"active+clean","last_fresh":"2020-12-05T17:31:41.640996+0000","last_change":"2020-12-05T17:31:38.262338+0000","last_active":"2020-12-05T17:31:41.640996+0000","last_peered":"2020-12-05T17:31:41.640996+0000","last_clean":"2020-12-05T17:31:41.640996+0000","last_became_active":"2020-12-05T17:31:38.261896+0000","last_became_peered":"2020-12-05T17:31:38.261896+0000","last_unstale":"2020-12-05T17:31:41.640996+0000","last_undegraded":"2020-12-05T17:31:41.640996+0000","last_fullsized":"2020-12-05T17:31:41.640996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_clean_scrub_stamp":"2020-12-05T17:31:37.227982+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.288807+0000","last_change":"2020-12-05T17:31:41.287845+0000","last_active":"2020-12-05T17:31:41.288807+0000","last_peered":"2020-12-05T17:31:41.288807+0000","last_clean":"2020-12-05T17:31:41.288807+0000","last_became_active":"2020-12-05T17:31:41.287109+0000","last_became_peered":"2020-12-05T17:31:41.287109+0000","last_unstale":"2020-12-05T17:31:41.288807+0000","last_undegraded":"2020-12-05T17:31:41.288807+0000","last_fullsized":"2020-12-05T17:31:41.288807+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_clean_scrub_stamp":"2020-12-05T17:31:40.257305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"25'7","reported_seq":"19","reported_epoch":"25","state":"active+clean","last_fresh":"2020-12-05T17:31:41.635837+0000","last_change":"2020-12-05T17:31:38.672245+0000","last_active":"2020-12-05T17:31:41.635837+0000","last_peered":"2020-12-05T17:31:41.635837+0000","last_clean":"2020-12-05T17:31:41.635837+0000","last_became_active":"2020-12-05T17:31:38.671182+0000","last_became_peered":"2020-12-05T17:31:38.671182+0000","last_unstale":"2020-12-05T17:31:41.635837+0000","last_undegraded":"2020-12-05T17:31:41.635837+0000","last_fullsized":"2020-12-05T17:31:41.635837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_clean_scrub_stamp":"2020-12-05T17:31:37.227982+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"14","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.269072+0000","last_change":"2020-12-05T17:31:35.123497+0000","last_active":"2020-12-05T17:31:41.269072+0000","last_peered":"2020-12-05T17:31:41.269072+0000","last_clean":"2020-12-05T17:31:41.269072+0000","last_became_active":"2020-12-05T17:31:35.122972+0000","last_became_peered":"2020-12-05T17:31:35.122972+0000","last_unstale":"2020-12-05T17:31:41.269072+0000","last_undegraded":"2020-12-05T17:31:41.269072+0000","last_fullsized":"2020-12-05T17:31:41.269072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.296339+0000","last_change":"2020-12-05T17:31:41.295397+0000","last_active":"2020-12-05T17:31:41.296339+0000","last_peered":"2020-12-05T17:31:41.296339+0000","last_clean":"2020-12-05T17:31:41.296339+0000","last_became_active":"2020-12-05T17:31:41.294256+0000","last_became_peered":"2020-12-05T17:31:41.294256+0000","last_unstale":"2020-12-05T17:31:41.296339+0000","last_undegraded":"2020-12-05T17:31:41.296339+0000","last_fullsized":"2020-12-05T17:31:41.296339+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:40.257305+0000","last_clean_scrub_stamp":"2020-12-05T17:31:40.257305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"14","reported_epoch":"24","state":"active+clean","last_fresh":"2020-12-05T17:31:41.271162+0000","last_change":"2020-12-05T17:31:35.127416+0000","last_active":"2020-12-05T17:31:41.271162+0000","last_peered":"2020-12-05T17:31:41.271162+0000","last_clean":"2020-12-05T17:31:41.271162+0000","last_became_active":"2020-12-05T17:31:35.126829+0000","last_became_peered":"2020-12-05T17:31:35.126829+0000","last_unstale":"2020-12-05T17:31:41.271162+0000","last_undegraded":"2020-12-05T17:31:41.271162+0000","last_fullsized":"2020-12-05T17:31:41.271162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"25'4","reported_seq":"16","reported_epoch":"25","state":"active+clean","last_fresh":"2020-12-05T17:31:41.641987+0000","last_change":"2020-12-05T17:31:38.262588+0000","last_active":"2020-12-05T17:31:41.641987+0000","last_peered":"2020-12-05T17:31:41.641987+0000","last_clean":"2020-12-05T17:31:41.641987+0000","last_became_active":"2020-12-05T17:31:38.262071+0000","last_became_peered":"2020-12-05T17:31:38.262071+0000","last_unstale":"2020-12-05T17:31:41.641987+0000","last_undegraded":"2020-12-05T17:31:41.641987+0000","last_fullsized":"2020-12-05T17:31:41.641987+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:37.227982+0000","last_clean_scrub_stamp":"2020-12-05T17:31:37.227982+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048848,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322992,"statfs":{"total":96636764160,"available":95562743808,"internally_reserved":1073741824,"allocated":270336,"data_stored":103606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1880000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1269999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1829999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7350000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2730000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.657}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7010000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048840,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323000,"statfs":{"total":96636764160,"available":95562752000,"internally_reserved":1073741824,"allocated":262144,"data_stored":103031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8109999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4830000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0379999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.125}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.847}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.7589999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048852,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322988,"statfs":{"total":96636764160,"available":95562739712,"internally_reserved":1073741824,"allocated":274432,"data_stored":104566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6619999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5669999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.27}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7149999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6720000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.786}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1880000000000002}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":103065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9730000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.677}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7889999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.008}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7330000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9220000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2269999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048852,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322988,"statfs":{"total":96636764160,"available":95562739712,"internally_reserved":1073741824,"allocated":274432,"data_stored":104566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2789999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9419999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.228}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1360000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.637}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6120000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8260000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048856,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322984,"statfs":{"total":96636764160,"available":95562735616,"internally_reserved":1073741824,"allocated":278528,"data_stored":103766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.333}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8680000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2240000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.964}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5390000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0169999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":103065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7709999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7250000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6909999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6379999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5960000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5030000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048848,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322992,"statfs":{"total":96636764160,"available":95562743808,"internally_reserved":1073741824,"allocated":270336,"data_stored":103191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.397}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2080000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4580000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2869999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6019999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0510000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7719999999999998}]}]}]}} 2020-12-05T17:31:51.254 INFO:tasks.ceph.ceph_manager.ceph:clean! 2020-12-05T17:31:51.254 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2020-12-05T17:31:51.255 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2020-12-05T17:31:51.255 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early health --format=json 2020-12-05T17:31:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.363+0000 7fc7196a0700 1 Processor -- start 2020-12-05T17:31:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.363+0000 7fc7196a0700 1 -- start start 2020-12-05T17:31:51.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.363+0000 7fc7196a0700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc7140afe50 0x7fc714146a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:51.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.363+0000 7fc7196a0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7140b3aa0 0x7fc7141429e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.363+0000 7fc7196a0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7140b0230 con 0x7fc7140b1880 2020-12-05T17:31:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.363+0000 7fc7196a0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7140b6a50 con 0x7fc7140b3aa0 2020-12-05T17:31:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.363+0000 7fc7196a0700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc714146080 con 0x7fc7140afe50 2020-12-05T17:31:51.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.363+0000 7fc7127fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7140b3aa0 0x7fc7141429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:51.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.363+0000 7fc7137fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc7140b1880 0x7fc714142ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35540/0 (socket says 172.21.15.32:35540) 2020-12-05T17:31:51.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.363+0000 7fc7137fe700 1 -- 172.21.15.32:0/2514740969 learned_addr learned my addr 172.21.15.32:0/2514740969 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:51.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.363+0000 7fc712ffd700 1 --2- 172.21.15.32:0/2514740969 >> v2:172.21.15.61:3300/0 conn(0x7fc7140afe50 0x7fc714146a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:51.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.363+0000 7fc711ffb700 1 -- 172.21.15.32:0/2514740969 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1731379304 0 0) 0x7fc7140b0230 con 0x7fc7140b1880 2020-12-05T17:31:51.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.363+0000 7fc711ffb700 1 -- 172.21.15.32:0/2514740969 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6f8003230 con 0x7fc7140b1880 2020-12-05T17:31:51.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.363+0000 7fc7127fc700 1 -- 172.21.15.32:0/2514740969 >> v2:172.21.15.61:3300/0 conn(0x7fc7140afe50 msgr2=0x7fc714146a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:51.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.363+0000 7fc7127fc700 1 --2- 172.21.15.32:0/2514740969 >> v2:172.21.15.61:3300/0 conn(0x7fc7140afe50 0x7fc714146a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.363+0000 7fc7127fc700 1 -- 172.21.15.32:0/2514740969 >> v1:172.21.15.32:6789/0 conn(0x7fc7140b1880 legacy=0x7fc714142ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:51.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.363+0000 7fc7127fc700 1 -- 172.21.15.32:0/2514740969 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc714146320 con 0x7fc7140b3aa0 2020-12-05T17:31:51.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.367+0000 7fc7127fc700 1 --2- 172.21.15.32:0/2514740969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7140b3aa0 0x7fc7141429e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fc708002f30 tx=0x7fc708006dc0).ready entity=mon.0 client_cookie=3c573591119fb1c1 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:51.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.367+0000 7fc711ffb700 1 -- 172.21.15.32:0/2514740969 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7080096f0 con 0x7fc7140b3aa0 2020-12-05T17:31:51.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.367+0000 7fc711ffb700 1 -- 172.21.15.32:0/2514740969 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc708009850 con 0x7fc7140b3aa0 2020-12-05T17:31:51.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.367+0000 7fc711ffb700 1 -- 172.21.15.32:0/2514740969 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc708009bb0 con 0x7fc7140b3aa0 2020-12-05T17:31:51.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.367+0000 7fc7196a0700 1 -- 172.21.15.32:0/2514740969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7140b3aa0 msgr2=0x7fc7141429e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:51.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.367+0000 7fc7196a0700 1 --2- 172.21.15.32:0/2514740969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7140b3aa0 0x7fc7141429e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fc708002f30 tx=0x7fc708006dc0).stop 2020-12-05T17:31:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.367+0000 7fc7196a0700 1 -- 172.21.15.32:0/2514740969 shutdown_connections 2020-12-05T17:31:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.367+0000 7fc7196a0700 1 --2- 172.21.15.32:0/2514740969 >> v2:172.21.15.61:3300/0 conn(0x7fc7140afe50 0x7fc714146a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.367+0000 7fc7196a0700 1 --2- 172.21.15.32:0/2514740969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7140b3aa0 0x7fc7141429e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.367+0000 7fc7196a0700 1 -- 172.21.15.32:0/2514740969 >> 172.21.15.32:0/2514740969 conn(0x7fc7140b9100 msgr2=0x7fc7140b9510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.367+0000 7fc7196a0700 1 -- 172.21.15.32:0/2514740969 shutdown_connections 2020-12-05T17:31:51.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.367+0000 7fc7196a0700 1 -- 172.21.15.32:0/2514740969 wait complete. 2020-12-05T17:31:51.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.367+0000 7fc7196a0700 1 Processor -- start 2020-12-05T17:31:51.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.367+0000 7fc7196a0700 1 -- start start 2020-12-05T17:31:51.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.371+0000 7fc7196a0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7140afe50 0x7fc7141427e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:51.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.371+0000 7fc7196a0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc7140b1880 0x7fc714141360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:51.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.371+0000 7fc7196a0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc7140b3aa0 0x7fc714137f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:51.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.371+0000 7fc7196a0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc714142210 con 0x7fc7140afe50 2020-12-05T17:31:51.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.371+0000 7fc7196a0700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71413ec30 con 0x7fc7140b3aa0 2020-12-05T17:31:51.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.371+0000 7fc7196a0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc714142cd0 con 0x7fc7140b1880 2020-12-05T17:31:51.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.371+0000 7fc712ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7140afe50 0x7fc7141427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:51.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.371+0000 7fc712ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7140afe50 0x7fc7141427e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32894/0 (socket says 172.21.15.32:32894) 2020-12-05T17:31:51.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.371+0000 7fc712ffd700 1 -- 172.21.15.32:0/3012039431 learned_addr learned my addr 172.21.15.32:0/3012039431 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:51.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.371+0000 7fc7127fc700 1 --2- 172.21.15.32:0/3012039431 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc7140b1880 0x7fc714141360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:51.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.371+0000 7fc7137fe700 1 --2- 172.21.15.32:0/3012039431 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc7140b3aa0 0x7fc714137f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.371+0000 7fc712ffd700 1 -- 172.21.15.32:0/3012039431 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc7140b1880 msgr2=0x7fc714141360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.371+0000 7fc712ffd700 1 --2- 172.21.15.32:0/3012039431 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc7140b1880 0x7fc714141360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.371+0000 7fc712ffd700 1 -- 172.21.15.32:0/3012039431 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc7140b3aa0 msgr2=0x7fc714137f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.371+0000 7fc712ffd700 1 --2- 172.21.15.32:0/3012039431 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc7140b3aa0 0x7fc714137f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.371+0000 7fc712ffd700 1 -- 172.21.15.32:0/3012039431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71413e7f0 con 0x7fc7140afe50 2020-12-05T17:31:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.371+0000 7fc712ffd700 1 --2- 172.21.15.32:0/3012039431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7140afe50 0x7fc7141427e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fc70400c000 tx=0x7fc704006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.371+0000 7fc703fff700 1 -- 172.21.15.32:0/3012039431 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc70400ca80 con 0x7fc7140afe50 2020-12-05T17:31:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.371+0000 7fc703fff700 1 -- 172.21.15.32:0/3012039431 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70400cbe0 con 0x7fc7140afe50 2020-12-05T17:31:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.371+0000 7fc7196a0700 1 -- 172.21.15.32:0/3012039431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc714141850 con 0x7fc7140afe50 2020-12-05T17:31:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.371+0000 7fc703fff700 1 -- 172.21.15.32:0/3012039431 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7040123f0 con 0x7fc7140afe50 2020-12-05T17:31:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.371+0000 7fc7196a0700 1 -- 172.21.15.32:0/3012039431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7141393d0 con 0x7fc7140afe50 2020-12-05T17:31:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.371+0000 7fc7196a0700 1 -- 172.21.15.32:0/3012039431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc714142410 con 0x7fc7140afe50 2020-12-05T17:31:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.375+0000 7fc703fff700 1 -- 172.21.15.32:0/3012039431 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc704012550 con 0x7fc7140afe50 2020-12-05T17:31:51.383 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.375+0000 7fc703fff700 1 --2- 172.21.15.32:0/3012039431 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc6f0055440 0x7fc6f0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:51.383 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.375+0000 7fc703fff700 1 -- 172.21.15.32:0/3012039431 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc70406e500 con 0x7fc7140afe50 2020-12-05T17:31:51.383 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.375+0000 7fc7127fc700 1 --2- 172.21.15.32:0/3012039431 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc6f0055440 0x7fc6f0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:51.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.375+0000 7fc7127fc700 1 --2- 172.21.15.32:0/3012039431 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc6f0055440 0x7fc6f0057900 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fc708019000 tx=0x7fc708015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.383+0000 7fc703fff700 1 -- 172.21.15.32:0/3012039431 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc704044640 con 0x7fc7140afe50 2020-12-05T17:31:51.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.579+0000 7fc7196a0700 1 -- 172.21.15.32:0/3012039431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc71413eff0 con 0x7fc7140afe50 2020-12-05T17:31:51.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.579+0000 7fc703fff700 1 -- 172.21.15.32:0/3012039431 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7fc704047c60 con 0x7fc7140afe50 2020-12-05T17:31:51.583 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:31:51.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.579+0000 7fc7196a0700 1 -- 172.21.15.32:0/3012039431 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc6f0055440 msgr2=0x7fc6f0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:51.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.579+0000 7fc7196a0700 1 --2- 172.21.15.32:0/3012039431 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc6f0055440 0x7fc6f0057900 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fc708019000 tx=0x7fc708015000).stop 2020-12-05T17:31:51.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.579+0000 7fc7196a0700 1 -- 172.21.15.32:0/3012039431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7140afe50 msgr2=0x7fc7141427e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:51.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.579+0000 7fc7196a0700 1 --2- 172.21.15.32:0/3012039431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7140afe50 0x7fc7141427e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fc70400c000 tx=0x7fc704006dc0).stop 2020-12-05T17:31:51.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.579+0000 7fc7196a0700 1 -- 172.21.15.32:0/3012039431 shutdown_connections 2020-12-05T17:31:51.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.579+0000 7fc7196a0700 1 --2- 172.21.15.32:0/3012039431 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc7140b3aa0 0x7fc714137f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.579+0000 7fc7196a0700 1 --2- 172.21.15.32:0/3012039431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7140afe50 0x7fc7141427e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.579+0000 7fc7196a0700 1 --2- 172.21.15.32:0/3012039431 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc6f0055440 0x7fc6f0057900 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.583+0000 7fc7196a0700 1 --2- 172.21.15.32:0/3012039431 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc7140b1880 0x7fc714141360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.583+0000 7fc7196a0700 1 -- 172.21.15.32:0/3012039431 >> 172.21.15.32:0/3012039431 conn(0x7fc7140b9100 msgr2=0x7fc714143f30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:51.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.583+0000 7fc7196a0700 1 -- 172.21.15.32:0/3012039431 shutdown_connections 2020-12-05T17:31:51.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.583+0000 7fc7196a0700 1 -- 172.21.15.32:0/3012039431 wait complete. 2020-12-05T17:31:51.598 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2020-12-05T17:31:51.598 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2020-12-05T17:31:51.599 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:31:51.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.707+0000 7f047ffff700 1 Processor -- start 2020-12-05T17:31:51.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.707+0000 7f047ffff700 1 -- start start 2020-12-05T17:31:51.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.707+0000 7f047ffff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f048014f680 0x7f0480151a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:51.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.707+0000 7f047ffff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04800b5a30 con 0x7f04800b5650 2020-12-05T17:31:51.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.707+0000 7f047ffff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f048014ae80 con 0x7f04800afd20 2020-12-05T17:31:51.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.707+0000 7f047ffff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0480067990 con 0x7f048014f680 2020-12-05T17:31:51.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.707+0000 7f047effd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f04800b5650 0x7f048014a060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35550/0 (socket says 172.21.15.32:35550) 2020-12-05T17:31:51.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.707+0000 7f047effd700 1 -- 172.21.15.32:0/2352556967 learned_addr learned my addr 172.21.15.32:0/2352556967 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:51.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.707+0000 7f047f7fe700 1 --2- 172.21.15.32:0/2352556967 >> v2:172.21.15.61:3300/0 conn(0x7f048014f680 0x7f0480151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:51.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.711+0000 7f047f7fe700 1 -- 172.21.15.32:0/2352556967 >> v1:172.21.15.61:6789/0 conn(0x7f04800afd20 legacy=0x7f048014a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:51.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.711+0000 7f047f7fe700 1 -- 172.21.15.32:0/2352556967 >> v1:172.21.15.32:6789/0 conn(0x7f04800b5650 legacy=0x7f048014a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:51.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.711+0000 7f047f7fe700 1 -- 172.21.15.32:0/2352556967 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04800aea20 con 0x7f048014f680 2020-12-05T17:31:51.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.711+0000 7f047f7fe700 1 --2- 172.21.15.32:0/2352556967 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f048014f680 0x7f0480151a70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f0478002f30 tx=0x7f0478006dc0).ready entity=mon.1 client_cookie=110f45543ee49b14 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:51.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.711+0000 7f047dffb700 1 -- 172.21.15.32:0/2352556967 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04780097f0 con 0x7f048014f680 2020-12-05T17:31:51.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.711+0000 7f047dffb700 1 -- 172.21.15.32:0/2352556967 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0478009950 con 0x7f048014f680 2020-12-05T17:31:51.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.711+0000 7f047dffb700 1 -- 172.21.15.32:0/2352556967 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0478009cb0 con 0x7f048014f680 2020-12-05T17:31:51.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.711+0000 7f047ffff700 1 -- 172.21.15.32:0/2352556967 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f048014f680 msgr2=0x7f0480151a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:51.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.711+0000 7f047ffff700 1 --2- 172.21.15.32:0/2352556967 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f048014f680 0x7f0480151a70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f0478002f30 tx=0x7f0478006dc0).stop 2020-12-05T17:31:51.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.711+0000 7f047ffff700 1 -- 172.21.15.32:0/2352556967 shutdown_connections 2020-12-05T17:31:51.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.711+0000 7f047ffff700 1 --2- 172.21.15.32:0/2352556967 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f048014f680 0x7f0480151a70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.711+0000 7f047ffff700 1 -- 172.21.15.32:0/2352556967 >> 172.21.15.32:0/2352556967 conn(0x7f048005bbb0 msgr2=0x7f0480062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:51.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.711+0000 7f047ffff700 1 -- 172.21.15.32:0/2352556967 shutdown_connections 2020-12-05T17:31:51.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.711+0000 7f047ffff700 1 -- 172.21.15.32:0/2352556967 wait complete. 2020-12-05T17:31:51.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.715+0000 7f047ffff700 1 Processor -- start 2020-12-05T17:31:51.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.715+0000 7f047ffff700 1 -- start start 2020-12-05T17:31:51.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.715+0000 7f047ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04800afd20 0x7f04801567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:51.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.715+0000 7f047ffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04800b5650 0x7f048013a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:51.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.715+0000 7f047ffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f048014f680 0x7f048013a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:51.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.715+0000 7f047ffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048014a590 con 0x7f04800afd20 2020-12-05T17:31:51.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.715+0000 7f047ffff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048014f490 con 0x7f04800b5650 2020-12-05T17:31:51.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.715+0000 7f047ffff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048014a8e0 con 0x7f048014f680 2020-12-05T17:31:51.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.715+0000 7f047effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04800afd20 0x7f04801567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:51.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.715+0000 7f047effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04800afd20 0x7f04801567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32908/0 (socket says 172.21.15.32:32908) 2020-12-05T17:31:51.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.715+0000 7f047effd700 1 -- 172.21.15.32:0/1595511408 learned_addr learned my addr 172.21.15.32:0/1595511408 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:51.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.715+0000 7f047e7fc700 1 --2- 172.21.15.32:0/1595511408 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04800b5650 0x7f048013a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:51.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.715+0000 7f047f7fe700 1 --2- 172.21.15.32:0/1595511408 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f048014f680 0x7f048013a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:51.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.715+0000 7f047effd700 1 -- 172.21.15.32:0/1595511408 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f048014f680 msgr2=0x7f048013a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:51.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.715+0000 7f047effd700 1 --2- 172.21.15.32:0/1595511408 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f048014f680 0x7f048013a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.715+0000 7f047effd700 1 -- 172.21.15.32:0/1595511408 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04800b5650 msgr2=0x7f048013a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:51.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.715+0000 7f047effd700 1 --2- 172.21.15.32:0/1595511408 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04800b5650 0x7f048013a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.715+0000 7f047effd700 1 -- 172.21.15.32:0/1595511408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048014f1d0 con 0x7f04800afd20 2020-12-05T17:31:51.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.715+0000 7f047effd700 1 --2- 172.21.15.32:0/1595511408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04800afd20 0x7f04801567c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f0480062be0 tx=0x7f0470007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:51.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.719+0000 7f0477fff700 1 -- 172.21.15.32:0/1595511408 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f047000ce40 con 0x7f04800afd20 2020-12-05T17:31:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.719+0000 7f047ffff700 1 -- 172.21.15.32:0/1595511408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f048013add0 con 0x7f04800afd20 2020-12-05T17:31:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.719+0000 7f0477fff700 1 -- 172.21.15.32:0/1595511408 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0470005b70 con 0x7f04800afd20 2020-12-05T17:31:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.719+0000 7f0477fff700 1 -- 172.21.15.32:0/1595511408 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0470007110 con 0x7f04800afd20 2020-12-05T17:31:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.719+0000 7f047ffff700 1 -- 172.21.15.32:0/1595511408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048014cd10 con 0x7f04800afd20 2020-12-05T17:31:51.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.719+0000 7f047ffff700 1 -- 172.21.15.32:0/1595511408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f048014aaa0 con 0x7f04800afd20 2020-12-05T17:31:51.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.719+0000 7f0477fff700 1 -- 172.21.15.32:0/1595511408 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0470010030 con 0x7f04800afd20 2020-12-05T17:31:51.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.719+0000 7f0477fff700 1 --2- 172.21.15.32:0/1595511408 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0464055330 0x7f04640577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:51.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.719+0000 7f0477fff700 1 -- 172.21.15.32:0/1595511408 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f047006e030 con 0x7f04800afd20 2020-12-05T17:31:51.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.719+0000 7f047e7fc700 1 --2- 172.21.15.32:0/1595511408 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0464055330 0x7f04640577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:51.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.723+0000 7f047e7fc700 1 --2- 172.21.15.32:0/1595511408 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0464055330 0x7f04640577f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f046800d000 tx=0x7f0468007b60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:51.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.731+0000 7f0477fff700 1 -- 172.21.15.32:0/1595511408 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0470044130 con 0x7f04800afd20 2020-12-05T17:31:51.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.871+0000 7f047ffff700 1 -- 172.21.15.32:0/1595511408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0480057050 con 0x7f04800afd20 2020-12-05T17:31:51.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.875+0000 7f0477fff700 1 -- 172.21.15.32:0/1595511408 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f0470047750 con 0x7f04800afd20 2020-12-05T17:31:51.878 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:31:51.878 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:31:41.610914+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:31:36.706361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-12-05T17:31:39.298269+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:31:51.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.875+0000 7f047ffff700 1 -- 172.21.15.32:0/1595511408 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0464055330 msgr2=0x7f04640577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:51.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.875+0000 7f047ffff700 1 --2- 172.21.15.32:0/1595511408 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0464055330 0x7f04640577f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f046800d000 tx=0x7f0468007b60).stop 2020-12-05T17:31:51.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.875+0000 7f047ffff700 1 -- 172.21.15.32:0/1595511408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04800afd20 msgr2=0x7f04801567c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:51.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.875+0000 7f047ffff700 1 --2- 172.21.15.32:0/1595511408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04800afd20 0x7f04801567c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f0480062be0 tx=0x7f0470007f90).stop 2020-12-05T17:31:51.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.875+0000 7f047ffff700 1 -- 172.21.15.32:0/1595511408 shutdown_connections 2020-12-05T17:31:51.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.875+0000 7f047ffff700 1 --2- 172.21.15.32:0/1595511408 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0464055330 0x7f04640577f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.875+0000 7f047ffff700 1 --2- 172.21.15.32:0/1595511408 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f048014f680 0x7f048013a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.875+0000 7f047ffff700 1 --2- 172.21.15.32:0/1595511408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04800afd20 0x7f04801567c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.875+0000 7f047ffff700 1 --2- 172.21.15.32:0/1595511408 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04800b5650 0x7f048013a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:51.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.875+0000 7f047ffff700 1 -- 172.21.15.32:0/1595511408 >> 172.21.15.32:0/1595511408 conn(0x7f048005bbb0 msgr2=0x7f048013ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:51.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.875+0000 7f047ffff700 1 -- 172.21.15.32:0/1595511408 shutdown_connections 2020-12-05T17:31:51.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:51.875+0000 7f047ffff700 1 -- 172.21.15.32:0/1595511408 wait complete. 2020-12-05T17:31:51.892 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:31:36.706361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:31:39.298269+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:31:51.892 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:31:52.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.011+0000 7f5e7184c700 1 Processor -- start 2020-12-05T17:31:52.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.011+0000 7f5e7184c700 1 -- start start 2020-12-05T17:31:52.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.011+0000 7f5e7184c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f5e6c0afd30 0x7f5e6c0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:52.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.011+0000 7f5e7184c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e6c1468e0 con 0x7f5e6c0b1dc0 2020-12-05T17:31:52.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.011+0000 7f5e7184c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e6c146b10 con 0x7f5e6c0af5f0 2020-12-05T17:31:52.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.011+0000 7f5e7184c700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e6c146db0 con 0x7f5e6c0afd30 2020-12-05T17:31:52.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.011+0000 7f5e7084a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5e6c0b1dc0 0x7f5e6c0ae600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35564/0 (socket says 172.21.15.32:35564) 2020-12-05T17:31:52.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.011+0000 7f5e7084a700 1 -- 172.21.15.32:0/1994473079 learned_addr learned my addr 172.21.15.32:0/1994473079 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:52.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.011+0000 7f5e6bfff700 1 --2- 172.21.15.32:0/1994473079 >> v2:172.21.15.61:3300/0 conn(0x7f5e6c0afd30 0x7f5e6c0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:52.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.011+0000 7f5e6b7fe700 1 -- 172.21.15.32:0/1994473079 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 725746437 0 0) 0x7f5e6c1468e0 con 0x7f5e6c0b1dc0 2020-12-05T17:31:52.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.011+0000 7f5e6b7fe700 1 -- 172.21.15.32:0/1994473079 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e58003230 con 0x7f5e6c0b1dc0 2020-12-05T17:31:52.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.015+0000 7f5e6bfff700 1 -- 172.21.15.32:0/1994473079 >> v1:172.21.15.61:6789/0 conn(0x7f5e6c0af5f0 legacy=0x7f5e6c141c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.015+0000 7f5e6bfff700 1 -- 172.21.15.32:0/1994473079 >> v1:172.21.15.32:6789/0 conn(0x7f5e6c0b1dc0 legacy=0x7f5e6c0ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.015+0000 7f5e6bfff700 1 -- 172.21.15.32:0/1994473079 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6c0ae160 con 0x7f5e6c0afd30 2020-12-05T17:31:52.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.015+0000 7f5e6bfff700 1 --2- 172.21.15.32:0/1994473079 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e6c0afd30 0x7f5e6c0b21a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f5e60002f30 tx=0x7f5e60006dc0).ready entity=mon.1 client_cookie=e9b289afea149eba server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:52.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.015+0000 7f5e6b7fe700 1 -- 172.21.15.32:0/1994473079 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e60009800 con 0x7f5e6c0afd30 2020-12-05T17:31:52.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.015+0000 7f5e6b7fe700 1 -- 172.21.15.32:0/1994473079 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e60009960 con 0x7f5e6c0afd30 2020-12-05T17:31:52.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.015+0000 7f5e6b7fe700 1 -- 172.21.15.32:0/1994473079 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e60009cc0 con 0x7f5e6c0afd30 2020-12-05T17:31:52.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.015+0000 7f5e7184c700 1 -- 172.21.15.32:0/1994473079 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e6c0afd30 msgr2=0x7f5e6c0b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.015+0000 7f5e7184c700 1 --2- 172.21.15.32:0/1994473079 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e6c0afd30 0x7f5e6c0b21a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f5e60002f30 tx=0x7f5e60006dc0).stop 2020-12-05T17:31:52.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.015+0000 7f5e7184c700 1 -- 172.21.15.32:0/1994473079 shutdown_connections 2020-12-05T17:31:52.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.015+0000 7f5e7184c700 1 --2- 172.21.15.32:0/1994473079 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e6c0afd30 0x7f5e6c0b21a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.015+0000 7f5e7184c700 1 -- 172.21.15.32:0/1994473079 >> 172.21.15.32:0/1994473079 conn(0x7f5e6c0620b0 msgr2=0x7f5e6c062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:52.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.015+0000 7f5e7184c700 1 -- 172.21.15.32:0/1994473079 shutdown_connections 2020-12-05T17:31:52.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.015+0000 7f5e7184c700 1 -- 172.21.15.32:0/1994473079 wait complete. 2020-12-05T17:31:52.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.015+0000 7f5e7184c700 1 Processor -- start 2020-12-05T17:31:52.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.015+0000 7f5e7184c700 1 -- start start 2020-12-05T17:31:52.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e7184c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e6c0af5f0 0x7f5e6c13f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:52.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e7184c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e6c0afd30 0x7f5e6c148820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:52.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e7184c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5e6c0b1dc0 0x7f5e6c065b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:52.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e7184c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c148ec0 con 0x7f5e6c0afd30 2020-12-05T17:31:52.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e7184c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c065430 con 0x7f5e6c0af5f0 2020-12-05T17:31:52.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e7184c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c0656d0 con 0x7f5e6c0b1dc0 2020-12-05T17:31:52.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e6bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e6c0afd30 0x7f5e6c148820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:52.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e6bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e6c0afd30 0x7f5e6c148820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32924/0 (socket says 172.21.15.32:32924) 2020-12-05T17:31:52.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e6bfff700 1 -- 172.21.15.32:0/1908340842 learned_addr learned my addr 172.21.15.32:0/1908340842 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:52.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e7084a700 1 --2- 172.21.15.32:0/1908340842 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e6c0af5f0 0x7f5e6c13f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:52.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e7104b700 1 --2- 172.21.15.32:0/1908340842 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5e6c0b1dc0 0x7f5e6c065b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:52.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e6bfff700 1 -- 172.21.15.32:0/1908340842 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5e6c0b1dc0 msgr2=0x7f5e6c065b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e6bfff700 1 --2- 172.21.15.32:0/1908340842 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5e6c0b1dc0 0x7f5e6c065b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e6bfff700 1 -- 172.21.15.32:0/1908340842 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e6c0af5f0 msgr2=0x7f5e6c13f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e6bfff700 1 --2- 172.21.15.32:0/1908340842 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e6c0af5f0 0x7f5e6c13f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e6bfff700 1 -- 172.21.15.32:0/1908340842 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6c066060 con 0x7f5e6c0afd30 2020-12-05T17:31:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e7104b700 1 --2- 172.21.15.32:0/1908340842 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5e6c0b1dc0 0x7f5e6c065b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e7084a700 1 --2- 172.21.15.32:0/1908340842 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e6c0af5f0 0x7f5e6c13f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e6bfff700 1 --2- 172.21.15.32:0/1908340842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e6c0afd30 0x7f5e6c148820 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f5e60002f30 tx=0x7f5e60015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:52.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e69ffb700 1 -- 172.21.15.32:0/1908340842 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e60011b00 con 0x7f5e6c0afd30 2020-12-05T17:31:52.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e69ffb700 1 -- 172.21.15.32:0/1908340842 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e60011c60 con 0x7f5e6c0afd30 2020-12-05T17:31:52.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e7184c700 1 -- 172.21.15.32:0/1908340842 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e6c149ab0 con 0x7f5e6c0afd30 2020-12-05T17:31:52.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e69ffb700 1 -- 172.21.15.32:0/1908340842 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e6001d440 con 0x7f5e6c0afd30 2020-12-05T17:31:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e7184c700 1 -- 172.21.15.32:0/1908340842 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e6c146ef0 con 0x7f5e6c0afd30 2020-12-05T17:31:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.019+0000 7f5e7184c700 1 -- 172.21.15.32:0/1908340842 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e6c148d10 con 0x7f5e6c0afd30 2020-12-05T17:31:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.023+0000 7f5e69ffb700 1 -- 172.21.15.32:0/1908340842 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5e60024030 con 0x7f5e6c0afd30 2020-12-05T17:31:52.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.023+0000 7f5e69ffb700 1 --2- 172.21.15.32:0/1908340842 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5e4c055470 0x7f5e4c057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:52.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.023+0000 7f5e7084a700 1 --2- 172.21.15.32:0/1908340842 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5e4c055470 0x7f5e4c057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:52.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.023+0000 7f5e69ffb700 1 -- 172.21.15.32:0/1908340842 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5e6006ef80 con 0x7f5e6c0afd30 2020-12-05T17:31:52.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.023+0000 7f5e7084a700 1 --2- 172.21.15.32:0/1908340842 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5e4c055470 0x7f5e4c057930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f5e6400b000 tx=0x7f5e64009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.031+0000 7f5e69ffb700 1 -- 172.21.15.32:0/1908340842 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5e60045120 con 0x7f5e6c0afd30 2020-12-05T17:31:52.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.171+0000 7f5e7184c700 1 -- 172.21.15.32:0/1908340842 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f5e6c065890 con 0x7f5e6c0afd30 2020-12-05T17:31:52.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.171+0000 7f5e69ffb700 1 -- 172.21.15.32:0/1908340842 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7f5e60022070 con 0x7f5e6c0afd30 2020-12-05T17:31:52.174 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:31:52.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.171+0000 7f5e7184c700 1 -- 172.21.15.32:0/1908340842 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5e4c055470 msgr2=0x7f5e4c057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.171+0000 7f5e7184c700 1 --2- 172.21.15.32:0/1908340842 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5e4c055470 0x7f5e4c057930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f5e6400b000 tx=0x7f5e64009040).stop 2020-12-05T17:31:52.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.171+0000 7f5e7184c700 1 -- 172.21.15.32:0/1908340842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e6c0afd30 msgr2=0x7f5e6c148820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.171+0000 7f5e7184c700 1 --2- 172.21.15.32:0/1908340842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e6c0afd30 0x7f5e6c148820 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f5e60002f30 tx=0x7f5e60015000).stop 2020-12-05T17:31:52.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.171+0000 7f5e7184c700 1 -- 172.21.15.32:0/1908340842 shutdown_connections 2020-12-05T17:31:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.171+0000 7f5e7184c700 1 --2- 172.21.15.32:0/1908340842 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5e4c055470 0x7f5e4c057930 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.171+0000 7f5e7184c700 1 --2- 172.21.15.32:0/1908340842 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5e6c0b1dc0 0x7f5e6c065b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.171+0000 7f5e7184c700 1 --2- 172.21.15.32:0/1908340842 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e6c0af5f0 0x7f5e6c13f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.171+0000 7f5e7184c700 1 --2- 172.21.15.32:0/1908340842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e6c0afd30 0x7f5e6c148820 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.171+0000 7f5e7184c700 1 -- 172.21.15.32:0/1908340842 >> 172.21.15.32:0/1908340842 conn(0x7f5e6c0620b0 msgr2=0x7f5e6c145de0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:52.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.171+0000 7f5e7184c700 1 -- 172.21.15.32:0/1908340842 shutdown_connections 2020-12-05T17:31:52.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.171+0000 7f5e7184c700 1 -- 172.21.15.32:0/1908340842 wait complete. 2020-12-05T17:31:52.188 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:31:52.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.299+0000 7fb419218700 1 Processor -- start 2020-12-05T17:31:52.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.299+0000 7fb419218700 1 -- start start 2020-12-05T17:31:52.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.303+0000 7fb419218700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4141433d0 0x7fb41405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:52.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.303+0000 7fb419218700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4140b2df0 con 0x7fb41405feb0 2020-12-05T17:31:52.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.303+0000 7fb419218700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4140b3c90 con 0x7fb41405f3b0 2020-12-05T17:31:52.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.303+0000 7fb419218700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4140b2b50 con 0x7fb4141433d0 2020-12-05T17:31:52.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.303+0000 7fb4137fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb41405feb0 0x7fb414142cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35580/0 (socket says 172.21.15.32:35580) 2020-12-05T17:31:52.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.303+0000 7fb4137fe700 1 -- 172.21.15.32:0/396983918 learned_addr learned my addr 172.21.15.32:0/396983918 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:52.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.303+0000 7fb418a17700 1 --2- 172.21.15.32:0/396983918 >> v2:172.21.15.32:3300/0 conn(0x7fb4141433d0 0x7fb41405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:52.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.303+0000 7fb418a17700 1 -- 172.21.15.32:0/396983918 >> v1:172.21.15.61:6789/0 conn(0x7fb41405f3b0 legacy=0x7fb41413c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.303+0000 7fb418a17700 1 -- 172.21.15.32:0/396983918 >> v1:172.21.15.32:6789/0 conn(0x7fb41405feb0 legacy=0x7fb414142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.303+0000 7fb418a17700 1 -- 172.21.15.32:0/396983918 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb414142820 con 0x7fb4141433d0 2020-12-05T17:31:52.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.303+0000 7fb418a17700 1 --2- 172.21.15.32:0/396983918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4141433d0 0x7fb41405f790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fb3fc0010d0 tx=0x7fb40c006dc0).ready entity=mon.0 client_cookie=8cb5865290376739 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:52.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.303+0000 7fb412ffd700 1 -- 172.21.15.32:0/396983918 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb40c0096b0 con 0x7fb4141433d0 2020-12-05T17:31:52.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.303+0000 7fb412ffd700 1 -- 172.21.15.32:0/396983918 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40c009810 con 0x7fb4141433d0 2020-12-05T17:31:52.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.303+0000 7fb412ffd700 1 -- 172.21.15.32:0/396983918 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb40c009b70 con 0x7fb4141433d0 2020-12-05T17:31:52.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.303+0000 7fb419218700 1 -- 172.21.15.32:0/396983918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4141433d0 msgr2=0x7fb41405f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.303+0000 7fb419218700 1 --2- 172.21.15.32:0/396983918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4141433d0 0x7fb41405f790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fb3fc0010d0 tx=0x7fb40c006dc0).stop 2020-12-05T17:31:52.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.303+0000 7fb419218700 1 -- 172.21.15.32:0/396983918 shutdown_connections 2020-12-05T17:31:52.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.303+0000 7fb419218700 1 --2- 172.21.15.32:0/396983918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4141433d0 0x7fb41405f790 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.303+0000 7fb419218700 1 -- 172.21.15.32:0/396983918 >> 172.21.15.32:0/396983918 conn(0x7fb4140b8f20 msgr2=0x7fb4140b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:52.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.303+0000 7fb419218700 1 -- 172.21.15.32:0/396983918 shutdown_connections 2020-12-05T17:31:52.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.303+0000 7fb419218700 1 -- 172.21.15.32:0/396983918 wait complete. 2020-12-05T17:31:52.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.303+0000 7fb419218700 1 Processor -- start 2020-12-05T17:31:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.303+0000 7fb419218700 1 -- start start 2020-12-05T17:31:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.307+0000 7fb419218700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb41405f3b0 0x7fb414146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.307+0000 7fb419218700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb41405feb0 0x7fb4141465f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.307+0000 7fb419218700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4141433d0 0x7fb4141396e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.307+0000 7fb419218700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb414147200 con 0x7fb4141433d0 2020-12-05T17:31:52.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.307+0000 7fb419218700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb414145640 con 0x7fb41405feb0 2020-12-05T17:31:52.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.307+0000 7fb419218700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4141488b0 con 0x7fb41405f3b0 2020-12-05T17:31:52.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.307+0000 7fb418a17700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4141433d0 0x7fb4141396e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:52.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.307+0000 7fb418a17700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4141433d0 0x7fb4141396e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32940/0 (socket says 172.21.15.32:32940) 2020-12-05T17:31:52.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.307+0000 7fb418a17700 1 -- 172.21.15.32:0/3148616377 learned_addr learned my addr 172.21.15.32:0/3148616377 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:52.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.307+0000 7fb413fff700 1 --2- 172.21.15.32:0/3148616377 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb41405f3b0 0x7fb414146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:52.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.307+0000 7fb4137fe700 1 --2- 172.21.15.32:0/3148616377 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb41405feb0 0x7fb4141465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:52.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.307+0000 7fb418a17700 1 -- 172.21.15.32:0/3148616377 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb41405f3b0 msgr2=0x7fb414146100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.307+0000 7fb418a17700 1 --2- 172.21.15.32:0/3148616377 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb41405f3b0 0x7fb414146100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.307+0000 7fb418a17700 1 -- 172.21.15.32:0/3148616377 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb41405feb0 msgr2=0x7fb4141465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.307+0000 7fb418a17700 1 --2- 172.21.15.32:0/3148616377 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb41405feb0 0x7fb4141465f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.307+0000 7fb418a17700 1 -- 172.21.15.32:0/3148616377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4141458a0 con 0x7fb4141433d0 2020-12-05T17:31:52.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.307+0000 7fb418a17700 1 --2- 172.21.15.32:0/3148616377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4141433d0 0x7fb4141396e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fb3fc0010d0 tx=0x7fb40c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:52.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.307+0000 7fb4117fa700 1 -- 172.21.15.32:0/3148616377 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb40c011b80 con 0x7fb4141433d0 2020-12-05T17:31:52.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.307+0000 7fb4117fa700 1 -- 172.21.15.32:0/3148616377 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40c011ce0 con 0x7fb4141433d0 2020-12-05T17:31:52.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.307+0000 7fb4117fa700 1 -- 172.21.15.32:0/3148616377 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb40c01d640 con 0x7fb4141433d0 2020-12-05T17:31:52.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.307+0000 7fb419218700 1 -- 172.21.15.32:0/3148616377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb414066920 con 0x7fb4141433d0 2020-12-05T17:31:52.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.307+0000 7fb419218700 1 -- 172.21.15.32:0/3148616377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb414144290 con 0x7fb4141433d0 2020-12-05T17:31:52.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.311+0000 7fb419218700 1 -- 172.21.15.32:0/3148616377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb414057050 con 0x7fb4141433d0 2020-12-05T17:31:52.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.311+0000 7fb4117fa700 1 -- 172.21.15.32:0/3148616377 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb40c00f070 con 0x7fb4141433d0 2020-12-05T17:31:52.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.311+0000 7fb4117fa700 1 --2- 172.21.15.32:0/3148616377 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3ec055500 0x7fb3ec0579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:52.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.311+0000 7fb4117fa700 1 -- 172.21.15.32:0/3148616377 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb40c06f9a0 con 0x7fb4141433d0 2020-12-05T17:31:52.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.311+0000 7fb413fff700 1 --2- 172.21.15.32:0/3148616377 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3ec055500 0x7fb3ec0579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:52.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.311+0000 7fb413fff700 1 --2- 172.21.15.32:0/3148616377 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3ec055500 0x7fb3ec0579c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fb40400d000 tx=0x7fb404007ee0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:52.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.319+0000 7fb4117fa700 1 -- 172.21.15.32:0/3148616377 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb40c045ae0 con 0x7fb4141433d0 2020-12-05T17:31:52.463 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.459+0000 7fb419218700 1 -- 172.21.15.32:0/3148616377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb41405aab0 con 0x7fb4141433d0 2020-12-05T17:31:52.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.459+0000 7fb4117fa700 1 -- 172.21.15.32:0/3148616377 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb40c049100 con 0x7fb4141433d0 2020-12-05T17:31:52.464 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:31:52.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.463+0000 7fb419218700 1 -- 172.21.15.32:0/3148616377 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3ec055500 msgr2=0x7fb3ec0579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.463+0000 7fb419218700 1 --2- 172.21.15.32:0/3148616377 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3ec055500 0x7fb3ec0579c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fb40400d000 tx=0x7fb404007ee0).stop 2020-12-05T17:31:52.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.463+0000 7fb419218700 1 -- 172.21.15.32:0/3148616377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4141433d0 msgr2=0x7fb4141396e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.463+0000 7fb419218700 1 --2- 172.21.15.32:0/3148616377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4141433d0 0x7fb4141396e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fb3fc0010d0 tx=0x7fb40c015000).stop 2020-12-05T17:31:52.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.463+0000 7fb419218700 1 -- 172.21.15.32:0/3148616377 shutdown_connections 2020-12-05T17:31:52.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.463+0000 7fb419218700 1 --2- 172.21.15.32:0/3148616377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4141433d0 0x7fb4141396e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.463+0000 7fb419218700 1 --2- 172.21.15.32:0/3148616377 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3ec055500 0x7fb3ec0579c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.463+0000 7fb419218700 1 --2- 172.21.15.32:0/3148616377 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb41405f3b0 0x7fb414146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.463+0000 7fb419218700 1 --2- 172.21.15.32:0/3148616377 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb41405feb0 0x7fb4141465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.463+0000 7fb419218700 1 -- 172.21.15.32:0/3148616377 >> 172.21.15.32:0/3148616377 conn(0x7fb4140b8f20 msgr2=0x7fb4140b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.463+0000 7fb419218700 1 -- 172.21.15.32:0/3148616377 shutdown_connections 2020-12-05T17:31:52.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.463+0000 7fb419218700 1 -- 172.21.15.32:0/3148616377 wait complete. 2020-12-05T17:31:52.482 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:31:52.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.603+0000 7fdb5ffff700 1 Processor -- start 2020-12-05T17:31:52.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.603+0000 7fdb5ffff700 1 -- start start 2020-12-05T17:31:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb6005f550 0x7fdb600acb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5ffff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fdb60063040 0x7fdb6013fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5ffff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb60067110 con 0x7fdb600af090 2020-12-05T17:31:52.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5ffff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb6013f760 con 0x7fdb6005f550 2020-12-05T17:31:52.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5ffff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb6013fa00 con 0x7fdb60063040 2020-12-05T17:31:52.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb6005f550 0x7fdb600acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:52.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdb600af090 0x7fdb600af500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35596/0 (socket says 172.21.15.32:35596) 2020-12-05T17:31:52.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb6005f550 0x7fdb600acb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32944/0 (socket says 172.21.15.32:32944) 2020-12-05T17:31:52.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5f7fe700 1 -- 172.21.15.32:0/3309110897 learned_addr learned my addr 172.21.15.32:0/3309110897 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:52.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb567fc700 1 --2- 172.21.15.32:0/3309110897 >> v2:172.21.15.61:3300/0 conn(0x7fdb60063040 0x7fdb6013fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:52.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5effd700 1 -- 172.21.15.32:0/3309110897 >> v2:172.21.15.61:3300/0 conn(0x7fdb60063040 msgr2=0x7fdb6013fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:52.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5effd700 1 --2- 172.21.15.32:0/3309110897 >> v2:172.21.15.61:3300/0 conn(0x7fdb60063040 0x7fdb6013fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5effd700 1 -- 172.21.15.32:0/3309110897 >> v1:172.21.15.32:6789/0 conn(0x7fdb600af090 legacy=0x7fdb600af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5effd700 1 -- 172.21.15.32:0/3309110897 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb60066d90 con 0x7fdb6005f550 2020-12-05T17:31:52.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5e7fc700 1 -- 172.21.15.32:0/3309110897 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4024247805 0 0) 0x7fdb60067110 con 0x7fdb600af090 2020-12-05T17:31:52.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5effd700 1 --2- 172.21.15.32:0/3309110897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb6005f550 0x7fdb600acb80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fdb50002f30 tx=0x7fdb50006dc0).ready entity=mon.0 client_cookie=bf4dd2349f12b3be server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:52.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5e7fc700 1 -- 172.21.15.32:0/3309110897 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb500097b0 con 0x7fdb6005f550 2020-12-05T17:31:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5e7fc700 1 -- 172.21.15.32:0/3309110897 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb50009910 con 0x7fdb6005f550 2020-12-05T17:31:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5e7fc700 1 -- 172.21.15.32:0/3309110897 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb50009c70 con 0x7fdb6005f550 2020-12-05T17:31:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5ffff700 1 -- 172.21.15.32:0/3309110897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb6005f550 msgr2=0x7fdb600acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5ffff700 1 --2- 172.21.15.32:0/3309110897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb6005f550 0x7fdb600acb80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fdb50002f30 tx=0x7fdb50006dc0).stop 2020-12-05T17:31:52.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5ffff700 1 -- 172.21.15.32:0/3309110897 shutdown_connections 2020-12-05T17:31:52.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5ffff700 1 --2- 172.21.15.32:0/3309110897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb6005f550 0x7fdb600acb80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5ffff700 1 --2- 172.21.15.32:0/3309110897 >> v2:172.21.15.61:3300/0 conn(0x7fdb60063040 0x7fdb6013fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5ffff700 1 -- 172.21.15.32:0/3309110897 >> 172.21.15.32:0/3309110897 conn(0x7fdb600620b0 msgr2=0x7fdb60062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:52.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5ffff700 1 -- 172.21.15.32:0/3309110897 shutdown_connections 2020-12-05T17:31:52.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.607+0000 7fdb5ffff700 1 -- 172.21.15.32:0/3309110897 wait complete. 2020-12-05T17:31:52.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5ffff700 1 Processor -- start 2020-12-05T17:31:52.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5ffff700 1 -- start start 2020-12-05T17:31:52.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb6005f550 0x7fdb600b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:52.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5ffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb60063040 0x7fdb6010f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:52.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5ffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb600af090 0x7fdb601515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:52.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5ffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb60144f50 con 0x7fdb6005f550 2020-12-05T17:31:52.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5ffff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6010f5f0 con 0x7fdb600af090 2020-12-05T17:31:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5ffff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb60142940 con 0x7fdb60063040 2020-12-05T17:31:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb6005f550 0x7fdb600b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb6005f550 0x7fdb600b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32950/0 (socket says 172.21.15.32:32950) 2020-12-05T17:31:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5effd700 1 -- 172.21.15.32:0/4285453445 learned_addr learned my addr 172.21.15.32:0/4285453445 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5f7fe700 1 --2- 172.21.15.32:0/4285453445 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb600af090 0x7fdb601515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:52.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb567fc700 1 --2- 172.21.15.32:0/4285453445 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb60063040 0x7fdb6010f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:52.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5effd700 1 -- 172.21.15.32:0/4285453445 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb60063040 msgr2=0x7fdb6010f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5effd700 1 --2- 172.21.15.32:0/4285453445 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb60063040 0x7fdb6010f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5effd700 1 -- 172.21.15.32:0/4285453445 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb600af090 msgr2=0x7fdb601515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5effd700 1 --2- 172.21.15.32:0/4285453445 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb600af090 0x7fdb601515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5effd700 1 -- 172.21.15.32:0/4285453445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb601426a0 con 0x7fdb6005f550 2020-12-05T17:31:52.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5f7fe700 1 --2- 172.21.15.32:0/4285453445 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb600af090 0x7fdb601515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:52.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5effd700 1 --2- 172.21.15.32:0/4285453445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb6005f550 0x7fdb600b2170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fdb600b26e0 tx=0x7fdb50015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:52.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5cff9700 1 -- 172.21.15.32:0/4285453445 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb50011c20 con 0x7fdb6005f550 2020-12-05T17:31:52.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5ffff700 1 -- 172.21.15.32:0/4285453445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb60154cd0 con 0x7fdb6005f550 2020-12-05T17:31:52.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5cff9700 1 -- 172.21.15.32:0/4285453445 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb50011d80 con 0x7fdb6005f550 2020-12-05T17:31:52.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5ffff700 1 -- 172.21.15.32:0/4285453445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb60142270 con 0x7fdb6005f550 2020-12-05T17:31:52.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.611+0000 7fdb5cff9700 1 -- 172.21.15.32:0/4285453445 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb5001d630 con 0x7fdb6005f550 2020-12-05T17:31:52.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.615+0000 7fdb5ffff700 1 -- 172.21.15.32:0/4285453445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb60154e60 con 0x7fdb6005f550 2020-12-05T17:31:52.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.615+0000 7fdb5cff9700 1 -- 172.21.15.32:0/4285453445 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fdb50017070 con 0x7fdb6005f550 2020-12-05T17:31:52.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.615+0000 7fdb5cff9700 1 --2- 172.21.15.32:0/4285453445 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb440553f0 0x7fdb440578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:52.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.615+0000 7fdb5cff9700 1 -- 172.21.15.32:0/4285453445 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fdb5006ecc0 con 0x7fdb6005f550 2020-12-05T17:31:52.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.615+0000 7fdb567fc700 1 --2- 172.21.15.32:0/4285453445 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb440553f0 0x7fdb440578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:52.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.615+0000 7fdb567fc700 1 --2- 172.21.15.32:0/4285453445 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb440553f0 0x7fdb440578b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fdb48002000 tx=0x7fdb48008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:52.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.623+0000 7fdb5cff9700 1 -- 172.21.15.32:0/4285453445 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fdb50044dc0 con 0x7fdb6005f550 2020-12-05T17:31:52.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.763+0000 7fdb5ffff700 1 -- 172.21.15.32:0/4285453445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdb60057050 con 0x7fdb6005f550 2020-12-05T17:31:52.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.767+0000 7fdb5cff9700 1 -- 172.21.15.32:0/4285453445 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7fdb500220a0 con 0x7fdb6005f550 2020-12-05T17:31:52.769 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:31:52.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.767+0000 7fdb5ffff700 1 -- 172.21.15.32:0/4285453445 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb440553f0 msgr2=0x7fdb440578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.767+0000 7fdb5ffff700 1 --2- 172.21.15.32:0/4285453445 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb440553f0 0x7fdb440578b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fdb48002000 tx=0x7fdb48008040).stop 2020-12-05T17:31:52.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.767+0000 7fdb5ffff700 1 -- 172.21.15.32:0/4285453445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb6005f550 msgr2=0x7fdb600b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.767+0000 7fdb5ffff700 1 --2- 172.21.15.32:0/4285453445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb6005f550 0x7fdb600b2170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fdb600b26e0 tx=0x7fdb50015000).stop 2020-12-05T17:31:52.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.767+0000 7fdb5ffff700 1 -- 172.21.15.32:0/4285453445 shutdown_connections 2020-12-05T17:31:52.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.767+0000 7fdb5ffff700 1 --2- 172.21.15.32:0/4285453445 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb600af090 0x7fdb601515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.767+0000 7fdb5ffff700 1 --2- 172.21.15.32:0/4285453445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb6005f550 0x7fdb600b2170 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.767+0000 7fdb5ffff700 1 --2- 172.21.15.32:0/4285453445 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb440553f0 0x7fdb440578b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.767+0000 7fdb5ffff700 1 --2- 172.21.15.32:0/4285453445 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb60063040 0x7fdb6010f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.767+0000 7fdb5ffff700 1 -- 172.21.15.32:0/4285453445 >> 172.21.15.32:0/4285453445 conn(0x7fdb600620b0 msgr2=0x7fdb600658b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.767+0000 7fdb5ffff700 1 -- 172.21.15.32:0/4285453445 shutdown_connections 2020-12-05T17:31:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.767+0000 7fdb5ffff700 1 -- 172.21.15.32:0/4285453445 wait complete. 2020-12-05T17:31:52.783 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:31:52.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.903+0000 7ff52359e700 1 Processor -- start 2020-12-05T17:31:52.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.903+0000 7ff52359e700 1 -- start start 2020-12-05T17:31:52.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.903+0000 7ff52359e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff5241433d0 0x7ff52405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:52.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.903+0000 7ff52359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5240b2df0 con 0x7ff52405feb0 2020-12-05T17:31:52.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.903+0000 7ff52359e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5240b3c90 con 0x7ff52405f3b0 2020-12-05T17:31:52.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.903+0000 7ff52359e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5240b2b50 con 0x7ff5241433d0 2020-12-05T17:31:52.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.903+0000 7ff52259c700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7ff52405f3b0 0x7ff52413c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:48246/0 (socket says 172.21.15.32:48246) 2020-12-05T17:31:52.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.903+0000 7ff52259c700 1 -- 172.21.15.32:0/3596626903 learned_addr learned my addr 172.21.15.32:0/3596626903 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:52.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.903+0000 7ff522d9d700 1 --2- 172.21.15.32:0/3596626903 >> v2:172.21.15.61:3300/0 conn(0x7ff5241433d0 0x7ff52405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:52.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.903+0000 7ff519d9b700 1 --1- 172.21.15.32:0/3596626903 >> v1:172.21.15.32:6789/0 conn(0x7ff52405feb0 0x7ff524142cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35608/0 (socket says 172.21.15.32:35608) 2020-12-05T17:31:52.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.903+0000 7ff521d9b700 1 -- 172.21.15.32:0/3596626903 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4094620514 0 0) 0x7ff5240b2df0 con 0x7ff52405feb0 2020-12-05T17:31:52.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.903+0000 7ff521d9b700 1 -- 172.21.15.32:0/3596626903 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff504003230 con 0x7ff52405feb0 2020-12-05T17:31:52.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.903+0000 7ff522d9d700 1 -- 172.21.15.32:0/3596626903 >> v1:172.21.15.61:6789/0 conn(0x7ff52405f3b0 legacy=0x7ff52413c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.903+0000 7ff521d9b700 1 -- 172.21.15.32:0/3596626903 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1459564825 0 0) 0x7ff5240b3c90 con 0x7ff52405f3b0 2020-12-05T17:31:52.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.903+0000 7ff522d9d700 1 -- 172.21.15.32:0/3596626903 >> v1:172.21.15.32:6789/0 conn(0x7ff52405feb0 legacy=0x7ff524142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.903+0000 7ff522d9d700 1 -- 172.21.15.32:0/3596626903 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff524142820 con 0x7ff5241433d0 2020-12-05T17:31:52.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.903+0000 7ff522d9d700 1 --2- 172.21.15.32:0/3596626903 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff5241433d0 0x7ff52405f790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7ff51c0032b0 tx=0x7ff51c0091a0).ready entity=mon.1 client_cookie=59b8ca8bff119319 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:52.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.907+0000 7ff521d9b700 1 -- 172.21.15.32:0/3596626903 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff51c00b9a0 con 0x7ff5241433d0 2020-12-05T17:31:52.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.907+0000 7ff521d9b700 1 -- 172.21.15.32:0/3596626903 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51c00bb00 con 0x7ff5241433d0 2020-12-05T17:31:52.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.907+0000 7ff521d9b700 1 -- 172.21.15.32:0/3596626903 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff51c00bc70 con 0x7ff5241433d0 2020-12-05T17:31:52.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.907+0000 7ff52359e700 1 -- 172.21.15.32:0/3596626903 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff5241433d0 msgr2=0x7ff52405f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.907+0000 7ff52359e700 1 --2- 172.21.15.32:0/3596626903 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff5241433d0 0x7ff52405f790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7ff51c0032b0 tx=0x7ff51c0091a0).stop 2020-12-05T17:31:52.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.907+0000 7ff52359e700 1 -- 172.21.15.32:0/3596626903 shutdown_connections 2020-12-05T17:31:52.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.907+0000 7ff52359e700 1 --2- 172.21.15.32:0/3596626903 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff5241433d0 0x7ff52405f790 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.907+0000 7ff52359e700 1 -- 172.21.15.32:0/3596626903 >> 172.21.15.32:0/3596626903 conn(0x7ff5240b8f20 msgr2=0x7ff5240b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:52.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.907+0000 7ff52359e700 1 -- 172.21.15.32:0/3596626903 shutdown_connections 2020-12-05T17:31:52.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.907+0000 7ff52359e700 1 -- 172.21.15.32:0/3596626903 wait complete. 2020-12-05T17:31:52.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.907+0000 7ff52359e700 1 Processor -- start 2020-12-05T17:31:52.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.907+0000 7ff52359e700 1 -- start start 2020-12-05T17:31:52.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.907+0000 7ff52359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff52405f3b0 0x7ff52413dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:52.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.907+0000 7ff52359e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff52405feb0 0x7ff52413b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:52.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.907+0000 7ff52359e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff5241433d0 0x7ff52413bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:52.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.907+0000 7ff52359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff524145720 con 0x7ff52405f3b0 2020-12-05T17:31:52.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.907+0000 7ff52359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52413c050 con 0x7ff52405feb0 2020-12-05T17:31:52.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.907+0000 7ff52359e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52413e940 con 0x7ff5241433d0 2020-12-05T17:31:52.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.907+0000 7ff52259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff52405f3b0 0x7ff52413dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:52.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.911+0000 7ff52259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff52405f3b0 0x7ff52413dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32964/0 (socket says 172.21.15.32:32964) 2020-12-05T17:31:52.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.911+0000 7ff52259c700 1 -- 172.21.15.32:0/1122188381 learned_addr learned my addr 172.21.15.32:0/1122188381 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:52.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.911+0000 7ff522d9d700 1 --2- 172.21.15.32:0/1122188381 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff5241433d0 0x7ff52413bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:52.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.911+0000 7ff519d9b700 1 --2- 172.21.15.32:0/1122188381 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff52405feb0 0x7ff52413b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:52.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.911+0000 7ff52259c700 1 -- 172.21.15.32:0/1122188381 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff5241433d0 msgr2=0x7ff52413bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.911+0000 7ff52259c700 1 --2- 172.21.15.32:0/1122188381 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff5241433d0 0x7ff52413bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.911+0000 7ff52259c700 1 -- 172.21.15.32:0/1122188381 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff52405feb0 msgr2=0x7ff52413b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:52.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.911+0000 7ff52259c700 1 --2- 172.21.15.32:0/1122188381 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff52405feb0 0x7ff52413b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:52.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.911+0000 7ff52259c700 1 -- 172.21.15.32:0/1122188381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5241520d0 con 0x7ff52405f3b0 2020-12-05T17:31:52.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.911+0000 7ff522d9d700 1 --2- 172.21.15.32:0/1122188381 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff5241433d0 0x7ff52413bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:31:52.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.911+0000 7ff519d9b700 1 --2- 172.21.15.32:0/1122188381 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff52405feb0 0x7ff52413b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:31:52.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.911+0000 7ff52259c700 1 --2- 172.21.15.32:0/1122188381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff52405f3b0 0x7ff52413dfc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7ff52413db70 tx=0x7ff514007b20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:52.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.911+0000 7ff51bfff700 1 -- 172.21.15.32:0/1122188381 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff514007320 con 0x7ff52405f3b0 2020-12-05T17:31:52.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.911+0000 7ff51bfff700 1 -- 172.21.15.32:0/1122188381 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff514007480 con 0x7ff52405f3b0 2020-12-05T17:31:52.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.911+0000 7ff52359e700 1 -- 172.21.15.32:0/1122188381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff524152370 con 0x7ff52405f3b0 2020-12-05T17:31:52.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.911+0000 7ff51bfff700 1 -- 172.21.15.32:0/1122188381 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5140057d0 con 0x7ff52405f3b0 2020-12-05T17:31:52.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.911+0000 7ff52359e700 1 -- 172.21.15.32:0/1122188381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52413e4b0 con 0x7ff52405f3b0 2020-12-05T17:31:52.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.911+0000 7ff52359e700 1 -- 172.21.15.32:0/1122188381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff524152500 con 0x7ff52405f3b0 2020-12-05T17:31:52.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.911+0000 7ff51bfff700 1 -- 172.21.15.32:0/1122188381 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff514010030 con 0x7ff52405f3b0 2020-12-05T17:31:52.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.911+0000 7ff51bfff700 1 --2- 172.21.15.32:0/1122188381 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff508055350 0x7ff508057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:52.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.911+0000 7ff519d9b700 1 --2- 172.21.15.32:0/1122188381 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff508055350 0x7ff508057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:52.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.911+0000 7ff51bfff700 1 -- 172.21.15.32:0/1122188381 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff51406df20 con 0x7ff52405f3b0 2020-12-05T17:31:52.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.915+0000 7ff519d9b700 1 --2- 172.21.15.32:0/1122188381 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff508055350 0x7ff508057810 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7ff50c008000 tx=0x7ff50c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:52.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:52.919+0000 7ff51bfff700 1 -- 172.21.15.32:0/1122188381 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff514043fc0 con 0x7ff52405f3b0 2020-12-05T17:31:53.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.055+0000 7ff52359e700 1 -- 172.21.15.32:0/1122188381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff524057050 con 0x7ff52405f3b0 2020-12-05T17:31:53.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.055+0000 7ff51bfff700 1 -- 172.21.15.32:0/1122188381 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff5140475e0 con 0x7ff52405f3b0 2020-12-05T17:31:53.059 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:31:53.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.059+0000 7ff52359e700 1 -- 172.21.15.32:0/1122188381 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff508055350 msgr2=0x7ff508057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:53.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.059+0000 7ff52359e700 1 --2- 172.21.15.32:0/1122188381 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff508055350 0x7ff508057810 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7ff50c008000 tx=0x7ff50c009040).stop 2020-12-05T17:31:53.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.059+0000 7ff52359e700 1 -- 172.21.15.32:0/1122188381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff52405f3b0 msgr2=0x7ff52413dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:53.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.059+0000 7ff52359e700 1 --2- 172.21.15.32:0/1122188381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff52405f3b0 0x7ff52413dfc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7ff52413db70 tx=0x7ff514007b20).stop 2020-12-05T17:31:53.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.059+0000 7ff52359e700 1 -- 172.21.15.32:0/1122188381 shutdown_connections 2020-12-05T17:31:53.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.059+0000 7ff52359e700 1 --2- 172.21.15.32:0/1122188381 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff508055350 0x7ff508057810 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:53.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.059+0000 7ff52359e700 1 --2- 172.21.15.32:0/1122188381 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff5241433d0 0x7ff52413bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:53.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.059+0000 7ff52359e700 1 --2- 172.21.15.32:0/1122188381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff52405f3b0 0x7ff52413dfc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:53.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.059+0000 7ff52359e700 1 --2- 172.21.15.32:0/1122188381 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff52405feb0 0x7ff52413b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:53.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.059+0000 7ff52359e700 1 -- 172.21.15.32:0/1122188381 >> 172.21.15.32:0/1122188381 conn(0x7ff5240b8f20 msgr2=0x7ff524061910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:53.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.059+0000 7ff52359e700 1 -- 172.21.15.32:0/1122188381 shutdown_connections 2020-12-05T17:31:53.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.059+0000 7ff52359e700 1 -- 172.21.15.32:0/1122188381 wait complete. 2020-12-05T17:31:53.073 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:31:53.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.179+0000 7f6aae644700 1 Processor -- start 2020-12-05T17:31:53.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.179+0000 7f6aae644700 1 -- start start 2020-12-05T17:31:53.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aae644700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6aa805feb0 0x7f6aa80b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:53.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aae644700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa805fad0 con 0x7f6aa8062210 2020-12-05T17:31:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aae644700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa8062670 con 0x7f6aa805f6f0 2020-12-05T17:31:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aae644700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa80b3360 con 0x7f6aa805feb0 2020-12-05T17:31:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aace41700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6aa805feb0 0x7f6aa80b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aace41700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6aa805feb0 0x7f6aa80b15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32972/0 (socket says 172.21.15.32:32972) 2020-12-05T17:31:53.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aade43700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6aa8062210 0x7f6aa8146970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35624/0 (socket says 172.21.15.32:35624) 2020-12-05T17:31:53.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aace41700 1 -- 172.21.15.32:0/1978107582 learned_addr learned my addr 172.21.15.32:0/1978107582 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:53.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aace41700 1 -- 172.21.15.32:0/1978107582 >> v1:172.21.15.61:6789/0 conn(0x7f6aa805f6f0 legacy=0x7f6aa8061b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:53.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aace41700 1 -- 172.21.15.32:0/1978107582 >> v1:172.21.15.32:6789/0 conn(0x7f6aa8062210 legacy=0x7f6aa8146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:53.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aace41700 1 -- 172.21.15.32:0/1978107582 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa80b4cc0 con 0x7f6aa805feb0 2020-12-05T17:31:53.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aace41700 1 --2- 172.21.15.32:0/1978107582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6aa805feb0 0x7f6aa80b15a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f6a90002f30 tx=0x7f6a90006e80).ready entity=mon.0 client_cookie=6edba115eeec8c51 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:53.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aa7fff700 1 -- 172.21.15.32:0/1978107582 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a900097f0 con 0x7f6aa805feb0 2020-12-05T17:31:53.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aa7fff700 1 -- 172.21.15.32:0/1978107582 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a90009950 con 0x7f6aa805feb0 2020-12-05T17:31:53.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aae644700 1 -- 172.21.15.32:0/1978107582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6aa805feb0 msgr2=0x7f6aa80b15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:53.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aa7fff700 1 -- 172.21.15.32:0/1978107582 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a90009cb0 con 0x7f6aa805feb0 2020-12-05T17:31:53.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aae644700 1 --2- 172.21.15.32:0/1978107582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6aa805feb0 0x7f6aa80b15a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f6a90002f30 tx=0x7f6a90006e80).stop 2020-12-05T17:31:53.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aae644700 1 -- 172.21.15.32:0/1978107582 shutdown_connections 2020-12-05T17:31:53.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aae644700 1 --2- 172.21.15.32:0/1978107582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6aa805feb0 0x7f6aa80b15a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:53.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aae644700 1 -- 172.21.15.32:0/1978107582 >> 172.21.15.32:0/1978107582 conn(0x7f6aa80b8ee0 msgr2=0x7f6aa80b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:53.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aae644700 1 -- 172.21.15.32:0/1978107582 shutdown_connections 2020-12-05T17:31:53.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aae644700 1 -- 172.21.15.32:0/1978107582 wait complete. 2020-12-05T17:31:53.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aae644700 1 Processor -- start 2020-12-05T17:31:53.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.183+0000 7f6aae644700 1 -- start start 2020-12-05T17:31:53.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aae644700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6aa805f6f0 0x7f6aa81433d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:53.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aae644700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6aa805feb0 0x7f6aa8066400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:53.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aae644700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6aa8062210 0x7f6aa80668f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:53.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aae644700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa8146210 con 0x7f6aa805feb0 2020-12-05T17:31:53.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aace41700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6aa805feb0 0x7f6aa8066400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:53.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aae644700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa8149bd0 con 0x7f6aa8062210 2020-12-05T17:31:53.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aae644700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa8147860 con 0x7f6aa805f6f0 2020-12-05T17:31:53.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aace41700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6aa805feb0 0x7f6aa8066400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32980/0 (socket says 172.21.15.32:32980) 2020-12-05T17:31:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aace41700 1 -- 172.21.15.32:0/1217048024 learned_addr learned my addr 172.21.15.32:0/1217048024 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aad642700 1 --2- 172.21.15.32:0/1217048024 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6aa805f6f0 0x7f6aa81433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aace41700 1 -- 172.21.15.32:0/1217048024 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6aa805f6f0 msgr2=0x7f6aa81433d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aace41700 1 --2- 172.21.15.32:0/1217048024 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6aa805f6f0 0x7f6aa81433d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aade43700 1 --2- 172.21.15.32:0/1217048024 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6aa8062210 0x7f6aa80668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:53.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aace41700 1 -- 172.21.15.32:0/1217048024 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6aa8062210 msgr2=0x7f6aa80668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:53.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aace41700 1 --2- 172.21.15.32:0/1217048024 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6aa8062210 0x7f6aa80668f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:53.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aace41700 1 -- 172.21.15.32:0/1217048024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa814a190 con 0x7f6aa805feb0 2020-12-05T17:31:53.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aad642700 1 --2- 172.21.15.32:0/1217048024 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6aa805f6f0 0x7f6aa81433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:31:53.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aade43700 1 --2- 172.21.15.32:0/1217048024 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6aa8062210 0x7f6aa80668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:31:53.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aace41700 1 --2- 172.21.15.32:0/1217048024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6aa805feb0 0x7f6aa8066400 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f6a90002f30 tx=0x7f6a90015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:53.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aa67fc700 1 -- 172.21.15.32:0/1217048024 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a90011e30 con 0x7f6aa805feb0 2020-12-05T17:31:53.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aa67fc700 1 -- 172.21.15.32:0/1217048024 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9001e460 con 0x7f6aa805feb0 2020-12-05T17:31:53.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aae644700 1 -- 172.21.15.32:0/1217048024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa8066de0 con 0x7f6aa805feb0 2020-12-05T17:31:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aa67fc700 1 -- 172.21.15.32:0/1217048024 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a9001e7c0 con 0x7f6aa805feb0 2020-12-05T17:31:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aae644700 1 -- 172.21.15.32:0/1217048024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa8155b00 con 0x7f6aa805feb0 2020-12-05T17:31:53.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.187+0000 7f6aae644700 1 -- 172.21.15.32:0/1217048024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa8057050 con 0x7f6aa805feb0 2020-12-05T17:31:53.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.191+0000 7f6aa67fc700 1 -- 172.21.15.32:0/1217048024 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6a9001a040 con 0x7f6aa805feb0 2020-12-05T17:31:53.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.191+0000 7f6aa67fc700 1 --2- 172.21.15.32:0/1217048024 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6a8c0553b0 0x7f6a8c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:53.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.191+0000 7f6aa67fc700 1 -- 172.21.15.32:0/1217048024 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f6a9006f2f0 con 0x7f6aa805feb0 2020-12-05T17:31:53.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.191+0000 7f6aad642700 1 --2- 172.21.15.32:0/1217048024 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6a8c0553b0 0x7f6a8c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:53.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.191+0000 7f6aad642700 1 --2- 172.21.15.32:0/1217048024 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6a8c0553b0 0x7f6a8c057870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f6aa0008000 tx=0x7f6aa0009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:53.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.195+0000 7f6aa67fc700 1 -- 172.21.15.32:0/1217048024 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6a90045430 con 0x7f6aa805feb0 2020-12-05T17:31:53.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.383+0000 7f6aae644700 1 -- 172.21.15.32:0/1217048024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6aa8149e70 con 0x7f6aa805feb0 2020-12-05T17:31:53.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.383+0000 7f6aa67fc700 1 -- 172.21.15.32:0/1217048024 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2134 (secure 0 0 0) 0x7f6a90023090 con 0x7f6aa805feb0 2020-12-05T17:31:53.387 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:31:53.387 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4023797731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4023797731},{"type":"v1","addr":"172.21.15.61:6833","nonce":4023797731}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:31:41.611214+0000","modified":"2020-12-05T17:31:42.614053+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3503848021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3503848021},{"type":"v1","addr":"172.21.15.32:6835","nonce":3503848021}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-12-05T17:31:53.389 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.387+0000 7f6aae644700 1 -- 172.21.15.32:0/1217048024 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6a8c0553b0 msgr2=0x7f6a8c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:53.389 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.387+0000 7f6aae644700 1 --2- 172.21.15.32:0/1217048024 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6a8c0553b0 0x7f6a8c057870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f6aa0008000 tx=0x7f6aa0009040).stop 2020-12-05T17:31:53.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.387+0000 7f6aae644700 1 -- 172.21.15.32:0/1217048024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6aa805feb0 msgr2=0x7f6aa8066400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:53.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.387+0000 7f6aae644700 1 --2- 172.21.15.32:0/1217048024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6aa805feb0 0x7f6aa8066400 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f6a90002f30 tx=0x7f6a90015000).stop 2020-12-05T17:31:53.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.387+0000 7f6aae644700 1 -- 172.21.15.32:0/1217048024 shutdown_connections 2020-12-05T17:31:53.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.387+0000 7f6aae644700 1 --2- 172.21.15.32:0/1217048024 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6aa8062210 0x7f6aa80668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:53.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.387+0000 7f6aae644700 1 --2- 172.21.15.32:0/1217048024 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6a8c0553b0 0x7f6a8c057870 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:53.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.387+0000 7f6aae644700 1 --2- 172.21.15.32:0/1217048024 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6aa805f6f0 0x7f6aa81433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:53.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.387+0000 7f6aae644700 1 --2- 172.21.15.32:0/1217048024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6aa805feb0 0x7f6aa8066400 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:53.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.387+0000 7f6aae644700 1 -- 172.21.15.32:0/1217048024 >> 172.21.15.32:0/1217048024 conn(0x7f6aa80b8ee0 msgr2=0x7f6aa80b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:53.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.387+0000 7f6aae644700 1 -- 172.21.15.32:0/1217048024 shutdown_connections 2020-12-05T17:31:53.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.387+0000 7f6aae644700 1 -- 172.21.15.32:0/1217048024 wait complete. 2020-12-05T17:31:53.393 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 5 2020-12-05T17:31:53.607 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:31:41.611214+0000', 'modified': '2020-12-05T17:31:42.614053+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4275}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4275': {'gid': 4275, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/3503848021', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 3503848021}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 3503848021}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:31:53.608 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:31:53.608 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:31:53.738 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":4275,"want_state":"up:active","state":"up:active","rank_uptime":12.114592319,"mdsmap_epoch":5,"osdmap_epoch":25,"osdmap_epoch_barrier":25,"uptime":18.520568934} 2020-12-05T17:31:53.738 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 4275, "mdsmap_epoch": 5, "osdmap_epoch": 25, "osdmap_epoch_barrier": 25, "rank_uptime": 12.114592319, "state": "up:active", "uptime": 18.520568934, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:31:53.739 INFO:teuthology.run_tasks:Running task ceph-fuse... 2020-12-05T17:31:53.758 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2020-12-05T17:31:53.758 INFO:tasks.ceph_fuse:config is None 2020-12-05T17:31:53.759 INFO:tasks.ceph_fuse:new config is {'client.0': None, 'client.1': None, 'client.2': None} 2020-12-05T17:31:53.759 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2020-12-05T17:31:53.760 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:31:53.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.871+0000 7f583313b700 1 Processor -- start 2020-12-05T17:31:53.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.871+0000 7f583313b700 1 -- start start 2020-12-05T17:31:53.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.871+0000 7f583313b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f582c05fe00 0x7f582c0b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:53.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.871+0000 7f583313b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f582c13c730 0x7f582c13cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:53.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.871+0000 7f583313b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f582c0601e0 con 0x7f582c142d00 2020-12-05T17:31:53.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.871+0000 7f583313b700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f582c061480 con 0x7f582c05fe00 2020-12-05T17:31:53.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.871+0000 7f583313b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582c0ad510 con 0x7f582c13c730 2020-12-05T17:31:53.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.871+0000 7f58316d8700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f582c142d00 0x7f582c1431b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35636/0 (socket says 172.21.15.32:35636) 2020-12-05T17:31:53.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.871+0000 7f58316d8700 1 -- 172.21.15.32:0/606194111 learned_addr learned my addr 172.21.15.32:0/606194111 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:53.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.871+0000 7f582bfff700 1 --2- 172.21.15.32:0/606194111 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f582c13c730 0x7f582c13cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:53.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.871+0000 7f5830ed7700 1 --2- 172.21.15.32:0/606194111 >> v2:172.21.15.61:3300/0 conn(0x7f582c05fe00 0x7f582c0b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:53.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.871+0000 7f582b7fe700 1 -- 172.21.15.32:0/606194111 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2174140090 0 0) 0x7f582c0601e0 con 0x7f582c142d00 2020-12-05T17:31:53.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.871+0000 7f582b7fe700 1 -- 172.21.15.32:0/606194111 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5818003230 con 0x7f582c142d00 2020-12-05T17:31:53.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.871+0000 7f582bfff700 1 -- 172.21.15.32:0/606194111 >> v2:172.21.15.61:3300/0 conn(0x7f582c05fe00 msgr2=0x7f582c0b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:53.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.871+0000 7f582bfff700 1 --2- 172.21.15.32:0/606194111 >> v2:172.21.15.61:3300/0 conn(0x7f582c05fe00 0x7f582c0b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:53.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.871+0000 7f582bfff700 1 -- 172.21.15.32:0/606194111 >> v1:172.21.15.32:6789/0 conn(0x7f582c142d00 legacy=0x7f582c1431b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:53.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.871+0000 7f582bfff700 1 -- 172.21.15.32:0/606194111 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582c0af250 con 0x7f582c13c730 2020-12-05T17:31:53.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.871+0000 7f582bfff700 1 --2- 172.21.15.32:0/606194111 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f582c13c730 0x7f582c13cb90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f5820002f30 tx=0x7f5820006dc0).ready entity=mon.2 client_cookie=f6423c62140f6910 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:53.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.871+0000 7f582b7fe700 1 -- 172.21.15.32:0/606194111 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58200096a0 con 0x7f582c13c730 2020-12-05T17:31:53.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.871+0000 7f582b7fe700 1 -- 172.21.15.32:0/606194111 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5820009800 con 0x7f582c13c730 2020-12-05T17:31:53.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.875+0000 7f583313b700 1 -- 172.21.15.32:0/606194111 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f582c13c730 msgr2=0x7f582c13cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:53.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.875+0000 7f582b7fe700 1 -- 172.21.15.32:0/606194111 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5820009b60 con 0x7f582c13c730 2020-12-05T17:31:53.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.875+0000 7f583313b700 1 --2- 172.21.15.32:0/606194111 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f582c13c730 0x7f582c13cb90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f5820002f30 tx=0x7f5820006dc0).stop 2020-12-05T17:31:53.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.875+0000 7f583313b700 1 -- 172.21.15.32:0/606194111 shutdown_connections 2020-12-05T17:31:53.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.875+0000 7f583313b700 1 --2- 172.21.15.32:0/606194111 >> v2:172.21.15.61:3300/0 conn(0x7f582c05fe00 0x7f582c0b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:53.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.875+0000 7f583313b700 1 --2- 172.21.15.32:0/606194111 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f582c13c730 0x7f582c13cb90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:53.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.875+0000 7f583313b700 1 -- 172.21.15.32:0/606194111 >> 172.21.15.32:0/606194111 conn(0x7f582c0b57c0 msgr2=0x7f582c05f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:53.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.875+0000 7f583313b700 1 -- 172.21.15.32:0/606194111 shutdown_connections 2020-12-05T17:31:53.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.875+0000 7f583313b700 1 -- 172.21.15.32:0/606194111 wait complete. 2020-12-05T17:31:53.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.875+0000 7f583313b700 1 Processor -- start 2020-12-05T17:31:53.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.875+0000 7f583313b700 1 -- start start 2020-12-05T17:31:53.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f583313b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f582c05fe00 0x7f582c0b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:53.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f583313b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f582c13c730 0x7f582c136f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:53.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f583313b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f582c142d00 0x7f582c153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:53.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f583313b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c14f450 con 0x7f582c05fe00 2020-12-05T17:31:53.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f583313b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c0b4620 con 0x7f582c13c730 2020-12-05T17:31:53.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f583313b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582c0b40e0 con 0x7f582c142d00 2020-12-05T17:31:53.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f5830ed7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f582c05fe00 0x7f582c0b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:53.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f5830ed7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f582c05fe00 0x7f582c0b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32992/0 (socket says 172.21.15.32:32992) 2020-12-05T17:31:53.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f5830ed7700 1 -- 172.21.15.32:0/4138247752 learned_addr learned my addr 172.21.15.32:0/4138247752 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:53.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f582bfff700 1 --2- 172.21.15.32:0/4138247752 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f582c13c730 0x7f582c136f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:53.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f58316d8700 1 --2- 172.21.15.32:0/4138247752 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f582c142d00 0x7f582c153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:53.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f5830ed7700 1 -- 172.21.15.32:0/4138247752 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f582c142d00 msgr2=0x7f582c153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:53.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f5830ed7700 1 --2- 172.21.15.32:0/4138247752 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f582c142d00 0x7f582c153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:53.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f5830ed7700 1 -- 172.21.15.32:0/4138247752 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f582c13c730 msgr2=0x7f582c136f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:53.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f5830ed7700 1 --2- 172.21.15.32:0/4138247752 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f582c13c730 0x7f582c136f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:53.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f5830ed7700 1 -- 172.21.15.32:0/4138247752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582c0b4380 con 0x7f582c05fe00 2020-12-05T17:31:53.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f582bfff700 1 --2- 172.21.15.32:0/4138247752 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f582c13c730 0x7f582c136f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:53.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f5830ed7700 1 --2- 172.21.15.32:0/4138247752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f582c05fe00 0x7f582c0b3ab0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f581c00c000 tx=0x7f581c006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:53.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f5829ffb700 1 -- 172.21.15.32:0/4138247752 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f581c00ca80 con 0x7f582c05fe00 2020-12-05T17:31:53.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f5829ffb700 1 -- 172.21.15.32:0/4138247752 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581c00cbe0 con 0x7f582c05fe00 2020-12-05T17:31:53.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f583313b700 1 -- 172.21.15.32:0/4138247752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582c13cb50 con 0x7f582c05fe00 2020-12-05T17:31:53.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f5829ffb700 1 -- 172.21.15.32:0/4138247752 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f581c0123f0 con 0x7f582c05fe00 2020-12-05T17:31:53.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f583313b700 1 -- 172.21.15.32:0/4138247752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582c066570 con 0x7f582c05fe00 2020-12-05T17:31:53.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f5829ffb700 1 -- 172.21.15.32:0/4138247752 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f581c012550 con 0x7f582c05fe00 2020-12-05T17:31:53.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f583313b700 1 -- 172.21.15.32:0/4138247752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582c13cd40 con 0x7f582c05fe00 2020-12-05T17:31:53.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f5829ffb700 1 --2- 172.21.15.32:0/4138247752 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f580c0553b0 0x7f580c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:53.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f5829ffb700 1 -- 172.21.15.32:0/4138247752 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f581c06e4d0 con 0x7f582c05fe00 2020-12-05T17:31:53.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.879+0000 7f582bfff700 1 --2- 172.21.15.32:0/4138247752 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f580c0553b0 0x7f580c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:53.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.883+0000 7f582bfff700 1 --2- 172.21.15.32:0/4138247752 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f580c0553b0 0x7f580c057870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f5820019000 tx=0x7f5820015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:53.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:53.887+0000 7f5829ffb700 1 -- 172.21.15.32:0/4138247752 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f581c044610 con 0x7f582c05fe00 2020-12-05T17:31:54.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.031+0000 7f583313b700 1 -- 172.21.15.32:0/4138247752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f582c057050 con 0x7f582c05fe00 2020-12-05T17:31:54.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.031+0000 7f5829ffb700 1 -- 172.21.15.32:0/4138247752 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f581c047c30 con 0x7f582c05fe00 2020-12-05T17:31:54.037 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:31:54.037 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:31:41.610914+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:31:36.706361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-12-05T17:31:39.298269+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:31:54.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.035+0000 7f583313b700 1 -- 172.21.15.32:0/4138247752 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f580c0553b0 msgr2=0x7f580c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:54.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.035+0000 7f583313b700 1 --2- 172.21.15.32:0/4138247752 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f580c0553b0 0x7f580c057870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f5820019000 tx=0x7f5820015000).stop 2020-12-05T17:31:54.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.035+0000 7f583313b700 1 -- 172.21.15.32:0/4138247752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f582c05fe00 msgr2=0x7f582c0b3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:54.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.035+0000 7f583313b700 1 --2- 172.21.15.32:0/4138247752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f582c05fe00 0x7f582c0b3ab0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f581c00c000 tx=0x7f581c006dc0).stop 2020-12-05T17:31:54.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.035+0000 7f583313b700 1 -- 172.21.15.32:0/4138247752 shutdown_connections 2020-12-05T17:31:54.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.035+0000 7f583313b700 1 --2- 172.21.15.32:0/4138247752 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f580c0553b0 0x7f580c057870 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.035+0000 7f583313b700 1 --2- 172.21.15.32:0/4138247752 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f582c142d00 0x7f582c153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.035+0000 7f583313b700 1 --2- 172.21.15.32:0/4138247752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f582c05fe00 0x7f582c0b3ab0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.035+0000 7f583313b700 1 --2- 172.21.15.32:0/4138247752 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f582c13c730 0x7f582c136f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.035+0000 7f583313b700 1 -- 172.21.15.32:0/4138247752 >> 172.21.15.32:0/4138247752 conn(0x7f582c0b57c0 msgr2=0x7f582c13f680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:54.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.035+0000 7f583313b700 1 -- 172.21.15.32:0/4138247752 shutdown_connections 2020-12-05T17:31:54.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.035+0000 7f583313b700 1 -- 172.21.15.32:0/4138247752 wait complete. 2020-12-05T17:31:54.050 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:31:36.706361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:31:39.298269+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:31:54.051 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:31:54.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.171+0000 7f688e916700 1 Processor -- start 2020-12-05T17:31:54.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.171+0000 7f688e916700 1 -- start start 2020-12-05T17:31:54.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.171+0000 7f688e916700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68880af0d0 0x7f688813daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:54.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.171+0000 7f688e916700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f688813df90 0x7f68880637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:54.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.171+0000 7f688e916700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68880b0d70 0x7f6888063ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:54.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.171+0000 7f688e916700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f688805f5c0 con 0x7f68880b0d70 2020-12-05T17:31:54.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688e916700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68880af4b0 con 0x7f688813df90 2020-12-05T17:31:54.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688e916700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688805f7f0 con 0x7f68880af0d0 2020-12-05T17:31:54.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688e115700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68880b0d70 0x7f6888063ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:54.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688e115700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68880b0d70 0x7f6888063ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33004/0 (socket says 172.21.15.32:33004) 2020-12-05T17:31:54.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688e115700 1 -- 172.21.15.32:0/610324717 learned_addr learned my addr 172.21.15.32:0/610324717 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:54.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688d914700 1 --2- 172.21.15.32:0/610324717 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68880af0d0 0x7f688813daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:54.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688d113700 1 --2- 172.21.15.32:0/610324717 >> v2:172.21.15.61:3300/0 conn(0x7f688813df90 0x7f68880637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:54.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688e115700 1 -- 172.21.15.32:0/610324717 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68880af0d0 msgr2=0x7f688813daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:54.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688e115700 1 --2- 172.21.15.32:0/610324717 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68880af0d0 0x7f688813daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688e115700 1 -- 172.21.15.32:0/610324717 >> v2:172.21.15.61:3300/0 conn(0x7f688813df90 msgr2=0x7f68880637f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:54.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688e115700 1 --2- 172.21.15.32:0/610324717 >> v2:172.21.15.61:3300/0 conn(0x7f688813df90 0x7f68880637f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688e115700 1 -- 172.21.15.32:0/610324717 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6888066ae0 con 0x7f68880b0d70 2020-12-05T17:31:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688e115700 1 --2- 172.21.15.32:0/610324717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68880b0d70 0x7f6888063ce0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f68800032b0 tx=0x7f6880009270).ready entity=mon.0 client_cookie=dd098a7c46a330b5 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688c912700 1 -- 172.21.15.32:0/610324717 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6880009e60 con 0x7f68880b0d70 2020-12-05T17:31:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688c912700 1 -- 172.21.15.32:0/610324717 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688000bb50 con 0x7f68880b0d70 2020-12-05T17:31:54.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688c912700 1 -- 172.21.15.32:0/610324717 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f688000bcc0 con 0x7f68880b0d70 2020-12-05T17:31:54.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688e916700 1 -- 172.21.15.32:0/610324717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68880b0d70 msgr2=0x7f6888063ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:54.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688e916700 1 --2- 172.21.15.32:0/610324717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68880b0d70 0x7f6888063ce0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f68800032b0 tx=0x7f6880009270).stop 2020-12-05T17:31:54.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688e916700 1 -- 172.21.15.32:0/610324717 shutdown_connections 2020-12-05T17:31:54.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688e916700 1 --2- 172.21.15.32:0/610324717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68880b0d70 0x7f6888063ce0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688e916700 1 --2- 172.21.15.32:0/610324717 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68880af0d0 0x7f688813daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688e916700 1 --2- 172.21.15.32:0/610324717 >> v2:172.21.15.61:3300/0 conn(0x7f688813df90 0x7f68880637f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688e916700 1 -- 172.21.15.32:0/610324717 >> 172.21.15.32:0/610324717 conn(0x7f68880620b0 msgr2=0x7f6888062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:54.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688e916700 1 -- 172.21.15.32:0/610324717 shutdown_connections 2020-12-05T17:31:54.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.175+0000 7f688e916700 1 -- 172.21.15.32:0/610324717 wait complete. 2020-12-05T17:31:54.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.179+0000 7f688e916700 1 Processor -- start 2020-12-05T17:31:54.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.179+0000 7f688e916700 1 -- start start 2020-12-05T17:31:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.179+0000 7f688e916700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68880af0d0 0x7f6888147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.179+0000 7f688e916700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68880b0d70 0x7f6888147530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.179+0000 7f688e916700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f688813df90 0x7f688813ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:54.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.179+0000 7f688e916700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6888110350 con 0x7f688813df90 2020-12-05T17:31:54.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.179+0000 7f688e916700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68881105b0 con 0x7f68880b0d70 2020-12-05T17:31:54.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.179+0000 7f688e916700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6888147a90 con 0x7f68880af0d0 2020-12-05T17:31:54.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.179+0000 7f688e115700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f688813df90 0x7f688813ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:54.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.179+0000 7f688d914700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68880af0d0 0x7f6888147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:54.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.179+0000 7f688e115700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f688813df90 0x7f688813ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33010/0 (socket says 172.21.15.32:33010) 2020-12-05T17:31:54.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.179+0000 7f688e115700 1 -- 172.21.15.32:0/1930041353 learned_addr learned my addr 172.21.15.32:0/1930041353 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:54.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.179+0000 7f688d113700 1 --2- 172.21.15.32:0/1930041353 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68880b0d70 0x7f6888147530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.183+0000 7f688e115700 1 -- 172.21.15.32:0/1930041353 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68880af0d0 msgr2=0x7f6888147040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.183+0000 7f688e115700 1 --2- 172.21.15.32:0/1930041353 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68880af0d0 0x7f6888147040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.183+0000 7f688e115700 1 -- 172.21.15.32:0/1930041353 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68880b0d70 msgr2=0x7f6888147530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.183+0000 7f688e115700 1 --2- 172.21.15.32:0/1930041353 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68880b0d70 0x7f6888147530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.183+0000 7f688e115700 1 -- 172.21.15.32:0/1930041353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6888147c50 con 0x7f688813df90 2020-12-05T17:31:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.183+0000 7f688e115700 1 --2- 172.21.15.32:0/1930041353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f688813df90 0x7f688813ce70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f68800032b0 tx=0x7f6880016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.183+0000 7f6876ffd700 1 -- 172.21.15.32:0/1930041353 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f688001e610 con 0x7f688813df90 2020-12-05T17:31:54.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.183+0000 7f6876ffd700 1 -- 172.21.15.32:0/1930041353 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688001e770 con 0x7f688813df90 2020-12-05T17:31:54.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.183+0000 7f6876ffd700 1 -- 172.21.15.32:0/1930041353 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f688001ead0 con 0x7f688813df90 2020-12-05T17:31:54.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.183+0000 7f688e916700 1 -- 172.21.15.32:0/1930041353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6888146230 con 0x7f688813df90 2020-12-05T17:31:54.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.183+0000 7f688e916700 1 -- 172.21.15.32:0/1930041353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6888145e40 con 0x7f688813df90 2020-12-05T17:31:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.183+0000 7f688e916700 1 -- 172.21.15.32:0/1930041353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6888110720 con 0x7f688813df90 2020-12-05T17:31:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.183+0000 7f6876ffd700 1 -- 172.21.15.32:0/1930041353 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f688000f070 con 0x7f688813df90 2020-12-05T17:31:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.183+0000 7f6876ffd700 1 --2- 172.21.15.32:0/1930041353 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6868055500 0x7f68680579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:54.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.183+0000 7f688d914700 1 --2- 172.21.15.32:0/1930041353 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6868055500 0x7f68680579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:54.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.183+0000 7f6876ffd700 1 -- 172.21.15.32:0/1930041353 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f688006f650 con 0x7f688813df90 2020-12-05T17:31:54.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.187+0000 7f688d914700 1 --2- 172.21.15.32:0/1930041353 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6868055500 0x7f68680579c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f688400b000 tx=0x7f6884006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:54.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.191+0000 7f6876ffd700 1 -- 172.21.15.32:0/1930041353 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f68800457f0 con 0x7f688813df90 2020-12-05T17:31:54.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.335+0000 7f688e916700 1 -- 172.21.15.32:0/1930041353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6888057050 con 0x7f688813df90 2020-12-05T17:31:54.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.335+0000 7f6876ffd700 1 -- 172.21.15.32:0/1930041353 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7f68800150b0 con 0x7f688813df90 2020-12-05T17:31:54.341 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:31:54.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.339+0000 7f688e916700 1 -- 172.21.15.32:0/1930041353 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6868055500 msgr2=0x7f68680579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:54.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.339+0000 7f688e916700 1 --2- 172.21.15.32:0/1930041353 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6868055500 0x7f68680579c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f688400b000 tx=0x7f6884006dc0).stop 2020-12-05T17:31:54.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.339+0000 7f688e916700 1 -- 172.21.15.32:0/1930041353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f688813df90 msgr2=0x7f688813ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:54.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.339+0000 7f688e916700 1 --2- 172.21.15.32:0/1930041353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f688813df90 0x7f688813ce70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f68800032b0 tx=0x7f6880016000).stop 2020-12-05T17:31:54.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.339+0000 7f688e916700 1 -- 172.21.15.32:0/1930041353 shutdown_connections 2020-12-05T17:31:54.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.339+0000 7f688e916700 1 --2- 172.21.15.32:0/1930041353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f688813df90 0x7f688813ce70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.339+0000 7f688e916700 1 --2- 172.21.15.32:0/1930041353 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6868055500 0x7f68680579c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.339+0000 7f688e916700 1 --2- 172.21.15.32:0/1930041353 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68880af0d0 0x7f6888147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.339+0000 7f688e916700 1 --2- 172.21.15.32:0/1930041353 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68880b0d70 0x7f6888147530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.339+0000 7f688e916700 1 -- 172.21.15.32:0/1930041353 >> 172.21.15.32:0/1930041353 conn(0x7f68880620b0 msgr2=0x7f68880b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:54.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.339+0000 7f688e916700 1 -- 172.21.15.32:0/1930041353 shutdown_connections 2020-12-05T17:31:54.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.339+0000 7f688e916700 1 -- 172.21.15.32:0/1930041353 wait complete. 2020-12-05T17:31:54.354 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:31:54.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.471+0000 7f36615c9700 1 Processor -- start 2020-12-05T17:31:54.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.471+0000 7f36615c9700 1 -- start start 2020-12-05T17:31:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.471+0000 7f36615c9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f365c0b2000 0x7f365c145e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.471+0000 7f36615c9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f365c0af4b0 0x7f365c1462f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.471+0000 7f36615c9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f365c0b23e0 con 0x7f365c061780 2020-12-05T17:31:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.471+0000 7f36615c9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f365c0ae7b0 con 0x7f365c0af4b0 2020-12-05T17:31:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.471+0000 7f36615c9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f365c0aea50 con 0x7f365c0b2000 2020-12-05T17:31:54.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.471+0000 7f365b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f365c0af4b0 0x7f365c1462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:54.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.471+0000 7f3660dc8700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f365c061780 0x7f365c0afe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35662/0 (socket says 172.21.15.32:35662) 2020-12-05T17:31:54.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.471+0000 7f3660dc8700 1 -- 172.21.15.32:0/2596055289 learned_addr learned my addr 172.21.15.32:0/2596055289 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:54.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.471+0000 7f365bfff700 1 --2- 172.21.15.32:0/2596055289 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f365c0b2000 0x7f365c145e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:54.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.471+0000 7f365b7fe700 1 -- 172.21.15.32:0/2596055289 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f365c0b2000 msgr2=0x7f365c145e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:54.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.471+0000 7f365b7fe700 1 --2- 172.21.15.32:0/2596055289 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f365c0b2000 0x7f365c145e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.471+0000 7f365b7fe700 1 -- 172.21.15.32:0/2596055289 >> v1:172.21.15.32:6789/0 conn(0x7f365c061780 legacy=0x7f365c0afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:54.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.471+0000 7f365affd700 1 -- 172.21.15.32:0/2596055289 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3232129359 0 0) 0x7f365c0b23e0 con 0x7f365c061780 2020-12-05T17:31:54.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.471+0000 7f365b7fe700 1 -- 172.21.15.32:0/2596055289 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c0b0f60 con 0x7f365c0af4b0 2020-12-05T17:31:54.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f365b7fe700 1 --2- 172.21.15.32:0/2596055289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f365c0af4b0 0x7f365c1462f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f3644002f30 tx=0x7f3644006dc0).ready entity=mon.0 client_cookie=9e2678a166dbf0f3 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:54.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f365affd700 1 -- 172.21.15.32:0/2596055289 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3644009760 con 0x7f365c0af4b0 2020-12-05T17:31:54.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f365affd700 1 -- 172.21.15.32:0/2596055289 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36440098c0 con 0x7f365c0af4b0 2020-12-05T17:31:54.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f36615c9700 1 -- 172.21.15.32:0/2596055289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f365c0af4b0 msgr2=0x7f365c1462f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:54.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f365affd700 1 -- 172.21.15.32:0/2596055289 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3644009c20 con 0x7f365c0af4b0 2020-12-05T17:31:54.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f36615c9700 1 --2- 172.21.15.32:0/2596055289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f365c0af4b0 0x7f365c1462f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f3644002f30 tx=0x7f3644006dc0).stop 2020-12-05T17:31:54.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f36615c9700 1 -- 172.21.15.32:0/2596055289 shutdown_connections 2020-12-05T17:31:54.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f36615c9700 1 --2- 172.21.15.32:0/2596055289 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f365c0b2000 0x7f365c145e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f36615c9700 1 --2- 172.21.15.32:0/2596055289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f365c0af4b0 0x7f365c1462f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f36615c9700 1 -- 172.21.15.32:0/2596055289 >> 172.21.15.32:0/2596055289 conn(0x7f365c0b5620 msgr2=0x7f365c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:54.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f36615c9700 1 -- 172.21.15.32:0/2596055289 shutdown_connections 2020-12-05T17:31:54.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f36615c9700 1 -- 172.21.15.32:0/2596055289 wait complete. 2020-12-05T17:31:54.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f36615c9700 1 Processor -- start 2020-12-05T17:31:54.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f36615c9700 1 -- start start 2020-12-05T17:31:54.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f36615c9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f365c061780 0x7f365c063f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:54.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f36615c9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f365c0af4b0 0x7f365c065b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:54.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f36615c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f365c0b2000 0x7f365c066070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:54.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f36615c9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365c064e10 con 0x7f365c0b2000 2020-12-05T17:31:54.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f36615c9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365c156670 con 0x7f365c0af4b0 2020-12-05T17:31:54.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f36615c9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f365c156910 con 0x7f365c061780 2020-12-05T17:31:54.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f3660dc8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f365c0b2000 0x7f365c066070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:54.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f3660dc8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f365c0b2000 0x7f365c066070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33022/0 (socket says 172.21.15.32:33022) 2020-12-05T17:31:54.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f3660dc8700 1 -- 172.21.15.32:0/1603923910 learned_addr learned my addr 172.21.15.32:0/1603923910 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:54.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f365bfff700 1 --2- 172.21.15.32:0/1603923910 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f365c061780 0x7f365c063f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:54.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f365b7fe700 1 --2- 172.21.15.32:0/1603923910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f365c0af4b0 0x7f365c065b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:54.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f3660dc8700 1 -- 172.21.15.32:0/1603923910 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f365c061780 msgr2=0x7f365c063f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:54.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f3660dc8700 1 --2- 172.21.15.32:0/1603923910 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f365c061780 0x7f365c063f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f3660dc8700 1 -- 172.21.15.32:0/1603923910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f365c0af4b0 msgr2=0x7f365c065b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:54.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f3660dc8700 1 --2- 172.21.15.32:0/1603923910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f365c0af4b0 0x7f365c065b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f3660dc8700 1 -- 172.21.15.32:0/1603923910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c064880 con 0x7f365c0b2000 2020-12-05T17:31:54.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.475+0000 7f365b7fe700 1 --2- 172.21.15.32:0/1603923910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f365c0af4b0 0x7f365c065b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:54.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.479+0000 7f3660dc8700 1 --2- 172.21.15.32:0/1603923910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f365c0b2000 0x7f365c066070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f3654000b80 tx=0x7f3654007a10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:54.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.479+0000 7f36597fa700 1 -- 172.21.15.32:0/1603923910 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36540072e0 con 0x7f365c0b2000 2020-12-05T17:31:54.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.479+0000 7f36615c9700 1 -- 172.21.15.32:0/1603923910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f365c111cb0 con 0x7f365c0b2000 2020-12-05T17:31:54.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.479+0000 7f36597fa700 1 -- 172.21.15.32:0/1603923910 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3654007440 con 0x7f365c0b2000 2020-12-05T17:31:54.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.479+0000 7f36615c9700 1 -- 172.21.15.32:0/1603923910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f365c065200 con 0x7f365c0b2000 2020-12-05T17:31:54.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.479+0000 7f36597fa700 1 -- 172.21.15.32:0/1603923910 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36540057b0 con 0x7f365c0b2000 2020-12-05T17:31:54.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.479+0000 7f36615c9700 1 -- 172.21.15.32:0/1603923910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f365c057050 con 0x7f365c0b2000 2020-12-05T17:31:54.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.479+0000 7f36597fa700 1 -- 172.21.15.32:0/1603923910 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f365401a060 con 0x7f365c0b2000 2020-12-05T17:31:54.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.479+0000 7f36597fa700 1 --2- 172.21.15.32:0/1603923910 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3634055440 0x7f3634057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:54.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.479+0000 7f36597fa700 1 -- 172.21.15.32:0/1603923910 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f365406d940 con 0x7f365c0b2000 2020-12-05T17:31:54.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.479+0000 7f365bfff700 1 --2- 172.21.15.32:0/1603923910 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3634055440 0x7f3634057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:54.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.479+0000 7f365bfff700 1 --2- 172.21.15.32:0/1603923910 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3634055440 0x7f3634057900 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f364c00b000 tx=0x7f364c0091a0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:54.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.487+0000 7f36597fa700 1 -- 172.21.15.32:0/1603923910 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3654043a40 con 0x7f365c0b2000 2020-12-05T17:31:54.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.627+0000 7f36615c9700 1 -- 172.21.15.32:0/1603923910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f365c065620 con 0x7f365c0b2000 2020-12-05T17:31:54.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.627+0000 7f36597fa700 1 -- 172.21.15.32:0/1603923910 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3654018070 con 0x7f365c0b2000 2020-12-05T17:31:54.630 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:31:54.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.627+0000 7f36615c9700 1 -- 172.21.15.32:0/1603923910 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3634055440 msgr2=0x7f3634057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:54.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.627+0000 7f36615c9700 1 --2- 172.21.15.32:0/1603923910 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3634055440 0x7f3634057900 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f364c00b000 tx=0x7f364c0091a0).stop 2020-12-05T17:31:54.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.627+0000 7f36615c9700 1 -- 172.21.15.32:0/1603923910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f365c0b2000 msgr2=0x7f365c066070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:54.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.627+0000 7f36615c9700 1 --2- 172.21.15.32:0/1603923910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f365c0b2000 0x7f365c066070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f3654000b80 tx=0x7f3654007a10).stop 2020-12-05T17:31:54.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.627+0000 7f36615c9700 1 -- 172.21.15.32:0/1603923910 shutdown_connections 2020-12-05T17:31:54.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.627+0000 7f36615c9700 1 --2- 172.21.15.32:0/1603923910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f365c0b2000 0x7f365c066070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.627+0000 7f36615c9700 1 --2- 172.21.15.32:0/1603923910 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3634055440 0x7f3634057900 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.627+0000 7f36615c9700 1 --2- 172.21.15.32:0/1603923910 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f365c061780 0x7f365c063f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.627+0000 7f36615c9700 1 --2- 172.21.15.32:0/1603923910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f365c0af4b0 0x7f365c065b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.627+0000 7f36615c9700 1 -- 172.21.15.32:0/1603923910 >> 172.21.15.32:0/1603923910 conn(0x7f365c0b5620 msgr2=0x7f365c0b0120 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:54.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.627+0000 7f36615c9700 1 -- 172.21.15.32:0/1603923910 shutdown_connections 2020-12-05T17:31:54.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.627+0000 7f36615c9700 1 -- 172.21.15.32:0/1603923910 wait complete. 2020-12-05T17:31:54.644 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:31:54.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.759+0000 7fa05128c700 1 Processor -- start 2020-12-05T17:31:54.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.759+0000 7fa05128c700 1 -- start start 2020-12-05T17:31:54.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.759+0000 7fa05128c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa04c05feb0 0x7fa04c05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:54.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.759+0000 7fa05128c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa04c0b2df0 con 0x7fa04c0b2fb0 2020-12-05T17:31:54.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.759+0000 7fa05128c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa04c0b3c90 con 0x7fa04c05f3b0 2020-12-05T17:31:54.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.759+0000 7fa05128c700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa04c0b2b50 con 0x7fa04c05feb0 2020-12-05T17:31:54.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.763+0000 7fa050a8b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa04c0b2fb0 0x7fa04c142cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35680/0 (socket says 172.21.15.32:35680) 2020-12-05T17:31:54.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.763+0000 7fa050a8b700 1 -- 172.21.15.32:0/2977409190 learned_addr learned my addr 172.21.15.32:0/2977409190 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:54.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.763+0000 7fa04b7fe700 1 --2- 172.21.15.32:0/2977409190 >> v2:172.21.15.61:3300/0 conn(0x7fa04c05feb0 0x7fa04c05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:54.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.763+0000 7fa04affd700 1 -- 172.21.15.32:0/2977409190 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 582962562 0 0) 0x7fa04c0b2df0 con 0x7fa04c0b2fb0 2020-12-05T17:31:54.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.763+0000 7fa04affd700 1 -- 172.21.15.32:0/2977409190 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa02c003230 con 0x7fa04c0b2fb0 2020-12-05T17:31:54.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.763+0000 7fa04affd700 1 -- 172.21.15.32:0/2977409190 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3832717111 0 0) 0x7fa02c003230 con 0x7fa04c0b2fb0 2020-12-05T17:31:54.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.763+0000 7fa04b7fe700 1 -- 172.21.15.32:0/2977409190 >> v1:172.21.15.61:6789/0 conn(0x7fa04c05f3b0 legacy=0x7fa04c13c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:54.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.763+0000 7fa04b7fe700 1 -- 172.21.15.32:0/2977409190 >> v1:172.21.15.32:6789/0 conn(0x7fa04c0b2fb0 legacy=0x7fa04c142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:54.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.763+0000 7fa04b7fe700 1 -- 172.21.15.32:0/2977409190 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04c142820 con 0x7fa04c05feb0 2020-12-05T17:31:54.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.763+0000 7fa04b7fe700 1 --2- 172.21.15.32:0/2977409190 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa04c05feb0 0x7fa04c05f790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fa044001bf0 tx=0x7fa034006dc0).ready entity=mon.1 client_cookie=992ef877d0bba35c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:54.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.763+0000 7fa04affd700 1 -- 172.21.15.32:0/2977409190 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa044001a50 con 0x7fa04c05feb0 2020-12-05T17:31:54.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.763+0000 7fa04affd700 1 -- 172.21.15.32:0/2977409190 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa034009620 con 0x7fa04c05feb0 2020-12-05T17:31:54.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.763+0000 7fa04affd700 1 -- 172.21.15.32:0/2977409190 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa034009790 con 0x7fa04c05feb0 2020-12-05T17:31:54.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.763+0000 7fa05128c700 1 -- 172.21.15.32:0/2977409190 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa04c05feb0 msgr2=0x7fa04c05f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:54.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.763+0000 7fa05128c700 1 --2- 172.21.15.32:0/2977409190 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa04c05feb0 0x7fa04c05f790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fa044001bf0 tx=0x7fa034006dc0).stop 2020-12-05T17:31:54.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.763+0000 7fa05128c700 1 -- 172.21.15.32:0/2977409190 shutdown_connections 2020-12-05T17:31:54.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.763+0000 7fa05128c700 1 --2- 172.21.15.32:0/2977409190 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa04c05feb0 0x7fa04c05f790 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.763+0000 7fa05128c700 1 -- 172.21.15.32:0/2977409190 >> 172.21.15.32:0/2977409190 conn(0x7fa04c0b8f20 msgr2=0x7fa04c0b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:54.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.763+0000 7fa05128c700 1 -- 172.21.15.32:0/2977409190 shutdown_connections 2020-12-05T17:31:54.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.763+0000 7fa05128c700 1 -- 172.21.15.32:0/2977409190 wait complete. 2020-12-05T17:31:54.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa05128c700 1 Processor -- start 2020-12-05T17:31:54.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa05128c700 1 -- start start 2020-12-05T17:31:54.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa05128c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa04c05f3b0 0x7fa04c13e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:54.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa05128c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa04c05feb0 0x7fa04c155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:54.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa05128c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa04c0b2fb0 0x7fa04c155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:54.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa05128c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c155a10 con 0x7fa04c05feb0 2020-12-05T17:31:54.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa05128c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c143b70 con 0x7fa04c05f3b0 2020-12-05T17:31:54.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa05128c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c143e10 con 0x7fa04c0b2fb0 2020-12-05T17:31:54.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa04b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa04c05feb0 0x7fa04c155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:54.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa04b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa04c05feb0 0x7fa04c155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33038/0 (socket says 172.21.15.32:33038) 2020-12-05T17:31:54.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa04b7fe700 1 -- 172.21.15.32:0/2730655013 learned_addr learned my addr 172.21.15.32:0/2730655013 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:54.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa050a8b700 1 --2- 172.21.15.32:0/2730655013 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa04c0b2fb0 0x7fa04c155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:54.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa04bfff700 1 --2- 172.21.15.32:0/2730655013 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa04c05f3b0 0x7fa04c13e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:54.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa04b7fe700 1 -- 172.21.15.32:0/2730655013 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa04c0b2fb0 msgr2=0x7fa04c155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:54.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa04b7fe700 1 --2- 172.21.15.32:0/2730655013 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa04c0b2fb0 0x7fa04c155520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa04b7fe700 1 -- 172.21.15.32:0/2730655013 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa04c05f3b0 msgr2=0x7fa04c13e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:54.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa04b7fe700 1 --2- 172.21.15.32:0/2730655013 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa04c05f3b0 0x7fa04c13e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa04b7fe700 1 -- 172.21.15.32:0/2730655013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04c146b20 con 0x7fa04c05feb0 2020-12-05T17:31:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa04b7fe700 1 --2- 172.21.15.32:0/2730655013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa04c05feb0 0x7fa04c155030 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fa044001bf0 tx=0x7fa04c0b9f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa0497fa700 1 -- 172.21.15.32:0/2730655013 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa034013de0 con 0x7fa04c05feb0 2020-12-05T17:31:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa0497fa700 1 -- 172.21.15.32:0/2730655013 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03401e400 con 0x7fa04c05feb0 2020-12-05T17:31:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa05128c700 1 -- 172.21.15.32:0/2730655013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa04c13edf0 con 0x7fa04c05feb0 2020-12-05T17:31:54.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa0497fa700 1 -- 172.21.15.32:0/2730655013 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa03401e760 con 0x7fa04c05feb0 2020-12-05T17:31:54.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.767+0000 7fa05128c700 1 -- 172.21.15.32:0/2730655013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa04c154a80 con 0x7fa04c05feb0 2020-12-05T17:31:54.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.771+0000 7fa05128c700 1 -- 172.21.15.32:0/2730655013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa04c146ec0 con 0x7fa04c05feb0 2020-12-05T17:31:54.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.771+0000 7fa0497fa700 1 -- 172.21.15.32:0/2730655013 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa034025030 con 0x7fa04c05feb0 2020-12-05T17:31:54.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.771+0000 7fa0497fa700 1 --2- 172.21.15.32:0/2730655013 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa030055330 0x7fa0300577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:54.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.771+0000 7fa0497fa700 1 -- 172.21.15.32:0/2730655013 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa03406ef00 con 0x7fa04c05feb0 2020-12-05T17:31:54.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.771+0000 7fa04bfff700 1 --2- 172.21.15.32:0/2730655013 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa030055330 0x7fa0300577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:54.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.771+0000 7fa04bfff700 1 --2- 172.21.15.32:0/2730655013 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa030055330 0x7fa0300577f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fa03c00d000 tx=0x7fa03c007b40).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:54.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.779+0000 7fa0497fa700 1 -- 172.21.15.32:0/2730655013 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa04c146ec0 con 0x7fa04c05feb0 2020-12-05T17:31:54.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.919+0000 7fa05128c700 1 -- 172.21.15.32:0/2730655013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa04c146ec0 con 0x7fa04c05feb0 2020-12-05T17:31:54.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.919+0000 7fa0497fa700 1 -- 172.21.15.32:0/2730655013 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7fa04c146ec0 con 0x7fa04c05feb0 2020-12-05T17:31:54.924 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:31:54.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.923+0000 7fa05128c700 1 -- 172.21.15.32:0/2730655013 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa030055330 msgr2=0x7fa0300577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:54.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.923+0000 7fa05128c700 1 --2- 172.21.15.32:0/2730655013 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa030055330 0x7fa0300577f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fa03c00d000 tx=0x7fa03c007b40).stop 2020-12-05T17:31:54.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.923+0000 7fa05128c700 1 -- 172.21.15.32:0/2730655013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa04c05feb0 msgr2=0x7fa04c155030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:54.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.923+0000 7fa05128c700 1 --2- 172.21.15.32:0/2730655013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa04c05feb0 0x7fa04c155030 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fa044001bf0 tx=0x7fa04c0b9f00).stop 2020-12-05T17:31:54.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.923+0000 7fa05128c700 1 -- 172.21.15.32:0/2730655013 shutdown_connections 2020-12-05T17:31:54.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.923+0000 7fa05128c700 1 --2- 172.21.15.32:0/2730655013 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa030055330 0x7fa0300577f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.923+0000 7fa05128c700 1 --2- 172.21.15.32:0/2730655013 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa04c0b2fb0 0x7fa04c155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.923+0000 7fa05128c700 1 --2- 172.21.15.32:0/2730655013 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa04c05f3b0 0x7fa04c13e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.923+0000 7fa05128c700 1 --2- 172.21.15.32:0/2730655013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa04c05feb0 0x7fa04c155030 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:54.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.923+0000 7fa05128c700 1 -- 172.21.15.32:0/2730655013 >> 172.21.15.32:0/2730655013 conn(0x7fa04c0b8f20 msgr2=0x7fa04c0b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:54.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.923+0000 7fa05128c700 1 -- 172.21.15.32:0/2730655013 shutdown_connections 2020-12-05T17:31:54.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:54.923+0000 7fa05128c700 1 -- 172.21.15.32:0/2730655013 wait complete. 2020-12-05T17:31:54.944 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:31:55.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.063+0000 7f9988ede700 1 Processor -- start 2020-12-05T17:31:55.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.063+0000 7f9988ede700 1 -- start start 2020-12-05T17:31:55.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.063+0000 7f9988ede700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f99840667f0 0x7f99840aef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:55.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.063+0000 7f9988ede700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99840ad9b0 con 0x7f99840b2960 2020-12-05T17:31:55.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.063+0000 7f9988ede700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f998413f4f0 con 0x7f99840ad280 2020-12-05T17:31:55.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.063+0000 7f9988ede700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99840b32c0 con 0x7f99840667f0 2020-12-05T17:31:55.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.063+0000 7f99837fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f99840b2960 0x7f99840660e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35690/0 (socket says 172.21.15.32:35690) 2020-12-05T17:31:55.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.063+0000 7f99837fe700 1 -- 172.21.15.32:0/616318454 learned_addr learned my addr 172.21.15.32:0/616318454 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:55.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.063+0000 7f9983fff700 1 --2- 172.21.15.32:0/616318454 >> v2:172.21.15.32:3300/0 conn(0x7f99840667f0 0x7f99840aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:55.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.063+0000 7f99827fc700 1 -- 172.21.15.32:0/616318454 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1045240181 0 0) 0x7f99840ad9b0 con 0x7f99840b2960 2020-12-05T17:31:55.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.063+0000 7f99827fc700 1 -- 172.21.15.32:0/616318454 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9970003230 con 0x7f99840b2960 2020-12-05T17:31:55.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.063+0000 7f9983fff700 1 -- 172.21.15.32:0/616318454 >> v1:172.21.15.61:6789/0 conn(0x7f99840ad280 legacy=0x7f99840ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:55.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.063+0000 7f9983fff700 1 -- 172.21.15.32:0/616318454 >> v1:172.21.15.32:6789/0 conn(0x7f99840b2960 legacy=0x7f99840660e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:55.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.063+0000 7f9983fff700 1 -- 172.21.15.32:0/616318454 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99840b3020 con 0x7f99840667f0 2020-12-05T17:31:55.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.063+0000 7f9983fff700 1 --2- 172.21.15.32:0/616318454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99840667f0 0x7f99840aef40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f997c0032b0 tx=0x7f997c0091a0).ready entity=mon.0 client_cookie=3c39cb9e7be7b864 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:55.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.067+0000 7f99827fc700 1 -- 172.21.15.32:0/616318454 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f997c00b9a0 con 0x7f99840667f0 2020-12-05T17:31:55.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.067+0000 7f99827fc700 1 -- 172.21.15.32:0/616318454 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997c00bb00 con 0x7f99840667f0 2020-12-05T17:31:55.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.067+0000 7f99827fc700 1 -- 172.21.15.32:0/616318454 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f997c00bc70 con 0x7f99840667f0 2020-12-05T17:31:55.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.067+0000 7f9988ede700 1 -- 172.21.15.32:0/616318454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99840667f0 msgr2=0x7f99840aef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:55.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.067+0000 7f9988ede700 1 --2- 172.21.15.32:0/616318454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99840667f0 0x7f99840aef40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f997c0032b0 tx=0x7f997c0091a0).stop 2020-12-05T17:31:55.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.067+0000 7f9988ede700 1 -- 172.21.15.32:0/616318454 shutdown_connections 2020-12-05T17:31:55.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.067+0000 7f9988ede700 1 --2- 172.21.15.32:0/616318454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99840667f0 0x7f99840aef40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:55.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.067+0000 7f9988ede700 1 -- 172.21.15.32:0/616318454 >> 172.21.15.32:0/616318454 conn(0x7f9984061010 msgr2=0x7f9984061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:55.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.067+0000 7f9988ede700 1 -- 172.21.15.32:0/616318454 shutdown_connections 2020-12-05T17:31:55.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.067+0000 7f9988ede700 1 -- 172.21.15.32:0/616318454 wait complete. 2020-12-05T17:31:55.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.067+0000 7f9988ede700 1 Processor -- start 2020-12-05T17:31:55.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.067+0000 7f9988ede700 1 -- start start 2020-12-05T17:31:55.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.067+0000 7f9988ede700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f99840667f0 0x7f99841395c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:55.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.067+0000 7f9988ede700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99840ad280 0x7f998413e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:55.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.067+0000 7f9988ede700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f99840b2960 0x7f998413e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:55.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.067+0000 7f9988ede700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998413a7b0 con 0x7f99840ad280 2020-12-05T17:31:55.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.067+0000 7f9988ede700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9984065ba0 con 0x7f99840667f0 2020-12-05T17:31:55.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.067+0000 7f9988ede700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9984065900 con 0x7f99840b2960 2020-12-05T17:31:55.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.067+0000 7f9982ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99840ad280 0x7f998413e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:55.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.067+0000 7f9982ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99840ad280 0x7f998413e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33050/0 (socket says 172.21.15.32:33050) 2020-12-05T17:31:55.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.067+0000 7f9982ffd700 1 -- 172.21.15.32:0/3072920811 learned_addr learned my addr 172.21.15.32:0/3072920811 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:55.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.067+0000 7f99837fe700 1 --2- 172.21.15.32:0/3072920811 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f99840667f0 0x7f99841395c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:55.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.067+0000 7f9983fff700 1 --2- 172.21.15.32:0/3072920811 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f99840b2960 0x7f998413e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:55.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.071+0000 7f9982ffd700 1 -- 172.21.15.32:0/3072920811 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f99840b2960 msgr2=0x7f998413e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:55.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.071+0000 7f9982ffd700 1 --2- 172.21.15.32:0/3072920811 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f99840b2960 0x7f998413e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:55.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.071+0000 7f9982ffd700 1 -- 172.21.15.32:0/3072920811 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f99840667f0 msgr2=0x7f99841395c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:55.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.071+0000 7f9982ffd700 1 --2- 172.21.15.32:0/3072920811 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f99840667f0 0x7f99841395c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:55.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.071+0000 7f9982ffd700 1 -- 172.21.15.32:0/3072920811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998413eed0 con 0x7f99840ad280 2020-12-05T17:31:55.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.071+0000 7f9982ffd700 1 --2- 172.21.15.32:0/3072920811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99840ad280 0x7f998413e4f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f99840621a0 tx=0x7f9978007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:55.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.071+0000 7f9980ff9700 1 -- 172.21.15.32:0/3072920811 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9978013410 con 0x7f99840ad280 2020-12-05T17:31:55.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.071+0000 7f9980ff9700 1 -- 172.21.15.32:0/3072920811 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9978013570 con 0x7f99840ad280 2020-12-05T17:31:55.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.071+0000 7f9988ede700 1 -- 172.21.15.32:0/3072920811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9984139ab0 con 0x7f99840ad280 2020-12-05T17:31:55.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.071+0000 7f9980ff9700 1 -- 172.21.15.32:0/3072920811 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99780138d0 con 0x7f99840ad280 2020-12-05T17:31:55.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.071+0000 7f9988ede700 1 -- 172.21.15.32:0/3072920811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9984138f70 con 0x7f99840ad280 2020-12-05T17:31:55.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.071+0000 7f9988ede700 1 -- 172.21.15.32:0/3072920811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9984057050 con 0x7f99840ad280 2020-12-05T17:31:55.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.071+0000 7f9980ff9700 1 -- 172.21.15.32:0/3072920811 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9978010030 con 0x7f99840ad280 2020-12-05T17:31:55.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.071+0000 7f9980ff9700 1 --2- 172.21.15.32:0/3072920811 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9964055440 0x7f9964057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:55.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.071+0000 7f9980ff9700 1 -- 172.21.15.32:0/3072920811 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f997806ddb0 con 0x7f99840ad280 2020-12-05T17:31:55.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.071+0000 7f99837fe700 1 --2- 172.21.15.32:0/3072920811 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9964055440 0x7f9964057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:55.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.071+0000 7f99837fe700 1 --2- 172.21.15.32:0/3072920811 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9964055440 0x7f9964057900 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f997400d000 tx=0x7f9974009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:55.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.079+0000 7f9980ff9700 1 -- 172.21.15.32:0/3072920811 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9978043f50 con 0x7f99840ad280 2020-12-05T17:31:55.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.219+0000 7f9988ede700 1 -- 172.21.15.32:0/3072920811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9984069690 con 0x7f99840ad280 2020-12-05T17:31:55.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.219+0000 7f9980ff9700 1 -- 172.21.15.32:0/3072920811 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7f9978047570 con 0x7f99840ad280 2020-12-05T17:31:55.224 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:31:55.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.223+0000 7f9988ede700 1 -- 172.21.15.32:0/3072920811 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9964055440 msgr2=0x7f9964057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:55.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.223+0000 7f9988ede700 1 --2- 172.21.15.32:0/3072920811 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9964055440 0x7f9964057900 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f997400d000 tx=0x7f9974009040).stop 2020-12-05T17:31:55.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.223+0000 7f9988ede700 1 -- 172.21.15.32:0/3072920811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99840ad280 msgr2=0x7f998413e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:55.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.223+0000 7f9988ede700 1 --2- 172.21.15.32:0/3072920811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99840ad280 0x7f998413e4f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f99840621a0 tx=0x7f9978007bf0).stop 2020-12-05T17:31:55.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.223+0000 7f9988ede700 1 -- 172.21.15.32:0/3072920811 shutdown_connections 2020-12-05T17:31:55.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.223+0000 7f9988ede700 1 --2- 172.21.15.32:0/3072920811 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9964055440 0x7f9964057900 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:55.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.223+0000 7f9988ede700 1 --2- 172.21.15.32:0/3072920811 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f99840b2960 0x7f998413e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:55.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.223+0000 7f9988ede700 1 --2- 172.21.15.32:0/3072920811 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f99840667f0 0x7f99841395c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:55.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.223+0000 7f9988ede700 1 --2- 172.21.15.32:0/3072920811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99840ad280 0x7f998413e4f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:55.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.223+0000 7f9988ede700 1 -- 172.21.15.32:0/3072920811 >> 172.21.15.32:0/3072920811 conn(0x7f9984061010 msgr2=0x7f99840ae7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:55.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.223+0000 7f9988ede700 1 -- 172.21.15.32:0/3072920811 shutdown_connections 2020-12-05T17:31:55.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.223+0000 7f9988ede700 1 -- 172.21.15.32:0/3072920811 wait complete. 2020-12-05T17:31:55.241 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:31:55.242 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:31:55.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f32474e8700 1 Processor -- start 2020-12-05T17:31:55.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f32474e8700 1 -- start start 2020-12-05T17:31:55.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f32474e8700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f32400b7720 0x7f3240061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:55.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f32474e8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3240144540 0x7f324013d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:55.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f32474e8700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3240143a00 con 0x7f32401403f0 2020-12-05T17:31:55.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f32474e8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32400b7b00 con 0x7f3240144540 2020-12-05T17:31:55.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f32474e8700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f324005fe60 con 0x7f32400b7720 2020-12-05T17:31:55.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f3246ce7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3240144540 0x7f324013d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:55.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f3245ce5700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f32401403f0 0x7f3240143e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:48346/0 (socket says 172.21.15.32:48346) 2020-12-05T17:31:55.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f3245ce5700 1 -- 172.21.15.32:0/3648246797 learned_addr learned my addr 172.21.15.32:0/3648246797 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:55.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f32464e6700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f32400b7720 0x7f3240061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:55.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f3246ce7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3240144540 0x7f324013d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33060/0 (socket says 172.21.15.32:33060) 2020-12-05T17:31:55.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f32454e4700 1 -- 172.21.15.32:0/3648246797 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190909381 0 0) 0x7f3240143a00 con 0x7f32401403f0 2020-12-05T17:31:55.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f32454e4700 1 -- 172.21.15.32:0/3648246797 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f322c003230 con 0x7f32401403f0 2020-12-05T17:31:55.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f3246ce7700 1 -- 172.21.15.32:0/3648246797 >> v2:172.21.15.61:3300/0 conn(0x7f32400b7720 msgr2=0x7f3240061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:55.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f3246ce7700 1 --2- 172.21.15.32:0/3648246797 >> v2:172.21.15.61:3300/0 conn(0x7f32400b7720 0x7f3240061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:55.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f3246ce7700 1 -- 172.21.15.32:0/3648246797 >> v1:172.21.15.61:6789/0 conn(0x7f32401403f0 legacy=0x7f3240143e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:55.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f3246ce7700 1 -- 172.21.15.32:0/3648246797 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3240140850 con 0x7f3240144540 2020-12-05T17:31:55.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f3246ce7700 1 --2- 172.21.15.32:0/3648246797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3240144540 0x7f324013d3c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f3238002f30 tx=0x7f3238006dc0).ready entity=mon.0 client_cookie=5dc759529e56b1c9 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:55.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f32454e4700 1 -- 172.21.15.32:0/3648246797 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32380096f0 con 0x7f3240144540 2020-12-05T17:31:55.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f32454e4700 1 -- 172.21.15.32:0/3648246797 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3238009850 con 0x7f3240144540 2020-12-05T17:31:55.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f32474e8700 1 -- 172.21.15.32:0/3648246797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3240144540 msgr2=0x7f324013d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:55.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f32454e4700 1 -- 172.21.15.32:0/3648246797 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3238009bb0 con 0x7f3240144540 2020-12-05T17:31:55.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.351+0000 7f32474e8700 1 --2- 172.21.15.32:0/3648246797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3240144540 0x7f324013d3c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f3238002f30 tx=0x7f3238006dc0).stop 2020-12-05T17:31:55.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f32474e8700 1 -- 172.21.15.32:0/3648246797 shutdown_connections 2020-12-05T17:31:55.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f32474e8700 1 --2- 172.21.15.32:0/3648246797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3240144540 0x7f324013d3c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:55.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f32474e8700 1 --2- 172.21.15.32:0/3648246797 >> v2:172.21.15.61:3300/0 conn(0x7f32400b7720 0x7f3240061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:55.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f32474e8700 1 -- 172.21.15.32:0/3648246797 >> 172.21.15.32:0/3648246797 conn(0x7f324005bc20 msgr2=0x7f32400b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:55.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f32474e8700 1 -- 172.21.15.32:0/3648246797 shutdown_connections 2020-12-05T17:31:55.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f32474e8700 1 -- 172.21.15.32:0/3648246797 wait complete. 2020-12-05T17:31:55.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f32474e8700 1 Processor -- start 2020-12-05T17:31:55.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f32474e8700 1 -- start start 2020-12-05T17:31:55.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f32474e8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32400b7720 0x7f32400b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:55.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f32474e8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32401403f0 0x7f32400b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:55.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f32474e8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3240144540 0x7f32400b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:55.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f32474e8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3240143250 con 0x7f32401403f0 2020-12-05T17:31:55.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f32474e8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32401434f0 con 0x7f32400b7720 2020-12-05T17:31:55.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f32474e8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3240152460 con 0x7f3240144540 2020-12-05T17:31:55.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f3245ce5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32401403f0 0x7f32400b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:55.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f3245ce5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32401403f0 0x7f32400b0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33064/0 (socket says 172.21.15.32:33064) 2020-12-05T17:31:55.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f3245ce5700 1 -- 172.21.15.32:0/1966468967 learned_addr learned my addr 172.21.15.32:0/1966468967 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:55.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f32464e6700 1 --2- 172.21.15.32:0/1966468967 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32400b7720 0x7f32400b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:55.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f3246ce7700 1 --2- 172.21.15.32:0/1966468967 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3240144540 0x7f32400b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:55.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f3245ce5700 1 -- 172.21.15.32:0/1966468967 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3240144540 msgr2=0x7f32400b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:55.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f3245ce5700 1 --2- 172.21.15.32:0/1966468967 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3240144540 0x7f32400b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:55.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f3245ce5700 1 -- 172.21.15.32:0/1966468967 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32400b7720 msgr2=0x7f32400b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:55.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f3245ce5700 1 --2- 172.21.15.32:0/1966468967 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32400b7720 0x7f32400b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:55.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f3245ce5700 1 -- 172.21.15.32:0/1966468967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32400afe90 con 0x7f32401403f0 2020-12-05T17:31:55.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f3245ce5700 1 --2- 172.21.15.32:0/1966468967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32401403f0 0x7f32400b0c20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f32340010d0 tx=0x7f3234007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:55.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f323f7fe700 1 -- 172.21.15.32:0/1966468967 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32340072e0 con 0x7f32401403f0 2020-12-05T17:31:55.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f323f7fe700 1 -- 172.21.15.32:0/1966468967 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3234007440 con 0x7f32401403f0 2020-12-05T17:31:55.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f32474e8700 1 -- 172.21.15.32:0/1966468967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3240142e20 con 0x7f32401403f0 2020-12-05T17:31:55.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f323f7fe700 1 -- 172.21.15.32:0/1966468967 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32340057d0 con 0x7f32401403f0 2020-12-05T17:31:55.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.355+0000 7f32474e8700 1 -- 172.21.15.32:0/1966468967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3240152aa0 con 0x7f32401403f0 2020-12-05T17:31:55.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.359+0000 7f32474e8700 1 -- 172.21.15.32:0/1966468967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3240152620 con 0x7f32401403f0 2020-12-05T17:31:55.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.359+0000 7f323f7fe700 1 -- 172.21.15.32:0/1966468967 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f323401a060 con 0x7f32401403f0 2020-12-05T17:31:55.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.359+0000 7f323f7fe700 1 --2- 172.21.15.32:0/1966468967 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f32200554e0 0x7f32200579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:55.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.359+0000 7f32464e6700 1 --2- 172.21.15.32:0/1966468967 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f32200554e0 0x7f32200579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:55.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.359+0000 7f323f7fe700 1 -- 172.21.15.32:0/1966468967 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f323406d620 con 0x7f32401403f0 2020-12-05T17:31:55.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.359+0000 7f32464e6700 1 --2- 172.21.15.32:0/1966468967 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f32200554e0 0x7f32200579a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f323000b000 tx=0x7f32300091a0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:55.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.367+0000 7f323f7fe700 1 -- 172.21.15.32:0/1966468967 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f32340436b0 con 0x7f32401403f0 2020-12-05T17:31:55.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.547+0000 7f32474e8700 1 -- 172.21.15.32:0/1966468967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3240057050 con 0x7f32401403f0 2020-12-05T17:31:55.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.547+0000 7f323f7fe700 1 -- 172.21.15.32:0/1966468967 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2134 (secure 0 0 0) 0x7f3234018020 con 0x7f32401403f0 2020-12-05T17:31:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:31:55.552 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4023797731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4023797731},{"type":"v1","addr":"172.21.15.61:6833","nonce":4023797731}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:31:41.611214+0000","modified":"2020-12-05T17:31:42.614053+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3503848021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3503848021},{"type":"v1","addr":"172.21.15.32:6835","nonce":3503848021}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-12-05T17:31:55.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.551+0000 7f32474e8700 1 -- 172.21.15.32:0/1966468967 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f32200554e0 msgr2=0x7f32200579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:55.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.551+0000 7f32474e8700 1 --2- 172.21.15.32:0/1966468967 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f32200554e0 0x7f32200579a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f323000b000 tx=0x7f32300091a0).stop 2020-12-05T17:31:55.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.551+0000 7f32474e8700 1 -- 172.21.15.32:0/1966468967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32401403f0 msgr2=0x7f32400b0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:55.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.551+0000 7f32474e8700 1 --2- 172.21.15.32:0/1966468967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32401403f0 0x7f32400b0c20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f32340010d0 tx=0x7f3234007b50).stop 2020-12-05T17:31:55.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.551+0000 7f32474e8700 1 -- 172.21.15.32:0/1966468967 shutdown_connections 2020-12-05T17:31:55.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.551+0000 7f32474e8700 1 --2- 172.21.15.32:0/1966468967 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f32200554e0 0x7f32200579a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:55.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.551+0000 7f32474e8700 1 --2- 172.21.15.32:0/1966468967 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3240144540 0x7f32400b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:55.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.551+0000 7f32474e8700 1 --2- 172.21.15.32:0/1966468967 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32400b7720 0x7f32400b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:55.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.551+0000 7f32474e8700 1 --2- 172.21.15.32:0/1966468967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32401403f0 0x7f32400b0c20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:55.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.551+0000 7f32474e8700 1 -- 172.21.15.32:0/1966468967 >> 172.21.15.32:0/1966468967 conn(0x7f324005bc20 msgr2=0x7f3240141b10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:55.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.551+0000 7f32474e8700 1 -- 172.21.15.32:0/1966468967 shutdown_connections 2020-12-05T17:31:55.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:55.551+0000 7f32474e8700 1 -- 172.21.15.32:0/1966468967 wait complete. 2020-12-05T17:31:55.556 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 5 2020-12-05T17:31:55.772 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:31:41.611214+0000', 'modified': '2020-12-05T17:31:42.614053+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4275}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4275': {'gid': 4275, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/3503848021', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 3503848021}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 3503848021}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:31:55.773 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:31:55.773 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:31:55.906 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":4275,"want_state":"up:active","state":"up:active","rank_uptime":14.281694303,"mdsmap_epoch":5,"osdmap_epoch":25,"osdmap_epoch_barrier":25,"uptime":20.687673556} 2020-12-05T17:31:55.907 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 4275, "mdsmap_epoch": 5, "osdmap_epoch": 25, "osdmap_epoch_barrier": 25, "rank_uptime": 14.281694303, "state": "up:active", "uptime": 20.687673556, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:31:55.907 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:31:55.907 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-12-05T17:31:55.908 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-12-05T17:31:55.908 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:31:55.908 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:31:55.917 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:31:55.975 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:31:55.975 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:31:56.029 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:31:56.031 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:31:56.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:31:56.032 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:31:56.078 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:31:56.079 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:31:56.122 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:31:56.138 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:31:56.138 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:31:56.194 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:31:56.195 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:31:56.195 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:31:56.196 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:31:56.204 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[10276]: starting ceph client2020-12-05T17:31:56.199+0000 7f0e7bdff0c0 -1 init, newargv = 0x5653b4d8fac0 newargc=9 2020-12-05T17:31:56.205 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr: 2020-12-05T17:31:56.216 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[10276]: starting fuse 2020-12-05T17:31:57.312 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:31:57.328 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:31:57.328 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:31:57.383 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:31:57.384 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:31:57.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:31:57.385 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:31:57.430 INFO:teuthology.orchestra.run.smithi036.stdout:49 2020-12-05T17:31:57.431 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-12-05T17:31:57.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:31:57.431 INFO:teuthology.orchestra.run.smithi036:> sudo python3 -c ' 2020-12-05T17:31:57.432 INFO:teuthology.orchestra.run.smithi036:> import glob 2020-12-05T17:31:57.432 INFO:teuthology.orchestra.run.smithi036:> import re 2020-12-05T17:31:57.432 INFO:teuthology.orchestra.run.smithi036:> import os 2020-12-05T17:31:57.433 INFO:teuthology.orchestra.run.smithi036:> import subprocess 2020-12-05T17:31:57.433 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:31:57.433 INFO:teuthology.orchestra.run.smithi036:> def find_socket(client_name): 2020-12-05T17:31:57.434 INFO:teuthology.orchestra.run.smithi036:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-12-05T17:31:57.434 INFO:teuthology.orchestra.run.smithi036:> files = glob.glob(asok_path) 2020-12-05T17:31:57.434 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:31:57.435 INFO:teuthology.orchestra.run.smithi036:> # Given a non-glob path, it better be there 2020-12-05T17:31:57.435 INFO:teuthology.orchestra.run.smithi036:> if "*" not in asok_path: 2020-12-05T17:31:57.435 INFO:teuthology.orchestra.run.smithi036:> assert(len(files) == 1) 2020-12-05T17:31:57.436 INFO:teuthology.orchestra.run.smithi036:> return files[0] 2020-12-05T17:31:57.436 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:31:57.436 INFO:teuthology.orchestra.run.smithi036:> for f in files: 2020-12-05T17:31:57.437 INFO:teuthology.orchestra.run.smithi036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:31:57.437 INFO:teuthology.orchestra.run.smithi036:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:31:57.437 INFO:teuthology.orchestra.run.smithi036:> return f 2020-12-05T17:31:57.438 INFO:teuthology.orchestra.run.smithi036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:31:57.438 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:31:57.438 INFO:teuthology.orchestra.run.smithi036:> print(find_socket("client.0")) 2020-12-05T17:31:57.439 INFO:teuthology.orchestra.run.smithi036:> ' 2020-12-05T17:31:57.523 INFO:teuthology.orchestra.run.smithi036.stdout:/var/run/ceph/ceph-client.0.10276.asok 2020-12-05T17:31:57.529 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.10276.asok 2020-12-05T17:31:57.529 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:31:57.529 INFO:teuthology.orchestra.run.smithi036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.10276.asok status 2020-12-05T17:31:57.732 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2020-12-05T17:31:57.732 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": { 2020-12-05T17:31:57.733 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:31:57.733 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:31:57.733 INFO:teuthology.orchestra.run.smithi036.stdout: "entity_id": "0", 2020-12-05T17:31:57.733 INFO:teuthology.orchestra.run.smithi036.stdout: "hostname": "smithi036", 2020-12-05T17:31:57.733 INFO:teuthology.orchestra.run.smithi036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-12-05T17:31:57.734 INFO:teuthology.orchestra.run.smithi036.stdout: "pid": "10276", 2020-12-05T17:31:57.734 INFO:teuthology.orchestra.run.smithi036.stdout: "root": "/" 2020-12-05T17:31:57.734 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:31:57.734 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_count": 0, 2020-12-05T17:31:57.735 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_pinned_count": 0, 2020-12-05T17:31:57.735 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 4475, 2020-12-05T17:31:57.735 INFO:teuthology.orchestra.run.smithi036.stdout: "inst": { 2020-12-05T17:31:57.735 INFO:teuthology.orchestra.run.smithi036.stdout: "name": { 2020-12-05T17:31:57.735 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "client", 2020-12-05T17:31:57.736 INFO:teuthology.orchestra.run.smithi036.stdout: "num": 4475 2020-12-05T17:31:57.736 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:31:57.736 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:31:57.736 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:31:57.737 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:31:57.737 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 3187922835 2020-12-05T17:31:57.737 INFO:teuthology.orchestra.run.smithi036.stdout: } 2020-12-05T17:31:57.737 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:31:57.737 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:31:57.738 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:31:57.738 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:31:57.738 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 3187922835 2020-12-05T17:31:57.738 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:31:57.738 INFO:teuthology.orchestra.run.smithi036.stdout: "inst_str": "client.4475 172.21.15.36:0/3187922835", 2020-12-05T17:31:57.739 INFO:teuthology.orchestra.run.smithi036.stdout: "addr_str": "172.21.15.36:0/3187922835", 2020-12-05T17:31:57.739 INFO:teuthology.orchestra.run.smithi036.stdout: "inode_count": 1, 2020-12-05T17:31:57.739 INFO:teuthology.orchestra.run.smithi036.stdout: "mds_epoch": 5, 2020-12-05T17:31:57.739 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch": 25, 2020-12-05T17:31:57.740 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:31:57.740 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklisted": false 2020-12-05T17:31:57.740 INFO:teuthology.orchestra.run.smithi036.stdout:} 2020-12-05T17:31:57.742 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:31:57.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.863+0000 7f9bdf2eb700 1 Processor -- start 2020-12-05T17:31:57.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.863+0000 7f9bdf2eb700 1 -- start start 2020-12-05T17:31:57.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.863+0000 7f9bdf2eb700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9bd004b030 0x7f9bd00d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:57.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.863+0000 7f9bdf2eb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9bd00d13e0 0x7f9bd0045540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:57.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.863+0000 7f9bdf2eb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bd004b410 con 0x7f9bd00d7420 2020-12-05T17:31:57.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.863+0000 7f9bdf2eb700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd00d4300 con 0x7f9bd004b030 2020-12-05T17:31:57.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.863+0000 7f9bdf2eb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd00d1850 con 0x7f9bd00d13e0 2020-12-05T17:31:57.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.863+0000 7f9bddae8700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9bd00d7420 0x7f9bd00d0cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35718/0 (socket says 172.21.15.32:35718) 2020-12-05T17:31:57.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.863+0000 7f9bddae8700 1 -- 172.21.15.32:0/2938402385 learned_addr learned my addr 172.21.15.32:0/2938402385 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:57.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.863+0000 7f9bdeaea700 1 --2- 172.21.15.32:0/2938402385 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9bd00d13e0 0x7f9bd0045540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:57.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.863+0000 7f9bde2e9700 1 --2- 172.21.15.32:0/2938402385 >> v2:172.21.15.61:3300/0 conn(0x7f9bd004b030 0x7f9bd00d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:57.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.863+0000 7f9bdd2e7700 1 -- 172.21.15.32:0/2938402385 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1505489948 0 0) 0x7f9bd004b410 con 0x7f9bd00d7420 2020-12-05T17:31:57.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.863+0000 7f9bdd2e7700 1 -- 172.21.15.32:0/2938402385 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bc0003230 con 0x7f9bd00d7420 2020-12-05T17:31:57.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.863+0000 7f9bdeaea700 1 -- 172.21.15.32:0/2938402385 >> v2:172.21.15.61:3300/0 conn(0x7f9bd004b030 msgr2=0x7f9bd00d3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:57.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.863+0000 7f9bdeaea700 1 --2- 172.21.15.32:0/2938402385 >> v2:172.21.15.61:3300/0 conn(0x7f9bd004b030 0x7f9bd00d3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:57.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.863+0000 7f9bdeaea700 1 -- 172.21.15.32:0/2938402385 >> v1:172.21.15.32:6789/0 conn(0x7f9bd00d7420 legacy=0x7f9bd00d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:57.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.863+0000 7f9bdeaea700 1 -- 172.21.15.32:0/2938402385 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd00d44f0 con 0x7f9bd00d13e0 2020-12-05T17:31:57.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.863+0000 7f9bde2e9700 1 --2- 172.21.15.32:0/2938402385 >> v2:172.21.15.61:3300/0 conn(0x7f9bd004b030 0x7f9bd00d3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:31:57.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.863+0000 7f9bdeaea700 1 --2- 172.21.15.32:0/2938402385 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9bd00d13e0 0x7f9bd0045540 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f9bd8068ca0 tx=0x7f9bd805a760).ready entity=mon.2 client_cookie=3bbbdcd0664d4657 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:57.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bdd2e7700 1 -- 172.21.15.32:0/2938402385 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bd8085070 con 0x7f9bd00d13e0 2020-12-05T17:31:57.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bdd2e7700 1 -- 172.21.15.32:0/2938402385 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd8068f70 con 0x7f9bd00d13e0 2020-12-05T17:31:57.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bdd2e7700 1 -- 172.21.15.32:0/2938402385 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bd8044b10 con 0x7f9bd00d13e0 2020-12-05T17:31:57.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bdf2eb700 1 -- 172.21.15.32:0/2938402385 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9bd00d13e0 msgr2=0x7f9bd0045540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bdf2eb700 1 --2- 172.21.15.32:0/2938402385 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9bd00d13e0 0x7f9bd0045540 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f9bd8068ca0 tx=0x7f9bd805a760).stop 2020-12-05T17:31:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bdf2eb700 1 -- 172.21.15.32:0/2938402385 shutdown_connections 2020-12-05T17:31:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bdf2eb700 1 --2- 172.21.15.32:0/2938402385 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9bd00d13e0 0x7f9bd0045540 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bdf2eb700 1 --2- 172.21.15.32:0/2938402385 >> v2:172.21.15.61:3300/0 conn(0x7f9bd004b030 0x7f9bd00d3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bdf2eb700 1 -- 172.21.15.32:0/2938402385 >> 172.21.15.32:0/2938402385 conn(0x7f9bd0011400 msgr2=0x7f9bd004a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bdf2eb700 1 -- 172.21.15.32:0/2938402385 shutdown_connections 2020-12-05T17:31:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bdf2eb700 1 -- 172.21.15.32:0/2938402385 wait complete. 2020-12-05T17:31:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bdf2eb700 1 Processor -- start 2020-12-05T17:31:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bdf2eb700 1 -- start start 2020-12-05T17:31:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bdf2eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bd004b030 0x7f9bd00db0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bdf2eb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9bd00d13e0 0x7f9bd00d65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bdf2eb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9bd00d7420 0x7f9bd00411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bdf2eb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd00d9d50 con 0x7f9bd004b030 2020-12-05T17:31:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bdf2eb700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd003f890 con 0x7f9bd00d13e0 2020-12-05T17:31:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bdf2eb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd00d9fb0 con 0x7f9bd00d7420 2020-12-05T17:31:57.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bde2e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bd004b030 0x7f9bd00db0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:57.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bde2e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bd004b030 0x7f9bd00db0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33076/0 (socket says 172.21.15.32:33076) 2020-12-05T17:31:57.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bde2e9700 1 -- 172.21.15.32:0/2744708646 learned_addr learned my addr 172.21.15.32:0/2744708646 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:57.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bdeaea700 1 --2- 172.21.15.32:0/2744708646 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9bd00d7420 0x7f9bd00411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:57.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bddae8700 1 --2- 172.21.15.32:0/2744708646 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9bd00d13e0 0x7f9bd00d65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bde2e9700 1 -- 172.21.15.32:0/2744708646 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9bd00d7420 msgr2=0x7f9bd00411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bde2e9700 1 --2- 172.21.15.32:0/2744708646 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9bd00d7420 0x7f9bd00411e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bde2e9700 1 -- 172.21.15.32:0/2744708646 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9bd00d13e0 msgr2=0x7f9bd00d65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bde2e9700 1 --2- 172.21.15.32:0/2744708646 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9bd00d13e0 0x7f9bd00d65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bde2e9700 1 -- 172.21.15.32:0/2744708646 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd003fcf0 con 0x7f9bd004b030 2020-12-05T17:31:57.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bdeaea700 1 --2- 172.21.15.32:0/2744708646 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9bd00d7420 0x7f9bd00411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:57.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.867+0000 7f9bde2e9700 1 --2- 172.21.15.32:0/2744708646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bd004b030 0x7f9bd00db0a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f9bd400c000 tx=0x7f9bd4006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:57.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.871+0000 7f9bc77fe700 1 -- 172.21.15.32:0/2744708646 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bd400cda0 con 0x7f9bd004b030 2020-12-05T17:31:57.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.871+0000 7f9bc77fe700 1 -- 172.21.15.32:0/2744708646 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd40123f0 con 0x7f9bd004b030 2020-12-05T17:31:57.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.871+0000 7f9bc77fe700 1 -- 172.21.15.32:0/2744708646 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bd4012750 con 0x7f9bd004b030 2020-12-05T17:31:57.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.871+0000 7f9bdf2eb700 1 -- 172.21.15.32:0/2744708646 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd00d98f0 con 0x7f9bd004b030 2020-12-05T17:31:57.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.871+0000 7f9bdf2eb700 1 -- 172.21.15.32:0/2744708646 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd003f5b0 con 0x7f9bd004b030 2020-12-05T17:31:57.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.871+0000 7f9bdf2eb700 1 -- 172.21.15.32:0/2744708646 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd0004c10 con 0x7f9bd004b030 2020-12-05T17:31:57.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.871+0000 7f9bc77fe700 1 -- 172.21.15.32:0/2744708646 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9bd40128b0 con 0x7f9bd004b030 2020-12-05T17:31:57.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.871+0000 7f9bc77fe700 1 --2- 172.21.15.32:0/2744708646 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9bb8055440 0x7f9bb8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:57.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.871+0000 7f9bc77fe700 1 -- 172.21.15.32:0/2744708646 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f9bd406e8d0 con 0x7f9bd004b030 2020-12-05T17:31:57.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.871+0000 7f9bddae8700 1 --2- 172.21.15.32:0/2744708646 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9bb8055440 0x7f9bb8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:57.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.871+0000 7f9bddae8700 1 --2- 172.21.15.32:0/2744708646 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9bb8055440 0x7f9bb8057900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f9bc800b000 tx=0x7f9bc8009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:57.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:57.879+0000 7f9bc77fe700 1 -- 172.21.15.32:0/2744708646 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9bd40449d0 con 0x7f9bd004b030 2020-12-05T17:31:58.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.023+0000 7f9bdf2eb700 1 -- 172.21.15.32:0/2744708646 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bd00082b0 con 0x7f9bd004b030 2020-12-05T17:31:58.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.023+0000 7f9bc77fe700 1 -- 172.21.15.32:0/2744708646 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f9bd4047ff0 con 0x7f9bd004b030 2020-12-05T17:31:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:31:58.028 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:31:41.610914+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:31:36.706361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-12-05T17:31:39.298269+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:31:58.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.027+0000 7f9bdf2eb700 1 -- 172.21.15.32:0/2744708646 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9bb8055440 msgr2=0x7f9bb8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:58.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.027+0000 7f9bdf2eb700 1 --2- 172.21.15.32:0/2744708646 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9bb8055440 0x7f9bb8057900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f9bc800b000 tx=0x7f9bc8009040).stop 2020-12-05T17:31:58.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.027+0000 7f9bdf2eb700 1 -- 172.21.15.32:0/2744708646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bd004b030 msgr2=0x7f9bd00db0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:58.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.027+0000 7f9bdf2eb700 1 --2- 172.21.15.32:0/2744708646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bd004b030 0x7f9bd00db0a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f9bd400c000 tx=0x7f9bd4006dc0).stop 2020-12-05T17:31:58.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.027+0000 7f9bdf2eb700 1 -- 172.21.15.32:0/2744708646 shutdown_connections 2020-12-05T17:31:58.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.027+0000 7f9bdf2eb700 1 --2- 172.21.15.32:0/2744708646 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9bb8055440 0x7f9bb8057900 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.027+0000 7f9bdf2eb700 1 --2- 172.21.15.32:0/2744708646 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9bd00d7420 0x7f9bd00411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.027+0000 7f9bdf2eb700 1 --2- 172.21.15.32:0/2744708646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bd004b030 0x7f9bd00db0a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.027+0000 7f9bdf2eb700 1 --2- 172.21.15.32:0/2744708646 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9bd00d13e0 0x7f9bd00d65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.027+0000 7f9bdf2eb700 1 -- 172.21.15.32:0/2744708646 >> 172.21.15.32:0/2744708646 conn(0x7f9bd0011400 msgr2=0x7f9bd0048fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:58.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.027+0000 7f9bdf2eb700 1 -- 172.21.15.32:0/2744708646 shutdown_connections 2020-12-05T17:31:58.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.027+0000 7f9bdf2eb700 1 -- 172.21.15.32:0/2744708646 wait complete. 2020-12-05T17:31:58.042 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:31:36.706361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:31:39.298269+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:31:58.043 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:31:58.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.159+0000 7f67908f2700 1 Processor -- start 2020-12-05T17:31:58.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.159+0000 7f67908f2700 1 -- start start 2020-12-05T17:31:58.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.163+0000 7f67908f2700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f678c063040 0x7f678c13fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:58.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.163+0000 7f67908f2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f678c067110 con 0x7f678c0af090 2020-12-05T17:31:58.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.163+0000 7f67908f2700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f678c0ad290 con 0x7f678c05f550 2020-12-05T17:31:58.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.163+0000 7f67908f2700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678c13f760 con 0x7f678c063040 2020-12-05T17:31:58.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.163+0000 7f678b59e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f678c0af090 0x7f678c0af500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35736/0 (socket says 172.21.15.32:35736) 2020-12-05T17:31:58.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.163+0000 7f678b59e700 1 -- 172.21.15.32:0/2224308280 learned_addr learned my addr 172.21.15.32:0/2224308280 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:31:58.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.163+0000 7f678a59c700 1 --2- 172.21.15.32:0/2224308280 >> v2:172.21.15.61:3300/0 conn(0x7f678c063040 0x7f678c13fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:58.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.163+0000 7f6789d9b700 1 -- 172.21.15.32:0/2224308280 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4055270483 0 0) 0x7f678c067110 con 0x7f678c0af090 2020-12-05T17:31:58.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.163+0000 7f6789d9b700 1 -- 172.21.15.32:0/2224308280 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6770003230 con 0x7f678c0af090 2020-12-05T17:31:58.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.163+0000 7f678a59c700 1 -- 172.21.15.32:0/2224308280 >> v1:172.21.15.61:6789/0 conn(0x7f678c05f550 legacy=0x7f678c0acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:58.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.163+0000 7f678a59c700 1 -- 172.21.15.32:0/2224308280 >> v1:172.21.15.32:6789/0 conn(0x7f678c0af090 legacy=0x7f678c0af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:58.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.163+0000 7f678a59c700 1 -- 172.21.15.32:0/2224308280 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c13fa00 con 0x7f678c063040 2020-12-05T17:31:58.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.163+0000 7f678a59c700 1 --2- 172.21.15.32:0/2224308280 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f678c063040 0x7f678c13fea0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f677c002f30 tx=0x7f677c006dc0).ready entity=mon.1 client_cookie=7163e8fc671c4b0b server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:58.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.163+0000 7f6789d9b700 1 -- 172.21.15.32:0/2224308280 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f677c009800 con 0x7f678c063040 2020-12-05T17:31:58.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.163+0000 7f6789d9b700 1 -- 172.21.15.32:0/2224308280 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677c009960 con 0x7f678c063040 2020-12-05T17:31:58.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.163+0000 7f6789d9b700 1 -- 172.21.15.32:0/2224308280 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f677c009cc0 con 0x7f678c063040 2020-12-05T17:31:58.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.163+0000 7f67908f2700 1 -- 172.21.15.32:0/2224308280 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f678c063040 msgr2=0x7f678c13fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:58.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.163+0000 7f67908f2700 1 --2- 172.21.15.32:0/2224308280 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f678c063040 0x7f678c13fea0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f677c002f30 tx=0x7f677c006dc0).stop 2020-12-05T17:31:58.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.163+0000 7f67908f2700 1 -- 172.21.15.32:0/2224308280 shutdown_connections 2020-12-05T17:31:58.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.163+0000 7f67908f2700 1 --2- 172.21.15.32:0/2224308280 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f678c063040 0x7f678c13fea0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.163+0000 7f67908f2700 1 -- 172.21.15.32:0/2224308280 >> 172.21.15.32:0/2224308280 conn(0x7f678c0620b0 msgr2=0x7f678c062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:58.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.163+0000 7f67908f2700 1 -- 172.21.15.32:0/2224308280 shutdown_connections 2020-12-05T17:31:58.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.163+0000 7f67908f2700 1 -- 172.21.15.32:0/2224308280 wait complete. 2020-12-05T17:31:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f67908f2700 1 Processor -- start 2020-12-05T17:31:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f67908f2700 1 -- start start 2020-12-05T17:31:58.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f67908f2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678c05f550 0x7f678c143100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:58.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f67908f2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f678c063040 0x7f678c13cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:58.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f67908f2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f678c0af090 0x7f678c13d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:58.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f67908f2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c066670 con 0x7f678c05f550 2020-12-05T17:31:58.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f67908f2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c13ad00 con 0x7f678c0af090 2020-12-05T17:31:58.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f67908f2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678c13a8a0 con 0x7f678c063040 2020-12-05T17:31:58.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f678ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678c05f550 0x7f678c143100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:58.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f678ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678c05f550 0x7f678c143100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33092/0 (socket says 172.21.15.32:33092) 2020-12-05T17:31:58.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f678ad9d700 1 -- 172.21.15.32:0/502876040 learned_addr learned my addr 172.21.15.32:0/502876040 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:58.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f678a59c700 1 --2- 172.21.15.32:0/502876040 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f678c063040 0x7f678c13cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:58.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f678b59e700 1 --2- 172.21.15.32:0/502876040 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f678c0af090 0x7f678c13d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:58.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f678ad9d700 1 -- 172.21.15.32:0/502876040 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f678c063040 msgr2=0x7f678c13cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:58.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f678ad9d700 1 --2- 172.21.15.32:0/502876040 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f678c063040 0x7f678c13cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f678ad9d700 1 -- 172.21.15.32:0/502876040 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f678c0af090 msgr2=0x7f678c13d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:58.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f678ad9d700 1 --2- 172.21.15.32:0/502876040 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f678c0af090 0x7f678c13d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f678ad9d700 1 -- 172.21.15.32:0/502876040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c0663d0 con 0x7f678c05f550 2020-12-05T17:31:58.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f678ad9d700 1 --2- 172.21.15.32:0/502876040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678c05f550 0x7f678c143100 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f678c13ae40 tx=0x7f6780007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:58.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f677bfff700 1 -- 172.21.15.32:0/502876040 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f678000e4a0 con 0x7f678c05f550 2020-12-05T17:31:58.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f677bfff700 1 -- 172.21.15.32:0/502876040 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678000e600 con 0x7f678c05f550 2020-12-05T17:31:58.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f677bfff700 1 -- 172.21.15.32:0/502876040 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f678000e960 con 0x7f678c05f550 2020-12-05T17:31:58.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f67908f2700 1 -- 172.21.15.32:0/502876040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678c065770 con 0x7f678c05f550 2020-12-05T17:31:58.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.167+0000 7f67908f2700 1 -- 172.21.15.32:0/502876040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678c139c00 con 0x7f678c05f550 2020-12-05T17:31:58.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.171+0000 7f67908f2700 1 -- 172.21.15.32:0/502876040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678c057050 con 0x7f678c05f550 2020-12-05T17:31:58.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.171+0000 7f677bfff700 1 -- 172.21.15.32:0/502876040 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6780010030 con 0x7f678c05f550 2020-12-05T17:31:58.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.171+0000 7f677bfff700 1 --2- 172.21.15.32:0/502876040 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6768055440 0x7f6768057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:58.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.171+0000 7f678a59c700 1 --2- 172.21.15.32:0/502876040 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6768055440 0x7f6768057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:58.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.171+0000 7f677bfff700 1 -- 172.21.15.32:0/502876040 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f678006df20 con 0x7f678c05f550 2020-12-05T17:31:58.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.171+0000 7f678a59c700 1 --2- 172.21.15.32:0/502876040 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6768055440 0x7f6768057900 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f677c01a000 tx=0x7f677c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:58.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.175+0000 7f677bfff700 1 -- 172.21.15.32:0/502876040 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f67800440c0 con 0x7f678c05f550 2020-12-05T17:31:58.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.319+0000 7f67908f2700 1 -- 172.21.15.32:0/502876040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f678c13f4a0 con 0x7f678c05f550 2020-12-05T17:31:58.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.319+0000 7f677bfff700 1 -- 172.21.15.32:0/502876040 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7f67800476e0 con 0x7f678c05f550 2020-12-05T17:31:58.323 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:31:58.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.323+0000 7f67908f2700 1 -- 172.21.15.32:0/502876040 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6768055440 msgr2=0x7f6768057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:58.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.323+0000 7f67908f2700 1 --2- 172.21.15.32:0/502876040 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6768055440 0x7f6768057900 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f677c01a000 tx=0x7f677c015000).stop 2020-12-05T17:31:58.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.323+0000 7f67908f2700 1 -- 172.21.15.32:0/502876040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678c05f550 msgr2=0x7f678c143100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:58.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.323+0000 7f67908f2700 1 --2- 172.21.15.32:0/502876040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678c05f550 0x7f678c143100 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f678c13ae40 tx=0x7f6780007c20).stop 2020-12-05T17:31:58.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.323+0000 7f67908f2700 1 -- 172.21.15.32:0/502876040 shutdown_connections 2020-12-05T17:31:58.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.323+0000 7f67908f2700 1 --2- 172.21.15.32:0/502876040 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f678c0af090 0x7f678c13d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.323+0000 7f67908f2700 1 --2- 172.21.15.32:0/502876040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678c05f550 0x7f678c143100 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.323+0000 7f67908f2700 1 --2- 172.21.15.32:0/502876040 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6768055440 0x7f6768057900 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.323+0000 7f67908f2700 1 --2- 172.21.15.32:0/502876040 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f678c063040 0x7f678c13cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.323+0000 7f67908f2700 1 -- 172.21.15.32:0/502876040 >> 172.21.15.32:0/502876040 conn(0x7f678c0620b0 msgr2=0x7f678c0b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:58.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.323+0000 7f67908f2700 1 -- 172.21.15.32:0/502876040 shutdown_connections 2020-12-05T17:31:58.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.323+0000 7f67908f2700 1 -- 172.21.15.32:0/502876040 wait complete. 2020-12-05T17:31:58.339 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:31:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.455+0000 7f0adffff700 1 Processor -- start 2020-12-05T17:31:58.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.455+0000 7f0adffff700 1 -- start start 2020-12-05T17:31:58.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.455+0000 7f0adffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ae005f3b0 0x7f0ae005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:58.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.455+0000 7f0adffff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae00b2df0 con 0x7f0ae00b2fb0 2020-12-05T17:31:58.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.455+0000 7f0adffff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae00b3c90 con 0x7f0ae005feb0 2020-12-05T17:31:58.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.455+0000 7f0adffff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ae00b2b50 con 0x7f0ae005f3b0 2020-12-05T17:31:58.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.455+0000 7f0adeffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ae005f3b0 0x7f0ae005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:58.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.455+0000 7f0adeffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ae005f3b0 0x7f0ae005f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33098/0 (socket says 172.21.15.32:33098) 2020-12-05T17:31:58.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.455+0000 7f0adeffd700 1 -- 172.21.15.32:0/1740522903 learned_addr learned my addr 172.21.15.32:0/1740522903 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:58.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.455+0000 7f0adeffd700 1 -- 172.21.15.32:0/1740522903 >> v1:172.21.15.61:6789/0 conn(0x7f0ae005feb0 legacy=0x7f0ae013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:58.461 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.455+0000 7f0adeffd700 1 -- 172.21.15.32:0/1740522903 >> v1:172.21.15.32:6789/0 conn(0x7f0ae00b2fb0 legacy=0x7f0ae0142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:58.461 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.455+0000 7f0adeffd700 1 -- 172.21.15.32:0/1740522903 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae0142820 con 0x7f0ae005f3b0 2020-12-05T17:31:58.461 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.455+0000 7f0adeffd700 1 --2- 172.21.15.32:0/1740522903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ae005f3b0 0x7f0ae005f790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f0ac8002f30 tx=0x7f0ac8006e80).ready entity=mon.0 client_cookie=57fd7998a47fd215 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:58.463 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.455+0000 7f0addffb700 1 -- 172.21.15.32:0/1740522903 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ac80097f0 con 0x7f0ae005f3b0 2020-12-05T17:31:58.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.455+0000 7f0addffb700 1 -- 172.21.15.32:0/1740522903 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac8009950 con 0x7f0ae005f3b0 2020-12-05T17:31:58.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.455+0000 7f0addffb700 1 -- 172.21.15.32:0/1740522903 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ac8009cb0 con 0x7f0ae005f3b0 2020-12-05T17:31:58.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.455+0000 7f0adffff700 1 -- 172.21.15.32:0/1740522903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ae005f3b0 msgr2=0x7f0ae005f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:58.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.455+0000 7f0adffff700 1 --2- 172.21.15.32:0/1740522903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ae005f3b0 0x7f0ae005f790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f0ac8002f30 tx=0x7f0ac8006e80).stop 2020-12-05T17:31:58.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.455+0000 7f0adffff700 1 -- 172.21.15.32:0/1740522903 shutdown_connections 2020-12-05T17:31:58.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.455+0000 7f0adffff700 1 --2- 172.21.15.32:0/1740522903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ae005f3b0 0x7f0ae005f790 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.455+0000 7f0adffff700 1 -- 172.21.15.32:0/1740522903 >> 172.21.15.32:0/1740522903 conn(0x7f0ae00b8f20 msgr2=0x7f0ae00b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:58.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.455+0000 7f0adffff700 1 -- 172.21.15.32:0/1740522903 shutdown_connections 2020-12-05T17:31:58.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.455+0000 7f0adffff700 1 -- 172.21.15.32:0/1740522903 wait complete. 2020-12-05T17:31:58.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0adffff700 1 Processor -- start 2020-12-05T17:31:58.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0adffff700 1 -- start start 2020-12-05T17:31:58.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0adffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ae005f3b0 0x7f0ae0145fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:58.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0adffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ae005feb0 0x7f0ae0154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:58.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0adffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ae00b2fb0 0x7f0ae01552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:58.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0adffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae0065770 con 0x7f0ae005f3b0 2020-12-05T17:31:58.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0adffff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae01516f0 con 0x7f0ae005feb0 2020-12-05T17:31:58.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0adffff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae0066800 con 0x7f0ae00b2fb0 2020-12-05T17:31:58.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0adeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ae005f3b0 0x7f0ae0145fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:58.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0adeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ae005f3b0 0x7f0ae0145fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33104/0 (socket says 172.21.15.32:33104) 2020-12-05T17:31:58.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0adeffd700 1 -- 172.21.15.32:0/1327970613 learned_addr learned my addr 172.21.15.32:0/1327970613 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:58.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0adf7fe700 1 --2- 172.21.15.32:0/1327970613 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ae00b2fb0 0x7f0ae01552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:58.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0ade7fc700 1 --2- 172.21.15.32:0/1327970613 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ae005feb0 0x7f0ae0154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:58.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0adeffd700 1 -- 172.21.15.32:0/1327970613 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ae00b2fb0 msgr2=0x7f0ae01552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:58.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0adeffd700 1 --2- 172.21.15.32:0/1327970613 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ae00b2fb0 0x7f0ae01552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0adeffd700 1 -- 172.21.15.32:0/1327970613 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ae005feb0 msgr2=0x7f0ae0154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:58.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0adeffd700 1 --2- 172.21.15.32:0/1327970613 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ae005feb0 0x7f0ae0154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0adeffd700 1 -- 172.21.15.32:0/1327970613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae0064f20 con 0x7f0ae005f3b0 2020-12-05T17:31:58.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0adeffd700 1 --2- 172.21.15.32:0/1327970613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ae005f3b0 0x7f0ae0145fc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f0ac8002f30 tx=0x7f0ac8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:58.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0ad7fff700 1 -- 172.21.15.32:0/1327970613 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ac8011d40 con 0x7f0ae005f3b0 2020-12-05T17:31:58.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0ad7fff700 1 -- 172.21.15.32:0/1327970613 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac801e460 con 0x7f0ae005f3b0 2020-12-05T17:31:58.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0adffff700 1 -- 172.21.15.32:0/1327970613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae0066bd0 con 0x7f0ae005f3b0 2020-12-05T17:31:58.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0ad7fff700 1 -- 172.21.15.32:0/1327970613 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ac8011ea0 con 0x7f0ae005f3b0 2020-12-05T17:31:58.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.459+0000 7f0adffff700 1 -- 172.21.15.32:0/1327970613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae0155930 con 0x7f0ae005f3b0 2020-12-05T17:31:58.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.463+0000 7f0adffff700 1 -- 172.21.15.32:0/1327970613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ae00655c0 con 0x7f0ae005f3b0 2020-12-05T17:31:58.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.463+0000 7f0ad7fff700 1 -- 172.21.15.32:0/1327970613 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0ac801a040 con 0x7f0ae005f3b0 2020-12-05T17:31:58.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.463+0000 7f0ad7fff700 1 --2- 172.21.15.32:0/1327970613 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0ac4055330 0x7f0ac40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:58.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.463+0000 7f0ad7fff700 1 -- 172.21.15.32:0/1327970613 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0ac806f240 con 0x7f0ae005f3b0 2020-12-05T17:31:58.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.463+0000 7f0ade7fc700 1 --2- 172.21.15.32:0/1327970613 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0ac4055330 0x7f0ac40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:58.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.463+0000 7f0ade7fc700 1 --2- 172.21.15.32:0/1327970613 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0ac4055330 0x7f0ac40577f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f0ad800b000 tx=0x7f0ad8009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:58.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.471+0000 7f0ad7fff700 1 -- 172.21.15.32:0/1327970613 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0ac8045380 con 0x7f0ae005f3b0 2020-12-05T17:31:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.611+0000 7f0adffff700 1 -- 172.21.15.32:0/1327970613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0ae0057050 con 0x7f0ae005f3b0 2020-12-05T17:31:58.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.611+0000 7f0ad7fff700 1 -- 172.21.15.32:0/1327970613 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0ac80230a0 con 0x7f0ae005f3b0 2020-12-05T17:31:58.616 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:31:58.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.615+0000 7f0adffff700 1 -- 172.21.15.32:0/1327970613 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0ac4055330 msgr2=0x7f0ac40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:58.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.615+0000 7f0adffff700 1 --2- 172.21.15.32:0/1327970613 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0ac4055330 0x7f0ac40577f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f0ad800b000 tx=0x7f0ad8009040).stop 2020-12-05T17:31:58.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.615+0000 7f0adffff700 1 -- 172.21.15.32:0/1327970613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ae005f3b0 msgr2=0x7f0ae0145fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:58.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.615+0000 7f0adffff700 1 --2- 172.21.15.32:0/1327970613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ae005f3b0 0x7f0ae0145fc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f0ac8002f30 tx=0x7f0ac8015000).stop 2020-12-05T17:31:58.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.615+0000 7f0adffff700 1 -- 172.21.15.32:0/1327970613 shutdown_connections 2020-12-05T17:31:58.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.615+0000 7f0adffff700 1 --2- 172.21.15.32:0/1327970613 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0ac4055330 0x7f0ac40577f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.615+0000 7f0adffff700 1 --2- 172.21.15.32:0/1327970613 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ae00b2fb0 0x7f0ae01552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.615+0000 7f0adffff700 1 --2- 172.21.15.32:0/1327970613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ae005f3b0 0x7f0ae0145fc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.615+0000 7f0adffff700 1 --2- 172.21.15.32:0/1327970613 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ae005feb0 0x7f0ae0154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.615+0000 7f0adffff700 1 -- 172.21.15.32:0/1327970613 >> 172.21.15.32:0/1327970613 conn(0x7f0ae00b8f20 msgr2=0x7f0ae00b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:58.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.615+0000 7f0adffff700 1 -- 172.21.15.32:0/1327970613 shutdown_connections 2020-12-05T17:31:58.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.615+0000 7f0adffff700 1 -- 172.21.15.32:0/1327970613 wait complete. 2020-12-05T17:31:58.629 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:31:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.743+0000 7f56e8964700 1 Processor -- start 2020-12-05T17:31:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.743+0000 7f56e8964700 1 -- start start 2020-12-05T17:31:58.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e8964700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56e405feb0 0x7f56e405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:58.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e8964700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56e40b2fb0 0x7f56e4061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:58.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e8964700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e40b2df0 con 0x7f56e405f3b0 2020-12-05T17:31:58.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e8964700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e40b3c90 con 0x7f56e40b2fb0 2020-12-05T17:31:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e8964700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e40b2b50 con 0x7f56e405feb0 2020-12-05T17:31:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56e40b2fb0 0x7f56e4061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56e40b2fb0 0x7f56e4061780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33116/0 (socket says 172.21.15.32:33116) 2020-12-05T17:31:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e359e700 1 -- 172.21.15.32:0/93243021 learned_addr learned my addr 172.21.15.32:0/93243021 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e259c700 1 --2- 172.21.15.32:0/93243021 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56e405feb0 0x7f56e405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e359e700 1 -- 172.21.15.32:0/93243021 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56e405feb0 msgr2=0x7f56e405f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e359e700 1 --2- 172.21.15.32:0/93243021 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56e405feb0 0x7f56e405f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e359e700 1 -- 172.21.15.32:0/93243021 >> v1:172.21.15.61:6789/0 conn(0x7f56e405f3b0 legacy=0x7f56e413c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:58.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e359e700 1 -- 172.21.15.32:0/93243021 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e4142820 con 0x7f56e40b2fb0 2020-12-05T17:31:58.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e359e700 1 --2- 172.21.15.32:0/93243021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e40b2fb0 0x7f56e4061780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f56dc0032b0 tx=0x7f56dc009270).ready entity=mon.0 client_cookie=229c475179abf8dc server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:58.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e1d9b700 1 -- 172.21.15.32:0/93243021 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56dc00bc00 con 0x7f56e40b2fb0 2020-12-05T17:31:58.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e1d9b700 1 -- 172.21.15.32:0/93243021 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56dc00bd60 con 0x7f56e40b2fb0 2020-12-05T17:31:58.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e1d9b700 1 -- 172.21.15.32:0/93243021 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56dc011400 con 0x7f56e40b2fb0 2020-12-05T17:31:58.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e8964700 1 -- 172.21.15.32:0/93243021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e40b2fb0 msgr2=0x7f56e4061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:58.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e8964700 1 --2- 172.21.15.32:0/93243021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e40b2fb0 0x7f56e4061780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f56dc0032b0 tx=0x7f56dc009270).stop 2020-12-05T17:31:58.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e8964700 1 -- 172.21.15.32:0/93243021 shutdown_connections 2020-12-05T17:31:58.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e8964700 1 --2- 172.21.15.32:0/93243021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e40b2fb0 0x7f56e4061780 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e8964700 1 --2- 172.21.15.32:0/93243021 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56e405feb0 0x7f56e405f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e8964700 1 -- 172.21.15.32:0/93243021 >> 172.21.15.32:0/93243021 conn(0x7f56e40b8f20 msgr2=0x7f56e40b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:58.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e8964700 1 -- 172.21.15.32:0/93243021 shutdown_connections 2020-12-05T17:31:58.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.747+0000 7f56e8964700 1 -- 172.21.15.32:0/93243021 wait complete. 2020-12-05T17:31:58.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.751+0000 7f56e8964700 1 Processor -- start 2020-12-05T17:31:58.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.751+0000 7f56e8964700 1 -- start start 2020-12-05T17:31:58.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.751+0000 7f56e8964700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56e405f3b0 0x7f56e4149100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:58.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.751+0000 7f56e8964700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e405feb0 0x7f56e41495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:58.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.751+0000 7f56e8964700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f56e40b2fb0 0x7f56e4140a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:58.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.751+0000 7f56e8964700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e4114330 con 0x7f56e405feb0 2020-12-05T17:31:58.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.751+0000 7f56e8964700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e4113e10 con 0x7f56e40b2fb0 2020-12-05T17:31:58.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.751+0000 7f56e8964700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e4114090 con 0x7f56e405f3b0 2020-12-05T17:31:58.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.751+0000 7f56e259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e405feb0 0x7f56e41495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:58.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.751+0000 7f56e259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e405feb0 0x7f56e41495f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33120/0 (socket says 172.21.15.32:33120) 2020-12-05T17:31:58.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.751+0000 7f56e259c700 1 -- 172.21.15.32:0/2097585197 learned_addr learned my addr 172.21.15.32:0/2097585197 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:58.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.751+0000 7f56e2d9d700 1 --2- 172.21.15.32:0/2097585197 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56e405f3b0 0x7f56e4149100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:58.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.751+0000 7f56e359e700 1 --2- 172.21.15.32:0/2097585197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f56e40b2fb0 0x7f56e4140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:58.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.755+0000 7f56e259c700 1 -- 172.21.15.32:0/2097585197 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56e405f3b0 msgr2=0x7f56e4149100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:58.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.755+0000 7f56e259c700 1 --2- 172.21.15.32:0/2097585197 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56e405f3b0 0x7f56e4149100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.755+0000 7f56e259c700 1 -- 172.21.15.32:0/2097585197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f56e40b2fb0 msgr2=0x7f56e4140a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:58.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.755+0000 7f56e259c700 1 --2- 172.21.15.32:0/2097585197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f56e40b2fb0 0x7f56e4140a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.755+0000 7f56e259c700 1 -- 172.21.15.32:0/2097585197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e4147600 con 0x7f56e405feb0 2020-12-05T17:31:58.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.755+0000 7f56e259c700 1 --2- 172.21.15.32:0/2097585197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e405feb0 0x7f56e41495f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f56cc003070 tx=0x7f56cc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:58.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.755+0000 7f56dbfff700 1 -- 172.21.15.32:0/2097585197 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56cc009790 con 0x7f56e405feb0 2020-12-05T17:31:58.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.755+0000 7f56dbfff700 1 -- 172.21.15.32:0/2097585197 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56cc0098f0 con 0x7f56e405feb0 2020-12-05T17:31:58.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.755+0000 7f56e8964700 1 -- 172.21.15.32:0/2097585197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e4147aa0 con 0x7f56e405feb0 2020-12-05T17:31:58.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.755+0000 7f56dbfff700 1 -- 172.21.15.32:0/2097585197 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56cc009c50 con 0x7f56e405feb0 2020-12-05T17:31:58.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.755+0000 7f56e8964700 1 -- 172.21.15.32:0/2097585197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e4148a40 con 0x7f56e405feb0 2020-12-05T17:31:58.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.755+0000 7f56e8964700 1 -- 172.21.15.32:0/2097585197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e4057050 con 0x7f56e405feb0 2020-12-05T17:31:58.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.755+0000 7f56dbfff700 1 -- 172.21.15.32:0/2097585197 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f56cc013480 con 0x7f56e405feb0 2020-12-05T17:31:58.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.755+0000 7f56dbfff700 1 --2- 172.21.15.32:0/2097585197 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f56c8055330 0x7f56c80577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:58.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.759+0000 7f56dbfff700 1 -- 172.21.15.32:0/2097585197 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f56cc06e700 con 0x7f56e405feb0 2020-12-05T17:31:58.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.759+0000 7f56e2d9d700 1 --2- 172.21.15.32:0/2097585197 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f56c8055330 0x7f56c80577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:58.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.759+0000 7f56e2d9d700 1 --2- 172.21.15.32:0/2097585197 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f56c8055330 0x7f56c80577f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f56d400d000 tx=0x7f56d4000b80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:58.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.763+0000 7f56dbfff700 1 -- 172.21.15.32:0/2097585197 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f56cc044840 con 0x7f56e405feb0 2020-12-05T17:31:58.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.907+0000 7f56e8964700 1 -- 172.21.15.32:0/2097585197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f56e405aab0 con 0x7f56e405feb0 2020-12-05T17:31:58.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.907+0000 7f56dbfff700 1 -- 172.21.15.32:0/2097585197 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7f56cc009db0 con 0x7f56e405feb0 2020-12-05T17:31:58.910 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:31:58.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.907+0000 7f56e8964700 1 -- 172.21.15.32:0/2097585197 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f56c8055330 msgr2=0x7f56c80577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:58.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.907+0000 7f56e8964700 1 --2- 172.21.15.32:0/2097585197 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f56c8055330 0x7f56c80577f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f56d400d000 tx=0x7f56d4000b80).stop 2020-12-05T17:31:58.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.907+0000 7f56e8964700 1 -- 172.21.15.32:0/2097585197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e405feb0 msgr2=0x7f56e41495f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:58.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.907+0000 7f56e8964700 1 --2- 172.21.15.32:0/2097585197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e405feb0 0x7f56e41495f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f56cc003070 tx=0x7f56cc008040).stop 2020-12-05T17:31:58.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.907+0000 7f56e8964700 1 -- 172.21.15.32:0/2097585197 shutdown_connections 2020-12-05T17:31:58.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.907+0000 7f56e8964700 1 --2- 172.21.15.32:0/2097585197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f56e40b2fb0 0x7f56e4140a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.907+0000 7f56e8964700 1 --2- 172.21.15.32:0/2097585197 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f56c8055330 0x7f56c80577f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.907+0000 7f56e8964700 1 --2- 172.21.15.32:0/2097585197 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56e405f3b0 0x7f56e4149100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.907+0000 7f56e8964700 1 --2- 172.21.15.32:0/2097585197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e405feb0 0x7f56e41495f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:58.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.907+0000 7f56e8964700 1 -- 172.21.15.32:0/2097585197 >> 172.21.15.32:0/2097585197 conn(0x7f56e40b8f20 msgr2=0x7f56e40b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:58.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.907+0000 7f56e8964700 1 -- 172.21.15.32:0/2097585197 shutdown_connections 2020-12-05T17:31:58.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:58.907+0000 7f56e8964700 1 -- 172.21.15.32:0/2097585197 wait complete. 2020-12-05T17:31:58.924 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:31:59.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.039+0000 7f904f4a6700 1 Processor -- start 2020-12-05T17:31:59.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.039+0000 7f904f4a6700 1 -- start start 2020-12-05T17:31:59.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904f4a6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90480b1dc0 0x7f90480b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:59.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904f4a6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f90480af5f0 0x7f90480aed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:59.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904f4a6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90481468e0 con 0x7f90480afd30 2020-12-05T17:31:59.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904f4a6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9048142c90 con 0x7f90480af5f0 2020-12-05T17:31:59.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904f4a6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9048146b10 con 0x7f90480b1dc0 2020-12-05T17:31:59.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904eca5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f90480af5f0 0x7f90480aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:59.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904eca5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f90480af5f0 0x7f90480aed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33130/0 (socket says 172.21.15.32:33130) 2020-12-05T17:31:59.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904eca5700 1 -- 172.21.15.32:0/1112364087 learned_addr learned my addr 172.21.15.32:0/1112364087 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:59.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904dca3700 1 --1- 172.21.15.32:0/1112364087 >> v1:172.21.15.32:6789/0 conn(0x7f90480afd30 0x7f90480ae600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35774/0 (socket says 172.21.15.32:35774) 2020-12-05T17:31:59.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904e4a4700 1 --2- 172.21.15.32:0/1112364087 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90480b1dc0 0x7f90480b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:59.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904eca5700 1 -- 172.21.15.32:0/1112364087 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90480b1dc0 msgr2=0x7f90480b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:31:59.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904eca5700 1 --2- 172.21.15.32:0/1112364087 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90480b1dc0 0x7f90480b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:59.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904eca5700 1 -- 172.21.15.32:0/1112364087 >> v1:172.21.15.32:6789/0 conn(0x7f90480afd30 legacy=0x7f90480ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:59.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904eca5700 1 -- 172.21.15.32:0/1112364087 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9048146db0 con 0x7f90480af5f0 2020-12-05T17:31:59.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904e4a4700 1 --2- 172.21.15.32:0/1112364087 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90480b1dc0 0x7f90480b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:59.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904eca5700 1 --2- 172.21.15.32:0/1112364087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90480af5f0 0x7f90480aed10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f90400032b0 tx=0x7f9040009270).ready entity=mon.0 client_cookie=8187d63573e35aba server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:59.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904d4a2700 1 -- 172.21.15.32:0/1112364087 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f904000bc00 con 0x7f90480af5f0 2020-12-05T17:31:59.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904d4a2700 1 -- 172.21.15.32:0/1112364087 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904000bd60 con 0x7f90480af5f0 2020-12-05T17:31:59.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904d4a2700 1 -- 172.21.15.32:0/1112364087 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9040011400 con 0x7f90480af5f0 2020-12-05T17:31:59.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904f4a6700 1 -- 172.21.15.32:0/1112364087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90480af5f0 msgr2=0x7f90480aed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:59.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904f4a6700 1 --2- 172.21.15.32:0/1112364087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90480af5f0 0x7f90480aed10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f90400032b0 tx=0x7f9040009270).stop 2020-12-05T17:31:59.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904f4a6700 1 -- 172.21.15.32:0/1112364087 shutdown_connections 2020-12-05T17:31:59.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904f4a6700 1 --2- 172.21.15.32:0/1112364087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90480af5f0 0x7f90480aed10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:59.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904f4a6700 1 --2- 172.21.15.32:0/1112364087 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90480b1dc0 0x7f90480b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:59.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904f4a6700 1 -- 172.21.15.32:0/1112364087 >> 172.21.15.32:0/1112364087 conn(0x7f90480620b0 msgr2=0x7f9048062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:59.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904f4a6700 1 -- 172.21.15.32:0/1112364087 shutdown_connections 2020-12-05T17:31:59.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.043+0000 7f904f4a6700 1 -- 172.21.15.32:0/1112364087 wait complete. 2020-12-05T17:31:59.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.047+0000 7f904f4a6700 1 Processor -- start 2020-12-05T17:31:59.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.047+0000 7f904f4a6700 1 -- start start 2020-12-05T17:31:59.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.047+0000 7f904f4a6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f90480af5f0 0x7f9048063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:59.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.047+0000 7f904f4a6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90480afd30 0x7f90480669b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:59.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.047+0000 7f904f4a6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90480b1dc0 0x7f904813f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:59.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.047+0000 7f904f4a6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90481550e0 con 0x7f90480afd30 2020-12-05T17:31:59.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.047+0000 7f904f4a6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9048066470 con 0x7f90480af5f0 2020-12-05T17:31:59.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.047+0000 7f904f4a6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90480653b0 con 0x7f90480b1dc0 2020-12-05T17:31:59.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.047+0000 7f904dca3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90480afd30 0x7f90480669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:59.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.047+0000 7f904dca3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90480afd30 0x7f90480669b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33132/0 (socket says 172.21.15.32:33132) 2020-12-05T17:31:59.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.047+0000 7f904dca3700 1 -- 172.21.15.32:0/4091101590 learned_addr learned my addr 172.21.15.32:0/4091101590 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:59.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.047+0000 7f904eca5700 1 --2- 172.21.15.32:0/4091101590 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90480b1dc0 0x7f904813f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:59.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.047+0000 7f904e4a4700 1 --2- 172.21.15.32:0/4091101590 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f90480af5f0 0x7f9048063040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:59.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.047+0000 7f904dca3700 1 -- 172.21.15.32:0/4091101590 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90480b1dc0 msgr2=0x7f904813f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:59.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.051+0000 7f904dca3700 1 --2- 172.21.15.32:0/4091101590 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90480b1dc0 0x7f904813f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:59.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.051+0000 7f904dca3700 1 -- 172.21.15.32:0/4091101590 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f90480af5f0 msgr2=0x7f9048063040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:59.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.051+0000 7f904dca3700 1 --2- 172.21.15.32:0/4091101590 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f90480af5f0 0x7f9048063040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:59.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.051+0000 7f904dca3700 1 -- 172.21.15.32:0/4091101590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9048065650 con 0x7f90480afd30 2020-12-05T17:31:59.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.051+0000 7f904e4a4700 1 --2- 172.21.15.32:0/4091101590 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f90480af5f0 0x7f9048063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:59.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.051+0000 7f904eca5700 1 --2- 172.21.15.32:0/4091101590 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90480b1dc0 0x7f904813f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:59.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.051+0000 7f904dca3700 1 --2- 172.21.15.32:0/4091101590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90480afd30 0x7f90480669b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f90480ae4e0 tx=0x7f9038007ba0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:59.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.051+0000 7f90377fe700 1 -- 172.21.15.32:0/4091101590 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90380072e0 con 0x7f90480afd30 2020-12-05T17:31:59.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.051+0000 7f90377fe700 1 -- 172.21.15.32:0/4091101590 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9038005520 con 0x7f90480afd30 2020-12-05T17:31:59.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.051+0000 7f904f4a6700 1 -- 172.21.15.32:0/4091101590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90481463a0 con 0x7f90480afd30 2020-12-05T17:31:59.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.051+0000 7f90377fe700 1 -- 172.21.15.32:0/4091101590 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9038005880 con 0x7f90480afd30 2020-12-05T17:31:59.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.051+0000 7f904f4a6700 1 -- 172.21.15.32:0/4091101590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9048145210 con 0x7f90480afd30 2020-12-05T17:31:59.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.051+0000 7f904f4a6700 1 -- 172.21.15.32:0/4091101590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9048146720 con 0x7f90480afd30 2020-12-05T17:31:59.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.051+0000 7f90377fe700 1 -- 172.21.15.32:0/4091101590 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f903801a060 con 0x7f90480afd30 2020-12-05T17:31:59.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.051+0000 7f90377fe700 1 --2- 172.21.15.32:0/4091101590 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f90280554e0 0x7f90280579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:59.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.051+0000 7f90377fe700 1 -- 172.21.15.32:0/4091101590 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f903806d7d0 con 0x7f90480afd30 2020-12-05T17:31:59.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.051+0000 7f904e4a4700 1 --2- 172.21.15.32:0/4091101590 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f90280554e0 0x7f90280579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:59.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.055+0000 7f904e4a4700 1 --2- 172.21.15.32:0/4091101590 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f90280554e0 0x7f90280579a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f904400c000 tx=0x7f9044008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:59.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.059+0000 7f90377fe700 1 -- 172.21.15.32:0/4091101590 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9038043970 con 0x7f90480afd30 2020-12-05T17:31:59.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.199+0000 7f904f4a6700 1 -- 172.21.15.32:0/4091101590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9048155220 con 0x7f90480afd30 2020-12-05T17:31:59.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.199+0000 7f90377fe700 1 -- 172.21.15.32:0/4091101590 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7f9038018070 con 0x7f90480afd30 2020-12-05T17:31:59.203 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:31:59.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.199+0000 7f904f4a6700 1 -- 172.21.15.32:0/4091101590 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f90280554e0 msgr2=0x7f90280579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:59.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.199+0000 7f904f4a6700 1 --2- 172.21.15.32:0/4091101590 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f90280554e0 0x7f90280579a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f904400c000 tx=0x7f9044008040).stop 2020-12-05T17:31:59.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.199+0000 7f904f4a6700 1 -- 172.21.15.32:0/4091101590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90480afd30 msgr2=0x7f90480669b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:59.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.199+0000 7f904f4a6700 1 --2- 172.21.15.32:0/4091101590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90480afd30 0x7f90480669b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f90480ae4e0 tx=0x7f9038007ba0).stop 2020-12-05T17:31:59.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.203+0000 7f904f4a6700 1 -- 172.21.15.32:0/4091101590 shutdown_connections 2020-12-05T17:31:59.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.203+0000 7f904f4a6700 1 --2- 172.21.15.32:0/4091101590 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f90280554e0 0x7f90280579a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:59.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.203+0000 7f904f4a6700 1 --2- 172.21.15.32:0/4091101590 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90480b1dc0 0x7f904813f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:59.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.203+0000 7f904f4a6700 1 --2- 172.21.15.32:0/4091101590 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f90480af5f0 0x7f9048063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:59.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.203+0000 7f904f4a6700 1 --2- 172.21.15.32:0/4091101590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90480afd30 0x7f90480669b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:59.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.203+0000 7f904f4a6700 1 -- 172.21.15.32:0/4091101590 >> 172.21.15.32:0/4091101590 conn(0x7f90480620b0 msgr2=0x7f904805f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:59.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.203+0000 7f904f4a6700 1 -- 172.21.15.32:0/4091101590 shutdown_connections 2020-12-05T17:31:59.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.203+0000 7f904f4a6700 1 -- 172.21.15.32:0/4091101590 wait complete. 2020-12-05T17:31:59.217 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:31:59.217 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:31:59.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.327+0000 7f7b8ffff700 1 Processor -- start 2020-12-05T17:31:59.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.327+0000 7f7b8ffff700 1 -- start start 2020-12-05T17:31:59.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.327+0000 7f7b8ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7b90061780 0x7f7b900b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:59.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.327+0000 7f7b8ffff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b90062970 con 0x7f7b9013c6f0 2020-12-05T17:31:59.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.327+0000 7f7b8ffff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b900add70 con 0x7f7b900b78d0 2020-12-05T17:31:59.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.327+0000 7f7b8ffff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b900b2340 con 0x7f7b90061780 2020-12-05T17:31:59.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.327+0000 7f7b8e7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7b90061780 0x7f7b900b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:59.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.327+0000 7f7b8e7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7b90061780 0x7f7b900b73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33142/0 (socket says 172.21.15.32:33142) 2020-12-05T17:31:59.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.327+0000 7f7b8e7fc700 1 -- 172.21.15.32:0/3126989177 learned_addr learned my addr 172.21.15.32:0/3126989177 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:59.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.331+0000 7f7b8e7fc700 1 -- 172.21.15.32:0/3126989177 >> v1:172.21.15.61:6789/0 conn(0x7f7b900b78d0 legacy=0x7f7b90145c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:59.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.331+0000 7f7b8e7fc700 1 -- 172.21.15.32:0/3126989177 >> v1:172.21.15.32:6789/0 conn(0x7f7b9013c6f0 legacy=0x7f7b900ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:59.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.331+0000 7f7b8dffb700 1 -- 172.21.15.32:0/3126989177 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 501413674 0 0) 0x7f7b90062970 con 0x7f7b9013c6f0 2020-12-05T17:31:59.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.331+0000 7f7b8e7fc700 1 -- 172.21.15.32:0/3126989177 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b900ae010 con 0x7f7b90061780 2020-12-05T17:31:59.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.331+0000 7f7b8e7fc700 1 --2- 172.21.15.32:0/3126989177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b90061780 0x7f7b900b73e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f7b780032b0 tx=0x7f7b78009270).ready entity=mon.0 client_cookie=d8af2d58c6db157b server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:59.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.331+0000 7f7b8dffb700 1 -- 172.21.15.32:0/3126989177 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b7800bc00 con 0x7f7b90061780 2020-12-05T17:31:59.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.331+0000 7f7b8dffb700 1 -- 172.21.15.32:0/3126989177 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7800bd60 con 0x7f7b90061780 2020-12-05T17:31:59.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.331+0000 7f7b8dffb700 1 -- 172.21.15.32:0/3126989177 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b78011400 con 0x7f7b90061780 2020-12-05T17:31:59.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.331+0000 7f7b8ffff700 1 -- 172.21.15.32:0/3126989177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b90061780 msgr2=0x7f7b900b73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:59.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.331+0000 7f7b8ffff700 1 --2- 172.21.15.32:0/3126989177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b90061780 0x7f7b900b73e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f7b780032b0 tx=0x7f7b78009270).stop 2020-12-05T17:31:59.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.331+0000 7f7b8ffff700 1 -- 172.21.15.32:0/3126989177 shutdown_connections 2020-12-05T17:31:59.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.331+0000 7f7b8ffff700 1 --2- 172.21.15.32:0/3126989177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b90061780 0x7f7b900b73e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:59.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.331+0000 7f7b8ffff700 1 -- 172.21.15.32:0/3126989177 >> 172.21.15.32:0/3126989177 conn(0x7f7b9005bc20 msgr2=0x7f7b900b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:59.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.331+0000 7f7b8ffff700 1 -- 172.21.15.32:0/3126989177 shutdown_connections 2020-12-05T17:31:59.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.331+0000 7f7b8ffff700 1 -- 172.21.15.32:0/3126989177 wait complete. 2020-12-05T17:31:59.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.331+0000 7f7b8ffff700 1 Processor -- start 2020-12-05T17:31:59.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.331+0000 7f7b8ffff700 1 -- start start 2020-12-05T17:31:59.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b8ffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7b90061780 0x7f7b90062fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:59.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b8ffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b900b78d0 0x7f7b900af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:59.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b8ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b9013c6f0 0x7f7b90151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:59.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b8ffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b90064f40 con 0x7f7b9013c6f0 2020-12-05T17:31:59.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b8ffff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b900651e0 con 0x7f7b900b78d0 2020-12-05T17:31:59.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b8ffff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b90064bc0 con 0x7f7b90061780 2020-12-05T17:31:59.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b8f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b9013c6f0 0x7f7b90151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:59.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b8f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b9013c6f0 0x7f7b90151af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33148/0 (socket says 172.21.15.32:33148) 2020-12-05T17:31:59.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b8f7fe700 1 -- 172.21.15.32:0/2836076038 learned_addr learned my addr 172.21.15.32:0/2836076038 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:31:59.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b8effd700 1 --2- 172.21.15.32:0/2836076038 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7b90061780 0x7f7b90062fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b8e7fc700 1 --2- 172.21.15.32:0/2836076038 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b900b78d0 0x7f7b900af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b8f7fe700 1 -- 172.21.15.32:0/2836076038 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7b90061780 msgr2=0x7f7b90062fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b8f7fe700 1 --2- 172.21.15.32:0/2836076038 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7b90061780 0x7f7b90062fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b8f7fe700 1 -- 172.21.15.32:0/2836076038 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b900b78d0 msgr2=0x7f7b900af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b8f7fe700 1 --2- 172.21.15.32:0/2836076038 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b900b78d0 0x7f7b900af6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b8f7fe700 1 -- 172.21.15.32:0/2836076038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b90064500 con 0x7f7b9013c6f0 2020-12-05T17:31:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b8e7fc700 1 --2- 172.21.15.32:0/2836076038 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b900b78d0 0x7f7b900af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:31:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b8f7fe700 1 --2- 172.21.15.32:0/2836076038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b9013c6f0 0x7f7b90151af0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f7b9005c4c0 tx=0x7f7b88007b60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b87fff700 1 -- 172.21.15.32:0/2836076038 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b88013410 con 0x7f7b9013c6f0 2020-12-05T17:31:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b87fff700 1 -- 172.21.15.32:0/2836076038 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b88013570 con 0x7f7b9013c6f0 2020-12-05T17:31:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b8ffff700 1 -- 172.21.15.32:0/2836076038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b90065400 con 0x7f7b9013c6f0 2020-12-05T17:31:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b87fff700 1 -- 172.21.15.32:0/2836076038 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b880138d0 con 0x7f7b9013c6f0 2020-12-05T17:31:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b8ffff700 1 -- 172.21.15.32:0/2836076038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b90152180 con 0x7f7b9013c6f0 2020-12-05T17:31:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b8ffff700 1 -- 172.21.15.32:0/2836076038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b900646f0 con 0x7f7b9013c6f0 2020-12-05T17:31:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.335+0000 7f7b87fff700 1 -- 172.21.15.32:0/2836076038 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7b88010030 con 0x7f7b9013c6f0 2020-12-05T17:31:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.339+0000 7f7b87fff700 1 --2- 172.21.15.32:0/2836076038 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7b74055330 0x7f7b740577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:31:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.339+0000 7f7b87fff700 1 -- 172.21.15.32:0/2836076038 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f7b88001e40 con 0x7f7b9013c6f0 2020-12-05T17:31:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.339+0000 7f7b8effd700 1 --2- 172.21.15.32:0/2836076038 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7b74055330 0x7f7b740577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:31:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.339+0000 7f7b8effd700 1 --2- 172.21.15.32:0/2836076038 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7b74055330 0x7f7b740577f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f7b8000d000 tx=0x7f7b80009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:31:59.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.347+0000 7f7b87fff700 1 -- 172.21.15.32:0/2836076038 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7b900646f0 con 0x7f7b9013c6f0 2020-12-05T17:31:59.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.535+0000 7f7b8ffff700 1 -- 172.21.15.32:0/2836076038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b900646f0 con 0x7f7b9013c6f0 2020-12-05T17:31:59.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.535+0000 7f7b87fff700 1 -- 172.21.15.32:0/2836076038 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2134 (secure 0 0 0) 0x7f7b900646f0 con 0x7f7b9013c6f0 2020-12-05T17:31:59.540 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:31:59.540 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4023797731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4023797731},{"type":"v1","addr":"172.21.15.61:6833","nonce":4023797731}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:31:41.611214+0000","modified":"2020-12-05T17:31:42.614053+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3503848021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3503848021},{"type":"v1","addr":"172.21.15.32:6835","nonce":3503848021}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-12-05T17:31:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.539+0000 7f7b8ffff700 1 -- 172.21.15.32:0/2836076038 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7b74055330 msgr2=0x7f7b740577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.539+0000 7f7b8ffff700 1 --2- 172.21.15.32:0/2836076038 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7b74055330 0x7f7b740577f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f7b8000d000 tx=0x7f7b80009040).stop 2020-12-05T17:31:59.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.539+0000 7f7b8ffff700 1 -- 172.21.15.32:0/2836076038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b9013c6f0 msgr2=0x7f7b90151af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:31:59.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.539+0000 7f7b8ffff700 1 --2- 172.21.15.32:0/2836076038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b9013c6f0 0x7f7b90151af0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f7b9005c4c0 tx=0x7f7b88007b60).stop 2020-12-05T17:31:59.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.539+0000 7f7b8ffff700 1 -- 172.21.15.32:0/2836076038 shutdown_connections 2020-12-05T17:31:59.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.539+0000 7f7b8ffff700 1 --2- 172.21.15.32:0/2836076038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b9013c6f0 0x7f7b90151af0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:59.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.539+0000 7f7b8ffff700 1 --2- 172.21.15.32:0/2836076038 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7b74055330 0x7f7b740577f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:59.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.539+0000 7f7b8ffff700 1 --2- 172.21.15.32:0/2836076038 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7b90061780 0x7f7b90062fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:59.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.539+0000 7f7b8ffff700 1 --2- 172.21.15.32:0/2836076038 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b900b78d0 0x7f7b900af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:31:59.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.539+0000 7f7b8ffff700 1 -- 172.21.15.32:0/2836076038 >> 172.21.15.32:0/2836076038 conn(0x7f7b9005bc20 msgr2=0x7f7b900b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:31:59.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.539+0000 7f7b8ffff700 1 -- 172.21.15.32:0/2836076038 shutdown_connections 2020-12-05T17:31:59.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:31:59.539+0000 7f7b8ffff700 1 -- 172.21.15.32:0/2836076038 wait complete. 2020-12-05T17:31:59.544 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 5 2020-12-05T17:31:59.755 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:31:41.611214+0000', 'modified': '2020-12-05T17:31:42.614053+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4275}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4275': {'gid': 4275, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/3503848021', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 3503848021}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 3503848021}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:31:59.755 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:31:59.756 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:31:59.887 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":4275,"want_state":"up:active","state":"up:active","rank_uptime":18.264521361,"mdsmap_epoch":5,"osdmap_epoch":25,"osdmap_epoch_barrier":25,"uptime":24.670498691999999} 2020-12-05T17:31:59.887 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 4275, "mdsmap_epoch": 5, "osdmap_epoch": 25, "osdmap_epoch_barrier": 25, "rank_uptime": 18.264521361, "state": "up:active", "uptime": 24.670498692, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:31:59.888 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:31:59.888 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-12-05T17:31:59.889 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi002.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-12-05T17:31:59.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:31:59.889 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:31:59.897 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:31:59.956 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:31:59.957 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:32:00.014 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:32:00.016 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:32:00.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:00.017 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:32:00.063 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:32:00.064 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:32:00.107 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:32:00.124 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:32:00.124 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:32:00.178 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:32:00.179 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:32:00.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:00.181 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:32:00.196 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:2020-12-05T17:32:00.191+0000 7fb1733a80c0 -1 init, newargv = 0x55d041965dc0 newargc=9 2020-12-05T17:32:00.196 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[10374]: starting ceph client 2020-12-05T17:32:00.208 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[10374]: starting fuse 2020-12-05T17:32:01.302 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:32:01.318 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:32:01.318 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:32:01.376 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:32:01.377 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:32:01.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:01.378 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:32:01.423 INFO:teuthology.orchestra.run.smithi002.stdout:50 2020-12-05T17:32:01.424 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [50] 2020-12-05T17:32:01.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:01.424 INFO:teuthology.orchestra.run.smithi002:> sudo python3 -c ' 2020-12-05T17:32:01.425 INFO:teuthology.orchestra.run.smithi002:> import glob 2020-12-05T17:32:01.425 INFO:teuthology.orchestra.run.smithi002:> import re 2020-12-05T17:32:01.425 INFO:teuthology.orchestra.run.smithi002:> import os 2020-12-05T17:32:01.426 INFO:teuthology.orchestra.run.smithi002:> import subprocess 2020-12-05T17:32:01.426 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:32:01.426 INFO:teuthology.orchestra.run.smithi002:> def find_socket(client_name): 2020-12-05T17:32:01.427 INFO:teuthology.orchestra.run.smithi002:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-12-05T17:32:01.427 INFO:teuthology.orchestra.run.smithi002:> files = glob.glob(asok_path) 2020-12-05T17:32:01.427 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:32:01.428 INFO:teuthology.orchestra.run.smithi002:> # Given a non-glob path, it better be there 2020-12-05T17:32:01.428 INFO:teuthology.orchestra.run.smithi002:> if "*" not in asok_path: 2020-12-05T17:32:01.428 INFO:teuthology.orchestra.run.smithi002:> assert(len(files) == 1) 2020-12-05T17:32:01.429 INFO:teuthology.orchestra.run.smithi002:> return files[0] 2020-12-05T17:32:01.429 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:32:01.429 INFO:teuthology.orchestra.run.smithi002:> for f in files: 2020-12-05T17:32:01.429 INFO:teuthology.orchestra.run.smithi002:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:32:01.430 INFO:teuthology.orchestra.run.smithi002:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:32:01.430 INFO:teuthology.orchestra.run.smithi002:> return f 2020-12-05T17:32:01.430 INFO:teuthology.orchestra.run.smithi002:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:32:01.431 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:32:01.431 INFO:teuthology.orchestra.run.smithi002:> print(find_socket("client.1")) 2020-12-05T17:32:01.431 INFO:teuthology.orchestra.run.smithi002:> ' 2020-12-05T17:32:01.516 INFO:teuthology.orchestra.run.smithi002.stdout:/var/run/ceph/ceph-client.1.10374.asok 2020-12-05T17:32:01.521 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.10374.asok 2020-12-05T17:32:01.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:01.522 INFO:teuthology.orchestra.run.smithi002:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.10374.asok status 2020-12-05T17:32:01.723 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2020-12-05T17:32:01.724 INFO:teuthology.orchestra.run.smithi002.stdout: "metadata": { 2020-12-05T17:32:01.724 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:32:01.724 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:32:01.724 INFO:teuthology.orchestra.run.smithi002.stdout: "entity_id": "1", 2020-12-05T17:32:01.725 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": "smithi002", 2020-12-05T17:32:01.725 INFO:teuthology.orchestra.run.smithi002.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-12-05T17:32:01.725 INFO:teuthology.orchestra.run.smithi002.stdout: "pid": "10374", 2020-12-05T17:32:01.726 INFO:teuthology.orchestra.run.smithi002.stdout: "root": "/" 2020-12-05T17:32:01.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:32:01.729 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_count": 0, 2020-12-05T17:32:01.729 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_pinned_count": 0, 2020-12-05T17:32:01.730 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 4716, 2020-12-05T17:32:01.730 INFO:teuthology.orchestra.run.smithi002.stdout: "inst": { 2020-12-05T17:32:01.730 INFO:teuthology.orchestra.run.smithi002.stdout: "name": { 2020-12-05T17:32:01.731 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "client", 2020-12-05T17:32:01.731 INFO:teuthology.orchestra.run.smithi002.stdout: "num": 4716 2020-12-05T17:32:01.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:32:01.731 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2020-12-05T17:32:01.732 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2020-12-05T17:32:01.732 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2020-12-05T17:32:01.732 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2272898768 2020-12-05T17:32:01.732 INFO:teuthology.orchestra.run.smithi002.stdout: } 2020-12-05T17:32:01.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:32:01.733 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2020-12-05T17:32:01.733 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2020-12-05T17:32:01.733 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2020-12-05T17:32:01.734 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2272898768 2020-12-05T17:32:01.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:32:01.734 INFO:teuthology.orchestra.run.smithi002.stdout: "inst_str": "client.4716 172.21.15.2:0/2272898768", 2020-12-05T17:32:01.735 INFO:teuthology.orchestra.run.smithi002.stdout: "addr_str": "172.21.15.2:0/2272898768", 2020-12-05T17:32:01.735 INFO:teuthology.orchestra.run.smithi002.stdout: "inode_count": 1, 2020-12-05T17:32:01.735 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_epoch": 5, 2020-12-05T17:32:01.735 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch": 25, 2020-12-05T17:32:01.736 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:32:01.736 INFO:teuthology.orchestra.run.smithi002.stdout: "blacklisted": false 2020-12-05T17:32:01.736 INFO:teuthology.orchestra.run.smithi002.stdout:} 2020-12-05T17:32:01.738 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:32:01.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.851+0000 7f0a4c91f700 1 Processor -- start 2020-12-05T17:32:01.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.851+0000 7f0a4c91f700 1 -- start start 2020-12-05T17:32:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a4c91f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0a4805f3b0 0x7f0a4805f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:01.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a4c91f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0a4805feb0 0x7f0a480b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:01.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a4c91f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a48142450 con 0x7f0a48062400 2020-12-05T17:32:01.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a4c91f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a480b42f0 con 0x7f0a4805feb0 2020-12-05T17:32:01.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a4c91f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a48061780 con 0x7f0a4805f3b0 2020-12-05T17:32:01.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a4759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0a4805feb0 0x7f0a480b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:01.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a4659c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0a48062400 0x7f0a480b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35802/0 (socket says 172.21.15.32:35802) 2020-12-05T17:32:01.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a4659c700 1 -- 172.21.15.32:0/3364579801 learned_addr learned my addr 172.21.15.32:0/3364579801 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:01.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a46d9d700 1 --2- 172.21.15.32:0/3364579801 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0a4805f3b0 0x7f0a4805f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:01.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a45d9b700 1 -- 172.21.15.32:0/3364579801 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 690335126 0 0) 0x7f0a48142450 con 0x7f0a48062400 2020-12-05T17:32:01.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a45d9b700 1 -- 172.21.15.32:0/3364579801 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a28003230 con 0x7f0a48062400 2020-12-05T17:32:01.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a4759e700 1 -- 172.21.15.32:0/3364579801 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0a4805f3b0 msgr2=0x7f0a4805f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:01.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a4759e700 1 --2- 172.21.15.32:0/3364579801 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0a4805f3b0 0x7f0a4805f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:01.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a4759e700 1 -- 172.21.15.32:0/3364579801 >> v1:172.21.15.32:6789/0 conn(0x7f0a48062400 legacy=0x7f0a480b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:01.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a4759e700 1 -- 172.21.15.32:0/3364579801 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a48061a20 con 0x7f0a4805feb0 2020-12-05T17:32:01.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a4759e700 1 --2- 172.21.15.32:0/3364579801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a4805feb0 0x7f0a480b3e00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f0a400032b0 tx=0x7f0a400091a0).ready entity=mon.0 client_cookie=87df5eea9425c633 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:01.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a45d9b700 1 -- 172.21.15.32:0/3364579801 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a4000b940 con 0x7f0a4805feb0 2020-12-05T17:32:01.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a45d9b700 1 -- 172.21.15.32:0/3364579801 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4000baa0 con 0x7f0a4805feb0 2020-12-05T17:32:01.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a45d9b700 1 -- 172.21.15.32:0/3364579801 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a4000bc10 con 0x7f0a4805feb0 2020-12-05T17:32:01.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a4c91f700 1 -- 172.21.15.32:0/3364579801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a4805feb0 msgr2=0x7f0a480b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:01.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a4c91f700 1 --2- 172.21.15.32:0/3364579801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a4805feb0 0x7f0a480b3e00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f0a400032b0 tx=0x7f0a400091a0).stop 2020-12-05T17:32:01.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a4c91f700 1 -- 172.21.15.32:0/3364579801 shutdown_connections 2020-12-05T17:32:01.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a4c91f700 1 --2- 172.21.15.32:0/3364579801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a4805feb0 0x7f0a480b3e00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a4c91f700 1 --2- 172.21.15.32:0/3364579801 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0a4805f3b0 0x7f0a4805f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a4c91f700 1 -- 172.21.15.32:0/3364579801 >> 172.21.15.32:0/3364579801 conn(0x7f0a480b8ee0 msgr2=0x7f0a480b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a4c91f700 1 -- 172.21.15.32:0/3364579801 shutdown_connections 2020-12-05T17:32:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.855+0000 7f0a4c91f700 1 -- 172.21.15.32:0/3364579801 wait complete. 2020-12-05T17:32:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.859+0000 7f0a4c91f700 1 Processor -- start 2020-12-05T17:32:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.859+0000 7f0a4c91f700 1 -- start start 2020-12-05T17:32:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.859+0000 7f0a4c91f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a4805f3b0 0x7f0a480b7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.859+0000 7f0a4c91f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a4805feb0 0x7f0a481518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.859+0000 7f0a4c91f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0a48062400 0x7f0a48151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.859+0000 7f0a4c91f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a48145b90 con 0x7f0a4805feb0 2020-12-05T17:32:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.859+0000 7f0a4c91f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a48145e30 con 0x7f0a4805f3b0 2020-12-05T17:32:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.859+0000 7f0a4c91f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a481460d0 con 0x7f0a48062400 2020-12-05T17:32:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.859+0000 7f0a4659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a4805feb0 0x7f0a481518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.859+0000 7f0a4659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a4805feb0 0x7f0a481518b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33162/0 (socket says 172.21.15.32:33162) 2020-12-05T17:32:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.859+0000 7f0a4659c700 1 -- 172.21.15.32:0/3260399690 learned_addr learned my addr 172.21.15.32:0/3260399690 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.859+0000 7f0a46d9d700 1 --2- 172.21.15.32:0/3260399690 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a4805f3b0 0x7f0a480b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.859+0000 7f0a4759e700 1 --2- 172.21.15.32:0/3260399690 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0a48062400 0x7f0a48151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:01.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.859+0000 7f0a4659c700 1 -- 172.21.15.32:0/3260399690 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0a48062400 msgr2=0x7f0a48151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:01.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.859+0000 7f0a4659c700 1 --2- 172.21.15.32:0/3260399690 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0a48062400 0x7f0a48151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:01.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.859+0000 7f0a4659c700 1 -- 172.21.15.32:0/3260399690 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a4805f3b0 msgr2=0x7f0a480b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:01.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.859+0000 7f0a4659c700 1 --2- 172.21.15.32:0/3260399690 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a4805f3b0 0x7f0a480b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.859+0000 7f0a4659c700 1 -- 172.21.15.32:0/3260399690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a48151410 con 0x7f0a4805feb0 2020-12-05T17:32:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.859+0000 7f0a4659c700 1 --2- 172.21.15.32:0/3260399690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a4805feb0 0x7f0a481518b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f0a300010d0 tx=0x7f0a30007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.863+0000 7f0a3ffff700 1 -- 172.21.15.32:0/3260399690 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a300072e0 con 0x7f0a4805feb0 2020-12-05T17:32:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.863+0000 7f0a4c91f700 1 -- 172.21.15.32:0/3260399690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a48145810 con 0x7f0a4805feb0 2020-12-05T17:32:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.863+0000 7f0a3ffff700 1 -- 172.21.15.32:0/3260399690 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a30007440 con 0x7f0a4805feb0 2020-12-05T17:32:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.863+0000 7f0a3ffff700 1 -- 172.21.15.32:0/3260399690 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a300057d0 con 0x7f0a4805feb0 2020-12-05T17:32:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.863+0000 7f0a4c91f700 1 -- 172.21.15.32:0/3260399690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a48152290 con 0x7f0a4805feb0 2020-12-05T17:32:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.863+0000 7f0a4c91f700 1 -- 172.21.15.32:0/3260399690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a48146290 con 0x7f0a4805feb0 2020-12-05T17:32:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.863+0000 7f0a3ffff700 1 -- 172.21.15.32:0/3260399690 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0a30010070 con 0x7f0a4805feb0 2020-12-05T17:32:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.863+0000 7f0a3ffff700 1 --2- 172.21.15.32:0/3260399690 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0a2c055440 0x7f0a2c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.863+0000 7f0a3ffff700 1 -- 172.21.15.32:0/3260399690 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0a3006d840 con 0x7f0a4805feb0 2020-12-05T17:32:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.863+0000 7f0a46d9d700 1 --2- 172.21.15.32:0/3260399690 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0a2c055440 0x7f0a2c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.863+0000 7f0a46d9d700 1 --2- 172.21.15.32:0/3260399690 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0a2c055440 0x7f0a2c057900 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f0a3800b000 tx=0x7f0a38006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:01.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:01.871+0000 7f0a3ffff700 1 -- 172.21.15.32:0/3260399690 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0a30043940 con 0x7f0a4805feb0 2020-12-05T17:32:02.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.011+0000 7f0a4c91f700 1 -- 172.21.15.32:0/3260399690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a48145a00 con 0x7f0a4805feb0 2020-12-05T17:32:02.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.015+0000 7f0a3ffff700 1 -- 172.21.15.32:0/3260399690 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f0a30046f60 con 0x7f0a4805feb0 2020-12-05T17:32:02.019 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:32:02.019 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:31:41.610914+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:31:36.706361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-12-05T17:31:39.298269+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:32:02.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.015+0000 7f0a4c91f700 1 -- 172.21.15.32:0/3260399690 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0a2c055440 msgr2=0x7f0a2c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:02.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.015+0000 7f0a4c91f700 1 --2- 172.21.15.32:0/3260399690 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0a2c055440 0x7f0a2c057900 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f0a3800b000 tx=0x7f0a38006dc0).stop 2020-12-05T17:32:02.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.015+0000 7f0a4c91f700 1 -- 172.21.15.32:0/3260399690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a4805feb0 msgr2=0x7f0a481518b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:02.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.015+0000 7f0a4c91f700 1 --2- 172.21.15.32:0/3260399690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a4805feb0 0x7f0a481518b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f0a300010d0 tx=0x7f0a30007b50).stop 2020-12-05T17:32:02.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.015+0000 7f0a4c91f700 1 -- 172.21.15.32:0/3260399690 shutdown_connections 2020-12-05T17:32:02.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.015+0000 7f0a4c91f700 1 --2- 172.21.15.32:0/3260399690 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0a2c055440 0x7f0a2c057900 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.015+0000 7f0a4c91f700 1 --2- 172.21.15.32:0/3260399690 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0a48062400 0x7f0a48151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.015+0000 7f0a4c91f700 1 --2- 172.21.15.32:0/3260399690 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a4805f3b0 0x7f0a480b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.015+0000 7f0a4c91f700 1 --2- 172.21.15.32:0/3260399690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a4805feb0 0x7f0a481518b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.015+0000 7f0a4c91f700 1 -- 172.21.15.32:0/3260399690 >> 172.21.15.32:0/3260399690 conn(0x7f0a480b8ee0 msgr2=0x7f0a4813fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:02.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.015+0000 7f0a4c91f700 1 -- 172.21.15.32:0/3260399690 shutdown_connections 2020-12-05T17:32:02.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.019+0000 7f0a4c91f700 1 -- 172.21.15.32:0/3260399690 wait complete. 2020-12-05T17:32:02.033 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:31:36.706361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:31:39.298269+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:32:02.033 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:32:02.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.155+0000 7fdc1a6d8700 1 Processor -- start 2020-12-05T17:32:02.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.155+0000 7fdc1a6d8700 1 -- start start 2020-12-05T17:32:02.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.155+0000 7fdc1a6d8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc140b2a70 0x7fdc140b0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:02.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.155+0000 7fdc1a6d8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc140b06b0 con 0x7fdc140b2690 2020-12-05T17:32:02.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.155+0000 7fdc1a6d8700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1414a280 con 0x7fdc140b15e0 2020-12-05T17:32:02.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.155+0000 7fdc1a6d8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1414a520 con 0x7fdc140b2a70 2020-12-05T17:32:02.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.155+0000 7fdc13fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdc140b2690 0x7fdc1414a9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35816/0 (socket says 172.21.15.32:35816) 2020-12-05T17:32:02.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.155+0000 7fdc13fff700 1 -- 172.21.15.32:0/3237632022 learned_addr learned my addr 172.21.15.32:0/3237632022 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:02.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.155+0000 7fdc18c75700 1 --2- 172.21.15.32:0/3237632022 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc140b2a70 0x7fdc140b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:02.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.155+0000 7fdc12ffd700 1 -- 172.21.15.32:0/3237632022 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1036575119 0 0) 0x7fdc140b06b0 con 0x7fdc140b2690 2020-12-05T17:32:02.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.155+0000 7fdc12ffd700 1 -- 172.21.15.32:0/3237632022 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc00003230 con 0x7fdc140b2690 2020-12-05T17:32:02.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.155+0000 7fdc12ffd700 1 -- 172.21.15.32:0/3237632022 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3987293284 0 0) 0x7fdc1414a280 con 0x7fdc140b15e0 2020-12-05T17:32:02.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.155+0000 7fdc12ffd700 1 -- 172.21.15.32:0/3237632022 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc140b06b0 con 0x7fdc140b15e0 2020-12-05T17:32:02.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.155+0000 7fdc18c75700 1 -- 172.21.15.32:0/3237632022 >> v1:172.21.15.61:6789/0 conn(0x7fdc140b15e0 legacy=0x7fdc14146280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:02.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.155+0000 7fdc18c75700 1 -- 172.21.15.32:0/3237632022 >> v1:172.21.15.32:6789/0 conn(0x7fdc140b2690 legacy=0x7fdc1414a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:02.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.155+0000 7fdc18c75700 1 -- 172.21.15.32:0/3237632022 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1414b0d0 con 0x7fdc140b2a70 2020-12-05T17:32:02.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.155+0000 7fdc18c75700 1 --2- 172.21.15.32:0/3237632022 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc140b2a70 0x7fdc140b0010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fdc0c0032b0 tx=0x7fdc0c0091a0).ready entity=mon.2 client_cookie=2696cd1cd28d8a9e server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:02.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.159+0000 7fdc12ffd700 1 -- 172.21.15.32:0/3237632022 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc0c00b940 con 0x7fdc140b2a70 2020-12-05T17:32:02.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.159+0000 7fdc12ffd700 1 -- 172.21.15.32:0/3237632022 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0c00baa0 con 0x7fdc140b2a70 2020-12-05T17:32:02.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.159+0000 7fdc12ffd700 1 -- 172.21.15.32:0/3237632022 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc0c00bc10 con 0x7fdc140b2a70 2020-12-05T17:32:02.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.159+0000 7fdc1a6d8700 1 -- 172.21.15.32:0/3237632022 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc140b2a70 msgr2=0x7fdc140b0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:02.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.159+0000 7fdc1a6d8700 1 --2- 172.21.15.32:0/3237632022 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc140b2a70 0x7fdc140b0010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fdc0c0032b0 tx=0x7fdc0c0091a0).stop 2020-12-05T17:32:02.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.159+0000 7fdc1a6d8700 1 -- 172.21.15.32:0/3237632022 shutdown_connections 2020-12-05T17:32:02.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.159+0000 7fdc1a6d8700 1 --2- 172.21.15.32:0/3237632022 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc140b2a70 0x7fdc140b0010 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.159+0000 7fdc1a6d8700 1 -- 172.21.15.32:0/3237632022 >> 172.21.15.32:0/3237632022 conn(0x7fdc14062310 msgr2=0x7fdc140626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:02.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.159+0000 7fdc1a6d8700 1 -- 172.21.15.32:0/3237632022 shutdown_connections 2020-12-05T17:32:02.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.159+0000 7fdc1a6d8700 1 -- 172.21.15.32:0/3237632022 wait complete. 2020-12-05T17:32:02.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.159+0000 7fdc1a6d8700 1 Processor -- start 2020-12-05T17:32:02.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.159+0000 7fdc1a6d8700 1 -- start start 2020-12-05T17:32:02.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.163+0000 7fdc1a6d8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc140b15e0 0x7fdc14118ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:02.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.163+0000 7fdc1a6d8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc140b2690 0x7fdc140b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:02.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.163+0000 7fdc1a6d8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc140b2a70 0x7fdc141447c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:02.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.163+0000 7fdc1a6d8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc140651d0 con 0x7fdc140b15e0 2020-12-05T17:32:02.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.163+0000 7fdc1a6d8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc14065470 con 0x7fdc140b2a70 2020-12-05T17:32:02.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.163+0000 7fdc1a6d8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc14065710 con 0x7fdc140b2690 2020-12-05T17:32:02.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.163+0000 7fdc13fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc140b15e0 0x7fdc14118ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:02.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.163+0000 7fdc13fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc140b15e0 0x7fdc14118ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33178/0 (socket says 172.21.15.32:33178) 2020-12-05T17:32:02.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.163+0000 7fdc13fff700 1 -- 172.21.15.32:0/3422744668 learned_addr learned my addr 172.21.15.32:0/3422744668 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:02.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.163+0000 7fdc18c75700 1 --2- 172.21.15.32:0/3422744668 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc140b2a70 0x7fdc141447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:02.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.163+0000 7fdc137fe700 1 --2- 172.21.15.32:0/3422744668 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc140b2690 0x7fdc140b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:02.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.163+0000 7fdc13fff700 1 -- 172.21.15.32:0/3422744668 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc140b2690 msgr2=0x7fdc140b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:02.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.163+0000 7fdc13fff700 1 --2- 172.21.15.32:0/3422744668 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc140b2690 0x7fdc140b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.163+0000 7fdc13fff700 1 -- 172.21.15.32:0/3422744668 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc140b2a70 msgr2=0x7fdc141447c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.163+0000 7fdc13fff700 1 --2- 172.21.15.32:0/3422744668 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc140b2a70 0x7fdc141447c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.163+0000 7fdc13fff700 1 -- 172.21.15.32:0/3422744668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc140b4450 con 0x7fdc140b15e0 2020-12-05T17:32:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.163+0000 7fdc13fff700 1 --2- 172.21.15.32:0/3422744668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc140b15e0 0x7fdc14118ee0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fdc04001190 tx=0x7fdc04007b20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.163+0000 7fdc117fa700 1 -- 172.21.15.32:0/3422744668 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc040072e0 con 0x7fdc140b15e0 2020-12-05T17:32:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.163+0000 7fdc1a6d8700 1 -- 172.21.15.32:0/3422744668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc14117000 con 0x7fdc140b15e0 2020-12-05T17:32:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.163+0000 7fdc117fa700 1 -- 172.21.15.32:0/3422744668 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc04007440 con 0x7fdc140b15e0 2020-12-05T17:32:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.163+0000 7fdc117fa700 1 -- 172.21.15.32:0/3422744668 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc040057d0 con 0x7fdc140b15e0 2020-12-05T17:32:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.163+0000 7fdc1a6d8700 1 -- 172.21.15.32:0/3422744668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc141594e0 con 0x7fdc140b15e0 2020-12-05T17:32:02.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.163+0000 7fdc117fa700 1 -- 172.21.15.32:0/3422744668 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fdc04010070 con 0x7fdc140b15e0 2020-12-05T17:32:02.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.167+0000 7fdc117fa700 1 --2- 172.21.15.32:0/3422744668 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdbf4055330 0x7fdbf40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:02.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.167+0000 7fdc137fe700 1 --2- 172.21.15.32:0/3422744668 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdbf4055330 0x7fdbf40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:02.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.167+0000 7fdc117fa700 1 -- 172.21.15.32:0/3422744668 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fdc0406d8c0 con 0x7fdc140b15e0 2020-12-05T17:32:02.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.167+0000 7fdc1a6d8700 1 -- 172.21.15.32:0/3422744668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc14064ff0 con 0x7fdc140b15e0 2020-12-05T17:32:02.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.167+0000 7fdc137fe700 1 --2- 172.21.15.32:0/3422744668 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdbf4055330 0x7fdbf40577f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fdc0800d000 tx=0x7fdc08007a30).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:02.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.171+0000 7fdc117fa700 1 -- 172.21.15.32:0/3422744668 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fdc040438d0 con 0x7fdc140b15e0 2020-12-05T17:32:02.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.315+0000 7fdc1a6d8700 1 -- 172.21.15.32:0/3422744668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fdc14057050 con 0x7fdc140b15e0 2020-12-05T17:32:02.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.315+0000 7fdc117fa700 1 -- 172.21.15.32:0/3422744668 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7fdc0404b070 con 0x7fdc140b15e0 2020-12-05T17:32:02.319 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:32:02.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.315+0000 7fdc1a6d8700 1 -- 172.21.15.32:0/3422744668 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdbf4055330 msgr2=0x7fdbf40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:02.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.315+0000 7fdc1a6d8700 1 --2- 172.21.15.32:0/3422744668 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdbf4055330 0x7fdbf40577f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fdc0800d000 tx=0x7fdc08007a30).stop 2020-12-05T17:32:02.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.315+0000 7fdc1a6d8700 1 -- 172.21.15.32:0/3422744668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc140b15e0 msgr2=0x7fdc14118ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:02.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.315+0000 7fdc1a6d8700 1 --2- 172.21.15.32:0/3422744668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc140b15e0 0x7fdc14118ee0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fdc04001190 tx=0x7fdc04007b20).stop 2020-12-05T17:32:02.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.319+0000 7fdc1a6d8700 1 -- 172.21.15.32:0/3422744668 shutdown_connections 2020-12-05T17:32:02.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.319+0000 7fdc1a6d8700 1 --2- 172.21.15.32:0/3422744668 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc140b2a70 0x7fdc141447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.319+0000 7fdc1a6d8700 1 --2- 172.21.15.32:0/3422744668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc140b15e0 0x7fdc14118ee0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.319+0000 7fdc1a6d8700 1 --2- 172.21.15.32:0/3422744668 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdbf4055330 0x7fdbf40577f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.319+0000 7fdc1a6d8700 1 --2- 172.21.15.32:0/3422744668 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc140b2690 0x7fdc140b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.323 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.319+0000 7fdc1a6d8700 1 -- 172.21.15.32:0/3422744668 >> 172.21.15.32:0/3422744668 conn(0x7fdc14062310 msgr2=0x7fdc141498f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:02.323 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.319+0000 7fdc1a6d8700 1 -- 172.21.15.32:0/3422744668 shutdown_connections 2020-12-05T17:32:02.323 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.319+0000 7fdc1a6d8700 1 -- 172.21.15.32:0/3422744668 wait complete. 2020-12-05T17:32:02.333 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:32:02.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f51458c4700 1 Processor -- start 2020-12-05T17:32:02.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f51458c4700 1 -- start start 2020-12-05T17:32:02.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f51458c4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51400b56f0 0x7f5140143290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:02.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f51458c4700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f5140062b00 0x7f5140143780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:02.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f51458c4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f51400a8660 0x7f51400a8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:02.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f51458c4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51400a9f50 con 0x7f51400b56f0 2020-12-05T17:32:02.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f51458c4700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5140143c70 con 0x7f5140062b00 2020-12-05T17:32:02.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f51458c4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51400abda0 con 0x7f51400a8660 2020-12-05T17:32:02.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f513effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51400b56f0 0x7f5140143290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:02.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f513effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51400b56f0 0x7f5140143290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33182/0 (socket says 172.21.15.32:33182) 2020-12-05T17:32:02.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f513effd700 1 -- 172.21.15.32:0/2625779068 learned_addr learned my addr 172.21.15.32:0/2625779068 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:02.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f513e7fc700 1 --2- 172.21.15.32:0/2625779068 >> v2:172.21.15.61:3300/0 conn(0x7f5140062b00 0x7f5140143780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:02.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f513f7fe700 1 --2- 172.21.15.32:0/2625779068 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f51400a8660 0x7f51400a8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:02.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f513effd700 1 -- 172.21.15.32:0/2625779068 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f51400a8660 msgr2=0x7f51400a8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:02.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f513effd700 1 --2- 172.21.15.32:0/2625779068 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f51400a8660 0x7f51400a8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f513effd700 1 -- 172.21.15.32:0/2625779068 >> v2:172.21.15.61:3300/0 conn(0x7f5140062b00 msgr2=0x7f5140143780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:02.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f513effd700 1 --2- 172.21.15.32:0/2625779068 >> v2:172.21.15.61:3300/0 conn(0x7f5140062b00 0x7f5140143780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f513effd700 1 -- 172.21.15.32:0/2625779068 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51400b5ad0 con 0x7f51400b56f0 2020-12-05T17:32:02.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f513e7fc700 1 --2- 172.21.15.32:0/2625779068 >> v2:172.21.15.61:3300/0 conn(0x7f5140062b00 0x7f5140143780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:02.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f513f7fe700 1 --2- 172.21.15.32:0/2625779068 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f51400a8660 0x7f51400a8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:32:02.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f513effd700 1 --2- 172.21.15.32:0/2625779068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51400b56f0 0x7f5140143290 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f5140143db0 tx=0x7f5130006e80).ready entity=mon.0 client_cookie=2ca0b5962360f06f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:02.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f513dffb700 1 -- 172.21.15.32:0/2625779068 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51300097b0 con 0x7f51400b56f0 2020-12-05T17:32:02.461 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f513dffb700 1 -- 172.21.15.32:0/2625779068 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5130009910 con 0x7f51400b56f0 2020-12-05T17:32:02.461 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f513dffb700 1 -- 172.21.15.32:0/2625779068 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5130009a80 con 0x7f51400b56f0 2020-12-05T17:32:02.461 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f51458c4700 1 -- 172.21.15.32:0/2625779068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51400b56f0 msgr2=0x7f5140143290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:02.461 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.451+0000 7f51458c4700 1 --2- 172.21.15.32:0/2625779068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51400b56f0 0x7f5140143290 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f5140143db0 tx=0x7f5130006e80).stop 2020-12-05T17:32:02.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.455+0000 7f51458c4700 1 -- 172.21.15.32:0/2625779068 shutdown_connections 2020-12-05T17:32:02.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.455+0000 7f51458c4700 1 --2- 172.21.15.32:0/2625779068 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f51400a8660 0x7f51400a8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.455+0000 7f51458c4700 1 --2- 172.21.15.32:0/2625779068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51400b56f0 0x7f5140143290 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.455+0000 7f51458c4700 1 --2- 172.21.15.32:0/2625779068 >> v2:172.21.15.61:3300/0 conn(0x7f5140062b00 0x7f5140143780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.455+0000 7f51458c4700 1 -- 172.21.15.32:0/2625779068 >> 172.21.15.32:0/2625779068 conn(0x7f514005fe80 msgr2=0x7f51400b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.455+0000 7f51458c4700 1 -- 172.21.15.32:0/2625779068 shutdown_connections 2020-12-05T17:32:02.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.455+0000 7f51458c4700 1 -- 172.21.15.32:0/2625779068 wait complete. 2020-12-05T17:32:02.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.455+0000 7f51458c4700 1 Processor -- start 2020-12-05T17:32:02.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.455+0000 7f51458c4700 1 -- start start 2020-12-05T17:32:02.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.455+0000 7f51458c4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5140062b00 0x7f5140061d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:02.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.455+0000 7f51458c4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f51400a8660 0x7f5140065a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:02.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.455+0000 7f51458c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51400b56f0 0x7f514014f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:02.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.455+0000 7f51458c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51400a03b0 con 0x7f51400b56f0 2020-12-05T17:32:02.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.455+0000 7f51458c4700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51400652f0 con 0x7f5140062b00 2020-12-05T17:32:02.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.455+0000 7f51458c4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5140065590 con 0x7f51400a8660 2020-12-05T17:32:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.455+0000 7f513f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51400b56f0 0x7f514014f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.455+0000 7f513f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51400b56f0 0x7f514014f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33190/0 (socket says 172.21.15.32:33190) 2020-12-05T17:32:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.455+0000 7f513f7fe700 1 -- 172.21.15.32:0/237596631 learned_addr learned my addr 172.21.15.32:0/237596631 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.455+0000 7f513effd700 1 --2- 172.21.15.32:0/237596631 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5140062b00 0x7f5140061d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:02.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.455+0000 7f513e7fc700 1 --2- 172.21.15.32:0/237596631 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f51400a8660 0x7f5140065a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:02.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.459+0000 7f513f7fe700 1 -- 172.21.15.32:0/237596631 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f51400a8660 msgr2=0x7f5140065a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:02.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.459+0000 7f513f7fe700 1 --2- 172.21.15.32:0/237596631 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f51400a8660 0x7f5140065a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.459+0000 7f513f7fe700 1 -- 172.21.15.32:0/237596631 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5140062b00 msgr2=0x7f5140061d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:02.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.459+0000 7f513f7fe700 1 --2- 172.21.15.32:0/237596631 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5140062b00 0x7f5140061d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.459+0000 7f513f7fe700 1 -- 172.21.15.32:0/237596631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5140066b80 con 0x7f51400b56f0 2020-12-05T17:32:02.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.459+0000 7f513f7fe700 1 --2- 172.21.15.32:0/237596631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51400b56f0 0x7f514014f600 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f5138007000 tx=0x7f5138008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:02.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.459+0000 7f512ffff700 1 -- 172.21.15.32:0/237596631 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f513800a8f0 con 0x7f51400b56f0 2020-12-05T17:32:02.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.459+0000 7f512ffff700 1 -- 172.21.15.32:0/237596631 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513800aa50 con 0x7f51400b56f0 2020-12-05T17:32:02.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.459+0000 7f51458c4700 1 -- 172.21.15.32:0/237596631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5140065f20 con 0x7f51400b56f0 2020-12-05T17:32:02.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.459+0000 7f512ffff700 1 -- 172.21.15.32:0/237596631 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f513800adb0 con 0x7f51400b56f0 2020-12-05T17:32:02.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.459+0000 7f51458c4700 1 -- 172.21.15.32:0/237596631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514014fd10 con 0x7f51400b56f0 2020-12-05T17:32:02.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.459+0000 7f512ffff700 1 -- 172.21.15.32:0/237596631 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f513801b460 con 0x7f51400b56f0 2020-12-05T17:32:02.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.459+0000 7f51458c4700 1 -- 172.21.15.32:0/237596631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51400a81f0 con 0x7f51400b56f0 2020-12-05T17:32:02.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.459+0000 7f512ffff700 1 --2- 172.21.15.32:0/237596631 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f511c055440 0x7f511c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:02.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.459+0000 7f512ffff700 1 -- 172.21.15.32:0/237596631 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f513806e660 con 0x7f51400b56f0 2020-12-05T17:32:02.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.459+0000 7f513effd700 1 --2- 172.21.15.32:0/237596631 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f511c055440 0x7f511c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:02.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.459+0000 7f513effd700 1 --2- 172.21.15.32:0/237596631 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f511c055440 0x7f511c057900 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f513001a000 tx=0x7f5130016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.467+0000 7f512ffff700 1 -- 172.21.15.32:0/237596631 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f51400a81f0 con 0x7f51400b56f0 2020-12-05T17:32:02.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.611+0000 7f51458c4700 1 -- 172.21.15.32:0/237596631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f51400a81f0 con 0x7f51400b56f0 2020-12-05T17:32:02.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.611+0000 7f512ffff700 1 -- 172.21.15.32:0/237596631 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f51400a81f0 con 0x7f51400b56f0 2020-12-05T17:32:02.614 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:32:02.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.611+0000 7f51458c4700 1 -- 172.21.15.32:0/237596631 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f511c055440 msgr2=0x7f511c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:02.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.611+0000 7f51458c4700 1 --2- 172.21.15.32:0/237596631 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f511c055440 0x7f511c057900 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f513001a000 tx=0x7f5130016000).stop 2020-12-05T17:32:02.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.611+0000 7f51458c4700 1 -- 172.21.15.32:0/237596631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51400b56f0 msgr2=0x7f514014f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:02.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.611+0000 7f51458c4700 1 --2- 172.21.15.32:0/237596631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51400b56f0 0x7f514014f600 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f5138007000 tx=0x7f5138008040).stop 2020-12-05T17:32:02.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.611+0000 7f51458c4700 1 -- 172.21.15.32:0/237596631 shutdown_connections 2020-12-05T17:32:02.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.611+0000 7f51458c4700 1 --2- 172.21.15.32:0/237596631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51400b56f0 0x7f514014f600 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.611+0000 7f51458c4700 1 --2- 172.21.15.32:0/237596631 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5140062b00 0x7f5140061d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.611+0000 7f51458c4700 1 --2- 172.21.15.32:0/237596631 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f511c055440 0x7f511c057900 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.611+0000 7f51458c4700 1 --2- 172.21.15.32:0/237596631 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f51400a8660 0x7f5140065a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.611+0000 7f51458c4700 1 -- 172.21.15.32:0/237596631 >> 172.21.15.32:0/237596631 conn(0x7f514005fe80 msgr2=0x7f51400afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:02.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.611+0000 7f51458c4700 1 -- 172.21.15.32:0/237596631 shutdown_connections 2020-12-05T17:32:02.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.611+0000 7f51458c4700 1 -- 172.21.15.32:0/237596631 wait complete. 2020-12-05T17:32:02.628 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:32:02.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.743+0000 7fad8de97700 1 Processor -- start 2020-12-05T17:32:02.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.743+0000 7fad8de97700 1 -- start start 2020-12-05T17:32:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.743+0000 7fad8de97700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad88061780 0x7fad88146510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.743+0000 7fad8de97700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad880b23e0 con 0x7fad880af4b0 2020-12-05T17:32:02.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.743+0000 7fad8de97700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad880ae7b0 con 0x7fad880b2000 2020-12-05T17:32:02.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.743+0000 7fad8de97700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad88146a00 con 0x7fad88061780 2020-12-05T17:32:02.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.743+0000 7fad8d696700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad88061780 0x7fad88146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:02.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.743+0000 7fad8d696700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad88061780 0x7fad88146510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33200/0 (socket says 172.21.15.32:33200) 2020-12-05T17:32:02.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.743+0000 7fad8d696700 1 -- 172.21.15.32:0/54594530 learned_addr learned my addr 172.21.15.32:0/54594530 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:02.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.743+0000 7fad87fff700 1 --1- 172.21.15.32:0/54594530 >> v1:172.21.15.32:6789/0 conn(0x7fad880af4b0 0x7fad880afe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35846/0 (socket says 172.21.15.32:35846) 2020-12-05T17:32:02.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.743+0000 7fad8d696700 1 -- 172.21.15.32:0/54594530 >> v1:172.21.15.61:6789/0 conn(0x7fad880b2000 legacy=0x7fad88145e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.743+0000 7fad8d696700 1 -- 172.21.15.32:0/54594530 >> v1:172.21.15.32:6789/0 conn(0x7fad880af4b0 legacy=0x7fad880afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.743+0000 7fad8d696700 1 -- 172.21.15.32:0/54594530 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad880aea50 con 0x7fad88061780 2020-12-05T17:32:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.743+0000 7fad8d696700 1 --2- 172.21.15.32:0/54594530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad88061780 0x7fad88146510 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fad800032b0 tx=0x7fad80009270).ready entity=mon.0 client_cookie=ef7b1eefba6f4783 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.743+0000 7fad877fe700 1 -- 172.21.15.32:0/54594530 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad8000bc00 con 0x7fad88061780 2020-12-05T17:32:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.743+0000 7fad877fe700 1 -- 172.21.15.32:0/54594530 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8000bd60 con 0x7fad88061780 2020-12-05T17:32:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.743+0000 7fad877fe700 1 -- 172.21.15.32:0/54594530 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad80011400 con 0x7fad88061780 2020-12-05T17:32:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.743+0000 7fad8de97700 1 -- 172.21.15.32:0/54594530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad88061780 msgr2=0x7fad88146510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:02.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.743+0000 7fad8de97700 1 --2- 172.21.15.32:0/54594530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad88061780 0x7fad88146510 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fad800032b0 tx=0x7fad80009270).stop 2020-12-05T17:32:02.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8de97700 1 -- 172.21.15.32:0/54594530 shutdown_connections 2020-12-05T17:32:02.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8de97700 1 --2- 172.21.15.32:0/54594530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad88061780 0x7fad88146510 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8de97700 1 -- 172.21.15.32:0/54594530 >> 172.21.15.32:0/54594530 conn(0x7fad880b5620 msgr2=0x7fad880b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:02.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8de97700 1 -- 172.21.15.32:0/54594530 shutdown_connections 2020-12-05T17:32:02.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8de97700 1 -- 172.21.15.32:0/54594530 wait complete. 2020-12-05T17:32:02.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8de97700 1 Processor -- start 2020-12-05T17:32:02.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8de97700 1 -- start start 2020-12-05T17:32:02.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8de97700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad88061780 0x7fad8814a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:02.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8de97700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad880af4b0 0x7fad88145c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:02.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8de97700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fad880b2000 0x7fad881441c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:02.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8de97700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad88063ac0 con 0x7fad88061780 2020-12-05T17:32:02.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8de97700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad88148c00 con 0x7fad880b2000 2020-12-05T17:32:02.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8de97700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad88148ea0 con 0x7fad880af4b0 2020-12-05T17:32:02.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8ce95700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad88061780 0x7fad8814a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:02.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8ce95700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad88061780 0x7fad8814a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33202/0 (socket says 172.21.15.32:33202) 2020-12-05T17:32:02.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8ce95700 1 -- 172.21.15.32:0/3549167272 learned_addr learned my addr 172.21.15.32:0/3549167272 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:02.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad87fff700 1 --2- 172.21.15.32:0/3549167272 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad880af4b0 0x7fad88145c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:02.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8d696700 1 --2- 172.21.15.32:0/3549167272 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fad880b2000 0x7fad881441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:02.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8ce95700 1 -- 172.21.15.32:0/3549167272 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad880af4b0 msgr2=0x7fad88145c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:02.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8ce95700 1 --2- 172.21.15.32:0/3549167272 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad880af4b0 0x7fad88145c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8ce95700 1 -- 172.21.15.32:0/3549167272 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fad880b2000 msgr2=0x7fad881441c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:02.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8ce95700 1 --2- 172.21.15.32:0/3549167272 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fad880b2000 0x7fad881441c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8ce95700 1 -- 172.21.15.32:0/3549167272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad88149860 con 0x7fad88061780 2020-12-05T17:32:02.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad87fff700 1 --2- 172.21.15.32:0/3549167272 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad880af4b0 0x7fad88145c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:02.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8ce95700 1 --2- 172.21.15.32:0/3549167272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad88061780 0x7fad8814a320 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fad8805cef0 tx=0x7fad70007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:02.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad85ffb700 1 -- 172.21.15.32:0/3549167272 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad70007240 con 0x7fad88061780 2020-12-05T17:32:02.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad85ffb700 1 -- 172.21.15.32:0/3549167272 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad700073a0 con 0x7fad88061780 2020-12-05T17:32:02.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad85ffb700 1 -- 172.21.15.32:0/3549167272 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad700066d0 con 0x7fad88061780 2020-12-05T17:32:02.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.747+0000 7fad8de97700 1 -- 172.21.15.32:0/3549167272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad88149b10 con 0x7fad88061780 2020-12-05T17:32:02.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.751+0000 7fad8de97700 1 -- 172.21.15.32:0/3549167272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad881448d0 con 0x7fad88061780 2020-12-05T17:32:02.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.751+0000 7fad8de97700 1 -- 172.21.15.32:0/3549167272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad88149d00 con 0x7fad88061780 2020-12-05T17:32:02.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.751+0000 7fad85ffb700 1 -- 172.21.15.32:0/3549167272 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fad70010030 con 0x7fad88061780 2020-12-05T17:32:02.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.751+0000 7fad85ffb700 1 --2- 172.21.15.32:0/3549167272 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fad6c055330 0x7fad6c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:02.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.751+0000 7fad85ffb700 1 -- 172.21.15.32:0/3549167272 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fad7006dc80 con 0x7fad88061780 2020-12-05T17:32:02.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.751+0000 7fad87fff700 1 --2- 172.21.15.32:0/3549167272 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fad6c055330 0x7fad6c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:02.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.755+0000 7fad87fff700 1 --2- 172.21.15.32:0/3549167272 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fad6c055330 0x7fad6c0577f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fad7800b000 tx=0x7fad780070e0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:02.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.759+0000 7fad85ffb700 1 -- 172.21.15.32:0/3549167272 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fad70043e20 con 0x7fad88061780 2020-12-05T17:32:02.902 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.899+0000 7fad8de97700 1 -- 172.21.15.32:0/3549167272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fad88149060 con 0x7fad88061780 2020-12-05T17:32:02.903 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.899+0000 7fad85ffb700 1 -- 172.21.15.32:0/3549167272 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7fad70047440 con 0x7fad88061780 2020-12-05T17:32:02.903 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:32:02.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.899+0000 7fad8de97700 1 -- 172.21.15.32:0/3549167272 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fad6c055330 msgr2=0x7fad6c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:02.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.899+0000 7fad8de97700 1 --2- 172.21.15.32:0/3549167272 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fad6c055330 0x7fad6c0577f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fad7800b000 tx=0x7fad780070e0).stop 2020-12-05T17:32:02.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.903+0000 7fad8de97700 1 -- 172.21.15.32:0/3549167272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad88061780 msgr2=0x7fad8814a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:02.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.903+0000 7fad8de97700 1 --2- 172.21.15.32:0/3549167272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad88061780 0x7fad8814a320 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fad8805cef0 tx=0x7fad70007bd0).stop 2020-12-05T17:32:02.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.903+0000 7fad8de97700 1 -- 172.21.15.32:0/3549167272 shutdown_connections 2020-12-05T17:32:02.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.903+0000 7fad8de97700 1 --2- 172.21.15.32:0/3549167272 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fad880b2000 0x7fad881441c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.903+0000 7fad8de97700 1 --2- 172.21.15.32:0/3549167272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad88061780 0x7fad8814a320 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.903+0000 7fad8de97700 1 --2- 172.21.15.32:0/3549167272 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fad6c055330 0x7fad6c0577f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.903+0000 7fad8de97700 1 --2- 172.21.15.32:0/3549167272 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad880af4b0 0x7fad88145c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:02.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.903+0000 7fad8de97700 1 -- 172.21.15.32:0/3549167272 >> 172.21.15.32:0/3549167272 conn(0x7fad880b5620 msgr2=0x7fad880b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:02.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.903+0000 7fad8de97700 1 -- 172.21.15.32:0/3549167272 shutdown_connections 2020-12-05T17:32:02.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:02.903+0000 7fad8de97700 1 -- 172.21.15.32:0/3549167272 wait complete. 2020-12-05T17:32:02.920 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:32:03.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.031+0000 7f4cdb59e700 1 Processor -- start 2020-12-05T17:32:03.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.031+0000 7f4cdb59e700 1 -- start start 2020-12-05T17:32:03.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cdb59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4cdc1396c0 0x7f4cdc0af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:03.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cdb59e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f4cdc062ac0 0x7f4cdc13f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:03.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cdb59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cdc143390 con 0x7f4cdc0b1e60 2020-12-05T17:32:03.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cdb59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cdc0ae0a0 con 0x7f4cdc1396c0 2020-12-05T17:32:03.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cdb59e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cdc143610 con 0x7f4cdc062ac0 2020-12-05T17:32:03.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cd9d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4cdc1396c0 0x7f4cdc0af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:03.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cd9d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4cdc1396c0 0x7f4cdc0af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33212/0 (socket says 172.21.15.32:33212) 2020-12-05T17:32:03.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cd9d9b700 1 -- 172.21.15.32:0/4010018208 learned_addr learned my addr 172.21.15.32:0/4010018208 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:03.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cdad9d700 1 --2- 172.21.15.32:0/4010018208 >> v2:172.21.15.61:3300/0 conn(0x7f4cdc062ac0 0x7f4cdc13f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:03.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cd9d9b700 1 -- 172.21.15.32:0/4010018208 >> v2:172.21.15.61:3300/0 conn(0x7f4cdc062ac0 msgr2=0x7f4cdc13f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:03.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cd9d9b700 1 --2- 172.21.15.32:0/4010018208 >> v2:172.21.15.61:3300/0 conn(0x7f4cdc062ac0 0x7f4cdc13f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:03.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cd9d9b700 1 -- 172.21.15.32:0/4010018208 >> v1:172.21.15.61:6789/0 conn(0x7f4cdc0b1e60 legacy=0x7f4cdc138fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:03.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cd9d9b700 1 -- 172.21.15.32:0/4010018208 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cdc1438b0 con 0x7f4cdc1396c0 2020-12-05T17:32:03.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cd9d9b700 1 --2- 172.21.15.32:0/4010018208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cdc1396c0 0x7f4cdc0af4b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f4cc4002f30 tx=0x7f4cc4006e80).ready entity=mon.0 client_cookie=4367fd00b287af62 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:03.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cd959a700 1 -- 172.21.15.32:0/4010018208 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4cc40097f0 con 0x7f4cdc1396c0 2020-12-05T17:32:03.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cd959a700 1 -- 172.21.15.32:0/4010018208 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc4009950 con 0x7f4cdc1396c0 2020-12-05T17:32:03.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cd959a700 1 -- 172.21.15.32:0/4010018208 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4cc4009cb0 con 0x7f4cdc1396c0 2020-12-05T17:32:03.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cdb59e700 1 -- 172.21.15.32:0/4010018208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cdc1396c0 msgr2=0x7f4cdc0af4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:03.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cdb59e700 1 --2- 172.21.15.32:0/4010018208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cdc1396c0 0x7f4cdc0af4b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f4cc4002f30 tx=0x7f4cc4006e80).stop 2020-12-05T17:32:03.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cdb59e700 1 -- 172.21.15.32:0/4010018208 shutdown_connections 2020-12-05T17:32:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cdb59e700 1 --2- 172.21.15.32:0/4010018208 >> v2:172.21.15.61:3300/0 conn(0x7f4cdc062ac0 0x7f4cdc13f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cdb59e700 1 --2- 172.21.15.32:0/4010018208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cdc1396c0 0x7f4cdc0af4b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cdb59e700 1 -- 172.21.15.32:0/4010018208 >> 172.21.15.32:0/4010018208 conn(0x7f4cdc0b5620 msgr2=0x7f4cdc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cdb59e700 1 -- 172.21.15.32:0/4010018208 shutdown_connections 2020-12-05T17:32:03.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.035+0000 7f4cdb59e700 1 -- 172.21.15.32:0/4010018208 wait complete. 2020-12-05T17:32:03.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cdb59e700 1 Processor -- start 2020-12-05T17:32:03.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cdb59e700 1 -- start start 2020-12-05T17:32:03.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cdb59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cdc062ac0 0x7f4cdc061ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:03.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cdb59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4cdc0b1e60 0x7f4cdc13bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:03.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cdb59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4cdc1396c0 0x7f4cdc13bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:03.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cdb59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc065cb0 con 0x7f4cdc062ac0 2020-12-05T17:32:03.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cdb59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc065a10 con 0x7f4cdc0b1e60 2020-12-05T17:32:03.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cdb59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc066550 con 0x7f4cdc1396c0 2020-12-05T17:32:03.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cda59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cdc062ac0 0x7f4cdc061ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:03.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cda59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cdc062ac0 0x7f4cdc061ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33216/0 (socket says 172.21.15.32:33216) 2020-12-05T17:32:03.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cda59c700 1 -- 172.21.15.32:0/1974153198 learned_addr learned my addr 172.21.15.32:0/1974153198 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:03.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cd9d9b700 1 --2- 172.21.15.32:0/1974153198 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4cdc0b1e60 0x7f4cdc13bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:03.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cdad9d700 1 --2- 172.21.15.32:0/1974153198 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4cdc1396c0 0x7f4cdc13bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:03.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cda59c700 1 -- 172.21.15.32:0/1974153198 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4cdc1396c0 msgr2=0x7f4cdc13bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:03.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cda59c700 1 --2- 172.21.15.32:0/1974153198 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4cdc1396c0 0x7f4cdc13bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:03.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cda59c700 1 -- 172.21.15.32:0/1974153198 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4cdc0b1e60 msgr2=0x7f4cdc13bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:03.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cda59c700 1 --2- 172.21.15.32:0/1974153198 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4cdc0b1e60 0x7f4cdc13bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:03.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cda59c700 1 -- 172.21.15.32:0/1974153198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cdc13c4b0 con 0x7f4cdc062ac0 2020-12-05T17:32:03.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cda59c700 1 --2- 172.21.15.32:0/1974153198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cdc062ac0 0x7f4cdc061ba0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f4cdc13f890 tx=0x7f4ccc007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:03.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cd37fe700 1 -- 172.21.15.32:0/1974153198 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ccc013410 con 0x7f4cdc062ac0 2020-12-05T17:32:03.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cd37fe700 1 -- 172.21.15.32:0/1974153198 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ccc013570 con 0x7f4cdc062ac0 2020-12-05T17:32:03.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cdb59e700 1 -- 172.21.15.32:0/1974153198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cdc13a650 con 0x7f4cdc062ac0 2020-12-05T17:32:03.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cd37fe700 1 -- 172.21.15.32:0/1974153198 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ccc0138d0 con 0x7f4cdc062ac0 2020-12-05T17:32:03.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.039+0000 7f4cdb59e700 1 -- 172.21.15.32:0/1974153198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cdc065610 con 0x7f4cdc062ac0 2020-12-05T17:32:03.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.043+0000 7f4cdb59e700 1 -- 172.21.15.32:0/1974153198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cdc057050 con 0x7f4cdc062ac0 2020-12-05T17:32:03.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.043+0000 7f4cd37fe700 1 -- 172.21.15.32:0/1974153198 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4ccc010030 con 0x7f4cdc062ac0 2020-12-05T17:32:03.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.043+0000 7f4cd37fe700 1 --2- 172.21.15.32:0/1974153198 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4cc0055440 0x7f4cc0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:03.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.043+0000 7f4cd37fe700 1 -- 172.21.15.32:0/1974153198 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f4ccc06dfe0 con 0x7f4cdc062ac0 2020-12-05T17:32:03.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.043+0000 7f4cd9d9b700 1 --2- 172.21.15.32:0/1974153198 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4cc0055440 0x7f4cc0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:03.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.043+0000 7f4cd9d9b700 1 --2- 172.21.15.32:0/1974153198 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4cc0055440 0x7f4cc0057900 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f4cc401c000 tx=0x7f4cc4015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:03.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.047+0000 7f4cd37fe700 1 -- 172.21.15.32:0/1974153198 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4ccc043ff0 con 0x7f4cdc062ac0 2020-12-05T17:32:03.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.191+0000 7f4cdb59e700 1 -- 172.21.15.32:0/1974153198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4cdc05aab0 con 0x7f4cdc062ac0 2020-12-05T17:32:03.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.191+0000 7f4cd37fe700 1 -- 172.21.15.32:0/1974153198 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7f4ccc047610 con 0x7f4cdc062ac0 2020-12-05T17:32:03.195 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:32:03.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.191+0000 7f4cdb59e700 1 -- 172.21.15.32:0/1974153198 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4cc0055440 msgr2=0x7f4cc0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:03.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.191+0000 7f4cdb59e700 1 --2- 172.21.15.32:0/1974153198 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4cc0055440 0x7f4cc0057900 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f4cc401c000 tx=0x7f4cc4015000).stop 2020-12-05T17:32:03.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.191+0000 7f4cdb59e700 1 -- 172.21.15.32:0/1974153198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cdc062ac0 msgr2=0x7f4cdc061ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:03.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.191+0000 7f4cdb59e700 1 --2- 172.21.15.32:0/1974153198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cdc062ac0 0x7f4cdc061ba0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f4cdc13f890 tx=0x7f4ccc007c20).stop 2020-12-05T17:32:03.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.191+0000 7f4cdb59e700 1 -- 172.21.15.32:0/1974153198 shutdown_connections 2020-12-05T17:32:03.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.191+0000 7f4cdb59e700 1 --2- 172.21.15.32:0/1974153198 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4cc0055440 0x7f4cc0057900 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:03.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.191+0000 7f4cdb59e700 1 --2- 172.21.15.32:0/1974153198 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4cdc1396c0 0x7f4cdc13bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:03.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.191+0000 7f4cdb59e700 1 --2- 172.21.15.32:0/1974153198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cdc062ac0 0x7f4cdc061ba0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.191+0000 7f4cdb59e700 1 --2- 172.21.15.32:0/1974153198 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4cdc0b1e60 0x7f4cdc13bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.191+0000 7f4cdb59e700 1 -- 172.21.15.32:0/1974153198 >> 172.21.15.32:0/1974153198 conn(0x7f4cdc0b5620 msgr2=0x7f4cdc0b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.195+0000 7f4cdb59e700 1 -- 172.21.15.32:0/1974153198 shutdown_connections 2020-12-05T17:32:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.195+0000 7f4cdb59e700 1 -- 172.21.15.32:0/1974153198 wait complete. 2020-12-05T17:32:03.208 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:32:03.208 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:32:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.319+0000 7fc9f7fff700 1 Processor -- start 2020-12-05T17:32:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.319+0000 7fc9f7fff700 1 -- start start 2020-12-05T17:32:03.324 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.319+0000 7fc9f7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc9f805f3b0 0x7fc9f805f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:03.324 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.319+0000 7fc9f7fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc9f8062400 0x7fc9f80b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:03.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.319+0000 7fc9f7fff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f8142450 con 0x7fc9f805feb0 2020-12-05T17:32:03.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.319+0000 7fc9f7fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f80b42f0 con 0x7fc9f805f3b0 2020-12-05T17:32:03.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.319+0000 7fc9f7fff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f8061780 con 0x7fc9f8062400 2020-12-05T17:32:03.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.319+0000 7fc9f6ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc9f805f3b0 0x7fc9f805f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.319+0000 7fc9f6ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc9f805f3b0 0x7fc9f805f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33224/0 (socket says 172.21.15.32:33224) 2020-12-05T17:32:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.319+0000 7fc9f6ffd700 1 -- 172.21.15.32:0/450659105 learned_addr learned my addr 172.21.15.32:0/450659105 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.319+0000 7fc9f67fc700 1 --2- 172.21.15.32:0/450659105 >> v2:172.21.15.61:3300/0 conn(0x7fc9f8062400 0x7fc9f80b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.319+0000 7fc9f6ffd700 1 -- 172.21.15.32:0/450659105 >> v2:172.21.15.61:3300/0 conn(0x7fc9f8062400 msgr2=0x7fc9f80b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:03.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.319+0000 7fc9f6ffd700 1 --2- 172.21.15.32:0/450659105 >> v2:172.21.15.61:3300/0 conn(0x7fc9f8062400 0x7fc9f80b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:03.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.319+0000 7fc9f6ffd700 1 -- 172.21.15.32:0/450659105 >> v1:172.21.15.61:6789/0 conn(0x7fc9f805feb0 legacy=0x7fc9f80b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:03.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.319+0000 7fc9f5ffb700 1 -- 172.21.15.32:0/450659105 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2666288222 0 0) 0x7fc9f8142450 con 0x7fc9f805feb0 2020-12-05T17:32:03.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.319+0000 7fc9f6ffd700 1 -- 172.21.15.32:0/450659105 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f8061a20 con 0x7fc9f805f3b0 2020-12-05T17:32:03.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.323+0000 7fc9f6ffd700 1 --2- 172.21.15.32:0/450659105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9f805f3b0 0x7fc9f805f790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fc9e0002f30 tx=0x7fc9e0006e80).ready entity=mon.0 client_cookie=709a28056f0c1dd6 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:03.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.323+0000 7fc9f5ffb700 1 -- 172.21.15.32:0/450659105 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9e00097f0 con 0x7fc9f805f3b0 2020-12-05T17:32:03.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.323+0000 7fc9f5ffb700 1 -- 172.21.15.32:0/450659105 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e0009950 con 0x7fc9f805f3b0 2020-12-05T17:32:03.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.323+0000 7fc9f5ffb700 1 -- 172.21.15.32:0/450659105 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9e0009cb0 con 0x7fc9f805f3b0 2020-12-05T17:32:03.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.323+0000 7fc9f7fff700 1 -- 172.21.15.32:0/450659105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9f805f3b0 msgr2=0x7fc9f805f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:03.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.323+0000 7fc9f7fff700 1 --2- 172.21.15.32:0/450659105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9f805f3b0 0x7fc9f805f790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fc9e0002f30 tx=0x7fc9e0006e80).stop 2020-12-05T17:32:03.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.323+0000 7fc9f7fff700 1 -- 172.21.15.32:0/450659105 shutdown_connections 2020-12-05T17:32:03.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.323+0000 7fc9f7fff700 1 --2- 172.21.15.32:0/450659105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9f805f3b0 0x7fc9f805f790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:03.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.323+0000 7fc9f7fff700 1 --2- 172.21.15.32:0/450659105 >> v2:172.21.15.61:3300/0 conn(0x7fc9f8062400 0x7fc9f80b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:03.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.323+0000 7fc9f7fff700 1 -- 172.21.15.32:0/450659105 >> 172.21.15.32:0/450659105 conn(0x7fc9f80b8ee0 msgr2=0x7fc9f80b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:03.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.323+0000 7fc9f7fff700 1 -- 172.21.15.32:0/450659105 shutdown_connections 2020-12-05T17:32:03.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.323+0000 7fc9f7fff700 1 -- 172.21.15.32:0/450659105 wait complete. 2020-12-05T17:32:03.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9f7fff700 1 Processor -- start 2020-12-05T17:32:03.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9f7fff700 1 -- start start 2020-12-05T17:32:03.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9f7fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc9f805f3b0 0x7fc9f8151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:03.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9f7fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9f805feb0 0x7fc9f8144930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:03.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9f7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9f8062400 0x7fc9f8145470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:03.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9f7fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f8066f20 con 0x7fc9f8062400 2020-12-05T17:32:03.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9f7fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f8151920 con 0x7fc9f805feb0 2020-12-05T17:32:03.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9f7fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f8151bc0 con 0x7fc9f805f3b0 2020-12-05T17:32:03.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9f77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9f8062400 0x7fc9f8145470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:03.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9f77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9f8062400 0x7fc9f8145470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33232/0 (socket says 172.21.15.32:33232) 2020-12-05T17:32:03.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9f77fe700 1 -- 172.21.15.32:0/1551427378 learned_addr learned my addr 172.21.15.32:0/1551427378 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:03.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9f6ffd700 1 --2- 172.21.15.32:0/1551427378 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc9f805f3b0 0x7fc9f8151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:03.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9f67fc700 1 --2- 172.21.15.32:0/1551427378 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9f805feb0 0x7fc9f8144930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:03.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9f77fe700 1 -- 172.21.15.32:0/1551427378 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc9f805f3b0 msgr2=0x7fc9f8151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:03.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9f77fe700 1 --2- 172.21.15.32:0/1551427378 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc9f805f3b0 0x7fc9f8151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:03.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9f77fe700 1 -- 172.21.15.32:0/1551427378 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9f805feb0 msgr2=0x7fc9f8144930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:03.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9f77fe700 1 --2- 172.21.15.32:0/1551427378 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9f805feb0 0x7fc9f8144930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:03.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9f77fe700 1 -- 172.21.15.32:0/1551427378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f8151e60 con 0x7fc9f8062400 2020-12-05T17:32:03.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9f6ffd700 1 --2- 172.21.15.32:0/1551427378 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc9f805f3b0 0x7fc9f8151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:03.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9f77fe700 1 --2- 172.21.15.32:0/1551427378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9f8062400 0x7fc9f8145470 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fc9f0000f00 tx=0x7fc9f0007da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:03.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9effff700 1 -- 172.21.15.32:0/1551427378 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9f000a880 con 0x7fc9f8062400 2020-12-05T17:32:03.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9effff700 1 -- 172.21.15.32:0/1551427378 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f000a9e0 con 0x7fc9f8062400 2020-12-05T17:32:03.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9f7fff700 1 -- 172.21.15.32:0/1551427378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f8063120 con 0x7fc9f8062400 2020-12-05T17:32:03.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9effff700 1 -- 172.21.15.32:0/1551427378 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9f000ad40 con 0x7fc9f8062400 2020-12-05T17:32:03.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9f7fff700 1 -- 172.21.15.32:0/1551427378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f8152520 con 0x7fc9f8062400 2020-12-05T17:32:03.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.327+0000 7fc9f7fff700 1 -- 172.21.15.32:0/1551427378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f8057050 con 0x7fc9f8062400 2020-12-05T17:32:03.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.331+0000 7fc9effff700 1 -- 172.21.15.32:0/1551427378 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc9f0019030 con 0x7fc9f8062400 2020-12-05T17:32:03.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.331+0000 7fc9effff700 1 --2- 172.21.15.32:0/1551427378 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc9dc055440 0x7fc9dc057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:03.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.331+0000 7fc9effff700 1 -- 172.21.15.32:0/1551427378 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc9f006da30 con 0x7fc9f8062400 2020-12-05T17:32:03.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.331+0000 7fc9f6ffd700 1 --2- 172.21.15.32:0/1551427378 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc9dc055440 0x7fc9dc057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:03.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.331+0000 7fc9f6ffd700 1 --2- 172.21.15.32:0/1551427378 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc9dc055440 0x7fc9dc057900 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fc9e001a000 tx=0x7fc9e0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:03.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.335+0000 7fc9effff700 1 -- 172.21.15.32:0/1551427378 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc9f0041df0 con 0x7fc9f8062400 2020-12-05T17:32:03.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.527+0000 7fc9f7fff700 1 -- 172.21.15.32:0/1551427378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9f805aab0 con 0x7fc9f8062400 2020-12-05T17:32:03.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.527+0000 7fc9effff700 1 -- 172.21.15.32:0/1551427378 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2134 (secure 0 0 0) 0x7fc9f0043c80 con 0x7fc9f8062400 2020-12-05T17:32:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:32:03.530 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4023797731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4023797731},{"type":"v1","addr":"172.21.15.61:6833","nonce":4023797731}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:31:41.611214+0000","modified":"2020-12-05T17:31:42.614053+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3503848021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3503848021},{"type":"v1","addr":"172.21.15.32:6835","nonce":3503848021}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-12-05T17:32:03.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.527+0000 7fc9f7fff700 1 -- 172.21.15.32:0/1551427378 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc9dc055440 msgr2=0x7fc9dc057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:03.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.527+0000 7fc9f7fff700 1 --2- 172.21.15.32:0/1551427378 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc9dc055440 0x7fc9dc057900 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fc9e001a000 tx=0x7fc9e0015000).stop 2020-12-05T17:32:03.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.527+0000 7fc9f7fff700 1 -- 172.21.15.32:0/1551427378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9f8062400 msgr2=0x7fc9f8145470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:03.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.527+0000 7fc9f7fff700 1 --2- 172.21.15.32:0/1551427378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9f8062400 0x7fc9f8145470 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fc9f0000f00 tx=0x7fc9f0007da0).stop 2020-12-05T17:32:03.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.527+0000 7fc9f7fff700 1 -- 172.21.15.32:0/1551427378 shutdown_connections 2020-12-05T17:32:03.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.527+0000 7fc9f7fff700 1 --2- 172.21.15.32:0/1551427378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9f8062400 0x7fc9f8145470 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:03.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.527+0000 7fc9f7fff700 1 --2- 172.21.15.32:0/1551427378 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc9dc055440 0x7fc9dc057900 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:03.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.527+0000 7fc9f7fff700 1 --2- 172.21.15.32:0/1551427378 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc9f805f3b0 0x7fc9f8151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:03.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.527+0000 7fc9f7fff700 1 --2- 172.21.15.32:0/1551427378 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9f805feb0 0x7fc9f8144930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:03.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.527+0000 7fc9f7fff700 1 -- 172.21.15.32:0/1551427378 >> 172.21.15.32:0/1551427378 conn(0x7fc9f80b8ee0 msgr2=0x7fc9f80b92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:03.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.527+0000 7fc9f7fff700 1 -- 172.21.15.32:0/1551427378 shutdown_connections 2020-12-05T17:32:03.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:03.527+0000 7fc9f7fff700 1 -- 172.21.15.32:0/1551427378 wait complete. 2020-12-05T17:32:03.536 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 5 2020-12-05T17:32:03.745 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:31:41.611214+0000', 'modified': '2020-12-05T17:31:42.614053+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4275}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4275': {'gid': 4275, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/3503848021', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 3503848021}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 3503848021}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:32:03.746 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:32:03.746 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:32:03.877 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":4275,"want_state":"up:active","state":"up:active","rank_uptime":22.253818617,"mdsmap_epoch":5,"osdmap_epoch":25,"osdmap_epoch_barrier":25,"uptime":28.659802064000001} 2020-12-05T17:32:03.877 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 4275, "mdsmap_epoch": 5, "osdmap_epoch": 25, "osdmap_epoch_barrier": 25, "rank_uptime": 22.253818617, "state": "up:active", "uptime": 28.659802064, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:32:03.878 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:32:03.878 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2020-12-05T17:32:03.879 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.2 at ubuntu@smithi053.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.2... 2020-12-05T17:32:03.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:03.879 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:32:03.887 INFO:teuthology.orchestra.run.smithi053:> sudo modprobe fuse 2020-12-05T17:32:03.946 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:32:03.946 INFO:teuthology.orchestra.run.smithi053:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:32:04.003 INFO:teuthology.orchestra.run.smithi053.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:32:04.004 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:32:04.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:04.005 INFO:teuthology.orchestra.run.smithi053:> ls /sys/fs/fuse/connections 2020-12-05T17:32:04.050 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:32:04.051 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.2 /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:32:04.094 INFO:teuthology.orchestra.run.smithi053:> sudo modprobe fuse 2020-12-05T17:32:04.109 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:32:04.109 INFO:teuthology.orchestra.run.smithi053:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:32:04.167 INFO:teuthology.orchestra.run.smithi053.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:32:04.168 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:32:04.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:04.169 INFO:teuthology.orchestra.run.smithi053:> ls /sys/fs/fuse/connections 2020-12-05T17:32:04.176 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi053.stderr:2020-12-05T17:32:04.172+0000 7ff3818d30c0 -1 init, newargv = 0x55a5a228f490 newargc=9ceph-fuse[10341]: starting ceph client 2020-12-05T17:32:04.176 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi053.stderr: 2020-12-05T17:32:04.186 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi053.stderr:ceph-fuse[10341]: starting fuse 2020-12-05T17:32:05.285 INFO:teuthology.orchestra.run.smithi053:> sudo modprobe fuse 2020-12-05T17:32:05.301 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:32:05.301 INFO:teuthology.orchestra.run.smithi053:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:32:05.359 INFO:teuthology.orchestra.run.smithi053.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:32:05.360 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:32:05.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:05.360 INFO:teuthology.orchestra.run.smithi053:> ls /sys/fs/fuse/connections 2020-12-05T17:32:05.410 INFO:teuthology.orchestra.run.smithi053.stdout:50 2020-12-05T17:32:05.411 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [50] 2020-12-05T17:32:05.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:05.412 INFO:teuthology.orchestra.run.smithi053:> sudo python3 -c ' 2020-12-05T17:32:05.413 INFO:teuthology.orchestra.run.smithi053:> import glob 2020-12-05T17:32:05.413 INFO:teuthology.orchestra.run.smithi053:> import re 2020-12-05T17:32:05.414 INFO:teuthology.orchestra.run.smithi053:> import os 2020-12-05T17:32:05.414 INFO:teuthology.orchestra.run.smithi053:> import subprocess 2020-12-05T17:32:05.415 INFO:teuthology.orchestra.run.smithi053:> 2020-12-05T17:32:05.415 INFO:teuthology.orchestra.run.smithi053:> def find_socket(client_name): 2020-12-05T17:32:05.415 INFO:teuthology.orchestra.run.smithi053:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2020-12-05T17:32:05.416 INFO:teuthology.orchestra.run.smithi053:> files = glob.glob(asok_path) 2020-12-05T17:32:05.416 INFO:teuthology.orchestra.run.smithi053:> 2020-12-05T17:32:05.416 INFO:teuthology.orchestra.run.smithi053:> # Given a non-glob path, it better be there 2020-12-05T17:32:05.417 INFO:teuthology.orchestra.run.smithi053:> if "*" not in asok_path: 2020-12-05T17:32:05.417 INFO:teuthology.orchestra.run.smithi053:> assert(len(files) == 1) 2020-12-05T17:32:05.418 INFO:teuthology.orchestra.run.smithi053:> return files[0] 2020-12-05T17:32:05.418 INFO:teuthology.orchestra.run.smithi053:> 2020-12-05T17:32:05.418 INFO:teuthology.orchestra.run.smithi053:> for f in files: 2020-12-05T17:32:05.419 INFO:teuthology.orchestra.run.smithi053:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:32:05.419 INFO:teuthology.orchestra.run.smithi053:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:32:05.420 INFO:teuthology.orchestra.run.smithi053:> return f 2020-12-05T17:32:05.420 INFO:teuthology.orchestra.run.smithi053:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:32:05.420 INFO:teuthology.orchestra.run.smithi053:> 2020-12-05T17:32:05.421 INFO:teuthology.orchestra.run.smithi053:> print(find_socket("client.2")) 2020-12-05T17:32:05.421 INFO:teuthology.orchestra.run.smithi053:> ' 2020-12-05T17:32:05.502 INFO:teuthology.orchestra.run.smithi053.stdout:/var/run/ceph/ceph-client.2.10341.asok 2020-12-05T17:32:05.508 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.10341.asok 2020-12-05T17:32:05.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:05.509 INFO:teuthology.orchestra.run.smithi053:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.10341.asok status 2020-12-05T17:32:05.714 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2020-12-05T17:32:05.714 INFO:teuthology.orchestra.run.smithi053.stdout: "metadata": { 2020-12-05T17:32:05.715 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:32:05.715 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:32:05.715 INFO:teuthology.orchestra.run.smithi053.stdout: "entity_id": "2", 2020-12-05T17:32:05.715 INFO:teuthology.orchestra.run.smithi053.stdout: "hostname": "smithi053", 2020-12-05T17:32:05.715 INFO:teuthology.orchestra.run.smithi053.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2020-12-05T17:32:05.716 INFO:teuthology.orchestra.run.smithi053.stdout: "pid": "10341", 2020-12-05T17:32:05.716 INFO:teuthology.orchestra.run.smithi053.stdout: "root": "/" 2020-12-05T17:32:05.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2020-12-05T17:32:05.716 INFO:teuthology.orchestra.run.smithi053.stdout: "dentry_count": 0, 2020-12-05T17:32:05.716 INFO:teuthology.orchestra.run.smithi053.stdout: "dentry_pinned_count": 0, 2020-12-05T17:32:05.717 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 4532, 2020-12-05T17:32:05.717 INFO:teuthology.orchestra.run.smithi053.stdout: "inst": { 2020-12-05T17:32:05.717 INFO:teuthology.orchestra.run.smithi053.stdout: "name": { 2020-12-05T17:32:05.717 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "client", 2020-12-05T17:32:05.718 INFO:teuthology.orchestra.run.smithi053.stdout: "num": 4532 2020-12-05T17:32:05.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2020-12-05T17:32:05.718 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": { 2020-12-05T17:32:05.718 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2020-12-05T17:32:05.718 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:0", 2020-12-05T17:32:05.719 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3065605959 2020-12-05T17:32:05.719 INFO:teuthology.orchestra.run.smithi053.stdout: } 2020-12-05T17:32:05.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2020-12-05T17:32:05.719 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": { 2020-12-05T17:32:05.719 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2020-12-05T17:32:05.720 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:0", 2020-12-05T17:32:05.720 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3065605959 2020-12-05T17:32:05.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2020-12-05T17:32:05.720 INFO:teuthology.orchestra.run.smithi053.stdout: "inst_str": "client.4532 172.21.15.53:0/3065605959", 2020-12-05T17:32:05.720 INFO:teuthology.orchestra.run.smithi053.stdout: "addr_str": "172.21.15.53:0/3065605959", 2020-12-05T17:32:05.721 INFO:teuthology.orchestra.run.smithi053.stdout: "inode_count": 1, 2020-12-05T17:32:05.721 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_epoch": 5, 2020-12-05T17:32:05.721 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_epoch": 25, 2020-12-05T17:32:05.721 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:32:05.721 INFO:teuthology.orchestra.run.smithi053.stdout: "blacklisted": false 2020-12-05T17:32:05.722 INFO:teuthology.orchestra.run.smithi053.stdout:} 2020-12-05T17:32:05.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:05.724 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:32:05.725 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:32:05.732 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:32:05.733 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:32:05.734 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:05.734 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:32:05.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:05.796 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:32:05.797 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:32:05.805 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:32:05.806 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:32:05.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:05.807 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:32:05.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:05.874 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:32:05.874 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:32:05.882 INFO:teuthology.orchestra.run.smithi053.stdout:fuseblk 2020-12-05T17:32:05.883 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2020-12-05T17:32:05.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:05.884 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:32:05.949 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2020-12-05T17:32:05.970 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:32:06.094 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.091+0000 7fbddffff700 1 Processor -- start 2020-12-05T17:32:06.094 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.091+0000 7fbddffff700 1 -- start start 2020-12-05T17:32:06.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.091+0000 7fbddffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbde00aed90 0x7fbde00af170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:06.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.091+0000 7fbddffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbde01441e0 0x7fbde0144690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:06.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.091+0000 7fbddffff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde0143b10 con 0x7fbde00b2140 2020-12-05T17:32:06.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.091+0000 7fbddffff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde013df90 con 0x7fbde01441e0 2020-12-05T17:32:06.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.091+0000 7fbddffff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde00b1c30 con 0x7fbde00aed90 2020-12-05T17:32:06.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.091+0000 7fbddf7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbde01441e0 0x7fbde0144690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:06.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.091+0000 7fbdde7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbde00b2140 0x7fbde01404a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35888/0 (socket says 172.21.15.32:35888) 2020-12-05T17:32:06.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.091+0000 7fbdde7fc700 1 -- 172.21.15.32:0/3017475141 learned_addr learned my addr 172.21.15.32:0/3017475141 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:06.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.091+0000 7fbddeffd700 1 --2- 172.21.15.32:0/3017475141 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbde00aed90 0x7fbde00af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:06.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.091+0000 7fbddf7fe700 1 -- 172.21.15.32:0/3017475141 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbde00aed90 msgr2=0x7fbde00af170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:06.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.091+0000 7fbddf7fe700 1 --2- 172.21.15.32:0/3017475141 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbde00aed90 0x7fbde00af170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.091+0000 7fbddf7fe700 1 -- 172.21.15.32:0/3017475141 >> v1:172.21.15.32:6789/0 conn(0x7fbde00b2140 legacy=0x7fbde01404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:06.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.091+0000 7fbdddffb700 1 -- 172.21.15.32:0/3017475141 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3188555500 0 0) 0x7fbde0143b10 con 0x7fbde00b2140 2020-12-05T17:32:06.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.091+0000 7fbddf7fe700 1 -- 172.21.15.32:0/3017475141 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde00b2650 con 0x7fbde01441e0 2020-12-05T17:32:06.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.091+0000 7fbddf7fe700 1 --2- 172.21.15.32:0/3017475141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbde01441e0 0x7fbde0144690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fbde00679d0 tx=0x7fbdd8006dc0).ready entity=mon.0 client_cookie=1c0c3966dce8ab32 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:06.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.095+0000 7fbdddffb700 1 -- 172.21.15.32:0/3017475141 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdd8009720 con 0x7fbde01441e0 2020-12-05T17:32:06.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.095+0000 7fbdddffb700 1 -- 172.21.15.32:0/3017475141 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd8009880 con 0x7fbde01441e0 2020-12-05T17:32:06.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.095+0000 7fbdddffb700 1 -- 172.21.15.32:0/3017475141 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdd8009be0 con 0x7fbde01441e0 2020-12-05T17:32:06.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.095+0000 7fbddffff700 1 -- 172.21.15.32:0/3017475141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbde01441e0 msgr2=0x7fbde0144690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:06.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.095+0000 7fbddffff700 1 --2- 172.21.15.32:0/3017475141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbde01441e0 0x7fbde0144690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fbde00679d0 tx=0x7fbdd8006dc0).stop 2020-12-05T17:32:06.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.095+0000 7fbddffff700 1 -- 172.21.15.32:0/3017475141 shutdown_connections 2020-12-05T17:32:06.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.095+0000 7fbddffff700 1 --2- 172.21.15.32:0/3017475141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbde01441e0 0x7fbde0144690 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.095+0000 7fbddffff700 1 --2- 172.21.15.32:0/3017475141 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbde00aed90 0x7fbde00af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.095+0000 7fbddffff700 1 -- 172.21.15.32:0/3017475141 >> 172.21.15.32:0/3017475141 conn(0x7fbde005bbb0 msgr2=0x7fbde0062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:06.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.095+0000 7fbddffff700 1 -- 172.21.15.32:0/3017475141 shutdown_connections 2020-12-05T17:32:06.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.095+0000 7fbddffff700 1 -- 172.21.15.32:0/3017475141 wait complete. 2020-12-05T17:32:06.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.095+0000 7fbddffff700 1 Processor -- start 2020-12-05T17:32:06.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.095+0000 7fbddffff700 1 -- start start 2020-12-05T17:32:06.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.095+0000 7fbddffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbde00aed90 0x7fbde01525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:06.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.095+0000 7fbddffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbde00b2140 0x7fbde00639e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:06.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.095+0000 7fbddffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbde01441e0 0x7fbde0142c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:06.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.095+0000 7fbddffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde0063350 con 0x7fbde00aed90 2020-12-05T17:32:06.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.095+0000 7fbddffff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde0065db0 con 0x7fbde01441e0 2020-12-05T17:32:06.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.095+0000 7fbddffff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde013f2c0 con 0x7fbde00b2140 2020-12-05T17:32:06.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.099+0000 7fbddeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbde00aed90 0x7fbde01525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:06.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.099+0000 7fbddeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbde00aed90 0x7fbde01525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33244/0 (socket says 172.21.15.32:33244) 2020-12-05T17:32:06.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.099+0000 7fbddeffd700 1 -- 172.21.15.32:0/1807260102 learned_addr learned my addr 172.21.15.32:0/1807260102 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:06.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.099+0000 7fbdde7fc700 1 --2- 172.21.15.32:0/1807260102 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbde00b2140 0x7fbde00639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:06.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.099+0000 7fbddf7fe700 1 --2- 172.21.15.32:0/1807260102 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbde01441e0 0x7fbde0142c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:06.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.099+0000 7fbddeffd700 1 -- 172.21.15.32:0/1807260102 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbde00b2140 msgr2=0x7fbde00639e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:06.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.099+0000 7fbddeffd700 1 --2- 172.21.15.32:0/1807260102 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbde00b2140 0x7fbde00639e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.099+0000 7fbddeffd700 1 -- 172.21.15.32:0/1807260102 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbde01441e0 msgr2=0x7fbde0142c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:06.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.099+0000 7fbddeffd700 1 --2- 172.21.15.32:0/1807260102 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbde01441e0 0x7fbde0142c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.099+0000 7fbddeffd700 1 -- 172.21.15.32:0/1807260102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde0063580 con 0x7fbde00aed90 2020-12-05T17:32:06.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.099+0000 7fbddf7fe700 1 --2- 172.21.15.32:0/1807260102 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbde01441e0 0x7fbde0142c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:06.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.099+0000 7fbddeffd700 1 --2- 172.21.15.32:0/1807260102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbde00aed90 0x7fbde01525c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fbdd000c000 tx=0x7fbdd0006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:06.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.099+0000 7fbdcffff700 1 -- 172.21.15.32:0/1807260102 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdd000ca10 con 0x7fbde00aed90 2020-12-05T17:32:06.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.099+0000 7fbddffff700 1 -- 172.21.15.32:0/1807260102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde0065fe0 con 0x7fbde00aed90 2020-12-05T17:32:06.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.099+0000 7fbdcffff700 1 -- 172.21.15.32:0/1807260102 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd000cb70 con 0x7fbde00aed90 2020-12-05T17:32:06.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.099+0000 7fbdcffff700 1 -- 172.21.15.32:0/1807260102 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdd00123f0 con 0x7fbde00aed90 2020-12-05T17:32:06.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.099+0000 7fbddffff700 1 -- 172.21.15.32:0/1807260102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde0140fe0 con 0x7fbde00aed90 2020-12-05T17:32:06.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.099+0000 7fbdcffff700 1 -- 172.21.15.32:0/1807260102 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbdd0012550 con 0x7fbde00aed90 2020-12-05T17:32:06.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.099+0000 7fbddffff700 1 -- 172.21.15.32:0/1807260102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde0063710 con 0x7fbde00aed90 2020-12-05T17:32:06.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.099+0000 7fbdcffff700 1 --2- 172.21.15.32:0/1807260102 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbdbc0554e0 0x7fbdbc0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:06.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.099+0000 7fbdcffff700 1 -- 172.21.15.32:0/1807260102 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fbdd006e560 con 0x7fbde00aed90 2020-12-05T17:32:06.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.099+0000 7fbdde7fc700 1 --2- 172.21.15.32:0/1807260102 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbdbc0554e0 0x7fbdbc0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:06.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.103+0000 7fbdde7fc700 1 --2- 172.21.15.32:0/1807260102 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbdbc0554e0 0x7fbdbc0579a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fbdd400d000 tx=0x7fbdd4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:06.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.107+0000 7fbdcffff700 1 -- 172.21.15.32:0/1807260102 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbdd0044600 con 0x7fbde00aed90 2020-12-05T17:32:06.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.251+0000 7fbddffff700 1 -- 172.21.15.32:0/1807260102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbde0066170 con 0x7fbde00aed90 2020-12-05T17:32:06.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.255+0000 7fbdcffff700 1 -- 172.21.15.32:0/1807260102 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7fbdd0047c20 con 0x7fbde00aed90 2020-12-05T17:32:06.258 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:32:06.259 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:31:41.610914+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:31:36.706361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-12-05T17:31:39.298269+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:32:06.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.255+0000 7fbddffff700 1 -- 172.21.15.32:0/1807260102 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbdbc0554e0 msgr2=0x7fbdbc0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:06.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.255+0000 7fbddffff700 1 --2- 172.21.15.32:0/1807260102 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbdbc0554e0 0x7fbdbc0579a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fbdd400d000 tx=0x7fbdd4009040).stop 2020-12-05T17:32:06.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.255+0000 7fbddffff700 1 -- 172.21.15.32:0/1807260102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbde00aed90 msgr2=0x7fbde01525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:06.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.255+0000 7fbddffff700 1 --2- 172.21.15.32:0/1807260102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbde00aed90 0x7fbde01525c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fbdd000c000 tx=0x7fbdd0006dc0).stop 2020-12-05T17:32:06.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.255+0000 7fbddffff700 1 -- 172.21.15.32:0/1807260102 shutdown_connections 2020-12-05T17:32:06.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.255+0000 7fbddffff700 1 --2- 172.21.15.32:0/1807260102 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbde01441e0 0x7fbde0142c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.255+0000 7fbddffff700 1 --2- 172.21.15.32:0/1807260102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbde00aed90 0x7fbde01525c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.255+0000 7fbddffff700 1 --2- 172.21.15.32:0/1807260102 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbdbc0554e0 0x7fbdbc0579a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.255+0000 7fbddffff700 1 --2- 172.21.15.32:0/1807260102 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbde00b2140 0x7fbde00639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.255+0000 7fbddffff700 1 -- 172.21.15.32:0/1807260102 >> 172.21.15.32:0/1807260102 conn(0x7fbde005bbb0 msgr2=0x7fbde00b3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:06.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.255+0000 7fbddffff700 1 -- 172.21.15.32:0/1807260102 shutdown_connections 2020-12-05T17:32:06.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.255+0000 7fbddffff700 1 -- 172.21.15.32:0/1807260102 wait complete. 2020-12-05T17:32:06.271 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:31:36.706361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:31:39.298269+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:32:06.272 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:32:06.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.387+0000 7f38f0ea5700 1 Processor -- start 2020-12-05T17:32:06.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.387+0000 7f38f0ea5700 1 -- start start 2020-12-05T17:32:06.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38f0ea5700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f38ec0b3020 0x7f38ec066e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38f0ea5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38ec144380 0x7f38ec13a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38f0ea5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38ec0b3540 con 0x7f38ec061ba0 2020-12-05T17:32:06.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38f0ea5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ec061f80 con 0x7f38ec144380 2020-12-05T17:32:06.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38f0ea5700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ec066750 con 0x7f38ec0b3020 2020-12-05T17:32:06.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38ebfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38ec144380 0x7f38ec13a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38eb7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f38ec061ba0 0x7f38ec143c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35900/0 (socket says 172.21.15.32:35900) 2020-12-05T17:32:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38eb7fe700 1 -- 172.21.15.32:0/1714502613 learned_addr learned my addr 172.21.15.32:0/1714502613 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38eaffd700 1 --2- 172.21.15.32:0/1714502613 >> v2:172.21.15.61:3300/0 conn(0x7f38ec0b3020 0x7f38ec066e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38ea7fc700 1 -- 172.21.15.32:0/1714502613 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 8654285 0 0) 0x7f38ec0b3540 con 0x7f38ec061ba0 2020-12-05T17:32:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38ea7fc700 1 -- 172.21.15.32:0/1714502613 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38d8003230 con 0x7f38ec061ba0 2020-12-05T17:32:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38ebfff700 1 -- 172.21.15.32:0/1714502613 >> v2:172.21.15.61:3300/0 conn(0x7f38ec0b3020 msgr2=0x7f38ec066e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38ebfff700 1 --2- 172.21.15.32:0/1714502613 >> v2:172.21.15.61:3300/0 conn(0x7f38ec0b3020 0x7f38ec066e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38ebfff700 1 -- 172.21.15.32:0/1714502613 >> v1:172.21.15.32:6789/0 conn(0x7f38ec061ba0 legacy=0x7f38ec143c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38ea7fc700 1 -- 172.21.15.32:0/1714502613 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2346259127 0 0) 0x7f38d8003230 con 0x7f38ec061ba0 2020-12-05T17:32:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38ebfff700 1 -- 172.21.15.32:0/1714502613 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec0669f0 con 0x7f38ec144380 2020-12-05T17:32:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38eaffd700 1 --2- 172.21.15.32:0/1714502613 >> v2:172.21.15.61:3300/0 conn(0x7f38ec0b3020 0x7f38ec066e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:32:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38ebfff700 1 --2- 172.21.15.32:0/1714502613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec144380 0x7f38ec13a1a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f38dc001870 tx=0x7f38e40091a0).ready entity=mon.0 client_cookie=32b0dc62a7e6d344 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38ea7fc700 1 -- 172.21.15.32:0/1714502613 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38dc0016d0 con 0x7f38ec144380 2020-12-05T17:32:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38ea7fc700 1 -- 172.21.15.32:0/1714502613 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e400b800 con 0x7f38ec144380 2020-12-05T17:32:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38ea7fc700 1 -- 172.21.15.32:0/1714502613 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38e400b970 con 0x7f38ec144380 2020-12-05T17:32:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38f0ea5700 1 -- 172.21.15.32:0/1714502613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec144380 msgr2=0x7f38ec13a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38f0ea5700 1 --2- 172.21.15.32:0/1714502613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec144380 0x7f38ec13a1a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f38dc001870 tx=0x7f38e40091a0).stop 2020-12-05T17:32:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38f0ea5700 1 -- 172.21.15.32:0/1714502613 shutdown_connections 2020-12-05T17:32:06.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38f0ea5700 1 --2- 172.21.15.32:0/1714502613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec144380 0x7f38ec13a1a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38f0ea5700 1 --2- 172.21.15.32:0/1714502613 >> v2:172.21.15.61:3300/0 conn(0x7f38ec0b3020 0x7f38ec066e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38f0ea5700 1 -- 172.21.15.32:0/1714502613 >> 172.21.15.32:0/1714502613 conn(0x7f38ec0b6d10 msgr2=0x7f38ec0b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:06.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38f0ea5700 1 -- 172.21.15.32:0/1714502613 shutdown_connections 2020-12-05T17:32:06.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.391+0000 7f38f0ea5700 1 -- 172.21.15.32:0/1714502613 wait complete. 2020-12-05T17:32:06.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.395+0000 7f38f0ea5700 1 Processor -- start 2020-12-05T17:32:06.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.395+0000 7f38f0ea5700 1 -- start start 2020-12-05T17:32:06.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.395+0000 7f38f0ea5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f38ec061ba0 0x7f38ec153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.395+0000 7f38f0ea5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f38ec0b3020 0x7f38ec13e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:06.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.395+0000 7f38f0ea5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec144380 0x7f38ec13a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:06.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.395+0000 7f38f0ea5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec1526f0 con 0x7f38ec144380 2020-12-05T17:32:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.395+0000 7f38f0ea5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0b69e0 con 0x7f38ec0b3020 2020-12-05T17:32:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.395+0000 7f38f0ea5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec152920 con 0x7f38ec061ba0 2020-12-05T17:32:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.395+0000 7f38ebfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec144380 0x7f38ec13a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.395+0000 7f38ebfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec144380 0x7f38ec13a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33262/0 (socket says 172.21.15.32:33262) 2020-12-05T17:32:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.395+0000 7f38ebfff700 1 -- 172.21.15.32:0/23309025 learned_addr learned my addr 172.21.15.32:0/23309025 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.395+0000 7f38eb7fe700 1 --2- 172.21.15.32:0/23309025 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f38ec061ba0 0x7f38ec153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:06.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.395+0000 7f38eaffd700 1 --2- 172.21.15.32:0/23309025 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f38ec0b3020 0x7f38ec13e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:06.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.395+0000 7f38ebfff700 1 -- 172.21.15.32:0/23309025 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f38ec061ba0 msgr2=0x7f38ec153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:06.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.395+0000 7f38ebfff700 1 --2- 172.21.15.32:0/23309025 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f38ec061ba0 0x7f38ec153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.395+0000 7f38ebfff700 1 -- 172.21.15.32:0/23309025 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f38ec0b3020 msgr2=0x7f38ec13e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.395+0000 7f38ebfff700 1 --2- 172.21.15.32:0/23309025 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f38ec0b3020 0x7f38ec13e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.395+0000 7f38ebfff700 1 -- 172.21.15.32:0/23309025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec152bc0 con 0x7f38ec144380 2020-12-05T17:32:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.395+0000 7f38ebfff700 1 --2- 172.21.15.32:0/23309025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec144380 0x7f38ec13a210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f38dc001870 tx=0x7f38e4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.399+0000 7f38e8ff9700 1 -- 172.21.15.32:0/23309025 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38e4011b40 con 0x7f38ec144380 2020-12-05T17:32:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.399+0000 7f38e8ff9700 1 -- 172.21.15.32:0/23309025 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e4011ca0 con 0x7f38ec144380 2020-12-05T17:32:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.399+0000 7f38f0ea5700 1 -- 172.21.15.32:0/23309025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ec152e60 con 0x7f38ec144380 2020-12-05T17:32:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.399+0000 7f38e8ff9700 1 -- 172.21.15.32:0/23309025 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38e4020400 con 0x7f38ec144380 2020-12-05T17:32:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.399+0000 7f38f0ea5700 1 -- 172.21.15.32:0/23309025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ec13de80 con 0x7f38ec144380 2020-12-05T17:32:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.399+0000 7f38f0ea5700 1 -- 172.21.15.32:0/23309025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ec1102b0 con 0x7f38ec144380 2020-12-05T17:32:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.399+0000 7f38e8ff9700 1 -- 172.21.15.32:0/23309025 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f38e4027030 con 0x7f38ec144380 2020-12-05T17:32:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.399+0000 7f38e8ff9700 1 --2- 172.21.15.32:0/23309025 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f38cc0553f0 0x7f38cc0578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.399+0000 7f38eb7fe700 1 --2- 172.21.15.32:0/23309025 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f38cc0553f0 0x7f38cc0578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.399+0000 7f38e8ff9700 1 -- 172.21.15.32:0/23309025 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f38e406e8b0 con 0x7f38ec144380 2020-12-05T17:32:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.403+0000 7f38eb7fe700 1 --2- 172.21.15.32:0/23309025 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f38cc0553f0 0x7f38cc0578b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f38dc00d000 tx=0x7f38dc001a60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.407+0000 7f38e8ff9700 1 -- 172.21.15.32:0/23309025 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f38ec1102b0 con 0x7f38ec144380 2020-12-05T17:32:06.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.547+0000 7f38f0ea5700 1 -- 172.21.15.32:0/23309025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f38ec1102b0 con 0x7f38ec144380 2020-12-05T17:32:06.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.547+0000 7f38e8ff9700 1 -- 172.21.15.32:0/23309025 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7f38ec1102b0 con 0x7f38ec144380 2020-12-05T17:32:06.552 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:32:06.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.551+0000 7f38f0ea5700 1 -- 172.21.15.32:0/23309025 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f38cc0553f0 msgr2=0x7f38cc0578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:06.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.551+0000 7f38f0ea5700 1 --2- 172.21.15.32:0/23309025 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f38cc0553f0 0x7f38cc0578b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f38dc00d000 tx=0x7f38dc001a60).stop 2020-12-05T17:32:06.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.551+0000 7f38f0ea5700 1 -- 172.21.15.32:0/23309025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec144380 msgr2=0x7f38ec13a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:06.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.551+0000 7f38f0ea5700 1 --2- 172.21.15.32:0/23309025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec144380 0x7f38ec13a210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f38dc001870 tx=0x7f38e4016000).stop 2020-12-05T17:32:06.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.551+0000 7f38f0ea5700 1 -- 172.21.15.32:0/23309025 shutdown_connections 2020-12-05T17:32:06.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.551+0000 7f38f0ea5700 1 --2- 172.21.15.32:0/23309025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec144380 0x7f38ec13a210 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.551+0000 7f38f0ea5700 1 --2- 172.21.15.32:0/23309025 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f38cc0553f0 0x7f38cc0578b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.551+0000 7f38f0ea5700 1 --2- 172.21.15.32:0/23309025 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f38ec061ba0 0x7f38ec153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.551+0000 7f38f0ea5700 1 --2- 172.21.15.32:0/23309025 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f38ec0b3020 0x7f38ec13e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.551+0000 7f38f0ea5700 1 -- 172.21.15.32:0/23309025 >> 172.21.15.32:0/23309025 conn(0x7f38ec0b6d10 msgr2=0x7f38ec143ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:06.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.551+0000 7f38f0ea5700 1 -- 172.21.15.32:0/23309025 shutdown_connections 2020-12-05T17:32:06.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.551+0000 7f38f0ea5700 1 -- 172.21.15.32:0/23309025 wait complete. 2020-12-05T17:32:06.573 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:32:06.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b2cd9700 1 Processor -- start 2020-12-05T17:32:06.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b2cd9700 1 -- start start 2020-12-05T17:32:06.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b2cd9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd2ac0b2100 0x7fd2ac0af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:06.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b2cd9700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd2ac0ad250 0x7fd2ac142600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:06.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b2cd9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ac0b24e0 con 0x7fd2ac13ed60 2020-12-05T17:32:06.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b2cd9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2ac13fd10 con 0x7fd2ac0b2100 2020-12-05T17:32:06.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b2cd9700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2ac0b0780 con 0x7fd2ac0ad250 2020-12-05T17:32:06.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b1cd7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd2ac0b2100 0x7fd2ac0af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:06.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b14d6700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd2ac13ed60 0x7fd2ac138820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35914/0 (socket says 172.21.15.32:35914) 2020-12-05T17:32:06.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b1cd7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd2ac0b2100 0x7fd2ac0af360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33270/0 (socket says 172.21.15.32:33270) 2020-12-05T17:32:06.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b14d6700 1 -- 172.21.15.32:0/2853606076 learned_addr learned my addr 172.21.15.32:0/2853606076 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:06.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b24d8700 1 --2- 172.21.15.32:0/2853606076 >> v2:172.21.15.61:3300/0 conn(0x7fd2ac0ad250 0x7fd2ac142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:06.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b0cd5700 1 -- 172.21.15.32:0/2853606076 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 886016332 0 0) 0x7fd2ac0b24e0 con 0x7fd2ac13ed60 2020-12-05T17:32:06.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b1cd7700 1 -- 172.21.15.32:0/2853606076 >> v2:172.21.15.61:3300/0 conn(0x7fd2ac0ad250 msgr2=0x7fd2ac142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b1cd7700 1 --2- 172.21.15.32:0/2853606076 >> v2:172.21.15.61:3300/0 conn(0x7fd2ac0ad250 0x7fd2ac142600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b1cd7700 1 -- 172.21.15.32:0/2853606076 >> v1:172.21.15.32:6789/0 conn(0x7fd2ac13ed60 legacy=0x7fd2ac138820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b1cd7700 1 -- 172.21.15.32:0/2853606076 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac0b0240 con 0x7fd2ac0b2100 2020-12-05T17:32:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b1cd7700 1 --2- 172.21.15.32:0/2853606076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac0b2100 0x7fd2ac0af360 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fd2a8002f30 tx=0x7fd2a8006dc0).ready entity=mon.0 client_cookie=9964dfa7f59e0773 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b0cd5700 1 -- 172.21.15.32:0/2853606076 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2a80097b0 con 0x7fd2ac0b2100 2020-12-05T17:32:06.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b0cd5700 1 -- 172.21.15.32:0/2853606076 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a8009910 con 0x7fd2ac0b2100 2020-12-05T17:32:06.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b0cd5700 1 -- 172.21.15.32:0/2853606076 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2a8009c70 con 0x7fd2ac0b2100 2020-12-05T17:32:06.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b2cd9700 1 -- 172.21.15.32:0/2853606076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac0b2100 msgr2=0x7fd2ac0af360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:06.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b2cd9700 1 --2- 172.21.15.32:0/2853606076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac0b2100 0x7fd2ac0af360 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fd2a8002f30 tx=0x7fd2a8006dc0).stop 2020-12-05T17:32:06.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b2cd9700 1 -- 172.21.15.32:0/2853606076 shutdown_connections 2020-12-05T17:32:06.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b2cd9700 1 --2- 172.21.15.32:0/2853606076 >> v2:172.21.15.61:3300/0 conn(0x7fd2ac0ad250 0x7fd2ac142600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b2cd9700 1 --2- 172.21.15.32:0/2853606076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac0b2100 0x7fd2ac0af360 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b2cd9700 1 -- 172.21.15.32:0/2853606076 >> 172.21.15.32:0/2853606076 conn(0x7fd2ac0620b0 msgr2=0x7fd2ac062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:06.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b2cd9700 1 -- 172.21.15.32:0/2853606076 shutdown_connections 2020-12-05T17:32:06.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.699+0000 7fd2b2cd9700 1 -- 172.21.15.32:0/2853606076 wait complete. 2020-12-05T17:32:06.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.703+0000 7fd2b2cd9700 1 Processor -- start 2020-12-05T17:32:06.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.703+0000 7fd2b2cd9700 1 -- start start 2020-12-05T17:32:06.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.703+0000 7fd2b2cd9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2ac0ad250 0x7fd2ac141780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:06.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.703+0000 7fd2b2cd9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd2ac0b2100 0x7fd2ac0662b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.703+0000 7fd2b2cd9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac064bd0 0x7fd2ac0667a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.703+0000 7fd2b2cd9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac0631c0 con 0x7fd2ac064bd0 2020-12-05T17:32:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.703+0000 7fd2b2cd9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac063460 con 0x7fd2ac0b2100 2020-12-05T17:32:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.703+0000 7fd2b2cd9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac13c930 con 0x7fd2ac0ad250 2020-12-05T17:32:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.703+0000 7fd2b24d8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac064bd0 0x7fd2ac0667a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.703+0000 7fd2b24d8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac064bd0 0x7fd2ac0667a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33276/0 (socket says 172.21.15.32:33276) 2020-12-05T17:32:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.703+0000 7fd2b24d8700 1 -- 172.21.15.32:0/2543581036 learned_addr learned my addr 172.21.15.32:0/2543581036 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.703+0000 7fd2b1cd7700 1 --2- 172.21.15.32:0/2543581036 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2ac0ad250 0x7fd2ac141780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.703+0000 7fd2b14d6700 1 --2- 172.21.15.32:0/2543581036 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd2ac0b2100 0x7fd2ac0662b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:06.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.703+0000 7fd2b24d8700 1 -- 172.21.15.32:0/2543581036 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2ac0ad250 msgr2=0x7fd2ac141780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:06.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.703+0000 7fd2b24d8700 1 --2- 172.21.15.32:0/2543581036 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2ac0ad250 0x7fd2ac141780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.703+0000 7fd2b24d8700 1 -- 172.21.15.32:0/2543581036 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd2ac0b2100 msgr2=0x7fd2ac0662b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:06.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.703+0000 7fd2b24d8700 1 --2- 172.21.15.32:0/2543581036 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd2ac0b2100 0x7fd2ac0662b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.703+0000 7fd2b24d8700 1 -- 172.21.15.32:0/2543581036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac063990 con 0x7fd2ac064bd0 2020-12-05T17:32:06.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.707+0000 7fd2b24d8700 1 --2- 172.21.15.32:0/2543581036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac064bd0 0x7fd2ac0667a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7fd2a4006dd0 tx=0x7fd2a40091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:06.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.707+0000 7fd29affd700 1 -- 172.21.15.32:0/2543581036 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2a400c950 con 0x7fd2ac064bd0 2020-12-05T17:32:06.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.707+0000 7fd29affd700 1 -- 172.21.15.32:0/2543581036 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a400cab0 con 0x7fd2ac064bd0 2020-12-05T17:32:06.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.707+0000 7fd2b2cd9700 1 -- 172.21.15.32:0/2543581036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ac13c2f0 con 0x7fd2ac064bd0 2020-12-05T17:32:06.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.707+0000 7fd29affd700 1 -- 172.21.15.32:0/2543581036 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2a400ce10 con 0x7fd2ac064bd0 2020-12-05T17:32:06.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.707+0000 7fd2b2cd9700 1 -- 172.21.15.32:0/2543581036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ac066eb0 con 0x7fd2ac064bd0 2020-12-05T17:32:06.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.707+0000 7fd2b2cd9700 1 -- 172.21.15.32:0/2543581036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2ac063020 con 0x7fd2ac064bd0 2020-12-05T17:32:06.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.707+0000 7fd29affd700 1 -- 172.21.15.32:0/2543581036 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd2a4022070 con 0x7fd2ac064bd0 2020-12-05T17:32:06.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.707+0000 7fd29affd700 1 --2- 172.21.15.32:0/2543581036 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd28c0554e0 0x7fd28c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:06.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.707+0000 7fd29affd700 1 -- 172.21.15.32:0/2543581036 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd2a406e640 con 0x7fd2ac064bd0 2020-12-05T17:32:06.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.707+0000 7fd2b1cd7700 1 --2- 172.21.15.32:0/2543581036 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd28c0554e0 0x7fd28c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:06.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.707+0000 7fd2b1cd7700 1 --2- 172.21.15.32:0/2543581036 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd28c0554e0 0x7fd28c0579a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fd2a801a000 tx=0x7fd2a8015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:06.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.719+0000 7fd29affd700 1 -- 172.21.15.32:0/2543581036 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd2ac063020 con 0x7fd2ac064bd0 2020-12-05T17:32:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.859+0000 7fd2b2cd9700 1 -- 172.21.15.32:0/2543581036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd2ac063020 con 0x7fd2ac064bd0 2020-12-05T17:32:06.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.859+0000 7fd29affd700 1 -- 172.21.15.32:0/2543581036 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd2ac063020 con 0x7fd2ac064bd0 2020-12-05T17:32:06.864 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:32:06.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.863+0000 7fd2b2cd9700 1 -- 172.21.15.32:0/2543581036 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd28c0554e0 msgr2=0x7fd28c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:06.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.863+0000 7fd2b2cd9700 1 --2- 172.21.15.32:0/2543581036 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd28c0554e0 0x7fd28c0579a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fd2a801a000 tx=0x7fd2a8015000).stop 2020-12-05T17:32:06.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.863+0000 7fd2b2cd9700 1 -- 172.21.15.32:0/2543581036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac064bd0 msgr2=0x7fd2ac0667a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:06.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.863+0000 7fd2b2cd9700 1 --2- 172.21.15.32:0/2543581036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac064bd0 0x7fd2ac0667a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7fd2a4006dd0 tx=0x7fd2a40091a0).stop 2020-12-05T17:32:06.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.863+0000 7fd2b2cd9700 1 -- 172.21.15.32:0/2543581036 shutdown_connections 2020-12-05T17:32:06.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.863+0000 7fd2b2cd9700 1 --2- 172.21.15.32:0/2543581036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac064bd0 0x7fd2ac0667a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.863+0000 7fd2b2cd9700 1 --2- 172.21.15.32:0/2543581036 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd28c0554e0 0x7fd28c0579a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.863+0000 7fd2b2cd9700 1 --2- 172.21.15.32:0/2543581036 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2ac0ad250 0x7fd2ac141780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.863+0000 7fd2b2cd9700 1 --2- 172.21.15.32:0/2543581036 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd2ac0b2100 0x7fd2ac0662b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.863+0000 7fd2b2cd9700 1 -- 172.21.15.32:0/2543581036 >> 172.21.15.32:0/2543581036 conn(0x7fd2ac0620b0 msgr2=0x7fd2ac0b52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.863+0000 7fd2b2cd9700 1 -- 172.21.15.32:0/2543581036 shutdown_connections 2020-12-05T17:32:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.863+0000 7fd2b2cd9700 1 -- 172.21.15.32:0/2543581036 wait complete. 2020-12-05T17:32:06.878 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:32:06.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.995+0000 7f437ed5c700 1 Processor -- start 2020-12-05T17:32:06.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.995+0000 7f437ed5c700 1 -- start start 2020-12-05T17:32:06.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.995+0000 7f437ed5c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43780b3910 0x7f43780b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:06.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.995+0000 7f437ed5c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43781468d0 con 0x7f43780b6970 2020-12-05T17:32:06.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.995+0000 7f437ed5c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4378146da0 con 0x7f43780b42e0 2020-12-05T17:32:07.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.995+0000 7f437ed5c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43780b5e90 con 0x7f43780b3910 2020-12-05T17:32:07.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.995+0000 7f437caf8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43780b3910 0x7f43780b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:07.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.995+0000 7f437caf8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43780b3910 0x7f43780b3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33280/0 (socket says 172.21.15.32:33280) 2020-12-05T17:32:07.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.995+0000 7f437d2f9700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f43780b6970 0x7f43781428c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35930/0 (socket says 172.21.15.32:35930) 2020-12-05T17:32:07.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.995+0000 7f437caf8700 1 -- 172.21.15.32:0/4097632515 learned_addr learned my addr 172.21.15.32:0/4097632515 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:07.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.995+0000 7f437caf8700 1 -- 172.21.15.32:0/4097632515 >> v1:172.21.15.61:6789/0 conn(0x7f43780b42e0 legacy=0x7f43780b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:07.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.995+0000 7f437caf8700 1 -- 172.21.15.32:0/4097632515 >> v1:172.21.15.32:6789/0 conn(0x7f43780b6970 legacy=0x7f43781428c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:07.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.995+0000 7f437caf8700 1 -- 172.21.15.32:0/4097632515 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4378146b00 con 0x7f43780b3910 2020-12-05T17:32:07.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.995+0000 7f43777fe700 1 -- 172.21.15.32:0/4097632515 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2745182055 0 0) 0x7f43781468d0 con 0x7f43780b6970 2020-12-05T17:32:07.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.995+0000 7f437caf8700 1 --2- 172.21.15.32:0/4097632515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43780b3910 0x7f43780b3cf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f43680032b0 tx=0x7f4368009270).ready entity=mon.0 client_cookie=8daa27a5057b418d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:07.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.995+0000 7f43777fe700 1 -- 172.21.15.32:0/4097632515 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f436800bc00 con 0x7f43780b3910 2020-12-05T17:32:07.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.995+0000 7f43777fe700 1 -- 172.21.15.32:0/4097632515 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436800bd60 con 0x7f43780b3910 2020-12-05T17:32:07.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.995+0000 7f43777fe700 1 -- 172.21.15.32:0/4097632515 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4368011400 con 0x7f43780b3910 2020-12-05T17:32:07.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.995+0000 7f437ed5c700 1 -- 172.21.15.32:0/4097632515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43780b3910 msgr2=0x7f43780b3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:07.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.995+0000 7f437ed5c700 1 --2- 172.21.15.32:0/4097632515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43780b3910 0x7f43780b3cf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f43680032b0 tx=0x7f4368009270).stop 2020-12-05T17:32:07.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.999+0000 7f437ed5c700 1 -- 172.21.15.32:0/4097632515 shutdown_connections 2020-12-05T17:32:07.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.999+0000 7f437ed5c700 1 --2- 172.21.15.32:0/4097632515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43780b3910 0x7f43780b3cf0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.999+0000 7f437ed5c700 1 -- 172.21.15.32:0/4097632515 >> 172.21.15.32:0/4097632515 conn(0x7f437805f3b0 msgr2=0x7f437805f790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:07.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.999+0000 7f437ed5c700 1 -- 172.21.15.32:0/4097632515 shutdown_connections 2020-12-05T17:32:07.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.999+0000 7f437ed5c700 1 -- 172.21.15.32:0/4097632515 wait complete. 2020-12-05T17:32:07.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.999+0000 7f437ed5c700 1 Processor -- start 2020-12-05T17:32:07.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.999+0000 7f437ed5c700 1 -- start start 2020-12-05T17:32:07.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.999+0000 7f437ed5c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f43780b3910 0x7f4378137fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:07.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.999+0000 7f437ed5c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f43780b42e0 0x7f43780b0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:07.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.999+0000 7f437ed5c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43780b6970 0x7f43780b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:07.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.999+0000 7f437ed5c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43780ae360 con 0x7f43780b6970 2020-12-05T17:32:07.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.999+0000 7f437ed5c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43780ae600 con 0x7f43780b3910 2020-12-05T17:32:07.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:06.999+0000 7f437ed5c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43780ae8a0 con 0x7f43780b42e0 2020-12-05T17:32:07.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.003+0000 7f437d2f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43780b6970 0x7f43780b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:07.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.003+0000 7f437d2f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43780b6970 0x7f43780b0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33290/0 (socket says 172.21.15.32:33290) 2020-12-05T17:32:07.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.003+0000 7f437d2f9700 1 -- 172.21.15.32:0/1478707311 learned_addr learned my addr 172.21.15.32:0/1478707311 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:07.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.003+0000 7f437caf8700 1 --2- 172.21.15.32:0/1478707311 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f43780b3910 0x7f4378137fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:07.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.003+0000 7f4377fff700 1 --2- 172.21.15.32:0/1478707311 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f43780b42e0 0x7f43780b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:07.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.003+0000 7f437d2f9700 1 -- 172.21.15.32:0/1478707311 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f43780b42e0 msgr2=0x7f43780b0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:07.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.003+0000 7f437d2f9700 1 --2- 172.21.15.32:0/1478707311 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f43780b42e0 0x7f43780b0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.003+0000 7f437d2f9700 1 -- 172.21.15.32:0/1478707311 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f43780b3910 msgr2=0x7f4378137fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:07.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.003+0000 7f437d2f9700 1 --2- 172.21.15.32:0/1478707311 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f43780b3910 0x7f4378137fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.003+0000 7f437d2f9700 1 -- 172.21.15.32:0/1478707311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4378139880 con 0x7f43780b6970 2020-12-05T17:32:07.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.003+0000 7f4377fff700 1 --2- 172.21.15.32:0/1478707311 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f43780b42e0 0x7f43780b0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.003+0000 7f437d2f9700 1 --2- 172.21.15.32:0/1478707311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43780b6970 0x7f43780b0730 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f4370000b80 tx=0x7f4370007b40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.003+0000 7f4375ffb700 1 -- 172.21.15.32:0/1478707311 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43700073b0 con 0x7f43780b6970 2020-12-05T17:32:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.003+0000 7f4375ffb700 1 -- 172.21.15.32:0/1478707311 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4370005520 con 0x7f43780b6970 2020-12-05T17:32:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.003+0000 7f437ed5c700 1 -- 172.21.15.32:0/1478707311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4378138c50 con 0x7f43780b6970 2020-12-05T17:32:07.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.003+0000 7f4375ffb700 1 -- 172.21.15.32:0/1478707311 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4370005880 con 0x7f43780b6970 2020-12-05T17:32:07.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.003+0000 7f437ed5c700 1 -- 172.21.15.32:0/1478707311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43780aed40 con 0x7f43780b6970 2020-12-05T17:32:07.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.003+0000 7f437ed5c700 1 -- 172.21.15.32:0/1478707311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4378057050 con 0x7f43780b6970 2020-12-05T17:32:07.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.003+0000 7f4375ffb700 1 -- 172.21.15.32:0/1478707311 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f437001a060 con 0x7f43780b6970 2020-12-05T17:32:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.003+0000 7f4375ffb700 1 --2- 172.21.15.32:0/1478707311 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4358055330 0x7f43580577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.003+0000 7f4375ffb700 1 -- 172.21.15.32:0/1478707311 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f437006e060 con 0x7f43780b6970 2020-12-05T17:32:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.003+0000 7f437caf8700 1 --2- 172.21.15.32:0/1478707311 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4358055330 0x7f43580577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.007+0000 7f437caf8700 1 --2- 172.21.15.32:0/1478707311 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4358055330 0x7f43580577f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f436801a000 tx=0x7f4368016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:07.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.011+0000 7f4375ffb700 1 -- 172.21.15.32:0/1478707311 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f43700440f0 con 0x7f43780b6970 2020-12-05T17:32:07.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.151+0000 7f437ed5c700 1 -- 172.21.15.32:0/1478707311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4378069690 con 0x7f43780b6970 2020-12-05T17:32:07.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.151+0000 7f4375ffb700 1 -- 172.21.15.32:0/1478707311 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7f4370018070 con 0x7f43780b6970 2020-12-05T17:32:07.156 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:32:07.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.155+0000 7f437ed5c700 1 -- 172.21.15.32:0/1478707311 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4358055330 msgr2=0x7f43580577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:07.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.155+0000 7f437ed5c700 1 --2- 172.21.15.32:0/1478707311 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4358055330 0x7f43580577f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f436801a000 tx=0x7f4368016000).stop 2020-12-05T17:32:07.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.155+0000 7f437ed5c700 1 -- 172.21.15.32:0/1478707311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43780b6970 msgr2=0x7f43780b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:07.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.155+0000 7f437ed5c700 1 --2- 172.21.15.32:0/1478707311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43780b6970 0x7f43780b0730 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f4370000b80 tx=0x7f4370007b40).stop 2020-12-05T17:32:07.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.155+0000 7f437ed5c700 1 -- 172.21.15.32:0/1478707311 shutdown_connections 2020-12-05T17:32:07.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.155+0000 7f437ed5c700 1 --2- 172.21.15.32:0/1478707311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43780b6970 0x7f43780b0730 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.155+0000 7f437ed5c700 1 --2- 172.21.15.32:0/1478707311 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f43780b3910 0x7f4378137fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.155+0000 7f437ed5c700 1 --2- 172.21.15.32:0/1478707311 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4358055330 0x7f43580577f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.155+0000 7f437ed5c700 1 --2- 172.21.15.32:0/1478707311 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f43780b42e0 0x7f43780b0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.155+0000 7f437ed5c700 1 -- 172.21.15.32:0/1478707311 >> 172.21.15.32:0/1478707311 conn(0x7f437805f3b0 msgr2=0x7f4378062120 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:07.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.155+0000 7f437ed5c700 1 -- 172.21.15.32:0/1478707311 shutdown_connections 2020-12-05T17:32:07.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.155+0000 7f437ed5c700 1 -- 172.21.15.32:0/1478707311 wait complete. 2020-12-05T17:32:07.169 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:32:07.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.287+0000 7f0c2759e700 1 Processor -- start 2020-12-05T17:32:07.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.287+0000 7f0c2759e700 1 -- start start 2020-12-05T17:32:07.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.287+0000 7f0c2759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0c280b2960 0x7f0c28140400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:07.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.287+0000 7f0c2759e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0c280667f0 0x7f0c280ae830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:07.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.287+0000 7f0c2759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c280ad9b0 con 0x7f0c280ad280 2020-12-05T17:32:07.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.287+0000 7f0c2759e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c280b32c0 con 0x7f0c280b2960 2020-12-05T17:32:07.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.287+0000 7f0c2759e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2813f4f0 con 0x7f0c280667f0 2020-12-05T17:32:07.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.287+0000 7f0c1dd9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0c280ad280 0x7f0c280660e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35942/0 (socket says 172.21.15.32:35942) 2020-12-05T17:32:07.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.287+0000 7f0c1dd9b700 1 -- 172.21.15.32:0/2178601088 learned_addr learned my addr 172.21.15.32:0/2178601088 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:07.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.287+0000 7f0c2659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0c280b2960 0x7f0c28140400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:07.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.287+0000 7f0c26d9d700 1 --2- 172.21.15.32:0/2178601088 >> v2:172.21.15.61:3300/0 conn(0x7f0c280667f0 0x7f0c280ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:07.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.291+0000 7f0c25d9b700 1 -- 172.21.15.32:0/2178601088 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2277965300 0 0) 0x7f0c280ad9b0 con 0x7f0c280ad280 2020-12-05T17:32:07.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.291+0000 7f0c25d9b700 1 -- 172.21.15.32:0/2178601088 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c08003230 con 0x7f0c280ad280 2020-12-05T17:32:07.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.291+0000 7f0c2659c700 1 -- 172.21.15.32:0/2178601088 >> v2:172.21.15.61:3300/0 conn(0x7f0c280667f0 msgr2=0x7f0c280ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:07.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.291+0000 7f0c2659c700 1 --2- 172.21.15.32:0/2178601088 >> v2:172.21.15.61:3300/0 conn(0x7f0c280667f0 0x7f0c280ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.291+0000 7f0c2659c700 1 -- 172.21.15.32:0/2178601088 >> v1:172.21.15.32:6789/0 conn(0x7f0c280ad280 legacy=0x7f0c280660e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:07.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.291+0000 7f0c2659c700 1 -- 172.21.15.32:0/2178601088 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c280b3020 con 0x7f0c280b2960 2020-12-05T17:32:07.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.291+0000 7f0c2659c700 1 --2- 172.21.15.32:0/2178601088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c280b2960 0x7f0c28140400 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f0c18002f30 tx=0x7f0c18006dc0).ready entity=mon.0 client_cookie=d2c21f9ac24cdc7b server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:07.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.291+0000 7f0c25d9b700 1 -- 172.21.15.32:0/2178601088 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c18009580 con 0x7f0c280b2960 2020-12-05T17:32:07.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.291+0000 7f0c25d9b700 1 -- 172.21.15.32:0/2178601088 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c180096e0 con 0x7f0c280b2960 2020-12-05T17:32:07.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.291+0000 7f0c25d9b700 1 -- 172.21.15.32:0/2178601088 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c18009850 con 0x7f0c280b2960 2020-12-05T17:32:07.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.291+0000 7f0c2759e700 1 -- 172.21.15.32:0/2178601088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c280b2960 msgr2=0x7f0c28140400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:07.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.291+0000 7f0c2759e700 1 --2- 172.21.15.32:0/2178601088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c280b2960 0x7f0c28140400 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f0c18002f30 tx=0x7f0c18006dc0).stop 2020-12-05T17:32:07.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.291+0000 7f0c2759e700 1 -- 172.21.15.32:0/2178601088 shutdown_connections 2020-12-05T17:32:07.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.291+0000 7f0c2759e700 1 --2- 172.21.15.32:0/2178601088 >> v2:172.21.15.61:3300/0 conn(0x7f0c280667f0 0x7f0c280ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.291+0000 7f0c2759e700 1 --2- 172.21.15.32:0/2178601088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c280b2960 0x7f0c28140400 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.291+0000 7f0c2759e700 1 -- 172.21.15.32:0/2178601088 >> 172.21.15.32:0/2178601088 conn(0x7f0c28061010 msgr2=0x7f0c28061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.291+0000 7f0c2759e700 1 -- 172.21.15.32:0/2178601088 shutdown_connections 2020-12-05T17:32:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.291+0000 7f0c2759e700 1 -- 172.21.15.32:0/2178601088 wait complete. 2020-12-05T17:32:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.291+0000 7f0c2759e700 1 Processor -- start 2020-12-05T17:32:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c2759e700 1 -- start start 2020-12-05T17:32:07.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c2759e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c280667f0 0x7f0c28142b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:07.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c2759e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c280ad280 0x7f0c28142ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:07.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c2759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c280b2960 0x7f0c2813df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:07.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c2759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c281420c0 con 0x7f0c280b2960 2020-12-05T17:32:07.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c2759e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c281412c0 con 0x7f0c280667f0 2020-12-05T17:32:07.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c2759e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c281422f0 con 0x7f0c280ad280 2020-12-05T17:32:07.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c26d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c280b2960 0x7f0c2813df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:07.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c26d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c280b2960 0x7f0c2813df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33304/0 (socket says 172.21.15.32:33304) 2020-12-05T17:32:07.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c26d9d700 1 -- 172.21.15.32:0/975420215 learned_addr learned my addr 172.21.15.32:0/975420215 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:07.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c1dd9b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c280ad280 0x7f0c28142ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:07.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c2659c700 1 --2- 172.21.15.32:0/975420215 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c280667f0 0x7f0c28142b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:07.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c26d9d700 1 -- 172.21.15.32:0/975420215 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c280ad280 msgr2=0x7f0c28142ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:07.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c26d9d700 1 --2- 172.21.15.32:0/975420215 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c280ad280 0x7f0c28142ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c26d9d700 1 -- 172.21.15.32:0/975420215 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c280667f0 msgr2=0x7f0c28142b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:07.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c26d9d700 1 --2- 172.21.15.32:0/975420215 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c280667f0 0x7f0c28142b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c26d9d700 1 -- 172.21.15.32:0/975420215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c281525c0 con 0x7f0c280b2960 2020-12-05T17:32:07.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c2659c700 1 --2- 172.21.15.32:0/975420215 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c280667f0 0x7f0c28142b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:07.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c26d9d700 1 --2- 172.21.15.32:0/975420215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c280b2960 0x7f0c2813df10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f0c20006dd0 tx=0x7f0c200091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:07.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c1ffff700 1 -- 172.21.15.32:0/975420215 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c2000ca00 con 0x7f0c280b2960 2020-12-05T17:32:07.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c1ffff700 1 -- 172.21.15.32:0/975420215 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2000cb60 con 0x7f0c280b2960 2020-12-05T17:32:07.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c2759e700 1 -- 172.21.15.32:0/975420215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c28140d40 con 0x7f0c280b2960 2020-12-05T17:32:07.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c1ffff700 1 -- 172.21.15.32:0/975420215 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c200123f0 con 0x7f0c280b2960 2020-12-05T17:32:07.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.295+0000 7f0c2759e700 1 -- 172.21.15.32:0/975420215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c28064a00 con 0x7f0c280b2960 2020-12-05T17:32:07.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.299+0000 7f0c2759e700 1 -- 172.21.15.32:0/975420215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c28057050 con 0x7f0c280b2960 2020-12-05T17:32:07.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.299+0000 7f0c1ffff700 1 -- 172.21.15.32:0/975420215 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0c20022070 con 0x7f0c280b2960 2020-12-05T17:32:07.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.299+0000 7f0c1ffff700 1 --2- 172.21.15.32:0/975420215 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c0c055440 0x7f0c0c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:07.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.299+0000 7f0c1ffff700 1 -- 172.21.15.32:0/975420215 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0c2006e740 con 0x7f0c280b2960 2020-12-05T17:32:07.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.299+0000 7f0c2659c700 1 --2- 172.21.15.32:0/975420215 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c0c055440 0x7f0c0c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:07.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.299+0000 7f0c2659c700 1 --2- 172.21.15.32:0/975420215 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c0c055440 0x7f0c0c057900 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f0c18019000 tx=0x7f0c18016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:07.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.303+0000 7f0c1ffff700 1 -- 172.21.15.32:0/975420215 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0c200448e0 con 0x7f0c280b2960 2020-12-05T17:32:07.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.447+0000 7f0c2759e700 1 -- 172.21.15.32:0/975420215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0c28069690 con 0x7f0c280b2960 2020-12-05T17:32:07.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.447+0000 7f0c1ffff700 1 -- 172.21.15.32:0/975420215 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7f0c20016020 con 0x7f0c280b2960 2020-12-05T17:32:07.452 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:32:07.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.451+0000 7f0c2759e700 1 -- 172.21.15.32:0/975420215 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c0c055440 msgr2=0x7f0c0c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:07.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.451+0000 7f0c2759e700 1 --2- 172.21.15.32:0/975420215 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c0c055440 0x7f0c0c057900 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f0c18019000 tx=0x7f0c18016000).stop 2020-12-05T17:32:07.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.451+0000 7f0c2759e700 1 -- 172.21.15.32:0/975420215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c280b2960 msgr2=0x7f0c2813df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:07.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.451+0000 7f0c2759e700 1 --2- 172.21.15.32:0/975420215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c280b2960 0x7f0c2813df10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f0c20006dd0 tx=0x7f0c200091a0).stop 2020-12-05T17:32:07.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.451+0000 7f0c2759e700 1 -- 172.21.15.32:0/975420215 shutdown_connections 2020-12-05T17:32:07.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.451+0000 7f0c2759e700 1 --2- 172.21.15.32:0/975420215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c280b2960 0x7f0c2813df10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.451+0000 7f0c2759e700 1 --2- 172.21.15.32:0/975420215 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c280667f0 0x7f0c28142b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.451+0000 7f0c2759e700 1 --2- 172.21.15.32:0/975420215 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c0c055440 0x7f0c0c057900 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.451+0000 7f0c2759e700 1 --2- 172.21.15.32:0/975420215 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c280ad280 0x7f0c28142ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.451+0000 7f0c2759e700 1 -- 172.21.15.32:0/975420215 >> 172.21.15.32:0/975420215 conn(0x7f0c28061010 msgr2=0x7f0c2805f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:07.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.451+0000 7f0c2759e700 1 -- 172.21.15.32:0/975420215 shutdown_connections 2020-12-05T17:32:07.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.451+0000 7f0c2759e700 1 -- 172.21.15.32:0/975420215 wait complete. 2020-12-05T17:32:07.466 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:32:07.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.571+0000 7f6404cbd700 1 Processor -- start 2020-12-05T17:32:07.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.571+0000 7f6404cbd700 1 -- start start 2020-12-05T17:32:07.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.571+0000 7f6404cbd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f640005f6f0 0x7f64000b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:07.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.571+0000 7f6404cbd700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f64000af920 0x7f6400063040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:07.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.571+0000 7f6404cbd700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f640005fad0 con 0x7f64000afea0 2020-12-05T17:32:07.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.571+0000 7f6404cbd700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64000b2cd0 con 0x7f64000af920 2020-12-05T17:32:07.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.571+0000 7f6404cbd700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64000b3560 con 0x7f640005f6f0 2020-12-05T17:32:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.571+0000 7f63ff7fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f640005f6f0 0x7f64000b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.571+0000 7f63ff7fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f640005f6f0 0x7f64000b1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:58330/0 (socket says 172.21.15.32:58330) 2020-12-05T17:32:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.571+0000 7f63ff7fe700 1 -- 172.21.15.32:0/3785751409 learned_addr learned my addr 172.21.15.32:0/3785751409 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.571+0000 7f63feffd700 1 --2- 172.21.15.32:0/3785751409 >> v2:172.21.15.61:3300/0 conn(0x7f64000af920 0x7f6400063040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.571+0000 7f63ff7fe700 1 -- 172.21.15.32:0/3785751409 >> v2:172.21.15.61:3300/0 conn(0x7f64000af920 msgr2=0x7f6400063040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.571+0000 7f63ff7fe700 1 --2- 172.21.15.32:0/3785751409 >> v2:172.21.15.61:3300/0 conn(0x7f64000af920 0x7f6400063040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.571+0000 7f63ff7fe700 1 -- 172.21.15.32:0/3785751409 >> v1:172.21.15.61:6789/0 conn(0x7f64000afea0 legacy=0x7f640013f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.571+0000 7f63ff7fe700 1 -- 172.21.15.32:0/3785751409 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6400144650 con 0x7f640005f6f0 2020-12-05T17:32:07.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.571+0000 7f63ff7fe700 1 --2- 172.21.15.32:0/3785751409 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f640005f6f0 0x7f64000b1d90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f63f0002f30 tx=0x7f63f0006e80).ready entity=mon.2 client_cookie=efa0ea781c82f69f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:07.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.571+0000 7f63fe7fc700 1 -- 172.21.15.32:0/3785751409 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63f00097f0 con 0x7f640005f6f0 2020-12-05T17:32:07.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.571+0000 7f63fe7fc700 1 -- 172.21.15.32:0/3785751409 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f0009950 con 0x7f640005f6f0 2020-12-05T17:32:07.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.571+0000 7f63fe7fc700 1 -- 172.21.15.32:0/3785751409 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63f0009cb0 con 0x7f640005f6f0 2020-12-05T17:32:07.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.571+0000 7f6404cbd700 1 -- 172.21.15.32:0/3785751409 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f640005f6f0 msgr2=0x7f64000b1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:07.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.571+0000 7f6404cbd700 1 --2- 172.21.15.32:0/3785751409 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f640005f6f0 0x7f64000b1d90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f63f0002f30 tx=0x7f63f0006e80).stop 2020-12-05T17:32:07.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.575+0000 7f6404cbd700 1 -- 172.21.15.32:0/3785751409 shutdown_connections 2020-12-05T17:32:07.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.575+0000 7f6404cbd700 1 --2- 172.21.15.32:0/3785751409 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f640005f6f0 0x7f64000b1d90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.575+0000 7f6404cbd700 1 --2- 172.21.15.32:0/3785751409 >> v2:172.21.15.61:3300/0 conn(0x7f64000af920 0x7f6400063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.575+0000 7f6404cbd700 1 -- 172.21.15.32:0/3785751409 >> 172.21.15.32:0/3785751409 conn(0x7f64000604c0 msgr2=0x7f640005c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:07.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.575+0000 7f6404cbd700 1 -- 172.21.15.32:0/3785751409 shutdown_connections 2020-12-05T17:32:07.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.575+0000 7f6404cbd700 1 -- 172.21.15.32:0/3785751409 wait complete. 2020-12-05T17:32:07.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.575+0000 7f6404cbd700 1 Processor -- start 2020-12-05T17:32:07.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.575+0000 7f6404cbd700 1 -- start start 2020-12-05T17:32:07.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.575+0000 7f6404cbd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f640005f6f0 0x7f6400147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.575+0000 7f6404cbd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f64000af920 0x7f6400110fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.575+0000 7f6404cbd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f64000afea0 0x7f64001114c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.575+0000 7f6404cbd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640010ff80 con 0x7f640005f6f0 2020-12-05T17:32:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.575+0000 7f6404cbd700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6400065200 con 0x7f64000af920 2020-12-05T17:32:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.579+0000 7f6404cbd700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6400066420 con 0x7f64000afea0 2020-12-05T17:32:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.579+0000 7f63ff7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f640005f6f0 0x7f6400147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.579+0000 7f63ff7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f640005f6f0 0x7f6400147640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33314/0 (socket says 172.21.15.32:33314) 2020-12-05T17:32:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.579+0000 7f63ff7fe700 1 -- 172.21.15.32:0/1733888484 learned_addr learned my addr 172.21.15.32:0/1733888484 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.579+0000 7f63feffd700 1 --2- 172.21.15.32:0/1733888484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f64000af920 0x7f6400110fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.579+0000 7f63fffff700 1 --2- 172.21.15.32:0/1733888484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f64000afea0 0x7f64001114c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.579+0000 7f63ff7fe700 1 -- 172.21.15.32:0/1733888484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f64000afea0 msgr2=0x7f64001114c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.579+0000 7f63ff7fe700 1 --2- 172.21.15.32:0/1733888484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f64000afea0 0x7f64001114c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.579+0000 7f63ff7fe700 1 -- 172.21.15.32:0/1733888484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f64000af920 msgr2=0x7f6400110fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.579+0000 7f63ff7fe700 1 --2- 172.21.15.32:0/1733888484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f64000af920 0x7f6400110fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.579+0000 7f63ff7fe700 1 -- 172.21.15.32:0/1733888484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6400066180 con 0x7f640005f6f0 2020-12-05T17:32:07.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.579+0000 7f63fffff700 1 --2- 172.21.15.32:0/1733888484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f64000afea0 0x7f64001114c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:07.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.579+0000 7f63feffd700 1 --2- 172.21.15.32:0/1733888484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f64000af920 0x7f6400110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:07.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.579+0000 7f63ff7fe700 1 --2- 172.21.15.32:0/1733888484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f640005f6f0 0x7f6400147640 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f64000665b0 tx=0x7f63f0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:07.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.579+0000 7f63fcff9700 1 -- 172.21.15.32:0/1733888484 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63f0011c20 con 0x7f640005f6f0 2020-12-05T17:32:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.579+0000 7f63fcff9700 1 -- 172.21.15.32:0/1733888484 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f0011d80 con 0x7f640005f6f0 2020-12-05T17:32:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.579+0000 7f63fcff9700 1 -- 172.21.15.32:0/1733888484 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63f001e650 con 0x7f640005f6f0 2020-12-05T17:32:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.579+0000 7f6404cbd700 1 -- 172.21.15.32:0/1733888484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6400065430 con 0x7f640005f6f0 2020-12-05T17:32:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.579+0000 7f6404cbd700 1 -- 172.21.15.32:0/1733888484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6400066630 con 0x7f640005f6f0 2020-12-05T17:32:07.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.579+0000 7f6404cbd700 1 -- 172.21.15.32:0/1733888484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6400057050 con 0x7f640005f6f0 2020-12-05T17:32:07.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.583+0000 7f63fcff9700 1 -- 172.21.15.32:0/1733888484 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f63f0025030 con 0x7f640005f6f0 2020-12-05T17:32:07.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.583+0000 7f63fcff9700 1 --2- 172.21.15.32:0/1733888484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f63e0055440 0x7f63e0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:07.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.583+0000 7f63fcff9700 1 -- 172.21.15.32:0/1733888484 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f63f006f3e0 con 0x7f640005f6f0 2020-12-05T17:32:07.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.583+0000 7f63feffd700 1 --2- 172.21.15.32:0/1733888484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f63e0055440 0x7f63e0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:07.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.583+0000 7f63feffd700 1 --2- 172.21.15.32:0/1733888484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f63e0055440 0x7f63e0057900 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f63f400b000 tx=0x7f63f4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:07.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.587+0000 7f63fcff9700 1 -- 172.21.15.32:0/1733888484 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f63f0045580 con 0x7f640005f6f0 2020-12-05T17:32:07.730 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.727+0000 7f6404cbd700 1 -- 172.21.15.32:0/1733888484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6400110d50 con 0x7f640005f6f0 2020-12-05T17:32:07.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.731+0000 7f63fcff9700 1 -- 172.21.15.32:0/1733888484 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f63f0023090 con 0x7f640005f6f0 2020-12-05T17:32:07.733 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:32:07.733 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:31:41.610914+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:31:36.706361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-12-05T17:31:39.298269+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:32:07.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.731+0000 7f6404cbd700 1 -- 172.21.15.32:0/1733888484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f63e0055440 msgr2=0x7f63e0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:07.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.731+0000 7f6404cbd700 1 --2- 172.21.15.32:0/1733888484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f63e0055440 0x7f63e0057900 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f63f400b000 tx=0x7f63f4008040).stop 2020-12-05T17:32:07.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.731+0000 7f6404cbd700 1 -- 172.21.15.32:0/1733888484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f640005f6f0 msgr2=0x7f6400147640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:07.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.731+0000 7f6404cbd700 1 --2- 172.21.15.32:0/1733888484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f640005f6f0 0x7f6400147640 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f64000665b0 tx=0x7f63f0015000).stop 2020-12-05T17:32:07.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.731+0000 7f6404cbd700 1 -- 172.21.15.32:0/1733888484 shutdown_connections 2020-12-05T17:32:07.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.731+0000 7f6404cbd700 1 --2- 172.21.15.32:0/1733888484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f63e0055440 0x7f63e0057900 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.731+0000 7f6404cbd700 1 --2- 172.21.15.32:0/1733888484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f64000afea0 0x7f64001114c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.731+0000 7f6404cbd700 1 --2- 172.21.15.32:0/1733888484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f640005f6f0 0x7f6400147640 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.731+0000 7f6404cbd700 1 --2- 172.21.15.32:0/1733888484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f64000af920 0x7f6400110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.731+0000 7f6404cbd700 1 -- 172.21.15.32:0/1733888484 >> 172.21.15.32:0/1733888484 conn(0x7f64000604c0 msgr2=0x7f64000b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:07.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.731+0000 7f6404cbd700 1 -- 172.21.15.32:0/1733888484 shutdown_connections 2020-12-05T17:32:07.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.731+0000 7f6404cbd700 1 -- 172.21.15.32:0/1733888484 wait complete. 2020-12-05T17:32:07.747 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:31:36.706361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:31:39.298269+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:32:07.748 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:32:07.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.867+0000 7f81b65b9700 1 Processor -- start 2020-12-05T17:32:07.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.867+0000 7f81b65b9700 1 -- start start 2020-12-05T17:32:07.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.867+0000 7f81b65b9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81b005feb0 0x7f81b005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:07.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.867+0000 7f81b65b9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f81b00b2fb0 0x7f81b0061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:07.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.867+0000 7f81b65b9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b00b2df0 con 0x7f81b005f3b0 2020-12-05T17:32:07.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.867+0000 7f81b65b9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b00b3c90 con 0x7f81b005feb0 2020-12-05T17:32:07.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.867+0000 7f81b65b9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b00b2b50 con 0x7f81b00b2fb0 2020-12-05T17:32:07.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.867+0000 7f81b4db6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81b005feb0 0x7f81b005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:07.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.867+0000 7f81b4db6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81b005feb0 0x7f81b005f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33324/0 (socket says 172.21.15.32:33324) 2020-12-05T17:32:07.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.867+0000 7f81b4db6700 1 -- 172.21.15.32:0/3967645917 learned_addr learned my addr 172.21.15.32:0/3967645917 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:07.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.867+0000 7f81b5db8700 1 --2- 172.21.15.32:0/3967645917 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f81b00b2fb0 0x7f81b0061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:07.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.867+0000 7f81b4db6700 1 -- 172.21.15.32:0/3967645917 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f81b00b2fb0 msgr2=0x7f81b0061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:07.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.867+0000 7f81b4db6700 1 --2- 172.21.15.32:0/3967645917 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f81b00b2fb0 0x7f81b0061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.867+0000 7f81b4db6700 1 -- 172.21.15.32:0/3967645917 >> v1:172.21.15.32:6789/0 conn(0x7f81b005f3b0 legacy=0x7f81b013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:07.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.867+0000 7f81b4db6700 1 -- 172.21.15.32:0/3967645917 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b0142820 con 0x7f81b005feb0 2020-12-05T17:32:07.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.867+0000 7f81b4db6700 1 --2- 172.21.15.32:0/3967645917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b005feb0 0x7f81b005f790 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f81ac002f30 tx=0x7f81ac006e80).ready entity=mon.0 client_cookie=8a7edceb73bab82e server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:07.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81a7fff700 1 -- 172.21.15.32:0/3967645917 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81ac0097f0 con 0x7f81b005feb0 2020-12-05T17:32:07.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81a7fff700 1 -- 172.21.15.32:0/3967645917 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ac009950 con 0x7f81b005feb0 2020-12-05T17:32:07.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81a7fff700 1 -- 172.21.15.32:0/3967645917 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81ac009ac0 con 0x7f81b005feb0 2020-12-05T17:32:07.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b65b9700 1 -- 172.21.15.32:0/3967645917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b005feb0 msgr2=0x7f81b005f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:07.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b65b9700 1 --2- 172.21.15.32:0/3967645917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b005feb0 0x7f81b005f790 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f81ac002f30 tx=0x7f81ac006e80).stop 2020-12-05T17:32:07.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b65b9700 1 -- 172.21.15.32:0/3967645917 shutdown_connections 2020-12-05T17:32:07.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b65b9700 1 --2- 172.21.15.32:0/3967645917 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f81b00b2fb0 0x7f81b0061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b65b9700 1 --2- 172.21.15.32:0/3967645917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b005feb0 0x7f81b005f790 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b65b9700 1 -- 172.21.15.32:0/3967645917 >> 172.21.15.32:0/3967645917 conn(0x7f81b00b8f20 msgr2=0x7f81b00b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:07.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b65b9700 1 -- 172.21.15.32:0/3967645917 shutdown_connections 2020-12-05T17:32:07.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b65b9700 1 -- 172.21.15.32:0/3967645917 wait complete. 2020-12-05T17:32:07.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b65b9700 1 Processor -- start 2020-12-05T17:32:07.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b65b9700 1 -- start start 2020-12-05T17:32:07.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b65b9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b005f3b0 0x7f81b0065130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:07.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b65b9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f81b005feb0 0x7f81b0148420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:07.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b65b9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f81b00b2fb0 0x7f81b013cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:07.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b65b9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b0066ab0 con 0x7f81b005f3b0 2020-12-05T17:32:07.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b65b9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b01463b0 con 0x7f81b00b2fb0 2020-12-05T17:32:07.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b65b9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b0151e70 con 0x7f81b005feb0 2020-12-05T17:32:07.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b55b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b005f3b0 0x7f81b0065130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:07.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b55b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b005f3b0 0x7f81b0065130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33328/0 (socket says 172.21.15.32:33328) 2020-12-05T17:32:07.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b55b7700 1 -- 172.21.15.32:0/341309279 learned_addr learned my addr 172.21.15.32:0/341309279 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:07.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b4db6700 1 --2- 172.21.15.32:0/341309279 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f81b005feb0 0x7f81b0148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:07.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b5db8700 1 --2- 172.21.15.32:0/341309279 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f81b00b2fb0 0x7f81b013cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:07.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b55b7700 1 -- 172.21.15.32:0/341309279 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f81b005feb0 msgr2=0x7f81b0148420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:07.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b55b7700 1 --2- 172.21.15.32:0/341309279 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f81b005feb0 0x7f81b0148420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b55b7700 1 -- 172.21.15.32:0/341309279 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f81b00b2fb0 msgr2=0x7f81b013cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:07.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b55b7700 1 --2- 172.21.15.32:0/341309279 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f81b00b2fb0 0x7f81b013cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:07.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b55b7700 1 -- 172.21.15.32:0/341309279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b0065dd0 con 0x7f81b005f3b0 2020-12-05T17:32:07.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.871+0000 7f81b55b7700 1 --2- 172.21.15.32:0/341309279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b005f3b0 0x7f81b0065130 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f81b0152000 tx=0x7f81a0007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:07.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.875+0000 7f81a67fc700 1 -- 172.21.15.32:0/341309279 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81a0007240 con 0x7f81b005f3b0 2020-12-05T17:32:07.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.875+0000 7f81a67fc700 1 -- 172.21.15.32:0/341309279 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a00073a0 con 0x7f81b005f3b0 2020-12-05T17:32:07.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.875+0000 7f81b65b9700 1 -- 172.21.15.32:0/341309279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b0146e60 con 0x7f81b005f3b0 2020-12-05T17:32:07.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.875+0000 7f81a67fc700 1 -- 172.21.15.32:0/341309279 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81a00066d0 con 0x7f81b005f3b0 2020-12-05T17:32:07.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.875+0000 7f81b65b9700 1 -- 172.21.15.32:0/341309279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b0148910 con 0x7f81b005f3b0 2020-12-05T17:32:07.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.875+0000 7f81a67fc700 1 -- 172.21.15.32:0/341309279 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f81a0010070 con 0x7f81b005f3b0 2020-12-05T17:32:07.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.875+0000 7f81b65b9700 1 -- 172.21.15.32:0/341309279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b013d440 con 0x7f81b005f3b0 2020-12-05T17:32:07.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.875+0000 7f81a67fc700 1 --2- 172.21.15.32:0/341309279 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8198055330 0x7f81980577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:07.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.875+0000 7f81a67fc700 1 -- 172.21.15.32:0/341309279 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f81a006d750 con 0x7f81b005f3b0 2020-12-05T17:32:07.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.875+0000 7f81b4db6700 1 --2- 172.21.15.32:0/341309279 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8198055330 0x7f81980577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:07.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.875+0000 7f81b4db6700 1 --2- 172.21.15.32:0/341309279 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8198055330 0x7f81980577f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f81ac01b000 tx=0x7f81ac016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:07.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:07.883+0000 7f81a67fc700 1 -- 172.21.15.32:0/341309279 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f81a00438f0 con 0x7f81b005f3b0 2020-12-05T17:32:08.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.019+0000 7f81b65b9700 1 -- 172.21.15.32:0/341309279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f81b0146c80 con 0x7f81b005f3b0 2020-12-05T17:32:08.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.019+0000 7f81a67fc700 1 -- 172.21.15.32:0/341309279 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7f81a0018020 con 0x7f81b005f3b0 2020-12-05T17:32:08.024 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:32:08.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.023+0000 7f81b65b9700 1 -- 172.21.15.32:0/341309279 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8198055330 msgr2=0x7f81980577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:08.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.023+0000 7f81b65b9700 1 --2- 172.21.15.32:0/341309279 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8198055330 0x7f81980577f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f81ac01b000 tx=0x7f81ac016000).stop 2020-12-05T17:32:08.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.023+0000 7f81b65b9700 1 -- 172.21.15.32:0/341309279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b005f3b0 msgr2=0x7f81b0065130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:08.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.023+0000 7f81b65b9700 1 --2- 172.21.15.32:0/341309279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b005f3b0 0x7f81b0065130 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f81b0152000 tx=0x7f81a0007be0).stop 2020-12-05T17:32:08.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.023+0000 7f81b65b9700 1 -- 172.21.15.32:0/341309279 shutdown_connections 2020-12-05T17:32:08.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.023+0000 7f81b65b9700 1 --2- 172.21.15.32:0/341309279 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f81b00b2fb0 0x7f81b013cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.023+0000 7f81b65b9700 1 --2- 172.21.15.32:0/341309279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b005f3b0 0x7f81b0065130 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.023+0000 7f81b65b9700 1 --2- 172.21.15.32:0/341309279 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8198055330 0x7f81980577f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.023+0000 7f81b65b9700 1 --2- 172.21.15.32:0/341309279 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f81b005feb0 0x7f81b0148420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.023+0000 7f81b65b9700 1 -- 172.21.15.32:0/341309279 >> 172.21.15.32:0/341309279 conn(0x7f81b00b8f20 msgr2=0x7f81b00b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:08.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.023+0000 7f81b65b9700 1 -- 172.21.15.32:0/341309279 shutdown_connections 2020-12-05T17:32:08.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.023+0000 7f81b65b9700 1 -- 172.21.15.32:0/341309279 wait complete. 2020-12-05T17:32:08.038 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:32:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.151+0000 7f45423df700 1 Processor -- start 2020-12-05T17:32:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.151+0000 7f45423df700 1 -- start start 2020-12-05T17:32:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.151+0000 7f45423df700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f453c05feb0 0x7f453c05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.151+0000 7f45423df700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f453c0b2fb0 0x7f453c061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.151+0000 7f45423df700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f453c0b2df0 con 0x7f453c05f3b0 2020-12-05T17:32:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.151+0000 7f45423df700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f453c0b3c90 con 0x7f453c05feb0 2020-12-05T17:32:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.151+0000 7f45423df700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453c0b2b50 con 0x7f453c0b2fb0 2020-12-05T17:32:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.151+0000 7f45413dd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f453c05f3b0 0x7f453c13c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35984/0 (socket says 172.21.15.32:35984) 2020-12-05T17:32:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.151+0000 7f45413dd700 1 -- 172.21.15.32:0/1202352627 learned_addr learned my addr 172.21.15.32:0/1202352627 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:08.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.151+0000 7f4541bde700 1 --2- 172.21.15.32:0/1202352627 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f453c0b2fb0 0x7f453c061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:08.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.151+0000 7f4540bdc700 1 --2- 172.21.15.32:0/1202352627 >> v2:172.21.15.61:3300/0 conn(0x7f453c05feb0 0x7f453c05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:08.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.151+0000 7f453bfff700 1 -- 172.21.15.32:0/1202352627 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 126251811 0 0) 0x7f453c0b2df0 con 0x7f453c05f3b0 2020-12-05T17:32:08.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.151+0000 7f453bfff700 1 -- 172.21.15.32:0/1202352627 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f451c003230 con 0x7f453c05f3b0 2020-12-05T17:32:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.151+0000 7f4540bdc700 1 -- 172.21.15.32:0/1202352627 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f453c0b2fb0 msgr2=0x7f453c061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.151+0000 7f4540bdc700 1 --2- 172.21.15.32:0/1202352627 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f453c0b2fb0 0x7f453c061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.151+0000 7f4540bdc700 1 -- 172.21.15.32:0/1202352627 >> v1:172.21.15.32:6789/0 conn(0x7f453c05f3b0 legacy=0x7f453c13c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.151+0000 7f4540bdc700 1 -- 172.21.15.32:0/1202352627 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453c142820 con 0x7f453c05feb0 2020-12-05T17:32:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.151+0000 7f4541bde700 1 --2- 172.21.15.32:0/1202352627 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f453c0b2fb0 0x7f453c061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:32:08.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.155+0000 7f4540bdc700 1 --2- 172.21.15.32:0/1202352627 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f453c05feb0 0x7f453c05f790 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f4524002f30 tx=0x7f4524006dc0).ready entity=mon.1 client_cookie=74c6dd068497f768 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:08.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.155+0000 7f453bfff700 1 -- 172.21.15.32:0/1202352627 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45240096a0 con 0x7f453c05feb0 2020-12-05T17:32:08.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.155+0000 7f453bfff700 1 -- 172.21.15.32:0/1202352627 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4524009800 con 0x7f453c05feb0 2020-12-05T17:32:08.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.155+0000 7f453bfff700 1 -- 172.21.15.32:0/1202352627 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4524009b60 con 0x7f453c05feb0 2020-12-05T17:32:08.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.155+0000 7f45423df700 1 -- 172.21.15.32:0/1202352627 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f453c05feb0 msgr2=0x7f453c05f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:08.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.155+0000 7f45423df700 1 --2- 172.21.15.32:0/1202352627 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f453c05feb0 0x7f453c05f790 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f4524002f30 tx=0x7f4524006dc0).stop 2020-12-05T17:32:08.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.155+0000 7f45423df700 1 -- 172.21.15.32:0/1202352627 shutdown_connections 2020-12-05T17:32:08.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.155+0000 7f45423df700 1 --2- 172.21.15.32:0/1202352627 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f453c0b2fb0 0x7f453c061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.155+0000 7f45423df700 1 --2- 172.21.15.32:0/1202352627 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f453c05feb0 0x7f453c05f790 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.155+0000 7f45423df700 1 -- 172.21.15.32:0/1202352627 >> 172.21.15.32:0/1202352627 conn(0x7f453c0b8f20 msgr2=0x7f453c0b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.155+0000 7f45423df700 1 -- 172.21.15.32:0/1202352627 shutdown_connections 2020-12-05T17:32:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.155+0000 7f45423df700 1 -- 172.21.15.32:0/1202352627 wait complete. 2020-12-05T17:32:08.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.155+0000 7f45423df700 1 Processor -- start 2020-12-05T17:32:08.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.155+0000 7f45423df700 1 -- start start 2020-12-05T17:32:08.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f45423df700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f453c05f3b0 0x7f453c154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:08.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f45423df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f453c05feb0 0x7f453c145fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:08.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f45423df700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f453c0b2fb0 0x7f453c113680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:08.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f45423df700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c064d30 con 0x7f453c05feb0 2020-12-05T17:32:08.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f45423df700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c064fd0 con 0x7f453c0b2fb0 2020-12-05T17:32:08.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f45423df700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453c0647b0 con 0x7f453c05f3b0 2020-12-05T17:32:08.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f4540bdc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f453c05feb0 0x7f453c145fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:08.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f4540bdc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f453c05feb0 0x7f453c145fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33344/0 (socket says 172.21.15.32:33344) 2020-12-05T17:32:08.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f4540bdc700 1 -- 172.21.15.32:0/1299590453 learned_addr learned my addr 172.21.15.32:0/1299590453 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:08.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f45413dd700 1 --2- 172.21.15.32:0/1299590453 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f453c05f3b0 0x7f453c154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:08.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f4541bde700 1 --2- 172.21.15.32:0/1299590453 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f453c0b2fb0 0x7f453c113680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:08.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f4540bdc700 1 -- 172.21.15.32:0/1299590453 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f453c05f3b0 msgr2=0x7f453c154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:08.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f4540bdc700 1 --2- 172.21.15.32:0/1299590453 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f453c05f3b0 0x7f453c154920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f4540bdc700 1 -- 172.21.15.32:0/1299590453 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f453c0b2fb0 msgr2=0x7f453c113680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:08.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f4540bdc700 1 --2- 172.21.15.32:0/1299590453 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f453c0b2fb0 0x7f453c113680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f4540bdc700 1 -- 172.21.15.32:0/1299590453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453c064920 con 0x7f453c05feb0 2020-12-05T17:32:08.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f4540bdc700 1 --2- 172.21.15.32:0/1299590453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f453c05feb0 0x7f453c145fc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f4524002f30 tx=0x7f4524015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:08.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f453a7fc700 1 -- 172.21.15.32:0/1299590453 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4524011b60 con 0x7f453c05feb0 2020-12-05T17:32:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f453a7fc700 1 -- 172.21.15.32:0/1299590453 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4524011cc0 con 0x7f453c05feb0 2020-12-05T17:32:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f453a7fc700 1 -- 172.21.15.32:0/1299590453 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f452401d640 con 0x7f453c05feb0 2020-12-05T17:32:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f45423df700 1 -- 172.21.15.32:0/1299590453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f453c065b60 con 0x7f453c05feb0 2020-12-05T17:32:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f45423df700 1 -- 172.21.15.32:0/1299590453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f453c113b70 con 0x7f453c05feb0 2020-12-05T17:32:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f45423df700 1 -- 172.21.15.32:0/1299590453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f453c064b70 con 0x7f453c05feb0 2020-12-05T17:32:08.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.159+0000 7f453a7fc700 1 -- 172.21.15.32:0/1299590453 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4524024030 con 0x7f453c05feb0 2020-12-05T17:32:08.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.163+0000 7f453a7fc700 1 --2- 172.21.15.32:0/1299590453 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f45200553b0 0x7f4520057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:08.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.163+0000 7f453a7fc700 1 -- 172.21.15.32:0/1299590453 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f452406ee90 con 0x7f453c05feb0 2020-12-05T17:32:08.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.163+0000 7f45413dd700 1 --2- 172.21.15.32:0/1299590453 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f45200553b0 0x7f4520057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:08.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.163+0000 7f45413dd700 1 --2- 172.21.15.32:0/1299590453 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f45200553b0 0x7f4520057870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f452c00d000 tx=0x7f452c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:08.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.171+0000 7f453a7fc700 1 -- 172.21.15.32:0/1299590453 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4524045070 con 0x7f453c05feb0 2020-12-05T17:32:08.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.311+0000 7f45423df700 1 -- 172.21.15.32:0/1299590453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f453c065140 con 0x7f453c05feb0 2020-12-05T17:32:08.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.311+0000 7f453a7fc700 1 -- 172.21.15.32:0/1299590453 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4524022070 con 0x7f453c05feb0 2020-12-05T17:32:08.316 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:32:08.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.311+0000 7f45423df700 1 -- 172.21.15.32:0/1299590453 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f45200553b0 msgr2=0x7f4520057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:08.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.311+0000 7f45423df700 1 --2- 172.21.15.32:0/1299590453 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f45200553b0 0x7f4520057870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f452c00d000 tx=0x7f452c009040).stop 2020-12-05T17:32:08.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.311+0000 7f45423df700 1 -- 172.21.15.32:0/1299590453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f453c05feb0 msgr2=0x7f453c145fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:08.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.311+0000 7f45423df700 1 --2- 172.21.15.32:0/1299590453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f453c05feb0 0x7f453c145fc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f4524002f30 tx=0x7f4524015000).stop 2020-12-05T17:32:08.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.311+0000 7f45423df700 1 -- 172.21.15.32:0/1299590453 shutdown_connections 2020-12-05T17:32:08.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.311+0000 7f45423df700 1 --2- 172.21.15.32:0/1299590453 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f453c0b2fb0 0x7f453c113680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.311+0000 7f45423df700 1 --2- 172.21.15.32:0/1299590453 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f45200553b0 0x7f4520057870 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.311+0000 7f45423df700 1 --2- 172.21.15.32:0/1299590453 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f453c05f3b0 0x7f453c154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.311+0000 7f45423df700 1 --2- 172.21.15.32:0/1299590453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f453c05feb0 0x7f453c145fc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.311+0000 7f45423df700 1 -- 172.21.15.32:0/1299590453 >> 172.21.15.32:0/1299590453 conn(0x7f453c0b8f20 msgr2=0x7f453c0b7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:08.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.315+0000 7f45423df700 1 -- 172.21.15.32:0/1299590453 shutdown_connections 2020-12-05T17:32:08.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.315+0000 7f45423df700 1 -- 172.21.15.32:0/1299590453 wait complete. 2020-12-05T17:32:08.328 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:32:08.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.447+0000 7f33e1786700 1 Processor -- start 2020-12-05T17:32:08.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.447+0000 7f33e1786700 1 -- start start 2020-12-05T17:32:08.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.447+0000 7f33e1786700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f33dc05f3b0 0x7f33dc05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:08.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.447+0000 7f33e1786700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f33dc05feb0 0x7f33dc061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:08.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.447+0000 7f33e1786700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33dc0b2df0 con 0x7f33dc0b2fb0 2020-12-05T17:32:08.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.447+0000 7f33e1786700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33dc0b3c90 con 0x7f33dc05feb0 2020-12-05T17:32:08.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.447+0000 7f33e1786700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33dc0b2b50 con 0x7f33dc05f3b0 2020-12-05T17:32:08.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.447+0000 7f33d37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f33dc05feb0 0x7f33dc061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:08.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.447+0000 7f33e0f85700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f33dc0b2fb0 0x7f33dc13c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36002/0 (socket says 172.21.15.32:36002) 2020-12-05T17:32:08.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.447+0000 7f33e0f85700 1 -- 172.21.15.32:0/2020242058 learned_addr learned my addr 172.21.15.32:0/2020242058 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:08.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.447+0000 7f33d37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f33dc05feb0 0x7f33dc061780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33352/0 (socket says 172.21.15.32:33352) 2020-12-05T17:32:08.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.447+0000 7f33dbfff700 1 --2- 172.21.15.32:0/2020242058 >> v2:172.21.15.61:3300/0 conn(0x7f33dc05f3b0 0x7f33dc05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:08.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.447+0000 7f33d37fe700 1 -- 172.21.15.32:0/2020242058 >> v2:172.21.15.61:3300/0 conn(0x7f33dc05f3b0 msgr2=0x7f33dc05f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:08.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.447+0000 7f33d37fe700 1 --2- 172.21.15.32:0/2020242058 >> v2:172.21.15.61:3300/0 conn(0x7f33dc05f3b0 0x7f33dc05f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.447+0000 7f33d37fe700 1 -- 172.21.15.32:0/2020242058 >> v1:172.21.15.32:6789/0 conn(0x7f33dc0b2fb0 legacy=0x7f33dc13c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:08.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.447+0000 7f33d37fe700 1 -- 172.21.15.32:0/2020242058 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33dc142820 con 0x7f33dc05feb0 2020-12-05T17:32:08.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.451+0000 7f33d37fe700 1 --2- 172.21.15.32:0/2020242058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33dc05feb0 0x7f33dc061780 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f33c4002f30 tx=0x7f33c4006dc0).ready entity=mon.0 client_cookie=1cecf21e1c63f7a4 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:08.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.451+0000 7f33db7fe700 1 -- 172.21.15.32:0/2020242058 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33c40097f0 con 0x7f33dc05feb0 2020-12-05T17:32:08.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.451+0000 7f33db7fe700 1 -- 172.21.15.32:0/2020242058 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c4009950 con 0x7f33dc05feb0 2020-12-05T17:32:08.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.451+0000 7f33db7fe700 1 -- 172.21.15.32:0/2020242058 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33c4009cb0 con 0x7f33dc05feb0 2020-12-05T17:32:08.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.451+0000 7f33e1786700 1 -- 172.21.15.32:0/2020242058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33dc05feb0 msgr2=0x7f33dc061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:08.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.451+0000 7f33e1786700 1 --2- 172.21.15.32:0/2020242058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33dc05feb0 0x7f33dc061780 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f33c4002f30 tx=0x7f33c4006dc0).stop 2020-12-05T17:32:08.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.451+0000 7f33e1786700 1 -- 172.21.15.32:0/2020242058 shutdown_connections 2020-12-05T17:32:08.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.451+0000 7f33e1786700 1 --2- 172.21.15.32:0/2020242058 >> v2:172.21.15.61:3300/0 conn(0x7f33dc05f3b0 0x7f33dc05f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.451+0000 7f33e1786700 1 --2- 172.21.15.32:0/2020242058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33dc05feb0 0x7f33dc061780 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.451+0000 7f33e1786700 1 -- 172.21.15.32:0/2020242058 >> 172.21.15.32:0/2020242058 conn(0x7f33dc0b8f20 msgr2=0x7f33dc0b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:08.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.451+0000 7f33e1786700 1 -- 172.21.15.32:0/2020242058 shutdown_connections 2020-12-05T17:32:08.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.451+0000 7f33e1786700 1 -- 172.21.15.32:0/2020242058 wait complete. 2020-12-05T17:32:08.462 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33e1786700 1 Processor -- start 2020-12-05T17:32:08.463 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33e1786700 1 -- start start 2020-12-05T17:32:08.463 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33e1786700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33dc05f3b0 0x7f33dc065fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:08.463 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33e1786700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f33dc05feb0 0x7f33dc151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:08.463 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33e1786700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f33dc0b2fb0 0x7f33dc154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:08.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33e1786700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc064840 con 0x7f33dc05f3b0 2020-12-05T17:32:08.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33e1786700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc151970 con 0x7f33dc05feb0 2020-12-05T17:32:08.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33e1786700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc0645a0 con 0x7f33dc0b2fb0 2020-12-05T17:32:08.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33dbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33dc05f3b0 0x7f33dc065fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:08.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33dbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33dc05f3b0 0x7f33dc065fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33358/0 (socket says 172.21.15.32:33358) 2020-12-05T17:32:08.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33dbfff700 1 -- 172.21.15.32:0/3464782488 learned_addr learned my addr 172.21.15.32:0/3464782488 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:08.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33e0f85700 1 --2- 172.21.15.32:0/3464782488 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f33dc0b2fb0 0x7f33dc154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:08.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33d37fe700 1 --2- 172.21.15.32:0/3464782488 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f33dc05feb0 0x7f33dc151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:08.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33dbfff700 1 -- 172.21.15.32:0/3464782488 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f33dc0b2fb0 msgr2=0x7f33dc154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:08.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33dbfff700 1 --2- 172.21.15.32:0/3464782488 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f33dc0b2fb0 0x7f33dc154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33dbfff700 1 -- 172.21.15.32:0/3464782488 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f33dc05feb0 msgr2=0x7f33dc151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:08.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33dbfff700 1 --2- 172.21.15.32:0/3464782488 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f33dc05feb0 0x7f33dc151480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33dbfff700 1 -- 172.21.15.32:0/3464782488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33dc154b20 con 0x7f33dc05f3b0 2020-12-05T17:32:08.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33dbfff700 1 --2- 172.21.15.32:0/3464782488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33dc05f3b0 0x7f33dc065fe0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f33cc003070 tx=0x7f33cc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:08.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33d9ffb700 1 -- 172.21.15.32:0/3464782488 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33cc009790 con 0x7f33dc05f3b0 2020-12-05T17:32:08.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33d9ffb700 1 -- 172.21.15.32:0/3464782488 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33cc0098f0 con 0x7f33dc05f3b0 2020-12-05T17:32:08.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33d9ffb700 1 -- 172.21.15.32:0/3464782488 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33cc009c50 con 0x7f33dc05f3b0 2020-12-05T17:32:08.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33e1786700 1 -- 172.21.15.32:0/3464782488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33dc064a80 con 0x7f33dc05f3b0 2020-12-05T17:32:08.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.455+0000 7f33e1786700 1 -- 172.21.15.32:0/3464782488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33dc064310 con 0x7f33dc05f3b0 2020-12-05T17:32:08.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.459+0000 7f33e1786700 1 -- 172.21.15.32:0/3464782488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33dc065860 con 0x7f33dc05f3b0 2020-12-05T17:32:08.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.459+0000 7f33d9ffb700 1 -- 172.21.15.32:0/3464782488 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f33cc009db0 con 0x7f33dc05f3b0 2020-12-05T17:32:08.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.459+0000 7f33d9ffb700 1 --2- 172.21.15.32:0/3464782488 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f33c0055470 0x7f33c0057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:08.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.459+0000 7f33d9ffb700 1 -- 172.21.15.32:0/3464782488 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f33cc06e4f0 con 0x7f33dc05f3b0 2020-12-05T17:32:08.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.459+0000 7f33d37fe700 1 --2- 172.21.15.32:0/3464782488 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f33c0055470 0x7f33c0057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.459+0000 7f33d37fe700 1 --2- 172.21.15.32:0/3464782488 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f33c0055470 0x7f33c0057930 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f33c401b000 tx=0x7f33c4015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.463+0000 7f33d9ffb700 1 -- 172.21.15.32:0/3464782488 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f33dc065860 con 0x7f33dc05f3b0 2020-12-05T17:32:08.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.603+0000 7f33e1786700 1 -- 172.21.15.32:0/3464782488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f33dc065860 con 0x7f33dc05f3b0 2020-12-05T17:32:08.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.607+0000 7f33d9ffb700 1 -- 172.21.15.32:0/3464782488 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7f33dc065860 con 0x7f33dc05f3b0 2020-12-05T17:32:08.608 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:32:08.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.607+0000 7f33e1786700 1 -- 172.21.15.32:0/3464782488 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f33c0055470 msgr2=0x7f33c0057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:08.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.607+0000 7f33e1786700 1 --2- 172.21.15.32:0/3464782488 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f33c0055470 0x7f33c0057930 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f33c401b000 tx=0x7f33c4015000).stop 2020-12-05T17:32:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.607+0000 7f33e1786700 1 -- 172.21.15.32:0/3464782488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33dc05f3b0 msgr2=0x7f33dc065fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.607+0000 7f33e1786700 1 --2- 172.21.15.32:0/3464782488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33dc05f3b0 0x7f33dc065fe0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f33cc003070 tx=0x7f33cc008040).stop 2020-12-05T17:32:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.607+0000 7f33e1786700 1 -- 172.21.15.32:0/3464782488 shutdown_connections 2020-12-05T17:32:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.607+0000 7f33e1786700 1 --2- 172.21.15.32:0/3464782488 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f33c0055470 0x7f33c0057930 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.607+0000 7f33e1786700 1 --2- 172.21.15.32:0/3464782488 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f33dc0b2fb0 0x7f33dc154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.607+0000 7f33e1786700 1 --2- 172.21.15.32:0/3464782488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33dc05f3b0 0x7f33dc065fe0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.607+0000 7f33e1786700 1 --2- 172.21.15.32:0/3464782488 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f33dc05feb0 0x7f33dc151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.607+0000 7f33e1786700 1 -- 172.21.15.32:0/3464782488 >> 172.21.15.32:0/3464782488 conn(0x7f33dc0b8f20 msgr2=0x7f33dc0667b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:08.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.607+0000 7f33e1786700 1 -- 172.21.15.32:0/3464782488 shutdown_connections 2020-12-05T17:32:08.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.607+0000 7f33e1786700 1 -- 172.21.15.32:0/3464782488 wait complete. 2020-12-05T17:32:08.621 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:32:08.740 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.735+0000 7fa669f0e700 1 Processor -- start 2020-12-05T17:32:08.740 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.735+0000 7fa669f0e700 1 -- start start 2020-12-05T17:32:08.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa669f0e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa6640b2000 0x7fa6640af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:08.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa669f0e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa6640aed90 0x7fa664140db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:08.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa669f0e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa664129180 0x7fa664129610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:08.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa669f0e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6640649e0 con 0x7fa6640b2000 2020-12-05T17:32:08.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa669f0e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6640b23e0 con 0x7fa6640aed90 2020-12-05T17:32:08.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa669f0e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa664062d80 con 0x7fa664129180 2020-12-05T17:32:08.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa668f0c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa6640b2000 0x7fa6640af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:08.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa663fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa6640aed90 0x7fa664140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:08.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa668f0c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa6640b2000 0x7fa6640af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33366/0 (socket says 172.21.15.32:33366) 2020-12-05T17:32:08.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa66970d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa664129180 0x7fa664129610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:08.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa668f0c700 1 -- 172.21.15.32:0/164936393 learned_addr learned my addr 172.21.15.32:0/164936393 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa663fff700 1 -- 172.21.15.32:0/164936393 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa664129180 msgr2=0x7fa664129610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa663fff700 1 --2- 172.21.15.32:0/164936393 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa664129180 0x7fa664129610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa663fff700 1 -- 172.21.15.32:0/164936393 >> v2:172.21.15.32:3300/0 conn(0x7fa6640b2000 msgr2=0x7fa6640af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:08.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa663fff700 1 --2- 172.21.15.32:0/164936393 >> v2:172.21.15.32:3300/0 conn(0x7fa6640b2000 0x7fa6640af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa663fff700 1 -- 172.21.15.32:0/164936393 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa664140670 con 0x7fa6640aed90 2020-12-05T17:32:08.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa663fff700 1 --2- 172.21.15.32:0/164936393 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa6640aed90 0x7fa664140db0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fa658002f30 tx=0x7fa658006dc0).ready entity=mon.1 client_cookie=ae2d38f57d3ad751 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:08.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa6637fe700 1 -- 172.21.15.32:0/164936393 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa658009650 con 0x7fa6640aed90 2020-12-05T17:32:08.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa6637fe700 1 -- 172.21.15.32:0/164936393 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6580097b0 con 0x7fa6640aed90 2020-12-05T17:32:08.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa6637fe700 1 -- 172.21.15.32:0/164936393 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa658009b10 con 0x7fa6640aed90 2020-12-05T17:32:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa669f0e700 1 -- 172.21.15.32:0/164936393 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa6640aed90 msgr2=0x7fa664140db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa669f0e700 1 --2- 172.21.15.32:0/164936393 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa6640aed90 0x7fa664140db0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fa658002f30 tx=0x7fa658006dc0).stop 2020-12-05T17:32:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa669f0e700 1 -- 172.21.15.32:0/164936393 shutdown_connections 2020-12-05T17:32:08.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa669f0e700 1 --2- 172.21.15.32:0/164936393 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa664129180 0x7fa664129610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa669f0e700 1 --2- 172.21.15.32:0/164936393 >> v2:172.21.15.32:3300/0 conn(0x7fa6640b2000 0x7fa6640af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa669f0e700 1 --2- 172.21.15.32:0/164936393 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa6640aed90 0x7fa664140db0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa669f0e700 1 -- 172.21.15.32:0/164936393 >> 172.21.15.32:0/164936393 conn(0x7fa6640b5620 msgr2=0x7fa6640b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:08.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa669f0e700 1 -- 172.21.15.32:0/164936393 shutdown_connections 2020-12-05T17:32:08.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.739+0000 7fa669f0e700 1 -- 172.21.15.32:0/164936393 wait complete. 2020-12-05T17:32:08.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa669f0e700 1 Processor -- start 2020-12-05T17:32:08.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa669f0e700 1 -- start start 2020-12-05T17:32:08.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa669f0e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6640aed90 0x7fa66414f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:08.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa669f0e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6640b2000 0x7fa6640afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:08.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa669f0e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa664129180 0x7fa664152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:08.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa669f0e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6641422d0 con 0x7fa6640b2000 2020-12-05T17:32:08.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa669f0e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa664152bc0 con 0x7fa664129180 2020-12-05T17:32:08.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa669f0e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa664065b60 con 0x7fa6640aed90 2020-12-05T17:32:08.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa663fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6640b2000 0x7fa6640afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:08.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa663fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6640b2000 0x7fa6640afd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33370/0 (socket says 172.21.15.32:33370) 2020-12-05T17:32:08.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa663fff700 1 -- 172.21.15.32:0/3331997909 learned_addr learned my addr 172.21.15.32:0/3331997909 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:08.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa66970d700 1 --2- 172.21.15.32:0/3331997909 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa664129180 0x7fa664152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:08.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa668f0c700 1 --2- 172.21.15.32:0/3331997909 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6640aed90 0x7fa66414f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:08.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa663fff700 1 -- 172.21.15.32:0/3331997909 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6640aed90 msgr2=0x7fa66414f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:08.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa663fff700 1 --2- 172.21.15.32:0/3331997909 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6640aed90 0x7fa66414f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa663fff700 1 -- 172.21.15.32:0/3331997909 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa664129180 msgr2=0x7fa664152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:08.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa663fff700 1 --2- 172.21.15.32:0/3331997909 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa664129180 0x7fa664152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa663fff700 1 -- 172.21.15.32:0/3331997909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6640b0230 con 0x7fa6640b2000 2020-12-05T17:32:08.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa663fff700 1 --2- 172.21.15.32:0/3331997909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6640b2000 0x7fa6640afd40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fa658002f30 tx=0x7fa658015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:08.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa661ffb700 1 -- 172.21.15.32:0/3331997909 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa658011c30 con 0x7fa6640b2000 2020-12-05T17:32:08.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa661ffb700 1 -- 172.21.15.32:0/3331997909 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa658011d90 con 0x7fa6640b2000 2020-12-05T17:32:08.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa661ffb700 1 -- 172.21.15.32:0/3331997909 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa65801d640 con 0x7fa6640b2000 2020-12-05T17:32:08.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa669f0e700 1 -- 172.21.15.32:0/3331997909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa664065e00 con 0x7fa6640b2000 2020-12-05T17:32:08.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.743+0000 7fa669f0e700 1 -- 172.21.15.32:0/3331997909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa664126110 con 0x7fa6640b2000 2020-12-05T17:32:08.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.747+0000 7fa661ffb700 1 -- 172.21.15.32:0/3331997909 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa65800f070 con 0x7fa6640b2000 2020-12-05T17:32:08.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.747+0000 7fa669f0e700 1 -- 172.21.15.32:0/3331997909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6641263f0 con 0x7fa6640b2000 2020-12-05T17:32:08.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.747+0000 7fa661ffb700 1 --2- 172.21.15.32:0/3331997909 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa6440554e0 0x7fa6440579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.747+0000 7fa661ffb700 1 -- 172.21.15.32:0/3331997909 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa65806ee90 con 0x7fa6640b2000 2020-12-05T17:32:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.747+0000 7fa668f0c700 1 --2- 172.21.15.32:0/3331997909 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa6440554e0 0x7fa6440579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.747+0000 7fa668f0c700 1 --2- 172.21.15.32:0/3331997909 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa6440554e0 0x7fa6440579a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fa65400b000 tx=0x7fa654009270).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.751+0000 7fa661ffb700 1 -- 172.21.15.32:0/3331997909 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa658044f20 con 0x7fa6640b2000 2020-12-05T17:32:08.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.891+0000 7fa669f0e700 1 -- 172.21.15.32:0/3331997909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa664057050 con 0x7fa6640b2000 2020-12-05T17:32:08.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.895+0000 7fa661ffb700 1 -- 172.21.15.32:0/3331997909 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7fa658070020 con 0x7fa6640b2000 2020-12-05T17:32:08.897 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:32:08.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.895+0000 7fa669f0e700 1 -- 172.21.15.32:0/3331997909 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa6440554e0 msgr2=0x7fa6440579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:08.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.895+0000 7fa669f0e700 1 --2- 172.21.15.32:0/3331997909 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa6440554e0 0x7fa6440579a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fa65400b000 tx=0x7fa654009270).stop 2020-12-05T17:32:08.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.895+0000 7fa669f0e700 1 -- 172.21.15.32:0/3331997909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6640b2000 msgr2=0x7fa6640afd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:08.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.895+0000 7fa669f0e700 1 --2- 172.21.15.32:0/3331997909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6640b2000 0x7fa6640afd40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fa658002f30 tx=0x7fa658015000).stop 2020-12-05T17:32:08.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.895+0000 7fa669f0e700 1 -- 172.21.15.32:0/3331997909 shutdown_connections 2020-12-05T17:32:08.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.895+0000 7fa669f0e700 1 --2- 172.21.15.32:0/3331997909 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa664129180 0x7fa664152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.895+0000 7fa669f0e700 1 --2- 172.21.15.32:0/3331997909 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa6440554e0 0x7fa6440579a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.895+0000 7fa669f0e700 1 --2- 172.21.15.32:0/3331997909 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6640aed90 0x7fa66414f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.895+0000 7fa669f0e700 1 --2- 172.21.15.32:0/3331997909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6640b2000 0x7fa6640afd40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.895+0000 7fa669f0e700 1 -- 172.21.15.32:0/3331997909 >> 172.21.15.32:0/3331997909 conn(0x7fa6640b5620 msgr2=0x7fa6640b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.895+0000 7fa669f0e700 1 -- 172.21.15.32:0/3331997909 shutdown_connections 2020-12-05T17:32:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:08.895+0000 7fa669f0e700 1 -- 172.21.15.32:0/3331997909 wait complete. 2020-12-05T17:32:08.911 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:32:09.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.023+0000 7f32d9bf5700 1 Processor -- start 2020-12-05T17:32:09.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.023+0000 7f32d9bf5700 1 -- start start 2020-12-05T17:32:09.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.023+0000 7f32d9bf5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f32d405fe00 0x7f32d40b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:09.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.023+0000 7f32d9bf5700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f32d413c730 0x7f32d413cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:09.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.023+0000 7f32d9bf5700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d40601e0 con 0x7f32d4142d00 2020-12-05T17:32:09.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.023+0000 7f32d9bf5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d4061480 con 0x7f32d405fe00 2020-12-05T17:32:09.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.023+0000 7f32d9bf5700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d40ad510 con 0x7f32d413c730 2020-12-05T17:32:09.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.023+0000 7f32d37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f32d405fe00 0x7f32d40b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:09.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.023+0000 7f32d37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f32d405fe00 0x7f32d40b0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33376/0 (socket says 172.21.15.32:33376) 2020-12-05T17:32:09.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.023+0000 7f32d2ffd700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f32d413c730 0x7f32d413cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:09.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.023+0000 7f32d3fff700 1 --1- 172.21.15.32:0/2750833613 >> v1:172.21.15.61:6789/0 conn(0x7f32d4142d00 0x7f32d4143140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:48666/0 (socket says 172.21.15.32:48666) 2020-12-05T17:32:09.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.023+0000 7f32d37fe700 1 -- 172.21.15.32:0/2750833613 learned_addr learned my addr 172.21.15.32:0/2750833613 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:09.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.023+0000 7f32d37fe700 1 -- 172.21.15.32:0/2750833613 >> v2:172.21.15.61:3300/0 conn(0x7f32d413c730 msgr2=0x7f32d413cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:09.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.023+0000 7f32d37fe700 1 --2- 172.21.15.32:0/2750833613 >> v2:172.21.15.61:3300/0 conn(0x7f32d413c730 0x7f32d413cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.023+0000 7f32d37fe700 1 -- 172.21.15.32:0/2750833613 >> v1:172.21.15.61:6789/0 conn(0x7f32d4142d00 legacy=0x7f32d4143140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:09.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.023+0000 7f32d27fc700 1 -- 172.21.15.32:0/2750833613 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477628942 0 0) 0x7f32d40601e0 con 0x7f32d4142d00 2020-12-05T17:32:09.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.023+0000 7f32d37fe700 1 -- 172.21.15.32:0/2750833613 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d40af250 con 0x7f32d405fe00 2020-12-05T17:32:09.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.023+0000 7f32d37fe700 1 --2- 172.21.15.32:0/2750833613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32d405fe00 0x7f32d40b0f40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f32c40032b0 tx=0x7f32c4009270).ready entity=mon.0 client_cookie=5deac7974a019f3a server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:09.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.023+0000 7f32d27fc700 1 -- 172.21.15.32:0/2750833613 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32c4009e60 con 0x7f32d405fe00 2020-12-05T17:32:09.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.023+0000 7f32d27fc700 1 -- 172.21.15.32:0/2750833613 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c400bc00 con 0x7f32d405fe00 2020-12-05T17:32:09.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.023+0000 7f32d27fc700 1 -- 172.21.15.32:0/2750833613 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32c400bd70 con 0x7f32d405fe00 2020-12-05T17:32:09.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.023+0000 7f32d9bf5700 1 -- 172.21.15.32:0/2750833613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32d405fe00 msgr2=0x7f32d40b0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:09.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.023+0000 7f32d9bf5700 1 --2- 172.21.15.32:0/2750833613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32d405fe00 0x7f32d40b0f40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f32c40032b0 tx=0x7f32c4009270).stop 2020-12-05T17:32:09.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.027+0000 7f32d9bf5700 1 -- 172.21.15.32:0/2750833613 shutdown_connections 2020-12-05T17:32:09.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.027+0000 7f32d9bf5700 1 --2- 172.21.15.32:0/2750833613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32d405fe00 0x7f32d40b0f40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.027+0000 7f32d9bf5700 1 --2- 172.21.15.32:0/2750833613 >> v2:172.21.15.61:3300/0 conn(0x7f32d413c730 0x7f32d413cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.027+0000 7f32d9bf5700 1 -- 172.21.15.32:0/2750833613 >> 172.21.15.32:0/2750833613 conn(0x7f32d40b57c0 msgr2=0x7f32d405f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:09.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.027+0000 7f32d9bf5700 1 -- 172.21.15.32:0/2750833613 shutdown_connections 2020-12-05T17:32:09.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.027+0000 7f32d9bf5700 1 -- 172.21.15.32:0/2750833613 wait complete. 2020-12-05T17:32:09.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.027+0000 7f32d9bf5700 1 Processor -- start 2020-12-05T17:32:09.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.027+0000 7f32d9bf5700 1 -- start start 2020-12-05T17:32:09.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.027+0000 7f32d9bf5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32d405fe00 0x7f32d40b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:09.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.027+0000 7f32d9bf5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32d413c730 0x7f32d413d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:09.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.027+0000 7f32d9bf5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32d4142d00 0x7f32d4154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:09.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.027+0000 7f32d9bf5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d414a6c0 con 0x7f32d413c730 2020-12-05T17:32:09.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.027+0000 7f32d9bf5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d413db40 con 0x7f32d405fe00 2020-12-05T17:32:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.027+0000 7f32d9bf5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d4142680 con 0x7f32d4142d00 2020-12-05T17:32:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.027+0000 7f32d2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32d413c730 0x7f32d413d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.027+0000 7f32d37fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32d405fe00 0x7f32d40b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.027+0000 7f32d3fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32d4142d00 0x7f32d4154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.027+0000 7f32d2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32d413c730 0x7f32d413d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33384/0 (socket says 172.21.15.32:33384) 2020-12-05T17:32:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.027+0000 7f32d2ffd700 1 -- 172.21.15.32:0/834358552 learned_addr learned my addr 172.21.15.32:0/834358552 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.031+0000 7f32d2ffd700 1 -- 172.21.15.32:0/834358552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32d4142d00 msgr2=0x7f32d4154f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.031+0000 7f32d2ffd700 1 --2- 172.21.15.32:0/834358552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32d4142d00 0x7f32d4154f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.031+0000 7f32d2ffd700 1 -- 172.21.15.32:0/834358552 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32d405fe00 msgr2=0x7f32d40b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.031+0000 7f32d2ffd700 1 --2- 172.21.15.32:0/834358552 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32d405fe00 0x7f32d40b1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.031+0000 7f32d2ffd700 1 -- 172.21.15.32:0/834358552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d4142920 con 0x7f32d413c730 2020-12-05T17:32:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.031+0000 7f32d2ffd700 1 --2- 172.21.15.32:0/834358552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32d413c730 0x7f32d413d650 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f32bc00c000 tx=0x7f32bc006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:09.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.031+0000 7f32d0ff9700 1 -- 172.21.15.32:0/834358552 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32bc00ca10 con 0x7f32d413c730 2020-12-05T17:32:09.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.031+0000 7f32d0ff9700 1 -- 172.21.15.32:0/834358552 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32bc00cb70 con 0x7f32d413c730 2020-12-05T17:32:09.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.031+0000 7f32d9bf5700 1 -- 172.21.15.32:0/834358552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d4149850 con 0x7f32d413c730 2020-12-05T17:32:09.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.031+0000 7f32d0ff9700 1 -- 172.21.15.32:0/834358552 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32bc00ee80 con 0x7f32d413c730 2020-12-05T17:32:09.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.031+0000 7f32d9bf5700 1 -- 172.21.15.32:0/834358552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d4155450 con 0x7f32d413c730 2020-12-05T17:32:09.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.031+0000 7f32d9bf5700 1 -- 172.21.15.32:0/834358552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d4142ab0 con 0x7f32d413c730 2020-12-05T17:32:09.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.031+0000 7f32d0ff9700 1 -- 172.21.15.32:0/834358552 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f32bc01a030 con 0x7f32d413c730 2020-12-05T17:32:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.035+0000 7f32d0ff9700 1 --2- 172.21.15.32:0/834358552 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f32b80554e0 0x7f32b80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.035+0000 7f32d37fe700 1 --2- 172.21.15.32:0/834358552 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f32b80554e0 0x7f32b80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.035+0000 7f32d0ff9700 1 -- 172.21.15.32:0/834358552 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f32bc06df50 con 0x7f32d413c730 2020-12-05T17:32:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.035+0000 7f32d37fe700 1 --2- 172.21.15.32:0/834358552 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f32b80554e0 0x7f32b80579a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f32c401a000 tx=0x7f32c4016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.039+0000 7f32d0ff9700 1 -- 172.21.15.32:0/834358552 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f32bc0440f0 con 0x7f32d413c730 2020-12-05T17:32:09.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.183+0000 7f32d9bf5700 1 -- 172.21.15.32:0/834358552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32d4057050 con 0x7f32d413c730 2020-12-05T17:32:09.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.183+0000 7f32d0ff9700 1 -- 172.21.15.32:0/834358552 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f32bc047710 con 0x7f32d413c730 2020-12-05T17:32:09.188 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:32:09.188 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:31:41.610914+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:31:36.706361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-12-05T17:31:39.298269+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:32:09.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.187+0000 7f32d9bf5700 1 -- 172.21.15.32:0/834358552 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f32b80554e0 msgr2=0x7f32b80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:09.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.187+0000 7f32d9bf5700 1 --2- 172.21.15.32:0/834358552 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f32b80554e0 0x7f32b80579a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f32c401a000 tx=0x7f32c4016000).stop 2020-12-05T17:32:09.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.187+0000 7f32d9bf5700 1 -- 172.21.15.32:0/834358552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32d413c730 msgr2=0x7f32d413d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:09.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.187+0000 7f32d9bf5700 1 --2- 172.21.15.32:0/834358552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32d413c730 0x7f32d413d650 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f32bc00c000 tx=0x7f32bc006dc0).stop 2020-12-05T17:32:09.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.187+0000 7f32d9bf5700 1 -- 172.21.15.32:0/834358552 shutdown_connections 2020-12-05T17:32:09.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.187+0000 7f32d9bf5700 1 --2- 172.21.15.32:0/834358552 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f32b80554e0 0x7f32b80579a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.187+0000 7f32d9bf5700 1 --2- 172.21.15.32:0/834358552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32d4142d00 0x7f32d4154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.187+0000 7f32d9bf5700 1 --2- 172.21.15.32:0/834358552 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32d405fe00 0x7f32d40b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.187+0000 7f32d9bf5700 1 --2- 172.21.15.32:0/834358552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32d413c730 0x7f32d413d650 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.187+0000 7f32d9bf5700 1 -- 172.21.15.32:0/834358552 >> 172.21.15.32:0/834358552 conn(0x7f32d40b57c0 msgr2=0x7f32d40af890 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:09.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.187+0000 7f32d9bf5700 1 -- 172.21.15.32:0/834358552 shutdown_connections 2020-12-05T17:32:09.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.187+0000 7f32d9bf5700 1 -- 172.21.15.32:0/834358552 wait complete. 2020-12-05T17:32:09.202 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:31:36.706361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:31:39.298269+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:32:09.203 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:32:09.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.323+0000 7f9a9b61c700 1 Processor -- start 2020-12-05T17:32:09.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.323+0000 7f9a9b61c700 1 -- start start 2020-12-05T17:32:09.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.323+0000 7f9a9b61c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a940666a0 0x7f9a94144600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:09.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.323+0000 7f9a9b61c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a940627a0 con 0x7f9a940b5730 2020-12-05T17:32:09.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.323+0000 7f9a9b61c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a940b5b10 con 0x7f9a94061bb0 2020-12-05T17:32:09.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.323+0000 7f9a9b61c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a940b04c0 con 0x7f9a940666a0 2020-12-05T17:32:09.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.323+0000 7f9a993b8700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9a940b5730 0x7f9a94143ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36036/0 (socket says 172.21.15.32:36036) 2020-12-05T17:32:09.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.323+0000 7f9a993b8700 1 -- 172.21.15.32:0/3769945926 learned_addr learned my addr 172.21.15.32:0/3769945926 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:09.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.323+0000 7f9a99bb9700 1 --2- 172.21.15.32:0/3769945926 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a940666a0 0x7f9a94144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.323+0000 7f9a8bfff700 1 -- 172.21.15.32:0/3769945926 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 832219521 0 0) 0x7f9a940627a0 con 0x7f9a940b5730 2020-12-05T17:32:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.323+0000 7f9a8bfff700 1 -- 172.21.15.32:0/3769945926 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a78003230 con 0x7f9a940b5730 2020-12-05T17:32:09.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.323+0000 7f9a8bfff700 1 -- 172.21.15.32:0/3769945926 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3972251052 0 0) 0x7f9a940b5b10 con 0x7f9a94061bb0 2020-12-05T17:32:09.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.323+0000 7f9a8bfff700 1 -- 172.21.15.32:0/3769945926 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a940627a0 con 0x7f9a94061bb0 2020-12-05T17:32:09.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.323+0000 7f9a8bfff700 1 -- 172.21.15.32:0/3769945926 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1033731072 0 0) 0x7f9a78003230 con 0x7f9a940b5730 2020-12-05T17:32:09.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.323+0000 7f9a99bb9700 1 -- 172.21.15.32:0/3769945926 >> v1:172.21.15.61:6789/0 conn(0x7f9a94061bb0 legacy=0x7f9a94140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:09.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.323+0000 7f9a99bb9700 1 -- 172.21.15.32:0/3769945926 >> v1:172.21.15.32:6789/0 conn(0x7f9a940b5730 legacy=0x7f9a94143ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:09.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.323+0000 7f9a99bb9700 1 -- 172.21.15.32:0/3769945926 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a94062a00 con 0x7f9a940666a0 2020-12-05T17:32:09.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.327+0000 7f9a99bb9700 1 --2- 172.21.15.32:0/3769945926 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a940666a0 0x7f9a94144600 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f9a900010d0 tx=0x7f9a8c0091a0).ready entity=mon.2 client_cookie=9ab0d969b66280dc server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:09.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.327+0000 7f9a8bfff700 1 -- 172.21.15.32:0/3769945926 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a900016d0 con 0x7f9a940666a0 2020-12-05T17:32:09.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.327+0000 7f9a8bfff700 1 -- 172.21.15.32:0/3769945926 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8c00b800 con 0x7f9a940666a0 2020-12-05T17:32:09.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.327+0000 7f9a8bfff700 1 -- 172.21.15.32:0/3769945926 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a8c00b970 con 0x7f9a940666a0 2020-12-05T17:32:09.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.327+0000 7f9a9b61c700 1 -- 172.21.15.32:0/3769945926 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a940666a0 msgr2=0x7f9a94144600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:09.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.327+0000 7f9a9b61c700 1 --2- 172.21.15.32:0/3769945926 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a940666a0 0x7f9a94144600 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f9a900010d0 tx=0x7f9a8c0091a0).stop 2020-12-05T17:32:09.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.327+0000 7f9a9b61c700 1 -- 172.21.15.32:0/3769945926 shutdown_connections 2020-12-05T17:32:09.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.327+0000 7f9a9b61c700 1 --2- 172.21.15.32:0/3769945926 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a940666a0 0x7f9a94144600 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.327+0000 7f9a9b61c700 1 -- 172.21.15.32:0/3769945926 >> 172.21.15.32:0/3769945926 conn(0x7f9a9405fe80 msgr2=0x7f9a940b4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:09.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.327+0000 7f9a9b61c700 1 -- 172.21.15.32:0/3769945926 shutdown_connections 2020-12-05T17:32:09.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.327+0000 7f9a9b61c700 1 -- 172.21.15.32:0/3769945926 wait complete. 2020-12-05T17:32:09.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.327+0000 7f9a9b61c700 1 Processor -- start 2020-12-05T17:32:09.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.327+0000 7f9a9b61c700 1 -- start start 2020-12-05T17:32:09.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.331+0000 7f9a9b61c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a940666a0 0x7f9a9409e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:09.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.331+0000 7f9a9b61c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a940b5730 0x7f9a940aad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:09.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.331+0000 7f9a9b61c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a94146820 0x7f9a940ab280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:09.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.331+0000 7f9a9b61c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a940b2f40 con 0x7f9a940b5730 2020-12-05T17:32:09.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.331+0000 7f9a9b61c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a940b3170 con 0x7f9a94146820 2020-12-05T17:32:09.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.331+0000 7f9a9b61c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a940b3410 con 0x7f9a940666a0 2020-12-05T17:32:09.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.331+0000 7f9a98bb7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a940b5730 0x7f9a940aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:09.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.331+0000 7f9a98bb7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a940b5730 0x7f9a940aad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33396/0 (socket says 172.21.15.32:33396) 2020-12-05T17:32:09.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.331+0000 7f9a98bb7700 1 -- 172.21.15.32:0/4259314213 learned_addr learned my addr 172.21.15.32:0/4259314213 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:09.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.331+0000 7f9a993b8700 1 --2- 172.21.15.32:0/4259314213 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a940666a0 0x7f9a9409e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:09.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.331+0000 7f9a99bb9700 1 --2- 172.21.15.32:0/4259314213 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a94146820 0x7f9a940ab280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:09.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.331+0000 7f9a98bb7700 1 -- 172.21.15.32:0/4259314213 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a940666a0 msgr2=0x7f9a9409e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:09.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.331+0000 7f9a98bb7700 1 --2- 172.21.15.32:0/4259314213 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a940666a0 0x7f9a9409e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.331+0000 7f9a98bb7700 1 -- 172.21.15.32:0/4259314213 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a94146820 msgr2=0x7f9a940ab280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:09.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.331+0000 7f9a98bb7700 1 --2- 172.21.15.32:0/4259314213 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a94146820 0x7f9a940ab280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.331+0000 7f9a98bb7700 1 -- 172.21.15.32:0/4259314213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a940ab770 con 0x7f9a940b5730 2020-12-05T17:32:09.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.331+0000 7f9a98bb7700 1 --2- 172.21.15.32:0/4259314213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a940b5730 0x7f9a940aad90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f9a840010d0 tx=0x7f9a84007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:09.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.331+0000 7f9a8a7fc700 1 -- 172.21.15.32:0/4259314213 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a8400e4a0 con 0x7f9a940b5730 2020-12-05T17:32:09.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.331+0000 7f9a8a7fc700 1 -- 172.21.15.32:0/4259314213 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8400e600 con 0x7f9a940b5730 2020-12-05T17:32:09.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.331+0000 7f9a9b61c700 1 -- 172.21.15.32:0/4259314213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a94144740 con 0x7f9a940b5730 2020-12-05T17:32:09.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.331+0000 7f9a8a7fc700 1 -- 172.21.15.32:0/4259314213 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a8400e960 con 0x7f9a940b5730 2020-12-05T17:32:09.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.331+0000 7f9a9b61c700 1 -- 172.21.15.32:0/4259314213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a941443d0 con 0x7f9a940b5730 2020-12-05T17:32:09.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.335+0000 7f9a8a7fc700 1 -- 172.21.15.32:0/4259314213 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9a84010070 con 0x7f9a940b5730 2020-12-05T17:32:09.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.335+0000 7f9a9b61c700 1 -- 172.21.15.32:0/4259314213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a940b35d0 con 0x7f9a940b5730 2020-12-05T17:32:09.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.335+0000 7f9a8a7fc700 1 --2- 172.21.15.32:0/4259314213 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9a7c055330 0x7f9a7c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:09.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.335+0000 7f9a993b8700 1 --2- 172.21.15.32:0/4259314213 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9a7c055330 0x7f9a7c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:09.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.335+0000 7f9a8a7fc700 1 -- 172.21.15.32:0/4259314213 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f9a8406d3d0 con 0x7f9a940b5730 2020-12-05T17:32:09.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.335+0000 7f9a993b8700 1 --2- 172.21.15.32:0/4259314213 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9a7c055330 0x7f9a7c0577f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f9a9000d000 tx=0x7f9a90000b80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:09.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.339+0000 7f9a8a7fc700 1 -- 172.21.15.32:0/4259314213 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9a84043460 con 0x7f9a940b5730 2020-12-05T17:32:09.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.479+0000 7f9a9b61c700 1 -- 172.21.15.32:0/4259314213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9a94057050 con 0x7f9a940b5730 2020-12-05T17:32:09.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.479+0000 7f9a8a7fc700 1 -- 172.21.15.32:0/4259314213 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9a84046a80 con 0x7f9a940b5730 2020-12-05T17:32:09.484 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:32:09.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.483+0000 7f9a9b61c700 1 -- 172.21.15.32:0/4259314213 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9a7c055330 msgr2=0x7f9a7c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:09.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.483+0000 7f9a9b61c700 1 --2- 172.21.15.32:0/4259314213 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9a7c055330 0x7f9a7c0577f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f9a9000d000 tx=0x7f9a90000b80).stop 2020-12-05T17:32:09.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.483+0000 7f9a9b61c700 1 -- 172.21.15.32:0/4259314213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a940b5730 msgr2=0x7f9a940aad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:09.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.483+0000 7f9a9b61c700 1 --2- 172.21.15.32:0/4259314213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a940b5730 0x7f9a940aad90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f9a840010d0 tx=0x7f9a84007ad0).stop 2020-12-05T17:32:09.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.483+0000 7f9a9b61c700 1 -- 172.21.15.32:0/4259314213 shutdown_connections 2020-12-05T17:32:09.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.483+0000 7f9a9b61c700 1 --2- 172.21.15.32:0/4259314213 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a94146820 0x7f9a940ab280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.483+0000 7f9a9b61c700 1 --2- 172.21.15.32:0/4259314213 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9a7c055330 0x7f9a7c0577f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.483+0000 7f9a9b61c700 1 --2- 172.21.15.32:0/4259314213 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a940666a0 0x7f9a9409e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.483+0000 7f9a9b61c700 1 --2- 172.21.15.32:0/4259314213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a940b5730 0x7f9a940aad90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.483+0000 7f9a9b61c700 1 -- 172.21.15.32:0/4259314213 >> 172.21.15.32:0/4259314213 conn(0x7f9a9405fe80 msgr2=0x7f9a94065900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:09.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.483+0000 7f9a9b61c700 1 -- 172.21.15.32:0/4259314213 shutdown_connections 2020-12-05T17:32:09.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.483+0000 7f9a9b61c700 1 -- 172.21.15.32:0/4259314213 wait complete. 2020-12-05T17:32:09.497 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:32:09.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.611+0000 7f5b61203700 1 Processor -- start 2020-12-05T17:32:09.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.611+0000 7f5b61203700 1 -- start start 2020-12-05T17:32:09.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.611+0000 7f5b61203700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b5c0af5c0 0x7f5b5c0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:09.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.611+0000 7f5b61203700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f5b5c143f00 0x7f5b5c1443b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:09.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.611+0000 7f5b61203700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b5c1448a0 0x7f5b5c05f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:09.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.611+0000 7f5b61203700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b5c1405f0 con 0x7f5b5c0af5c0 2020-12-05T17:32:09.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.611+0000 7f5b61203700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b5c1407e0 con 0x7f5b5c143f00 2020-12-05T17:32:09.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.611+0000 7f5b61203700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c061b30 con 0x7f5b5c1448a0 2020-12-05T17:32:09.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.611+0000 7f5b5bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b5c0af5c0 0x7f5b5c0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:09.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.611+0000 7f5b60a02700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b5c1448a0 0x7f5b5c05f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:09.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.611+0000 7f5b5bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b5c0af5c0 0x7f5b5c0af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33404/0 (socket says 172.21.15.32:33404) 2020-12-05T17:32:09.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.611+0000 7f5b60a02700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b5c1448a0 0x7f5b5c05f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:58428/0 (socket says 172.21.15.32:58428) 2020-12-05T17:32:09.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.611+0000 7f5b537fe700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f5b5c143f00 0x7f5b5c1443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:09.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.611+0000 7f5b5bfff700 1 -- 172.21.15.32:0/412633632 learned_addr learned my addr 172.21.15.32:0/412633632 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:09.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.611+0000 7f5b5bfff700 1 -- 172.21.15.32:0/412633632 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b5c1448a0 msgr2=0x7f5b5c05f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:09.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.611+0000 7f5b5bfff700 1 --2- 172.21.15.32:0/412633632 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b5c1448a0 0x7f5b5c05f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.611+0000 7f5b5bfff700 1 -- 172.21.15.32:0/412633632 >> v2:172.21.15.61:3300/0 conn(0x7f5b5c143f00 msgr2=0x7f5b5c1443b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:09.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.611+0000 7f5b5bfff700 1 --2- 172.21.15.32:0/412633632 >> v2:172.21.15.61:3300/0 conn(0x7f5b5c143f00 0x7f5b5c1443b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.611+0000 7f5b5bfff700 1 -- 172.21.15.32:0/412633632 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5c0b0c70 con 0x7f5b5c0af5c0 2020-12-05T17:32:09.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.615+0000 7f5b5bfff700 1 --2- 172.21.15.32:0/412633632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b5c0af5c0 0x7f5b5c0af9a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f5b4c002f30 tx=0x7f5b4c006e20).ready entity=mon.0 client_cookie=3e68c42e26ad4442 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:09.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.615+0000 7f5b5b7fe700 1 -- 172.21.15.32:0/412633632 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b4c009710 con 0x7f5b5c0af5c0 2020-12-05T17:32:09.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.615+0000 7f5b5b7fe700 1 -- 172.21.15.32:0/412633632 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4c009870 con 0x7f5b5c0af5c0 2020-12-05T17:32:09.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.615+0000 7f5b5b7fe700 1 -- 172.21.15.32:0/412633632 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b4c009bd0 con 0x7f5b5c0af5c0 2020-12-05T17:32:09.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.615+0000 7f5b61203700 1 -- 172.21.15.32:0/412633632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b5c0af5c0 msgr2=0x7f5b5c0af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:09.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.615+0000 7f5b61203700 1 --2- 172.21.15.32:0/412633632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b5c0af5c0 0x7f5b5c0af9a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f5b4c002f30 tx=0x7f5b4c006e20).stop 2020-12-05T17:32:09.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.615+0000 7f5b61203700 1 -- 172.21.15.32:0/412633632 shutdown_connections 2020-12-05T17:32:09.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.615+0000 7f5b61203700 1 --2- 172.21.15.32:0/412633632 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b5c1448a0 0x7f5b5c05f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.615+0000 7f5b61203700 1 --2- 172.21.15.32:0/412633632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b5c0af5c0 0x7f5b5c0af9a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.615+0000 7f5b61203700 1 --2- 172.21.15.32:0/412633632 >> v2:172.21.15.61:3300/0 conn(0x7f5b5c143f00 0x7f5b5c1443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.615+0000 7f5b61203700 1 -- 172.21.15.32:0/412633632 >> 172.21.15.32:0/412633632 conn(0x7f5b5c061010 msgr2=0x7f5b5c061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:09.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.615+0000 7f5b61203700 1 -- 172.21.15.32:0/412633632 shutdown_connections 2020-12-05T17:32:09.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.615+0000 7f5b61203700 1 -- 172.21.15.32:0/412633632 wait complete. 2020-12-05T17:32:09.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.615+0000 7f5b61203700 1 Processor -- start 2020-12-05T17:32:09.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.615+0000 7f5b61203700 1 -- start start 2020-12-05T17:32:09.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.619+0000 7f5b61203700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b5c0af5c0 0x7f5b5c066250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:09.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.619+0000 7f5b61203700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b5c143f00 0x7f5b5c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:09.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.619+0000 7f5b61203700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5b5c1448a0 0x7f5b5c13fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:09.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.619+0000 7f5b61203700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c136620 con 0x7f5b5c143f00 2020-12-05T17:32:09.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.619+0000 7f5b61203700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c0b5810 con 0x7f5b5c1448a0 2020-12-05T17:32:09.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.619+0000 7f5b61203700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c0648f0 con 0x7f5b5c0af5c0 2020-12-05T17:32:09.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.619+0000 7f5b537fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b5c143f00 0x7f5b5c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:09.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.619+0000 7f5b537fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b5c143f00 0x7f5b5c0b6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33410/0 (socket says 172.21.15.32:33410) 2020-12-05T17:32:09.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.619+0000 7f5b537fe700 1 -- 172.21.15.32:0/1407501326 learned_addr learned my addr 172.21.15.32:0/1407501326 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:09.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.619+0000 7f5b5bfff700 1 --2- 172.21.15.32:0/1407501326 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b5c0af5c0 0x7f5b5c066250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:09.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.619+0000 7f5b60a02700 1 --2- 172.21.15.32:0/1407501326 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5b5c1448a0 0x7f5b5c13fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:09.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.619+0000 7f5b537fe700 1 -- 172.21.15.32:0/1407501326 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b5c0af5c0 msgr2=0x7f5b5c066250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:09.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.619+0000 7f5b537fe700 1 --2- 172.21.15.32:0/1407501326 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b5c0af5c0 0x7f5b5c066250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.619+0000 7f5b537fe700 1 -- 172.21.15.32:0/1407501326 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5b5c1448a0 msgr2=0x7f5b5c13fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:09.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.619+0000 7f5b537fe700 1 --2- 172.21.15.32:0/1407501326 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5b5c1448a0 0x7f5b5c13fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.619+0000 7f5b537fe700 1 -- 172.21.15.32:0/1407501326 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5c064b90 con 0x7f5b5c143f00 2020-12-05T17:32:09.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.619+0000 7f5b537fe700 1 --2- 172.21.15.32:0/1407501326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b5c143f00 0x7f5b5c0b6200 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f5b4400c000 tx=0x7f5b44006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:09.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.619+0000 7f5b59ffb700 1 -- 172.21.15.32:0/1407501326 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b4400ca10 con 0x7f5b5c143f00 2020-12-05T17:32:09.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.619+0000 7f5b59ffb700 1 -- 172.21.15.32:0/1407501326 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4400cb70 con 0x7f5b5c143f00 2020-12-05T17:32:09.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.619+0000 7f5b61203700 1 -- 172.21.15.32:0/1407501326 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5c066740 con 0x7f5b5c143f00 2020-12-05T17:32:09.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.619+0000 7f5b59ffb700 1 -- 172.21.15.32:0/1407501326 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b440123f0 con 0x7f5b5c143f00 2020-12-05T17:32:09.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.619+0000 7f5b61203700 1 -- 172.21.15.32:0/1407501326 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5c0b6890 con 0x7f5b5c143f00 2020-12-05T17:32:09.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.619+0000 7f5b61203700 1 -- 172.21.15.32:0/1407501326 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b5c064750 con 0x7f5b5c143f00 2020-12-05T17:32:09.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.623+0000 7f5b59ffb700 1 -- 172.21.15.32:0/1407501326 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5b44012550 con 0x7f5b5c143f00 2020-12-05T17:32:09.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.623+0000 7f5b59ffb700 1 --2- 172.21.15.32:0/1407501326 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b40055470 0x7f5b40057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:09.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.623+0000 7f5b59ffb700 1 -- 172.21.15.32:0/1407501326 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5b4406e370 con 0x7f5b5c143f00 2020-12-05T17:32:09.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.623+0000 7f5b5bfff700 1 --2- 172.21.15.32:0/1407501326 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b40055470 0x7f5b40057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:09.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.623+0000 7f5b5bfff700 1 --2- 172.21.15.32:0/1407501326 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b40055470 0x7f5b40057930 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f5b4c01a000 tx=0x7f5b4c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:09.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.631+0000 7f5b59ffb700 1 -- 172.21.15.32:0/1407501326 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5b5c064750 con 0x7f5b5c143f00 2020-12-05T17:32:09.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.771+0000 7f5b61203700 1 -- 172.21.15.32:0/1407501326 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5b5c064750 con 0x7f5b5c143f00 2020-12-05T17:32:09.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.771+0000 7f5b59ffb700 1 -- 172.21.15.32:0/1407501326 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5b5c064750 con 0x7f5b5c143f00 2020-12-05T17:32:09.776 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:32:09.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.775+0000 7f5b61203700 1 -- 172.21.15.32:0/1407501326 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b40055470 msgr2=0x7f5b40057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:09.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.775+0000 7f5b61203700 1 --2- 172.21.15.32:0/1407501326 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b40055470 0x7f5b40057930 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f5b4c01a000 tx=0x7f5b4c015000).stop 2020-12-05T17:32:09.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.775+0000 7f5b61203700 1 -- 172.21.15.32:0/1407501326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b5c143f00 msgr2=0x7f5b5c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:09.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.775+0000 7f5b61203700 1 --2- 172.21.15.32:0/1407501326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b5c143f00 0x7f5b5c0b6200 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f5b4400c000 tx=0x7f5b44006dc0).stop 2020-12-05T17:32:09.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.775+0000 7f5b61203700 1 -- 172.21.15.32:0/1407501326 shutdown_connections 2020-12-05T17:32:09.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.775+0000 7f5b61203700 1 --2- 172.21.15.32:0/1407501326 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5b5c1448a0 0x7f5b5c13fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.775+0000 7f5b61203700 1 --2- 172.21.15.32:0/1407501326 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b40055470 0x7f5b40057930 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.775+0000 7f5b61203700 1 --2- 172.21.15.32:0/1407501326 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b5c0af5c0 0x7f5b5c066250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.775+0000 7f5b61203700 1 --2- 172.21.15.32:0/1407501326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b5c143f00 0x7f5b5c0b6200 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.775+0000 7f5b61203700 1 -- 172.21.15.32:0/1407501326 >> 172.21.15.32:0/1407501326 conn(0x7f5b5c061010 msgr2=0x7f5b5c140b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:09.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.775+0000 7f5b61203700 1 -- 172.21.15.32:0/1407501326 shutdown_connections 2020-12-05T17:32:09.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.775+0000 7f5b61203700 1 -- 172.21.15.32:0/1407501326 wait complete. 2020-12-05T17:32:09.789 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:32:09.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.903+0000 7f15d359e700 1 Processor -- start 2020-12-05T17:32:09.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.903+0000 7f15d359e700 1 -- start start 2020-12-05T17:32:09.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f15d40b0d70 0x7f15d40637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:09.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d405f5c0 con 0x7f15d4063ce0 2020-12-05T17:32:09.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d359e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d40af4b0 con 0x7f15d40af0d0 2020-12-05T17:32:09.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d359e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d413e1b0 con 0x7f15d40b0d70 2020-12-05T17:32:09.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d1d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f15d40b0d70 0x7f15d40637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:09.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d2d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f15d4063ce0 0x7f15d414b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36068/0 (socket says 172.21.15.32:36068) 2020-12-05T17:32:09.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d2d9d700 1 -- 172.21.15.32:0/2760516245 learned_addr learned my addr 172.21.15.32:0/2760516245 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:09.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d1d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f15d40b0d70 0x7f15d40637f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33418/0 (socket says 172.21.15.32:33418) 2020-12-05T17:32:09.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d1d9b700 1 -- 172.21.15.32:0/2760516245 >> v1:172.21.15.61:6789/0 conn(0x7f15d40af0d0 legacy=0x7f15d413daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:09.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d159a700 1 -- 172.21.15.32:0/2760516245 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964453396 0 0) 0x7f15d405f5c0 con 0x7f15d4063ce0 2020-12-05T17:32:09.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d1d9b700 1 -- 172.21.15.32:0/2760516245 >> v1:172.21.15.32:6789/0 conn(0x7f15d4063ce0 legacy=0x7f15d414b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:09.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d1d9b700 1 -- 172.21.15.32:0/2760516245 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d405f7f0 con 0x7f15d40b0d70 2020-12-05T17:32:09.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d1d9b700 1 --2- 172.21.15.32:0/2760516245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15d40b0d70 0x7f15d40637f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f15c80032b0 tx=0x7f15c80091a0).ready entity=mon.0 client_cookie=aa2c57bc61b98dbc server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:09.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d159a700 1 -- 172.21.15.32:0/2760516245 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15c800f430 con 0x7f15d40b0d70 2020-12-05T17:32:09.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d159a700 1 -- 172.21.15.32:0/2760516245 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c800e4e0 con 0x7f15d40b0d70 2020-12-05T17:32:09.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d159a700 1 -- 172.21.15.32:0/2760516245 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15c800e840 con 0x7f15d40b0d70 2020-12-05T17:32:09.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d359e700 1 -- 172.21.15.32:0/2760516245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15d40b0d70 msgr2=0x7f15d40637f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:09.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d359e700 1 --2- 172.21.15.32:0/2760516245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15d40b0d70 0x7f15d40637f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f15c80032b0 tx=0x7f15c80091a0).stop 2020-12-05T17:32:09.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d359e700 1 -- 172.21.15.32:0/2760516245 shutdown_connections 2020-12-05T17:32:09.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d359e700 1 --2- 172.21.15.32:0/2760516245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15d40b0d70 0x7f15d40637f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d359e700 1 -- 172.21.15.32:0/2760516245 >> 172.21.15.32:0/2760516245 conn(0x7f15d40620b0 msgr2=0x7f15d4062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:09.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d359e700 1 -- 172.21.15.32:0/2760516245 shutdown_connections 2020-12-05T17:32:09.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d359e700 1 -- 172.21.15.32:0/2760516245 wait complete. 2020-12-05T17:32:09.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d359e700 1 Processor -- start 2020-12-05T17:32:09.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.907+0000 7f15d359e700 1 -- start start 2020-12-05T17:32:09.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.911+0000 7f15d359e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15d4063ce0 0x7f15d40b3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:09.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.911+0000 7f15d359e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15d40af0d0 0x7f15d413bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:09.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.911+0000 7f15d359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15d40b0d70 0x7f15d413bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:09.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.911+0000 7f15d359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d4141cf0 con 0x7f15d40b0d70 2020-12-05T17:32:09.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.911+0000 7f15d359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d4141f90 con 0x7f15d40af0d0 2020-12-05T17:32:09.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.911+0000 7f15d359e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d4142230 con 0x7f15d4063ce0 2020-12-05T17:32:09.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.911+0000 7f15d2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15d40b0d70 0x7f15d413bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:09.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.911+0000 7f15d1d9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15d40af0d0 0x7f15d413bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:09.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.911+0000 7f15d2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15d40b0d70 0x7f15d413bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33426/0 (socket says 172.21.15.32:33426) 2020-12-05T17:32:09.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.911+0000 7f15d2d9d700 1 -- 172.21.15.32:0/2041600328 learned_addr learned my addr 172.21.15.32:0/2041600328 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:09.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.911+0000 7f15d259c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15d4063ce0 0x7f15d40b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:09.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.911+0000 7f15d2d9d700 1 -- 172.21.15.32:0/2041600328 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15d4063ce0 msgr2=0x7f15d40b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:09.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.911+0000 7f15d2d9d700 1 --2- 172.21.15.32:0/2041600328 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15d4063ce0 0x7f15d40b3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.911+0000 7f15d2d9d700 1 -- 172.21.15.32:0/2041600328 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15d40af0d0 msgr2=0x7f15d413bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:09.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.911+0000 7f15d2d9d700 1 --2- 172.21.15.32:0/2041600328 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15d40af0d0 0x7f15d413bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:09.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.911+0000 7f15d2d9d700 1 -- 172.21.15.32:0/2041600328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d413cde0 con 0x7f15d40b0d70 2020-12-05T17:32:09.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.911+0000 7f15d2d9d700 1 --2- 172.21.15.32:0/2041600328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15d40b0d70 0x7f15d413bfd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f15cc000b80 tx=0x7f15cc007a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:09.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.911+0000 7f15c37fe700 1 -- 172.21.15.32:0/2041600328 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15cc0072e0 con 0x7f15d40b0d70 2020-12-05T17:32:09.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.911+0000 7f15c37fe700 1 -- 172.21.15.32:0/2041600328 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15cc007440 con 0x7f15d40b0d70 2020-12-05T17:32:09.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.911+0000 7f15d359e700 1 -- 172.21.15.32:0/2041600328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d414d580 con 0x7f15d40b0d70 2020-12-05T17:32:09.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.911+0000 7f15c37fe700 1 -- 172.21.15.32:0/2041600328 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15cc0057d0 con 0x7f15d40b0d70 2020-12-05T17:32:09.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.911+0000 7f15d359e700 1 -- 172.21.15.32:0/2041600328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d4142780 con 0x7f15d40b0d70 2020-12-05T17:32:09.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.911+0000 7f15d359e700 1 -- 172.21.15.32:0/2041600328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d4057050 con 0x7f15d40b0d70 2020-12-05T17:32:09.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.915+0000 7f15c37fe700 1 -- 172.21.15.32:0/2041600328 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f15cc01a060 con 0x7f15d40b0d70 2020-12-05T17:32:09.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.915+0000 7f15c37fe700 1 --2- 172.21.15.32:0/2041600328 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15b0055440 0x7f15b0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:09.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.915+0000 7f15c37fe700 1 -- 172.21.15.32:0/2041600328 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f15cc06d770 con 0x7f15d40b0d70 2020-12-05T17:32:09.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.915+0000 7f15d259c700 1 --2- 172.21.15.32:0/2041600328 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15b0055440 0x7f15b0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:09.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.915+0000 7f15d259c700 1 --2- 172.21.15.32:0/2041600328 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15b0055440 0x7f15b0057900 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f15c400d000 tx=0x7f15c4007b60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:09.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:09.923+0000 7f15c37fe700 1 -- 172.21.15.32:0/2041600328 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f15cc043800 con 0x7f15d40b0d70 2020-12-05T17:32:10.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.071+0000 7f15d359e700 1 -- 172.21.15.32:0/2041600328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f15d4142780 con 0x7f15d40b0d70 2020-12-05T17:32:10.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.071+0000 7f15c37fe700 1 -- 172.21.15.32:0/2041600328 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7f15d4142780 con 0x7f15d40b0d70 2020-12-05T17:32:10.074 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:32:10.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.071+0000 7f15d359e700 1 -- 172.21.15.32:0/2041600328 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15b0055440 msgr2=0x7f15b0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:10.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.071+0000 7f15d359e700 1 --2- 172.21.15.32:0/2041600328 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15b0055440 0x7f15b0057900 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f15c400d000 tx=0x7f15c4007b60).stop 2020-12-05T17:32:10.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.071+0000 7f15d359e700 1 -- 172.21.15.32:0/2041600328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15d40b0d70 msgr2=0x7f15d413bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:10.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.071+0000 7f15d359e700 1 --2- 172.21.15.32:0/2041600328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15d40b0d70 0x7f15d413bfd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f15cc000b80 tx=0x7f15cc007a50).stop 2020-12-05T17:32:10.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.071+0000 7f15d359e700 1 -- 172.21.15.32:0/2041600328 shutdown_connections 2020-12-05T17:32:10.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.071+0000 7f15d359e700 1 --2- 172.21.15.32:0/2041600328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15d40b0d70 0x7f15d413bfd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.071+0000 7f15d359e700 1 --2- 172.21.15.32:0/2041600328 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15b0055440 0x7f15b0057900 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.071+0000 7f15d359e700 1 --2- 172.21.15.32:0/2041600328 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15d4063ce0 0x7f15d40b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.071+0000 7f15d359e700 1 --2- 172.21.15.32:0/2041600328 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15d40af0d0 0x7f15d413bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.071+0000 7f15d359e700 1 -- 172.21.15.32:0/2041600328 >> 172.21.15.32:0/2041600328 conn(0x7f15d40620b0 msgr2=0x7f15d4062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:10.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.075+0000 7f15d359e700 1 -- 172.21.15.32:0/2041600328 shutdown_connections 2020-12-05T17:32:10.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.075+0000 7f15d359e700 1 -- 172.21.15.32:0/2041600328 wait complete. 2020-12-05T17:32:10.092 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:32:10.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.207+0000 7f0cb6f6b700 1 Processor -- start 2020-12-05T17:32:10.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.207+0000 7f0cb6f6b700 1 -- start start 2020-12-05T17:32:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb6f6b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0cb00b1e60 0x7f0cb00af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb6f6b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0cb01396c0 0x7f0cb013f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb6f6b700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb0143390 con 0x7f0cb0062ac0 2020-12-05T17:32:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb6f6b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb00ae0a0 con 0x7f0cb01396c0 2020-12-05T17:32:10.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb6f6b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb0143610 con 0x7f0cb00b1e60 2020-12-05T17:32:10.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb676a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0cb01396c0 0x7f0cb013f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:10.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb676a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0cb01396c0 0x7f0cb013f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33432/0 (socket says 172.21.15.32:33432) 2020-12-05T17:32:10.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb676a700 1 -- 172.21.15.32:0/1394546694 learned_addr learned my addr 172.21.15.32:0/1394546694 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:10.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb5f69700 1 --2- 172.21.15.32:0/1394546694 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0cb00b1e60 0x7f0cb00af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:10.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb676a700 1 -- 172.21.15.32:0/1394546694 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0cb00b1e60 msgr2=0x7f0cb00af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:10.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb676a700 1 --2- 172.21.15.32:0/1394546694 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0cb00b1e60 0x7f0cb00af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb676a700 1 -- 172.21.15.32:0/1394546694 >> v1:172.21.15.61:6789/0 conn(0x7f0cb0062ac0 legacy=0x7f0cb0138fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:10.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb676a700 1 -- 172.21.15.32:0/1394546694 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb01438b0 con 0x7f0cb01396c0 2020-12-05T17:32:10.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb676a700 1 --2- 172.21.15.32:0/1394546694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb01396c0 0x7f0cb013f9b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f0ca80032b0 tx=0x7f0ca8009270).ready entity=mon.0 client_cookie=7930116620ce3faf server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:10.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb4f67700 1 -- 172.21.15.32:0/1394546694 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ca8009e60 con 0x7f0cb01396c0 2020-12-05T17:32:10.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb4f67700 1 -- 172.21.15.32:0/1394546694 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca800bc00 con 0x7f0cb01396c0 2020-12-05T17:32:10.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb4f67700 1 -- 172.21.15.32:0/1394546694 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ca800bd70 con 0x7f0cb01396c0 2020-12-05T17:32:10.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb6f6b700 1 -- 172.21.15.32:0/1394546694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb01396c0 msgr2=0x7f0cb013f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:10.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb6f6b700 1 --2- 172.21.15.32:0/1394546694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb01396c0 0x7f0cb013f9b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f0ca80032b0 tx=0x7f0ca8009270).stop 2020-12-05T17:32:10.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb6f6b700 1 -- 172.21.15.32:0/1394546694 shutdown_connections 2020-12-05T17:32:10.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb6f6b700 1 --2- 172.21.15.32:0/1394546694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb01396c0 0x7f0cb013f9b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb6f6b700 1 --2- 172.21.15.32:0/1394546694 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0cb00b1e60 0x7f0cb00af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb6f6b700 1 -- 172.21.15.32:0/1394546694 >> 172.21.15.32:0/1394546694 conn(0x7f0cb00b5620 msgr2=0x7f0cb00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:10.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb6f6b700 1 -- 172.21.15.32:0/1394546694 shutdown_connections 2020-12-05T17:32:10.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.211+0000 7f0cb6f6b700 1 -- 172.21.15.32:0/1394546694 wait complete. 2020-12-05T17:32:10.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0cb6f6b700 1 Processor -- start 2020-12-05T17:32:10.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0cb6f6b700 1 -- start start 2020-12-05T17:32:10.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0cb6f6b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0cb0062ac0 0x7f0cb0144ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:10.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0cb6f6b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0cb00b1e60 0x7f0cb013b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:10.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0cb6f6b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb01396c0 0x7f0cb01524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:10.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0cb6f6b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb00647b0 con 0x7f0cb01396c0 2020-12-05T17:32:10.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0cb6f6b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb0146710 con 0x7f0cb00b1e60 2020-12-05T17:32:10.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0cb6f6b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb0145ae0 con 0x7f0cb0062ac0 2020-12-05T17:32:10.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0cb676a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb01396c0 0x7f0cb01524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:10.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0cb676a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb01396c0 0x7f0cb01524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33440/0 (socket says 172.21.15.32:33440) 2020-12-05T17:32:10.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0cb676a700 1 -- 172.21.15.32:0/3745671244 learned_addr learned my addr 172.21.15.32:0/3745671244 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:10.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0cb5f69700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0cb0062ac0 0x7f0cb0144ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:10.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0cb5768700 1 --2- 172.21.15.32:0/3745671244 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0cb00b1e60 0x7f0cb013b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:10.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0cb676a700 1 -- 172.21.15.32:0/3745671244 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0cb0062ac0 msgr2=0x7f0cb0144ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:10.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0cb676a700 1 --2- 172.21.15.32:0/3745671244 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0cb0062ac0 0x7f0cb0144ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0cb676a700 1 -- 172.21.15.32:0/3745671244 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0cb00b1e60 msgr2=0x7f0cb013b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:10.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0cb676a700 1 --2- 172.21.15.32:0/3745671244 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0cb00b1e60 0x7f0cb013b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0cb676a700 1 -- 172.21.15.32:0/3745671244 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb0151d60 con 0x7f0cb01396c0 2020-12-05T17:32:10.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0cb676a700 1 --2- 172.21.15.32:0/3745671244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb01396c0 0x7f0cb01524a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f0ca80032b0 tx=0x7f0ca8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:10.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0c9effd700 1 -- 172.21.15.32:0/3745671244 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ca801f5e0 con 0x7f0cb01396c0 2020-12-05T17:32:10.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0c9effd700 1 -- 172.21.15.32:0/3745671244 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca801f740 con 0x7f0cb01396c0 2020-12-05T17:32:10.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0c9effd700 1 -- 172.21.15.32:0/3745671244 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ca801faa0 con 0x7f0cb01396c0 2020-12-05T17:32:10.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0cb6f6b700 1 -- 172.21.15.32:0/3745671244 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb0152000 con 0x7f0cb01396c0 2020-12-05T17:32:10.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.215+0000 7f0cb6f6b700 1 -- 172.21.15.32:0/3745671244 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb013be40 con 0x7f0cb01396c0 2020-12-05T17:32:10.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.219+0000 7f0cb6f6b700 1 -- 172.21.15.32:0/3745671244 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb0064620 con 0x7f0cb01396c0 2020-12-05T17:32:10.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.219+0000 7f0c9effd700 1 -- 172.21.15.32:0/3745671244 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0ca8026030 con 0x7f0cb01396c0 2020-12-05T17:32:10.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.219+0000 7f0c9effd700 1 --2- 172.21.15.32:0/3745671244 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c900553b0 0x7f0c90057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:10.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.219+0000 7f0c9effd700 1 -- 172.21.15.32:0/3745671244 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0ca806ef50 con 0x7f0cb01396c0 2020-12-05T17:32:10.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.219+0000 7f0cb5f69700 1 --2- 172.21.15.32:0/3745671244 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c900553b0 0x7f0c90057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:10.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.219+0000 7f0cb5f69700 1 --2- 172.21.15.32:0/3745671244 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c900553b0 0x7f0c90057870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f0cac00b000 tx=0x7f0cac006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:10.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.223+0000 7f0c9effd700 1 -- 172.21.15.32:0/3745671244 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0ca8045170 con 0x7f0cb01396c0 2020-12-05T17:32:10.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.367+0000 7f0cb6f6b700 1 -- 172.21.15.32:0/3745671244 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0cb0057050 con 0x7f0cb01396c0 2020-12-05T17:32:10.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.371+0000 7f0c9effd700 1 -- 172.21.15.32:0/3745671244 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7f0ca8024020 con 0x7f0cb01396c0 2020-12-05T17:32:10.373 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:32:10.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.371+0000 7f0cb6f6b700 1 -- 172.21.15.32:0/3745671244 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c900553b0 msgr2=0x7f0c90057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:10.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.371+0000 7f0cb6f6b700 1 --2- 172.21.15.32:0/3745671244 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c900553b0 0x7f0c90057870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f0cac00b000 tx=0x7f0cac006dc0).stop 2020-12-05T17:32:10.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.371+0000 7f0cb6f6b700 1 -- 172.21.15.32:0/3745671244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb01396c0 msgr2=0x7f0cb01524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:10.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.371+0000 7f0cb6f6b700 1 --2- 172.21.15.32:0/3745671244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb01396c0 0x7f0cb01524a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f0ca80032b0 tx=0x7f0ca8016000).stop 2020-12-05T17:32:10.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.371+0000 7f0cb6f6b700 1 -- 172.21.15.32:0/3745671244 shutdown_connections 2020-12-05T17:32:10.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.371+0000 7f0cb6f6b700 1 --2- 172.21.15.32:0/3745671244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb01396c0 0x7f0cb01524a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.371+0000 7f0cb6f6b700 1 --2- 172.21.15.32:0/3745671244 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c900553b0 0x7f0c90057870 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.371+0000 7f0cb6f6b700 1 --2- 172.21.15.32:0/3745671244 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0cb0062ac0 0x7f0cb0144ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.371+0000 7f0cb6f6b700 1 --2- 172.21.15.32:0/3745671244 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0cb00b1e60 0x7f0cb013b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.371+0000 7f0cb6f6b700 1 -- 172.21.15.32:0/3745671244 >> 172.21.15.32:0/3745671244 conn(0x7f0cb00b5620 msgr2=0x7f0cb01428e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:10.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.371+0000 7f0cb6f6b700 1 -- 172.21.15.32:0/3745671244 shutdown_connections 2020-12-05T17:32:10.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.371+0000 7f0cb6f6b700 1 -- 172.21.15.32:0/3745671244 wait complete. 2020-12-05T17:32:10.390 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:32:10.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74f0903700 1 Processor -- start 2020-12-05T17:32:10.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74f0903700 1 -- start start 2020-12-05T17:32:10.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74f0903700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f74ec0aead0 0x7f74ec0aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:10.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74f0903700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74ec0ad1b0 0x7f74ec1389f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:10.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74f0903700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74ec13fb70 con 0x7f74ec138ee0 2020-12-05T17:32:10.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74f0903700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74ec067b90 con 0x7f74ec0aead0 2020-12-05T17:32:10.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74f0903700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec0af460 con 0x7f74ec0ad1b0 2020-12-05T17:32:10.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74ea59c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f74ec0aead0 0x7f74ec0aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:10.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74e9d9b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74ec0ad1b0 0x7f74ec1389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:10.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74ead9d700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f74ec138ee0 0x7f74ec067480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:48736/0 (socket says 172.21.15.32:48736) 2020-12-05T17:32:10.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74ead9d700 1 -- 172.21.15.32:0/999445767 learned_addr learned my addr 172.21.15.32:0/999445767 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:10.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74ea59c700 1 --2- 172.21.15.32:0/999445767 >> v2:172.21.15.61:3300/0 conn(0x7f74ec0aead0 0x7f74ec0aeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:53336/0 (socket says 172.21.15.32:53336) 2020-12-05T17:32:10.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74ea59c700 1 -- 172.21.15.32:0/999445767 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74ec0ad1b0 msgr2=0x7f74ec1389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:10.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74ea59c700 1 --2- 172.21.15.32:0/999445767 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74ec0ad1b0 0x7f74ec1389f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74ea59c700 1 -- 172.21.15.32:0/999445767 >> v1:172.21.15.61:6789/0 conn(0x7f74ec138ee0 legacy=0x7f74ec067480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:10.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74e959a700 1 -- 172.21.15.32:0/999445767 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 995180720 0 0) 0x7f74ec13fb70 con 0x7f74ec138ee0 2020-12-05T17:32:10.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74ea59c700 1 -- 172.21.15.32:0/999445767 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ec0af700 con 0x7f74ec0aead0 2020-12-05T17:32:10.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74e9d9b700 1 --2- 172.21.15.32:0/999445767 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74ec0ad1b0 0x7f74ec1389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:32:10.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74ea59c700 1 --2- 172.21.15.32:0/999445767 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74ec0aead0 0x7f74ec0aeeb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f74dc0032b0 tx=0x7f74dc0091a0).ready entity=mon.1 client_cookie=cb775beae48d3420 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:10.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74e959a700 1 -- 172.21.15.32:0/999445767 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74dc00b9a0 con 0x7f74ec0aead0 2020-12-05T17:32:10.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74e959a700 1 -- 172.21.15.32:0/999445767 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74dc00bb00 con 0x7f74ec0aead0 2020-12-05T17:32:10.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74e959a700 1 -- 172.21.15.32:0/999445767 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74dc00bc70 con 0x7f74ec0aead0 2020-12-05T17:32:10.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74f0903700 1 -- 172.21.15.32:0/999445767 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74ec0aead0 msgr2=0x7f74ec0aeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:10.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.499+0000 7f74f0903700 1 --2- 172.21.15.32:0/999445767 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74ec0aead0 0x7f74ec0aeeb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f74dc0032b0 tx=0x7f74dc0091a0).stop 2020-12-05T17:32:10.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.503+0000 7f74f0903700 1 -- 172.21.15.32:0/999445767 shutdown_connections 2020-12-05T17:32:10.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.503+0000 7f74f0903700 1 --2- 172.21.15.32:0/999445767 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74ec0aead0 0x7f74ec0aeeb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.503+0000 7f74f0903700 1 --2- 172.21.15.32:0/999445767 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74ec0ad1b0 0x7f74ec1389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.503+0000 7f74f0903700 1 -- 172.21.15.32:0/999445767 >> 172.21.15.32:0/999445767 conn(0x7f74ec0b5580 msgr2=0x7f74ec0b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:10.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.503+0000 7f74f0903700 1 -- 172.21.15.32:0/999445767 shutdown_connections 2020-12-05T17:32:10.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.503+0000 7f74f0903700 1 -- 172.21.15.32:0/999445767 wait complete. 2020-12-05T17:32:10.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.503+0000 7f74f0903700 1 Processor -- start 2020-12-05T17:32:10.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.503+0000 7f74f0903700 1 -- start start 2020-12-05T17:32:10.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.503+0000 7f74f0903700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74ec0ad1b0 0x7f74ec13b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:10.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.503+0000 7f74f0903700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74ec0aead0 0x7f74ec066b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:10.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.503+0000 7f74f0903700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ec138ee0 0x7f74ec067020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:10.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.503+0000 7f74f0903700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec13afd0 con 0x7f74ec138ee0 2020-12-05T17:32:10.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.503+0000 7f74f0903700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec067510 con 0x7f74ec0ad1b0 2020-12-05T17:32:10.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.503+0000 7f74f0903700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec13bd90 con 0x7f74ec0aead0 2020-12-05T17:32:10.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.503+0000 7f74ead9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ec138ee0 0x7f74ec067020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:10.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.503+0000 7f74e9d9b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74ec0aead0 0x7f74ec066b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:10.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.503+0000 7f74ea59c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74ec0ad1b0 0x7f74ec13b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:10.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.503+0000 7f74ead9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ec138ee0 0x7f74ec067020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33454/0 (socket says 172.21.15.32:33454) 2020-12-05T17:32:10.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.503+0000 7f74ead9d700 1 -- 172.21.15.32:0/1028297993 learned_addr learned my addr 172.21.15.32:0/1028297993 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:10.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.507+0000 7f74ead9d700 1 -- 172.21.15.32:0/1028297993 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74ec0aead0 msgr2=0x7f74ec066b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:10.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.507+0000 7f74ead9d700 1 --2- 172.21.15.32:0/1028297993 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74ec0aead0 0x7f74ec066b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.507+0000 7f74ead9d700 1 -- 172.21.15.32:0/1028297993 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74ec0ad1b0 msgr2=0x7f74ec13b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.507+0000 7f74ead9d700 1 --2- 172.21.15.32:0/1028297993 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74ec0ad1b0 0x7f74ec13b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.507+0000 7f74ead9d700 1 -- 172.21.15.32:0/1028297993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ec13b960 con 0x7f74ec138ee0 2020-12-05T17:32:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.507+0000 7f74ead9d700 1 --2- 172.21.15.32:0/1028297993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ec138ee0 0x7f74ec067020 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f74e4000b80 tx=0x7f74e4007b40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.507+0000 7f74e37fe700 1 -- 172.21.15.32:0/1028297993 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74e40072e0 con 0x7f74ec138ee0 2020-12-05T17:32:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.507+0000 7f74e37fe700 1 -- 172.21.15.32:0/1028297993 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e4007440 con 0x7f74ec138ee0 2020-12-05T17:32:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.507+0000 7f74f0903700 1 -- 172.21.15.32:0/1028297993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ec13c030 con 0x7f74ec138ee0 2020-12-05T17:32:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.507+0000 7f74e37fe700 1 -- 172.21.15.32:0/1028297993 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74e40057d0 con 0x7f74ec138ee0 2020-12-05T17:32:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.507+0000 7f74f0903700 1 -- 172.21.15.32:0/1028297993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ec0647b0 con 0x7f74ec138ee0 2020-12-05T17:32:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.507+0000 7f74f0903700 1 -- 172.21.15.32:0/1028297993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ec057050 con 0x7f74ec138ee0 2020-12-05T17:32:10.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.507+0000 7f74e37fe700 1 -- 172.21.15.32:0/1028297993 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f74e4010070 con 0x7f74ec138ee0 2020-12-05T17:32:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.507+0000 7f74e37fe700 1 --2- 172.21.15.32:0/1028297993 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f74c40553b0 0x7f74c4057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.507+0000 7f74ea59c700 1 --2- 172.21.15.32:0/1028297993 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f74c40553b0 0x7f74c4057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.507+0000 7f74e37fe700 1 -- 172.21.15.32:0/1028297993 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f74e406d650 con 0x7f74ec138ee0 2020-12-05T17:32:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.511+0000 7f74ea59c700 1 --2- 172.21.15.32:0/1028297993 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f74c40553b0 0x7f74c4057870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f74dc01b000 tx=0x7f74dc016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:10.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.515+0000 7f74e37fe700 1 -- 172.21.15.32:0/1028297993 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f74e40437f0 con 0x7f74ec138ee0 2020-12-05T17:32:10.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.655+0000 7f74f0903700 1 -- 172.21.15.32:0/1028297993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74ec05aab0 con 0x7f74ec138ee0 2020-12-05T17:32:10.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.655+0000 7f74e37fe700 1 -- 172.21.15.32:0/1028297993 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f74e4046e10 con 0x7f74ec138ee0 2020-12-05T17:32:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:32:10.661 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:31:41.610914+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:31:36.706361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-12-05T17:31:39.298269+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:32:10.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.659+0000 7f74f0903700 1 -- 172.21.15.32:0/1028297993 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f74c40553b0 msgr2=0x7f74c4057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:10.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.659+0000 7f74f0903700 1 --2- 172.21.15.32:0/1028297993 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f74c40553b0 0x7f74c4057870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f74dc01b000 tx=0x7f74dc016000).stop 2020-12-05T17:32:10.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.659+0000 7f74f0903700 1 -- 172.21.15.32:0/1028297993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ec138ee0 msgr2=0x7f74ec067020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:10.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.659+0000 7f74f0903700 1 --2- 172.21.15.32:0/1028297993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ec138ee0 0x7f74ec067020 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f74e4000b80 tx=0x7f74e4007b40).stop 2020-12-05T17:32:10.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.659+0000 7f74f0903700 1 -- 172.21.15.32:0/1028297993 shutdown_connections 2020-12-05T17:32:10.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.659+0000 7f74f0903700 1 --2- 172.21.15.32:0/1028297993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ec138ee0 0x7f74ec067020 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.659+0000 7f74f0903700 1 --2- 172.21.15.32:0/1028297993 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74ec0ad1b0 0x7f74ec13b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.659+0000 7f74f0903700 1 --2- 172.21.15.32:0/1028297993 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f74c40553b0 0x7f74c4057870 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.659+0000 7f74f0903700 1 --2- 172.21.15.32:0/1028297993 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74ec0aead0 0x7f74ec066b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.659+0000 7f74f0903700 1 -- 172.21.15.32:0/1028297993 >> 172.21.15.32:0/1028297993 conn(0x7f74ec0b5580 msgr2=0x7f74ec13ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:10.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.659+0000 7f74f0903700 1 -- 172.21.15.32:0/1028297993 shutdown_connections 2020-12-05T17:32:10.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.659+0000 7f74f0903700 1 -- 172.21.15.32:0/1028297993 wait complete. 2020-12-05T17:32:10.679 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:31:36.706361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:31:39.298269+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:32:10.681 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:32:10.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.799+0000 7efd608eb700 1 Processor -- start 2020-12-05T17:32:10.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.799+0000 7efd608eb700 1 -- start start 2020-12-05T17:32:10.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.799+0000 7efd608eb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd5c05feb0 0x7efd5c05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:10.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.799+0000 7efd608eb700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7efd5c0b2fb0 0x7efd5c061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:10.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.799+0000 7efd608eb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd5c0b2df0 con 0x7efd5c05f3b0 2020-12-05T17:32:10.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.799+0000 7efd608eb700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd5c0b3c90 con 0x7efd5c0b2fb0 2020-12-05T17:32:10.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.799+0000 7efd608eb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c0b2b50 con 0x7efd5c05feb0 2020-12-05T17:32:10.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.799+0000 7efd5b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efd5c05f3b0 0x7efd5c13c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36106/0 (socket says 172.21.15.32:36106) 2020-12-05T17:32:10.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.799+0000 7efd5b7fe700 1 -- 172.21.15.32:0/920667753 learned_addr learned my addr 172.21.15.32:0/920667753 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:10.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.799+0000 7efd5affd700 1 --2- 172.21.15.32:0/920667753 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd5c05feb0 0x7efd5c05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:10.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.799+0000 7efd5bfff700 1 --2- 172.21.15.32:0/920667753 >> v2:172.21.15.61:3300/0 conn(0x7efd5c0b2fb0 0x7efd5c061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:10.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.799+0000 7efd5a7fc700 1 -- 172.21.15.32:0/920667753 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1180744404 0 0) 0x7efd5c0b2df0 con 0x7efd5c05f3b0 2020-12-05T17:32:10.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.799+0000 7efd5a7fc700 1 -- 172.21.15.32:0/920667753 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd3c003230 con 0x7efd5c05f3b0 2020-12-05T17:32:10.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.799+0000 7efd5affd700 1 -- 172.21.15.32:0/920667753 >> v2:172.21.15.61:3300/0 conn(0x7efd5c0b2fb0 msgr2=0x7efd5c061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:10.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.799+0000 7efd5affd700 1 --2- 172.21.15.32:0/920667753 >> v2:172.21.15.61:3300/0 conn(0x7efd5c0b2fb0 0x7efd5c061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.799+0000 7efd5affd700 1 -- 172.21.15.32:0/920667753 >> v1:172.21.15.32:6789/0 conn(0x7efd5c05f3b0 legacy=0x7efd5c13c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:10.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.799+0000 7efd5affd700 1 -- 172.21.15.32:0/920667753 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5c142820 con 0x7efd5c05feb0 2020-12-05T17:32:10.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.799+0000 7efd5affd700 1 --2- 172.21.15.32:0/920667753 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd5c05feb0 0x7efd5c05f790 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7efd44002f30 tx=0x7efd44006dc0).ready entity=mon.2 client_cookie=b86b5d853fe72373 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:10.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd5a7fc700 1 -- 172.21.15.32:0/920667753 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd440096a0 con 0x7efd5c05feb0 2020-12-05T17:32:10.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd5a7fc700 1 -- 172.21.15.32:0/920667753 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd44009800 con 0x7efd5c05feb0 2020-12-05T17:32:10.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd608eb700 1 -- 172.21.15.32:0/920667753 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd5c05feb0 msgr2=0x7efd5c05f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:10.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd608eb700 1 --2- 172.21.15.32:0/920667753 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd5c05feb0 0x7efd5c05f790 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7efd44002f30 tx=0x7efd44006dc0).stop 2020-12-05T17:32:10.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd5a7fc700 1 -- 172.21.15.32:0/920667753 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd44009b60 con 0x7efd5c05feb0 2020-12-05T17:32:10.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd608eb700 1 -- 172.21.15.32:0/920667753 shutdown_connections 2020-12-05T17:32:10.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd608eb700 1 --2- 172.21.15.32:0/920667753 >> v2:172.21.15.61:3300/0 conn(0x7efd5c0b2fb0 0x7efd5c061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd608eb700 1 --2- 172.21.15.32:0/920667753 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd5c05feb0 0x7efd5c05f790 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd608eb700 1 -- 172.21.15.32:0/920667753 >> 172.21.15.32:0/920667753 conn(0x7efd5c0b8f20 msgr2=0x7efd5c0b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:10.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd608eb700 1 -- 172.21.15.32:0/920667753 shutdown_connections 2020-12-05T17:32:10.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd608eb700 1 -- 172.21.15.32:0/920667753 wait complete. 2020-12-05T17:32:10.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd608eb700 1 Processor -- start 2020-12-05T17:32:10.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd608eb700 1 -- start start 2020-12-05T17:32:10.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd608eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd5c05f3b0 0x7efd5c154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:10.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd608eb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd5c05feb0 0x7efd5c13e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:10.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd608eb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efd5c0b2fb0 0x7efd5c149000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:10.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd608eb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c13c450 con 0x7efd5c05f3b0 2020-12-05T17:32:10.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd608eb700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c143d40 con 0x7efd5c0b2fb0 2020-12-05T17:32:10.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd608eb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c1444f0 con 0x7efd5c05feb0 2020-12-05T17:32:10.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd5b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd5c05f3b0 0x7efd5c154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:10.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd5b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd5c05f3b0 0x7efd5c154850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33468/0 (socket says 172.21.15.32:33468) 2020-12-05T17:32:10.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd5b7fe700 1 -- 172.21.15.32:0/2876971757 learned_addr learned my addr 172.21.15.32:0/2876971757 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:10.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd5bfff700 1 --2- 172.21.15.32:0/2876971757 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efd5c0b2fb0 0x7efd5c149000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:10.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd5affd700 1 --2- 172.21.15.32:0/2876971757 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd5c05feb0 0x7efd5c13e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:10.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd5b7fe700 1 -- 172.21.15.32:0/2876971757 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd5c05feb0 msgr2=0x7efd5c13e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:10.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd5b7fe700 1 --2- 172.21.15.32:0/2876971757 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd5c05feb0 0x7efd5c13e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd5b7fe700 1 -- 172.21.15.32:0/2876971757 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efd5c0b2fb0 msgr2=0x7efd5c149000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:10.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.803+0000 7efd5b7fe700 1 --2- 172.21.15.32:0/2876971757 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efd5c0b2fb0 0x7efd5c149000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.807+0000 7efd5b7fe700 1 -- 172.21.15.32:0/2876971757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5c144790 con 0x7efd5c05f3b0 2020-12-05T17:32:10.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.807+0000 7efd5b7fe700 1 --2- 172.21.15.32:0/2876971757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd5c05f3b0 0x7efd5c154850 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7efd4c0010d0 tx=0x7efd4c007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:10.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.807+0000 7efd58ff9700 1 -- 172.21.15.32:0/2876971757 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd4c0072e0 con 0x7efd5c05f3b0 2020-12-05T17:32:10.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.807+0000 7efd608eb700 1 -- 172.21.15.32:0/2876971757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd5c1422d0 con 0x7efd5c05f3b0 2020-12-05T17:32:10.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.807+0000 7efd58ff9700 1 -- 172.21.15.32:0/2876971757 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4c007440 con 0x7efd5c05f3b0 2020-12-05T17:32:10.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.807+0000 7efd58ff9700 1 -- 172.21.15.32:0/2876971757 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd4c0057d0 con 0x7efd5c05f3b0 2020-12-05T17:32:10.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.807+0000 7efd608eb700 1 -- 172.21.15.32:0/2876971757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd5c143f70 con 0x7efd5c05f3b0 2020-12-05T17:32:10.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.807+0000 7efd608eb700 1 -- 172.21.15.32:0/2876971757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd5c144920 con 0x7efd5c05f3b0 2020-12-05T17:32:10.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.807+0000 7efd58ff9700 1 -- 172.21.15.32:0/2876971757 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7efd4c010070 con 0x7efd5c05f3b0 2020-12-05T17:32:10.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.811+0000 7efd58ff9700 1 --2- 172.21.15.32:0/2876971757 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efd40055440 0x7efd40057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:10.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.811+0000 7efd58ff9700 1 -- 172.21.15.32:0/2876971757 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7efd4c06d910 con 0x7efd5c05f3b0 2020-12-05T17:32:10.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.811+0000 7efd5affd700 1 --2- 172.21.15.32:0/2876971757 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efd40055440 0x7efd40057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:10.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.811+0000 7efd5affd700 1 --2- 172.21.15.32:0/2876971757 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efd40055440 0x7efd40057900 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7efd4401b000 tx=0x7efd44015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:10.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.815+0000 7efd58ff9700 1 -- 172.21.15.32:0/2876971757 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7efd4c0439b0 con 0x7efd5c05f3b0 2020-12-05T17:32:10.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.959+0000 7efd608eb700 1 -- 172.21.15.32:0/2876971757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7efd5c057050 con 0x7efd5c05f3b0 2020-12-05T17:32:10.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.959+0000 7efd58ff9700 1 -- 172.21.15.32:0/2876971757 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7efd4c046fd0 con 0x7efd5c05f3b0 2020-12-05T17:32:10.962 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:32:10.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.959+0000 7efd608eb700 1 -- 172.21.15.32:0/2876971757 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efd40055440 msgr2=0x7efd40057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:10.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.959+0000 7efd608eb700 1 --2- 172.21.15.32:0/2876971757 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efd40055440 0x7efd40057900 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7efd4401b000 tx=0x7efd44015000).stop 2020-12-05T17:32:10.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.959+0000 7efd608eb700 1 -- 172.21.15.32:0/2876971757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd5c05f3b0 msgr2=0x7efd5c154850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:10.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.959+0000 7efd608eb700 1 --2- 172.21.15.32:0/2876971757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd5c05f3b0 0x7efd5c154850 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7efd4c0010d0 tx=0x7efd4c007b50).stop 2020-12-05T17:32:10.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.959+0000 7efd608eb700 1 -- 172.21.15.32:0/2876971757 shutdown_connections 2020-12-05T17:32:10.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.959+0000 7efd608eb700 1 --2- 172.21.15.32:0/2876971757 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efd5c0b2fb0 0x7efd5c149000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.959+0000 7efd608eb700 1 --2- 172.21.15.32:0/2876971757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd5c05f3b0 0x7efd5c154850 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.959+0000 7efd608eb700 1 --2- 172.21.15.32:0/2876971757 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efd40055440 0x7efd40057900 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.959+0000 7efd608eb700 1 --2- 172.21.15.32:0/2876971757 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd5c05feb0 0x7efd5c13e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:10.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.959+0000 7efd608eb700 1 -- 172.21.15.32:0/2876971757 >> 172.21.15.32:0/2876971757 conn(0x7efd5c0b8f20 msgr2=0x7efd5c0b7800 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:10.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.959+0000 7efd608eb700 1 -- 172.21.15.32:0/2876971757 shutdown_connections 2020-12-05T17:32:10.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:10.959+0000 7efd608eb700 1 -- 172.21.15.32:0/2876971757 wait complete. 2020-12-05T17:32:10.975 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:32:11.094 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.091+0000 7f34f4a96700 1 Processor -- start 2020-12-05T17:32:11.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.091+0000 7f34f4a96700 1 -- start start 2020-12-05T17:32:11.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.091+0000 7f34f4a96700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34f005f3b0 0x7f34f005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:11.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.091+0000 7f34f4a96700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34f005feb0 0x7f34f0061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.091+0000 7f34f4a96700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f34f00b2fb0 0x7f34f013c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.091+0000 7f34f4a96700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f00b2df0 con 0x7f34f005f3b0 2020-12-05T17:32:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.091+0000 7f34f4a96700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f00b3c90 con 0x7f34f00b2fb0 2020-12-05T17:32:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.091+0000 7f34f4a96700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f00b2b50 con 0x7f34f005feb0 2020-12-05T17:32:11.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.091+0000 7f34ef7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34f005f3b0 0x7f34f005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:11.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.091+0000 7f34ef7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34f005f3b0 0x7f34f005f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33472/0 (socket says 172.21.15.32:33472) 2020-12-05T17:32:11.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.091+0000 7f34e6ffd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34f005feb0 0x7f34f0061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:11.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.091+0000 7f34ef7fe700 1 -- 172.21.15.32:0/2735078270 learned_addr learned my addr 172.21.15.32:0/2735078270 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:11.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34effff700 1 --2- 172.21.15.32:0/2735078270 >> v2:172.21.15.61:3300/0 conn(0x7f34f00b2fb0 0x7f34f013c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:11.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34ef7fe700 1 -- 172.21.15.32:0/2735078270 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34f005feb0 msgr2=0x7f34f0061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:11.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34ef7fe700 1 --2- 172.21.15.32:0/2735078270 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34f005feb0 0x7f34f0061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34ef7fe700 1 -- 172.21.15.32:0/2735078270 >> v2:172.21.15.61:3300/0 conn(0x7f34f00b2fb0 msgr2=0x7f34f013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:11.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34ef7fe700 1 --2- 172.21.15.32:0/2735078270 >> v2:172.21.15.61:3300/0 conn(0x7f34f00b2fb0 0x7f34f013c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34ef7fe700 1 -- 172.21.15.32:0/2735078270 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f0142820 con 0x7f34f005f3b0 2020-12-05T17:32:11.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34ef7fe700 1 --2- 172.21.15.32:0/2735078270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34f005f3b0 0x7f34f005f790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f34e0002f30 tx=0x7f34e0006e80).ready entity=mon.0 client_cookie=420647a54c7b6f04 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:11.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34eeffd700 1 -- 172.21.15.32:0/2735078270 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34e0009640 con 0x7f34f005f3b0 2020-12-05T17:32:11.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34eeffd700 1 -- 172.21.15.32:0/2735078270 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e00097a0 con 0x7f34f005f3b0 2020-12-05T17:32:11.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34eeffd700 1 -- 172.21.15.32:0/2735078270 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34e0009b00 con 0x7f34f005f3b0 2020-12-05T17:32:11.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34f4a96700 1 -- 172.21.15.32:0/2735078270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34f005f3b0 msgr2=0x7f34f005f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:11.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34f4a96700 1 --2- 172.21.15.32:0/2735078270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34f005f3b0 0x7f34f005f790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f34e0002f30 tx=0x7f34e0006e80).stop 2020-12-05T17:32:11.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34f4a96700 1 -- 172.21.15.32:0/2735078270 shutdown_connections 2020-12-05T17:32:11.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34f4a96700 1 --2- 172.21.15.32:0/2735078270 >> v2:172.21.15.61:3300/0 conn(0x7f34f00b2fb0 0x7f34f013c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34f4a96700 1 --2- 172.21.15.32:0/2735078270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34f005f3b0 0x7f34f005f790 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34f4a96700 1 --2- 172.21.15.32:0/2735078270 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34f005feb0 0x7f34f0061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34f4a96700 1 -- 172.21.15.32:0/2735078270 >> 172.21.15.32:0/2735078270 conn(0x7f34f00b8f20 msgr2=0x7f34f00b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:11.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34f4a96700 1 -- 172.21.15.32:0/2735078270 shutdown_connections 2020-12-05T17:32:11.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34f4a96700 1 -- 172.21.15.32:0/2735078270 wait complete. 2020-12-05T17:32:11.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34f4a96700 1 Processor -- start 2020-12-05T17:32:11.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34f4a96700 1 -- start start 2020-12-05T17:32:11.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34f4a96700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34f005f3b0 0x7f34f0141810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:11.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34f4a96700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34f005feb0 0x7f34f0146790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:11.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34f4a96700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34f00b2fb0 0x7f34f01407d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:11.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34f4a96700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f0140f10 con 0x7f34f005feb0 2020-12-05T17:32:11.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34f4a96700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f0145db0 con 0x7f34f005f3b0 2020-12-05T17:32:11.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.095+0000 7f34f4a96700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f0146050 con 0x7f34f00b2fb0 2020-12-05T17:32:11.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.099+0000 7f34e6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34f005feb0 0x7f34f0146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:11.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.099+0000 7f34e6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34f005feb0 0x7f34f0146790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33478/0 (socket says 172.21.15.32:33478) 2020-12-05T17:32:11.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.099+0000 7f34e6ffd700 1 -- 172.21.15.32:0/1614515080 learned_addr learned my addr 172.21.15.32:0/1614515080 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:11.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.099+0000 7f34effff700 1 --2- 172.21.15.32:0/1614515080 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34f00b2fb0 0x7f34f01407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:11.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.099+0000 7f34ef7fe700 1 --2- 172.21.15.32:0/1614515080 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34f005f3b0 0x7f34f0141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:11.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.099+0000 7f34e6ffd700 1 -- 172.21.15.32:0/1614515080 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34f00b2fb0 msgr2=0x7f34f01407d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:11.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.099+0000 7f34e6ffd700 1 --2- 172.21.15.32:0/1614515080 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34f00b2fb0 0x7f34f01407d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.099+0000 7f34e6ffd700 1 -- 172.21.15.32:0/1614515080 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34f005f3b0 msgr2=0x7f34f0141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:11.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.099+0000 7f34e6ffd700 1 --2- 172.21.15.32:0/1614515080 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34f005f3b0 0x7f34f0141810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.099+0000 7f34e6ffd700 1 -- 172.21.15.32:0/1614515080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f01462f0 con 0x7f34f005feb0 2020-12-05T17:32:11.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.099+0000 7f34e6ffd700 1 --2- 172.21.15.32:0/1614515080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34f005feb0 0x7f34f0146790 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f34d800c000 tx=0x7f34d8006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:11.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.099+0000 7f34ed7fa700 1 -- 172.21.15.32:0/1614515080 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34d800ca10 con 0x7f34f005feb0 2020-12-05T17:32:11.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.099+0000 7f34ed7fa700 1 -- 172.21.15.32:0/1614515080 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d800cb70 con 0x7f34f005feb0 2020-12-05T17:32:11.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.099+0000 7f34f4a96700 1 -- 172.21.15.32:0/1614515080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f0144580 con 0x7f34f005feb0 2020-12-05T17:32:11.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.099+0000 7f34ed7fa700 1 -- 172.21.15.32:0/1614515080 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34d80123f0 con 0x7f34f005feb0 2020-12-05T17:32:11.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.099+0000 7f34f4a96700 1 -- 172.21.15.32:0/1614515080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f0149250 con 0x7f34f005feb0 2020-12-05T17:32:11.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.099+0000 7f34ed7fa700 1 -- 172.21.15.32:0/1614515080 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f34d8012550 con 0x7f34f005feb0 2020-12-05T17:32:11.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.099+0000 7f34ed7fa700 1 --2- 172.21.15.32:0/1614515080 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f34c80553b0 0x7f34c8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:11.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.099+0000 7f34ed7fa700 1 -- 172.21.15.32:0/1614515080 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f34d806e540 con 0x7f34f005feb0 2020-12-05T17:32:11.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.099+0000 7f34ef7fe700 1 --2- 172.21.15.32:0/1614515080 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f34c80553b0 0x7f34c8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:11.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.099+0000 7f34f4a96700 1 -- 172.21.15.32:0/1614515080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f0057050 con 0x7f34f005feb0 2020-12-05T17:32:11.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.103+0000 7f34ef7fe700 1 --2- 172.21.15.32:0/1614515080 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f34c80553b0 0x7f34c8057870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f34e0019000 tx=0x7f34e0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:11.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.107+0000 7f34ed7fa700 1 -- 172.21.15.32:0/1614515080 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f34d8044680 con 0x7f34f005feb0 2020-12-05T17:32:11.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.251+0000 7f34f4a96700 1 -- 172.21.15.32:0/1614515080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f34f005aab0 con 0x7f34f005feb0 2020-12-05T17:32:11.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.255+0000 7f34ed7fa700 1 -- 172.21.15.32:0/1614515080 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f34d8047ca0 con 0x7f34f005feb0 2020-12-05T17:32:11.257 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:32:11.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.255+0000 7f34f4a96700 1 -- 172.21.15.32:0/1614515080 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f34c80553b0 msgr2=0x7f34c8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:11.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.255+0000 7f34f4a96700 1 --2- 172.21.15.32:0/1614515080 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f34c80553b0 0x7f34c8057870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f34e0019000 tx=0x7f34e0015000).stop 2020-12-05T17:32:11.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.255+0000 7f34f4a96700 1 -- 172.21.15.32:0/1614515080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34f005feb0 msgr2=0x7f34f0146790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:11.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.255+0000 7f34f4a96700 1 --2- 172.21.15.32:0/1614515080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34f005feb0 0x7f34f0146790 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f34d800c000 tx=0x7f34d8006dc0).stop 2020-12-05T17:32:11.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.255+0000 7f34f4a96700 1 -- 172.21.15.32:0/1614515080 shutdown_connections 2020-12-05T17:32:11.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.255+0000 7f34f4a96700 1 --2- 172.21.15.32:0/1614515080 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f34c80553b0 0x7f34c8057870 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.255+0000 7f34f4a96700 1 --2- 172.21.15.32:0/1614515080 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34f00b2fb0 0x7f34f01407d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.255+0000 7f34f4a96700 1 --2- 172.21.15.32:0/1614515080 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34f005f3b0 0x7f34f0141810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.255+0000 7f34f4a96700 1 --2- 172.21.15.32:0/1614515080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34f005feb0 0x7f34f0146790 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.255+0000 7f34f4a96700 1 -- 172.21.15.32:0/1614515080 >> 172.21.15.32:0/1614515080 conn(0x7f34f00b8f20 msgr2=0x7f34f00b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:11.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.255+0000 7f34f4a96700 1 -- 172.21.15.32:0/1614515080 shutdown_connections 2020-12-05T17:32:11.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.255+0000 7f34f4a96700 1 -- 172.21.15.32:0/1614515080 wait complete. 2020-12-05T17:32:11.270 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:32:11.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.387+0000 7fb1df59e700 1 Processor -- start 2020-12-05T17:32:11.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.387+0000 7fb1df59e700 1 -- start start 2020-12-05T17:32:11.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.387+0000 7fb1df59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb1e00af5c0 0x7fb1e00af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:11.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.387+0000 7fb1df59e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb1e0143f00 0x7fb1e0144380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:11.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.387+0000 7fb1df59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e01407e0 con 0x7fb1e0144870 2020-12-05T17:32:11.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.387+0000 7fb1df59e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e01405f0 con 0x7fb1e0143f00 2020-12-05T17:32:11.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.387+0000 7fb1df59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e0061b30 con 0x7fb1e00af5c0 2020-12-05T17:32:11.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.387+0000 7fb1de59c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb1e00af5c0 0x7fb1e00af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:11.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.387+0000 7fb1ddd9b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb1e0143f00 0x7fb1e0144380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:11.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.387+0000 7fb1ded9d700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fb1e0144870 0x7fb1e005f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:48778/0 (socket says 172.21.15.32:48778) 2020-12-05T17:32:11.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.387+0000 7fb1ded9d700 1 -- 172.21.15.32:0/1754010852 learned_addr learned my addr 172.21.15.32:0/1754010852 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:11.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.387+0000 7fb1de59c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb1e00af5c0 0x7fb1e00af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:58510/0 (socket says 172.21.15.32:58510) 2020-12-05T17:32:11.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.391+0000 7fb1de59c700 1 -- 172.21.15.32:0/1754010852 >> v2:172.21.15.61:3300/0 conn(0x7fb1e0143f00 msgr2=0x7fb1e0144380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:11.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.391+0000 7fb1de59c700 1 --2- 172.21.15.32:0/1754010852 >> v2:172.21.15.61:3300/0 conn(0x7fb1e0143f00 0x7fb1e0144380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.391+0000 7fb1de59c700 1 -- 172.21.15.32:0/1754010852 >> v1:172.21.15.61:6789/0 conn(0x7fb1e0144870 legacy=0x7fb1e005f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:11.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.391+0000 7fb1de59c700 1 -- 172.21.15.32:0/1754010852 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e0061c70 con 0x7fb1e00af5c0 2020-12-05T17:32:11.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.391+0000 7fb1de59c700 1 --2- 172.21.15.32:0/1754010852 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb1e00af5c0 0x7fb1e00af9a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fb1d0002f30 tx=0x7fb1d0006d60).ready entity=mon.2 client_cookie=c34882be0de05db9 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:11.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.391+0000 7fb1dd59a700 1 -- 172.21.15.32:0/1754010852 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1d0009600 con 0x7fb1e00af5c0 2020-12-05T17:32:11.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.391+0000 7fb1dd59a700 1 -- 172.21.15.32:0/1754010852 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d0009760 con 0x7fb1e00af5c0 2020-12-05T17:32:11.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.391+0000 7fb1dd59a700 1 -- 172.21.15.32:0/1754010852 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1d00098d0 con 0x7fb1e00af5c0 2020-12-05T17:32:11.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.391+0000 7fb1df59e700 1 -- 172.21.15.32:0/1754010852 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb1e00af5c0 msgr2=0x7fb1e00af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:11.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.391+0000 7fb1df59e700 1 --2- 172.21.15.32:0/1754010852 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb1e00af5c0 0x7fb1e00af9a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fb1d0002f30 tx=0x7fb1d0006d60).stop 2020-12-05T17:32:11.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.391+0000 7fb1df59e700 1 -- 172.21.15.32:0/1754010852 shutdown_connections 2020-12-05T17:32:11.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.391+0000 7fb1df59e700 1 --2- 172.21.15.32:0/1754010852 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb1e00af5c0 0x7fb1e00af9a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.391+0000 7fb1df59e700 1 --2- 172.21.15.32:0/1754010852 >> v2:172.21.15.61:3300/0 conn(0x7fb1e0143f00 0x7fb1e0144380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.391+0000 7fb1df59e700 1 -- 172.21.15.32:0/1754010852 >> 172.21.15.32:0/1754010852 conn(0x7fb1e0061010 msgr2=0x7fb1e0061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:11.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.391+0000 7fb1df59e700 1 -- 172.21.15.32:0/1754010852 shutdown_connections 2020-12-05T17:32:11.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.391+0000 7fb1df59e700 1 -- 172.21.15.32:0/1754010852 wait complete. 2020-12-05T17:32:11.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.391+0000 7fb1df59e700 1 Processor -- start 2020-12-05T17:32:11.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.391+0000 7fb1df59e700 1 -- start start 2020-12-05T17:32:11.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.395+0000 7fb1df59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1e00af5c0 0x7fb1e00b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:11.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.395+0000 7fb1df59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1e0143f00 0x7fb1e00b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:11.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.395+0000 7fb1df59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb1e0144870 0x7fb1e00b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:11.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.395+0000 7fb1df59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e00b6430 con 0x7fb1e00af5c0 2020-12-05T17:32:11.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.395+0000 7fb1df59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e0063bc0 con 0x7fb1e0143f00 2020-12-05T17:32:11.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.395+0000 7fb1df59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e00b5d10 con 0x7fb1e0144870 2020-12-05T17:32:11.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.395+0000 7fb1de59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1e00af5c0 0x7fb1e00b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:11.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.395+0000 7fb1de59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1e00af5c0 0x7fb1e00b3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33492/0 (socket says 172.21.15.32:33492) 2020-12-05T17:32:11.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.395+0000 7fb1de59c700 1 -- 172.21.15.32:0/3109200062 learned_addr learned my addr 172.21.15.32:0/3109200062 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:11.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.395+0000 7fb1ddd9b700 1 --2- 172.21.15.32:0/3109200062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1e0143f00 0x7fb1e00b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:11.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.395+0000 7fb1ded9d700 1 --2- 172.21.15.32:0/3109200062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb1e0144870 0x7fb1e00b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:11.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.395+0000 7fb1de59c700 1 -- 172.21.15.32:0/3109200062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb1e0144870 msgr2=0x7fb1e00b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:11.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.395+0000 7fb1de59c700 1 --2- 172.21.15.32:0/3109200062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb1e0144870 0x7fb1e00b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.395+0000 7fb1de59c700 1 -- 172.21.15.32:0/3109200062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1e0143f00 msgr2=0x7fb1e00b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:11.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.395+0000 7fb1de59c700 1 --2- 172.21.15.32:0/3109200062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1e0143f00 0x7fb1e00b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.395+0000 7fb1de59c700 1 -- 172.21.15.32:0/3109200062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e0063df0 con 0x7fb1e00af5c0 2020-12-05T17:32:11.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.395+0000 7fb1de59c700 1 --2- 172.21.15.32:0/3109200062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1e00af5c0 0x7fb1e00b3da0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7fb1e00659a0 tx=0x7fb1d0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:11.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.395+0000 7fb1cf7fe700 1 -- 172.21.15.32:0/3109200062 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1d00065e0 con 0x7fb1e00af5c0 2020-12-05T17:32:11.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.395+0000 7fb1df59e700 1 -- 172.21.15.32:0/3109200062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e00b5fb0 con 0x7fb1e00af5c0 2020-12-05T17:32:11.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.395+0000 7fb1cf7fe700 1 -- 172.21.15.32:0/3109200062 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d00056c0 con 0x7fb1e00af5c0 2020-12-05T17:32:11.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.395+0000 7fb1cf7fe700 1 -- 172.21.15.32:0/3109200062 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1d0005a20 con 0x7fb1e00af5c0 2020-12-05T17:32:11.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.395+0000 7fb1df59e700 1 -- 172.21.15.32:0/3109200062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e00636b0 con 0x7fb1e00af5c0 2020-12-05T17:32:11.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.399+0000 7fb1cf7fe700 1 -- 172.21.15.32:0/3109200062 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb1d0026030 con 0x7fb1e00af5c0 2020-12-05T17:32:11.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.399+0000 7fb1cf7fe700 1 --2- 172.21.15.32:0/3109200062 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb1bc055440 0x7fb1bc057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:11.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.399+0000 7fb1df59e700 1 -- 172.21.15.32:0/3109200062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e0063f80 con 0x7fb1e00af5c0 2020-12-05T17:32:11.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.399+0000 7fb1ddd9b700 1 --2- 172.21.15.32:0/3109200062 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb1bc055440 0x7fb1bc057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:11.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.399+0000 7fb1cf7fe700 1 -- 172.21.15.32:0/3109200062 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb1d006e720 con 0x7fb1e00af5c0 2020-12-05T17:32:11.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.399+0000 7fb1ddd9b700 1 --2- 172.21.15.32:0/3109200062 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb1bc055440 0x7fb1bc057900 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fb1d400b000 tx=0x7fb1d4006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:11.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.403+0000 7fb1cf7fe700 1 -- 172.21.15.32:0/3109200062 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb1d0024080 con 0x7fb1e00af5c0 2020-12-05T17:32:11.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.547+0000 7fb1df59e700 1 -- 172.21.15.32:0/3109200062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb1e0061c70 con 0x7fb1e00af5c0 2020-12-05T17:32:11.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.547+0000 7fb1cf7fe700 1 -- 172.21.15.32:0/3109200062 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7fb1e0061c70 con 0x7fb1e00af5c0 2020-12-05T17:32:11.553 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:32:11.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.551+0000 7fb1df59e700 1 -- 172.21.15.32:0/3109200062 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb1bc055440 msgr2=0x7fb1bc057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:11.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.551+0000 7fb1df59e700 1 --2- 172.21.15.32:0/3109200062 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb1bc055440 0x7fb1bc057900 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fb1d400b000 tx=0x7fb1d4006dc0).stop 2020-12-05T17:32:11.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.551+0000 7fb1df59e700 1 -- 172.21.15.32:0/3109200062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1e00af5c0 msgr2=0x7fb1e00b3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:11.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.551+0000 7fb1df59e700 1 --2- 172.21.15.32:0/3109200062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1e00af5c0 0x7fb1e00b3da0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7fb1e00659a0 tx=0x7fb1d0016000).stop 2020-12-05T17:32:11.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.551+0000 7fb1df59e700 1 -- 172.21.15.32:0/3109200062 shutdown_connections 2020-12-05T17:32:11.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.551+0000 7fb1df59e700 1 --2- 172.21.15.32:0/3109200062 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb1bc055440 0x7fb1bc057900 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.551+0000 7fb1df59e700 1 --2- 172.21.15.32:0/3109200062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb1e0144870 0x7fb1e00b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.551+0000 7fb1df59e700 1 --2- 172.21.15.32:0/3109200062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1e00af5c0 0x7fb1e00b3da0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.551+0000 7fb1df59e700 1 --2- 172.21.15.32:0/3109200062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1e0143f00 0x7fb1e00b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.551+0000 7fb1df59e700 1 -- 172.21.15.32:0/3109200062 >> 172.21.15.32:0/3109200062 conn(0x7fb1e0061010 msgr2=0x7fb1e0141a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:11.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.551+0000 7fb1df59e700 1 -- 172.21.15.32:0/3109200062 shutdown_connections 2020-12-05T17:32:11.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.551+0000 7fb1df59e700 1 -- 172.21.15.32:0/3109200062 wait complete. 2020-12-05T17:32:11.576 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:32:11.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.695+0000 7fb107fff700 1 Processor -- start 2020-12-05T17:32:11.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.695+0000 7fb107fff700 1 -- start start 2020-12-05T17:32:11.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.695+0000 7fb107fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb10805f3b0 0x7fb10805f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:11.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.695+0000 7fb107fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1080b2df0 con 0x7fb10805feb0 2020-12-05T17:32:11.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.695+0000 7fb107fff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1080b3c90 con 0x7fb1080b2fb0 2020-12-05T17:32:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.695+0000 7fb107fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1080b2b50 con 0x7fb10805f3b0 2020-12-05T17:32:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.695+0000 7fb106ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb10805f3b0 0x7fb10805f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.695+0000 7fb1067fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb10805feb0 0x7fb10813c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36148/0 (socket says 172.21.15.32:36148) 2020-12-05T17:32:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.695+0000 7fb1067fc700 1 -- 172.21.15.32:0/3917467324 learned_addr learned my addr 172.21.15.32:0/3917467324 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.695+0000 7fb106ffd700 1 -- 172.21.15.32:0/3917467324 >> v1:172.21.15.61:6789/0 conn(0x7fb1080b2fb0 legacy=0x7fb108142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.695+0000 7fb106ffd700 1 -- 172.21.15.32:0/3917467324 >> v1:172.21.15.32:6789/0 conn(0x7fb10805feb0 legacy=0x7fb10813c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.695+0000 7fb105ffb700 1 -- 172.21.15.32:0/3917467324 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2722503540 0 0) 0x7fb1080b2df0 con 0x7fb10805feb0 2020-12-05T17:32:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.695+0000 7fb106ffd700 1 -- 172.21.15.32:0/3917467324 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb108142820 con 0x7fb10805f3b0 2020-12-05T17:32:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.699+0000 7fb106ffd700 1 --2- 172.21.15.32:0/3917467324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb10805f3b0 0x7fb10805f790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fb0f8002f30 tx=0x7fb0f8006dc0).ready entity=mon.0 client_cookie=4bde54a6b397000c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.699+0000 7fb105ffb700 1 -- 172.21.15.32:0/3917467324 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0f80097f0 con 0x7fb10805f3b0 2020-12-05T17:32:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.699+0000 7fb105ffb700 1 -- 172.21.15.32:0/3917467324 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f8009950 con 0x7fb10805f3b0 2020-12-05T17:32:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.699+0000 7fb105ffb700 1 -- 172.21.15.32:0/3917467324 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0f8009cb0 con 0x7fb10805f3b0 2020-12-05T17:32:11.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.699+0000 7fb107fff700 1 -- 172.21.15.32:0/3917467324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb10805f3b0 msgr2=0x7fb10805f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:11.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.699+0000 7fb107fff700 1 --2- 172.21.15.32:0/3917467324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb10805f3b0 0x7fb10805f790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fb0f8002f30 tx=0x7fb0f8006dc0).stop 2020-12-05T17:32:11.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.699+0000 7fb107fff700 1 -- 172.21.15.32:0/3917467324 shutdown_connections 2020-12-05T17:32:11.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.699+0000 7fb107fff700 1 --2- 172.21.15.32:0/3917467324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb10805f3b0 0x7fb10805f790 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.699+0000 7fb107fff700 1 -- 172.21.15.32:0/3917467324 >> 172.21.15.32:0/3917467324 conn(0x7fb1080b8f20 msgr2=0x7fb1080b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.699+0000 7fb107fff700 1 -- 172.21.15.32:0/3917467324 shutdown_connections 2020-12-05T17:32:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.699+0000 7fb107fff700 1 -- 172.21.15.32:0/3917467324 wait complete. 2020-12-05T17:32:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.699+0000 7fb107fff700 1 Processor -- start 2020-12-05T17:32:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.699+0000 7fb107fff700 1 -- start start 2020-12-05T17:32:11.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb107fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb10805f3b0 0x7fb108151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:11.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb107fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb10805feb0 0x7fb108145fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:11.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb107fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1080b2fb0 0x7fb108063b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:11.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb107fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb108063700 con 0x7fb10805feb0 2020-12-05T17:32:11.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb107fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb108154cd0 con 0x7fb1080b2fb0 2020-12-05T17:32:11.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb107fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb108154a30 con 0x7fb10805f3b0 2020-12-05T17:32:11.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb1067fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb10805feb0 0x7fb108145fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:11.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb1067fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb10805feb0 0x7fb108145fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33508/0 (socket says 172.21.15.32:33508) 2020-12-05T17:32:11.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb1067fc700 1 -- 172.21.15.32:0/2684993160 learned_addr learned my addr 172.21.15.32:0/2684993160 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:11.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb106ffd700 1 --2- 172.21.15.32:0/2684993160 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb10805f3b0 0x7fb108151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:11.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb1077fe700 1 --2- 172.21.15.32:0/2684993160 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1080b2fb0 0x7fb108063b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:11.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb1067fc700 1 -- 172.21.15.32:0/2684993160 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb10805f3b0 msgr2=0x7fb108151440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:11.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb1067fc700 1 --2- 172.21.15.32:0/2684993160 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb10805f3b0 0x7fb108151440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb1067fc700 1 -- 172.21.15.32:0/2684993160 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1080b2fb0 msgr2=0x7fb108063b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:11.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb1067fc700 1 --2- 172.21.15.32:0/2684993160 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1080b2fb0 0x7fb108063b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb1067fc700 1 -- 172.21.15.32:0/2684993160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10813d4d0 con 0x7fb10805feb0 2020-12-05T17:32:11.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb1077fe700 1 --2- 172.21.15.32:0/2684993160 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1080b2fb0 0x7fb108063b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:11.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb1067fc700 1 --2- 172.21.15.32:0/2684993160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb10805feb0 0x7fb108145fc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fb0fc000b80 tx=0x7fb0fc007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:11.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb0f7fff700 1 -- 172.21.15.32:0/2684993160 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0fc0072e0 con 0x7fb10805feb0 2020-12-05T17:32:11.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb0f7fff700 1 -- 172.21.15.32:0/2684993160 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc007440 con 0x7fb10805feb0 2020-12-05T17:32:11.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb107fff700 1 -- 172.21.15.32:0/2684993160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1080650e0 con 0x7fb10805feb0 2020-12-05T17:32:11.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb0f7fff700 1 -- 172.21.15.32:0/2684993160 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0fc0057d0 con 0x7fb10805feb0 2020-12-05T17:32:11.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb107fff700 1 -- 172.21.15.32:0/2684993160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb108065b90 con 0x7fb10805feb0 2020-12-05T17:32:11.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.703+0000 7fb107fff700 1 -- 172.21.15.32:0/2684993160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb108057050 con 0x7fb10805feb0 2020-12-05T17:32:11.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.707+0000 7fb0f7fff700 1 -- 172.21.15.32:0/2684993160 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb0fc01a060 con 0x7fb10805feb0 2020-12-05T17:32:11.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.707+0000 7fb0f7fff700 1 --2- 172.21.15.32:0/2684993160 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb0e4055330 0x7fb0e40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:11.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.707+0000 7fb0f7fff700 1 -- 172.21.15.32:0/2684993160 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb0fc06d950 con 0x7fb10805feb0 2020-12-05T17:32:11.728 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.707+0000 7fb106ffd700 1 --2- 172.21.15.32:0/2684993160 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb0e4055330 0x7fb0e40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:11.728 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.707+0000 7fb106ffd700 1 --2- 172.21.15.32:0/2684993160 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb0e4055330 0x7fb0e40577f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fb0f801b000 tx=0x7fb0f8015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:11.729 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.715+0000 7fb0f7fff700 1 -- 172.21.15.32:0/2684993160 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb0fc043af0 con 0x7fb10805feb0 2020-12-05T17:32:11.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.855+0000 7fb107fff700 1 -- 172.21.15.32:0/2684993160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb10805aab0 con 0x7fb10805feb0 2020-12-05T17:32:11.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.855+0000 7fb0f7fff700 1 -- 172.21.15.32:0/2684993160 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7fb0fc018070 con 0x7fb10805feb0 2020-12-05T17:32:11.859 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:32:11.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.859+0000 7fb107fff700 1 -- 172.21.15.32:0/2684993160 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb0e4055330 msgr2=0x7fb0e40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:11.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.859+0000 7fb107fff700 1 --2- 172.21.15.32:0/2684993160 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb0e4055330 0x7fb0e40577f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fb0f801b000 tx=0x7fb0f8015000).stop 2020-12-05T17:32:11.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.859+0000 7fb107fff700 1 -- 172.21.15.32:0/2684993160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb10805feb0 msgr2=0x7fb108145fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:11.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.859+0000 7fb107fff700 1 --2- 172.21.15.32:0/2684993160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb10805feb0 0x7fb108145fc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fb0fc000b80 tx=0x7fb0fc007ac0).stop 2020-12-05T17:32:11.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.859+0000 7fb107fff700 1 -- 172.21.15.32:0/2684993160 shutdown_connections 2020-12-05T17:32:11.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.859+0000 7fb107fff700 1 --2- 172.21.15.32:0/2684993160 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1080b2fb0 0x7fb108063b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.859+0000 7fb107fff700 1 --2- 172.21.15.32:0/2684993160 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb0e4055330 0x7fb0e40577f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.859+0000 7fb107fff700 1 --2- 172.21.15.32:0/2684993160 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb10805f3b0 0x7fb108151440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.859+0000 7fb107fff700 1 --2- 172.21.15.32:0/2684993160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb10805feb0 0x7fb108145fc0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:11.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.859+0000 7fb107fff700 1 -- 172.21.15.32:0/2684993160 >> 172.21.15.32:0/2684993160 conn(0x7fb1080b8f20 msgr2=0x7fb1080b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:11.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.859+0000 7fb107fff700 1 -- 172.21.15.32:0/2684993160 shutdown_connections 2020-12-05T17:32:11.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:11.859+0000 7fb107fff700 1 -- 172.21.15.32:0/2684993160 wait complete. 2020-12-05T17:32:11.879 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop) 2020-12-05T17:32:11.885 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command' 2020-12-05T17:32:12.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.003+0000 7fc8bffff700 1 Processor -- start 2020-12-05T17:32:12.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.003+0000 7fc8bffff700 1 -- start start 2020-12-05T17:32:12.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8bffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8c00ab710 0x7fc8c00af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:12.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8bffff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc8c0147400 0x7fc8c0145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:12.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8bffff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8c0146a90 con 0x7fc8c00b1fc0 2020-12-05T17:32:12.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8bffff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8c0061e90 con 0x7fc8c00ab710 2020-12-05T17:32:12.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8bffff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8c00b0110 con 0x7fc8c0147400 2020-12-05T17:32:12.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8be7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8c00ab710 0x7fc8c00af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:12.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8be7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8c00ab710 0x7fc8c00af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33518/0 (socket says 172.21.15.32:33518) 2020-12-05T17:32:12.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8be7fc700 1 -- 172.21.15.32:0/2928561928 learned_addr learned my addr 172.21.15.32:0/2928561928 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:12.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8bf7fe700 1 --2- 172.21.15.32:0/2928561928 >> v2:172.21.15.61:3300/0 conn(0x7fc8c0147400 0x7fc8c0145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:12.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8be7fc700 1 -- 172.21.15.32:0/2928561928 >> v2:172.21.15.61:3300/0 conn(0x7fc8c0147400 msgr2=0x7fc8c0145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:12.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8be7fc700 1 --2- 172.21.15.32:0/2928561928 >> v2:172.21.15.61:3300/0 conn(0x7fc8c0147400 0x7fc8c0145d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:12.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8be7fc700 1 -- 172.21.15.32:0/2928561928 >> v1:172.21.15.61:6789/0 conn(0x7fc8c00b1fc0 legacy=0x7fc8c0061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:12.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8be7fc700 1 -- 172.21.15.32:0/2928561928 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c00b23a0 con 0x7fc8c00ab710 2020-12-05T17:32:12.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8bf7fe700 1 --2- 172.21.15.32:0/2928561928 >> v2:172.21.15.61:3300/0 conn(0x7fc8c0147400 0x7fc8c0145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:12.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8be7fc700 1 --2- 172.21.15.32:0/2928561928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8c00ab710 0x7fc8c00af230 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fc8b0002f30 tx=0x7fc8b0006e80).ready entity=mon.0 client_cookie=97134c1d8b897f89 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:12.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8bdffb700 1 -- 172.21.15.32:0/2928561928 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8b00097f0 con 0x7fc8c00ab710 2020-12-05T17:32:12.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8bdffb700 1 -- 172.21.15.32:0/2928561928 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b0009950 con 0x7fc8c00ab710 2020-12-05T17:32:12.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8bdffb700 1 -- 172.21.15.32:0/2928561928 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8b0009cb0 con 0x7fc8c00ab710 2020-12-05T17:32:12.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8bffff700 1 -- 172.21.15.32:0/2928561928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8c00ab710 msgr2=0x7fc8c00af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:12.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8bffff700 1 --2- 172.21.15.32:0/2928561928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8c00ab710 0x7fc8c00af230 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fc8b0002f30 tx=0x7fc8b0006e80).stop 2020-12-05T17:32:12.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8bffff700 1 -- 172.21.15.32:0/2928561928 shutdown_connections 2020-12-05T17:32:12.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8bffff700 1 --2- 172.21.15.32:0/2928561928 >> v2:172.21.15.61:3300/0 conn(0x7fc8c0147400 0x7fc8c0145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:12.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8bffff700 1 --2- 172.21.15.32:0/2928561928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8c00ab710 0x7fc8c00af230 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:12.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8bffff700 1 -- 172.21.15.32:0/2928561928 >> 172.21.15.32:0/2928561928 conn(0x7fc8c00b5530 msgr2=0x7fc8c00b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:12.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8bffff700 1 -- 172.21.15.32:0/2928561928 shutdown_connections 2020-12-05T17:32:12.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8bffff700 1 -- 172.21.15.32:0/2928561928 wait complete. 2020-12-05T17:32:12.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8bffff700 1 Processor -- start 2020-12-05T17:32:12.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.007+0000 7fc8bffff700 1 -- start start 2020-12-05T17:32:12.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.011+0000 7fc8bffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc8c00ab710 0x7fc8c00ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:12.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.011+0000 7fc8bffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8c00b1fc0 0x7fc8c01563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:12.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.011+0000 7fc8bffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8c0147400 0x7fc8c014b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:12.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.011+0000 7fc8bffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c00ad760 con 0x7fc8c0147400 2020-12-05T17:32:12.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.011+0000 7fc8bffff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c00aca80 con 0x7fc8c00b1fc0 2020-12-05T17:32:12.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.011+0000 7fc8bffff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c014d920 con 0x7fc8c00ab710 2020-12-05T17:32:12.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.011+0000 7fc8bf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8c0147400 0x7fc8c014b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:12.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.011+0000 7fc8be7fc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8c00b1fc0 0x7fc8c01563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:12.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.011+0000 7fc8beffd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc8c00ab710 0x7fc8c00ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:12.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.011+0000 7fc8bf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8c0147400 0x7fc8c014b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33524/0 (socket says 172.21.15.32:33524) 2020-12-05T17:32:12.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.011+0000 7fc8bf7fe700 1 -- 172.21.15.32:0/3247585195 learned_addr learned my addr 172.21.15.32:0/3247585195 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:12.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.011+0000 7fc8bf7fe700 1 -- 172.21.15.32:0/3247585195 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc8c00ab710 msgr2=0x7fc8c00ad0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:12.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.011+0000 7fc8bf7fe700 1 --2- 172.21.15.32:0/3247585195 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc8c00ab710 0x7fc8c00ad0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:12.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.011+0000 7fc8bf7fe700 1 -- 172.21.15.32:0/3247585195 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8c00b1fc0 msgr2=0x7fc8c01563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:12.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.011+0000 7fc8bf7fe700 1 --2- 172.21.15.32:0/3247585195 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8c00b1fc0 0x7fc8c01563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:12.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.011+0000 7fc8bf7fe700 1 -- 172.21.15.32:0/3247585195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c014dbc0 con 0x7fc8c0147400 2020-12-05T17:32:12.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.011+0000 7fc8bf7fe700 1 --2- 172.21.15.32:0/3247585195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8c0147400 0x7fc8c014b0b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fc8b8006dd0 tx=0x7fc8b8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:12.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.011+0000 7fc8b7fff700 1 -- 172.21.15.32:0/3247585195 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8b800b9a0 con 0x7fc8c0147400 2020-12-05T17:32:12.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.011+0000 7fc8b7fff700 1 -- 172.21.15.32:0/3247585195 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b800bb00 con 0x7fc8c0147400 2020-12-05T17:32:12.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.011+0000 7fc8bffff700 1 -- 172.21.15.32:0/3247585195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c00ac770 con 0x7fc8c0147400 2020-12-05T17:32:12.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.011+0000 7fc8b7fff700 1 -- 172.21.15.32:0/3247585195 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8b800be60 con 0x7fc8c0147400 2020-12-05T17:32:12.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.011+0000 7fc8bffff700 1 -- 172.21.15.32:0/3247585195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c00abb50 con 0x7fc8c0147400 2020-12-05T17:32:12.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.015+0000 7fc8bffff700 1 -- 172.21.15.32:0/3247585195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8c01568d0 con 0x7fc8c0147400 2020-12-05T17:32:12.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.015+0000 7fc8b7fff700 1 -- 172.21.15.32:0/3247585195 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc8b801c460 con 0x7fc8c0147400 2020-12-05T17:32:12.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.015+0000 7fc8b7fff700 1 --2- 172.21.15.32:0/3247585195 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc8a4055330 0x7fc8a40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:12.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.015+0000 7fc8b7fff700 1 -- 172.21.15.32:0/3247585195 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc8b806e860 con 0x7fc8c0147400 2020-12-05T17:32:12.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.015+0000 7fc8beffd700 1 --2- 172.21.15.32:0/3247585195 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc8a4055330 0x7fc8a40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:12.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.015+0000 7fc8beffd700 1 --2- 172.21.15.32:0/3247585195 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc8a4055330 0x7fc8a40577f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fc8a800d000 tx=0x7fc8a8007b60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:12.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.019+0000 7fc8b7fff700 1 -- 172.21.15.32:0/3247585195 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc8b8044890 con 0x7fc8c0147400 2020-12-05T17:32:12.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.215+0000 7fc8bffff700 1 -- 172.21.15.32:0/3247585195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]} v 0) v1 -- 0x7fc8c0057050 con 0x7fc8c0147400 2020-12-05T17:32:12.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.903+0000 7fc8b7fff700 1 -- 172.21.15.32:0/3247585195 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]}]=0 v51) v1 ==== 142+0+0 (secure 0 0 0) 0x7fc8b8047eb0 con 0x7fc8c0147400 2020-12-05T17:32:12.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.907+0000 7fc8bffff700 1 -- 172.21.15.32:0/3247585195 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc8a4055330 msgr2=0x7fc8a40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:12.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.907+0000 7fc8bffff700 1 --2- 172.21.15.32:0/3247585195 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc8a4055330 0x7fc8a40577f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fc8a800d000 tx=0x7fc8a8007b60).stop 2020-12-05T17:32:12.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.907+0000 7fc8bffff700 1 -- 172.21.15.32:0/3247585195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8c0147400 msgr2=0x7fc8c014b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:12.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.907+0000 7fc8bffff700 1 --2- 172.21.15.32:0/3247585195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8c0147400 0x7fc8c014b0b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fc8b8006dd0 tx=0x7fc8b8008040).stop 2020-12-05T17:32:12.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.907+0000 7fc8bffff700 1 -- 172.21.15.32:0/3247585195 shutdown_connections 2020-12-05T17:32:12.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.907+0000 7fc8bffff700 1 --2- 172.21.15.32:0/3247585195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8c0147400 0x7fc8c014b0b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:12.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.907+0000 7fc8bffff700 1 --2- 172.21.15.32:0/3247585195 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc8a4055330 0x7fc8a40577f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:12.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.907+0000 7fc8bffff700 1 --2- 172.21.15.32:0/3247585195 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc8c00ab710 0x7fc8c00ad0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:12.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.907+0000 7fc8bffff700 1 --2- 172.21.15.32:0/3247585195 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8c00b1fc0 0x7fc8c01563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:12.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.907+0000 7fc8bffff700 1 -- 172.21.15.32:0/3247585195 >> 172.21.15.32:0/3247585195 conn(0x7fc8c00b5530 msgr2=0x7fc8c00af740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:12.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.907+0000 7fc8bffff700 1 -- 172.21.15.32:0/3247585195 shutdown_connections 2020-12-05T17:32:12.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:12.907+0000 7fc8bffff700 1 -- 172.21.15.32:0/3247585195 wait complete. 2020-12-05T17:32:12.926 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-12-05T17:32:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.047+0000 7fa782200700 1 Processor -- start 2020-12-05T17:32:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.047+0000 7fa782200700 1 -- start start 2020-12-05T17:32:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.047+0000 7fa782200700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa77c05f3b0 0x7fa77c05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.047+0000 7fa782200700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa77c0b2fb0 0x7fa77c061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.047+0000 7fa782200700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77c0b2df0 con 0x7fa77c05feb0 2020-12-05T17:32:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.047+0000 7fa782200700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa77c0b3c90 con 0x7fa77c0b2fb0 2020-12-05T17:32:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.047+0000 7fa782200700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c0b2b50 con 0x7fa77c05f3b0 2020-12-05T17:32:13.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.047+0000 7fa7811fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa77c05f3b0 0x7fa77c05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:13.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.047+0000 7fa7819ff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa77c0b2fb0 0x7fa77c061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.047+0000 7fa7809fd700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fa77c05feb0 0x7fa77c13c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:48816/0 (socket says 172.21.15.32:48816) 2020-12-05T17:32:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.047+0000 7fa7811fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa77c05f3b0 0x7fa77c05f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:58550/0 (socket says 172.21.15.32:58550) 2020-12-05T17:32:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.047+0000 7fa7809fd700 1 -- 172.21.15.32:0/1157817004 learned_addr learned my addr 172.21.15.32:0/1157817004 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.047+0000 7fa7811fe700 1 -- 172.21.15.32:0/1157817004 >> v2:172.21.15.61:3300/0 conn(0x7fa77c0b2fb0 msgr2=0x7fa77c061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.047+0000 7fa7811fe700 1 --2- 172.21.15.32:0/1157817004 >> v2:172.21.15.61:3300/0 conn(0x7fa77c0b2fb0 0x7fa77c061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.047+0000 7fa7811fe700 1 -- 172.21.15.32:0/1157817004 >> v1:172.21.15.61:6789/0 conn(0x7fa77c05feb0 legacy=0x7fa77c13c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.047+0000 7fa7811fe700 1 -- 172.21.15.32:0/1157817004 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c142820 con 0x7fa77c05f3b0 2020-12-05T17:32:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.051+0000 7fa7811fe700 1 --2- 172.21.15.32:0/1157817004 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa77c05f3b0 0x7fa77c05f790 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fa76c002f30 tx=0x7fa76c006dc0).ready entity=mon.2 client_cookie=189f185f0776035e server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.051+0000 7fa773fff700 1 -- 172.21.15.32:0/1157817004 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa76c009760 con 0x7fa77c05f3b0 2020-12-05T17:32:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.051+0000 7fa773fff700 1 -- 172.21.15.32:0/1157817004 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76c0098c0 con 0x7fa77c05f3b0 2020-12-05T17:32:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.051+0000 7fa773fff700 1 -- 172.21.15.32:0/1157817004 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa76c009c20 con 0x7fa77c05f3b0 2020-12-05T17:32:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.051+0000 7fa782200700 1 -- 172.21.15.32:0/1157817004 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa77c05f3b0 msgr2=0x7fa77c05f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.051+0000 7fa782200700 1 --2- 172.21.15.32:0/1157817004 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa77c05f3b0 0x7fa77c05f790 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fa76c002f30 tx=0x7fa76c006dc0).stop 2020-12-05T17:32:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.051+0000 7fa782200700 1 -- 172.21.15.32:0/1157817004 shutdown_connections 2020-12-05T17:32:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.051+0000 7fa782200700 1 --2- 172.21.15.32:0/1157817004 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa77c05f3b0 0x7fa77c05f790 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.051+0000 7fa782200700 1 --2- 172.21.15.32:0/1157817004 >> v2:172.21.15.61:3300/0 conn(0x7fa77c0b2fb0 0x7fa77c061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.051+0000 7fa782200700 1 -- 172.21.15.32:0/1157817004 >> 172.21.15.32:0/1157817004 conn(0x7fa77c0b8f20 msgr2=0x7fa77c0b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.051+0000 7fa782200700 1 -- 172.21.15.32:0/1157817004 shutdown_connections 2020-12-05T17:32:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.051+0000 7fa782200700 1 -- 172.21.15.32:0/1157817004 wait complete. 2020-12-05T17:32:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.051+0000 7fa782200700 1 Processor -- start 2020-12-05T17:32:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.051+0000 7fa782200700 1 -- start start 2020-12-05T17:32:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.055+0000 7fa782200700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa77c05f3b0 0x7fa77c149740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.055+0000 7fa782200700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77c05feb0 0x7fa77c13b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.055+0000 7fa782200700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa77c0b2fb0 0x7fa77c13bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.055+0000 7fa782200700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c1483a0 con 0x7fa77c05feb0 2020-12-05T17:32:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.055+0000 7fa782200700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c148640 con 0x7fa77c05f3b0 2020-12-05T17:32:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.055+0000 7fa782200700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c1488e0 con 0x7fa77c0b2fb0 2020-12-05T17:32:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.055+0000 7fa7809fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77c05feb0 0x7fa77c13b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.055+0000 7fa7811fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa77c05f3b0 0x7fa77c149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.055+0000 7fa7819ff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa77c0b2fb0 0x7fa77c13bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.055+0000 7fa7809fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77c05feb0 0x7fa77c13b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33536/0 (socket says 172.21.15.32:33536) 2020-12-05T17:32:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.055+0000 7fa7809fd700 1 -- 172.21.15.32:0/3280150037 learned_addr learned my addr 172.21.15.32:0/3280150037 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.055+0000 7fa7819ff700 1 -- 172.21.15.32:0/3280150037 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa77c05f3b0 msgr2=0x7fa77c149740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.055+0000 7fa7819ff700 1 --2- 172.21.15.32:0/3280150037 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa77c05f3b0 0x7fa77c149740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.055+0000 7fa7819ff700 1 -- 172.21.15.32:0/3280150037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77c05feb0 msgr2=0x7fa77c13b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.055+0000 7fa7819ff700 1 --2- 172.21.15.32:0/3280150037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77c05feb0 0x7fa77c13b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.055+0000 7fa7819ff700 1 -- 172.21.15.32:0/3280150037 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c13c0e0 con 0x7fa77c0b2fb0 2020-12-05T17:32:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.055+0000 7fa7819ff700 1 --2- 172.21.15.32:0/3280150037 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa77c0b2fb0 0x7fa77c13bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fa77400c000 tx=0x7fa7740091a0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.055+0000 7fa7727fc700 1 -- 172.21.15.32:0/3280150037 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa77400c8f0 con 0x7fa77c0b2fb0 2020-12-05T17:32:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.055+0000 7fa7727fc700 1 -- 172.21.15.32:0/3280150037 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77400ca50 con 0x7fa77c0b2fb0 2020-12-05T17:32:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.055+0000 7fa782200700 1 -- 172.21.15.32:0/3280150037 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa77c145f60 con 0x7fa77c0b2fb0 2020-12-05T17:32:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.055+0000 7fa7727fc700 1 -- 172.21.15.32:0/3280150037 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa77400cdb0 con 0x7fa77c0b2fb0 2020-12-05T17:32:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.055+0000 7fa782200700 1 -- 172.21.15.32:0/3280150037 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa77c146ab0 con 0x7fa77c0b2fb0 2020-12-05T17:32:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.055+0000 7fa782200700 1 -- 172.21.15.32:0/3280150037 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77c147f30 con 0x7fa77c0b2fb0 2020-12-05T17:32:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.059+0000 7fa7727fc700 1 -- 172.21.15.32:0/3280150037 <== mon.2 v2:172.21.15.61:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa774022070 con 0x7fa77c0b2fb0 2020-12-05T17:32:13.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.059+0000 7fa7727fc700 1 --2- 172.21.15.32:0/3280150037 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa764055440 0x7fa764057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:13.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.059+0000 7fa7811fe700 1 --2- 172.21.15.32:0/3280150037 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa764055440 0x7fa764057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:13.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.059+0000 7fa7727fc700 1 -- 172.21.15.32:0/3280150037 <== mon.2 v2:172.21.15.61:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa77406e6d0 con 0x7fa77c0b2fb0 2020-12-05T17:32:13.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.059+0000 7fa7811fe700 1 --2- 172.21.15.32:0/3280150037 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa764055440 0x7fa764057900 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fa76c01a000 tx=0x7fa76c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.067+0000 7fa7727fc700 1 -- 172.21.15.32:0/3280150037 <== mon.2 v2:172.21.15.61:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa774044760 con 0x7fa77c0b2fb0 2020-12-05T17:32:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.215+0000 7fa782200700 1 -- 172.21.15.32:0/3280150037 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7fa77c057050 con 0x7fa77c0b2fb0 2020-12-05T17:32:13.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.223+0000 7fa7727fc700 1 -- 172.21.15.32:0/3280150037 <== mon.2 v2:172.21.15.61:3301/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa774016020 con 0x7fa77c0b2fb0 2020-12-05T17:32:13.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.227+0000 7fa782200700 1 -- 172.21.15.32:0/3280150037 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa764055440 msgr2=0x7fa764057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:13.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.227+0000 7fa782200700 1 --2- 172.21.15.32:0/3280150037 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa764055440 0x7fa764057900 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fa76c01a000 tx=0x7fa76c015000).stop 2020-12-05T17:32:13.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.227+0000 7fa782200700 1 -- 172.21.15.32:0/3280150037 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa77c0b2fb0 msgr2=0x7fa77c13bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:13.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.227+0000 7fa782200700 1 --2- 172.21.15.32:0/3280150037 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa77c0b2fb0 0x7fa77c13bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fa77400c000 tx=0x7fa7740091a0).stop 2020-12-05T17:32:13.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.227+0000 7fa782200700 1 -- 172.21.15.32:0/3280150037 shutdown_connections 2020-12-05T17:32:13.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.227+0000 7fa782200700 1 --2- 172.21.15.32:0/3280150037 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa764055440 0x7fa764057900 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:13.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.227+0000 7fa782200700 1 --2- 172.21.15.32:0/3280150037 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa77c0b2fb0 0x7fa77c13bbf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:13.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.227+0000 7fa782200700 1 --2- 172.21.15.32:0/3280150037 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa77c05f3b0 0x7fa77c149740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:13.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.227+0000 7fa782200700 1 --2- 172.21.15.32:0/3280150037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77c05feb0 0x7fa77c13b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:13.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.227+0000 7fa782200700 1 -- 172.21.15.32:0/3280150037 >> 172.21.15.32:0/3280150037 conn(0x7fa77c0b8f20 msgr2=0x7fa77c0b9530 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:13.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.227+0000 7fa782200700 1 -- 172.21.15.32:0/3280150037 shutdown_connections 2020-12-05T17:32:13.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:13.227+0000 7fa782200700 1 -- 172.21.15.32:0/3280150037 wait complete. 2020-12-05T17:32:13.244 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:32:13.244 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:32:13.270 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:32:13.291 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:32:13.320 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:32:13.349 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:32:13.377 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:32:13.378 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:13.378 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:32:13.379 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:32:13.387 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:32:13.388 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:32:13.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:13.389 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:32:13.389 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:32:13.441 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:32:13.441 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:32:13.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:13.442 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:32:13.443 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:32:13.490 INFO:teuthology.orchestra.run.smithi036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2020-12-05T17:32:13.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:32:13.492 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2020-12-05T17:32:13.492 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi036.front.sepia.ceph.com... 2020-12-05T17:32:13.492 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:32:13.493 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:32:13.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:13.603 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:32:13.604 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:32:13.610 INFO:teuthology.orchestra.run.smithi036.stdout:ext2/ext3 2020-12-05T17:32:13.612 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:32:13.613 INFO:teuthology.orchestra.run:waiting for 900 2020-12-05T17:32:13.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:32:13.614 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:32:13.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:13.666 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:32:13.666 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:32:13.674 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:32:13.676 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:32:13.676 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:13.677 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:32:13.678 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:32:13.724 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:32:13.725 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:32:13.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:13.726 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:32:13.727 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:32:13.772 INFO:teuthology.orchestra.run.smithi002.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': Transport endpoint is not connected 2020-12-05T17:32:13.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:32:13.774 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.1 2020-12-05T17:32:13.775 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi002.front.sepia.ceph.com... 2020-12-05T17:32:13.775 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:32:13.776 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:32:13.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:13.876 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:32:13.877 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:32:13.883 INFO:teuthology.orchestra.run.smithi002.stdout:ext2/ext3 2020-12-05T17:32:13.884 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:32:13.885 INFO:teuthology.orchestra.run:waiting for 900 2020-12-05T17:32:13.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:32:13.886 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:32:13.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:13.939 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:32:13.940 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:32:13.947 INFO:teuthology.orchestra.run.smithi053.stdout:fuseblk 2020-12-05T17:32:13.948 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2020-12-05T17:32:13.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:13.949 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:32:13.949 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:32:13.999 INFO:teuthology.orchestra.run.smithi053.stdout:fuseblk 2020-12-05T17:32:14.000 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2020-12-05T17:32:14.001 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:14.001 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:32:14.002 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:32:14.051 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': Transport endpoint is not connected 2020-12-05T17:32:14.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:32:14.052 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.2 2020-12-05T17:32:14.053 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi053.front.sepia.ceph.com... 2020-12-05T17:32:14.053 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:32:14.054 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:32:14.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:14.151 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:32:14.151 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:32:14.158 INFO:teuthology.orchestra.run.smithi053.stdout:ext2/ext3 2020-12-05T17:32:14.159 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:32:14.159 INFO:teuthology.orchestra.run:waiting for 900 2020-12-05T17:32:14.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:32:14.161 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:32:14.222 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-12-05T17:32:14.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.343+0000 7fac889f4700 1 Processor -- start 2020-12-05T17:32:14.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.343+0000 7fac889f4700 1 -- start start 2020-12-05T17:32:14.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.347+0000 7fac889f4700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fac841400e0 0x7fac840acb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:14.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.347+0000 7fac889f4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac840adc20 con 0x7fac840aedc0 2020-12-05T17:32:14.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.347+0000 7fac889f4700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac840adec0 con 0x7fac840b2050 2020-12-05T17:32:14.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.347+0000 7fac889f4700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac840b2870 con 0x7fac841400e0 2020-12-05T17:32:14.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.347+0000 7fac81d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fac840aedc0 0x7fac8413f9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36192/0 (socket says 172.21.15.32:36192) 2020-12-05T17:32:14.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.347+0000 7fac81d9b700 1 -- 172.21.15.32:0/339667691 learned_addr learned my addr 172.21.15.32:0/339667691 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:14.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.347+0000 7fac82d9d700 1 --2- 172.21.15.32:0/339667691 >> v2:172.21.15.61:3300/0 conn(0x7fac841400e0 0x7fac840acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:14.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.347+0000 7fac8159a700 1 -- 172.21.15.32:0/339667691 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3292521295 0 0) 0x7fac840adc20 con 0x7fac840aedc0 2020-12-05T17:32:14.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.347+0000 7fac8159a700 1 -- 172.21.15.32:0/339667691 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac64003230 con 0x7fac840aedc0 2020-12-05T17:32:14.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.347+0000 7fac82d9d700 1 -- 172.21.15.32:0/339667691 >> v1:172.21.15.61:6789/0 conn(0x7fac840b2050 legacy=0x7fac840ae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:14.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.347+0000 7fac82d9d700 1 -- 172.21.15.32:0/339667691 >> v1:172.21.15.32:6789/0 conn(0x7fac840aedc0 legacy=0x7fac8413f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:14.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.347+0000 7fac82d9d700 1 -- 172.21.15.32:0/339667691 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac840b2430 con 0x7fac841400e0 2020-12-05T17:32:14.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.347+0000 7fac82d9d700 1 --2- 172.21.15.32:0/339667691 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac841400e0 0x7fac840acb80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fac6c002f30 tx=0x7fac6c006dc0).ready entity=mon.1 client_cookie=92a504bc552235f4 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:14.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.347+0000 7fac8159a700 1 -- 172.21.15.32:0/339667691 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac6c009800 con 0x7fac841400e0 2020-12-05T17:32:14.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.347+0000 7fac8159a700 1 -- 172.21.15.32:0/339667691 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6c009960 con 0x7fac841400e0 2020-12-05T17:32:14.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.347+0000 7fac8159a700 1 -- 172.21.15.32:0/339667691 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac6c009cc0 con 0x7fac841400e0 2020-12-05T17:32:14.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.347+0000 7fac889f4700 1 -- 172.21.15.32:0/339667691 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac841400e0 msgr2=0x7fac840acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:14.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.347+0000 7fac889f4700 1 --2- 172.21.15.32:0/339667691 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac841400e0 0x7fac840acb80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fac6c002f30 tx=0x7fac6c006dc0).stop 2020-12-05T17:32:14.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.347+0000 7fac889f4700 1 -- 172.21.15.32:0/339667691 shutdown_connections 2020-12-05T17:32:14.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.347+0000 7fac889f4700 1 --2- 172.21.15.32:0/339667691 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac841400e0 0x7fac840acb80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:14.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.347+0000 7fac889f4700 1 -- 172.21.15.32:0/339667691 >> 172.21.15.32:0/339667691 conn(0x7fac840b6ef0 msgr2=0x7fac840b72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:14.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.347+0000 7fac889f4700 1 -- 172.21.15.32:0/339667691 shutdown_connections 2020-12-05T17:32:14.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.347+0000 7fac889f4700 1 -- 172.21.15.32:0/339667691 wait complete. 2020-12-05T17:32:14.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.351+0000 7fac889f4700 1 Processor -- start 2020-12-05T17:32:14.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.351+0000 7fac889f4700 1 -- start start 2020-12-05T17:32:14.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.351+0000 7fac889f4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac840aedc0 0x7fac841466b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:14.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.351+0000 7fac889f4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac840b2050 0x7fac84066000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:14.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.351+0000 7fac889f4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac841400e0 0x7fac84064fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:14.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.351+0000 7fac889f4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac84064280 con 0x7fac840b2050 2020-12-05T17:32:14.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.351+0000 7fac889f4700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac84142cd0 con 0x7fac841400e0 2020-12-05T17:32:14.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.351+0000 7fac889f4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac840664f0 con 0x7fac840aedc0 2020-12-05T17:32:14.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.351+0000 7fac81d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac840b2050 0x7fac84066000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:14.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.351+0000 7fac81d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac840b2050 0x7fac84066000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33550/0 (socket says 172.21.15.32:33550) 2020-12-05T17:32:14.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.351+0000 7fac81d9b700 1 -- 172.21.15.32:0/2784964363 learned_addr learned my addr 172.21.15.32:0/2784964363 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:14.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.351+0000 7fac8259c700 1 --2- 172.21.15.32:0/2784964363 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac840aedc0 0x7fac841466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:14.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.351+0000 7fac82d9d700 1 --2- 172.21.15.32:0/2784964363 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac841400e0 0x7fac84064fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:14.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.351+0000 7fac81d9b700 1 -- 172.21.15.32:0/2784964363 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac840aedc0 msgr2=0x7fac841466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:14.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.351+0000 7fac81d9b700 1 --2- 172.21.15.32:0/2784964363 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac840aedc0 0x7fac841466b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:14.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.351+0000 7fac81d9b700 1 -- 172.21.15.32:0/2784964363 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac841400e0 msgr2=0x7fac84064fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:14.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.351+0000 7fac81d9b700 1 --2- 172.21.15.32:0/2784964363 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac841400e0 0x7fac84064fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:14.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.351+0000 7fac81d9b700 1 -- 172.21.15.32:0/2784964363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac84142a30 con 0x7fac840b2050 2020-12-05T17:32:14.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.351+0000 7fac82d9d700 1 --2- 172.21.15.32:0/2784964363 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac841400e0 0x7fac84064fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:14.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.351+0000 7fac81d9b700 1 --2- 172.21.15.32:0/2784964363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac840b2050 0x7fac84066000 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fac740010d0 tx=0x7fac74007af0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:14.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.355+0000 7fac7b7fe700 1 -- 172.21.15.32:0/2784964363 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac740072e0 con 0x7fac840b2050 2020-12-05T17:32:14.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.355+0000 7fac7b7fe700 1 -- 172.21.15.32:0/2784964363 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac74007440 con 0x7fac840b2050 2020-12-05T17:32:14.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.355+0000 7fac889f4700 1 -- 172.21.15.32:0/2784964363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac84142610 con 0x7fac840b2050 2020-12-05T17:32:14.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.355+0000 7fac7b7fe700 1 -- 172.21.15.32:0/2784964363 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac740057d0 con 0x7fac840b2050 2020-12-05T17:32:14.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.355+0000 7fac889f4700 1 -- 172.21.15.32:0/2784964363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac840639a0 con 0x7fac840b2050 2020-12-05T17:32:14.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.355+0000 7fac889f4700 1 -- 172.21.15.32:0/2784964363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac84143830 con 0x7fac840b2050 2020-12-05T17:32:14.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.355+0000 7fac7b7fe700 1 -- 172.21.15.32:0/2784964363 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fac74010070 con 0x7fac840b2050 2020-12-05T17:32:14.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.355+0000 7fac7b7fe700 1 --2- 172.21.15.32:0/2784964363 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fac680554e0 0x7fac680579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:14.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.355+0000 7fac8259c700 1 --2- 172.21.15.32:0/2784964363 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fac680554e0 0x7fac680579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:14.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.355+0000 7fac7b7fe700 1 -- 172.21.15.32:0/2784964363 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fac7406d7b0 con 0x7fac840b2050 2020-12-05T17:32:14.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.359+0000 7fac8259c700 1 --2- 172.21.15.32:0/2784964363 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fac680554e0 0x7fac680579a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fac7c00d000 tx=0x7fac7c007ee0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:14.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.363+0000 7fac7b7fe700 1 -- 172.21.15.32:0/2784964363 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fac740438b0 con 0x7fac840b2050 2020-12-05T17:32:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.511+0000 7fac889f4700 1 -- 172.21.15.32:0/2784964363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fac841427a0 con 0x7fac840b2050 2020-12-05T17:32:14.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.511+0000 7fac7b7fe700 1 -- 172.21.15.32:0/2784964363 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v25) v1 ==== 81+0+34065 (secure 0 0 0) 0x7fac74018070 con 0x7fac840b2050 2020-12-05T17:32:14.516 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:32:14.516 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:32:14.516 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 25, 2020-12-05T17:32:14.516 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", 2020-12-05T17:32:14.517 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2020-12-05T17:31:20.975238+0000", 2020-12-05T17:32:14.517 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2020-12-05T17:31:41.610914+0000", 2020-12-05T17:32:14.517 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:32:14.517 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:32:14.518 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-12-05T17:32:14.518 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2020-12-05T17:32:14.518 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2020-12-05T17:32:14.518 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2020-12-05T17:32:14.519 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2020-12-05T17:32:14.519 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2020-12-05T17:32:14.519 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2020-12-05T17:32:14.519 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:32:14.520 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2020-12-05T17:32:14.520 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2020-12-05T17:32:14.520 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-12-05T17:32:14.521 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2020-12-05T17:32:14.521 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2020-12-05T17:32:14.521 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 4, 2020-12-05T17:32:14.521 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2020-12-05T17:32:14.521 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2020-12-05T17:32:14.522 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2020-12-05T17:32:14.522 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2020-12-05T17:32:14.522 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2020-12-05T17:32:14.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.523 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2020-12-05T17:32:14.523 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2020-12-05T17:32:14.523 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:24.858732+0000", 2020-12-05T17:32:14.523 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:32:14.523 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:32:14.524 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:32:14.524 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:32:14.524 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:32:14.524 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:32:14.524 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:32:14.525 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:32:14.525 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2020-12-05T17:32:14.525 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2020-12-05T17:32:14.525 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2020-12-05T17:32:14.526 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2020-12-05T17:32:14.526 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2020-12-05T17:32:14.526 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:32:14.526 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:32:14.527 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:32:14.527 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:32:14.527 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:32:14.527 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:32:14.527 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:32:14.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.528 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "16", 2020-12-05T17:32:14.528 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:32:14.528 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:32:14.528 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:32:14.529 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:32:14.529 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:32:14.529 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:32:14.529 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:32:14.529 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:32:14.530 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:32:14.530 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:32:14.530 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:32:14.530 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:32:14.531 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:32:14.531 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:32:14.531 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:32:14.531 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:32:14.531 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:32:14.532 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:32:14.532 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:32:14.532 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:32:14.532 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:32:14.532 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:32:14.533 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:32:14.533 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:32:14.533 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:32:14.533 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:32:14.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.534 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:32:14.534 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:32:14.534 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:32:14.534 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:32:14.535 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:32:14.535 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:32:14.535 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:32:14.535 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:32:14.535 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:32:14.536 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:32:14.536 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:32:14.536 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:32:14.536 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2020-12-05T17:32:14.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.537 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:32:14.537 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2020-12-05T17:32:14.537 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.538 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2020-12-05T17:32:14.538 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2020-12-05T17:32:14.539 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:33.305595+0000", 2020-12-05T17:32:14.539 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:32:14.539 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:32:14.539 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:32:14.540 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:32:14.540 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:32:14.540 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:32:14.540 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:32:14.540 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:32:14.541 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:32:14.541 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:32:14.541 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:32:14.541 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:32:14.542 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:32:14.542 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:32:14.542 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:32:14.542 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:32:14.543 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:32:14.543 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:32:14.543 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:32:14.543 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:32:14.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.544 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "19", 2020-12-05T17:32:14.544 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:32:14.544 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:32:14.544 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:32:14.545 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:32:14.545 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:32:14.545 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:32:14.545 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:32:14.546 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:32:14.546 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:32:14.546 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:32:14.546 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:32:14.547 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:32:14.547 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:32:14.547 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:32:14.547 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:32:14.548 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:32:14.548 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:32:14.548 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:32:14.548 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:32:14.549 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:32:14.549 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:32:14.549 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:32:14.549 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:32:14.550 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:32:14.550 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:32:14.550 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:32:14.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.551 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:32:14.551 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:32:14.551 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:32:14.551 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:32:14.552 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:32:14.552 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:32:14.552 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:32:14.552 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:32:14.552 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:32:14.553 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:32:14.553 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:32:14.553 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:32:14.553 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:32:14.554 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2020-12-05T17:32:14.554 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.554 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2020-12-05T17:32:14.555 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2020-12-05T17:32:14.555 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:36.706361+0000", 2020-12-05T17:32:14.555 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:32:14.555 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:32:14.556 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:32:14.556 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:32:14.556 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:32:14.556 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:32:14.557 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:32:14.557 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:32:14.557 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:32:14.557 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:32:14.557 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:32:14.558 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:32:14.558 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:32:14.558 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:32:14.558 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:32:14.559 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:32:14.559 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:32:14.559 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:32:14.559 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:32:14.560 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:32:14.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.560 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "25", 2020-12-05T17:32:14.560 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:32:14.560 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:32:14.561 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:32:14.561 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:32:14.561 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:32:14.561 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:32:14.562 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:32:14.562 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:32:14.562 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:32:14.562 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:32:14.563 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:32:14.563 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:32:14.563 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:32:14.563 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:32:14.563 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:32:14.564 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:32:14.564 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:32:14.564 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:32:14.564 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:32:14.565 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:32:14.565 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:32:14.565 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:32:14.565 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:32:14.565 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:32:14.566 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:32:14.566 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:32:14.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.566 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:32:14.567 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:32:14.567 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:32:14.567 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:32:14.567 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:32:14.567 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:32:14.568 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:32:14.568 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:32:14.568 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:32:14.568 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:32:14.569 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:32:14.569 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:32:14.569 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2020-12-05T17:32:14.569 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2020-12-05T17:32:14.570 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2020-12-05T17:32:14.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.570 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:32:14.570 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:32:14.571 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2020-12-05T17:32:14.571 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.571 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.572 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 4, 2020-12-05T17:32:14.572 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2020-12-05T17:32:14.572 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:39.298269+0000", 2020-12-05T17:32:14.572 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 5, 2020-12-05T17:32:14.573 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,ec_overwrites", 2020-12-05T17:32:14.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 3, 2020-12-05T17:32:14.573 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 4, 2020-12-05T17:32:14.573 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 3, 2020-12-05T17:32:14.573 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 1, 2020-12-05T17:32:14.574 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:32:14.574 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:32:14.574 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:32:14.574 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:32:14.574 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:32:14.575 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:32:14.575 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:32:14.575 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:32:14.575 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:32:14.576 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:32:14.576 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:32:14.576 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:32:14.576 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:32:14.577 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:32:14.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.577 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "25", 2020-12-05T17:32:14.577 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:32:14.578 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:32:14.578 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:32:14.578 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:32:14.578 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:32:14.579 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:32:14.579 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:32:14.579 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:32:14.579 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:32:14.580 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:32:14.580 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:32:14.580 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:32:14.580 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:32:14.580 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:32:14.581 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:32:14.581 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:32:14.581 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:32:14.581 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:32:14.582 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:32:14.582 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:32:14.582 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:32:14.582 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:32:14.583 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:32:14.583 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "cephfs_data", 2020-12-05T17:32:14.583 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:32:14.583 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:32:14.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.584 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:32:14.584 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:32:14.584 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:32:14.584 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:32:14.584 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:32:14.585 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:32:14.585 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:32:14.585 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:32:14.585 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 8192, 2020-12-05T17:32:14.585 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:32:14.586 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:32:14.586 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:32:14.586 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:32:14.586 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:32:14.587 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2020-12-05T17:32:14.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.587 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:32:14.588 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2020-12-05T17:32:14.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.588 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:32:14.588 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "b941127b-0a49-4e06-8d11-17794ff8b013", 2020-12-05T17:32:14.588 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:32:14.589 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:32:14.589 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:32:14.589 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:32:14.589 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:32:14.590 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:32:14.590 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:32:14.590 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 23, 2020-12-05T17:32:14.590 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:32:14.590 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:32:14.591 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:32:14.591 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.591 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2020-12-05T17:32:14.592 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:32:14.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.592 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.592 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2020-12-05T17:32:14.593 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:32:14.593 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.593 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.593 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:32:14.594 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.594 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.594 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2020-12-05T17:32:14.594 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:32:14.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.595 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.595 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2020-12-05T17:32:14.595 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:32:14.596 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.596 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.596 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:32:14.597 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.597 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.597 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2020-12-05T17:32:14.597 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:32:14.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.598 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.598 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2020-12-05T17:32:14.598 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:32:14.599 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.599 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.599 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:32:14.599 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.600 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.600 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2020-12-05T17:32:14.600 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:32:14.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.601 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.601 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2020-12-05T17:32:14.601 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:32:14.602 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.602 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.602 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6803/11526", 2020-12-05T17:32:14.602 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6805/11526", 2020-12-05T17:32:14.603 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6809/11526", 2020-12-05T17:32:14.603 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6807/11526", 2020-12-05T17:32:14.603 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:32:14.603 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:32:14.604 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:32:14.604 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.604 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:32:14.605 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "01bd7ff5-a86b-4e11-ba1e-484c309c0fba", 2020-12-05T17:32:14.605 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:32:14.605 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:32:14.605 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:32:14.606 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:32:14.606 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:32:14.606 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:32:14.606 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:32:14.607 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 23, 2020-12-05T17:32:14.607 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:32:14.607 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:32:14.607 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:32:14.608 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.608 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.608 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2020-12-05T17:32:14.608 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:32:14.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.609 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.609 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2020-12-05T17:32:14.609 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:32:14.610 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.610 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.610 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:32:14.610 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.611 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.611 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2020-12-05T17:32:14.611 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:32:14.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.612 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.612 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2020-12-05T17:32:14.612 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:32:14.613 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.613 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.613 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:32:14.613 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.614 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.614 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2020-12-05T17:32:14.614 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:32:14.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.615 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.615 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2020-12-05T17:32:14.615 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:32:14.615 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.616 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.616 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:32:14.616 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.617 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.617 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2020-12-05T17:32:14.617 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:32:14.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.618 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.618 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2020-12-05T17:32:14.618 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:32:14.618 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.619 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.619 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6819/11538", 2020-12-05T17:32:14.619 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6821/11538", 2020-12-05T17:32:14.619 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6825/11538", 2020-12-05T17:32:14.620 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6823/11538", 2020-12-05T17:32:14.620 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:32:14.620 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:32:14.620 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:32:14.620 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.621 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:32:14.621 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "94736ef5-64c2-4a97-b963-bd38cddd9c52", 2020-12-05T17:32:14.622 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:32:14.622 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:32:14.622 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:32:14.622 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:32:14.622 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:32:14.623 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:32:14.623 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:32:14.623 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 17, 2020-12-05T17:32:14.623 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:32:14.623 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:32:14.624 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:32:14.624 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.624 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.624 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2020-12-05T17:32:14.625 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:32:14.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.625 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.626 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2020-12-05T17:32:14.626 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:32:14.626 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.626 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.627 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:32:14.627 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.627 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.627 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2020-12-05T17:32:14.628 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:32:14.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.628 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.629 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2020-12-05T17:32:14.629 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:32:14.629 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.629 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.630 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:32:14.630 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.630 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.630 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2020-12-05T17:32:14.631 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:32:14.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.631 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.632 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2020-12-05T17:32:14.632 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:32:14.632 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.632 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.633 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:32:14.633 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.633 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.633 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2020-12-05T17:32:14.634 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:32:14.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.634 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.634 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2020-12-05T17:32:14.635 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:32:14.635 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.635 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.636 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6811/11521", 2020-12-05T17:32:14.636 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6813/11521", 2020-12-05T17:32:14.636 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6817/11521", 2020-12-05T17:32:14.636 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6815/11521", 2020-12-05T17:32:14.637 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:32:14.637 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:32:14.637 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:32:14.637 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.638 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:32:14.638 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "69a1f3ff-fd25-43df-b60c-e32a89c35234", 2020-12-05T17:32:14.638 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:32:14.638 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:32:14.639 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:32:14.639 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:32:14.639 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:32:14.639 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:32:14.640 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:32:14.640 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 23, 2020-12-05T17:32:14.640 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:32:14.640 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:32:14.640 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:32:14.641 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.641 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.641 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2020-12-05T17:32:14.641 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:32:14.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.642 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.642 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2020-12-05T17:32:14.643 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:32:14.643 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.643 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.643 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:32:14.644 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.644 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.644 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2020-12-05T17:32:14.644 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:32:14.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.645 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.645 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2020-12-05T17:32:14.645 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:32:14.646 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.646 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.646 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:32:14.647 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.647 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.647 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6832", 2020-12-05T17:32:14.647 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:32:14.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.648 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.648 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6833", 2020-12-05T17:32:14.648 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:32:14.649 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.649 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.649 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:32:14.649 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.650 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.650 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2020-12-05T17:32:14.650 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:32:14.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.651 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.651 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2020-12-05T17:32:14.651 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:32:14.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.652 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.652 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6827/11535", 2020-12-05T17:32:14.652 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6829/11535", 2020-12-05T17:32:14.653 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6833/11535", 2020-12-05T17:32:14.653 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6831/11535", 2020-12-05T17:32:14.653 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:32:14.653 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:32:14.653 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:32:14.654 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.654 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:32:14.654 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c301fe4e-5eab-402c-ae6f-db3e6a7181af", 2020-12-05T17:32:14.655 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:32:14.655 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:32:14.655 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:32:14.655 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:32:14.655 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:32:14.656 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:32:14.656 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:32:14.656 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 23, 2020-12-05T17:32:14.656 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:32:14.656 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:32:14.657 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:32:14.657 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.657 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.657 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6808", 2020-12-05T17:32:14.658 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:32:14.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.658 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.659 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6809", 2020-12-05T17:32:14.659 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:32:14.659 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.659 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.660 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:32:14.660 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.660 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.660 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6810", 2020-12-05T17:32:14.661 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:32:14.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.661 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.661 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6811", 2020-12-05T17:32:14.662 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:32:14.662 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.662 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.662 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:32:14.663 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.663 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.663 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6817", 2020-12-05T17:32:14.664 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:32:14.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.664 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.664 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6819", 2020-12-05T17:32:14.665 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:32:14.665 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.665 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.665 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:32:14.666 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.666 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.666 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6812", 2020-12-05T17:32:14.666 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:32:14.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.667 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.667 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6814", 2020-12-05T17:32:14.667 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:32:14.668 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.668 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.668 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6809/11541", 2020-12-05T17:32:14.668 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6811/11541", 2020-12-05T17:32:14.669 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6819/11541", 2020-12-05T17:32:14.669 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6814/11541", 2020-12-05T17:32:14.669 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:32:14.669 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:32:14.669 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:32:14.670 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.670 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:32:14.670 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "9476eff4-345b-46c5-a473-1d9a89a6d327", 2020-12-05T17:32:14.671 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:32:14.671 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:32:14.671 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:32:14.671 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:32:14.671 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:32:14.672 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:32:14.672 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:32:14.672 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 20, 2020-12-05T17:32:14.672 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:32:14.672 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:32:14.673 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:32:14.673 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.673 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.673 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6800", 2020-12-05T17:32:14.674 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:32:14.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.674 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.674 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6801", 2020-12-05T17:32:14.675 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:32:14.675 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.675 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.675 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:32:14.676 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.676 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.676 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6802", 2020-12-05T17:32:14.676 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:32:14.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.677 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.677 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6803", 2020-12-05T17:32:14.678 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:32:14.678 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.678 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.678 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:32:14.679 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.679 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.679 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6806", 2020-12-05T17:32:14.679 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:32:14.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.680 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.680 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6807", 2020-12-05T17:32:14.680 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:32:14.681 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.681 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.681 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:32:14.681 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.682 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.682 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6804", 2020-12-05T17:32:14.682 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:32:14.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.683 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.683 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6805", 2020-12-05T17:32:14.683 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:32:14.684 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.684 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.684 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6801/11539", 2020-12-05T17:32:14.684 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6803/11539", 2020-12-05T17:32:14.685 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6807/11539", 2020-12-05T17:32:14.685 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6805/11539", 2020-12-05T17:32:14.685 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:32:14.685 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:32:14.685 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:32:14.686 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.686 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:32:14.686 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8dfd8567-9b1c-4d13-9c96-071183e469c2", 2020-12-05T17:32:14.687 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:32:14.687 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:32:14.687 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:32:14.687 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:32:14.687 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:32:14.688 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:32:14.688 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:32:14.688 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 23, 2020-12-05T17:32:14.688 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:32:14.689 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:32:14.689 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:32:14.689 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.689 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.690 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6820", 2020-12-05T17:32:14.690 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:32:14.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.690 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.691 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6822", 2020-12-05T17:32:14.691 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:32:14.691 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.691 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.692 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:32:14.692 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.692 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.692 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6823", 2020-12-05T17:32:14.693 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:32:14.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.693 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.694 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6825", 2020-12-05T17:32:14.694 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:32:14.694 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.694 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.695 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:32:14.695 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.695 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.696 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6829", 2020-12-05T17:32:14.696 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:32:14.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.696 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.697 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6830", 2020-12-05T17:32:14.697 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:32:14.697 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.697 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.698 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:32:14.698 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.698 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.698 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6826", 2020-12-05T17:32:14.699 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:32:14.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.699 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.699 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6827", 2020-12-05T17:32:14.700 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:32:14.700 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.700 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.700 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6822/11542", 2020-12-05T17:32:14.701 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6825/11542", 2020-12-05T17:32:14.701 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6830/11542", 2020-12-05T17:32:14.701 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6827/11542", 2020-12-05T17:32:14.701 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:32:14.701 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:32:14.702 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:32:14.702 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.703 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:32:14.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "24d466b3-01dc-4ba2-8b81-85250d163bd2", 2020-12-05T17:32:14.703 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:32:14.703 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:32:14.703 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:32:14.704 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:32:14.704 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:32:14.704 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:32:14.704 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:32:14.704 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 17, 2020-12-05T17:32:14.705 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:32:14.705 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:32:14.705 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:32:14.705 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.706 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.706 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6813", 2020-12-05T17:32:14.706 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:32:14.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.707 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.707 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6815", 2020-12-05T17:32:14.707 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:32:14.707 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.708 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.708 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:32:14.708 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.709 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.709 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6816", 2020-12-05T17:32:14.709 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:32:14.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.710 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.710 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6818", 2020-12-05T17:32:14.710 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:32:14.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.711 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.711 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:32:14.711 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.712 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.712 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6828", 2020-12-05T17:32:14.712 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:32:14.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.713 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.713 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6831", 2020-12-05T17:32:14.713 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:32:14.713 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.713 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.714 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:32:14.714 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:32:14.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.714 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:32:14.715 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6821", 2020-12-05T17:32:14.715 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:32:14.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.715 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:32:14.716 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6824", 2020-12-05T17:32:14.716 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:32:14.716 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.716 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.717 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6815/11540", 2020-12-05T17:32:14.717 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6818/11540", 2020-12-05T17:32:14.717 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6831/11540", 2020-12-05T17:32:14.717 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6824/11540", 2020-12-05T17:32:14.717 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:32:14.718 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:32:14.718 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:32:14.718 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:14.718 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.718 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:32:14.719 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2020-12-05T17:32:14.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.719 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:32:14.719 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:32:14.720 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:32:14.720 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:32:14.720 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:32:14.720 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:32:14.720 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.398868+0000", 2020-12-05T17:32:14.721 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:32:14.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.721 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:32:14.721 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:32:14.722 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:32:14.722 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:32:14.722 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:32:14.722 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:32:14.722 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.675748+0000", 2020-12-05T17:32:14.723 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:32:14.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.723 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:32:14.724 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:32:14.724 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:32:14.724 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:32:14.724 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:32:14.724 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:32:14.725 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.620888+0000", 2020-12-05T17:32:14.725 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:32:14.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.725 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:32:14.726 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:32:14.726 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:32:14.726 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:32:14.726 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:32:14.726 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:32:14.727 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.681600+0000", 2020-12-05T17:32:14.727 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:32:14.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.728 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:32:14.728 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:32:14.728 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:32:14.728 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:32:14.728 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:32:14.729 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:32:14.729 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.658769+0000", 2020-12-05T17:32:14.729 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:32:14.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.730 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:32:14.730 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:32:14.730 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:32:14.730 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:32:14.731 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:32:14.731 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:32:14.731 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.630708+0000", 2020-12-05T17:32:14.731 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:32:14.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.732 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:32:14.732 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:32:14.732 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:32:14.733 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:32:14.733 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:32:14.733 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:32:14.733 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.648352+0000", 2020-12-05T17:32:14.733 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:32:14.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:14.734 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:32:14.734 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:32:14.735 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:32:14.735 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:32:14.735 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:32:14.735 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:32:14.735 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.629399+0000", 2020-12-05T17:32:14.736 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:32:14.736 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.736 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:32:14.736 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2020-12-05T17:32:14.737 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2020-12-05T17:32:14.737 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2020-12-05T17:32:14.737 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2020-12-05T17:32:14.737 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": {}, 2020-12-05T17:32:14.738 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2020-12-05T17:32:14.738 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2020-12-05T17:32:14.738 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2020-12-05T17:32:14.738 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:32:14.738 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2020-12-05T17:32:14.739 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2020-12-05T17:32:14.739 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:32:14.739 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2020-12-05T17:32:14.739 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:32:14.740 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:32:14.740 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2020-12-05T17:32:14.740 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2020-12-05T17:32:14.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.740 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2020-12-05T17:32:14.741 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:32:14.741 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:32:14.741 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2020-12-05T17:32:14.741 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:32:14.741 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:32:14.742 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2020-12-05T17:32:14.742 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:14.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:14.742 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2020-12-05T17:32:14.743 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2020-12-05T17:32:14.743 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2020-12-05T17:32:14.743 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2020-12-05T17:32:14.743 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2020-12-05T17:32:14.743 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:32:14.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.515+0000 7fac889f4700 1 -- 172.21.15.32:0/2784964363 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fac680554e0 msgr2=0x7fac680579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:14.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.515+0000 7fac889f4700 1 --2- 172.21.15.32:0/2784964363 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fac680554e0 0x7fac680579a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fac7c00d000 tx=0x7fac7c007ee0).stop 2020-12-05T17:32:14.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.515+0000 7fac889f4700 1 -- 172.21.15.32:0/2784964363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac840b2050 msgr2=0x7fac84066000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:14.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.515+0000 7fac889f4700 1 --2- 172.21.15.32:0/2784964363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac840b2050 0x7fac84066000 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fac740010d0 tx=0x7fac74007af0).stop 2020-12-05T17:32:14.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.515+0000 7fac889f4700 1 -- 172.21.15.32:0/2784964363 shutdown_connections 2020-12-05T17:32:14.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.515+0000 7fac889f4700 1 --2- 172.21.15.32:0/2784964363 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac841400e0 0x7fac84064fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:14.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.515+0000 7fac889f4700 1 --2- 172.21.15.32:0/2784964363 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fac680554e0 0x7fac680579a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:14.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.515+0000 7fac889f4700 1 --2- 172.21.15.32:0/2784964363 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac840aedc0 0x7fac841466b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:14.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.515+0000 7fac889f4700 1 --2- 172.21.15.32:0/2784964363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac840b2050 0x7fac84066000 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:14.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.515+0000 7fac889f4700 1 -- 172.21.15.32:0/2784964363 >> 172.21.15.32:0/2784964363 conn(0x7fac840b6ef0 msgr2=0x7fac840b72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:14.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.515+0000 7fac889f4700 1 -- 172.21.15.32:0/2784964363 shutdown_connections 2020-12-05T17:32:14.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.515+0000 7fac889f4700 1 -- 172.21.15.32:0/2784964363 wait complete. 2020-12-05T17:32:14.748 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:32:14.764 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:32:14.767 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:32:14.794 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:32:14.796 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:32:14.798 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:32:14.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.867+0000 7f499962e700 1 Processor -- start 2020-12-05T17:32:14.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.867+0000 7f499962e700 1 -- start start 2020-12-05T17:32:14.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f499962e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f499405feb0 0x7f499405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:14.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f499962e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4994142450 con 0x7f4994062400 2020-12-05T17:32:14.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f499962e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4994061780 con 0x7f499405f3b0 2020-12-05T17:32:14.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f499962e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4994061a20 con 0x7f499405feb0 2020-12-05T17:32:14.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f49937fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4994062400 0x7f49940b0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36206/0 (socket says 172.21.15.32:36206) 2020-12-05T17:32:14.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f49937fe700 1 -- 172.21.15.32:0/3934023374 learned_addr learned my addr 172.21.15.32:0/3934023374 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:14.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f4998e2d700 1 --2- 172.21.15.32:0/3934023374 >> v2:172.21.15.32:3300/0 conn(0x7f499405feb0 0x7f499405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:14.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f4992ffd700 1 -- 172.21.15.32:0/3934023374 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3551805190 0 0) 0x7f4994142450 con 0x7f4994062400 2020-12-05T17:32:14.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f4992ffd700 1 -- 172.21.15.32:0/3934023374 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4974003230 con 0x7f4994062400 2020-12-05T17:32:14.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f4998e2d700 1 -- 172.21.15.32:0/3934023374 >> v1:172.21.15.61:6789/0 conn(0x7f499405f3b0 legacy=0x7f49940b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:14.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f4998e2d700 1 -- 172.21.15.32:0/3934023374 >> v1:172.21.15.32:6789/0 conn(0x7f4994062400 legacy=0x7f49940b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:14.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f4998e2d700 1 -- 172.21.15.32:0/3934023374 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4994061cc0 con 0x7f499405feb0 2020-12-05T17:32:14.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f4998e2d700 1 --2- 172.21.15.32:0/3934023374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499405feb0 0x7f499405f790 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f4984002f30 tx=0x7f4984006dc0).ready entity=mon.0 client_cookie=5de257b6f65166cc server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:14.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f4992ffd700 1 -- 172.21.15.32:0/3934023374 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4984009800 con 0x7f499405feb0 2020-12-05T17:32:14.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f4992ffd700 1 -- 172.21.15.32:0/3934023374 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4984009960 con 0x7f499405feb0 2020-12-05T17:32:14.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f4992ffd700 1 -- 172.21.15.32:0/3934023374 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4984009cc0 con 0x7f499405feb0 2020-12-05T17:32:14.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f499962e700 1 -- 172.21.15.32:0/3934023374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499405feb0 msgr2=0x7f499405f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:14.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f499962e700 1 --2- 172.21.15.32:0/3934023374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499405feb0 0x7f499405f790 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f4984002f30 tx=0x7f4984006dc0).stop 2020-12-05T17:32:14.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f499962e700 1 -- 172.21.15.32:0/3934023374 shutdown_connections 2020-12-05T17:32:14.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f499962e700 1 --2- 172.21.15.32:0/3934023374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499405feb0 0x7f499405f790 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:14.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f499962e700 1 -- 172.21.15.32:0/3934023374 >> 172.21.15.32:0/3934023374 conn(0x7f49940b8ee0 msgr2=0x7f49940b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:14.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f499962e700 1 -- 172.21.15.32:0/3934023374 shutdown_connections 2020-12-05T17:32:14.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f499962e700 1 -- 172.21.15.32:0/3934023374 wait complete. 2020-12-05T17:32:14.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f499962e700 1 Processor -- start 2020-12-05T17:32:14.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.871+0000 7f499962e700 1 -- start start 2020-12-05T17:32:14.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f499962e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f499405f3b0 0x7f4994066a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f499962e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499405feb0 0x7f4994063470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f499962e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4994062400 0x7f4994151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:14.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f499962e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49941447b0 con 0x7f499405feb0 2020-12-05T17:32:14.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f499962e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49941449e0 con 0x7f499405f3b0 2020-12-05T17:32:14.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f499962e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49940af9b0 con 0x7f4994062400 2020-12-05T17:32:14.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f49937fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499405feb0 0x7f4994063470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:14.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f49937fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499405feb0 0x7f4994063470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33564/0 (socket says 172.21.15.32:33564) 2020-12-05T17:32:14.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f49937fe700 1 -- 172.21.15.32:0/3476224309 learned_addr learned my addr 172.21.15.32:0/3476224309 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:14.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f4998e2d700 1 --2- 172.21.15.32:0/3476224309 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4994062400 0x7f4994151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:14.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f4993fff700 1 --2- 172.21.15.32:0/3476224309 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f499405f3b0 0x7f4994066a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:14.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f49937fe700 1 -- 172.21.15.32:0/3476224309 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4994062400 msgr2=0x7f4994151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:14.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f49937fe700 1 --2- 172.21.15.32:0/3476224309 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4994062400 0x7f4994151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:14.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f49937fe700 1 -- 172.21.15.32:0/3476224309 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f499405f3b0 msgr2=0x7f4994066a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:14.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f49937fe700 1 --2- 172.21.15.32:0/3476224309 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f499405f3b0 0x7f4994066a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:14.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f49937fe700 1 -- 172.21.15.32:0/3476224309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49941512c0 con 0x7f499405feb0 2020-12-05T17:32:14.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f4993fff700 1 --2- 172.21.15.32:0/3476224309 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f499405f3b0 0x7f4994066a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:14.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f49937fe700 1 --2- 172.21.15.32:0/3476224309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499405feb0 0x7f4994063470 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f497c001450 tx=0x7f497c007e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:14.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f49917fa700 1 -- 172.21.15.32:0/3476224309 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f497c005b70 con 0x7f499405feb0 2020-12-05T17:32:14.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f49917fa700 1 -- 172.21.15.32:0/3476224309 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497c005cd0 con 0x7f499405feb0 2020-12-05T17:32:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f49917fa700 1 -- 172.21.15.32:0/3476224309 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f497c007300 con 0x7f499405feb0 2020-12-05T17:32:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f499962e700 1 -- 172.21.15.32:0/3476224309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49941514b0 con 0x7f499405feb0 2020-12-05T17:32:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f499962e700 1 -- 172.21.15.32:0/3476224309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4994146230 con 0x7f499405feb0 2020-12-05T17:32:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f499962e700 1 -- 172.21.15.32:0/3476224309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4994057050 con 0x7f499405feb0 2020-12-05T17:32:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.875+0000 7f49917fa700 1 -- 172.21.15.32:0/3476224309 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f497c010030 con 0x7f499405feb0 2020-12-05T17:32:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.879+0000 7f49917fa700 1 --2- 172.21.15.32:0/3476224309 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49780553b0 0x7f4978057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.879+0000 7f4993fff700 1 --2- 172.21.15.32:0/3476224309 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49780553b0 0x7f4978057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.879+0000 7f49917fa700 1 -- 172.21.15.32:0/3476224309 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f497c06e170 con 0x7f499405feb0 2020-12-05T17:32:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.879+0000 7f4993fff700 1 --2- 172.21.15.32:0/3476224309 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49780553b0 0x7f4978057870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f498c00b000 tx=0x7f498c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:14.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:14.883+0000 7f49917fa700 1 -- 172.21.15.32:0/3476224309 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f497c044310 con 0x7f499405feb0 2020-12-05T17:32:15.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.067+0000 7f499962e700 1 -- 172.21.15.32:0/3476224309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49940008d0 con 0x7f499405feb0 2020-12-05T17:32:15.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.067+0000 7f49917fa700 1 -- 172.21.15.32:0/3476224309 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2134 (secure 0 0 0) 0x7f497c047930 con 0x7f499405feb0 2020-12-05T17:32:15.072 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:32:15.072 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4023797731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4023797731},{"type":"v1","addr":"172.21.15.61:6833","nonce":4023797731}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:31:41.611214+0000","modified":"2020-12-05T17:31:42.614053+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3503848021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3503848021},{"type":"v1","addr":"172.21.15.32:6835","nonce":3503848021}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-12-05T17:32:15.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.071+0000 7f499962e700 1 -- 172.21.15.32:0/3476224309 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49780553b0 msgr2=0x7f4978057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:15.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.071+0000 7f499962e700 1 --2- 172.21.15.32:0/3476224309 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49780553b0 0x7f4978057870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f498c00b000 tx=0x7f498c009040).stop 2020-12-05T17:32:15.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.071+0000 7f499962e700 1 -- 172.21.15.32:0/3476224309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499405feb0 msgr2=0x7f4994063470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:15.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.071+0000 7f499962e700 1 --2- 172.21.15.32:0/3476224309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499405feb0 0x7f4994063470 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f497c001450 tx=0x7f497c007e40).stop 2020-12-05T17:32:15.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.071+0000 7f499962e700 1 -- 172.21.15.32:0/3476224309 shutdown_connections 2020-12-05T17:32:15.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.071+0000 7f499962e700 1 --2- 172.21.15.32:0/3476224309 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49780553b0 0x7f4978057870 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:15.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.071+0000 7f499962e700 1 --2- 172.21.15.32:0/3476224309 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4994062400 0x7f4994151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:15.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.071+0000 7f499962e700 1 --2- 172.21.15.32:0/3476224309 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f499405f3b0 0x7f4994066a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:15.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.071+0000 7f499962e700 1 --2- 172.21.15.32:0/3476224309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499405feb0 0x7f4994063470 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:15.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.071+0000 7f499962e700 1 -- 172.21.15.32:0/3476224309 >> 172.21.15.32:0/3476224309 conn(0x7f49940b8ee0 msgr2=0x7f49940b7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:15.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.071+0000 7f499962e700 1 -- 172.21.15.32:0/3476224309 shutdown_connections 2020-12-05T17:32:15.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.071+0000 7f499962e700 1 -- 172.21.15.32:0/3476224309 wait complete. 2020-12-05T17:32:15.076 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 5 2020-12-05T17:32:15.086 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-12-05T17:32:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.199+0000 7fbd068ef700 1 Processor -- start 2020-12-05T17:32:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.199+0000 7fbd068ef700 1 -- start start 2020-12-05T17:32:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.199+0000 7fbd068ef700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd001444d0 0x7fbd0014aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.199+0000 7fbd068ef700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd000b25a0 con 0x7fbd000afc50 2020-12-05T17:32:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.199+0000 7fbd068ef700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd00061a90 con 0x7fbd000b21c0 2020-12-05T17:32:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.199+0000 7fbd068ef700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd00061d30 con 0x7fbd001444d0 2020-12-05T17:32:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.199+0000 7fbcff7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbd000afc50 0x7fbd0014a3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36218/0 (socket says 172.21.15.32:36218) 2020-12-05T17:32:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.199+0000 7fbcff7fe700 1 -- 172.21.15.32:0/2692797690 learned_addr learned my addr 172.21.15.32:0/2692797690 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.199+0000 7fbd04e8c700 1 --2- 172.21.15.32:0/2692797690 >> v2:172.21.15.32:3300/0 conn(0x7fbd001444d0 0x7fbd0014aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.199+0000 7fbcfeffd700 1 -- 172.21.15.32:0/2692797690 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3915036083 0 0) 0x7fbd000b25a0 con 0x7fbd000afc50 2020-12-05T17:32:15.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.199+0000 7fbcfeffd700 1 -- 172.21.15.32:0/2692797690 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbce0003230 con 0x7fbd000afc50 2020-12-05T17:32:15.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.203+0000 7fbd04e8c700 1 -- 172.21.15.32:0/2692797690 >> v1:172.21.15.61:6789/0 conn(0x7fbd000b21c0 legacy=0x7fbd00143dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:15.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.203+0000 7fbcfeffd700 1 -- 172.21.15.32:0/2692797690 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2525872519 0 0) 0x7fbd00061a90 con 0x7fbd000b21c0 2020-12-05T17:32:15.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.203+0000 7fbd04e8c700 1 -- 172.21.15.32:0/2692797690 >> v1:172.21.15.32:6789/0 conn(0x7fbd000afc50 legacy=0x7fbd0014a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:15.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.203+0000 7fbd04e8c700 1 -- 172.21.15.32:0/2692797690 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd00062a90 con 0x7fbd001444d0 2020-12-05T17:32:15.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.203+0000 7fbd04e8c700 1 --2- 172.21.15.32:0/2692797690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd001444d0 0x7fbd0014aaf0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7fbce8001bf0 tx=0x7fbcf8006dc0).ready entity=mon.0 client_cookie=e88c2b21f29a9e80 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:15.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.203+0000 7fbcfeffd700 1 -- 172.21.15.32:0/2692797690 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbce8001a50 con 0x7fbd001444d0 2020-12-05T17:32:15.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.203+0000 7fbcfeffd700 1 -- 172.21.15.32:0/2692797690 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf8009670 con 0x7fbd001444d0 2020-12-05T17:32:15.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.203+0000 7fbcfeffd700 1 -- 172.21.15.32:0/2692797690 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcf80097e0 con 0x7fbd001444d0 2020-12-05T17:32:15.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.203+0000 7fbd068ef700 1 -- 172.21.15.32:0/2692797690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd001444d0 msgr2=0x7fbd0014aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:15.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.203+0000 7fbd068ef700 1 --2- 172.21.15.32:0/2692797690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd001444d0 0x7fbd0014aaf0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7fbce8001bf0 tx=0x7fbcf8006dc0).stop 2020-12-05T17:32:15.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.203+0000 7fbd068ef700 1 -- 172.21.15.32:0/2692797690 shutdown_connections 2020-12-05T17:32:15.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.203+0000 7fbd068ef700 1 --2- 172.21.15.32:0/2692797690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd001444d0 0x7fbd0014aaf0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:15.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.203+0000 7fbd068ef700 1 -- 172.21.15.32:0/2692797690 >> 172.21.15.32:0/2692797690 conn(0x7fbd000b57a0 msgr2=0x7fbd000b1770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:15.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.203+0000 7fbd068ef700 1 -- 172.21.15.32:0/2692797690 shutdown_connections 2020-12-05T17:32:15.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.203+0000 7fbd068ef700 1 -- 172.21.15.32:0/2692797690 wait complete. 2020-12-05T17:32:15.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbd068ef700 1 Processor -- start 2020-12-05T17:32:15.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbd068ef700 1 -- start start 2020-12-05T17:32:15.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbd068ef700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbd000afc50 0x7fbd000657b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:15.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbd068ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd000b21c0 0x7fbd00149270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:15.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbd068ef700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbd001444d0 0x7fbd00149760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:15.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbd068ef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd00143aa0 con 0x7fbd000b21c0 2020-12-05T17:32:15.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbd068ef700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd00063d70 con 0x7fbd000afc50 2020-12-05T17:32:15.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbd068ef700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd00063830 con 0x7fbd001444d0 2020-12-05T17:32:15.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbcff7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd000b21c0 0x7fbd00149270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:15.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbd04e8c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbd001444d0 0x7fbd00149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:15.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbcff7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd000b21c0 0x7fbd00149270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33578/0 (socket says 172.21.15.32:33578) 2020-12-05T17:32:15.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbcff7fe700 1 -- 172.21.15.32:0/1770409944 learned_addr learned my addr 172.21.15.32:0/1770409944 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:15.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbcfffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbd000afc50 0x7fbd000657b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:15.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbcff7fe700 1 -- 172.21.15.32:0/1770409944 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbd001444d0 msgr2=0x7fbd00149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:15.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbcff7fe700 1 --2- 172.21.15.32:0/1770409944 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbd001444d0 0x7fbd00149760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:15.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbcff7fe700 1 -- 172.21.15.32:0/1770409944 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbd000afc50 msgr2=0x7fbd000657b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:15.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbcff7fe700 1 --2- 172.21.15.32:0/1770409944 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbd000afc50 0x7fbd000657b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:15.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbcff7fe700 1 -- 172.21.15.32:0/1770409944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd00063ad0 con 0x7fbd000b21c0 2020-12-05T17:32:15.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbcff7fe700 1 --2- 172.21.15.32:0/1770409944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd000b21c0 0x7fbd00149270 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fbce8001c30 tx=0x7fbce8001e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:15.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbcfd7fa700 1 -- 172.21.15.32:0/1770409944 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbce800ce60 con 0x7fbd000b21c0 2020-12-05T17:32:15.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbcfd7fa700 1 -- 172.21.15.32:0/1770409944 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce8004a80 con 0x7fbd000b21c0 2020-12-05T17:32:15.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbd068ef700 1 -- 172.21.15.32:0/1770409944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd0014b000 con 0x7fbd000b21c0 2020-12-05T17:32:15.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbcfd7fa700 1 -- 172.21.15.32:0/1770409944 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbce8004de0 con 0x7fbd000b21c0 2020-12-05T17:32:15.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.207+0000 7fbd068ef700 1 -- 172.21.15.32:0/1770409944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd00063fc0 con 0x7fbd000b21c0 2020-12-05T17:32:15.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.211+0000 7fbd068ef700 1 -- 172.21.15.32:0/1770409944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd00057050 con 0x7fbd000b21c0 2020-12-05T17:32:15.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.211+0000 7fbcfd7fa700 1 -- 172.21.15.32:0/1770409944 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbce8010070 con 0x7fbd000b21c0 2020-12-05T17:32:15.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.211+0000 7fbcfd7fa700 1 --2- 172.21.15.32:0/1770409944 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbce4055330 0x7fbce40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:15.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.211+0000 7fbcfd7fa700 1 -- 172.21.15.32:0/1770409944 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fbce806e090 con 0x7fbd000b21c0 2020-12-05T17:32:15.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.211+0000 7fbcfffff700 1 --2- 172.21.15.32:0/1770409944 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbce4055330 0x7fbce40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:15.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.211+0000 7fbcfffff700 1 --2- 172.21.15.32:0/1770409944 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbce4055330 0x7fbce40577f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fbcf000d000 tx=0x7fbcf0007a10).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:15.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.219+0000 7fbcfd7fa700 1 -- 172.21.15.32:0/1770409944 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbce8044230 con 0x7fbd000b21c0 2020-12-05T17:32:15.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.407+0000 7fbd068ef700 1 -- 172.21.15.32:0/1770409944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbd00069690 con 0x7fbd000b21c0 2020-12-05T17:32:15.913 INFO:tasks.ceph.mon.a.smithi032.stderr:2020-12-05T17:32:15.911+0000 7fdef8136700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-12-05T17:32:15.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.919+0000 7fbcfd7fa700 1 -- 172.21.15.32:0/1770409944 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v6) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbce8047850 con 0x7fbd000b21c0 2020-12-05T17:32:15.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.923+0000 7fbd068ef700 1 -- 172.21.15.32:0/1770409944 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbce4055330 msgr2=0x7fbce40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:15.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.923+0000 7fbd068ef700 1 --2- 172.21.15.32:0/1770409944 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbce4055330 0x7fbce40577f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fbcf000d000 tx=0x7fbcf0007a10).stop 2020-12-05T17:32:15.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.923+0000 7fbd068ef700 1 -- 172.21.15.32:0/1770409944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd000b21c0 msgr2=0x7fbd00149270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:15.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.923+0000 7fbd068ef700 1 --2- 172.21.15.32:0/1770409944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd000b21c0 0x7fbd00149270 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fbce8001c30 tx=0x7fbce8001e50).stop 2020-12-05T17:32:15.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.923+0000 7fbd068ef700 1 -- 172.21.15.32:0/1770409944 shutdown_connections 2020-12-05T17:32:15.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.923+0000 7fbd068ef700 1 --2- 172.21.15.32:0/1770409944 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbce4055330 0x7fbce40577f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:15.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.923+0000 7fbd068ef700 1 --2- 172.21.15.32:0/1770409944 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbd001444d0 0x7fbd00149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:15.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.923+0000 7fbd068ef700 1 --2- 172.21.15.32:0/1770409944 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbd000afc50 0x7fbd000657b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:15.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.923+0000 7fbd068ef700 1 --2- 172.21.15.32:0/1770409944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd000b21c0 0x7fbd00149270 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:15.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.923+0000 7fbd068ef700 1 -- 172.21.15.32:0/1770409944 >> 172.21.15.32:0/1770409944 conn(0x7fbd000b57a0 msgr2=0x7fbd00061e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:15.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.923+0000 7fbd068ef700 1 -- 172.21.15.32:0/1770409944 shutdown_connections 2020-12-05T17:32:15.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:15.923+0000 7fbd068ef700 1 -- 172.21.15.32:0/1770409944 wait complete. 2020-12-05T17:32:15.928 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-12-05T17:32:15.946 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:32:15.949 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:32:16.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.051+0000 7f50cede6700 1 Processor -- start 2020-12-05T17:32:16.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.051+0000 7f50cede6700 1 -- start start 2020-12-05T17:32:16.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50cede6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f50c80b52f0 0x7f50c80b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:16.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50cede6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f50c80ae860 0x7f50c80aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:16.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50cede6700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c81420f0 con 0x7f50c80b1870 2020-12-05T17:32:16.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50cede6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c80ad4f0 con 0x7f50c80ae860 2020-12-05T17:32:16.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50cede6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c80b0380 con 0x7f50c80b52f0 2020-12-05T17:32:16.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50ce5e5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f50c80ae860 0x7f50c80aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:16.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50ce5e5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f50c80ae860 0x7f50c80aecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33600/0 (socket says 172.21.15.32:33600) 2020-12-05T17:32:16.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50ce5e5700 1 -- 172.21.15.32:0/3663170360 learned_addr learned my addr 172.21.15.32:0/3663170360 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:16.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50cdde4700 1 --2- 172.21.15.32:0/3663170360 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f50c80b52f0 0x7f50c80b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:16.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50ce5e5700 1 -- 172.21.15.32:0/3663170360 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f50c80b52f0 msgr2=0x7f50c80b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:16.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50ce5e5700 1 --2- 172.21.15.32:0/3663170360 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f50c80b52f0 0x7f50c80b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:16.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50ce5e5700 1 -- 172.21.15.32:0/3663170360 >> v1:172.21.15.61:6789/0 conn(0x7f50c80b1870 legacy=0x7f50c8138810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:16.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50ce5e5700 1 -- 172.21.15.32:0/3663170360 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c80b0620 con 0x7f50c80ae860 2020-12-05T17:32:16.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50ce5e5700 1 --2- 172.21.15.32:0/3663170360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50c80ae860 0x7f50c80aecf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f50c00032b0 tx=0x7f50c0009270).ready entity=mon.0 client_cookie=a713e01ef7715e88 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:16.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50ccde2700 1 -- 172.21.15.32:0/3663170360 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50c000bc00 con 0x7f50c80ae860 2020-12-05T17:32:16.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50ccde2700 1 -- 172.21.15.32:0/3663170360 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c000bd60 con 0x7f50c80ae860 2020-12-05T17:32:16.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50ccde2700 1 -- 172.21.15.32:0/3663170360 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50c0011400 con 0x7f50c80ae860 2020-12-05T17:32:16.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50cede6700 1 -- 172.21.15.32:0/3663170360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50c80ae860 msgr2=0x7f50c80aecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:16.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50cede6700 1 --2- 172.21.15.32:0/3663170360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50c80ae860 0x7f50c80aecf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f50c00032b0 tx=0x7f50c0009270).stop 2020-12-05T17:32:16.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50cede6700 1 -- 172.21.15.32:0/3663170360 shutdown_connections 2020-12-05T17:32:16.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50cede6700 1 --2- 172.21.15.32:0/3663170360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50c80ae860 0x7f50c80aecf0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:16.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50cede6700 1 --2- 172.21.15.32:0/3663170360 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f50c80b52f0 0x7f50c80b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:16.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50cede6700 1 -- 172.21.15.32:0/3663170360 >> 172.21.15.32:0/3663170360 conn(0x7f50c805bbb0 msgr2=0x7f50c8062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:16.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50cede6700 1 -- 172.21.15.32:0/3663170360 shutdown_connections 2020-12-05T17:32:16.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.055+0000 7f50cede6700 1 -- 172.21.15.32:0/3663170360 wait complete. 2020-12-05T17:32:16.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.059+0000 7f50cede6700 1 Processor -- start 2020-12-05T17:32:16.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.059+0000 7f50cede6700 1 -- start start 2020-12-05T17:32:16.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.059+0000 7f50cede6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50c80ae860 0x7f50c80b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:16.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.059+0000 7f50cede6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f50c80b1870 0x7f50c8063f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:16.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.059+0000 7f50cede6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f50c80b52f0 0x7f50c8064470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:16.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.059+0000 7f50cede6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c80661f0 con 0x7f50c80ae860 2020-12-05T17:32:16.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.059+0000 7f50cede6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c8066440 con 0x7f50c80b1870 2020-12-05T17:32:16.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.059+0000 7f50cede6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c80666e0 con 0x7f50c80b52f0 2020-12-05T17:32:16.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.059+0000 7f50cdde4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50c80ae860 0x7f50c80b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:16.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.059+0000 7f50cdde4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50c80ae860 0x7f50c80b1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33602/0 (socket says 172.21.15.32:33602) 2020-12-05T17:32:16.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.059+0000 7f50cdde4700 1 -- 172.21.15.32:0/168047651 learned_addr learned my addr 172.21.15.32:0/168047651 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:16.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.059+0000 7f50cd5e3700 1 --2- 172.21.15.32:0/168047651 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f50c80b1870 0x7f50c8063f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:16.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.059+0000 7f50ce5e5700 1 --2- 172.21.15.32:0/168047651 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f50c80b52f0 0x7f50c8064470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:16.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.059+0000 7f50cdde4700 1 -- 172.21.15.32:0/168047651 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f50c80b52f0 msgr2=0x7f50c8064470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:16.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.059+0000 7f50cdde4700 1 --2- 172.21.15.32:0/168047651 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f50c80b52f0 0x7f50c8064470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:16.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.059+0000 7f50cdde4700 1 -- 172.21.15.32:0/168047651 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f50c80b1870 msgr2=0x7f50c8063f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:16.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.063+0000 7f50cdde4700 1 --2- 172.21.15.32:0/168047651 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f50c80b1870 0x7f50c8063f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:16.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.063+0000 7f50cdde4700 1 -- 172.21.15.32:0/168047651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c8066980 con 0x7f50c80ae860 2020-12-05T17:32:16.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.063+0000 7f50cd5e3700 1 --2- 172.21.15.32:0/168047651 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f50c80b1870 0x7f50c8063f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:16.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.063+0000 7f50cdde4700 1 --2- 172.21.15.32:0/168047651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50c80ae860 0x7f50c80b1f50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f50c4003070 tx=0x7f50c4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:16.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.063+0000 7f50b6ffd700 1 -- 172.21.15.32:0/168047651 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50c4009980 con 0x7f50c80ae860 2020-12-05T17:32:16.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.063+0000 7f50b6ffd700 1 -- 172.21.15.32:0/168047651 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c4009ae0 con 0x7f50c80ae860 2020-12-05T17:32:16.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.063+0000 7f50cede6700 1 -- 172.21.15.32:0/168047651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c80b2440 con 0x7f50c80ae860 2020-12-05T17:32:16.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.063+0000 7f50b6ffd700 1 -- 172.21.15.32:0/168047651 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50c4009e40 con 0x7f50c80ae860 2020-12-05T17:32:16.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.063+0000 7f50cede6700 1 -- 172.21.15.32:0/168047651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c8064960 con 0x7f50c80ae860 2020-12-05T17:32:16.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.063+0000 7f50cede6700 1 -- 172.21.15.32:0/168047651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c8057050 con 0x7f50c80ae860 2020-12-05T17:32:16.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.063+0000 7f50b6ffd700 1 -- 172.21.15.32:0/168047651 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f50c4013480 con 0x7f50c80ae860 2020-12-05T17:32:16.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.063+0000 7f50b6ffd700 1 --2- 172.21.15.32:0/168047651 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f50a8055330 0x7f50a80577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:16.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.063+0000 7f50cd5e3700 1 --2- 172.21.15.32:0/168047651 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f50a8055330 0x7f50a80577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:16.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.067+0000 7f50b6ffd700 1 -- 172.21.15.32:0/168047651 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6517+0+0 (secure 0 0 0) 0x7f50c406e850 con 0x7f50c80ae860 2020-12-05T17:32:16.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.067+0000 7f50cd5e3700 1 --2- 172.21.15.32:0/168047651 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f50a8055330 0x7f50a80577f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f50b800b000 tx=0x7f50b8009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:16.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.071+0000 7f50b6ffd700 1 -- 172.21.15.32:0/168047651 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f50c4044930 con 0x7f50c80ae860 2020-12-05T17:32:16.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.259+0000 7f50cede6700 1 -- 172.21.15.32:0/168047651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50c805c300 con 0x7f50c80ae860 2020-12-05T17:32:16.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.263+0000 7f50b6ffd700 1 -- 172.21.15.32:0/168047651 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+1774 (secure 0 0 0) 0x7f50c4047f50 con 0x7f50c80ae860 2020-12-05T17:32:16.265 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:32:16.265 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4023797731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4023797731},{"type":"v1","addr":"172.21.15.61:6833","nonce":4023797731}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:31:41.611214+0000","modified":"2020-12-05T17:32:15.915087+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-12-05T17:32:16.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.263+0000 7f50cede6700 1 -- 172.21.15.32:0/168047651 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f50a8055330 msgr2=0x7f50a80577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:16.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.263+0000 7f50cede6700 1 --2- 172.21.15.32:0/168047651 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f50a8055330 0x7f50a80577f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f50b800b000 tx=0x7f50b8009040).stop 2020-12-05T17:32:16.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.263+0000 7f50cede6700 1 -- 172.21.15.32:0/168047651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50c80ae860 msgr2=0x7f50c80b1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:16.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.263+0000 7f50cede6700 1 --2- 172.21.15.32:0/168047651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50c80ae860 0x7f50c80b1f50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f50c4003070 tx=0x7f50c4008040).stop 2020-12-05T17:32:16.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.263+0000 7f50cede6700 1 -- 172.21.15.32:0/168047651 shutdown_connections 2020-12-05T17:32:16.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.263+0000 7f50cede6700 1 --2- 172.21.15.32:0/168047651 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f50a8055330 0x7f50a80577f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:16.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.263+0000 7f50cede6700 1 --2- 172.21.15.32:0/168047651 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f50c80b52f0 0x7f50c8064470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:16.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.263+0000 7f50cede6700 1 --2- 172.21.15.32:0/168047651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50c80ae860 0x7f50c80b1f50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:16.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.263+0000 7f50cede6700 1 --2- 172.21.15.32:0/168047651 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f50c80b1870 0x7f50c8063f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:16.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.263+0000 7f50cede6700 1 -- 172.21.15.32:0/168047651 >> 172.21.15.32:0/168047651 conn(0x7f50c805bbb0 msgr2=0x7f50c80b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:16.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.263+0000 7f50cede6700 1 -- 172.21.15.32:0/168047651 shutdown_connections 2020-12-05T17:32:16.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.263+0000 7f50cede6700 1 -- 172.21.15.32:0/168047651 wait complete. 2020-12-05T17:32:16.270 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 6 2020-12-05T17:32:16.279 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-12-05T17:32:16.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.435+0000 7f87a704d700 1 Processor -- start 2020-12-05T17:32:16.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.435+0000 7f87a704d700 1 -- start start 2020-12-05T17:32:16.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a704d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f87a00b2070 0x7f87a0138e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:16.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a704d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87a00b2ca0 0x7f87a0139320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:16.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a704d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a00b2450 con 0x7f87a0142980 2020-12-05T17:32:16.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a704d700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a00acb80 con 0x7f87a00b2070 2020-12-05T17:32:16.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a704d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a0139810 con 0x7f87a00b2ca0 2020-12-05T17:32:16.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a684c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f87a0142980 0x7f87a0142e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36262/0 (socket says 172.21.15.32:36262) 2020-12-05T17:32:16.443 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a684c700 1 -- 172.21.15.32:0/1391947953 learned_addr learned my addr 172.21.15.32:0/1391947953 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:16.443 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a584a700 1 --2- 172.21.15.32:0/1391947953 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87a00b2ca0 0x7f87a0139320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:16.443 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a604b700 1 --2- 172.21.15.32:0/1391947953 >> v2:172.21.15.61:3300/0 conn(0x7f87a00b2070 0x7f87a0138e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:16.443 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a5049700 1 -- 172.21.15.32:0/1391947953 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4196554279 0 0) 0x7f87a00b2450 con 0x7f87a0142980 2020-12-05T17:32:16.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a5049700 1 -- 172.21.15.32:0/1391947953 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8788003230 con 0x7f87a0142980 2020-12-05T17:32:16.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a5049700 1 -- 172.21.15.32:0/1391947953 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 853149839 0 0) 0x7f8788003230 con 0x7f87a0142980 2020-12-05T17:32:16.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a604b700 1 -- 172.21.15.32:0/1391947953 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87a00b2ca0 msgr2=0x7f87a0139320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:16.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a604b700 1 --2- 172.21.15.32:0/1391947953 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87a00b2ca0 0x7f87a0139320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:16.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a604b700 1 -- 172.21.15.32:0/1391947953 >> v1:172.21.15.32:6789/0 conn(0x7f87a0142980 legacy=0x7f87a0142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:16.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a604b700 1 -- 172.21.15.32:0/1391947953 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a0066d40 con 0x7f87a00b2070 2020-12-05T17:32:16.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a584a700 1 --2- 172.21.15.32:0/1391947953 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87a00b2ca0 0x7f87a0139320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:32:16.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a604b700 1 --2- 172.21.15.32:0/1391947953 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87a00b2070 0x7f87a0138e30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f8798001bf0 tx=0x7f879c006dc0).ready entity=mon.1 client_cookie=dea0dd33abb501e2 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:16.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a5049700 1 -- 172.21.15.32:0/1391947953 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8798001a50 con 0x7f87a00b2070 2020-12-05T17:32:16.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a5049700 1 -- 172.21.15.32:0/1391947953 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879c00d570 con 0x7f87a00b2070 2020-12-05T17:32:16.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a5049700 1 -- 172.21.15.32:0/1391947953 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f879c00d6e0 con 0x7f87a00b2070 2020-12-05T17:32:16.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a704d700 1 -- 172.21.15.32:0/1391947953 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87a00b2070 msgr2=0x7f87a0138e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:16.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a704d700 1 --2- 172.21.15.32:0/1391947953 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87a00b2070 0x7f87a0138e30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f8798001bf0 tx=0x7f879c006dc0).stop 2020-12-05T17:32:16.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a704d700 1 -- 172.21.15.32:0/1391947953 shutdown_connections 2020-12-05T17:32:16.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a704d700 1 --2- 172.21.15.32:0/1391947953 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87a00b2070 0x7f87a0138e30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a704d700 1 --2- 172.21.15.32:0/1391947953 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87a00b2ca0 0x7f87a0139320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a704d700 1 -- 172.21.15.32:0/1391947953 >> 172.21.15.32:0/1391947953 conn(0x7f87a005bbb0 msgr2=0x7f87a0062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a704d700 1 -- 172.21.15.32:0/1391947953 shutdown_connections 2020-12-05T17:32:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.439+0000 7f87a704d700 1 -- 172.21.15.32:0/1391947953 wait complete. 2020-12-05T17:32:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87a704d700 1 Processor -- start 2020-12-05T17:32:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87a704d700 1 -- start start 2020-12-05T17:32:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87a704d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87a00b2070 0x7f87a0138120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87a704d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87a00b2ca0 0x7f87a0151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87a704d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87a0142980 0x7f87a0151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87a704d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a01559a0 con 0x7f87a00b2ca0 2020-12-05T17:32:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87a704d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a0064590 con 0x7f87a00b2070 2020-12-05T17:32:16.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87a704d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a0154b40 con 0x7f87a0142980 2020-12-05T17:32:16.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87a584a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87a00b2ca0 0x7f87a0151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:16.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87a584a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87a00b2ca0 0x7f87a0151840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33618/0 (socket says 172.21.15.32:33618) 2020-12-05T17:32:16.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87a584a700 1 -- 172.21.15.32:0/1561768990 learned_addr learned my addr 172.21.15.32:0/1561768990 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:16.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87a604b700 1 --2- 172.21.15.32:0/1561768990 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87a00b2070 0x7f87a0138120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87a684c700 1 --2- 172.21.15.32:0/1561768990 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87a0142980 0x7f87a0151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87a584a700 1 -- 172.21.15.32:0/1561768990 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87a0142980 msgr2=0x7f87a0151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87a584a700 1 --2- 172.21.15.32:0/1561768990 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87a0142980 0x7f87a0151d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87a584a700 1 -- 172.21.15.32:0/1561768990 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87a00b2070 msgr2=0x7f87a0138120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87a584a700 1 --2- 172.21.15.32:0/1561768990 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87a00b2070 0x7f87a0138120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87a584a700 1 -- 172.21.15.32:0/1561768990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a01402f0 con 0x7f87a00b2ca0 2020-12-05T17:32:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87a584a700 1 --2- 172.21.15.32:0/1561768990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87a00b2ca0 0x7f87a0151840 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f879400c000 tx=0x7f8794006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87937fe700 1 -- 172.21.15.32:0/1561768990 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f879400cda0 con 0x7f87a00b2ca0 2020-12-05T17:32:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87937fe700 1 -- 172.21.15.32:0/1561768990 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87940123f0 con 0x7f87a00b2ca0 2020-12-05T17:32:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87a704d700 1 -- 172.21.15.32:0/1561768990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a013ffe0 con 0x7f87a00b2ca0 2020-12-05T17:32:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87937fe700 1 -- 172.21.15.32:0/1561768990 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8794012750 con 0x7f87a00b2ca0 2020-12-05T17:32:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.443+0000 7f87a704d700 1 -- 172.21.15.32:0/1561768990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a00633e0 con 0x7f87a00b2ca0 2020-12-05T17:32:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.447+0000 7f87a704d700 1 -- 172.21.15.32:0/1561768990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a0151520 con 0x7f87a00b2ca0 2020-12-05T17:32:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.447+0000 7f87937fe700 1 -- 172.21.15.32:0/1561768990 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f87940128b0 con 0x7f87a00b2ca0 2020-12-05T17:32:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.447+0000 7f87937fe700 1 --2- 172.21.15.32:0/1561768990 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f87800553b0 0x7f8780057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.447+0000 7f87937fe700 1 -- 172.21.15.32:0/1561768990 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6517+0+0 (secure 0 0 0) 0x7f879406ea10 con 0x7f87a00b2ca0 2020-12-05T17:32:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.447+0000 7f87a604b700 1 --2- 172.21.15.32:0/1561768990 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f87800553b0 0x7f8780057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.447+0000 7f87a604b700 1 --2- 172.21.15.32:0/1561768990 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f87800553b0 0x7f8780057870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f879c018000 tx=0x7f879c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.451+0000 7f87937fe700 1 -- 172.21.15.32:0/1561768990 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8794044a40 con 0x7f87a00b2ca0 2020-12-05T17:32:16.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.643+0000 7f87a704d700 1 -- 172.21.15.32:0/1561768990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f87a0057050 con 0x7f87a00b2ca0 2020-12-05T17:32:16.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.935+0000 7f87937fe700 1 -- 172.21.15.32:0/1561768990 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8794048060 con 0x7f87a00b2ca0 2020-12-05T17:32:16.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.939+0000 7f87a704d700 1 -- 172.21.15.32:0/1561768990 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f87800553b0 msgr2=0x7f8780057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:16.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.939+0000 7f87a704d700 1 --2- 172.21.15.32:0/1561768990 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f87800553b0 0x7f8780057870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f879c018000 tx=0x7f879c015000).stop 2020-12-05T17:32:16.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.939+0000 7f87a704d700 1 -- 172.21.15.32:0/1561768990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87a00b2ca0 msgr2=0x7f87a0151840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:16.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.939+0000 7f87a704d700 1 --2- 172.21.15.32:0/1561768990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87a00b2ca0 0x7f87a0151840 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f879400c000 tx=0x7f8794006dc0).stop 2020-12-05T17:32:16.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.939+0000 7f87a704d700 1 -- 172.21.15.32:0/1561768990 shutdown_connections 2020-12-05T17:32:16.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.939+0000 7f87a704d700 1 --2- 172.21.15.32:0/1561768990 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f87800553b0 0x7f8780057870 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:16.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.939+0000 7f87a704d700 1 --2- 172.21.15.32:0/1561768990 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87a0142980 0x7f87a0151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:16.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.939+0000 7f87a704d700 1 --2- 172.21.15.32:0/1561768990 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87a00b2070 0x7f87a0138120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:16.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.939+0000 7f87a704d700 1 --2- 172.21.15.32:0/1561768990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87a00b2ca0 0x7f87a0151840 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:16.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.939+0000 7f87a704d700 1 -- 172.21.15.32:0/1561768990 >> 172.21.15.32:0/1561768990 conn(0x7f87a005bbb0 msgr2=0x7f87a00b5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:16.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.939+0000 7f87a704d700 1 -- 172.21.15.32:0/1561768990 shutdown_connections 2020-12-05T17:32:16.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:16.939+0000 7f87a704d700 1 -- 172.21.15.32:0/1561768990 wait complete. 2020-12-05T17:32:16.961 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-12-05T17:32:17.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.087+0000 7f85e359e700 1 Processor -- start 2020-12-05T17:32:17.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.087+0000 7f85e359e700 1 -- start start 2020-12-05T17:32:17.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e359e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f85e405f3b0 0x7f85e405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:17.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e359e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85e405feb0 0x7f85e4061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:17.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85e40b2fb0 0x7f85e413c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:17.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e359e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e40b2df0 con 0x7f85e40b2fb0 2020-12-05T17:32:17.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e359e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e40b3c90 con 0x7f85e405f3b0 2020-12-05T17:32:17.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e359e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e40b2b50 con 0x7f85e405feb0 2020-12-05T17:32:17.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e2d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85e40b2fb0 0x7f85e413c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:17.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e2d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85e40b2fb0 0x7f85e413c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33628/0 (socket says 172.21.15.32:33628) 2020-12-05T17:32:17.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e2d9d700 1 -- 172.21.15.32:0/3550987474 learned_addr learned my addr 172.21.15.32:0/3550987474 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:17.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e259c700 1 --2- 172.21.15.32:0/3550987474 >> v2:172.21.15.61:3300/0 conn(0x7f85e405f3b0 0x7f85e405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:17.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e1d9b700 1 --2- 172.21.15.32:0/3550987474 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85e405feb0 0x7f85e4061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:17.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e2d9d700 1 -- 172.21.15.32:0/3550987474 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85e405feb0 msgr2=0x7f85e4061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:17.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e2d9d700 1 --2- 172.21.15.32:0/3550987474 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85e405feb0 0x7f85e4061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:17.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e2d9d700 1 -- 172.21.15.32:0/3550987474 >> v2:172.21.15.61:3300/0 conn(0x7f85e405f3b0 msgr2=0x7f85e405f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:17.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e2d9d700 1 --2- 172.21.15.32:0/3550987474 >> v2:172.21.15.61:3300/0 conn(0x7f85e405f3b0 0x7f85e405f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:17.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e2d9d700 1 -- 172.21.15.32:0/3550987474 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e4142820 con 0x7f85e40b2fb0 2020-12-05T17:32:17.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e2d9d700 1 --2- 172.21.15.32:0/3550987474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e40b2fb0 0x7f85e413c3b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f85dc002f30 tx=0x7f85dc006e80).ready entity=mon.0 client_cookie=437198071fa2d2cd server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:17.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e159a700 1 -- 172.21.15.32:0/3550987474 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85dc009640 con 0x7f85e40b2fb0 2020-12-05T17:32:17.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e159a700 1 -- 172.21.15.32:0/3550987474 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85dc0097a0 con 0x7f85e40b2fb0 2020-12-05T17:32:17.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e159a700 1 -- 172.21.15.32:0/3550987474 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85dc009b00 con 0x7f85e40b2fb0 2020-12-05T17:32:17.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e359e700 1 -- 172.21.15.32:0/3550987474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e40b2fb0 msgr2=0x7f85e413c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:17.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e359e700 1 --2- 172.21.15.32:0/3550987474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e40b2fb0 0x7f85e413c3b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f85dc002f30 tx=0x7f85dc006e80).stop 2020-12-05T17:32:17.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e359e700 1 -- 172.21.15.32:0/3550987474 shutdown_connections 2020-12-05T17:32:17.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e359e700 1 --2- 172.21.15.32:0/3550987474 >> v2:172.21.15.61:3300/0 conn(0x7f85e405f3b0 0x7f85e405f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:17.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e359e700 1 --2- 172.21.15.32:0/3550987474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e40b2fb0 0x7f85e413c3b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e359e700 1 --2- 172.21.15.32:0/3550987474 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85e405feb0 0x7f85e4061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e359e700 1 -- 172.21.15.32:0/3550987474 >> 172.21.15.32:0/3550987474 conn(0x7f85e40b8f20 msgr2=0x7f85e40b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e359e700 1 -- 172.21.15.32:0/3550987474 shutdown_connections 2020-12-05T17:32:17.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e359e700 1 -- 172.21.15.32:0/3550987474 wait complete. 2020-12-05T17:32:17.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e359e700 1 Processor -- start 2020-12-05T17:32:17.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.091+0000 7f85e359e700 1 -- start start 2020-12-05T17:32:17.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85e359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e405f3b0 0x7f85e4144b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:17.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85e359e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85e405feb0 0x7f85e4113df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:17.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85e359e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85e40b2fb0 0x7f85e4149480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:17.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85e359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e4146890 con 0x7f85e405f3b0 2020-12-05T17:32:17.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85e359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e4146b30 con 0x7f85e405feb0 2020-12-05T17:32:17.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85e359e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e41476a0 con 0x7f85e40b2fb0 2020-12-05T17:32:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85e259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e405f3b0 0x7f85e4144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85e259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e405f3b0 0x7f85e4144b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33630/0 (socket says 172.21.15.32:33630) 2020-12-05T17:32:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85e259c700 1 -- 172.21.15.32:0/4061077414 learned_addr learned my addr 172.21.15.32:0/4061077414 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:17.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85e1d9b700 1 --2- 172.21.15.32:0/4061077414 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85e405feb0 0x7f85e4113df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:17.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85e2d9d700 1 --2- 172.21.15.32:0/4061077414 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85e40b2fb0 0x7f85e4149480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:17.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85e259c700 1 -- 172.21.15.32:0/4061077414 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85e40b2fb0 msgr2=0x7f85e4149480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:17.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85e259c700 1 --2- 172.21.15.32:0/4061077414 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85e40b2fb0 0x7f85e4149480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:17.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85e259c700 1 -- 172.21.15.32:0/4061077414 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85e405feb0 msgr2=0x7f85e4113df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:17.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85e259c700 1 --2- 172.21.15.32:0/4061077414 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85e405feb0 0x7f85e4113df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:17.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85e259c700 1 -- 172.21.15.32:0/4061077414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e4147860 con 0x7f85e405f3b0 2020-12-05T17:32:17.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85e2d9d700 1 --2- 172.21.15.32:0/4061077414 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85e40b2fb0 0x7f85e4149480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:17.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85e259c700 1 --2- 172.21.15.32:0/4061077414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e405f3b0 0x7f85e4144b00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f85d800c000 tx=0x7f85d8006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:17.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85d37fe700 1 -- 172.21.15.32:0/4061077414 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85d800ca10 con 0x7f85e405f3b0 2020-12-05T17:32:17.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85d37fe700 1 -- 172.21.15.32:0/4061077414 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d800cb70 con 0x7f85e405f3b0 2020-12-05T17:32:17.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85e359e700 1 -- 172.21.15.32:0/4061077414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e4140aa0 con 0x7f85e405f3b0 2020-12-05T17:32:17.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85d37fe700 1 -- 172.21.15.32:0/4061077414 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85d80123f0 con 0x7f85e405f3b0 2020-12-05T17:32:17.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85e359e700 1 -- 172.21.15.32:0/4061077414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e4140fa0 con 0x7f85e405f3b0 2020-12-05T17:32:17.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85e359e700 1 -- 172.21.15.32:0/4061077414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e4146cf0 con 0x7f85e405f3b0 2020-12-05T17:32:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85d37fe700 1 -- 172.21.15.32:0/4061077414 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f85d8012550 con 0x7f85e405f3b0 2020-12-05T17:32:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85d37fe700 1 --2- 172.21.15.32:0/4061077414 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f85c00553b0 0x7f85c0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85d37fe700 1 -- 172.21.15.32:0/4061077414 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6517+0+0 (secure 0 0 0) 0x7f85d806e4c0 con 0x7f85e405f3b0 2020-12-05T17:32:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.095+0000 7f85e1d9b700 1 --2- 172.21.15.32:0/4061077414 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f85c00553b0 0x7f85c0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.099+0000 7f85e1d9b700 1 --2- 172.21.15.32:0/4061077414 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f85c00553b0 0x7f85c0057870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f85d400b000 tx=0x7f85d40091a0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.103+0000 7f85d37fe700 1 -- 172.21.15.32:0/4061077414 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f85d80445a0 con 0x7f85e405f3b0 2020-12-05T17:32:17.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.247+0000 7f85e359e700 1 -- 172.21.15.32:0/4061077414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f85e4141530 con 0x7f85e405f3b0 2020-12-05T17:32:17.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.947+0000 7f85d37fe700 1 -- 172.21.15.32:0/4061077414 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v27) v1 ==== 185+0+0 (secure 0 0 0) 0x7f85d8047bc0 con 0x7f85e405f3b0 2020-12-05T17:32:17.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.951+0000 7f85e359e700 1 -- 172.21.15.32:0/4061077414 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f85c00553b0 msgr2=0x7f85c0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.951+0000 7f85e359e700 1 --2- 172.21.15.32:0/4061077414 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f85c00553b0 0x7f85c0057870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f85d400b000 tx=0x7f85d40091a0).stop 2020-12-05T17:32:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.951+0000 7f85e359e700 1 -- 172.21.15.32:0/4061077414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e405f3b0 msgr2=0x7f85e4144b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.951+0000 7f85e359e700 1 --2- 172.21.15.32:0/4061077414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e405f3b0 0x7f85e4144b00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f85d800c000 tx=0x7f85d8006dc0).stop 2020-12-05T17:32:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.951+0000 7f85e359e700 1 -- 172.21.15.32:0/4061077414 shutdown_connections 2020-12-05T17:32:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.951+0000 7f85e359e700 1 --2- 172.21.15.32:0/4061077414 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f85c00553b0 0x7f85c0057870 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.951+0000 7f85e359e700 1 --2- 172.21.15.32:0/4061077414 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85e40b2fb0 0x7f85e4149480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.951+0000 7f85e359e700 1 --2- 172.21.15.32:0/4061077414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e405f3b0 0x7f85e4144b00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.951+0000 7f85e359e700 1 --2- 172.21.15.32:0/4061077414 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85e405feb0 0x7f85e4113df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:17.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.951+0000 7f85e359e700 1 -- 172.21.15.32:0/4061077414 >> 172.21.15.32:0/4061077414 conn(0x7f85e40b8f20 msgr2=0x7f85e40b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:17.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.951+0000 7f85e359e700 1 -- 172.21.15.32:0/4061077414 shutdown_connections 2020-12-05T17:32:17.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:17.951+0000 7f85e359e700 1 -- 172.21.15.32:0/4061077414 wait complete. 2020-12-05T17:32:17.957 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2020-12-05T17:32:17.971 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-12-05T17:32:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.095+0000 7f6faa31e700 1 Processor -- start 2020-12-05T17:32:18.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.095+0000 7f6faa31e700 1 -- start start 2020-12-05T17:32:18.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6faa31e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f6fa405f550 0x7f6fa40acb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:18.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6faa31e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6fa4063040 0x7f6fa413fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:18.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6faa31e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6fa40af090 0x7f6fa40af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:18.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6faa31e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fa4067110 con 0x7f6fa40af090 2020-12-05T17:32:18.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6faa31e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fa413f760 con 0x7f6fa405f550 2020-12-05T17:32:18.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6faa31e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa413fa00 con 0x7f6fa4063040 2020-12-05T17:32:18.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6fa9b1d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6fa40af090 0x7f6fa40af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:18.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6fa9b1d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6fa40af090 0x7f6fa40af4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33642/0 (socket says 172.21.15.32:33642) 2020-12-05T17:32:18.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6fa9b1d700 1 -- 172.21.15.32:0/1231723635 learned_addr learned my addr 172.21.15.32:0/1231723635 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:18.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6fa8b1b700 1 --2- 172.21.15.32:0/1231723635 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6fa4063040 0x7f6fa413fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:18.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6fa931c700 1 --2- 172.21.15.32:0/1231723635 >> v2:172.21.15.61:3300/0 conn(0x7f6fa405f550 0x7f6fa40acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:18.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6fa9b1d700 1 -- 172.21.15.32:0/1231723635 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6fa4063040 msgr2=0x7f6fa413fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:18.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6fa9b1d700 1 --2- 172.21.15.32:0/1231723635 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6fa4063040 0x7f6fa413fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:18.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6fa9b1d700 1 -- 172.21.15.32:0/1231723635 >> v2:172.21.15.61:3300/0 conn(0x7f6fa405f550 msgr2=0x7f6fa40acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:18.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6fa9b1d700 1 --2- 172.21.15.32:0/1231723635 >> v2:172.21.15.61:3300/0 conn(0x7f6fa405f550 0x7f6fa40acb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:18.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6fa9b1d700 1 -- 172.21.15.32:0/1231723635 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa40af9d0 con 0x7f6fa40af090 2020-12-05T17:32:18.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6fa931c700 1 --2- 172.21.15.32:0/1231723635 >> v2:172.21.15.61:3300/0 conn(0x7f6fa405f550 0x7f6fa40acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:32:18.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6fa9b1d700 1 --2- 172.21.15.32:0/1231723635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa40af090 0x7f6fa40af4e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f6f9c0032b0 tx=0x7f6f9c009270).ready entity=mon.0 client_cookie=e3367b250ec742e5 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:18.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6f93fff700 1 -- 172.21.15.32:0/1231723635 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f9c00bc00 con 0x7f6fa40af090 2020-12-05T17:32:18.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6f93fff700 1 -- 172.21.15.32:0/1231723635 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9c00bd60 con 0x7f6fa40af090 2020-12-05T17:32:18.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6f93fff700 1 -- 172.21.15.32:0/1231723635 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f9c011400 con 0x7f6fa40af090 2020-12-05T17:32:18.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6faa31e700 1 -- 172.21.15.32:0/1231723635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa40af090 msgr2=0x7f6fa40af4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:18.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6faa31e700 1 --2- 172.21.15.32:0/1231723635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa40af090 0x7f6fa40af4e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f6f9c0032b0 tx=0x7f6f9c009270).stop 2020-12-05T17:32:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6faa31e700 1 -- 172.21.15.32:0/1231723635 shutdown_connections 2020-12-05T17:32:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6faa31e700 1 --2- 172.21.15.32:0/1231723635 >> v2:172.21.15.61:3300/0 conn(0x7f6fa405f550 0x7f6fa40acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6faa31e700 1 --2- 172.21.15.32:0/1231723635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa40af090 0x7f6fa40af4e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6faa31e700 1 --2- 172.21.15.32:0/1231723635 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6fa4063040 0x7f6fa413fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6faa31e700 1 -- 172.21.15.32:0/1231723635 >> 172.21.15.32:0/1231723635 conn(0x7f6fa40620b0 msgr2=0x7f6fa4062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6faa31e700 1 -- 172.21.15.32:0/1231723635 shutdown_connections 2020-12-05T17:32:18.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.099+0000 7f6faa31e700 1 -- 172.21.15.32:0/1231723635 wait complete. 2020-12-05T17:32:18.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.103+0000 7f6faa31e700 1 Processor -- start 2020-12-05T17:32:18.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.103+0000 7f6faa31e700 1 -- start start 2020-12-05T17:32:18.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.107+0000 7f6faa31e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6fa405f550 0x7f6fa4066020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:18.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.107+0000 7f6faa31e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6fa4063040 0x7f6fa4143930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.107+0000 7f6faa31e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa40af090 0x7f6fa413e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.107+0000 7f6faa31e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa40ad640 con 0x7f6fa40af090 2020-12-05T17:32:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.107+0000 7f6faa31e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa4142f50 con 0x7f6fa4063040 2020-12-05T17:32:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.107+0000 7f6faa31e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa41431f0 con 0x7f6fa405f550 2020-12-05T17:32:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.107+0000 7f6fa9b1d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa40af090 0x7f6fa413e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.107+0000 7f6fa931c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6fa405f550 0x7f6fa4066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.107+0000 7f6fa9b1d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa40af090 0x7f6fa413e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33648/0 (socket says 172.21.15.32:33648) 2020-12-05T17:32:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.107+0000 7f6fa9b1d700 1 -- 172.21.15.32:0/3319538512 learned_addr learned my addr 172.21.15.32:0/3319538512 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.107+0000 7f6fa8b1b700 1 --2- 172.21.15.32:0/3319538512 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6fa4063040 0x7f6fa4143930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:18.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.107+0000 7f6fa9b1d700 1 -- 172.21.15.32:0/3319538512 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6fa405f550 msgr2=0x7f6fa4066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:18.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.107+0000 7f6fa9b1d700 1 --2- 172.21.15.32:0/3319538512 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6fa405f550 0x7f6fa4066020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:18.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.107+0000 7f6fa9b1d700 1 -- 172.21.15.32:0/3319538512 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6fa4063040 msgr2=0x7f6fa4143930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:18.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.107+0000 7f6fa9b1d700 1 --2- 172.21.15.32:0/3319538512 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6fa4063040 0x7f6fa4143930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:18.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.107+0000 7f6fa9b1d700 1 -- 172.21.15.32:0/3319538512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa4143490 con 0x7f6fa40af090 2020-12-05T17:32:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.107+0000 7f6fa9b1d700 1 --2- 172.21.15.32:0/3319538512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa40af090 0x7f6fa413e440 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f6f9c0032b0 tx=0x7f6f9c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.107+0000 7f6f927fc700 1 -- 172.21.15.32:0/3319538512 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f9c011ea0 con 0x7f6fa40af090 2020-12-05T17:32:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.107+0000 7f6f927fc700 1 -- 172.21.15.32:0/3319538512 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9c01f3f0 con 0x7f6fa40af090 2020-12-05T17:32:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.107+0000 7f6faa31e700 1 -- 172.21.15.32:0/3319538512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa40659a0 con 0x7f6fa40af090 2020-12-05T17:32:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.107+0000 7f6f927fc700 1 -- 172.21.15.32:0/3319538512 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f9c01f750 con 0x7f6fa40af090 2020-12-05T17:32:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.107+0000 7f6faa31e700 1 -- 172.21.15.32:0/3319538512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa41516a0 con 0x7f6fa40af090 2020-12-05T17:32:18.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.107+0000 7f6faa31e700 1 -- 172.21.15.32:0/3319538512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa4143620 con 0x7f6fa40af090 2020-12-05T17:32:18.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.111+0000 7f6f927fc700 1 -- 172.21.15.32:0/3319538512 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6f9c026030 con 0x7f6fa40af090 2020-12-05T17:32:18.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.111+0000 7f6f927fc700 1 --2- 172.21.15.32:0/3319538512 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6f840553b0 0x7f6f84057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:18.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.111+0000 7f6f927fc700 1 -- 172.21.15.32:0/3319538512 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6079+0+0 (secure 0 0 0) 0x7f6f9c06ead0 con 0x7f6fa40af090 2020-12-05T17:32:18.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.111+0000 7f6fa931c700 1 --2- 172.21.15.32:0/3319538512 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6f840553b0 0x7f6f84057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:18.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.111+0000 7f6fa931c700 1 --2- 172.21.15.32:0/3319538512 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6f840553b0 0x7f6f84057870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f6fa000d000 tx=0x7f6fa0008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:18.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.115+0000 7f6f927fc700 1 -- 172.21.15.32:0/3319538512 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6f9c044d30 con 0x7f6fa40af090 2020-12-05T17:32:18.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.259+0000 7f6faa31e700 1 -- 172.21.15.32:0/3319538512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6fa4057050 con 0x7f6fa40af090 2020-12-05T17:32:18.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.959+0000 7f6f927fc700 1 -- 172.21.15.32:0/3319538512 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v28) v1 ==== 173+0+0 (secure 0 0 0) 0x7f6f9c0240d0 con 0x7f6fa40af090 2020-12-05T17:32:18.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.959+0000 7f6faa31e700 1 -- 172.21.15.32:0/3319538512 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6f840553b0 msgr2=0x7f6f84057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:18.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.959+0000 7f6faa31e700 1 --2- 172.21.15.32:0/3319538512 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6f840553b0 0x7f6f84057870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f6fa000d000 tx=0x7f6fa0008040).stop 2020-12-05T17:32:18.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.963+0000 7f6faa31e700 1 -- 172.21.15.32:0/3319538512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa40af090 msgr2=0x7f6fa413e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:18.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.963+0000 7f6faa31e700 1 --2- 172.21.15.32:0/3319538512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa40af090 0x7f6fa413e440 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f6f9c0032b0 tx=0x7f6f9c016000).stop 2020-12-05T17:32:18.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.963+0000 7f6faa31e700 1 -- 172.21.15.32:0/3319538512 shutdown_connections 2020-12-05T17:32:18.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.963+0000 7f6faa31e700 1 --2- 172.21.15.32:0/3319538512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa40af090 0x7f6fa413e440 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:18.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.963+0000 7f6faa31e700 1 --2- 172.21.15.32:0/3319538512 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6f840553b0 0x7f6f84057870 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:18.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.963+0000 7f6faa31e700 1 --2- 172.21.15.32:0/3319538512 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6fa405f550 0x7f6fa4066020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:18.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.963+0000 7f6faa31e700 1 --2- 172.21.15.32:0/3319538512 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6fa4063040 0x7f6fa4143930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:18.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.963+0000 7f6faa31e700 1 -- 172.21.15.32:0/3319538512 >> 172.21.15.32:0/3319538512 conn(0x7f6fa40620b0 msgr2=0x7f6fa40b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:18.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.963+0000 7f6faa31e700 1 -- 172.21.15.32:0/3319538512 shutdown_connections 2020-12-05T17:32:18.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:18.963+0000 7f6faa31e700 1 -- 172.21.15.32:0/3319538512 wait complete. 2020-12-05T17:32:18.967 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2020-12-05T17:32:18.981 INFO:tasks.ceph.mds.a:Restarting daemon 2020-12-05T17:32:18.982 INFO:tasks.ceph.mds.a:Stopping old one... 2020-12-05T17:32:18.982 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-12-05T17:32:18.983 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:32:18.983 INFO:tasks.ceph.mds.b:Restarting daemon 2020-12-05T17:32:18.983 INFO:tasks.ceph.mds.b:Stopping old one... 2020-12-05T17:32:18.984 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-12-05T17:32:18.984 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:32:18.994 INFO:tasks.ceph.mds.b:Stopped 2020-12-05T17:32:18.995 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-12-05T17:32:18.996 INFO:tasks.ceph.mds.a:Stopped 2020-12-05T17:32:18.997 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-12-05T17:32:19.039 INFO:tasks.ceph.mds.b:Started 2020-12-05T17:32:19.039 DEBUG:teuthology.parallel:result is None 2020-12-05T17:32:19.045 INFO:tasks.ceph.mds.a:Started 2020-12-05T17:32:19.046 DEBUG:teuthology.parallel:result is None 2020-12-05T17:32:19.047 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-12-05T17:32:19.116 INFO:tasks.ceph.mds.b.smithi061.stdout:starting mds.b at 2020-12-05T17:32:19.123 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:32:19.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.167+0000 7f7773ba5700 1 Processor -- start 2020-12-05T17:32:19.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.167+0000 7f7773ba5700 1 -- start start 2020-12-05T17:32:19.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7773ba5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f776c05f6f0 0x7f776c0b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:19.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7773ba5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f776c143650 0x7f776c061de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:19.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7773ba5700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f776c0619b0 con 0x7f776c05feb0 2020-12-05T17:32:19.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7773ba5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f776c05fad0 con 0x7f776c05f6f0 2020-12-05T17:32:19.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7773ba5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776c0afa50 con 0x7f776c143650 2020-12-05T17:32:19.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7771941700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f776c05f6f0 0x7f776c0b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:19.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7771941700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f776c05f6f0 0x7f776c0b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33664/0 (socket says 172.21.15.32:33664) 2020-12-05T17:32:19.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7771941700 1 -- 172.21.15.32:0/3496072268 learned_addr learned my addr 172.21.15.32:0/3496072268 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:19.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7771140700 1 --2- 172.21.15.32:0/3496072268 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f776c143650 0x7f776c061de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7771941700 1 -- 172.21.15.32:0/3496072268 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f776c143650 msgr2=0x7f776c061de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7771941700 1 --2- 172.21.15.32:0/3496072268 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f776c143650 0x7f776c061de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7771941700 1 -- 172.21.15.32:0/3496072268 >> v1:172.21.15.61:6789/0 conn(0x7f776c05feb0 legacy=0x7f776c145fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:19.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7771941700 1 -- 172.21.15.32:0/3496072268 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776c0afcf0 con 0x7f776c05f6f0 2020-12-05T17:32:19.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7771140700 1 --2- 172.21.15.32:0/3496072268 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f776c143650 0x7f776c061de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:19.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7771941700 1 --2- 172.21.15.32:0/3496072268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f776c05f6f0 0x7f776c0b3a80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f7764002f30 tx=0x7f7764006e80).ready entity=mon.0 client_cookie=389d6df9ecc975a3 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:19.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f777093f700 1 -- 172.21.15.32:0/3496072268 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77640097f0 con 0x7f776c05f6f0 2020-12-05T17:32:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f777093f700 1 -- 172.21.15.32:0/3496072268 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7764009950 con 0x7f776c05f6f0 2020-12-05T17:32:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f777093f700 1 -- 172.21.15.32:0/3496072268 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7764009cb0 con 0x7f776c05f6f0 2020-12-05T17:32:19.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7773ba5700 1 -- 172.21.15.32:0/3496072268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f776c05f6f0 msgr2=0x7f776c0b3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:19.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7773ba5700 1 --2- 172.21.15.32:0/3496072268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f776c05f6f0 0x7f776c0b3a80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f7764002f30 tx=0x7f7764006e80).stop 2020-12-05T17:32:19.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7773ba5700 1 -- 172.21.15.32:0/3496072268 shutdown_connections 2020-12-05T17:32:19.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7773ba5700 1 --2- 172.21.15.32:0/3496072268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f776c05f6f0 0x7f776c0b3a80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7773ba5700 1 --2- 172.21.15.32:0/3496072268 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f776c143650 0x7f776c061de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7773ba5700 1 -- 172.21.15.32:0/3496072268 >> 172.21.15.32:0/3496072268 conn(0x7f776c0b90e0 msgr2=0x7f776c0b94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7773ba5700 1 -- 172.21.15.32:0/3496072268 shutdown_connections 2020-12-05T17:32:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7773ba5700 1 -- 172.21.15.32:0/3496072268 wait complete. 2020-12-05T17:32:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7773ba5700 1 Processor -- start 2020-12-05T17:32:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.171+0000 7f7773ba5700 1 -- start start 2020-12-05T17:32:19.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.175+0000 7f7773ba5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f776c05f6f0 0x7f776c155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:19.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.175+0000 7f7773ba5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f776c05feb0 0x7f776c146020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.175+0000 7f7773ba5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f776c143650 0x7f776c141fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.175+0000 7f7773ba5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776c13e9d0 con 0x7f776c143650 2020-12-05T17:32:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.175+0000 7f7773ba5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776c13d720 con 0x7f776c05f6f0 2020-12-05T17:32:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.175+0000 7f7773ba5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776c13dc90 con 0x7f776c05feb0 2020-12-05T17:32:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.175+0000 7f7772142700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f776c143650 0x7f776c141fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.175+0000 7f7771941700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f776c05f6f0 0x7f776c155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.175+0000 7f7772142700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f776c143650 0x7f776c141fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33674/0 (socket says 172.21.15.32:33674) 2020-12-05T17:32:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.175+0000 7f7772142700 1 -- 172.21.15.32:0/2555555581 learned_addr learned my addr 172.21.15.32:0/2555555581 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.175+0000 7f7771140700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f776c05feb0 0x7f776c146020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.175+0000 7f7772142700 1 -- 172.21.15.32:0/2555555581 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f776c05feb0 msgr2=0x7f776c146020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.175+0000 7f7772142700 1 --2- 172.21.15.32:0/2555555581 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f776c05feb0 0x7f776c146020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:19.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.175+0000 7f7772142700 1 -- 172.21.15.32:0/2555555581 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f776c05f6f0 msgr2=0x7f776c155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:19.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.175+0000 7f7772142700 1 --2- 172.21.15.32:0/2555555581 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f776c05f6f0 0x7f776c155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:19.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.175+0000 7f7772142700 1 -- 172.21.15.32:0/2555555581 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776c13e0c0 con 0x7f776c143650 2020-12-05T17:32:19.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.175+0000 7f7772142700 1 --2- 172.21.15.32:0/2555555581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f776c143650 0x7f776c141fa0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f776c061cc0 tx=0x7f7768007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:19.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.175+0000 7f775affd700 1 -- 172.21.15.32:0/2555555581 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f776800a5a0 con 0x7f776c143650 2020-12-05T17:32:19.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.175+0000 7f775affd700 1 -- 172.21.15.32:0/2555555581 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776800a700 con 0x7f776c143650 2020-12-05T17:32:19.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.175+0000 7f7773ba5700 1 -- 172.21.15.32:0/2555555581 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f776c13d950 con 0x7f776c143650 2020-12-05T17:32:19.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.175+0000 7f775affd700 1 -- 172.21.15.32:0/2555555581 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f776800aa60 con 0x7f776c143650 2020-12-05T17:32:19.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.175+0000 7f7773ba5700 1 -- 172.21.15.32:0/2555555581 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f776c140350 con 0x7f776c143650 2020-12-05T17:32:19.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.175+0000 7f7773ba5700 1 -- 172.21.15.32:0/2555555581 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f776c13dae0 con 0x7f776c143650 2020-12-05T17:32:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.179+0000 7f775affd700 1 -- 172.21.15.32:0/2555555581 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7768019030 con 0x7f776c143650 2020-12-05T17:32:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.179+0000 7f775affd700 1 --2- 172.21.15.32:0/2555555581 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f774c055330 0x7f774c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.179+0000 7f775affd700 1 -- 172.21.15.32:0/2555555581 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f776806d600 con 0x7f776c143650 2020-12-05T17:32:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.179+0000 7f7771941700 1 --2- 172.21.15.32:0/2555555581 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f774c055330 0x7f774c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:19.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.179+0000 7f7771941700 1 --2- 172.21.15.32:0/2555555581 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f774c055330 0x7f774c0577f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f776401c000 tx=0x7f7764015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:19.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.187+0000 7f775affd700 1 -- 172.21.15.32:0/2555555581 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7768043a80 con 0x7f776c143650 2020-12-05T17:32:19.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.343+0000 7f7773ba5700 1 -- 172.21.15.32:0/2555555581 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f776c057050 con 0x7f776c143650 2020-12-05T17:32:19.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.959+0000 7f775affd700 1 -- 172.21.15.32:0/2555555581 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v29) v1 ==== 97+0+0 (secure 0 0 0) 0x7f77680470a0 con 0x7f776c143650 2020-12-05T17:32:19.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.963+0000 7f7773ba5700 1 -- 172.21.15.32:0/2555555581 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f774c055330 msgr2=0x7f774c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:19.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.963+0000 7f7773ba5700 1 --2- 172.21.15.32:0/2555555581 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f774c055330 0x7f774c0577f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f776401c000 tx=0x7f7764015000).stop 2020-12-05T17:32:19.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.963+0000 7f7773ba5700 1 -- 172.21.15.32:0/2555555581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f776c143650 msgr2=0x7f776c141fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:19.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.963+0000 7f7773ba5700 1 --2- 172.21.15.32:0/2555555581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f776c143650 0x7f776c141fa0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f776c061cc0 tx=0x7f7768007f10).stop 2020-12-05T17:32:19.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.963+0000 7f7773ba5700 1 -- 172.21.15.32:0/2555555581 shutdown_connections 2020-12-05T17:32:19.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.963+0000 7f7773ba5700 1 --2- 172.21.15.32:0/2555555581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f776c143650 0x7f776c141fa0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:19.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.963+0000 7f7773ba5700 1 --2- 172.21.15.32:0/2555555581 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f776c05f6f0 0x7f776c155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:19.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.963+0000 7f7773ba5700 1 --2- 172.21.15.32:0/2555555581 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f774c055330 0x7f774c0577f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:19.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.963+0000 7f7773ba5700 1 --2- 172.21.15.32:0/2555555581 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f776c05feb0 0x7f776c146020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:19.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.963+0000 7f7773ba5700 1 -- 172.21.15.32:0/2555555581 >> 172.21.15.32:0/2555555581 conn(0x7f776c0b90e0 msgr2=0x7f776c0b6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:19.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.963+0000 7f7773ba5700 1 -- 172.21.15.32:0/2555555581 shutdown_connections 2020-12-05T17:32:19.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:19.963+0000 7f7773ba5700 1 -- 172.21.15.32:0/2555555581 wait complete. 2020-12-05T17:32:19.970 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2020-12-05T17:32:19.983 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2020-12-05T17:32:19.984 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-12-05T17:32:20.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.095+0000 7fb4863bd700 1 Processor -- start 2020-12-05T17:32:20.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.095+0000 7fb4863bd700 1 -- start start 2020-12-05T17:32:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.095+0000 7fb4863bd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb4800b1fc0 0x7fb4800af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.095+0000 7fb4863bd700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb480147400 0x7fb480145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.095+0000 7fb4863bd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb480146a90 con 0x7fb4800ab710 2020-12-05T17:32:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.095+0000 7fb4863bd700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb480061e90 con 0x7fb480147400 2020-12-05T17:32:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.095+0000 7fb4863bd700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4800b0110 con 0x7fb4800b1fc0 2020-12-05T17:32:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.095+0000 7fb484bba700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb4800ab710 0x7fb480061780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36326/0 (socket says 172.21.15.32:36326) 2020-12-05T17:32:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.095+0000 7fb484bba700 1 -- 172.21.15.32:0/2127167616 learned_addr learned my addr 172.21.15.32:0/2127167616 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.095+0000 7fb485bbc700 1 --2- 172.21.15.32:0/2127167616 >> v2:172.21.15.61:3300/0 conn(0x7fb480147400 0x7fb480145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.095+0000 7fb4853bb700 1 --2- 172.21.15.32:0/2127167616 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb4800b1fc0 0x7fb4800af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.095+0000 7fb47ffff700 1 -- 172.21.15.32:0/2127167616 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2079661628 0 0) 0x7fb480146a90 con 0x7fb4800ab710 2020-12-05T17:32:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.095+0000 7fb47ffff700 1 -- 172.21.15.32:0/2127167616 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb46c003230 con 0x7fb4800ab710 2020-12-05T17:32:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.095+0000 7fb485bbc700 1 -- 172.21.15.32:0/2127167616 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb4800b1fc0 msgr2=0x7fb4800af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:20.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.095+0000 7fb485bbc700 1 --2- 172.21.15.32:0/2127167616 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb4800b1fc0 0x7fb4800af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:20.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.095+0000 7fb485bbc700 1 -- 172.21.15.32:0/2127167616 >> v1:172.21.15.32:6789/0 conn(0x7fb4800ab710 legacy=0x7fb480061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:20.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.095+0000 7fb485bbc700 1 -- 172.21.15.32:0/2127167616 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4800b23a0 con 0x7fb480147400 2020-12-05T17:32:20.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.095+0000 7fb485bbc700 1 --2- 172.21.15.32:0/2127167616 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb480147400 0x7fb480145d10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fb4780032b0 tx=0x7fb4780091a0).ready entity=mon.1 client_cookie=829d7cb9585b09ff server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:20.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.095+0000 7fb47ffff700 1 -- 172.21.15.32:0/2127167616 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb47800b940 con 0x7fb480147400 2020-12-05T17:32:20.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.095+0000 7fb47ffff700 1 -- 172.21.15.32:0/2127167616 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47800baa0 con 0x7fb480147400 2020-12-05T17:32:20.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.095+0000 7fb47ffff700 1 -- 172.21.15.32:0/2127167616 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb47800bc10 con 0x7fb480147400 2020-12-05T17:32:20.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.099+0000 7fb4863bd700 1 -- 172.21.15.32:0/2127167616 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb480147400 msgr2=0x7fb480145d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:20.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.099+0000 7fb4863bd700 1 --2- 172.21.15.32:0/2127167616 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb480147400 0x7fb480145d10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fb4780032b0 tx=0x7fb4780091a0).stop 2020-12-05T17:32:20.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.099+0000 7fb4863bd700 1 -- 172.21.15.32:0/2127167616 shutdown_connections 2020-12-05T17:32:20.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.099+0000 7fb4863bd700 1 --2- 172.21.15.32:0/2127167616 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb480147400 0x7fb480145d10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:20.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.099+0000 7fb4863bd700 1 --2- 172.21.15.32:0/2127167616 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb4800b1fc0 0x7fb4800af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:20.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.099+0000 7fb4863bd700 1 -- 172.21.15.32:0/2127167616 >> 172.21.15.32:0/2127167616 conn(0x7fb4800b5530 msgr2=0x7fb4800b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:20.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.099+0000 7fb4863bd700 1 -- 172.21.15.32:0/2127167616 shutdown_connections 2020-12-05T17:32:20.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.099+0000 7fb4863bd700 1 -- 172.21.15.32:0/2127167616 wait complete. 2020-12-05T17:32:20.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.099+0000 7fb4863bd700 1 Processor -- start 2020-12-05T17:32:20.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.103+0000 7fb4863bd700 1 -- start start 2020-12-05T17:32:20.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.103+0000 7fb4863bd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4800ab710 0x7fb48009e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:20.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.103+0000 7fb4863bd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb4800b1fc0 0x7fb4800a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:20.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.103+0000 7fb4863bd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb480147400 0x7fb480158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:20.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.103+0000 7fb4863bd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4800a0790 con 0x7fb480147400 2020-12-05T17:32:20.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.103+0000 7fb4863bd700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48009fb60 con 0x7fb4800ab710 2020-12-05T17:32:20.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.103+0000 7fb4863bd700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48009fe00 con 0x7fb4800b1fc0 2020-12-05T17:32:20.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.103+0000 7fb485bbc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb480147400 0x7fb480158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:20.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.103+0000 7fb485bbc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb480147400 0x7fb480158b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33686/0 (socket says 172.21.15.32:33686) 2020-12-05T17:32:20.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.103+0000 7fb485bbc700 1 -- 172.21.15.32:0/584214793 learned_addr learned my addr 172.21.15.32:0/584214793 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:20.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.103+0000 7fb4853bb700 1 --2- 172.21.15.32:0/584214793 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4800ab710 0x7fb48009e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:20.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.103+0000 7fb484bba700 1 --2- 172.21.15.32:0/584214793 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb4800b1fc0 0x7fb4800a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:20.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.103+0000 7fb485bbc700 1 -- 172.21.15.32:0/584214793 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb4800b1fc0 msgr2=0x7fb4800a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:20.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.103+0000 7fb485bbc700 1 --2- 172.21.15.32:0/584214793 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb4800b1fc0 0x7fb4800a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:20.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.103+0000 7fb485bbc700 1 -- 172.21.15.32:0/584214793 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4800ab710 msgr2=0x7fb48009e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:20.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.103+0000 7fb485bbc700 1 --2- 172.21.15.32:0/584214793 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4800ab710 0x7fb48009e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:20.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.103+0000 7fb485bbc700 1 -- 172.21.15.32:0/584214793 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4800acd40 con 0x7fb480147400 2020-12-05T17:32:20.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.103+0000 7fb485bbc700 1 --2- 172.21.15.32:0/584214793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb480147400 0x7fb480158b00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7fb4780032b0 tx=0x7fb478016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:20.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.103+0000 7fb47e7fc700 1 -- 172.21.15.32:0/584214793 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb478011e80 con 0x7fb480147400 2020-12-05T17:32:20.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.103+0000 7fb47e7fc700 1 -- 172.21.15.32:0/584214793 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4780105b0 con 0x7fb480147400 2020-12-05T17:32:20.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.103+0000 7fb4863bd700 1 -- 172.21.15.32:0/584214793 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4800ae3a0 con 0x7fb480147400 2020-12-05T17:32:20.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.103+0000 7fb47e7fc700 1 -- 172.21.15.32:0/584214793 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb478010910 con 0x7fb480147400 2020-12-05T17:32:20.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.103+0000 7fb4863bd700 1 -- 172.21.15.32:0/584214793 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4800ad210 con 0x7fb480147400 2020-12-05T17:32:20.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.107+0000 7fb4863bd700 1 -- 172.21.15.32:0/584214793 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb48009f9a0 con 0x7fb480147400 2020-12-05T17:32:20.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.107+0000 7fb47e7fc700 1 -- 172.21.15.32:0/584214793 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb478027030 con 0x7fb480147400 2020-12-05T17:32:20.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.107+0000 7fb47e7fc700 1 --2- 172.21.15.32:0/584214793 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb4600553f0 0x7fb4600578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:20.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.107+0000 7fb4853bb700 1 --2- 172.21.15.32:0/584214793 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb4600553f0 0x7fb4600578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:20.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.107+0000 7fb47e7fc700 1 -- 172.21.15.32:0/584214793 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fb47806eca0 con 0x7fb480147400 2020-12-05T17:32:20.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.107+0000 7fb4853bb700 1 --2- 172.21.15.32:0/584214793 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb4600553f0 0x7fb4600578b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fb47400b000 tx=0x7fb474006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:20.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.111+0000 7fb47e7fc700 1 -- 172.21.15.32:0/584214793 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb4780450b0 con 0x7fb480147400 2020-12-05T17:32:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.307+0000 7fb4863bd700 1 -- 172.21.15.32:0/584214793 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb480057050 con 0x7fb480147400 2020-12-05T17:32:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.307+0000 7fb47e7fc700 1 -- 172.21.15.32:0/584214793 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+5627 (secure 0 0 0) 0x7fb478025090 con 0x7fb480147400 2020-12-05T17:32:20.310 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:32:20.310 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:32:20.311 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2020-12-05T17:32:20.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:20.311 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2020-12-05T17:32:20.311 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfpb/aExAAdMKW4yIgC2zhkzyW6l8xug==", 2020-12-05T17:32:20.312 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:32:20.312 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:32:20.312 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:32:20.312 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:32:20.312 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:32:20.313 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:20.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:20.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:20.313 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2020-12-05T17:32:20.313 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfBKTYGRAAaPkezwmTiAvkUfTuDGDNZA==", 2020-12-05T17:32:20.314 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:32:20.314 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:32:20.314 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:32:20.314 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:32:20.314 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:32:20.315 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:20.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:20.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:20.315 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2020-12-05T17:32:20.315 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHw8tflo5EKhAAwV9D64fAoY3uvXCm+Aksag==", 2020-12-05T17:32:20.316 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:32:20.316 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:32:20.316 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:32:20.316 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:32:20.316 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:20.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:20.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:20.317 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2020-12-05T17:32:20.317 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDKw8tfTjEiERAAkW6mkgApoXwyF49kPjZBGg==", 2020-12-05T17:32:20.317 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:32:20.318 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:32:20.318 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:32:20.318 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:32:20.318 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:20.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:20.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:20.319 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2020-12-05T17:32:20.319 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDMw8tfLEzBMxAAiJ2NifsOLEkt1e3vmhwSTA==", 2020-12-05T17:32:20.319 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:32:20.319 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:32:20.320 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:32:20.320 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:32:20.320 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:20.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:20.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:20.321 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2020-12-05T17:32:20.321 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPw8tfV+vnGxAAQHvXJRW5X9Vhe/eYJBJ/uQ==", 2020-12-05T17:32:20.321 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:32:20.321 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:32:20.321 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:32:20.322 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:32:20.322 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:20.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:20.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:20.322 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2020-12-05T17:32:20.323 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUw8tfN8wMJxAApc5LY9HGDeIV/o0eil/g4A==", 2020-12-05T17:32:20.323 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:32:20.323 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:32:20.323 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:32:20.323 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:32:20.324 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:20.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:20.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:20.324 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2020-12-05T17:32:20.324 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXw8tfvZ5dDxAAtAfftExkljlBvn7ZSoD/Og==", 2020-12-05T17:32:20.325 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:32:20.325 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:32:20.325 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:32:20.325 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:32:20.325 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:20.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:20.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:20.326 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2020-12-05T17:32:20.326 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDZw8tfhXFzMhAAZXNwNGfKv360Tq4N/DX5Mw==", 2020-12-05T17:32:20.326 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:32:20.327 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:32:20.327 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:32:20.327 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:32:20.327 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:20.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:20.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:20.328 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2020-12-05T17:32:20.328 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDcw8tfROfSGhAA/Y2Y1b4k5zQju0MM7LASeQ==", 2020-12-05T17:32:20.328 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:32:20.328 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:32:20.329 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:32:20.329 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:32:20.329 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:20.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:20.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:20.330 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2020-12-05T17:32:20.330 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfWDp/HxAAmvLPj7C3WDgj0OC2o7lBJQ==", 2020-12-05T17:32:20.330 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:32:20.330 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:32:20.331 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2020-12-05T17:32:20.331 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2020-12-05T17:32:20.331 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2020-12-05T17:32:20.331 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:20.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:20.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:20.332 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2020-12-05T17:32:20.332 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfugnmIRAAW6+/QGMLkHw+Z97RHpV4kw==", 2020-12-05T17:32:20.332 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:32:20.332 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:32:20.333 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2020-12-05T17:32:20.333 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2020-12-05T17:32:20.333 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2020-12-05T17:32:20.333 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:20.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:20.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:20.334 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2020-12-05T17:32:20.334 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfojyqJBAAyta+dSy2yGnC1kKe3YOAYw==", 2020-12-05T17:32:20.334 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:32:20.334 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:32:20.335 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2020-12-05T17:32:20.335 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2020-12-05T17:32:20.335 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2020-12-05T17:32:20.335 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:20.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:20.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:20.336 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2020-12-05T17:32:20.336 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDFw8tfES0PLBAAA1cUV44snAUDF2TyAmXImw==", 2020-12-05T17:32:20.336 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:32:20.336 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:32:20.337 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:32:20.337 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:32:20.337 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:32:20.337 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:20.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:20.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:20.338 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-mds", 2020-12-05T17:32:20.338 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDow8tfLp1DOhAAQIgvczAemsgctx4+2tajpw==", 2020-12-05T17:32:20.338 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:32:20.339 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-mds" 2020-12-05T17:32:20.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:20.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:20.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:20.339 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-mgr", 2020-12-05T17:32:20.340 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDow8tfYbxDOhAAxVkSu6yBSpuSZUn8FPBXNQ==", 2020-12-05T17:32:20.340 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:32:20.340 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-mgr" 2020-12-05T17:32:20.340 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:20.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:20.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:20.341 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-osd", 2020-12-05T17:32:20.341 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDow8tfpddDOhAAHGI5xaDRsGNJzLvaJQPwCQ==", 2020-12-05T17:32:20.341 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:32:20.341 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-osd" 2020-12-05T17:32:20.341 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:20.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:20.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:20.342 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rbd", 2020-12-05T17:32:20.342 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDow8tfffBDOhAAOaz4fX74SlGkvnTx14PW9w==", 2020-12-05T17:32:20.342 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:32:20.343 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rbd" 2020-12-05T17:32:20.343 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:20.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:20.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:20.343 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rbd-mirror", 2020-12-05T17:32:20.344 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDow8tfDwpEOhAAdRU1bGM+iM8AEf/Z5Toi7A==", 2020-12-05T17:32:20.344 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:32:20.344 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2020-12-05T17:32:20.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:20.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:20.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:20.345 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rgw", 2020-12-05T17:32:20.345 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDow8tfvyJEOhAAqGSLDttC1jxqooFn48FQxg==", 2020-12-05T17:32:20.345 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:32:20.345 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rgw" 2020-12-05T17:32:20.346 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:20.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:20.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:20.346 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2020-12-05T17:32:20.346 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfDr8mERAAS7bpAQXgW78W8GDq1v1zaA==", 2020-12-05T17:32:20.347 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:32:20.347 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:32:20.347 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:32:20.347 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:32:20.347 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:20.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:20.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:20.348 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2020-12-05T17:32:20.348 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tf3HmcDhAATNKHhH9zgasedaNEzgG0OQ==", 2020-12-05T17:32:20.348 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:32:20.349 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:32:20.349 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:32:20.349 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:32:20.349 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:20.349 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:20.350 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:32:20.350 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:32:20.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.307+0000 7fb4863bd700 1 -- 172.21.15.32:0/584214793 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb4600553f0 msgr2=0x7fb4600578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:20.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.307+0000 7fb4863bd700 1 --2- 172.21.15.32:0/584214793 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb4600553f0 0x7fb4600578b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fb47400b000 tx=0x7fb474006dc0).stop 2020-12-05T17:32:20.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.307+0000 7fb4863bd700 1 -- 172.21.15.32:0/584214793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb480147400 msgr2=0x7fb480158b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:20.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.307+0000 7fb4863bd700 1 --2- 172.21.15.32:0/584214793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb480147400 0x7fb480158b00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7fb4780032b0 tx=0x7fb478016000).stop 2020-12-05T17:32:20.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.311+0000 7fb4863bd700 1 -- 172.21.15.32:0/584214793 shutdown_connections 2020-12-05T17:32:20.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.311+0000 7fb4863bd700 1 --2- 172.21.15.32:0/584214793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb480147400 0x7fb480158b00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:20.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.311+0000 7fb4863bd700 1 --2- 172.21.15.32:0/584214793 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4800ab710 0x7fb48009e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:20.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.311+0000 7fb4863bd700 1 --2- 172.21.15.32:0/584214793 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb4600553f0 0x7fb4600578b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:20.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.311+0000 7fb4863bd700 1 --2- 172.21.15.32:0/584214793 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb4800b1fc0 0x7fb4800a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:20.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.311+0000 7fb4863bd700 1 -- 172.21.15.32:0/584214793 >> 172.21.15.32:0/584214793 conn(0x7fb4800b5530 msgr2=0x7fb4800af740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:20.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.311+0000 7fb4863bd700 1 -- 172.21.15.32:0/584214793 shutdown_connections 2020-12-05T17:32:20.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.311+0000 7fb4863bd700 1 -- 172.21.15.32:0/584214793 wait complete. 2020-12-05T17:32:20.354 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-mds 2020-12-05T17:32:20.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.467+0000 7f003d42a700 1 Processor -- start 2020-12-05T17:32:20.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.467+0000 7f003d42a700 1 -- start start 2020-12-05T17:32:20.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.467+0000 7f003d42a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0038140170 0x7f0038142a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:20.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.467+0000 7f003d42a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f003805fee0 con 0x7f0038067760 2020-12-05T17:32:20.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.467+0000 7f003d42a700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0038143060 con 0x7f00380acf30 2020-12-05T17:32:20.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.467+0000 7f003d42a700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00380b1bb0 con 0x7f0038140170 2020-12-05T17:32:20.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.467+0000 7f00367fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0038067760 0x7f00380ae4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36340/0 (socket says 172.21.15.32:36340) 2020-12-05T17:32:20.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.467+0000 7f00367fc700 1 -- 172.21.15.32:0/1432773293 learned_addr learned my addr 172.21.15.32:0/1432773293 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:20.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.467+0000 7f00377fe700 1 --2- 172.21.15.32:0/1432773293 >> v2:172.21.15.61:3300/0 conn(0x7f0038140170 0x7f0038142a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:20.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.467+0000 7f0035ffb700 1 -- 172.21.15.32:0/1432773293 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 786156663 0 0) 0x7f003805fee0 con 0x7f0038067760 2020-12-05T17:32:20.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.467+0000 7f0035ffb700 1 -- 172.21.15.32:0/1432773293 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0018003230 con 0x7f0038067760 2020-12-05T17:32:20.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.467+0000 7f0035ffb700 1 -- 172.21.15.32:0/1432773293 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828854964 0 0) 0x7f0038143060 con 0x7f00380acf30 2020-12-05T17:32:20.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.467+0000 7f00377fe700 1 -- 172.21.15.32:0/1432773293 >> v1:172.21.15.61:6789/0 conn(0x7f00380acf30 legacy=0x7f003813fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:20.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.467+0000 7f00377fe700 1 -- 172.21.15.32:0/1432773293 >> v1:172.21.15.32:6789/0 conn(0x7f0038067760 legacy=0x7f00380ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:20.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.467+0000 7f00377fe700 1 -- 172.21.15.32:0/1432773293 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0038143300 con 0x7f0038140170 2020-12-05T17:32:20.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.467+0000 7f00377fe700 1 --2- 172.21.15.32:0/1432773293 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0038140170 0x7f0038142a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f0028002f30 tx=0x7f0028006dc0).ready entity=mon.1 client_cookie=368f798bb0ac81bc server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:20.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.467+0000 7f0035ffb700 1 -- 172.21.15.32:0/1432773293 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0028009800 con 0x7f0038140170 2020-12-05T17:32:20.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.467+0000 7f0035ffb700 1 -- 172.21.15.32:0/1432773293 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0028009960 con 0x7f0038140170 2020-12-05T17:32:20.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.467+0000 7f0035ffb700 1 -- 172.21.15.32:0/1432773293 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0028009cc0 con 0x7f0038140170 2020-12-05T17:32:20.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.467+0000 7f003d42a700 1 -- 172.21.15.32:0/1432773293 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0038140170 msgr2=0x7f0038142a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:20.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.467+0000 7f003d42a700 1 --2- 172.21.15.32:0/1432773293 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0038140170 0x7f0038142a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f0028002f30 tx=0x7f0028006dc0).stop 2020-12-05T17:32:20.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f003d42a700 1 -- 172.21.15.32:0/1432773293 shutdown_connections 2020-12-05T17:32:20.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f003d42a700 1 --2- 172.21.15.32:0/1432773293 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0038140170 0x7f0038142a70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:20.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f003d42a700 1 -- 172.21.15.32:0/1432773293 >> 172.21.15.32:0/1432773293 conn(0x7f00380b5580 msgr2=0x7f00380b5990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:20.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f003d42a700 1 -- 172.21.15.32:0/1432773293 shutdown_connections 2020-12-05T17:32:20.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f003d42a700 1 -- 172.21.15.32:0/1432773293 wait complete. 2020-12-05T17:32:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f003d42a700 1 Processor -- start 2020-12-05T17:32:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f003d42a700 1 -- start start 2020-12-05T17:32:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f003d42a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0038067760 0x7f003810f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f003d42a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00380acf30 0x7f003813e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f003d42a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0038140170 0x7f003813e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f003d42a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0038148e50 con 0x7f00380acf30 2020-12-05T17:32:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f003d42a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003810eb00 con 0x7f0038140170 2020-12-05T17:32:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f003d42a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003810e860 con 0x7f0038067760 2020-12-05T17:32:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f00367fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00380acf30 0x7f003813e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f00367fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00380acf30 0x7f003813e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33696/0 (socket says 172.21.15.32:33696) 2020-12-05T17:32:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f00367fc700 1 -- 172.21.15.32:0/335161677 learned_addr learned my addr 172.21.15.32:0/335161677 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f0036ffd700 1 --2- 172.21.15.32:0/335161677 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0038067760 0x7f003810f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f00377fe700 1 --2- 172.21.15.32:0/335161677 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0038140170 0x7f003813e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f00367fc700 1 -- 172.21.15.32:0/335161677 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0038067760 msgr2=0x7f003810f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:20.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f00367fc700 1 --2- 172.21.15.32:0/335161677 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0038067760 0x7f003810f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:20.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f00367fc700 1 -- 172.21.15.32:0/335161677 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0038140170 msgr2=0x7f003813e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:20.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f00367fc700 1 --2- 172.21.15.32:0/335161677 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0038140170 0x7f003813e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:20.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f00367fc700 1 -- 172.21.15.32:0/335161677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003813ee70 con 0x7f00380acf30 2020-12-05T17:32:20.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f00377fe700 1 --2- 172.21.15.32:0/335161677 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0038140170 0x7f003813e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:32:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f0036ffd700 1 --2- 172.21.15.32:0/335161677 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0038067760 0x7f003810f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:32:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.471+0000 7f00367fc700 1 --2- 172.21.15.32:0/335161677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00380acf30 0x7f003813e490 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f0020001450 tx=0x7f0020007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.475+0000 7f002ffff700 1 -- 172.21.15.32:0/335161677 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0020005c40 con 0x7f00380acf30 2020-12-05T17:32:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.475+0000 7f003d42a700 1 -- 172.21.15.32:0/335161677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0038111950 con 0x7f00380acf30 2020-12-05T17:32:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.475+0000 7f002ffff700 1 -- 172.21.15.32:0/335161677 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0020005da0 con 0x7f00380acf30 2020-12-05T17:32:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.475+0000 7f002ffff700 1 -- 172.21.15.32:0/335161677 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0020007300 con 0x7f00380acf30 2020-12-05T17:32:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.475+0000 7f003d42a700 1 -- 172.21.15.32:0/335161677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f003810f500 con 0x7f00380acf30 2020-12-05T17:32:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.475+0000 7f003d42a700 1 -- 172.21.15.32:0/335161677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00381480d0 con 0x7f00380acf30 2020-12-05T17:32:20.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.475+0000 7f002ffff700 1 -- 172.21.15.32:0/335161677 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0020010030 con 0x7f00380acf30 2020-12-05T17:32:20.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.475+0000 7f002ffff700 1 --2- 172.21.15.32:0/335161677 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f001c0554e0 0x7f001c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:20.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.475+0000 7f002ffff700 1 -- 172.21.15.32:0/335161677 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f002006dec0 con 0x7f00380acf30 2020-12-05T17:32:20.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.475+0000 7f0036ffd700 1 --2- 172.21.15.32:0/335161677 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f001c0554e0 0x7f001c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:20.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.475+0000 7f0036ffd700 1 --2- 172.21.15.32:0/335161677 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f001c0554e0 0x7f001c0579a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f0030008000 tx=0x7f0030009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:20.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.483+0000 7f002ffff700 1 -- 172.21.15.32:0/335161677 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f00200443b0 con 0x7f00380acf30 2020-12-05T17:32:20.676 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.671+0000 7f003d42a700 1 -- 172.21.15.32:0/335161677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f0038148c70 con 0x7f00380acf30 2020-12-05T17:32:20.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.679+0000 7f002ffff700 1 -- 172.21.15.32:0/335161677 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 updated v5) v1 ==== 97+0+0 (secure 0 0 0) 0x7f00200479d0 con 0x7f00380acf30 2020-12-05T17:32:20.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.683+0000 7f003d42a700 1 -- 172.21.15.32:0/335161677 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f001c0554e0 msgr2=0x7f001c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:20.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.683+0000 7f003d42a700 1 --2- 172.21.15.32:0/335161677 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f001c0554e0 0x7f001c0579a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f0030008000 tx=0x7f0030009040).stop 2020-12-05T17:32:20.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.683+0000 7f003d42a700 1 -- 172.21.15.32:0/335161677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00380acf30 msgr2=0x7f003813e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:20.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.683+0000 7f003d42a700 1 --2- 172.21.15.32:0/335161677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00380acf30 0x7f003813e490 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f0020001450 tx=0x7f0020007df0).stop 2020-12-05T17:32:20.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.683+0000 7f003d42a700 1 -- 172.21.15.32:0/335161677 shutdown_connections 2020-12-05T17:32:20.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.683+0000 7f003d42a700 1 --2- 172.21.15.32:0/335161677 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0038140170 0x7f003813e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:20.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.683+0000 7f003d42a700 1 --2- 172.21.15.32:0/335161677 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f001c0554e0 0x7f001c0579a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:20.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.683+0000 7f003d42a700 1 --2- 172.21.15.32:0/335161677 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0038067760 0x7f003810f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:20.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.683+0000 7f003d42a700 1 --2- 172.21.15.32:0/335161677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00380acf30 0x7f003813e490 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:20.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.683+0000 7f003d42a700 1 -- 172.21.15.32:0/335161677 >> 172.21.15.32:0/335161677 conn(0x7f00380b5580 msgr2=0x7f0038142690 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:20.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.683+0000 7f003d42a700 1 -- 172.21.15.32:0/335161677 shutdown_connections 2020-12-05T17:32:20.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.683+0000 7f003d42a700 1 -- 172.21.15.32:0/335161677 wait complete. 2020-12-05T17:32:20.689 INFO:teuthology.orchestra.run.smithi032.stderr:updated 2020-12-05T17:32:20.699 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-mgr 2020-12-05T17:32:20.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.811+0000 7f39bb089700 1 Processor -- start 2020-12-05T17:32:20.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.811+0000 7f39bb089700 1 -- start start 2020-12-05T17:32:20.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.811+0000 7f39bb089700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f39b40b52f0 0x7f39b40b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:20.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.811+0000 7f39bb089700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f39b40b1870 0x7f39b4138810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:20.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.811+0000 7f39bb089700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39b41420f0 con 0x7f39b4138d00 2020-12-05T17:32:20.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.811+0000 7f39bb089700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b40ad4f0 con 0x7f39b40b1870 2020-12-05T17:32:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.811+0000 7f39bb089700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b40b0380 con 0x7f39b40b52f0 2020-12-05T17:32:20.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.815+0000 7f39b9886700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f39b40b1870 0x7f39b4138810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:20.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.815+0000 7f39ba087700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f39b40b52f0 0x7f39b40b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:20.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.815+0000 7f39b9886700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f39b40b1870 0x7f39b4138810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:53596/0 (socket says 172.21.15.32:53596) 2020-12-05T17:32:20.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.815+0000 7f39ba888700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f39b4138d00 0x7f39b40ae860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:48996/0 (socket says 172.21.15.32:48996) 2020-12-05T17:32:20.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.815+0000 7f39b9886700 1 -- 172.21.15.32:0/172423583 learned_addr learned my addr 172.21.15.32:0/172423583 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:20.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.815+0000 7f39b9886700 1 -- 172.21.15.32:0/172423583 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f39b40b52f0 msgr2=0x7f39b40b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:20.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.815+0000 7f39b9886700 1 --2- 172.21.15.32:0/172423583 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f39b40b52f0 0x7f39b40b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:20.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.815+0000 7f39b9886700 1 -- 172.21.15.32:0/172423583 >> v1:172.21.15.61:6789/0 conn(0x7f39b4138d00 legacy=0x7f39b40ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:20.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.815+0000 7f39b9886700 1 -- 172.21.15.32:0/172423583 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b40b0620 con 0x7f39b40b1870 2020-12-05T17:32:20.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.815+0000 7f39b9886700 1 --2- 172.21.15.32:0/172423583 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f39b40b1870 0x7f39b4138810 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f39b40adc80 tx=0x7f39a4006e80).ready entity=mon.1 client_cookie=707b31bf8e162f5 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:20.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.815+0000 7f39b9085700 1 -- 172.21.15.32:0/172423583 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39a40097b0 con 0x7f39b40b1870 2020-12-05T17:32:20.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.815+0000 7f39b9085700 1 -- 172.21.15.32:0/172423583 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a4009910 con 0x7f39b40b1870 2020-12-05T17:32:20.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.815+0000 7f39bb089700 1 -- 172.21.15.32:0/172423583 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f39b40b1870 msgr2=0x7f39b4138810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:20.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.815+0000 7f39bb089700 1 --2- 172.21.15.32:0/172423583 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f39b40b1870 0x7f39b4138810 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f39b40adc80 tx=0x7f39a4006e80).stop 2020-12-05T17:32:20.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.815+0000 7f39b9085700 1 -- 172.21.15.32:0/172423583 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39a4009c70 con 0x7f39b40b1870 2020-12-05T17:32:20.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.815+0000 7f39bb089700 1 -- 172.21.15.32:0/172423583 shutdown_connections 2020-12-05T17:32:20.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.815+0000 7f39bb089700 1 --2- 172.21.15.32:0/172423583 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f39b40b52f0 0x7f39b40b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:20.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.815+0000 7f39bb089700 1 --2- 172.21.15.32:0/172423583 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f39b40b1870 0x7f39b4138810 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:20.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.815+0000 7f39bb089700 1 -- 172.21.15.32:0/172423583 >> 172.21.15.32:0/172423583 conn(0x7f39b405bbb0 msgr2=0x7f39b4062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:20.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.815+0000 7f39bb089700 1 -- 172.21.15.32:0/172423583 shutdown_connections 2020-12-05T17:32:20.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.815+0000 7f39bb089700 1 -- 172.21.15.32:0/172423583 wait complete. 2020-12-05T17:32:20.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39bb089700 1 Processor -- start 2020-12-05T17:32:20.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39bb089700 1 -- start start 2020-12-05T17:32:20.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39bb089700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39b40b1870 0x7f39b41394a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:20.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39bb089700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f39b40b52f0 0x7f39b4153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:20.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39bb089700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f39b4138d00 0x7f39b4153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:20.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39bb089700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b4141ea0 con 0x7f39b40b1870 2020-12-05T17:32:20.826 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39bb089700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b413aec0 con 0x7f39b40b52f0 2020-12-05T17:32:20.826 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39bb089700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b413b160 con 0x7f39b4138d00 2020-12-05T17:32:20.826 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39ba087700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39b40b1870 0x7f39b41394a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:20.826 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39ba087700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39b40b1870 0x7f39b41394a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33710/0 (socket says 172.21.15.32:33710) 2020-12-05T17:32:20.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39ba087700 1 -- 172.21.15.32:0/876737148 learned_addr learned my addr 172.21.15.32:0/876737148 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:20.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39ba888700 1 --2- 172.21.15.32:0/876737148 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f39b4138d00 0x7f39b4153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:20.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39b9886700 1 --2- 172.21.15.32:0/876737148 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f39b40b52f0 0x7f39b4153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:20.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39ba087700 1 -- 172.21.15.32:0/876737148 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f39b4138d00 msgr2=0x7f39b4153520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:20.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39ba087700 1 --2- 172.21.15.32:0/876737148 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f39b4138d00 0x7f39b4153520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:20.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39ba087700 1 -- 172.21.15.32:0/876737148 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f39b40b52f0 msgr2=0x7f39b4153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:20.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39ba087700 1 --2- 172.21.15.32:0/876737148 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f39b40b52f0 0x7f39b4153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:20.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39ba087700 1 -- 172.21.15.32:0/876737148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b413b400 con 0x7f39b40b1870 2020-12-05T17:32:20.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39ba087700 1 --2- 172.21.15.32:0/876737148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39b40b1870 0x7f39b41394a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f39b000c000 tx=0x7f39b0006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:20.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39a37fe700 1 -- 172.21.15.32:0/876737148 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39b000ca10 con 0x7f39b40b1870 2020-12-05T17:32:20.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39a37fe700 1 -- 172.21.15.32:0/876737148 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b000cb70 con 0x7f39b40b1870 2020-12-05T17:32:20.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39bb089700 1 -- 172.21.15.32:0/876737148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b4153a10 con 0x7f39b40b1870 2020-12-05T17:32:20.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39a37fe700 1 -- 172.21.15.32:0/876737148 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39b00123f0 con 0x7f39b40b1870 2020-12-05T17:32:20.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.819+0000 7f39bb089700 1 -- 172.21.15.32:0/876737148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b41416e0 con 0x7f39b40b1870 2020-12-05T17:32:20.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.823+0000 7f39a37fe700 1 -- 172.21.15.32:0/876737148 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f39b0012550 con 0x7f39b40b1870 2020-12-05T17:32:20.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.823+0000 7f39a37fe700 1 --2- 172.21.15.32:0/876737148 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3994055440 0x7f3994057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:20.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.823+0000 7f39bb089700 1 -- 172.21.15.32:0/876737148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b4057050 con 0x7f39b40b1870 2020-12-05T17:32:20.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.823+0000 7f39b9886700 1 --2- 172.21.15.32:0/876737148 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3994055440 0x7f3994057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:20.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.823+0000 7f39a37fe700 1 -- 172.21.15.32:0/876737148 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f39b006e080 con 0x7f39b40b1870 2020-12-05T17:32:20.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.823+0000 7f39b9886700 1 --2- 172.21.15.32:0/876737148 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3994055440 0x7f3994057900 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f39a401c000 tx=0x7f39a4015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:20.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:20.831+0000 7f39a37fe700 1 -- 172.21.15.32:0/876737148 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f39b0044500 con 0x7f39b40b1870 2020-12-05T17:32:21.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.027+0000 7f39bb089700 1 -- 172.21.15.32:0/876737148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f39b4067080 con 0x7f39b40b1870 2020-12-05T17:32:21.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.035+0000 7f39a37fe700 1 -- 172.21.15.32:0/876737148 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 updated v6) v1 ==== 97+0+0 (secure 0 0 0) 0x7f39b0047b20 con 0x7f39b40b1870 2020-12-05T17:32:21.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.035+0000 7f39bb089700 1 -- 172.21.15.32:0/876737148 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3994055440 msgr2=0x7f3994057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:21.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.035+0000 7f39bb089700 1 --2- 172.21.15.32:0/876737148 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3994055440 0x7f3994057900 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f39a401c000 tx=0x7f39a4015000).stop 2020-12-05T17:32:21.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.035+0000 7f39bb089700 1 -- 172.21.15.32:0/876737148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39b40b1870 msgr2=0x7f39b41394a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:21.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.035+0000 7f39bb089700 1 --2- 172.21.15.32:0/876737148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39b40b1870 0x7f39b41394a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f39b000c000 tx=0x7f39b0006dc0).stop 2020-12-05T17:32:21.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.035+0000 7f39bb089700 1 -- 172.21.15.32:0/876737148 shutdown_connections 2020-12-05T17:32:21.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.035+0000 7f39bb089700 1 --2- 172.21.15.32:0/876737148 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3994055440 0x7f3994057900 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.035+0000 7f39bb089700 1 --2- 172.21.15.32:0/876737148 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f39b4138d00 0x7f39b4153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.035+0000 7f39bb089700 1 --2- 172.21.15.32:0/876737148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39b40b1870 0x7f39b41394a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.035+0000 7f39bb089700 1 --2- 172.21.15.32:0/876737148 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f39b40b52f0 0x7f39b4153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.035+0000 7f39bb089700 1 -- 172.21.15.32:0/876737148 >> 172.21.15.32:0/876737148 conn(0x7f39b405bbb0 msgr2=0x7f39b4067320 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:21.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.035+0000 7f39bb089700 1 -- 172.21.15.32:0/876737148 shutdown_connections 2020-12-05T17:32:21.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.035+0000 7f39bb089700 1 -- 172.21.15.32:0/876737148 wait complete. 2020-12-05T17:32:21.042 INFO:teuthology.orchestra.run.smithi032.stderr:updated 2020-12-05T17:32:21.051 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-osd 2020-12-05T17:32:21.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.163+0000 7fa42bfff700 1 Processor -- start 2020-12-05T17:32:21.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.163+0000 7fa42bfff700 1 -- start start 2020-12-05T17:32:21.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.163+0000 7fa42bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa42c144540 0x7fa42c061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:21.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.163+0000 7fa42bfff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa42c1403f0 0x7fa42c13d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:21.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.163+0000 7fa42bfff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa42c143a00 con 0x7fa42c0b7720 2020-12-05T17:32:21.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.163+0000 7fa42bfff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa42c140880 con 0x7fa42c1403f0 2020-12-05T17:32:21.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.163+0000 7fa42bfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c0b7b00 con 0x7fa42c144540 2020-12-05T17:32:21.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.167+0000 7fa42a7fc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa42c144540 0x7fa42c061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:21.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.167+0000 7fa42b7fe700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa42c1403f0 0x7fa42c13d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:21.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.167+0000 7fa42affd700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fa42c0b7720 0x7fa42c143e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:49006/0 (socket says 172.21.15.32:49006) 2020-12-05T17:32:21.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.167+0000 7fa42a7fc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa42c144540 0x7fa42c061780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:58744/0 (socket says 172.21.15.32:58744) 2020-12-05T17:32:21.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.167+0000 7fa42affd700 1 -- 172.21.15.32:0/600517882 learned_addr learned my addr 172.21.15.32:0/600517882 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:21.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.167+0000 7fa429ffb700 1 -- 172.21.15.32:0/600517882 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1252936515 0 0) 0x7fa42c143a00 con 0x7fa42c0b7720 2020-12-05T17:32:21.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.167+0000 7fa429ffb700 1 -- 172.21.15.32:0/600517882 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa410003230 con 0x7fa42c0b7720 2020-12-05T17:32:21.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.167+0000 7fa42a7fc700 1 -- 172.21.15.32:0/600517882 >> v2:172.21.15.61:3300/0 conn(0x7fa42c1403f0 msgr2=0x7fa42c13d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:21.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.167+0000 7fa42a7fc700 1 --2- 172.21.15.32:0/600517882 >> v2:172.21.15.61:3300/0 conn(0x7fa42c1403f0 0x7fa42c13d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.167+0000 7fa42a7fc700 1 -- 172.21.15.32:0/600517882 >> v1:172.21.15.61:6789/0 conn(0x7fa42c0b7720 legacy=0x7fa42c143e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:21.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.167+0000 7fa42a7fc700 1 -- 172.21.15.32:0/600517882 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42c05fe60 con 0x7fa42c144540 2020-12-05T17:32:21.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.167+0000 7fa42a7fc700 1 --2- 172.21.15.32:0/600517882 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa42c144540 0x7fa42c061780 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fa414002f30 tx=0x7fa414006dc0).ready entity=mon.2 client_cookie=dadb4aec7abe1ed8 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:21.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.167+0000 7fa429ffb700 1 -- 172.21.15.32:0/600517882 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4140096a0 con 0x7fa42c144540 2020-12-05T17:32:21.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.167+0000 7fa429ffb700 1 -- 172.21.15.32:0/600517882 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa414009800 con 0x7fa42c144540 2020-12-05T17:32:21.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.167+0000 7fa429ffb700 1 -- 172.21.15.32:0/600517882 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa414009b60 con 0x7fa42c144540 2020-12-05T17:32:21.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.167+0000 7fa42bfff700 1 -- 172.21.15.32:0/600517882 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa42c144540 msgr2=0x7fa42c061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:21.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.167+0000 7fa42bfff700 1 --2- 172.21.15.32:0/600517882 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa42c144540 0x7fa42c061780 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fa414002f30 tx=0x7fa414006dc0).stop 2020-12-05T17:32:21.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.167+0000 7fa42bfff700 1 -- 172.21.15.32:0/600517882 shutdown_connections 2020-12-05T17:32:21.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.167+0000 7fa42bfff700 1 --2- 172.21.15.32:0/600517882 >> v2:172.21.15.61:3300/0 conn(0x7fa42c1403f0 0x7fa42c13d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.167+0000 7fa42bfff700 1 --2- 172.21.15.32:0/600517882 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa42c144540 0x7fa42c061780 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.167+0000 7fa42bfff700 1 -- 172.21.15.32:0/600517882 >> 172.21.15.32:0/600517882 conn(0x7fa42c05bc20 msgr2=0x7fa42c0b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:21.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.167+0000 7fa42bfff700 1 -- 172.21.15.32:0/600517882 shutdown_connections 2020-12-05T17:32:21.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.167+0000 7fa42bfff700 1 -- 172.21.15.32:0/600517882 wait complete. 2020-12-05T17:32:21.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.171+0000 7fa42bfff700 1 Processor -- start 2020-12-05T17:32:21.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.171+0000 7fa42bfff700 1 -- start start 2020-12-05T17:32:21.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.171+0000 7fa42bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa42c0b7720 0x7fa42c065010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:21.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.171+0000 7fa42bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa42c1403f0 0x7fa42c0add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.171+0000 7fa42bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa42c144540 0x7fa42c154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.171+0000 7fa42bfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c154720 con 0x7fa42c144540 2020-12-05T17:32:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.171+0000 7fa42bfff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c0b3830 con 0x7fa42c1403f0 2020-12-05T17:32:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.171+0000 7fa42bfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c154950 con 0x7fa42c0b7720 2020-12-05T17:32:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.171+0000 7fa42b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa42c144540 0x7fa42c154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:21.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.171+0000 7fa42b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa42c144540 0x7fa42c154df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33728/0 (socket says 172.21.15.32:33728) 2020-12-05T17:32:21.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.171+0000 7fa42b7fe700 1 -- 172.21.15.32:0/2483426817 learned_addr learned my addr 172.21.15.32:0/2483426817 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:21.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.171+0000 7fa42affd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa42c0b7720 0x7fa42c065010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:21.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.171+0000 7fa42a7fc700 1 --2- 172.21.15.32:0/2483426817 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa42c1403f0 0x7fa42c0add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:21.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.175+0000 7fa42b7fe700 1 -- 172.21.15.32:0/2483426817 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa42c0b7720 msgr2=0x7fa42c065010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:21.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.175+0000 7fa42b7fe700 1 --2- 172.21.15.32:0/2483426817 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa42c0b7720 0x7fa42c065010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.175+0000 7fa42b7fe700 1 -- 172.21.15.32:0/2483426817 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa42c1403f0 msgr2=0x7fa42c0add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:21.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.175+0000 7fa42b7fe700 1 --2- 172.21.15.32:0/2483426817 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa42c1403f0 0x7fa42c0add50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.175+0000 7fa42b7fe700 1 -- 172.21.15.32:0/2483426817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42c065500 con 0x7fa42c144540 2020-12-05T17:32:21.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.175+0000 7fa42b7fe700 1 --2- 172.21.15.32:0/2483426817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa42c144540 0x7fa42c154df0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fa42400c000 tx=0x7fa424006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:21.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.175+0000 7fa423fff700 1 -- 172.21.15.32:0/2483426817 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa42400ca10 con 0x7fa42c144540 2020-12-05T17:32:21.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.175+0000 7fa423fff700 1 -- 172.21.15.32:0/2483426817 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42400cb70 con 0x7fa42c144540 2020-12-05T17:32:21.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.175+0000 7fa42bfff700 1 -- 172.21.15.32:0/2483426817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa42c0ae240 con 0x7fa42c144540 2020-12-05T17:32:21.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.175+0000 7fa423fff700 1 -- 172.21.15.32:0/2483426817 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa42400be70 con 0x7fa42c144540 2020-12-05T17:32:21.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.175+0000 7fa42bfff700 1 -- 172.21.15.32:0/2483426817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa42c0b4230 con 0x7fa42c144540 2020-12-05T17:32:21.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.175+0000 7fa42bfff700 1 -- 172.21.15.32:0/2483426817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa42c154b10 con 0x7fa42c144540 2020-12-05T17:32:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.175+0000 7fa423fff700 1 -- 172.21.15.32:0/2483426817 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa42401c460 con 0x7fa42c144540 2020-12-05T17:32:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.175+0000 7fa423fff700 1 --2- 172.21.15.32:0/2483426817 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa404055440 0x7fa404057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.175+0000 7fa423fff700 1 -- 172.21.15.32:0/2483426817 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fa42406e1c0 con 0x7fa42c144540 2020-12-05T17:32:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.179+0000 7fa42affd700 1 --2- 172.21.15.32:0/2483426817 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa404055440 0x7fa404057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.179+0000 7fa42affd700 1 --2- 172.21.15.32:0/2483426817 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa404055440 0x7fa404057900 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fa41c00d000 tx=0x7fa41c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.183+0000 7fa423fff700 1 -- 172.21.15.32:0/2483426817 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa424044640 con 0x7fa42c144540 2020-12-05T17:32:21.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.379+0000 7fa42bfff700 1 -- 172.21.15.32:0/2483426817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fa42c057050 con 0x7fa42c144540 2020-12-05T17:32:21.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.383+0000 7fa423fff700 1 -- 172.21.15.32:0/2483426817 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 updated v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa424047c60 con 0x7fa42c144540 2020-12-05T17:32:21.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.387+0000 7fa42bfff700 1 -- 172.21.15.32:0/2483426817 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa404055440 msgr2=0x7fa404057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:21.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.387+0000 7fa42bfff700 1 --2- 172.21.15.32:0/2483426817 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa404055440 0x7fa404057900 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fa41c00d000 tx=0x7fa41c009040).stop 2020-12-05T17:32:21.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.387+0000 7fa42bfff700 1 -- 172.21.15.32:0/2483426817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa42c144540 msgr2=0x7fa42c154df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:21.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.387+0000 7fa42bfff700 1 --2- 172.21.15.32:0/2483426817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa42c144540 0x7fa42c154df0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fa42400c000 tx=0x7fa424006dc0).stop 2020-12-05T17:32:21.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.387+0000 7fa42bfff700 1 -- 172.21.15.32:0/2483426817 shutdown_connections 2020-12-05T17:32:21.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.387+0000 7fa42bfff700 1 --2- 172.21.15.32:0/2483426817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa42c144540 0x7fa42c154df0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.387+0000 7fa42bfff700 1 --2- 172.21.15.32:0/2483426817 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa404055440 0x7fa404057900 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.387+0000 7fa42bfff700 1 --2- 172.21.15.32:0/2483426817 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa42c0b7720 0x7fa42c065010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.387+0000 7fa42bfff700 1 --2- 172.21.15.32:0/2483426817 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa42c1403f0 0x7fa42c0add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.387+0000 7fa42bfff700 1 -- 172.21.15.32:0/2483426817 >> 172.21.15.32:0/2483426817 conn(0x7fa42c05bc20 msgr2=0x7fa42c05f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:21.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.387+0000 7fa42bfff700 1 -- 172.21.15.32:0/2483426817 shutdown_connections 2020-12-05T17:32:21.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.387+0000 7fa42bfff700 1 -- 172.21.15.32:0/2483426817 wait complete. 2020-12-05T17:32:21.393 INFO:teuthology.orchestra.run.smithi032.stderr:updated 2020-12-05T17:32:21.405 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-rbd 2020-12-05T17:32:21.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.519+0000 7f098ffff700 1 Processor -- start 2020-12-05T17:32:21.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.519+0000 7f098ffff700 1 -- start start 2020-12-05T17:32:21.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.519+0000 7f098ffff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f09900b1e20 0x7f0990143b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:21.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.519+0000 7f098ffff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09900ab4a0 con 0x7f0990144790 2020-12-05T17:32:21.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.519+0000 7f098ffff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09900aca70 con 0x7f09900aec10 2020-12-05T17:32:21.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.519+0000 7f098ffff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09900ad800 con 0x7f09900b1e20 2020-12-05T17:32:21.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.519+0000 7f098f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0990144790 0x7f0990140960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36380/0 (socket says 172.21.15.32:36380) 2020-12-05T17:32:21.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.519+0000 7f098f7fe700 1 -- 172.21.15.32:0/3099909126 learned_addr learned my addr 172.21.15.32:0/3099909126 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:21.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.519+0000 7f098effd700 1 --2- 172.21.15.32:0/3099909126 >> v2:172.21.15.61:3300/0 conn(0x7f09900b1e20 0x7f0990143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:21.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.519+0000 7f098dffb700 1 -- 172.21.15.32:0/3099909126 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1723646045 0 0) 0x7f09900ab4a0 con 0x7f0990144790 2020-12-05T17:32:21.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.519+0000 7f098dffb700 1 -- 172.21.15.32:0/3099909126 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0970003230 con 0x7f0990144790 2020-12-05T17:32:21.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.519+0000 7f098dffb700 1 -- 172.21.15.32:0/3099909126 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2271381457 0 0) 0x7f09900aca70 con 0x7f09900aec10 2020-12-05T17:32:21.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.519+0000 7f098dffb700 1 -- 172.21.15.32:0/3099909126 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09900ab4a0 con 0x7f09900aec10 2020-12-05T17:32:21.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.519+0000 7f098dffb700 1 -- 172.21.15.32:0/3099909126 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4052788799 0 0) 0x7f0970003230 con 0x7f0990144790 2020-12-05T17:32:21.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.519+0000 7f098dffb700 1 -- 172.21.15.32:0/3099909126 >> v2:172.21.15.61:3300/0 conn(0x7f09900b1e20 msgr2=0x7f0990143b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:21.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.519+0000 7f098dffb700 1 --2- 172.21.15.32:0/3099909126 >> v2:172.21.15.61:3300/0 conn(0x7f09900b1e20 0x7f0990143b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.519+0000 7f098dffb700 1 -- 172.21.15.32:0/3099909126 >> v1:172.21.15.61:6789/0 conn(0x7f09900aec10 legacy=0x7f0990144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:21.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.519+0000 7f098dffb700 1 -- 172.21.15.32:0/3099909126 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09900ab6d0 con 0x7f0990144790 2020-12-05T17:32:21.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.519+0000 7f098dffb700 1 -- 172.21.15.32:0/3099909126 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f0988001a50 con 0x7f0990144790 2020-12-05T17:32:21.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098dffb700 1 -- 172.21.15.32:0/3099909126 >> v1:172.21.15.32:6789/0 conn(0x7f0990144790 legacy=0x7f0990140960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:21.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098dffb700 1 --2- 172.21.15.32:0/3099909126 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09700038b0 0x7f0970003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:21.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098dffb700 1 --2- 172.21.15.32:0/3099909126 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f09700044d0 0x7f0970004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:21.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098dffb700 1 --2- 172.21.15.32:0/3099909126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0970004e40 0x7f097000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:21.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098dffb700 1 -- 172.21.15.32:0/3099909126 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0970005370 con 0x7f0970004e40 2020-12-05T17:32:21.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098dffb700 1 -- 172.21.15.32:0/3099909126 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097000bc10 con 0x7f09700044d0 2020-12-05T17:32:21.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098dffb700 1 -- 172.21.15.32:0/3099909126 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f097000bdd0 con 0x7f09700038b0 2020-12-05T17:32:21.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098f7fe700 1 --2- 172.21.15.32:0/3099909126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0970004e40 0x7f097000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:21.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098effd700 1 --2- 172.21.15.32:0/3099909126 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09700038b0 0x7f0970003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:21.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098e7fc700 1 --2- 172.21.15.32:0/3099909126 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f09700044d0 0x7f0970004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:21.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098f7fe700 1 -- 172.21.15.32:0/3099909126 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09700038b0 msgr2=0x7f0970003cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:21.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098f7fe700 1 --2- 172.21.15.32:0/3099909126 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09700038b0 0x7f0970003cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098f7fe700 1 -- 172.21.15.32:0/3099909126 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f09700044d0 msgr2=0x7f0970004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:21.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098f7fe700 1 --2- 172.21.15.32:0/3099909126 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f09700044d0 0x7f0970004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098f7fe700 1 -- 172.21.15.32:0/3099909126 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0970003230 con 0x7f0970004e40 2020-12-05T17:32:21.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098f7fe700 1 -- 172.21.15.32:0/3099909126 reap_dead start 2020-12-05T17:32:21.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098f7fe700 1 --2- 172.21.15.32:0/3099909126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0970004e40 0x7f097000b720 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f0988001820 tx=0x7f09900ab860).ready entity=mon.0 client_cookie=6b745ecc1c27816f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:21.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098dffb700 1 -- 172.21.15.32:0/3099909126 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0988005c40 con 0x7f0970004e40 2020-12-05T17:32:21.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098dffb700 1 -- 172.21.15.32:0/3099909126 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0988005da0 con 0x7f0970004e40 2020-12-05T17:32:21.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098dffb700 1 -- 172.21.15.32:0/3099909126 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09880072c0 con 0x7f0970004e40 2020-12-05T17:32:21.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098ffff700 1 -- 172.21.15.32:0/3099909126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0970004e40 msgr2=0x7f097000b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:21.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098ffff700 1 --2- 172.21.15.32:0/3099909126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0970004e40 0x7f097000b720 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f0988001820 tx=0x7f09900ab860).stop 2020-12-05T17:32:21.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098ffff700 1 -- 172.21.15.32:0/3099909126 shutdown_connections 2020-12-05T17:32:21.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098ffff700 1 --2- 172.21.15.32:0/3099909126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0970004e40 0x7f097000b720 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098ffff700 1 -- 172.21.15.32:0/3099909126 >> 172.21.15.32:0/3099909126 conn(0x7f09900b5530 msgr2=0x7f09900b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:21.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098ffff700 1 -- 172.21.15.32:0/3099909126 shutdown_connections 2020-12-05T17:32:21.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098ffff700 1 -- 172.21.15.32:0/3099909126 wait complete. 2020-12-05T17:32:21.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098ffff700 1 Processor -- start 2020-12-05T17:32:21.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.523+0000 7f098ffff700 1 -- start start 2020-12-05T17:32:21.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f098ffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0970004e40 0x7f0990064310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:21.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f098ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099009ed90 0x7f099009f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f098ffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09900a2ca0 0x7f0990152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f098ffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099009ebd0 con 0x7f099009ed90 2020-12-05T17:32:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f098ffff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099009f540 con 0x7f0970004e40 2020-12-05T17:32:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f098ffff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099009f2a0 con 0x7f09900a2ca0 2020-12-05T17:32:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f098e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099009ed90 0x7f099009f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f098e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099009ed90 0x7f099009f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33746/0 (socket says 172.21.15.32:33746) 2020-12-05T17:32:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f098e7fc700 1 -- 172.21.15.32:0/891942758 learned_addr learned my addr 172.21.15.32:0/891942758 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f098effd700 1 --2- 172.21.15.32:0/891942758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0970004e40 0x7f0990064310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f098f7fe700 1 --2- 172.21.15.32:0/891942758 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09900a2ca0 0x7f0990152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:21.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f098e7fc700 1 -- 172.21.15.32:0/891942758 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09900a2ca0 msgr2=0x7f0990152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:21.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f098e7fc700 1 --2- 172.21.15.32:0/891942758 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09900a2ca0 0x7f0990152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f098e7fc700 1 -- 172.21.15.32:0/891942758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0970004e40 msgr2=0x7f0990064310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:21.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f098e7fc700 1 --2- 172.21.15.32:0/891942758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0970004e40 0x7f0990064310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f098e7fc700 1 -- 172.21.15.32:0/891942758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0990152880 con 0x7f099009ed90 2020-12-05T17:32:21.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f098f7fe700 1 --2- 172.21.15.32:0/891942758 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09900a2ca0 0x7f0990152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:21.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f098e7fc700 1 --2- 172.21.15.32:0/891942758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099009ed90 0x7f099009f820 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f097800d000 tx=0x7f0978007b60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:21.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f0987fff700 1 -- 172.21.15.32:0/891942758 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f097800fb40 con 0x7f099009ed90 2020-12-05T17:32:21.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f0987fff700 1 -- 172.21.15.32:0/891942758 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097800fca0 con 0x7f099009ed90 2020-12-05T17:32:21.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f0987fff700 1 -- 172.21.15.32:0/891942758 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0978014450 con 0x7f099009ed90 2020-12-05T17:32:21.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f098ffff700 1 -- 172.21.15.32:0/891942758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09900a3130 con 0x7f099009ed90 2020-12-05T17:32:21.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f098ffff700 1 -- 172.21.15.32:0/891942758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09901533b0 con 0x7f099009ed90 2020-12-05T17:32:21.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.527+0000 7f0987fff700 1 -- 172.21.15.32:0/891942758 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f097801b030 con 0x7f099009ed90 2020-12-05T17:32:21.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.531+0000 7f098ffff700 1 -- 172.21.15.32:0/891942758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0990152a10 con 0x7f099009ed90 2020-12-05T17:32:21.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.531+0000 7f0987fff700 1 --2- 172.21.15.32:0/891942758 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0974055330 0x7f09740577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:21.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.531+0000 7f0987fff700 1 -- 172.21.15.32:0/891942758 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f097806e010 con 0x7f099009ed90 2020-12-05T17:32:21.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.531+0000 7f098effd700 1 --2- 172.21.15.32:0/891942758 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0974055330 0x7f09740577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:21.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.531+0000 7f098effd700 1 --2- 172.21.15.32:0/891942758 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0974055330 0x7f09740577f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f0980008000 tx=0x7f0980008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:21.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.539+0000 7f0987fff700 1 -- 172.21.15.32:0/891942758 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0978044450 con 0x7f099009ed90 2020-12-05T17:32:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.723+0000 7f098ffff700 1 -- 172.21.15.32:0/891942758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f0990057050 con 0x7f099009ed90 2020-12-05T17:32:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.727+0000 7f0987fff700 1 -- 172.21.15.32:0/891942758 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 updated v8) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0978047a70 con 0x7f099009ed90 2020-12-05T17:32:21.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.731+0000 7f098ffff700 1 -- 172.21.15.32:0/891942758 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0974055330 msgr2=0x7f09740577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:21.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.731+0000 7f098ffff700 1 --2- 172.21.15.32:0/891942758 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0974055330 0x7f09740577f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f0980008000 tx=0x7f0980008040).stop 2020-12-05T17:32:21.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.731+0000 7f098ffff700 1 -- 172.21.15.32:0/891942758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099009ed90 msgr2=0x7f099009f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:21.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.731+0000 7f098ffff700 1 --2- 172.21.15.32:0/891942758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099009ed90 0x7f099009f820 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f097800d000 tx=0x7f0978007b60).stop 2020-12-05T17:32:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.731+0000 7f098ffff700 1 -- 172.21.15.32:0/891942758 shutdown_connections 2020-12-05T17:32:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.731+0000 7f098ffff700 1 --2- 172.21.15.32:0/891942758 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0974055330 0x7f09740577f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.731+0000 7f098ffff700 1 --2- 172.21.15.32:0/891942758 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09900a2ca0 0x7f0990152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.731+0000 7f098ffff700 1 --2- 172.21.15.32:0/891942758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0970004e40 0x7f0990064310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.731+0000 7f098ffff700 1 --2- 172.21.15.32:0/891942758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099009ed90 0x7f099009f820 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.731+0000 7f098ffff700 1 -- 172.21.15.32:0/891942758 >> 172.21.15.32:0/891942758 conn(0x7f09900b5530 msgr2=0x7f0990061780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:21.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.731+0000 7f098ffff700 1 -- 172.21.15.32:0/891942758 shutdown_connections 2020-12-05T17:32:21.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.731+0000 7f098ffff700 1 -- 172.21.15.32:0/891942758 wait complete. 2020-12-05T17:32:21.737 INFO:teuthology.orchestra.run.smithi032.stderr:updated 2020-12-05T17:32:21.746 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-rbd-mirror 2020-12-05T17:32:21.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.855+0000 7f1b853d2700 1 Processor -- start 2020-12-05T17:32:21.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.855+0000 7f1b853d2700 1 -- start start 2020-12-05T17:32:21.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.855+0000 7f1b853d2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b800b21c0 0x7f1b800af880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:21.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.855+0000 7f1b853d2700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1b801433f0 0x7f1b80062160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:21.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.855+0000 7f1b853d2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b801438d0 con 0x7f1b800b1190 2020-12-05T17:32:21.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.855+0000 7f1b853d2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b800b25a0 con 0x7f1b800b21c0 2020-12-05T17:32:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.855+0000 7f1b853d2700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b80061a20 con 0x7f1b801433f0 2020-12-05T17:32:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b7effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b800b21c0 0x7f1b800af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b7f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1b800b1190 0x7f1b800b0100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36402/0 (socket says 172.21.15.32:36402) 2020-12-05T17:32:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b7f7fe700 1 -- 172.21.15.32:0/2333598949 learned_addr learned my addr 172.21.15.32:0/2333598949 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b7e7fc700 1 --2- 172.21.15.32:0/2333598949 >> v2:172.21.15.61:3300/0 conn(0x7f1b801433f0 0x7f1b80062160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b7dffb700 1 -- 172.21.15.32:0/2333598949 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729490328 0 0) 0x7f1b801438d0 con 0x7f1b800b1190 2020-12-05T17:32:21.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b7dffb700 1 -- 172.21.15.32:0/2333598949 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b60003230 con 0x7f1b800b1190 2020-12-05T17:32:21.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b7effd700 1 -- 172.21.15.32:0/2333598949 >> v2:172.21.15.61:3300/0 conn(0x7f1b801433f0 msgr2=0x7f1b80062160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:21.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b7effd700 1 --2- 172.21.15.32:0/2333598949 >> v2:172.21.15.61:3300/0 conn(0x7f1b801433f0 0x7f1b80062160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b7effd700 1 -- 172.21.15.32:0/2333598949 >> v1:172.21.15.32:6789/0 conn(0x7f1b800b1190 legacy=0x7f1b800b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:21.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b7effd700 1 -- 172.21.15.32:0/2333598949 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b80061cc0 con 0x7f1b800b21c0 2020-12-05T17:32:21.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b7effd700 1 --2- 172.21.15.32:0/2333598949 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b800b21c0 0x7f1b800af880 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f1b80061940 tx=0x7f1b78006dc0).ready entity=mon.0 client_cookie=269bf71bf8896cff server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:21.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b7dffb700 1 -- 172.21.15.32:0/2333598949 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b78009690 con 0x7f1b800b21c0 2020-12-05T17:32:21.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b7dffb700 1 -- 172.21.15.32:0/2333598949 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b780097f0 con 0x7f1b800b21c0 2020-12-05T17:32:21.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b7dffb700 1 -- 172.21.15.32:0/2333598949 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b78009b50 con 0x7f1b800b21c0 2020-12-05T17:32:21.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b853d2700 1 -- 172.21.15.32:0/2333598949 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b800b21c0 msgr2=0x7f1b800af880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:21.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b853d2700 1 --2- 172.21.15.32:0/2333598949 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b800b21c0 0x7f1b800af880 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f1b80061940 tx=0x7f1b78006dc0).stop 2020-12-05T17:32:21.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b853d2700 1 -- 172.21.15.32:0/2333598949 shutdown_connections 2020-12-05T17:32:21.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b853d2700 1 --2- 172.21.15.32:0/2333598949 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b800b21c0 0x7f1b800af880 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b853d2700 1 --2- 172.21.15.32:0/2333598949 >> v2:172.21.15.61:3300/0 conn(0x7f1b801433f0 0x7f1b80062160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b853d2700 1 -- 172.21.15.32:0/2333598949 >> 172.21.15.32:0/2333598949 conn(0x7f1b800b57a0 msgr2=0x7f1b800b1770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:21.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b853d2700 1 -- 172.21.15.32:0/2333598949 shutdown_connections 2020-12-05T17:32:21.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b853d2700 1 -- 172.21.15.32:0/2333598949 wait complete. 2020-12-05T17:32:21.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b853d2700 1 Processor -- start 2020-12-05T17:32:21.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.859+0000 7f1b853d2700 1 -- start start 2020-12-05T17:32:21.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.863+0000 7f1b853d2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b800b1190 0x7f1b80154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:21.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.863+0000 7f1b853d2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b800b21c0 0x7f1b8013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:21.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.863+0000 7f1b853d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b801433f0 0x7f1b8013cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:21.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.863+0000 7f1b853d2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b80146770 con 0x7f1b801433f0 2020-12-05T17:32:21.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.863+0000 7f1b853d2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b801469a0 con 0x7f1b800b21c0 2020-12-05T17:32:21.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.863+0000 7f1b853d2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b80146c40 con 0x7f1b800b1190 2020-12-05T17:32:21.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.863+0000 7f1b7f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b801433f0 0x7f1b8013cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:21.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.863+0000 7f1b7f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b801433f0 0x7f1b8013cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33762/0 (socket says 172.21.15.32:33762) 2020-12-05T17:32:21.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.863+0000 7f1b7f7fe700 1 -- 172.21.15.32:0/332132840 learned_addr learned my addr 172.21.15.32:0/332132840 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:21.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.863+0000 7f1b7e7fc700 1 --2- 172.21.15.32:0/332132840 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b800b21c0 0x7f1b8013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:21.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.863+0000 7f1b7effd700 1 --2- 172.21.15.32:0/332132840 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b800b1190 0x7f1b80154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:21.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.863+0000 7f1b7f7fe700 1 -- 172.21.15.32:0/332132840 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b800b1190 msgr2=0x7f1b80154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:21.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.863+0000 7f1b7f7fe700 1 --2- 172.21.15.32:0/332132840 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b800b1190 0x7f1b80154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.863+0000 7f1b7f7fe700 1 -- 172.21.15.32:0/332132840 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b800b21c0 msgr2=0x7f1b8013fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:21.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.863+0000 7f1b7f7fe700 1 --2- 172.21.15.32:0/332132840 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b800b21c0 0x7f1b8013fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:21.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.863+0000 7f1b7f7fe700 1 -- 172.21.15.32:0/332132840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b80145870 con 0x7f1b801433f0 2020-12-05T17:32:21.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.863+0000 7f1b7effd700 1 --2- 172.21.15.32:0/332132840 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b800b1190 0x7f1b80154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:21.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.863+0000 7f1b7f7fe700 1 --2- 172.21.15.32:0/332132840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b801433f0 0x7f1b8013cce0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f1b680010d0 tx=0x7f1b68007a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:21.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.863+0000 7f1b77fff700 1 -- 172.21.15.32:0/332132840 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b680072e0 con 0x7f1b801433f0 2020-12-05T17:32:21.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.863+0000 7f1b77fff700 1 -- 172.21.15.32:0/332132840 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b68007440 con 0x7f1b801433f0 2020-12-05T17:32:21.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.863+0000 7f1b853d2700 1 -- 172.21.15.32:0/332132840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b80063040 con 0x7f1b801433f0 2020-12-05T17:32:21.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.863+0000 7f1b77fff700 1 -- 172.21.15.32:0/332132840 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b680057d0 con 0x7f1b801433f0 2020-12-05T17:32:21.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.863+0000 7f1b853d2700 1 -- 172.21.15.32:0/332132840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b80066d80 con 0x7f1b801433f0 2020-12-05T17:32:21.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.867+0000 7f1b77fff700 1 -- 172.21.15.32:0/332132840 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1b6801a060 con 0x7f1b801433f0 2020-12-05T17:32:21.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.867+0000 7f1b853d2700 1 -- 172.21.15.32:0/332132840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b80063320 con 0x7f1b801433f0 2020-12-05T17:32:21.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.867+0000 7f1b77fff700 1 --2- 172.21.15.32:0/332132840 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1b64055330 0x7f1b640577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:21.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.867+0000 7f1b77fff700 1 -- 172.21.15.32:0/332132840 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f1b6806d520 con 0x7f1b801433f0 2020-12-05T17:32:21.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.867+0000 7f1b7effd700 1 --2- 172.21.15.32:0/332132840 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1b64055330 0x7f1b640577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:21.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.867+0000 7f1b7effd700 1 --2- 172.21.15.32:0/332132840 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1b64055330 0x7f1b640577f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f1b78019000 tx=0x7f1b78015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:21.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:21.875+0000 7f1b77fff700 1 -- 172.21.15.32:0/332132840 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1b68043a00 con 0x7f1b801433f0 2020-12-05T17:32:22.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.063+0000 7f1b853d2700 1 -- 172.21.15.32:0/332132840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f1b80146e00 con 0x7f1b801433f0 2020-12-05T17:32:22.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.071+0000 7f1b77fff700 1 -- 172.21.15.32:0/332132840 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 updated v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1b68018020 con 0x7f1b801433f0 2020-12-05T17:32:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.071+0000 7f1b853d2700 1 -- 172.21.15.32:0/332132840 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1b64055330 msgr2=0x7f1b640577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.071+0000 7f1b853d2700 1 --2- 172.21.15.32:0/332132840 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1b64055330 0x7f1b640577f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f1b78019000 tx=0x7f1b78015000).stop 2020-12-05T17:32:22.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.071+0000 7f1b853d2700 1 -- 172.21.15.32:0/332132840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b801433f0 msgr2=0x7f1b8013cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:22.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.071+0000 7f1b853d2700 1 --2- 172.21.15.32:0/332132840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b801433f0 0x7f1b8013cce0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f1b680010d0 tx=0x7f1b68007a90).stop 2020-12-05T17:32:22.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.071+0000 7f1b853d2700 1 -- 172.21.15.32:0/332132840 shutdown_connections 2020-12-05T17:32:22.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.071+0000 7f1b853d2700 1 --2- 172.21.15.32:0/332132840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b801433f0 0x7f1b8013cce0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.071+0000 7f1b853d2700 1 --2- 172.21.15.32:0/332132840 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1b64055330 0x7f1b640577f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.071+0000 7f1b853d2700 1 --2- 172.21.15.32:0/332132840 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b800b1190 0x7f1b80154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.071+0000 7f1b853d2700 1 --2- 172.21.15.32:0/332132840 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b800b21c0 0x7f1b8013fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.071+0000 7f1b853d2700 1 -- 172.21.15.32:0/332132840 >> 172.21.15.32:0/332132840 conn(0x7f1b800b57a0 msgr2=0x7f1b80141200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:22.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.071+0000 7f1b853d2700 1 -- 172.21.15.32:0/332132840 shutdown_connections 2020-12-05T17:32:22.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.071+0000 7f1b853d2700 1 -- 172.21.15.32:0/332132840 wait complete. 2020-12-05T17:32:22.079 INFO:teuthology.orchestra.run.smithi032.stderr:updated 2020-12-05T17:32:22.089 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-rgw 2020-12-05T17:32:22.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b10a4e700 1 Processor -- start 2020-12-05T17:32:22.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b10a4e700 1 -- start start 2020-12-05T17:32:22.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b10a4e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f8b0c05f6f0 0x7f8b0c1428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:22.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b10a4e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b0c0b1980 0x7f8b0c146580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:22.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b10a4e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8b0c146a70 0x7f8b0c066070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:22.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b10a4e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b0c05fad0 con 0x7f8b0c146a70 2020-12-05T17:32:22.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b10a4e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b0c0b2f80 con 0x7f8b0c05f6f0 2020-12-05T17:32:22.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b10a4e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c145e40 con 0x7f8b0c0b1980 2020-12-05T17:32:22.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b0bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8b0c146a70 0x7f8b0c066070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:22.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b0bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8b0c146a70 0x7f8b0c066070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33770/0 (socket says 172.21.15.32:33770) 2020-12-05T17:32:22.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b0bfff700 1 -- 172.21.15.32:0/2228284258 learned_addr learned my addr 172.21.15.32:0/2228284258 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:22.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b0b7fe700 1 --2- 172.21.15.32:0/2228284258 >> v2:172.21.15.61:3300/0 conn(0x7f8b0c05f6f0 0x7f8b0c1428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:22.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b0affd700 1 --2- 172.21.15.32:0/2228284258 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b0c0b1980 0x7f8b0c146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:22.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b0bfff700 1 -- 172.21.15.32:0/2228284258 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b0c0b1980 msgr2=0x7f8b0c146580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:22.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b0bfff700 1 --2- 172.21.15.32:0/2228284258 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b0c0b1980 0x7f8b0c146580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b0bfff700 1 -- 172.21.15.32:0/2228284258 >> v2:172.21.15.61:3300/0 conn(0x7f8b0c05f6f0 msgr2=0x7f8b0c1428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:22.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b0bfff700 1 --2- 172.21.15.32:0/2228284258 >> v2:172.21.15.61:3300/0 conn(0x7f8b0c05f6f0 0x7f8b0c1428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b0bfff700 1 -- 172.21.15.32:0/2228284258 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0c1460e0 con 0x7f8b0c146a70 2020-12-05T17:32:22.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b0bfff700 1 --2- 172.21.15.32:0/2228284258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b0c146a70 0x7f8b0c066070 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f8b040032b0 tx=0x7f8b04009270).ready entity=mon.0 client_cookie=6757323e7dec33d0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:22.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b0a7fc700 1 -- 172.21.15.32:0/2228284258 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b04009e60 con 0x7f8b0c146a70 2020-12-05T17:32:22.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b0a7fc700 1 -- 172.21.15.32:0/2228284258 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0400bb50 con 0x7f8b0c146a70 2020-12-05T17:32:22.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b0a7fc700 1 -- 172.21.15.32:0/2228284258 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b0400bcc0 con 0x7f8b0c146a70 2020-12-05T17:32:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b10a4e700 1 -- 172.21.15.32:0/2228284258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b0c146a70 msgr2=0x7f8b0c066070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.207+0000 7f8b10a4e700 1 --2- 172.21.15.32:0/2228284258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b0c146a70 0x7f8b0c066070 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f8b040032b0 tx=0x7f8b04009270).stop 2020-12-05T17:32:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b10a4e700 1 -- 172.21.15.32:0/2228284258 shutdown_connections 2020-12-05T17:32:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b10a4e700 1 --2- 172.21.15.32:0/2228284258 >> v2:172.21.15.61:3300/0 conn(0x7f8b0c05f6f0 0x7f8b0c1428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b10a4e700 1 --2- 172.21.15.32:0/2228284258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b0c146a70 0x7f8b0c066070 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b10a4e700 1 --2- 172.21.15.32:0/2228284258 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b0c0b1980 0x7f8b0c146580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b10a4e700 1 -- 172.21.15.32:0/2228284258 >> 172.21.15.32:0/2228284258 conn(0x7f8b0c0604c0 msgr2=0x7f8b0c05c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:22.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b10a4e700 1 -- 172.21.15.32:0/2228284258 shutdown_connections 2020-12-05T17:32:22.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b10a4e700 1 -- 172.21.15.32:0/2228284258 wait complete. 2020-12-05T17:32:22.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b10a4e700 1 Processor -- start 2020-12-05T17:32:22.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b10a4e700 1 -- start start 2020-12-05T17:32:22.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b10a4e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b0c05f6f0 0x7f8b0c151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:22.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b10a4e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b0c0b1980 0x7f8b0c063fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:22.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b10a4e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b0c146a70 0x7f8b0c1549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:22.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b10a4e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c065ca0 con 0x7f8b0c0b1980 2020-12-05T17:32:22.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b10a4e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c065f40 con 0x7f8b0c05f6f0 2020-12-05T17:32:22.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b10a4e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c0b01b0 con 0x7f8b0c146a70 2020-12-05T17:32:22.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b0affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b0c0b1980 0x7f8b0c063fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b0affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b0c0b1980 0x7f8b0c063fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33776/0 (socket says 172.21.15.32:33776) 2020-12-05T17:32:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b0affd700 1 -- 172.21.15.32:0/851588941 learned_addr learned my addr 172.21.15.32:0/851588941 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b0b7fe700 1 --2- 172.21.15.32:0/851588941 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b0c05f6f0 0x7f8b0c151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b0bfff700 1 --2- 172.21.15.32:0/851588941 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b0c146a70 0x7f8b0c1549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b0affd700 1 -- 172.21.15.32:0/851588941 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b0c146a70 msgr2=0x7f8b0c1549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:22.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b0affd700 1 --2- 172.21.15.32:0/851588941 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b0c146a70 0x7f8b0c1549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b0affd700 1 -- 172.21.15.32:0/851588941 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b0c05f6f0 msgr2=0x7f8b0c151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:22.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b0affd700 1 --2- 172.21.15.32:0/851588941 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b0c05f6f0 0x7f8b0c151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.211+0000 7f8b0affd700 1 -- 172.21.15.32:0/851588941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0c0b3a60 con 0x7f8b0c0b1980 2020-12-05T17:32:22.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.215+0000 7f8b0affd700 1 --2- 172.21.15.32:0/851588941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b0c0b1980 0x7f8b0c063fc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f8b0000c000 tx=0x7f8b00006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:22.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.215+0000 7f8b08ff9700 1 -- 172.21.15.32:0/851588941 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b0000ca10 con 0x7f8b0c0b1980 2020-12-05T17:32:22.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.215+0000 7f8b08ff9700 1 -- 172.21.15.32:0/851588941 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0000cb70 con 0x7f8b0c0b1980 2020-12-05T17:32:22.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.215+0000 7f8b08ff9700 1 -- 172.21.15.32:0/851588941 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b00013450 con 0x7f8b0c0b1980 2020-12-05T17:32:22.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.215+0000 7f8b10a4e700 1 -- 172.21.15.32:0/851588941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0c1523a0 con 0x7f8b0c0b1980 2020-12-05T17:32:22.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.215+0000 7f8b10a4e700 1 -- 172.21.15.32:0/851588941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0c063680 con 0x7f8b0c0b1980 2020-12-05T17:32:22.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.215+0000 7f8b10a4e700 1 -- 172.21.15.32:0/851588941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b0c065af0 con 0x7f8b0c0b1980 2020-12-05T17:32:22.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.215+0000 7f8b08ff9700 1 -- 172.21.15.32:0/851588941 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8b0001a030 con 0x7f8b0c0b1980 2020-12-05T17:32:22.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.215+0000 7f8b08ff9700 1 --2- 172.21.15.32:0/851588941 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8aec055500 0x7f8aec0579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:22.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.215+0000 7f8b08ff9700 1 -- 172.21.15.32:0/851588941 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f8b0006da00 con 0x7f8b0c0b1980 2020-12-05T17:32:22.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.215+0000 7f8b0b7fe700 1 --2- 172.21.15.32:0/851588941 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8aec055500 0x7f8aec0579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:22.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.215+0000 7f8b0b7fe700 1 --2- 172.21.15.32:0/851588941 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8aec055500 0x7f8aec0579c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f8afc00b000 tx=0x7f8afc006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:22.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.223+0000 7f8b08ff9700 1 -- 172.21.15.32:0/851588941 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8b00042df0 con 0x7f8b0c0b1980 2020-12-05T17:32:22.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.419+0000 7f8b10a4e700 1 -- 172.21.15.32:0/851588941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f8b0c057050 con 0x7f8b0c0b1980 2020-12-05T17:32:22.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.427+0000 7f8b08ff9700 1 -- 172.21.15.32:0/851588941 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 updated v10) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8b00043f90 con 0x7f8b0c0b1980 2020-12-05T17:32:22.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.427+0000 7f8b10a4e700 1 -- 172.21.15.32:0/851588941 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8aec055500 msgr2=0x7f8aec0579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:22.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.427+0000 7f8b10a4e700 1 --2- 172.21.15.32:0/851588941 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8aec055500 0x7f8aec0579c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f8afc00b000 tx=0x7f8afc006dc0).stop 2020-12-05T17:32:22.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.427+0000 7f8b10a4e700 1 -- 172.21.15.32:0/851588941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b0c0b1980 msgr2=0x7f8b0c063fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:22.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.427+0000 7f8b10a4e700 1 --2- 172.21.15.32:0/851588941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b0c0b1980 0x7f8b0c063fc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f8b0000c000 tx=0x7f8b00006dc0).stop 2020-12-05T17:32:22.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.427+0000 7f8b10a4e700 1 -- 172.21.15.32:0/851588941 shutdown_connections 2020-12-05T17:32:22.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.427+0000 7f8b10a4e700 1 --2- 172.21.15.32:0/851588941 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8aec055500 0x7f8aec0579c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.427+0000 7f8b10a4e700 1 --2- 172.21.15.32:0/851588941 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b0c146a70 0x7f8b0c1549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.427+0000 7f8b10a4e700 1 --2- 172.21.15.32:0/851588941 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b0c05f6f0 0x7f8b0c151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.427+0000 7f8b10a4e700 1 --2- 172.21.15.32:0/851588941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b0c0b1980 0x7f8b0c063fc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.427+0000 7f8b10a4e700 1 -- 172.21.15.32:0/851588941 >> 172.21.15.32:0/851588941 conn(0x7f8b0c0604c0 msgr2=0x7f8b0c143d40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:22.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.427+0000 7f8b10a4e700 1 -- 172.21.15.32:0/851588941 shutdown_connections 2020-12-05T17:32:22.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.427+0000 7f8b10a4e700 1 -- 172.21.15.32:0/851588941 wait complete. 2020-12-05T17:32:22.435 INFO:teuthology.orchestra.run.smithi032.stderr:updated 2020-12-05T17:32:22.445 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:32:22.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.555+0000 7efe14fd8700 1 Processor -- start 2020-12-05T17:32:22.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.555+0000 7efe14fd8700 1 -- start start 2020-12-05T17:32:22.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.555+0000 7efe14fd8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efe100afe50 0x7efe10146a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:22.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.555+0000 7efe14fd8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efe100b3aa0 0x7efe101429e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:22.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.555+0000 7efe14fd8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe100b0230 con 0x7efe100b1880 2020-12-05T17:32:22.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.555+0000 7efe14fd8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe100b6a50 con 0x7efe100afe50 2020-12-05T17:32:22.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.555+0000 7efe14fd8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe10146080 con 0x7efe100b3aa0 2020-12-05T17:32:22.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.559+0000 7efe0e59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efe100afe50 0x7efe10146a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:22.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.559+0000 7efe0ed9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efe100b1880 0x7efe10142ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36432/0 (socket says 172.21.15.32:36432) 2020-12-05T17:32:22.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.559+0000 7efe0ed9d700 1 -- 172.21.15.32:0/3924420223 learned_addr learned my addr 172.21.15.32:0/3924420223 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:22.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.559+0000 7efe05d9b700 1 --2- 172.21.15.32:0/3924420223 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efe100b3aa0 0x7efe101429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:22.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.559+0000 7efe0dd9b700 1 -- 172.21.15.32:0/3924420223 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319950601 0 0) 0x7efe100b0230 con 0x7efe100b1880 2020-12-05T17:32:22.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.559+0000 7efe0dd9b700 1 -- 172.21.15.32:0/3924420223 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdf0003230 con 0x7efe100b1880 2020-12-05T17:32:22.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.559+0000 7efe0e59c700 1 -- 172.21.15.32:0/3924420223 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efe100b3aa0 msgr2=0x7efe101429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:22.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.559+0000 7efe0e59c700 1 --2- 172.21.15.32:0/3924420223 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efe100b3aa0 0x7efe101429e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.559+0000 7efe0e59c700 1 -- 172.21.15.32:0/3924420223 >> v1:172.21.15.32:6789/0 conn(0x7efe100b1880 legacy=0x7efe10142ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:22.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.559+0000 7efe0e59c700 1 -- 172.21.15.32:0/3924420223 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe10146320 con 0x7efe100afe50 2020-12-05T17:32:22.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.559+0000 7efe0e59c700 1 --2- 172.21.15.32:0/3924420223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe100afe50 0x7efe10146a60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7efe00002f30 tx=0x7efe00006dc0).ready entity=mon.0 client_cookie=eb3f847e812ccb82 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:22.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.559+0000 7efe0dd9b700 1 -- 172.21.15.32:0/3924420223 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe000096a0 con 0x7efe100afe50 2020-12-05T17:32:22.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.559+0000 7efe0dd9b700 1 -- 172.21.15.32:0/3924420223 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe00009800 con 0x7efe100afe50 2020-12-05T17:32:22.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.559+0000 7efe0dd9b700 1 -- 172.21.15.32:0/3924420223 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe00009b60 con 0x7efe100afe50 2020-12-05T17:32:22.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.559+0000 7efe14fd8700 1 -- 172.21.15.32:0/3924420223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe100afe50 msgr2=0x7efe10146a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:22.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.559+0000 7efe14fd8700 1 --2- 172.21.15.32:0/3924420223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe100afe50 0x7efe10146a60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7efe00002f30 tx=0x7efe00006dc0).stop 2020-12-05T17:32:22.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.559+0000 7efe14fd8700 1 -- 172.21.15.32:0/3924420223 shutdown_connections 2020-12-05T17:32:22.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.559+0000 7efe14fd8700 1 --2- 172.21.15.32:0/3924420223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe100afe50 0x7efe10146a60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.559+0000 7efe14fd8700 1 --2- 172.21.15.32:0/3924420223 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efe100b3aa0 0x7efe101429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.559+0000 7efe14fd8700 1 -- 172.21.15.32:0/3924420223 >> 172.21.15.32:0/3924420223 conn(0x7efe100b9100 msgr2=0x7efe100b9510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:22.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.559+0000 7efe14fd8700 1 -- 172.21.15.32:0/3924420223 shutdown_connections 2020-12-05T17:32:22.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.559+0000 7efe14fd8700 1 -- 172.21.15.32:0/3924420223 wait complete. 2020-12-05T17:32:22.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe14fd8700 1 Processor -- start 2020-12-05T17:32:22.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe14fd8700 1 -- start start 2020-12-05T17:32:22.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe14fd8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efe100afe50 0x7efe100b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:22.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe14fd8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efe100b1880 0x7efe10066730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe14fd8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe100b3aa0 0x7efe10154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe14fd8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe10142870 con 0x7efe100b3aa0 2020-12-05T17:32:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe14fd8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe10065fa0 con 0x7efe100afe50 2020-12-05T17:32:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe14fd8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe10065d00 con 0x7efe100b1880 2020-12-05T17:32:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe0ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe100b3aa0 0x7efe10154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:22.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe0ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe100b3aa0 0x7efe10154d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33790/0 (socket says 172.21.15.32:33790) 2020-12-05T17:32:22.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe0ed9d700 1 -- 172.21.15.32:0/2985047748 learned_addr learned my addr 172.21.15.32:0/2985047748 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:22.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe0e59c700 1 --2- 172.21.15.32:0/2985047748 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efe100afe50 0x7efe100b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:22.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe05d9b700 1 --2- 172.21.15.32:0/2985047748 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efe100b1880 0x7efe10066730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:22.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe0ed9d700 1 -- 172.21.15.32:0/2985047748 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efe100b1880 msgr2=0x7efe10066730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:22.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe0ed9d700 1 --2- 172.21.15.32:0/2985047748 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efe100b1880 0x7efe10066730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe0ed9d700 1 -- 172.21.15.32:0/2985047748 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efe100afe50 msgr2=0x7efe100b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:22.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe0ed9d700 1 --2- 172.21.15.32:0/2985047748 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efe100afe50 0x7efe100b5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe0ed9d700 1 -- 172.21.15.32:0/2985047748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe10066290 con 0x7efe100b3aa0 2020-12-05T17:32:22.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe0ed9d700 1 --2- 172.21.15.32:0/2985047748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe100b3aa0 0x7efe10154d30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7efe08001450 tx=0x7efe08007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:22.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe07fff700 1 -- 172.21.15.32:0/2985047748 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe08005b70 con 0x7efe100b3aa0 2020-12-05T17:32:22.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe14fd8700 1 -- 172.21.15.32:0/2985047748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe10066c20 con 0x7efe100b3aa0 2020-12-05T17:32:22.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe07fff700 1 -- 172.21.15.32:0/2985047748 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe08005cd0 con 0x7efe100b3aa0 2020-12-05T17:32:22.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe07fff700 1 -- 172.21.15.32:0/2985047748 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe08007300 con 0x7efe100b3aa0 2020-12-05T17:32:22.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.563+0000 7efe14fd8700 1 -- 172.21.15.32:0/2985047748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe10155440 con 0x7efe100b3aa0 2020-12-05T17:32:22.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.567+0000 7efe14fd8700 1 -- 172.21.15.32:0/2985047748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe10065b50 con 0x7efe100b3aa0 2020-12-05T17:32:22.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.567+0000 7efe07fff700 1 -- 172.21.15.32:0/2985047748 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7efe08010030 con 0x7efe100b3aa0 2020-12-05T17:32:22.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.567+0000 7efe07fff700 1 --2- 172.21.15.32:0/2985047748 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efdf40554e0 0x7efdf40579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:22.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.567+0000 7efe07fff700 1 -- 172.21.15.32:0/2985047748 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5600+0+0 (secure 0 0 0) 0x7efe0806de70 con 0x7efe100b3aa0 2020-12-05T17:32:22.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.567+0000 7efe0e59c700 1 --2- 172.21.15.32:0/2985047748 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efdf40554e0 0x7efdf40579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:22.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.567+0000 7efe0e59c700 1 --2- 172.21.15.32:0/2985047748 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efdf40554e0 0x7efdf40579a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7efe0001b000 tx=0x7efe00015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:22.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.575+0000 7efe07fff700 1 -- 172.21.15.32:0/2985047748 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7efe080442b0 con 0x7efe100b3aa0 2020-12-05T17:32:22.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.715+0000 7efe14fd8700 1 -- 172.21.15.32:0/2985047748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe10057050 con 0x7efe100b3aa0 2020-12-05T17:32:22.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.715+0000 7efe07fff700 1 -- 172.21.15.32:0/2985047748 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+13414 (secure 0 0 0) 0x7efe080478d0 con 0x7efe100b3aa0 2020-12-05T17:32:22.721 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:32:22.721 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":29,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:32:19.957588+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:32:22.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.719+0000 7efe14fd8700 1 -- 172.21.15.32:0/2985047748 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efdf40554e0 msgr2=0x7efdf40579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:22.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.719+0000 7efe14fd8700 1 --2- 172.21.15.32:0/2985047748 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efdf40554e0 0x7efdf40579a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7efe0001b000 tx=0x7efe00015000).stop 2020-12-05T17:32:22.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.719+0000 7efe14fd8700 1 -- 172.21.15.32:0/2985047748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe100b3aa0 msgr2=0x7efe10154d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:22.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.719+0000 7efe14fd8700 1 --2- 172.21.15.32:0/2985047748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe100b3aa0 0x7efe10154d30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7efe08001450 tx=0x7efe08007e10).stop 2020-12-05T17:32:22.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.719+0000 7efe14fd8700 1 -- 172.21.15.32:0/2985047748 shutdown_connections 2020-12-05T17:32:22.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.719+0000 7efe14fd8700 1 --2- 172.21.15.32:0/2985047748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe100b3aa0 0x7efe10154d30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.719+0000 7efe14fd8700 1 --2- 172.21.15.32:0/2985047748 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efe100afe50 0x7efe100b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.719+0000 7efe14fd8700 1 --2- 172.21.15.32:0/2985047748 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efdf40554e0 0x7efdf40579a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.719+0000 7efe14fd8700 1 --2- 172.21.15.32:0/2985047748 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efe100b1880 0x7efe10066730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.719+0000 7efe14fd8700 1 -- 172.21.15.32:0/2985047748 >> 172.21.15.32:0/2985047748 conn(0x7efe100b9100 msgr2=0x7efe100b79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:22.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.719+0000 7efe14fd8700 1 -- 172.21.15.32:0/2985047748 shutdown_connections 2020-12-05T17:32:22.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.719+0000 7efe14fd8700 1 -- 172.21.15.32:0/2985047748 wait complete. 2020-12-05T17:32:22.734 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-12-05T17:32:22.735 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:32:22.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.851+0000 7f8d0df2d700 1 Processor -- start 2020-12-05T17:32:22.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.851+0000 7f8d0df2d700 1 -- start start 2020-12-05T17:32:22.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.851+0000 7f8d0df2d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8d080b3020 0x7f8d08066e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:22.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.851+0000 7f8d0df2d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f8d08144380 0x7f8d0813a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:22.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.851+0000 7f8d0df2d700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d080b3540 con 0x7f8d08061ba0 2020-12-05T17:32:22.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.851+0000 7f8d0df2d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d08061f80 con 0x7f8d080b3020 2020-12-05T17:32:22.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.851+0000 7f8d0df2d700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d08066750 con 0x7f8d08144380 2020-12-05T17:32:22.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.855+0000 7f8d07fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8d080b3020 0x7f8d08066e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:22.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.855+0000 7f8d07fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8d080b3020 0x7f8d08066e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33796/0 (socket says 172.21.15.32:33796) 2020-12-05T17:32:22.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.855+0000 7f8d07fff700 1 -- 172.21.15.32:0/2594285983 learned_addr learned my addr 172.21.15.32:0/2594285983 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:22.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.855+0000 7f8d0d72c700 1 --2- 172.21.15.32:0/2594285983 >> v2:172.21.15.61:3300/0 conn(0x7f8d08144380 0x7f8d0813a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:22.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.855+0000 7f8d0cf2b700 1 --1- 172.21.15.32:0/2594285983 >> v1:172.21.15.61:6789/0 conn(0x7f8d08061ba0 0x7f8d08143c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:49082/0 (socket says 172.21.15.32:49082) 2020-12-05T17:32:22.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.855+0000 7f8d07fff700 1 -- 172.21.15.32:0/2594285983 >> v2:172.21.15.61:3300/0 conn(0x7f8d08144380 msgr2=0x7f8d0813a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:22.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.855+0000 7f8d07fff700 1 --2- 172.21.15.32:0/2594285983 >> v2:172.21.15.61:3300/0 conn(0x7f8d08144380 0x7f8d0813a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.855+0000 7f8d07fff700 1 -- 172.21.15.32:0/2594285983 >> v1:172.21.15.61:6789/0 conn(0x7f8d08061ba0 legacy=0x7f8d08143c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:22.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.855+0000 7f8d07fff700 1 -- 172.21.15.32:0/2594285983 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d080669f0 con 0x7f8d080b3020 2020-12-05T17:32:22.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.855+0000 7f8d07fff700 1 --2- 172.21.15.32:0/2594285983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d080b3020 0x7f8d08066e90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f8cf4002f30 tx=0x7f8cf4006e80).ready entity=mon.0 client_cookie=2421ec16cd9e6a5d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:22.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.855+0000 7f8d077fe700 1 -- 172.21.15.32:0/2594285983 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cf40097f0 con 0x7f8d080b3020 2020-12-05T17:32:22.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.855+0000 7f8d077fe700 1 -- 172.21.15.32:0/2594285983 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf4009950 con 0x7f8d080b3020 2020-12-05T17:32:22.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.855+0000 7f8d077fe700 1 -- 172.21.15.32:0/2594285983 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cf4009cb0 con 0x7f8d080b3020 2020-12-05T17:32:22.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.855+0000 7f8d0df2d700 1 -- 172.21.15.32:0/2594285983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d080b3020 msgr2=0x7f8d08066e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:22.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.855+0000 7f8d0df2d700 1 --2- 172.21.15.32:0/2594285983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d080b3020 0x7f8d08066e90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f8cf4002f30 tx=0x7f8cf4006e80).stop 2020-12-05T17:32:22.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.855+0000 7f8d0df2d700 1 -- 172.21.15.32:0/2594285983 shutdown_connections 2020-12-05T17:32:22.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.855+0000 7f8d0df2d700 1 --2- 172.21.15.32:0/2594285983 >> v2:172.21.15.61:3300/0 conn(0x7f8d08144380 0x7f8d0813a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.855+0000 7f8d0df2d700 1 --2- 172.21.15.32:0/2594285983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d080b3020 0x7f8d08066e90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.855+0000 7f8d0df2d700 1 -- 172.21.15.32:0/2594285983 >> 172.21.15.32:0/2594285983 conn(0x7f8d080b6d10 msgr2=0x7f8d080b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.855+0000 7f8d0df2d700 1 -- 172.21.15.32:0/2594285983 shutdown_connections 2020-12-05T17:32:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.855+0000 7f8d0df2d700 1 -- 172.21.15.32:0/2594285983 wait complete. 2020-12-05T17:32:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.855+0000 7f8d0df2d700 1 Processor -- start 2020-12-05T17:32:22.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.855+0000 7f8d0df2d700 1 -- start start 2020-12-05T17:32:22.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.859+0000 7f8d0df2d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8d08061ba0 0x7f8d080b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:22.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.859+0000 7f8d0df2d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d080b3020 0x7f8d08110c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:22.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.859+0000 7f8d0df2d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8d08144380 0x7f8d08111150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:22.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.859+0000 7f8d0df2d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d08140200 con 0x7f8d080b3020 2020-12-05T17:32:22.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.859+0000 7f8d0df2d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d08111640 con 0x7f8d08061ba0 2020-12-05T17:32:22.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.859+0000 7f8d0df2d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d081117b0 con 0x7f8d08144380 2020-12-05T17:32:22.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.859+0000 7f8d07fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d080b3020 0x7f8d08110c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:22.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.859+0000 7f8d07fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d080b3020 0x7f8d08110c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33802/0 (socket says 172.21.15.32:33802) 2020-12-05T17:32:22.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.859+0000 7f8d07fff700 1 -- 172.21.15.32:0/1335138033 learned_addr learned my addr 172.21.15.32:0/1335138033 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:22.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.859+0000 7f8d0d72c700 1 --2- 172.21.15.32:0/1335138033 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8d08144380 0x7f8d08111150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:22.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.859+0000 7f8d0cf2b700 1 --2- 172.21.15.32:0/1335138033 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8d08061ba0 0x7f8d080b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:22.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.859+0000 7f8d07fff700 1 -- 172.21.15.32:0/1335138033 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8d08144380 msgr2=0x7f8d08111150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:22.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.859+0000 7f8d07fff700 1 --2- 172.21.15.32:0/1335138033 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8d08144380 0x7f8d08111150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.859+0000 7f8d07fff700 1 -- 172.21.15.32:0/1335138033 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8d08061ba0 msgr2=0x7f8d080b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:22.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.859+0000 7f8d07fff700 1 --2- 172.21.15.32:0/1335138033 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8d08061ba0 0x7f8d080b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:22.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.859+0000 7f8d07fff700 1 -- 172.21.15.32:0/1335138033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0813fbd0 con 0x7f8d080b3020 2020-12-05T17:32:22.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.859+0000 7f8d07fff700 1 --2- 172.21.15.32:0/1335138033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d080b3020 0x7f8d08110c60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f8cf4002f30 tx=0x7f8cf4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:22.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.859+0000 7f8d05ffb700 1 -- 172.21.15.32:0/1335138033 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cf4011b80 con 0x7f8d080b3020 2020-12-05T17:32:22.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.859+0000 7f8d05ffb700 1 -- 172.21.15.32:0/1335138033 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf4011ce0 con 0x7f8d080b3020 2020-12-05T17:32:22.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.859+0000 7f8d0df2d700 1 -- 172.21.15.32:0/1335138033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d0813e7b0 con 0x7f8d080b3020 2020-12-05T17:32:22.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.859+0000 7f8d05ffb700 1 -- 172.21.15.32:0/1335138033 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cf401e650 con 0x7f8d080b3020 2020-12-05T17:32:22.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.859+0000 7f8d0df2d700 1 -- 172.21.15.32:0/1335138033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d080b5f80 con 0x7f8d080b3020 2020-12-05T17:32:22.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.863+0000 7f8d0df2d700 1 -- 172.21.15.32:0/1335138033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d08140060 con 0x7f8d080b3020 2020-12-05T17:32:22.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.863+0000 7f8d05ffb700 1 -- 172.21.15.32:0/1335138033 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8cf4025030 con 0x7f8d080b3020 2020-12-05T17:32:22.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.863+0000 7f8d05ffb700 1 --2- 172.21.15.32:0/1335138033 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8ce4055440 0x7f8ce4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:22.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.863+0000 7f8d05ffb700 1 -- 172.21.15.32:0/1335138033 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f8cf406ef30 con 0x7f8d080b3020 2020-12-05T17:32:22.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.863+0000 7f8d0cf2b700 1 --2- 172.21.15.32:0/1335138033 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8ce4055440 0x7f8ce4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:22.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.863+0000 7f8d0cf2b700 1 --2- 172.21.15.32:0/1335138033 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8ce4055440 0x7f8ce4057900 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f8cf800d000 tx=0x7f8cf8007b50).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:22.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:22.871+0000 7f8d05ffb700 1 -- 172.21.15.32:0/1335138033 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8d08140060 con 0x7f8d080b3020 2020-12-05T17:32:23.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.015+0000 7f8d0df2d700 1 -- 172.21.15.32:0/1335138033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f8d08140060 con 0x7f8d080b3020 2020-12-05T17:32:23.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.015+0000 7f8d05ffb700 1 -- 172.21.15.32:0/1335138033 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v29) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8d08140060 con 0x7f8d080b3020 2020-12-05T17:32:23.020 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:32:23.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.019+0000 7f8d0df2d700 1 -- 172.21.15.32:0/1335138033 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8ce4055440 msgr2=0x7f8ce4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:23.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.019+0000 7f8d0df2d700 1 --2- 172.21.15.32:0/1335138033 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8ce4055440 0x7f8ce4057900 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f8cf800d000 tx=0x7f8cf8007b50).stop 2020-12-05T17:32:23.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.019+0000 7f8d0df2d700 1 -- 172.21.15.32:0/1335138033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d080b3020 msgr2=0x7f8d08110c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:23.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.019+0000 7f8d0df2d700 1 --2- 172.21.15.32:0/1335138033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d080b3020 0x7f8d08110c60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f8cf4002f30 tx=0x7f8cf4015000).stop 2020-12-05T17:32:23.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.019+0000 7f8d0df2d700 1 -- 172.21.15.32:0/1335138033 shutdown_connections 2020-12-05T17:32:23.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.019+0000 7f8d0df2d700 1 --2- 172.21.15.32:0/1335138033 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8ce4055440 0x7f8ce4057900 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.019+0000 7f8d0df2d700 1 --2- 172.21.15.32:0/1335138033 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8d08144380 0x7f8d08111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.019+0000 7f8d0df2d700 1 --2- 172.21.15.32:0/1335138033 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8d08061ba0 0x7f8d080b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.019+0000 7f8d0df2d700 1 --2- 172.21.15.32:0/1335138033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d080b3020 0x7f8d08110c60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.019+0000 7f8d0df2d700 1 -- 172.21.15.32:0/1335138033 >> 172.21.15.32:0/1335138033 conn(0x7f8d080b6d10 msgr2=0x7f8d0813a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:23.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.019+0000 7f8d0df2d700 1 -- 172.21.15.32:0/1335138033 shutdown_connections 2020-12-05T17:32:23.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.019+0000 7f8d0df2d700 1 -- 172.21.15.32:0/1335138033 wait complete. 2020-12-05T17:32:23.033 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:32:23.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe902c59700 1 Processor -- start 2020-12-05T17:32:23.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe902c59700 1 -- start start 2020-12-05T17:32:23.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe902c59700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fe8fc0aa730 0x7fe8fc0b7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:23.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe902c59700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8fc066e50 0x7fe8fc0b1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:23.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe902c59700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8fc0aab10 con 0x7fe8fc140f50 2020-12-05T17:32:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe902c59700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8fc0b7a00 con 0x7fe8fc0aa730 2020-12-05T17:32:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe902c59700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc140ab0 con 0x7fe8fc066e50 2020-12-05T17:32:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe9011f6700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe8fc140f50 0x7fe8fc13fb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36458/0 (socket says 172.21.15.32:36458) 2020-12-05T17:32:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe9011f6700 1 -- 172.21.15.32:0/3599313032 learned_addr learned my addr 172.21.15.32:0/3599313032 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:23.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe9009f5700 1 --2- 172.21.15.32:0/3599313032 >> v2:172.21.15.61:3300/0 conn(0x7fe8fc0aa730 0x7fe8fc0b7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:23.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fbfff700 1 --2- 172.21.15.32:0/3599313032 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8fc066e50 0x7fe8fc0b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:23.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fb7fe700 1 -- 172.21.15.32:0/3599313032 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1582621109 0 0) 0x7fe8fc0aab10 con 0x7fe8fc140f50 2020-12-05T17:32:23.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fb7fe700 1 -- 172.21.15.32:0/3599313032 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8dc003230 con 0x7fe8fc140f50 2020-12-05T17:32:23.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fb7fe700 1 -- 172.21.15.32:0/3599313032 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1312533059 0 0) 0x7fe8dc003230 con 0x7fe8fc140f50 2020-12-05T17:32:23.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fb7fe700 1 -- 172.21.15.32:0/3599313032 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8fc066e50 msgr2=0x7fe8fc0b1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:23.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fb7fe700 1 --2- 172.21.15.32:0/3599313032 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8fc066e50 0x7fe8fc0b1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fb7fe700 1 -- 172.21.15.32:0/3599313032 >> v2:172.21.15.61:3300/0 conn(0x7fe8fc0aa730 msgr2=0x7fe8fc0b7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:23.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fb7fe700 1 --2- 172.21.15.32:0/3599313032 >> v2:172.21.15.61:3300/0 conn(0x7fe8fc0aa730 0x7fe8fc0b7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fb7fe700 1 -- 172.21.15.32:0/3599313032 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8fc0b1d40 con 0x7fe8fc140f50 2020-12-05T17:32:23.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fb7fe700 1 -- 172.21.15.32:0/3599313032 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fe8ec001a50 con 0x7fe8fc140f50 2020-12-05T17:32:23.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fb7fe700 1 -- 172.21.15.32:0/3599313032 >> v1:172.21.15.32:6789/0 conn(0x7fe8fc140f50 legacy=0x7fe8fc13fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:23.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fb7fe700 1 --2- 172.21.15.32:0/3599313032 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8dc003720 0x7fe8dc003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:23.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fb7fe700 1 --2- 172.21.15.32:0/3599313032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8dc0044d0 0x7fe8dc004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fb7fe700 1 --2- 172.21.15.32:0/3599313032 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8fc066e50 0x7fe8dc004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fb7fe700 1 -- 172.21.15.32:0/3599313032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc0b7a00 con 0x7fe8dc0044d0 2020-12-05T17:32:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fb7fe700 1 -- 172.21.15.32:0/3599313032 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc140ab0 con 0x7fe8dc003720 2020-12-05T17:32:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fb7fe700 1 -- 172.21.15.32:0/3599313032 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc004260 con 0x7fe8fc066e50 2020-12-05T17:32:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fbfff700 1 --2- 172.21.15.32:0/3599313032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8dc0044d0 0x7fe8dc004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:23.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe9009f5700 1 --2- 172.21.15.32:0/3599313032 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8dc003720 0x7fe8dc003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:23.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe9011f6700 1 --2- 172.21.15.32:0/3599313032 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8fc066e50 0x7fe8dc004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:23.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fbfff700 1 -- 172.21.15.32:0/3599313032 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8fc066e50 msgr2=0x7fe8dc004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:23.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fbfff700 1 --2- 172.21.15.32:0/3599313032 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8fc066e50 0x7fe8dc004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fbfff700 1 -- 172.21.15.32:0/3599313032 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8dc003720 msgr2=0x7fe8dc003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fbfff700 1 --2- 172.21.15.32:0/3599313032 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8dc003720 0x7fe8dc003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fbfff700 1 -- 172.21.15.32:0/3599313032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc003230 con 0x7fe8dc0044d0 2020-12-05T17:32:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.147+0000 7fe8fbfff700 1 --2- 172.21.15.32:0/3599313032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8dc0044d0 0x7fe8dc004950 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7fe8e400c000 tx=0x7fe8e4008040).ready entity=mon.0 client_cookie=b5fca617e40912cd server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe8fb7fe700 1 -- 172.21.15.32:0/3599313032 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8e400e970 con 0x7fe8dc0044d0 2020-12-05T17:32:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe8fb7fe700 1 -- 172.21.15.32:0/3599313032 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e400ead0 con 0x7fe8dc0044d0 2020-12-05T17:32:23.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe8fb7fe700 1 -- 172.21.15.32:0/3599313032 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8e400ee30 con 0x7fe8dc0044d0 2020-12-05T17:32:23.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe902c59700 1 -- 172.21.15.32:0/3599313032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8dc0044d0 msgr2=0x7fe8dc004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:23.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe902c59700 1 --2- 172.21.15.32:0/3599313032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8dc0044d0 0x7fe8dc004950 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7fe8e400c000 tx=0x7fe8e4008040).stop 2020-12-05T17:32:23.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe9011f6700 1 -- 172.21.15.32:0/3599313032 reap_dead start 2020-12-05T17:32:23.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe902c59700 1 -- 172.21.15.32:0/3599313032 shutdown_connections 2020-12-05T17:32:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe902c59700 1 -- 172.21.15.32:0/3599313032 >> 172.21.15.32:0/3599313032 conn(0x7fe8fc05c960 msgr2=0x7fe8fc05cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe902c59700 1 -- 172.21.15.32:0/3599313032 shutdown_connections 2020-12-05T17:32:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe902c59700 1 -- 172.21.15.32:0/3599313032 wait complete. 2020-12-05T17:32:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe902c59700 1 Processor -- start 2020-12-05T17:32:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe902c59700 1 -- start start 2020-12-05T17:32:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe902c59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8fc146430 0x7fe8fc0b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe902c59700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8fc064c30 0x7fe8fc0b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe902c59700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8fc154e70 0x7fe8fc155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe902c59700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc152700 con 0x7fe8fc146430 2020-12-05T17:32:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe902c59700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc146810 con 0x7fe8fc064c30 2020-12-05T17:32:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe902c59700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc0aca90 con 0x7fe8fc154e70 2020-12-05T17:32:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe9009f5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8fc146430 0x7fe8fc0b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe9009f5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8fc146430 0x7fe8fc0b0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33820/0 (socket says 172.21.15.32:33820) 2020-12-05T17:32:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe9009f5700 1 -- 172.21.15.32:0/1485006434 learned_addr learned my addr 172.21.15.32:0/1485006434 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe9011f6700 1 --2- 172.21.15.32:0/1485006434 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8fc154e70 0x7fe8fc155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe8fbfff700 1 --2- 172.21.15.32:0/1485006434 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8fc064c30 0x7fe8fc0b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe9009f5700 1 -- 172.21.15.32:0/1485006434 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8fc154e70 msgr2=0x7fe8fc155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe9009f5700 1 --2- 172.21.15.32:0/1485006434 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8fc154e70 0x7fe8fc155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe9009f5700 1 -- 172.21.15.32:0/1485006434 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8fc064c30 msgr2=0x7fe8fc0b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe9009f5700 1 --2- 172.21.15.32:0/1485006434 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8fc064c30 0x7fe8fc0b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.151+0000 7fe9009f5700 1 -- 172.21.15.32:0/1485006434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8fc0acd30 con 0x7fe8fc146430 2020-12-05T17:32:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.155+0000 7fe9009f5700 1 --2- 172.21.15.32:0/1485006434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8fc146430 0x7fe8fc0b0710 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7fe8fc1469a0 tx=0x7fe8f4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.155+0000 7fe8f9ffb700 1 -- 172.21.15.32:0/1485006434 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8f400fa40 con 0x7fe8fc146430 2020-12-05T17:32:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.155+0000 7fe8f9ffb700 1 -- 172.21.15.32:0/1485006434 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f400fba0 con 0x7fe8fc146430 2020-12-05T17:32:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.155+0000 7fe902c59700 1 -- 172.21.15.32:0/1485006434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8fc0b10f0 con 0x7fe8fc146430 2020-12-05T17:32:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.155+0000 7fe8f9ffb700 1 -- 172.21.15.32:0/1485006434 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8f400fd10 con 0x7fe8fc146430 2020-12-05T17:32:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.155+0000 7fe902c59700 1 -- 172.21.15.32:0/1485006434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8fc1517f0 con 0x7fe8fc146430 2020-12-05T17:32:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.155+0000 7fe8f9ffb700 1 -- 172.21.15.32:0/1485006434 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe8f4024070 con 0x7fe8fc146430 2020-12-05T17:32:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.155+0000 7fe902c59700 1 -- 172.21.15.32:0/1485006434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8fc057050 con 0x7fe8fc146430 2020-12-05T17:32:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.155+0000 7fe8f9ffb700 1 --2- 172.21.15.32:0/1485006434 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe8e00553b0 0x7fe8e0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.155+0000 7fe8f9ffb700 1 -- 172.21.15.32:0/1485006434 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fe8f406e6f0 con 0x7fe8fc146430 2020-12-05T17:32:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.155+0000 7fe8fbfff700 1 --2- 172.21.15.32:0/1485006434 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe8e00553b0 0x7fe8e0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.155+0000 7fe8fbfff700 1 --2- 172.21.15.32:0/1485006434 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe8e00553b0 0x7fe8e0057870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fe8e401b000 tx=0x7fe8e401a000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.163+0000 7fe8f9ffb700 1 -- 172.21.15.32:0/1485006434 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe8f4044bd0 con 0x7fe8fc146430 2020-12-05T17:32:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.303+0000 7fe902c59700 1 -- 172.21.15.32:0/1485006434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe8fc13db60 con 0x7fe8fc146430 2020-12-05T17:32:23.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.303+0000 7fe8f9ffb700 1 -- 172.21.15.32:0/1485006434 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe8f40481f0 con 0x7fe8fc146430 2020-12-05T17:32:23.307 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:32:23.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.307+0000 7fe902c59700 1 -- 172.21.15.32:0/1485006434 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe8e00553b0 msgr2=0x7fe8e0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:23.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.307+0000 7fe902c59700 1 --2- 172.21.15.32:0/1485006434 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe8e00553b0 0x7fe8e0057870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fe8e401b000 tx=0x7fe8e401a000).stop 2020-12-05T17:32:23.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.307+0000 7fe902c59700 1 -- 172.21.15.32:0/1485006434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8fc146430 msgr2=0x7fe8fc0b0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:23.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.307+0000 7fe902c59700 1 --2- 172.21.15.32:0/1485006434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8fc146430 0x7fe8fc0b0710 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7fe8fc1469a0 tx=0x7fe8f4008040).stop 2020-12-05T17:32:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.307+0000 7fe902c59700 1 -- 172.21.15.32:0/1485006434 shutdown_connections 2020-12-05T17:32:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.307+0000 7fe902c59700 1 --2- 172.21.15.32:0/1485006434 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe8e00553b0 0x7fe8e0057870 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.307+0000 7fe902c59700 1 --2- 172.21.15.32:0/1485006434 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8fc154e70 0x7fe8fc155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.307+0000 7fe902c59700 1 --2- 172.21.15.32:0/1485006434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8fc146430 0x7fe8fc0b0710 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.307+0000 7fe902c59700 1 --2- 172.21.15.32:0/1485006434 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8fc064c30 0x7fe8fc0b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.307+0000 7fe902c59700 1 -- 172.21.15.32:0/1485006434 >> 172.21.15.32:0/1485006434 conn(0x7fe8fc05c960 msgr2=0x7fe8fc061a30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:23.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.307+0000 7fe902c59700 1 -- 172.21.15.32:0/1485006434 shutdown_connections 2020-12-05T17:32:23.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.307+0000 7fe902c59700 1 -- 172.21.15.32:0/1485006434 wait complete. 2020-12-05T17:32:23.322 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-12-05T17:32:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.431+0000 7fb934bc2700 1 Processor -- start 2020-12-05T17:32:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.431+0000 7fb934bc2700 1 -- start start 2020-12-05T17:32:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.431+0000 7fb934bc2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb930062400 0x7fb93005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.431+0000 7fb934bc2700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb93005feb0 0x7fb9300b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.431+0000 7fb934bc2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb930142450 con 0x7fb93005f3b0 2020-12-05T17:32:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.431+0000 7fb934bc2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9300b42f0 con 0x7fb930062400 2020-12-05T17:32:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.431+0000 7fb934bc2700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb930061780 con 0x7fb93005feb0 2020-12-05T17:32:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.431+0000 7fb927fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb930062400 0x7fb93005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.431+0000 7fb927fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb930062400 0x7fb93005f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33828/0 (socket says 172.21.15.32:33828) 2020-12-05T17:32:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.431+0000 7fb927fff700 1 -- 172.21.15.32:0/901932490 learned_addr learned my addr 172.21.15.32:0/901932490 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.431+0000 7fb92ffff700 1 --2- 172.21.15.32:0/901932490 >> v2:172.21.15.61:3300/0 conn(0x7fb93005feb0 0x7fb9300b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.431+0000 7fb927fff700 1 -- 172.21.15.32:0/901932490 >> v2:172.21.15.61:3300/0 conn(0x7fb93005feb0 msgr2=0x7fb9300b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.431+0000 7fb927fff700 1 --2- 172.21.15.32:0/901932490 >> v2:172.21.15.61:3300/0 conn(0x7fb93005feb0 0x7fb9300b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.431+0000 7fb927fff700 1 -- 172.21.15.32:0/901932490 >> v1:172.21.15.32:6789/0 conn(0x7fb93005f3b0 legacy=0x7fb9300b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.431+0000 7fb92effd700 1 -- 172.21.15.32:0/901932490 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3352545458 0 0) 0x7fb930142450 con 0x7fb93005f3b0 2020-12-05T17:32:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.431+0000 7fb927fff700 1 -- 172.21.15.32:0/901932490 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb930061a20 con 0x7fb930062400 2020-12-05T17:32:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.431+0000 7fb927fff700 1 --2- 172.21.15.32:0/901932490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb930062400 0x7fb93005f790 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fb918002f30 tx=0x7fb918006e80).ready entity=mon.0 client_cookie=643e21bdc53b6858 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:23.443 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.431+0000 7fb92effd700 1 -- 172.21.15.32:0/901932490 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9180097f0 con 0x7fb930062400 2020-12-05T17:32:23.443 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.431+0000 7fb92effd700 1 -- 172.21.15.32:0/901932490 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb918009950 con 0x7fb930062400 2020-12-05T17:32:23.443 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.431+0000 7fb92effd700 1 -- 172.21.15.32:0/901932490 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb918009cb0 con 0x7fb930062400 2020-12-05T17:32:23.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.431+0000 7fb934bc2700 1 -- 172.21.15.32:0/901932490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb930062400 msgr2=0x7fb93005f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:23.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.431+0000 7fb934bc2700 1 --2- 172.21.15.32:0/901932490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb930062400 0x7fb93005f790 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fb918002f30 tx=0x7fb918006e80).stop 2020-12-05T17:32:23.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.431+0000 7fb934bc2700 1 -- 172.21.15.32:0/901932490 shutdown_connections 2020-12-05T17:32:23.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb934bc2700 1 --2- 172.21.15.32:0/901932490 >> v2:172.21.15.61:3300/0 conn(0x7fb93005feb0 0x7fb9300b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb934bc2700 1 --2- 172.21.15.32:0/901932490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb930062400 0x7fb93005f790 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb934bc2700 1 -- 172.21.15.32:0/901932490 >> 172.21.15.32:0/901932490 conn(0x7fb9300b8ee0 msgr2=0x7fb9300b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:23.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb934bc2700 1 -- 172.21.15.32:0/901932490 shutdown_connections 2020-12-05T17:32:23.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb934bc2700 1 -- 172.21.15.32:0/901932490 wait complete. 2020-12-05T17:32:23.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb934bc2700 1 Processor -- start 2020-12-05T17:32:23.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb934bc2700 1 -- start start 2020-12-05T17:32:23.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb934bc2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb93005f3b0 0x7fb930063340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:23.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb934bc2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb93005feb0 0x7fb930146c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:23.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb934bc2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb930062400 0x7fb930147130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:23.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb934bc2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93013d960 con 0x7fb93005f3b0 2020-12-05T17:32:23.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb934bc2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb930149390 con 0x7fb93005feb0 2020-12-05T17:32:23.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb934bc2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9301495c0 con 0x7fb930062400 2020-12-05T17:32:23.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb92f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb93005f3b0 0x7fb930063340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:23.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb92f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb93005f3b0 0x7fb930063340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33834/0 (socket says 172.21.15.32:33834) 2020-12-05T17:32:23.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb92f7fe700 1 -- 172.21.15.32:0/2418375989 learned_addr learned my addr 172.21.15.32:0/2418375989 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:23.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb927fff700 1 --2- 172.21.15.32:0/2418375989 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb93005feb0 0x7fb930146c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:23.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb92ffff700 1 --2- 172.21.15.32:0/2418375989 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb930062400 0x7fb930147130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:23.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb92f7fe700 1 -- 172.21.15.32:0/2418375989 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb930062400 msgr2=0x7fb930147130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:23.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb92f7fe700 1 --2- 172.21.15.32:0/2418375989 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb930062400 0x7fb930147130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb92f7fe700 1 -- 172.21.15.32:0/2418375989 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb93005feb0 msgr2=0x7fb930146c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:23.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb92f7fe700 1 --2- 172.21.15.32:0/2418375989 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb93005feb0 0x7fb930146c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb92f7fe700 1 -- 172.21.15.32:0/2418375989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb930144930 con 0x7fb93005f3b0 2020-12-05T17:32:23.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb927fff700 1 --2- 172.21.15.32:0/2418375989 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb93005feb0 0x7fb930146c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:23.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb92ffff700 1 --2- 172.21.15.32:0/2418375989 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb930062400 0x7fb930147130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:23.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb92f7fe700 1 --2- 172.21.15.32:0/2418375989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb93005f3b0 0x7fb930063340 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fb920000b80 tx=0x7fb920007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb92d7fa700 1 -- 172.21.15.32:0/2418375989 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb920010510 con 0x7fb93005f3b0 2020-12-05T17:32:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb92d7fa700 1 -- 172.21.15.32:0/2418375989 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb920010670 con 0x7fb93005f3b0 2020-12-05T17:32:23.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb934bc2700 1 -- 172.21.15.32:0/2418375989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb930144bd0 con 0x7fb93005f3b0 2020-12-05T17:32:23.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb92d7fa700 1 -- 172.21.15.32:0/2418375989 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9200107e0 con 0x7fb93005f3b0 2020-12-05T17:32:23.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.435+0000 7fb934bc2700 1 -- 172.21.15.32:0/2418375989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb930147620 con 0x7fb93005f3b0 2020-12-05T17:32:23.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.439+0000 7fb934bc2700 1 -- 172.21.15.32:0/2418375989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb930144d60 con 0x7fb93005f3b0 2020-12-05T17:32:23.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.439+0000 7fb92d7fa700 1 -- 172.21.15.32:0/2418375989 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb920010070 con 0x7fb93005f3b0 2020-12-05T17:32:23.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.439+0000 7fb92d7fa700 1 --2- 172.21.15.32:0/2418375989 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb9140554e0 0x7fb9140579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:23.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.439+0000 7fb927fff700 1 --2- 172.21.15.32:0/2418375989 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb9140554e0 0x7fb9140579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:23.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.439+0000 7fb92d7fa700 1 -- 172.21.15.32:0/2418375989 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fb92006d710 con 0x7fb93005f3b0 2020-12-05T17:32:23.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.439+0000 7fb927fff700 1 --2- 172.21.15.32:0/2418375989 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb9140554e0 0x7fb9140579a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fb91801a000 tx=0x7fb918015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:23.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.447+0000 7fb92d7fa700 1 -- 172.21.15.32:0/2418375989 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb920043bf0 con 0x7fb93005f3b0 2020-12-05T17:32:23.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.587+0000 7fb934bc2700 1 -- 172.21.15.32:0/2418375989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb930057050 con 0x7fb93005f3b0 2020-12-05T17:32:23.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.591+0000 7fb92d7fa700 1 -- 172.21.15.32:0/2418375989 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+143 (secure 0 0 0) 0x7fb920047210 con 0x7fb93005f3b0 2020-12-05T17:32:23.593 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:32:23.594 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2020-12-05T17:32:23.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:23.594 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2020-12-05T17:32:23.594 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2020-12-05T17:32:23.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:32:23.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:32:23.595 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2020-12-05T17:32:23.595 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2020-12-05T17:32:23.596 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:32:23.596 INFO:teuthology.orchestra.run.smithi032.stdout:] 2020-12-05T17:32:23.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.591+0000 7fb934bc2700 1 -- 172.21.15.32:0/2418375989 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb9140554e0 msgr2=0x7fb9140579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:23.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.591+0000 7fb934bc2700 1 --2- 172.21.15.32:0/2418375989 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb9140554e0 0x7fb9140579a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fb91801a000 tx=0x7fb918015000).stop 2020-12-05T17:32:23.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.591+0000 7fb934bc2700 1 -- 172.21.15.32:0/2418375989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb93005f3b0 msgr2=0x7fb930063340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:23.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.591+0000 7fb934bc2700 1 --2- 172.21.15.32:0/2418375989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb93005f3b0 0x7fb930063340 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fb920000b80 tx=0x7fb920007ac0).stop 2020-12-05T17:32:23.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.591+0000 7fb934bc2700 1 -- 172.21.15.32:0/2418375989 shutdown_connections 2020-12-05T17:32:23.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.591+0000 7fb934bc2700 1 --2- 172.21.15.32:0/2418375989 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb9140554e0 0x7fb9140579a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.591+0000 7fb934bc2700 1 --2- 172.21.15.32:0/2418375989 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb930062400 0x7fb930147130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.591+0000 7fb934bc2700 1 --2- 172.21.15.32:0/2418375989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb93005f3b0 0x7fb930063340 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.591+0000 7fb934bc2700 1 --2- 172.21.15.32:0/2418375989 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb93005feb0 0x7fb930146c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.591+0000 7fb934bc2700 1 -- 172.21.15.32:0/2418375989 >> 172.21.15.32:0/2418375989 conn(0x7fb9300b8ee0 msgr2=0x7fb9300b8320 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:23.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.591+0000 7fb934bc2700 1 -- 172.21.15.32:0/2418375989 shutdown_connections 2020-12-05T17:32:23.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.591+0000 7fb934bc2700 1 -- 172.21.15.32:0/2418375989 wait complete. 2020-12-05T17:32:23.612 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-12-05T17:32:23.613 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-12-05T17:32:23.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.731+0000 7f1a2b187700 1 Processor -- start 2020-12-05T17:32:23.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.731+0000 7f1a2b187700 1 -- start start 2020-12-05T17:32:23.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a2b187700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a24061ba0 0x7f1a24066e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:23.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a2b187700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a240b3540 con 0x7f1a240b3020 2020-12-05T17:32:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a2b187700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a24061f80 con 0x7f1a24144380 2020-12-05T17:32:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a2b187700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a24066750 con 0x7f1a24061ba0 2020-12-05T17:32:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29984700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1a240b3020 0x7f1a24143c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36490/0 (socket says 172.21.15.32:36490) 2020-12-05T17:32:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29984700 1 -- 172.21.15.32:0/1037539335 learned_addr learned my addr 172.21.15.32:0/1037539335 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a2a185700 1 --2- 172.21.15.32:0/1037539335 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a24061ba0 0x7f1a24066e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:23.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29183700 1 -- 172.21.15.32:0/1037539335 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4194086130 0 0) 0x7f1a240b3540 con 0x7f1a240b3020 2020-12-05T17:32:23.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29183700 1 -- 172.21.15.32:0/1037539335 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a0c003230 con 0x7f1a240b3020 2020-12-05T17:32:23.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29183700 1 -- 172.21.15.32:0/1037539335 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4270971592 0 0) 0x7f1a24061f80 con 0x7f1a24144380 2020-12-05T17:32:23.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29183700 1 -- 172.21.15.32:0/1037539335 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a240b3540 con 0x7f1a24144380 2020-12-05T17:32:23.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29183700 1 -- 172.21.15.32:0/1037539335 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2008681807 0 0) 0x7f1a0c003230 con 0x7f1a240b3020 2020-12-05T17:32:23.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29183700 1 -- 172.21.15.32:0/1037539335 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a24061ba0 msgr2=0x7f1a24066e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:23.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29183700 1 --2- 172.21.15.32:0/1037539335 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a24061ba0 0x7f1a24066e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29183700 1 -- 172.21.15.32:0/1037539335 >> v1:172.21.15.61:6789/0 conn(0x7f1a24144380 legacy=0x7f1a2413a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:23.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29183700 1 -- 172.21.15.32:0/1037539335 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a240669f0 con 0x7f1a240b3020 2020-12-05T17:32:23.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29183700 1 -- 172.21.15.32:0/1037539335 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f1a200016d0 con 0x7f1a240b3020 2020-12-05T17:32:23.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a2a185700 1 --2- 172.21.15.32:0/1037539335 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a24061ba0 0x7f1a24066e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:32:23.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29183700 1 -- 172.21.15.32:0/1037539335 >> v1:172.21.15.32:6789/0 conn(0x7f1a240b3020 legacy=0x7f1a24143c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:23.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29183700 1 --2- 172.21.15.32:0/1037539335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a0c0038e0 0x7f1a0c003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:23.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29183700 1 --2- 172.21.15.32:0/1037539335 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a0c0044d0 0x7f1a0c004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:23.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29183700 1 --2- 172.21.15.32:0/1037539335 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a24061ba0 0x7f1a0c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:23.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a2a185700 1 --2- 172.21.15.32:0/1037539335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a0c0038e0 0x7f1a0c003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:23.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29183700 1 -- 172.21.15.32:0/1037539335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a24066750 con 0x7f1a0c0038e0 2020-12-05T17:32:23.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29183700 1 -- 172.21.15.32:0/1037539335 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c005330 con 0x7f1a0c0044d0 2020-12-05T17:32:23.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29183700 1 -- 172.21.15.32:0/1037539335 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c00b750 con 0x7f1a24061ba0 2020-12-05T17:32:23.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29984700 1 --2- 172.21.15.32:0/1037539335 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a0c0044d0 0x7f1a0c004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:23.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a2a986700 1 --2- 172.21.15.32:0/1037539335 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a24061ba0 0x7f1a0c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:23.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a2a185700 1 -- 172.21.15.32:0/1037539335 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a24061ba0 msgr2=0x7f1a0c004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:23.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a2a185700 1 --2- 172.21.15.32:0/1037539335 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a24061ba0 0x7f1a0c004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a2a185700 1 -- 172.21.15.32:0/1037539335 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a0c0044d0 msgr2=0x7f1a0c004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:23.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a2a185700 1 --2- 172.21.15.32:0/1037539335 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a0c0044d0 0x7f1a0c004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a2a185700 1 -- 172.21.15.32:0/1037539335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0c003230 con 0x7f1a0c0038e0 2020-12-05T17:32:23.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29984700 1 --2- 172.21.15.32:0/1037539335 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a0c0044d0 0x7f1a0c004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:23.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a2a986700 1 --2- 172.21.15.32:0/1037539335 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a24061ba0 0x7f1a0c004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:32:23.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a2a185700 1 --2- 172.21.15.32:0/1037539335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a0c0038e0 0x7f1a0c003ce0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f1a1800c000 tx=0x7f1a18008040).ready entity=mon.0 client_cookie=1862bae7922ed815 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:23.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29183700 1 -- 172.21.15.32:0/1037539335 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a1800ec30 con 0x7f1a0c0038e0 2020-12-05T17:32:23.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29183700 1 -- 172.21.15.32:0/1037539335 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1800ed90 con 0x7f1a0c0038e0 2020-12-05T17:32:23.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a29183700 1 -- 172.21.15.32:0/1037539335 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a180125e0 con 0x7f1a0c0038e0 2020-12-05T17:32:23.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a2b187700 1 -- 172.21.15.32:0/1037539335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a0c0038e0 msgr2=0x7f1a0c003ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:23.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a2b187700 1 --2- 172.21.15.32:0/1037539335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a0c0038e0 0x7f1a0c003ce0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f1a1800c000 tx=0x7f1a18008040).stop 2020-12-05T17:32:23.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a2a986700 1 -- 172.21.15.32:0/1037539335 reap_dead start 2020-12-05T17:32:23.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a2b187700 1 -- 172.21.15.32:0/1037539335 shutdown_connections 2020-12-05T17:32:23.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.735+0000 7f1a2b187700 1 -- 172.21.15.32:0/1037539335 >> 172.21.15.32:0/1037539335 conn(0x7f1a240b6d10 msgr2=0x7f1a240b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:23.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.739+0000 7f1a2b187700 1 -- 172.21.15.32:0/1037539335 shutdown_connections 2020-12-05T17:32:23.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.739+0000 7f1a2b187700 1 -- 172.21.15.32:0/1037539335 wait complete. 2020-12-05T17:32:23.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.739+0000 7f1a2b187700 1 Processor -- start 2020-12-05T17:32:23.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.739+0000 7f1a2b187700 1 -- start start 2020-12-05T17:32:23.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a2b187700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a2413f5d0 0x7f1a240aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:23.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a2b187700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a2414f1e0 0x7f1a2413c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:23.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a2b187700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a2413cb80 0x7f1a24110330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:23.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a2b187700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a240b0ab0 con 0x7f1a2413f5d0 2020-12-05T17:32:23.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a2b187700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2413d030 con 0x7f1a2414f1e0 2020-12-05T17:32:23.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a2b187700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2413f9b0 con 0x7f1a2413cb80 2020-12-05T17:32:23.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a2a185700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a2413f5d0 0x7f1a240aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:23.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a2a185700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a2413f5d0 0x7f1a240aee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33854/0 (socket says 172.21.15.32:33854) 2020-12-05T17:32:23.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a2a185700 1 -- 172.21.15.32:0/1529540108 learned_addr learned my addr 172.21.15.32:0/1529540108 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:23.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a29984700 1 --2- 172.21.15.32:0/1529540108 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a2414f1e0 0x7f1a2413c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:23.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a2a986700 1 --2- 172.21.15.32:0/1529540108 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a2413cb80 0x7f1a24110330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:23.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a2a185700 1 -- 172.21.15.32:0/1529540108 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a2413cb80 msgr2=0x7f1a24110330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:23.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a2a185700 1 --2- 172.21.15.32:0/1529540108 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a2413cb80 0x7f1a24110330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a2a185700 1 -- 172.21.15.32:0/1529540108 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a2414f1e0 msgr2=0x7f1a2413c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:23.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a2a185700 1 --2- 172.21.15.32:0/1529540108 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a2414f1e0 0x7f1a2413c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:23.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a2a185700 1 -- 172.21.15.32:0/1529540108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a240ae150 con 0x7f1a2413f5d0 2020-12-05T17:32:23.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a29984700 1 --2- 172.21.15.32:0/1529540108 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a2414f1e0 0x7f1a2413c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:23.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a2a185700 1 --2- 172.21.15.32:0/1529540108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a2413f5d0 0x7f1a240aee50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f1a1800c000 tx=0x7f1a1801a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:23.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a177fe700 1 -- 172.21.15.32:0/1529540108 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a1801c430 con 0x7f1a2413f5d0 2020-12-05T17:32:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a177fe700 1 -- 172.21.15.32:0/1529540108 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1801c590 con 0x7f1a2413f5d0 2020-12-05T17:32:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a2b187700 1 -- 172.21.15.32:0/1529540108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a240add20 con 0x7f1a2413f5d0 2020-12-05T17:32:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a177fe700 1 -- 172.21.15.32:0/1529540108 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a1801c8f0 con 0x7f1a2413f5d0 2020-12-05T17:32:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a2b187700 1 -- 172.21.15.32:0/1529540108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a240aff90 con 0x7f1a2413f5d0 2020-12-05T17:32:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.743+0000 7f1a2b187700 1 -- 172.21.15.32:0/1529540108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a240b0180 con 0x7f1a2413f5d0 2020-12-05T17:32:23.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.747+0000 7f1a177fe700 1 -- 172.21.15.32:0/1529540108 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1a1802c070 con 0x7f1a2413f5d0 2020-12-05T17:32:23.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.747+0000 7f1a177fe700 1 --2- 172.21.15.32:0/1529540108 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1a04055470 0x7f1a04057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:23.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.747+0000 7f1a177fe700 1 -- 172.21.15.32:0/1529540108 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f1a1806f7e0 con 0x7f1a2413f5d0 2020-12-05T17:32:23.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.747+0000 7f1a29984700 1 --2- 172.21.15.32:0/1529540108 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1a04055470 0x7f1a04057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:23.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.747+0000 7f1a29984700 1 --2- 172.21.15.32:0/1529540108 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1a04055470 0x7f1a04057930 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f1a2000d000 tx=0x7f1a20009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:23.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.755+0000 7f1a177fe700 1 -- 172.21.15.32:0/1529540108 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1a18045b30 con 0x7f1a2413f5d0 2020-12-05T17:32:23.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:23.899+0000 7f1a2b187700 1 -- 172.21.15.32:0/1529540108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f1a24057050 con 0x7f1a2413f5d0 2020-12-05T17:32:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.431+0000 7f1a177fe700 1 -- 172.21.15.32:0/1529540108 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v31) v1 ==== 133+0+0 (secure 0 0 0) 0x7f1a18016070 con 0x7f1a2413f5d0 2020-12-05T17:32:24.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.435+0000 7f1a2b187700 1 -- 172.21.15.32:0/1529540108 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1a04055470 msgr2=0x7f1a04057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:24.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.435+0000 7f1a2b187700 1 --2- 172.21.15.32:0/1529540108 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1a04055470 0x7f1a04057930 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f1a2000d000 tx=0x7f1a20009040).stop 2020-12-05T17:32:24.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.435+0000 7f1a2b187700 1 -- 172.21.15.32:0/1529540108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a2413f5d0 msgr2=0x7f1a240aee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:24.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.435+0000 7f1a2b187700 1 --2- 172.21.15.32:0/1529540108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a2413f5d0 0x7f1a240aee50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f1a1800c000 tx=0x7f1a1801a000).stop 2020-12-05T17:32:24.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.435+0000 7f1a2b187700 1 -- 172.21.15.32:0/1529540108 shutdown_connections 2020-12-05T17:32:24.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.435+0000 7f1a2b187700 1 --2- 172.21.15.32:0/1529540108 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1a04055470 0x7f1a04057930 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0x7f1a2000d000 tx=0x7f1a20009040).stop 2020-12-05T17:32:24.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.435+0000 7f1a2b187700 1 --2- 172.21.15.32:0/1529540108 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a2413cb80 0x7f1a24110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:24.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.435+0000 7f1a2b187700 1 --2- 172.21.15.32:0/1529540108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a2413f5d0 0x7f1a240aee50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:24.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.435+0000 7f1a2b187700 1 --2- 172.21.15.32:0/1529540108 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a2414f1e0 0x7f1a2413c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:24.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.435+0000 7f1a2b187700 1 -- 172.21.15.32:0/1529540108 >> 172.21.15.32:0/1529540108 conn(0x7f1a240b6d10 msgr2=0x7f1a240b5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:24.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.439+0000 7f1a2b187700 1 -- 172.21.15.32:0/1529540108 shutdown_connections 2020-12-05T17:32:24.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.439+0000 7f1a2b187700 1 -- 172.21.15.32:0/1529540108 wait complete. 2020-12-05T17:32:24.446 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2020-12-05T17:32:24.458 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-12-05T17:32:24.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.579+0000 7f84eab19700 1 Processor -- start 2020-12-05T17:32:24.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.579+0000 7f84eab19700 1 -- start start 2020-12-05T17:32:24.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.579+0000 7f84eab19700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84e405feb0 0x7f84e405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.579+0000 7f84eab19700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e40b2df0 con 0x7f84e405f3b0 2020-12-05T17:32:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.579+0000 7f84eab19700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e40b3c90 con 0x7f84e40b2fb0 2020-12-05T17:32:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.579+0000 7f84eab19700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e40b2b50 con 0x7f84e405feb0 2020-12-05T17:32:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.579+0000 7f84e9316700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84e405feb0 0x7f84e405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.579+0000 7f84e9b17700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f84e405f3b0 0x7f84e413c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36512/0 (socket says 172.21.15.32:36512) 2020-12-05T17:32:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.579+0000 7f84e9b17700 1 -- 172.21.15.32:0/2761331927 learned_addr learned my addr 172.21.15.32:0/2761331927 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:24.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.583+0000 7f84e9316700 1 -- 172.21.15.32:0/2761331927 >> v1:172.21.15.61:6789/0 conn(0x7f84e40b2fb0 legacy=0x7f84e4142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.583+0000 7f84e8b15700 1 -- 172.21.15.32:0/2761331927 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3004693689 0 0) 0x7f84e40b2df0 con 0x7f84e405f3b0 2020-12-05T17:32:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.583+0000 7f84e9316700 1 -- 172.21.15.32:0/2761331927 >> v1:172.21.15.32:6789/0 conn(0x7f84e405f3b0 legacy=0x7f84e413c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.583+0000 7f84e9316700 1 -- 172.21.15.32:0/2761331927 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e4142820 con 0x7f84e405feb0 2020-12-05T17:32:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.583+0000 7f84e9316700 1 --2- 172.21.15.32:0/2761331927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e405feb0 0x7f84e405f790 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f84d4002f30 tx=0x7f84d4006dc0).ready entity=mon.0 client_cookie=99b56b2b444aa649 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:24.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.583+0000 7f84e8b15700 1 -- 172.21.15.32:0/2761331927 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84d40097f0 con 0x7f84e405feb0 2020-12-05T17:32:24.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.583+0000 7f84e8b15700 1 -- 172.21.15.32:0/2761331927 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d4009950 con 0x7f84e405feb0 2020-12-05T17:32:24.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.583+0000 7f84e8b15700 1 -- 172.21.15.32:0/2761331927 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84d4009cb0 con 0x7f84e405feb0 2020-12-05T17:32:24.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.583+0000 7f84eab19700 1 -- 172.21.15.32:0/2761331927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e405feb0 msgr2=0x7f84e405f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:24.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.583+0000 7f84eab19700 1 --2- 172.21.15.32:0/2761331927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e405feb0 0x7f84e405f790 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f84d4002f30 tx=0x7f84d4006dc0).stop 2020-12-05T17:32:24.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.583+0000 7f84eab19700 1 -- 172.21.15.32:0/2761331927 shutdown_connections 2020-12-05T17:32:24.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.583+0000 7f84eab19700 1 --2- 172.21.15.32:0/2761331927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e405feb0 0x7f84e405f790 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:24.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.583+0000 7f84eab19700 1 -- 172.21.15.32:0/2761331927 >> 172.21.15.32:0/2761331927 conn(0x7f84e40b8f20 msgr2=0x7f84e40b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:24.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.583+0000 7f84eab19700 1 -- 172.21.15.32:0/2761331927 shutdown_connections 2020-12-05T17:32:24.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.583+0000 7f84eab19700 1 -- 172.21.15.32:0/2761331927 wait complete. 2020-12-05T17:32:24.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.583+0000 7f84eab19700 1 Processor -- start 2020-12-05T17:32:24.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.583+0000 7f84eab19700 1 -- start start 2020-12-05T17:32:24.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.587+0000 7f84eab19700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f84e405f3b0 0x7f84e413d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:24.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.587+0000 7f84eab19700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e405feb0 0x7f84e41513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:24.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.587+0000 7f84eab19700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f84e40b2fb0 0x7f84e41518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:24.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.587+0000 7f84eab19700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e413e860 con 0x7f84e405feb0 2020-12-05T17:32:24.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.587+0000 7f84eab19700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e413eb00 con 0x7f84e405f3b0 2020-12-05T17:32:24.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.587+0000 7f84eab19700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e413eda0 con 0x7f84e40b2fb0 2020-12-05T17:32:24.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.587+0000 7f84e9316700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e405feb0 0x7f84e41513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:24.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.587+0000 7f84e9316700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e405feb0 0x7f84e41513f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33872/0 (socket says 172.21.15.32:33872) 2020-12-05T17:32:24.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.587+0000 7f84e9316700 1 -- 172.21.15.32:0/3934649076 learned_addr learned my addr 172.21.15.32:0/3934649076 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:24.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.587+0000 7f84e9b17700 1 --2- 172.21.15.32:0/3934649076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f84e405f3b0 0x7f84e413d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:24.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.587+0000 7f84ea318700 1 --2- 172.21.15.32:0/3934649076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f84e40b2fb0 0x7f84e41518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:24.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.587+0000 7f84e9316700 1 -- 172.21.15.32:0/3934649076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f84e40b2fb0 msgr2=0x7f84e41518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:24.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.587+0000 7f84e9316700 1 --2- 172.21.15.32:0/3934649076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f84e40b2fb0 0x7f84e41518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:24.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.587+0000 7f84e9316700 1 -- 172.21.15.32:0/3934649076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f84e405f3b0 msgr2=0x7f84e413d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:24.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.587+0000 7f84e9316700 1 --2- 172.21.15.32:0/3934649076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f84e405f3b0 0x7f84e413d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:24.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.587+0000 7f84e9316700 1 -- 172.21.15.32:0/3934649076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e413f240 con 0x7f84e405feb0 2020-12-05T17:32:24.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.587+0000 7f84e9316700 1 --2- 172.21.15.32:0/3934649076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e405feb0 0x7f84e41513f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f84d4002f30 tx=0x7f84d4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:24.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.587+0000 7f84d2ffd700 1 -- 172.21.15.32:0/3934649076 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84d4011e10 con 0x7f84e405feb0 2020-12-05T17:32:24.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.587+0000 7f84d2ffd700 1 -- 172.21.15.32:0/3934649076 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d401d450 con 0x7f84e405feb0 2020-12-05T17:32:24.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.587+0000 7f84eab19700 1 -- 172.21.15.32:0/3934649076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e413e4e0 con 0x7f84e405feb0 2020-12-05T17:32:24.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.587+0000 7f84d2ffd700 1 -- 172.21.15.32:0/3934649076 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84d401d7b0 con 0x7f84e405feb0 2020-12-05T17:32:24.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.587+0000 7f84eab19700 1 -- 172.21.15.32:0/3934649076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e4146c50 con 0x7f84e405feb0 2020-12-05T17:32:24.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.591+0000 7f84d2ffd700 1 -- 172.21.15.32:0/3934649076 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f84d4024030 con 0x7f84e405feb0 2020-12-05T17:32:24.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.591+0000 7f84eab19700 1 -- 172.21.15.32:0/3934649076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e413e6d0 con 0x7f84e405feb0 2020-12-05T17:32:24.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.591+0000 7f84d2ffd700 1 --2- 172.21.15.32:0/3934649076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f84c4055440 0x7f84c4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:24.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.591+0000 7f84d2ffd700 1 -- 172.21.15.32:0/3934649076 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f84d406f1f0 con 0x7f84e405feb0 2020-12-05T17:32:24.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.591+0000 7f84e9b17700 1 --2- 172.21.15.32:0/3934649076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f84c4055440 0x7f84c4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:24.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.591+0000 7f84e9b17700 1 --2- 172.21.15.32:0/3934649076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f84c4055440 0x7f84c4057900 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f84e000d000 tx=0x7f84e0009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:24.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.595+0000 7f84d2ffd700 1 -- 172.21.15.32:0/3934649076 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f84d4045560 con 0x7f84e405feb0 2020-12-05T17:32:24.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:24.743+0000 7f84eab19700 1 -- 172.21.15.32:0/3934649076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f84e4146a70 con 0x7f84e405feb0 2020-12-05T17:32:25.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.443+0000 7f84d2ffd700 1 -- 172.21.15.32:0/3934649076 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v32) v1 ==== 125+0+0 (secure 0 0 0) 0x7f84d4022020 con 0x7f84e405feb0 2020-12-05T17:32:25.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.451+0000 7f84eab19700 1 -- 172.21.15.32:0/3934649076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f84c4055440 msgr2=0x7f84c4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:25.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.451+0000 7f84eab19700 1 --2- 172.21.15.32:0/3934649076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f84c4055440 0x7f84c4057900 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f84e000d000 tx=0x7f84e0009040).stop 2020-12-05T17:32:25.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.451+0000 7f84eab19700 1 -- 172.21.15.32:0/3934649076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e405feb0 msgr2=0x7f84e41513f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:25.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.451+0000 7f84eab19700 1 --2- 172.21.15.32:0/3934649076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e405feb0 0x7f84e41513f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f84d4002f30 tx=0x7f84d4015000).stop 2020-12-05T17:32:25.461 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.455+0000 7f84eab19700 1 -- 172.21.15.32:0/3934649076 shutdown_connections 2020-12-05T17:32:25.461 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.455+0000 7f84eab19700 1 --2- 172.21.15.32:0/3934649076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f84c4055440 0x7f84c4057900 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0x7f84e000d000 tx=0x7f84e0009040).stop 2020-12-05T17:32:25.462 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.455+0000 7f84eab19700 1 --2- 172.21.15.32:0/3934649076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f84e40b2fb0 0x7f84e41518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:25.462 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.455+0000 7f84eab19700 1 --2- 172.21.15.32:0/3934649076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f84e405f3b0 0x7f84e413d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:25.462 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.455+0000 7f84eab19700 1 --2- 172.21.15.32:0/3934649076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e405feb0 0x7f84e41513f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:25.463 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.455+0000 7f84eab19700 1 -- 172.21.15.32:0/3934649076 >> 172.21.15.32:0/3934649076 conn(0x7f84e40b8f20 msgr2=0x7f84e40b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:25.463 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.455+0000 7f84eab19700 1 -- 172.21.15.32:0/3934649076 shutdown_connections 2020-12-05T17:32:25.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.455+0000 7f84eab19700 1 -- 172.21.15.32:0/3934649076 wait complete. 2020-12-05T17:32:25.464 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2020-12-05T17:32:25.473 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-12-05T17:32:25.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.579+0000 7fbb37231700 1 Processor -- start 2020-12-05T17:32:25.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.579+0000 7fbb37231700 1 -- start start 2020-12-05T17:32:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.583+0000 7fbb37231700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fbb300af0d0 0x7fbb3013daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:25.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.583+0000 7fbb37231700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb3005f5c0 con 0x7fbb3013df90 2020-12-05T17:32:25.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.583+0000 7fbb37231700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb300af4b0 con 0x7fbb300b0d70 2020-12-05T17:32:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.583+0000 7fbb37231700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb3005f7f0 con 0x7fbb300af0d0 2020-12-05T17:32:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.583+0000 7fbb35a2e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbb3013df90 0x7fbb300637f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36532/0 (socket says 172.21.15.32:36532) 2020-12-05T17:32:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.583+0000 7fbb35a2e700 1 -- 172.21.15.32:0/937033677 learned_addr learned my addr 172.21.15.32:0/937033677 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:25.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.583+0000 7fbb3622f700 1 --2- 172.21.15.32:0/937033677 >> v2:172.21.15.61:3300/0 conn(0x7fbb300af0d0 0x7fbb3013daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:25.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.583+0000 7fbb3522d700 1 -- 172.21.15.32:0/937033677 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1591167667 0 0) 0x7fbb3005f5c0 con 0x7fbb3013df90 2020-12-05T17:32:25.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.583+0000 7fbb3522d700 1 -- 172.21.15.32:0/937033677 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb1c003230 con 0x7fbb3013df90 2020-12-05T17:32:25.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.583+0000 7fbb3622f700 1 -- 172.21.15.32:0/937033677 >> v1:172.21.15.61:6789/0 conn(0x7fbb300b0d70 legacy=0x7fbb30063f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:25.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.583+0000 7fbb3622f700 1 -- 172.21.15.32:0/937033677 >> v1:172.21.15.32:6789/0 conn(0x7fbb3013df90 legacy=0x7fbb300637f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.583+0000 7fbb3622f700 1 -- 172.21.15.32:0/937033677 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb30066ae0 con 0x7fbb300af0d0 2020-12-05T17:32:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.583+0000 7fbb3622f700 1 --2- 172.21.15.32:0/937033677 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb300af0d0 0x7fbb3013daa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fbb20002f30 tx=0x7fbb20006dc0).ready entity=mon.1 client_cookie=8ac91c2b878ec0fa server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:25.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.583+0000 7fbb3522d700 1 -- 172.21.15.32:0/937033677 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb20009800 con 0x7fbb300af0d0 2020-12-05T17:32:25.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.583+0000 7fbb3522d700 1 -- 172.21.15.32:0/937033677 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb20009960 con 0x7fbb300af0d0 2020-12-05T17:32:25.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.583+0000 7fbb3522d700 1 -- 172.21.15.32:0/937033677 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb20009cc0 con 0x7fbb300af0d0 2020-12-05T17:32:25.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.583+0000 7fbb37231700 1 -- 172.21.15.32:0/937033677 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb300af0d0 msgr2=0x7fbb3013daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:25.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.583+0000 7fbb37231700 1 --2- 172.21.15.32:0/937033677 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb300af0d0 0x7fbb3013daa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fbb20002f30 tx=0x7fbb20006dc0).stop 2020-12-05T17:32:25.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.583+0000 7fbb37231700 1 -- 172.21.15.32:0/937033677 shutdown_connections 2020-12-05T17:32:25.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.583+0000 7fbb37231700 1 --2- 172.21.15.32:0/937033677 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb300af0d0 0x7fbb3013daa0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:25.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.583+0000 7fbb37231700 1 -- 172.21.15.32:0/937033677 >> 172.21.15.32:0/937033677 conn(0x7fbb300620b0 msgr2=0x7fbb30062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:25.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.583+0000 7fbb37231700 1 -- 172.21.15.32:0/937033677 shutdown_connections 2020-12-05T17:32:25.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.583+0000 7fbb37231700 1 -- 172.21.15.32:0/937033677 wait complete. 2020-12-05T17:32:25.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb37231700 1 Processor -- start 2020-12-05T17:32:25.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb37231700 1 -- start start 2020-12-05T17:32:25.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb37231700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb300af0d0 0x7fbb301434a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:25.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb37231700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb300b0d70 0x7fbb30143990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:25.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb37231700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb3013df90 0x7fbb3013ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:25.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb37231700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb30110180 con 0x7fbb3013df90 2020-12-05T17:32:25.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb37231700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb30110420 con 0x7fbb300b0d70 2020-12-05T17:32:25.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb37231700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb30143f30 con 0x7fbb300af0d0 2020-12-05T17:32:25.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb36a30700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb3013df90 0x7fbb3013ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:25.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb36a30700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb3013df90 0x7fbb3013ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33890/0 (socket says 172.21.15.32:33890) 2020-12-05T17:32:25.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb36a30700 1 -- 172.21.15.32:0/1276276394 learned_addr learned my addr 172.21.15.32:0/1276276394 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:25.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb3622f700 1 --2- 172.21.15.32:0/1276276394 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb300af0d0 0x7fbb301434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:25.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb35a2e700 1 --2- 172.21.15.32:0/1276276394 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb300b0d70 0x7fbb30143990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:25.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb36a30700 1 -- 172.21.15.32:0/1276276394 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb300af0d0 msgr2=0x7fbb301434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:25.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb36a30700 1 --2- 172.21.15.32:0/1276276394 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb300af0d0 0x7fbb301434a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:25.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb36a30700 1 -- 172.21.15.32:0/1276276394 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb300b0d70 msgr2=0x7fbb30143990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:25.609 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb36a30700 1 --2- 172.21.15.32:0/1276276394 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb300b0d70 0x7fbb30143990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:25.609 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb36a30700 1 -- 172.21.15.32:0/1276276394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb30142470 con 0x7fbb3013df90 2020-12-05T17:32:25.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb36a30700 1 --2- 172.21.15.32:0/1276276394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb3013df90 0x7fbb3013ce00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fbb28000fe0 tx=0x7fbb28007ba0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:25.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb2f7fe700 1 -- 172.21.15.32:0/1276276394 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb28007240 con 0x7fbb3013df90 2020-12-05T17:32:25.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb2f7fe700 1 -- 172.21.15.32:0/1276276394 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb280073a0 con 0x7fbb3013df90 2020-12-05T17:32:25.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb37231700 1 -- 172.21.15.32:0/1276276394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb301443d0 con 0x7fbb3013df90 2020-12-05T17:32:25.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb2f7fe700 1 -- 172.21.15.32:0/1276276394 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb280066d0 con 0x7fbb3013df90 2020-12-05T17:32:25.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.587+0000 7fbb37231700 1 -- 172.21.15.32:0/1276276394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb30142030 con 0x7fbb3013df90 2020-12-05T17:32:25.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.591+0000 7fbb37231700 1 -- 172.21.15.32:0/1276276394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb30057050 con 0x7fbb3013df90 2020-12-05T17:32:25.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.591+0000 7fbb2f7fe700 1 -- 172.21.15.32:0/1276276394 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbb2801a060 con 0x7fbb3013df90 2020-12-05T17:32:25.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.591+0000 7fbb2f7fe700 1 --2- 172.21.15.32:0/1276276394 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbb100553b0 0x7fbb10057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:25.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.591+0000 7fbb3622f700 1 --2- 172.21.15.32:0/1276276394 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbb100553b0 0x7fbb10057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:25.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.591+0000 7fbb2f7fe700 1 -- 172.21.15.32:0/1276276394 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fbb2806d890 con 0x7fbb3013df90 2020-12-05T17:32:25.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.595+0000 7fbb3622f700 1 --2- 172.21.15.32:0/1276276394 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbb100553b0 0x7fbb10057870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fbb2001a000 tx=0x7fbb20015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:25.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.599+0000 7fbb2f7fe700 1 -- 172.21.15.32:0/1276276394 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbb28043a50 con 0x7fbb3013df90 2020-12-05T17:32:25.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.787+0000 7fbb37231700 1 -- 172.21.15.32:0/1276276394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fbb301445c0 con 0x7fbb3013df90 2020-12-05T17:32:25.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.799+0000 7fbb2f7fe700 1 -- 172.21.15.32:0/1276276394 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 5 and data pool 6 v9) v1 ==== 187+0+0 (secure 0 0 0) 0x7fbb280180d0 con 0x7fbb3013df90 2020-12-05T17:32:25.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.803+0000 7fbb37231700 1 -- 172.21.15.32:0/1276276394 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbb100553b0 msgr2=0x7fbb10057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:25.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.803+0000 7fbb37231700 1 --2- 172.21.15.32:0/1276276394 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbb100553b0 0x7fbb10057870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fbb2001a000 tx=0x7fbb20015000).stop 2020-12-05T17:32:25.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.803+0000 7fbb37231700 1 -- 172.21.15.32:0/1276276394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb3013df90 msgr2=0x7fbb3013ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:25.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.803+0000 7fbb37231700 1 --2- 172.21.15.32:0/1276276394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb3013df90 0x7fbb3013ce00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fbb28000fe0 tx=0x7fbb28007ba0).stop 2020-12-05T17:32:25.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.803+0000 7fbb37231700 1 -- 172.21.15.32:0/1276276394 shutdown_connections 2020-12-05T17:32:25.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.803+0000 7fbb37231700 1 --2- 172.21.15.32:0/1276276394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb3013df90 0x7fbb3013ce00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:25.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.803+0000 7fbb37231700 1 --2- 172.21.15.32:0/1276276394 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbb100553b0 0x7fbb10057870 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:25.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.803+0000 7fbb37231700 1 --2- 172.21.15.32:0/1276276394 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb300af0d0 0x7fbb301434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:25.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.803+0000 7fbb37231700 1 --2- 172.21.15.32:0/1276276394 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb300b0d70 0x7fbb30143990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:25.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.803+0000 7fbb37231700 1 -- 172.21.15.32:0/1276276394 >> 172.21.15.32:0/1276276394 conn(0x7fbb300620b0 msgr2=0x7fbb30141160 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:25.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.803+0000 7fbb37231700 1 -- 172.21.15.32:0/1276276394 shutdown_connections 2020-12-05T17:32:25.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.803+0000 7fbb37231700 1 -- 172.21.15.32:0/1276276394 wait complete. 2020-12-05T17:32:25.813 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 5 and data pool 6 2020-12-05T17:32:25.828 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:32:25.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.927+0000 7f492915d700 1 Processor -- start 2020-12-05T17:32:25.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.927+0000 7f492915d700 1 -- start start 2020-12-05T17:32:25.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.927+0000 7f492915d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49240b1fc0 0x7f49240af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:25.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.927+0000 7f492915d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f49240ab710 0x7f4924061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:25.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.927+0000 7f492915d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4924146a90 con 0x7f4924147400 2020-12-05T17:32:25.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.927+0000 7f492915d700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49240b0110 con 0x7f49240ab710 2020-12-05T17:32:25.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.927+0000 7f492915d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49240b23a0 con 0x7f49240b1fc0 2020-12-05T17:32:25.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.927+0000 7f492895c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4924147400 0x7f4924145d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36560/0 (socket says 172.21.15.32:36560) 2020-12-05T17:32:25.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.927+0000 7f492895c700 1 -- 172.21.15.32:0/929324394 learned_addr learned my addr 172.21.15.32:0/929324394 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:25.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.927+0000 7f49237fe700 1 --2- 172.21.15.32:0/929324394 >> v2:172.21.15.61:3300/0 conn(0x7f49240ab710 0x7f4924061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:25.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.927+0000 7f4923fff700 1 --2- 172.21.15.32:0/929324394 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49240b1fc0 0x7f49240af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:25.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.927+0000 7f4922ffd700 1 -- 172.21.15.32:0/929324394 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804783453 0 0) 0x7f4924146a90 con 0x7f4924147400 2020-12-05T17:32:25.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.927+0000 7f4922ffd700 1 -- 172.21.15.32:0/929324394 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4904003230 con 0x7f4924147400 2020-12-05T17:32:25.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.927+0000 7f49237fe700 1 -- 172.21.15.32:0/929324394 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49240b1fc0 msgr2=0x7f49240af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:25.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.927+0000 7f49237fe700 1 --2- 172.21.15.32:0/929324394 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49240b1fc0 0x7f49240af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:25.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.927+0000 7f49237fe700 1 -- 172.21.15.32:0/929324394 >> v1:172.21.15.32:6789/0 conn(0x7f4924147400 legacy=0x7f4924145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:25.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.927+0000 7f49237fe700 1 -- 172.21.15.32:0/929324394 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4924146cc0 con 0x7f49240ab710 2020-12-05T17:32:25.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f49237fe700 1 --2- 172.21.15.32:0/929324394 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49240ab710 0x7f4924061780 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f491c0032b0 tx=0x7f491c0091a0).ready entity=mon.1 client_cookie=2d4667d21d7b2a71 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:25.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f4922ffd700 1 -- 172.21.15.32:0/929324394 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f491c00b940 con 0x7f49240ab710 2020-12-05T17:32:25.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f4922ffd700 1 -- 172.21.15.32:0/929324394 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491c00baa0 con 0x7f49240ab710 2020-12-05T17:32:25.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f4922ffd700 1 -- 172.21.15.32:0/929324394 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f491c00bc10 con 0x7f49240ab710 2020-12-05T17:32:25.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f492915d700 1 -- 172.21.15.32:0/929324394 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49240ab710 msgr2=0x7f4924061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:25.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f492915d700 1 --2- 172.21.15.32:0/929324394 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49240ab710 0x7f4924061780 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f491c0032b0 tx=0x7f491c0091a0).stop 2020-12-05T17:32:25.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f492915d700 1 -- 172.21.15.32:0/929324394 shutdown_connections 2020-12-05T17:32:25.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f492915d700 1 --2- 172.21.15.32:0/929324394 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49240b1fc0 0x7f49240af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:25.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f492915d700 1 --2- 172.21.15.32:0/929324394 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49240ab710 0x7f4924061780 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:25.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f492915d700 1 -- 172.21.15.32:0/929324394 >> 172.21.15.32:0/929324394 conn(0x7f49240b5530 msgr2=0x7f49240b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:25.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f492915d700 1 -- 172.21.15.32:0/929324394 shutdown_connections 2020-12-05T17:32:25.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f492915d700 1 -- 172.21.15.32:0/929324394 wait complete. 2020-12-05T17:32:25.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f492915d700 1 Processor -- start 2020-12-05T17:32:25.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f492915d700 1 -- start start 2020-12-05T17:32:25.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f492915d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49240ab710 0x7f49240ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:25.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f492915d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49240b1fc0 0x7f49240b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:25.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f492915d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4924147400 0x7f4924159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:25.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f492915d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4924158bd0 con 0x7f4924147400 2020-12-05T17:32:25.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f492915d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4924158e70 con 0x7f49240b1fc0 2020-12-05T17:32:25.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f492915d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49241565b0 con 0x7f49240ab710 2020-12-05T17:32:25.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f492895c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4924147400 0x7f4924159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:25.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f4923fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49240ab710 0x7f49240ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:25.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f49237fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49240b1fc0 0x7f49240b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:25.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f492895c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4924147400 0x7f4924159310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33918/0 (socket says 172.21.15.32:33918) 2020-12-05T17:32:25.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f49237fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49240b1fc0 0x7f49240b2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:53808/0 (socket says 172.21.15.32:53808) 2020-12-05T17:32:25.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f4923fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49240ab710 0x7f49240ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:58936/0 (socket says 172.21.15.32:58936) 2020-12-05T17:32:25.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.931+0000 7f492895c700 1 -- 172.21.15.32:0/2697697204 learned_addr learned my addr 172.21.15.32:0/2697697204 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:25.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.935+0000 7f492895c700 1 -- 172.21.15.32:0/2697697204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49240ab710 msgr2=0x7f49240ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:25.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.935+0000 7f492895c700 1 --2- 172.21.15.32:0/2697697204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49240ab710 0x7f49240ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:25.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.935+0000 7f492895c700 1 -- 172.21.15.32:0/2697697204 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49240b1fc0 msgr2=0x7f49240b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:25.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.935+0000 7f492895c700 1 --2- 172.21.15.32:0/2697697204 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49240b1fc0 0x7f49240b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:25.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.935+0000 7f492895c700 1 -- 172.21.15.32:0/2697697204 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4924159800 con 0x7f4924147400 2020-12-05T17:32:25.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.935+0000 7f492895c700 1 --2- 172.21.15.32:0/2697697204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4924147400 0x7f4924159310 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f49140010d0 tx=0x7f4914007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:25.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.935+0000 7f49217fa700 1 -- 172.21.15.32:0/2697697204 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49140072e0 con 0x7f4924147400 2020-12-05T17:32:25.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.935+0000 7f49217fa700 1 -- 172.21.15.32:0/2697697204 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4914007440 con 0x7f4924147400 2020-12-05T17:32:25.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.935+0000 7f492915d700 1 -- 172.21.15.32:0/2697697204 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49240a1ab0 con 0x7f4924147400 2020-12-05T17:32:25.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.935+0000 7f49217fa700 1 -- 172.21.15.32:0/2697697204 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49140057d0 con 0x7f4924147400 2020-12-05T17:32:25.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.935+0000 7f492915d700 1 -- 172.21.15.32:0/2697697204 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49240acc40 con 0x7f4924147400 2020-12-05T17:32:25.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.935+0000 7f492915d700 1 -- 172.21.15.32:0/2697697204 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4924057050 con 0x7f4924147400 2020-12-05T17:32:25.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.935+0000 7f49217fa700 1 -- 172.21.15.32:0/2697697204 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f491401a060 con 0x7f4924147400 2020-12-05T17:32:25.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.939+0000 7f49217fa700 1 --2- 172.21.15.32:0/2697697204 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4908055330 0x7f49080577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:25.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.939+0000 7f4923fff700 1 --2- 172.21.15.32:0/2697697204 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4908055330 0x7f49080577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:25.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.939+0000 7f49217fa700 1 -- 172.21.15.32:0/2697697204 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f491406d830 con 0x7f4924147400 2020-12-05T17:32:25.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.939+0000 7f4923fff700 1 --2- 172.21.15.32:0/2697697204 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4908055330 0x7f49080577f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f490c008000 tx=0x7f490c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:25.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:25.943+0000 7f49217fa700 1 -- 172.21.15.32:0/2697697204 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f49140439d0 con 0x7f4924147400 2020-12-05T17:32:26.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.083+0000 7f492915d700 1 -- 172.21.15.32:0/2697697204 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49240ad1d0 con 0x7f4924147400 2020-12-05T17:32:26.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.087+0000 7f49217fa700 1 -- 172.21.15.32:0/2697697204 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16284 (secure 0 0 0) 0x7f4914018070 con 0x7f4924147400 2020-12-05T17:32:26.090 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:32:26.091 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:32:25.792127+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:32:23.957088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-12-05T17:32:24.800777+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:32:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.087+0000 7f492915d700 1 -- 172.21.15.32:0/2697697204 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4908055330 msgr2=0x7f49080577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.087+0000 7f492915d700 1 --2- 172.21.15.32:0/2697697204 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4908055330 0x7f49080577f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f490c008000 tx=0x7f490c008040).stop 2020-12-05T17:32:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.087+0000 7f492915d700 1 -- 172.21.15.32:0/2697697204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4924147400 msgr2=0x7f4924159310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.087+0000 7f492915d700 1 --2- 172.21.15.32:0/2697697204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4924147400 0x7f4924159310 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f49140010d0 tx=0x7f4914007ac0).stop 2020-12-05T17:32:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.087+0000 7f492915d700 1 -- 172.21.15.32:0/2697697204 shutdown_connections 2020-12-05T17:32:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.087+0000 7f492915d700 1 --2- 172.21.15.32:0/2697697204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4924147400 0x7f4924159310 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.094 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.087+0000 7f492915d700 1 --2- 172.21.15.32:0/2697697204 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4908055330 0x7f49080577f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.087+0000 7f492915d700 1 --2- 172.21.15.32:0/2697697204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49240ab710 0x7f49240ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.087+0000 7f492915d700 1 --2- 172.21.15.32:0/2697697204 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49240b1fc0 0x7f49240b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.087+0000 7f492915d700 1 -- 172.21.15.32:0/2697697204 >> 172.21.15.32:0/2697697204 conn(0x7f49240b5530 msgr2=0x7f4924061d90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:26.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.087+0000 7f492915d700 1 -- 172.21.15.32:0/2697697204 shutdown_connections 2020-12-05T17:32:26.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.087+0000 7f492915d700 1 -- 172.21.15.32:0/2697697204 wait complete. 2020-12-05T17:32:26.104 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:32:26.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.211+0000 7fa1bab39700 1 Processor -- start 2020-12-05T17:32:26.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.211+0000 7fa1bab39700 1 -- start start 2020-12-05T17:32:26.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1bab39700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1b40b5f00 0x7fa1b40b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:26.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1bab39700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1b40ad510 0x7fa1b40af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:26.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1bab39700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1b4142ab0 con 0x7fa1b4066370 2020-12-05T17:32:26.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1bab39700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1b40b62e0 con 0x7fa1b40ad510 2020-12-05T17:32:26.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1bab39700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b40afbf0 con 0x7fa1b40b5f00 2020-12-05T17:32:26.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1b9336700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1b40ad510 0x7fa1b40af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:26.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1ba338700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa1b4066370 0x7fa1b4142ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36574/0 (socket says 172.21.15.32:36574) 2020-12-05T17:32:26.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1ba338700 1 -- 172.21.15.32:0/1443183216 learned_addr learned my addr 172.21.15.32:0/1443183216 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:26.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1b9b37700 1 --2- 172.21.15.32:0/1443183216 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1b40b5f00 0x7fa1b40b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:26.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1b8b35700 1 -- 172.21.15.32:0/1443183216 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 759496238 0 0) 0x7fa1b4142ab0 con 0x7fa1b4066370 2020-12-05T17:32:26.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1b8b35700 1 -- 172.21.15.32:0/1443183216 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa19c003230 con 0x7fa1b4066370 2020-12-05T17:32:26.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1b9336700 1 -- 172.21.15.32:0/1443183216 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1b40b5f00 msgr2=0x7fa1b40b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:26.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1b9336700 1 --2- 172.21.15.32:0/1443183216 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1b40b5f00 0x7fa1b40b14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1b9336700 1 -- 172.21.15.32:0/1443183216 >> v1:172.21.15.32:6789/0 conn(0x7fa1b4066370 legacy=0x7fa1b4142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:26.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1b9336700 1 -- 172.21.15.32:0/1443183216 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b40afe90 con 0x7fa1b40ad510 2020-12-05T17:32:26.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1b9336700 1 --2- 172.21.15.32:0/1443183216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1b40ad510 0x7fa1b40af480 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fa1b0002f30 tx=0x7fa1b0006dc0).ready entity=mon.0 client_cookie=6a1e9db1d78d608d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:26.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1b8b35700 1 -- 172.21.15.32:0/1443183216 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1b000f9c0 con 0x7fa1b40ad510 2020-12-05T17:32:26.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1b8b35700 1 -- 172.21.15.32:0/1443183216 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b000fb20 con 0x7fa1b40ad510 2020-12-05T17:32:26.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1bab39700 1 -- 172.21.15.32:0/1443183216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1b40ad510 msgr2=0x7fa1b40af480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:26.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1b8b35700 1 -- 172.21.15.32:0/1443183216 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1b000fe80 con 0x7fa1b40ad510 2020-12-05T17:32:26.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1bab39700 1 --2- 172.21.15.32:0/1443183216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1b40ad510 0x7fa1b40af480 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fa1b0002f30 tx=0x7fa1b0006dc0).stop 2020-12-05T17:32:26.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1bab39700 1 -- 172.21.15.32:0/1443183216 shutdown_connections 2020-12-05T17:32:26.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1bab39700 1 --2- 172.21.15.32:0/1443183216 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1b40b5f00 0x7fa1b40b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1bab39700 1 --2- 172.21.15.32:0/1443183216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1b40ad510 0x7fa1b40af480 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1bab39700 1 -- 172.21.15.32:0/1443183216 >> 172.21.15.32:0/1443183216 conn(0x7fa1b40604c0 msgr2=0x7fa1b405c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:26.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1bab39700 1 -- 172.21.15.32:0/1443183216 shutdown_connections 2020-12-05T17:32:26.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1bab39700 1 -- 172.21.15.32:0/1443183216 wait complete. 2020-12-05T17:32:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1bab39700 1 Processor -- start 2020-12-05T17:32:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.215+0000 7fa1bab39700 1 -- start start 2020-12-05T17:32:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1bab39700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1b4066370 0x7fa1b413e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1bab39700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1b40ad510 0x7fa1b40b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1bab39700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1b40b5f00 0x7fa1b41511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1bab39700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b4150af0 con 0x7fa1b40ad510 2020-12-05T17:32:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1bab39700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b4064270 con 0x7fa1b40b5f00 2020-12-05T17:32:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1bab39700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b4150d20 con 0x7fa1b4066370 2020-12-05T17:32:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1b9336700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1b40ad510 0x7fa1b40b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1b9b37700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1b4066370 0x7fa1b413e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1b9336700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1b40ad510 0x7fa1b40b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33930/0 (socket says 172.21.15.32:33930) 2020-12-05T17:32:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1b9336700 1 -- 172.21.15.32:0/288015407 learned_addr learned my addr 172.21.15.32:0/288015407 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:26.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1ba338700 1 --2- 172.21.15.32:0/288015407 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1b40b5f00 0x7fa1b41511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:26.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1b9336700 1 -- 172.21.15.32:0/288015407 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1b4066370 msgr2=0x7fa1b413e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:26.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1b9336700 1 --2- 172.21.15.32:0/288015407 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1b4066370 0x7fa1b413e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1b9336700 1 -- 172.21.15.32:0/288015407 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1b40b5f00 msgr2=0x7fa1b41511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:26.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1b9336700 1 --2- 172.21.15.32:0/288015407 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1b40b5f00 0x7fa1b41511c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1b9336700 1 -- 172.21.15.32:0/288015407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b4063fa0 con 0x7fa1b40ad510 2020-12-05T17:32:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1b9336700 1 --2- 172.21.15.32:0/288015407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1b40ad510 0x7fa1b40b3d10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fa1b0002f30 tx=0x7fa1b0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1a6ffd700 1 -- 172.21.15.32:0/288015407 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1b0010ca0 con 0x7fa1b40ad510 2020-12-05T17:32:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1a6ffd700 1 -- 172.21.15.32:0/288015407 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b0010e00 con 0x7fa1b40ad510 2020-12-05T17:32:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1bab39700 1 -- 172.21.15.32:0/288015407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b413e540 con 0x7fa1b40ad510 2020-12-05T17:32:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1a6ffd700 1 -- 172.21.15.32:0/288015407 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1b001a670 con 0x7fa1b40ad510 2020-12-05T17:32:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1bab39700 1 -- 172.21.15.32:0/288015407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b41516b0 con 0x7fa1b40ad510 2020-12-05T17:32:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1a6ffd700 1 -- 172.21.15.32:0/288015407 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa1b002a070 con 0x7fa1b40ad510 2020-12-05T17:32:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.219+0000 7fa1bab39700 1 -- 172.21.15.32:0/288015407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b4150ee0 con 0x7fa1b40ad510 2020-12-05T17:32:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.223+0000 7fa1a6ffd700 1 --2- 172.21.15.32:0/288015407 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa1940554e0 0x7fa1940579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.223+0000 7fa1a6ffd700 1 -- 172.21.15.32:0/288015407 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa1b006f690 con 0x7fa1b40ad510 2020-12-05T17:32:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.223+0000 7fa1b9b37700 1 --2- 172.21.15.32:0/288015407 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa1940554e0 0x7fa1940579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.223+0000 7fa1b9b37700 1 --2- 172.21.15.32:0/288015407 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa1940554e0 0x7fa1940579a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fa1ac00b000 tx=0x7fa1ac0091a0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:26.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.227+0000 7fa1a6ffd700 1 -- 172.21.15.32:0/288015407 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa1b0045830 con 0x7fa1b40ad510 2020-12-05T17:32:26.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.371+0000 7fa1bab39700 1 -- 172.21.15.32:0/288015407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1b4057050 con 0x7fa1b40ad510 2020-12-05T17:32:26.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.371+0000 7fa1a6ffd700 1 -- 172.21.15.32:0/288015407 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16284 (secure 0 0 0) 0x7fa1b0048e50 con 0x7fa1b40ad510 2020-12-05T17:32:26.376 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:32:26.376 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:32:25.792127+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:32:23.957088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-12-05T17:32:24.800777+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:32:26.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.375+0000 7fa1bab39700 1 -- 172.21.15.32:0/288015407 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa1940554e0 msgr2=0x7fa1940579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:26.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.375+0000 7fa1bab39700 1 --2- 172.21.15.32:0/288015407 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa1940554e0 0x7fa1940579a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fa1ac00b000 tx=0x7fa1ac0091a0).stop 2020-12-05T17:32:26.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.375+0000 7fa1bab39700 1 -- 172.21.15.32:0/288015407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1b40ad510 msgr2=0x7fa1b40b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:26.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.375+0000 7fa1bab39700 1 --2- 172.21.15.32:0/288015407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1b40ad510 0x7fa1b40b3d10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fa1b0002f30 tx=0x7fa1b0017000).stop 2020-12-05T17:32:26.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.375+0000 7fa1bab39700 1 -- 172.21.15.32:0/288015407 shutdown_connections 2020-12-05T17:32:26.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.375+0000 7fa1bab39700 1 --2- 172.21.15.32:0/288015407 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1b40b5f00 0x7fa1b41511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.381 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.375+0000 7fa1bab39700 1 --2- 172.21.15.32:0/288015407 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa1940554e0 0x7fa1940579a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.381 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.375+0000 7fa1bab39700 1 --2- 172.21.15.32:0/288015407 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1b4066370 0x7fa1b413e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.375+0000 7fa1bab39700 1 --2- 172.21.15.32:0/288015407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1b40ad510 0x7fa1b40b3d10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.375+0000 7fa1bab39700 1 -- 172.21.15.32:0/288015407 >> 172.21.15.32:0/288015407 conn(0x7fa1b40604c0 msgr2=0x7fa1b405f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:26.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.375+0000 7fa1bab39700 1 -- 172.21.15.32:0/288015407 shutdown_connections 2020-12-05T17:32:26.383 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.375+0000 7fa1bab39700 1 -- 172.21.15.32:0/288015407 wait complete. 2020-12-05T17:32:26.390 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-12-05T17:32:26.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.503+0000 7f6b5ef6f700 1 Processor -- start 2020-12-05T17:32:26.679 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.503+0000 7f6b5ef6f700 1 -- start start 2020-12-05T17:32:26.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b5ef6f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f6b5805fb10 0x7f6b5805ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b5ef6f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b580b0de0 0x7f6b580af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b5ef6f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5805fef0 con 0x7f6b5813fbe0 2020-12-05T17:32:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b5ef6f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b5805d060 con 0x7f6b580b0de0 2020-12-05T17:32:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b5ef6f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b580af080 con 0x7f6b5805fb10 2020-12-05T17:32:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b57fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b580b0de0 0x7f6b580af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b5d50c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6b5813fbe0 0x7f6b58140030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36586/0 (socket says 172.21.15.32:36586) 2020-12-05T17:32:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b5d50c700 1 -- 172.21.15.32:0/3892166817 learned_addr learned my addr 172.21.15.32:0/3892166817 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b5cd0b700 1 --2- 172.21.15.32:0/3892166817 >> v2:172.21.15.61:3300/0 conn(0x7f6b5805fb10 0x7f6b5805ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b57fff700 1 -- 172.21.15.32:0/3892166817 >> v2:172.21.15.61:3300/0 conn(0x7f6b5805fb10 msgr2=0x7f6b5805ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b57fff700 1 --2- 172.21.15.32:0/3892166817 >> v2:172.21.15.61:3300/0 conn(0x7f6b5805fb10 0x7f6b5805ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b57fff700 1 -- 172.21.15.32:0/3892166817 >> v1:172.21.15.32:6789/0 conn(0x7f6b5813fbe0 legacy=0x7f6b58140030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b57fff700 1 -- 172.21.15.32:0/3892166817 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b580af320 con 0x7f6b580b0de0 2020-12-05T17:32:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b57fff700 1 --2- 172.21.15.32:0/3892166817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b580b0de0 0x7f6b580af7c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f6b4c002f30 tx=0x7f6b4c006d60).ready entity=mon.0 client_cookie=b992722c94c491e7 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:26.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b577fe700 1 -- 172.21.15.32:0/3892166817 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b4c009600 con 0x7f6b580b0de0 2020-12-05T17:32:26.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b577fe700 1 -- 172.21.15.32:0/3892166817 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4c009760 con 0x7f6b580b0de0 2020-12-05T17:32:26.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b577fe700 1 -- 172.21.15.32:0/3892166817 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b4c0098d0 con 0x7f6b580b0de0 2020-12-05T17:32:26.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b5ef6f700 1 -- 172.21.15.32:0/3892166817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b580b0de0 msgr2=0x7f6b580af7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:26.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b5ef6f700 1 --2- 172.21.15.32:0/3892166817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b580b0de0 0x7f6b580af7c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f6b4c002f30 tx=0x7f6b4c006d60).stop 2020-12-05T17:32:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b5ef6f700 1 -- 172.21.15.32:0/3892166817 shutdown_connections 2020-12-05T17:32:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b5ef6f700 1 --2- 172.21.15.32:0/3892166817 >> v2:172.21.15.61:3300/0 conn(0x7f6b5805fb10 0x7f6b5805ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b5ef6f700 1 --2- 172.21.15.32:0/3892166817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b580b0de0 0x7f6b580af7c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b5ef6f700 1 -- 172.21.15.32:0/3892166817 >> 172.21.15.32:0/3892166817 conn(0x7f6b58069cb0 msgr2=0x7f6b580604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:26.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b5ef6f700 1 -- 172.21.15.32:0/3892166817 shutdown_connections 2020-12-05T17:32:26.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.507+0000 7f6b5ef6f700 1 -- 172.21.15.32:0/3892166817 wait complete. 2020-12-05T17:32:26.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b5ef6f700 1 Processor -- start 2020-12-05T17:32:26.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b5ef6f700 1 -- start start 2020-12-05T17:32:26.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b5ef6f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6b5805fb10 0x7f6b58065ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:26.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b5ef6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b580b0de0 0x7f6b5810fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:26.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b5ef6f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6b5813fbe0 0x7f6b58063380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:26.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b5ef6f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b58110ab0 con 0x7f6b580b0de0 2020-12-05T17:32:26.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b5ef6f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b58065680 con 0x7f6b5805fb10 2020-12-05T17:32:26.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b5ef6f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b58112010 con 0x7f6b5813fbe0 2020-12-05T17:32:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b57fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b580b0de0 0x7f6b5810fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b57fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b580b0de0 0x7f6b5810fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33942/0 (socket says 172.21.15.32:33942) 2020-12-05T17:32:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b57fff700 1 -- 172.21.15.32:0/1565167824 learned_addr learned my addr 172.21.15.32:0/1565167824 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b5cd0b700 1 --2- 172.21.15.32:0/1565167824 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6b5805fb10 0x7f6b58065ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b5d50c700 1 --2- 172.21.15.32:0/1565167824 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6b5813fbe0 0x7f6b58063380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b57fff700 1 -- 172.21.15.32:0/1565167824 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6b5813fbe0 msgr2=0x7f6b58063380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b57fff700 1 --2- 172.21.15.32:0/1565167824 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6b5813fbe0 0x7f6b58063380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b57fff700 1 -- 172.21.15.32:0/1565167824 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6b5805fb10 msgr2=0x7f6b58065ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b57fff700 1 --2- 172.21.15.32:0/1565167824 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6b5805fb10 0x7f6b58065ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b57fff700 1 -- 172.21.15.32:0/1565167824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b581122b0 con 0x7f6b580b0de0 2020-12-05T17:32:26.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b57fff700 1 --2- 172.21.15.32:0/1565167824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b580b0de0 0x7f6b5810fbd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f6b58064b90 tx=0x7f6b4c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:26.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b55ffb700 1 -- 172.21.15.32:0/1565167824 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b4c0065e0 con 0x7f6b580b0de0 2020-12-05T17:32:26.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b55ffb700 1 -- 172.21.15.32:0/1565167824 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4c0056c0 con 0x7f6b580b0de0 2020-12-05T17:32:26.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b5ef6f700 1 -- 172.21.15.32:0/1565167824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b58152bd0 con 0x7f6b580b0de0 2020-12-05T17:32:26.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b55ffb700 1 -- 172.21.15.32:0/1565167824 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b4c005a20 con 0x7f6b580b0de0 2020-12-05T17:32:26.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.511+0000 7f6b5ef6f700 1 -- 172.21.15.32:0/1565167824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b580663a0 con 0x7f6b580b0de0 2020-12-05T17:32:26.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.515+0000 7f6b5ef6f700 1 -- 172.21.15.32:0/1565167824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b58057050 con 0x7f6b580b0de0 2020-12-05T17:32:26.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.515+0000 7f6b55ffb700 1 -- 172.21.15.32:0/1565167824 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6b4c026030 con 0x7f6b580b0de0 2020-12-05T17:32:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.515+0000 7f6b55ffb700 1 --2- 172.21.15.32:0/1565167824 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6b38055500 0x7f6b380579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.515+0000 7f6b55ffb700 1 -- 172.21.15.32:0/1565167824 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6b4c06e770 con 0x7f6b580b0de0 2020-12-05T17:32:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.515+0000 7f6b5cd0b700 1 --2- 172.21.15.32:0/1565167824 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6b38055500 0x7f6b380579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.515+0000 7f6b5cd0b700 1 --2- 172.21.15.32:0/1565167824 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6b38055500 0x7f6b380579c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f6b4800b000 tx=0x7f6b48006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.523+0000 7f6b55ffb700 1 -- 172.21.15.32:0/1565167824 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6b4c048340 con 0x7f6b580b0de0 2020-12-05T17:32:26.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.711+0000 7f6b5ef6f700 1 -- 172.21.15.32:0/1565167824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6b58069690 con 0x7f6b580b0de0 2020-12-05T17:32:26.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.799+0000 7f6b55ffb700 1 -- 172.21.15.32:0/1565167824 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v10) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6b4c0150b0 con 0x7f6b580b0de0 2020-12-05T17:32:26.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.803+0000 7f6b5ef6f700 1 -- 172.21.15.32:0/1565167824 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6b38055500 msgr2=0x7f6b380579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:26.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.803+0000 7f6b5ef6f700 1 --2- 172.21.15.32:0/1565167824 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6b38055500 0x7f6b380579c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f6b4800b000 tx=0x7f6b48006dc0).stop 2020-12-05T17:32:26.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.803+0000 7f6b5ef6f700 1 -- 172.21.15.32:0/1565167824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b580b0de0 msgr2=0x7f6b5810fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:26.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.803+0000 7f6b5ef6f700 1 --2- 172.21.15.32:0/1565167824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b580b0de0 0x7f6b5810fbd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f6b58064b90 tx=0x7f6b4c016000).stop 2020-12-05T17:32:26.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.803+0000 7f6b5ef6f700 1 -- 172.21.15.32:0/1565167824 shutdown_connections 2020-12-05T17:32:26.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.803+0000 7f6b5ef6f700 1 --2- 172.21.15.32:0/1565167824 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6b38055500 0x7f6b380579c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.803+0000 7f6b5ef6f700 1 --2- 172.21.15.32:0/1565167824 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6b5813fbe0 0x7f6b58063380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.803+0000 7f6b5ef6f700 1 --2- 172.21.15.32:0/1565167824 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6b5805fb10 0x7f6b58065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.803+0000 7f6b5ef6f700 1 --2- 172.21.15.32:0/1565167824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b580b0de0 0x7f6b5810fbd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.803+0000 7f6b5ef6f700 1 -- 172.21.15.32:0/1565167824 >> 172.21.15.32:0/1565167824 conn(0x7f6b58069cb0 msgr2=0x7f6b58142540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:26.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.807+0000 7f6b5ef6f700 1 -- 172.21.15.32:0/1565167824 shutdown_connections 2020-12-05T17:32:26.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.807+0000 7f6b5ef6f700 1 -- 172.21.15.32:0/1565167824 wait complete. 2020-12-05T17:32:26.825 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:32:26.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.931+0000 7f6fff59e700 1 Processor -- start 2020-12-05T17:32:26.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.931+0000 7f6fff59e700 1 -- start start 2020-12-05T17:32:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6fff59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70000b1fc0 0x7f70000af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6fff59e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f70000ab710 0x7f7000061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6fff59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7000146a90 con 0x7f7000147400 2020-12-05T17:32:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6fff59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70000b0110 con 0x7f70000b1fc0 2020-12-05T17:32:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6fff59e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70000b23a0 con 0x7f70000ab710 2020-12-05T17:32:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6ffe59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70000b1fc0 0x7f70000af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6ffe59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70000b1fc0 0x7f70000af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33950/0 (socket says 172.21.15.32:33950) 2020-12-05T17:32:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6ffe59c700 1 -- 172.21.15.32:0/1562425099 learned_addr learned my addr 172.21.15.32:0/1562425099 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:26.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6ff5d9b700 1 --2- 172.21.15.32:0/1562425099 >> v2:172.21.15.61:3300/0 conn(0x7f70000ab710 0x7f7000061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:26.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6ffe59c700 1 -- 172.21.15.32:0/1562425099 >> v2:172.21.15.61:3300/0 conn(0x7f70000ab710 msgr2=0x7f7000061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:26.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6ffe59c700 1 --2- 172.21.15.32:0/1562425099 >> v2:172.21.15.61:3300/0 conn(0x7f70000ab710 0x7f7000061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6ffe59c700 1 -- 172.21.15.32:0/1562425099 >> v1:172.21.15.61:6789/0 conn(0x7f7000147400 legacy=0x7f7000145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:26.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6ffe59c700 1 -- 172.21.15.32:0/1562425099 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7000146cc0 con 0x7f70000b1fc0 2020-12-05T17:32:26.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6ffe59c700 1 --2- 172.21.15.32:0/1562425099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70000b1fc0 0x7f70000af230 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f6ff0002f30 tx=0x7f6ff0006e80).ready entity=mon.0 client_cookie=a4c121fc743abead server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:26.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6ffdd9b700 1 -- 172.21.15.32:0/1562425099 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ff00097f0 con 0x7f70000b1fc0 2020-12-05T17:32:26.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6ffdd9b700 1 -- 172.21.15.32:0/1562425099 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff0009950 con 0x7f70000b1fc0 2020-12-05T17:32:26.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6ffdd9b700 1 -- 172.21.15.32:0/1562425099 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ff0009cb0 con 0x7f70000b1fc0 2020-12-05T17:32:26.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6fff59e700 1 -- 172.21.15.32:0/1562425099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70000b1fc0 msgr2=0x7f70000af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:26.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6fff59e700 1 --2- 172.21.15.32:0/1562425099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70000b1fc0 0x7f70000af230 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f6ff0002f30 tx=0x7f6ff0006e80).stop 2020-12-05T17:32:26.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6fff59e700 1 -- 172.21.15.32:0/1562425099 shutdown_connections 2020-12-05T17:32:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6fff59e700 1 --2- 172.21.15.32:0/1562425099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70000b1fc0 0x7f70000af230 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6fff59e700 1 --2- 172.21.15.32:0/1562425099 >> v2:172.21.15.61:3300/0 conn(0x7f70000ab710 0x7f7000061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6fff59e700 1 -- 172.21.15.32:0/1562425099 >> 172.21.15.32:0/1562425099 conn(0x7f70000b5530 msgr2=0x7f70000b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6fff59e700 1 -- 172.21.15.32:0/1562425099 shutdown_connections 2020-12-05T17:32:26.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6fff59e700 1 -- 172.21.15.32:0/1562425099 wait complete. 2020-12-05T17:32:26.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.935+0000 7f6fff59e700 1 Processor -- start 2020-12-05T17:32:26.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6fff59e700 1 -- start start 2020-12-05T17:32:26.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6fff59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70000ab710 0x7f7000141e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:26.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6fff59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f70000b1fc0 0x7f7000147e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:26.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6fff59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7000147400 0x7f7000148340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:26.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6fff59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7000149ee0 con 0x7f70000ab710 2020-12-05T17:32:26.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6fff59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700014ad30 con 0x7f70000b1fc0 2020-12-05T17:32:26.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6fff59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7000149c40 con 0x7f7000147400 2020-12-05T17:32:26.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6ffe59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70000ab710 0x7f7000141e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:26.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6ffe59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70000ab710 0x7f7000141e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33954/0 (socket says 172.21.15.32:33954) 2020-12-05T17:32:26.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6ffe59c700 1 -- 172.21.15.32:0/3917635193 learned_addr learned my addr 172.21.15.32:0/3917635193 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:26.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6ff5d9b700 1 --2- 172.21.15.32:0/3917635193 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f70000b1fc0 0x7f7000147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:26.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6ffed9d700 1 --2- 172.21.15.32:0/3917635193 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7000147400 0x7f7000148340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:26.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6ffe59c700 1 -- 172.21.15.32:0/3917635193 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7000147400 msgr2=0x7f7000148340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:26.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6ffe59c700 1 --2- 172.21.15.32:0/3917635193 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7000147400 0x7f7000148340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6ffe59c700 1 -- 172.21.15.32:0/3917635193 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f70000b1fc0 msgr2=0x7f7000147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:26.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6ffe59c700 1 --2- 172.21.15.32:0/3917635193 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f70000b1fc0 0x7f7000147e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:26.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6ffe59c700 1 -- 172.21.15.32:0/3917635193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700014aa90 con 0x7f70000ab710 2020-12-05T17:32:26.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6ff5d9b700 1 --2- 172.21.15.32:0/3917635193 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f70000b1fc0 0x7f7000147e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:26.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6ffe59c700 1 --2- 172.21.15.32:0/3917635193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70000ab710 0x7f7000141e20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f700014aec0 tx=0x7f6ff0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:26.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6ff7fff700 1 -- 172.21.15.32:0/3917635193 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ff0011ca0 con 0x7f70000ab710 2020-12-05T17:32:26.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6ff7fff700 1 -- 172.21.15.32:0/3917635193 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff0011e00 con 0x7f70000ab710 2020-12-05T17:32:26.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6fff59e700 1 -- 172.21.15.32:0/3917635193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70001415d0 con 0x7f70000ab710 2020-12-05T17:32:26.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6ff7fff700 1 -- 172.21.15.32:0/3917635193 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ff001e650 con 0x7f70000ab710 2020-12-05T17:32:26.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6fff59e700 1 -- 172.21.15.32:0/3917635193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7000149860 con 0x7f70000ab710 2020-12-05T17:32:26.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.939+0000 7f6fff59e700 1 -- 172.21.15.32:0/3917635193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7000057050 con 0x7f70000ab710 2020-12-05T17:32:26.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.943+0000 7f6ff7fff700 1 -- 172.21.15.32:0/3917635193 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6ff001a040 con 0x7f70000ab710 2020-12-05T17:32:26.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.943+0000 7f6ff7fff700 1 --2- 172.21.15.32:0/3917635193 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6fe4055330 0x7f6fe40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:26.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.943+0000 7f6ff7fff700 1 -- 172.21.15.32:0/3917635193 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6ff006f0d0 con 0x7f70000ab710 2020-12-05T17:32:26.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.943+0000 7f6ff5d9b700 1 --2- 172.21.15.32:0/3917635193 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6fe4055330 0x7f6fe40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:26.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.943+0000 7f6ff5d9b700 1 --2- 172.21.15.32:0/3917635193 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6fe4055330 0x7f6fe40577f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f6fe800b000 tx=0x7f6fe8008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:26.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:26.947+0000 7f6ff7fff700 1 -- 172.21.15.32:0/3917635193 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6ff00453b0 con 0x7f70000ab710 2020-12-05T17:32:27.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.131+0000 7f6fff59e700 1 -- 172.21.15.32:0/3917635193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70000ae950 con 0x7f70000ab710 2020-12-05T17:32:27.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.131+0000 7f6ff7fff700 1 -- 172.21.15.32:0/3917635193 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2136 (secure 0 0 0) 0x7f6ff0023020 con 0x7f70000ab710 2020-12-05T17:32:27.135 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:32:27.135 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4679,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/2850341272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2850341272},{"type":"v1","addr":"172.21.15.61:6833","nonce":2850341272}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:32:25.792490+0000","modified":"2020-12-05T17:32:26.797858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4873},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4873":{"gid":4873,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3316547885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3316547885},{"type":"v1","addr":"172.21.15.32:6835","nonce":3316547885}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-12-05T17:32:27.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.135+0000 7f6fff59e700 1 -- 172.21.15.32:0/3917635193 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6fe4055330 msgr2=0x7f6fe40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:27.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.135+0000 7f6fff59e700 1 --2- 172.21.15.32:0/3917635193 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6fe4055330 0x7f6fe40577f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f6fe800b000 tx=0x7f6fe8008040).stop 2020-12-05T17:32:27.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.135+0000 7f6fff59e700 1 -- 172.21.15.32:0/3917635193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70000ab710 msgr2=0x7f7000141e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:27.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.135+0000 7f6fff59e700 1 --2- 172.21.15.32:0/3917635193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70000ab710 0x7f7000141e20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f700014aec0 tx=0x7f6ff0015000).stop 2020-12-05T17:32:27.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.135+0000 7f6fff59e700 1 -- 172.21.15.32:0/3917635193 shutdown_connections 2020-12-05T17:32:27.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.135+0000 7f6fff59e700 1 --2- 172.21.15.32:0/3917635193 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6fe4055330 0x7f6fe40577f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.135+0000 7f6fff59e700 1 --2- 172.21.15.32:0/3917635193 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7000147400 0x7f7000148340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.135+0000 7f6fff59e700 1 --2- 172.21.15.32:0/3917635193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70000ab710 0x7f7000141e20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.135+0000 7f6fff59e700 1 --2- 172.21.15.32:0/3917635193 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f70000b1fc0 0x7f7000147e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.135+0000 7f6fff59e700 1 -- 172.21.15.32:0/3917635193 >> 172.21.15.32:0/3917635193 conn(0x7f70000b5530 msgr2=0x7f7000061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:27.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.135+0000 7f6fff59e700 1 -- 172.21.15.32:0/3917635193 shutdown_connections 2020-12-05T17:32:27.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.135+0000 7f6fff59e700 1 -- 172.21.15.32:0/3917635193 wait complete. 2020-12-05T17:32:27.140 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 10 2020-12-05T17:32:27.149 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:32:27.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.255+0000 7fb3f8832700 1 Processor -- start 2020-12-05T17:32:27.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.255+0000 7fb3f8832700 1 -- start start 2020-12-05T17:32:27.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.255+0000 7fb3f8832700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3f40b54b0 0x7fb3f40b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:27.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.255+0000 7fb3f8832700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3f4141ad0 0x7fb3f4141f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:27.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.255+0000 7fb3f8832700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f40adfe0 con 0x7fb3f40afd20 2020-12-05T17:32:27.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.255+0000 7fb3f8832700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f40b0630 con 0x7fb3f40b54b0 2020-12-05T17:32:27.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.255+0000 7fb3f8832700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f40ae210 con 0x7fb3f4141ad0 2020-12-05T17:32:27.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.255+0000 7fb3f37fe700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3f40b54b0 0x7fb3f40b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:27.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.255+0000 7fb3f37fe700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3f40b54b0 0x7fb3f40b0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:53854/0 (socket says 172.21.15.32:53854) 2020-12-05T17:32:27.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.255+0000 7fb3f37fe700 1 -- 172.21.15.32:0/379939805 learned_addr learned my addr 172.21.15.32:0/379939805 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:27.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.255+0000 7fb3f2ffd700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fb3f40afd20 0x7fb3f40b0f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:49252/0 (socket says 172.21.15.32:49252) 2020-12-05T17:32:27.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.255+0000 7fb3f3fff700 1 --2- 172.21.15.32:0/379939805 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3f4141ad0 0x7fb3f4141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:27.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.255+0000 7fb3f37fe700 1 -- 172.21.15.32:0/379939805 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3f4141ad0 msgr2=0x7fb3f4141f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:27.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.255+0000 7fb3f37fe700 1 --2- 172.21.15.32:0/379939805 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3f4141ad0 0x7fb3f4141f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.255+0000 7fb3f37fe700 1 -- 172.21.15.32:0/379939805 >> v1:172.21.15.61:6789/0 conn(0x7fb3f40afd20 legacy=0x7fb3f40b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:27.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.255+0000 7fb3f27fc700 1 -- 172.21.15.32:0/379939805 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2070520287 0 0) 0x7fb3f40adfe0 con 0x7fb3f40afd20 2020-12-05T17:32:27.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.255+0000 7fb3f37fe700 1 -- 172.21.15.32:0/379939805 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f40677f0 con 0x7fb3f40b54b0 2020-12-05T17:32:27.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.255+0000 7fb3f3fff700 1 --2- 172.21.15.32:0/379939805 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3f4141ad0 0x7fb3f4141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:32:27.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.255+0000 7fb3f37fe700 1 --2- 172.21.15.32:0/379939805 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3f40b54b0 0x7fb3f40b0a90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fb3e40032b0 tx=0x7fb3e4009270).ready entity=mon.1 client_cookie=c1c06381748b1d84 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:27.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.259+0000 7fb3f27fc700 1 -- 172.21.15.32:0/379939805 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3e4009e60 con 0x7fb3f40b54b0 2020-12-05T17:32:27.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.259+0000 7fb3f27fc700 1 -- 172.21.15.32:0/379939805 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e400bc00 con 0x7fb3f40b54b0 2020-12-05T17:32:27.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.259+0000 7fb3f27fc700 1 -- 172.21.15.32:0/379939805 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3e400bd70 con 0x7fb3f40b54b0 2020-12-05T17:32:27.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.259+0000 7fb3f8832700 1 -- 172.21.15.32:0/379939805 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3f40b54b0 msgr2=0x7fb3f40b0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:27.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.259+0000 7fb3f8832700 1 --2- 172.21.15.32:0/379939805 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3f40b54b0 0x7fb3f40b0a90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fb3e40032b0 tx=0x7fb3e4009270).stop 2020-12-05T17:32:27.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.259+0000 7fb3f8832700 1 -- 172.21.15.32:0/379939805 shutdown_connections 2020-12-05T17:32:27.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.259+0000 7fb3f8832700 1 --2- 172.21.15.32:0/379939805 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3f4141ad0 0x7fb3f4141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.259+0000 7fb3f8832700 1 --2- 172.21.15.32:0/379939805 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3f40b54b0 0x7fb3f40b0a90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.259+0000 7fb3f8832700 1 -- 172.21.15.32:0/379939805 >> 172.21.15.32:0/379939805 conn(0x7fb3f405bbb0 msgr2=0x7fb3f4062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:27.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.259+0000 7fb3f8832700 1 -- 172.21.15.32:0/379939805 shutdown_connections 2020-12-05T17:32:27.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.259+0000 7fb3f8832700 1 -- 172.21.15.32:0/379939805 wait complete. 2020-12-05T17:32:27.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.259+0000 7fb3f8832700 1 Processor -- start 2020-12-05T17:32:27.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.259+0000 7fb3f8832700 1 -- start start 2020-12-05T17:32:27.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.259+0000 7fb3f8832700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3f40afd20 0x7fb3f40b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:27.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.259+0000 7fb3f8832700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3f40b54b0 0x7fb3f4145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:27.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.259+0000 7fb3f8832700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3f4141ad0 0x7fb3f41513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:27.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.259+0000 7fb3f8832700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f4145610 con 0x7fb3f40afd20 2020-12-05T17:32:27.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.259+0000 7fb3f8832700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f41458b0 con 0x7fb3f40b54b0 2020-12-05T17:32:27.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.259+0000 7fb3f8832700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f4145b50 con 0x7fb3f4141ad0 2020-12-05T17:32:27.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.263+0000 7fb3f37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3f40afd20 0x7fb3f40b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:27.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.263+0000 7fb3f37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3f40afd20 0x7fb3f40b3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33970/0 (socket says 172.21.15.32:33970) 2020-12-05T17:32:27.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.263+0000 7fb3f37fe700 1 -- 172.21.15.32:0/2562524788 learned_addr learned my addr 172.21.15.32:0/2562524788 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:27.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.263+0000 7fb3f3fff700 1 --2- 172.21.15.32:0/2562524788 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3f4141ad0 0x7fb3f41513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:27.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.263+0000 7fb3f2ffd700 1 --2- 172.21.15.32:0/2562524788 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3f40b54b0 0x7fb3f4145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:27.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.263+0000 7fb3f37fe700 1 -- 172.21.15.32:0/2562524788 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3f4141ad0 msgr2=0x7fb3f41513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:27.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.263+0000 7fb3f37fe700 1 --2- 172.21.15.32:0/2562524788 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3f4141ad0 0x7fb3f41513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.263+0000 7fb3f37fe700 1 -- 172.21.15.32:0/2562524788 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3f40b54b0 msgr2=0x7fb3f4145ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:27.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.263+0000 7fb3f37fe700 1 --2- 172.21.15.32:0/2562524788 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3f40b54b0 0x7fb3f4145ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.263+0000 7fb3f37fe700 1 -- 172.21.15.32:0/2562524788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f4146b10 con 0x7fb3f40afd20 2020-12-05T17:32:27.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.263+0000 7fb3f37fe700 1 --2- 172.21.15.32:0/2562524788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3f40afd20 0x7fb3f40b3fa0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7fb3e40032b0 tx=0x7fb3e4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:27.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.263+0000 7fb3f0ff9700 1 -- 172.21.15.32:0/2562524788 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3e401e610 con 0x7fb3f40afd20 2020-12-05T17:32:27.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.263+0000 7fb3f0ff9700 1 -- 172.21.15.32:0/2562524788 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e401e770 con 0x7fb3f40afd20 2020-12-05T17:32:27.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.263+0000 7fb3f8832700 1 -- 172.21.15.32:0/2562524788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f41464e0 con 0x7fb3f40afd20 2020-12-05T17:32:27.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.263+0000 7fb3f0ff9700 1 -- 172.21.15.32:0/2562524788 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3e401ead0 con 0x7fb3f40afd20 2020-12-05T17:32:27.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.263+0000 7fb3f8832700 1 -- 172.21.15.32:0/2562524788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f4151ae0 con 0x7fb3f40afd20 2020-12-05T17:32:27.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.263+0000 7fb3f8832700 1 -- 172.21.15.32:0/2562524788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f4057050 con 0x7fb3f40afd20 2020-12-05T17:32:27.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.263+0000 7fb3f0ff9700 1 -- 172.21.15.32:0/2562524788 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb3e400f070 con 0x7fb3f40afd20 2020-12-05T17:32:27.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.267+0000 7fb3f0ff9700 1 --2- 172.21.15.32:0/2562524788 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3d40554e0 0x7fb3d40579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:27.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.267+0000 7fb3f0ff9700 1 -- 172.21.15.32:0/2562524788 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb3e406f520 con 0x7fb3f40afd20 2020-12-05T17:32:27.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.267+0000 7fb3f2ffd700 1 --2- 172.21.15.32:0/2562524788 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3d40554e0 0x7fb3d40579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:27.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.267+0000 7fb3f2ffd700 1 --2- 172.21.15.32:0/2562524788 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3d40554e0 0x7fb3d40579a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fb3e800d000 tx=0x7fb3e8007a40).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:27.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.271+0000 7fb3f0ff9700 1 -- 172.21.15.32:0/2562524788 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb3e40456c0 con 0x7fb3f40afd20 2020-12-05T17:32:27.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.411+0000 7fb3f8832700 1 -- 172.21.15.32:0/2562524788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3f405c300 con 0x7fb3f40afd20 2020-12-05T17:32:27.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.411+0000 7fb3f0ff9700 1 -- 172.21.15.32:0/2562524788 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16271 (secure 0 0 0) 0x7fb3e4048ce0 con 0x7fb3f40afd20 2020-12-05T17:32:27.416 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:32:27.416 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:32:26.798545+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:32:23.957088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-12-05T17:32:24.800777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:32:27.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.415+0000 7fb3f8832700 1 -- 172.21.15.32:0/2562524788 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3d40554e0 msgr2=0x7fb3d40579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:27.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.415+0000 7fb3f8832700 1 --2- 172.21.15.32:0/2562524788 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3d40554e0 0x7fb3d40579a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fb3e800d000 tx=0x7fb3e8007a40).stop 2020-12-05T17:32:27.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.415+0000 7fb3f8832700 1 -- 172.21.15.32:0/2562524788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3f40afd20 msgr2=0x7fb3f40b3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:27.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.415+0000 7fb3f8832700 1 --2- 172.21.15.32:0/2562524788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3f40afd20 0x7fb3f40b3fa0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7fb3e40032b0 tx=0x7fb3e4016000).stop 2020-12-05T17:32:27.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.415+0000 7fb3f8832700 1 -- 172.21.15.32:0/2562524788 shutdown_connections 2020-12-05T17:32:27.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.415+0000 7fb3f8832700 1 --2- 172.21.15.32:0/2562524788 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3d40554e0 0x7fb3d40579a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.415+0000 7fb3f8832700 1 --2- 172.21.15.32:0/2562524788 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3f4141ad0 0x7fb3f41513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.415+0000 7fb3f8832700 1 --2- 172.21.15.32:0/2562524788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3f40afd20 0x7fb3f40b3fa0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.415+0000 7fb3f8832700 1 --2- 172.21.15.32:0/2562524788 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3f40b54b0 0x7fb3f4145ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.415+0000 7fb3f8832700 1 -- 172.21.15.32:0/2562524788 >> 172.21.15.32:0/2562524788 conn(0x7fb3f405bbb0 msgr2=0x7fb3f40b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:27.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.415+0000 7fb3f8832700 1 -- 172.21.15.32:0/2562524788 shutdown_connections 2020-12-05T17:32:27.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.415+0000 7fb3f8832700 1 -- 172.21.15.32:0/2562524788 wait complete. 2020-12-05T17:32:27.431 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:32:27.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.551+0000 7f6be759e700 1 Processor -- start 2020-12-05T17:32:27.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.551+0000 7f6be759e700 1 -- start start 2020-12-05T17:32:27.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be759e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6be80611b0 0x7f6be8153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:27.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6be8080a90 con 0x7f6be80b0240 2020-12-05T17:32:27.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be759e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6be8152860 con 0x7f6be80ae860 2020-12-05T17:32:27.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be759e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be80aecb0 con 0x7f6be80611b0 2020-12-05T17:32:27.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be659c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6be80b0240 0x7f6be814c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36624/0 (socket says 172.21.15.32:36624) 2020-12-05T17:32:27.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be659c700 1 -- 172.21.15.32:0/4271467552 learned_addr learned my addr 172.21.15.32:0/4271467552 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:27.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be6d9d700 1 --2- 172.21.15.32:0/4271467552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6be80611b0 0x7f6be8153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:27.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be5d9b700 1 -- 172.21.15.32:0/4271467552 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1780779717 0 0) 0x7f6be8080a90 con 0x7f6be80b0240 2020-12-05T17:32:27.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be5d9b700 1 -- 172.21.15.32:0/4271467552 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bd4003230 con 0x7f6be80b0240 2020-12-05T17:32:27.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be5d9b700 1 -- 172.21.15.32:0/4271467552 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 175086368 0 0) 0x7f6be8152860 con 0x7f6be80ae860 2020-12-05T17:32:27.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be5d9b700 1 -- 172.21.15.32:0/4271467552 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6be8080a90 con 0x7f6be80ae860 2020-12-05T17:32:27.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be5d9b700 1 -- 172.21.15.32:0/4271467552 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 596922539 0 0) 0x7f6bd4003230 con 0x7f6be80b0240 2020-12-05T17:32:27.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be5d9b700 1 -- 172.21.15.32:0/4271467552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6be80611b0 msgr2=0x7f6be8153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:27.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be5d9b700 1 --2- 172.21.15.32:0/4271467552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6be80611b0 0x7f6be8153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be5d9b700 1 -- 172.21.15.32:0/4271467552 >> v1:172.21.15.61:6789/0 conn(0x7f6be80ae860 legacy=0x7f6be814f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:27.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be5d9b700 1 -- 172.21.15.32:0/4271467552 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be8152b00 con 0x7f6be80b0240 2020-12-05T17:32:27.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be5d9b700 1 -- 172.21.15.32:0/4271467552 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f6bd00016d0 con 0x7f6be80b0240 2020-12-05T17:32:27.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be6d9d700 1 --2- 172.21.15.32:0/4271467552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6be80611b0 0x7f6be8153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:32:27.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be5d9b700 1 -- 172.21.15.32:0/4271467552 >> v1:172.21.15.32:6789/0 conn(0x7f6be80b0240 legacy=0x7f6be814c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:27.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be5d9b700 1 --2- 172.21.15.32:0/4271467552 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6bd40038e0 0x7f6bd4003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:27.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be5d9b700 1 --2- 172.21.15.32:0/4271467552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bd40044d0 0x7f6bd4004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:27.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be5d9b700 1 --2- 172.21.15.32:0/4271467552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6be80611b0 0x7f6bd4004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:27.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be5d9b700 1 -- 172.21.15.32:0/4271467552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be80aecb0 con 0x7f6bd40044d0 2020-12-05T17:32:27.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be5d9b700 1 -- 172.21.15.32:0/4271467552 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd4005330 con 0x7f6bd40038e0 2020-12-05T17:32:27.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be5d9b700 1 -- 172.21.15.32:0/4271467552 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd404f760 con 0x7f6be80611b0 2020-12-05T17:32:27.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be6d9d700 1 --2- 172.21.15.32:0/4271467552 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6bd40038e0 0x7f6bd4003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:27.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be659c700 1 --2- 172.21.15.32:0/4271467552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bd40044d0 0x7f6bd4004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:27.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6bece5c700 1 --2- 172.21.15.32:0/4271467552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6be80611b0 0x7f6bd4004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:27.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be6d9d700 1 -- 172.21.15.32:0/4271467552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6be80611b0 msgr2=0x7f6bd4004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:27.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be6d9d700 1 --2- 172.21.15.32:0/4271467552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6be80611b0 0x7f6bd4004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be6d9d700 1 -- 172.21.15.32:0/4271467552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bd40044d0 msgr2=0x7f6bd4004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:27.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be6d9d700 1 --2- 172.21.15.32:0/4271467552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bd40044d0 0x7f6bd4004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be6d9d700 1 -- 172.21.15.32:0/4271467552 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd4003230 con 0x7f6bd40038e0 2020-12-05T17:32:27.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be6d9d700 1 --2- 172.21.15.32:0/4271467552 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6bd40038e0 0x7f6bd4003ce0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f6bd800c000 tx=0x7f6bd80091a0).ready entity=mon.1 client_cookie=4b83e79dd3a821bb server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:27.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be5d9b700 1 -- 172.21.15.32:0/4271467552 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bd800ed60 con 0x7f6bd40038e0 2020-12-05T17:32:27.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be5d9b700 1 -- 172.21.15.32:0/4271467552 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd80123f0 con 0x7f6bd40038e0 2020-12-05T17:32:27.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be5d9b700 1 -- 172.21.15.32:0/4271467552 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bd8012750 con 0x7f6bd40038e0 2020-12-05T17:32:27.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be759e700 1 -- 172.21.15.32:0/4271467552 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6bd40038e0 msgr2=0x7f6bd4003ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:27.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6be759e700 1 --2- 172.21.15.32:0/4271467552 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6bd40038e0 0x7f6bd4003ce0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f6bd800c000 tx=0x7f6bd80091a0).stop 2020-12-05T17:32:27.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.555+0000 7f6bece5c700 1 -- 172.21.15.32:0/4271467552 reap_dead start 2020-12-05T17:32:27.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.559+0000 7f6be759e700 1 -- 172.21.15.32:0/4271467552 shutdown_connections 2020-12-05T17:32:27.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.559+0000 7f6be759e700 1 -- 172.21.15.32:0/4271467552 >> 172.21.15.32:0/4271467552 conn(0x7f6be80b5f90 msgr2=0x7f6be80b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:27.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.559+0000 7f6be759e700 1 -- 172.21.15.32:0/4271467552 shutdown_connections 2020-12-05T17:32:27.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.559+0000 7f6be759e700 1 -- 172.21.15.32:0/4271467552 wait complete. 2020-12-05T17:32:27.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.559+0000 7f6be759e700 1 Processor -- start 2020-12-05T17:32:27.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.559+0000 7f6be759e700 1 -- start start 2020-12-05T17:32:27.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6be759e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6be80b2460 0x7f6be8070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:27.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6be759e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6be807f8a0 0x7f6be813b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:27.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6be759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6be813db30 0x7f6be813dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:27.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6be759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be813a540 con 0x7f6be813db30 2020-12-05T17:32:27.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6be759e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be813a730 con 0x7f6be807f8a0 2020-12-05T17:32:27.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6be759e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be813a9d0 con 0x7f6be80b2460 2020-12-05T17:32:27.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6bece5c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6be813db30 0x7f6be813dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:27.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6be6d9d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6be80b2460 0x7f6be8070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:27.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6be659c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6be807f8a0 0x7f6be813b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:27.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6bece5c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6be813db30 0x7f6be813dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33992/0 (socket says 172.21.15.32:33992) 2020-12-05T17:32:27.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6bece5c700 1 -- 172.21.15.32:0/1830018043 learned_addr learned my addr 172.21.15.32:0/1830018043 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:27.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6be6d9d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6be80b2460 0x7f6be8070b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:59010/0 (socket says 172.21.15.32:59010) 2020-12-05T17:32:27.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6bece5c700 1 -- 172.21.15.32:0/1830018043 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6be80b2460 msgr2=0x7f6be8070b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:27.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6bece5c700 1 --2- 172.21.15.32:0/1830018043 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6be80b2460 0x7f6be8070b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6bece5c700 1 -- 172.21.15.32:0/1830018043 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6be807f8a0 msgr2=0x7f6be813b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:27.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6bece5c700 1 --2- 172.21.15.32:0/1830018043 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6be807f8a0 0x7f6be813b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6bece5c700 1 -- 172.21.15.32:0/1830018043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be813ac70 con 0x7f6be813db30 2020-12-05T17:32:27.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6be659c700 1 --2- 172.21.15.32:0/1830018043 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6be807f8a0 0x7f6be813b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:32:27.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6bece5c700 1 --2- 172.21.15.32:0/1830018043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6be813db30 0x7f6be813dfb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f6be813a8f0 tx=0x7f6be0007b60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:27.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6bdf7fe700 1 -- 172.21.15.32:0/1830018043 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6be0011850 con 0x7f6be813db30 2020-12-05T17:32:27.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6bdf7fe700 1 -- 172.21.15.32:0/1830018043 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be00119b0 con 0x7f6be813db30 2020-12-05T17:32:27.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6bdf7fe700 1 -- 172.21.15.32:0/1830018043 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6be0011d10 con 0x7f6be813db30 2020-12-05T17:32:27.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6be759e700 1 -- 172.21.15.32:0/1830018043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be8071000 con 0x7f6be813db30 2020-12-05T17:32:27.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.563+0000 7f6be759e700 1 -- 172.21.15.32:0/1830018043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be80b2840 con 0x7f6be813db30 2020-12-05T17:32:27.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.567+0000 7f6be759e700 1 -- 172.21.15.32:0/1830018043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be813ae00 con 0x7f6be813db30 2020-12-05T17:32:27.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.567+0000 7f6bdf7fe700 1 -- 172.21.15.32:0/1830018043 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6be00133b0 con 0x7f6be813db30 2020-12-05T17:32:27.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.567+0000 7f6bdf7fe700 1 --2- 172.21.15.32:0/1830018043 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6bcc055320 0x7f6bcc0577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:27.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.567+0000 7f6bdf7fe700 1 -- 172.21.15.32:0/1830018043 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6be006db80 con 0x7f6be813db30 2020-12-05T17:32:27.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.567+0000 7f6be6d9d700 1 --2- 172.21.15.32:0/1830018043 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6bcc055320 0x7f6bcc0577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:27.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.567+0000 7f6be6d9d700 1 --2- 172.21.15.32:0/1830018043 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6bcc055320 0x7f6bcc0577e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f6bd801d000 tx=0x7f6bd801a000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:27.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.575+0000 7f6bdf7fe700 1 -- 172.21.15.32:0/1830018043 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6be0043b30 con 0x7f6be813db30 2020-12-05T17:32:27.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.763+0000 7f6be759e700 1 -- 172.21.15.32:0/1830018043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f6be8057050 con 0x7f6be813db30 2020-12-05T17:32:27.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.771+0000 7f6bdf7fe700 1 -- 172.21.15.32:0/1830018043 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v11) v1 ==== 184+0+0 (secure 0 0 0) 0x7f6be0047150 con 0x7f6be813db30 2020-12-05T17:32:27.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.771+0000 7f6be759e700 1 -- 172.21.15.32:0/1830018043 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6bcc055320 msgr2=0x7f6bcc0577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:27.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.771+0000 7f6be759e700 1 --2- 172.21.15.32:0/1830018043 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6bcc055320 0x7f6bcc0577e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f6bd801d000 tx=0x7f6bd801a000).stop 2020-12-05T17:32:27.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.771+0000 7f6be759e700 1 -- 172.21.15.32:0/1830018043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6be813db30 msgr2=0x7f6be813dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:27.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.771+0000 7f6be759e700 1 --2- 172.21.15.32:0/1830018043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6be813db30 0x7f6be813dfb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f6be813a8f0 tx=0x7f6be0007b60).stop 2020-12-05T17:32:27.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.771+0000 7f6be759e700 1 -- 172.21.15.32:0/1830018043 shutdown_connections 2020-12-05T17:32:27.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.771+0000 7f6be759e700 1 --2- 172.21.15.32:0/1830018043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6be813db30 0x7f6be813dfb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.771+0000 7f6be759e700 1 --2- 172.21.15.32:0/1830018043 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6bcc055320 0x7f6bcc0577e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.771+0000 7f6be759e700 1 --2- 172.21.15.32:0/1830018043 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6be80b2460 0x7f6be8070b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.771+0000 7f6be759e700 1 --2- 172.21.15.32:0/1830018043 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6be807f8a0 0x7f6be813b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.771+0000 7f6be759e700 1 -- 172.21.15.32:0/1830018043 >> 172.21.15.32:0/1830018043 conn(0x7f6be80b5f90 msgr2=0x7f6be80b5790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:27.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.771+0000 7f6be759e700 1 -- 172.21.15.32:0/1830018043 shutdown_connections 2020-12-05T17:32:27.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.775+0000 7f6be759e700 1 -- 172.21.15.32:0/1830018043 wait complete. 2020-12-05T17:32:27.780 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2020-12-05T17:32:27.789 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:32:27.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.915+0000 7f0b2ffff700 1 Processor -- start 2020-12-05T17:32:27.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.915+0000 7f0b2ffff700 1 -- start start 2020-12-05T17:32:27.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.915+0000 7f0b2ffff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0b300611b0 0x7f0b30153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:27.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.915+0000 7f0b2ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b300ae860 0x7f0b3014f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:27.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.915+0000 7f0b2ffff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b30080a90 con 0x7f0b300b0240 2020-12-05T17:32:27.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.915+0000 7f0b2ffff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b300aec40 con 0x7f0b300ae860 2020-12-05T17:32:27.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.915+0000 7f0b2ffff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b30152860 con 0x7f0b300611b0 2020-12-05T17:32:27.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.915+0000 7f0b3558a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b300ae860 0x7f0b3014f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:27.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.915+0000 7f0b26ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0b300b0240 0x7f0b3014c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36646/0 (socket says 172.21.15.32:36646) 2020-12-05T17:32:27.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.915+0000 7f0b26ffd700 1 -- 172.21.15.32:0/1335488455 learned_addr learned my addr 172.21.15.32:0/1335488455 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:27.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.915+0000 7f0b2f7fe700 1 --2- 172.21.15.32:0/1335488455 >> v2:172.21.15.61:3300/0 conn(0x7f0b300611b0 0x7f0b30153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:27.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.915+0000 7f0b3558a700 1 -- 172.21.15.32:0/1335488455 >> v2:172.21.15.61:3300/0 conn(0x7f0b300611b0 msgr2=0x7f0b30153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:27.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.915+0000 7f0b3558a700 1 --2- 172.21.15.32:0/1335488455 >> v2:172.21.15.61:3300/0 conn(0x7f0b300611b0 0x7f0b30153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.915+0000 7f0b3558a700 1 -- 172.21.15.32:0/1335488455 >> v1:172.21.15.32:6789/0 conn(0x7f0b300b0240 legacy=0x7f0b3014c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:27.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.915+0000 7f0b2effd700 1 -- 172.21.15.32:0/1335488455 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2556930880 0 0) 0x7f0b30080a90 con 0x7f0b300b0240 2020-12-05T17:32:27.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.915+0000 7f0b3558a700 1 -- 172.21.15.32:0/1335488455 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b30152b00 con 0x7f0b300ae860 2020-12-05T17:32:27.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.915+0000 7f0b3558a700 1 --2- 172.21.15.32:0/1335488455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b300ae860 0x7f0b3014f0d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f0b280032b0 tx=0x7f0b280091a0).ready entity=mon.0 client_cookie=db1b556cd2a0e9bf server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:27.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.915+0000 7f0b2effd700 1 -- 172.21.15.32:0/1335488455 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b2800b9a0 con 0x7f0b300ae860 2020-12-05T17:32:27.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.915+0000 7f0b2effd700 1 -- 172.21.15.32:0/1335488455 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2800bb00 con 0x7f0b300ae860 2020-12-05T17:32:27.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.915+0000 7f0b2effd700 1 -- 172.21.15.32:0/1335488455 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b2800bc70 con 0x7f0b300ae860 2020-12-05T17:32:27.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.919+0000 7f0b2ffff700 1 -- 172.21.15.32:0/1335488455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b300ae860 msgr2=0x7f0b3014f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:27.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.919+0000 7f0b2ffff700 1 --2- 172.21.15.32:0/1335488455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b300ae860 0x7f0b3014f0d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f0b280032b0 tx=0x7f0b280091a0).stop 2020-12-05T17:32:27.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.919+0000 7f0b2ffff700 1 -- 172.21.15.32:0/1335488455 shutdown_connections 2020-12-05T17:32:27.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.919+0000 7f0b2ffff700 1 --2- 172.21.15.32:0/1335488455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b300ae860 0x7f0b3014f0d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.919+0000 7f0b2ffff700 1 --2- 172.21.15.32:0/1335488455 >> v2:172.21.15.61:3300/0 conn(0x7f0b300611b0 0x7f0b30153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.919+0000 7f0b2ffff700 1 -- 172.21.15.32:0/1335488455 >> 172.21.15.32:0/1335488455 conn(0x7f0b300b5f90 msgr2=0x7f0b300b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:27.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.919+0000 7f0b2ffff700 1 -- 172.21.15.32:0/1335488455 shutdown_connections 2020-12-05T17:32:27.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.919+0000 7f0b2ffff700 1 -- 172.21.15.32:0/1335488455 wait complete. 2020-12-05T17:32:27.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.919+0000 7f0b2ffff700 1 Processor -- start 2020-12-05T17:32:27.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.919+0000 7f0b2ffff700 1 -- start start 2020-12-05T17:32:27.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.919+0000 7f0b2ffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b300611b0 0x7f0b300b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:27.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.919+0000 7f0b2ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b300ae860 0x7f0b30155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:27.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.919+0000 7f0b2ffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b300b0240 0x7f0b300b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:27.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.919+0000 7f0b2ffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b30066e50 con 0x7f0b300ae860 2020-12-05T17:32:27.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.919+0000 7f0b2ffff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b301558c0 con 0x7f0b300611b0 2020-12-05T17:32:27.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.919+0000 7f0b2ffff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b301555f0 con 0x7f0b300b0240 2020-12-05T17:32:27.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.923+0000 7f0b2f7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b300611b0 0x7f0b300b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:27.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.923+0000 7f0b26ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b300ae860 0x7f0b30155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:27.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.923+0000 7f0b2f7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b300611b0 0x7f0b300b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:53894/0 (socket says 172.21.15.32:53894) 2020-12-05T17:32:27.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.923+0000 7f0b2f7fe700 1 -- 172.21.15.32:0/3757124913 learned_addr learned my addr 172.21.15.32:0/3757124913 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:27.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.923+0000 7f0b3558a700 1 --2- 172.21.15.32:0/3757124913 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b300b0240 0x7f0b300b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:27.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.923+0000 7f0b26ffd700 1 -- 172.21.15.32:0/3757124913 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b300b0240 msgr2=0x7f0b300b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:27.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.923+0000 7f0b26ffd700 1 --2- 172.21.15.32:0/3757124913 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b300b0240 0x7f0b300b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.923+0000 7f0b26ffd700 1 -- 172.21.15.32:0/3757124913 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b300611b0 msgr2=0x7f0b300b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:27.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.923+0000 7f0b26ffd700 1 --2- 172.21.15.32:0/3757124913 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b300611b0 0x7f0b300b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:27.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.923+0000 7f0b26ffd700 1 -- 172.21.15.32:0/3757124913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b300b4450 con 0x7f0b300ae860 2020-12-05T17:32:27.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.923+0000 7f0b26ffd700 1 --2- 172.21.15.32:0/3757124913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b300ae860 0x7f0b30155d60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f0b18000fa0 tx=0x7f0b18007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:27.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.923+0000 7f0b2cff9700 1 -- 172.21.15.32:0/3757124913 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b180072e0 con 0x7f0b300ae860 2020-12-05T17:32:27.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.923+0000 7f0b2cff9700 1 -- 172.21.15.32:0/3757124913 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b18007440 con 0x7f0b300ae860 2020-12-05T17:32:27.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.923+0000 7f0b2ffff700 1 -- 172.21.15.32:0/3757124913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b300665c0 con 0x7f0b300ae860 2020-12-05T17:32:27.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.923+0000 7f0b2cff9700 1 -- 172.21.15.32:0/3757124913 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b180057d0 con 0x7f0b300ae860 2020-12-05T17:32:27.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.923+0000 7f0b2ffff700 1 -- 172.21.15.32:0/3757124913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b30066a20 con 0x7f0b300ae860 2020-12-05T17:32:27.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.923+0000 7f0b2ffff700 1 -- 172.21.15.32:0/3757124913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b30155a80 con 0x7f0b300ae860 2020-12-05T17:32:27.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.923+0000 7f0b2cff9700 1 -- 172.21.15.32:0/3757124913 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0b1801a060 con 0x7f0b300ae860 2020-12-05T17:32:27.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.927+0000 7f0b2cff9700 1 --2- 172.21.15.32:0/3757124913 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0b14055440 0x7f0b14057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:27.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.927+0000 7f0b2cff9700 1 -- 172.21.15.32:0/3757124913 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0b1806d780 con 0x7f0b300ae860 2020-12-05T17:32:27.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.927+0000 7f0b2f7fe700 1 --2- 172.21.15.32:0/3757124913 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0b14055440 0x7f0b14057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:27.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.927+0000 7f0b2f7fe700 1 --2- 172.21.15.32:0/3757124913 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0b14055440 0x7f0b14057900 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f0b2000b000 tx=0x7f0b20006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:27.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:27.931+0000 7f0b2cff9700 1 -- 172.21.15.32:0/3757124913 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0b18043920 con 0x7f0b300ae860 2020-12-05T17:32:28.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.123+0000 7f0b2ffff700 1 -- 172.21.15.32:0/3757124913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f0b30057050 con 0x7f0b300ae860 2020-12-05T17:32:28.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.131+0000 7f0b2cff9700 1 -- 172.21.15.32:0/3757124913 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v12) v1 ==== 184+0+0 (secure 0 0 0) 0x7f0b18018020 con 0x7f0b300ae860 2020-12-05T17:32:28.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.135+0000 7f0b2ffff700 1 -- 172.21.15.32:0/3757124913 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0b14055440 msgr2=0x7f0b14057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:28.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.135+0000 7f0b2ffff700 1 --2- 172.21.15.32:0/3757124913 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0b14055440 0x7f0b14057900 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f0b2000b000 tx=0x7f0b20006dc0).stop 2020-12-05T17:32:28.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.135+0000 7f0b2ffff700 1 -- 172.21.15.32:0/3757124913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b300ae860 msgr2=0x7f0b30155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:28.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.135+0000 7f0b2ffff700 1 --2- 172.21.15.32:0/3757124913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b300ae860 0x7f0b30155d60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f0b18000fa0 tx=0x7f0b18007b50).stop 2020-12-05T17:32:28.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.135+0000 7f0b2ffff700 1 -- 172.21.15.32:0/3757124913 shutdown_connections 2020-12-05T17:32:28.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.135+0000 7f0b2ffff700 1 --2- 172.21.15.32:0/3757124913 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0b14055440 0x7f0b14057900 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:28.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.135+0000 7f0b2ffff700 1 --2- 172.21.15.32:0/3757124913 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b300b0240 0x7f0b300b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.135+0000 7f0b2ffff700 1 --2- 172.21.15.32:0/3757124913 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b300611b0 0x7f0b300b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.135+0000 7f0b2ffff700 1 --2- 172.21.15.32:0/3757124913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b300ae860 0x7f0b30155d60 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.135+0000 7f0b2ffff700 1 -- 172.21.15.32:0/3757124913 >> 172.21.15.32:0/3757124913 conn(0x7f0b300b5f90 msgr2=0x7f0b300b6370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.135+0000 7f0b2ffff700 1 -- 172.21.15.32:0/3757124913 shutdown_connections 2020-12-05T17:32:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.135+0000 7f0b2ffff700 1 -- 172.21.15.32:0/3757124913 wait complete. 2020-12-05T17:32:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2020-12-05T17:32:28.151 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:32:28.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.271+0000 7f7201e3e700 1 Processor -- start 2020-12-05T17:32:28.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.271+0000 7f7201e3e700 1 -- start start 2020-12-05T17:32:28.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.271+0000 7f7201e3e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f71fc0b0240 0x7f71fc153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:28.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.271+0000 7f7201e3e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71fc080a90 con 0x7f71fc0ae860 2020-12-05T17:32:28.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.271+0000 7f7201e3e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71fc0aec40 con 0x7f71fc0611b0 2020-12-05T17:32:28.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.271+0000 7f7201e3e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71fc152860 con 0x7f71fc0b0240 2020-12-05T17:32:28.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.271+0000 7f72030a0700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f71fc0ae860 0x7f71fc14f0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36662/0 (socket says 172.21.15.32:36662) 2020-12-05T17:32:28.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.271+0000 7f72030a0700 1 -- 172.21.15.32:0/2818087294 learned_addr learned my addr 172.21.15.32:0/2818087294 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:28.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.271+0000 7f7200e3c700 1 --2- 172.21.15.32:0/2818087294 >> v2:172.21.15.61:3300/0 conn(0x7f71fc0b0240 0x7f71fc153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:28.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.271+0000 7f71f3fff700 1 -- 172.21.15.32:0/2818087294 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3442808829 0 0) 0x7f71fc080a90 con 0x7f71fc0ae860 2020-12-05T17:32:28.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.271+0000 7f71f3fff700 1 -- 172.21.15.32:0/2818087294 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71e8003230 con 0x7f71fc0ae860 2020-12-05T17:32:28.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.271+0000 7f71f3fff700 1 -- 172.21.15.32:0/2818087294 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2196992169 0 0) 0x7f71fc0aec40 con 0x7f71fc0611b0 2020-12-05T17:32:28.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.271+0000 7f71f3fff700 1 -- 172.21.15.32:0/2818087294 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71fc080a90 con 0x7f71fc0611b0 2020-12-05T17:32:28.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.271+0000 7f71f3fff700 1 -- 172.21.15.32:0/2818087294 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 618687690 0 0) 0x7f71e8003230 con 0x7f71fc0ae860 2020-12-05T17:32:28.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.271+0000 7f7200e3c700 1 -- 172.21.15.32:0/2818087294 >> v1:172.21.15.61:6789/0 conn(0x7f71fc0611b0 legacy=0x7f71fc14c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:28.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.271+0000 7f7200e3c700 1 -- 172.21.15.32:0/2818087294 >> v1:172.21.15.32:6789/0 conn(0x7f71fc0ae860 legacy=0x7f71fc14f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:28.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.271+0000 7f7200e3c700 1 -- 172.21.15.32:0/2818087294 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc152b00 con 0x7f71fc0b0240 2020-12-05T17:32:28.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.271+0000 7f7200e3c700 1 --2- 172.21.15.32:0/2818087294 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71fc0b0240 0x7f71fc153240 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f71f4001450 tx=0x7f71ec006dc0).ready entity=mon.1 client_cookie=1bc27a828ae88f59 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:28.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.271+0000 7f71f3fff700 1 -- 172.21.15.32:0/2818087294 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71f4001a50 con 0x7f71fc0b0240 2020-12-05T17:32:28.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.275+0000 7f71f3fff700 1 -- 172.21.15.32:0/2818087294 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ec00d610 con 0x7f71fc0b0240 2020-12-05T17:32:28.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.275+0000 7f71f3fff700 1 -- 172.21.15.32:0/2818087294 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71ec00d780 con 0x7f71fc0b0240 2020-12-05T17:32:28.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.275+0000 7f7201e3e700 1 -- 172.21.15.32:0/2818087294 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71fc0b0240 msgr2=0x7f71fc153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:28.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.275+0000 7f7201e3e700 1 --2- 172.21.15.32:0/2818087294 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71fc0b0240 0x7f71fc153240 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f71f4001450 tx=0x7f71ec006dc0).stop 2020-12-05T17:32:28.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.275+0000 7f7201e3e700 1 -- 172.21.15.32:0/2818087294 shutdown_connections 2020-12-05T17:32:28.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.275+0000 7f7201e3e700 1 --2- 172.21.15.32:0/2818087294 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71fc0b0240 0x7f71fc153240 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:28.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.275+0000 7f7201e3e700 1 -- 172.21.15.32:0/2818087294 >> 172.21.15.32:0/2818087294 conn(0x7f71fc0b5f90 msgr2=0x7f71fc0b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:28.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.275+0000 7f7201e3e700 1 -- 172.21.15.32:0/2818087294 shutdown_connections 2020-12-05T17:32:28.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.275+0000 7f7201e3e700 1 -- 172.21.15.32:0/2818087294 wait complete. 2020-12-05T17:32:28.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.275+0000 7f7201e3e700 1 Processor -- start 2020-12-05T17:32:28.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.275+0000 7f7201e3e700 1 -- start start 2020-12-05T17:32:28.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.275+0000 7f7201e3e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71fc0611b0 0x7f71fc0746e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:28.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.275+0000 7f7201e3e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71fc0ae860 0x7f71fc07f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:28.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.275+0000 7f7201e3e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71fc0b0240 0x7f71fc0645f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:28.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.275+0000 7f7201e3e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc0641c0 con 0x7f71fc0611b0 2020-12-05T17:32:28.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.275+0000 7f7201e3e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc074bd0 con 0x7f71fc0b0240 2020-12-05T17:32:28.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.275+0000 7f7201e3e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc0636e0 con 0x7f71fc0ae860 2020-12-05T17:32:28.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.275+0000 7f720163d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71fc0611b0 0x7f71fc0746e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:28.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.275+0000 7f720163d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71fc0611b0 0x7f71fc0746e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34020/0 (socket says 172.21.15.32:34020) 2020-12-05T17:32:28.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.275+0000 7f720163d700 1 -- 172.21.15.32:0/3151136057 learned_addr learned my addr 172.21.15.32:0/3151136057 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:28.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.275+0000 7f7200e3c700 1 --2- 172.21.15.32:0/3151136057 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71fc0ae860 0x7f71fc07f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:28.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.275+0000 7f72030a0700 1 --2- 172.21.15.32:0/3151136057 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71fc0b0240 0x7f71fc0645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:28.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.279+0000 7f720163d700 1 -- 172.21.15.32:0/3151136057 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71fc0ae860 msgr2=0x7f71fc07f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:28.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.279+0000 7f720163d700 1 --2- 172.21.15.32:0/3151136057 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71fc0ae860 0x7f71fc07f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:28.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.279+0000 7f720163d700 1 -- 172.21.15.32:0/3151136057 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71fc0b0240 msgr2=0x7f71fc0645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:28.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.279+0000 7f720163d700 1 --2- 172.21.15.32:0/3151136057 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71fc0b0240 0x7f71fc0645f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:28.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.279+0000 7f720163d700 1 -- 172.21.15.32:0/3151136057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc063980 con 0x7f71fc0611b0 2020-12-05T17:32:28.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.279+0000 7f7200e3c700 1 --2- 172.21.15.32:0/3151136057 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71fc0ae860 0x7f71fc07f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:28.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.279+0000 7f720163d700 1 --2- 172.21.15.32:0/3151136057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71fc0611b0 0x7f71fc0746e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f71f8001110 tx=0x7f71f8007ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:28.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.279+0000 7f71f1ffb700 1 -- 172.21.15.32:0/3151136057 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71f8010510 con 0x7f71fc0611b0 2020-12-05T17:32:28.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.279+0000 7f71f1ffb700 1 -- 172.21.15.32:0/3151136057 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f8010670 con 0x7f71fc0611b0 2020-12-05T17:32:28.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.279+0000 7f7201e3e700 1 -- 172.21.15.32:0/3151136057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71fc063360 con 0x7f71fc0611b0 2020-12-05T17:32:28.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.279+0000 7f71f1ffb700 1 -- 172.21.15.32:0/3151136057 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71f80107e0 con 0x7f71fc0611b0 2020-12-05T17:32:28.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.279+0000 7f7201e3e700 1 -- 172.21.15.32:0/3151136057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71fc063e90 con 0x7f71fc0611b0 2020-12-05T17:32:28.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.279+0000 7f71f1ffb700 1 -- 172.21.15.32:0/3151136057 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f71f8010070 con 0x7f71fc0611b0 2020-12-05T17:32:28.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.279+0000 7f71f1ffb700 1 --2- 172.21.15.32:0/3151136057 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71dc055320 0x7f71dc0577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:28.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.279+0000 7f71f1ffb700 1 -- 172.21.15.32:0/3151136057 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f71f806da50 con 0x7f71fc0611b0 2020-12-05T17:32:28.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.279+0000 7f7201e3e700 1 -- 172.21.15.32:0/3151136057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71fc063e90 con 0x7f71fc0611b0 2020-12-05T17:32:28.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.279+0000 7f7200e3c700 1 --2- 172.21.15.32:0/3151136057 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71dc055320 0x7f71dc0577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:28.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.279+0000 7f7200e3c700 1 --2- 172.21.15.32:0/3151136057 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71dc055320 0x7f71dc0577e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f71ec018000 tx=0x7f71ec015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:28.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.287+0000 7f71f1ffb700 1 -- 172.21.15.32:0/3151136057 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f71fc063e90 con 0x7f71fc0611b0 2020-12-05T17:32:28.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.475+0000 7f7201e3e700 1 -- 172.21.15.32:0/3151136057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f71fc063e90 con 0x7f71fc0611b0 2020-12-05T17:32:28.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.479+0000 7f71f1ffb700 1 -- 172.21.15.32:0/3151136057 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v13) v1 ==== 184+0+0 (secure 0 0 0) 0x7f71fc063e90 con 0x7f71fc0611b0 2020-12-05T17:32:28.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.483+0000 7f7201e3e700 1 -- 172.21.15.32:0/3151136057 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71dc055320 msgr2=0x7f71dc0577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:28.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.483+0000 7f7201e3e700 1 --2- 172.21.15.32:0/3151136057 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71dc055320 0x7f71dc0577e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f71ec018000 tx=0x7f71ec015000).stop 2020-12-05T17:32:28.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.483+0000 7f7201e3e700 1 -- 172.21.15.32:0/3151136057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71fc0611b0 msgr2=0x7f71fc0746e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:28.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.483+0000 7f7201e3e700 1 --2- 172.21.15.32:0/3151136057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71fc0611b0 0x7f71fc0746e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f71f8001110 tx=0x7f71f8007ab0).stop 2020-12-05T17:32:28.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.483+0000 7f7201e3e700 1 -- 172.21.15.32:0/3151136057 shutdown_connections 2020-12-05T17:32:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.483+0000 7f7201e3e700 1 --2- 172.21.15.32:0/3151136057 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71fc0b0240 0x7f71fc0645f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.483+0000 7f7201e3e700 1 --2- 172.21.15.32:0/3151136057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71fc0611b0 0x7f71fc0746e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.483+0000 7f7201e3e700 1 --2- 172.21.15.32:0/3151136057 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71dc055320 0x7f71dc0577e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.483+0000 7f7201e3e700 1 --2- 172.21.15.32:0/3151136057 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71fc0ae860 0x7f71fc07f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.483+0000 7f7201e3e700 1 -- 172.21.15.32:0/3151136057 >> 172.21.15.32:0/3151136057 conn(0x7f71fc0b5f90 msgr2=0x7f71fc12dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.483+0000 7f7201e3e700 1 -- 172.21.15.32:0/3151136057 shutdown_connections 2020-12-05T17:32:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.483+0000 7f7201e3e700 1 -- 172.21.15.32:0/3151136057 wait complete. 2020-12-05T17:32:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.2 2020-12-05T17:32:28.503 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:32:28.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.615+0000 7fa99f26c700 1 Processor -- start 2020-12-05T17:32:28.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.615+0000 7fa99f26c700 1 -- start start 2020-12-05T17:32:28.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.615+0000 7fa99f26c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa998062400 0x7fa99805f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:28.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.615+0000 7fa99f26c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa99805feb0 0x7fa9980b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:28.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.615+0000 7fa99f26c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa998142450 con 0x7fa99805f3b0 2020-12-05T17:32:28.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.615+0000 7fa99f26c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9980b42f0 con 0x7fa99805feb0 2020-12-05T17:32:28.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.615+0000 7fa99f26c700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa998061780 con 0x7fa998062400 2020-12-05T17:32:28.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.615+0000 7fa99ea6b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa99805feb0 0x7fa9980b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:28.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.615+0000 7fa99ea6b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa99805feb0 0x7fa9980b3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34028/0 (socket says 172.21.15.32:34028) 2020-12-05T17:32:28.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.615+0000 7fa99ea6b700 1 -- 172.21.15.32:0/1890564275 learned_addr learned my addr 172.21.15.32:0/1890564275 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:28.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.615+0000 7fa99da69700 1 --2- 172.21.15.32:0/1890564275 >> v2:172.21.15.61:3300/0 conn(0x7fa998062400 0x7fa99805f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:28.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.615+0000 7fa99ea6b700 1 -- 172.21.15.32:0/1890564275 >> v2:172.21.15.61:3300/0 conn(0x7fa998062400 msgr2=0x7fa99805f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:28.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.615+0000 7fa99ea6b700 1 --2- 172.21.15.32:0/1890564275 >> v2:172.21.15.61:3300/0 conn(0x7fa998062400 0x7fa99805f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:28.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.615+0000 7fa99ea6b700 1 -- 172.21.15.32:0/1890564275 >> v1:172.21.15.61:6789/0 conn(0x7fa99805f3b0 legacy=0x7fa9980b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:28.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.615+0000 7fa99ea6b700 1 -- 172.21.15.32:0/1890564275 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa998061a20 con 0x7fa99805feb0 2020-12-05T17:32:28.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.615+0000 7fa99ea6b700 1 --2- 172.21.15.32:0/1890564275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa99805feb0 0x7fa9980b3e00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7fa9900032b0 tx=0x7fa990009270).ready entity=mon.0 client_cookie=1b381da8156629c8 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:28.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.615+0000 7fa99d268700 1 -- 172.21.15.32:0/1890564275 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa990009e60 con 0x7fa99805feb0 2020-12-05T17:32:28.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.615+0000 7fa99d268700 1 -- 172.21.15.32:0/1890564275 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99000bc00 con 0x7fa99805feb0 2020-12-05T17:32:28.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.615+0000 7fa99d268700 1 -- 172.21.15.32:0/1890564275 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa99000bd70 con 0x7fa99805feb0 2020-12-05T17:32:28.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.615+0000 7fa99f26c700 1 -- 172.21.15.32:0/1890564275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa99805feb0 msgr2=0x7fa9980b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:28.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.615+0000 7fa99f26c700 1 --2- 172.21.15.32:0/1890564275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa99805feb0 0x7fa9980b3e00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7fa9900032b0 tx=0x7fa990009270).stop 2020-12-05T17:32:28.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.619+0000 7fa99f26c700 1 -- 172.21.15.32:0/1890564275 shutdown_connections 2020-12-05T17:32:28.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.619+0000 7fa99f26c700 1 --2- 172.21.15.32:0/1890564275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa99805feb0 0x7fa9980b3e00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:28.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.619+0000 7fa99f26c700 1 --2- 172.21.15.32:0/1890564275 >> v2:172.21.15.61:3300/0 conn(0x7fa998062400 0x7fa99805f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:28.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.619+0000 7fa99f26c700 1 -- 172.21.15.32:0/1890564275 >> 172.21.15.32:0/1890564275 conn(0x7fa9980b8ee0 msgr2=0x7fa9980b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:28.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.619+0000 7fa99f26c700 1 -- 172.21.15.32:0/1890564275 shutdown_connections 2020-12-05T17:32:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.619+0000 7fa99f26c700 1 -- 172.21.15.32:0/1890564275 wait complete. 2020-12-05T17:32:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.619+0000 7fa99f26c700 1 Processor -- start 2020-12-05T17:32:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.619+0000 7fa99f26c700 1 -- start start 2020-12-05T17:32:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.619+0000 7fa99f26c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa99805f3b0 0x7fa998063340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:28.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.619+0000 7fa99f26c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa99805feb0 0x7fa9981552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:28.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.619+0000 7fa99f26c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa998062400 0x7fa99813e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:28.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.619+0000 7fa99f26c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa998063c10 con 0x7fa99805f3b0 2020-12-05T17:32:28.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.619+0000 7fa99f26c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9981557b0 con 0x7fa998062400 2020-12-05T17:32:28.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.619+0000 7fa99f26c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9980b2370 con 0x7fa99805feb0 2020-12-05T17:32:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.619+0000 7fa99e26a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa99805f3b0 0x7fa998063340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.623+0000 7fa99e26a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa99805f3b0 0x7fa998063340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34030/0 (socket says 172.21.15.32:34030) 2020-12-05T17:32:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.623+0000 7fa99e26a700 1 -- 172.21.15.32:0/3453502609 learned_addr learned my addr 172.21.15.32:0/3453502609 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.623+0000 7fa99ea6b700 1 --2- 172.21.15.32:0/3453502609 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa998062400 0x7fa99813e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.623+0000 7fa99da69700 1 --2- 172.21.15.32:0/3453502609 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa99805feb0 0x7fa9981552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.623+0000 7fa99e26a700 1 -- 172.21.15.32:0/3453502609 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa99805feb0 msgr2=0x7fa9981552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.623+0000 7fa99e26a700 1 --2- 172.21.15.32:0/3453502609 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa99805feb0 0x7fa9981552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.623+0000 7fa99e26a700 1 -- 172.21.15.32:0/3453502609 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa998062400 msgr2=0x7fa99813e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:28.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.623+0000 7fa99e26a700 1 --2- 172.21.15.32:0/3453502609 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa998062400 0x7fa99813e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:28.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.623+0000 7fa99e26a700 1 -- 172.21.15.32:0/3453502609 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9980b2610 con 0x7fa99805f3b0 2020-12-05T17:32:28.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.623+0000 7fa99da69700 1 --2- 172.21.15.32:0/3453502609 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa99805feb0 0x7fa9981552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:32:28.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.623+0000 7fa99e26a700 1 --2- 172.21.15.32:0/3453502609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa99805f3b0 0x7fa998063340 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fa9980622e0 tx=0x7fa994007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:28.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.623+0000 7fa98b7fe700 1 -- 172.21.15.32:0/3453502609 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa99400e4a0 con 0x7fa99805f3b0 2020-12-05T17:32:28.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.623+0000 7fa98b7fe700 1 -- 172.21.15.32:0/3453502609 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99400e600 con 0x7fa99805f3b0 2020-12-05T17:32:28.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.623+0000 7fa99f26c700 1 -- 172.21.15.32:0/3453502609 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9980b28b0 con 0x7fa99805f3b0 2020-12-05T17:32:28.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.623+0000 7fa98b7fe700 1 -- 172.21.15.32:0/3453502609 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa99400e960 con 0x7fa99805f3b0 2020-12-05T17:32:28.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.623+0000 7fa99f26c700 1 -- 172.21.15.32:0/3453502609 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa998144c20 con 0x7fa99805f3b0 2020-12-05T17:32:28.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.623+0000 7fa98b7fe700 1 -- 172.21.15.32:0/3453502609 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa994010030 con 0x7fa99805f3b0 2020-12-05T17:32:28.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.623+0000 7fa99f26c700 1 -- 172.21.15.32:0/3453502609 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa998057050 con 0x7fa99805f3b0 2020-12-05T17:32:28.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.627+0000 7fa98b7fe700 1 --2- 172.21.15.32:0/3453502609 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa978055440 0x7fa978057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:28.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.627+0000 7fa99da69700 1 --2- 172.21.15.32:0/3453502609 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa978055440 0x7fa978057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:28.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.627+0000 7fa98b7fe700 1 -- 172.21.15.32:0/3453502609 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa99406df30 con 0x7fa99805f3b0 2020-12-05T17:32:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.627+0000 7fa99da69700 1 --2- 172.21.15.32:0/3453502609 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa978055440 0x7fa978057900 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fa98c00b000 tx=0x7fa98c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.631+0000 7fa98b7fe700 1 -- 172.21.15.32:0/3453502609 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa9940440d0 con 0x7fa99805f3b0 2020-12-05T17:32:28.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.815+0000 7fa99f26c700 1 -- 172.21.15.32:0/3453502609 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa99805aab0 con 0x7fa99805f3b0 2020-12-05T17:32:28.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.815+0000 7fa98b7fe700 1 -- 172.21.15.32:0/3453502609 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2136 (secure 0 0 0) 0x7fa9940476f0 con 0x7fa99805f3b0 2020-12-05T17:32:28.819 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:32:28.819 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4679,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/2850341272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2850341272},{"type":"v1","addr":"172.21.15.61:6833","nonce":2850341272}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:32:25.792490+0000","modified":"2020-12-05T17:32:26.797858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4873},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4873":{"gid":4873,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3316547885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3316547885},{"type":"v1","addr":"172.21.15.32:6835","nonce":3316547885}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-12-05T17:32:28.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.815+0000 7fa99f26c700 1 -- 172.21.15.32:0/3453502609 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa978055440 msgr2=0x7fa978057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:28.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.815+0000 7fa99f26c700 1 --2- 172.21.15.32:0/3453502609 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa978055440 0x7fa978057900 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fa98c00b000 tx=0x7fa98c008040).stop 2020-12-05T17:32:28.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.815+0000 7fa99f26c700 1 -- 172.21.15.32:0/3453502609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa99805f3b0 msgr2=0x7fa998063340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:28.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.815+0000 7fa99f26c700 1 --2- 172.21.15.32:0/3453502609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa99805f3b0 0x7fa998063340 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fa9980622e0 tx=0x7fa994007c20).stop 2020-12-05T17:32:28.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.819+0000 7fa99f26c700 1 -- 172.21.15.32:0/3453502609 shutdown_connections 2020-12-05T17:32:28.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.819+0000 7fa99f26c700 1 --2- 172.21.15.32:0/3453502609 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa998062400 0x7fa99813e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:28.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.819+0000 7fa99f26c700 1 --2- 172.21.15.32:0/3453502609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa99805f3b0 0x7fa998063340 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:28.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.819+0000 7fa99f26c700 1 --2- 172.21.15.32:0/3453502609 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa978055440 0x7fa978057900 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:28.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.819+0000 7fa99f26c700 1 --2- 172.21.15.32:0/3453502609 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa99805feb0 0x7fa9981552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:28.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.819+0000 7fa99f26c700 1 -- 172.21.15.32:0/3453502609 >> 172.21.15.32:0/3453502609 conn(0x7fa9980b8ee0 msgr2=0x7fa9980b54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:28.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.819+0000 7fa99f26c700 1 -- 172.21.15.32:0/3453502609 shutdown_connections 2020-12-05T17:32:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:28.819+0000 7fa99f26c700 1 -- 172.21.15.32:0/3453502609 wait complete. 2020-12-05T17:32:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 10 2020-12-05T17:32:29.035 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 10, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:32:25.792490+0000', 'modified': '2020-12-05T17:32:26.797858+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4873}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4873': {'gid': 4873, 'name': 'a', 'rank': 0, 'incarnation': 9, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/3316547885', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 3316547885}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 3316547885}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:32:29.035 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:32:29.036 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:32:29.172 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":4873,"want_state":"up:active","state":"up:active","rank_uptime":3.3636427480000002,"mdsmap_epoch":10,"osdmap_epoch":34,"osdmap_epoch_barrier":33,"uptime":10.036501295000001} 2020-12-05T17:32:29.173 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 4873, "mdsmap_epoch": 10, "osdmap_epoch": 34, "osdmap_epoch_barrier": 33, "rank_uptime": 3.363642748, "state": "up:active", "uptime": 10.036501295, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:32:29.174 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:32:29.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.287+0000 7fc945585700 1 Processor -- start 2020-12-05T17:32:29.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.287+0000 7fc945585700 1 -- start start 2020-12-05T17:32:29.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.287+0000 7fc945585700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc9400b5650 0x7fc940141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:29.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.287+0000 7fc945585700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc9400afd20 0x7fc940141fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:29.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.287+0000 7fc945585700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9400af300 con 0x7fc940142a30 2020-12-05T17:32:29.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.287+0000 7fc945585700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9400b5a30 con 0x7fc9400b5650 2020-12-05T17:32:29.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.287+0000 7fc945585700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9400aea90 con 0x7fc9400afd20 2020-12-05T17:32:29.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.287+0000 7fc93ffff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc9400b5650 0x7fc940141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:29.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.287+0000 7fc937fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc9400afd20 0x7fc940141fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:29.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.287+0000 7fc93ffff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc9400b5650 0x7fc940141ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:53930/0 (socket says 172.21.15.32:53930) 2020-12-05T17:32:29.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.287+0000 7fc93ffff700 1 -- 172.21.15.32:0/667513182 learned_addr learned my addr 172.21.15.32:0/667513182 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:29.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.287+0000 7fc944d84700 1 --1- 172.21.15.32:0/667513182 >> v1:172.21.15.61:6789/0 conn(0x7fc940142a30 0x7fc940142ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:49328/0 (socket says 172.21.15.32:49328) 2020-12-05T17:32:29.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.287+0000 7fc937fff700 1 -- 172.21.15.32:0/667513182 >> v2:172.21.15.61:3300/0 conn(0x7fc9400b5650 msgr2=0x7fc940141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:29.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.287+0000 7fc937fff700 1 --2- 172.21.15.32:0/667513182 >> v2:172.21.15.61:3300/0 conn(0x7fc9400b5650 0x7fc940141ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:29.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.287+0000 7fc937fff700 1 -- 172.21.15.32:0/667513182 >> v1:172.21.15.61:6789/0 conn(0x7fc940142a30 legacy=0x7fc940142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:29.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.287+0000 7fc937fff700 1 -- 172.21.15.32:0/667513182 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9400aed30 con 0x7fc9400afd20 2020-12-05T17:32:29.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.287+0000 7fc937fff700 1 --2- 172.21.15.32:0/667513182 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc9400afd20 0x7fc940141fc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fc928002f30 tx=0x7fc928006dc0).ready entity=mon.2 client_cookie=862407f836808ad3 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:29.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.287+0000 7fc93f7fe700 1 -- 172.21.15.32:0/667513182 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc928009760 con 0x7fc9400afd20 2020-12-05T17:32:29.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.287+0000 7fc93f7fe700 1 -- 172.21.15.32:0/667513182 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9280098c0 con 0x7fc9400afd20 2020-12-05T17:32:29.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.287+0000 7fc93f7fe700 1 -- 172.21.15.32:0/667513182 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc928009c20 con 0x7fc9400afd20 2020-12-05T17:32:29.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.287+0000 7fc945585700 1 -- 172.21.15.32:0/667513182 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc9400afd20 msgr2=0x7fc940141fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:29.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.287+0000 7fc945585700 1 --2- 172.21.15.32:0/667513182 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc9400afd20 0x7fc940141fc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fc928002f30 tx=0x7fc928006dc0).stop 2020-12-05T17:32:29.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.291+0000 7fc945585700 1 -- 172.21.15.32:0/667513182 shutdown_connections 2020-12-05T17:32:29.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.291+0000 7fc945585700 1 --2- 172.21.15.32:0/667513182 >> v2:172.21.15.61:3300/0 conn(0x7fc9400b5650 0x7fc940141ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:29.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.291+0000 7fc945585700 1 --2- 172.21.15.32:0/667513182 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc9400afd20 0x7fc940141fc0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:29.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.291+0000 7fc945585700 1 -- 172.21.15.32:0/667513182 >> 172.21.15.32:0/667513182 conn(0x7fc94005bbb0 msgr2=0x7fc940062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:29.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.291+0000 7fc945585700 1 -- 172.21.15.32:0/667513182 shutdown_connections 2020-12-05T17:32:29.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.291+0000 7fc945585700 1 -- 172.21.15.32:0/667513182 wait complete. 2020-12-05T17:32:29.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.291+0000 7fc945585700 1 Processor -- start 2020-12-05T17:32:29.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.291+0000 7fc945585700 1 -- start start 2020-12-05T17:32:29.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.291+0000 7fc945585700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9400afd20 0x7fc9401408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:29.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.291+0000 7fc945585700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9400b5650 0x7fc940111d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:29.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.291+0000 7fc945585700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc940142a30 0x7fc940112280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:29.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.291+0000 7fc945585700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94010ec00 con 0x7fc9400afd20 2020-12-05T17:32:29.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.291+0000 7fc945585700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94010eea0 con 0x7fc9400b5650 2020-12-05T17:32:29.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.291+0000 7fc945585700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc940112770 con 0x7fc940142a30 2020-12-05T17:32:29.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.291+0000 7fc93ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9400afd20 0x7fc9401408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:29.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.291+0000 7fc93ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9400afd20 0x7fc9401408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34044/0 (socket says 172.21.15.32:34044) 2020-12-05T17:32:29.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.291+0000 7fc93ffff700 1 -- 172.21.15.32:0/2958329001 learned_addr learned my addr 172.21.15.32:0/2958329001 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:29.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.291+0000 7fc937fff700 1 --2- 172.21.15.32:0/2958329001 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9400b5650 0x7fc940111d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:29.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.291+0000 7fc944d84700 1 --2- 172.21.15.32:0/2958329001 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc940142a30 0x7fc940112280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:29.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.295+0000 7fc93ffff700 1 -- 172.21.15.32:0/2958329001 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc940142a30 msgr2=0x7fc940112280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.295+0000 7fc93ffff700 1 --2- 172.21.15.32:0/2958329001 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc940142a30 0x7fc940112280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.295+0000 7fc93ffff700 1 -- 172.21.15.32:0/2958329001 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9400b5650 msgr2=0x7fc940111d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.295+0000 7fc93ffff700 1 --2- 172.21.15.32:0/2958329001 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9400b5650 0x7fc940111d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.295+0000 7fc93ffff700 1 -- 172.21.15.32:0/2958329001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc940112930 con 0x7fc9400afd20 2020-12-05T17:32:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.295+0000 7fc93ffff700 1 --2- 172.21.15.32:0/2958329001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9400afd20 0x7fc9401408b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7fc93000c000 tx=0x7fc930008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:29.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.295+0000 7fc93dffb700 1 -- 172.21.15.32:0/2958329001 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc93000ee80 con 0x7fc9400afd20 2020-12-05T17:32:29.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.295+0000 7fc945585700 1 -- 172.21.15.32:0/2958329001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94010f340 con 0x7fc9400afd20 2020-12-05T17:32:29.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.295+0000 7fc93dffb700 1 -- 172.21.15.32:0/2958329001 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93000c820 con 0x7fc9400afd20 2020-12-05T17:32:29.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.295+0000 7fc945585700 1 -- 172.21.15.32:0/2958329001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc940151730 con 0x7fc9400afd20 2020-12-05T17:32:29.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.295+0000 7fc93dffb700 1 -- 172.21.15.32:0/2958329001 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc93000cb80 con 0x7fc9400afd20 2020-12-05T17:32:29.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.295+0000 7fc945585700 1 -- 172.21.15.32:0/2958329001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc940057050 con 0x7fc9400afd20 2020-12-05T17:32:29.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.295+0000 7fc93dffb700 1 -- 172.21.15.32:0/2958329001 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc930022070 con 0x7fc9400afd20 2020-12-05T17:32:29.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.299+0000 7fc93dffb700 1 --2- 172.21.15.32:0/2958329001 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc924055330 0x7fc9240577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:29.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.299+0000 7fc93dffb700 1 -- 172.21.15.32:0/2958329001 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc93006e430 con 0x7fc9400afd20 2020-12-05T17:32:29.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.299+0000 7fc937fff700 1 --2- 172.21.15.32:0/2958329001 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc924055330 0x7fc9240577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:29.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.299+0000 7fc937fff700 1 --2- 172.21.15.32:0/2958329001 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc924055330 0x7fc9240577f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fc92801a000 tx=0x7fc928015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:29.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.303+0000 7fc93dffb700 1 -- 172.21.15.32:0/2958329001 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc930044530 con 0x7fc9400afd20 2020-12-05T17:32:29.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.447+0000 7fc945585700 1 -- 172.21.15.32:0/2958329001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc94005c300 con 0x7fc9400afd20 2020-12-05T17:32:29.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.451+0000 7fc93dffb700 1 -- 172.21.15.32:0/2958329001 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16271 (secure 0 0 0) 0x7fc930047b50 con 0x7fc9400afd20 2020-12-05T17:32:29.453 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:32:29.454 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:32:26.798545+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:32:23.957088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-12-05T17:32:24.800777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:32:29.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.451+0000 7fc945585700 1 -- 172.21.15.32:0/2958329001 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc924055330 msgr2=0x7fc9240577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:29.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.451+0000 7fc945585700 1 --2- 172.21.15.32:0/2958329001 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc924055330 0x7fc9240577f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fc92801a000 tx=0x7fc928015000).stop 2020-12-05T17:32:29.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.451+0000 7fc945585700 1 -- 172.21.15.32:0/2958329001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9400afd20 msgr2=0x7fc9401408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:29.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.451+0000 7fc945585700 1 --2- 172.21.15.32:0/2958329001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9400afd20 0x7fc9401408b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7fc93000c000 tx=0x7fc930008040).stop 2020-12-05T17:32:29.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.451+0000 7fc945585700 1 -- 172.21.15.32:0/2958329001 shutdown_connections 2020-12-05T17:32:29.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.451+0000 7fc945585700 1 --2- 172.21.15.32:0/2958329001 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc924055330 0x7fc9240577f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:29.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.451+0000 7fc945585700 1 --2- 172.21.15.32:0/2958329001 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc940142a30 0x7fc940112280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:29.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.451+0000 7fc945585700 1 --2- 172.21.15.32:0/2958329001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9400afd20 0x7fc9401408b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:29.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.451+0000 7fc945585700 1 --2- 172.21.15.32:0/2958329001 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9400b5650 0x7fc940111d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:29.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.451+0000 7fc945585700 1 -- 172.21.15.32:0/2958329001 >> 172.21.15.32:0/2958329001 conn(0x7fc94005bbb0 msgr2=0x7fc940145b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:29.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.451+0000 7fc945585700 1 -- 172.21.15.32:0/2958329001 shutdown_connections 2020-12-05T17:32:29.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.451+0000 7fc945585700 1 -- 172.21.15.32:0/2958329001 wait complete. 2020-12-05T17:32:29.467 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:32:23.957088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:32:24.800777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:32:29.467 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:32:29.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.583+0000 7f7dc10d7700 1 Processor -- start 2020-12-05T17:32:29.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.583+0000 7f7dc10d7700 1 -- start start 2020-12-05T17:32:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dc10d7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7dbc0b29d0 0x7f7dbc13ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dc10d7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7dbc142b70 0x7f7dbc13ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dc10d7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dbc13f460 con 0x7f7dbc0af340 2020-12-05T17:32:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dc10d7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dbc143b60 con 0x7f7dbc0b29d0 2020-12-05T17:32:29.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dc10d7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc0b2db0 con 0x7f7dbc142b70 2020-12-05T17:32:29.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dbad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7dbc0b29d0 0x7f7dbc13ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:29.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dbb59e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7dbc0af340 0x7f7dbc0b0320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36704/0 (socket says 172.21.15.32:36704) 2020-12-05T17:32:29.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dbb59e700 1 -- 172.21.15.32:0/1081515768 learned_addr learned my addr 172.21.15.32:0/1081515768 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:29.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dbad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7dbc0b29d0 0x7f7dbc13ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34052/0 (socket says 172.21.15.32:34052) 2020-12-05T17:32:29.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7db259c700 1 --2- 172.21.15.32:0/1081515768 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7dbc142b70 0x7f7dbc13ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:29.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dbad9d700 1 -- 172.21.15.32:0/1081515768 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7dbc142b70 msgr2=0x7f7dbc13ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:29.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dbad9d700 1 --2- 172.21.15.32:0/1081515768 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7dbc142b70 0x7f7dbc13ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:29.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dbad9d700 1 -- 172.21.15.32:0/1081515768 >> v1:172.21.15.32:6789/0 conn(0x7f7dbc0af340 legacy=0x7f7dbc0b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:29.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dbad9d700 1 -- 172.21.15.32:0/1081515768 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dbc0afd50 con 0x7f7dbc0b29d0 2020-12-05T17:32:29.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dba59c700 1 -- 172.21.15.32:0/1081515768 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684920481 0 0) 0x7f7dbc13f460 con 0x7f7dbc0af340 2020-12-05T17:32:29.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dbad9d700 1 --2- 172.21.15.32:0/1081515768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc0b29d0 0x7f7dbc13ea80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f7dac002f30 tx=0x7f7dac006dc0).ready entity=mon.0 client_cookie=56ea8880a3a69665 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:29.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dba59c700 1 -- 172.21.15.32:0/1081515768 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7dac009760 con 0x7f7dbc0b29d0 2020-12-05T17:32:29.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dba59c700 1 -- 172.21.15.32:0/1081515768 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dac0098c0 con 0x7f7dbc0b29d0 2020-12-05T17:32:29.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dba59c700 1 -- 172.21.15.32:0/1081515768 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7dac009c20 con 0x7f7dbc0b29d0 2020-12-05T17:32:29.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dc10d7700 1 -- 172.21.15.32:0/1081515768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc0b29d0 msgr2=0x7f7dbc13ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:29.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dc10d7700 1 --2- 172.21.15.32:0/1081515768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc0b29d0 0x7f7dbc13ea80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f7dac002f30 tx=0x7f7dac006dc0).stop 2020-12-05T17:32:29.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dc10d7700 1 -- 172.21.15.32:0/1081515768 shutdown_connections 2020-12-05T17:32:29.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dc10d7700 1 --2- 172.21.15.32:0/1081515768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc0b29d0 0x7f7dbc13ea80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:29.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dc10d7700 1 --2- 172.21.15.32:0/1081515768 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7dbc142b70 0x7f7dbc13ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:29.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dc10d7700 1 -- 172.21.15.32:0/1081515768 >> 172.21.15.32:0/1081515768 conn(0x7f7dbc062310 msgr2=0x7f7dbc0626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:29.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dc10d7700 1 -- 172.21.15.32:0/1081515768 shutdown_connections 2020-12-05T17:32:29.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.587+0000 7f7dc10d7700 1 -- 172.21.15.32:0/1081515768 wait complete. 2020-12-05T17:32:29.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.591+0000 7f7dc10d7700 1 Processor -- start 2020-12-05T17:32:29.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.591+0000 7f7dc10d7700 1 -- start start 2020-12-05T17:32:29.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.591+0000 7f7dc10d7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7dbc0af340 0x7f7dbc152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:29.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.591+0000 7f7dc10d7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7dbc0b29d0 0x7f7dbc141f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:29.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.591+0000 7f7dc10d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc0663b0 0x7f7dbc14fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:29.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.591+0000 7f7dc10d7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc153aa0 con 0x7f7dbc0663b0 2020-12-05T17:32:29.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.591+0000 7f7dc10d7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc14f630 con 0x7f7dbc0b29d0 2020-12-05T17:32:29.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.591+0000 7f7dc10d7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc13c3f0 con 0x7f7dbc0af340 2020-12-05T17:32:29.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.591+0000 7f7dbb59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc0663b0 0x7f7dbc14fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:29.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.591+0000 7f7dbb59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc0663b0 0x7f7dbc14fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34062/0 (socket says 172.21.15.32:34062) 2020-12-05T17:32:29.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.591+0000 7f7dbb59e700 1 -- 172.21.15.32:0/3733308209 learned_addr learned my addr 172.21.15.32:0/3733308209 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:29.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.591+0000 7f7dbad9d700 1 --2- 172.21.15.32:0/3733308209 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7dbc0af340 0x7f7dbc152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:29.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.591+0000 7f7db259c700 1 --2- 172.21.15.32:0/3733308209 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7dbc0b29d0 0x7f7dbc141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:29.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.595+0000 7f7dbb59e700 1 -- 172.21.15.32:0/3733308209 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7dbc0af340 msgr2=0x7f7dbc152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:29.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.595+0000 7f7dbb59e700 1 --2- 172.21.15.32:0/3733308209 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7dbc0af340 0x7f7dbc152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:29.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.595+0000 7f7dbb59e700 1 -- 172.21.15.32:0/3733308209 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7dbc0b29d0 msgr2=0x7f7dbc141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:29.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.595+0000 7f7dbb59e700 1 --2- 172.21.15.32:0/3733308209 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7dbc0b29d0 0x7f7dbc141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:29.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.595+0000 7f7dbb59e700 1 -- 172.21.15.32:0/3733308209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dbc065290 con 0x7f7dbc0663b0 2020-12-05T17:32:29.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.595+0000 7f7dbb59e700 1 --2- 172.21.15.32:0/3733308209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc0663b0 0x7f7dbc14fa60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f7db4001320 tx=0x7f7db4007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:29.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.595+0000 7f7db8d99700 1 -- 172.21.15.32:0/3733308209 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7db4005c40 con 0x7f7dbc0663b0 2020-12-05T17:32:29.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.595+0000 7f7db8d99700 1 -- 172.21.15.32:0/3733308209 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db4005da0 con 0x7f7dbc0663b0 2020-12-05T17:32:29.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.595+0000 7f7db8d99700 1 -- 172.21.15.32:0/3733308209 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7db4007300 con 0x7f7dbc0663b0 2020-12-05T17:32:29.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.595+0000 7f7dc10d7700 1 -- 172.21.15.32:0/3733308209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dbc13c690 con 0x7f7dbc0663b0 2020-12-05T17:32:29.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.595+0000 7f7dc10d7700 1 -- 172.21.15.32:0/3733308209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dbc065860 con 0x7f7dbc0663b0 2020-12-05T17:32:29.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.595+0000 7f7dc10d7700 1 -- 172.21.15.32:0/3733308209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dbc14f7c0 con 0x7f7dbc0663b0 2020-12-05T17:32:29.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.595+0000 7f7db8d99700 1 -- 172.21.15.32:0/3733308209 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7db4010030 con 0x7f7dbc0663b0 2020-12-05T17:32:29.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.595+0000 7f7db8d99700 1 --2- 172.21.15.32:0/3733308209 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7da0055500 0x7f7da00579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:29.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.595+0000 7f7dbad9d700 1 --2- 172.21.15.32:0/3733308209 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7da0055500 0x7f7da00579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:29.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.595+0000 7f7db8d99700 1 -- 172.21.15.32:0/3733308209 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7db406e080 con 0x7f7dbc0663b0 2020-12-05T17:32:29.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.599+0000 7f7dbad9d700 1 --2- 172.21.15.32:0/3733308209 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7da0055500 0x7f7da00579c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f7dac01a000 tx=0x7f7dac015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:29.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.603+0000 7f7db8d99700 1 -- 172.21.15.32:0/3733308209 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7dbc14f7c0 con 0x7f7dbc0663b0 2020-12-05T17:32:29.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.747+0000 7f7dc10d7700 1 -- 172.21.15.32:0/3733308209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7dbc14f7c0 con 0x7f7dbc0663b0 2020-12-05T17:32:29.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.747+0000 7f7db8d99700 1 -- 172.21.15.32:0/3733308209 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v34) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7dbc14f7c0 con 0x7f7dbc0663b0 2020-12-05T17:32:29.750 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:32:29.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.747+0000 7f7dc10d7700 1 -- 172.21.15.32:0/3733308209 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7da0055500 msgr2=0x7f7da00579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:29.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.747+0000 7f7dc10d7700 1 --2- 172.21.15.32:0/3733308209 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7da0055500 0x7f7da00579c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f7dac01a000 tx=0x7f7dac015000).stop 2020-12-05T17:32:29.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.747+0000 7f7dc10d7700 1 -- 172.21.15.32:0/3733308209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc0663b0 msgr2=0x7f7dbc14fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:29.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.747+0000 7f7dc10d7700 1 --2- 172.21.15.32:0/3733308209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc0663b0 0x7f7dbc14fa60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f7db4001320 tx=0x7f7db4007df0).stop 2020-12-05T17:32:29.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.747+0000 7f7dc10d7700 1 -- 172.21.15.32:0/3733308209 shutdown_connections 2020-12-05T17:32:29.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.747+0000 7f7dc10d7700 1 --2- 172.21.15.32:0/3733308209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc0663b0 0x7f7dbc14fa60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:29.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.747+0000 7f7dc10d7700 1 --2- 172.21.15.32:0/3733308209 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7da0055500 0x7f7da00579c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:29.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.747+0000 7f7dc10d7700 1 --2- 172.21.15.32:0/3733308209 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7dbc0af340 0x7f7dbc152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:29.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.747+0000 7f7dc10d7700 1 --2- 172.21.15.32:0/3733308209 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7dbc0b29d0 0x7f7dbc141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:29.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.747+0000 7f7dc10d7700 1 -- 172.21.15.32:0/3733308209 >> 172.21.15.32:0/3733308209 conn(0x7f7dbc062310 msgr2=0x7f7dbc0b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:29.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.747+0000 7f7dc10d7700 1 -- 172.21.15.32:0/3733308209 shutdown_connections 2020-12-05T17:32:29.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.747+0000 7f7dc10d7700 1 -- 172.21.15.32:0/3733308209 wait complete. 2020-12-05T17:32:29.763 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:32:29.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.883+0000 7f48b7fff700 1 Processor -- start 2020-12-05T17:32:29.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.883+0000 7f48b7fff700 1 -- start start 2020-12-05T17:32:29.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.883+0000 7f48b7fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f48b80b1dc0 0x7f48b80b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:29.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.883+0000 7f48b7fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48b80afd30 0x7f48b80ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:29.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.883+0000 7f48b7fff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b81468e0 con 0x7f48b80af5f0 2020-12-05T17:32:29.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.883+0000 7f48b7fff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b8142c90 con 0x7f48b80b1dc0 2020-12-05T17:32:29.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.883+0000 7f48b7fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b8146b10 con 0x7f48b80afd30 2020-12-05T17:32:29.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.883+0000 7f48b6ffd700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f48b80b1dc0 0x7f48b80b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:29.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.883+0000 7f48b77fe700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f48b80af5f0 0x7f48b80aeaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:49358/0 (socket says 172.21.15.32:49358) 2020-12-05T17:32:29.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.883+0000 7f48b77fe700 1 -- 172.21.15.32:0/560732080 learned_addr learned my addr 172.21.15.32:0/560732080 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:29.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.883+0000 7f48b67fc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48b80afd30 0x7f48b80ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:29.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.883+0000 7f48b6ffd700 1 -- 172.21.15.32:0/560732080 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48b80afd30 msgr2=0x7f48b80ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:29.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.883+0000 7f48b6ffd700 1 --2- 172.21.15.32:0/560732080 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48b80afd30 0x7f48b80ae600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:29.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.883+0000 7f48b6ffd700 1 -- 172.21.15.32:0/560732080 >> v1:172.21.15.61:6789/0 conn(0x7f48b80af5f0 legacy=0x7f48b80aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:29.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.883+0000 7f48b5ffb700 1 -- 172.21.15.32:0/560732080 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802179755 0 0) 0x7f48b81468e0 con 0x7f48b80af5f0 2020-12-05T17:32:29.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.887+0000 7f48b6ffd700 1 -- 172.21.15.32:0/560732080 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b8146db0 con 0x7f48b80b1dc0 2020-12-05T17:32:29.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.887+0000 7f48b6ffd700 1 --2- 172.21.15.32:0/560732080 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48b80b1dc0 0x7f48b80b21a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f48a8002f30 tx=0x7f48a8006dc0).ready entity=mon.1 client_cookie=cd857cc63772c689 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:29.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.887+0000 7f48b5ffb700 1 -- 172.21.15.32:0/560732080 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48a8009760 con 0x7f48b80b1dc0 2020-12-05T17:32:29.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.887+0000 7f48b5ffb700 1 -- 172.21.15.32:0/560732080 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a80098c0 con 0x7f48b80b1dc0 2020-12-05T17:32:29.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.887+0000 7f48b5ffb700 1 -- 172.21.15.32:0/560732080 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48a8009c20 con 0x7f48b80b1dc0 2020-12-05T17:32:29.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.887+0000 7f48b7fff700 1 -- 172.21.15.32:0/560732080 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48b80b1dc0 msgr2=0x7f48b80b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:29.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.887+0000 7f48b7fff700 1 --2- 172.21.15.32:0/560732080 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48b80b1dc0 0x7f48b80b21a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f48a8002f30 tx=0x7f48a8006dc0).stop 2020-12-05T17:32:29.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.887+0000 7f48b7fff700 1 -- 172.21.15.32:0/560732080 shutdown_connections 2020-12-05T17:32:29.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.887+0000 7f48b7fff700 1 --2- 172.21.15.32:0/560732080 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48b80b1dc0 0x7f48b80b21a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:29.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.887+0000 7f48b7fff700 1 --2- 172.21.15.32:0/560732080 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48b80afd30 0x7f48b80ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:29.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.887+0000 7f48b7fff700 1 -- 172.21.15.32:0/560732080 >> 172.21.15.32:0/560732080 conn(0x7f48b80620b0 msgr2=0x7f48b8062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:29.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.887+0000 7f48b7fff700 1 -- 172.21.15.32:0/560732080 shutdown_connections 2020-12-05T17:32:29.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.887+0000 7f48b7fff700 1 -- 172.21.15.32:0/560732080 wait complete. 2020-12-05T17:32:29.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.887+0000 7f48b7fff700 1 Processor -- start 2020-12-05T17:32:29.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.887+0000 7f48b7fff700 1 -- start start 2020-12-05T17:32:29.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.891+0000 7f48b7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b80af5f0 0x7f48b8144110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:29.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.891+0000 7f48b7fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48b80afd30 0x7f48b8149670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:29.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.891+0000 7f48b7fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48b80b1dc0 0x7f48b8063f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:29.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.891+0000 7f48b7fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b80668f0 con 0x7f48b80af5f0 2020-12-05T17:32:29.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.891+0000 7f48b7fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b813d670 con 0x7f48b80b1dc0 2020-12-05T17:32:29.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.891+0000 7f48b7fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b813d910 con 0x7f48b80afd30 2020-12-05T17:32:29.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.891+0000 7f48b6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b80af5f0 0x7f48b8144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:29.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.891+0000 7f48b6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b80af5f0 0x7f48b8144110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34074/0 (socket says 172.21.15.32:34074) 2020-12-05T17:32:29.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.891+0000 7f48b6ffd700 1 -- 172.21.15.32:0/1469424056 learned_addr learned my addr 172.21.15.32:0/1469424056 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:29.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.891+0000 7f48b67fc700 1 --2- 172.21.15.32:0/1469424056 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48b80afd30 0x7f48b8149670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:29.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.891+0000 7f48b77fe700 1 --2- 172.21.15.32:0/1469424056 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48b80b1dc0 0x7f48b8063f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:29.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.891+0000 7f48b6ffd700 1 -- 172.21.15.32:0/1469424056 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48b80afd30 msgr2=0x7f48b8149670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:29.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.891+0000 7f48b6ffd700 1 --2- 172.21.15.32:0/1469424056 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48b80afd30 0x7f48b8149670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:29.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.891+0000 7f48b6ffd700 1 -- 172.21.15.32:0/1469424056 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48b80b1dc0 msgr2=0x7f48b8063f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:29.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.891+0000 7f48b6ffd700 1 --2- 172.21.15.32:0/1469424056 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48b80b1dc0 0x7f48b8063f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:29.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.891+0000 7f48b6ffd700 1 -- 172.21.15.32:0/1469424056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b8149b60 con 0x7f48b80af5f0 2020-12-05T17:32:29.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.891+0000 7f48b6ffd700 1 --2- 172.21.15.32:0/1469424056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b80af5f0 0x7f48b8144110 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f48a8002f30 tx=0x7f48a8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:29.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.891+0000 7f48a7fff700 1 -- 172.21.15.32:0/1469424056 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48a8011a10 con 0x7f48b80af5f0 2020-12-05T17:32:29.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.891+0000 7f48a7fff700 1 -- 172.21.15.32:0/1469424056 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a8011b70 con 0x7f48b80af5f0 2020-12-05T17:32:29.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.891+0000 7f48a7fff700 1 -- 172.21.15.32:0/1469424056 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48a801d440 con 0x7f48b80af5f0 2020-12-05T17:32:29.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.891+0000 7f48b7fff700 1 -- 172.21.15.32:0/1469424056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b8064430 con 0x7f48b80af5f0 2020-12-05T17:32:29.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.891+0000 7f48b7fff700 1 -- 172.21.15.32:0/1469424056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b8147600 con 0x7f48b80af5f0 2020-12-05T17:32:29.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.895+0000 7f48b7fff700 1 -- 172.21.15.32:0/1469424056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b81488f0 con 0x7f48b80af5f0 2020-12-05T17:32:29.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.895+0000 7f48a7fff700 1 -- 172.21.15.32:0/1469424056 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f48a8024030 con 0x7f48b80af5f0 2020-12-05T17:32:29.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.895+0000 7f48a7fff700 1 --2- 172.21.15.32:0/1469424056 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4894055470 0x7f4894057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:29.902 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.895+0000 7f48a7fff700 1 -- 172.21.15.32:0/1469424056 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f48a806f150 con 0x7f48b80af5f0 2020-12-05T17:32:29.902 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.895+0000 7f48b67fc700 1 --2- 172.21.15.32:0/1469424056 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4894055470 0x7f4894057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:29.902 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.895+0000 7f48b67fc700 1 --2- 172.21.15.32:0/1469424056 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4894055470 0x7f4894057930 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f48ac00b000 tx=0x7f48ac006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:29.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:29.903+0000 7f48a7fff700 1 -- 172.21.15.32:0/1469424056 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f48a80451e0 con 0x7f48b80af5f0 2020-12-05T17:32:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.051+0000 7f48b7fff700 1 -- 172.21.15.32:0/1469424056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f48b8057050 con 0x7f48b80af5f0 2020-12-05T17:32:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.051+0000 7f48a7fff700 1 -- 172.21.15.32:0/1469424056 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v34) v1 ==== 92+0+10 (secure 0 0 0) 0x7f48a80220a0 con 0x7f48b80af5f0 2020-12-05T17:32:30.053 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:32:30.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.051+0000 7f48b7fff700 1 -- 172.21.15.32:0/1469424056 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4894055470 msgr2=0x7f4894057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:30.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.051+0000 7f48b7fff700 1 --2- 172.21.15.32:0/1469424056 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4894055470 0x7f4894057930 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f48ac00b000 tx=0x7f48ac006dc0).stop 2020-12-05T17:32:30.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.051+0000 7f48b7fff700 1 -- 172.21.15.32:0/1469424056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b80af5f0 msgr2=0x7f48b8144110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:30.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.051+0000 7f48b7fff700 1 --2- 172.21.15.32:0/1469424056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b80af5f0 0x7f48b8144110 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f48a8002f30 tx=0x7f48a8015000).stop 2020-12-05T17:32:30.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.051+0000 7f48b7fff700 1 -- 172.21.15.32:0/1469424056 shutdown_connections 2020-12-05T17:32:30.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.051+0000 7f48b7fff700 1 --2- 172.21.15.32:0/1469424056 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48b80b1dc0 0x7f48b8063f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.051+0000 7f48b7fff700 1 --2- 172.21.15.32:0/1469424056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b80af5f0 0x7f48b8144110 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.051+0000 7f48b7fff700 1 --2- 172.21.15.32:0/1469424056 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4894055470 0x7f4894057930 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.051+0000 7f48b7fff700 1 --2- 172.21.15.32:0/1469424056 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48b80afd30 0x7f48b8149670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.051+0000 7f48b7fff700 1 -- 172.21.15.32:0/1469424056 >> 172.21.15.32:0/1469424056 conn(0x7f48b80620b0 msgr2=0x7f48b80b01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.051+0000 7f48b7fff700 1 -- 172.21.15.32:0/1469424056 shutdown_connections 2020-12-05T17:32:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.051+0000 7f48b7fff700 1 -- 172.21.15.32:0/1469424056 wait complete. 2020-12-05T17:32:30.067 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:32:30.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.179+0000 7f0795608700 1 Processor -- start 2020-12-05T17:32:30.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.179+0000 7f0795608700 1 -- start start 2020-12-05T17:32:30.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.179+0000 7f0795608700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07900b5b20 0x7f07900b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:30.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.179+0000 7f0795608700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07900b6690 0x7f07900b5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:30.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.183+0000 7f0795608700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07900b1920 con 0x7f07900b2060 2020-12-05T17:32:30.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.183+0000 7f0795608700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07900b5730 con 0x7f07900b6690 2020-12-05T17:32:30.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.183+0000 7f0795608700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07900b1bc0 con 0x7f07900b5b20 2020-12-05T17:32:30.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.183+0000 7f078e7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07900b6690 0x7f07900b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:30.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.183+0000 7f078e7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07900b6690 0x7f07900b5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34082/0 (socket says 172.21.15.32:34082) 2020-12-05T17:32:30.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.183+0000 7f078e7fc700 1 -- 172.21.15.32:0/2584960654 learned_addr learned my addr 172.21.15.32:0/2584960654 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:30.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.183+0000 7f078effd700 1 --2- 172.21.15.32:0/2584960654 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07900b5b20 0x7f07900b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:30.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.183+0000 7f078e7fc700 1 -- 172.21.15.32:0/2584960654 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07900b5b20 msgr2=0x7f07900b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:30.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.183+0000 7f078e7fc700 1 --2- 172.21.15.32:0/2584960654 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07900b5b20 0x7f07900b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.183+0000 7f078e7fc700 1 -- 172.21.15.32:0/2584960654 >> v1:172.21.15.32:6789/0 conn(0x7f07900b2060 legacy=0x7f0790145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:30.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.183+0000 7f078e7fc700 1 -- 172.21.15.32:0/2584960654 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07900b24c0 con 0x7f07900b6690 2020-12-05T17:32:30.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.183+0000 7f078e7fc700 1 --2- 172.21.15.32:0/2584960654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07900b6690 0x7f07900b5240 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f0784002f30 tx=0x7f0784006e80).ready entity=mon.0 client_cookie=3232d12303b613e8 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:30.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.183+0000 7f078dffb700 1 -- 172.21.15.32:0/2584960654 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07840097f0 con 0x7f07900b6690 2020-12-05T17:32:30.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.183+0000 7f078dffb700 1 -- 172.21.15.32:0/2584960654 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0784009950 con 0x7f07900b6690 2020-12-05T17:32:30.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.183+0000 7f078dffb700 1 -- 172.21.15.32:0/2584960654 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0784009cb0 con 0x7f07900b6690 2020-12-05T17:32:30.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.183+0000 7f0795608700 1 -- 172.21.15.32:0/2584960654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07900b6690 msgr2=0x7f07900b5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:30.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.183+0000 7f0795608700 1 --2- 172.21.15.32:0/2584960654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07900b6690 0x7f07900b5240 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f0784002f30 tx=0x7f0784006e80).stop 2020-12-05T17:32:30.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.183+0000 7f0795608700 1 -- 172.21.15.32:0/2584960654 shutdown_connections 2020-12-05T17:32:30.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.183+0000 7f0795608700 1 --2- 172.21.15.32:0/2584960654 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07900b5b20 0x7f07900b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.183+0000 7f0795608700 1 --2- 172.21.15.32:0/2584960654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07900b6690 0x7f07900b5240 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.183+0000 7f0795608700 1 -- 172.21.15.32:0/2584960654 >> 172.21.15.32:0/2584960654 conn(0x7f079005f3b0 msgr2=0x7f079005f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:30.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.183+0000 7f0795608700 1 -- 172.21.15.32:0/2584960654 shutdown_connections 2020-12-05T17:32:30.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.183+0000 7f0795608700 1 -- 172.21.15.32:0/2584960654 wait complete. 2020-12-05T17:32:30.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.187+0000 7f0795608700 1 Processor -- start 2020-12-05T17:32:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.187+0000 7f0795608700 1 -- start start 2020-12-05T17:32:30.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.187+0000 7f0795608700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07900b2060 0x7f07900a27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:30.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.187+0000 7f0795608700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07900b5b20 0x7f07901467b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.187+0000 7f0795608700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07900b6690 0x7f0790148620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.187+0000 7f0795608700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0790147060 con 0x7f07900b5b20 2020-12-05T17:32:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.187+0000 7f0795608700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07901455f0 con 0x7f07900b2060 2020-12-05T17:32:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.187+0000 7f0795608700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0790146e00 con 0x7f07900b6690 2020-12-05T17:32:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.187+0000 7f078e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07900b5b20 0x7f07901467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:30.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.187+0000 7f078e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07900b5b20 0x7f07901467b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34088/0 (socket says 172.21.15.32:34088) 2020-12-05T17:32:30.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.187+0000 7f078e7fc700 1 -- 172.21.15.32:0/1203316358 learned_addr learned my addr 172.21.15.32:0/1203316358 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:30.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.187+0000 7f078effd700 1 --2- 172.21.15.32:0/1203316358 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07900b2060 0x7f07900a27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:30.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.187+0000 7f078f7fe700 1 --2- 172.21.15.32:0/1203316358 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07900b6690 0x7f0790148620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:30.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.187+0000 7f078e7fc700 1 -- 172.21.15.32:0/1203316358 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07900b6690 msgr2=0x7f0790148620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:30.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.187+0000 7f078e7fc700 1 --2- 172.21.15.32:0/1203316358 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07900b6690 0x7f0790148620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.187+0000 7f078e7fc700 1 -- 172.21.15.32:0/1203316358 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07900b2060 msgr2=0x7f07900a27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:30.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.187+0000 7f078e7fc700 1 --2- 172.21.15.32:0/1203316358 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07900b2060 0x7f07900a27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.187+0000 7f078e7fc700 1 -- 172.21.15.32:0/1203316358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0790145f70 con 0x7f07900b5b20 2020-12-05T17:32:30.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.187+0000 7f078effd700 1 --2- 172.21.15.32:0/1203316358 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07900b2060 0x7f07900a27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:30.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.187+0000 7f078e7fc700 1 --2- 172.21.15.32:0/1203316358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07900b5b20 0x7f07901467b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f0784002f30 tx=0x7f0784015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:30.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.187+0000 7f077ffff700 1 -- 172.21.15.32:0/1203316358 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0784011ce0 con 0x7f07900b5b20 2020-12-05T17:32:30.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.191+0000 7f077ffff700 1 -- 172.21.15.32:0/1203316358 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0784011e40 con 0x7f07900b5b20 2020-12-05T17:32:30.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.191+0000 7f0795608700 1 -- 172.21.15.32:0/1203316358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0790146210 con 0x7f07900b5b20 2020-12-05T17:32:30.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.191+0000 7f077ffff700 1 -- 172.21.15.32:0/1203316358 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f078401e650 con 0x7f07900b5b20 2020-12-05T17:32:30.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.191+0000 7f0795608700 1 -- 172.21.15.32:0/1203316358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0790144cf0 con 0x7f07900b5b20 2020-12-05T17:32:30.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.191+0000 7f0795608700 1 -- 172.21.15.32:0/1203316358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0790057050 con 0x7f07900b5b20 2020-12-05T17:32:30.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.191+0000 7f077ffff700 1 -- 172.21.15.32:0/1203316358 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0784025030 con 0x7f07900b5b20 2020-12-05T17:32:30.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.191+0000 7f077ffff700 1 --2- 172.21.15.32:0/1203316358 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f076c055440 0x7f076c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:30.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.191+0000 7f077ffff700 1 -- 172.21.15.32:0/1203316358 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f078406f470 con 0x7f07900b5b20 2020-12-05T17:32:30.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.191+0000 7f078effd700 1 --2- 172.21.15.32:0/1203316358 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f076c055440 0x7f076c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:30.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.191+0000 7f078effd700 1 --2- 172.21.15.32:0/1203316358 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f076c055440 0x7f076c057900 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f078000b000 tx=0x7f0780008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:30.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.199+0000 7f077ffff700 1 -- 172.21.15.32:0/1203316358 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0784045610 con 0x7f07900b5b20 2020-12-05T17:32:30.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.339+0000 7f0795608700 1 -- 172.21.15.32:0/1203316358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f079005bae0 con 0x7f07900b5b20 2020-12-05T17:32:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.339+0000 7f077ffff700 1 -- 172.21.15.32:0/1203316358 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0784048c30 con 0x7f07900b5b20 2020-12-05T17:32:30.342 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:32:30.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.339+0000 7f0795608700 1 -- 172.21.15.32:0/1203316358 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f076c055440 msgr2=0x7f076c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.339+0000 7f0795608700 1 --2- 172.21.15.32:0/1203316358 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f076c055440 0x7f076c057900 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f078000b000 tx=0x7f0780008040).stop 2020-12-05T17:32:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.339+0000 7f0795608700 1 -- 172.21.15.32:0/1203316358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07900b5b20 msgr2=0x7f07901467b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.339+0000 7f0795608700 1 --2- 172.21.15.32:0/1203316358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07900b5b20 0x7f07901467b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f0784002f30 tx=0x7f0784015000).stop 2020-12-05T17:32:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.343+0000 7f0795608700 1 -- 172.21.15.32:0/1203316358 shutdown_connections 2020-12-05T17:32:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.343+0000 7f0795608700 1 --2- 172.21.15.32:0/1203316358 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f076c055440 0x7f076c057900 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.343+0000 7f0795608700 1 --2- 172.21.15.32:0/1203316358 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07900b6690 0x7f0790148620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.343+0000 7f0795608700 1 --2- 172.21.15.32:0/1203316358 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07900b2060 0x7f07900a27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.343+0000 7f0795608700 1 --2- 172.21.15.32:0/1203316358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07900b5b20 0x7f07901467b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.343+0000 7f0795608700 1 -- 172.21.15.32:0/1203316358 >> 172.21.15.32:0/1203316358 conn(0x7f079005f3b0 msgr2=0x7f07900b6f20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:30.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.343+0000 7f0795608700 1 -- 172.21.15.32:0/1203316358 shutdown_connections 2020-12-05T17:32:30.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.343+0000 7f0795608700 1 -- 172.21.15.32:0/1203316358 wait complete. 2020-12-05T17:32:30.361 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:32:30.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72c695b700 1 Processor -- start 2020-12-05T17:32:30.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72c695b700 1 -- start start 2020-12-05T17:32:30.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72c695b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f72c00b2690 0x7f72c00b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:30.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72c695b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72c00b0010 0x7f72c0146280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:30.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72c695b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c00b06b0 con 0x7f72c00b15e0 2020-12-05T17:32:30.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72c695b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c00b04a0 con 0x7f72c00b0010 2020-12-05T17:32:30.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72c695b700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c014a280 con 0x7f72c00b2690 2020-12-05T17:32:30.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72c4ef8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72c00b0010 0x7f72c0146280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:30.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72c4ef8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72c00b0010 0x7f72c0146280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34096/0 (socket says 172.21.15.32:34096) 2020-12-05T17:32:30.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72bf7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f72c00b15e0 0x7f72c014a9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36746/0 (socket says 172.21.15.32:36746) 2020-12-05T17:32:30.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72c4ef8700 1 -- 172.21.15.32:0/3086401590 learned_addr learned my addr 172.21.15.32:0/3086401590 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:30.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72bffff700 1 --2- 172.21.15.32:0/3086401590 >> v2:172.21.15.61:3300/0 conn(0x7f72c00b2690 0x7f72c00b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:30.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72c4ef8700 1 -- 172.21.15.32:0/3086401590 >> v2:172.21.15.61:3300/0 conn(0x7f72c00b2690 msgr2=0x7f72c00b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:30.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72c4ef8700 1 --2- 172.21.15.32:0/3086401590 >> v2:172.21.15.61:3300/0 conn(0x7f72c00b2690 0x7f72c00b2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72c4ef8700 1 -- 172.21.15.32:0/3086401590 >> v1:172.21.15.32:6789/0 conn(0x7f72c00b15e0 legacy=0x7f72c014a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:30.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72c4ef8700 1 -- 172.21.15.32:0/3086401590 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c014a520 con 0x7f72c00b0010 2020-12-05T17:32:30.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72c4ef8700 1 --2- 172.21.15.32:0/3086401590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c00b0010 0x7f72c0146280 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f72b80032b0 tx=0x7f72b8009270).ready entity=mon.0 client_cookie=50893af780d90b40 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:30.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72beffd700 1 -- 172.21.15.32:0/3086401590 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72b800bc00 con 0x7f72c00b0010 2020-12-05T17:32:30.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72beffd700 1 -- 172.21.15.32:0/3086401590 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b800bd60 con 0x7f72c00b0010 2020-12-05T17:32:30.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72beffd700 1 -- 172.21.15.32:0/3086401590 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72b8011400 con 0x7f72c00b0010 2020-12-05T17:32:30.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72c695b700 1 -- 172.21.15.32:0/3086401590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c00b0010 msgr2=0x7f72c0146280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:30.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72c695b700 1 --2- 172.21.15.32:0/3086401590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c00b0010 0x7f72c0146280 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f72b80032b0 tx=0x7f72b8009270).stop 2020-12-05T17:32:30.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72c695b700 1 -- 172.21.15.32:0/3086401590 shutdown_connections 2020-12-05T17:32:30.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72c695b700 1 --2- 172.21.15.32:0/3086401590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c00b0010 0x7f72c0146280 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72c695b700 1 --2- 172.21.15.32:0/3086401590 >> v2:172.21.15.61:3300/0 conn(0x7f72c00b2690 0x7f72c00b2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72c695b700 1 -- 172.21.15.32:0/3086401590 >> 172.21.15.32:0/3086401590 conn(0x7f72c0062310 msgr2=0x7f72c00626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:30.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72c695b700 1 -- 172.21.15.32:0/3086401590 shutdown_connections 2020-12-05T17:32:30.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.479+0000 7f72c695b700 1 -- 172.21.15.32:0/3086401590 wait complete. 2020-12-05T17:32:30.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.483+0000 7f72c695b700 1 Processor -- start 2020-12-05T17:32:30.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.483+0000 7f72c695b700 1 -- start start 2020-12-05T17:32:30.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72c695b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c00b0010 0x7f72c0146d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:30.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72c695b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72c00b15e0 0x7f72c0064050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:30.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72c695b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72c00b2690 0x7f72c01596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:30.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72c695b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c0064540 con 0x7f72c00b0010 2020-12-05T17:32:30.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72bffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c00b0010 0x7f72c0146d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:30.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72c695b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c0159be0 con 0x7f72c00b15e0 2020-12-05T17:32:30.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72c695b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c0064760 con 0x7f72c00b2690 2020-12-05T17:32:30.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72bffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c00b0010 0x7f72c0146d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34100/0 (socket says 172.21.15.32:34100) 2020-12-05T17:32:30.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72bffff700 1 -- 172.21.15.32:0/1693048102 learned_addr learned my addr 172.21.15.32:0/1693048102 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:30.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72bf7fe700 1 --2- 172.21.15.32:0/1693048102 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72c00b15e0 0x7f72c0064050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:30.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72c4ef8700 1 --2- 172.21.15.32:0/1693048102 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72c00b2690 0x7f72c01596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:30.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72bffff700 1 -- 172.21.15.32:0/1693048102 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72c00b2690 msgr2=0x7f72c01596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:30.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72bffff700 1 --2- 172.21.15.32:0/1693048102 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72c00b2690 0x7f72c01596f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72bffff700 1 -- 172.21.15.32:0/1693048102 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72c00b15e0 msgr2=0x7f72c0064050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:30.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72bffff700 1 --2- 172.21.15.32:0/1693048102 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72c00b15e0 0x7f72c0064050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72bffff700 1 -- 172.21.15.32:0/1693048102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c0064a00 con 0x7f72c00b0010 2020-12-05T17:32:30.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72bf7fe700 1 --2- 172.21.15.32:0/1693048102 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72c00b15e0 0x7f72c0064050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:30.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72bffff700 1 --2- 172.21.15.32:0/1693048102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c00b0010 0x7f72c0146d90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f72b0003070 tx=0x7f72b0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:30.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72bd7fa700 1 -- 172.21.15.32:0/1693048102 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72b0009980 con 0x7f72c00b0010 2020-12-05T17:32:30.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72bd7fa700 1 -- 172.21.15.32:0/1693048102 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b0009ae0 con 0x7f72c00b0010 2020-12-05T17:32:30.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72c695b700 1 -- 172.21.15.32:0/1693048102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c01178b0 con 0x7f72c00b0010 2020-12-05T17:32:30.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72bd7fa700 1 -- 172.21.15.32:0/1693048102 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72b0009e40 con 0x7f72c00b0010 2020-12-05T17:32:30.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72c695b700 1 -- 172.21.15.32:0/1693048102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c0065630 con 0x7f72c00b0010 2020-12-05T17:32:30.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.487+0000 7f72c695b700 1 -- 172.21.15.32:0/1693048102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c0057050 con 0x7f72c00b0010 2020-12-05T17:32:30.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.491+0000 7f72bd7fa700 1 -- 172.21.15.32:0/1693048102 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f72b0013480 con 0x7f72c00b0010 2020-12-05T17:32:30.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.491+0000 7f72bd7fa700 1 --2- 172.21.15.32:0/1693048102 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f72a00554e0 0x7f72a00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:30.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.491+0000 7f72bf7fe700 1 --2- 172.21.15.32:0/1693048102 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f72a00554e0 0x7f72a00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:30.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.491+0000 7f72bd7fa700 1 -- 172.21.15.32:0/1693048102 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f72b006e7b0 con 0x7f72c00b0010 2020-12-05T17:32:30.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.491+0000 7f72bf7fe700 1 --2- 172.21.15.32:0/1693048102 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f72a00554e0 0x7f72a00579a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f72b400b000 tx=0x7f72b40070e0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:30.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.495+0000 7f72bd7fa700 1 -- 172.21.15.32:0/1693048102 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f72b00448f0 con 0x7f72c00b0010 2020-12-05T17:32:30.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.639+0000 7f72c695b700 1 -- 172.21.15.32:0/1693048102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f72c00b2da0 con 0x7f72c00b0010 2020-12-05T17:32:30.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.643+0000 7f72bd7fa700 1 -- 172.21.15.32:0/1693048102 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+10 (secure 0 0 0) 0x7f72b0047f10 con 0x7f72c00b0010 2020-12-05T17:32:30.645 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:32:30.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.643+0000 7f72c695b700 1 -- 172.21.15.32:0/1693048102 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f72a00554e0 msgr2=0x7f72a00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:30.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.643+0000 7f72c695b700 1 --2- 172.21.15.32:0/1693048102 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f72a00554e0 0x7f72a00579a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f72b400b000 tx=0x7f72b40070e0).stop 2020-12-05T17:32:30.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.643+0000 7f72c695b700 1 -- 172.21.15.32:0/1693048102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c00b0010 msgr2=0x7f72c0146d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:30.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.643+0000 7f72c695b700 1 --2- 172.21.15.32:0/1693048102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c00b0010 0x7f72c0146d90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f72b0003070 tx=0x7f72b0008040).stop 2020-12-05T17:32:30.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.643+0000 7f72c695b700 1 -- 172.21.15.32:0/1693048102 shutdown_connections 2020-12-05T17:32:30.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.643+0000 7f72c695b700 1 --2- 172.21.15.32:0/1693048102 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f72a00554e0 0x7f72a00579a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.643+0000 7f72c695b700 1 --2- 172.21.15.32:0/1693048102 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72c00b2690 0x7f72c01596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.643+0000 7f72c695b700 1 --2- 172.21.15.32:0/1693048102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c00b0010 0x7f72c0146d90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.643+0000 7f72c695b700 1 --2- 172.21.15.32:0/1693048102 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72c00b15e0 0x7f72c0064050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.643+0000 7f72c695b700 1 -- 172.21.15.32:0/1693048102 >> 172.21.15.32:0/1693048102 conn(0x7f72c0062310 msgr2=0x7f72c00ae1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:30.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.643+0000 7f72c695b700 1 -- 172.21.15.32:0/1693048102 shutdown_connections 2020-12-05T17:32:30.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.643+0000 7f72c695b700 1 -- 172.21.15.32:0/1693048102 wait complete. 2020-12-05T17:32:30.664 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:32:30.665 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:32:30.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.775+0000 7f96affff700 1 Processor -- start 2020-12-05T17:32:30.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.775+0000 7f96affff700 1 -- start start 2020-12-05T17:32:30.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.775+0000 7f96affff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f96b00b7720 0x7f96b0061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:30.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.775+0000 7f96affff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96b01403f0 0x7f96b0143e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:30.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.775+0000 7f96affff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b0143a00 con 0x7f96b0144320 2020-12-05T17:32:30.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.775+0000 7f96affff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b00b7b00 con 0x7f96b00b7720 2020-12-05T17:32:30.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.775+0000 7f96affff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b005fe60 con 0x7f96b01403f0 2020-12-05T17:32:30.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.779+0000 7f96ae7fc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96b01403f0 0x7f96b0143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:30.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.779+0000 7f96af7fe700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f96b0144320 0x7f96b013d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:49400/0 (socket says 172.21.15.32:49400) 2020-12-05T17:32:30.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.779+0000 7f96aeffd700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f96b00b7720 0x7f96b0061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:30.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.779+0000 7f96af7fe700 1 -- 172.21.15.32:0/1396740467 learned_addr learned my addr 172.21.15.32:0/1396740467 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:30.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.779+0000 7f96ae7fc700 1 -- 172.21.15.32:0/1396740467 >> v2:172.21.15.61:3300/0 conn(0x7f96b00b7720 msgr2=0x7f96b0061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:30.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.779+0000 7f96ae7fc700 1 --2- 172.21.15.32:0/1396740467 >> v2:172.21.15.61:3300/0 conn(0x7f96b00b7720 0x7f96b0061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.779+0000 7f96ae7fc700 1 -- 172.21.15.32:0/1396740467 >> v1:172.21.15.61:6789/0 conn(0x7f96b0144320 legacy=0x7f96b013d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:30.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.779+0000 7f96ae7fc700 1 -- 172.21.15.32:0/1396740467 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b0140850 con 0x7f96b01403f0 2020-12-05T17:32:30.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.779+0000 7f96ae7fc700 1 --2- 172.21.15.32:0/1396740467 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96b01403f0 0x7f96b0143e30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f9698002f30 tx=0x7f9698006dc0).ready entity=mon.2 client_cookie=ad6cf694e3b04527 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:30.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.779+0000 7f96adffb700 1 -- 172.21.15.32:0/1396740467 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9698009760 con 0x7f96b01403f0 2020-12-05T17:32:30.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.779+0000 7f96adffb700 1 -- 172.21.15.32:0/1396740467 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96980098c0 con 0x7f96b01403f0 2020-12-05T17:32:30.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.779+0000 7f96adffb700 1 -- 172.21.15.32:0/1396740467 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9698009c20 con 0x7f96b01403f0 2020-12-05T17:32:30.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.779+0000 7f96affff700 1 -- 172.21.15.32:0/1396740467 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96b01403f0 msgr2=0x7f96b0143e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:30.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.779+0000 7f96affff700 1 --2- 172.21.15.32:0/1396740467 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96b01403f0 0x7f96b0143e30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f9698002f30 tx=0x7f9698006dc0).stop 2020-12-05T17:32:30.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.779+0000 7f96affff700 1 -- 172.21.15.32:0/1396740467 shutdown_connections 2020-12-05T17:32:30.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.779+0000 7f96affff700 1 --2- 172.21.15.32:0/1396740467 >> v2:172.21.15.61:3300/0 conn(0x7f96b00b7720 0x7f96b0061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.779+0000 7f96affff700 1 --2- 172.21.15.32:0/1396740467 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96b01403f0 0x7f96b0143e30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.779+0000 7f96affff700 1 -- 172.21.15.32:0/1396740467 >> 172.21.15.32:0/1396740467 conn(0x7f96b005bc20 msgr2=0x7f96b00b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:30.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.779+0000 7f96affff700 1 -- 172.21.15.32:0/1396740467 shutdown_connections 2020-12-05T17:32:30.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.779+0000 7f96affff700 1 -- 172.21.15.32:0/1396740467 wait complete. 2020-12-05T17:32:30.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96affff700 1 Processor -- start 2020-12-05T17:32:30.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96affff700 1 -- start start 2020-12-05T17:32:30.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96affff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96b00b7720 0x7f96b0137130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:30.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96affff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96b01403f0 0x7f96b0151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:30.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96affff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96b0144320 0x7f96b0151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96affff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00b1ac0 con 0x7f96b00b7720 2020-12-05T17:32:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96affff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00b33b0 con 0x7f96b01403f0 2020-12-05T17:32:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96affff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00b2e70 con 0x7f96b0144320 2020-12-05T17:32:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96aeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96b00b7720 0x7f96b0137130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:30.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96aeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96b00b7720 0x7f96b0137130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34114/0 (socket says 172.21.15.32:34114) 2020-12-05T17:32:30.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96aeffd700 1 -- 172.21.15.32:0/3841353656 learned_addr learned my addr 172.21.15.32:0/3841353656 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:30.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96ae7fc700 1 --2- 172.21.15.32:0/3841353656 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96b01403f0 0x7f96b0151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:30.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96af7fe700 1 --2- 172.21.15.32:0/3841353656 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96b0144320 0x7f96b0151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:30.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96aeffd700 1 -- 172.21.15.32:0/3841353656 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96b0144320 msgr2=0x7f96b0151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:30.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96aeffd700 1 --2- 172.21.15.32:0/3841353656 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96b0144320 0x7f96b0151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96aeffd700 1 -- 172.21.15.32:0/3841353656 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96b01403f0 msgr2=0x7f96b0151910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:30.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96aeffd700 1 --2- 172.21.15.32:0/3841353656 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96b01403f0 0x7f96b0151910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96aeffd700 1 -- 172.21.15.32:0/3841353656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b00b3110 con 0x7f96b00b7720 2020-12-05T17:32:30.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96ae7fc700 1 --2- 172.21.15.32:0/3841353656 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96b01403f0 0x7f96b0151910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:30.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96aeffd700 1 --2- 172.21.15.32:0/3841353656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96b00b7720 0x7f96b0137130 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f96a000c000 tx=0x7f96a0006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:30.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96a7fff700 1 -- 172.21.15.32:0/3841353656 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96a000ca80 con 0x7f96b00b7720 2020-12-05T17:32:30.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96a7fff700 1 -- 172.21.15.32:0/3841353656 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a000cbe0 con 0x7f96b00b7720 2020-12-05T17:32:30.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96affff700 1 -- 172.21.15.32:0/3841353656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b01522f0 con 0x7f96b00b7720 2020-12-05T17:32:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96a7fff700 1 -- 172.21.15.32:0/3841353656 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96a000b370 con 0x7f96b00b7720 2020-12-05T17:32:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96affff700 1 -- 172.21.15.32:0/3841353656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b00b3a80 con 0x7f96b00b7720 2020-12-05T17:32:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.783+0000 7f96affff700 1 -- 172.21.15.32:0/3841353656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b0057050 con 0x7f96b00b7720 2020-12-05T17:32:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.787+0000 7f96a7fff700 1 -- 172.21.15.32:0/3841353656 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f96a001c460 con 0x7f96b00b7720 2020-12-05T17:32:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.787+0000 7f96a7fff700 1 --2- 172.21.15.32:0/3841353656 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9694055470 0x7f9694057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:30.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.787+0000 7f96ae7fc700 1 --2- 172.21.15.32:0/3841353656 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9694055470 0x7f9694057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:30.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.787+0000 7f96a7fff700 1 -- 172.21.15.32:0/3841353656 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f96a006f130 con 0x7f96b00b7720 2020-12-05T17:32:30.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.787+0000 7f96ae7fc700 1 --2- 172.21.15.32:0/3841353656 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9694055470 0x7f9694057930 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f9698018000 tx=0x7f9698015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:30.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.795+0000 7f96a7fff700 1 -- 172.21.15.32:0/3841353656 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f96a0045160 con 0x7f96b00b7720 2020-12-05T17:32:30.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.983+0000 7f96affff700 1 -- 172.21.15.32:0/3841353656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96b00b3d30 con 0x7f96b00b7720 2020-12-05T17:32:30.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.983+0000 7f96a7fff700 1 -- 172.21.15.32:0/3841353656 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2136 (secure 0 0 0) 0x7f96a0048780 con 0x7f96b00b7720 2020-12-05T17:32:30.986 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:32:30.987 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4679,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/2850341272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2850341272},{"type":"v1","addr":"172.21.15.61:6833","nonce":2850341272}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:32:25.792490+0000","modified":"2020-12-05T17:32:26.797858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4873},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4873":{"gid":4873,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3316547885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3316547885},{"type":"v1","addr":"172.21.15.32:6835","nonce":3316547885}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-12-05T17:32:30.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.983+0000 7f96affff700 1 -- 172.21.15.32:0/3841353656 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9694055470 msgr2=0x7f9694057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:30.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.983+0000 7f96affff700 1 --2- 172.21.15.32:0/3841353656 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9694055470 0x7f9694057930 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f9698018000 tx=0x7f9698015000).stop 2020-12-05T17:32:30.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.983+0000 7f96affff700 1 -- 172.21.15.32:0/3841353656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96b00b7720 msgr2=0x7f96b0137130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:30.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.983+0000 7f96affff700 1 --2- 172.21.15.32:0/3841353656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96b00b7720 0x7f96b0137130 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f96a000c000 tx=0x7f96a0006dc0).stop 2020-12-05T17:32:30.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.983+0000 7f96affff700 1 -- 172.21.15.32:0/3841353656 shutdown_connections 2020-12-05T17:32:30.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.983+0000 7f96affff700 1 --2- 172.21.15.32:0/3841353656 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9694055470 0x7f9694057930 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.983+0000 7f96affff700 1 --2- 172.21.15.32:0/3841353656 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96b0144320 0x7f96b0151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.983+0000 7f96affff700 1 --2- 172.21.15.32:0/3841353656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96b00b7720 0x7f96b0137130 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.983+0000 7f96affff700 1 --2- 172.21.15.32:0/3841353656 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96b01403f0 0x7f96b0151910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:30.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.983+0000 7f96affff700 1 -- 172.21.15.32:0/3841353656 >> 172.21.15.32:0/3841353656 conn(0x7f96b005bc20 msgr2=0x7f96b0143b90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:30.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.983+0000 7f96affff700 1 -- 172.21.15.32:0/3841353656 shutdown_connections 2020-12-05T17:32:30.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:30.983+0000 7f96affff700 1 -- 172.21.15.32:0/3841353656 wait complete. 2020-12-05T17:32:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 10 2020-12-05T17:32:31.201 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 10, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:32:25.792490+0000', 'modified': '2020-12-05T17:32:26.797858+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4873}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4873': {'gid': 4873, 'name': 'a', 'rank': 0, 'incarnation': 9, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/3316547885', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 3316547885}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 3316547885}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:32:31.201 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:32:31.202 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:32:31.339 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":4873,"want_state":"up:active","state":"up:active","rank_uptime":5.5317876750000003,"mdsmap_epoch":10,"osdmap_epoch":34,"osdmap_epoch_barrier":33,"uptime":12.204648288} 2020-12-05T17:32:31.340 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 4873, "mdsmap_epoch": 10, "osdmap_epoch": 34, "osdmap_epoch_barrier": 33, "rank_uptime": 5.531787675, "state": "up:active", "uptime": 12.204648288, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:32:31.340 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:32:31.341 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-12-05T17:32:31.341 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-12-05T17:32:31.341 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:31.342 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:32:31.354 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:32:31.417 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:32:31.417 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:32:31.474 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:32:31.475 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:32:31.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:31.476 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:32:31.529 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:32:31.529 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:32:31.578 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:32:31.598 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:32:31.598 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:32:31.656 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:32:31.659 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:32:31.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:31.660 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:32:31.662 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:2020-12-05T17:32:31.659+0000 7f87f63710c0 -1 init, newargv = 0x557b1fe97a50 newargc=9 2020-12-05T17:32:31.662 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[10361]: starting ceph client 2020-12-05T17:32:31.674 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[10361]: starting fuse 2020-12-05T17:32:32.772 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:32:32.790 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:32:32.791 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:32:32.847 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:32:32.849 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:32:32.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:32.850 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:32:32.899 INFO:teuthology.orchestra.run.smithi036.stdout:49 2020-12-05T17:32:32.899 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-12-05T17:32:32.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:32.901 INFO:teuthology.orchestra.run.smithi036:> sudo python3 -c ' 2020-12-05T17:32:32.901 INFO:teuthology.orchestra.run.smithi036:> import glob 2020-12-05T17:32:32.901 INFO:teuthology.orchestra.run.smithi036:> import re 2020-12-05T17:32:32.902 INFO:teuthology.orchestra.run.smithi036:> import os 2020-12-05T17:32:32.903 INFO:teuthology.orchestra.run.smithi036:> import subprocess 2020-12-05T17:32:32.903 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:32:32.904 INFO:teuthology.orchestra.run.smithi036:> def find_socket(client_name): 2020-12-05T17:32:32.905 INFO:teuthology.orchestra.run.smithi036:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-12-05T17:32:32.905 INFO:teuthology.orchestra.run.smithi036:> files = glob.glob(asok_path) 2020-12-05T17:32:32.905 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:32:32.906 INFO:teuthology.orchestra.run.smithi036:> # Given a non-glob path, it better be there 2020-12-05T17:32:32.906 INFO:teuthology.orchestra.run.smithi036:> if "*" not in asok_path: 2020-12-05T17:32:32.906 INFO:teuthology.orchestra.run.smithi036:> assert(len(files) == 1) 2020-12-05T17:32:32.907 INFO:teuthology.orchestra.run.smithi036:> return files[0] 2020-12-05T17:32:32.907 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:32:32.908 INFO:teuthology.orchestra.run.smithi036:> for f in files: 2020-12-05T17:32:32.909 INFO:teuthology.orchestra.run.smithi036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:32:32.909 INFO:teuthology.orchestra.run.smithi036:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:32:32.910 INFO:teuthology.orchestra.run.smithi036:> return f 2020-12-05T17:32:32.910 INFO:teuthology.orchestra.run.smithi036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:32:32.911 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:32:32.911 INFO:teuthology.orchestra.run.smithi036:> print(find_socket("client.0")) 2020-12-05T17:32:32.912 INFO:teuthology.orchestra.run.smithi036:> ' 2020-12-05T17:32:32.990 INFO:teuthology.orchestra.run.smithi036.stdout:/var/run/ceph/ceph-client.0.10361.asok 2020-12-05T17:32:32.995 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.10361.asok 2020-12-05T17:32:32.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:32.996 INFO:teuthology.orchestra.run.smithi036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.10361.asok status 2020-12-05T17:32:33.094 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2020-12-05T17:32:33.095 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": { 2020-12-05T17:32:33.095 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:32:33.095 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:32:33.095 INFO:teuthology.orchestra.run.smithi036.stdout: "entity_id": "0", 2020-12-05T17:32:33.096 INFO:teuthology.orchestra.run.smithi036.stdout: "hostname": "smithi036", 2020-12-05T17:32:33.096 INFO:teuthology.orchestra.run.smithi036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-12-05T17:32:33.096 INFO:teuthology.orchestra.run.smithi036.stdout: "pid": "10361", 2020-12-05T17:32:33.096 INFO:teuthology.orchestra.run.smithi036.stdout: "root": "/" 2020-12-05T17:32:33.097 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:32:33.097 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_count": 0, 2020-12-05T17:32:33.097 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_pinned_count": 0, 2020-12-05T17:32:33.097 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 5169, 2020-12-05T17:32:33.097 INFO:teuthology.orchestra.run.smithi036.stdout: "inst": { 2020-12-05T17:32:33.098 INFO:teuthology.orchestra.run.smithi036.stdout: "name": { 2020-12-05T17:32:33.098 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "client", 2020-12-05T17:32:33.098 INFO:teuthology.orchestra.run.smithi036.stdout: "num": 5169 2020-12-05T17:32:33.098 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:32:33.098 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:32:33.099 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:32:33.099 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:32:33.099 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 467138591 2020-12-05T17:32:33.099 INFO:teuthology.orchestra.run.smithi036.stdout: } 2020-12-05T17:32:33.099 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:32:33.100 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:32:33.100 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:32:33.100 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:32:33.100 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 467138591 2020-12-05T17:32:33.100 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:32:33.101 INFO:teuthology.orchestra.run.smithi036.stdout: "inst_str": "client.5169 172.21.15.36:0/467138591", 2020-12-05T17:32:33.101 INFO:teuthology.orchestra.run.smithi036.stdout: "addr_str": "172.21.15.36:0/467138591", 2020-12-05T17:32:33.101 INFO:teuthology.orchestra.run.smithi036.stdout: "inode_count": 1, 2020-12-05T17:32:33.101 INFO:teuthology.orchestra.run.smithi036.stdout: "mds_epoch": 10, 2020-12-05T17:32:33.102 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch": 34, 2020-12-05T17:32:33.102 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:32:33.102 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklisted": false 2020-12-05T17:32:33.102 INFO:teuthology.orchestra.run.smithi036.stdout:} 2020-12-05T17:32:33.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:33.104 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:32:33.104 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:32:33.112 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:32:33.112 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:32:33.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:33.113 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:32:33.173 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_min_caps_per_client 2020-12-05T17:32:33.290 INFO:teuthology.orchestra.run.smithi032.stdout:{"mds_min_caps_per_client":"100"} 2020-12-05T17:32:33.291 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_min_caps_per_client": "100" } 2020-12-05T17:32:33.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:32:33.292 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:32:33.292 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:32:33.301 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:32:33.303 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:32:33.305 INFO:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-12-05T17:32:33.305 INFO:teuthology.orchestra.run.smithi036:> import sys 2020-12-05T17:32:33.306 INFO:teuthology.orchestra.run.smithi036:> import time 2020-12-05T17:32:33.307 INFO:teuthology.orchestra.run.smithi036:> import os 2020-12-05T17:32:33.308 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:32:33.308 INFO:teuthology.orchestra.run.smithi036:> n = 1000 2020-12-05T17:32:33.309 INFO:teuthology.orchestra.run.smithi036:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2020-12-05T17:32:33.310 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:32:33.310 INFO:teuthology.orchestra.run.smithi036:> if not os.path.exists(os.path.dirname(abs_path)): 2020-12-05T17:32:33.311 INFO:teuthology.orchestra.run.smithi036:> os.makedirs(os.path.dirname(abs_path)) 2020-12-05T17:32:33.312 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:32:33.313 INFO:teuthology.orchestra.run.smithi036:> for i in range(0, n): 2020-12-05T17:32:33.313 INFO:teuthology.orchestra.run.smithi036:> fname = "{0}_{1}".format(abs_path, i) 2020-12-05T17:32:33.314 INFO:teuthology.orchestra.run.smithi036:> with open(fname, '"'"'w'"'"') as f: 2020-12-05T17:32:33.315 INFO:teuthology.orchestra.run.smithi036:> f.write('"'"'content'"'"') 2020-12-05T17:32:33.315 INFO:teuthology.orchestra.run.smithi036:> if True: 2020-12-05T17:32:33.316 INFO:teuthology.orchestra.run.smithi036:> f.flush() 2020-12-05T17:32:33.317 INFO:teuthology.orchestra.run.smithi036:> os.fsync(f.fileno()) 2020-12-05T17:32:33.318 INFO:teuthology.orchestra.run.smithi036:> ' 2020-12-05T17:32:44.850 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:32:44.853 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:32:44.855 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:32:44.857 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:32:44.859 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:32:46.231 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:32:46.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a8a63700 1 Processor -- start 2020-12-05T17:32:46.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a8a63700 1 -- start start 2020-12-05T17:32:46.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a8a63700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5a405f3b0 0x7fa5a405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a8a63700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a4142450 con 0x7fa5a4062400 2020-12-05T17:32:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a8a63700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a4061780 con 0x7fa5a405feb0 2020-12-05T17:32:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a8a63700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a4061a20 con 0x7fa5a405f3b0 2020-12-05T17:32:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a2ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa5a4062400 0x7fa5a40b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36774/0 (socket says 172.21.15.32:36774) 2020-12-05T17:32:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a2ffd700 1 -- 172.21.15.32:0/3626358978 learned_addr learned my addr 172.21.15.32:0/3626358978 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:46.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a37fe700 1 --2- 172.21.15.32:0/3626358978 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5a405f3b0 0x7fa5a405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:46.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a27fc700 1 -- 172.21.15.32:0/3626358978 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3394854630 0 0) 0x7fa5a4142450 con 0x7fa5a4062400 2020-12-05T17:32:46.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a27fc700 1 -- 172.21.15.32:0/3626358978 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa588003230 con 0x7fa5a4062400 2020-12-05T17:32:46.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a37fe700 1 -- 172.21.15.32:0/3626358978 >> v1:172.21.15.61:6789/0 conn(0x7fa5a405feb0 legacy=0x7fa5a40b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:46.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a37fe700 1 -- 172.21.15.32:0/3626358978 >> v1:172.21.15.32:6789/0 conn(0x7fa5a4062400 legacy=0x7fa5a40b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:46.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a27fc700 1 -- 172.21.15.32:0/3626358978 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4003768443 0 0) 0x7fa5a4061780 con 0x7fa5a405feb0 2020-12-05T17:32:46.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a37fe700 1 -- 172.21.15.32:0/3626358978 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a4061cc0 con 0x7fa5a405f3b0 2020-12-05T17:32:46.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a37fe700 1 --2- 172.21.15.32:0/3626358978 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5a405f3b0 0x7fa5a405f790 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fa594002f30 tx=0x7fa594006dc0).ready entity=mon.2 client_cookie=fb0072dec8ed2510 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:46.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a27fc700 1 -- 172.21.15.32:0/3626358978 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5940097b0 con 0x7fa5a405f3b0 2020-12-05T17:32:46.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a27fc700 1 -- 172.21.15.32:0/3626358978 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa594009910 con 0x7fa5a405f3b0 2020-12-05T17:32:46.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a27fc700 1 -- 172.21.15.32:0/3626358978 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa594009c70 con 0x7fa5a405f3b0 2020-12-05T17:32:46.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a8a63700 1 -- 172.21.15.32:0/3626358978 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5a405f3b0 msgr2=0x7fa5a405f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:46.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a8a63700 1 --2- 172.21.15.32:0/3626358978 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5a405f3b0 0x7fa5a405f790 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fa594002f30 tx=0x7fa594006dc0).stop 2020-12-05T17:32:46.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a8a63700 1 -- 172.21.15.32:0/3626358978 shutdown_connections 2020-12-05T17:32:46.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a8a63700 1 --2- 172.21.15.32:0/3626358978 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5a405f3b0 0x7fa5a405f790 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:46.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.351+0000 7fa5a8a63700 1 -- 172.21.15.32:0/3626358978 >> 172.21.15.32:0/3626358978 conn(0x7fa5a40b8ee0 msgr2=0x7fa5a40b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:46.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.355+0000 7fa5a8a63700 1 -- 172.21.15.32:0/3626358978 shutdown_connections 2020-12-05T17:32:46.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.355+0000 7fa5a8a63700 1 -- 172.21.15.32:0/3626358978 wait complete. 2020-12-05T17:32:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.355+0000 7fa5a8a63700 1 Processor -- start 2020-12-05T17:32:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.355+0000 7fa5a8a63700 1 -- start start 2020-12-05T17:32:46.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.355+0000 7fa5a8a63700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5a405f3b0 0x7fa5a4146c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:46.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.355+0000 7fa5a8a63700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5a405feb0 0x7fa5a4154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:46.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.355+0000 7fa5a8a63700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a4062400 0x7fa5a4144830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:46.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.355+0000 7fa5a8a63700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a4154f70 con 0x7fa5a4062400 2020-12-05T17:32:46.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.355+0000 7fa5a8a63700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a41551b0 con 0x7fa5a405feb0 2020-12-05T17:32:46.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.355+0000 7fa5a8a63700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a4148e60 con 0x7fa5a405f3b0 2020-12-05T17:32:46.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.355+0000 7fa5a3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a4062400 0x7fa5a4144830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:46.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.355+0000 7fa5a3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a4062400 0x7fa5a4144830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34134/0 (socket says 172.21.15.32:34134) 2020-12-05T17:32:46.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.355+0000 7fa5a3fff700 1 -- 172.21.15.32:0/810189564 learned_addr learned my addr 172.21.15.32:0/810189564 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:46.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.355+0000 7fa5a37fe700 1 --2- 172.21.15.32:0/810189564 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5a405f3b0 0x7fa5a4146c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:46.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.355+0000 7fa5a2ffd700 1 --2- 172.21.15.32:0/810189564 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5a405feb0 0x7fa5a4154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:46.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.355+0000 7fa5a3fff700 1 -- 172.21.15.32:0/810189564 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5a405f3b0 msgr2=0x7fa5a4146c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:46.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.355+0000 7fa5a3fff700 1 --2- 172.21.15.32:0/810189564 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5a405f3b0 0x7fa5a4146c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:46.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.355+0000 7fa5a3fff700 1 -- 172.21.15.32:0/810189564 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5a405feb0 msgr2=0x7fa5a4154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.355+0000 7fa5a3fff700 1 --2- 172.21.15.32:0/810189564 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5a405feb0 0x7fa5a4154a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.355+0000 7fa5a3fff700 1 -- 172.21.15.32:0/810189564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a41491b0 con 0x7fa5a4062400 2020-12-05T17:32:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.355+0000 7fa5a2ffd700 1 --2- 172.21.15.32:0/810189564 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5a405feb0 0x7fa5a4154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.355+0000 7fa5a37fe700 1 --2- 172.21.15.32:0/810189564 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5a405f3b0 0x7fa5a4146c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.355+0000 7fa5a3fff700 1 --2- 172.21.15.32:0/810189564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a4062400 0x7fa5a4144830 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fa59c000f00 tx=0x7fa59c007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.359+0000 7fa5a0ff9700 1 -- 172.21.15.32:0/810189564 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa59c00a880 con 0x7fa5a4062400 2020-12-05T17:32:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.359+0000 7fa5a0ff9700 1 -- 172.21.15.32:0/810189564 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59c00a9e0 con 0x7fa5a4062400 2020-12-05T17:32:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.359+0000 7fa5a8a63700 1 -- 172.21.15.32:0/810189564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a41493a0 con 0x7fa5a4062400 2020-12-05T17:32:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.359+0000 7fa5a0ff9700 1 -- 172.21.15.32:0/810189564 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa59c00ad40 con 0x7fa5a4062400 2020-12-05T17:32:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.359+0000 7fa5a8a63700 1 -- 172.21.15.32:0/810189564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a4146830 con 0x7fa5a4062400 2020-12-05T17:32:46.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.359+0000 7fa5a8a63700 1 -- 172.21.15.32:0/810189564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a4057050 con 0x7fa5a4062400 2020-12-05T17:32:46.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.359+0000 7fa5a0ff9700 1 -- 172.21.15.32:0/810189564 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa59c019030 con 0x7fa5a4062400 2020-12-05T17:32:46.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.359+0000 7fa5a0ff9700 1 --2- 172.21.15.32:0/810189564 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa57c0554e0 0x7fa57c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:46.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.359+0000 7fa5a0ff9700 1 -- 172.21.15.32:0/810189564 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa59c06dea0 con 0x7fa5a4062400 2020-12-05T17:32:46.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.359+0000 7fa5a37fe700 1 --2- 172.21.15.32:0/810189564 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa57c0554e0 0x7fa57c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:46.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.359+0000 7fa5a37fe700 1 --2- 172.21.15.32:0/810189564 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa57c0554e0 0x7fa57c0579a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fa594018000 tx=0x7fa594015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:46.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.367+0000 7fa5a0ff9700 1 -- 172.21.15.32:0/810189564 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa59c043eb0 con 0x7fa5a4062400 2020-12-05T17:32:46.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.551+0000 7fa5a8a63700 1 -- 172.21.15.32:0/810189564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5a4149530 con 0x7fa5a4062400 2020-12-05T17:32:46.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.551+0000 7fa5a0ff9700 1 -- 172.21.15.32:0/810189564 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2136 (secure 0 0 0) 0x7fa59c0474d0 con 0x7fa5a4062400 2020-12-05T17:32:46.554 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:32:46.555 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4679,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/2850341272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2850341272},{"type":"v1","addr":"172.21.15.61:6833","nonce":2850341272}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:32:25.792490+0000","modified":"2020-12-05T17:32:26.797858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4873},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4873":{"gid":4873,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3316547885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3316547885},{"type":"v1","addr":"172.21.15.32:6835","nonce":3316547885}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-12-05T17:32:46.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.555+0000 7fa5a8a63700 1 -- 172.21.15.32:0/810189564 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa57c0554e0 msgr2=0x7fa57c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:46.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.555+0000 7fa5a8a63700 1 --2- 172.21.15.32:0/810189564 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa57c0554e0 0x7fa57c0579a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fa594018000 tx=0x7fa594015000).stop 2020-12-05T17:32:46.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.555+0000 7fa5a8a63700 1 -- 172.21.15.32:0/810189564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a4062400 msgr2=0x7fa5a4144830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:46.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.555+0000 7fa5a8a63700 1 --2- 172.21.15.32:0/810189564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a4062400 0x7fa5a4144830 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fa59c000f00 tx=0x7fa59c007dd0).stop 2020-12-05T17:32:46.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.555+0000 7fa5a8a63700 1 -- 172.21.15.32:0/810189564 shutdown_connections 2020-12-05T17:32:46.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.555+0000 7fa5a8a63700 1 --2- 172.21.15.32:0/810189564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a4062400 0x7fa5a4144830 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:46.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.555+0000 7fa5a8a63700 1 --2- 172.21.15.32:0/810189564 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa57c0554e0 0x7fa57c0579a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:46.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.555+0000 7fa5a8a63700 1 --2- 172.21.15.32:0/810189564 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5a405f3b0 0x7fa5a4146c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:46.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.555+0000 7fa5a8a63700 1 --2- 172.21.15.32:0/810189564 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5a405feb0 0x7fa5a4154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:46.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.555+0000 7fa5a8a63700 1 -- 172.21.15.32:0/810189564 >> 172.21.15.32:0/810189564 conn(0x7fa5a40b8ee0 msgr2=0x7fa5a41430e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:46.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.555+0000 7fa5a8a63700 1 -- 172.21.15.32:0/810189564 shutdown_connections 2020-12-05T17:32:46.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.555+0000 7fa5a8a63700 1 -- 172.21.15.32:0/810189564 wait complete. 2020-12-05T17:32:46.559 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 10 2020-12-05T17:32:46.568 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_caps 20 2020-12-05T17:32:46.689 INFO:teuthology.orchestra.run.smithi032.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2020-12-05T17:32:46.690 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2020-12-05T17:32:46.690 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:32:46.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.795+0000 7f8833fff700 1 Processor -- start 2020-12-05T17:32:46.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.795+0000 7f8833fff700 1 -- start start 2020-12-05T17:32:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.799+0000 7f8833fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f88340b52f0 0x7f88340b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.799+0000 7f8833fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88340b1870 0x7f8834138810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.799+0000 7f8833fff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88341420f0 con 0x7f8834138d00 2020-12-05T17:32:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.799+0000 7f8833fff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88340ad4f0 con 0x7f88340b52f0 2020-12-05T17:32:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.799+0000 7f8833fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88340b0380 con 0x7f88340b1870 2020-12-05T17:32:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.799+0000 7f8832ffd700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f88340b52f0 0x7f88340b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:46.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.799+0000 7f88337fe700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f8834138d00 0x7f88340ae860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:49430/0 (socket says 172.21.15.32:49430) 2020-12-05T17:32:46.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.799+0000 7f88337fe700 1 -- 172.21.15.32:0/2978519448 learned_addr learned my addr 172.21.15.32:0/2978519448 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:46.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.799+0000 7f88327fc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88340b1870 0x7f8834138810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.799+0000 7f88327fc700 1 -- 172.21.15.32:0/2978519448 >> v2:172.21.15.61:3300/0 conn(0x7f88340b52f0 msgr2=0x7f88340b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.799+0000 7f88327fc700 1 --2- 172.21.15.32:0/2978519448 >> v2:172.21.15.61:3300/0 conn(0x7f88340b52f0 0x7f88340b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.799+0000 7f88327fc700 1 -- 172.21.15.32:0/2978519448 >> v1:172.21.15.61:6789/0 conn(0x7f8834138d00 legacy=0x7f88340ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.799+0000 7f88327fc700 1 -- 172.21.15.32:0/2978519448 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88340b0620 con 0x7f88340b1870 2020-12-05T17:32:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.799+0000 7f88327fc700 1 --2- 172.21.15.32:0/2978519448 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88340b1870 0x7f8834138810 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f88340adc80 tx=0x7f8828006dc0).ready entity=mon.2 client_cookie=cadd2023cb60d556 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.799+0000 7f8831ffb700 1 -- 172.21.15.32:0/2978519448 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8828009720 con 0x7f88340b1870 2020-12-05T17:32:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.799+0000 7f8831ffb700 1 -- 172.21.15.32:0/2978519448 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8828009880 con 0x7f88340b1870 2020-12-05T17:32:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.799+0000 7f8831ffb700 1 -- 172.21.15.32:0/2978519448 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8828009be0 con 0x7f88340b1870 2020-12-05T17:32:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.799+0000 7f8833fff700 1 -- 172.21.15.32:0/2978519448 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88340b1870 msgr2=0x7f8834138810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.799+0000 7f8833fff700 1 --2- 172.21.15.32:0/2978519448 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88340b1870 0x7f8834138810 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f88340adc80 tx=0x7f8828006dc0).stop 2020-12-05T17:32:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.803+0000 7f8833fff700 1 -- 172.21.15.32:0/2978519448 shutdown_connections 2020-12-05T17:32:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.803+0000 7f8833fff700 1 --2- 172.21.15.32:0/2978519448 >> v2:172.21.15.61:3300/0 conn(0x7f88340b52f0 0x7f88340b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.803+0000 7f8833fff700 1 --2- 172.21.15.32:0/2978519448 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88340b1870 0x7f8834138810 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.803+0000 7f8833fff700 1 -- 172.21.15.32:0/2978519448 >> 172.21.15.32:0/2978519448 conn(0x7f883405bbb0 msgr2=0x7f8834062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:46.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.803+0000 7f8833fff700 1 -- 172.21.15.32:0/2978519448 shutdown_connections 2020-12-05T17:32:46.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.803+0000 7f8833fff700 1 -- 172.21.15.32:0/2978519448 wait complete. 2020-12-05T17:32:46.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.803+0000 7f8833fff700 1 Processor -- start 2020-12-05T17:32:46.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.803+0000 7f8833fff700 1 -- start start 2020-12-05T17:32:46.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.807+0000 7f8833fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88340b1870 0x7f883413f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:46.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.807+0000 7f8833fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88340b52f0 0x7f8834062ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:46.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.807+0000 7f8833fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8834138d00 0x7f8834144950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:46.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.807+0000 7f8833fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88340634e0 con 0x7f88340b52f0 2020-12-05T17:32:46.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.807+0000 7f8833fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8834065110 con 0x7f8834138d00 2020-12-05T17:32:46.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.807+0000 7f8833fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88340653b0 con 0x7f88340b1870 2020-12-05T17:32:46.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.807+0000 7f88327fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88340b52f0 0x7f8834062ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:46.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.807+0000 7f88327fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88340b52f0 0x7f8834062ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34148/0 (socket says 172.21.15.32:34148) 2020-12-05T17:32:46.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.807+0000 7f8832ffd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88340b1870 0x7f883413f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:46.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.807+0000 7f88327fc700 1 -- 172.21.15.32:0/3943064921 learned_addr learned my addr 172.21.15.32:0/3943064921 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:46.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.807+0000 7f88337fe700 1 --2- 172.21.15.32:0/3943064921 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8834138d00 0x7f8834144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:46.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.807+0000 7f88327fc700 1 -- 172.21.15.32:0/3943064921 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88340b1870 msgr2=0x7f883413f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:46.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.807+0000 7f88327fc700 1 --2- 172.21.15.32:0/3943064921 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88340b1870 0x7f883413f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:46.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.807+0000 7f88327fc700 1 -- 172.21.15.32:0/3943064921 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8834138d00 msgr2=0x7f8834144950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:46.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.807+0000 7f88327fc700 1 --2- 172.21.15.32:0/3943064921 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8834138d00 0x7f8834144950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:46.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.807+0000 7f88327fc700 1 -- 172.21.15.32:0/3943064921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88341444b0 con 0x7f88340b52f0 2020-12-05T17:32:46.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.807+0000 7f88327fc700 1 --2- 172.21.15.32:0/3943064921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88340b52f0 0x7f8834062ff0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f88340adc80 tx=0x7f8828015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:46.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.807+0000 7f8823fff700 1 -- 172.21.15.32:0/3943064921 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8828011b40 con 0x7f88340b52f0 2020-12-05T17:32:46.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.807+0000 7f8823fff700 1 -- 172.21.15.32:0/3943064921 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8828011ca0 con 0x7f88340b52f0 2020-12-05T17:32:46.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.807+0000 7f8833fff700 1 -- 172.21.15.32:0/3943064921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8834064e00 con 0x7f88340b52f0 2020-12-05T17:32:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.807+0000 7f8823fff700 1 -- 172.21.15.32:0/3943064921 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f882801d450 con 0x7f88340b52f0 2020-12-05T17:32:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.807+0000 7f8833fff700 1 -- 172.21.15.32:0/3943064921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8834066fb0 con 0x7f88340b52f0 2020-12-05T17:32:46.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.807+0000 7f8833fff700 1 -- 172.21.15.32:0/3943064921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88340b5a50 con 0x7f88340b52f0 2020-12-05T17:32:46.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.811+0000 7f8823fff700 1 -- 172.21.15.32:0/3943064921 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f882800f070 con 0x7f88340b52f0 2020-12-05T17:32:46.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.811+0000 7f8823fff700 1 --2- 172.21.15.32:0/3943064921 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8810055440 0x7f8810057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:46.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.811+0000 7f8823fff700 1 -- 172.21.15.32:0/3943064921 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f882806eef0 con 0x7f88340b52f0 2020-12-05T17:32:46.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.811+0000 7f8832ffd700 1 --2- 172.21.15.32:0/3943064921 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8810055440 0x7f8810057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:46.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.811+0000 7f8832ffd700 1 --2- 172.21.15.32:0/3943064921 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8810055440 0x7f8810057900 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f882400b000 tx=0x7f88240091a0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:46.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:46.815+0000 7f8823fff700 1 -- 172.21.15.32:0/3943064921 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8828045090 con 0x7f88340b52f0 2020-12-05T17:32:47.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.011+0000 7f8833fff700 1 -- 172.21.15.32:0/3943064921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88340660e0 con 0x7f88340b52f0 2020-12-05T17:32:47.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.011+0000 7f8823fff700 1 -- 172.21.15.32:0/3943064921 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2136 (secure 0 0 0) 0x7f88340660e0 con 0x7f88340b52f0 2020-12-05T17:32:47.013 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:32:47.014 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4679,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/2850341272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2850341272},{"type":"v1","addr":"172.21.15.61:6833","nonce":2850341272}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:32:25.792490+0000","modified":"2020-12-05T17:32:26.797858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4873},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4873":{"gid":4873,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3316547885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3316547885},{"type":"v1","addr":"172.21.15.32:6835","nonce":3316547885}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-12-05T17:32:47.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.011+0000 7f8833fff700 1 -- 172.21.15.32:0/3943064921 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8810055440 msgr2=0x7f8810057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:47.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.011+0000 7f8833fff700 1 --2- 172.21.15.32:0/3943064921 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8810055440 0x7f8810057900 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f882400b000 tx=0x7f88240091a0).stop 2020-12-05T17:32:47.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.011+0000 7f8833fff700 1 -- 172.21.15.32:0/3943064921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88340b52f0 msgr2=0x7f8834062ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:47.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.011+0000 7f8833fff700 1 --2- 172.21.15.32:0/3943064921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88340b52f0 0x7f8834062ff0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f88340adc80 tx=0x7f8828015000).stop 2020-12-05T17:32:47.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.011+0000 7f8833fff700 1 -- 172.21.15.32:0/3943064921 shutdown_connections 2020-12-05T17:32:47.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.011+0000 7f8833fff700 1 --2- 172.21.15.32:0/3943064921 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8834138d00 0x7f8834144950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.011+0000 7f8833fff700 1 --2- 172.21.15.32:0/3943064921 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8810055440 0x7f8810057900 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.011+0000 7f8833fff700 1 --2- 172.21.15.32:0/3943064921 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88340b1870 0x7f883413f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.011+0000 7f8833fff700 1 --2- 172.21.15.32:0/3943064921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88340b52f0 0x7f8834062ff0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.011+0000 7f8833fff700 1 -- 172.21.15.32:0/3943064921 >> 172.21.15.32:0/3943064921 conn(0x7f883405bbb0 msgr2=0x7f88340b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:47.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.011+0000 7f8833fff700 1 -- 172.21.15.32:0/3943064921 shutdown_connections 2020-12-05T17:32:47.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.011+0000 7f8833fff700 1 -- 172.21.15.32:0/3943064921 wait complete. 2020-12-05T17:32:47.018 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 10 2020-12-05T17:32:47.027 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_decay_threshold 400 2020-12-05T17:32:47.144 INFO:teuthology.orchestra.run.smithi032.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2020-12-05T17:32:47.145 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2020-12-05T17:32:47.145 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:32:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f0072053700 1 Processor -- start 2020-12-05T17:32:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f0072053700 1 -- start start 2020-12-05T17:32:47.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f0072053700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f006c0b5f00 0x7f006c0b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f0072053700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f006c0af480 0x7f006c142ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f0072053700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f006c142ab0 con 0x7f006c0ad510 2020-12-05T17:32:47.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f0072053700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f006c0af8f0 con 0x7f006c0af480 2020-12-05T17:32:47.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f0072053700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006c0b62e0 con 0x7f006c0b5f00 2020-12-05T17:32:47.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f0070850700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f006c0ad510 0x7f006c066370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36802/0 (socket says 172.21.15.32:36802) 2020-12-05T17:32:47.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f0070850700 1 -- 172.21.15.32:0/3162974443 learned_addr learned my addr 172.21.15.32:0/3162974443 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:47.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f0071852700 1 --2- 172.21.15.32:0/3162974443 >> v2:172.21.15.32:3300/0 conn(0x7f006c0af480 0x7f006c142ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f0071051700 1 --2- 172.21.15.32:0/3162974443 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f006c0b5f00 0x7f006c0b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f005bfff700 1 -- 172.21.15.32:0/3162974443 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3660993466 0 0) 0x7f006c142ab0 con 0x7f006c0ad510 2020-12-05T17:32:47.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f005bfff700 1 -- 172.21.15.32:0/3162974443 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0054003230 con 0x7f006c0ad510 2020-12-05T17:32:47.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f0071852700 1 -- 172.21.15.32:0/3162974443 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f006c0b5f00 msgr2=0x7f006c0b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:47.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f0071852700 1 --2- 172.21.15.32:0/3162974443 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f006c0b5f00 0x7f006c0b14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f0071852700 1 -- 172.21.15.32:0/3162974443 >> v1:172.21.15.32:6789/0 conn(0x7f006c0ad510 legacy=0x7f006c066370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:47.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f0071852700 1 -- 172.21.15.32:0/3162974443 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c0afbf0 con 0x7f006c0af480 2020-12-05T17:32:47.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f0071852700 1 --2- 172.21.15.32:0/3162974443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f006c0af480 0x7f006c142ee0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f00640032b0 tx=0x7f00640091a0).ready entity=mon.0 client_cookie=63e687965806a213 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:47.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f005bfff700 1 -- 172.21.15.32:0/3162974443 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f006400b940 con 0x7f006c0af480 2020-12-05T17:32:47.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f005bfff700 1 -- 172.21.15.32:0/3162974443 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006400baa0 con 0x7f006c0af480 2020-12-05T17:32:47.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f005bfff700 1 -- 172.21.15.32:0/3162974443 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f006400bc10 con 0x7f006c0af480 2020-12-05T17:32:47.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f0072053700 1 -- 172.21.15.32:0/3162974443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f006c0af480 msgr2=0x7f006c142ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:47.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f0072053700 1 --2- 172.21.15.32:0/3162974443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f006c0af480 0x7f006c142ee0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f00640032b0 tx=0x7f00640091a0).stop 2020-12-05T17:32:47.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f0072053700 1 -- 172.21.15.32:0/3162974443 shutdown_connections 2020-12-05T17:32:47.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f0072053700 1 --2- 172.21.15.32:0/3162974443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f006c0af480 0x7f006c142ee0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f0072053700 1 --2- 172.21.15.32:0/3162974443 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f006c0b5f00 0x7f006c0b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.251+0000 7f0072053700 1 -- 172.21.15.32:0/3162974443 >> 172.21.15.32:0/3162974443 conn(0x7f006c0604c0 msgr2=0x7f006c05c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:47.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.255+0000 7f0072053700 1 -- 172.21.15.32:0/3162974443 shutdown_connections 2020-12-05T17:32:47.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.255+0000 7f0072053700 1 -- 172.21.15.32:0/3162974443 wait complete. 2020-12-05T17:32:47.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.255+0000 7f0072053700 1 Processor -- start 2020-12-05T17:32:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.255+0000 7f0072053700 1 -- start start 2020-12-05T17:32:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.255+0000 7f0072053700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f006c0ad510 0x7f006c134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.255+0000 7f0072053700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f006c0af480 0x7f006c139f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.255+0000 7f0072053700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f006c0b5f00 0x7f006c135850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.255+0000 7f0072053700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c0b4ea0 con 0x7f006c0ad510 2020-12-05T17:32:47.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.255+0000 7f0072053700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c063120 con 0x7f006c0af480 2020-12-05T17:32:47.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.255+0000 7f0072053700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006c0633c0 con 0x7f006c0b5f00 2020-12-05T17:32:47.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.255+0000 7f0071051700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f006c0ad510 0x7f006c134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.255+0000 7f0071051700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f006c0ad510 0x7f006c134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34158/0 (socket says 172.21.15.32:34158) 2020-12-05T17:32:47.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.255+0000 7f0071051700 1 -- 172.21.15.32:0/1687948389 learned_addr learned my addr 172.21.15.32:0/1687948389 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:47.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.255+0000 7f0071852700 1 --2- 172.21.15.32:0/1687948389 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f006c0b5f00 0x7f006c135850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.255+0000 7f0070850700 1 --2- 172.21.15.32:0/1687948389 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f006c0af480 0x7f006c139f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.255+0000 7f0071051700 1 -- 172.21.15.32:0/1687948389 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f006c0b5f00 msgr2=0x7f006c135850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:47.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.255+0000 7f0071051700 1 --2- 172.21.15.32:0/1687948389 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f006c0b5f00 0x7f006c135850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.255+0000 7f0071051700 1 -- 172.21.15.32:0/1687948389 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f006c0af480 msgr2=0x7f006c139f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:47.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.255+0000 7f0071051700 1 --2- 172.21.15.32:0/1687948389 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f006c0af480 0x7f006c139f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.255+0000 7f0071051700 1 -- 172.21.15.32:0/1687948389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c136060 con 0x7f006c0ad510 2020-12-05T17:32:47.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.255+0000 7f0071051700 1 --2- 172.21.15.32:0/1687948389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f006c0ad510 0x7f006c134ab0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f006800c000 tx=0x7f0068008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:47.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.259+0000 7f005a7fc700 1 -- 172.21.15.32:0/1687948389 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f006800be30 con 0x7f006c0ad510 2020-12-05T17:32:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.259+0000 7f005a7fc700 1 -- 172.21.15.32:0/1687948389 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006800c820 con 0x7f006c0ad510 2020-12-05T17:32:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.259+0000 7f0072053700 1 -- 172.21.15.32:0/1687948389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f006c0b5300 con 0x7f006c0ad510 2020-12-05T17:32:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.259+0000 7f005a7fc700 1 -- 172.21.15.32:0/1687948389 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f006800cb80 con 0x7f006c0ad510 2020-12-05T17:32:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.259+0000 7f0072053700 1 -- 172.21.15.32:0/1687948389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f006c065f70 con 0x7f006c0ad510 2020-12-05T17:32:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.259+0000 7f0072053700 1 -- 172.21.15.32:0/1687948389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f006c057050 con 0x7f006c0ad510 2020-12-05T17:32:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.259+0000 7f005a7fc700 1 -- 172.21.15.32:0/1687948389 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f006800cce0 con 0x7f006c0ad510 2020-12-05T17:32:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.259+0000 7f005a7fc700 1 --2- 172.21.15.32:0/1687948389 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f004c055440 0x7f004c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.259+0000 7f005a7fc700 1 -- 172.21.15.32:0/1687948389 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f006806e390 con 0x7f006c0ad510 2020-12-05T17:32:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.259+0000 7f0070850700 1 --2- 172.21.15.32:0/1687948389 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f004c055440 0x7f004c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.259+0000 7f0070850700 1 --2- 172.21.15.32:0/1687948389 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f004c055440 0x7f004c057900 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f005c00d000 tx=0x7f005c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:47.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.263+0000 7f005a7fc700 1 -- 172.21.15.32:0/1687948389 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0068044530 con 0x7f006c0ad510 2020-12-05T17:32:47.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.447+0000 7f0072053700 1 -- 172.21.15.32:0/1687948389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f006c0b12c0 con 0x7f006c0ad510 2020-12-05T17:32:47.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.447+0000 7f005a7fc700 1 -- 172.21.15.32:0/1687948389 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2136 (secure 0 0 0) 0x7f0068047b50 con 0x7f006c0ad510 2020-12-05T17:32:47.450 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:32:47.450 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4679,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/2850341272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2850341272},{"type":"v1","addr":"172.21.15.61:6833","nonce":2850341272}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:32:25.792490+0000","modified":"2020-12-05T17:32:26.797858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4873},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4873":{"gid":4873,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3316547885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3316547885},{"type":"v1","addr":"172.21.15.32:6835","nonce":3316547885}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-12-05T17:32:47.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.451+0000 7f0072053700 1 -- 172.21.15.32:0/1687948389 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f004c055440 msgr2=0x7f004c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:47.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.451+0000 7f0072053700 1 --2- 172.21.15.32:0/1687948389 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f004c055440 0x7f004c057900 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f005c00d000 tx=0x7f005c009040).stop 2020-12-05T17:32:47.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.451+0000 7f0072053700 1 -- 172.21.15.32:0/1687948389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f006c0ad510 msgr2=0x7f006c134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:47.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.451+0000 7f0072053700 1 --2- 172.21.15.32:0/1687948389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f006c0ad510 0x7f006c134ab0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f006800c000 tx=0x7f0068008040).stop 2020-12-05T17:32:47.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.451+0000 7f0072053700 1 -- 172.21.15.32:0/1687948389 shutdown_connections 2020-12-05T17:32:47.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.451+0000 7f0072053700 1 --2- 172.21.15.32:0/1687948389 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f004c055440 0x7f004c057900 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.451+0000 7f0072053700 1 --2- 172.21.15.32:0/1687948389 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f006c0b5f00 0x7f006c135850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.451+0000 7f0072053700 1 --2- 172.21.15.32:0/1687948389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f006c0ad510 0x7f006c134ab0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.451+0000 7f0072053700 1 --2- 172.21.15.32:0/1687948389 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f006c0af480 0x7f006c139f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.451+0000 7f0072053700 1 -- 172.21.15.32:0/1687948389 >> 172.21.15.32:0/1687948389 conn(0x7f006c0604c0 msgr2=0x7f006c142e00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:47.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.451+0000 7f0072053700 1 -- 172.21.15.32:0/1687948389 shutdown_connections 2020-12-05T17:32:47.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.451+0000 7f0072053700 1 -- 172.21.15.32:0/1687948389 wait complete. 2020-12-05T17:32:47.455 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 10 2020-12-05T17:32:47.464 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell mds.a cache drop 2020-12-05T17:32:47.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.579+0000 7f15cd45b700 1 Processor -- start 2020-12-05T17:32:47.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.579+0000 7f15cd45b700 1 -- start start 2020-12-05T17:32:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15cd45b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15c80666a0 0x7f15c8144600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15cd45b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15c80627a0 con 0x7f15c80b5730 2020-12-05T17:32:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15cd45b700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15c80b5b10 con 0x7f15c8061bb0 2020-12-05T17:32:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15cd45b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c80b04c0 con 0x7f15c80666a0 2020-12-05T17:32:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c6ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f15c80b5730 0x7f15c8143ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36814/0 (socket says 172.21.15.32:36814) 2020-12-05T17:32:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c6ffd700 1 -- 172.21.15.32:0/1516689880 learned_addr learned my addr 172.21.15.32:0/1516689880 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:32:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c77fe700 1 --2- 172.21.15.32:0/1516689880 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15c80666a0 0x7f15c8144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c67fc700 1 -- 172.21.15.32:0/1516689880 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1432346435 0 0) 0x7f15c80627a0 con 0x7f15c80b5730 2020-12-05T17:32:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c67fc700 1 -- 172.21.15.32:0/1516689880 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15a8003230 con 0x7f15c80b5730 2020-12-05T17:32:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c67fc700 1 -- 172.21.15.32:0/1516689880 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1020302975 0 0) 0x7f15c80b5b10 con 0x7f15c8061bb0 2020-12-05T17:32:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c67fc700 1 -- 172.21.15.32:0/1516689880 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15c80627a0 con 0x7f15c8061bb0 2020-12-05T17:32:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c67fc700 1 -- 172.21.15.32:0/1516689880 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2343989652 0 0) 0x7f15a8003230 con 0x7f15c80b5730 2020-12-05T17:32:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c67fc700 1 -- 172.21.15.32:0/1516689880 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15c80666a0 msgr2=0x7f15c8144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c67fc700 1 --2- 172.21.15.32:0/1516689880 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15c80666a0 0x7f15c8144600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c67fc700 1 -- 172.21.15.32:0/1516689880 >> v1:172.21.15.61:6789/0 conn(0x7f15c8061bb0 legacy=0x7f15c8140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:47.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c67fc700 1 -- 172.21.15.32:0/1516689880 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c8062a00 con 0x7f15c80b5730 2020-12-05T17:32:47.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c67fc700 1 -- 172.21.15.32:0/1516689880 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f15b80016d0 con 0x7f15c80b5730 2020-12-05T17:32:47.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c67fc700 1 -- 172.21.15.32:0/1516689880 >> v1:172.21.15.32:6789/0 conn(0x7f15c80b5730 legacy=0x7f15c8143ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:47.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c67fc700 1 --2- 172.21.15.32:0/1516689880 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15c80666a0 0x7f15a80038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c67fc700 1 --2- 172.21.15.32:0/1516689880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15a8003dd0 0x7f15a80044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c67fc700 1 --2- 172.21.15.32:0/1516689880 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15a80049c0 0x7f15a8004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c67fc700 1 -- 172.21.15.32:0/1516689880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c80b04c0 con 0x7f15a8003dd0 2020-12-05T17:32:47.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c67fc700 1 -- 172.21.15.32:0/1516689880 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a8005360 con 0x7f15a80049c0 2020-12-05T17:32:47.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c67fc700 1 -- 172.21.15.32:0/1516689880 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a800b770 con 0x7f15c80666a0 2020-12-05T17:32:47.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15be7fc700 1 --2- 172.21.15.32:0/1516689880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15a8003dd0 0x7f15a80044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c6ffd700 1 --2- 172.21.15.32:0/1516689880 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15c80666a0 0x7f15a80038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c77fe700 1 --2- 172.21.15.32:0/1516689880 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15a80049c0 0x7f15a8004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15be7fc700 1 -- 172.21.15.32:0/1516689880 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15c80666a0 msgr2=0x7f15a80038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:47.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15be7fc700 1 --2- 172.21.15.32:0/1516689880 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15c80666a0 0x7f15a80038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15be7fc700 1 -- 172.21.15.32:0/1516689880 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15a80049c0 msgr2=0x7f15a8004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:47.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15be7fc700 1 --2- 172.21.15.32:0/1516689880 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15a80049c0 0x7f15a8004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15be7fc700 1 -- 172.21.15.32:0/1516689880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a8003230 con 0x7f15a8003dd0 2020-12-05T17:32:47.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15be7fc700 1 --2- 172.21.15.32:0/1516689880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15a8003dd0 0x7f15a80044d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f15a80054a0 tx=0x7f15b0007b60).ready entity=mon.0 client_cookie=f50cda21e95a4153 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:47.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c67fc700 1 -- 172.21.15.32:0/1516689880 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15b000c830 con 0x7f15a8003dd0 2020-12-05T17:32:47.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c67fc700 1 -- 172.21.15.32:0/1516689880 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b000c990 con 0x7f15a8003dd0 2020-12-05T17:32:47.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c67fc700 1 -- 172.21.15.32:0/1516689880 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15b000ccf0 con 0x7f15a8003dd0 2020-12-05T17:32:47.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15cd45b700 1 -- 172.21.15.32:0/1516689880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15a8003dd0 msgr2=0x7f15a80044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:47.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15cd45b700 1 --2- 172.21.15.32:0/1516689880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15a8003dd0 0x7f15a80044d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f15a80054a0 tx=0x7f15b0007b60).stop 2020-12-05T17:32:47.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15c77fe700 1 -- 172.21.15.32:0/1516689880 reap_dead start 2020-12-05T17:32:47.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15cd45b700 1 -- 172.21.15.32:0/1516689880 shutdown_connections 2020-12-05T17:32:47.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15cd45b700 1 -- 172.21.15.32:0/1516689880 >> 172.21.15.32:0/1516689880 conn(0x7f15c805fe80 msgr2=0x7f15c80b4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:47.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15cd45b700 1 -- 172.21.15.32:0/1516689880 shutdown_connections 2020-12-05T17:32:47.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.583+0000 7f15cd45b700 1 -- 172.21.15.32:0/1516689880 wait complete. 2020-12-05T17:32:47.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15cd45b700 1 Processor -- start 2020-12-05T17:32:47.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15cd45b700 1 -- start start 2020-12-05T17:32:47.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15cd45b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15c81421e0 0x7f15c809e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15cd45b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15c8144a60 0x7f15c8151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15cd45b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15c8151a10 0x7f15c80aa750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15cd45b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c809f110 con 0x7f15c8151a10 2020-12-05T17:32:47.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15cd45b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c81425c0 con 0x7f15c81421e0 2020-12-05T17:32:47.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15cd45b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c8151e90 con 0x7f15c8144a60 2020-12-05T17:32:47.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15c77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15c8151a10 0x7f15c80aa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15c77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15c8151a10 0x7f15c80aa750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34182/0 (socket says 172.21.15.32:34182) 2020-12-05T17:32:47.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15c77fe700 1 -- 172.21.15.32:0/4013371575 learned_addr learned my addr 172.21.15.32:0/4013371575 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:47.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15c6ffd700 1 --2- 172.21.15.32:0/4013371575 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15c81421e0 0x7f15c809e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15be7fc700 1 --2- 172.21.15.32:0/4013371575 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15c8144a60 0x7f15c8151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15c77fe700 1 -- 172.21.15.32:0/4013371575 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15c8144a60 msgr2=0x7f15c8151520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:47.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15c77fe700 1 --2- 172.21.15.32:0/4013371575 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15c8144a60 0x7f15c8151520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15c77fe700 1 -- 172.21.15.32:0/4013371575 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15c81421e0 msgr2=0x7f15c809e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:47.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15c77fe700 1 --2- 172.21.15.32:0/4013371575 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15c81421e0 0x7f15c809e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15c77fe700 1 -- 172.21.15.32:0/4013371575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c80a11d0 con 0x7f15c8151a10 2020-12-05T17:32:47.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15be7fc700 1 --2- 172.21.15.32:0/4013371575 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15c8144a60 0x7f15c8151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:47.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15c77fe700 1 --2- 172.21.15.32:0/4013371575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15c8151a10 0x7f15c80aa750 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f15c000b000 tx=0x7f15c00091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:47.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15c4ff9700 1 -- 172.21.15.32:0/4013371575 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15c001a070 con 0x7f15c8151a10 2020-12-05T17:32:47.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15c4ff9700 1 -- 172.21.15.32:0/4013371575 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c000fb10 con 0x7f15c8151a10 2020-12-05T17:32:47.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15cd45b700 1 -- 172.21.15.32:0/4013371575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c80a1470 con 0x7f15c8151a10 2020-12-05T17:32:47.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15c4ff9700 1 -- 172.21.15.32:0/4013371575 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15c000fe70 con 0x7f15c8151a10 2020-12-05T17:32:47.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.587+0000 7f15cd45b700 1 -- 172.21.15.32:0/4013371575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c81450c0 con 0x7f15c8151a10 2020-12-05T17:32:47.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.591+0000 7f15c4ff9700 1 -- 172.21.15.32:0/4013371575 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f15c0014440 con 0x7f15c8151a10 2020-12-05T17:32:47.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.591+0000 7f15c4ff9700 1 --2- 172.21.15.32:0/4013371575 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15ac055330 0x7f15ac0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.591+0000 7f15c4ff9700 1 -- 172.21.15.32:0/4013371575 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f15c006e960 con 0x7f15c8151a10 2020-12-05T17:32:47.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.591+0000 7f15c6ffd700 1 --2- 172.21.15.32:0/4013371575 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15ac055330 0x7f15ac0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.591+0000 7f15c6ffd700 1 --2- 172.21.15.32:0/4013371575 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15ac055330 0x7f15ac0577f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f15b8005000 tx=0x7f15b8007740).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:47.609 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15dbdf6740 1 Processor -- start 2020-12-05T17:32:47.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15dbdf6740 1 -- start start 2020-12-05T17:32:47.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15dbdf6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xe6d060 0xeba4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15dbdf6740 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0xd55e90 0xef10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15dbdf6740 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0xcd5800 0xe9bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15dbdf6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0xde1ff0 con 0xe6d060 2020-12-05T17:32:47.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15dbdf6740 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0xcee690 con 0xcd5800 2020-12-05T17:32:47.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15dbdf6740 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0xc9c410 con 0xd55e90 2020-12-05T17:32:47.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15c6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xe6d060 0xeba4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15c6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xe6d060 0xeba4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34186/0 (socket says 172.21.15.32:34186) 2020-12-05T17:32:47.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15c6ffd700 1 -- 172.21.15.32:0/545589808 learned_addr learned my addr 172.21.15.32:0/545589808 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:47.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15be7fc700 1 --2- 172.21.15.32:0/545589808 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0xd55e90 0xef10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15c77fe700 1 --2- 172.21.15.32:0/545589808 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0xcd5800 0xe9bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15c6ffd700 1 -- 172.21.15.32:0/545589808 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0xd55e90 msgr2=0xef10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:47.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15c6ffd700 1 --2- 172.21.15.32:0/545589808 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0xd55e90 0xef10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15c6ffd700 1 -- 172.21.15.32:0/545589808 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0xcd5800 msgr2=0xe9bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:47.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15c6ffd700 1 --2- 172.21.15.32:0/545589808 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0xcd5800 0xe9bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15c6ffd700 1 -- 172.21.15.32:0/545589808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xe02f10 con 0xe6d060 2020-12-05T17:32:47.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15c77fe700 1 --2- 172.21.15.32:0/545589808 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0xcd5800 0xe9bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:47.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15c6ffd700 1 --2- 172.21.15.32:0/545589808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xe6d060 0xeba4e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0xebb340 tx=0x7f15b8006000).ready entity=mon.0 client_cookie=bc46077342111033 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:47.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15cd45b700 1 -- 172.21.15.32:0/545589808 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15b80178c0 con 0xe6d060 2020-12-05T17:32:47.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15cd45b700 1 -- 172.21.15.32:0/545589808 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b8017a20 con 0xe6d060 2020-12-05T17:32:47.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15cd45b700 1 -- 172.21.15.32:0/545589808 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15b8017d80 con 0xe6d060 2020-12-05T17:32:47.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15dbdf6740 1 -- 172.21.15.32:0/545589808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xe6d060 msgr2=0xeba4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:47.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.595+0000 7f15dbdf6740 1 --2- 172.21.15.32:0/545589808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xe6d060 0xeba4e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0xebb340 tx=0x7f15b8006000).stop 2020-12-05T17:32:47.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15dbdf6740 1 -- 172.21.15.32:0/545589808 shutdown_connections 2020-12-05T17:32:47.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15dbdf6740 1 --2- 172.21.15.32:0/545589808 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0xcd5800 0xe9bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15dbdf6740 1 --2- 172.21.15.32:0/545589808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xe6d060 0xeba4e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15dbdf6740 1 --2- 172.21.15.32:0/545589808 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0xd55e90 0xef10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15dbdf6740 1 -- 172.21.15.32:0/545589808 >> 172.21.15.32:0/545589808 conn(0xe9c9f0 msgr2=0xeb9810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:47.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15dbdf6740 1 -- 172.21.15.32:0/545589808 shutdown_connections 2020-12-05T17:32:47.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15dbdf6740 1 -- 172.21.15.32:0/545589808 wait complete. 2020-12-05T17:32:47.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15dbdf6740 1 -- start start 2020-12-05T17:32:47.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15dbdf6740 1 Processor -- start 2020-12-05T17:32:47.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15dbdf6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xcd5800 0xeee6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15dbdf6740 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0xd55e90 0xec2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15dbdf6740 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0xe6d060 0xec2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15dbdf6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0xe11c70 con 0xcd5800 2020-12-05T17:32:47.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15dbdf6740 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0xe13240 con 0xd55e90 2020-12-05T17:32:47.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15dbdf6740 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0xdf4710 con 0xe6d060 2020-12-05T17:32:47.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15dbdf6740 10 client.0 ll_register_callbacks cb 0xe9c950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-12-05T17:32:47.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15c6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xcd5800 0xeee6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15c6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xcd5800 0xeee6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34192/0 (socket says 172.21.15.32:34192) 2020-12-05T17:32:47.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15c6ffd700 1 -- 172.21.15.32:0/223581183 learned_addr learned my addr 172.21.15.32:0/223581183 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:47.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15be7fc700 1 --2- 172.21.15.32:0/223581183 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0xd55e90 0xec2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15c77fe700 1 --2- 172.21.15.32:0/223581183 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0xe6d060 0xec2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15c6ffd700 1 -- 172.21.15.32:0/223581183 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0xe6d060 msgr2=0xec2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:47.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15c6ffd700 1 --2- 172.21.15.32:0/223581183 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0xe6d060 0xec2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15c6ffd700 1 -- 172.21.15.32:0/223581183 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0xd55e90 msgr2=0xec2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15c6ffd700 1 --2- 172.21.15.32:0/223581183 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0xd55e90 0xec2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15c6ffd700 1 -- 172.21.15.32:0/223581183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xe0b440 con 0xcd5800 2020-12-05T17:32:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15c77fe700 1 --2- 172.21.15.32:0/223581183 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0xe6d060 0xec2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:32:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15c6ffd700 1 --2- 172.21.15.32:0/223581183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xcd5800 0xeee6b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0xebb340 tx=0x7f15b8017000).ready entity=mon.0 client_cookie=d6740d0b603c144a server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:47.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2020-12-05T17:32:47.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15a5ffb700 1 -- 172.21.15.32:0/223581183 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15b802a070 con 0xcd5800 2020-12-05T17:32:47.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15a5ffb700 1 -- 172.21.15.32:0/223581183 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f15b8015820 con 0xcd5800 2020-12-05T17:32:47.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15dbdf6740 1 -- 172.21.15.32:0/223581183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xcfa8b0 con 0xcd5800 2020-12-05T17:32:47.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15a5ffb700 1 -- 172.21.15.32:0/223581183 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=10) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15b80159e0 con 0xcd5800 2020-12-05T17:32:47.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15dbdf6740 10 client.5199 fetch_fsmap learned FSMap version 10 2020-12-05T17:32:47.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.599+0000 7f15dbdf6740 1 -- 172.21.15.32:0/223581183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=10,monmap=2+}) v3 -- 0xc85770 con 0xcd5800 2020-12-05T17:32:47.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15a5ffb700 1 -- 172.21.15.32:0/223581183 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xcfa8b0 con 0xcd5800 2020-12-05T17:32:47.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15a5ffb700 1 -- 172.21.15.32:0/223581183 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 10) v1 ==== 1247+0+0 (secure 0 0 0) 0x7f15b802d030 con 0xcd5800 2020-12-05T17:32:47.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15dbdf6740 10 client.5199 fetch_fsmap finished waiting for FSMap version 10 2020-12-05T17:32:47.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15dbdf6740 10 client.5199 resolve_mds: resolved ID 'a' to GID 4873 2020-12-05T17:32:47.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15dbdf6740 20 client.5199 populate_metadata read hostname 'smithi032' 2020-12-05T17:32:47.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15dbdf6740 1 --2- 172.21.15.32:0/223581183 >> [v2:172.21.15.32:6834/3316547885,v1:172.21.15.32:6835/3316547885] conn(0xc67c00 0xebd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15dbdf6740 4 client.5199 mds_command: new command op to 4873 tid=0[{"prefix": "get_command_descriptions"}] 2020-12-05T17:32:47.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15dbdf6740 1 -- 172.21.15.32:0/223581183 --> [v2:172.21.15.32:6834/3316547885,v1:172.21.15.32:6835/3316547885] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xc67150 con 0xc67c00 2020-12-05T17:32:47.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15be7fc700 1 --2- 172.21.15.32:0/223581183 >> [v2:172.21.15.32:6834/3316547885,v1:172.21.15.32:6835/3316547885] conn(0xc67c00 0xebd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15be7fc700 1 --2- 172.21.15.32:0/223581183 >> [v2:172.21.15.32:6834/3316547885,v1:172.21.15.32:6835/3316547885] conn(0xc67c00 0xebd390 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=886501039cb96cf6 server_cookie=94a1ee3a53b9306a in_seq=0 out_seq=0 2020-12-05T17:32:47.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15a5ffb700 10 client.5199 ms_handle_connect on v2:172.21.15.32:6834/3316547885 2020-12-05T17:32:47.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15a5ffb700 1 -- 172.21.15.32:0/223581183 <== mds.0 v2:172.21.15.32:6834/3316547885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16606 (crc 0 0 0) 0xc67150 con 0xc67c00 2020-12-05T17:32:47.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15a5ffb700 10 client.5199 handle_command_reply: tid=0 2020-12-05T17:32:47.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15dbdf6740 1 client.5199 shutdown 2020-12-05T17:32:47.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15dbdf6740 1 -- 172.21.15.32:0/223581183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xcd5800 msgr2=0xeee6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:47.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15dbdf6740 1 --2- 172.21.15.32:0/223581183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xcd5800 0xeee6b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0xebb340 tx=0x7f15b8017000).stop 2020-12-05T17:32:47.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15dbdf6740 1 -- 172.21.15.32:0/223581183 shutdown_connections 2020-12-05T17:32:47.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15dbdf6740 1 --2- 172.21.15.32:0/223581183 >> [v2:172.21.15.32:6834/3316547885,v1:172.21.15.32:6835/3316547885] conn(0xc67c00 0xebd390 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15dbdf6740 1 --2- 172.21.15.32:0/223581183 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0xe6d060 0xec2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15dbdf6740 1 --2- 172.21.15.32:0/223581183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xcd5800 0xeee6b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15dbdf6740 1 --2- 172.21.15.32:0/223581183 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0xd55e90 0xec2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15a5ffb700 0 client.5199 ms_handle_reset on v2:172.21.15.32:6834/3316547885 2020-12-05T17:32:47.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15dbdf6740 1 -- 172.21.15.32:0/223581183 >> 172.21.15.32:0/223581183 conn(0xe9c9f0 msgr2=0xef10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:47.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15dbdf6740 1 -- 172.21.15.32:0/223581183 shutdown_connections 2020-12-05T17:32:47.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15dbdf6740 1 -- 172.21.15.32:0/223581183 wait complete. 2020-12-05T17:32:47.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.603+0000 7f15dbdf6740 20 client.5199 trim_cache size 0 max 16384 2020-12-05T17:32:47.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.619+0000 7f15dbdf6740 1 Processor -- start 2020-12-05T17:32:47.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.619+0000 7f15dbdf6740 1 -- start start 2020-12-05T17:32:47.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.619+0000 7f15dbdf6740 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0xcd5800 0xeb9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.619+0000 7f15dbdf6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xd55e90 0xef0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.619+0000 7f15dbdf6740 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0xe6d060 0xef10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.619+0000 7f15dbdf6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0xcfa8b0 con 0xd55e90 2020-12-05T17:32:47.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.619+0000 7f15dbdf6740 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0xdedbf0 con 0xcd5800 2020-12-05T17:32:47.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.619+0000 7f15dbdf6740 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0xe91c50 con 0xe6d060 2020-12-05T17:32:47.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.619+0000 7f15be7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xd55e90 0xef0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.619+0000 7f15be7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xd55e90 0xef0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34202/0 (socket says 172.21.15.32:34202) 2020-12-05T17:32:47.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.619+0000 7f15c6ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0xcd5800 0xeb9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.619+0000 7f15be7fc700 1 -- 172.21.15.32:0/513221060 learned_addr learned my addr 172.21.15.32:0/513221060 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:47.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.619+0000 7f15c77fe700 1 --2- 172.21.15.32:0/513221060 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0xe6d060 0xef10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.623+0000 7f15be7fc700 1 -- 172.21.15.32:0/513221060 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0xe6d060 msgr2=0xef10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:47.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.623+0000 7f15be7fc700 1 --2- 172.21.15.32:0/513221060 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0xe6d060 0xef10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.623+0000 7f15be7fc700 1 -- 172.21.15.32:0/513221060 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0xcd5800 msgr2=0xeb9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:47.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.623+0000 7f15be7fc700 1 --2- 172.21.15.32:0/513221060 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0xcd5800 0xeb9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.623+0000 7f15be7fc700 1 -- 172.21.15.32:0/513221060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c8158590 con 0xd55e90 2020-12-05T17:32:47.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.623+0000 7f15be7fc700 1 --2- 172.21.15.32:0/513221060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xd55e90 0xef0c00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0xebffe0 tx=0x7f15b0003040).ready entity=mon.0 client_cookie=79d0298b3b702a05 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:47.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.623+0000 7f15a7fff700 1 -- 172.21.15.32:0/513221060 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15b0023070 con 0xd55e90 2020-12-05T17:32:47.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.623+0000 7f15a7fff700 1 -- 172.21.15.32:0/513221060 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b0024040 con 0xd55e90 2020-12-05T17:32:47.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.623+0000 7f15dbdf6740 1 -- 172.21.15.32:0/513221060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xd55e90 msgr2=0xef0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:32:47.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.623+0000 7f15a7fff700 1 -- 172.21.15.32:0/513221060 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15b00270f0 con 0xd55e90 2020-12-05T17:32:47.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.623+0000 7f15dbdf6740 1 --2- 172.21.15.32:0/513221060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xd55e90 0xef0c00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0xebffe0 tx=0x7f15b0003040).stop 2020-12-05T17:32:47.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.623+0000 7f15dbdf6740 1 -- 172.21.15.32:0/513221060 shutdown_connections 2020-12-05T17:32:47.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.623+0000 7f15dbdf6740 1 --2- 172.21.15.32:0/513221060 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0xe6d060 0xef10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.623+0000 7f15dbdf6740 1 --2- 172.21.15.32:0/513221060 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0xcd5800 0xeb9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.623+0000 7f15dbdf6740 1 --2- 172.21.15.32:0/513221060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xd55e90 0xef0c00 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.623+0000 7f15dbdf6740 1 -- 172.21.15.32:0/513221060 >> 172.21.15.32:0/513221060 conn(0xe9c9f0 msgr2=0xef03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:32:47.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.623+0000 7f15dbdf6740 1 -- 172.21.15.32:0/513221060 shutdown_connections 2020-12-05T17:32:47.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.623+0000 7f15dbdf6740 1 -- 172.21.15.32:0/513221060 wait complete. 2020-12-05T17:32:47.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.623+0000 7f15dbdf6740 1 -- start start 2020-12-05T17:32:47.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.623+0000 7f15dbdf6740 1 Processor -- start 2020-12-05T17:32:47.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.627+0000 7f15dbdf6740 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0xcd5800 0xeb3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.627+0000 7f15dbdf6740 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0xd55e90 0xeb63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.627+0000 7f15dbdf6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xe6d060 0xeb68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.627+0000 7f15dbdf6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0xd90240 con 0xe6d060 2020-12-05T17:32:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.627+0000 7f15dbdf6740 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0xe9bd40 con 0xd55e90 2020-12-05T17:32:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.627+0000 7f15dbdf6740 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0xefeca0 con 0xcd5800 2020-12-05T17:32:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.627+0000 7f15dbdf6740 10 client.0 ll_register_callbacks cb 0x7f15b8025070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-12-05T17:32:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.627+0000 7f15c77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xe6d060 0xeb68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.627+0000 7f15be7fc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0xd55e90 0xeb63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.627+0000 7f15c6ffd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0xcd5800 0xeb3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.627+0000 7f15be7fc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0xd55e90 0xeb63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:54100/0 (socket says 172.21.15.32:54100) 2020-12-05T17:32:47.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.627+0000 7f15be7fc700 1 -- 172.21.15.32:0/1809304921 learned_addr learned my addr 172.21.15.32:0/1809304921 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:32:47.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.627+0000 7f15be7fc700 1 -- 172.21.15.32:0/1809304921 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0xcd5800 msgr2=0xeb3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:47.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.627+0000 7f15be7fc700 1 --2- 172.21.15.32:0/1809304921 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0xcd5800 0xeb3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.627+0000 7f15be7fc700 1 -- 172.21.15.32:0/1809304921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xe6d060 msgr2=0xeb68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:32:47.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.627+0000 7f15be7fc700 1 --2- 172.21.15.32:0/1809304921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xe6d060 0xeb68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:32:47.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.627+0000 7f15be7fc700 1 -- 172.21.15.32:0/1809304921 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xc90e50 con 0xd55e90 2020-12-05T17:32:47.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.627+0000 7f15be7fc700 1 --2- 172.21.15.32:0/1809304921 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0xd55e90 0xeb63b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0xeba6e0 tx=0x7f15b0026000).ready entity=mon.1 client_cookie=4a0371fba853f7b7 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:32:47.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.627+0000 7f15a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2020-12-05T17:32:47.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.631+0000 7f15a6ffd700 1 -- 172.21.15.32:0/1809304921 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15b0023070 con 0xd55e90 2020-12-05T17:32:47.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.631+0000 7f15dbdf6740 1 -- 172.21.15.32:0/1809304921 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xd19710 con 0xd55e90 2020-12-05T17:32:47.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.631+0000 7f15a6ffd700 1 -- 172.21.15.32:0/1809304921 <== mon.1 v2:172.21.15.61:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f15b0001e40 con 0xd55e90 2020-12-05T17:32:47.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.631+0000 7f15a6ffd700 1 -- 172.21.15.32:0/1809304921 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=10) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15b000cc30 con 0xd55e90 2020-12-05T17:32:47.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.631+0000 7f15dbdf6740 10 client.5122 fetch_fsmap learned FSMap version 10 2020-12-05T17:32:47.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.631+0000 7f15dbdf6740 1 -- 172.21.15.32:0/1809304921 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=10,monmap=2+}) v3 -- 0xeefd20 con 0xd55e90 2020-12-05T17:32:47.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.631+0000 7f15a6ffd700 1 -- 172.21.15.32:0/1809304921 <== mon.1 v2:172.21.15.61:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xd19710 con 0xd55e90 2020-12-05T17:32:47.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.631+0000 7f15a6ffd700 1 -- 172.21.15.32:0/1809304921 <== mon.1 v2:172.21.15.61:3300/0 5 ==== fsmap(e 10) v1 ==== 1247+0+0 (secure 0 0 0) 0x7f15b0027070 con 0xd55e90 2020-12-05T17:32:47.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.631+0000 7f15dbdf6740 10 client.5122 fetch_fsmap finished waiting for FSMap version 10 2020-12-05T17:32:47.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.631+0000 7f15dbdf6740 10 client.5122 resolve_mds: resolved ID 'a' to GID 4873 2020-12-05T17:32:47.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.631+0000 7f15dbdf6740 20 client.5122 populate_metadata read hostname 'smithi032' 2020-12-05T17:32:47.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.631+0000 7f15dbdf6740 1 --2- 172.21.15.32:0/1809304921 >> [v2:172.21.15.32:6834/3316547885,v1:172.21.15.32:6835/3316547885] conn(0xebd390 0xebd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:32:47.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.631+0000 7f15dbdf6740 4 client.5122 mds_command: new command op to 4873 tid=0[{"prefix": "cache drop"}] 2020-12-05T17:32:47.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.631+0000 7f15dbdf6740 1 -- 172.21.15.32:0/1809304921 --> [v2:172.21.15.32:6834/3316547885,v1:172.21.15.32:6835/3316547885] -- command(tid 0: {"prefix": "cache drop"}) v1 -- 0xcf0420 con 0xebd390 2020-12-05T17:32:47.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.631+0000 7f15c6ffd700 1 --2- 172.21.15.32:0/1809304921 >> [v2:172.21.15.32:6834/3316547885,v1:172.21.15.32:6835/3316547885] conn(0xebd390 0xebd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:32:47.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.631+0000 7f15a6ffd700 10 client.5122 ms_handle_connect on v2:172.21.15.32:6834/3316547885 2020-12-05T17:32:47.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:32:47.631+0000 7f15c6ffd700 1 --2- 172.21.15.32:0/1809304921 >> [v2:172.21.15.32:6834/3316547885,v1:172.21.15.32:6835/3316547885] conn(0xebd390 0xebd770 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=f694d835f1adf0ad server_cookie=1cfb73ab8ce0d31a in_seq=0 out_seq=0 2020-12-05T17:33:14.896 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:33:14.899 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:33:14.901 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:33:14.903 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:33:14.906 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:33:32.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.099+0000 7f15a6ffd700 1 -- 172.21.15.32:0/1809304921 <== mds.0 v2:172.21.15.32:6834/3316547885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+386 (crc 0 0 0) 0xcf0420 con 0xebd390 2020-12-05T17:33:32.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.099+0000 7f15a6ffd700 10 client.5122 handle_command_reply: tid=0 2020-12-05T17:33:32.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.099+0000 7f15dbdf6740 1 client.5122 shutdown 2020-12-05T17:33:32.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.099+0000 7f15dbdf6740 1 -- 172.21.15.32:0/1809304921 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0xd55e90 msgr2=0xeb63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:32.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.099+0000 7f15dbdf6740 1 --2- 172.21.15.32:0/1809304921 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0xd55e90 0xeb63b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0xeba6e0 tx=0x7f15b0026000).stop 2020-12-05T17:33:32.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.099+0000 7f15dbdf6740 1 -- 172.21.15.32:0/1809304921 shutdown_connections 2020-12-05T17:33:32.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.099+0000 7f15dbdf6740 1 --2- 172.21.15.32:0/1809304921 >> [v2:172.21.15.32:6834/3316547885,v1:172.21.15.32:6835/3316547885] conn(0xebd390 0xebd770 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:32.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.099+0000 7f15dbdf6740 1 --2- 172.21.15.32:0/1809304921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xe6d060 0xeb68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:32.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.099+0000 7f15a6ffd700 0 client.5122 ms_handle_reset on v2:172.21.15.32:6834/3316547885 2020-12-05T17:33:32.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.099+0000 7f15dbdf6740 1 --2- 172.21.15.32:0/1809304921 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0xcd5800 0xeb3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:32.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.099+0000 7f15dbdf6740 1 --2- 172.21.15.32:0/1809304921 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0xd55e90 0xeb63b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:32.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.099+0000 7f15dbdf6740 1 -- 172.21.15.32:0/1809304921 >> 172.21.15.32:0/1809304921 conn(0xe9c9f0 msgr2=0xeb9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:32.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.099+0000 7f15dbdf6740 1 -- 172.21.15.32:0/1809304921 shutdown_connections 2020-12-05T17:33:32.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.099+0000 7f15dbdf6740 1 -- 172.21.15.32:0/1809304921 wait complete. 2020-12-05T17:33:32.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.103+0000 7f15dbdf6740 20 client.5122 trim_cache size 0 max 16384 2020-12-05T17:33:32.106 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:33:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "client_recall": { 2020-12-05T17:33:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2020-12-05T17:33:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "(0) Success", 2020-12-05T17:33:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "recalled": 920 2020-12-05T17:33:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "flush_journal": { 2020-12-05T17:33:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2020-12-05T17:33:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "" 2020-12-05T17:33:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "trim_cache": { 2020-12-05T17:33:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "trimmed": 900 2020-12-05T17:33:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "cache": { 2020-12-05T17:33:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": { 2020-12-05T17:33:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "items": 2525, 2020-12-05T17:33:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes": 431920 2020-12-05T17:33:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 44.457641221999999 2020-12-05T17:33:32.110 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:33:32.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.103+0000 7f15cd45b700 1 -- 172.21.15.32:0/4013371575 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15ac055330 msgr2=0x7f15ac0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:32.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.103+0000 7f15cd45b700 1 --2- 172.21.15.32:0/4013371575 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15ac055330 0x7f15ac0577f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f15b8005000 tx=0x7f15b8007740).stop 2020-12-05T17:33:32.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.103+0000 7f15cd45b700 1 -- 172.21.15.32:0/4013371575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15c8151a10 msgr2=0x7f15c80aa750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:32.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.103+0000 7f15cd45b700 1 --2- 172.21.15.32:0/4013371575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15c8151a10 0x7f15c80aa750 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f15c000b000 tx=0x7f15c00091a0).stop 2020-12-05T17:33:32.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.103+0000 7f15cd45b700 1 -- 172.21.15.32:0/4013371575 shutdown_connections 2020-12-05T17:33:32.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.103+0000 7f15cd45b700 1 --2- 172.21.15.32:0/4013371575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15c8151a10 0x7f15c80aa750 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:32.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.103+0000 7f15cd45b700 1 --2- 172.21.15.32:0/4013371575 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15c81421e0 0x7f15c809e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:32.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.103+0000 7f15cd45b700 1 --2- 172.21.15.32:0/4013371575 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15ac055330 0x7f15ac0577f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:32.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.103+0000 7f15cd45b700 1 --2- 172.21.15.32:0/4013371575 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15c8144a60 0x7f15c8151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:32.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.103+0000 7f15cd45b700 1 -- 172.21.15.32:0/4013371575 >> 172.21.15.32:0/4013371575 conn(0x7f15c805fe80 msgr2=0x7f15c80b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:32.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.103+0000 7f15cd45b700 1 -- 172.21.15.32:0/4013371575 shutdown_connections 2020-12-05T17:33:32.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.103+0000 7f15cd45b700 1 -- 172.21.15.32:0/4013371575 wait complete. 2020-12-05T17:33:32.118 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:33:32.119 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:33:32.136 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:33:32.152 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:33:32.169 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:33:32.186 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:33:32.204 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:33:32.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:33:32.205 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:33:32.205 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:33:32.218 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:33:32.219 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:33:32.220 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi036.front.sepia.ceph.com... 2020-12-05T17:33:32.220 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:33:32.221 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:33:32.304 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[10361]: fuse finished with error 0 and tester_r 0 2020-12-05T17:33:32.306 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:33:32.306 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:33:32.306 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:33:32.353 INFO:teuthology.orchestra.run.smithi036.stdout:ext2/ext3 2020-12-05T17:33:32.354 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:33:32.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:33:32.472 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:33:32.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:33:32.519 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:33:32.520 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:33:32.528 INFO:teuthology.orchestra.run.smithi002.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-12-05T17:33:32.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:33:32.530 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-12-05T17:33:32.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:33:32.532 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:33:32.577 INFO:teuthology.orchestra.run.smithi002.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-12-05T17:33:32.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:33:32.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:33:32.580 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:33:32.581 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:33:32.588 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:33:32.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:33:32.591 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2020-12-05T17:33:32.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:33:32.593 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:33:32.636 INFO:teuthology.orchestra.run.smithi053.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:33:32.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:33:32.639 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2020-12-05T17:33:32.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.767+0000 7ff86ffff700 1 Processor -- start 2020-12-05T17:33:32.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.767+0000 7ff86ffff700 1 -- start start 2020-12-05T17:33:32.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.767+0000 7ff86ffff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff8700af230 0x7ff870145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:32.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.767+0000 7ff86ffff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff870146a90 con 0x7ff8700b1fc0 2020-12-05T17:33:32.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.767+0000 7ff86ffff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8700b23a0 con 0x7ff8700ab710 2020-12-05T17:33:32.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.767+0000 7ff86ffff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff870061e90 con 0x7ff8700af230 2020-12-05T17:33:32.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.767+0000 7ff86effd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff8700b1fc0 0x7ff870061780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36866/0 (socket says 172.21.15.32:36866) 2020-12-05T17:33:32.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.767+0000 7ff86effd700 1 -- 172.21.15.32:0/2929131761 learned_addr learned my addr 172.21.15.32:0/2929131761 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:33:32.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.767+0000 7ff86f7fe700 1 --2- 172.21.15.32:0/2929131761 >> v2:172.21.15.61:3300/0 conn(0x7ff8700af230 0x7ff870145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:32.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.767+0000 7ff86dffb700 1 -- 172.21.15.32:0/2929131761 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051079187 0 0) 0x7ff870146a90 con 0x7ff8700b1fc0 2020-12-05T17:33:32.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.767+0000 7ff86dffb700 1 -- 172.21.15.32:0/2929131761 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff850003230 con 0x7ff8700b1fc0 2020-12-05T17:33:32.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.767+0000 7ff86f7fe700 1 -- 172.21.15.32:0/2929131761 >> v1:172.21.15.61:6789/0 conn(0x7ff8700ab710 legacy=0x7ff870147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:32.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.767+0000 7ff86f7fe700 1 -- 172.21.15.32:0/2929131761 >> v1:172.21.15.32:6789/0 conn(0x7ff8700b1fc0 legacy=0x7ff870061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:32.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.767+0000 7ff86f7fe700 1 -- 172.21.15.32:0/2929131761 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff870146cc0 con 0x7ff8700af230 2020-12-05T17:33:32.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.767+0000 7ff86f7fe700 1 --2- 172.21.15.32:0/2929131761 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff8700af230 0x7ff870145d10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7ff8680032b0 tx=0x7ff8680091a0).ready entity=mon.1 client_cookie=b689731519a1484 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:32.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.771+0000 7ff86dffb700 1 -- 172.21.15.32:0/2929131761 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff86800b9a0 con 0x7ff8700af230 2020-12-05T17:33:32.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.771+0000 7ff86dffb700 1 -- 172.21.15.32:0/2929131761 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86800bb00 con 0x7ff8700af230 2020-12-05T17:33:32.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.771+0000 7ff86dffb700 1 -- 172.21.15.32:0/2929131761 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff86800bc70 con 0x7ff8700af230 2020-12-05T17:33:32.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.771+0000 7ff86ffff700 1 -- 172.21.15.32:0/2929131761 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff8700af230 msgr2=0x7ff870145d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:32.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.771+0000 7ff86ffff700 1 --2- 172.21.15.32:0/2929131761 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff8700af230 0x7ff870145d10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7ff8680032b0 tx=0x7ff8680091a0).stop 2020-12-05T17:33:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.771+0000 7ff86ffff700 1 -- 172.21.15.32:0/2929131761 shutdown_connections 2020-12-05T17:33:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.771+0000 7ff86ffff700 1 --2- 172.21.15.32:0/2929131761 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff8700af230 0x7ff870145d10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.771+0000 7ff86ffff700 1 -- 172.21.15.32:0/2929131761 >> 172.21.15.32:0/2929131761 conn(0x7ff8700b5530 msgr2=0x7ff8700b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.771+0000 7ff86ffff700 1 -- 172.21.15.32:0/2929131761 shutdown_connections 2020-12-05T17:33:32.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.771+0000 7ff86ffff700 1 -- 172.21.15.32:0/2929131761 wait complete. 2020-12-05T17:33:32.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.771+0000 7ff86ffff700 1 Processor -- start 2020-12-05T17:33:32.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.771+0000 7ff86ffff700 1 -- start start 2020-12-05T17:33:32.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.771+0000 7ff86ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8700ab710 0x7ff870155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:32.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.771+0000 7ff86ffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8700af230 0x7ff870143330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:32.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.771+0000 7ff86ffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff8700b1fc0 0x7ff8701499e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.771+0000 7ff86ffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff870142a20 con 0x7ff8700ab710 2020-12-05T17:33:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.771+0000 7ff86ffff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8700ad870 con 0x7ff8700b1fc0 2020-12-05T17:33:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.771+0000 7ff86ffff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8700add40 con 0x7ff8700af230 2020-12-05T17:33:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.771+0000 7ff86effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8700ab710 0x7ff870155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.771+0000 7ff86effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8700ab710 0x7ff870155700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34222/0 (socket says 172.21.15.32:34222) 2020-12-05T17:33:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.771+0000 7ff86effd700 1 -- 172.21.15.32:0/1573832374 learned_addr learned my addr 172.21.15.32:0/1573832374 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:32.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.771+0000 7ff86e7fc700 1 --2- 172.21.15.32:0/1573832374 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8700af230 0x7ff870143330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:32.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.771+0000 7ff86f7fe700 1 --2- 172.21.15.32:0/1573832374 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff8700b1fc0 0x7ff8701499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:32.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.775+0000 7ff86effd700 1 -- 172.21.15.32:0/1573832374 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8700af230 msgr2=0x7ff870143330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:32.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.775+0000 7ff86effd700 1 --2- 172.21.15.32:0/1573832374 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8700af230 0x7ff870143330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:32.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.775+0000 7ff86effd700 1 -- 172.21.15.32:0/1573832374 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff8700b1fc0 msgr2=0x7ff8701499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:32.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.775+0000 7ff86effd700 1 --2- 172.21.15.32:0/1573832374 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff8700b1fc0 0x7ff8701499e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:32.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.775+0000 7ff86effd700 1 -- 172.21.15.32:0/1573832374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8700adaa0 con 0x7ff8700ab710 2020-12-05T17:33:32.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.775+0000 7ff86effd700 1 --2- 172.21.15.32:0/1573832374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8700ab710 0x7ff870155700 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7ff8580010d0 tx=0x7ff858007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:32.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.775+0000 7ff867fff700 1 -- 172.21.15.32:0/1573832374 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8580072e0 con 0x7ff8700ab710 2020-12-05T17:33:32.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.775+0000 7ff867fff700 1 -- 172.21.15.32:0/1573832374 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff858007440 con 0x7ff8700ab710 2020-12-05T17:33:32.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.775+0000 7ff86ffff700 1 -- 172.21.15.32:0/1573832374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff87014ac10 con 0x7ff8700ab710 2020-12-05T17:33:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.775+0000 7ff867fff700 1 -- 172.21.15.32:0/1573832374 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8580057d0 con 0x7ff8700ab710 2020-12-05T17:33:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.775+0000 7ff86ffff700 1 -- 172.21.15.32:0/1573832374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff87014a070 con 0x7ff8700ab710 2020-12-05T17:33:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.775+0000 7ff86ffff700 1 -- 172.21.15.32:0/1573832374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff870057050 con 0x7ff8700ab710 2020-12-05T17:33:32.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.775+0000 7ff867fff700 1 -- 172.21.15.32:0/1573832374 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff85801a060 con 0x7ff8700ab710 2020-12-05T17:33:32.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.775+0000 7ff867fff700 1 --2- 172.21.15.32:0/1573832374 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff854055350 0x7ff854057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:32.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.779+0000 7ff86e7fc700 1 --2- 172.21.15.32:0/1573832374 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff854055350 0x7ff854057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:32.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.779+0000 7ff867fff700 1 -- 172.21.15.32:0/1573832374 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff85806d340 con 0x7ff8700ab710 2020-12-05T17:33:32.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.779+0000 7ff86e7fc700 1 --2- 172.21.15.32:0/1573832374 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff854055350 0x7ff854057810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7ff86000d000 tx=0x7ff860007b40).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:32.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.783+0000 7ff867fff700 1 -- 172.21.15.32:0/1573832374 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff8580434e0 con 0x7ff8700ab710 2020-12-05T17:33:32.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.927+0000 7ff86ffff700 1 -- 172.21.15.32:0/1573832374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff87005aab0 con 0x7ff8700ab710 2020-12-05T17:33:32.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.935+0000 7ff867fff700 1 -- 172.21.15.32:0/1573832374 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff8580180a0 con 0x7ff8700ab710 2020-12-05T17:33:32.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.935+0000 7ff86ffff700 1 -- 172.21.15.32:0/1573832374 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff854055350 msgr2=0x7ff854057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:32.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.935+0000 7ff86ffff700 1 --2- 172.21.15.32:0/1573832374 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff854055350 0x7ff854057810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7ff86000d000 tx=0x7ff860007b40).stop 2020-12-05T17:33:32.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.935+0000 7ff86ffff700 1 -- 172.21.15.32:0/1573832374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8700ab710 msgr2=0x7ff870155700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:32.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.935+0000 7ff86ffff700 1 --2- 172.21.15.32:0/1573832374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8700ab710 0x7ff870155700 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7ff8580010d0 tx=0x7ff858007aa0).stop 2020-12-05T17:33:32.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.935+0000 7ff86ffff700 1 -- 172.21.15.32:0/1573832374 shutdown_connections 2020-12-05T17:33:32.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.935+0000 7ff86ffff700 1 --2- 172.21.15.32:0/1573832374 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff8700b1fc0 0x7ff8701499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:32.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.935+0000 7ff86ffff700 1 --2- 172.21.15.32:0/1573832374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8700ab710 0x7ff870155700 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:32.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.935+0000 7ff86ffff700 1 --2- 172.21.15.32:0/1573832374 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff854055350 0x7ff854057810 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:32.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.935+0000 7ff86ffff700 1 --2- 172.21.15.32:0/1573832374 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8700af230 0x7ff870143330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:32.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.935+0000 7ff86ffff700 1 -- 172.21.15.32:0/1573832374 >> 172.21.15.32:0/1573832374 conn(0x7ff8700b5530 msgr2=0x7ff8700b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:32.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.935+0000 7ff86ffff700 1 -- 172.21.15.32:0/1573832374 shutdown_connections 2020-12-05T17:33:32.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:32.935+0000 7ff86ffff700 1 -- 172.21.15.32:0/1573832374 wait complete. 2020-12-05T17:33:32.953 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command' 2020-12-05T17:33:33.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.067+0000 7f0ff3176700 1 Processor -- start 2020-12-05T17:33:33.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.067+0000 7f0ff3176700 1 -- start start 2020-12-05T17:33:33.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.067+0000 7f0ff3176700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0fec0b7720 0x7f0fec061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:33.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.067+0000 7f0ff3176700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0fec144540 0x7f0fec13d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:33.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.067+0000 7f0ff3176700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fec143a00 con 0x7f0fec1403f0 2020-12-05T17:33:33.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.067+0000 7f0ff3176700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fec140870 con 0x7f0fec144540 2020-12-05T17:33:33.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.067+0000 7f0ff3176700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec0b7b00 con 0x7f0fec0b7720 2020-12-05T17:33:33.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.067+0000 7f0ff1973700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0fec1403f0 0x7f0fec143e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36878/0 (socket says 172.21.15.32:36878) 2020-12-05T17:33:33.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.067+0000 7f0ff1973700 1 -- 172.21.15.32:0/835329731 learned_addr learned my addr 172.21.15.32:0/835329731 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:33:33.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.067+0000 7f0ff2975700 1 --2- 172.21.15.32:0/835329731 >> v2:172.21.15.32:3300/0 conn(0x7f0fec144540 0x7f0fec13d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:33.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.067+0000 7f0ff2174700 1 --2- 172.21.15.32:0/835329731 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0fec0b7720 0x7f0fec061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:33.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.067+0000 7f0ff1172700 1 -- 172.21.15.32:0/835329731 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1282723676 0 0) 0x7f0fec143a00 con 0x7f0fec1403f0 2020-12-05T17:33:33.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.067+0000 7f0ff1172700 1 -- 172.21.15.32:0/835329731 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fcc003230 con 0x7f0fec1403f0 2020-12-05T17:33:33.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.067+0000 7f0ff2975700 1 -- 172.21.15.32:0/835329731 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0fec0b7720 msgr2=0x7f0fec061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:33.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.067+0000 7f0ff2975700 1 --2- 172.21.15.32:0/835329731 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0fec0b7720 0x7f0fec061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:33.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.067+0000 7f0ff2975700 1 -- 172.21.15.32:0/835329731 >> v1:172.21.15.32:6789/0 conn(0x7f0fec1403f0 legacy=0x7f0fec143e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:33.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.067+0000 7f0ff2975700 1 -- 172.21.15.32:0/835329731 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fec05fe60 con 0x7f0fec144540 2020-12-05T17:33:33.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.067+0000 7f0ff2174700 1 --2- 172.21.15.32:0/835329731 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0fec0b7720 0x7f0fec061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:33:33.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff2975700 1 --2- 172.21.15.32:0/835329731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fec144540 0x7f0fec13d3c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f0fd4002f30 tx=0x7f0fd4006dc0).ready entity=mon.0 client_cookie=ddda1616ba5da8f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:33.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff1172700 1 -- 172.21.15.32:0/835329731 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fd4009760 con 0x7f0fec144540 2020-12-05T17:33:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff1172700 1 -- 172.21.15.32:0/835329731 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd40098c0 con 0x7f0fec144540 2020-12-05T17:33:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff1172700 1 -- 172.21.15.32:0/835329731 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fd4009c20 con 0x7f0fec144540 2020-12-05T17:33:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff3176700 1 -- 172.21.15.32:0/835329731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fec144540 msgr2=0x7f0fec13d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:33.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff3176700 1 --2- 172.21.15.32:0/835329731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fec144540 0x7f0fec13d3c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f0fd4002f30 tx=0x7f0fd4006dc0).stop 2020-12-05T17:33:33.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff3176700 1 -- 172.21.15.32:0/835329731 shutdown_connections 2020-12-05T17:33:33.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff3176700 1 --2- 172.21.15.32:0/835329731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fec144540 0x7f0fec13d3c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:33.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff3176700 1 --2- 172.21.15.32:0/835329731 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0fec0b7720 0x7f0fec061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:33.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff3176700 1 -- 172.21.15.32:0/835329731 >> 172.21.15.32:0/835329731 conn(0x7f0fec05bc20 msgr2=0x7f0fec0b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:33.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff3176700 1 -- 172.21.15.32:0/835329731 shutdown_connections 2020-12-05T17:33:33.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff3176700 1 -- 172.21.15.32:0/835329731 wait complete. 2020-12-05T17:33:33.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff3176700 1 Processor -- start 2020-12-05T17:33:33.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff3176700 1 -- start start 2020-12-05T17:33:33.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff3176700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0fec0b7720 0x7f0fec0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:33.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff3176700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fec1403f0 0x7f0fec0b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:33.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff3176700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0fec144540 0x7f0fec0b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:33.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff3176700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec0b3310 con 0x7f0fec1403f0 2020-12-05T17:33:33.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff3176700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec0b35b0 con 0x7f0fec0b7720 2020-12-05T17:33:33.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff3176700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec111eb0 con 0x7f0fec144540 2020-12-05T17:33:33.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff1973700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fec1403f0 0x7f0fec0b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:33.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff1973700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fec1403f0 0x7f0fec0b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34238/0 (socket says 172.21.15.32:34238) 2020-12-05T17:33:33.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff1973700 1 -- 172.21.15.32:0/1569590507 learned_addr learned my addr 172.21.15.32:0/1569590507 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:33.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff2174700 1 --2- 172.21.15.32:0/1569590507 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0fec0b7720 0x7f0fec0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:33.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff2975700 1 --2- 172.21.15.32:0/1569590507 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0fec144540 0x7f0fec0b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:33.093 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff1973700 1 -- 172.21.15.32:0/1569590507 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0fec144540 msgr2=0x7f0fec0b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:33.093 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff1973700 1 --2- 172.21.15.32:0/1569590507 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0fec144540 0x7f0fec0b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:33.093 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff1973700 1 -- 172.21.15.32:0/1569590507 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0fec0b7720 msgr2=0x7f0fec0b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:33.094 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff1973700 1 --2- 172.21.15.32:0/1569590507 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0fec0b7720 0x7f0fec0b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:33.094 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.071+0000 7f0ff1973700 1 -- 172.21.15.32:0/1569590507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fec0b4090 con 0x7f0fec1403f0 2020-12-05T17:33:33.094 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.075+0000 7f0ff1973700 1 --2- 172.21.15.32:0/1569590507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fec1403f0 0x7f0fec0b04d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f0fdc001450 tx=0x7f0fdc007ed0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:33.094 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.075+0000 7f0feb7fe700 1 -- 172.21.15.32:0/1569590507 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fdc005b70 con 0x7f0fec1403f0 2020-12-05T17:33:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.075+0000 7f0feb7fe700 1 -- 172.21.15.32:0/1569590507 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fdc005cd0 con 0x7f0fec1403f0 2020-12-05T17:33:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.075+0000 7f0ff3176700 1 -- 172.21.15.32:0/1569590507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fec0b4340 con 0x7f0fec1403f0 2020-12-05T17:33:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.075+0000 7f0feb7fe700 1 -- 172.21.15.32:0/1569590507 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fdc007300 con 0x7f0fec1403f0 2020-12-05T17:33:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.075+0000 7f0ff3176700 1 -- 172.21.15.32:0/1569590507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fec1124f0 con 0x7f0fec1403f0 2020-12-05T17:33:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.075+0000 7f0ff3176700 1 -- 172.21.15.32:0/1569590507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fec112070 con 0x7f0fec1403f0 2020-12-05T17:33:33.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.075+0000 7f0feb7fe700 1 -- 172.21.15.32:0/1569590507 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0fdc010030 con 0x7f0fec1403f0 2020-12-05T17:33:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.075+0000 7f0feb7fe700 1 --2- 172.21.15.32:0/1569590507 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0fd00553b0 0x7f0fd0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.079+0000 7f0ff2174700 1 --2- 172.21.15.32:0/1569590507 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0fd00553b0 0x7f0fd0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.079+0000 7f0feb7fe700 1 -- 172.21.15.32:0/1569590507 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0fdc06e060 con 0x7f0fec1403f0 2020-12-05T17:33:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.079+0000 7f0ff2174700 1 --2- 172.21.15.32:0/1569590507 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0fd00553b0 0x7f0fd0057870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f0fe400c000 tx=0x7f0fe4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:33.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.083+0000 7f0feb7fe700 1 -- 172.21.15.32:0/1569590507 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0fdc0441a0 con 0x7f0fec1403f0 2020-12-05T17:33:33.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.271+0000 7f0ff3176700 1 -- 172.21.15.32:0/1569590507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]} v 0) v1 -- 0x7f0fec057050 con 0x7f0fec1403f0 2020-12-05T17:33:33.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.891+0000 7f0feb7fe700 1 -- 172.21.15.32:0/1569590507 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]}]=0 v102) v1 ==== 139+0+0 (secure 0 0 0) 0x7f0fdc0477c0 con 0x7f0fec1403f0 2020-12-05T17:33:33.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.895+0000 7f0ff3176700 1 -- 172.21.15.32:0/1569590507 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0fd00553b0 msgr2=0x7f0fd0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:33.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.895+0000 7f0ff3176700 1 --2- 172.21.15.32:0/1569590507 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0fd00553b0 0x7f0fd0057870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f0fe400c000 tx=0x7f0fe4008040).stop 2020-12-05T17:33:33.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.895+0000 7f0ff3176700 1 -- 172.21.15.32:0/1569590507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fec1403f0 msgr2=0x7f0fec0b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:33.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.895+0000 7f0ff3176700 1 --2- 172.21.15.32:0/1569590507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fec1403f0 0x7f0fec0b04d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f0fdc001450 tx=0x7f0fdc007ed0).stop 2020-12-05T17:33:33.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.895+0000 7f0ff3176700 1 -- 172.21.15.32:0/1569590507 shutdown_connections 2020-12-05T17:33:33.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.895+0000 7f0ff3176700 1 --2- 172.21.15.32:0/1569590507 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0fd00553b0 0x7f0fd0057870 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:33.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.895+0000 7f0ff3176700 1 --2- 172.21.15.32:0/1569590507 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0fec144540 0x7f0fec0b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:33.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.895+0000 7f0ff3176700 1 --2- 172.21.15.32:0/1569590507 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0fec0b7720 0x7f0fec0b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:33.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.895+0000 7f0ff3176700 1 --2- 172.21.15.32:0/1569590507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fec1403f0 0x7f0fec0b04d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:33.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.895+0000 7f0ff3176700 1 -- 172.21.15.32:0/1569590507 >> 172.21.15.32:0/1569590507 conn(0x7f0fec05bc20 msgr2=0x7f0fec0b6300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:33.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.895+0000 7f0ff3176700 1 -- 172.21.15.32:0/1569590507 shutdown_connections 2020-12-05T17:33:33.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:33.895+0000 7f0ff3176700 1 -- 172.21.15.32:0/1569590507 wait complete. 2020-12-05T17:33:33.913 INFO:tasks.cephfs_test_runner:test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2020-12-05T17:33:33.913 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_dead (tasks.cephfs.test_misc.TestCacheDrop) 2020-12-05T17:33:33.914 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead' 2020-12-05T17:33:34.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.031+0000 7f02fceba700 1 Processor -- start 2020-12-05T17:33:34.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.031+0000 7f02fceba700 1 -- start start 2020-12-05T17:33:34.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02fceba700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f02f8142570 0x7f02f8142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:34.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02fceba700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02f80af0d0 0x7f02f80af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:34.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02fceba700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02f80b2390 con 0x7f02f8061cf0 2020-12-05T17:33:34.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02fceba700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02f813ca50 con 0x7f02f80af0d0 2020-12-05T17:33:34.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02fceba700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02f8143410 con 0x7f02f8142570 2020-12-05T17:33:34.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02f7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02f80af0d0 0x7f02f80af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:34.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02f7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02f80af0d0 0x7f02f80af4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34248/0 (socket says 172.21.15.32:34248) 2020-12-05T17:33:34.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02f7fff700 1 -- 172.21.15.32:0/1078126124 learned_addr learned my addr 172.21.15.32:0/1078126124 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:34.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02eeffd700 1 --2- 172.21.15.32:0/1078126124 >> v2:172.21.15.61:3300/0 conn(0x7f02f8142570 0x7f02f8142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:34.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02f7fff700 1 -- 172.21.15.32:0/1078126124 >> v2:172.21.15.61:3300/0 conn(0x7f02f8142570 msgr2=0x7f02f8142f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:34.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02f7fff700 1 --2- 172.21.15.32:0/1078126124 >> v2:172.21.15.61:3300/0 conn(0x7f02f8142570 0x7f02f8142f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:34.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02f6ffd700 1 -- 172.21.15.32:0/1078126124 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 437390172 0 0) 0x7f02f80b2390 con 0x7f02f8061cf0 2020-12-05T17:33:34.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02f7fff700 1 -- 172.21.15.32:0/1078126124 >> v1:172.21.15.32:6789/0 conn(0x7f02f8061cf0 legacy=0x7f02f80b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:34.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02f7fff700 1 -- 172.21.15.32:0/1078126124 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f8062430 con 0x7f02f80af0d0 2020-12-05T17:33:34.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02f7fff700 1 --2- 172.21.15.32:0/1078126124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f80af0d0 0x7f02f80af4e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f02f00032b0 tx=0x7f02f0009270).ready entity=mon.0 client_cookie=374b9edc2e277d42 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:34.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02f6ffd700 1 -- 172.21.15.32:0/1078126124 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02f000bc00 con 0x7f02f80af0d0 2020-12-05T17:33:34.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02f6ffd700 1 -- 172.21.15.32:0/1078126124 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f000bd60 con 0x7f02f80af0d0 2020-12-05T17:33:34.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02f6ffd700 1 -- 172.21.15.32:0/1078126124 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02f0011400 con 0x7f02f80af0d0 2020-12-05T17:33:34.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02fceba700 1 -- 172.21.15.32:0/1078126124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f80af0d0 msgr2=0x7f02f80af4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:34.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02fceba700 1 --2- 172.21.15.32:0/1078126124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f80af0d0 0x7f02f80af4e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f02f00032b0 tx=0x7f02f0009270).stop 2020-12-05T17:33:34.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02fceba700 1 -- 172.21.15.32:0/1078126124 shutdown_connections 2020-12-05T17:33:34.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02fceba700 1 --2- 172.21.15.32:0/1078126124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f80af0d0 0x7f02f80af4e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:34.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02fceba700 1 --2- 172.21.15.32:0/1078126124 >> v2:172.21.15.61:3300/0 conn(0x7f02f8142570 0x7f02f8142f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:34.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02fceba700 1 -- 172.21.15.32:0/1078126124 >> 172.21.15.32:0/1078126124 conn(0x7f02f80b5530 msgr2=0x7f02f80b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:34.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02fceba700 1 -- 172.21.15.32:0/1078126124 shutdown_connections 2020-12-05T17:33:34.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.035+0000 7f02fceba700 1 -- 172.21.15.32:0/1078126124 wait complete. 2020-12-05T17:33:34.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02fceba700 1 Processor -- start 2020-12-05T17:33:34.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02fceba700 1 -- start start 2020-12-05T17:33:34.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02fceba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f8061cf0 0x7f02f8151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:34.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02fceba700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f02f80af0d0 0x7f02f813f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:34.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02fceba700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02f8142570 0x7f02f8151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:34.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02fceba700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f8063000 con 0x7f02f8061cf0 2020-12-05T17:33:34.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02fceba700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f81460d0 con 0x7f02f8142570 2020-12-05T17:33:34.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02fceba700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f8063230 con 0x7f02f80af0d0 2020-12-05T17:33:34.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02f77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f8061cf0 0x7f02f8151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:34.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02eeffd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f02f80af0d0 0x7f02f813f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:34.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02f7fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02f8142570 0x7f02f8151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:34.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02f77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f8061cf0 0x7f02f8151610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34250/0 (socket says 172.21.15.32:34250) 2020-12-05T17:33:34.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02f77fe700 1 -- 172.21.15.32:0/2153206887 learned_addr learned my addr 172.21.15.32:0/2153206887 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:34.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02f77fe700 1 -- 172.21.15.32:0/2153206887 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f02f80af0d0 msgr2=0x7f02f813f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:34.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02f77fe700 1 --2- 172.21.15.32:0/2153206887 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f02f80af0d0 0x7f02f813f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:34.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02f77fe700 1 -- 172.21.15.32:0/2153206887 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02f8142570 msgr2=0x7f02f8151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:34.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02f77fe700 1 --2- 172.21.15.32:0/2153206887 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02f8142570 0x7f02f8151d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:34.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02f77fe700 1 -- 172.21.15.32:0/2153206887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f8146330 con 0x7f02f8061cf0 2020-12-05T17:33:34.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02f77fe700 1 --2- 172.21.15.32:0/2153206887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f8061cf0 0x7f02f8151610 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f02e8000fa0 tx=0x7f02e8007b20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:34.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02f57fa700 1 -- 172.21.15.32:0/2153206887 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02e800e4a0 con 0x7f02f8061cf0 2020-12-05T17:33:34.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02f57fa700 1 -- 172.21.15.32:0/2153206887 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e800e600 con 0x7f02f8061cf0 2020-12-05T17:33:34.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02fceba700 1 -- 172.21.15.32:0/2153206887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f8152830 con 0x7f02f8061cf0 2020-12-05T17:33:34.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02f57fa700 1 -- 172.21.15.32:0/2153206887 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02e800e960 con 0x7f02f8061cf0 2020-12-05T17:33:34.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.039+0000 7f02fceba700 1 -- 172.21.15.32:0/2153206887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f8145cd0 con 0x7f02f8061cf0 2020-12-05T17:33:34.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.043+0000 7f02fceba700 1 -- 172.21.15.32:0/2153206887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f81466d0 con 0x7f02f8061cf0 2020-12-05T17:33:34.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.043+0000 7f02f57fa700 1 -- 172.21.15.32:0/2153206887 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f02e8019030 con 0x7f02f8061cf0 2020-12-05T17:33:34.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.043+0000 7f02f57fa700 1 --2- 172.21.15.32:0/2153206887 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f02dc055330 0x7f02dc0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:34.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.043+0000 7f02f57fa700 1 -- 172.21.15.32:0/2153206887 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f02e806d400 con 0x7f02f8061cf0 2020-12-05T17:33:34.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.043+0000 7f02eeffd700 1 --2- 172.21.15.32:0/2153206887 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f02dc055330 0x7f02dc0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:34.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.043+0000 7f02eeffd700 1 --2- 172.21.15.32:0/2153206887 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f02dc055330 0x7f02dc0577f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f02e000c000 tx=0x7f02e0008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:34.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.051+0000 7f02f57fa700 1 -- 172.21.15.32:0/2153206887 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f02e8041df0 con 0x7f02f8061cf0 2020-12-05T17:33:34.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.247+0000 7f02fceba700 1 -- 172.21.15.32:0/2153206887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]} v 0) v1 -- 0x7f02f81464c0 con 0x7f02f8061cf0 2020-12-05T17:33:34.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.899+0000 7f02f57fa700 1 -- 172.21.15.32:0/2153206887 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]}]=0 v103) v1 ==== 147+0+0 (secure 0 0 0) 0x7f02f81466d0 con 0x7f02f8061cf0 2020-12-05T17:33:34.904 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.903+0000 7f02fceba700 1 -- 172.21.15.32:0/2153206887 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f02dc055330 msgr2=0x7f02dc0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:34.904 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.903+0000 7f02fceba700 1 --2- 172.21.15.32:0/2153206887 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f02dc055330 0x7f02dc0577f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f02e000c000 tx=0x7f02e0008040).stop 2020-12-05T17:33:34.904 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.903+0000 7f02fceba700 1 -- 172.21.15.32:0/2153206887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f8061cf0 msgr2=0x7f02f8151610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:34.904 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.903+0000 7f02fceba700 1 --2- 172.21.15.32:0/2153206887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f8061cf0 0x7f02f8151610 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f02e8000fa0 tx=0x7f02e8007b20).stop 2020-12-05T17:33:34.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.903+0000 7f02fceba700 1 -- 172.21.15.32:0/2153206887 shutdown_connections 2020-12-05T17:33:34.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.903+0000 7f02fceba700 1 --2- 172.21.15.32:0/2153206887 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02f8142570 0x7f02f8151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:34.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.903+0000 7f02fceba700 1 --2- 172.21.15.32:0/2153206887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f8061cf0 0x7f02f8151610 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:34.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.903+0000 7f02fceba700 1 --2- 172.21.15.32:0/2153206887 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f02dc055330 0x7f02dc0577f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:34.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.903+0000 7f02fceba700 1 --2- 172.21.15.32:0/2153206887 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f02f80af0d0 0x7f02f813f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:34.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.903+0000 7f02fceba700 1 -- 172.21.15.32:0/2153206887 >> 172.21.15.32:0/2153206887 conn(0x7f02f80b5530 msgr2=0x7f02f80b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:34.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.903+0000 7f02fceba700 1 -- 172.21.15.32:0/2153206887 shutdown_connections 2020-12-05T17:33:34.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:34.903+0000 7f02fceba700 1 -- 172.21.15.32:0/2153206887 wait complete. 2020-12-05T17:33:34.919 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-12-05T17:33:35.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.043+0000 7f207f59e700 1 Processor -- start 2020-12-05T17:33:35.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.043+0000 7f207f59e700 1 -- start start 2020-12-05T17:33:35.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.043+0000 7f207f59e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f208005f6f0 0x7f2080141700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:35.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.043+0000 7f207f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2080142300 0x7f2080142790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:35.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.043+0000 7f207f59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20801439c0 con 0x7f2080061b30 2020-12-05T17:33:35.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.043+0000 7f207f59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f208005fe20 con 0x7f2080142300 2020-12-05T17:33:35.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.043+0000 7f207f59e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2080146910 con 0x7f208005f6f0 2020-12-05T17:33:35.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.043+0000 7f207ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2080142300 0x7f2080142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:35.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.043+0000 7f207ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2080142300 0x7f2080142790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34262/0 (socket says 172.21.15.32:34262) 2020-12-05T17:33:35.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.043+0000 7f207ed9d700 1 -- 172.21.15.32:0/2162567824 learned_addr learned my addr 172.21.15.32:0/2162567824 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:35.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.043+0000 7f207e59c700 1 --2- 172.21.15.32:0/2162567824 >> v2:172.21.15.61:3300/0 conn(0x7f208005f6f0 0x7f2080141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:35.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.047+0000 7f207ed9d700 1 -- 172.21.15.32:0/2162567824 >> v2:172.21.15.61:3300/0 conn(0x7f208005f6f0 msgr2=0x7f2080141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:35.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.047+0000 7f207ed9d700 1 --2- 172.21.15.32:0/2162567824 >> v2:172.21.15.61:3300/0 conn(0x7f208005f6f0 0x7f2080141700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:35.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.047+0000 7f207ed9d700 1 -- 172.21.15.32:0/2162567824 >> v1:172.21.15.61:6789/0 conn(0x7f2080061b30 legacy=0x7f2080141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:35.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.047+0000 7f207ed9d700 1 -- 172.21.15.32:0/2162567824 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208005fad0 con 0x7f2080142300 2020-12-05T17:33:35.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.047+0000 7f207ed9d700 1 --2- 172.21.15.32:0/2162567824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2080142300 0x7f2080142790 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f20780032b0 tx=0x7f2078009270).ready entity=mon.0 client_cookie=dd4ec92849b952b4 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:35.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.047+0000 7f207d59a700 1 -- 172.21.15.32:0/2162567824 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2078009e60 con 0x7f2080142300 2020-12-05T17:33:35.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.047+0000 7f207d59a700 1 -- 172.21.15.32:0/2162567824 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207800bc00 con 0x7f2080142300 2020-12-05T17:33:35.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.047+0000 7f207d59a700 1 -- 172.21.15.32:0/2162567824 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f207800bd70 con 0x7f2080142300 2020-12-05T17:33:35.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.047+0000 7f207f59e700 1 -- 172.21.15.32:0/2162567824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2080142300 msgr2=0x7f2080142790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:35.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.047+0000 7f207f59e700 1 --2- 172.21.15.32:0/2162567824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2080142300 0x7f2080142790 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f20780032b0 tx=0x7f2078009270).stop 2020-12-05T17:33:35.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.047+0000 7f207f59e700 1 -- 172.21.15.32:0/2162567824 shutdown_connections 2020-12-05T17:33:35.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.047+0000 7f207f59e700 1 --2- 172.21.15.32:0/2162567824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2080142300 0x7f2080142790 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:35.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.047+0000 7f207f59e700 1 --2- 172.21.15.32:0/2162567824 >> v2:172.21.15.61:3300/0 conn(0x7f208005f6f0 0x7f2080141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:35.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.047+0000 7f207f59e700 1 -- 172.21.15.32:0/2162567824 >> 172.21.15.32:0/2162567824 conn(0x7f2080061010 msgr2=0x7f2080061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:35.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.047+0000 7f207f59e700 1 -- 172.21.15.32:0/2162567824 shutdown_connections 2020-12-05T17:33:35.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.047+0000 7f207f59e700 1 -- 172.21.15.32:0/2162567824 wait complete. 2020-12-05T17:33:35.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.047+0000 7f207f59e700 1 Processor -- start 2020-12-05T17:33:35.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.047+0000 7f207f59e700 1 -- start start 2020-12-05T17:33:35.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.051+0000 7f207f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f208005f6f0 0x7f20801137c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:35.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.051+0000 7f207f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2080061b30 0x7f2080140b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:35.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.051+0000 7f207f59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2080142300 0x7f2080140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:35.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.051+0000 7f207f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20801414e0 con 0x7f2080061b30 2020-12-05T17:33:35.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.051+0000 7f207f59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2080152180 con 0x7f2080142300 2020-12-05T17:33:35.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.051+0000 7f207f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2080152420 con 0x7f208005f6f0 2020-12-05T17:33:35.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.051+0000 7f207dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2080061b30 0x7f2080140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:35.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.051+0000 7f207dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2080061b30 0x7f2080140b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34266/0 (socket says 172.21.15.32:34266) 2020-12-05T17:33:35.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.051+0000 7f207dd9b700 1 -- 172.21.15.32:0/1811740483 learned_addr learned my addr 172.21.15.32:0/1811740483 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:35.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.051+0000 7f207e59c700 1 --2- 172.21.15.32:0/1811740483 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f208005f6f0 0x7f20801137c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:35.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.051+0000 7f207ed9d700 1 --2- 172.21.15.32:0/1811740483 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2080142300 0x7f2080140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:35.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.051+0000 7f207dd9b700 1 -- 172.21.15.32:0/1811740483 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f208005f6f0 msgr2=0x7f20801137c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:35.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.051+0000 7f207dd9b700 1 --2- 172.21.15.32:0/1811740483 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f208005f6f0 0x7f20801137c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:35.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.051+0000 7f207dd9b700 1 -- 172.21.15.32:0/1811740483 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2080142300 msgr2=0x7f2080140ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:35.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.051+0000 7f207dd9b700 1 --2- 172.21.15.32:0/1811740483 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2080142300 0x7f2080140ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:35.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.051+0000 7f207dd9b700 1 -- 172.21.15.32:0/1811740483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2080063870 con 0x7f2080061b30 2020-12-05T17:33:35.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.051+0000 7f207dd9b700 1 --2- 172.21.15.32:0/1811740483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2080061b30 0x7f2080140b00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f20800b37d0 tx=0x7f2068007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:35.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.051+0000 7f20777fe700 1 -- 172.21.15.32:0/1811740483 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2068007240 con 0x7f2080061b30 2020-12-05T17:33:35.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.051+0000 7f20777fe700 1 -- 172.21.15.32:0/1811740483 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20680073a0 con 0x7f2080061b30 2020-12-05T17:33:35.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.051+0000 7f207f59e700 1 -- 172.21.15.32:0/1811740483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2080065400 con 0x7f2080061b30 2020-12-05T17:33:35.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.051+0000 7f20777fe700 1 -- 172.21.15.32:0/1811740483 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20680066d0 con 0x7f2080061b30 2020-12-05T17:33:35.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.051+0000 7f207f59e700 1 -- 172.21.15.32:0/1811740483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2080063480 con 0x7f2080061b30 2020-12-05T17:33:35.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.051+0000 7f207f59e700 1 -- 172.21.15.32:0/1811740483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2080063020 con 0x7f2080061b30 2020-12-05T17:33:35.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.055+0000 7f20777fe700 1 -- 172.21.15.32:0/1811740483 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2068010070 con 0x7f2080061b30 2020-12-05T17:33:35.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.055+0000 7f20777fe700 1 --2- 172.21.15.32:0/1811740483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2064055440 0x7f2064057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:35.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.055+0000 7f20777fe700 1 -- 172.21.15.32:0/1811740483 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f206806d6f0 con 0x7f2080061b30 2020-12-05T17:33:35.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.055+0000 7f207e59c700 1 --2- 172.21.15.32:0/1811740483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2064055440 0x7f2064057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:35.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.055+0000 7f207e59c700 1 --2- 172.21.15.32:0/1811740483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2064055440 0x7f2064057900 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f2070006000 tx=0x7f2070006d60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:35.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.059+0000 7f20777fe700 1 -- 172.21.15.32:0/1811740483 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2080063020 con 0x7f2080061b30 2020-12-05T17:33:35.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.203+0000 7f207f59e700 1 -- 172.21.15.32:0/1811740483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f2080063020 con 0x7f2080061b30 2020-12-05T17:33:35.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.207+0000 7f20777fe700 1 -- 172.21.15.32:0/1811740483 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2080063020 con 0x7f2080061b30 2020-12-05T17:33:35.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.211+0000 7f207f59e700 1 -- 172.21.15.32:0/1811740483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2064055440 msgr2=0x7f2064057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:35.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.211+0000 7f207f59e700 1 --2- 172.21.15.32:0/1811740483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2064055440 0x7f2064057900 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f2070006000 tx=0x7f2070006d60).stop 2020-12-05T17:33:35.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.211+0000 7f207f59e700 1 -- 172.21.15.32:0/1811740483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2080061b30 msgr2=0x7f2080140b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:35.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.211+0000 7f207f59e700 1 --2- 172.21.15.32:0/1811740483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2080061b30 0x7f2080140b00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f20800b37d0 tx=0x7f2068007be0).stop 2020-12-05T17:33:35.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.211+0000 7f207f59e700 1 -- 172.21.15.32:0/1811740483 shutdown_connections 2020-12-05T17:33:35.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.211+0000 7f207f59e700 1 --2- 172.21.15.32:0/1811740483 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2080142300 0x7f2080140ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:35.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.211+0000 7f207f59e700 1 --2- 172.21.15.32:0/1811740483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2064055440 0x7f2064057900 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:35.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.211+0000 7f207f59e700 1 --2- 172.21.15.32:0/1811740483 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f208005f6f0 0x7f20801137c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:35.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.211+0000 7f207f59e700 1 --2- 172.21.15.32:0/1811740483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2080061b30 0x7f2080140b00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:35.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.211+0000 7f207f59e700 1 -- 172.21.15.32:0/1811740483 >> 172.21.15.32:0/1811740483 conn(0x7f2080061010 msgr2=0x7f20800ad660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:35.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.211+0000 7f207f59e700 1 -- 172.21.15.32:0/1811740483 shutdown_connections 2020-12-05T17:33:35.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.211+0000 7f207f59e700 1 -- 172.21.15.32:0/1811740483 wait complete. 2020-12-05T17:33:35.223 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:33:35.224 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:33:35.241 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:33:35.258 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:33:35.275 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:33:35.292 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:33:35.310 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:33:35.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:33:35.311 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:33:35.312 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:33:35.325 INFO:teuthology.orchestra.run.smithi036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-12-05T17:33:35.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:33:35.326 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-12-05T17:33:35.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:33:35.327 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:33:35.328 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:33:35.334 INFO:teuthology.orchestra.run.smithi002.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-12-05T17:33:35.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:33:35.336 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-12-05T17:33:35.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:33:35.336 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:33:35.337 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:33:35.343 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:33:35.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:33:35.345 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2020-12-05T17:33:35.345 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-12-05T17:33:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.463+0000 7f3654985700 1 Processor -- start 2020-12-05T17:33:35.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.463+0000 7f3654985700 1 -- start start 2020-12-05T17:33:35.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f3654985700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f36500afd20 0x7f36501421e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:35.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f3654985700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3650142a30 0x7f3650142ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:35.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f3654985700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36500af300 con 0x7f36500b5650 2020-12-05T17:33:35.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f3654985700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36500b5a30 con 0x7f36500afd20 2020-12-05T17:33:35.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f3654985700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36500aea90 con 0x7f3650142a30 2020-12-05T17:33:35.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f3647fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f36500afd20 0x7f36501421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:35.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f3647fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f36500afd20 0x7f36501421e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34274/0 (socket says 172.21.15.32:34274) 2020-12-05T17:33:35.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f3647fff700 1 -- 172.21.15.32:0/2511219141 learned_addr learned my addr 172.21.15.32:0/2511219141 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:35.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f364f59e700 1 --2- 172.21.15.32:0/2511219141 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3650142a30 0x7f3650142ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:35.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f3647fff700 1 -- 172.21.15.32:0/2511219141 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3650142a30 msgr2=0x7f3650142ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:35.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f3647fff700 1 --2- 172.21.15.32:0/2511219141 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3650142a30 0x7f3650142ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:35.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f3647fff700 1 -- 172.21.15.32:0/2511219141 >> v1:172.21.15.61:6789/0 conn(0x7f36500b5650 legacy=0x7f3650141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:35.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f364e59c700 1 -- 172.21.15.32:0/2511219141 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4157105866 0 0) 0x7f36500af300 con 0x7f36500b5650 2020-12-05T17:33:35.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f3647fff700 1 -- 172.21.15.32:0/2511219141 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36500aed30 con 0x7f36500afd20 2020-12-05T17:33:35.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f3647fff700 1 --2- 172.21.15.32:0/2511219141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36500afd20 0x7f36501421e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f3638002f30 tx=0x7f3638006e80).ready entity=mon.0 client_cookie=490942c459d8ea2b server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:35.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f364e59c700 1 -- 172.21.15.32:0/2511219141 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36380097f0 con 0x7f36500afd20 2020-12-05T17:33:35.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f364e59c700 1 -- 172.21.15.32:0/2511219141 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3638009950 con 0x7f36500afd20 2020-12-05T17:33:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f364e59c700 1 -- 172.21.15.32:0/2511219141 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3638009cb0 con 0x7f36500afd20 2020-12-05T17:33:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f3654985700 1 -- 172.21.15.32:0/2511219141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36500afd20 msgr2=0x7f36501421e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f3654985700 1 --2- 172.21.15.32:0/2511219141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36500afd20 0x7f36501421e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f3638002f30 tx=0x7f3638006e80).stop 2020-12-05T17:33:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f3654985700 1 -- 172.21.15.32:0/2511219141 shutdown_connections 2020-12-05T17:33:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f3654985700 1 --2- 172.21.15.32:0/2511219141 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3650142a30 0x7f3650142ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f3654985700 1 --2- 172.21.15.32:0/2511219141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36500afd20 0x7f36501421e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f3654985700 1 -- 172.21.15.32:0/2511219141 >> 172.21.15.32:0/2511219141 conn(0x7f365005bbb0 msgr2=0x7f3650062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f3654985700 1 -- 172.21.15.32:0/2511219141 shutdown_connections 2020-12-05T17:33:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.467+0000 7f3654985700 1 -- 172.21.15.32:0/2511219141 wait complete. 2020-12-05T17:33:35.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.471+0000 7f3654985700 1 Processor -- start 2020-12-05T17:33:35.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.471+0000 7f3654985700 1 -- start start 2020-12-05T17:33:35.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f3654985700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36500afd20 0x7f3650141420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:35.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f3654985700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f36500b5650 0x7f365013ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:35.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f3654985700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3650142a30 0x7f365013f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:35.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f3654985700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365010fee0 con 0x7f36500afd20 2020-12-05T17:33:35.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f3654985700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36501103b0 con 0x7f3650142a30 2020-12-05T17:33:35.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f3654985700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3650110110 con 0x7f36500b5650 2020-12-05T17:33:35.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f364ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36500afd20 0x7f3650141420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:35.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f364ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36500afd20 0x7f3650141420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34278/0 (socket says 172.21.15.32:34278) 2020-12-05T17:33:35.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f364ed9d700 1 -- 172.21.15.32:0/4017563113 learned_addr learned my addr 172.21.15.32:0/4017563113 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:35.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f3647fff700 1 --2- 172.21.15.32:0/4017563113 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f36500b5650 0x7f365013ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:35.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f364f59e700 1 --2- 172.21.15.32:0/4017563113 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3650142a30 0x7f365013f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:35.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f364ed9d700 1 -- 172.21.15.32:0/4017563113 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f36500b5650 msgr2=0x7f365013ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:35.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f364ed9d700 1 --2- 172.21.15.32:0/4017563113 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f36500b5650 0x7f365013ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:35.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f364ed9d700 1 -- 172.21.15.32:0/4017563113 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3650142a30 msgr2=0x7f365013f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:35.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f364ed9d700 1 --2- 172.21.15.32:0/4017563113 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3650142a30 0x7f365013f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f364ed9d700 1 -- 172.21.15.32:0/4017563113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3650141d00 con 0x7f36500afd20 2020-12-05T17:33:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f364ed9d700 1 --2- 172.21.15.32:0/4017563113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36500afd20 0x7f3650141420 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f3640000b80 tx=0x7f3640007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f364cd99700 1 -- 172.21.15.32:0/4017563113 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f364000e4a0 con 0x7f36500afd20 2020-12-05T17:33:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f364cd99700 1 -- 172.21.15.32:0/4017563113 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364000e600 con 0x7f36500afd20 2020-12-05T17:33:35.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f3654985700 1 -- 172.21.15.32:0/4017563113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f365013f930 con 0x7f36500afd20 2020-12-05T17:33:35.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f364cd99700 1 -- 172.21.15.32:0/4017563113 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f364000e960 con 0x7f36500afd20 2020-12-05T17:33:35.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f3654985700 1 -- 172.21.15.32:0/4017563113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3650149720 con 0x7f36500afd20 2020-12-05T17:33:35.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f3654985700 1 -- 172.21.15.32:0/4017563113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3650057050 con 0x7f36500afd20 2020-12-05T17:33:35.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.475+0000 7f364cd99700 1 -- 172.21.15.32:0/4017563113 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3640019030 con 0x7f36500afd20 2020-12-05T17:33:35.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.479+0000 7f364cd99700 1 --2- 172.21.15.32:0/4017563113 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f36340553b0 0x7f3634057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:35.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.479+0000 7f364cd99700 1 -- 172.21.15.32:0/4017563113 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f364006d470 con 0x7f36500afd20 2020-12-05T17:33:35.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.479+0000 7f3647fff700 1 --2- 172.21.15.32:0/4017563113 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f36340553b0 0x7f3634057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:35.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.479+0000 7f3647fff700 1 --2- 172.21.15.32:0/4017563113 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f36340553b0 0x7f3634057870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f363801c000 tx=0x7f3638015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:35.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.483+0000 7f364cd99700 1 -- 172.21.15.32:0/4017563113 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3640043610 con 0x7f36500afd20 2020-12-05T17:33:35.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.623+0000 7f3654985700 1 -- 172.21.15.32:0/4017563113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f365005c300 con 0x7f36500afd20 2020-12-05T17:33:35.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.623+0000 7f364cd99700 1 -- 172.21.15.32:0/4017563113 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v34) v1 ==== 81+0+34037 (secure 0 0 0) 0x7f3640046c30 con 0x7f36500afd20 2020-12-05T17:33:35.627 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:33:35.627 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:33:35.627 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 34, 2020-12-05T17:33:35.627 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", 2020-12-05T17:33:35.628 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2020-12-05T17:31:20.975238+0000", 2020-12-05T17:33:35.628 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2020-12-05T17:32:26.798545+0000", 2020-12-05T17:33:35.628 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:33:35.628 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:33:35.628 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-12-05T17:33:35.629 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2020-12-05T17:33:35.629 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2020-12-05T17:33:35.629 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2020-12-05T17:33:35.629 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2020-12-05T17:33:35.629 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2020-12-05T17:33:35.630 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2020-12-05T17:33:35.630 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:33:35.630 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2020-12-05T17:33:35.630 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2020-12-05T17:33:35.630 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-12-05T17:33:35.631 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2020-12-05T17:33:35.631 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2020-12-05T17:33:35.631 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 6, 2020-12-05T17:33:35.631 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2020-12-05T17:33:35.631 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2020-12-05T17:33:35.632 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2020-12-05T17:33:35.632 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2020-12-05T17:33:35.632 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2020-12-05T17:33:35.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.632 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2020-12-05T17:33:35.633 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2020-12-05T17:33:35.633 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:24.858732+0000", 2020-12-05T17:33:35.633 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:33:35.633 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:33:35.634 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:33:35.634 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:33:35.634 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:33:35.634 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:33:35.634 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:33:35.635 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:33:35.635 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2020-12-05T17:33:35.635 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2020-12-05T17:33:35.635 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2020-12-05T17:33:35.635 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2020-12-05T17:33:35.636 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2020-12-05T17:33:35.636 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:33:35.636 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:33:35.636 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:33:35.636 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:33:35.637 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:33:35.637 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:33:35.637 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:33:35.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.637 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "16", 2020-12-05T17:33:35.638 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:33:35.638 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:33:35.638 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:33:35.638 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:33:35.638 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:33:35.639 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:33:35.639 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:33:35.639 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:33:35.639 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:33:35.640 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:33:35.640 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:33:35.640 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:33:35.640 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:33:35.640 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:33:35.641 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:33:35.641 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:33:35.641 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:33:35.641 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:33:35.641 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:33:35.642 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:33:35.642 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:33:35.642 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:33:35.642 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:33:35.642 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:33:35.643 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:33:35.643 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:33:35.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.643 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:33:35.643 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:33:35.644 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:33:35.644 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:33:35.644 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:33:35.644 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:33:35.644 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:33:35.645 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:33:35.645 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:33:35.645 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:33:35.645 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:33:35.645 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:33:35.646 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2020-12-05T17:33:35.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.646 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:33:35.646 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2020-12-05T17:33:35.646 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.647 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2020-12-05T17:33:35.647 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2020-12-05T17:33:35.647 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:33.305595+0000", 2020-12-05T17:33:35.648 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:33:35.648 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:33:35.648 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:33:35.648 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:33:35.648 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:33:35.649 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:33:35.649 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:33:35.649 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:33:35.649 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:33:35.649 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:33:35.650 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:33:35.650 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:33:35.650 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:33:35.650 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:33:35.650 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:33:35.651 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:33:35.651 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:33:35.651 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:33:35.651 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:33:35.651 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:33:35.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.652 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "19", 2020-12-05T17:33:35.652 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:33:35.652 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:33:35.652 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:33:35.653 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:33:35.653 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:33:35.653 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:33:35.653 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:33:35.653 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:33:35.654 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:33:35.654 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:33:35.654 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:33:35.654 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:33:35.655 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:33:35.655 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:33:35.655 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:33:35.655 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:33:35.655 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:33:35.656 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:33:35.656 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:33:35.656 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:33:35.656 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:33:35.656 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:33:35.657 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:33:35.657 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:33:35.657 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:33:35.657 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:33:35.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.658 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:33:35.658 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:33:35.658 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:33:35.658 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:33:35.658 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:33:35.659 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:33:35.659 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:33:35.659 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:33:35.659 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:33:35.659 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:33:35.660 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:33:35.660 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:33:35.660 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:33:35.660 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2020-12-05T17:33:35.660 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.661 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 5, 2020-12-05T17:33:35.661 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2020-12-05T17:33:35.661 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:32:23.957088+0000", 2020-12-05T17:33:35.662 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:33:35.662 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:33:35.662 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:33:35.662 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:33:35.662 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:33:35.663 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:33:35.663 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:33:35.663 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:33:35.663 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:33:35.663 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:33:35.664 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:33:35.664 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:33:35.664 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:33:35.664 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:33:35.664 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:33:35.665 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:33:35.665 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:33:35.665 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:33:35.665 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:33:35.665 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:33:35.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.666 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "33", 2020-12-05T17:33:35.666 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:33:35.666 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:33:35.666 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:33:35.667 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:33:35.667 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:33:35.667 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:33:35.667 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:33:35.667 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:33:35.668 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:33:35.668 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:33:35.668 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:33:35.668 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:33:35.668 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:33:35.669 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:33:35.669 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:33:35.669 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:33:35.669 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:33:35.669 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:33:35.670 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:33:35.670 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:33:35.670 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:33:35.670 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:33:35.670 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:33:35.671 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:33:35.671 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:33:35.671 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:33:35.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.671 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:33:35.672 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:33:35.672 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:33:35.672 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:33:35.672 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:33:35.672 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:33:35.673 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:33:35.673 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:33:35.673 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:33:35.673 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:33:35.673 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:33:35.674 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:33:35.674 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2020-12-05T17:33:35.674 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2020-12-05T17:33:35.674 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2020-12-05T17:33:35.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.675 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:33:35.675 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:33:35.675 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2020-12-05T17:33:35.675 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.675 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.676 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 6, 2020-12-05T17:33:35.676 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2020-12-05T17:33:35.676 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:32:24.800777+0000", 2020-12-05T17:33:35.677 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:33:35.677 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:33:35.677 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:33:35.677 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:33:35.677 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:33:35.678 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:33:35.678 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:33:35.678 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:33:35.678 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:33:35.678 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:33:35.679 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:33:35.679 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:33:35.679 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:33:35.679 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:33:35.679 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:33:35.680 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:33:35.680 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:33:35.680 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:33:35.680 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:33:35.680 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:33:35.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.681 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "34", 2020-12-05T17:33:35.681 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:33:35.681 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:33:35.681 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:33:35.682 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:33:35.682 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:33:35.682 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:33:35.682 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:33:35.682 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:33:35.683 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:33:35.683 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:33:35.683 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:33:35.683 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:33:35.683 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:33:35.684 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:33:35.684 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:33:35.684 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:33:35.684 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:33:35.684 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:33:35.685 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:33:35.685 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:33:35.685 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:33:35.685 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:33:35.685 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:33:35.686 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:33:35.686 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:33:35.686 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:33:35.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.686 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:33:35.687 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:33:35.687 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:33:35.687 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:33:35.687 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:33:35.687 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:33:35.688 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:33:35.688 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:33:35.688 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:33:35.688 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:33:35.688 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:33:35.689 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:33:35.689 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:33:35.689 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:33:35.689 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2020-12-05T17:33:35.689 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.690 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.690 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.690 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:33:35.690 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2020-12-05T17:33:35.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.691 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:33:35.691 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "b941127b-0a49-4e06-8d11-17794ff8b013", 2020-12-05T17:33:35.691 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:33:35.691 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:33:35.691 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:33:35.692 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:33:35.692 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:33:35.692 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:33:35.692 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:33:35.692 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 32, 2020-12-05T17:33:35.693 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:33:35.693 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:33:35.693 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:33:35.693 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.693 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.694 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2020-12-05T17:33:35.694 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:33:35.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.695 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.695 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2020-12-05T17:33:35.695 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:33:35.695 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.695 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.696 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:33:35.696 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.696 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.697 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2020-12-05T17:33:35.697 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:33:35.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.697 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.698 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2020-12-05T17:33:35.698 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:33:35.698 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.698 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.699 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:33:35.699 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.699 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.699 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2020-12-05T17:33:35.700 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:33:35.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.700 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.700 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2020-12-05T17:33:35.701 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:33:35.701 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.701 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.701 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:33:35.702 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.702 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.702 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2020-12-05T17:33:35.702 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:33:35.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.703 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.703 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2020-12-05T17:33:35.703 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:33:35.704 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.704 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.704 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6803/11526", 2020-12-05T17:33:35.704 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6805/11526", 2020-12-05T17:33:35.705 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6809/11526", 2020-12-05T17:33:35.705 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6807/11526", 2020-12-05T17:33:35.705 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:33:35.705 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:33:35.705 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:33:35.706 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.706 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:33:35.706 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "01bd7ff5-a86b-4e11-ba1e-484c309c0fba", 2020-12-05T17:33:35.707 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:33:35.707 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:33:35.707 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:33:35.707 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:33:35.707 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:33:35.708 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:33:35.708 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:33:35.708 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 32, 2020-12-05T17:33:35.708 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:33:35.708 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:33:35.709 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:33:35.709 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.709 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.709 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2020-12-05T17:33:35.710 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:33:35.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.710 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.710 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2020-12-05T17:33:35.711 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:33:35.711 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.711 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.711 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:33:35.712 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.712 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.712 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2020-12-05T17:33:35.712 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:33:35.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.713 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.713 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2020-12-05T17:33:35.713 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:33:35.714 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.714 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.714 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:33:35.714 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.715 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.715 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2020-12-05T17:33:35.715 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:33:35.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.716 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.716 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2020-12-05T17:33:35.716 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:33:35.716 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.717 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.717 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:33:35.717 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.718 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.718 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2020-12-05T17:33:35.718 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:33:35.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.719 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.719 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2020-12-05T17:33:35.719 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:33:35.719 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.719 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.720 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6819/11538", 2020-12-05T17:33:35.720 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6821/11538", 2020-12-05T17:33:35.720 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6825/11538", 2020-12-05T17:33:35.720 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6823/11538", 2020-12-05T17:33:35.721 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:33:35.721 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:33:35.721 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:33:35.721 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.722 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:33:35.722 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "94736ef5-64c2-4a97-b963-bd38cddd9c52", 2020-12-05T17:33:35.722 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:33:35.722 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:33:35.723 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:33:35.723 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:33:35.723 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:33:35.723 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:33:35.723 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:33:35.724 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 31, 2020-12-05T17:33:35.724 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:33:35.724 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:33:35.724 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:33:35.724 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.725 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.725 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2020-12-05T17:33:35.725 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:33:35.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.726 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.726 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2020-12-05T17:33:35.726 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:33:35.726 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.727 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.727 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:33:35.727 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.728 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.728 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2020-12-05T17:33:35.728 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:33:35.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.729 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.729 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2020-12-05T17:33:35.729 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:33:35.729 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.729 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.730 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:33:35.730 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.730 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.731 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2020-12-05T17:33:35.731 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:33:35.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.731 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.732 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2020-12-05T17:33:35.732 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:33:35.732 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.732 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.733 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:33:35.733 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.733 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.733 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2020-12-05T17:33:35.734 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:33:35.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.734 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.734 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2020-12-05T17:33:35.735 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:33:35.735 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.735 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.735 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6811/11521", 2020-12-05T17:33:35.736 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6813/11521", 2020-12-05T17:33:35.736 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6817/11521", 2020-12-05T17:33:35.736 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6815/11521", 2020-12-05T17:33:35.736 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:33:35.736 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:33:35.737 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:33:35.737 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.737 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:33:35.738 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "69a1f3ff-fd25-43df-b60c-e32a89c35234", 2020-12-05T17:33:35.738 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:33:35.738 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:33:35.738 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:33:35.738 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:33:35.739 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:33:35.739 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:33:35.739 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:33:35.739 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 32, 2020-12-05T17:33:35.739 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:33:35.740 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:33:35.740 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:33:35.740 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.740 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.741 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2020-12-05T17:33:35.741 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:33:35.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.741 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2020-12-05T17:33:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:33:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.743 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:33:35.743 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.743 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.743 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2020-12-05T17:33:35.744 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:33:35.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.744 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.744 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2020-12-05T17:33:35.745 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:33:35.745 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.745 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.745 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:33:35.746 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.746 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.746 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6832", 2020-12-05T17:33:35.746 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:33:35.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.747 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.747 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6833", 2020-12-05T17:33:35.747 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:33:35.748 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.748 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.748 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:33:35.748 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.749 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.749 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2020-12-05T17:33:35.749 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:33:35.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.750 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2020-12-05T17:33:35.750 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:33:35.750 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.751 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.751 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6827/11535", 2020-12-05T17:33:35.751 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6829/11535", 2020-12-05T17:33:35.751 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6833/11535", 2020-12-05T17:33:35.752 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6831/11535", 2020-12-05T17:33:35.752 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:33:35.752 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:33:35.752 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:33:35.752 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.753 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:33:35.753 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c301fe4e-5eab-402c-ae6f-db3e6a7181af", 2020-12-05T17:33:35.753 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:33:35.754 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:33:35.754 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:33:35.754 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:33:35.754 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:33:35.754 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:33:35.755 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:33:35.755 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 32, 2020-12-05T17:33:35.755 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:33:35.755 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:33:35.755 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:33:35.756 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.756 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.756 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6808", 2020-12-05T17:33:35.756 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:33:35.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.757 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.757 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6809", 2020-12-05T17:33:35.757 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:33:35.758 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.758 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.758 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:33:35.758 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.759 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.759 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6810", 2020-12-05T17:33:35.759 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:33:35.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.760 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.760 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6811", 2020-12-05T17:33:35.760 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:33:35.760 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.761 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.761 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:33:35.761 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.762 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.762 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6817", 2020-12-05T17:33:35.762 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:33:35.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.763 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.763 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6819", 2020-12-05T17:33:35.763 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:33:35.763 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.763 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:33:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.765 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6812", 2020-12-05T17:33:35.765 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:33:35.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.765 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.766 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6814", 2020-12-05T17:33:35.766 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:33:35.766 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.766 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.767 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6809/11541", 2020-12-05T17:33:35.767 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6811/11541", 2020-12-05T17:33:35.767 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6819/11541", 2020-12-05T17:33:35.767 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6814/11541", 2020-12-05T17:33:35.767 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:33:35.768 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:33:35.768 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:33:35.768 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.769 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:33:35.769 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "9476eff4-345b-46c5-a473-1d9a89a6d327", 2020-12-05T17:33:35.769 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:33:35.769 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:33:35.769 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:33:35.770 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:33:35.770 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:33:35.770 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:33:35.770 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:33:35.770 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 32, 2020-12-05T17:33:35.771 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:33:35.771 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:33:35.771 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:33:35.771 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.772 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.772 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6800", 2020-12-05T17:33:35.772 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:33:35.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.773 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.773 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6801", 2020-12-05T17:33:35.773 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:33:35.773 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.773 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.774 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:33:35.774 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.774 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.775 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6802", 2020-12-05T17:33:35.775 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:33:35.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.775 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.776 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6803", 2020-12-05T17:33:35.776 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:33:35.776 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.776 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.777 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:33:35.777 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.777 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.777 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6806", 2020-12-05T17:33:35.778 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:33:35.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.778 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.778 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6807", 2020-12-05T17:33:35.779 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:33:35.779 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.779 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.779 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:33:35.780 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.780 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.780 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6804", 2020-12-05T17:33:35.780 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:33:35.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.781 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.781 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6805", 2020-12-05T17:33:35.781 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:33:35.782 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.782 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.782 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6801/11539", 2020-12-05T17:33:35.782 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6803/11539", 2020-12-05T17:33:35.783 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6807/11539", 2020-12-05T17:33:35.783 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6805/11539", 2020-12-05T17:33:35.783 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:33:35.783 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:33:35.783 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:33:35.784 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.784 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:33:35.784 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8dfd8567-9b1c-4d13-9c96-071183e469c2", 2020-12-05T17:33:35.785 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:33:35.785 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:33:35.785 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:33:35.785 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:33:35.785 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:33:35.786 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:33:35.786 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:33:35.786 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 31, 2020-12-05T17:33:35.786 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:33:35.786 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:33:35.787 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:33:35.787 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.787 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6820", 2020-12-05T17:33:35.788 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:33:35.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.788 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.788 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6822", 2020-12-05T17:33:35.789 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:33:35.789 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.789 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.789 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:33:35.789 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.790 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.790 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6823", 2020-12-05T17:33:35.790 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:33:35.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.791 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.791 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6825", 2020-12-05T17:33:35.791 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:33:35.792 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.792 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.792 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:33:35.792 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.793 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.793 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6829", 2020-12-05T17:33:35.793 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:33:35.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.794 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.794 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6830", 2020-12-05T17:33:35.794 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:33:35.794 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.795 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.795 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:33:35.795 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.796 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.796 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6826", 2020-12-05T17:33:35.796 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:33:35.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.797 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.797 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6827", 2020-12-05T17:33:35.797 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:33:35.797 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.797 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.798 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6822/11542", 2020-12-05T17:33:35.798 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6825/11542", 2020-12-05T17:33:35.798 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6830/11542", 2020-12-05T17:33:35.798 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6827/11542", 2020-12-05T17:33:35.799 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:33:35.799 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:33:35.799 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:33:35.799 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.800 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:33:35.800 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "24d466b3-01dc-4ba2-8b81-85250d163bd2", 2020-12-05T17:33:35.800 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:33:35.800 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:33:35.801 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:33:35.801 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:33:35.801 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:33:35.801 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:33:35.801 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:33:35.802 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 32, 2020-12-05T17:33:35.802 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:33:35.802 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:33:35.802 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:33:35.802 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.803 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.803 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6813", 2020-12-05T17:33:35.803 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:33:35.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.804 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.804 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6815", 2020-12-05T17:33:35.804 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:33:35.804 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.805 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.805 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:33:35.805 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.805 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.806 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6816", 2020-12-05T17:33:35.806 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:33:35.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.807 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.807 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6818", 2020-12-05T17:33:35.807 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:33:35.807 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.807 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.808 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:33:35.808 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.808 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.809 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6828", 2020-12-05T17:33:35.809 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:33:35.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.809 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.810 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6831", 2020-12-05T17:33:35.810 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:33:35.810 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.810 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.811 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:33:35.811 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:33:35.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.811 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:33:35.811 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6821", 2020-12-05T17:33:35.812 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:33:35.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.812 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:33:35.812 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6824", 2020-12-05T17:33:35.812 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:33:35.813 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.813 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.813 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6815/11540", 2020-12-05T17:33:35.813 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6818/11540", 2020-12-05T17:33:35.814 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6831/11540", 2020-12-05T17:33:35.814 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6824/11540", 2020-12-05T17:33:35.814 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:33:35.814 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:33:35.814 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:33:35.815 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:35.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.815 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:33:35.815 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2020-12-05T17:33:35.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.816 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:33:35.816 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:33:35.816 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:33:35.816 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:33:35.816 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:33:35.817 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:33:35.817 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.398868+0000", 2020-12-05T17:33:35.817 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:33:35.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.818 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:33:35.818 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:33:35.818 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:33:35.818 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:33:35.819 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:33:35.819 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:33:35.819 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.675748+0000", 2020-12-05T17:33:35.819 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:33:35.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.820 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:33:35.820 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:33:35.820 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:33:35.820 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:33:35.821 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:33:35.821 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:33:35.821 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.620888+0000", 2020-12-05T17:33:35.821 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:33:35.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.822 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:33:35.822 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:33:35.822 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:33:35.822 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:33:35.823 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:33:35.823 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:33:35.823 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.681600+0000", 2020-12-05T17:33:35.823 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:33:35.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.824 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:33:35.824 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:33:35.824 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:33:35.824 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:33:35.824 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:33:35.825 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:33:35.825 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.658769+0000", 2020-12-05T17:33:35.825 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:33:35.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.826 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:33:35.826 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:33:35.826 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:33:35.826 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:33:35.827 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:33:35.827 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:33:35.827 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.630708+0000", 2020-12-05T17:33:35.827 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:33:35.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.828 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:33:35.828 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:33:35.828 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:33:35.828 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:33:35.828 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:33:35.829 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:33:35.829 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.648352+0000", 2020-12-05T17:33:35.829 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:33:35.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:35.830 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:33:35.830 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:33:35.830 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:33:35.830 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:33:35.831 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:33:35.831 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:33:35.831 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.629399+0000", 2020-12-05T17:33:35.831 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:33:35.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.832 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:33:35.832 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2020-12-05T17:33:35.832 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2020-12-05T17:33:35.832 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2020-12-05T17:33:35.832 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2020-12-05T17:33:35.833 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": {}, 2020-12-05T17:33:35.833 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2020-12-05T17:33:35.833 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2020-12-05T17:33:35.833 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2020-12-05T17:33:35.833 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:33:35.833 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2020-12-05T17:33:35.834 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2020-12-05T17:33:35.834 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:33:35.834 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2020-12-05T17:33:35.834 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:33:35.835 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:33:35.835 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2020-12-05T17:33:35.835 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2020-12-05T17:33:35.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.835 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2020-12-05T17:33:35.835 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:33:35.836 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:33:35.836 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2020-12-05T17:33:35.836 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:33:35.836 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:33:35.836 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2020-12-05T17:33:35.837 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:35.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:35.837 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2020-12-05T17:33:35.837 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2020-12-05T17:33:35.837 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2020-12-05T17:33:35.838 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2020-12-05T17:33:35.838 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2020-12-05T17:33:35.838 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:33:35.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.627+0000 7f3654985700 1 -- 172.21.15.32:0/4017563113 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f36340553b0 msgr2=0x7f3634057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:35.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.627+0000 7f3654985700 1 --2- 172.21.15.32:0/4017563113 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f36340553b0 0x7f3634057870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f363801c000 tx=0x7f3638015000).stop 2020-12-05T17:33:35.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.627+0000 7f3654985700 1 -- 172.21.15.32:0/4017563113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36500afd20 msgr2=0x7f3650141420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:35.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.627+0000 7f3654985700 1 --2- 172.21.15.32:0/4017563113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36500afd20 0x7f3650141420 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f3640000b80 tx=0x7f3640007ac0).stop 2020-12-05T17:33:35.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.627+0000 7f3654985700 1 -- 172.21.15.32:0/4017563113 shutdown_connections 2020-12-05T17:33:35.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.627+0000 7f3654985700 1 --2- 172.21.15.32:0/4017563113 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3650142a30 0x7f365013f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:35.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.627+0000 7f3654985700 1 --2- 172.21.15.32:0/4017563113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36500afd20 0x7f3650141420 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:35.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.627+0000 7f3654985700 1 --2- 172.21.15.32:0/4017563113 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f36340553b0 0x7f3634057870 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:35.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.627+0000 7f3654985700 1 --2- 172.21.15.32:0/4017563113 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f36500b5650 0x7f365013ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:35.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.627+0000 7f3654985700 1 -- 172.21.15.32:0/4017563113 >> 172.21.15.32:0/4017563113 conn(0x7f365005bbb0 msgr2=0x7f36500b45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:35.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.627+0000 7f3654985700 1 -- 172.21.15.32:0/4017563113 shutdown_connections 2020-12-05T17:33:35.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.627+0000 7f3654985700 1 -- 172.21.15.32:0/4017563113 wait complete. 2020-12-05T17:33:35.843 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:33:35.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.963+0000 7fcf1326f700 1 Processor -- start 2020-12-05T17:33:35.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.963+0000 7fcf1326f700 1 -- start start 2020-12-05T17:33:35.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.963+0000 7fcf1326f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf0c0b2050 0x7fcf0c0acb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:35.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.963+0000 7fcf1326f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcf0c145f50 0x7fcf0c0ad580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:35.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.963+0000 7fcf1326f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf0c0adc20 con 0x7fcf0c0ae6b0 2020-12-05T17:33:35.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.963+0000 7fcf1326f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf0c0b2870 con 0x7fcf0c145f50 2020-12-05T17:33:35.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.963+0000 7fcf1326f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c0adec0 con 0x7fcf0c0b2050 2020-12-05T17:33:35.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.963+0000 7fcf1080a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcf0c145f50 0x7fcf0c0ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:35.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.963+0000 7fcf1180c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcf0c0ae6b0 0x7fcf0c0aeb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36938/0 (socket says 172.21.15.32:36938) 2020-12-05T17:33:35.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.963+0000 7fcf1180c700 1 -- 172.21.15.32:0/3824909712 learned_addr learned my addr 172.21.15.32:0/3824909712 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:33:35.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.963+0000 7fcf1100b700 1 --2- 172.21.15.32:0/3824909712 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf0c0b2050 0x7fcf0c0acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:35.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.963+0000 7fcf1080a700 1 -- 172.21.15.32:0/3824909712 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf0c0b2050 msgr2=0x7fcf0c0acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:35.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.963+0000 7fcf1080a700 1 --2- 172.21.15.32:0/3824909712 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf0c0b2050 0x7fcf0c0acb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:35.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.963+0000 7fcf1080a700 1 -- 172.21.15.32:0/3824909712 >> v1:172.21.15.32:6789/0 conn(0x7fcf0c0ae6b0 legacy=0x7fcf0c0aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:35.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.963+0000 7fcefbfff700 1 -- 172.21.15.32:0/3824909712 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 50334934 0 0) 0x7fcf0c0adc20 con 0x7fcf0c0ae6b0 2020-12-05T17:33:35.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.963+0000 7fcf1080a700 1 -- 172.21.15.32:0/3824909712 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0c0b2430 con 0x7fcf0c145f50 2020-12-05T17:33:35.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.963+0000 7fcf1080a700 1 --2- 172.21.15.32:0/3824909712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf0c145f50 0x7fcf0c0ad580 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7fcf040032b0 tx=0x7fcf040091a0).ready entity=mon.0 client_cookie=93b139e9e1dfdeae server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:35.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.963+0000 7fcefbfff700 1 -- 172.21.15.32:0/3824909712 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf0400b9a0 con 0x7fcf0c145f50 2020-12-05T17:33:35.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.963+0000 7fcefbfff700 1 -- 172.21.15.32:0/3824909712 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0400bb00 con 0x7fcf0c145f50 2020-12-05T17:33:35.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.963+0000 7fcefbfff700 1 -- 172.21.15.32:0/3824909712 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf0400bc70 con 0x7fcf0c145f50 2020-12-05T17:33:35.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.963+0000 7fcf1326f700 1 -- 172.21.15.32:0/3824909712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf0c145f50 msgr2=0x7fcf0c0ad580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:35.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.963+0000 7fcf1326f700 1 --2- 172.21.15.32:0/3824909712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf0c145f50 0x7fcf0c0ad580 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7fcf040032b0 tx=0x7fcf040091a0).stop 2020-12-05T17:33:35.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1326f700 1 -- 172.21.15.32:0/3824909712 shutdown_connections 2020-12-05T17:33:35.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1326f700 1 --2- 172.21.15.32:0/3824909712 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf0c0b2050 0x7fcf0c0acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1326f700 1 --2- 172.21.15.32:0/3824909712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf0c145f50 0x7fcf0c0ad580 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1326f700 1 -- 172.21.15.32:0/3824909712 >> 172.21.15.32:0/3824909712 conn(0x7fcf0c0b6ef0 msgr2=0x7fcf0c0b72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1326f700 1 -- 172.21.15.32:0/3824909712 shutdown_connections 2020-12-05T17:33:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1326f700 1 -- 172.21.15.32:0/3824909712 wait complete. 2020-12-05T17:33:35.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1326f700 1 Processor -- start 2020-12-05T17:33:35.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1326f700 1 -- start start 2020-12-05T17:33:35.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1326f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf0c0ae6b0 0x7fcf0c147820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:35.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1326f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf0c0b2050 0x7fcf0c064370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:35.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1326f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf0c145f50 0x7fcf0c1516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1326f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c155b10 con 0x7fcf0c0b2050 2020-12-05T17:33:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1326f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c1480f0 con 0x7fcf0c0ae6b0 2020-12-05T17:33:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1326f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c064a30 con 0x7fcf0c145f50 2020-12-05T17:33:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1080a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf0c0b2050 0x7fcf0c064370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1080a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf0c0b2050 0x7fcf0c064370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34294/0 (socket says 172.21.15.32:34294) 2020-12-05T17:33:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1080a700 1 -- 172.21.15.32:0/2749316934 learned_addr learned my addr 172.21.15.32:0/2749316934 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1100b700 1 --2- 172.21.15.32:0/2749316934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf0c0ae6b0 0x7fcf0c147820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1180c700 1 --2- 172.21.15.32:0/2749316934 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf0c145f50 0x7fcf0c1516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1080a700 1 -- 172.21.15.32:0/2749316934 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf0c145f50 msgr2=0x7fcf0c1516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:35.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1080a700 1 --2- 172.21.15.32:0/2749316934 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf0c145f50 0x7fcf0c1516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:35.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1080a700 1 -- 172.21.15.32:0/2749316934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf0c0ae6b0 msgr2=0x7fcf0c147820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:35.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1080a700 1 --2- 172.21.15.32:0/2749316934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf0c0ae6b0 0x7fcf0c147820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:35.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.967+0000 7fcf1080a700 1 -- 172.21.15.32:0/2749316934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0c064cd0 con 0x7fcf0c0b2050 2020-12-05T17:33:35.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.971+0000 7fcf1080a700 1 --2- 172.21.15.32:0/2749316934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf0c0b2050 0x7fcf0c064370 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7fcf040032b0 tx=0x7fcf04016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:35.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.971+0000 7fcefa7fc700 1 -- 172.21.15.32:0/2749316934 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf040107a0 con 0x7fcf0c0b2050 2020-12-05T17:33:35.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.971+0000 7fcefa7fc700 1 -- 172.21.15.32:0/2749316934 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf04010900 con 0x7fcf0c0b2050 2020-12-05T17:33:35.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.971+0000 7fcf1326f700 1 -- 172.21.15.32:0/2749316934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf0c155650 con 0x7fcf0c0b2050 2020-12-05T17:33:35.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.971+0000 7fcefa7fc700 1 -- 172.21.15.32:0/2749316934 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf04010c60 con 0x7fcf0c0b2050 2020-12-05T17:33:35.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.971+0000 7fcf1326f700 1 -- 172.21.15.32:0/2749316934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf0c148c50 con 0x7fcf0c0b2050 2020-12-05T17:33:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.971+0000 7fcf1326f700 1 -- 172.21.15.32:0/2749316934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf0c064860 con 0x7fcf0c0b2050 2020-12-05T17:33:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.971+0000 7fcefa7fc700 1 -- 172.21.15.32:0/2749316934 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fcf04027030 con 0x7fcf0c0b2050 2020-12-05T17:33:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.971+0000 7fcefa7fc700 1 --2- 172.21.15.32:0/2749316934 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fceec055330 0x7fceec0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.971+0000 7fcf1100b700 1 --2- 172.21.15.32:0/2749316934 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fceec055330 0x7fceec0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.971+0000 7fcefa7fc700 1 -- 172.21.15.32:0/2749316934 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcf0406f260 con 0x7fcf0c0b2050 2020-12-05T17:33:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.971+0000 7fcf1100b700 1 --2- 172.21.15.32:0/2749316934 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fceec055330 0x7fceec0577f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fcefc00b000 tx=0x7fcefc006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:35.979+0000 7fcefa7fc700 1 -- 172.21.15.32:0/2749316934 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fcf04045330 con 0x7fcf0c0b2050 2020-12-05T17:33:36.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.163+0000 7fcf1326f700 1 -- 172.21.15.32:0/2749316934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf0c155900 con 0x7fcf0c0b2050 2020-12-05T17:33:36.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.163+0000 7fcefa7fc700 1 -- 172.21.15.32:0/2749316934 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2136 (secure 0 0 0) 0x7fcf04025870 con 0x7fcf0c0b2050 2020-12-05T17:33:36.165 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:33:36.166 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4679,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/2850341272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2850341272},{"type":"v1","addr":"172.21.15.61:6833","nonce":2850341272}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:32:25.792490+0000","modified":"2020-12-05T17:32:26.797858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4873},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4873":{"gid":4873,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3316547885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3316547885},{"type":"v1","addr":"172.21.15.32:6835","nonce":3316547885}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-12-05T17:33:36.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.167+0000 7fcf1326f700 1 -- 172.21.15.32:0/2749316934 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fceec055330 msgr2=0x7fceec0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:36.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.167+0000 7fcf1326f700 1 --2- 172.21.15.32:0/2749316934 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fceec055330 0x7fceec0577f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fcefc00b000 tx=0x7fcefc006dc0).stop 2020-12-05T17:33:36.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.167+0000 7fcf1326f700 1 -- 172.21.15.32:0/2749316934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf0c0b2050 msgr2=0x7fcf0c064370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:36.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.167+0000 7fcf1326f700 1 --2- 172.21.15.32:0/2749316934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf0c0b2050 0x7fcf0c064370 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7fcf040032b0 tx=0x7fcf04016000).stop 2020-12-05T17:33:36.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.167+0000 7fcf1326f700 1 -- 172.21.15.32:0/2749316934 shutdown_connections 2020-12-05T17:33:36.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.167+0000 7fcf1326f700 1 --2- 172.21.15.32:0/2749316934 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fceec055330 0x7fceec0577f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:36.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.167+0000 7fcf1326f700 1 --2- 172.21.15.32:0/2749316934 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf0c145f50 0x7fcf0c1516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:36.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.167+0000 7fcf1326f700 1 --2- 172.21.15.32:0/2749316934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf0c0ae6b0 0x7fcf0c147820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:36.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.167+0000 7fcf1326f700 1 --2- 172.21.15.32:0/2749316934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf0c0b2050 0x7fcf0c064370 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:36.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.167+0000 7fcf1326f700 1 -- 172.21.15.32:0/2749316934 >> 172.21.15.32:0/2749316934 conn(0x7fcf0c0b6ef0 msgr2=0x7fcf0c146370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:36.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.167+0000 7fcf1326f700 1 -- 172.21.15.32:0/2749316934 shutdown_connections 2020-12-05T17:33:36.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.167+0000 7fcf1326f700 1 -- 172.21.15.32:0/2749316934 wait complete. 2020-12-05T17:33:36.172 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 10 2020-12-05T17:33:36.185 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-12-05T17:33:36.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f869e2700 1 Processor -- start 2020-12-05T17:33:36.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f869e2700 1 -- start start 2020-12-05T17:33:36.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f869e2700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f5f800b1fc0 0x7f5f800af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:36.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f869e2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5f800ab710 0x7f5f80061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:36.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f869e2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5f80147400 0x7f5f80145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:36.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f869e2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f800b0110 con 0x7f5f80147400 2020-12-05T17:33:36.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f869e2700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f80146a90 con 0x7f5f800b1fc0 2020-12-05T17:33:36.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f869e2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f800b23a0 con 0x7f5f800ab710 2020-12-05T17:33:36.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f861e1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5f80147400 0x7f5f80145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:36.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f861e1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5f80147400 0x7f5f80145d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34304/0 (socket says 172.21.15.32:34304) 2020-12-05T17:33:36.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f861e1700 1 -- 172.21.15.32:0/459645839 learned_addr learned my addr 172.21.15.32:0/459645839 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:36.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f859e0700 1 --2- 172.21.15.32:0/459645839 >> v2:172.21.15.61:3300/0 conn(0x7f5f800b1fc0 0x7f5f800af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:36.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f851df700 1 --2- 172.21.15.32:0/459645839 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5f800ab710 0x7f5f80061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:36.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f861e1700 1 -- 172.21.15.32:0/459645839 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5f800ab710 msgr2=0x7f5f80061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:36.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f861e1700 1 --2- 172.21.15.32:0/459645839 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5f800ab710 0x7f5f80061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:36.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f861e1700 1 -- 172.21.15.32:0/459645839 >> v2:172.21.15.61:3300/0 conn(0x7f5f800b1fc0 msgr2=0x7f5f800af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:36.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f861e1700 1 --2- 172.21.15.32:0/459645839 >> v2:172.21.15.61:3300/0 conn(0x7f5f800b1fc0 0x7f5f800af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:36.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f861e1700 1 -- 172.21.15.32:0/459645839 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f80146cc0 con 0x7f5f80147400 2020-12-05T17:33:36.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f861e1700 1 --2- 172.21.15.32:0/459645839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f80147400 0x7f5f80145d10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f5f78002f30 tx=0x7f5f78006e80).ready entity=mon.0 client_cookie=105067c85c59ebd8 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:36.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f849de700 1 -- 172.21.15.32:0/459645839 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f78009640 con 0x7f5f80147400 2020-12-05T17:33:36.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f849de700 1 -- 172.21.15.32:0/459645839 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f780097a0 con 0x7f5f80147400 2020-12-05T17:33:36.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f849de700 1 -- 172.21.15.32:0/459645839 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f78009b00 con 0x7f5f80147400 2020-12-05T17:33:36.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f869e2700 1 -- 172.21.15.32:0/459645839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f80147400 msgr2=0x7f5f80145d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:36.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f869e2700 1 --2- 172.21.15.32:0/459645839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f80147400 0x7f5f80145d10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f5f78002f30 tx=0x7f5f78006e80).stop 2020-12-05T17:33:36.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f869e2700 1 -- 172.21.15.32:0/459645839 shutdown_connections 2020-12-05T17:33:36.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f869e2700 1 --2- 172.21.15.32:0/459645839 >> v2:172.21.15.61:3300/0 conn(0x7f5f800b1fc0 0x7f5f800af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:36.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f869e2700 1 --2- 172.21.15.32:0/459645839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f80147400 0x7f5f80145d10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:36.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f869e2700 1 --2- 172.21.15.32:0/459645839 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5f800ab710 0x7f5f80061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:36.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.303+0000 7f5f869e2700 1 -- 172.21.15.32:0/459645839 >> 172.21.15.32:0/459645839 conn(0x7f5f800b5530 msgr2=0x7f5f800b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:36.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f869e2700 1 -- 172.21.15.32:0/459645839 shutdown_connections 2020-12-05T17:33:36.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f869e2700 1 -- 172.21.15.32:0/459645839 wait complete. 2020-12-05T17:33:36.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f869e2700 1 Processor -- start 2020-12-05T17:33:36.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f869e2700 1 -- start start 2020-12-05T17:33:36.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f869e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f800ab710 0x7f5f801421f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:36.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f869e2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5f800b1fc0 0x7f5f80147e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:36.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f869e2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5f80147400 0x7f5f800ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:36.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f869e2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f800ae040 con 0x7f5f800ab710 2020-12-05T17:33:36.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f869e2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f80149c80 con 0x7f5f80147400 2020-12-05T17:33:36.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f869e2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f800acac0 con 0x7f5f800b1fc0 2020-12-05T17:33:36.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f859e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f800ab710 0x7f5f801421f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:36.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f859e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f800ab710 0x7f5f801421f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34306/0 (socket says 172.21.15.32:34306) 2020-12-05T17:33:36.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f859e0700 1 -- 172.21.15.32:0/1364565419 learned_addr learned my addr 172.21.15.32:0/1364565419 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:36.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f861e1700 1 --2- 172.21.15.32:0/1364565419 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5f80147400 0x7f5f800ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:36.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f851df700 1 --2- 172.21.15.32:0/1364565419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5f800b1fc0 0x7f5f80147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:36.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f859e0700 1 -- 172.21.15.32:0/1364565419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5f800b1fc0 msgr2=0x7f5f80147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:36.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f859e0700 1 --2- 172.21.15.32:0/1364565419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5f800b1fc0 0x7f5f80147e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:36.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f859e0700 1 -- 172.21.15.32:0/1364565419 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5f80147400 msgr2=0x7f5f800ae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:36.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f859e0700 1 --2- 172.21.15.32:0/1364565419 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5f80147400 0x7f5f800ae4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:36.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f859e0700 1 -- 172.21.15.32:0/1364565419 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f801499e0 con 0x7f5f800ab710 2020-12-05T17:33:36.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f859e0700 1 --2- 172.21.15.32:0/1364565419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f800ab710 0x7f5f801421f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f5f800ad630 tx=0x7f5f700091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:36.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f7effd700 1 -- 172.21.15.32:0/1364565419 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f7000cb30 con 0x7f5f800ab710 2020-12-05T17:33:36.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f7effd700 1 -- 172.21.15.32:0/1364565419 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7000cc90 con 0x7f5f800ab710 2020-12-05T17:33:36.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f7effd700 1 -- 172.21.15.32:0/1364565419 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f700123f0 con 0x7f5f800ab710 2020-12-05T17:33:36.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f869e2700 1 -- 172.21.15.32:0/1364565419 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f80148340 con 0x7f5f800ab710 2020-12-05T17:33:36.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.307+0000 7f5f869e2700 1 -- 172.21.15.32:0/1364565419 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f8014aca0 con 0x7f5f800ab710 2020-12-05T17:33:36.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.311+0000 7f5f869e2700 1 -- 172.21.15.32:0/1364565419 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f800abe70 con 0x7f5f800ab710 2020-12-05T17:33:36.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.311+0000 7f5f7effd700 1 -- 172.21.15.32:0/1364565419 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5f70022070 con 0x7f5f800ab710 2020-12-05T17:33:36.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.311+0000 7f5f7effd700 1 --2- 172.21.15.32:0/1364565419 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5f600553b0 0x7f5f60057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:36.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.311+0000 7f5f7effd700 1 -- 172.21.15.32:0/1364565419 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5f7006e7c0 con 0x7f5f800ab710 2020-12-05T17:33:36.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.311+0000 7f5f851df700 1 --2- 172.21.15.32:0/1364565419 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5f600553b0 0x7f5f60057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:36.323 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.311+0000 7f5f851df700 1 --2- 172.21.15.32:0/1364565419 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5f600553b0 0x7f5f60057870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f5f7400b000 tx=0x7f5f74006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:36.323 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.315+0000 7f5f7effd700 1 -- 172.21.15.32:0/1364565419 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5f70044960 con 0x7f5f800ab710 2020-12-05T17:33:36.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.495+0000 7f5f869e2700 1 -- 172.21.15.32:0/1364565419 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5f80057050 con 0x7f5f800ab710 2020-12-05T17:33:36.900 INFO:tasks.ceph.mon.a.smithi032.stderr:2020-12-05T17:33:36.899+0000 7fdef8136700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-12-05T17:33:36.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.915+0000 7f5f7effd700 1 -- 172.21.15.32:0/1364565419 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v11) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5f70047f80 con 0x7f5f800ab710 2020-12-05T17:33:36.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.915+0000 7f5f869e2700 1 -- 172.21.15.32:0/1364565419 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5f600553b0 msgr2=0x7f5f60057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:36.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.915+0000 7f5f869e2700 1 --2- 172.21.15.32:0/1364565419 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5f600553b0 0x7f5f60057870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f5f7400b000 tx=0x7f5f74006dc0).stop 2020-12-05T17:33:36.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.915+0000 7f5f869e2700 1 -- 172.21.15.32:0/1364565419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f800ab710 msgr2=0x7f5f801421f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:36.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.915+0000 7f5f869e2700 1 --2- 172.21.15.32:0/1364565419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f800ab710 0x7f5f801421f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f5f800ad630 tx=0x7f5f700091a0).stop 2020-12-05T17:33:36.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.915+0000 7f5f869e2700 1 -- 172.21.15.32:0/1364565419 shutdown_connections 2020-12-05T17:33:36.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.915+0000 7f5f869e2700 1 --2- 172.21.15.32:0/1364565419 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5f80147400 0x7f5f800ae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:36.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.915+0000 7f5f869e2700 1 --2- 172.21.15.32:0/1364565419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f800ab710 0x7f5f801421f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:36.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.915+0000 7f5f869e2700 1 --2- 172.21.15.32:0/1364565419 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5f600553b0 0x7f5f60057870 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:36.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.915+0000 7f5f869e2700 1 --2- 172.21.15.32:0/1364565419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5f800b1fc0 0x7f5f80147e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:36.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.915+0000 7f5f869e2700 1 -- 172.21.15.32:0/1364565419 >> 172.21.15.32:0/1364565419 conn(0x7f5f800b5530 msgr2=0x7f5f80144820 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:36.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.915+0000 7f5f869e2700 1 -- 172.21.15.32:0/1364565419 shutdown_connections 2020-12-05T17:33:36.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:36.915+0000 7f5f869e2700 1 -- 172.21.15.32:0/1364565419 wait complete. 2020-12-05T17:33:36.921 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-12-05T17:33:36.935 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:33:36.957 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:33:37.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.039+0000 7f594359e700 1 Processor -- start 2020-12-05T17:33:37.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.039+0000 7f594359e700 1 -- start start 2020-12-05T17:33:37.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.039+0000 7f594359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59440ab710 0x7f59440af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:37.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.039+0000 7f594359e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5944147400 0x7f5944145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:37.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.039+0000 7f594359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5944146a90 con 0x7f59440b1fc0 2020-12-05T17:33:37.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.039+0000 7f594359e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5944061e90 con 0x7f59440ab710 2020-12-05T17:33:37.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.039+0000 7f594359e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59440b0110 con 0x7f5944147400 2020-12-05T17:33:37.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.039+0000 7f594259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f59440b1fc0 0x7f5944061780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36974/0 (socket says 172.21.15.32:36974) 2020-12-05T17:33:37.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.039+0000 7f594259c700 1 -- 172.21.15.32:0/1815993222 learned_addr learned my addr 172.21.15.32:0/1815993222 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:33:37.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.039+0000 7f5941d9b700 1 --2- 172.21.15.32:0/1815993222 >> v2:172.21.15.32:3300/0 conn(0x7f59440ab710 0x7f59440af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:37.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.039+0000 7f5942d9d700 1 --2- 172.21.15.32:0/1815993222 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5944147400 0x7f5944145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:37.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.043+0000 7f594159a700 1 -- 172.21.15.32:0/1815993222 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2001726529 0 0) 0x7f5944146a90 con 0x7f59440b1fc0 2020-12-05T17:33:37.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.043+0000 7f5941d9b700 1 -- 172.21.15.32:0/1815993222 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5944147400 msgr2=0x7f5944145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:37.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.043+0000 7f5941d9b700 1 --2- 172.21.15.32:0/1815993222 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5944147400 0x7f5944145d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:37.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.043+0000 7f5941d9b700 1 -- 172.21.15.32:0/1815993222 >> v1:172.21.15.32:6789/0 conn(0x7f59440b1fc0 legacy=0x7f5944061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:37.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.043+0000 7f5941d9b700 1 -- 172.21.15.32:0/1815993222 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59440b23a0 con 0x7f59440ab710 2020-12-05T17:33:37.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.043+0000 7f5942d9d700 1 --2- 172.21.15.32:0/1815993222 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5944147400 0x7f5944145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:33:37.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.043+0000 7f5941d9b700 1 --2- 172.21.15.32:0/1815993222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59440ab710 0x7f59440af230 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f592c002f30 tx=0x7f592c006dc0).ready entity=mon.0 client_cookie=673f357a30a4956a server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:37.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.043+0000 7f594159a700 1 -- 172.21.15.32:0/1815993222 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f592c009760 con 0x7f59440ab710 2020-12-05T17:33:37.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.043+0000 7f594159a700 1 -- 172.21.15.32:0/1815993222 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592c0098c0 con 0x7f59440ab710 2020-12-05T17:33:37.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.043+0000 7f594159a700 1 -- 172.21.15.32:0/1815993222 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f592c009c20 con 0x7f59440ab710 2020-12-05T17:33:37.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.043+0000 7f594359e700 1 -- 172.21.15.32:0/1815993222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59440ab710 msgr2=0x7f59440af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:37.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.043+0000 7f594359e700 1 --2- 172.21.15.32:0/1815993222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59440ab710 0x7f59440af230 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f592c002f30 tx=0x7f592c006dc0).stop 2020-12-05T17:33:37.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.043+0000 7f594359e700 1 -- 172.21.15.32:0/1815993222 shutdown_connections 2020-12-05T17:33:37.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.043+0000 7f594359e700 1 --2- 172.21.15.32:0/1815993222 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5944147400 0x7f5944145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.043+0000 7f594359e700 1 --2- 172.21.15.32:0/1815993222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59440ab710 0x7f59440af230 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.043+0000 7f594359e700 1 -- 172.21.15.32:0/1815993222 >> 172.21.15.32:0/1815993222 conn(0x7f59440b5530 msgr2=0x7f59440b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.043+0000 7f594359e700 1 -- 172.21.15.32:0/1815993222 shutdown_connections 2020-12-05T17:33:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.043+0000 7f594359e700 1 -- 172.21.15.32:0/1815993222 wait complete. 2020-12-05T17:33:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.043+0000 7f594359e700 1 Processor -- start 2020-12-05T17:33:37.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.043+0000 7f594359e700 1 -- start start 2020-12-05T17:33:37.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f594359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59440ab710 0x7f59441596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:37.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f594359e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59440b1fc0 0x7f5944144d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:37.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f594359e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5944147400 0x7f59440ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:37.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f594359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59440abc60 con 0x7f59440ab710 2020-12-05T17:33:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f594359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59440a1ed0 con 0x7f59440b1fc0 2020-12-05T17:33:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f594359e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594414dcd0 con 0x7f5944147400 2020-12-05T17:33:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f594259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59440ab710 0x7f59441596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f594259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59440ab710 0x7f59441596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34334/0 (socket says 172.21.15.32:34334) 2020-12-05T17:33:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f5942d9d700 1 --2- 172.21.15.32:0/1841899270 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5944147400 0x7f59440ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:37.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f594259c700 1 -- 172.21.15.32:0/1841899270 learned_addr learned my addr 172.21.15.32:0/1841899270 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:37.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f5941d9b700 1 --2- 172.21.15.32:0/1841899270 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59440b1fc0 0x7f5944144d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:37.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f594259c700 1 -- 172.21.15.32:0/1841899270 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5944147400 msgr2=0x7f59440ac5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:37.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f594259c700 1 --2- 172.21.15.32:0/1841899270 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5944147400 0x7f59440ac5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:37.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f594259c700 1 -- 172.21.15.32:0/1841899270 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59440b1fc0 msgr2=0x7f5944144d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:37.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f594259c700 1 --2- 172.21.15.32:0/1841899270 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59440b1fc0 0x7f5944144d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:37.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f594259c700 1 -- 172.21.15.32:0/1841899270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594414de90 con 0x7f59440ab710 2020-12-05T17:33:37.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f5942d9d700 1 --2- 172.21.15.32:0/1841899270 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5944147400 0x7f59440ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:33:37.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f5941d9b700 1 --2- 172.21.15.32:0/1841899270 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59440b1fc0 0x7f5944144d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:33:37.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f594259c700 1 --2- 172.21.15.32:0/1841899270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59440ab710 0x7f59441596f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f593c000b80 tx=0x7f593c007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:37.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f593b7fe700 1 -- 172.21.15.32:0/1841899270 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f593c0072e0 con 0x7f59440ab710 2020-12-05T17:33:37.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f593b7fe700 1 -- 172.21.15.32:0/1841899270 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593c007440 con 0x7f59440ab710 2020-12-05T17:33:37.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f593b7fe700 1 -- 172.21.15.32:0/1841899270 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f593c0057d0 con 0x7f59440ab710 2020-12-05T17:33:37.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f594359e700 1 -- 172.21.15.32:0/1841899270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59440ac060 con 0x7f59440ab710 2020-12-05T17:33:37.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.047+0000 7f594359e700 1 -- 172.21.15.32:0/1841899270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59440acc30 con 0x7f59440ab710 2020-12-05T17:33:37.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.051+0000 7f594359e700 1 -- 172.21.15.32:0/1841899270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59440abdd0 con 0x7f59440ab710 2020-12-05T17:33:37.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.051+0000 7f593b7fe700 1 -- 172.21.15.32:0/1841899270 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f593c010070 con 0x7f59440ab710 2020-12-05T17:33:37.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.051+0000 7f593b7fe700 1 --2- 172.21.15.32:0/1841899270 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f59280553b0 0x7f5928057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:37.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.051+0000 7f593b7fe700 1 -- 172.21.15.32:0/1841899270 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f593c06d820 con 0x7f59440ab710 2020-12-05T17:33:37.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.051+0000 7f5941d9b700 1 --2- 172.21.15.32:0/1841899270 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f59280553b0 0x7f5928057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:37.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.051+0000 7f5941d9b700 1 --2- 172.21.15.32:0/1841899270 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f59280553b0 0x7f5928057870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f592c01a000 tx=0x7f592c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:37.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.059+0000 7f593b7fe700 1 -- 172.21.15.32:0/1841899270 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f593c043970 con 0x7f59440ab710 2020-12-05T17:33:37.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.243+0000 7f594359e700 1 -- 172.21.15.32:0/1841899270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5944057050 con 0x7f59440ab710 2020-12-05T17:33:37.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.243+0000 7f593b7fe700 1 -- 172.21.15.32:0/1841899270 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+1776 (secure 0 0 0) 0x7f593c046f90 con 0x7f59440ab710 2020-12-05T17:33:37.244 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:33:37.244 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4679,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/2850341272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2850341272},{"type":"v1","addr":"172.21.15.61:6833","nonce":2850341272}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":11,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:32:25.792490+0000","modified":"2020-12-05T17:33:36.904310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-12-05T17:33:37.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.243+0000 7f594359e700 1 -- 172.21.15.32:0/1841899270 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f59280553b0 msgr2=0x7f5928057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:37.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.243+0000 7f594359e700 1 --2- 172.21.15.32:0/1841899270 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f59280553b0 0x7f5928057870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f592c01a000 tx=0x7f592c015000).stop 2020-12-05T17:33:37.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.243+0000 7f594359e700 1 -- 172.21.15.32:0/1841899270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59440ab710 msgr2=0x7f59441596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:37.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.243+0000 7f594359e700 1 --2- 172.21.15.32:0/1841899270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59440ab710 0x7f59441596f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f593c000b80 tx=0x7f593c007ac0).stop 2020-12-05T17:33:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.243+0000 7f594359e700 1 -- 172.21.15.32:0/1841899270 shutdown_connections 2020-12-05T17:33:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.243+0000 7f594359e700 1 --2- 172.21.15.32:0/1841899270 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f59280553b0 0x7f5928057870 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.243+0000 7f594359e700 1 --2- 172.21.15.32:0/1841899270 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5944147400 0x7f59440ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.243+0000 7f594359e700 1 --2- 172.21.15.32:0/1841899270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59440ab710 0x7f59441596f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.243+0000 7f594359e700 1 --2- 172.21.15.32:0/1841899270 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59440b1fc0 0x7f5944144d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:37.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.243+0000 7f594359e700 1 -- 172.21.15.32:0/1841899270 >> 172.21.15.32:0/1841899270 conn(0x7f59440b5530 msgr2=0x7f59440af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:37.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.243+0000 7f594359e700 1 -- 172.21.15.32:0/1841899270 shutdown_connections 2020-12-05T17:33:37.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.243+0000 7f594359e700 1 -- 172.21.15.32:0/1841899270 wait complete. 2020-12-05T17:33:37.248 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 11 2020-12-05T17:33:37.258 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-12-05T17:33:37.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.419+0000 7f23fb59e700 1 Processor -- start 2020-12-05T17:33:37.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.419+0000 7f23fb59e700 1 -- start start 2020-12-05T17:33:37.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.419+0000 7f23fb59e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f23fc0b1fc0 0x7f23fc0af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:37.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.419+0000 7f23fb59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23fc146a90 con 0x7f23fc0ab710 2020-12-05T17:33:37.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.419+0000 7f23fb59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23fc0b23a0 con 0x7f23fc147400 2020-12-05T17:33:37.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.419+0000 7f23fb59e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23fc061e90 con 0x7f23fc0b1fc0 2020-12-05T17:33:37.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.419+0000 7f23f9d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f23fc0ab710 0x7f23fc061780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36988/0 (socket says 172.21.15.32:36988) 2020-12-05T17:33:37.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.419+0000 7f23f9d9b700 1 -- 172.21.15.32:0/837522913 learned_addr learned my addr 172.21.15.32:0/837522913 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:33:37.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.423+0000 7f23fa59c700 1 --2- 172.21.15.32:0/837522913 >> v2:172.21.15.61:3300/0 conn(0x7f23fc0b1fc0 0x7f23fc0af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:37.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.423+0000 7f23f959a700 1 -- 172.21.15.32:0/837522913 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2796689642 0 0) 0x7f23fc146a90 con 0x7f23fc0ab710 2020-12-05T17:33:37.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.423+0000 7f23f959a700 1 -- 172.21.15.32:0/837522913 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23dc003230 con 0x7f23fc0ab710 2020-12-05T17:33:37.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.423+0000 7f23f959a700 1 -- 172.21.15.32:0/837522913 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2923498751 0 0) 0x7f23fc0b23a0 con 0x7f23fc147400 2020-12-05T17:33:37.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.423+0000 7f23f959a700 1 -- 172.21.15.32:0/837522913 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23fc146a90 con 0x7f23fc147400 2020-12-05T17:33:37.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.423+0000 7f23fa59c700 1 -- 172.21.15.32:0/837522913 >> v1:172.21.15.61:6789/0 conn(0x7f23fc147400 legacy=0x7f23fc145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:37.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.423+0000 7f23fa59c700 1 -- 172.21.15.32:0/837522913 >> v1:172.21.15.32:6789/0 conn(0x7f23fc0ab710 legacy=0x7f23fc061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:37.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.423+0000 7f23fa59c700 1 -- 172.21.15.32:0/837522913 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23fc146cc0 con 0x7f23fc0b1fc0 2020-12-05T17:33:37.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.423+0000 7f23fa59c700 1 --2- 172.21.15.32:0/837522913 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23fc0b1fc0 0x7f23fc0af230 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f23e4002f30 tx=0x7f23e4006dc0).ready entity=mon.1 client_cookie=5baa300215c48cd9 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:37.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.423+0000 7f23f959a700 1 -- 172.21.15.32:0/837522913 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23e4009740 con 0x7f23fc0b1fc0 2020-12-05T17:33:37.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.423+0000 7f23f959a700 1 -- 172.21.15.32:0/837522913 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e40098a0 con 0x7f23fc0b1fc0 2020-12-05T17:33:37.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.423+0000 7f23f959a700 1 -- 172.21.15.32:0/837522913 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23e4009c00 con 0x7f23fc0b1fc0 2020-12-05T17:33:37.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.423+0000 7f23fb59e700 1 -- 172.21.15.32:0/837522913 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23fc0b1fc0 msgr2=0x7f23fc0af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:37.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.423+0000 7f23fb59e700 1 --2- 172.21.15.32:0/837522913 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23fc0b1fc0 0x7f23fc0af230 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f23e4002f30 tx=0x7f23e4006dc0).stop 2020-12-05T17:33:37.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.423+0000 7f23fb59e700 1 -- 172.21.15.32:0/837522913 shutdown_connections 2020-12-05T17:33:37.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.423+0000 7f23fb59e700 1 --2- 172.21.15.32:0/837522913 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23fc0b1fc0 0x7f23fc0af230 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:37.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.423+0000 7f23fb59e700 1 -- 172.21.15.32:0/837522913 >> 172.21.15.32:0/837522913 conn(0x7f23fc0b5530 msgr2=0x7f23fc0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:37.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.423+0000 7f23fb59e700 1 -- 172.21.15.32:0/837522913 shutdown_connections 2020-12-05T17:33:37.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.423+0000 7f23fb59e700 1 -- 172.21.15.32:0/837522913 wait complete. 2020-12-05T17:33:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.423+0000 7f23fb59e700 1 Processor -- start 2020-12-05T17:33:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.423+0000 7f23fb59e700 1 -- start start 2020-12-05T17:33:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23fb59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23fc0ab710 0x7f23fc14a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23fb59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23fc0b1fc0 0x7f23fc0a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:37.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23fb59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23fc147400 0x7f23fc0a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:37.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23fb59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc066460 con 0x7f23fc0ab710 2020-12-05T17:33:37.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23fb59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc066700 con 0x7f23fc0b1fc0 2020-12-05T17:33:37.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23fb59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc1566f0 con 0x7f23fc147400 2020-12-05T17:33:37.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23fa59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23fc0ab710 0x7f23fc14a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:37.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23fa59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23fc0ab710 0x7f23fc14a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34346/0 (socket says 172.21.15.32:34346) 2020-12-05T17:33:37.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23fa59c700 1 -- 172.21.15.32:0/2553375134 learned_addr learned my addr 172.21.15.32:0/2553375134 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:37.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23f9d9b700 1 --2- 172.21.15.32:0/2553375134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23fc0b1fc0 0x7f23fc0a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:37.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23fad9d700 1 --2- 172.21.15.32:0/2553375134 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23fc147400 0x7f23fc0a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:37.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23fa59c700 1 -- 172.21.15.32:0/2553375134 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23fc147400 msgr2=0x7f23fc0a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:37.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23fa59c700 1 --2- 172.21.15.32:0/2553375134 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23fc147400 0x7f23fc0a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:37.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23fa59c700 1 -- 172.21.15.32:0/2553375134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23fc0b1fc0 msgr2=0x7f23fc0a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:37.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23fa59c700 1 --2- 172.21.15.32:0/2553375134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23fc0b1fc0 0x7f23fc0a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:37.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23fa59c700 1 -- 172.21.15.32:0/2553375134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23fc063200 con 0x7f23fc0ab710 2020-12-05T17:33:37.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23fad9d700 1 --2- 172.21.15.32:0/2553375134 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23fc147400 0x7f23fc0a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:33:37.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23fa59c700 1 --2- 172.21.15.32:0/2553375134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23fc0ab710 0x7f23fc14a1a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f23e4002f30 tx=0x7f23e4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:37.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23f37fe700 1 -- 172.21.15.32:0/2553375134 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23e4011c30 con 0x7f23fc0ab710 2020-12-05T17:33:37.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23f37fe700 1 -- 172.21.15.32:0/2553375134 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e4011d90 con 0x7f23fc0ab710 2020-12-05T17:33:37.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23fb59e700 1 -- 172.21.15.32:0/2553375134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23fc0a1e00 con 0x7f23fc0ab710 2020-12-05T17:33:37.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23f37fe700 1 -- 172.21.15.32:0/2553375134 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23e401d640 con 0x7f23fc0ab710 2020-12-05T17:33:37.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23fb59e700 1 -- 172.21.15.32:0/2553375134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23fc156440 con 0x7f23fc0ab710 2020-12-05T17:33:37.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23fb59e700 1 -- 172.21.15.32:0/2553375134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23fc0668c0 con 0x7f23fc0ab710 2020-12-05T17:33:37.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23f37fe700 1 -- 172.21.15.32:0/2553375134 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f23e400f070 con 0x7f23fc0ab710 2020-12-05T17:33:37.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.427+0000 7f23f37fe700 1 --2- 172.21.15.32:0/2553375134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f23e00553b0 0x7f23e0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:37.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.431+0000 7f23f9d9b700 1 --2- 172.21.15.32:0/2553375134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f23e00553b0 0x7f23e0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:37.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.431+0000 7f23f37fe700 1 -- 172.21.15.32:0/2553375134 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f23e406fa50 con 0x7f23fc0ab710 2020-12-05T17:33:37.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.431+0000 7f23f9d9b700 1 --2- 172.21.15.32:0/2553375134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f23e00553b0 0x7f23e0057870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f23ec00b000 tx=0x7f23ec009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:37.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.435+0000 7f23f37fe700 1 -- 172.21.15.32:0/2553375134 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f23e4045ba0 con 0x7f23fc0ab710 2020-12-05T17:33:37.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.623+0000 7f23fb59e700 1 -- 172.21.15.32:0/2553375134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f23fc057050 con 0x7f23fc0ab710 2020-12-05T17:33:37.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.915+0000 7f23f37fe700 1 -- 172.21.15.32:0/2553375134 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v12) v1 ==== 104+0+0 (secure 0 0 0) 0x7f23e40491c0 con 0x7f23fc0ab710 2020-12-05T17:33:37.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.919+0000 7f23fb59e700 1 -- 172.21.15.32:0/2553375134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f23e00553b0 msgr2=0x7f23e0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:37.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.919+0000 7f23fb59e700 1 --2- 172.21.15.32:0/2553375134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f23e00553b0 0x7f23e0057870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f23ec00b000 tx=0x7f23ec009040).stop 2020-12-05T17:33:37.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.919+0000 7f23fb59e700 1 -- 172.21.15.32:0/2553375134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23fc0ab710 msgr2=0x7f23fc14a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:37.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.919+0000 7f23fb59e700 1 --2- 172.21.15.32:0/2553375134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23fc0ab710 0x7f23fc14a1a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f23e4002f30 tx=0x7f23e4015000).stop 2020-12-05T17:33:37.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.919+0000 7f23fb59e700 1 -- 172.21.15.32:0/2553375134 shutdown_connections 2020-12-05T17:33:37.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.919+0000 7f23fb59e700 1 --2- 172.21.15.32:0/2553375134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f23e00553b0 0x7f23e0057870 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:37.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.919+0000 7f23fb59e700 1 --2- 172.21.15.32:0/2553375134 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23fc147400 0x7f23fc0a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:37.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.919+0000 7f23fb59e700 1 --2- 172.21.15.32:0/2553375134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23fc0ab710 0x7f23fc14a1a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:37.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.919+0000 7f23fb59e700 1 --2- 172.21.15.32:0/2553375134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23fc0b1fc0 0x7f23fc0a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:37.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.919+0000 7f23fb59e700 1 -- 172.21.15.32:0/2553375134 >> 172.21.15.32:0/2553375134 conn(0x7f23fc0b5530 msgr2=0x7f23fc0afb60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:37.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.919+0000 7f23fb59e700 1 -- 172.21.15.32:0/2553375134 shutdown_connections 2020-12-05T17:33:37.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:37.919+0000 7f23fb59e700 1 -- 172.21.15.32:0/2553375134 wait complete. 2020-12-05T17:33:37.939 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-12-05T17:33:38.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.063+0000 7f9f05a6d700 1 Processor -- start 2020-12-05T17:33:38.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.063+0000 7f9f05a6d700 1 -- start start 2020-12-05T17:33:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.063+0000 7f9f05a6d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9f0005f3b0 0x7f9f0005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.063+0000 7f9f05a6d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9f000b2fb0 0x7f9f00061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.063+0000 7f9f05a6d700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f000b2df0 con 0x7f9f0005feb0 2020-12-05T17:33:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.063+0000 7f9f05a6d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f000b3c90 con 0x7f9f000b2fb0 2020-12-05T17:33:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.063+0000 7f9f05a6d700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f000b2b50 con 0x7f9f0005f3b0 2020-12-05T17:33:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.063+0000 7f9f0526c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9f000b2fb0 0x7f9f00061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.063+0000 7f9f0526c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9f000b2fb0 0x7f9f00061780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34360/0 (socket says 172.21.15.32:34360) 2020-12-05T17:33:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.063+0000 7f9f0526c700 1 -- 172.21.15.32:0/3378805915 learned_addr learned my addr 172.21.15.32:0/3378805915 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.063+0000 7f9f04a6b700 1 --2- 172.21.15.32:0/3378805915 >> v2:172.21.15.61:3300/0 conn(0x7f9f0005f3b0 0x7f9f0005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:38.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.067+0000 7f9f0526c700 1 -- 172.21.15.32:0/3378805915 >> v2:172.21.15.61:3300/0 conn(0x7f9f0005f3b0 msgr2=0x7f9f0005f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:38.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.067+0000 7f9f0526c700 1 --2- 172.21.15.32:0/3378805915 >> v2:172.21.15.61:3300/0 conn(0x7f9f0005f3b0 0x7f9f0005f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:38.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.067+0000 7f9f0526c700 1 -- 172.21.15.32:0/3378805915 >> v1:172.21.15.61:6789/0 conn(0x7f9f0005feb0 legacy=0x7f9f0013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:38.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.067+0000 7f9f0526c700 1 -- 172.21.15.32:0/3378805915 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f00142820 con 0x7f9f000b2fb0 2020-12-05T17:33:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.067+0000 7f9f0526c700 1 --2- 172.21.15.32:0/3378805915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f000b2fb0 0x7f9f00061780 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f9ef80032b0 tx=0x7f9ef8009270).ready entity=mon.0 client_cookie=ddb0a408f903797d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.067+0000 7f9eff7fe700 1 -- 172.21.15.32:0/3378805915 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ef8009e60 con 0x7f9f000b2fb0 2020-12-05T17:33:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.067+0000 7f9eff7fe700 1 -- 172.21.15.32:0/3378805915 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef800bc00 con 0x7f9f000b2fb0 2020-12-05T17:33:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.067+0000 7f9eff7fe700 1 -- 172.21.15.32:0/3378805915 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ef800bd70 con 0x7f9f000b2fb0 2020-12-05T17:33:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.067+0000 7f9f05a6d700 1 -- 172.21.15.32:0/3378805915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f000b2fb0 msgr2=0x7f9f00061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.067+0000 7f9f05a6d700 1 --2- 172.21.15.32:0/3378805915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f000b2fb0 0x7f9f00061780 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f9ef80032b0 tx=0x7f9ef8009270).stop 2020-12-05T17:33:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.067+0000 7f9f05a6d700 1 -- 172.21.15.32:0/3378805915 shutdown_connections 2020-12-05T17:33:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.067+0000 7f9f05a6d700 1 --2- 172.21.15.32:0/3378805915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f000b2fb0 0x7f9f00061780 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.067+0000 7f9f05a6d700 1 --2- 172.21.15.32:0/3378805915 >> v2:172.21.15.61:3300/0 conn(0x7f9f0005f3b0 0x7f9f0005f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.067+0000 7f9f05a6d700 1 -- 172.21.15.32:0/3378805915 >> 172.21.15.32:0/3378805915 conn(0x7f9f000b8f20 msgr2=0x7f9f000b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:38.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.067+0000 7f9f05a6d700 1 -- 172.21.15.32:0/3378805915 shutdown_connections 2020-12-05T17:33:38.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.067+0000 7f9f05a6d700 1 -- 172.21.15.32:0/3378805915 wait complete. 2020-12-05T17:33:38.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.067+0000 7f9f05a6d700 1 Processor -- start 2020-12-05T17:33:38.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.067+0000 7f9f05a6d700 1 -- start start 2020-12-05T17:33:38.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.071+0000 7f9f05a6d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9f0005f3b0 0x7f9f00141270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:38.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.071+0000 7f9f05a6d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9f0005feb0 0x7f9f0013f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:38.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.071+0000 7f9f05a6d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f000b2fb0 0x7f9f00154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:38.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.071+0000 7f9f05a6d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f00141b20 con 0x7f9f000b2fb0 2020-12-05T17:33:38.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.071+0000 7f9f05a6d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f00140bf0 con 0x7f9f0005f3b0 2020-12-05T17:33:38.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.071+0000 7f9f05a6d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f001556b0 con 0x7f9f0005feb0 2020-12-05T17:33:38.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.071+0000 7f9f0526c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f000b2fb0 0x7f9f00154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:38.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.071+0000 7f9f0526c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f000b2fb0 0x7f9f00154d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34364/0 (socket says 172.21.15.32:34364) 2020-12-05T17:33:38.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.071+0000 7f9f04a6b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9f0005f3b0 0x7f9f00141270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:38.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.071+0000 7f9f0526c700 1 -- 172.21.15.32:0/1131398349 learned_addr learned my addr 172.21.15.32:0/1131398349 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:38.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.071+0000 7f9efffff700 1 --2- 172.21.15.32:0/1131398349 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9f0005feb0 0x7f9f0013f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:38.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.071+0000 7f9f0526c700 1 -- 172.21.15.32:0/1131398349 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9f0005feb0 msgr2=0x7f9f0013f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:38.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.071+0000 7f9f0526c700 1 --2- 172.21.15.32:0/1131398349 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9f0005feb0 0x7f9f0013f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:38.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.071+0000 7f9f0526c700 1 -- 172.21.15.32:0/1131398349 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9f0005f3b0 msgr2=0x7f9f00141270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:38.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.071+0000 7f9f0526c700 1 --2- 172.21.15.32:0/1131398349 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9f0005f3b0 0x7f9f00141270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:38.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.071+0000 7f9f0526c700 1 -- 172.21.15.32:0/1131398349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f00140dd0 con 0x7f9f000b2fb0 2020-12-05T17:33:38.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.071+0000 7f9f0526c700 1 --2- 172.21.15.32:0/1131398349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f000b2fb0 0x7f9f00154d10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f9ef80032b0 tx=0x7f9ef8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:38.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.071+0000 7f9efdffb700 1 -- 172.21.15.32:0/1131398349 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ef80105b0 con 0x7f9f000b2fb0 2020-12-05T17:33:38.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.071+0000 7f9efdffb700 1 -- 172.21.15.32:0/1131398349 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef8010710 con 0x7f9f000b2fb0 2020-12-05T17:33:38.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.071+0000 7f9f05a6d700 1 -- 172.21.15.32:0/1131398349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f0013f190 con 0x7f9f000b2fb0 2020-12-05T17:33:38.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.071+0000 7f9efdffb700 1 -- 172.21.15.32:0/1131398349 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ef8010a70 con 0x7f9f000b2fb0 2020-12-05T17:33:38.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.071+0000 7f9f05a6d700 1 -- 172.21.15.32:0/1131398349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f00155420 con 0x7f9f000b2fb0 2020-12-05T17:33:38.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.075+0000 7f9f05a6d700 1 -- 172.21.15.32:0/1131398349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f00140f60 con 0x7f9f000b2fb0 2020-12-05T17:33:38.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.075+0000 7f9efdffb700 1 -- 172.21.15.32:0/1131398349 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9ef800f070 con 0x7f9f000b2fb0 2020-12-05T17:33:38.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.075+0000 7f9efdffb700 1 --2- 172.21.15.32:0/1131398349 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9ee4055440 0x7f9ee4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:38.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.075+0000 7f9f04a6b700 1 --2- 172.21.15.32:0/1131398349 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9ee4055440 0x7f9ee4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:38.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.075+0000 7f9efdffb700 1 -- 172.21.15.32:0/1131398349 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f9ef806f830 con 0x7f9f000b2fb0 2020-12-05T17:33:38.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.075+0000 7f9f04a6b700 1 --2- 172.21.15.32:0/1131398349 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9ee4055440 0x7f9ee4057900 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f9ef000b000 tx=0x7f9ef0006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:38.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.083+0000 7f9efdffb700 1 -- 172.21.15.32:0/1131398349 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9ef8045990 con 0x7f9f000b2fb0 2020-12-05T17:33:38.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.227+0000 7f9f05a6d700 1 -- 172.21.15.32:0/1131398349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9f00057050 con 0x7f9f000b2fb0 2020-12-05T17:33:38.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.927+0000 7f9efdffb700 1 -- 172.21.15.32:0/1131398349 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) v1 ==== 185+0+0 (secure 0 0 0) 0x7f9ef80150e0 con 0x7f9f000b2fb0 2020-12-05T17:33:38.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.931+0000 7f9f05a6d700 1 -- 172.21.15.32:0/1131398349 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9ee4055440 msgr2=0x7f9ee4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:38.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.931+0000 7f9f05a6d700 1 --2- 172.21.15.32:0/1131398349 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9ee4055440 0x7f9ee4057900 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f9ef000b000 tx=0x7f9ef0006dc0).stop 2020-12-05T17:33:38.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.931+0000 7f9f05a6d700 1 -- 172.21.15.32:0/1131398349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f000b2fb0 msgr2=0x7f9f00154d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:38.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.931+0000 7f9f05a6d700 1 --2- 172.21.15.32:0/1131398349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f000b2fb0 0x7f9f00154d10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f9ef80032b0 tx=0x7f9ef8016000).stop 2020-12-05T17:33:38.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.931+0000 7f9f05a6d700 1 -- 172.21.15.32:0/1131398349 shutdown_connections 2020-12-05T17:33:38.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.931+0000 7f9f05a6d700 1 --2- 172.21.15.32:0/1131398349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f000b2fb0 0x7f9f00154d10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:38.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.931+0000 7f9f05a6d700 1 --2- 172.21.15.32:0/1131398349 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9f0005f3b0 0x7f9f00141270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:38.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.931+0000 7f9f05a6d700 1 --2- 172.21.15.32:0/1131398349 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9ee4055440 0x7f9ee4057900 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:38.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.931+0000 7f9f05a6d700 1 --2- 172.21.15.32:0/1131398349 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9f0005feb0 0x7f9f0013f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:38.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.931+0000 7f9f05a6d700 1 -- 172.21.15.32:0/1131398349 >> 172.21.15.32:0/1131398349 conn(0x7f9f000b8f20 msgr2=0x7f9f00061780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:38.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.931+0000 7f9f05a6d700 1 -- 172.21.15.32:0/1131398349 shutdown_connections 2020-12-05T17:33:38.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:38.931+0000 7f9f05a6d700 1 -- 172.21.15.32:0/1131398349 wait complete. 2020-12-05T17:33:38.934 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2020-12-05T17:33:38.946 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-12-05T17:33:39.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.063+0000 7fccb6a77700 1 Processor -- start 2020-12-05T17:33:39.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.063+0000 7fccb6a77700 1 -- start start 2020-12-05T17:33:39.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fccb6a77700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fccb00acbf0 0x7fccb0146540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:39.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fccb6a77700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fccb00b0010 0x7fccb0141830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:39.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fccb6a77700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccb00b2db0 con 0x7fccb00b29d0 2020-12-05T17:33:39.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fccb6a77700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccb00b07e0 con 0x7fccb00b0010 2020-12-05T17:33:39.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fccb6a77700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb00b0a80 con 0x7fccb00acbf0 2020-12-05T17:33:39.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fccb4813700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fccb00b29d0 0x7fccb0145e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37018/0 (socket says 172.21.15.32:37018) 2020-12-05T17:33:39.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fccb4813700 1 -- 172.21.15.32:0/1254111814 learned_addr learned my addr 172.21.15.32:0/1254111814 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:33:39.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fccb5014700 1 --2- 172.21.15.32:0/1254111814 >> v2:172.21.15.32:3300/0 conn(0x7fccb00b0010 0x7fccb0141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:39.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fcca7fff700 1 --2- 172.21.15.32:0/1254111814 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fccb00acbf0 0x7fccb0146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:39.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fcca77fe700 1 -- 172.21.15.32:0/1254111814 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2557653730 0 0) 0x7fccb00b2db0 con 0x7fccb00b29d0 2020-12-05T17:33:39.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fcca77fe700 1 -- 172.21.15.32:0/1254111814 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc9c003230 con 0x7fccb00b29d0 2020-12-05T17:33:39.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fccb5014700 1 -- 172.21.15.32:0/1254111814 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fccb00acbf0 msgr2=0x7fccb0146540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:39.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fccb5014700 1 --2- 172.21.15.32:0/1254111814 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fccb00acbf0 0x7fccb0146540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:39.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fccb5014700 1 -- 172.21.15.32:0/1254111814 >> v1:172.21.15.32:6789/0 conn(0x7fccb00b29d0 legacy=0x7fccb0145e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:39.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fccb5014700 1 -- 172.21.15.32:0/1254111814 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb00b0d20 con 0x7fccb00b0010 2020-12-05T17:33:39.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fccb5014700 1 --2- 172.21.15.32:0/1254111814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb00b0010 0x7fccb0141830 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fcca80032b0 tx=0x7fcca80091a0).ready entity=mon.0 client_cookie=19920abd9fb6b0ce server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:39.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fcca77fe700 1 -- 172.21.15.32:0/1254111814 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcca800b940 con 0x7fccb00b0010 2020-12-05T17:33:39.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fcca77fe700 1 -- 172.21.15.32:0/1254111814 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca800baa0 con 0x7fccb00b0010 2020-12-05T17:33:39.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fcca77fe700 1 -- 172.21.15.32:0/1254111814 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcca800bc10 con 0x7fccb00b0010 2020-12-05T17:33:39.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fccb6a77700 1 -- 172.21.15.32:0/1254111814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb00b0010 msgr2=0x7fccb0141830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:39.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fccb6a77700 1 --2- 172.21.15.32:0/1254111814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb00b0010 0x7fccb0141830 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fcca80032b0 tx=0x7fcca80091a0).stop 2020-12-05T17:33:39.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fccb6a77700 1 -- 172.21.15.32:0/1254111814 shutdown_connections 2020-12-05T17:33:39.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fccb6a77700 1 --2- 172.21.15.32:0/1254111814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb00b0010 0x7fccb0141830 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:39.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fccb6a77700 1 --2- 172.21.15.32:0/1254111814 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fccb00acbf0 0x7fccb0146540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:39.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fccb6a77700 1 -- 172.21.15.32:0/1254111814 >> 172.21.15.32:0/1254111814 conn(0x7fccb0062310 msgr2=0x7fccb00626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:39.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fccb6a77700 1 -- 172.21.15.32:0/1254111814 shutdown_connections 2020-12-05T17:33:39.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.067+0000 7fccb6a77700 1 -- 172.21.15.32:0/1254111814 wait complete. 2020-12-05T17:33:39.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fccb6a77700 1 Processor -- start 2020-12-05T17:33:39.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fccb6a77700 1 -- start start 2020-12-05T17:33:39.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fccb6a77700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fccb00acbf0 0x7fccb01431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:39.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fccb6a77700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fccb00b0010 0x7fccb013be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:39.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fccb6a77700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb00b29d0 0x7fccb013c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:39.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fccb6a77700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb010f7e0 con 0x7fccb00b29d0 2020-12-05T17:33:39.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fccb6a77700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb0151b50 con 0x7fccb00b0010 2020-12-05T17:33:39.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fccb6a77700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb0151df0 con 0x7fccb00acbf0 2020-12-05T17:33:39.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fccb5014700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb00b29d0 0x7fccb013c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:39.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fccb5014700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb00b29d0 0x7fccb013c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34380/0 (socket says 172.21.15.32:34380) 2020-12-05T17:33:39.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fccb5014700 1 -- 172.21.15.32:0/2092527478 learned_addr learned my addr 172.21.15.32:0/2092527478 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:39.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fccb4813700 1 --2- 172.21.15.32:0/2092527478 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fccb00acbf0 0x7fccb01431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:39.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fcca7fff700 1 --2- 172.21.15.32:0/2092527478 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fccb00b0010 0x7fccb013be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:39.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fccb5014700 1 -- 172.21.15.32:0/2092527478 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fccb00acbf0 msgr2=0x7fccb01431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fccb5014700 1 --2- 172.21.15.32:0/2092527478 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fccb00acbf0 0x7fccb01431c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fccb5014700 1 -- 172.21.15.32:0/2092527478 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fccb00b0010 msgr2=0x7fccb013be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fccb5014700 1 --2- 172.21.15.32:0/2092527478 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fccb00b0010 0x7fccb013be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fccb5014700 1 -- 172.21.15.32:0/2092527478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb0152090 con 0x7fccb00b29d0 2020-12-05T17:33:39.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fccb5014700 1 --2- 172.21.15.32:0/2092527478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb00b29d0 0x7fccb013c300 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7fcca80032b0 tx=0x7fcca8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:39.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fcca5ffb700 1 -- 172.21.15.32:0/2092527478 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcca800be80 con 0x7fccb00b29d0 2020-12-05T17:33:39.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fccb6a77700 1 -- 172.21.15.32:0/2092527478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb013c7f0 con 0x7fccb00b29d0 2020-12-05T17:33:39.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fcca5ffb700 1 -- 172.21.15.32:0/2092527478 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca801f3f0 con 0x7fccb00b29d0 2020-12-05T17:33:39.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fcca5ffb700 1 -- 172.21.15.32:0/2092527478 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcca801f750 con 0x7fccb00b29d0 2020-12-05T17:33:39.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fccb6a77700 1 -- 172.21.15.32:0/2092527478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb0149b80 con 0x7fccb00b29d0 2020-12-05T17:33:39.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fcca5ffb700 1 -- 172.21.15.32:0/2092527478 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fcca8026030 con 0x7fccb00b29d0 2020-12-05T17:33:39.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.071+0000 7fccb6a77700 1 -- 172.21.15.32:0/2092527478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccb0151990 con 0x7fccb00b29d0 2020-12-05T17:33:39.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.075+0000 7fcca5ffb700 1 --2- 172.21.15.32:0/2092527478 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcc90055440 0x7fcc90057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:39.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.075+0000 7fcca5ffb700 1 -- 172.21.15.32:0/2092527478 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6068+0+0 (secure 0 0 0) 0x7fcca806ea20 con 0x7fccb00b29d0 2020-12-05T17:33:39.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.075+0000 7fccb4813700 1 --2- 172.21.15.32:0/2092527478 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcc90055440 0x7fcc90057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:39.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.075+0000 7fccb4813700 1 --2- 172.21.15.32:0/2092527478 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcc90055440 0x7fcc90057900 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fccac00b000 tx=0x7fccac009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:39.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.083+0000 7fcca5ffb700 1 -- 172.21.15.32:0/2092527478 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fcca8044c50 con 0x7fccb00b29d0 2020-12-05T17:33:39.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.235+0000 7fccb6a77700 1 -- 172.21.15.32:0/2092527478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fccb0152220 con 0x7fccb00b29d0 2020-12-05T17:33:39.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.931+0000 7fcca5ffb700 1 -- 172.21.15.32:0/2092527478 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) v1 ==== 173+0+0 (secure 0 0 0) 0x7fcca80240d0 con 0x7fccb00b29d0 2020-12-05T17:33:39.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.935+0000 7fccb6a77700 1 -- 172.21.15.32:0/2092527478 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcc90055440 msgr2=0x7fcc90057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:39.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.935+0000 7fccb6a77700 1 --2- 172.21.15.32:0/2092527478 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcc90055440 0x7fcc90057900 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fccac00b000 tx=0x7fccac009040).stop 2020-12-05T17:33:39.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.939+0000 7fccb6a77700 1 -- 172.21.15.32:0/2092527478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb00b29d0 msgr2=0x7fccb013c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:39.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.939+0000 7fccb6a77700 1 --2- 172.21.15.32:0/2092527478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb00b29d0 0x7fccb013c300 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7fcca80032b0 tx=0x7fcca8016000).stop 2020-12-05T17:33:39.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.939+0000 7fccb6a77700 1 -- 172.21.15.32:0/2092527478 shutdown_connections 2020-12-05T17:33:39.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.939+0000 7fccb6a77700 1 --2- 172.21.15.32:0/2092527478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb00b29d0 0x7fccb013c300 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:39.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.939+0000 7fccb6a77700 1 --2- 172.21.15.32:0/2092527478 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcc90055440 0x7fcc90057900 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:39.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.939+0000 7fccb6a77700 1 --2- 172.21.15.32:0/2092527478 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fccb00acbf0 0x7fccb01431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:39.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.939+0000 7fccb6a77700 1 --2- 172.21.15.32:0/2092527478 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fccb00b0010 0x7fccb013be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:39.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.939+0000 7fccb6a77700 1 -- 172.21.15.32:0/2092527478 >> 172.21.15.32:0/2092527478 conn(0x7fccb0062310 msgr2=0x7fccb00ae300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:39.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.939+0000 7fccb6a77700 1 -- 172.21.15.32:0/2092527478 shutdown_connections 2020-12-05T17:33:39.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:39.939+0000 7fccb6a77700 1 -- 172.21.15.32:0/2092527478 wait complete. 2020-12-05T17:33:39.942 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2020-12-05T17:33:39.957 INFO:tasks.ceph.mds.a:Restarting daemon 2020-12-05T17:33:39.957 INFO:tasks.ceph.mds.a:Stopping old one... 2020-12-05T17:33:39.957 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-12-05T17:33:39.958 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:33:39.958 INFO:tasks.ceph.mds.b:Restarting daemon 2020-12-05T17:33:39.958 INFO:tasks.ceph.mds.b:Stopping old one... 2020-12-05T17:33:39.959 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-12-05T17:33:39.959 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:33:39.967 INFO:tasks.ceph.mds.a:Stopped 2020-12-05T17:33:39.967 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-12-05T17:33:39.970 INFO:tasks.ceph.mds.b:Stopped 2020-12-05T17:33:39.970 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-12-05T17:33:40.012 INFO:tasks.ceph.mds.a:Started 2020-12-05T17:33:40.012 DEBUG:teuthology.parallel:result is None 2020-12-05T17:33:40.013 INFO:tasks.ceph.mds.b:Started 2020-12-05T17:33:40.014 DEBUG:teuthology.parallel:result is None 2020-12-05T17:33:40.014 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-12-05T17:33:40.088 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:33:40.089 INFO:tasks.ceph.mds.b.smithi061.stdout:starting mds.b at 2020-12-05T17:33:40.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.123+0000 7f5bef59e700 1 Processor -- start 2020-12-05T17:33:40.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.123+0000 7f5bef59e700 1 -- start start 2020-12-05T17:33:40.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.123+0000 7f5bef59e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f5bf0062400 0x7f5bf005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:40.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.123+0000 7f5bef59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf0142450 con 0x7f5bf005feb0 2020-12-05T17:33:40.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.123+0000 7f5bef59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf0061780 con 0x7f5bf005f3b0 2020-12-05T17:33:40.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.123+0000 7f5bef59e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bf0061a20 con 0x7f5bf0062400 2020-12-05T17:33:40.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.123+0000 7f5bee59c700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f5bf005f3b0 0x7f5bf00b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:49684/0 (socket says 172.21.15.32:49684) 2020-12-05T17:33:40.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.123+0000 7f5bee59c700 1 -- 172.21.15.32:0/414074701 learned_addr learned my addr 172.21.15.32:0/414074701 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:33:40.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.123+0000 7f5bedd9b700 1 --2- 172.21.15.32:0/414074701 >> v2:172.21.15.61:3300/0 conn(0x7f5bf0062400 0x7f5bf005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:40.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.123+0000 7f5beed9d700 1 --1- 172.21.15.32:0/414074701 >> v1:172.21.15.32:6789/0 conn(0x7f5bf005feb0 0x7f5bf00b0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37048/0 (socket says 172.21.15.32:37048) 2020-12-05T17:33:40.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.123+0000 7f5bed59a700 1 -- 172.21.15.32:0/414074701 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2981836207 0 0) 0x7f5bf0061780 con 0x7f5bf005f3b0 2020-12-05T17:33:40.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.123+0000 7f5bed59a700 1 -- 172.21.15.32:0/414074701 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bd4003230 con 0x7f5bf005f3b0 2020-12-05T17:33:40.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.123+0000 7f5bed59a700 1 -- 172.21.15.32:0/414074701 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1127504434 0 0) 0x7f5bf0142450 con 0x7f5bf005feb0 2020-12-05T17:33:40.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.123+0000 7f5bed59a700 1 -- 172.21.15.32:0/414074701 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bf0061780 con 0x7f5bf005feb0 2020-12-05T17:33:40.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.123+0000 7f5bedd9b700 1 -- 172.21.15.32:0/414074701 >> v1:172.21.15.61:6789/0 conn(0x7f5bf005f3b0 legacy=0x7f5bf00b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:40.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.123+0000 7f5bedd9b700 1 -- 172.21.15.32:0/414074701 >> v1:172.21.15.32:6789/0 conn(0x7f5bf005feb0 legacy=0x7f5bf00b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:40.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.123+0000 7f5bed59a700 1 -- 172.21.15.32:0/414074701 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 551947854 0 0) 0x7f5bf0061780 con 0x7f5bf005feb0 2020-12-05T17:33:40.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.123+0000 7f5bedd9b700 1 -- 172.21.15.32:0/414074701 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf0061cc0 con 0x7f5bf0062400 2020-12-05T17:33:40.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.123+0000 7f5bedd9b700 1 --2- 172.21.15.32:0/414074701 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5bf0062400 0x7f5bf005f790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f5be8001450 tx=0x7f5be4006dc0).ready entity=mon.1 client_cookie=4629e1e3b6b404d4 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:40.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.127+0000 7f5bed59a700 1 -- 172.21.15.32:0/414074701 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5be8001a50 con 0x7f5bf0062400 2020-12-05T17:33:40.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.127+0000 7f5bed59a700 1 -- 172.21.15.32:0/414074701 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be400d610 con 0x7f5bf0062400 2020-12-05T17:33:40.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.127+0000 7f5bed59a700 1 -- 172.21.15.32:0/414074701 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5be400d780 con 0x7f5bf0062400 2020-12-05T17:33:40.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.127+0000 7f5bef59e700 1 -- 172.21.15.32:0/414074701 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5bf0062400 msgr2=0x7f5bf005f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:40.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.127+0000 7f5bef59e700 1 --2- 172.21.15.32:0/414074701 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5bf0062400 0x7f5bf005f790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f5be8001450 tx=0x7f5be4006dc0).stop 2020-12-05T17:33:40.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.127+0000 7f5bef59e700 1 -- 172.21.15.32:0/414074701 shutdown_connections 2020-12-05T17:33:40.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.127+0000 7f5bef59e700 1 --2- 172.21.15.32:0/414074701 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5bf0062400 0x7f5bf005f790 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:40.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.127+0000 7f5bef59e700 1 -- 172.21.15.32:0/414074701 >> 172.21.15.32:0/414074701 conn(0x7f5bf00b8ee0 msgr2=0x7f5bf00b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:40.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.127+0000 7f5bef59e700 1 -- 172.21.15.32:0/414074701 shutdown_connections 2020-12-05T17:33:40.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.127+0000 7f5bef59e700 1 -- 172.21.15.32:0/414074701 wait complete. 2020-12-05T17:33:40.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.127+0000 7f5bef59e700 1 Processor -- start 2020-12-05T17:33:40.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.127+0000 7f5bef59e700 1 -- start start 2020-12-05T17:33:40.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5bef59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5bf005f3b0 0x7f5bf01481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:40.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5bef59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf005feb0 0x7f5bf0064cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:40.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5bef59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5bf0062400 0x7f5bf00651a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:40.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5bef59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf0063550 con 0x7f5bf005feb0 2020-12-05T17:33:40.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5bedd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf005feb0 0x7f5bf0064cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:40.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5bef59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf00637f0 con 0x7f5bf0062400 2020-12-05T17:33:40.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5bedd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf005feb0 0x7f5bf0064cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34404/0 (socket says 172.21.15.32:34404) 2020-12-05T17:33:40.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5bedd9b700 1 -- 172.21.15.32:0/2327636989 learned_addr learned my addr 172.21.15.32:0/2327636989 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:40.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5bef59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf0064810 con 0x7f5bf005f3b0 2020-12-05T17:33:40.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5bee59c700 1 --2- 172.21.15.32:0/2327636989 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5bf005f3b0 0x7f5bf01481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:40.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5beed9d700 1 --2- 172.21.15.32:0/2327636989 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5bf0062400 0x7f5bf00651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:40.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5bedd9b700 1 -- 172.21.15.32:0/2327636989 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5bf005f3b0 msgr2=0x7f5bf01481a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:40.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5bedd9b700 1 --2- 172.21.15.32:0/2327636989 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5bf005f3b0 0x7f5bf01481a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:40.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5bedd9b700 1 -- 172.21.15.32:0/2327636989 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5bf0062400 msgr2=0x7f5bf00651a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:40.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5bedd9b700 1 --2- 172.21.15.32:0/2327636989 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5bf0062400 0x7f5bf00651a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:40.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5bedd9b700 1 -- 172.21.15.32:0/2327636989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf0148df0 con 0x7f5bf005feb0 2020-12-05T17:33:40.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5bedd9b700 1 --2- 172.21.15.32:0/2327636989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf005feb0 0x7f5bf0064cb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f5be8001450 tx=0x7f5be4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:40.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5bdf7fe700 1 -- 172.21.15.32:0/2327636989 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5be4011860 con 0x7f5bf005feb0 2020-12-05T17:33:40.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5bdf7fe700 1 -- 172.21.15.32:0/2327636989 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be40119c0 con 0x7f5bf005feb0 2020-12-05T17:33:40.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5bdf7fe700 1 -- 172.21.15.32:0/2327636989 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5be4011d20 con 0x7f5bf005feb0 2020-12-05T17:33:40.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5bef59e700 1 -- 172.21.15.32:0/2327636989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf0146090 con 0x7f5bf005feb0 2020-12-05T17:33:40.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5bef59e700 1 -- 172.21.15.32:0/2327636989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf00631e0 con 0x7f5bf005feb0 2020-12-05T17:33:40.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5bef59e700 1 -- 172.21.15.32:0/2327636989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf00649d0 con 0x7f5bf005feb0 2020-12-05T17:33:40.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.131+0000 7f5bdf7fe700 1 -- 172.21.15.32:0/2327636989 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5be4024030 con 0x7f5bf005feb0 2020-12-05T17:33:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.135+0000 7f5bdf7fe700 1 --2- 172.21.15.32:0/2327636989 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5bcc055440 0x7f5bcc057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.135+0000 7f5bee59c700 1 --2- 172.21.15.32:0/2327636989 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5bcc055440 0x7f5bcc057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.135+0000 7f5bdf7fe700 1 -- 172.21.15.32:0/2327636989 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f5be406ed00 con 0x7f5bf005feb0 2020-12-05T17:33:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.135+0000 7f5bee59c700 1 --2- 172.21.15.32:0/2327636989 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5bcc055440 0x7f5bcc057900 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f5be000d000 tx=0x7f5be0009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:40.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.143+0000 7f5bdf7fe700 1 -- 172.21.15.32:0/2327636989 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5be4072030 con 0x7f5bf005feb0 2020-12-05T17:33:40.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:40.303+0000 7f5bef59e700 1 -- 172.21.15.32:0/2327636989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f5bf0057050 con 0x7f5bf005feb0 2020-12-05T17:33:41.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.115+0000 7f5bdf7fe700 1 -- 172.21.15.32:0/2327636989 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v39) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5be4022070 con 0x7f5bf005feb0 2020-12-05T17:33:41.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.115+0000 7f5bef59e700 1 -- 172.21.15.32:0/2327636989 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5bcc055440 msgr2=0x7f5bcc057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:41.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.115+0000 7f5bef59e700 1 --2- 172.21.15.32:0/2327636989 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5bcc055440 0x7f5bcc057900 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f5be000d000 tx=0x7f5be0009040).stop 2020-12-05T17:33:41.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.115+0000 7f5bef59e700 1 -- 172.21.15.32:0/2327636989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf005feb0 msgr2=0x7f5bf0064cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:41.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.115+0000 7f5bef59e700 1 --2- 172.21.15.32:0/2327636989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf005feb0 0x7f5bf0064cb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f5be8001450 tx=0x7f5be4015000).stop 2020-12-05T17:33:41.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.115+0000 7f5bef59e700 1 -- 172.21.15.32:0/2327636989 shutdown_connections 2020-12-05T17:33:41.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.115+0000 7f5bef59e700 1 --2- 172.21.15.32:0/2327636989 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5bf0062400 0x7f5bf00651a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.115+0000 7f5bef59e700 1 --2- 172.21.15.32:0/2327636989 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5bcc055440 0x7f5bcc057900 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.115+0000 7f5bef59e700 1 --2- 172.21.15.32:0/2327636989 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5bf005f3b0 0x7f5bf01481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.115+0000 7f5bef59e700 1 --2- 172.21.15.32:0/2327636989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf005feb0 0x7f5bf0064cb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.115+0000 7f5bef59e700 1 -- 172.21.15.32:0/2327636989 >> 172.21.15.32:0/2327636989 conn(0x7f5bf00b8ee0 msgr2=0x7f5bf00b8290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:41.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.115+0000 7f5bef59e700 1 -- 172.21.15.32:0/2327636989 shutdown_connections 2020-12-05T17:33:41.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.119+0000 7f5bef59e700 1 -- 172.21.15.32:0/2327636989 wait complete. 2020-12-05T17:33:41.121 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2020-12-05T17:33:41.136 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2020-12-05T17:33:41.137 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-12-05T17:33:41.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.251+0000 7f924bfff700 1 Processor -- start 2020-12-05T17:33:41.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.251+0000 7f924bfff700 1 -- start start 2020-12-05T17:33:41.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.251+0000 7f924bfff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f924c0abeb0 0x7f924c0ab610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:41.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.251+0000 7f924bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f924c0ae6e0 0x7f924c13dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:41.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.251+0000 7f924bfff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f924c0ac290 con 0x7f924c062660 2020-12-05T17:33:41.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.251+0000 7f924bfff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f924c0ad8a0 con 0x7f924c0abeb0 2020-12-05T17:33:41.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.251+0000 7f924bfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924c0b0930 con 0x7f924c0ae6e0 2020-12-05T17:33:41.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.255+0000 7f924affd700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f924c0abeb0 0x7f924c0ab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:41.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.255+0000 7f924b7fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f924c0ae6e0 0x7f924c13dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:41.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.255+0000 7f924affd700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f924c0abeb0 0x7f924c0ab610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:54304/0 (socket says 172.21.15.32:54304) 2020-12-05T17:33:41.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.255+0000 7f92427fc700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f924c062660 0x7f924c0ad190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:49704/0 (socket says 172.21.15.32:49704) 2020-12-05T17:33:41.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.255+0000 7f924affd700 1 -- 172.21.15.32:0/948482143 learned_addr learned my addr 172.21.15.32:0/948482143 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:41.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.255+0000 7f924affd700 1 -- 172.21.15.32:0/948482143 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f924c0ae6e0 msgr2=0x7f924c13dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:41.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.255+0000 7f924affd700 1 --2- 172.21.15.32:0/948482143 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f924c0ae6e0 0x7f924c13dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.255+0000 7f924affd700 1 -- 172.21.15.32:0/948482143 >> v1:172.21.15.61:6789/0 conn(0x7f924c062660 legacy=0x7f924c0ad190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:41.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.255+0000 7f924a7fc700 1 -- 172.21.15.32:0/948482143 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2636215585 0 0) 0x7f924c0ac290 con 0x7f924c062660 2020-12-05T17:33:41.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.255+0000 7f924affd700 1 -- 172.21.15.32:0/948482143 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c0b0bd0 con 0x7f924c0abeb0 2020-12-05T17:33:41.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.255+0000 7f924affd700 1 --2- 172.21.15.32:0/948482143 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f924c0abeb0 0x7f924c0ab610 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f923c002f30 tx=0x7f923c006f10).ready entity=mon.1 client_cookie=ede71055822a3f61 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:41.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.255+0000 7f924a7fc700 1 -- 172.21.15.32:0/948482143 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f923c00cb70 con 0x7f924c0abeb0 2020-12-05T17:33:41.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.255+0000 7f924a7fc700 1 -- 172.21.15.32:0/948482143 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923c00ccd0 con 0x7f924c0abeb0 2020-12-05T17:33:41.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.255+0000 7f924a7fc700 1 -- 172.21.15.32:0/948482143 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f923c010600 con 0x7f924c0abeb0 2020-12-05T17:33:41.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.255+0000 7f924bfff700 1 -- 172.21.15.32:0/948482143 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f924c0abeb0 msgr2=0x7f924c0ab610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:41.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.255+0000 7f924bfff700 1 --2- 172.21.15.32:0/948482143 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f924c0abeb0 0x7f924c0ab610 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f923c002f30 tx=0x7f923c006f10).stop 2020-12-05T17:33:41.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.255+0000 7f924bfff700 1 -- 172.21.15.32:0/948482143 shutdown_connections 2020-12-05T17:33:41.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.255+0000 7f924bfff700 1 --2- 172.21.15.32:0/948482143 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f924c0ae6e0 0x7f924c13dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.255+0000 7f924bfff700 1 --2- 172.21.15.32:0/948482143 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f924c0abeb0 0x7f924c0ab610 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.255+0000 7f924bfff700 1 -- 172.21.15.32:0/948482143 >> 172.21.15.32:0/948482143 conn(0x7f924c0b5530 msgr2=0x7f924c0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:41.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.255+0000 7f924bfff700 1 -- 172.21.15.32:0/948482143 shutdown_connections 2020-12-05T17:33:41.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.255+0000 7f924bfff700 1 -- 172.21.15.32:0/948482143 wait complete. 2020-12-05T17:33:41.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.255+0000 7f924bfff700 1 Processor -- start 2020-12-05T17:33:41.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f924bfff700 1 -- start start 2020-12-05T17:33:41.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f924bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f924c062660 0x7f924c13f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:41.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f924bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f924c0abeb0 0x7f924c065930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:41.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f924bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f924c0ae6e0 0x7f924c065e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:41.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f924bfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c0644d0 con 0x7f924c0abeb0 2020-12-05T17:33:41.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f924bfff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c063af0 con 0x7f924c062660 2020-12-05T17:33:41.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f924bfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924c063d90 con 0x7f924c0ae6e0 2020-12-05T17:33:41.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f92427fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f924c0abeb0 0x7f924c065930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:41.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f92427fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f924c0abeb0 0x7f924c065930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34420/0 (socket says 172.21.15.32:34420) 2020-12-05T17:33:41.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f92427fc700 1 -- 172.21.15.32:0/3750665961 learned_addr learned my addr 172.21.15.32:0/3750665961 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:41.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f924b7fe700 1 --2- 172.21.15.32:0/3750665961 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f924c0ae6e0 0x7f924c065e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:41.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f924affd700 1 --2- 172.21.15.32:0/3750665961 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f924c062660 0x7f924c13f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:41.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f92427fc700 1 -- 172.21.15.32:0/3750665961 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f924c0ae6e0 msgr2=0x7f924c065e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:41.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f92427fc700 1 --2- 172.21.15.32:0/3750665961 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f924c0ae6e0 0x7f924c065e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f92427fc700 1 -- 172.21.15.32:0/3750665961 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f924c062660 msgr2=0x7f924c13f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:41.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f92427fc700 1 --2- 172.21.15.32:0/3750665961 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f924c062660 0x7f924c13f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f92427fc700 1 -- 172.21.15.32:0/3750665961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c064030 con 0x7f924c0abeb0 2020-12-05T17:33:41.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f924affd700 1 --2- 172.21.15.32:0/3750665961 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f924c062660 0x7f924c13f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:33:41.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f92427fc700 1 --2- 172.21.15.32:0/3750665961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f924c0abeb0 0x7f924c065930 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f9234000b80 tx=0x7f9234007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:41.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f9248ff9700 1 -- 172.21.15.32:0/3750665961 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92340072e0 con 0x7f924c0abeb0 2020-12-05T17:33:41.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f9248ff9700 1 -- 172.21.15.32:0/3750665961 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9234005520 con 0x7f924c0abeb0 2020-12-05T17:33:41.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f924bfff700 1 -- 172.21.15.32:0/3750665961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f924c063770 con 0x7f924c0abeb0 2020-12-05T17:33:41.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f9248ff9700 1 -- 172.21.15.32:0/3750665961 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9234005880 con 0x7f924c0abeb0 2020-12-05T17:33:41.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f924bfff700 1 -- 172.21.15.32:0/3750665961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f924c13fd00 con 0x7f924c0abeb0 2020-12-05T17:33:41.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f9248ff9700 1 -- 172.21.15.32:0/3750665961 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f923401a060 con 0x7f924c0abeb0 2020-12-05T17:33:41.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.259+0000 7f924bfff700 1 -- 172.21.15.32:0/3750665961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f924c057050 con 0x7f924c0abeb0 2020-12-05T17:33:41.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.263+0000 7f9248ff9700 1 --2- 172.21.15.32:0/3750665961 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9224055440 0x7f9224057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:41.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.263+0000 7f924affd700 1 --2- 172.21.15.32:0/3750665961 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9224055440 0x7f9224057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:41.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.263+0000 7f9248ff9700 1 -- 172.21.15.32:0/3750665961 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f923406d6b0 con 0x7f924c0abeb0 2020-12-05T17:33:41.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.263+0000 7f924affd700 1 --2- 172.21.15.32:0/3750665961 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9224055440 0x7f9224057900 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f923c019000 tx=0x7f924c061200).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:41.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.267+0000 7f9248ff9700 1 -- 172.21.15.32:0/3750665961 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9234043b30 con 0x7f924c0abeb0 2020-12-05T17:33:41.462 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.459+0000 7f924bfff700 1 -- 172.21.15.32:0/3750665961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f924c069ee0 con 0x7f924c0abeb0 2020-12-05T17:33:41.462 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.463+0000 7f9248ff9700 1 -- 172.21.15.32:0/3750665961 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f9234018070 con 0x7f924c0abeb0 2020-12-05T17:33:41.463 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:33:41.463 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:33:41.463 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2020-12-05T17:33:41.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:41.464 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2020-12-05T17:33:41.464 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfpb/aExAAdMKW4yIgC2zhkzyW6l8xug==", 2020-12-05T17:33:41.464 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:33:41.464 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:33:41.465 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:33:41.465 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:33:41.465 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:33:41.465 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:41.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:41.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:41.466 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2020-12-05T17:33:41.466 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfBKTYGRAAaPkezwmTiAvkUfTuDGDNZA==", 2020-12-05T17:33:41.466 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:33:41.467 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:33:41.467 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:33:41.467 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:33:41.467 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:33:41.467 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:41.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:41.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:41.468 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2020-12-05T17:33:41.468 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHw8tflo5EKhAAwV9D64fAoY3uvXCm+Aksag==", 2020-12-05T17:33:41.469 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:33:41.469 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:33:41.469 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:33:41.469 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:33:41.469 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:41.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:41.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:41.470 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2020-12-05T17:33:41.470 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDKw8tfTjEiERAAkW6mkgApoXwyF49kPjZBGg==", 2020-12-05T17:33:41.470 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:33:41.471 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:33:41.471 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:33:41.471 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:33:41.471 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:41.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:41.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:41.472 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2020-12-05T17:33:41.472 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDMw8tfLEzBMxAAiJ2NifsOLEkt1e3vmhwSTA==", 2020-12-05T17:33:41.472 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:33:41.472 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:33:41.473 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:33:41.473 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:33:41.473 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:41.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:41.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:41.474 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2020-12-05T17:33:41.474 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPw8tfV+vnGxAAQHvXJRW5X9Vhe/eYJBJ/uQ==", 2020-12-05T17:33:41.474 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:33:41.474 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:33:41.474 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:33:41.475 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:33:41.475 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:41.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:41.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:41.475 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2020-12-05T17:33:41.476 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUw8tfN8wMJxAApc5LY9HGDeIV/o0eil/g4A==", 2020-12-05T17:33:41.476 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:33:41.476 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:33:41.476 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:33:41.476 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:33:41.477 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:41.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:41.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:41.477 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2020-12-05T17:33:41.478 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXw8tfvZ5dDxAAtAfftExkljlBvn7ZSoD/Og==", 2020-12-05T17:33:41.478 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:33:41.478 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:33:41.478 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:33:41.478 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:33:41.479 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:41.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:41.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:41.479 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2020-12-05T17:33:41.479 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDZw8tfhXFzMhAAZXNwNGfKv360Tq4N/DX5Mw==", 2020-12-05T17:33:41.480 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:33:41.480 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:33:41.480 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:33:41.480 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:33:41.481 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:41.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:41.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:41.481 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2020-12-05T17:33:41.481 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDcw8tfROfSGhAA/Y2Y1b4k5zQju0MM7LASeQ==", 2020-12-05T17:33:41.482 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:33:41.482 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:33:41.482 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:33:41.482 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:33:41.482 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:41.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:41.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:41.483 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2020-12-05T17:33:41.483 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfWDp/HxAAmvLPj7C3WDgj0OC2o7lBJQ==", 2020-12-05T17:33:41.484 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:33:41.484 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:33:41.484 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:33:41.484 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:33:41.484 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:41.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:41.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:41.485 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2020-12-05T17:33:41.485 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfugnmIRAAW6+/QGMLkHw+Z97RHpV4kw==", 2020-12-05T17:33:41.485 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:33:41.486 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:33:41.486 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:33:41.486 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:33:41.486 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:41.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:41.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:41.487 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2020-12-05T17:33:41.487 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfojyqJBAAyta+dSy2yGnC1kKe3YOAYw==", 2020-12-05T17:33:41.487 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:33:41.487 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:33:41.488 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:33:41.488 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:33:41.488 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:41.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:41.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:41.489 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2020-12-05T17:33:41.489 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDFw8tfES0PLBAAA1cUV44snAUDF2TyAmXImw==", 2020-12-05T17:33:41.489 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:33:41.489 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:33:41.489 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:33:41.490 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:33:41.490 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:33:41.490 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:41.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:41.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:41.491 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2020-12-05T17:33:41.491 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfDr8mERAAS7bpAQXgW78W8GDq1v1zaA==", 2020-12-05T17:33:41.491 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:33:41.491 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:33:41.491 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:33:41.492 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:33:41.492 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:41.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:41.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:41.493 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2020-12-05T17:33:41.493 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tf3HmcDhAATNKHhH9zgasedaNEzgG0OQ==", 2020-12-05T17:33:41.493 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:33:41.493 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:33:41.493 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:33:41.494 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:33:41.494 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:41.494 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:41.494 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:33:41.494 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:33:41.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.463+0000 7f924bfff700 1 -- 172.21.15.32:0/3750665961 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9224055440 msgr2=0x7f9224057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:41.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.463+0000 7f924bfff700 1 --2- 172.21.15.32:0/3750665961 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9224055440 0x7f9224057900 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f923c019000 tx=0x7f924c061200).stop 2020-12-05T17:33:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.463+0000 7f924bfff700 1 -- 172.21.15.32:0/3750665961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f924c0abeb0 msgr2=0x7f924c065930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.463+0000 7f924bfff700 1 --2- 172.21.15.32:0/3750665961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f924c0abeb0 0x7f924c065930 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f9234000b80 tx=0x7f9234007aa0).stop 2020-12-05T17:33:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.463+0000 7f924bfff700 1 -- 172.21.15.32:0/3750665961 shutdown_connections 2020-12-05T17:33:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.463+0000 7f924bfff700 1 --2- 172.21.15.32:0/3750665961 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9224055440 0x7f9224057900 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.463+0000 7f924bfff700 1 --2- 172.21.15.32:0/3750665961 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f924c0ae6e0 0x7f924c065e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.463+0000 7f924bfff700 1 --2- 172.21.15.32:0/3750665961 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f924c062660 0x7f924c13f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.463+0000 7f924bfff700 1 --2- 172.21.15.32:0/3750665961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f924c0abeb0 0x7f924c065930 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.463+0000 7f924bfff700 1 -- 172.21.15.32:0/3750665961 >> 172.21.15.32:0/3750665961 conn(0x7f924c0b5530 msgr2=0x7f924c0b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.463+0000 7f924bfff700 1 -- 172.21.15.32:0/3750665961 shutdown_connections 2020-12-05T17:33:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.463+0000 7f924bfff700 1 -- 172.21.15.32:0/3750665961 wait complete. 2020-12-05T17:33:41.499 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:33:41.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.611+0000 7f409b59e700 1 Processor -- start 2020-12-05T17:33:41.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.611+0000 7f409b59e700 1 -- start start 2020-12-05T17:33:41.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.611+0000 7f409b59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f409c05f6f0 0x7f409c0b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:41.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.611+0000 7f409b59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409c05fad0 con 0x7f409c05feb0 2020-12-05T17:33:41.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.611+0000 7f409b59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409c062670 con 0x7f409c062210 2020-12-05T17:33:41.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.611+0000 7f409b59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409c0b3360 con 0x7f409c05f6f0 2020-12-05T17:33:41.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.611+0000 7f4093fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f409c05feb0 0x7f409c061b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37076/0 (socket says 172.21.15.32:37076) 2020-12-05T17:33:41.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.611+0000 7f4093fff700 1 -- 172.21.15.32:0/3106869052 learned_addr learned my addr 172.21.15.32:0/3106869052 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:33:41.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.611+0000 7f409a59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f409c05f6f0 0x7f409c0b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:41.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.611+0000 7f4099d9b700 1 -- 172.21.15.32:0/3106869052 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3267461225 0 0) 0x7f409c05fad0 con 0x7f409c05feb0 2020-12-05T17:33:41.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.611+0000 7f4099d9b700 1 -- 172.21.15.32:0/3106869052 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f407c003230 con 0x7f409c05feb0 2020-12-05T17:33:41.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.611+0000 7f4099d9b700 1 -- 172.21.15.32:0/3106869052 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972300300 0 0) 0x7f409c062670 con 0x7f409c062210 2020-12-05T17:33:41.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.611+0000 7f4099d9b700 1 -- 172.21.15.32:0/3106869052 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f409c05fad0 con 0x7f409c062210 2020-12-05T17:33:41.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.611+0000 7f409a59c700 1 -- 172.21.15.32:0/3106869052 >> v1:172.21.15.61:6789/0 conn(0x7f409c062210 legacy=0x7f409c146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:41.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.611+0000 7f409a59c700 1 -- 172.21.15.32:0/3106869052 >> v1:172.21.15.32:6789/0 conn(0x7f409c05feb0 legacy=0x7f409c061b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:41.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.611+0000 7f409a59c700 1 -- 172.21.15.32:0/3106869052 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c0b4cc0 con 0x7f409c05f6f0 2020-12-05T17:33:41.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.611+0000 7f409a59c700 1 --2- 172.21.15.32:0/3106869052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f409c05f6f0 0x7f409c0b15a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f4094001490 tx=0x7f408c006dc0).ready entity=mon.0 client_cookie=e5f791c2a92a67bf server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:41.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.611+0000 7f4099d9b700 1 -- 172.21.15.32:0/3106869052 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f408c009690 con 0x7f409c05f6f0 2020-12-05T17:33:41.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.611+0000 7f4099d9b700 1 -- 172.21.15.32:0/3106869052 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408c0097f0 con 0x7f409c05f6f0 2020-12-05T17:33:41.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.611+0000 7f4099d9b700 1 -- 172.21.15.32:0/3106869052 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f408c009b50 con 0x7f409c05f6f0 2020-12-05T17:33:41.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.611+0000 7f409b59e700 1 -- 172.21.15.32:0/3106869052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f409c05f6f0 msgr2=0x7f409c0b15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:41.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.611+0000 7f409b59e700 1 --2- 172.21.15.32:0/3106869052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f409c05f6f0 0x7f409c0b15a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f4094001490 tx=0x7f408c006dc0).stop 2020-12-05T17:33:41.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.615+0000 7f409b59e700 1 -- 172.21.15.32:0/3106869052 shutdown_connections 2020-12-05T17:33:41.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.615+0000 7f409b59e700 1 --2- 172.21.15.32:0/3106869052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f409c05f6f0 0x7f409c0b15a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.615+0000 7f409b59e700 1 -- 172.21.15.32:0/3106869052 >> 172.21.15.32:0/3106869052 conn(0x7f409c0b8ee0 msgr2=0x7f409c0b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:41.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.615+0000 7f409b59e700 1 -- 172.21.15.32:0/3106869052 shutdown_connections 2020-12-05T17:33:41.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.615+0000 7f409b59e700 1 -- 172.21.15.32:0/3106869052 wait complete. 2020-12-05T17:33:41.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.615+0000 7f409b59e700 1 Processor -- start 2020-12-05T17:33:41.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.615+0000 7f409b59e700 1 -- start start 2020-12-05T17:33:41.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.615+0000 7f409b59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f409c05f6f0 0x7f409c13ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:41.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.615+0000 7f409b59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f409c05feb0 0x7f409c1455c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:41.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.615+0000 7f409b59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f409c062210 0x7f409c142940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:41.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.615+0000 7f409b59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c146250 con 0x7f409c05f6f0 2020-12-05T17:33:41.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.615+0000 7f409b59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c156040 con 0x7f409c062210 2020-12-05T17:33:41.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.615+0000 7f409a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f409c05f6f0 0x7f409c13ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:41.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.615+0000 7f409b59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409c145fb0 con 0x7f409c05feb0 2020-12-05T17:33:41.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.615+0000 7f409a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f409c05f6f0 0x7f409c13ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34432/0 (socket says 172.21.15.32:34432) 2020-12-05T17:33:41.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.615+0000 7f409a59c700 1 -- 172.21.15.32:0/1690974751 learned_addr learned my addr 172.21.15.32:0/1690974751 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:41.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.619+0000 7f4093fff700 1 --2- 172.21.15.32:0/1690974751 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f409c05feb0 0x7f409c1455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:41.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.619+0000 7f409ad9d700 1 --2- 172.21.15.32:0/1690974751 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f409c062210 0x7f409c142940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:41.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.619+0000 7f409a59c700 1 -- 172.21.15.32:0/1690974751 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f409c05feb0 msgr2=0x7f409c1455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:41.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.619+0000 7f409a59c700 1 --2- 172.21.15.32:0/1690974751 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f409c05feb0 0x7f409c1455c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.619+0000 7f409a59c700 1 -- 172.21.15.32:0/1690974751 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f409c062210 msgr2=0x7f409c142940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:41.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.619+0000 7f409a59c700 1 --2- 172.21.15.32:0/1690974751 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f409c062210 0x7f409c142940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.619+0000 7f409a59c700 1 -- 172.21.15.32:0/1690974751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c1562d0 con 0x7f409c05f6f0 2020-12-05T17:33:41.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.619+0000 7f409ad9d700 1 --2- 172.21.15.32:0/1690974751 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f409c062210 0x7f409c142940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:33:41.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.619+0000 7f409a59c700 1 --2- 172.21.15.32:0/1690974751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f409c05f6f0 0x7f409c13ed10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f4094001490 tx=0x7f408c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:41.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.619+0000 7f40937fe700 1 -- 172.21.15.32:0/1690974751 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f408c011ca0 con 0x7f409c05f6f0 2020-12-05T17:33:41.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.619+0000 7f40937fe700 1 -- 172.21.15.32:0/1690974751 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408c011e00 con 0x7f409c05f6f0 2020-12-05T17:33:41.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.619+0000 7f409b59e700 1 -- 172.21.15.32:0/1690974751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409c156770 con 0x7f409c05f6f0 2020-12-05T17:33:41.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.619+0000 7f40937fe700 1 -- 172.21.15.32:0/1690974751 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f408c01d640 con 0x7f409c05f6f0 2020-12-05T17:33:41.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.619+0000 7f409b59e700 1 -- 172.21.15.32:0/1690974751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409c13f420 con 0x7f409c05f6f0 2020-12-05T17:33:41.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.619+0000 7f409b59e700 1 -- 172.21.15.32:0/1690974751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f409c145df0 con 0x7f409c05f6f0 2020-12-05T17:33:41.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.619+0000 7f40937fe700 1 -- 172.21.15.32:0/1690974751 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f408c00f070 con 0x7f409c05f6f0 2020-12-05T17:33:41.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.619+0000 7f40937fe700 1 --2- 172.21.15.32:0/1690974751 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f40800553b0 0x7f4080057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:41.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.619+0000 7f40937fe700 1 -- 172.21.15.32:0/1690974751 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f408c06f820 con 0x7f409c05f6f0 2020-12-05T17:33:41.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.619+0000 7f4093fff700 1 --2- 172.21.15.32:0/1690974751 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f40800553b0 0x7f4080057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:41.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.623+0000 7f4093fff700 1 --2- 172.21.15.32:0/1690974751 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f40800553b0 0x7f4080057870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f408400b000 tx=0x7f4084009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:41.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.627+0000 7f40937fe700 1 -- 172.21.15.32:0/1690974751 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f408c045c60 con 0x7f409c05f6f0 2020-12-05T17:33:41.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.767+0000 7f409b59e700 1 -- 172.21.15.32:0/1690974751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f409c057050 con 0x7f409c05f6f0 2020-12-05T17:33:41.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.771+0000 7f40937fe700 1 -- 172.21.15.32:0/1690974751 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+13414 (secure 0 0 0) 0x7f408c049280 con 0x7f409c05f6f0 2020-12-05T17:33:41.771 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:33:41.771 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":39,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:33:41.109162+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:33:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.771+0000 7f409b59e700 1 -- 172.21.15.32:0/1690974751 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f40800553b0 msgr2=0x7f4080057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.771+0000 7f409b59e700 1 --2- 172.21.15.32:0/1690974751 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f40800553b0 0x7f4080057870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f408400b000 tx=0x7f4084009040).stop 2020-12-05T17:33:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.771+0000 7f409b59e700 1 -- 172.21.15.32:0/1690974751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f409c05f6f0 msgr2=0x7f409c13ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.771+0000 7f409b59e700 1 --2- 172.21.15.32:0/1690974751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f409c05f6f0 0x7f409c13ed10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f4094001490 tx=0x7f408c015000).stop 2020-12-05T17:33:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.771+0000 7f409b59e700 1 -- 172.21.15.32:0/1690974751 shutdown_connections 2020-12-05T17:33:41.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.771+0000 7f409b59e700 1 --2- 172.21.15.32:0/1690974751 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f409c062210 0x7f409c142940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.771+0000 7f409b59e700 1 --2- 172.21.15.32:0/1690974751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f409c05f6f0 0x7f409c13ed10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.771+0000 7f409b59e700 1 --2- 172.21.15.32:0/1690974751 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f40800553b0 0x7f4080057870 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.771+0000 7f409b59e700 1 --2- 172.21.15.32:0/1690974751 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f409c05feb0 0x7f409c1455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.771+0000 7f409b59e700 1 -- 172.21.15.32:0/1690974751 >> 172.21.15.32:0/1690974751 conn(0x7f409c0b8ee0 msgr2=0x7f409c0b5110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.771+0000 7f409b59e700 1 -- 172.21.15.32:0/1690974751 shutdown_connections 2020-12-05T17:33:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.771+0000 7f409b59e700 1 -- 172.21.15.32:0/1690974751 wait complete. 2020-12-05T17:33:41.785 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-12-05T17:33:41.785 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:33:41.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.911+0000 7f2773050700 1 Processor -- start 2020-12-05T17:33:41.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.911+0000 7f2773050700 1 -- start start 2020-12-05T17:33:41.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.911+0000 7f2773050700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f276c0b1e60 0x7f276c0af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:41.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.911+0000 7f2773050700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f276c062ac0 0x7f276c138fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:41.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.911+0000 7f2773050700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f276c0aedf0 0x7f276c1394a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:41.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.911+0000 7f2773050700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f276c0ae0a0 con 0x7f276c0aedf0 2020-12-05T17:33:41.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.911+0000 7f2773050700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f276c139990 con 0x7f276c062ac0 2020-12-05T17:33:41.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.911+0000 7f2773050700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f276c143390 con 0x7f276c0b1e60 2020-12-05T17:33:41.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.911+0000 7f277284f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f276c0aedf0 0x7f276c1394a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:41.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.911+0000 7f277284f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f276c0aedf0 0x7f276c1394a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34444/0 (socket says 172.21.15.32:34444) 2020-12-05T17:33:41.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.911+0000 7f277184d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f276c062ac0 0x7f276c138fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:41.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.911+0000 7f277284f700 1 -- 172.21.15.32:0/2978752731 learned_addr learned my addr 172.21.15.32:0/2978752731 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:41.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.911+0000 7f277204e700 1 --2- 172.21.15.32:0/2978752731 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f276c0b1e60 0x7f276c0af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:41.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.911+0000 7f277284f700 1 -- 172.21.15.32:0/2978752731 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f276c0b1e60 msgr2=0x7f276c0af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:41.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.911+0000 7f277284f700 1 --2- 172.21.15.32:0/2978752731 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f276c0b1e60 0x7f276c0af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.911+0000 7f277284f700 1 -- 172.21.15.32:0/2978752731 >> v2:172.21.15.61:3300/0 conn(0x7f276c062ac0 msgr2=0x7f276c138fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:41.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.911+0000 7f277284f700 1 --2- 172.21.15.32:0/2978752731 >> v2:172.21.15.61:3300/0 conn(0x7f276c062ac0 0x7f276c138fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.911+0000 7f277284f700 1 -- 172.21.15.32:0/2978752731 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276c143610 con 0x7f276c0aedf0 2020-12-05T17:33:41.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.911+0000 7f277204e700 1 --2- 172.21.15.32:0/2978752731 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f276c0b1e60 0x7f276c0af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:33:41.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.915+0000 7f277284f700 1 --2- 172.21.15.32:0/2978752731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276c0aedf0 0x7f276c1394a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f27640032b0 tx=0x7f2764009270).ready entity=mon.0 client_cookie=fbffd0653f9a0277 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:41.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.915+0000 7f277104c700 1 -- 172.21.15.32:0/2978752731 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2764009e60 con 0x7f276c0aedf0 2020-12-05T17:33:41.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.915+0000 7f277104c700 1 -- 172.21.15.32:0/2978752731 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276400bc00 con 0x7f276c0aedf0 2020-12-05T17:33:41.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.915+0000 7f2773050700 1 -- 172.21.15.32:0/2978752731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276c0aedf0 msgr2=0x7f276c1394a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:41.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.915+0000 7f277104c700 1 -- 172.21.15.32:0/2978752731 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f276400bd70 con 0x7f276c0aedf0 2020-12-05T17:33:41.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.915+0000 7f2773050700 1 --2- 172.21.15.32:0/2978752731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276c0aedf0 0x7f276c1394a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f27640032b0 tx=0x7f2764009270).stop 2020-12-05T17:33:41.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.915+0000 7f2773050700 1 -- 172.21.15.32:0/2978752731 shutdown_connections 2020-12-05T17:33:41.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.915+0000 7f2773050700 1 --2- 172.21.15.32:0/2978752731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276c0aedf0 0x7f276c1394a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.915+0000 7f2773050700 1 --2- 172.21.15.32:0/2978752731 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f276c0b1e60 0x7f276c0af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.915+0000 7f2773050700 1 --2- 172.21.15.32:0/2978752731 >> v2:172.21.15.61:3300/0 conn(0x7f276c062ac0 0x7f276c138fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.915+0000 7f2773050700 1 -- 172.21.15.32:0/2978752731 >> 172.21.15.32:0/2978752731 conn(0x7f276c0b5620 msgr2=0x7f276c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:41.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.915+0000 7f2773050700 1 -- 172.21.15.32:0/2978752731 shutdown_connections 2020-12-05T17:33:41.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.915+0000 7f2773050700 1 -- 172.21.15.32:0/2978752731 wait complete. 2020-12-05T17:33:41.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.915+0000 7f2773050700 1 Processor -- start 2020-12-05T17:33:41.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.915+0000 7f2773050700 1 -- start start 2020-12-05T17:33:41.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f2773050700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f276c062ac0 0x7f276c13a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:41.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f2773050700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f276c0aedf0 0x7f276c0633c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f2773050700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276c0b1e60 0x7f276c1412a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f2773050700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c13f500 con 0x7f276c0b1e60 2020-12-05T17:33:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f2773050700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c12fa40 con 0x7f276c062ac0 2020-12-05T17:33:41.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f2773050700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f276c12fce0 con 0x7f276c0aedf0 2020-12-05T17:33:41.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f277284f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276c0b1e60 0x7f276c1412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:41.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f277284f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276c0b1e60 0x7f276c1412a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34450/0 (socket says 172.21.15.32:34450) 2020-12-05T17:33:41.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f277284f700 1 -- 172.21.15.32:0/1674552300 learned_addr learned my addr 172.21.15.32:0/1674552300 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:41.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f277204e700 1 --2- 172.21.15.32:0/1674552300 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f276c062ac0 0x7f276c13a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:41.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f277184d700 1 --2- 172.21.15.32:0/1674552300 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f276c0aedf0 0x7f276c0633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:41.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f277284f700 1 -- 172.21.15.32:0/1674552300 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f276c0aedf0 msgr2=0x7f276c0633c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:41.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f277284f700 1 --2- 172.21.15.32:0/1674552300 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f276c0aedf0 0x7f276c0633c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f277284f700 1 -- 172.21.15.32:0/1674552300 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f276c062ac0 msgr2=0x7f276c13a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:41.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f277284f700 1 --2- 172.21.15.32:0/1674552300 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f276c062ac0 0x7f276c13a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:41.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f277284f700 1 -- 172.21.15.32:0/1674552300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276c12f140 con 0x7f276c0b1e60 2020-12-05T17:33:41.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f277284f700 1 --2- 172.21.15.32:0/1674552300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276c0b1e60 0x7f276c1412a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f27640032b0 tx=0x7f2764016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:41.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f275b7fe700 1 -- 172.21.15.32:0/1674552300 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27640107a0 con 0x7f276c0b1e60 2020-12-05T17:33:41.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f275b7fe700 1 -- 172.21.15.32:0/1674552300 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276400bed0 con 0x7f276c0b1e60 2020-12-05T17:33:41.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f2773050700 1 -- 172.21.15.32:0/1674552300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f276c139b10 con 0x7f276c0b1e60 2020-12-05T17:33:41.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f275b7fe700 1 -- 172.21.15.32:0/1674552300 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2764010af0 con 0x7f276c0b1e60 2020-12-05T17:33:41.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f2773050700 1 -- 172.21.15.32:0/1674552300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f276c13a6b0 con 0x7f276c0b1e60 2020-12-05T17:33:41.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f2773050700 1 -- 172.21.15.32:0/1674552300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f276c057050 con 0x7f276c0b1e60 2020-12-05T17:33:41.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.919+0000 7f275b7fe700 1 -- 172.21.15.32:0/1674552300 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f276400f070 con 0x7f276c0b1e60 2020-12-05T17:33:41.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.923+0000 7f275b7fe700 1 --2- 172.21.15.32:0/1674552300 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f274c055330 0x7f274c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:41.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.923+0000 7f277204e700 1 --2- 172.21.15.32:0/1674552300 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f274c055330 0x7f274c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:41.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.923+0000 7f275b7fe700 1 -- 172.21.15.32:0/1674552300 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f276406f580 con 0x7f276c0b1e60 2020-12-05T17:33:41.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.923+0000 7f277204e700 1 --2- 172.21.15.32:0/1674552300 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f274c055330 0x7f274c0577f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f276800c000 tx=0x7f2768008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:41.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:41.931+0000 7f275b7fe700 1 -- 172.21.15.32:0/1674552300 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2764049380 con 0x7f276c0b1e60 2020-12-05T17:33:42.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.071+0000 7f2773050700 1 -- 172.21.15.32:0/1674552300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f276c05aab0 con 0x7f276c0b1e60 2020-12-05T17:33:42.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.075+0000 7f275b7fe700 1 -- 172.21.15.32:0/1674552300 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v39) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2764015020 con 0x7f276c0b1e60 2020-12-05T17:33:42.076 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:33:42.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.075+0000 7f2773050700 1 -- 172.21.15.32:0/1674552300 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f274c055330 msgr2=0x7f274c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:42.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.075+0000 7f2773050700 1 --2- 172.21.15.32:0/1674552300 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f274c055330 0x7f274c0577f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f276800c000 tx=0x7f2768008040).stop 2020-12-05T17:33:42.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.075+0000 7f2773050700 1 -- 172.21.15.32:0/1674552300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276c0b1e60 msgr2=0x7f276c1412a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:42.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.075+0000 7f2773050700 1 --2- 172.21.15.32:0/1674552300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276c0b1e60 0x7f276c1412a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f27640032b0 tx=0x7f2764016000).stop 2020-12-05T17:33:42.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.075+0000 7f2773050700 1 -- 172.21.15.32:0/1674552300 shutdown_connections 2020-12-05T17:33:42.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.075+0000 7f2773050700 1 --2- 172.21.15.32:0/1674552300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276c0b1e60 0x7f276c1412a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.075+0000 7f2773050700 1 --2- 172.21.15.32:0/1674552300 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f276c062ac0 0x7f276c13a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.075+0000 7f2773050700 1 --2- 172.21.15.32:0/1674552300 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f274c055330 0x7f274c0577f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.075+0000 7f2773050700 1 --2- 172.21.15.32:0/1674552300 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f276c0aedf0 0x7f276c0633c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.075+0000 7f2773050700 1 -- 172.21.15.32:0/1674552300 >> 172.21.15.32:0/1674552300 conn(0x7f276c0b5620 msgr2=0x7f276c13f960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:42.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.075+0000 7f2773050700 1 -- 172.21.15.32:0/1674552300 shutdown_connections 2020-12-05T17:33:42.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.075+0000 7f2773050700 1 -- 172.21.15.32:0/1674552300 wait complete. 2020-12-05T17:33:42.096 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:33:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce319b6700 1 Processor -- start 2020-12-05T17:33:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce319b6700 1 -- start start 2020-12-05T17:33:42.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce319b6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fce2c05f3b0 0x7fce2c05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:42.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce319b6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce2c05feb0 0x7fce2c061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:42.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce319b6700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fce2c0b2fb0 0x7fce2c13c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:42.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce319b6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce2c0b2df0 con 0x7fce2c05f3b0 2020-12-05T17:33:42.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce319b6700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce2c0b3c90 con 0x7fce2c0b2fb0 2020-12-05T17:33:42.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce319b6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c0b2b50 con 0x7fce2c05feb0 2020-12-05T17:33:42.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce309b4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fce2c05f3b0 0x7fce2c05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:42.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce309b4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fce2c05f3b0 0x7fce2c05f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34454/0 (socket says 172.21.15.32:34454) 2020-12-05T17:33:42.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce309b4700 1 -- 172.21.15.32:0/1652239630 learned_addr learned my addr 172.21.15.32:0/1652239630 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:42.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce311b5700 1 --2- 172.21.15.32:0/1652239630 >> v2:172.21.15.61:3300/0 conn(0x7fce2c0b2fb0 0x7fce2c13c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:42.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce2bfff700 1 --2- 172.21.15.32:0/1652239630 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce2c05feb0 0x7fce2c061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:42.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce309b4700 1 -- 172.21.15.32:0/1652239630 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce2c05feb0 msgr2=0x7fce2c061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:42.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce309b4700 1 --2- 172.21.15.32:0/1652239630 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce2c05feb0 0x7fce2c061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce309b4700 1 -- 172.21.15.32:0/1652239630 >> v2:172.21.15.61:3300/0 conn(0x7fce2c0b2fb0 msgr2=0x7fce2c13c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:42.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce309b4700 1 --2- 172.21.15.32:0/1652239630 >> v2:172.21.15.61:3300/0 conn(0x7fce2c0b2fb0 0x7fce2c13c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce309b4700 1 -- 172.21.15.32:0/1652239630 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c142820 con 0x7fce2c05f3b0 2020-12-05T17:33:42.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce2bfff700 1 --2- 172.21.15.32:0/1652239630 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce2c05feb0 0x7fce2c061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:33:42.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce309b4700 1 --2- 172.21.15.32:0/1652239630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce2c05f3b0 0x7fce2c05f790 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fce1c002f30 tx=0x7fce1c006e80).ready entity=mon.0 client_cookie=231c763cb6bbca4d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:42.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce2b7fe700 1 -- 172.21.15.32:0/1652239630 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce1c0097f0 con 0x7fce2c05f3b0 2020-12-05T17:33:42.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce2b7fe700 1 -- 172.21.15.32:0/1652239630 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c009950 con 0x7fce2c05f3b0 2020-12-05T17:33:42.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce2b7fe700 1 -- 172.21.15.32:0/1652239630 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce1c009cb0 con 0x7fce2c05f3b0 2020-12-05T17:33:42.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce319b6700 1 -- 172.21.15.32:0/1652239630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce2c05f3b0 msgr2=0x7fce2c05f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:42.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.215+0000 7fce319b6700 1 --2- 172.21.15.32:0/1652239630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce2c05f3b0 0x7fce2c05f790 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fce1c002f30 tx=0x7fce1c006e80).stop 2020-12-05T17:33:42.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce319b6700 1 -- 172.21.15.32:0/1652239630 shutdown_connections 2020-12-05T17:33:42.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce319b6700 1 --2- 172.21.15.32:0/1652239630 >> v2:172.21.15.61:3300/0 conn(0x7fce2c0b2fb0 0x7fce2c13c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce319b6700 1 --2- 172.21.15.32:0/1652239630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce2c05f3b0 0x7fce2c05f790 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce319b6700 1 --2- 172.21.15.32:0/1652239630 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce2c05feb0 0x7fce2c061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce319b6700 1 -- 172.21.15.32:0/1652239630 >> 172.21.15.32:0/1652239630 conn(0x7fce2c0b8f20 msgr2=0x7fce2c0b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:42.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce319b6700 1 -- 172.21.15.32:0/1652239630 shutdown_connections 2020-12-05T17:33:42.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce319b6700 1 -- 172.21.15.32:0/1652239630 wait complete. 2020-12-05T17:33:42.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce319b6700 1 Processor -- start 2020-12-05T17:33:42.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce319b6700 1 -- start start 2020-12-05T17:33:42.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce319b6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce2c05f3b0 0x7fce2c13e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:42.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce319b6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce2c05feb0 0x7fce2c13f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:42.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce319b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce2c0b2fb0 0x7fce2c143b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:42.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce319b6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c1413d0 con 0x7fce2c0b2fb0 2020-12-05T17:33:42.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce319b6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c13f940 con 0x7fce2c05f3b0 2020-12-05T17:33:42.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce319b6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c141130 con 0x7fce2c05feb0 2020-12-05T17:33:42.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce311b5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce2c0b2fb0 0x7fce2c143b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:42.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce311b5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce2c0b2fb0 0x7fce2c143b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34462/0 (socket says 172.21.15.32:34462) 2020-12-05T17:33:42.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce311b5700 1 -- 172.21.15.32:0/936132271 learned_addr learned my addr 172.21.15.32:0/936132271 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:42.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce309b4700 1 --2- 172.21.15.32:0/936132271 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce2c05f3b0 0x7fce2c13e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce2bfff700 1 --2- 172.21.15.32:0/936132271 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce2c05feb0 0x7fce2c13f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce311b5700 1 -- 172.21.15.32:0/936132271 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce2c05feb0 msgr2=0x7fce2c13f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce311b5700 1 --2- 172.21.15.32:0/936132271 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce2c05feb0 0x7fce2c13f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce311b5700 1 -- 172.21.15.32:0/936132271 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce2c05f3b0 msgr2=0x7fce2c13e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:42.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce311b5700 1 --2- 172.21.15.32:0/936132271 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce2c05f3b0 0x7fce2c13e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce311b5700 1 -- 172.21.15.32:0/936132271 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c1465e0 con 0x7fce2c0b2fb0 2020-12-05T17:33:42.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce311b5700 1 --2- 172.21.15.32:0/936132271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce2c0b2fb0 0x7fce2c143b70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fce2400a000 tx=0x7fce24008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:42.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce29ffb700 1 -- 172.21.15.32:0/936132271 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce2400b9a0 con 0x7fce2c0b2fb0 2020-12-05T17:33:42.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce29ffb700 1 -- 172.21.15.32:0/936132271 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2400bb00 con 0x7fce2c0b2fb0 2020-12-05T17:33:42.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce319b6700 1 -- 172.21.15.32:0/936132271 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2c1415e0 con 0x7fce2c0b2fb0 2020-12-05T17:33:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce29ffb700 1 -- 172.21.15.32:0/936132271 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce2400be60 con 0x7fce2c0b2fb0 2020-12-05T17:33:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.219+0000 7fce319b6700 1 -- 172.21.15.32:0/936132271 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2c142050 con 0x7fce2c0b2fb0 2020-12-05T17:33:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.223+0000 7fce319b6700 1 -- 172.21.15.32:0/936132271 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce2c146770 con 0x7fce2c0b2fb0 2020-12-05T17:33:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.223+0000 7fce29ffb700 1 -- 172.21.15.32:0/936132271 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fce2401c460 con 0x7fce2c0b2fb0 2020-12-05T17:33:42.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.223+0000 7fce29ffb700 1 --2- 172.21.15.32:0/936132271 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fce0c055440 0x7fce0c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:42.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.223+0000 7fce309b4700 1 --2- 172.21.15.32:0/936132271 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fce0c055440 0x7fce0c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:42.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.223+0000 7fce29ffb700 1 -- 172.21.15.32:0/936132271 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fce2406e460 con 0x7fce2c0b2fb0 2020-12-05T17:33:42.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.223+0000 7fce309b4700 1 --2- 172.21.15.32:0/936132271 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fce0c055440 0x7fce0c057900 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fce1c01c000 tx=0x7fce1c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:42.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.231+0000 7fce29ffb700 1 -- 172.21.15.32:0/936132271 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fce240447d0 con 0x7fce2c0b2fb0 2020-12-05T17:33:42.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.375+0000 7fce319b6700 1 -- 172.21.15.32:0/936132271 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fce2c057050 con 0x7fce2c0b2fb0 2020-12-05T17:33:42.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.375+0000 7fce29ffb700 1 -- 172.21.15.32:0/936132271 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v39) v1 ==== 92+0+10 (secure 0 0 0) 0x7fce24047df0 con 0x7fce2c0b2fb0 2020-12-05T17:33:42.378 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:33:42.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.379+0000 7fce319b6700 1 -- 172.21.15.32:0/936132271 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fce0c055440 msgr2=0x7fce0c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:42.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.379+0000 7fce319b6700 1 --2- 172.21.15.32:0/936132271 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fce0c055440 0x7fce0c057900 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fce1c01c000 tx=0x7fce1c015000).stop 2020-12-05T17:33:42.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.379+0000 7fce319b6700 1 -- 172.21.15.32:0/936132271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce2c0b2fb0 msgr2=0x7fce2c143b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.379+0000 7fce319b6700 1 --2- 172.21.15.32:0/936132271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce2c0b2fb0 0x7fce2c143b70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fce2400a000 tx=0x7fce24008040).stop 2020-12-05T17:33:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.379+0000 7fce319b6700 1 -- 172.21.15.32:0/936132271 shutdown_connections 2020-12-05T17:33:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.379+0000 7fce319b6700 1 --2- 172.21.15.32:0/936132271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce2c0b2fb0 0x7fce2c143b70 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.379+0000 7fce319b6700 1 --2- 172.21.15.32:0/936132271 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce2c05f3b0 0x7fce2c13e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.381 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.379+0000 7fce319b6700 1 --2- 172.21.15.32:0/936132271 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fce0c055440 0x7fce0c057900 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.381 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.379+0000 7fce319b6700 1 --2- 172.21.15.32:0/936132271 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce2c05feb0 0x7fce2c13f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.381 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.379+0000 7fce319b6700 1 -- 172.21.15.32:0/936132271 >> 172.21.15.32:0/936132271 conn(0x7fce2c0b8f20 msgr2=0x7fce2c0b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:42.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.379+0000 7fce319b6700 1 -- 172.21.15.32:0/936132271 shutdown_connections 2020-12-05T17:33:42.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.379+0000 7fce319b6700 1 -- 172.21.15.32:0/936132271 wait complete. 2020-12-05T17:33:42.392 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-12-05T17:33:42.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.503+0000 7f2ee664f700 1 Processor -- start 2020-12-05T17:33:42.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.503+0000 7f2ee664f700 1 -- start start 2020-12-05T17:33:42.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2ee664f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ee00afe50 0x7f2ee0146a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:42.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2ee664f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2ee00b3aa0 0x7f2ee01429e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:42.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2ee664f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee00b0230 con 0x7f2ee00b1880 2020-12-05T17:33:42.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2ee664f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee00b6a50 con 0x7f2ee00afe50 2020-12-05T17:33:42.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2ee664f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee0146080 con 0x7f2ee00b3aa0 2020-12-05T17:33:42.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2edffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ee00afe50 0x7f2ee0146a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:42.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2ee4bec700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2ee00b1880 0x7f2ee0142ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37120/0 (socket says 172.21.15.32:37120) 2020-12-05T17:33:42.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2ee4bec700 1 -- 172.21.15.32:0/2087276546 learned_addr learned my addr 172.21.15.32:0/2087276546 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:33:42.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2edf7fe700 1 --2- 172.21.15.32:0/2087276546 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2ee00b3aa0 0x7f2ee01429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:42.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2edeffd700 1 -- 172.21.15.32:0/2087276546 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1423714903 0 0) 0x7f2ee00b0230 con 0x7f2ee00b1880 2020-12-05T17:33:42.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2edeffd700 1 -- 172.21.15.32:0/2087276546 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ecc003230 con 0x7f2ee00b1880 2020-12-05T17:33:42.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2edffff700 1 -- 172.21.15.32:0/2087276546 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2ee00b3aa0 msgr2=0x7f2ee01429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:42.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2edffff700 1 --2- 172.21.15.32:0/2087276546 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2ee00b3aa0 0x7f2ee01429e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2edffff700 1 -- 172.21.15.32:0/2087276546 >> v1:172.21.15.32:6789/0 conn(0x7f2ee00b1880 legacy=0x7f2ee0142ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:42.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2edffff700 1 -- 172.21.15.32:0/2087276546 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee0146320 con 0x7f2ee00afe50 2020-12-05T17:33:42.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2edffff700 1 --2- 172.21.15.32:0/2087276546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee00afe50 0x7f2ee0146a60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f2ed0002f30 tx=0x7f2ed0006dc0).ready entity=mon.0 client_cookie=3cd3b96b6d165d44 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:42.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2edeffd700 1 -- 172.21.15.32:0/2087276546 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ed00096a0 con 0x7f2ee00afe50 2020-12-05T17:33:42.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2edeffd700 1 -- 172.21.15.32:0/2087276546 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed0009800 con 0x7f2ee00afe50 2020-12-05T17:33:42.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2edeffd700 1 -- 172.21.15.32:0/2087276546 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ed0009b60 con 0x7f2ee00afe50 2020-12-05T17:33:42.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2ee664f700 1 -- 172.21.15.32:0/2087276546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee00afe50 msgr2=0x7f2ee0146a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:42.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2ee664f700 1 --2- 172.21.15.32:0/2087276546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee00afe50 0x7f2ee0146a60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f2ed0002f30 tx=0x7f2ed0006dc0).stop 2020-12-05T17:33:42.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2ee664f700 1 -- 172.21.15.32:0/2087276546 shutdown_connections 2020-12-05T17:33:42.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2ee664f700 1 --2- 172.21.15.32:0/2087276546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee00afe50 0x7f2ee0146a60 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2ee664f700 1 --2- 172.21.15.32:0/2087276546 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2ee00b3aa0 0x7f2ee01429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2ee664f700 1 -- 172.21.15.32:0/2087276546 >> 172.21.15.32:0/2087276546 conn(0x7f2ee00b9100 msgr2=0x7f2ee00b9510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:42.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2ee664f700 1 -- 172.21.15.32:0/2087276546 shutdown_connections 2020-12-05T17:33:42.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.507+0000 7f2ee664f700 1 -- 172.21.15.32:0/2087276546 wait complete. 2020-12-05T17:33:42.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2ee664f700 1 Processor -- start 2020-12-05T17:33:42.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2ee664f700 1 -- start start 2020-12-05T17:33:42.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2ee664f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2ee00afe50 0x7f2ee00b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:42.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2ee664f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2ee00b1880 0x7f2ee01418d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:42.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2ee664f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee00b3aa0 0x7f2ee0064330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:42.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2ee664f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee0062ff0 con 0x7f2ee00b3aa0 2020-12-05T17:33:42.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2ee664f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee0064820 con 0x7f2ee00b1880 2020-12-05T17:33:42.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2ee664f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee01424f0 con 0x7f2ee00afe50 2020-12-05T17:33:42.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2ee4bec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee00b3aa0 0x7f2ee0064330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:42.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2ee4bec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee00b3aa0 0x7f2ee0064330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34476/0 (socket says 172.21.15.32:34476) 2020-12-05T17:33:42.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2ee4bec700 1 -- 172.21.15.32:0/3734060788 learned_addr learned my addr 172.21.15.32:0/3734060788 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:42.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2edf7fe700 1 --2- 172.21.15.32:0/3734060788 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2ee00b1880 0x7f2ee01418d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:42.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2edffff700 1 --2- 172.21.15.32:0/3734060788 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2ee00afe50 0x7f2ee00b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:42.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2ee4bec700 1 -- 172.21.15.32:0/3734060788 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2ee00afe50 msgr2=0x7f2ee00b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:42.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2ee4bec700 1 --2- 172.21.15.32:0/3734060788 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2ee00afe50 0x7f2ee00b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2ee4bec700 1 -- 172.21.15.32:0/3734060788 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2ee00b1880 msgr2=0x7f2ee01418d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:42.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2ee4bec700 1 --2- 172.21.15.32:0/3734060788 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2ee00b1880 0x7f2ee01418d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2ee4bec700 1 -- 172.21.15.32:0/3734060788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee0151b10 con 0x7f2ee00b3aa0 2020-12-05T17:33:42.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2ee4bec700 1 --2- 172.21.15.32:0/3734060788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee00b3aa0 0x7f2ee0064330 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f2ed8001450 tx=0x7f2ed8007e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:42.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2edd7fa700 1 -- 172.21.15.32:0/3734060788 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ed8005b70 con 0x7f2ee00b3aa0 2020-12-05T17:33:42.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2edd7fa700 1 -- 172.21.15.32:0/3734060788 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed8005cd0 con 0x7f2ee00b3aa0 2020-12-05T17:33:42.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2ee664f700 1 -- 172.21.15.32:0/3734060788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee0151db0 con 0x7f2ee00b3aa0 2020-12-05T17:33:42.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2edd7fa700 1 -- 172.21.15.32:0/3734060788 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ed8007300 con 0x7f2ee00b3aa0 2020-12-05T17:33:42.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.511+0000 7f2ee664f700 1 -- 172.21.15.32:0/3734060788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee01426f0 con 0x7f2ee00b3aa0 2020-12-05T17:33:42.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.515+0000 7f2ee664f700 1 -- 172.21.15.32:0/3734060788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee0151f70 con 0x7f2ee00b3aa0 2020-12-05T17:33:42.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.515+0000 7f2edd7fa700 1 -- 172.21.15.32:0/3734060788 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2ed8010030 con 0x7f2ee00b3aa0 2020-12-05T17:33:42.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.515+0000 7f2edd7fa700 1 --2- 172.21.15.32:0/3734060788 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2ec00553b0 0x7f2ec0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:42.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.515+0000 7f2edd7fa700 1 -- 172.21.15.32:0/3734060788 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f2ed806dea0 con 0x7f2ee00b3aa0 2020-12-05T17:33:42.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.515+0000 7f2edffff700 1 --2- 172.21.15.32:0/3734060788 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2ec00553b0 0x7f2ec0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:42.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.515+0000 7f2edffff700 1 --2- 172.21.15.32:0/3734060788 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2ec00553b0 0x7f2ec0057870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f2ed001b000 tx=0x7f2ed0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:42.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.523+0000 7f2edd7fa700 1 -- 172.21.15.32:0/3734060788 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2ed8044270 con 0x7f2ee00b3aa0 2020-12-05T17:33:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.667+0000 7f2ee664f700 1 -- 172.21.15.32:0/3734060788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2ee0057050 con 0x7f2ee00b3aa0 2020-12-05T17:33:42.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.667+0000 7f2edd7fa700 1 -- 172.21.15.32:0/3734060788 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v39) v1 ==== 84+0+143 (secure 0 0 0) 0x7f2ed8047890 con 0x7f2ee00b3aa0 2020-12-05T17:33:42.668 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:33:42.668 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2020-12-05T17:33:42.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:42.669 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2020-12-05T17:33:42.669 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2020-12-05T17:33:42.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:33:42.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:33:42.670 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2020-12-05T17:33:42.670 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2020-12-05T17:33:42.670 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:33:42.670 INFO:teuthology.orchestra.run.smithi032.stdout:] 2020-12-05T17:33:42.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.667+0000 7f2ee664f700 1 -- 172.21.15.32:0/3734060788 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2ec00553b0 msgr2=0x7f2ec0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:42.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.667+0000 7f2ee664f700 1 --2- 172.21.15.32:0/3734060788 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2ec00553b0 0x7f2ec0057870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f2ed001b000 tx=0x7f2ed0015000).stop 2020-12-05T17:33:42.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.667+0000 7f2ee664f700 1 -- 172.21.15.32:0/3734060788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee00b3aa0 msgr2=0x7f2ee0064330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:42.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.667+0000 7f2ee664f700 1 --2- 172.21.15.32:0/3734060788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee00b3aa0 0x7f2ee0064330 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f2ed8001450 tx=0x7f2ed8007e40).stop 2020-12-05T17:33:42.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.667+0000 7f2ee664f700 1 -- 172.21.15.32:0/3734060788 shutdown_connections 2020-12-05T17:33:42.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.667+0000 7f2ee664f700 1 --2- 172.21.15.32:0/3734060788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee00b3aa0 0x7f2ee0064330 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.667+0000 7f2ee664f700 1 --2- 172.21.15.32:0/3734060788 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2ec00553b0 0x7f2ec0057870 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.667+0000 7f2ee664f700 1 --2- 172.21.15.32:0/3734060788 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2ee00afe50 0x7f2ee00b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.667+0000 7f2ee664f700 1 --2- 172.21.15.32:0/3734060788 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2ee00b1880 0x7f2ee01418d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.667+0000 7f2ee664f700 1 -- 172.21.15.32:0/3734060788 >> 172.21.15.32:0/3734060788 conn(0x7f2ee00b9100 msgr2=0x7f2ee00b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:42.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.667+0000 7f2ee664f700 1 -- 172.21.15.32:0/3734060788 shutdown_connections 2020-12-05T17:33:42.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.667+0000 7f2ee664f700 1 -- 172.21.15.32:0/3734060788 wait complete. 2020-12-05T17:33:42.682 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-12-05T17:33:42.682 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-12-05T17:33:42.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.807+0000 7f51230a1700 1 Processor -- start 2020-12-05T17:33:42.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.807+0000 7f51230a1700 1 -- start start 2020-12-05T17:33:42.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.807+0000 7f51230a1700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f511c0af090 0x7f511c0af500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:42.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.807+0000 7f51230a1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f511c067110 con 0x7f511c05f550 2020-12-05T17:33:42.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.807+0000 7f51230a1700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f511c0ad290 con 0x7f511c063040 2020-12-05T17:33:42.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.807+0000 7f51230a1700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f511c1405b0 con 0x7f511c0af090 2020-12-05T17:33:42.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.807+0000 7f512209f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f511c05f550 0x7f511c0acb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37128/0 (socket says 172.21.15.32:37128) 2020-12-05T17:33:42.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.807+0000 7f512209f700 1 -- 172.21.15.32:0/3411524982 learned_addr learned my addr 172.21.15.32:0/3411524982 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:33:42.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.807+0000 7f51228a0700 1 --2- 172.21.15.32:0/3411524982 >> v2:172.21.15.61:3300/0 conn(0x7f511c0af090 0x7f511c0af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:42.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.807+0000 7f512109d700 1 -- 172.21.15.32:0/3411524982 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2374286950 0 0) 0x7f511c067110 con 0x7f511c05f550 2020-12-05T17:33:42.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.807+0000 7f512109d700 1 -- 172.21.15.32:0/3411524982 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5104003230 con 0x7f511c05f550 2020-12-05T17:33:42.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.811+0000 7f512109d700 1 -- 172.21.15.32:0/3411524982 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 435895463 0 0) 0x7f5104003230 con 0x7f511c05f550 2020-12-05T17:33:42.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.811+0000 7f51228a0700 1 -- 172.21.15.32:0/3411524982 >> v1:172.21.15.61:6789/0 conn(0x7f511c063040 legacy=0x7f511c13fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:42.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.811+0000 7f51228a0700 1 -- 172.21.15.32:0/3411524982 >> v1:172.21.15.32:6789/0 conn(0x7f511c05f550 legacy=0x7f511c0acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:42.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.811+0000 7f51228a0700 1 -- 172.21.15.32:0/3411524982 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511c13f760 con 0x7f511c0af090 2020-12-05T17:33:42.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.811+0000 7f51228a0700 1 --2- 172.21.15.32:0/3411524982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f511c0af090 0x7f511c0af500 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f5118001870 tx=0x7f51140091a0).ready entity=mon.1 client_cookie=67a8522518b26044 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:42.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.811+0000 7f512109d700 1 -- 172.21.15.32:0/3411524982 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51180016d0 con 0x7f511c0af090 2020-12-05T17:33:42.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.811+0000 7f512109d700 1 -- 172.21.15.32:0/3411524982 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511400b860 con 0x7f511c0af090 2020-12-05T17:33:42.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.811+0000 7f512109d700 1 -- 172.21.15.32:0/3411524982 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f511400b9d0 con 0x7f511c0af090 2020-12-05T17:33:42.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.811+0000 7f51230a1700 1 -- 172.21.15.32:0/3411524982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f511c0af090 msgr2=0x7f511c0af500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:42.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.811+0000 7f51230a1700 1 --2- 172.21.15.32:0/3411524982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f511c0af090 0x7f511c0af500 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f5118001870 tx=0x7f51140091a0).stop 2020-12-05T17:33:42.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.811+0000 7f51230a1700 1 -- 172.21.15.32:0/3411524982 shutdown_connections 2020-12-05T17:33:42.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.811+0000 7f51230a1700 1 --2- 172.21.15.32:0/3411524982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f511c0af090 0x7f511c0af500 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.811+0000 7f51230a1700 1 -- 172.21.15.32:0/3411524982 >> 172.21.15.32:0/3411524982 conn(0x7f511c0620b0 msgr2=0x7f511c062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:42.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.811+0000 7f51230a1700 1 -- 172.21.15.32:0/3411524982 shutdown_connections 2020-12-05T17:33:42.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.811+0000 7f51230a1700 1 -- 172.21.15.32:0/3411524982 wait complete. 2020-12-05T17:33:42.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.811+0000 7f51230a1700 1 Processor -- start 2020-12-05T17:33:42.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.811+0000 7f51230a1700 1 -- start start 2020-12-05T17:33:42.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.815+0000 7f51230a1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f511c05f550 0x7f511c135e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:42.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.815+0000 7f51230a1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f511c063040 0x7f511c142490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:42.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.815+0000 7f51230a1700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f511c0af090 0x7f511c137950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:42.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.815+0000 7f51230a1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c142e90 con 0x7f511c063040 2020-12-05T17:33:42.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.815+0000 7f51230a1700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c0649a0 con 0x7f511c0af090 2020-12-05T17:33:42.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.815+0000 7f51230a1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f511c064c40 con 0x7f511c05f550 2020-12-05T17:33:42.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.815+0000 7f512189e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f511c063040 0x7f511c142490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:42.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.815+0000 7f512189e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f511c063040 0x7f511c142490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34488/0 (socket says 172.21.15.32:34488) 2020-12-05T17:33:42.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.815+0000 7f512189e700 1 -- 172.21.15.32:0/1860655207 learned_addr learned my addr 172.21.15.32:0/1860655207 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:42.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.815+0000 7f512209f700 1 --2- 172.21.15.32:0/1860655207 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f511c05f550 0x7f511c135e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:42.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.815+0000 7f51228a0700 1 --2- 172.21.15.32:0/1860655207 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f511c0af090 0x7f511c137950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:42.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.815+0000 7f512189e700 1 -- 172.21.15.32:0/1860655207 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f511c05f550 msgr2=0x7f511c135e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:42.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.815+0000 7f512189e700 1 --2- 172.21.15.32:0/1860655207 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f511c05f550 0x7f511c135e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.815+0000 7f512189e700 1 -- 172.21.15.32:0/1860655207 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f511c0af090 msgr2=0x7f511c137950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:42.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.815+0000 7f512189e700 1 --2- 172.21.15.32:0/1860655207 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f511c0af090 0x7f511c137950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:42.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.815+0000 7f512189e700 1 -- 172.21.15.32:0/1860655207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511c064ee0 con 0x7f511c063040 2020-12-05T17:33:42.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.815+0000 7f512189e700 1 --2- 172.21.15.32:0/1860655207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f511c063040 0x7f511c142490 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f511c139c00 tx=0x7f510c007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:42.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.815+0000 7f510b7fe700 1 -- 172.21.15.32:0/1860655207 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f510c013410 con 0x7f511c063040 2020-12-05T17:33:42.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.815+0000 7f510b7fe700 1 -- 172.21.15.32:0/1860655207 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510c013570 con 0x7f511c063040 2020-12-05T17:33:42.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.815+0000 7f510b7fe700 1 -- 172.21.15.32:0/1860655207 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f510c0138d0 con 0x7f511c063040 2020-12-05T17:33:42.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.815+0000 7f51230a1700 1 -- 172.21.15.32:0/1860655207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f511c142980 con 0x7f511c063040 2020-12-05T17:33:42.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.815+0000 7f51230a1700 1 -- 172.21.15.32:0/1860655207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511c066230 con 0x7f511c063040 2020-12-05T17:33:42.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.815+0000 7f51230a1700 1 -- 172.21.15.32:0/1860655207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511c057050 con 0x7f511c063040 2020-12-05T17:33:42.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.819+0000 7f510b7fe700 1 -- 172.21.15.32:0/1860655207 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f510c010030 con 0x7f511c063040 2020-12-05T17:33:42.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.819+0000 7f510b7fe700 1 --2- 172.21.15.32:0/1860655207 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f50fc055330 0x7f50fc0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:42.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.819+0000 7f510b7fe700 1 -- 172.21.15.32:0/1860655207 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f510c06db50 con 0x7f511c063040 2020-12-05T17:33:42.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.819+0000 7f512209f700 1 --2- 172.21.15.32:0/1860655207 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f50fc055330 0x7f50fc0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:42.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.819+0000 7f512209f700 1 --2- 172.21.15.32:0/1860655207 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f50fc055330 0x7f50fc0577f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f511800d000 tx=0x7f5118000b80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:42.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.823+0000 7f510b7fe700 1 -- 172.21.15.32:0/1860655207 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f510c044030 con 0x7f511c063040 2020-12-05T17:33:42.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:42.967+0000 7f51230a1700 1 -- 172.21.15.32:0/1860655207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f511c060c10 con 0x7f511c063040 2020-12-05T17:33:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:43.975+0000 7f510b7fe700 1 -- 172.21.15.32:0/1860655207 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v40) v1 ==== 133+0+0 (secure 0 0 0) 0x7f510c047650 con 0x7f511c063040 2020-12-05T17:33:43.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:43.979+0000 7f51230a1700 1 -- 172.21.15.32:0/1860655207 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f50fc055330 msgr2=0x7f50fc0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:43.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:43.979+0000 7f51230a1700 1 --2- 172.21.15.32:0/1860655207 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f50fc055330 0x7f50fc0577f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f511800d000 tx=0x7f5118000b80).stop 2020-12-05T17:33:43.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:43.979+0000 7f51230a1700 1 -- 172.21.15.32:0/1860655207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f511c063040 msgr2=0x7f511c142490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:43.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:43.979+0000 7f51230a1700 1 --2- 172.21.15.32:0/1860655207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f511c063040 0x7f511c142490 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f511c139c00 tx=0x7f510c007c20).stop 2020-12-05T17:33:43.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:43.983+0000 7f51230a1700 1 -- 172.21.15.32:0/1860655207 shutdown_connections 2020-12-05T17:33:43.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:43.983+0000 7f51230a1700 1 --2- 172.21.15.32:0/1860655207 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f511c0af090 0x7f511c137950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:43.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:43.983+0000 7f51230a1700 1 --2- 172.21.15.32:0/1860655207 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f50fc055330 0x7f50fc0577f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:43.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:43.983+0000 7f51230a1700 1 --2- 172.21.15.32:0/1860655207 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f511c05f550 0x7f511c135e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:43.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:43.983+0000 7f51230a1700 1 --2- 172.21.15.32:0/1860655207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f511c063040 0x7f511c142490 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:43.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:43.983+0000 7f51230a1700 1 -- 172.21.15.32:0/1860655207 >> 172.21.15.32:0/1860655207 conn(0x7f511c0620b0 msgr2=0x7f511c067460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:43.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:43.983+0000 7f51230a1700 1 -- 172.21.15.32:0/1860655207 shutdown_connections 2020-12-05T17:33:43.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:43.983+0000 7f51230a1700 1 -- 172.21.15.32:0/1860655207 wait complete. 2020-12-05T17:33:43.985 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2020-12-05T17:33:44.000 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-12-05T17:33:44.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.119+0000 7f4389ad2700 1 Processor -- start 2020-12-05T17:33:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.119+0000 7f4389ad2700 1 -- start start 2020-12-05T17:33:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.119+0000 7f4389ad2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43840b5f00 0x7f43840b5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.119+0000 7f4389ad2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43840b1920 con 0x7f43840b5b20 2020-12-05T17:33:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.119+0000 7f4389ad2700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43840b1bc0 con 0x7f43840b6690 2020-12-05T17:33:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.119+0000 7f4389ad2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43840b5730 con 0x7f43840b5f00 2020-12-05T17:33:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.119+0000 7f4383fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43840b5f00 0x7f43840b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.119+0000 7f43837fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f43840b5b20 0x7f43840b2060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37142/0 (socket says 172.21.15.32:37142) 2020-12-05T17:33:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.119+0000 7f4383fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43840b5f00 0x7f43840b5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34498/0 (socket says 172.21.15.32:34498) 2020-12-05T17:33:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.119+0000 7f43837fe700 1 -- 172.21.15.32:0/2263550314 learned_addr learned my addr 172.21.15.32:0/2263550314 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:33:44.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.119+0000 7f4383fff700 1 -- 172.21.15.32:0/2263550314 >> v1:172.21.15.61:6789/0 conn(0x7f43840b6690 legacy=0x7f4384145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:44.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.119+0000 7f4383fff700 1 -- 172.21.15.32:0/2263550314 >> v1:172.21.15.32:6789/0 conn(0x7f43840b5b20 legacy=0x7f43840b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:44.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.119+0000 7f4383fff700 1 -- 172.21.15.32:0/2263550314 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4384146620 con 0x7f43840b5f00 2020-12-05T17:33:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.119+0000 7f4383fff700 1 --2- 172.21.15.32:0/2263550314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43840b5f00 0x7f43840b5240 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f437c0032b0 tx=0x7f437c0091a0).ready entity=mon.0 client_cookie=a06273c33db4f112 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.119+0000 7f43827fc700 1 -- 172.21.15.32:0/2263550314 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f437c00ba50 con 0x7f43840b5f00 2020-12-05T17:33:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.119+0000 7f43827fc700 1 -- 172.21.15.32:0/2263550314 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f437c00bbb0 con 0x7f43840b5f00 2020-12-05T17:33:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.119+0000 7f43827fc700 1 -- 172.21.15.32:0/2263550314 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f437c00bd20 con 0x7f43840b5f00 2020-12-05T17:33:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.119+0000 7f4389ad2700 1 -- 172.21.15.32:0/2263550314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43840b5f00 msgr2=0x7f43840b5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.119+0000 7f4389ad2700 1 --2- 172.21.15.32:0/2263550314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43840b5f00 0x7f43840b5240 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f437c0032b0 tx=0x7f437c0091a0).stop 2020-12-05T17:33:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.123+0000 7f4389ad2700 1 -- 172.21.15.32:0/2263550314 shutdown_connections 2020-12-05T17:33:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.123+0000 7f4389ad2700 1 --2- 172.21.15.32:0/2263550314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43840b5f00 0x7f43840b5240 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.123+0000 7f4389ad2700 1 -- 172.21.15.32:0/2263550314 >> 172.21.15.32:0/2263550314 conn(0x7f438405f3b0 msgr2=0x7f438405f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.123+0000 7f4389ad2700 1 -- 172.21.15.32:0/2263550314 shutdown_connections 2020-12-05T17:33:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.123+0000 7f4389ad2700 1 -- 172.21.15.32:0/2263550314 wait complete. 2020-12-05T17:33:44.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.123+0000 7f4389ad2700 1 Processor -- start 2020-12-05T17:33:44.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.123+0000 7f4389ad2700 1 -- start start 2020-12-05T17:33:44.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.127+0000 7f4389ad2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43840b5b20 0x7f43840a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:44.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.127+0000 7f4389ad2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f43840b5f00 0x7f43840a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:44.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.127+0000 7f4389ad2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f43840b6690 0x7f43840aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:44.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.127+0000 7f4389ad2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4384141100 con 0x7f43840b5b20 2020-12-05T17:33:44.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.127+0000 7f4389ad2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43840668e0 con 0x7f43840b6690 2020-12-05T17:33:44.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.127+0000 7f4389ad2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43840a3090 con 0x7f43840b5f00 2020-12-05T17:33:44.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.127+0000 7f43837fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43840b5b20 0x7f43840a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:44.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.127+0000 7f43837fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43840b5b20 0x7f43840a3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34500/0 (socket says 172.21.15.32:34500) 2020-12-05T17:33:44.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.127+0000 7f43837fe700 1 -- 172.21.15.32:0/3824682772 learned_addr learned my addr 172.21.15.32:0/3824682772 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:44.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.127+0000 7f4382ffd700 1 --2- 172.21.15.32:0/3824682772 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f43840b5f00 0x7f43840a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:44.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.127+0000 7f4383fff700 1 --2- 172.21.15.32:0/3824682772 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f43840b6690 0x7f43840aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:44.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.127+0000 7f43837fe700 1 -- 172.21.15.32:0/3824682772 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f43840b5f00 msgr2=0x7f43840a3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:44.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.127+0000 7f43837fe700 1 --2- 172.21.15.32:0/3824682772 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f43840b5f00 0x7f43840a3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:44.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.127+0000 7f43837fe700 1 -- 172.21.15.32:0/3824682772 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f43840b6690 msgr2=0x7f43840aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:44.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.127+0000 7f43837fe700 1 --2- 172.21.15.32:0/3824682772 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f43840b6690 0x7f43840aebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:44.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.127+0000 7f43837fe700 1 -- 172.21.15.32:0/3824682772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4384066b10 con 0x7f43840b5b20 2020-12-05T17:33:44.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.127+0000 7f43837fe700 1 --2- 172.21.15.32:0/3824682772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43840b5b20 0x7f43840a3940 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f43840b2770 tx=0x7f4374007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:44.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.127+0000 7f4380ff9700 1 -- 172.21.15.32:0/3824682772 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43740072e0 con 0x7f43840b5b20 2020-12-05T17:33:44.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.127+0000 7f4380ff9700 1 -- 172.21.15.32:0/3824682772 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4374007440 con 0x7f43840b5b20 2020-12-05T17:33:44.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.127+0000 7f4389ad2700 1 -- 172.21.15.32:0/3824682772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43840a3330 con 0x7f43840b5b20 2020-12-05T17:33:44.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.127+0000 7f4380ff9700 1 -- 172.21.15.32:0/3824682772 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43740057d0 con 0x7f43840b5b20 2020-12-05T17:33:44.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.127+0000 7f4389ad2700 1 -- 172.21.15.32:0/3824682772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43840a65f0 con 0x7f43840b5b20 2020-12-05T17:33:44.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.127+0000 7f4389ad2700 1 -- 172.21.15.32:0/3824682772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4384057050 con 0x7f43840b5b20 2020-12-05T17:33:44.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.131+0000 7f4380ff9700 1 -- 172.21.15.32:0/3824682772 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4374010070 con 0x7f43840b5b20 2020-12-05T17:33:44.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.131+0000 7f4380ff9700 1 --2- 172.21.15.32:0/3824682772 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f43640554e0 0x7f43640579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:44.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.131+0000 7f4380ff9700 1 -- 172.21.15.32:0/3824682772 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f437406d510 con 0x7f43840b5b20 2020-12-05T17:33:44.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.131+0000 7f4382ffd700 1 --2- 172.21.15.32:0/3824682772 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f43640554e0 0x7f43640579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:44.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.131+0000 7f4382ffd700 1 --2- 172.21.15.32:0/3824682772 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f43640554e0 0x7f43640579a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f437800d000 tx=0x7f4378007b60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:44.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.139+0000 7f4380ff9700 1 -- 172.21.15.32:0/3824682772 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4374043880 con 0x7f43840b5b20 2020-12-05T17:33:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:44.283+0000 7f4389ad2700 1 -- 172.21.15.32:0/3824682772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f438405bae0 con 0x7f43840b5b20 2020-12-05T17:33:44.941 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:33:44.944 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:33:44.949 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:33:44.952 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:33:44.954 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:33:45.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.163+0000 7f4380ff9700 1 -- 172.21.15.32:0/3824682772 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v42) v1 ==== 125+0+0 (secure 0 0 0) 0x7f4374046ea0 con 0x7f43840b5b20 2020-12-05T17:33:45.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.167+0000 7f4389ad2700 1 -- 172.21.15.32:0/3824682772 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f43640554e0 msgr2=0x7f43640579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:45.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.167+0000 7f4389ad2700 1 --2- 172.21.15.32:0/3824682772 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f43640554e0 0x7f43640579a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f437800d000 tx=0x7f4378007b60).stop 2020-12-05T17:33:45.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.167+0000 7f4389ad2700 1 -- 172.21.15.32:0/3824682772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43840b5b20 msgr2=0x7f43840a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:45.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.167+0000 7f4389ad2700 1 --2- 172.21.15.32:0/3824682772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43840b5b20 0x7f43840a3940 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f43840b2770 tx=0x7f4374007c10).stop 2020-12-05T17:33:45.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.167+0000 7f4389ad2700 1 -- 172.21.15.32:0/3824682772 shutdown_connections 2020-12-05T17:33:45.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.167+0000 7f4389ad2700 1 --2- 172.21.15.32:0/3824682772 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f43840b6690 0x7f43840aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.167+0000 7f4389ad2700 1 --2- 172.21.15.32:0/3824682772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43840b5b20 0x7f43840a3940 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.167+0000 7f4389ad2700 1 --2- 172.21.15.32:0/3824682772 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f43640554e0 0x7f43640579a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.167+0000 7f4389ad2700 1 --2- 172.21.15.32:0/3824682772 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f43840b5f00 0x7f43840a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.167+0000 7f4389ad2700 1 -- 172.21.15.32:0/3824682772 >> 172.21.15.32:0/3824682772 conn(0x7f438405f3b0 msgr2=0x7f43841453e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:45.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.167+0000 7f4389ad2700 1 -- 172.21.15.32:0/3824682772 shutdown_connections 2020-12-05T17:33:45.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.167+0000 7f4389ad2700 1 -- 172.21.15.32:0/3824682772 wait complete. 2020-12-05T17:33:45.173 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2020-12-05T17:33:45.186 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-12-05T17:33:45.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.299+0000 7fe021b92700 1 Processor -- start 2020-12-05T17:33:45.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.299+0000 7fe021b92700 1 -- start start 2020-12-05T17:33:45.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe021b92700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe01c0b6690 0x7fe01c0b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:45.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe021b92700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe01c0b5240 0x7fe01c145f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:45.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe021b92700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe01c0b56f0 con 0x7fe01c0b5b20 2020-12-05T17:33:45.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe021b92700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe01c0b1920 con 0x7fe01c0b5240 2020-12-05T17:33:45.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe021b92700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c0b1bc0 con 0x7fe01c0b6690 2020-12-05T17:33:45.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe01bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe01c0b5240 0x7fe01c145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:45.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe01bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe01c0b5240 0x7fe01c145f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34512/0 (socket says 172.21.15.32:34512) 2020-12-05T17:33:45.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe01bfff700 1 -- 172.21.15.32:0/11514786 learned_addr learned my addr 172.21.15.32:0/11514786 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:45.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe01affd700 1 --2- 172.21.15.32:0/11514786 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe01c0b6690 0x7fe01c0b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:45.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe01bfff700 1 -- 172.21.15.32:0/11514786 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe01c0b6690 msgr2=0x7fe01c0b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:45.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe01bfff700 1 --2- 172.21.15.32:0/11514786 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe01c0b6690 0x7fe01c0b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe01bfff700 1 -- 172.21.15.32:0/11514786 >> v1:172.21.15.61:6789/0 conn(0x7fe01c0b5b20 legacy=0x7fe01c0b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:45.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe01bfff700 1 -- 172.21.15.32:0/11514786 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c146400 con 0x7fe01c0b5240 2020-12-05T17:33:45.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe01bfff700 1 --2- 172.21.15.32:0/11514786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe01c0b5240 0x7fe01c145f10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7fe0140032b0 tx=0x7fe014009270).ready entity=mon.0 client_cookie=9bf932601ad25cee server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:45.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe01a7fc700 1 -- 172.21.15.32:0/11514786 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe014009e60 con 0x7fe01c0b5240 2020-12-05T17:33:45.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe01a7fc700 1 -- 172.21.15.32:0/11514786 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01400bc00 con 0x7fe01c0b5240 2020-12-05T17:33:45.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe01a7fc700 1 -- 172.21.15.32:0/11514786 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe01400bd70 con 0x7fe01c0b5240 2020-12-05T17:33:45.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe021b92700 1 -- 172.21.15.32:0/11514786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe01c0b5240 msgr2=0x7fe01c145f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:45.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe021b92700 1 --2- 172.21.15.32:0/11514786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe01c0b5240 0x7fe01c145f10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7fe0140032b0 tx=0x7fe014009270).stop 2020-12-05T17:33:45.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe021b92700 1 -- 172.21.15.32:0/11514786 shutdown_connections 2020-12-05T17:33:45.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe021b92700 1 --2- 172.21.15.32:0/11514786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe01c0b5240 0x7fe01c145f10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe021b92700 1 --2- 172.21.15.32:0/11514786 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe01c0b6690 0x7fe01c0b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe021b92700 1 -- 172.21.15.32:0/11514786 >> 172.21.15.32:0/11514786 conn(0x7fe01c05f3b0 msgr2=0x7fe01c05f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:45.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe021b92700 1 -- 172.21.15.32:0/11514786 shutdown_connections 2020-12-05T17:33:45.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.303+0000 7fe021b92700 1 -- 172.21.15.32:0/11514786 wait complete. 2020-12-05T17:33:45.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe021b92700 1 Processor -- start 2020-12-05T17:33:45.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe021b92700 1 -- start start 2020-12-05T17:33:45.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe021b92700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe01c0b5240 0x7fe01c0a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:45.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe021b92700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe01c0b5b20 0x7fe01c141f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe021b92700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe01c0b6690 0x7fe01c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe021b92700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c0a3d00 con 0x7fe01c0b6690 2020-12-05T17:33:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe021b92700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c151be0 con 0x7fe01c0b5b20 2020-12-05T17:33:45.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe021b92700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c0a38a0 con 0x7fe01c0b5240 2020-12-05T17:33:45.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe01bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe01c0b6690 0x7fe01c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:45.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe01bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe01c0b6690 0x7fe01c065150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34518/0 (socket says 172.21.15.32:34518) 2020-12-05T17:33:45.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe01bfff700 1 -- 172.21.15.32:0/1637816359 learned_addr learned my addr 172.21.15.32:0/1637816359 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:45.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe01b7fe700 1 --2- 172.21.15.32:0/1637816359 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe01c0b5240 0x7fe01c0a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:45.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe01affd700 1 --2- 172.21.15.32:0/1637816359 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe01c0b5b20 0x7fe01c141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:45.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe01bfff700 1 -- 172.21.15.32:0/1637816359 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe01c0b5240 msgr2=0x7fe01c0a2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:45.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe01bfff700 1 --2- 172.21.15.32:0/1637816359 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe01c0b5240 0x7fe01c0a2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.323 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe01bfff700 1 -- 172.21.15.32:0/1637816359 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe01c0b5b20 msgr2=0x7fe01c141f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:45.323 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe01bfff700 1 --2- 172.21.15.32:0/1637816359 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe01c0b5b20 0x7fe01c141f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.324 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe01bfff700 1 -- 172.21.15.32:0/1637816359 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c0a6c50 con 0x7fe01c0b6690 2020-12-05T17:33:45.324 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe01bfff700 1 --2- 172.21.15.32:0/1637816359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe01c0b6690 0x7fe01c065150 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fe0140032b0 tx=0x7fe014016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:45.324 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe018ff9700 1 -- 172.21.15.32:0/1637816359 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0140107a0 con 0x7fe01c0b6690 2020-12-05T17:33:45.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe018ff9700 1 -- 172.21.15.32:0/1637816359 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe014010900 con 0x7fe01c0b6690 2020-12-05T17:33:45.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe021b92700 1 -- 172.21.15.32:0/1637816359 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01c151e10 con 0x7fe01c0b6690 2020-12-05T17:33:45.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe018ff9700 1 -- 172.21.15.32:0/1637816359 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe014010c60 con 0x7fe01c0b6690 2020-12-05T17:33:45.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.307+0000 7fe021b92700 1 -- 172.21.15.32:0/1637816359 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01c0a4060 con 0x7fe01c0b6690 2020-12-05T17:33:45.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.311+0000 7fe021b92700 1 -- 172.21.15.32:0/1637816359 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe01c057050 con 0x7fe01c0b6690 2020-12-05T17:33:45.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.311+0000 7fe018ff9700 1 -- 172.21.15.32:0/1637816359 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe014026030 con 0x7fe01c0b6690 2020-12-05T17:33:45.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.311+0000 7fe018ff9700 1 --2- 172.21.15.32:0/1637816359 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe0000553b0 0x7fe000057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:45.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.311+0000 7fe018ff9700 1 -- 172.21.15.32:0/1637816359 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fe01406f1c0 con 0x7fe01c0b6690 2020-12-05T17:33:45.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.311+0000 7fe01b7fe700 1 --2- 172.21.15.32:0/1637816359 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe0000553b0 0x7fe000057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:45.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.311+0000 7fe01b7fe700 1 --2- 172.21.15.32:0/1637816359 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe0000553b0 0x7fe000057870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fe00400b000 tx=0x7fe004009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:45.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.319+0000 7fe018ff9700 1 -- 172.21.15.32:0/1637816359 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe0140453e0 con 0x7fe01c0b6690 2020-12-05T17:33:45.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.507+0000 7fe021b92700 1 -- 172.21.15.32:0/1637816359 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fe01c05bae0 con 0x7fe01c0b6690 2020-12-05T17:33:45.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.519+0000 7fe018ff9700 1 -- 172.21.15.32:0/1637816359 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 7 and data pool 8 v15) v1 ==== 187+0+0 (secure 0 0 0) 0x7fe0140240d0 con 0x7fe01c0b6690 2020-12-05T17:33:45.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.527+0000 7fe021b92700 1 -- 172.21.15.32:0/1637816359 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe0000553b0 msgr2=0x7fe000057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:45.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.527+0000 7fe021b92700 1 --2- 172.21.15.32:0/1637816359 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe0000553b0 0x7fe000057870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fe00400b000 tx=0x7fe004009040).stop 2020-12-05T17:33:45.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.527+0000 7fe021b92700 1 -- 172.21.15.32:0/1637816359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe01c0b6690 msgr2=0x7fe01c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:45.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.527+0000 7fe021b92700 1 --2- 172.21.15.32:0/1637816359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe01c0b6690 0x7fe01c065150 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fe0140032b0 tx=0x7fe014016000).stop 2020-12-05T17:33:45.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.527+0000 7fe021b92700 1 -- 172.21.15.32:0/1637816359 shutdown_connections 2020-12-05T17:33:45.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.527+0000 7fe021b92700 1 --2- 172.21.15.32:0/1637816359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe01c0b6690 0x7fe01c065150 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.527+0000 7fe021b92700 1 --2- 172.21.15.32:0/1637816359 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe0000553b0 0x7fe000057870 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.527+0000 7fe021b92700 1 --2- 172.21.15.32:0/1637816359 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe01c0b5240 0x7fe01c0a2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.527+0000 7fe021b92700 1 --2- 172.21.15.32:0/1637816359 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe01c0b5b20 0x7fe01c141f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.527+0000 7fe021b92700 1 -- 172.21.15.32:0/1637816359 >> 172.21.15.32:0/1637816359 conn(0x7fe01c05f3b0 msgr2=0x7fe01c0b4130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:45.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.527+0000 7fe021b92700 1 -- 172.21.15.32:0/1637816359 shutdown_connections 2020-12-05T17:33:45.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.531+0000 7fe021b92700 1 -- 172.21.15.32:0/1637816359 wait complete. 2020-12-05T17:33:45.534 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 7 and data pool 8 2020-12-05T17:33:45.547 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:33:45.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.651+0000 7fca10b86700 1 Processor -- start 2020-12-05T17:33:45.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.651+0000 7fca10b86700 1 -- start start 2020-12-05T17:33:45.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.651+0000 7fca10b86700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fca0c0ad1b0 0x7fca0c0aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:45.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.651+0000 7fca10b86700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fca0c05f490 0x7fca0c05f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:45.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.651+0000 7fca10b86700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca0c13fb70 con 0x7fca0c0aead0 2020-12-05T17:33:45.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.651+0000 7fca10b86700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca0c0af460 con 0x7fca0c0ad1b0 2020-12-05T17:33:45.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.651+0000 7fca10b86700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca0c0af700 con 0x7fca0c05f490 2020-12-05T17:33:45.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.651+0000 7fca0a59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fca0c0aead0 0x7fca0c1389f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37180/0 (socket says 172.21.15.32:37180) 2020-12-05T17:33:45.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.651+0000 7fca0a59c700 1 -- 172.21.15.32:0/1734852910 learned_addr learned my addr 172.21.15.32:0/1734852910 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:33:45.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.651+0000 7fca09d9b700 1 --2- 172.21.15.32:0/1734852910 >> v2:172.21.15.32:3300/0 conn(0x7fca0c0ad1b0 0x7fca0c0aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:45.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.655+0000 7fca0ad9d700 1 --2- 172.21.15.32:0/1734852910 >> v2:172.21.15.61:3300/0 conn(0x7fca0c05f490 0x7fca0c05f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:45.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.655+0000 7fca0959a700 1 -- 172.21.15.32:0/1734852910 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 742319316 0 0) 0x7fca0c13fb70 con 0x7fca0c0aead0 2020-12-05T17:33:45.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.655+0000 7fca0959a700 1 -- 172.21.15.32:0/1734852910 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9ec003230 con 0x7fca0c0aead0 2020-12-05T17:33:45.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.655+0000 7fca0ad9d700 1 -- 172.21.15.32:0/1734852910 >> v2:172.21.15.32:3300/0 conn(0x7fca0c0ad1b0 msgr2=0x7fca0c0aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:45.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.655+0000 7fca0ad9d700 1 --2- 172.21.15.32:0/1734852910 >> v2:172.21.15.32:3300/0 conn(0x7fca0c0ad1b0 0x7fca0c0aeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.655+0000 7fca0ad9d700 1 -- 172.21.15.32:0/1734852910 >> v1:172.21.15.32:6789/0 conn(0x7fca0c0aead0 legacy=0x7fca0c1389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:45.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.655+0000 7fca0ad9d700 1 -- 172.21.15.32:0/1734852910 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0c139100 con 0x7fca0c05f490 2020-12-05T17:33:45.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.655+0000 7fca0ad9d700 1 --2- 172.21.15.32:0/1734852910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca0c05f490 0x7fca0c05f940 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fca04002f30 tx=0x7fca04006dc0).ready entity=mon.1 client_cookie=35480fe6c6f887ba server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:45.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.655+0000 7fca0959a700 1 -- 172.21.15.32:0/1734852910 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca040096f0 con 0x7fca0c05f490 2020-12-05T17:33:45.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.655+0000 7fca0959a700 1 -- 172.21.15.32:0/1734852910 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca04009850 con 0x7fca0c05f490 2020-12-05T17:33:45.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.655+0000 7fca0959a700 1 -- 172.21.15.32:0/1734852910 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca04009bb0 con 0x7fca0c05f490 2020-12-05T17:33:45.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.655+0000 7fca10b86700 1 -- 172.21.15.32:0/1734852910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca0c05f490 msgr2=0x7fca0c05f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:45.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.655+0000 7fca10b86700 1 --2- 172.21.15.32:0/1734852910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca0c05f490 0x7fca0c05f940 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fca04002f30 tx=0x7fca04006dc0).stop 2020-12-05T17:33:45.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.655+0000 7fca10b86700 1 -- 172.21.15.32:0/1734852910 shutdown_connections 2020-12-05T17:33:45.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.655+0000 7fca10b86700 1 --2- 172.21.15.32:0/1734852910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca0c05f490 0x7fca0c05f940 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.655+0000 7fca10b86700 1 --2- 172.21.15.32:0/1734852910 >> v2:172.21.15.32:3300/0 conn(0x7fca0c0ad1b0 0x7fca0c0aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.655+0000 7fca10b86700 1 -- 172.21.15.32:0/1734852910 >> 172.21.15.32:0/1734852910 conn(0x7fca0c0b5580 msgr2=0x7fca0c0b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:45.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.655+0000 7fca10b86700 1 -- 172.21.15.32:0/1734852910 shutdown_connections 2020-12-05T17:33:45.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.655+0000 7fca10b86700 1 -- 172.21.15.32:0/1734852910 wait complete. 2020-12-05T17:33:45.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca10b86700 1 Processor -- start 2020-12-05T17:33:45.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca10b86700 1 -- start start 2020-12-05T17:33:45.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca10b86700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca0c05f490 0x7fca0c0676d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:45.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca10b86700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fca0c0ad1b0 0x7fca0c14f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:45.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca10b86700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca0c0aead0 0x7fca0c13e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:45.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca10b86700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c14f780 con 0x7fca0c05f490 2020-12-05T17:33:45.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca10b86700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c0640d0 con 0x7fca0c0aead0 2020-12-05T17:33:45.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca10b86700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c152a30 con 0x7fca0c0ad1b0 2020-12-05T17:33:45.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca0a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca0c05f490 0x7fca0c0676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:45.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca0a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca0c05f490 0x7fca0c0676d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34538/0 (socket says 172.21.15.32:34538) 2020-12-05T17:33:45.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca0a59c700 1 -- 172.21.15.32:0/1047776716 learned_addr learned my addr 172.21.15.32:0/1047776716 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:45.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca0ad9d700 1 --2- 172.21.15.32:0/1047776716 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca0c0aead0 0x7fca0c13e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:45.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca09d9b700 1 --2- 172.21.15.32:0/1047776716 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fca0c0ad1b0 0x7fca0c14f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:45.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca0a59c700 1 -- 172.21.15.32:0/1047776716 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fca0c0ad1b0 msgr2=0x7fca0c14f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:45.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca0a59c700 1 --2- 172.21.15.32:0/1047776716 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fca0c0ad1b0 0x7fca0c14f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca0a59c700 1 -- 172.21.15.32:0/1047776716 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca0c0aead0 msgr2=0x7fca0c13e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:45.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca0a59c700 1 --2- 172.21.15.32:0/1047776716 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca0c0aead0 0x7fca0c13e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca0a59c700 1 -- 172.21.15.32:0/1047776716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0c13ec80 con 0x7fca0c05f490 2020-12-05T17:33:45.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca0ad9d700 1 --2- 172.21.15.32:0/1047776716 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca0c0aead0 0x7fca0c13e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:33:45.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca0a59c700 1 --2- 172.21.15.32:0/1047776716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca0c05f490 0x7fca0c0676d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7fc9fc001450 tx=0x7fc9fc007ed0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:45.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca037fe700 1 -- 172.21.15.32:0/1047776716 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9fc005b70 con 0x7fca0c05f490 2020-12-05T17:33:45.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca037fe700 1 -- 172.21.15.32:0/1047776716 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9fc005cd0 con 0x7fca0c05f490 2020-12-05T17:33:45.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca10b86700 1 -- 172.21.15.32:0/1047776716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0c063150 con 0x7fca0c05f490 2020-12-05T17:33:45.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca037fe700 1 -- 172.21.15.32:0/1047776716 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9fc007300 con 0x7fca0c05f490 2020-12-05T17:33:45.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca10b86700 1 -- 172.21.15.32:0/1047776716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0c063dd0 con 0x7fca0c05f490 2020-12-05T17:33:45.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.659+0000 7fca10b86700 1 -- 172.21.15.32:0/1047776716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0c0642d0 con 0x7fca0c05f490 2020-12-05T17:33:45.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.663+0000 7fca037fe700 1 -- 172.21.15.32:0/1047776716 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc9fc010030 con 0x7fca0c05f490 2020-12-05T17:33:45.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.663+0000 7fca037fe700 1 --2- 172.21.15.32:0/1047776716 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc9f00554e0 0x7fc9f00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:45.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.663+0000 7fca09d9b700 1 --2- 172.21.15.32:0/1047776716 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc9f00554e0 0x7fc9f00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:45.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.663+0000 7fca037fe700 1 -- 172.21.15.32:0/1047776716 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc9fc06e170 con 0x7fca0c05f490 2020-12-05T17:33:45.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.663+0000 7fca09d9b700 1 --2- 172.21.15.32:0/1047776716 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc9f00554e0 0x7fc9f00579a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fc9f400b000 tx=0x7fc9f4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:45.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.671+0000 7fca037fe700 1 -- 172.21.15.32:0/1047776716 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc9fc044310 con 0x7fca0c05f490 2020-12-05T17:33:45.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.811+0000 7fca10b86700 1 -- 172.21.15.32:0/1047776716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca0c057050 con 0x7fca0c05f490 2020-12-05T17:33:45.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.815+0000 7fca037fe700 1 -- 172.21.15.32:0/1047776716 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16284 (secure 0 0 0) 0x7fc9fc047930 con 0x7fca0c05f490 2020-12-05T17:33:45.815 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:33:45.815 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":43,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:33:45.513330+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:33:43.025173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-12-05T17:33:44.344341+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:33:45.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.815+0000 7fca10b86700 1 -- 172.21.15.32:0/1047776716 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc9f00554e0 msgr2=0x7fc9f00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:45.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.815+0000 7fca10b86700 1 --2- 172.21.15.32:0/1047776716 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc9f00554e0 0x7fc9f00579a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fc9f400b000 tx=0x7fc9f4008040).stop 2020-12-05T17:33:45.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.815+0000 7fca10b86700 1 -- 172.21.15.32:0/1047776716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca0c05f490 msgr2=0x7fca0c0676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:45.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.815+0000 7fca10b86700 1 --2- 172.21.15.32:0/1047776716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca0c05f490 0x7fca0c0676d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7fc9fc001450 tx=0x7fc9fc007ed0).stop 2020-12-05T17:33:45.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.815+0000 7fca10b86700 1 -- 172.21.15.32:0/1047776716 shutdown_connections 2020-12-05T17:33:45.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.815+0000 7fca10b86700 1 --2- 172.21.15.32:0/1047776716 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca0c0aead0 0x7fca0c13e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.815+0000 7fca10b86700 1 --2- 172.21.15.32:0/1047776716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca0c05f490 0x7fca0c0676d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.815+0000 7fca10b86700 1 --2- 172.21.15.32:0/1047776716 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc9f00554e0 0x7fc9f00579a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.815+0000 7fca10b86700 1 --2- 172.21.15.32:0/1047776716 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fca0c0ad1b0 0x7fca0c14f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.815+0000 7fca10b86700 1 -- 172.21.15.32:0/1047776716 >> 172.21.15.32:0/1047776716 conn(0x7fca0c0b5580 msgr2=0x7fca0c1382b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:45.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.815+0000 7fca10b86700 1 -- 172.21.15.32:0/1047776716 shutdown_connections 2020-12-05T17:33:45.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.815+0000 7fca10b86700 1 -- 172.21.15.32:0/1047776716 wait complete. 2020-12-05T17:33:45.829 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:33:45.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.939+0000 7f2345a8e700 1 Processor -- start 2020-12-05T17:33:45.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.939+0000 7f2345a8e700 1 -- start start 2020-12-05T17:33:45.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.939+0000 7f2345a8e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f234005f6f0 0x7f23401428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:45.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.939+0000 7f2345a8e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f23400b0880 0x7f2340066070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:45.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.939+0000 7f2345a8e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f234005fad0 con 0x7f23400b1980 2020-12-05T17:33:45.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.939+0000 7f2345a8e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23400b2f80 con 0x7f23400b0880 2020-12-05T17:33:45.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.939+0000 7f2345a8e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2340145e40 con 0x7f234005f6f0 2020-12-05T17:33:45.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.939+0000 7f234528d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f23400b0880 0x7f2340066070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:45.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.939+0000 7f2344a8c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f234005f6f0 0x7f23401428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:45.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.939+0000 7f234528d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f23400b0880 0x7f2340066070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:54442/0 (socket says 172.21.15.32:54442) 2020-12-05T17:33:45.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.939+0000 7f234528d700 1 -- 172.21.15.32:0/949514524 learned_addr learned my addr 172.21.15.32:0/949514524 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:45.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.939+0000 7f233ffff700 1 --1- 172.21.15.32:0/949514524 >> v1:172.21.15.61:6789/0 conn(0x7f23400b1980 0x7f2340146580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:49836/0 (socket says 172.21.15.32:49836) 2020-12-05T17:33:45.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.939+0000 7f2344a8c700 1 -- 172.21.15.32:0/949514524 >> v2:172.21.15.61:3300/0 conn(0x7f23400b0880 msgr2=0x7f2340066070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:45.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.939+0000 7f2344a8c700 1 --2- 172.21.15.32:0/949514524 >> v2:172.21.15.61:3300/0 conn(0x7f23400b0880 0x7f2340066070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.939+0000 7f2344a8c700 1 -- 172.21.15.32:0/949514524 >> v1:172.21.15.61:6789/0 conn(0x7f23400b1980 legacy=0x7f2340146580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:45.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.939+0000 7f2344a8c700 1 -- 172.21.15.32:0/949514524 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23401460e0 con 0x7f234005f6f0 2020-12-05T17:33:45.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.939+0000 7f234528d700 1 --2- 172.21.15.32:0/949514524 >> v2:172.21.15.61:3300/0 conn(0x7f23400b0880 0x7f2340066070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:33:45.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.939+0000 7f2344a8c700 1 --2- 172.21.15.32:0/949514524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f234005f6f0 0x7f23401428f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f2330002f30 tx=0x7f2330006dc0).ready entity=mon.2 client_cookie=222e9c17bb59cba7 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:45.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.943+0000 7f233f7fe700 1 -- 172.21.15.32:0/949514524 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2330009760 con 0x7f234005f6f0 2020-12-05T17:33:45.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.943+0000 7f233f7fe700 1 -- 172.21.15.32:0/949514524 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23300098c0 con 0x7f234005f6f0 2020-12-05T17:33:45.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.943+0000 7f2345a8e700 1 -- 172.21.15.32:0/949514524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f234005f6f0 msgr2=0x7f23401428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:45.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.943+0000 7f2345a8e700 1 --2- 172.21.15.32:0/949514524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f234005f6f0 0x7f23401428f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f2330002f30 tx=0x7f2330006dc0).stop 2020-12-05T17:33:45.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.943+0000 7f233f7fe700 1 -- 172.21.15.32:0/949514524 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 0 0 0) 0x7f2330009c20 con 0x7f234005f6f0 2020-12-05T17:33:45.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.943+0000 7f2345a8e700 1 -- 172.21.15.32:0/949514524 shutdown_connections 2020-12-05T17:33:45.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.943+0000 7f2345a8e700 1 --2- 172.21.15.32:0/949514524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f234005f6f0 0x7f23401428f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.943+0000 7f2345a8e700 1 --2- 172.21.15.32:0/949514524 >> v2:172.21.15.61:3300/0 conn(0x7f23400b0880 0x7f2340066070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.943+0000 7f2345a8e700 1 -- 172.21.15.32:0/949514524 >> 172.21.15.32:0/949514524 conn(0x7f23400604c0 msgr2=0x7f234005c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:45.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.943+0000 7f2345a8e700 1 -- 172.21.15.32:0/949514524 shutdown_connections 2020-12-05T17:33:45.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.943+0000 7f2345a8e700 1 -- 172.21.15.32:0/949514524 wait complete. 2020-12-05T17:33:45.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.943+0000 7f2345a8e700 1 Processor -- start 2020-12-05T17:33:45.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.943+0000 7f2345a8e700 1 -- start start 2020-12-05T17:33:45.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.943+0000 7f2345a8e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f234005f6f0 0x7f23401159f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:45.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.943+0000 7f2345a8e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23400b0880 0x7f2340140f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:45.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.943+0000 7f2345a8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23400b1980 0x7f23401413f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:45.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.943+0000 7f2345a8e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2340114650 con 0x7f23400b1980 2020-12-05T17:33:45.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.943+0000 7f2345a8e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2340117fc0 con 0x7f234005f6f0 2020-12-05T17:33:45.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.943+0000 7f2345a8e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2340118220 con 0x7f23400b0880 2020-12-05T17:33:45.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.947+0000 7f234528d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23400b1980 0x7f23401413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:45.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.947+0000 7f2344a8c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f234005f6f0 0x7f23401159f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:45.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.947+0000 7f234528d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23400b1980 0x7f23401413f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34556/0 (socket says 172.21.15.32:34556) 2020-12-05T17:33:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.947+0000 7f233ffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23400b0880 0x7f2340140f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.947+0000 7f234528d700 1 -- 172.21.15.32:0/2953798316 learned_addr learned my addr 172.21.15.32:0/2953798316 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.947+0000 7f2344a8c700 1 -- 172.21.15.32:0/2953798316 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23400b0880 msgr2=0x7f2340140f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.947+0000 7f2344a8c700 1 --2- 172.21.15.32:0/2953798316 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23400b0880 0x7f2340140f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.947+0000 7f2344a8c700 1 -- 172.21.15.32:0/2953798316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23400b1980 msgr2=0x7f23401413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:45.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.947+0000 7f2344a8c700 1 --2- 172.21.15.32:0/2953798316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23400b1980 0x7f23401413f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:45.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.947+0000 7f2344a8c700 1 -- 172.21.15.32:0/2953798316 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2340113c90 con 0x7f234005f6f0 2020-12-05T17:33:45.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.947+0000 7f2344a8c700 1 --2- 172.21.15.32:0/2953798316 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f234005f6f0 0x7f23401159f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f2330002f30 tx=0x7f2330015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:45.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.947+0000 7f233dffb700 1 -- 172.21.15.32:0/2953798316 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2330011940 con 0x7f234005f6f0 2020-12-05T17:33:45.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.947+0000 7f233dffb700 1 -- 172.21.15.32:0/2953798316 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2330011aa0 con 0x7f234005f6f0 2020-12-05T17:33:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.947+0000 7f2345a8e700 1 -- 172.21.15.32:0/2953798316 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2340114130 con 0x7f234005f6f0 2020-12-05T17:33:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.947+0000 7f233dffb700 1 -- 172.21.15.32:0/2953798316 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2330011e00 con 0x7f234005f6f0 2020-12-05T17:33:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.947+0000 7f2345a8e700 1 -- 172.21.15.32:0/2953798316 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2340116100 con 0x7f234005f6f0 2020-12-05T17:33:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.947+0000 7f2345a8e700 1 -- 172.21.15.32:0/2953798316 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2340115280 con 0x7f234005f6f0 2020-12-05T17:33:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.947+0000 7f233dffb700 1 -- 172.21.15.32:0/2953798316 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2330017070 con 0x7f234005f6f0 2020-12-05T17:33:45.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.951+0000 7f233dffb700 1 --2- 172.21.15.32:0/2953798316 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2324055500 0x7f23240579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:45.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.951+0000 7f233dffb700 1 -- 172.21.15.32:0/2953798316 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f233006e810 con 0x7f234005f6f0 2020-12-05T17:33:45.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.951+0000 7f233ffff700 1 --2- 172.21.15.32:0/2953798316 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2324055500 0x7f23240579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:45.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.951+0000 7f233ffff700 1 --2- 172.21.15.32:0/2953798316 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2324055500 0x7f23240579c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f232800d000 tx=0x7f2328007b50).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:45.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:45.959+0000 7f233dffb700 1 -- 172.21.15.32:0/2953798316 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f23300448a0 con 0x7f234005f6f0 2020-12-05T17:33:46.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.099+0000 7f2345a8e700 1 -- 172.21.15.32:0/2953798316 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2340115280 con 0x7f234005f6f0 2020-12-05T17:33:46.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.103+0000 7f233dffb700 1 -- 172.21.15.32:0/2953798316 <== mon.1 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16284 (secure 0 0 0) 0x7f2340115280 con 0x7f234005f6f0 2020-12-05T17:33:46.103 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:33:46.104 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":43,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:33:45.513330+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:33:43.025173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-12-05T17:33:44.344341+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:33:46.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.103+0000 7f2345a8e700 1 -- 172.21.15.32:0/2953798316 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2324055500 msgr2=0x7f23240579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:46.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.103+0000 7f2345a8e700 1 --2- 172.21.15.32:0/2953798316 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2324055500 0x7f23240579c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f232800d000 tx=0x7f2328007b50).stop 2020-12-05T17:33:46.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.103+0000 7f2345a8e700 1 -- 172.21.15.32:0/2953798316 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f234005f6f0 msgr2=0x7f23401159f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:46.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.103+0000 7f2345a8e700 1 --2- 172.21.15.32:0/2953798316 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f234005f6f0 0x7f23401159f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f2330002f30 tx=0x7f2330015000).stop 2020-12-05T17:33:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.103+0000 7f2345a8e700 1 -- 172.21.15.32:0/2953798316 shutdown_connections 2020-12-05T17:33:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.103+0000 7f2345a8e700 1 --2- 172.21.15.32:0/2953798316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23400b1980 0x7f23401413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.103+0000 7f2345a8e700 1 --2- 172.21.15.32:0/2953798316 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f234005f6f0 0x7f23401159f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.103+0000 7f2345a8e700 1 --2- 172.21.15.32:0/2953798316 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2324055500 0x7f23240579c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.103+0000 7f2345a8e700 1 --2- 172.21.15.32:0/2953798316 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23400b0880 0x7f2340140f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:46.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.103+0000 7f2345a8e700 1 -- 172.21.15.32:0/2953798316 >> 172.21.15.32:0/2953798316 conn(0x7f23400604c0 msgr2=0x7f23400b64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:46.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.103+0000 7f2345a8e700 1 -- 172.21.15.32:0/2953798316 shutdown_connections 2020-12-05T17:33:46.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.103+0000 7f2345a8e700 1 -- 172.21.15.32:0/2953798316 wait complete. 2020-12-05T17:33:46.121 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-12-05T17:33:46.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.247+0000 7f4e4759e700 1 Processor -- start 2020-12-05T17:33:46.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.247+0000 7f4e4759e700 1 -- start start 2020-12-05T17:33:46.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.247+0000 7f4e4759e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f4e48061920 0x7f4e480aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:46.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.247+0000 7f4e4759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e480b1380 con 0x7f4e480621b0 2020-12-05T17:33:46.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.247+0000 7f4e4759e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e480ae620 con 0x7f4e48061d00 2020-12-05T17:33:46.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.247+0000 7f4e4759e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e48062ba0 con 0x7f4e48061920 2020-12-05T17:33:46.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.247+0000 7f4e46d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4e480621b0 0x7f4e4813e000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37212/0 (socket says 172.21.15.32:37212) 2020-12-05T17:33:46.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.247+0000 7f4e46d9d700 1 -- 172.21.15.32:0/3026037637 learned_addr learned my addr 172.21.15.32:0/3026037637 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:33:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.247+0000 7f4e4659c700 1 --2- 172.21.15.32:0/3026037637 >> v2:172.21.15.61:3300/0 conn(0x7f4e48061920 0x7f4e480aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.247+0000 7f4e45d9b700 1 -- 172.21.15.32:0/3026037637 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1246402620 0 0) 0x7f4e480b1380 con 0x7f4e480621b0 2020-12-05T17:33:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.247+0000 7f4e45d9b700 1 -- 172.21.15.32:0/3026037637 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e28003230 con 0x7f4e480621b0 2020-12-05T17:33:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.247+0000 7f4e45d9b700 1 -- 172.21.15.32:0/3026037637 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029834494 0 0) 0x7f4e480ae620 con 0x7f4e48061d00 2020-12-05T17:33:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.247+0000 7f4e45d9b700 1 -- 172.21.15.32:0/3026037637 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e480b1380 con 0x7f4e48061d00 2020-12-05T17:33:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.247+0000 7f4e45d9b700 1 -- 172.21.15.32:0/3026037637 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2561786889 0 0) 0x7f4e28003230 con 0x7f4e480621b0 2020-12-05T17:33:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.247+0000 7f4e4659c700 1 -- 172.21.15.32:0/3026037637 >> v1:172.21.15.61:6789/0 conn(0x7f4e48061d00 legacy=0x7f4e48142200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.247+0000 7f4e4659c700 1 -- 172.21.15.32:0/3026037637 >> v1:172.21.15.32:6789/0 conn(0x7f4e480621b0 legacy=0x7f4e4813e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.247+0000 7f4e4659c700 1 -- 172.21.15.32:0/3026037637 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e480ae8c0 con 0x7f4e48061920 2020-12-05T17:33:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.247+0000 7f4e4659c700 1 --2- 172.21.15.32:0/3026037637 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4e48061920 0x7f4e480aed60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f4e40001450 tx=0x7f4e38006dc0).ready entity=mon.1 client_cookie=411943b5bef4bc56 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.247+0000 7f4e45d9b700 1 -- 172.21.15.32:0/3026037637 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e40001a50 con 0x7f4e48061920 2020-12-05T17:33:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.247+0000 7f4e45d9b700 1 -- 172.21.15.32:0/3026037637 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3800d610 con 0x7f4e48061920 2020-12-05T17:33:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.247+0000 7f4e45d9b700 1 -- 172.21.15.32:0/3026037637 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e3800d780 con 0x7f4e48061920 2020-12-05T17:33:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.251+0000 7f4e4759e700 1 -- 172.21.15.32:0/3026037637 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4e48061920 msgr2=0x7f4e480aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.251+0000 7f4e4759e700 1 --2- 172.21.15.32:0/3026037637 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4e48061920 0x7f4e480aed60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f4e40001450 tx=0x7f4e38006dc0).stop 2020-12-05T17:33:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.251+0000 7f4e4759e700 1 -- 172.21.15.32:0/3026037637 shutdown_connections 2020-12-05T17:33:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.251+0000 7f4e4759e700 1 --2- 172.21.15.32:0/3026037637 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4e48061920 0x7f4e480aed60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.251+0000 7f4e4759e700 1 -- 172.21.15.32:0/3026037637 >> 172.21.15.32:0/3026037637 conn(0x7f4e480b5620 msgr2=0x7f4e480b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.251+0000 7f4e4759e700 1 -- 172.21.15.32:0/3026037637 shutdown_connections 2020-12-05T17:33:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.251+0000 7f4e4759e700 1 -- 172.21.15.32:0/3026037637 wait complete. 2020-12-05T17:33:46.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.251+0000 7f4e4759e700 1 Processor -- start 2020-12-05T17:33:46.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.251+0000 7f4e4759e700 1 -- start start 2020-12-05T17:33:46.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e4759e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4e48061920 0x7f4e4810e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:46.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e4759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e48061d00 0x7f4e48145b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:46.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e4759e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4e480621b0 0x7f4e480647a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:46.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e4759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e48065e10 con 0x7f4e48061d00 2020-12-05T17:33:46.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e4759e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e481103b0 con 0x7f4e480621b0 2020-12-05T17:33:46.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e4759e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e48065b70 con 0x7f4e48061920 2020-12-05T17:33:46.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e3dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e48061d00 0x7f4e48145b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:46.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e3dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e48061d00 0x7f4e48145b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34568/0 (socket says 172.21.15.32:34568) 2020-12-05T17:33:46.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e3dd9b700 1 -- 172.21.15.32:0/838473719 learned_addr learned my addr 172.21.15.32:0/838473719 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:46.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e4659c700 1 --2- 172.21.15.32:0/838473719 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4e48061920 0x7f4e4810e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:46.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e46d9d700 1 --2- 172.21.15.32:0/838473719 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4e480621b0 0x7f4e480647a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:46.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e3dd9b700 1 -- 172.21.15.32:0/838473719 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4e48061920 msgr2=0x7f4e4810e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:46.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e3dd9b700 1 --2- 172.21.15.32:0/838473719 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4e48061920 0x7f4e4810e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:46.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e3dd9b700 1 -- 172.21.15.32:0/838473719 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4e480621b0 msgr2=0x7f4e480647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:46.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e3dd9b700 1 --2- 172.21.15.32:0/838473719 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4e480621b0 0x7f4e480647a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:46.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e3dd9b700 1 -- 172.21.15.32:0/838473719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e480b2230 con 0x7f4e48061d00 2020-12-05T17:33:46.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e46d9d700 1 --2- 172.21.15.32:0/838473719 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4e480621b0 0x7f4e480647a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:33:46.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e4659c700 1 --2- 172.21.15.32:0/838473719 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4e48061920 0x7f4e4810e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:33:46.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e3dd9b700 1 --2- 172.21.15.32:0/838473719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e48061d00 0x7f4e48145b90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f4e30001110 tx=0x7f4e30007b00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:46.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e3ffff700 1 -- 172.21.15.32:0/838473719 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e30010510 con 0x7f4e48061d00 2020-12-05T17:33:46.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e4759e700 1 -- 172.21.15.32:0/838473719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e48064c90 con 0x7f4e48061d00 2020-12-05T17:33:46.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e3ffff700 1 -- 172.21.15.32:0/838473719 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e30010670 con 0x7f4e48061d00 2020-12-05T17:33:46.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e3ffff700 1 -- 172.21.15.32:0/838473719 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e300107e0 con 0x7f4e48061d00 2020-12-05T17:33:46.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.255+0000 7f4e4759e700 1 -- 172.21.15.32:0/838473719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e4810df20 con 0x7f4e48061d00 2020-12-05T17:33:46.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.259+0000 7f4e4759e700 1 -- 172.21.15.32:0/838473719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e48057050 con 0x7f4e48061d00 2020-12-05T17:33:46.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.259+0000 7f4e3ffff700 1 -- 172.21.15.32:0/838473719 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4e30010070 con 0x7f4e48061d00 2020-12-05T17:33:46.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.259+0000 7f4e3ffff700 1 --2- 172.21.15.32:0/838473719 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4e2c0575c0 0x7f4e2c059a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:46.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.259+0000 7f4e3ffff700 1 -- 172.21.15.32:0/838473719 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4e3006db40 con 0x7f4e48061d00 2020-12-05T17:33:46.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.259+0000 7f4e4659c700 1 --2- 172.21.15.32:0/838473719 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4e2c0575c0 0x7f4e2c059a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:46.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.259+0000 7f4e4659c700 1 --2- 172.21.15.32:0/838473719 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4e2c0575c0 0x7f4e2c059a80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f4e38018000 tx=0x7f4e38015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:46.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.267+0000 7f4e3ffff700 1 -- 172.21.15.32:0/838473719 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4e30043c40 con 0x7f4e48061d00 2020-12-05T17:33:46.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.459+0000 7f4e4759e700 1 -- 172.21.15.32:0/838473719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4e48110520 con 0x7f4e48061d00 2020-12-05T17:33:46.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.995+0000 7f4e3ffff700 1 -- 172.21.15.32:0/838473719 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4e30047260 con 0x7f4e48061d00 2020-12-05T17:33:47.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.999+0000 7f4e4759e700 1 -- 172.21.15.32:0/838473719 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4e2c0575c0 msgr2=0x7f4e2c059a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:47.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.999+0000 7f4e4759e700 1 --2- 172.21.15.32:0/838473719 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4e2c0575c0 0x7f4e2c059a80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f4e38018000 tx=0x7f4e38015000).stop 2020-12-05T17:33:47.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.999+0000 7f4e4759e700 1 -- 172.21.15.32:0/838473719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e48061d00 msgr2=0x7f4e48145b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:47.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.999+0000 7f4e4759e700 1 --2- 172.21.15.32:0/838473719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e48061d00 0x7f4e48145b90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f4e30001110 tx=0x7f4e30007b00).stop 2020-12-05T17:33:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.999+0000 7f4e4759e700 1 -- 172.21.15.32:0/838473719 shutdown_connections 2020-12-05T17:33:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.999+0000 7f4e4759e700 1 --2- 172.21.15.32:0/838473719 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4e480621b0 0x7f4e480647a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.999+0000 7f4e4759e700 1 --2- 172.21.15.32:0/838473719 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4e2c0575c0 0x7f4e2c059a80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.999+0000 7f4e4759e700 1 --2- 172.21.15.32:0/838473719 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4e48061920 0x7f4e4810e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.999+0000 7f4e4759e700 1 --2- 172.21.15.32:0/838473719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e48061d00 0x7f4e48145b90 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.999+0000 7f4e4759e700 1 -- 172.21.15.32:0/838473719 >> 172.21.15.32:0/838473719 conn(0x7f4e480b5620 msgr2=0x7f4e480b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:46.999+0000 7f4e4759e700 1 -- 172.21.15.32:0/838473719 shutdown_connections 2020-12-05T17:33:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.003+0000 7f4e4759e700 1 -- 172.21.15.32:0/838473719 wait complete. 2020-12-05T17:33:47.019 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:33:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a03443700 1 Processor -- start 2020-12-05T17:33:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a03443700 1 -- start start 2020-12-05T17:33:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a03443700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89fc0af810 0x7f89fc143b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a03443700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f89fc0aff10 0x7f89fc13d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a03443700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89fc063040 con 0x7f89fc05f6f0 2020-12-05T17:33:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a03443700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89fc05fad0 con 0x7f89fc0aff10 2020-12-05T17:33:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a03443700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0b3030 con 0x7f89fc0af810 2020-12-05T17:33:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a02441700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f89fc05f6f0 0x7f89fc0b1d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37222/0 (socket says 172.21.15.32:37222) 2020-12-05T17:33:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a02441700 1 -- 172.21.15.32:0/510376133 learned_addr learned my addr 172.21.15.32:0/510376133 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:33:47.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a01c40700 1 --2- 172.21.15.32:0/510376133 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89fc0af810 0x7f89fc143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:47.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a02c42700 1 --2- 172.21.15.32:0/510376133 >> v2:172.21.15.61:3300/0 conn(0x7f89fc0aff10 0x7f89fc13d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:47.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a0143f700 1 -- 172.21.15.32:0/510376133 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1599699409 0 0) 0x7f89fc063040 con 0x7f89fc05f6f0 2020-12-05T17:33:47.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a0143f700 1 -- 172.21.15.32:0/510376133 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89e4003230 con 0x7f89fc05f6f0 2020-12-05T17:33:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a0143f700 1 -- 172.21.15.32:0/510376133 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1908089847 0 0) 0x7f89e4003230 con 0x7f89fc05f6f0 2020-12-05T17:33:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a0143f700 1 -- 172.21.15.32:0/510376133 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89fc0af810 msgr2=0x7f89fc143b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a0143f700 1 --2- 172.21.15.32:0/510376133 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89fc0af810 0x7f89fc143b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a0143f700 1 -- 172.21.15.32:0/510376133 >> v2:172.21.15.61:3300/0 conn(0x7f89fc0aff10 msgr2=0x7f89fc13d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a0143f700 1 --2- 172.21.15.32:0/510376133 >> v2:172.21.15.61:3300/0 conn(0x7f89fc0aff10 0x7f89fc13d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a0143f700 1 -- 172.21.15.32:0/510376133 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc0b24a0 con 0x7f89fc05f6f0 2020-12-05T17:33:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a0143f700 1 -- 172.21.15.32:0/510376133 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f89f80016d0 con 0x7f89fc05f6f0 2020-12-05T17:33:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a0143f700 1 -- 172.21.15.32:0/510376133 >> v1:172.21.15.32:6789/0 conn(0x7f89fc05f6f0 legacy=0x7f89fc0b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a0143f700 1 --2- 172.21.15.32:0/510376133 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f89e4003720 0x7f89e4003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a0143f700 1 --2- 172.21.15.32:0/510376133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e40044d0 0x7f89e4004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a0143f700 1 --2- 172.21.15.32:0/510376133 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89fc0af810 0x7f89e4004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a0143f700 1 -- 172.21.15.32:0/510376133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc05fad0 con 0x7f89e40044d0 2020-12-05T17:33:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a0143f700 1 -- 172.21.15.32:0/510376133 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0b3030 con 0x7f89e4003720 2020-12-05T17:33:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a0143f700 1 -- 172.21.15.32:0/510376133 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e4004260 con 0x7f89fc0af810 2020-12-05T17:33:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a01c40700 1 --2- 172.21.15.32:0/510376133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e40044d0 0x7f89e4004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.139+0000 7f8a02441700 1 --2- 172.21.15.32:0/510376133 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f89e4003720 0x7f89e4003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.143+0000 7f8a02c42700 1 --2- 172.21.15.32:0/510376133 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89fc0af810 0x7f89e4004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.143+0000 7f8a01c40700 1 -- 172.21.15.32:0/510376133 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89fc0af810 msgr2=0x7f89e4004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.143+0000 7f8a01c40700 1 --2- 172.21.15.32:0/510376133 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89fc0af810 0x7f89e4004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.143+0000 7f8a01c40700 1 -- 172.21.15.32:0/510376133 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f89e4003720 msgr2=0x7f89e4003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.143+0000 7f8a01c40700 1 --2- 172.21.15.32:0/510376133 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f89e4003720 0x7f89e4003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.143+0000 7f8a01c40700 1 -- 172.21.15.32:0/510376133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e4003230 con 0x7f89e40044d0 2020-12-05T17:33:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.143+0000 7f8a01c40700 1 --2- 172.21.15.32:0/510376133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e40044d0 0x7f89e4004950 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f89ec00c000 tx=0x7f89ec008040).ready entity=mon.0 client_cookie=56fdd9890bb35c6e server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.143+0000 7f8a0143f700 1 -- 172.21.15.32:0/510376133 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89ec00e970 con 0x7f89e40044d0 2020-12-05T17:33:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.143+0000 7f8a0143f700 1 -- 172.21.15.32:0/510376133 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ec00ead0 con 0x7f89e40044d0 2020-12-05T17:33:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.143+0000 7f8a0143f700 1 -- 172.21.15.32:0/510376133 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89ec00ee30 con 0x7f89e40044d0 2020-12-05T17:33:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.143+0000 7f8a03443700 1 -- 172.21.15.32:0/510376133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e40044d0 msgr2=0x7f89e4004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.143+0000 7f8a03443700 1 --2- 172.21.15.32:0/510376133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e40044d0 0x7f89e4004950 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f89ec00c000 tx=0x7f89ec008040).stop 2020-12-05T17:33:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.143+0000 7f8a02c42700 1 -- 172.21.15.32:0/510376133 reap_dead start 2020-12-05T17:33:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.143+0000 7f8a03443700 1 -- 172.21.15.32:0/510376133 shutdown_connections 2020-12-05T17:33:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.143+0000 7f8a03443700 1 -- 172.21.15.32:0/510376133 >> 172.21.15.32:0/510376133 conn(0x7f89fc0604c0 msgr2=0x7f89fc05c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.143+0000 7f8a03443700 1 -- 172.21.15.32:0/510376133 shutdown_connections 2020-12-05T17:33:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.143+0000 7f8a03443700 1 -- 172.21.15.32:0/510376133 wait complete. 2020-12-05T17:33:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f8a03443700 1 Processor -- start 2020-12-05T17:33:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f8a03443700 1 -- start start 2020-12-05T17:33:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f8a03443700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89fc1388f0 0x7f89fc0ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f8a03443700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89fc145bd0 0x7f89fc0651f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f8a03443700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f89fc0656e0 0x7f89fc154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f8a03443700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc145990 con 0x7f89fc145bd0 2020-12-05T17:33:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f8a03443700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc138cd0 con 0x7f89fc0656e0 2020-12-05T17:33:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f8a03443700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc1456e0 con 0x7f89fc1388f0 2020-12-05T17:33:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f8a01c40700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89fc145bd0 0x7f89fc0651f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f8a01c40700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89fc145bd0 0x7f89fc0651f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34588/0 (socket says 172.21.15.32:34588) 2020-12-05T17:33:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f8a01c40700 1 -- 172.21.15.32:0/1273996705 learned_addr learned my addr 172.21.15.32:0/1273996705 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f8a02441700 1 --2- 172.21.15.32:0/1273996705 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89fc1388f0 0x7f89fc0ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f8a02c42700 1 --2- 172.21.15.32:0/1273996705 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f89fc0656e0 0x7f89fc154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f8a01c40700 1 -- 172.21.15.32:0/1273996705 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89fc1388f0 msgr2=0x7f89fc0ad000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f8a01c40700 1 --2- 172.21.15.32:0/1273996705 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89fc1388f0 0x7f89fc0ad000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f8a01c40700 1 -- 172.21.15.32:0/1273996705 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f89fc0656e0 msgr2=0x7f89fc154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f8a01c40700 1 --2- 172.21.15.32:0/1273996705 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f89fc0656e0 0x7f89fc154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f8a01c40700 1 -- 172.21.15.32:0/1273996705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc146740 con 0x7f89fc145bd0 2020-12-05T17:33:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f8a01c40700 1 --2- 172.21.15.32:0/1273996705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89fc145bd0 0x7f89fc0651f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f89fc145870 tx=0x7f89ec01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f89eb7fe700 1 -- 172.21.15.32:0/1273996705 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89ec012d50 con 0x7f89fc145bd0 2020-12-05T17:33:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f89eb7fe700 1 -- 172.21.15.32:0/1273996705 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ec01c430 con 0x7f89fc145bd0 2020-12-05T17:33:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f8a03443700 1 -- 172.21.15.32:0/1273996705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89fc1469e0 con 0x7f89fc145bd0 2020-12-05T17:33:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f89eb7fe700 1 -- 172.21.15.32:0/1273996705 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89ec01c790 con 0x7f89fc145bd0 2020-12-05T17:33:47.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.147+0000 7f8a03443700 1 -- 172.21.15.32:0/1273996705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89fc065b60 con 0x7f89fc145bd0 2020-12-05T17:33:47.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.151+0000 7f8a03443700 1 -- 172.21.15.32:0/1273996705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89fc057050 con 0x7f89fc145bd0 2020-12-05T17:33:47.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.151+0000 7f89eb7fe700 1 -- 172.21.15.32:0/1273996705 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f89ec02c070 con 0x7f89fc145bd0 2020-12-05T17:33:47.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.151+0000 7f89eb7fe700 1 --2- 172.21.15.32:0/1273996705 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f89dc0553b0 0x7f89dc057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:47.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.151+0000 7f89eb7fe700 1 -- 172.21.15.32:0/1273996705 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f89ec06fa70 con 0x7f89fc145bd0 2020-12-05T17:33:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.151+0000 7f8a02441700 1 --2- 172.21.15.32:0/1273996705 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f89dc0553b0 0x7f89dc057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.151+0000 7f8a02441700 1 --2- 172.21.15.32:0/1273996705 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f89dc0553b0 0x7f89dc057870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f89f8005000 tx=0x7f89f80077c0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.159+0000 7f89eb7fe700 1 -- 172.21.15.32:0/1273996705 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f89ec045c50 con 0x7f89fc145bd0 2020-12-05T17:33:47.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.343+0000 7f8a03443700 1 -- 172.21.15.32:0/1273996705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89fc1447d0 con 0x7f89fc145bd0 2020-12-05T17:33:47.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.347+0000 7f89eb7fe700 1 -- 172.21.15.32:0/1273996705 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2138 (secure 0 0 0) 0x7f89ec06fe60 con 0x7f89fc145bd0 2020-12-05T17:33:47.347 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:33:47.347 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4919,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/3405415630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3405415630},{"type":"v1","addr":"172.21.15.61:6833","nonce":3405415630}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:33:45.513696+0000","modified":"2020-12-05T17:33:46.989395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5301":{"gid":5301,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3450615455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3450615455},{"type":"v1","addr":"172.21.15.32:6835","nonce":3450615455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-12-05T17:33:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.347+0000 7f8a03443700 1 -- 172.21.15.32:0/1273996705 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f89dc0553b0 msgr2=0x7f89dc057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.347+0000 7f8a03443700 1 --2- 172.21.15.32:0/1273996705 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f89dc0553b0 0x7f89dc057870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f89f8005000 tx=0x7f89f80077c0).stop 2020-12-05T17:33:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.347+0000 7f8a03443700 1 -- 172.21.15.32:0/1273996705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89fc145bd0 msgr2=0x7f89fc0651f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.347+0000 7f8a03443700 1 --2- 172.21.15.32:0/1273996705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89fc145bd0 0x7f89fc0651f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f89fc145870 tx=0x7f89ec01a000).stop 2020-12-05T17:33:47.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.347+0000 7f8a03443700 1 -- 172.21.15.32:0/1273996705 shutdown_connections 2020-12-05T17:33:47.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.347+0000 7f8a03443700 1 --2- 172.21.15.32:0/1273996705 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f89fc0656e0 0x7f89fc154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.347+0000 7f8a03443700 1 --2- 172.21.15.32:0/1273996705 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f89dc0553b0 0x7f89dc057870 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.347+0000 7f8a03443700 1 --2- 172.21.15.32:0/1273996705 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89fc1388f0 0x7f89fc0ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.347+0000 7f8a03443700 1 --2- 172.21.15.32:0/1273996705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89fc145bd0 0x7f89fc0651f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.347+0000 7f8a03443700 1 -- 172.21.15.32:0/1273996705 >> 172.21.15.32:0/1273996705 conn(0x7f89fc0604c0 msgr2=0x7f89fc0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:47.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.347+0000 7f8a03443700 1 -- 172.21.15.32:0/1273996705 shutdown_connections 2020-12-05T17:33:47.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.347+0000 7f8a03443700 1 -- 172.21.15.32:0/1273996705 wait complete. 2020-12-05T17:33:47.352 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 17 2020-12-05T17:33:47.361 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:33:47.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc96b395700 1 Processor -- start 2020-12-05T17:33:47.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc96b395700 1 -- start start 2020-12-05T17:33:47.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc96b395700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc96405fe00 0x7fc964061910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:47.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc96b395700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc9640b00a0 0x7fc96414a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:47.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc96b395700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9640601e0 con 0x7fc9640ac590 2020-12-05T17:33:47.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc96b395700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9640ac1a0 con 0x7fc9640b00a0 2020-12-05T17:33:47.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc96b395700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc964061fa0 con 0x7fc96405fe00 2020-12-05T17:33:47.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc968930700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc9640b00a0 0x7fc96414a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:47.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc968930700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc9640b00a0 0x7fc96414a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34594/0 (socket says 172.21.15.32:34594) 2020-12-05T17:33:47.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc968930700 1 -- 172.21.15.32:0/3779920637 learned_addr learned my addr 172.21.15.32:0/3779920637 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:47.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc969131700 1 --2- 172.21.15.32:0/3779920637 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc96405fe00 0x7fc964061910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:47.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc968930700 1 -- 172.21.15.32:0/3779920637 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc96405fe00 msgr2=0x7fc964061910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:47.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc968930700 1 --2- 172.21.15.32:0/3779920637 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc96405fe00 0x7fc964061910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc968930700 1 -- 172.21.15.32:0/3779920637 >> v1:172.21.15.61:6789/0 conn(0x7fc9640ac590 legacy=0x7fc9640ac9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc968930700 1 -- 172.21.15.32:0/3779920637 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc964062240 con 0x7fc9640b00a0 2020-12-05T17:33:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc969131700 1 --2- 172.21.15.32:0/3779920637 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc96405fe00 0x7fc964061910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:33:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc968930700 1 --2- 172.21.15.32:0/3779920637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9640b00a0 0x7fc96414a3d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7fc958002f30 tx=0x7fc958006e80).ready entity=mon.0 client_cookie=d3b76db1902af377 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc963fff700 1 -- 172.21.15.32:0/3779920637 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9580097f0 con 0x7fc9640b00a0 2020-12-05T17:33:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc963fff700 1 -- 172.21.15.32:0/3779920637 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc958009950 con 0x7fc9640b00a0 2020-12-05T17:33:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc963fff700 1 -- 172.21.15.32:0/3779920637 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc958009cb0 con 0x7fc9640b00a0 2020-12-05T17:33:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc96b395700 1 -- 172.21.15.32:0/3779920637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9640b00a0 msgr2=0x7fc96414a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc96b395700 1 --2- 172.21.15.32:0/3779920637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9640b00a0 0x7fc96414a3d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7fc958002f30 tx=0x7fc958006e80).stop 2020-12-05T17:33:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc96b395700 1 -- 172.21.15.32:0/3779920637 shutdown_connections 2020-12-05T17:33:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc96b395700 1 --2- 172.21.15.32:0/3779920637 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc96405fe00 0x7fc964061910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc96b395700 1 --2- 172.21.15.32:0/3779920637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9640b00a0 0x7fc96414a3d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc96b395700 1 -- 172.21.15.32:0/3779920637 >> 172.21.15.32:0/3779920637 conn(0x7fc9640b57c0 msgr2=0x7fc96405f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc96b395700 1 -- 172.21.15.32:0/3779920637 shutdown_connections 2020-12-05T17:33:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.475+0000 7fc96b395700 1 -- 172.21.15.32:0/3779920637 wait complete. 2020-12-05T17:33:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc96b395700 1 Processor -- start 2020-12-05T17:33:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc96b395700 1 -- start start 2020-12-05T17:33:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc96b395700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc96405fe00 0x7fc964159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc96b395700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9640ac590 0x7fc964141880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc96b395700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9640b00a0 0x7fc96414cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc96b395700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96414df70 con 0x7fc9640b00a0 2020-12-05T17:33:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc96b395700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96414e210 con 0x7fc9640ac590 2020-12-05T17:33:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc96b395700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96414d9c0 con 0x7fc96405fe00 2020-12-05T17:33:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc969932700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9640b00a0 0x7fc96414cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc969131700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc96405fe00 0x7fc964159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc969932700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9640b00a0 0x7fc96414cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34604/0 (socket says 172.21.15.32:34604) 2020-12-05T17:33:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc969932700 1 -- 172.21.15.32:0/3130946135 learned_addr learned my addr 172.21.15.32:0/3130946135 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc968930700 1 --2- 172.21.15.32:0/3130946135 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9640ac590 0x7fc964141880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc969932700 1 -- 172.21.15.32:0/3130946135 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc96405fe00 msgr2=0x7fc964159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc969932700 1 --2- 172.21.15.32:0/3130946135 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc96405fe00 0x7fc964159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc969932700 1 -- 172.21.15.32:0/3130946135 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9640ac590 msgr2=0x7fc964141880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc969932700 1 --2- 172.21.15.32:0/3130946135 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9640ac590 0x7fc964141880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc969932700 1 -- 172.21.15.32:0/3130946135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc964159ef0 con 0x7fc9640b00a0 2020-12-05T17:33:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc968930700 1 --2- 172.21.15.32:0/3130946135 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9640ac590 0x7fc964141880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:33:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc969932700 1 --2- 172.21.15.32:0/3130946135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9640b00a0 0x7fc96414cda0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7fc96405d3a0 tx=0x7fc95c007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc9627fc700 1 -- 172.21.15.32:0/3130946135 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc95c013410 con 0x7fc9640b00a0 2020-12-05T17:33:47.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc9627fc700 1 -- 172.21.15.32:0/3130946135 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95c013570 con 0x7fc9640b00a0 2020-12-05T17:33:47.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc96b395700 1 -- 172.21.15.32:0/3130946135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc964141d70 con 0x7fc9640b00a0 2020-12-05T17:33:47.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc9627fc700 1 -- 172.21.15.32:0/3130946135 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc95c0138d0 con 0x7fc9640b00a0 2020-12-05T17:33:47.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.479+0000 7fc96b395700 1 -- 172.21.15.32:0/3130946135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96414bb00 con 0x7fc9640b00a0 2020-12-05T17:33:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.483+0000 7fc96b395700 1 -- 172.21.15.32:0/3130946135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc964057050 con 0x7fc9640b00a0 2020-12-05T17:33:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.483+0000 7fc9627fc700 1 -- 172.21.15.32:0/3130946135 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc95c019020 con 0x7fc9640b00a0 2020-12-05T17:33:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.483+0000 7fc9627fc700 1 --2- 172.21.15.32:0/3130946135 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc944055440 0x7fc944057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.483+0000 7fc969131700 1 --2- 172.21.15.32:0/3130946135 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc944055440 0x7fc944057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.483+0000 7fc9627fc700 1 -- 172.21.15.32:0/3130946135 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc95c06da70 con 0x7fc9640b00a0 2020-12-05T17:33:47.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.483+0000 7fc969131700 1 --2- 172.21.15.32:0/3130946135 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc944055440 0x7fc944057900 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fc95400d000 tx=0x7fc954008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:47.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.491+0000 7fc9627fc700 1 -- 172.21.15.32:0/3130946135 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc95c043aa0 con 0x7fc9640b00a0 2020-12-05T17:33:47.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.635+0000 7fc96b395700 1 -- 172.21.15.32:0/3130946135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc964069690 con 0x7fc9640b00a0 2020-12-05T17:33:47.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.635+0000 7fc9627fc700 1 -- 172.21.15.32:0/3130946135 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16271 (secure 0 0 0) 0x7fc95c0470c0 con 0x7fc9640b00a0 2020-12-05T17:33:47.637 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:33:47.638 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":44,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:33:46.516416+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:33:43.025173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-12-05T17:33:44.344341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:33:47.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.639+0000 7fc96b395700 1 -- 172.21.15.32:0/3130946135 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc944055440 msgr2=0x7fc944057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:47.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.639+0000 7fc96b395700 1 --2- 172.21.15.32:0/3130946135 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc944055440 0x7fc944057900 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fc95400d000 tx=0x7fc954008040).stop 2020-12-05T17:33:47.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.639+0000 7fc96b395700 1 -- 172.21.15.32:0/3130946135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9640b00a0 msgr2=0x7fc96414cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:47.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.639+0000 7fc96b395700 1 --2- 172.21.15.32:0/3130946135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9640b00a0 0x7fc96414cda0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7fc96405d3a0 tx=0x7fc95c007b50).stop 2020-12-05T17:33:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.639+0000 7fc96b395700 1 -- 172.21.15.32:0/3130946135 shutdown_connections 2020-12-05T17:33:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.639+0000 7fc96b395700 1 --2- 172.21.15.32:0/3130946135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9640b00a0 0x7fc96414cda0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.639+0000 7fc96b395700 1 --2- 172.21.15.32:0/3130946135 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc944055440 0x7fc944057900 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.639+0000 7fc96b395700 1 --2- 172.21.15.32:0/3130946135 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc96405fe00 0x7fc964159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.639+0000 7fc96b395700 1 --2- 172.21.15.32:0/3130946135 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9640ac590 0x7fc964141880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.639+0000 7fc96b395700 1 -- 172.21.15.32:0/3130946135 >> 172.21.15.32:0/3130946135 conn(0x7fc9640b57c0 msgr2=0x7fc9640aebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:47.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.639+0000 7fc96b395700 1 -- 172.21.15.32:0/3130946135 shutdown_connections 2020-12-05T17:33:47.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.639+0000 7fc96b395700 1 -- 172.21.15.32:0/3130946135 wait complete. 2020-12-05T17:33:47.651 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:33:47.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.771+0000 7f064d9e6700 1 Processor -- start 2020-12-05T17:33:47.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.771+0000 7f064d9e6700 1 -- start start 2020-12-05T17:33:47.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.771+0000 7f064d9e6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f064805feb0 0x7f06480ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:47.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.771+0000 7f064d9e6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0648061780 0x7f064814e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.771+0000 7f064d9e6700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06480b0380 con 0x7f064805f6f0 2020-12-05T17:33:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.771+0000 7f064d9e6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06480aeaf0 con 0x7f0648061780 2020-12-05T17:33:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.771+0000 7f064d9e6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06480aef70 con 0x7f064805feb0 2020-12-05T17:33:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.771+0000 7f0647fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0648061780 0x7f064814e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.771+0000 7f0647fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0648061780 0x7f064814e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34612/0 (socket says 172.21.15.32:34612) 2020-12-05T17:33:47.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.771+0000 7f0647fff700 1 -- 172.21.15.32:0/1535344036 learned_addr learned my addr 172.21.15.32:0/1535344036 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:47.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.771+0000 7f0646ffd700 1 --2- 172.21.15.32:0/1535344036 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f064805feb0 0x7f06480ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:47.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.771+0000 7f0647fff700 1 -- 172.21.15.32:0/1535344036 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f064805feb0 msgr2=0x7f06480ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:47.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.771+0000 7f0647fff700 1 --2- 172.21.15.32:0/1535344036 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f064805feb0 0x7f06480ae600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.771+0000 7f0647fff700 1 -- 172.21.15.32:0/1535344036 >> v1:172.21.15.61:6789/0 conn(0x7f064805f6f0 legacy=0x7f06480adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.771+0000 7f0647fff700 1 -- 172.21.15.32:0/1535344036 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064805fad0 con 0x7f0648061780 2020-12-05T17:33:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.771+0000 7f0647fff700 1 --2- 172.21.15.32:0/1535344036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0648061780 0x7f064814e240 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f0640002f30 tx=0x7f0640006e80).ready entity=mon.0 client_cookie=cc0879d150ccccd6 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.771+0000 7f06467fc700 1 -- 172.21.15.32:0/1535344036 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06400097f0 con 0x7f0648061780 2020-12-05T17:33:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.771+0000 7f06467fc700 1 -- 172.21.15.32:0/1535344036 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0640009950 con 0x7f0648061780 2020-12-05T17:33:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.771+0000 7f06467fc700 1 -- 172.21.15.32:0/1535344036 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0640009cb0 con 0x7f0648061780 2020-12-05T17:33:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.771+0000 7f064d9e6700 1 -- 172.21.15.32:0/1535344036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0648061780 msgr2=0x7f064814e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.771+0000 7f064d9e6700 1 --2- 172.21.15.32:0/1535344036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0648061780 0x7f064814e240 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f0640002f30 tx=0x7f0640006e80).stop 2020-12-05T17:33:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.775+0000 7f064d9e6700 1 -- 172.21.15.32:0/1535344036 shutdown_connections 2020-12-05T17:33:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.775+0000 7f064d9e6700 1 --2- 172.21.15.32:0/1535344036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0648061780 0x7f064814e240 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.775+0000 7f064d9e6700 1 --2- 172.21.15.32:0/1535344036 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f064805feb0 0x7f06480ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.775+0000 7f064d9e6700 1 -- 172.21.15.32:0/1535344036 >> 172.21.15.32:0/1535344036 conn(0x7f06480b4b10 msgr2=0x7f06480b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:47.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.775+0000 7f064d9e6700 1 -- 172.21.15.32:0/1535344036 shutdown_connections 2020-12-05T17:33:47.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.775+0000 7f064d9e6700 1 -- 172.21.15.32:0/1535344036 wait complete. 2020-12-05T17:33:47.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.775+0000 7f064d9e6700 1 Processor -- start 2020-12-05T17:33:47.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.775+0000 7f064d9e6700 1 -- start start 2020-12-05T17:33:47.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f064d9e6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f064805f6f0 0x7f064811e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:47.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f064d9e6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f064805feb0 0x7f0648072890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:47.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f064d9e6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0648061780 0x7f064806fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:47.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f064d9e6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06480731a0 con 0x7f064805f6f0 2020-12-05T17:33:47.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f064d9e6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064811d9f0 con 0x7f064805feb0 2020-12-05T17:33:47.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f064d9e6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064811dc90 con 0x7f0648061780 2020-12-05T17:33:47.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f06477fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f064805f6f0 0x7f064811e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:47.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f06477fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f064805f6f0 0x7f064811e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34614/0 (socket says 172.21.15.32:34614) 2020-12-05T17:33:47.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f06477fe700 1 -- 172.21.15.32:0/1005818553 learned_addr learned my addr 172.21.15.32:0/1005818553 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f0646ffd700 1 --2- 172.21.15.32:0/1005818553 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f064805feb0 0x7f0648072890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f0647fff700 1 --2- 172.21.15.32:0/1005818553 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0648061780 0x7f064806fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f06477fe700 1 -- 172.21.15.32:0/1005818553 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0648061780 msgr2=0x7f064806fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f06477fe700 1 --2- 172.21.15.32:0/1005818553 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0648061780 0x7f064806fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f06477fe700 1 -- 172.21.15.32:0/1005818553 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f064805feb0 msgr2=0x7f0648072890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:47.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f06477fe700 1 --2- 172.21.15.32:0/1005818553 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f064805feb0 0x7f0648072890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f06477fe700 1 -- 172.21.15.32:0/1005818553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0648066c60 con 0x7f064805f6f0 2020-12-05T17:33:47.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f0647fff700 1 --2- 172.21.15.32:0/1005818553 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0648061780 0x7f064806fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:33:47.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f0646ffd700 1 --2- 172.21.15.32:0/1005818553 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f064805feb0 0x7f0648072890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:33:47.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f06477fe700 1 --2- 172.21.15.32:0/1005818553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f064805f6f0 0x7f064811e130 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f0648061550 tx=0x7f0638007f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:47.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f063ffff700 1 -- 172.21.15.32:0/1005818553 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0638006a70 con 0x7f064805f6f0 2020-12-05T17:33:47.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f063ffff700 1 -- 172.21.15.32:0/1005818553 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0638006bd0 con 0x7f064805f6f0 2020-12-05T17:33:47.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f064d9e6700 1 -- 172.21.15.32:0/1005818553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f064811e620 con 0x7f064805f6f0 2020-12-05T17:33:47.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f063ffff700 1 -- 172.21.15.32:0/1005818553 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0638007030 con 0x7f064805f6f0 2020-12-05T17:33:47.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f064d9e6700 1 -- 172.21.15.32:0/1005818553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0648072f20 con 0x7f064805f6f0 2020-12-05T17:33:47.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f064d9e6700 1 -- 172.21.15.32:0/1005818553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0648057050 con 0x7f064805f6f0 2020-12-05T17:33:47.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.779+0000 7f063ffff700 1 -- 172.21.15.32:0/1005818553 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0638010030 con 0x7f064805f6f0 2020-12-05T17:33:47.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.783+0000 7f063ffff700 1 --2- 172.21.15.32:0/1005818553 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f062c0554d0 0x7f062c057990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:47.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.783+0000 7f063ffff700 1 -- 172.21.15.32:0/1005818553 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f063806e1d0 con 0x7f064805f6f0 2020-12-05T17:33:47.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.783+0000 7f0646ffd700 1 --2- 172.21.15.32:0/1005818553 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f062c0554d0 0x7f062c057990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:47.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.783+0000 7f0646ffd700 1 --2- 172.21.15.32:0/1005818553 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f062c0554d0 0x7f062c057990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f063000b000 tx=0x7f0630008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:47.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.787+0000 7f063ffff700 1 -- 172.21.15.32:0/1005818553 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0638044370 con 0x7f064805f6f0 2020-12-05T17:33:47.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.979+0000 7f064d9e6700 1 -- 172.21.15.32:0/1005818553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f064811de00 con 0x7f064805f6f0 2020-12-05T17:33:47.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.987+0000 7f063ffff700 1 -- 172.21.15.32:0/1005818553 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v14) v1 ==== 184+0+0 (secure 0 0 0) 0x7f0638047990 con 0x7f064805f6f0 2020-12-05T17:33:47.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.991+0000 7f064d9e6700 1 -- 172.21.15.32:0/1005818553 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f062c0554d0 msgr2=0x7f062c057990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:47.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.991+0000 7f064d9e6700 1 --2- 172.21.15.32:0/1005818553 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f062c0554d0 0x7f062c057990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f063000b000 tx=0x7f0630008040).stop 2020-12-05T17:33:47.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.991+0000 7f064d9e6700 1 -- 172.21.15.32:0/1005818553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f064805f6f0 msgr2=0x7f064811e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:47.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.991+0000 7f064d9e6700 1 --2- 172.21.15.32:0/1005818553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f064805f6f0 0x7f064811e130 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f0648061550 tx=0x7f0638007f60).stop 2020-12-05T17:33:47.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.991+0000 7f064d9e6700 1 -- 172.21.15.32:0/1005818553 shutdown_connections 2020-12-05T17:33:47.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.991+0000 7f064d9e6700 1 --2- 172.21.15.32:0/1005818553 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f062c0554d0 0x7f062c057990 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.991+0000 7f064d9e6700 1 --2- 172.21.15.32:0/1005818553 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0648061780 0x7f064806fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.991+0000 7f064d9e6700 1 --2- 172.21.15.32:0/1005818553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f064805f6f0 0x7f064811e130 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.991+0000 7f064d9e6700 1 --2- 172.21.15.32:0/1005818553 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f064805feb0 0x7f0648072890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:47.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.991+0000 7f064d9e6700 1 -- 172.21.15.32:0/1005818553 >> 172.21.15.32:0/1005818553 conn(0x7f06480b4b10 msgr2=0x7f0648130d40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:47.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.991+0000 7f064d9e6700 1 -- 172.21.15.32:0/1005818553 shutdown_connections 2020-12-05T17:33:47.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:47.991+0000 7f064d9e6700 1 -- 172.21.15.32:0/1005818553 wait complete. 2020-12-05T17:33:47.995 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2020-12-05T17:33:48.008 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:33:48.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.119+0000 7ff75184f700 1 Processor -- start 2020-12-05T17:33:48.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.119+0000 7ff75184f700 1 -- start start 2020-12-05T17:33:48.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.119+0000 7ff75184f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff74c05feb0 0x7ff74c0ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:48.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.119+0000 7ff75184f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff74c061780 0x7ff74c14e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:48.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.119+0000 7ff75184f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff74c0b0380 con 0x7ff74c05f6f0 2020-12-05T17:33:48.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.119+0000 7ff75184f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff74c0aeaf0 con 0x7ff74c05feb0 2020-12-05T17:33:48.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.119+0000 7ff75184f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff74c0aef70 con 0x7ff74c061780 2020-12-05T17:33:48.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.119+0000 7ff74affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff74c05feb0 0x7ff74c0ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:48.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.119+0000 7ff74b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff74c05f6f0 0x7ff74c0adef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37270/0 (socket says 172.21.15.32:37270) 2020-12-05T17:33:48.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.119+0000 7ff74b7fe700 1 -- 172.21.15.32:0/323071004 learned_addr learned my addr 172.21.15.32:0/323071004 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:33:48.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.119+0000 7ff74bfff700 1 --2- 172.21.15.32:0/323071004 >> v2:172.21.15.61:3300/0 conn(0x7ff74c061780 0x7ff74c14e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.119+0000 7ff74affd700 1 -- 172.21.15.32:0/323071004 >> v2:172.21.15.61:3300/0 conn(0x7ff74c061780 msgr2=0x7ff74c14e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.119+0000 7ff74affd700 1 --2- 172.21.15.32:0/323071004 >> v2:172.21.15.61:3300/0 conn(0x7ff74c061780 0x7ff74c14e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.119+0000 7ff74affd700 1 -- 172.21.15.32:0/323071004 >> v1:172.21.15.32:6789/0 conn(0x7ff74c05f6f0 legacy=0x7ff74c0adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.119+0000 7ff74affd700 1 -- 172.21.15.32:0/323071004 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74c05fad0 con 0x7ff74c05feb0 2020-12-05T17:33:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.119+0000 7ff74affd700 1 --2- 172.21.15.32:0/323071004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff74c05feb0 0x7ff74c0ae600 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7ff740002f30 tx=0x7ff740006dc0).ready entity=mon.0 client_cookie=c8facb136e6d0cc8 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:48.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.119+0000 7ff74a7fc700 1 -- 172.21.15.32:0/323071004 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7400097b0 con 0x7ff74c05feb0 2020-12-05T17:33:48.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.119+0000 7ff74a7fc700 1 -- 172.21.15.32:0/323071004 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff740009910 con 0x7ff74c05feb0 2020-12-05T17:33:48.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.119+0000 7ff74a7fc700 1 -- 172.21.15.32:0/323071004 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff740009c70 con 0x7ff74c05feb0 2020-12-05T17:33:48.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.123+0000 7ff75184f700 1 -- 172.21.15.32:0/323071004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff74c05feb0 msgr2=0x7ff74c0ae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:48.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.123+0000 7ff75184f700 1 --2- 172.21.15.32:0/323071004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff74c05feb0 0x7ff74c0ae600 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7ff740002f30 tx=0x7ff740006dc0).stop 2020-12-05T17:33:48.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.123+0000 7ff75184f700 1 -- 172.21.15.32:0/323071004 shutdown_connections 2020-12-05T17:33:48.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.123+0000 7ff75184f700 1 --2- 172.21.15.32:0/323071004 >> v2:172.21.15.61:3300/0 conn(0x7ff74c061780 0x7ff74c14e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.123+0000 7ff75184f700 1 --2- 172.21.15.32:0/323071004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff74c05feb0 0x7ff74c0ae600 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.123+0000 7ff75184f700 1 -- 172.21.15.32:0/323071004 >> 172.21.15.32:0/323071004 conn(0x7ff74c0b4b10 msgr2=0x7ff74c0b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:48.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.123+0000 7ff75184f700 1 -- 172.21.15.32:0/323071004 shutdown_connections 2020-12-05T17:33:48.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.123+0000 7ff75184f700 1 -- 172.21.15.32:0/323071004 wait complete. 2020-12-05T17:33:48.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.127+0000 7ff75184f700 1 Processor -- start 2020-12-05T17:33:48.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.127+0000 7ff75184f700 1 -- start start 2020-12-05T17:33:48.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.127+0000 7ff75184f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff74c05f6f0 0x7ff74c14e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:48.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.127+0000 7ff75184f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff74c05feb0 0x7ff74c121430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:48.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.127+0000 7ff75184f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff74c061780 0x7ff74c121920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:48.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.127+0000 7ff75184f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c065250 con 0x7ff74c061780 2020-12-05T17:33:48.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.127+0000 7ff75184f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c1239b0 con 0x7ff74c05feb0 2020-12-05T17:33:48.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.127+0000 7ff75184f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c0667f0 con 0x7ff74c05f6f0 2020-12-05T17:33:48.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.127+0000 7ff74bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff74c061780 0x7ff74c121920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:48.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.127+0000 7ff74bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff74c061780 0x7ff74c121920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34632/0 (socket says 172.21.15.32:34632) 2020-12-05T17:33:48.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.127+0000 7ff74bfff700 1 -- 172.21.15.32:0/779712212 learned_addr learned my addr 172.21.15.32:0/779712212 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:48.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.127+0000 7ff74affd700 1 --2- 172.21.15.32:0/779712212 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff74c05feb0 0x7ff74c121430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:48.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.127+0000 7ff74b7fe700 1 --2- 172.21.15.32:0/779712212 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff74c05f6f0 0x7ff74c14e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:48.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.127+0000 7ff74bfff700 1 -- 172.21.15.32:0/779712212 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff74c05f6f0 msgr2=0x7ff74c14e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.127+0000 7ff74bfff700 1 --2- 172.21.15.32:0/779712212 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff74c05f6f0 0x7ff74c14e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.127+0000 7ff74bfff700 1 -- 172.21.15.32:0/779712212 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff74c05feb0 msgr2=0x7ff74c121430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.127+0000 7ff74bfff700 1 --2- 172.21.15.32:0/779712212 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff74c05feb0 0x7ff74c121430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.127+0000 7ff74bfff700 1 -- 172.21.15.32:0/779712212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74c121e10 con 0x7ff74c061780 2020-12-05T17:33:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.127+0000 7ff74bfff700 1 --2- 172.21.15.32:0/779712212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff74c061780 0x7ff74c121920 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7ff744006dd0 tx=0x7ff7440091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.127+0000 7ff733fff700 1 -- 172.21.15.32:0/779712212 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff74400c8f0 con 0x7ff74c061780 2020-12-05T17:33:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.127+0000 7ff733fff700 1 -- 172.21.15.32:0/779712212 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74400ca50 con 0x7ff74c061780 2020-12-05T17:33:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.131+0000 7ff75184f700 1 -- 172.21.15.32:0/779712212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff74c1599b0 con 0x7ff74c061780 2020-12-05T17:33:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.131+0000 7ff733fff700 1 -- 172.21.15.32:0/779712212 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff74400cdb0 con 0x7ff74c061780 2020-12-05T17:33:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.131+0000 7ff75184f700 1 -- 172.21.15.32:0/779712212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74c159c90 con 0x7ff74c061780 2020-12-05T17:33:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.131+0000 7ff75184f700 1 -- 172.21.15.32:0/779712212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff74c123810 con 0x7ff74c061780 2020-12-05T17:33:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.131+0000 7ff733fff700 1 -- 172.21.15.32:0/779712212 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff744022070 con 0x7ff74c061780 2020-12-05T17:33:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.131+0000 7ff733fff700 1 --2- 172.21.15.32:0/779712212 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff72c0553b0 0x7ff72c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.131+0000 7ff733fff700 1 -- 172.21.15.32:0/779712212 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff74406e680 con 0x7ff74c061780 2020-12-05T17:33:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.131+0000 7ff74b7fe700 1 --2- 172.21.15.32:0/779712212 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff72c0553b0 0x7ff72c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.135+0000 7ff74b7fe700 1 --2- 172.21.15.32:0/779712212 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff72c0553b0 0x7ff72c057870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7ff73c00d000 tx=0x7ff73c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.139+0000 7ff733fff700 1 -- 172.21.15.32:0/779712212 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff74c123810 con 0x7ff74c061780 2020-12-05T17:33:48.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.335+0000 7ff75184f700 1 -- 172.21.15.32:0/779712212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7ff74c123810 con 0x7ff74c061780 2020-12-05T17:33:48.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.339+0000 7ff733fff700 1 -- 172.21.15.32:0/779712212 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v15) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff74c123810 con 0x7ff74c061780 2020-12-05T17:33:48.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.339+0000 7ff75184f700 1 -- 172.21.15.32:0/779712212 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff72c0553b0 msgr2=0x7ff72c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:48.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.339+0000 7ff75184f700 1 --2- 172.21.15.32:0/779712212 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff72c0553b0 0x7ff72c057870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7ff73c00d000 tx=0x7ff73c009040).stop 2020-12-05T17:33:48.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.339+0000 7ff75184f700 1 -- 172.21.15.32:0/779712212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff74c061780 msgr2=0x7ff74c121920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:48.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.339+0000 7ff75184f700 1 --2- 172.21.15.32:0/779712212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff74c061780 0x7ff74c121920 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7ff744006dd0 tx=0x7ff7440091a0).stop 2020-12-05T17:33:48.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.343+0000 7ff75184f700 1 -- 172.21.15.32:0/779712212 shutdown_connections 2020-12-05T17:33:48.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.343+0000 7ff75184f700 1 --2- 172.21.15.32:0/779712212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff74c061780 0x7ff74c121920 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.343+0000 7ff75184f700 1 --2- 172.21.15.32:0/779712212 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff72c0553b0 0x7ff72c057870 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.343+0000 7ff75184f700 1 --2- 172.21.15.32:0/779712212 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff74c05f6f0 0x7ff74c14e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.343+0000 7ff75184f700 1 --2- 172.21.15.32:0/779712212 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff74c05feb0 0x7ff74c121430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.343+0000 7ff75184f700 1 -- 172.21.15.32:0/779712212 >> 172.21.15.32:0/779712212 conn(0x7ff74c0b4b10 msgr2=0x7ff74c0b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.343+0000 7ff75184f700 1 -- 172.21.15.32:0/779712212 shutdown_connections 2020-12-05T17:33:48.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.343+0000 7ff75184f700 1 -- 172.21.15.32:0/779712212 wait complete. 2020-12-05T17:33:48.345 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2020-12-05T17:33:48.354 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:33:48.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.479+0000 7f9b4134a700 1 Processor -- start 2020-12-05T17:33:48.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.479+0000 7f9b4134a700 1 -- start start 2020-12-05T17:33:48.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.479+0000 7f9b4134a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9b3c153240 0x7f9b3c0ae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:48.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.479+0000 7f9b4134a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b3c080a90 con 0x7f9b3c0611b0 2020-12-05T17:33:48.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.479+0000 7f9b4134a700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b3c152860 con 0x7f9b3c0b0240 2020-12-05T17:33:48.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.479+0000 7f9b4134a700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b3c152b00 con 0x7f9b3c153240 2020-12-05T17:33:48.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.479+0000 7f9b40b49700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9b3c0611b0 0x7f9b3c14c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37284/0 (socket says 172.21.15.32:37284) 2020-12-05T17:33:48.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.479+0000 7f9b40b49700 1 -- 172.21.15.32:0/2791115182 learned_addr learned my addr 172.21.15.32:0/2791115182 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:33:48.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.479+0000 7f9b425ac700 1 --2- 172.21.15.32:0/2791115182 >> v2:172.21.15.61:3300/0 conn(0x7f9b3c153240 0x7f9b3c0ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:48.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b3b7fe700 1 -- 172.21.15.32:0/2791115182 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296326083 0 0) 0x7f9b3c080a90 con 0x7f9b3c0611b0 2020-12-05T17:33:48.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b3b7fe700 1 -- 172.21.15.32:0/2791115182 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b24003230 con 0x7f9b3c0611b0 2020-12-05T17:33:48.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b3b7fe700 1 -- 172.21.15.32:0/2791115182 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 358083275 0 0) 0x7f9b24003230 con 0x7f9b3c0611b0 2020-12-05T17:33:48.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b3b7fe700 1 -- 172.21.15.32:0/2791115182 >> v2:172.21.15.61:3300/0 conn(0x7f9b3c153240 msgr2=0x7f9b3c0ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:48.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b3b7fe700 1 --2- 172.21.15.32:0/2791115182 >> v2:172.21.15.61:3300/0 conn(0x7f9b3c153240 0x7f9b3c0ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b3b7fe700 1 -- 172.21.15.32:0/2791115182 >> v1:172.21.15.61:6789/0 conn(0x7f9b3c0b0240 legacy=0x7f9b3c14f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:48.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b3b7fe700 1 -- 172.21.15.32:0/2791115182 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c152da0 con 0x7f9b3c0611b0 2020-12-05T17:33:48.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b3b7fe700 1 -- 172.21.15.32:0/2791115182 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f9b2c001a50 con 0x7f9b3c0611b0 2020-12-05T17:33:48.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b3b7fe700 1 -- 172.21.15.32:0/2791115182 >> v1:172.21.15.32:6789/0 conn(0x7f9b3c0611b0 legacy=0x7f9b3c14c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:48.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b3b7fe700 1 --2- 172.21.15.32:0/2791115182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b24003720 0x7f9b24003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:48.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b3b7fe700 1 --2- 172.21.15.32:0/2791115182 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b240044d0 0x7f9b24004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:48.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b3b7fe700 1 --2- 172.21.15.32:0/2791115182 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b24004e40 0x7f9b2400b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:48.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b3b7fe700 1 -- 172.21.15.32:0/2791115182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c152b00 con 0x7f9b24003720 2020-12-05T17:33:48.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b3b7fe700 1 -- 172.21.15.32:0/2791115182 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b24004250 con 0x7f9b240044d0 2020-12-05T17:33:48.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b3b7fe700 1 -- 172.21.15.32:0/2791115182 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2404fc20 con 0x7f9b24004e40 2020-12-05T17:33:48.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b40b49700 1 --2- 172.21.15.32:0/2791115182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b24003720 0x7f9b24003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:48.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b425ac700 1 --2- 172.21.15.32:0/2791115182 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b24004e40 0x7f9b2400b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:48.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b3bfff700 1 --2- 172.21.15.32:0/2791115182 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b240044d0 0x7f9b24004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:48.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b40b49700 1 -- 172.21.15.32:0/2791115182 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b24004e40 msgr2=0x7f9b2400b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:48.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b40b49700 1 --2- 172.21.15.32:0/2791115182 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b24004e40 0x7f9b2400b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b40b49700 1 -- 172.21.15.32:0/2791115182 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b240044d0 msgr2=0x7f9b24004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:48.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b40b49700 1 --2- 172.21.15.32:0/2791115182 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b240044d0 0x7f9b24004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b425ac700 1 -- 172.21.15.32:0/2791115182 reap_dead start 2020-12-05T17:33:48.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b40b49700 1 -- 172.21.15.32:0/2791115182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b24003230 con 0x7f9b24003720 2020-12-05T17:33:48.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b40b49700 1 --2- 172.21.15.32:0/2791115182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b24003720 0x7f9b24003b40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f9b2c001760 tx=0x7f9b3c1529f0).ready entity=mon.0 client_cookie=c6a5df204d24c57 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:48.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b3b7fe700 1 -- 172.21.15.32:0/2791115182 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b2c005b70 con 0x7f9b24003720 2020-12-05T17:33:48.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b3b7fe700 1 -- 172.21.15.32:0/2791115182 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2c005cd0 con 0x7f9b24003720 2020-12-05T17:33:48.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b3b7fe700 1 -- 172.21.15.32:0/2791115182 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b2c0072c0 con 0x7f9b24003720 2020-12-05T17:33:48.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b4134a700 1 -- 172.21.15.32:0/2791115182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b24003720 msgr2=0x7f9b24003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:48.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b4134a700 1 --2- 172.21.15.32:0/2791115182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b24003720 0x7f9b24003b40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f9b2c001760 tx=0x7f9b3c1529f0).stop 2020-12-05T17:33:48.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b4134a700 1 -- 172.21.15.32:0/2791115182 shutdown_connections 2020-12-05T17:33:48.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b4134a700 1 --2- 172.21.15.32:0/2791115182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b24003720 0x7f9b24003b40 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b4134a700 1 -- 172.21.15.32:0/2791115182 >> 172.21.15.32:0/2791115182 conn(0x7f9b3c0b5f90 msgr2=0x7f9b3c0b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:48.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b4134a700 1 -- 172.21.15.32:0/2791115182 shutdown_connections 2020-12-05T17:33:48.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.483+0000 7f9b4134a700 1 -- 172.21.15.32:0/2791115182 wait complete. 2020-12-05T17:33:48.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.487+0000 7f9b4134a700 1 Processor -- start 2020-12-05T17:33:48.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.487+0000 7f9b4134a700 1 -- start start 2020-12-05T17:33:48.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.487+0000 7f9b4134a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b24003720 0x7f9b3c064c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:48.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.487+0000 7f9b4134a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b3c139f40 0x7f9b3c13df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.487+0000 7f9b4134a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b3c13a7f0 0x7f9b3c13ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.487+0000 7f9b4134a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c13a3c0 con 0x7f9b3c139f40 2020-12-05T17:33:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.487+0000 7f9b4134a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c0646d0 con 0x7f9b3c13a7f0 2020-12-05T17:33:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.487+0000 7f9b4134a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c13e430 con 0x7f9b24003720 2020-12-05T17:33:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.487+0000 7f9b3bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b3c139f40 0x7f9b3c13df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:48.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.487+0000 7f9b3bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b3c139f40 0x7f9b3c13df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34648/0 (socket says 172.21.15.32:34648) 2020-12-05T17:33:48.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.487+0000 7f9b3bfff700 1 -- 172.21.15.32:0/1324354196 learned_addr learned my addr 172.21.15.32:0/1324354196 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:48.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.487+0000 7f9b40b49700 1 --2- 172.21.15.32:0/1324354196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b24003720 0x7f9b3c064c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:48.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.487+0000 7f9b425ac700 1 --2- 172.21.15.32:0/1324354196 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b3c13a7f0 0x7f9b3c13ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:48.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.487+0000 7f9b3bfff700 1 -- 172.21.15.32:0/1324354196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b24003720 msgr2=0x7f9b3c064c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:48.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.491+0000 7f9b3bfff700 1 --2- 172.21.15.32:0/1324354196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b24003720 0x7f9b3c064c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.491+0000 7f9b3bfff700 1 -- 172.21.15.32:0/1324354196 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b3c13a7f0 msgr2=0x7f9b3c13ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:48.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.491+0000 7f9b3bfff700 1 --2- 172.21.15.32:0/1324354196 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b3c13a7f0 0x7f9b3c13ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.491+0000 7f9b3bfff700 1 -- 172.21.15.32:0/1324354196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c1114a0 con 0x7f9b3c139f40 2020-12-05T17:33:48.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.491+0000 7f9b40b49700 1 --2- 172.21.15.32:0/1324354196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b24003720 0x7f9b3c064c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:33:48.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.491+0000 7f9b3bfff700 1 --2- 172.21.15.32:0/1324354196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b3c139f40 0x7f9b3c13df40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f9b3c064810 tx=0x7f9b30007b60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:48.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.491+0000 7f9b397fa700 1 -- 172.21.15.32:0/1324354196 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b300118c0 con 0x7f9b3c139f40 2020-12-05T17:33:48.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.491+0000 7f9b397fa700 1 -- 172.21.15.32:0/1324354196 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b30011a20 con 0x7f9b3c139f40 2020-12-05T17:33:48.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.491+0000 7f9b397fa700 1 -- 172.21.15.32:0/1324354196 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b30011d80 con 0x7f9b3c139f40 2020-12-05T17:33:48.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.491+0000 7f9b4134a700 1 -- 172.21.15.32:0/1324354196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b3c111740 con 0x7f9b3c139f40 2020-12-05T17:33:48.503 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.491+0000 7f9b4134a700 1 -- 172.21.15.32:0/1324354196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b3c13b160 con 0x7f9b3c139f40 2020-12-05T17:33:48.503 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.491+0000 7f9b4134a700 1 -- 172.21.15.32:0/1324354196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b3c057050 con 0x7f9b3c139f40 2020-12-05T17:33:48.503 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.491+0000 7f9b397fa700 1 -- 172.21.15.32:0/1324354196 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9b3001c030 con 0x7f9b3c139f40 2020-12-05T17:33:48.503 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.491+0000 7f9b397fa700 1 --2- 172.21.15.32:0/1324354196 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9b1c055320 0x7f9b1c0577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:48.503 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.491+0000 7f9b397fa700 1 -- 172.21.15.32:0/1324354196 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9b3006fee0 con 0x7f9b3c139f40 2020-12-05T17:33:48.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.491+0000 7f9b40b49700 1 --2- 172.21.15.32:0/1324354196 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9b1c055320 0x7f9b1c0577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:48.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.495+0000 7f9b40b49700 1 --2- 172.21.15.32:0/1324354196 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9b1c055320 0x7f9b1c0577e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f9b2c017000 tx=0x7f9b2c013000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:48.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.499+0000 7f9b397fa700 1 -- 172.21.15.32:0/1324354196 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9b30046030 con 0x7f9b3c139f40 2020-12-05T17:33:48.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.691+0000 7f9b4134a700 1 -- 172.21.15.32:0/1324354196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f9b3c07fd30 con 0x7f9b3c139f40 2020-12-05T17:33:48.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.699+0000 7f9b397fa700 1 -- 172.21.15.32:0/1324354196 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v16) v1 ==== 184+0+0 (secure 0 0 0) 0x7f9b30049650 con 0x7f9b3c139f40 2020-12-05T17:33:48.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.699+0000 7f9b4134a700 1 -- 172.21.15.32:0/1324354196 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9b1c055320 msgr2=0x7f9b1c0577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:48.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.699+0000 7f9b4134a700 1 --2- 172.21.15.32:0/1324354196 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9b1c055320 0x7f9b1c0577e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f9b2c017000 tx=0x7f9b2c013000).stop 2020-12-05T17:33:48.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.699+0000 7f9b4134a700 1 -- 172.21.15.32:0/1324354196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b3c139f40 msgr2=0x7f9b3c13df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:48.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.699+0000 7f9b4134a700 1 --2- 172.21.15.32:0/1324354196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b3c139f40 0x7f9b3c13df40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f9b3c064810 tx=0x7f9b30007b60).stop 2020-12-05T17:33:48.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.699+0000 7f9b4134a700 1 -- 172.21.15.32:0/1324354196 shutdown_connections 2020-12-05T17:33:48.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.699+0000 7f9b4134a700 1 --2- 172.21.15.32:0/1324354196 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b3c13a7f0 0x7f9b3c13ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.699+0000 7f9b4134a700 1 --2- 172.21.15.32:0/1324354196 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9b1c055320 0x7f9b1c0577e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.699+0000 7f9b4134a700 1 --2- 172.21.15.32:0/1324354196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b24003720 0x7f9b3c064c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.699+0000 7f9b4134a700 1 --2- 172.21.15.32:0/1324354196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b3c139f40 0x7f9b3c13df40 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.699+0000 7f9b4134a700 1 -- 172.21.15.32:0/1324354196 >> 172.21.15.32:0/1324354196 conn(0x7f9b3c0b5f90 msgr2=0x7f9b3c0ae480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:48.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.699+0000 7f9b4134a700 1 -- 172.21.15.32:0/1324354196 shutdown_connections 2020-12-05T17:33:48.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.699+0000 7f9b4134a700 1 -- 172.21.15.32:0/1324354196 wait complete. 2020-12-05T17:33:48.704 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.2 2020-12-05T17:33:48.715 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:33:48.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.827+0000 7f2d2d7a6700 1 Processor -- start 2020-12-05T17:33:48.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.827+0000 7f2d2d7a6700 1 -- start start 2020-12-05T17:33:48.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.827+0000 7f2d2d7a6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d280b5f00 0x7f2d280b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:48.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.827+0000 7f2d2d7a6700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f2d280ad510 0x7f2d280af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:48.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.827+0000 7f2d2d7a6700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d28142ab0 con 0x7f2d28066370 2020-12-05T17:33:48.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.827+0000 7f2d2d7a6700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d280b62e0 con 0x7f2d280ad510 2020-12-05T17:33:48.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.827+0000 7f2d2d7a6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d280afbf0 con 0x7f2d280b5f00 2020-12-05T17:33:48.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.827+0000 7f2d277fe700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f2d280ad510 0x7f2d280af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:48.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.827+0000 7f2d27fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d280b5f00 0x7f2d280b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:48.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.827+0000 7f2d27fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d280b5f00 0x7f2d280b14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:59678/0 (socket says 172.21.15.32:59678) 2020-12-05T17:33:48.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.827+0000 7f2d2cfa5700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f2d28066370 0x7f2d28142ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:49946/0 (socket says 172.21.15.32:49946) 2020-12-05T17:33:48.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.827+0000 7f2d277fe700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f2d280ad510 0x7f2d280af480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:54550/0 (socket says 172.21.15.32:54550) 2020-12-05T17:33:48.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.827+0000 7f2d27fff700 1 -- 172.21.15.32:0/3520688806 learned_addr learned my addr 172.21.15.32:0/3520688806 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:48.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.827+0000 7f2d26ffd700 1 -- 172.21.15.32:0/3520688806 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 923211246 0 0) 0x7f2d28142ab0 con 0x7f2d28066370 2020-12-05T17:33:48.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.827+0000 7f2d26ffd700 1 -- 172.21.15.32:0/3520688806 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d14003230 con 0x7f2d28066370 2020-12-05T17:33:48.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.827+0000 7f2d27fff700 1 -- 172.21.15.32:0/3520688806 >> v2:172.21.15.61:3300/0 conn(0x7f2d280ad510 msgr2=0x7f2d280af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:48.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.827+0000 7f2d27fff700 1 --2- 172.21.15.32:0/3520688806 >> v2:172.21.15.61:3300/0 conn(0x7f2d280ad510 0x7f2d280af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.827+0000 7f2d27fff700 1 -- 172.21.15.32:0/3520688806 >> v1:172.21.15.61:6789/0 conn(0x7f2d28066370 legacy=0x7f2d28142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:48.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.827+0000 7f2d27fff700 1 -- 172.21.15.32:0/3520688806 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d280afe90 con 0x7f2d280b5f00 2020-12-05T17:33:48.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.827+0000 7f2d277fe700 1 --2- 172.21.15.32:0/3520688806 >> v2:172.21.15.61:3300/0 conn(0x7f2d280ad510 0x7f2d280af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:33:48.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.831+0000 7f2d27fff700 1 --2- 172.21.15.32:0/3520688806 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d280b5f00 0x7f2d280b14e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f2d180032b0 tx=0x7f2d18009270).ready entity=mon.2 client_cookie=16c2ca9ec1fe47f6 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:48.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.831+0000 7f2d26ffd700 1 -- 172.21.15.32:0/3520688806 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d18009e60 con 0x7f2d280b5f00 2020-12-05T17:33:48.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.831+0000 7f2d26ffd700 1 -- 172.21.15.32:0/3520688806 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1800bb50 con 0x7f2d280b5f00 2020-12-05T17:33:48.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.831+0000 7f2d26ffd700 1 -- 172.21.15.32:0/3520688806 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d1800bcc0 con 0x7f2d280b5f00 2020-12-05T17:33:48.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.831+0000 7f2d2d7a6700 1 -- 172.21.15.32:0/3520688806 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d280b5f00 msgr2=0x7f2d280b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:48.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.831+0000 7f2d2d7a6700 1 --2- 172.21.15.32:0/3520688806 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d280b5f00 0x7f2d280b14e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f2d180032b0 tx=0x7f2d18009270).stop 2020-12-05T17:33:48.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.831+0000 7f2d2d7a6700 1 -- 172.21.15.32:0/3520688806 shutdown_connections 2020-12-05T17:33:48.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.831+0000 7f2d2d7a6700 1 --2- 172.21.15.32:0/3520688806 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d280b5f00 0x7f2d280b14e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.831+0000 7f2d2d7a6700 1 --2- 172.21.15.32:0/3520688806 >> v2:172.21.15.61:3300/0 conn(0x7f2d280ad510 0x7f2d280af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.831+0000 7f2d2d7a6700 1 -- 172.21.15.32:0/3520688806 >> 172.21.15.32:0/3520688806 conn(0x7f2d280604c0 msgr2=0x7f2d2805c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:48.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.831+0000 7f2d2d7a6700 1 -- 172.21.15.32:0/3520688806 shutdown_connections 2020-12-05T17:33:48.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.831+0000 7f2d2d7a6700 1 -- 172.21.15.32:0/3520688806 wait complete. 2020-12-05T17:33:48.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.831+0000 7f2d2d7a6700 1 Processor -- start 2020-12-05T17:33:48.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.831+0000 7f2d2d7a6700 1 -- start start 2020-12-05T17:33:48.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d2d7a6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d28066370 0x7f2d2813b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:48.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d2d7a6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2d280ad510 0x7f2d28144a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:48.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d2d7a6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d280b5f00 0x7f2d2813deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:48.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d2d7a6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d28136600 con 0x7f2d28066370 2020-12-05T17:33:48.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d2d7a6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2813d7e0 con 0x7f2d280ad510 2020-12-05T17:33:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d2d7a6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2813c3f0 con 0x7f2d280b5f00 2020-12-05T17:33:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d27fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d28066370 0x7f2d2813b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d27fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d28066370 0x7f2d2813b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34662/0 (socket says 172.21.15.32:34662) 2020-12-05T17:33:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d27fff700 1 -- 172.21.15.32:0/2351207248 learned_addr learned my addr 172.21.15.32:0/2351207248 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d277fe700 1 --2- 172.21.15.32:0/2351207248 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2d280ad510 0x7f2d28144a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d2cfa5700 1 --2- 172.21.15.32:0/2351207248 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d280b5f00 0x7f2d2813deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d27fff700 1 -- 172.21.15.32:0/2351207248 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d280b5f00 msgr2=0x7f2d2813deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d27fff700 1 --2- 172.21.15.32:0/2351207248 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d280b5f00 0x7f2d2813deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d27fff700 1 -- 172.21.15.32:0/2351207248 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2d280ad510 msgr2=0x7f2d28144a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d27fff700 1 --2- 172.21.15.32:0/2351207248 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2d280ad510 0x7f2d28144a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:48.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d27fff700 1 -- 172.21.15.32:0/2351207248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d281367f0 con 0x7f2d28066370 2020-12-05T17:33:48.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d27fff700 1 --2- 172.21.15.32:0/2351207248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d28066370 0x7f2d2813b930 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f2d180032b0 tx=0x7f2d18016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:48.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d257fa700 1 -- 172.21.15.32:0/2351207248 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d1801f3f0 con 0x7f2d28066370 2020-12-05T17:33:48.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d257fa700 1 -- 172.21.15.32:0/2351207248 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1801f550 con 0x7f2d28066370 2020-12-05T17:33:48.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d2d7a6700 1 -- 172.21.15.32:0/2351207248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2813da10 con 0x7f2d28066370 2020-12-05T17:33:48.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d257fa700 1 -- 172.21.15.32:0/2351207248 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d1801f8b0 con 0x7f2d28066370 2020-12-05T17:33:48.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d2d7a6700 1 -- 172.21.15.32:0/2351207248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2813d600 con 0x7f2d28066370 2020-12-05T17:33:48.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d2d7a6700 1 -- 172.21.15.32:0/2351207248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d2813dba0 con 0x7f2d28066370 2020-12-05T17:33:48.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.835+0000 7f2d257fa700 1 -- 172.21.15.32:0/2351207248 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2d18026030 con 0x7f2d28066370 2020-12-05T17:33:48.845 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.839+0000 7f2d257fa700 1 --2- 172.21.15.32:0/2351207248 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2d080554e0 0x7f2d080579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:48.845 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.839+0000 7f2d277fe700 1 --2- 172.21.15.32:0/2351207248 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2d080554e0 0x7f2d080579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:48.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.839+0000 7f2d257fa700 1 -- 172.21.15.32:0/2351207248 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2d1806e960 con 0x7f2d28066370 2020-12-05T17:33:48.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.839+0000 7f2d277fe700 1 --2- 172.21.15.32:0/2351207248 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2d080554e0 0x7f2d080579a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f2d1c00d000 tx=0x7f2d1c006e50).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:48.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:48.843+0000 7f2d257fa700 1 -- 172.21.15.32:0/2351207248 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2d180449f0 con 0x7f2d28066370 2020-12-05T17:33:49.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.031+0000 7f2d2d7a6700 1 -- 172.21.15.32:0/2351207248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d28057050 con 0x7f2d28066370 2020-12-05T17:33:49.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.031+0000 7f2d257fa700 1 -- 172.21.15.32:0/2351207248 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2138 (secure 0 0 0) 0x7f2d18024090 con 0x7f2d28066370 2020-12-05T17:33:49.032 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:33:49.033 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4919,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/3405415630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3405415630},{"type":"v1","addr":"172.21.15.61:6833","nonce":3405415630}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:33:45.513696+0000","modified":"2020-12-05T17:33:46.989395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5301":{"gid":5301,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3450615455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3450615455},{"type":"v1","addr":"172.21.15.32:6835","nonce":3450615455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-12-05T17:33:49.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.035+0000 7f2d2d7a6700 1 -- 172.21.15.32:0/2351207248 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2d080554e0 msgr2=0x7f2d080579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:49.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.035+0000 7f2d2d7a6700 1 --2- 172.21.15.32:0/2351207248 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2d080554e0 0x7f2d080579a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f2d1c00d000 tx=0x7f2d1c006e50).stop 2020-12-05T17:33:49.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.035+0000 7f2d2d7a6700 1 -- 172.21.15.32:0/2351207248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d28066370 msgr2=0x7f2d2813b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:49.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.035+0000 7f2d2d7a6700 1 --2- 172.21.15.32:0/2351207248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d28066370 0x7f2d2813b930 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f2d180032b0 tx=0x7f2d18016000).stop 2020-12-05T17:33:49.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.035+0000 7f2d2d7a6700 1 -- 172.21.15.32:0/2351207248 shutdown_connections 2020-12-05T17:33:49.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.035+0000 7f2d2d7a6700 1 --2- 172.21.15.32:0/2351207248 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2d080554e0 0x7f2d080579a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:49.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.035+0000 7f2d2d7a6700 1 --2- 172.21.15.32:0/2351207248 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d280b5f00 0x7f2d2813deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:49.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.035+0000 7f2d2d7a6700 1 --2- 172.21.15.32:0/2351207248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d28066370 0x7f2d2813b930 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:49.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.035+0000 7f2d2d7a6700 1 --2- 172.21.15.32:0/2351207248 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2d280ad510 0x7f2d28144a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:49.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.035+0000 7f2d2d7a6700 1 -- 172.21.15.32:0/2351207248 >> 172.21.15.32:0/2351207248 conn(0x7f2d280604c0 msgr2=0x7f2d2805f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:49.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.035+0000 7f2d2d7a6700 1 -- 172.21.15.32:0/2351207248 shutdown_connections 2020-12-05T17:33:49.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.035+0000 7f2d2d7a6700 1 -- 172.21.15.32:0/2351207248 wait complete. 2020-12-05T17:33:49.038 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 17 2020-12-05T17:33:49.255 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:33:45.513696+0000', 'modified': '2020-12-05T17:33:46.989395+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5301}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5301': {'gid': 5301, 'name': 'a', 'rank': 0, 'incarnation': 15, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/3450615455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 3450615455}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 3450615455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:33:49.256 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:33:49.257 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:33:49.387 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":5301,"want_state":"up:active","state":"up:active","rank_uptime":3.8640399109999999,"mdsmap_epoch":17,"osdmap_epoch":44,"osdmap_epoch_barrier":44,"uptime":9.2881413360000007} 2020-12-05T17:33:49.388 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 5301, "mdsmap_epoch": 17, "osdmap_epoch": 44, "osdmap_epoch_barrier": 44, "rank_uptime": 3.864039911, "state": "up:active", "uptime": 9.288141336, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:33:49.389 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:33:49.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5c640e700 1 Processor -- start 2020-12-05T17:33:49.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5c640e700 1 -- start start 2020-12-05T17:33:49.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5c640e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5c005feb0 0x7fe5c005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:49.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5c640e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c0142450 con 0x7fe5c0062400 2020-12-05T17:33:49.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5c640e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c0061780 con 0x7fe5c005f3b0 2020-12-05T17:33:49.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5c640e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c0061a20 con 0x7fe5c005feb0 2020-12-05T17:33:49.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5c4c0b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe5c0062400 0x7fe5c00b0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37320/0 (socket says 172.21.15.32:37320) 2020-12-05T17:33:49.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5c4c0b700 1 -- 172.21.15.32:0/3113955440 learned_addr learned my addr 172.21.15.32:0/3113955440 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:33:49.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5c5c0d700 1 --2- 172.21.15.32:0/3113955440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5c005feb0 0x7fe5c005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:49.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5bffff700 1 -- 172.21.15.32:0/3113955440 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2321871986 0 0) 0x7fe5c0142450 con 0x7fe5c0062400 2020-12-05T17:33:49.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5bffff700 1 -- 172.21.15.32:0/3113955440 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5ac003230 con 0x7fe5c0062400 2020-12-05T17:33:49.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5bffff700 1 -- 172.21.15.32:0/3113955440 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274400744 0 0) 0x7fe5c0061780 con 0x7fe5c005f3b0 2020-12-05T17:33:49.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5bffff700 1 -- 172.21.15.32:0/3113955440 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5c0142450 con 0x7fe5c005f3b0 2020-12-05T17:33:49.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5bffff700 1 -- 172.21.15.32:0/3113955440 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3117473425 0 0) 0x7fe5ac003230 con 0x7fe5c0062400 2020-12-05T17:33:49.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5bffff700 1 -- 172.21.15.32:0/3113955440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5c005feb0 msgr2=0x7fe5c005f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:49.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5bffff700 1 --2- 172.21.15.32:0/3113955440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5c005feb0 0x7fe5c005f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:49.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5bffff700 1 -- 172.21.15.32:0/3113955440 >> v1:172.21.15.61:6789/0 conn(0x7fe5c005f3b0 legacy=0x7fe5c00b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:49.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5bffff700 1 -- 172.21.15.32:0/3113955440 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c0061cc0 con 0x7fe5c0062400 2020-12-05T17:33:49.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5bffff700 1 -- 172.21.15.32:0/3113955440 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fe5b40016d0 con 0x7fe5c0062400 2020-12-05T17:33:49.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5bffff700 1 -- 172.21.15.32:0/3113955440 >> v1:172.21.15.32:6789/0 conn(0x7fe5c0062400 legacy=0x7fe5c00b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:49.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5bffff700 1 --2- 172.21.15.32:0/3113955440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ac0038e0 0x7fe5ac003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:49.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5bffff700 1 --2- 172.21.15.32:0/3113955440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5c005feb0 0x7fe5ac0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:49.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5bffff700 1 --2- 172.21.15.32:0/3113955440 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe5ac0049c0 0x7fe5ac004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:49.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5bffff700 1 -- 172.21.15.32:0/3113955440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c0061a20 con 0x7fe5ac0038e0 2020-12-05T17:33:49.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5bffff700 1 -- 172.21.15.32:0/3113955440 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac005330 con 0x7fe5ac0049c0 2020-12-05T17:33:49.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5bffff700 1 -- 172.21.15.32:0/3113955440 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac00b720 con 0x7fe5c005feb0 2020-12-05T17:33:49.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.499+0000 7fe5c540c700 1 --2- 172.21.15.32:0/3113955440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ac0038e0 0x7fe5ac003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:49.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.503+0000 7fe5c4c0b700 1 --2- 172.21.15.32:0/3113955440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5c005feb0 0x7fe5ac0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:49.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.503+0000 7fe5c540c700 1 -- 172.21.15.32:0/3113955440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5c005feb0 msgr2=0x7fe5ac0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:49.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.503+0000 7fe5c540c700 1 --2- 172.21.15.32:0/3113955440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5c005feb0 0x7fe5ac0044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:49.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.503+0000 7fe5c540c700 1 -- 172.21.15.32:0/3113955440 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe5ac0049c0 msgr2=0x7fe5ac004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:49.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.503+0000 7fe5c540c700 1 --2- 172.21.15.32:0/3113955440 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe5ac0049c0 0x7fe5ac004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-12-05T17:33:49.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.503+0000 7fe5c540c700 1 -- 172.21.15.32:0/3113955440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ac003230 con 0x7fe5ac0038e0 2020-12-05T17:33:49.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.503+0000 7fe5c4c0b700 1 --2- 172.21.15.32:0/3113955440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5c005feb0 0x7fe5ac0044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:33:49.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.503+0000 7fe5c540c700 1 --2- 172.21.15.32:0/3113955440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ac0038e0 0x7fe5ac003ce0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7fe5b0001450 tx=0x7fe5b0007ee0).ready entity=mon.0 client_cookie=c8ad988dd741dfbc server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:49.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.503+0000 7fe5bffff700 1 -- 172.21.15.32:0/3113955440 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5b000cc00 con 0x7fe5ac0038e0 2020-12-05T17:33:49.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.503+0000 7fe5bffff700 1 -- 172.21.15.32:0/3113955440 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b000cd60 con 0x7fe5ac0038e0 2020-12-05T17:33:49.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.503+0000 7fe5bffff700 1 -- 172.21.15.32:0/3113955440 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5b00125f0 con 0x7fe5ac0038e0 2020-12-05T17:33:49.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.503+0000 7fe5c640e700 1 -- 172.21.15.32:0/3113955440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ac0038e0 msgr2=0x7fe5ac003ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:49.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.503+0000 7fe5c640e700 1 --2- 172.21.15.32:0/3113955440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ac0038e0 0x7fe5ac003ce0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7fe5b0001450 tx=0x7fe5b0007ee0).stop 2020-12-05T17:33:49.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.503+0000 7fe5c5c0d700 1 -- 172.21.15.32:0/3113955440 reap_dead start 2020-12-05T17:33:49.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.503+0000 7fe5c640e700 1 -- 172.21.15.32:0/3113955440 shutdown_connections 2020-12-05T17:33:49.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.503+0000 7fe5c640e700 1 -- 172.21.15.32:0/3113955440 >> 172.21.15.32:0/3113955440 conn(0x7fe5c00b8ee0 msgr2=0x7fe5c00b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:49.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.503+0000 7fe5c640e700 1 -- 172.21.15.32:0/3113955440 shutdown_connections 2020-12-05T17:33:49.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.503+0000 7fe5c640e700 1 -- 172.21.15.32:0/3113955440 wait complete. 2020-12-05T17:33:49.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.503+0000 7fe5c640e700 1 Processor -- start 2020-12-05T17:33:49.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.503+0000 7fe5c640e700 1 -- start start 2020-12-05T17:33:49.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5c640e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe5c00b3800 0x7fe5c013e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:49.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5c640e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5c0145630 0x7fe5c013e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:49.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5c640e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c0066aa0 0x7fe5c00647a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:49.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5c640e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c00b3be0 con 0x7fe5c0066aa0 2020-12-05T17:33:49.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5c640e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c0145ad0 con 0x7fe5c00b3800 2020-12-05T17:33:49.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5c640e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c0145d70 con 0x7fe5c0145630 2020-12-05T17:33:49.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5c5c0d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c0066aa0 0x7fe5c00647a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:49.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5c5c0d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c0066aa0 0x7fe5c00647a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34686/0 (socket says 172.21.15.32:34686) 2020-12-05T17:33:49.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5c5c0d700 1 -- 172.21.15.32:0/2594367750 learned_addr learned my addr 172.21.15.32:0/2594367750 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:49.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5c4c0b700 1 --2- 172.21.15.32:0/2594367750 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5c0145630 0x7fe5c013e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:49.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5c540c700 1 --2- 172.21.15.32:0/2594367750 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe5c00b3800 0x7fe5c013e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:49.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5c5c0d700 1 -- 172.21.15.32:0/2594367750 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5c0145630 msgr2=0x7fe5c013e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:49.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5c5c0d700 1 --2- 172.21.15.32:0/2594367750 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5c0145630 0x7fe5c013e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:49.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5c5c0d700 1 -- 172.21.15.32:0/2594367750 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe5c00b3800 msgr2=0x7fe5c013e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:49.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5c5c0d700 1 --2- 172.21.15.32:0/2594367750 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe5c00b3800 0x7fe5c013e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:49.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5c5c0d700 1 -- 172.21.15.32:0/2594367750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c0146010 con 0x7fe5c0066aa0 2020-12-05T17:33:49.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5c5c0d700 1 --2- 172.21.15.32:0/2594367750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c0066aa0 0x7fe5c00647a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7fe5b8000e50 tx=0x7fe5b8006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:49.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5be7fc700 1 -- 172.21.15.32:0/2594367750 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5b8016070 con 0x7fe5c0066aa0 2020-12-05T17:33:49.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5be7fc700 1 -- 172.21.15.32:0/2594367750 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b8011750 con 0x7fe5c0066aa0 2020-12-05T17:33:49.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5c640e700 1 -- 172.21.15.32:0/2594367750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c00af9b0 con 0x7fe5c0066aa0 2020-12-05T17:33:49.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5be7fc700 1 -- 172.21.15.32:0/2594367750 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5b8011ab0 con 0x7fe5c0066aa0 2020-12-05T17:33:49.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5c640e700 1 -- 172.21.15.32:0/2594367750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c0064c90 con 0x7fe5c0066aa0 2020-12-05T17:33:49.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5be7fc700 1 -- 172.21.15.32:0/2594367750 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe5b8011c10 con 0x7fe5c0066aa0 2020-12-05T17:33:49.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.507+0000 7fe5c640e700 1 -- 172.21.15.32:0/2594367750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c01461a0 con 0x7fe5c0066aa0 2020-12-05T17:33:49.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.511+0000 7fe5be7fc700 1 --2- 172.21.15.32:0/2594367750 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe5a0055470 0x7fe5a0057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:49.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.511+0000 7fe5c540c700 1 --2- 172.21.15.32:0/2594367750 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe5a0055470 0x7fe5a0057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:49.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.511+0000 7fe5be7fc700 1 -- 172.21.15.32:0/2594367750 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe5b806dc60 con 0x7fe5c0066aa0 2020-12-05T17:33:49.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.511+0000 7fe5c540c700 1 --2- 172.21.15.32:0/2594367750 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe5a0055470 0x7fe5a0057930 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fe5b001b000 tx=0x7fe5b0017000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:49.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.515+0000 7fe5be7fc700 1 -- 172.21.15.32:0/2594367750 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe5b8043cf0 con 0x7fe5c0066aa0 2020-12-05T17:33:49.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.659+0000 7fe5c640e700 1 -- 172.21.15.32:0/2594367750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5c0057050 con 0x7fe5c0066aa0 2020-12-05T17:33:49.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.659+0000 7fe5be7fc700 1 -- 172.21.15.32:0/2594367750 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16271 (secure 0 0 0) 0x7fe5b8047310 con 0x7fe5c0066aa0 2020-12-05T17:33:49.661 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:33:49.661 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":44,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:33:46.516416+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:33:43.025173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-12-05T17:33:44.344341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:33:49.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.663+0000 7fe5c640e700 1 -- 172.21.15.32:0/2594367750 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe5a0055470 msgr2=0x7fe5a0057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:49.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.663+0000 7fe5c640e700 1 --2- 172.21.15.32:0/2594367750 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe5a0055470 0x7fe5a0057930 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fe5b001b000 tx=0x7fe5b0017000).stop 2020-12-05T17:33:49.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.663+0000 7fe5c640e700 1 -- 172.21.15.32:0/2594367750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c0066aa0 msgr2=0x7fe5c00647a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:49.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.663+0000 7fe5c640e700 1 --2- 172.21.15.32:0/2594367750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c0066aa0 0x7fe5c00647a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7fe5b8000e50 tx=0x7fe5b8006dc0).stop 2020-12-05T17:33:49.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.663+0000 7fe5c640e700 1 -- 172.21.15.32:0/2594367750 shutdown_connections 2020-12-05T17:33:49.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.663+0000 7fe5c640e700 1 --2- 172.21.15.32:0/2594367750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c0066aa0 0x7fe5c00647a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:49.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.663+0000 7fe5c640e700 1 --2- 172.21.15.32:0/2594367750 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe5c00b3800 0x7fe5c013e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:49.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.663+0000 7fe5c640e700 1 --2- 172.21.15.32:0/2594367750 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe5a0055470 0x7fe5a0057930 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:49.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.663+0000 7fe5c640e700 1 --2- 172.21.15.32:0/2594367750 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5c0145630 0x7fe5c013e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:49.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.663+0000 7fe5c640e700 1 -- 172.21.15.32:0/2594367750 >> 172.21.15.32:0/2594367750 conn(0x7fe5c00b8ee0 msgr2=0x7fe5c00b7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:49.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.663+0000 7fe5c640e700 1 -- 172.21.15.32:0/2594367750 shutdown_connections 2020-12-05T17:33:49.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.663+0000 7fe5c640e700 1 -- 172.21.15.32:0/2594367750 wait complete. 2020-12-05T17:33:49.675 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:33:43.025173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:33:44.344341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:33:49.676 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:33:49.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.795+0000 7f52fb7a7700 1 Processor -- start 2020-12-05T17:33:49.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.795+0000 7f52fb7a7700 1 -- start start 2020-12-05T17:33:49.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52fb7a7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52f40b29d0 0x7f52f413ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:49.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52fb7a7700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f52f4142b70 0x7f52f413ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:49.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52fb7a7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52f40af340 0x7f52f40b0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:49.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52fb7a7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f4143b60 con 0x7f52f40af340 2020-12-05T17:33:49.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52fb7a7700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f413f460 con 0x7f52f4142b70 2020-12-05T17:33:49.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52fb7a7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f40b2db0 con 0x7f52f40b29d0 2020-12-05T17:33:49.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52f9d44700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52f40af340 0x7f52f40b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:49.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52f9d44700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52f40af340 0x7f52f40b0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34694/0 (socket says 172.21.15.32:34694) 2020-12-05T17:33:49.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52f9d44700 1 -- 172.21.15.32:0/993058637 learned_addr learned my addr 172.21.15.32:0/993058637 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:49.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52f9543700 1 --2- 172.21.15.32:0/993058637 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52f40b29d0 0x7f52f413ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:49.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52f8d42700 1 --2- 172.21.15.32:0/993058637 >> v2:172.21.15.61:3300/0 conn(0x7f52f4142b70 0x7f52f413ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:49.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52f9d44700 1 -- 172.21.15.32:0/993058637 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52f40b29d0 msgr2=0x7f52f413ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:49.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52f9d44700 1 --2- 172.21.15.32:0/993058637 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52f40b29d0 0x7f52f413ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:49.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52f9d44700 1 -- 172.21.15.32:0/993058637 >> v2:172.21.15.61:3300/0 conn(0x7f52f4142b70 msgr2=0x7f52f413ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:49.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52f9d44700 1 --2- 172.21.15.32:0/993058637 >> v2:172.21.15.61:3300/0 conn(0x7f52f4142b70 0x7f52f413ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:49.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52f9d44700 1 -- 172.21.15.32:0/993058637 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f40afd50 con 0x7f52f40af340 2020-12-05T17:33:49.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52f9d44700 1 --2- 172.21.15.32:0/993058637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52f40af340 0x7f52f40b0320 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f52ec0032b0 tx=0x7f52ec009270).ready entity=mon.0 client_cookie=1148f9f957034201 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:49.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52e3fff700 1 -- 172.21.15.32:0/993058637 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52ec00bc00 con 0x7f52f40af340 2020-12-05T17:33:49.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52e3fff700 1 -- 172.21.15.32:0/993058637 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ec00bd60 con 0x7f52f40af340 2020-12-05T17:33:49.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52e3fff700 1 -- 172.21.15.32:0/993058637 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52ec011400 con 0x7f52f40af340 2020-12-05T17:33:49.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52fb7a7700 1 -- 172.21.15.32:0/993058637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52f40af340 msgr2=0x7f52f40b0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:49.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52fb7a7700 1 --2- 172.21.15.32:0/993058637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52f40af340 0x7f52f40b0320 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f52ec0032b0 tx=0x7f52ec009270).stop 2020-12-05T17:33:49.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52fb7a7700 1 -- 172.21.15.32:0/993058637 shutdown_connections 2020-12-05T17:33:49.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52fb7a7700 1 --2- 172.21.15.32:0/993058637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52f40af340 0x7f52f40b0320 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:49.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52fb7a7700 1 --2- 172.21.15.32:0/993058637 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52f40b29d0 0x7f52f413ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:49.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52fb7a7700 1 --2- 172.21.15.32:0/993058637 >> v2:172.21.15.61:3300/0 conn(0x7f52f4142b70 0x7f52f413ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:49.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52fb7a7700 1 -- 172.21.15.32:0/993058637 >> 172.21.15.32:0/993058637 conn(0x7f52f4062310 msgr2=0x7f52f40626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:49.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52fb7a7700 1 -- 172.21.15.32:0/993058637 shutdown_connections 2020-12-05T17:33:49.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.799+0000 7f52fb7a7700 1 -- 172.21.15.32:0/993058637 wait complete. 2020-12-05T17:33:49.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52fb7a7700 1 Processor -- start 2020-12-05T17:33:49.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52fb7a7700 1 -- start start 2020-12-05T17:33:49.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52fb7a7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52f40af340 0x7f52f414ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:49.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52fb7a7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52f40b29d0 0x7f52f410eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:49.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52fb7a7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52f4063040 0x7f52f40634c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:49.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52fb7a7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f4065380 con 0x7f52f4063040 2020-12-05T17:33:49.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52fb7a7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f4066610 con 0x7f52f40b29d0 2020-12-05T17:33:49.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52fb7a7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f40639b0 con 0x7f52f40af340 2020-12-05T17:33:49.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52f9d44700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52f4063040 0x7f52f40634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:49.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52f9d44700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52f4063040 0x7f52f40634c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34700/0 (socket says 172.21.15.32:34700) 2020-12-05T17:33:49.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52f9d44700 1 -- 172.21.15.32:0/2270506554 learned_addr learned my addr 172.21.15.32:0/2270506554 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:49.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52f9543700 1 --2- 172.21.15.32:0/2270506554 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52f40af340 0x7f52f414ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:49.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52f8d42700 1 --2- 172.21.15.32:0/2270506554 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52f40b29d0 0x7f52f410eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:49.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52f9d44700 1 -- 172.21.15.32:0/2270506554 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52f40af340 msgr2=0x7f52f414ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:49.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52f9d44700 1 --2- 172.21.15.32:0/2270506554 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52f40af340 0x7f52f414ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:49.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52f9d44700 1 -- 172.21.15.32:0/2270506554 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52f40b29d0 msgr2=0x7f52f410eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:49.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52f9d44700 1 --2- 172.21.15.32:0/2270506554 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52f40b29d0 0x7f52f410eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:49.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52f9d44700 1 -- 172.21.15.32:0/2270506554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f40655b0 con 0x7f52f4063040 2020-12-05T17:33:49.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52f9543700 1 --2- 172.21.15.32:0/2270506554 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52f40af340 0x7f52f414ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:33:49.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52f9d44700 1 --2- 172.21.15.32:0/2270506554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52f4063040 0x7f52f40634c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f52f40667a0 tx=0x7f52ec016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:49.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52e27fc700 1 -- 172.21.15.32:0/2270506554 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52ec01e610 con 0x7f52f4063040 2020-12-05T17:33:49.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52e27fc700 1 -- 172.21.15.32:0/2270506554 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ec01e770 con 0x7f52f4063040 2020-12-05T17:33:49.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52fb7a7700 1 -- 172.21.15.32:0/2270506554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f4066370 con 0x7f52f4063040 2020-12-05T17:33:49.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.803+0000 7f52e27fc700 1 -- 172.21.15.32:0/2270506554 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52ec01ead0 con 0x7f52f4063040 2020-12-05T17:33:49.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.807+0000 7f52fb7a7700 1 -- 172.21.15.32:0/2270506554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f410efb0 con 0x7f52f4063040 2020-12-05T17:33:49.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.807+0000 7f52fb7a7700 1 -- 172.21.15.32:0/2270506554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f4057050 con 0x7f52f4063040 2020-12-05T17:33:49.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.807+0000 7f52e27fc700 1 -- 172.21.15.32:0/2270506554 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f52ec00f070 con 0x7f52f4063040 2020-12-05T17:33:49.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.807+0000 7f52e27fc700 1 --2- 172.21.15.32:0/2270506554 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f52d4055440 0x7f52d4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:49.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.807+0000 7f52e27fc700 1 -- 172.21.15.32:0/2270506554 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f52ec06f7d0 con 0x7f52f4063040 2020-12-05T17:33:49.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.807+0000 7f52f9543700 1 --2- 172.21.15.32:0/2270506554 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f52d4055440 0x7f52d4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.807+0000 7f52f9543700 1 --2- 172.21.15.32:0/2270506554 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f52d4055440 0x7f52d4057900 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f52f000c000 tx=0x7f52f0008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.815+0000 7f52e27fc700 1 -- 172.21.15.32:0/2270506554 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f52ec045970 con 0x7f52f4063040 2020-12-05T17:33:49.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.955+0000 7f52fb7a7700 1 -- 172.21.15.32:0/2270506554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f52f405aab0 con 0x7f52f4063040 2020-12-05T17:33:49.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.955+0000 7f52e27fc700 1 -- 172.21.15.32:0/2270506554 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v44) v1 ==== 110+0+10 (secure 0 0 0) 0x7f52ec0150b0 con 0x7f52f4063040 2020-12-05T17:33:49.957 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:33:49.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.959+0000 7f52fb7a7700 1 -- 172.21.15.32:0/2270506554 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f52d4055440 msgr2=0x7f52d4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:49.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.959+0000 7f52fb7a7700 1 --2- 172.21.15.32:0/2270506554 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f52d4055440 0x7f52d4057900 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f52f000c000 tx=0x7f52f0008040).stop 2020-12-05T17:33:49.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.959+0000 7f52fb7a7700 1 -- 172.21.15.32:0/2270506554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52f4063040 msgr2=0x7f52f40634c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:49.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.959+0000 7f52fb7a7700 1 --2- 172.21.15.32:0/2270506554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52f4063040 0x7f52f40634c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f52f40667a0 tx=0x7f52ec016000).stop 2020-12-05T17:33:49.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.959+0000 7f52fb7a7700 1 -- 172.21.15.32:0/2270506554 shutdown_connections 2020-12-05T17:33:49.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.959+0000 7f52fb7a7700 1 --2- 172.21.15.32:0/2270506554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52f4063040 0x7f52f40634c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:49.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.959+0000 7f52fb7a7700 1 --2- 172.21.15.32:0/2270506554 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f52d4055440 0x7f52d4057900 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:49.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.959+0000 7f52fb7a7700 1 --2- 172.21.15.32:0/2270506554 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52f40af340 0x7f52f414ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:49.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.959+0000 7f52fb7a7700 1 --2- 172.21.15.32:0/2270506554 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52f40b29d0 0x7f52f410eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:49.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.959+0000 7f52fb7a7700 1 -- 172.21.15.32:0/2270506554 >> 172.21.15.32:0/2270506554 conn(0x7f52f4062310 msgr2=0x7f52f40b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:49.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.959+0000 7f52fb7a7700 1 -- 172.21.15.32:0/2270506554 shutdown_connections 2020-12-05T17:33:49.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:49.959+0000 7f52fb7a7700 1 -- 172.21.15.32:0/2270506554 wait complete. 2020-12-05T17:33:49.970 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:33:50.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6b099c700 1 Processor -- start 2020-12-05T17:33:50.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6b099c700 1 -- start start 2020-12-05T17:33:50.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6b099c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe6ac0b1dc0 0x7fe6ac0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:50.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6b099c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fe6ac0af5f0 0x7fe6ac0aed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:50.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6b099c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6ac1468e0 con 0x7fe6ac0afd30 2020-12-05T17:33:50.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6b099c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6ac142c90 con 0x7fe6ac0b1dc0 2020-12-05T17:33:50.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6b099c700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6ac146b10 con 0x7fe6ac0af5f0 2020-12-05T17:33:50.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6aad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe6ac0b1dc0 0x7fe6ac0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:50.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6a3fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe6ac0afd30 0x7fe6ac0ae600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37354/0 (socket says 172.21.15.32:37354) 2020-12-05T17:33:50.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6a3fff700 1 -- 172.21.15.32:0/1651815368 learned_addr learned my addr 172.21.15.32:0/1651815368 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:33:50.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6ab59e700 1 --2- 172.21.15.32:0/1651815368 >> v2:172.21.15.61:3300/0 conn(0x7fe6ac0af5f0 0x7fe6ac0aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:50.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6aad9d700 1 -- 172.21.15.32:0/1651815368 >> v2:172.21.15.61:3300/0 conn(0x7fe6ac0af5f0 msgr2=0x7fe6ac0aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:50.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6aad9d700 1 --2- 172.21.15.32:0/1651815368 >> v2:172.21.15.61:3300/0 conn(0x7fe6ac0af5f0 0x7fe6ac0aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6aad9d700 1 -- 172.21.15.32:0/1651815368 >> v1:172.21.15.32:6789/0 conn(0x7fe6ac0afd30 legacy=0x7fe6ac0ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6aa59c700 1 -- 172.21.15.32:0/1651815368 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3642404456 0 0) 0x7fe6ac1468e0 con 0x7fe6ac0afd30 2020-12-05T17:33:50.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6aad9d700 1 -- 172.21.15.32:0/1651815368 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ac146db0 con 0x7fe6ac0b1dc0 2020-12-05T17:33:50.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6aad9d700 1 --2- 172.21.15.32:0/1651815368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac0b1dc0 0x7fe6ac0b21a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fe69c002f30 tx=0x7fe69c006dc0).ready entity=mon.0 client_cookie=95f6f6bcb9a86f01 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:50.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6aa59c700 1 -- 172.21.15.32:0/1651815368 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe69c0097b0 con 0x7fe6ac0b1dc0 2020-12-05T17:33:50.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6aa59c700 1 -- 172.21.15.32:0/1651815368 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c009910 con 0x7fe6ac0b1dc0 2020-12-05T17:33:50.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6aa59c700 1 -- 172.21.15.32:0/1651815368 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe69c009c70 con 0x7fe6ac0b1dc0 2020-12-05T17:33:50.093 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6b099c700 1 -- 172.21.15.32:0/1651815368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac0b1dc0 msgr2=0x7fe6ac0b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.093 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6b099c700 1 --2- 172.21.15.32:0/1651815368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac0b1dc0 0x7fe6ac0b21a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fe69c002f30 tx=0x7fe69c006dc0).stop 2020-12-05T17:33:50.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6b099c700 1 -- 172.21.15.32:0/1651815368 shutdown_connections 2020-12-05T17:33:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6b099c700 1 --2- 172.21.15.32:0/1651815368 >> v2:172.21.15.61:3300/0 conn(0x7fe6ac0af5f0 0x7fe6ac0aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6b099c700 1 --2- 172.21.15.32:0/1651815368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac0b1dc0 0x7fe6ac0b21a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6b099c700 1 -- 172.21.15.32:0/1651815368 >> 172.21.15.32:0/1651815368 conn(0x7fe6ac0620b0 msgr2=0x7fe6ac062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6b099c700 1 -- 172.21.15.32:0/1651815368 shutdown_connections 2020-12-05T17:33:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.087+0000 7fe6b099c700 1 -- 172.21.15.32:0/1651815368 wait complete. 2020-12-05T17:33:50.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6b099c700 1 Processor -- start 2020-12-05T17:33:50.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6b099c700 1 -- start start 2020-12-05T17:33:50.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6b099c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe6ac0af5f0 0x7fe6ac149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:50.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6b099c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe6ac0afd30 0x7fe6ac063f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:50.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6b099c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac0b1dc0 0x7fe6ac13f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:50.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6b099c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac064860 con 0x7fe6ac0b1dc0 2020-12-05T17:33:50.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6b099c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac064b00 con 0x7fe6ac0afd30 2020-12-05T17:33:50.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6b099c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac0645a0 con 0x7fe6ac0af5f0 2020-12-05T17:33:50.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6ab59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac0b1dc0 0x7fe6ac13f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:50.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6ab59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac0b1dc0 0x7fe6ac13f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34714/0 (socket says 172.21.15.32:34714) 2020-12-05T17:33:50.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6ab59e700 1 -- 172.21.15.32:0/3792183113 learned_addr learned my addr 172.21.15.32:0/3792183113 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:50.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6aad9d700 1 --2- 172.21.15.32:0/3792183113 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe6ac0af5f0 0x7fe6ac149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:50.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6a3fff700 1 --2- 172.21.15.32:0/3792183113 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe6ac0afd30 0x7fe6ac063f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:50.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6ab59e700 1 -- 172.21.15.32:0/3792183113 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe6ac0af5f0 msgr2=0x7fe6ac149940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6ab59e700 1 --2- 172.21.15.32:0/3792183113 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe6ac0af5f0 0x7fe6ac149940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6ab59e700 1 -- 172.21.15.32:0/3792183113 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe6ac0afd30 msgr2=0x7fe6ac063f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6ab59e700 1 --2- 172.21.15.32:0/3792183113 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe6ac0afd30 0x7fe6ac063f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6ab59e700 1 -- 172.21.15.32:0/3792183113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ac064da0 con 0x7fe6ac0b1dc0 2020-12-05T17:33:50.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6ab59e700 1 --2- 172.21.15.32:0/3792183113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac0b1dc0 0x7fe6ac13f020 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fe6a4006dd0 tx=0x7fe6a40091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:50.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6a8d99700 1 -- 172.21.15.32:0/3792183113 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6a400c950 con 0x7fe6ac0b1dc0 2020-12-05T17:33:50.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6a8d99700 1 -- 172.21.15.32:0/3792183113 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a400cab0 con 0x7fe6ac0b1dc0 2020-12-05T17:33:50.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6a8d99700 1 -- 172.21.15.32:0/3792183113 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6a400ce10 con 0x7fe6ac0b1dc0 2020-12-05T17:33:50.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6b099c700 1 -- 172.21.15.32:0/3792183113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6ac063040 con 0x7fe6ac0b1dc0 2020-12-05T17:33:50.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.091+0000 7fe6b099c700 1 -- 172.21.15.32:0/3792183113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6ac066d20 con 0x7fe6ac0b1dc0 2020-12-05T17:33:50.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.095+0000 7fe6b099c700 1 -- 172.21.15.32:0/3792183113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6ac063350 con 0x7fe6ac0b1dc0 2020-12-05T17:33:50.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.095+0000 7fe6a8d99700 1 -- 172.21.15.32:0/3792183113 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe6a4022070 con 0x7fe6ac0b1dc0 2020-12-05T17:33:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.095+0000 7fe6a8d99700 1 --2- 172.21.15.32:0/3792183113 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe6900553b0 0x7fe690057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.095+0000 7fe6a8d99700 1 -- 172.21.15.32:0/3792183113 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe6a406e640 con 0x7fe6ac0b1dc0 2020-12-05T17:33:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.095+0000 7fe6aad9d700 1 --2- 172.21.15.32:0/3792183113 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe6900553b0 0x7fe690057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:50.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.095+0000 7fe6aad9d700 1 --2- 172.21.15.32:0/3792183113 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe6900553b0 0x7fe690057870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fe69c01a000 tx=0x7fe69c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:50.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.103+0000 7fe6a8d99700 1 -- 172.21.15.32:0/3792183113 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe6a40447e0 con 0x7fe6ac0b1dc0 2020-12-05T17:33:50.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.247+0000 7fe6b099c700 1 -- 172.21.15.32:0/3792183113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe6ac057050 con 0x7fe6ac0b1dc0 2020-12-05T17:33:50.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.247+0000 7fe6a8d99700 1 -- 172.21.15.32:0/3792183113 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v44) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe6a4016020 con 0x7fe6ac0b1dc0 2020-12-05T17:33:50.248 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:33:50.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.251+0000 7fe6b099c700 1 -- 172.21.15.32:0/3792183113 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe6900553b0 msgr2=0x7fe690057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.251+0000 7fe6b099c700 1 --2- 172.21.15.32:0/3792183113 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe6900553b0 0x7fe690057870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fe69c01a000 tx=0x7fe69c015000).stop 2020-12-05T17:33:50.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.251+0000 7fe6b099c700 1 -- 172.21.15.32:0/3792183113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac0b1dc0 msgr2=0x7fe6ac13f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.251+0000 7fe6b099c700 1 --2- 172.21.15.32:0/3792183113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac0b1dc0 0x7fe6ac13f020 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fe6a4006dd0 tx=0x7fe6a40091a0).stop 2020-12-05T17:33:50.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.251+0000 7fe6b099c700 1 -- 172.21.15.32:0/3792183113 shutdown_connections 2020-12-05T17:33:50.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.251+0000 7fe6b099c700 1 --2- 172.21.15.32:0/3792183113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac0b1dc0 0x7fe6ac13f020 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.251+0000 7fe6b099c700 1 --2- 172.21.15.32:0/3792183113 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe6900553b0 0x7fe690057870 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.251+0000 7fe6b099c700 1 --2- 172.21.15.32:0/3792183113 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe6ac0af5f0 0x7fe6ac149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.251+0000 7fe6b099c700 1 --2- 172.21.15.32:0/3792183113 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe6ac0afd30 0x7fe6ac063f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.251+0000 7fe6b099c700 1 -- 172.21.15.32:0/3792183113 >> 172.21.15.32:0/3792183113 conn(0x7fe6ac0620b0 msgr2=0x7fe6ac145de0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:50.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.251+0000 7fe6b099c700 1 -- 172.21.15.32:0/3792183113 shutdown_connections 2020-12-05T17:33:50.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.251+0000 7fe6b099c700 1 -- 172.21.15.32:0/3792183113 wait complete. 2020-12-05T17:33:50.267 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:33:50.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.383+0000 7f905678a700 1 Processor -- start 2020-12-05T17:33:50.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.383+0000 7f905678a700 1 -- start start 2020-12-05T17:33:50.385 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.383+0000 7f905678a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f905005f3b0 0x7f905005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:50.385 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.383+0000 7f905678a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90500b2df0 con 0x7f90500b2fb0 2020-12-05T17:33:50.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.383+0000 7f905678a700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90500b3c90 con 0x7f905005feb0 2020-12-05T17:33:50.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.383+0000 7f905678a700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90500b2b50 con 0x7f905005f3b0 2020-12-05T17:33:50.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.383+0000 7f9055f89700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f90500b2fb0 0x7f9050142cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37370/0 (socket says 172.21.15.32:37370) 2020-12-05T17:33:50.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.383+0000 7f9055f89700 1 -- 172.21.15.32:0/3796166117 learned_addr learned my addr 172.21.15.32:0/3796166117 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:33:50.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.383+0000 7f9055788700 1 --2- 172.21.15.32:0/3796166117 >> v2:172.21.15.61:3300/0 conn(0x7f905005f3b0 0x7f905005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:50.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.383+0000 7f9043fff700 1 -- 172.21.15.32:0/3796166117 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3514882887 0 0) 0x7f90500b2df0 con 0x7f90500b2fb0 2020-12-05T17:33:50.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.383+0000 7f9043fff700 1 -- 172.21.15.32:0/3796166117 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9034003230 con 0x7f90500b2fb0 2020-12-05T17:33:50.388 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.383+0000 7f9055788700 1 -- 172.21.15.32:0/3796166117 >> v1:172.21.15.61:6789/0 conn(0x7f905005feb0 legacy=0x7f905013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.388 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.383+0000 7f9043fff700 1 -- 172.21.15.32:0/3796166117 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 654282562 0 0) 0x7f9034003230 con 0x7f90500b2fb0 2020-12-05T17:33:50.389 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.383+0000 7f9055788700 1 -- 172.21.15.32:0/3796166117 >> v1:172.21.15.32:6789/0 conn(0x7f90500b2fb0 legacy=0x7f9050142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.389 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.383+0000 7f9055788700 1 -- 172.21.15.32:0/3796166117 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9050142820 con 0x7f905005f3b0 2020-12-05T17:33:50.389 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.387+0000 7f9055788700 1 --2- 172.21.15.32:0/3796166117 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f905005f3b0 0x7f905005f790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f9048001bf0 tx=0x7f9044006dc0).ready entity=mon.1 client_cookie=54455cf75aa0e315 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:50.389 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.387+0000 7f9043fff700 1 -- 172.21.15.32:0/3796166117 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9048001a50 con 0x7f905005f3b0 2020-12-05T17:33:50.389 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.387+0000 7f9043fff700 1 -- 172.21.15.32:0/3796166117 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9044009620 con 0x7f905005f3b0 2020-12-05T17:33:50.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.387+0000 7f9043fff700 1 -- 172.21.15.32:0/3796166117 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9044009790 con 0x7f905005f3b0 2020-12-05T17:33:50.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.387+0000 7f905678a700 1 -- 172.21.15.32:0/3796166117 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f905005f3b0 msgr2=0x7f905005f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.387+0000 7f905678a700 1 --2- 172.21.15.32:0/3796166117 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f905005f3b0 0x7f905005f790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f9048001bf0 tx=0x7f9044006dc0).stop 2020-12-05T17:33:50.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.387+0000 7f905678a700 1 -- 172.21.15.32:0/3796166117 shutdown_connections 2020-12-05T17:33:50.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.387+0000 7f905678a700 1 --2- 172.21.15.32:0/3796166117 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f905005f3b0 0x7f905005f790 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.387+0000 7f905678a700 1 -- 172.21.15.32:0/3796166117 >> 172.21.15.32:0/3796166117 conn(0x7f90500b8f20 msgr2=0x7f90500b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:50.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.387+0000 7f905678a700 1 -- 172.21.15.32:0/3796166117 shutdown_connections 2020-12-05T17:33:50.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.387+0000 7f905678a700 1 -- 172.21.15.32:0/3796166117 wait complete. 2020-12-05T17:33:50.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.387+0000 7f905678a700 1 Processor -- start 2020-12-05T17:33:50.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.387+0000 7f905678a700 1 -- start start 2020-12-05T17:33:50.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f905678a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f905005f3b0 0x7f90501482a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:50.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f905678a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f905005feb0 0x7f905013f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:50.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f905678a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90500b2fb0 0x7f905013f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:50.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f905678a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9050147800 con 0x7f905005feb0 2020-12-05T17:33:50.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f905678a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90501472c0 con 0x7f905005f3b0 2020-12-05T17:33:50.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f905678a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9050147560 con 0x7f90500b2fb0 2020-12-05T17:33:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f9054f87700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f905005feb0 0x7f905013f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f9055788700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f905005f3b0 0x7f90501482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f9055788700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f905005f3b0 0x7f90501482a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:54616/0 (socket says 172.21.15.32:54616) 2020-12-05T17:33:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f9055788700 1 -- 172.21.15.32:0/3596685587 learned_addr learned my addr 172.21.15.32:0/3596685587 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f9054f87700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f905005feb0 0x7f905013f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34728/0 (socket says 172.21.15.32:34728) 2020-12-05T17:33:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f9055f89700 1 --2- 172.21.15.32:0/3596685587 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90500b2fb0 0x7f905013f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f9054f87700 1 -- 172.21.15.32:0/3596685587 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90500b2fb0 msgr2=0x7f905013f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f9054f87700 1 --2- 172.21.15.32:0/3596685587 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90500b2fb0 0x7f905013f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f9054f87700 1 -- 172.21.15.32:0/3596685587 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f905005f3b0 msgr2=0x7f90501482a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f9054f87700 1 --2- 172.21.15.32:0/3596685587 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f905005f3b0 0x7f90501482a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f9054f87700 1 -- 172.21.15.32:0/3596685587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905013fa80 con 0x7f905005feb0 2020-12-05T17:33:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f9054f87700 1 --2- 172.21.15.32:0/3596685587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f905005feb0 0x7f905013f0a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f904c001110 tx=0x7f904c007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f90427fc700 1 -- 172.21.15.32:0/3596685587 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f904c013410 con 0x7f905005feb0 2020-12-05T17:33:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f90427fc700 1 -- 172.21.15.32:0/3596685587 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c013570 con 0x7f905005feb0 2020-12-05T17:33:50.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f905678a700 1 -- 172.21.15.32:0/3596685587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9050142320 con 0x7f905005feb0 2020-12-05T17:33:50.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f90427fc700 1 -- 172.21.15.32:0/3596685587 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f904c0138d0 con 0x7f905005feb0 2020-12-05T17:33:50.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f905678a700 1 -- 172.21.15.32:0/3596685587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f905013d8e0 con 0x7f905005feb0 2020-12-05T17:33:50.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.391+0000 7f905678a700 1 -- 172.21.15.32:0/3596685587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f905013ede0 con 0x7f905005feb0 2020-12-05T17:33:50.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.395+0000 7f90427fc700 1 -- 172.21.15.32:0/3596685587 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f904c010030 con 0x7f905005feb0 2020-12-05T17:33:50.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.395+0000 7f90427fc700 1 --2- 172.21.15.32:0/3596685587 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f90380554e0 0x7f90380579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:50.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.395+0000 7f90427fc700 1 -- 172.21.15.32:0/3596685587 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f904c06df80 con 0x7f905005feb0 2020-12-05T17:33:50.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.395+0000 7f9055788700 1 --2- 172.21.15.32:0/3596685587 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f90380554e0 0x7f90380579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:50.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.395+0000 7f9055788700 1 --2- 172.21.15.32:0/3596685587 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f90380554e0 0x7f90380579a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f9044019000 tx=0x7f90500b9f00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:50.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.399+0000 7f90427fc700 1 -- 172.21.15.32:0/3596685587 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f905013ede0 con 0x7f905005feb0 2020-12-05T17:33:50.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.539+0000 7f905678a700 1 -- 172.21.15.32:0/3596685587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f905013ede0 con 0x7f905005feb0 2020-12-05T17:33:50.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.543+0000 7f90427fc700 1 -- 172.21.15.32:0/3596685587 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+10 (secure 0 0 0) 0x7f905013ede0 con 0x7f905005feb0 2020-12-05T17:33:50.543 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:33:50.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.543+0000 7f905678a700 1 -- 172.21.15.32:0/3596685587 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f90380554e0 msgr2=0x7f90380579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.543+0000 7f905678a700 1 --2- 172.21.15.32:0/3596685587 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f90380554e0 0x7f90380579a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f9044019000 tx=0x7f90500b9f00).stop 2020-12-05T17:33:50.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.543+0000 7f905678a700 1 -- 172.21.15.32:0/3596685587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f905005feb0 msgr2=0x7f905013f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.543+0000 7f905678a700 1 --2- 172.21.15.32:0/3596685587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f905005feb0 0x7f905013f0a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f904c001110 tx=0x7f904c007ac0).stop 2020-12-05T17:33:50.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.543+0000 7f905678a700 1 -- 172.21.15.32:0/3596685587 shutdown_connections 2020-12-05T17:33:50.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.543+0000 7f905678a700 1 --2- 172.21.15.32:0/3596685587 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f90380554e0 0x7f90380579a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.543+0000 7f905678a700 1 --2- 172.21.15.32:0/3596685587 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90500b2fb0 0x7f905013f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.543+0000 7f905678a700 1 --2- 172.21.15.32:0/3596685587 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f905005f3b0 0x7f90501482a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.543+0000 7f905678a700 1 --2- 172.21.15.32:0/3596685587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f905005feb0 0x7f905013f0a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.543+0000 7f905678a700 1 -- 172.21.15.32:0/3596685587 >> 172.21.15.32:0/3596685587 conn(0x7f90500b8f20 msgr2=0x7f90500b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:50.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.543+0000 7f905678a700 1 -- 172.21.15.32:0/3596685587 shutdown_connections 2020-12-05T17:33:50.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.543+0000 7f905678a700 1 -- 172.21.15.32:0/3596685587 wait complete. 2020-12-05T17:33:50.564 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:33:50.676 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.675+0000 7f7551f78700 1 Processor -- start 2020-12-05T17:33:50.676 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.675+0000 7f7551f78700 1 -- start start 2020-12-05T17:33:50.678 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.675+0000 7f7551f78700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f754c0af4b0 0x7f754c146510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:50.679 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.675+0000 7f7551f78700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f754c061780 0x7f754c0afe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:50.679 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.675+0000 7f7551f78700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f754c0b23e0 con 0x7f754c0b2000 2020-12-05T17:33:50.679 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.675+0000 7f7551f78700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f754c146a00 con 0x7f754c061780 2020-12-05T17:33:50.679 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.675+0000 7f7551f78700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f754c0ae7b0 con 0x7f754c0af4b0 2020-12-05T17:33:50.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.675+0000 7f7551777700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f754c061780 0x7f754c0afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:50.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.675+0000 7f7551777700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f754c061780 0x7f754c0afe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34734/0 (socket says 172.21.15.32:34734) 2020-12-05T17:33:50.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.675+0000 7f7551777700 1 -- 172.21.15.32:0/2982088920 learned_addr learned my addr 172.21.15.32:0/2982088920 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:50.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.675+0000 7f754bfff700 1 --2- 172.21.15.32:0/2982088920 >> v2:172.21.15.61:3300/0 conn(0x7f754c0af4b0 0x7f754c146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:50.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.675+0000 7f7551777700 1 -- 172.21.15.32:0/2982088920 >> v2:172.21.15.61:3300/0 conn(0x7f754c0af4b0 msgr2=0x7f754c146510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:33:50.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.675+0000 7f7551777700 1 --2- 172.21.15.32:0/2982088920 >> v2:172.21.15.61:3300/0 conn(0x7f754c0af4b0 0x7f754c146510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.675+0000 7f7551777700 1 -- 172.21.15.32:0/2982088920 >> v1:172.21.15.32:6789/0 conn(0x7f754c0b2000 legacy=0x7f754c145e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.675+0000 7f7551777700 1 -- 172.21.15.32:0/2982088920 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c0aea50 con 0x7f754c061780 2020-12-05T17:33:50.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.675+0000 7f754b7fe700 1 -- 172.21.15.32:0/2982088920 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989664763 0 0) 0x7f754c0b23e0 con 0x7f754c0b2000 2020-12-05T17:33:50.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.675+0000 7f7551777700 1 --2- 172.21.15.32:0/2982088920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f754c061780 0x7f754c0afe20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f75440032b0 tx=0x7f7544009270).ready entity=mon.0 client_cookie=b66393522d2e95a0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:50.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.679+0000 7f754b7fe700 1 -- 172.21.15.32:0/2982088920 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7544009e60 con 0x7f754c061780 2020-12-05T17:33:50.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.679+0000 7f754b7fe700 1 -- 172.21.15.32:0/2982088920 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754400bc00 con 0x7f754c061780 2020-12-05T17:33:50.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.679+0000 7f754b7fe700 1 -- 172.21.15.32:0/2982088920 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f754400bd70 con 0x7f754c061780 2020-12-05T17:33:50.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.679+0000 7f7551f78700 1 -- 172.21.15.32:0/2982088920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f754c061780 msgr2=0x7f754c0afe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.679+0000 7f7551f78700 1 --2- 172.21.15.32:0/2982088920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f754c061780 0x7f754c0afe20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f75440032b0 tx=0x7f7544009270).stop 2020-12-05T17:33:50.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.679+0000 7f7551f78700 1 -- 172.21.15.32:0/2982088920 shutdown_connections 2020-12-05T17:33:50.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.679+0000 7f7551f78700 1 --2- 172.21.15.32:0/2982088920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f754c061780 0x7f754c0afe20 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.679+0000 7f7551f78700 1 --2- 172.21.15.32:0/2982088920 >> v2:172.21.15.61:3300/0 conn(0x7f754c0af4b0 0x7f754c146510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.679+0000 7f7551f78700 1 -- 172.21.15.32:0/2982088920 >> 172.21.15.32:0/2982088920 conn(0x7f754c0b5620 msgr2=0x7f754c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:50.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.679+0000 7f7551f78700 1 -- 172.21.15.32:0/2982088920 shutdown_connections 2020-12-05T17:33:50.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.679+0000 7f7551f78700 1 -- 172.21.15.32:0/2982088920 wait complete. 2020-12-05T17:33:50.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.679+0000 7f7551f78700 1 Processor -- start 2020-12-05T17:33:50.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.679+0000 7f7551f78700 1 -- start start 2020-12-05T17:33:50.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.679+0000 7f7551f78700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f754c061780 0x7f754c063380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:50.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.679+0000 7f7551f78700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f754c0af4b0 0x7f754c065000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:50.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.679+0000 7f7551f78700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f754c0b2000 0x7f754c0654f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:50.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.679+0000 7f7551f78700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c14ab10 con 0x7f754c0af4b0 2020-12-05T17:33:50.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.679+0000 7f7551f78700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c149d00 con 0x7f754c061780 2020-12-05T17:33:50.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.679+0000 7f7551f78700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754c063870 con 0x7f754c0b2000 2020-12-05T17:33:50.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.679+0000 7f754bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f754c0af4b0 0x7f754c065000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:50.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.679+0000 7f754bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f754c0af4b0 0x7f754c065000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34738/0 (socket says 172.21.15.32:34738) 2020-12-05T17:33:50.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.679+0000 7f754bfff700 1 -- 172.21.15.32:0/1979123969 learned_addr learned my addr 172.21.15.32:0/1979123969 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:50.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.683+0000 7f7551777700 1 --2- 172.21.15.32:0/1979123969 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f754c0b2000 0x7f754c0654f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:50.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.683+0000 7f7550f76700 1 --2- 172.21.15.32:0/1979123969 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f754c061780 0x7f754c063380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:50.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.683+0000 7f754bfff700 1 -- 172.21.15.32:0/1979123969 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f754c0b2000 msgr2=0x7f754c0654f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.683+0000 7f754bfff700 1 --2- 172.21.15.32:0/1979123969 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f754c0b2000 0x7f754c0654f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.683+0000 7f754bfff700 1 -- 172.21.15.32:0/1979123969 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f754c061780 msgr2=0x7f754c063380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.683+0000 7f754bfff700 1 --2- 172.21.15.32:0/1979123969 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f754c061780 0x7f754c063380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.683+0000 7f754bfff700 1 -- 172.21.15.32:0/1979123969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c063ab0 con 0x7f754c0af4b0 2020-12-05T17:33:50.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.683+0000 7f754bfff700 1 --2- 172.21.15.32:0/1979123969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f754c0af4b0 0x7f754c065000 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f753400c000 tx=0x7f7534006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:50.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.683+0000 7f7549ffb700 1 -- 172.21.15.32:0/1979123969 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f753400c9b0 con 0x7f754c0af4b0 2020-12-05T17:33:50.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.683+0000 7f7549ffb700 1 -- 172.21.15.32:0/1979123969 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753400cb10 con 0x7f754c0af4b0 2020-12-05T17:33:50.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.683+0000 7f7549ffb700 1 -- 172.21.15.32:0/1979123969 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f753400ce70 con 0x7f754c0af4b0 2020-12-05T17:33:50.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.683+0000 7f7551f78700 1 -- 172.21.15.32:0/1979123969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f754c149a60 con 0x7f754c0af4b0 2020-12-05T17:33:50.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.683+0000 7f7551f78700 1 -- 172.21.15.32:0/1979123969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f754c149660 con 0x7f754c0af4b0 2020-12-05T17:33:50.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.683+0000 7f7551f78700 1 -- 172.21.15.32:0/1979123969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f754c057050 con 0x7f754c0af4b0 2020-12-05T17:33:50.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.683+0000 7f7549ffb700 1 -- 172.21.15.32:0/1979123969 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f753401c460 con 0x7f754c0af4b0 2020-12-05T17:33:50.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.683+0000 7f7549ffb700 1 --2- 172.21.15.32:0/1979123969 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7524055440 0x7f7524057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:50.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.683+0000 7f7550f76700 1 --2- 172.21.15.32:0/1979123969 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7524055440 0x7f7524057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:50.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.683+0000 7f7549ffb700 1 -- 172.21.15.32:0/1979123969 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f753406e550 con 0x7f754c0af4b0 2020-12-05T17:33:50.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.687+0000 7f7550f76700 1 --2- 172.21.15.32:0/1979123969 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7524055440 0x7f7524057900 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f753c00d000 tx=0x7f753c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:50.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.691+0000 7f7549ffb700 1 -- 172.21.15.32:0/1979123969 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7534044580 con 0x7f754c0af4b0 2020-12-05T17:33:50.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.831+0000 7f7551f78700 1 -- 172.21.15.32:0/1979123969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f754c14a800 con 0x7f754c0af4b0 2020-12-05T17:33:50.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.831+0000 7f7549ffb700 1 -- 172.21.15.32:0/1979123969 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7534047ba0 con 0x7f754c0af4b0 2020-12-05T17:33:50.831 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:33:50.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.831+0000 7f7551f78700 1 -- 172.21.15.32:0/1979123969 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7524055440 msgr2=0x7f7524057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.831+0000 7f7551f78700 1 --2- 172.21.15.32:0/1979123969 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7524055440 0x7f7524057900 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f753c00d000 tx=0x7f753c009040).stop 2020-12-05T17:33:50.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.831+0000 7f7551f78700 1 -- 172.21.15.32:0/1979123969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f754c0af4b0 msgr2=0x7f754c065000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.831+0000 7f7551f78700 1 --2- 172.21.15.32:0/1979123969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f754c0af4b0 0x7f754c065000 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f753400c000 tx=0x7f7534006dc0).stop 2020-12-05T17:33:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.831+0000 7f7551f78700 1 -- 172.21.15.32:0/1979123969 shutdown_connections 2020-12-05T17:33:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.831+0000 7f7551f78700 1 --2- 172.21.15.32:0/1979123969 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7524055440 0x7f7524057900 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.831+0000 7f7551f78700 1 --2- 172.21.15.32:0/1979123969 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f754c0b2000 0x7f754c0654f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.831+0000 7f7551f78700 1 --2- 172.21.15.32:0/1979123969 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f754c061780 0x7f754c063380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.831+0000 7f7551f78700 1 --2- 172.21.15.32:0/1979123969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f754c0af4b0 0x7f754c065000 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.831+0000 7f7551f78700 1 -- 172.21.15.32:0/1979123969 >> 172.21.15.32:0/1979123969 conn(0x7f754c0b5620 msgr2=0x7f754c0b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:50.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.831+0000 7f7551f78700 1 -- 172.21.15.32:0/1979123969 shutdown_connections 2020-12-05T17:33:50.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.831+0000 7f7551f78700 1 -- 172.21.15.32:0/1979123969 wait complete. 2020-12-05T17:33:50.844 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:33:50.845 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:33:50.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.955+0000 7fdddbfff700 1 Processor -- start 2020-12-05T17:33:50.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.955+0000 7fdddbfff700 1 -- start start 2020-12-05T17:33:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.955+0000 7fdddbfff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fdddc0aec10 0x7fdddc1442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.955+0000 7fdddbfff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdddc0ab4a0 con 0x7fdddc0b1e20 2020-12-05T17:33:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.955+0000 7fdddbfff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdddc0aca70 con 0x7fdddc144790 2020-12-05T17:33:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.955+0000 7fdddbfff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdddc0ad800 con 0x7fdddc0aec10 2020-12-05T17:33:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.959+0000 7fdddaffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdddc0b1e20 0x7fdddc143b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37392/0 (socket says 172.21.15.32:37392) 2020-12-05T17:33:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.959+0000 7fdddaffd700 1 -- 172.21.15.32:0/1653268730 learned_addr learned my addr 172.21.15.32:0/1653268730 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:33:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.959+0000 7fddda7fc700 1 --2- 172.21.15.32:0/1653268730 >> v2:172.21.15.61:3300/0 conn(0x7fdddc0aec10 0x7fdddc1442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.959+0000 7fddd9ffb700 1 -- 172.21.15.32:0/1653268730 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3730242343 0 0) 0x7fdddc0ab4a0 con 0x7fdddc0b1e20 2020-12-05T17:33:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.959+0000 7fddd9ffb700 1 -- 172.21.15.32:0/1653268730 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddbc003230 con 0x7fdddc0b1e20 2020-12-05T17:33:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.959+0000 7fddd9ffb700 1 -- 172.21.15.32:0/1653268730 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2969310144 0 0) 0x7fdddc0aca70 con 0x7fdddc144790 2020-12-05T17:33:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.959+0000 7fddd9ffb700 1 -- 172.21.15.32:0/1653268730 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdddc0ab4a0 con 0x7fdddc144790 2020-12-05T17:33:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.959+0000 7fddd9ffb700 1 -- 172.21.15.32:0/1653268730 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1972955213 0 0) 0x7fddbc003230 con 0x7fdddc0b1e20 2020-12-05T17:33:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.959+0000 7fddda7fc700 1 -- 172.21.15.32:0/1653268730 >> v1:172.21.15.61:6789/0 conn(0x7fdddc144790 legacy=0x7fdddc140960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.959+0000 7fddda7fc700 1 -- 172.21.15.32:0/1653268730 >> v1:172.21.15.32:6789/0 conn(0x7fdddc0b1e20 legacy=0x7fdddc143b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.959+0000 7fddda7fc700 1 -- 172.21.15.32:0/1653268730 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdddc0ab6d0 con 0x7fdddc0aec10 2020-12-05T17:33:50.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.959+0000 7fddda7fc700 1 --2- 172.21.15.32:0/1653268730 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdddc0aec10 0x7fdddc1442a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fddc40010d0 tx=0x7fddcc006dc0).ready entity=mon.1 client_cookie=e486eba9199b5666 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:50.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.959+0000 7fddd9ffb700 1 -- 172.21.15.32:0/1653268730 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddc40016d0 con 0x7fdddc0aec10 2020-12-05T17:33:50.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.959+0000 7fddd9ffb700 1 -- 172.21.15.32:0/1653268730 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddcc00d610 con 0x7fdddc0aec10 2020-12-05T17:33:50.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.959+0000 7fddd9ffb700 1 -- 172.21.15.32:0/1653268730 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddcc00d780 con 0x7fdddc0aec10 2020-12-05T17:33:50.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.959+0000 7fdddbfff700 1 -- 172.21.15.32:0/1653268730 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdddc0aec10 msgr2=0x7fdddc1442a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.959+0000 7fdddbfff700 1 --2- 172.21.15.32:0/1653268730 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdddc0aec10 0x7fdddc1442a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fddc40010d0 tx=0x7fddcc006dc0).stop 2020-12-05T17:33:50.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.959+0000 7fdddbfff700 1 -- 172.21.15.32:0/1653268730 shutdown_connections 2020-12-05T17:33:50.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.959+0000 7fdddbfff700 1 --2- 172.21.15.32:0/1653268730 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdddc0aec10 0x7fdddc1442a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.959+0000 7fdddbfff700 1 -- 172.21.15.32:0/1653268730 >> 172.21.15.32:0/1653268730 conn(0x7fdddc0b5530 msgr2=0x7fdddc0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:50.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.959+0000 7fdddbfff700 1 -- 172.21.15.32:0/1653268730 shutdown_connections 2020-12-05T17:33:50.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.959+0000 7fdddbfff700 1 -- 172.21.15.32:0/1653268730 wait complete. 2020-12-05T17:33:50.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.963+0000 7fdddbfff700 1 Processor -- start 2020-12-05T17:33:50.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.963+0000 7fdddbfff700 1 -- start start 2020-12-05T17:33:50.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.963+0000 7fdddbfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdddc0aec10 0x7fdddc0b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:50.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.963+0000 7fdddbfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdddc0b1e20 0x7fdddc064290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:50.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.963+0000 7fdddbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdddc144790 0x7fdddc064780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:50.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.963+0000 7fdddbfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc065710 con 0x7fdddc144790 2020-12-05T17:33:50.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.967+0000 7fdddbfff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc13d840 con 0x7fdddc0aec10 2020-12-05T17:33:50.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.967+0000 7fdddbfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc063eb0 con 0x7fdddc0b1e20 2020-12-05T17:33:50.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.967+0000 7fdddb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdddc144790 0x7fdddc064780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:50.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.967+0000 7fdddb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdddc144790 0x7fdddc064780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34754/0 (socket says 172.21.15.32:34754) 2020-12-05T17:33:50.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.967+0000 7fdddb7fe700 1 -- 172.21.15.32:0/203935476 learned_addr learned my addr 172.21.15.32:0/203935476 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:33:50.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.967+0000 7fdddaffd700 1 --2- 172.21.15.32:0/203935476 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdddc0aec10 0x7fdddc0b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:50.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.967+0000 7fddda7fc700 1 --2- 172.21.15.32:0/203935476 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdddc0b1e20 0x7fdddc064290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:50.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.967+0000 7fdddb7fe700 1 -- 172.21.15.32:0/203935476 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdddc0b1e20 msgr2=0x7fdddc064290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.967+0000 7fdddb7fe700 1 --2- 172.21.15.32:0/203935476 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdddc0b1e20 0x7fdddc064290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.967+0000 7fdddb7fe700 1 -- 172.21.15.32:0/203935476 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdddc0aec10 msgr2=0x7fdddc0b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:50.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.967+0000 7fdddb7fe700 1 --2- 172.21.15.32:0/203935476 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdddc0aec10 0x7fdddc0b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:50.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.967+0000 7fdddb7fe700 1 -- 172.21.15.32:0/203935476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdddc152a20 con 0x7fdddc144790 2020-12-05T17:33:50.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.967+0000 7fdddb7fe700 1 --2- 172.21.15.32:0/203935476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdddc144790 0x7fdddc064780 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fddd4001490 tx=0x7fddd4007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:50.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.967+0000 7fddd3fff700 1 -- 172.21.15.32:0/203935476 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddd4002240 con 0x7fdddc144790 2020-12-05T17:33:50.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.967+0000 7fddd3fff700 1 -- 172.21.15.32:0/203935476 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd40023a0 con 0x7fdddc144790 2020-12-05T17:33:50.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.967+0000 7fdddbfff700 1 -- 172.21.15.32:0/203935476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdddc143dd0 con 0x7fdddc144790 2020-12-05T17:33:50.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.967+0000 7fddd3fff700 1 -- 172.21.15.32:0/203935476 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddd400a790 con 0x7fdddc144790 2020-12-05T17:33:50.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.967+0000 7fdddbfff700 1 -- 172.21.15.32:0/203935476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdddc064e90 con 0x7fdddc144790 2020-12-05T17:33:50.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.967+0000 7fdddbfff700 1 -- 172.21.15.32:0/203935476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdddc057050 con 0x7fdddc144790 2020-12-05T17:33:50.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.971+0000 7fddd3fff700 1 -- 172.21.15.32:0/203935476 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fddd4019030 con 0x7fdddc144790 2020-12-05T17:33:50.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.971+0000 7fddd3fff700 1 --2- 172.21.15.32:0/203935476 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fddc0055440 0x7fddc0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:33:50.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.971+0000 7fddd3fff700 1 -- 172.21.15.32:0/203935476 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fddd406d960 con 0x7fdddc144790 2020-12-05T17:33:50.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.971+0000 7fdddaffd700 1 --2- 172.21.15.32:0/203935476 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fddc0055440 0x7fddc0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:33:50.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.971+0000 7fdddaffd700 1 --2- 172.21.15.32:0/203935476 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fddc0055440 0x7fddc0057900 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fddc400d000 tx=0x7fddc4000b80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:33:50.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:50.975+0000 7fddd3fff700 1 -- 172.21.15.32:0/203935476 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fddd40439f0 con 0x7fdddc144790 2020-12-05T17:33:51.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:51.159+0000 7fdddbfff700 1 -- 172.21.15.32:0/203935476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdddc0b12e0 con 0x7fdddc144790 2020-12-05T17:33:51.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:51.159+0000 7fddd3fff700 1 -- 172.21.15.32:0/203935476 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2138 (secure 0 0 0) 0x7fddd4047010 con 0x7fdddc144790 2020-12-05T17:33:51.162 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:33:51.162 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4919,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/3405415630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3405415630},{"type":"v1","addr":"172.21.15.61:6833","nonce":3405415630}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:33:45.513696+0000","modified":"2020-12-05T17:33:46.989395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5301":{"gid":5301,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3450615455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3450615455},{"type":"v1","addr":"172.21.15.32:6835","nonce":3450615455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-12-05T17:33:51.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:51.163+0000 7fdddbfff700 1 -- 172.21.15.32:0/203935476 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fddc0055440 msgr2=0x7fddc0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:51.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:51.163+0000 7fdddbfff700 1 --2- 172.21.15.32:0/203935476 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fddc0055440 0x7fddc0057900 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fddc400d000 tx=0x7fddc4000b80).stop 2020-12-05T17:33:51.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:51.163+0000 7fdddbfff700 1 -- 172.21.15.32:0/203935476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdddc144790 msgr2=0x7fdddc064780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:33:51.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:51.163+0000 7fdddbfff700 1 --2- 172.21.15.32:0/203935476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdddc144790 0x7fdddc064780 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fddd4001490 tx=0x7fddd4007e00).stop 2020-12-05T17:33:51.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:51.163+0000 7fdddbfff700 1 -- 172.21.15.32:0/203935476 shutdown_connections 2020-12-05T17:33:51.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:51.163+0000 7fdddbfff700 1 --2- 172.21.15.32:0/203935476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdddc144790 0x7fdddc064780 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:51.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:51.163+0000 7fdddbfff700 1 --2- 172.21.15.32:0/203935476 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdddc0aec10 0x7fdddc0b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:51.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:51.163+0000 7fdddbfff700 1 --2- 172.21.15.32:0/203935476 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fddc0055440 0x7fddc0057900 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:51.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:51.163+0000 7fdddbfff700 1 --2- 172.21.15.32:0/203935476 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdddc0b1e20 0x7fdddc064290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:33:51.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:51.163+0000 7fdddbfff700 1 -- 172.21.15.32:0/203935476 >> 172.21.15.32:0/203935476 conn(0x7fdddc0b5530 msgr2=0x7fdddc0ac0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:33:51.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:51.163+0000 7fdddbfff700 1 -- 172.21.15.32:0/203935476 shutdown_connections 2020-12-05T17:33:51.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:33:51.163+0000 7fdddbfff700 1 -- 172.21.15.32:0/203935476 wait complete. 2020-12-05T17:33:51.166 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 17 2020-12-05T17:33:51.377 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:33:45.513696+0000', 'modified': '2020-12-05T17:33:46.989395+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5301}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5301': {'gid': 5301, 'name': 'a', 'rank': 0, 'incarnation': 15, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/3450615455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 3450615455}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 3450615455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:33:51.378 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:33:51.378 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:33:51.508 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":5301,"want_state":"up:active","state":"up:active","rank_uptime":5.9850872280000003,"mdsmap_epoch":17,"osdmap_epoch":44,"osdmap_epoch_barrier":44,"uptime":11.409187229} 2020-12-05T17:33:51.509 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 5301, "mdsmap_epoch": 17, "osdmap_epoch": 44, "osdmap_epoch_barrier": 44, "rank_uptime": 5.985087228, "state": "up:active", "uptime": 11.409187229, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:33:51.509 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:33:51.510 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-12-05T17:33:51.510 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-12-05T17:33:51.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:33:51.511 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:33:51.518 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:33:51.572 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:33:51.572 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:33:51.626 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:33:51.628 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:33:51.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:33:51.629 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:33:51.676 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:33:51.676 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:33:51.721 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:33:51.737 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:33:51.737 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:33:51.792 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:33:51.794 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:33:51.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:33:51.795 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:33:51.799 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:2020-12-05T17:33:51.795+0000 7f211d80e0c0 -1 init, newargv = 0x555576f453d0 newargc=9ceph-fuse[ 2020-12-05T17:33:51.799 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:10550]: starting ceph client 2020-12-05T17:33:51.809 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[10550]: starting fuse 2020-12-05T17:33:52.905 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:33:52.921 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:33:52.921 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:33:52.978 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:33:52.979 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:33:52.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:33:52.979 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:33:53.025 INFO:teuthology.orchestra.run.smithi036.stdout:49 2020-12-05T17:33:53.025 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-12-05T17:33:53.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:33:53.026 INFO:teuthology.orchestra.run.smithi036:> sudo python3 -c ' 2020-12-05T17:33:53.026 INFO:teuthology.orchestra.run.smithi036:> import glob 2020-12-05T17:33:53.027 INFO:teuthology.orchestra.run.smithi036:> import re 2020-12-05T17:33:53.027 INFO:teuthology.orchestra.run.smithi036:> import os 2020-12-05T17:33:53.028 INFO:teuthology.orchestra.run.smithi036:> import subprocess 2020-12-05T17:33:53.028 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:33:53.028 INFO:teuthology.orchestra.run.smithi036:> def find_socket(client_name): 2020-12-05T17:33:53.029 INFO:teuthology.orchestra.run.smithi036:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-12-05T17:33:53.029 INFO:teuthology.orchestra.run.smithi036:> files = glob.glob(asok_path) 2020-12-05T17:33:53.030 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:33:53.030 INFO:teuthology.orchestra.run.smithi036:> # Given a non-glob path, it better be there 2020-12-05T17:33:53.030 INFO:teuthology.orchestra.run.smithi036:> if "*" not in asok_path: 2020-12-05T17:33:53.031 INFO:teuthology.orchestra.run.smithi036:> assert(len(files) == 1) 2020-12-05T17:33:53.031 INFO:teuthology.orchestra.run.smithi036:> return files[0] 2020-12-05T17:33:53.032 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:33:53.032 INFO:teuthology.orchestra.run.smithi036:> for f in files: 2020-12-05T17:33:53.032 INFO:teuthology.orchestra.run.smithi036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:33:53.033 INFO:teuthology.orchestra.run.smithi036:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:33:53.033 INFO:teuthology.orchestra.run.smithi036:> return f 2020-12-05T17:33:53.034 INFO:teuthology.orchestra.run.smithi036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:33:53.034 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:33:53.034 INFO:teuthology.orchestra.run.smithi036:> print(find_socket("client.0")) 2020-12-05T17:33:53.035 INFO:teuthology.orchestra.run.smithi036:> ' 2020-12-05T17:33:53.118 INFO:teuthology.orchestra.run.smithi036.stdout:/var/run/ceph/ceph-client.0.10550.asok 2020-12-05T17:33:53.123 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.10550.asok 2020-12-05T17:33:53.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:33:53.124 INFO:teuthology.orchestra.run.smithi036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.10550.asok status 2020-12-05T17:33:53.224 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2020-12-05T17:33:53.225 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": { 2020-12-05T17:33:53.225 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:33:53.225 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:33:53.225 INFO:teuthology.orchestra.run.smithi036.stdout: "entity_id": "0", 2020-12-05T17:33:53.226 INFO:teuthology.orchestra.run.smithi036.stdout: "hostname": "smithi036", 2020-12-05T17:33:53.226 INFO:teuthology.orchestra.run.smithi036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-12-05T17:33:53.226 INFO:teuthology.orchestra.run.smithi036.stdout: "pid": "10550", 2020-12-05T17:33:53.226 INFO:teuthology.orchestra.run.smithi036.stdout: "root": "/" 2020-12-05T17:33:53.226 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:33:53.227 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_count": 0, 2020-12-05T17:33:53.227 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_pinned_count": 0, 2020-12-05T17:33:53.227 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 5389, 2020-12-05T17:33:53.227 INFO:teuthology.orchestra.run.smithi036.stdout: "inst": { 2020-12-05T17:33:53.227 INFO:teuthology.orchestra.run.smithi036.stdout: "name": { 2020-12-05T17:33:53.228 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "client", 2020-12-05T17:33:53.228 INFO:teuthology.orchestra.run.smithi036.stdout: "num": 5389 2020-12-05T17:33:53.228 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:33:53.228 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:33:53.228 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:33:53.229 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:33:53.229 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 685290876 2020-12-05T17:33:53.229 INFO:teuthology.orchestra.run.smithi036.stdout: } 2020-12-05T17:33:53.229 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:33:53.229 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:33:53.230 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:33:53.230 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:33:53.230 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 685290876 2020-12-05T17:33:53.230 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:33:53.230 INFO:teuthology.orchestra.run.smithi036.stdout: "inst_str": "client.5389 172.21.15.36:0/685290876", 2020-12-05T17:33:53.231 INFO:teuthology.orchestra.run.smithi036.stdout: "addr_str": "172.21.15.36:0/685290876", 2020-12-05T17:33:53.231 INFO:teuthology.orchestra.run.smithi036.stdout: "inode_count": 1, 2020-12-05T17:33:53.231 INFO:teuthology.orchestra.run.smithi036.stdout: "mds_epoch": 17, 2020-12-05T17:33:53.231 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch": 44, 2020-12-05T17:33:53.231 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:33:53.231 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklisted": false 2020-12-05T17:33:53.232 INFO:teuthology.orchestra.run.smithi036.stdout:} 2020-12-05T17:33:53.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:33:53.235 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:33:53.235 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:33:53.243 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:33:53.243 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:33:53.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:33:53.244 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:33:53.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:33:53.304 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:33:53.305 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:33:53.354 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:33:53.355 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:33:53.355 INFO:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-12-05T17:33:53.356 INFO:teuthology.orchestra.run.smithi036:> import sys 2020-12-05T17:33:53.356 INFO:teuthology.orchestra.run.smithi036:> import time 2020-12-05T17:33:53.356 INFO:teuthology.orchestra.run.smithi036:> import os 2020-12-05T17:33:53.357 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:33:53.357 INFO:teuthology.orchestra.run.smithi036:> n = 1000 2020-12-05T17:33:53.357 INFO:teuthology.orchestra.run.smithi036:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2020-12-05T17:33:53.358 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:33:53.358 INFO:teuthology.orchestra.run.smithi036:> if not os.path.exists(os.path.dirname(abs_path)): 2020-12-05T17:33:53.359 INFO:teuthology.orchestra.run.smithi036:> os.makedirs(os.path.dirname(abs_path)) 2020-12-05T17:33:53.359 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:33:53.359 INFO:teuthology.orchestra.run.smithi036:> for i in range(0, n): 2020-12-05T17:33:53.360 INFO:teuthology.orchestra.run.smithi036:> fname = "{0}_{1}".format(abs_path, i) 2020-12-05T17:33:53.360 INFO:teuthology.orchestra.run.smithi036:> with open(fname, '"'"'w'"'"') as f: 2020-12-05T17:33:53.361 INFO:teuthology.orchestra.run.smithi036:> f.write('"'"'content'"'"') 2020-12-05T17:33:53.361 INFO:teuthology.orchestra.run.smithi036:> if True: 2020-12-05T17:33:53.361 INFO:teuthology.orchestra.run.smithi036:> f.flush() 2020-12-05T17:33:53.362 INFO:teuthology.orchestra.run.smithi036:> os.fsync(f.fileno()) 2020-12-05T17:33:53.362 INFO:teuthology.orchestra.run.smithi036:> ' 2020-12-05T17:34:06.082 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:34:06.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.199+0000 7fca0376c700 1 Processor -- start 2020-12-05T17:34:06.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.199+0000 7fca0376c700 1 -- start start 2020-12-05T17:34:06.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.199+0000 7fca0376c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc9fc0acf60 0x7fc9fc05fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:06.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.199+0000 7fca0376c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9fc0affa0 con 0x7fc9fc0acb80 2020-12-05T17:34:06.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.199+0000 7fca0376c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9fc0ad6a0 con 0x7fc9fc067480 2020-12-05T17:34:06.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.199+0000 7fca0376c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc067a00 con 0x7fc9fc0acf60 2020-12-05T17:34:06.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.199+0000 7fca01508700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc9fc0acb80 0x7fc9fc142800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37408/0 (socket says 172.21.15.32:37408) 2020-12-05T17:34:06.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.199+0000 7fca01508700 1 -- 172.21.15.32:0/2687171003 learned_addr learned my addr 172.21.15.32:0/2687171003 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:34:06.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.199+0000 7fca01d09700 1 --2- 172.21.15.32:0/2687171003 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc9fc0acf60 0x7fc9fc05fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.199+0000 7fc9fbfff700 1 -- 172.21.15.32:0/2687171003 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2413348437 0 0) 0x7fc9fc0affa0 con 0x7fc9fc0acb80 2020-12-05T17:34:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.199+0000 7fc9fbfff700 1 -- 172.21.15.32:0/2687171003 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9dc003230 con 0x7fc9fc0acb80 2020-12-05T17:34:06.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.199+0000 7fca01d09700 1 -- 172.21.15.32:0/2687171003 >> v1:172.21.15.61:6789/0 conn(0x7fc9fc067480 legacy=0x7fc9fc13fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:06.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.199+0000 7fc9fbfff700 1 -- 172.21.15.32:0/2687171003 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3242783164 0 0) 0x7fc9fc0ad6a0 con 0x7fc9fc067480 2020-12-05T17:34:06.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.199+0000 7fca01d09700 1 -- 172.21.15.32:0/2687171003 >> v1:172.21.15.32:6789/0 conn(0x7fc9fc0acb80 legacy=0x7fc9fc142800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:06.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.199+0000 7fca01d09700 1 -- 172.21.15.32:0/2687171003 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9fc0b01d0 con 0x7fc9fc0acf60 2020-12-05T17:34:06.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.199+0000 7fca01d09700 1 --2- 172.21.15.32:0/2687171003 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc9fc0acf60 0x7fc9fc05fdc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fc9ec001870 tx=0x7fc9f40091a0).ready entity=mon.2 client_cookie=df1bae3ea82e108e server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:06.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.199+0000 7fc9fbfff700 1 -- 172.21.15.32:0/2687171003 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9ec0016d0 con 0x7fc9fc0acf60 2020-12-05T17:34:06.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.199+0000 7fc9fbfff700 1 -- 172.21.15.32:0/2687171003 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f400b8b0 con 0x7fc9fc0acf60 2020-12-05T17:34:06.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.199+0000 7fc9fbfff700 1 -- 172.21.15.32:0/2687171003 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9f400ba20 con 0x7fc9fc0acf60 2020-12-05T17:34:06.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.199+0000 7fca0376c700 1 -- 172.21.15.32:0/2687171003 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc9fc0acf60 msgr2=0x7fc9fc05fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:06.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.199+0000 7fca0376c700 1 --2- 172.21.15.32:0/2687171003 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc9fc0acf60 0x7fc9fc05fdc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fc9ec001870 tx=0x7fc9f40091a0).stop 2020-12-05T17:34:06.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.203+0000 7fca0376c700 1 -- 172.21.15.32:0/2687171003 shutdown_connections 2020-12-05T17:34:06.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.203+0000 7fca0376c700 1 --2- 172.21.15.32:0/2687171003 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc9fc0acf60 0x7fc9fc05fdc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:06.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.203+0000 7fca0376c700 1 -- 172.21.15.32:0/2687171003 >> 172.21.15.32:0/2687171003 conn(0x7fc9fc0b5580 msgr2=0x7fc9fc0b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:06.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.203+0000 7fca0376c700 1 -- 172.21.15.32:0/2687171003 shutdown_connections 2020-12-05T17:34:06.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.203+0000 7fca0376c700 1 -- 172.21.15.32:0/2687171003 wait complete. 2020-12-05T17:34:06.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.203+0000 7fca0376c700 1 Processor -- start 2020-12-05T17:34:06.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.203+0000 7fca0376c700 1 -- start start 2020-12-05T17:34:06.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.203+0000 7fca0376c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc9fc067480 0x7fc9fc1114c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:06.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.203+0000 7fca0376c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9fc0acb80 0x7fc9fc140270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:06.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.203+0000 7fca0376c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9fc0acf60 0x7fc9fc151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:06.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.203+0000 7fca0376c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc110950 con 0x7fc9fc0acf60 2020-12-05T17:34:06.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.203+0000 7fca0376c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc10f820 con 0x7fc9fc0acb80 2020-12-05T17:34:06.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.203+0000 7fca0376c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc10fac0 con 0x7fc9fc067480 2020-12-05T17:34:06.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.207+0000 7fca01d09700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9fc0acf60 0x7fc9fc151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:06.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.207+0000 7fca01d09700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9fc0acf60 0x7fc9fc151570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34770/0 (socket says 172.21.15.32:34770) 2020-12-05T17:34:06.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.207+0000 7fca01d09700 1 -- 172.21.15.32:0/8063704 learned_addr learned my addr 172.21.15.32:0/8063704 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:34:06.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.207+0000 7fca01508700 1 --2- 172.21.15.32:0/8063704 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc9fc067480 0x7fc9fc1114c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:06.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.207+0000 7fca00d07700 1 --2- 172.21.15.32:0/8063704 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9fc0acb80 0x7fc9fc140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:06.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.207+0000 7fca01d09700 1 -- 172.21.15.32:0/8063704 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc9fc067480 msgr2=0x7fc9fc1114c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:06.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.207+0000 7fca01d09700 1 --2- 172.21.15.32:0/8063704 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc9fc067480 0x7fc9fc1114c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:06.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.207+0000 7fca01d09700 1 -- 172.21.15.32:0/8063704 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9fc0acb80 msgr2=0x7fc9fc140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:06.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.207+0000 7fca01d09700 1 --2- 172.21.15.32:0/8063704 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9fc0acb80 0x7fc9fc140270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:06.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.207+0000 7fca01d09700 1 -- 172.21.15.32:0/8063704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9fc10f2a0 con 0x7fc9fc0acf60 2020-12-05T17:34:06.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.207+0000 7fca01d09700 1 --2- 172.21.15.32:0/8063704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9fc0acf60 0x7fc9fc151570 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fc9ec001870 tx=0x7fc9fc067ca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:06.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.207+0000 7fc9fa7fc700 1 -- 172.21.15.32:0/8063704 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9f4011b40 con 0x7fc9fc0acf60 2020-12-05T17:34:06.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.207+0000 7fc9fa7fc700 1 -- 172.21.15.32:0/8063704 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f4011ca0 con 0x7fc9fc0acf60 2020-12-05T17:34:06.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.207+0000 7fca0376c700 1 -- 172.21.15.32:0/8063704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9fc10f4f0 con 0x7fc9fc0acf60 2020-12-05T17:34:06.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.207+0000 7fc9fa7fc700 1 -- 172.21.15.32:0/8063704 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9f4011e10 con 0x7fc9fc0acf60 2020-12-05T17:34:06.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.207+0000 7fca0376c700 1 -- 172.21.15.32:0/8063704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9fc113260 con 0x7fc9fc0acf60 2020-12-05T17:34:06.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.207+0000 7fca0376c700 1 -- 172.21.15.32:0/8063704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9fc057050 con 0x7fc9fc0acf60 2020-12-05T17:34:06.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.207+0000 7fc9fa7fc700 1 -- 172.21.15.32:0/8063704 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc9f400f070 con 0x7fc9fc0acf60 2020-12-05T17:34:06.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.211+0000 7fc9fa7fc700 1 --2- 172.21.15.32:0/8063704 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc9e00554e0 0x7fc9e00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:06.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.211+0000 7fc9fa7fc700 1 -- 172.21.15.32:0/8063704 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc9f406f870 con 0x7fc9fc0acf60 2020-12-05T17:34:06.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.211+0000 7fca01508700 1 --2- 172.21.15.32:0/8063704 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc9e00554e0 0x7fc9e00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:06.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.211+0000 7fca01508700 1 --2- 172.21.15.32:0/8063704 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc9e00554e0 0x7fc9e00579a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fc9ec00d000 tx=0x7fc9ec000b80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:06.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.215+0000 7fc9fa7fc700 1 -- 172.21.15.32:0/8063704 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc9f4045a50 con 0x7fc9fc0acf60 2020-12-05T17:34:06.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.403+0000 7fca0376c700 1 -- 172.21.15.32:0/8063704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9fc10fc80 con 0x7fc9fc0acf60 2020-12-05T17:34:06.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.403+0000 7fc9fa7fc700 1 -- 172.21.15.32:0/8063704 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2138 (secure 0 0 0) 0x7fc9f4049030 con 0x7fc9fc0acf60 2020-12-05T17:34:06.403 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:34:06.404 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4919,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/3405415630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3405415630},{"type":"v1","addr":"172.21.15.61:6833","nonce":3405415630}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:33:45.513696+0000","modified":"2020-12-05T17:33:46.989395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5301":{"gid":5301,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3450615455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3450615455},{"type":"v1","addr":"172.21.15.32:6835","nonce":3450615455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-12-05T17:34:06.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.407+0000 7fca0376c700 1 -- 172.21.15.32:0/8063704 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc9e00554e0 msgr2=0x7fc9e00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:06.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.407+0000 7fca0376c700 1 --2- 172.21.15.32:0/8063704 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc9e00554e0 0x7fc9e00579a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fc9ec00d000 tx=0x7fc9ec000b80).stop 2020-12-05T17:34:06.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.407+0000 7fca0376c700 1 -- 172.21.15.32:0/8063704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9fc0acf60 msgr2=0x7fc9fc151570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:06.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.407+0000 7fca0376c700 1 --2- 172.21.15.32:0/8063704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9fc0acf60 0x7fc9fc151570 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fc9ec001870 tx=0x7fc9fc067ca0).stop 2020-12-05T17:34:06.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.407+0000 7fca0376c700 1 -- 172.21.15.32:0/8063704 shutdown_connections 2020-12-05T17:34:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.407+0000 7fca0376c700 1 --2- 172.21.15.32:0/8063704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9fc0acf60 0x7fc9fc151570 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.407+0000 7fca0376c700 1 --2- 172.21.15.32:0/8063704 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc9e00554e0 0x7fc9e00579a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.407+0000 7fca0376c700 1 --2- 172.21.15.32:0/8063704 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc9fc067480 0x7fc9fc1114c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.407+0000 7fca0376c700 1 --2- 172.21.15.32:0/8063704 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9fc0acb80 0x7fc9fc140270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:06.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.407+0000 7fca0376c700 1 -- 172.21.15.32:0/8063704 >> 172.21.15.32:0/8063704 conn(0x7fc9fc0b5580 msgr2=0x7fc9fc0b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:06.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.407+0000 7fca0376c700 1 -- 172.21.15.32:0/8063704 shutdown_connections 2020-12-05T17:34:06.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.407+0000 7fca0376c700 1 -- 172.21.15.32:0/8063704 wait complete. 2020-12-05T17:34:06.409 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 17 2020-12-05T17:34:06.424 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_caps 20 2020-12-05T17:34:06.538 INFO:teuthology.orchestra.run.smithi032.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2020-12-05T17:34:06.539 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2020-12-05T17:34:06.539 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:34:06.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.651+0000 7f0afd310700 1 Processor -- start 2020-12-05T17:34:06.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.655+0000 7f0afd310700 1 -- start start 2020-12-05T17:34:06.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.655+0000 7f0afd310700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0af805f6f0 0x7f0af81428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:06.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.655+0000 7f0afd310700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0af80b0880 0x7f0af8066070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:06.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.655+0000 7f0afd310700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af805fad0 con 0x7f0af80b1980 2020-12-05T17:34:06.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.655+0000 7f0afd310700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af80b2f80 con 0x7f0af805f6f0 2020-12-05T17:34:06.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.655+0000 7f0afd310700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af8145e40 con 0x7f0af80b0880 2020-12-05T17:34:06.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.655+0000 7f0af7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0af805f6f0 0x7f0af81428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:06.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.655+0000 7f0af77fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0af80b1980 0x7f0af8146580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37426/0 (socket says 172.21.15.32:37426) 2020-12-05T17:34:06.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.655+0000 7f0af77fe700 1 -- 172.21.15.32:0/331831117 learned_addr learned my addr 172.21.15.32:0/331831117 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:34:06.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.655+0000 7f0afcb0f700 1 --2- 172.21.15.32:0/331831117 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0af80b0880 0x7f0af8066070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.655+0000 7f0af7fff700 1 -- 172.21.15.32:0/331831117 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0af80b0880 msgr2=0x7f0af8066070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:34:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.655+0000 7f0af7fff700 1 --2- 172.21.15.32:0/331831117 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0af80b0880 0x7f0af8066070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.655+0000 7f0af7fff700 1 -- 172.21.15.32:0/331831117 >> v1:172.21.15.32:6789/0 conn(0x7f0af80b1980 legacy=0x7f0af8146580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.655+0000 7f0af6ffd700 1 -- 172.21.15.32:0/331831117 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3246236437 0 0) 0x7f0af805fad0 con 0x7f0af80b1980 2020-12-05T17:34:06.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.655+0000 7f0af7fff700 1 -- 172.21.15.32:0/331831117 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af81460e0 con 0x7f0af805f6f0 2020-12-05T17:34:06.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.655+0000 7f0af7fff700 1 --2- 172.21.15.32:0/331831117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0af805f6f0 0x7f0af81428f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f0ae8002f30 tx=0x7f0ae8006dc0).ready entity=mon.0 client_cookie=3f47270db38e3841 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:06.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.655+0000 7f0af6ffd700 1 -- 172.21.15.32:0/331831117 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ae8009760 con 0x7f0af805f6f0 2020-12-05T17:34:06.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.655+0000 7f0af6ffd700 1 -- 172.21.15.32:0/331831117 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae80098c0 con 0x7f0af805f6f0 2020-12-05T17:34:06.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.655+0000 7f0af6ffd700 1 -- 172.21.15.32:0/331831117 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ae8009c20 con 0x7f0af805f6f0 2020-12-05T17:34:06.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.655+0000 7f0afd310700 1 -- 172.21.15.32:0/331831117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0af805f6f0 msgr2=0x7f0af81428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:06.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.655+0000 7f0afd310700 1 --2- 172.21.15.32:0/331831117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0af805f6f0 0x7f0af81428f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f0ae8002f30 tx=0x7f0ae8006dc0).stop 2020-12-05T17:34:06.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0afd310700 1 -- 172.21.15.32:0/331831117 shutdown_connections 2020-12-05T17:34:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0afd310700 1 --2- 172.21.15.32:0/331831117 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0af80b0880 0x7f0af8066070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0afd310700 1 --2- 172.21.15.32:0/331831117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0af805f6f0 0x7f0af81428f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0afd310700 1 -- 172.21.15.32:0/331831117 >> 172.21.15.32:0/331831117 conn(0x7f0af80604c0 msgr2=0x7f0af805c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0afd310700 1 -- 172.21.15.32:0/331831117 shutdown_connections 2020-12-05T17:34:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0afd310700 1 -- 172.21.15.32:0/331831117 wait complete. 2020-12-05T17:34:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0afd310700 1 Processor -- start 2020-12-05T17:34:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0afd310700 1 -- start start 2020-12-05T17:34:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0afd310700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0af805f6f0 0x7f0af80637d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0afd310700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0af80b0880 0x7f0af8154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0afd310700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0af80b1980 0x7f0af8116700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0afd310700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af8115550 con 0x7f0af80b0880 2020-12-05T17:34:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0afd310700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80644c0 con 0x7f0af80b1980 2020-12-05T17:34:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0afd310700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af81162a0 con 0x7f0af805f6f0 2020-12-05T17:34:06.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0af77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0af80b0880 0x7f0af8154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:06.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0af77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0af80b0880 0x7f0af8154d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34784/0 (socket says 172.21.15.32:34784) 2020-12-05T17:34:06.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0af77fe700 1 -- 172.21.15.32:0/541710446 learned_addr learned my addr 172.21.15.32:0/541710446 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:34:06.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0afcb0f700 1 --2- 172.21.15.32:0/541710446 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0af80b1980 0x7f0af8116700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:06.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0af7fff700 1 --2- 172.21.15.32:0/541710446 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0af805f6f0 0x7f0af80637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:06.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0af77fe700 1 -- 172.21.15.32:0/541710446 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0af805f6f0 msgr2=0x7f0af80637d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:06.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0af77fe700 1 --2- 172.21.15.32:0/541710446 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0af805f6f0 0x7f0af80637d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:06.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0af77fe700 1 -- 172.21.15.32:0/541710446 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0af80b1980 msgr2=0x7f0af8116700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:06.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0af77fe700 1 --2- 172.21.15.32:0/541710446 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0af80b1980 0x7f0af8116700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:06.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.659+0000 7f0af77fe700 1 -- 172.21.15.32:0/541710446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af8066360 con 0x7f0af80b0880 2020-12-05T17:34:06.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.663+0000 7f0af77fe700 1 --2- 172.21.15.32:0/541710446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0af80b0880 0x7f0af8154d80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f0af8151860 tx=0x7f0aec007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:06.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.663+0000 7f0af57fa700 1 -- 172.21.15.32:0/541710446 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0aec0072e0 con 0x7f0af80b0880 2020-12-05T17:34:06.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.663+0000 7f0af57fa700 1 -- 172.21.15.32:0/541710446 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aec007440 con 0x7f0af80b0880 2020-12-05T17:34:06.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.663+0000 7f0afd310700 1 -- 172.21.15.32:0/541710446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af8066600 con 0x7f0af80b0880 2020-12-05T17:34:06.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.663+0000 7f0af57fa700 1 -- 172.21.15.32:0/541710446 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0aec0057d0 con 0x7f0af80b0880 2020-12-05T17:34:06.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.663+0000 7f0afd310700 1 -- 172.21.15.32:0/541710446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af8066050 con 0x7f0af80b0880 2020-12-05T17:34:06.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.663+0000 7f0afd310700 1 -- 172.21.15.32:0/541710446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af8057050 con 0x7f0af80b0880 2020-12-05T17:34:06.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.663+0000 7f0af57fa700 1 -- 172.21.15.32:0/541710446 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0aec010070 con 0x7f0af80b0880 2020-12-05T17:34:06.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.663+0000 7f0af57fa700 1 --2- 172.21.15.32:0/541710446 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0ad80553b0 0x7f0ad8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:06.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.663+0000 7f0af57fa700 1 -- 172.21.15.32:0/541710446 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0aec06d750 con 0x7f0af80b0880 2020-12-05T17:34:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.663+0000 7f0af7fff700 1 --2- 172.21.15.32:0/541710446 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0ad80553b0 0x7f0ad8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.667+0000 7f0af7fff700 1 --2- 172.21.15.32:0/541710446 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0ad80553b0 0x7f0ad8057870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f0ae801a000 tx=0x7f0ae8015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.671+0000 7f0af57fa700 1 -- 172.21.15.32:0/541710446 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0aec043850 con 0x7f0af80b0880 2020-12-05T17:34:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.859+0000 7f0afd310700 1 -- 172.21.15.32:0/541710446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0af80b1700 con 0x7f0af80b0880 2020-12-05T17:34:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.859+0000 7f0af57fa700 1 -- 172.21.15.32:0/541710446 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2138 (secure 0 0 0) 0x7f0aec046e70 con 0x7f0af80b0880 2020-12-05T17:34:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:34:06.862 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4919,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/3405415630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3405415630},{"type":"v1","addr":"172.21.15.61:6833","nonce":3405415630}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:33:45.513696+0000","modified":"2020-12-05T17:33:46.989395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5301":{"gid":5301,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3450615455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3450615455},{"type":"v1","addr":"172.21.15.32:6835","nonce":3450615455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-12-05T17:34:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.863+0000 7f0afd310700 1 -- 172.21.15.32:0/541710446 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0ad80553b0 msgr2=0x7f0ad8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.863+0000 7f0afd310700 1 --2- 172.21.15.32:0/541710446 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0ad80553b0 0x7f0ad8057870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f0ae801a000 tx=0x7f0ae8015000).stop 2020-12-05T17:34:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.863+0000 7f0afd310700 1 -- 172.21.15.32:0/541710446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0af80b0880 msgr2=0x7f0af8154d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.863+0000 7f0afd310700 1 --2- 172.21.15.32:0/541710446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0af80b0880 0x7f0af8154d80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f0af8151860 tx=0x7f0aec007bc0).stop 2020-12-05T17:34:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.863+0000 7f0afd310700 1 -- 172.21.15.32:0/541710446 shutdown_connections 2020-12-05T17:34:06.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.863+0000 7f0afd310700 1 --2- 172.21.15.32:0/541710446 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0af80b1980 0x7f0af8116700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:06.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.863+0000 7f0afd310700 1 --2- 172.21.15.32:0/541710446 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0ad80553b0 0x7f0ad8057870 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:06.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.863+0000 7f0afd310700 1 --2- 172.21.15.32:0/541710446 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0af805f6f0 0x7f0af80637d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:06.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.863+0000 7f0afd310700 1 --2- 172.21.15.32:0/541710446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0af80b0880 0x7f0af8154d80 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:06.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.863+0000 7f0afd310700 1 -- 172.21.15.32:0/541710446 >> 172.21.15.32:0/541710446 conn(0x7f0af80604c0 msgr2=0x7f0af81410a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:06.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.863+0000 7f0afd310700 1 -- 172.21.15.32:0/541710446 shutdown_connections 2020-12-05T17:34:06.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:06.863+0000 7f0afd310700 1 -- 172.21.15.32:0/541710446 wait complete. 2020-12-05T17:34:06.867 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 17 2020-12-05T17:34:06.881 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_decay_threshold 400 2020-12-05T17:34:06.998 INFO:teuthology.orchestra.run.smithi032.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2020-12-05T17:34:06.998 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2020-12-05T17:34:06.999 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@smithi036.front.sepia.ceph.com... 2020-12-05T17:34:07.016 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:34:07.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a49a3d700 1 Processor -- start 2020-12-05T17:34:07.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a49a3d700 1 -- start start 2020-12-05T17:34:07.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a49a3d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f8a440b7720 0x7f8a44061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a49a3d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a441403f0 0x7f8a44143e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a49a3d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a44143a00 con 0x7f8a44144320 2020-12-05T17:34:07.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a49a3d700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a440b7b00 con 0x7f8a440b7720 2020-12-05T17:34:07.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a49a3d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4405fe60 con 0x7f8a441403f0 2020-12-05T17:34:07.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a4923c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8a44144320 0x7f8a4413d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37436/0 (socket says 172.21.15.32:37436) 2020-12-05T17:34:07.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a4923c700 1 -- 172.21.15.32:0/2360939344 learned_addr learned my addr 172.21.15.32:0/2360939344 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:34:07.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a48a3b700 1 --2- 172.21.15.32:0/2360939344 >> v2:172.21.15.61:3300/0 conn(0x7f8a440b7720 0x7f8a44061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a43fff700 1 --2- 172.21.15.32:0/2360939344 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a441403f0 0x7f8a44143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a437fe700 1 -- 172.21.15.32:0/2360939344 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1942380255 0 0) 0x7f8a44143a00 con 0x7f8a44144320 2020-12-05T17:34:07.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a437fe700 1 -- 172.21.15.32:0/2360939344 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a28003230 con 0x7f8a44144320 2020-12-05T17:34:07.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a437fe700 1 -- 172.21.15.32:0/2360939344 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2649467699 0 0) 0x7f8a28003230 con 0x7f8a44144320 2020-12-05T17:34:07.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a437fe700 1 -- 172.21.15.32:0/2360939344 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a441403f0 msgr2=0x7f8a44143e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:34:07.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a437fe700 1 --2- 172.21.15.32:0/2360939344 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a441403f0 0x7f8a44143e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a437fe700 1 -- 172.21.15.32:0/2360939344 >> v2:172.21.15.61:3300/0 conn(0x7f8a440b7720 msgr2=0x7f8a44061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:34:07.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a437fe700 1 --2- 172.21.15.32:0/2360939344 >> v2:172.21.15.61:3300/0 conn(0x7f8a440b7720 0x7f8a44061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a437fe700 1 -- 172.21.15.32:0/2360939344 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a44140850 con 0x7f8a44144320 2020-12-05T17:34:07.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a437fe700 1 -- 172.21.15.32:0/2360939344 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f8a3c001a50 con 0x7f8a44144320 2020-12-05T17:34:07.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a437fe700 1 -- 172.21.15.32:0/2360939344 >> v1:172.21.15.32:6789/0 conn(0x7f8a44144320 legacy=0x7f8a4413d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:07.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a437fe700 1 --2- 172.21.15.32:0/2360939344 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a28003720 0x7f8a28003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a437fe700 1 --2- 172.21.15.32:0/2360939344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a280044d0 0x7f8a28004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a437fe700 1 --2- 172.21.15.32:0/2360939344 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a441403f0 0x7f8a28004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a437fe700 1 -- 172.21.15.32:0/2360939344 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a440b7b00 con 0x7f8a280044d0 2020-12-05T17:34:07.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a437fe700 1 -- 172.21.15.32:0/2360939344 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4405fe60 con 0x7f8a28003720 2020-12-05T17:34:07.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a437fe700 1 -- 172.21.15.32:0/2360939344 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a28004260 con 0x7f8a441403f0 2020-12-05T17:34:07.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a43fff700 1 --2- 172.21.15.32:0/2360939344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a280044d0 0x7f8a28004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a48a3b700 1 --2- 172.21.15.32:0/2360939344 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a28003720 0x7f8a28003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a4923c700 1 --2- 172.21.15.32:0/2360939344 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a441403f0 0x7f8a28004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a43fff700 1 -- 172.21.15.32:0/2360939344 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a441403f0 msgr2=0x7f8a28004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:34:07.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a43fff700 1 --2- 172.21.15.32:0/2360939344 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a441403f0 0x7f8a28004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a43fff700 1 -- 172.21.15.32:0/2360939344 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a28003720 msgr2=0x7f8a28003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:34:07.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a43fff700 1 --2- 172.21.15.32:0/2360939344 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a28003720 0x7f8a28003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a43fff700 1 -- 172.21.15.32:0/2360939344 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a28003230 con 0x7f8a280044d0 2020-12-05T17:34:07.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.131+0000 7f8a43fff700 1 --2- 172.21.15.32:0/2360939344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a280044d0 0x7f8a28004950 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f8a3000c000 tx=0x7f8a30008040).ready entity=mon.0 client_cookie=84b0b048454a2076 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:07.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.135+0000 7f8a437fe700 1 -- 172.21.15.32:0/2360939344 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a3000e9d0 con 0x7f8a280044d0 2020-12-05T17:34:07.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.135+0000 7f8a437fe700 1 -- 172.21.15.32:0/2360939344 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3000eb30 con 0x7f8a280044d0 2020-12-05T17:34:07.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.135+0000 7f8a437fe700 1 -- 172.21.15.32:0/2360939344 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a3000ee90 con 0x7f8a280044d0 2020-12-05T17:34:07.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.135+0000 7f8a49a3d700 1 -- 172.21.15.32:0/2360939344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a280044d0 msgr2=0x7f8a28004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:07.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.135+0000 7f8a49a3d700 1 --2- 172.21.15.32:0/2360939344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a280044d0 0x7f8a28004950 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f8a3000c000 tx=0x7f8a30008040).stop 2020-12-05T17:34:07.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.135+0000 7f8a4923c700 1 -- 172.21.15.32:0/2360939344 reap_dead start 2020-12-05T17:34:07.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.135+0000 7f8a49a3d700 1 -- 172.21.15.32:0/2360939344 shutdown_connections 2020-12-05T17:34:07.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.135+0000 7f8a49a3d700 1 -- 172.21.15.32:0/2360939344 >> 172.21.15.32:0/2360939344 conn(0x7f8a4405bc20 msgr2=0x7f8a440b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:07.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.135+0000 7f8a49a3d700 1 -- 172.21.15.32:0/2360939344 shutdown_connections 2020-12-05T17:34:07.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.135+0000 7f8a49a3d700 1 -- 172.21.15.32:0/2360939344 wait complete. 2020-12-05T17:34:07.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.135+0000 7f8a49a3d700 1 Processor -- start 2020-12-05T17:34:07.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.135+0000 7f8a49a3d700 1 -- start start 2020-12-05T17:34:07.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.139+0000 7f8a49a3d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a440b29e0 0x7f8a440b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.139+0000 7f8a49a3d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a440b2dc0 0x7f8a4413b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.139+0000 7f8a49a3d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a4413bd40 0x7f8a44150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.139+0000 7f8a49a3d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a44110d60 con 0x7f8a4413bd40 2020-12-05T17:34:07.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.139+0000 7f8a49a3d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a44063da0 con 0x7f8a440b2dc0 2020-12-05T17:34:07.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.139+0000 7f8a49a3d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a440b3240 con 0x7f8a440b29e0 2020-12-05T17:34:07.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.139+0000 7f8a4923c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a4413bd40 0x7f8a44150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.139+0000 7f8a48a3b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a440b29e0 0x7f8a440b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.139+0000 7f8a4923c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a4413bd40 0x7f8a44150fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34804/0 (socket says 172.21.15.32:34804) 2020-12-05T17:34:07.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.139+0000 7f8a4923c700 1 -- 172.21.15.32:0/3090732395 learned_addr learned my addr 172.21.15.32:0/3090732395 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:34:07.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.139+0000 7f8a43fff700 1 --2- 172.21.15.32:0/3090732395 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a440b2dc0 0x7f8a4413b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.139+0000 7f8a4923c700 1 -- 172.21.15.32:0/3090732395 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a440b29e0 msgr2=0x7f8a440b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:07.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.139+0000 7f8a4923c700 1 --2- 172.21.15.32:0/3090732395 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a440b29e0 0x7f8a440b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.139+0000 7f8a4923c700 1 -- 172.21.15.32:0/3090732395 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a440b2dc0 msgr2=0x7f8a4413b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:07.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.139+0000 7f8a4923c700 1 --2- 172.21.15.32:0/3090732395 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a440b2dc0 0x7f8a4413b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.139+0000 7f8a4923c700 1 -- 172.21.15.32:0/3090732395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a44111000 con 0x7f8a4413bd40 2020-12-05T17:34:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.139+0000 7f8a4923c700 1 --2- 172.21.15.32:0/3090732395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a4413bd40 0x7f8a44150fb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f8a3c0011d0 tx=0x7f8a3c00ee60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.139+0000 7f8a41ffb700 1 -- 172.21.15.32:0/3090732395 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a3c00c9a0 con 0x7f8a4413bd40 2020-12-05T17:34:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.139+0000 7f8a41ffb700 1 -- 172.21.15.32:0/3090732395 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3c00cb00 con 0x7f8a4413bd40 2020-12-05T17:34:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.139+0000 7f8a49a3d700 1 -- 172.21.15.32:0/3090732395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a44064240 con 0x7f8a4413bd40 2020-12-05T17:34:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.139+0000 7f8a41ffb700 1 -- 172.21.15.32:0/3090732395 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a3c00ce60 con 0x7f8a4413bd40 2020-12-05T17:34:07.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.139+0000 7f8a49a3d700 1 -- 172.21.15.32:0/3090732395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a441514a0 con 0x7f8a4413bd40 2020-12-05T17:34:07.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.143+0000 7f8a49a3d700 1 -- 172.21.15.32:0/3090732395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a44110bc0 con 0x7f8a4413bd40 2020-12-05T17:34:07.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.143+0000 7f8a41ffb700 1 -- 172.21.15.32:0/3090732395 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8a3c0133e0 con 0x7f8a4413bd40 2020-12-05T17:34:07.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.143+0000 7f8a41ffb700 1 --2- 172.21.15.32:0/3090732395 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8a2c055440 0x7f8a2c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.143+0000 7f8a41ffb700 1 -- 172.21.15.32:0/3090732395 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8a3c06df50 con 0x7f8a4413bd40 2020-12-05T17:34:07.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.143+0000 7f8a48a3b700 1 --2- 172.21.15.32:0/3090732395 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8a2c055440 0x7f8a2c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.143+0000 7f8a48a3b700 1 --2- 172.21.15.32:0/3090732395 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8a2c055440 0x7f8a2c057900 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f8a38008000 tx=0x7f8a38008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:07.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.151+0000 7f8a41ffb700 1 -- 172.21.15.32:0/3090732395 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8a44110bc0 con 0x7f8a4413bd40 2020-12-05T17:34:07.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.331+0000 7f8a49a3d700 1 -- 172.21.15.32:0/3090732395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a44110bc0 con 0x7f8a4413bd40 2020-12-05T17:34:07.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.331+0000 7f8a41ffb700 1 -- 172.21.15.32:0/3090732395 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2138 (secure 0 0 0) 0x7f8a44110bc0 con 0x7f8a4413bd40 2020-12-05T17:34:07.334 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:34:07.334 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4919,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/3405415630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3405415630},{"type":"v1","addr":"172.21.15.61:6833","nonce":3405415630}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:33:45.513696+0000","modified":"2020-12-05T17:33:46.989395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5301":{"gid":5301,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3450615455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3450615455},{"type":"v1","addr":"172.21.15.32:6835","nonce":3450615455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-12-05T17:34:07.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.335+0000 7f8a49a3d700 1 -- 172.21.15.32:0/3090732395 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8a2c055440 msgr2=0x7f8a2c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:07.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.335+0000 7f8a49a3d700 1 --2- 172.21.15.32:0/3090732395 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8a2c055440 0x7f8a2c057900 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f8a38008000 tx=0x7f8a38008040).stop 2020-12-05T17:34:07.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.335+0000 7f8a49a3d700 1 -- 172.21.15.32:0/3090732395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a4413bd40 msgr2=0x7f8a44150fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:07.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.335+0000 7f8a49a3d700 1 --2- 172.21.15.32:0/3090732395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a4413bd40 0x7f8a44150fb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f8a3c0011d0 tx=0x7f8a3c00ee60).stop 2020-12-05T17:34:07.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.335+0000 7f8a49a3d700 1 -- 172.21.15.32:0/3090732395 shutdown_connections 2020-12-05T17:34:07.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.335+0000 7f8a49a3d700 1 --2- 172.21.15.32:0/3090732395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a4413bd40 0x7f8a44150fb0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.335+0000 7f8a49a3d700 1 --2- 172.21.15.32:0/3090732395 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8a2c055440 0x7f8a2c057900 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.335+0000 7f8a49a3d700 1 --2- 172.21.15.32:0/3090732395 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a440b29e0 0x7f8a440b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.335+0000 7f8a49a3d700 1 --2- 172.21.15.32:0/3090732395 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a440b2dc0 0x7f8a4413b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.335+0000 7f8a49a3d700 1 -- 172.21.15.32:0/3090732395 >> 172.21.15.32:0/3090732395 conn(0x7f8a4405bc20 msgr2=0x7f8a44142ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:07.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.335+0000 7f8a49a3d700 1 -- 172.21.15.32:0/3090732395 shutdown_connections 2020-12-05T17:34:07.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.335+0000 7f8a49a3d700 1 -- 172.21.15.32:0/3090732395 wait complete. 2020-12-05T17:34:07.339 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 17 2020-12-05T17:34:07.354 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell mds.a cache drop 2020-12-05T17:34:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.467+0000 7fd545c5d700 1 Processor -- start 2020-12-05T17:34:07.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.467+0000 7fd545c5d700 1 -- start start 2020-12-05T17:34:07.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.467+0000 7fd545c5d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5400b5b20 0x7fd5400b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.467+0000 7fd545c5d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd5400b5240 0x7fd540145f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.467+0000 7fd545c5d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5400b1920 con 0x7fd5400b6690 2020-12-05T17:34:07.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.467+0000 7fd545c5d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5400b1bc0 con 0x7fd5400b5b20 2020-12-05T17:34:07.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.467+0000 7fd545c5d700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5400b5680 con 0x7fd5400b5240 2020-12-05T17:34:07.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.467+0000 7fd53f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5400b5b20 0x7fd5400b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.467+0000 7fd53effd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd5400b6690 0x7fd5400b2060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37456/0 (socket says 172.21.15.32:37456) 2020-12-05T17:34:07.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.467+0000 7fd53effd700 1 -- 172.21.15.32:0/1057329581 learned_addr learned my addr 172.21.15.32:0/1057329581 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:34:07.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.467+0000 7fd53ffff700 1 --2- 172.21.15.32:0/1057329581 >> v2:172.21.15.61:3300/0 conn(0x7fd5400b5240 0x7fd540145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.471+0000 7fd53e7fc700 1 -- 172.21.15.32:0/1057329581 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2927032650 0 0) 0x7fd5400b1920 con 0x7fd5400b6690 2020-12-05T17:34:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.471+0000 7fd53f7fe700 1 -- 172.21.15.32:0/1057329581 >> v2:172.21.15.61:3300/0 conn(0x7fd5400b5240 msgr2=0x7fd540145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:34:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.471+0000 7fd53f7fe700 1 --2- 172.21.15.32:0/1057329581 >> v2:172.21.15.61:3300/0 conn(0x7fd5400b5240 0x7fd540145f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.471+0000 7fd53f7fe700 1 -- 172.21.15.32:0/1057329581 >> v1:172.21.15.32:6789/0 conn(0x7fd5400b6690 legacy=0x7fd5400b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.471+0000 7fd53f7fe700 1 -- 172.21.15.32:0/1057329581 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd540146400 con 0x7fd5400b5b20 2020-12-05T17:34:07.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.471+0000 7fd53ffff700 1 --2- 172.21.15.32:0/1057329581 >> v2:172.21.15.61:3300/0 conn(0x7fd5400b5240 0x7fd540145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:34:07.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.471+0000 7fd53f7fe700 1 --2- 172.21.15.32:0/1057329581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5400b5b20 0x7fd5400b5f00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7fd534002f30 tx=0x7fd534006dc0).ready entity=mon.0 client_cookie=939a2a070e141df8 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:07.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.471+0000 7fd53e7fc700 1 -- 172.21.15.32:0/1057329581 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5340097b0 con 0x7fd5400b5b20 2020-12-05T17:34:07.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.471+0000 7fd53e7fc700 1 -- 172.21.15.32:0/1057329581 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd534009910 con 0x7fd5400b5b20 2020-12-05T17:34:07.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.471+0000 7fd53e7fc700 1 -- 172.21.15.32:0/1057329581 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd534009c70 con 0x7fd5400b5b20 2020-12-05T17:34:07.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.471+0000 7fd545c5d700 1 -- 172.21.15.32:0/1057329581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5400b5b20 msgr2=0x7fd5400b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:07.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.471+0000 7fd545c5d700 1 --2- 172.21.15.32:0/1057329581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5400b5b20 0x7fd5400b5f00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7fd534002f30 tx=0x7fd534006dc0).stop 2020-12-05T17:34:07.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.471+0000 7fd545c5d700 1 -- 172.21.15.32:0/1057329581 shutdown_connections 2020-12-05T17:34:07.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.471+0000 7fd545c5d700 1 --2- 172.21.15.32:0/1057329581 >> v2:172.21.15.61:3300/0 conn(0x7fd5400b5240 0x7fd540145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.471+0000 7fd545c5d700 1 --2- 172.21.15.32:0/1057329581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5400b5b20 0x7fd5400b5f00 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.471+0000 7fd545c5d700 1 -- 172.21.15.32:0/1057329581 >> 172.21.15.32:0/1057329581 conn(0x7fd54005f3b0 msgr2=0x7fd54005f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:07.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.471+0000 7fd545c5d700 1 -- 172.21.15.32:0/1057329581 shutdown_connections 2020-12-05T17:34:07.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.471+0000 7fd545c5d700 1 -- 172.21.15.32:0/1057329581 wait complete. 2020-12-05T17:34:07.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.471+0000 7fd545c5d700 1 Processor -- start 2020-12-05T17:34:07.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.471+0000 7fd545c5d700 1 -- start start 2020-12-05T17:34:07.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.475+0000 7fd545c5d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5400b5240 0x7fd5400a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.475+0000 7fd545c5d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd5400b5b20 0x7fd5400a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.475+0000 7fd545c5d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd5400b6690 0x7fd5401518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.475+0000 7fd545c5d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5400a46f0 con 0x7fd5400b5240 2020-12-05T17:34:07.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.475+0000 7fd545c5d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd540066d00 con 0x7fd5400b5b20 2020-12-05T17:34:07.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.475+0000 7fd545c5d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5400a3890 con 0x7fd5400b6690 2020-12-05T17:34:07.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.475+0000 7fd53f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5400b5240 0x7fd5400a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.475+0000 7fd53f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5400b5240 0x7fd5400a5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34814/0 (socket says 172.21.15.32:34814) 2020-12-05T17:34:07.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.475+0000 7fd53f7fe700 1 -- 172.21.15.32:0/920427025 learned_addr learned my addr 172.21.15.32:0/920427025 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:34:07.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.475+0000 7fd53effd700 1 --2- 172.21.15.32:0/920427025 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd5400b5b20 0x7fd5400a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.475+0000 7fd53ffff700 1 --2- 172.21.15.32:0/920427025 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd5400b6690 0x7fd5401518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.475+0000 7fd53f7fe700 1 -- 172.21.15.32:0/920427025 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd5400b6690 msgr2=0x7fd5401518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:07.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.475+0000 7fd53f7fe700 1 --2- 172.21.15.32:0/920427025 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd5400b6690 0x7fd5401518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.475+0000 7fd53f7fe700 1 -- 172.21.15.32:0/920427025 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd5400b5b20 msgr2=0x7fd5400a5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:07.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.475+0000 7fd53f7fe700 1 --2- 172.21.15.32:0/920427025 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd5400b5b20 0x7fd5400a5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.475+0000 7fd53f7fe700 1 -- 172.21.15.32:0/920427025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5400a3d30 con 0x7fd5400b5240 2020-12-05T17:34:07.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.475+0000 7fd53f7fe700 1 --2- 172.21.15.32:0/920427025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5400b5240 0x7fd5400a5780 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7fd534002f30 tx=0x7fd534015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:07.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.475+0000 7fd53cff9700 1 -- 172.21.15.32:0/920427025 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd534011900 con 0x7fd5400b5240 2020-12-05T17:34:07.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.475+0000 7fd53cff9700 1 -- 172.21.15.32:0/920427025 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd534011a60 con 0x7fd5400b5240 2020-12-05T17:34:07.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.475+0000 7fd545c5d700 1 -- 172.21.15.32:0/920427025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5400659a0 con 0x7fd5400b5240 2020-12-05T17:34:07.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.475+0000 7fd53cff9700 1 -- 172.21.15.32:0/920427025 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd534011dc0 con 0x7fd5400b5240 2020-12-05T17:34:07.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.475+0000 7fd545c5d700 1 -- 172.21.15.32:0/920427025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd540066a50 con 0x7fd5400b5240 2020-12-05T17:34:07.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.479+0000 7fd53cff9700 1 -- 172.21.15.32:0/920427025 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd534024030 con 0x7fd5400b5240 2020-12-05T17:34:07.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.479+0000 7fd53cff9700 1 --2- 172.21.15.32:0/920427025 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd520055330 0x7fd5200577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.479+0000 7fd53cff9700 1 -- 172.21.15.32:0/920427025 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd53406eda0 con 0x7fd5400b5240 2020-12-05T17:34:07.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.479+0000 7fd53effd700 1 --2- 172.21.15.32:0/920427025 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd520055330 0x7fd5200577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.479+0000 7fd53effd700 1 --2- 172.21.15.32:0/920427025 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd520055330 0x7fd5200577f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fd53000d000 tx=0x7fd530009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:07.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd5545f8740 1 Processor -- start 2020-12-05T17:34:07.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd5545f8740 1 -- start start 2020-12-05T17:34:07.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd5545f8740 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x2895060 0x28e24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd5545f8740 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x277de90 0x29190a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd5545f8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x26fd800 0x28c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd5545f8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x2809ff0 con 0x26fd800 2020-12-05T17:34:07.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd5545f8740 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x2716690 con 0x277de90 2020-12-05T17:34:07.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd5545f8740 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x26c4410 con 0x2895060 2020-12-05T17:34:07.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd53ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x26fd800 0x28c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd53ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x26fd800 0x28c3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34826/0 (socket says 172.21.15.32:34826) 2020-12-05T17:34:07.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd53ffff700 1 -- 172.21.15.32:0/3380317481 learned_addr learned my addr 172.21.15.32:0/3380317481 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:34:07.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd53effd700 1 --2- 172.21.15.32:0/3380317481 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x277de90 0x29190a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd53f7fe700 1 --2- 172.21.15.32:0/3380317481 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x2895060 0x28e24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd53ffff700 1 -- 172.21.15.32:0/3380317481 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x2895060 msgr2=0x28e24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:34:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd53ffff700 1 --2- 172.21.15.32:0/3380317481 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x2895060 0x28e24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd53ffff700 1 -- 172.21.15.32:0/3380317481 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x277de90 msgr2=0x29190a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:34:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd53ffff700 1 --2- 172.21.15.32:0/3380317481 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x277de90 0x29190a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd53ffff700 1 -- 172.21.15.32:0/3380317481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x282af10 con 0x26fd800 2020-12-05T17:34:07.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd53effd700 1 --2- 172.21.15.32:0/3380317481 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x277de90 0x29190a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:34:07.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd53ffff700 1 --2- 172.21.15.32:0/3380317481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x26fd800 0x28c3ca0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x28e3340 tx=0x7fd5380091a0).ready entity=mon.0 client_cookie=adecc3c96ea37457 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:07.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd545c5d700 1 -- 172.21.15.32:0/3380317481 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd538009a80 con 0x26fd800 2020-12-05T17:34:07.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd545c5d700 1 -- 172.21.15.32:0/3380317481 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd538014650 con 0x26fd800 2020-12-05T17:34:07.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd545c5d700 1 -- 172.21.15.32:0/3380317481 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5380149b0 con 0x26fd800 2020-12-05T17:34:07.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd5545f8740 1 -- 172.21.15.32:0/3380317481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x26fd800 msgr2=0x28c3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:07.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd5545f8740 1 --2- 172.21.15.32:0/3380317481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x26fd800 0x28c3ca0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x28e3340 tx=0x7fd5380091a0).stop 2020-12-05T17:34:07.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd5545f8740 1 -- 172.21.15.32:0/3380317481 shutdown_connections 2020-12-05T17:34:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd5545f8740 1 --2- 172.21.15.32:0/3380317481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x26fd800 0x28c3ca0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd5545f8740 1 --2- 172.21.15.32:0/3380317481 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x2895060 0x28e24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd5545f8740 1 --2- 172.21.15.32:0/3380317481 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x277de90 0x29190a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd5545f8740 1 -- 172.21.15.32:0/3380317481 >> 172.21.15.32:0/3380317481 conn(0x28c49f0 msgr2=0x28e1810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd5545f8740 1 -- 172.21.15.32:0/3380317481 shutdown_connections 2020-12-05T17:34:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd5545f8740 1 -- 172.21.15.32:0/3380317481 wait complete. 2020-12-05T17:34:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd5545f8740 1 -- start start 2020-12-05T17:34:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.483+0000 7fd5545f8740 1 Processor -- start 2020-12-05T17:34:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd5545f8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x26fd800 0x28ea780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd5545f8740 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x277de90 0x28e7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd5545f8740 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x2895060 0x28e0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd5545f8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x2839c70 con 0x26fd800 2020-12-05T17:34:07.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd5545f8740 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x283b240 con 0x277de90 2020-12-05T17:34:07.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd5545f8740 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x281c710 con 0x2895060 2020-12-05T17:34:07.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd5545f8740 10 client.0 ll_register_callbacks cb 0x28c4950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-12-05T17:34:07.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd53f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x26fd800 0x28ea780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd53f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x26fd800 0x28ea780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34828/0 (socket says 172.21.15.32:34828) 2020-12-05T17:34:07.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd53f7fe700 1 -- 172.21.15.32:0/2808190352 learned_addr learned my addr 172.21.15.32:0/2808190352 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:34:07.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd53ffff700 1 --2- 172.21.15.32:0/2808190352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x2895060 0x28e0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd53effd700 1 --2- 172.21.15.32:0/2808190352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x277de90 0x28e7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd53f7fe700 1 -- 172.21.15.32:0/2808190352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x2895060 msgr2=0x28e0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:34:07.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd53f7fe700 1 --2- 172.21.15.32:0/2808190352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x2895060 0x28e0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd53f7fe700 1 -- 172.21.15.32:0/2808190352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x277de90 msgr2=0x28e7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:34:07.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd53f7fe700 1 --2- 172.21.15.32:0/2808190352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x277de90 0x28e7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd53f7fe700 1 -- 172.21.15.32:0/2808190352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2833440 con 0x26fd800 2020-12-05T17:34:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd53f7fe700 1 --2- 172.21.15.32:0/2808190352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x26fd800 0x28ea780 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fd53402b000 tx=0x7fd534011000).ready entity=mon.0 client_cookie=2cb2c154649c196 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd51a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2020-12-05T17:34:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd51a7fc700 1 -- 172.21.15.32:0/2808190352 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd534047c60 con 0x26fd800 2020-12-05T17:34:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd5545f8740 1 -- 172.21.15.32:0/2808190352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x27228b0 con 0x26fd800 2020-12-05T17:34:07.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd51a7fc700 1 -- 172.21.15.32:0/2808190352 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd534047dc0 con 0x26fd800 2020-12-05T17:34:07.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd51a7fc700 1 -- 172.21.15.32:0/2808190352 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd534047150 con 0x26fd800 2020-12-05T17:34:07.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd5545f8740 10 client.5502 fetch_fsmap learned FSMap version 17 2020-12-05T17:34:07.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd5545f8740 1 -- 172.21.15.32:0/2808190352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x26ad770 con 0x26fd800 2020-12-05T17:34:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd51a7fc700 1 -- 172.21.15.32:0/2808190352 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x27228b0 con 0x26fd800 2020-12-05T17:34:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd51a7fc700 1 -- 172.21.15.32:0/2808190352 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 17) v1 ==== 1247+0+0 (secure 0 0 0) 0x7fd534025070 con 0x26fd800 2020-12-05T17:34:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd5545f8740 10 client.5502 fetch_fsmap finished waiting for FSMap version 17 2020-12-05T17:34:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd5545f8740 10 client.5502 resolve_mds: resolved ID 'a' to GID 5301 2020-12-05T17:34:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd5545f8740 20 client.5502 populate_metadata read hostname 'smithi032' 2020-12-05T17:34:07.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd5545f8740 1 --2- 172.21.15.32:0/2808190352 >> [v2:172.21.15.32:6834/3450615455,v1:172.21.15.32:6835/3450615455] conn(0x268fc00 0x2925c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd5545f8740 4 client.5502 mds_command: new command op to 5301 tid=0[{"prefix": "get_command_descriptions"}] 2020-12-05T17:34:07.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd5545f8740 1 -- 172.21.15.32:0/2808190352 --> [v2:172.21.15.32:6834/3450615455,v1:172.21.15.32:6835/3450615455] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x268f150 con 0x268fc00 2020-12-05T17:34:07.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd53effd700 1 --2- 172.21.15.32:0/2808190352 >> [v2:172.21.15.32:6834/3450615455,v1:172.21.15.32:6835/3450615455] conn(0x268fc00 0x2925c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd51a7fc700 10 client.5502 ms_handle_connect on v2:172.21.15.32:6834/3450615455 2020-12-05T17:34:07.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.487+0000 7fd53effd700 1 --2- 172.21.15.32:0/2808190352 >> [v2:172.21.15.32:6834/3450615455,v1:172.21.15.32:6835/3450615455] conn(0x268fc00 0x2925c70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=dc22c074ea8af31d server_cookie=5611a11c668b869f in_seq=0 out_seq=0 2020-12-05T17:34:07.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.491+0000 7fd51a7fc700 1 -- 172.21.15.32:0/2808190352 <== mds.0 v2:172.21.15.32:6834/3450615455 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16606 (crc 0 0 0) 0x268f150 con 0x268fc00 2020-12-05T17:34:07.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.491+0000 7fd51a7fc700 10 client.5502 handle_command_reply: tid=0 2020-12-05T17:34:07.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.491+0000 7fd5545f8740 1 client.5502 shutdown 2020-12-05T17:34:07.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.491+0000 7fd5545f8740 1 -- 172.21.15.32:0/2808190352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x26fd800 msgr2=0x28ea780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:07.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.491+0000 7fd5545f8740 1 --2- 172.21.15.32:0/2808190352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x26fd800 0x28ea780 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fd53402b000 tx=0x7fd534011000).stop 2020-12-05T17:34:07.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.491+0000 7fd5545f8740 1 -- 172.21.15.32:0/2808190352 shutdown_connections 2020-12-05T17:34:07.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.491+0000 7fd5545f8740 1 --2- 172.21.15.32:0/2808190352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x2895060 0x28e0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.491+0000 7fd5545f8740 1 --2- 172.21.15.32:0/2808190352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x26fd800 0x28ea780 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.491+0000 7fd5545f8740 1 --2- 172.21.15.32:0/2808190352 >> [v2:172.21.15.32:6834/3450615455,v1:172.21.15.32:6835/3450615455] conn(0x268fc00 0x2925c70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.491+0000 7fd5545f8740 1 --2- 172.21.15.32:0/2808190352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x277de90 0x28e7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.491+0000 7fd51a7fc700 0 client.5502 ms_handle_reset on v2:172.21.15.32:6834/3450615455 2020-12-05T17:34:07.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.491+0000 7fd5545f8740 1 -- 172.21.15.32:0/2808190352 >> 172.21.15.32:0/2808190352 conn(0x28c49f0 msgr2=0x28e1810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:07.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.491+0000 7fd5545f8740 1 -- 172.21.15.32:0/2808190352 shutdown_connections 2020-12-05T17:34:07.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.491+0000 7fd5545f8740 1 -- 172.21.15.32:0/2808190352 wait complete. 2020-12-05T17:34:07.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.491+0000 7fd5545f8740 20 client.5502 trim_cache size 0 max 16384 2020-12-05T17:34:07.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.507+0000 7fd5545f8740 1 Processor -- start 2020-12-05T17:34:07.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.507+0000 7fd5545f8740 1 -- start start 2020-12-05T17:34:07.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.507+0000 7fd5545f8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x26fd800 0x2918c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.507+0000 7fd5545f8740 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x277de90 0x28db7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.507+0000 7fd5545f8740 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x2895060 0x29158e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.507+0000 7fd5545f8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x2919590 con 0x26fd800 2020-12-05T17:34:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.507+0000 7fd5545f8740 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x28e8d70 con 0x2895060 2020-12-05T17:34:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.507+0000 7fd5545f8740 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x28dfc20 con 0x277de90 2020-12-05T17:34:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.507+0000 7fd53f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x26fd800 0x2918c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.507+0000 7fd53f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x26fd800 0x2918c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34838/0 (socket says 172.21.15.32:34838) 2020-12-05T17:34:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.507+0000 7fd53f7fe700 1 -- 172.21.15.32:0/500742495 learned_addr learned my addr 172.21.15.32:0/500742495 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:34:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.507+0000 7fd53ffff700 1 --2- 172.21.15.32:0/500742495 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x2895060 0x29158e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.507+0000 7fd53effd700 1 --2- 172.21.15.32:0/500742495 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x277de90 0x28db7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.507+0000 7fd53f7fe700 1 -- 172.21.15.32:0/500742495 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x277de90 msgr2=0x28db7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:34:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.507+0000 7fd53f7fe700 1 --2- 172.21.15.32:0/500742495 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x277de90 0x28db7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd53f7fe700 1 -- 172.21.15.32:0/500742495 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x2895060 msgr2=0x29158e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:34:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd53f7fe700 1 --2- 172.21.15.32:0/500742495 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x2895060 0x29158e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd53f7fe700 1 -- 172.21.15.32:0/500742495 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5401448d0 con 0x26fd800 2020-12-05T17:34:07.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd53f7fe700 1 --2- 172.21.15.32:0/500742495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x26fd800 0x2918c00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7fd53402b000 tx=0x7fd534029030).ready entity=mon.0 client_cookie=556257ba81532997 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:07.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd524ff9700 1 -- 172.21.15.32:0/500742495 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd53406e420 con 0x26fd800 2020-12-05T17:34:07.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd524ff9700 1 -- 172.21.15.32:0/500742495 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53406e580 con 0x26fd800 2020-12-05T17:34:07.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd5545f8740 1 -- 172.21.15.32:0/500742495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x26fd800 msgr2=0x2918c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:07.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd524ff9700 1 -- 172.21.15.32:0/500742495 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd53406ea00 con 0x26fd800 2020-12-05T17:34:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd5545f8740 1 --2- 172.21.15.32:0/500742495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x26fd800 0x2918c00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7fd53402b000 tx=0x7fd534029030).stop 2020-12-05T17:34:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd5545f8740 1 -- 172.21.15.32:0/500742495 shutdown_connections 2020-12-05T17:34:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd5545f8740 1 --2- 172.21.15.32:0/500742495 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x2895060 0x29158e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd5545f8740 1 --2- 172.21.15.32:0/500742495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x26fd800 0x2918c00 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd5545f8740 1 --2- 172.21.15.32:0/500742495 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x277de90 0x28db7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd5545f8740 1 -- 172.21.15.32:0/500742495 >> 172.21.15.32:0/500742495 conn(0x28c49f0 msgr2=0x281e710 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:07.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd5545f8740 1 -- 172.21.15.32:0/500742495 shutdown_connections 2020-12-05T17:34:07.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd5545f8740 1 -- 172.21.15.32:0/500742495 wait complete. 2020-12-05T17:34:07.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd5545f8740 1 -- start start 2020-12-05T17:34:07.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd5545f8740 1 Processor -- start 2020-12-05T17:34:07.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd5545f8740 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x26fd800 0x28dba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd5545f8740 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x277de90 0x2924f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd5545f8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2895060 0x2924140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd5545f8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x2925420 con 0x2895060 2020-12-05T17:34:07.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd5545f8740 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x2917730 con 0x277de90 2020-12-05T17:34:07.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd5545f8740 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x2915d40 con 0x26fd800 2020-12-05T17:34:07.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd5545f8740 10 client.0 ll_register_callbacks cb 0x28e99f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-12-05T17:34:07.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd53ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2895060 0x2924140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd53ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2895060 0x2924140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34846/0 (socket says 172.21.15.32:34846) 2020-12-05T17:34:07.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd53ffff700 1 -- 172.21.15.32:0/1010975003 learned_addr learned my addr 172.21.15.32:0/1010975003 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:34:07.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd53f7fe700 1 --2- 172.21.15.32:0/1010975003 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x26fd800 0x28dba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd53effd700 1 --2- 172.21.15.32:0/1010975003 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x277de90 0x2924f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd53ffff700 1 -- 172.21.15.32:0/1010975003 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x26fd800 msgr2=0x28dba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:34:07.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd53ffff700 1 --2- 172.21.15.32:0/1010975003 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x26fd800 0x28dba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd53ffff700 1 -- 172.21.15.32:0/1010975003 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x277de90 msgr2=0x2924f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:34:07.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd53ffff700 1 --2- 172.21.15.32:0/1010975003 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x277de90 0x2924f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:07.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd53ffff700 1 -- 172.21.15.32:0/1010975003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x26b8e50 con 0x2895060 2020-12-05T17:34:07.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd53ffff700 1 --2- 172.21.15.32:0/1010975003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2895060 0x2924140 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x2917870 tx=0x7fd53801d000).ready entity=mon.0 client_cookie=2d3af893c04006d4 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:07.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.511+0000 7fd51b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2020-12-05T17:34:07.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.515+0000 7fd51b7fe700 1 -- 172.21.15.32:0/1010975003 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd538022040 con 0x2895060 2020-12-05T17:34:07.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.515+0000 7fd51b7fe700 1 -- 172.21.15.32:0/1010975003 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd538029450 con 0x2895060 2020-12-05T17:34:07.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.515+0000 7fd5545f8740 1 -- 172.21.15.32:0/1010975003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2741710 con 0x2895060 2020-12-05T17:34:07.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.515+0000 7fd51b7fe700 1 -- 172.21.15.32:0/1010975003 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd538009a80 con 0x2895060 2020-12-05T17:34:07.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.515+0000 7fd5545f8740 10 client.5508 fetch_fsmap learned FSMap version 17 2020-12-05T17:34:07.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.515+0000 7fd5545f8740 1 -- 172.21.15.32:0/1010975003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x2935d40 con 0x2895060 2020-12-05T17:34:07.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.515+0000 7fd51b7fe700 1 -- 172.21.15.32:0/1010975003 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2741710 con 0x2895060 2020-12-05T17:34:07.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.515+0000 7fd51b7fe700 1 -- 172.21.15.32:0/1010975003 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 17) v1 ==== 1247+0+0 (secure 0 0 0) 0x7fd53801b070 con 0x2895060 2020-12-05T17:34:07.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.515+0000 7fd5545f8740 10 client.5508 fetch_fsmap finished waiting for FSMap version 17 2020-12-05T17:34:07.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.515+0000 7fd5545f8740 10 client.5508 resolve_mds: resolved ID 'a' to GID 5301 2020-12-05T17:34:07.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.515+0000 7fd5545f8740 20 client.5508 populate_metadata read hostname 'smithi032' 2020-12-05T17:34:07.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.515+0000 7fd5545f8740 1 --2- 172.21.15.32:0/1010975003 >> [v2:172.21.15.32:6834/3450615455,v1:172.21.15.32:6835/3450615455] conn(0x2924630 0x2916550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:07.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.515+0000 7fd5545f8740 4 client.5508 mds_command: new command op to 5301 tid=0[{"prefix": "cache drop"}] 2020-12-05T17:34:07.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.515+0000 7fd5545f8740 1 -- 172.21.15.32:0/1010975003 --> [v2:172.21.15.32:6834/3450615455,v1:172.21.15.32:6835/3450615455] -- command(tid 0: {"prefix": "cache drop"}) v1 -- 0x2718420 con 0x2924630 2020-12-05T17:34:07.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.515+0000 7fd53f7fe700 1 --2- 172.21.15.32:0/1010975003 >> [v2:172.21.15.32:6834/3450615455,v1:172.21.15.32:6835/3450615455] conn(0x2924630 0x2916550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:07.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.515+0000 7fd51b7fe700 10 client.5508 ms_handle_connect on v2:172.21.15.32:6834/3450615455 2020-12-05T17:34:07.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:07.515+0000 7fd53f7fe700 1 --2- 172.21.15.32:0/1010975003 >> [v2:172.21.15.32:6834/3450615455,v1:172.21.15.32:6835/3450615455] conn(0x2924630 0x2916550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=6adee4547346f3cb server_cookie=eb084b18aed701cd in_seq=0 out_seq=0 2020-12-05T17:34:15.002 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:34:15.005 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:34:15.007 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:34:15.009 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:34:15.011 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:34:18.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd51b7fe700 1 -- 172.21.15.32:0/1010975003 <== mds.0 v2:172.21.15.32:6834/3450615455 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+380 (crc 0 0 0) 0x2718420 con 0x2924630 2020-12-05T17:34:18.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd51b7fe700 10 client.5508 handle_command_reply: tid=0 2020-12-05T17:34:18.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd5545f8740 1 client.5508 shutdown 2020-12-05T17:34:18.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd5545f8740 1 -- 172.21.15.32:0/1010975003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2895060 msgr2=0x2924140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:18.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd5545f8740 1 --2- 172.21.15.32:0/1010975003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2895060 0x2924140 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x2917870 tx=0x7fd53801d000).stop 2020-12-05T17:34:18.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd5545f8740 1 -- 172.21.15.32:0/1010975003 shutdown_connections 2020-12-05T17:34:18.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd5545f8740 1 --2- 172.21.15.32:0/1010975003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2895060 0x2924140 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:18.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd5545f8740 1 --2- 172.21.15.32:0/1010975003 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x26fd800 0x28dba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:18.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd5545f8740 1 --2- 172.21.15.32:0/1010975003 >> [v2:172.21.15.32:6834/3450615455,v1:172.21.15.32:6835/3450615455] conn(0x2924630 0x2916550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:18.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd5545f8740 1 --2- 172.21.15.32:0/1010975003 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x277de90 0x2924f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:18.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd5545f8740 1 -- 172.21.15.32:0/1010975003 >> 172.21.15.32:0/1010975003 conn(0x28c49f0 msgr2=0x2918a90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:18.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd51b7fe700 0 client.5508 ms_handle_reset on v2:172.21.15.32:6834/3450615455 2020-12-05T17:34:18.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd5545f8740 1 -- 172.21.15.32:0/1010975003 shutdown_connections 2020-12-05T17:34:18.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd5545f8740 1 -- 172.21.15.32:0/1010975003 wait complete. 2020-12-05T17:34:18.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd5545f8740 20 client.5508 trim_cache size 0 max 16384 2020-12-05T17:34:18.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd545c5d700 1 -- 172.21.15.32:0/920427025 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd520055330 msgr2=0x7fd5200577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:18.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd545c5d700 1 --2- 172.21.15.32:0/920427025 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd520055330 0x7fd5200577f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fd53000d000 tx=0x7fd530009040).stop 2020-12-05T17:34:18.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd545c5d700 1 -- 172.21.15.32:0/920427025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5400b5240 msgr2=0x7fd5400a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:18.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd545c5d700 1 --2- 172.21.15.32:0/920427025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5400b5240 0x7fd5400a5780 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7fd534002f30 tx=0x7fd534015000).stop 2020-12-05T17:34:18.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd545c5d700 1 -- 172.21.15.32:0/920427025 shutdown_connections 2020-12-05T17:34:18.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd545c5d700 1 --2- 172.21.15.32:0/920427025 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd520055330 0x7fd5200577f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:18.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd545c5d700 1 --2- 172.21.15.32:0/920427025 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd5400b6690 0x7fd5401518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:18.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd545c5d700 1 --2- 172.21.15.32:0/920427025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5400b5240 0x7fd5400a5780 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:18.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd545c5d700 1 --2- 172.21.15.32:0/920427025 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd5400b5b20 0x7fd5400a5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:18.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd545c5d700 1 -- 172.21.15.32:0/920427025 >> 172.21.15.32:0/920427025 conn(0x7fd54005f3b0 msgr2=0x7fd5400b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:18.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd545c5d700 1 -- 172.21.15.32:0/920427025 shutdown_connections 2020-12-05T17:34:18.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:18.807+0000 7fd545c5d700 1 -- 172.21.15.32:0/920427025 wait complete. 2020-12-05T17:34:18.822 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:34:18.822 INFO:teuthology.orchestra.run.smithi032.stdout: "client_recall": { 2020-12-05T17:34:18.823 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2020-12-05T17:34:18.823 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "(0) Success", 2020-12-05T17:34:18.823 INFO:teuthology.orchestra.run.smithi032.stdout: "recalled": 220 2020-12-05T17:34:18.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:34:18.823 INFO:teuthology.orchestra.run.smithi032.stdout: "flush_journal": { 2020-12-05T17:34:18.824 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2020-12-05T17:34:18.824 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "" 2020-12-05T17:34:18.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:34:18.824 INFO:teuthology.orchestra.run.smithi032.stdout: "trim_cache": { 2020-12-05T17:34:18.824 INFO:teuthology.orchestra.run.smithi032.stdout: "trimmed": 0 2020-12-05T17:34:18.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:34:18.825 INFO:teuthology.orchestra.run.smithi032.stdout: "cache": { 2020-12-05T17:34:18.825 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": { 2020-12-05T17:34:18.825 INFO:teuthology.orchestra.run.smithi032.stdout: "items": 19916, 2020-12-05T17:34:18.825 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes": 3094944 2020-12-05T17:34:18.826 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:34:18.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:34:18.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 11.292496802 2020-12-05T17:34:18.826 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:34:18.827 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2020-12-05T17:34:18.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:34:18.828 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:34:18.829 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:34:18.836 INFO:teuthology.orchestra.run.smithi036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2020-12-05T17:34:18.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:34:18.837 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2020-12-05T17:34:18.837 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi036.front.sepia.ceph.com... 2020-12-05T17:34:18.838 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:34:18.838 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:34:18.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:34:18.916 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:34:18.917 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:34:18.965 INFO:teuthology.orchestra.run.smithi036.stdout:ext2/ext3 2020-12-05T17:34:18.966 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:34:18.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:34:18.967 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:34:19.013 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:34:19.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.135+0000 7f9ca88aa700 1 Processor -- start 2020-12-05T17:34:19.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.135+0000 7f9ca88aa700 1 -- start start 2020-12-05T17:34:19.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.135+0000 7f9ca88aa700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9ca40b5650 0x7f9ca414a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:19.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.135+0000 7f9ca88aa700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca40b5a30 con 0x7f9ca40afd20 2020-12-05T17:34:19.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.135+0000 7f9ca88aa700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca4067990 con 0x7f9ca414ac60 2020-12-05T17:34:19.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.135+0000 7f9ca88aa700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca40aea20 con 0x7f9ca40b5650 2020-12-05T17:34:19.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.135+0000 7f9ca37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9ca40b5650 0x7f9ca414a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:19.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.135+0000 7f9ca2ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9ca40afd20 0x7f9ca414a550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37504/0 (socket says 172.21.15.32:37504) 2020-12-05T17:34:19.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.135+0000 7f9ca2ffd700 1 -- 172.21.15.32:0/2518758054 learned_addr learned my addr 172.21.15.32:0/2518758054 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:34:19.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.135+0000 7f9ca37fe700 1 -- 172.21.15.32:0/2518758054 >> v1:172.21.15.61:6789/0 conn(0x7f9ca414ac60 legacy=0x7f9ca4146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:34:19.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.135+0000 7f9ca37fe700 1 -- 172.21.15.32:0/2518758054 >> v1:172.21.15.32:6789/0 conn(0x7f9ca40afd20 legacy=0x7f9ca414a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:19.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.135+0000 7f9ca27fc700 1 -- 172.21.15.32:0/2518758054 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4201589764 0 0) 0x7f9ca40b5a30 con 0x7f9ca40afd20 2020-12-05T17:34:19.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.135+0000 7f9ca37fe700 1 -- 172.21.15.32:0/2518758054 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca40aecc0 con 0x7f9ca40b5650 2020-12-05T17:34:19.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.135+0000 7f9ca37fe700 1 --2- 172.21.15.32:0/2518758054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca40b5650 0x7f9ca414a060 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f9ca40b5bc0 tx=0x7f9c8c006dc0).ready entity=mon.0 client_cookie=b90f2903c061b22c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:19.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.135+0000 7f9ca27fc700 1 -- 172.21.15.32:0/2518758054 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c8c0097b0 con 0x7f9ca40b5650 2020-12-05T17:34:19.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.135+0000 7f9ca27fc700 1 -- 172.21.15.32:0/2518758054 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8c009910 con 0x7f9ca40b5650 2020-12-05T17:34:19.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.135+0000 7f9ca27fc700 1 -- 172.21.15.32:0/2518758054 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c8c009c70 con 0x7f9ca40b5650 2020-12-05T17:34:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.135+0000 7f9ca88aa700 1 -- 172.21.15.32:0/2518758054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca40b5650 msgr2=0x7f9ca414a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.135+0000 7f9ca88aa700 1 --2- 172.21.15.32:0/2518758054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca40b5650 0x7f9ca414a060 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f9ca40b5bc0 tx=0x7f9c8c006dc0).stop 2020-12-05T17:34:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.139+0000 7f9ca88aa700 1 -- 172.21.15.32:0/2518758054 shutdown_connections 2020-12-05T17:34:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.139+0000 7f9ca88aa700 1 --2- 172.21.15.32:0/2518758054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca40b5650 0x7f9ca414a060 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.139+0000 7f9ca88aa700 1 -- 172.21.15.32:0/2518758054 >> 172.21.15.32:0/2518758054 conn(0x7f9ca405bbb0 msgr2=0x7f9ca4062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:19.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.139+0000 7f9ca88aa700 1 -- 172.21.15.32:0/2518758054 shutdown_connections 2020-12-05T17:34:19.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.139+0000 7f9ca88aa700 1 -- 172.21.15.32:0/2518758054 wait complete. 2020-12-05T17:34:19.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.139+0000 7f9ca88aa700 1 Processor -- start 2020-12-05T17:34:19.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.139+0000 7f9ca88aa700 1 -- start start 2020-12-05T17:34:19.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.143+0000 7f9ca88aa700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ca40afd20 0x7f9ca40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:19.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.143+0000 7f9ca88aa700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ca40b5650 0x7f9ca4143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:19.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.143+0000 7f9ca88aa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca414ac60 0x7f9ca41443e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:19.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.143+0000 7f9ca88aa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca4064630 con 0x7f9ca414ac60 2020-12-05T17:34:19.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.143+0000 7f9ca88aa700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca4066540 con 0x7f9ca40afd20 2020-12-05T17:34:19.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.143+0000 7f9ca88aa700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca4065100 con 0x7f9ca40b5650 2020-12-05T17:34:19.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.143+0000 7f9ca3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca414ac60 0x7f9ca41443e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:19.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.143+0000 7f9ca37fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ca40afd20 0x7f9ca40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:19.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.143+0000 7f9ca3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca414ac60 0x7f9ca41443e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34862/0 (socket says 172.21.15.32:34862) 2020-12-05T17:34:19.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.143+0000 7f9ca3fff700 1 -- 172.21.15.32:0/742020874 learned_addr learned my addr 172.21.15.32:0/742020874 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:34:19.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.143+0000 7f9ca2ffd700 1 --2- 172.21.15.32:0/742020874 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ca40b5650 0x7f9ca4143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:19.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.143+0000 7f9ca3fff700 1 -- 172.21.15.32:0/742020874 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ca40b5650 msgr2=0x7f9ca4143ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:19.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.143+0000 7f9ca3fff700 1 --2- 172.21.15.32:0/742020874 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ca40b5650 0x7f9ca4143ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.143+0000 7f9ca3fff700 1 -- 172.21.15.32:0/742020874 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ca40afd20 msgr2=0x7f9ca40b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:19.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.143+0000 7f9ca3fff700 1 --2- 172.21.15.32:0/742020874 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ca40afd20 0x7f9ca40b2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.143+0000 7f9ca3fff700 1 -- 172.21.15.32:0/742020874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca40653a0 con 0x7f9ca414ac60 2020-12-05T17:34:19.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.143+0000 7f9ca3fff700 1 --2- 172.21.15.32:0/742020874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca414ac60 0x7f9ca41443e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f9c9c002ef0 tx=0x7f9c9c006c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:19.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.143+0000 7f9ca0ff9700 1 -- 172.21.15.32:0/742020874 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c9c00f510 con 0x7f9ca414ac60 2020-12-05T17:34:19.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.143+0000 7f9ca0ff9700 1 -- 172.21.15.32:0/742020874 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9c00f670 con 0x7f9ca414ac60 2020-12-05T17:34:19.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.143+0000 7f9ca0ff9700 1 -- 172.21.15.32:0/742020874 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c9c00f7e0 con 0x7f9ca414ac60 2020-12-05T17:34:19.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.143+0000 7f9ca88aa700 1 -- 172.21.15.32:0/742020874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca40661c0 con 0x7f9ca414ac60 2020-12-05T17:34:19.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.143+0000 7f9ca88aa700 1 -- 172.21.15.32:0/742020874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca4066c20 con 0x7f9ca414ac60 2020-12-05T17:34:19.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.147+0000 7f9ca88aa700 1 -- 172.21.15.32:0/742020874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca4057050 con 0x7f9ca414ac60 2020-12-05T17:34:19.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.147+0000 7f9ca0ff9700 1 -- 172.21.15.32:0/742020874 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9c9c00f070 con 0x7f9ca414ac60 2020-12-05T17:34:19.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.147+0000 7f9ca0ff9700 1 --2- 172.21.15.32:0/742020874 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9c7c0553b0 0x7f9c7c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:19.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.147+0000 7f9ca0ff9700 1 -- 172.21.15.32:0/742020874 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9c9c06d760 con 0x7f9ca414ac60 2020-12-05T17:34:19.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.147+0000 7f9ca37fe700 1 --2- 172.21.15.32:0/742020874 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9c7c0553b0 0x7f9c7c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.147+0000 7f9ca37fe700 1 --2- 172.21.15.32:0/742020874 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9c7c0553b0 0x7f9c7c057870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f9c8c01b000 tx=0x7f9c8c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:19.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.155+0000 7f9ca0ff9700 1 -- 172.21.15.32:0/742020874 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9c9c043900 con 0x7f9ca414ac60 2020-12-05T17:34:19.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.291+0000 7f9ca88aa700 1 -- 172.21.15.32:0/742020874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ca40663b0 con 0x7f9ca414ac60 2020-12-05T17:34:19.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.291+0000 7f9ca0ff9700 1 -- 172.21.15.32:0/742020874 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16271 (secure 0 0 0) 0x7f9c9c006da0 con 0x7f9ca414ac60 2020-12-05T17:34:19.292 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:34:19.293 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":44,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:33:46.516416+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:33:43.025173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-12-05T17:33:44.344341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:34:19.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.295+0000 7f9ca88aa700 1 -- 172.21.15.32:0/742020874 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9c7c0553b0 msgr2=0x7f9c7c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:19.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.295+0000 7f9ca88aa700 1 --2- 172.21.15.32:0/742020874 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9c7c0553b0 0x7f9c7c057870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f9c8c01b000 tx=0x7f9c8c015000).stop 2020-12-05T17:34:19.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.295+0000 7f9ca88aa700 1 -- 172.21.15.32:0/742020874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca414ac60 msgr2=0x7f9ca41443e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.295+0000 7f9ca88aa700 1 --2- 172.21.15.32:0/742020874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca414ac60 0x7f9ca41443e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f9c9c002ef0 tx=0x7f9c9c006c80).stop 2020-12-05T17:34:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.295+0000 7f9ca88aa700 1 -- 172.21.15.32:0/742020874 shutdown_connections 2020-12-05T17:34:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.295+0000 7f9ca88aa700 1 --2- 172.21.15.32:0/742020874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca414ac60 0x7f9ca41443e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.295+0000 7f9ca88aa700 1 --2- 172.21.15.32:0/742020874 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ca40afd20 0x7f9ca40b2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.295+0000 7f9ca88aa700 1 --2- 172.21.15.32:0/742020874 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9c7c0553b0 0x7f9c7c057870 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.295+0000 7f9ca88aa700 1 --2- 172.21.15.32:0/742020874 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ca40b5650 0x7f9ca4143ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.295+0000 7f9ca88aa700 1 -- 172.21.15.32:0/742020874 >> 172.21.15.32:0/742020874 conn(0x7f9ca405bbb0 msgr2=0x7f9ca4149fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:19.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.295+0000 7f9ca88aa700 1 -- 172.21.15.32:0/742020874 shutdown_connections 2020-12-05T17:34:19.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.295+0000 7f9ca88aa700 1 -- 172.21.15.32:0/742020874 wait complete. 2020-12-05T17:34:19.309 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:33:43.025173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:33:44.344341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:34:19.310 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:34:19.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e3de40700 1 Processor -- start 2020-12-05T17:34:19.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e3de40700 1 -- start start 2020-12-05T17:34:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e3de40700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e380b0060 0x7f2e381476b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:19.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e3de40700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f2e38062b20 0x7f2e38145d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:19.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e3de40700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e38061910 con 0x7f2e380b5730 2020-12-05T17:34:19.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e3de40700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e380614f0 con 0x7f2e38062b20 2020-12-05T17:34:19.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e3de40700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e380b5b10 con 0x7f2e380b0060 2020-12-05T17:34:19.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e377fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2e380b5730 0x7f2e38146fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37514/0 (socket says 172.21.15.32:37514) 2020-12-05T17:34:19.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e377fe700 1 -- 172.21.15.32:0/424687157 learned_addr learned my addr 172.21.15.32:0/424687157 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:34:19.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e37fff700 1 --2- 172.21.15.32:0/424687157 >> v2:172.21.15.61:3300/0 conn(0x7f2e38062b20 0x7f2e38145d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:19.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e36ffd700 1 --2- 172.21.15.32:0/424687157 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e380b0060 0x7f2e381476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:19.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e367fc700 1 -- 172.21.15.32:0/424687157 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1654900810 0 0) 0x7f2e38061910 con 0x7f2e380b5730 2020-12-05T17:34:19.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e367fc700 1 -- 172.21.15.32:0/424687157 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e18003230 con 0x7f2e380b5730 2020-12-05T17:34:19.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e36ffd700 1 -- 172.21.15.32:0/424687157 >> v2:172.21.15.61:3300/0 conn(0x7f2e38062b20 msgr2=0x7f2e38145d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:34:19.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e36ffd700 1 --2- 172.21.15.32:0/424687157 >> v2:172.21.15.61:3300/0 conn(0x7f2e38062b20 0x7f2e38145d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e36ffd700 1 -- 172.21.15.32:0/424687157 >> v1:172.21.15.32:6789/0 conn(0x7f2e380b5730 legacy=0x7f2e38146fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:19.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e36ffd700 1 -- 172.21.15.32:0/424687157 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e38061f00 con 0x7f2e380b0060 2020-12-05T17:34:19.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e367fc700 1 -- 172.21.15.32:0/424687157 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2046226629 0 0) 0x7f2e18003230 con 0x7f2e380b5730 2020-12-05T17:34:19.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e36ffd700 1 --2- 172.21.15.32:0/424687157 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e380b0060 0x7f2e381476b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f2e20001870 tx=0x7f2e28006dc0).ready entity=mon.2 client_cookie=31e481535a3bd31d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:19.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e367fc700 1 -- 172.21.15.32:0/424687157 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e200016d0 con 0x7f2e380b0060 2020-12-05T17:34:19.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e367fc700 1 -- 172.21.15.32:0/424687157 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2800d570 con 0x7f2e380b0060 2020-12-05T17:34:19.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e3de40700 1 -- 172.21.15.32:0/424687157 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e380b0060 msgr2=0x7f2e381476b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:19.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e367fc700 1 -- 172.21.15.32:0/424687157 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e2800d6e0 con 0x7f2e380b0060 2020-12-05T17:34:19.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.423+0000 7f2e3de40700 1 --2- 172.21.15.32:0/424687157 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e380b0060 0x7f2e381476b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f2e20001870 tx=0x7f2e28006dc0).stop 2020-12-05T17:34:19.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.427+0000 7f2e3de40700 1 -- 172.21.15.32:0/424687157 shutdown_connections 2020-12-05T17:34:19.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.427+0000 7f2e3de40700 1 --2- 172.21.15.32:0/424687157 >> v2:172.21.15.61:3300/0 conn(0x7f2e38062b20 0x7f2e38145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.427+0000 7f2e3de40700 1 --2- 172.21.15.32:0/424687157 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e380b0060 0x7f2e381476b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.427+0000 7f2e3de40700 1 -- 172.21.15.32:0/424687157 >> 172.21.15.32:0/424687157 conn(0x7f2e3805fe80 msgr2=0x7f2e380b4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:19.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.427+0000 7f2e3de40700 1 -- 172.21.15.32:0/424687157 shutdown_connections 2020-12-05T17:34:19.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.427+0000 7f2e3de40700 1 -- 172.21.15.32:0/424687157 wait complete. 2020-12-05T17:34:19.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.427+0000 7f2e3de40700 1 Processor -- start 2020-12-05T17:34:19.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.427+0000 7f2e3de40700 1 -- start start 2020-12-05T17:34:19.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.427+0000 7f2e3de40700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e38062b20 0x7f2e3814dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:19.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.427+0000 7f2e3de40700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e380b0060 0x7f2e38159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:19.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.427+0000 7f2e3de40700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e380b5730 0x7f2e38159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:19.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.427+0000 7f2e3de40700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e380ae4f0 con 0x7f2e380b0060 2020-12-05T17:34:19.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.427+0000 7f2e3de40700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e380639a0 con 0x7f2e38062b20 2020-12-05T17:34:19.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.427+0000 7f2e3de40700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3815a450 con 0x7f2e380b5730 2020-12-05T17:34:19.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.427+0000 7f2e36ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e380b0060 0x7f2e38159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:19.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.427+0000 7f2e36ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e380b0060 0x7f2e38159a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34874/0 (socket says 172.21.15.32:34874) 2020-12-05T17:34:19.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.427+0000 7f2e36ffd700 1 -- 172.21.15.32:0/1734824075 learned_addr learned my addr 172.21.15.32:0/1734824075 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:34:19.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.431+0000 7f2e37fff700 1 --2- 172.21.15.32:0/1734824075 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e380b5730 0x7f2e38159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.431+0000 7f2e377fe700 1 --2- 172.21.15.32:0/1734824075 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e38062b20 0x7f2e3814dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.431+0000 7f2e36ffd700 1 -- 172.21.15.32:0/1734824075 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e380b5730 msgr2=0x7f2e38159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.431+0000 7f2e36ffd700 1 --2- 172.21.15.32:0/1734824075 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e380b5730 0x7f2e38159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.431+0000 7f2e36ffd700 1 -- 172.21.15.32:0/1734824075 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e38062b20 msgr2=0x7f2e3814dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.431+0000 7f2e36ffd700 1 --2- 172.21.15.32:0/1734824075 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e38062b20 0x7f2e3814dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.431+0000 7f2e36ffd700 1 -- 172.21.15.32:0/1734824075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e38159610 con 0x7f2e380b0060 2020-12-05T17:34:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.431+0000 7f2e36ffd700 1 --2- 172.21.15.32:0/1734824075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e380b0060 0x7f2e38159a70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f2e20001870 tx=0x7f2e28015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.431+0000 7f2e34ff9700 1 -- 172.21.15.32:0/1734824075 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e280117e0 con 0x7f2e380b0060 2020-12-05T17:34:19.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.431+0000 7f2e34ff9700 1 -- 172.21.15.32:0/1734824075 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e28011940 con 0x7f2e380b0060 2020-12-05T17:34:19.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.431+0000 7f2e3de40700 1 -- 172.21.15.32:0/1734824075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e38063700 con 0x7f2e380b0060 2020-12-05T17:34:19.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.431+0000 7f2e34ff9700 1 -- 172.21.15.32:0/1734824075 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e28011ca0 con 0x7f2e380b0060 2020-12-05T17:34:19.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.431+0000 7f2e3de40700 1 -- 172.21.15.32:0/1734824075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e38066a40 con 0x7f2e380b0060 2020-12-05T17:34:19.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.431+0000 7f2e3de40700 1 -- 172.21.15.32:0/1734824075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e38057050 con 0x7f2e380b0060 2020-12-05T17:34:19.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.431+0000 7f2e34ff9700 1 -- 172.21.15.32:0/1734824075 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2e28024030 con 0x7f2e380b0060 2020-12-05T17:34:19.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.431+0000 7f2e34ff9700 1 --2- 172.21.15.32:0/1734824075 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2e1c0553f0 0x7f2e1c0578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:19.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.431+0000 7f2e34ff9700 1 -- 172.21.15.32:0/1734824075 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2e2806ee90 con 0x7f2e380b0060 2020-12-05T17:34:19.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.431+0000 7f2e377fe700 1 --2- 172.21.15.32:0/1734824075 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2e1c0553f0 0x7f2e1c0578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:19.443 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.435+0000 7f2e377fe700 1 --2- 172.21.15.32:0/1734824075 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2e1c0553f0 0x7f2e1c0578b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f2e2000d000 tx=0x7f2e20001790).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:19.443 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.439+0000 7f2e34ff9700 1 -- 172.21.15.32:0/1734824075 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2e28045070 con 0x7f2e380b0060 2020-12-05T17:34:19.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.579+0000 7f2e3de40700 1 -- 172.21.15.32:0/1734824075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f2e380b33d0 con 0x7f2e380b0060 2020-12-05T17:34:19.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.579+0000 7f2e34ff9700 1 -- 172.21.15.32:0/1734824075 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v44) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2e28022020 con 0x7f2e380b0060 2020-12-05T17:34:19.581 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:34:19.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.583+0000 7f2e3de40700 1 -- 172.21.15.32:0/1734824075 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2e1c0553f0 msgr2=0x7f2e1c0578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:19.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.583+0000 7f2e3de40700 1 --2- 172.21.15.32:0/1734824075 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2e1c0553f0 0x7f2e1c0578b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f2e2000d000 tx=0x7f2e20001790).stop 2020-12-05T17:34:19.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.583+0000 7f2e3de40700 1 -- 172.21.15.32:0/1734824075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e380b0060 msgr2=0x7f2e38159a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:19.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.583+0000 7f2e3de40700 1 --2- 172.21.15.32:0/1734824075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e380b0060 0x7f2e38159a70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f2e20001870 tx=0x7f2e28015000).stop 2020-12-05T17:34:19.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.583+0000 7f2e3de40700 1 -- 172.21.15.32:0/1734824075 shutdown_connections 2020-12-05T17:34:19.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.583+0000 7f2e3de40700 1 --2- 172.21.15.32:0/1734824075 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2e1c0553f0 0x7f2e1c0578b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.583+0000 7f2e3de40700 1 --2- 172.21.15.32:0/1734824075 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e380b5730 0x7f2e38159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.583+0000 7f2e3de40700 1 --2- 172.21.15.32:0/1734824075 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e38062b20 0x7f2e3814dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.583+0000 7f2e3de40700 1 --2- 172.21.15.32:0/1734824075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e380b0060 0x7f2e38159a70 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.583+0000 7f2e3de40700 1 -- 172.21.15.32:0/1734824075 >> 172.21.15.32:0/1734824075 conn(0x7f2e3805fe80 msgr2=0x7f2e380b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:19.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.583+0000 7f2e3de40700 1 -- 172.21.15.32:0/1734824075 shutdown_connections 2020-12-05T17:34:19.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.583+0000 7f2e3de40700 1 -- 172.21.15.32:0/1734824075 wait complete. 2020-12-05T17:34:19.593 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:34:19.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.707+0000 7f5cb3d35700 1 Processor -- start 2020-12-05T17:34:19.707 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.707+0000 7f5cb3d35700 1 -- start start 2020-12-05T17:34:19.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.707+0000 7f5cb3d35700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5cac0b2830 0x7f5cac0b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:19.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.707+0000 7f5cb3d35700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cac063fd0 0x7f5cac143c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:19.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.707+0000 7f5cb3d35700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cac0b0840 con 0x7f5cac05fd70 2020-12-05T17:34:19.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.707+0000 7f5cb3d35700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cac064420 con 0x7f5cac0b2830 2020-12-05T17:34:19.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.707+0000 7f5cb3d35700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac05f710 con 0x7f5cac063fd0 2020-12-05T17:34:19.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.707+0000 7f5cb1ad1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5cac0b2830 0x7f5cac0b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:19.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.707+0000 7f5cb1ad1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5cac0b2830 0x7f5cac0b0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34880/0 (socket says 172.21.15.32:34880) 2020-12-05T17:34:19.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.707+0000 7f5cb1ad1700 1 -- 172.21.15.32:0/3730835330 learned_addr learned my addr 172.21.15.32:0/3730835330 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:34:19.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.707+0000 7f5cb22d2700 1 --2- 172.21.15.32:0/3730835330 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cac063fd0 0x7f5cac143c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:19.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.707+0000 7f5cb1ad1700 1 -- 172.21.15.32:0/3730835330 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cac063fd0 msgr2=0x7f5cac143c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:34:19.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.707+0000 7f5cb1ad1700 1 --2- 172.21.15.32:0/3730835330 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cac063fd0 0x7f5cac143c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.707+0000 7f5cb1ad1700 1 -- 172.21.15.32:0/3730835330 >> v1:172.21.15.61:6789/0 conn(0x7f5cac05fd70 legacy=0x7f5cac13fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:19.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.707+0000 7f5cb1ad1700 1 -- 172.21.15.32:0/3730835330 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac05f9b0 con 0x7f5cac0b2830 2020-12-05T17:34:19.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.707+0000 7f5cb1ad1700 1 --2- 172.21.15.32:0/3730835330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cac0b2830 0x7f5cac0b0c70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f5ca8002f30 tx=0x7f5ca8006e80).ready entity=mon.0 client_cookie=5033157e52c54aee server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:19.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.707+0000 7f5cb0acf700 1 -- 172.21.15.32:0/3730835330 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ca80097f0 con 0x7f5cac0b2830 2020-12-05T17:34:19.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.707+0000 7f5cb0acf700 1 -- 172.21.15.32:0/3730835330 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca8009950 con 0x7f5cac0b2830 2020-12-05T17:34:19.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.707+0000 7f5cb0acf700 1 -- 172.21.15.32:0/3730835330 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ca8009cb0 con 0x7f5cac0b2830 2020-12-05T17:34:19.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.707+0000 7f5cb3d35700 1 -- 172.21.15.32:0/3730835330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cac0b2830 msgr2=0x7f5cac0b0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:19.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.707+0000 7f5cb3d35700 1 --2- 172.21.15.32:0/3730835330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cac0b2830 0x7f5cac0b0c70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f5ca8002f30 tx=0x7f5ca8006e80).stop 2020-12-05T17:34:19.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb3d35700 1 -- 172.21.15.32:0/3730835330 shutdown_connections 2020-12-05T17:34:19.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb3d35700 1 --2- 172.21.15.32:0/3730835330 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cac063fd0 0x7f5cac143c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb3d35700 1 --2- 172.21.15.32:0/3730835330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cac0b2830 0x7f5cac0b0c70 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb3d35700 1 -- 172.21.15.32:0/3730835330 >> 172.21.15.32:0/3730835330 conn(0x7f5cac062310 msgr2=0x7f5cac0626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:19.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb3d35700 1 -- 172.21.15.32:0/3730835330 shutdown_connections 2020-12-05T17:34:19.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb3d35700 1 -- 172.21.15.32:0/3730835330 wait complete. 2020-12-05T17:34:19.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb3d35700 1 Processor -- start 2020-12-05T17:34:19.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb3d35700 1 -- start start 2020-12-05T17:34:19.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb3d35700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cac05fd70 0x7f5cac0b4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:19.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb3d35700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cac063fd0 0x7f5cac141ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:19.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb3d35700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cac0b2830 0x7f5cac1423d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:19.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb3d35700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac10f5b0 con 0x7f5cac05fd70 2020-12-05T17:34:19.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb3d35700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac10f310 con 0x7f5cac063fd0 2020-12-05T17:34:19.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb3d35700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac143500 con 0x7f5cac0b2830 2020-12-05T17:34:19.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb1ad1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cac05fd70 0x7f5cac0b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:19.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb22d2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cac0b2830 0x7f5cac1423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:19.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb12d0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cac063fd0 0x7f5cac141ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:19.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb1ad1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cac05fd70 0x7f5cac0b4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34886/0 (socket says 172.21.15.32:34886) 2020-12-05T17:34:19.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb1ad1700 1 -- 172.21.15.32:0/1029810550 learned_addr learned my addr 172.21.15.32:0/1029810550 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:34:19.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb1ad1700 1 -- 172.21.15.32:0/1029810550 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cac0b2830 msgr2=0x7f5cac1423d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb1ad1700 1 --2- 172.21.15.32:0/1029810550 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cac0b2830 0x7f5cac1423d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb1ad1700 1 -- 172.21.15.32:0/1029810550 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cac063fd0 msgr2=0x7f5cac141ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb1ad1700 1 --2- 172.21.15.32:0/1029810550 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cac063fd0 0x7f5cac141ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb1ad1700 1 -- 172.21.15.32:0/1029810550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac1436c0 con 0x7f5cac05fd70 2020-12-05T17:34:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb22d2700 1 --2- 172.21.15.32:0/1029810550 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cac0b2830 0x7f5cac1423d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:34:19.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb12d0700 1 --2- 172.21.15.32:0/1029810550 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cac063fd0 0x7f5cac141ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:34:19.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.711+0000 7f5cb1ad1700 1 --2- 172.21.15.32:0/1029810550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cac05fd70 0x7f5cac0b4430 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f5ca8002f30 tx=0x7f5ca8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:19.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.715+0000 7f5c9affd700 1 -- 172.21.15.32:0/1029810550 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ca8011b40 con 0x7f5cac05fd70 2020-12-05T17:34:19.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.715+0000 7f5c9affd700 1 -- 172.21.15.32:0/1029810550 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca8011ca0 con 0x7f5cac05fd70 2020-12-05T17:34:19.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.715+0000 7f5cb3d35700 1 -- 172.21.15.32:0/1029810550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cac1447b0 con 0x7f5cac05fd70 2020-12-05T17:34:19.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.715+0000 7f5c9affd700 1 -- 172.21.15.32:0/1029810550 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ca801e460 con 0x7f5cac05fd70 2020-12-05T17:34:19.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.715+0000 7f5cb3d35700 1 -- 172.21.15.32:0/1029810550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cac142ae0 con 0x7f5cac05fd70 2020-12-05T17:34:19.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.715+0000 7f5cb3d35700 1 -- 172.21.15.32:0/1029810550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cac10f150 con 0x7f5cac05fd70 2020-12-05T17:34:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.715+0000 7f5c9affd700 1 -- 172.21.15.32:0/1029810550 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5ca8025030 con 0x7f5cac05fd70 2020-12-05T17:34:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.715+0000 7f5c9affd700 1 --2- 172.21.15.32:0/1029810550 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5c8c0554e0 0x7f5c8c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.715+0000 7f5c9affd700 1 -- 172.21.15.32:0/1029810550 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5ca806f410 con 0x7f5cac05fd70 2020-12-05T17:34:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.715+0000 7f5cb12d0700 1 --2- 172.21.15.32:0/1029810550 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5c8c0554e0 0x7f5c8c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.715+0000 7f5cb12d0700 1 --2- 172.21.15.32:0/1029810550 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5c8c0554e0 0x7f5c8c0579a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f5c9c00d000 tx=0x7f5c9c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:19.728 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.723+0000 7f5c9affd700 1 -- 172.21.15.32:0/1029810550 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5ca80455b0 con 0x7f5cac05fd70 2020-12-05T17:34:19.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.863+0000 7f5cb3d35700 1 -- 172.21.15.32:0/1029810550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5cac144af0 con 0x7f5cac05fd70 2020-12-05T17:34:19.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.863+0000 7f5c9affd700 1 -- 172.21.15.32:0/1029810550 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v44) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5ca80230a0 con 0x7f5cac05fd70 2020-12-05T17:34:19.866 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:34:19.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.863+0000 7f5cb3d35700 1 -- 172.21.15.32:0/1029810550 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5c8c0554e0 msgr2=0x7f5c8c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:19.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.863+0000 7f5cb3d35700 1 --2- 172.21.15.32:0/1029810550 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5c8c0554e0 0x7f5c8c0579a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f5c9c00d000 tx=0x7f5c9c009040).stop 2020-12-05T17:34:19.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.863+0000 7f5cb3d35700 1 -- 172.21.15.32:0/1029810550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cac05fd70 msgr2=0x7f5cac0b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:19.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.863+0000 7f5cb3d35700 1 --2- 172.21.15.32:0/1029810550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cac05fd70 0x7f5cac0b4430 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f5ca8002f30 tx=0x7f5ca8015000).stop 2020-12-05T17:34:19.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.863+0000 7f5cb3d35700 1 -- 172.21.15.32:0/1029810550 shutdown_connections 2020-12-05T17:34:19.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.863+0000 7f5cb3d35700 1 --2- 172.21.15.32:0/1029810550 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5c8c0554e0 0x7f5c8c0579a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.863+0000 7f5cb3d35700 1 --2- 172.21.15.32:0/1029810550 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cac0b2830 0x7f5cac1423d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.863+0000 7f5cb3d35700 1 --2- 172.21.15.32:0/1029810550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cac05fd70 0x7f5cac0b4430 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.863+0000 7f5cb3d35700 1 --2- 172.21.15.32:0/1029810550 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cac063fd0 0x7f5cac141ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:19.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.863+0000 7f5cb3d35700 1 -- 172.21.15.32:0/1029810550 >> 172.21.15.32:0/1029810550 conn(0x7f5cac062310 msgr2=0x7f5cac13efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:19.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.863+0000 7f5cb3d35700 1 -- 172.21.15.32:0/1029810550 shutdown_connections 2020-12-05T17:34:19.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.863+0000 7f5cb3d35700 1 -- 172.21.15.32:0/1029810550 wait complete. 2020-12-05T17:34:19.877 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:34:19.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.999+0000 7fe2a2a0f700 1 Processor -- start 2020-12-05T17:34:19.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.999+0000 7fe2a2a0f700 1 -- start start 2020-12-05T17:34:20.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.999+0000 7fe2a2a0f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe29c05f6f0 0x7fe29c141700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:20.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.999+0000 7fe2a2a0f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe29c142300 0x7fe29c142790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:20.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.999+0000 7fe2a2a0f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe29c1439c0 con 0x7fe29c061b30 2020-12-05T17:34:20.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.999+0000 7fe2a2a0f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe29c05fe20 con 0x7fe29c05f6f0 2020-12-05T17:34:20.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.999+0000 7fe2a2a0f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c146910 con 0x7fe29c142300 2020-12-05T17:34:20.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.999+0000 7fe2a1a0d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe29c05f6f0 0x7fe29c141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:20.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.999+0000 7fe2a1a0d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe29c05f6f0 0x7fe29c141700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34896/0 (socket says 172.21.15.32:34896) 2020-12-05T17:34:20.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.999+0000 7fe2a1a0d700 1 -- 172.21.15.32:0/550052771 learned_addr learned my addr 172.21.15.32:0/550052771 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:34:20.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.999+0000 7fe2a220e700 1 --2- 172.21.15.32:0/550052771 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe29c142300 0x7fe29c142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:20.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.999+0000 7fe2a1a0d700 1 -- 172.21.15.32:0/550052771 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe29c142300 msgr2=0x7fe29c142790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:34:20.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.999+0000 7fe2a1a0d700 1 --2- 172.21.15.32:0/550052771 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe29c142300 0x7fe29c142790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.999+0000 7fe2a1a0d700 1 -- 172.21.15.32:0/550052771 >> v1:172.21.15.61:6789/0 conn(0x7fe29c061b30 legacy=0x7fe29c141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:20.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.999+0000 7fe2a1a0d700 1 -- 172.21.15.32:0/550052771 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c05fad0 con 0x7fe29c05f6f0 2020-12-05T17:34:20.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.999+0000 7fe2a1a0d700 1 --2- 172.21.15.32:0/550052771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe29c05f6f0 0x7fe29c141700 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fe298002f30 tx=0x7fe298006e20).ready entity=mon.0 client_cookie=ea7d1c2e4d9971f5 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:20.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.999+0000 7fe2a0a0b700 1 -- 172.21.15.32:0/550052771 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2980097f0 con 0x7fe29c05f6f0 2020-12-05T17:34:20.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.999+0000 7fe2a0a0b700 1 -- 172.21.15.32:0/550052771 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe298009950 con 0x7fe29c05f6f0 2020-12-05T17:34:20.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.999+0000 7fe2a0a0b700 1 -- 172.21.15.32:0/550052771 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe298009cb0 con 0x7fe29c05f6f0 2020-12-05T17:34:20.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.999+0000 7fe2a2a0f700 1 -- 172.21.15.32:0/550052771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe29c05f6f0 msgr2=0x7fe29c141700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:20.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:19.999+0000 7fe2a2a0f700 1 --2- 172.21.15.32:0/550052771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe29c05f6f0 0x7fe29c141700 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fe298002f30 tx=0x7fe298006e20).stop 2020-12-05T17:34:20.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.003+0000 7fe2a2a0f700 1 -- 172.21.15.32:0/550052771 shutdown_connections 2020-12-05T17:34:20.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.003+0000 7fe2a2a0f700 1 --2- 172.21.15.32:0/550052771 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe29c142300 0x7fe29c142790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.003+0000 7fe2a2a0f700 1 --2- 172.21.15.32:0/550052771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe29c05f6f0 0x7fe29c141700 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.003+0000 7fe2a2a0f700 1 -- 172.21.15.32:0/550052771 >> 172.21.15.32:0/550052771 conn(0x7fe29c061010 msgr2=0x7fe29c061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:20.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.003+0000 7fe2a2a0f700 1 -- 172.21.15.32:0/550052771 shutdown_connections 2020-12-05T17:34:20.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.003+0000 7fe2a2a0f700 1 -- 172.21.15.32:0/550052771 wait complete. 2020-12-05T17:34:20.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.003+0000 7fe2a2a0f700 1 Processor -- start 2020-12-05T17:34:20.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.003+0000 7fe2a2a0f700 1 -- start start 2020-12-05T17:34:20.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.003+0000 7fe2a2a0f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe29c05f6f0 0x7fe29c063ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:20.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.003+0000 7fe2a2a0f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe29c061b30 0x7fe29c063fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:20.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.003+0000 7fe2a2a0f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe29c142300 0x7fe29c144c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:20.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.003+0000 7fe2a2a0f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c065e80 con 0x7fe29c142300 2020-12-05T17:34:20.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.003+0000 7fe2a2a0f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c0660b0 con 0x7fe29c061b30 2020-12-05T17:34:20.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.003+0000 7fe2a2a0f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c066350 con 0x7fe29c05f6f0 2020-12-05T17:34:20.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.003+0000 7fe2a220e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe29c142300 0x7fe29c144c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:20.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.003+0000 7fe2a220e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe29c142300 0x7fe29c144c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34904/0 (socket says 172.21.15.32:34904) 2020-12-05T17:34:20.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.003+0000 7fe2a220e700 1 -- 172.21.15.32:0/2233329580 learned_addr learned my addr 172.21.15.32:0/2233329580 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:34:20.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.003+0000 7fe2a1a0d700 1 --2- 172.21.15.32:0/2233329580 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe29c05f6f0 0x7fe29c063ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:20.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.003+0000 7fe2a120c700 1 --2- 172.21.15.32:0/2233329580 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe29c061b30 0x7fe29c063fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:20.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.007+0000 7fe2a220e700 1 -- 172.21.15.32:0/2233329580 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe29c05f6f0 msgr2=0x7fe29c063ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:20.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.007+0000 7fe2a220e700 1 --2- 172.21.15.32:0/2233329580 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe29c05f6f0 0x7fe29c063ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.007+0000 7fe2a220e700 1 -- 172.21.15.32:0/2233329580 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe29c061b30 msgr2=0x7fe29c063fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:20.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.007+0000 7fe2a220e700 1 --2- 172.21.15.32:0/2233329580 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe29c061b30 0x7fe29c063fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.007+0000 7fe2a220e700 1 -- 172.21.15.32:0/2233329580 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c1521f0 con 0x7fe29c142300 2020-12-05T17:34:20.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.007+0000 7fe2a220e700 1 --2- 172.21.15.32:0/2233329580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe29c142300 0x7fe29c144c90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fe294006dd0 tx=0x7fe2940091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:20.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.007+0000 7fe28effd700 1 -- 172.21.15.32:0/2233329580 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe29400c8f0 con 0x7fe29c142300 2020-12-05T17:34:20.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.007+0000 7fe28effd700 1 -- 172.21.15.32:0/2233329580 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29400ca50 con 0x7fe29c142300 2020-12-05T17:34:20.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.007+0000 7fe2a2a0f700 1 -- 172.21.15.32:0/2233329580 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29c152690 con 0x7fe29c142300 2020-12-05T17:34:20.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.007+0000 7fe28effd700 1 -- 172.21.15.32:0/2233329580 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe29400cdb0 con 0x7fe29c142300 2020-12-05T17:34:20.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.007+0000 7fe2a2a0f700 1 -- 172.21.15.32:0/2233329580 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29c0b3f70 con 0x7fe29c142300 2020-12-05T17:34:20.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.007+0000 7fe2a2a0f700 1 -- 172.21.15.32:0/2233329580 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29c152380 con 0x7fe29c142300 2020-12-05T17:34:20.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.007+0000 7fe28effd700 1 -- 172.21.15.32:0/2233329580 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe294022070 con 0x7fe29c142300 2020-12-05T17:34:20.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.007+0000 7fe28effd700 1 --2- 172.21.15.32:0/2233329580 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe27c055440 0x7fe27c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.011+0000 7fe28effd700 1 -- 172.21.15.32:0/2233329580 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe29406e750 con 0x7fe29c142300 2020-12-05T17:34:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.011+0000 7fe2a1a0d700 1 --2- 172.21.15.32:0/2233329580 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe27c055440 0x7fe27c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.011+0000 7fe2a1a0d700 1 --2- 172.21.15.32:0/2233329580 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe27c055440 0x7fe27c057900 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fe29801b000 tx=0x7fe298015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:20.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.015+0000 7fe28effd700 1 -- 172.21.15.32:0/2233329580 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe2940447e0 con 0x7fe29c142300 2020-12-05T17:34:20.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.155+0000 7fe2a2a0f700 1 -- 172.21.15.32:0/2233329580 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe29c057050 con 0x7fe29c142300 2020-12-05T17:34:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.155+0000 7fe28effd700 1 -- 172.21.15.32:0/2233329580 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+10 (secure 0 0 0) 0x7fe294016020 con 0x7fe29c142300 2020-12-05T17:34:20.155 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:34:20.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.155+0000 7fe2a2a0f700 1 -- 172.21.15.32:0/2233329580 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe27c055440 msgr2=0x7fe27c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:20.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.155+0000 7fe2a2a0f700 1 --2- 172.21.15.32:0/2233329580 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe27c055440 0x7fe27c057900 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fe29801b000 tx=0x7fe298015000).stop 2020-12-05T17:34:20.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.155+0000 7fe2a2a0f700 1 -- 172.21.15.32:0/2233329580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe29c142300 msgr2=0x7fe29c144c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:20.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.155+0000 7fe2a2a0f700 1 --2- 172.21.15.32:0/2233329580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe29c142300 0x7fe29c144c90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fe294006dd0 tx=0x7fe2940091a0).stop 2020-12-05T17:34:20.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.155+0000 7fe2a2a0f700 1 -- 172.21.15.32:0/2233329580 shutdown_connections 2020-12-05T17:34:20.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.155+0000 7fe2a2a0f700 1 --2- 172.21.15.32:0/2233329580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe29c142300 0x7fe29c144c90 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.155+0000 7fe2a2a0f700 1 --2- 172.21.15.32:0/2233329580 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe27c055440 0x7fe27c057900 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.155+0000 7fe2a2a0f700 1 --2- 172.21.15.32:0/2233329580 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe29c05f6f0 0x7fe29c063ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.155+0000 7fe2a2a0f700 1 --2- 172.21.15.32:0/2233329580 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe29c061b30 0x7fe29c063fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.155+0000 7fe2a2a0f700 1 -- 172.21.15.32:0/2233329580 >> 172.21.15.32:0/2233329580 conn(0x7fe29c061010 msgr2=0x7fe29c146e40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:20.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.155+0000 7fe2a2a0f700 1 -- 172.21.15.32:0/2233329580 shutdown_connections 2020-12-05T17:34:20.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.155+0000 7fe2a2a0f700 1 -- 172.21.15.32:0/2233329580 wait complete. 2020-12-05T17:34:20.168 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:34:20.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.287+0000 7f5662e36700 1 Processor -- start 2020-12-05T17:34:20.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.287+0000 7f5662e36700 1 -- start start 2020-12-05T17:34:20.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.287+0000 7f5662e36700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f565c0b2100 0x7f565c0af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:20.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.287+0000 7f5662e36700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f565c0b24e0 con 0x7f565c13ed60 2020-12-05T17:34:20.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.287+0000 7f5662e36700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f565c0b0780 con 0x7f565c0ad250 2020-12-05T17:34:20.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.287+0000 7f5662e36700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f565c13fd10 con 0x7f565c0b2100 2020-12-05T17:34:20.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.287+0000 7f5661633700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f565c13ed60 0x7f565c138820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37560/0 (socket says 172.21.15.32:37560) 2020-12-05T17:34:20.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.287+0000 7f5661633700 1 -- 172.21.15.32:0/2580586432 learned_addr learned my addr 172.21.15.32:0/2580586432 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:34:20.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.287+0000 7f5661e34700 1 --2- 172.21.15.32:0/2580586432 >> v2:172.21.15.61:3300/0 conn(0x7f565c0b2100 0x7f565c0af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.287+0000 7f5660e32700 1 -- 172.21.15.32:0/2580586432 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104546048 0 0) 0x7f565c0b24e0 con 0x7f565c13ed60 2020-12-05T17:34:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.287+0000 7f5660e32700 1 -- 172.21.15.32:0/2580586432 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5644003230 con 0x7f565c13ed60 2020-12-05T17:34:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.287+0000 7f5661e34700 1 -- 172.21.15.32:0/2580586432 >> v1:172.21.15.61:6789/0 conn(0x7f565c0ad250 legacy=0x7f565c142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.287+0000 7f5661e34700 1 -- 172.21.15.32:0/2580586432 >> v1:172.21.15.32:6789/0 conn(0x7f565c13ed60 legacy=0x7f565c138820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:20.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.287+0000 7f5661e34700 1 -- 172.21.15.32:0/2580586432 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565c0b0240 con 0x7f565c0b2100 2020-12-05T17:34:20.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.287+0000 7f5660e32700 1 -- 172.21.15.32:0/2580586432 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3988226778 0 0) 0x7f5644003230 con 0x7f565c13ed60 2020-12-05T17:34:20.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.287+0000 7f5661e34700 1 --2- 172.21.15.32:0/2580586432 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f565c0b2100 0x7f565c0af360 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f5658002f30 tx=0x7f5658006dc0).ready entity=mon.1 client_cookie=eed006257b4abe70 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:20.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.287+0000 7f5660e32700 1 -- 172.21.15.32:0/2580586432 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f565800fa60 con 0x7f565c0b2100 2020-12-05T17:34:20.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.287+0000 7f5660e32700 1 -- 172.21.15.32:0/2580586432 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565800fbc0 con 0x7f565c0b2100 2020-12-05T17:34:20.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.287+0000 7f5660e32700 1 -- 172.21.15.32:0/2580586432 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5658010470 con 0x7f565c0b2100 2020-12-05T17:34:20.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.287+0000 7f5662e36700 1 -- 172.21.15.32:0/2580586432 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f565c0b2100 msgr2=0x7f565c0af360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:20.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.287+0000 7f5662e36700 1 --2- 172.21.15.32:0/2580586432 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f565c0b2100 0x7f565c0af360 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f5658002f30 tx=0x7f5658006dc0).stop 2020-12-05T17:34:20.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.291+0000 7f5662e36700 1 -- 172.21.15.32:0/2580586432 shutdown_connections 2020-12-05T17:34:20.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.291+0000 7f5662e36700 1 --2- 172.21.15.32:0/2580586432 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f565c0b2100 0x7f565c0af360 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.291+0000 7f5662e36700 1 -- 172.21.15.32:0/2580586432 >> 172.21.15.32:0/2580586432 conn(0x7f565c0620b0 msgr2=0x7f565c062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:20.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.291+0000 7f5662e36700 1 -- 172.21.15.32:0/2580586432 shutdown_connections 2020-12-05T17:34:20.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.291+0000 7f5662e36700 1 -- 172.21.15.32:0/2580586432 wait complete. 2020-12-05T17:34:20.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.291+0000 7f5662e36700 1 Processor -- start 2020-12-05T17:34:20.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.291+0000 7f5662e36700 1 -- start start 2020-12-05T17:34:20.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.295+0000 7f5662e36700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f565c0ad250 0x7f565c066c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:20.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.295+0000 7f5662e36700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f565c0b2100 0x7f565c0638e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:20.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.295+0000 7f5662e36700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f565c065450 0x7f565c138f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:20.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.295+0000 7f5662e36700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c064d60 con 0x7f565c065450 2020-12-05T17:34:20.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.295+0000 7f5662e36700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c0658e0 con 0x7f565c0ad250 2020-12-05T17:34:20.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.295+0000 7f5662e36700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565c0647e0 con 0x7f565c0b2100 2020-12-05T17:34:20.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.295+0000 7f5661e34700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f565c0ad250 0x7f565c066c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:20.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.295+0000 7f5661e34700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f565c0ad250 0x7f565c066c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:54806/0 (socket says 172.21.15.32:54806) 2020-12-05T17:34:20.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.295+0000 7f5661e34700 1 -- 172.21.15.32:0/1771709863 learned_addr learned my addr 172.21.15.32:0/1771709863 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:34:20.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.295+0000 7f5662635700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f565c065450 0x7f565c138f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:20.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.295+0000 7f5661633700 1 --2- 172.21.15.32:0/1771709863 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f565c0b2100 0x7f565c0638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:20.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.295+0000 7f5661e34700 1 -- 172.21.15.32:0/1771709863 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f565c0b2100 msgr2=0x7f565c0638e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:20.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.295+0000 7f5661e34700 1 --2- 172.21.15.32:0/1771709863 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f565c0b2100 0x7f565c0638e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.295+0000 7f5661e34700 1 -- 172.21.15.32:0/1771709863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f565c065450 msgr2=0x7f565c138f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.295+0000 7f5661e34700 1 --2- 172.21.15.32:0/1771709863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f565c065450 0x7f565c138f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.295+0000 7f5661e34700 1 -- 172.21.15.32:0/1771709863 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565c064fb0 con 0x7f565c0ad250 2020-12-05T17:34:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.295+0000 7f5661e34700 1 --2- 172.21.15.32:0/1771709863 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f565c0ad250 0x7f565c066c40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f5658002f30 tx=0x7f5658017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:20.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.295+0000 7f564affd700 1 -- 172.21.15.32:0/1771709863 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f565801a450 con 0x7f565c0ad250 2020-12-05T17:34:20.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.295+0000 7f564affd700 1 -- 172.21.15.32:0/1771709863 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565801e070 con 0x7f565c0ad250 2020-12-05T17:34:20.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.295+0000 7f5662e36700 1 -- 172.21.15.32:0/1771709863 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f565c13e480 con 0x7f565c0ad250 2020-12-05T17:34:20.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.295+0000 7f564affd700 1 -- 172.21.15.32:0/1771709863 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f565801a7a0 con 0x7f565c0ad250 2020-12-05T17:34:20.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.295+0000 7f5662e36700 1 -- 172.21.15.32:0/1771709863 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f565c139470 con 0x7f565c0ad250 2020-12-05T17:34:20.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.299+0000 7f5662e36700 1 -- 172.21.15.32:0/1771709863 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f565c064620 con 0x7f565c0ad250 2020-12-05T17:34:20.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.299+0000 7f564affd700 1 -- 172.21.15.32:0/1771709863 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f565802a070 con 0x7f565c0ad250 2020-12-05T17:34:20.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.299+0000 7f564affd700 1 --2- 172.21.15.32:0/1771709863 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f563c0554e0 0x7f563c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:20.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.299+0000 7f5661633700 1 --2- 172.21.15.32:0/1771709863 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f563c0554e0 0x7f563c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:20.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.299+0000 7f564affd700 1 -- 172.21.15.32:0/1771709863 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f565806f820 con 0x7f565c0ad250 2020-12-05T17:34:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.299+0000 7f5661633700 1 --2- 172.21.15.32:0/1771709863 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f563c0554e0 0x7f563c0579a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f564c00d000 tx=0x7f564c007a60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.311+0000 7f564affd700 1 -- 172.21.15.32:0/1771709863 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f56580459c0 con 0x7f565c0ad250 2020-12-05T17:34:20.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.455+0000 7f5662e36700 1 -- 172.21.15.32:0/1771709863 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f565c065140 con 0x7f565c0ad250 2020-12-05T17:34:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.455+0000 7f564affd700 1 -- 172.21.15.32:0/1771709863 <== mon.1 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5658048fe0 con 0x7f565c0ad250 2020-12-05T17:34:20.457 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:34:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.459+0000 7f5662e36700 1 -- 172.21.15.32:0/1771709863 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f563c0554e0 msgr2=0x7f563c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.459+0000 7f5662e36700 1 --2- 172.21.15.32:0/1771709863 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f563c0554e0 0x7f563c0579a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f564c00d000 tx=0x7f564c007a60).stop 2020-12-05T17:34:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.459+0000 7f5662e36700 1 -- 172.21.15.32:0/1771709863 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f565c0ad250 msgr2=0x7f565c066c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.459+0000 7f5662e36700 1 --2- 172.21.15.32:0/1771709863 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f565c0ad250 0x7f565c066c40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f5658002f30 tx=0x7f5658017000).stop 2020-12-05T17:34:20.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.459+0000 7f5662e36700 1 -- 172.21.15.32:0/1771709863 shutdown_connections 2020-12-05T17:34:20.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.459+0000 7f5662e36700 1 --2- 172.21.15.32:0/1771709863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f565c065450 0x7f565c138f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.459+0000 7f5662e36700 1 --2- 172.21.15.32:0/1771709863 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f565c0ad250 0x7f565c066c40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.459+0000 7f5662e36700 1 --2- 172.21.15.32:0/1771709863 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f563c0554e0 0x7f563c0579a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.461 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.459+0000 7f5662e36700 1 --2- 172.21.15.32:0/1771709863 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f565c0b2100 0x7f565c0638e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.461 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.459+0000 7f5662e36700 1 -- 172.21.15.32:0/1771709863 >> 172.21.15.32:0/1771709863 conn(0x7f565c0620b0 msgr2=0x7f565c13b370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:20.461 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.459+0000 7f5662e36700 1 -- 172.21.15.32:0/1771709863 shutdown_connections 2020-12-05T17:34:20.462 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.459+0000 7f5662e36700 1 -- 172.21.15.32:0/1771709863 wait complete. 2020-12-05T17:34:20.471 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:34:20.471 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:34:20.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.583+0000 7fae55f09700 1 Processor -- start 2020-12-05T17:34:20.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.583+0000 7fae55f09700 1 -- start start 2020-12-05T17:34:20.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.583+0000 7fae55f09700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fae5005f7f0 0x7fae500b0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:20.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.583+0000 7fae55f09700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fae50142f50 0x7fae501433e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:20.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.583+0000 7fae55f09700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae501438d0 con 0x7fae500b2870 2020-12-05T17:34:20.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.583+0000 7fae55f09700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae501428c0 con 0x7fae5005f7f0 2020-12-05T17:34:20.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.583+0000 7fae55f09700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5005fd00 con 0x7fae50142f50 2020-12-05T17:34:20.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.583+0000 7fae4effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fae5005f7f0 0x7fae500b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:20.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.583+0000 7fae4effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fae5005f7f0 0x7fae500b0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34924/0 (socket says 172.21.15.32:34924) 2020-12-05T17:34:20.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.583+0000 7fae4effd700 1 -- 172.21.15.32:0/2154308680 learned_addr learned my addr 172.21.15.32:0/2154308680 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:34:20.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.583+0000 7fae4ffff700 1 --2- 172.21.15.32:0/2154308680 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fae50142f50 0x7fae501433e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:20.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.587+0000 7fae4effd700 1 -- 172.21.15.32:0/2154308680 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fae50142f50 msgr2=0x7fae501433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:34:20.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.587+0000 7fae4effd700 1 --2- 172.21.15.32:0/2154308680 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fae50142f50 0x7fae501433e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.587+0000 7fae4effd700 1 -- 172.21.15.32:0/2154308680 >> v1:172.21.15.61:6789/0 conn(0x7fae500b2870 legacy=0x7fae50146860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:20.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.587+0000 7fae4effd700 1 -- 172.21.15.32:0/2154308680 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae50142ab0 con 0x7fae5005f7f0 2020-12-05T17:34:20.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.587+0000 7fae4effd700 1 --2- 172.21.15.32:0/2154308680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae5005f7f0 0x7fae500b0940 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7fae40002f30 tx=0x7fae40006e80).ready entity=mon.0 client_cookie=2016d9c9780d7350 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:20.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.587+0000 7fae4e7fc700 1 -- 172.21.15.32:0/2154308680 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae400097f0 con 0x7fae5005f7f0 2020-12-05T17:34:20.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.587+0000 7fae4e7fc700 1 -- 172.21.15.32:0/2154308680 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae40009950 con 0x7fae5005f7f0 2020-12-05T17:34:20.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.587+0000 7fae4e7fc700 1 -- 172.21.15.32:0/2154308680 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae40009cb0 con 0x7fae5005f7f0 2020-12-05T17:34:20.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.587+0000 7fae55f09700 1 -- 172.21.15.32:0/2154308680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae5005f7f0 msgr2=0x7fae500b0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:20.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.587+0000 7fae55f09700 1 --2- 172.21.15.32:0/2154308680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae5005f7f0 0x7fae500b0940 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7fae40002f30 tx=0x7fae40006e80).stop 2020-12-05T17:34:20.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.587+0000 7fae55f09700 1 -- 172.21.15.32:0/2154308680 shutdown_connections 2020-12-05T17:34:20.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.587+0000 7fae55f09700 1 --2- 172.21.15.32:0/2154308680 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fae50142f50 0x7fae501433e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.587+0000 7fae55f09700 1 --2- 172.21.15.32:0/2154308680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae5005f7f0 0x7fae500b0940 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.587+0000 7fae55f09700 1 -- 172.21.15.32:0/2154308680 >> 172.21.15.32:0/2154308680 conn(0x7fae500604c0 msgr2=0x7fae5005c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:20.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.587+0000 7fae55f09700 1 -- 172.21.15.32:0/2154308680 shutdown_connections 2020-12-05T17:34:20.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.587+0000 7fae55f09700 1 -- 172.21.15.32:0/2154308680 wait complete. 2020-12-05T17:34:20.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae55f09700 1 Processor -- start 2020-12-05T17:34:20.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae55f09700 1 -- start start 2020-12-05T17:34:20.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae55f09700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fae5005f7f0 0x7fae500660f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:20.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae55f09700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae500b2870 0x7fae500665e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:20.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae55f09700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae50142f50 0x7fae50151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:20.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae55f09700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae500b6430 con 0x7fae50142f50 2020-12-05T17:34:20.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae55f09700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae500b56e0 con 0x7fae500b2870 2020-12-05T17:34:20.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae55f09700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae50064350 con 0x7fae5005f7f0 2020-12-05T17:34:20.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae4ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae50142f50 0x7fae50151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:20.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae4ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae50142f50 0x7fae50151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34932/0 (socket says 172.21.15.32:34932) 2020-12-05T17:34:20.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae4ffff700 1 -- 172.21.15.32:0/1509473801 learned_addr learned my addr 172.21.15.32:0/1509473801 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:34:20.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae4effd700 1 --2- 172.21.15.32:0/1509473801 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae500b2870 0x7fae500665e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:20.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae4f7fe700 1 --2- 172.21.15.32:0/1509473801 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fae5005f7f0 0x7fae500660f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:20.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae4ffff700 1 -- 172.21.15.32:0/1509473801 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fae5005f7f0 msgr2=0x7fae500660f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:20.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae4ffff700 1 --2- 172.21.15.32:0/1509473801 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fae5005f7f0 0x7fae500660f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae4ffff700 1 -- 172.21.15.32:0/1509473801 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae500b2870 msgr2=0x7fae500665e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:20.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae4ffff700 1 --2- 172.21.15.32:0/1509473801 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae500b2870 0x7fae500665e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae4ffff700 1 -- 172.21.15.32:0/1509473801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae500645f0 con 0x7fae50142f50 2020-12-05T17:34:20.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae4ffff700 1 --2- 172.21.15.32:0/1509473801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae50142f50 0x7fae50151620 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7fae4800a000 tx=0x7fae48008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:20.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae4cff9700 1 -- 172.21.15.32:0/1509473801 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae4800b980 con 0x7fae50142f50 2020-12-05T17:34:20.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae4cff9700 1 -- 172.21.15.32:0/1509473801 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4800bae0 con 0x7fae50142f50 2020-12-05T17:34:20.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae55f09700 1 -- 172.21.15.32:0/1509473801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae500b6070 con 0x7fae50142f50 2020-12-05T17:34:20.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae4cff9700 1 -- 172.21.15.32:0/1509473801 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae4800be40 con 0x7fae50142f50 2020-12-05T17:34:20.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.591+0000 7fae55f09700 1 -- 172.21.15.32:0/1509473801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae500b6a20 con 0x7fae50142f50 2020-12-05T17:34:20.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.595+0000 7fae55f09700 1 -- 172.21.15.32:0/1509473801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae50064780 con 0x7fae50142f50 2020-12-05T17:34:20.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.595+0000 7fae4cff9700 1 -- 172.21.15.32:0/1509473801 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fae4801c460 con 0x7fae50142f50 2020-12-05T17:34:20.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.595+0000 7fae4cff9700 1 --2- 172.21.15.32:0/1509473801 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fae300554e0 0x7fae300579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:34:20.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.595+0000 7fae4cff9700 1 -- 172.21.15.32:0/1509473801 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fae4806e740 con 0x7fae50142f50 2020-12-05T17:34:20.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.595+0000 7fae4f7fe700 1 --2- 172.21.15.32:0/1509473801 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fae300554e0 0x7fae300579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:34:20.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.595+0000 7fae4f7fe700 1 --2- 172.21.15.32:0/1509473801 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fae300554e0 0x7fae300579a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fae4400d000 tx=0x7fae44007b60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:34:20.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.603+0000 7fae4cff9700 1 -- 172.21.15.32:0/1509473801 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fae48044880 con 0x7fae50142f50 2020-12-05T17:34:20.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.791+0000 7fae55f09700 1 -- 172.21.15.32:0/1509473801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae500b6570 con 0x7fae50142f50 2020-12-05T17:34:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.791+0000 7fae4cff9700 1 -- 172.21.15.32:0/1509473801 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2138 (secure 0 0 0) 0x7fae480074b0 con 0x7fae50142f50 2020-12-05T17:34:20.792 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:34:20.792 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4919,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/3405415630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3405415630},{"type":"v1","addr":"172.21.15.61:6833","nonce":3405415630}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:33:45.513696+0000","modified":"2020-12-05T17:33:46.989395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5301":{"gid":5301,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3450615455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3450615455},{"type":"v1","addr":"172.21.15.32:6835","nonce":3450615455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-12-05T17:34:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.791+0000 7fae55f09700 1 -- 172.21.15.32:0/1509473801 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fae300554e0 msgr2=0x7fae300579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.791+0000 7fae55f09700 1 --2- 172.21.15.32:0/1509473801 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fae300554e0 0x7fae300579a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fae4400d000 tx=0x7fae44007b60).stop 2020-12-05T17:34:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.791+0000 7fae55f09700 1 -- 172.21.15.32:0/1509473801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae50142f50 msgr2=0x7fae50151620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:34:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.791+0000 7fae55f09700 1 --2- 172.21.15.32:0/1509473801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae50142f50 0x7fae50151620 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7fae4800a000 tx=0x7fae48008040).stop 2020-12-05T17:34:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.791+0000 7fae55f09700 1 -- 172.21.15.32:0/1509473801 shutdown_connections 2020-12-05T17:34:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.795+0000 7fae55f09700 1 --2- 172.21.15.32:0/1509473801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae50142f50 0x7fae50151620 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.795+0000 7fae55f09700 1 --2- 172.21.15.32:0/1509473801 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fae300554e0 0x7fae300579a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.795+0000 7fae55f09700 1 --2- 172.21.15.32:0/1509473801 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fae5005f7f0 0x7fae500660f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.795+0000 7fae55f09700 1 --2- 172.21.15.32:0/1509473801 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae500b2870 0x7fae500665e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:34:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.795+0000 7fae55f09700 1 -- 172.21.15.32:0/1509473801 >> 172.21.15.32:0/1509473801 conn(0x7fae500604c0 msgr2=0x7fae500b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:34:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.795+0000 7fae55f09700 1 -- 172.21.15.32:0/1509473801 shutdown_connections 2020-12-05T17:34:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:34:20.795+0000 7fae55f09700 1 -- 172.21.15.32:0/1509473801 wait complete. 2020-12-05T17:34:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 17 2020-12-05T17:34:21.009 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:33:45.513696+0000', 'modified': '2020-12-05T17:33:46.989395+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5301}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5301': {'gid': 5301, 'name': 'a', 'rank': 0, 'incarnation': 15, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/3450615455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 3450615455}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 3450615455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:34:21.009 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:34:21.010 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:34:21.141 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":5301,"want_state":"up:active","state":"up:active","rank_uptime":35.618211809999998,"mdsmap_epoch":17,"osdmap_epoch":44,"osdmap_epoch_barrier":44,"uptime":41.042324221000001} 2020-12-05T17:34:21.141 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 5301, "mdsmap_epoch": 17, "osdmap_epoch": 44, "osdmap_epoch_barrier": 44, "rank_uptime": 35.61821181, "state": "up:active", "uptime": 41.042324221, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:34:21.142 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:34:21.142 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-12-05T17:34:21.143 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-12-05T17:34:21.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:34:21.144 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:34:21.151 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:34:21.208 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:34:21.209 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:34:21.265 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:34:21.266 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:34:21.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:34:21.267 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:34:21.313 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:34:21.313 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:34:21.356 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:34:21.372 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:34:21.373 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:34:21.430 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:34:21.431 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:34:21.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:34:21.432 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:34:21.435 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:2020-12-05T17:34:21.431+0000 7f8439b200c0 -1 init, newargv = 0x561330fad380 newargc=9ceph-fuse[10631]: starting ceph client 2020-12-05T17:34:21.435 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr: 2020-12-05T17:34:21.444 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[10631]: starting fuse 2020-12-05T17:34:22.540 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:34:22.557 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:34:22.558 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:34:22.613 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:34:22.614 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:34:22.615 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:34:22.615 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:34:22.660 INFO:teuthology.orchestra.run.smithi036.stdout:49 2020-12-05T17:34:22.662 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-12-05T17:34:22.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:34:22.663 INFO:teuthology.orchestra.run.smithi036:> sudo python3 -c ' 2020-12-05T17:34:22.663 INFO:teuthology.orchestra.run.smithi036:> import glob 2020-12-05T17:34:22.664 INFO:teuthology.orchestra.run.smithi036:> import re 2020-12-05T17:34:22.664 INFO:teuthology.orchestra.run.smithi036:> import os 2020-12-05T17:34:22.664 INFO:teuthology.orchestra.run.smithi036:> import subprocess 2020-12-05T17:34:22.665 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:34:22.665 INFO:teuthology.orchestra.run.smithi036:> def find_socket(client_name): 2020-12-05T17:34:22.665 INFO:teuthology.orchestra.run.smithi036:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-12-05T17:34:22.666 INFO:teuthology.orchestra.run.smithi036:> files = glob.glob(asok_path) 2020-12-05T17:34:22.666 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:34:22.667 INFO:teuthology.orchestra.run.smithi036:> # Given a non-glob path, it better be there 2020-12-05T17:34:22.667 INFO:teuthology.orchestra.run.smithi036:> if "*" not in asok_path: 2020-12-05T17:34:22.668 INFO:teuthology.orchestra.run.smithi036:> assert(len(files) == 1) 2020-12-05T17:34:22.668 INFO:teuthology.orchestra.run.smithi036:> return files[0] 2020-12-05T17:34:22.669 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:34:22.669 INFO:teuthology.orchestra.run.smithi036:> for f in files: 2020-12-05T17:34:22.669 INFO:teuthology.orchestra.run.smithi036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:34:22.670 INFO:teuthology.orchestra.run.smithi036:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:34:22.670 INFO:teuthology.orchestra.run.smithi036:> return f 2020-12-05T17:34:22.671 INFO:teuthology.orchestra.run.smithi036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:34:22.671 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:34:22.671 INFO:teuthology.orchestra.run.smithi036:> print(find_socket("client.0")) 2020-12-05T17:34:22.672 INFO:teuthology.orchestra.run.smithi036:> ' 2020-12-05T17:34:22.754 INFO:teuthology.orchestra.run.smithi036.stdout:/var/run/ceph/ceph-client.0.10631.asok 2020-12-05T17:34:22.759 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.10631.asok 2020-12-05T17:34:22.760 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:34:22.760 INFO:teuthology.orchestra.run.smithi036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.10631.asok status 2020-12-05T17:34:22.864 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2020-12-05T17:34:22.865 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": { 2020-12-05T17:34:22.865 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:34:22.865 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:34:22.865 INFO:teuthology.orchestra.run.smithi036.stdout: "entity_id": "0", 2020-12-05T17:34:22.866 INFO:teuthology.orchestra.run.smithi036.stdout: "hostname": "smithi036", 2020-12-05T17:34:22.866 INFO:teuthology.orchestra.run.smithi036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-12-05T17:34:22.866 INFO:teuthology.orchestra.run.smithi036.stdout: "pid": "10631", 2020-12-05T17:34:22.867 INFO:teuthology.orchestra.run.smithi036.stdout: "root": "/" 2020-12-05T17:34:22.867 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:34:22.867 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_count": 0, 2020-12-05T17:34:22.867 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_pinned_count": 0, 2020-12-05T17:34:22.868 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 5556, 2020-12-05T17:34:22.868 INFO:teuthology.orchestra.run.smithi036.stdout: "inst": { 2020-12-05T17:34:22.868 INFO:teuthology.orchestra.run.smithi036.stdout: "name": { 2020-12-05T17:34:22.868 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "client", 2020-12-05T17:34:22.869 INFO:teuthology.orchestra.run.smithi036.stdout: "num": 5556 2020-12-05T17:34:22.869 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:34:22.869 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:34:22.869 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:34:22.869 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:34:22.870 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 3309890406 2020-12-05T17:34:22.870 INFO:teuthology.orchestra.run.smithi036.stdout: } 2020-12-05T17:34:22.870 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:34:22.870 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:34:22.871 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:34:22.871 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:34:22.871 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 3309890406 2020-12-05T17:34:22.871 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:34:22.872 INFO:teuthology.orchestra.run.smithi036.stdout: "inst_str": "client.5556 172.21.15.36:0/3309890406", 2020-12-05T17:34:22.872 INFO:teuthology.orchestra.run.smithi036.stdout: "addr_str": "172.21.15.36:0/3309890406", 2020-12-05T17:34:22.872 INFO:teuthology.orchestra.run.smithi036.stdout: "inode_count": 1, 2020-12-05T17:34:22.872 INFO:teuthology.orchestra.run.smithi036.stdout: "mds_epoch": 17, 2020-12-05T17:34:22.872 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch": 44, 2020-12-05T17:34:22.873 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:34:22.873 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklisted": false 2020-12-05T17:34:22.873 INFO:teuthology.orchestra.run.smithi036.stdout:} 2020-12-05T17:34:22.875 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:34:22.875 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:34:22.876 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:34:22.882 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:34:22.883 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:34:22.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:34:22.884 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:34:45.046 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:34:45.048 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:34:45.051 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:34:45.053 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:34:45.056 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:35:10.095 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:35:10.096 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:35:10.121 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:35:10.151 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:35:10.181 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:35:10.215 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:35:10.243 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:35:10.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:10.244 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:35:10.244 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:35:10.259 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:35:10.260 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:35:10.261 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi036.front.sepia.ceph.com... 2020-12-05T17:35:10.261 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:35:10.262 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:35:10.351 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[10631]: fuse finished with error 0 and tester_r 0 2020-12-05T17:35:10.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:10.353 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:35:10.353 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:35:10.414 INFO:teuthology.orchestra.run.smithi036.stdout:ext2/ext3 2020-12-05T17:35:10.415 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:35:10.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:35:10.499 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:35:10.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:10.517 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:35:10.517 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:35:10.533 INFO:teuthology.orchestra.run.smithi002.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-12-05T17:35:10.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:35:10.536 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-12-05T17:35:10.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:35:10.537 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:35:10.596 INFO:teuthology.orchestra.run.smithi002.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-12-05T17:35:10.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:35:10.599 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:10.599 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:35:10.600 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:35:10.609 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:35:10.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:35:10.610 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2020-12-05T17:35:10.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:35:10.611 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:35:10.663 INFO:teuthology.orchestra.run.smithi053.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:35:10.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:35:10.664 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2020-12-05T17:35:10.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.803+0000 7f9745615700 1 Processor -- start 2020-12-05T17:35:10.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.803+0000 7f9745615700 1 -- start start 2020-12-05T17:35:10.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.803+0000 7f9745615700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f974005f3b0 0x7f974005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:10.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.803+0000 7f9745615700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f974005feb0 0x7f97400b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:10.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.803+0000 7f9745615700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9740142450 con 0x7f9740062400 2020-12-05T17:35:10.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.803+0000 7f9745615700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97400b42f0 con 0x7f974005feb0 2020-12-05T17:35:10.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.803+0000 7f9745615700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9740061780 con 0x7f974005f3b0 2020-12-05T17:35:10.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.803+0000 7f9744e14700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f974005feb0 0x7f97400b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:10.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.803+0000 7f973ffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f974005f3b0 0x7f974005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:10.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.803+0000 7f973f7fe700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f9740062400 0x7f97400b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:50224/0 (socket says 172.21.15.32:50224) 2020-12-05T17:35:10.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.803+0000 7f973f7fe700 1 -- 172.21.15.32:0/3540280923 learned_addr learned my addr 172.21.15.32:0/3540280923 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:35:10.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.803+0000 7f973ffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f974005f3b0 0x7f974005f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:59960/0 (socket says 172.21.15.32:59960) 2020-12-05T17:35:10.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.803+0000 7f9744e14700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f974005feb0 0x7f97400b3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:54832/0 (socket says 172.21.15.32:54832) 2020-12-05T17:35:10.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.807+0000 7f973effd700 1 -- 172.21.15.32:0/3540280923 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2462171486 0 0) 0x7f9740142450 con 0x7f9740062400 2020-12-05T17:35:10.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.807+0000 7f973effd700 1 -- 172.21.15.32:0/3540280923 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f972c003230 con 0x7f9740062400 2020-12-05T17:35:10.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.807+0000 7f9744e14700 1 -- 172.21.15.32:0/3540280923 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f974005f3b0 msgr2=0x7f974005f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:10.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.807+0000 7f9744e14700 1 --2- 172.21.15.32:0/3540280923 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f974005f3b0 0x7f974005f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:10.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.807+0000 7f9744e14700 1 -- 172.21.15.32:0/3540280923 >> v1:172.21.15.61:6789/0 conn(0x7f9740062400 legacy=0x7f97400b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:10.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.807+0000 7f9744e14700 1 -- 172.21.15.32:0/3540280923 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9740061a20 con 0x7f974005feb0 2020-12-05T17:35:10.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.807+0000 7f9744e14700 1 --2- 172.21.15.32:0/3540280923 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f974005feb0 0x7f97400b3e00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f9730002f30 tx=0x7f9730006dc0).ready entity=mon.1 client_cookie=44fc2c47fb8ef22c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:10.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.807+0000 7f973effd700 1 -- 172.21.15.32:0/3540280923 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97300096a0 con 0x7f974005feb0 2020-12-05T17:35:10.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.807+0000 7f973effd700 1 -- 172.21.15.32:0/3540280923 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9730009800 con 0x7f974005feb0 2020-12-05T17:35:10.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.807+0000 7f973effd700 1 -- 172.21.15.32:0/3540280923 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9730009b60 con 0x7f974005feb0 2020-12-05T17:35:10.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.807+0000 7f9745615700 1 -- 172.21.15.32:0/3540280923 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f974005feb0 msgr2=0x7f97400b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:10.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.807+0000 7f9745615700 1 --2- 172.21.15.32:0/3540280923 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f974005feb0 0x7f97400b3e00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f9730002f30 tx=0x7f9730006dc0).stop 2020-12-05T17:35:10.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.807+0000 7f9745615700 1 -- 172.21.15.32:0/3540280923 shutdown_connections 2020-12-05T17:35:10.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.807+0000 7f9745615700 1 --2- 172.21.15.32:0/3540280923 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f974005f3b0 0x7f974005f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:10.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.807+0000 7f9745615700 1 --2- 172.21.15.32:0/3540280923 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f974005feb0 0x7f97400b3e00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:10.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.807+0000 7f9745615700 1 -- 172.21.15.32:0/3540280923 >> 172.21.15.32:0/3540280923 conn(0x7f97400b8ee0 msgr2=0x7f97400b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:10.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.807+0000 7f9745615700 1 -- 172.21.15.32:0/3540280923 shutdown_connections 2020-12-05T17:35:10.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.807+0000 7f9745615700 1 -- 172.21.15.32:0/3540280923 wait complete. 2020-12-05T17:35:10.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.811+0000 7f9745615700 1 Processor -- start 2020-12-05T17:35:10.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.811+0000 7f9745615700 1 -- start start 2020-12-05T17:35:10.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.811+0000 7f9745615700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f974005f3b0 0x7f9740148040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:10.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.811+0000 7f9745615700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f974005feb0 0x7f9740146160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:10.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.811+0000 7f9745615700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9740062400 0x7f9740064960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:10.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.811+0000 7f9745615700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9740063b70 con 0x7f974005f3b0 2020-12-05T17:35:10.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.811+0000 7f9745615700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9740144cd0 con 0x7f974005feb0 2020-12-05T17:35:10.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.811+0000 7f9745615700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9740063040 con 0x7f9740062400 2020-12-05T17:35:10.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.811+0000 7f973ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f974005f3b0 0x7f9740148040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:10.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.811+0000 7f973ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f974005f3b0 0x7f9740148040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34942/0 (socket says 172.21.15.32:34942) 2020-12-05T17:35:10.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.811+0000 7f973ffff700 1 -- 172.21.15.32:0/3471423927 learned_addr learned my addr 172.21.15.32:0/3471423927 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:10.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.811+0000 7f973f7fe700 1 --2- 172.21.15.32:0/3471423927 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f974005feb0 0x7f9740146160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:10.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.811+0000 7f9744e14700 1 --2- 172.21.15.32:0/3471423927 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9740062400 0x7f9740064960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:10.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.811+0000 7f973ffff700 1 -- 172.21.15.32:0/3471423927 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9740062400 msgr2=0x7f9740064960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:10.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.811+0000 7f973ffff700 1 --2- 172.21.15.32:0/3471423927 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9740062400 0x7f9740064960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:10.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.811+0000 7f973ffff700 1 -- 172.21.15.32:0/3471423927 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f974005feb0 msgr2=0x7f9740146160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:10.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.811+0000 7f973ffff700 1 --2- 172.21.15.32:0/3471423927 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f974005feb0 0x7f9740146160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:10.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.811+0000 7f973ffff700 1 -- 172.21.15.32:0/3471423927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9740063200 con 0x7f974005f3b0 2020-12-05T17:35:10.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.815+0000 7f973ffff700 1 --2- 172.21.15.32:0/3471423927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f974005f3b0 0x7f9740148040 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f973800c000 tx=0x7f97380091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:10.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.815+0000 7f973d7fa700 1 -- 172.21.15.32:0/3471423927 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f973800cb00 con 0x7f974005f3b0 2020-12-05T17:35:10.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.815+0000 7f973d7fa700 1 -- 172.21.15.32:0/3471423927 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973800cc60 con 0x7f974005f3b0 2020-12-05T17:35:10.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.815+0000 7f9745615700 1 -- 172.21.15.32:0/3471423927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9740148530 con 0x7f974005f3b0 2020-12-05T17:35:10.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.815+0000 7f973d7fa700 1 -- 172.21.15.32:0/3471423927 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97380123f0 con 0x7f974005f3b0 2020-12-05T17:35:10.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.815+0000 7f9745615700 1 -- 172.21.15.32:0/3471423927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97401467f0 con 0x7f974005f3b0 2020-12-05T17:35:10.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.815+0000 7f9745615700 1 -- 172.21.15.32:0/3471423927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9740057050 con 0x7f974005f3b0 2020-12-05T17:35:10.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.815+0000 7f973d7fa700 1 -- 172.21.15.32:0/3471423927 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9738022070 con 0x7f974005f3b0 2020-12-05T17:35:10.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.815+0000 7f973d7fa700 1 --2- 172.21.15.32:0/3471423927 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9720055440 0x7f9720057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:10.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.815+0000 7f973d7fa700 1 -- 172.21.15.32:0/3471423927 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f973806e780 con 0x7f974005f3b0 2020-12-05T17:35:10.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.815+0000 7f973f7fe700 1 --2- 172.21.15.32:0/3471423927 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9720055440 0x7f9720057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:10.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.819+0000 7f973f7fe700 1 --2- 172.21.15.32:0/3471423927 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9720055440 0x7f9720057900 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f973400d000 tx=0x7f9734009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:10.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.823+0000 7f973d7fa700 1 -- 172.21.15.32:0/3471423927 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9738044880 con 0x7f974005f3b0 2020-12-05T17:35:10.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.971+0000 7f9745615700 1 -- 172.21.15.32:0/3471423927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f974005aab0 con 0x7f974005f3b0 2020-12-05T17:35:10.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.975+0000 7f973d7fa700 1 -- 172.21.15.32:0/3471423927 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9738047ea0 con 0x7f974005f3b0 2020-12-05T17:35:10.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.979+0000 7f9745615700 1 -- 172.21.15.32:0/3471423927 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9720055440 msgr2=0x7f9720057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.979+0000 7f9745615700 1 --2- 172.21.15.32:0/3471423927 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9720055440 0x7f9720057900 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f973400d000 tx=0x7f9734009040).stop 2020-12-05T17:35:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.979+0000 7f9745615700 1 -- 172.21.15.32:0/3471423927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f974005f3b0 msgr2=0x7f9740148040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.979+0000 7f9745615700 1 --2- 172.21.15.32:0/3471423927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f974005f3b0 0x7f9740148040 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f973800c000 tx=0x7f97380091a0).stop 2020-12-05T17:35:10.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.979+0000 7f9745615700 1 -- 172.21.15.32:0/3471423927 shutdown_connections 2020-12-05T17:35:10.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.979+0000 7f9745615700 1 --2- 172.21.15.32:0/3471423927 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9720055440 0x7f9720057900 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:10.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.979+0000 7f9745615700 1 --2- 172.21.15.32:0/3471423927 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9740062400 0x7f9740064960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:10.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.979+0000 7f9745615700 1 --2- 172.21.15.32:0/3471423927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f974005f3b0 0x7f9740148040 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:10.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.979+0000 7f9745615700 1 --2- 172.21.15.32:0/3471423927 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f974005feb0 0x7f9740146160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:10.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.979+0000 7f9745615700 1 -- 172.21.15.32:0/3471423927 >> 172.21.15.32:0/3471423927 conn(0x7f97400b8ee0 msgr2=0x7f97400b7630 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:10.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.979+0000 7f9745615700 1 -- 172.21.15.32:0/3471423927 shutdown_connections 2020-12-05T17:35:10.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:10.979+0000 7f9745615700 1 -- 172.21.15.32:0/3471423927 wait complete. 2020-12-05T17:35:10.989 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead' 2020-12-05T17:35:11.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.107+0000 7fd09984b700 1 Processor -- start 2020-12-05T17:35:11.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.107+0000 7fd09984b700 1 -- start start 2020-12-05T17:35:11.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.107+0000 7fd09984b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd0940b1fc0 0x7fd0940af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:11.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.107+0000 7fd09984b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd0940ab710 0x7fd094061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:11.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.107+0000 7fd09984b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd094147400 0x7fd094145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:11.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.107+0000 7fd09984b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0940b0110 con 0x7fd0940b1fc0 2020-12-05T17:35:11.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.107+0000 7fd09984b700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd094146a90 con 0x7fd0940ab710 2020-12-05T17:35:11.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.107+0000 7fd09984b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0940b23a0 con 0x7fd094147400 2020-12-05T17:35:11.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.107+0000 7fd098849700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd0940b1fc0 0x7fd0940af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:11.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.107+0000 7fd098849700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd0940b1fc0 0x7fd0940af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34952/0 (socket says 172.21.15.32:34952) 2020-12-05T17:35:11.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.107+0000 7fd098849700 1 -- 172.21.15.32:0/2424130094 learned_addr learned my addr 172.21.15.32:0/2424130094 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:11.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.107+0000 7fd093fff700 1 --2- 172.21.15.32:0/2424130094 >> v2:172.21.15.61:3300/0 conn(0x7fd0940ab710 0x7fd094061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:11.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.107+0000 7fd09904a700 1 --2- 172.21.15.32:0/2424130094 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd094147400 0x7fd094145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:11.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.111+0000 7fd098849700 1 -- 172.21.15.32:0/2424130094 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd094147400 msgr2=0x7fd094145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:11.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.111+0000 7fd098849700 1 --2- 172.21.15.32:0/2424130094 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd094147400 0x7fd094145d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:11.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.111+0000 7fd098849700 1 -- 172.21.15.32:0/2424130094 >> v2:172.21.15.61:3300/0 conn(0x7fd0940ab710 msgr2=0x7fd094061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:11.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.111+0000 7fd098849700 1 --2- 172.21.15.32:0/2424130094 >> v2:172.21.15.61:3300/0 conn(0x7fd0940ab710 0x7fd094061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:11.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.111+0000 7fd098849700 1 -- 172.21.15.32:0/2424130094 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd094146cc0 con 0x7fd0940b1fc0 2020-12-05T17:35:11.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.111+0000 7fd093fff700 1 --2- 172.21.15.32:0/2424130094 >> v2:172.21.15.61:3300/0 conn(0x7fd0940ab710 0x7fd094061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:11.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.111+0000 7fd09904a700 1 --2- 172.21.15.32:0/2424130094 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd094147400 0x7fd094145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:11.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.111+0000 7fd098849700 1 --2- 172.21.15.32:0/2424130094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0940b1fc0 0x7fd0940af230 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7fd084002f30 tx=0x7fd084006e80).ready entity=mon.0 client_cookie=430a9949cd82371c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:11.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.111+0000 7fd0937fe700 1 -- 172.21.15.32:0/2424130094 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0840097f0 con 0x7fd0940b1fc0 2020-12-05T17:35:11.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.111+0000 7fd0937fe700 1 -- 172.21.15.32:0/2424130094 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd084009950 con 0x7fd0940b1fc0 2020-12-05T17:35:11.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.111+0000 7fd0937fe700 1 -- 172.21.15.32:0/2424130094 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd084009cb0 con 0x7fd0940b1fc0 2020-12-05T17:35:11.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.111+0000 7fd09984b700 1 -- 172.21.15.32:0/2424130094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0940b1fc0 msgr2=0x7fd0940af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:11.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.111+0000 7fd09984b700 1 --2- 172.21.15.32:0/2424130094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0940b1fc0 0x7fd0940af230 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7fd084002f30 tx=0x7fd084006e80).stop 2020-12-05T17:35:11.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.111+0000 7fd09984b700 1 -- 172.21.15.32:0/2424130094 shutdown_connections 2020-12-05T17:35:11.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.111+0000 7fd09984b700 1 --2- 172.21.15.32:0/2424130094 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd094147400 0x7fd094145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:11.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.111+0000 7fd09984b700 1 --2- 172.21.15.32:0/2424130094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0940b1fc0 0x7fd0940af230 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:11.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.111+0000 7fd09984b700 1 --2- 172.21.15.32:0/2424130094 >> v2:172.21.15.61:3300/0 conn(0x7fd0940ab710 0x7fd094061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:11.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.111+0000 7fd09984b700 1 -- 172.21.15.32:0/2424130094 >> 172.21.15.32:0/2424130094 conn(0x7fd0940b5530 msgr2=0x7fd0940b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:11.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.111+0000 7fd09984b700 1 -- 172.21.15.32:0/2424130094 shutdown_connections 2020-12-05T17:35:11.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.111+0000 7fd09984b700 1 -- 172.21.15.32:0/2424130094 wait complete. 2020-12-05T17:35:11.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.111+0000 7fd09984b700 1 Processor -- start 2020-12-05T17:35:11.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.111+0000 7fd09984b700 1 -- start start 2020-12-05T17:35:11.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.115+0000 7fd09984b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0940ab710 0x7fd0940ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:11.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.115+0000 7fd09984b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0940b1fc0 0x7fd0940b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:11.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.115+0000 7fd09984b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd094147400 0x7fd09414d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:11.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.115+0000 7fd09984b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0940ade60 con 0x7fd0940ab710 2020-12-05T17:35:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.115+0000 7fd09984b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0940ae100 con 0x7fd0940b1fc0 2020-12-05T17:35:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.115+0000 7fd09984b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0940ae3a0 con 0x7fd094147400 2020-12-05T17:35:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.115+0000 7fd098849700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0940ab710 0x7fd0940ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:11.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.115+0000 7fd098849700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0940ab710 0x7fd0940ae9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34956/0 (socket says 172.21.15.32:34956) 2020-12-05T17:35:11.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.115+0000 7fd098849700 1 -- 172.21.15.32:0/972230744 learned_addr learned my addr 172.21.15.32:0/972230744 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:11.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.115+0000 7fd093fff700 1 --2- 172.21.15.32:0/972230744 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0940b1fc0 0x7fd0940b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:11.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.115+0000 7fd09904a700 1 --2- 172.21.15.32:0/972230744 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd094147400 0x7fd09414d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:11.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.115+0000 7fd098849700 1 -- 172.21.15.32:0/972230744 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd094147400 msgr2=0x7fd09414d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.115+0000 7fd098849700 1 --2- 172.21.15.32:0/972230744 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd094147400 0x7fd09414d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.115+0000 7fd098849700 1 -- 172.21.15.32:0/972230744 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0940b1fc0 msgr2=0x7fd0940b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.115+0000 7fd098849700 1 --2- 172.21.15.32:0/972230744 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0940b1fc0 0x7fd0940b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.115+0000 7fd098849700 1 -- 172.21.15.32:0/972230744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0940acd40 con 0x7fd0940ab710 2020-12-05T17:35:11.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.115+0000 7fd098849700 1 --2- 172.21.15.32:0/972230744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0940ab710 0x7fd0940ae9b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7fd084002f30 tx=0x7fd084015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:11.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.115+0000 7fd091ffb700 1 -- 172.21.15.32:0/972230744 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd084011b40 con 0x7fd0940ab710 2020-12-05T17:35:11.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.115+0000 7fd091ffb700 1 -- 172.21.15.32:0/972230744 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd084011ca0 con 0x7fd0940ab710 2020-12-05T17:35:11.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.115+0000 7fd091ffb700 1 -- 172.21.15.32:0/972230744 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd08401e460 con 0x7fd0940ab710 2020-12-05T17:35:11.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.115+0000 7fd09984b700 1 -- 172.21.15.32:0/972230744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0940acf90 con 0x7fd0940ab710 2020-12-05T17:35:11.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.115+0000 7fd09984b700 1 -- 172.21.15.32:0/972230744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd09414ce00 con 0x7fd0940ab710 2020-12-05T17:35:11.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.115+0000 7fd091ffb700 1 -- 172.21.15.32:0/972230744 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd08401a040 con 0x7fd0940ab710 2020-12-05T17:35:11.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.119+0000 7fd091ffb700 1 --2- 172.21.15.32:0/972230744 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd078055440 0x7fd078057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:11.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.119+0000 7fd093fff700 1 --2- 172.21.15.32:0/972230744 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd078055440 0x7fd078057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:11.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.119+0000 7fd091ffb700 1 -- 172.21.15.32:0/972230744 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd08406f070 con 0x7fd0940ab710 2020-12-05T17:35:11.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.119+0000 7fd09984b700 1 -- 172.21.15.32:0/972230744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0940ae560 con 0x7fd0940ab710 2020-12-05T17:35:11.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.119+0000 7fd093fff700 1 --2- 172.21.15.32:0/972230744 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd078055440 0x7fd078057900 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fd07c00c000 tx=0x7fd07c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:11.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.123+0000 7fd091ffb700 1 -- 172.21.15.32:0/972230744 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd084023020 con 0x7fd0940ab710 2020-12-05T17:35:11.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.315+0000 7fd09984b700 1 -- 172.21.15.32:0/972230744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]} v 0) v1 -- 0x7fd094057050 con 0x7fd0940ab710 2020-12-05T17:35:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.779+0000 7fd091ffb700 1 -- 172.21.15.32:0/972230744 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]}]=0 v162) v1 ==== 144+0+0 (secure 0 0 0) 0x7fd084023030 con 0x7fd0940ab710 2020-12-05T17:35:11.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.779+0000 7fd09984b700 1 -- 172.21.15.32:0/972230744 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd078055440 msgr2=0x7fd078057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:11.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.779+0000 7fd09984b700 1 --2- 172.21.15.32:0/972230744 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd078055440 0x7fd078057900 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fd07c00c000 tx=0x7fd07c008040).stop 2020-12-05T17:35:11.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.779+0000 7fd09984b700 1 -- 172.21.15.32:0/972230744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0940ab710 msgr2=0x7fd0940ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:11.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.779+0000 7fd09984b700 1 --2- 172.21.15.32:0/972230744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0940ab710 0x7fd0940ae9b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7fd084002f30 tx=0x7fd084015000).stop 2020-12-05T17:35:11.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.779+0000 7fd09984b700 1 -- 172.21.15.32:0/972230744 shutdown_connections 2020-12-05T17:35:11.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.779+0000 7fd09984b700 1 --2- 172.21.15.32:0/972230744 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd078055440 0x7fd078057900 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:11.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.779+0000 7fd09984b700 1 --2- 172.21.15.32:0/972230744 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd094147400 0x7fd09414d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:11.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.779+0000 7fd09984b700 1 --2- 172.21.15.32:0/972230744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0940ab710 0x7fd0940ae9b0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:11.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.779+0000 7fd09984b700 1 --2- 172.21.15.32:0/972230744 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0940b1fc0 0x7fd0940b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:11.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.783+0000 7fd09984b700 1 -- 172.21.15.32:0/972230744 >> 172.21.15.32:0/972230744 conn(0x7fd0940b5530 msgr2=0x7fd0940af090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:11.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.783+0000 7fd09984b700 1 -- 172.21.15.32:0/972230744 shutdown_connections 2020-12-05T17:35:11.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.783+0000 7fd09984b700 1 -- 172.21.15.32:0/972230744 wait complete. 2020-12-05T17:35:11.795 INFO:tasks.cephfs_test_runner:test_drop_cache_command_dead (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2020-12-05T17:35:11.796 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_dead_timeout (tasks.cephfs.test_misc.TestCacheDrop) 2020-12-05T17:35:11.796 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout' 2020-12-05T17:35:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.915+0000 7fdabbfff700 1 Processor -- start 2020-12-05T17:35:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.915+0000 7fdabbfff700 1 -- start start 2020-12-05T17:35:11.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.915+0000 7fdabbfff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fdabc062400 0x7fdabc05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:11.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.915+0000 7fdabbfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdabc05feb0 0x7fdabc0b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:11.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.915+0000 7fdabbfff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdabc142450 con 0x7fdabc05f3b0 2020-12-05T17:35:11.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.915+0000 7fdabbfff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdabc0b42f0 con 0x7fdabc062400 2020-12-05T17:35:11.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.915+0000 7fdabbfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc061780 con 0x7fdabc05feb0 2020-12-05T17:35:11.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.919+0000 7fdaba7fc700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fdabc062400 0x7fdabc05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:11.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.919+0000 7fdabb7fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdabc05feb0 0x7fdabc0b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:11.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.919+0000 7fdabb7fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdabc05feb0 0x7fdabc0b3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:59990/0 (socket says 172.21.15.32:59990) 2020-12-05T17:35:11.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.919+0000 7fdabb7fe700 1 -- 172.21.15.32:0/2668905226 learned_addr learned my addr 172.21.15.32:0/2668905226 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:11.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.919+0000 7fdabaffd700 1 --1- 172.21.15.32:0/2668905226 >> v1:172.21.15.61:6789/0 conn(0x7fdabc05f3b0 0x7fdabc0b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:50252/0 (socket says 172.21.15.32:50252) 2020-12-05T17:35:11.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.919+0000 7fdabb7fe700 1 -- 172.21.15.32:0/2668905226 >> v2:172.21.15.61:3300/0 conn(0x7fdabc062400 msgr2=0x7fdabc05f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.919+0000 7fdabb7fe700 1 --2- 172.21.15.32:0/2668905226 >> v2:172.21.15.61:3300/0 conn(0x7fdabc062400 0x7fdabc05f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.919+0000 7fdabb7fe700 1 -- 172.21.15.32:0/2668905226 >> v1:172.21.15.61:6789/0 conn(0x7fdabc05f3b0 legacy=0x7fdabc0b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.919+0000 7fdabb7fe700 1 -- 172.21.15.32:0/2668905226 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdabc061a20 con 0x7fdabc05feb0 2020-12-05T17:35:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.919+0000 7fdabb7fe700 1 --2- 172.21.15.32:0/2668905226 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdabc05feb0 0x7fdabc0b3e00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fdab40032b0 tx=0x7fdab4009270).ready entity=mon.2 client_cookie=9d0aad7c5bdf481e server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.919+0000 7fdab9ffb700 1 -- 172.21.15.32:0/2668905226 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdab4009e60 con 0x7fdabc05feb0 2020-12-05T17:35:11.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.919+0000 7fdab9ffb700 1 -- 172.21.15.32:0/2668905226 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab400bc00 con 0x7fdabc05feb0 2020-12-05T17:35:11.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.919+0000 7fdab9ffb700 1 -- 172.21.15.32:0/2668905226 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdab400bd70 con 0x7fdabc05feb0 2020-12-05T17:35:11.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.919+0000 7fdabbfff700 1 -- 172.21.15.32:0/2668905226 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdabc05feb0 msgr2=0x7fdabc0b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:11.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.919+0000 7fdabbfff700 1 --2- 172.21.15.32:0/2668905226 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdabc05feb0 0x7fdabc0b3e00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fdab40032b0 tx=0x7fdab4009270).stop 2020-12-05T17:35:11.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.919+0000 7fdabbfff700 1 -- 172.21.15.32:0/2668905226 shutdown_connections 2020-12-05T17:35:11.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.919+0000 7fdabbfff700 1 --2- 172.21.15.32:0/2668905226 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdabc05feb0 0x7fdabc0b3e00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:11.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.919+0000 7fdabbfff700 1 --2- 172.21.15.32:0/2668905226 >> v2:172.21.15.61:3300/0 conn(0x7fdabc062400 0x7fdabc05f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:11.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.919+0000 7fdabbfff700 1 -- 172.21.15.32:0/2668905226 >> 172.21.15.32:0/2668905226 conn(0x7fdabc0b8ee0 msgr2=0x7fdabc0b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:11.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.919+0000 7fdabbfff700 1 -- 172.21.15.32:0/2668905226 shutdown_connections 2020-12-05T17:35:11.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.919+0000 7fdabbfff700 1 -- 172.21.15.32:0/2668905226 wait complete. 2020-12-05T17:35:11.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.919+0000 7fdabbfff700 1 Processor -- start 2020-12-05T17:35:11.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.919+0000 7fdabbfff700 1 -- start start 2020-12-05T17:35:11.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdabbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc05f3b0 0x7fdabc1483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:11.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdabbfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdabc05feb0 0x7fdabc146060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:11.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdabbfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdabc062400 0x7fdabc154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:11.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdabbfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc0630a0 con 0x7fdabc05f3b0 2020-12-05T17:35:11.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdabbfff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc063300 con 0x7fdabc062400 2020-12-05T17:35:11.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdabbfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc0635a0 con 0x7fdabc05feb0 2020-12-05T17:35:11.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdabaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc05f3b0 0x7fdabc1483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:11.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdabaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc05f3b0 0x7fdabc1483b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34970/0 (socket says 172.21.15.32:34970) 2020-12-05T17:35:11.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdabaffd700 1 -- 172.21.15.32:0/1529394534 learned_addr learned my addr 172.21.15.32:0/1529394534 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:11.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdabb7fe700 1 --2- 172.21.15.32:0/1529394534 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdabc062400 0x7fdabc154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:11.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdaba7fc700 1 --2- 172.21.15.32:0/1529394534 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdabc05feb0 0x7fdabc146060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:11.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdabaffd700 1 -- 172.21.15.32:0/1529394534 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdabc05feb0 msgr2=0x7fdabc146060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:11.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdabaffd700 1 --2- 172.21.15.32:0/1529394534 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdabc05feb0 0x7fdabc146060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:11.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdabaffd700 1 -- 172.21.15.32:0/1529394534 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdabc062400 msgr2=0x7fdabc154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:11.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdabaffd700 1 --2- 172.21.15.32:0/1529394534 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdabc062400 0x7fdabc154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:11.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdabaffd700 1 -- 172.21.15.32:0/1529394534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdabc146650 con 0x7fdabc05f3b0 2020-12-05T17:35:11.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdaba7fc700 1 --2- 172.21.15.32:0/1529394534 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdabc05feb0 0x7fdabc146060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:11.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdabaffd700 1 --2- 172.21.15.32:0/1529394534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc05f3b0 0x7fdabc1483b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7fdabc05c5c0 tx=0x7fdab0007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:11.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdaabfff700 1 -- 172.21.15.32:0/1529394534 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdab00072e0 con 0x7fdabc05f3b0 2020-12-05T17:35:11.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdaabfff700 1 -- 172.21.15.32:0/1529394534 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab0005520 con 0x7fdabc05f3b0 2020-12-05T17:35:11.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdabbfff700 1 -- 172.21.15.32:0/1529394534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdabc1468a0 con 0x7fdabc05f3b0 2020-12-05T17:35:11.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdaabfff700 1 -- 172.21.15.32:0/1529394534 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdab0005880 con 0x7fdabc05f3b0 2020-12-05T17:35:11.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdabbfff700 1 -- 172.21.15.32:0/1529394534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdabc154f10 con 0x7fdabc05f3b0 2020-12-05T17:35:11.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.923+0000 7fdabbfff700 1 -- 172.21.15.32:0/1529394534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdabc057050 con 0x7fdabc05f3b0 2020-12-05T17:35:11.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.927+0000 7fdaabfff700 1 -- 172.21.15.32:0/1529394534 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fdab0010070 con 0x7fdabc05f3b0 2020-12-05T17:35:11.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.927+0000 7fdaabfff700 1 --2- 172.21.15.32:0/1529394534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fda98055440 0x7fda98057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:11.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.927+0000 7fdaabfff700 1 -- 172.21.15.32:0/1529394534 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdab006d610 con 0x7fdabc05f3b0 2020-12-05T17:35:11.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.927+0000 7fdaba7fc700 1 --2- 172.21.15.32:0/1529394534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fda98055440 0x7fda98057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:11.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.927+0000 7fdaba7fc700 1 --2- 172.21.15.32:0/1529394534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fda98055440 0x7fda98057900 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fdaac00b000 tx=0x7fdaac008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:11.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:11.935+0000 7fdaabfff700 1 -- 172.21.15.32:0/1529394534 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fdab00437b0 con 0x7fdabc05f3b0 2020-12-05T17:35:12.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.123+0000 7fdabbfff700 1 -- 172.21.15.32:0/1529394534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]} v 0) v1 -- 0x7fdabc05aab0 con 0x7fdabc05f3b0 2020-12-05T17:35:12.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.783+0000 7fdaabfff700 1 -- 172.21.15.32:0/1529394534 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]}]=0 v163) v1 ==== 155+0+0 (secure 0 0 0) 0x7fdab0046dd0 con 0x7fdabc05f3b0 2020-12-05T17:35:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.787+0000 7fdabbfff700 1 -- 172.21.15.32:0/1529394534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fda98055440 msgr2=0x7fda98057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.787+0000 7fdabbfff700 1 --2- 172.21.15.32:0/1529394534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fda98055440 0x7fda98057900 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fdaac00b000 tx=0x7fdaac008040).stop 2020-12-05T17:35:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.787+0000 7fdabbfff700 1 -- 172.21.15.32:0/1529394534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc05f3b0 msgr2=0x7fdabc1483b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.787+0000 7fdabbfff700 1 --2- 172.21.15.32:0/1529394534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc05f3b0 0x7fdabc1483b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7fdabc05c5c0 tx=0x7fdab0007c10).stop 2020-12-05T17:35:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.787+0000 7fdabbfff700 1 -- 172.21.15.32:0/1529394534 shutdown_connections 2020-12-05T17:35:12.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.787+0000 7fdabbfff700 1 --2- 172.21.15.32:0/1529394534 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdabc062400 0x7fdabc154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:12.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.787+0000 7fdabbfff700 1 --2- 172.21.15.32:0/1529394534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc05f3b0 0x7fdabc1483b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:12.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.787+0000 7fdabbfff700 1 --2- 172.21.15.32:0/1529394534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fda98055440 0x7fda98057900 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:12.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.787+0000 7fdabbfff700 1 --2- 172.21.15.32:0/1529394534 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdabc05feb0 0x7fdabc146060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:12.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.787+0000 7fdabbfff700 1 -- 172.21.15.32:0/1529394534 >> 172.21.15.32:0/1529394534 conn(0x7fdabc0b8ee0 msgr2=0x7fdabc0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:12.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.787+0000 7fdabbfff700 1 -- 172.21.15.32:0/1529394534 shutdown_connections 2020-12-05T17:35:12.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.787+0000 7fdabbfff700 1 -- 172.21.15.32:0/1529394534 wait complete. 2020-12-05T17:35:12.804 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-12-05T17:35:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.927+0000 7f8f37fff700 1 Processor -- start 2020-12-05T17:35:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.927+0000 7f8f37fff700 1 -- start start 2020-12-05T17:35:12.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.927+0000 7f8f37fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f380b1dc0 0x7f8f380b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:12.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.927+0000 7f8f37fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f381468e0 con 0x7f8f380af5f0 2020-12-05T17:35:12.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.927+0000 7f8f37fff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f38146b10 con 0x7f8f380afd30 2020-12-05T17:35:12.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.927+0000 7f8f37fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f38146db0 con 0x7f8f380b1dc0 2020-12-05T17:35:12.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.927+0000 7f8f36ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f380b1dc0 0x7f8f380b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:12.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.927+0000 7f8f36ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f380b1dc0 0x7f8f380b21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34978/0 (socket says 172.21.15.32:34978) 2020-12-05T17:35:12.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.927+0000 7f8f36ffd700 1 -- 172.21.15.32:0/926153966 learned_addr learned my addr 172.21.15.32:0/926153966 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:12.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.927+0000 7f8f36ffd700 1 -- 172.21.15.32:0/926153966 >> v1:172.21.15.61:6789/0 conn(0x7f8f380afd30 legacy=0x7f8f380ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:12.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.927+0000 7f8f36ffd700 1 -- 172.21.15.32:0/926153966 >> v1:172.21.15.32:6789/0 conn(0x7f8f380af5f0 legacy=0x7f8f38141c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:12.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.927+0000 7f8f36ffd700 1 -- 172.21.15.32:0/926153966 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f380ae160 con 0x7f8f380b1dc0 2020-12-05T17:35:12.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.927+0000 7f8f36ffd700 1 --2- 172.21.15.32:0/926153966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f380b1dc0 0x7f8f380b21a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f8f20002f30 tx=0x7f8f20006e80).ready entity=mon.0 client_cookie=37f470d2c26552b0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:12.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.927+0000 7f8f35ffb700 1 -- 172.21.15.32:0/926153966 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f200097f0 con 0x7f8f380b1dc0 2020-12-05T17:35:12.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.927+0000 7f8f35ffb700 1 -- 172.21.15.32:0/926153966 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f20009950 con 0x7f8f380b1dc0 2020-12-05T17:35:12.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.927+0000 7f8f35ffb700 1 -- 172.21.15.32:0/926153966 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f20009ac0 con 0x7f8f380b1dc0 2020-12-05T17:35:12.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.927+0000 7f8f37fff700 1 -- 172.21.15.32:0/926153966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f380b1dc0 msgr2=0x7f8f380b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:12.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.927+0000 7f8f37fff700 1 --2- 172.21.15.32:0/926153966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f380b1dc0 0x7f8f380b21a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f8f20002f30 tx=0x7f8f20006e80).stop 2020-12-05T17:35:12.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.931+0000 7f8f37fff700 1 -- 172.21.15.32:0/926153966 shutdown_connections 2020-12-05T17:35:12.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.931+0000 7f8f37fff700 1 --2- 172.21.15.32:0/926153966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f380b1dc0 0x7f8f380b21a0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:12.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.931+0000 7f8f37fff700 1 -- 172.21.15.32:0/926153966 >> 172.21.15.32:0/926153966 conn(0x7f8f380620b0 msgr2=0x7f8f38062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:12.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.931+0000 7f8f37fff700 1 -- 172.21.15.32:0/926153966 shutdown_connections 2020-12-05T17:35:12.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.931+0000 7f8f37fff700 1 -- 172.21.15.32:0/926153966 wait complete. 2020-12-05T17:35:12.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.931+0000 7f8f37fff700 1 Processor -- start 2020-12-05T17:35:12.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.931+0000 7f8f37fff700 1 -- start start 2020-12-05T17:35:12.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.935+0000 7f8f37fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8f380af5f0 0x7f8f38066a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:12.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.935+0000 7f8f37fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f380afd30 0x7f8f380b0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:12.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.935+0000 7f8f37fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8f380b1dc0 0x7f8f380656f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:12.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.935+0000 7f8f37fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f38146020 con 0x7f8f380afd30 2020-12-05T17:35:12.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.935+0000 7f8f37fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f380b1390 con 0x7f8f380b1dc0 2020-12-05T17:35:12.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.935+0000 7f8f37fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f38146250 con 0x7f8f380af5f0 2020-12-05T17:35:12.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.935+0000 7f8f367fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f380afd30 0x7f8f380b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:12.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.935+0000 7f8f367fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f380afd30 0x7f8f380b0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34986/0 (socket says 172.21.15.32:34986) 2020-12-05T17:35:12.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.935+0000 7f8f367fc700 1 -- 172.21.15.32:0/1175871899 learned_addr learned my addr 172.21.15.32:0/1175871899 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:12.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.935+0000 7f8f36ffd700 1 --2- 172.21.15.32:0/1175871899 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8f380af5f0 0x7f8f38066a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:12.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.935+0000 7f8f377fe700 1 --2- 172.21.15.32:0/1175871899 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8f380b1dc0 0x7f8f380656f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:12.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.935+0000 7f8f367fc700 1 -- 172.21.15.32:0/1175871899 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8f380af5f0 msgr2=0x7f8f38066a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:12.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.935+0000 7f8f367fc700 1 --2- 172.21.15.32:0/1175871899 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8f380af5f0 0x7f8f38066a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:12.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.935+0000 7f8f367fc700 1 -- 172.21.15.32:0/1175871899 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8f380b1dc0 msgr2=0x7f8f380656f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:12.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.935+0000 7f8f367fc700 1 --2- 172.21.15.32:0/1175871899 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8f380b1dc0 0x7f8f380656f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:12.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.935+0000 7f8f367fc700 1 -- 172.21.15.32:0/1175871899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f38063f10 con 0x7f8f380afd30 2020-12-05T17:35:12.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.935+0000 7f8f367fc700 1 --2- 172.21.15.32:0/1175871899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f380afd30 0x7f8f380b0690 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f8f3805c820 tx=0x7f8f30007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:12.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.935+0000 7f8f2ffff700 1 -- 172.21.15.32:0/1175871899 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f3000e4a0 con 0x7f8f380afd30 2020-12-05T17:35:12.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.935+0000 7f8f2ffff700 1 -- 172.21.15.32:0/1175871899 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3000e600 con 0x7f8f380afd30 2020-12-05T17:35:12.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.935+0000 7f8f37fff700 1 -- 172.21.15.32:0/1175871899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f380641b0 con 0x7f8f380afd30 2020-12-05T17:35:12.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.935+0000 7f8f2ffff700 1 -- 172.21.15.32:0/1175871899 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f3000e960 con 0x7f8f380afd30 2020-12-05T17:35:12.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.935+0000 7f8f37fff700 1 -- 172.21.15.32:0/1175871899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f38145500 con 0x7f8f380afd30 2020-12-05T17:35:12.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.939+0000 7f8f37fff700 1 -- 172.21.15.32:0/1175871899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f380b04d0 con 0x7f8f380afd30 2020-12-05T17:35:12.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.939+0000 7f8f2ffff700 1 -- 172.21.15.32:0/1175871899 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8f30010070 con 0x7f8f380afd30 2020-12-05T17:35:12.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.939+0000 7f8f2ffff700 1 --2- 172.21.15.32:0/1175871899 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8f10055330 0x7f8f100577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:12.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.939+0000 7f8f36ffd700 1 --2- 172.21.15.32:0/1175871899 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8f10055330 0x7f8f100577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:12.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.939+0000 7f8f2ffff700 1 -- 172.21.15.32:0/1175871899 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8f3006d420 con 0x7f8f380afd30 2020-12-05T17:35:12.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.939+0000 7f8f36ffd700 1 --2- 172.21.15.32:0/1175871899 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8f10055330 0x7f8f100577f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f8f2001a000 tx=0x7f8f20016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:12.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:12.943+0000 7f8f2ffff700 1 -- 172.21.15.32:0/1175871899 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8f300434b0 con 0x7f8f380afd30 2020-12-05T17:35:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.091+0000 7f8f37fff700 1 -- 172.21.15.32:0/1175871899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f8f38064340 con 0x7f8f380afd30 2020-12-05T17:35:13.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.099+0000 7f8f2ffff700 1 -- 172.21.15.32:0/1175871899 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8f30046ad0 con 0x7f8f380afd30 2020-12-05T17:35:13.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.099+0000 7f8f37fff700 1 -- 172.21.15.32:0/1175871899 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8f10055330 msgr2=0x7f8f100577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:13.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.099+0000 7f8f37fff700 1 --2- 172.21.15.32:0/1175871899 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8f10055330 0x7f8f100577f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f8f2001a000 tx=0x7f8f20016000).stop 2020-12-05T17:35:13.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.099+0000 7f8f37fff700 1 -- 172.21.15.32:0/1175871899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f380afd30 msgr2=0x7f8f380b0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:13.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.099+0000 7f8f37fff700 1 --2- 172.21.15.32:0/1175871899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f380afd30 0x7f8f380b0690 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f8f3805c820 tx=0x7f8f30007c20).stop 2020-12-05T17:35:13.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.099+0000 7f8f37fff700 1 -- 172.21.15.32:0/1175871899 shutdown_connections 2020-12-05T17:35:13.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.099+0000 7f8f37fff700 1 --2- 172.21.15.32:0/1175871899 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8f380b1dc0 0x7f8f380656f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:13.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.099+0000 7f8f37fff700 1 --2- 172.21.15.32:0/1175871899 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8f10055330 0x7f8f100577f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:13.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.099+0000 7f8f37fff700 1 --2- 172.21.15.32:0/1175871899 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8f380af5f0 0x7f8f38066a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:13.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.099+0000 7f8f37fff700 1 --2- 172.21.15.32:0/1175871899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f380afd30 0x7f8f380b0690 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:13.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.099+0000 7f8f37fff700 1 -- 172.21.15.32:0/1175871899 >> 172.21.15.32:0/1175871899 conn(0x7f8f380620b0 msgr2=0x7f8f380ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:13.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.099+0000 7f8f37fff700 1 -- 172.21.15.32:0/1175871899 shutdown_connections 2020-12-05T17:35:13.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.099+0000 7f8f37fff700 1 -- 172.21.15.32:0/1175871899 wait complete. 2020-12-05T17:35:13.117 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:35:13.118 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:35:13.134 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:35:13.150 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:35:13.167 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:35:13.184 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:35:13.201 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:35:13.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:13.202 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:35:13.203 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:35:13.216 INFO:teuthology.orchestra.run.smithi036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-12-05T17:35:13.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:35:13.217 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-12-05T17:35:13.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:13.218 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:35:13.219 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:35:13.225 INFO:teuthology.orchestra.run.smithi002.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-12-05T17:35:13.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:35:13.227 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-12-05T17:35:13.227 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:13.227 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:35:13.228 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:35:13.234 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:35:13.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:35:13.236 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2020-12-05T17:35:13.236 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-12-05T17:35:13.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.355+0000 7f650e13e700 1 Processor -- start 2020-12-05T17:35:13.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f650e13e700 1 -- start start 2020-12-05T17:35:13.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f650e13e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6508144540 0x7f6508061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:13.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f650e13e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6508143a00 con 0x7f65081403f0 2020-12-05T17:35:13.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f650e13e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65080b7b00 con 0x7f65080b7720 2020-12-05T17:35:13.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f650e13e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6508140880 con 0x7f6508144540 2020-12-05T17:35:13.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f650d93d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f65081403f0 0x7f650813d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37644/0 (socket says 172.21.15.32:37644) 2020-12-05T17:35:13.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f650d93d700 1 -- 172.21.15.32:0/390469240 learned_addr learned my addr 172.21.15.32:0/390469240 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:35:13.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f650c93b700 1 --2- 172.21.15.32:0/390469240 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6508144540 0x7f6508061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:13.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f64fbfff700 1 -- 172.21.15.32:0/390469240 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3170947832 0 0) 0x7f6508143a00 con 0x7f65081403f0 2020-12-05T17:35:13.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f64fbfff700 1 -- 172.21.15.32:0/390469240 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64ec003230 con 0x7f65081403f0 2020-12-05T17:35:13.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f64fbfff700 1 -- 172.21.15.32:0/390469240 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3098076392 0 0) 0x7f65080b7b00 con 0x7f65080b7720 2020-12-05T17:35:13.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f64fbfff700 1 -- 172.21.15.32:0/390469240 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6508143a00 con 0x7f65080b7720 2020-12-05T17:35:13.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f650c93b700 1 -- 172.21.15.32:0/390469240 >> v1:172.21.15.61:6789/0 conn(0x7f65080b7720 legacy=0x7f6508143e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:13.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f650c93b700 1 -- 172.21.15.32:0/390469240 >> v1:172.21.15.32:6789/0 conn(0x7f65081403f0 legacy=0x7f650813d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:13.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f650c93b700 1 -- 172.21.15.32:0/390469240 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650805fe60 con 0x7f6508144540 2020-12-05T17:35:13.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f650c93b700 1 --2- 172.21.15.32:0/390469240 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6508144540 0x7f6508061780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f64fc002f30 tx=0x7f64fc006dc0).ready entity=mon.2 client_cookie=e5458fbcdeacbf4c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:13.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f64fbfff700 1 -- 172.21.15.32:0/390469240 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64fc009730 con 0x7f6508144540 2020-12-05T17:35:13.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f64fbfff700 1 -- 172.21.15.32:0/390469240 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64fc009890 con 0x7f6508144540 2020-12-05T17:35:13.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f64fbfff700 1 -- 172.21.15.32:0/390469240 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64fc009bf0 con 0x7f6508144540 2020-12-05T17:35:13.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f650e13e700 1 -- 172.21.15.32:0/390469240 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6508144540 msgr2=0x7f6508061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:13.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f650e13e700 1 --2- 172.21.15.32:0/390469240 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6508144540 0x7f6508061780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f64fc002f30 tx=0x7f64fc006dc0).stop 2020-12-05T17:35:13.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f650e13e700 1 -- 172.21.15.32:0/390469240 shutdown_connections 2020-12-05T17:35:13.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f650e13e700 1 --2- 172.21.15.32:0/390469240 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6508144540 0x7f6508061780 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:13.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f650e13e700 1 -- 172.21.15.32:0/390469240 >> 172.21.15.32:0/390469240 conn(0x7f650805bc20 msgr2=0x7f65080b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:13.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f650e13e700 1 -- 172.21.15.32:0/390469240 shutdown_connections 2020-12-05T17:35:13.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.359+0000 7f650e13e700 1 -- 172.21.15.32:0/390469240 wait complete. 2020-12-05T17:35:13.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.363+0000 7f650e13e700 1 Processor -- start 2020-12-05T17:35:13.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.363+0000 7f650e13e700 1 -- start start 2020-12-05T17:35:13.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.367+0000 7f650e13e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65080b7720 0x7f6508146e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:13.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.367+0000 7f650e13e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65081403f0 0x7f6508063db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:13.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.367+0000 7f650e13e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6508144540 0x7f65080642a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:13.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.367+0000 7f650e13e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6508111660 con 0x7f65080b7720 2020-12-05T17:35:13.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.367+0000 7f650e13e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6508110530 con 0x7f65081403f0 2020-12-05T17:35:13.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.367+0000 7f650e13e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65080632e0 con 0x7f6508144540 2020-12-05T17:35:13.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.367+0000 7f650d13c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65080b7720 0x7f6508146e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.367+0000 7f650d13c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65080b7720 0x7f6508146e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34998/0 (socket says 172.21.15.32:34998) 2020-12-05T17:35:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.367+0000 7f650d13c700 1 -- 172.21.15.32:0/208309906 learned_addr learned my addr 172.21.15.32:0/208309906 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.367+0000 7f650c93b700 1 --2- 172.21.15.32:0/208309906 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65081403f0 0x7f6508063db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.367+0000 7f650d93d700 1 --2- 172.21.15.32:0/208309906 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6508144540 0x7f65080642a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.367+0000 7f650d13c700 1 -- 172.21.15.32:0/208309906 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6508144540 msgr2=0x7f65080642a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:13.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.367+0000 7f650d13c700 1 --2- 172.21.15.32:0/208309906 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6508144540 0x7f65080642a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:13.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.367+0000 7f650d13c700 1 -- 172.21.15.32:0/208309906 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65081403f0 msgr2=0x7f6508063db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:13.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.367+0000 7f650d13c700 1 --2- 172.21.15.32:0/208309906 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65081403f0 0x7f6508063db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:13.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.367+0000 7f650d13c700 1 -- 172.21.15.32:0/208309906 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65080b3460 con 0x7f65080b7720 2020-12-05T17:35:13.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.367+0000 7f650d13c700 1 --2- 172.21.15.32:0/208309906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65080b7720 0x7f6508146e90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f65040010d0 tx=0x7f6504007af0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:13.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.367+0000 7f64fa7fc700 1 -- 172.21.15.32:0/208309906 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f650400f470 con 0x7f65080b7720 2020-12-05T17:35:13.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.367+0000 7f64fa7fc700 1 -- 172.21.15.32:0/208309906 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f650400f5d0 con 0x7f65080b7720 2020-12-05T17:35:13.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.367+0000 7f650e13e700 1 -- 172.21.15.32:0/208309906 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6508063040 con 0x7f65080b7720 2020-12-05T17:35:13.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.367+0000 7f64fa7fc700 1 -- 172.21.15.32:0/208309906 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f650400f930 con 0x7f65080b7720 2020-12-05T17:35:13.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.367+0000 7f650e13e700 1 -- 172.21.15.32:0/208309906 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6508064810 con 0x7f65080b7720 2020-12-05T17:35:13.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.367+0000 7f650e13e700 1 -- 172.21.15.32:0/208309906 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6508057050 con 0x7f65080b7720 2020-12-05T17:35:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.371+0000 7f64fa7fc700 1 -- 172.21.15.32:0/208309906 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6504019030 con 0x7f65080b7720 2020-12-05T17:35:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.371+0000 7f64fa7fc700 1 --2- 172.21.15.32:0/208309906 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f64f0055330 0x7f64f00577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.371+0000 7f650c93b700 1 --2- 172.21.15.32:0/208309906 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f64f0055330 0x7f64f00577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.371+0000 7f64fa7fc700 1 -- 172.21.15.32:0/208309906 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f650406d580 con 0x7f65080b7720 2020-12-05T17:35:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.371+0000 7f650c93b700 1 --2- 172.21.15.32:0/208309906 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f64f0055330 0x7f64f00577f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f64fc019000 tx=0x7f64fc015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:13.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.379+0000 7f64fa7fc700 1 -- 172.21.15.32:0/208309906 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6504043610 con 0x7f65080b7720 2020-12-05T17:35:13.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.527+0000 7f650e13e700 1 -- 172.21.15.32:0/208309906 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f650805aab0 con 0x7f65080b7720 2020-12-05T17:35:13.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.531+0000 7f64fa7fc700 1 -- 172.21.15.32:0/208309906 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v44) v1 ==== 81+0+34037 (secure 0 0 0) 0x7f6504046c30 con 0x7f65080b7720 2020-12-05T17:35:13.529 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:35:13.530 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:35:13.530 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 44, 2020-12-05T17:35:13.530 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", 2020-12-05T17:35:13.530 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2020-12-05T17:31:20.975238+0000", 2020-12-05T17:35:13.531 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2020-12-05T17:33:46.516416+0000", 2020-12-05T17:35:13.531 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:35:13.531 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:35:13.531 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-12-05T17:35:13.531 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2020-12-05T17:35:13.532 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2020-12-05T17:35:13.532 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2020-12-05T17:35:13.532 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2020-12-05T17:35:13.532 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2020-12-05T17:35:13.532 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2020-12-05T17:35:13.533 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:35:13.533 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2020-12-05T17:35:13.533 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2020-12-05T17:35:13.533 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-12-05T17:35:13.534 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2020-12-05T17:35:13.534 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2020-12-05T17:35:13.534 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 8, 2020-12-05T17:35:13.534 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2020-12-05T17:35:13.534 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2020-12-05T17:35:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2020-12-05T17:35:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2020-12-05T17:35:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2020-12-05T17:35:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2020-12-05T17:35:13.536 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2020-12-05T17:35:13.536 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:24.858732+0000", 2020-12-05T17:35:13.536 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:35:13.536 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:35:13.537 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:35:13.537 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:35:13.537 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:35:13.537 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:35:13.537 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:35:13.538 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:35:13.538 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2020-12-05T17:35:13.538 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2020-12-05T17:35:13.538 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2020-12-05T17:35:13.539 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2020-12-05T17:35:13.539 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2020-12-05T17:35:13.539 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:35:13.539 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:35:13.539 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:35:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:35:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:35:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:35:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:35:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.541 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "16", 2020-12-05T17:35:13.541 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:35:13.541 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:35:13.541 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:35:13.541 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:35:13.542 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:35:13.542 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:35:13.542 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:35:13.542 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:35:13.543 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:35:13.543 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:35:13.543 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:35:13.543 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:35:13.543 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:35:13.544 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:35:13.544 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:35:13.544 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:35:13.544 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:35:13.544 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:35:13.545 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:35:13.545 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:35:13.545 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:35:13.545 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:35:13.546 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:35:13.546 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:35:13.546 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:35:13.546 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:35:13.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.547 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:35:13.547 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:35:13.547 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:35:13.547 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:35:13.547 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:35:13.548 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:35:13.548 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:35:13.548 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:35:13.548 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:35:13.548 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:35:13.549 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:35:13.549 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:35:13.549 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2020-12-05T17:35:13.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.549 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:35:13.550 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2020-12-05T17:35:13.550 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.551 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2020-12-05T17:35:13.551 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2020-12-05T17:35:13.551 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:33.305595+0000", 2020-12-05T17:35:13.551 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:35:13.551 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:35:13.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:35:13.552 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:35:13.552 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:35:13.552 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:35:13.552 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:35:13.553 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:35:13.553 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:35:13.553 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:35:13.553 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:35:13.553 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:35:13.554 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:35:13.554 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:35:13.554 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:35:13.554 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:35:13.554 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:35:13.555 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:35:13.555 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:35:13.555 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:35:13.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.556 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "19", 2020-12-05T17:35:13.556 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:35:13.556 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:35:13.556 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:35:13.556 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:35:13.557 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:35:13.557 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:35:13.557 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:35:13.557 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:35:13.557 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:35:13.558 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:35:13.558 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:35:13.558 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:35:13.558 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:35:13.559 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:35:13.559 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:35:13.559 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:35:13.559 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:35:13.560 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:35:13.560 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:35:13.560 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:35:13.560 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:35:13.561 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:35:13.561 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:35:13.561 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:35:13.561 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:35:13.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:35:13.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.562 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:35:13.562 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:35:13.562 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:35:13.562 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:35:13.563 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:35:13.563 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:35:13.563 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:35:13.563 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:35:13.563 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:35:13.564 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:35:13.564 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:35:13.564 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:35:13.564 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:35:13.564 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2020-12-05T17:35:13.565 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.565 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 7, 2020-12-05T17:35:13.565 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2020-12-05T17:35:13.566 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:33:43.025173+0000", 2020-12-05T17:35:13.566 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:35:13.566 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:35:13.566 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:35:13.567 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:35:13.567 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:35:13.567 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:35:13.567 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:35:13.567 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:35:13.568 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:35:13.568 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:35:13.568 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:35:13.568 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:35:13.568 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:35:13.569 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:35:13.569 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:35:13.569 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:35:13.569 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:35:13.569 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:35:13.570 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:35:13.570 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:35:13.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.570 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "43", 2020-12-05T17:35:13.571 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:35:13.571 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:35:13.571 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:35:13.571 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:35:13.571 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:35:13.572 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:35:13.572 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:35:13.572 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:35:13.572 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:35:13.572 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:35:13.573 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:35:13.573 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:35:13.573 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:35:13.573 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:35:13.573 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:35:13.574 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:35:13.574 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:35:13.574 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:35:13.574 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:35:13.575 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:35:13.575 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:35:13.575 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:35:13.575 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:35:13.575 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:35:13.576 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:35:13.576 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:35:13.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.576 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:35:13.576 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:35:13.577 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:35:13.577 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:35:13.577 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:35:13.577 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:35:13.577 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:35:13.578 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:35:13.578 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:35:13.578 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:35:13.578 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:35:13.578 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:35:13.579 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2020-12-05T17:35:13.579 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2020-12-05T17:35:13.579 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2020-12-05T17:35:13.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.580 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:35:13.580 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:35:13.580 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2020-12-05T17:35:13.580 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 8, 2020-12-05T17:35:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2020-12-05T17:35:13.582 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:33:44.344341+0000", 2020-12-05T17:35:13.582 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:35:13.582 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:35:13.582 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:35:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:35:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:35:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:35:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:35:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:35:13.584 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:35:13.584 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:35:13.584 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:35:13.584 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:35:13.584 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:35:13.585 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:35:13.585 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:35:13.585 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:35:13.585 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:35:13.585 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:35:13.586 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:35:13.586 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:35:13.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.586 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "44", 2020-12-05T17:35:13.586 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:35:13.587 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:35:13.587 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:35:13.587 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:35:13.587 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:35:13.588 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:35:13.588 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:35:13.588 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:35:13.588 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:35:13.588 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:35:13.589 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:35:13.589 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:35:13.589 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:35:13.589 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:35:13.589 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:35:13.590 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:35:13.590 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:35:13.590 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:35:13.590 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:35:13.590 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:35:13.591 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:35:13.591 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:35:13.591 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:35:13.591 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:35:13.591 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:35:13.592 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:35:13.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.592 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:35:13.592 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:35:13.593 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:35:13.593 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:35:13.593 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:35:13.593 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:35:13.593 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:35:13.594 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:35:13.594 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:35:13.594 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:35:13.594 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:35:13.595 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:35:13.595 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:35:13.595 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:35:13.595 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2020-12-05T17:35:13.595 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.596 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.596 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.596 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:35:13.596 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2020-12-05T17:35:13.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.597 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:35:13.597 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "b941127b-0a49-4e06-8d11-17794ff8b013", 2020-12-05T17:35:13.597 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:35:13.597 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:35:13.597 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:35:13.598 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:35:13.598 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:35:13.598 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:35:13.598 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:35:13.599 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 42, 2020-12-05T17:35:13.599 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:35:13.599 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:35:13.599 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:35:13.599 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.600 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.600 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2020-12-05T17:35:13.600 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:35:13.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.601 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.601 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2020-12-05T17:35:13.601 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:35:13.601 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.602 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.602 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:35:13.602 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.603 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.603 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2020-12-05T17:35:13.603 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:35:13.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.604 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.604 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2020-12-05T17:35:13.604 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:35:13.604 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.605 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.605 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:35:13.605 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.606 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.606 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2020-12-05T17:35:13.606 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:35:13.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.607 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.607 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2020-12-05T17:35:13.607 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:35:13.607 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.608 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.608 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:35:13.608 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.609 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.609 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2020-12-05T17:35:13.609 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:35:13.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.610 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.610 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2020-12-05T17:35:13.610 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:35:13.610 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.611 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.611 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6803/11526", 2020-12-05T17:35:13.611 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6805/11526", 2020-12-05T17:35:13.611 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6809/11526", 2020-12-05T17:35:13.612 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6807/11526", 2020-12-05T17:35:13.612 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:35:13.612 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:35:13.612 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:35:13.612 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.613 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:35:13.613 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "01bd7ff5-a86b-4e11-ba1e-484c309c0fba", 2020-12-05T17:35:13.613 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:35:13.614 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:35:13.614 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:35:13.614 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:35:13.614 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:35:13.614 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:35:13.615 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:35:13.615 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 42, 2020-12-05T17:35:13.615 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:35:13.615 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:35:13.616 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:35:13.616 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.616 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.616 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2020-12-05T17:35:13.617 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:35:13.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.617 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.617 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2020-12-05T17:35:13.618 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:35:13.618 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.618 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.618 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:35:13.619 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.619 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.619 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2020-12-05T17:35:13.620 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:35:13.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.620 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.620 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2020-12-05T17:35:13.621 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:35:13.621 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.621 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.622 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:35:13.622 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.622 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.622 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2020-12-05T17:35:13.623 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:35:13.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.623 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.624 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2020-12-05T17:35:13.624 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:35:13.624 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.624 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.625 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:35:13.625 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.625 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.625 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2020-12-05T17:35:13.626 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:35:13.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.626 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.626 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2020-12-05T17:35:13.627 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:35:13.627 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.627 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.628 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6819/11538", 2020-12-05T17:35:13.628 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6821/11538", 2020-12-05T17:35:13.628 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6825/11538", 2020-12-05T17:35:13.628 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6823/11538", 2020-12-05T17:35:13.628 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:35:13.629 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:35:13.629 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:35:13.629 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.630 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:35:13.630 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "94736ef5-64c2-4a97-b963-bd38cddd9c52", 2020-12-05T17:35:13.630 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:35:13.630 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:35:13.630 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:35:13.631 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:35:13.631 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:35:13.631 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:35:13.631 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:35:13.631 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 31, 2020-12-05T17:35:13.632 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:35:13.632 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:35:13.632 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:35:13.632 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.633 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.633 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2020-12-05T17:35:13.633 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:35:13.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.634 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.634 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2020-12-05T17:35:13.634 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:35:13.634 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.635 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.635 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:35:13.635 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.636 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.636 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2020-12-05T17:35:13.636 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:35:13.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.637 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.637 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2020-12-05T17:35:13.637 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:35:13.637 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.637 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.638 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:35:13.638 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.639 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.639 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2020-12-05T17:35:13.639 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:35:13.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.640 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.640 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2020-12-05T17:35:13.640 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:35:13.640 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.640 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.641 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:35:13.641 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.641 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.642 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2020-12-05T17:35:13.642 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:35:13.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.642 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.643 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2020-12-05T17:35:13.643 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:35:13.643 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.643 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.644 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6811/11521", 2020-12-05T17:35:13.644 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6813/11521", 2020-12-05T17:35:13.644 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6817/11521", 2020-12-05T17:35:13.644 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6815/11521", 2020-12-05T17:35:13.644 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:35:13.645 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:35:13.645 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:35:13.645 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.646 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:35:13.646 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "69a1f3ff-fd25-43df-b60c-e32a89c35234", 2020-12-05T17:35:13.646 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:35:13.646 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:35:13.647 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:35:13.647 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:35:13.647 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:35:13.647 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:35:13.647 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:35:13.648 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 42, 2020-12-05T17:35:13.648 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:35:13.648 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:35:13.648 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:35:13.648 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.649 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.649 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2020-12-05T17:35:13.649 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:35:13.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.650 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.650 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2020-12-05T17:35:13.650 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:35:13.651 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.651 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.651 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:35:13.651 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.652 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.652 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2020-12-05T17:35:13.652 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:35:13.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.653 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.653 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2020-12-05T17:35:13.653 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:35:13.653 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.654 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.654 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:35:13.654 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.655 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.655 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6832", 2020-12-05T17:35:13.655 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:35:13.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.656 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.656 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6833", 2020-12-05T17:35:13.656 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:35:13.656 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.657 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.657 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:35:13.657 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.658 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.658 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2020-12-05T17:35:13.658 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:35:13.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.659 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.659 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2020-12-05T17:35:13.659 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:35:13.659 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.660 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.660 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6827/11535", 2020-12-05T17:35:13.660 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6829/11535", 2020-12-05T17:35:13.660 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6833/11535", 2020-12-05T17:35:13.661 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6831/11535", 2020-12-05T17:35:13.661 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:35:13.661 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:35:13.661 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:35:13.661 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.662 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:35:13.662 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c301fe4e-5eab-402c-ae6f-db3e6a7181af", 2020-12-05T17:35:13.662 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:35:13.663 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:35:13.663 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:35:13.663 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:35:13.663 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:35:13.664 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:35:13.664 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:35:13.664 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 42, 2020-12-05T17:35:13.664 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:35:13.664 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:35:13.665 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:35:13.665 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.665 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.665 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6808", 2020-12-05T17:35:13.666 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:35:13.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.666 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.666 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6809", 2020-12-05T17:35:13.667 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:35:13.667 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.667 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.667 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:35:13.668 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.668 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.668 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6810", 2020-12-05T17:35:13.668 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:35:13.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.669 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.669 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6811", 2020-12-05T17:35:13.669 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:35:13.670 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.670 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.670 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:35:13.670 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.671 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.671 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6817", 2020-12-05T17:35:13.671 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:35:13.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.672 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.672 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6819", 2020-12-05T17:35:13.672 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:35:13.673 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.673 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.673 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:35:13.673 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.674 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.674 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6812", 2020-12-05T17:35:13.674 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:35:13.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.675 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.675 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6814", 2020-12-05T17:35:13.675 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:35:13.675 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.676 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.676 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6809/11541", 2020-12-05T17:35:13.676 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6811/11541", 2020-12-05T17:35:13.677 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6819/11541", 2020-12-05T17:35:13.677 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6814/11541", 2020-12-05T17:35:13.677 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:35:13.677 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:35:13.677 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:35:13.678 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.678 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:35:13.678 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "9476eff4-345b-46c5-a473-1d9a89a6d327", 2020-12-05T17:35:13.679 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:35:13.679 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:35:13.679 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:35:13.679 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:35:13.680 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:35:13.680 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:35:13.680 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:35:13.680 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 42, 2020-12-05T17:35:13.680 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:35:13.681 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:35:13.681 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:35:13.681 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.681 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.682 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6800", 2020-12-05T17:35:13.682 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:35:13.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.682 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.683 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6801", 2020-12-05T17:35:13.683 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:35:13.683 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.683 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.684 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:35:13.684 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.684 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.684 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6802", 2020-12-05T17:35:13.685 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:35:13.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.685 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.685 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6803", 2020-12-05T17:35:13.686 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:35:13.686 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.686 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.686 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:35:13.687 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.687 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.687 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6806", 2020-12-05T17:35:13.688 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:35:13.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.688 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.688 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6807", 2020-12-05T17:35:13.689 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:35:13.689 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.689 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.690 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:35:13.690 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.690 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.690 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6804", 2020-12-05T17:35:13.691 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:35:13.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.691 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.691 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6805", 2020-12-05T17:35:13.692 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:35:13.692 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.692 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.692 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6801/11539", 2020-12-05T17:35:13.693 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6803/11539", 2020-12-05T17:35:13.693 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6807/11539", 2020-12-05T17:35:13.693 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6805/11539", 2020-12-05T17:35:13.693 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:35:13.694 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:35:13.694 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:35:13.694 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.695 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:35:13.695 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8dfd8567-9b1c-4d13-9c96-071183e469c2", 2020-12-05T17:35:13.695 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:35:13.695 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:35:13.695 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:35:13.696 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:35:13.696 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:35:13.696 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:35:13.696 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:35:13.696 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 42, 2020-12-05T17:35:13.697 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:35:13.697 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:35:13.697 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:35:13.697 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.698 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.698 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6820", 2020-12-05T17:35:13.698 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:35:13.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.699 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.699 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6822", 2020-12-05T17:35:13.699 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:35:13.699 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.699 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.700 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:35:13.700 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.701 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.701 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6823", 2020-12-05T17:35:13.701 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:35:13.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.702 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.702 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6825", 2020-12-05T17:35:13.702 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:35:13.702 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.702 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.703 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:35:13.703 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.703 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.704 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6829", 2020-12-05T17:35:13.704 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:35:13.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.704 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.705 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6830", 2020-12-05T17:35:13.705 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:35:13.705 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.705 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.706 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:35:13.706 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.706 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.707 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6826", 2020-12-05T17:35:13.707 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:35:13.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.707 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.708 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6827", 2020-12-05T17:35:13.708 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:35:13.708 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.708 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.709 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6822/11542", 2020-12-05T17:35:13.709 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6825/11542", 2020-12-05T17:35:13.709 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6830/11542", 2020-12-05T17:35:13.709 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6827/11542", 2020-12-05T17:35:13.710 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:35:13.710 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:35:13.710 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:35:13.710 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.711 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:35:13.711 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "24d466b3-01dc-4ba2-8b81-85250d163bd2", 2020-12-05T17:35:13.711 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:35:13.711 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:35:13.712 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:35:13.712 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:35:13.712 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:35:13.712 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:35:13.713 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:35:13.713 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 40, 2020-12-05T17:35:13.713 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:35:13.713 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:35:13.713 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:35:13.714 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.714 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.714 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6813", 2020-12-05T17:35:13.714 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:35:13.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.715 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.715 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6815", 2020-12-05T17:35:13.715 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:35:13.716 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.716 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.716 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:35:13.717 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.717 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.717 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6816", 2020-12-05T17:35:13.717 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:35:13.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.718 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.718 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6818", 2020-12-05T17:35:13.718 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:35:13.719 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.719 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.719 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:35:13.719 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.720 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.720 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6828", 2020-12-05T17:35:13.720 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:35:13.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.721 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.721 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6831", 2020-12-05T17:35:13.721 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:35:13.722 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.722 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.722 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:35:13.722 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:35:13.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.723 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:35:13.723 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6821", 2020-12-05T17:35:13.723 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:35:13.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.724 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:35:13.724 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6824", 2020-12-05T17:35:13.724 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:35:13.724 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.725 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.725 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6815/11540", 2020-12-05T17:35:13.725 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6818/11540", 2020-12-05T17:35:13.726 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6831/11540", 2020-12-05T17:35:13.726 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6824/11540", 2020-12-05T17:35:13.726 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:35:13.726 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:35:13.726 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:35:13.727 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:13.727 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.727 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:35:13.727 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2020-12-05T17:35:13.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.728 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:35:13.728 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:35:13.728 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:35:13.728 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:35:13.728 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:35:13.729 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:35:13.729 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.398868+0000", 2020-12-05T17:35:13.729 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:35:13.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.730 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:35:13.730 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:35:13.730 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:35:13.730 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:35:13.731 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:35:13.731 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:35:13.731 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.675748+0000", 2020-12-05T17:35:13.731 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:35:13.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.732 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:35:13.732 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:35:13.732 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:35:13.732 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:35:13.733 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:35:13.733 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:35:13.733 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.620888+0000", 2020-12-05T17:35:13.733 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:35:13.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.734 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:35:13.734 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:35:13.734 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:35:13.734 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:35:13.735 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:35:13.735 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:35:13.735 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.681600+0000", 2020-12-05T17:35:13.735 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:35:13.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.736 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:35:13.736 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:35:13.736 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:35:13.737 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:35:13.737 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:35:13.737 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:35:13.737 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.658769+0000", 2020-12-05T17:35:13.737 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:35:13.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.738 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:35:13.738 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:35:13.738 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:35:13.739 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:35:13.739 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:35:13.739 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:35:13.739 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.630708+0000", 2020-12-05T17:35:13.739 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:35:13.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.740 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:35:13.740 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:35:13.740 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:35:13.741 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:35:13.741 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:35:13.741 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:35:13.741 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.648352+0000", 2020-12-05T17:35:13.742 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:35:13.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:13.742 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:35:13.743 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:35:13.743 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:35:13.743 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:35:13.743 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:35:13.743 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:35:13.744 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.629399+0000", 2020-12-05T17:35:13.744 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:35:13.744 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.744 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:35:13.744 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2020-12-05T17:35:13.745 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2020-12-05T17:35:13.745 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2020-12-05T17:35:13.745 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2020-12-05T17:35:13.745 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": {}, 2020-12-05T17:35:13.745 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2020-12-05T17:35:13.746 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2020-12-05T17:35:13.746 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2020-12-05T17:35:13.746 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:35:13.746 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2020-12-05T17:35:13.746 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2020-12-05T17:35:13.747 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:35:13.747 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2020-12-05T17:35:13.747 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:35:13.747 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:35:13.747 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2020-12-05T17:35:13.748 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2020-12-05T17:35:13.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.748 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2020-12-05T17:35:13.748 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:35:13.748 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:35:13.749 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2020-12-05T17:35:13.749 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:35:13.749 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:35:13.749 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2020-12-05T17:35:13.749 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:13.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:13.750 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2020-12-05T17:35:13.750 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2020-12-05T17:35:13.750 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2020-12-05T17:35:13.750 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2020-12-05T17:35:13.751 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2020-12-05T17:35:13.751 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:35:13.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.531+0000 7f650e13e700 1 -- 172.21.15.32:0/208309906 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f64f0055330 msgr2=0x7f64f00577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:13.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.531+0000 7f650e13e700 1 --2- 172.21.15.32:0/208309906 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f64f0055330 0x7f64f00577f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f64fc019000 tx=0x7f64fc015000).stop 2020-12-05T17:35:13.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.531+0000 7f650e13e700 1 -- 172.21.15.32:0/208309906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65080b7720 msgr2=0x7f6508146e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:13.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.531+0000 7f650e13e700 1 --2- 172.21.15.32:0/208309906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65080b7720 0x7f6508146e90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f65040010d0 tx=0x7f6504007af0).stop 2020-12-05T17:35:13.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.531+0000 7f650e13e700 1 -- 172.21.15.32:0/208309906 shutdown_connections 2020-12-05T17:35:13.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.531+0000 7f650e13e700 1 --2- 172.21.15.32:0/208309906 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f64f0055330 0x7f64f00577f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:13.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.531+0000 7f650e13e700 1 --2- 172.21.15.32:0/208309906 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6508144540 0x7f65080642a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:13.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.531+0000 7f650e13e700 1 --2- 172.21.15.32:0/208309906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65080b7720 0x7f6508146e90 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:13.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.531+0000 7f650e13e700 1 --2- 172.21.15.32:0/208309906 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65081403f0 0x7f6508063db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:13.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.531+0000 7f650e13e700 1 -- 172.21.15.32:0/208309906 >> 172.21.15.32:0/208309906 conn(0x7f650805bc20 msgr2=0x7f65080b6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:13.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.531+0000 7f650e13e700 1 -- 172.21.15.32:0/208309906 shutdown_connections 2020-12-05T17:35:13.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.531+0000 7f650e13e700 1 -- 172.21.15.32:0/208309906 wait complete. 2020-12-05T17:35:13.756 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:35:13.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c3441700 1 Processor -- start 2020-12-05T17:35:13.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c3441700 1 -- start start 2020-12-05T17:35:13.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c3441700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f23bc0b1fc0 0x7f23bc0af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:13.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c3441700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23bc147400 0x7f23bc145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:13.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c3441700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23bc146a90 con 0x7f23bc0ab710 2020-12-05T17:35:13.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c3441700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23bc061e90 con 0x7f23bc0b1fc0 2020-12-05T17:35:13.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c3441700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc0b0110 con 0x7f23bc147400 2020-12-05T17:35:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c1c3e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f23bc0ab710 0x7f23bc061780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37658/0 (socket says 172.21.15.32:37658) 2020-12-05T17:35:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c1c3e700 1 -- 172.21.15.32:0/1549783929 learned_addr learned my addr 172.21.15.32:0/1549783929 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:35:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c243f700 1 --2- 172.21.15.32:0/1549783929 >> v2:172.21.15.61:3300/0 conn(0x7f23bc0b1fc0 0x7f23bc0af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c2c40700 1 --2- 172.21.15.32:0/1549783929 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23bc147400 0x7f23bc145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c143d700 1 -- 172.21.15.32:0/1549783929 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 99541549 0 0) 0x7f23bc146a90 con 0x7f23bc0ab710 2020-12-05T17:35:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c143d700 1 -- 172.21.15.32:0/1549783929 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f239c003230 con 0x7f23bc0ab710 2020-12-05T17:35:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c243f700 1 -- 172.21.15.32:0/1549783929 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23bc147400 msgr2=0x7f23bc145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c243f700 1 --2- 172.21.15.32:0/1549783929 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23bc147400 0x7f23bc145d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c143d700 1 -- 172.21.15.32:0/1549783929 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1316032716 0 0) 0x7f239c003230 con 0x7f23bc0ab710 2020-12-05T17:35:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c243f700 1 -- 172.21.15.32:0/1549783929 >> v1:172.21.15.32:6789/0 conn(0x7f23bc0ab710 legacy=0x7f23bc061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:13.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c243f700 1 -- 172.21.15.32:0/1549783929 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc0b23a0 con 0x7f23bc0b1fc0 2020-12-05T17:35:13.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c2c40700 1 --2- 172.21.15.32:0/1549783929 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23bc147400 0x7f23bc145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:35:13.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c243f700 1 --2- 172.21.15.32:0/1549783929 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23bc0b1fc0 0x7f23bc0af230 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f23a4001870 tx=0x7f23ac0091a0).ready entity=mon.1 client_cookie=2befd9886fa9c0d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:13.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c143d700 1 -- 172.21.15.32:0/1549783929 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23a40016d0 con 0x7f23bc0b1fc0 2020-12-05T17:35:13.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c143d700 1 -- 172.21.15.32:0/1549783929 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ac00b800 con 0x7f23bc0b1fc0 2020-12-05T17:35:13.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c143d700 1 -- 172.21.15.32:0/1549783929 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23ac00b970 con 0x7f23bc0b1fc0 2020-12-05T17:35:13.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c3441700 1 -- 172.21.15.32:0/1549783929 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23bc0b1fc0 msgr2=0x7f23bc0af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:13.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.883+0000 7f23c3441700 1 --2- 172.21.15.32:0/1549783929 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23bc0b1fc0 0x7f23bc0af230 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f23a4001870 tx=0x7f23ac0091a0).stop 2020-12-05T17:35:13.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c3441700 1 -- 172.21.15.32:0/1549783929 shutdown_connections 2020-12-05T17:35:13.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c3441700 1 --2- 172.21.15.32:0/1549783929 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23bc147400 0x7f23bc145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:13.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c3441700 1 --2- 172.21.15.32:0/1549783929 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23bc0b1fc0 0x7f23bc0af230 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:13.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c3441700 1 -- 172.21.15.32:0/1549783929 >> 172.21.15.32:0/1549783929 conn(0x7f23bc0b5530 msgr2=0x7f23bc0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:13.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c3441700 1 -- 172.21.15.32:0/1549783929 shutdown_connections 2020-12-05T17:35:13.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c3441700 1 -- 172.21.15.32:0/1549783929 wait complete. 2020-12-05T17:35:13.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c3441700 1 Processor -- start 2020-12-05T17:35:13.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c3441700 1 -- start start 2020-12-05T17:35:13.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c3441700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23bc0ab710 0x7f23bc0abe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:13.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c3441700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23bc0b1fc0 0x7f23bc158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:13.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c3441700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23bc147400 0x7f23bc09e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:13.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c3441700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc0ae300 con 0x7f23bc0b1fc0 2020-12-05T17:35:13.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c3441700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc0adfc0 con 0x7f23bc0ab710 2020-12-05T17:35:13.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c3441700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc0b3260 con 0x7f23bc147400 2020-12-05T17:35:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c1c3e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23bc0b1fc0 0x7f23bc158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c1c3e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23bc0b1fc0 0x7f23bc158cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35016/0 (socket says 172.21.15.32:35016) 2020-12-05T17:35:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c243f700 1 --2- 172.21.15.32:0/963524977 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23bc0ab710 0x7f23bc0abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c1c3e700 1 -- 172.21.15.32:0/963524977 learned_addr learned my addr 172.21.15.32:0/963524977 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:13.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c2c40700 1 --2- 172.21.15.32:0/963524977 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23bc147400 0x7f23bc09e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:13.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c1c3e700 1 -- 172.21.15.32:0/963524977 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23bc147400 msgr2=0x7f23bc09e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:13.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c1c3e700 1 --2- 172.21.15.32:0/963524977 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23bc147400 0x7f23bc09e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:13.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c1c3e700 1 -- 172.21.15.32:0/963524977 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23bc0ab710 msgr2=0x7f23bc0abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:13.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c1c3e700 1 --2- 172.21.15.32:0/963524977 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23bc0ab710 0x7f23bc0abe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c1c3e700 1 -- 172.21.15.32:0/963524977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc0b3500 con 0x7f23bc0b1fc0 2020-12-05T17:35:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c243f700 1 --2- 172.21.15.32:0/963524977 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23bc0ab710 0x7f23bc0abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:35:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.887+0000 7f23c1c3e700 1 --2- 172.21.15.32:0/963524977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23bc0b1fc0 0x7f23bc158cf0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f23bc0ae640 tx=0x7f23a4001ba0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.891+0000 7f23bb7fe700 1 -- 172.21.15.32:0/963524977 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23a4007170 con 0x7f23bc0b1fc0 2020-12-05T17:35:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.891+0000 7f23bb7fe700 1 -- 172.21.15.32:0/963524977 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a40072d0 con 0x7f23bc0b1fc0 2020-12-05T17:35:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.891+0000 7f23c3441700 1 -- 172.21.15.32:0/963524977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23bc09e3c0 con 0x7f23bc0b1fc0 2020-12-05T17:35:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.891+0000 7f23bb7fe700 1 -- 172.21.15.32:0/963524977 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23a400a900 con 0x7f23bc0b1fc0 2020-12-05T17:35:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.891+0000 7f23c3441700 1 -- 172.21.15.32:0/963524977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23bc0b2ee0 con 0x7f23bc0b1fc0 2020-12-05T17:35:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.891+0000 7f23c3441700 1 -- 172.21.15.32:0/963524977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23bc057050 con 0x7f23bc0b1fc0 2020-12-05T17:35:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.891+0000 7f23bb7fe700 1 -- 172.21.15.32:0/963524977 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f23a4010070 con 0x7f23bc0b1fc0 2020-12-05T17:35:13.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.891+0000 7f23bb7fe700 1 --2- 172.21.15.32:0/963524977 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f23a00554e0 0x7f23a00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:13.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.891+0000 7f23bb7fe700 1 -- 172.21.15.32:0/963524977 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f23a406dd40 con 0x7f23bc0b1fc0 2020-12-05T17:35:13.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.891+0000 7f23c243f700 1 --2- 172.21.15.32:0/963524977 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f23a00554e0 0x7f23a00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:13.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.895+0000 7f23c243f700 1 --2- 172.21.15.32:0/963524977 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f23a00554e0 0x7f23a00579a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f23ac01b000 tx=0x7f23ac016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:13.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:13.899+0000 7f23bb7fe700 1 -- 172.21.15.32:0/963524977 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f23a4043dd0 con 0x7f23bc0b1fc0 2020-12-05T17:35:14.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.087+0000 7f23c3441700 1 -- 172.21.15.32:0/963524977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23bc05aab0 con 0x7f23bc0b1fc0 2020-12-05T17:35:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.087+0000 7f23bb7fe700 1 -- 172.21.15.32:0/963524977 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2138 (secure 0 0 0) 0x7f23a40473f0 con 0x7f23bc0b1fc0 2020-12-05T17:35:14.087 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:35:14.087 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4919,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/3405415630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3405415630},{"type":"v1","addr":"172.21.15.61:6833","nonce":3405415630}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:33:45.513696+0000","modified":"2020-12-05T17:33:46.989395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5301":{"gid":5301,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3450615455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3450615455},{"type":"v1","addr":"172.21.15.32:6835","nonce":3450615455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-12-05T17:35:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.087+0000 7f23c3441700 1 -- 172.21.15.32:0/963524977 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f23a00554e0 msgr2=0x7f23a00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.087+0000 7f23c3441700 1 --2- 172.21.15.32:0/963524977 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f23a00554e0 0x7f23a00579a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f23ac01b000 tx=0x7f23ac016000).stop 2020-12-05T17:35:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.087+0000 7f23c3441700 1 -- 172.21.15.32:0/963524977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23bc0b1fc0 msgr2=0x7f23bc158cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.087+0000 7f23c3441700 1 --2- 172.21.15.32:0/963524977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23bc0b1fc0 0x7f23bc158cf0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f23bc0ae640 tx=0x7f23a4001ba0).stop 2020-12-05T17:35:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.087+0000 7f23c3441700 1 -- 172.21.15.32:0/963524977 shutdown_connections 2020-12-05T17:35:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.087+0000 7f23c3441700 1 --2- 172.21.15.32:0/963524977 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f23a00554e0 0x7f23a00579a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:14.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.087+0000 7f23c3441700 1 --2- 172.21.15.32:0/963524977 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23bc147400 0x7f23bc09e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:14.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.087+0000 7f23c3441700 1 --2- 172.21.15.32:0/963524977 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23bc0ab710 0x7f23bc0abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:14.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.087+0000 7f23c3441700 1 --2- 172.21.15.32:0/963524977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23bc0b1fc0 0x7f23bc158cf0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:14.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.087+0000 7f23c3441700 1 -- 172.21.15.32:0/963524977 >> 172.21.15.32:0/963524977 conn(0x7f23bc0b5530 msgr2=0x7f23bc149e60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:14.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.091+0000 7f23c3441700 1 -- 172.21.15.32:0/963524977 shutdown_connections 2020-12-05T17:35:14.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.091+0000 7f23c3441700 1 -- 172.21.15.32:0/963524977 wait complete. 2020-12-05T17:35:14.091 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 17 2020-12-05T17:35:14.101 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-12-05T17:35:14.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.211+0000 7fb836a4f700 1 Processor -- start 2020-12-05T17:35:14.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.211+0000 7fb836a4f700 1 -- start start 2020-12-05T17:35:14.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.211+0000 7fb836a4f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb83005feb0 0x7fb83005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:14.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.211+0000 7fb836a4f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb830142450 con 0x7fb83005f3b0 2020-12-05T17:35:14.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.211+0000 7fb836a4f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb830061780 con 0x7fb830062400 2020-12-05T17:35:14.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.211+0000 7fb836a4f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb830061a20 con 0x7fb83005feb0 2020-12-05T17:35:14.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.215+0000 7fb835a4d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb83005f3b0 0x7fb8300b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37668/0 (socket says 172.21.15.32:37668) 2020-12-05T17:35:14.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.215+0000 7fb835a4d700 1 -- 172.21.15.32:0/2302889940 learned_addr learned my addr 172.21.15.32:0/2302889940 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:35:14.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.215+0000 7fb83624e700 1 --2- 172.21.15.32:0/2302889940 >> v2:172.21.15.61:3300/0 conn(0x7fb83005feb0 0x7fb83005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:14.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.215+0000 7fb834a4b700 1 -- 172.21.15.32:0/2302889940 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3297077138 0 0) 0x7fb830142450 con 0x7fb83005f3b0 2020-12-05T17:35:14.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.215+0000 7fb834a4b700 1 -- 172.21.15.32:0/2302889940 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb810003230 con 0x7fb83005f3b0 2020-12-05T17:35:14.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.215+0000 7fb83624e700 1 -- 172.21.15.32:0/2302889940 >> v1:172.21.15.61:6789/0 conn(0x7fb830062400 legacy=0x7fb8300b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:14.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.215+0000 7fb83624e700 1 -- 172.21.15.32:0/2302889940 >> v1:172.21.15.32:6789/0 conn(0x7fb83005f3b0 legacy=0x7fb8300b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:14.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.215+0000 7fb83624e700 1 -- 172.21.15.32:0/2302889940 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb830061cc0 con 0x7fb83005feb0 2020-12-05T17:35:14.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.215+0000 7fb83624e700 1 --2- 172.21.15.32:0/2302889940 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb83005feb0 0x7fb83005f790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fb8200032b0 tx=0x7fb8200091a0).ready entity=mon.1 client_cookie=cb3b2a9205e6e88 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.215+0000 7fb834a4b700 1 -- 172.21.15.32:0/2302889940 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb82000b9a0 con 0x7fb83005feb0 2020-12-05T17:35:14.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.215+0000 7fb834a4b700 1 -- 172.21.15.32:0/2302889940 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82000bb00 con 0x7fb83005feb0 2020-12-05T17:35:14.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.215+0000 7fb834a4b700 1 -- 172.21.15.32:0/2302889940 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb82000bc70 con 0x7fb83005feb0 2020-12-05T17:35:14.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.215+0000 7fb836a4f700 1 -- 172.21.15.32:0/2302889940 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb83005feb0 msgr2=0x7fb83005f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:14.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.215+0000 7fb836a4f700 1 --2- 172.21.15.32:0/2302889940 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb83005feb0 0x7fb83005f790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fb8200032b0 tx=0x7fb8200091a0).stop 2020-12-05T17:35:14.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.215+0000 7fb836a4f700 1 -- 172.21.15.32:0/2302889940 shutdown_connections 2020-12-05T17:35:14.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.215+0000 7fb836a4f700 1 --2- 172.21.15.32:0/2302889940 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb83005feb0 0x7fb83005f790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:14.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.215+0000 7fb836a4f700 1 -- 172.21.15.32:0/2302889940 >> 172.21.15.32:0/2302889940 conn(0x7fb8300b8ee0 msgr2=0x7fb8300b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:14.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.215+0000 7fb836a4f700 1 -- 172.21.15.32:0/2302889940 shutdown_connections 2020-12-05T17:35:14.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.215+0000 7fb836a4f700 1 -- 172.21.15.32:0/2302889940 wait complete. 2020-12-05T17:35:14.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.215+0000 7fb836a4f700 1 Processor -- start 2020-12-05T17:35:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.215+0000 7fb836a4f700 1 -- start start 2020-12-05T17:35:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb836a4f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb83005f3b0 0x7fb830063380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb836a4f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb83005feb0 0x7fb830064fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb836a4f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb830062400 0x7fb8301513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:14.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb836a4f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb830143d80 con 0x7fb83005f3b0 2020-12-05T17:35:14.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb836a4f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb830149710 con 0x7fb830062400 2020-12-05T17:35:14.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb836a4f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb830144de0 con 0x7fb83005feb0 2020-12-05T17:35:14.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb835a4d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb83005f3b0 0x7fb830063380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:14.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb835a4d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb83005f3b0 0x7fb830063380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35028/0 (socket says 172.21.15.32:35028) 2020-12-05T17:35:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb835a4d700 1 -- 172.21.15.32:0/1250243704 learned_addr learned my addr 172.21.15.32:0/1250243704 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb83624e700 1 --2- 172.21.15.32:0/1250243704 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb830062400 0x7fb8301513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb83524c700 1 --2- 172.21.15.32:0/1250243704 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb83005feb0 0x7fb830064fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb835a4d700 1 -- 172.21.15.32:0/1250243704 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb83005feb0 msgr2=0x7fb830064fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb835a4d700 1 --2- 172.21.15.32:0/1250243704 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb83005feb0 0x7fb830064fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb835a4d700 1 -- 172.21.15.32:0/1250243704 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb830062400 msgr2=0x7fb8301513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb835a4d700 1 --2- 172.21.15.32:0/1250243704 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb830062400 0x7fb8301513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb835a4d700 1 -- 172.21.15.32:0/1250243704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb830064880 con 0x7fb83005f3b0 2020-12-05T17:35:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb835a4d700 1 --2- 172.21.15.32:0/1250243704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb83005f3b0 0x7fb830063380 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7fb8300638f0 tx=0x7fb828007a70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:14.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb82effd700 1 -- 172.21.15.32:0/1250243704 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8280072e0 con 0x7fb83005f3b0 2020-12-05T17:35:14.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb82effd700 1 -- 172.21.15.32:0/1250243704 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb828007440 con 0x7fb83005f3b0 2020-12-05T17:35:14.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb836a4f700 1 -- 172.21.15.32:0/1250243704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb830064b20 con 0x7fb83005f3b0 2020-12-05T17:35:14.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb82effd700 1 -- 172.21.15.32:0/1250243704 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8280057d0 con 0x7fb83005f3b0 2020-12-05T17:35:14.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb836a4f700 1 -- 172.21.15.32:0/1250243704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb830144a60 con 0x7fb83005f3b0 2020-12-05T17:35:14.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb836a4f700 1 -- 172.21.15.32:0/1250243704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb830057050 con 0x7fb83005f3b0 2020-12-05T17:35:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.219+0000 7fb82effd700 1 -- 172.21.15.32:0/1250243704 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb828010070 con 0x7fb83005f3b0 2020-12-05T17:35:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.223+0000 7fb82effd700 1 --2- 172.21.15.32:0/1250243704 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8140554e0 0x7fb8140579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.223+0000 7fb83524c700 1 --2- 172.21.15.32:0/1250243704 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8140554e0 0x7fb8140579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.223+0000 7fb82effd700 1 -- 172.21.15.32:0/1250243704 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb82806d930 con 0x7fb83005f3b0 2020-12-05T17:35:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.223+0000 7fb83524c700 1 --2- 172.21.15.32:0/1250243704 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8140554e0 0x7fb8140579a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fb81800b000 tx=0x7fb818009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:14.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.227+0000 7fb82effd700 1 -- 172.21.15.32:0/1250243704 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb828043940 con 0x7fb83005f3b0 2020-12-05T17:35:14.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.411+0000 7fb836a4f700 1 -- 172.21.15.32:0/1250243704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb83005aab0 con 0x7fb83005f3b0 2020-12-05T17:35:14.788 INFO:tasks.ceph.mon.a.smithi032.stderr:2020-12-05T17:35:14.791+0000 7fdef8136700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-12-05T17:35:14.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.803+0000 7fb82effd700 1 -- 172.21.15.32:0/1250243704 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb828046f60 con 0x7fb83005f3b0 2020-12-05T17:35:14.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.803+0000 7fb836a4f700 1 -- 172.21.15.32:0/1250243704 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8140554e0 msgr2=0x7fb8140579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:14.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.803+0000 7fb836a4f700 1 --2- 172.21.15.32:0/1250243704 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8140554e0 0x7fb8140579a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fb81800b000 tx=0x7fb818009040).stop 2020-12-05T17:35:14.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.803+0000 7fb836a4f700 1 -- 172.21.15.32:0/1250243704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb83005f3b0 msgr2=0x7fb830063380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:14.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.803+0000 7fb836a4f700 1 --2- 172.21.15.32:0/1250243704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb83005f3b0 0x7fb830063380 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7fb8300638f0 tx=0x7fb828007a70).stop 2020-12-05T17:35:14.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.807+0000 7fb836a4f700 1 -- 172.21.15.32:0/1250243704 shutdown_connections 2020-12-05T17:35:14.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.807+0000 7fb836a4f700 1 --2- 172.21.15.32:0/1250243704 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb830062400 0x7fb8301513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:14.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.807+0000 7fb836a4f700 1 --2- 172.21.15.32:0/1250243704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb83005f3b0 0x7fb830063380 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:14.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.807+0000 7fb836a4f700 1 --2- 172.21.15.32:0/1250243704 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8140554e0 0x7fb8140579a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:14.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.807+0000 7fb836a4f700 1 --2- 172.21.15.32:0/1250243704 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb83005feb0 0x7fb830064fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.807+0000 7fb836a4f700 1 -- 172.21.15.32:0/1250243704 >> 172.21.15.32:0/1250243704 conn(0x7fb8300b8ee0 msgr2=0x7fb8300b7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.807+0000 7fb836a4f700 1 -- 172.21.15.32:0/1250243704 shutdown_connections 2020-12-05T17:35:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.807+0000 7fb836a4f700 1 -- 172.21.15.32:0/1250243704 wait complete. 2020-12-05T17:35:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-12-05T17:35:14.820 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:35:14.839 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:35:14.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.927+0000 7fc76a3b3700 1 Processor -- start 2020-12-05T17:35:14.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.931+0000 7fc76a3b3700 1 -- start start 2020-12-05T17:35:14.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.931+0000 7fc76a3b3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc7640b1870 0x7fc7640b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:14.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.931+0000 7fc76a3b3700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc7640ae860 0x7fc7640aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:14.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.931+0000 7fc76a3b3700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7641420f0 con 0x7fc7640b52f0 2020-12-05T17:35:14.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.931+0000 7fc76a3b3700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7640ad4f0 con 0x7fc7640ae860 2020-12-05T17:35:14.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.931+0000 7fc76a3b3700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7640b0380 con 0x7fc7640b1870 2020-12-05T17:35:14.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.931+0000 7fc769bb2700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc7640ae860 0x7fc7640aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:14.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.931+0000 7fc7693b1700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fc7640b52f0 0x7fc764138810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:50334/0 (socket says 172.21.15.32:50334) 2020-12-05T17:35:14.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.931+0000 7fc769bb2700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc7640ae860 0x7fc7640aecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:54942/0 (socket says 172.21.15.32:54942) 2020-12-05T17:35:14.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.931+0000 7fc768bb0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc7640b1870 0x7fc7640b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:14.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.931+0000 7fc7693b1700 1 -- 172.21.15.32:0/4142859764 learned_addr learned my addr 172.21.15.32:0/4142859764 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:35:14.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.931+0000 7fc769bb2700 1 -- 172.21.15.32:0/4142859764 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc7640b1870 msgr2=0x7fc7640b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:14.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.931+0000 7fc769bb2700 1 --2- 172.21.15.32:0/4142859764 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc7640b1870 0x7fc7640b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:14.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.931+0000 7fc769bb2700 1 -- 172.21.15.32:0/4142859764 >> v1:172.21.15.61:6789/0 conn(0x7fc7640b52f0 legacy=0x7fc764138810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:14.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.931+0000 7fc769bb2700 1 -- 172.21.15.32:0/4142859764 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7640b0620 con 0x7fc7640ae860 2020-12-05T17:35:14.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.931+0000 7fc753fff700 1 -- 172.21.15.32:0/4142859764 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1531263777 0 0) 0x7fc7641420f0 con 0x7fc7640b52f0 2020-12-05T17:35:14.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.931+0000 7fc768bb0700 1 --2- 172.21.15.32:0/4142859764 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc7640b1870 0x7fc7640b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:35:14.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.931+0000 7fc769bb2700 1 --2- 172.21.15.32:0/4142859764 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc7640ae860 0x7fc7640aecf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fc75c0032b0 tx=0x7fc75c0091a0).ready entity=mon.1 client_cookie=ee8757b63505436a server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:14.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.931+0000 7fc753fff700 1 -- 172.21.15.32:0/4142859764 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc75c00b9a0 con 0x7fc7640ae860 2020-12-05T17:35:14.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.931+0000 7fc753fff700 1 -- 172.21.15.32:0/4142859764 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc75c00bb00 con 0x7fc7640ae860 2020-12-05T17:35:14.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.931+0000 7fc753fff700 1 -- 172.21.15.32:0/4142859764 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc75c00bc70 con 0x7fc7640ae860 2020-12-05T17:35:14.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.931+0000 7fc76a3b3700 1 -- 172.21.15.32:0/4142859764 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc7640ae860 msgr2=0x7fc7640aecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:14.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.931+0000 7fc76a3b3700 1 --2- 172.21.15.32:0/4142859764 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc7640ae860 0x7fc7640aecf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fc75c0032b0 tx=0x7fc75c0091a0).stop 2020-12-05T17:35:14.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.935+0000 7fc76a3b3700 1 -- 172.21.15.32:0/4142859764 shutdown_connections 2020-12-05T17:35:14.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.935+0000 7fc76a3b3700 1 --2- 172.21.15.32:0/4142859764 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc7640ae860 0x7fc7640aecf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:14.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.935+0000 7fc76a3b3700 1 --2- 172.21.15.32:0/4142859764 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc7640b1870 0x7fc7640b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:14.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.935+0000 7fc76a3b3700 1 -- 172.21.15.32:0/4142859764 >> 172.21.15.32:0/4142859764 conn(0x7fc76405bbb0 msgr2=0x7fc764062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:14.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.935+0000 7fc76a3b3700 1 -- 172.21.15.32:0/4142859764 shutdown_connections 2020-12-05T17:35:14.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.935+0000 7fc76a3b3700 1 -- 172.21.15.32:0/4142859764 wait complete. 2020-12-05T17:35:14.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.935+0000 7fc76a3b3700 1 Processor -- start 2020-12-05T17:35:14.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.935+0000 7fc76a3b3700 1 -- start start 2020-12-05T17:35:14.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.939+0000 7fc76a3b3700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc7640ae860 0x7fc76413f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:14.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.939+0000 7fc76a3b3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7640b1870 0x7fc764062ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:14.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.939+0000 7fc76a3b3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc7640b52f0 0x7fc7641141d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:14.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.939+0000 7fc76a3b3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc764113da0 con 0x7fc7640b1870 2020-12-05T17:35:14.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.939+0000 7fc76a3b3700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc764066dc0 con 0x7fc7640ae860 2020-12-05T17:35:14.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.939+0000 7fc76a3b3700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc764065c90 con 0x7fc7640b52f0 2020-12-05T17:35:14.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.939+0000 7fc768bb0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7640b1870 0x7fc764062ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:14.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.939+0000 7fc768bb0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7640b1870 0x7fc764062ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35054/0 (socket says 172.21.15.32:35054) 2020-12-05T17:35:14.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.939+0000 7fc768bb0700 1 -- 172.21.15.32:0/1495452132 learned_addr learned my addr 172.21.15.32:0/1495452132 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:14.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.939+0000 7fc7693b1700 1 --2- 172.21.15.32:0/1495452132 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc7640ae860 0x7fc76413f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:14.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.939+0000 7fc769bb2700 1 --2- 172.21.15.32:0/1495452132 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc7640b52f0 0x7fc7641141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:14.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.939+0000 7fc768bb0700 1 -- 172.21.15.32:0/1495452132 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc7640b52f0 msgr2=0x7fc7641141d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:14.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.939+0000 7fc768bb0700 1 --2- 172.21.15.32:0/1495452132 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc7640b52f0 0x7fc7641141d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:14.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.939+0000 7fc768bb0700 1 -- 172.21.15.32:0/1495452132 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc7640ae860 msgr2=0x7fc76413f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:14.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.939+0000 7fc768bb0700 1 --2- 172.21.15.32:0/1495452132 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc7640ae860 0x7fc76413f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:14.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.939+0000 7fc768bb0700 1 -- 172.21.15.32:0/1495452132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7640651f0 con 0x7fc7640b1870 2020-12-05T17:35:14.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.939+0000 7fc768bb0700 1 --2- 172.21.15.32:0/1495452132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7640b1870 0x7fc764062ff0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7fc75400c000 tx=0x7fc754006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:14.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.939+0000 7fc7527fc700 1 -- 172.21.15.32:0/1495452132 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc75400cb40 con 0x7fc7640b1870 2020-12-05T17:35:14.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.939+0000 7fc7527fc700 1 -- 172.21.15.32:0/1495452132 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc75400cca0 con 0x7fc7640b1870 2020-12-05T17:35:14.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.939+0000 7fc76a3b3700 1 -- 172.21.15.32:0/1495452132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc764065f30 con 0x7fc7640b1870 2020-12-05T17:35:14.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.939+0000 7fc7527fc700 1 -- 172.21.15.32:0/1495452132 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7540123f0 con 0x7fc7640b1870 2020-12-05T17:35:14.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.939+0000 7fc76a3b3700 1 -- 172.21.15.32:0/1495452132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc764064ce0 con 0x7fc7640b1870 2020-12-05T17:35:14.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.943+0000 7fc76a3b3700 1 -- 172.21.15.32:0/1495452132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc764113f10 con 0x7fc7640b1870 2020-12-05T17:35:14.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.943+0000 7fc7527fc700 1 -- 172.21.15.32:0/1495452132 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc754012550 con 0x7fc7640b1870 2020-12-05T17:35:14.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.943+0000 7fc7527fc700 1 --2- 172.21.15.32:0/1495452132 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc7440553b0 0x7fc744057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:14.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.943+0000 7fc7527fc700 1 -- 172.21.15.32:0/1495452132 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fc75406e7c0 con 0x7fc7640b1870 2020-12-05T17:35:14.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.943+0000 7fc7693b1700 1 --2- 172.21.15.32:0/1495452132 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc7440553b0 0x7fc744057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:14.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.943+0000 7fc7693b1700 1 --2- 172.21.15.32:0/1495452132 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc7440553b0 0x7fc744057870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fc76000d000 tx=0x7fc760009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:14.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:14.951+0000 7fc7527fc700 1 -- 172.21.15.32:0/1495452132 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc764113f10 con 0x7fc7640b1870 2020-12-05T17:35:15.097 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:35:15.100 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:35:15.102 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:35:15.105 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:35:15.107 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:35:15.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.139+0000 7fc76a3b3700 1 -- 172.21.15.32:0/1495452132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc764113ee0 con 0x7fc7640b1870 2020-12-05T17:35:15.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.139+0000 7fc7527fc700 1 -- 172.21.15.32:0/1495452132 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+1777 (secure 0 0 0) 0x7fc754044810 con 0x7fc7640b1870 2020-12-05T17:35:15.140 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:35:15.140 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4919,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/3405415630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3405415630},{"type":"v1","addr":"172.21.15.61:6833","nonce":3405415630}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:33:45.513696+0000","modified":"2020-12-05T17:35:14.794201+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-12-05T17:35:15.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.143+0000 7fc76a3b3700 1 -- 172.21.15.32:0/1495452132 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc7440553b0 msgr2=0x7fc744057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:15.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.143+0000 7fc76a3b3700 1 --2- 172.21.15.32:0/1495452132 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc7440553b0 0x7fc744057870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fc76000d000 tx=0x7fc760009040).stop 2020-12-05T17:35:15.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.143+0000 7fc76a3b3700 1 -- 172.21.15.32:0/1495452132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7640b1870 msgr2=0x7fc764062ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:15.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.143+0000 7fc76a3b3700 1 --2- 172.21.15.32:0/1495452132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7640b1870 0x7fc764062ff0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7fc75400c000 tx=0x7fc754006dc0).stop 2020-12-05T17:35:15.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.143+0000 7fc76a3b3700 1 -- 172.21.15.32:0/1495452132 shutdown_connections 2020-12-05T17:35:15.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.143+0000 7fc76a3b3700 1 --2- 172.21.15.32:0/1495452132 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc7440553b0 0x7fc744057870 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:15.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.143+0000 7fc76a3b3700 1 --2- 172.21.15.32:0/1495452132 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc7640b52f0 0x7fc7641141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:15.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.143+0000 7fc76a3b3700 1 --2- 172.21.15.32:0/1495452132 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc7640ae860 0x7fc76413f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:15.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.143+0000 7fc76a3b3700 1 --2- 172.21.15.32:0/1495452132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7640b1870 0x7fc764062ff0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:15.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.143+0000 7fc76a3b3700 1 -- 172.21.15.32:0/1495452132 >> 172.21.15.32:0/1495452132 conn(0x7fc76405bbb0 msgr2=0x7fc76413f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:15.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.143+0000 7fc76a3b3700 1 -- 172.21.15.32:0/1495452132 shutdown_connections 2020-12-05T17:35:15.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.143+0000 7fc76a3b3700 1 -- 172.21.15.32:0/1495452132 wait complete. 2020-12-05T17:35:15.146 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2020-12-05T17:35:15.154 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-12-05T17:35:15.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.319+0000 7ff087fff700 1 Processor -- start 2020-12-05T17:35:15.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.323+0000 7ff087fff700 1 -- start start 2020-12-05T17:35:15.324 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.323+0000 7ff087fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff0880b52f0 0x7ff0880b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:15.324 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.323+0000 7ff087fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0880b1870 0x7ff088138810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:15.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.323+0000 7ff087fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0881420f0 con 0x7ff088138d00 2020-12-05T17:35:15.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.323+0000 7ff087fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0880ad4f0 con 0x7ff0880b1870 2020-12-05T17:35:15.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.323+0000 7ff087fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0880b0380 con 0x7ff0880b52f0 2020-12-05T17:35:15.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.323+0000 7ff0867fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0880b1870 0x7ff088138810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:15.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.323+0000 7ff0867fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0880b1870 0x7ff088138810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35062/0 (socket says 172.21.15.32:35062) 2020-12-05T17:35:15.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.323+0000 7ff0867fc700 1 -- 172.21.15.32:0/1312925021 learned_addr learned my addr 172.21.15.32:0/1312925021 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:15.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.323+0000 7ff0877fe700 1 --1- 172.21.15.32:0/1312925021 >> v1:172.21.15.32:6789/0 conn(0x7ff088138d00 0x7ff0880ae860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37712/0 (socket says 172.21.15.32:37712) 2020-12-05T17:35:15.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.323+0000 7ff086ffd700 1 --2- 172.21.15.32:0/1312925021 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff0880b52f0 0x7ff0880b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:15.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.323+0000 7ff0867fc700 1 -- 172.21.15.32:0/1312925021 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff0880b52f0 msgr2=0x7ff0880b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:15.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.323+0000 7ff0867fc700 1 --2- 172.21.15.32:0/1312925021 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff0880b52f0 0x7ff0880b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:15.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.323+0000 7ff0867fc700 1 -- 172.21.15.32:0/1312925021 >> v1:172.21.15.32:6789/0 conn(0x7ff088138d00 legacy=0x7ff0880ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:15.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.323+0000 7ff085ffb700 1 -- 172.21.15.32:0/1312925021 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3758669988 0 0) 0x7ff0881420f0 con 0x7ff088138d00 2020-12-05T17:35:15.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.323+0000 7ff0867fc700 1 -- 172.21.15.32:0/1312925021 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0880b0620 con 0x7ff0880b1870 2020-12-05T17:35:15.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.323+0000 7ff086ffd700 1 --2- 172.21.15.32:0/1312925021 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff0880b52f0 0x7ff0880b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:15.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.323+0000 7ff0867fc700 1 --2- 172.21.15.32:0/1312925021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0880b1870 0x7ff088138810 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7ff0880adc80 tx=0x7ff07c006e80).ready entity=mon.0 client_cookie=b943bddc37bf8277 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:15.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.323+0000 7ff085ffb700 1 -- 172.21.15.32:0/1312925021 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff07c014070 con 0x7ff0880b1870 2020-12-05T17:35:15.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.323+0000 7ff085ffb700 1 -- 172.21.15.32:0/1312925021 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07c0099a0 con 0x7ff0880b1870 2020-12-05T17:35:15.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.323+0000 7ff085ffb700 1 -- 172.21.15.32:0/1312925021 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff07c009d00 con 0x7ff0880b1870 2020-12-05T17:35:15.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.323+0000 7ff087fff700 1 -- 172.21.15.32:0/1312925021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0880b1870 msgr2=0x7ff088138810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:15.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.323+0000 7ff087fff700 1 --2- 172.21.15.32:0/1312925021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0880b1870 0x7ff088138810 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7ff0880adc80 tx=0x7ff07c006e80).stop 2020-12-05T17:35:15.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.327+0000 7ff087fff700 1 -- 172.21.15.32:0/1312925021 shutdown_connections 2020-12-05T17:35:15.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.327+0000 7ff087fff700 1 --2- 172.21.15.32:0/1312925021 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff0880b52f0 0x7ff0880b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:15.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.327+0000 7ff087fff700 1 --2- 172.21.15.32:0/1312925021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0880b1870 0x7ff088138810 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:15.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.327+0000 7ff087fff700 1 -- 172.21.15.32:0/1312925021 >> 172.21.15.32:0/1312925021 conn(0x7ff08805bbb0 msgr2=0x7ff088062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:15.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.327+0000 7ff087fff700 1 -- 172.21.15.32:0/1312925021 shutdown_connections 2020-12-05T17:35:15.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.327+0000 7ff087fff700 1 -- 172.21.15.32:0/1312925021 wait complete. 2020-12-05T17:35:15.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.327+0000 7ff087fff700 1 Processor -- start 2020-12-05T17:35:15.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.327+0000 7ff087fff700 1 -- start start 2020-12-05T17:35:15.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.327+0000 7ff087fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff0880b1870 0x7ff08813ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:15.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.327+0000 7ff087fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0880b52f0 0x7ff088064390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:15.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.327+0000 7ff087fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff088138d00 0x7ff088063b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:15.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.327+0000 7ff087fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0880652c0 con 0x7ff0880b52f0 2020-12-05T17:35:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.327+0000 7ff087fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08813f8d0 con 0x7ff0880b1870 2020-12-05T17:35:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.327+0000 7ff087fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08813f630 con 0x7ff088138d00 2020-12-05T17:35:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.327+0000 7ff0867fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0880b52f0 0x7ff088064390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.327+0000 7ff0867fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0880b52f0 0x7ff088064390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35068/0 (socket says 172.21.15.32:35068) 2020-12-05T17:35:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.327+0000 7ff0867fc700 1 -- 172.21.15.32:0/3720728356 learned_addr learned my addr 172.21.15.32:0/3720728356 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.331+0000 7ff0877fe700 1 --2- 172.21.15.32:0/3720728356 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff088138d00 0x7ff088063b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.331+0000 7ff086ffd700 1 --2- 172.21.15.32:0/3720728356 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff0880b1870 0x7ff08813ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.331+0000 7ff0867fc700 1 -- 172.21.15.32:0/3720728356 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff088138d00 msgr2=0x7ff088063b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.331+0000 7ff0867fc700 1 --2- 172.21.15.32:0/3720728356 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff088138d00 0x7ff088063b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.331+0000 7ff0867fc700 1 -- 172.21.15.32:0/3720728356 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff0880b1870 msgr2=0x7ff08813ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.331+0000 7ff0867fc700 1 --2- 172.21.15.32:0/3720728356 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff0880b1870 0x7ff08813ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.331+0000 7ff0867fc700 1 -- 172.21.15.32:0/3720728356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff088066920 con 0x7ff0880b52f0 2020-12-05T17:35:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.331+0000 7ff0877fe700 1 --2- 172.21.15.32:0/3720728356 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff088138d00 0x7ff088063b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.331+0000 7ff0867fc700 1 --2- 172.21.15.32:0/3720728356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0880b52f0 0x7ff088064390 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7ff0880adc80 tx=0x7ff07c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:15.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.331+0000 7ff077fff700 1 -- 172.21.15.32:0/3720728356 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff07c01d450 con 0x7ff0880b52f0 2020-12-05T17:35:15.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.331+0000 7ff077fff700 1 -- 172.21.15.32:0/3720728356 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07c01d5b0 con 0x7ff0880b52f0 2020-12-05T17:35:15.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.331+0000 7ff077fff700 1 -- 172.21.15.32:0/3720728356 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff07c01d910 con 0x7ff0880b52f0 2020-12-05T17:35:15.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.331+0000 7ff087fff700 1 -- 172.21.15.32:0/3720728356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0880630b0 con 0x7ff0880b52f0 2020-12-05T17:35:15.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.331+0000 7ff087fff700 1 -- 172.21.15.32:0/3720728356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff08813fb30 con 0x7ff0880b52f0 2020-12-05T17:35:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.331+0000 7ff087fff700 1 -- 172.21.15.32:0/3720728356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff088066ab0 con 0x7ff0880b52f0 2020-12-05T17:35:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.331+0000 7ff077fff700 1 -- 172.21.15.32:0/3720728356 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff07c00e040 con 0x7ff0880b52f0 2020-12-05T17:35:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.331+0000 7ff077fff700 1 --2- 172.21.15.32:0/3720728356 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff064055440 0x7ff064057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.331+0000 7ff077fff700 1 -- 172.21.15.32:0/3720728356 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6506+0+0 (secure 0 0 0) 0x7ff07c071090 con 0x7ff0880b52f0 2020-12-05T17:35:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.331+0000 7ff086ffd700 1 --2- 172.21.15.32:0/3720728356 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff064055440 0x7ff064057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.331+0000 7ff086ffd700 1 --2- 172.21.15.32:0/3720728356 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff064055440 0x7ff064057900 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7ff07800c000 tx=0x7ff078008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.339+0000 7ff077fff700 1 -- 172.21.15.32:0/3720728356 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff07c0471e0 con 0x7ff0880b52f0 2020-12-05T17:35:15.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.531+0000 7ff087fff700 1 -- 172.21.15.32:0/3720728356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff088057050 con 0x7ff0880b52f0 2020-12-05T17:35:15.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.811+0000 7ff077fff700 1 -- 172.21.15.32:0/3720728356 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff07c04a800 con 0x7ff0880b52f0 2020-12-05T17:35:15.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.815+0000 7ff087fff700 1 -- 172.21.15.32:0/3720728356 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff064055440 msgr2=0x7ff064057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:15.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.815+0000 7ff087fff700 1 --2- 172.21.15.32:0/3720728356 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff064055440 0x7ff064057900 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7ff07800c000 tx=0x7ff078008040).stop 2020-12-05T17:35:15.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.815+0000 7ff087fff700 1 -- 172.21.15.32:0/3720728356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0880b52f0 msgr2=0x7ff088064390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:15.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.815+0000 7ff087fff700 1 --2- 172.21.15.32:0/3720728356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0880b52f0 0x7ff088064390 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7ff0880adc80 tx=0x7ff07c016000).stop 2020-12-05T17:35:15.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.815+0000 7ff087fff700 1 -- 172.21.15.32:0/3720728356 shutdown_connections 2020-12-05T17:35:15.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.815+0000 7ff087fff700 1 --2- 172.21.15.32:0/3720728356 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff064055440 0x7ff064057900 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:15.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.815+0000 7ff087fff700 1 --2- 172.21.15.32:0/3720728356 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff088138d00 0x7ff088063b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:15.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.815+0000 7ff087fff700 1 --2- 172.21.15.32:0/3720728356 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff0880b1870 0x7ff08813ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:15.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.815+0000 7ff087fff700 1 --2- 172.21.15.32:0/3720728356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0880b52f0 0x7ff088064390 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:15.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.815+0000 7ff087fff700 1 -- 172.21.15.32:0/3720728356 >> 172.21.15.32:0/3720728356 conn(0x7ff08805bbb0 msgr2=0x7ff0880af060 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:15.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.815+0000 7ff087fff700 1 -- 172.21.15.32:0/3720728356 shutdown_connections 2020-12-05T17:35:15.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.815+0000 7ff087fff700 1 -- 172.21.15.32:0/3720728356 wait complete. 2020-12-05T17:35:15.832 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-12-05T17:35:15.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.951+0000 7f2de29d4700 1 Processor -- start 2020-12-05T17:35:15.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.951+0000 7f2de29d4700 1 -- start start 2020-12-05T17:35:15.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.951+0000 7f2de29d4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ddc05feb0 0x7f2ddc05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:15.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.951+0000 7f2de29d4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ddc0b2df0 con 0x7f2ddc05f3b0 2020-12-05T17:35:15.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.951+0000 7f2de29d4700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ddc0b3c90 con 0x7f2ddc0b2fb0 2020-12-05T17:35:15.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.951+0000 7f2de29d4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ddc0b2b50 con 0x7f2ddc05feb0 2020-12-05T17:35:15.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.955+0000 7f2de11d1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ddc05feb0 0x7f2ddc05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:15.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.955+0000 7f2de19d2700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2ddc05f3b0 0x7f2ddc13c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37722/0 (socket says 172.21.15.32:37722) 2020-12-05T17:35:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.955+0000 7f2de19d2700 1 -- 172.21.15.32:0/3357741226 learned_addr learned my addr 172.21.15.32:0/3357741226 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:35:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.955+0000 7f2de11d1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ddc05feb0 0x7f2ddc05f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35076/0 (socket says 172.21.15.32:35076) 2020-12-05T17:35:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.955+0000 7f2de11d1700 1 -- 172.21.15.32:0/3357741226 >> v1:172.21.15.61:6789/0 conn(0x7f2ddc0b2fb0 legacy=0x7f2ddc142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.955+0000 7f2de11d1700 1 -- 172.21.15.32:0/3357741226 >> v1:172.21.15.32:6789/0 conn(0x7f2ddc05f3b0 legacy=0x7f2ddc13c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.955+0000 7f2de11d1700 1 -- 172.21.15.32:0/3357741226 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ddc142820 con 0x7f2ddc05feb0 2020-12-05T17:35:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.955+0000 7f2de11d1700 1 --2- 172.21.15.32:0/3357741226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ddc05feb0 0x7f2ddc05f790 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f2dd0002f30 tx=0x7f2dd0006dc0).ready entity=mon.0 client_cookie=385a789a95dea943 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.955+0000 7f2de09d0700 1 -- 172.21.15.32:0/3357741226 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2dd00097f0 con 0x7f2ddc05feb0 2020-12-05T17:35:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.955+0000 7f2de09d0700 1 -- 172.21.15.32:0/3357741226 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd0009950 con 0x7f2ddc05feb0 2020-12-05T17:35:15.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.955+0000 7f2de09d0700 1 -- 172.21.15.32:0/3357741226 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2dd0009cb0 con 0x7f2ddc05feb0 2020-12-05T17:35:15.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.955+0000 7f2de29d4700 1 -- 172.21.15.32:0/3357741226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ddc05feb0 msgr2=0x7f2ddc05f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:15.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.955+0000 7f2de29d4700 1 --2- 172.21.15.32:0/3357741226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ddc05feb0 0x7f2ddc05f790 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f2dd0002f30 tx=0x7f2dd0006dc0).stop 2020-12-05T17:35:15.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.955+0000 7f2de29d4700 1 -- 172.21.15.32:0/3357741226 shutdown_connections 2020-12-05T17:35:15.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.955+0000 7f2de29d4700 1 --2- 172.21.15.32:0/3357741226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ddc05feb0 0x7f2ddc05f790 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:15.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.955+0000 7f2de29d4700 1 -- 172.21.15.32:0/3357741226 >> 172.21.15.32:0/3357741226 conn(0x7f2ddc0b8f20 msgr2=0x7f2ddc0b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:15.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.955+0000 7f2de29d4700 1 -- 172.21.15.32:0/3357741226 shutdown_connections 2020-12-05T17:35:15.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.955+0000 7f2de29d4700 1 -- 172.21.15.32:0/3357741226 wait complete. 2020-12-05T17:35:15.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.955+0000 7f2de29d4700 1 Processor -- start 2020-12-05T17:35:15.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.955+0000 7f2de29d4700 1 -- start start 2020-12-05T17:35:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2de29d4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2ddc05f3b0 0x7f2ddc0b9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2de29d4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ddc05feb0 0x7f2ddc13e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2de29d4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2ddc0b2fb0 0x7f2ddc13e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2de29d4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc13d040 con 0x7f2ddc05feb0 2020-12-05T17:35:15.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2de29d4700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc148000 con 0x7f2ddc0b2fb0 2020-12-05T17:35:15.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2de29d4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc148260 con 0x7f2ddc05f3b0 2020-12-05T17:35:15.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2de11d1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ddc05feb0 0x7f2ddc13e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:15.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2de11d1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ddc05feb0 0x7f2ddc13e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35082/0 (socket says 172.21.15.32:35082) 2020-12-05T17:35:15.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2de11d1700 1 -- 172.21.15.32:0/3548630654 learned_addr learned my addr 172.21.15.32:0/3548630654 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:15.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2de19d2700 1 --2- 172.21.15.32:0/3548630654 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2ddc05f3b0 0x7f2ddc0b9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:15.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2de21d3700 1 --2- 172.21.15.32:0/3548630654 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2ddc0b2fb0 0x7f2ddc13e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:15.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2de11d1700 1 -- 172.21.15.32:0/3548630654 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2ddc05f3b0 msgr2=0x7f2ddc0b9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:15.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2de11d1700 1 --2- 172.21.15.32:0/3548630654 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2ddc05f3b0 0x7f2ddc0b9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2de11d1700 1 -- 172.21.15.32:0/3548630654 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2ddc0b2fb0 msgr2=0x7f2ddc13e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2de11d1700 1 --2- 172.21.15.32:0/3548630654 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2ddc0b2fb0 0x7f2ddc13e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2de11d1700 1 -- 172.21.15.32:0/3548630654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ddc13edb0 con 0x7f2ddc05feb0 2020-12-05T17:35:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2de19d2700 1 --2- 172.21.15.32:0/3548630654 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2ddc05f3b0 0x7f2ddc0b9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2de21d3700 1 --2- 172.21.15.32:0/3548630654 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2ddc0b2fb0 0x7f2ddc13e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2de11d1700 1 --2- 172.21.15.32:0/3548630654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ddc05feb0 0x7f2ddc13e3d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f2dd0002f30 tx=0x7f2dd0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2ddaffd700 1 -- 172.21.15.32:0/3548630654 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2dd001d450 con 0x7f2ddc05feb0 2020-12-05T17:35:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2ddaffd700 1 -- 172.21.15.32:0/3548630654 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd001d5b0 con 0x7f2ddc05feb0 2020-12-05T17:35:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2ddaffd700 1 -- 172.21.15.32:0/3548630654 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2dd001d910 con 0x7f2ddc05feb0 2020-12-05T17:35:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2de29d4700 1 -- 172.21.15.32:0/3548630654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ddc146730 con 0x7f2ddc05feb0 2020-12-05T17:35:15.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2de29d4700 1 -- 172.21.15.32:0/3548630654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ddc146cd0 con 0x7f2ddc05feb0 2020-12-05T17:35:15.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.959+0000 7f2de29d4700 1 -- 172.21.15.32:0/3548630654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ddc148460 con 0x7f2ddc05feb0 2020-12-05T17:35:15.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.963+0000 7f2ddaffd700 1 -- 172.21.15.32:0/3548630654 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2dd0024030 con 0x7f2ddc05feb0 2020-12-05T17:35:15.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.963+0000 7f2ddaffd700 1 --2- 172.21.15.32:0/3548630654 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2dbc055440 0x7f2dbc057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:15.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.963+0000 7f2ddaffd700 1 -- 172.21.15.32:0/3548630654 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f2dd006f510 con 0x7f2ddc05feb0 2020-12-05T17:35:15.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.963+0000 7f2de19d2700 1 --2- 172.21.15.32:0/3548630654 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2dbc055440 0x7f2dbc057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:15.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.963+0000 7f2de19d2700 1 --2- 172.21.15.32:0/3548630654 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2dbc055440 0x7f2dbc057900 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f2dcc00b000 tx=0x7f2dcc009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:15.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:15.967+0000 7f2ddaffd700 1 -- 172.21.15.32:0/3548630654 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2dd0045660 con 0x7f2ddc05feb0 2020-12-05T17:35:16.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.115+0000 7f2de29d4700 1 -- 172.21.15.32:0/3548630654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2ddc057050 con 0x7f2ddc05feb0 2020-12-05T17:35:16.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.815+0000 7f2ddaffd700 1 -- 172.21.15.32:0/3548630654 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v46) v1 ==== 185+0+0 (secure 0 0 0) 0x7f2dd0022020 con 0x7f2ddc05feb0 2020-12-05T17:35:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.823+0000 7f2de29d4700 1 -- 172.21.15.32:0/3548630654 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2dbc055440 msgr2=0x7f2dbc057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.823+0000 7f2de29d4700 1 --2- 172.21.15.32:0/3548630654 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2dbc055440 0x7f2dbc057900 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f2dcc00b000 tx=0x7f2dcc009040).stop 2020-12-05T17:35:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.823+0000 7f2de29d4700 1 -- 172.21.15.32:0/3548630654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ddc05feb0 msgr2=0x7f2ddc13e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.823+0000 7f2de29d4700 1 --2- 172.21.15.32:0/3548630654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ddc05feb0 0x7f2ddc13e3d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f2dd0002f30 tx=0x7f2dd0015000).stop 2020-12-05T17:35:16.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.823+0000 7f2de29d4700 1 -- 172.21.15.32:0/3548630654 shutdown_connections 2020-12-05T17:35:16.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.823+0000 7f2de29d4700 1 --2- 172.21.15.32:0/3548630654 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2ddc0b2fb0 0x7f2ddc13e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:16.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.823+0000 7f2de29d4700 1 --2- 172.21.15.32:0/3548630654 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2dbc055440 0x7f2dbc057900 secure :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0x7f2dcc00b000 tx=0x7f2dcc009040).stop 2020-12-05T17:35:16.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.823+0000 7f2de29d4700 1 --2- 172.21.15.32:0/3548630654 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2ddc05f3b0 0x7f2ddc0b9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:16.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.823+0000 7f2de29d4700 1 --2- 172.21.15.32:0/3548630654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ddc05feb0 0x7f2ddc13e3d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:16.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.823+0000 7f2de29d4700 1 -- 172.21.15.32:0/3548630654 >> 172.21.15.32:0/3548630654 conn(0x7f2ddc0b8f20 msgr2=0x7f2ddc1430e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:16.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.823+0000 7f2de29d4700 1 -- 172.21.15.32:0/3548630654 shutdown_connections 2020-12-05T17:35:16.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.823+0000 7f2de29d4700 1 -- 172.21.15.32:0/3548630654 wait complete. 2020-12-05T17:35:16.824 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2020-12-05T17:35:16.837 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-12-05T17:35:16.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.951+0000 7f658319c700 1 Processor -- start 2020-12-05T17:35:16.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.951+0000 7f658319c700 1 -- start start 2020-12-05T17:35:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f658319c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f657c0b2b00 0x7f657c13bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f658319c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f657c0aea80 0x7f657c13c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f658319c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f657c0b2ee0 con 0x7f657c0ad230 2020-12-05T17:35:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f658319c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f657c13ed30 con 0x7f657c0aea80 2020-12-05T17:35:16.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f658319c700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f657c0b0c30 con 0x7f657c0b2b00 2020-12-05T17:35:16.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f6573fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f657c0aea80 0x7f657c13c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:16.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f6573fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f657c0aea80 0x7f657c13c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35090/0 (socket says 172.21.15.32:35090) 2020-12-05T17:35:16.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f6573fff700 1 -- 172.21.15.32:0/1753226475 learned_addr learned my addr 172.21.15.32:0/1753226475 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:16.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f6580f38700 1 --2- 172.21.15.32:0/1753226475 >> v2:172.21.15.61:3300/0 conn(0x7f657c0b2b00 0x7f657c13bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f6573fff700 1 -- 172.21.15.32:0/1753226475 >> v2:172.21.15.61:3300/0 conn(0x7f657c0b2b00 msgr2=0x7f657c13bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f6573fff700 1 --2- 172.21.15.32:0/1753226475 >> v2:172.21.15.61:3300/0 conn(0x7f657c0b2b00 0x7f657c13bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f6573fff700 1 -- 172.21.15.32:0/1753226475 >> v1:172.21.15.61:6789/0 conn(0x7f657c0ad230 legacy=0x7f657c13f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f6573fff700 1 -- 172.21.15.32:0/1753226475 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c142de0 con 0x7f657c0aea80 2020-12-05T17:35:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f6580f38700 1 --2- 172.21.15.32:0/1753226475 >> v2:172.21.15.61:3300/0 conn(0x7f657c0b2b00 0x7f657c13bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f6573fff700 1 --2- 172.21.15.32:0/1753226475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f657c0aea80 0x7f657c13c170 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f656c002f30 tx=0x7f656c006f10).ready entity=mon.0 client_cookie=dbb6fbd825c807f2 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:16.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f65737fe700 1 -- 172.21.15.32:0/1753226475 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f656c00cc20 con 0x7f657c0aea80 2020-12-05T17:35:16.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f65737fe700 1 -- 172.21.15.32:0/1753226475 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656c00cd80 con 0x7f657c0aea80 2020-12-05T17:35:16.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f65737fe700 1 -- 172.21.15.32:0/1753226475 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f656c010600 con 0x7f657c0aea80 2020-12-05T17:35:16.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f658319c700 1 -- 172.21.15.32:0/1753226475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f657c0aea80 msgr2=0x7f657c13c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:16.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f658319c700 1 --2- 172.21.15.32:0/1753226475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f657c0aea80 0x7f657c13c170 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f656c002f30 tx=0x7f656c006f10).stop 2020-12-05T17:35:16.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f658319c700 1 -- 172.21.15.32:0/1753226475 shutdown_connections 2020-12-05T17:35:16.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f658319c700 1 --2- 172.21.15.32:0/1753226475 >> v2:172.21.15.61:3300/0 conn(0x7f657c0b2b00 0x7f657c13bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f658319c700 1 --2- 172.21.15.32:0/1753226475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f657c0aea80 0x7f657c13c170 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f658319c700 1 -- 172.21.15.32:0/1753226475 >> 172.21.15.32:0/1753226475 conn(0x7f657c062310 msgr2=0x7f657c0626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f658319c700 1 -- 172.21.15.32:0/1753226475 shutdown_connections 2020-12-05T17:35:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.955+0000 7f658319c700 1 -- 172.21.15.32:0/1753226475 wait complete. 2020-12-05T17:35:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f658319c700 1 Processor -- start 2020-12-05T17:35:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f658319c700 1 -- start start 2020-12-05T17:35:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f658319c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f657c0ad230 0x7f657c13d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f658319c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f657c0aea80 0x7f657c14f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f658319c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f657c0b2b00 0x7f657c1530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f658319c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c113a70 con 0x7f657c0b2b00 2020-12-05T17:35:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f658319c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c152c40 con 0x7f657c0ad230 2020-12-05T17:35:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f658319c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657c1137d0 con 0x7f657c0aea80 2020-12-05T17:35:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f6581739700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f657c0b2b00 0x7f657c1530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f6581739700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f657c0b2b00 0x7f657c1530a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35098/0 (socket says 172.21.15.32:35098) 2020-12-05T17:35:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f6581739700 1 -- 172.21.15.32:0/4176346444 learned_addr learned my addr 172.21.15.32:0/4176346444 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f6573fff700 1 --2- 172.21.15.32:0/4176346444 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f657c0aea80 0x7f657c14f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f6580f38700 1 --2- 172.21.15.32:0/4176346444 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f657c0ad230 0x7f657c13d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f6581739700 1 -- 172.21.15.32:0/4176346444 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f657c0aea80 msgr2=0x7f657c14f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f6581739700 1 --2- 172.21.15.32:0/4176346444 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f657c0aea80 0x7f657c14f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:16.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f6581739700 1 -- 172.21.15.32:0/4176346444 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f657c0ad230 msgr2=0x7f657c13d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:16.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f6581739700 1 --2- 172.21.15.32:0/4176346444 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f657c0ad230 0x7f657c13d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:16.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f6581739700 1 -- 172.21.15.32:0/4176346444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c14f9e0 con 0x7f657c0b2b00 2020-12-05T17:35:16.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f6581739700 1 --2- 172.21.15.32:0/4176346444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f657c0b2b00 0x7f657c1530a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f657c05eee0 tx=0x7f6574007ed0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:16.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f6571ffb700 1 -- 172.21.15.32:0/4176346444 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f657400a7b0 con 0x7f657c0b2b00 2020-12-05T17:35:16.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f6571ffb700 1 -- 172.21.15.32:0/4176346444 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657400a910 con 0x7f657c0b2b00 2020-12-05T17:35:16.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f658319c700 1 -- 172.21.15.32:0/4176346444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f657c142610 con 0x7f657c0b2b00 2020-12-05T17:35:16.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f6571ffb700 1 -- 172.21.15.32:0/4176346444 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f657400ac70 con 0x7f657c0b2b00 2020-12-05T17:35:16.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f658319c700 1 -- 172.21.15.32:0/4176346444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657c0667b0 con 0x7f657c0b2b00 2020-12-05T17:35:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.959+0000 7f6571ffb700 1 -- 172.21.15.32:0/4176346444 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6574019030 con 0x7f657c0b2b00 2020-12-05T17:35:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.963+0000 7f6571ffb700 1 --2- 172.21.15.32:0/4176346444 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f655c0553b0 0x7f655c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.963+0000 7f6580f38700 1 --2- 172.21.15.32:0/4176346444 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f655c0553b0 0x7f655c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.963+0000 7f6571ffb700 1 -- 172.21.15.32:0/4176346444 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f657406d8c0 con 0x7f657c0b2b00 2020-12-05T17:35:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.963+0000 7f658319c700 1 -- 172.21.15.32:0/4176346444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657c152e00 con 0x7f657c0b2b00 2020-12-05T17:35:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.963+0000 7f6580f38700 1 --2- 172.21.15.32:0/4176346444 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f655c0553b0 0x7f655c057870 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f657800c000 tx=0x7f6578008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:16.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:16.971+0000 7f6571ffb700 1 -- 172.21.15.32:0/4176346444 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f657c152e00 con 0x7f657c0b2b00 2020-12-05T17:35:17.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:17.111+0000 7f658319c700 1 -- 172.21.15.32:0/4176346444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f657c152e00 con 0x7f657c0b2b00 2020-12-05T17:35:17.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:17.823+0000 7f6571ffb700 1 -- 172.21.15.32:0/4176346444 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v47) v1 ==== 173+0+0 (secure 0 0 0) 0x7f657c152e00 con 0x7f657c0b2b00 2020-12-05T17:35:17.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:17.827+0000 7f658319c700 1 -- 172.21.15.32:0/4176346444 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f655c0553b0 msgr2=0x7f655c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:17.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:17.827+0000 7f658319c700 1 --2- 172.21.15.32:0/4176346444 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f655c0553b0 0x7f655c057870 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f657800c000 tx=0x7f6578008040).stop 2020-12-05T17:35:17.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:17.827+0000 7f658319c700 1 -- 172.21.15.32:0/4176346444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f657c0b2b00 msgr2=0x7f657c1530a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:17.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:17.827+0000 7f658319c700 1 --2- 172.21.15.32:0/4176346444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f657c0b2b00 0x7f657c1530a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f657c05eee0 tx=0x7f6574007ed0).stop 2020-12-05T17:35:17.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:17.827+0000 7f658319c700 1 -- 172.21.15.32:0/4176346444 shutdown_connections 2020-12-05T17:35:17.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:17.827+0000 7f658319c700 1 --2- 172.21.15.32:0/4176346444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f657c0b2b00 0x7f657c1530a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:17.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:17.827+0000 7f658319c700 1 --2- 172.21.15.32:0/4176346444 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f657c0ad230 0x7f657c13d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:17.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:17.827+0000 7f658319c700 1 --2- 172.21.15.32:0/4176346444 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f655c0553b0 0x7f655c057870 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:17.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:17.827+0000 7f658319c700 1 --2- 172.21.15.32:0/4176346444 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f657c0aea80 0x7f657c14f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:17.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:17.827+0000 7f658319c700 1 -- 172.21.15.32:0/4176346444 >> 172.21.15.32:0/4176346444 conn(0x7f657c062310 msgr2=0x7f657c0b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:17.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:17.827+0000 7f658319c700 1 -- 172.21.15.32:0/4176346444 shutdown_connections 2020-12-05T17:35:17.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:17.827+0000 7f658319c700 1 -- 172.21.15.32:0/4176346444 wait complete. 2020-12-05T17:35:17.830 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2020-12-05T17:35:17.844 INFO:tasks.ceph.mds.a:Restarting daemon 2020-12-05T17:35:17.844 INFO:tasks.ceph.mds.a:Stopping old one... 2020-12-05T17:35:17.845 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-12-05T17:35:17.845 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:35:17.845 INFO:tasks.ceph.mds.b:Restarting daemon 2020-12-05T17:35:17.846 INFO:tasks.ceph.mds.b:Stopping old one... 2020-12-05T17:35:17.846 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-12-05T17:35:17.846 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:35:17.858 INFO:tasks.ceph.mds.b:Stopped 2020-12-05T17:35:17.858 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-12-05T17:35:17.860 INFO:tasks.ceph.mds.a:Stopped 2020-12-05T17:35:17.861 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-12-05T17:35:17.907 INFO:tasks.ceph.mds.b:Started 2020-12-05T17:35:17.908 DEBUG:teuthology.parallel:result is None 2020-12-05T17:35:17.910 INFO:tasks.ceph.mds.a:Started 2020-12-05T17:35:17.910 DEBUG:teuthology.parallel:result is None 2020-12-05T17:35:17.910 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-12-05T17:35:17.976 INFO:tasks.ceph.mds.b.smithi061.stdout:starting mds.b at 2020-12-05T17:35:17.979 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:35:18.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.027+0000 7effb5437700 1 Processor -- start 2020-12-05T17:35:18.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.027+0000 7effb5437700 1 -- start start 2020-12-05T17:35:18.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.027+0000 7effb5437700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7effb00b1fc0 0x7effb00af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:18.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.027+0000 7effb5437700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7effb00ab710 0x7effb0061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:18.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.027+0000 7effb5437700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb0146a90 con 0x7effb0147400 2020-12-05T17:35:18.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.027+0000 7effb5437700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb00b0110 con 0x7effb00b1fc0 2020-12-05T17:35:18.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.027+0000 7effb5437700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb00b23a0 con 0x7effb00ab710 2020-12-05T17:35:18.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.027+0000 7effaffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7effb00b1fc0 0x7effb00af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:18.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.027+0000 7effaffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7effb00b1fc0 0x7effb00af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35114/0 (socket says 172.21.15.32:35114) 2020-12-05T17:35:18.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.027+0000 7effaffff700 1 -- 172.21.15.32:0/518312524 learned_addr learned my addr 172.21.15.32:0/518312524 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:18.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.027+0000 7effaf7fe700 1 --2- 172.21.15.32:0/518312524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7effb00ab710 0x7effb0061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:18.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.027+0000 7effaffff700 1 -- 172.21.15.32:0/518312524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7effb00ab710 msgr2=0x7effb0061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:18.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.027+0000 7effaffff700 1 --2- 172.21.15.32:0/518312524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7effb00ab710 0x7effb0061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:18.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.027+0000 7effaffff700 1 -- 172.21.15.32:0/518312524 >> v1:172.21.15.61:6789/0 conn(0x7effb0147400 legacy=0x7effb0145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:18.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.027+0000 7effaffff700 1 -- 172.21.15.32:0/518312524 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb0146cc0 con 0x7effb00b1fc0 2020-12-05T17:35:18.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.027+0000 7effaffff700 1 --2- 172.21.15.32:0/518312524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effb00b1fc0 0x7effb00af230 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7effa0002f30 tx=0x7effa0006e80).ready entity=mon.0 client_cookie=9e898b5f36169645 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:18.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.031+0000 7effaeffd700 1 -- 172.21.15.32:0/518312524 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effa00097f0 con 0x7effb00b1fc0 2020-12-05T17:35:18.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.031+0000 7effaeffd700 1 -- 172.21.15.32:0/518312524 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa0009950 con 0x7effb00b1fc0 2020-12-05T17:35:18.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.031+0000 7effb5437700 1 -- 172.21.15.32:0/518312524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effb00b1fc0 msgr2=0x7effb00af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:18.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.031+0000 7effb5437700 1 --2- 172.21.15.32:0/518312524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effb00b1fc0 0x7effb00af230 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7effa0002f30 tx=0x7effa0006e80).stop 2020-12-05T17:35:18.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.031+0000 7effaeffd700 1 -- 172.21.15.32:0/518312524 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effa0009cb0 con 0x7effb00b1fc0 2020-12-05T17:35:18.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.031+0000 7effb5437700 1 -- 172.21.15.32:0/518312524 shutdown_connections 2020-12-05T17:35:18.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.031+0000 7effb5437700 1 --2- 172.21.15.32:0/518312524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effb00b1fc0 0x7effb00af230 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:18.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.031+0000 7effb5437700 1 --2- 172.21.15.32:0/518312524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7effb00ab710 0x7effb0061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:18.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.031+0000 7effb5437700 1 -- 172.21.15.32:0/518312524 >> 172.21.15.32:0/518312524 conn(0x7effb00b5530 msgr2=0x7effb00b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:18.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.031+0000 7effb5437700 1 -- 172.21.15.32:0/518312524 shutdown_connections 2020-12-05T17:35:18.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.031+0000 7effb5437700 1 -- 172.21.15.32:0/518312524 wait complete. 2020-12-05T17:35:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.031+0000 7effb5437700 1 Processor -- start 2020-12-05T17:35:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.031+0000 7effb5437700 1 -- start start 2020-12-05T17:35:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effb5437700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7effb00ab710 0x7effb0143260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effb5437700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effb00b1fc0 0x7effb0147e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effb5437700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7effb0147400 0x7effb00ae380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:18.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effb5437700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb0148340 con 0x7effb00b1fc0 2020-12-05T17:35:18.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effb5437700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb0149c20 con 0x7effb0147400 2020-12-05T17:35:18.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effb5437700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb0149ec0 con 0x7effb00ab710 2020-12-05T17:35:18.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effaf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effb00b1fc0 0x7effb0147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:18.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effaf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effb00b1fc0 0x7effb0147e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35122/0 (socket says 172.21.15.32:35122) 2020-12-05T17:35:18.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effaf7fe700 1 -- 172.21.15.32:0/710828325 learned_addr learned my addr 172.21.15.32:0/710828325 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:18.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effaffff700 1 --2- 172.21.15.32:0/710828325 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7effb00ab710 0x7effb0143260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:18.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effb4c36700 1 --2- 172.21.15.32:0/710828325 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7effb0147400 0x7effb00ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:18.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effaf7fe700 1 -- 172.21.15.32:0/710828325 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7effb00ab710 msgr2=0x7effb0143260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:18.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effaf7fe700 1 --2- 172.21.15.32:0/710828325 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7effb00ab710 0x7effb0143260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:18.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effaf7fe700 1 -- 172.21.15.32:0/710828325 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7effb0147400 msgr2=0x7effb00ae380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:18.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effaf7fe700 1 --2- 172.21.15.32:0/710828325 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7effb0147400 0x7effb00ae380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:18.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effaf7fe700 1 -- 172.21.15.32:0/710828325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb014a160 con 0x7effb00b1fc0 2020-12-05T17:35:18.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effaf7fe700 1 --2- 172.21.15.32:0/710828325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effb00b1fc0 0x7effb0147e50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7effa4003070 tx=0x7effa4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:18.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effad7fa700 1 -- 172.21.15.32:0/710828325 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effa4009790 con 0x7effb00b1fc0 2020-12-05T17:35:18.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effad7fa700 1 -- 172.21.15.32:0/710828325 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa40098f0 con 0x7effb00b1fc0 2020-12-05T17:35:18.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effad7fa700 1 -- 172.21.15.32:0/710828325 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effa4009c50 con 0x7effb00b1fc0 2020-12-05T17:35:18.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effb5437700 1 -- 172.21.15.32:0/710828325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb00adee0 con 0x7effb00b1fc0 2020-12-05T17:35:18.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effb5437700 1 -- 172.21.15.32:0/710828325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb00ac930 con 0x7effb00b1fc0 2020-12-05T17:35:18.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effb5437700 1 -- 172.21.15.32:0/710828325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb014ad10 con 0x7effb00b1fc0 2020-12-05T17:35:18.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.035+0000 7effad7fa700 1 -- 172.21.15.32:0/710828325 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7effa4013480 con 0x7effb00b1fc0 2020-12-05T17:35:18.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.039+0000 7effad7fa700 1 --2- 172.21.15.32:0/710828325 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7eff90055440 0x7eff90057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:18.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.039+0000 7effad7fa700 1 -- 172.21.15.32:0/710828325 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5686+0+0 (secure 0 0 0) 0x7effa406e150 con 0x7effb00b1fc0 2020-12-05T17:35:18.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.039+0000 7effaffff700 1 --2- 172.21.15.32:0/710828325 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7eff90055440 0x7eff90057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:18.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.039+0000 7effaffff700 1 --2- 172.21.15.32:0/710828325 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7eff90055440 0x7eff90057900 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7effa001c000 tx=0x7effa0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:18.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.043+0000 7effad7fa700 1 -- 172.21.15.32:0/710828325 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7effb014ad10 con 0x7effb00b1fc0 2020-12-05T17:35:18.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.199+0000 7effb5437700 1 -- 172.21.15.32:0/710828325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7effb014ad10 con 0x7effb00b1fc0 2020-12-05T17:35:18.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.995+0000 7effad7fa700 1 -- 172.21.15.32:0/710828325 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v48) v1 ==== 97+0+0 (secure 0 0 0) 0x7effb014ad10 con 0x7effb00b1fc0 2020-12-05T17:35:19.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.999+0000 7effb5437700 1 -- 172.21.15.32:0/710828325 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7eff90055440 msgr2=0x7eff90057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:19.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.999+0000 7effb5437700 1 --2- 172.21.15.32:0/710828325 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7eff90055440 0x7eff90057900 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7effa001c000 tx=0x7effa0015000).stop 2020-12-05T17:35:19.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.999+0000 7effb5437700 1 -- 172.21.15.32:0/710828325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effb00b1fc0 msgr2=0x7effb0147e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:19.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.999+0000 7effb5437700 1 --2- 172.21.15.32:0/710828325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effb00b1fc0 0x7effb0147e50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7effa4003070 tx=0x7effa4008040).stop 2020-12-05T17:35:19.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.999+0000 7effb5437700 1 -- 172.21.15.32:0/710828325 shutdown_connections 2020-12-05T17:35:19.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.999+0000 7effb5437700 1 --2- 172.21.15.32:0/710828325 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7effb0147400 0x7effb00ae380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.999+0000 7effb5437700 1 --2- 172.21.15.32:0/710828325 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7eff90055440 0x7eff90057900 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.999+0000 7effb5437700 1 --2- 172.21.15.32:0/710828325 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7effb00ab710 0x7effb0143260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.999+0000 7effb5437700 1 --2- 172.21.15.32:0/710828325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effb00b1fc0 0x7effb0147e50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.999+0000 7effb5437700 1 -- 172.21.15.32:0/710828325 >> 172.21.15.32:0/710828325 conn(0x7effb00b5530 msgr2=0x7effb0144880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:19.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.999+0000 7effb5437700 1 -- 172.21.15.32:0/710828325 shutdown_connections 2020-12-05T17:35:19.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:18.999+0000 7effb5437700 1 -- 172.21.15.32:0/710828325 wait complete. 2020-12-05T17:35:19.004 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2020-12-05T17:35:19.017 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2020-12-05T17:35:19.018 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-12-05T17:35:19.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.127+0000 7f4216819700 1 Processor -- start 2020-12-05T17:35:19.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.127+0000 7f4216819700 1 -- start start 2020-12-05T17:35:19.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4216819700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42100b2070 0x7f4210138e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:19.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4216819700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4210139320 0x7f4210143090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:19.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4216819700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42100b2450 con 0x7f42100b2ca0 2020-12-05T17:35:19.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4216819700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42100acb80 con 0x7f4210139320 2020-12-05T17:35:19.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4216819700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4210066d40 con 0x7f42100b2070 2020-12-05T17:35:19.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4216018700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4210139320 0x7f4210143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:19.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4215016700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f42100b2ca0 0x7f4210142980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37780/0 (socket says 172.21.15.32:37780) 2020-12-05T17:35:19.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4215016700 1 -- 172.21.15.32:0/3645676664 learned_addr learned my addr 172.21.15.32:0/3645676664 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:35:19.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4215817700 1 --2- 172.21.15.32:0/3645676664 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42100b2070 0x7f4210138e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:19.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4216018700 1 -- 172.21.15.32:0/3645676664 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42100b2070 msgr2=0x7f4210138e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:19.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4216018700 1 --2- 172.21.15.32:0/3645676664 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42100b2070 0x7f4210138e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4216018700 1 -- 172.21.15.32:0/3645676664 >> v1:172.21.15.32:6789/0 conn(0x7f42100b2ca0 legacy=0x7f4210142980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:19.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4216018700 1 -- 172.21.15.32:0/3645676664 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4210066fe0 con 0x7f4210139320 2020-12-05T17:35:19.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4216018700 1 --2- 172.21.15.32:0/3645676664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4210139320 0x7f4210143090 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f42000010d0 tx=0x7f42080091a0).ready entity=mon.0 client_cookie=8bee972e8a56da05 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:19.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4214815700 1 -- 172.21.15.32:0/3645676664 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f420800b860 con 0x7f4210139320 2020-12-05T17:35:19.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4214815700 1 -- 172.21.15.32:0/3645676664 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420800b9c0 con 0x7f4210139320 2020-12-05T17:35:19.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4214815700 1 -- 172.21.15.32:0/3645676664 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f420800bb30 con 0x7f4210139320 2020-12-05T17:35:19.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4216819700 1 -- 172.21.15.32:0/3645676664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4210139320 msgr2=0x7f4210143090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:19.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4216819700 1 --2- 172.21.15.32:0/3645676664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4210139320 0x7f4210143090 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f42000010d0 tx=0x7f42080091a0).stop 2020-12-05T17:35:19.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4216819700 1 -- 172.21.15.32:0/3645676664 shutdown_connections 2020-12-05T17:35:19.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4216819700 1 --2- 172.21.15.32:0/3645676664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4210139320 0x7f4210143090 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4216819700 1 --2- 172.21.15.32:0/3645676664 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42100b2070 0x7f4210138e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4216819700 1 -- 172.21.15.32:0/3645676664 >> 172.21.15.32:0/3645676664 conn(0x7f421005bbb0 msgr2=0x7f4210062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:19.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4216819700 1 -- 172.21.15.32:0/3645676664 shutdown_connections 2020-12-05T17:35:19.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.131+0000 7f4216819700 1 -- 172.21.15.32:0/3645676664 wait complete. 2020-12-05T17:35:19.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.135+0000 7f4216819700 1 Processor -- start 2020-12-05T17:35:19.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.135+0000 7f4216819700 1 -- start start 2020-12-05T17:35:19.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.135+0000 7f4216819700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42100b2070 0x7f4210145e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:19.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.135+0000 7f4216819700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42100b2ca0 0x7f42101399d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:19.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.135+0000 7f4216819700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4210139320 0x7f421014e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:19.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.135+0000 7f4216819700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4210139ec0 con 0x7f42100b2070 2020-12-05T17:35:19.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.135+0000 7f4216819700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42101523b0 con 0x7f4210139320 2020-12-05T17:35:19.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.135+0000 7f4216819700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4210152650 con 0x7f42100b2ca0 2020-12-05T17:35:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.135+0000 7f4215817700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42100b2070 0x7f4210145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.135+0000 7f4215817700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42100b2070 0x7f4210145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35136/0 (socket says 172.21.15.32:35136) 2020-12-05T17:35:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.135+0000 7f4215817700 1 -- 172.21.15.32:0/19274421 learned_addr learned my addr 172.21.15.32:0/19274421 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.135+0000 7f4215016700 1 --2- 172.21.15.32:0/19274421 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42100b2ca0 0x7f42101399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.135+0000 7f4216018700 1 --2- 172.21.15.32:0/19274421 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4210139320 0x7f421014e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.139+0000 7f4215817700 1 -- 172.21.15.32:0/19274421 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42100b2ca0 msgr2=0x7f42101399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.139+0000 7f4215817700 1 --2- 172.21.15.32:0/19274421 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42100b2ca0 0x7f42101399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.139+0000 7f4215817700 1 -- 172.21.15.32:0/19274421 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4210139320 msgr2=0x7f421014e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:19.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.139+0000 7f4215817700 1 --2- 172.21.15.32:0/19274421 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4210139320 0x7f421014e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.139+0000 7f4215817700 1 -- 172.21.15.32:0/19274421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421013ea50 con 0x7f42100b2070 2020-12-05T17:35:19.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.139+0000 7f4216018700 1 --2- 172.21.15.32:0/19274421 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4210139320 0x7f421014e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:19.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.139+0000 7f4215817700 1 --2- 172.21.15.32:0/19274421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42100b2070 0x7f4210145e10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f420c006a50 tx=0x7f420c006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:19.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.139+0000 7f41feffd700 1 -- 172.21.15.32:0/19274421 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f420c00c9b0 con 0x7f42100b2070 2020-12-05T17:35:19.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.139+0000 7f41feffd700 1 -- 172.21.15.32:0/19274421 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420c00cb10 con 0x7f42100b2070 2020-12-05T17:35:19.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.139+0000 7f4216819700 1 -- 172.21.15.32:0/19274421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4210064900 con 0x7f42100b2070 2020-12-05T17:35:19.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.139+0000 7f41feffd700 1 -- 172.21.15.32:0/19274421 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f420c00ce70 con 0x7f42100b2070 2020-12-05T17:35:19.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.139+0000 7f4216819700 1 -- 172.21.15.32:0/19274421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f421014e750 con 0x7f42100b2070 2020-12-05T17:35:19.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.139+0000 7f4216819700 1 -- 172.21.15.32:0/19274421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42101521f0 con 0x7f42100b2070 2020-12-05T17:35:19.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.139+0000 7f41feffd700 1 -- 172.21.15.32:0/19274421 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f420c01c460 con 0x7f42100b2070 2020-12-05T17:35:19.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.139+0000 7f41feffd700 1 --2- 172.21.15.32:0/19274421 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f41f0055440 0x7f41f0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:19.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.139+0000 7f41feffd700 1 -- 172.21.15.32:0/19274421 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f420c06e230 con 0x7f42100b2070 2020-12-05T17:35:19.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.139+0000 7f4215016700 1 --2- 172.21.15.32:0/19274421 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f41f0055440 0x7f41f0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:19.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.139+0000 7f4215016700 1 --2- 172.21.15.32:0/19274421 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f41f0055440 0x7f41f0057900 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f420000d000 tx=0x7f4200009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:19.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.147+0000 7f41feffd700 1 -- 172.21.15.32:0/19274421 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f420c0446b0 con 0x7f42100b2070 2020-12-05T17:35:19.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.339+0000 7f4216819700 1 -- 172.21.15.32:0/19274421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4210057050 con 0x7f42100b2070 2020-12-05T17:35:19.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.339+0000 7f41feffd700 1 -- 172.21.15.32:0/19274421 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f420c047cd0 con 0x7f42100b2070 2020-12-05T17:35:19.339 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:35:19.339 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:35:19.339 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2020-12-05T17:35:19.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:19.340 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2020-12-05T17:35:19.340 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfpb/aExAAdMKW4yIgC2zhkzyW6l8xug==", 2020-12-05T17:35:19.341 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:35:19.341 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:35:19.341 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:35:19.341 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:35:19.342 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:35:19.342 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:19.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:19.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:19.343 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2020-12-05T17:35:19.343 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfBKTYGRAAaPkezwmTiAvkUfTuDGDNZA==", 2020-12-05T17:35:19.343 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:35:19.343 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:35:19.344 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:35:19.344 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:35:19.344 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:35:19.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:19.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:19.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:19.345 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2020-12-05T17:35:19.345 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHw8tflo5EKhAAwV9D64fAoY3uvXCm+Aksag==", 2020-12-05T17:35:19.345 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:35:19.345 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:35:19.346 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:35:19.346 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:35:19.346 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:19.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:19.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:19.347 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2020-12-05T17:35:19.347 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDKw8tfTjEiERAAkW6mkgApoXwyF49kPjZBGg==", 2020-12-05T17:35:19.347 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:35:19.347 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:35:19.348 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:35:19.348 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:35:19.348 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:19.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:19.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:19.349 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2020-12-05T17:35:19.349 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDMw8tfLEzBMxAAiJ2NifsOLEkt1e3vmhwSTA==", 2020-12-05T17:35:19.349 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:35:19.350 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:35:19.350 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:35:19.351 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:35:19.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:19.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:19.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:19.351 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2020-12-05T17:35:19.352 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPw8tfV+vnGxAAQHvXJRW5X9Vhe/eYJBJ/uQ==", 2020-12-05T17:35:19.352 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:35:19.352 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:35:19.352 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:35:19.353 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:35:19.353 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:19.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:19.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:19.354 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2020-12-05T17:35:19.354 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUw8tfN8wMJxAApc5LY9HGDeIV/o0eil/g4A==", 2020-12-05T17:35:19.354 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:35:19.354 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:35:19.354 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:35:19.355 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:35:19.355 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:19.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:19.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:19.356 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2020-12-05T17:35:19.356 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXw8tfvZ5dDxAAtAfftExkljlBvn7ZSoD/Og==", 2020-12-05T17:35:19.356 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:35:19.356 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:35:19.356 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:35:19.357 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:35:19.357 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:19.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:19.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:19.358 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2020-12-05T17:35:19.358 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDZw8tfhXFzMhAAZXNwNGfKv360Tq4N/DX5Mw==", 2020-12-05T17:35:19.358 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:35:19.359 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:35:19.359 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:35:19.359 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:35:19.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:19.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:19.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:19.360 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2020-12-05T17:35:19.360 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDcw8tfROfSGhAA/Y2Y1b4k5zQju0MM7LASeQ==", 2020-12-05T17:35:19.361 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:35:19.361 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:35:19.361 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:35:19.361 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:35:19.361 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:19.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:19.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:19.362 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2020-12-05T17:35:19.362 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfWDp/HxAAmvLPj7C3WDgj0OC2o7lBJQ==", 2020-12-05T17:35:19.363 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:35:19.363 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:35:19.363 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:35:19.363 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:35:19.363 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:19.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:19.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:19.364 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2020-12-05T17:35:19.364 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfugnmIRAAW6+/QGMLkHw+Z97RHpV4kw==", 2020-12-05T17:35:19.364 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:35:19.365 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:35:19.365 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:35:19.365 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:35:19.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:19.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:19.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:19.366 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2020-12-05T17:35:19.366 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfojyqJBAAyta+dSy2yGnC1kKe3YOAYw==", 2020-12-05T17:35:19.366 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:35:19.367 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:35:19.367 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:35:19.367 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:35:19.367 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:19.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:19.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:19.368 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2020-12-05T17:35:19.368 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDFw8tfES0PLBAAA1cUV44snAUDF2TyAmXImw==", 2020-12-05T17:35:19.368 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:35:19.368 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:35:19.369 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:35:19.369 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:35:19.369 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:35:19.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:19.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:19.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:19.370 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2020-12-05T17:35:19.370 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfDr8mERAAS7bpAQXgW78W8GDq1v1zaA==", 2020-12-05T17:35:19.370 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:35:19.371 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:35:19.371 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:35:19.371 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:35:19.371 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:19.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:19.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:19.372 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2020-12-05T17:35:19.372 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tf3HmcDhAATNKHhH9zgasedaNEzgG0OQ==", 2020-12-05T17:35:19.372 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:35:19.372 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:35:19.373 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:35:19.373 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:35:19.373 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:19.373 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:19.374 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:35:19.374 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:35:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.343+0000 7f4216819700 1 -- 172.21.15.32:0/19274421 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f41f0055440 msgr2=0x7f41f0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.343+0000 7f4216819700 1 --2- 172.21.15.32:0/19274421 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f41f0055440 0x7f41f0057900 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f420000d000 tx=0x7f4200009040).stop 2020-12-05T17:35:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.343+0000 7f4216819700 1 -- 172.21.15.32:0/19274421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42100b2070 msgr2=0x7f4210145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.343+0000 7f4216819700 1 --2- 172.21.15.32:0/19274421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42100b2070 0x7f4210145e10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f420c006a50 tx=0x7f420c006dc0).stop 2020-12-05T17:35:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.343+0000 7f4216819700 1 -- 172.21.15.32:0/19274421 shutdown_connections 2020-12-05T17:35:19.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.343+0000 7f4216819700 1 --2- 172.21.15.32:0/19274421 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4210139320 0x7f421014e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.343+0000 7f4216819700 1 --2- 172.21.15.32:0/19274421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42100b2070 0x7f4210145e10 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.343+0000 7f4216819700 1 --2- 172.21.15.32:0/19274421 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f41f0055440 0x7f41f0057900 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.343+0000 7f4216819700 1 --2- 172.21.15.32:0/19274421 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42100b2ca0 0x7f42101399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.343+0000 7f4216819700 1 -- 172.21.15.32:0/19274421 >> 172.21.15.32:0/19274421 conn(0x7f421005bbb0 msgr2=0x7f42100afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:19.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.343+0000 7f4216819700 1 -- 172.21.15.32:0/19274421 shutdown_connections 2020-12-05T17:35:19.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.343+0000 7f4216819700 1 -- 172.21.15.32:0/19274421 wait complete. 2020-12-05T17:35:19.379 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:35:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e72ff2700 1 Processor -- start 2020-12-05T17:35:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e72ff2700 1 -- start start 2020-12-05T17:35:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e72ff2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e6c13c6f0 0x7f1e6c061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e72ff2700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1e6c0ae4b0 0x7f1e6c0b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e72ff2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e6c062970 con 0x7f1e6c0b78d0 2020-12-05T17:35:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e72ff2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e6c0b2340 con 0x7f1e6c13c6f0 2020-12-05T17:35:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e72ff2700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e6c0add70 con 0x7f1e6c0ae4b0 2020-12-05T17:35:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e71ff0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e6c13c6f0 0x7f1e6c061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e727f1700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1e6c0b78d0 0x7f1e6c145c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37796/0 (socket says 172.21.15.32:37796) 2020-12-05T17:35:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e727f1700 1 -- 172.21.15.32:0/3998647004 learned_addr learned my addr 172.21.15.32:0/3998647004 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:35:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e717ef700 1 --2- 172.21.15.32:0/3998647004 >> v2:172.21.15.61:3300/0 conn(0x7f1e6c0ae4b0 0x7f1e6c0b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e71ff0700 1 -- 172.21.15.32:0/3998647004 >> v2:172.21.15.61:3300/0 conn(0x7f1e6c0ae4b0 msgr2=0x7f1e6c0b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e71ff0700 1 --2- 172.21.15.32:0/3998647004 >> v2:172.21.15.61:3300/0 conn(0x7f1e6c0ae4b0 0x7f1e6c0b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e71ff0700 1 -- 172.21.15.32:0/3998647004 >> v1:172.21.15.32:6789/0 conn(0x7f1e6c0b78d0 legacy=0x7f1e6c145c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e70fee700 1 -- 172.21.15.32:0/3998647004 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1495988171 0 0) 0x7f1e6c062970 con 0x7f1e6c0b78d0 2020-12-05T17:35:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e71ff0700 1 -- 172.21.15.32:0/3998647004 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6c0ae010 con 0x7f1e6c13c6f0 2020-12-05T17:35:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e71ff0700 1 --2- 172.21.15.32:0/3998647004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6c13c6f0 0x7f1e6c061780 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f1e68002f30 tx=0x7f1e68006dc0).ready entity=mon.0 client_cookie=e8f294e77856cc7 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:19.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e70fee700 1 -- 172.21.15.32:0/3998647004 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e680097b0 con 0x7f1e6c13c6f0 2020-12-05T17:35:19.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e70fee700 1 -- 172.21.15.32:0/3998647004 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e68009910 con 0x7f1e6c13c6f0 2020-12-05T17:35:19.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e70fee700 1 -- 172.21.15.32:0/3998647004 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e68009c70 con 0x7f1e6c13c6f0 2020-12-05T17:35:19.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e72ff2700 1 -- 172.21.15.32:0/3998647004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6c13c6f0 msgr2=0x7f1e6c061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:19.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e72ff2700 1 --2- 172.21.15.32:0/3998647004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6c13c6f0 0x7f1e6c061780 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f1e68002f30 tx=0x7f1e68006dc0).stop 2020-12-05T17:35:19.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e72ff2700 1 -- 172.21.15.32:0/3998647004 shutdown_connections 2020-12-05T17:35:19.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e72ff2700 1 --2- 172.21.15.32:0/3998647004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6c13c6f0 0x7f1e6c061780 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e72ff2700 1 --2- 172.21.15.32:0/3998647004 >> v2:172.21.15.61:3300/0 conn(0x7f1e6c0ae4b0 0x7f1e6c0b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e72ff2700 1 -- 172.21.15.32:0/3998647004 >> 172.21.15.32:0/3998647004 conn(0x7f1e6c05bc20 msgr2=0x7f1e6c0b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:19.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e72ff2700 1 -- 172.21.15.32:0/3998647004 shutdown_connections 2020-12-05T17:35:19.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.487+0000 7f1e72ff2700 1 -- 172.21.15.32:0/3998647004 wait complete. 2020-12-05T17:35:19.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e72ff2700 1 Processor -- start 2020-12-05T17:35:19.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e72ff2700 1 -- start start 2020-12-05T17:35:19.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e72ff2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1e6c0ae4b0 0x7f1e6c0aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:19.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e72ff2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6c0b78d0 0x7f1e6c151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:19.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e72ff2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1e6c13c6f0 0x7f1e6c064ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:19.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e72ff2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c1410d0 con 0x7f1e6c0b78d0 2020-12-05T17:35:19.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e72ff2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c151340 con 0x7f1e6c0ae4b0 2020-12-05T17:35:19.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e72ff2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c1515e0 con 0x7f1e6c13c6f0 2020-12-05T17:35:19.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e717ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6c0b78d0 0x7f1e6c151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:19.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e717ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6c0b78d0 0x7f1e6c151d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35152/0 (socket says 172.21.15.32:35152) 2020-12-05T17:35:19.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e717ef700 1 -- 172.21.15.32:0/1625176421 learned_addr learned my addr 172.21.15.32:0/1625176421 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:19.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e71ff0700 1 --2- 172.21.15.32:0/1625176421 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1e6c0ae4b0 0x7f1e6c0aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:19.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e727f1700 1 --2- 172.21.15.32:0/1625176421 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1e6c13c6f0 0x7f1e6c064ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:19.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e717ef700 1 -- 172.21.15.32:0/1625176421 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1e6c13c6f0 msgr2=0x7f1e6c064ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:19.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e717ef700 1 --2- 172.21.15.32:0/1625176421 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1e6c13c6f0 0x7f1e6c064ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e717ef700 1 -- 172.21.15.32:0/1625176421 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1e6c0ae4b0 msgr2=0x7f1e6c0aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:19.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e717ef700 1 --2- 172.21.15.32:0/1625176421 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1e6c0ae4b0 0x7f1e6c0aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e717ef700 1 -- 172.21.15.32:0/1625176421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6c151880 con 0x7f1e6c0b78d0 2020-12-05T17:35:19.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e727f1700 1 --2- 172.21.15.32:0/1625176421 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1e6c13c6f0 0x7f1e6c064ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:19.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e717ef700 1 --2- 172.21.15.32:0/1625176421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6c0b78d0 0x7f1e6c151d20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f1e5c00c000 tx=0x7f1e5c006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:19.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e62ffd700 1 -- 172.21.15.32:0/1625176421 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e5c00ca10 con 0x7f1e6c0b78d0 2020-12-05T17:35:19.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e62ffd700 1 -- 172.21.15.32:0/1625176421 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5c00cb70 con 0x7f1e6c0b78d0 2020-12-05T17:35:19.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e62ffd700 1 -- 172.21.15.32:0/1625176421 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e5c0123f0 con 0x7f1e6c0b78d0 2020-12-05T17:35:19.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e72ff2700 1 -- 172.21.15.32:0/1625176421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e6c152210 con 0x7f1e6c0b78d0 2020-12-05T17:35:19.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.491+0000 7f1e72ff2700 1 -- 172.21.15.32:0/1625176421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e6c140210 con 0x7f1e6c0b78d0 2020-12-05T17:35:19.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.495+0000 7f1e72ff2700 1 -- 172.21.15.32:0/1625176421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e6c140520 con 0x7f1e6c0b78d0 2020-12-05T17:35:19.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.495+0000 7f1e62ffd700 1 -- 172.21.15.32:0/1625176421 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1e5c012550 con 0x7f1e6c0b78d0 2020-12-05T17:35:19.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.495+0000 7f1e62ffd700 1 --2- 172.21.15.32:0/1625176421 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1e54055440 0x7f1e54057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:19.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.495+0000 7f1e62ffd700 1 -- 172.21.15.32:0/1625176421 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f1e5c06e1c0 con 0x7f1e6c0b78d0 2020-12-05T17:35:19.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.495+0000 7f1e71ff0700 1 --2- 172.21.15.32:0/1625176421 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1e54055440 0x7f1e54057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:19.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.495+0000 7f1e71ff0700 1 --2- 172.21.15.32:0/1625176421 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1e54055440 0x7f1e54057900 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f1e6801a000 tx=0x7f1e68015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:19.503 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.499+0000 7f1e62ffd700 1 -- 172.21.15.32:0/1625176421 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1e5c0445a0 con 0x7f1e6c0b78d0 2020-12-05T17:35:19.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.651+0000 7f1e72ff2700 1 -- 172.21.15.32:0/1625176421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e6c151a10 con 0x7f1e6c0b78d0 2020-12-05T17:35:19.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.651+0000 7f1e62ffd700 1 -- 172.21.15.32:0/1625176421 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+13414 (secure 0 0 0) 0x7f1e5c047bc0 con 0x7f1e6c0b78d0 2020-12-05T17:35:19.651 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:35:19.652 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":48,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:35:18.992863+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:35:19.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.655+0000 7f1e72ff2700 1 -- 172.21.15.32:0/1625176421 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1e54055440 msgr2=0x7f1e54057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:19.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.655+0000 7f1e72ff2700 1 --2- 172.21.15.32:0/1625176421 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1e54055440 0x7f1e54057900 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f1e6801a000 tx=0x7f1e68015000).stop 2020-12-05T17:35:19.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.655+0000 7f1e72ff2700 1 -- 172.21.15.32:0/1625176421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6c0b78d0 msgr2=0x7f1e6c151d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:19.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.655+0000 7f1e72ff2700 1 --2- 172.21.15.32:0/1625176421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6c0b78d0 0x7f1e6c151d20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f1e5c00c000 tx=0x7f1e5c006dc0).stop 2020-12-05T17:35:19.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.655+0000 7f1e72ff2700 1 -- 172.21.15.32:0/1625176421 shutdown_connections 2020-12-05T17:35:19.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.655+0000 7f1e72ff2700 1 --2- 172.21.15.32:0/1625176421 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1e54055440 0x7f1e54057900 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.655+0000 7f1e72ff2700 1 --2- 172.21.15.32:0/1625176421 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1e6c13c6f0 0x7f1e6c064ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.655+0000 7f1e72ff2700 1 --2- 172.21.15.32:0/1625176421 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1e6c0ae4b0 0x7f1e6c0aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.655+0000 7f1e72ff2700 1 --2- 172.21.15.32:0/1625176421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6c0b78d0 0x7f1e6c151d20 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.655+0000 7f1e72ff2700 1 -- 172.21.15.32:0/1625176421 >> 172.21.15.32:0/1625176421 conn(0x7f1e6c05bc20 msgr2=0x7f1e6c0b3770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:19.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.655+0000 7f1e72ff2700 1 -- 172.21.15.32:0/1625176421 shutdown_connections 2020-12-05T17:35:19.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.655+0000 7f1e72ff2700 1 -- 172.21.15.32:0/1625176421 wait complete. 2020-12-05T17:35:19.665 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-12-05T17:35:19.666 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:35:19.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.779+0000 7f9209712700 1 Processor -- start 2020-12-05T17:35:19.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.779+0000 7f9209712700 1 -- start start 2020-12-05T17:35:19.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9209712700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9204061920 0x7f9204064a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:19.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9209712700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92040b16b0 0x7f9204146860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:19.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9209712700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92041428b0 0x7f9204142d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:19.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9209712700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9204146d50 con 0x7f92041428b0 2020-12-05T17:35:19.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9209712700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9204143210 con 0x7f9204061920 2020-12-05T17:35:19.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9209712700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9204145e80 con 0x7f92040b16b0 2020-12-05T17:35:19.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9208f11700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92041428b0 0x7f9204142d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:19.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9208f11700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92041428b0 0x7f9204142d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35162/0 (socket says 172.21.15.32:35162) 2020-12-05T17:35:19.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9208f11700 1 -- 172.21.15.32:0/3828605503 learned_addr learned my addr 172.21.15.32:0/3828605503 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:19.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9203fff700 1 --2- 172.21.15.32:0/3828605503 >> v2:172.21.15.61:3300/0 conn(0x7f9204061920 0x7f9204064a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:19.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f91fbfff700 1 --2- 172.21.15.32:0/3828605503 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92040b16b0 0x7f9204146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:19.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9208f11700 1 -- 172.21.15.32:0/3828605503 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92040b16b0 msgr2=0x7f9204146860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:19.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9208f11700 1 --2- 172.21.15.32:0/3828605503 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92040b16b0 0x7f9204146860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9208f11700 1 -- 172.21.15.32:0/3828605503 >> v2:172.21.15.61:3300/0 conn(0x7f9204061920 msgr2=0x7f9204064a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:19.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9208f11700 1 --2- 172.21.15.32:0/3828605503 >> v2:172.21.15.61:3300/0 conn(0x7f9204061920 0x7f9204064a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9208f11700 1 -- 172.21.15.32:0/3828605503 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9204146120 con 0x7f92041428b0 2020-12-05T17:35:19.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9208f11700 1 --2- 172.21.15.32:0/3828605503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92041428b0 0x7f9204142d20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f91fc0032b0 tx=0x7f91fc009270).ready entity=mon.0 client_cookie=e614e3deb2a877f5 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:19.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f92037fe700 1 -- 172.21.15.32:0/3828605503 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91fc009e60 con 0x7f92041428b0 2020-12-05T17:35:19.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f92037fe700 1 -- 172.21.15.32:0/3828605503 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91fc00bb50 con 0x7f92041428b0 2020-12-05T17:35:19.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f92037fe700 1 -- 172.21.15.32:0/3828605503 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91fc00bcc0 con 0x7f92041428b0 2020-12-05T17:35:19.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9209712700 1 -- 172.21.15.32:0/3828605503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92041428b0 msgr2=0x7f9204142d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:19.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9209712700 1 --2- 172.21.15.32:0/3828605503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92041428b0 0x7f9204142d20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f91fc0032b0 tx=0x7f91fc009270).stop 2020-12-05T17:35:19.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9209712700 1 -- 172.21.15.32:0/3828605503 shutdown_connections 2020-12-05T17:35:19.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9209712700 1 --2- 172.21.15.32:0/3828605503 >> v2:172.21.15.61:3300/0 conn(0x7f9204061920 0x7f9204064a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9209712700 1 --2- 172.21.15.32:0/3828605503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92041428b0 0x7f9204142d20 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9209712700 1 --2- 172.21.15.32:0/3828605503 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92040b16b0 0x7f9204146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9209712700 1 -- 172.21.15.32:0/3828605503 >> 172.21.15.32:0/3828605503 conn(0x7f92040b6d10 msgr2=0x7f92040b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9209712700 1 -- 172.21.15.32:0/3828605503 shutdown_connections 2020-12-05T17:35:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9209712700 1 -- 172.21.15.32:0/3828605503 wait complete. 2020-12-05T17:35:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.783+0000 7f9209712700 1 Processor -- start 2020-12-05T17:35:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.787+0000 7f9209712700 1 -- start start 2020-12-05T17:35:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.787+0000 7f9209712700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9204061920 0x7f920413b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.787+0000 7f9209712700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92040b16b0 0x7f9204151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.787+0000 7f9209712700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92041428b0 0x7f9204149420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.787+0000 7f9209712700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9204147aa0 con 0x7f9204061920 2020-12-05T17:35:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.787+0000 7f9209712700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9204151bf0 con 0x7f92040b16b0 2020-12-05T17:35:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.787+0000 7f9209712700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9204067110 con 0x7f92041428b0 2020-12-05T17:35:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.787+0000 7f9203fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9204061920 0x7f920413b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:19.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.787+0000 7f9203fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9204061920 0x7f920413b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35164/0 (socket says 172.21.15.32:35164) 2020-12-05T17:35:19.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.787+0000 7f9203fff700 1 -- 172.21.15.32:0/2974643430 learned_addr learned my addr 172.21.15.32:0/2974643430 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:19.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.787+0000 7f91fbfff700 1 --2- 172.21.15.32:0/2974643430 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92040b16b0 0x7f9204151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:19.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.787+0000 7f9208f11700 1 --2- 172.21.15.32:0/2974643430 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92041428b0 0x7f9204149420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:19.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.787+0000 7f9203fff700 1 -- 172.21.15.32:0/2974643430 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92041428b0 msgr2=0x7f9204149420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:19.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.787+0000 7f9203fff700 1 --2- 172.21.15.32:0/2974643430 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92041428b0 0x7f9204149420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.787+0000 7f9203fff700 1 -- 172.21.15.32:0/2974643430 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92040b16b0 msgr2=0x7f9204151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:19.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.787+0000 7f9203fff700 1 --2- 172.21.15.32:0/2974643430 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92040b16b0 0x7f9204151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.787+0000 7f9203fff700 1 -- 172.21.15.32:0/2974643430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9204147cd0 con 0x7f9204061920 2020-12-05T17:35:19.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.787+0000 7f9203fff700 1 --2- 172.21.15.32:0/2974643430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9204061920 0x7f920413b130 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f91f400c000 tx=0x7f91f4006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:19.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.787+0000 7f9201ffb700 1 -- 172.21.15.32:0/2974643430 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91f400ca10 con 0x7f9204061920 2020-12-05T17:35:19.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.787+0000 7f9201ffb700 1 -- 172.21.15.32:0/2974643430 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f400cb70 con 0x7f9204061920 2020-12-05T17:35:19.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.787+0000 7f9209712700 1 -- 172.21.15.32:0/2974643430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9204065100 con 0x7f9204061920 2020-12-05T17:35:19.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.787+0000 7f9201ffb700 1 -- 172.21.15.32:0/2974643430 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91f40123f0 con 0x7f9204061920 2020-12-05T17:35:19.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.787+0000 7f9209712700 1 -- 172.21.15.32:0/2974643430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9204065fe0 con 0x7f9204061920 2020-12-05T17:35:19.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.791+0000 7f9209712700 1 -- 172.21.15.32:0/2974643430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92040672d0 con 0x7f9204061920 2020-12-05T17:35:19.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.791+0000 7f9201ffb700 1 -- 172.21.15.32:0/2974643430 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f91f4012550 con 0x7f9204061920 2020-12-05T17:35:19.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.791+0000 7f9201ffb700 1 --2- 172.21.15.32:0/2974643430 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f91e8055440 0x7f91e8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:19.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.791+0000 7f9201ffb700 1 -- 172.21.15.32:0/2974643430 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f91f406e200 con 0x7f9204061920 2020-12-05T17:35:19.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.791+0000 7f91fbfff700 1 --2- 172.21.15.32:0/2974643430 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f91e8055440 0x7f91e8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:19.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.791+0000 7f91fbfff700 1 --2- 172.21.15.32:0/2974643430 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f91e8055440 0x7f91e8057900 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f91ec00b000 tx=0x7f91ec006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:19.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.795+0000 7f9201ffb700 1 -- 172.21.15.32:0/2974643430 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f91f4044680 con 0x7f9204061920 2020-12-05T17:35:19.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.939+0000 7f9209712700 1 -- 172.21.15.32:0/2974643430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9204147e60 con 0x7f9204061920 2020-12-05T17:35:19.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.939+0000 7f9201ffb700 1 -- 172.21.15.32:0/2974643430 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v48) v1 ==== 110+0+10 (secure 0 0 0) 0x7f91f4047ca0 con 0x7f9204061920 2020-12-05T17:35:19.940 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:35:19.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.943+0000 7f9209712700 1 -- 172.21.15.32:0/2974643430 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f91e8055440 msgr2=0x7f91e8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:19.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.943+0000 7f9209712700 1 --2- 172.21.15.32:0/2974643430 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f91e8055440 0x7f91e8057900 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f91ec00b000 tx=0x7f91ec006dc0).stop 2020-12-05T17:35:19.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.943+0000 7f9209712700 1 -- 172.21.15.32:0/2974643430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9204061920 msgr2=0x7f920413b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:19.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.943+0000 7f9209712700 1 --2- 172.21.15.32:0/2974643430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9204061920 0x7f920413b130 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f91f400c000 tx=0x7f91f4006dc0).stop 2020-12-05T17:35:19.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.943+0000 7f9209712700 1 -- 172.21.15.32:0/2974643430 shutdown_connections 2020-12-05T17:35:19.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.943+0000 7f9209712700 1 --2- 172.21.15.32:0/2974643430 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f91e8055440 0x7f91e8057900 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.943+0000 7f9209712700 1 --2- 172.21.15.32:0/2974643430 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92041428b0 0x7f9204149420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.943+0000 7f9209712700 1 --2- 172.21.15.32:0/2974643430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9204061920 0x7f920413b130 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.943+0000 7f9209712700 1 --2- 172.21.15.32:0/2974643430 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92040b16b0 0x7f9204151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:19.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.943+0000 7f9209712700 1 -- 172.21.15.32:0/2974643430 >> 172.21.15.32:0/2974643430 conn(0x7f92040b6d10 msgr2=0x7f9204145100 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:19.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.943+0000 7f9209712700 1 -- 172.21.15.32:0/2974643430 shutdown_connections 2020-12-05T17:35:19.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:19.943+0000 7f9209712700 1 -- 172.21.15.32:0/2974643430 wait complete. 2020-12-05T17:35:19.953 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:35:20.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.071+0000 7f491956f700 1 Processor -- start 2020-12-05T17:35:20.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.071+0000 7f491956f700 1 -- start start 2020-12-05T17:35:20.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.071+0000 7f491956f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f49140b2000 0x7f4914145e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:20.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.071+0000 7f491956f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49140af4b0 0x7f49141462f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:20.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.071+0000 7f491956f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f4914061780 0x7f49140afe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:20.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.071+0000 7f491956f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49140b23e0 con 0x7f49140b2000 2020-12-05T17:35:20.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.071+0000 7f491956f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49140ae7b0 con 0x7f4914061780 2020-12-05T17:35:20.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.071+0000 7f491956f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49140aea50 con 0x7f49140af4b0 2020-12-05T17:35:20.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.071+0000 7f4913fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f49140b2000 0x7f4914145e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:20.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.071+0000 7f4913fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f49140b2000 0x7f4914145e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35176/0 (socket says 172.21.15.32:35176) 2020-12-05T17:35:20.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.071+0000 7f4913fff700 1 -- 172.21.15.32:0/2638727524 learned_addr learned my addr 172.21.15.32:0/2638727524 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:20.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.071+0000 7f4918d6e700 1 --2- 172.21.15.32:0/2638727524 >> v2:172.21.15.61:3300/0 conn(0x7f4914061780 0x7f49140afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:20.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.071+0000 7f49137fe700 1 --2- 172.21.15.32:0/2638727524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49140af4b0 0x7f49141462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:20.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.075+0000 7f4913fff700 1 -- 172.21.15.32:0/2638727524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49140af4b0 msgr2=0x7f49141462f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:20.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.075+0000 7f4913fff700 1 --2- 172.21.15.32:0/2638727524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49140af4b0 0x7f49141462f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.075+0000 7f4913fff700 1 -- 172.21.15.32:0/2638727524 >> v2:172.21.15.61:3300/0 conn(0x7f4914061780 msgr2=0x7f49140afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:20.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.075+0000 7f4913fff700 1 --2- 172.21.15.32:0/2638727524 >> v2:172.21.15.61:3300/0 conn(0x7f4914061780 0x7f49140afe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.075+0000 7f4913fff700 1 -- 172.21.15.32:0/2638727524 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49140b0f60 con 0x7f49140b2000 2020-12-05T17:35:20.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.075+0000 7f4913fff700 1 --2- 172.21.15.32:0/2638727524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49140b2000 0x7f4914145e00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f49040032b0 tx=0x7f4904009270).ready entity=mon.0 client_cookie=d5e484e84414d099 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:20.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.075+0000 7f4912ffd700 1 -- 172.21.15.32:0/2638727524 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4904009e60 con 0x7f49140b2000 2020-12-05T17:35:20.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.075+0000 7f4912ffd700 1 -- 172.21.15.32:0/2638727524 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490400bb50 con 0x7f49140b2000 2020-12-05T17:35:20.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.075+0000 7f4912ffd700 1 -- 172.21.15.32:0/2638727524 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f490400bcc0 con 0x7f49140b2000 2020-12-05T17:35:20.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.075+0000 7f491956f700 1 -- 172.21.15.32:0/2638727524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49140b2000 msgr2=0x7f4914145e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:20.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.075+0000 7f491956f700 1 --2- 172.21.15.32:0/2638727524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49140b2000 0x7f4914145e00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f49040032b0 tx=0x7f4904009270).stop 2020-12-05T17:35:20.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.075+0000 7f491956f700 1 -- 172.21.15.32:0/2638727524 shutdown_connections 2020-12-05T17:35:20.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.075+0000 7f491956f700 1 --2- 172.21.15.32:0/2638727524 >> v2:172.21.15.61:3300/0 conn(0x7f4914061780 0x7f49140afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.075+0000 7f491956f700 1 --2- 172.21.15.32:0/2638727524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49140b2000 0x7f4914145e00 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.075+0000 7f491956f700 1 --2- 172.21.15.32:0/2638727524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49140af4b0 0x7f49141462f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.075+0000 7f491956f700 1 -- 172.21.15.32:0/2638727524 >> 172.21.15.32:0/2638727524 conn(0x7f49140b5620 msgr2=0x7f49140b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:20.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.075+0000 7f491956f700 1 -- 172.21.15.32:0/2638727524 shutdown_connections 2020-12-05T17:35:20.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.075+0000 7f491956f700 1 -- 172.21.15.32:0/2638727524 wait complete. 2020-12-05T17:35:20.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.075+0000 7f491956f700 1 Processor -- start 2020-12-05T17:35:20.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.075+0000 7f491956f700 1 -- start start 2020-12-05T17:35:20.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f491956f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4914061780 0x7f49140b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:20.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f491956f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49140af4b0 0x7f491414c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:20.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f491956f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49140b2000 0x7f4914156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:20.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f491956f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49140b3ae0 con 0x7f49140af4b0 2020-12-05T17:35:20.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f491956f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49140b3d80 con 0x7f4914061780 2020-12-05T17:35:20.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f491956f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4914063b30 con 0x7f49140b2000 2020-12-05T17:35:20.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f49137fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49140af4b0 0x7f491414c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:20.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f49137fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49140af4b0 0x7f491414c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35180/0 (socket says 172.21.15.32:35180) 2020-12-05T17:35:20.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f49137fe700 1 -- 172.21.15.32:0/1107162680 learned_addr learned my addr 172.21.15.32:0/1107162680 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:20.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f4918d6e700 1 --2- 172.21.15.32:0/1107162680 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49140b2000 0x7f4914156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:20.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f4913fff700 1 --2- 172.21.15.32:0/1107162680 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4914061780 0x7f49140b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:20.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f49137fe700 1 -- 172.21.15.32:0/1107162680 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49140b2000 msgr2=0x7f4914156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:20.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f49137fe700 1 --2- 172.21.15.32:0/1107162680 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49140b2000 0x7f4914156410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f49137fe700 1 -- 172.21.15.32:0/1107162680 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4914061780 msgr2=0x7f49140b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:20.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f49137fe700 1 --2- 172.21.15.32:0/1107162680 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4914061780 0x7f49140b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f49137fe700 1 -- 172.21.15.32:0/1107162680 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49140b4020 con 0x7f49140af4b0 2020-12-05T17:35:20.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f49137fe700 1 --2- 172.21.15.32:0/1107162680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49140af4b0 0x7f491414c750 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f48fc00c000 tx=0x7f48fc006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:20.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f49117fa700 1 -- 172.21.15.32:0/1107162680 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48fc00ca10 con 0x7f49140af4b0 2020-12-05T17:35:20.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f49117fa700 1 -- 172.21.15.32:0/1107162680 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48fc00cb70 con 0x7f49140af4b0 2020-12-05T17:35:20.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f491956f700 1 -- 172.21.15.32:0/1107162680 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4914155bc0 con 0x7f49140af4b0 2020-12-05T17:35:20.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f49117fa700 1 -- 172.21.15.32:0/1107162680 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48fc0123f0 con 0x7f49140af4b0 2020-12-05T17:35:20.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f491956f700 1 -- 172.21.15.32:0/1107162680 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49140b3780 con 0x7f49140af4b0 2020-12-05T17:35:20.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f491956f700 1 -- 172.21.15.32:0/1107162680 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4914057050 con 0x7f49140af4b0 2020-12-05T17:35:20.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.079+0000 7f49117fa700 1 -- 172.21.15.32:0/1107162680 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f48fc012550 con 0x7f49140af4b0 2020-12-05T17:35:20.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.083+0000 7f49117fa700 1 --2- 172.21.15.32:0/1107162680 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f48ec0554e0 0x7f48ec0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:20.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.083+0000 7f49117fa700 1 -- 172.21.15.32:0/1107162680 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f48fc06e230 con 0x7f49140af4b0 2020-12-05T17:35:20.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.083+0000 7f4913fff700 1 --2- 172.21.15.32:0/1107162680 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f48ec0554e0 0x7f48ec0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:20.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.083+0000 7f4913fff700 1 --2- 172.21.15.32:0/1107162680 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f48ec0554e0 0x7f48ec0579a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f490401b000 tx=0x7f4904016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:20.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.091+0000 7f49117fa700 1 -- 172.21.15.32:0/1107162680 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f48fc0445a0 con 0x7f49140af4b0 2020-12-05T17:35:20.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.235+0000 7f491956f700 1 -- 172.21.15.32:0/1107162680 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f491414ad80 con 0x7f49140af4b0 2020-12-05T17:35:20.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.235+0000 7f49117fa700 1 -- 172.21.15.32:0/1107162680 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v48) v1 ==== 92+0+10 (secure 0 0 0) 0x7f48fc047bc0 con 0x7f49140af4b0 2020-12-05T17:35:20.233 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:35:20.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.235+0000 7f491956f700 1 -- 172.21.15.32:0/1107162680 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f48ec0554e0 msgr2=0x7f48ec0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:20.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.235+0000 7f491956f700 1 --2- 172.21.15.32:0/1107162680 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f48ec0554e0 0x7f48ec0579a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f490401b000 tx=0x7f4904016000).stop 2020-12-05T17:35:20.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.235+0000 7f491956f700 1 -- 172.21.15.32:0/1107162680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49140af4b0 msgr2=0x7f491414c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:20.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.235+0000 7f491956f700 1 --2- 172.21.15.32:0/1107162680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49140af4b0 0x7f491414c750 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f48fc00c000 tx=0x7f48fc006dc0).stop 2020-12-05T17:35:20.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.239+0000 7f491956f700 1 -- 172.21.15.32:0/1107162680 shutdown_connections 2020-12-05T17:35:20.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.239+0000 7f491956f700 1 --2- 172.21.15.32:0/1107162680 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f48ec0554e0 0x7f48ec0579a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.239+0000 7f491956f700 1 --2- 172.21.15.32:0/1107162680 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49140b2000 0x7f4914156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.239+0000 7f491956f700 1 --2- 172.21.15.32:0/1107162680 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4914061780 0x7f49140b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.239+0000 7f491956f700 1 --2- 172.21.15.32:0/1107162680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49140af4b0 0x7f491414c750 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.239+0000 7f491956f700 1 -- 172.21.15.32:0/1107162680 >> 172.21.15.32:0/1107162680 conn(0x7f49140b5620 msgr2=0x7f4914147fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:20.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.239+0000 7f491956f700 1 -- 172.21.15.32:0/1107162680 shutdown_connections 2020-12-05T17:35:20.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.239+0000 7f491956f700 1 -- 172.21.15.32:0/1107162680 wait complete. 2020-12-05T17:35:20.252 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-12-05T17:35:20.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.363+0000 7f23ee5ed700 1 Processor -- start 2020-12-05T17:35:20.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.363+0000 7f23ee5ed700 1 -- start start 2020-12-05T17:35:20.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.363+0000 7f23ee5ed700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23e80afe50 0x7f23e8146a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:20.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.363+0000 7f23ee5ed700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23e80b1880 0x7f23e81430f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:20.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.363+0000 7f23ee5ed700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e80b0230 con 0x7f23e80b3aa0 2020-12-05T17:35:20.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.363+0000 7f23ee5ed700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23e80b6a50 con 0x7f23e80afe50 2020-12-05T17:35:20.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.363+0000 7f23ee5ed700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e8146080 con 0x7f23e80b1880 2020-12-05T17:35:20.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.363+0000 7f23e7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23e80afe50 0x7f23e8146a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:20.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.363+0000 7f23e7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23e80afe50 0x7f23e8146a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35186/0 (socket says 172.21.15.32:35186) 2020-12-05T17:35:20.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.363+0000 7f23e7fff700 1 -- 172.21.15.32:0/2545299055 learned_addr learned my addr 172.21.15.32:0/2545299055 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:20.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.363+0000 7f23ecb8a700 1 --2- 172.21.15.32:0/2545299055 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23e80b1880 0x7f23e81430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:20.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.363+0000 7f23e7fff700 1 -- 172.21.15.32:0/2545299055 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23e80b1880 msgr2=0x7f23e81430f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:20.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.363+0000 7f23e7fff700 1 --2- 172.21.15.32:0/2545299055 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23e80b1880 0x7f23e81430f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.363+0000 7f23e7fff700 1 -- 172.21.15.32:0/2545299055 >> v1:172.21.15.61:6789/0 conn(0x7f23e80b3aa0 legacy=0x7f23e81429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:20.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.363+0000 7f23e7fff700 1 -- 172.21.15.32:0/2545299055 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e8146320 con 0x7f23e80afe50 2020-12-05T17:35:20.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.363+0000 7f23ecb8a700 1 --2- 172.21.15.32:0/2545299055 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23e80b1880 0x7f23e81430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:20.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.363+0000 7f23e7fff700 1 --2- 172.21.15.32:0/2545299055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e80afe50 0x7f23e8146a60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f23d80032b0 tx=0x7f23d8009270).ready entity=mon.0 client_cookie=22695b40dc2708bd server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:20.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.363+0000 7f23e6ffd700 1 -- 172.21.15.32:0/2545299055 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23d800bc00 con 0x7f23e80afe50 2020-12-05T17:35:20.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.363+0000 7f23e6ffd700 1 -- 172.21.15.32:0/2545299055 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d800bd60 con 0x7f23e80afe50 2020-12-05T17:35:20.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.363+0000 7f23e6ffd700 1 -- 172.21.15.32:0/2545299055 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23d8011400 con 0x7f23e80afe50 2020-12-05T17:35:20.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.363+0000 7f23ee5ed700 1 -- 172.21.15.32:0/2545299055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e80afe50 msgr2=0x7f23e8146a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:20.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.363+0000 7f23ee5ed700 1 --2- 172.21.15.32:0/2545299055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e80afe50 0x7f23e8146a60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f23d80032b0 tx=0x7f23d8009270).stop 2020-12-05T17:35:20.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23ee5ed700 1 -- 172.21.15.32:0/2545299055 shutdown_connections 2020-12-05T17:35:20.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23ee5ed700 1 --2- 172.21.15.32:0/2545299055 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23e80b1880 0x7f23e81430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23ee5ed700 1 --2- 172.21.15.32:0/2545299055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e80afe50 0x7f23e8146a60 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23ee5ed700 1 -- 172.21.15.32:0/2545299055 >> 172.21.15.32:0/2545299055 conn(0x7f23e80b9100 msgr2=0x7f23e80b9510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:20.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23ee5ed700 1 -- 172.21.15.32:0/2545299055 shutdown_connections 2020-12-05T17:35:20.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23ee5ed700 1 -- 172.21.15.32:0/2545299055 wait complete. 2020-12-05T17:35:20.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23ee5ed700 1 Processor -- start 2020-12-05T17:35:20.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23ee5ed700 1 -- start start 2020-12-05T17:35:20.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23ee5ed700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23e80afe50 0x7f23e81523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:20.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23ee5ed700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23e80b1880 0x7f23e8063620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:20.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23ee5ed700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e80b3aa0 0x7f23e8063b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:20.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23ee5ed700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e8140820 con 0x7f23e80b3aa0 2020-12-05T17:35:20.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23ee5ed700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e8140a10 con 0x7f23e80afe50 2020-12-05T17:35:20.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23ee5ed700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e8140cb0 con 0x7f23e80b1880 2020-12-05T17:35:20.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23ecb8a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e80b3aa0 0x7f23e8063b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:20.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23ecb8a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e80b3aa0 0x7f23e8063b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35196/0 (socket says 172.21.15.32:35196) 2020-12-05T17:35:20.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23e7fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23e80afe50 0x7f23e81523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:20.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23ecb8a700 1 -- 172.21.15.32:0/2056245860 learned_addr learned my addr 172.21.15.32:0/2056245860 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:20.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23e77fe700 1 --2- 172.21.15.32:0/2056245860 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23e80b1880 0x7f23e8063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:20.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23ecb8a700 1 -- 172.21.15.32:0/2056245860 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23e80b1880 msgr2=0x7f23e8063620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:20.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23ecb8a700 1 --2- 172.21.15.32:0/2056245860 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23e80b1880 0x7f23e8063620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23ecb8a700 1 -- 172.21.15.32:0/2056245860 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23e80afe50 msgr2=0x7f23e81523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:20.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23ecb8a700 1 --2- 172.21.15.32:0/2056245860 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23e80afe50 0x7f23e81523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23ecb8a700 1 -- 172.21.15.32:0/2056245860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e81528d0 con 0x7f23e80b3aa0 2020-12-05T17:35:20.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.367+0000 7f23ecb8a700 1 --2- 172.21.15.32:0/2056245860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e80b3aa0 0x7f23e8063b10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f23e0006910 tx=0x7f23e0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:20.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.371+0000 7f23e57fa700 1 -- 172.21.15.32:0/2056245860 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23e00127e0 con 0x7f23e80b3aa0 2020-12-05T17:35:20.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.371+0000 7f23e57fa700 1 -- 172.21.15.32:0/2056245860 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e0012940 con 0x7f23e80b3aa0 2020-12-05T17:35:20.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.371+0000 7f23e57fa700 1 -- 172.21.15.32:0/2056245860 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23e0012ca0 con 0x7f23e80b3aa0 2020-12-05T17:35:20.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.371+0000 7f23ee5ed700 1 -- 172.21.15.32:0/2056245860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e813e5a0 con 0x7f23e80b3aa0 2020-12-05T17:35:20.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.371+0000 7f23ee5ed700 1 -- 172.21.15.32:0/2056245860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e8142b60 con 0x7f23e80b3aa0 2020-12-05T17:35:20.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.371+0000 7f23ee5ed700 1 -- 172.21.15.32:0/2056245860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e8140e70 con 0x7f23e80b3aa0 2020-12-05T17:35:20.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.371+0000 7f23e57fa700 1 -- 172.21.15.32:0/2056245860 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f23e0013480 con 0x7f23e80b3aa0 2020-12-05T17:35:20.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.371+0000 7f23e57fa700 1 --2- 172.21.15.32:0/2056245860 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f23c8055440 0x7f23c8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:20.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.371+0000 7f23e7fff700 1 --2- 172.21.15.32:0/2056245860 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f23c8055440 0x7f23c8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:20.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.371+0000 7f23e57fa700 1 -- 172.21.15.32:0/2056245860 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f23e006e450 con 0x7f23e80b3aa0 2020-12-05T17:35:20.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.371+0000 7f23e7fff700 1 --2- 172.21.15.32:0/2056245860 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f23c8055440 0x7f23c8057900 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f23d801a000 tx=0x7f23d8016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:20.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.379+0000 7f23e57fa700 1 -- 172.21.15.32:0/2056245860 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f23e00448d0 con 0x7f23e80b3aa0 2020-12-05T17:35:20.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.519+0000 7f23ee5ed700 1 -- 172.21.15.32:0/2056245860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f23e8057050 con 0x7f23e80b3aa0 2020-12-05T17:35:20.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.523+0000 7f23e57fa700 1 -- 172.21.15.32:0/2056245860 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v48) v1 ==== 84+0+143 (secure 0 0 0) 0x7f23e0047ef0 con 0x7f23e80b3aa0 2020-12-05T17:35:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:35:20.521 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2020-12-05T17:35:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2020-12-05T17:35:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2020-12-05T17:35:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:35:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2020-12-05T17:35:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2020-12-05T17:35:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:20.523 INFO:teuthology.orchestra.run.smithi032.stdout:] 2020-12-05T17:35:20.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.523+0000 7f23ee5ed700 1 -- 172.21.15.32:0/2056245860 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f23c8055440 msgr2=0x7f23c8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:20.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.523+0000 7f23ee5ed700 1 --2- 172.21.15.32:0/2056245860 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f23c8055440 0x7f23c8057900 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f23d801a000 tx=0x7f23d8016000).stop 2020-12-05T17:35:20.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.523+0000 7f23ee5ed700 1 -- 172.21.15.32:0/2056245860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e80b3aa0 msgr2=0x7f23e8063b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:20.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.523+0000 7f23ee5ed700 1 --2- 172.21.15.32:0/2056245860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e80b3aa0 0x7f23e8063b10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f23e0006910 tx=0x7f23e0008040).stop 2020-12-05T17:35:20.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.523+0000 7f23ee5ed700 1 -- 172.21.15.32:0/2056245860 shutdown_connections 2020-12-05T17:35:20.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.523+0000 7f23ee5ed700 1 --2- 172.21.15.32:0/2056245860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e80b3aa0 0x7f23e8063b10 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.523+0000 7f23ee5ed700 1 --2- 172.21.15.32:0/2056245860 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f23e80afe50 0x7f23e81523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.523+0000 7f23ee5ed700 1 --2- 172.21.15.32:0/2056245860 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f23c8055440 0x7f23c8057900 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.523+0000 7f23ee5ed700 1 --2- 172.21.15.32:0/2056245860 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f23e80b1880 0x7f23e8063620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.523+0000 7f23ee5ed700 1 -- 172.21.15.32:0/2056245860 >> 172.21.15.32:0/2056245860 conn(0x7f23e80b9100 msgr2=0x7f23e80b7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:20.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.523+0000 7f23ee5ed700 1 -- 172.21.15.32:0/2056245860 shutdown_connections 2020-12-05T17:35:20.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.523+0000 7f23ee5ed700 1 -- 172.21.15.32:0/2056245860 wait complete. 2020-12-05T17:35:20.534 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-12-05T17:35:20.535 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-12-05T17:35:20.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.647+0000 7fc4b6d51700 1 Processor -- start 2020-12-05T17:35:20.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.647+0000 7fc4b6d51700 1 -- start start 2020-12-05T17:35:20.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b6d51700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc4b005f6f0 0x7fc4b0141700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:20.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b6d51700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc4b0142300 0x7fc4b0142790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:20.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b6d51700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b01439c0 con 0x7fc4b0061b30 2020-12-05T17:35:20.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b6d51700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b005fe20 con 0x7fc4b005f6f0 2020-12-05T17:35:20.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b6d51700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b0146910 con 0x7fc4b0142300 2020-12-05T17:35:20.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b5d4f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc4b005f6f0 0x7fc4b0141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:20.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b5d4f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc4b005f6f0 0x7fc4b0141700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35202/0 (socket says 172.21.15.32:35202) 2020-12-05T17:35:20.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b5d4f700 1 -- 172.21.15.32:0/950650845 learned_addr learned my addr 172.21.15.32:0/950650845 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:20.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b6550700 1 --2- 172.21.15.32:0/950650845 >> v2:172.21.15.61:3300/0 conn(0x7fc4b0142300 0x7fc4b0142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:20.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b4d4d700 1 -- 172.21.15.32:0/950650845 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697789250 0 0) 0x7fc4b01439c0 con 0x7fc4b0061b30 2020-12-05T17:35:20.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b4d4d700 1 -- 172.21.15.32:0/950650845 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc498003230 con 0x7fc4b0061b30 2020-12-05T17:35:20.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b5d4f700 1 -- 172.21.15.32:0/950650845 >> v2:172.21.15.61:3300/0 conn(0x7fc4b0142300 msgr2=0x7fc4b0142790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:20.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b5d4f700 1 --2- 172.21.15.32:0/950650845 >> v2:172.21.15.61:3300/0 conn(0x7fc4b0142300 0x7fc4b0142790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b5d4f700 1 -- 172.21.15.32:0/950650845 >> v1:172.21.15.61:6789/0 conn(0x7fc4b0061b30 legacy=0x7fc4b0141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:20.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b5d4f700 1 -- 172.21.15.32:0/950650845 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b005fad0 con 0x7fc4b005f6f0 2020-12-05T17:35:20.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b5d4f700 1 --2- 172.21.15.32:0/950650845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4b005f6f0 0x7fc4b0141700 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fc4ac002f30 tx=0x7fc4ac006e20).ready entity=mon.0 client_cookie=4e0d5e85673cf9c0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:20.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b4d4d700 1 -- 172.21.15.32:0/950650845 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4ac0097b0 con 0x7fc4b005f6f0 2020-12-05T17:35:20.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b4d4d700 1 -- 172.21.15.32:0/950650845 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ac009910 con 0x7fc4b005f6f0 2020-12-05T17:35:20.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b4d4d700 1 -- 172.21.15.32:0/950650845 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4ac009c70 con 0x7fc4b005f6f0 2020-12-05T17:35:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b6d51700 1 -- 172.21.15.32:0/950650845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4b005f6f0 msgr2=0x7fc4b0141700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b6d51700 1 --2- 172.21.15.32:0/950650845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4b005f6f0 0x7fc4b0141700 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fc4ac002f30 tx=0x7fc4ac006e20).stop 2020-12-05T17:35:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b6d51700 1 -- 172.21.15.32:0/950650845 shutdown_connections 2020-12-05T17:35:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b6d51700 1 --2- 172.21.15.32:0/950650845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4b005f6f0 0x7fc4b0141700 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b6d51700 1 --2- 172.21.15.32:0/950650845 >> v2:172.21.15.61:3300/0 conn(0x7fc4b0142300 0x7fc4b0142790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b6d51700 1 -- 172.21.15.32:0/950650845 >> 172.21.15.32:0/950650845 conn(0x7fc4b0061010 msgr2=0x7fc4b0061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:20.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b6d51700 1 -- 172.21.15.32:0/950650845 shutdown_connections 2020-12-05T17:35:20.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.651+0000 7fc4b6d51700 1 -- 172.21.15.32:0/950650845 wait complete. 2020-12-05T17:35:20.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc4b6d51700 1 Processor -- start 2020-12-05T17:35:20.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc4b6d51700 1 -- start start 2020-12-05T17:35:20.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc4b6d51700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4b005f6f0 0x7fc4b00b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:20.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc4b6d51700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4b0061b30 0x7fc4b0158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:20.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc4b6d51700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4b0142300 0x7fc4b0065df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:20.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc4b6d51700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b0159d60 con 0x7fc4b005f6f0 2020-12-05T17:35:20.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc4b6d51700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b00637f0 con 0x7fc4b0061b30 2020-12-05T17:35:20.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc4b6d51700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b00b6340 con 0x7fc4b0142300 2020-12-05T17:35:20.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc4b5d4f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4b005f6f0 0x7fc4b00b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:20.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc4b5d4f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4b005f6f0 0x7fc4b00b3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35208/0 (socket says 172.21.15.32:35208) 2020-12-05T17:35:20.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc4b5d4f700 1 -- 172.21.15.32:0/1116680656 learned_addr learned my addr 172.21.15.32:0/1116680656 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:20.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc4b554e700 1 --2- 172.21.15.32:0/1116680656 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4b0061b30 0x7fc4b0158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:20.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc4b6550700 1 --2- 172.21.15.32:0/1116680656 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4b0142300 0x7fc4b0065df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:20.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc4b5d4f700 1 -- 172.21.15.32:0/1116680656 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4b0142300 msgr2=0x7fc4b0065df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:20.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc4b5d4f700 1 --2- 172.21.15.32:0/1116680656 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4b0142300 0x7fc4b0065df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc4b5d4f700 1 -- 172.21.15.32:0/1116680656 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4b0061b30 msgr2=0x7fc4b0158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:20.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc4b5d4f700 1 --2- 172.21.15.32:0/1116680656 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4b0061b30 0x7fc4b0158d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:20.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc4b5d4f700 1 -- 172.21.15.32:0/1116680656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b0159260 con 0x7fc4b005f6f0 2020-12-05T17:35:20.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc4b554e700 1 --2- 172.21.15.32:0/1116680656 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4b0061b30 0x7fc4b0158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:20.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc4b5d4f700 1 --2- 172.21.15.32:0/1116680656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4b005f6f0 0x7fc4b00b3da0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7fc4ac002f30 tx=0x7fc4ac015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:20.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc49effd700 1 -- 172.21.15.32:0/1116680656 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4ac0056c0 con 0x7fc4b005f6f0 2020-12-05T17:35:20.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc49effd700 1 -- 172.21.15.32:0/1116680656 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ac005820 con 0x7fc4b005f6f0 2020-12-05T17:35:20.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc49effd700 1 -- 172.21.15.32:0/1116680656 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4ac005b80 con 0x7fc4b005f6f0 2020-12-05T17:35:20.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc4b6d51700 1 -- 172.21.15.32:0/1116680656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b00b60a0 con 0x7fc4b005f6f0 2020-12-05T17:35:20.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.655+0000 7fc4b6d51700 1 -- 172.21.15.32:0/1116680656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b00b1310 con 0x7fc4b005f6f0 2020-12-05T17:35:20.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.659+0000 7fc4b6d51700 1 -- 172.21.15.32:0/1116680656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b00b5e40 con 0x7fc4b005f6f0 2020-12-05T17:35:20.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.659+0000 7fc49effd700 1 -- 172.21.15.32:0/1116680656 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc4ac017070 con 0x7fc4b005f6f0 2020-12-05T17:35:20.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.659+0000 7fc49effd700 1 --2- 172.21.15.32:0/1116680656 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc4900553f0 0x7fc4900578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:20.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.659+0000 7fc49effd700 1 -- 172.21.15.32:0/1116680656 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fc4ac06e780 con 0x7fc4b005f6f0 2020-12-05T17:35:20.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.659+0000 7fc4b554e700 1 --2- 172.21.15.32:0/1116680656 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc4900553f0 0x7fc4900578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:20.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.659+0000 7fc4b554e700 1 --2- 172.21.15.32:0/1116680656 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc4900553f0 0x7fc4900578b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fc4a000b000 tx=0x7fc4a0009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:20.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.667+0000 7fc49effd700 1 -- 172.21.15.32:0/1116680656 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc4ac044c60 con 0x7fc4b005f6f0 2020-12-05T17:35:20.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:20.807+0000 7fc4b6d51700 1 -- 172.21.15.32:0/1116680656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fc4b0057050 con 0x7fc4b005f6f0 2020-12-05T17:35:21.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.783+0000 7fc49effd700 1 -- 172.21.15.32:0/1116680656 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v49) v1 ==== 133+0+0 (secure 0 0 0) 0x7fc4ac022070 con 0x7fc4b005f6f0 2020-12-05T17:35:21.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.787+0000 7fc4b6d51700 1 -- 172.21.15.32:0/1116680656 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc4900553f0 msgr2=0x7fc4900578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:21.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.787+0000 7fc4b6d51700 1 --2- 172.21.15.32:0/1116680656 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc4900553f0 0x7fc4900578b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fc4a000b000 tx=0x7fc4a0009040).stop 2020-12-05T17:35:21.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.787+0000 7fc4b6d51700 1 -- 172.21.15.32:0/1116680656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4b005f6f0 msgr2=0x7fc4b00b3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:21.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.787+0000 7fc4b6d51700 1 --2- 172.21.15.32:0/1116680656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4b005f6f0 0x7fc4b00b3da0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7fc4ac002f30 tx=0x7fc4ac015000).stop 2020-12-05T17:35:21.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.787+0000 7fc4b6d51700 1 -- 172.21.15.32:0/1116680656 shutdown_connections 2020-12-05T17:35:21.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.787+0000 7fc4b6d51700 1 --2- 172.21.15.32:0/1116680656 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc4900553f0 0x7fc4900578b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:21.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.787+0000 7fc4b6d51700 1 --2- 172.21.15.32:0/1116680656 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4b0142300 0x7fc4b0065df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:21.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.787+0000 7fc4b6d51700 1 --2- 172.21.15.32:0/1116680656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4b005f6f0 0x7fc4b00b3da0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:21.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.787+0000 7fc4b6d51700 1 --2- 172.21.15.32:0/1116680656 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4b0061b30 0x7fc4b0158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:21.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.787+0000 7fc4b6d51700 1 -- 172.21.15.32:0/1116680656 >> 172.21.15.32:0/1116680656 conn(0x7fc4b0061010 msgr2=0x7fc4b00ade50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:21.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.787+0000 7fc4b6d51700 1 -- 172.21.15.32:0/1116680656 shutdown_connections 2020-12-05T17:35:21.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.787+0000 7fc4b6d51700 1 -- 172.21.15.32:0/1116680656 wait complete. 2020-12-05T17:35:21.789 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2020-12-05T17:35:21.800 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-12-05T17:35:21.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.919+0000 7f4601f59700 1 Processor -- start 2020-12-05T17:35:21.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.919+0000 7f4601f59700 1 -- start start 2020-12-05T17:35:21.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.919+0000 7f4601f59700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45fc05ce60 0x7f45fc13ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:21.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.919+0000 7f4601f59700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45fc142cc0 0x7f45fc0b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:21.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.919+0000 7f4601f59700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45fc0b0ca0 con 0x7f45fc0ad220 2020-12-05T17:35:21.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.919+0000 7f4601f59700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45fc13f670 con 0x7f45fc05ce60 2020-12-05T17:35:21.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.919+0000 7f4601f59700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc143c20 con 0x7f45fc142cc0 2020-12-05T17:35:21.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.919+0000 7f45fb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45fc05ce60 0x7f45fc13ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:21.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.919+0000 7f45fb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45fc05ce60 0x7f45fc13ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35214/0 (socket says 172.21.15.32:35214) 2020-12-05T17:35:21.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.919+0000 7f45fb7fe700 1 -- 172.21.15.32:0/469307357 learned_addr learned my addr 172.21.15.32:0/469307357 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:21.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.919+0000 7f45fbfff700 1 --2- 172.21.15.32:0/469307357 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45fc142cc0 0x7f45fc0b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:21.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.919+0000 7f45fb7fe700 1 -- 172.21.15.32:0/469307357 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45fc142cc0 msgr2=0x7f45fc0b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:21.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.919+0000 7f45fb7fe700 1 --2- 172.21.15.32:0/469307357 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45fc142cc0 0x7f45fc0b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:21.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.919+0000 7f45fb7fe700 1 -- 172.21.15.32:0/469307357 >> v1:172.21.15.61:6789/0 conn(0x7f45fc0ad220 legacy=0x7f45fc13ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:21.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.919+0000 7f45fb7fe700 1 -- 172.21.15.32:0/469307357 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45fc05d240 con 0x7f45fc05ce60 2020-12-05T17:35:21.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.923+0000 7f45fb7fe700 1 --2- 172.21.15.32:0/469307357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45fc05ce60 0x7f45fc13ea70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f45ec002f30 tx=0x7f45ec006e80).ready entity=mon.0 client_cookie=8abfdfd8cb04d7f2 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:21.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.923+0000 7f45faffd700 1 -- 172.21.15.32:0/469307357 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45ec0097f0 con 0x7f45fc05ce60 2020-12-05T17:35:21.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.923+0000 7f45faffd700 1 -- 172.21.15.32:0/469307357 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec009950 con 0x7f45fc05ce60 2020-12-05T17:35:21.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.923+0000 7f45faffd700 1 -- 172.21.15.32:0/469307357 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45ec009cb0 con 0x7f45fc05ce60 2020-12-05T17:35:21.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.923+0000 7f4601f59700 1 -- 172.21.15.32:0/469307357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45fc05ce60 msgr2=0x7f45fc13ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:21.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.923+0000 7f4601f59700 1 --2- 172.21.15.32:0/469307357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45fc05ce60 0x7f45fc13ea70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f45ec002f30 tx=0x7f45ec006e80).stop 2020-12-05T17:35:21.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.923+0000 7f4601f59700 1 -- 172.21.15.32:0/469307357 shutdown_connections 2020-12-05T17:35:21.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.923+0000 7f4601f59700 1 --2- 172.21.15.32:0/469307357 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45fc142cc0 0x7f45fc0b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:21.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.923+0000 7f4601f59700 1 --2- 172.21.15.32:0/469307357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45fc05ce60 0x7f45fc13ea70 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:21.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.923+0000 7f4601f59700 1 -- 172.21.15.32:0/469307357 >> 172.21.15.32:0/469307357 conn(0x7f45fc069cb0 msgr2=0x7f45fc0604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:21.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.923+0000 7f4601f59700 1 -- 172.21.15.32:0/469307357 shutdown_connections 2020-12-05T17:35:21.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.923+0000 7f4601f59700 1 -- 172.21.15.32:0/469307357 wait complete. 2020-12-05T17:35:21.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.923+0000 7f4601f59700 1 Processor -- start 2020-12-05T17:35:21.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.927+0000 7f4601f59700 1 -- start start 2020-12-05T17:35:21.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.927+0000 7f4601f59700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45fc05ce60 0x7f45fc141250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:21.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.927+0000 7f4601f59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45fc0ad220 0x7f45fc1384f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:21.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.927+0000 7f4601f59700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45fc142cc0 0x7f45fc151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:21.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.927+0000 7f4601f59700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc137970 con 0x7f45fc0ad220 2020-12-05T17:35:21.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.927+0000 7f4601f59700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc141740 con 0x7f45fc142cc0 2020-12-05T17:35:21.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.927+0000 7f4601f59700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc063c40 con 0x7f45fc05ce60 2020-12-05T17:35:21.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.927+0000 7f45f2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45fc0ad220 0x7f45fc1384f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:21.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.927+0000 7f45f2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45fc0ad220 0x7f45fc1384f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35222/0 (socket says 172.21.15.32:35222) 2020-12-05T17:35:21.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.927+0000 7f45fb7fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45fc05ce60 0x7f45fc141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:21.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.927+0000 7f45fbfff700 1 --2- 172.21.15.32:0/918411151 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45fc142cc0 0x7f45fc151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:21.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.927+0000 7f45f2ffd700 1 -- 172.21.15.32:0/918411151 learned_addr learned my addr 172.21.15.32:0/918411151 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:21.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.927+0000 7f45f2ffd700 1 -- 172.21.15.32:0/918411151 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45fc05ce60 msgr2=0x7f45fc141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:21.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.927+0000 7f45f2ffd700 1 --2- 172.21.15.32:0/918411151 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45fc05ce60 0x7f45fc141250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:21.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.927+0000 7f45f2ffd700 1 -- 172.21.15.32:0/918411151 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45fc142cc0 msgr2=0x7f45fc151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:21.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.927+0000 7f45f2ffd700 1 --2- 172.21.15.32:0/918411151 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45fc142cc0 0x7f45fc151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:21.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.927+0000 7f45f2ffd700 1 -- 172.21.15.32:0/918411151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45fc1510b0 con 0x7f45fc0ad220 2020-12-05T17:35:21.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.927+0000 7f45f2ffd700 1 --2- 172.21.15.32:0/918411151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45fc0ad220 0x7f45fc1384f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f45fc05ca30 tx=0x7f45e4007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:21.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.927+0000 7f45f97fa700 1 -- 172.21.15.32:0/918411151 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45e4013410 con 0x7f45fc0ad220 2020-12-05T17:35:21.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.927+0000 7f45f97fa700 1 -- 172.21.15.32:0/918411151 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e4001e90 con 0x7f45fc0ad220 2020-12-05T17:35:21.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.927+0000 7f4601f59700 1 -- 172.21.15.32:0/918411151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45fc1389e0 con 0x7f45fc0ad220 2020-12-05T17:35:21.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.927+0000 7f45f97fa700 1 -- 172.21.15.32:0/918411151 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45e4013760 con 0x7f45fc0ad220 2020-12-05T17:35:21.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.927+0000 7f4601f59700 1 -- 172.21.15.32:0/918411151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45fc140f70 con 0x7f45fc0ad220 2020-12-05T17:35:21.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.931+0000 7f4601f59700 1 -- 172.21.15.32:0/918411151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45fc13a740 con 0x7f45fc0ad220 2020-12-05T17:35:21.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.931+0000 7f45f97fa700 1 -- 172.21.15.32:0/918411151 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f45e4010030 con 0x7f45fc0ad220 2020-12-05T17:35:21.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.931+0000 7f45f97fa700 1 --2- 172.21.15.32:0/918411151 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f45e0055470 0x7f45e0057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:21.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.931+0000 7f45f97fa700 1 -- 172.21.15.32:0/918411151 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f45e406da50 con 0x7f45fc0ad220 2020-12-05T17:35:21.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.931+0000 7f45fb7fe700 1 --2- 172.21.15.32:0/918411151 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f45e0055470 0x7f45e0057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:21.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.931+0000 7f45fb7fe700 1 --2- 172.21.15.32:0/918411151 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f45e0055470 0x7f45e0057930 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f45ec01c000 tx=0x7f45ec015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:21.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:21.935+0000 7f45f97fa700 1 -- 172.21.15.32:0/918411151 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f45fc13a740 con 0x7f45fc0ad220 2020-12-05T17:35:22.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:22.083+0000 7f4601f59700 1 -- 172.21.15.32:0/918411151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f45fc13a740 con 0x7f45fc0ad220 2020-12-05T17:35:23.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.031+0000 7f45f97fa700 1 -- 172.21.15.32:0/918411151 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v51) v1 ==== 125+0+0 (secure 0 0 0) 0x7f45fc13a740 con 0x7f45fc0ad220 2020-12-05T17:35:23.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.039+0000 7f4601f59700 1 -- 172.21.15.32:0/918411151 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f45e0055470 msgr2=0x7f45e0057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:23.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.039+0000 7f4601f59700 1 --2- 172.21.15.32:0/918411151 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f45e0055470 0x7f45e0057930 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f45ec01c000 tx=0x7f45ec015000).stop 2020-12-05T17:35:23.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.039+0000 7f4601f59700 1 -- 172.21.15.32:0/918411151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45fc0ad220 msgr2=0x7f45fc1384f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:23.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.039+0000 7f4601f59700 1 --2- 172.21.15.32:0/918411151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45fc0ad220 0x7f45fc1384f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f45fc05ca30 tx=0x7f45e4007bc0).stop 2020-12-05T17:35:23.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.039+0000 7f4601f59700 1 -- 172.21.15.32:0/918411151 shutdown_connections 2020-12-05T17:35:23.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.039+0000 7f4601f59700 1 --2- 172.21.15.32:0/918411151 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45fc142cc0 0x7f45fc151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.039+0000 7f4601f59700 1 --2- 172.21.15.32:0/918411151 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f45e0055470 0x7f45e0057930 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.039+0000 7f4601f59700 1 --2- 172.21.15.32:0/918411151 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45fc05ce60 0x7f45fc141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.039+0000 7f4601f59700 1 --2- 172.21.15.32:0/918411151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45fc0ad220 0x7f45fc1384f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.039+0000 7f4601f59700 1 -- 172.21.15.32:0/918411151 >> 172.21.15.32:0/918411151 conn(0x7f45fc069cb0 msgr2=0x7f45fc0af330 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:23.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.039+0000 7f4601f59700 1 -- 172.21.15.32:0/918411151 shutdown_connections 2020-12-05T17:35:23.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.039+0000 7f4601f59700 1 -- 172.21.15.32:0/918411151 wait complete. 2020-12-05T17:35:23.044 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2020-12-05T17:35:23.053 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-12-05T17:35:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.171+0000 7efd8c965700 1 Processor -- start 2020-12-05T17:35:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.171+0000 7efd8c965700 1 -- start start 2020-12-05T17:35:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd8c965700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd8805f3b0 0x7efd8805f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd8c965700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7efd8805feb0 0x7efd88061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd8c965700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efd880b2fb0 0x7efd8813c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd8c965700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd880b2df0 con 0x7efd880b2fb0 2020-12-05T17:35:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd8c965700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd880b3c90 con 0x7efd8805feb0 2020-12-05T17:35:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd8c965700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd880b2b50 con 0x7efd8805f3b0 2020-12-05T17:35:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd87fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efd880b2fb0 0x7efd8813c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd877fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd8805f3b0 0x7efd8805f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd87fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efd880b2fb0 0x7efd8813c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35232/0 (socket says 172.21.15.32:35232) 2020-12-05T17:35:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd87fff700 1 -- 172.21.15.32:0/1732238750 learned_addr learned my addr 172.21.15.32:0/1732238750 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd86ffd700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7efd8805feb0 0x7efd88061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd87fff700 1 -- 172.21.15.32:0/1732238750 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd8805f3b0 msgr2=0x7efd8805f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:23.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd87fff700 1 --2- 172.21.15.32:0/1732238750 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd8805f3b0 0x7efd8805f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd87fff700 1 -- 172.21.15.32:0/1732238750 >> v2:172.21.15.61:3300/0 conn(0x7efd8805feb0 msgr2=0x7efd88061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:23.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd87fff700 1 --2- 172.21.15.32:0/1732238750 >> v2:172.21.15.61:3300/0 conn(0x7efd8805feb0 0x7efd88061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd87fff700 1 -- 172.21.15.32:0/1732238750 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd88142820 con 0x7efd880b2fb0 2020-12-05T17:35:23.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd87fff700 1 --2- 172.21.15.32:0/1732238750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd880b2fb0 0x7efd8813c3b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7efd800032b0 tx=0x7efd80009270).ready entity=mon.0 client_cookie=58527da33e245cf5 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:23.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd867fc700 1 -- 172.21.15.32:0/1732238750 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd80009e60 con 0x7efd880b2fb0 2020-12-05T17:35:23.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd867fc700 1 -- 172.21.15.32:0/1732238750 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8000bb50 con 0x7efd880b2fb0 2020-12-05T17:35:23.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd867fc700 1 -- 172.21.15.32:0/1732238750 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd8000bcc0 con 0x7efd880b2fb0 2020-12-05T17:35:23.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd8c965700 1 -- 172.21.15.32:0/1732238750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd880b2fb0 msgr2=0x7efd8813c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:23.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd8c965700 1 --2- 172.21.15.32:0/1732238750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd880b2fb0 0x7efd8813c3b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7efd800032b0 tx=0x7efd80009270).stop 2020-12-05T17:35:23.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd8c965700 1 -- 172.21.15.32:0/1732238750 shutdown_connections 2020-12-05T17:35:23.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd8c965700 1 --2- 172.21.15.32:0/1732238750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd880b2fb0 0x7efd8813c3b0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd8c965700 1 --2- 172.21.15.32:0/1732238750 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd8805f3b0 0x7efd8805f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd8c965700 1 --2- 172.21.15.32:0/1732238750 >> v2:172.21.15.61:3300/0 conn(0x7efd8805feb0 0x7efd88061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd8c965700 1 -- 172.21.15.32:0/1732238750 >> 172.21.15.32:0/1732238750 conn(0x7efd880b8f20 msgr2=0x7efd880b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:23.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd8c965700 1 -- 172.21.15.32:0/1732238750 shutdown_connections 2020-12-05T17:35:23.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.175+0000 7efd8c965700 1 -- 172.21.15.32:0/1732238750 wait complete. 2020-12-05T17:35:23.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd8c965700 1 Processor -- start 2020-12-05T17:35:23.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd8c965700 1 -- start start 2020-12-05T17:35:23.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd8c965700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd8805f3b0 0x7efd88113750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:23.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd8c965700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd8805feb0 0x7efd88113c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:23.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd8c965700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efd880b2fb0 0x7efd88140ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:23.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd8c965700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd88145a10 con 0x7efd8805f3b0 2020-12-05T17:35:23.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd8c965700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd881467a0 con 0x7efd880b2fb0 2020-12-05T17:35:23.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd8c965700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd88146a40 con 0x7efd8805feb0 2020-12-05T17:35:23.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd877fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd8805f3b0 0x7efd88113750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:23.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd877fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd8805f3b0 0x7efd88113750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35234/0 (socket says 172.21.15.32:35234) 2020-12-05T17:35:23.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd877fe700 1 -- 172.21.15.32:0/502504965 learned_addr learned my addr 172.21.15.32:0/502504965 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:23.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd86ffd700 1 --2- 172.21.15.32:0/502504965 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd8805feb0 0x7efd88113c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:23.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd87fff700 1 --2- 172.21.15.32:0/502504965 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efd880b2fb0 0x7efd88140ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:23.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd877fe700 1 -- 172.21.15.32:0/502504965 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd8805feb0 msgr2=0x7efd88113c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:23.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd877fe700 1 --2- 172.21.15.32:0/502504965 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd8805feb0 0x7efd88113c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd877fe700 1 -- 172.21.15.32:0/502504965 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efd880b2fb0 msgr2=0x7efd88140ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:23.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd877fe700 1 --2- 172.21.15.32:0/502504965 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efd880b2fb0 0x7efd88140ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd877fe700 1 -- 172.21.15.32:0/502504965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd881481e0 con 0x7efd8805f3b0 2020-12-05T17:35:23.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd86ffd700 1 --2- 172.21.15.32:0/502504965 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd8805feb0 0x7efd88113c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:35:23.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd87fff700 1 --2- 172.21.15.32:0/502504965 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efd880b2fb0 0x7efd88140ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:35:23.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd877fe700 1 --2- 172.21.15.32:0/502504965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd8805f3b0 0x7efd88113750 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7efd7800c000 tx=0x7efd78006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:23.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd84ff9700 1 -- 172.21.15.32:0/502504965 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd7800ca10 con 0x7efd8805f3b0 2020-12-05T17:35:23.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd84ff9700 1 -- 172.21.15.32:0/502504965 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7800cb70 con 0x7efd8805f3b0 2020-12-05T17:35:23.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd84ff9700 1 -- 172.21.15.32:0/502504965 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd780123f0 con 0x7efd8805f3b0 2020-12-05T17:35:23.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.179+0000 7efd8c965700 1 -- 172.21.15.32:0/502504965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd88146ce0 con 0x7efd8805f3b0 2020-12-05T17:35:23.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.183+0000 7efd8c965700 1 -- 172.21.15.32:0/502504965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd88148680 con 0x7efd8805f3b0 2020-12-05T17:35:23.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.183+0000 7efd8c965700 1 -- 172.21.15.32:0/502504965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd88148370 con 0x7efd8805f3b0 2020-12-05T17:35:23.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.183+0000 7efd84ff9700 1 -- 172.21.15.32:0/502504965 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7efd78012550 con 0x7efd8805f3b0 2020-12-05T17:35:23.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.183+0000 7efd84ff9700 1 --2- 172.21.15.32:0/502504965 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efd680553b0 0x7efd68057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:23.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.183+0000 7efd86ffd700 1 --2- 172.21.15.32:0/502504965 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efd680553b0 0x7efd68057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:23.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.183+0000 7efd84ff9700 1 -- 172.21.15.32:0/502504965 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6304+0+0 (secure 0 0 0) 0x7efd7806e440 con 0x7efd8805f3b0 2020-12-05T17:35:23.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.187+0000 7efd86ffd700 1 --2- 172.21.15.32:0/502504965 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efd680553b0 0x7efd68057870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7efd7c00b000 tx=0x7efd7c006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:23.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.191+0000 7efd84ff9700 1 -- 172.21.15.32:0/502504965 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7efd78044600 con 0x7efd8805f3b0 2020-12-05T17:35:23.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.379+0000 7efd8c965700 1 -- 172.21.15.32:0/502504965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7efd88057050 con 0x7efd8805f3b0 2020-12-05T17:35:23.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.391+0000 7efd84ff9700 1 -- 172.21.15.32:0/502504965 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 9 and data pool 10 v22) v1 ==== 188+0+0 (secure 0 0 0) 0x7efd78047c20 con 0x7efd8805f3b0 2020-12-05T17:35:23.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.399+0000 7efd8c965700 1 -- 172.21.15.32:0/502504965 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efd680553b0 msgr2=0x7efd68057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:23.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.399+0000 7efd8c965700 1 --2- 172.21.15.32:0/502504965 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efd680553b0 0x7efd68057870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7efd7c00b000 tx=0x7efd7c006dc0).stop 2020-12-05T17:35:23.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.399+0000 7efd8c965700 1 -- 172.21.15.32:0/502504965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd8805f3b0 msgr2=0x7efd88113750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:23.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.399+0000 7efd8c965700 1 --2- 172.21.15.32:0/502504965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd8805f3b0 0x7efd88113750 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7efd7800c000 tx=0x7efd78006dc0).stop 2020-12-05T17:35:23.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.399+0000 7efd8c965700 1 -- 172.21.15.32:0/502504965 shutdown_connections 2020-12-05T17:35:23.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.399+0000 7efd8c965700 1 --2- 172.21.15.32:0/502504965 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efd880b2fb0 0x7efd88140ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.399+0000 7efd8c965700 1 --2- 172.21.15.32:0/502504965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd8805f3b0 0x7efd88113750 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.399+0000 7efd8c965700 1 --2- 172.21.15.32:0/502504965 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efd680553b0 0x7efd68057870 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.399+0000 7efd8c965700 1 --2- 172.21.15.32:0/502504965 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd8805feb0 0x7efd88113c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.399+0000 7efd8c965700 1 -- 172.21.15.32:0/502504965 >> 172.21.15.32:0/502504965 conn(0x7efd880b8f20 msgr2=0x7efd880b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:23.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.399+0000 7efd8c965700 1 -- 172.21.15.32:0/502504965 shutdown_connections 2020-12-05T17:35:23.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.399+0000 7efd8c965700 1 -- 172.21.15.32:0/502504965 wait complete. 2020-12-05T17:35:23.404 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 9 and data pool 10 2020-12-05T17:35:23.416 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:35:23.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.515+0000 7fc1db348700 1 Processor -- start 2020-12-05T17:35:23.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1db348700 1 -- start start 2020-12-05T17:35:23.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1db348700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc1d405f6f0 0x7fc1d40b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:23.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1db348700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d405fad0 con 0x7fc1d40af920 2020-12-05T17:35:23.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1db348700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d4144650 con 0x7fc1d40afea0 2020-12-05T17:35:23.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1db348700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1d40b2cd0 con 0x7fc1d405f6f0 2020-12-05T17:35:23.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9b45700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc1d40af920 0x7fc1d4063040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37902/0 (socket says 172.21.15.32:37902) 2020-12-05T17:35:23.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9b45700 1 -- 172.21.15.32:0/1279401405 learned_addr learned my addr 172.21.15.32:0/1279401405 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:35:23.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1da346700 1 --2- 172.21.15.32:0/1279401405 >> v2:172.21.15.61:3300/0 conn(0x7fc1d405f6f0 0x7fc1d40b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:23.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9344700 1 -- 172.21.15.32:0/1279401405 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 712031923 0 0) 0x7fc1d405fad0 con 0x7fc1d40af920 2020-12-05T17:35:23.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9344700 1 -- 172.21.15.32:0/1279401405 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1bc003230 con 0x7fc1d40af920 2020-12-05T17:35:23.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9344700 1 -- 172.21.15.32:0/1279401405 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812563930 0 0) 0x7fc1d4144650 con 0x7fc1d40afea0 2020-12-05T17:35:23.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9344700 1 -- 172.21.15.32:0/1279401405 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1d405fad0 con 0x7fc1d40afea0 2020-12-05T17:35:23.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9344700 1 -- 172.21.15.32:0/1279401405 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1483916237 0 0) 0x7fc1bc003230 con 0x7fc1d40af920 2020-12-05T17:35:23.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9344700 1 -- 172.21.15.32:0/1279401405 >> v2:172.21.15.61:3300/0 conn(0x7fc1d405f6f0 msgr2=0x7fc1d40b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:23.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9344700 1 --2- 172.21.15.32:0/1279401405 >> v2:172.21.15.61:3300/0 conn(0x7fc1d405f6f0 0x7fc1d40b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9344700 1 -- 172.21.15.32:0/1279401405 >> v1:172.21.15.61:6789/0 conn(0x7fc1d40afea0 legacy=0x7fc1d41409c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:23.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9344700 1 -- 172.21.15.32:0/1279401405 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d41448f0 con 0x7fc1d40af920 2020-12-05T17:35:23.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9344700 1 -- 172.21.15.32:0/1279401405 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fc1c40016d0 con 0x7fc1d40af920 2020-12-05T17:35:23.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9344700 1 -- 172.21.15.32:0/1279401405 >> v1:172.21.15.32:6789/0 conn(0x7fc1d40af920 legacy=0x7fc1d4063040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:23.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9344700 1 --2- 172.21.15.32:0/1279401405 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc1bc0038b0 0x7fc1bc003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:23.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9344700 1 --2- 172.21.15.32:0/1279401405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1bc0044d0 0x7fc1bc004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:23.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9344700 1 --2- 172.21.15.32:0/1279401405 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1bc004e40 0x7fc1bc00b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:23.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9344700 1 -- 172.21.15.32:0/1279401405 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc005370 con 0x7fc1bc0044d0 2020-12-05T17:35:23.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9344700 1 -- 172.21.15.32:0/1279401405 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc00bc10 con 0x7fc1bc004e40 2020-12-05T17:35:23.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9344700 1 -- 172.21.15.32:0/1279401405 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc00bdd0 con 0x7fc1bc0038b0 2020-12-05T17:35:23.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9b45700 1 --2- 172.21.15.32:0/1279401405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1bc0044d0 0x7fc1bc004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:23.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1da346700 1 --2- 172.21.15.32:0/1279401405 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc1bc0038b0 0x7fc1bc003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:23.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1dab47700 1 --2- 172.21.15.32:0/1279401405 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1bc004e40 0x7fc1bc00b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:23.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9b45700 1 -- 172.21.15.32:0/1279401405 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc1bc0038b0 msgr2=0x7fc1bc003cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:23.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9b45700 1 --2- 172.21.15.32:0/1279401405 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc1bc0038b0 0x7fc1bc003cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9b45700 1 -- 172.21.15.32:0/1279401405 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1bc004e40 msgr2=0x7fc1bc00b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:23.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9b45700 1 --2- 172.21.15.32:0/1279401405 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1bc004e40 0x7fc1bc00b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1dab47700 1 -- 172.21.15.32:0/1279401405 reap_dead start 2020-12-05T17:35:23.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1d9b45700 1 -- 172.21.15.32:0/1279401405 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc003230 con 0x7fc1bc0044d0 2020-12-05T17:35:23.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.519+0000 7fc1da346700 1 --2- 172.21.15.32:0/1279401405 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc1bc0038b0 0x7fc1bc003cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:35:23.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.523+0000 7fc1d9b45700 1 --2- 172.21.15.32:0/1279401405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1bc0044d0 0x7fc1bc004950 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7fc1c40014a0 tx=0x7fc1d41447e0).ready entity=mon.0 client_cookie=9337d520a0359f72 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:23.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.523+0000 7fc1d9344700 1 -- 172.21.15.32:0/1279401405 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1c40070d0 con 0x7fc1bc0044d0 2020-12-05T17:35:23.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.523+0000 7fc1d9344700 1 -- 172.21.15.32:0/1279401405 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c4007230 con 0x7fc1bc0044d0 2020-12-05T17:35:23.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.523+0000 7fc1d9344700 1 -- 172.21.15.32:0/1279401405 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1c40055e0 con 0x7fc1bc0044d0 2020-12-05T17:35:23.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.523+0000 7fc1db348700 1 -- 172.21.15.32:0/1279401405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1bc0044d0 msgr2=0x7fc1bc004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:23.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.523+0000 7fc1db348700 1 --2- 172.21.15.32:0/1279401405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1bc0044d0 0x7fc1bc004950 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7fc1c40014a0 tx=0x7fc1d41447e0).stop 2020-12-05T17:35:23.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.523+0000 7fc1db348700 1 -- 172.21.15.32:0/1279401405 shutdown_connections 2020-12-05T17:35:23.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.523+0000 7fc1db348700 1 --2- 172.21.15.32:0/1279401405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1bc0044d0 0x7fc1bc004950 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.523+0000 7fc1db348700 1 -- 172.21.15.32:0/1279401405 >> 172.21.15.32:0/1279401405 conn(0x7fc1d40604c0 msgr2=0x7fc1d405c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:23.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.523+0000 7fc1db348700 1 -- 172.21.15.32:0/1279401405 shutdown_connections 2020-12-05T17:35:23.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.523+0000 7fc1db348700 1 -- 172.21.15.32:0/1279401405 wait complete. 2020-12-05T17:35:23.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.523+0000 7fc1db348700 1 Processor -- start 2020-12-05T17:35:23.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.523+0000 7fc1db348700 1 -- start start 2020-12-05T17:35:23.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.527+0000 7fc1db348700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1bc0044d0 0x7fc1d4063140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:23.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.527+0000 7fc1db348700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1d40adf40 0x7fc1d413a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:23.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.527+0000 7fc1db348700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc1d4063b60 0x7fc1d413abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:23.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.527+0000 7fc1db348700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d40af010 con 0x7fc1d40adf40 2020-12-05T17:35:23.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.527+0000 7fc1db348700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d4065d50 con 0x7fc1bc0044d0 2020-12-05T17:35:23.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.527+0000 7fc1db348700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d4065ff0 con 0x7fc1d4063b60 2020-12-05T17:35:23.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.527+0000 7fc1d9b45700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1d40adf40 0x7fc1d413a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:23.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.527+0000 7fc1d9b45700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1d40adf40 0x7fc1d413a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35266/0 (socket says 172.21.15.32:35266) 2020-12-05T17:35:23.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.527+0000 7fc1d9b45700 1 -- 172.21.15.32:0/3237811274 learned_addr learned my addr 172.21.15.32:0/3237811274 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:23.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.527+0000 7fc1dab47700 1 --2- 172.21.15.32:0/3237811274 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc1d4063b60 0x7fc1d413abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:23.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.527+0000 7fc1da346700 1 --2- 172.21.15.32:0/3237811274 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1bc0044d0 0x7fc1d4063140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:23.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.527+0000 7fc1d9b45700 1 -- 172.21.15.32:0/3237811274 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc1d4063b60 msgr2=0x7fc1d413abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:23.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.527+0000 7fc1d9b45700 1 --2- 172.21.15.32:0/3237811274 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc1d4063b60 0x7fc1d413abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.527+0000 7fc1d9b45700 1 -- 172.21.15.32:0/3237811274 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1bc0044d0 msgr2=0x7fc1d4063140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:23.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.527+0000 7fc1d9b45700 1 --2- 172.21.15.32:0/3237811274 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1bc0044d0 0x7fc1d4063140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.527+0000 7fc1d9b45700 1 -- 172.21.15.32:0/3237811274 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d4063630 con 0x7fc1d40adf40 2020-12-05T17:35:23.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.527+0000 7fc1d9b45700 1 --2- 172.21.15.32:0/3237811274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1d40adf40 0x7fc1d413a6d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fc1c40014a0 tx=0x7fc1c4013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:23.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.527+0000 7fc1c37fe700 1 -- 172.21.15.32:0/3237811274 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1c401b070 con 0x7fc1d40adf40 2020-12-05T17:35:23.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.527+0000 7fc1c37fe700 1 -- 172.21.15.32:0/3237811274 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c4005df0 con 0x7fc1d40adf40 2020-12-05T17:35:23.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.527+0000 7fc1db348700 1 -- 172.21.15.32:0/3237811274 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d40aed00 con 0x7fc1d40adf40 2020-12-05T17:35:23.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.527+0000 7fc1c37fe700 1 -- 172.21.15.32:0/3237811274 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1c4001750 con 0x7fc1d40adf40 2020-12-05T17:35:23.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.527+0000 7fc1db348700 1 -- 172.21.15.32:0/3237811274 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d413b250 con 0x7fc1d40adf40 2020-12-05T17:35:23.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.527+0000 7fc1db348700 1 -- 172.21.15.32:0/3237811274 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1d4057050 con 0x7fc1d40adf40 2020-12-05T17:35:23.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.531+0000 7fc1c37fe700 1 -- 172.21.15.32:0/3237811274 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc1c4027070 con 0x7fc1d40adf40 2020-12-05T17:35:23.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.531+0000 7fc1c37fe700 1 --2- 172.21.15.32:0/3237811274 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc1b40553f0 0x7fc1b40578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:23.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.531+0000 7fc1c37fe700 1 -- 172.21.15.32:0/3237811274 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc1c406e540 con 0x7fc1d40adf40 2020-12-05T17:35:23.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.531+0000 7fc1da346700 1 --2- 172.21.15.32:0/3237811274 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc1b40553f0 0x7fc1b40578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:23.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.531+0000 7fc1da346700 1 --2- 172.21.15.32:0/3237811274 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc1b40553f0 0x7fc1b40578b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fc1d0008000 tx=0x7fc1d0008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:23.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.539+0000 7fc1c37fe700 1 -- 172.21.15.32:0/3237811274 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc1c40446e0 con 0x7fc1d40adf40 2020-12-05T17:35:23.679 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.679+0000 7fc1db348700 1 -- 172.21.15.32:0/3237811274 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1d4063940 con 0x7fc1d40adf40 2020-12-05T17:35:23.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.683+0000 7fc1c37fe700 1 -- 172.21.15.32:0/3237811274 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16299 (secure 0 0 0) 0x7fc1c4047d00 con 0x7fc1d40adf40 2020-12-05T17:35:23.681 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:35:23.681 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":52,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:35:23.386011+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:35:20.867930+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-12-05T17:35:22.139660+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:35:23.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.683+0000 7fc1db348700 1 -- 172.21.15.32:0/3237811274 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc1b40553f0 msgr2=0x7fc1b40578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:23.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.683+0000 7fc1db348700 1 --2- 172.21.15.32:0/3237811274 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc1b40553f0 0x7fc1b40578b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fc1d0008000 tx=0x7fc1d0008040).stop 2020-12-05T17:35:23.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.683+0000 7fc1db348700 1 -- 172.21.15.32:0/3237811274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1d40adf40 msgr2=0x7fc1d413a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:23.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.683+0000 7fc1db348700 1 --2- 172.21.15.32:0/3237811274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1d40adf40 0x7fc1d413a6d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fc1c40014a0 tx=0x7fc1c4013000).stop 2020-12-05T17:35:23.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.683+0000 7fc1db348700 1 -- 172.21.15.32:0/3237811274 shutdown_connections 2020-12-05T17:35:23.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.683+0000 7fc1db348700 1 --2- 172.21.15.32:0/3237811274 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc1b40553f0 0x7fc1b40578b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.683+0000 7fc1db348700 1 --2- 172.21.15.32:0/3237811274 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc1d4063b60 0x7fc1d413abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.683+0000 7fc1db348700 1 --2- 172.21.15.32:0/3237811274 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1bc0044d0 0x7fc1d4063140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.683+0000 7fc1db348700 1 --2- 172.21.15.32:0/3237811274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1d40adf40 0x7fc1d413a6d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.683+0000 7fc1db348700 1 -- 172.21.15.32:0/3237811274 >> 172.21.15.32:0/3237811274 conn(0x7fc1d40604c0 msgr2=0x7fc1d4141220 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:23.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.683+0000 7fc1db348700 1 -- 172.21.15.32:0/3237811274 shutdown_connections 2020-12-05T17:35:23.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.683+0000 7fc1db348700 1 -- 172.21.15.32:0/3237811274 wait complete. 2020-12-05T17:35:23.695 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:35:23.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.811+0000 7f7205af4700 1 Processor -- start 2020-12-05T17:35:23.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.811+0000 7f7205af4700 1 -- start start 2020-12-05T17:35:23.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.811+0000 7f7205af4700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f72000b1880 0x7f72001430f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:23.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.811+0000 7f7205af4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72000b0230 con 0x7f72000afe50 2020-12-05T17:35:23.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.811+0000 7f7205af4700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72000b6a50 con 0x7f72000b3aa0 2020-12-05T17:35:23.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.811+0000 7f7205af4700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7200146080 con 0x7f72000b1880 2020-12-05T17:35:23.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.811+0000 7f71ff7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f72000afe50 0x7f72001429e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37922/0 (socket says 172.21.15.32:37922) 2020-12-05T17:35:23.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.811+0000 7f71ff7fe700 1 -- 172.21.15.32:0/1259896063 learned_addr learned my addr 172.21.15.32:0/1259896063 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:35:23.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.811+0000 7f71fffff700 1 --2- 172.21.15.32:0/1259896063 >> v2:172.21.15.61:3300/0 conn(0x7f72000b1880 0x7f72001430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:23.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.811+0000 7f71fe7fc700 1 -- 172.21.15.32:0/1259896063 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771178384 0 0) 0x7f72000b0230 con 0x7f72000afe50 2020-12-05T17:35:23.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.811+0000 7f71fe7fc700 1 -- 172.21.15.32:0/1259896063 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71e0003230 con 0x7f72000afe50 2020-12-05T17:35:23.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.811+0000 7f71fffff700 1 -- 172.21.15.32:0/1259896063 >> v1:172.21.15.61:6789/0 conn(0x7f72000b3aa0 legacy=0x7f720013f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.811+0000 7f71fffff700 1 -- 172.21.15.32:0/1259896063 >> v1:172.21.15.32:6789/0 conn(0x7f72000afe50 legacy=0x7f72001429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.811+0000 7f71fffff700 1 -- 172.21.15.32:0/1259896063 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7200146320 con 0x7f72000b1880 2020-12-05T17:35:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.811+0000 7f71fffff700 1 --2- 172.21.15.32:0/1259896063 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72000b1880 0x7f72001430f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f71f80032b0 tx=0x7f71f80091a0).ready entity=mon.1 client_cookie=bc6cc46c1ff71b55 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.811+0000 7f71fe7fc700 1 -- 172.21.15.32:0/1259896063 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71f800b9a0 con 0x7f72000b1880 2020-12-05T17:35:23.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.811+0000 7f71fe7fc700 1 -- 172.21.15.32:0/1259896063 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f800bb00 con 0x7f72000b1880 2020-12-05T17:35:23.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.811+0000 7f71fe7fc700 1 -- 172.21.15.32:0/1259896063 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71f800bc70 con 0x7f72000b1880 2020-12-05T17:35:23.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.811+0000 7f7205af4700 1 -- 172.21.15.32:0/1259896063 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72000b1880 msgr2=0x7f72001430f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:23.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.811+0000 7f7205af4700 1 --2- 172.21.15.32:0/1259896063 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72000b1880 0x7f72001430f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f71f80032b0 tx=0x7f71f80091a0).stop 2020-12-05T17:35:23.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f7205af4700 1 -- 172.21.15.32:0/1259896063 shutdown_connections 2020-12-05T17:35:23.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f7205af4700 1 --2- 172.21.15.32:0/1259896063 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72000b1880 0x7f72001430f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f7205af4700 1 -- 172.21.15.32:0/1259896063 >> 172.21.15.32:0/1259896063 conn(0x7f72000b9100 msgr2=0x7f72000b9510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:23.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f7205af4700 1 -- 172.21.15.32:0/1259896063 shutdown_connections 2020-12-05T17:35:23.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f7205af4700 1 -- 172.21.15.32:0/1259896063 wait complete. 2020-12-05T17:35:23.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f7205af4700 1 Processor -- start 2020-12-05T17:35:23.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f7205af4700 1 -- start start 2020-12-05T17:35:23.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f7205af4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72000afe50 0x7f72001459f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:23.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f7205af4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72000b1880 0x7f7200141f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:23.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f7205af4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72000b3aa0 0x7f7200154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:23.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f7205af4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7200061d40 con 0x7f72000b3aa0 2020-12-05T17:35:23.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f7205af4700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72001431e0 con 0x7f72000b1880 2020-12-05T17:35:23.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f7205af4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7200143680 con 0x7f72000afe50 2020-12-05T17:35:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f71fffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72000b3aa0 0x7f7200154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f71fffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72000b3aa0 0x7f7200154f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35280/0 (socket says 172.21.15.32:35280) 2020-12-05T17:35:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f71fffff700 1 -- 172.21.15.32:0/175528769 learned_addr learned my addr 172.21.15.32:0/175528769 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f71ff7fe700 1 --2- 172.21.15.32:0/175528769 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72000afe50 0x7f72001459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:23.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f71feffd700 1 --2- 172.21.15.32:0/175528769 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72000b1880 0x7f7200141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:23.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f71fffff700 1 -- 172.21.15.32:0/175528769 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72000afe50 msgr2=0x7f72001459f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:23.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f71fffff700 1 --2- 172.21.15.32:0/175528769 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72000afe50 0x7f72001459f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f71fffff700 1 -- 172.21.15.32:0/175528769 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72000b1880 msgr2=0x7f7200141f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:23.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f71fffff700 1 --2- 172.21.15.32:0/175528769 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72000b1880 0x7f7200141f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f71fffff700 1 -- 172.21.15.32:0/175528769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72000669a0 con 0x7f72000b3aa0 2020-12-05T17:35:23.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f71ff7fe700 1 --2- 172.21.15.32:0/175528769 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72000afe50 0x7f72001459f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:23.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f71fffff700 1 --2- 172.21.15.32:0/175528769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72000b3aa0 0x7f7200154f50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f71f80032b0 tx=0x7f71f8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:23.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f71fcff9700 1 -- 172.21.15.32:0/175528769 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71f800bc70 con 0x7f72000b3aa0 2020-12-05T17:35:23.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f71fcff9700 1 -- 172.21.15.32:0/175528769 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f801f5e0 con 0x7f72000b3aa0 2020-12-05T17:35:23.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f7205af4700 1 -- 172.21.15.32:0/175528769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7200142440 con 0x7f72000b3aa0 2020-12-05T17:35:23.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f71fcff9700 1 -- 172.21.15.32:0/175528769 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71f801f940 con 0x7f72000b3aa0 2020-12-05T17:35:23.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.815+0000 7f7205af4700 1 -- 172.21.15.32:0/175528769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720013ecc0 con 0x7f72000b3aa0 2020-12-05T17:35:23.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.819+0000 7f7205af4700 1 -- 172.21.15.32:0/175528769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72001433a0 con 0x7f72000b3aa0 2020-12-05T17:35:23.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.819+0000 7f71fcff9700 1 -- 172.21.15.32:0/175528769 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f71f800f070 con 0x7f72000b3aa0 2020-12-05T17:35:23.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.819+0000 7f71fcff9700 1 --2- 172.21.15.32:0/175528769 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71e4055330 0x7f71e40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:23.826 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.819+0000 7f71ff7fe700 1 --2- 172.21.15.32:0/175528769 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71e4055330 0x7f71e40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:23.826 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.819+0000 7f71fcff9700 1 -- 172.21.15.32:0/175528769 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f71f806edc0 con 0x7f72000b3aa0 2020-12-05T17:35:23.826 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.819+0000 7f71ff7fe700 1 --2- 172.21.15.32:0/175528769 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71e4055330 0x7f71e40577f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f71f000b000 tx=0x7f71f0009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:23.826 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.827+0000 7f71fcff9700 1 -- 172.21.15.32:0/175528769 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f71f8044e50 con 0x7f72000b3aa0 2020-12-05T17:35:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.971+0000 7f7205af4700 1 -- 172.21.15.32:0/175528769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7200061b30 con 0x7f72000b3aa0 2020-12-05T17:35:23.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.971+0000 7f71fcff9700 1 -- 172.21.15.32:0/175528769 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f71f8024090 con 0x7f72000b3aa0 2020-12-05T17:35:23.970 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:35:23.971 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":52,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:35:23.386011+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:35:20.867930+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-12-05T17:35:22.139660+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:35:23.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.975+0000 7f7205af4700 1 -- 172.21.15.32:0/175528769 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71e4055330 msgr2=0x7f71e40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:23.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.975+0000 7f7205af4700 1 --2- 172.21.15.32:0/175528769 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71e4055330 0x7f71e40577f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f71f000b000 tx=0x7f71f0009040).stop 2020-12-05T17:35:23.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.975+0000 7f7205af4700 1 -- 172.21.15.32:0/175528769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72000b3aa0 msgr2=0x7f7200154f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:23.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.975+0000 7f7205af4700 1 --2- 172.21.15.32:0/175528769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72000b3aa0 0x7f7200154f50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f71f80032b0 tx=0x7f71f8016000).stop 2020-12-05T17:35:23.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.975+0000 7f7205af4700 1 -- 172.21.15.32:0/175528769 shutdown_connections 2020-12-05T17:35:23.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.975+0000 7f7205af4700 1 --2- 172.21.15.32:0/175528769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72000b3aa0 0x7f7200154f50 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.975+0000 7f7205af4700 1 --2- 172.21.15.32:0/175528769 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71e4055330 0x7f71e40577f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.975+0000 7f7205af4700 1 --2- 172.21.15.32:0/175528769 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72000afe50 0x7f72001459f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.975+0000 7f7205af4700 1 --2- 172.21.15.32:0/175528769 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72000b1880 0x7f7200141f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.975+0000 7f7205af4700 1 -- 172.21.15.32:0/175528769 >> 172.21.15.32:0/175528769 conn(0x7f72000b9100 msgr2=0x7f7200139350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.975+0000 7f7205af4700 1 -- 172.21.15.32:0/175528769 shutdown_connections 2020-12-05T17:35:23.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:23.975+0000 7f7205af4700 1 -- 172.21.15.32:0/175528769 wait complete. 2020-12-05T17:35:23.992 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-12-05T17:35:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.111+0000 7f015942e700 1 Processor -- start 2020-12-05T17:35:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.111+0000 7f015942e700 1 -- start start 2020-12-05T17:35:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f015942e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0154063f00 0x7f015414c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f015942e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f015405f5c0 con 0x7f01540af0d0 2020-12-05T17:35:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0153fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f01540af0d0 0x7f015413daa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37934/0 (socket says 172.21.15.32:37934) 2020-12-05T17:35:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0153fff700 1 -- 172.21.15.32:0/3244144604 learned_addr learned my addr 172.21.15.32:0/3244144604 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:35:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f015942e700 1 -- 172.21.15.32:0/3244144604 --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01540af4b0 con 0x7f01540b0d70 2020-12-05T17:35:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f015942e700 1 -- 172.21.15.32:0/3244144604 --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f015413e1b0 con 0x7f0154063f00 2020-12-05T17:35:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0158c2d700 1 --2- 172.21.15.32:0/3244144604 >> v2:172.21.15.61:3300/0 conn(0x7f0154063f00 0x7f015414c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0152ffd700 1 -- 172.21.15.32:0/3244144604 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2044311689 0 0) 0x7f015405f5c0 con 0x7f01540af0d0 2020-12-05T17:35:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0152ffd700 1 -- 172.21.15.32:0/3244144604 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0134003230 con 0x7f01540af0d0 2020-12-05T17:35:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0152ffd700 1 -- 172.21.15.32:0/3244144604 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1702648693 0 0) 0x7f0134003230 con 0x7f01540af0d0 2020-12-05T17:35:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0152ffd700 1 -- 172.21.15.32:0/3244144604 >> v2:172.21.15.61:3300/0 conn(0x7f0154063f00 msgr2=0x7f015414c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0152ffd700 1 --2- 172.21.15.32:0/3244144604 >> v2:172.21.15.61:3300/0 conn(0x7f0154063f00 0x7f015414c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0152ffd700 1 -- 172.21.15.32:0/3244144604 >> v1:172.21.15.61:6789/0 conn(0x7f01540b0d70 legacy=0x7f01540637f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0152ffd700 1 -- 172.21.15.32:0/3244144604 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015405f7f0 con 0x7f01540af0d0 2020-12-05T17:35:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0152ffd700 1 -- 172.21.15.32:0/3244144604 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f0144001a50 con 0x7f01540af0d0 2020-12-05T17:35:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0152ffd700 1 -- 172.21.15.32:0/3244144604 >> v1:172.21.15.32:6789/0 conn(0x7f01540af0d0 legacy=0x7f015413daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0152ffd700 1 --2- 172.21.15.32:0/3244144604 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0134003720 0x7f0134003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:24.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0152ffd700 1 --2- 172.21.15.32:0/3244144604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01340044d0 0x7f0134004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:24.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0152ffd700 1 --2- 172.21.15.32:0/3244144604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0134004e40 0x7f013400b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:24.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0152ffd700 1 -- 172.21.15.32:0/3244144604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015413e1b0 con 0x7f0134004e40 2020-12-05T17:35:24.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0152ffd700 1 -- 172.21.15.32:0/3244144604 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0134004250 con 0x7f0134003720 2020-12-05T17:35:24.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0152ffd700 1 -- 172.21.15.32:0/3244144604 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f013400bc10 con 0x7f01340044d0 2020-12-05T17:35:24.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0158c2d700 1 --2- 172.21.15.32:0/3244144604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0134004e40 0x7f013400b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:24.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0153fff700 1 --2- 172.21.15.32:0/3244144604 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0134003720 0x7f0134003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:24.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f01537fe700 1 --2- 172.21.15.32:0/3244144604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01340044d0 0x7f0134004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:24.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0158c2d700 1 -- 172.21.15.32:0/3244144604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01340044d0 msgr2=0x7f0134004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0158c2d700 1 --2- 172.21.15.32:0/3244144604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01340044d0 0x7f0134004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0158c2d700 1 -- 172.21.15.32:0/3244144604 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0134003720 msgr2=0x7f0134003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0158c2d700 1 --2- 172.21.15.32:0/3244144604 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0134003720 0x7f0134003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0158c2d700 1 -- 172.21.15.32:0/3244144604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0134003230 con 0x7f0134004e40 2020-12-05T17:35:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0158c2d700 1 -- 172.21.15.32:0/3244144604 reap_dead start 2020-12-05T17:35:24.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0158c2d700 1 --2- 172.21.15.32:0/3244144604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0134004e40 0x7f013400b720 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f014c00c000 tx=0x7f014c008040).ready entity=mon.0 client_cookie=db63b445f0155fc3 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:24.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0152ffd700 1 -- 172.21.15.32:0/3244144604 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f014c00e8a0 con 0x7f0134004e40 2020-12-05T17:35:24.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0152ffd700 1 -- 172.21.15.32:0/3244144604 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014c00ea00 con 0x7f0134004e40 2020-12-05T17:35:24.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f0152ffd700 1 -- 172.21.15.32:0/3244144604 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f014c00ed60 con 0x7f0134004e40 2020-12-05T17:35:24.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f015942e700 1 -- 172.21.15.32:0/3244144604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0134004e40 msgr2=0x7f013400b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:24.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.115+0000 7f015942e700 1 --2- 172.21.15.32:0/3244144604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0134004e40 0x7f013400b720 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f014c00c000 tx=0x7f014c008040).stop 2020-12-05T17:35:24.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.119+0000 7f015942e700 1 -- 172.21.15.32:0/3244144604 shutdown_connections 2020-12-05T17:35:24.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.119+0000 7f015942e700 1 --2- 172.21.15.32:0/3244144604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0134004e40 0x7f013400b720 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.119+0000 7f015942e700 1 -- 172.21.15.32:0/3244144604 >> 172.21.15.32:0/3244144604 conn(0x7f01540620b0 msgr2=0x7f0154062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:24.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.119+0000 7f015942e700 1 -- 172.21.15.32:0/3244144604 shutdown_connections 2020-12-05T17:35:24.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.119+0000 7f015942e700 1 -- 172.21.15.32:0/3244144604 wait complete. 2020-12-05T17:35:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.119+0000 7f015942e700 1 Processor -- start 2020-12-05T17:35:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.119+0000 7f015942e700 1 -- start start 2020-12-05T17:35:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.119+0000 7f015942e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0134004e40 0x7f015414f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.119+0000 7f015942e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01540b3f40 0x7f0154140980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.119+0000 7f015942e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01541530c0 0x7f0154153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.119+0000 7f015942e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0154153a30 con 0x7f01540b3f40 2020-12-05T17:35:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.119+0000 7f015942e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0154110610 con 0x7f01541530c0 2020-12-05T17:35:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.119+0000 7f015942e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0154110370 con 0x7f0134004e40 2020-12-05T17:35:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.119+0000 7f01537fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01540b3f40 0x7f0154140980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.119+0000 7f01537fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01540b3f40 0x7f0154140980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35300/0 (socket says 172.21.15.32:35300) 2020-12-05T17:35:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.119+0000 7f01537fe700 1 -- 172.21.15.32:0/1423810275 learned_addr learned my addr 172.21.15.32:0/1423810275 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.119+0000 7f0153fff700 1 --2- 172.21.15.32:0/1423810275 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0134004e40 0x7f015414f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.119+0000 7f0158c2d700 1 --2- 172.21.15.32:0/1423810275 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01541530c0 0x7f0154153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.123+0000 7f01537fe700 1 -- 172.21.15.32:0/1423810275 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0134004e40 msgr2=0x7f015414f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.123+0000 7f01537fe700 1 --2- 172.21.15.32:0/1423810275 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0134004e40 0x7f015414f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.123+0000 7f01537fe700 1 -- 172.21.15.32:0/1423810275 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01541530c0 msgr2=0x7f0154153540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.123+0000 7f01537fe700 1 --2- 172.21.15.32:0/1423810275 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01541530c0 0x7f0154153540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.123+0000 7f01537fe700 1 -- 172.21.15.32:0/1423810275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0154152c20 con 0x7f01540b3f40 2020-12-05T17:35:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.123+0000 7f01537fe700 1 --2- 172.21.15.32:0/1423810275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01540b3f40 0x7f0154140980 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f0154152fa0 tx=0x7f013c007b60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.123+0000 7f01517fa700 1 -- 172.21.15.32:0/1423810275 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f013c002930 con 0x7f01540b3f40 2020-12-05T17:35:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.123+0000 7f01517fa700 1 -- 172.21.15.32:0/1423810275 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013c002a90 con 0x7f01540b3f40 2020-12-05T17:35:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.123+0000 7f015942e700 1 -- 172.21.15.32:0/1423810275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015414fa30 con 0x7f01540b3f40 2020-12-05T17:35:24.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.123+0000 7f01517fa700 1 -- 172.21.15.32:0/1423810275 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f013c002df0 con 0x7f01540b3f40 2020-12-05T17:35:24.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.123+0000 7f015942e700 1 -- 172.21.15.32:0/1423810275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0154140e70 con 0x7f01540b3f40 2020-12-05T17:35:24.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.123+0000 7f015942e700 1 -- 172.21.15.32:0/1423810275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01541101a0 con 0x7f01540b3f40 2020-12-05T17:35:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.123+0000 7f01517fa700 1 -- 172.21.15.32:0/1423810275 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f013c01b030 con 0x7f01540b3f40 2020-12-05T17:35:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.123+0000 7f01517fa700 1 --2- 172.21.15.32:0/1423810275 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0138055330 0x7f01380577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.123+0000 7f01517fa700 1 -- 172.21.15.32:0/1423810275 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f013c06dde0 con 0x7f01540b3f40 2020-12-05T17:35:24.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.123+0000 7f0153fff700 1 --2- 172.21.15.32:0/1423810275 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0138055330 0x7f01380577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:24.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.123+0000 7f0153fff700 1 --2- 172.21.15.32:0/1423810275 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0138055330 0x7f01380577f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f0144005000 tx=0x7f0154063580).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:24.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.131+0000 7f01517fa700 1 -- 172.21.15.32:0/1423810275 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f013c044030 con 0x7f01540b3f40 2020-12-05T17:35:24.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.315+0000 7f015942e700 1 -- 172.21.15.32:0/1423810275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0154152db0 con 0x7f01540b3f40 2020-12-05T17:35:24.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.391+0000 7f01517fa700 1 -- 172.21.15.32:0/1423810275 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7f013c0230b0 con 0x7f01540b3f40 2020-12-05T17:35:24.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.395+0000 7f015942e700 1 -- 172.21.15.32:0/1423810275 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0138055330 msgr2=0x7f01380577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:24.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.395+0000 7f015942e700 1 --2- 172.21.15.32:0/1423810275 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0138055330 0x7f01380577f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f0144005000 tx=0x7f0154063580).stop 2020-12-05T17:35:24.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.395+0000 7f015942e700 1 -- 172.21.15.32:0/1423810275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01540b3f40 msgr2=0x7f0154140980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:24.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.395+0000 7f015942e700 1 --2- 172.21.15.32:0/1423810275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01540b3f40 0x7f0154140980 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f0154152fa0 tx=0x7f013c007b60).stop 2020-12-05T17:35:24.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.395+0000 7f015942e700 1 -- 172.21.15.32:0/1423810275 shutdown_connections 2020-12-05T17:35:24.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.395+0000 7f015942e700 1 --2- 172.21.15.32:0/1423810275 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01541530c0 0x7f0154153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.395+0000 7f015942e700 1 --2- 172.21.15.32:0/1423810275 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0138055330 0x7f01380577f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.395+0000 7f015942e700 1 --2- 172.21.15.32:0/1423810275 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0134004e40 0x7f015414f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.395+0000 7f015942e700 1 --2- 172.21.15.32:0/1423810275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01540b3f40 0x7f0154140980 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.395+0000 7f015942e700 1 -- 172.21.15.32:0/1423810275 >> 172.21.15.32:0/1423810275 conn(0x7f01540620b0 msgr2=0x7f015413f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:24.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.395+0000 7f015942e700 1 -- 172.21.15.32:0/1423810275 shutdown_connections 2020-12-05T17:35:24.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.399+0000 7f015942e700 1 -- 172.21.15.32:0/1423810275 wait complete. 2020-12-05T17:35:24.414 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:35:24.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.531+0000 7f71e2654700 1 Processor -- start 2020-12-05T17:35:24.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.531+0000 7f71e2654700 1 -- start start 2020-12-05T17:35:24.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.531+0000 7f71e2654700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f71dc0b1fc0 0x7f71dc0af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:24.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.531+0000 7f71e2654700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71dc0ab710 0x7f71dc061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:24.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.531+0000 7f71e2654700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71dc147400 0x7f71dc145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:24.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.531+0000 7f71e2654700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71dc0b0110 con 0x7f71dc0ab710 2020-12-05T17:35:24.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.531+0000 7f71e2654700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71dc146a90 con 0x7f71dc0b1fc0 2020-12-05T17:35:24.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.531+0000 7f71e2654700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc0b23a0 con 0x7f71dc147400 2020-12-05T17:35:24.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.531+0000 7f71e0e51700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71dc0ab710 0x7f71dc061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:24.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.531+0000 7f71e0e51700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71dc0ab710 0x7f71dc061780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35310/0 (socket says 172.21.15.32:35310) 2020-12-05T17:35:24.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.531+0000 7f71e0e51700 1 -- 172.21.15.32:0/1310416283 learned_addr learned my addr 172.21.15.32:0/1310416283 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:24.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.531+0000 7f71e1e53700 1 --2- 172.21.15.32:0/1310416283 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71dc147400 0x7f71dc145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:24.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.531+0000 7f71e1652700 1 --2- 172.21.15.32:0/1310416283 >> v2:172.21.15.61:3300/0 conn(0x7f71dc0b1fc0 0x7f71dc0af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:24.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e0e51700 1 -- 172.21.15.32:0/1310416283 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71dc147400 msgr2=0x7f71dc145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:24.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e0e51700 1 --2- 172.21.15.32:0/1310416283 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71dc147400 0x7f71dc145d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e0e51700 1 -- 172.21.15.32:0/1310416283 >> v2:172.21.15.61:3300/0 conn(0x7f71dc0b1fc0 msgr2=0x7f71dc0af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:24.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e0e51700 1 --2- 172.21.15.32:0/1310416283 >> v2:172.21.15.61:3300/0 conn(0x7f71dc0b1fc0 0x7f71dc0af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e0e51700 1 -- 172.21.15.32:0/1310416283 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71dc146cc0 con 0x7f71dc0ab710 2020-12-05T17:35:24.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e0e51700 1 --2- 172.21.15.32:0/1310416283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71dc0ab710 0x7f71dc061780 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f71d0002f30 tx=0x7f71d0006e80).ready entity=mon.0 client_cookie=40a2217225fc49d9 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:24.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71dbfff700 1 -- 172.21.15.32:0/1310416283 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71d0009640 con 0x7f71dc0ab710 2020-12-05T17:35:24.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71dbfff700 1 -- 172.21.15.32:0/1310416283 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d00097a0 con 0x7f71dc0ab710 2020-12-05T17:35:24.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71dbfff700 1 -- 172.21.15.32:0/1310416283 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71d0009b00 con 0x7f71dc0ab710 2020-12-05T17:35:24.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e2654700 1 -- 172.21.15.32:0/1310416283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71dc0ab710 msgr2=0x7f71dc061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:24.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e2654700 1 --2- 172.21.15.32:0/1310416283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71dc0ab710 0x7f71dc061780 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f71d0002f30 tx=0x7f71d0006e80).stop 2020-12-05T17:35:24.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e2654700 1 -- 172.21.15.32:0/1310416283 shutdown_connections 2020-12-05T17:35:24.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e2654700 1 --2- 172.21.15.32:0/1310416283 >> v2:172.21.15.61:3300/0 conn(0x7f71dc0b1fc0 0x7f71dc0af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e2654700 1 --2- 172.21.15.32:0/1310416283 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71dc147400 0x7f71dc145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e2654700 1 --2- 172.21.15.32:0/1310416283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71dc0ab710 0x7f71dc061780 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e2654700 1 -- 172.21.15.32:0/1310416283 >> 172.21.15.32:0/1310416283 conn(0x7f71dc0b5530 msgr2=0x7f71dc0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:24.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e2654700 1 -- 172.21.15.32:0/1310416283 shutdown_connections 2020-12-05T17:35:24.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e2654700 1 -- 172.21.15.32:0/1310416283 wait complete. 2020-12-05T17:35:24.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e2654700 1 Processor -- start 2020-12-05T17:35:24.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e2654700 1 -- start start 2020-12-05T17:35:24.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e2654700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71dc0ab710 0x7f71dc142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:24.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e2654700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71dc0b1fc0 0x7f71dc148390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:24.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e2654700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71dc147400 0x7f71dc148880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:24.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e2654700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc0ad520 con 0x7f71dc147400 2020-12-05T17:35:24.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e2654700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc0ad7c0 con 0x7f71dc0ab710 2020-12-05T17:35:24.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e2654700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc142dc0 con 0x7f71dc0b1fc0 2020-12-05T17:35:24.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e1e53700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71dc147400 0x7f71dc148880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:24.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e1e53700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71dc147400 0x7f71dc148880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35316/0 (socket says 172.21.15.32:35316) 2020-12-05T17:35:24.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e1e53700 1 -- 172.21.15.32:0/2463933266 learned_addr learned my addr 172.21.15.32:0/2463933266 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:24.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e1652700 1 --2- 172.21.15.32:0/2463933266 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71dc0ab710 0x7f71dc142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:24.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.535+0000 7f71e0e51700 1 --2- 172.21.15.32:0/2463933266 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71dc0b1fc0 0x7f71dc148390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:24.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.539+0000 7f71e1e53700 1 -- 172.21.15.32:0/2463933266 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71dc0b1fc0 msgr2=0x7f71dc148390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:24.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.539+0000 7f71e1e53700 1 --2- 172.21.15.32:0/2463933266 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71dc0b1fc0 0x7f71dc148390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.539+0000 7f71e1e53700 1 -- 172.21.15.32:0/2463933266 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71dc0ab710 msgr2=0x7f71dc142250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:24.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.539+0000 7f71e1e53700 1 --2- 172.21.15.32:0/2463933266 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71dc0ab710 0x7f71dc142250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.539+0000 7f71e1e53700 1 -- 172.21.15.32:0/2463933266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71dc0ada60 con 0x7f71dc147400 2020-12-05T17:35:24.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.539+0000 7f71e1e53700 1 --2- 172.21.15.32:0/2463933266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71dc147400 0x7f71dc148880 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f71cc006a50 tx=0x7f71cc006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:24.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.539+0000 7f71da7fc700 1 -- 172.21.15.32:0/2463933266 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71cc00c7e0 con 0x7f71dc147400 2020-12-05T17:35:24.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.539+0000 7f71da7fc700 1 -- 172.21.15.32:0/2463933266 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71cc00c940 con 0x7f71dc147400 2020-12-05T17:35:24.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.539+0000 7f71e2654700 1 -- 172.21.15.32:0/2463933266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71dc148d70 con 0x7f71dc147400 2020-12-05T17:35:24.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.539+0000 7f71da7fc700 1 -- 172.21.15.32:0/2463933266 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71cc00cca0 con 0x7f71dc147400 2020-12-05T17:35:24.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.539+0000 7f71e2654700 1 -- 172.21.15.32:0/2463933266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71dc14aba0 con 0x7f71dc147400 2020-12-05T17:35:24.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.539+0000 7f71e2654700 1 -- 172.21.15.32:0/2463933266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71dc057050 con 0x7f71dc147400 2020-12-05T17:35:24.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.539+0000 7f71da7fc700 1 -- 172.21.15.32:0/2463933266 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f71cc026430 con 0x7f71dc147400 2020-12-05T17:35:24.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.539+0000 7f71da7fc700 1 --2- 172.21.15.32:0/2463933266 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71bc055440 0x7f71bc057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:24.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.539+0000 7f71da7fc700 1 -- 172.21.15.32:0/2463933266 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f71cc06ee90 con 0x7f71dc147400 2020-12-05T17:35:24.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.539+0000 7f71e1652700 1 --2- 172.21.15.32:0/2463933266 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71bc055440 0x7f71bc057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:24.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.543+0000 7f71e1652700 1 --2- 172.21.15.32:0/2463933266 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71bc055440 0x7f71bc057900 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f71d400b000 tx=0x7f71d40091a0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:24.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.547+0000 7f71da7fc700 1 -- 172.21.15.32:0/2463933266 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f71cc044fd0 con 0x7f71dc147400 2020-12-05T17:35:24.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.731+0000 7f71e2654700 1 -- 172.21.15.32:0/2463933266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71dc0b33c0 con 0x7f71dc147400 2020-12-05T17:35:24.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.735+0000 7f71da7fc700 1 -- 172.21.15.32:0/2463933266 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2136 (secure 0 0 0) 0x7f71cc0485f0 con 0x7f71dc147400 2020-12-05T17:35:24.733 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:35:24.733 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5159,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/2123765277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2123765277},{"type":"v1","addr":"172.21.15.61:6833","nonce":2123765277}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:35:23.386407+0000","modified":"2020-12-05T17:35:24.389244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5640},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5640":{"gid":5640,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/681412338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":681412338},{"type":"v1","addr":"172.21.15.32:6835","nonce":681412338}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-12-05T17:35:24.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.735+0000 7f71e2654700 1 -- 172.21.15.32:0/2463933266 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71bc055440 msgr2=0x7f71bc057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:24.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.735+0000 7f71e2654700 1 --2- 172.21.15.32:0/2463933266 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71bc055440 0x7f71bc057900 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f71d400b000 tx=0x7f71d40091a0).stop 2020-12-05T17:35:24.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.735+0000 7f71e2654700 1 -- 172.21.15.32:0/2463933266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71dc147400 msgr2=0x7f71dc148880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:24.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.735+0000 7f71e2654700 1 --2- 172.21.15.32:0/2463933266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71dc147400 0x7f71dc148880 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f71cc006a50 tx=0x7f71cc006dc0).stop 2020-12-05T17:35:24.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.735+0000 7f71e2654700 1 -- 172.21.15.32:0/2463933266 shutdown_connections 2020-12-05T17:35:24.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.735+0000 7f71e2654700 1 --2- 172.21.15.32:0/2463933266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71dc147400 0x7f71dc148880 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.735+0000 7f71e2654700 1 --2- 172.21.15.32:0/2463933266 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71dc0ab710 0x7f71dc142250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.735+0000 7f71e2654700 1 --2- 172.21.15.32:0/2463933266 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71bc055440 0x7f71bc057900 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.735+0000 7f71e2654700 1 --2- 172.21.15.32:0/2463933266 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71dc0b1fc0 0x7f71dc148390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.735+0000 7f71e2654700 1 -- 172.21.15.32:0/2463933266 >> 172.21.15.32:0/2463933266 conn(0x7f71dc0b5530 msgr2=0x7f71dc1444c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:24.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.735+0000 7f71e2654700 1 -- 172.21.15.32:0/2463933266 shutdown_connections 2020-12-05T17:35:24.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.735+0000 7f71e2654700 1 -- 172.21.15.32:0/2463933266 wait complete. 2020-12-05T17:35:24.737 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 23 2020-12-05T17:35:24.746 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:35:24.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.859+0000 7f3743fff700 1 Processor -- start 2020-12-05T17:35:24.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.859+0000 7f3743fff700 1 -- start start 2020-12-05T17:35:24.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.859+0000 7f3743fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f37440b2070 0x7f3744138e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:24.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.859+0000 7f3743fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37440b2450 con 0x7f3744139320 2020-12-05T17:35:24.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.859+0000 7f3743fff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37441437a0 con 0x7f37440b2ca0 2020-12-05T17:35:24.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.859+0000 7f3743fff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37440acb80 con 0x7f37440b2070 2020-12-05T17:35:24.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.859+0000 7f37437fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3744139320 0x7f3744143090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37972/0 (socket says 172.21.15.32:37972) 2020-12-05T17:35:24.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.859+0000 7f37437fe700 1 -- 172.21.15.32:0/1084782355 learned_addr learned my addr 172.21.15.32:0/1084782355 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:35:24.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.859+0000 7f3742ffd700 1 --2- 172.21.15.32:0/1084782355 >> v2:172.21.15.61:3300/0 conn(0x7f37440b2070 0x7f3744138e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:24.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.859+0000 7f37427fc700 1 -- 172.21.15.32:0/1084782355 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 622726141 0 0) 0x7f37440b2450 con 0x7f3744139320 2020-12-05T17:35:24.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.859+0000 7f37427fc700 1 -- 172.21.15.32:0/1084782355 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3724003230 con 0x7f3744139320 2020-12-05T17:35:24.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.859+0000 7f37427fc700 1 -- 172.21.15.32:0/1084782355 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445523553 0 0) 0x7f37441437a0 con 0x7f37440b2ca0 2020-12-05T17:35:24.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.859+0000 7f37427fc700 1 -- 172.21.15.32:0/1084782355 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37440b2450 con 0x7f37440b2ca0 2020-12-05T17:35:24.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.859+0000 7f3742ffd700 1 -- 172.21.15.32:0/1084782355 >> v1:172.21.15.61:6789/0 conn(0x7f37440b2ca0 legacy=0x7f3744142980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:24.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.859+0000 7f3742ffd700 1 -- 172.21.15.32:0/1084782355 >> v1:172.21.15.32:6789/0 conn(0x7f3744139320 legacy=0x7f3744143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:24.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.859+0000 7f3742ffd700 1 -- 172.21.15.32:0/1084782355 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3744066d40 con 0x7f37440b2070 2020-12-05T17:35:24.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.859+0000 7f3742ffd700 1 --2- 172.21.15.32:0/1084782355 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37440b2070 0x7f3744138e30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f3744067360 tx=0x7f3734006dc0).ready entity=mon.1 client_cookie=49e4ed7c01b46c3d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:24.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.859+0000 7f37427fc700 1 -- 172.21.15.32:0/1084782355 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3734009740 con 0x7f37440b2070 2020-12-05T17:35:24.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.859+0000 7f37427fc700 1 -- 172.21.15.32:0/1084782355 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37340098a0 con 0x7f37440b2070 2020-12-05T17:35:24.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.859+0000 7f37427fc700 1 -- 172.21.15.32:0/1084782355 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3734009c00 con 0x7f37440b2070 2020-12-05T17:35:24.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.859+0000 7f3743fff700 1 -- 172.21.15.32:0/1084782355 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37440b2070 msgr2=0x7f3744138e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:24.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.859+0000 7f3743fff700 1 --2- 172.21.15.32:0/1084782355 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37440b2070 0x7f3744138e30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f3744067360 tx=0x7f3734006dc0).stop 2020-12-05T17:35:24.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.863+0000 7f3743fff700 1 -- 172.21.15.32:0/1084782355 shutdown_connections 2020-12-05T17:35:24.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.863+0000 7f3743fff700 1 --2- 172.21.15.32:0/1084782355 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37440b2070 0x7f3744138e30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.863+0000 7f3743fff700 1 -- 172.21.15.32:0/1084782355 >> 172.21.15.32:0/1084782355 conn(0x7f374405bbb0 msgr2=0x7f3744062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:24.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.863+0000 7f3743fff700 1 -- 172.21.15.32:0/1084782355 shutdown_connections 2020-12-05T17:35:24.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.863+0000 7f3743fff700 1 -- 172.21.15.32:0/1084782355 wait complete. 2020-12-05T17:35:24.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.863+0000 7f3743fff700 1 Processor -- start 2020-12-05T17:35:24.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.863+0000 7f3743fff700 1 -- start start 2020-12-05T17:35:24.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.863+0000 7f3743fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37440b2070 0x7f3744151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:24.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.863+0000 7f3743fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f37440b2ca0 0x7f3744142f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:24.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.863+0000 7f3743fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3744139320 0x7f37441462b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:24.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.863+0000 7f3743fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3744143dd0 con 0x7f37440b2070 2020-12-05T17:35:24.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.863+0000 7f3743fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3744065730 con 0x7f3744139320 2020-12-05T17:35:24.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.863+0000 7f3743fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37441467a0 con 0x7f37440b2ca0 2020-12-05T17:35:24.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.863+0000 7f3742ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37440b2070 0x7f3744151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:24.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.863+0000 7f3742ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37440b2070 0x7f3744151740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35326/0 (socket says 172.21.15.32:35326) 2020-12-05T17:35:24.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.863+0000 7f3742ffd700 1 -- 172.21.15.32:0/1228413601 learned_addr learned my addr 172.21.15.32:0/1228413601 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:24.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.863+0000 7f37437fe700 1 --2- 172.21.15.32:0/1228413601 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3744139320 0x7f37441462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:24.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.863+0000 7f373a7fc700 1 --2- 172.21.15.32:0/1228413601 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f37440b2ca0 0x7f3744142f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:24.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.867+0000 7f3742ffd700 1 -- 172.21.15.32:0/1228413601 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f37440b2ca0 msgr2=0x7f3744142f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:24.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.867+0000 7f3742ffd700 1 --2- 172.21.15.32:0/1228413601 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f37440b2ca0 0x7f3744142f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.867+0000 7f3742ffd700 1 -- 172.21.15.32:0/1228413601 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3744139320 msgr2=0x7f37441462b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:24.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.867+0000 7f3742ffd700 1 --2- 172.21.15.32:0/1228413601 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3744139320 0x7f37441462b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:24.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.867+0000 7f3742ffd700 1 -- 172.21.15.32:0/1228413601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3744065350 con 0x7f37440b2070 2020-12-05T17:35:24.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.867+0000 7f373a7fc700 1 --2- 172.21.15.32:0/1228413601 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f37440b2ca0 0x7f3744142f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:24.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.867+0000 7f3742ffd700 1 --2- 172.21.15.32:0/1228413601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37440b2070 0x7f3744151740 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f3744067360 tx=0x7f3734015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:24.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.867+0000 7f3740ff9700 1 -- 172.21.15.32:0/1228413601 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3734011a50 con 0x7f37440b2070 2020-12-05T17:35:24.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.867+0000 7f3740ff9700 1 -- 172.21.15.32:0/1228413601 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3734011bb0 con 0x7f37440b2070 2020-12-05T17:35:24.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.867+0000 7f3743fff700 1 -- 172.21.15.32:0/1228413601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3744066150 con 0x7f37440b2070 2020-12-05T17:35:24.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.867+0000 7f3740ff9700 1 -- 172.21.15.32:0/1228413601 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f373401d450 con 0x7f37440b2070 2020-12-05T17:35:24.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.867+0000 7f3743fff700 1 -- 172.21.15.32:0/1228413601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3744139d00 con 0x7f37440b2070 2020-12-05T17:35:24.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.867+0000 7f3743fff700 1 -- 172.21.15.32:0/1228413601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3744143f40 con 0x7f37440b2070 2020-12-05T17:35:24.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.867+0000 7f3740ff9700 1 -- 172.21.15.32:0/1228413601 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f373400f070 con 0x7f37440b2070 2020-12-05T17:35:24.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.867+0000 7f3740ff9700 1 --2- 172.21.15.32:0/1228413601 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3728055440 0x7f3728057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:24.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.867+0000 7f373a7fc700 1 --2- 172.21.15.32:0/1228413601 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3728055440 0x7f3728057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:24.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.867+0000 7f3740ff9700 1 -- 172.21.15.32:0/1228413601 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f373406ef60 con 0x7f37440b2070 2020-12-05T17:35:24.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.871+0000 7f373a7fc700 1 --2- 172.21.15.32:0/1228413601 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3728055440 0x7f3728057900 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f372c00b000 tx=0x7f372c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:24.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:24.875+0000 7f3740ff9700 1 -- 172.21.15.32:0/1228413601 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3734045060 con 0x7f37440b2070 2020-12-05T17:35:25.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.019+0000 7f3743fff700 1 -- 172.21.15.32:0/1228413601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3744057050 con 0x7f37440b2070 2020-12-05T17:35:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.023+0000 7f3740ff9700 1 -- 172.21.15.32:0/1228413601 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16286 (secure 0 0 0) 0x7f3734022070 con 0x7f37440b2070 2020-12-05T17:35:25.022 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:35:25.022 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":53,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:35:24.389886+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:35:20.867930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-12-05T17:35:22.139660+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:35:25.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.023+0000 7f3743fff700 1 -- 172.21.15.32:0/1228413601 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3728055440 msgr2=0x7f3728057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:25.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.023+0000 7f3743fff700 1 --2- 172.21.15.32:0/1228413601 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3728055440 0x7f3728057900 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f372c00b000 tx=0x7f372c009040).stop 2020-12-05T17:35:25.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.023+0000 7f3743fff700 1 -- 172.21.15.32:0/1228413601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37440b2070 msgr2=0x7f3744151740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:25.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.023+0000 7f3743fff700 1 --2- 172.21.15.32:0/1228413601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37440b2070 0x7f3744151740 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f3744067360 tx=0x7f3734015000).stop 2020-12-05T17:35:25.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.023+0000 7f3743fff700 1 -- 172.21.15.32:0/1228413601 shutdown_connections 2020-12-05T17:35:25.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.023+0000 7f3743fff700 1 --2- 172.21.15.32:0/1228413601 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3744139320 0x7f37441462b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:25.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.023+0000 7f3743fff700 1 --2- 172.21.15.32:0/1228413601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37440b2070 0x7f3744151740 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:25.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.023+0000 7f3743fff700 1 --2- 172.21.15.32:0/1228413601 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3728055440 0x7f3728057900 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:25.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.023+0000 7f3743fff700 1 --2- 172.21.15.32:0/1228413601 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f37440b2ca0 0x7f3744142f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:25.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.023+0000 7f3743fff700 1 -- 172.21.15.32:0/1228413601 >> 172.21.15.32:0/1228413601 conn(0x7f374405bbb0 msgr2=0x7f37440b49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:25.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.023+0000 7f3743fff700 1 -- 172.21.15.32:0/1228413601 shutdown_connections 2020-12-05T17:35:25.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.023+0000 7f3743fff700 1 -- 172.21.15.32:0/1228413601 wait complete. 2020-12-05T17:35:25.036 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:35:25.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.163+0000 7fa0d8e78700 1 Processor -- start 2020-12-05T17:35:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.163+0000 7fa0d8e78700 1 -- start start 2020-12-05T17:35:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d8e78700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa0d40611b0 0x7fa0d4153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d8e78700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0d40ae860 0x7fa0d414f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d8e78700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0d4080a90 con 0x7fa0d40b0240 2020-12-05T17:35:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d8e78700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0d4152860 con 0x7fa0d40611b0 2020-12-05T17:35:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d8e78700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d4152b00 con 0x7fa0d40ae860 2020-12-05T17:35:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d37fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa0d40b0240 0x7fa0d414c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37986/0 (socket says 172.21.15.32:37986) 2020-12-05T17:35:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d37fe700 1 -- 172.21.15.32:0/4223350019 learned_addr learned my addr 172.21.15.32:0/4223350019 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:35:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0da0da700 1 --2- 172.21.15.32:0/4223350019 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0d40ae860 0x7fa0d414f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:25.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d3fff700 1 --2- 172.21.15.32:0/4223350019 >> v2:172.21.15.61:3300/0 conn(0x7fa0d40611b0 0x7fa0d4153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:25.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d2ffd700 1 -- 172.21.15.32:0/4223350019 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2076280888 0 0) 0x7fa0d4080a90 con 0x7fa0d40b0240 2020-12-05T17:35:25.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d2ffd700 1 -- 172.21.15.32:0/4223350019 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0bc003230 con 0x7fa0d40b0240 2020-12-05T17:35:25.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d3fff700 1 -- 172.21.15.32:0/4223350019 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0d40ae860 msgr2=0x7fa0d414f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:25.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d3fff700 1 --2- 172.21.15.32:0/4223350019 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0d40ae860 0x7fa0d414f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:25.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d3fff700 1 -- 172.21.15.32:0/4223350019 >> v1:172.21.15.32:6789/0 conn(0x7fa0d40b0240 legacy=0x7fa0d414c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:25.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d3fff700 1 -- 172.21.15.32:0/4223350019 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d40aec90 con 0x7fa0d40611b0 2020-12-05T17:35:25.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0da0da700 1 --2- 172.21.15.32:0/4223350019 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0d40ae860 0x7fa0d414f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:35:25.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d3fff700 1 --2- 172.21.15.32:0/4223350019 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0d40611b0 0x7fa0d4153240 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fa0c4002f30 tx=0x7fa0c4006dc0).ready entity=mon.1 client_cookie=f57705eb5c73067f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:25.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d2ffd700 1 -- 172.21.15.32:0/4223350019 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0c40096a0 con 0x7fa0d40611b0 2020-12-05T17:35:25.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d2ffd700 1 -- 172.21.15.32:0/4223350019 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c4009800 con 0x7fa0d40611b0 2020-12-05T17:35:25.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d2ffd700 1 -- 172.21.15.32:0/4223350019 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0c4009b60 con 0x7fa0d40611b0 2020-12-05T17:35:25.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d8e78700 1 -- 172.21.15.32:0/4223350019 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0d40611b0 msgr2=0x7fa0d4153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:25.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d8e78700 1 --2- 172.21.15.32:0/4223350019 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0d40611b0 0x7fa0d4153240 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fa0c4002f30 tx=0x7fa0c4006dc0).stop 2020-12-05T17:35:25.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d8e78700 1 -- 172.21.15.32:0/4223350019 shutdown_connections 2020-12-05T17:35:25.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d8e78700 1 --2- 172.21.15.32:0/4223350019 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0d40ae860 0x7fa0d414f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:25.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d8e78700 1 --2- 172.21.15.32:0/4223350019 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0d40611b0 0x7fa0d4153240 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:25.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d8e78700 1 -- 172.21.15.32:0/4223350019 >> 172.21.15.32:0/4223350019 conn(0x7fa0d40b5f90 msgr2=0x7fa0d40b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:25.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d8e78700 1 -- 172.21.15.32:0/4223350019 shutdown_connections 2020-12-05T17:35:25.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.167+0000 7fa0d8e78700 1 -- 172.21.15.32:0/4223350019 wait complete. 2020-12-05T17:35:25.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0d8e78700 1 Processor -- start 2020-12-05T17:35:25.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0d8e78700 1 -- start start 2020-12-05T17:35:25.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0d8e78700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0d40611b0 0x7fa0d40b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:25.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0d8e78700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0d40ae860 0x7fa0d4064e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:25.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0d8e78700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0d40b0240 0x7fa0d40652f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:25.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0d8e78700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d40b3270 con 0x7fa0d40ae860 2020-12-05T17:35:25.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0d8e78700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d40b3510 con 0x7fa0d40611b0 2020-12-05T17:35:25.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0d8e78700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d40b37b0 con 0x7fa0d40b0240 2020-12-05T17:35:25.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0d37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0d40ae860 0x7fa0d4064e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:25.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0d37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0d40ae860 0x7fa0d4064e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35342/0 (socket says 172.21.15.32:35342) 2020-12-05T17:35:25.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0d37fe700 1 -- 172.21.15.32:0/1393632446 learned_addr learned my addr 172.21.15.32:0/1393632446 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:25.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0d3fff700 1 --2- 172.21.15.32:0/1393632446 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0d40611b0 0x7fa0d40b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:25.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0da0da700 1 --2- 172.21.15.32:0/1393632446 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0d40b0240 0x7fa0d40652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:25.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0d37fe700 1 -- 172.21.15.32:0/1393632446 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0d40b0240 msgr2=0x7fa0d40652f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:25.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0d37fe700 1 --2- 172.21.15.32:0/1393632446 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0d40b0240 0x7fa0d40652f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:25.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0d37fe700 1 -- 172.21.15.32:0/1393632446 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0d40611b0 msgr2=0x7fa0d40b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:25.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0d37fe700 1 --2- 172.21.15.32:0/1393632446 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0d40611b0 0x7fa0d40b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:25.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0d37fe700 1 -- 172.21.15.32:0/1393632446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d40b29e0 con 0x7fa0d40ae860 2020-12-05T17:35:25.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0d37fe700 1 --2- 172.21.15.32:0/1393632446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0d40ae860 0x7fa0d4064e00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fa0c80010d0 tx=0x7fa0c8007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:25.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0d0ff9700 1 -- 172.21.15.32:0/1393632446 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0c80072e0 con 0x7fa0d40ae860 2020-12-05T17:35:25.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0d0ff9700 1 -- 172.21.15.32:0/1393632446 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c8007440 con 0x7fa0d40ae860 2020-12-05T17:35:25.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0d8e78700 1 -- 172.21.15.32:0/1393632446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d40638b0 con 0x7fa0d40ae860 2020-12-05T17:35:25.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0d0ff9700 1 -- 172.21.15.32:0/1393632446 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0c80057d0 con 0x7fa0d40ae860 2020-12-05T17:35:25.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.171+0000 7fa0d8e78700 1 -- 172.21.15.32:0/1393632446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d40657e0 con 0x7fa0d40ae860 2020-12-05T17:35:25.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.175+0000 7fa0d0ff9700 1 -- 172.21.15.32:0/1393632446 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa0c801a060 con 0x7fa0d40ae860 2020-12-05T17:35:25.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.175+0000 7fa0d8e78700 1 -- 172.21.15.32:0/1393632446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0d40b3060 con 0x7fa0d40ae860 2020-12-05T17:35:25.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.175+0000 7fa0d0ff9700 1 --2- 172.21.15.32:0/1393632446 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa0b40554d0 0x7fa0b4057990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:25.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.175+0000 7fa0d0ff9700 1 -- 172.21.15.32:0/1393632446 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa0c806d880 con 0x7fa0d40ae860 2020-12-05T17:35:25.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.175+0000 7fa0d3fff700 1 --2- 172.21.15.32:0/1393632446 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa0b40554d0 0x7fa0b4057990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:25.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.175+0000 7fa0d3fff700 1 --2- 172.21.15.32:0/1393632446 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa0b40554d0 0x7fa0b4057990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fa0c401b000 tx=0x7fa0c4015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:25.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.179+0000 7fa0d0ff9700 1 -- 172.21.15.32:0/1393632446 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa0c8043a20 con 0x7fa0d40ae860 2020-12-05T17:35:25.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.371+0000 7fa0d8e78700 1 -- 172.21.15.32:0/1393632446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fa0d4057050 con 0x7fa0d40ae860 2020-12-05T17:35:25.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.375+0000 7fa0d0ff9700 1 -- 172.21.15.32:0/1393632446 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v17) v1 ==== 184+0+0 (secure 0 0 0) 0x7fa0c8018020 con 0x7fa0d40ae860 2020-12-05T17:35:25.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.379+0000 7fa0d8e78700 1 -- 172.21.15.32:0/1393632446 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa0b40554d0 msgr2=0x7fa0b4057990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:25.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.379+0000 7fa0d8e78700 1 --2- 172.21.15.32:0/1393632446 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa0b40554d0 0x7fa0b4057990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fa0c401b000 tx=0x7fa0c4015000).stop 2020-12-05T17:35:25.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.379+0000 7fa0d8e78700 1 -- 172.21.15.32:0/1393632446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0d40ae860 msgr2=0x7fa0d4064e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:25.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.379+0000 7fa0d8e78700 1 --2- 172.21.15.32:0/1393632446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0d40ae860 0x7fa0d4064e00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fa0c80010d0 tx=0x7fa0c8007b50).stop 2020-12-05T17:35:25.381 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.379+0000 7fa0d8e78700 1 -- 172.21.15.32:0/1393632446 shutdown_connections 2020-12-05T17:35:25.381 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.379+0000 7fa0d8e78700 1 --2- 172.21.15.32:0/1393632446 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa0b40554d0 0x7fa0b4057990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:25.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.379+0000 7fa0d8e78700 1 --2- 172.21.15.32:0/1393632446 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0d40b0240 0x7fa0d40652f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:25.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.379+0000 7fa0d8e78700 1 --2- 172.21.15.32:0/1393632446 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0d40611b0 0x7fa0d40b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:25.383 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.379+0000 7fa0d8e78700 1 --2- 172.21.15.32:0/1393632446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0d40ae860 0x7fa0d4064e00 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:25.383 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.379+0000 7fa0d8e78700 1 -- 172.21.15.32:0/1393632446 >> 172.21.15.32:0/1393632446 conn(0x7fa0d40b5f90 msgr2=0x7fa0d4061590 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:25.383 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.379+0000 7fa0d8e78700 1 -- 172.21.15.32:0/1393632446 shutdown_connections 2020-12-05T17:35:25.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.379+0000 7fa0d8e78700 1 -- 172.21.15.32:0/1393632446 wait complete. 2020-12-05T17:35:25.384 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2020-12-05T17:35:25.392 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:35:25.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.507+0000 7efd1e8ad700 1 Processor -- start 2020-12-05T17:35:25.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.507+0000 7efd1e8ad700 1 -- start start 2020-12-05T17:35:25.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.507+0000 7efd1e8ad700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efd1805f6f0 0x7efd180adef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:25.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.507+0000 7efd1e8ad700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd180b0380 con 0x7efd18061780 2020-12-05T17:35:25.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.507+0000 7efd1e8ad700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd1805fad0 con 0x7efd1805feb0 2020-12-05T17:35:25.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.507+0000 7efd1e8ad700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd180aeaf0 con 0x7efd1805f6f0 2020-12-05T17:35:25.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.507+0000 7efd1ce4a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efd1805f6f0 0x7efd180adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.507+0000 7efd1d64b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efd18061780 0x7efd1814e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38000/0 (socket says 172.21.15.32:38000) 2020-12-05T17:35:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.507+0000 7efd1d64b700 1 -- 172.21.15.32:0/3293731893 learned_addr learned my addr 172.21.15.32:0/3293731893 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:35:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.507+0000 7efd1ce4a700 1 -- 172.21.15.32:0/3293731893 >> v1:172.21.15.61:6789/0 conn(0x7efd1805feb0 legacy=0x7efd180ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.507+0000 7efd1ce4a700 1 -- 172.21.15.32:0/3293731893 >> v1:172.21.15.32:6789/0 conn(0x7efd18061780 legacy=0x7efd1814e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.507+0000 7efd177fe700 1 -- 172.21.15.32:0/3293731893 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3065482818 0 0) 0x7efd180b0380 con 0x7efd18061780 2020-12-05T17:35:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.507+0000 7efd1ce4a700 1 -- 172.21.15.32:0/3293731893 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd180aecd0 con 0x7efd1805f6f0 2020-12-05T17:35:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.507+0000 7efd1ce4a700 1 --2- 172.21.15.32:0/3293731893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd1805f6f0 0x7efd180adef0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7efd080032b0 tx=0x7efd080091a0).ready entity=mon.0 client_cookie=f78e4b125f8ffaea server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.511+0000 7efd177fe700 1 -- 172.21.15.32:0/3293731893 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd0800ba50 con 0x7efd1805f6f0 2020-12-05T17:35:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.511+0000 7efd177fe700 1 -- 172.21.15.32:0/3293731893 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0800bbb0 con 0x7efd1805f6f0 2020-12-05T17:35:25.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.511+0000 7efd177fe700 1 -- 172.21.15.32:0/3293731893 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd0800bd20 con 0x7efd1805f6f0 2020-12-05T17:35:25.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.511+0000 7efd1e8ad700 1 -- 172.21.15.32:0/3293731893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd1805f6f0 msgr2=0x7efd180adef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:25.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.511+0000 7efd1e8ad700 1 --2- 172.21.15.32:0/3293731893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd1805f6f0 0x7efd180adef0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7efd080032b0 tx=0x7efd080091a0).stop 2020-12-05T17:35:25.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.511+0000 7efd1e8ad700 1 -- 172.21.15.32:0/3293731893 shutdown_connections 2020-12-05T17:35:25.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.511+0000 7efd1e8ad700 1 --2- 172.21.15.32:0/3293731893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd1805f6f0 0x7efd180adef0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:25.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.511+0000 7efd1e8ad700 1 -- 172.21.15.32:0/3293731893 >> 172.21.15.32:0/3293731893 conn(0x7efd180b4b10 msgr2=0x7efd180b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:25.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.511+0000 7efd1e8ad700 1 -- 172.21.15.32:0/3293731893 shutdown_connections 2020-12-05T17:35:25.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.511+0000 7efd1e8ad700 1 -- 172.21.15.32:0/3293731893 wait complete. 2020-12-05T17:35:25.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.511+0000 7efd1e8ad700 1 Processor -- start 2020-12-05T17:35:25.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.511+0000 7efd1e8ad700 1 -- start start 2020-12-05T17:35:25.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.511+0000 7efd1e8ad700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd1805f6f0 0x7efd1815a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:25.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.511+0000 7efd1e8ad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd1805feb0 0x7efd18151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:25.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.511+0000 7efd1e8ad700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efd18061780 0x7efd18134a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:25.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.511+0000 7efd1e8ad700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd18159c30 con 0x7efd1805feb0 2020-12-05T17:35:25.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.511+0000 7efd1e8ad700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd180721d0 con 0x7efd18061780 2020-12-05T17:35:25.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.511+0000 7efd1e8ad700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd18071d40 con 0x7efd1805f6f0 2020-12-05T17:35:25.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.511+0000 7efd17fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd1805feb0 0x7efd18151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:25.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.511+0000 7efd1ce4a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd1805f6f0 0x7efd1815a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:25.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.511+0000 7efd17fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd1805feb0 0x7efd18151970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35356/0 (socket says 172.21.15.32:35356) 2020-12-05T17:35:25.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.511+0000 7efd17fff700 1 -- 172.21.15.32:0/4128887704 learned_addr learned my addr 172.21.15.32:0/4128887704 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:25.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.511+0000 7efd1d64b700 1 --2- 172.21.15.32:0/4128887704 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efd18061780 0x7efd18134a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:25.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.515+0000 7efd17fff700 1 -- 172.21.15.32:0/4128887704 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd1805f6f0 msgr2=0x7efd1815a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:25.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.515+0000 7efd17fff700 1 --2- 172.21.15.32:0/4128887704 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd1805f6f0 0x7efd1815a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:25.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.515+0000 7efd17fff700 1 -- 172.21.15.32:0/4128887704 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efd18061780 msgr2=0x7efd18134a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:25.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.515+0000 7efd17fff700 1 --2- 172.21.15.32:0/4128887704 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efd18061780 0x7efd18134a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:25.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.515+0000 7efd17fff700 1 -- 172.21.15.32:0/4128887704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd18159e60 con 0x7efd1805feb0 2020-12-05T17:35:25.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.515+0000 7efd17fff700 1 --2- 172.21.15.32:0/4128887704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd1805feb0 0x7efd18151970 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7efd18061550 tx=0x7efd00007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:25.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.515+0000 7efd157fa700 1 -- 172.21.15.32:0/4128887704 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd00013410 con 0x7efd1805feb0 2020-12-05T17:35:25.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.515+0000 7efd157fa700 1 -- 172.21.15.32:0/4128887704 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd00013570 con 0x7efd1805feb0 2020-12-05T17:35:25.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.515+0000 7efd1e8ad700 1 -- 172.21.15.32:0/4128887704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1806fca0 con 0x7efd1805feb0 2020-12-05T17:35:25.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.515+0000 7efd157fa700 1 -- 172.21.15.32:0/4128887704 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd000138d0 con 0x7efd1805feb0 2020-12-05T17:35:25.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.515+0000 7efd1e8ad700 1 -- 172.21.15.32:0/4128887704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd18070320 con 0x7efd1805feb0 2020-12-05T17:35:25.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.515+0000 7efd157fa700 1 -- 172.21.15.32:0/4128887704 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7efd00010030 con 0x7efd1805feb0 2020-12-05T17:35:25.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.515+0000 7efd1e8ad700 1 -- 172.21.15.32:0/4128887704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd18057050 con 0x7efd1805feb0 2020-12-05T17:35:25.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.515+0000 7efd157fa700 1 --2- 172.21.15.32:0/4128887704 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efcfc055440 0x7efcfc057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:25.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.519+0000 7efd157fa700 1 -- 172.21.15.32:0/4128887704 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7efd0006df10 con 0x7efd1805feb0 2020-12-05T17:35:25.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.519+0000 7efd1ce4a700 1 --2- 172.21.15.32:0/4128887704 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efcfc055440 0x7efcfc057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:25.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.519+0000 7efd1ce4a700 1 --2- 172.21.15.32:0/4128887704 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efcfc055440 0x7efcfc057900 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7efd0801b000 tx=0x7efd08016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:25.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.523+0000 7efd157fa700 1 -- 172.21.15.32:0/4128887704 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7efd00043fa0 con 0x7efd1805feb0 2020-12-05T17:35:25.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.711+0000 7efd1e8ad700 1 -- 172.21.15.32:0/4128887704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7efd1806fe30 con 0x7efd1805feb0 2020-12-05T17:35:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.719+0000 7efd157fa700 1 -- 172.21.15.32:0/4128887704 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v18) v1 ==== 184+0+0 (secure 0 0 0) 0x7efd1806fe30 con 0x7efd1805feb0 2020-12-05T17:35:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.719+0000 7efd1e8ad700 1 -- 172.21.15.32:0/4128887704 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efcfc055440 msgr2=0x7efcfc057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.719+0000 7efd1e8ad700 1 --2- 172.21.15.32:0/4128887704 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efcfc055440 0x7efcfc057900 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7efd0801b000 tx=0x7efd08016000).stop 2020-12-05T17:35:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.719+0000 7efd1e8ad700 1 -- 172.21.15.32:0/4128887704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd1805feb0 msgr2=0x7efd18151970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.719+0000 7efd1e8ad700 1 --2- 172.21.15.32:0/4128887704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd1805feb0 0x7efd18151970 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7efd18061550 tx=0x7efd00007bf0).stop 2020-12-05T17:35:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.719+0000 7efd1e8ad700 1 -- 172.21.15.32:0/4128887704 shutdown_connections 2020-12-05T17:35:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.719+0000 7efd1e8ad700 1 --2- 172.21.15.32:0/4128887704 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efd18061780 0x7efd18134a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.719+0000 7efd1e8ad700 1 --2- 172.21.15.32:0/4128887704 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efcfc055440 0x7efcfc057900 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.719+0000 7efd1e8ad700 1 --2- 172.21.15.32:0/4128887704 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efd1805f6f0 0x7efd1815a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.719+0000 7efd1e8ad700 1 --2- 172.21.15.32:0/4128887704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd1805feb0 0x7efd18151970 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.719+0000 7efd1e8ad700 1 -- 172.21.15.32:0/4128887704 >> 172.21.15.32:0/4128887704 conn(0x7efd180b4b10 msgr2=0x7efd180b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.719+0000 7efd1e8ad700 1 -- 172.21.15.32:0/4128887704 shutdown_connections 2020-12-05T17:35:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.719+0000 7efd1e8ad700 1 -- 172.21.15.32:0/4128887704 wait complete. 2020-12-05T17:35:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2020-12-05T17:35:25.736 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:35:25.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.859+0000 7f595dd7e700 1 Processor -- start 2020-12-05T17:35:25.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.859+0000 7f595dd7e700 1 -- start start 2020-12-05T17:35:25.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.859+0000 7f595dd7e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59580611b0 0x7f5958153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:25.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.859+0000 7f595dd7e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5958080a90 con 0x7f59580ae860 2020-12-05T17:35:25.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.859+0000 7f595dd7e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59580aec40 con 0x7f59580b0240 2020-12-05T17:35:25.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.859+0000 7f595dd7e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5958152860 con 0x7f59580611b0 2020-12-05T17:35:25.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.859+0000 7f595d57d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59580611b0 0x7f5958153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:25.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.859+0000 7f595d57d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59580611b0 0x7f5958153240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35362/0 (socket says 172.21.15.32:35362) 2020-12-05T17:35:25.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.859+0000 7f595d57d700 1 -- 172.21.15.32:0/3703658940 learned_addr learned my addr 172.21.15.32:0/3703658940 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:25.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.859+0000 7f595d57d700 1 -- 172.21.15.32:0/3703658940 >> v1:172.21.15.61:6789/0 conn(0x7f59580b0240 legacy=0x7f595814c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:25.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.859+0000 7f595d57d700 1 -- 172.21.15.32:0/3703658940 >> v1:172.21.15.32:6789/0 conn(0x7f59580ae860 legacy=0x7f595814f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:25.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.859+0000 7f595d57d700 1 -- 172.21.15.32:0/3703658940 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5958152b00 con 0x7f59580611b0 2020-12-05T17:35:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.863+0000 7f595d57d700 1 --2- 172.21.15.32:0/3703658940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59580611b0 0x7f5958153240 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f59480032b0 tx=0x7f5948009270).ready entity=mon.0 client_cookie=c87ebbad4eaf2527 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.863+0000 7f5957fff700 1 -- 172.21.15.32:0/3703658940 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f594800bc00 con 0x7f59580611b0 2020-12-05T17:35:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.863+0000 7f5957fff700 1 -- 172.21.15.32:0/3703658940 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594800bd60 con 0x7f59580611b0 2020-12-05T17:35:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.863+0000 7f5957fff700 1 -- 172.21.15.32:0/3703658940 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5948011400 con 0x7f59580611b0 2020-12-05T17:35:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.863+0000 7f595dd7e700 1 -- 172.21.15.32:0/3703658940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59580611b0 msgr2=0x7f5958153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:25.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.863+0000 7f595dd7e700 1 --2- 172.21.15.32:0/3703658940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59580611b0 0x7f5958153240 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f59480032b0 tx=0x7f5948009270).stop 2020-12-05T17:35:25.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.863+0000 7f595dd7e700 1 -- 172.21.15.32:0/3703658940 shutdown_connections 2020-12-05T17:35:25.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.863+0000 7f595dd7e700 1 --2- 172.21.15.32:0/3703658940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59580611b0 0x7f5958153240 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:25.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.863+0000 7f595dd7e700 1 -- 172.21.15.32:0/3703658940 >> 172.21.15.32:0/3703658940 conn(0x7f59580b5f90 msgr2=0x7f59580b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:25.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.863+0000 7f595dd7e700 1 -- 172.21.15.32:0/3703658940 shutdown_connections 2020-12-05T17:35:25.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.863+0000 7f595dd7e700 1 -- 172.21.15.32:0/3703658940 wait complete. 2020-12-05T17:35:25.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f595dd7e700 1 Processor -- start 2020-12-05T17:35:25.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f595dd7e700 1 -- start start 2020-12-05T17:35:25.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f595dd7e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59580611b0 0x7f5958155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:25.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f595dd7e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f59580ae860 0x7f5958063b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:25.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f595dd7e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59580b0240 0x7f59580b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:25.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f595dd7e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59580663a0 con 0x7f59580611b0 2020-12-05T17:35:25.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f595dd7e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5958156150 con 0x7f59580b0240 2020-12-05T17:35:25.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f595dd7e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5958156380 con 0x7f59580ae860 2020-12-05T17:35:25.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f595d57d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59580611b0 0x7f5958155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:25.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f595d57d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59580611b0 0x7f5958155530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35368/0 (socket says 172.21.15.32:35368) 2020-12-05T17:35:25.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f595d57d700 1 -- 172.21.15.32:0/794521676 learned_addr learned my addr 172.21.15.32:0/794521676 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:25.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f595cd7c700 1 --2- 172.21.15.32:0/794521676 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f59580ae860 0x7f5958063b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:25.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f595efe0700 1 --2- 172.21.15.32:0/794521676 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59580b0240 0x7f59580b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:25.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f595d57d700 1 -- 172.21.15.32:0/794521676 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f59580ae860 msgr2=0x7f5958063b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:25.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f595d57d700 1 --2- 172.21.15.32:0/794521676 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f59580ae860 0x7f5958063b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:25.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f595d57d700 1 -- 172.21.15.32:0/794521676 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59580b0240 msgr2=0x7f59580b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:25.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f595d57d700 1 --2- 172.21.15.32:0/794521676 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59580b0240 0x7f59580b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:25.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f595d57d700 1 -- 172.21.15.32:0/794521676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5958065f00 con 0x7f59580611b0 2020-12-05T17:35:25.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f595cd7c700 1 --2- 172.21.15.32:0/794521676 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f59580ae860 0x7f5958063b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:25.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f595efe0700 1 --2- 172.21.15.32:0/794521676 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59580b0240 0x7f59580b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:25.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f595d57d700 1 --2- 172.21.15.32:0/794521676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59580611b0 0x7f5958155530 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f59480032b0 tx=0x7f5948016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:25.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f5955ffb700 1 -- 172.21.15.32:0/794521676 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59480107a0 con 0x7f59580611b0 2020-12-05T17:35:25.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f5955ffb700 1 -- 172.21.15.32:0/794521676 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5948010900 con 0x7f59580611b0 2020-12-05T17:35:25.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f595dd7e700 1 -- 172.21.15.32:0/794521676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5958063040 con 0x7f59580611b0 2020-12-05T17:35:25.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f5955ffb700 1 -- 172.21.15.32:0/794521676 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5948010c60 con 0x7f59580611b0 2020-12-05T17:35:25.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.867+0000 7f595dd7e700 1 -- 172.21.15.32:0/794521676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5958066e70 con 0x7f59580611b0 2020-12-05T17:35:25.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.871+0000 7f595dd7e700 1 -- 172.21.15.32:0/794521676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5958066090 con 0x7f59580611b0 2020-12-05T17:35:25.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.871+0000 7f5955ffb700 1 -- 172.21.15.32:0/794521676 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f594800f070 con 0x7f59580611b0 2020-12-05T17:35:25.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.871+0000 7f5955ffb700 1 --2- 172.21.15.32:0/794521676 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5938055320 0x7f59380577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:25.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.871+0000 7f595cd7c700 1 --2- 172.21.15.32:0/794521676 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5938055320 0x7f59380577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:25.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.871+0000 7f5955ffb700 1 -- 172.21.15.32:0/794521676 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f594806f810 con 0x7f59580611b0 2020-12-05T17:35:25.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.871+0000 7f595cd7c700 1 --2- 172.21.15.32:0/794521676 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5938055320 0x7f59380577e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f594c00a000 tx=0x7f594c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:25.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:25.879+0000 7f5955ffb700 1 -- 172.21.15.32:0/794521676 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5948045ae0 con 0x7f59580611b0 2020-12-05T17:35:26.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.063+0000 7f595dd7e700 1 -- 172.21.15.32:0/794521676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f5958066090 con 0x7f59580611b0 2020-12-05T17:35:26.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.075+0000 7f5955ffb700 1 -- 172.21.15.32:0/794521676 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v19) v1 ==== 184+0+0 (secure 0 0 0) 0x7f59480150e0 con 0x7f59580611b0 2020-12-05T17:35:26.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.075+0000 7f595dd7e700 1 -- 172.21.15.32:0/794521676 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5938055320 msgr2=0x7f59380577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:26.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.075+0000 7f595dd7e700 1 --2- 172.21.15.32:0/794521676 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5938055320 0x7f59380577e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f594c00a000 tx=0x7f594c008040).stop 2020-12-05T17:35:26.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.075+0000 7f595dd7e700 1 -- 172.21.15.32:0/794521676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59580611b0 msgr2=0x7f5958155530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:26.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.075+0000 7f595dd7e700 1 --2- 172.21.15.32:0/794521676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59580611b0 0x7f5958155530 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f59480032b0 tx=0x7f5948016000).stop 2020-12-05T17:35:26.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.075+0000 7f595dd7e700 1 -- 172.21.15.32:0/794521676 shutdown_connections 2020-12-05T17:35:26.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.075+0000 7f595dd7e700 1 --2- 172.21.15.32:0/794521676 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59580b0240 0x7f59580b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:26.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.075+0000 7f595dd7e700 1 --2- 172.21.15.32:0/794521676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59580611b0 0x7f5958155530 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:26.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.075+0000 7f595dd7e700 1 --2- 172.21.15.32:0/794521676 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5938055320 0x7f59380577e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:26.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.075+0000 7f595dd7e700 1 --2- 172.21.15.32:0/794521676 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f59580ae860 0x7f5958063b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:26.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.075+0000 7f595dd7e700 1 -- 172.21.15.32:0/794521676 >> 172.21.15.32:0/794521676 conn(0x7f59580b5f90 msgr2=0x7f595814f960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:26.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.075+0000 7f595dd7e700 1 -- 172.21.15.32:0/794521676 shutdown_connections 2020-12-05T17:35:26.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.075+0000 7f595dd7e700 1 -- 172.21.15.32:0/794521676 wait complete. 2020-12-05T17:35:26.078 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.2 2020-12-05T17:35:26.088 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:35:26.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.199+0000 7ff66b2e5700 1 Processor -- start 2020-12-05T17:35:26.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.199+0000 7ff66b2e5700 1 -- start start 2020-12-05T17:35:26.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.199+0000 7ff66b2e5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff664062400 0x7ff66405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:26.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.199+0000 7ff66b2e5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff664142450 con 0x7ff66405f3b0 2020-12-05T17:35:26.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.199+0000 7ff66b2e5700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff664061780 con 0x7ff66405feb0 2020-12-05T17:35:26.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.199+0000 7ff66b2e5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff664061a20 con 0x7ff664062400 2020-12-05T17:35:26.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.199+0000 7ff669ae2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff664062400 0x7ff66405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:26.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.199+0000 7ff669ae2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff664062400 0x7ff66405f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35378/0 (socket says 172.21.15.32:35378) 2020-12-05T17:35:26.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.199+0000 7ff66a2e3700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff66405f3b0 0x7ff6640b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38024/0 (socket says 172.21.15.32:38024) 2020-12-05T17:35:26.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.199+0000 7ff669ae2700 1 -- 172.21.15.32:0/2926916154 learned_addr learned my addr 172.21.15.32:0/2926916154 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:26.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.199+0000 7ff669ae2700 1 -- 172.21.15.32:0/2926916154 >> v1:172.21.15.61:6789/0 conn(0x7ff66405feb0 legacy=0x7ff6640b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:26.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.199+0000 7ff6692e1700 1 -- 172.21.15.32:0/2926916154 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1208612785 0 0) 0x7ff664142450 con 0x7ff66405f3b0 2020-12-05T17:35:26.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.199+0000 7ff669ae2700 1 -- 172.21.15.32:0/2926916154 >> v1:172.21.15.32:6789/0 conn(0x7ff66405f3b0 legacy=0x7ff6640b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:26.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.199+0000 7ff669ae2700 1 -- 172.21.15.32:0/2926916154 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff664061cc0 con 0x7ff664062400 2020-12-05T17:35:26.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.203+0000 7ff669ae2700 1 --2- 172.21.15.32:0/2926916154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff664062400 0x7ff66405f790 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7ff660002f30 tx=0x7ff660006e80).ready entity=mon.0 client_cookie=2b2c462422b161fd server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:26.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.203+0000 7ff6692e1700 1 -- 172.21.15.32:0/2926916154 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6600097f0 con 0x7ff664062400 2020-12-05T17:35:26.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.203+0000 7ff6692e1700 1 -- 172.21.15.32:0/2926916154 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff660009950 con 0x7ff664062400 2020-12-05T17:35:26.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.203+0000 7ff6692e1700 1 -- 172.21.15.32:0/2926916154 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff660009cb0 con 0x7ff664062400 2020-12-05T17:35:26.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.203+0000 7ff66b2e5700 1 -- 172.21.15.32:0/2926916154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff664062400 msgr2=0x7ff66405f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:26.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.203+0000 7ff66b2e5700 1 --2- 172.21.15.32:0/2926916154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff664062400 0x7ff66405f790 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7ff660002f30 tx=0x7ff660006e80).stop 2020-12-05T17:35:26.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.203+0000 7ff66b2e5700 1 -- 172.21.15.32:0/2926916154 shutdown_connections 2020-12-05T17:35:26.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.203+0000 7ff66b2e5700 1 --2- 172.21.15.32:0/2926916154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff664062400 0x7ff66405f790 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:26.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.203+0000 7ff66b2e5700 1 -- 172.21.15.32:0/2926916154 >> 172.21.15.32:0/2926916154 conn(0x7ff6640b8ee0 msgr2=0x7ff6640b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:26.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.203+0000 7ff66b2e5700 1 -- 172.21.15.32:0/2926916154 shutdown_connections 2020-12-05T17:35:26.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.203+0000 7ff66b2e5700 1 -- 172.21.15.32:0/2926916154 wait complete. 2020-12-05T17:35:26.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.203+0000 7ff66b2e5700 1 Processor -- start 2020-12-05T17:35:26.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.203+0000 7ff66b2e5700 1 -- start start 2020-12-05T17:35:26.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff66b2e5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff66405f3b0 0x7ff664145180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:26.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff66b2e5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff66405feb0 0x7ff664151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:26.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff66b2e5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff664062400 0x7ff6641549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:26.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff66b2e5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff664064e50 con 0x7ff66405feb0 2020-12-05T17:35:26.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff66b2e5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff664145e20 con 0x7ff66405f3b0 2020-12-05T17:35:26.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff66b2e5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6640a31e0 con 0x7ff664062400 2020-12-05T17:35:26.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff669ae2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff66405feb0 0x7ff664151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:26.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff669ae2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff66405feb0 0x7ff664151c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35384/0 (socket says 172.21.15.32:35384) 2020-12-05T17:35:26.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff66a2e3700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff66405f3b0 0x7ff664145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:26.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff669ae2700 1 -- 172.21.15.32:0/5461010 learned_addr learned my addr 172.21.15.32:0/5461010 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:26.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff66aae4700 1 --2- 172.21.15.32:0/5461010 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff664062400 0x7ff6641549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:26.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff669ae2700 1 -- 172.21.15.32:0/5461010 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff664062400 msgr2=0x7ff6641549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:26.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff669ae2700 1 --2- 172.21.15.32:0/5461010 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff664062400 0x7ff6641549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:26.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff669ae2700 1 -- 172.21.15.32:0/5461010 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff66405f3b0 msgr2=0x7ff664145180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:26.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff669ae2700 1 --2- 172.21.15.32:0/5461010 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff66405f3b0 0x7ff664145180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:26.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff669ae2700 1 -- 172.21.15.32:0/5461010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff664151520 con 0x7ff66405feb0 2020-12-05T17:35:26.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff66aae4700 1 --2- 172.21.15.32:0/5461010 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff664062400 0x7ff6641549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:26.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff669ae2700 1 --2- 172.21.15.32:0/5461010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff66405feb0 0x7ff664151c60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7ff660002f30 tx=0x7ff660015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:26.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff6537fe700 1 -- 172.21.15.32:0/5461010 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff660011d40 con 0x7ff66405feb0 2020-12-05T17:35:26.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff6537fe700 1 -- 172.21.15.32:0/5461010 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66001e460 con 0x7ff66405feb0 2020-12-05T17:35:26.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff6537fe700 1 -- 172.21.15.32:0/5461010 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff660011ea0 con 0x7ff66405feb0 2020-12-05T17:35:26.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff66b2e5700 1 -- 172.21.15.32:0/5461010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6641517c0 con 0x7ff66405feb0 2020-12-05T17:35:26.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff66b2e5700 1 -- 172.21.15.32:0/5461010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff664152150 con 0x7ff66405feb0 2020-12-05T17:35:26.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.207+0000 7ff6537fe700 1 -- 172.21.15.32:0/5461010 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff66001a040 con 0x7ff66405feb0 2020-12-05T17:35:26.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.211+0000 7ff66b2e5700 1 -- 172.21.15.32:0/5461010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff664151950 con 0x7ff66405feb0 2020-12-05T17:35:26.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.211+0000 7ff6537fe700 1 --2- 172.21.15.32:0/5461010 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff6440554e0 0x7ff6440579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:26.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.211+0000 7ff66a2e3700 1 --2- 172.21.15.32:0/5461010 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff6440554e0 0x7ff6440579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:26.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.211+0000 7ff6537fe700 1 -- 172.21.15.32:0/5461010 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff66006f190 con 0x7ff66405feb0 2020-12-05T17:35:26.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.211+0000 7ff66a2e3700 1 --2- 172.21.15.32:0/5461010 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff6440554e0 0x7ff6440579a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7ff65400d000 tx=0x7ff654007a40).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:26.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.215+0000 7ff6537fe700 1 -- 172.21.15.32:0/5461010 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff660045330 con 0x7ff66405feb0 2020-12-05T17:35:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.407+0000 7ff66b2e5700 1 -- 172.21.15.32:0/5461010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff664057050 con 0x7ff66405feb0 2020-12-05T17:35:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.407+0000 7ff6537fe700 1 -- 172.21.15.32:0/5461010 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2136 (secure 0 0 0) 0x7ff660023020 con 0x7ff66405feb0 2020-12-05T17:35:26.406 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:35:26.406 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5159,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/2123765277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2123765277},{"type":"v1","addr":"172.21.15.61:6833","nonce":2123765277}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:35:23.386407+0000","modified":"2020-12-05T17:35:24.389244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5640},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5640":{"gid":5640,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/681412338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":681412338},{"type":"v1","addr":"172.21.15.32:6835","nonce":681412338}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-12-05T17:35:26.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.407+0000 7ff66b2e5700 1 -- 172.21.15.32:0/5461010 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff6440554e0 msgr2=0x7ff6440579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.407+0000 7ff66b2e5700 1 --2- 172.21.15.32:0/5461010 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff6440554e0 0x7ff6440579a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7ff65400d000 tx=0x7ff654007a40).stop 2020-12-05T17:35:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.407+0000 7ff66b2e5700 1 -- 172.21.15.32:0/5461010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff66405feb0 msgr2=0x7ff664151c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.407+0000 7ff66b2e5700 1 --2- 172.21.15.32:0/5461010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff66405feb0 0x7ff664151c60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7ff660002f30 tx=0x7ff660015000).stop 2020-12-05T17:35:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.407+0000 7ff66b2e5700 1 -- 172.21.15.32:0/5461010 shutdown_connections 2020-12-05T17:35:26.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.407+0000 7ff66b2e5700 1 --2- 172.21.15.32:0/5461010 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff6440554e0 0x7ff6440579a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:26.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.407+0000 7ff66b2e5700 1 --2- 172.21.15.32:0/5461010 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff664062400 0x7ff6641549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:26.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.407+0000 7ff66b2e5700 1 --2- 172.21.15.32:0/5461010 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff66405f3b0 0x7ff664145180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:26.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.407+0000 7ff66b2e5700 1 --2- 172.21.15.32:0/5461010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff66405feb0 0x7ff664151c60 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:26.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.407+0000 7ff66b2e5700 1 -- 172.21.15.32:0/5461010 >> 172.21.15.32:0/5461010 conn(0x7ff6640b8ee0 msgr2=0x7ff6640b6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:26.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.407+0000 7ff66b2e5700 1 -- 172.21.15.32:0/5461010 shutdown_connections 2020-12-05T17:35:26.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.407+0000 7ff66b2e5700 1 -- 172.21.15.32:0/5461010 wait complete. 2020-12-05T17:35:26.410 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 23 2020-12-05T17:35:26.626 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:35:23.386407+0000', 'modified': '2020-12-05T17:35:24.389244+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5640}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5640': {'gid': 5640, 'name': 'a', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/681412338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 681412338}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 681412338}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:35:26.627 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:35:26.627 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:35:26.757 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":5640,"want_state":"up:active","state":"up:active","rank_uptime":3.3634511800000002,"mdsmap_epoch":23,"osdmap_epoch":54,"osdmap_epoch_barrier":52,"uptime":8.7657943169999992} 2020-12-05T17:35:26.758 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 5640, "mdsmap_epoch": 23, "osdmap_epoch": 54, "osdmap_epoch_barrier": 52, "rank_uptime": 3.36345118, "state": "up:active", "uptime": 8.765794317, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:35:26.759 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:35:26.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4b457700 1 Processor -- start 2020-12-05T17:35:26.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4b457700 1 -- start start 2020-12-05T17:35:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4b457700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1c440b1fc0 0x7f1c440af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4b457700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c440ab710 0x7f1c44061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4b457700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1c44147400 0x7f1c44145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4b457700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c440b0110 con 0x7f1c44147400 2020-12-05T17:35:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4b457700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c44146a90 con 0x7f1c440b1fc0 2020-12-05T17:35:26.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4b457700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c440b23a0 con 0x7f1c440ab710 2020-12-05T17:35:26.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4ac56700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1c44147400 0x7f1c44145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:26.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4ac56700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1c44147400 0x7f1c44145d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35392/0 (socket says 172.21.15.32:35392) 2020-12-05T17:35:26.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4ac56700 1 -- 172.21.15.32:0/2139556927 learned_addr learned my addr 172.21.15.32:0/2139556927 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:26.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c49c54700 1 --2- 172.21.15.32:0/2139556927 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c440ab710 0x7f1c44061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:26.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4a455700 1 --2- 172.21.15.32:0/2139556927 >> v2:172.21.15.61:3300/0 conn(0x7f1c440b1fc0 0x7f1c440af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:26.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4ac56700 1 -- 172.21.15.32:0/2139556927 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c440ab710 msgr2=0x7f1c44061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:26.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4ac56700 1 --2- 172.21.15.32:0/2139556927 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c440ab710 0x7f1c44061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:26.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4ac56700 1 -- 172.21.15.32:0/2139556927 >> v2:172.21.15.61:3300/0 conn(0x7f1c440b1fc0 msgr2=0x7f1c440af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:26.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4ac56700 1 --2- 172.21.15.32:0/2139556927 >> v2:172.21.15.61:3300/0 conn(0x7f1c440b1fc0 0x7f1c440af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:26.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4ac56700 1 -- 172.21.15.32:0/2139556927 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c44146cc0 con 0x7f1c44147400 2020-12-05T17:35:26.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4ac56700 1 --2- 172.21.15.32:0/2139556927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c44147400 0x7f1c44145d10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f1c3c0032b0 tx=0x7f1c3c009270).ready entity=mon.0 client_cookie=c5df484ff48f3295 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:26.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c49453700 1 -- 172.21.15.32:0/2139556927 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c3c00bc00 con 0x7f1c44147400 2020-12-05T17:35:26.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c49453700 1 -- 172.21.15.32:0/2139556927 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3c00bd60 con 0x7f1c44147400 2020-12-05T17:35:26.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c49453700 1 -- 172.21.15.32:0/2139556927 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c3c011400 con 0x7f1c44147400 2020-12-05T17:35:26.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4b457700 1 -- 172.21.15.32:0/2139556927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c44147400 msgr2=0x7f1c44145d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:26.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4b457700 1 --2- 172.21.15.32:0/2139556927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c44147400 0x7f1c44145d10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f1c3c0032b0 tx=0x7f1c3c009270).stop 2020-12-05T17:35:26.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4b457700 1 -- 172.21.15.32:0/2139556927 shutdown_connections 2020-12-05T17:35:26.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4b457700 1 --2- 172.21.15.32:0/2139556927 >> v2:172.21.15.61:3300/0 conn(0x7f1c440b1fc0 0x7f1c440af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:26.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4b457700 1 --2- 172.21.15.32:0/2139556927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c44147400 0x7f1c44145d10 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:26.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4b457700 1 --2- 172.21.15.32:0/2139556927 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c440ab710 0x7f1c44061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:26.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4b457700 1 -- 172.21.15.32:0/2139556927 >> 172.21.15.32:0/2139556927 conn(0x7f1c440b5530 msgr2=0x7f1c440b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:26.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4b457700 1 -- 172.21.15.32:0/2139556927 shutdown_connections 2020-12-05T17:35:26.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.879+0000 7f1c4b457700 1 -- 172.21.15.32:0/2139556927 wait complete. 2020-12-05T17:35:26.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.883+0000 7f1c4b457700 1 Processor -- start 2020-12-05T17:35:26.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.883+0000 7f1c4b457700 1 -- start start 2020-12-05T17:35:26.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c4b457700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c440ab710 0x7f1c441432d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:26.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c4b457700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c440b1fc0 0x7f1c440abe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:26.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c4b457700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c44147400 0x7f1c440ae490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:26.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c4b457700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c441495c0 con 0x7f1c44147400 2020-12-05T17:35:26.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c4b457700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c44149830 con 0x7f1c440b1fc0 2020-12-05T17:35:26.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c4b457700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c44149a60 con 0x7f1c440ab710 2020-12-05T17:35:26.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c4ac56700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c44147400 0x7f1c440ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:26.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c4ac56700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c44147400 0x7f1c440ae490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35400/0 (socket says 172.21.15.32:35400) 2020-12-05T17:35:26.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c4ac56700 1 -- 172.21.15.32:0/1437180977 learned_addr learned my addr 172.21.15.32:0/1437180977 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:26.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c49c54700 1 --2- 172.21.15.32:0/1437180977 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c440b1fc0 0x7f1c440abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:26.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c4a455700 1 --2- 172.21.15.32:0/1437180977 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c440ab710 0x7f1c441432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:26.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c4ac56700 1 -- 172.21.15.32:0/1437180977 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c440ab710 msgr2=0x7f1c441432d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:26.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c4ac56700 1 --2- 172.21.15.32:0/1437180977 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c440ab710 0x7f1c441432d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:26.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c4ac56700 1 -- 172.21.15.32:0/1437180977 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c440b1fc0 msgr2=0x7f1c440abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:26.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c4ac56700 1 --2- 172.21.15.32:0/1437180977 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c440b1fc0 0x7f1c440abe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:26.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c4ac56700 1 -- 172.21.15.32:0/1437180977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4414aa50 con 0x7f1c44147400 2020-12-05T17:35:26.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c4ac56700 1 --2- 172.21.15.32:0/1437180977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c44147400 0x7f1c440ae490 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f1c44149700 tx=0x7f1c3c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:26.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c337fe700 1 -- 172.21.15.32:0/1437180977 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c3c011e90 con 0x7f1c44147400 2020-12-05T17:35:26.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c337fe700 1 -- 172.21.15.32:0/1437180977 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3c01f3f0 con 0x7f1c44147400 2020-12-05T17:35:26.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c4b457700 1 -- 172.21.15.32:0/1437180977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c440abb30 con 0x7f1c44147400 2020-12-05T17:35:26.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c337fe700 1 -- 172.21.15.32:0/1437180977 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c3c01f750 con 0x7f1c44147400 2020-12-05T17:35:26.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c4b457700 1 -- 172.21.15.32:0/1437180977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c44158b50 con 0x7f1c44147400 2020-12-05T17:35:26.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c4b457700 1 -- 172.21.15.32:0/1437180977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c4414a770 con 0x7f1c44147400 2020-12-05T17:35:26.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.887+0000 7f1c337fe700 1 -- 172.21.15.32:0/1437180977 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1c3c026030 con 0x7f1c44147400 2020-12-05T17:35:26.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.891+0000 7f1c337fe700 1 --2- 172.21.15.32:0/1437180977 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1c240553b0 0x7f1c24057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:26.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.891+0000 7f1c4a455700 1 --2- 172.21.15.32:0/1437180977 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1c240553b0 0x7f1c24057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:26.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.891+0000 7f1c337fe700 1 -- 172.21.15.32:0/1437180977 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1c3c06eb30 con 0x7f1c44147400 2020-12-05T17:35:26.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.891+0000 7f1c4a455700 1 --2- 172.21.15.32:0/1437180977 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1c240553b0 0x7f1c24057870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f1c4000c000 tx=0x7f1c40008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:26.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:26.895+0000 7f1c337fe700 1 -- 172.21.15.32:0/1437180977 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1c3c048700 con 0x7f1c44147400 2020-12-05T17:35:27.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.043+0000 7f1c4b457700 1 -- 172.21.15.32:0/1437180977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c4414abe0 con 0x7f1c44147400 2020-12-05T17:35:27.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.043+0000 7f1c337fe700 1 -- 172.21.15.32:0/1437180977 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16273 (secure 0 0 0) 0x7f1c3c024020 con 0x7f1c44147400 2020-12-05T17:35:27.043 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:35:27.043 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":54,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:35:25.399857+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:35:20.867930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-12-05T17:35:22.139660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:35:27.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.043+0000 7f1c4b457700 1 -- 172.21.15.32:0/1437180977 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1c240553b0 msgr2=0x7f1c24057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:27.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.043+0000 7f1c4b457700 1 --2- 172.21.15.32:0/1437180977 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1c240553b0 0x7f1c24057870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f1c4000c000 tx=0x7f1c40008040).stop 2020-12-05T17:35:27.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.047+0000 7f1c4b457700 1 -- 172.21.15.32:0/1437180977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c44147400 msgr2=0x7f1c440ae490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:27.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.047+0000 7f1c4b457700 1 --2- 172.21.15.32:0/1437180977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c44147400 0x7f1c440ae490 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f1c44149700 tx=0x7f1c3c016000).stop 2020-12-05T17:35:27.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.047+0000 7f1c4b457700 1 -- 172.21.15.32:0/1437180977 shutdown_connections 2020-12-05T17:35:27.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.047+0000 7f1c4b457700 1 --2- 172.21.15.32:0/1437180977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c44147400 0x7f1c440ae490 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.047+0000 7f1c4b457700 1 --2- 172.21.15.32:0/1437180977 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1c240553b0 0x7f1c24057870 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.047+0000 7f1c4b457700 1 --2- 172.21.15.32:0/1437180977 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c440ab710 0x7f1c441432d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.047+0000 7f1c4b457700 1 --2- 172.21.15.32:0/1437180977 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c440b1fc0 0x7f1c440abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.047+0000 7f1c4b457700 1 -- 172.21.15.32:0/1437180977 >> 172.21.15.32:0/1437180977 conn(0x7f1c440b5530 msgr2=0x7f1c441449a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:27.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.047+0000 7f1c4b457700 1 -- 172.21.15.32:0/1437180977 shutdown_connections 2020-12-05T17:35:27.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.047+0000 7f1c4b457700 1 -- 172.21.15.32:0/1437180977 wait complete. 2020-12-05T17:35:27.057 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:35:20.867930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:35:22.139660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:35:27.058 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:35:27.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.175+0000 7f0c00eba700 1 Processor -- start 2020-12-05T17:35:27.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.175+0000 7f0c00eba700 1 -- start start 2020-12-05T17:35:27.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.175+0000 7f0c00eba700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0bfc05feb0 0x7f0bfc05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:27.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.175+0000 7f0c00eba700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bfc0b2df0 con 0x7f0bfc05f3b0 2020-12-05T17:35:27.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.175+0000 7f0c00eba700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bfc0b3c90 con 0x7f0bfc0b2fb0 2020-12-05T17:35:27.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.175+0000 7f0c00eba700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bfc0b2b50 con 0x7f0bfc05feb0 2020-12-05T17:35:27.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.175+0000 7f0bfb7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0bfc05f3b0 0x7f0bfc13c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38052/0 (socket says 172.21.15.32:38052) 2020-12-05T17:35:27.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.175+0000 7f0bfb7fe700 1 -- 172.21.15.32:0/330517132 learned_addr learned my addr 172.21.15.32:0/330517132 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:35:27.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.175+0000 7f0bfaffd700 1 --2- 172.21.15.32:0/330517132 >> v2:172.21.15.32:3300/0 conn(0x7f0bfc05feb0 0x7f0bfc05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:27.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.175+0000 7f0bfa7fc700 1 -- 172.21.15.32:0/330517132 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198627977 0 0) 0x7f0bfc0b2df0 con 0x7f0bfc05f3b0 2020-12-05T17:35:27.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0bfa7fc700 1 -- 172.21.15.32:0/330517132 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0be8003230 con 0x7f0bfc05f3b0 2020-12-05T17:35:27.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0bfaffd700 1 -- 172.21.15.32:0/330517132 >> v1:172.21.15.61:6789/0 conn(0x7f0bfc0b2fb0 legacy=0x7f0bfc142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:27.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0bfaffd700 1 -- 172.21.15.32:0/330517132 >> v1:172.21.15.32:6789/0 conn(0x7f0bfc05f3b0 legacy=0x7f0bfc13c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:27.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0bfaffd700 1 -- 172.21.15.32:0/330517132 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bfc142820 con 0x7f0bfc05feb0 2020-12-05T17:35:27.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0bfaffd700 1 --2- 172.21.15.32:0/330517132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bfc05feb0 0x7f0bfc05f790 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f0bf0002f30 tx=0x7f0bf0006dc0).ready entity=mon.0 client_cookie=2041f44893125ad8 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:27.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0bfa7fc700 1 -- 172.21.15.32:0/330517132 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bf0009800 con 0x7f0bfc05feb0 2020-12-05T17:35:27.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0bfa7fc700 1 -- 172.21.15.32:0/330517132 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf0009960 con 0x7f0bfc05feb0 2020-12-05T17:35:27.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0bfa7fc700 1 -- 172.21.15.32:0/330517132 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bf0009cc0 con 0x7f0bfc05feb0 2020-12-05T17:35:27.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0c00eba700 1 -- 172.21.15.32:0/330517132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bfc05feb0 msgr2=0x7f0bfc05f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:27.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0c00eba700 1 --2- 172.21.15.32:0/330517132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bfc05feb0 0x7f0bfc05f790 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f0bf0002f30 tx=0x7f0bf0006dc0).stop 2020-12-05T17:35:27.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0c00eba700 1 -- 172.21.15.32:0/330517132 shutdown_connections 2020-12-05T17:35:27.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0c00eba700 1 --2- 172.21.15.32:0/330517132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bfc05feb0 0x7f0bfc05f790 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0c00eba700 1 -- 172.21.15.32:0/330517132 >> 172.21.15.32:0/330517132 conn(0x7f0bfc0b8f20 msgr2=0x7f0bfc0b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:27.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0c00eba700 1 -- 172.21.15.32:0/330517132 shutdown_connections 2020-12-05T17:35:27.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0c00eba700 1 -- 172.21.15.32:0/330517132 wait complete. 2020-12-05T17:35:27.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0c00eba700 1 Processor -- start 2020-12-05T17:35:27.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0c00eba700 1 -- start start 2020-12-05T17:35:27.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0c00eba700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0bfc05f3b0 0x7f0bfc13d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:27.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0c00eba700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0bfc05feb0 0x7f0bfc0b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:27.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0c00eba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bfc0b2fb0 0x7f0bfc0b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:27.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0c00eba700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc148910 con 0x7f0bfc0b2fb0 2020-12-05T17:35:27.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0c00eba700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc146210 con 0x7f0bfc05feb0 2020-12-05T17:35:27.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0c00eba700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc13f3c0 con 0x7f0bfc05f3b0 2020-12-05T17:35:27.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0bfbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bfc0b2fb0 0x7f0bfc0b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:27.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0bfbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bfc0b2fb0 0x7f0bfc0b6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35414/0 (socket says 172.21.15.32:35414) 2020-12-05T17:35:27.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.179+0000 7f0bfbfff700 1 -- 172.21.15.32:0/1035530358 learned_addr learned my addr 172.21.15.32:0/1035530358 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:27.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.183+0000 7f0bfaffd700 1 --2- 172.21.15.32:0/1035530358 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0bfc05feb0 0x7f0bfc0b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:27.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.183+0000 7f0bfb7fe700 1 --2- 172.21.15.32:0/1035530358 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0bfc05f3b0 0x7f0bfc13d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:27.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.183+0000 7f0bfbfff700 1 -- 172.21.15.32:0/1035530358 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0bfc05f3b0 msgr2=0x7f0bfc13d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.183+0000 7f0bfbfff700 1 --2- 172.21.15.32:0/1035530358 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0bfc05f3b0 0x7f0bfc13d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.183+0000 7f0bfbfff700 1 -- 172.21.15.32:0/1035530358 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0bfc05feb0 msgr2=0x7f0bfc0b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.183+0000 7f0bfbfff700 1 --2- 172.21.15.32:0/1035530358 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0bfc05feb0 0x7f0bfc0b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.183+0000 7f0bfbfff700 1 -- 172.21.15.32:0/1035530358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bfc13f660 con 0x7f0bfc0b2fb0 2020-12-05T17:35:27.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.183+0000 7f0bfbfff700 1 --2- 172.21.15.32:0/1035530358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bfc0b2fb0 0x7f0bfc0b6380 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f0bfc0b1e60 tx=0x7f0bf4007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:27.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.183+0000 7f0bf8ff9700 1 -- 172.21.15.32:0/1035530358 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bf400a5a0 con 0x7f0bfc0b2fb0 2020-12-05T17:35:27.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.183+0000 7f0bf8ff9700 1 -- 172.21.15.32:0/1035530358 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf400a700 con 0x7f0bfc0b2fb0 2020-12-05T17:35:27.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.183+0000 7f0c00eba700 1 -- 172.21.15.32:0/1035530358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bfc154a80 con 0x7f0bfc0b2fb0 2020-12-05T17:35:27.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.183+0000 7f0bf8ff9700 1 -- 172.21.15.32:0/1035530358 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bf400aa60 con 0x7f0bfc0b2fb0 2020-12-05T17:35:27.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.183+0000 7f0c00eba700 1 -- 172.21.15.32:0/1035530358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bfc146c10 con 0x7f0bfc0b2fb0 2020-12-05T17:35:27.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.183+0000 7f0c00eba700 1 -- 172.21.15.32:0/1035530358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bfc148710 con 0x7f0bfc0b2fb0 2020-12-05T17:35:27.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.183+0000 7f0bf8ff9700 1 -- 172.21.15.32:0/1035530358 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0bf4019030 con 0x7f0bfc0b2fb0 2020-12-05T17:35:27.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.183+0000 7f0bf8ff9700 1 --2- 172.21.15.32:0/1035530358 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0bdc055330 0x7f0bdc0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:27.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.183+0000 7f0bfb7fe700 1 --2- 172.21.15.32:0/1035530358 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0bdc055330 0x7f0bdc0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:27.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.183+0000 7f0bf8ff9700 1 -- 172.21.15.32:0/1035530358 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0bf406d7d0 con 0x7f0bfc0b2fb0 2020-12-05T17:35:27.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.183+0000 7f0bfb7fe700 1 --2- 172.21.15.32:0/1035530358 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0bdc055330 0x7f0bdc0577f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f0bec00d000 tx=0x7f0bec009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:27.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.191+0000 7f0bf8ff9700 1 -- 172.21.15.32:0/1035530358 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0bf4043970 con 0x7f0bfc0b2fb0 2020-12-05T17:35:27.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.335+0000 7f0c00eba700 1 -- 172.21.15.32:0/1035530358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0bfc057050 con 0x7f0bfc0b2fb0 2020-12-05T17:35:27.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.335+0000 7f0bf8ff9700 1 -- 172.21.15.32:0/1035530358 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v54) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0bf4046f90 con 0x7f0bfc0b2fb0 2020-12-05T17:35:27.335 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:35:27.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.339+0000 7f0c00eba700 1 -- 172.21.15.32:0/1035530358 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0bdc055330 msgr2=0x7f0bdc0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:27.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.339+0000 7f0c00eba700 1 --2- 172.21.15.32:0/1035530358 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0bdc055330 0x7f0bdc0577f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f0bec00d000 tx=0x7f0bec009040).stop 2020-12-05T17:35:27.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.339+0000 7f0c00eba700 1 -- 172.21.15.32:0/1035530358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bfc0b2fb0 msgr2=0x7f0bfc0b6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:27.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.339+0000 7f0c00eba700 1 --2- 172.21.15.32:0/1035530358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bfc0b2fb0 0x7f0bfc0b6380 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f0bfc0b1e60 tx=0x7f0bf4007f70).stop 2020-12-05T17:35:27.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.339+0000 7f0c00eba700 1 -- 172.21.15.32:0/1035530358 shutdown_connections 2020-12-05T17:35:27.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.339+0000 7f0c00eba700 1 --2- 172.21.15.32:0/1035530358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bfc0b2fb0 0x7f0bfc0b6380 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.339+0000 7f0c00eba700 1 --2- 172.21.15.32:0/1035530358 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0bdc055330 0x7f0bdc0577f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.339+0000 7f0c00eba700 1 --2- 172.21.15.32:0/1035530358 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0bfc05f3b0 0x7f0bfc13d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.339+0000 7f0c00eba700 1 --2- 172.21.15.32:0/1035530358 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0bfc05feb0 0x7f0bfc0b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.339+0000 7f0c00eba700 1 -- 172.21.15.32:0/1035530358 >> 172.21.15.32:0/1035530358 conn(0x7f0bfc0b8f20 msgr2=0x7f0bfc0b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:27.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.339+0000 7f0c00eba700 1 -- 172.21.15.32:0/1035530358 shutdown_connections 2020-12-05T17:35:27.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.339+0000 7f0c00eba700 1 -- 172.21.15.32:0/1035530358 wait complete. 2020-12-05T17:35:27.354 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:35:27.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.471+0000 7fdc73c90700 1 Processor -- start 2020-12-05T17:35:27.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.471+0000 7fdc73c90700 1 -- start start 2020-12-05T17:35:27.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.471+0000 7fdc73c90700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc6c0b6690 0x7fdc6c0b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:27.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.471+0000 7fdc73c90700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc6c0b56f0 con 0x7fdc6c0b5240 2020-12-05T17:35:27.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.471+0000 7fdc73c90700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc6c0b1920 con 0x7fdc6c0b5b20 2020-12-05T17:35:27.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.471+0000 7fdc73c90700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc6c0b1bc0 con 0x7fdc6c0b6690 2020-12-05T17:35:27.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.471+0000 7fdc7122b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc6c0b6690 0x7fdc6c0b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:27.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.471+0000 7fdc7222d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdc6c0b5240 0x7fdc6c145f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38070/0 (socket says 172.21.15.32:38070) 2020-12-05T17:35:27.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.471+0000 7fdc7222d700 1 -- 172.21.15.32:0/1901385319 learned_addr learned my addr 172.21.15.32:0/1901385319 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:35:27.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.471+0000 7fdc70a2a700 1 -- 172.21.15.32:0/1901385319 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 369251622 0 0) 0x7fdc6c0b56f0 con 0x7fdc6c0b5240 2020-12-05T17:35:27.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.471+0000 7fdc70a2a700 1 -- 172.21.15.32:0/1901385319 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc54003230 con 0x7fdc6c0b5240 2020-12-05T17:35:27.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.471+0000 7fdc7122b700 1 -- 172.21.15.32:0/1901385319 >> v1:172.21.15.61:6789/0 conn(0x7fdc6c0b5b20 legacy=0x7fdc6c0b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:27.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc7122b700 1 -- 172.21.15.32:0/1901385319 >> v1:172.21.15.32:6789/0 conn(0x7fdc6c0b5240 legacy=0x7fdc6c145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:27.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc7122b700 1 -- 172.21.15.32:0/1901385319 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c146620 con 0x7fdc6c0b6690 2020-12-05T17:35:27.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc7122b700 1 --2- 172.21.15.32:0/1901385319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6c0b6690 0x7fdc6c0b5f00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7fdc68002f30 tx=0x7fdc68006dc0).ready entity=mon.0 client_cookie=86d87fe8c40b4891 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:27.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc70a2a700 1 -- 172.21.15.32:0/1901385319 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc68009800 con 0x7fdc6c0b6690 2020-12-05T17:35:27.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc70a2a700 1 -- 172.21.15.32:0/1901385319 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc68009960 con 0x7fdc6c0b6690 2020-12-05T17:35:27.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc70a2a700 1 -- 172.21.15.32:0/1901385319 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc68009cc0 con 0x7fdc6c0b6690 2020-12-05T17:35:27.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc73c90700 1 -- 172.21.15.32:0/1901385319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6c0b6690 msgr2=0x7fdc6c0b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:27.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc73c90700 1 --2- 172.21.15.32:0/1901385319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6c0b6690 0x7fdc6c0b5f00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7fdc68002f30 tx=0x7fdc68006dc0).stop 2020-12-05T17:35:27.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc73c90700 1 -- 172.21.15.32:0/1901385319 shutdown_connections 2020-12-05T17:35:27.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc73c90700 1 --2- 172.21.15.32:0/1901385319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6c0b6690 0x7fdc6c0b5f00 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc73c90700 1 -- 172.21.15.32:0/1901385319 >> 172.21.15.32:0/1901385319 conn(0x7fdc6c05f3b0 msgr2=0x7fdc6c05f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:27.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc73c90700 1 -- 172.21.15.32:0/1901385319 shutdown_connections 2020-12-05T17:35:27.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc73c90700 1 -- 172.21.15.32:0/1901385319 wait complete. 2020-12-05T17:35:27.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc73c90700 1 Processor -- start 2020-12-05T17:35:27.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc73c90700 1 -- start start 2020-12-05T17:35:27.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc73c90700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6c0b5240 0x7fdc6c0a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:27.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc73c90700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc6c0b5b20 0x7fdc6c0aea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:27.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc73c90700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc6c0b6690 0x7fdc6c0aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:27.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc73c90700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0a3dd0 con 0x7fdc6c0b5240 2020-12-05T17:35:27.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc73c90700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0a6390 con 0x7fdc6c0b5b20 2020-12-05T17:35:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc73c90700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0a60d0 con 0x7fdc6c0b6690 2020-12-05T17:35:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc71a2c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6c0b5240 0x7fdc6c0a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc71a2c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6c0b5240 0x7fdc6c0a38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35424/0 (socket says 172.21.15.32:35424) 2020-12-05T17:35:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc71a2c700 1 -- 172.21.15.32:0/99362145 learned_addr learned my addr 172.21.15.32:0/99362145 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:27.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc7222d700 1 --2- 172.21.15.32:0/99362145 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc6c0b6690 0x7fdc6c0aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:27.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc7122b700 1 --2- 172.21.15.32:0/99362145 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc6c0b5b20 0x7fdc6c0aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:27.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc71a2c700 1 -- 172.21.15.32:0/99362145 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc6c0b6690 msgr2=0x7fdc6c0aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:27.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc71a2c700 1 --2- 172.21.15.32:0/99362145 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc6c0b6690 0x7fdc6c0aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc71a2c700 1 -- 172.21.15.32:0/99362145 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc6c0b5b20 msgr2=0x7fdc6c0aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:27.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc71a2c700 1 --2- 172.21.15.32:0/99362145 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc6c0b5b20 0x7fdc6c0aea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc71a2c700 1 -- 172.21.15.32:0/99362145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c0a26f0 con 0x7fdc6c0b5240 2020-12-05T17:35:27.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc7122b700 1 --2- 172.21.15.32:0/99362145 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc6c0b5b20 0x7fdc6c0aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:35:27.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.475+0000 7fdc71a2c700 1 --2- 172.21.15.32:0/99362145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6c0b5240 0x7fdc6c0a38e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7fdc6c0b2770 tx=0x7fdc60007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:27.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.479+0000 7fdc5effd700 1 -- 172.21.15.32:0/99362145 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc6000e4a0 con 0x7fdc6c0b5240 2020-12-05T17:35:27.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.479+0000 7fdc5effd700 1 -- 172.21.15.32:0/99362145 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6000e600 con 0x7fdc6c0b5240 2020-12-05T17:35:27.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.479+0000 7fdc73c90700 1 -- 172.21.15.32:0/99362145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc6c0af3f0 con 0x7fdc6c0b5240 2020-12-05T17:35:27.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.479+0000 7fdc5effd700 1 -- 172.21.15.32:0/99362145 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc6000e960 con 0x7fdc6c0b5240 2020-12-05T17:35:27.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.479+0000 7fdc73c90700 1 -- 172.21.15.32:0/99362145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6c0a43b0 con 0x7fdc6c0b5240 2020-12-05T17:35:27.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.479+0000 7fdc73c90700 1 -- 172.21.15.32:0/99362145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc6c057050 con 0x7fdc6c0b5240 2020-12-05T17:35:27.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.479+0000 7fdc5effd700 1 -- 172.21.15.32:0/99362145 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fdc60019030 con 0x7fdc6c0b5240 2020-12-05T17:35:27.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.479+0000 7fdc5effd700 1 --2- 172.21.15.32:0/99362145 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdc4c0553b0 0x7fdc4c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:27.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.479+0000 7fdc5effd700 1 -- 172.21.15.32:0/99362145 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdc6006d7d0 con 0x7fdc6c0b5240 2020-12-05T17:35:27.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.479+0000 7fdc7122b700 1 --2- 172.21.15.32:0/99362145 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdc4c0553b0 0x7fdc4c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:27.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.479+0000 7fdc7122b700 1 --2- 172.21.15.32:0/99362145 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdc4c0553b0 0x7fdc4c057870 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fdc68007000 tx=0x7fdc68015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:27.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.487+0000 7fdc5effd700 1 -- 172.21.15.32:0/99362145 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fdc600438d0 con 0x7fdc6c0b5240 2020-12-05T17:35:27.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.631+0000 7fdc73c90700 1 -- 172.21.15.32:0/99362145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdc6c146400 con 0x7fdc6c0b5240 2020-12-05T17:35:27.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.631+0000 7fdc5effd700 1 -- 172.21.15.32:0/99362145 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v54) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdc60046ef0 con 0x7fdc6c0b5240 2020-12-05T17:35:27.631 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:35:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.631+0000 7fdc73c90700 1 -- 172.21.15.32:0/99362145 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdc4c0553b0 msgr2=0x7fdc4c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.631+0000 7fdc73c90700 1 --2- 172.21.15.32:0/99362145 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdc4c0553b0 0x7fdc4c057870 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fdc68007000 tx=0x7fdc68015000).stop 2020-12-05T17:35:27.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.631+0000 7fdc73c90700 1 -- 172.21.15.32:0/99362145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6c0b5240 msgr2=0x7fdc6c0a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:27.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.631+0000 7fdc73c90700 1 --2- 172.21.15.32:0/99362145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6c0b5240 0x7fdc6c0a38e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7fdc6c0b2770 tx=0x7fdc60007c20).stop 2020-12-05T17:35:27.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.635+0000 7fdc73c90700 1 -- 172.21.15.32:0/99362145 shutdown_connections 2020-12-05T17:35:27.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.635+0000 7fdc73c90700 1 --2- 172.21.15.32:0/99362145 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdc4c0553b0 0x7fdc4c057870 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.635+0000 7fdc73c90700 1 --2- 172.21.15.32:0/99362145 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc6c0b6690 0x7fdc6c0aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.635+0000 7fdc73c90700 1 --2- 172.21.15.32:0/99362145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6c0b5240 0x7fdc6c0a38e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.635+0000 7fdc73c90700 1 --2- 172.21.15.32:0/99362145 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc6c0b5b20 0x7fdc6c0aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.635+0000 7fdc73c90700 1 -- 172.21.15.32:0/99362145 >> 172.21.15.32:0/99362145 conn(0x7fdc6c05f3b0 msgr2=0x7fdc6c0b6f20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:27.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.635+0000 7fdc73c90700 1 -- 172.21.15.32:0/99362145 shutdown_connections 2020-12-05T17:35:27.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.635+0000 7fdc73c90700 1 -- 172.21.15.32:0/99362145 wait complete. 2020-12-05T17:35:27.644 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:35:27.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.767+0000 7f6a820e9700 1 Processor -- start 2020-12-05T17:35:27.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.767+0000 7f6a820e9700 1 -- start start 2020-12-05T17:35:27.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.767+0000 7f6a820e9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6a7c05f3b0 0x7f6a7c05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:27.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.767+0000 7f6a820e9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7c0b2df0 con 0x7f6a7c0b2fb0 2020-12-05T17:35:27.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.767+0000 7f6a820e9700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7c0b3c90 con 0x7f6a7c05feb0 2020-12-05T17:35:27.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.767+0000 7f6a820e9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a7c0b2b50 con 0x7f6a7c05f3b0 2020-12-05T17:35:27.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.767+0000 7f6a810e7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6a7c05f3b0 0x7f6a7c05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:27.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.767+0000 7f6a818e8700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6a7c0b2fb0 0x7f6a7c142cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38084/0 (socket says 172.21.15.32:38084) 2020-12-05T17:35:27.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.767+0000 7f6a818e8700 1 -- 172.21.15.32:0/545109167 learned_addr learned my addr 172.21.15.32:0/545109167 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:35:27.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.767+0000 7f6a6ffff700 1 -- 172.21.15.32:0/545109167 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1907889338 0 0) 0x7f6a7c0b2df0 con 0x7f6a7c0b2fb0 2020-12-05T17:35:27.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.767+0000 7f6a6ffff700 1 -- 172.21.15.32:0/545109167 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a60003230 con 0x7f6a7c0b2fb0 2020-12-05T17:35:27.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.767+0000 7f6a810e7700 1 -- 172.21.15.32:0/545109167 >> v1:172.21.15.61:6789/0 conn(0x7f6a7c05feb0 legacy=0x7f6a7c13c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:27.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.767+0000 7f6a810e7700 1 -- 172.21.15.32:0/545109167 >> v1:172.21.15.32:6789/0 conn(0x7f6a7c0b2fb0 legacy=0x7f6a7c142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.767+0000 7f6a810e7700 1 -- 172.21.15.32:0/545109167 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c142820 con 0x7f6a7c05f3b0 2020-12-05T17:35:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.767+0000 7f6a810e7700 1 --2- 172.21.15.32:0/545109167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a7c05f3b0 0x7f6a7c05f790 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f6a78002f30 tx=0x7f6a78006dc0).ready entity=mon.0 client_cookie=7f564b0ff75cad7c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:27.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.767+0000 7f6a6ffff700 1 -- 172.21.15.32:0/545109167 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a78009800 con 0x7f6a7c05f3b0 2020-12-05T17:35:27.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.771+0000 7f6a6ffff700 1 -- 172.21.15.32:0/545109167 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a78009960 con 0x7f6a7c05f3b0 2020-12-05T17:35:27.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.771+0000 7f6a6ffff700 1 -- 172.21.15.32:0/545109167 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a78009cc0 con 0x7f6a7c05f3b0 2020-12-05T17:35:27.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.771+0000 7f6a820e9700 1 -- 172.21.15.32:0/545109167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a7c05f3b0 msgr2=0x7f6a7c05f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:27.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.771+0000 7f6a820e9700 1 --2- 172.21.15.32:0/545109167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a7c05f3b0 0x7f6a7c05f790 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f6a78002f30 tx=0x7f6a78006dc0).stop 2020-12-05T17:35:27.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.771+0000 7f6a820e9700 1 -- 172.21.15.32:0/545109167 shutdown_connections 2020-12-05T17:35:27.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.771+0000 7f6a820e9700 1 --2- 172.21.15.32:0/545109167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a7c05f3b0 0x7f6a7c05f790 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.771+0000 7f6a820e9700 1 -- 172.21.15.32:0/545109167 >> 172.21.15.32:0/545109167 conn(0x7f6a7c0b8f20 msgr2=0x7f6a7c0b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:27.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.771+0000 7f6a820e9700 1 -- 172.21.15.32:0/545109167 shutdown_connections 2020-12-05T17:35:27.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.771+0000 7f6a820e9700 1 -- 172.21.15.32:0/545109167 wait complete. 2020-12-05T17:35:27.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.771+0000 7f6a820e9700 1 Processor -- start 2020-12-05T17:35:27.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.771+0000 7f6a820e9700 1 -- start start 2020-12-05T17:35:27.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.771+0000 7f6a820e9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a7c05f3b0 0x7f6a7c066680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:27.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.771+0000 7f6a820e9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a7c05feb0 0x7f6a7c145e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:27.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.771+0000 7f6a820e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a7c0b2fb0 0x7f6a7c141ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:27.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.771+0000 7f6a820e9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c065130 con 0x7f6a7c0b2fb0 2020-12-05T17:35:27.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.771+0000 7f6a820e9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c064bf0 con 0x7f6a7c05f3b0 2020-12-05T17:35:27.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.771+0000 7f6a820e9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c064e90 con 0x7f6a7c05feb0 2020-12-05T17:35:27.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.771+0000 7f6a818e8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a7c0b2fb0 0x7f6a7c141ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:27.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.775+0000 7f6a818e8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a7c0b2fb0 0x7f6a7c141ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35440/0 (socket says 172.21.15.32:35440) 2020-12-05T17:35:27.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.775+0000 7f6a818e8700 1 -- 172.21.15.32:0/3115660833 learned_addr learned my addr 172.21.15.32:0/3115660833 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:27.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.775+0000 7f6a810e7700 1 --2- 172.21.15.32:0/3115660833 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a7c05f3b0 0x7f6a7c066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:27.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.775+0000 7f6a808e6700 1 --2- 172.21.15.32:0/3115660833 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a7c05feb0 0x7f6a7c145e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:27.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.775+0000 7f6a818e8700 1 -- 172.21.15.32:0/3115660833 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a7c05feb0 msgr2=0x7f6a7c145e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:27.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.775+0000 7f6a818e8700 1 --2- 172.21.15.32:0/3115660833 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a7c05feb0 0x7f6a7c145e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.775+0000 7f6a818e8700 1 -- 172.21.15.32:0/3115660833 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a7c05f3b0 msgr2=0x7f6a7c066680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:27.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.775+0000 7f6a818e8700 1 --2- 172.21.15.32:0/3115660833 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a7c05f3b0 0x7f6a7c066680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.775+0000 7f6a818e8700 1 -- 172.21.15.32:0/3115660833 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c146310 con 0x7f6a7c0b2fb0 2020-12-05T17:35:27.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.775+0000 7f6a808e6700 1 --2- 172.21.15.32:0/3115660833 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a7c05feb0 0x7f6a7c145e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:27.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.775+0000 7f6a818e8700 1 --2- 172.21.15.32:0/3115660833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a7c0b2fb0 0x7f6a7c141ba0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f6a74001450 tx=0x7f6a74007e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:27.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.775+0000 7f6a6e7fc700 1 -- 172.21.15.32:0/3115660833 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a74005b70 con 0x7f6a7c0b2fb0 2020-12-05T17:35:27.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.775+0000 7f6a6e7fc700 1 -- 172.21.15.32:0/3115660833 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a74005cd0 con 0x7f6a7c0b2fb0 2020-12-05T17:35:27.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.775+0000 7f6a820e9700 1 -- 172.21.15.32:0/3115660833 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7c154920 con 0x7f6a7c0b2fb0 2020-12-05T17:35:27.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.775+0000 7f6a6e7fc700 1 -- 172.21.15.32:0/3115660833 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a74007300 con 0x7f6a7c0b2fb0 2020-12-05T17:35:27.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.775+0000 7f6a820e9700 1 -- 172.21.15.32:0/3115660833 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c154c30 con 0x7f6a7c0b2fb0 2020-12-05T17:35:27.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.775+0000 7f6a820e9700 1 -- 172.21.15.32:0/3115660833 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7c13e940 con 0x7f6a7c0b2fb0 2020-12-05T17:35:27.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.775+0000 7f6a6e7fc700 1 -- 172.21.15.32:0/3115660833 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6a74010030 con 0x7f6a7c0b2fb0 2020-12-05T17:35:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.779+0000 7f6a6e7fc700 1 --2- 172.21.15.32:0/3115660833 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6a640553b0 0x7f6a64057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.779+0000 7f6a6e7fc700 1 -- 172.21.15.32:0/3115660833 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6a7406dff0 con 0x7f6a7c0b2fb0 2020-12-05T17:35:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.779+0000 7f6a810e7700 1 --2- 172.21.15.32:0/3115660833 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6a640553b0 0x7f6a64057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.779+0000 7f6a810e7700 1 --2- 172.21.15.32:0/3115660833 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6a640553b0 0x7f6a64057870 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f6a7801a000 tx=0x7f6a78015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.783+0000 7f6a6e7fc700 1 -- 172.21.15.32:0/3115660833 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6a74044240 con 0x7f6a7c0b2fb0 2020-12-05T17:35:27.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.919+0000 7f6a820e9700 1 -- 172.21.15.32:0/3115660833 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6a7c057050 con 0x7f6a7c0b2fb0 2020-12-05T17:35:27.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.923+0000 7f6a6e7fc700 1 -- 172.21.15.32:0/3115660833 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+10 (secure 0 0 0) 0x7f6a74047860 con 0x7f6a7c0b2fb0 2020-12-05T17:35:27.921 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:35:27.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.923+0000 7f6a820e9700 1 -- 172.21.15.32:0/3115660833 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6a640553b0 msgr2=0x7f6a64057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:27.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.923+0000 7f6a820e9700 1 --2- 172.21.15.32:0/3115660833 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6a640553b0 0x7f6a64057870 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f6a7801a000 tx=0x7f6a78015000).stop 2020-12-05T17:35:27.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.923+0000 7f6a820e9700 1 -- 172.21.15.32:0/3115660833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a7c0b2fb0 msgr2=0x7f6a7c141ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:27.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.923+0000 7f6a820e9700 1 --2- 172.21.15.32:0/3115660833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a7c0b2fb0 0x7f6a7c141ba0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f6a74001450 tx=0x7f6a74007e40).stop 2020-12-05T17:35:27.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.923+0000 7f6a820e9700 1 -- 172.21.15.32:0/3115660833 shutdown_connections 2020-12-05T17:35:27.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.923+0000 7f6a820e9700 1 --2- 172.21.15.32:0/3115660833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a7c0b2fb0 0x7f6a7c141ba0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.923+0000 7f6a820e9700 1 --2- 172.21.15.32:0/3115660833 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a7c05f3b0 0x7f6a7c066680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.923+0000 7f6a820e9700 1 --2- 172.21.15.32:0/3115660833 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6a640553b0 0x7f6a64057870 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.923+0000 7f6a820e9700 1 --2- 172.21.15.32:0/3115660833 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a7c05feb0 0x7f6a7c145e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:27.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.923+0000 7f6a820e9700 1 -- 172.21.15.32:0/3115660833 >> 172.21.15.32:0/3115660833 conn(0x7f6a7c0b8f20 msgr2=0x7f6a7c0b5480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:27.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.923+0000 7f6a820e9700 1 -- 172.21.15.32:0/3115660833 shutdown_connections 2020-12-05T17:35:27.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:27.923+0000 7f6a820e9700 1 -- 172.21.15.32:0/3115660833 wait complete. 2020-12-05T17:35:27.935 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:35:28.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.059+0000 7f452d888700 1 Processor -- start 2020-12-05T17:35:28.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.059+0000 7f452d888700 1 -- start start 2020-12-05T17:35:28.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.059+0000 7f452d888700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f452805f6f0 0x7f4528141700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:28.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.059+0000 7f452d888700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4528061b30 0x7f4528141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:28.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.059+0000 7f452d888700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45281420e0 0x7f45280b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:28.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.059+0000 7f452d888700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45281439c0 con 0x7f45281420e0 2020-12-05T17:35:28.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.059+0000 7f452d888700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4528142550 con 0x7f452805f6f0 2020-12-05T17:35:28.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.059+0000 7f452d888700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452805fe20 con 0x7f4528061b30 2020-12-05T17:35:28.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.059+0000 7f452d087700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45281420e0 0x7f45280b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:28.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.059+0000 7f452d087700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45281420e0 0x7f45280b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35450/0 (socket says 172.21.15.32:35450) 2020-12-05T17:35:28.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.059+0000 7f452d087700 1 -- 172.21.15.32:0/3123447354 learned_addr learned my addr 172.21.15.32:0/3123447354 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:28.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f4527fff700 1 --2- 172.21.15.32:0/3123447354 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4528061b30 0x7f4528141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:28.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452c886700 1 --2- 172.21.15.32:0/3123447354 >> v2:172.21.15.61:3300/0 conn(0x7f452805f6f0 0x7f4528141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:28.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452d087700 1 -- 172.21.15.32:0/3123447354 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4528061b30 msgr2=0x7f4528141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:28.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452d087700 1 --2- 172.21.15.32:0/3123447354 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4528061b30 0x7f4528141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:28.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452d087700 1 -- 172.21.15.32:0/3123447354 >> v2:172.21.15.61:3300/0 conn(0x7f452805f6f0 msgr2=0x7f4528141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:28.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452d087700 1 --2- 172.21.15.32:0/3123447354 >> v2:172.21.15.61:3300/0 conn(0x7f452805f6f0 0x7f4528141700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:28.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452d087700 1 -- 172.21.15.32:0/3123447354 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4528146910 con 0x7f45281420e0 2020-12-05T17:35:28.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f4527fff700 1 --2- 172.21.15.32:0/3123447354 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4528061b30 0x7f4528141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:28.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452c886700 1 --2- 172.21.15.32:0/3123447354 >> v2:172.21.15.61:3300/0 conn(0x7f452805f6f0 0x7f4528141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:28.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452d087700 1 --2- 172.21.15.32:0/3123447354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45281420e0 0x7f45280b2660 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f45200032b0 tx=0x7f4520009270).ready entity=mon.0 client_cookie=9ec59f270593a653 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:28.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f45277fe700 1 -- 172.21.15.32:0/3123447354 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f452000bc00 con 0x7f45281420e0 2020-12-05T17:35:28.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f45277fe700 1 -- 172.21.15.32:0/3123447354 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452000bd60 con 0x7f45281420e0 2020-12-05T17:35:28.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f45277fe700 1 -- 172.21.15.32:0/3123447354 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4520011400 con 0x7f45281420e0 2020-12-05T17:35:28.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452d888700 1 -- 172.21.15.32:0/3123447354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45281420e0 msgr2=0x7f45280b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:28.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452d888700 1 --2- 172.21.15.32:0/3123447354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45281420e0 0x7f45280b2660 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f45200032b0 tx=0x7f4520009270).stop 2020-12-05T17:35:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452d888700 1 -- 172.21.15.32:0/3123447354 shutdown_connections 2020-12-05T17:35:28.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452d888700 1 --2- 172.21.15.32:0/3123447354 >> v2:172.21.15.61:3300/0 conn(0x7f452805f6f0 0x7f4528141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:28.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452d888700 1 --2- 172.21.15.32:0/3123447354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45281420e0 0x7f45280b2660 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:28.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452d888700 1 --2- 172.21.15.32:0/3123447354 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4528061b30 0x7f4528141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:28.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452d888700 1 -- 172.21.15.32:0/3123447354 >> 172.21.15.32:0/3123447354 conn(0x7f4528061010 msgr2=0x7f4528061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:28.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452d888700 1 -- 172.21.15.32:0/3123447354 shutdown_connections 2020-12-05T17:35:28.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452d888700 1 -- 172.21.15.32:0/3123447354 wait complete. 2020-12-05T17:35:28.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452d888700 1 Processor -- start 2020-12-05T17:35:28.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452d888700 1 -- start start 2020-12-05T17:35:28.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452d888700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f452805f6f0 0x7f45281548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:28.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452d888700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4528061b30 0x7f45280b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:28.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452d888700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45281420e0 0x7f4528113500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:28.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452d888700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4528065a80 con 0x7f4528061b30 2020-12-05T17:35:28.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452d888700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4528064e60 con 0x7f45281420e0 2020-12-05T17:35:28.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f452d888700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4528065100 con 0x7f452805f6f0 2020-12-05T17:35:28.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f4527fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4528061b30 0x7f45280b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:28.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f4527fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4528061b30 0x7f45280b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35456/0 (socket says 172.21.15.32:35456) 2020-12-05T17:35:28.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.063+0000 7f4527fff700 1 -- 172.21.15.32:0/1212546636 learned_addr learned my addr 172.21.15.32:0/1212546636 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:28.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.067+0000 7f452d087700 1 --2- 172.21.15.32:0/1212546636 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45281420e0 0x7f4528113500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:28.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.067+0000 7f452c886700 1 --2- 172.21.15.32:0/1212546636 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f452805f6f0 0x7f45281548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:28.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.067+0000 7f4527fff700 1 -- 172.21.15.32:0/1212546636 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f452805f6f0 msgr2=0x7f45281548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:28.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.067+0000 7f4527fff700 1 --2- 172.21.15.32:0/1212546636 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f452805f6f0 0x7f45281548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:28.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.067+0000 7f4527fff700 1 -- 172.21.15.32:0/1212546636 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45281420e0 msgr2=0x7f4528113500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:28.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.067+0000 7f4527fff700 1 --2- 172.21.15.32:0/1212546636 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45281420e0 0x7f4528113500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:28.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.067+0000 7f4527fff700 1 -- 172.21.15.32:0/1212546636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45280653a0 con 0x7f4528061b30 2020-12-05T17:35:28.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.067+0000 7f452d087700 1 --2- 172.21.15.32:0/1212546636 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45281420e0 0x7f4528113500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:28.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.067+0000 7f4527fff700 1 --2- 172.21.15.32:0/1212546636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4528061b30 0x7f45280b41c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f451400a000 tx=0x7f4514008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:28.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.067+0000 7f4525ffb700 1 -- 172.21.15.32:0/1212546636 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45140127e0 con 0x7f4528061b30 2020-12-05T17:35:28.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.067+0000 7f452d888700 1 -- 172.21.15.32:0/1212546636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45280b4d60 con 0x7f4528061b30 2020-12-05T17:35:28.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.067+0000 7f4525ffb700 1 -- 172.21.15.32:0/1212546636 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4514012940 con 0x7f4528061b30 2020-12-05T17:35:28.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.067+0000 7f4525ffb700 1 -- 172.21.15.32:0/1212546636 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4514012ca0 con 0x7f4528061b30 2020-12-05T17:35:28.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.067+0000 7f452d888700 1 -- 172.21.15.32:0/1212546636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4528063940 con 0x7f4528061b30 2020-12-05T17:35:28.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.067+0000 7f452d888700 1 -- 172.21.15.32:0/1212546636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45280b4ef0 con 0x7f4528061b30 2020-12-05T17:35:28.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.067+0000 7f4525ffb700 1 -- 172.21.15.32:0/1212546636 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4514013480 con 0x7f4528061b30 2020-12-05T17:35:28.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.067+0000 7f4525ffb700 1 --2- 172.21.15.32:0/1212546636 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f45100553b0 0x7f4510057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:28.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.067+0000 7f4525ffb700 1 -- 172.21.15.32:0/1212546636 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f451406e750 con 0x7f4528061b30 2020-12-05T17:35:28.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.067+0000 7f452c886700 1 --2- 172.21.15.32:0/1212546636 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f45100553b0 0x7f4510057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:28.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.071+0000 7f452c886700 1 --2- 172.21.15.32:0/1212546636 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f45100553b0 0x7f4510057870 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f451c00c000 tx=0x7f451c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:28.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.079+0000 7f4525ffb700 1 -- 172.21.15.32:0/1212546636 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f45140447f0 con 0x7f4528061b30 2020-12-05T17:35:28.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.219+0000 7f452d888700 1 -- 172.21.15.32:0/1212546636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4528063940 con 0x7f4528061b30 2020-12-05T17:35:28.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.219+0000 7f4525ffb700 1 -- 172.21.15.32:0/1212546636 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+10 (secure 0 0 0) 0x7f4528063940 con 0x7f4528061b30 2020-12-05T17:35:28.221 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:35:28.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.223+0000 7f452d888700 1 -- 172.21.15.32:0/1212546636 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f45100553b0 msgr2=0x7f4510057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:28.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.223+0000 7f452d888700 1 --2- 172.21.15.32:0/1212546636 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f45100553b0 0x7f4510057870 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f451c00c000 tx=0x7f451c008040).stop 2020-12-05T17:35:28.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.223+0000 7f452d888700 1 -- 172.21.15.32:0/1212546636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4528061b30 msgr2=0x7f45280b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:28.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.223+0000 7f452d888700 1 --2- 172.21.15.32:0/1212546636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4528061b30 0x7f45280b41c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f451400a000 tx=0x7f4514008040).stop 2020-12-05T17:35:28.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.223+0000 7f452d888700 1 -- 172.21.15.32:0/1212546636 shutdown_connections 2020-12-05T17:35:28.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.223+0000 7f452d888700 1 --2- 172.21.15.32:0/1212546636 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45281420e0 0x7f4528113500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:28.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.223+0000 7f452d888700 1 --2- 172.21.15.32:0/1212546636 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f45100553b0 0x7f4510057870 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:28.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.223+0000 7f452d888700 1 --2- 172.21.15.32:0/1212546636 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f452805f6f0 0x7f45281548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:28.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.223+0000 7f452d888700 1 --2- 172.21.15.32:0/1212546636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4528061b30 0x7f45280b41c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:28.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.223+0000 7f452d888700 1 -- 172.21.15.32:0/1212546636 >> 172.21.15.32:0/1212546636 conn(0x7f4528061010 msgr2=0x7f4528148e50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:28.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.223+0000 7f452d888700 1 -- 172.21.15.32:0/1212546636 shutdown_connections 2020-12-05T17:35:28.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.223+0000 7f452d888700 1 -- 172.21.15.32:0/1212546636 wait complete. 2020-12-05T17:35:28.233 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:35:28.234 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:35:28.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.347+0000 7fbe6f59e700 1 Processor -- start 2020-12-05T17:35:28.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.347+0000 7fbe6f59e700 1 -- start start 2020-12-05T17:35:28.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.347+0000 7fbe6f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbe70061780 0x7fbe700b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:28.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.347+0000 7fbe6f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe700b78d0 0x7fbe70145c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:28.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.347+0000 7fbe6f59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe70062970 con 0x7fbe7013c6f0 2020-12-05T17:35:28.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.347+0000 7fbe6f59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe700b2340 con 0x7fbe70061780 2020-12-05T17:35:28.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.347+0000 7fbe6f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe700add70 con 0x7fbe700b78d0 2020-12-05T17:35:28.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.347+0000 7fbe6dd9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbe70061780 0x7fbe700b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:28.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.347+0000 7fbe6dd9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbe70061780 0x7fbe700b73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35462/0 (socket says 172.21.15.32:35462) 2020-12-05T17:35:28.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.347+0000 7fbe6dd9b700 1 -- 172.21.15.32:0/1406046419 learned_addr learned my addr 172.21.15.32:0/1406046419 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:28.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.347+0000 7fbe6ed9d700 1 --2- 172.21.15.32:0/1406046419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe700b78d0 0x7fbe70145c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:28.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.347+0000 7fbe6dd9b700 1 -- 172.21.15.32:0/1406046419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe700b78d0 msgr2=0x7fbe70145c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:28.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.347+0000 7fbe6dd9b700 1 --2- 172.21.15.32:0/1406046419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe700b78d0 0x7fbe70145c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:28.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.347+0000 7fbe6dd9b700 1 -- 172.21.15.32:0/1406046419 >> v1:172.21.15.61:6789/0 conn(0x7fbe7013c6f0 legacy=0x7fbe700ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:28.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.347+0000 7fbe6dd9b700 1 -- 172.21.15.32:0/1406046419 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe700ae010 con 0x7fbe70061780 2020-12-05T17:35:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.347+0000 7fbe6dd9b700 1 --2- 172.21.15.32:0/1406046419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe70061780 0x7fbe700b73e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7fbe64002f30 tx=0x7fbe64006e80).ready entity=mon.0 client_cookie=740ee6052c04cbbc server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.347+0000 7fbe6d59a700 1 -- 172.21.15.32:0/1406046419 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe640097f0 con 0x7fbe70061780 2020-12-05T17:35:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.347+0000 7fbe6d59a700 1 -- 172.21.15.32:0/1406046419 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe64009950 con 0x7fbe70061780 2020-12-05T17:35:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.347+0000 7fbe6d59a700 1 -- 172.21.15.32:0/1406046419 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe64009cb0 con 0x7fbe70061780 2020-12-05T17:35:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.347+0000 7fbe6f59e700 1 -- 172.21.15.32:0/1406046419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe70061780 msgr2=0x7fbe700b73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:28.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.347+0000 7fbe6f59e700 1 --2- 172.21.15.32:0/1406046419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe70061780 0x7fbe700b73e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7fbe64002f30 tx=0x7fbe64006e80).stop 2020-12-05T17:35:28.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.351+0000 7fbe6f59e700 1 -- 172.21.15.32:0/1406046419 shutdown_connections 2020-12-05T17:35:28.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.351+0000 7fbe6f59e700 1 --2- 172.21.15.32:0/1406046419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe700b78d0 0x7fbe70145c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:28.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.351+0000 7fbe6f59e700 1 --2- 172.21.15.32:0/1406046419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe70061780 0x7fbe700b73e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:28.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.351+0000 7fbe6f59e700 1 -- 172.21.15.32:0/1406046419 >> 172.21.15.32:0/1406046419 conn(0x7fbe7005bc20 msgr2=0x7fbe700b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:28.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.351+0000 7fbe6f59e700 1 -- 172.21.15.32:0/1406046419 shutdown_connections 2020-12-05T17:35:28.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.351+0000 7fbe6f59e700 1 -- 172.21.15.32:0/1406046419 wait complete. 2020-12-05T17:35:28.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.351+0000 7fbe6f59e700 1 Processor -- start 2020-12-05T17:35:28.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.351+0000 7fbe6f59e700 1 -- start start 2020-12-05T17:35:28.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.355+0000 7fbe6f59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbe70061780 0x7fbe700b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:28.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.355+0000 7fbe6f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe700b78d0 0x7fbe70151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:28.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.355+0000 7fbe6f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe7013c6f0 0x7fbe701517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:28.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.355+0000 7fbe6f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe701524d0 con 0x7fbe700b78d0 2020-12-05T17:35:28.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.355+0000 7fbe6f59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe700aff30 con 0x7fbe70061780 2020-12-05T17:35:28.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.355+0000 7fbe6f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe700b01d0 con 0x7fbe7013c6f0 2020-12-05T17:35:28.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.355+0000 7fbe6dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe700b78d0 0x7fbe70151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:28.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.355+0000 7fbe6dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe700b78d0 0x7fbe70151fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35466/0 (socket says 172.21.15.32:35466) 2020-12-05T17:35:28.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.355+0000 7fbe6dd9b700 1 -- 172.21.15.32:0/554842957 learned_addr learned my addr 172.21.15.32:0/554842957 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:28.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.355+0000 7fbe6e59c700 1 --2- 172.21.15.32:0/554842957 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbe70061780 0x7fbe700b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:28.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.355+0000 7fbe6ed9d700 1 --2- 172.21.15.32:0/554842957 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe7013c6f0 0x7fbe701517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:28.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.355+0000 7fbe6dd9b700 1 -- 172.21.15.32:0/554842957 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe7013c6f0 msgr2=0x7fbe701517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:28.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.355+0000 7fbe6dd9b700 1 --2- 172.21.15.32:0/554842957 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe7013c6f0 0x7fbe701517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:28.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.355+0000 7fbe6dd9b700 1 -- 172.21.15.32:0/554842957 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbe70061780 msgr2=0x7fbe700b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:28.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.355+0000 7fbe6dd9b700 1 --2- 172.21.15.32:0/554842957 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbe70061780 0x7fbe700b0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:28.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.355+0000 7fbe6dd9b700 1 -- 172.21.15.32:0/554842957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe700b0470 con 0x7fbe700b78d0 2020-12-05T17:35:28.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.355+0000 7fbe6dd9b700 1 --2- 172.21.15.32:0/554842957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe700b78d0 0x7fbe70151fe0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fbe64002f30 tx=0x7fbe64015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:28.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.355+0000 7fbe5f7fe700 1 -- 172.21.15.32:0/554842957 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe64011bf0 con 0x7fbe700b78d0 2020-12-05T17:35:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.355+0000 7fbe5f7fe700 1 -- 172.21.15.32:0/554842957 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe64011d50 con 0x7fbe700b78d0 2020-12-05T17:35:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.355+0000 7fbe6f59e700 1 -- 172.21.15.32:0/554842957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe70151cd0 con 0x7fbe700b78d0 2020-12-05T17:35:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.355+0000 7fbe5f7fe700 1 -- 172.21.15.32:0/554842957 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe6401e650 con 0x7fbe700b78d0 2020-12-05T17:35:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.355+0000 7fbe6f59e700 1 -- 172.21.15.32:0/554842957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe700aeba0 con 0x7fbe700b78d0 2020-12-05T17:35:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.355+0000 7fbe6f59e700 1 -- 172.21.15.32:0/554842957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe70057050 con 0x7fbe700b78d0 2020-12-05T17:35:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.359+0000 7fbe5f7fe700 1 -- 172.21.15.32:0/554842957 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbe64025030 con 0x7fbe700b78d0 2020-12-05T17:35:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.359+0000 7fbe5f7fe700 1 --2- 172.21.15.32:0/554842957 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbe4c0553f0 0x7fbe4c0578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.359+0000 7fbe5f7fe700 1 -- 172.21.15.32:0/554842957 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbe6406f4c0 con 0x7fbe700b78d0 2020-12-05T17:35:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.359+0000 7fbe6e59c700 1 --2- 172.21.15.32:0/554842957 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbe4c0553f0 0x7fbe4c0578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.359+0000 7fbe6e59c700 1 --2- 172.21.15.32:0/554842957 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbe4c0553f0 0x7fbe4c0578b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fbe6000d000 tx=0x7fbe60007ee0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:28.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.367+0000 7fbe5f7fe700 1 -- 172.21.15.32:0/554842957 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbe64045550 con 0x7fbe700b78d0 2020-12-05T17:35:28.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.559+0000 7fbe6f59e700 1 -- 172.21.15.32:0/554842957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe700b1fe0 con 0x7fbe700b78d0 2020-12-05T17:35:28.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.563+0000 7fbe5f7fe700 1 -- 172.21.15.32:0/554842957 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2136 (secure 0 0 0) 0x7fbe64048b70 con 0x7fbe700b78d0 2020-12-05T17:35:28.561 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:35:28.561 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5159,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/2123765277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2123765277},{"type":"v1","addr":"172.21.15.61:6833","nonce":2123765277}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:35:23.386407+0000","modified":"2020-12-05T17:35:24.389244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5640},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5640":{"gid":5640,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/681412338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":681412338},{"type":"v1","addr":"172.21.15.32:6835","nonce":681412338}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-12-05T17:35:28.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.563+0000 7fbe6f59e700 1 -- 172.21.15.32:0/554842957 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbe4c0553f0 msgr2=0x7fbe4c0578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:28.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.563+0000 7fbe6f59e700 1 --2- 172.21.15.32:0/554842957 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbe4c0553f0 0x7fbe4c0578b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fbe6000d000 tx=0x7fbe60007ee0).stop 2020-12-05T17:35:28.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.563+0000 7fbe6f59e700 1 -- 172.21.15.32:0/554842957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe700b78d0 msgr2=0x7fbe70151fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:28.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.563+0000 7fbe6f59e700 1 --2- 172.21.15.32:0/554842957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe700b78d0 0x7fbe70151fe0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fbe64002f30 tx=0x7fbe64015000).stop 2020-12-05T17:35:28.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.563+0000 7fbe6f59e700 1 -- 172.21.15.32:0/554842957 shutdown_connections 2020-12-05T17:35:28.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.563+0000 7fbe6f59e700 1 --2- 172.21.15.32:0/554842957 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbe4c0553f0 0x7fbe4c0578b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:28.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.563+0000 7fbe6f59e700 1 --2- 172.21.15.32:0/554842957 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe7013c6f0 0x7fbe701517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:28.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.563+0000 7fbe6f59e700 1 --2- 172.21.15.32:0/554842957 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbe70061780 0x7fbe700b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:28.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.563+0000 7fbe6f59e700 1 --2- 172.21.15.32:0/554842957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe700b78d0 0x7fbe70151fe0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:28.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.563+0000 7fbe6f59e700 1 -- 172.21.15.32:0/554842957 >> 172.21.15.32:0/554842957 conn(0x7fbe7005bc20 msgr2=0x7fbe70144510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:28.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.563+0000 7fbe6f59e700 1 -- 172.21.15.32:0/554842957 shutdown_connections 2020-12-05T17:35:28.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:28.563+0000 7fbe6f59e700 1 -- 172.21.15.32:0/554842957 wait complete. 2020-12-05T17:35:28.567 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 23 2020-12-05T17:35:28.780 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:35:23.386407+0000', 'modified': '2020-12-05T17:35:24.389244+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5640}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5640': {'gid': 5640, 'name': 'a', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/681412338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 681412338}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 681412338}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:35:28.781 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:35:28.781 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:35:28.911 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":5640,"want_state":"up:active","state":"up:active","rank_uptime":5.5175507359999996,"mdsmap_epoch":23,"osdmap_epoch":54,"osdmap_epoch_barrier":52,"uptime":10.919893546000001} 2020-12-05T17:35:28.912 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 5640, "mdsmap_epoch": 23, "osdmap_epoch": 54, "osdmap_epoch_barrier": 52, "rank_uptime": 5.517550736, "state": "up:active", "uptime": 10.919893546, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:35:28.913 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:35:28.913 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-12-05T17:35:28.913 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-12-05T17:35:28.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:28.914 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:35:28.922 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:35:28.983 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:35:28.984 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:35:29.040 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:35:29.043 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:35:29.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:29.044 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:35:29.093 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:35:29.093 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:35:29.136 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:35:29.152 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:35:29.153 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:35:29.208 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:35:29.209 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:35:29.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:29.210 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:35:29.215 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:2020-12-05T17:35:29.215+0000 7f4bd7aae0c0 -1 init, newargv = 0x55e2522a0900 newargc=9ceph-fuse[ 2020-12-05T17:35:29.215 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:10727]: starting ceph client 2020-12-05T17:35:29.225 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[10727]: starting fuse 2020-12-05T17:35:30.322 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:35:30.339 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:35:30.340 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:35:30.397 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:35:30.398 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:35:30.399 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:30.400 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:35:30.448 INFO:teuthology.orchestra.run.smithi036.stdout:49 2020-12-05T17:35:30.448 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-12-05T17:35:30.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:30.449 INFO:teuthology.orchestra.run.smithi036:> sudo python3 -c ' 2020-12-05T17:35:30.450 INFO:teuthology.orchestra.run.smithi036:> import glob 2020-12-05T17:35:30.450 INFO:teuthology.orchestra.run.smithi036:> import re 2020-12-05T17:35:30.450 INFO:teuthology.orchestra.run.smithi036:> import os 2020-12-05T17:35:30.451 INFO:teuthology.orchestra.run.smithi036:> import subprocess 2020-12-05T17:35:30.451 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:35:30.452 INFO:teuthology.orchestra.run.smithi036:> def find_socket(client_name): 2020-12-05T17:35:30.452 INFO:teuthology.orchestra.run.smithi036:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-12-05T17:35:30.452 INFO:teuthology.orchestra.run.smithi036:> files = glob.glob(asok_path) 2020-12-05T17:35:30.453 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:35:30.453 INFO:teuthology.orchestra.run.smithi036:> # Given a non-glob path, it better be there 2020-12-05T17:35:30.453 INFO:teuthology.orchestra.run.smithi036:> if "*" not in asok_path: 2020-12-05T17:35:30.454 INFO:teuthology.orchestra.run.smithi036:> assert(len(files) == 1) 2020-12-05T17:35:30.454 INFO:teuthology.orchestra.run.smithi036:> return files[0] 2020-12-05T17:35:30.455 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:35:30.455 INFO:teuthology.orchestra.run.smithi036:> for f in files: 2020-12-05T17:35:30.455 INFO:teuthology.orchestra.run.smithi036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:35:30.456 INFO:teuthology.orchestra.run.smithi036:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:35:30.456 INFO:teuthology.orchestra.run.smithi036:> return f 2020-12-05T17:35:30.456 INFO:teuthology.orchestra.run.smithi036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:35:30.457 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:35:30.457 INFO:teuthology.orchestra.run.smithi036:> print(find_socket("client.0")) 2020-12-05T17:35:30.458 INFO:teuthology.orchestra.run.smithi036:> ' 2020-12-05T17:35:30.538 INFO:teuthology.orchestra.run.smithi036.stdout:/var/run/ceph/ceph-client.0.10727.asok 2020-12-05T17:35:30.543 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.10727.asok 2020-12-05T17:35:30.543 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:30.544 INFO:teuthology.orchestra.run.smithi036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.10727.asok status 2020-12-05T17:35:30.650 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2020-12-05T17:35:30.651 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": { 2020-12-05T17:35:30.651 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:35:30.651 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:35:30.651 INFO:teuthology.orchestra.run.smithi036.stdout: "entity_id": "0", 2020-12-05T17:35:30.652 INFO:teuthology.orchestra.run.smithi036.stdout: "hostname": "smithi036", 2020-12-05T17:35:30.652 INFO:teuthology.orchestra.run.smithi036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-12-05T17:35:30.652 INFO:teuthology.orchestra.run.smithi036.stdout: "pid": "10727", 2020-12-05T17:35:30.652 INFO:teuthology.orchestra.run.smithi036.stdout: "root": "/" 2020-12-05T17:35:30.653 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:35:30.653 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_count": 0, 2020-12-05T17:35:30.653 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_pinned_count": 0, 2020-12-05T17:35:30.653 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 5276, 2020-12-05T17:35:30.653 INFO:teuthology.orchestra.run.smithi036.stdout: "inst": { 2020-12-05T17:35:30.654 INFO:teuthology.orchestra.run.smithi036.stdout: "name": { 2020-12-05T17:35:30.654 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "client", 2020-12-05T17:35:30.654 INFO:teuthology.orchestra.run.smithi036.stdout: "num": 5276 2020-12-05T17:35:30.654 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:35:30.654 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:35:30.655 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:35:30.655 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:35:30.655 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 766086136 2020-12-05T17:35:30.655 INFO:teuthology.orchestra.run.smithi036.stdout: } 2020-12-05T17:35:30.656 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:35:30.656 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:35:30.656 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:35:30.656 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:35:30.657 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 766086136 2020-12-05T17:35:30.657 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:35:30.657 INFO:teuthology.orchestra.run.smithi036.stdout: "inst_str": "client.5276 172.21.15.36:0/766086136", 2020-12-05T17:35:30.657 INFO:teuthology.orchestra.run.smithi036.stdout: "addr_str": "172.21.15.36:0/766086136", 2020-12-05T17:35:30.657 INFO:teuthology.orchestra.run.smithi036.stdout: "inode_count": 1, 2020-12-05T17:35:30.658 INFO:teuthology.orchestra.run.smithi036.stdout: "mds_epoch": 23, 2020-12-05T17:35:30.658 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch": 54, 2020-12-05T17:35:30.658 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:35:30.658 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklisted": false 2020-12-05T17:35:30.659 INFO:teuthology.orchestra.run.smithi036.stdout:} 2020-12-05T17:35:30.661 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:30.661 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:35:30.662 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:35:30.670 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:35:30.671 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:35:30.672 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:30.672 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:35:30.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:30.732 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:35:30.732 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:35:30.781 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:35:30.782 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:35:30.782 INFO:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-12-05T17:35:30.783 INFO:teuthology.orchestra.run.smithi036:> import sys 2020-12-05T17:35:30.783 INFO:teuthology.orchestra.run.smithi036:> import time 2020-12-05T17:35:30.784 INFO:teuthology.orchestra.run.smithi036:> import os 2020-12-05T17:35:30.784 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:35:30.784 INFO:teuthology.orchestra.run.smithi036:> n = 1000 2020-12-05T17:35:30.785 INFO:teuthology.orchestra.run.smithi036:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2020-12-05T17:35:30.785 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:35:30.785 INFO:teuthology.orchestra.run.smithi036:> if not os.path.exists(os.path.dirname(abs_path)): 2020-12-05T17:35:30.786 INFO:teuthology.orchestra.run.smithi036:> os.makedirs(os.path.dirname(abs_path)) 2020-12-05T17:35:30.786 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:35:30.786 INFO:teuthology.orchestra.run.smithi036:> for i in range(0, n): 2020-12-05T17:35:30.787 INFO:teuthology.orchestra.run.smithi036:> fname = "{0}_{1}".format(abs_path, i) 2020-12-05T17:35:30.787 INFO:teuthology.orchestra.run.smithi036:> with open(fname, '"'"'w'"'"') as f: 2020-12-05T17:35:30.788 INFO:teuthology.orchestra.run.smithi036:> f.write('"'"'content'"'"') 2020-12-05T17:35:30.788 INFO:teuthology.orchestra.run.smithi036:> if True: 2020-12-05T17:35:30.788 INFO:teuthology.orchestra.run.smithi036:> f.flush() 2020-12-05T17:35:30.789 INFO:teuthology.orchestra.run.smithi036:> os.fsync(f.fileno()) 2020-12-05T17:35:30.789 INFO:teuthology.orchestra.run.smithi036:> ' 2020-12-05T17:35:43.713 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:35:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.839+0000 7f4bf5ab0700 1 Processor -- start 2020-12-05T17:35:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.839+0000 7f4bf5ab0700 1 -- start start 2020-12-05T17:35:43.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.839+0000 7f4bf5ab0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4bf00b1fc0 0x7f4bf00af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:43.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.839+0000 7f4bf5ab0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf0146a90 con 0x7f4bf00ab710 2020-12-05T17:35:43.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.839+0000 7f4bf5ab0700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf00b23a0 con 0x7f4bf0147400 2020-12-05T17:35:43.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.839+0000 7f4bf5ab0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf0061e90 con 0x7f4bf00b1fc0 2020-12-05T17:35:43.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.839+0000 7f4beffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4bf00ab710 0x7f4bf0061780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38124/0 (socket says 172.21.15.32:38124) 2020-12-05T17:35:43.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.839+0000 7f4beffff700 1 -- 172.21.15.32:0/341961638 learned_addr learned my addr 172.21.15.32:0/341961638 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:35:43.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.839+0000 7f4bf4aae700 1 --2- 172.21.15.32:0/341961638 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4bf00b1fc0 0x7f4bf00af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:43.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.843+0000 7f4bef7fe700 1 -- 172.21.15.32:0/341961638 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4200477635 0 0) 0x7f4bf0146a90 con 0x7f4bf00ab710 2020-12-05T17:35:43.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.843+0000 7f4bef7fe700 1 -- 172.21.15.32:0/341961638 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bd0003230 con 0x7f4bf00ab710 2020-12-05T17:35:43.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.843+0000 7f4bf4aae700 1 -- 172.21.15.32:0/341961638 >> v1:172.21.15.61:6789/0 conn(0x7f4bf0147400 legacy=0x7f4bf0145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:43.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.843+0000 7f4bf4aae700 1 -- 172.21.15.32:0/341961638 >> v1:172.21.15.32:6789/0 conn(0x7f4bf00ab710 legacy=0x7f4bf0061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:43.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.843+0000 7f4bf4aae700 1 -- 172.21.15.32:0/341961638 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf0146cc0 con 0x7f4bf00b1fc0 2020-12-05T17:35:43.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.843+0000 7f4bf4aae700 1 --2- 172.21.15.32:0/341961638 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4bf00b1fc0 0x7f4bf00af230 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f4be80032b0 tx=0x7f4be80091a0).ready entity=mon.2 client_cookie=39e0685d1046808c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:43.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.843+0000 7f4bef7fe700 1 -- 172.21.15.32:0/341961638 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4be800b9f0 con 0x7f4bf00b1fc0 2020-12-05T17:35:43.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.843+0000 7f4bef7fe700 1 -- 172.21.15.32:0/341961638 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be800bb50 con 0x7f4bf00b1fc0 2020-12-05T17:35:43.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.843+0000 7f4bef7fe700 1 -- 172.21.15.32:0/341961638 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4be800bcc0 con 0x7f4bf00b1fc0 2020-12-05T17:35:43.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.843+0000 7f4bf5ab0700 1 -- 172.21.15.32:0/341961638 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4bf00b1fc0 msgr2=0x7f4bf00af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:43.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.843+0000 7f4bf5ab0700 1 --2- 172.21.15.32:0/341961638 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4bf00b1fc0 0x7f4bf00af230 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f4be80032b0 tx=0x7f4be80091a0).stop 2020-12-05T17:35:43.845 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.843+0000 7f4bf5ab0700 1 -- 172.21.15.32:0/341961638 shutdown_connections 2020-12-05T17:35:43.845 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.843+0000 7f4bf5ab0700 1 --2- 172.21.15.32:0/341961638 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4bf00b1fc0 0x7f4bf00af230 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:43.845 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.843+0000 7f4bf5ab0700 1 -- 172.21.15.32:0/341961638 >> 172.21.15.32:0/341961638 conn(0x7f4bf00b5530 msgr2=0x7f4bf00b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:43.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.843+0000 7f4bf5ab0700 1 -- 172.21.15.32:0/341961638 shutdown_connections 2020-12-05T17:35:43.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.843+0000 7f4bf5ab0700 1 -- 172.21.15.32:0/341961638 wait complete. 2020-12-05T17:35:43.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.847+0000 7f4bf5ab0700 1 Processor -- start 2020-12-05T17:35:43.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.847+0000 7f4bf5ab0700 1 -- start start 2020-12-05T17:35:43.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.847+0000 7f4bf5ab0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf00ab710 0x7f4bf009e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:43.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.847+0000 7f4bf5ab0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4bf00b1fc0 0x7f4bf0142f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:43.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.847+0000 7f4bf5ab0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4bf0147400 0x7f4bf01433f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:43.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.847+0000 7f4bf5ab0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf00ad410 con 0x7f4bf00ab710 2020-12-05T17:35:43.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.847+0000 7f4bf5ab0700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf01562c0 con 0x7f4bf00b1fc0 2020-12-05T17:35:43.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.847+0000 7f4bf5ab0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf0156560 con 0x7f4bf0147400 2020-12-05T17:35:43.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.847+0000 7f4bf4aae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf00ab710 0x7f4bf009e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:43.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.847+0000 7f4bf4aae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf00ab710 0x7f4bf009e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35480/0 (socket says 172.21.15.32:35480) 2020-12-05T17:35:43.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.847+0000 7f4bf4aae700 1 -- 172.21.15.32:0/318086738 learned_addr learned my addr 172.21.15.32:0/318086738 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:43.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.847+0000 7f4beffff700 1 --2- 172.21.15.32:0/318086738 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4bf00b1fc0 0x7f4bf0142f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:43.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.847+0000 7f4bf52af700 1 --2- 172.21.15.32:0/318086738 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4bf0147400 0x7f4bf01433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:43.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.847+0000 7f4bf4aae700 1 -- 172.21.15.32:0/318086738 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4bf0147400 msgr2=0x7f4bf01433f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:43.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.847+0000 7f4bf4aae700 1 --2- 172.21.15.32:0/318086738 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4bf0147400 0x7f4bf01433f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:43.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.847+0000 7f4bf4aae700 1 -- 172.21.15.32:0/318086738 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4bf00b1fc0 msgr2=0x7f4bf0142f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:43.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.847+0000 7f4bf4aae700 1 --2- 172.21.15.32:0/318086738 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4bf00b1fc0 0x7f4bf0142f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:43.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.847+0000 7f4bf4aae700 1 -- 172.21.15.32:0/318086738 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf00aece0 con 0x7f4bf00ab710 2020-12-05T17:35:43.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.847+0000 7f4beffff700 1 --2- 172.21.15.32:0/318086738 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4bf00b1fc0 0x7f4bf0142f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:35:43.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.847+0000 7f4bf4aae700 1 --2- 172.21.15.32:0/318086738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf00ab710 0x7f4bf009e460 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f4be80032b0 tx=0x7f4be8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:43.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.851+0000 7f4bedffb700 1 -- 172.21.15.32:0/318086738 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4be801f3f0 con 0x7f4bf00ab710 2020-12-05T17:35:43.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.851+0000 7f4bedffb700 1 -- 172.21.15.32:0/318086738 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be801f550 con 0x7f4bf00ab710 2020-12-05T17:35:43.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.851+0000 7f4bf5ab0700 1 -- 172.21.15.32:0/318086738 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf0143e60 con 0x7f4bf00ab710 2020-12-05T17:35:43.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.851+0000 7f4bedffb700 1 -- 172.21.15.32:0/318086738 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4be801f8b0 con 0x7f4bf00ab710 2020-12-05T17:35:43.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.851+0000 7f4bf5ab0700 1 -- 172.21.15.32:0/318086738 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf0158ef0 con 0x7f4bf00ab710 2020-12-05T17:35:43.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.851+0000 7f4bf5ab0700 1 -- 172.21.15.32:0/318086738 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf0057050 con 0x7f4bf00ab710 2020-12-05T17:35:43.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.851+0000 7f4bedffb700 1 -- 172.21.15.32:0/318086738 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4be8026030 con 0x7f4bf00ab710 2020-12-05T17:35:43.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.851+0000 7f4bedffb700 1 --2- 172.21.15.32:0/318086738 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4bd4055330 0x7f4bd40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:43.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.851+0000 7f4bedffb700 1 -- 172.21.15.32:0/318086738 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4be806eb40 con 0x7f4bf00ab710 2020-12-05T17:35:43.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.851+0000 7f4beffff700 1 --2- 172.21.15.32:0/318086738 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4bd4055330 0x7f4bd40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:43.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.851+0000 7f4beffff700 1 --2- 172.21.15.32:0/318086738 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4bd4055330 0x7f4bd40577f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f4be000d000 tx=0x7f4be0009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:43.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:43.859+0000 7f4bedffb700 1 -- 172.21.15.32:0/318086738 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4be8044ce0 con 0x7f4bf00ab710 2020-12-05T17:35:44.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.047+0000 7f4bf5ab0700 1 -- 172.21.15.32:0/318086738 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4bf0158ef0 con 0x7f4bf00ab710 2020-12-05T17:35:44.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.051+0000 7f4bedffb700 1 -- 172.21.15.32:0/318086738 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2136 (secure 0 0 0) 0x7f4bf0158ef0 con 0x7f4bf00ab710 2020-12-05T17:35:44.048 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:35:44.048 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5159,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/2123765277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2123765277},{"type":"v1","addr":"172.21.15.61:6833","nonce":2123765277}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:35:23.386407+0000","modified":"2020-12-05T17:35:24.389244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5640},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5640":{"gid":5640,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/681412338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":681412338},{"type":"v1","addr":"172.21.15.32:6835","nonce":681412338}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-12-05T17:35:44.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.051+0000 7f4bf5ab0700 1 -- 172.21.15.32:0/318086738 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4bd4055330 msgr2=0x7f4bd40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:44.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.051+0000 7f4bf5ab0700 1 --2- 172.21.15.32:0/318086738 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4bd4055330 0x7f4bd40577f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f4be000d000 tx=0x7f4be0009040).stop 2020-12-05T17:35:44.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.051+0000 7f4bf5ab0700 1 -- 172.21.15.32:0/318086738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf00ab710 msgr2=0x7f4bf009e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:44.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.051+0000 7f4bf5ab0700 1 --2- 172.21.15.32:0/318086738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf00ab710 0x7f4bf009e460 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f4be80032b0 tx=0x7f4be8016000).stop 2020-12-05T17:35:44.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.051+0000 7f4bf5ab0700 1 -- 172.21.15.32:0/318086738 shutdown_connections 2020-12-05T17:35:44.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.051+0000 7f4bf5ab0700 1 --2- 172.21.15.32:0/318086738 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4bd4055330 0x7f4bd40577f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:44.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.051+0000 7f4bf5ab0700 1 --2- 172.21.15.32:0/318086738 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4bf0147400 0x7f4bf01433f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:44.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.051+0000 7f4bf5ab0700 1 --2- 172.21.15.32:0/318086738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf00ab710 0x7f4bf009e460 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:44.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.051+0000 7f4bf5ab0700 1 --2- 172.21.15.32:0/318086738 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4bf00b1fc0 0x7f4bf0142f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:44.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.051+0000 7f4bf5ab0700 1 -- 172.21.15.32:0/318086738 >> 172.21.15.32:0/318086738 conn(0x7f4bf00b5530 msgr2=0x7f4bf00afe30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:44.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.051+0000 7f4bf5ab0700 1 -- 172.21.15.32:0/318086738 shutdown_connections 2020-12-05T17:35:44.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.051+0000 7f4bf5ab0700 1 -- 172.21.15.32:0/318086738 wait complete. 2020-12-05T17:35:44.053 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 23 2020-12-05T17:35:44.062 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_caps 20 2020-12-05T17:35:44.176 INFO:teuthology.orchestra.run.smithi032.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2020-12-05T17:35:44.177 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2020-12-05T17:35:44.177 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:35:44.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.303+0000 7f4e6dd65700 1 Processor -- start 2020-12-05T17:35:44.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.303+0000 7f4e6dd65700 1 -- start start 2020-12-05T17:35:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.303+0000 7f4e6dd65700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4e680af920 0x7f4e68063040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.303+0000 7f4e6dd65700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f4e680afea0 0x7f4e681409c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.303+0000 7f4e6dd65700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6805fad0 con 0x7f4e6805f6f0 2020-12-05T17:35:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.303+0000 7f4e6dd65700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e680b2cd0 con 0x7f4e680afea0 2020-12-05T17:35:44.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.303+0000 7f4e6dd65700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e680b3560 con 0x7f4e680af920 2020-12-05T17:35:44.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.303+0000 7f4e6cd63700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4e6805f6f0 0x7f4e680b1d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38136/0 (socket says 172.21.15.32:38136) 2020-12-05T17:35:44.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.303+0000 7f4e6cd63700 1 -- 172.21.15.32:0/3052157826 learned_addr learned my addr 172.21.15.32:0/3052157826 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:35:44.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.303+0000 7f4e67fff700 1 --2- 172.21.15.32:0/3052157826 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4e680af920 0x7f4e68063040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:44.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.303+0000 7f4e6d564700 1 --2- 172.21.15.32:0/3052157826 >> v2:172.21.15.61:3300/0 conn(0x7f4e680afea0 0x7f4e681409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:44.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.303+0000 7f4e677fe700 1 -- 172.21.15.32:0/3052157826 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2135252656 0 0) 0x7f4e6805fad0 con 0x7f4e6805f6f0 2020-12-05T17:35:44.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.303+0000 7f4e677fe700 1 -- 172.21.15.32:0/3052157826 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e4c003230 con 0x7f4e6805f6f0 2020-12-05T17:35:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.303+0000 7f4e67fff700 1 -- 172.21.15.32:0/3052157826 >> v2:172.21.15.61:3300/0 conn(0x7f4e680afea0 msgr2=0x7f4e681409c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.303+0000 7f4e67fff700 1 --2- 172.21.15.32:0/3052157826 >> v2:172.21.15.61:3300/0 conn(0x7f4e680afea0 0x7f4e681409c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.303+0000 7f4e67fff700 1 -- 172.21.15.32:0/3052157826 >> v1:172.21.15.32:6789/0 conn(0x7f4e6805f6f0 legacy=0x7f4e680b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:44.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.303+0000 7f4e677fe700 1 -- 172.21.15.32:0/3052157826 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3380826195 0 0) 0x7f4e4c003230 con 0x7f4e6805f6f0 2020-12-05T17:35:44.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.303+0000 7f4e67fff700 1 -- 172.21.15.32:0/3052157826 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e68144650 con 0x7f4e680af920 2020-12-05T17:35:44.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.303+0000 7f4e67fff700 1 --2- 172.21.15.32:0/3052157826 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4e680af920 0x7f4e68063040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f4e5c001870 tx=0x7f4e54006dc0).ready entity=mon.2 client_cookie=a18b5dcf834cff60 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:44.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.303+0000 7f4e677fe700 1 -- 172.21.15.32:0/3052157826 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e5c0016d0 con 0x7f4e680af920 2020-12-05T17:35:44.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e677fe700 1 -- 172.21.15.32:0/3052157826 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5400d570 con 0x7f4e680af920 2020-12-05T17:35:44.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e677fe700 1 -- 172.21.15.32:0/3052157826 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e5400d6e0 con 0x7f4e680af920 2020-12-05T17:35:44.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6dd65700 1 -- 172.21.15.32:0/3052157826 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4e680af920 msgr2=0x7f4e68063040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:44.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6dd65700 1 --2- 172.21.15.32:0/3052157826 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4e680af920 0x7f4e68063040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f4e5c001870 tx=0x7f4e54006dc0).stop 2020-12-05T17:35:44.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6dd65700 1 -- 172.21.15.32:0/3052157826 shutdown_connections 2020-12-05T17:35:44.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6dd65700 1 --2- 172.21.15.32:0/3052157826 >> v2:172.21.15.61:3300/0 conn(0x7f4e680afea0 0x7f4e681409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:44.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6dd65700 1 --2- 172.21.15.32:0/3052157826 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4e680af920 0x7f4e68063040 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:44.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6dd65700 1 -- 172.21.15.32:0/3052157826 >> 172.21.15.32:0/3052157826 conn(0x7f4e680604c0 msgr2=0x7f4e6805c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:44.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6dd65700 1 -- 172.21.15.32:0/3052157826 shutdown_connections 2020-12-05T17:35:44.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6dd65700 1 -- 172.21.15.32:0/3052157826 wait complete. 2020-12-05T17:35:44.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6dd65700 1 Processor -- start 2020-12-05T17:35:44.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6dd65700 1 -- start start 2020-12-05T17:35:44.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6dd65700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e6805f6f0 0x7f4e68146980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:44.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6dd65700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4e680af920 0x7f4e68065a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:44.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6dd65700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4e680afea0 0x7f4e680b6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:44.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6dd65700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e68063040 con 0x7f4e6805f6f0 2020-12-05T17:35:44.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6dd65700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e680632a0 con 0x7f4e680af920 2020-12-05T17:35:44.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6dd65700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e680b4bc0 con 0x7f4e680afea0 2020-12-05T17:35:44.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6cd63700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e6805f6f0 0x7f4e68146980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:44.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6cd63700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e6805f6f0 0x7f4e68146980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35494/0 (socket says 172.21.15.32:35494) 2020-12-05T17:35:44.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6cd63700 1 -- 172.21.15.32:0/2658598076 learned_addr learned my addr 172.21.15.32:0/2658598076 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:44.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6d564700 1 --2- 172.21.15.32:0/2658598076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4e680afea0 0x7f4e680b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:44.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e67fff700 1 --2- 172.21.15.32:0/2658598076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4e680af920 0x7f4e68065a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:44.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6cd63700 1 -- 172.21.15.32:0/2658598076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4e680afea0 msgr2=0x7f4e680b6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:44.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6cd63700 1 --2- 172.21.15.32:0/2658598076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4e680afea0 0x7f4e680b6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:44.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6cd63700 1 -- 172.21.15.32:0/2658598076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4e680af920 msgr2=0x7f4e68065a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:44.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6cd63700 1 --2- 172.21.15.32:0/2658598076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4e680af920 0x7f4e68065a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:44.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6cd63700 1 -- 172.21.15.32:0/2658598076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e680af010 con 0x7f4e6805f6f0 2020-12-05T17:35:44.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e67fff700 1 --2- 172.21.15.32:0/2658598076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4e680af920 0x7f4e68065a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:44.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e6cd63700 1 --2- 172.21.15.32:0/2658598076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e6805f6f0 0x7f4e68146980 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f4e5c001930 tx=0x7f4e5c001330).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:44.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.307+0000 7f4e65ffb700 1 -- 172.21.15.32:0/2658598076 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e5c007170 con 0x7f4e6805f6f0 2020-12-05T17:35:44.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.311+0000 7f4e65ffb700 1 -- 172.21.15.32:0/2658598076 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5c0072d0 con 0x7f4e6805f6f0 2020-12-05T17:35:44.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.311+0000 7f4e65ffb700 1 -- 172.21.15.32:0/2658598076 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e5c0048f0 con 0x7f4e6805f6f0 2020-12-05T17:35:44.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.311+0000 7f4e6dd65700 1 -- 172.21.15.32:0/2658598076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e68065f30 con 0x7f4e6805f6f0 2020-12-05T17:35:44.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.311+0000 7f4e6dd65700 1 -- 172.21.15.32:0/2658598076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e680b6900 con 0x7f4e6805f6f0 2020-12-05T17:35:44.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.311+0000 7f4e6dd65700 1 -- 172.21.15.32:0/2658598076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e68057050 con 0x7f4e6805f6f0 2020-12-05T17:35:44.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.311+0000 7f4e65ffb700 1 -- 172.21.15.32:0/2658598076 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4e5c010070 con 0x7f4e6805f6f0 2020-12-05T17:35:44.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.311+0000 7f4e65ffb700 1 --2- 172.21.15.32:0/2658598076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4e500554e0 0x7f4e500579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:44.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.311+0000 7f4e65ffb700 1 -- 172.21.15.32:0/2658598076 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4e5c06dba0 con 0x7f4e6805f6f0 2020-12-05T17:35:44.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.311+0000 7f4e67fff700 1 --2- 172.21.15.32:0/2658598076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4e500554e0 0x7f4e500579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:44.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.311+0000 7f4e67fff700 1 --2- 172.21.15.32:0/2658598076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4e500554e0 0x7f4e500579a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f4e54018000 tx=0x7f4e54015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:44.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.315+0000 7f4e65ffb700 1 -- 172.21.15.32:0/2658598076 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4e5c043ca0 con 0x7f4e6805f6f0 2020-12-05T17:35:44.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.507+0000 7f4e6dd65700 1 -- 172.21.15.32:0/2658598076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e68069690 con 0x7f4e6805f6f0 2020-12-05T17:35:44.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.507+0000 7f4e65ffb700 1 -- 172.21.15.32:0/2658598076 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2136 (secure 0 0 0) 0x7f4e5c0472c0 con 0x7f4e6805f6f0 2020-12-05T17:35:44.508 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:35:44.508 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5159,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/2123765277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2123765277},{"type":"v1","addr":"172.21.15.61:6833","nonce":2123765277}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:35:23.386407+0000","modified":"2020-12-05T17:35:24.389244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5640},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5640":{"gid":5640,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/681412338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":681412338},{"type":"v1","addr":"172.21.15.32:6835","nonce":681412338}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-12-05T17:35:44.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.507+0000 7f4e6dd65700 1 -- 172.21.15.32:0/2658598076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4e500554e0 msgr2=0x7f4e500579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:44.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.507+0000 7f4e6dd65700 1 --2- 172.21.15.32:0/2658598076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4e500554e0 0x7f4e500579a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f4e54018000 tx=0x7f4e54015000).stop 2020-12-05T17:35:44.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.507+0000 7f4e6dd65700 1 -- 172.21.15.32:0/2658598076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e6805f6f0 msgr2=0x7f4e68146980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:44.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.507+0000 7f4e6dd65700 1 --2- 172.21.15.32:0/2658598076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e6805f6f0 0x7f4e68146980 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f4e5c001930 tx=0x7f4e5c001330).stop 2020-12-05T17:35:44.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.507+0000 7f4e6dd65700 1 -- 172.21.15.32:0/2658598076 shutdown_connections 2020-12-05T17:35:44.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.507+0000 7f4e6dd65700 1 --2- 172.21.15.32:0/2658598076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4e500554e0 0x7f4e500579a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:44.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.507+0000 7f4e6dd65700 1 --2- 172.21.15.32:0/2658598076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4e680afea0 0x7f4e680b6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:44.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.507+0000 7f4e6dd65700 1 --2- 172.21.15.32:0/2658598076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e6805f6f0 0x7f4e68146980 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:44.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.507+0000 7f4e6dd65700 1 --2- 172.21.15.32:0/2658598076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4e680af920 0x7f4e68065a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:44.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.511+0000 7f4e6dd65700 1 -- 172.21.15.32:0/2658598076 >> 172.21.15.32:0/2658598076 conn(0x7f4e680604c0 msgr2=0x7f4e68141210 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:44.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.511+0000 7f4e6dd65700 1 -- 172.21.15.32:0/2658598076 shutdown_connections 2020-12-05T17:35:44.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.511+0000 7f4e6dd65700 1 -- 172.21.15.32:0/2658598076 wait complete. 2020-12-05T17:35:44.513 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 23 2020-12-05T17:35:44.520 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_decay_threshold 400 2020-12-05T17:35:44.631 INFO:teuthology.orchestra.run.smithi032.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2020-12-05T17:35:44.632 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2020-12-05T17:35:44.632 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@smithi036.front.sepia.ceph.com... 2020-12-05T17:35:44.651 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:35:44.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8dba785700 1 Processor -- start 2020-12-05T17:35:44.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8dba785700 1 -- start start 2020-12-05T17:35:44.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8dba785700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8db405cd70 0x7f8db40b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:44.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8dba785700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8db40b6960 0x7f8db4146050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:44.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8dba785700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db405f6d0 con 0x7f8db40b0770 2020-12-05T17:35:44.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8dba785700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db40b1c40 con 0x7f8db40b6960 2020-12-05T17:35:44.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8dba785700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db405d150 con 0x7f8db405cd70 2020-12-05T17:35:44.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8db8d22700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8db40b6960 0x7f8db4146050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:44.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8db8d22700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8db40b6960 0x7f8db4146050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35506/0 (socket says 172.21.15.32:35506) 2020-12-05T17:35:44.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8db8d22700 1 -- 172.21.15.32:0/2720391356 learned_addr learned my addr 172.21.15.32:0/2720391356 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:44.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8db3fff700 1 --2- 172.21.15.32:0/2720391356 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8db405cd70 0x7f8db40b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:44.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8db8d22700 1 -- 172.21.15.32:0/2720391356 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8db405cd70 msgr2=0x7f8db40b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:44.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8db8d22700 1 --2- 172.21.15.32:0/2720391356 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8db405cd70 0x7f8db40b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:44.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8db8d22700 1 -- 172.21.15.32:0/2720391356 >> v1:172.21.15.61:6789/0 conn(0x7f8db40b0770 legacy=0x7f8db40b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:44.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8db37fe700 1 -- 172.21.15.32:0/2720391356 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868486175 0 0) 0x7f8db405f6d0 con 0x7f8db40b0770 2020-12-05T17:35:44.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8db8d22700 1 -- 172.21.15.32:0/2720391356 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db40602f0 con 0x7f8db40b6960 2020-12-05T17:35:44.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8db3fff700 1 --2- 172.21.15.32:0/2720391356 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8db405cd70 0x7f8db40b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:35:44.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8db8d22700 1 --2- 172.21.15.32:0/2720391356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db40b6960 0x7f8db4146050 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f8dac0032b0 tx=0x7f8dac009270).ready entity=mon.0 client_cookie=f985fe5fb1ea79d6 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:44.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8db37fe700 1 -- 172.21.15.32:0/2720391356 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8dac009e60 con 0x7f8db40b6960 2020-12-05T17:35:44.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8db37fe700 1 -- 172.21.15.32:0/2720391356 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dac00bc00 con 0x7f8db40b6960 2020-12-05T17:35:44.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8db37fe700 1 -- 172.21.15.32:0/2720391356 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8dac00bd70 con 0x7f8db40b6960 2020-12-05T17:35:44.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8dba785700 1 -- 172.21.15.32:0/2720391356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db40b6960 msgr2=0x7f8db4146050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:44.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8dba785700 1 --2- 172.21.15.32:0/2720391356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db40b6960 0x7f8db4146050 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f8dac0032b0 tx=0x7f8dac009270).stop 2020-12-05T17:35:44.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8dba785700 1 -- 172.21.15.32:0/2720391356 shutdown_connections 2020-12-05T17:35:44.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8dba785700 1 --2- 172.21.15.32:0/2720391356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db40b6960 0x7f8db4146050 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:44.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8dba785700 1 --2- 172.21.15.32:0/2720391356 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8db405cd70 0x7f8db40b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:44.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8dba785700 1 -- 172.21.15.32:0/2720391356 >> 172.21.15.32:0/2720391356 conn(0x7f8db4069cb0 msgr2=0x7f8db40604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:44.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8dba785700 1 -- 172.21.15.32:0/2720391356 shutdown_connections 2020-12-05T17:35:44.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.767+0000 7f8dba785700 1 -- 172.21.15.32:0/2720391356 wait complete. 2020-12-05T17:35:44.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.771+0000 7f8dba785700 1 Processor -- start 2020-12-05T17:35:44.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.771+0000 7f8dba785700 1 -- start start 2020-12-05T17:35:44.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.775+0000 7f8dba785700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8db405cd70 0x7f8db4147270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:44.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.775+0000 7f8dba785700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8db40b0770 0x7f8db4065710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:44.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.775+0000 7f8dba785700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db40b6960 0x7f8db4145cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.775+0000 7f8dba785700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db4113950 con 0x7f8db40b6960 2020-12-05T17:35:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.775+0000 7f8dba785700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db4149870 con 0x7f8db405cd70 2020-12-05T17:35:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.775+0000 7f8dba785700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db4149b10 con 0x7f8db40b0770 2020-12-05T17:35:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.775+0000 7f8db8d22700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db40b6960 0x7f8db4145cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.775+0000 7f8db8d22700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db40b6960 0x7f8db4145cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35512/0 (socket says 172.21.15.32:35512) 2020-12-05T17:35:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.775+0000 7f8db8d22700 1 -- 172.21.15.32:0/1150520153 learned_addr learned my addr 172.21.15.32:0/1150520153 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.775+0000 7f8dabfff700 1 --2- 172.21.15.32:0/1150520153 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8db40b0770 0x7f8db4065710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.775+0000 7f8db3fff700 1 --2- 172.21.15.32:0/1150520153 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8db405cd70 0x7f8db4147270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.775+0000 7f8db8d22700 1 -- 172.21.15.32:0/1150520153 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8db40b0770 msgr2=0x7f8db4065710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:44.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.775+0000 7f8db8d22700 1 --2- 172.21.15.32:0/1150520153 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8db40b0770 0x7f8db4065710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:44.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.775+0000 7f8db8d22700 1 -- 172.21.15.32:0/1150520153 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8db405cd70 msgr2=0x7f8db4147270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:44.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.775+0000 7f8db8d22700 1 --2- 172.21.15.32:0/1150520153 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8db405cd70 0x7f8db4147270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:44.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.775+0000 7f8db8d22700 1 -- 172.21.15.32:0/1150520153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db41514c0 con 0x7f8db40b6960 2020-12-05T17:35:44.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.775+0000 7f8db8d22700 1 --2- 172.21.15.32:0/1150520153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db40b6960 0x7f8db4145cf0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f8dac0032b0 tx=0x7f8dac016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.775+0000 7f8db1ffb700 1 -- 172.21.15.32:0/1150520153 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8dac01f3f0 con 0x7f8db40b6960 2020-12-05T17:35:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.775+0000 7f8db1ffb700 1 -- 172.21.15.32:0/1150520153 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dac01f550 con 0x7f8db40b6960 2020-12-05T17:35:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.775+0000 7f8dba785700 1 -- 172.21.15.32:0/1150520153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db41516b0 con 0x7f8db40b6960 2020-12-05T17:35:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.775+0000 7f8db1ffb700 1 -- 172.21.15.32:0/1150520153 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8dac01f8b0 con 0x7f8db40b6960 2020-12-05T17:35:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.775+0000 7f8dba785700 1 -- 172.21.15.32:0/1150520153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db4113570 con 0x7f8db40b6960 2020-12-05T17:35:44.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.775+0000 7f8dba785700 1 -- 172.21.15.32:0/1150520153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db4113790 con 0x7f8db40b6960 2020-12-05T17:35:44.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.779+0000 7f8db1ffb700 1 -- 172.21.15.32:0/1150520153 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8dac00f070 con 0x7f8db40b6960 2020-12-05T17:35:44.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.779+0000 7f8db1ffb700 1 --2- 172.21.15.32:0/1150520153 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8d98055490 0x7f8d98057950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:44.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.779+0000 7f8db1ffb700 1 -- 172.21.15.32:0/1150520153 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8dac06e9d0 con 0x7f8db40b6960 2020-12-05T17:35:44.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.779+0000 7f8db3fff700 1 --2- 172.21.15.32:0/1150520153 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8d98055490 0x7f8d98057950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:44.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.779+0000 7f8db3fff700 1 --2- 172.21.15.32:0/1150520153 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8d98055490 0x7f8d98057950 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f8d9c00b000 tx=0x7f8d9c006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.787+0000 7f8db1ffb700 1 -- 172.21.15.32:0/1150520153 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8dac044b70 con 0x7f8db40b6960 2020-12-05T17:35:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.975+0000 7f8dba785700 1 -- 172.21.15.32:0/1150520153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8db4149cd0 con 0x7f8db40b6960 2020-12-05T17:35:44.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.975+0000 7f8db1ffb700 1 -- 172.21.15.32:0/1150520153 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2136 (secure 0 0 0) 0x7f8dac015070 con 0x7f8db40b6960 2020-12-05T17:35:44.974 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:35:44.974 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5159,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/2123765277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2123765277},{"type":"v1","addr":"172.21.15.61:6833","nonce":2123765277}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:35:23.386407+0000","modified":"2020-12-05T17:35:24.389244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5640},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5640":{"gid":5640,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/681412338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":681412338},{"type":"v1","addr":"172.21.15.32:6835","nonce":681412338}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-12-05T17:35:44.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.979+0000 7f8dba785700 1 -- 172.21.15.32:0/1150520153 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8d98055490 msgr2=0x7f8d98057950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:44.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.979+0000 7f8dba785700 1 --2- 172.21.15.32:0/1150520153 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8d98055490 0x7f8d98057950 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f8d9c00b000 tx=0x7f8d9c006dc0).stop 2020-12-05T17:35:44.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.979+0000 7f8dba785700 1 -- 172.21.15.32:0/1150520153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db40b6960 msgr2=0x7f8db4145cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:44.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.979+0000 7f8dba785700 1 --2- 172.21.15.32:0/1150520153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db40b6960 0x7f8db4145cf0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f8dac0032b0 tx=0x7f8dac016000).stop 2020-12-05T17:35:44.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.979+0000 7f8dba785700 1 -- 172.21.15.32:0/1150520153 shutdown_connections 2020-12-05T17:35:44.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.979+0000 7f8dba785700 1 --2- 172.21.15.32:0/1150520153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db40b6960 0x7f8db4145cf0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.979+0000 7f8dba785700 1 --2- 172.21.15.32:0/1150520153 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8db405cd70 0x7f8db4147270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.979+0000 7f8dba785700 1 --2- 172.21.15.32:0/1150520153 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8d98055490 0x7f8d98057950 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.979+0000 7f8dba785700 1 --2- 172.21.15.32:0/1150520153 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8db40b0770 0x7f8db4065710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.979+0000 7f8dba785700 1 -- 172.21.15.32:0/1150520153 >> 172.21.15.32:0/1150520153 conn(0x7f8db4069cb0 msgr2=0x7f8db4143440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.979+0000 7f8dba785700 1 -- 172.21.15.32:0/1150520153 shutdown_connections 2020-12-05T17:35:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:44.979+0000 7f8dba785700 1 -- 172.21.15.32:0/1150520153 wait complete. 2020-12-05T17:35:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 23 2020-12-05T17:35:44.989 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell mds.a cache drop 5 2020-12-05T17:35:45.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.107+0000 7f6dcee55700 1 Processor -- start 2020-12-05T17:35:45.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.107+0000 7f6dcee55700 1 -- start start 2020-12-05T17:35:45.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.107+0000 7f6dcee55700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6dc8061ba0 0x7f6dc8066e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:45.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.107+0000 7f6dcee55700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f6dc80b3020 0x7f6dc8143c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:45.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.107+0000 7f6dcee55700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6dc80625c0 0x7f6dc8144160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:45.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.107+0000 7f6dcee55700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc80b3540 con 0x7f6dc80625c0 2020-12-05T17:35:45.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.107+0000 7f6dcee55700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc8061f80 con 0x7f6dc80b3020 2020-12-05T17:35:45.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.107+0000 7f6dcee55700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc8144650 con 0x7f6dc8061ba0 2020-12-05T17:35:45.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.107+0000 7f6dce654700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6dc80625c0 0x7f6dc8144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:45.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.107+0000 7f6dce654700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6dc80625c0 0x7f6dc8144160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35520/0 (socket says 172.21.15.32:35520) 2020-12-05T17:35:45.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.107+0000 7f6dce654700 1 -- 172.21.15.32:0/2687415925 learned_addr learned my addr 172.21.15.32:0/2687415925 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:45.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.107+0000 7f6dcde53700 1 --2- 172.21.15.32:0/2687415925 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6dc8061ba0 0x7f6dc8066e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.107+0000 7f6dcd652700 1 --2- 172.21.15.32:0/2687415925 >> v2:172.21.15.61:3300/0 conn(0x7f6dc80b3020 0x7f6dc8143c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dce654700 1 -- 172.21.15.32:0/2687415925 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6dc8061ba0 msgr2=0x7f6dc8066e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dce654700 1 --2- 172.21.15.32:0/2687415925 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6dc8061ba0 0x7f6dc8066e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dce654700 1 -- 172.21.15.32:0/2687415925 >> v2:172.21.15.61:3300/0 conn(0x7f6dc80b3020 msgr2=0x7f6dc8143c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:45.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dce654700 1 --2- 172.21.15.32:0/2687415925 >> v2:172.21.15.61:3300/0 conn(0x7f6dc80b3020 0x7f6dc8143c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dce654700 1 -- 172.21.15.32:0/2687415925 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc8066750 con 0x7f6dc80625c0 2020-12-05T17:35:45.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dcd652700 1 --2- 172.21.15.32:0/2687415925 >> v2:172.21.15.61:3300/0 conn(0x7f6dc80b3020 0x7f6dc8143c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:45.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dce654700 1 --2- 172.21.15.32:0/2687415925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc80625c0 0x7f6dc8144160 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f6db8002f30 tx=0x7f6db8006e80).ready entity=mon.0 client_cookie=ed341cc0bbca9697 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:45.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dcce51700 1 -- 172.21.15.32:0/2687415925 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6db80097f0 con 0x7f6dc80625c0 2020-12-05T17:35:45.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dcce51700 1 -- 172.21.15.32:0/2687415925 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db8009950 con 0x7f6dc80625c0 2020-12-05T17:35:45.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dcce51700 1 -- 172.21.15.32:0/2687415925 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6db8009cb0 con 0x7f6dc80625c0 2020-12-05T17:35:45.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dcee55700 1 -- 172.21.15.32:0/2687415925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc80625c0 msgr2=0x7f6dc8144160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:45.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dcee55700 1 --2- 172.21.15.32:0/2687415925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc80625c0 0x7f6dc8144160 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f6db8002f30 tx=0x7f6db8006e80).stop 2020-12-05T17:35:45.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dcee55700 1 -- 172.21.15.32:0/2687415925 shutdown_connections 2020-12-05T17:35:45.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dcee55700 1 --2- 172.21.15.32:0/2687415925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc80625c0 0x7f6dc8144160 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dcee55700 1 --2- 172.21.15.32:0/2687415925 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6dc8061ba0 0x7f6dc8066e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dcee55700 1 --2- 172.21.15.32:0/2687415925 >> v2:172.21.15.61:3300/0 conn(0x7f6dc80b3020 0x7f6dc8143c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dcee55700 1 -- 172.21.15.32:0/2687415925 >> 172.21.15.32:0/2687415925 conn(0x7f6dc80b6d10 msgr2=0x7f6dc80b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:45.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dcee55700 1 -- 172.21.15.32:0/2687415925 shutdown_connections 2020-12-05T17:35:45.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dcee55700 1 -- 172.21.15.32:0/2687415925 wait complete. 2020-12-05T17:35:45.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dcee55700 1 Processor -- start 2020-12-05T17:35:45.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dcee55700 1 -- start start 2020-12-05T17:35:45.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dcee55700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc8061ba0 0x7f6dc8151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:45.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dcee55700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6dc80625c0 0x7f6dc8151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:45.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dcee55700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6dc80b3020 0x7f6dc810ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:45.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dcee55700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc8111990 con 0x7f6dc8061ba0 2020-12-05T17:35:45.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dcee55700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc8063a10 con 0x7f6dc80625c0 2020-12-05T17:35:45.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dcee55700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc8111bf0 con 0x7f6dc80b3020 2020-12-05T17:35:45.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dcde53700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc8061ba0 0x7f6dc8151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:45.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.111+0000 7f6dcde53700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc8061ba0 0x7f6dc8151910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35522/0 (socket says 172.21.15.32:35522) 2020-12-05T17:35:45.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.115+0000 7f6dcde53700 1 -- 172.21.15.32:0/1224129483 learned_addr learned my addr 172.21.15.32:0/1224129483 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:45.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.115+0000 7f6dcd652700 1 --2- 172.21.15.32:0/1224129483 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6dc80625c0 0x7f6dc8151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:45.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.115+0000 7f6dce654700 1 --2- 172.21.15.32:0/1224129483 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6dc80b3020 0x7f6dc810ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:45.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.115+0000 7f6dcde53700 1 -- 172.21.15.32:0/1224129483 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6dc80b3020 msgr2=0x7f6dc810ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:45.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.115+0000 7f6dcde53700 1 --2- 172.21.15.32:0/1224129483 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6dc80b3020 0x7f6dc810ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.115+0000 7f6dcde53700 1 -- 172.21.15.32:0/1224129483 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6dc80625c0 msgr2=0x7f6dc8151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:45.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.115+0000 7f6dcde53700 1 --2- 172.21.15.32:0/1224129483 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6dc80625c0 0x7f6dc8151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.115+0000 7f6dcde53700 1 -- 172.21.15.32:0/1224129483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc80b2170 con 0x7f6dc8061ba0 2020-12-05T17:35:45.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.115+0000 7f6dcde53700 1 --2- 172.21.15.32:0/1224129483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc8061ba0 0x7f6dc8151910 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f6dc00033f0 tx=0x7f6dc0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:45.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.115+0000 7f6dc6ffd700 1 -- 172.21.15.32:0/1224129483 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6dc000bad0 con 0x7f6dc8061ba0 2020-12-05T17:35:45.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.115+0000 7f6dc6ffd700 1 -- 172.21.15.32:0/1224129483 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc000bc30 con 0x7f6dc8061ba0 2020-12-05T17:35:45.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.115+0000 7f6dcee55700 1 -- 172.21.15.32:0/1224129483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc8152490 con 0x7f6dc8061ba0 2020-12-05T17:35:45.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.115+0000 7f6dc6ffd700 1 -- 172.21.15.32:0/1224129483 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6dc0012420 con 0x7f6dc8061ba0 2020-12-05T17:35:45.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.115+0000 7f6dcee55700 1 -- 172.21.15.32:0/1224129483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc81103f0 con 0x7f6dc8061ba0 2020-12-05T17:35:45.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.119+0000 7f6dc6ffd700 1 -- 172.21.15.32:0/1224129483 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6dc0012580 con 0x7f6dc8061ba0 2020-12-05T17:35:45.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.119+0000 7f6dc6ffd700 1 --2- 172.21.15.32:0/1224129483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6da80554e0 0x7f6da80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:45.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.119+0000 7f6dc6ffd700 1 -- 172.21.15.32:0/1224129483 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6dc006e730 con 0x7f6dc8061ba0 2020-12-05T17:35:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.119+0000 7f6dcd652700 1 --2- 172.21.15.32:0/1224129483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6da80554e0 0x7f6da80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.119+0000 7f6dcd652700 1 --2- 172.21.15.32:0/1224129483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6da80554e0 0x7f6da80579a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f6dbc00c000 tx=0x7f6dbc008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.119+0000 7f6ddea52740 1 Processor -- start 2020-12-05T17:35:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.119+0000 7f6ddea52740 1 -- start start 2020-12-05T17:35:45.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6ddea52740 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1d65080 0x1daf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:45.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6ddea52740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1c4deb0 0x1db1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:45.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6ddea52740 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1bcd820 0x1d93c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:45.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6ddea52740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cda010 con 0x1c4deb0 2020-12-05T17:35:45.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6ddea52740 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x1be66b0 con 0x1d65080 2020-12-05T17:35:45.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6ddea52740 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b94430 con 0x1bcd820 2020-12-05T17:35:45.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6dcd652700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1c4deb0 0x1db1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:45.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6dcd652700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1c4deb0 0x1db1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35532/0 (socket says 172.21.15.32:35532) 2020-12-05T17:35:45.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6dcd652700 1 -- 172.21.15.32:0/1668292571 learned_addr learned my addr 172.21.15.32:0/1668292571 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:45.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6dcde53700 1 --2- 172.21.15.32:0/1668292571 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1d65080 0x1daf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:45.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6dce654700 1 --2- 172.21.15.32:0/1668292571 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1bcd820 0x1d93c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:45.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6dcd652700 1 -- 172.21.15.32:0/1668292571 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1bcd820 msgr2=0x1d93c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:45.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6dcd652700 1 --2- 172.21.15.32:0/1668292571 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1bcd820 0x1d93c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6dcd652700 1 -- 172.21.15.32:0/1668292571 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1d65080 msgr2=0x1daf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:45.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6dcd652700 1 --2- 172.21.15.32:0/1668292571 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1d65080 0x1daf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6dcd652700 1 -- 172.21.15.32:0/1668292571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1cfaf30 con 0x1c4deb0 2020-12-05T17:35:45.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6dcd652700 1 --2- 172.21.15.32:0/1668292571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1c4deb0 0x1db1b60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x1db10f0 tx=0x7f6dbc015000).ready entity=mon.0 client_cookie=ab9ad5c9d62ae79b server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:45.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6dcee55700 1 -- 172.21.15.32:0/1668292571 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6dbc010ed0 con 0x1c4deb0 2020-12-05T17:35:45.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6dcee55700 1 -- 172.21.15.32:0/1668292571 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dbc01f400 con 0x1c4deb0 2020-12-05T17:35:45.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6dcee55700 1 -- 172.21.15.32:0/1668292571 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6dbc01f7a0 con 0x1c4deb0 2020-12-05T17:35:45.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6ddea52740 1 -- 172.21.15.32:0/1668292571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1c4deb0 msgr2=0x1db1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:45.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6ddea52740 1 --2- 172.21.15.32:0/1668292571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1c4deb0 0x1db1b60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x1db10f0 tx=0x7f6dbc015000).stop 2020-12-05T17:35:45.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6ddea52740 1 -- 172.21.15.32:0/1668292571 shutdown_connections 2020-12-05T17:35:45.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6ddea52740 1 --2- 172.21.15.32:0/1668292571 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1bcd820 0x1d93c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6ddea52740 1 --2- 172.21.15.32:0/1668292571 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1d65080 0x1daf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6ddea52740 1 --2- 172.21.15.32:0/1668292571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1c4deb0 0x1db1b60 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.123+0000 7f6ddea52740 1 -- 172.21.15.32:0/1668292571 >> 172.21.15.32:0/1668292571 conn(0x1d94c60 msgr2=0x1db0140 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:45.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6ddea52740 1 -- 172.21.15.32:0/1668292571 shutdown_connections 2020-12-05T17:35:45.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6ddea52740 1 -- 172.21.15.32:0/1668292571 wait complete. 2020-12-05T17:35:45.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6ddea52740 1 -- start start 2020-12-05T17:35:45.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6ddea52740 1 Processor -- start 2020-12-05T17:35:45.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6ddea52740 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1bcd820 0x1db3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:45.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6ddea52740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1c4deb0 0x1de5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:45.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6ddea52740 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1d65080 0x1db4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:45.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6ddea52740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d09c90 con 0x1c4deb0 2020-12-05T17:35:45.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6ddea52740 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d0b260 con 0x1d65080 2020-12-05T17:35:45.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6ddea52740 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x1cec730 con 0x1bcd820 2020-12-05T17:35:45.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6ddea52740 10 client.0 ll_register_callbacks cb 0x1d94bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-12-05T17:35:45.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6dcd652700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1c4deb0 0x1de5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:45.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6dcd652700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1c4deb0 0x1de5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35538/0 (socket says 172.21.15.32:35538) 2020-12-05T17:35:45.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6dcd652700 1 -- 172.21.15.32:0/198434819 learned_addr learned my addr 172.21.15.32:0/198434819 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:45.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6dce654700 1 --2- 172.21.15.32:0/198434819 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1d65080 0x1db4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:45.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6dcde53700 1 --2- 172.21.15.32:0/198434819 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1bcd820 0x1db3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:45.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6dcd652700 1 -- 172.21.15.32:0/198434819 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1bcd820 msgr2=0x1db3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:45.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6dcd652700 1 --2- 172.21.15.32:0/198434819 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1bcd820 0x1db3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6dcd652700 1 -- 172.21.15.32:0/198434819 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1d65080 msgr2=0x1db4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:45.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6dcd652700 1 --2- 172.21.15.32:0/198434819 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1d65080 0x1db4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6dcd652700 1 -- 172.21.15.32:0/198434819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1d03460 con 0x1c4deb0 2020-12-05T17:35:45.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6dcde53700 1 --2- 172.21.15.32:0/198434819 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1bcd820 0x1db3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:35:45.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6dcd652700 1 --2- 172.21.15.32:0/198434819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1c4deb0 0x1de5360 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x1db10f0 tx=0x7f6dbc022000).ready entity=mon.0 client_cookie=5b6f84447eecadb0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:45.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6da4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2020-12-05T17:35:45.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6da4ff9700 1 -- 172.21.15.32:0/198434819 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6dbc029810 con 0x1c4deb0 2020-12-05T17:35:45.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6da4ff9700 1 -- 172.21.15.32:0/198434819 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6dbc029970 con 0x1c4deb0 2020-12-05T17:35:45.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.127+0000 7f6ddea52740 1 -- 172.21.15.32:0/198434819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1bf28d0 con 0x1c4deb0 2020-12-05T17:35:45.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6da4ff9700 1 -- 172.21.15.32:0/198434819 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dbc010ed0 con 0x1c4deb0 2020-12-05T17:35:45.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6ddea52740 10 client.5844 fetch_fsmap learned FSMap version 23 2020-12-05T17:35:45.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6ddea52740 1 -- 172.21.15.32:0/198434819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x1b7d790 con 0x1c4deb0 2020-12-05T17:35:45.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6da4ff9700 1 -- 172.21.15.32:0/198434819 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1bf28d0 con 0x1c4deb0 2020-12-05T17:35:45.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6da4ff9700 1 -- 172.21.15.32:0/198434819 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 23) v1 ==== 1247+0+0 (secure 0 0 0) 0x7f6dbc012040 con 0x1c4deb0 2020-12-05T17:35:45.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6ddea52740 10 client.5844 fetch_fsmap finished waiting for FSMap version 23 2020-12-05T17:35:45.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6ddea52740 10 client.5844 resolve_mds: resolved ID 'a' to GID 5640 2020-12-05T17:35:45.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6ddea52740 20 client.5844 populate_metadata read hostname 'smithi032' 2020-12-05T17:35:45.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6ddea52740 1 --2- 172.21.15.32:0/198434819 >> [v2:172.21.15.32:6834/681412338,v1:172.21.15.32:6835/681412338] conn(0x1b5fc20 0x1df5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:45.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6ddea52740 4 client.5844 mds_command: new command op to 5640 tid=0[{"prefix": "get_command_descriptions"}] 2020-12-05T17:35:45.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6ddea52740 1 -- 172.21.15.32:0/198434819 --> [v2:172.21.15.32:6834/681412338,v1:172.21.15.32:6835/681412338] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1b5f170 con 0x1b5fc20 2020-12-05T17:35:45.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6dcde53700 1 --2- 172.21.15.32:0/198434819 >> [v2:172.21.15.32:6834/681412338,v1:172.21.15.32:6835/681412338] conn(0x1b5fc20 0x1df5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:45.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6da4ff9700 10 client.5844 ms_handle_connect on v2:172.21.15.32:6834/681412338 2020-12-05T17:35:45.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6dcde53700 1 --2- 172.21.15.32:0/198434819 >> [v2:172.21.15.32:6834/681412338,v1:172.21.15.32:6835/681412338] conn(0x1b5fc20 0x1df5ce0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=58915b2a7bac7b3b server_cookie=42bd3f75222e81e0 in_seq=0 out_seq=0 2020-12-05T17:35:45.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6da4ff9700 1 -- 172.21.15.32:0/198434819 <== mds.0 v2:172.21.15.32:6834/681412338 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16606 (crc 0 0 0) 0x1b5f170 con 0x1b5fc20 2020-12-05T17:35:45.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6da4ff9700 10 client.5844 handle_command_reply: tid=0 2020-12-05T17:35:45.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6ddea52740 1 client.5844 shutdown 2020-12-05T17:35:45.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6ddea52740 1 -- 172.21.15.32:0/198434819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1c4deb0 msgr2=0x1de5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:45.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6ddea52740 1 --2- 172.21.15.32:0/198434819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1c4deb0 0x1de5360 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x1db10f0 tx=0x7f6dbc022000).stop 2020-12-05T17:35:45.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6ddea52740 1 -- 172.21.15.32:0/198434819 shutdown_connections 2020-12-05T17:35:45.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6ddea52740 1 --2- 172.21.15.32:0/198434819 >> [v2:172.21.15.32:6834/681412338,v1:172.21.15.32:6835/681412338] conn(0x1b5fc20 0x1df5ce0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6ddea52740 1 --2- 172.21.15.32:0/198434819 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1d65080 0x1db4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6ddea52740 1 --2- 172.21.15.32:0/198434819 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1bcd820 0x1db3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6ddea52740 1 --2- 172.21.15.32:0/198434819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1c4deb0 0x1de5360 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6ddea52740 1 -- 172.21.15.32:0/198434819 >> 172.21.15.32:0/198434819 conn(0x1d94c60 msgr2=0x1de2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:45.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6da4ff9700 0 client.5844 ms_handle_reset on v2:172.21.15.32:6834/681412338 2020-12-05T17:35:45.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6ddea52740 1 -- 172.21.15.32:0/198434819 shutdown_connections 2020-12-05T17:35:45.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6ddea52740 1 -- 172.21.15.32:0/198434819 wait complete. 2020-12-05T17:35:45.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.131+0000 7f6ddea52740 20 client.5844 trim_cache size 0 max 16384 2020-12-05T17:35:45.154 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:35:45.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.143+0000 7f6ddea52740 1 Processor -- start 2020-12-05T17:35:45.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.143+0000 7f6ddea52740 1 -- start start 2020-12-05T17:35:45.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.143+0000 7f6ddea52740 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1bcd820 0x1dae210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:45.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.143+0000 7f6ddea52740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1c4deb0 0x1dab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:45.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.143+0000 7f6ddea52740 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1d65080 0x1cee730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:45.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.143+0000 7f6ddea52740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x1db0e80 con 0x1c4deb0 2020-12-05T17:35:45.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.143+0000 7f6ddea52740 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x1de1340 con 0x1d65080 2020-12-05T17:35:45.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.143+0000 7f6ddea52740 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x1dabfe0 con 0x1bcd820 2020-12-05T17:35:45.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.143+0000 7f6dcd652700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1c4deb0 0x1dab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:45.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.143+0000 7f6dcd652700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1c4deb0 0x1dab5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35544/0 (socket says 172.21.15.32:35544) 2020-12-05T17:35:45.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.143+0000 7f6dcd652700 1 -- 172.21.15.32:0/3413645980 learned_addr learned my addr 172.21.15.32:0/3413645980 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:45.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.143+0000 7f6dcde53700 1 --2- 172.21.15.32:0/3413645980 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1bcd820 0x1dae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:45.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.143+0000 7f6dce654700 1 --2- 172.21.15.32:0/3413645980 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1d65080 0x1cee730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:45.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6dcd652700 1 -- 172.21.15.32:0/3413645980 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1bcd820 msgr2=0x1dae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:45.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6dcd652700 1 --2- 172.21.15.32:0/3413645980 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1bcd820 0x1dae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6dcd652700 1 -- 172.21.15.32:0/3413645980 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1d65080 msgr2=0x1cee730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:45.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6dcd652700 1 --2- 172.21.15.32:0/3413645980 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1d65080 0x1cee730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6dcd652700 1 -- 172.21.15.32:0/3413645980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc8140c40 con 0x1c4deb0 2020-12-05T17:35:45.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6dcde53700 1 --2- 172.21.15.32:0/3413645980 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1bcd820 0x1dae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:35:45.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6dcd652700 1 --2- 172.21.15.32:0/3413645980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1c4deb0 0x1dab5a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x1d957e0 tx=0x7f6dbc029000).ready entity=mon.0 client_cookie=896faa81de8a2819 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:45.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6da6ffd700 1 -- 172.21.15.32:0/3413645980 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6dbc031590 con 0x1c4deb0 2020-12-05T17:35:45.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6da6ffd700 1 -- 172.21.15.32:0/3413645980 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dbc010ed0 con 0x1c4deb0 2020-12-05T17:35:45.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6da6ffd700 1 -- 172.21.15.32:0/3413645980 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6dbc031ab0 con 0x1c4deb0 2020-12-05T17:35:45.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6ddea52740 1 -- 172.21.15.32:0/3413645980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1c4deb0 msgr2=0x1dab5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:45.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6ddea52740 1 --2- 172.21.15.32:0/3413645980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1c4deb0 0x1dab5a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x1d957e0 tx=0x7f6dbc029000).stop 2020-12-05T17:35:45.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6ddea52740 1 -- 172.21.15.32:0/3413645980 shutdown_connections 2020-12-05T17:35:45.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6ddea52740 1 --2- 172.21.15.32:0/3413645980 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1d65080 0x1cee730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6ddea52740 1 --2- 172.21.15.32:0/3413645980 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1bcd820 0x1dae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6ddea52740 1 --2- 172.21.15.32:0/3413645980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1c4deb0 0x1dab5a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6ddea52740 1 -- 172.21.15.32:0/3413645980 >> 172.21.15.32:0/3413645980 conn(0x1d94c60 msgr2=0x1de5130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:45.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6ddea52740 1 -- 172.21.15.32:0/3413645980 shutdown_connections 2020-12-05T17:35:45.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6ddea52740 1 -- 172.21.15.32:0/3413645980 wait complete. 2020-12-05T17:35:45.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6ddea52740 1 -- start start 2020-12-05T17:35:45.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6ddea52740 1 Processor -- start 2020-12-05T17:35:45.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6ddea52740 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1bcd820 0x1dad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:45.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6ddea52740 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1c4deb0 0x1df6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:45.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6ddea52740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1d65080 0x1df6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:45.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6ddea52740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cd99a0 con 0x1d65080 2020-12-05T17:35:45.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6ddea52740 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dae4c0 con 0x1c4deb0 2020-12-05T17:35:45.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6ddea52740 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x1cd9420 con 0x1bcd820 2020-12-05T17:35:45.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6ddea52740 10 client.0 ll_register_callbacks cb 0x1de6950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-12-05T17:35:45.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6dce654700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1d65080 0x1df6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:45.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6dce654700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1d65080 0x1df6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35552/0 (socket says 172.21.15.32:35552) 2020-12-05T17:35:45.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6dce654700 1 -- 172.21.15.32:0/619425987 learned_addr learned my addr 172.21.15.32:0/619425987 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:45.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6dcde53700 1 --2- 172.21.15.32:0/619425987 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1bcd820 0x1dad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:45.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6dcd652700 1 --2- 172.21.15.32:0/619425987 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1c4deb0 0x1df6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:45.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6dce654700 1 -- 172.21.15.32:0/619425987 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1bcd820 msgr2=0x1dad4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:45.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6dce654700 1 --2- 172.21.15.32:0/619425987 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1bcd820 0x1dad4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6dce654700 1 -- 172.21.15.32:0/619425987 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1c4deb0 msgr2=0x1df6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:45.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6dce654700 1 --2- 172.21.15.32:0/619425987 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1c4deb0 0x1df6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:45.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6dce654700 1 -- 172.21.15.32:0/619425987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1b88e70 con 0x1d65080 2020-12-05T17:35:45.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6dce654700 1 --2- 172.21.15.32:0/619425987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1d65080 0x1df6bf0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x1de0880 tx=0x7f6db8015000).ready entity=mon.0 client_cookie=1e3c7f7cb2ba31cd server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:45.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6da5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2020-12-05T17:35:45.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6da5ffb700 1 -- 172.21.15.32:0/619425987 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6db8025070 con 0x1d65080 2020-12-05T17:35:45.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6da5ffb700 1 -- 172.21.15.32:0/619425987 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6db802cda0 con 0x1d65080 2020-12-05T17:35:45.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6ddea52740 1 -- 172.21.15.32:0/619425987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1c11730 con 0x1d65080 2020-12-05T17:35:45.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.147+0000 7f6da5ffb700 1 -- 172.21.15.32:0/619425987 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6db801f650 con 0x1d65080 2020-12-05T17:35:45.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.151+0000 7f6ddea52740 10 client.5850 fetch_fsmap learned FSMap version 23 2020-12-05T17:35:45.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.151+0000 7f6ddea52740 1 -- 172.21.15.32:0/619425987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x1de1a00 con 0x1d65080 2020-12-05T17:35:45.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.151+0000 7f6da5ffb700 1 -- 172.21.15.32:0/619425987 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1c11730 con 0x1d65080 2020-12-05T17:35:45.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.151+0000 7f6da5ffb700 1 -- 172.21.15.32:0/619425987 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 23) v1 ==== 1247+0+0 (secure 0 0 0) 0x7f6db8029020 con 0x1d65080 2020-12-05T17:35:45.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.151+0000 7f6ddea52740 10 client.5850 fetch_fsmap finished waiting for FSMap version 23 2020-12-05T17:35:45.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.151+0000 7f6ddea52740 10 client.5850 resolve_mds: resolved ID 'a' to GID 5640 2020-12-05T17:35:45.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.151+0000 7f6ddea52740 20 client.5850 populate_metadata read hostname 'smithi032' 2020-12-05T17:35:45.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.151+0000 7f6ddea52740 1 --2- 172.21.15.32:0/619425987 >> [v2:172.21.15.32:6834/681412338,v1:172.21.15.32:6835/681412338] conn(0x1de7a30 0x1de7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:45.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.151+0000 7f6ddea52740 4 client.5850 mds_command: new command op to 5640 tid=0[{"prefix": "cache drop", "timeout": 5}] 2020-12-05T17:35:45.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.151+0000 7f6ddea52740 1 -- 172.21.15.32:0/619425987 --> [v2:172.21.15.32:6834/681412338,v1:172.21.15.32:6835/681412338] -- command(tid 0: {"prefix": "cache drop", "timeout": 5}) v1 -- 0x1be8440 con 0x1de7a30 2020-12-05T17:35:45.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.151+0000 7f6dcde53700 1 --2- 172.21.15.32:0/619425987 >> [v2:172.21.15.32:6834/681412338,v1:172.21.15.32:6835/681412338] conn(0x1de7a30 0x1de7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:45.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.151+0000 7f6dcde53700 1 --2- 172.21.15.32:0/619425987 >> [v2:172.21.15.32:6834/681412338,v1:172.21.15.32:6835/681412338] conn(0x1de7a30 0x1de7e10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=60bad5e19175e326 server_cookie=11cdc1456f60c281 in_seq=0 out_seq=0 2020-12-05T17:35:45.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:45.151+0000 7f6da5ffb700 10 client.5850 ms_handle_connect on v2:172.21.15.32:6834/681412338 2020-12-05T17:35:45.176 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:35:45.178 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:35:45.180 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:35:45.183 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:35:50.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6da5ffb700 1 -- 172.21.15.32:0/619425987 <== mds.0 v2:172.21.15.32:6834/681412338 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+404 (crc 0 0 0) 0x1be8440 con 0x1de7a30 2020-12-05T17:35:50.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6da5ffb700 10 client.5850 handle_command_reply: tid=0 2020-12-05T17:35:50.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6ddea52740 1 client.5850 shutdown 2020-12-05T17:35:50.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6ddea52740 1 -- 172.21.15.32:0/619425987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1d65080 msgr2=0x1df6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:50.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6ddea52740 1 --2- 172.21.15.32:0/619425987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1d65080 0x1df6bf0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x1de0880 tx=0x7f6db8015000).stop 2020-12-05T17:35:50.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6ddea52740 1 -- 172.21.15.32:0/619425987 shutdown_connections 2020-12-05T17:35:50.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6ddea52740 1 --2- 172.21.15.32:0/619425987 >> [v2:172.21.15.32:6834/681412338,v1:172.21.15.32:6835/681412338] conn(0x1de7a30 0x1de7e10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:50.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6ddea52740 1 --2- 172.21.15.32:0/619425987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1d65080 0x1df6bf0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:50.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6da5ffb700 0 client.5850 ms_handle_reset on v2:172.21.15.32:6834/681412338 2020-12-05T17:35:50.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6ddea52740 1 --2- 172.21.15.32:0/619425987 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1bcd820 0x1dad4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:50.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6ddea52740 1 --2- 172.21.15.32:0/619425987 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1c4deb0 0x1df6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:50.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6ddea52740 1 -- 172.21.15.32:0/619425987 >> 172.21.15.32:0/619425987 conn(0x1d94c60 msgr2=0x1db3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:50.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6ddea52740 1 -- 172.21.15.32:0/619425987 shutdown_connections 2020-12-05T17:35:50.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6ddea52740 1 -- 172.21.15.32:0/619425987 wait complete. 2020-12-05T17:35:50.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6ddea52740 20 client.5850 trim_cache size 0 max 16384 2020-12-05T17:35:50.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6dcee55700 1 -- 172.21.15.32:0/1224129483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6da80554e0 msgr2=0x7f6da80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:50.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6dcee55700 1 --2- 172.21.15.32:0/1224129483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6da80554e0 0x7f6da80579a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f6dbc00c000 tx=0x7f6dbc008040).stop 2020-12-05T17:35:50.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6dcee55700 1 -- 172.21.15.32:0/1224129483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc8061ba0 msgr2=0x7f6dc8151910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:50.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6dcee55700 1 --2- 172.21.15.32:0/1224129483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc8061ba0 0x7f6dc8151910 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f6dc00033f0 tx=0x7f6dc0008040).stop 2020-12-05T17:35:50.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6dcee55700 1 -- 172.21.15.32:0/1224129483 shutdown_connections 2020-12-05T17:35:50.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6dcee55700 1 --2- 172.21.15.32:0/1224129483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6da80554e0 0x7f6da80579a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:50.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6dcee55700 1 --2- 172.21.15.32:0/1224129483 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6dc80b3020 0x7f6dc810ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:50.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6dcee55700 1 --2- 172.21.15.32:0/1224129483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc8061ba0 0x7f6dc8151910 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:50.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6dcee55700 1 --2- 172.21.15.32:0/1224129483 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6dc80625c0 0x7f6dc8151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:50.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6dcee55700 1 -- 172.21.15.32:0/1224129483 >> 172.21.15.32:0/1224129483 conn(0x7f6dc80b6d10 msgr2=0x7f6dc805fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:50.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6dcee55700 1 -- 172.21.15.32:0/1224129483 shutdown_connections 2020-12-05T17:35:50.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.403+0000 7f6dcee55700 1 -- 172.21.15.32:0/1224129483 wait complete. 2020-12-05T17:35:50.412 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:35:50.412 INFO:teuthology.orchestra.run.smithi032.stdout: "client_recall": { 2020-12-05T17:35:50.413 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": -110, 2020-12-05T17:35:50.413 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "(110) Connection timed out", 2020-12-05T17:35:50.413 INFO:teuthology.orchestra.run.smithi032.stdout: "recalled": 120 2020-12-05T17:35:50.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:50.414 INFO:teuthology.orchestra.run.smithi032.stdout: "flush_journal": { 2020-12-05T17:35:50.414 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2020-12-05T17:35:50.414 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "" 2020-12-05T17:35:50.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:50.414 INFO:teuthology.orchestra.run.smithi032.stdout: "trim_cache": { 2020-12-05T17:35:50.415 INFO:teuthology.orchestra.run.smithi032.stdout: "trimmed": 0 2020-12-05T17:35:50.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:50.415 INFO:teuthology.orchestra.run.smithi032.stdout: "cache": { 2020-12-05T17:35:50.415 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": { 2020-12-05T17:35:50.415 INFO:teuthology.orchestra.run.smithi032.stdout: "items": 19930, 2020-12-05T17:35:50.416 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes": 3095168 2020-12-05T17:35:50.416 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:35:50.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:35:50.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2515731849999998 2020-12-05T17:35:50.417 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:35:50.418 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2020-12-05T17:35:50.418 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:50.419 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:35:50.419 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:35:50.426 INFO:teuthology.orchestra.run.smithi036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2020-12-05T17:35:50.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:35:50.428 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2020-12-05T17:35:50.428 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi036.front.sepia.ceph.com... 2020-12-05T17:35:50.428 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:35:50.429 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:35:50.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:50.512 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:35:50.512 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:35:50.560 INFO:teuthology.orchestra.run.smithi036.stdout:ext2/ext3 2020-12-05T17:35:50.561 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:35:50.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:35:50.562 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:35:50.613 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:35:50.729 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.731+0000 7f07397be700 1 Processor -- start 2020-12-05T17:35:50.730 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.731+0000 7f07397be700 1 -- start start 2020-12-05T17:35:50.731 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.731+0000 7f07397be700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0734145b60 0x7f073414df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:50.731 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.731+0000 7f07397be700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0734061b30 con 0x7f07340ae750 2020-12-05T17:35:50.731 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.731+0000 7f07397be700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0734061d70 con 0x7f07340b7580 2020-12-05T17:35:50.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.731+0000 7f07397be700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0734061890 con 0x7f0734145b60 2020-12-05T17:35:50.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.731+0000 7f07337fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f07340ae750 0x7f07340b1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38214/0 (socket says 172.21.15.32:38214) 2020-12-05T17:35:50.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.731+0000 7f07337fe700 1 -- 172.21.15.32:0/2389376834 learned_addr learned my addr 172.21.15.32:0/2389376834 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:35:50.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.731+0000 7f0738fbd700 1 --2- 172.21.15.32:0/2389376834 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0734145b60 0x7f073414df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:50.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.731+0000 7f0732ffd700 1 -- 172.21.15.32:0/2389376834 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765089986 0 0) 0x7f0734061b30 con 0x7f07340ae750 2020-12-05T17:35:50.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.731+0000 7f0732ffd700 1 -- 172.21.15.32:0/2389376834 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0714003230 con 0x7f07340ae750 2020-12-05T17:35:50.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.731+0000 7f0732ffd700 1 -- 172.21.15.32:0/2389376834 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2500070747 0 0) 0x7f0734061d70 con 0x7f07340b7580 2020-12-05T17:35:50.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.731+0000 7f0732ffd700 1 -- 172.21.15.32:0/2389376834 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0734061b30 con 0x7f07340b7580 2020-12-05T17:35:50.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.731+0000 7f0732ffd700 1 -- 172.21.15.32:0/2389376834 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2945267337 0 0) 0x7f0714003230 con 0x7f07340ae750 2020-12-05T17:35:50.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.731+0000 7f0732ffd700 1 -- 172.21.15.32:0/2389376834 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0734145b60 msgr2=0x7f073414df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:50.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.731+0000 7f0732ffd700 1 --2- 172.21.15.32:0/2389376834 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0734145b60 0x7f073414df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:50.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.731+0000 7f0732ffd700 1 -- 172.21.15.32:0/2389376834 >> v1:172.21.15.61:6789/0 conn(0x7f07340b7580 legacy=0x7f073413f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:50.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.731+0000 7f0732ffd700 1 -- 172.21.15.32:0/2389376834 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07340b13e0 con 0x7f07340ae750 2020-12-05T17:35:50.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f0732ffd700 1 -- 172.21.15.32:0/2389376834 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f071c001a50 con 0x7f07340ae750 2020-12-05T17:35:50.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f0732ffd700 1 -- 172.21.15.32:0/2389376834 >> v1:172.21.15.32:6789/0 conn(0x7f07340ae750 legacy=0x7f07340b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:50.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f0732ffd700 1 --2- 172.21.15.32:0/2389376834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07140038e0 0x7f0714003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:50.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f0732ffd700 1 --2- 172.21.15.32:0/2389376834 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0734145b60 0x7f07140044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:50.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f0732ffd700 1 --2- 172.21.15.32:0/2389376834 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07140049c0 0x7f0714004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:50.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f0732ffd700 1 -- 172.21.15.32:0/2389376834 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0734061890 con 0x7f07140038e0 2020-12-05T17:35:50.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f0732ffd700 1 -- 172.21.15.32:0/2389376834 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0714005330 con 0x7f07140049c0 2020-12-05T17:35:50.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f0732ffd700 1 -- 172.21.15.32:0/2389376834 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071400b750 con 0x7f0734145b60 2020-12-05T17:35:50.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f0733fff700 1 --2- 172.21.15.32:0/2389376834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07140038e0 0x7f0714003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:50.740 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f0738fbd700 1 --2- 172.21.15.32:0/2389376834 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07140049c0 0x7f0714004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:50.741 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f07337fe700 1 --2- 172.21.15.32:0/2389376834 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0734145b60 0x7f07140044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:50.741 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f0733fff700 1 -- 172.21.15.32:0/2389376834 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0734145b60 msgr2=0x7f07140044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:50.741 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f0733fff700 1 --2- 172.21.15.32:0/2389376834 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0734145b60 0x7f07140044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:50.741 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f0733fff700 1 -- 172.21.15.32:0/2389376834 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07140049c0 msgr2=0x7f0714004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:50.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f0733fff700 1 --2- 172.21.15.32:0/2389376834 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07140049c0 0x7f0714004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:50.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f0733fff700 1 -- 172.21.15.32:0/2389376834 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0714003230 con 0x7f07140038e0 2020-12-05T17:35:50.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f07337fe700 1 --2- 172.21.15.32:0/2389376834 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0734145b60 0x7f07140044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:35:50.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f0733fff700 1 --2- 172.21.15.32:0/2389376834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07140038e0 0x7f0714003ce0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f07240010d0 tx=0x7f0724007b60).ready entity=mon.0 client_cookie=9a521e50a142a041 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:50.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f0732ffd700 1 -- 172.21.15.32:0/2389376834 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f072400c870 con 0x7f07140038e0 2020-12-05T17:35:50.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f0732ffd700 1 -- 172.21.15.32:0/2389376834 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072400c9d0 con 0x7f07140038e0 2020-12-05T17:35:50.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f0732ffd700 1 -- 172.21.15.32:0/2389376834 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f072400cd30 con 0x7f07140038e0 2020-12-05T17:35:50.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f07397be700 1 -- 172.21.15.32:0/2389376834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07140038e0 msgr2=0x7f0714003ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:50.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f07397be700 1 --2- 172.21.15.32:0/2389376834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07140038e0 0x7f0714003ce0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f07240010d0 tx=0x7f0724007b60).stop 2020-12-05T17:35:50.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f0738fbd700 1 -- 172.21.15.32:0/2389376834 reap_dead start 2020-12-05T17:35:50.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f07397be700 1 -- 172.21.15.32:0/2389376834 shutdown_connections 2020-12-05T17:35:50.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f07397be700 1 -- 172.21.15.32:0/2389376834 >> 172.21.15.32:0/2389376834 conn(0x7f073405bc20 msgr2=0x7f07340b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:50.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f07397be700 1 -- 172.21.15.32:0/2389376834 shutdown_connections 2020-12-05T17:35:50.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.735+0000 7f07397be700 1 -- 172.21.15.32:0/2389376834 wait complete. 2020-12-05T17:35:50.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f07397be700 1 Processor -- start 2020-12-05T17:35:50.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f07397be700 1 -- start start 2020-12-05T17:35:50.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f07397be700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07340b0890 0x7f0734064c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:50.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f07397be700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07340659e0 0x7f0734151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:50.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f07397be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0734154eb0 0x7f0734155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:50.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f07397be700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07341547b0 con 0x7f0734154eb0 2020-12-05T17:35:50.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f07397be700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07340b0c70 con 0x7f07340659e0 2020-12-05T17:35:50.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f07397be700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07341152e0 con 0x7f07340b0890 2020-12-05T17:35:50.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f0738fbd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0734154eb0 0x7f0734155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:50.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f07337fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07340659e0 0x7f0734151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:50.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f0738fbd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0734154eb0 0x7f0734155330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35580/0 (socket says 172.21.15.32:35580) 2020-12-05T17:35:50.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f0738fbd700 1 -- 172.21.15.32:0/2123233799 learned_addr learned my addr 172.21.15.32:0/2123233799 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:50.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f0733fff700 1 --2- 172.21.15.32:0/2123233799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07340b0890 0x7f0734064c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:50.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f0738fbd700 1 -- 172.21.15.32:0/2123233799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07340b0890 msgr2=0x7f0734064c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:50.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f0738fbd700 1 --2- 172.21.15.32:0/2123233799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07340b0890 0x7f0734064c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:50.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f0738fbd700 1 -- 172.21.15.32:0/2123233799 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07340659e0 msgr2=0x7f0734151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:50.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f0738fbd700 1 --2- 172.21.15.32:0/2123233799 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07340659e0 0x7f0734151260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:50.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f0738fbd700 1 -- 172.21.15.32:0/2123233799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0734115560 con 0x7f0734154eb0 2020-12-05T17:35:50.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f0738fbd700 1 --2- 172.21.15.32:0/2123233799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0734154eb0 0x7f0734155330 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f0734114910 tx=0x7f072c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:50.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f07317fa700 1 -- 172.21.15.32:0/2123233799 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f072c013750 con 0x7f0734154eb0 2020-12-05T17:35:50.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f07397be700 1 -- 172.21.15.32:0/2123233799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0734154a10 con 0x7f0734154eb0 2020-12-05T17:35:50.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f07317fa700 1 -- 172.21.15.32:0/2123233799 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072c0138b0 con 0x7f0734154eb0 2020-12-05T17:35:50.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f07317fa700 1 -- 172.21.15.32:0/2123233799 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f072c013c10 con 0x7f0734154eb0 2020-12-05T17:35:50.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.739+0000 7f07397be700 1 -- 172.21.15.32:0/2123233799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073413f8a0 con 0x7f0734154eb0 2020-12-05T17:35:50.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.743+0000 7f07397be700 1 -- 172.21.15.32:0/2123233799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0734154ba0 con 0x7f0734154eb0 2020-12-05T17:35:50.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.743+0000 7f07317fa700 1 -- 172.21.15.32:0/2123233799 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f072c013d70 con 0x7f0734154eb0 2020-12-05T17:35:50.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.743+0000 7f07317fa700 1 --2- 172.21.15.32:0/2123233799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0718055500 0x7f07180579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:50.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.743+0000 7f07317fa700 1 -- 172.21.15.32:0/2123233799 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f072c06e6c0 con 0x7f0734154eb0 2020-12-05T17:35:50.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.743+0000 7f0733fff700 1 --2- 172.21.15.32:0/2123233799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0718055500 0x7f07180579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:50.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.743+0000 7f0733fff700 1 --2- 172.21.15.32:0/2123233799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0718055500 0x7f07180579c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f072401b000 tx=0x7f0724017000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:50.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.751+0000 7f07317fa700 1 -- 172.21.15.32:0/2123233799 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f072c0447c0 con 0x7f0734154eb0 2020-12-05T17:35:50.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.887+0000 7f07397be700 1 -- 172.21.15.32:0/2123233799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0734057050 con 0x7f0734154eb0 2020-12-05T17:35:50.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.891+0000 7f07317fa700 1 -- 172.21.15.32:0/2123233799 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16273 (secure 0 0 0) 0x7f072c047de0 con 0x7f0734154eb0 2020-12-05T17:35:50.888 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:35:50.889 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":54,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:35:25.399857+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:35:20.867930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-12-05T17:35:22.139660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:35:50.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.891+0000 7f07397be700 1 -- 172.21.15.32:0/2123233799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0718055500 msgr2=0x7f07180579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:50.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.891+0000 7f07397be700 1 --2- 172.21.15.32:0/2123233799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0718055500 0x7f07180579c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f072401b000 tx=0x7f0724017000).stop 2020-12-05T17:35:50.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.891+0000 7f07397be700 1 -- 172.21.15.32:0/2123233799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0734154eb0 msgr2=0x7f0734155330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:50.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.891+0000 7f07397be700 1 --2- 172.21.15.32:0/2123233799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0734154eb0 0x7f0734155330 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f0734114910 tx=0x7f072c008040).stop 2020-12-05T17:35:50.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.891+0000 7f07397be700 1 -- 172.21.15.32:0/2123233799 shutdown_connections 2020-12-05T17:35:50.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.891+0000 7f07397be700 1 --2- 172.21.15.32:0/2123233799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0734154eb0 0x7f0734155330 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:50.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.891+0000 7f07397be700 1 --2- 172.21.15.32:0/2123233799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0718055500 0x7f07180579c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:50.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.891+0000 7f07397be700 1 --2- 172.21.15.32:0/2123233799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07340b0890 0x7f0734064c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:50.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.891+0000 7f07397be700 1 --2- 172.21.15.32:0/2123233799 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07340659e0 0x7f0734151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:50.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.891+0000 7f07397be700 1 -- 172.21.15.32:0/2123233799 >> 172.21.15.32:0/2123233799 conn(0x7f073405bc20 msgr2=0x7f07340b6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:50.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.891+0000 7f07397be700 1 -- 172.21.15.32:0/2123233799 shutdown_connections 2020-12-05T17:35:50.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:50.891+0000 7f07397be700 1 -- 172.21.15.32:0/2123233799 wait complete. 2020-12-05T17:35:50.901 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:35:20.867930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:35:22.139660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:35:50.902 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:35:51.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.019+0000 7ff95cf6f700 1 Processor -- start 2020-12-05T17:35:51.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.019+0000 7ff95cf6f700 1 -- start start 2020-12-05T17:35:51.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.019+0000 7ff95cf6f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff95805feb0 0x7ff95805f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.019+0000 7ff95cf6f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9580b2fb0 0x7ff958061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.019+0000 7ff95cf6f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9580b2df0 con 0x7ff95805f3b0 2020-12-05T17:35:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.019+0000 7ff95cf6f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9580b3c90 con 0x7ff95805feb0 2020-12-05T17:35:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.019+0000 7ff95cf6f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9580b2b50 con 0x7ff9580b2fb0 2020-12-05T17:35:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.019+0000 7ff94ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff95805feb0 0x7ff95805f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.019+0000 7ff94ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff95805feb0 0x7ff95805f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35588/0 (socket says 172.21.15.32:35588) 2020-12-05T17:35:51.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.019+0000 7ff94ffff700 1 -- 172.21.15.32:0/3374038330 learned_addr learned my addr 172.21.15.32:0/3374038330 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:51.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff957fff700 1 --2- 172.21.15.32:0/3374038330 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9580b2fb0 0x7ff958061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff94ffff700 1 -- 172.21.15.32:0/3374038330 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9580b2fb0 msgr2=0x7ff958061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:51.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff94ffff700 1 --2- 172.21.15.32:0/3374038330 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9580b2fb0 0x7ff958061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff94ffff700 1 -- 172.21.15.32:0/3374038330 >> v1:172.21.15.61:6789/0 conn(0x7ff95805f3b0 legacy=0x7ff95813c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff94ffff700 1 -- 172.21.15.32:0/3374038330 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff958142820 con 0x7ff95805feb0 2020-12-05T17:35:51.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff94ffff700 1 --2- 172.21.15.32:0/3374038330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff95805feb0 0x7ff95805f790 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7ff940002f30 tx=0x7ff940006e80).ready entity=mon.0 client_cookie=719d13e33896a566 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:51.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff956ffd700 1 -- 172.21.15.32:0/3374038330 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9400097f0 con 0x7ff95805feb0 2020-12-05T17:35:51.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff956ffd700 1 -- 172.21.15.32:0/3374038330 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff940009950 con 0x7ff95805feb0 2020-12-05T17:35:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff956ffd700 1 -- 172.21.15.32:0/3374038330 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff940009cb0 con 0x7ff95805feb0 2020-12-05T17:35:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff95cf6f700 1 -- 172.21.15.32:0/3374038330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff95805feb0 msgr2=0x7ff95805f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff95cf6f700 1 --2- 172.21.15.32:0/3374038330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff95805feb0 0x7ff95805f790 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7ff940002f30 tx=0x7ff940006e80).stop 2020-12-05T17:35:51.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff95cf6f700 1 -- 172.21.15.32:0/3374038330 shutdown_connections 2020-12-05T17:35:51.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff95cf6f700 1 --2- 172.21.15.32:0/3374038330 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9580b2fb0 0x7ff958061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff95cf6f700 1 --2- 172.21.15.32:0/3374038330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff95805feb0 0x7ff95805f790 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff95cf6f700 1 -- 172.21.15.32:0/3374038330 >> 172.21.15.32:0/3374038330 conn(0x7ff9580b8f20 msgr2=0x7ff9580b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:51.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff95cf6f700 1 -- 172.21.15.32:0/3374038330 shutdown_connections 2020-12-05T17:35:51.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff95cf6f700 1 -- 172.21.15.32:0/3374038330 wait complete. 2020-12-05T17:35:51.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff95cf6f700 1 Processor -- start 2020-12-05T17:35:51.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff95cf6f700 1 -- start start 2020-12-05T17:35:51.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff95cf6f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff95805f3b0 0x7ff958148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff95cf6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff95805feb0 0x7ff958114140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff95cf6f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9580b2fb0 0x7ff9581477e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff95cf6f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9581137a0 con 0x7ff95805feb0 2020-12-05T17:35:51.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff95cf6f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff958113a00 con 0x7ff95805f3b0 2020-12-05T17:35:51.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff95cf6f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff958113ca0 con 0x7ff9580b2fb0 2020-12-05T17:35:51.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff94ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff95805feb0 0x7ff958114140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff94ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff95805feb0 0x7ff958114140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35592/0 (socket says 172.21.15.32:35592) 2020-12-05T17:35:51.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff94ffff700 1 -- 172.21.15.32:0/2895573954 learned_addr learned my addr 172.21.15.32:0/2895573954 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:51.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff957fff700 1 --2- 172.21.15.32:0/2895573954 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9580b2fb0 0x7ff9581477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.023+0000 7ff9577fe700 1 --2- 172.21.15.32:0/2895573954 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff95805f3b0 0x7ff958148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.027+0000 7ff94ffff700 1 -- 172.21.15.32:0/2895573954 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9580b2fb0 msgr2=0x7ff9581477e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.027+0000 7ff94ffff700 1 --2- 172.21.15.32:0/2895573954 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9580b2fb0 0x7ff9581477e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.027+0000 7ff94ffff700 1 -- 172.21.15.32:0/2895573954 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff95805f3b0 msgr2=0x7ff958148cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.027+0000 7ff94ffff700 1 --2- 172.21.15.32:0/2895573954 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff95805f3b0 0x7ff958148cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.027+0000 7ff94ffff700 1 -- 172.21.15.32:0/2895573954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff958112aa0 con 0x7ff95805feb0 2020-12-05T17:35:51.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.027+0000 7ff9577fe700 1 --2- 172.21.15.32:0/2895573954 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff95805f3b0 0x7ff958148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:35:51.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.027+0000 7ff94ffff700 1 --2- 172.21.15.32:0/2895573954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff95805feb0 0x7ff958114140 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7ff940002f30 tx=0x7ff940015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:51.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.027+0000 7ff9557fa700 1 -- 172.21.15.32:0/2895573954 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff940011d30 con 0x7ff95805feb0 2020-12-05T17:35:51.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.027+0000 7ff9557fa700 1 -- 172.21.15.32:0/2895573954 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff940011e90 con 0x7ff95805feb0 2020-12-05T17:35:51.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.027+0000 7ff95cf6f700 1 -- 172.21.15.32:0/2895573954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff958147cd0 con 0x7ff95805feb0 2020-12-05T17:35:51.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.027+0000 7ff9557fa700 1 -- 172.21.15.32:0/2895573954 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff94001e650 con 0x7ff95805feb0 2020-12-05T17:35:51.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.027+0000 7ff95cf6f700 1 -- 172.21.15.32:0/2895573954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff958140ca0 con 0x7ff95805feb0 2020-12-05T17:35:51.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.027+0000 7ff9557fa700 1 -- 172.21.15.32:0/2895573954 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff940025030 con 0x7ff95805feb0 2020-12-05T17:35:51.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.027+0000 7ff95cf6f700 1 -- 172.21.15.32:0/2895573954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff958113e60 con 0x7ff95805feb0 2020-12-05T17:35:51.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.027+0000 7ff9557fa700 1 --2- 172.21.15.32:0/2895573954 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff9300554e0 0x7ff9300579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.027+0000 7ff9577fe700 1 --2- 172.21.15.32:0/2895573954 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff9300554e0 0x7ff9300579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.027+0000 7ff9557fa700 1 -- 172.21.15.32:0/2895573954 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff94006f600 con 0x7ff95805feb0 2020-12-05T17:35:51.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.027+0000 7ff9577fe700 1 --2- 172.21.15.32:0/2895573954 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff9300554e0 0x7ff9300579a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7ff948008000 tx=0x7ff948009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:51.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.035+0000 7ff9557fa700 1 -- 172.21.15.32:0/2895573954 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff940045690 con 0x7ff95805feb0 2020-12-05T17:35:51.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.171+0000 7ff95cf6f700 1 -- 172.21.15.32:0/2895573954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff958057050 con 0x7ff95805feb0 2020-12-05T17:35:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.171+0000 7ff9557fa700 1 -- 172.21.15.32:0/2895573954 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v54) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff940023020 con 0x7ff95805feb0 2020-12-05T17:35:51.171 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:35:51.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.175+0000 7ff95cf6f700 1 -- 172.21.15.32:0/2895573954 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff9300554e0 msgr2=0x7ff9300579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.175+0000 7ff95cf6f700 1 --2- 172.21.15.32:0/2895573954 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff9300554e0 0x7ff9300579a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7ff948008000 tx=0x7ff948009040).stop 2020-12-05T17:35:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.175+0000 7ff95cf6f700 1 -- 172.21.15.32:0/2895573954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff95805feb0 msgr2=0x7ff958114140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.175+0000 7ff95cf6f700 1 --2- 172.21.15.32:0/2895573954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff95805feb0 0x7ff958114140 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7ff940002f30 tx=0x7ff940015000).stop 2020-12-05T17:35:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.175+0000 7ff95cf6f700 1 -- 172.21.15.32:0/2895573954 shutdown_connections 2020-12-05T17:35:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.175+0000 7ff95cf6f700 1 --2- 172.21.15.32:0/2895573954 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff9300554e0 0x7ff9300579a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.175+0000 7ff95cf6f700 1 --2- 172.21.15.32:0/2895573954 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9580b2fb0 0x7ff9581477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.175+0000 7ff95cf6f700 1 --2- 172.21.15.32:0/2895573954 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff95805f3b0 0x7ff958148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.175+0000 7ff95cf6f700 1 --2- 172.21.15.32:0/2895573954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff95805feb0 0x7ff958114140 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.175+0000 7ff95cf6f700 1 -- 172.21.15.32:0/2895573954 >> 172.21.15.32:0/2895573954 conn(0x7ff9580b8f20 msgr2=0x7ff9580b97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.175+0000 7ff95cf6f700 1 -- 172.21.15.32:0/2895573954 shutdown_connections 2020-12-05T17:35:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.175+0000 7ff95cf6f700 1 -- 172.21.15.32:0/2895573954 wait complete. 2020-12-05T17:35:51.183 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:35:51.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.295+0000 7f92a3fff700 1 Processor -- start 2020-12-05T17:35:51.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.295+0000 7f92a3fff700 1 -- start start 2020-12-05T17:35:51.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.295+0000 7f92a3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92a405f3b0 0x7f92a405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.295+0000 7f92a3fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a40b2df0 con 0x7f92a405feb0 2020-12-05T17:35:51.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.295+0000 7f92a3fff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a40b3c90 con 0x7f92a40b2fb0 2020-12-05T17:35:51.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.295+0000 7f92a3fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a40b2b50 con 0x7f92a405f3b0 2020-12-05T17:35:51.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.295+0000 7f92a27fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f92a405feb0 0x7f92a413c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38246/0 (socket says 172.21.15.32:38246) 2020-12-05T17:35:51.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.295+0000 7f92a27fc700 1 -- 172.21.15.32:0/3770222899 learned_addr learned my addr 172.21.15.32:0/3770222899 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:35:51.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.295+0000 7f92a2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92a405f3b0 0x7f92a405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.295+0000 7f92a1ffb700 1 -- 172.21.15.32:0/3770222899 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4063047277 0 0) 0x7f92a40b2df0 con 0x7f92a405feb0 2020-12-05T17:35:51.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.295+0000 7f92a1ffb700 1 -- 172.21.15.32:0/3770222899 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9288003230 con 0x7f92a405feb0 2020-12-05T17:35:51.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.295+0000 7f92a1ffb700 1 -- 172.21.15.32:0/3770222899 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3263561048 0 0) 0x7f92a40b3c90 con 0x7f92a40b2fb0 2020-12-05T17:35:51.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.295+0000 7f92a2ffd700 1 -- 172.21.15.32:0/3770222899 >> v1:172.21.15.61:6789/0 conn(0x7f92a40b2fb0 legacy=0x7f92a4142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.295+0000 7f92a2ffd700 1 -- 172.21.15.32:0/3770222899 >> v1:172.21.15.32:6789/0 conn(0x7f92a405feb0 legacy=0x7f92a413c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.295+0000 7f92a2ffd700 1 -- 172.21.15.32:0/3770222899 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a4142820 con 0x7f92a405f3b0 2020-12-05T17:35:51.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.299+0000 7f92a2ffd700 1 --2- 172.21.15.32:0/3770222899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a405f3b0 0x7f92a405f790 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f9298001bf0 tx=0x7f9294006dc0).ready entity=mon.0 client_cookie=64b7a9e199072f3d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.299+0000 7f92a1ffb700 1 -- 172.21.15.32:0/3770222899 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9298001a50 con 0x7f92a405f3b0 2020-12-05T17:35:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.299+0000 7f92a1ffb700 1 -- 172.21.15.32:0/3770222899 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9294009670 con 0x7f92a405f3b0 2020-12-05T17:35:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.299+0000 7f92a1ffb700 1 -- 172.21.15.32:0/3770222899 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92940097e0 con 0x7f92a405f3b0 2020-12-05T17:35:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.299+0000 7f92a3fff700 1 -- 172.21.15.32:0/3770222899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a405f3b0 msgr2=0x7f92a405f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.299+0000 7f92a3fff700 1 --2- 172.21.15.32:0/3770222899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a405f3b0 0x7f92a405f790 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f9298001bf0 tx=0x7f9294006dc0).stop 2020-12-05T17:35:51.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.299+0000 7f92a3fff700 1 -- 172.21.15.32:0/3770222899 shutdown_connections 2020-12-05T17:35:51.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.299+0000 7f92a3fff700 1 --2- 172.21.15.32:0/3770222899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a405f3b0 0x7f92a405f790 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.299+0000 7f92a3fff700 1 -- 172.21.15.32:0/3770222899 >> 172.21.15.32:0/3770222899 conn(0x7f92a40b8f20 msgr2=0x7f92a40b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:51.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.299+0000 7f92a3fff700 1 -- 172.21.15.32:0/3770222899 shutdown_connections 2020-12-05T17:35:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.299+0000 7f92a3fff700 1 -- 172.21.15.32:0/3770222899 wait complete. 2020-12-05T17:35:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.299+0000 7f92a3fff700 1 Processor -- start 2020-12-05T17:35:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.299+0000 7f92a3fff700 1 -- start start 2020-12-05T17:35:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f92a3fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92a405f3b0 0x7f92a413fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f92a3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a405feb0 0x7f92a40667d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f92a3fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92a40b2fb0 0x7f92a4140fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f92a3fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a4143c90 con 0x7f92a405feb0 2020-12-05T17:35:51.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f92a3fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a4143ec0 con 0x7f92a405f3b0 2020-12-05T17:35:51.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f92a3fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a4144160 con 0x7f92a40b2fb0 2020-12-05T17:35:51.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f92a27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a405feb0 0x7f92a40667d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f92a27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a405feb0 0x7f92a40667d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35606/0 (socket says 172.21.15.32:35606) 2020-12-05T17:35:51.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f92a27fc700 1 -- 172.21.15.32:0/2173054567 learned_addr learned my addr 172.21.15.32:0/2173054567 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:51.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f92a2ffd700 1 --2- 172.21.15.32:0/2173054567 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92a405f3b0 0x7f92a413fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f92a37fe700 1 --2- 172.21.15.32:0/2173054567 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92a40b2fb0 0x7f92a4140fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f92a27fc700 1 -- 172.21.15.32:0/2173054567 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92a40b2fb0 msgr2=0x7f92a4140fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f92a27fc700 1 --2- 172.21.15.32:0/2173054567 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92a40b2fb0 0x7f92a4140fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f92a27fc700 1 -- 172.21.15.32:0/2173054567 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92a405f3b0 msgr2=0x7f92a413fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f92a27fc700 1 --2- 172.21.15.32:0/2173054567 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92a405f3b0 0x7f92a413fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f92a27fc700 1 -- 172.21.15.32:0/2173054567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a4151240 con 0x7f92a405feb0 2020-12-05T17:35:51.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f92a27fc700 1 --2- 172.21.15.32:0/2173054567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a405feb0 0x7f92a40667d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f9298001510 tx=0x7f9298001ed0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:51.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f9293fff700 1 -- 172.21.15.32:0/2173054567 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9298004a80 con 0x7f92a405feb0 2020-12-05T17:35:51.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f9293fff700 1 -- 172.21.15.32:0/2173054567 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9298004be0 con 0x7f92a405feb0 2020-12-05T17:35:51.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f9293fff700 1 -- 172.21.15.32:0/2173054567 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9298007170 con 0x7f92a405feb0 2020-12-05T17:35:51.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f92a3fff700 1 -- 172.21.15.32:0/2173054567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a41553b0 con 0x7f92a405feb0 2020-12-05T17:35:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f92a3fff700 1 -- 172.21.15.32:0/2173054567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a41414a0 con 0x7f92a405feb0 2020-12-05T17:35:51.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f92a3fff700 1 -- 172.21.15.32:0/2173054567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a4144320 con 0x7f92a405feb0 2020-12-05T17:35:51.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.303+0000 7f9293fff700 1 -- 172.21.15.32:0/2173054567 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9298010070 con 0x7f92a405feb0 2020-12-05T17:35:51.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.307+0000 7f9293fff700 1 --2- 172.21.15.32:0/2173054567 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9280055350 0x7f9280057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.307+0000 7f9293fff700 1 -- 172.21.15.32:0/2173054567 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f929806dd50 con 0x7f92a405feb0 2020-12-05T17:35:51.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.307+0000 7f92a2ffd700 1 --2- 172.21.15.32:0/2173054567 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9280055350 0x7f9280057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.307+0000 7f92a2ffd700 1 --2- 172.21.15.32:0/2173054567 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9280055350 0x7f9280057810 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f9294019000 tx=0x7f92a40b9f00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:51.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.311+0000 7f9293fff700 1 -- 172.21.15.32:0/2173054567 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9298043ef0 con 0x7f92a405feb0 2020-12-05T17:35:51.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.451+0000 7f92a3fff700 1 -- 172.21.15.32:0/2173054567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f92a4057050 con 0x7f92a405feb0 2020-12-05T17:35:51.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.451+0000 7f9293fff700 1 -- 172.21.15.32:0/2173054567 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v54) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9298019030 con 0x7f92a405feb0 2020-12-05T17:35:51.449 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:35:51.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.451+0000 7f92a3fff700 1 -- 172.21.15.32:0/2173054567 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9280055350 msgr2=0x7f9280057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.451+0000 7f92a3fff700 1 --2- 172.21.15.32:0/2173054567 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9280055350 0x7f9280057810 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f9294019000 tx=0x7f92a40b9f00).stop 2020-12-05T17:35:51.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.451+0000 7f92a3fff700 1 -- 172.21.15.32:0/2173054567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a405feb0 msgr2=0x7f92a40667d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.451+0000 7f92a3fff700 1 --2- 172.21.15.32:0/2173054567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a405feb0 0x7f92a40667d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f9298001510 tx=0x7f9298001ed0).stop 2020-12-05T17:35:51.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.451+0000 7f92a3fff700 1 -- 172.21.15.32:0/2173054567 shutdown_connections 2020-12-05T17:35:51.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.451+0000 7f92a3fff700 1 --2- 172.21.15.32:0/2173054567 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9280055350 0x7f9280057810 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.451+0000 7f92a3fff700 1 --2- 172.21.15.32:0/2173054567 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92a40b2fb0 0x7f92a4140fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.451+0000 7f92a3fff700 1 --2- 172.21.15.32:0/2173054567 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92a405f3b0 0x7f92a413fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.451+0000 7f92a3fff700 1 --2- 172.21.15.32:0/2173054567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a405feb0 0x7f92a40667d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.451+0000 7f92a3fff700 1 -- 172.21.15.32:0/2173054567 >> 172.21.15.32:0/2173054567 conn(0x7f92a40b8f20 msgr2=0x7f92a4141f30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:51.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.451+0000 7f92a3fff700 1 -- 172.21.15.32:0/2173054567 shutdown_connections 2020-12-05T17:35:51.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.451+0000 7f92a3fff700 1 -- 172.21.15.32:0/2173054567 wait complete. 2020-12-05T17:35:51.462 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:35:51.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.579+0000 7fae82a82700 1 Processor -- start 2020-12-05T17:35:51.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.579+0000 7fae82a82700 1 -- start start 2020-12-05T17:35:51.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.579+0000 7fae82a82700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fae7c05ce60 0x7fae7c13ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.579+0000 7fae82a82700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7c0b0ca0 con 0x7fae7c142cc0 2020-12-05T17:35:51.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.579+0000 7fae82a82700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7c05d240 con 0x7fae7c0ad220 2020-12-05T17:35:51.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.579+0000 7fae82a82700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae7c13f670 con 0x7fae7c05ce60 2020-12-05T17:35:51.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.579+0000 7fae8081e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fae7c05ce60 0x7fae7c13ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.579+0000 7fae8081e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fae7c05ce60 0x7fae7c13ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35612/0 (socket says 172.21.15.32:35612) 2020-12-05T17:35:51.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.579+0000 7fae8081e700 1 -- 172.21.15.32:0/27876498 learned_addr learned my addr 172.21.15.32:0/27876498 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:51.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.579+0000 7fae8081e700 1 -- 172.21.15.32:0/27876498 >> v1:172.21.15.61:6789/0 conn(0x7fae7c0ad220 legacy=0x7fae7c13ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:51.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.579+0000 7fae8081e700 1 -- 172.21.15.32:0/27876498 >> v1:172.21.15.32:6789/0 conn(0x7fae7c142cc0 legacy=0x7fae7c0b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.579+0000 7fae8081e700 1 -- 172.21.15.32:0/27876498 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c143240 con 0x7fae7c05ce60 2020-12-05T17:35:51.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.579+0000 7fae8081e700 1 --2- 172.21.15.32:0/27876498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae7c05ce60 0x7fae7c13ea70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7fae7c0b0ee0 tx=0x7fae6c006e80).ready entity=mon.0 client_cookie=7bf3ea69a57993a server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:51.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.579+0000 7fae7b7fe700 1 -- 172.21.15.32:0/27876498 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae6c009770 con 0x7fae7c05ce60 2020-12-05T17:35:51.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae7b7fe700 1 -- 172.21.15.32:0/27876498 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6c0098d0 con 0x7fae7c05ce60 2020-12-05T17:35:51.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae7b7fe700 1 -- 172.21.15.32:0/27876498 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae6c009a40 con 0x7fae7c05ce60 2020-12-05T17:35:51.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae82a82700 1 -- 172.21.15.32:0/27876498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae7c05ce60 msgr2=0x7fae7c13ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae82a82700 1 --2- 172.21.15.32:0/27876498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae7c05ce60 0x7fae7c13ea70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7fae7c0b0ee0 tx=0x7fae6c006e80).stop 2020-12-05T17:35:51.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae82a82700 1 -- 172.21.15.32:0/27876498 shutdown_connections 2020-12-05T17:35:51.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae82a82700 1 --2- 172.21.15.32:0/27876498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae7c05ce60 0x7fae7c13ea70 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae82a82700 1 -- 172.21.15.32:0/27876498 >> 172.21.15.32:0/27876498 conn(0x7fae7c069cb0 msgr2=0x7fae7c0604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:51.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae82a82700 1 -- 172.21.15.32:0/27876498 shutdown_connections 2020-12-05T17:35:51.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae82a82700 1 -- 172.21.15.32:0/27876498 wait complete. 2020-12-05T17:35:51.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae82a82700 1 Processor -- start 2020-12-05T17:35:51.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae82a82700 1 -- start start 2020-12-05T17:35:51.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae82a82700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae7c05ce60 0x7fae7c0b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae82a82700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fae7c0ad220 0x7fae7c155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae82a82700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae7c142cc0 0x7fae7c156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae82a82700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c114160 con 0x7fae7c142cc0 2020-12-05T17:35:51.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae82a82700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c156560 con 0x7fae7c05ce60 2020-12-05T17:35:51.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae82a82700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c155720 con 0x7fae7c0ad220 2020-12-05T17:35:51.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae8101f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae7c142cc0 0x7fae7c156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae8101f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae7c142cc0 0x7fae7c156070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35620/0 (socket says 172.21.15.32:35620) 2020-12-05T17:35:51.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae8101f700 1 -- 172.21.15.32:0/1068059321 learned_addr learned my addr 172.21.15.32:0/1068059321 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:51.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae8081e700 1 --2- 172.21.15.32:0/1068059321 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae7c05ce60 0x7fae7c0b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae7bfff700 1 --2- 172.21.15.32:0/1068059321 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fae7c0ad220 0x7fae7c155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae8101f700 1 -- 172.21.15.32:0/1068059321 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fae7c0ad220 msgr2=0x7fae7c155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae8101f700 1 --2- 172.21.15.32:0/1068059321 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fae7c0ad220 0x7fae7c155b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae8101f700 1 -- 172.21.15.32:0/1068059321 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae7c05ce60 msgr2=0x7fae7c0b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae8101f700 1 --2- 172.21.15.32:0/1068059321 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae7c05ce60 0x7fae7c0b5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae8101f700 1 -- 172.21.15.32:0/1068059321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c113ec0 con 0x7fae7c142cc0 2020-12-05T17:35:51.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae8101f700 1 --2- 172.21.15.32:0/1068059321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae7c142cc0 0x7fae7c156070 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fae7c0b3250 tx=0x7fae74007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:51.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae79ffb700 1 -- 172.21.15.32:0/1068059321 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae7400a5a0 con 0x7fae7c142cc0 2020-12-05T17:35:51.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae79ffb700 1 -- 172.21.15.32:0/1068059321 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7400a700 con 0x7fae7c142cc0 2020-12-05T17:35:51.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae82a82700 1 -- 172.21.15.32:0/1068059321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7c0b6370 con 0x7fae7c142cc0 2020-12-05T17:35:51.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae79ffb700 1 -- 172.21.15.32:0/1068059321 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae7400aa60 con 0x7fae7c142cc0 2020-12-05T17:35:51.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.583+0000 7fae82a82700 1 -- 172.21.15.32:0/1068059321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7c05f5f0 con 0x7fae7c142cc0 2020-12-05T17:35:51.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.587+0000 7fae82a82700 1 -- 172.21.15.32:0/1068059321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae7c057050 con 0x7fae7c142cc0 2020-12-05T17:35:51.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.587+0000 7fae79ffb700 1 -- 172.21.15.32:0/1068059321 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fae74019030 con 0x7fae7c142cc0 2020-12-05T17:35:51.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.587+0000 7fae79ffb700 1 --2- 172.21.15.32:0/1068059321 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fae5c055330 0x7fae5c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.587+0000 7fae79ffb700 1 -- 172.21.15.32:0/1068059321 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fae7406d6d0 con 0x7fae7c142cc0 2020-12-05T17:35:51.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.587+0000 7fae8081e700 1 --2- 172.21.15.32:0/1068059321 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fae5c055330 0x7fae5c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.587+0000 7fae8081e700 1 --2- 172.21.15.32:0/1068059321 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fae5c055330 0x7fae5c0577f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fae6c01b000 tx=0x7fae6c016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:51.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.595+0000 7fae79ffb700 1 -- 172.21.15.32:0/1068059321 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fae74043870 con 0x7fae7c142cc0 2020-12-05T17:35:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.731+0000 7fae82a82700 1 -- 172.21.15.32:0/1068059321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fae7c1408d0 con 0x7fae7c142cc0 2020-12-05T17:35:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.731+0000 7fae79ffb700 1 -- 172.21.15.32:0/1068059321 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+10 (secure 0 0 0) 0x7fae74046e90 con 0x7fae7c142cc0 2020-12-05T17:35:51.730 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:35:51.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.735+0000 7fae82a82700 1 -- 172.21.15.32:0/1068059321 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fae5c055330 msgr2=0x7fae5c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.735+0000 7fae82a82700 1 --2- 172.21.15.32:0/1068059321 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fae5c055330 0x7fae5c0577f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fae6c01b000 tx=0x7fae6c016000).stop 2020-12-05T17:35:51.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.735+0000 7fae82a82700 1 -- 172.21.15.32:0/1068059321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae7c142cc0 msgr2=0x7fae7c156070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.735+0000 7fae82a82700 1 --2- 172.21.15.32:0/1068059321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae7c142cc0 0x7fae7c156070 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fae7c0b3250 tx=0x7fae74007f70).stop 2020-12-05T17:35:51.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.735+0000 7fae82a82700 1 -- 172.21.15.32:0/1068059321 shutdown_connections 2020-12-05T17:35:51.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.735+0000 7fae82a82700 1 --2- 172.21.15.32:0/1068059321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae7c142cc0 0x7fae7c156070 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.735+0000 7fae82a82700 1 --2- 172.21.15.32:0/1068059321 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae7c05ce60 0x7fae7c0b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.735+0000 7fae82a82700 1 --2- 172.21.15.32:0/1068059321 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fae5c055330 0x7fae5c0577f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.735+0000 7fae82a82700 1 --2- 172.21.15.32:0/1068059321 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fae7c0ad220 0x7fae7c155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.735+0000 7fae82a82700 1 -- 172.21.15.32:0/1068059321 >> 172.21.15.32:0/1068059321 conn(0x7fae7c069cb0 msgr2=0x7fae7c145690 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:51.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.735+0000 7fae82a82700 1 -- 172.21.15.32:0/1068059321 shutdown_connections 2020-12-05T17:35:51.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.735+0000 7fae82a82700 1 -- 172.21.15.32:0/1068059321 wait complete. 2020-12-05T17:35:51.742 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:35:51.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56c1660700 1 Processor -- start 2020-12-05T17:35:51.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56c1660700 1 -- start start 2020-12-05T17:35:51.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56c1660700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56bc061b30 0x7f56bc141e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56c1660700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56bc1439c0 con 0x7f56bc05f6f0 2020-12-05T17:35:51.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56bbfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f56bc05f6f0 0x7f56bc141700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38272/0 (socket says 172.21.15.32:38272) 2020-12-05T17:35:51.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56c1660700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56bc05fe20 con 0x7f56bc142300 2020-12-05T17:35:51.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56bbfff700 1 -- 172.21.15.32:0/2155750024 learned_addr learned my addr 172.21.15.32:0/2155750024 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:35:51.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56c1660700 1 -- 172.21.15.32:0/2155750024 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc146910 con 0x7f56bc061b30 2020-12-05T17:35:51.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56bb7fe700 1 --2- 172.21.15.32:0/2155750024 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56bc061b30 0x7f56bc141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56baffd700 1 -- 172.21.15.32:0/2155750024 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3233123224 0 0) 0x7f56bc1439c0 con 0x7f56bc05f6f0 2020-12-05T17:35:51.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56baffd700 1 -- 172.21.15.32:0/2155750024 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f569c003230 con 0x7f56bc05f6f0 2020-12-05T17:35:51.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56baffd700 1 -- 172.21.15.32:0/2155750024 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3066038292 0 0) 0x7f56bc05fe20 con 0x7f56bc142300 2020-12-05T17:35:51.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56baffd700 1 -- 172.21.15.32:0/2155750024 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56bc1439c0 con 0x7f56bc142300 2020-12-05T17:35:51.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56baffd700 1 -- 172.21.15.32:0/2155750024 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3613119068 0 0) 0x7f569c003230 con 0x7f56bc05f6f0 2020-12-05T17:35:51.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56baffd700 1 -- 172.21.15.32:0/2155750024 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56bc061b30 msgr2=0x7f56bc141e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:51.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56baffd700 1 --2- 172.21.15.32:0/2155750024 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56bc061b30 0x7f56bc141e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56baffd700 1 -- 172.21.15.32:0/2155750024 >> v1:172.21.15.61:6789/0 conn(0x7f56bc142300 legacy=0x7f56bc142790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56baffd700 1 -- 172.21.15.32:0/2155750024 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc05fad0 con 0x7f56bc05f6f0 2020-12-05T17:35:51.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56baffd700 1 -- 172.21.15.32:0/2155750024 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f56a4001850 con 0x7f56bc05f6f0 2020-12-05T17:35:51.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56baffd700 1 -- 172.21.15.32:0/2155750024 >> v1:172.21.15.32:6789/0 conn(0x7f56bc05f6f0 legacy=0x7f56bc141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56baffd700 1 --2- 172.21.15.32:0/2155750024 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56bc061b30 0x7f569c0038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56baffd700 1 --2- 172.21.15.32:0/2155750024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c003e20 0x7f569c0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56baffd700 1 --2- 172.21.15.32:0/2155750024 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f569c0049c0 0x7f569c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56baffd700 1 -- 172.21.15.32:0/2155750024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc146910 con 0x7f569c003e20 2020-12-05T17:35:51.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56baffd700 1 -- 172.21.15.32:0/2155750024 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c00b720 con 0x7f569c0049c0 2020-12-05T17:35:51.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56baffd700 1 -- 172.21.15.32:0/2155750024 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569c00b860 con 0x7f56bc061b30 2020-12-05T17:35:51.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56bbfff700 1 --2- 172.21.15.32:0/2155750024 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56bc061b30 0x7f569c0038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56bb7fe700 1 --2- 172.21.15.32:0/2155750024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c003e20 0x7f569c0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56c0e5f700 1 --2- 172.21.15.32:0/2155750024 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f569c0049c0 0x7f569c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56bb7fe700 1 -- 172.21.15.32:0/2155750024 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56bc061b30 msgr2=0x7f569c0038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:51.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56bb7fe700 1 --2- 172.21.15.32:0/2155750024 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56bc061b30 0x7f569c0038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56bb7fe700 1 -- 172.21.15.32:0/2155750024 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f569c0049c0 msgr2=0x7f569c004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:35:51.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56bb7fe700 1 --2- 172.21.15.32:0/2155750024 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f569c0049c0 0x7f569c004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56bb7fe700 1 -- 172.21.15.32:0/2155750024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c003230 con 0x7f569c003e20 2020-12-05T17:35:51.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.867+0000 7f56bbfff700 1 --2- 172.21.15.32:0/2155750024 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56bc061b30 0x7f569c0038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:35:51.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.871+0000 7f56bb7fe700 1 --2- 172.21.15.32:0/2155750024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c003e20 0x7f569c0044d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f56ac00c000 tx=0x7f56ac0091a0).ready entity=mon.0 client_cookie=b1f017efcba97f16 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:51.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.871+0000 7f56baffd700 1 -- 172.21.15.32:0/2155750024 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56ac00eb80 con 0x7f569c003e20 2020-12-05T17:35:51.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.871+0000 7f56baffd700 1 -- 172.21.15.32:0/2155750024 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ac00ece0 con 0x7f569c003e20 2020-12-05T17:35:51.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.871+0000 7f56baffd700 1 -- 172.21.15.32:0/2155750024 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56ac0125e0 con 0x7f569c003e20 2020-12-05T17:35:51.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.871+0000 7f56c1660700 1 -- 172.21.15.32:0/2155750024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c003e20 msgr2=0x7f569c0044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.871+0000 7f56c1660700 1 --2- 172.21.15.32:0/2155750024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c003e20 0x7f569c0044d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f56ac00c000 tx=0x7f56ac0091a0).stop 2020-12-05T17:35:51.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.871+0000 7f56c0e5f700 1 -- 172.21.15.32:0/2155750024 reap_dead start 2020-12-05T17:35:51.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.871+0000 7f56c1660700 1 -- 172.21.15.32:0/2155750024 shutdown_connections 2020-12-05T17:35:51.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.871+0000 7f56c1660700 1 -- 172.21.15.32:0/2155750024 >> 172.21.15.32:0/2155750024 conn(0x7f56bc061010 msgr2=0x7f56bc061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:51.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.871+0000 7f56c1660700 1 -- 172.21.15.32:0/2155750024 shutdown_connections 2020-12-05T17:35:51.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.871+0000 7f56c1660700 1 -- 172.21.15.32:0/2155750024 wait complete. 2020-12-05T17:35:51.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.871+0000 7f56c1660700 1 Processor -- start 2020-12-05T17:35:51.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.871+0000 7f56c1660700 1 -- start start 2020-12-05T17:35:51.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.875+0000 7f56c1660700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56bc0ae670 0x7f56bc063110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.875+0000 7f56c1660700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f56bc148f20 0x7f56bc154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.875+0000 7f56c1660700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56bc0657b0 0x7f56bc13d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.875+0000 7f56c1660700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc0b6b30 con 0x7f56bc0657b0 2020-12-05T17:35:51.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.875+0000 7f56c1660700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc064360 con 0x7f56bc148f20 2020-12-05T17:35:51.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.875+0000 7f56c1660700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc149960 con 0x7f56bc0ae670 2020-12-05T17:35:51.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.875+0000 7f56c0e5f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56bc0657b0 0x7f56bc13d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.875+0000 7f56bb7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f56bc148f20 0x7f56bc154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.875+0000 7f56c0e5f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56bc0657b0 0x7f56bc13d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35640/0 (socket says 172.21.15.32:35640) 2020-12-05T17:35:51.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.875+0000 7f56c0e5f700 1 -- 172.21.15.32:0/3242754265 learned_addr learned my addr 172.21.15.32:0/3242754265 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:51.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.875+0000 7f56bbfff700 1 --2- 172.21.15.32:0/3242754265 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56bc0ae670 0x7f56bc063110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.875+0000 7f56c0e5f700 1 -- 172.21.15.32:0/3242754265 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56bc0ae670 msgr2=0x7f56bc063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.875+0000 7f56c0e5f700 1 --2- 172.21.15.32:0/3242754265 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56bc0ae670 0x7f56bc063110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.875+0000 7f56c0e5f700 1 -- 172.21.15.32:0/3242754265 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f56bc148f20 msgr2=0x7f56bc154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:51.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.875+0000 7f56c0e5f700 1 --2- 172.21.15.32:0/3242754265 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f56bc148f20 0x7f56bc154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:51.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.875+0000 7f56c0e5f700 1 -- 172.21.15.32:0/3242754265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc149c00 con 0x7f56bc0657b0 2020-12-05T17:35:51.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.875+0000 7f56bb7fe700 1 --2- 172.21.15.32:0/3242754265 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f56bc148f20 0x7f56bc154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:35:51.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.875+0000 7f56c0e5f700 1 --2- 172.21.15.32:0/3242754265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56bc0657b0 0x7f56bc13d4d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f56b4000b20 tx=0x7f56b4007b60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:51.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.875+0000 7f56b97fa700 1 -- 172.21.15.32:0/3242754265 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56b4011850 con 0x7f56bc0657b0 2020-12-05T17:35:51.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.875+0000 7f56b97fa700 1 -- 172.21.15.32:0/3242754265 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b40119b0 con 0x7f56bc0657b0 2020-12-05T17:35:51.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.875+0000 7f56c1660700 1 -- 172.21.15.32:0/3242754265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56bc0b6690 con 0x7f56bc0657b0 2020-12-05T17:35:51.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.875+0000 7f56b97fa700 1 -- 172.21.15.32:0/3242754265 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56b4011d10 con 0x7f56bc0657b0 2020-12-05T17:35:51.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.875+0000 7f56c1660700 1 -- 172.21.15.32:0/3242754265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56bc0aea50 con 0x7f56bc0657b0 2020-12-05T17:35:51.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.879+0000 7f56c1660700 1 -- 172.21.15.32:0/3242754265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56bc057050 con 0x7f56bc0657b0 2020-12-05T17:35:51.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.879+0000 7f56b97fa700 1 -- 172.21.15.32:0/3242754265 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f56b401b030 con 0x7f56bc0657b0 2020-12-05T17:35:51.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.879+0000 7f56b97fa700 1 --2- 172.21.15.32:0/3242754265 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f56a00554e0 0x7f56a00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:51.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.879+0000 7f56b97fa700 1 -- 172.21.15.32:0/3242754265 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f56b406dc00 con 0x7f56bc0657b0 2020-12-05T17:35:51.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.879+0000 7f56bbfff700 1 --2- 172.21.15.32:0/3242754265 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f56a00554e0 0x7f56a00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:51.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.879+0000 7f56bbfff700 1 --2- 172.21.15.32:0/3242754265 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f56a00554e0 0x7f56a00579a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f56a4009000 tx=0x7f56a4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:51.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:51.887+0000 7f56b97fa700 1 -- 172.21.15.32:0/3242754265 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f56b4073020 con 0x7f56bc0657b0 2020-12-05T17:35:52.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.023+0000 7f56c1660700 1 -- 172.21.15.32:0/3242754265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f56bc069690 con 0x7f56bc0657b0 2020-12-05T17:35:52.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.027+0000 7f56b97fa700 1 -- 172.21.15.32:0/3242754265 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+10 (secure 0 0 0) 0x7f56b4019070 con 0x7f56bc0657b0 2020-12-05T17:35:52.024 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:35:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.027+0000 7f56c1660700 1 -- 172.21.15.32:0/3242754265 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f56a00554e0 msgr2=0x7f56a00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.027+0000 7f56c1660700 1 --2- 172.21.15.32:0/3242754265 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f56a00554e0 0x7f56a00579a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f56a4009000 tx=0x7f56a4009040).stop 2020-12-05T17:35:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.027+0000 7f56c1660700 1 -- 172.21.15.32:0/3242754265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56bc0657b0 msgr2=0x7f56bc13d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:52.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.027+0000 7f56c1660700 1 --2- 172.21.15.32:0/3242754265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56bc0657b0 0x7f56bc13d4d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f56b4000b20 tx=0x7f56b4007b60).stop 2020-12-05T17:35:52.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.027+0000 7f56c1660700 1 -- 172.21.15.32:0/3242754265 shutdown_connections 2020-12-05T17:35:52.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.027+0000 7f56c1660700 1 --2- 172.21.15.32:0/3242754265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56bc0657b0 0x7f56bc13d4d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:52.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.031+0000 7f56c1660700 1 --2- 172.21.15.32:0/3242754265 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f56a00554e0 0x7f56a00579a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:52.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.031+0000 7f56c1660700 1 --2- 172.21.15.32:0/3242754265 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56bc0ae670 0x7f56bc063110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.031+0000 7f56c1660700 1 --2- 172.21.15.32:0/3242754265 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f56bc148f20 0x7f56bc154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.031+0000 7f56c1660700 1 -- 172.21.15.32:0/3242754265 >> 172.21.15.32:0/3242754265 conn(0x7f56bc061010 msgr2=0x7f56bc066110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.031+0000 7f56c1660700 1 -- 172.21.15.32:0/3242754265 shutdown_connections 2020-12-05T17:35:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.031+0000 7f56c1660700 1 -- 172.21.15.32:0/3242754265 wait complete. 2020-12-05T17:35:52.044 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:35:52.045 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:35:52.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.151+0000 7f9163512700 1 Processor -- start 2020-12-05T17:35:52.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.151+0000 7f9163512700 1 -- start start 2020-12-05T17:35:52.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f9163512700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f915c0b1fc0 0x7f915c0af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:52.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f9163512700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f915c146a90 con 0x7f915c0ab710 2020-12-05T17:35:52.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f9163512700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f915c0b23a0 con 0x7f915c147400 2020-12-05T17:35:52.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f9163512700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f915c061e90 con 0x7f915c0b1fc0 2020-12-05T17:35:52.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f9161d0f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f915c0ab710 0x7f915c061780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38292/0 (socket says 172.21.15.32:38292) 2020-12-05T17:35:52.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f9161d0f700 1 -- 172.21.15.32:0/1899417917 learned_addr learned my addr 172.21.15.32:0/1899417917 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:35:52.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f9162510700 1 --2- 172.21.15.32:0/1899417917 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f915c0b1fc0 0x7f915c0af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:52.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f916150e700 1 -- 172.21.15.32:0/1899417917 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1808356801 0 0) 0x7f915c146a90 con 0x7f915c0ab710 2020-12-05T17:35:52.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f916150e700 1 -- 172.21.15.32:0/1899417917 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9144003230 con 0x7f915c0ab710 2020-12-05T17:35:52.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f916150e700 1 -- 172.21.15.32:0/1899417917 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418300105 0 0) 0x7f915c0b23a0 con 0x7f915c147400 2020-12-05T17:35:52.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f916150e700 1 -- 172.21.15.32:0/1899417917 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f915c146a90 con 0x7f915c147400 2020-12-05T17:35:52.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f9162510700 1 -- 172.21.15.32:0/1899417917 >> v1:172.21.15.61:6789/0 conn(0x7f915c147400 legacy=0x7f915c145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:52.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f9162510700 1 -- 172.21.15.32:0/1899417917 >> v1:172.21.15.32:6789/0 conn(0x7f915c0ab710 legacy=0x7f915c061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:52.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f9162510700 1 -- 172.21.15.32:0/1899417917 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f915c146cc0 con 0x7f915c0b1fc0 2020-12-05T17:35:52.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f9162510700 1 --2- 172.21.15.32:0/1899417917 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f915c0b1fc0 0x7f915c0af230 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f9158002f30 tx=0x7f9158006dc0).ready entity=mon.2 client_cookie=93d876d481637a1f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:52.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f916150e700 1 -- 172.21.15.32:0/1899417917 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9158009730 con 0x7f915c0b1fc0 2020-12-05T17:35:52.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f916150e700 1 -- 172.21.15.32:0/1899417917 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9158009890 con 0x7f915c0b1fc0 2020-12-05T17:35:52.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f916150e700 1 -- 172.21.15.32:0/1899417917 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9158009bf0 con 0x7f915c0b1fc0 2020-12-05T17:35:52.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f9163512700 1 -- 172.21.15.32:0/1899417917 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f915c0b1fc0 msgr2=0x7f915c0af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:52.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f9163512700 1 --2- 172.21.15.32:0/1899417917 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f915c0b1fc0 0x7f915c0af230 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f9158002f30 tx=0x7f9158006dc0).stop 2020-12-05T17:35:52.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f9163512700 1 -- 172.21.15.32:0/1899417917 shutdown_connections 2020-12-05T17:35:52.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f9163512700 1 --2- 172.21.15.32:0/1899417917 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f915c0b1fc0 0x7f915c0af230 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:52.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f9163512700 1 -- 172.21.15.32:0/1899417917 >> 172.21.15.32:0/1899417917 conn(0x7f915c0b5530 msgr2=0x7f915c0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:52.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f9163512700 1 -- 172.21.15.32:0/1899417917 shutdown_connections 2020-12-05T17:35:52.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f9163512700 1 -- 172.21.15.32:0/1899417917 wait complete. 2020-12-05T17:35:52.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f9163512700 1 Processor -- start 2020-12-05T17:35:52.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.155+0000 7f9163512700 1 -- start start 2020-12-05T17:35:52.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.159+0000 7f9163512700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f915c0ab710 0x7f915c0aca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:52.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.159+0000 7f9163512700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f915c0b1fc0 0x7f915c14b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:52.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.159+0000 7f9163512700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f915c147400 0x7f915c158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:52.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.159+0000 7f9163512700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915c0abf80 con 0x7f915c0b1fc0 2020-12-05T17:35:52.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.159+0000 7f9163512700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915c0ac220 con 0x7f915c0ab710 2020-12-05T17:35:52.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.159+0000 7f9163512700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f915c09e190 con 0x7f915c147400 2020-12-05T17:35:52.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.159+0000 7f9161d0f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f915c0b1fc0 0x7f915c14b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:52.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.159+0000 7f9161d0f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f915c0b1fc0 0x7f915c14b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35652/0 (socket says 172.21.15.32:35652) 2020-12-05T17:35:52.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.159+0000 7f9161d0f700 1 -- 172.21.15.32:0/3391584005 learned_addr learned my addr 172.21.15.32:0/3391584005 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:35:52.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.159+0000 7f9162d11700 1 --2- 172.21.15.32:0/3391584005 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f915c147400 0x7f915c158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:52.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.159+0000 7f9162510700 1 --2- 172.21.15.32:0/3391584005 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f915c0ab710 0x7f915c0aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:52.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.159+0000 7f9161d0f700 1 -- 172.21.15.32:0/3391584005 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f915c147400 msgr2=0x7f915c158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:52.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.159+0000 7f9161d0f700 1 --2- 172.21.15.32:0/3391584005 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f915c147400 0x7f915c158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:52.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.159+0000 7f9161d0f700 1 -- 172.21.15.32:0/3391584005 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f915c0ab710 msgr2=0x7f915c0aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:52.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.159+0000 7f9161d0f700 1 --2- 172.21.15.32:0/3391584005 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f915c0ab710 0x7f915c0aca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:52.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.159+0000 7f9161d0f700 1 -- 172.21.15.32:0/3391584005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f915c0ad9f0 con 0x7f915c0b1fc0 2020-12-05T17:35:52.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.159+0000 7f9161d0f700 1 --2- 172.21.15.32:0/3391584005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f915c0b1fc0 0x7f915c14b3f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f9150001090 tx=0x7f9150007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:52.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.159+0000 7f914f7fe700 1 -- 172.21.15.32:0/3391584005 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91500072e0 con 0x7f915c0b1fc0 2020-12-05T17:35:52.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.159+0000 7f914f7fe700 1 -- 172.21.15.32:0/3391584005 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9150007440 con 0x7f915c0b1fc0 2020-12-05T17:35:52.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.159+0000 7f9163512700 1 -- 172.21.15.32:0/3391584005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f915c0abb50 con 0x7f915c0b1fc0 2020-12-05T17:35:52.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.159+0000 7f914f7fe700 1 -- 172.21.15.32:0/3391584005 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91500057d0 con 0x7f915c0b1fc0 2020-12-05T17:35:52.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.159+0000 7f9163512700 1 -- 172.21.15.32:0/3391584005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f915c09e850 con 0x7f915c0b1fc0 2020-12-05T17:35:52.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.159+0000 7f9163512700 1 -- 172.21.15.32:0/3391584005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91380033e0 con 0x7f915c0b1fc0 2020-12-05T17:35:52.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.163+0000 7f914f7fe700 1 -- 172.21.15.32:0/3391584005 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9150010070 con 0x7f915c0b1fc0 2020-12-05T17:35:52.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.163+0000 7f914f7fe700 1 --2- 172.21.15.32:0/3391584005 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f913c055330 0x7f913c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:35:52.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.163+0000 7f9162510700 1 --2- 172.21.15.32:0/3391584005 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f913c055330 0x7f913c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:35:52.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.163+0000 7f914f7fe700 1 -- 172.21.15.32:0/3391584005 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f915006d500 con 0x7f915c0b1fc0 2020-12-05T17:35:52.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.163+0000 7f9162510700 1 --2- 172.21.15.32:0/3391584005 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f913c055330 0x7f913c0577f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f915801b000 tx=0x7f9158015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:35:52.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.171+0000 7f914f7fe700 1 -- 172.21.15.32:0/3391584005 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9150043640 con 0x7f915c0b1fc0 2020-12-05T17:35:52.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.347+0000 7f9163512700 1 -- 172.21.15.32:0/3391584005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9138003820 con 0x7f915c0b1fc0 2020-12-05T17:35:52.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.347+0000 7f914f7fe700 1 -- 172.21.15.32:0/3391584005 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2136 (secure 0 0 0) 0x7f9150046c60 con 0x7f915c0b1fc0 2020-12-05T17:35:52.348 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:35:52.348 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5159,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/2123765277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2123765277},{"type":"v1","addr":"172.21.15.61:6833","nonce":2123765277}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:35:23.386407+0000","modified":"2020-12-05T17:35:24.389244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5640},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5640":{"gid":5640,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/681412338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":681412338},{"type":"v1","addr":"172.21.15.32:6835","nonce":681412338}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-12-05T17:35:52.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.351+0000 7f9163512700 1 -- 172.21.15.32:0/3391584005 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f913c055330 msgr2=0x7f913c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:52.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.351+0000 7f9163512700 1 --2- 172.21.15.32:0/3391584005 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f913c055330 0x7f913c0577f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f915801b000 tx=0x7f9158015000).stop 2020-12-05T17:35:52.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.351+0000 7f9163512700 1 -- 172.21.15.32:0/3391584005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f915c0b1fc0 msgr2=0x7f915c14b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:35:52.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.351+0000 7f9163512700 1 --2- 172.21.15.32:0/3391584005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f915c0b1fc0 0x7f915c14b3f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f9150001090 tx=0x7f9150007ad0).stop 2020-12-05T17:35:52.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.351+0000 7f9163512700 1 -- 172.21.15.32:0/3391584005 shutdown_connections 2020-12-05T17:35:52.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.351+0000 7f9163512700 1 --2- 172.21.15.32:0/3391584005 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f913c055330 0x7f913c0577f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:52.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.351+0000 7f9163512700 1 --2- 172.21.15.32:0/3391584005 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f915c147400 0x7f915c158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:52.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.351+0000 7f9163512700 1 --2- 172.21.15.32:0/3391584005 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f915c0ab710 0x7f915c0aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:52.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.351+0000 7f9163512700 1 --2- 172.21.15.32:0/3391584005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f915c0b1fc0 0x7f915c14b3f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:35:52.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.351+0000 7f9163512700 1 -- 172.21.15.32:0/3391584005 >> 172.21.15.32:0/3391584005 conn(0x7f915c0b5530 msgr2=0x7f915c0b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:35:52.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.351+0000 7f9163512700 1 -- 172.21.15.32:0/3391584005 shutdown_connections 2020-12-05T17:35:52.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:35:52.351+0000 7f9163512700 1 -- 172.21.15.32:0/3391584005 wait complete. 2020-12-05T17:35:52.352 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 23 2020-12-05T17:35:52.562 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:35:23.386407+0000', 'modified': '2020-12-05T17:35:24.389244+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5640}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5640': {'gid': 5640, 'name': 'a', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/681412338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 681412338}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 681412338}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:35:52.562 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:35:52.563 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:35:52.691 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":5640,"want_state":"up:active","state":"up:active","rank_uptime":29.297837336000001,"mdsmap_epoch":23,"osdmap_epoch":54,"osdmap_epoch_barrier":52,"uptime":34.700180865999997} 2020-12-05T17:35:52.691 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 5640, "mdsmap_epoch": 23, "osdmap_epoch": 54, "osdmap_epoch_barrier": 52, "rank_uptime": 29.297837336, "state": "up:active", "uptime": 34.700180866, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:35:52.692 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:35:52.692 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-12-05T17:35:52.692 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-12-05T17:35:52.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:52.693 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:35:52.702 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:35:52.760 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:35:52.761 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:35:52.820 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:35:52.821 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:35:52.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:52.822 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:35:52.868 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:35:52.868 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:35:52.911 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:35:52.925 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:35:52.925 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:35:52.980 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:35:52.981 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:35:52.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:52.982 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:35:52.989 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:2020-12-05T17:35:52.987+0000 7f71d98c60c0 -1 init, newargv = 0x5566d8e7b150 newargc=9 2020-12-05T17:35:52.989 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[10808]: starting ceph client 2020-12-05T17:35:53.001 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[10808]: starting fuse 2020-12-05T17:35:54.096 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:35:54.114 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:35:54.114 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:35:54.171 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:35:54.172 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:35:54.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:54.173 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:35:54.219 INFO:teuthology.orchestra.run.smithi036.stdout:49 2020-12-05T17:35:54.220 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-12-05T17:35:54.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:54.221 INFO:teuthology.orchestra.run.smithi036:> sudo python3 -c ' 2020-12-05T17:35:54.221 INFO:teuthology.orchestra.run.smithi036:> import glob 2020-12-05T17:35:54.221 INFO:teuthology.orchestra.run.smithi036:> import re 2020-12-05T17:35:54.222 INFO:teuthology.orchestra.run.smithi036:> import os 2020-12-05T17:35:54.222 INFO:teuthology.orchestra.run.smithi036:> import subprocess 2020-12-05T17:35:54.223 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:35:54.223 INFO:teuthology.orchestra.run.smithi036:> def find_socket(client_name): 2020-12-05T17:35:54.223 INFO:teuthology.orchestra.run.smithi036:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-12-05T17:35:54.224 INFO:teuthology.orchestra.run.smithi036:> files = glob.glob(asok_path) 2020-12-05T17:35:54.224 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:35:54.224 INFO:teuthology.orchestra.run.smithi036:> # Given a non-glob path, it better be there 2020-12-05T17:35:54.225 INFO:teuthology.orchestra.run.smithi036:> if "*" not in asok_path: 2020-12-05T17:35:54.225 INFO:teuthology.orchestra.run.smithi036:> assert(len(files) == 1) 2020-12-05T17:35:54.226 INFO:teuthology.orchestra.run.smithi036:> return files[0] 2020-12-05T17:35:54.226 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:35:54.227 INFO:teuthology.orchestra.run.smithi036:> for f in files: 2020-12-05T17:35:54.227 INFO:teuthology.orchestra.run.smithi036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:35:54.227 INFO:teuthology.orchestra.run.smithi036:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:35:54.228 INFO:teuthology.orchestra.run.smithi036:> return f 2020-12-05T17:35:54.228 INFO:teuthology.orchestra.run.smithi036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:35:54.228 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:35:54.229 INFO:teuthology.orchestra.run.smithi036:> print(find_socket("client.0")) 2020-12-05T17:35:54.229 INFO:teuthology.orchestra.run.smithi036:> ' 2020-12-05T17:35:54.313 INFO:teuthology.orchestra.run.smithi036.stdout:/var/run/ceph/ceph-client.0.10808.asok 2020-12-05T17:35:54.318 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.10808.asok 2020-12-05T17:35:54.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:54.319 INFO:teuthology.orchestra.run.smithi036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.10808.asok status 2020-12-05T17:35:54.419 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2020-12-05T17:35:54.419 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": { 2020-12-05T17:35:54.419 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:35:54.420 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:35:54.420 INFO:teuthology.orchestra.run.smithi036.stdout: "entity_id": "0", 2020-12-05T17:35:54.420 INFO:teuthology.orchestra.run.smithi036.stdout: "hostname": "smithi036", 2020-12-05T17:35:54.420 INFO:teuthology.orchestra.run.smithi036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-12-05T17:35:54.420 INFO:teuthology.orchestra.run.smithi036.stdout: "pid": "10808", 2020-12-05T17:35:54.421 INFO:teuthology.orchestra.run.smithi036.stdout: "root": "/" 2020-12-05T17:35:54.421 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:35:54.421 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_count": 0, 2020-12-05T17:35:54.421 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_pinned_count": 0, 2020-12-05T17:35:54.421 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 5785, 2020-12-05T17:35:54.422 INFO:teuthology.orchestra.run.smithi036.stdout: "inst": { 2020-12-05T17:35:54.422 INFO:teuthology.orchestra.run.smithi036.stdout: "name": { 2020-12-05T17:35:54.422 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "client", 2020-12-05T17:35:54.422 INFO:teuthology.orchestra.run.smithi036.stdout: "num": 5785 2020-12-05T17:35:54.423 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:35:54.423 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:35:54.423 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:35:54.423 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:35:54.423 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 2885015938 2020-12-05T17:35:54.424 INFO:teuthology.orchestra.run.smithi036.stdout: } 2020-12-05T17:35:54.424 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:35:54.424 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:35:54.424 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:35:54.424 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:35:54.425 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 2885015938 2020-12-05T17:35:54.425 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:35:54.425 INFO:teuthology.orchestra.run.smithi036.stdout: "inst_str": "client.5785 172.21.15.36:0/2885015938", 2020-12-05T17:35:54.425 INFO:teuthology.orchestra.run.smithi036.stdout: "addr_str": "172.21.15.36:0/2885015938", 2020-12-05T17:35:54.426 INFO:teuthology.orchestra.run.smithi036.stdout: "inode_count": 1, 2020-12-05T17:35:54.426 INFO:teuthology.orchestra.run.smithi036.stdout: "mds_epoch": 23, 2020-12-05T17:35:54.426 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch": 54, 2020-12-05T17:35:54.426 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:35:54.426 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklisted": false 2020-12-05T17:35:54.427 INFO:teuthology.orchestra.run.smithi036.stdout:} 2020-12-05T17:35:54.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:54.429 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:35:54.429 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:35:54.435 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:35:54.436 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:35:54.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:35:54.437 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:36:15.336 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:36:15.339 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:36:15.341 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:36:15.343 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:36:15.346 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:36:45.403 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:36:45.408 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:36:45.411 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:36:45.414 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:36:45.417 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:36:47.992 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:36:47.993 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:36:48.010 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:36:48.029 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:36:48.045 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:36:48.063 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:36:48.081 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:36:48.082 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:36:48.082 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:36:48.083 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:36:48.100 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:36:48.101 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:36:48.102 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi036.front.sepia.ceph.com... 2020-12-05T17:36:48.102 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:36:48.102 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:36:48.178 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[10808]: fuse finished with error 0 and tester_r 0 2020-12-05T17:36:48.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:36:48.180 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:36:48.181 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:36:48.227 INFO:teuthology.orchestra.run.smithi036.stdout:ext2/ext3 2020-12-05T17:36:48.228 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:36:48.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:36:48.264 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:36:48.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:36:48.313 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:36:48.313 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:36:48.320 INFO:teuthology.orchestra.run.smithi002.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-12-05T17:36:48.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:36:48.322 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-12-05T17:36:48.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:36:48.323 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:36:48.368 INFO:teuthology.orchestra.run.smithi002.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-12-05T17:36:48.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:36:48.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:36:48.370 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:36:48.371 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:36:48.378 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:36:48.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:36:48.379 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2020-12-05T17:36:48.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:36:48.380 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:36:48.428 INFO:teuthology.orchestra.run.smithi053.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:36:48.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:36:48.430 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2020-12-05T17:36:48.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.556+0000 7f0b3af0d700 1 Processor -- start 2020-12-05T17:36:48.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.556+0000 7f0b3af0d700 1 -- start start 2020-12-05T17:36:48.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b3af0d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0b340b5650 0x7f0b34141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:48.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b3af0d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b340afd20 0x7f0b34141fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:48.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b3af0d700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b340af300 con 0x7f0b34142a30 2020-12-05T17:36:48.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b3af0d700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b340b5a30 con 0x7f0b340b5650 2020-12-05T17:36:48.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b3af0d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b340aea90 con 0x7f0b340afd20 2020-12-05T17:36:48.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b39f0b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0b340b5650 0x7f0b34141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:48.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b3970a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b340afd20 0x7f0b34141fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:48.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b3970a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b340afd20 0x7f0b34141fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:60682/0 (socket says 172.21.15.32:60682) 2020-12-05T17:36:48.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b39f0b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0b340b5650 0x7f0b34141ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:55550/0 (socket says 172.21.15.32:55550) 2020-12-05T17:36:48.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b3970a700 1 -- 172.21.15.32:0/3027074737 learned_addr learned my addr 172.21.15.32:0/3027074737 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:48.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b3a70c700 1 --1- 172.21.15.32:0/3027074737 >> v1:172.21.15.61:6789/0 conn(0x7f0b34142a30 0x7f0b34142ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:50950/0 (socket says 172.21.15.32:50950) 2020-12-05T17:36:48.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b3970a700 1 -- 172.21.15.32:0/3027074737 >> v2:172.21.15.61:3300/0 conn(0x7f0b340b5650 msgr2=0x7f0b34141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:36:48.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b3970a700 1 --2- 172.21.15.32:0/3027074737 >> v2:172.21.15.61:3300/0 conn(0x7f0b340b5650 0x7f0b34141ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:48.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b3970a700 1 -- 172.21.15.32:0/3027074737 >> v1:172.21.15.61:6789/0 conn(0x7f0b34142a30 legacy=0x7f0b34142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:48.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b3970a700 1 -- 172.21.15.32:0/3027074737 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b340aed30 con 0x7f0b340afd20 2020-12-05T17:36:48.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b3970a700 1 --2- 172.21.15.32:0/3027074737 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b340afd20 0x7f0b34141fc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f0b24002f30 tx=0x7f0b24006e80).ready entity=mon.2 client_cookie=17611b91f32e07d6 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:48.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b38f09700 1 -- 172.21.15.32:0/3027074737 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b240097f0 con 0x7f0b340afd20 2020-12-05T17:36:48.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b38f09700 1 -- 172.21.15.32:0/3027074737 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b24009950 con 0x7f0b340afd20 2020-12-05T17:36:48.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b38f09700 1 -- 172.21.15.32:0/3027074737 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b24009cb0 con 0x7f0b340afd20 2020-12-05T17:36:48.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b3af0d700 1 -- 172.21.15.32:0/3027074737 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b340afd20 msgr2=0x7f0b34141fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:48.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b3af0d700 1 --2- 172.21.15.32:0/3027074737 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b340afd20 0x7f0b34141fc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f0b24002f30 tx=0x7f0b24006e80).stop 2020-12-05T17:36:48.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b3af0d700 1 -- 172.21.15.32:0/3027074737 shutdown_connections 2020-12-05T17:36:48.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b3af0d700 1 --2- 172.21.15.32:0/3027074737 >> v2:172.21.15.61:3300/0 conn(0x7f0b340b5650 0x7f0b34141ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:48.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b3af0d700 1 --2- 172.21.15.32:0/3027074737 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b340afd20 0x7f0b34141fc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:48.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b3af0d700 1 -- 172.21.15.32:0/3027074737 >> 172.21.15.32:0/3027074737 conn(0x7f0b3405bbb0 msgr2=0x7f0b34062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:48.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b3af0d700 1 -- 172.21.15.32:0/3027074737 shutdown_connections 2020-12-05T17:36:48.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.560+0000 7f0b3af0d700 1 -- 172.21.15.32:0/3027074737 wait complete. 2020-12-05T17:36:48.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b3af0d700 1 Processor -- start 2020-12-05T17:36:48.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b3af0d700 1 -- start start 2020-12-05T17:36:48.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b3af0d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b340afd20 0x7f0b3410ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:48.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b3af0d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b340b5650 0x7f0b3413e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:48.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b3af0d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b34142a30 0x7f0b3413e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:48.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b3af0d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3413eb10 con 0x7f0b340afd20 2020-12-05T17:36:48.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b3af0d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b340b0a00 con 0x7f0b340b5650 2020-12-05T17:36:48.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b3af0d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b34111380 con 0x7f0b34142a30 2020-12-05T17:36:48.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b39f0b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b340afd20 0x7f0b3410ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:48.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b39f0b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b340afd20 0x7f0b3410ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35664/0 (socket says 172.21.15.32:35664) 2020-12-05T17:36:48.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b39f0b700 1 -- 172.21.15.32:0/1487163583 learned_addr learned my addr 172.21.15.32:0/1487163583 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:48.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b3970a700 1 --2- 172.21.15.32:0/1487163583 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b340b5650 0x7f0b3413e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:48.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b3a70c700 1 --2- 172.21.15.32:0/1487163583 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b34142a30 0x7f0b3413e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:48.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b39f0b700 1 -- 172.21.15.32:0/1487163583 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b34142a30 msgr2=0x7f0b3413e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:48.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b39f0b700 1 --2- 172.21.15.32:0/1487163583 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b34142a30 0x7f0b3413e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:48.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b39f0b700 1 -- 172.21.15.32:0/1487163583 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b340b5650 msgr2=0x7f0b3413e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:48.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b39f0b700 1 --2- 172.21.15.32:0/1487163583 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b340b5650 0x7f0b3413e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:48.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b39f0b700 1 -- 172.21.15.32:0/1487163583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b340b16a0 con 0x7f0b340afd20 2020-12-05T17:36:48.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b39f0b700 1 --2- 172.21.15.32:0/1487163583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b340afd20 0x7f0b3410ec00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f0b3000c000 tx=0x7f0b30006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:48.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b22ffd700 1 -- 172.21.15.32:0/1487163583 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b3000ca10 con 0x7f0b340afd20 2020-12-05T17:36:48.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b3af0d700 1 -- 172.21.15.32:0/1487163583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b340b1940 con 0x7f0b340afd20 2020-12-05T17:36:48.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b22ffd700 1 -- 172.21.15.32:0/1487163583 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3000cb70 con 0x7f0b340afd20 2020-12-05T17:36:48.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b3af0d700 1 -- 172.21.15.32:0/1487163583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b340b12a0 con 0x7f0b340afd20 2020-12-05T17:36:48.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.564+0000 7f0b22ffd700 1 -- 172.21.15.32:0/1487163583 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b300123f0 con 0x7f0b340afd20 2020-12-05T17:36:48.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.568+0000 7f0b3af0d700 1 -- 172.21.15.32:0/1487163583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b340b1ad0 con 0x7f0b340afd20 2020-12-05T17:36:48.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.568+0000 7f0b22ffd700 1 -- 172.21.15.32:0/1487163583 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0b30012550 con 0x7f0b340afd20 2020-12-05T17:36:48.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.568+0000 7f0b22ffd700 1 --2- 172.21.15.32:0/1487163583 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0b140553b0 0x7f0b14057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:48.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.568+0000 7f0b22ffd700 1 -- 172.21.15.32:0/1487163583 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0b3006e440 con 0x7f0b340afd20 2020-12-05T17:36:48.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.568+0000 7f0b3970a700 1 --2- 172.21.15.32:0/1487163583 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0b140553b0 0x7f0b14057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:48.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.568+0000 7f0b3970a700 1 --2- 172.21.15.32:0/1487163583 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0b140553b0 0x7f0b14057870 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f0b2401c000 tx=0x7f0b24015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:48.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.572+0000 7f0b22ffd700 1 -- 172.21.15.32:0/1487163583 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0b300444e0 con 0x7f0b340afd20 2020-12-05T17:36:48.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.716+0000 7f0b3af0d700 1 -- 172.21.15.32:0/1487163583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0b340b1ce0 con 0x7f0b340afd20 2020-12-05T17:36:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.728+0000 7f0b22ffd700 1 -- 172.21.15.32:0/1487163583 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0b340b1ce0 con 0x7f0b340afd20 2020-12-05T17:36:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.728+0000 7f0b3af0d700 1 -- 172.21.15.32:0/1487163583 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0b140553b0 msgr2=0x7f0b14057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:48.728 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.728+0000 7f0b3af0d700 1 --2- 172.21.15.32:0/1487163583 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0b140553b0 0x7f0b14057870 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f0b2401c000 tx=0x7f0b24015000).stop 2020-12-05T17:36:48.728 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.728+0000 7f0b3af0d700 1 -- 172.21.15.32:0/1487163583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b340afd20 msgr2=0x7f0b3410ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:48.728 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.728+0000 7f0b3af0d700 1 --2- 172.21.15.32:0/1487163583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b340afd20 0x7f0b3410ec00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f0b3000c000 tx=0x7f0b30006dc0).stop 2020-12-05T17:36:48.728 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.728+0000 7f0b3af0d700 1 -- 172.21.15.32:0/1487163583 shutdown_connections 2020-12-05T17:36:48.729 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.728+0000 7f0b3af0d700 1 --2- 172.21.15.32:0/1487163583 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0b140553b0 0x7f0b14057870 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:48.729 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.728+0000 7f0b3af0d700 1 --2- 172.21.15.32:0/1487163583 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b34142a30 0x7f0b3413e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:48.729 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.728+0000 7f0b3af0d700 1 --2- 172.21.15.32:0/1487163583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b340afd20 0x7f0b3410ec00 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:48.729 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.728+0000 7f0b3af0d700 1 --2- 172.21.15.32:0/1487163583 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b340b5650 0x7f0b3413e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:48.729 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.728+0000 7f0b3af0d700 1 -- 172.21.15.32:0/1487163583 >> 172.21.15.32:0/1487163583 conn(0x7f0b3405bbb0 msgr2=0x7f0b34144900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:48.730 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.728+0000 7f0b3af0d700 1 -- 172.21.15.32:0/1487163583 shutdown_connections 2020-12-05T17:36:48.730 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.728+0000 7f0b3af0d700 1 -- 172.21.15.32:0/1487163583 wait complete. 2020-12-05T17:36:48.746 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout' 2020-12-05T17:36:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1dbb59e700 1 Processor -- start 2020-12-05T17:36:48.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1dbb59e700 1 -- start start 2020-12-05T17:36:48.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1dbb59e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1dbc0af810 0x7f1dbc143b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:48.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1dbb59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dbc063040 con 0x7f1dbc05f6f0 2020-12-05T17:36:48.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1dbb59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dbc0b24a0 con 0x7f1dbc0aff10 2020-12-05T17:36:48.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1dbb59e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dbc05fad0 con 0x7f1dbc0af810 2020-12-05T17:36:48.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1dba59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1dbc05f6f0 0x7f1dbc0b1d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38320/0 (socket says 172.21.15.32:38320) 2020-12-05T17:36:48.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1dba59c700 1 -- 172.21.15.32:0/1049889000 learned_addr learned my addr 172.21.15.32:0/1049889000 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:36:48.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1db9d9b700 1 --2- 172.21.15.32:0/1049889000 >> v2:172.21.15.61:3300/0 conn(0x7f1dbc0af810 0x7f1dbc143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:48.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1db959a700 1 -- 172.21.15.32:0/1049889000 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729527606 0 0) 0x7f1dbc063040 con 0x7f1dbc05f6f0 2020-12-05T17:36:48.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1db959a700 1 -- 172.21.15.32:0/1049889000 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d9c003230 con 0x7f1dbc05f6f0 2020-12-05T17:36:48.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1db9d9b700 1 -- 172.21.15.32:0/1049889000 >> v1:172.21.15.61:6789/0 conn(0x7f1dbc0aff10 legacy=0x7f1dbc13d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:48.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1db9d9b700 1 -- 172.21.15.32:0/1049889000 >> v1:172.21.15.32:6789/0 conn(0x7f1dbc05f6f0 legacy=0x7f1dbc0b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:48.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1db9d9b700 1 -- 172.21.15.32:0/1049889000 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dbc0b3030 con 0x7f1dbc0af810 2020-12-05T17:36:48.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1db9d9b700 1 --2- 172.21.15.32:0/1049889000 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1dbc0af810 0x7f1dbc143b90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f1dac002f30 tx=0x7f1dac006dc0).ready entity=mon.1 client_cookie=c3d63e873df28ea server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:48.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1db959a700 1 -- 172.21.15.32:0/1049889000 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1dac009800 con 0x7f1dbc0af810 2020-12-05T17:36:48.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1db959a700 1 -- 172.21.15.32:0/1049889000 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dac009960 con 0x7f1dbc0af810 2020-12-05T17:36:48.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1db959a700 1 -- 172.21.15.32:0/1049889000 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1dac009cc0 con 0x7f1dbc0af810 2020-12-05T17:36:48.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1dbb59e700 1 -- 172.21.15.32:0/1049889000 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1dbc0af810 msgr2=0x7f1dbc143b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:48.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1dbb59e700 1 --2- 172.21.15.32:0/1049889000 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1dbc0af810 0x7f1dbc143b90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f1dac002f30 tx=0x7f1dac006dc0).stop 2020-12-05T17:36:48.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1dbb59e700 1 -- 172.21.15.32:0/1049889000 shutdown_connections 2020-12-05T17:36:48.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1dbb59e700 1 --2- 172.21.15.32:0/1049889000 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1dbc0af810 0x7f1dbc143b90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:48.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1dbb59e700 1 -- 172.21.15.32:0/1049889000 >> 172.21.15.32:0/1049889000 conn(0x7f1dbc0604c0 msgr2=0x7f1dbc05c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:48.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1dbb59e700 1 -- 172.21.15.32:0/1049889000 shutdown_connections 2020-12-05T17:36:48.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.860+0000 7f1dbb59e700 1 -- 172.21.15.32:0/1049889000 wait complete. 2020-12-05T17:36:48.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1dbb59e700 1 Processor -- start 2020-12-05T17:36:48.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1dbb59e700 1 -- start start 2020-12-05T17:36:48.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1dbb59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1dbc05f6f0 0x7f1dbc063390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:48.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1dbb59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dbc0af810 0x7f1dbc145c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:48.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1dbb59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1dbc0aff10 0x7f1dbc142e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:48.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1dbb59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dbc13a6d0 con 0x7f1dbc0af810 2020-12-05T17:36:48.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1dbb59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dbc13d8f0 con 0x7f1dbc0aff10 2020-12-05T17:36:48.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1dbb59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dbc143be0 con 0x7f1dbc05f6f0 2020-12-05T17:36:48.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1db9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dbc0af810 0x7f1dbc145c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:48.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1db9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dbc0af810 0x7f1dbc145c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35680/0 (socket says 172.21.15.32:35680) 2020-12-05T17:36:48.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1db9d9b700 1 -- 172.21.15.32:0/1771210120 learned_addr learned my addr 172.21.15.32:0/1771210120 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:48.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1dba59c700 1 --2- 172.21.15.32:0/1771210120 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1dbc05f6f0 0x7f1dbc063390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:48.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1dbad9d700 1 --2- 172.21.15.32:0/1771210120 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1dbc0aff10 0x7f1dbc142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:48.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1db9d9b700 1 -- 172.21.15.32:0/1771210120 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1dbc05f6f0 msgr2=0x7f1dbc063390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:48.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1db9d9b700 1 --2- 172.21.15.32:0/1771210120 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1dbc05f6f0 0x7f1dbc063390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:48.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1db9d9b700 1 -- 172.21.15.32:0/1771210120 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1dbc0aff10 msgr2=0x7f1dbc142e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:48.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1db9d9b700 1 --2- 172.21.15.32:0/1771210120 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1dbc0aff10 0x7f1dbc142e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:48.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1db9d9b700 1 -- 172.21.15.32:0/1771210120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dbc143e80 con 0x7f1dbc0af810 2020-12-05T17:36:48.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1db9d9b700 1 --2- 172.21.15.32:0/1771210120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dbc0af810 0x7f1dbc145c00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f1dbc13f920 tx=0x7f1dac015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:48.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1db37fe700 1 -- 172.21.15.32:0/1771210120 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1dac0119a0 con 0x7f1dbc0af810 2020-12-05T17:36:48.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1db37fe700 1 -- 172.21.15.32:0/1771210120 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dac011b00 con 0x7f1dbc0af810 2020-12-05T17:36:48.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1db37fe700 1 -- 172.21.15.32:0/1771210120 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1dac011e60 con 0x7f1dbc0af810 2020-12-05T17:36:48.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1dbb59e700 1 -- 172.21.15.32:0/1771210120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dbc154b30 con 0x7f1dbc0af810 2020-12-05T17:36:48.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.864+0000 7f1dbb59e700 1 -- 172.21.15.32:0/1771210120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dbc155030 con 0x7f1dbc0af810 2020-12-05T17:36:48.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.868+0000 7f1dbb59e700 1 -- 172.21.15.32:0/1771210120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dbc057050 con 0x7f1dbc0af810 2020-12-05T17:36:48.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.868+0000 7f1db37fe700 1 -- 172.21.15.32:0/1771210120 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1dac024030 con 0x7f1dbc0af810 2020-12-05T17:36:48.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.868+0000 7f1db37fe700 1 --2- 172.21.15.32:0/1771210120 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1da0055330 0x7f1da00577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:48.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.868+0000 7f1db37fe700 1 -- 172.21.15.32:0/1771210120 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1dac06ee00 con 0x7f1dbc0af810 2020-12-05T17:36:48.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.868+0000 7f1dba59c700 1 --2- 172.21.15.32:0/1771210120 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1da0055330 0x7f1da00577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:48.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.868+0000 7f1dba59c700 1 --2- 172.21.15.32:0/1771210120 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1da0055330 0x7f1da00577f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f1da400d000 tx=0x7f1da4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:48.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:48.872+0000 7f1db37fe700 1 -- 172.21.15.32:0/1771210120 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1dac044fa0 con 0x7f1dbc0af810 2020-12-05T17:36:49.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:49.068+0000 7f1dbb59e700 1 -- 172.21.15.32:0/1771210120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]} v 0) v1 -- 0x7f1dbc154cc0 con 0x7f1dbc0af810 2020-12-05T17:36:49.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:49.972+0000 7f1db37fe700 1 -- 172.21.15.32:0/1771210120 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]}]=0 v223) v1 ==== 152+0+0 (secure 0 0 0) 0x7f1dbc154cc0 con 0x7f1dbc0af810 2020-12-05T17:36:49.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:49.976+0000 7f1dbb59e700 1 -- 172.21.15.32:0/1771210120 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1da0055330 msgr2=0x7f1da00577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:49.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:49.976+0000 7f1dbb59e700 1 --2- 172.21.15.32:0/1771210120 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1da0055330 0x7f1da00577f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f1da400d000 tx=0x7f1da4009040).stop 2020-12-05T17:36:49.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:49.976+0000 7f1dbb59e700 1 -- 172.21.15.32:0/1771210120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dbc0af810 msgr2=0x7f1dbc145c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:49.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:49.976+0000 7f1dbb59e700 1 --2- 172.21.15.32:0/1771210120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dbc0af810 0x7f1dbc145c00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f1dbc13f920 tx=0x7f1dac015000).stop 2020-12-05T17:36:49.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:49.976+0000 7f1dbb59e700 1 -- 172.21.15.32:0/1771210120 shutdown_connections 2020-12-05T17:36:49.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:49.976+0000 7f1dbb59e700 1 --2- 172.21.15.32:0/1771210120 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1dbc0aff10 0x7f1dbc142e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:49.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:49.976+0000 7f1dbb59e700 1 --2- 172.21.15.32:0/1771210120 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1da0055330 0x7f1da00577f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:49.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:49.976+0000 7f1dbb59e700 1 --2- 172.21.15.32:0/1771210120 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1dbc05f6f0 0x7f1dbc063390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:49.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:49.976+0000 7f1dbb59e700 1 --2- 172.21.15.32:0/1771210120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dbc0af810 0x7f1dbc145c00 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:49.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:49.976+0000 7f1dbb59e700 1 -- 172.21.15.32:0/1771210120 >> 172.21.15.32:0/1771210120 conn(0x7f1dbc0604c0 msgr2=0x7f1dbc13a970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:49.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:49.976+0000 7f1dbb59e700 1 -- 172.21.15.32:0/1771210120 shutdown_connections 2020-12-05T17:36:49.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:49.976+0000 7f1dbb59e700 1 -- 172.21.15.32:0/1771210120 wait complete. 2020-12-05T17:36:49.993 INFO:tasks.cephfs_test_runner:test_drop_cache_command_dead_timeout (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2020-12-05T17:36:49.994 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_timeout (tasks.cephfs.test_misc.TestCacheDrop) 2020-12-05T17:36:49.994 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout' 2020-12-05T17:36:50.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.108+0000 7f01b2774700 1 Processor -- start 2020-12-05T17:36:50.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.108+0000 7f01b2774700 1 -- start start 2020-12-05T17:36:50.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01b2774700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f01ac144870 0x7f01ac0af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:50.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01b2774700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ac0b29d0 0x7f01ac13fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:50.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01b2774700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ac0afbb0 con 0x7f01ac0af2e0 2020-12-05T17:36:50.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01b2774700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ac05fa20 con 0x7f01ac144870 2020-12-05T17:36:50.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01b2774700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac05fcc0 con 0x7f01ac0b29d0 2020-12-05T17:36:50.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01b0d11700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ac0b29d0 0x7f01ac13fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:50.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01ab7fe700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f01ac144870 0x7f01ac0af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:50.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01b0d11700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ac0b29d0 0x7f01ac13fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:60712/0 (socket says 172.21.15.32:60712) 2020-12-05T17:36:50.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01ab7fe700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f01ac144870 0x7f01ac0af6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:55580/0 (socket says 172.21.15.32:55580) 2020-12-05T17:36:50.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01b0d11700 1 -- 172.21.15.32:0/321334982 learned_addr learned my addr 172.21.15.32:0/321334982 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:50.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01abfff700 1 --1- 172.21.15.32:0/321334982 >> v1:172.21.15.61:6789/0 conn(0x7f01ac0af2e0 0x7f01ac0b22c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:50974/0 (socket says 172.21.15.32:50974) 2020-12-05T17:36:50.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01ab7fe700 1 -- 172.21.15.32:0/321334982 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ac0b29d0 msgr2=0x7f01ac13fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:36:50.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01ab7fe700 1 --2- 172.21.15.32:0/321334982 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ac0b29d0 0x7f01ac13fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:50.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01ab7fe700 1 -- 172.21.15.32:0/321334982 >> v1:172.21.15.61:6789/0 conn(0x7f01ac0af2e0 legacy=0x7f01ac0b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:50.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01ab7fe700 1 -- 172.21.15.32:0/321334982 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac0b2e60 con 0x7f01ac144870 2020-12-05T17:36:50.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01b0d11700 1 --2- 172.21.15.32:0/321334982 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ac0b29d0 0x7f01ac13fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:36:50.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01ab7fe700 1 --2- 172.21.15.32:0/321334982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01ac144870 0x7f01ac0af6c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f01ac0b21a0 tx=0x7f01a0006dc0).ready entity=mon.1 client_cookie=519e079af593de1f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:50.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01aaffd700 1 -- 172.21.15.32:0/321334982 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01a0009720 con 0x7f01ac144870 2020-12-05T17:36:50.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01aaffd700 1 -- 172.21.15.32:0/321334982 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a0009880 con 0x7f01ac144870 2020-12-05T17:36:50.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01aaffd700 1 -- 172.21.15.32:0/321334982 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01a0009be0 con 0x7f01ac144870 2020-12-05T17:36:50.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01b2774700 1 -- 172.21.15.32:0/321334982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01ac144870 msgr2=0x7f01ac0af6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:50.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01b2774700 1 --2- 172.21.15.32:0/321334982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01ac144870 0x7f01ac0af6c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f01ac0b21a0 tx=0x7f01a0006dc0).stop 2020-12-05T17:36:50.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01b2774700 1 -- 172.21.15.32:0/321334982 shutdown_connections 2020-12-05T17:36:50.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01b2774700 1 --2- 172.21.15.32:0/321334982 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ac0b29d0 0x7f01ac13fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:50.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01b2774700 1 --2- 172.21.15.32:0/321334982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01ac144870 0x7f01ac0af6c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:50.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01b2774700 1 -- 172.21.15.32:0/321334982 >> 172.21.15.32:0/321334982 conn(0x7f01ac0604c0 msgr2=0x7f01ac05c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:50.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01b2774700 1 -- 172.21.15.32:0/321334982 shutdown_connections 2020-12-05T17:36:50.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.112+0000 7f01b2774700 1 -- 172.21.15.32:0/321334982 wait complete. 2020-12-05T17:36:50.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.116+0000 7f01b2774700 1 Processor -- start 2020-12-05T17:36:50.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.116+0000 7f01b2774700 1 -- start start 2020-12-05T17:36:50.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.116+0000 7f01b2774700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ac0af2e0 0x7f01ac150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:50.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.116+0000 7f01b2774700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01ac0b29d0 0x7f01ac110a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:50.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.116+0000 7f01b2774700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01ac144870 0x7f01ac063f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:50.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.116+0000 7f01b2774700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac1513c0 con 0x7f01ac144870 2020-12-05T17:36:50.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.116+0000 7f01b2774700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac110f30 con 0x7f01ac0b29d0 2020-12-05T17:36:50.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.116+0000 7f01b2774700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac151a50 con 0x7f01ac0af2e0 2020-12-05T17:36:50.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.116+0000 7f01abfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ac0af2e0 0x7f01ac150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:50.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.116+0000 7f01b0d11700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01ac144870 0x7f01ac063f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:50.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.116+0000 7f01abfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ac0af2e0 0x7f01ac150ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:60714/0 (socket says 172.21.15.32:60714) 2020-12-05T17:36:50.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.116+0000 7f01b0d11700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01ac144870 0x7f01ac063f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35694/0 (socket says 172.21.15.32:35694) 2020-12-05T17:36:50.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.116+0000 7f01abfff700 1 -- 172.21.15.32:0/2207979761 learned_addr learned my addr 172.21.15.32:0/2207979761 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:50.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.116+0000 7f01ab7fe700 1 --2- 172.21.15.32:0/2207979761 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01ac0b29d0 0x7f01ac110a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:50.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.116+0000 7f01b0d11700 1 -- 172.21.15.32:0/2207979761 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ac0af2e0 msgr2=0x7f01ac150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:50.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.116+0000 7f01b0d11700 1 --2- 172.21.15.32:0/2207979761 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ac0af2e0 0x7f01ac150ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:50.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.116+0000 7f01b0d11700 1 -- 172.21.15.32:0/2207979761 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01ac0b29d0 msgr2=0x7f01ac110a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:50.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.116+0000 7f01b0d11700 1 --2- 172.21.15.32:0/2207979761 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01ac0b29d0 0x7f01ac110a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:50.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.116+0000 7f01b0d11700 1 -- 172.21.15.32:0/2207979761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac1515a0 con 0x7f01ac144870 2020-12-05T17:36:50.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.116+0000 7f01abfff700 1 --2- 172.21.15.32:0/2207979761 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ac0af2e0 0x7f01ac150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:36:50.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.116+0000 7f01b0d11700 1 --2- 172.21.15.32:0/2207979761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01ac144870 0x7f01ac063f80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f01ac065d40 tx=0x7f01a4009270).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:50.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.120+0000 7f01a97fa700 1 -- 172.21.15.32:0/2207979761 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01a4015070 con 0x7f01ac144870 2020-12-05T17:36:50.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.120+0000 7f01b2774700 1 -- 172.21.15.32:0/2207979761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ac10ef80 con 0x7f01ac144870 2020-12-05T17:36:50.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.120+0000 7f01a97fa700 1 -- 172.21.15.32:0/2207979761 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a401a610 con 0x7f01ac144870 2020-12-05T17:36:50.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.120+0000 7f01a97fa700 1 -- 172.21.15.32:0/2207979761 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01a401a970 con 0x7f01ac144870 2020-12-05T17:36:50.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.120+0000 7f01b2774700 1 -- 172.21.15.32:0/2207979761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ac10f490 con 0x7f01ac144870 2020-12-05T17:36:50.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.120+0000 7f01b2774700 1 -- 172.21.15.32:0/2207979761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ac151730 con 0x7f01ac144870 2020-12-05T17:36:50.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.120+0000 7f01a97fa700 1 -- 172.21.15.32:0/2207979761 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f01a401aad0 con 0x7f01ac144870 2020-12-05T17:36:50.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.120+0000 7f01a97fa700 1 --2- 172.21.15.32:0/2207979761 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f018c055500 0x7f018c0579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:50.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.120+0000 7f01a97fa700 1 -- 172.21.15.32:0/2207979761 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f01a4072d40 con 0x7f01ac144870 2020-12-05T17:36:50.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.120+0000 7f01abfff700 1 --2- 172.21.15.32:0/2207979761 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f018c055500 0x7f018c0579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:50.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.124+0000 7f01abfff700 1 --2- 172.21.15.32:0/2207979761 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f018c055500 0x7f018c0579c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f019c00d000 tx=0x7f019c007c10).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:50.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.128+0000 7f01a97fa700 1 -- 172.21.15.32:0/2207979761 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f01a4048d50 con 0x7f01ac144870 2020-12-05T17:36:50.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.320+0000 7f01b2774700 1 -- 172.21.15.32:0/2207979761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]} v 0) v1 -- 0x7f01ac057050 con 0x7f01ac144870 2020-12-05T17:36:50.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.976+0000 7f01a97fa700 1 -- 172.21.15.32:0/2207979761 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]}]=0 v224) v1 ==== 150+0+0 (secure 0 0 0) 0x7f01a404c370 con 0x7f01ac144870 2020-12-05T17:36:50.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.976+0000 7f01b2774700 1 -- 172.21.15.32:0/2207979761 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f018c055500 msgr2=0x7f018c0579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:50.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.976+0000 7f01b2774700 1 --2- 172.21.15.32:0/2207979761 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f018c055500 0x7f018c0579c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f019c00d000 tx=0x7f019c007c10).stop 2020-12-05T17:36:50.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.976+0000 7f01b2774700 1 -- 172.21.15.32:0/2207979761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01ac144870 msgr2=0x7f01ac063f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:50.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.976+0000 7f01b2774700 1 --2- 172.21.15.32:0/2207979761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01ac144870 0x7f01ac063f80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f01ac065d40 tx=0x7f01a4009270).stop 2020-12-05T17:36:50.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.980+0000 7f01b2774700 1 -- 172.21.15.32:0/2207979761 shutdown_connections 2020-12-05T17:36:50.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.980+0000 7f01b2774700 1 --2- 172.21.15.32:0/2207979761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01ac144870 0x7f01ac063f80 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:50.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.980+0000 7f01b2774700 1 --2- 172.21.15.32:0/2207979761 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f018c055500 0x7f018c0579c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:50.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.980+0000 7f01b2774700 1 --2- 172.21.15.32:0/2207979761 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ac0af2e0 0x7f01ac150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:50.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.980+0000 7f01b2774700 1 --2- 172.21.15.32:0/2207979761 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01ac0b29d0 0x7f01ac110a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.980+0000 7f01b2774700 1 -- 172.21.15.32:0/2207979761 >> 172.21.15.32:0/2207979761 conn(0x7f01ac0604c0 msgr2=0x7f01ac0ae2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.980+0000 7f01b2774700 1 -- 172.21.15.32:0/2207979761 shutdown_connections 2020-12-05T17:36:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:50.980+0000 7f01b2774700 1 -- 172.21.15.32:0/2207979761 wait complete. 2020-12-05T17:36:50.995 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-12-05T17:36:51.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.116+0000 7f925a8b3700 1 Processor -- start 2020-12-05T17:36:51.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.116+0000 7f925a8b3700 1 -- start start 2020-12-05T17:36:51.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f925a8b3700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f92540b2830 0x7f92540b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:51.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f925a8b3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f925405fd70 0x7f9254063fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:51.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f925a8b3700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92540b0840 con 0x7f925413fcc0 2020-12-05T17:36:51.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f925a8b3700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f925405f710 con 0x7f92540b2830 2020-12-05T17:36:51.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f925a8b3700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f925405f9b0 con 0x7f925405fd70 2020-12-05T17:36:51.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f9253fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f92540b2830 0x7f92540b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:51.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f924b7fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f925405fd70 0x7f9254063fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:51.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f9253fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f92540b2830 0x7f92540b0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:55592/0 (socket says 172.21.15.32:55592) 2020-12-05T17:36:51.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f924b7fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f925405fd70 0x7f9254063fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:60724/0 (socket says 172.21.15.32:60724) 2020-12-05T17:36:51.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f9253fff700 1 -- 172.21.15.32:0/3619165900 learned_addr learned my addr 172.21.15.32:0/3619165900 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:51.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f9258e50700 1 --1- 172.21.15.32:0/3619165900 >> v1:172.21.15.61:6789/0 conn(0x7f925413fcc0 0x7f9254143c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:50992/0 (socket says 172.21.15.32:50992) 2020-12-05T17:36:51.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f9253fff700 1 -- 172.21.15.32:0/3619165900 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f925405fd70 msgr2=0x7f9254063fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:36:51.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f9253fff700 1 --2- 172.21.15.32:0/3619165900 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f925405fd70 0x7f9254063fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:51.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f9253fff700 1 -- 172.21.15.32:0/3619165900 >> v1:172.21.15.61:6789/0 conn(0x7f925413fcc0 legacy=0x7f9254143c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:51.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f9253fff700 1 -- 172.21.15.32:0/3619165900 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92540b2c10 con 0x7f92540b2830 2020-12-05T17:36:51.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f9253fff700 1 --2- 172.21.15.32:0/3619165900 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92540b2830 0x7f92540b0c70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f9244002f30 tx=0x7f9244006e80).ready entity=mon.1 client_cookie=c4fa0a1a6375940d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:51.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f92537fe700 1 -- 172.21.15.32:0/3619165900 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92440097f0 con 0x7f92540b2830 2020-12-05T17:36:51.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f92537fe700 1 -- 172.21.15.32:0/3619165900 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9244009950 con 0x7f92540b2830 2020-12-05T17:36:51.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f92537fe700 1 -- 172.21.15.32:0/3619165900 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9244009cb0 con 0x7f92540b2830 2020-12-05T17:36:51.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f925a8b3700 1 -- 172.21.15.32:0/3619165900 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92540b2830 msgr2=0x7f92540b0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:51.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f925a8b3700 1 --2- 172.21.15.32:0/3619165900 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92540b2830 0x7f92540b0c70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f9244002f30 tx=0x7f9244006e80).stop 2020-12-05T17:36:51.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f925a8b3700 1 -- 172.21.15.32:0/3619165900 shutdown_connections 2020-12-05T17:36:51.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f925a8b3700 1 --2- 172.21.15.32:0/3619165900 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92540b2830 0x7f92540b0c70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:51.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f925a8b3700 1 --2- 172.21.15.32:0/3619165900 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f925405fd70 0x7f9254063fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:51.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f925a8b3700 1 -- 172.21.15.32:0/3619165900 >> 172.21.15.32:0/3619165900 conn(0x7f9254062310 msgr2=0x7f92540626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:51.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f925a8b3700 1 -- 172.21.15.32:0/3619165900 shutdown_connections 2020-12-05T17:36:51.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.120+0000 7f925a8b3700 1 -- 172.21.15.32:0/3619165900 wait complete. 2020-12-05T17:36:51.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.124+0000 7f925a8b3700 1 Processor -- start 2020-12-05T17:36:51.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.124+0000 7f925a8b3700 1 -- start start 2020-12-05T17:36:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.124+0000 7f925a8b3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f925405fd70 0x7f92541442b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.124+0000 7f925a8b3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92540b2830 0x7f925413b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.124+0000 7f925a8b3700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f925413fcc0 0x7f925413b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.124+0000 7f925a8b3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9254143a70 con 0x7f92540b2830 2020-12-05T17:36:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.124+0000 7f925a8b3700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9254143d30 con 0x7f925413fcc0 2020-12-05T17:36:51.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.124+0000 7f925a8b3700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92541437d0 con 0x7f925405fd70 2020-12-05T17:36:51.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.128+0000 7f924b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92540b2830 0x7f925413b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:51.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.128+0000 7f924b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92540b2830 0x7f925413b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35708/0 (socket says 172.21.15.32:35708) 2020-12-05T17:36:51.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.128+0000 7f924b7fe700 1 -- 172.21.15.32:0/2614604044 learned_addr learned my addr 172.21.15.32:0/2614604044 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:51.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.128+0000 7f9253fff700 1 --2- 172.21.15.32:0/2614604044 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f925405fd70 0x7f92541442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:51.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.128+0000 7f9258e50700 1 --2- 172.21.15.32:0/2614604044 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f925413fcc0 0x7f925413b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:51.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.128+0000 7f924b7fe700 1 -- 172.21.15.32:0/2614604044 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f925405fd70 msgr2=0x7f92541442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:51.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.128+0000 7f924b7fe700 1 --2- 172.21.15.32:0/2614604044 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f925405fd70 0x7f92541442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:51.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.128+0000 7f924b7fe700 1 -- 172.21.15.32:0/2614604044 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f925413fcc0 msgr2=0x7f925413b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.128+0000 7f924b7fe700 1 --2- 172.21.15.32:0/2614604044 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f925413fcc0 0x7f925413b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.128+0000 7f924b7fe700 1 -- 172.21.15.32:0/2614604044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92541428a0 con 0x7f92540b2830 2020-12-05T17:36:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.128+0000 7f924b7fe700 1 --2- 172.21.15.32:0/2614604044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92540b2830 0x7f925413b250 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7f923c00c000 tx=0x7f923c006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.128+0000 7f9251ffb700 1 -- 172.21.15.32:0/2614604044 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f923c00ca10 con 0x7f92540b2830 2020-12-05T17:36:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.128+0000 7f9251ffb700 1 -- 172.21.15.32:0/2614604044 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923c00cb70 con 0x7f92540b2830 2020-12-05T17:36:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.128+0000 7f925a8b3700 1 -- 172.21.15.32:0/2614604044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9254142d40 con 0x7f92540b2830 2020-12-05T17:36:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.128+0000 7f9251ffb700 1 -- 172.21.15.32:0/2614604044 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f923c0123f0 con 0x7f92540b2830 2020-12-05T17:36:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.128+0000 7f925a8b3700 1 -- 172.21.15.32:0/2614604044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f925413bc30 con 0x7f92540b2830 2020-12-05T17:36:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.128+0000 7f9251ffb700 1 -- 172.21.15.32:0/2614604044 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f923c012550 con 0x7f92540b2830 2020-12-05T17:36:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.128+0000 7f925a8b3700 1 -- 172.21.15.32:0/2614604044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92541425d0 con 0x7f92540b2830 2020-12-05T17:36:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.128+0000 7f9251ffb700 1 --2- 172.21.15.32:0/2614604044 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9238055500 0x7f92380579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.128+0000 7f9251ffb700 1 -- 172.21.15.32:0/2614604044 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f923c06e380 con 0x7f92540b2830 2020-12-05T17:36:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.128+0000 7f9253fff700 1 --2- 172.21.15.32:0/2614604044 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9238055500 0x7f92380579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.132+0000 7f9253fff700 1 --2- 172.21.15.32:0/2614604044 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9238055500 0x7f92380579c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f924401c000 tx=0x7f9244015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:51.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.140+0000 7f9251ffb700 1 -- 172.21.15.32:0/2614604044 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f92541425d0 con 0x7f92540b2830 2020-12-05T17:36:51.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.284+0000 7f925a8b3700 1 -- 172.21.15.32:0/2614604044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f92541425d0 con 0x7f92540b2830 2020-12-05T17:36:51.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.292+0000 7f9251ffb700 1 -- 172.21.15.32:0/2614604044 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f92541425d0 con 0x7f92540b2830 2020-12-05T17:36:51.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.292+0000 7f925a8b3700 1 -- 172.21.15.32:0/2614604044 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9238055500 msgr2=0x7f92380579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:51.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.292+0000 7f925a8b3700 1 --2- 172.21.15.32:0/2614604044 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9238055500 0x7f92380579c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f924401c000 tx=0x7f9244015000).stop 2020-12-05T17:36:51.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.292+0000 7f925a8b3700 1 -- 172.21.15.32:0/2614604044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92540b2830 msgr2=0x7f925413b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:51.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.292+0000 7f925a8b3700 1 --2- 172.21.15.32:0/2614604044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92540b2830 0x7f925413b250 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7f923c00c000 tx=0x7f923c006dc0).stop 2020-12-05T17:36:51.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.292+0000 7f925a8b3700 1 -- 172.21.15.32:0/2614604044 shutdown_connections 2020-12-05T17:36:51.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.292+0000 7f925a8b3700 1 --2- 172.21.15.32:0/2614604044 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f925413fcc0 0x7f925413b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:51.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.292+0000 7f925a8b3700 1 --2- 172.21.15.32:0/2614604044 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9238055500 0x7f92380579c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:51.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.292+0000 7f925a8b3700 1 --2- 172.21.15.32:0/2614604044 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f925405fd70 0x7f92541442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:51.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.292+0000 7f925a8b3700 1 --2- 172.21.15.32:0/2614604044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92540b2830 0x7f925413b250 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:51.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.292+0000 7f925a8b3700 1 -- 172.21.15.32:0/2614604044 >> 172.21.15.32:0/2614604044 conn(0x7f9254062310 msgr2=0x7f92540b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:51.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.292+0000 7f925a8b3700 1 -- 172.21.15.32:0/2614604044 shutdown_connections 2020-12-05T17:36:51.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.292+0000 7f925a8b3700 1 -- 172.21.15.32:0/2614604044 wait complete. 2020-12-05T17:36:51.304 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:36:51.304 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:36:51.322 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:36:51.338 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:36:51.356 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:36:51.374 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:36:51.391 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:36:51.391 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:36:51.392 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:36:51.392 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:36:51.403 INFO:teuthology.orchestra.run.smithi036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-12-05T17:36:51.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:36:51.405 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-12-05T17:36:51.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:36:51.406 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:36:51.406 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:36:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-12-05T17:36:51.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:36:51.414 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-12-05T17:36:51.414 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:36:51.415 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:36:51.415 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:36:51.423 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:36:51.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:36:51.425 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2020-12-05T17:36:51.425 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-12-05T17:36:51.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.548+0000 7f9ec0e6c700 1 Processor -- start 2020-12-05T17:36:51.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.548+0000 7f9ec0e6c700 1 -- start start 2020-12-05T17:36:51.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.548+0000 7f9ec0e6c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ebc0afe50 0x7f9ebc146a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:51.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.548+0000 7f9ec0e6c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9ebc0b3aa0 0x7f9ebc1429e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:51.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.548+0000 7f9ec0e6c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ebc0b0230 con 0x7f9ebc0b1880 2020-12-05T17:36:51.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.548+0000 7f9ec0e6c700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ebc0b6a50 con 0x7f9ebc0b3aa0 2020-12-05T17:36:51.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.548+0000 7f9ec0e6c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc146080 con 0x7f9ebc0afe50 2020-12-05T17:36:51.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.548+0000 7f9eba59c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ebc0afe50 0x7f9ebc146a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:51.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.548+0000 7f9eb9d9b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9ebc0b3aa0 0x7f9ebc1429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:51.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.548+0000 7f9ebad9d700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f9ebc0b1880 0x7f9ebc142ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:51006/0 (socket says 172.21.15.32:51006) 2020-12-05T17:36:51.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.548+0000 7f9ebad9d700 1 -- 172.21.15.32:0/3229943111 learned_addr learned my addr 172.21.15.32:0/3229943111 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:36:51.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.548+0000 7f9eba59c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ebc0afe50 0x7f9ebc146a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:60736/0 (socket says 172.21.15.32:60736) 2020-12-05T17:36:51.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.548+0000 7f9eb9d9b700 1 -- 172.21.15.32:0/3229943111 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ebc0afe50 msgr2=0x7f9ebc146a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:36:51.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.548+0000 7f9eb9d9b700 1 --2- 172.21.15.32:0/3229943111 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ebc0afe50 0x7f9ebc146a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:51.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.548+0000 7f9eb9d9b700 1 -- 172.21.15.32:0/3229943111 >> v1:172.21.15.61:6789/0 conn(0x7f9ebc0b1880 legacy=0x7f9ebc142ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:51.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.548+0000 7f9eb959a700 1 -- 172.21.15.32:0/3229943111 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1983974847 0 0) 0x7f9ebc0b0230 con 0x7f9ebc0b1880 2020-12-05T17:36:51.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.548+0000 7f9eb9d9b700 1 -- 172.21.15.32:0/3229943111 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ebc146320 con 0x7f9ebc0b3aa0 2020-12-05T17:36:51.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.548+0000 7f9eb9d9b700 1 --2- 172.21.15.32:0/3229943111 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ebc0b3aa0 0x7f9ebc1429e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f9eb0002f30 tx=0x7f9eb0006dc0).ready entity=mon.1 client_cookie=9a61b4769f99794d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:51.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.552+0000 7f9eb959a700 1 -- 172.21.15.32:0/3229943111 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9eb0009760 con 0x7f9ebc0b3aa0 2020-12-05T17:36:51.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.552+0000 7f9eb959a700 1 -- 172.21.15.32:0/3229943111 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb00098c0 con 0x7f9ebc0b3aa0 2020-12-05T17:36:51.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.552+0000 7f9eb959a700 1 -- 172.21.15.32:0/3229943111 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9eb0009c20 con 0x7f9ebc0b3aa0 2020-12-05T17:36:51.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.552+0000 7f9ec0e6c700 1 -- 172.21.15.32:0/3229943111 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ebc0b3aa0 msgr2=0x7f9ebc1429e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:51.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.552+0000 7f9ec0e6c700 1 --2- 172.21.15.32:0/3229943111 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ebc0b3aa0 0x7f9ebc1429e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f9eb0002f30 tx=0x7f9eb0006dc0).stop 2020-12-05T17:36:51.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.552+0000 7f9ec0e6c700 1 -- 172.21.15.32:0/3229943111 shutdown_connections 2020-12-05T17:36:51.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.552+0000 7f9ec0e6c700 1 --2- 172.21.15.32:0/3229943111 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ebc0afe50 0x7f9ebc146a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:51.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.552+0000 7f9ec0e6c700 1 --2- 172.21.15.32:0/3229943111 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ebc0b3aa0 0x7f9ebc1429e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:51.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.552+0000 7f9ec0e6c700 1 -- 172.21.15.32:0/3229943111 >> 172.21.15.32:0/3229943111 conn(0x7f9ebc0b9100 msgr2=0x7f9ebc0b9510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:51.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.552+0000 7f9ec0e6c700 1 -- 172.21.15.32:0/3229943111 shutdown_connections 2020-12-05T17:36:51.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.552+0000 7f9ec0e6c700 1 -- 172.21.15.32:0/3229943111 wait complete. 2020-12-05T17:36:51.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.552+0000 7f9ec0e6c700 1 Processor -- start 2020-12-05T17:36:51.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.552+0000 7f9ec0e6c700 1 -- start start 2020-12-05T17:36:51.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.556+0000 7f9ec0e6c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ebc0afe50 0x7f9ebc141b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:51.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.556+0000 7f9ec0e6c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ebc0b1880 0x7f9ebc151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:51.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.556+0000 7f9ec0e6c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ebc0b3aa0 0x7f9ebc152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:51.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.556+0000 7f9ec0e6c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc152530 con 0x7f9ebc0b1880 2020-12-05T17:36:51.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.556+0000 7f9ec0e6c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc1409a0 con 0x7f9ebc0b3aa0 2020-12-05T17:36:51.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.556+0000 7f9ec0e6c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc140c40 con 0x7f9ebc0afe50 2020-12-05T17:36:51.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.556+0000 7f9eb9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ebc0b1880 0x7f9ebc151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:51.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.556+0000 7f9eb9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ebc0b1880 0x7f9ebc151b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35722/0 (socket says 172.21.15.32:35722) 2020-12-05T17:36:51.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.556+0000 7f9eb9d9b700 1 -- 172.21.15.32:0/1127753266 learned_addr learned my addr 172.21.15.32:0/1127753266 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:51.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.556+0000 7f9ebad9d700 1 --2- 172.21.15.32:0/1127753266 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ebc0b3aa0 0x7f9ebc152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:51.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.556+0000 7f9eba59c700 1 --2- 172.21.15.32:0/1127753266 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ebc0afe50 0x7f9ebc141b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:51.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.556+0000 7f9eb9d9b700 1 -- 172.21.15.32:0/1127753266 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ebc0afe50 msgr2=0x7f9ebc141b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:51.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.556+0000 7f9eb9d9b700 1 --2- 172.21.15.32:0/1127753266 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ebc0afe50 0x7f9ebc141b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:51.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.556+0000 7f9eb9d9b700 1 -- 172.21.15.32:0/1127753266 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ebc0b3aa0 msgr2=0x7f9ebc152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:51.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.556+0000 7f9eb9d9b700 1 --2- 172.21.15.32:0/1127753266 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ebc0b3aa0 0x7f9ebc152040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:51.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.556+0000 7f9eb9d9b700 1 -- 172.21.15.32:0/1127753266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ebc140ee0 con 0x7f9ebc0b1880 2020-12-05T17:36:51.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.556+0000 7f9eb9d9b700 1 --2- 172.21.15.32:0/1127753266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ebc0b1880 0x7f9ebc151b50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f9eb0002f30 tx=0x7f9eb0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:51.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.556+0000 7f9eab7fe700 1 -- 172.21.15.32:0/1127753266 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9eb0011950 con 0x7f9ebc0b1880 2020-12-05T17:36:51.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.556+0000 7f9eab7fe700 1 -- 172.21.15.32:0/1127753266 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb0011ab0 con 0x7f9ebc0b1880 2020-12-05T17:36:51.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.556+0000 7f9ec0e6c700 1 -- 172.21.15.32:0/1127753266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ebc0b5d90 con 0x7f9ebc0b1880 2020-12-05T17:36:51.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.556+0000 7f9eab7fe700 1 -- 172.21.15.32:0/1127753266 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9eb0011e10 con 0x7f9ebc0b1880 2020-12-05T17:36:51.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.556+0000 7f9ec0e6c700 1 -- 172.21.15.32:0/1127753266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ebc142ef0 con 0x7f9ebc0b1880 2020-12-05T17:36:51.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.556+0000 7f9ec0e6c700 1 -- 172.21.15.32:0/1127753266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ebc057050 con 0x7f9ebc0b1880 2020-12-05T17:36:51.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.560+0000 7f9eab7fe700 1 -- 172.21.15.32:0/1127753266 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9eb0024030 con 0x7f9ebc0b1880 2020-12-05T17:36:51.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.560+0000 7f9eab7fe700 1 --2- 172.21.15.32:0/1127753266 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9e980554e0 0x7f9e980579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:51.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.560+0000 7f9eab7fe700 1 -- 172.21.15.32:0/1127753266 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9eb006eeb0 con 0x7f9ebc0b1880 2020-12-05T17:36:51.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.560+0000 7f9eba59c700 1 --2- 172.21.15.32:0/1127753266 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9e980554e0 0x7f9e980579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:51.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.560+0000 7f9eba59c700 1 --2- 172.21.15.32:0/1127753266 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9e980554e0 0x7f9e980579a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f9eac00b000 tx=0x7f9eac0091a0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:51.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.564+0000 7f9eab7fe700 1 -- 172.21.15.32:0/1127753266 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9eb0045050 con 0x7f9ebc0b1880 2020-12-05T17:36:51.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.712+0000 7f9ec0e6c700 1 -- 172.21.15.32:0/1127753266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f9ebc069690 con 0x7f9ebc0b1880 2020-12-05T17:36:51.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.712+0000 7f9eab7fe700 1 -- 172.21.15.32:0/1127753266 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v54) v1 ==== 81+0+34039 (secure 0 0 0) 0x7f9eb0022090 con 0x7f9ebc0b1880 2020-12-05T17:36:51.714 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:36:51.715 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:36:51.715 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 54, 2020-12-05T17:36:51.715 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", 2020-12-05T17:36:51.715 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2020-12-05T17:31:20.975238+0000", 2020-12-05T17:36:51.716 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2020-12-05T17:35:25.399857+0000", 2020-12-05T17:36:51.716 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:36:51.716 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:36:51.716 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-12-05T17:36:51.716 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2020-12-05T17:36:51.717 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2020-12-05T17:36:51.717 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2020-12-05T17:36:51.717 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2020-12-05T17:36:51.717 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2020-12-05T17:36:51.717 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2020-12-05T17:36:51.718 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:36:51.718 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2020-12-05T17:36:51.718 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2020-12-05T17:36:51.718 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-12-05T17:36:51.719 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2020-12-05T17:36:51.719 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2020-12-05T17:36:51.719 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 10, 2020-12-05T17:36:51.719 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2020-12-05T17:36:51.719 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2020-12-05T17:36:51.720 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2020-12-05T17:36:51.720 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2020-12-05T17:36:51.720 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2020-12-05T17:36:51.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.721 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2020-12-05T17:36:51.721 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2020-12-05T17:36:51.721 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:24.858732+0000", 2020-12-05T17:36:51.721 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:36:51.721 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:36:51.722 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:36:51.722 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:36:51.722 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:36:51.722 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:36:51.722 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:36:51.723 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:36:51.723 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2020-12-05T17:36:51.723 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2020-12-05T17:36:51.723 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2020-12-05T17:36:51.723 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2020-12-05T17:36:51.724 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2020-12-05T17:36:51.724 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:36:51.724 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:36:51.724 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:36:51.724 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:36:51.725 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:36:51.725 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:36:51.725 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:36:51.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.726 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "16", 2020-12-05T17:36:51.726 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:36:51.726 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:36:51.726 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:36:51.726 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:36:51.727 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:36:51.727 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:36:51.727 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:36:51.727 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:36:51.727 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:36:51.728 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:36:51.728 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:36:51.728 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:36:51.728 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:36:51.728 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:36:51.729 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:36:51.729 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:36:51.729 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:36:51.729 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:36:51.730 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:36:51.730 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:36:51.731 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:36:51.731 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:36:51.731 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:36:51.731 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:36:51.731 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:36:51.732 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:36:51.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.732 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:36:51.732 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:36:51.732 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:36:51.733 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:36:51.733 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:36:51.733 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:36:51.733 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:36:51.734 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:36:51.734 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:36:51.734 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:36:51.734 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:36:51.735 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:36:51.735 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2020-12-05T17:36:51.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.735 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:36:51.735 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2020-12-05T17:36:51.736 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.736 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2020-12-05T17:36:51.736 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2020-12-05T17:36:51.737 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:33.305595+0000", 2020-12-05T17:36:51.737 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:36:51.737 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:36:51.737 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:36:51.738 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:36:51.738 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:36:51.738 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:36:51.738 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:36:51.739 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:36:51.739 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:36:51.739 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:36:51.739 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:36:51.740 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:36:51.740 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:36:51.740 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:36:51.740 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:36:51.740 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:36:51.741 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:36:51.741 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:36:51.741 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:36:51.741 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:36:51.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.742 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "19", 2020-12-05T17:36:51.742 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:36:51.742 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:36:51.742 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:36:51.743 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:36:51.743 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:36:51.743 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:36:51.743 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:36:51.743 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:36:51.744 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:36:51.744 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:36:51.744 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:36:51.744 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:36:51.744 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:36:51.745 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:36:51.745 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:36:51.745 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:36:51.745 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:36:51.746 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:36:51.746 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:36:51.746 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:36:51.746 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:36:51.747 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:36:51.747 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:36:51.747 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:36:51.747 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:36:51.747 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:36:51.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.748 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:36:51.748 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:36:51.748 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:36:51.748 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:36:51.749 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:36:51.749 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:36:51.749 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:36:51.749 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:36:51.749 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:36:51.750 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:36:51.750 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:36:51.750 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:36:51.750 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:36:51.750 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2020-12-05T17:36:51.751 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.751 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 9, 2020-12-05T17:36:51.751 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2020-12-05T17:36:51.752 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:35:20.867930+0000", 2020-12-05T17:36:51.752 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:36:51.752 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:36:51.752 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:36:51.752 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:36:51.753 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:36:51.753 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:36:51.753 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:36:51.753 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:36:51.753 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:36:51.754 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:36:51.754 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:36:51.754 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:36:51.754 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:36:51.755 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:36:51.755 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:36:51.755 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:36:51.755 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:36:51.755 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:36:51.756 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:36:51.756 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:36:51.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.756 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "53", 2020-12-05T17:36:51.756 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:36:51.757 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:36:51.757 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:36:51.757 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:36:51.757 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:36:51.757 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:36:51.758 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:36:51.758 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:36:51.758 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:36:51.758 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:36:51.758 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:36:51.759 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:36:51.759 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:36:51.759 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:36:51.759 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:36:51.759 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:36:51.760 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:36:51.760 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:36:51.760 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:36:51.760 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:36:51.761 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:36:51.761 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:36:51.761 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:36:51.761 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:36:51.761 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:36:51.762 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:36:51.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.762 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:36:51.762 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:36:51.762 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:36:51.763 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:36:51.763 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:36:51.763 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:36:51.763 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:36:51.763 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:36:51.764 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:36:51.764 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:36:51.764 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:36:51.764 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:36:51.765 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2020-12-05T17:36:51.765 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2020-12-05T17:36:51.765 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2020-12-05T17:36:51.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.765 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:36:51.766 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:36:51.766 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2020-12-05T17:36:51.766 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.766 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.767 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 10, 2020-12-05T17:36:51.767 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2020-12-05T17:36:51.767 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:35:22.139660+0000", 2020-12-05T17:36:51.768 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:36:51.768 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:36:51.768 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:36:51.768 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:36:51.768 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:36:51.769 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:36:51.769 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:36:51.769 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:36:51.769 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:36:51.770 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:36:51.770 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:36:51.770 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:36:51.770 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:36:51.770 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:36:51.771 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:36:51.771 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:36:51.771 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:36:51.771 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:36:51.771 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:36:51.772 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:36:51.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.772 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "54", 2020-12-05T17:36:51.772 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:36:51.772 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:36:51.773 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:36:51.773 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:36:51.773 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:36:51.773 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:36:51.773 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:36:51.774 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:36:51.774 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:36:51.774 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:36:51.774 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:36:51.775 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:36:51.775 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:36:51.775 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:36:51.775 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:36:51.775 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:36:51.776 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:36:51.776 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:36:51.776 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:36:51.776 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:36:51.776 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:36:51.777 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:36:51.777 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:36:51.777 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:36:51.777 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:36:51.778 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:36:51.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.778 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:36:51.778 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:36:51.778 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:36:51.779 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:36:51.779 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:36:51.779 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:36:51.779 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:36:51.780 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:36:51.780 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:36:51.780 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:36:51.780 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:36:51.780 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:36:51.781 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:36:51.781 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:36:51.781 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2020-12-05T17:36:51.781 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.781 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.782 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.782 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:36:51.782 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2020-12-05T17:36:51.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.782 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:36:51.783 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "b941127b-0a49-4e06-8d11-17794ff8b013", 2020-12-05T17:36:51.783 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:36:51.783 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:36:51.783 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:36:51.783 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:36:51.784 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:36:51.784 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:36:51.784 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:36:51.784 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 51, 2020-12-05T17:36:51.785 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:36:51.785 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:36:51.785 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:36:51.785 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.786 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.786 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2020-12-05T17:36:51.786 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:36:51.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.787 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2020-12-05T17:36:51.787 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:36:51.787 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.788 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.788 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:36:51.788 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.789 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.789 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2020-12-05T17:36:51.789 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:36:51.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.790 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.790 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2020-12-05T17:36:51.790 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:36:51.790 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.791 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.791 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:36:51.791 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.792 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.792 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2020-12-05T17:36:51.792 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:36:51.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.793 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.793 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2020-12-05T17:36:51.793 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:36:51.793 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.794 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.794 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:36:51.794 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.795 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.795 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2020-12-05T17:36:51.795 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:36:51.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.796 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.796 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2020-12-05T17:36:51.796 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:36:51.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.797 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.797 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6803/11526", 2020-12-05T17:36:51.797 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6805/11526", 2020-12-05T17:36:51.798 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6809/11526", 2020-12-05T17:36:51.798 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6807/11526", 2020-12-05T17:36:51.798 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:36:51.798 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:36:51.799 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:36:51.799 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.800 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:36:51.800 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "01bd7ff5-a86b-4e11-ba1e-484c309c0fba", 2020-12-05T17:36:51.800 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:36:51.800 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:36:51.800 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:36:51.801 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:36:51.801 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:36:51.801 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:36:51.801 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:36:51.802 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 51, 2020-12-05T17:36:51.802 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:36:51.802 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:36:51.802 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:36:51.802 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.803 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.803 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2020-12-05T17:36:51.803 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:36:51.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.804 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.804 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2020-12-05T17:36:51.804 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:36:51.804 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.805 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.805 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:36:51.805 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.806 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.806 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2020-12-05T17:36:51.806 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:36:51.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.807 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.807 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2020-12-05T17:36:51.807 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:36:51.807 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.807 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.808 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:36:51.808 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.808 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.809 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2020-12-05T17:36:51.809 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:36:51.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.809 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.810 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2020-12-05T17:36:51.810 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:36:51.810 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.810 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.811 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:36:51.811 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.811 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.812 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2020-12-05T17:36:51.812 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:36:51.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.812 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.813 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2020-12-05T17:36:51.813 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:36:51.813 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.813 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.814 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6819/11538", 2020-12-05T17:36:51.814 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6821/11538", 2020-12-05T17:36:51.814 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6825/11538", 2020-12-05T17:36:51.814 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6823/11538", 2020-12-05T17:36:51.815 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:36:51.815 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:36:51.815 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:36:51.815 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.816 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:36:51.816 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "94736ef5-64c2-4a97-b963-bd38cddd9c52", 2020-12-05T17:36:51.816 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:36:51.816 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:36:51.817 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:36:51.817 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:36:51.817 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:36:51.817 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:36:51.818 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:36:51.818 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 52, 2020-12-05T17:36:51.818 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:36:51.818 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:36:51.818 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:36:51.819 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.819 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.819 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2020-12-05T17:36:51.819 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:36:51.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.820 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.820 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2020-12-05T17:36:51.820 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:36:51.821 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.821 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.821 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:36:51.822 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.822 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.822 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2020-12-05T17:36:51.822 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:36:51.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.823 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.823 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2020-12-05T17:36:51.823 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:36:51.824 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.824 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.824 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:36:51.824 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.825 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.825 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2020-12-05T17:36:51.825 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:36:51.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.826 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.826 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2020-12-05T17:36:51.826 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:36:51.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.827 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.827 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:36:51.827 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.828 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.828 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2020-12-05T17:36:51.828 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:36:51.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.829 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.829 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2020-12-05T17:36:51.829 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:36:51.830 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.830 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.830 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6811/11521", 2020-12-05T17:36:51.830 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6813/11521", 2020-12-05T17:36:51.831 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6817/11521", 2020-12-05T17:36:51.831 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6815/11521", 2020-12-05T17:36:51.831 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:36:51.831 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:36:51.831 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:36:51.832 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.832 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:36:51.832 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "69a1f3ff-fd25-43df-b60c-e32a89c35234", 2020-12-05T17:36:51.833 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:36:51.833 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:36:51.833 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:36:51.833 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:36:51.834 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:36:51.834 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:36:51.834 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:36:51.834 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 50, 2020-12-05T17:36:51.834 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:36:51.835 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:36:51.835 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:36:51.835 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.835 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.836 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2020-12-05T17:36:51.836 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:36:51.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.837 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.837 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2020-12-05T17:36:51.837 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:36:51.837 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.837 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.838 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:36:51.838 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.838 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.839 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2020-12-05T17:36:51.839 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:36:51.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.840 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.840 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2020-12-05T17:36:51.840 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:36:51.840 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.840 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.841 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:36:51.841 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.841 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.842 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6832", 2020-12-05T17:36:51.842 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:36:51.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.842 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.843 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6833", 2020-12-05T17:36:51.843 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:36:51.843 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.843 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.844 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:36:51.844 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.844 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.845 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2020-12-05T17:36:51.845 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:36:51.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.845 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.846 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2020-12-05T17:36:51.846 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:36:51.846 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.846 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.847 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6827/11535", 2020-12-05T17:36:51.847 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6829/11535", 2020-12-05T17:36:51.847 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6833/11535", 2020-12-05T17:36:51.847 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6831/11535", 2020-12-05T17:36:51.847 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:36:51.848 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:36:51.848 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:36:51.848 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.849 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:36:51.849 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c301fe4e-5eab-402c-ae6f-db3e6a7181af", 2020-12-05T17:36:51.849 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:36:51.849 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:36:51.849 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:36:51.850 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:36:51.850 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:36:51.850 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:36:51.850 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:36:51.850 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 51, 2020-12-05T17:36:51.851 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:36:51.851 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:36:51.851 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:36:51.851 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.852 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.852 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6808", 2020-12-05T17:36:51.852 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:36:51.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.853 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.853 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6809", 2020-12-05T17:36:51.853 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:36:51.853 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.853 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.854 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:36:51.854 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.854 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.855 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6810", 2020-12-05T17:36:51.855 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:36:51.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.856 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.856 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6811", 2020-12-05T17:36:51.856 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:36:51.856 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.856 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.857 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:36:51.857 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.857 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.858 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6817", 2020-12-05T17:36:51.858 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:36:51.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.858 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.859 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6819", 2020-12-05T17:36:51.859 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:36:51.859 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.859 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.860 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:36:51.860 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.860 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.861 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6812", 2020-12-05T17:36:51.861 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:36:51.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.861 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.862 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6814", 2020-12-05T17:36:51.862 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:36:51.862 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.862 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.863 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6809/11541", 2020-12-05T17:36:51.863 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6811/11541", 2020-12-05T17:36:51.863 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6819/11541", 2020-12-05T17:36:51.863 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6814/11541", 2020-12-05T17:36:51.864 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:36:51.864 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:36:51.864 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:36:51.864 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.865 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:36:51.865 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "9476eff4-345b-46c5-a473-1d9a89a6d327", 2020-12-05T17:36:51.865 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:36:51.865 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:36:51.866 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:36:51.866 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:36:51.866 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:36:51.866 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:36:51.867 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:36:51.867 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 51, 2020-12-05T17:36:51.867 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:36:51.867 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:36:51.867 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:36:51.868 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.868 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.868 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6800", 2020-12-05T17:36:51.868 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:36:51.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.869 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.869 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6801", 2020-12-05T17:36:51.869 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:36:51.870 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.870 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.870 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:36:51.871 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.871 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.871 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6802", 2020-12-05T17:36:51.871 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:36:51.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.872 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.872 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6803", 2020-12-05T17:36:51.872 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:36:51.873 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.873 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.873 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:36:51.873 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.874 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.874 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6806", 2020-12-05T17:36:51.874 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:36:51.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.875 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.875 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6807", 2020-12-05T17:36:51.875 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:36:51.876 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.876 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.876 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:36:51.876 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.877 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.877 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6804", 2020-12-05T17:36:51.877 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:36:51.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.878 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.878 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6805", 2020-12-05T17:36:51.878 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:36:51.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.879 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.879 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6801/11539", 2020-12-05T17:36:51.879 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6803/11539", 2020-12-05T17:36:51.880 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6807/11539", 2020-12-05T17:36:51.880 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6805/11539", 2020-12-05T17:36:51.880 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:36:51.880 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:36:51.880 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:36:51.881 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.881 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:36:51.881 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8dfd8567-9b1c-4d13-9c96-071183e469c2", 2020-12-05T17:36:51.882 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:36:51.882 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:36:51.882 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:36:51.882 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:36:51.882 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:36:51.883 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:36:51.883 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:36:51.883 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 52, 2020-12-05T17:36:51.883 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:36:51.884 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:36:51.884 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:36:51.884 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.884 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.885 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6820", 2020-12-05T17:36:51.885 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:36:51.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.885 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.886 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6822", 2020-12-05T17:36:51.886 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:36:51.886 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.886 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.887 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:36:51.887 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.887 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.888 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6823", 2020-12-05T17:36:51.888 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:36:51.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.888 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.889 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6825", 2020-12-05T17:36:51.889 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:36:51.889 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.889 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.890 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:36:51.890 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.890 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.891 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6829", 2020-12-05T17:36:51.891 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:36:51.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.891 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.892 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6830", 2020-12-05T17:36:51.892 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:36:51.892 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.892 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.893 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:36:51.893 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.893 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.894 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6826", 2020-12-05T17:36:51.894 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:36:51.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.894 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.895 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6827", 2020-12-05T17:36:51.895 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:36:51.895 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.895 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.896 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6822/11542", 2020-12-05T17:36:51.896 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6825/11542", 2020-12-05T17:36:51.896 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6830/11542", 2020-12-05T17:36:51.896 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6827/11542", 2020-12-05T17:36:51.897 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:36:51.897 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:36:51.897 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:36:51.897 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.898 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:36:51.898 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "24d466b3-01dc-4ba2-8b81-85250d163bd2", 2020-12-05T17:36:51.898 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:36:51.898 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:36:51.899 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:36:51.899 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:36:51.899 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:36:51.899 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:36:51.900 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:36:51.900 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 50, 2020-12-05T17:36:51.900 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:36:51.900 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:36:51.900 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:36:51.901 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.901 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.901 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6813", 2020-12-05T17:36:51.901 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:36:51.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.902 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.902 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6815", 2020-12-05T17:36:51.903 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:36:51.903 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.903 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.903 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:36:51.904 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.904 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.904 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6816", 2020-12-05T17:36:51.904 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:36:51.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.905 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.905 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6818", 2020-12-05T17:36:51.905 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:36:51.906 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.906 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.906 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:36:51.907 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.907 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.907 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6828", 2020-12-05T17:36:51.907 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:36:51.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.908 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.908 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6831", 2020-12-05T17:36:51.908 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:36:51.909 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.909 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.909 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:36:51.909 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:36:51.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.910 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:36:51.910 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6821", 2020-12-05T17:36:51.910 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:36:51.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.911 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:36:51.911 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6824", 2020-12-05T17:36:51.911 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:36:51.912 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.912 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.912 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6815/11540", 2020-12-05T17:36:51.912 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6818/11540", 2020-12-05T17:36:51.913 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6831/11540", 2020-12-05T17:36:51.913 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6824/11540", 2020-12-05T17:36:51.913 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:36:51.913 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:36:51.913 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:36:51.914 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:51.914 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.914 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:36:51.914 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2020-12-05T17:36:51.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.915 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:36:51.915 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:36:51.915 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:36:51.915 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:36:51.915 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:36:51.916 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:36:51.916 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.398868+0000", 2020-12-05T17:36:51.916 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:36:51.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.917 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:36:51.917 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:36:51.917 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:36:51.917 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:36:51.917 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:36:51.918 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:36:51.918 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.675748+0000", 2020-12-05T17:36:51.918 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:36:51.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.919 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:36:51.919 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:36:51.919 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:36:51.919 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:36:51.920 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:36:51.920 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:36:51.920 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.620888+0000", 2020-12-05T17:36:51.920 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:36:51.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.921 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:36:51.921 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:36:51.921 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:36:51.921 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:36:51.922 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:36:51.922 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:36:51.922 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.681600+0000", 2020-12-05T17:36:51.922 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:36:51.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.923 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:36:51.923 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:36:51.923 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:36:51.923 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:36:51.924 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:36:51.924 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:36:51.924 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.658769+0000", 2020-12-05T17:36:51.924 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:36:51.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.925 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:36:51.925 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:36:51.925 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:36:51.925 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:36:51.926 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:36:51.926 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:36:51.926 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.630708+0000", 2020-12-05T17:36:51.926 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:36:51.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.927 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:36:51.927 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:36:51.927 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:36:51.927 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:36:51.928 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:36:51.928 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:36:51.928 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.648352+0000", 2020-12-05T17:36:51.928 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:36:51.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:51.929 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:36:51.929 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:36:51.929 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:36:51.930 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:36:51.930 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:36:51.930 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:36:51.930 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.629399+0000", 2020-12-05T17:36:51.930 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:36:51.931 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.931 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:36:51.931 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2020-12-05T17:36:51.931 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2020-12-05T17:36:51.931 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2020-12-05T17:36:51.932 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2020-12-05T17:36:51.932 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": {}, 2020-12-05T17:36:51.932 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2020-12-05T17:36:51.932 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2020-12-05T17:36:51.932 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2020-12-05T17:36:51.933 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:36:51.933 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2020-12-05T17:36:51.933 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2020-12-05T17:36:51.933 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:36:51.933 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2020-12-05T17:36:51.934 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:36:51.934 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:36:51.934 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2020-12-05T17:36:51.934 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2020-12-05T17:36:51.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.935 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2020-12-05T17:36:51.935 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:36:51.935 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:36:51.935 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2020-12-05T17:36:51.935 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:36:51.936 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:36:51.936 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2020-12-05T17:36:51.936 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:51.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:51.937 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2020-12-05T17:36:51.937 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2020-12-05T17:36:51.937 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2020-12-05T17:36:51.937 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2020-12-05T17:36:51.937 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2020-12-05T17:36:51.938 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:36:51.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.716+0000 7f9ec0e6c700 1 -- 172.21.15.32:0/1127753266 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9e980554e0 msgr2=0x7f9e980579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:51.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.716+0000 7f9ec0e6c700 1 --2- 172.21.15.32:0/1127753266 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9e980554e0 0x7f9e980579a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f9eac00b000 tx=0x7f9eac0091a0).stop 2020-12-05T17:36:51.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.716+0000 7f9ec0e6c700 1 -- 172.21.15.32:0/1127753266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ebc0b1880 msgr2=0x7f9ebc151b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:51.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.716+0000 7f9ec0e6c700 1 --2- 172.21.15.32:0/1127753266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ebc0b1880 0x7f9ebc151b50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f9eb0002f30 tx=0x7f9eb0015000).stop 2020-12-05T17:36:51.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.716+0000 7f9ec0e6c700 1 -- 172.21.15.32:0/1127753266 shutdown_connections 2020-12-05T17:36:51.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.716+0000 7f9ec0e6c700 1 --2- 172.21.15.32:0/1127753266 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ebc0b3aa0 0x7f9ebc152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:51.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.716+0000 7f9ec0e6c700 1 --2- 172.21.15.32:0/1127753266 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9e980554e0 0x7f9e980579a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:51.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.716+0000 7f9ec0e6c700 1 --2- 172.21.15.32:0/1127753266 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ebc0afe50 0x7f9ebc141b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:51.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.716+0000 7f9ec0e6c700 1 --2- 172.21.15.32:0/1127753266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ebc0b1880 0x7f9ebc151b50 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:51.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.716+0000 7f9ec0e6c700 1 -- 172.21.15.32:0/1127753266 >> 172.21.15.32:0/1127753266 conn(0x7f9ebc0b9100 msgr2=0x7f9ebc13f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:51.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.716+0000 7f9ec0e6c700 1 -- 172.21.15.32:0/1127753266 shutdown_connections 2020-12-05T17:36:51.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:51.716+0000 7f9ec0e6c700 1 -- 172.21.15.32:0/1127753266 wait complete. 2020-12-05T17:36:51.943 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:36:52.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.072+0000 7f9fb5ed4700 1 Processor -- start 2020-12-05T17:36:52.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.072+0000 7f9fb5ed4700 1 -- start start 2020-12-05T17:36:52.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9fb5ed4700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9fb005f3b0 0x7f9fb005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:52.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9fb5ed4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9fb005feb0 0x7f9fb00b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:52.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9fb5ed4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb0142450 con 0x7f9fb0062400 2020-12-05T17:36:52.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9fb5ed4700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb00b42f0 con 0x7f9fb005f3b0 2020-12-05T17:36:52.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9fb5ed4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb0061780 con 0x7f9fb005feb0 2020-12-05T17:36:52.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9faffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9fb0062400 0x7f9fb00b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38376/0 (socket says 172.21.15.32:38376) 2020-12-05T17:36:52.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9faffff700 1 -- 172.21.15.32:0/3226148079 learned_addr learned my addr 172.21.15.32:0/3226148079 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:36:52.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9fb56d3700 1 --2- 172.21.15.32:0/3226148079 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9fb005feb0 0x7f9fb00b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:52.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9fb4ed2700 1 --2- 172.21.15.32:0/3226148079 >> v2:172.21.15.61:3300/0 conn(0x7f9fb005f3b0 0x7f9fb005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:52.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9faf7fe700 1 -- 172.21.15.32:0/3226148079 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 287335946 0 0) 0x7f9fb0142450 con 0x7f9fb0062400 2020-12-05T17:36:52.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9faf7fe700 1 -- 172.21.15.32:0/3226148079 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f9c003230 con 0x7f9fb0062400 2020-12-05T17:36:52.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9fb56d3700 1 -- 172.21.15.32:0/3226148079 >> v2:172.21.15.61:3300/0 conn(0x7f9fb005f3b0 msgr2=0x7f9fb005f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:36:52.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9fb56d3700 1 --2- 172.21.15.32:0/3226148079 >> v2:172.21.15.61:3300/0 conn(0x7f9fb005f3b0 0x7f9fb005f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:52.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9fb56d3700 1 -- 172.21.15.32:0/3226148079 >> v1:172.21.15.32:6789/0 conn(0x7f9fb0062400 legacy=0x7f9fb00b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:52.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9fb56d3700 1 -- 172.21.15.32:0/3226148079 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb0061a20 con 0x7f9fb005feb0 2020-12-05T17:36:52.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9fb4ed2700 1 --2- 172.21.15.32:0/3226148079 >> v2:172.21.15.61:3300/0 conn(0x7f9fb005f3b0 0x7f9fb005f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:36:52.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9fb56d3700 1 --2- 172.21.15.32:0/3226148079 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9fb005feb0 0x7f9fb00b3e00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f9fa80032b0 tx=0x7f9fa80091a0).ready entity=mon.2 client_cookie=f4b9a60a3da52bb3 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:52.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9faf7fe700 1 -- 172.21.15.32:0/3226148079 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fa800b940 con 0x7f9fb005feb0 2020-12-05T17:36:52.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9faf7fe700 1 -- 172.21.15.32:0/3226148079 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa800baa0 con 0x7f9fb005feb0 2020-12-05T17:36:52.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9faf7fe700 1 -- 172.21.15.32:0/3226148079 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fa800bc10 con 0x7f9fb005feb0 2020-12-05T17:36:52.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9fb5ed4700 1 -- 172.21.15.32:0/3226148079 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9fb005feb0 msgr2=0x7f9fb00b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:52.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9fb5ed4700 1 --2- 172.21.15.32:0/3226148079 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9fb005feb0 0x7f9fb00b3e00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f9fa80032b0 tx=0x7f9fa80091a0).stop 2020-12-05T17:36:52.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9fb5ed4700 1 -- 172.21.15.32:0/3226148079 shutdown_connections 2020-12-05T17:36:52.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9fb5ed4700 1 --2- 172.21.15.32:0/3226148079 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9fb005feb0 0x7f9fb00b3e00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9fb5ed4700 1 --2- 172.21.15.32:0/3226148079 >> v2:172.21.15.61:3300/0 conn(0x7f9fb005f3b0 0x7f9fb005f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9fb5ed4700 1 -- 172.21.15.32:0/3226148079 >> 172.21.15.32:0/3226148079 conn(0x7f9fb00b8ee0 msgr2=0x7f9fb00b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9fb5ed4700 1 -- 172.21.15.32:0/3226148079 shutdown_connections 2020-12-05T17:36:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.076+0000 7f9fb5ed4700 1 -- 172.21.15.32:0/3226148079 wait complete. 2020-12-05T17:36:52.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.080+0000 7f9fb5ed4700 1 Processor -- start 2020-12-05T17:36:52.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.080+0000 7f9fb5ed4700 1 -- start start 2020-12-05T17:36:52.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.080+0000 7f9fb5ed4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9fb005f3b0 0x7f9fb0148280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:52.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.080+0000 7f9fb5ed4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fb005feb0 0x7f9fb0064ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:52.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.080+0000 7f9fb5ed4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9fb0062400 0x7f9fb0064fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:52.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.080+0000 7f9fb5ed4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb0148f60 con 0x7f9fb005feb0 2020-12-05T17:36:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.080+0000 7f9fb5ed4700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb0063380 con 0x7f9fb0062400 2020-12-05T17:36:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.080+0000 7f9fb5ed4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb0063620 con 0x7f9fb005f3b0 2020-12-05T17:36:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.080+0000 7f9faffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fb005feb0 0x7f9fb0064ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.080+0000 7f9faffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fb005feb0 0x7f9fb0064ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35736/0 (socket says 172.21.15.32:35736) 2020-12-05T17:36:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.080+0000 7f9fb56d3700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9fb0062400 0x7f9fb0064fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.080+0000 7f9faffff700 1 -- 172.21.15.32:0/2513569192 learned_addr learned my addr 172.21.15.32:0/2513569192 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.080+0000 7f9fb4ed2700 1 --2- 172.21.15.32:0/2513569192 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9fb005f3b0 0x7f9fb0148280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.080+0000 7f9faffff700 1 -- 172.21.15.32:0/2513569192 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9fb005f3b0 msgr2=0x7f9fb0148280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.080+0000 7f9faffff700 1 --2- 172.21.15.32:0/2513569192 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9fb005f3b0 0x7f9fb0148280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.080+0000 7f9faffff700 1 -- 172.21.15.32:0/2513569192 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9fb0062400 msgr2=0x7f9fb0064fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:52.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.080+0000 7f9faffff700 1 --2- 172.21.15.32:0/2513569192 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9fb0062400 0x7f9fb0064fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:52.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.080+0000 7f9faffff700 1 -- 172.21.15.32:0/2513569192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb0065490 con 0x7f9fb005feb0 2020-12-05T17:36:52.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.080+0000 7f9faffff700 1 --2- 172.21.15.32:0/2513569192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fb005feb0 0x7f9fb0064ab0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f9fa40010d0 tx=0x7f9fa4007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:52.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.080+0000 7f9fadffb700 1 -- 172.21.15.32:0/2513569192 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fa40073b0 con 0x7f9fb005feb0 2020-12-05T17:36:52.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.084+0000 7f9fadffb700 1 -- 172.21.15.32:0/2513569192 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa40055e0 con 0x7f9fb005feb0 2020-12-05T17:36:52.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.084+0000 7f9fb5ed4700 1 -- 172.21.15.32:0/2513569192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb00af930 con 0x7f9fb005feb0 2020-12-05T17:36:52.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.084+0000 7f9fadffb700 1 -- 172.21.15.32:0/2513569192 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fa4005940 con 0x7f9fb005feb0 2020-12-05T17:36:52.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.084+0000 7f9fb5ed4700 1 -- 172.21.15.32:0/2513569192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb0145f70 con 0x7f9fb005feb0 2020-12-05T17:36:52.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.084+0000 7f9fb5ed4700 1 -- 172.21.15.32:0/2513569192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb00631a0 con 0x7f9fb005feb0 2020-12-05T17:36:52.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.084+0000 7f9fadffb700 1 -- 172.21.15.32:0/2513569192 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9fa4010070 con 0x7f9fb005feb0 2020-12-05T17:36:52.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.084+0000 7f9fadffb700 1 --2- 172.21.15.32:0/2513569192 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9f90055440 0x7f9f90057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:52.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.084+0000 7f9fadffb700 1 -- 172.21.15.32:0/2513569192 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9fa406d980 con 0x7f9fb005feb0 2020-12-05T17:36:52.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.084+0000 7f9fb4ed2700 1 --2- 172.21.15.32:0/2513569192 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9f90055440 0x7f9f90057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:52.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.088+0000 7f9fb4ed2700 1 --2- 172.21.15.32:0/2513569192 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9f90055440 0x7f9f90057900 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f9fa000b000 tx=0x7f9fa0006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:52.094 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.096+0000 7f9fadffb700 1 -- 172.21.15.32:0/2513569192 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9fa4043990 con 0x7f9fb005feb0 2020-12-05T17:36:52.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.284+0000 7f9fb5ed4700 1 -- 172.21.15.32:0/2513569192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fb00637e0 con 0x7f9fb005feb0 2020-12-05T17:36:52.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.284+0000 7f9fadffb700 1 -- 172.21.15.32:0/2513569192 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2136 (secure 0 0 0) 0x7f9fa4046fb0 con 0x7f9fb005feb0 2020-12-05T17:36:52.283 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:36:52.284 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5159,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/2123765277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2123765277},{"type":"v1","addr":"172.21.15.61:6833","nonce":2123765277}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:35:23.386407+0000","modified":"2020-12-05T17:35:24.389244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5640},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5640":{"gid":5640,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/681412338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":681412338},{"type":"v1","addr":"172.21.15.32:6835","nonce":681412338}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-12-05T17:36:52.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.284+0000 7f9fb5ed4700 1 -- 172.21.15.32:0/2513569192 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9f90055440 msgr2=0x7f9f90057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:52.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.284+0000 7f9fb5ed4700 1 --2- 172.21.15.32:0/2513569192 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9f90055440 0x7f9f90057900 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f9fa000b000 tx=0x7f9fa0006dc0).stop 2020-12-05T17:36:52.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.284+0000 7f9fb5ed4700 1 -- 172.21.15.32:0/2513569192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fb005feb0 msgr2=0x7f9fb0064ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:52.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.284+0000 7f9fb5ed4700 1 --2- 172.21.15.32:0/2513569192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fb005feb0 0x7f9fb0064ab0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f9fa40010d0 tx=0x7f9fa4007b50).stop 2020-12-05T17:36:52.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.284+0000 7f9fb5ed4700 1 -- 172.21.15.32:0/2513569192 shutdown_connections 2020-12-05T17:36:52.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.284+0000 7f9fb5ed4700 1 --2- 172.21.15.32:0/2513569192 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9fb0062400 0x7f9fb0064fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:52.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.284+0000 7f9fb5ed4700 1 --2- 172.21.15.32:0/2513569192 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9f90055440 0x7f9f90057900 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:52.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.284+0000 7f9fb5ed4700 1 --2- 172.21.15.32:0/2513569192 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9fb005f3b0 0x7f9fb0148280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:52.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.284+0000 7f9fb5ed4700 1 --2- 172.21.15.32:0/2513569192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fb005feb0 0x7f9fb0064ab0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:52.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.284+0000 7f9fb5ed4700 1 -- 172.21.15.32:0/2513569192 >> 172.21.15.32:0/2513569192 conn(0x7f9fb00b8ee0 msgr2=0x7f9fb0142620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:52.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.284+0000 7f9fb5ed4700 1 -- 172.21.15.32:0/2513569192 shutdown_connections 2020-12-05T17:36:52.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.284+0000 7f9fb5ed4700 1 -- 172.21.15.32:0/2513569192 wait complete. 2020-12-05T17:36:52.288 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 23 2020-12-05T17:36:52.297 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-12-05T17:36:52.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.412+0000 7f1029033700 1 Processor -- start 2020-12-05T17:36:52.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.412+0000 7f1029033700 1 -- start start 2020-12-05T17:36:52.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.412+0000 7f1029033700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f10240b1fc0 0x7f10240af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:52.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.412+0000 7f1029033700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1024147400 0x7f1024145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:52.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.412+0000 7f1029033700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1024146a90 con 0x7f10240ab710 2020-12-05T17:36:52.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.412+0000 7f1029033700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1024061e90 con 0x7f1024147400 2020-12-05T17:36:52.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.412+0000 7f1029033700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10240b0110 con 0x7f10240b1fc0 2020-12-05T17:36:52.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.412+0000 7f10237fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f10240ab710 0x7f1024061780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38392/0 (socket says 172.21.15.32:38392) 2020-12-05T17:36:52.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.412+0000 7f10237fe700 1 -- 172.21.15.32:0/4270195560 learned_addr learned my addr 172.21.15.32:0/4270195560 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:36:52.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.412+0000 7f1023fff700 1 --2- 172.21.15.32:0/4270195560 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f10240b1fc0 0x7f10240af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:52.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.412+0000 7f1028832700 1 --2- 172.21.15.32:0/4270195560 >> v2:172.21.15.61:3300/0 conn(0x7f1024147400 0x7f1024145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:52.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1022ffd700 1 -- 172.21.15.32:0/4270195560 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1571265175 0 0) 0x7f1024146a90 con 0x7f10240ab710 2020-12-05T17:36:52.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1022ffd700 1 -- 172.21.15.32:0/4270195560 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1004003230 con 0x7f10240ab710 2020-12-05T17:36:52.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1023fff700 1 -- 172.21.15.32:0/4270195560 >> v2:172.21.15.61:3300/0 conn(0x7f1024147400 msgr2=0x7f1024145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:36:52.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1023fff700 1 --2- 172.21.15.32:0/4270195560 >> v2:172.21.15.61:3300/0 conn(0x7f1024147400 0x7f1024145d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:52.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1023fff700 1 -- 172.21.15.32:0/4270195560 >> v1:172.21.15.32:6789/0 conn(0x7f10240ab710 legacy=0x7f1024061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:52.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1023fff700 1 -- 172.21.15.32:0/4270195560 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10240b23a0 con 0x7f10240b1fc0 2020-12-05T17:36:52.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1023fff700 1 --2- 172.21.15.32:0/4270195560 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f10240b1fc0 0x7f10240af230 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f101c0032b0 tx=0x7f101c0091a0).ready entity=mon.2 client_cookie=eb0798cad3658d40 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:52.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1022ffd700 1 -- 172.21.15.32:0/4270195560 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f101c00b940 con 0x7f10240b1fc0 2020-12-05T17:36:52.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1022ffd700 1 -- 172.21.15.32:0/4270195560 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101c00baa0 con 0x7f10240b1fc0 2020-12-05T17:36:52.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1022ffd700 1 -- 172.21.15.32:0/4270195560 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f101c00bc10 con 0x7f10240b1fc0 2020-12-05T17:36:52.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1029033700 1 -- 172.21.15.32:0/4270195560 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f10240b1fc0 msgr2=0x7f10240af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:52.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1029033700 1 --2- 172.21.15.32:0/4270195560 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f10240b1fc0 0x7f10240af230 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f101c0032b0 tx=0x7f101c0091a0).stop 2020-12-05T17:36:52.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1029033700 1 -- 172.21.15.32:0/4270195560 shutdown_connections 2020-12-05T17:36:52.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1029033700 1 --2- 172.21.15.32:0/4270195560 >> v2:172.21.15.61:3300/0 conn(0x7f1024147400 0x7f1024145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:52.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1029033700 1 --2- 172.21.15.32:0/4270195560 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f10240b1fc0 0x7f10240af230 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:52.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1029033700 1 -- 172.21.15.32:0/4270195560 >> 172.21.15.32:0/4270195560 conn(0x7f10240b5530 msgr2=0x7f10240b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:52.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1029033700 1 -- 172.21.15.32:0/4270195560 shutdown_connections 2020-12-05T17:36:52.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1029033700 1 -- 172.21.15.32:0/4270195560 wait complete. 2020-12-05T17:36:52.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1029033700 1 Processor -- start 2020-12-05T17:36:52.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1029033700 1 -- start start 2020-12-05T17:36:52.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1029033700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f10240ab710 0x7f10240abcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:52.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1029033700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f10240b1fc0 0x7f1024066470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:52.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1029033700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1024147400 0x7f1024066960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:52.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1029033700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1024065750 con 0x7f1024147400 2020-12-05T17:36:52.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1029033700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102409e120 con 0x7f10240b1fc0 2020-12-05T17:36:52.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1029033700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1024066e50 con 0x7f10240ab710 2020-12-05T17:36:52.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.416+0000 7f1028832700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1024147400 0x7f1024066960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.420+0000 7f1028832700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1024147400 0x7f1024066960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35750/0 (socket says 172.21.15.32:35750) 2020-12-05T17:36:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.420+0000 7f1028832700 1 -- 172.21.15.32:0/4138585507 learned_addr learned my addr 172.21.15.32:0/4138585507 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.420+0000 7f1023fff700 1 --2- 172.21.15.32:0/4138585507 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f10240ab710 0x7f10240abcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.420+0000 7f10237fe700 1 --2- 172.21.15.32:0/4138585507 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f10240b1fc0 0x7f1024066470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.420+0000 7f1028832700 1 -- 172.21.15.32:0/4138585507 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f10240ab710 msgr2=0x7f10240abcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:52.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.420+0000 7f1028832700 1 --2- 172.21.15.32:0/4138585507 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f10240ab710 0x7f10240abcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:52.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.420+0000 7f1028832700 1 -- 172.21.15.32:0/4138585507 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f10240b1fc0 msgr2=0x7f1024066470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:52.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.420+0000 7f1028832700 1 --2- 172.21.15.32:0/4138585507 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f10240b1fc0 0x7f1024066470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:52.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.420+0000 7f1028832700 1 -- 172.21.15.32:0/4138585507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102409e350 con 0x7f1024147400 2020-12-05T17:36:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.420+0000 7f10237fe700 1 --2- 172.21.15.32:0/4138585507 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f10240b1fc0 0x7f1024066470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:36:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.420+0000 7f1028832700 1 --2- 172.21.15.32:0/4138585507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1024147400 0x7f1024066960 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f101400c000 tx=0x7f1014006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.420+0000 7f10217fa700 1 -- 172.21.15.32:0/4138585507 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f101400ca10 con 0x7f1024147400 2020-12-05T17:36:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.420+0000 7f1029033700 1 -- 172.21.15.32:0/4138585507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1024063150 con 0x7f1024147400 2020-12-05T17:36:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.420+0000 7f10217fa700 1 -- 172.21.15.32:0/4138585507 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101400cb70 con 0x7f1024147400 2020-12-05T17:36:52.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.420+0000 7f10217fa700 1 -- 172.21.15.32:0/4138585507 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f101400ee80 con 0x7f1024147400 2020-12-05T17:36:52.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.420+0000 7f1029033700 1 -- 172.21.15.32:0/4138585507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1024159940 con 0x7f1024147400 2020-12-05T17:36:52.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.420+0000 7f1029033700 1 -- 172.21.15.32:0/4138585507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10240ae890 con 0x7f1024147400 2020-12-05T17:36:52.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.420+0000 7f10217fa700 1 -- 172.21.15.32:0/4138585507 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f101401c460 con 0x7f1024147400 2020-12-05T17:36:52.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.424+0000 7f10217fa700 1 --2- 172.21.15.32:0/4138585507 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f10080553b0 0x7f1008057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:52.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.424+0000 7f10217fa700 1 -- 172.21.15.32:0/4138585507 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f101406e580 con 0x7f1024147400 2020-12-05T17:36:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.424+0000 7f1023fff700 1 --2- 172.21.15.32:0/4138585507 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f10080553b0 0x7f1008057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.424+0000 7f1023fff700 1 --2- 172.21.15.32:0/4138585507 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f10080553b0 0x7f1008057870 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f101c01b000 tx=0x7f101c016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.428+0000 7f10217fa700 1 -- 172.21.15.32:0/4138585507 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1014044620 con 0x7f1024147400 2020-12-05T17:36:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.616+0000 7f1029033700 1 -- 172.21.15.32:0/4138585507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1024057050 con 0x7f1024147400 2020-12-05T17:36:52.972 INFO:tasks.ceph.mon.a.smithi032.stderr:2020-12-05T17:36:52.971+0000 7fdef8136700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-12-05T17:36:52.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.979+0000 7f10217fa700 1 -- 172.21.15.32:0/4138585507 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1014047c40 con 0x7f1024147400 2020-12-05T17:36:52.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.983+0000 7f1029033700 1 -- 172.21.15.32:0/4138585507 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f10080553b0 msgr2=0x7f1008057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:52.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.983+0000 7f1029033700 1 --2- 172.21.15.32:0/4138585507 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f10080553b0 0x7f1008057870 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f101c01b000 tx=0x7f101c016000).stop 2020-12-05T17:36:52.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.983+0000 7f1029033700 1 -- 172.21.15.32:0/4138585507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1024147400 msgr2=0x7f1024066960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:52.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.983+0000 7f1029033700 1 --2- 172.21.15.32:0/4138585507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1024147400 0x7f1024066960 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f101400c000 tx=0x7f1014006dc0).stop 2020-12-05T17:36:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.983+0000 7f1029033700 1 -- 172.21.15.32:0/4138585507 shutdown_connections 2020-12-05T17:36:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.983+0000 7f1029033700 1 --2- 172.21.15.32:0/4138585507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1024147400 0x7f1024066960 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.983+0000 7f1029033700 1 --2- 172.21.15.32:0/4138585507 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f10080553b0 0x7f1008057870 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.983+0000 7f1029033700 1 --2- 172.21.15.32:0/4138585507 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f10240ab710 0x7f10240abcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.983+0000 7f1029033700 1 --2- 172.21.15.32:0/4138585507 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f10240b1fc0 0x7f1024066470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.983+0000 7f1029033700 1 -- 172.21.15.32:0/4138585507 >> 172.21.15.32:0/4138585507 conn(0x7f10240b5530 msgr2=0x7f10240af740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.983+0000 7f1029033700 1 -- 172.21.15.32:0/4138585507 shutdown_connections 2020-12-05T17:36:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:52.983+0000 7f1029033700 1 -- 172.21.15.32:0/4138585507 wait complete. 2020-12-05T17:36:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-12-05T17:36:53.000 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:36:53.017 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:36:53.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.111+0000 7f6b58312700 1 Processor -- start 2020-12-05T17:36:53.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.111+0000 7f6b58312700 1 -- start start 2020-12-05T17:36:53.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.111+0000 7f6b58312700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6b50065d90 0x7f6b5014c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:53.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.111+0000 7f6b58312700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b500acb80 con 0x7f6b5005f6f0 2020-12-05T17:36:53.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.111+0000 7f6b58312700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b50065250 con 0x7f6b50143d40 2020-12-05T17:36:53.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.111+0000 7f6b58312700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b500b2860 con 0x7f6b50065d90 2020-12-05T17:36:53.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.111+0000 7f6b560ae700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6b5005f6f0 0x7f6b5013a380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38424/0 (socket says 172.21.15.32:38424) 2020-12-05T17:36:53.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.111+0000 7f6b560ae700 1 -- 172.21.15.32:0/1971851995 learned_addr learned my addr 172.21.15.32:0/1971851995 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:36:53.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.111+0000 7f6b568af700 1 --2- 172.21.15.32:0/1971851995 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6b50065d90 0x7f6b5014c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:53.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.111+0000 7f6b550ac700 1 -- 172.21.15.32:0/1971851995 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2883278349 0 0) 0x7f6b500acb80 con 0x7f6b5005f6f0 2020-12-05T17:36:53.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.111+0000 7f6b550ac700 1 -- 172.21.15.32:0/1971851995 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b38003230 con 0x7f6b5005f6f0 2020-12-05T17:36:53.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.111+0000 7f6b550ac700 1 -- 172.21.15.32:0/1971851995 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3096695764 0 0) 0x7f6b50065250 con 0x7f6b50143d40 2020-12-05T17:36:53.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.111+0000 7f6b550ac700 1 -- 172.21.15.32:0/1971851995 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b500acb80 con 0x7f6b50143d40 2020-12-05T17:36:53.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.111+0000 7f6b550ac700 1 -- 172.21.15.32:0/1971851995 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4046631160 0 0) 0x7f6b38003230 con 0x7f6b5005f6f0 2020-12-05T17:36:53.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.111+0000 7f6b568af700 1 -- 172.21.15.32:0/1971851995 >> v1:172.21.15.61:6789/0 conn(0x7f6b50143d40 legacy=0x7f6b50065680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:53.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.111+0000 7f6b568af700 1 -- 172.21.15.32:0/1971851995 >> v1:172.21.15.32:6789/0 conn(0x7f6b5005f6f0 legacy=0x7f6b5013a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:53.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.111+0000 7f6b568af700 1 -- 172.21.15.32:0/1971851995 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5005fad0 con 0x7f6b50065d90 2020-12-05T17:36:53.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.111+0000 7f6b568af700 1 --2- 172.21.15.32:0/1971851995 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6b50065d90 0x7f6b5014c0f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f6b4c0010d0 tx=0x7f6b480091a0).ready entity=mon.2 client_cookie=9dde9e8c8d7e693d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:53.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.111+0000 7f6b550ac700 1 -- 172.21.15.32:0/1971851995 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b4c0016d0 con 0x7f6b50065d90 2020-12-05T17:36:53.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.111+0000 7f6b550ac700 1 -- 172.21.15.32:0/1971851995 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4800b800 con 0x7f6b50065d90 2020-12-05T17:36:53.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.111+0000 7f6b58312700 1 -- 172.21.15.32:0/1971851995 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6b50065d90 msgr2=0x7f6b5014c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:53.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.111+0000 7f6b58312700 1 --2- 172.21.15.32:0/1971851995 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6b50065d90 0x7f6b5014c0f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f6b4c0010d0 tx=0x7f6b480091a0).stop 2020-12-05T17:36:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.115+0000 7f6b58312700 1 -- 172.21.15.32:0/1971851995 shutdown_connections 2020-12-05T17:36:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.115+0000 7f6b58312700 1 --2- 172.21.15.32:0/1971851995 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6b50065d90 0x7f6b5014c0f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.115+0000 7f6b58312700 1 -- 172.21.15.32:0/1971851995 >> 172.21.15.32:0/1971851995 conn(0x7f6b500b6ef0 msgr2=0x7f6b500b72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.115+0000 7f6b58312700 1 -- 172.21.15.32:0/1971851995 shutdown_connections 2020-12-05T17:36:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.115+0000 7f6b58312700 1 -- 172.21.15.32:0/1971851995 wait complete. 2020-12-05T17:36:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.115+0000 7f6b58312700 1 Processor -- start 2020-12-05T17:36:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.115+0000 7f6b58312700 1 -- start start 2020-12-05T17:36:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.115+0000 7f6b58312700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b5005f6f0 0x7f6b50153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:53.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.115+0000 7f6b58312700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6b50065d90 0x7f6b500ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:53.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.115+0000 7f6b58312700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6b50143d40 0x7f6b500aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:53.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.115+0000 7f6b58312700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5013ffa0 con 0x7f6b5005f6f0 2020-12-05T17:36:53.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.115+0000 7f6b58312700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b50152b70 con 0x7f6b50065d90 2020-12-05T17:36:53.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.115+0000 7f6b58312700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b50152e10 con 0x7f6b50143d40 2020-12-05T17:36:53.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.115+0000 7f6b560ae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b5005f6f0 0x7f6b50153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:53.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.115+0000 7f6b560ae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b5005f6f0 0x7f6b50153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35780/0 (socket says 172.21.15.32:35780) 2020-12-05T17:36:53.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.115+0000 7f6b560ae700 1 -- 172.21.15.32:0/3058917673 learned_addr learned my addr 172.21.15.32:0/3058917673 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:53.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.115+0000 7f6b568af700 1 --2- 172.21.15.32:0/3058917673 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6b50143d40 0x7f6b500aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:53.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.115+0000 7f6b558ad700 1 --2- 172.21.15.32:0/3058917673 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6b50065d90 0x7f6b500ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:53.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.119+0000 7f6b560ae700 1 -- 172.21.15.32:0/3058917673 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6b50143d40 msgr2=0x7f6b500aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:53.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.119+0000 7f6b560ae700 1 --2- 172.21.15.32:0/3058917673 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6b50143d40 0x7f6b500aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:53.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.119+0000 7f6b560ae700 1 -- 172.21.15.32:0/3058917673 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6b50065d90 msgr2=0x7f6b500ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:53.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.119+0000 7f6b560ae700 1 --2- 172.21.15.32:0/3058917673 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6b50065d90 0x7f6b500ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:53.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.119+0000 7f6b560ae700 1 -- 172.21.15.32:0/3058917673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b501530b0 con 0x7f6b5005f6f0 2020-12-05T17:36:53.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.119+0000 7f6b558ad700 1 --2- 172.21.15.32:0/3058917673 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6b50065d90 0x7f6b500ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:36:53.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.119+0000 7f6b560ae700 1 --2- 172.21.15.32:0/3058917673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b5005f6f0 0x7f6b50153550 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f6b4c000fa0 tx=0x7f6b4c001b20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:53.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.119+0000 7f6b3f7fe700 1 -- 172.21.15.32:0/3058917673 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b4c007170 con 0x7f6b5005f6f0 2020-12-05T17:36:53.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.119+0000 7f6b3f7fe700 1 -- 172.21.15.32:0/3058917673 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4c0072d0 con 0x7f6b5005f6f0 2020-12-05T17:36:53.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.119+0000 7f6b58312700 1 -- 172.21.15.32:0/3058917673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b5013b6d0 con 0x7f6b5005f6f0 2020-12-05T17:36:53.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.119+0000 7f6b3f7fe700 1 -- 172.21.15.32:0/3058917673 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b4c00a900 con 0x7f6b5005f6f0 2020-12-05T17:36:53.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.119+0000 7f6b58312700 1 -- 172.21.15.32:0/3058917673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b500af3e0 con 0x7f6b5005f6f0 2020-12-05T17:36:53.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.119+0000 7f6b58312700 1 -- 172.21.15.32:0/3058917673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b50057050 con 0x7f6b5005f6f0 2020-12-05T17:36:53.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.119+0000 7f6b3f7fe700 1 -- 172.21.15.32:0/3058917673 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6b4c010070 con 0x7f6b5005f6f0 2020-12-05T17:36:53.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.119+0000 7f6b3f7fe700 1 --2- 172.21.15.32:0/3058917673 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6b30055440 0x7f6b30057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:53.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.119+0000 7f6b3f7fe700 1 -- 172.21.15.32:0/3058917673 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f6b4c06de90 con 0x7f6b5005f6f0 2020-12-05T17:36:53.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.123+0000 7f6b558ad700 1 --2- 172.21.15.32:0/3058917673 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6b30055440 0x7f6b30057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:53.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.123+0000 7f6b558ad700 1 --2- 172.21.15.32:0/3058917673 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6b30055440 0x7f6b30057900 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f6b4000b000 tx=0x7f6b40009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:53.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.127+0000 7f6b3f7fe700 1 -- 172.21.15.32:0/3058917673 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6b4c043fe0 con 0x7f6b5005f6f0 2020-12-05T17:36:53.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.311+0000 7f6b58312700 1 -- 172.21.15.32:0/3058917673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b50069690 con 0x7f6b5005f6f0 2020-12-05T17:36:54.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.311+0000 7f6b3f7fe700 1 -- 172.21.15.32:0/3058917673 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+1778 (secure 0 0 0) 0x7f6b4c047600 con 0x7f6b5005f6f0 2020-12-05T17:36:54.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.311+0000 7f6b58312700 1 -- 172.21.15.32:0/3058917673 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6b30055440 msgr2=0x7f6b30057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:54.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.311+0000 7f6b58312700 1 --2- 172.21.15.32:0/3058917673 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6b30055440 0x7f6b30057900 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f6b4000b000 tx=0x7f6b40009040).stop 2020-12-05T17:36:54.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.311+0000 7f6b58312700 1 -- 172.21.15.32:0/3058917673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b5005f6f0 msgr2=0x7f6b50153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:54.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.311+0000 7f6b58312700 1 --2- 172.21.15.32:0/3058917673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b5005f6f0 0x7f6b50153550 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f6b4c000fa0 tx=0x7f6b4c001b20).stop 2020-12-05T17:36:54.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.311+0000 7f6b58312700 1 -- 172.21.15.32:0/3058917673 shutdown_connections 2020-12-05T17:36:54.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.311+0000 7f6b58312700 1 --2- 172.21.15.32:0/3058917673 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6b30055440 0x7f6b30057900 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:54.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.311+0000 7f6b58312700 1 --2- 172.21.15.32:0/3058917673 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6b50143d40 0x7f6b500aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:54.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.311+0000 7f6b58312700 1 --2- 172.21.15.32:0/3058917673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b5005f6f0 0x7f6b50153550 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:54.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.311+0000 7f6b58312700 1 --2- 172.21.15.32:0/3058917673 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6b50065d90 0x7f6b500ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:54.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.311+0000 7f6b58312700 1 -- 172.21.15.32:0/3058917673 >> 172.21.15.32:0/3058917673 conn(0x7f6b500b6ef0 msgr2=0x7f6b501407c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:54.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.311+0000 7f6b58312700 1 -- 172.21.15.32:0/3058917673 shutdown_connections 2020-12-05T17:36:54.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:53.311+0000 7f6b58312700 1 -- 172.21.15.32:0/3058917673 wait complete. 2020-12-05T17:36:54.090 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2020-12-05T17:36:54.090 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:36:54.091 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5159,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/2123765277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2123765277},{"type":"v1","addr":"172.21.15.61:6833","nonce":2123765277}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:35:23.386407+0000","modified":"2020-12-05T17:36:52.976686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-12-05T17:36:54.091 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-12-05T17:36:54.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.255+0000 7f79c3fff700 1 Processor -- start 2020-12-05T17:36:54.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.255+0000 7f79c3fff700 1 -- start start 2020-12-05T17:36:54.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f79c40b5f00 0x7f79c40b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:54.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c3fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f79c40ad510 0x7f79c40af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:54.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c3fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79c4142ab0 con 0x7f79c4066370 2020-12-05T17:36:54.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c3fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79c40b62e0 con 0x7f79c40b5f00 2020-12-05T17:36:54.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c3fff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79c40afbf0 con 0x7f79c40ad510 2020-12-05T17:36:54.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c37fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f79c4066370 0x7f79c4142ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38440/0 (socket says 172.21.15.32:38440) 2020-12-05T17:36:54.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f79c40b5f00 0x7f79c40b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:54.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c37fe700 1 -- 172.21.15.32:0/660649794 learned_addr learned my addr 172.21.15.32:0/660649794 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:36:54.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79ba7fc700 1 --2- 172.21.15.32:0/660649794 >> v2:172.21.15.61:3300/0 conn(0x7f79c40ad510 0x7f79c40af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:54.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c2ffd700 1 -- 172.21.15.32:0/660649794 >> v2:172.21.15.61:3300/0 conn(0x7f79c40ad510 msgr2=0x7f79c40af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:36:54.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c2ffd700 1 --2- 172.21.15.32:0/660649794 >> v2:172.21.15.61:3300/0 conn(0x7f79c40ad510 0x7f79c40af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:54.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c2ffd700 1 -- 172.21.15.32:0/660649794 >> v1:172.21.15.32:6789/0 conn(0x7f79c4066370 legacy=0x7f79c4142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:54.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c2ffd700 1 -- 172.21.15.32:0/660649794 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c40afe90 con 0x7f79c40b5f00 2020-12-05T17:36:54.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c2ffd700 1 --2- 172.21.15.32:0/660649794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c40b5f00 0x7f79c40b14e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f79bc001450 tx=0x7f79b4006dc0).ready entity=mon.0 client_cookie=57de056802f63ca server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:54.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c27fc700 1 -- 172.21.15.32:0/660649794 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79b40096b0 con 0x7f79c40b5f00 2020-12-05T17:36:54.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c27fc700 1 -- 172.21.15.32:0/660649794 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b4009810 con 0x7f79c40b5f00 2020-12-05T17:36:54.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c27fc700 1 -- 172.21.15.32:0/660649794 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79b4009b70 con 0x7f79c40b5f00 2020-12-05T17:36:54.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c3fff700 1 -- 172.21.15.32:0/660649794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c40b5f00 msgr2=0x7f79c40b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:54.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c3fff700 1 --2- 172.21.15.32:0/660649794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c40b5f00 0x7f79c40b14e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f79bc001450 tx=0x7f79b4006dc0).stop 2020-12-05T17:36:54.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c3fff700 1 -- 172.21.15.32:0/660649794 shutdown_connections 2020-12-05T17:36:54.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c3fff700 1 --2- 172.21.15.32:0/660649794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c40b5f00 0x7f79c40b14e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:54.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c3fff700 1 --2- 172.21.15.32:0/660649794 >> v2:172.21.15.61:3300/0 conn(0x7f79c40ad510 0x7f79c40af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:54.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c3fff700 1 -- 172.21.15.32:0/660649794 >> 172.21.15.32:0/660649794 conn(0x7f79c40604c0 msgr2=0x7f79c405c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:54.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c3fff700 1 -- 172.21.15.32:0/660649794 shutdown_connections 2020-12-05T17:36:54.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.259+0000 7f79c3fff700 1 -- 172.21.15.32:0/660649794 wait complete. 2020-12-05T17:36:54.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.263+0000 7f79c3fff700 1 Processor -- start 2020-12-05T17:36:54.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.263+0000 7f79c3fff700 1 -- start start 2020-12-05T17:36:54.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.267+0000 7f79c3fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f79c4066370 0x7f79c412e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.267+0000 7f79c3fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f79c40ad510 0x7f79c41359e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.267+0000 7f79c3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c40b5f00 0x7f79c4135ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.267+0000 7f79c3fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c40b4ed0 con 0x7f79c40b5f00 2020-12-05T17:36:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.267+0000 7f79c3fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c40b3d00 con 0x7f79c4066370 2020-12-05T17:36:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.267+0000 7f79c3fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c40b3a60 con 0x7f79c40ad510 2020-12-05T17:36:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.267+0000 7f79c37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c40b5f00 0x7f79c4135ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.267+0000 7f79c37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c40b5f00 0x7f79c4135ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35798/0 (socket says 172.21.15.32:35798) 2020-12-05T17:36:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.267+0000 7f79c37fe700 1 -- 172.21.15.32:0/1879000571 learned_addr learned my addr 172.21.15.32:0/1879000571 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.267+0000 7f79ba7fc700 1 --2- 172.21.15.32:0/1879000571 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f79c40ad510 0x7f79c41359e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:54.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.267+0000 7f79c2ffd700 1 --2- 172.21.15.32:0/1879000571 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f79c4066370 0x7f79c412e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:54.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.267+0000 7f79c37fe700 1 -- 172.21.15.32:0/1879000571 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f79c40ad510 msgr2=0x7f79c41359e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:54.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.267+0000 7f79c37fe700 1 --2- 172.21.15.32:0/1879000571 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f79c40ad510 0x7f79c41359e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:54.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.267+0000 7f79c37fe700 1 -- 172.21.15.32:0/1879000571 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f79c4066370 msgr2=0x7f79c412e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:54.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.267+0000 7f79c37fe700 1 --2- 172.21.15.32:0/1879000571 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f79c4066370 0x7f79c412e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:54.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.267+0000 7f79c37fe700 1 -- 172.21.15.32:0/1879000571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c40b5630 con 0x7f79c40b5f00 2020-12-05T17:36:54.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.267+0000 7f79c37fe700 1 --2- 172.21.15.32:0/1879000571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c40b5f00 0x7f79c4135ed0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f79bc001510 tx=0x7f79bc007ed0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:54.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.267+0000 7f79c0ff9700 1 -- 172.21.15.32:0/1879000571 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79bc005b70 con 0x7f79c40b5f00 2020-12-05T17:36:54.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.267+0000 7f79c0ff9700 1 -- 172.21.15.32:0/1879000571 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79bc005cd0 con 0x7f79c40b5f00 2020-12-05T17:36:54.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.267+0000 7f79c0ff9700 1 -- 172.21.15.32:0/1879000571 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79bc007300 con 0x7f79c40b5f00 2020-12-05T17:36:54.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.267+0000 7f79c3fff700 1 -- 172.21.15.32:0/1879000571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c40b3f70 con 0x7f79c40b5f00 2020-12-05T17:36:54.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.267+0000 7f79c3fff700 1 -- 172.21.15.32:0/1879000571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c4137770 con 0x7f79c40b5f00 2020-12-05T17:36:54.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.271+0000 7f79c3fff700 1 -- 172.21.15.32:0/1879000571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c4057050 con 0x7f79c40b5f00 2020-12-05T17:36:54.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.271+0000 7f79c0ff9700 1 -- 172.21.15.32:0/1879000571 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f79bc010030 con 0x7f79c40b5f00 2020-12-05T17:36:54.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.271+0000 7f79c0ff9700 1 --2- 172.21.15.32:0/1879000571 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79a80554e0 0x7f79a80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:54.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.271+0000 7f79c0ff9700 1 -- 172.21.15.32:0/1879000571 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f79bc06e230 con 0x7f79c40b5f00 2020-12-05T17:36:54.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.271+0000 7f79c2ffd700 1 --2- 172.21.15.32:0/1879000571 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79a80554e0 0x7f79a80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:54.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.271+0000 7f79c2ffd700 1 --2- 172.21.15.32:0/1879000571 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79a80554e0 0x7f79a80579a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f79b401b000 tx=0x7f79b4015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:54.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.275+0000 7f79c0ff9700 1 -- 172.21.15.32:0/1879000571 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f79bc044380 con 0x7f79c40b5f00 2020-12-05T17:36:54.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.459+0000 7f79c3fff700 1 -- 172.21.15.32:0/1879000571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f79c40660f0 con 0x7f79c40b5f00 2020-12-05T17:36:55.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:54.999+0000 7f79c0ff9700 1 -- 172.21.15.32:0/1879000571 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f79bc0479a0 con 0x7f79c40b5f00 2020-12-05T17:36:55.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.003+0000 7f79c3fff700 1 -- 172.21.15.32:0/1879000571 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79a80554e0 msgr2=0x7f79a80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:55.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.003+0000 7f79c3fff700 1 --2- 172.21.15.32:0/1879000571 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79a80554e0 0x7f79a80579a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f79b401b000 tx=0x7f79b4015000).stop 2020-12-05T17:36:55.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.003+0000 7f79c3fff700 1 -- 172.21.15.32:0/1879000571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c40b5f00 msgr2=0x7f79c4135ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:55.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.003+0000 7f79c3fff700 1 --2- 172.21.15.32:0/1879000571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c40b5f00 0x7f79c4135ed0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f79bc001510 tx=0x7f79bc007ed0).stop 2020-12-05T17:36:55.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.003+0000 7f79c3fff700 1 -- 172.21.15.32:0/1879000571 shutdown_connections 2020-12-05T17:36:55.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.003+0000 7f79c3fff700 1 --2- 172.21.15.32:0/1879000571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c40b5f00 0x7f79c4135ed0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:55.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.003+0000 7f79c3fff700 1 --2- 172.21.15.32:0/1879000571 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f79c4066370 0x7f79c412e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:55.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.003+0000 7f79c3fff700 1 --2- 172.21.15.32:0/1879000571 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79a80554e0 0x7f79a80579a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:55.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.003+0000 7f79c3fff700 1 --2- 172.21.15.32:0/1879000571 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f79c40ad510 0x7f79c41359e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:55.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.003+0000 7f79c3fff700 1 -- 172.21.15.32:0/1879000571 >> 172.21.15.32:0/1879000571 conn(0x7f79c40604c0 msgr2=0x7f79c4137b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:55.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.003+0000 7f79c3fff700 1 -- 172.21.15.32:0/1879000571 shutdown_connections 2020-12-05T17:36:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.003+0000 7f79c3fff700 1 -- 172.21.15.32:0/1879000571 wait complete. 2020-12-05T17:36:55.018 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-12-05T17:36:55.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.139+0000 7f18b70b2700 1 Processor -- start 2020-12-05T17:36:55.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.139+0000 7f18b70b2700 1 -- start start 2020-12-05T17:36:55.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.139+0000 7f18b70b2700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f18b00b5b20 0x7f18b00b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:55.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18b70b2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f18b00b5240 0x7f18b0145f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:55.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18b70b2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b00b1920 con 0x7f18b00b6690 2020-12-05T17:36:55.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18b70b2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b00b1bc0 con 0x7f18b00b5240 2020-12-05T17:36:55.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18b70b2700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b00b5680 con 0x7f18b00b5b20 2020-12-05T17:36:55.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18b564f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f18b00b5240 0x7f18b0145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:55.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18affff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f18b00b6690 0x7f18b00b2060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38450/0 (socket says 172.21.15.32:38450) 2020-12-05T17:36:55.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18affff700 1 -- 172.21.15.32:0/2187156628 learned_addr learned my addr 172.21.15.32:0/2187156628 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:36:55.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18b564f700 1 --2- 172.21.15.32:0/2187156628 >> v2:172.21.15.32:3300/0 conn(0x7f18b00b5240 0x7f18b0145f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35806/0 (socket says 172.21.15.32:35806) 2020-12-05T17:36:55.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18b4e4e700 1 --2- 172.21.15.32:0/2187156628 >> v2:172.21.15.61:3300/0 conn(0x7f18b00b5b20 0x7f18b00b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:55.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18b564f700 1 -- 172.21.15.32:0/2187156628 >> v2:172.21.15.61:3300/0 conn(0x7f18b00b5b20 msgr2=0x7f18b00b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:36:55.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18b564f700 1 --2- 172.21.15.32:0/2187156628 >> v2:172.21.15.61:3300/0 conn(0x7f18b00b5b20 0x7f18b00b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:55.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18b564f700 1 -- 172.21.15.32:0/2187156628 >> v1:172.21.15.32:6789/0 conn(0x7f18b00b6690 legacy=0x7f18b00b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:55.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18b564f700 1 -- 172.21.15.32:0/2187156628 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b0146400 con 0x7f18b00b5240 2020-12-05T17:36:55.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18b564f700 1 --2- 172.21.15.32:0/2187156628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b00b5240 0x7f18b0145f10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f18a80032b0 tx=0x7f18a80091a0).ready entity=mon.0 client_cookie=eed8ffc915a1923c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:55.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18af7fe700 1 -- 172.21.15.32:0/2187156628 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18a800ba50 con 0x7f18b00b5240 2020-12-05T17:36:55.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18af7fe700 1 -- 172.21.15.32:0/2187156628 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a800bbb0 con 0x7f18b00b5240 2020-12-05T17:36:55.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18af7fe700 1 -- 172.21.15.32:0/2187156628 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18a800bd20 con 0x7f18b00b5240 2020-12-05T17:36:55.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18b70b2700 1 -- 172.21.15.32:0/2187156628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b00b5240 msgr2=0x7f18b0145f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:55.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18b70b2700 1 --2- 172.21.15.32:0/2187156628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b00b5240 0x7f18b0145f10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f18a80032b0 tx=0x7f18a80091a0).stop 2020-12-05T17:36:55.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18b70b2700 1 -- 172.21.15.32:0/2187156628 shutdown_connections 2020-12-05T17:36:55.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18b70b2700 1 --2- 172.21.15.32:0/2187156628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b00b5240 0x7f18b0145f10 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:55.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18b70b2700 1 --2- 172.21.15.32:0/2187156628 >> v2:172.21.15.61:3300/0 conn(0x7f18b00b5b20 0x7f18b00b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:55.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18b70b2700 1 -- 172.21.15.32:0/2187156628 >> 172.21.15.32:0/2187156628 conn(0x7f18b005f3b0 msgr2=0x7f18b005f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:55.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18b70b2700 1 -- 172.21.15.32:0/2187156628 shutdown_connections 2020-12-05T17:36:55.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18b70b2700 1 -- 172.21.15.32:0/2187156628 wait complete. 2020-12-05T17:36:55.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18b70b2700 1 Processor -- start 2020-12-05T17:36:55.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.143+0000 7f18b70b2700 1 -- start start 2020-12-05T17:36:55.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.147+0000 7f18b70b2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18b00b5240 0x7f18b00a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:55.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.147+0000 7f18b70b2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b00b5b20 0x7f18b00aead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:55.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.147+0000 7f18b70b2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18b00b6690 0x7f18b00aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:55.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.147+0000 7f18b70b2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b01528d0 con 0x7f18b00b5b20 2020-12-05T17:36:55.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.147+0000 7f18b70b2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b0152d00 con 0x7f18b00b6690 2020-12-05T17:36:55.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.147+0000 7f18b70b2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b00a65d0 con 0x7f18b00b5240 2020-12-05T17:36:55.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.147+0000 7f18affff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b00b5b20 0x7f18b00aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:55.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.147+0000 7f18affff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b00b5b20 0x7f18b00aead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35810/0 (socket says 172.21.15.32:35810) 2020-12-05T17:36:55.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.147+0000 7f18affff700 1 -- 172.21.15.32:0/2747241232 learned_addr learned my addr 172.21.15.32:0/2747241232 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:55.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.147+0000 7f18b564f700 1 --2- 172.21.15.32:0/2747241232 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18b00b6690 0x7f18b00aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:55.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.147+0000 7f18b4e4e700 1 --2- 172.21.15.32:0/2747241232 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18b00b5240 0x7f18b00a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:55.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.147+0000 7f18affff700 1 -- 172.21.15.32:0/2747241232 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18b00b5240 msgr2=0x7f18b00a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:55.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.147+0000 7f18affff700 1 --2- 172.21.15.32:0/2747241232 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18b00b5240 0x7f18b00a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:55.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.147+0000 7f18affff700 1 -- 172.21.15.32:0/2747241232 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18b00b6690 msgr2=0x7f18b00aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:55.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.147+0000 7f18affff700 1 --2- 172.21.15.32:0/2747241232 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18b00b6690 0x7f18b00aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:55.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.147+0000 7f18affff700 1 -- 172.21.15.32:0/2747241232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b00af4b0 con 0x7f18b00b5b20 2020-12-05T17:36:55.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.147+0000 7f18affff700 1 --2- 172.21.15.32:0/2747241232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b00b5b20 0x7f18b00aead0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f18b0061150 tx=0x7f18a0007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:55.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.147+0000 7f18adffb700 1 -- 172.21.15.32:0/2747241232 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18a0007320 con 0x7f18b00b5b20 2020-12-05T17:36:55.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.147+0000 7f18adffb700 1 -- 172.21.15.32:0/2747241232 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a0007480 con 0x7f18b00b5b20 2020-12-05T17:36:55.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.147+0000 7f18b70b2700 1 -- 172.21.15.32:0/2747241232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b0066ad0 con 0x7f18b00b5b20 2020-12-05T17:36:55.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.147+0000 7f18adffb700 1 -- 172.21.15.32:0/2747241232 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18a0005220 con 0x7f18b00b5b20 2020-12-05T17:36:55.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.147+0000 7f18b70b2700 1 -- 172.21.15.32:0/2747241232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b00a2c60 con 0x7f18b00b5b20 2020-12-05T17:36:55.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.147+0000 7f18b70b2700 1 -- 172.21.15.32:0/2747241232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b0152a40 con 0x7f18b00b5b20 2020-12-05T17:36:55.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.151+0000 7f18adffb700 1 -- 172.21.15.32:0/2747241232 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f18a001b070 con 0x7f18b00b5b20 2020-12-05T17:36:55.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.151+0000 7f18adffb700 1 --2- 172.21.15.32:0/2747241232 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1890055440 0x7f1890057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:55.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.151+0000 7f18adffb700 1 -- 172.21.15.32:0/2747241232 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f18a006d520 con 0x7f18b00b5b20 2020-12-05T17:36:55.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.151+0000 7f18b4e4e700 1 --2- 172.21.15.32:0/2747241232 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1890055440 0x7f1890057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:55.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.151+0000 7f18b4e4e700 1 --2- 172.21.15.32:0/2747241232 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1890055440 0x7f1890057900 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f18a400c000 tx=0x7f18a4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:55.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.155+0000 7f18adffb700 1 -- 172.21.15.32:0/2747241232 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f18b0152a40 con 0x7f18b00b5b20 2020-12-05T17:36:55.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:55.299+0000 7f18b70b2700 1 -- 172.21.15.32:0/2747241232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f18b00a6410 con 0x7f18b00b5b20 2020-12-05T17:36:56.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.003+0000 7f18adffb700 1 -- 172.21.15.32:0/2747241232 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v56) v1 ==== 185+0+0 (secure 0 0 0) 0x7f18a0047000 con 0x7f18b00b5b20 2020-12-05T17:36:56.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.007+0000 7f18b70b2700 1 -- 172.21.15.32:0/2747241232 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1890055440 msgr2=0x7f1890057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:56.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.007+0000 7f18b70b2700 1 --2- 172.21.15.32:0/2747241232 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1890055440 0x7f1890057900 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f18a400c000 tx=0x7f18a4008040).stop 2020-12-05T17:36:56.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.007+0000 7f18b70b2700 1 -- 172.21.15.32:0/2747241232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b00b5b20 msgr2=0x7f18b00aead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:56.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.007+0000 7f18b70b2700 1 --2- 172.21.15.32:0/2747241232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b00b5b20 0x7f18b00aead0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f18b0061150 tx=0x7f18a0007c30).stop 2020-12-05T17:36:56.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.007+0000 7f18b70b2700 1 -- 172.21.15.32:0/2747241232 shutdown_connections 2020-12-05T17:36:56.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.007+0000 7f18b70b2700 1 --2- 172.21.15.32:0/2747241232 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18b00b6690 0x7f18b00aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:56.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.007+0000 7f18b70b2700 1 --2- 172.21.15.32:0/2747241232 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1890055440 0x7f1890057900 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:56.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.007+0000 7f18b70b2700 1 --2- 172.21.15.32:0/2747241232 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18b00b5240 0x7f18b00a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:56.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.007+0000 7f18b70b2700 1 --2- 172.21.15.32:0/2747241232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b00b5b20 0x7f18b00aead0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:56.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.007+0000 7f18b70b2700 1 -- 172.21.15.32:0/2747241232 >> 172.21.15.32:0/2747241232 conn(0x7f18b005f3b0 msgr2=0x7f18b0143e10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:56.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.007+0000 7f18b70b2700 1 -- 172.21.15.32:0/2747241232 shutdown_connections 2020-12-05T17:36:56.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.007+0000 7f18b70b2700 1 -- 172.21.15.32:0/2747241232 wait complete. 2020-12-05T17:36:56.012 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2020-12-05T17:36:56.024 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-12-05T17:36:56.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.143+0000 7fd531897700 1 Processor -- start 2020-12-05T17:36:56.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.143+0000 7fd531897700 1 -- start start 2020-12-05T17:36:56.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.143+0000 7fd531897700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd52c061b30 0x7fd52c141e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:56.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.143+0000 7fd531897700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd52c142300 0x7fd52c142790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:56.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.143+0000 7fd531897700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd52c1439c0 con 0x7fd52c05f6f0 2020-12-05T17:36:56.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.143+0000 7fd531897700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd52c05fe20 con 0x7fd52c142300 2020-12-05T17:36:56.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.143+0000 7fd531897700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c146910 con 0x7fd52c061b30 2020-12-05T17:36:56.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.143+0000 7fd530895700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd52c05f6f0 0x7fd52c141700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38464/0 (socket says 172.21.15.32:38464) 2020-12-05T17:36:56.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd530895700 1 -- 172.21.15.32:0/4209623580 learned_addr learned my addr 172.21.15.32:0/4209623580 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:36:56.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd531096700 1 --2- 172.21.15.32:0/4209623580 >> v2:172.21.15.61:3300/0 conn(0x7fd52c142300 0x7fd52c142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:56.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd52bfff700 1 --2- 172.21.15.32:0/4209623580 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd52c061b30 0x7fd52c141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:56.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd52b7fe700 1 -- 172.21.15.32:0/4209623580 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1560067267 0 0) 0x7fd52c1439c0 con 0x7fd52c05f6f0 2020-12-05T17:36:56.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd52b7fe700 1 -- 172.21.15.32:0/4209623580 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd50c003230 con 0x7fd52c05f6f0 2020-12-05T17:36:56.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd52bfff700 1 -- 172.21.15.32:0/4209623580 >> v2:172.21.15.61:3300/0 conn(0x7fd52c142300 msgr2=0x7fd52c142790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:36:56.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd52bfff700 1 --2- 172.21.15.32:0/4209623580 >> v2:172.21.15.61:3300/0 conn(0x7fd52c142300 0x7fd52c142790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:56.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd52bfff700 1 -- 172.21.15.32:0/4209623580 >> v1:172.21.15.32:6789/0 conn(0x7fd52c05f6f0 legacy=0x7fd52c141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:56.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd52bfff700 1 -- 172.21.15.32:0/4209623580 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52c05fad0 con 0x7fd52c061b30 2020-12-05T17:36:56.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd531096700 1 --2- 172.21.15.32:0/4209623580 >> v2:172.21.15.61:3300/0 conn(0x7fd52c142300 0x7fd52c142790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:36:56.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd52bfff700 1 --2- 172.21.15.32:0/4209623580 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd52c061b30 0x7fd52c141e10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fd514002f30 tx=0x7fd514006dc0).ready entity=mon.2 client_cookie=c1b08e3fd5d6f405 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:56.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd52b7fe700 1 -- 172.21.15.32:0/4209623580 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5140096a0 con 0x7fd52c061b30 2020-12-05T17:36:56.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd52b7fe700 1 -- 172.21.15.32:0/4209623580 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd514009800 con 0x7fd52c061b30 2020-12-05T17:36:56.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd52b7fe700 1 -- 172.21.15.32:0/4209623580 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd514009b60 con 0x7fd52c061b30 2020-12-05T17:36:56.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd531897700 1 -- 172.21.15.32:0/4209623580 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd52c061b30 msgr2=0x7fd52c141e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:56.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd531897700 1 --2- 172.21.15.32:0/4209623580 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd52c061b30 0x7fd52c141e10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fd514002f30 tx=0x7fd514006dc0).stop 2020-12-05T17:36:56.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd531897700 1 -- 172.21.15.32:0/4209623580 shutdown_connections 2020-12-05T17:36:56.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd531897700 1 --2- 172.21.15.32:0/4209623580 >> v2:172.21.15.61:3300/0 conn(0x7fd52c142300 0x7fd52c142790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:56.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd531897700 1 --2- 172.21.15.32:0/4209623580 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd52c061b30 0x7fd52c141e10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:56.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd531897700 1 -- 172.21.15.32:0/4209623580 >> 172.21.15.32:0/4209623580 conn(0x7fd52c061010 msgr2=0x7fd52c061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:56.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd531897700 1 -- 172.21.15.32:0/4209623580 shutdown_connections 2020-12-05T17:36:56.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd531897700 1 -- 172.21.15.32:0/4209623580 wait complete. 2020-12-05T17:36:56.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd531897700 1 Processor -- start 2020-12-05T17:36:56.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.147+0000 7fd531897700 1 -- start start 2020-12-05T17:36:56.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd531897700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52c05f6f0 0x7fd52c144f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:56.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd531897700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd52c061b30 0x7fd52c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd531897700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd52c142300 0x7fd52c154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd531897700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c0b5c40 con 0x7fd52c05f6f0 2020-12-05T17:36:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd531897700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c0b5e70 con 0x7fd52c061b30 2020-12-05T17:36:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd531897700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c0b6110 con 0x7fd52c142300 2020-12-05T17:36:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd530895700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52c05f6f0 0x7fd52c144f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd530895700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52c05f6f0 0x7fd52c144f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35822/0 (socket says 172.21.15.32:35822) 2020-12-05T17:36:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd530895700 1 -- 172.21.15.32:0/1424692240 learned_addr learned my addr 172.21.15.32:0/1424692240 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd52bfff700 1 --2- 172.21.15.32:0/1424692240 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd52c061b30 0x7fd52c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd531096700 1 --2- 172.21.15.32:0/1424692240 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd52c142300 0x7fd52c154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:56.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd530895700 1 -- 172.21.15.32:0/1424692240 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd52c142300 msgr2=0x7fd52c154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:56.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd530895700 1 --2- 172.21.15.32:0/1424692240 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd52c142300 0x7fd52c154b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:56.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd530895700 1 -- 172.21.15.32:0/1424692240 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd52c061b30 msgr2=0x7fd52c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:56.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd530895700 1 --2- 172.21.15.32:0/1424692240 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd52c061b30 0x7fd52c0b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:56.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd530895700 1 -- 172.21.15.32:0/1424692240 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52c0b63b0 con 0x7fd52c05f6f0 2020-12-05T17:36:56.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd531096700 1 --2- 172.21.15.32:0/1424692240 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd52c142300 0x7fd52c154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:36:56.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd530895700 1 --2- 172.21.15.32:0/1424692240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52c05f6f0 0x7fd52c144f50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fd51c001250 tx=0x7fd51c002910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:56.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd529ffb700 1 -- 172.21.15.32:0/1424692240 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd51c007320 con 0x7fd52c05f6f0 2020-12-05T17:36:56.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd531897700 1 -- 172.21.15.32:0/1424692240 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52c0b5380 con 0x7fd52c05f6f0 2020-12-05T17:36:56.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd529ffb700 1 -- 172.21.15.32:0/1424692240 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51c0054b0 con 0x7fd52c05f6f0 2020-12-05T17:36:56.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd529ffb700 1 -- 172.21.15.32:0/1424692240 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd51c005810 con 0x7fd52c05f6f0 2020-12-05T17:36:56.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd531897700 1 -- 172.21.15.32:0/1424692240 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52c155270 con 0x7fd52c05f6f0 2020-12-05T17:36:56.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd531897700 1 -- 172.21.15.32:0/1424692240 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52c057050 con 0x7fd52c05f6f0 2020-12-05T17:36:56.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd529ffb700 1 -- 172.21.15.32:0/1424692240 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd51c010030 con 0x7fd52c05f6f0 2020-12-05T17:36:56.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd529ffb700 1 --2- 172.21.15.32:0/1424692240 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd5100554e0 0x7fd5100579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:56.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd529ffb700 1 -- 172.21.15.32:0/1424692240 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6068+0+0 (secure 0 0 0) 0x7fd51c06dd20 con 0x7fd52c05f6f0 2020-12-05T17:36:56.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.151+0000 7fd52bfff700 1 --2- 172.21.15.32:0/1424692240 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd5100554e0 0x7fd5100579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:56.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.155+0000 7fd52bfff700 1 --2- 172.21.15.32:0/1424692240 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd5100554e0 0x7fd5100579a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fd51401b000 tx=0x7fd514015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:56.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.159+0000 7fd529ffb700 1 -- 172.21.15.32:0/1424692240 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd51c043f80 con 0x7fd52c05f6f0 2020-12-05T17:36:56.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:56.295+0000 7fd531897700 1 -- 172.21.15.32:0/1424692240 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd52c0b6540 con 0x7fd52c05f6f0 2020-12-05T17:36:57.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.019+0000 7fd529ffb700 1 -- 172.21.15.32:0/1424692240 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v57) v1 ==== 173+0+0 (secure 0 0 0) 0x7fd51c0475a0 con 0x7fd52c05f6f0 2020-12-05T17:36:57.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.019+0000 7fd531897700 1 -- 172.21.15.32:0/1424692240 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd5100554e0 msgr2=0x7fd5100579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:57.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.019+0000 7fd531897700 1 --2- 172.21.15.32:0/1424692240 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd5100554e0 0x7fd5100579a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fd51401b000 tx=0x7fd514015000).stop 2020-12-05T17:36:57.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.019+0000 7fd531897700 1 -- 172.21.15.32:0/1424692240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52c05f6f0 msgr2=0x7fd52c144f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:57.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.019+0000 7fd531897700 1 --2- 172.21.15.32:0/1424692240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52c05f6f0 0x7fd52c144f50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fd51c001250 tx=0x7fd51c002910).stop 2020-12-05T17:36:57.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.019+0000 7fd531897700 1 -- 172.21.15.32:0/1424692240 shutdown_connections 2020-12-05T17:36:57.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.019+0000 7fd531897700 1 --2- 172.21.15.32:0/1424692240 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd5100554e0 0x7fd5100579a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:57.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.019+0000 7fd531897700 1 --2- 172.21.15.32:0/1424692240 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd52c142300 0x7fd52c154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:57.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.019+0000 7fd531897700 1 --2- 172.21.15.32:0/1424692240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52c05f6f0 0x7fd52c144f50 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:57.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.019+0000 7fd531897700 1 --2- 172.21.15.32:0/1424692240 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd52c061b30 0x7fd52c0b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:57.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.019+0000 7fd531897700 1 -- 172.21.15.32:0/1424692240 >> 172.21.15.32:0/1424692240 conn(0x7fd52c061010 msgr2=0x7fd52c142710 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:57.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.023+0000 7fd531897700 1 -- 172.21.15.32:0/1424692240 shutdown_connections 2020-12-05T17:36:57.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.023+0000 7fd531897700 1 -- 172.21.15.32:0/1424692240 wait complete. 2020-12-05T17:36:57.025 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2020-12-05T17:36:57.040 INFO:tasks.ceph.mds.a:Restarting daemon 2020-12-05T17:36:57.041 INFO:tasks.ceph.mds.a:Stopping old one... 2020-12-05T17:36:57.041 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-12-05T17:36:57.041 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:36:57.042 INFO:tasks.ceph.mds.b:Restarting daemon 2020-12-05T17:36:57.042 INFO:tasks.ceph.mds.b:Stopping old one... 2020-12-05T17:36:57.042 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-12-05T17:36:57.043 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:36:57.050 INFO:tasks.ceph.mds.a:Stopped 2020-12-05T17:36:57.050 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-12-05T17:36:57.054 INFO:tasks.ceph.mds.b:Stopped 2020-12-05T17:36:57.054 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-12-05T17:36:57.091 INFO:tasks.ceph.mds.a:Started 2020-12-05T17:36:57.092 DEBUG:teuthology.parallel:result is None 2020-12-05T17:36:57.099 INFO:tasks.ceph.mds.b:Started 2020-12-05T17:36:57.100 DEBUG:teuthology.parallel:result is None 2020-12-05T17:36:57.100 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-12-05T17:36:57.172 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:36:57.178 INFO:tasks.ceph.mds.b.smithi061.stdout:starting mds.b at 2020-12-05T17:36:57.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.219+0000 7fb7f9457700 1 Processor -- start 2020-12-05T17:36:57.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.219+0000 7fb7f9457700 1 -- start start 2020-12-05T17:36:57.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f9457700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7f405f6f0 0x7fb7f4143b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:57.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f9457700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7f40aff10 0x7fb7f413d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:57.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f9457700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f4063040 con 0x7fb7f40af810 2020-12-05T17:36:57.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f9457700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f405fad0 con 0x7fb7f405f6f0 2020-12-05T17:36:57.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f9457700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f40b3030 con 0x7fb7f40aff10 2020-12-05T17:36:57.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f37fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb7f40af810 0x7fb7f40b1d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38492/0 (socket says 172.21.15.32:38492) 2020-12-05T17:36:57.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f37fe700 1 -- 172.21.15.32:0/61041785 learned_addr learned my addr 172.21.15.32:0/61041785 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:36:57.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7f405f6f0 0x7fb7f4143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:57.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f8c56700 1 --2- 172.21.15.32:0/61041785 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7f40aff10 0x7fb7f413d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:57.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f3fff700 1 -- 172.21.15.32:0/61041785 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7f40aff10 msgr2=0x7fb7f413d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:36:57.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f3fff700 1 --2- 172.21.15.32:0/61041785 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7f40aff10 0x7fb7f413d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:57.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f3fff700 1 -- 172.21.15.32:0/61041785 >> v1:172.21.15.32:6789/0 conn(0x7fb7f40af810 legacy=0x7fb7f40b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:57.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f3fff700 1 -- 172.21.15.32:0/61041785 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f40b24a0 con 0x7fb7f405f6f0 2020-12-05T17:36:57.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f2ffd700 1 -- 172.21.15.32:0/61041785 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1763977985 0 0) 0x7fb7f4063040 con 0x7fb7f40af810 2020-12-05T17:36:57.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f3fff700 1 --2- 172.21.15.32:0/61041785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f405f6f0 0x7fb7f4143b90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7fb7e4002f30 tx=0x7fb7e4006dc0).ready entity=mon.0 client_cookie=bc0b505b2566db07 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:57.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f2ffd700 1 -- 172.21.15.32:0/61041785 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7e4009760 con 0x7fb7f405f6f0 2020-12-05T17:36:57.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f2ffd700 1 -- 172.21.15.32:0/61041785 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e40098c0 con 0x7fb7f405f6f0 2020-12-05T17:36:57.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f2ffd700 1 -- 172.21.15.32:0/61041785 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7e4009c20 con 0x7fb7f405f6f0 2020-12-05T17:36:57.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f9457700 1 -- 172.21.15.32:0/61041785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f405f6f0 msgr2=0x7fb7f4143b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:57.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f9457700 1 --2- 172.21.15.32:0/61041785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f405f6f0 0x7fb7f4143b90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7fb7e4002f30 tx=0x7fb7e4006dc0).stop 2020-12-05T17:36:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f9457700 1 -- 172.21.15.32:0/61041785 shutdown_connections 2020-12-05T17:36:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f9457700 1 --2- 172.21.15.32:0/61041785 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7f40aff10 0x7fb7f413d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f9457700 1 --2- 172.21.15.32:0/61041785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f405f6f0 0x7fb7f4143b90 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f9457700 1 -- 172.21.15.32:0/61041785 >> 172.21.15.32:0/61041785 conn(0x7fb7f40604c0 msgr2=0x7fb7f405c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f9457700 1 -- 172.21.15.32:0/61041785 shutdown_connections 2020-12-05T17:36:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.223+0000 7fb7f9457700 1 -- 172.21.15.32:0/61041785 wait complete. 2020-12-05T17:36:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f9457700 1 Processor -- start 2020-12-05T17:36:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f9457700 1 -- start start 2020-12-05T17:36:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f9457700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb7f405f6f0 0x7fb7f413eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f9457700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7f40af810 0x7fb7f413efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f9457700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f40aff10 0x7fb7f41409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f9457700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f413dcd0 con 0x7fb7f40aff10 2020-12-05T17:36:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f9457700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f4066630 con 0x7fb7f405f6f0 2020-12-05T17:36:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f9457700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f40641a0 con 0x7fb7f40af810 2020-12-05T17:36:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f8c56700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f40aff10 0x7fb7f41409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f8c56700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f40aff10 0x7fb7f41409d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35850/0 (socket says 172.21.15.32:35850) 2020-12-05T17:36:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f8c56700 1 -- 172.21.15.32:0/238530256 learned_addr learned my addr 172.21.15.32:0/238530256 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f3fff700 1 --2- 172.21.15.32:0/238530256 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb7f405f6f0 0x7fb7f413eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f37fe700 1 --2- 172.21.15.32:0/238530256 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7f40af810 0x7fb7f413efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f8c56700 1 -- 172.21.15.32:0/238530256 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7f40af810 msgr2=0x7fb7f413efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f8c56700 1 --2- 172.21.15.32:0/238530256 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7f40af810 0x7fb7f413efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f8c56700 1 -- 172.21.15.32:0/238530256 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb7f405f6f0 msgr2=0x7fb7f413eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f8c56700 1 --2- 172.21.15.32:0/238530256 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb7f405f6f0 0x7fb7f413eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f8c56700 1 -- 172.21.15.32:0/238530256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f4064440 con 0x7fb7f40aff10 2020-12-05T17:36:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f37fe700 1 --2- 172.21.15.32:0/238530256 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7f40af810 0x7fb7f413efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:36:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f8c56700 1 --2- 172.21.15.32:0/238530256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f40aff10 0x7fb7f41409d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fb7ec006dd0 tx=0x7fb7ec0091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f17fa700 1 -- 172.21.15.32:0/238530256 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7ec00c950 con 0x7fb7f40aff10 2020-12-05T17:36:57.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f17fa700 1 -- 172.21.15.32:0/238530256 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ec00cab0 con 0x7fb7f40aff10 2020-12-05T17:36:57.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f17fa700 1 -- 172.21.15.32:0/238530256 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7ec00ce10 con 0x7fb7f40aff10 2020-12-05T17:36:57.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f9457700 1 -- 172.21.15.32:0/238530256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f4065be0 con 0x7fb7f40aff10 2020-12-05T17:36:57.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.227+0000 7fb7f9457700 1 -- 172.21.15.32:0/238530256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f4066080 con 0x7fb7f40aff10 2020-12-05T17:36:57.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.231+0000 7fb7f9457700 1 -- 172.21.15.32:0/238530256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f4065d70 con 0x7fb7f40aff10 2020-12-05T17:36:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.231+0000 7fb7f17fa700 1 -- 172.21.15.32:0/238530256 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb7ec022070 con 0x7fb7f40aff10 2020-12-05T17:36:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.231+0000 7fb7f17fa700 1 --2- 172.21.15.32:0/238530256 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb7d80553b0 0x7fb7d8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.231+0000 7fb7f17fa700 1 -- 172.21.15.32:0/238530256 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fb7ec06e4a0 con 0x7fb7f40aff10 2020-12-05T17:36:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.231+0000 7fb7f3fff700 1 --2- 172.21.15.32:0/238530256 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb7d80553b0 0x7fb7d8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:57.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.231+0000 7fb7f3fff700 1 --2- 172.21.15.32:0/238530256 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb7d80553b0 0x7fb7d8057870 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7fb7e401a000 tx=0x7fb7e4015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:57.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.235+0000 7fb7f17fa700 1 -- 172.21.15.32:0/238530256 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb7ec044880 con 0x7fb7f40aff10 2020-12-05T17:36:57.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:57.387+0000 7fb7f9457700 1 -- 172.21.15.32:0/238530256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fb7f4057050 con 0x7fb7f40aff10 2020-12-05T17:36:58.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.019+0000 7fb7f17fa700 1 -- 172.21.15.32:0/238530256 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v58) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb7ec016070 con 0x7fb7f40aff10 2020-12-05T17:36:58.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.023+0000 7fb7f9457700 1 -- 172.21.15.32:0/238530256 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb7d80553b0 msgr2=0x7fb7d8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:58.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.023+0000 7fb7f9457700 1 --2- 172.21.15.32:0/238530256 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb7d80553b0 0x7fb7d8057870 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7fb7e401a000 tx=0x7fb7e4015000).stop 2020-12-05T17:36:58.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.023+0000 7fb7f9457700 1 -- 172.21.15.32:0/238530256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f40aff10 msgr2=0x7fb7f41409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:58.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.023+0000 7fb7f9457700 1 --2- 172.21.15.32:0/238530256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f40aff10 0x7fb7f41409d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fb7ec006dd0 tx=0x7fb7ec0091a0).stop 2020-12-05T17:36:58.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.023+0000 7fb7f9457700 1 -- 172.21.15.32:0/238530256 shutdown_connections 2020-12-05T17:36:58.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.023+0000 7fb7f9457700 1 --2- 172.21.15.32:0/238530256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f40aff10 0x7fb7f41409d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.023+0000 7fb7f9457700 1 --2- 172.21.15.32:0/238530256 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb7f405f6f0 0x7fb7f413eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.023+0000 7fb7f9457700 1 --2- 172.21.15.32:0/238530256 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb7d80553b0 0x7fb7d8057870 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.023+0000 7fb7f9457700 1 --2- 172.21.15.32:0/238530256 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7f40af810 0x7fb7f413efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.023+0000 7fb7f9457700 1 -- 172.21.15.32:0/238530256 >> 172.21.15.32:0/238530256 conn(0x7fb7f40604c0 msgr2=0x7fb7f40b3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.023+0000 7fb7f9457700 1 -- 172.21.15.32:0/238530256 shutdown_connections 2020-12-05T17:36:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.023+0000 7fb7f9457700 1 -- 172.21.15.32:0/238530256 wait complete. 2020-12-05T17:36:58.028 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2020-12-05T17:36:58.042 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2020-12-05T17:36:58.042 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-12-05T17:36:58.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.159+0000 7fddc8a3d700 1 Processor -- start 2020-12-05T17:36:58.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.159+0000 7fddc8a3d700 1 -- start start 2020-12-05T17:36:58.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.159+0000 7fddc8a3d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fddc40b52f0 0x7fddc40b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:58.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.159+0000 7fddc8a3d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fddc40b1870 0x7fddc4138810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:58.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.159+0000 7fddc8a3d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc41420f0 con 0x7fddc4138d00 2020-12-05T17:36:58.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.159+0000 7fddc8a3d700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc40ad4f0 con 0x7fddc40b52f0 2020-12-05T17:36:58.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.159+0000 7fddc8a3d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc40b0380 con 0x7fddc40b1870 2020-12-05T17:36:58.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.159+0000 7fddc3fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fddc4138d00 0x7fddc40ae860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38506/0 (socket says 172.21.15.32:38506) 2020-12-05T17:36:58.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.159+0000 7fddc3fff700 1 -- 172.21.15.32:0/3679540702 learned_addr learned my addr 172.21.15.32:0/3679540702 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:36:58.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.163+0000 7fddc2ffd700 1 --2- 172.21.15.32:0/3679540702 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fddc40b1870 0x7fddc4138810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:58.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.163+0000 7fddc37fe700 1 --2- 172.21.15.32:0/3679540702 >> v2:172.21.15.61:3300/0 conn(0x7fddc40b52f0 0x7fddc40b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:58.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.163+0000 7fddc27fc700 1 -- 172.21.15.32:0/3679540702 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4114433201 0 0) 0x7fddc41420f0 con 0x7fddc4138d00 2020-12-05T17:36:58.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.163+0000 7fddc27fc700 1 -- 172.21.15.32:0/3679540702 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddb0003230 con 0x7fddc4138d00 2020-12-05T17:36:58.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.163+0000 7fddc2ffd700 1 -- 172.21.15.32:0/3679540702 >> v2:172.21.15.61:3300/0 conn(0x7fddc40b52f0 msgr2=0x7fddc40b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:36:58.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.163+0000 7fddc2ffd700 1 --2- 172.21.15.32:0/3679540702 >> v2:172.21.15.61:3300/0 conn(0x7fddc40b52f0 0x7fddc40b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.163+0000 7fddc2ffd700 1 -- 172.21.15.32:0/3679540702 >> v1:172.21.15.32:6789/0 conn(0x7fddc4138d00 legacy=0x7fddc40ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:58.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.163+0000 7fddc2ffd700 1 -- 172.21.15.32:0/3679540702 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc40b0620 con 0x7fddc40b1870 2020-12-05T17:36:58.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.163+0000 7fddc2ffd700 1 --2- 172.21.15.32:0/3679540702 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fddc40b1870 0x7fddc4138810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fddc40adc80 tx=0x7fddb8006dc0).ready entity=mon.2 client_cookie=499c35503673bf1 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:58.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.163+0000 7fddc27fc700 1 -- 172.21.15.32:0/3679540702 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddb800f9c0 con 0x7fddc40b1870 2020-12-05T17:36:58.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.163+0000 7fddc27fc700 1 -- 172.21.15.32:0/3679540702 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb800fb20 con 0x7fddc40b1870 2020-12-05T17:36:58.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.163+0000 7fddc27fc700 1 -- 172.21.15.32:0/3679540702 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddb800fe80 con 0x7fddc40b1870 2020-12-05T17:36:58.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.163+0000 7fddc8a3d700 1 -- 172.21.15.32:0/3679540702 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fddc40b1870 msgr2=0x7fddc4138810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:58.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.163+0000 7fddc8a3d700 1 --2- 172.21.15.32:0/3679540702 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fddc40b1870 0x7fddc4138810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fddc40adc80 tx=0x7fddb8006dc0).stop 2020-12-05T17:36:58.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.163+0000 7fddc8a3d700 1 -- 172.21.15.32:0/3679540702 shutdown_connections 2020-12-05T17:36:58.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.163+0000 7fddc8a3d700 1 --2- 172.21.15.32:0/3679540702 >> v2:172.21.15.61:3300/0 conn(0x7fddc40b52f0 0x7fddc40b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.163+0000 7fddc8a3d700 1 --2- 172.21.15.32:0/3679540702 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fddc40b1870 0x7fddc4138810 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.163+0000 7fddc8a3d700 1 -- 172.21.15.32:0/3679540702 >> 172.21.15.32:0/3679540702 conn(0x7fddc405bbb0 msgr2=0x7fddc4062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:58.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.163+0000 7fddc8a3d700 1 -- 172.21.15.32:0/3679540702 shutdown_connections 2020-12-05T17:36:58.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.163+0000 7fddc8a3d700 1 -- 172.21.15.32:0/3679540702 wait complete. 2020-12-05T17:36:58.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc8a3d700 1 Processor -- start 2020-12-05T17:36:58.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc8a3d700 1 -- start start 2020-12-05T17:36:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc8a3d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc40b1870 0x7fddc4150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc8a3d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddc40b52f0 0x7fddc40af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc8a3d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fddc4138d00 0x7fddc4064a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:58.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc8a3d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc413b300 con 0x7fddc40b1870 2020-12-05T17:36:58.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc8a3d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc4139ed0 con 0x7fddc40b52f0 2020-12-05T17:36:58.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc8a3d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc413a170 con 0x7fddc4138d00 2020-12-05T17:36:58.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc40b1870 0x7fddc4150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:58.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc40b1870 0x7fddc4150880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35862/0 (socket says 172.21.15.32:35862) 2020-12-05T17:36:58.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc37fe700 1 -- 172.21.15.32:0/709461831 learned_addr learned my addr 172.21.15.32:0/709461831 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:58.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc2ffd700 1 --2- 172.21.15.32:0/709461831 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddc40b52f0 0x7fddc40af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:58.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc3fff700 1 --2- 172.21.15.32:0/709461831 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fddc4138d00 0x7fddc4064a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:58.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc37fe700 1 -- 172.21.15.32:0/709461831 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fddc4138d00 msgr2=0x7fddc4064a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:58.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc37fe700 1 --2- 172.21.15.32:0/709461831 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fddc4138d00 0x7fddc4064a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc37fe700 1 -- 172.21.15.32:0/709461831 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddc40b52f0 msgr2=0x7fddc40af220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:58.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc37fe700 1 --2- 172.21.15.32:0/709461831 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddc40b52f0 0x7fddc40af220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc37fe700 1 -- 172.21.15.32:0/709461831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc413a660 con 0x7fddc40b1870 2020-12-05T17:36:58.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc37fe700 1 --2- 172.21.15.32:0/709461831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc40b1870 0x7fddc4150880 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7fddb400c000 tx=0x7fddb40091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:58.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc0ff9700 1 -- 172.21.15.32:0/709461831 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddb400cbb0 con 0x7fddc40b1870 2020-12-05T17:36:58.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc8a3d700 1 -- 172.21.15.32:0/709461831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc413a910 con 0x7fddc40b1870 2020-12-05T17:36:58.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc0ff9700 1 -- 172.21.15.32:0/709461831 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb400cd10 con 0x7fddc40b1870 2020-12-05T17:36:58.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc0ff9700 1 -- 172.21.15.32:0/709461831 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddb40125e0 con 0x7fddc40b1870 2020-12-05T17:36:58.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc8a3d700 1 -- 172.21.15.32:0/709461831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc40666f0 con 0x7fddc40b1870 2020-12-05T17:36:58.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.167+0000 7fddc8a3d700 1 -- 172.21.15.32:0/709461831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc413b470 con 0x7fddc40b1870 2020-12-05T17:36:58.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.171+0000 7fddc0ff9700 1 -- 172.21.15.32:0/709461831 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fddb4022070 con 0x7fddc40b1870 2020-12-05T17:36:58.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.171+0000 7fddc0ff9700 1 --2- 172.21.15.32:0/709461831 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdda4055440 0x7fdda4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:58.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.171+0000 7fddc0ff9700 1 -- 172.21.15.32:0/709461831 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fddb406e510 con 0x7fddc40b1870 2020-12-05T17:36:58.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.171+0000 7fddc2ffd700 1 --2- 172.21.15.32:0/709461831 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdda4055440 0x7fdda4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:58.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.171+0000 7fddc2ffd700 1 --2- 172.21.15.32:0/709461831 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdda4055440 0x7fdda4057900 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fddb801b000 tx=0x7fddb8017000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:58.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.175+0000 7fddc0ff9700 1 -- 172.21.15.32:0/709461831 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fddc413b470 con 0x7fddc40b1870 2020-12-05T17:36:58.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.367+0000 7fddc8a3d700 1 -- 172.21.15.32:0/709461831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fddc413b440 con 0x7fddc40b1870 2020-12-05T17:36:58.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.367+0000 7fddc0ff9700 1 -- 172.21.15.32:0/709461831 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+4242 (secure 0 0 0) 0x7fddb4044950 con 0x7fddc40b1870 2020-12-05T17:36:58.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.367+0000 7fddc8a3d700 1 -- 172.21.15.32:0/709461831 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdda4055440 msgr2=0x7fdda4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:58.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.367+0000 7fddc8a3d700 1 --2- 172.21.15.32:0/709461831 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdda4055440 0x7fdda4057900 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fddb801b000 tx=0x7fddb8017000).stop 2020-12-05T17:36:58.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.367+0000 7fddc8a3d700 1 -- 172.21.15.32:0/709461831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc40b1870 msgr2=0x7fddc4150880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:58.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.367+0000 7fddc8a3d700 1 --2- 172.21.15.32:0/709461831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc40b1870 0x7fddc4150880 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7fddb400c000 tx=0x7fddb40091a0).stop 2020-12-05T17:36:58.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.367+0000 7fddc8a3d700 1 -- 172.21.15.32:0/709461831 shutdown_connections 2020-12-05T17:36:58.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.367+0000 7fddc8a3d700 1 --2- 172.21.15.32:0/709461831 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdda4055440 0x7fdda4057900 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.367+0000 7fddc8a3d700 1 --2- 172.21.15.32:0/709461831 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fddc4138d00 0x7fddc4064a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.367+0000 7fddc8a3d700 1 --2- 172.21.15.32:0/709461831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc40b1870 0x7fddc4150880 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.367+0000 7fddc8a3d700 1 --2- 172.21.15.32:0/709461831 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddc40b52f0 0x7fddc40af220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.367+0000 7fddc8a3d700 1 -- 172.21.15.32:0/709461831 >> 172.21.15.32:0/709461831 conn(0x7fddc405bbb0 msgr2=0x7fddc40b3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:58.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.367+0000 7fddc8a3d700 1 -- 172.21.15.32:0/709461831 shutdown_connections 2020-12-05T17:36:58.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.367+0000 7fddc8a3d700 1 -- 172.21.15.32:0/709461831 wait complete. 2020-12-05T17:36:58.380 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:36:58.380 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:36:58.381 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2020-12-05T17:36:58.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:58.382 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2020-12-05T17:36:58.382 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfpb/aExAAdMKW4yIgC2zhkzyW6l8xug==", 2020-12-05T17:36:58.382 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:36:58.383 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:36:58.383 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:36:58.383 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:36:58.384 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:36:58.384 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:58.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:58.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:58.385 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2020-12-05T17:36:58.385 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfBKTYGRAAaPkezwmTiAvkUfTuDGDNZA==", 2020-12-05T17:36:58.385 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:36:58.386 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:36:58.386 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:36:58.386 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:36:58.387 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:36:58.387 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:58.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:58.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:58.388 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2020-12-05T17:36:58.389 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHw8tflo5EKhAAwV9D64fAoY3uvXCm+Aksag==", 2020-12-05T17:36:58.389 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:36:58.389 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:36:58.390 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:36:58.390 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:36:58.390 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:58.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:58.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:58.391 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2020-12-05T17:36:58.392 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDKw8tfTjEiERAAkW6mkgApoXwyF49kPjZBGg==", 2020-12-05T17:36:58.392 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:36:58.392 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:36:58.393 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:36:58.393 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:36:58.393 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:58.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:58.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:58.395 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2020-12-05T17:36:58.395 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDMw8tfLEzBMxAAiJ2NifsOLEkt1e3vmhwSTA==", 2020-12-05T17:36:58.395 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:36:58.396 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:36:58.396 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:36:58.396 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:36:58.397 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:58.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:58.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:58.397 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2020-12-05T17:36:58.398 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPw8tfV+vnGxAAQHvXJRW5X9Vhe/eYJBJ/uQ==", 2020-12-05T17:36:58.398 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:36:58.398 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:36:58.399 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:36:58.399 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:36:58.399 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:58.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:58.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:58.400 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2020-12-05T17:36:58.400 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUw8tfN8wMJxAApc5LY9HGDeIV/o0eil/g4A==", 2020-12-05T17:36:58.400 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:36:58.401 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:36:58.401 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:36:58.401 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:36:58.401 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:58.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:58.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:58.402 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2020-12-05T17:36:58.403 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXw8tfvZ5dDxAAtAfftExkljlBvn7ZSoD/Og==", 2020-12-05T17:36:58.403 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:36:58.404 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:36:58.404 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:36:58.404 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:36:58.405 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:58.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:58.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:58.405 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2020-12-05T17:36:58.406 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDZw8tfhXFzMhAAZXNwNGfKv360Tq4N/DX5Mw==", 2020-12-05T17:36:58.406 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:36:58.406 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:36:58.407 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:36:58.407 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:36:58.407 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:58.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:58.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:58.408 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2020-12-05T17:36:58.408 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDcw8tfROfSGhAA/Y2Y1b4k5zQju0MM7LASeQ==", 2020-12-05T17:36:58.409 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:36:58.409 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:36:58.409 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:36:58.409 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:36:58.410 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:58.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:58.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:58.410 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2020-12-05T17:36:58.411 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfWDp/HxAAmvLPj7C3WDgj0OC2o7lBJQ==", 2020-12-05T17:36:58.411 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:36:58.412 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:36:58.412 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:36:58.412 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:36:58.413 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:58.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:58.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:58.414 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2020-12-05T17:36:58.414 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfugnmIRAAW6+/QGMLkHw+Z97RHpV4kw==", 2020-12-05T17:36:58.414 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:36:58.415 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:36:58.415 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:36:58.415 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:36:58.415 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:58.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:58.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:58.416 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2020-12-05T17:36:58.416 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfojyqJBAAyta+dSy2yGnC1kKe3YOAYw==", 2020-12-05T17:36:58.417 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:36:58.417 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:36:58.417 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:36:58.418 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:36:58.418 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:58.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:58.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:58.419 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2020-12-05T17:36:58.419 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDFw8tfES0PLBAAA1cUV44snAUDF2TyAmXImw==", 2020-12-05T17:36:58.419 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:36:58.419 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:36:58.420 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:36:58.420 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:36:58.420 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:36:58.420 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:58.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:58.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:58.421 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2020-12-05T17:36:58.422 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfDr8mERAAS7bpAQXgW78W8GDq1v1zaA==", 2020-12-05T17:36:58.422 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:36:58.422 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:36:58.423 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:36:58.423 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:36:58.424 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:58.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:58.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:58.424 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2020-12-05T17:36:58.425 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tf3HmcDhAATNKHhH9zgasedaNEzgG0OQ==", 2020-12-05T17:36:58.425 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:36:58.425 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:36:58.425 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:36:58.426 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:36:58.426 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:58.426 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:58.427 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:36:58.427 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:36:58.429 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:36:58.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.547+0000 7faed8777700 1 Processor -- start 2020-12-05T17:36:58.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.547+0000 7faed8777700 1 -- start start 2020-12-05T17:36:58.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed8777700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faed00ad1b0 0x7faed00aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:58.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed8777700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed013fb70 con 0x7faed00aead0 2020-12-05T17:36:58.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed8777700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed00af460 con 0x7faed005f490 2020-12-05T17:36:58.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed8777700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed00af700 con 0x7faed00ad1b0 2020-12-05T17:36:58.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed6513700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7faed00aead0 0x7faed01389f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38518/0 (socket says 172.21.15.32:38518) 2020-12-05T17:36:58.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed6513700 1 -- 172.21.15.32:0/164118444 learned_addr learned my addr 172.21.15.32:0/164118444 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:36:58.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed5d12700 1 --2- 172.21.15.32:0/164118444 >> v2:172.21.15.32:3300/0 conn(0x7faed00ad1b0 0x7faed00aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:58.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed5511700 1 -- 172.21.15.32:0/164118444 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2875160855 0 0) 0x7faed013fb70 con 0x7faed00aead0 2020-12-05T17:36:58.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed5511700 1 -- 172.21.15.32:0/164118444 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faeb8003230 con 0x7faed00aead0 2020-12-05T17:36:58.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed5511700 1 -- 172.21.15.32:0/164118444 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1107892996 0 0) 0x7faed00af460 con 0x7faed005f490 2020-12-05T17:36:58.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed5511700 1 -- 172.21.15.32:0/164118444 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faed013fb70 con 0x7faed005f490 2020-12-05T17:36:58.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed5d12700 1 -- 172.21.15.32:0/164118444 >> v1:172.21.15.61:6789/0 conn(0x7faed005f490 legacy=0x7faed005f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:58.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed5d12700 1 -- 172.21.15.32:0/164118444 >> v1:172.21.15.32:6789/0 conn(0x7faed00aead0 legacy=0x7faed01389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:58.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed5511700 1 -- 172.21.15.32:0/164118444 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1439750 0 0) 0x7faeb8003230 con 0x7faed00aead0 2020-12-05T17:36:58.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed5d12700 1 -- 172.21.15.32:0/164118444 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed0139100 con 0x7faed00ad1b0 2020-12-05T17:36:58.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed5d12700 1 --2- 172.21.15.32:0/164118444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed00ad1b0 0x7faed00aeeb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7faecc0010d0 tx=0x7faec0006dc0).ready entity=mon.0 client_cookie=98e9a790444bc9aa server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed5511700 1 -- 172.21.15.32:0/164118444 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faecc0016d0 con 0x7faed00ad1b0 2020-12-05T17:36:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed5511700 1 -- 172.21.15.32:0/164118444 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec000d610 con 0x7faed00ad1b0 2020-12-05T17:36:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed5511700 1 -- 172.21.15.32:0/164118444 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faec000d780 con 0x7faed00ad1b0 2020-12-05T17:36:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed8777700 1 -- 172.21.15.32:0/164118444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed00ad1b0 msgr2=0x7faed00aeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:58.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed8777700 1 --2- 172.21.15.32:0/164118444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed00ad1b0 0x7faed00aeeb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7faecc0010d0 tx=0x7faec0006dc0).stop 2020-12-05T17:36:58.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed8777700 1 -- 172.21.15.32:0/164118444 shutdown_connections 2020-12-05T17:36:58.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed8777700 1 --2- 172.21.15.32:0/164118444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed00ad1b0 0x7faed00aeeb0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.551+0000 7faed8777700 1 -- 172.21.15.32:0/164118444 >> 172.21.15.32:0/164118444 conn(0x7faed00b5580 msgr2=0x7faed00b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.555+0000 7faed8777700 1 -- 172.21.15.32:0/164118444 shutdown_connections 2020-12-05T17:36:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.555+0000 7faed8777700 1 -- 172.21.15.32:0/164118444 wait complete. 2020-12-05T17:36:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.555+0000 7faed8777700 1 Processor -- start 2020-12-05T17:36:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.555+0000 7faed8777700 1 -- start start 2020-12-05T17:36:58.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faed8777700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed005f490 0x7faed00674c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:58.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faed8777700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faed00ad1b0 0x7faed0135920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:58.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faed8777700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faed00aead0 0x7faed0135e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:58.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faed8777700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed013a950 con 0x7faed005f490 2020-12-05T17:36:58.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faed8777700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed013abf0 con 0x7faed00ad1b0 2020-12-05T17:36:58.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faed8777700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed00b3990 con 0x7faed00aead0 2020-12-05T17:36:58.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faed6513700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed005f490 0x7faed00674c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:58.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faed6513700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed005f490 0x7faed00674c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35876/0 (socket says 172.21.15.32:35876) 2020-12-05T17:36:58.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faed6513700 1 -- 172.21.15.32:0/2345674308 learned_addr learned my addr 172.21.15.32:0/2345674308 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:58.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faed6d14700 1 --2- 172.21.15.32:0/2345674308 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faed00aead0 0x7faed0135e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:58.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faed5d12700 1 --2- 172.21.15.32:0/2345674308 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faed00ad1b0 0x7faed0135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:58.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faed6513700 1 -- 172.21.15.32:0/2345674308 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faed00aead0 msgr2=0x7faed0135e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faed6513700 1 --2- 172.21.15.32:0/2345674308 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faed00aead0 0x7faed0135e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faed6513700 1 -- 172.21.15.32:0/2345674308 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faed00ad1b0 msgr2=0x7faed0135920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faed6513700 1 --2- 172.21.15.32:0/2345674308 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faed00ad1b0 0x7faed0135920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faed6513700 1 -- 172.21.15.32:0/2345674308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed0136300 con 0x7faed005f490 2020-12-05T17:36:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faed6d14700 1 --2- 172.21.15.32:0/2345674308 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faed00aead0 0x7faed0135e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:36:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faed5d12700 1 --2- 172.21.15.32:0/2345674308 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faed00ad1b0 0x7faed0135920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:36:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faed6513700 1 --2- 172.21.15.32:0/2345674308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed005f490 0x7faed00674c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7faecc000fa0 tx=0x7faecc0011f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faebf7fe700 1 -- 172.21.15.32:0/2345674308 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faecc007170 con 0x7faed005f490 2020-12-05T17:36:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faebf7fe700 1 -- 172.21.15.32:0/2345674308 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faecc0072d0 con 0x7faed005f490 2020-12-05T17:36:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faed8777700 1 -- 172.21.15.32:0/2345674308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed00b2160 con 0x7faed005f490 2020-12-05T17:36:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faebf7fe700 1 -- 172.21.15.32:0/2345674308 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faecc0048f0 con 0x7faed005f490 2020-12-05T17:36:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faed8777700 1 -- 172.21.15.32:0/2345674308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed01394b0 con 0x7faed005f490 2020-12-05T17:36:58.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.559+0000 7faed8777700 1 -- 172.21.15.32:0/2345674308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed013adf0 con 0x7faed005f490 2020-12-05T17:36:58.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.563+0000 7faebf7fe700 1 -- 172.21.15.32:0/2345674308 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7faecc010070 con 0x7faed005f490 2020-12-05T17:36:58.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.563+0000 7faebf7fe700 1 --2- 172.21.15.32:0/2345674308 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faeb0055440 0x7faeb0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:58.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.563+0000 7faed5d12700 1 --2- 172.21.15.32:0/2345674308 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faeb0055440 0x7faeb0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.563+0000 7faebf7fe700 1 -- 172.21.15.32:0/2345674308 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5600+0+0 (secure 0 0 0) 0x7faecc06d660 con 0x7faed005f490 2020-12-05T17:36:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.563+0000 7faed5d12700 1 --2- 172.21.15.32:0/2345674308 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faeb0055440 0x7faeb0057900 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7faec0018000 tx=0x7faec0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.571+0000 7faebf7fe700 1 -- 172.21.15.32:0/2345674308 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7faecc043aa0 con 0x7faed005f490 2020-12-05T17:36:58.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.711+0000 7faed8777700 1 -- 172.21.15.32:0/2345674308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faed0057050 con 0x7faed005f490 2020-12-05T17:36:58.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.715+0000 7faebf7fe700 1 -- 172.21.15.32:0/2345674308 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+13415 (secure 0 0 0) 0x7faecc019030 con 0x7faed005f490 2020-12-05T17:36:58.715 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:36:58.715 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":58,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:36:58.015636+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:36:58.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.715+0000 7faed8777700 1 -- 172.21.15.32:0/2345674308 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faeb0055440 msgr2=0x7faeb0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:58.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.715+0000 7faed8777700 1 --2- 172.21.15.32:0/2345674308 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faeb0055440 0x7faeb0057900 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7faec0018000 tx=0x7faec0015000).stop 2020-12-05T17:36:58.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.715+0000 7faed8777700 1 -- 172.21.15.32:0/2345674308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed005f490 msgr2=0x7faed00674c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:58.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.715+0000 7faed8777700 1 --2- 172.21.15.32:0/2345674308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed005f490 0x7faed00674c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7faecc000fa0 tx=0x7faecc0011f0).stop 2020-12-05T17:36:58.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.715+0000 7faed8777700 1 -- 172.21.15.32:0/2345674308 shutdown_connections 2020-12-05T17:36:58.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.715+0000 7faed8777700 1 --2- 172.21.15.32:0/2345674308 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faeb0055440 0x7faeb0057900 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.715+0000 7faed8777700 1 --2- 172.21.15.32:0/2345674308 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faed00aead0 0x7faed0135e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.715+0000 7faed8777700 1 --2- 172.21.15.32:0/2345674308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed005f490 0x7faed00674c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.715+0000 7faed8777700 1 --2- 172.21.15.32:0/2345674308 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faed00ad1b0 0x7faed0135920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.715+0000 7faed8777700 1 -- 172.21.15.32:0/2345674308 >> 172.21.15.32:0/2345674308 conn(0x7faed00b5580 msgr2=0x7faed01365d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:58.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.715+0000 7faed8777700 1 -- 172.21.15.32:0/2345674308 shutdown_connections 2020-12-05T17:36:58.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.715+0000 7faed8777700 1 -- 172.21.15.32:0/2345674308 wait complete. 2020-12-05T17:36:58.730 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-12-05T17:36:58.731 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:36:58.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.851+0000 7f1de57a8700 1 Processor -- start 2020-12-05T17:36:58.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.851+0000 7f1de57a8700 1 -- start start 2020-12-05T17:36:58.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.851+0000 7f1de57a8700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1de00b3020 0x7f1de0066e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:58.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.851+0000 7f1de57a8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1de0144380 0x7f1de013a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:58.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.851+0000 7f1de57a8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1de00b3540 con 0x7f1de0061ba0 2020-12-05T17:36:58.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.851+0000 7f1de57a8700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1de0061f80 con 0x7f1de00b3020 2020-12-05T17:36:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.851+0000 7f1de57a8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de0066750 con 0x7f1de0144380 2020-12-05T17:36:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.851+0000 7f1ddffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1de0061ba0 0x7f1de0143c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38532/0 (socket says 172.21.15.32:38532) 2020-12-05T17:36:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.851+0000 7f1ddffff700 1 -- 172.21.15.32:0/2697149288 learned_addr learned my addr 172.21.15.32:0/2697149288 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:36:58.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.851+0000 7f1de4fa7700 1 --2- 172.21.15.32:0/2697149288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1de0144380 0x7f1de013a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:58.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.851+0000 7f1ddf7fe700 1 --2- 172.21.15.32:0/2697149288 >> v2:172.21.15.61:3300/0 conn(0x7f1de00b3020 0x7f1de0066e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:58.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.851+0000 7f1ddeffd700 1 -- 172.21.15.32:0/2697149288 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3042301849 0 0) 0x7f1de00b3540 con 0x7f1de0061ba0 2020-12-05T17:36:58.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.851+0000 7f1ddeffd700 1 -- 172.21.15.32:0/2697149288 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dc0003230 con 0x7f1de0061ba0 2020-12-05T17:36:58.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.851+0000 7f1ddeffd700 1 -- 172.21.15.32:0/2697149288 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4003178357 0 0) 0x7f1dc0003230 con 0x7f1de0061ba0 2020-12-05T17:36:58.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.851+0000 7f1ddeffd700 1 -- 172.21.15.32:0/2697149288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1de0144380 msgr2=0x7f1de013a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:36:58.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.851+0000 7f1ddeffd700 1 --2- 172.21.15.32:0/2697149288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1de0144380 0x7f1de013a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.851+0000 7f1ddeffd700 1 -- 172.21.15.32:0/2697149288 >> v2:172.21.15.61:3300/0 conn(0x7f1de00b3020 msgr2=0x7f1de0066e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:36:58.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1ddeffd700 1 --2- 172.21.15.32:0/2697149288 >> v2:172.21.15.61:3300/0 conn(0x7f1de00b3020 0x7f1de0066e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1ddeffd700 1 -- 172.21.15.32:0/2697149288 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de00669f0 con 0x7f1de0061ba0 2020-12-05T17:36:58.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1ddeffd700 1 -- 172.21.15.32:0/2697149288 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f1dd80016d0 con 0x7f1de0061ba0 2020-12-05T17:36:58.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1ddeffd700 1 -- 172.21.15.32:0/2697149288 >> v1:172.21.15.32:6789/0 conn(0x7f1de0061ba0 legacy=0x7f1de0143c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:58.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1ddeffd700 1 --2- 172.21.15.32:0/2697149288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1dc0003720 0x7f1dc0003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:58.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1ddeffd700 1 --2- 172.21.15.32:0/2697149288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1de0144380 0x7f1dc00044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:58.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1ddeffd700 1 --2- 172.21.15.32:0/2697149288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dc00049c0 0x7f1dc0004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:58.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1ddeffd700 1 -- 172.21.15.32:0/2697149288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de0061f80 con 0x7f1dc00049c0 2020-12-05T17:36:58.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1ddeffd700 1 -- 172.21.15.32:0/2697149288 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de0066750 con 0x7f1dc0003720 2020-12-05T17:36:58.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1ddeffd700 1 -- 172.21.15.32:0/2697149288 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc00041c0 con 0x7f1de0144380 2020-12-05T17:36:58.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1de4fa7700 1 --2- 172.21.15.32:0/2697149288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dc00049c0 0x7f1dc0004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:58.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1ddffff700 1 --2- 172.21.15.32:0/2697149288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1dc0003720 0x7f1dc0003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:58.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1ddf7fe700 1 --2- 172.21.15.32:0/2697149288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1de0144380 0x7f1dc00044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:58.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1de4fa7700 1 -- 172.21.15.32:0/2697149288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1de0144380 msgr2=0x7f1dc00044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:36:58.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1de4fa7700 1 --2- 172.21.15.32:0/2697149288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1de0144380 0x7f1dc00044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1de4fa7700 1 -- 172.21.15.32:0/2697149288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1dc0003720 msgr2=0x7f1dc0003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:36:58.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1de4fa7700 1 --2- 172.21.15.32:0/2697149288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1dc0003720 0x7f1dc0003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1de4fa7700 1 -- 172.21.15.32:0/2697149288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc0003230 con 0x7f1dc00049c0 2020-12-05T17:36:58.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1de4fa7700 1 --2- 172.21.15.32:0/2697149288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dc00049c0 0x7f1dc0004e40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f1dd000c000 tx=0x7f1dd0008040).ready entity=mon.0 client_cookie=63c6a17ffc10dd97 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:58.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1ddeffd700 1 -- 172.21.15.32:0/2697149288 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1dd000e970 con 0x7f1dc00049c0 2020-12-05T17:36:58.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1ddeffd700 1 -- 172.21.15.32:0/2697149288 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd000ead0 con 0x7f1dc00049c0 2020-12-05T17:36:58.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1ddeffd700 1 -- 172.21.15.32:0/2697149288 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1dd000ee30 con 0x7f1dc00049c0 2020-12-05T17:36:58.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1de57a8700 1 -- 172.21.15.32:0/2697149288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dc00049c0 msgr2=0x7f1dc0004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:58.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1de57a8700 1 --2- 172.21.15.32:0/2697149288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dc00049c0 0x7f1dc0004e40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f1dd000c000 tx=0x7f1dd0008040).stop 2020-12-05T17:36:58.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1de4fa7700 1 -- 172.21.15.32:0/2697149288 reap_dead start 2020-12-05T17:36:58.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1de57a8700 1 -- 172.21.15.32:0/2697149288 shutdown_connections 2020-12-05T17:36:58.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1de57a8700 1 -- 172.21.15.32:0/2697149288 >> 172.21.15.32:0/2697149288 conn(0x7f1de00b6d10 msgr2=0x7f1de00b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:58.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1de57a8700 1 -- 172.21.15.32:0/2697149288 shutdown_connections 2020-12-05T17:36:58.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.855+0000 7f1de57a8700 1 -- 172.21.15.32:0/2697149288 wait complete. 2020-12-05T17:36:58.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1de57a8700 1 Processor -- start 2020-12-05T17:36:58.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1de57a8700 1 -- start start 2020-12-05T17:36:58.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1de57a8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1de0113660 0x7f1de0113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:58.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1de57a8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1de00b0e20 0x7f1de013d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:58.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1de57a8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1de013dc50 0x7f1de014f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:58.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1de57a8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de0112750 con 0x7f1de0113660 2020-12-05T17:36:58.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1de57a8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de013e0d0 con 0x7f1de013dc50 2020-12-05T17:36:58.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1de57a8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de00b05d0 con 0x7f1de00b0e20 2020-12-05T17:36:58.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1ddffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1de0113660 0x7f1de0113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:58.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1ddffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1de0113660 0x7f1de0113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35896/0 (socket says 172.21.15.32:35896) 2020-12-05T17:36:58.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1ddffff700 1 -- 172.21.15.32:0/3429999229 learned_addr learned my addr 172.21.15.32:0/3429999229 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:58.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1de4fa7700 1 --2- 172.21.15.32:0/3429999229 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1de013dc50 0x7f1de014f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:58.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1ddf7fe700 1 --2- 172.21.15.32:0/3429999229 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1de00b0e20 0x7f1de013d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:58.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1ddffff700 1 -- 172.21.15.32:0/3429999229 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1de00b0e20 msgr2=0x7f1de013d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:58.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1ddffff700 1 --2- 172.21.15.32:0/3429999229 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1de00b0e20 0x7f1de013d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1ddffff700 1 -- 172.21.15.32:0/3429999229 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1de013dc50 msgr2=0x7f1de014f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:58.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1ddffff700 1 --2- 172.21.15.32:0/3429999229 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1de013dc50 0x7f1de014f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:58.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1ddffff700 1 -- 172.21.15.32:0/3429999229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de0111af0 con 0x7f1de0113660 2020-12-05T17:36:58.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1ddf7fe700 1 --2- 172.21.15.32:0/3429999229 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1de00b0e20 0x7f1de013d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:36:58.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1ddffff700 1 --2- 172.21.15.32:0/3429999229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1de0113660 0x7f1de0113a40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f1de0110280 tx=0x7f1de00664b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:58.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1ddd7fa700 1 -- 172.21.15.32:0/3429999229 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1dd800a990 con 0x7f1de0113660 2020-12-05T17:36:58.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1ddd7fa700 1 -- 172.21.15.32:0/3429999229 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd8010040 con 0x7f1de0113660 2020-12-05T17:36:58.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1ddd7fa700 1 -- 172.21.15.32:0/3429999229 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1dd800ace0 con 0x7f1de0113660 2020-12-05T17:36:58.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1de57a8700 1 -- 172.21.15.32:0/3429999229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de0111d90 con 0x7f1de0113660 2020-12-05T17:36:58.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.859+0000 7f1de57a8700 1 -- 172.21.15.32:0/3429999229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de0113fb0 con 0x7f1de0113660 2020-12-05T17:36:58.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.863+0000 7f1de57a8700 1 -- 172.21.15.32:0/3429999229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de0057050 con 0x7f1de0113660 2020-12-05T17:36:58.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.863+0000 7f1ddd7fa700 1 -- 172.21.15.32:0/3429999229 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1dd801e430 con 0x7f1de0113660 2020-12-05T17:36:58.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.863+0000 7f1ddd7fa700 1 --2- 172.21.15.32:0/3429999229 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1dc4055470 0x7f1dc4057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:58.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.863+0000 7f1ddf7fe700 1 --2- 172.21.15.32:0/3429999229 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1dc4055470 0x7f1dc4057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:58.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.863+0000 7f1ddd7fa700 1 -- 172.21.15.32:0/3429999229 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f1dd806da50 con 0x7f1de0113660 2020-12-05T17:36:58.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.863+0000 7f1ddf7fe700 1 --2- 172.21.15.32:0/3429999229 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1dc4055470 0x7f1dc4057930 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f1dc800b000 tx=0x7f1dc8008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:58.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:58.871+0000 7f1ddd7fa700 1 -- 172.21.15.32:0/3429999229 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1dd8043ed0 con 0x7f1de0113660 2020-12-05T17:36:59.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.011+0000 7f1de57a8700 1 -- 172.21.15.32:0/3429999229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f1de00b4630 con 0x7f1de0113660 2020-12-05T17:36:59.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.015+0000 7f1ddd7fa700 1 -- 172.21.15.32:0/3429999229 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v58) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1dd80474f0 con 0x7f1de0113660 2020-12-05T17:36:59.015 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:36:59.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.015+0000 7f1de57a8700 1 -- 172.21.15.32:0/3429999229 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1dc4055470 msgr2=0x7f1dc4057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:59.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.015+0000 7f1de57a8700 1 --2- 172.21.15.32:0/3429999229 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1dc4055470 0x7f1dc4057930 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f1dc800b000 tx=0x7f1dc8008040).stop 2020-12-05T17:36:59.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.015+0000 7f1de57a8700 1 -- 172.21.15.32:0/3429999229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1de0113660 msgr2=0x7f1de0113a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:59.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.015+0000 7f1de57a8700 1 --2- 172.21.15.32:0/3429999229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1de0113660 0x7f1de0113a40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f1de0110280 tx=0x7f1de00664b0).stop 2020-12-05T17:36:59.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.015+0000 7f1de57a8700 1 -- 172.21.15.32:0/3429999229 shutdown_connections 2020-12-05T17:36:59.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.015+0000 7f1de57a8700 1 --2- 172.21.15.32:0/3429999229 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1de013dc50 0x7f1de014f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.015+0000 7f1de57a8700 1 --2- 172.21.15.32:0/3429999229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1de0113660 0x7f1de0113a40 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.015+0000 7f1de57a8700 1 --2- 172.21.15.32:0/3429999229 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1dc4055470 0x7f1dc4057930 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.015+0000 7f1de57a8700 1 --2- 172.21.15.32:0/3429999229 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1de00b0e20 0x7f1de013d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.015+0000 7f1de57a8700 1 -- 172.21.15.32:0/3429999229 >> 172.21.15.32:0/3429999229 conn(0x7f1de00b6d10 msgr2=0x7f1de013e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:59.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.015+0000 7f1de57a8700 1 -- 172.21.15.32:0/3429999229 shutdown_connections 2020-12-05T17:36:59.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.015+0000 7f1de57a8700 1 -- 172.21.15.32:0/3429999229 wait complete. 2020-12-05T17:36:59.029 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:36:59.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e9b3e700 1 Processor -- start 2020-12-05T17:36:59.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e9b3e700 1 -- start start 2020-12-05T17:36:59.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e9b3e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85e405feb0 0x7f85e405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:59.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e9b3e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85e40b2fb0 0x7f85e4061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:59.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e9b3e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e40b2df0 con 0x7f85e405f3b0 2020-12-05T17:36:59.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e9b3e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e40b3c90 con 0x7f85e405feb0 2020-12-05T17:36:59.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e9b3e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e40b2b50 con 0x7f85e40b2fb0 2020-12-05T17:36:59.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85e405feb0 0x7f85e405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:59.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85e405feb0 0x7f85e405f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35904/0 (socket says 172.21.15.32:35904) 2020-12-05T17:36:59.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e3fff700 1 -- 172.21.15.32:0/1910496534 learned_addr learned my addr 172.21.15.32:0/1910496534 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:59.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e8b3c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f85e405f3b0 0x7f85e413c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38552/0 (socket says 172.21.15.32:38552) 2020-12-05T17:36:59.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e933d700 1 --2- 172.21.15.32:0/1910496534 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85e40b2fb0 0x7f85e4061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:59.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e3fff700 1 -- 172.21.15.32:0/1910496534 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85e40b2fb0 msgr2=0x7f85e4061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:36:59.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e3fff700 1 --2- 172.21.15.32:0/1910496534 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85e40b2fb0 0x7f85e4061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e37fe700 1 -- 172.21.15.32:0/1910496534 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 971077104 0 0) 0x7f85e40b2df0 con 0x7f85e405f3b0 2020-12-05T17:36:59.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e3fff700 1 -- 172.21.15.32:0/1910496534 >> v1:172.21.15.32:6789/0 conn(0x7f85e405f3b0 legacy=0x7f85e413c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:59.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e3fff700 1 -- 172.21.15.32:0/1910496534 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e4142820 con 0x7f85e405feb0 2020-12-05T17:36:59.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e933d700 1 --2- 172.21.15.32:0/1910496534 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85e40b2fb0 0x7f85e4061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:36:59.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e3fff700 1 --2- 172.21.15.32:0/1910496534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e405feb0 0x7f85e405f790 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f85cc0032b0 tx=0x7f85cc009270).ready entity=mon.0 client_cookie=ac19a7065fe5591a server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:59.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e37fe700 1 -- 172.21.15.32:0/1910496534 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85cc00bc00 con 0x7f85e405feb0 2020-12-05T17:36:59.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e37fe700 1 -- 172.21.15.32:0/1910496534 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85cc00bd60 con 0x7f85e405feb0 2020-12-05T17:36:59.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e37fe700 1 -- 172.21.15.32:0/1910496534 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85cc011400 con 0x7f85e405feb0 2020-12-05T17:36:59.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e9b3e700 1 -- 172.21.15.32:0/1910496534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e405feb0 msgr2=0x7f85e405f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:59.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e9b3e700 1 --2- 172.21.15.32:0/1910496534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e405feb0 0x7f85e405f790 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f85cc0032b0 tx=0x7f85cc009270).stop 2020-12-05T17:36:59.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e9b3e700 1 -- 172.21.15.32:0/1910496534 shutdown_connections 2020-12-05T17:36:59.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e9b3e700 1 --2- 172.21.15.32:0/1910496534 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85e40b2fb0 0x7f85e4061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e9b3e700 1 --2- 172.21.15.32:0/1910496534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e405feb0 0x7f85e405f790 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.143+0000 7f85e9b3e700 1 -- 172.21.15.32:0/1910496534 >> 172.21.15.32:0/1910496534 conn(0x7f85e40b8f20 msgr2=0x7f85e40b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:59.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e9b3e700 1 -- 172.21.15.32:0/1910496534 shutdown_connections 2020-12-05T17:36:59.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e9b3e700 1 -- 172.21.15.32:0/1910496534 wait complete. 2020-12-05T17:36:59.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e9b3e700 1 Processor -- start 2020-12-05T17:36:59.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e9b3e700 1 -- start start 2020-12-05T17:36:59.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e9b3e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85e405f3b0 0x7f85e4143f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:59.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e9b3e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e405feb0 0x7f85e4144430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:59.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e9b3e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85e40b2fb0 0x7f85e4148f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:59.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e9b3e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e4145250 con 0x7f85e405feb0 2020-12-05T17:36:59.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e9b3e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e4144920 con 0x7f85e40b2fb0 2020-12-05T17:36:59.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e9b3e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e413c450 con 0x7f85e405f3b0 2020-12-05T17:36:59.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e405feb0 0x7f85e4144430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:59.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e405feb0 0x7f85e4144430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35914/0 (socket says 172.21.15.32:35914) 2020-12-05T17:36:59.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e3fff700 1 -- 172.21.15.32:0/1319314333 learned_addr learned my addr 172.21.15.32:0/1319314333 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:59.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e933d700 1 --2- 172.21.15.32:0/1319314333 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85e40b2fb0 0x7f85e4148f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:59.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e8b3c700 1 --2- 172.21.15.32:0/1319314333 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85e405f3b0 0x7f85e4143f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:59.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e3fff700 1 -- 172.21.15.32:0/1319314333 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85e405f3b0 msgr2=0x7f85e4143f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:59.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e3fff700 1 --2- 172.21.15.32:0/1319314333 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85e405f3b0 0x7f85e4143f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e3fff700 1 -- 172.21.15.32:0/1319314333 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85e40b2fb0 msgr2=0x7f85e4148f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:59.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e3fff700 1 --2- 172.21.15.32:0/1319314333 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85e40b2fb0 0x7f85e4148f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e3fff700 1 -- 172.21.15.32:0/1319314333 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e4146830 con 0x7f85e405feb0 2020-12-05T17:36:59.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e3fff700 1 --2- 172.21.15.32:0/1319314333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e405feb0 0x7f85e4144430 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f85cc0032b0 tx=0x7f85cc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:59.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e1ffb700 1 -- 172.21.15.32:0/1319314333 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85cc0107a0 con 0x7f85e405feb0 2020-12-05T17:36:59.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e1ffb700 1 -- 172.21.15.32:0/1319314333 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85cc00bed0 con 0x7f85e405feb0 2020-12-05T17:36:59.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e9b3e700 1 -- 172.21.15.32:0/1319314333 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e4146b30 con 0x7f85e405feb0 2020-12-05T17:36:59.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e1ffb700 1 -- 172.21.15.32:0/1319314333 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85cc010af0 con 0x7f85e405feb0 2020-12-05T17:36:59.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.147+0000 7f85e9b3e700 1 -- 172.21.15.32:0/1319314333 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e41422a0 con 0x7f85e405feb0 2020-12-05T17:36:59.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.151+0000 7f85e9b3e700 1 -- 172.21.15.32:0/1319314333 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e4057050 con 0x7f85e405feb0 2020-12-05T17:36:59.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.151+0000 7f85e1ffb700 1 -- 172.21.15.32:0/1319314333 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f85cc00f070 con 0x7f85e405feb0 2020-12-05T17:36:59.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.151+0000 7f85e1ffb700 1 --2- 172.21.15.32:0/1319314333 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f85bc0553b0 0x7f85bc057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:59.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.151+0000 7f85e8b3c700 1 --2- 172.21.15.32:0/1319314333 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f85bc0553b0 0x7f85bc057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:59.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.151+0000 7f85e1ffb700 1 -- 172.21.15.32:0/1319314333 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f85cc06f2f0 con 0x7f85e405feb0 2020-12-05T17:36:59.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.151+0000 7f85e8b3c700 1 --2- 172.21.15.32:0/1319314333 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f85bc0553b0 0x7f85bc057870 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f85d400d000 tx=0x7f85d4007a60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:59.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.159+0000 7f85e1ffb700 1 -- 172.21.15.32:0/1319314333 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f85cc049160 con 0x7f85e405feb0 2020-12-05T17:36:59.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.307+0000 7f85e9b3e700 1 -- 172.21.15.32:0/1319314333 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f85e405aab0 con 0x7f85e405feb0 2020-12-05T17:36:59.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.307+0000 7f85e1ffb700 1 -- 172.21.15.32:0/1319314333 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v58) v1 ==== 92+0+10 (secure 0 0 0) 0x7f85cc0150a0 con 0x7f85e405feb0 2020-12-05T17:36:59.310 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:36:59.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.311+0000 7f85e9b3e700 1 -- 172.21.15.32:0/1319314333 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f85bc0553b0 msgr2=0x7f85bc057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:59.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.311+0000 7f85e9b3e700 1 --2- 172.21.15.32:0/1319314333 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f85bc0553b0 0x7f85bc057870 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f85d400d000 tx=0x7f85d4007a60).stop 2020-12-05T17:36:59.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.311+0000 7f85e9b3e700 1 -- 172.21.15.32:0/1319314333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e405feb0 msgr2=0x7f85e4144430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:59.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.311+0000 7f85e9b3e700 1 --2- 172.21.15.32:0/1319314333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e405feb0 0x7f85e4144430 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f85cc0032b0 tx=0x7f85cc016000).stop 2020-12-05T17:36:59.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.311+0000 7f85e9b3e700 1 -- 172.21.15.32:0/1319314333 shutdown_connections 2020-12-05T17:36:59.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.311+0000 7f85e9b3e700 1 --2- 172.21.15.32:0/1319314333 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85e40b2fb0 0x7f85e4148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.311+0000 7f85e9b3e700 1 --2- 172.21.15.32:0/1319314333 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f85bc0553b0 0x7f85bc057870 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.311+0000 7f85e9b3e700 1 --2- 172.21.15.32:0/1319314333 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85e405f3b0 0x7f85e4143f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.311+0000 7f85e9b3e700 1 --2- 172.21.15.32:0/1319314333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e405feb0 0x7f85e4144430 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.311+0000 7f85e9b3e700 1 -- 172.21.15.32:0/1319314333 >> 172.21.15.32:0/1319314333 conn(0x7f85e40b8f20 msgr2=0x7f85e40b7830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:59.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.311+0000 7f85e9b3e700 1 -- 172.21.15.32:0/1319314333 shutdown_connections 2020-12-05T17:36:59.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.311+0000 7f85e9b3e700 1 -- 172.21.15.32:0/1319314333 wait complete. 2020-12-05T17:36:59.324 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-12-05T17:36:59.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.435+0000 7f3cb68da700 1 Processor -- start 2020-12-05T17:36:59.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.435+0000 7f3cb68da700 1 -- start start 2020-12-05T17:36:59.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.435+0000 7f3cb68da700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3cb005f6f0 0x7f3cb01428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:59.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.435+0000 7f3cb68da700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3cb00b1980 0x7f3cb0146580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:59.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.435+0000 7f3cb68da700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb005fad0 con 0x7f3cb0146a70 2020-12-05T17:36:59.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.435+0000 7f3cb68da700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb00b2f80 con 0x7f3cb005f6f0 2020-12-05T17:36:59.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.435+0000 7f3cb68da700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb0145e40 con 0x7f3cb00b1980 2020-12-05T17:36:59.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.435+0000 7f3cb58d8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3cb005f6f0 0x7f3cb01428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:59.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.435+0000 7f3cb58d8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3cb005f6f0 0x7f3cb01428f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35918/0 (socket says 172.21.15.32:35918) 2020-12-05T17:36:59.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.435+0000 7f3cb58d8700 1 -- 172.21.15.32:0/3349425388 learned_addr learned my addr 172.21.15.32:0/3349425388 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:59.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.435+0000 7f3cb50d7700 1 --2- 172.21.15.32:0/3349425388 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3cb00b1980 0x7f3cb0146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.435+0000 7f3cb58d8700 1 -- 172.21.15.32:0/3349425388 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3cb00b1980 msgr2=0x7f3cb0146580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:36:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.435+0000 7f3cb58d8700 1 --2- 172.21.15.32:0/3349425388 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3cb00b1980 0x7f3cb0146580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.435+0000 7f3cb58d8700 1 -- 172.21.15.32:0/3349425388 >> v1:172.21.15.61:6789/0 conn(0x7f3cb0146a70 legacy=0x7f3cb0066070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:36:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.435+0000 7f3cb58d8700 1 -- 172.21.15.32:0/3349425388 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb01460e0 con 0x7f3cb005f6f0 2020-12-05T17:36:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.435+0000 7f3cb50d7700 1 --2- 172.21.15.32:0/3349425388 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3cb00b1980 0x7f3cb0146580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:36:59.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.435+0000 7f3cb58d8700 1 --2- 172.21.15.32:0/3349425388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb005f6f0 0x7f3cb01428f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f3cb006a370 tx=0x7f3cac009270).ready entity=mon.0 client_cookie=8204b6987ecf227e server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:59.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.435+0000 7f3cb48d6700 1 -- 172.21.15.32:0/3349425388 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cac00bbc0 con 0x7f3cb005f6f0 2020-12-05T17:36:59.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.435+0000 7f3cb48d6700 1 -- 172.21.15.32:0/3349425388 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cac00bd20 con 0x7f3cb005f6f0 2020-12-05T17:36:59.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.435+0000 7f3cb48d6700 1 -- 172.21.15.32:0/3349425388 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cac011400 con 0x7f3cb005f6f0 2020-12-05T17:36:59.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.435+0000 7f3cb68da700 1 -- 172.21.15.32:0/3349425388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb005f6f0 msgr2=0x7f3cb01428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:59.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.435+0000 7f3cb68da700 1 --2- 172.21.15.32:0/3349425388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb005f6f0 0x7f3cb01428f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f3cb006a370 tx=0x7f3cac009270).stop 2020-12-05T17:36:59.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.439+0000 7f3cb68da700 1 -- 172.21.15.32:0/3349425388 shutdown_connections 2020-12-05T17:36:59.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.439+0000 7f3cb68da700 1 --2- 172.21.15.32:0/3349425388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb005f6f0 0x7f3cb01428f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.439+0000 7f3cb68da700 1 --2- 172.21.15.32:0/3349425388 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3cb00b1980 0x7f3cb0146580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.439+0000 7f3cb68da700 1 -- 172.21.15.32:0/3349425388 >> 172.21.15.32:0/3349425388 conn(0x7f3cb00604c0 msgr2=0x7f3cb005c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:59.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.439+0000 7f3cb68da700 1 -- 172.21.15.32:0/3349425388 shutdown_connections 2020-12-05T17:36:59.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.439+0000 7f3cb68da700 1 -- 172.21.15.32:0/3349425388 wait complete. 2020-12-05T17:36:59.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.439+0000 7f3cb68da700 1 Processor -- start 2020-12-05T17:36:59.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.439+0000 7f3cb68da700 1 -- start start 2020-12-05T17:36:59.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.439+0000 7f3cb68da700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3cb005f6f0 0x7f3cb0145010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:59.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.439+0000 7f3cb68da700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3cb00b1980 0x7f3cb013e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:59.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.439+0000 7f3cb68da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb0146a70 0x7f3cb013edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:59.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.439+0000 7f3cb68da700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb0065aa0 con 0x7f3cb0146a70 2020-12-05T17:36:59.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.439+0000 7f3cb68da700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb0064ca0 con 0x7f3cb00b1980 2020-12-05T17:36:59.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.439+0000 7f3cb68da700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb0065cd0 con 0x7f3cb005f6f0 2020-12-05T17:36:59.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.439+0000 7f3cb60d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb0146a70 0x7f3cb013edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:59.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.439+0000 7f3cb60d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb0146a70 0x7f3cb013edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35926/0 (socket says 172.21.15.32:35926) 2020-12-05T17:36:59.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.439+0000 7f3cb60d9700 1 -- 172.21.15.32:0/3056130378 learned_addr learned my addr 172.21.15.32:0/3056130378 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:59.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.439+0000 7f3cb50d7700 1 --2- 172.21.15.32:0/3056130378 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3cb00b1980 0x7f3cb013e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:59.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.439+0000 7f3cb58d8700 1 --2- 172.21.15.32:0/3056130378 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3cb005f6f0 0x7f3cb0145010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:59.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.439+0000 7f3cb60d9700 1 -- 172.21.15.32:0/3056130378 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3cb005f6f0 msgr2=0x7f3cb0145010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:59.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.439+0000 7f3cb60d9700 1 --2- 172.21.15.32:0/3056130378 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3cb005f6f0 0x7f3cb0145010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.443+0000 7f3cb60d9700 1 -- 172.21.15.32:0/3056130378 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3cb00b1980 msgr2=0x7f3cb013e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:59.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.443+0000 7f3cb60d9700 1 --2- 172.21.15.32:0/3056130378 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3cb00b1980 0x7f3cb013e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.443+0000 7f3cb60d9700 1 -- 172.21.15.32:0/3056130378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb013f2a0 con 0x7f3cb0146a70 2020-12-05T17:36:59.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.443+0000 7f3cb58d8700 1 --2- 172.21.15.32:0/3056130378 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3cb005f6f0 0x7f3cb0145010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:36:59.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.443+0000 7f3cb60d9700 1 --2- 172.21.15.32:0/3056130378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb0146a70 0x7f3cb013edb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f3ca8002ef0 tx=0x7f3ca8006cc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:59.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.443+0000 7f3c9effd700 1 -- 172.21.15.32:0/3056130378 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ca8002320 con 0x7f3cb0146a70 2020-12-05T17:36:59.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.443+0000 7f3c9effd700 1 -- 172.21.15.32:0/3056130378 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca8000e90 con 0x7f3cb0146a70 2020-12-05T17:36:59.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.443+0000 7f3cb68da700 1 -- 172.21.15.32:0/3056130378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb0065050 con 0x7f3cb0146a70 2020-12-05T17:36:59.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.443+0000 7f3c9effd700 1 -- 172.21.15.32:0/3056130378 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ca8002670 con 0x7f3cb0146a70 2020-12-05T17:36:59.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.443+0000 7f3cb68da700 1 -- 172.21.15.32:0/3056130378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb0154cb0 con 0x7f3cb0146a70 2020-12-05T17:36:59.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.443+0000 7f3cb68da700 1 -- 172.21.15.32:0/3056130378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb0064b00 con 0x7f3cb0146a70 2020-12-05T17:36:59.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.443+0000 7f3c9effd700 1 -- 172.21.15.32:0/3056130378 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3ca800f030 con 0x7f3cb0146a70 2020-12-05T17:36:59.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.443+0000 7f3c9effd700 1 --2- 172.21.15.32:0/3056130378 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3c900553b0 0x7f3c90057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:59.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.443+0000 7f3c9effd700 1 -- 172.21.15.32:0/3056130378 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f3ca806d940 con 0x7f3cb0146a70 2020-12-05T17:36:59.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.443+0000 7f3cb58d8700 1 --2- 172.21.15.32:0/3056130378 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3c900553b0 0x7f3c90057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:59.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.447+0000 7f3cb58d8700 1 --2- 172.21.15.32:0/3056130378 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3c900553b0 0x7f3c90057870 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f3cac018000 tx=0x7f3cac016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:59.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.451+0000 7f3c9effd700 1 -- 172.21.15.32:0/3056130378 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3cb0064b00 con 0x7f3cb0146a70 2020-12-05T17:36:59.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.591+0000 7f3cb68da700 1 -- 172.21.15.32:0/3056130378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3cb0064b00 con 0x7f3cb0146a70 2020-12-05T17:36:59.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.591+0000 7f3c9effd700 1 -- 172.21.15.32:0/3056130378 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v58) v1 ==== 84+0+143 (secure 0 0 0) 0x7f3cb0064b00 con 0x7f3cb0146a70 2020-12-05T17:36:59.593 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:36:59.593 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2020-12-05T17:36:59.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:59.594 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2020-12-05T17:36:59.594 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2020-12-05T17:36:59.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:36:59.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:36:59.595 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2020-12-05T17:36:59.595 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2020-12-05T17:36:59.595 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:36:59.596 INFO:teuthology.orchestra.run.smithi032.stdout:] 2020-12-05T17:36:59.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.591+0000 7f3cb68da700 1 -- 172.21.15.32:0/3056130378 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3c900553b0 msgr2=0x7f3c90057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:59.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.591+0000 7f3cb68da700 1 --2- 172.21.15.32:0/3056130378 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3c900553b0 0x7f3c90057870 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f3cac018000 tx=0x7f3cac016000).stop 2020-12-05T17:36:59.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.591+0000 7f3cb68da700 1 -- 172.21.15.32:0/3056130378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb0146a70 msgr2=0x7f3cb013edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:59.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.591+0000 7f3cb68da700 1 --2- 172.21.15.32:0/3056130378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb0146a70 0x7f3cb013edb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f3ca8002ef0 tx=0x7f3ca8006cc0).stop 2020-12-05T17:36:59.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.595+0000 7f3cb68da700 1 -- 172.21.15.32:0/3056130378 shutdown_connections 2020-12-05T17:36:59.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.595+0000 7f3cb68da700 1 --2- 172.21.15.32:0/3056130378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb0146a70 0x7f3cb013edb0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.595+0000 7f3cb68da700 1 --2- 172.21.15.32:0/3056130378 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3c900553b0 0x7f3c90057870 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.595+0000 7f3cb68da700 1 --2- 172.21.15.32:0/3056130378 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3cb005f6f0 0x7f3cb0145010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.595+0000 7f3cb68da700 1 --2- 172.21.15.32:0/3056130378 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3cb00b1980 0x7f3cb013e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.595+0000 7f3cb68da700 1 -- 172.21.15.32:0/3056130378 >> 172.21.15.32:0/3056130378 conn(0x7f3cb00604c0 msgr2=0x7f3cb0061010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:59.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.595+0000 7f3cb68da700 1 -- 172.21.15.32:0/3056130378 shutdown_connections 2020-12-05T17:36:59.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.595+0000 7f3cb68da700 1 -- 172.21.15.32:0/3056130378 wait complete. 2020-12-05T17:36:59.614 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-12-05T17:36:59.614 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-12-05T17:36:59.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.735+0000 7f2b7cbd7700 1 Processor -- start 2020-12-05T17:36:59.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.735+0000 7f2b7cbd7700 1 -- start start 2020-12-05T17:36:59.740 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b7cbd7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2b78061920 0x7f2b78064a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:59.740 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b7cbd7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2b780b16b0 0x7f2b78146860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:59.740 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b7cbd7700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b78146d50 con 0x7f2b781428b0 2020-12-05T17:36:59.740 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b7cbd7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b78145e80 con 0x7f2b780b16b0 2020-12-05T17:36:59.741 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b7cbd7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b78146120 con 0x7f2b78061920 2020-12-05T17:36:59.741 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b76ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2b780b16b0 0x7f2b78146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:59.741 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b76ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2b780b16b0 0x7f2b78146860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35934/0 (socket says 172.21.15.32:35934) 2020-12-05T17:36:59.741 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b76ffd700 1 -- 172.21.15.32:0/1691215757 learned_addr learned my addr 172.21.15.32:0/1691215757 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:59.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b777fe700 1 --2- 172.21.15.32:0/1691215757 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2b78061920 0x7f2b78064a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:59.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b76ffd700 1 -- 172.21.15.32:0/1691215757 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2b78061920 msgr2=0x7f2b78064a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:36:59.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b76ffd700 1 --2- 172.21.15.32:0/1691215757 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2b78061920 0x7f2b78064a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b76ffd700 1 -- 172.21.15.32:0/1691215757 >> v1:172.21.15.61:6789/0 conn(0x7f2b781428b0 legacy=0x7f2b7813c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:59.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b76ffd700 1 -- 172.21.15.32:0/1691215757 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b781463c0 con 0x7f2b780b16b0 2020-12-05T17:36:59.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b76ffd700 1 --2- 172.21.15.32:0/1691215757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b780b16b0 0x7f2b78146860 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f2b6c002f30 tx=0x7f2b6c006e80).ready entity=mon.0 client_cookie=e369b1045225f28f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:59.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b767fc700 1 -- 172.21.15.32:0/1691215757 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b6c0097f0 con 0x7f2b780b16b0 2020-12-05T17:36:59.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b767fc700 1 -- 172.21.15.32:0/1691215757 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6c009950 con 0x7f2b780b16b0 2020-12-05T17:36:59.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b767fc700 1 -- 172.21.15.32:0/1691215757 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b6c009cb0 con 0x7f2b780b16b0 2020-12-05T17:36:59.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b7cbd7700 1 -- 172.21.15.32:0/1691215757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b780b16b0 msgr2=0x7f2b78146860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:59.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b7cbd7700 1 --2- 172.21.15.32:0/1691215757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b780b16b0 0x7f2b78146860 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f2b6c002f30 tx=0x7f2b6c006e80).stop 2020-12-05T17:36:59.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b7cbd7700 1 -- 172.21.15.32:0/1691215757 shutdown_connections 2020-12-05T17:36:59.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b7cbd7700 1 --2- 172.21.15.32:0/1691215757 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2b78061920 0x7f2b78064a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b7cbd7700 1 --2- 172.21.15.32:0/1691215757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b780b16b0 0x7f2b78146860 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b7cbd7700 1 -- 172.21.15.32:0/1691215757 >> 172.21.15.32:0/1691215757 conn(0x7f2b780b6d10 msgr2=0x7f2b780b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:36:59.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b7cbd7700 1 -- 172.21.15.32:0/1691215757 shutdown_connections 2020-12-05T17:36:59.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.739+0000 7f2b7cbd7700 1 -- 172.21.15.32:0/1691215757 wait complete. 2020-12-05T17:36:59.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b7cbd7700 1 Processor -- start 2020-12-05T17:36:59.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b7cbd7700 1 -- start start 2020-12-05T17:36:59.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b7cbd7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2b78061920 0x7f2b780b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:59.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b7cbd7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b780b16b0 0x7f2b78140c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:59.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b7cbd7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2b781428b0 0x7f2b78152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:59.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b7cbd7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b780b43d0 con 0x7f2b780b16b0 2020-12-05T17:36:59.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b7cbd7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b781414b0 con 0x7f2b781428b0 2020-12-05T17:36:59.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b7cbd7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b780b3330 con 0x7f2b78061920 2020-12-05T17:36:59.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b76ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b780b16b0 0x7f2b78140c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:59.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b76ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b780b16b0 0x7f2b78140c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35940/0 (socket says 172.21.15.32:35940) 2020-12-05T17:36:59.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b76ffd700 1 -- 172.21.15.32:0/3607911620 learned_addr learned my addr 172.21.15.32:0/3607911620 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:36:59.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b777fe700 1 --2- 172.21.15.32:0/3607911620 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2b78061920 0x7f2b780b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:59.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b77fff700 1 --2- 172.21.15.32:0/3607911620 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2b781428b0 0x7f2b78152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:59.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b76ffd700 1 -- 172.21.15.32:0/3607911620 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2b78061920 msgr2=0x7f2b780b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:59.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b76ffd700 1 --2- 172.21.15.32:0/3607911620 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2b78061920 0x7f2b780b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b76ffd700 1 -- 172.21.15.32:0/3607911620 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2b781428b0 msgr2=0x7f2b78152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:36:59.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b76ffd700 1 --2- 172.21.15.32:0/3607911620 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2b781428b0 0x7f2b78152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:36:59.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b76ffd700 1 -- 172.21.15.32:0/3607911620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b780b3680 con 0x7f2b780b16b0 2020-12-05T17:36:59.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b76ffd700 1 --2- 172.21.15.32:0/3607911620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b780b16b0 0x7f2b78140c60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f2b6c002f30 tx=0x7f2b6c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:59.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b74ff9700 1 -- 172.21.15.32:0/3607911620 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b6c011c80 con 0x7f2b780b16b0 2020-12-05T17:36:59.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b74ff9700 1 -- 172.21.15.32:0/3607911620 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6c011de0 con 0x7f2b780b16b0 2020-12-05T17:36:59.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b74ff9700 1 -- 172.21.15.32:0/3607911620 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b6c01e650 con 0x7f2b780b16b0 2020-12-05T17:36:59.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b7cbd7700 1 -- 172.21.15.32:0/3607911620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b78141150 con 0x7f2b780b16b0 2020-12-05T17:36:59.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.743+0000 7f2b7cbd7700 1 -- 172.21.15.32:0/3607911620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b780b6aa0 con 0x7f2b780b16b0 2020-12-05T17:36:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.747+0000 7f2b7cbd7700 1 -- 172.21.15.32:0/3607911620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b78057050 con 0x7f2b780b16b0 2020-12-05T17:36:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.747+0000 7f2b74ff9700 1 -- 172.21.15.32:0/3607911620 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2b6c025030 con 0x7f2b780b16b0 2020-12-05T17:36:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.747+0000 7f2b74ff9700 1 --2- 172.21.15.32:0/3607911620 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2b58055470 0x7f2b58057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:36:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.747+0000 7f2b777fe700 1 --2- 172.21.15.32:0/3607911620 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2b58055470 0x7f2b58057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:36:59.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.747+0000 7f2b74ff9700 1 -- 172.21.15.32:0/3607911620 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f2b6c06f290 con 0x7f2b780b16b0 2020-12-05T17:36:59.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.747+0000 7f2b777fe700 1 --2- 172.21.15.32:0/3607911620 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2b58055470 0x7f2b58057930 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f2b6800c000 tx=0x7f2b68008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:36:59.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.755+0000 7f2b74ff9700 1 -- 172.21.15.32:0/3607911620 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2b6c0455e0 con 0x7f2b780b16b0 2020-12-05T17:36:59.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:36:59.899+0000 7f2b7cbd7700 1 -- 172.21.15.32:0/3607911620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f2b780b7a10 con 0x7f2b780b16b0 2020-12-05T17:37:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.043+0000 7f2b74ff9700 1 -- 172.21.15.32:0/3607911620 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v59) v1 ==== 133+0+0 (secure 0 0 0) 0x7f2b6c0230a0 con 0x7f2b780b16b0 2020-12-05T17:37:00.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.047+0000 7f2b7cbd7700 1 -- 172.21.15.32:0/3607911620 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2b58055470 msgr2=0x7f2b58057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:00.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.047+0000 7f2b7cbd7700 1 --2- 172.21.15.32:0/3607911620 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2b58055470 0x7f2b58057930 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f2b6800c000 tx=0x7f2b68008040).stop 2020-12-05T17:37:00.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.047+0000 7f2b7cbd7700 1 -- 172.21.15.32:0/3607911620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b780b16b0 msgr2=0x7f2b78140c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:00.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.047+0000 7f2b7cbd7700 1 --2- 172.21.15.32:0/3607911620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b780b16b0 0x7f2b78140c60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f2b6c002f30 tx=0x7f2b6c015000).stop 2020-12-05T17:37:00.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.047+0000 7f2b7cbd7700 1 -- 172.21.15.32:0/3607911620 shutdown_connections 2020-12-05T17:37:00.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.047+0000 7f2b7cbd7700 1 --2- 172.21.15.32:0/3607911620 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2b781428b0 0x7f2b78152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:00.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.047+0000 7f2b7cbd7700 1 --2- 172.21.15.32:0/3607911620 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2b58055470 0x7f2b58057930 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:00.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.047+0000 7f2b7cbd7700 1 --2- 172.21.15.32:0/3607911620 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2b78061920 0x7f2b780b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:00.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.047+0000 7f2b7cbd7700 1 --2- 172.21.15.32:0/3607911620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b780b16b0 0x7f2b78140c60 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:00.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.047+0000 7f2b7cbd7700 1 -- 172.21.15.32:0/3607911620 >> 172.21.15.32:0/3607911620 conn(0x7f2b780b6d10 msgr2=0x7f2b78144100 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:00.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.047+0000 7f2b7cbd7700 1 -- 172.21.15.32:0/3607911620 shutdown_connections 2020-12-05T17:37:00.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.047+0000 7f2b7cbd7700 1 -- 172.21.15.32:0/3607911620 wait complete. 2020-12-05T17:37:00.052 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2020-12-05T17:37:00.065 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-12-05T17:37:00.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.179+0000 7fbeef55f700 1 Processor -- start 2020-12-05T17:37:00.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.179+0000 7fbeef55f700 1 -- start start 2020-12-05T17:37:00.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeef55f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbee80b2100 0x7fbee80af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:00.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeef55f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fbee813ed60 0x7fbee8138820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:00.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeef55f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbee80ad250 0x7fbee8138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:00.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeef55f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbee813fd10 con 0x7fbee80b2100 2020-12-05T17:37:00.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeef55f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbee8139200 con 0x7fbee813ed60 2020-12-05T17:37:00.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeef55f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee80b24e0 con 0x7fbee80ad250 2020-12-05T17:37:00.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeee55d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbee80b2100 0x7fbee80af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:00.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeee55d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbee80b2100 0x7fbee80af360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35946/0 (socket says 172.21.15.32:35946) 2020-12-05T17:37:00.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeee55d700 1 -- 172.21.15.32:0/1408580208 learned_addr learned my addr 172.21.15.32:0/1408580208 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:00.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeeed5e700 1 --2- 172.21.15.32:0/1408580208 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbee80ad250 0x7fbee8138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:00.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeedd5c700 1 --2- 172.21.15.32:0/1408580208 >> v2:172.21.15.61:3300/0 conn(0x7fbee813ed60 0x7fbee8138820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:00.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeee55d700 1 -- 172.21.15.32:0/1408580208 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbee80ad250 msgr2=0x7fbee8138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:00.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeee55d700 1 --2- 172.21.15.32:0/1408580208 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbee80ad250 0x7fbee8138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:00.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeee55d700 1 -- 172.21.15.32:0/1408580208 >> v2:172.21.15.61:3300/0 conn(0x7fbee813ed60 msgr2=0x7fbee8138820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:00.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeee55d700 1 --2- 172.21.15.32:0/1408580208 >> v2:172.21.15.61:3300/0 conn(0x7fbee813ed60 0x7fbee8138820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:00.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeee55d700 1 -- 172.21.15.32:0/1408580208 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee80b0780 con 0x7fbee80b2100 2020-12-05T17:37:00.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeeed5e700 1 --2- 172.21.15.32:0/1408580208 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbee80ad250 0x7fbee8138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:00.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeee55d700 1 --2- 172.21.15.32:0/1408580208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbee80b2100 0x7fbee80af360 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7fbee4002f30 tx=0x7fbee4006f10).ready entity=mon.0 client_cookie=5f36dbc018506f69 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:00.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeed55b700 1 -- 172.21.15.32:0/1408580208 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbee400cb70 con 0x7fbee80b2100 2020-12-05T17:37:00.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeed55b700 1 -- 172.21.15.32:0/1408580208 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee400ccd0 con 0x7fbee80b2100 2020-12-05T17:37:00.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeed55b700 1 -- 172.21.15.32:0/1408580208 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbee4010600 con 0x7fbee80b2100 2020-12-05T17:37:00.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeef55f700 1 -- 172.21.15.32:0/1408580208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbee80b2100 msgr2=0x7fbee80af360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:00.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeef55f700 1 --2- 172.21.15.32:0/1408580208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbee80b2100 0x7fbee80af360 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7fbee4002f30 tx=0x7fbee4006f10).stop 2020-12-05T17:37:00.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeef55f700 1 -- 172.21.15.32:0/1408580208 shutdown_connections 2020-12-05T17:37:00.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeef55f700 1 --2- 172.21.15.32:0/1408580208 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbee80ad250 0x7fbee8138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:00.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeef55f700 1 --2- 172.21.15.32:0/1408580208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbee80b2100 0x7fbee80af360 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:00.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeef55f700 1 --2- 172.21.15.32:0/1408580208 >> v2:172.21.15.61:3300/0 conn(0x7fbee813ed60 0x7fbee8138820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:00.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeef55f700 1 -- 172.21.15.32:0/1408580208 >> 172.21.15.32:0/1408580208 conn(0x7fbee80620b0 msgr2=0x7fbee8062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:00.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeef55f700 1 -- 172.21.15.32:0/1408580208 shutdown_connections 2020-12-05T17:37:00.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeef55f700 1 -- 172.21.15.32:0/1408580208 wait complete. 2020-12-05T17:37:00.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeef55f700 1 Processor -- start 2020-12-05T17:37:00.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.183+0000 7fbeef55f700 1 -- start start 2020-12-05T17:37:00.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbeef55f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbee80ad250 0x7fbee8138cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:00.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbeef55f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbee80b2100 0x7fbee8135a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:00.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbeef55f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbee8136640 0x7fbee80b4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:00.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbeef55f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee8136340 con 0x7fbee80b2100 2020-12-05T17:37:00.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbeef55f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee8136ad0 con 0x7fbee8136640 2020-12-05T17:37:00.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbeef55f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee81360a0 con 0x7fbee80ad250 2020-12-05T17:37:00.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbeedd5c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbee80b2100 0x7fbee8135a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:00.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbeedd5c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbee80b2100 0x7fbee8135a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35952/0 (socket says 172.21.15.32:35952) 2020-12-05T17:37:00.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbeedd5c700 1 -- 172.21.15.32:0/2368543523 learned_addr learned my addr 172.21.15.32:0/2368543523 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:00.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbeee55d700 1 --2- 172.21.15.32:0/2368543523 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbee80ad250 0x7fbee8138cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:00.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbeeed5e700 1 --2- 172.21.15.32:0/2368543523 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbee8136640 0x7fbee80b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:00.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbeedd5c700 1 -- 172.21.15.32:0/2368543523 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbee80ad250 msgr2=0x7fbee8138cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:00.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbeedd5c700 1 --2- 172.21.15.32:0/2368543523 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbee80ad250 0x7fbee8138cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:00.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbeedd5c700 1 -- 172.21.15.32:0/2368543523 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbee8136640 msgr2=0x7fbee80b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:00.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbeedd5c700 1 --2- 172.21.15.32:0/2368543523 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbee8136640 0x7fbee80b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:00.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbeedd5c700 1 -- 172.21.15.32:0/2368543523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee8154ee0 con 0x7fbee80b2100 2020-12-05T17:37:00.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbeedd5c700 1 --2- 172.21.15.32:0/2368543523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbee80b2100 0x7fbee8135a00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7fbedc00c000 tx=0x7fbedc006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:00.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbedb7fe700 1 -- 172.21.15.32:0/2368543523 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbedc00cae0 con 0x7fbee80b2100 2020-12-05T17:37:00.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbedb7fe700 1 -- 172.21.15.32:0/2368543523 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbedc00cc40 con 0x7fbee80b2100 2020-12-05T17:37:00.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbeef55f700 1 -- 172.21.15.32:0/2368543523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee80b4a20 con 0x7fbee80b2100 2020-12-05T17:37:00.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbedb7fe700 1 -- 172.21.15.32:0/2368543523 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbedc0123f0 con 0x7fbee80b2100 2020-12-05T17:37:00.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbeef55f700 1 -- 172.21.15.32:0/2368543523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee81555a0 con 0x7fbee80b2100 2020-12-05T17:37:00.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbeef55f700 1 -- 172.21.15.32:0/2368543523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee8135ef0 con 0x7fbee80b2100 2020-12-05T17:37:00.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbedb7fe700 1 -- 172.21.15.32:0/2368543523 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbedc012550 con 0x7fbee80b2100 2020-12-05T17:37:00.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbedb7fe700 1 --2- 172.21.15.32:0/2368543523 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbec8055440 0x7fbec8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:00.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbedb7fe700 1 -- 172.21.15.32:0/2368543523 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5954+0+0 (secure 0 0 0) 0x7fbedc06ed80 con 0x7fbee80b2100 2020-12-05T17:37:00.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.187+0000 7fbeee55d700 1 --2- 172.21.15.32:0/2368543523 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbec8055440 0x7fbec8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:00.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.191+0000 7fbeee55d700 1 --2- 172.21.15.32:0/2368543523 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbec8055440 0x7fbec8057900 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7fbee4019000 tx=0x7fbee4006f10).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:00.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.199+0000 7fbedb7fe700 1 -- 172.21.15.32:0/2368543523 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbedc044fc0 con 0x7fbee80b2100 2020-12-05T17:37:00.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:00.343+0000 7fbeef55f700 1 -- 172.21.15.32:0/2368543523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fbee8155070 con 0x7fbee80b2100 2020-12-05T17:37:01.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.059+0000 7fbedb7fe700 1 -- 172.21.15.32:0/2368543523 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v60) v1 ==== 125+0+0 (secure 0 0 0) 0x7fbedc0290a0 con 0x7fbee80b2100 2020-12-05T17:37:01.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.067+0000 7fbeef55f700 1 -- 172.21.15.32:0/2368543523 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbec8055440 msgr2=0x7fbec8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:01.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.067+0000 7fbeef55f700 1 --2- 172.21.15.32:0/2368543523 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbec8055440 0x7fbec8057900 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7fbee4019000 tx=0x7fbee4006f10).stop 2020-12-05T17:37:01.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.067+0000 7fbeef55f700 1 -- 172.21.15.32:0/2368543523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbee80b2100 msgr2=0x7fbee8135a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:01.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.067+0000 7fbeef55f700 1 --2- 172.21.15.32:0/2368543523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbee80b2100 0x7fbee8135a00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7fbedc00c000 tx=0x7fbedc006dc0).stop 2020-12-05T17:37:01.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.067+0000 7fbeef55f700 1 -- 172.21.15.32:0/2368543523 shutdown_connections 2020-12-05T17:37:01.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.067+0000 7fbeef55f700 1 --2- 172.21.15.32:0/2368543523 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbee8136640 0x7fbee80b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.067+0000 7fbeef55f700 1 --2- 172.21.15.32:0/2368543523 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbec8055440 0x7fbec8057900 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.067+0000 7fbeef55f700 1 --2- 172.21.15.32:0/2368543523 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbee80ad250 0x7fbee8138cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.067+0000 7fbeef55f700 1 --2- 172.21.15.32:0/2368543523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbee80b2100 0x7fbee8135a00 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.067+0000 7fbeef55f700 1 -- 172.21.15.32:0/2368543523 >> 172.21.15.32:0/2368543523 conn(0x7fbee80620b0 msgr2=0x7fbee80b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:01.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.071+0000 7fbeef55f700 1 -- 172.21.15.32:0/2368543523 shutdown_connections 2020-12-05T17:37:01.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.071+0000 7fbeef55f700 1 -- 172.21.15.32:0/2368543523 wait complete. 2020-12-05T17:37:01.074 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2020-12-05T17:37:01.086 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-12-05T17:37:01.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc62a5af700 1 Processor -- start 2020-12-05T17:37:01.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc62a5af700 1 -- start start 2020-12-05T17:37:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc62a5af700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc624061920 0x7fc624064a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc62a5af700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc624142fc0 0x7fc624146860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:01.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc62a5af700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc624146d50 con 0x7fc6240b16b0 2020-12-05T17:37:01.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc62a5af700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc624145e80 con 0x7fc624142fc0 2020-12-05T17:37:01.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc62a5af700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc624146120 con 0x7fc624061920 2020-12-05T17:37:01.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc629dae700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc624142fc0 0x7fc624146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:01.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc629dae700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc624142fc0 0x7fc624146860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35964/0 (socket says 172.21.15.32:35964) 2020-12-05T17:37:01.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc629dae700 1 -- 172.21.15.32:0/4051969817 learned_addr learned my addr 172.21.15.32:0/4051969817 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:01.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc6295ad700 1 --2- 172.21.15.32:0/4051969817 >> v2:172.21.15.61:3300/0 conn(0x7fc624061920 0x7fc624064a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:01.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc629dae700 1 -- 172.21.15.32:0/4051969817 >> v2:172.21.15.61:3300/0 conn(0x7fc624061920 msgr2=0x7fc624064a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:01.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc629dae700 1 --2- 172.21.15.32:0/4051969817 >> v2:172.21.15.61:3300/0 conn(0x7fc624061920 0x7fc624064a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc629dae700 1 -- 172.21.15.32:0/4051969817 >> v1:172.21.15.61:6789/0 conn(0x7fc6240b16b0 legacy=0x7fc6241428b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:01.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc629dae700 1 -- 172.21.15.32:0/4051969817 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6241463c0 con 0x7fc624142fc0 2020-12-05T17:37:01.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc629dae700 1 --2- 172.21.15.32:0/4051969817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc624142fc0 0x7fc624146860 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7fc61c0032b0 tx=0x7fc61c009270).ready entity=mon.0 client_cookie=cf60184e9e3a9765 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:01.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc61bfff700 1 -- 172.21.15.32:0/4051969817 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc61c009e60 con 0x7fc624142fc0 2020-12-05T17:37:01.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc61bfff700 1 -- 172.21.15.32:0/4051969817 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61c00bc00 con 0x7fc624142fc0 2020-12-05T17:37:01.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc61bfff700 1 -- 172.21.15.32:0/4051969817 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc61c00bd70 con 0x7fc624142fc0 2020-12-05T17:37:01.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc62a5af700 1 -- 172.21.15.32:0/4051969817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc624142fc0 msgr2=0x7fc624146860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:01.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc62a5af700 1 --2- 172.21.15.32:0/4051969817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc624142fc0 0x7fc624146860 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7fc61c0032b0 tx=0x7fc61c009270).stop 2020-12-05T17:37:01.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc62a5af700 1 -- 172.21.15.32:0/4051969817 shutdown_connections 2020-12-05T17:37:01.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc62a5af700 1 --2- 172.21.15.32:0/4051969817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc624142fc0 0x7fc624146860 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc62a5af700 1 --2- 172.21.15.32:0/4051969817 >> v2:172.21.15.61:3300/0 conn(0x7fc624061920 0x7fc624064a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc62a5af700 1 -- 172.21.15.32:0/4051969817 >> 172.21.15.32:0/4051969817 conn(0x7fc6240b6d10 msgr2=0x7fc6240b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:01.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc62a5af700 1 -- 172.21.15.32:0/4051969817 shutdown_connections 2020-12-05T17:37:01.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.203+0000 7fc62a5af700 1 -- 172.21.15.32:0/4051969817 wait complete. 2020-12-05T17:37:01.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc62a5af700 1 Processor -- start 2020-12-05T17:37:01.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc62a5af700 1 -- start start 2020-12-05T17:37:01.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc62a5af700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc624061920 0x7fc6241417b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:01.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc62a5af700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6240b16b0 0x7fc62413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:01.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc62a5af700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc624142fc0 0x7fc62413fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:01.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc62a5af700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6241114c0 con 0x7fc6240b16b0 2020-12-05T17:37:01.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc62a5af700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc624148b00 con 0x7fc624061920 2020-12-05T17:37:01.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc62a5af700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc624112b00 con 0x7fc624142fc0 2020-12-05T17:37:01.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc628dac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6240b16b0 0x7fc62413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:01.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc628dac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6240b16b0 0x7fc62413f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35968/0 (socket says 172.21.15.32:35968) 2020-12-05T17:37:01.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc628dac700 1 -- 172.21.15.32:0/2215281773 learned_addr learned my addr 172.21.15.32:0/2215281773 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:01.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc629dae700 1 --2- 172.21.15.32:0/2215281773 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc624142fc0 0x7fc62413fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:01.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc6295ad700 1 --2- 172.21.15.32:0/2215281773 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc624061920 0x7fc6241417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:01.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc628dac700 1 -- 172.21.15.32:0/2215281773 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc624142fc0 msgr2=0x7fc62413fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:01.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc628dac700 1 --2- 172.21.15.32:0/2215281773 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc624142fc0 0x7fc62413fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc628dac700 1 -- 172.21.15.32:0/2215281773 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc624061920 msgr2=0x7fc6241417b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:01.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc628dac700 1 --2- 172.21.15.32:0/2215281773 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc624061920 0x7fc6241417b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc628dac700 1 -- 172.21.15.32:0/2215281773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62413ff80 con 0x7fc6240b16b0 2020-12-05T17:37:01.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc6295ad700 1 --2- 172.21.15.32:0/2215281773 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc624061920 0x7fc6241417b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:01.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc628dac700 1 --2- 172.21.15.32:0/2215281773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6240b16b0 0x7fc62413f5a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7fc62405f230 tx=0x7fc614007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc61a7fc700 1 -- 172.21.15.32:0/2215281773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc61400e4a0 con 0x7fc6240b16b0 2020-12-05T17:37:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc61a7fc700 1 -- 172.21.15.32:0/2215281773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61400e600 con 0x7fc6240b16b0 2020-12-05T17:37:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc61a7fc700 1 -- 172.21.15.32:0/2215281773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc61400e960 con 0x7fc6240b16b0 2020-12-05T17:37:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc62a5af700 1 -- 172.21.15.32:0/2215281773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6241486f0 con 0x7fc6240b16b0 2020-12-05T17:37:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc62a5af700 1 -- 172.21.15.32:0/2215281773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc624149140 con 0x7fc6240b16b0 2020-12-05T17:37:01.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.207+0000 7fc62a5af700 1 -- 172.21.15.32:0/2215281773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc624112940 con 0x7fc6240b16b0 2020-12-05T17:37:01.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.211+0000 7fc61a7fc700 1 -- 172.21.15.32:0/2215281773 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc614010030 con 0x7fc6240b16b0 2020-12-05T17:37:01.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.211+0000 7fc61a7fc700 1 --2- 172.21.15.32:0/2215281773 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc60c055470 0x7fc60c057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:01.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.211+0000 7fc61a7fc700 1 -- 172.21.15.32:0/2215281773 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fc61406e590 con 0x7fc6240b16b0 2020-12-05T17:37:01.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.215+0000 7fc6295ad700 1 --2- 172.21.15.32:0/2215281773 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc60c055470 0x7fc60c057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:01.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.215+0000 7fc61a7fc700 1 -- 172.21.15.32:0/2215281773 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc6140446a0 con 0x7fc6240b16b0 2020-12-05T17:37:01.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.223+0000 7fc6295ad700 1 --2- 172.21.15.32:0/2215281773 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc60c055470 0x7fc60c057930 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7fc62000b000 tx=0x7fc620008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:01.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.399+0000 7fc62a5af700 1 -- 172.21.15.32:0/2215281773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fc624057050 con 0x7fc6240b16b0 2020-12-05T17:37:01.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.403+0000 7fc61a7fc700 1 -- 172.21.15.32:0/2215281773 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 11 and data pool 12 v26) v1 ==== 189+0+0 (secure 0 0 0) 0x7fc614047cc0 con 0x7fc6240b16b0 2020-12-05T17:37:01.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.407+0000 7fc62a5af700 1 -- 172.21.15.32:0/2215281773 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc60c055470 msgr2=0x7fc60c057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:01.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.407+0000 7fc62a5af700 1 --2- 172.21.15.32:0/2215281773 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc60c055470 0x7fc60c057930 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7fc62000b000 tx=0x7fc620008040).stop 2020-12-05T17:37:01.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.407+0000 7fc62a5af700 1 -- 172.21.15.32:0/2215281773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6240b16b0 msgr2=0x7fc62413f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:01.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.407+0000 7fc62a5af700 1 --2- 172.21.15.32:0/2215281773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6240b16b0 0x7fc62413f5a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7fc62405f230 tx=0x7fc614007c20).stop 2020-12-05T17:37:01.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.407+0000 7fc62a5af700 1 -- 172.21.15.32:0/2215281773 shutdown_connections 2020-12-05T17:37:01.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.407+0000 7fc62a5af700 1 --2- 172.21.15.32:0/2215281773 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc60c055470 0x7fc60c057930 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.407+0000 7fc62a5af700 1 --2- 172.21.15.32:0/2215281773 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc624142fc0 0x7fc62413fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.407+0000 7fc62a5af700 1 --2- 172.21.15.32:0/2215281773 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc624061920 0x7fc6241417b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.407+0000 7fc62a5af700 1 --2- 172.21.15.32:0/2215281773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6240b16b0 0x7fc62413f5a0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.407+0000 7fc62a5af700 1 -- 172.21.15.32:0/2215281773 >> 172.21.15.32:0/2215281773 conn(0x7fc6240b6d10 msgr2=0x7fc624143d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:01.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.411+0000 7fc62a5af700 1 -- 172.21.15.32:0/2215281773 shutdown_connections 2020-12-05T17:37:01.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.411+0000 7fc62a5af700 1 -- 172.21.15.32:0/2215281773 wait complete. 2020-12-05T17:37:01.413 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 11 and data pool 12 2020-12-05T17:37:01.425 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:37:01.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2578f87700 1 Processor -- start 2020-12-05T17:37:01.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2578f87700 1 -- start start 2020-12-05T17:37:01.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2578f87700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f257413dad0 0x7f2574061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:01.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2578f87700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2574143a00 con 0x7f25740b7720 2020-12-05T17:37:01.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2578f87700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25740b7b00 con 0x7f2574144540 2020-12-05T17:37:01.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2578f87700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f257405fe60 con 0x7f257413dad0 2020-12-05T17:37:01.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f25737fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f25740b7720 0x7f2574143e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38636/0 (socket says 172.21.15.32:38636) 2020-12-05T17:37:01.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f25737fe700 1 -- 172.21.15.32:0/1548939540 learned_addr learned my addr 172.21.15.32:0/1548939540 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:01.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2573fff700 1 --2- 172.21.15.32:0/1548939540 >> v2:172.21.15.61:3300/0 conn(0x7f257413dad0 0x7f2574061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:01.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f25727fc700 1 -- 172.21.15.32:0/1548939540 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 612636430 0 0) 0x7f2574143a00 con 0x7f25740b7720 2020-12-05T17:37:01.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f25727fc700 1 -- 172.21.15.32:0/1548939540 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2560003230 con 0x7f25740b7720 2020-12-05T17:37:01.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f25727fc700 1 -- 172.21.15.32:0/1548939540 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3664948613 0 0) 0x7f2560003230 con 0x7f25740b7720 2020-12-05T17:37:01.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f25727fc700 1 -- 172.21.15.32:0/1548939540 >> v2:172.21.15.61:3300/0 conn(0x7f257413dad0 msgr2=0x7f2574061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:01.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f25727fc700 1 --2- 172.21.15.32:0/1548939540 >> v2:172.21.15.61:3300/0 conn(0x7f257413dad0 0x7f2574061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f25727fc700 1 -- 172.21.15.32:0/1548939540 >> v1:172.21.15.61:6789/0 conn(0x7f2574144540 legacy=0x7f257413d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:01.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f25727fc700 1 -- 172.21.15.32:0/1548939540 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2574060100 con 0x7f25740b7720 2020-12-05T17:37:01.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f25727fc700 1 -- 172.21.15.32:0/1548939540 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f25640016d0 con 0x7f25740b7720 2020-12-05T17:37:01.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f25727fc700 1 -- 172.21.15.32:0/1548939540 >> v1:172.21.15.32:6789/0 conn(0x7f25740b7720 legacy=0x7f2574143e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:01.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f25727fc700 1 --2- 172.21.15.32:0/1548939540 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2560003720 0x7f2560003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:01.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f25727fc700 1 --2- 172.21.15.32:0/1548939540 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f25600044d0 0x7f2560004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:01.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f25727fc700 1 --2- 172.21.15.32:0/1548939540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2560004e40 0x7f256000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:01.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f25727fc700 1 -- 172.21.15.32:0/1548939540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257405fe60 con 0x7f2560004e40 2020-12-05T17:37:01.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f25727fc700 1 -- 172.21.15.32:0/1548939540 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2560004250 con 0x7f25600044d0 2020-12-05T17:37:01.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f25727fc700 1 -- 172.21.15.32:0/1548939540 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256000bc10 con 0x7f2560003720 2020-12-05T17:37:01.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2573fff700 1 --2- 172.21.15.32:0/1548939540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2560004e40 0x7f256000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:01.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f25737fe700 1 --2- 172.21.15.32:0/1548939540 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2560003720 0x7f2560003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:01.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2572ffd700 1 --2- 172.21.15.32:0/1548939540 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f25600044d0 0x7f2560004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:01.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2573fff700 1 -- 172.21.15.32:0/1548939540 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2560003720 msgr2=0x7f2560003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:01.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2573fff700 1 --2- 172.21.15.32:0/1548939540 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2560003720 0x7f2560003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2573fff700 1 -- 172.21.15.32:0/1548939540 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f25600044d0 msgr2=0x7f2560004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:01.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2573fff700 1 --2- 172.21.15.32:0/1548939540 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f25600044d0 0x7f2560004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2573fff700 1 -- 172.21.15.32:0/1548939540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2560003230 con 0x7f2560004e40 2020-12-05T17:37:01.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2572ffd700 1 --2- 172.21.15.32:0/1548939540 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f25600044d0 0x7f2560004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:01.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f25737fe700 1 --2- 172.21.15.32:0/1548939540 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2560003720 0x7f2560003b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:01.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2573fff700 1 -- 172.21.15.32:0/1548939540 reap_dead start 2020-12-05T17:37:01.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2573fff700 1 --2- 172.21.15.32:0/1548939540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2560004e40 0x7f256000b720 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f256c00c000 tx=0x7f256c0091a0).ready entity=mon.0 client_cookie=22f8cd6a6d0945cf server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:01.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f25727fc700 1 -- 172.21.15.32:0/1548939540 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f256c00ed00 con 0x7f2560004e40 2020-12-05T17:37:01.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f25727fc700 1 -- 172.21.15.32:0/1548939540 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256c00ee60 con 0x7f2560004e40 2020-12-05T17:37:01.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f25727fc700 1 -- 172.21.15.32:0/1548939540 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f256c0125e0 con 0x7f2560004e40 2020-12-05T17:37:01.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2578f87700 1 -- 172.21.15.32:0/1548939540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2560004e40 msgr2=0x7f256000b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:01.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2578f87700 1 --2- 172.21.15.32:0/1548939540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2560004e40 0x7f256000b720 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f256c00c000 tx=0x7f256c0091a0).stop 2020-12-05T17:37:01.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2578f87700 1 -- 172.21.15.32:0/1548939540 shutdown_connections 2020-12-05T17:37:01.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2578f87700 1 --2- 172.21.15.32:0/1548939540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2560004e40 0x7f256000b720 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2578f87700 1 -- 172.21.15.32:0/1548939540 >> 172.21.15.32:0/1548939540 conn(0x7f257405bc20 msgr2=0x7f25740b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:01.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2578f87700 1 -- 172.21.15.32:0/1548939540 shutdown_connections 2020-12-05T17:37:01.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.527+0000 7f2578f87700 1 -- 172.21.15.32:0/1548939540 wait complete. 2020-12-05T17:37:01.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2578f87700 1 Processor -- start 2020-12-05T17:37:01.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2578f87700 1 -- start start 2020-12-05T17:37:01.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2578f87700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2560004e40 0x7f257413fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:01.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2578f87700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25741524d0 0x7f25740b3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:01.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2578f87700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25740b0a20 0x7f25740b3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:01.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2578f87700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25740b0120 con 0x7f25741524d0 2020-12-05T17:37:01.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2578f87700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25740b1ad0 con 0x7f2560004e40 2020-12-05T17:37:01.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2578f87700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25740afbe0 con 0x7f25740b0a20 2020-12-05T17:37:01.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2572ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25741524d0 0x7f25740b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:01.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2572ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25741524d0 0x7f25740b3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36002/0 (socket says 172.21.15.32:36002) 2020-12-05T17:37:01.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2572ffd700 1 -- 172.21.15.32:0/1296762409 learned_addr learned my addr 172.21.15.32:0/1296762409 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:01.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f25737fe700 1 --2- 172.21.15.32:0/1296762409 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2560004e40 0x7f257413fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:01.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2573fff700 1 --2- 172.21.15.32:0/1296762409 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25740b0a20 0x7f25740b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:01.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2572ffd700 1 -- 172.21.15.32:0/1296762409 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25740b0a20 msgr2=0x7f25740b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:01.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2572ffd700 1 --2- 172.21.15.32:0/1296762409 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25740b0a20 0x7f25740b3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2572ffd700 1 -- 172.21.15.32:0/1296762409 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2560004e40 msgr2=0x7f257413fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:01.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2572ffd700 1 --2- 172.21.15.32:0/1296762409 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2560004e40 0x7f257413fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2572ffd700 1 -- 172.21.15.32:0/1296762409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25740afe80 con 0x7f25741524d0 2020-12-05T17:37:01.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f25737fe700 1 --2- 172.21.15.32:0/1296762409 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2560004e40 0x7f257413fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:01.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2573fff700 1 --2- 172.21.15.32:0/1296762409 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25740b0a20 0x7f25740b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:01.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2572ffd700 1 --2- 172.21.15.32:0/1296762409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25741524d0 0x7f25740b3440 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f25680076b0 tx=0x7f2568009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:01.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2570ff9700 1 -- 172.21.15.32:0/1296762409 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2568010680 con 0x7f25741524d0 2020-12-05T17:37:01.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2570ff9700 1 -- 172.21.15.32:0/1296762409 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2568010040 con 0x7f25741524d0 2020-12-05T17:37:01.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2578f87700 1 -- 172.21.15.32:0/1296762409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25740b3e20 con 0x7f25741524d0 2020-12-05T17:37:01.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2570ff9700 1 -- 172.21.15.32:0/1296762409 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25680109d0 con 0x7f25741524d0 2020-12-05T17:37:01.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.531+0000 7f2578f87700 1 -- 172.21.15.32:0/1296762409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25740adde0 con 0x7f25741524d0 2020-12-05T17:37:01.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.535+0000 7f2578f87700 1 -- 172.21.15.32:0/1296762409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25740b3130 con 0x7f25741524d0 2020-12-05T17:37:01.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.535+0000 7f2570ff9700 1 -- 172.21.15.32:0/1296762409 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f256801c060 con 0x7f25741524d0 2020-12-05T17:37:01.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.535+0000 7f2570ff9700 1 --2- 172.21.15.32:0/1296762409 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2554055330 0x7f25540577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:01.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.535+0000 7f25737fe700 1 --2- 172.21.15.32:0/1296762409 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2554055330 0x7f25540577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:01.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.535+0000 7f2570ff9700 1 -- 172.21.15.32:0/1296762409 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f256806d7a0 con 0x7f25741524d0 2020-12-05T17:37:01.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.535+0000 7f25737fe700 1 --2- 172.21.15.32:0/1296762409 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2554055330 0x7f25540577f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f256400d000 tx=0x7f2564009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:01.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.543+0000 7f2570ff9700 1 -- 172.21.15.32:0/1296762409 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f25680438c0 con 0x7f25741524d0 2020-12-05T17:37:01.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.679+0000 7f2578f87700 1 -- 172.21.15.32:0/1296762409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2574057050 con 0x7f25741524d0 2020-12-05T17:37:01.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.679+0000 7f2570ff9700 1 -- 172.21.15.32:0/1296762409 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f256801a020 con 0x7f25741524d0 2020-12-05T17:37:01.682 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:01.682 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":62,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:37:01.403672+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:36:59.960851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-12-05T17:37:00.401054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:37:01.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.683+0000 7f2578f87700 1 -- 172.21.15.32:0/1296762409 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2554055330 msgr2=0x7f25540577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:01.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.683+0000 7f2578f87700 1 --2- 172.21.15.32:0/1296762409 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2554055330 0x7f25540577f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f256400d000 tx=0x7f2564009040).stop 2020-12-05T17:37:01.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.683+0000 7f2578f87700 1 -- 172.21.15.32:0/1296762409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25741524d0 msgr2=0x7f25740b3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:01.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.683+0000 7f2578f87700 1 --2- 172.21.15.32:0/1296762409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25741524d0 0x7f25740b3440 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f25680076b0 tx=0x7f2568009040).stop 2020-12-05T17:37:01.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.683+0000 7f2578f87700 1 -- 172.21.15.32:0/1296762409 shutdown_connections 2020-12-05T17:37:01.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.683+0000 7f2578f87700 1 --2- 172.21.15.32:0/1296762409 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2554055330 0x7f25540577f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.683+0000 7f2578f87700 1 --2- 172.21.15.32:0/1296762409 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25740b0a20 0x7f25740b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.683+0000 7f2578f87700 1 --2- 172.21.15.32:0/1296762409 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2560004e40 0x7f257413fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.683+0000 7f2578f87700 1 --2- 172.21.15.32:0/1296762409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25741524d0 0x7f25740b3440 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.683+0000 7f2578f87700 1 -- 172.21.15.32:0/1296762409 >> 172.21.15.32:0/1296762409 conn(0x7f257405bc20 msgr2=0x7f2574061780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:01.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.683+0000 7f2578f87700 1 -- 172.21.15.32:0/1296762409 shutdown_connections 2020-12-05T17:37:01.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.683+0000 7f2578f87700 1 -- 172.21.15.32:0/1296762409 wait complete. 2020-12-05T17:37:01.697 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:37:01.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f54647cc700 1 Processor -- start 2020-12-05T17:37:01.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f54647cc700 1 -- start start 2020-12-05T17:37:01.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f54647cc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f545c05fe00 0x7f545c0b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:01.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f54647cc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f545c13c730 0x7f545c13cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:01.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f54647cc700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545c0601e0 con 0x7f545c142d00 2020-12-05T17:37:01.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f54647cc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f545c061480 con 0x7f545c05fe00 2020-12-05T17:37:01.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f54647cc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545c0ad510 con 0x7f545c13c730 2020-12-05T17:37:01.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f5462568700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f545c05fe00 0x7f545c0b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:01.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f5462568700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f545c05fe00 0x7f545c0b0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36008/0 (socket says 172.21.15.32:36008) 2020-12-05T17:37:01.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f5462568700 1 -- 172.21.15.32:0/3252905120 learned_addr learned my addr 172.21.15.32:0/3252905120 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:01.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f5461d67700 1 --2- 172.21.15.32:0/3252905120 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f545c13c730 0x7f545c13cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:01.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f5462568700 1 -- 172.21.15.32:0/3252905120 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f545c13c730 msgr2=0x7f545c13cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:01.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f5462568700 1 --2- 172.21.15.32:0/3252905120 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f545c13c730 0x7f545c13cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f5462568700 1 -- 172.21.15.32:0/3252905120 >> v1:172.21.15.61:6789/0 conn(0x7f545c142d00 legacy=0x7f545c1431b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:01.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f5462568700 1 -- 172.21.15.32:0/3252905120 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c0af250 con 0x7f545c05fe00 2020-12-05T17:37:01.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f5462568700 1 --2- 172.21.15.32:0/3252905120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f545c05fe00 0x7f545c0b0f40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f54580032b0 tx=0x7f5458009270).ready entity=mon.0 client_cookie=10fd9270ab3c3919 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:01.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f5461566700 1 -- 172.21.15.32:0/3252905120 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f545800bc00 con 0x7f545c05fe00 2020-12-05T17:37:01.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f5461566700 1 -- 172.21.15.32:0/3252905120 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545800bd60 con 0x7f545c05fe00 2020-12-05T17:37:01.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f5461566700 1 -- 172.21.15.32:0/3252905120 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5458011400 con 0x7f545c05fe00 2020-12-05T17:37:01.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f54647cc700 1 -- 172.21.15.32:0/3252905120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f545c05fe00 msgr2=0x7f545c0b0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:01.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f54647cc700 1 --2- 172.21.15.32:0/3252905120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f545c05fe00 0x7f545c0b0f40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f54580032b0 tx=0x7f5458009270).stop 2020-12-05T17:37:01.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f54647cc700 1 -- 172.21.15.32:0/3252905120 shutdown_connections 2020-12-05T17:37:01.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f54647cc700 1 --2- 172.21.15.32:0/3252905120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f545c05fe00 0x7f545c0b0f40 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f54647cc700 1 --2- 172.21.15.32:0/3252905120 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f545c13c730 0x7f545c13cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f54647cc700 1 -- 172.21.15.32:0/3252905120 >> 172.21.15.32:0/3252905120 conn(0x7f545c0b57c0 msgr2=0x7f545c05f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:01.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f54647cc700 1 -- 172.21.15.32:0/3252905120 shutdown_connections 2020-12-05T17:37:01.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.807+0000 7f54647cc700 1 -- 172.21.15.32:0/3252905120 wait complete. 2020-12-05T17:37:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.811+0000 7f54647cc700 1 Processor -- start 2020-12-05T17:37:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.811+0000 7f54647cc700 1 -- start start 2020-12-05T17:37:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.811+0000 7f54647cc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f545c05fe00 0x7f545c061d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.811+0000 7f54647cc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f545c13c730 0x7f545c13af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.811+0000 7f54647cc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f545c142d00 0x7f545c13b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.811+0000 7f54647cc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c0641e0 con 0x7f545c05fe00 2020-12-05T17:37:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.811+0000 7f5462568700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f545c05fe00 0x7f545c061d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:01.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.811+0000 7f54647cc700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c13cce0 con 0x7f545c142d00 2020-12-05T17:37:01.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.811+0000 7f5462568700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f545c05fe00 0x7f545c061d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36014/0 (socket says 172.21.15.32:36014) 2020-12-05T17:37:01.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.811+0000 7f5462568700 1 -- 172.21.15.32:0/4200080048 learned_addr learned my addr 172.21.15.32:0/4200080048 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:01.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.811+0000 7f54647cc700 1 -- 172.21.15.32:0/4200080048 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545c13cf80 con 0x7f545c13c730 2020-12-05T17:37:01.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.811+0000 7f5461d67700 1 --2- 172.21.15.32:0/4200080048 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f545c13c730 0x7f545c13af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:01.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.811+0000 7f5462d69700 1 --2- 172.21.15.32:0/4200080048 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f545c142d00 0x7f545c13b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:01.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.815+0000 7f5462568700 1 -- 172.21.15.32:0/4200080048 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f545c13c730 msgr2=0x7f545c13af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:01.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.815+0000 7f5462568700 1 --2- 172.21.15.32:0/4200080048 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f545c13c730 0x7f545c13af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.815+0000 7f5462568700 1 -- 172.21.15.32:0/4200080048 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f545c142d00 msgr2=0x7f545c13b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:01.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.815+0000 7f5462568700 1 --2- 172.21.15.32:0/4200080048 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f545c142d00 0x7f545c13b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.815+0000 7f5462568700 1 -- 172.21.15.32:0/4200080048 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c066d60 con 0x7f545c05fe00 2020-12-05T17:37:01.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.815+0000 7f5462568700 1 --2- 172.21.15.32:0/4200080048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f545c05fe00 0x7f545c061d30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f54580032b0 tx=0x7f5458016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:01.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.815+0000 7f544b7fe700 1 -- 172.21.15.32:0/4200080048 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54580107a0 con 0x7f545c05fe00 2020-12-05T17:37:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.815+0000 7f544b7fe700 1 -- 172.21.15.32:0/4200080048 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5458010900 con 0x7f545c05fe00 2020-12-05T17:37:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.815+0000 7f544b7fe700 1 -- 172.21.15.32:0/4200080048 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5458010c60 con 0x7f545c05fe00 2020-12-05T17:37:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.815+0000 7f54647cc700 1 -- 172.21.15.32:0/4200080048 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f545c063d40 con 0x7f545c05fe00 2020-12-05T17:37:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.815+0000 7f54647cc700 1 -- 172.21.15.32:0/4200080048 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f545c066990 con 0x7f545c05fe00 2020-12-05T17:37:01.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.815+0000 7f54647cc700 1 -- 172.21.15.32:0/4200080048 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f545c063ff0 con 0x7f545c05fe00 2020-12-05T17:37:01.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.815+0000 7f544b7fe700 1 -- 172.21.15.32:0/4200080048 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f545800f070 con 0x7f545c05fe00 2020-12-05T17:37:01.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.815+0000 7f544b7fe700 1 --2- 172.21.15.32:0/4200080048 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f543c055440 0x7f543c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:01.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.815+0000 7f544b7fe700 1 -- 172.21.15.32:0/4200080048 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f545806f980 con 0x7f545c05fe00 2020-12-05T17:37:01.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.815+0000 7f5461d67700 1 --2- 172.21.15.32:0/4200080048 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f543c055440 0x7f543c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:01.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.815+0000 7f5461d67700 1 --2- 172.21.15.32:0/4200080048 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f543c055440 0x7f543c057900 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f544c00c000 tx=0x7f544c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:01.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.823+0000 7f544b7fe700 1 -- 172.21.15.32:0/4200080048 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5458045a80 con 0x7f545c05fe00 2020-12-05T17:37:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.963+0000 7f54647cc700 1 -- 172.21.15.32:0/4200080048 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f545c057050 con 0x7f545c05fe00 2020-12-05T17:37:01.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.967+0000 7f544b7fe700 1 -- 172.21.15.32:0/4200080048 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f5458015020 con 0x7f545c05fe00 2020-12-05T17:37:01.967 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:01.967 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":62,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:37:01.403672+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:36:59.960851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-12-05T17:37:00.401054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:37:01.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.967+0000 7f54647cc700 1 -- 172.21.15.32:0/4200080048 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f543c055440 msgr2=0x7f543c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:01.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.967+0000 7f54647cc700 1 --2- 172.21.15.32:0/4200080048 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f543c055440 0x7f543c057900 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f544c00c000 tx=0x7f544c008040).stop 2020-12-05T17:37:01.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.967+0000 7f54647cc700 1 -- 172.21.15.32:0/4200080048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f545c05fe00 msgr2=0x7f545c061d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:01.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.967+0000 7f54647cc700 1 --2- 172.21.15.32:0/4200080048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f545c05fe00 0x7f545c061d30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f54580032b0 tx=0x7f5458016000).stop 2020-12-05T17:37:01.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.967+0000 7f54647cc700 1 -- 172.21.15.32:0/4200080048 shutdown_connections 2020-12-05T17:37:01.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.967+0000 7f54647cc700 1 --2- 172.21.15.32:0/4200080048 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f545c142d00 0x7f545c13b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.967+0000 7f54647cc700 1 --2- 172.21.15.32:0/4200080048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f545c05fe00 0x7f545c061d30 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.967+0000 7f54647cc700 1 --2- 172.21.15.32:0/4200080048 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f543c055440 0x7f543c057900 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.967+0000 7f54647cc700 1 --2- 172.21.15.32:0/4200080048 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f545c13c730 0x7f545c13af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:01.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.967+0000 7f54647cc700 1 -- 172.21.15.32:0/4200080048 >> 172.21.15.32:0/4200080048 conn(0x7f545c0b57c0 msgr2=0x7f545c0b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:01.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.967+0000 7f54647cc700 1 -- 172.21.15.32:0/4200080048 shutdown_connections 2020-12-05T17:37:01.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:01.967+0000 7f54647cc700 1 -- 172.21.15.32:0/4200080048 wait complete. 2020-12-05T17:37:01.981 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-12-05T17:37:02.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.099+0000 7feec929a700 1 Processor -- start 2020-12-05T17:37:02.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.099+0000 7feec929a700 1 -- start start 2020-12-05T17:37:02.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.099+0000 7feec929a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feec40b1dc0 0x7feec40b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:02.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.099+0000 7feec929a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feec40af5f0 0x7feec40aed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:02.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.099+0000 7feec929a700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec41468e0 con 0x7feec40afd30 2020-12-05T17:37:02.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.099+0000 7feec929a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec4142c90 con 0x7feec40af5f0 2020-12-05T17:37:02.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.099+0000 7feec929a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec4146b10 con 0x7feec40b1dc0 2020-12-05T17:37:02.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.099+0000 7feec8a99700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feec40af5f0 0x7feec40aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:02.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.099+0000 7feec8a99700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feec40af5f0 0x7feec40aed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36026/0 (socket says 172.21.15.32:36026) 2020-12-05T17:37:02.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.099+0000 7feec8a99700 1 -- 172.21.15.32:0/3351893632 learned_addr learned my addr 172.21.15.32:0/3351893632 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:02.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.099+0000 7feec3fff700 1 --2- 172.21.15.32:0/3351893632 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feec40b1dc0 0x7feec40b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:02.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.099+0000 7feec8a99700 1 -- 172.21.15.32:0/3351893632 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feec40b1dc0 msgr2=0x7feec40b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:02.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.099+0000 7feec8a99700 1 --2- 172.21.15.32:0/3351893632 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feec40b1dc0 0x7feec40b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:02.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.099+0000 7feec8a99700 1 -- 172.21.15.32:0/3351893632 >> v1:172.21.15.61:6789/0 conn(0x7feec40afd30 legacy=0x7feec40ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:02.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.099+0000 7feec8a99700 1 -- 172.21.15.32:0/3351893632 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec4146db0 con 0x7feec40af5f0 2020-12-05T17:37:02.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.099+0000 7feec8a99700 1 --2- 172.21.15.32:0/3351893632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feec40af5f0 0x7feec40aed10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7feeb40032b0 tx=0x7feeb4009270).ready entity=mon.0 client_cookie=3ce37101fc937255 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:02.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.099+0000 7feec2ffd700 1 -- 172.21.15.32:0/3351893632 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feeb4009e60 con 0x7feec40af5f0 2020-12-05T17:37:02.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.099+0000 7feec2ffd700 1 -- 172.21.15.32:0/3351893632 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb400bc00 con 0x7feec40af5f0 2020-12-05T17:37:02.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.099+0000 7feec2ffd700 1 -- 172.21.15.32:0/3351893632 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feeb400bd70 con 0x7feec40af5f0 2020-12-05T17:37:02.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.099+0000 7feec929a700 1 -- 172.21.15.32:0/3351893632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feec40af5f0 msgr2=0x7feec40aed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:02.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.099+0000 7feec929a700 1 --2- 172.21.15.32:0/3351893632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feec40af5f0 0x7feec40aed10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7feeb40032b0 tx=0x7feeb4009270).stop 2020-12-05T17:37:02.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.103+0000 7feec929a700 1 -- 172.21.15.32:0/3351893632 shutdown_connections 2020-12-05T17:37:02.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.103+0000 7feec929a700 1 --2- 172.21.15.32:0/3351893632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feec40af5f0 0x7feec40aed10 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:02.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.103+0000 7feec929a700 1 --2- 172.21.15.32:0/3351893632 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feec40b1dc0 0x7feec40b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:02.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.103+0000 7feec929a700 1 -- 172.21.15.32:0/3351893632 >> 172.21.15.32:0/3351893632 conn(0x7feec40620b0 msgr2=0x7feec4062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:02.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.103+0000 7feec929a700 1 -- 172.21.15.32:0/3351893632 shutdown_connections 2020-12-05T17:37:02.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.103+0000 7feec929a700 1 -- 172.21.15.32:0/3351893632 wait complete. 2020-12-05T17:37:02.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.103+0000 7feec929a700 1 Processor -- start 2020-12-05T17:37:02.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.103+0000 7feec929a700 1 -- start start 2020-12-05T17:37:02.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.103+0000 7feec929a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feec40af5f0 0x7feec4063380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:02.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.103+0000 7feec929a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feec40afd30 0x7feec413f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:02.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.103+0000 7feec929a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feec40b1dc0 0x7feec413f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:02.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.103+0000 7feec929a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec4065bc0 con 0x7feec40af5f0 2020-12-05T17:37:02.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.103+0000 7feec929a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec4066f20 con 0x7feec40afd30 2020-12-05T17:37:02.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.103+0000 7feec929a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec41466b0 con 0x7feec40b1dc0 2020-12-05T17:37:02.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.103+0000 7feec3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feec40af5f0 0x7feec4063380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:02.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.103+0000 7feec3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feec40af5f0 0x7feec4063380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36028/0 (socket says 172.21.15.32:36028) 2020-12-05T17:37:02.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.103+0000 7feec3fff700 1 -- 172.21.15.32:0/3181717962 learned_addr learned my addr 172.21.15.32:0/3181717962 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:02.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.103+0000 7feec8a99700 1 --2- 172.21.15.32:0/3181717962 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feec40b1dc0 0x7feec413f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:02.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.103+0000 7feec37fe700 1 --2- 172.21.15.32:0/3181717962 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feec40afd30 0x7feec413f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:02.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.107+0000 7feec3fff700 1 -- 172.21.15.32:0/3181717962 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feec40b1dc0 msgr2=0x7feec413f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:02.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.107+0000 7feec3fff700 1 --2- 172.21.15.32:0/3181717962 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feec40b1dc0 0x7feec413f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:02.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.107+0000 7feec3fff700 1 -- 172.21.15.32:0/3181717962 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feec40afd30 msgr2=0x7feec413f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:02.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.107+0000 7feec3fff700 1 --2- 172.21.15.32:0/3181717962 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feec40afd30 0x7feec413f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:02.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.107+0000 7feec3fff700 1 -- 172.21.15.32:0/3181717962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec40649c0 con 0x7feec40af5f0 2020-12-05T17:37:02.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.107+0000 7feec3fff700 1 --2- 172.21.15.32:0/3181717962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feec40af5f0 0x7feec4063380 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7feebc00c000 tx=0x7feebc006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:02.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.107+0000 7feec17fa700 1 -- 172.21.15.32:0/3181717962 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feebc00ca10 con 0x7feec40af5f0 2020-12-05T17:37:02.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.107+0000 7feec929a700 1 -- 172.21.15.32:0/3181717962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec4064c60 con 0x7feec40af5f0 2020-12-05T17:37:02.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.107+0000 7feec17fa700 1 -- 172.21.15.32:0/3181717962 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feebc00cb70 con 0x7feec40af5f0 2020-12-05T17:37:02.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.107+0000 7feec17fa700 1 -- 172.21.15.32:0/3181717962 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feebc00b2d0 con 0x7feec40af5f0 2020-12-05T17:37:02.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.107+0000 7feec929a700 1 -- 172.21.15.32:0/3181717962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec40652a0 con 0x7feec40af5f0 2020-12-05T17:37:02.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.107+0000 7feec929a700 1 -- 172.21.15.32:0/3181717962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec4064df0 con 0x7feec40af5f0 2020-12-05T17:37:02.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.107+0000 7feec17fa700 1 -- 172.21.15.32:0/3181717962 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7feebc01b400 con 0x7feec40af5f0 2020-12-05T17:37:02.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.107+0000 7feec17fa700 1 --2- 172.21.15.32:0/3181717962 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feea80553f0 0x7feea80578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:02.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.107+0000 7feec17fa700 1 -- 172.21.15.32:0/3181717962 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7feebc06e160 con 0x7feec40af5f0 2020-12-05T17:37:02.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.107+0000 7feec37fe700 1 --2- 172.21.15.32:0/3181717962 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feea80553f0 0x7feea80578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:02.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.111+0000 7feec37fe700 1 --2- 172.21.15.32:0/3181717962 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feea80553f0 0x7feea80578b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7feeac00d000 tx=0x7feeac007b60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:02.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.115+0000 7feec17fa700 1 -- 172.21.15.32:0/3181717962 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7feebc044200 con 0x7feec40af5f0 2020-12-05T17:37:02.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.302+0000 7feec929a700 1 -- 172.21.15.32:0/3181717962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7feec4057050 con 0x7feec40af5f0 2020-12-05T17:37:02.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.406+0000 7feec17fa700 1 -- 172.21.15.32:0/3181717962 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v27) v1 ==== 118+0+0 (secure 0 0 0) 0x7feebc047820 con 0x7feec40af5f0 2020-12-05T17:37:02.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.410+0000 7feec929a700 1 -- 172.21.15.32:0/3181717962 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feea80553f0 msgr2=0x7feea80578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:02.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.410+0000 7feec929a700 1 --2- 172.21.15.32:0/3181717962 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feea80553f0 0x7feea80578b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7feeac00d000 tx=0x7feeac007b60).stop 2020-12-05T17:37:02.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.410+0000 7feec929a700 1 -- 172.21.15.32:0/3181717962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feec40af5f0 msgr2=0x7feec4063380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:02.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.410+0000 7feec929a700 1 --2- 172.21.15.32:0/3181717962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feec40af5f0 0x7feec4063380 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7feebc00c000 tx=0x7feebc006dc0).stop 2020-12-05T17:37:02.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.410+0000 7feec929a700 1 -- 172.21.15.32:0/3181717962 shutdown_connections 2020-12-05T17:37:02.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.410+0000 7feec929a700 1 --2- 172.21.15.32:0/3181717962 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feea80553f0 0x7feea80578b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:02.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.410+0000 7feec929a700 1 --2- 172.21.15.32:0/3181717962 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feec40b1dc0 0x7feec413f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:02.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.410+0000 7feec929a700 1 --2- 172.21.15.32:0/3181717962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feec40af5f0 0x7feec4063380 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:02.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.410+0000 7feec929a700 1 --2- 172.21.15.32:0/3181717962 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feec40afd30 0x7feec413f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:02.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.410+0000 7feec929a700 1 -- 172.21.15.32:0/3181717962 >> 172.21.15.32:0/3181717962 conn(0x7feec40620b0 msgr2=0x7feec405feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:02.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.410+0000 7feec929a700 1 -- 172.21.15.32:0/3181717962 shutdown_connections 2020-12-05T17:37:02.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.410+0000 7feec929a700 1 -- 172.21.15.32:0/3181717962 wait complete. 2020-12-05T17:37:02.428 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:37:02.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.542+0000 7fab515b1700 1 Processor -- start 2020-12-05T17:37:02.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.542+0000 7fab515b1700 1 -- start start 2020-12-05T17:37:02.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.542+0000 7fab515b1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab4c05f6f0 0x7fab4c0b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:02.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.542+0000 7fab515b1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab4c0af920 0x7fab4c063040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:02.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.542+0000 7fab515b1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab4c05fad0 con 0x7fab4c0afea0 2020-12-05T17:37:02.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.542+0000 7fab515b1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab4c0b2cd0 con 0x7fab4c0af920 2020-12-05T17:37:02.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.542+0000 7fab515b1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c0b3560 con 0x7fab4c05f6f0 2020-12-05T17:37:02.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.542+0000 7fab4b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab4c0af920 0x7fab4c063040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:02.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.542+0000 7fab4b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab4c0af920 0x7fab4c063040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36038/0 (socket says 172.21.15.32:36038) 2020-12-05T17:37:02.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.542+0000 7fab4b7fe700 1 -- 172.21.15.32:0/2462733288 learned_addr learned my addr 172.21.15.32:0/2462733288 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:02.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.542+0000 7fab50db0700 1 --1- 172.21.15.32:0/2462733288 >> v1:172.21.15.32:6789/0 conn(0x7fab4c0afea0 0x7fab4c13f9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38688/0 (socket says 172.21.15.32:38688) 2020-12-05T17:37:02.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.542+0000 7fab4bfff700 1 --2- 172.21.15.32:0/2462733288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab4c05f6f0 0x7fab4c0b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:02.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.542+0000 7fab4b7fe700 1 -- 172.21.15.32:0/2462733288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab4c05f6f0 msgr2=0x7fab4c0b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:02.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.542+0000 7fab4b7fe700 1 --2- 172.21.15.32:0/2462733288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab4c05f6f0 0x7fab4c0b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:02.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.542+0000 7fab4b7fe700 1 -- 172.21.15.32:0/2462733288 >> v1:172.21.15.32:6789/0 conn(0x7fab4c0afea0 legacy=0x7fab4c13f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:02.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.542+0000 7fab4affd700 1 -- 172.21.15.32:0/2462733288 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2864883058 0 0) 0x7fab4c05fad0 con 0x7fab4c0afea0 2020-12-05T17:37:02.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.542+0000 7fab4b7fe700 1 -- 172.21.15.32:0/2462733288 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4c144650 con 0x7fab4c0af920 2020-12-05T17:37:02.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.542+0000 7fab4b7fe700 1 --2- 172.21.15.32:0/2462733288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4c0af920 0x7fab4c063040 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7fab40002f30 tx=0x7fab40006e80).ready entity=mon.0 client_cookie=adc9da0386eb8ef7 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:02.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.542+0000 7fab4affd700 1 -- 172.21.15.32:0/2462733288 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab400097f0 con 0x7fab4c0af920 2020-12-05T17:37:02.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.542+0000 7fab4affd700 1 -- 172.21.15.32:0/2462733288 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab40009950 con 0x7fab4c0af920 2020-12-05T17:37:02.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.542+0000 7fab4affd700 1 -- 172.21.15.32:0/2462733288 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab40009cb0 con 0x7fab4c0af920 2020-12-05T17:37:02.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.542+0000 7fab515b1700 1 -- 172.21.15.32:0/2462733288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4c0af920 msgr2=0x7fab4c063040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:02.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.542+0000 7fab515b1700 1 --2- 172.21.15.32:0/2462733288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4c0af920 0x7fab4c063040 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7fab40002f30 tx=0x7fab40006e80).stop 2020-12-05T17:37:02.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.546+0000 7fab515b1700 1 -- 172.21.15.32:0/2462733288 shutdown_connections 2020-12-05T17:37:02.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.546+0000 7fab515b1700 1 --2- 172.21.15.32:0/2462733288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab4c05f6f0 0x7fab4c0b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:02.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.546+0000 7fab515b1700 1 --2- 172.21.15.32:0/2462733288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4c0af920 0x7fab4c063040 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:02.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.546+0000 7fab515b1700 1 -- 172.21.15.32:0/2462733288 >> 172.21.15.32:0/2462733288 conn(0x7fab4c0604c0 msgr2=0x7fab4c05c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:02.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.546+0000 7fab515b1700 1 -- 172.21.15.32:0/2462733288 shutdown_connections 2020-12-05T17:37:02.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.546+0000 7fab515b1700 1 -- 172.21.15.32:0/2462733288 wait complete. 2020-12-05T17:37:02.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.546+0000 7fab515b1700 1 Processor -- start 2020-12-05T17:37:02.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.546+0000 7fab515b1700 1 -- start start 2020-12-05T17:37:02.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.546+0000 7fab515b1700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab4c05f6f0 0x7fab4c0632e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:02.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.546+0000 7fab515b1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab4c0af920 0x7fab4c13cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:02.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.546+0000 7fab515b1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4c0afea0 0x7fab4c064860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:02.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.546+0000 7fab515b1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c063da0 con 0x7fab4c0afea0 2020-12-05T17:37:02.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.546+0000 7fab515b1700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c0b5cf0 con 0x7fab4c05f6f0 2020-12-05T17:37:02.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.546+0000 7fab515b1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c0b5f90 con 0x7fab4c0af920 2020-12-05T17:37:02.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.546+0000 7fab50db0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4c0afea0 0x7fab4c064860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:02.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.546+0000 7fab50db0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4c0afea0 0x7fab4c064860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36046/0 (socket says 172.21.15.32:36046) 2020-12-05T17:37:02.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.546+0000 7fab50db0700 1 -- 172.21.15.32:0/2776390548 learned_addr learned my addr 172.21.15.32:0/2776390548 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:02.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.546+0000 7fab4b7fe700 1 --2- 172.21.15.32:0/2776390548 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab4c0af920 0x7fab4c13cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:02.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.546+0000 7fab4bfff700 1 --2- 172.21.15.32:0/2776390548 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab4c05f6f0 0x7fab4c0632e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:02.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.550+0000 7fab50db0700 1 -- 172.21.15.32:0/2776390548 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab4c0af920 msgr2=0x7fab4c13cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:02.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.550+0000 7fab50db0700 1 --2- 172.21.15.32:0/2776390548 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab4c0af920 0x7fab4c13cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:02.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.550+0000 7fab50db0700 1 -- 172.21.15.32:0/2776390548 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab4c05f6f0 msgr2=0x7fab4c0632e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:02.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.550+0000 7fab50db0700 1 --2- 172.21.15.32:0/2776390548 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab4c05f6f0 0x7fab4c0632e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:02.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.550+0000 7fab50db0700 1 -- 172.21.15.32:0/2776390548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4c0b6230 con 0x7fab4c0afea0 2020-12-05T17:37:02.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.550+0000 7fab50db0700 1 --2- 172.21.15.32:0/2776390548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4c0afea0 0x7fab4c064860 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fab44000b80 tx=0x7fab44007b40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:02.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.550+0000 7fab497fa700 1 -- 172.21.15.32:0/2776390548 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab440072e0 con 0x7fab4c0afea0 2020-12-05T17:37:02.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.550+0000 7fab497fa700 1 -- 172.21.15.32:0/2776390548 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab44005520 con 0x7fab4c0afea0 2020-12-05T17:37:02.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.550+0000 7fab515b1700 1 -- 172.21.15.32:0/2776390548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab4c0b66d0 con 0x7fab4c0afea0 2020-12-05T17:37:02.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.550+0000 7fab497fa700 1 -- 172.21.15.32:0/2776390548 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab44005880 con 0x7fab4c0afea0 2020-12-05T17:37:02.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.550+0000 7fab515b1700 1 -- 172.21.15.32:0/2776390548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4c066240 con 0x7fab4c0afea0 2020-12-05T17:37:02.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.550+0000 7fab515b1700 1 -- 172.21.15.32:0/2776390548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab4c057050 con 0x7fab4c0afea0 2020-12-05T17:37:02.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.550+0000 7fab497fa700 1 -- 172.21.15.32:0/2776390548 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fab4401a060 con 0x7fab4c0afea0 2020-12-05T17:37:02.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.550+0000 7fab497fa700 1 --2- 172.21.15.32:0/2776390548 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fab2c0554c0 0x7fab2c057980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:02.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.550+0000 7fab497fa700 1 -- 172.21.15.32:0/2776390548 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fab4406d8b0 con 0x7fab4c0afea0 2020-12-05T17:37:02.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.550+0000 7fab4bfff700 1 --2- 172.21.15.32:0/2776390548 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fab2c0554c0 0x7fab2c057980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:02.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.554+0000 7fab4bfff700 1 --2- 172.21.15.32:0/2776390548 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fab2c0554c0 0x7fab2c057980 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7fab3c00b000 tx=0x7fab3c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:02.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.558+0000 7fab497fa700 1 -- 172.21.15.32:0/2776390548 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fab44043a50 con 0x7fab4c0afea0 2020-12-05T17:37:02.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.742+0000 7fab515b1700 1 -- 172.21.15.32:0/2776390548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab4c0b63c0 con 0x7fab4c0afea0 2020-12-05T17:37:02.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.742+0000 7fab497fa700 1 -- 172.21.15.32:0/2776390548 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+2137 (secure 0 0 0) 0x7fab44018020 con 0x7fab4c0afea0 2020-12-05T17:37:02.746 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:02.746 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5411,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/3995477718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3995477718},{"type":"v1","addr":"172.21.15.61:6833","nonce":3995477718}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:37:01.403820+0000","modified":"2020-12-05T17:37:02.406060+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5976},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5976":{"gid":5976,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/738464306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":738464306},{"type":"v1","addr":"172.21.15.32:6835","nonce":738464306}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-12-05T17:37:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.746+0000 7fab515b1700 1 -- 172.21.15.32:0/2776390548 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fab2c0554c0 msgr2=0x7fab2c057980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.746+0000 7fab515b1700 1 --2- 172.21.15.32:0/2776390548 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fab2c0554c0 0x7fab2c057980 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7fab3c00b000 tx=0x7fab3c008040).stop 2020-12-05T17:37:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.746+0000 7fab515b1700 1 -- 172.21.15.32:0/2776390548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4c0afea0 msgr2=0x7fab4c064860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.746+0000 7fab515b1700 1 --2- 172.21.15.32:0/2776390548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4c0afea0 0x7fab4c064860 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fab44000b80 tx=0x7fab44007b40).stop 2020-12-05T17:37:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.746+0000 7fab515b1700 1 -- 172.21.15.32:0/2776390548 shutdown_connections 2020-12-05T17:37:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.746+0000 7fab515b1700 1 --2- 172.21.15.32:0/2776390548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4c0afea0 0x7fab4c064860 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.746+0000 7fab515b1700 1 --2- 172.21.15.32:0/2776390548 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab4c05f6f0 0x7fab4c0632e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.746+0000 7fab515b1700 1 --2- 172.21.15.32:0/2776390548 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fab2c0554c0 0x7fab2c057980 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.746+0000 7fab515b1700 1 --2- 172.21.15.32:0/2776390548 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab4c0af920 0x7fab4c13cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.746+0000 7fab515b1700 1 -- 172.21.15.32:0/2776390548 >> 172.21.15.32:0/2776390548 conn(0x7fab4c0604c0 msgr2=0x7fab4c141000 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.746+0000 7fab515b1700 1 -- 172.21.15.32:0/2776390548 shutdown_connections 2020-12-05T17:37:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.746+0000 7fab515b1700 1 -- 172.21.15.32:0/2776390548 wait complete. 2020-12-05T17:37:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 27 2020-12-05T17:37:02.760 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:37:02.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.874+0000 7f8a4ab9f700 1 Processor -- start 2020-12-05T17:37:02.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.874+0000 7f8a4ab9f700 1 -- start start 2020-12-05T17:37:02.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.874+0000 7f8a4ab9f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f8a44062400 0x7f8a4405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:02.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.874+0000 7f8a4ab9f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a4405feb0 0x7f8a440b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:02.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.874+0000 7f8a4ab9f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a44142450 con 0x7f8a4405f3b0 2020-12-05T17:37:02.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.874+0000 7f8a4ab9f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a440b42f0 con 0x7f8a44062400 2020-12-05T17:37:02.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.874+0000 7f8a4ab9f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a44061780 con 0x7f8a4405feb0 2020-12-05T17:37:02.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a4a39e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a4405feb0 0x7f8a440b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:02.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a4939c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f8a44062400 0x7f8a4405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:02.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a4939c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f8a44062400 0x7f8a4405f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:55946/0 (socket says 172.21.15.32:55946) 2020-12-05T17:37:02.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a49b9d700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f8a4405f3b0 0x7f8a440b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:51338/0 (socket says 172.21.15.32:51338) 2020-12-05T17:37:02.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a4939c700 1 -- 172.21.15.32:0/2283385134 learned_addr learned my addr 172.21.15.32:0/2283385134 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:02.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a4a39e700 1 -- 172.21.15.32:0/2283385134 >> v2:172.21.15.61:3300/0 conn(0x7f8a44062400 msgr2=0x7f8a4405f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:02.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a4a39e700 1 --2- 172.21.15.32:0/2283385134 >> v2:172.21.15.61:3300/0 conn(0x7f8a44062400 0x7f8a4405f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:02.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a4a39e700 1 -- 172.21.15.32:0/2283385134 >> v1:172.21.15.61:6789/0 conn(0x7f8a4405f3b0 legacy=0x7f8a440b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:02.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a4a39e700 1 -- 172.21.15.32:0/2283385134 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a44061a20 con 0x7f8a4405feb0 2020-12-05T17:37:02.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a4939c700 1 --2- 172.21.15.32:0/2283385134 >> v2:172.21.15.61:3300/0 conn(0x7f8a44062400 0x7f8a4405f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:37:02.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a4a39e700 1 --2- 172.21.15.32:0/2283385134 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a4405feb0 0x7f8a440b3e00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f8a3c0032b0 tx=0x7f8a3c0091a0).ready entity=mon.2 client_cookie=295ef33f0bec3af2 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:02.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a48b9b700 1 -- 172.21.15.32:0/2283385134 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a3c00b9a0 con 0x7f8a4405feb0 2020-12-05T17:37:02.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a48b9b700 1 -- 172.21.15.32:0/2283385134 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3c00bb00 con 0x7f8a4405feb0 2020-12-05T17:37:02.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a48b9b700 1 -- 172.21.15.32:0/2283385134 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a3c00bc70 con 0x7f8a4405feb0 2020-12-05T17:37:02.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a4ab9f700 1 -- 172.21.15.32:0/2283385134 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a4405feb0 msgr2=0x7f8a440b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:02.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a4ab9f700 1 --2- 172.21.15.32:0/2283385134 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a4405feb0 0x7f8a440b3e00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f8a3c0032b0 tx=0x7f8a3c0091a0).stop 2020-12-05T17:37:02.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a4ab9f700 1 -- 172.21.15.32:0/2283385134 shutdown_connections 2020-12-05T17:37:02.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a4ab9f700 1 --2- 172.21.15.32:0/2283385134 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a4405feb0 0x7f8a440b3e00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:02.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a4ab9f700 1 --2- 172.21.15.32:0/2283385134 >> v2:172.21.15.61:3300/0 conn(0x7f8a44062400 0x7f8a4405f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:02.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a4ab9f700 1 -- 172.21.15.32:0/2283385134 >> 172.21.15.32:0/2283385134 conn(0x7f8a440b8ee0 msgr2=0x7f8a440b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:02.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a4ab9f700 1 -- 172.21.15.32:0/2283385134 shutdown_connections 2020-12-05T17:37:02.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a4ab9f700 1 -- 172.21.15.32:0/2283385134 wait complete. 2020-12-05T17:37:02.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a4ab9f700 1 Processor -- start 2020-12-05T17:37:02.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.878+0000 7f8a4ab9f700 1 -- start start 2020-12-05T17:37:02.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.882+0000 7f8a4ab9f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a4405f3b0 0x7f8a44065110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:02.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.882+0000 7f8a4ab9f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a4405feb0 0x7f8a440631a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:02.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.882+0000 7f8a4ab9f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a44062400 0x7f8a44151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:02.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.882+0000 7f8a4ab9f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a44064730 con 0x7f8a44062400 2020-12-05T17:37:02.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.882+0000 7f8a4ab9f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a440649d0 con 0x7f8a4405f3b0 2020-12-05T17:37:02.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.882+0000 7f8a4ab9f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a44064c70 con 0x7f8a4405feb0 2020-12-05T17:37:02.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.882+0000 7f8a4a39e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a44062400 0x7f8a44151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:02.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.882+0000 7f8a4a39e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a44062400 0x7f8a44151450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36060/0 (socket says 172.21.15.32:36060) 2020-12-05T17:37:02.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.882+0000 7f8a4a39e700 1 -- 172.21.15.32:0/3445859038 learned_addr learned my addr 172.21.15.32:0/3445859038 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:02.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.882+0000 7f8a4939c700 1 --2- 172.21.15.32:0/3445859038 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a4405feb0 0x7f8a440631a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:02.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.882+0000 7f8a49b9d700 1 --2- 172.21.15.32:0/3445859038 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a4405f3b0 0x7f8a44065110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:02.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.882+0000 7f8a4a39e700 1 -- 172.21.15.32:0/3445859038 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a4405feb0 msgr2=0x7f8a440631a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:02.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.882+0000 7f8a4a39e700 1 --2- 172.21.15.32:0/3445859038 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a4405feb0 0x7f8a440631a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:02.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.882+0000 7f8a4a39e700 1 -- 172.21.15.32:0/3445859038 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a4405f3b0 msgr2=0x7f8a44065110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:02.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.882+0000 7f8a4a39e700 1 --2- 172.21.15.32:0/3445859038 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a4405f3b0 0x7f8a44065110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:02.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.882+0000 7f8a4a39e700 1 -- 172.21.15.32:0/3445859038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a44063690 con 0x7f8a44062400 2020-12-05T17:37:02.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.882+0000 7f8a4a39e700 1 --2- 172.21.15.32:0/3445859038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a44062400 0x7f8a44151450 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f8a3c0032b0 tx=0x7f8a3c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:02.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.882+0000 7f8a32ffd700 1 -- 172.21.15.32:0/3445859038 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a3c011e60 con 0x7f8a44062400 2020-12-05T17:37:02.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.882+0000 7f8a32ffd700 1 -- 172.21.15.32:0/3445859038 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3c01f3f0 con 0x7f8a44062400 2020-12-05T17:37:02.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.882+0000 7f8a4ab9f700 1 -- 172.21.15.32:0/3445859038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a44144a70 con 0x7f8a44062400 2020-12-05T17:37:02.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.882+0000 7f8a32ffd700 1 -- 172.21.15.32:0/3445859038 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a3c01f750 con 0x7f8a44062400 2020-12-05T17:37:02.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.882+0000 7f8a4ab9f700 1 -- 172.21.15.32:0/3445859038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a44151940 con 0x7f8a44062400 2020-12-05T17:37:02.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.882+0000 7f8a4ab9f700 1 -- 172.21.15.32:0/3445859038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a44151f60 con 0x7f8a44062400 2020-12-05T17:37:02.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.886+0000 7f8a32ffd700 1 -- 172.21.15.32:0/3445859038 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8a3c026030 con 0x7f8a44062400 2020-12-05T17:37:02.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.886+0000 7f8a32ffd700 1 --2- 172.21.15.32:0/3445859038 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8a24055470 0x7f8a24057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:02.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.886+0000 7f8a32ffd700 1 -- 172.21.15.32:0/3445859038 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8a3c06ea20 con 0x7f8a44062400 2020-12-05T17:37:02.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.886+0000 7f8a49b9d700 1 --2- 172.21.15.32:0/3445859038 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8a24055470 0x7f8a24057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:02.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.886+0000 7f8a49b9d700 1 --2- 172.21.15.32:0/3445859038 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8a24055470 0x7f8a24057930 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f8a4000d000 tx=0x7f8a40007b10).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:02.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:02.894+0000 7f8a32ffd700 1 -- 172.21.15.32:0/3445859038 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8a3c044ab0 con 0x7f8a44062400 2020-12-05T17:37:03.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.038+0000 7f8a4ab9f700 1 -- 172.21.15.32:0/3445859038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a44057050 con 0x7f8a44062400 2020-12-05T17:37:03.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.038+0000 7f8a32ffd700 1 -- 172.21.15.32:0/3445859038 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f8a3c024020 con 0x7f8a44062400 2020-12-05T17:37:03.041 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:03.041 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":62,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:37:01.403672+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:36:59.960851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-12-05T17:37:00.401054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:37:03.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.038+0000 7f8a4ab9f700 1 -- 172.21.15.32:0/3445859038 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8a24055470 msgr2=0x7f8a24057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:03.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.038+0000 7f8a4ab9f700 1 --2- 172.21.15.32:0/3445859038 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8a24055470 0x7f8a24057930 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f8a4000d000 tx=0x7f8a40007b10).stop 2020-12-05T17:37:03.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.038+0000 7f8a4ab9f700 1 -- 172.21.15.32:0/3445859038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a44062400 msgr2=0x7f8a44151450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:03.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.038+0000 7f8a4ab9f700 1 --2- 172.21.15.32:0/3445859038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a44062400 0x7f8a44151450 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f8a3c0032b0 tx=0x7f8a3c016000).stop 2020-12-05T17:37:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.042+0000 7f8a4ab9f700 1 -- 172.21.15.32:0/3445859038 shutdown_connections 2020-12-05T17:37:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.042+0000 7f8a4ab9f700 1 --2- 172.21.15.32:0/3445859038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a44062400 0x7f8a44151450 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:03.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.042+0000 7f8a4ab9f700 1 --2- 172.21.15.32:0/3445859038 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a4405f3b0 0x7f8a44065110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:03.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.042+0000 7f8a4ab9f700 1 --2- 172.21.15.32:0/3445859038 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8a24055470 0x7f8a24057930 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:03.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.042+0000 7f8a4ab9f700 1 --2- 172.21.15.32:0/3445859038 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a4405feb0 0x7f8a440631a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:03.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.042+0000 7f8a4ab9f700 1 -- 172.21.15.32:0/3445859038 >> 172.21.15.32:0/3445859038 conn(0x7f8a440b8ee0 msgr2=0x7f8a440b78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:03.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.042+0000 7f8a4ab9f700 1 -- 172.21.15.32:0/3445859038 shutdown_connections 2020-12-05T17:37:03.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.042+0000 7f8a4ab9f700 1 -- 172.21.15.32:0/3445859038 wait complete. 2020-12-05T17:37:03.056 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:37:03.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.178+0000 7f0513095700 1 Processor -- start 2020-12-05T17:37:03.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.178+0000 7f0513095700 1 -- start start 2020-12-05T17:37:03.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f0513095700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f050c0b0240 0x7f050c153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:03.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f0513095700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f050c080a90 con 0x7f050c0ae860 2020-12-05T17:37:03.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f0513095700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f050c0aec40 con 0x7f050c0611b0 2020-12-05T17:37:03.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f0513095700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f050c152860 con 0x7f050c0b0240 2020-12-05T17:37:03.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f05142f7700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f050c0ae860 0x7f050c14f0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38716/0 (socket says 172.21.15.32:38716) 2020-12-05T17:37:03.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f05142f7700 1 -- 172.21.15.32:0/3085342445 learned_addr learned my addr 172.21.15.32:0/3085342445 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:03.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f0512093700 1 --2- 172.21.15.32:0/3085342445 >> v2:172.21.15.61:3300/0 conn(0x7f050c0b0240 0x7f050c153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:03.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f0511892700 1 -- 172.21.15.32:0/3085342445 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2040556122 0 0) 0x7f050c080a90 con 0x7f050c0ae860 2020-12-05T17:37:03.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f0511892700 1 -- 172.21.15.32:0/3085342445 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04f4003230 con 0x7f050c0ae860 2020-12-05T17:37:03.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f0512093700 1 -- 172.21.15.32:0/3085342445 >> v1:172.21.15.61:6789/0 conn(0x7f050c0611b0 legacy=0x7f050c14c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:03.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f0511892700 1 -- 172.21.15.32:0/3085342445 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2852657312 0 0) 0x7f050c0aec40 con 0x7f050c0611b0 2020-12-05T17:37:03.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f0512093700 1 -- 172.21.15.32:0/3085342445 >> v1:172.21.15.32:6789/0 conn(0x7f050c0ae860 legacy=0x7f050c14f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:03.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f0512093700 1 -- 172.21.15.32:0/3085342445 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f050c152b00 con 0x7f050c0b0240 2020-12-05T17:37:03.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f0512093700 1 --2- 172.21.15.32:0/3085342445 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f050c0b0240 0x7f050c153240 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f0500002f30 tx=0x7f0500006dc0).ready entity=mon.1 client_cookie=7005d555dddc6803 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:03.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f0511892700 1 -- 172.21.15.32:0/3085342445 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0500009800 con 0x7f050c0b0240 2020-12-05T17:37:03.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f0511892700 1 -- 172.21.15.32:0/3085342445 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0500009960 con 0x7f050c0b0240 2020-12-05T17:37:03.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f0511892700 1 -- 172.21.15.32:0/3085342445 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0500009cc0 con 0x7f050c0b0240 2020-12-05T17:37:03.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f0513095700 1 -- 172.21.15.32:0/3085342445 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f050c0b0240 msgr2=0x7f050c153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:03.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f0513095700 1 --2- 172.21.15.32:0/3085342445 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f050c0b0240 0x7f050c153240 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f0500002f30 tx=0x7f0500006dc0).stop 2020-12-05T17:37:03.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f0513095700 1 -- 172.21.15.32:0/3085342445 shutdown_connections 2020-12-05T17:37:03.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f0513095700 1 --2- 172.21.15.32:0/3085342445 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f050c0b0240 0x7f050c153240 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:03.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f0513095700 1 -- 172.21.15.32:0/3085342445 >> 172.21.15.32:0/3085342445 conn(0x7f050c0b5f90 msgr2=0x7f050c0b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:03.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f0513095700 1 -- 172.21.15.32:0/3085342445 shutdown_connections 2020-12-05T17:37:03.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.182+0000 7f0513095700 1 -- 172.21.15.32:0/3085342445 wait complete. 2020-12-05T17:37:03.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.186+0000 7f0513095700 1 Processor -- start 2020-12-05T17:37:03.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.186+0000 7f0513095700 1 -- start start 2020-12-05T17:37:03.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.186+0000 7f0513095700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f050c0611b0 0x7f050c14e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:03.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.186+0000 7f0513095700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f050c0ae860 0x7f050c14e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:03.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.186+0000 7f0513095700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f050c0b0240 0x7f050c063780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:03.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.186+0000 7f0513095700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c14d4c0 con 0x7f050c0b0240 2020-12-05T17:37:03.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.186+0000 7f0513095700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c14ee50 con 0x7f050c0ae860 2020-12-05T17:37:03.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.186+0000 7f0513095700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050c14f0f0 con 0x7f050c0611b0 2020-12-05T17:37:03.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.186+0000 7f05142f7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f050c0b0240 0x7f050c063780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:03.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.186+0000 7f05142f7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f050c0b0240 0x7f050c063780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36070/0 (socket says 172.21.15.32:36070) 2020-12-05T17:37:03.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.186+0000 7f05142f7700 1 -- 172.21.15.32:0/1508831999 learned_addr learned my addr 172.21.15.32:0/1508831999 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:03.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.186+0000 7f0512093700 1 --2- 172.21.15.32:0/1508831999 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f050c0ae860 0x7f050c14e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:03.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.186+0000 7f0512894700 1 --2- 172.21.15.32:0/1508831999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f050c0611b0 0x7f050c14e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:03.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.186+0000 7f05142f7700 1 -- 172.21.15.32:0/1508831999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f050c0611b0 msgr2=0x7f050c14e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:03.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.190+0000 7f05142f7700 1 --2- 172.21.15.32:0/1508831999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f050c0611b0 0x7f050c14e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:03.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.190+0000 7f05142f7700 1 -- 172.21.15.32:0/1508831999 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f050c0ae860 msgr2=0x7f050c14e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:03.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.190+0000 7f05142f7700 1 --2- 172.21.15.32:0/1508831999 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f050c0ae860 0x7f050c14e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:03.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.190+0000 7f05142f7700 1 -- 172.21.15.32:0/1508831999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f050c14f390 con 0x7f050c0b0240 2020-12-05T17:37:03.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.190+0000 7f05142f7700 1 --2- 172.21.15.32:0/1508831999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f050c0b0240 0x7f050c063780 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f05040010d0 tx=0x7f05040079f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.190+0000 7f050b7fe700 1 -- 172.21.15.32:0/1508831999 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05040072e0 con 0x7f050c0b0240 2020-12-05T17:37:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.190+0000 7f050b7fe700 1 -- 172.21.15.32:0/1508831999 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0504007440 con 0x7f050c0b0240 2020-12-05T17:37:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.190+0000 7f0513095700 1 -- 172.21.15.32:0/1508831999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f050c066fa0 con 0x7f050c0b0240 2020-12-05T17:37:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.190+0000 7f050b7fe700 1 -- 172.21.15.32:0/1508831999 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05040057d0 con 0x7f050c0b0240 2020-12-05T17:37:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.190+0000 7f0513095700 1 -- 172.21.15.32:0/1508831999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f050c14cd70 con 0x7f050c0b0240 2020-12-05T17:37:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.190+0000 7f0513095700 1 -- 172.21.15.32:0/1508831999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f050c14eca0 con 0x7f050c0b0240 2020-12-05T17:37:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.190+0000 7f050b7fe700 1 -- 172.21.15.32:0/1508831999 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0504010070 con 0x7f050c0b0240 2020-12-05T17:37:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.190+0000 7f050b7fe700 1 --2- 172.21.15.32:0/1508831999 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f04ec055440 0x7f04ec057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.190+0000 7f050b7fe700 1 -- 172.21.15.32:0/1508831999 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f050406d680 con 0x7f050c0b0240 2020-12-05T17:37:03.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.190+0000 7f0512894700 1 --2- 172.21.15.32:0/1508831999 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f04ec055440 0x7f04ec057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:03.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.194+0000 7f0512894700 1 --2- 172.21.15.32:0/1508831999 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f04ec055440 0x7f04ec057900 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f04fc00d000 tx=0x7f04fc009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.202+0000 7f050b7fe700 1 -- 172.21.15.32:0/1508831999 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0504043780 con 0x7f050c0b0240 2020-12-05T17:37:03.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.426+0000 7f0513095700 1 -- 172.21.15.32:0/1508831999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f050c057050 con 0x7f050c0b0240 2020-12-05T17:37:03.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.430+0000 7f050b7fe700 1 -- 172.21.15.32:0/1508831999 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v20) v1 ==== 184+0+0 (secure 0 0 0) 0x7f0504018070 con 0x7f050c0b0240 2020-12-05T17:37:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.434+0000 7f0513095700 1 -- 172.21.15.32:0/1508831999 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f04ec055440 msgr2=0x7f04ec057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.434+0000 7f0513095700 1 --2- 172.21.15.32:0/1508831999 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f04ec055440 0x7f04ec057900 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f04fc00d000 tx=0x7f04fc009040).stop 2020-12-05T17:37:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.434+0000 7f0513095700 1 -- 172.21.15.32:0/1508831999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f050c0b0240 msgr2=0x7f050c063780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.434+0000 7f0513095700 1 --2- 172.21.15.32:0/1508831999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f050c0b0240 0x7f050c063780 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f05040010d0 tx=0x7f05040079f0).stop 2020-12-05T17:37:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.434+0000 7f0513095700 1 -- 172.21.15.32:0/1508831999 shutdown_connections 2020-12-05T17:37:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.434+0000 7f0513095700 1 --2- 172.21.15.32:0/1508831999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f050c0b0240 0x7f050c063780 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.434+0000 7f0513095700 1 --2- 172.21.15.32:0/1508831999 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f04ec055440 0x7f04ec057900 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.434+0000 7f0513095700 1 --2- 172.21.15.32:0/1508831999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f050c0611b0 0x7f050c14e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.434+0000 7f0513095700 1 --2- 172.21.15.32:0/1508831999 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f050c0ae860 0x7f050c14e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.434+0000 7f0513095700 1 -- 172.21.15.32:0/1508831999 >> 172.21.15.32:0/1508831999 conn(0x7f050c0b5f90 msgr2=0x7f050c0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.434+0000 7f0513095700 1 -- 172.21.15.32:0/1508831999 shutdown_connections 2020-12-05T17:37:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.434+0000 7f0513095700 1 -- 172.21.15.32:0/1508831999 wait complete. 2020-12-05T17:37:03.439 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2020-12-05T17:37:03.452 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:37:03.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.570+0000 7f69189e8700 1 Processor -- start 2020-12-05T17:37:03.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.570+0000 7f69189e8700 1 -- start start 2020-12-05T17:37:03.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.570+0000 7f69189e8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f69140b0240 0x7f6914153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:03.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.570+0000 7f69189e8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6914080a90 con 0x7f69140ae860 2020-12-05T17:37:03.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.570+0000 7f69189e8700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69140aec40 con 0x7f69140611b0 2020-12-05T17:37:03.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.570+0000 7f69189e8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6914152860 con 0x7f69140b0240 2020-12-05T17:37:03.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.570+0000 7f6919c4a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f69140ae860 0x7f691414f0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38730/0 (socket says 172.21.15.32:38730) 2020-12-05T17:37:03.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.570+0000 7f69137fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f69140b0240 0x7f6914153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:03.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.570+0000 7f6919c4a700 1 -- 172.21.15.32:0/2603185642 learned_addr learned my addr 172.21.15.32:0/2603185642 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:03.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.570+0000 7f69137fe700 1 -- 172.21.15.32:0/2603185642 >> v1:172.21.15.61:6789/0 conn(0x7f69140611b0 legacy=0x7f691414c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:03.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.570+0000 7f6912ffd700 1 -- 172.21.15.32:0/2603185642 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1793251472 0 0) 0x7f6914080a90 con 0x7f69140ae860 2020-12-05T17:37:03.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.570+0000 7f69137fe700 1 -- 172.21.15.32:0/2603185642 >> v1:172.21.15.32:6789/0 conn(0x7f69140ae860 legacy=0x7f691414f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:03.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.570+0000 7f69137fe700 1 -- 172.21.15.32:0/2603185642 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6914152b00 con 0x7f69140b0240 2020-12-05T17:37:03.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.570+0000 7f69137fe700 1 --2- 172.21.15.32:0/2603185642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69140b0240 0x7f6914153240 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f6908002f30 tx=0x7f6908006dc0).ready entity=mon.0 client_cookie=d2bbcd35044ed8f8 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.570+0000 7f6912ffd700 1 -- 172.21.15.32:0/2603185642 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69080097f0 con 0x7f69140b0240 2020-12-05T17:37:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.570+0000 7f6912ffd700 1 -- 172.21.15.32:0/2603185642 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6908009950 con 0x7f69140b0240 2020-12-05T17:37:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.570+0000 7f6912ffd700 1 -- 172.21.15.32:0/2603185642 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6908009cb0 con 0x7f69140b0240 2020-12-05T17:37:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.570+0000 7f69189e8700 1 -- 172.21.15.32:0/2603185642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69140b0240 msgr2=0x7f6914153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.570+0000 7f69189e8700 1 --2- 172.21.15.32:0/2603185642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69140b0240 0x7f6914153240 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f6908002f30 tx=0x7f6908006dc0).stop 2020-12-05T17:37:03.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f69189e8700 1 -- 172.21.15.32:0/2603185642 shutdown_connections 2020-12-05T17:37:03.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f69189e8700 1 --2- 172.21.15.32:0/2603185642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69140b0240 0x7f6914153240 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:03.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f69189e8700 1 -- 172.21.15.32:0/2603185642 >> 172.21.15.32:0/2603185642 conn(0x7f69140b5f90 msgr2=0x7f69140b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:03.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f69189e8700 1 -- 172.21.15.32:0/2603185642 shutdown_connections 2020-12-05T17:37:03.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f69189e8700 1 -- 172.21.15.32:0/2603185642 wait complete. 2020-12-05T17:37:03.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f69189e8700 1 Processor -- start 2020-12-05T17:37:03.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f69189e8700 1 -- start start 2020-12-05T17:37:03.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f69189e8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69140611b0 0x7f691412a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:03.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f69189e8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69140ae860 0x7f69140b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:03.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f69189e8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f69140b0240 0x7f691414fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:03.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f69189e8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69140b1730 con 0x7f69140611b0 2020-12-05T17:37:03.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f69189e8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69140b1960 con 0x7f69140ae860 2020-12-05T17:37:03.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f69189e8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f691413a180 con 0x7f69140b0240 2020-12-05T17:37:03.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f6913fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69140611b0 0x7f691412a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:03.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f6913fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69140611b0 0x7f691412a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36084/0 (socket says 172.21.15.32:36084) 2020-12-05T17:37:03.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f6913fff700 1 -- 172.21.15.32:0/3199919128 learned_addr learned my addr 172.21.15.32:0/3199919128 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:03.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f69137fe700 1 --2- 172.21.15.32:0/3199919128 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69140ae860 0x7f69140b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:03.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f6919c4a700 1 --2- 172.21.15.32:0/3199919128 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f69140b0240 0x7f691414fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:03.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f6913fff700 1 -- 172.21.15.32:0/3199919128 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f69140b0240 msgr2=0x7f691414fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:03.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f6913fff700 1 --2- 172.21.15.32:0/3199919128 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f69140b0240 0x7f691414fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:03.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f6913fff700 1 -- 172.21.15.32:0/3199919128 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69140ae860 msgr2=0x7f69140b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:03.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f6913fff700 1 --2- 172.21.15.32:0/3199919128 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69140ae860 0x7f69140b0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:03.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f6913fff700 1 -- 172.21.15.32:0/3199919128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6914139eb0 con 0x7f69140611b0 2020-12-05T17:37:03.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.574+0000 7f6913fff700 1 --2- 172.21.15.32:0/3199919128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69140611b0 0x7f691412a7f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f6904001f60 tx=0x7f6904007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:03.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.578+0000 7f6910ff9700 1 -- 172.21.15.32:0/3199919128 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f690400e4a0 con 0x7f69140611b0 2020-12-05T17:37:03.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.578+0000 7f6910ff9700 1 -- 172.21.15.32:0/3199919128 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690400e600 con 0x7f69140611b0 2020-12-05T17:37:03.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.578+0000 7f6910ff9700 1 -- 172.21.15.32:0/3199919128 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f690400e960 con 0x7f69140611b0 2020-12-05T17:37:03.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.578+0000 7f69189e8700 1 -- 172.21.15.32:0/3199919128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69140b2560 con 0x7f69140611b0 2020-12-05T17:37:03.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.578+0000 7f69189e8700 1 -- 172.21.15.32:0/3199919128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69140b2e90 con 0x7f69140611b0 2020-12-05T17:37:03.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.578+0000 7f69189e8700 1 -- 172.21.15.32:0/3199919128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f691413a340 con 0x7f69140611b0 2020-12-05T17:37:03.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.578+0000 7f6910ff9700 1 -- 172.21.15.32:0/3199919128 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6904010030 con 0x7f69140611b0 2020-12-05T17:37:03.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.578+0000 7f6910ff9700 1 --2- 172.21.15.32:0/3199919128 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f68f4055350 0x7f68f4057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:03.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.578+0000 7f6910ff9700 1 -- 172.21.15.32:0/3199919128 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f690406db40 con 0x7f69140611b0 2020-12-05T17:37:03.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.578+0000 7f69137fe700 1 --2- 172.21.15.32:0/3199919128 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f68f4055350 0x7f68f4057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:03.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.582+0000 7f69137fe700 1 --2- 172.21.15.32:0/3199919128 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f68f4055350 0x7f68f4057810 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f690801b000 tx=0x7f6908015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:03.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.586+0000 7f6910ff9700 1 -- 172.21.15.32:0/3199919128 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6904043bd0 con 0x7f69140611b0 2020-12-05T17:37:03.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.778+0000 7f69189e8700 1 -- 172.21.15.32:0/3199919128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f6914057050 con 0x7f69140611b0 2020-12-05T17:37:03.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.786+0000 7f6910ff9700 1 -- 172.21.15.32:0/3199919128 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v21) v1 ==== 184+0+0 (secure 0 0 0) 0x7f69040471f0 con 0x7f69140611b0 2020-12-05T17:37:03.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.786+0000 7f69189e8700 1 -- 172.21.15.32:0/3199919128 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f68f4055350 msgr2=0x7f68f4057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:03.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.786+0000 7f69189e8700 1 --2- 172.21.15.32:0/3199919128 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f68f4055350 0x7f68f4057810 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f690801b000 tx=0x7f6908015000).stop 2020-12-05T17:37:03.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.786+0000 7f69189e8700 1 -- 172.21.15.32:0/3199919128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69140611b0 msgr2=0x7f691412a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:03.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.786+0000 7f69189e8700 1 --2- 172.21.15.32:0/3199919128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69140611b0 0x7f691412a7f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f6904001f60 tx=0x7f6904007c00).stop 2020-12-05T17:37:03.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.786+0000 7f69189e8700 1 -- 172.21.15.32:0/3199919128 shutdown_connections 2020-12-05T17:37:03.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.786+0000 7f69189e8700 1 --2- 172.21.15.32:0/3199919128 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f68f4055350 0x7f68f4057810 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:03.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.786+0000 7f69189e8700 1 --2- 172.21.15.32:0/3199919128 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f69140b0240 0x7f691414fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:03.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.786+0000 7f69189e8700 1 --2- 172.21.15.32:0/3199919128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69140611b0 0x7f691412a7f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:03.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.786+0000 7f69189e8700 1 --2- 172.21.15.32:0/3199919128 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69140ae860 0x7f69140b0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:03.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.786+0000 7f69189e8700 1 -- 172.21.15.32:0/3199919128 >> 172.21.15.32:0/3199919128 conn(0x7f69140b5f90 msgr2=0x7f69140aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:03.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.786+0000 7f69189e8700 1 -- 172.21.15.32:0/3199919128 shutdown_connections 2020-12-05T17:37:03.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.786+0000 7f69189e8700 1 -- 172.21.15.32:0/3199919128 wait complete. 2020-12-05T17:37:03.792 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2020-12-05T17:37:03.804 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:37:03.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.930+0000 7f247bfff700 1 Processor -- start 2020-12-05T17:37:03.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.930+0000 7f247bfff700 1 -- start start 2020-12-05T17:37:03.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.930+0000 7f247bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f247c0611b0 0x7f247c153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:03.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.930+0000 7f247bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f247c0b0240 0x7f247c0ae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:03.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.930+0000 7f247bfff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f247c080a90 con 0x7f247c14c340 2020-12-05T17:37:03.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.930+0000 7f247bfff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f247c152860 con 0x7f247c0b0240 2020-12-05T17:37:03.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.930+0000 7f247bfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f247c152b00 con 0x7f247c0611b0 2020-12-05T17:37:03.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.930+0000 7f248133e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f247c14c340 0x7f247c14f0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38744/0 (socket says 172.21.15.32:38744) 2020-12-05T17:37:03.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.930+0000 7f248133e700 1 -- 172.21.15.32:0/1427552419 learned_addr learned my addr 172.21.15.32:0/1427552419 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:03.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.930+0000 7f2472ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f247c0b0240 0x7f247c0ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:03.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.930+0000 7f247b7fe700 1 --2- 172.21.15.32:0/1427552419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f247c0611b0 0x7f247c153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:03.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.930+0000 7f2472ffd700 1 -- 172.21.15.32:0/1427552419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f247c0611b0 msgr2=0x7f247c153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:03.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.930+0000 7f2472ffd700 1 --2- 172.21.15.32:0/1427552419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f247c0611b0 0x7f247c153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:03.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.930+0000 7f2472ffd700 1 -- 172.21.15.32:0/1427552419 >> v1:172.21.15.32:6789/0 conn(0x7f247c14c340 legacy=0x7f247c14f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:03.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.930+0000 7f2472ffd700 1 -- 172.21.15.32:0/1427552419 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247c152da0 con 0x7f247c0b0240 2020-12-05T17:37:03.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.930+0000 7f247b7fe700 1 --2- 172.21.15.32:0/1427552419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f247c0611b0 0x7f247c153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:03.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.930+0000 7f2472ffd700 1 --2- 172.21.15.32:0/1427552419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247c0b0240 0x7f247c0ae860 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f2464002f30 tx=0x7f2464006dc0).ready entity=mon.0 client_cookie=3b4a2c87183af5c9 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:03.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.930+0000 7f247affd700 1 -- 172.21.15.32:0/1427552419 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24640097f0 con 0x7f247c0b0240 2020-12-05T17:37:03.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.930+0000 7f247affd700 1 -- 172.21.15.32:0/1427552419 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2464009950 con 0x7f247c0b0240 2020-12-05T17:37:03.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.930+0000 7f247affd700 1 -- 172.21.15.32:0/1427552419 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2464009cb0 con 0x7f247c0b0240 2020-12-05T17:37:03.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.930+0000 7f247bfff700 1 -- 172.21.15.32:0/1427552419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247c0b0240 msgr2=0x7f247c0ae860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:03.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.930+0000 7f247bfff700 1 --2- 172.21.15.32:0/1427552419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247c0b0240 0x7f247c0ae860 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f2464002f30 tx=0x7f2464006dc0).stop 2020-12-05T17:37:03.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.934+0000 7f247bfff700 1 -- 172.21.15.32:0/1427552419 shutdown_connections 2020-12-05T17:37:03.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.934+0000 7f247bfff700 1 --2- 172.21.15.32:0/1427552419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f247c0611b0 0x7f247c153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:03.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.934+0000 7f247bfff700 1 --2- 172.21.15.32:0/1427552419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247c0b0240 0x7f247c0ae860 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:03.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.934+0000 7f247bfff700 1 -- 172.21.15.32:0/1427552419 >> 172.21.15.32:0/1427552419 conn(0x7f247c0b5f90 msgr2=0x7f247c0b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:03.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.934+0000 7f247bfff700 1 -- 172.21.15.32:0/1427552419 shutdown_connections 2020-12-05T17:37:03.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.934+0000 7f247bfff700 1 -- 172.21.15.32:0/1427552419 wait complete. 2020-12-05T17:37:03.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.934+0000 7f247bfff700 1 Processor -- start 2020-12-05T17:37:03.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.934+0000 7f247bfff700 1 -- start start 2020-12-05T17:37:03.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.934+0000 7f247bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f247c0611b0 0x7f247c14ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:03.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.934+0000 7f247bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f247c0b0240 0x7f247c14b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:03.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.934+0000 7f247bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247c14c340 0x7f247c154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:03.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.934+0000 7f247bfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247c159700 con 0x7f247c14c340 2020-12-05T17:37:03.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.934+0000 7f247bfff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247c159460 con 0x7f247c0b0240 2020-12-05T17:37:03.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.934+0000 7f247bfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f247c0669d0 con 0x7f247c0611b0 2020-12-05T17:37:03.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.934+0000 7f248133e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247c14c340 0x7f247c154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:03.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.934+0000 7f248133e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247c14c340 0x7f247c154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36102/0 (socket says 172.21.15.32:36102) 2020-12-05T17:37:03.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.934+0000 7f248133e700 1 -- 172.21.15.32:0/519644028 learned_addr learned my addr 172.21.15.32:0/519644028 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:03.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.934+0000 7f2472ffd700 1 --2- 172.21.15.32:0/519644028 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f247c0b0240 0x7f247c14b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:03.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.934+0000 7f247b7fe700 1 --2- 172.21.15.32:0/519644028 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f247c0611b0 0x7f247c14ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:03.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.938+0000 7f248133e700 1 -- 172.21.15.32:0/519644028 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f247c0611b0 msgr2=0x7f247c14ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:03.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.938+0000 7f248133e700 1 --2- 172.21.15.32:0/519644028 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f247c0611b0 0x7f247c14ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:03.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.938+0000 7f248133e700 1 -- 172.21.15.32:0/519644028 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f247c0b0240 msgr2=0x7f247c14b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:03.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.938+0000 7f248133e700 1 --2- 172.21.15.32:0/519644028 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f247c0b0240 0x7f247c14b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:03.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.938+0000 7f248133e700 1 -- 172.21.15.32:0/519644028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247c066e70 con 0x7f247c14c340 2020-12-05T17:37:03.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.938+0000 7f248133e700 1 --2- 172.21.15.32:0/519644028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247c14c340 0x7f247c154c00 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f247c060810 tx=0x7f2474007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:03.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.938+0000 7f2478ff9700 1 -- 172.21.15.32:0/519644028 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2474005b70 con 0x7f247c14c340 2020-12-05T17:37:03.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.938+0000 7f2478ff9700 1 -- 172.21.15.32:0/519644028 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2474005cd0 con 0x7f247c14c340 2020-12-05T17:37:03.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.938+0000 7f247bfff700 1 -- 172.21.15.32:0/519644028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f247c064b70 con 0x7f247c14c340 2020-12-05T17:37:03.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.938+0000 7f2478ff9700 1 -- 172.21.15.32:0/519644028 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2474007300 con 0x7f247c14c340 2020-12-05T17:37:03.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.938+0000 7f247bfff700 1 -- 172.21.15.32:0/519644028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f247c14bed0 con 0x7f247c14c340 2020-12-05T17:37:03.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.938+0000 7f247bfff700 1 -- 172.21.15.32:0/519644028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f247c14c1b0 con 0x7f247c14c340 2020-12-05T17:37:03.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.938+0000 7f2478ff9700 1 -- 172.21.15.32:0/519644028 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2474010030 con 0x7f247c14c340 2020-12-05T17:37:03.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.938+0000 7f2478ff9700 1 --2- 172.21.15.32:0/519644028 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f24600553b0 0x7f2460057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:03.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.938+0000 7f2478ff9700 1 -- 172.21.15.32:0/519644028 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f247406df00 con 0x7f247c14c340 2020-12-05T17:37:03.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.938+0000 7f247b7fe700 1 --2- 172.21.15.32:0/519644028 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f24600553b0 0x7f2460057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:03.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.942+0000 7f247b7fe700 1 --2- 172.21.15.32:0/519644028 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f24600553b0 0x7f2460057870 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f246c00c000 tx=0x7f246c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:03.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:03.946+0000 7f2478ff9700 1 -- 172.21.15.32:0/519644028 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f24740440a0 con 0x7f247c14c340 2020-12-05T17:37:04.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.138+0000 7f247bfff700 1 -- 172.21.15.32:0/519644028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f247c0651c0 con 0x7f247c14c340 2020-12-05T17:37:04.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.146+0000 7f2478ff9700 1 -- 172.21.15.32:0/519644028 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v22) v1 ==== 184+0+0 (secure 0 0 0) 0x7f24740476c0 con 0x7f247c14c340 2020-12-05T17:37:04.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.146+0000 7f247bfff700 1 -- 172.21.15.32:0/519644028 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f24600553b0 msgr2=0x7f2460057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:04.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.146+0000 7f247bfff700 1 --2- 172.21.15.32:0/519644028 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f24600553b0 0x7f2460057870 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f246c00c000 tx=0x7f246c008040).stop 2020-12-05T17:37:04.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.146+0000 7f247bfff700 1 -- 172.21.15.32:0/519644028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247c14c340 msgr2=0x7f247c154c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.146+0000 7f247bfff700 1 --2- 172.21.15.32:0/519644028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247c14c340 0x7f247c154c00 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f247c060810 tx=0x7f2474007f30).stop 2020-12-05T17:37:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.146+0000 7f247bfff700 1 -- 172.21.15.32:0/519644028 shutdown_connections 2020-12-05T17:37:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.146+0000 7f247bfff700 1 --2- 172.21.15.32:0/519644028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247c14c340 0x7f247c154c00 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.146+0000 7f247bfff700 1 --2- 172.21.15.32:0/519644028 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f24600553b0 0x7f2460057870 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.146+0000 7f247bfff700 1 --2- 172.21.15.32:0/519644028 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f247c0611b0 0x7f247c14ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:04.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.146+0000 7f247bfff700 1 --2- 172.21.15.32:0/519644028 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f247c0b0240 0x7f247c14b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:04.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.146+0000 7f247bfff700 1 -- 172.21.15.32:0/519644028 >> 172.21.15.32:0/519644028 conn(0x7f247c0b5f90 msgr2=0x7f247c0b2940 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:04.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.146+0000 7f247bfff700 1 -- 172.21.15.32:0/519644028 shutdown_connections 2020-12-05T17:37:04.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.146+0000 7f247bfff700 1 -- 172.21.15.32:0/519644028 wait complete. 2020-12-05T17:37:04.153 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.2 2020-12-05T17:37:04.163 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:37:04.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.274+0000 7fe26e5b8700 1 Processor -- start 2020-12-05T17:37:04.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.274+0000 7fe26e5b8700 1 -- start start 2020-12-05T17:37:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.274+0000 7fe26e5b8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2680af3d0 0x7fe268138e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:04.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.274+0000 7fe26e5b8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe26813f4f0 0x7fe26813f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:04.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.274+0000 7fe26e5b8700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2680af7b0 con 0x7fe268063040 2020-12-05T17:37:04.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.274+0000 7fe26e5b8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe268066da0 con 0x7fe26813f4f0 2020-12-05T17:37:04.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.274+0000 7fe26e5b8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe268067040 con 0x7fe2680af3d0 2020-12-05T17:37:04.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.274+0000 7fe26ddb7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe26813f4f0 0x7fe26813f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:04.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.274+0000 7fe26d5b6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2680af3d0 0x7fe268138e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:04.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.274+0000 7fe26ddb7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe26813f4f0 0x7fe26813f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36110/0 (socket says 172.21.15.32:36110) 2020-12-05T17:37:04.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.274+0000 7fe26ddb7700 1 -- 172.21.15.32:0/3934283971 learned_addr learned my addr 172.21.15.32:0/3934283971 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.274+0000 7fe26ddb7700 1 -- 172.21.15.32:0/3934283971 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2680af3d0 msgr2=0x7fe268138e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.274+0000 7fe26ddb7700 1 --2- 172.21.15.32:0/3934283971 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2680af3d0 0x7fe268138e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.274+0000 7fe26ddb7700 1 -- 172.21.15.32:0/3934283971 >> v1:172.21.15.61:6789/0 conn(0x7fe268063040 legacy=0x7fe268139370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.274+0000 7fe26ddb7700 1 -- 172.21.15.32:0/3934283971 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2680b2dc0 con 0x7fe26813f4f0 2020-12-05T17:37:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.274+0000 7fe257fff700 1 -- 172.21.15.32:0/3934283971 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1975161226 0 0) 0x7fe2680af7b0 con 0x7fe268063040 2020-12-05T17:37:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.274+0000 7fe26ddb7700 1 --2- 172.21.15.32:0/3934283971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe26813f4f0 0x7fe26813f910 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7fe260002f30 tx=0x7fe260006e80).ready entity=mon.0 client_cookie=1cf02de044e783bb server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:04.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.274+0000 7fe257fff700 1 -- 172.21.15.32:0/3934283971 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2600097f0 con 0x7fe26813f4f0 2020-12-05T17:37:04.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.274+0000 7fe257fff700 1 -- 172.21.15.32:0/3934283971 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe260009950 con 0x7fe26813f4f0 2020-12-05T17:37:04.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.274+0000 7fe257fff700 1 -- 172.21.15.32:0/3934283971 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe260009cb0 con 0x7fe26813f4f0 2020-12-05T17:37:04.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.278+0000 7fe26e5b8700 1 -- 172.21.15.32:0/3934283971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe26813f4f0 msgr2=0x7fe26813f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:04.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.278+0000 7fe26e5b8700 1 --2- 172.21.15.32:0/3934283971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe26813f4f0 0x7fe26813f910 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7fe260002f30 tx=0x7fe260006e80).stop 2020-12-05T17:37:04.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.278+0000 7fe26e5b8700 1 -- 172.21.15.32:0/3934283971 shutdown_connections 2020-12-05T17:37:04.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.278+0000 7fe26e5b8700 1 --2- 172.21.15.32:0/3934283971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe26813f4f0 0x7fe26813f910 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:04.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.278+0000 7fe26e5b8700 1 --2- 172.21.15.32:0/3934283971 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2680af3d0 0x7fe268138e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:04.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.278+0000 7fe26e5b8700 1 -- 172.21.15.32:0/3934283971 >> 172.21.15.32:0/3934283971 conn(0x7fe26805bbb0 msgr2=0x7fe268062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:04.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.278+0000 7fe26e5b8700 1 -- 172.21.15.32:0/3934283971 shutdown_connections 2020-12-05T17:37:04.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.278+0000 7fe26e5b8700 1 -- 172.21.15.32:0/3934283971 wait complete. 2020-12-05T17:37:04.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.278+0000 7fe26e5b8700 1 Processor -- start 2020-12-05T17:37:04.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.278+0000 7fe26e5b8700 1 -- start start 2020-12-05T17:37:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe26e5b8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe268063040 0x7fe2680ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe26e5b8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2680af3d0 0x7fe268152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe26e5b8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe26813f4f0 0x7fe268152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe26e5b8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe268142750 con 0x7fe268063040 2020-12-05T17:37:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe26e5b8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe268140e60 con 0x7fe2680af3d0 2020-12-05T17:37:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe26e5b8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe268141020 con 0x7fe26813f4f0 2020-12-05T17:37:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe26d5b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe268063040 0x7fe2680ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:04.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe26d5b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe268063040 0x7fe2680ade50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36112/0 (socket says 172.21.15.32:36112) 2020-12-05T17:37:04.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe26d5b6700 1 -- 172.21.15.32:0/2678757783 learned_addr learned my addr 172.21.15.32:0/2678757783 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:04.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe26cdb5700 1 --2- 172.21.15.32:0/2678757783 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2680af3d0 0x7fe268152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:04.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe26ddb7700 1 --2- 172.21.15.32:0/2678757783 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe26813f4f0 0x7fe268152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:04.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe26d5b6700 1 -- 172.21.15.32:0/2678757783 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe26813f4f0 msgr2=0x7fe268152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:04.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe26d5b6700 1 --2- 172.21.15.32:0/2678757783 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe26813f4f0 0x7fe268152e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:04.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe26d5b6700 1 -- 172.21.15.32:0/2678757783 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2680af3d0 msgr2=0x7fe268152920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:04.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe26d5b6700 1 --2- 172.21.15.32:0/2678757783 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2680af3d0 0x7fe268152920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:04.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe26d5b6700 1 -- 172.21.15.32:0/2678757783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe268141650 con 0x7fe268063040 2020-12-05T17:37:04.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe26d5b6700 1 --2- 172.21.15.32:0/2678757783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe268063040 0x7fe2680ade50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7fe2681411b0 tx=0x7fe258006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:04.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe2567fc700 1 -- 172.21.15.32:0/2678757783 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe25800ca10 con 0x7fe268063040 2020-12-05T17:37:04.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe2567fc700 1 -- 172.21.15.32:0/2678757783 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe25800cb70 con 0x7fe268063040 2020-12-05T17:37:04.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe26e5b8700 1 -- 172.21.15.32:0/2678757783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe268153300 con 0x7fe268063040 2020-12-05T17:37:04.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe2567fc700 1 -- 172.21.15.32:0/2678757783 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2580123f0 con 0x7fe268063040 2020-12-05T17:37:04.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe26e5b8700 1 -- 172.21.15.32:0/2678757783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe26814f230 con 0x7fe268063040 2020-12-05T17:37:04.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe26e5b8700 1 -- 172.21.15.32:0/2678757783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe268152640 con 0x7fe268063040 2020-12-05T17:37:04.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.282+0000 7fe2567fc700 1 -- 172.21.15.32:0/2678757783 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe258012550 con 0x7fe268063040 2020-12-05T17:37:04.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.286+0000 7fe2567fc700 1 --2- 172.21.15.32:0/2678757783 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe248055500 0x7fe2480579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:04.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.286+0000 7fe2567fc700 1 -- 172.21.15.32:0/2678757783 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe25806e410 con 0x7fe268063040 2020-12-05T17:37:04.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.286+0000 7fe26cdb5700 1 --2- 172.21.15.32:0/2678757783 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe248055500 0x7fe2480579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:04.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.286+0000 7fe26cdb5700 1 --2- 172.21.15.32:0/2678757783 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe248055500 0x7fe2480579c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7fe26400d000 tx=0x7fe264007d90).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:04.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.290+0000 7fe2567fc700 1 -- 172.21.15.32:0/2678757783 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe2580444b0 con 0x7fe268063040 2020-12-05T17:37:04.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.482+0000 7fe26e5b8700 1 -- 172.21.15.32:0/2678757783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe268057050 con 0x7fe268063040 2020-12-05T17:37:04.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.482+0000 7fe2567fc700 1 -- 172.21.15.32:0/2678757783 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+2137 (secure 0 0 0) 0x7fe258047ad0 con 0x7fe268063040 2020-12-05T17:37:04.485 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:04.485 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5411,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/3995477718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3995477718},{"type":"v1","addr":"172.21.15.61:6833","nonce":3995477718}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:37:01.403820+0000","modified":"2020-12-05T17:37:02.406060+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5976},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5976":{"gid":5976,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/738464306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":738464306},{"type":"v1","addr":"172.21.15.32:6835","nonce":738464306}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-12-05T17:37:04.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.486+0000 7fe26e5b8700 1 -- 172.21.15.32:0/2678757783 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe248055500 msgr2=0x7fe2480579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:04.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.486+0000 7fe26e5b8700 1 --2- 172.21.15.32:0/2678757783 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe248055500 0x7fe2480579c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7fe26400d000 tx=0x7fe264007d90).stop 2020-12-05T17:37:04.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.486+0000 7fe26e5b8700 1 -- 172.21.15.32:0/2678757783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe268063040 msgr2=0x7fe2680ade50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:04.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.486+0000 7fe26e5b8700 1 --2- 172.21.15.32:0/2678757783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe268063040 0x7fe2680ade50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7fe2681411b0 tx=0x7fe258006dc0).stop 2020-12-05T17:37:04.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.486+0000 7fe26e5b8700 1 -- 172.21.15.32:0/2678757783 shutdown_connections 2020-12-05T17:37:04.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.486+0000 7fe26e5b8700 1 --2- 172.21.15.32:0/2678757783 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe248055500 0x7fe2480579c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:04.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.486+0000 7fe26e5b8700 1 --2- 172.21.15.32:0/2678757783 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe26813f4f0 0x7fe268152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:04.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.486+0000 7fe26e5b8700 1 --2- 172.21.15.32:0/2678757783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe268063040 0x7fe2680ade50 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:04.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.486+0000 7fe26e5b8700 1 --2- 172.21.15.32:0/2678757783 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2680af3d0 0x7fe268152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:04.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.486+0000 7fe26e5b8700 1 -- 172.21.15.32:0/2678757783 >> 172.21.15.32:0/2678757783 conn(0x7fe26805bbb0 msgr2=0x7fe268136570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:04.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.486+0000 7fe26e5b8700 1 -- 172.21.15.32:0/2678757783 shutdown_connections 2020-12-05T17:37:04.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.486+0000 7fe26e5b8700 1 -- 172.21.15.32:0/2678757783 wait complete. 2020-12-05T17:37:04.490 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 27 2020-12-05T17:37:04.701 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:37:01.403820+0000', 'modified': '2020-12-05T17:37:02.406060+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5976}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5976': {'gid': 5976, 'name': 'a', 'rank': 0, 'incarnation': 26, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/738464306', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 738464306}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 738464306}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:37:04.702 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:37:04.703 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:37:04.835 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":5976,"want_state":"up:active","state":"up:active","rank_uptime":3.4214823870000002,"mdsmap_epoch":27,"osdmap_epoch":62,"osdmap_epoch_barrier":62,"uptime":7.6497996940000004} 2020-12-05T17:37:04.836 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 5976, "mdsmap_epoch": 27, "osdmap_epoch": 62, "osdmap_epoch_barrier": 62, "rank_uptime": 3.421482387, "state": "up:active", "uptime": 7.649799694, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:37:04.837 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:37:04.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65c1575700 1 Processor -- start 2020-12-05T17:37:04.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65c1575700 1 -- start start 2020-12-05T17:37:04.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65c1575700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f65bc0b1fc0 0x7f65bc0af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:04.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65c1575700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f65bc0ab710 0x7f65bc061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:04.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65c1575700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65bc146a90 con 0x7f65bc147400 2020-12-05T17:37:04.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65c1575700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65bc0b0110 con 0x7f65bc0ab710 2020-12-05T17:37:04.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65c1575700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc0b23a0 con 0x7f65bc0b1fc0 2020-12-05T17:37:04.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65bbfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f65bc0b1fc0 0x7f65bc0af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:04.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65bb7fe700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f65bc0ab710 0x7f65bc061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:04.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65bbfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f65bc0b1fc0 0x7f65bc0af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:32910/0 (socket says 172.21.15.32:32910) 2020-12-05T17:37:04.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65bbfff700 1 -- 172.21.15.32:0/45150028 learned_addr learned my addr 172.21.15.32:0/45150028 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:04.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65bbfff700 1 -- 172.21.15.32:0/45150028 >> v2:172.21.15.61:3300/0 conn(0x7f65bc0ab710 msgr2=0x7f65bc061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:04.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65bbfff700 1 --2- 172.21.15.32:0/45150028 >> v2:172.21.15.61:3300/0 conn(0x7f65bc0ab710 0x7f65bc061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:04.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65bbfff700 1 -- 172.21.15.32:0/45150028 >> v1:172.21.15.61:6789/0 conn(0x7f65bc147400 legacy=0x7f65bc145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:04.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65bbfff700 1 -- 172.21.15.32:0/45150028 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc146cc0 con 0x7f65bc0b1fc0 2020-12-05T17:37:04.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65bbfff700 1 --2- 172.21.15.32:0/45150028 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f65bc0b1fc0 0x7f65bc0af230 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f65ac002f30 tx=0x7f65ac006e80).ready entity=mon.2 client_cookie=60ca48857d96f29e server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:04.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65baffd700 1 -- 172.21.15.32:0/45150028 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65ac0097f0 con 0x7f65bc0b1fc0 2020-12-05T17:37:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65baffd700 1 -- 172.21.15.32:0/45150028 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ac009950 con 0x7f65bc0b1fc0 2020-12-05T17:37:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65baffd700 1 -- 172.21.15.32:0/45150028 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65ac009cb0 con 0x7f65bc0b1fc0 2020-12-05T17:37:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65c1575700 1 -- 172.21.15.32:0/45150028 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f65bc0b1fc0 msgr2=0x7f65bc0af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65c1575700 1 --2- 172.21.15.32:0/45150028 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f65bc0b1fc0 0x7f65bc0af230 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f65ac002f30 tx=0x7f65ac006e80).stop 2020-12-05T17:37:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65c1575700 1 -- 172.21.15.32:0/45150028 shutdown_connections 2020-12-05T17:37:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65c1575700 1 --2- 172.21.15.32:0/45150028 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f65bc0b1fc0 0x7f65bc0af230 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65c1575700 1 --2- 172.21.15.32:0/45150028 >> v2:172.21.15.61:3300/0 conn(0x7f65bc0ab710 0x7f65bc061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65c1575700 1 -- 172.21.15.32:0/45150028 >> 172.21.15.32:0/45150028 conn(0x7f65bc0b5530 msgr2=0x7f65bc0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65c1575700 1 -- 172.21.15.32:0/45150028 shutdown_connections 2020-12-05T17:37:04.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.954+0000 7f65c1575700 1 -- 172.21.15.32:0/45150028 wait complete. 2020-12-05T17:37:04.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65c1575700 1 Processor -- start 2020-12-05T17:37:04.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65c1575700 1 -- start start 2020-12-05T17:37:04.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65c1575700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65bc0ab710 0x7f65bc158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65c1575700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f65bc0b1fc0 0x7f65bc0abe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65c1575700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65bc147400 0x7f65bc09e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65c1575700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc147e50 con 0x7f65bc0ab710 2020-12-05T17:37:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65c1575700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc1480f0 con 0x7f65bc147400 2020-12-05T17:37:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65c1575700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc148390 con 0x7f65bc0b1fc0 2020-12-05T17:37:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65bbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65bc0ab710 0x7f65bc158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65bbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65bc0ab710 0x7f65bc158f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36126/0 (socket says 172.21.15.32:36126) 2020-12-05T17:37:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65bbfff700 1 -- 172.21.15.32:0/1403610074 learned_addr learned my addr 172.21.15.32:0/1403610074 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:04.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65bb7fe700 1 --2- 172.21.15.32:0/1403610074 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f65bc0b1fc0 0x7f65bc0abe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:04.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65c0d74700 1 --2- 172.21.15.32:0/1403610074 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65bc147400 0x7f65bc09e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:04.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65bb7fe700 1 -- 172.21.15.32:0/1403610074 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65bc147400 msgr2=0x7f65bc09e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:04.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65bb7fe700 1 --2- 172.21.15.32:0/1403610074 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65bc147400 0x7f65bc09e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:04.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65bb7fe700 1 -- 172.21.15.32:0/1403610074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65bc0ab710 msgr2=0x7f65bc158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:04.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65bb7fe700 1 --2- 172.21.15.32:0/1403610074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65bc0ab710 0x7f65bc158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:04.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65bb7fe700 1 -- 172.21.15.32:0/1403610074 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc0ad7f0 con 0x7f65bc0b1fc0 2020-12-05T17:37:04.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65bbfff700 1 --2- 172.21.15.32:0/1403610074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65bc0ab710 0x7f65bc158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:37:04.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65bb7fe700 1 --2- 172.21.15.32:0/1403610074 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f65bc0b1fc0 0x7f65bc0abe30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f65a400c000 tx=0x7f65a4006dc0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:04.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65b97fa700 1 -- 172.21.15.32:0/1403610074 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65a400ca10 con 0x7f65bc0b1fc0 2020-12-05T17:37:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65b97fa700 1 -- 172.21.15.32:0/1403610074 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a400cb70 con 0x7f65bc0b1fc0 2020-12-05T17:37:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65c1575700 1 -- 172.21.15.32:0/1403610074 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65bc0ac800 con 0x7f65bc0b1fc0 2020-12-05T17:37:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65b97fa700 1 -- 172.21.15.32:0/1403610074 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65a400be40 con 0x7f65bc0b1fc0 2020-12-05T17:37:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.958+0000 7f65c1575700 1 -- 172.21.15.32:0/1403610074 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65bc09edd0 con 0x7f65bc0b1fc0 2020-12-05T17:37:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.962+0000 7f65c1575700 1 -- 172.21.15.32:0/1403610074 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65bc148550 con 0x7f65bc0b1fc0 2020-12-05T17:37:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.962+0000 7f65b97fa700 1 -- 172.21.15.32:0/1403610074 <== mon.2 v2:172.21.15.61:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f65a4032830 con 0x7f65bc0b1fc0 2020-12-05T17:37:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.962+0000 7f65b97fa700 1 --2- 172.21.15.32:0/1403610074 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f65a00553f0 0x7f65a00578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.962+0000 7f65b97fa700 1 -- 172.21.15.32:0/1403610074 <== mon.2 v2:172.21.15.61:3301/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f65a406e3c0 con 0x7f65bc0b1fc0 2020-12-05T17:37:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.962+0000 7f65bbfff700 1 --2- 172.21.15.32:0/1403610074 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f65a00553f0 0x7f65a00578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:04.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.962+0000 7f65bbfff700 1 --2- 172.21.15.32:0/1403610074 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f65a00553f0 0x7f65a00578b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f65ac01c000 tx=0x7f65ac015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:04.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:04.970+0000 7f65b97fa700 1 -- 172.21.15.32:0/1403610074 <== mon.2 v2:172.21.15.61:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f65a4044560 con 0x7f65bc0b1fc0 2020-12-05T17:37:05.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.114+0000 7f65c1575700 1 -- 172.21.15.32:0/1403610074 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65bc057050 con 0x7f65bc0b1fc0 2020-12-05T17:37:05.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.114+0000 7f65b97fa700 1 -- 172.21.15.32:0/1403610074 <== mon.2 v2:172.21.15.61:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f65a4047b80 con 0x7f65bc0b1fc0 2020-12-05T17:37:05.119 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:05.119 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":62,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:37:01.403672+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:36:59.960851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-12-05T17:37:00.401054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:37:05.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.118+0000 7f65c1575700 1 -- 172.21.15.32:0/1403610074 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f65a00553f0 msgr2=0x7f65a00578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:05.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.118+0000 7f65c1575700 1 --2- 172.21.15.32:0/1403610074 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f65a00553f0 0x7f65a00578b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f65ac01c000 tx=0x7f65ac015000).stop 2020-12-05T17:37:05.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.118+0000 7f65c1575700 1 -- 172.21.15.32:0/1403610074 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f65bc0b1fc0 msgr2=0x7f65bc0abe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:05.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.118+0000 7f65c1575700 1 --2- 172.21.15.32:0/1403610074 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f65bc0b1fc0 0x7f65bc0abe30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f65a400c000 tx=0x7f65a4006dc0).stop 2020-12-05T17:37:05.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.118+0000 7f65c1575700 1 -- 172.21.15.32:0/1403610074 shutdown_connections 2020-12-05T17:37:05.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.118+0000 7f65c1575700 1 --2- 172.21.15.32:0/1403610074 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65bc147400 0x7f65bc09e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.118+0000 7f65c1575700 1 --2- 172.21.15.32:0/1403610074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65bc0ab710 0x7f65bc158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.118+0000 7f65c1575700 1 --2- 172.21.15.32:0/1403610074 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f65a00553f0 0x7f65a00578b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.118+0000 7f65c1575700 1 --2- 172.21.15.32:0/1403610074 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f65bc0b1fc0 0x7f65bc0abe30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.118+0000 7f65c1575700 1 -- 172.21.15.32:0/1403610074 >> 172.21.15.32:0/1403610074 conn(0x7f65bc0b5530 msgr2=0x7f65bc0618e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:05.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.118+0000 7f65c1575700 1 -- 172.21.15.32:0/1403610074 shutdown_connections 2020-12-05T17:37:05.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.118+0000 7f65c1575700 1 -- 172.21.15.32:0/1403610074 wait complete. 2020-12-05T17:37:05.133 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:36:59.960851+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:37:00.401054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:37:05.134 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:37:05.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.250+0000 7f9c1525d700 1 Processor -- start 2020-12-05T17:37:05.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.250+0000 7f9c1525d700 1 -- start start 2020-12-05T17:37:05.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.250+0000 7f9c1525d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9c1005fe00 0x7f9c100ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:05.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.250+0000 7f9c1525d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9c100afb70 0x7f9c100aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:05.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.250+0000 7f9c1525d700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c100b13e0 con 0x7f9c10141cb0 2020-12-05T17:37:05.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.250+0000 7f9c1525d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c100601e0 con 0x7f9c100afb70 2020-12-05T17:37:05.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.250+0000 7f9c1525d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c100b0ea0 con 0x7f9c1005fe00 2020-12-05T17:37:05.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.250+0000 7f9c0f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9c100afb70 0x7f9c100aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:05.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.250+0000 7f9c0f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9c100afb70 0x7f9c100aebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36134/0 (socket says 172.21.15.32:36134) 2020-12-05T17:37:05.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.250+0000 7f9c0f7fe700 1 -- 172.21.15.32:0/665182366 learned_addr learned my addr 172.21.15.32:0/665182366 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.250+0000 7f9c0ffff700 1 --2- 172.21.15.32:0/665182366 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9c1005fe00 0x7f9c100ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.250+0000 7f9c0f7fe700 1 -- 172.21.15.32:0/665182366 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9c1005fe00 msgr2=0x7f9c100ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.250+0000 7f9c0f7fe700 1 --2- 172.21.15.32:0/665182366 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9c1005fe00 0x7f9c100ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.250+0000 7f9c0f7fe700 1 -- 172.21.15.32:0/665182366 >> v1:172.21.15.61:6789/0 conn(0x7f9c10141cb0 legacy=0x7f9c1013d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:05.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.250+0000 7f9c0f7fe700 1 -- 172.21.15.32:0/665182366 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c100b1140 con 0x7f9c100afb70 2020-12-05T17:37:05.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.250+0000 7f9c0ffff700 1 --2- 172.21.15.32:0/665182366 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9c1005fe00 0x7f9c100ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:37:05.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.250+0000 7f9c0f7fe700 1 --2- 172.21.15.32:0/665182366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c100afb70 0x7f9c100aebd0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f9c10061580 tx=0x7f9c04006e80).ready entity=mon.0 client_cookie=72d19f37f5136c35 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:05.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.250+0000 7f9c0effd700 1 -- 172.21.15.32:0/665182366 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c040097b0 con 0x7f9c100afb70 2020-12-05T17:37:05.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.250+0000 7f9c0effd700 1 -- 172.21.15.32:0/665182366 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c04009910 con 0x7f9c100afb70 2020-12-05T17:37:05.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.250+0000 7f9c0effd700 1 -- 172.21.15.32:0/665182366 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c04009ac0 con 0x7f9c100afb70 2020-12-05T17:37:05.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.250+0000 7f9c1525d700 1 -- 172.21.15.32:0/665182366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c100afb70 msgr2=0x7f9c100aebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:05.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.250+0000 7f9c1525d700 1 --2- 172.21.15.32:0/665182366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c100afb70 0x7f9c100aebd0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f9c10061580 tx=0x7f9c04006e80).stop 2020-12-05T17:37:05.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.254+0000 7f9c1525d700 1 -- 172.21.15.32:0/665182366 shutdown_connections 2020-12-05T17:37:05.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.254+0000 7f9c1525d700 1 --2- 172.21.15.32:0/665182366 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9c1005fe00 0x7f9c100ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.254+0000 7f9c1525d700 1 --2- 172.21.15.32:0/665182366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c100afb70 0x7f9c100aebd0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.254+0000 7f9c1525d700 1 -- 172.21.15.32:0/665182366 >> 172.21.15.32:0/665182366 conn(0x7f9c100b5a30 msgr2=0x7f9c1005f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:05.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.254+0000 7f9c1525d700 1 -- 172.21.15.32:0/665182366 shutdown_connections 2020-12-05T17:37:05.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.254+0000 7f9c1525d700 1 -- 172.21.15.32:0/665182366 wait complete. 2020-12-05T17:37:05.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.254+0000 7f9c1525d700 1 Processor -- start 2020-12-05T17:37:05.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.254+0000 7f9c1525d700 1 -- start start 2020-12-05T17:37:05.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c1525d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9c1005fe00 0x7f9c101388b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:05.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c1525d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9c100afb70 0x7f9c1013cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:05.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c1525d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c10141cb0 0x7f9c1013cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:05.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c1525d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c10138380 con 0x7f9c10141cb0 2020-12-05T17:37:05.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c1525d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1013b240 con 0x7f9c1005fe00 2020-12-05T17:37:05.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c1525d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1013b4e0 con 0x7f9c100afb70 2020-12-05T17:37:05.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c14a5c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c10141cb0 0x7f9c1013cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:05.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c14a5c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c10141cb0 0x7f9c1013cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36144/0 (socket says 172.21.15.32:36144) 2020-12-05T17:37:05.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c14a5c700 1 -- 172.21.15.32:0/910170775 learned_addr learned my addr 172.21.15.32:0/910170775 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:05.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c0f7fe700 1 --2- 172.21.15.32:0/910170775 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9c100afb70 0x7f9c1013cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:05.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c0ffff700 1 --2- 172.21.15.32:0/910170775 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9c1005fe00 0x7f9c101388b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:05.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c14a5c700 1 -- 172.21.15.32:0/910170775 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9c100afb70 msgr2=0x7f9c1013cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:05.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c14a5c700 1 --2- 172.21.15.32:0/910170775 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9c100afb70 0x7f9c1013cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c14a5c700 1 -- 172.21.15.32:0/910170775 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9c1005fe00 msgr2=0x7f9c101388b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:05.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c14a5c700 1 --2- 172.21.15.32:0/910170775 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9c1005fe00 0x7f9c101388b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c14a5c700 1 -- 172.21.15.32:0/910170775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c10138000 con 0x7f9c10141cb0 2020-12-05T17:37:05.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c14a5c700 1 --2- 172.21.15.32:0/910170775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c10141cb0 0x7f9c1013cfa0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f9c08002ef0 tx=0x7f9c08006c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:05.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c0d7fa700 1 -- 172.21.15.32:0/910170775 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c0800be70 con 0x7f9c10141cb0 2020-12-05T17:37:05.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c0d7fa700 1 -- 172.21.15.32:0/910170775 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c08000e90 con 0x7f9c10141cb0 2020-12-05T17:37:05.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c1525d700 1 -- 172.21.15.32:0/910170775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c10139100 con 0x7f9c10141cb0 2020-12-05T17:37:05.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c0d7fa700 1 -- 172.21.15.32:0/910170775 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c08002510 con 0x7f9c10141cb0 2020-12-05T17:37:05.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c1525d700 1 -- 172.21.15.32:0/910170775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c10066590 con 0x7f9c10141cb0 2020-12-05T17:37:05.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c1525d700 1 -- 172.21.15.32:0/910170775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c10057050 con 0x7f9c10141cb0 2020-12-05T17:37:05.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c0d7fa700 1 -- 172.21.15.32:0/910170775 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9c0800f030 con 0x7f9c10141cb0 2020-12-05T17:37:05.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c0d7fa700 1 --2- 172.21.15.32:0/910170775 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9bf0055440 0x7f9bf0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:05.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c0d7fa700 1 -- 172.21.15.32:0/910170775 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9c0806dae0 con 0x7f9c10141cb0 2020-12-05T17:37:05.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.258+0000 7f9c0ffff700 1 --2- 172.21.15.32:0/910170775 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9bf0055440 0x7f9bf0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:05.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.262+0000 7f9c0ffff700 1 --2- 172.21.15.32:0/910170775 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9bf0055440 0x7f9bf0057900 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f9c0000d000 tx=0x7f9c00008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:05.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.266+0000 7f9c0d7fa700 1 -- 172.21.15.32:0/910170775 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9c08043c80 con 0x7f9c10141cb0 2020-12-05T17:37:05.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.410+0000 7f9c1525d700 1 -- 172.21.15.32:0/910170775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9c10138da0 con 0x7f9c10141cb0 2020-12-05T17:37:05.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.410+0000 7f9c0d7fa700 1 -- 172.21.15.32:0/910170775 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v62) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9c080472a0 con 0x7f9c10141cb0 2020-12-05T17:37:05.413 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:37:05.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.410+0000 7f9c1525d700 1 -- 172.21.15.32:0/910170775 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9bf0055440 msgr2=0x7f9bf0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:05.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.410+0000 7f9c1525d700 1 --2- 172.21.15.32:0/910170775 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9bf0055440 0x7f9bf0057900 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f9c0000d000 tx=0x7f9c00008040).stop 2020-12-05T17:37:05.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.410+0000 7f9c1525d700 1 -- 172.21.15.32:0/910170775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c10141cb0 msgr2=0x7f9c1013cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:05.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.410+0000 7f9c1525d700 1 --2- 172.21.15.32:0/910170775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c10141cb0 0x7f9c1013cfa0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f9c08002ef0 tx=0x7f9c08006c90).stop 2020-12-05T17:37:05.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.414+0000 7f9c1525d700 1 -- 172.21.15.32:0/910170775 shutdown_connections 2020-12-05T17:37:05.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.414+0000 7f9c1525d700 1 --2- 172.21.15.32:0/910170775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c10141cb0 0x7f9c1013cfa0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.414+0000 7f9c1525d700 1 --2- 172.21.15.32:0/910170775 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9c1005fe00 0x7f9c101388b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.414+0000 7f9c1525d700 1 --2- 172.21.15.32:0/910170775 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9bf0055440 0x7f9bf0057900 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.414+0000 7f9c1525d700 1 --2- 172.21.15.32:0/910170775 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9c100afb70 0x7f9c1013cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.414+0000 7f9c1525d700 1 -- 172.21.15.32:0/910170775 >> 172.21.15.32:0/910170775 conn(0x7f9c100b5a30 msgr2=0x7f9c1005f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:05.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.414+0000 7f9c1525d700 1 -- 172.21.15.32:0/910170775 shutdown_connections 2020-12-05T17:37:05.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.414+0000 7f9c1525d700 1 -- 172.21.15.32:0/910170775 wait complete. 2020-12-05T17:37:05.427 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:37:05.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.530+0000 7fa3b1e3c700 1 Processor -- start 2020-12-05T17:37:05.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.530+0000 7fa3b1e3c700 1 -- start start 2020-12-05T17:37:05.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b1e3c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa3ac061920 0x7fa3ac064a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:05.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b1e3c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3ac0b16b0 0x7fa3ac146860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:05.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b1e3c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3ac1428b0 0x7fa3ac142d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:05.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b1e3c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3ac146d50 con 0x7fa3ac1428b0 2020-12-05T17:37:05.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b1e3c700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3ac143210 con 0x7fa3ac061920 2020-12-05T17:37:05.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b1e3c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac145e80 con 0x7fa3ac0b16b0 2020-12-05T17:37:05.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b163b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3ac1428b0 0x7fa3ac142d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:05.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b163b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3ac1428b0 0x7fa3ac142d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36152/0 (socket says 172.21.15.32:36152) 2020-12-05T17:37:05.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b163b700 1 -- 172.21.15.32:0/2865801933 learned_addr learned my addr 172.21.15.32:0/2865801933 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:05.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b0e3a700 1 --2- 172.21.15.32:0/2865801933 >> v2:172.21.15.61:3300/0 conn(0x7fa3ac061920 0x7fa3ac064a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:05.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3abfff700 1 --2- 172.21.15.32:0/2865801933 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3ac0b16b0 0x7fa3ac146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:05.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b163b700 1 -- 172.21.15.32:0/2865801933 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3ac0b16b0 msgr2=0x7fa3ac146860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:05.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b163b700 1 --2- 172.21.15.32:0/2865801933 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3ac0b16b0 0x7fa3ac146860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b163b700 1 -- 172.21.15.32:0/2865801933 >> v2:172.21.15.61:3300/0 conn(0x7fa3ac061920 msgr2=0x7fa3ac064a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:05.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b163b700 1 --2- 172.21.15.32:0/2865801933 >> v2:172.21.15.61:3300/0 conn(0x7fa3ac061920 0x7fa3ac064a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b163b700 1 -- 172.21.15.32:0/2865801933 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac146120 con 0x7fa3ac1428b0 2020-12-05T17:37:05.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b163b700 1 --2- 172.21.15.32:0/2865801933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3ac1428b0 0x7fa3ac142d20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fa3a40032b0 tx=0x7fa3a4009270).ready entity=mon.0 client_cookie=ae7f1c1a7c226f57 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:05.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3ab7fe700 1 -- 172.21.15.32:0/2865801933 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3a4009e60 con 0x7fa3ac1428b0 2020-12-05T17:37:05.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3ab7fe700 1 -- 172.21.15.32:0/2865801933 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a400bb50 con 0x7fa3ac1428b0 2020-12-05T17:37:05.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3ab7fe700 1 -- 172.21.15.32:0/2865801933 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3a400bcc0 con 0x7fa3ac1428b0 2020-12-05T17:37:05.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b1e3c700 1 -- 172.21.15.32:0/2865801933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3ac1428b0 msgr2=0x7fa3ac142d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:05.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b1e3c700 1 --2- 172.21.15.32:0/2865801933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3ac1428b0 0x7fa3ac142d20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fa3a40032b0 tx=0x7fa3a4009270).stop 2020-12-05T17:37:05.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b1e3c700 1 -- 172.21.15.32:0/2865801933 shutdown_connections 2020-12-05T17:37:05.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b1e3c700 1 --2- 172.21.15.32:0/2865801933 >> v2:172.21.15.61:3300/0 conn(0x7fa3ac061920 0x7fa3ac064a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b1e3c700 1 --2- 172.21.15.32:0/2865801933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3ac1428b0 0x7fa3ac142d20 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b1e3c700 1 --2- 172.21.15.32:0/2865801933 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3ac0b16b0 0x7fa3ac146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b1e3c700 1 -- 172.21.15.32:0/2865801933 >> 172.21.15.32:0/2865801933 conn(0x7fa3ac0b6d10 msgr2=0x7fa3ac0b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:05.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b1e3c700 1 -- 172.21.15.32:0/2865801933 shutdown_connections 2020-12-05T17:37:05.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.534+0000 7fa3b1e3c700 1 -- 172.21.15.32:0/2865801933 wait complete. 2020-12-05T17:37:05.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3b1e3c700 1 Processor -- start 2020-12-05T17:37:05.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3b1e3c700 1 -- start start 2020-12-05T17:37:05.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3b1e3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa3ac061920 0x7fa3ac066de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:05.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3b1e3c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3ac0b16b0 0x7fa3ac064ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:05.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3b1e3c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3ac1428b0 0x7fa3ac0663b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:05.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3b1e3c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac1497c0 con 0x7fa3ac1428b0 2020-12-05T17:37:05.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3b1e3c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac149a60 con 0x7fa3ac061920 2020-12-05T17:37:05.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3b1e3c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac13cfc0 con 0x7fa3ac0b16b0 2020-12-05T17:37:05.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3b163b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3ac1428b0 0x7fa3ac0663b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:05.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3abfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3ac0b16b0 0x7fa3ac064ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:05.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3b163b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3ac1428b0 0x7fa3ac0663b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36158/0 (socket says 172.21.15.32:36158) 2020-12-05T17:37:05.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3abfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3ac0b16b0 0x7fa3ac064ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:32946/0 (socket says 172.21.15.32:32946) 2020-12-05T17:37:05.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3b163b700 1 -- 172.21.15.32:0/1283194913 learned_addr learned my addr 172.21.15.32:0/1283194913 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:05.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3b0e3a700 1 --2- 172.21.15.32:0/1283194913 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa3ac061920 0x7fa3ac066de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:05.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3b163b700 1 -- 172.21.15.32:0/1283194913 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3ac0b16b0 msgr2=0x7fa3ac064ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:05.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3b163b700 1 --2- 172.21.15.32:0/1283194913 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3ac0b16b0 0x7fa3ac064ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3b163b700 1 -- 172.21.15.32:0/1283194913 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa3ac061920 msgr2=0x7fa3ac066de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:05.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3b163b700 1 --2- 172.21.15.32:0/1283194913 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa3ac061920 0x7fa3ac066de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3b163b700 1 -- 172.21.15.32:0/1283194913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac13d180 con 0x7fa3ac1428b0 2020-12-05T17:37:05.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3b163b700 1 --2- 172.21.15.32:0/1283194913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3ac1428b0 0x7fa3ac0663b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7fa3a40032b0 tx=0x7fa3a4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:05.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3a9ffb700 1 -- 172.21.15.32:0/1283194913 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3a401f3f0 con 0x7fa3ac1428b0 2020-12-05T17:37:05.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3b1e3c700 1 -- 172.21.15.32:0/1283194913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ac147780 con 0x7fa3ac1428b0 2020-12-05T17:37:05.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3a9ffb700 1 -- 172.21.15.32:0/1283194913 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a401f550 con 0x7fa3ac1428b0 2020-12-05T17:37:05.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3a9ffb700 1 -- 172.21.15.32:0/1283194913 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3a401f8b0 con 0x7fa3ac1428b0 2020-12-05T17:37:05.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.538+0000 7fa3b1e3c700 1 -- 172.21.15.32:0/1283194913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ac147bd0 con 0x7fa3ac1428b0 2020-12-05T17:37:05.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.542+0000 7fa3b1e3c700 1 -- 172.21.15.32:0/1283194913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3ac057050 con 0x7fa3ac1428b0 2020-12-05T17:37:05.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.542+0000 7fa3a9ffb700 1 -- 172.21.15.32:0/1283194913 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa3a4026030 con 0x7fa3ac1428b0 2020-12-05T17:37:05.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.542+0000 7fa3a9ffb700 1 --2- 172.21.15.32:0/1283194913 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa38c055440 0x7fa38c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:05.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.542+0000 7fa3a9ffb700 1 -- 172.21.15.32:0/1283194913 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa3a406ecd0 con 0x7fa3ac1428b0 2020-12-05T17:37:05.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.542+0000 7fa3b0e3a700 1 --2- 172.21.15.32:0/1283194913 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa38c055440 0x7fa38c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:05.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.542+0000 7fa3b0e3a700 1 --2- 172.21.15.32:0/1283194913 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa38c055440 0x7fa38c057900 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fa3a000b000 tx=0x7fa3a0006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:05.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.550+0000 7fa3a9ffb700 1 -- 172.21.15.32:0/1283194913 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa3a4044dd0 con 0x7fa3ac1428b0 2020-12-05T17:37:05.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.690+0000 7fa3b1e3c700 1 -- 172.21.15.32:0/1283194913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa3ac149bd0 con 0x7fa3ac1428b0 2020-12-05T17:37:05.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.690+0000 7fa3a9ffb700 1 -- 172.21.15.32:0/1283194913 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v62) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa3a40150a0 con 0x7fa3ac1428b0 2020-12-05T17:37:05.692 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:37:05.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.690+0000 7fa3b1e3c700 1 -- 172.21.15.32:0/1283194913 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa38c055440 msgr2=0x7fa38c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:05.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.690+0000 7fa3b1e3c700 1 --2- 172.21.15.32:0/1283194913 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa38c055440 0x7fa38c057900 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fa3a000b000 tx=0x7fa3a0006dc0).stop 2020-12-05T17:37:05.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.690+0000 7fa3b1e3c700 1 -- 172.21.15.32:0/1283194913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3ac1428b0 msgr2=0x7fa3ac0663b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:05.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.690+0000 7fa3b1e3c700 1 --2- 172.21.15.32:0/1283194913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3ac1428b0 0x7fa3ac0663b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7fa3a40032b0 tx=0x7fa3a4016000).stop 2020-12-05T17:37:05.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.690+0000 7fa3b1e3c700 1 -- 172.21.15.32:0/1283194913 shutdown_connections 2020-12-05T17:37:05.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.690+0000 7fa3b1e3c700 1 --2- 172.21.15.32:0/1283194913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3ac1428b0 0x7fa3ac0663b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.690+0000 7fa3b1e3c700 1 --2- 172.21.15.32:0/1283194913 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa3ac061920 0x7fa3ac066de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.690+0000 7fa3b1e3c700 1 --2- 172.21.15.32:0/1283194913 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa38c055440 0x7fa38c057900 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.690+0000 7fa3b1e3c700 1 --2- 172.21.15.32:0/1283194913 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3ac0b16b0 0x7fa3ac064ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.690+0000 7fa3b1e3c700 1 -- 172.21.15.32:0/1283194913 >> 172.21.15.32:0/1283194913 conn(0x7fa3ac0b6d10 msgr2=0x7fa3ac145100 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:05.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.690+0000 7fa3b1e3c700 1 -- 172.21.15.32:0/1283194913 shutdown_connections 2020-12-05T17:37:05.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.690+0000 7fa3b1e3c700 1 -- 172.21.15.32:0/1283194913 wait complete. 2020-12-05T17:37:05.705 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:37:05.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.814+0000 7fd3ea0fc700 1 Processor -- start 2020-12-05T17:37:05.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.814+0000 7fd3ea0fc700 1 -- start start 2020-12-05T17:37:05.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.814+0000 7fd3ea0fc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd3e405f3b0 0x7fd3e405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:05.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3ea0fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3e40b2fb0 0x7fd3e4061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:05.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3ea0fc700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3e40b2df0 con 0x7fd3e405feb0 2020-12-05T17:37:05.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3ea0fc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3e40b3c90 con 0x7fd3e40b2fb0 2020-12-05T17:37:05.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3ea0fc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e40b2b50 con 0x7fd3e405f3b0 2020-12-05T17:37:05.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3e98fb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3e40b2fb0 0x7fd3e4061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:05.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3e98fb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3e40b2fb0 0x7fd3e4061780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36162/0 (socket says 172.21.15.32:36162) 2020-12-05T17:37:05.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3e98fb700 1 -- 172.21.15.32:0/4272772948 learned_addr learned my addr 172.21.15.32:0/4272772948 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:05.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3e90fa700 1 --2- 172.21.15.32:0/4272772948 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd3e405f3b0 0x7fd3e405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:05.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3e98fb700 1 -- 172.21.15.32:0/4272772948 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd3e405f3b0 msgr2=0x7fd3e405f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:05.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3e98fb700 1 --2- 172.21.15.32:0/4272772948 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd3e405f3b0 0x7fd3e405f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3e98fb700 1 -- 172.21.15.32:0/4272772948 >> v1:172.21.15.61:6789/0 conn(0x7fd3e405feb0 legacy=0x7fd3e413c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3e98fb700 1 -- 172.21.15.32:0/4272772948 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e4142820 con 0x7fd3e40b2fb0 2020-12-05T17:37:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3e98fb700 1 --2- 172.21.15.32:0/4272772948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3e40b2fb0 0x7fd3e4061780 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7fd3dc0032b0 tx=0x7fd3dc009270).ready entity=mon.0 client_cookie=e5861628f61d206c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:05.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3dbfff700 1 -- 172.21.15.32:0/4272772948 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3dc00bc00 con 0x7fd3e40b2fb0 2020-12-05T17:37:05.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3dbfff700 1 -- 172.21.15.32:0/4272772948 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3dc00bd60 con 0x7fd3e40b2fb0 2020-12-05T17:37:05.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3dbfff700 1 -- 172.21.15.32:0/4272772948 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3dc011400 con 0x7fd3e40b2fb0 2020-12-05T17:37:05.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3ea0fc700 1 -- 172.21.15.32:0/4272772948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3e40b2fb0 msgr2=0x7fd3e4061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:05.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3ea0fc700 1 --2- 172.21.15.32:0/4272772948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3e40b2fb0 0x7fd3e4061780 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7fd3dc0032b0 tx=0x7fd3dc009270).stop 2020-12-05T17:37:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3ea0fc700 1 -- 172.21.15.32:0/4272772948 shutdown_connections 2020-12-05T17:37:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3ea0fc700 1 --2- 172.21.15.32:0/4272772948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3e40b2fb0 0x7fd3e4061780 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3ea0fc700 1 --2- 172.21.15.32:0/4272772948 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd3e405f3b0 0x7fd3e405f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3ea0fc700 1 -- 172.21.15.32:0/4272772948 >> 172.21.15.32:0/4272772948 conn(0x7fd3e40b8f20 msgr2=0x7fd3e40b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3ea0fc700 1 -- 172.21.15.32:0/4272772948 shutdown_connections 2020-12-05T17:37:05.826 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.818+0000 7fd3ea0fc700 1 -- 172.21.15.32:0/4272772948 wait complete. 2020-12-05T17:37:05.826 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3ea0fc700 1 Processor -- start 2020-12-05T17:37:05.826 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3ea0fc700 1 -- start start 2020-12-05T17:37:05.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3ea0fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3e405f3b0 0x7fd3e4145770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:05.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3ea0fc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd3e405feb0 0x7fd3e4143b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:05.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3ea0fc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd3e40b2fb0 0x7fd3e413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:05.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3ea0fc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e4145c60 con 0x7fd3e405f3b0 2020-12-05T17:37:05.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3ea0fc700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e4145e80 con 0x7fd3e405feb0 2020-12-05T17:37:05.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3ea0fc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e4146120 con 0x7fd3e40b2fb0 2020-12-05T17:37:05.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3e90fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3e405f3b0 0x7fd3e4145770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3e90fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3e405f3b0 0x7fd3e4145770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36168/0 (socket says 172.21.15.32:36168) 2020-12-05T17:37:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3e90fa700 1 -- 172.21.15.32:0/1659888873 learned_addr learned my addr 172.21.15.32:0/1659888873 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3e98fb700 1 --2- 172.21.15.32:0/1659888873 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd3e40b2fb0 0x7fd3e413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3e88f9700 1 --2- 172.21.15.32:0/1659888873 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd3e405feb0 0x7fd3e4143b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:05.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3e90fa700 1 -- 172.21.15.32:0/1659888873 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd3e40b2fb0 msgr2=0x7fd3e413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:05.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3e90fa700 1 --2- 172.21.15.32:0/1659888873 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd3e40b2fb0 0x7fd3e413b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3e90fa700 1 -- 172.21.15.32:0/1659888873 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd3e405feb0 msgr2=0x7fd3e4143b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:05.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3e90fa700 1 --2- 172.21.15.32:0/1659888873 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd3e405feb0 0x7fd3e4143b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3e90fa700 1 -- 172.21.15.32:0/1659888873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e41463c0 con 0x7fd3e405f3b0 2020-12-05T17:37:05.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3e90fa700 1 --2- 172.21.15.32:0/1659888873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3e405f3b0 0x7fd3e4145770 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7fd3e000a000 tx=0x7fd3e0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:05.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3da7fc700 1 -- 172.21.15.32:0/1659888873 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3e00127e0 con 0x7fd3e405f3b0 2020-12-05T17:37:05.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3da7fc700 1 -- 172.21.15.32:0/1659888873 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e0012940 con 0x7fd3e405f3b0 2020-12-05T17:37:05.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3ea0fc700 1 -- 172.21.15.32:0/1659888873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e4144060 con 0x7fd3e405f3b0 2020-12-05T17:37:05.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3da7fc700 1 -- 172.21.15.32:0/1659888873 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3e0012ca0 con 0x7fd3e405f3b0 2020-12-05T17:37:05.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.822+0000 7fd3ea0fc700 1 -- 172.21.15.32:0/1659888873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e4140700 con 0x7fd3e405f3b0 2020-12-05T17:37:05.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.826+0000 7fd3ea0fc700 1 -- 172.21.15.32:0/1659888873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3e4057050 con 0x7fd3e405f3b0 2020-12-05T17:37:05.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.826+0000 7fd3da7fc700 1 -- 172.21.15.32:0/1659888873 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd3e0013480 con 0x7fd3e405f3b0 2020-12-05T17:37:05.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.826+0000 7fd3da7fc700 1 --2- 172.21.15.32:0/1659888873 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd3cc055350 0x7fd3cc057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:05.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.826+0000 7fd3da7fc700 1 -- 172.21.15.32:0/1659888873 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd3e006e6a0 con 0x7fd3e405f3b0 2020-12-05T17:37:05.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.826+0000 7fd3e88f9700 1 --2- 172.21.15.32:0/1659888873 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd3cc055350 0x7fd3cc057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:05.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.826+0000 7fd3e88f9700 1 --2- 172.21.15.32:0/1659888873 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd3cc055350 0x7fd3cc057810 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fd3d400d000 tx=0x7fd3d4000b80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:05.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.834+0000 7fd3da7fc700 1 -- 172.21.15.32:0/1659888873 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd3e00447e0 con 0x7fd3e405f3b0 2020-12-05T17:37:05.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.978+0000 7fd3ea0fc700 1 -- 172.21.15.32:0/1659888873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd3e413bbd0 con 0x7fd3e405f3b0 2020-12-05T17:37:05.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.978+0000 7fd3da7fc700 1 -- 172.21.15.32:0/1659888873 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v62) v1 ==== 104+0+10 (secure 0 0 0) 0x7fd3e0047e00 con 0x7fd3e405f3b0 2020-12-05T17:37:05.980 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:37:05.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.982+0000 7fd3ea0fc700 1 -- 172.21.15.32:0/1659888873 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd3cc055350 msgr2=0x7fd3cc057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:05.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.982+0000 7fd3ea0fc700 1 --2- 172.21.15.32:0/1659888873 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd3cc055350 0x7fd3cc057810 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fd3d400d000 tx=0x7fd3d4000b80).stop 2020-12-05T17:37:05.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.982+0000 7fd3ea0fc700 1 -- 172.21.15.32:0/1659888873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3e405f3b0 msgr2=0x7fd3e4145770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:05.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.982+0000 7fd3ea0fc700 1 --2- 172.21.15.32:0/1659888873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3e405f3b0 0x7fd3e4145770 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7fd3e000a000 tx=0x7fd3e0008040).stop 2020-12-05T17:37:05.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.982+0000 7fd3ea0fc700 1 -- 172.21.15.32:0/1659888873 shutdown_connections 2020-12-05T17:37:05.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.982+0000 7fd3ea0fc700 1 --2- 172.21.15.32:0/1659888873 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd3cc055350 0x7fd3cc057810 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.982+0000 7fd3ea0fc700 1 --2- 172.21.15.32:0/1659888873 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd3e40b2fb0 0x7fd3e413b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.982+0000 7fd3ea0fc700 1 --2- 172.21.15.32:0/1659888873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3e405f3b0 0x7fd3e4145770 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.982+0000 7fd3ea0fc700 1 --2- 172.21.15.32:0/1659888873 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd3e405feb0 0x7fd3e4143b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:05.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.982+0000 7fd3ea0fc700 1 -- 172.21.15.32:0/1659888873 >> 172.21.15.32:0/1659888873 conn(0x7fd3e40b8f20 msgr2=0x7fd3e4061780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:05.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.982+0000 7fd3ea0fc700 1 -- 172.21.15.32:0/1659888873 shutdown_connections 2020-12-05T17:37:05.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:05.982+0000 7fd3ea0fc700 1 -- 172.21.15.32:0/1659888873 wait complete. 2020-12-05T17:37:06.000 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:37:06.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.118+0000 7f8f5e1f6700 1 Processor -- start 2020-12-05T17:37:06.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.118+0000 7f8f5e1f6700 1 -- start start 2020-12-05T17:37:06.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.118+0000 7f8f5e1f6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8f5805feb0 0x7f8f5805f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:06.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.118+0000 7f8f5e1f6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f580b2fb0 0x7f8f58061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:06.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.118+0000 7f8f5e1f6700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f580b2df0 con 0x7f8f5805f3b0 2020-12-05T17:37:06.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.118+0000 7f8f5e1f6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f580b3c90 con 0x7f8f580b2fb0 2020-12-05T17:37:06.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.118+0000 7f8f5e1f6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f580b2b50 con 0x7f8f5805feb0 2020-12-05T17:37:06.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.118+0000 7f8f5d9f5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f580b2fb0 0x7f8f58061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:06.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.118+0000 7f8f5d9f5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f580b2fb0 0x7f8f58061780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36180/0 (socket says 172.21.15.32:36180) 2020-12-05T17:37:06.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.118+0000 7f8f5d1f4700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f8f5805f3b0 0x7f8f5813c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:51464/0 (socket says 172.21.15.32:51464) 2020-12-05T17:37:06.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.118+0000 7f8f5d9f5700 1 -- 172.21.15.32:0/1132017115 learned_addr learned my addr 172.21.15.32:0/1132017115 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:06.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.118+0000 7f8f5c9f3700 1 --2- 172.21.15.32:0/1132017115 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8f5805feb0 0x7f8f5805f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:06.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.122+0000 7f8f4ffff700 1 -- 172.21.15.32:0/1132017115 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1591475191 0 0) 0x7f8f580b2df0 con 0x7f8f5805f3b0 2020-12-05T17:37:06.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.122+0000 7f8f4ffff700 1 -- 172.21.15.32:0/1132017115 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f3c003230 con 0x7f8f5805f3b0 2020-12-05T17:37:06.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.122+0000 7f8f5d9f5700 1 -- 172.21.15.32:0/1132017115 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8f5805feb0 msgr2=0x7f8f5805f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:06.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.122+0000 7f8f5d9f5700 1 --2- 172.21.15.32:0/1132017115 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8f5805feb0 0x7f8f5805f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:06.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.122+0000 7f8f5d9f5700 1 -- 172.21.15.32:0/1132017115 >> v1:172.21.15.61:6789/0 conn(0x7f8f5805f3b0 legacy=0x7f8f5813c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:06.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.122+0000 7f8f5d9f5700 1 -- 172.21.15.32:0/1132017115 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f58142820 con 0x7f8f580b2fb0 2020-12-05T17:37:06.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.122+0000 7f8f5d9f5700 1 --2- 172.21.15.32:0/1132017115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f580b2fb0 0x7f8f58061780 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f8f50002f30 tx=0x7f8f50006e80).ready entity=mon.0 client_cookie=b3f9a38b44dc6ea0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:06.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.122+0000 7f8f4ffff700 1 -- 172.21.15.32:0/1132017115 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f50009640 con 0x7f8f580b2fb0 2020-12-05T17:37:06.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.122+0000 7f8f4ffff700 1 -- 172.21.15.32:0/1132017115 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f500097a0 con 0x7f8f580b2fb0 2020-12-05T17:37:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.122+0000 7f8f4ffff700 1 -- 172.21.15.32:0/1132017115 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f50009b00 con 0x7f8f580b2fb0 2020-12-05T17:37:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.122+0000 7f8f5e1f6700 1 -- 172.21.15.32:0/1132017115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f580b2fb0 msgr2=0x7f8f58061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.122+0000 7f8f5e1f6700 1 --2- 172.21.15.32:0/1132017115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f580b2fb0 0x7f8f58061780 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f8f50002f30 tx=0x7f8f50006e80).stop 2020-12-05T17:37:06.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.122+0000 7f8f5e1f6700 1 -- 172.21.15.32:0/1132017115 shutdown_connections 2020-12-05T17:37:06.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.122+0000 7f8f5e1f6700 1 --2- 172.21.15.32:0/1132017115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f580b2fb0 0x7f8f58061780 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:06.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.122+0000 7f8f5e1f6700 1 --2- 172.21.15.32:0/1132017115 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8f5805feb0 0x7f8f5805f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:06.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.122+0000 7f8f5e1f6700 1 -- 172.21.15.32:0/1132017115 >> 172.21.15.32:0/1132017115 conn(0x7f8f580b8f20 msgr2=0x7f8f580b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:06.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.122+0000 7f8f5e1f6700 1 -- 172.21.15.32:0/1132017115 shutdown_connections 2020-12-05T17:37:06.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.122+0000 7f8f5e1f6700 1 -- 172.21.15.32:0/1132017115 wait complete. 2020-12-05T17:37:06.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.122+0000 7f8f5e1f6700 1 Processor -- start 2020-12-05T17:37:06.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.126+0000 7f8f5e1f6700 1 -- start start 2020-12-05T17:37:06.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.126+0000 7f8f5e1f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5805f3b0 0x7f8f58148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:06.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.126+0000 7f8f5e1f6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8f5805feb0 0x7f8f58113530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:06.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.126+0000 7f8f5e1f6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8f580b2fb0 0x7f8f58140820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:06.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.126+0000 7f8f5e1f6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f58113a20 con 0x7f8f5805f3b0 2020-12-05T17:37:06.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.126+0000 7f8f5e1f6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f58149560 con 0x7f8f5805feb0 2020-12-05T17:37:06.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.126+0000 7f8f5e1f6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f58149800 con 0x7f8f580b2fb0 2020-12-05T17:37:06.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.126+0000 7f8f5d1f4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5805f3b0 0x7f8f58148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:06.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.126+0000 7f8f5d1f4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5805f3b0 0x7f8f58148580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36182/0 (socket says 172.21.15.32:36182) 2020-12-05T17:37:06.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.126+0000 7f8f5d1f4700 1 -- 172.21.15.32:0/3528425188 learned_addr learned my addr 172.21.15.32:0/3528425188 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:06.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.126+0000 7f8f5c9f3700 1 --2- 172.21.15.32:0/3528425188 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8f5805feb0 0x7f8f58113530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:06.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.126+0000 7f8f5d9f5700 1 --2- 172.21.15.32:0/3528425188 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8f580b2fb0 0x7f8f58140820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:06.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.126+0000 7f8f5d1f4700 1 -- 172.21.15.32:0/3528425188 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8f580b2fb0 msgr2=0x7f8f58140820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:06.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.126+0000 7f8f5d1f4700 1 --2- 172.21.15.32:0/3528425188 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8f580b2fb0 0x7f8f58140820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:06.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.126+0000 7f8f5d1f4700 1 -- 172.21.15.32:0/3528425188 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8f5805feb0 msgr2=0x7f8f58113530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:06.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.126+0000 7f8f5d1f4700 1 --2- 172.21.15.32:0/3528425188 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8f5805feb0 0x7f8f58113530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:06.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.126+0000 7f8f5d1f4700 1 -- 172.21.15.32:0/3528425188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f581471e0 con 0x7f8f5805f3b0 2020-12-05T17:37:06.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.126+0000 7f8f5d1f4700 1 --2- 172.21.15.32:0/3528425188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5805f3b0 0x7f8f58148580 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f8f54001450 tx=0x7f8f54007e70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:06.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.126+0000 7f8f4e7fc700 1 -- 172.21.15.32:0/3528425188 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f54005ab0 con 0x7f8f5805f3b0 2020-12-05T17:37:06.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.126+0000 7f8f4e7fc700 1 -- 172.21.15.32:0/3528425188 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f54005c10 con 0x7f8f5805f3b0 2020-12-05T17:37:06.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.126+0000 7f8f5e1f6700 1 -- 172.21.15.32:0/3528425188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f581475e0 con 0x7f8f5805f3b0 2020-12-05T17:37:06.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.126+0000 7f8f4e7fc700 1 -- 172.21.15.32:0/3528425188 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f54007110 con 0x7f8f5805f3b0 2020-12-05T17:37:06.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.126+0000 7f8f5e1f6700 1 -- 172.21.15.32:0/3528425188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f58140d10 con 0x7f8f5805f3b0 2020-12-05T17:37:06.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.130+0000 7f8f5e1f6700 1 -- 172.21.15.32:0/3528425188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f581499c0 con 0x7f8f5805f3b0 2020-12-05T17:37:06.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.130+0000 7f8f4e7fc700 1 -- 172.21.15.32:0/3528425188 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8f54010030 con 0x7f8f5805f3b0 2020-12-05T17:37:06.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.130+0000 7f8f4e7fc700 1 --2- 172.21.15.32:0/3528425188 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8f400554e0 0x7f8f400579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:06.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.130+0000 7f8f4e7fc700 1 -- 172.21.15.32:0/3528425188 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8f5406e0e0 con 0x7f8f5805f3b0 2020-12-05T17:37:06.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.130+0000 7f8f5c9f3700 1 --2- 172.21.15.32:0/3528425188 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8f400554e0 0x7f8f400579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:06.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.130+0000 7f8f5c9f3700 1 --2- 172.21.15.32:0/3528425188 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8f400554e0 0x7f8f400579a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f8f4800b000 tx=0x7f8f48006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:06.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.138+0000 7f8f4e7fc700 1 -- 172.21.15.32:0/3528425188 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8f54044330 con 0x7f8f5805f3b0 2020-12-05T17:37:06.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.278+0000 7f8f5e1f6700 1 -- 172.21.15.32:0/3528425188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8f5805f950 con 0x7f8f5805f3b0 2020-12-05T17:37:06.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.278+0000 7f8f4e7fc700 1 -- 172.21.15.32:0/3528425188 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v62) v1 ==== 100+0+10 (secure 0 0 0) 0x7f8f54047950 con 0x7f8f5805f3b0 2020-12-05T17:37:06.281 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:37:06.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.282+0000 7f8f5e1f6700 1 -- 172.21.15.32:0/3528425188 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8f400554e0 msgr2=0x7f8f400579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:06.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.282+0000 7f8f5e1f6700 1 --2- 172.21.15.32:0/3528425188 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8f400554e0 0x7f8f400579a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f8f4800b000 tx=0x7f8f48006dc0).stop 2020-12-05T17:37:06.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.282+0000 7f8f5e1f6700 1 -- 172.21.15.32:0/3528425188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5805f3b0 msgr2=0x7f8f58148580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:06.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.282+0000 7f8f5e1f6700 1 --2- 172.21.15.32:0/3528425188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5805f3b0 0x7f8f58148580 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f8f54001450 tx=0x7f8f54007e70).stop 2020-12-05T17:37:06.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.282+0000 7f8f5e1f6700 1 -- 172.21.15.32:0/3528425188 shutdown_connections 2020-12-05T17:37:06.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.282+0000 7f8f5e1f6700 1 --2- 172.21.15.32:0/3528425188 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8f400554e0 0x7f8f400579a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:06.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.282+0000 7f8f5e1f6700 1 --2- 172.21.15.32:0/3528425188 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8f580b2fb0 0x7f8f58140820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:06.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.282+0000 7f8f5e1f6700 1 --2- 172.21.15.32:0/3528425188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5805f3b0 0x7f8f58148580 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:06.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.282+0000 7f8f5e1f6700 1 --2- 172.21.15.32:0/3528425188 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8f5805feb0 0x7f8f58113530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:06.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.282+0000 7f8f5e1f6700 1 -- 172.21.15.32:0/3528425188 >> 172.21.15.32:0/3528425188 conn(0x7f8f580b8f20 msgr2=0x7f8f580b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:06.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.282+0000 7f8f5e1f6700 1 -- 172.21.15.32:0/3528425188 shutdown_connections 2020-12-05T17:37:06.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.282+0000 7f8f5e1f6700 1 -- 172.21.15.32:0/3528425188 wait complete. 2020-12-05T17:37:06.299 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:37:06.299 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:37:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.410+0000 7fdb7b59e700 1 Processor -- start 2020-12-05T17:37:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.410+0000 7fdb7b59e700 1 -- start start 2020-12-05T17:37:06.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.410+0000 7fdb7b59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb7c0b1fc0 0x7fdb7c0b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:06.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.410+0000 7fdb7b59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb7c1401f0 con 0x7fdb7c13f670 2020-12-05T17:37:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.410+0000 7fdb7b59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb7c0b23a0 con 0x7fdb7c0af0d0 2020-12-05T17:37:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.410+0000 7fdb7b59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c13f280 con 0x7fdb7c0b1fc0 2020-12-05T17:37:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.410+0000 7fdb7ad9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdb7c13f670 0x7fdb7c13fae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38842/0 (socket says 172.21.15.32:38842) 2020-12-05T17:37:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.410+0000 7fdb7ad9d700 1 -- 172.21.15.32:0/2327210467 learned_addr learned my addr 172.21.15.32:0/2327210467 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:06.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.410+0000 7fdb7a59c700 1 --2- 172.21.15.32:0/2327210467 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb7c0b1fc0 0x7fdb7c0b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:06.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb79d9b700 1 -- 172.21.15.32:0/2327210467 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3476676576 0 0) 0x7fdb74001450 con 0x7fdb7c13f670 2020-12-05T17:37:06.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb79d9b700 1 -- 172.21.15.32:0/2327210467 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb5c003230 con 0x7fdb7c13f670 2020-12-05T17:37:06.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb79d9b700 1 -- 172.21.15.32:0/2327210467 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2308530951 0 0) 0x7fdb7c0b23a0 con 0x7fdb7c0af0d0 2020-12-05T17:37:06.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb79d9b700 1 -- 172.21.15.32:0/2327210467 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb74001450 con 0x7fdb7c0af0d0 2020-12-05T17:37:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb79d9b700 1 -- 172.21.15.32:0/2327210467 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3731808855 0 0) 0x7fdb5c003230 con 0x7fdb7c13f670 2020-12-05T17:37:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb79d9b700 1 -- 172.21.15.32:0/2327210467 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb7c0b1fc0 msgr2=0x7fdb7c0b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb79d9b700 1 --2- 172.21.15.32:0/2327210467 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb7c0b1fc0 0x7fdb7c0b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb79d9b700 1 -- 172.21.15.32:0/2327210467 >> v1:172.21.15.61:6789/0 conn(0x7fdb7c0af0d0 legacy=0x7fdb7c13c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb79d9b700 1 -- 172.21.15.32:0/2327210467 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c0abfa0 con 0x7fdb7c13f670 2020-12-05T17:37:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb79d9b700 1 -- 172.21.15.32:0/2327210467 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fdb74001be0 con 0x7fdb7c13f670 2020-12-05T17:37:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb7a59c700 1 --2- 172.21.15.32:0/2327210467 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb7c0b1fc0 0x7fdb7c0b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:37:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb79d9b700 1 -- 172.21.15.32:0/2327210467 >> v1:172.21.15.32:6789/0 conn(0x7fdb7c13f670 legacy=0x7fdb7c13fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb79d9b700 1 --2- 172.21.15.32:0/2327210467 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb5c0038e0 0x7fdb5c003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb79d9b700 1 --2- 172.21.15.32:0/2327210467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb5c0044d0 0x7fdb5c004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb79d9b700 1 --2- 172.21.15.32:0/2327210467 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb7c0b1fc0 0x7fdb5c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb79d9b700 1 -- 172.21.15.32:0/2327210467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c005330 con 0x7fdb5c0044d0 2020-12-05T17:37:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb79d9b700 1 -- 172.21.15.32:0/2327210467 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c00b720 con 0x7fdb5c0038e0 2020-12-05T17:37:06.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb79d9b700 1 -- 172.21.15.32:0/2327210467 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c00b860 con 0x7fdb7c0b1fc0 2020-12-05T17:37:06.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb71d9b700 1 --2- 172.21.15.32:0/2327210467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb5c0044d0 0x7fdb5c004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:06.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb7a59c700 1 --2- 172.21.15.32:0/2327210467 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb5c0038e0 0x7fdb5c003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:06.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb7ad9d700 1 --2- 172.21.15.32:0/2327210467 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb7c0b1fc0 0x7fdb5c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:06.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb71d9b700 1 -- 172.21.15.32:0/2327210467 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb7c0b1fc0 msgr2=0x7fdb5c004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:06.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb71d9b700 1 --2- 172.21.15.32:0/2327210467 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb7c0b1fc0 0x7fdb5c004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:06.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb71d9b700 1 -- 172.21.15.32:0/2327210467 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb5c0038e0 msgr2=0x7fdb5c003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:06.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb71d9b700 1 --2- 172.21.15.32:0/2327210467 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb5c0038e0 0x7fdb5c003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:06.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb71d9b700 1 -- 172.21.15.32:0/2327210467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5c003230 con 0x7fdb5c0044d0 2020-12-05T17:37:06.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb71d9b700 1 --2- 172.21.15.32:0/2327210467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb5c0044d0 0x7fdb5c004950 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fdb5c0054a0 tx=0x7fdb64007bf0).ready entity=mon.0 client_cookie=3d55df47ac152507 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb79d9b700 1 -- 172.21.15.32:0/2327210467 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb6400e9e0 con 0x7fdb5c0044d0 2020-12-05T17:37:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb79d9b700 1 -- 172.21.15.32:0/2327210467 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb6400eb40 con 0x7fdb5c0044d0 2020-12-05T17:37:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb79d9b700 1 -- 172.21.15.32:0/2327210467 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb6400eea0 con 0x7fdb5c0044d0 2020-12-05T17:37:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb7b59e700 1 -- 172.21.15.32:0/2327210467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb5c0044d0 msgr2=0x7fdb5c004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb7b59e700 1 --2- 172.21.15.32:0/2327210467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb5c0044d0 0x7fdb5c004950 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fdb5c0054a0 tx=0x7fdb64007bf0).stop 2020-12-05T17:37:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb7ad9d700 1 -- 172.21.15.32:0/2327210467 reap_dead start 2020-12-05T17:37:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb7b59e700 1 -- 172.21.15.32:0/2327210467 shutdown_connections 2020-12-05T17:37:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb7b59e700 1 -- 172.21.15.32:0/2327210467 >> 172.21.15.32:0/2327210467 conn(0x7fdb7c0b5530 msgr2=0x7fdb7c0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb7b59e700 1 -- 172.21.15.32:0/2327210467 shutdown_connections 2020-12-05T17:37:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.414+0000 7fdb7b59e700 1 -- 172.21.15.32:0/2327210467 wait complete. 2020-12-05T17:37:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.418+0000 7fdb7b59e700 1 Processor -- start 2020-12-05T17:37:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.418+0000 7fdb7b59e700 1 -- start start 2020-12-05T17:37:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb7b59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb7c0b2900 0x7fdb7c0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb7b59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb7c13b5b0 0x7fdb7c1358b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb7b59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb7c135da0 0x7fdb7c13cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb7b59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c13e2e0 con 0x7fdb7c0b2900 2020-12-05T17:37:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb7a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb7c0b2900 0x7fdb7c0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb7b59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c13c1e0 con 0x7fdb7c13b5b0 2020-12-05T17:37:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb7b59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c13c740 con 0x7fdb7c135da0 2020-12-05T17:37:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb7a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb7c0b2900 0x7fdb7c0b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36202/0 (socket says 172.21.15.32:36202) 2020-12-05T17:37:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb7a59c700 1 -- 172.21.15.32:0/140616564 learned_addr learned my addr 172.21.15.32:0/140616564 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb71d9b700 1 --2- 172.21.15.32:0/140616564 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb7c13b5b0 0x7fdb7c1358b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb7ad9d700 1 --2- 172.21.15.32:0/140616564 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb7c135da0 0x7fdb7c13cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb7a59c700 1 -- 172.21.15.32:0/140616564 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb7c135da0 msgr2=0x7fdb7c13cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb7a59c700 1 --2- 172.21.15.32:0/140616564 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb7c135da0 0x7fdb7c13cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb7a59c700 1 -- 172.21.15.32:0/140616564 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb7c13b5b0 msgr2=0x7fdb7c1358b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb7a59c700 1 --2- 172.21.15.32:0/140616564 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb7c13b5b0 0x7fdb7c1358b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb7a59c700 1 -- 172.21.15.32:0/140616564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c13b340 con 0x7fdb7c0b2900 2020-12-05T17:37:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb7a59c700 1 --2- 172.21.15.32:0/140616564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb7c0b2900 0x7fdb7c0b2ce0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fdb6c008000 tx=0x7fdb6c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb73fff700 1 -- 172.21.15.32:0/140616564 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb6c00f980 con 0x7fdb7c0b2900 2020-12-05T17:37:06.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb73fff700 1 -- 172.21.15.32:0/140616564 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb6c00fae0 con 0x7fdb7c0b2900 2020-12-05T17:37:06.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb7b59e700 1 -- 172.21.15.32:0/140616564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb7c13c440 con 0x7fdb7c0b2900 2020-12-05T17:37:06.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb73fff700 1 -- 172.21.15.32:0/140616564 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb6c00fc50 con 0x7fdb7c0b2900 2020-12-05T17:37:06.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb7b59e700 1 -- 172.21.15.32:0/140616564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb7c13cba0 con 0x7fdb7c0b2900 2020-12-05T17:37:06.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb7b59e700 1 -- 172.21.15.32:0/140616564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb7c13f0f0 con 0x7fdb7c0b2900 2020-12-05T17:37:06.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.422+0000 7fdb73fff700 1 -- 172.21.15.32:0/140616564 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fdb6c024070 con 0x7fdb7c0b2900 2020-12-05T17:37:06.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.426+0000 7fdb73fff700 1 --2- 172.21.15.32:0/140616564 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb600553f0 0x7fdb600578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:06.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.426+0000 7fdb71d9b700 1 --2- 172.21.15.32:0/140616564 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb600553f0 0x7fdb600578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:06.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.426+0000 7fdb73fff700 1 -- 172.21.15.32:0/140616564 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdb6c06e9e0 con 0x7fdb7c0b2900 2020-12-05T17:37:06.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.426+0000 7fdb71d9b700 1 --2- 172.21.15.32:0/140616564 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb600553f0 0x7fdb600578b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fdb6401e000 tx=0x7fdb64007bf0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:06.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.430+0000 7fdb73fff700 1 -- 172.21.15.32:0/140616564 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fdb7c13f0f0 con 0x7fdb7c0b2900 2020-12-05T17:37:06.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.618+0000 7fdb7b59e700 1 -- 172.21.15.32:0/140616564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb7c13c900 con 0x7fdb7c0b2900 2020-12-05T17:37:06.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.618+0000 7fdb73fff700 1 -- 172.21.15.32:0/140616564 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+2137 (secure 0 0 0) 0x7fdb7c13c900 con 0x7fdb7c0b2900 2020-12-05T17:37:06.622 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:06.622 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5411,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/3995477718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3995477718},{"type":"v1","addr":"172.21.15.61:6833","nonce":3995477718}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:37:01.403820+0000","modified":"2020-12-05T17:37:02.406060+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5976},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5976":{"gid":5976,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/738464306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":738464306},{"type":"v1","addr":"172.21.15.32:6835","nonce":738464306}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-12-05T17:37:06.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.622+0000 7fdb7b59e700 1 -- 172.21.15.32:0/140616564 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb600553f0 msgr2=0x7fdb600578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:06.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.622+0000 7fdb7b59e700 1 --2- 172.21.15.32:0/140616564 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb600553f0 0x7fdb600578b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fdb6401e000 tx=0x7fdb64007bf0).stop 2020-12-05T17:37:06.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.622+0000 7fdb7b59e700 1 -- 172.21.15.32:0/140616564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb7c0b2900 msgr2=0x7fdb7c0b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:06.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.622+0000 7fdb7b59e700 1 --2- 172.21.15.32:0/140616564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb7c0b2900 0x7fdb7c0b2ce0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fdb6c008000 tx=0x7fdb6c008040).stop 2020-12-05T17:37:06.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.622+0000 7fdb7b59e700 1 -- 172.21.15.32:0/140616564 shutdown_connections 2020-12-05T17:37:06.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.622+0000 7fdb7b59e700 1 --2- 172.21.15.32:0/140616564 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb600553f0 0x7fdb600578b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:06.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.622+0000 7fdb7b59e700 1 --2- 172.21.15.32:0/140616564 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb7c135da0 0x7fdb7c13cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:06.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.622+0000 7fdb7b59e700 1 --2- 172.21.15.32:0/140616564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb7c0b2900 0x7fdb7c0b2ce0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:06.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.622+0000 7fdb7b59e700 1 --2- 172.21.15.32:0/140616564 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb7c13b5b0 0x7fdb7c1358b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:06.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.622+0000 7fdb7b59e700 1 -- 172.21.15.32:0/140616564 >> 172.21.15.32:0/140616564 conn(0x7fdb7c0b5530 msgr2=0x7fdb7c0b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:06.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.622+0000 7fdb7b59e700 1 -- 172.21.15.32:0/140616564 shutdown_connections 2020-12-05T17:37:06.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:06.622+0000 7fdb7b59e700 1 -- 172.21.15.32:0/140616564 wait complete. 2020-12-05T17:37:06.627 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 27 2020-12-05T17:37:06.838 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:37:01.403820+0000', 'modified': '2020-12-05T17:37:02.406060+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5976}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5976': {'gid': 5976, 'name': 'a', 'rank': 0, 'incarnation': 26, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/738464306', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 738464306}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 738464306}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:37:06.838 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:37:06.839 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:37:06.972 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":5976,"want_state":"up:active","state":"up:active","rank_uptime":5.5576676919999999,"mdsmap_epoch":27,"osdmap_epoch":62,"osdmap_epoch_barrier":62,"uptime":9.7859934709999994} 2020-12-05T17:37:06.973 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 5976, "mdsmap_epoch": 27, "osdmap_epoch": 62, "osdmap_epoch_barrier": 62, "rank_uptime": 5.557667692, "state": "up:active", "uptime": 9.785993471, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:37:06.973 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:37:06.974 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-12-05T17:37:06.974 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-12-05T17:37:06.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:06.975 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:37:06.983 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:37:07.040 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:37:07.040 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:37:07.096 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:37:07.097 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:37:07.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:07.098 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:37:07.144 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:37:07.144 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:37:07.188 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:37:07.204 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:37:07.204 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:37:07.261 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:37:07.263 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:37:07.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:07.264 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:37:07.267 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:2020-12-05T17:37:07.262+0000 7f5d3bf3b0c0 -1 init, newargv = 0x55682a29c8f0 newargc=9 2020-12-05T17:37:07.268 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[10896]: starting ceph client 2020-12-05T17:37:07.276 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[10896]: starting fuse 2020-12-05T17:37:08.375 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:37:08.391 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:37:08.392 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:37:08.448 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:37:08.449 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:37:08.450 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:08.450 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:37:08.495 INFO:teuthology.orchestra.run.smithi036.stdout:49 2020-12-05T17:37:08.496 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-12-05T17:37:08.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:08.497 INFO:teuthology.orchestra.run.smithi036:> sudo python3 -c ' 2020-12-05T17:37:08.497 INFO:teuthology.orchestra.run.smithi036:> import glob 2020-12-05T17:37:08.497 INFO:teuthology.orchestra.run.smithi036:> import re 2020-12-05T17:37:08.498 INFO:teuthology.orchestra.run.smithi036:> import os 2020-12-05T17:37:08.498 INFO:teuthology.orchestra.run.smithi036:> import subprocess 2020-12-05T17:37:08.498 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:37:08.499 INFO:teuthology.orchestra.run.smithi036:> def find_socket(client_name): 2020-12-05T17:37:08.499 INFO:teuthology.orchestra.run.smithi036:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-12-05T17:37:08.500 INFO:teuthology.orchestra.run.smithi036:> files = glob.glob(asok_path) 2020-12-05T17:37:08.500 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:37:08.500 INFO:teuthology.orchestra.run.smithi036:> # Given a non-glob path, it better be there 2020-12-05T17:37:08.501 INFO:teuthology.orchestra.run.smithi036:> if "*" not in asok_path: 2020-12-05T17:37:08.501 INFO:teuthology.orchestra.run.smithi036:> assert(len(files) == 1) 2020-12-05T17:37:08.501 INFO:teuthology.orchestra.run.smithi036:> return files[0] 2020-12-05T17:37:08.502 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:37:08.502 INFO:teuthology.orchestra.run.smithi036:> for f in files: 2020-12-05T17:37:08.502 INFO:teuthology.orchestra.run.smithi036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:37:08.503 INFO:teuthology.orchestra.run.smithi036:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:37:08.503 INFO:teuthology.orchestra.run.smithi036:> return f 2020-12-05T17:37:08.504 INFO:teuthology.orchestra.run.smithi036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:37:08.504 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:37:08.504 INFO:teuthology.orchestra.run.smithi036:> print(find_socket("client.0")) 2020-12-05T17:37:08.505 INFO:teuthology.orchestra.run.smithi036:> ' 2020-12-05T17:37:08.587 INFO:teuthology.orchestra.run.smithi036.stdout:/var/run/ceph/ceph-client.0.10896.asok 2020-12-05T17:37:08.591 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.10896.asok 2020-12-05T17:37:08.592 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:08.592 INFO:teuthology.orchestra.run.smithi036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.10896.asok status 2020-12-05T17:37:08.693 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2020-12-05T17:37:08.694 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": { 2020-12-05T17:37:08.694 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:37:08.694 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:37:08.694 INFO:teuthology.orchestra.run.smithi036.stdout: "entity_id": "0", 2020-12-05T17:37:08.694 INFO:teuthology.orchestra.run.smithi036.stdout: "hostname": "smithi036", 2020-12-05T17:37:08.695 INFO:teuthology.orchestra.run.smithi036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-12-05T17:37:08.695 INFO:teuthology.orchestra.run.smithi036.stdout: "pid": "10896", 2020-12-05T17:37:08.695 INFO:teuthology.orchestra.run.smithi036.stdout: "root": "/" 2020-12-05T17:37:08.695 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:37:08.695 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_count": 0, 2020-12-05T17:37:08.696 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_pinned_count": 0, 2020-12-05T17:37:08.696 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 5540, 2020-12-05T17:37:08.696 INFO:teuthology.orchestra.run.smithi036.stdout: "inst": { 2020-12-05T17:37:08.696 INFO:teuthology.orchestra.run.smithi036.stdout: "name": { 2020-12-05T17:37:08.696 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "client", 2020-12-05T17:37:08.697 INFO:teuthology.orchestra.run.smithi036.stdout: "num": 5540 2020-12-05T17:37:08.697 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:37:08.697 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:37:08.697 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:37:08.697 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:37:08.698 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 781516587 2020-12-05T17:37:08.698 INFO:teuthology.orchestra.run.smithi036.stdout: } 2020-12-05T17:37:08.698 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:37:08.698 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:37:08.698 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:37:08.699 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:37:08.699 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 781516587 2020-12-05T17:37:08.699 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:37:08.699 INFO:teuthology.orchestra.run.smithi036.stdout: "inst_str": "client.5540 172.21.15.36:0/781516587", 2020-12-05T17:37:08.699 INFO:teuthology.orchestra.run.smithi036.stdout: "addr_str": "172.21.15.36:0/781516587", 2020-12-05T17:37:08.699 INFO:teuthology.orchestra.run.smithi036.stdout: "inode_count": 1, 2020-12-05T17:37:08.700 INFO:teuthology.orchestra.run.smithi036.stdout: "mds_epoch": 27, 2020-12-05T17:37:08.700 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch": 62, 2020-12-05T17:37:08.700 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:37:08.700 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklisted": false 2020-12-05T17:37:08.701 INFO:teuthology.orchestra.run.smithi036.stdout:} 2020-12-05T17:37:08.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:08.703 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:37:08.704 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:37:08.710 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:37:08.711 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:37:08.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:08.712 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:37:08.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:08.772 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:37:08.772 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:37:08.821 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:37:08.822 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:37:08.823 INFO:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-12-05T17:37:08.823 INFO:teuthology.orchestra.run.smithi036:> import sys 2020-12-05T17:37:08.824 INFO:teuthology.orchestra.run.smithi036:> import time 2020-12-05T17:37:08.824 INFO:teuthology.orchestra.run.smithi036:> import os 2020-12-05T17:37:08.825 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:37:08.825 INFO:teuthology.orchestra.run.smithi036:> n = 1000 2020-12-05T17:37:08.826 INFO:teuthology.orchestra.run.smithi036:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2020-12-05T17:37:08.826 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:37:08.826 INFO:teuthology.orchestra.run.smithi036:> if not os.path.exists(os.path.dirname(abs_path)): 2020-12-05T17:37:08.827 INFO:teuthology.orchestra.run.smithi036:> os.makedirs(os.path.dirname(abs_path)) 2020-12-05T17:37:08.827 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:37:08.828 INFO:teuthology.orchestra.run.smithi036:> for i in range(0, n): 2020-12-05T17:37:08.828 INFO:teuthology.orchestra.run.smithi036:> fname = "{0}_{1}".format(abs_path, i) 2020-12-05T17:37:08.828 INFO:teuthology.orchestra.run.smithi036:> with open(fname, '"'"'w'"'"') as f: 2020-12-05T17:37:08.829 INFO:teuthology.orchestra.run.smithi036:> f.write('"'"'content'"'"') 2020-12-05T17:37:08.829 INFO:teuthology.orchestra.run.smithi036:> if True: 2020-12-05T17:37:08.830 INFO:teuthology.orchestra.run.smithi036:> f.flush() 2020-12-05T17:37:08.830 INFO:teuthology.orchestra.run.smithi036:> os.fsync(f.fileno()) 2020-12-05T17:37:08.830 INFO:teuthology.orchestra.run.smithi036:> ' 2020-12-05T17:37:15.473 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:37:15.475 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:37:15.478 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:37:15.480 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:37:15.482 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:37:22.555 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:37:22.678 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.677+0000 7f1aecd9a700 1 Processor -- start 2020-12-05T17:37:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.677+0000 7f1aecd9a700 1 -- start start 2020-12-05T17:37:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.677+0000 7f1aecd9a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ae80b1fc0 0x7f1ae80af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:22.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.677+0000 7f1aecd9a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1ae80ab710 0x7f1ae8061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:22.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.677+0000 7f1aecd9a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ae8146a90 con 0x7f1ae8147400 2020-12-05T17:37:22.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.677+0000 7f1aecd9a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ae80b0110 con 0x7f1ae80b1fc0 2020-12-05T17:37:22.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.677+0000 7f1aecd9a700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ae80b23a0 con 0x7f1ae80ab710 2020-12-05T17:37:22.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.677+0000 7f1ae7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1ae8147400 0x7f1ae8145d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38862/0 (socket says 172.21.15.32:38862) 2020-12-05T17:37:22.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.677+0000 7f1ae77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ae80b1fc0 0x7f1ae80af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:22.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.677+0000 7f1ae7fff700 1 -- 172.21.15.32:0/130205432 learned_addr learned my addr 172.21.15.32:0/130205432 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:22.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.681+0000 7f1ae6ffd700 1 --2- 172.21.15.32:0/130205432 >> v2:172.21.15.61:3300/0 conn(0x7f1ae80ab710 0x7f1ae8061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:22.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.681+0000 7f1ae67fc700 1 -- 172.21.15.32:0/130205432 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 569087973 0 0) 0x7f1ae8146a90 con 0x7f1ae8147400 2020-12-05T17:37:22.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.681+0000 7f1ae67fc700 1 -- 172.21.15.32:0/130205432 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ac8003230 con 0x7f1ae8147400 2020-12-05T17:37:22.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.681+0000 7f1ae67fc700 1 -- 172.21.15.32:0/130205432 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2908612327 0 0) 0x7f1ac8003230 con 0x7f1ae8147400 2020-12-05T17:37:22.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.681+0000 7f1ae77fe700 1 -- 172.21.15.32:0/130205432 >> v2:172.21.15.61:3300/0 conn(0x7f1ae80ab710 msgr2=0x7f1ae8061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:22.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.681+0000 7f1ae77fe700 1 --2- 172.21.15.32:0/130205432 >> v2:172.21.15.61:3300/0 conn(0x7f1ae80ab710 0x7f1ae8061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:22.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.681+0000 7f1ae77fe700 1 -- 172.21.15.32:0/130205432 >> v1:172.21.15.32:6789/0 conn(0x7f1ae8147400 legacy=0x7f1ae8145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:22.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.681+0000 7f1ae77fe700 1 -- 172.21.15.32:0/130205432 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae8146cc0 con 0x7f1ae80b1fc0 2020-12-05T17:37:22.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.681+0000 7f1ae77fe700 1 --2- 172.21.15.32:0/130205432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae80b1fc0 0x7f1ae80af230 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f1ae0001bf0 tx=0x7f1ad0006dc0).ready entity=mon.0 client_cookie=d812eff42646e84e server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:22.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.681+0000 7f1ae67fc700 1 -- 172.21.15.32:0/130205432 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ae0001a50 con 0x7f1ae80b1fc0 2020-12-05T17:37:22.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.681+0000 7f1ae67fc700 1 -- 172.21.15.32:0/130205432 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad000d5c0 con 0x7f1ae80b1fc0 2020-12-05T17:37:22.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.681+0000 7f1ae67fc700 1 -- 172.21.15.32:0/130205432 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ad000d730 con 0x7f1ae80b1fc0 2020-12-05T17:37:22.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.681+0000 7f1aecd9a700 1 -- 172.21.15.32:0/130205432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae80b1fc0 msgr2=0x7f1ae80af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:22.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.681+0000 7f1aecd9a700 1 --2- 172.21.15.32:0/130205432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae80b1fc0 0x7f1ae80af230 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f1ae0001bf0 tx=0x7f1ad0006dc0).stop 2020-12-05T17:37:22.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.681+0000 7f1aecd9a700 1 -- 172.21.15.32:0/130205432 shutdown_connections 2020-12-05T17:37:22.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.681+0000 7f1aecd9a700 1 --2- 172.21.15.32:0/130205432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae80b1fc0 0x7f1ae80af230 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:22.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.681+0000 7f1aecd9a700 1 --2- 172.21.15.32:0/130205432 >> v2:172.21.15.61:3300/0 conn(0x7f1ae80ab710 0x7f1ae8061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:22.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.681+0000 7f1aecd9a700 1 -- 172.21.15.32:0/130205432 >> 172.21.15.32:0/130205432 conn(0x7f1ae80b5530 msgr2=0x7f1ae80b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:22.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.681+0000 7f1aecd9a700 1 -- 172.21.15.32:0/130205432 shutdown_connections 2020-12-05T17:37:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.681+0000 7f1aecd9a700 1 -- 172.21.15.32:0/130205432 wait complete. 2020-12-05T17:37:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.681+0000 7f1aecd9a700 1 Processor -- start 2020-12-05T17:37:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.681+0000 7f1aecd9a700 1 -- start start 2020-12-05T17:37:22.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.685+0000 7f1aecd9a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae80ab710 0x7f1ae80ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:22.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.685+0000 7f1aecd9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ae80b1fc0 0x7f1ae8149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:22.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.685+0000 7f1aecd9a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ae8147400 0x7f1ae80ad170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:22.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.685+0000 7f1aecd9a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae80ac650 con 0x7f1ae80ab710 2020-12-05T17:37:22.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.685+0000 7f1aecd9a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae8148de0 con 0x7f1ae80b1fc0 2020-12-05T17:37:22.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.685+0000 7f1aecd9a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae814d280 con 0x7f1ae8147400 2020-12-05T17:37:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.685+0000 7f1ae77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae80ab710 0x7f1ae80ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.685+0000 7f1ae77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae80ab710 0x7f1ae80ae4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36218/0 (socket says 172.21.15.32:36218) 2020-12-05T17:37:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.685+0000 7f1ae77fe700 1 -- 172.21.15.32:0/3185560366 learned_addr learned my addr 172.21.15.32:0/3185560366 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.685+0000 7f1ae6ffd700 1 --2- 172.21.15.32:0/3185560366 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ae80b1fc0 0x7f1ae8149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.685+0000 7f1ae7fff700 1 --2- 172.21.15.32:0/3185560366 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ae8147400 0x7f1ae80ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.685+0000 7f1ae77fe700 1 -- 172.21.15.32:0/3185560366 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ae8147400 msgr2=0x7f1ae80ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.685+0000 7f1ae77fe700 1 --2- 172.21.15.32:0/3185560366 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ae8147400 0x7f1ae80ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.685+0000 7f1ae77fe700 1 -- 172.21.15.32:0/3185560366 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ae80b1fc0 msgr2=0x7f1ae8149210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.685+0000 7f1ae77fe700 1 --2- 172.21.15.32:0/3185560366 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ae80b1fc0 0x7f1ae8149210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.685+0000 7f1ae77fe700 1 -- 172.21.15.32:0/3185560366 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae814d520 con 0x7f1ae80ab710 2020-12-05T17:37:22.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.685+0000 7f1ae77fe700 1 --2- 172.21.15.32:0/3185560366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae80ab710 0x7f1ae80ae4c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7f1ae0001bf0 tx=0x7f1ad0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:22.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.685+0000 7f1ae4ff9700 1 -- 172.21.15.32:0/3185560366 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ad00118e0 con 0x7f1ae80ab710 2020-12-05T17:37:22.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.685+0000 7f1aecd9a700 1 -- 172.21.15.32:0/3185560366 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae80aed90 con 0x7f1ae80ab710 2020-12-05T17:37:22.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.685+0000 7f1aecd9a700 1 -- 172.21.15.32:0/3185560366 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae8149a90 con 0x7f1ae80ab710 2020-12-05T17:37:22.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.685+0000 7f1ae4ff9700 1 -- 172.21.15.32:0/3185560366 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad0011a40 con 0x7f1ae80ab710 2020-12-05T17:37:22.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.685+0000 7f1ae4ff9700 1 -- 172.21.15.32:0/3185560366 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ad0011de0 con 0x7f1ae80ab710 2020-12-05T17:37:22.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.685+0000 7f1aecd9a700 1 -- 172.21.15.32:0/3185560366 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac00033e0 con 0x7f1ae80ab710 2020-12-05T17:37:22.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.689+0000 7f1ae4ff9700 1 -- 172.21.15.32:0/3185560366 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1ad0017030 con 0x7f1ae80ab710 2020-12-05T17:37:22.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.689+0000 7f1ae4ff9700 1 --2- 172.21.15.32:0/3185560366 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1acc055470 0x7f1acc057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:22.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.689+0000 7f1ae4ff9700 1 -- 172.21.15.32:0/3185560366 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1ad006ec00 con 0x7f1ae80ab710 2020-12-05T17:37:22.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.689+0000 7f1ae6ffd700 1 --2- 172.21.15.32:0/3185560366 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1acc055470 0x7f1acc057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:22.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.689+0000 7f1ae6ffd700 1 --2- 172.21.15.32:0/3185560366 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1acc055470 0x7f1acc057930 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f1ad800b000 tx=0x7f1ad8006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:22.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.697+0000 7f1ae4ff9700 1 -- 172.21.15.32:0/3185560366 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1ad0074020 con 0x7f1ae80ab710 2020-12-05T17:37:22.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.885+0000 7f1aecd9a700 1 -- 172.21.15.32:0/3185560366 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ac0003820 con 0x7f1ae80ab710 2020-12-05T17:37:22.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.885+0000 7f1ae4ff9700 1 -- 172.21.15.32:0/3185560366 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f1ad0022090 con 0x7f1ae80ab710 2020-12-05T17:37:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:22.887 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5411,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/3995477718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3995477718},{"type":"v1","addr":"172.21.15.61:6833","nonce":3995477718}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:37:01.403820+0000","modified":"2020-12-05T17:37:02.406060+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5976},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5976":{"gid":5976,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/738464306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":738464306},{"type":"v1","addr":"172.21.15.32:6835","nonce":738464306}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-12-05T17:37:22.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.889+0000 7f1aecd9a700 1 -- 172.21.15.32:0/3185560366 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1acc055470 msgr2=0x7f1acc057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:22.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.889+0000 7f1aecd9a700 1 --2- 172.21.15.32:0/3185560366 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1acc055470 0x7f1acc057930 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f1ad800b000 tx=0x7f1ad8006dc0).stop 2020-12-05T17:37:22.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.889+0000 7f1aecd9a700 1 -- 172.21.15.32:0/3185560366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae80ab710 msgr2=0x7f1ae80ae4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:22.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.889+0000 7f1aecd9a700 1 --2- 172.21.15.32:0/3185560366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae80ab710 0x7f1ae80ae4c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7f1ae0001bf0 tx=0x7f1ad0015000).stop 2020-12-05T17:37:22.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.889+0000 7f1aecd9a700 1 -- 172.21.15.32:0/3185560366 shutdown_connections 2020-12-05T17:37:22.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.889+0000 7f1aecd9a700 1 --2- 172.21.15.32:0/3185560366 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1acc055470 0x7f1acc057930 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:22.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.889+0000 7f1aecd9a700 1 --2- 172.21.15.32:0/3185560366 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ae8147400 0x7f1ae80ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:22.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.889+0000 7f1aecd9a700 1 --2- 172.21.15.32:0/3185560366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae80ab710 0x7f1ae80ae4c0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:22.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.889+0000 7f1aecd9a700 1 --2- 172.21.15.32:0/3185560366 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ae80b1fc0 0x7f1ae8149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:22.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.889+0000 7f1aecd9a700 1 -- 172.21.15.32:0/3185560366 >> 172.21.15.32:0/3185560366 conn(0x7f1ae80b5530 msgr2=0x7f1ae805f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:22.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.889+0000 7f1aecd9a700 1 -- 172.21.15.32:0/3185560366 shutdown_connections 2020-12-05T17:37:22.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:22.889+0000 7f1aecd9a700 1 -- 172.21.15.32:0/3185560366 wait complete. 2020-12-05T17:37:22.891 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 27 2020-12-05T17:37:22.900 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_caps 20 2020-12-05T17:37:23.021 INFO:teuthology.orchestra.run.smithi032.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2020-12-05T17:37:23.021 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2020-12-05T17:37:23.022 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:37:23.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.129+0000 7f49fcd47700 1 Processor -- start 2020-12-05T17:37:23.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.129+0000 7f49fcd47700 1 -- start start 2020-12-05T17:37:23.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.129+0000 7f49fcd47700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49f80b1980 0x7f49f8146580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.129+0000 7f49fcd47700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f49f8146a70 0x7f49f8066070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.129+0000 7f49fcd47700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f8143000 con 0x7f49f805f6f0 2020-12-05T17:37:23.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.129+0000 7f49fcd47700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49f805fad0 con 0x7f49f8146a70 2020-12-05T17:37:23.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.129+0000 7f49fcd47700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f80b2f80 con 0x7f49f80b1980 2020-12-05T17:37:23.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.129+0000 7f49f6ffd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49f80b1980 0x7f49f8146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.129+0000 7f49f6ffd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49f80b1980 0x7f49f8146580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:33016/0 (socket says 172.21.15.32:33016) 2020-12-05T17:37:23.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.129+0000 7f49f6ffd700 1 -- 172.21.15.32:0/390783977 learned_addr learned my addr 172.21.15.32:0/390783977 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:23.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.129+0000 7f49f7fff700 1 --2- 172.21.15.32:0/390783977 >> v2:172.21.15.61:3300/0 conn(0x7f49f8146a70 0x7f49f8066070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.129+0000 7f49f6ffd700 1 -- 172.21.15.32:0/390783977 >> v2:172.21.15.61:3300/0 conn(0x7f49f8146a70 msgr2=0x7f49f8066070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:23.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.129+0000 7f49f6ffd700 1 --2- 172.21.15.32:0/390783977 >> v2:172.21.15.61:3300/0 conn(0x7f49f8146a70 0x7f49f8066070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.129+0000 7f49f6ffd700 1 -- 172.21.15.32:0/390783977 >> v1:172.21.15.61:6789/0 conn(0x7f49f805f6f0 legacy=0x7f49f81428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:23.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.129+0000 7f49f6ffd700 1 -- 172.21.15.32:0/390783977 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f8145e40 con 0x7f49f80b1980 2020-12-05T17:37:23.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.129+0000 7f49f7fff700 1 --2- 172.21.15.32:0/390783977 >> v2:172.21.15.61:3300/0 conn(0x7f49f8146a70 0x7f49f8066070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:37:23.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.129+0000 7f49f6ffd700 1 --2- 172.21.15.32:0/390783977 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49f80b1980 0x7f49f8146580 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f49ec002f30 tx=0x7f49ec006e80).ready entity=mon.2 client_cookie=573a0b4bea90880a server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:23.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.133+0000 7f49f67fc700 1 -- 172.21.15.32:0/390783977 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49ec0097f0 con 0x7f49f80b1980 2020-12-05T17:37:23.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.133+0000 7f49f67fc700 1 -- 172.21.15.32:0/390783977 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ec009950 con 0x7f49f80b1980 2020-12-05T17:37:23.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.133+0000 7f49f67fc700 1 -- 172.21.15.32:0/390783977 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49ec009cb0 con 0x7f49f80b1980 2020-12-05T17:37:23.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.133+0000 7f49fcd47700 1 -- 172.21.15.32:0/390783977 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49f80b1980 msgr2=0x7f49f8146580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:23.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.133+0000 7f49fcd47700 1 --2- 172.21.15.32:0/390783977 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49f80b1980 0x7f49f8146580 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f49ec002f30 tx=0x7f49ec006e80).stop 2020-12-05T17:37:23.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.133+0000 7f49fcd47700 1 -- 172.21.15.32:0/390783977 shutdown_connections 2020-12-05T17:37:23.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.133+0000 7f49fcd47700 1 --2- 172.21.15.32:0/390783977 >> v2:172.21.15.61:3300/0 conn(0x7f49f8146a70 0x7f49f8066070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.133+0000 7f49fcd47700 1 --2- 172.21.15.32:0/390783977 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49f80b1980 0x7f49f8146580 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.133+0000 7f49fcd47700 1 -- 172.21.15.32:0/390783977 >> 172.21.15.32:0/390783977 conn(0x7f49f80604c0 msgr2=0x7f49f805c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:23.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.133+0000 7f49fcd47700 1 -- 172.21.15.32:0/390783977 shutdown_connections 2020-12-05T17:37:23.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.133+0000 7f49fcd47700 1 -- 172.21.15.32:0/390783977 wait complete. 2020-12-05T17:37:23.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.133+0000 7f49fcd47700 1 Processor -- start 2020-12-05T17:37:23.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.133+0000 7f49fcd47700 1 -- start start 2020-12-05T17:37:23.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.133+0000 7f49fcd47700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49f805f6f0 0x7f49f80b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.133+0000 7f49fcd47700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49f80b1980 0x7f49f8064170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.133+0000 7f49fcd47700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f8146a70 0x7f49f8151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.133+0000 7f49fcd47700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f81516a0 con 0x7f49f8146a70 2020-12-05T17:37:23.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.133+0000 7f49fcd47700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f8063c60 con 0x7f49f805f6f0 2020-12-05T17:37:23.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.133+0000 7f49fcd47700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f8151900 con 0x7f49f80b1980 2020-12-05T17:37:23.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.137+0000 7f49f7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f8146a70 0x7f49f8151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.137+0000 7f49f7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f8146a70 0x7f49f8151da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36234/0 (socket says 172.21.15.32:36234) 2020-12-05T17:37:23.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.137+0000 7f49f7fff700 1 -- 172.21.15.32:0/3254858561 learned_addr learned my addr 172.21.15.32:0/3254858561 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:23.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.137+0000 7f49f6ffd700 1 --2- 172.21.15.32:0/3254858561 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49f80b1980 0x7f49f8064170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.137+0000 7f49f77fe700 1 --2- 172.21.15.32:0/3254858561 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49f805f6f0 0x7f49f80b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.137+0000 7f49f7fff700 1 -- 172.21.15.32:0/3254858561 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49f80b1980 msgr2=0x7f49f8064170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:23.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.137+0000 7f49f7fff700 1 --2- 172.21.15.32:0/3254858561 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49f80b1980 0x7f49f8064170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.137+0000 7f49f7fff700 1 -- 172.21.15.32:0/3254858561 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49f805f6f0 msgr2=0x7f49f80b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:23.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.137+0000 7f49f7fff700 1 --2- 172.21.15.32:0/3254858561 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49f805f6f0 0x7f49f80b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.137+0000 7f49f7fff700 1 -- 172.21.15.32:0/3254858561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f8064660 con 0x7f49f8146a70 2020-12-05T17:37:23.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.137+0000 7f49f7fff700 1 --2- 172.21.15.32:0/3254858561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f8146a70 0x7f49f8151da0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f49f000c000 tx=0x7f49f00091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:23.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.137+0000 7f49f4ff9700 1 -- 172.21.15.32:0/3254858561 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49f000cb00 con 0x7f49f8146a70 2020-12-05T17:37:23.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.137+0000 7f49f4ff9700 1 -- 172.21.15.32:0/3254858561 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f000cc60 con 0x7f49f8146a70 2020-12-05T17:37:23.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.137+0000 7f49f4ff9700 1 -- 172.21.15.32:0/3254858561 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49f00123f0 con 0x7f49f8146a70 2020-12-05T17:37:23.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.137+0000 7f49fcd47700 1 -- 172.21.15.32:0/3254858561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f80638e0 con 0x7f49f8146a70 2020-12-05T17:37:23.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.137+0000 7f49fcd47700 1 -- 172.21.15.32:0/3254858561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f8065d70 con 0x7f49f8146a70 2020-12-05T17:37:23.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.137+0000 7f49fcd47700 1 -- 172.21.15.32:0/3254858561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f8151ac0 con 0x7f49f8146a70 2020-12-05T17:37:23.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.137+0000 7f49f4ff9700 1 -- 172.21.15.32:0/3254858561 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f49f0022070 con 0x7f49f8146a70 2020-12-05T17:37:23.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.137+0000 7f49f4ff9700 1 --2- 172.21.15.32:0/3254858561 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49d80554e0 0x7f49d80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.137+0000 7f49f4ff9700 1 -- 172.21.15.32:0/3254858561 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f49f006e8a0 con 0x7f49f8146a70 2020-12-05T17:37:23.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.137+0000 7f49f77fe700 1 --2- 172.21.15.32:0/3254858561 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49d80554e0 0x7f49d80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.141+0000 7f49f77fe700 1 --2- 172.21.15.32:0/3254858561 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49d80554e0 0x7f49d80579a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f49e800d000 tx=0x7f49e8000b80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:23.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.145+0000 7f49f4ff9700 1 -- 172.21.15.32:0/3254858561 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f49f0044a40 con 0x7f49f8146a70 2020-12-05T17:37:23.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.333+0000 7f49fcd47700 1 -- 172.21.15.32:0/3254858561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49f8057050 con 0x7f49f8146a70 2020-12-05T17:37:23.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.333+0000 7f49f4ff9700 1 -- 172.21.15.32:0/3254858561 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f49f0016090 con 0x7f49f8146a70 2020-12-05T17:37:23.334 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:23.335 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5411,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/3995477718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3995477718},{"type":"v1","addr":"172.21.15.61:6833","nonce":3995477718}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:37:01.403820+0000","modified":"2020-12-05T17:37:02.406060+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5976},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5976":{"gid":5976,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/738464306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":738464306},{"type":"v1","addr":"172.21.15.32:6835","nonce":738464306}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-12-05T17:37:23.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.337+0000 7f49fcd47700 1 -- 172.21.15.32:0/3254858561 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49d80554e0 msgr2=0x7f49d80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.337+0000 7f49fcd47700 1 --2- 172.21.15.32:0/3254858561 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49d80554e0 0x7f49d80579a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f49e800d000 tx=0x7f49e8000b80).stop 2020-12-05T17:37:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.337+0000 7f49fcd47700 1 -- 172.21.15.32:0/3254858561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f8146a70 msgr2=0x7f49f8151da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.337+0000 7f49fcd47700 1 --2- 172.21.15.32:0/3254858561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f8146a70 0x7f49f8151da0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f49f000c000 tx=0x7f49f00091a0).stop 2020-12-05T17:37:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.337+0000 7f49fcd47700 1 -- 172.21.15.32:0/3254858561 shutdown_connections 2020-12-05T17:37:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.337+0000 7f49fcd47700 1 --2- 172.21.15.32:0/3254858561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f8146a70 0x7f49f8151da0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.337+0000 7f49fcd47700 1 --2- 172.21.15.32:0/3254858561 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49f805f6f0 0x7f49f80b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.337+0000 7f49fcd47700 1 --2- 172.21.15.32:0/3254858561 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49d80554e0 0x7f49d80579a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.337+0000 7f49fcd47700 1 --2- 172.21.15.32:0/3254858561 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49f80b1980 0x7f49f8064170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.337+0000 7f49fcd47700 1 -- 172.21.15.32:0/3254858561 >> 172.21.15.32:0/3254858561 conn(0x7f49f80604c0 msgr2=0x7f49f8143d40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:23.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.337+0000 7f49fcd47700 1 -- 172.21.15.32:0/3254858561 shutdown_connections 2020-12-05T17:37:23.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.337+0000 7f49fcd47700 1 -- 172.21.15.32:0/3254858561 wait complete. 2020-12-05T17:37:23.339 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 27 2020-12-05T17:37:23.349 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_decay_threshold 400 2020-12-05T17:37:23.466 INFO:teuthology.orchestra.run.smithi032.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2020-12-05T17:37:23.466 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2020-12-05T17:37:23.467 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:37:23.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5a15f700 1 Processor -- start 2020-12-05T17:37:23.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5a15f700 1 -- start start 2020-12-05T17:37:23.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5a15f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c540b54b0 0x7f5c540b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5a15f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5c54141ad0 0x7f5c54141f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5a15f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c540adfe0 con 0x7f5c540afd20 2020-12-05T17:37:23.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5a15f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c540b0630 con 0x7f5c54141ad0 2020-12-05T17:37:23.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5a15f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c540ae210 con 0x7f5c540b54b0 2020-12-05T17:37:23.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5895c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5c540afd20 0x7f5c540b0f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38888/0 (socket says 172.21.15.32:38888) 2020-12-05T17:37:23.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5895c700 1 -- 172.21.15.32:0/812906819 learned_addr learned my addr 172.21.15.32:0/812906819 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:23.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5995e700 1 --2- 172.21.15.32:0/812906819 >> v2:172.21.15.32:3300/0 conn(0x7f5c54141ad0 0x7f5c54141f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5915d700 1 --2- 172.21.15.32:0/812906819 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c540b54b0 0x7f5c540b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c53fff700 1 -- 172.21.15.32:0/812906819 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1424213292 0 0) 0x7f5c540adfe0 con 0x7f5c540afd20 2020-12-05T17:37:23.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5915d700 1 -- 172.21.15.32:0/812906819 >> v2:172.21.15.32:3300/0 conn(0x7f5c54141ad0 msgr2=0x7f5c54141f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:23.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5915d700 1 --2- 172.21.15.32:0/812906819 >> v2:172.21.15.32:3300/0 conn(0x7f5c54141ad0 0x7f5c54141f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5915d700 1 -- 172.21.15.32:0/812906819 >> v1:172.21.15.32:6789/0 conn(0x7f5c540afd20 legacy=0x7f5c540b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:23.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5915d700 1 -- 172.21.15.32:0/812906819 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c540677f0 con 0x7f5c540b54b0 2020-12-05T17:37:23.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5915d700 1 --2- 172.21.15.32:0/812906819 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c540b54b0 0x7f5c540b0a90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f5c3c0032b0 tx=0x7f5c3c0091a0).ready entity=mon.2 client_cookie=a3056dfce6dcb4e5 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:23.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c53fff700 1 -- 172.21.15.32:0/812906819 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c3c00b9a0 con 0x7f5c540b54b0 2020-12-05T17:37:23.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c53fff700 1 -- 172.21.15.32:0/812906819 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c00bb00 con 0x7f5c540b54b0 2020-12-05T17:37:23.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c53fff700 1 -- 172.21.15.32:0/812906819 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c3c00bc70 con 0x7f5c540b54b0 2020-12-05T17:37:23.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5a15f700 1 -- 172.21.15.32:0/812906819 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c540b54b0 msgr2=0x7f5c540b0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:23.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5a15f700 1 --2- 172.21.15.32:0/812906819 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c540b54b0 0x7f5c540b0a90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f5c3c0032b0 tx=0x7f5c3c0091a0).stop 2020-12-05T17:37:23.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5a15f700 1 -- 172.21.15.32:0/812906819 shutdown_connections 2020-12-05T17:37:23.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5a15f700 1 --2- 172.21.15.32:0/812906819 >> v2:172.21.15.32:3300/0 conn(0x7f5c54141ad0 0x7f5c54141f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5a15f700 1 --2- 172.21.15.32:0/812906819 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c540b54b0 0x7f5c540b0a90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5a15f700 1 -- 172.21.15.32:0/812906819 >> 172.21.15.32:0/812906819 conn(0x7f5c5405bbb0 msgr2=0x7f5c54062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:23.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5a15f700 1 -- 172.21.15.32:0/812906819 shutdown_connections 2020-12-05T17:37:23.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.581+0000 7f5c5a15f700 1 -- 172.21.15.32:0/812906819 wait complete. 2020-12-05T17:37:23.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c5a15f700 1 Processor -- start 2020-12-05T17:37:23.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c5a15f700 1 -- start start 2020-12-05T17:37:23.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c5a15f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c540afd20 0x7f5c5410f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c5a15f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c540b54b0 0x7f5c54149110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c5a15f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c54141ad0 0x7f5c54154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c5a15f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c54146970 con 0x7f5c540afd20 2020-12-05T17:37:23.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c5a15f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c54145850 con 0x7f5c54141ad0 2020-12-05T17:37:23.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c5a15f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c54145450 con 0x7f5c540b54b0 2020-12-05T17:37:23.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c5915d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c540afd20 0x7f5c5410f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c5915d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c540afd20 0x7f5c5410f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36244/0 (socket says 172.21.15.32:36244) 2020-12-05T17:37:23.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c5915d700 1 -- 172.21.15.32:0/1279464974 learned_addr learned my addr 172.21.15.32:0/1279464974 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:23.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c5895c700 1 --2- 172.21.15.32:0/1279464974 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c540b54b0 0x7f5c54149110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c5995e700 1 --2- 172.21.15.32:0/1279464974 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c54141ad0 0x7f5c54154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c5915d700 1 -- 172.21.15.32:0/1279464974 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c540b54b0 msgr2=0x7f5c54149110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:23.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c5915d700 1 --2- 172.21.15.32:0/1279464974 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c540b54b0 0x7f5c54149110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c5915d700 1 -- 172.21.15.32:0/1279464974 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c54141ad0 msgr2=0x7f5c54154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:23.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c5915d700 1 --2- 172.21.15.32:0/1279464974 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c54141ad0 0x7f5c54154cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c5915d700 1 -- 172.21.15.32:0/1279464974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c54145610 con 0x7f5c540afd20 2020-12-05T17:37:23.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c5915d700 1 --2- 172.21.15.32:0/1279464974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c540afd20 0x7f5c5410f010 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f5c3c0032b0 tx=0x7f5c3c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:23.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c527fc700 1 -- 172.21.15.32:0/1279464974 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c3c01f3f0 con 0x7f5c540afd20 2020-12-05T17:37:23.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c527fc700 1 -- 172.21.15.32:0/1279464974 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c01f550 con 0x7f5c540afd20 2020-12-05T17:37:23.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c5a15f700 1 -- 172.21.15.32:0/1279464974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c541513e0 con 0x7f5c540afd20 2020-12-05T17:37:23.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c527fc700 1 -- 172.21.15.32:0/1279464974 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c3c01f8b0 con 0x7f5c540afd20 2020-12-05T17:37:23.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.585+0000 7f5c5a15f700 1 -- 172.21.15.32:0/1279464974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c541551a0 con 0x7f5c540afd20 2020-12-05T17:37:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.589+0000 7f5c5a15f700 1 -- 172.21.15.32:0/1279464974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c541557f0 con 0x7f5c540afd20 2020-12-05T17:37:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.589+0000 7f5c527fc700 1 -- 172.21.15.32:0/1279464974 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5c3c00f070 con 0x7f5c540afd20 2020-12-05T17:37:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.589+0000 7f5c527fc700 1 --2- 172.21.15.32:0/1279464974 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5c38055330 0x7f5c380577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.589+0000 7f5c527fc700 1 -- 172.21.15.32:0/1279464974 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5c3c06ee40 con 0x7f5c540afd20 2020-12-05T17:37:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.589+0000 7f5c5895c700 1 --2- 172.21.15.32:0/1279464974 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5c38055330 0x7f5c380577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.589+0000 7f5c5895c700 1 --2- 172.21.15.32:0/1279464974 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5c38055330 0x7f5c380577f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f5c4c00d000 tx=0x7f5c4c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:23.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.597+0000 7f5c527fc700 1 -- 172.21.15.32:0/1279464974 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5c3c044fe0 con 0x7f5c540afd20 2020-12-05T17:37:23.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.781+0000 7f5c5a15f700 1 -- 172.21.15.32:0/1279464974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c54057050 con 0x7f5c540afd20 2020-12-05T17:37:23.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.781+0000 7f5c527fc700 1 -- 172.21.15.32:0/1279464974 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f5c3c015070 con 0x7f5c540afd20 2020-12-05T17:37:23.784 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:23.784 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5411,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/3995477718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3995477718},{"type":"v1","addr":"172.21.15.61:6833","nonce":3995477718}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:37:01.403820+0000","modified":"2020-12-05T17:37:02.406060+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5976},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5976":{"gid":5976,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/738464306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":738464306},{"type":"v1","addr":"172.21.15.32:6835","nonce":738464306}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-12-05T17:37:23.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.785+0000 7f5c5a15f700 1 -- 172.21.15.32:0/1279464974 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5c38055330 msgr2=0x7f5c380577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:23.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.785+0000 7f5c5a15f700 1 --2- 172.21.15.32:0/1279464974 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5c38055330 0x7f5c380577f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f5c4c00d000 tx=0x7f5c4c009040).stop 2020-12-05T17:37:23.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.785+0000 7f5c5a15f700 1 -- 172.21.15.32:0/1279464974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c540afd20 msgr2=0x7f5c5410f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:23.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.785+0000 7f5c5a15f700 1 --2- 172.21.15.32:0/1279464974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c540afd20 0x7f5c5410f010 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f5c3c0032b0 tx=0x7f5c3c016000).stop 2020-12-05T17:37:23.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.785+0000 7f5c5a15f700 1 -- 172.21.15.32:0/1279464974 shutdown_connections 2020-12-05T17:37:23.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.785+0000 7f5c5a15f700 1 --2- 172.21.15.32:0/1279464974 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c54141ad0 0x7f5c54154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.785+0000 7f5c5a15f700 1 --2- 172.21.15.32:0/1279464974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c540afd20 0x7f5c5410f010 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.785+0000 7f5c5a15f700 1 --2- 172.21.15.32:0/1279464974 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5c38055330 0x7f5c380577f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.785+0000 7f5c5a15f700 1 --2- 172.21.15.32:0/1279464974 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c540b54b0 0x7f5c54149110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.785+0000 7f5c5a15f700 1 -- 172.21.15.32:0/1279464974 >> 172.21.15.32:0/1279464974 conn(0x7f5c5405bbb0 msgr2=0x7f5c540b0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.785+0000 7f5c5a15f700 1 -- 172.21.15.32:0/1279464974 shutdown_connections 2020-12-05T17:37:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.785+0000 7f5c5a15f700 1 -- 172.21.15.32:0/1279464974 wait complete. 2020-12-05T17:37:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 27 2020-12-05T17:37:23.802 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell mds.a cache drop 10 2020-12-05T17:37:23.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.925+0000 7f49c7fff700 1 Processor -- start 2020-12-05T17:37:23.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.925+0000 7f49c7fff700 1 -- start start 2020-12-05T17:37:23.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.929+0000 7f49c7fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f49c8061920 0x7f49c80aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.929+0000 7f49c7fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49c80621b0 0x7f49c813e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.929+0000 7f49c7fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c80b1380 con 0x7f49c8061d00 2020-12-05T17:37:23.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.929+0000 7f49c7fff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c8062ba0 con 0x7f49c8061920 2020-12-05T17:37:23.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.929+0000 7f49c7fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c80ae620 con 0x7f49c80621b0 2020-12-05T17:37:23.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.929+0000 7f49bffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f49c8061d00 0x7f49c8142200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38904/0 (socket says 172.21.15.32:38904) 2020-12-05T17:37:23.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.929+0000 7f49bffff700 1 -- 172.21.15.32:0/1855851292 learned_addr learned my addr 172.21.15.32:0/1855851292 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:23.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.929+0000 7f49c77fe700 1 --2- 172.21.15.32:0/1855851292 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49c80621b0 0x7f49c813e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.929+0000 7f49c67fc700 1 -- 172.21.15.32:0/1855851292 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1578894353 0 0) 0x7f49c80b1380 con 0x7f49c8061d00 2020-12-05T17:37:23.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.929+0000 7f49c67fc700 1 -- 172.21.15.32:0/1855851292 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49a8003230 con 0x7f49c8061d00 2020-12-05T17:37:23.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.929+0000 7f49c6ffd700 1 --2- 172.21.15.32:0/1855851292 >> v2:172.21.15.61:3300/0 conn(0x7f49c8061920 0x7f49c80aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.929+0000 7f49c77fe700 1 -- 172.21.15.32:0/1855851292 >> v2:172.21.15.61:3300/0 conn(0x7f49c8061920 msgr2=0x7f49c80aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:23.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.929+0000 7f49c77fe700 1 --2- 172.21.15.32:0/1855851292 >> v2:172.21.15.61:3300/0 conn(0x7f49c8061920 0x7f49c80aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.929+0000 7f49c77fe700 1 -- 172.21.15.32:0/1855851292 >> v1:172.21.15.32:6789/0 conn(0x7f49c8061d00 legacy=0x7f49c8142200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:23.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.929+0000 7f49c77fe700 1 -- 172.21.15.32:0/1855851292 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c80ae8c0 con 0x7f49c80621b0 2020-12-05T17:37:23.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.929+0000 7f49c77fe700 1 --2- 172.21.15.32:0/1855851292 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49c80621b0 0x7f49c813e000 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f49b0001870 tx=0x7f49c00091a0).ready entity=mon.2 client_cookie=4fc0466cf19f7ae9 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:23.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.933+0000 7f49c67fc700 1 -- 172.21.15.32:0/1855851292 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49b00016d0 con 0x7f49c80621b0 2020-12-05T17:37:23.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.933+0000 7f49c67fc700 1 -- 172.21.15.32:0/1855851292 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c000b8b0 con 0x7f49c80621b0 2020-12-05T17:37:23.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.933+0000 7f49c67fc700 1 -- 172.21.15.32:0/1855851292 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49c000ba20 con 0x7f49c80621b0 2020-12-05T17:37:23.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.933+0000 7f49c7fff700 1 -- 172.21.15.32:0/1855851292 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49c80621b0 msgr2=0x7f49c813e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:23.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.933+0000 7f49c7fff700 1 --2- 172.21.15.32:0/1855851292 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49c80621b0 0x7f49c813e000 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f49b0001870 tx=0x7f49c00091a0).stop 2020-12-05T17:37:23.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.933+0000 7f49c7fff700 1 -- 172.21.15.32:0/1855851292 shutdown_connections 2020-12-05T17:37:23.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.933+0000 7f49c7fff700 1 --2- 172.21.15.32:0/1855851292 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49c80621b0 0x7f49c813e000 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.933+0000 7f49c7fff700 1 --2- 172.21.15.32:0/1855851292 >> v2:172.21.15.61:3300/0 conn(0x7f49c8061920 0x7f49c80aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.933+0000 7f49c7fff700 1 -- 172.21.15.32:0/1855851292 >> 172.21.15.32:0/1855851292 conn(0x7f49c80b5620 msgr2=0x7f49c80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:23.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.933+0000 7f49c7fff700 1 -- 172.21.15.32:0/1855851292 shutdown_connections 2020-12-05T17:37:23.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.933+0000 7f49c7fff700 1 -- 172.21.15.32:0/1855851292 wait complete. 2020-12-05T17:37:23.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.933+0000 7f49c7fff700 1 Processor -- start 2020-12-05T17:37:23.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.933+0000 7f49c7fff700 1 -- start start 2020-12-05T17:37:23.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.937+0000 7f49c7fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49c8061920 0x7f49c8065590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.937+0000 7f49c7fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49c8061d00 0x7f49c813e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.937+0000 7f49c7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c80621b0 0x7f49c81528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.937+0000 7f49c7fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c80662a0 con 0x7f49c80621b0 2020-12-05T17:37:23.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.937+0000 7f49c7fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c813da10 con 0x7f49c8061920 2020-12-05T17:37:23.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.937+0000 7f49c7fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c813dcb0 con 0x7f49c8061d00 2020-12-05T17:37:23.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.937+0000 7f49c77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c80621b0 0x7f49c81528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.937+0000 7f49bffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49c8061d00 0x7f49c813e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.937+0000 7f49c6ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49c8061920 0x7f49c8065590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.937+0000 7f49c77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c80621b0 0x7f49c81528c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36262/0 (socket says 172.21.15.32:36262) 2020-12-05T17:37:23.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.937+0000 7f49bffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49c8061d00 0x7f49c813e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:33050/0 (socket says 172.21.15.32:33050) 2020-12-05T17:37:23.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.937+0000 7f49c77fe700 1 -- 172.21.15.32:0/1893383665 learned_addr learned my addr 172.21.15.32:0/1893383665 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:23.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.937+0000 7f49bffff700 1 -- 172.21.15.32:0/1893383665 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49c8061920 msgr2=0x7f49c8065590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:23.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.937+0000 7f49bffff700 1 --2- 172.21.15.32:0/1893383665 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49c8061920 0x7f49c8065590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.937+0000 7f49bffff700 1 -- 172.21.15.32:0/1893383665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c80621b0 msgr2=0x7f49c81528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:23.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.937+0000 7f49bffff700 1 --2- 172.21.15.32:0/1893383665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c80621b0 0x7f49c81528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.937+0000 7f49bffff700 1 -- 172.21.15.32:0/1893383665 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c813e640 con 0x7f49c8061d00 2020-12-05T17:37:23.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.937+0000 7f49bffff700 1 --2- 172.21.15.32:0/1893383665 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49c8061d00 0x7f49c813e150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f49b00013e0 tx=0x7f49b0001b00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:23.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.937+0000 7f49c4ff9700 1 -- 172.21.15.32:0/1893383665 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49b000ce60 con 0x7f49c8061d00 2020-12-05T17:37:23.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.937+0000 7f49c4ff9700 1 -- 172.21.15.32:0/1893383665 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b0007170 con 0x7f49c8061d00 2020-12-05T17:37:23.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.937+0000 7f49c7fff700 1 -- 172.21.15.32:0/1893383665 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c8065f20 con 0x7f49c8061d00 2020-12-05T17:37:23.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.937+0000 7f49c4ff9700 1 -- 172.21.15.32:0/1893383665 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49b000a710 con 0x7f49c8061d00 2020-12-05T17:37:23.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.937+0000 7f49c7fff700 1 -- 172.21.15.32:0/1893383665 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c813a3d0 con 0x7f49c8061d00 2020-12-05T17:37:23.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.941+0000 7f49c4ff9700 1 -- 172.21.15.32:0/1893383665 <== mon.2 v2:172.21.15.61:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f49b001b060 con 0x7f49c8061d00 2020-12-05T17:37:23.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.941+0000 7f49c4ff9700 1 --2- 172.21.15.32:0/1893383665 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49ac055330 0x7f49ac0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.941+0000 7f49c4ff9700 1 -- 172.21.15.32:0/1893383665 <== mon.2 v2:172.21.15.61:3301/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f49b006dc30 con 0x7f49c8061d00 2020-12-05T17:37:23.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.941+0000 7f49c6ffd700 1 --2- 172.21.15.32:0/1893383665 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49ac055330 0x7f49ac0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.941+0000 7f49c6ffd700 1 --2- 172.21.15.32:0/1893383665 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49ac055330 0x7f49ac0577f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f49b800c000 tx=0x7f49b8008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.941+0000 7f49dbd79740 1 Processor -- start 2020-12-05T17:37:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.941+0000 7f49dbd79740 1 -- start start 2020-12-05T17:37:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.941+0000 7f49dbd79740 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1a7a080 0x1ac4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.941+0000 7f49dbd79740 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1962eb0 0x1ac6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.941+0000 7f49dbd79740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x18e2820 0x1aa8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.941+0000 7f49dbd79740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x19ef010 con 0x18e2820 2020-12-05T17:37:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.941+0000 7f49dbd79740 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x18fb6b0 con 0x1a7a080 2020-12-05T17:37:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.941+0000 7f49dbd79740 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x18a9430 con 0x1962eb0 2020-12-05T17:37:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.941+0000 7f49c77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x18e2820 0x1aa8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.941+0000 7f49c77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x18e2820 0x1aa8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36270/0 (socket says 172.21.15.32:36270) 2020-12-05T17:37:23.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.941+0000 7f49c77fe700 1 -- 172.21.15.32:0/4180241058 learned_addr learned my addr 172.21.15.32:0/4180241058 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:23.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.941+0000 7f49c6ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1a7a080 0x1ac4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.941+0000 7f49bffff700 1 --2- 172.21.15.32:0/4180241058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1962eb0 0x1ac6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49c77fe700 1 -- 172.21.15.32:0/4180241058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1962eb0 msgr2=0x1ac6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:23.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49c77fe700 1 --2- 172.21.15.32:0/4180241058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1962eb0 0x1ac6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49c77fe700 1 -- 172.21.15.32:0/4180241058 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1a7a080 msgr2=0x1ac4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:23.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49c77fe700 1 --2- 172.21.15.32:0/4180241058 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1a7a080 0x1ac4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49c77fe700 1 -- 172.21.15.32:0/4180241058 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1a0ff30 con 0x18e2820 2020-12-05T17:37:23.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49c77fe700 1 --2- 172.21.15.32:0/4180241058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x18e2820 0x1aa8c50 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x1ac60f0 tx=0x7f49c80aea50).ready entity=mon.0 client_cookie=2f1b68fca4a6421b server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:23.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49c7fff700 1 -- 172.21.15.32:0/4180241058 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49c0006de0 con 0x18e2820 2020-12-05T17:37:23.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49c7fff700 1 -- 172.21.15.32:0/4180241058 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c000bde0 con 0x18e2820 2020-12-05T17:37:23.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49c7fff700 1 -- 172.21.15.32:0/4180241058 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49c001e430 con 0x18e2820 2020-12-05T17:37:23.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49dbd79740 1 -- 172.21.15.32:0/4180241058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x18e2820 msgr2=0x1aa8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:23.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49dbd79740 1 --2- 172.21.15.32:0/4180241058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x18e2820 0x1aa8c50 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x1ac60f0 tx=0x7f49c80aea50).stop 2020-12-05T17:37:23.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49dbd79740 1 -- 172.21.15.32:0/4180241058 shutdown_connections 2020-12-05T17:37:23.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49dbd79740 1 --2- 172.21.15.32:0/4180241058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x18e2820 0x1aa8c50 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49dbd79740 1 --2- 172.21.15.32:0/4180241058 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1a7a080 0x1ac4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49dbd79740 1 --2- 172.21.15.32:0/4180241058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1962eb0 0x1ac6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49dbd79740 1 -- 172.21.15.32:0/4180241058 >> 172.21.15.32:0/4180241058 conn(0x1aa9c60 msgr2=0x1ac5140 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:23.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49dbd79740 1 -- 172.21.15.32:0/4180241058 shutdown_connections 2020-12-05T17:37:23.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49dbd79740 1 -- 172.21.15.32:0/4180241058 wait complete. 2020-12-05T17:37:23.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49dbd79740 1 -- start start 2020-12-05T17:37:23.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49dbd79740 1 Processor -- start 2020-12-05T17:37:23.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49dbd79740 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x18e2820 0x1ac54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49dbd79740 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1962eb0 0x1b01910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49dbd79740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1a7a080 0x1aff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49dbd79740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a1ec90 con 0x1a7a080 2020-12-05T17:37:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49dbd79740 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a20260 con 0x1962eb0 2020-12-05T17:37:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49dbd79740 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a01730 con 0x18e2820 2020-12-05T17:37:23.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49dbd79740 10 client.0 ll_register_callbacks cb 0x1aa9bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-12-05T17:37:23.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49c77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1a7a080 0x1aff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49c77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1a7a080 0x1aff6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36276/0 (socket says 172.21.15.32:36276) 2020-12-05T17:37:23.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49c77fe700 1 -- 172.21.15.32:0/2568551255 learned_addr learned my addr 172.21.15.32:0/2568551255 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:23.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49c6ffd700 1 --2- 172.21.15.32:0/2568551255 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x18e2820 0x1ac54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49bffff700 1 --2- 172.21.15.32:0/2568551255 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1962eb0 0x1b01910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49c6ffd700 1 -- 172.21.15.32:0/2568551255 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1962eb0 msgr2=0x1b01910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:23.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49c6ffd700 1 --2- 172.21.15.32:0/2568551255 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1962eb0 0x1b01910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49c6ffd700 1 -- 172.21.15.32:0/2568551255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1a7a080 msgr2=0x1aff6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49c6ffd700 1 --2- 172.21.15.32:0/2568551255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1a7a080 0x1aff6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49c6ffd700 1 -- 172.21.15.32:0/2568551255 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1a18460 con 0x18e2820 2020-12-05T17:37:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49c6ffd700 1 --2- 172.21.15.32:0/2568551255 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x18e2820 0x1ac54a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f49b8019000 tx=0x7f49b8015000).ready entity=mon.2 client_cookie=6a332b19e1da78b2 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.61:3301/0 2020-12-05T17:37:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49a5ffb700 1 -- 172.21.15.32:0/2568551255 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49b8013a20 con 0x18e2820 2020-12-05T17:37:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49a5ffb700 1 -- 172.21.15.32:0/2568551255 <== mon.2 v2:172.21.15.61:3301/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f49b8022e10 con 0x18e2820 2020-12-05T17:37:23.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.945+0000 7f49dbd79740 1 -- 172.21.15.32:0/2568551255 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x19078d0 con 0x18e2820 2020-12-05T17:37:23.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49a5ffb700 1 -- 172.21.15.32:0/2568551255 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49b8013b80 con 0x18e2820 2020-12-05T17:37:23.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49dbd79740 10 client.5567 fetch_fsmap learned FSMap version 27 2020-12-05T17:37:23.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49dbd79740 1 -- 172.21.15.32:0/2568551255 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x1892790 con 0x18e2820 2020-12-05T17:37:23.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49a5ffb700 1 -- 172.21.15.32:0/2568551255 <== mon.2 v2:172.21.15.61:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x19078d0 con 0x18e2820 2020-12-05T17:37:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49a5ffb700 1 -- 172.21.15.32:0/2568551255 <== mon.2 v2:172.21.15.61:3301/0 5 ==== fsmap(e 27) v1 ==== 1247+0+0 (secure 0 0 0) 0x7f49b8012040 con 0x18e2820 2020-12-05T17:37:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49dbd79740 10 client.5567 fetch_fsmap finished waiting for FSMap version 27 2020-12-05T17:37:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49dbd79740 10 client.5567 resolve_mds: resolved ID 'a' to GID 5976 2020-12-05T17:37:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49dbd79740 20 client.5567 populate_metadata read hostname 'smithi032' 2020-12-05T17:37:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49dbd79740 1 --2- 172.21.15.32:0/2568551255 >> [v2:172.21.15.32:6834/738464306,v1:172.21.15.32:6835/738464306] conn(0x1874c20 0x1ac9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49dbd79740 4 client.5567 mds_command: new command op to 5976 tid=0[{"prefix": "get_command_descriptions"}] 2020-12-05T17:37:23.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49dbd79740 1 -- 172.21.15.32:0/2568551255 --> [v2:172.21.15.32:6834/738464306,v1:172.21.15.32:6835/738464306] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1874170 con 0x1874c20 2020-12-05T17:37:23.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49bffff700 1 --2- 172.21.15.32:0/2568551255 >> [v2:172.21.15.32:6834/738464306,v1:172.21.15.32:6835/738464306] conn(0x1874c20 0x1ac9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49bffff700 1 --2- 172.21.15.32:0/2568551255 >> [v2:172.21.15.32:6834/738464306,v1:172.21.15.32:6835/738464306] conn(0x1874c20 0x1ac9730 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=e1006e6c42a4234 server_cookie=c0fe03a0454b42b3 in_seq=0 out_seq=0 2020-12-05T17:37:23.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49a5ffb700 10 client.5567 ms_handle_connect on v2:172.21.15.32:6834/738464306 2020-12-05T17:37:23.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49a5ffb700 1 -- 172.21.15.32:0/2568551255 <== mds.0 v2:172.21.15.32:6834/738464306 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16606 (crc 0 0 0) 0x1874170 con 0x1874c20 2020-12-05T17:37:23.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49a5ffb700 10 client.5567 handle_command_reply: tid=0 2020-12-05T17:37:23.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49dbd79740 1 client.5567 shutdown 2020-12-05T17:37:23.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49dbd79740 1 -- 172.21.15.32:0/2568551255 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x18e2820 msgr2=0x1ac54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:23.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49dbd79740 1 --2- 172.21.15.32:0/2568551255 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x18e2820 0x1ac54a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f49b8019000 tx=0x7f49b8015000).stop 2020-12-05T17:37:23.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49dbd79740 1 -- 172.21.15.32:0/2568551255 shutdown_connections 2020-12-05T17:37:23.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49dbd79740 1 --2- 172.21.15.32:0/2568551255 >> [v2:172.21.15.32:6834/738464306,v1:172.21.15.32:6835/738464306] conn(0x1874c20 0x1ac9730 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49dbd79740 1 --2- 172.21.15.32:0/2568551255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1a7a080 0x1aff6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49dbd79740 1 --2- 172.21.15.32:0/2568551255 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x18e2820 0x1ac54a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49dbd79740 1 --2- 172.21.15.32:0/2568551255 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1962eb0 0x1b01910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49a5ffb700 0 client.5567 ms_handle_reset on v2:172.21.15.32:6834/738464306 2020-12-05T17:37:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49dbd79740 1 -- 172.21.15.32:0/2568551255 >> 172.21.15.32:0/2568551255 conn(0x1aa9c60 msgr2=0x1af7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49dbd79740 1 -- 172.21.15.32:0/2568551255 shutdown_connections 2020-12-05T17:37:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49dbd79740 1 -- 172.21.15.32:0/2568551255 wait complete. 2020-12-05T17:37:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.949+0000 7f49dbd79740 20 client.5567 trim_cache size 0 max 16384 2020-12-05T17:37:23.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.965+0000 7f49dbd79740 1 Processor -- start 2020-12-05T17:37:23.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.965+0000 7f49dbd79740 1 -- start start 2020-12-05T17:37:23.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.965+0000 7f49dbd79740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x18e2820 0x1af6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.965+0000 7f49dbd79740 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1962eb0 0x1ac45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.965+0000 7f49dbd79740 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1a7a080 0x1afda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.965+0000 7f49dbd79740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x1abe880 con 0x18e2820 2020-12-05T17:37:23.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.965+0000 7f49dbd79740 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ac78a0 con 0x1a7a080 2020-12-05T17:37:23.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.965+0000 7f49dbd79740 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x19cf6e0 con 0x1962eb0 2020-12-05T17:37:23.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.965+0000 7f49c6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x18e2820 0x1af6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.965+0000 7f49c77fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1a7a080 0x1afda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.965+0000 7f49c6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x18e2820 0x1af6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36280/0 (socket says 172.21.15.32:36280) 2020-12-05T17:37:23.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.965+0000 7f49c6ffd700 1 -- 172.21.15.32:0/4133095258 learned_addr learned my addr 172.21.15.32:0/4133095258 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:23.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.965+0000 7f49bffff700 1 --2- 172.21.15.32:0/4133095258 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1962eb0 0x1ac45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.965+0000 7f49c6ffd700 1 -- 172.21.15.32:0/4133095258 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1962eb0 msgr2=0x1ac45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:23.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.965+0000 7f49c6ffd700 1 --2- 172.21.15.32:0/4133095258 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1962eb0 0x1ac45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.965+0000 7f49c6ffd700 1 -- 172.21.15.32:0/4133095258 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1a7a080 msgr2=0x1afda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:23.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.965+0000 7f49c6ffd700 1 --2- 172.21.15.32:0/4133095258 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1a7a080 0x1afda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.965+0000 7f49c6ffd700 1 -- 172.21.15.32:0/4133095258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x19cf860 con 0x18e2820 2020-12-05T17:37:23.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.965+0000 7f49c77fe700 1 --2- 172.21.15.32:0/4133095258 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1a7a080 0x1afda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:37:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.965+0000 7f49c6ffd700 1 --2- 172.21.15.32:0/4133095258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x18e2820 0x1af6be0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f49b8019000 tx=0x7f49b8021000).ready entity=mon.0 client_cookie=429bd49afed810ba server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49a7fff700 1 -- 172.21.15.32:0/4133095258 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49b800de70 con 0x18e2820 2020-12-05T17:37:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49a7fff700 1 -- 172.21.15.32:0/4133095258 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b8010e60 con 0x18e2820 2020-12-05T17:37:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49a7fff700 1 -- 172.21.15.32:0/4133095258 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49b80205d0 con 0x18e2820 2020-12-05T17:37:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49dbd79740 1 -- 172.21.15.32:0/4133095258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x18e2820 msgr2=0x1af6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:23.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49dbd79740 1 --2- 172.21.15.32:0/4133095258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x18e2820 0x1af6be0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f49b8019000 tx=0x7f49b8021000).stop 2020-12-05T17:37:23.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49dbd79740 1 -- 172.21.15.32:0/4133095258 shutdown_connections 2020-12-05T17:37:23.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49dbd79740 1 --2- 172.21.15.32:0/4133095258 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1a7a080 0x1afda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49dbd79740 1 --2- 172.21.15.32:0/4133095258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x18e2820 0x1af6be0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49dbd79740 1 --2- 172.21.15.32:0/4133095258 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1962eb0 0x1ac45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49dbd79740 1 -- 172.21.15.32:0/4133095258 >> 172.21.15.32:0/4133095258 conn(0x1aa9c60 msgr2=0x1a03730 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49dbd79740 1 -- 172.21.15.32:0/4133095258 shutdown_connections 2020-12-05T17:37:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49dbd79740 1 -- 172.21.15.32:0/4133095258 wait complete. 2020-12-05T17:37:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49dbd79740 1 -- start start 2020-12-05T17:37:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49dbd79740 1 Processor -- start 2020-12-05T17:37:23.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49dbd79740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x18e2820 0x1b05630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49dbd79740 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1962eb0 0x1b05b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49dbd79740 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1a7a080 0x1ac53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49dbd79740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b06010 con 0x18e2820 2020-12-05T17:37:23.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49dbd79740 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b0bbb0 con 0x1962eb0 2020-12-05T17:37:23.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49dbd79740 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b0be50 con 0x1a7a080 2020-12-05T17:37:23.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49c6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x18e2820 0x1b05630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49c6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x18e2820 0x1b05630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36286/0 (socket says 172.21.15.32:36286) 2020-12-05T17:37:23.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49c6ffd700 1 -- 172.21.15.32:0/83776364 learned_addr learned my addr 172.21.15.32:0/83776364 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:23.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.969+0000 7f49dbd79740 10 client.0 ll_register_callbacks cb 0x1ac9420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-12-05T17:37:23.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49bffff700 1 --2- 172.21.15.32:0/83776364 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1962eb0 0x1b05b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49c77fe700 1 --2- 172.21.15.32:0/83776364 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1a7a080 0x1ac53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49c6ffd700 1 -- 172.21.15.32:0/83776364 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1a7a080 msgr2=0x1ac53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:23.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49c6ffd700 1 --2- 172.21.15.32:0/83776364 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1a7a080 0x1ac53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49c6ffd700 1 -- 172.21.15.32:0/83776364 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1962eb0 msgr2=0x1b05b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:23.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49c6ffd700 1 --2- 172.21.15.32:0/83776364 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1962eb0 0x1b05b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:23.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49c6ffd700 1 -- 172.21.15.32:0/83776364 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c8142050 con 0x18e2820 2020-12-05T17:37:23.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49c77fe700 1 --2- 172.21.15.32:0/83776364 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1a7a080 0x1ac53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:37:23.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49c6ffd700 1 --2- 172.21.15.32:0/83776364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x18e2820 0x1b05630 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f49b8019000 tx=0x7f49b8023040).ready entity=mon.0 client_cookie=330f7450aafd8871 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2020-12-05T17:37:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49a6ffd700 1 -- 172.21.15.32:0/83776364 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49b8010e60 con 0x18e2820 2020-12-05T17:37:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49dbd79740 1 -- 172.21.15.32:0/83776364 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1926730 con 0x18e2820 2020-12-05T17:37:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49a6ffd700 1 -- 172.21.15.32:0/83776364 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f49b8034460 con 0x18e2820 2020-12-05T17:37:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49a6ffd700 1 -- 172.21.15.32:0/83776364 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49b8034620 con 0x18e2820 2020-12-05T17:37:23.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49dbd79740 10 client.6180 fetch_fsmap learned FSMap version 27 2020-12-05T17:37:23.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49dbd79740 1 -- 172.21.15.32:0/83776364 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x189de70 con 0x18e2820 2020-12-05T17:37:23.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49a6ffd700 1 -- 172.21.15.32:0/83776364 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b8045020 con 0x18e2820 2020-12-05T17:37:23.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49a6ffd700 1 -- 172.21.15.32:0/83776364 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 27) v1 ==== 1247+0+0 (secure 0 0 0) 0x7f49b8012030 con 0x18e2820 2020-12-05T17:37:23.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49dbd79740 10 client.6180 fetch_fsmap finished waiting for FSMap version 27 2020-12-05T17:37:23.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49dbd79740 10 client.6180 resolve_mds: resolved ID 'a' to GID 5976 2020-12-05T17:37:23.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49dbd79740 20 client.6180 populate_metadata read hostname 'smithi032' 2020-12-05T17:37:23.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49dbd79740 1 --2- 172.21.15.32:0/83776364 >> [v2:172.21.15.32:6834/738464306,v1:172.21.15.32:6835/738464306] conn(0x1b5d4b0 0x1ac58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:23.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49dbd79740 4 client.6180 mds_command: new command op to 5976 tid=0[{"prefix": "cache drop", "timeout": 10}] 2020-12-05T17:37:23.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49dbd79740 1 -- 172.21.15.32:0/83776364 --> [v2:172.21.15.32:6834/738464306,v1:172.21.15.32:6835/738464306] -- command(tid 0: {"prefix": "cache drop", "timeout": 10}) v1 -- 0x18fd440 con 0x1b5d4b0 2020-12-05T17:37:23.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49bffff700 1 --2- 172.21.15.32:0/83776364 >> [v2:172.21.15.32:6834/738464306,v1:172.21.15.32:6835/738464306] conn(0x1b5d4b0 0x1ac58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:23.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49a6ffd700 10 client.6180 ms_handle_connect on v2:172.21.15.32:6834/738464306 2020-12-05T17:37:23.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:23.973+0000 7f49bffff700 1 --2- 172.21.15.32:0/83776364 >> [v2:172.21.15.32:6834/738464306,v1:172.21.15.32:6835/738464306] conn(0x1b5d4b0 0x1ac58d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=52e45cc1517b300f server_cookie=ce8a655905e13b13 in_seq=0 out_seq=0 2020-12-05T17:37:34.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.280+0000 7f49a6ffd700 1 -- 172.21.15.32:0/83776364 <== mds.0 v2:172.21.15.32:6834/738464306 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+406 (crc 0 0 0) 0x18fd440 con 0x1b5d4b0 2020-12-05T17:37:34.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.280+0000 7f49a6ffd700 10 client.6180 handle_command_reply: tid=0 2020-12-05T17:37:34.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.280+0000 7f49dbd79740 1 client.6180 shutdown 2020-12-05T17:37:34.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.280+0000 7f49dbd79740 1 -- 172.21.15.32:0/83776364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x18e2820 msgr2=0x1b05630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:34.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.280+0000 7f49dbd79740 1 --2- 172.21.15.32:0/83776364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x18e2820 0x1b05630 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f49b8019000 tx=0x7f49b8023040).stop 2020-12-05T17:37:34.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.280+0000 7f49dbd79740 1 -- 172.21.15.32:0/83776364 shutdown_connections 2020-12-05T17:37:34.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.280+0000 7f49dbd79740 1 --2- 172.21.15.32:0/83776364 >> [v2:172.21.15.32:6834/738464306,v1:172.21.15.32:6835/738464306] conn(0x1b5d4b0 0x1ac58d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:34.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.280+0000 7f49dbd79740 1 --2- 172.21.15.32:0/83776364 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x1a7a080 0x1ac53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:34.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.280+0000 7f49dbd79740 1 --2- 172.21.15.32:0/83776364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x18e2820 0x1b05630 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:34.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.280+0000 7f49dbd79740 1 --2- 172.21.15.32:0/83776364 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x1962eb0 0x1b05b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:34.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.280+0000 7f49dbd79740 1 -- 172.21.15.32:0/83776364 >> 172.21.15.32:0/83776364 conn(0x1aa9c60 msgr2=0x1ac6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:34.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.280+0000 7f49a6ffd700 0 client.6180 ms_handle_reset on v2:172.21.15.32:6834/738464306 2020-12-05T17:37:34.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.280+0000 7f49dbd79740 1 -- 172.21.15.32:0/83776364 shutdown_connections 2020-12-05T17:37:34.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.280+0000 7f49dbd79740 1 -- 172.21.15.32:0/83776364 wait complete. 2020-12-05T17:37:34.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.280+0000 7f49dbd79740 20 client.6180 trim_cache size 0 max 16384 2020-12-05T17:37:34.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.284+0000 7f49c7fff700 1 -- 172.21.15.32:0/1893383665 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49ac055330 msgr2=0x7f49ac0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:34.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.284+0000 7f49c7fff700 1 --2- 172.21.15.32:0/1893383665 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49ac055330 0x7f49ac0577f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f49b800c000 tx=0x7f49b8008040).stop 2020-12-05T17:37:34.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.284+0000 7f49c7fff700 1 -- 172.21.15.32:0/1893383665 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49c8061d00 msgr2=0x7f49c813e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:34.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.284+0000 7f49c7fff700 1 --2- 172.21.15.32:0/1893383665 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49c8061d00 0x7f49c813e150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f49b00013e0 tx=0x7f49b0001b00).stop 2020-12-05T17:37:34.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.284+0000 7f49c7fff700 1 -- 172.21.15.32:0/1893383665 shutdown_connections 2020-12-05T17:37:34.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.284+0000 7f49c7fff700 1 --2- 172.21.15.32:0/1893383665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c80621b0 0x7f49c81528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:34.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.284+0000 7f49c7fff700 1 --2- 172.21.15.32:0/1893383665 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49c8061920 0x7f49c8065590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:34.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.284+0000 7f49c7fff700 1 --2- 172.21.15.32:0/1893383665 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49ac055330 0x7f49ac0577f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:34.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.284+0000 7f49c7fff700 1 --2- 172.21.15.32:0/1893383665 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49c8061d00 0x7f49c813e150 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:34.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.284+0000 7f49c7fff700 1 -- 172.21.15.32:0/1893383665 >> 172.21.15.32:0/1893383665 conn(0x7f49c80b5620 msgr2=0x7f49c8135a00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:34.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.284+0000 7f49c7fff700 1 -- 172.21.15.32:0/1893383665 shutdown_connections 2020-12-05T17:37:34.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.284+0000 7f49c7fff700 1 -- 172.21.15.32:0/1893383665 wait complete. 2020-12-05T17:37:34.291 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:37:34.291 INFO:teuthology.orchestra.run.smithi032.stdout: "client_recall": { 2020-12-05T17:37:34.291 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": -110, 2020-12-05T17:37:34.292 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "(110) Connection timed out", 2020-12-05T17:37:34.292 INFO:teuthology.orchestra.run.smithi032.stdout: "recalled": 240 2020-12-05T17:37:34.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:34.292 INFO:teuthology.orchestra.run.smithi032.stdout: "flush_journal": { 2020-12-05T17:37:34.292 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2020-12-05T17:37:34.293 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "" 2020-12-05T17:37:34.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:34.293 INFO:teuthology.orchestra.run.smithi032.stdout: "trim_cache": { 2020-12-05T17:37:34.293 INFO:teuthology.orchestra.run.smithi032.stdout: "trimmed": 220 2020-12-05T17:37:34.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:34.294 INFO:teuthology.orchestra.run.smithi032.stdout: "cache": { 2020-12-05T17:37:34.294 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": { 2020-12-05T17:37:34.294 INFO:teuthology.orchestra.run.smithi032.stdout: "items": 15448, 2020-12-05T17:37:34.294 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes": 2428464 2020-12-05T17:37:34.295 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:34.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:34.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 10.308557005000001 2020-12-05T17:37:34.295 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:37:34.296 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:37:34.297 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:37:34.314 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:37:34.331 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:37:34.349 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:37:34.366 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:37:34.386 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:37:34.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:34.387 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:37:34.387 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:37:34.397 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:37:34.398 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:37:34.399 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi036.front.sepia.ceph.com... 2020-12-05T17:37:34.399 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:37:34.399 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:37:34.467 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[10896]: fuse finished with error 0 and tester_r 0 2020-12-05T17:37:34.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:34.470 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:37:34.470 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:37:34.516 INFO:teuthology.orchestra.run.smithi036.stdout:ext2/ext3 2020-12-05T17:37:34.517 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:37:34.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:37:34.633 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:37:34.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:34.682 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:37:34.682 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:37:34.689 INFO:teuthology.orchestra.run.smithi002.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-12-05T17:37:34.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:37:34.690 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-12-05T17:37:34.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:37:34.691 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:37:34.737 INFO:teuthology.orchestra.run.smithi002.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-12-05T17:37:34.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:37:34.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:34.739 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:37:34.740 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:37:34.746 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:37:34.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:37:34.748 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2020-12-05T17:37:34.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:37:34.749 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:37:34.795 INFO:teuthology.orchestra.run.smithi053.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:37:34.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:37:34.797 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2020-12-05T17:37:34.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.924+0000 7fd02953e700 1 Processor -- start 2020-12-05T17:37:34.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.924+0000 7fd02953e700 1 -- start start 2020-12-05T17:37:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.924+0000 7fd02953e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd0240abeb0 0x7fd0240ab610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.924+0000 7fd02953e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0240ae6e0 0x7fd02413dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.924+0000 7fd02953e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0240ac290 con 0x7fd024062660 2020-12-05T17:37:34.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.924+0000 7fd02953e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0240ad8a0 con 0x7fd0240abeb0 2020-12-05T17:37:34.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.924+0000 7fd02953e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0240b0930 con 0x7fd0240ae6e0 2020-12-05T17:37:34.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.924+0000 7fd01bfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd024062660 0x7fd0240ad190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38944/0 (socket says 172.21.15.32:38944) 2020-12-05T17:37:34.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.924+0000 7fd023fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd0240abeb0 0x7fd0240ab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.924+0000 7fd01bfff700 1 -- 172.21.15.32:0/719426960 learned_addr learned my addr 172.21.15.32:0/719426960 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.928+0000 7fd028d3d700 1 --2- 172.21.15.32:0/719426960 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0240ae6e0 0x7fd02413dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.928+0000 7fd0237fe700 1 -- 172.21.15.32:0/719426960 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2950135397 0 0) 0x7fd0240ac290 con 0x7fd024062660 2020-12-05T17:37:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.928+0000 7fd0237fe700 1 -- 172.21.15.32:0/719426960 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd004003230 con 0x7fd024062660 2020-12-05T17:37:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.928+0000 7fd023fff700 1 -- 172.21.15.32:0/719426960 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0240ae6e0 msgr2=0x7fd02413dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:34.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.928+0000 7fd023fff700 1 --2- 172.21.15.32:0/719426960 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0240ae6e0 0x7fd02413dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:34.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.928+0000 7fd023fff700 1 -- 172.21.15.32:0/719426960 >> v1:172.21.15.32:6789/0 conn(0x7fd024062660 legacy=0x7fd0240ad190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:34.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.928+0000 7fd023fff700 1 -- 172.21.15.32:0/719426960 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0240b0bd0 con 0x7fd0240abeb0 2020-12-05T17:37:34.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.928+0000 7fd023fff700 1 --2- 172.21.15.32:0/719426960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0240abeb0 0x7fd0240ab610 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7fd014002f30 tx=0x7fd014006dc0).ready entity=mon.0 client_cookie=b38ac133e6fc3a7 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:34.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.928+0000 7fd0237fe700 1 -- 172.21.15.32:0/719426960 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0140096a0 con 0x7fd0240abeb0 2020-12-05T17:37:34.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.928+0000 7fd0237fe700 1 -- 172.21.15.32:0/719426960 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd014009800 con 0x7fd0240abeb0 2020-12-05T17:37:34.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.928+0000 7fd02953e700 1 -- 172.21.15.32:0/719426960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0240abeb0 msgr2=0x7fd0240ab610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:34.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.928+0000 7fd0237fe700 1 -- 172.21.15.32:0/719426960 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd014009b60 con 0x7fd0240abeb0 2020-12-05T17:37:34.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.928+0000 7fd02953e700 1 --2- 172.21.15.32:0/719426960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0240abeb0 0x7fd0240ab610 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7fd014002f30 tx=0x7fd014006dc0).stop 2020-12-05T17:37:34.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.928+0000 7fd02953e700 1 -- 172.21.15.32:0/719426960 shutdown_connections 2020-12-05T17:37:34.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.928+0000 7fd02953e700 1 --2- 172.21.15.32:0/719426960 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0240ae6e0 0x7fd02413dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:34.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.928+0000 7fd02953e700 1 --2- 172.21.15.32:0/719426960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0240abeb0 0x7fd0240ab610 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:34.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.928+0000 7fd02953e700 1 -- 172.21.15.32:0/719426960 >> 172.21.15.32:0/719426960 conn(0x7fd0240b5530 msgr2=0x7fd0240b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:34.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.928+0000 7fd02953e700 1 -- 172.21.15.32:0/719426960 shutdown_connections 2020-12-05T17:37:34.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.928+0000 7fd02953e700 1 -- 172.21.15.32:0/719426960 wait complete. 2020-12-05T17:37:34.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.928+0000 7fd02953e700 1 Processor -- start 2020-12-05T17:37:34.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.928+0000 7fd02953e700 1 -- start start 2020-12-05T17:37:34.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.932+0000 7fd02953e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd024062660 0x7fd024142ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:34.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.932+0000 7fd02953e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0240abeb0 0x7fd024159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:34.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.932+0000 7fd02953e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0240ae6e0 0x7fd024159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:34.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.932+0000 7fd02953e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02413e570 con 0x7fd0240abeb0 2020-12-05T17:37:34.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.932+0000 7fd02953e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd024153320 con 0x7fd0240ae6e0 2020-12-05T17:37:34.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.932+0000 7fd02953e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02413f290 con 0x7fd024062660 2020-12-05T17:37:34.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.932+0000 7fd01bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0240abeb0 0x7fd024159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:34.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.932+0000 7fd01bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0240abeb0 0x7fd024159250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36304/0 (socket says 172.21.15.32:36304) 2020-12-05T17:37:34.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.932+0000 7fd01bfff700 1 -- 172.21.15.32:0/16163269 learned_addr learned my addr 172.21.15.32:0/16163269 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:34.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.932+0000 7fd023fff700 1 --2- 172.21.15.32:0/16163269 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd024062660 0x7fd024142ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:34.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.932+0000 7fd01bfff700 1 -- 172.21.15.32:0/16163269 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd024062660 msgr2=0x7fd024142ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:34.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.932+0000 7fd01bfff700 1 --2- 172.21.15.32:0/16163269 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd024062660 0x7fd024142ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:34.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.932+0000 7fd01bfff700 1 -- 172.21.15.32:0/16163269 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0240ae6e0 msgr2=0x7fd024159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:34.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.932+0000 7fd01bfff700 1 --2- 172.21.15.32:0/16163269 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0240ae6e0 0x7fd024159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-12-05T17:37:34.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.932+0000 7fd01bfff700 1 -- 172.21.15.32:0/16163269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd024158e20 con 0x7fd0240abeb0 2020-12-05T17:37:34.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.932+0000 7fd01bfff700 1 --2- 172.21.15.32:0/16163269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0240abeb0 0x7fd024159250 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fd024153460 tx=0x7fd00c007a70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:34.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.932+0000 7fd021ffb700 1 -- 172.21.15.32:0/16163269 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd00c0072e0 con 0x7fd0240abeb0 2020-12-05T17:37:34.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.932+0000 7fd021ffb700 1 -- 172.21.15.32:0/16163269 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00c007440 con 0x7fd0240abeb0 2020-12-05T17:37:34.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.932+0000 7fd02953e700 1 -- 172.21.15.32:0/16163269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd02413f4c0 con 0x7fd0240abeb0 2020-12-05T17:37:34.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.932+0000 7fd021ffb700 1 -- 172.21.15.32:0/16163269 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd00c0057d0 con 0x7fd0240abeb0 2020-12-05T17:37:34.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.932+0000 7fd02953e700 1 -- 172.21.15.32:0/16163269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd024159c30 con 0x7fd0240abeb0 2020-12-05T17:37:34.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.932+0000 7fd02953e700 1 -- 172.21.15.32:0/16163269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd024057050 con 0x7fd0240abeb0 2020-12-05T17:37:34.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.932+0000 7fd021ffb700 1 -- 172.21.15.32:0/16163269 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd00c010070 con 0x7fd0240abeb0 2020-12-05T17:37:34.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.936+0000 7fd021ffb700 1 --2- 172.21.15.32:0/16163269 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd008055440 0x7fd008057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:34.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.936+0000 7fd021ffb700 1 -- 172.21.15.32:0/16163269 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd00c06d860 con 0x7fd0240abeb0 2020-12-05T17:37:34.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.936+0000 7fd023fff700 1 --2- 172.21.15.32:0/16163269 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd008055440 0x7fd008057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:34.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.936+0000 7fd023fff700 1 --2- 172.21.15.32:0/16163269 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd008055440 0x7fd008057900 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fd014019000 tx=0x7fd014015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:34.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:34.940+0000 7fd021ffb700 1 -- 172.21.15.32:0/16163269 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd00c043a00 con 0x7fd0240abeb0 2020-12-05T17:37:35.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.084+0000 7fd02953e700 1 -- 172.21.15.32:0/16163269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd024069ee0 con 0x7fd0240abeb0 2020-12-05T17:37:35.093 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.092+0000 7fd021ffb700 1 -- 172.21.15.32:0/16163269 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd00c047020 con 0x7fd0240abeb0 2020-12-05T17:37:35.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.092+0000 7fd02953e700 1 -- 172.21.15.32:0/16163269 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd008055440 msgr2=0x7fd008057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:35.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.092+0000 7fd02953e700 1 --2- 172.21.15.32:0/16163269 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd008055440 0x7fd008057900 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fd014019000 tx=0x7fd014015000).stop 2020-12-05T17:37:35.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.092+0000 7fd02953e700 1 -- 172.21.15.32:0/16163269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0240abeb0 msgr2=0x7fd024159250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:35.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.092+0000 7fd02953e700 1 --2- 172.21.15.32:0/16163269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0240abeb0 0x7fd024159250 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fd024153460 tx=0x7fd00c007a70).stop 2020-12-05T17:37:35.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.092+0000 7fd02953e700 1 -- 172.21.15.32:0/16163269 shutdown_connections 2020-12-05T17:37:35.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.092+0000 7fd02953e700 1 --2- 172.21.15.32:0/16163269 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0240ae6e0 0x7fd024159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-12-05T17:37:35.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.092+0000 7fd02953e700 1 --2- 172.21.15.32:0/16163269 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd008055440 0x7fd008057900 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:35.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.092+0000 7fd02953e700 1 --2- 172.21.15.32:0/16163269 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd024062660 0x7fd024142ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:35.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.092+0000 7fd02953e700 1 --2- 172.21.15.32:0/16163269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0240abeb0 0x7fd024159250 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:35.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.092+0000 7fd02953e700 1 -- 172.21.15.32:0/16163269 >> 172.21.15.32:0/16163269 conn(0x7fd0240b5530 msgr2=0x7fd0240b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:35.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.092+0000 7fd02953e700 1 -- 172.21.15.32:0/16163269 shutdown_connections 2020-12-05T17:37:35.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.092+0000 7fd02953e700 1 -- 172.21.15.32:0/16163269 wait complete. 2020-12-05T17:37:35.107 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout' 2020-12-05T17:37:35.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f6391700 1 Processor -- start 2020-12-05T17:37:35.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f6391700 1 -- start start 2020-12-05T17:37:35.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f6391700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1f005f3b0 0x7fd1f005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:35.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f6391700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd1f005feb0 0x7fd1f00b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:35.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f6391700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f0142450 con 0x7fd1f0062400 2020-12-05T17:37:35.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f6391700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1f00b42f0 con 0x7fd1f005f3b0 2020-12-05T17:37:35.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f6391700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1f0061780 con 0x7fd1f005feb0 2020-12-05T17:37:35.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f538f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1f005f3b0 0x7fd1f005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:35.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f4b8e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd1f0062400 0x7fd1f00b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38962/0 (socket says 172.21.15.32:38962) 2020-12-05T17:37:35.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f538f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1f005f3b0 0x7fd1f005f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36312/0 (socket says 172.21.15.32:36312) 2020-12-05T17:37:35.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f4b8e700 1 -- 172.21.15.32:0/2697186673 learned_addr learned my addr 172.21.15.32:0/2697186673 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:35.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f5b90700 1 --2- 172.21.15.32:0/2697186673 >> v2:172.21.15.61:3300/0 conn(0x7fd1f005feb0 0x7fd1f00b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:35.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f538f700 1 -- 172.21.15.32:0/2697186673 >> v2:172.21.15.61:3300/0 conn(0x7fd1f005feb0 msgr2=0x7fd1f00b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:35.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f538f700 1 --2- 172.21.15.32:0/2697186673 >> v2:172.21.15.61:3300/0 conn(0x7fd1f005feb0 0x7fd1f00b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:35.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1e7fff700 1 -- 172.21.15.32:0/2697186673 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2190488061 0 0) 0x7fd1f0142450 con 0x7fd1f0062400 2020-12-05T17:37:35.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f538f700 1 -- 172.21.15.32:0/2697186673 >> v1:172.21.15.32:6789/0 conn(0x7fd1f0062400 legacy=0x7fd1f00b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:35.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f538f700 1 -- 172.21.15.32:0/2697186673 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f0061a20 con 0x7fd1f005f3b0 2020-12-05T17:37:35.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f538f700 1 --2- 172.21.15.32:0/2697186673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f005f3b0 0x7fd1f005f790 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7fd1e0002f30 tx=0x7fd1e0006dc0).ready entity=mon.0 client_cookie=988cbe7d9e2bf272 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:35.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1e7fff700 1 -- 172.21.15.32:0/2697186673 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1e00097b0 con 0x7fd1f005f3b0 2020-12-05T17:37:35.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1e7fff700 1 -- 172.21.15.32:0/2697186673 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e0009910 con 0x7fd1f005f3b0 2020-12-05T17:37:35.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1e7fff700 1 -- 172.21.15.32:0/2697186673 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1e0009c70 con 0x7fd1f005f3b0 2020-12-05T17:37:35.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f6391700 1 -- 172.21.15.32:0/2697186673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f005f3b0 msgr2=0x7fd1f005f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:35.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f6391700 1 --2- 172.21.15.32:0/2697186673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f005f3b0 0x7fd1f005f790 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7fd1e0002f30 tx=0x7fd1e0006dc0).stop 2020-12-05T17:37:35.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f6391700 1 -- 172.21.15.32:0/2697186673 shutdown_connections 2020-12-05T17:37:35.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f6391700 1 --2- 172.21.15.32:0/2697186673 >> v2:172.21.15.61:3300/0 conn(0x7fd1f005feb0 0x7fd1f00b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:35.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f6391700 1 --2- 172.21.15.32:0/2697186673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f005f3b0 0x7fd1f005f790 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:35.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f6391700 1 -- 172.21.15.32:0/2697186673 >> 172.21.15.32:0/2697186673 conn(0x7fd1f00b8ee0 msgr2=0x7fd1f00b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:35.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.212+0000 7fd1f6391700 1 -- 172.21.15.32:0/2697186673 shutdown_connections 2020-12-05T17:37:35.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.216+0000 7fd1f6391700 1 -- 172.21.15.32:0/2697186673 wait complete. 2020-12-05T17:37:35.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.216+0000 7fd1f6391700 1 Processor -- start 2020-12-05T17:37:35.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.216+0000 7fd1f6391700 1 -- start start 2020-12-05T17:37:35.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.216+0000 7fd1f6391700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f005f3b0 0x7fd1f0063340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:35.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.216+0000 7fd1f6391700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1f005feb0 0x7fd1f0064ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:35.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.216+0000 7fd1f6391700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1f0062400 0x7fd1f0065190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:35.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.216+0000 7fd1f6391700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f00afcb0 con 0x7fd1f005f3b0 2020-12-05T17:37:35.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.216+0000 7fd1f6391700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f0148420 con 0x7fd1f005feb0 2020-12-05T17:37:35.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.216+0000 7fd1f6391700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f01486c0 con 0x7fd1f0062400 2020-12-05T17:37:35.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.216+0000 7fd1f538f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f005f3b0 0x7fd1f0063340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:35.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.216+0000 7fd1f538f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f005f3b0 0x7fd1f0063340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36316/0 (socket says 172.21.15.32:36316) 2020-12-05T17:37:35.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.216+0000 7fd1f538f700 1 -- 172.21.15.32:0/3780980373 learned_addr learned my addr 172.21.15.32:0/3780980373 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:35.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.220+0000 7fd1f4b8e700 1 --2- 172.21.15.32:0/3780980373 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1f005feb0 0x7fd1f0064ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:35.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.220+0000 7fd1f5b90700 1 --2- 172.21.15.32:0/3780980373 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1f0062400 0x7fd1f0065190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:35.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.220+0000 7fd1f538f700 1 -- 172.21.15.32:0/3780980373 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1f0062400 msgr2=0x7fd1f0065190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:35.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.220+0000 7fd1f538f700 1 --2- 172.21.15.32:0/3780980373 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1f0062400 0x7fd1f0065190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:35.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.220+0000 7fd1f538f700 1 -- 172.21.15.32:0/3780980373 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1f005feb0 msgr2=0x7fd1f0064ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:35.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.220+0000 7fd1f538f700 1 --2- 172.21.15.32:0/3780980373 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1f005feb0 0x7fd1f0064ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:35.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.220+0000 7fd1f538f700 1 -- 172.21.15.32:0/3780980373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f0065680 con 0x7fd1f005f3b0 2020-12-05T17:37:35.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.220+0000 7fd1f538f700 1 --2- 172.21.15.32:0/3780980373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f005f3b0 0x7fd1f0063340 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7fd1e0002f30 tx=0x7fd1e0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:35.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.220+0000 7fd1e67fc700 1 -- 172.21.15.32:0/3780980373 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1e0011950 con 0x7fd1f005f3b0 2020-12-05T17:37:35.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.220+0000 7fd1e67fc700 1 -- 172.21.15.32:0/3780980373 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e0011ab0 con 0x7fd1f005f3b0 2020-12-05T17:37:35.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.220+0000 7fd1f6391700 1 -- 172.21.15.32:0/3780980373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f0146460 con 0x7fd1f005f3b0 2020-12-05T17:37:35.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.220+0000 7fd1e67fc700 1 -- 172.21.15.32:0/3780980373 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1e0011e10 con 0x7fd1f005f3b0 2020-12-05T17:37:35.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.220+0000 7fd1f6391700 1 -- 172.21.15.32:0/3780980373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f00af9b0 con 0x7fd1f005f3b0 2020-12-05T17:37:35.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.220+0000 7fd1f6391700 1 -- 172.21.15.32:0/3780980373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f01488c0 con 0x7fd1f005f3b0 2020-12-05T17:37:35.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.220+0000 7fd1e67fc700 1 -- 172.21.15.32:0/3780980373 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd1e0024030 con 0x7fd1f005f3b0 2020-12-05T17:37:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.220+0000 7fd1e67fc700 1 --2- 172.21.15.32:0/3780980373 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1d80554e0 0x7fd1d80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.220+0000 7fd1f4b8e700 1 --2- 172.21.15.32:0/3780980373 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1d80554e0 0x7fd1d80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.220+0000 7fd1e67fc700 1 -- 172.21.15.32:0/3780980373 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd1e006ee60 con 0x7fd1f005f3b0 2020-12-05T17:37:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.224+0000 7fd1f4b8e700 1 --2- 172.21.15.32:0/3780980373 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1d80554e0 0x7fd1d80579a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fd1ec00d000 tx=0x7fd1ec009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:35.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.228+0000 7fd1e67fc700 1 -- 172.21.15.32:0/3780980373 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd1f01488c0 con 0x7fd1f005f3b0 2020-12-05T17:37:35.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.408+0000 7fd1f6391700 1 -- 172.21.15.32:0/3780980373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]} v 0) v1 -- 0x7fd1f01488c0 con 0x7fd1f005f3b0 2020-12-05T17:37:35.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.976+0000 7fd1e67fc700 1 -- 172.21.15.32:0/3780980373 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]}]=0 v256) v1 ==== 147+0+0 (secure 0 0 0) 0x7fd1f01488c0 con 0x7fd1f005f3b0 2020-12-05T17:37:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.980+0000 7fd1f6391700 1 -- 172.21.15.32:0/3780980373 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1d80554e0 msgr2=0x7fd1d80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.980+0000 7fd1f6391700 1 --2- 172.21.15.32:0/3780980373 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1d80554e0 0x7fd1d80579a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fd1ec00d000 tx=0x7fd1ec009040).stop 2020-12-05T17:37:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.980+0000 7fd1f6391700 1 -- 172.21.15.32:0/3780980373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f005f3b0 msgr2=0x7fd1f0063340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.980+0000 7fd1f6391700 1 --2- 172.21.15.32:0/3780980373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f005f3b0 0x7fd1f0063340 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7fd1e0002f30 tx=0x7fd1e0015000).stop 2020-12-05T17:37:35.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.980+0000 7fd1f6391700 1 -- 172.21.15.32:0/3780980373 shutdown_connections 2020-12-05T17:37:35.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.980+0000 7fd1f6391700 1 --2- 172.21.15.32:0/3780980373 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1d80554e0 0x7fd1d80579a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:35.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.980+0000 7fd1f6391700 1 --2- 172.21.15.32:0/3780980373 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1f0062400 0x7fd1f0065190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:35.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.980+0000 7fd1f6391700 1 --2- 172.21.15.32:0/3780980373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f005f3b0 0x7fd1f0063340 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:35.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.980+0000 7fd1f6391700 1 --2- 172.21.15.32:0/3780980373 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1f005feb0 0x7fd1f0064ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:35.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.980+0000 7fd1f6391700 1 -- 172.21.15.32:0/3780980373 >> 172.21.15.32:0/3780980373 conn(0x7fd1f00b8ee0 msgr2=0x7fd1f0140b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:35.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.980+0000 7fd1f6391700 1 -- 172.21.15.32:0/3780980373 shutdown_connections 2020-12-05T17:37:35.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:35.980+0000 7fd1f6391700 1 -- 172.21.15.32:0/3780980373 wait complete. 2020-12-05T17:37:35.999 INFO:tasks.cephfs_test_runner:test_drop_cache_command_timeout (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2020-12-05T17:37:36.000 INFO:tasks.cephfs_test_runner:Starting test: test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc) 2020-12-05T17:37:36.000 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder' 2020-12-05T17:37:36.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.116+0000 7fa3d49dc700 1 Processor -- start 2020-12-05T17:37:36.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.116+0000 7fa3d49dc700 1 -- start start 2020-12-05T17:37:36.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.116+0000 7fa3d49dc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3d00af0d0 0x7fa3d00af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:36.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.116+0000 7fa3d49dc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3d0144320 0x7fa3d00b0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:36.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.116+0000 7fa3d49dc700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3d00af9a0 con 0x7fa3d0066cf0 2020-12-05T17:37:36.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.116+0000 7fa3d49dc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d0067190 con 0x7fa3d0144320 2020-12-05T17:37:36.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.116+0000 7fa3d49dc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d005fe60 con 0x7fa3d00af0d0 2020-12-05T17:37:36.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.116+0000 7fa3ced9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3d0144320 0x7fa3d00b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:36.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.116+0000 7fa3ced9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3d0144320 0x7fa3d00b0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36328/0 (socket says 172.21.15.32:36328) 2020-12-05T17:37:36.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.116+0000 7fa3ce59c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3d00af0d0 0x7fa3d00af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:36.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.116+0000 7fa3ced9d700 1 -- 172.21.15.32:0/2573553124 learned_addr learned my addr 172.21.15.32:0/2573553124 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:36.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.116+0000 7fa3ced9d700 1 -- 172.21.15.32:0/2573553124 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3d00af0d0 msgr2=0x7fa3d00af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:36.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.116+0000 7fa3ced9d700 1 --2- 172.21.15.32:0/2573553124 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3d00af0d0 0x7fa3d00af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:36.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.116+0000 7fa3cd59a700 1 -- 172.21.15.32:0/2573553124 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 702458037 0 0) 0x7fa3d00af9a0 con 0x7fa3d0066cf0 2020-12-05T17:37:36.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.116+0000 7fa3ced9d700 1 -- 172.21.15.32:0/2573553124 >> v1:172.21.15.61:6789/0 conn(0x7fa3d0066cf0 legacy=0x7fa3d0143c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:36.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.120+0000 7fa3ced9d700 1 -- 172.21.15.32:0/2573553124 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d0060100 con 0x7fa3d0144320 2020-12-05T17:37:36.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.120+0000 7fa3ced9d700 1 --2- 172.21.15.32:0/2573553124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d0144320 0x7fa3d00b0010 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7fa3c80032b0 tx=0x7fa3c8009270).ready entity=mon.0 client_cookie=244f65c7a0092ab6 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:36.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.120+0000 7fa3cd59a700 1 -- 172.21.15.32:0/2573553124 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3c8009e60 con 0x7fa3d0144320 2020-12-05T17:37:36.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.120+0000 7fa3cd59a700 1 -- 172.21.15.32:0/2573553124 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c800bc00 con 0x7fa3d0144320 2020-12-05T17:37:36.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.120+0000 7fa3cd59a700 1 -- 172.21.15.32:0/2573553124 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3c800bd70 con 0x7fa3d0144320 2020-12-05T17:37:36.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.120+0000 7fa3d49dc700 1 -- 172.21.15.32:0/2573553124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d0144320 msgr2=0x7fa3d00b0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:36.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.120+0000 7fa3d49dc700 1 --2- 172.21.15.32:0/2573553124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d0144320 0x7fa3d00b0010 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7fa3c80032b0 tx=0x7fa3c8009270).stop 2020-12-05T17:37:36.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.120+0000 7fa3d49dc700 1 -- 172.21.15.32:0/2573553124 shutdown_connections 2020-12-05T17:37:36.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.120+0000 7fa3d49dc700 1 --2- 172.21.15.32:0/2573553124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d0144320 0x7fa3d00b0010 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:36.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.120+0000 7fa3d49dc700 1 --2- 172.21.15.32:0/2573553124 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3d00af0d0 0x7fa3d00af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:36.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.120+0000 7fa3d49dc700 1 -- 172.21.15.32:0/2573553124 >> 172.21.15.32:0/2573553124 conn(0x7fa3d00b5580 msgr2=0x7fa3d00b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:36.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.120+0000 7fa3d49dc700 1 -- 172.21.15.32:0/2573553124 shutdown_connections 2020-12-05T17:37:36.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.120+0000 7fa3d49dc700 1 -- 172.21.15.32:0/2573553124 wait complete. 2020-12-05T17:37:36.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.120+0000 7fa3d49dc700 1 Processor -- start 2020-12-05T17:37:36.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.120+0000 7fa3d49dc700 1 -- start start 2020-12-05T17:37:36.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.124+0000 7fa3d49dc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa3d0066cf0 0x7fa3d0151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:36.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.124+0000 7fa3d49dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d00af0d0 0x7fa3d013bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:36.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.124+0000 7fa3d49dc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3d0144320 0x7fa3d0151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:36.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.124+0000 7fa3d49dc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d013cdc0 con 0x7fa3d00af0d0 2020-12-05T17:37:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.124+0000 7fa3d49dc700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d0147990 con 0x7fa3d0066cf0 2020-12-05T17:37:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.124+0000 7fa3d49dc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d013dc90 con 0x7fa3d0144320 2020-12-05T17:37:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.124+0000 7fa3cdd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d00af0d0 0x7fa3d013bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.124+0000 7fa3cdd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d00af0d0 0x7fa3d013bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36330/0 (socket says 172.21.15.32:36330) 2020-12-05T17:37:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.124+0000 7fa3cdd9b700 1 -- 172.21.15.32:0/2484245700 learned_addr learned my addr 172.21.15.32:0/2484245700 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:36.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.124+0000 7fa3ce59c700 1 --2- 172.21.15.32:0/2484245700 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa3d0066cf0 0x7fa3d0151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:36.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.124+0000 7fa3ced9d700 1 --2- 172.21.15.32:0/2484245700 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3d0144320 0x7fa3d0151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:36.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.124+0000 7fa3ce59c700 1 -- 172.21.15.32:0/2484245700 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3d0144320 msgr2=0x7fa3d0151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:36.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.124+0000 7fa3ce59c700 1 --2- 172.21.15.32:0/2484245700 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3d0144320 0x7fa3d0151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:36.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.124+0000 7fa3ce59c700 1 -- 172.21.15.32:0/2484245700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d00af0d0 msgr2=0x7fa3d013bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:36.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.124+0000 7fa3ce59c700 1 --2- 172.21.15.32:0/2484245700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d00af0d0 0x7fa3d013bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:36.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.124+0000 7fa3ce59c700 1 -- 172.21.15.32:0/2484245700 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d00ad5c0 con 0x7fa3d0066cf0 2020-12-05T17:37:36.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.124+0000 7fa3cdd9b700 1 --2- 172.21.15.32:0/2484245700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d00af0d0 0x7fa3d013bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:37:36.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.124+0000 7fa3ce59c700 1 --2- 172.21.15.32:0/2484245700 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa3d0066cf0 0x7fa3d0151610 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7fa3c000c000 tx=0x7fa3c0006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:36.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.124+0000 7fa3bf7fe700 1 -- 172.21.15.32:0/2484245700 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3c000ca10 con 0x7fa3d0066cf0 2020-12-05T17:37:36.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.124+0000 7fa3bf7fe700 1 -- 172.21.15.32:0/2484245700 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c000cb70 con 0x7fa3d0066cf0 2020-12-05T17:37:36.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.124+0000 7fa3d49dc700 1 -- 172.21.15.32:0/2484245700 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d00ad1c0 con 0x7fa3d0066cf0 2020-12-05T17:37:36.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.124+0000 7fa3bf7fe700 1 -- 172.21.15.32:0/2484245700 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3c000be40 con 0x7fa3d0066cf0 2020-12-05T17:37:36.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.124+0000 7fa3d49dc700 1 -- 172.21.15.32:0/2484245700 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d013eca0 con 0x7fa3d0066cf0 2020-12-05T17:37:36.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.128+0000 7fa3d49dc700 1 -- 172.21.15.32:0/2484245700 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d013dac0 con 0x7fa3d0066cf0 2020-12-05T17:37:36.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.128+0000 7fa3bf7fe700 1 -- 172.21.15.32:0/2484245700 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa3c001c460 con 0x7fa3d0066cf0 2020-12-05T17:37:36.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.128+0000 7fa3bf7fe700 1 --2- 172.21.15.32:0/2484245700 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa3ac055440 0x7fa3ac057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:36.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.128+0000 7fa3cdd9b700 1 --2- 172.21.15.32:0/2484245700 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa3ac055440 0x7fa3ac057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:36.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.128+0000 7fa3bf7fe700 1 -- 172.21.15.32:0/2484245700 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa3c006e470 con 0x7fa3d0066cf0 2020-12-05T17:37:36.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.128+0000 7fa3cdd9b700 1 --2- 172.21.15.32:0/2484245700 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa3ac055440 0x7fa3ac057900 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fa3c400d000 tx=0x7fa3c4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:36.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.136+0000 7fa3bf7fe700 1 -- 172.21.15.32:0/2484245700 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa3c0044510 con 0x7fa3d0066cf0 2020-12-05T17:37:36.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.336+0000 7fa3d49dc700 1 -- 172.21.15.32:0/2484245700 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]} v 0) v1 -- 0x7fa3d0057050 con 0x7fa3d0066cf0 2020-12-05T17:37:36.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.984+0000 7fa3bf7fe700 1 -- 172.21.15.32:0/2484245700 <== mon.1 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]}]=0 v257) v1 ==== 142+0+0 (secure 0 0 0) 0x7fa3c0047b30 con 0x7fa3d0066cf0 2020-12-05T17:37:36.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.988+0000 7fa3d49dc700 1 -- 172.21.15.32:0/2484245700 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa3ac055440 msgr2=0x7fa3ac057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:36.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.988+0000 7fa3d49dc700 1 --2- 172.21.15.32:0/2484245700 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa3ac055440 0x7fa3ac057900 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fa3c400d000 tx=0x7fa3c4009040).stop 2020-12-05T17:37:36.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.988+0000 7fa3d49dc700 1 -- 172.21.15.32:0/2484245700 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa3d0066cf0 msgr2=0x7fa3d0151610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:36.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.988+0000 7fa3d49dc700 1 --2- 172.21.15.32:0/2484245700 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa3d0066cf0 0x7fa3d0151610 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7fa3c000c000 tx=0x7fa3c0006dc0).stop 2020-12-05T17:37:36.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.988+0000 7fa3d49dc700 1 -- 172.21.15.32:0/2484245700 shutdown_connections 2020-12-05T17:37:36.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.988+0000 7fa3d49dc700 1 --2- 172.21.15.32:0/2484245700 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa3ac055440 0x7fa3ac057900 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:36.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.988+0000 7fa3d49dc700 1 --2- 172.21.15.32:0/2484245700 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3d0144320 0x7fa3d0151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:36.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.988+0000 7fa3d49dc700 1 --2- 172.21.15.32:0/2484245700 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa3d0066cf0 0x7fa3d0151610 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:36.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.988+0000 7fa3d49dc700 1 --2- 172.21.15.32:0/2484245700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d00af0d0 0x7fa3d013bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:36.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.988+0000 7fa3d49dc700 1 -- 172.21.15.32:0/2484245700 >> 172.21.15.32:0/2484245700 conn(0x7fa3d00b5580 msgr2=0x7fa3d01405d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:36.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.988+0000 7fa3d49dc700 1 -- 172.21.15.32:0/2484245700 shutdown_connections 2020-12-05T17:37:36.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:36.988+0000 7fa3d49dc700 1 -- 172.21.15.32:0/2484245700 wait complete. 2020-12-05T17:37:37.007 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-12-05T17:37:37.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.132+0000 7f1641d78700 1 Processor -- start 2020-12-05T17:37:37.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.132+0000 7f1641d78700 1 -- start start 2020-12-05T17:37:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.136+0000 7f1641d78700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f163c0af9a0 0x7f163c144610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:37.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.136+0000 7f1641d78700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f163c13f5b0 0x7f163c13fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:37.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.136+0000 7f1641d78700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f163c144b00 con 0x7f163c0af5c0 2020-12-05T17:37:37.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.136+0000 7f1641d78700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f163c1405f0 con 0x7f163c0af9a0 2020-12-05T17:37:37.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.136+0000 7f1641d78700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163c1407e0 con 0x7f163c13f5b0 2020-12-05T17:37:37.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.136+0000 7f1633fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f163c0af9a0 0x7f163c144610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:37.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.136+0000 7f1640d76700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f163c0af5c0 0x7f163c143f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:51626/0 (socket says 172.21.15.32:51626) 2020-12-05T17:37:37.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.136+0000 7f1640d76700 1 -- 172.21.15.32:0/646818446 learned_addr learned my addr 172.21.15.32:0/646818446 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:37.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.136+0000 7f1641577700 1 --2- 172.21.15.32:0/646818446 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f163c13f5b0 0x7f163c13fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.136+0000 7f16337fe700 1 -- 172.21.15.32:0/646818446 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 768170334 0 0) 0x7f163c144b00 con 0x7f163c0af5c0 2020-12-05T17:37:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.136+0000 7f16337fe700 1 -- 172.21.15.32:0/646818446 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1628003230 con 0x7f163c0af5c0 2020-12-05T17:37:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.136+0000 7f1641577700 1 -- 172.21.15.32:0/646818446 >> v2:172.21.15.61:3300/0 conn(0x7f163c0af9a0 msgr2=0x7f163c144610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:37.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.136+0000 7f1641577700 1 --2- 172.21.15.32:0/646818446 >> v2:172.21.15.61:3300/0 conn(0x7f163c0af9a0 0x7f163c144610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:37.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.136+0000 7f1641577700 1 -- 172.21.15.32:0/646818446 >> v1:172.21.15.61:6789/0 conn(0x7f163c0af5c0 legacy=0x7f163c143f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:37.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.136+0000 7f1641577700 1 -- 172.21.15.32:0/646818446 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c061b30 con 0x7f163c13f5b0 2020-12-05T17:37:37.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.136+0000 7f1633fff700 1 --2- 172.21.15.32:0/646818446 >> v2:172.21.15.61:3300/0 conn(0x7f163c0af9a0 0x7f163c144610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:37:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.136+0000 7f1641577700 1 --2- 172.21.15.32:0/646818446 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f163c13f5b0 0x7f163c13fa20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f16340032b0 tx=0x7f16340091a0).ready entity=mon.2 client_cookie=8b0d7aa197a8715c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.136+0000 7f16337fe700 1 -- 172.21.15.32:0/646818446 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f163400b940 con 0x7f163c13f5b0 2020-12-05T17:37:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.136+0000 7f16337fe700 1 -- 172.21.15.32:0/646818446 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163400baa0 con 0x7f163c13f5b0 2020-12-05T17:37:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.136+0000 7f16337fe700 1 -- 172.21.15.32:0/646818446 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f163400bc10 con 0x7f163c13f5b0 2020-12-05T17:37:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.136+0000 7f1641d78700 1 -- 172.21.15.32:0/646818446 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f163c13f5b0 msgr2=0x7f163c13fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.136+0000 7f1641d78700 1 --2- 172.21.15.32:0/646818446 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f163c13f5b0 0x7f163c13fa20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f16340032b0 tx=0x7f16340091a0).stop 2020-12-05T17:37:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.140+0000 7f1641d78700 1 -- 172.21.15.32:0/646818446 shutdown_connections 2020-12-05T17:37:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.140+0000 7f1641d78700 1 --2- 172.21.15.32:0/646818446 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f163c13f5b0 0x7f163c13fa20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.140+0000 7f1641d78700 1 --2- 172.21.15.32:0/646818446 >> v2:172.21.15.61:3300/0 conn(0x7f163c0af9a0 0x7f163c144610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.140+0000 7f1641d78700 1 -- 172.21.15.32:0/646818446 >> 172.21.15.32:0/646818446 conn(0x7f163c061010 msgr2=0x7f163c061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:37.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.140+0000 7f1641d78700 1 -- 172.21.15.32:0/646818446 shutdown_connections 2020-12-05T17:37:37.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.140+0000 7f1641d78700 1 -- 172.21.15.32:0/646818446 wait complete. 2020-12-05T17:37:37.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.140+0000 7f1641d78700 1 Processor -- start 2020-12-05T17:37:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.140+0000 7f1641d78700 1 -- start start 2020-12-05T17:37:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.144+0000 7f1641d78700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f163c0af5c0 0x7f163c10f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.144+0000 7f1641d78700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f163c0af9a0 0x7f163c14f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.144+0000 7f1641d78700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f163c13f5b0 0x7f163c147070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:37.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.144+0000 7f1641d78700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c153580 con 0x7f163c0af9a0 2020-12-05T17:37:37.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.144+0000 7f1641d78700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c064e90 con 0x7f163c13f5b0 2020-12-05T17:37:37.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.144+0000 7f1641d78700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163c065130 con 0x7f163c0af5c0 2020-12-05T17:37:37.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.144+0000 7f1633fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f163c0af9a0 0x7f163c14f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:37.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.144+0000 7f1640d76700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f163c0af5c0 0x7f163c10f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:37.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.144+0000 7f1633fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f163c0af9a0 0x7f163c14f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36346/0 (socket says 172.21.15.32:36346) 2020-12-05T17:37:37.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.144+0000 7f1633fff700 1 -- 172.21.15.32:0/3775037389 learned_addr learned my addr 172.21.15.32:0/3775037389 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:37.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.144+0000 7f1641577700 1 --2- 172.21.15.32:0/3775037389 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f163c13f5b0 0x7f163c147070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:37.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.144+0000 7f1633fff700 1 -- 172.21.15.32:0/3775037389 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f163c0af5c0 msgr2=0x7f163c10f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:37.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.144+0000 7f1633fff700 1 --2- 172.21.15.32:0/3775037389 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f163c0af5c0 0x7f163c10f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:37.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.144+0000 7f1633fff700 1 -- 172.21.15.32:0/3775037389 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f163c13f5b0 msgr2=0x7f163c147070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:37.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.144+0000 7f1633fff700 1 --2- 172.21.15.32:0/3775037389 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f163c13f5b0 0x7f163c147070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:37.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.144+0000 7f1633fff700 1 -- 172.21.15.32:0/3775037389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c14f670 con 0x7f163c0af9a0 2020-12-05T17:37:37.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.144+0000 7f1633fff700 1 --2- 172.21.15.32:0/3775037389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f163c0af9a0 0x7f163c14f180 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f162c00c000 tx=0x7f162c006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:37.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.144+0000 7f1631ffb700 1 -- 172.21.15.32:0/3775037389 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f162c00cb40 con 0x7f163c0af9a0 2020-12-05T17:37:37.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.144+0000 7f1631ffb700 1 -- 172.21.15.32:0/3775037389 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162c00cca0 con 0x7f163c0af9a0 2020-12-05T17:37:37.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.144+0000 7f1641d78700 1 -- 172.21.15.32:0/3775037389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163c0642a0 con 0x7f163c0af9a0 2020-12-05T17:37:37.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.144+0000 7f1631ffb700 1 -- 172.21.15.32:0/3775037389 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f162c0123f0 con 0x7f163c0af9a0 2020-12-05T17:37:37.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.144+0000 7f1641d78700 1 -- 172.21.15.32:0/3775037389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163c1532c0 con 0x7f163c0af9a0 2020-12-05T17:37:37.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.148+0000 7f1641d78700 1 -- 172.21.15.32:0/3775037389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f163c0654f0 con 0x7f163c0af9a0 2020-12-05T17:37:37.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.148+0000 7f1631ffb700 1 -- 172.21.15.32:0/3775037389 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f162c012550 con 0x7f163c0af9a0 2020-12-05T17:37:37.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.148+0000 7f1631ffb700 1 --2- 172.21.15.32:0/3775037389 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f161c055470 0x7f161c057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:37.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.148+0000 7f1631ffb700 1 -- 172.21.15.32:0/3775037389 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f162c06e7b0 con 0x7f163c0af9a0 2020-12-05T17:37:37.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.148+0000 7f1640d76700 1 --2- 172.21.15.32:0/3775037389 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f161c055470 0x7f161c057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:37.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.148+0000 7f1640d76700 1 --2- 172.21.15.32:0/3775037389 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f161c055470 0x7f161c057930 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f163800d000 tx=0x7f1638009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:37.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.156+0000 7f1631ffb700 1 -- 172.21.15.32:0/3775037389 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f162c0447e0 con 0x7f163c0af9a0 2020-12-05T17:37:37.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.304+0000 7f1641d78700 1 -- 172.21.15.32:0/3775037389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f163c0652f0 con 0x7f163c0af9a0 2020-12-05T17:37:37.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.312+0000 7f1631ffb700 1 -- 172.21.15.32:0/3775037389 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f162c047e00 con 0x7f163c0af9a0 2020-12-05T17:37:37.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.316+0000 7f1641d78700 1 -- 172.21.15.32:0/3775037389 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f161c055470 msgr2=0x7f161c057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:37.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.316+0000 7f1641d78700 1 --2- 172.21.15.32:0/3775037389 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f161c055470 0x7f161c057930 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f163800d000 tx=0x7f1638009040).stop 2020-12-05T17:37:37.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.316+0000 7f1641d78700 1 -- 172.21.15.32:0/3775037389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f163c0af9a0 msgr2=0x7f163c14f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:37.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.316+0000 7f1641d78700 1 --2- 172.21.15.32:0/3775037389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f163c0af9a0 0x7f163c14f180 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f162c00c000 tx=0x7f162c006dc0).stop 2020-12-05T17:37:37.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.316+0000 7f1641d78700 1 -- 172.21.15.32:0/3775037389 shutdown_connections 2020-12-05T17:37:37.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.316+0000 7f1641d78700 1 --2- 172.21.15.32:0/3775037389 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f163c13f5b0 0x7f163c147070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:37.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.316+0000 7f1641d78700 1 --2- 172.21.15.32:0/3775037389 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f161c055470 0x7f161c057930 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:37.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.316+0000 7f1641d78700 1 --2- 172.21.15.32:0/3775037389 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f163c0af5c0 0x7f163c10f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:37.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.316+0000 7f1641d78700 1 --2- 172.21.15.32:0/3775037389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f163c0af9a0 0x7f163c14f180 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:37.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.316+0000 7f1641d78700 1 -- 172.21.15.32:0/3775037389 >> 172.21.15.32:0/3775037389 conn(0x7f163c061010 msgr2=0x7f163c144450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:37.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.316+0000 7f1641d78700 1 -- 172.21.15.32:0/3775037389 shutdown_connections 2020-12-05T17:37:37.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.316+0000 7f1641d78700 1 -- 172.21.15.32:0/3775037389 wait complete. 2020-12-05T17:37:37.331 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:37:37.332 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:37:37.348 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:37:37.363 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:37:37.380 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:37:37.398 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:37:37.416 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:37:37.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:37.417 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:37:37.418 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:37:37.428 INFO:teuthology.orchestra.run.smithi036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-12-05T17:37:37.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:37:37.430 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-12-05T17:37:37.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:37.431 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:37:37.432 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:37:37.439 INFO:teuthology.orchestra.run.smithi002.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-12-05T17:37:37.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:37:37.441 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-12-05T17:37:37.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:37.443 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:37:37.444 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:37:37.451 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:37:37.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:37:37.453 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2020-12-05T17:37:37.453 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-12-05T17:37:37.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.576+0000 7f06f0d37700 1 Processor -- start 2020-12-05T17:37:37.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.576+0000 7f06f0d37700 1 -- start start 2020-12-05T17:37:37.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.576+0000 7f06f0d37700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06ec05f3b0 0x7f06ec05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:37.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.576+0000 7f06f0d37700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f06ec062400 0x7f06ec0b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:37.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.576+0000 7f06f0d37700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f06ec05feb0 0x7f06ec0b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:37.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.576+0000 7f06f0d37700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06ec142450 con 0x7f06ec05feb0 2020-12-05T17:37:37.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.576+0000 7f06f0d37700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06ec061780 con 0x7f06ec062400 2020-12-05T17:37:37.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06f0d37700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06ec061a20 con 0x7f06ec05f3b0 2020-12-05T17:37:37.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06ebfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f06ec05feb0 0x7f06ec0b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:37.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06ebfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f06ec05feb0 0x7f06ec0b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36356/0 (socket says 172.21.15.32:36356) 2020-12-05T17:37:37.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06ebfff700 1 -- 172.21.15.32:0/2993023065 learned_addr learned my addr 172.21.15.32:0/2993023065 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:37.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06e2ffd700 1 --2- 172.21.15.32:0/2993023065 >> v2:172.21.15.61:3300/0 conn(0x7f06ec062400 0x7f06ec0b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:37.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06ebfff700 1 -- 172.21.15.32:0/2993023065 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06ec05f3b0 msgr2=0x7f06ec05f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:37.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06eb7fe700 1 --2- 172.21.15.32:0/2993023065 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06ec05f3b0 0x7f06ec05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:37.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06ebfff700 1 --2- 172.21.15.32:0/2993023065 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06ec05f3b0 0x7f06ec05f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:37.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06ebfff700 1 -- 172.21.15.32:0/2993023065 >> v2:172.21.15.61:3300/0 conn(0x7f06ec062400 msgr2=0x7f06ec0b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:37.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06ebfff700 1 --2- 172.21.15.32:0/2993023065 >> v2:172.21.15.61:3300/0 conn(0x7f06ec062400 0x7f06ec0b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:37.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06ebfff700 1 -- 172.21.15.32:0/2993023065 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ec061cc0 con 0x7f06ec05feb0 2020-12-05T17:37:37.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06eb7fe700 1 --2- 172.21.15.32:0/2993023065 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06ec05f3b0 0x7f06ec05f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:37:37.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06ebfff700 1 --2- 172.21.15.32:0/2993023065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06ec05feb0 0x7f06ec0b3be0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f06e40032b0 tx=0x7f06e4009270).ready entity=mon.0 client_cookie=3ba7f558b11a3738 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:37.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06eaffd700 1 -- 172.21.15.32:0/2993023065 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06e400bc00 con 0x7f06ec05feb0 2020-12-05T17:37:37.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06eaffd700 1 -- 172.21.15.32:0/2993023065 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e400bd60 con 0x7f06ec05feb0 2020-12-05T17:37:37.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06eaffd700 1 -- 172.21.15.32:0/2993023065 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06e4011400 con 0x7f06ec05feb0 2020-12-05T17:37:37.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06f0d37700 1 -- 172.21.15.32:0/2993023065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06ec05feb0 msgr2=0x7f06ec0b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:37.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06f0d37700 1 --2- 172.21.15.32:0/2993023065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06ec05feb0 0x7f06ec0b3be0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f06e40032b0 tx=0x7f06e4009270).stop 2020-12-05T17:37:37.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06f0d37700 1 -- 172.21.15.32:0/2993023065 shutdown_connections 2020-12-05T17:37:37.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06f0d37700 1 --2- 172.21.15.32:0/2993023065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06ec05feb0 0x7f06ec0b3be0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:37.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06f0d37700 1 --2- 172.21.15.32:0/2993023065 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06ec05f3b0 0x7f06ec05f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:37.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06f0d37700 1 --2- 172.21.15.32:0/2993023065 >> v2:172.21.15.61:3300/0 conn(0x7f06ec062400 0x7f06ec0b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:37.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06f0d37700 1 -- 172.21.15.32:0/2993023065 >> 172.21.15.32:0/2993023065 conn(0x7f06ec0b8ee0 msgr2=0x7f06ec0b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:37.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06f0d37700 1 -- 172.21.15.32:0/2993023065 shutdown_connections 2020-12-05T17:37:37.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06f0d37700 1 -- 172.21.15.32:0/2993023065 wait complete. 2020-12-05T17:37:37.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06f0d37700 1 Processor -- start 2020-12-05T17:37:37.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.580+0000 7f06f0d37700 1 -- start start 2020-12-05T17:37:37.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.584+0000 7f06f0d37700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06ec05f3b0 0x7f06ec0a33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:37.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.584+0000 7f06f0d37700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06ec05feb0 0x7f06ec0a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:37.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.584+0000 7f06f0d37700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06ec062400 0x7f06ec0af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:37.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.584+0000 7f06f0d37700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ec0a25e0 con 0x7f06ec05feb0 2020-12-05T17:37:37.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.584+0000 7f06f0d37700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ec152270 con 0x7f06ec05f3b0 2020-12-05T17:37:37.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.584+0000 7f06f0d37700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06ec0649a0 con 0x7f06ec062400 2020-12-05T17:37:37.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.584+0000 7f06e2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06ec05feb0 0x7f06ec0a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:37.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.584+0000 7f06e2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06ec05feb0 0x7f06ec0a38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36360/0 (socket says 172.21.15.32:36360) 2020-12-05T17:37:37.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.584+0000 7f06e2ffd700 1 -- 172.21.15.32:0/2690048999 learned_addr learned my addr 172.21.15.32:0/2690048999 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:37.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.584+0000 7f06eb7fe700 1 --2- 172.21.15.32:0/2690048999 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06ec05f3b0 0x7f06ec0a33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:37.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.584+0000 7f06ebfff700 1 --2- 172.21.15.32:0/2690048999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06ec062400 0x7f06ec0af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:37.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.584+0000 7f06e2ffd700 1 -- 172.21.15.32:0/2690048999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06ec062400 msgr2=0x7f06ec0af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:37.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.584+0000 7f06e2ffd700 1 --2- 172.21.15.32:0/2690048999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06ec062400 0x7f06ec0af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:37.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.584+0000 7f06e2ffd700 1 -- 172.21.15.32:0/2690048999 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06ec05f3b0 msgr2=0x7f06ec0a33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:37.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.584+0000 7f06e2ffd700 1 --2- 172.21.15.32:0/2690048999 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06ec05f3b0 0x7f06ec0a33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:37.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.584+0000 7f06e2ffd700 1 -- 172.21.15.32:0/2690048999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ec151dd0 con 0x7f06ec05feb0 2020-12-05T17:37:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.584+0000 7f06e2ffd700 1 --2- 172.21.15.32:0/2690048999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06ec05feb0 0x7f06ec0a38c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f06d4003070 tx=0x7f06d4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.584+0000 7f06e97fa700 1 -- 172.21.15.32:0/2690048999 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06d4011780 con 0x7f06ec05feb0 2020-12-05T17:37:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.584+0000 7f06e97fa700 1 -- 172.21.15.32:0/2690048999 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d40118e0 con 0x7f06ec05feb0 2020-12-05T17:37:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.584+0000 7f06f0d37700 1 -- 172.21.15.32:0/2690048999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06ec151ae0 con 0x7f06ec05feb0 2020-12-05T17:37:37.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.584+0000 7f06e97fa700 1 -- 172.21.15.32:0/2690048999 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06d4011c40 con 0x7f06ec05feb0 2020-12-05T17:37:37.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.584+0000 7f06f0d37700 1 -- 172.21.15.32:0/2690048999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ec064500 con 0x7f06ec05feb0 2020-12-05T17:37:37.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.584+0000 7f06f0d37700 1 -- 172.21.15.32:0/2690048999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06ec151f60 con 0x7f06ec05feb0 2020-12-05T17:37:37.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.588+0000 7f06e97fa700 1 -- 172.21.15.32:0/2690048999 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f06d401c460 con 0x7f06ec05feb0 2020-12-05T17:37:37.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.588+0000 7f06e97fa700 1 --2- 172.21.15.32:0/2690048999 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f06c4055330 0x7f06c40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:37.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.588+0000 7f06e97fa700 1 -- 172.21.15.32:0/2690048999 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f06d406e390 con 0x7f06ec05feb0 2020-12-05T17:37:37.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.588+0000 7f06eb7fe700 1 --2- 172.21.15.32:0/2690048999 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f06c4055330 0x7f06c40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:37.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.588+0000 7f06eb7fe700 1 --2- 172.21.15.32:0/2690048999 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f06c4055330 0x7f06c40577f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f06dc00d000 tx=0x7f06dc008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:37.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.596+0000 7f06e97fa700 1 -- 172.21.15.32:0/2690048999 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f06d4044430 con 0x7f06ec05feb0 2020-12-05T17:37:37.740 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.740+0000 7f06f0d37700 1 -- 172.21.15.32:0/2690048999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f06ec057050 con 0x7f06ec05feb0 2020-12-05T17:37:37.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.740+0000 7f06e97fa700 1 -- 172.21.15.32:0/2690048999 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v62) v1 ==== 81+0+34040 (secure 0 0 0) 0x7f06d4047a50 con 0x7f06ec05feb0 2020-12-05T17:37:37.743 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:37.744 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:37:37.744 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 62, 2020-12-05T17:37:37.744 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", 2020-12-05T17:37:37.744 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2020-12-05T17:31:20.975238+0000", 2020-12-05T17:37:37.744 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2020-12-05T17:37:01.403672+0000", 2020-12-05T17:37:37.745 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:37:37.745 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:37:37.745 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-12-05T17:37:37.745 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2020-12-05T17:37:37.745 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2020-12-05T17:37:37.746 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2020-12-05T17:37:37.746 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2020-12-05T17:37:37.746 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2020-12-05T17:37:37.746 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2020-12-05T17:37:37.747 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:37:37.747 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2020-12-05T17:37:37.747 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2020-12-05T17:37:37.747 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-12-05T17:37:37.747 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2020-12-05T17:37:37.748 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2020-12-05T17:37:37.748 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 12, 2020-12-05T17:37:37.748 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2020-12-05T17:37:37.748 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2020-12-05T17:37:37.748 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2020-12-05T17:37:37.749 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2020-12-05T17:37:37.749 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2020-12-05T17:37:37.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.749 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2020-12-05T17:37:37.750 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2020-12-05T17:37:37.750 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:24.858732+0000", 2020-12-05T17:37:37.750 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:37:37.750 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:37:37.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:37:37.751 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:37:37.751 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:37:37.751 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:37:37.751 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:37:37.751 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:37:37.752 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2020-12-05T17:37:37.752 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2020-12-05T17:37:37.752 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2020-12-05T17:37:37.752 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2020-12-05T17:37:37.752 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2020-12-05T17:37:37.753 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:37:37.753 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:37:37.753 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:37:37.753 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:37:37.753 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:37:37.754 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:37:37.754 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:37:37.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.754 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "16", 2020-12-05T17:37:37.754 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:37:37.755 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:37:37.755 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:37:37.755 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:37:37.755 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:37:37.755 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:37:37.756 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:37:37.756 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:37:37.756 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:37:37.756 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:37:37.756 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:37:37.757 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:37:37.757 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:37:37.757 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:37:37.757 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:37:37.757 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:37:37.758 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:37:37.758 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:37:37.758 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:37:37.758 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:37:37.758 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:37:37.759 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:37:37.759 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:37:37.759 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:37:37.759 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:37:37.759 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:37:37.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.760 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:37:37.760 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:37:37.760 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:37:37.760 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:37:37.761 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:37:37.761 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:37:37.761 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:37:37.761 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:37:37.761 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:37:37.762 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:37:37.762 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:37:37.762 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:37:37.762 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2020-12-05T17:37:37.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.763 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:37:37.763 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2020-12-05T17:37:37.763 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.764 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2020-12-05T17:37:37.764 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2020-12-05T17:37:37.764 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:33.305595+0000", 2020-12-05T17:37:37.764 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:37:37.764 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:37:37.765 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:37:37.765 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:37:37.765 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:37:37.765 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:37:37.765 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:37:37.766 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:37:37.766 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:37:37.766 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:37:37.766 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:37:37.766 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:37:37.767 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:37:37.767 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:37:37.767 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:37:37.767 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:37:37.767 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:37:37.768 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:37:37.768 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:37:37.768 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:37:37.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.768 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "19", 2020-12-05T17:37:37.769 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:37:37.769 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:37:37.769 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:37:37.769 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:37:37.769 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:37:37.770 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:37:37.770 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:37:37.770 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:37:37.770 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:37:37.771 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:37:37.771 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:37:37.771 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:37:37.771 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:37:37.771 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:37:37.772 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:37:37.772 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:37:37.772 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:37:37.772 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:37:37.772 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:37:37.773 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:37:37.773 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:37:37.773 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:37:37.773 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:37:37.773 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:37:37.774 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:37:37.774 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:37:37.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.774 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:37:37.774 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:37:37.775 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:37:37.775 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:37:37.775 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:37:37.775 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:37:37.775 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:37:37.776 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:37:37.776 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:37:37.776 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:37:37.776 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:37:37.776 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:37:37.777 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:37:37.777 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2020-12-05T17:37:37.777 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.778 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 11, 2020-12-05T17:37:37.778 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2020-12-05T17:37:37.778 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:36:59.960851+0000", 2020-12-05T17:37:37.778 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:37:37.778 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:37:37.779 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:37:37.779 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:37:37.779 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:37:37.779 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:37:37.780 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:37:37.780 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:37:37.780 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:37:37.780 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:37:37.780 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:37:37.781 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:37:37.781 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:37:37.781 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:37:37.781 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:37:37.781 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:37:37.782 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:37:37.782 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:37:37.782 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:37:37.782 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:37:37.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.783 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "62", 2020-12-05T17:37:37.783 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:37:37.783 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:37:37.783 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:37:37.783 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:37:37.784 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:37:37.784 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:37:37.784 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:37:37.784 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:37:37.784 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:37:37.785 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:37:37.785 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:37:37.785 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:37:37.785 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:37:37.785 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:37:37.786 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:37:37.786 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:37:37.786 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:37:37.786 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:37:37.786 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:37:37.787 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:37:37.787 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:37:37.787 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:37:37.787 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:37:37.787 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:37:37.788 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:37:37.788 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:37:37.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.788 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:37:37.788 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:37:37.789 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:37:37.789 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:37:37.789 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:37:37.789 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:37:37.789 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:37:37.790 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:37:37.790 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:37:37.790 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:37:37.790 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:37:37.790 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:37:37.791 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2020-12-05T17:37:37.791 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2020-12-05T17:37:37.791 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2020-12-05T17:37:37.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.791 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:37:37.792 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:37:37.792 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2020-12-05T17:37:37.792 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.792 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.793 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 12, 2020-12-05T17:37:37.793 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2020-12-05T17:37:37.793 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:37:00.401054+0000", 2020-12-05T17:37:37.794 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:37:37.794 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:37:37.794 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:37:37.794 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:37:37.794 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:37:37.795 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:37:37.795 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:37:37.795 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:37:37.795 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:37:37.795 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:37:37.796 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:37:37.796 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:37:37.796 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:37:37.796 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:37:37.796 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:37:37.797 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:37:37.797 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:37:37.797 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:37:37.797 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:37:37.797 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:37:37.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.798 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "62", 2020-12-05T17:37:37.798 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:37:37.798 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:37:37.798 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:37:37.799 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:37:37.799 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:37:37.799 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:37:37.799 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:37:37.799 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:37:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:37:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:37:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:37:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:37:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:37:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:37:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:37:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:37:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:37:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:37:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:37:37.802 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:37:37.802 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:37:37.802 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:37:37.802 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:37:37.803 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:37:37.803 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:37:37.803 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:37:37.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.803 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:37:37.804 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:37:37.804 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:37:37.804 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:37:37.804 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:37:37.804 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:37:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:37:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:37:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:37:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:37:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:37:37.806 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:37:37.806 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:37:37.806 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:37:37.806 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2020-12-05T17:37:37.806 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:37:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2020-12-05T17:37:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:37:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "b941127b-0a49-4e06-8d11-17794ff8b013", 2020-12-05T17:37:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:37:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:37:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:37:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:37:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:37:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:37:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:37:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 60, 2020-12-05T17:37:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:37:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:37:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:37:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2020-12-05T17:37:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:37:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2020-12-05T17:37:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:37:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:37:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.814 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.814 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2020-12-05T17:37:37.814 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:37:37.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2020-12-05T17:37:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:37:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:37:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2020-12-05T17:37:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:37:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2020-12-05T17:37:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:37:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:37:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2020-12-05T17:37:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:37:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2020-12-05T17:37:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:37:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6803/11526", 2020-12-05T17:37:37.822 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6805/11526", 2020-12-05T17:37:37.822 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6809/11526", 2020-12-05T17:37:37.822 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6807/11526", 2020-12-05T17:37:37.822 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:37:37.822 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:37:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:37:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:37:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "01bd7ff5-a86b-4e11-ba1e-484c309c0fba", 2020-12-05T17:37:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:37:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:37:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:37:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:37:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:37:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:37:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:37:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 60, 2020-12-05T17:37:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:37:37.826 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:37:37.826 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:37:37.826 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.826 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.827 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2020-12-05T17:37:37.827 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:37:37.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.827 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.828 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2020-12-05T17:37:37.828 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:37:37.828 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.828 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:37:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2020-12-05T17:37:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:37:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2020-12-05T17:37:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:37:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:37:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2020-12-05T17:37:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:37:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2020-12-05T17:37:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:37:37.834 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.834 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.834 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:37:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2020-12-05T17:37:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:37:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2020-12-05T17:37:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:37:37.837 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.837 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.837 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6819/11538", 2020-12-05T17:37:37.837 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6821/11538", 2020-12-05T17:37:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6825/11538", 2020-12-05T17:37:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6823/11538", 2020-12-05T17:37:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:37:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:37:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:37:37.839 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.839 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:37:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "94736ef5-64c2-4a97-b963-bd38cddd9c52", 2020-12-05T17:37:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:37:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:37:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:37:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:37:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:37:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:37:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:37:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 59, 2020-12-05T17:37:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:37:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:37:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:37:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.843 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.843 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2020-12-05T17:37:37.843 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:37:37.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.844 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.844 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2020-12-05T17:37:37.844 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:37:37.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.845 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:37:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2020-12-05T17:37:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:37:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2020-12-05T17:37:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:37:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:37:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2020-12-05T17:37:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:37:37.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.850 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.850 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2020-12-05T17:37:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:37:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:37:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2020-12-05T17:37:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:37:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2020-12-05T17:37:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:37:37.854 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.854 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.854 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6811/11521", 2020-12-05T17:37:37.854 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6813/11521", 2020-12-05T17:37:37.855 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6817/11521", 2020-12-05T17:37:37.855 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6815/11521", 2020-12-05T17:37:37.855 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:37:37.855 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:37:37.855 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:37:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:37:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "69a1f3ff-fd25-43df-b60c-e32a89c35234", 2020-12-05T17:37:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:37:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:37:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:37:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:37:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:37:37.858 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:37:37.858 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:37:37.858 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 50, 2020-12-05T17:37:37.858 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:37:37.858 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:37:37.859 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:37:37.859 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.859 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.859 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2020-12-05T17:37:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:37:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2020-12-05T17:37:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:37:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:37:37.862 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.862 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.862 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2020-12-05T17:37:37.862 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:37:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2020-12-05T17:37:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:37:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:37:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6832", 2020-12-05T17:37:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:37:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.866 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.866 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6833", 2020-12-05T17:37:37.866 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:37:37.866 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:37:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2020-12-05T17:37:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:37:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2020-12-05T17:37:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:37:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6827/11535", 2020-12-05T17:37:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6829/11535", 2020-12-05T17:37:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6833/11535", 2020-12-05T17:37:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6831/11535", 2020-12-05T17:37:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:37:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:37:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:37:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:37:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c301fe4e-5eab-402c-ae6f-db3e6a7181af", 2020-12-05T17:37:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:37:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:37:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:37:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:37:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:37:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:37:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:37:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 60, 2020-12-05T17:37:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:37:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:37:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:37:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6808", 2020-12-05T17:37:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:37:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.876 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.876 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6809", 2020-12-05T17:37:37.876 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:37:37.876 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.877 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.877 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:37:37.877 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.878 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.878 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6810", 2020-12-05T17:37:37.878 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:37:37.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.879 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.879 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6811", 2020-12-05T17:37:37.879 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:37:37.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.879 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.880 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:37:37.880 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.880 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.881 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6817", 2020-12-05T17:37:37.881 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:37:37.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.881 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.882 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6819", 2020-12-05T17:37:37.882 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:37:37.882 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.882 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.883 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:37:37.883 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.883 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.884 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6812", 2020-12-05T17:37:37.884 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:37:37.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.884 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.885 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6814", 2020-12-05T17:37:37.885 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:37:37.885 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.885 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.886 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6809/11541", 2020-12-05T17:37:37.886 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6811/11541", 2020-12-05T17:37:37.886 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6819/11541", 2020-12-05T17:37:37.886 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6814/11541", 2020-12-05T17:37:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:37:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:37:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:37:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.888 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:37:37.888 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "9476eff4-345b-46c5-a473-1d9a89a6d327", 2020-12-05T17:37:37.888 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:37:37.888 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:37:37.889 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:37:37.889 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:37:37.889 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:37:37.889 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:37:37.889 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:37:37.890 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 59, 2020-12-05T17:37:37.890 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:37:37.890 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:37:37.890 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:37:37.890 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6800", 2020-12-05T17:37:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:37:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.892 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.892 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6801", 2020-12-05T17:37:37.892 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:37:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:37:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.894 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.894 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6802", 2020-12-05T17:37:37.894 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:37:37.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6803", 2020-12-05T17:37:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:37:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:37:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6806", 2020-12-05T17:37:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:37:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6807", 2020-12-05T17:37:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:37:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:37:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6804", 2020-12-05T17:37:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:37:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6805", 2020-12-05T17:37:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:37:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6801/11539", 2020-12-05T17:37:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6803/11539", 2020-12-05T17:37:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6807/11539", 2020-12-05T17:37:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6805/11539", 2020-12-05T17:37:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:37:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:37:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:37:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:37:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8dfd8567-9b1c-4d13-9c96-071183e469c2", 2020-12-05T17:37:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:37:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:37:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:37:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:37:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:37:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:37:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:37:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 60, 2020-12-05T17:37:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:37:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:37:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:37:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6820", 2020-12-05T17:37:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:37:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6822", 2020-12-05T17:37:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:37:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:37:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6823", 2020-12-05T17:37:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:37:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6825", 2020-12-05T17:37:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:37:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:37:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6829", 2020-12-05T17:37:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:37:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6830", 2020-12-05T17:37:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:37:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:37:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6826", 2020-12-05T17:37:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:37:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6827", 2020-12-05T17:37:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:37:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6822/11542", 2020-12-05T17:37:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6825/11542", 2020-12-05T17:37:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6830/11542", 2020-12-05T17:37:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6827/11542", 2020-12-05T17:37:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:37:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:37:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:37:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:37:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "24d466b3-01dc-4ba2-8b81-85250d163bd2", 2020-12-05T17:37:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:37:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:37:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:37:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:37:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:37:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:37:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:37:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 59, 2020-12-05T17:37:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:37:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:37:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:37:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6813", 2020-12-05T17:37:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:37:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6815", 2020-12-05T17:37:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:37:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:37:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6816", 2020-12-05T17:37:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:37:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6818", 2020-12-05T17:37:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:37:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:37:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6828", 2020-12-05T17:37:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:37:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6831", 2020-12-05T17:37:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:37:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:37:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:37:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:37:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6821", 2020-12-05T17:37:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:37:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:37:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6824", 2020-12-05T17:37:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:37:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6815/11540", 2020-12-05T17:37:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6818/11540", 2020-12-05T17:37:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6831/11540", 2020-12-05T17:37:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6824/11540", 2020-12-05T17:37:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:37:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:37:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:37:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:37:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2020-12-05T17:37:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:37:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:37:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:37:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:37:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:37:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:37:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.398868+0000", 2020-12-05T17:37:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:37:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:37:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:37:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:37:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:37:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:37:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:37:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.675748+0000", 2020-12-05T17:37:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:37:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:37:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:37:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:37:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:37:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:37:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:37:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.620888+0000", 2020-12-05T17:37:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:37:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:37:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:37:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:37:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:37:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:37:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:37:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.681600+0000", 2020-12-05T17:37:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:37:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:37:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:37:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:37:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:37:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:37:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:37:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.658769+0000", 2020-12-05T17:37:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:37:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:37:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:37:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:37:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:37:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:37:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:37:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.630708+0000", 2020-12-05T17:37:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:37:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:37:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:37:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:37:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:37:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:37:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:37:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.648352+0000", 2020-12-05T17:37:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:37:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:37:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:37:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:37:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:37:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:37:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:37:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.629399+0000", 2020-12-05T17:37:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:37:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:37:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2020-12-05T17:37:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2020-12-05T17:37:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2020-12-05T17:37:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2020-12-05T17:37:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": {}, 2020-12-05T17:37:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2020-12-05T17:37:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2020-12-05T17:37:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2020-12-05T17:37:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:37:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2020-12-05T17:37:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2020-12-05T17:37:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:37:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2020-12-05T17:37:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:37:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:37:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2020-12-05T17:37:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2020-12-05T17:37:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2020-12-05T17:37:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:37:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:37:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2020-12-05T17:37:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:37:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:37:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2020-12-05T17:37:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2020-12-05T17:37:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2020-12-05T17:37:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2020-12-05T17:37:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2020-12-05T17:37:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2020-12-05T17:37:37.960 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:37:37.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.744+0000 7f06f0d37700 1 -- 172.21.15.32:0/2690048999 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f06c4055330 msgr2=0x7f06c40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:37.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.744+0000 7f06f0d37700 1 --2- 172.21.15.32:0/2690048999 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f06c4055330 0x7f06c40577f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f06dc00d000 tx=0x7f06dc008040).stop 2020-12-05T17:37:37.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.744+0000 7f06f0d37700 1 -- 172.21.15.32:0/2690048999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06ec05feb0 msgr2=0x7f06ec0a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:37.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.744+0000 7f06f0d37700 1 --2- 172.21.15.32:0/2690048999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06ec05feb0 0x7f06ec0a38c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f06d4003070 tx=0x7f06d4008040).stop 2020-12-05T17:37:37.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.744+0000 7f06f0d37700 1 -- 172.21.15.32:0/2690048999 shutdown_connections 2020-12-05T17:37:37.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.744+0000 7f06f0d37700 1 --2- 172.21.15.32:0/2690048999 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f06c4055330 0x7f06c40577f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:37.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.744+0000 7f06f0d37700 1 --2- 172.21.15.32:0/2690048999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06ec062400 0x7f06ec0af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:37.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.744+0000 7f06f0d37700 1 --2- 172.21.15.32:0/2690048999 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06ec05f3b0 0x7f06ec0a33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:37.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.744+0000 7f06f0d37700 1 --2- 172.21.15.32:0/2690048999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06ec05feb0 0x7f06ec0a38c0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:37.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.744+0000 7f06f0d37700 1 -- 172.21.15.32:0/2690048999 >> 172.21.15.32:0/2690048999 conn(0x7f06ec0b8ee0 msgr2=0x7f06ec0b83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:37.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.744+0000 7f06f0d37700 1 -- 172.21.15.32:0/2690048999 shutdown_connections 2020-12-05T17:37:37.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:37.744+0000 7f06f0d37700 1 -- 172.21.15.32:0/2690048999 wait complete. 2020-12-05T17:37:37.965 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:37:38.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.088+0000 7f1893fff700 1 Processor -- start 2020-12-05T17:37:38.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.088+0000 7f1893fff700 1 -- start start 2020-12-05T17:37:38.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.088+0000 7f1893fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f18940b1fc0 0x7f18940af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:38.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.088+0000 7f1893fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18940ab710 0x7f1894061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:38.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.088+0000 7f1893fff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1894146a90 con 0x7f1894147400 2020-12-05T17:37:38.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.088+0000 7f1893fff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18940b0110 con 0x7f18940b1fc0 2020-12-05T17:37:38.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.088+0000 7f1893fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18940b23a0 con 0x7f18940ab710 2020-12-05T17:37:38.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.088+0000 7f1892ffd700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f18940b1fc0 0x7f18940af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:38.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.088+0000 7f18927fc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18940ab710 0x7f1894061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:38.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.088+0000 7f18937fe700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f1894147400 0x7f1894145d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:51658/0 (socket says 172.21.15.32:51658) 2020-12-05T17:37:38.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.088+0000 7f18937fe700 1 -- 172.21.15.32:0/1865842866 learned_addr learned my addr 172.21.15.32:0/1865842866 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:38.094 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.088+0000 7f18927fc700 1 -- 172.21.15.32:0/1865842866 >> v2:172.21.15.61:3300/0 conn(0x7f18940b1fc0 msgr2=0x7f18940af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:38.094 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.088+0000 7f18927fc700 1 --2- 172.21.15.32:0/1865842866 >> v2:172.21.15.61:3300/0 conn(0x7f18940b1fc0 0x7f18940af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:38.094 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.088+0000 7f18927fc700 1 -- 172.21.15.32:0/1865842866 >> v1:172.21.15.61:6789/0 conn(0x7f1894147400 legacy=0x7f1894145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:38.094 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.088+0000 7f18927fc700 1 -- 172.21.15.32:0/1865842866 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1894146cc0 con 0x7f18940ab710 2020-12-05T17:37:38.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.088+0000 7f1892ffd700 1 --2- 172.21.15.32:0/1865842866 >> v2:172.21.15.61:3300/0 conn(0x7f18940b1fc0 0x7f18940af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:37:38.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.088+0000 7f18927fc700 1 --2- 172.21.15.32:0/1865842866 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18940ab710 0x7f1894061780 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f187c002f30 tx=0x7f187c006dc0).ready entity=mon.2 client_cookie=1028be2905164514 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:38.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.092+0000 7f1891ffb700 1 -- 172.21.15.32:0/1865842866 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f187c009760 con 0x7f18940ab710 2020-12-05T17:37:38.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.092+0000 7f1891ffb700 1 -- 172.21.15.32:0/1865842866 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187c0098c0 con 0x7f18940ab710 2020-12-05T17:37:38.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.092+0000 7f1891ffb700 1 -- 172.21.15.32:0/1865842866 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f187c009c20 con 0x7f18940ab710 2020-12-05T17:37:38.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.092+0000 7f1893fff700 1 -- 172.21.15.32:0/1865842866 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18940ab710 msgr2=0x7f1894061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:38.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.092+0000 7f1893fff700 1 --2- 172.21.15.32:0/1865842866 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18940ab710 0x7f1894061780 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f187c002f30 tx=0x7f187c006dc0).stop 2020-12-05T17:37:38.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.092+0000 7f1893fff700 1 -- 172.21.15.32:0/1865842866 shutdown_connections 2020-12-05T17:37:38.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.092+0000 7f1893fff700 1 --2- 172.21.15.32:0/1865842866 >> v2:172.21.15.61:3300/0 conn(0x7f18940b1fc0 0x7f18940af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:38.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.092+0000 7f1893fff700 1 --2- 172.21.15.32:0/1865842866 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18940ab710 0x7f1894061780 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:38.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.092+0000 7f1893fff700 1 -- 172.21.15.32:0/1865842866 >> 172.21.15.32:0/1865842866 conn(0x7f18940b5530 msgr2=0x7f18940b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:38.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.092+0000 7f1893fff700 1 -- 172.21.15.32:0/1865842866 shutdown_connections 2020-12-05T17:37:38.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.092+0000 7f1893fff700 1 -- 172.21.15.32:0/1865842866 wait complete. 2020-12-05T17:37:38.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f1893fff700 1 Processor -- start 2020-12-05T17:37:38.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f1893fff700 1 -- start start 2020-12-05T17:37:38.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f1893fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18940ab710 0x7f18940abe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:38.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f1893fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18940b1fc0 0x7f18940acd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:38.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f1893fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1894147400 0x7f18940ad250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:38.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f1893fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189414a640 con 0x7f18940b1fc0 2020-12-05T17:37:38.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f1893fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18940a1f70 con 0x7f18940ab710 2020-12-05T17:37:38.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f1893fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189409f230 con 0x7f1894147400 2020-12-05T17:37:38.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f18927fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18940b1fc0 0x7f18940acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:38.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f18927fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18940b1fc0 0x7f18940acd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36372/0 (socket says 172.21.15.32:36372) 2020-12-05T17:37:38.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f18927fc700 1 -- 172.21.15.32:0/3045182416 learned_addr learned my addr 172.21.15.32:0/3045182416 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:38.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f18937fe700 1 --2- 172.21.15.32:0/3045182416 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1894147400 0x7f18940ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:38.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f1892ffd700 1 --2- 172.21.15.32:0/3045182416 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18940ab710 0x7f18940abe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:38.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f18927fc700 1 -- 172.21.15.32:0/3045182416 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1894147400 msgr2=0x7f18940ad250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:38.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f18927fc700 1 --2- 172.21.15.32:0/3045182416 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1894147400 0x7f18940ad250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:38.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f18927fc700 1 -- 172.21.15.32:0/3045182416 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18940ab710 msgr2=0x7f18940abe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:38.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f18927fc700 1 --2- 172.21.15.32:0/3045182416 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18940ab710 0x7f18940abe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:38.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f18927fc700 1 -- 172.21.15.32:0/3045182416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18940a1830 con 0x7f18940b1fc0 2020-12-05T17:37:38.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f18927fc700 1 --2- 172.21.15.32:0/3045182416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18940b1fc0 0x7f18940acd60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f187c002f30 tx=0x7f187c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:38.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f188bfff700 1 -- 172.21.15.32:0/3045182416 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f187c011960 con 0x7f18940b1fc0 2020-12-05T17:37:38.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f188bfff700 1 -- 172.21.15.32:0/3045182416 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187c011ac0 con 0x7f18940b1fc0 2020-12-05T17:37:38.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f1893fff700 1 -- 172.21.15.32:0/3045182416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18940a1ad0 con 0x7f18940b1fc0 2020-12-05T17:37:38.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f188bfff700 1 -- 172.21.15.32:0/3045182416 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f187c011e20 con 0x7f18940b1fc0 2020-12-05T17:37:38.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.096+0000 7f1893fff700 1 -- 172.21.15.32:0/3045182416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1894147ec0 con 0x7f18940b1fc0 2020-12-05T17:37:38.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.100+0000 7f1893fff700 1 -- 172.21.15.32:0/3045182416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189409f0a0 con 0x7f18940b1fc0 2020-12-05T17:37:38.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.100+0000 7f188bfff700 1 -- 172.21.15.32:0/3045182416 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f187c024030 con 0x7f18940b1fc0 2020-12-05T17:37:38.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.100+0000 7f188bfff700 1 --2- 172.21.15.32:0/3045182416 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1878055440 0x7f1878057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:38.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.100+0000 7f188bfff700 1 -- 172.21.15.32:0/3045182416 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f187c06eae0 con 0x7f18940b1fc0 2020-12-05T17:37:38.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.100+0000 7f1892ffd700 1 --2- 172.21.15.32:0/3045182416 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1878055440 0x7f1878057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:38.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.100+0000 7f1892ffd700 1 --2- 172.21.15.32:0/3045182416 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1878055440 0x7f1878057900 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f188400b000 tx=0x7f1884006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:38.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.104+0000 7f188bfff700 1 -- 172.21.15.32:0/3045182416 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f187c044c80 con 0x7f18940b1fc0 2020-12-05T17:37:38.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.296+0000 7f1893fff700 1 -- 172.21.15.32:0/3045182416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18940a1c90 con 0x7f18940b1fc0 2020-12-05T17:37:38.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.296+0000 7f188bfff700 1 -- 172.21.15.32:0/3045182416 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f187c022020 con 0x7f18940b1fc0 2020-12-05T17:37:38.298 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:38.299 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5411,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/3995477718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3995477718},{"type":"v1","addr":"172.21.15.61:6833","nonce":3995477718}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:37:01.403820+0000","modified":"2020-12-05T17:37:02.406060+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5976},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5976":{"gid":5976,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/738464306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":738464306},{"type":"v1","addr":"172.21.15.32:6835","nonce":738464306}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-12-05T17:37:38.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.296+0000 7f1893fff700 1 -- 172.21.15.32:0/3045182416 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1878055440 msgr2=0x7f1878057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:38.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.296+0000 7f1893fff700 1 --2- 172.21.15.32:0/3045182416 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1878055440 0x7f1878057900 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f188400b000 tx=0x7f1884006dc0).stop 2020-12-05T17:37:38.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.296+0000 7f1893fff700 1 -- 172.21.15.32:0/3045182416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18940b1fc0 msgr2=0x7f18940acd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:38.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.296+0000 7f1893fff700 1 --2- 172.21.15.32:0/3045182416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18940b1fc0 0x7f18940acd60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f187c002f30 tx=0x7f187c015000).stop 2020-12-05T17:37:38.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.296+0000 7f1893fff700 1 -- 172.21.15.32:0/3045182416 shutdown_connections 2020-12-05T17:37:38.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.296+0000 7f1893fff700 1 --2- 172.21.15.32:0/3045182416 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1878055440 0x7f1878057900 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:38.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.296+0000 7f1893fff700 1 --2- 172.21.15.32:0/3045182416 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1894147400 0x7f18940ad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:38.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.296+0000 7f1893fff700 1 --2- 172.21.15.32:0/3045182416 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18940ab710 0x7f18940abe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:38.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.296+0000 7f1893fff700 1 --2- 172.21.15.32:0/3045182416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18940b1fc0 0x7f18940acd60 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:38.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.296+0000 7f1893fff700 1 -- 172.21.15.32:0/3045182416 >> 172.21.15.32:0/3045182416 conn(0x7f18940b5530 msgr2=0x7f1894144b20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:38.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.296+0000 7f1893fff700 1 -- 172.21.15.32:0/3045182416 shutdown_connections 2020-12-05T17:37:38.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.296+0000 7f1893fff700 1 -- 172.21.15.32:0/3045182416 wait complete. 2020-12-05T17:37:38.306 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 27 2020-12-05T17:37:38.312 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-12-05T17:37:38.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cf496700 1 Processor -- start 2020-12-05T17:37:38.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cf496700 1 -- start start 2020-12-05T17:37:38.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cf496700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8c805feb0 0x7fb8c805f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:38.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cf496700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8c8142450 con 0x7fb8c8062400 2020-12-05T17:37:38.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cf496700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8c8061780 con 0x7fb8c805f3b0 2020-12-05T17:37:38.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cf496700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c8061a20 con 0x7fb8c805feb0 2020-12-05T17:37:38.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cdc93700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb8c8062400 0x7fb8c80b0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39030/0 (socket says 172.21.15.32:39030) 2020-12-05T17:37:38.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cdc93700 1 -- 172.21.15.32:0/4278257062 learned_addr learned my addr 172.21.15.32:0/4278257062 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:38.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cec95700 1 --2- 172.21.15.32:0/4278257062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8c805feb0 0x7fb8c805f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:38.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cd492700 1 -- 172.21.15.32:0/4278257062 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1076379433 0 0) 0x7fb8c8142450 con 0x7fb8c8062400 2020-12-05T17:37:38.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cd492700 1 -- 172.21.15.32:0/4278257062 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8b0003230 con 0x7fb8c8062400 2020-12-05T17:37:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cd492700 1 -- 172.21.15.32:0/4278257062 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3472994205 0 0) 0x7fb8b0003230 con 0x7fb8c8062400 2020-12-05T17:37:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cd492700 1 -- 172.21.15.32:0/4278257062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8c805feb0 msgr2=0x7fb8c805f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cd492700 1 --2- 172.21.15.32:0/4278257062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8c805feb0 0x7fb8c805f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cd492700 1 -- 172.21.15.32:0/4278257062 >> v1:172.21.15.61:6789/0 conn(0x7fb8c805f3b0 legacy=0x7fb8c80b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cd492700 1 -- 172.21.15.32:0/4278257062 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c8061cc0 con 0x7fb8c8062400 2020-12-05T17:37:38.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cd492700 1 -- 172.21.15.32:0/4278257062 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fb8bc001a50 con 0x7fb8c8062400 2020-12-05T17:37:38.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cd492700 1 -- 172.21.15.32:0/4278257062 >> v1:172.21.15.32:6789/0 conn(0x7fb8c8062400 legacy=0x7fb8c80b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:38.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cd492700 1 --2- 172.21.15.32:0/4278257062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8b0003750 0x7fb8b0003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:38.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cd492700 1 --2- 172.21.15.32:0/4278257062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8c805feb0 0x7fb8b00044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:38.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cd492700 1 --2- 172.21.15.32:0/4278257062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb8b00049c0 0x7fb8b0004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:38.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cd492700 1 -- 172.21.15.32:0/4278257062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c8061a20 con 0x7fb8b0003750 2020-12-05T17:37:38.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cd492700 1 -- 172.21.15.32:0/4278257062 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b0004160 con 0x7fb8b00049c0 2020-12-05T17:37:38.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cd492700 1 -- 172.21.15.32:0/4278257062 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b0004320 con 0x7fb8c805feb0 2020-12-05T17:37:38.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8ce494700 1 --2- 172.21.15.32:0/4278257062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8b0003750 0x7fb8b0003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:38.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cec95700 1 --2- 172.21.15.32:0/4278257062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb8b00049c0 0x7fb8b0004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:38.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cdc93700 1 --2- 172.21.15.32:0/4278257062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8c805feb0 0x7fb8b00044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:38.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8ce494700 1 -- 172.21.15.32:0/4278257062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8c805feb0 msgr2=0x7fb8b00044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:38.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8ce494700 1 --2- 172.21.15.32:0/4278257062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8c805feb0 0x7fb8b00044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:38.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8ce494700 1 -- 172.21.15.32:0/4278257062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb8b00049c0 msgr2=0x7fb8b0004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:38.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8ce494700 1 --2- 172.21.15.32:0/4278257062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb8b00049c0 0x7fb8b0004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8ce494700 1 -- 172.21.15.32:0/4278257062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b0003230 con 0x7fb8b0003750 2020-12-05T17:37:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cdc93700 1 --2- 172.21.15.32:0/4278257062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8c805feb0 0x7fb8b00044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:37:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8ce494700 1 --2- 172.21.15.32:0/4278257062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8b0003750 0x7fb8b0003b30 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7fb8c81425e0 tx=0x7fb8c0007b60).ready entity=mon.0 client_cookie=ee6f4f64e1d168e8 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cd492700 1 -- 172.21.15.32:0/4278257062 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8c000c660 con 0x7fb8b0003750 2020-12-05T17:37:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cd492700 1 -- 172.21.15.32:0/4278257062 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c000c7c0 con 0x7fb8b0003750 2020-12-05T17:37:38.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cd492700 1 -- 172.21.15.32:0/4278257062 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8c000cb20 con 0x7fb8b0003750 2020-12-05T17:37:38.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cf496700 1 -- 172.21.15.32:0/4278257062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8b0003750 msgr2=0x7fb8b0003b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:38.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cf496700 1 --2- 172.21.15.32:0/4278257062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8b0003750 0x7fb8b0003b30 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7fb8c81425e0 tx=0x7fb8c0007b60).stop 2020-12-05T17:37:38.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.424+0000 7fb8cec95700 1 -- 172.21.15.32:0/4278257062 reap_dead start 2020-12-05T17:37:38.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8cf496700 1 -- 172.21.15.32:0/4278257062 shutdown_connections 2020-12-05T17:37:38.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8cf496700 1 -- 172.21.15.32:0/4278257062 >> 172.21.15.32:0/4278257062 conn(0x7fb8c80b8ee0 msgr2=0x7fb8c80b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:38.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8cf496700 1 -- 172.21.15.32:0/4278257062 shutdown_connections 2020-12-05T17:37:38.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8cf496700 1 -- 172.21.15.32:0/4278257062 wait complete. 2020-12-05T17:37:38.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8cf496700 1 Processor -- start 2020-12-05T17:37:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8cf496700 1 -- start start 2020-12-05T17:37:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8cf496700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c8154d20 0x7fb8c81444a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8cf496700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8c8143a80 0x7fb8c80b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8cf496700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb8c8151490 0x7fb8c8151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8cf496700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c8155680 con 0x7fb8c8154d20 2020-12-05T17:37:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8cf496700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c80b1d70 con 0x7fb8c8151490 2020-12-05T17:37:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8cf496700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c81553e0 con 0x7fb8c8143a80 2020-12-05T17:37:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8ce494700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c8154d20 0x7fb8c81444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8ce494700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c8154d20 0x7fb8c81444a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36392/0 (socket says 172.21.15.32:36392) 2020-12-05T17:37:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8ce494700 1 -- 172.21.15.32:0/3647334288 learned_addr learned my addr 172.21.15.32:0/3647334288 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8cdc93700 1 --2- 172.21.15.32:0/3647334288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8c8143a80 0x7fb8c80b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8cec95700 1 --2- 172.21.15.32:0/3647334288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb8c8151490 0x7fb8c8151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8ce494700 1 -- 172.21.15.32:0/3647334288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8c8143a80 msgr2=0x7fb8c80b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8ce494700 1 --2- 172.21.15.32:0/3647334288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8c8143a80 0x7fb8c80b21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8ce494700 1 -- 172.21.15.32:0/3647334288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb8c8151490 msgr2=0x7fb8c8151940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:38.443 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8ce494700 1 --2- 172.21.15.32:0/3647334288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb8c8151490 0x7fb8c8151940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:38.443 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8ce494700 1 -- 172.21.15.32:0/3647334288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c80b4a50 con 0x7fb8c8154d20 2020-12-05T17:37:38.443 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8cdc93700 1 --2- 172.21.15.32:0/3647334288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8c8143a80 0x7fb8c80b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:38.443 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8ce494700 1 --2- 172.21.15.32:0/3647334288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c8154d20 0x7fb8c81444a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7fb8c81425e0 tx=0x7fb8c0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:38.443 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8bb7fe700 1 -- 172.21.15.32:0/3647334288 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8c0001ea0 con 0x7fb8c8154d20 2020-12-05T17:37:38.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8bb7fe700 1 -- 172.21.15.32:0/3647334288 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c00127a0 con 0x7fb8c8154d20 2020-12-05T17:37:38.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8cf496700 1 -- 172.21.15.32:0/3647334288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c8155100 con 0x7fb8c8154d20 2020-12-05T17:37:38.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8bb7fe700 1 -- 172.21.15.32:0/3647334288 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8c0012b00 con 0x7fb8c8154d20 2020-12-05T17:37:38.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.428+0000 7fb8cf496700 1 -- 172.21.15.32:0/3647334288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c80b2d90 con 0x7fb8c8154d20 2020-12-05T17:37:38.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.432+0000 7fb8bb7fe700 1 -- 172.21.15.32:0/3647334288 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb8c0010070 con 0x7fb8c8154d20 2020-12-05T17:37:38.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.432+0000 7fb8cf496700 1 -- 172.21.15.32:0/3647334288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c8057050 con 0x7fb8c8154d20 2020-12-05T17:37:38.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.432+0000 7fb8bb7fe700 1 --2- 172.21.15.32:0/3647334288 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8a80553f0 0x7fb8a80578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:38.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.432+0000 7fb8bb7fe700 1 -- 172.21.15.32:0/3647334288 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb8c006e860 con 0x7fb8c8154d20 2020-12-05T17:37:38.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.432+0000 7fb8cdc93700 1 --2- 172.21.15.32:0/3647334288 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8a80553f0 0x7fb8a80578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:38.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.432+0000 7fb8cdc93700 1 --2- 172.21.15.32:0/3647334288 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8a80553f0 0x7fb8a80578b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fb8bc00d000 tx=0x7fb8bc00a040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:38.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.440+0000 7fb8bb7fe700 1 -- 172.21.15.32:0/3647334288 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb8c00449a0 con 0x7fb8c8154d20 2020-12-05T17:37:38.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:38.620+0000 7fb8cf496700 1 -- 172.21.15.32:0/3647334288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb8c80b2d90 con 0x7fb8c8154d20 2020-12-05T17:37:38.986 INFO:tasks.ceph.mon.a.smithi032.stderr:2020-12-05T17:37:38.984+0000 7fdef8136700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-12-05T17:37:39.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.000+0000 7fb8bb7fe700 1 -- 172.21.15.32:0/3647334288 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb8c80b2d90 con 0x7fb8c8154d20 2020-12-05T17:37:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.004+0000 7fb8cf496700 1 -- 172.21.15.32:0/3647334288 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8a80553f0 msgr2=0x7fb8a80578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.004+0000 7fb8cf496700 1 --2- 172.21.15.32:0/3647334288 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8a80553f0 0x7fb8a80578b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fb8bc00d000 tx=0x7fb8bc00a040).stop 2020-12-05T17:37:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.004+0000 7fb8cf496700 1 -- 172.21.15.32:0/3647334288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c8154d20 msgr2=0x7fb8c81444a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.004+0000 7fb8cf496700 1 --2- 172.21.15.32:0/3647334288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c8154d20 0x7fb8c81444a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7fb8c81425e0 tx=0x7fb8c0017000).stop 2020-12-05T17:37:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.004+0000 7fb8cf496700 1 -- 172.21.15.32:0/3647334288 shutdown_connections 2020-12-05T17:37:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.004+0000 7fb8cf496700 1 --2- 172.21.15.32:0/3647334288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb8c8151490 0x7fb8c8151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.004+0000 7fb8cf496700 1 --2- 172.21.15.32:0/3647334288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c8154d20 0x7fb8c81444a0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.004+0000 7fb8cf496700 1 --2- 172.21.15.32:0/3647334288 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8a80553f0 0x7fb8a80578b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.004+0000 7fb8cf496700 1 --2- 172.21.15.32:0/3647334288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8c8143a80 0x7fb8c80b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:39.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.004+0000 7fb8cf496700 1 -- 172.21.15.32:0/3647334288 >> 172.21.15.32:0/3647334288 conn(0x7fb8c80b8ee0 msgr2=0x7fb8c813f430 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:39.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.004+0000 7fb8cf496700 1 -- 172.21.15.32:0/3647334288 shutdown_connections 2020-12-05T17:37:39.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.004+0000 7fb8cf496700 1 -- 172.21.15.32:0/3647334288 wait complete. 2020-12-05T17:37:39.013 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-12-05T17:37:39.024 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:37:39.037 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:37:39.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.128+0000 7f08b70ef700 1 Processor -- start 2020-12-05T17:37:39.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.128+0000 7f08b70ef700 1 -- start start 2020-12-05T17:37:39.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.128+0000 7f08b70ef700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08b005fe00 0x7f08b0061910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:39.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.128+0000 7f08b70ef700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f08b00b00a0 0x7f08b014a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:39.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.128+0000 7f08b70ef700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b00601e0 con 0x7f08b00ac590 2020-12-05T17:37:39.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.128+0000 7f08b70ef700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b00ac1a0 con 0x7f08b00b00a0 2020-12-05T17:37:39.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.128+0000 7f08b70ef700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b0061fa0 con 0x7f08b005fe00 2020-12-05T17:37:39.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.128+0000 7f08affff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f08b00b00a0 0x7f08b014a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:39.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.128+0000 7f08b4e8b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08b005fe00 0x7f08b0061910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.128+0000 7f08affff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f08b00b00a0 0x7f08b014a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:56306/0 (socket says 172.21.15.32:56306) 2020-12-05T17:37:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.128+0000 7f08affff700 1 -- 172.21.15.32:0/2439136099 learned_addr learned my addr 172.21.15.32:0/2439136099 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.128+0000 7f08b4e8b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08b005fe00 0x7f08b0061910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:33202/0 (socket says 172.21.15.32:33202) 2020-12-05T17:37:39.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.128+0000 7f08b4e8b700 1 -- 172.21.15.32:0/2439136099 >> v2:172.21.15.61:3300/0 conn(0x7f08b00b00a0 msgr2=0x7f08b014a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:39.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.128+0000 7f08b4e8b700 1 --2- 172.21.15.32:0/2439136099 >> v2:172.21.15.61:3300/0 conn(0x7f08b00b00a0 0x7f08b014a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:39.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.128+0000 7f08b4e8b700 1 -- 172.21.15.32:0/2439136099 >> v1:172.21.15.61:6789/0 conn(0x7f08b00ac590 legacy=0x7f08b00ac9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:39.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.128+0000 7f08b4e8b700 1 -- 172.21.15.32:0/2439136099 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b0062240 con 0x7f08b005fe00 2020-12-05T17:37:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.132+0000 7f08b4e8b700 1 --2- 172.21.15.32:0/2439136099 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08b005fe00 0x7f08b0061910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f08a8002f30 tx=0x7f08a8006dc0).ready entity=mon.2 client_cookie=5ac4976ed652f6ca server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.132+0000 7f08af7fe700 1 -- 172.21.15.32:0/2439136099 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08a8009760 con 0x7f08b005fe00 2020-12-05T17:37:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.132+0000 7f08af7fe700 1 -- 172.21.15.32:0/2439136099 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a80098c0 con 0x7f08b005fe00 2020-12-05T17:37:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.132+0000 7f08af7fe700 1 -- 172.21.15.32:0/2439136099 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08a8009c20 con 0x7f08b005fe00 2020-12-05T17:37:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.132+0000 7f08b70ef700 1 -- 172.21.15.32:0/2439136099 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08b005fe00 msgr2=0x7f08b0061910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.132+0000 7f08b70ef700 1 --2- 172.21.15.32:0/2439136099 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08b005fe00 0x7f08b0061910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f08a8002f30 tx=0x7f08a8006dc0).stop 2020-12-05T17:37:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.132+0000 7f08b70ef700 1 -- 172.21.15.32:0/2439136099 shutdown_connections 2020-12-05T17:37:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.132+0000 7f08b70ef700 1 --2- 172.21.15.32:0/2439136099 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08b005fe00 0x7f08b0061910 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.132+0000 7f08b70ef700 1 --2- 172.21.15.32:0/2439136099 >> v2:172.21.15.61:3300/0 conn(0x7f08b00b00a0 0x7f08b014a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.132+0000 7f08b70ef700 1 -- 172.21.15.32:0/2439136099 >> 172.21.15.32:0/2439136099 conn(0x7f08b00b57c0 msgr2=0x7f08b005f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.132+0000 7f08b70ef700 1 -- 172.21.15.32:0/2439136099 shutdown_connections 2020-12-05T17:37:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.132+0000 7f08b70ef700 1 -- 172.21.15.32:0/2439136099 wait complete. 2020-12-05T17:37:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.132+0000 7f08b70ef700 1 Processor -- start 2020-12-05T17:37:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.132+0000 7f08b70ef700 1 -- start start 2020-12-05T17:37:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08b70ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08b005fe00 0x7f08b009e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08b70ef700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f08b00ac590 0x7f08b0062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08b70ef700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08b00b00a0 0x7f08b01422f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08b70ef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b0063170 con 0x7f08b005fe00 2020-12-05T17:37:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08b70ef700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b01436f0 con 0x7f08b00ac590 2020-12-05T17:37:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08b70ef700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b0143990 con 0x7f08b00b00a0 2020-12-05T17:37:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08b4e8b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08b005fe00 0x7f08b009e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08b4e8b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08b005fe00 0x7f08b009e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36418/0 (socket says 172.21.15.32:36418) 2020-12-05T17:37:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08b4e8b700 1 -- 172.21.15.32:0/1397524203 learned_addr learned my addr 172.21.15.32:0/1397524203 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08affff700 1 --2- 172.21.15.32:0/1397524203 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f08b00ac590 0x7f08b0062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08b568c700 1 --2- 172.21.15.32:0/1397524203 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08b00b00a0 0x7f08b01422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:39.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08b4e8b700 1 -- 172.21.15.32:0/1397524203 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08b00b00a0 msgr2=0x7f08b01422f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:39.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08b4e8b700 1 --2- 172.21.15.32:0/1397524203 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08b00b00a0 0x7f08b01422f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:39.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08b4e8b700 1 -- 172.21.15.32:0/1397524203 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f08b00ac590 msgr2=0x7f08b0062c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:39.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08b4e8b700 1 --2- 172.21.15.32:0/1397524203 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f08b00ac590 0x7f08b0062c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:39.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08b4e8b700 1 -- 172.21.15.32:0/1397524203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b0143c30 con 0x7f08b005fe00 2020-12-05T17:37:39.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08b4e8b700 1 --2- 172.21.15.32:0/1397524203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08b005fe00 0x7f08b009e460 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f08a8002f30 tx=0x7f08a8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:39.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08adffb700 1 -- 172.21.15.32:0/1397524203 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08a8011a20 con 0x7f08b005fe00 2020-12-05T17:37:39.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08adffb700 1 -- 172.21.15.32:0/1397524203 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a8011b80 con 0x7f08b005fe00 2020-12-05T17:37:39.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08adffb700 1 -- 172.21.15.32:0/1397524203 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08a801d440 con 0x7f08b005fe00 2020-12-05T17:37:39.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08b70ef700 1 -- 172.21.15.32:0/1397524203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b0066ac0 con 0x7f08b005fe00 2020-12-05T17:37:39.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08b70ef700 1 -- 172.21.15.32:0/1397524203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b01427e0 con 0x7f08b005fe00 2020-12-05T17:37:39.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08b70ef700 1 -- 172.21.15.32:0/1397524203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b0057050 con 0x7f08b005fe00 2020-12-05T17:37:39.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.136+0000 7f08adffb700 1 -- 172.21.15.32:0/1397524203 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f08a8024030 con 0x7f08b005fe00 2020-12-05T17:37:39.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.140+0000 7f08adffb700 1 --2- 172.21.15.32:0/1397524203 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0894055440 0x7f0894057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:39.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.140+0000 7f08adffb700 1 -- 172.21.15.32:0/1397524203 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f08a806ef90 con 0x7f08b005fe00 2020-12-05T17:37:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.140+0000 7f08affff700 1 --2- 172.21.15.32:0/1397524203 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0894055440 0x7f0894057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.140+0000 7f08affff700 1 --2- 172.21.15.32:0/1397524203 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0894055440 0x7f0894057900 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f0898008000 tx=0x7f0898008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.144+0000 7f08adffb700 1 -- 172.21.15.32:0/1397524203 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f08a80450e0 con 0x7f08b005fe00 2020-12-05T17:37:39.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.332+0000 7f08b70ef700 1 -- 172.21.15.32:0/1397524203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08b0069690 con 0x7f08b005fe00 2020-12-05T17:37:39.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.332+0000 7f08adffb700 1 -- 172.21.15.32:0/1397524203 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+1779 (secure 0 0 0) 0x7f08a8022020 con 0x7f08b005fe00 2020-12-05T17:37:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:39.333 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5411,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/3995477718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3995477718},{"type":"v1","addr":"172.21.15.61:6833","nonce":3995477718}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":28,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:37:01.403820+0000","modified":"2020-12-05T17:37:38.987841+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-12-05T17:37:39.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.332+0000 7f08b70ef700 1 -- 172.21.15.32:0/1397524203 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0894055440 msgr2=0x7f0894057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:39.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.332+0000 7f08b70ef700 1 --2- 172.21.15.32:0/1397524203 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0894055440 0x7f0894057900 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f0898008000 tx=0x7f0898008040).stop 2020-12-05T17:37:39.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.332+0000 7f08b70ef700 1 -- 172.21.15.32:0/1397524203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08b005fe00 msgr2=0x7f08b009e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:39.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.332+0000 7f08b70ef700 1 --2- 172.21.15.32:0/1397524203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08b005fe00 0x7f08b009e460 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f08a8002f30 tx=0x7f08a8015000).stop 2020-12-05T17:37:39.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.332+0000 7f08b70ef700 1 -- 172.21.15.32:0/1397524203 shutdown_connections 2020-12-05T17:37:39.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.332+0000 7f08b70ef700 1 --2- 172.21.15.32:0/1397524203 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0894055440 0x7f0894057900 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:39.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.332+0000 7f08b70ef700 1 --2- 172.21.15.32:0/1397524203 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08b00b00a0 0x7f08b01422f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:39.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.332+0000 7f08b70ef700 1 --2- 172.21.15.32:0/1397524203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08b005fe00 0x7f08b009e460 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:39.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.332+0000 7f08b70ef700 1 --2- 172.21.15.32:0/1397524203 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f08b00ac590 0x7f08b0062c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:39.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.332+0000 7f08b70ef700 1 -- 172.21.15.32:0/1397524203 >> 172.21.15.32:0/1397524203 conn(0x7f08b00b57c0 msgr2=0x7f08b00ad1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:39.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.332+0000 7f08b70ef700 1 -- 172.21.15.32:0/1397524203 shutdown_connections 2020-12-05T17:37:39.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.336+0000 7f08b70ef700 1 -- 172.21.15.32:0/1397524203 wait complete. 2020-12-05T17:37:39.339 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2020-12-05T17:37:39.354 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-12-05T17:37:39.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258f4ed700 1 Processor -- start 2020-12-05T17:37:39.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258f4ed700 1 -- start start 2020-12-05T17:37:39.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258f4ed700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25880af230 0x7f2588145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:39.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258f4ed700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2588146a90 con 0x7f25880ab710 2020-12-05T17:37:39.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258f4ed700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25880b23a0 con 0x7f25880b1fc0 2020-12-05T17:37:39.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258f4ed700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2588061e90 con 0x7f25880af230 2020-12-05T17:37:39.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258e4eb700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f25880b1fc0 0x7f2588061780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:51714/0 (socket says 172.21.15.32:51714) 2020-12-05T17:37:39.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258dcea700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f25880ab710 0x7f2588147400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39078/0 (socket says 172.21.15.32:39078) 2020-12-05T17:37:39.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258e4eb700 1 -- 172.21.15.32:0/3030441093 learned_addr learned my addr 172.21.15.32:0/3030441093 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:39.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258ecec700 1 --2- 172.21.15.32:0/3030441093 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25880af230 0x7f2588145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:39.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258d4e9700 1 -- 172.21.15.32:0/3030441093 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198819403 0 0) 0x7f2588146a90 con 0x7f25880ab710 2020-12-05T17:37:39.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258d4e9700 1 -- 172.21.15.32:0/3030441093 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2570003230 con 0x7f25880ab710 2020-12-05T17:37:39.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258d4e9700 1 -- 172.21.15.32:0/3030441093 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 412540227 0 0) 0x7f25880b23a0 con 0x7f25880b1fc0 2020-12-05T17:37:39.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258d4e9700 1 -- 172.21.15.32:0/3030441093 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2588146a90 con 0x7f25880b1fc0 2020-12-05T17:37:39.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258ecec700 1 -- 172.21.15.32:0/3030441093 >> v1:172.21.15.61:6789/0 conn(0x7f25880b1fc0 legacy=0x7f2588061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258ecec700 1 -- 172.21.15.32:0/3030441093 >> v1:172.21.15.32:6789/0 conn(0x7f25880ab710 legacy=0x7f2588147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258ecec700 1 -- 172.21.15.32:0/3030441093 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2588146cc0 con 0x7f25880af230 2020-12-05T17:37:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258ecec700 1 --2- 172.21.15.32:0/3030441093 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25880af230 0x7f2588145d10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f2580002f30 tx=0x7f2580006dc0).ready entity=mon.2 client_cookie=c03b14c7ccb96dd6 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:39.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258d4e9700 1 -- 172.21.15.32:0/3030441093 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2580009730 con 0x7f25880af230 2020-12-05T17:37:39.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258d4e9700 1 -- 172.21.15.32:0/3030441093 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2580009890 con 0x7f25880af230 2020-12-05T17:37:39.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258d4e9700 1 -- 172.21.15.32:0/3030441093 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2580009bf0 con 0x7f25880af230 2020-12-05T17:37:39.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258f4ed700 1 -- 172.21.15.32:0/3030441093 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25880af230 msgr2=0x7f2588145d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:39.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258f4ed700 1 --2- 172.21.15.32:0/3030441093 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25880af230 0x7f2588145d10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f2580002f30 tx=0x7f2580006dc0).stop 2020-12-05T17:37:39.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258f4ed700 1 -- 172.21.15.32:0/3030441093 shutdown_connections 2020-12-05T17:37:39.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258f4ed700 1 --2- 172.21.15.32:0/3030441093 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25880af230 0x7f2588145d10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:39.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.516+0000 7f258f4ed700 1 -- 172.21.15.32:0/3030441093 >> 172.21.15.32:0/3030441093 conn(0x7f25880b5530 msgr2=0x7f25880b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:39.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f258f4ed700 1 -- 172.21.15.32:0/3030441093 shutdown_connections 2020-12-05T17:37:39.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f258f4ed700 1 -- 172.21.15.32:0/3030441093 wait complete. 2020-12-05T17:37:39.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f258f4ed700 1 Processor -- start 2020-12-05T17:37:39.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f258f4ed700 1 -- start start 2020-12-05T17:37:39.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f258f4ed700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f25880ab710 0x7f25881496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:39.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f258f4ed700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25880af230 0x7f2588155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:39.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f258f4ed700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25880b1fc0 0x7f25880abb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:39.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f258f4ed700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2588149bb0 con 0x7f25880b1fc0 2020-12-05T17:37:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f258f4ed700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2588147fd0 con 0x7f25880ab710 2020-12-05T17:37:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f258f4ed700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2588156490 con 0x7f25880af230 2020-12-05T17:37:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f258ecec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25880b1fc0 0x7f25880abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f258ecec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25880b1fc0 0x7f25880abb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36436/0 (socket says 172.21.15.32:36436) 2020-12-05T17:37:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f258ecec700 1 -- 172.21.15.32:0/1856967199 learned_addr learned my addr 172.21.15.32:0/1856967199 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f258dcea700 1 --2- 172.21.15.32:0/1856967199 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25880af230 0x7f2588155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f258e4eb700 1 --2- 172.21.15.32:0/1856967199 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f25880ab710 0x7f25881496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f258ecec700 1 -- 172.21.15.32:0/1856967199 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25880af230 msgr2=0x7f2588155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f258ecec700 1 --2- 172.21.15.32:0/1856967199 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25880af230 0x7f2588155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f258ecec700 1 -- 172.21.15.32:0/1856967199 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f25880ab710 msgr2=0x7f25881496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:39.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f258ecec700 1 --2- 172.21.15.32:0/1856967199 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f25880ab710 0x7f25881496c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:39.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f258ecec700 1 -- 172.21.15.32:0/1856967199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25881561a0 con 0x7f25880b1fc0 2020-12-05T17:37:39.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f258ecec700 1 --2- 172.21.15.32:0/1856967199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25880b1fc0 0x7f25880abb50 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f2580002f30 tx=0x7f2580015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:39.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f25777fe700 1 -- 172.21.15.32:0/1856967199 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2580009b90 con 0x7f25880b1fc0 2020-12-05T17:37:39.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f25777fe700 1 -- 172.21.15.32:0/1856967199 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2580011b60 con 0x7f25880b1fc0 2020-12-05T17:37:39.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f258f4ed700 1 -- 172.21.15.32:0/1856967199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25880ad990 con 0x7f25880b1fc0 2020-12-05T17:37:39.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.520+0000 7f25777fe700 1 -- 172.21.15.32:0/1856967199 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f258001d450 con 0x7f25880b1fc0 2020-12-05T17:37:39.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.524+0000 7f258f4ed700 1 -- 172.21.15.32:0/1856967199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25880ae4e0 con 0x7f25880b1fc0 2020-12-05T17:37:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.524+0000 7f258f4ed700 1 -- 172.21.15.32:0/1856967199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25881494c0 con 0x7f25880b1fc0 2020-12-05T17:37:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.524+0000 7f25777fe700 1 -- 172.21.15.32:0/1856967199 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2580024030 con 0x7f25880b1fc0 2020-12-05T17:37:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.524+0000 7f25777fe700 1 --2- 172.21.15.32:0/1856967199 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2568055470 0x7f2568057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.524+0000 7f25777fe700 1 -- 172.21.15.32:0/1856967199 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f258006f0d0 con 0x7f25880b1fc0 2020-12-05T17:37:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.524+0000 7f258e4eb700 1 --2- 172.21.15.32:0/1856967199 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2568055470 0x7f2568057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.524+0000 7f258e4eb700 1 --2- 172.21.15.32:0/1856967199 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2568055470 0x7f2568057930 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f258400d000 tx=0x7f2584009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.528+0000 7f25777fe700 1 -- 172.21.15.32:0/1856967199 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2580045220 con 0x7f25880b1fc0 2020-12-05T17:37:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:39.716+0000 7f258f4ed700 1 -- 172.21.15.32:0/1856967199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2588057050 con 0x7f25880b1fc0 2020-12-05T17:37:40.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.004+0000 7f25777fe700 1 -- 172.21.15.32:0/1856967199 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2580022020 con 0x7f25880b1fc0 2020-12-05T17:37:40.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.008+0000 7f258f4ed700 1 -- 172.21.15.32:0/1856967199 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2568055470 msgr2=0x7f2568057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:40.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.008+0000 7f258f4ed700 1 --2- 172.21.15.32:0/1856967199 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2568055470 0x7f2568057930 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f258400d000 tx=0x7f2584009040).stop 2020-12-05T17:37:40.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.008+0000 7f258f4ed700 1 -- 172.21.15.32:0/1856967199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25880b1fc0 msgr2=0x7f25880abb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:40.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.008+0000 7f258f4ed700 1 --2- 172.21.15.32:0/1856967199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25880b1fc0 0x7f25880abb50 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f2580002f30 tx=0x7f2580015000).stop 2020-12-05T17:37:40.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.008+0000 7f258f4ed700 1 -- 172.21.15.32:0/1856967199 shutdown_connections 2020-12-05T17:37:40.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.008+0000 7f258f4ed700 1 --2- 172.21.15.32:0/1856967199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25880b1fc0 0x7f25880abb50 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:40.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.008+0000 7f258f4ed700 1 --2- 172.21.15.32:0/1856967199 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f25880ab710 0x7f25881496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:40.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.008+0000 7f258f4ed700 1 --2- 172.21.15.32:0/1856967199 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2568055470 0x7f2568057930 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.008+0000 7f258f4ed700 1 --2- 172.21.15.32:0/1856967199 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25880af230 0x7f2588155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.008+0000 7f258f4ed700 1 -- 172.21.15.32:0/1856967199 >> 172.21.15.32:0/1856967199 conn(0x7f25880b5530 msgr2=0x7f25881473b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.008+0000 7f258f4ed700 1 -- 172.21.15.32:0/1856967199 shutdown_connections 2020-12-05T17:37:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.008+0000 7f258f4ed700 1 -- 172.21.15.32:0/1856967199 wait complete. 2020-12-05T17:37:40.025 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-12-05T17:37:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f544beef700 1 Processor -- start 2020-12-05T17:37:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f544beef700 1 -- start start 2020-12-05T17:37:40.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f544beef700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54440b6690 0x7f54440b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:40.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f544beef700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f54440b5240 0x7f5444145f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:40.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f544beef700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54440b56f0 con 0x7f54440b5b20 2020-12-05T17:37:40.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f544beef700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54440b1920 con 0x7f54440b5240 2020-12-05T17:37:40.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f544beef700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54440b1bc0 con 0x7f54440b6690 2020-12-05T17:37:40.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f544948a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54440b6690 0x7f54440b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:40.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f544a48c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f54440b5240 0x7f5444145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:40.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f544948a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54440b6690 0x7f54440b5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:33234/0 (socket says 172.21.15.32:33234) 2020-12-05T17:37:40.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f544948a700 1 -- 172.21.15.32:0/184324760 learned_addr learned my addr 172.21.15.32:0/184324760 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:40.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f544948a700 1 -- 172.21.15.32:0/184324760 >> v2:172.21.15.61:3300/0 conn(0x7f54440b5240 msgr2=0x7f5444145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:40.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f544948a700 1 --2- 172.21.15.32:0/184324760 >> v2:172.21.15.61:3300/0 conn(0x7f54440b5240 0x7f5444145f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:40.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f544948a700 1 -- 172.21.15.32:0/184324760 >> v1:172.21.15.61:6789/0 conn(0x7f54440b5b20 legacy=0x7f54440b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:40.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f544948a700 1 -- 172.21.15.32:0/184324760 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5444146400 con 0x7f54440b6690 2020-12-05T17:37:40.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f544948a700 1 --2- 172.21.15.32:0/184324760 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54440b6690 0x7f54440b5f00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f5438002f30 tx=0x7f5438006e80).ready entity=mon.2 client_cookie=6fb0a93c2fac5a27 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f5448c89700 1 -- 172.21.15.32:0/184324760 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54380097f0 con 0x7f54440b6690 2020-12-05T17:37:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f5448c89700 1 -- 172.21.15.32:0/184324760 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5438009950 con 0x7f54440b6690 2020-12-05T17:37:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f5448c89700 1 -- 172.21.15.32:0/184324760 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5438009cb0 con 0x7f54440b6690 2020-12-05T17:37:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f544beef700 1 -- 172.21.15.32:0/184324760 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54440b6690 msgr2=0x7f54440b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f544beef700 1 --2- 172.21.15.32:0/184324760 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54440b6690 0x7f54440b5f00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f5438002f30 tx=0x7f5438006e80).stop 2020-12-05T17:37:40.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f544beef700 1 -- 172.21.15.32:0/184324760 shutdown_connections 2020-12-05T17:37:40.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f544beef700 1 --2- 172.21.15.32:0/184324760 >> v2:172.21.15.61:3300/0 conn(0x7f54440b5240 0x7f5444145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:40.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f544beef700 1 --2- 172.21.15.32:0/184324760 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54440b6690 0x7f54440b5f00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:40.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.144+0000 7f544beef700 1 -- 172.21.15.32:0/184324760 >> 172.21.15.32:0/184324760 conn(0x7f544405f3b0 msgr2=0x7f544405f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:40.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f544beef700 1 -- 172.21.15.32:0/184324760 shutdown_connections 2020-12-05T17:37:40.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f544beef700 1 -- 172.21.15.32:0/184324760 wait complete. 2020-12-05T17:37:40.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f544beef700 1 Processor -- start 2020-12-05T17:37:40.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f544beef700 1 -- start start 2020-12-05T17:37:40.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f544beef700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54440b5240 0x7f5444141f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:40.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f544beef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54440b5b20 0x7f5444142440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:40.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f544beef700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f54440b6690 0x7f54440a3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:40.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f544beef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54441526b0 con 0x7f54440b5b20 2020-12-05T17:37:40.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f544beef700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5444152920 con 0x7f54440b6690 2020-12-05T17:37:40.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f544beef700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5444152bc0 con 0x7f54440b5240 2020-12-05T17:37:40.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f544948a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54440b5b20 0x7f5444142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:40.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f544948a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54440b5b20 0x7f5444142440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36450/0 (socket says 172.21.15.32:36450) 2020-12-05T17:37:40.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f544948a700 1 -- 172.21.15.32:0/2340430035 learned_addr learned my addr 172.21.15.32:0/2340430035 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:40.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f5449c8b700 1 --2- 172.21.15.32:0/2340430035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54440b5240 0x7f5444141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:40.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f544a48c700 1 --2- 172.21.15.32:0/2340430035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f54440b6690 0x7f54440a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:40.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f544948a700 1 -- 172.21.15.32:0/2340430035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54440b5240 msgr2=0x7f5444141f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:40.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f544948a700 1 --2- 172.21.15.32:0/2340430035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54440b5240 0x7f5444141f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:40.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f544948a700 1 -- 172.21.15.32:0/2340430035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f54440b6690 msgr2=0x7f54440a3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:40.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f544948a700 1 --2- 172.21.15.32:0/2340430035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f54440b6690 0x7f54440a3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:40.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f544948a700 1 -- 172.21.15.32:0/2340430035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5444063e60 con 0x7f54440b5b20 2020-12-05T17:37:40.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f5449c8b700 1 --2- 172.21.15.32:0/2340430035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54440b5240 0x7f5444141f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:40.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f544948a700 1 --2- 172.21.15.32:0/2340430035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54440b5b20 0x7f5444142440 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f5438002f30 tx=0x7f5438015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:40.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f54377fe700 1 -- 172.21.15.32:0/2340430035 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5438011ce0 con 0x7f54440b5b20 2020-12-05T17:37:40.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f54377fe700 1 -- 172.21.15.32:0/2340430035 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5438011e40 con 0x7f54440b5b20 2020-12-05T17:37:40.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f54377fe700 1 -- 172.21.15.32:0/2340430035 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f543801e650 con 0x7f54440b5b20 2020-12-05T17:37:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f544beef700 1 -- 172.21.15.32:0/2340430035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5444064300 con 0x7f54440b5b20 2020-12-05T17:37:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.148+0000 7f544beef700 1 -- 172.21.15.32:0/2340430035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5444064ef0 con 0x7f54440b5b20 2020-12-05T17:37:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.152+0000 7f544beef700 1 -- 172.21.15.32:0/2340430035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5444063ff0 con 0x7f54440b5b20 2020-12-05T17:37:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.152+0000 7f54377fe700 1 -- 172.21.15.32:0/2340430035 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5438025030 con 0x7f54440b5b20 2020-12-05T17:37:40.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.152+0000 7f54377fe700 1 --2- 172.21.15.32:0/2340430035 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5420055440 0x7f5420057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:40.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.152+0000 7f54377fe700 1 -- 172.21.15.32:0/2340430035 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f543806f450 con 0x7f54440b5b20 2020-12-05T17:37:40.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.152+0000 7f5449c8b700 1 --2- 172.21.15.32:0/2340430035 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5420055440 0x7f5420057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:40.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.152+0000 7f5449c8b700 1 --2- 172.21.15.32:0/2340430035 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5420055440 0x7f5420057900 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f544000b000 tx=0x7f5440009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:40.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.156+0000 7f54377fe700 1 -- 172.21.15.32:0/2340430035 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f54380455a0 con 0x7f54440b5b20 2020-12-05T17:37:40.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:40.304+0000 7f544beef700 1 -- 172.21.15.32:0/2340430035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5444057050 con 0x7f54440b5b20 2020-12-05T17:37:41.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.004+0000 7f54377fe700 1 -- 172.21.15.32:0/2340430035 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v64) v1 ==== 185+0+0 (secure 0 0 0) 0x7f5438048bc0 con 0x7f54440b5b20 2020-12-05T17:37:41.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.008+0000 7f544beef700 1 -- 172.21.15.32:0/2340430035 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5420055440 msgr2=0x7f5420057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:41.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.008+0000 7f544beef700 1 --2- 172.21.15.32:0/2340430035 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5420055440 0x7f5420057900 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f544000b000 tx=0x7f5440009040).stop 2020-12-05T17:37:41.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.008+0000 7f544beef700 1 -- 172.21.15.32:0/2340430035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54440b5b20 msgr2=0x7f5444142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:41.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.008+0000 7f544beef700 1 --2- 172.21.15.32:0/2340430035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54440b5b20 0x7f5444142440 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f5438002f30 tx=0x7f5438015000).stop 2020-12-05T17:37:41.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.008+0000 7f544beef700 1 -- 172.21.15.32:0/2340430035 shutdown_connections 2020-12-05T17:37:41.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.008+0000 7f544beef700 1 --2- 172.21.15.32:0/2340430035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f54440b6690 0x7f54440a3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:41.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.008+0000 7f544beef700 1 --2- 172.21.15.32:0/2340430035 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5420055440 0x7f5420057900 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:41.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.008+0000 7f544beef700 1 --2- 172.21.15.32:0/2340430035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54440b5240 0x7f5444141f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:41.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.008+0000 7f544beef700 1 --2- 172.21.15.32:0/2340430035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54440b5b20 0x7f5444142440 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:41.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.008+0000 7f544beef700 1 -- 172.21.15.32:0/2340430035 >> 172.21.15.32:0/2340430035 conn(0x7f544405f3b0 msgr2=0x7f54440b8560 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:41.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.008+0000 7f544beef700 1 -- 172.21.15.32:0/2340430035 shutdown_connections 2020-12-05T17:37:41.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.008+0000 7f544beef700 1 -- 172.21.15.32:0/2340430035 wait complete. 2020-12-05T17:37:41.014 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2020-12-05T17:37:41.029 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-12-05T17:37:41.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.152+0000 7f03d9e62700 1 Processor -- start 2020-12-05T17:37:41.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.152+0000 7f03d9e62700 1 -- start start 2020-12-05T17:37:41.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03d9e62700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f03d40b2100 0x7f03d40af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:41.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03d9e62700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f03d413ed60 0x7f03d4138820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:41.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03d9e62700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d40b24e0 con 0x7f03d40ad250 2020-12-05T17:37:41.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03d9e62700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d413fd10 con 0x7f03d40b2100 2020-12-05T17:37:41.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03d9e62700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d40b0780 con 0x7f03d413ed60 2020-12-05T17:37:41.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03d8e60700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f03d40b2100 0x7f03d40af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:41.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03d8e60700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f03d40b2100 0x7f03d40af360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:56348/0 (socket says 172.21.15.32:56348) 2020-12-05T17:37:41.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03d8e60700 1 -- 172.21.15.32:0/2820601220 learned_addr learned my addr 172.21.15.32:0/2820601220 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:41.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03cbfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f03d413ed60 0x7f03d4138820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:41.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03d9661700 1 --1- 172.21.15.32:0/2820601220 >> v1:172.21.15.61:6789/0 conn(0x7f03d40ad250 0x7f03d4142600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:51748/0 (socket says 172.21.15.32:51748) 2020-12-05T17:37:41.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03d8e60700 1 -- 172.21.15.32:0/2820601220 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f03d413ed60 msgr2=0x7f03d4138820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:41.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03d8e60700 1 --2- 172.21.15.32:0/2820601220 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f03d413ed60 0x7f03d4138820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:41.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03d8e60700 1 -- 172.21.15.32:0/2820601220 >> v1:172.21.15.61:6789/0 conn(0x7f03d40ad250 legacy=0x7f03d4142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:41.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03d8e60700 1 -- 172.21.15.32:0/2820601220 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d40b0240 con 0x7f03d40b2100 2020-12-05T17:37:41.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03cb7fe700 1 -- 172.21.15.32:0/2820601220 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2599753501 0 0) 0x7f03d40b24e0 con 0x7f03d40ad250 2020-12-05T17:37:41.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03d8e60700 1 --2- 172.21.15.32:0/2820601220 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03d40b2100 0x7f03d40af360 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f03d00032b0 tx=0x7f03d0009270).ready entity=mon.1 client_cookie=d05d30906acd617d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:41.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03cb7fe700 1 -- 172.21.15.32:0/2820601220 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03d0009e60 con 0x7f03d40b2100 2020-12-05T17:37:41.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03cb7fe700 1 -- 172.21.15.32:0/2820601220 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d000bc00 con 0x7f03d40b2100 2020-12-05T17:37:41.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03cb7fe700 1 -- 172.21.15.32:0/2820601220 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03d000bd70 con 0x7f03d40b2100 2020-12-05T17:37:41.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03d9e62700 1 -- 172.21.15.32:0/2820601220 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03d40b2100 msgr2=0x7f03d40af360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:41.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03d9e62700 1 --2- 172.21.15.32:0/2820601220 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03d40b2100 0x7f03d40af360 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f03d00032b0 tx=0x7f03d0009270).stop 2020-12-05T17:37:41.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03d9e62700 1 -- 172.21.15.32:0/2820601220 shutdown_connections 2020-12-05T17:37:41.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03d9e62700 1 --2- 172.21.15.32:0/2820601220 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03d40b2100 0x7f03d40af360 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:41.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03d9e62700 1 --2- 172.21.15.32:0/2820601220 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f03d413ed60 0x7f03d4138820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:41.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03d9e62700 1 -- 172.21.15.32:0/2820601220 >> 172.21.15.32:0/2820601220 conn(0x7f03d40620b0 msgr2=0x7f03d4062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:41.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03d9e62700 1 -- 172.21.15.32:0/2820601220 shutdown_connections 2020-12-05T17:37:41.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.156+0000 7f03d9e62700 1 -- 172.21.15.32:0/2820601220 wait complete. 2020-12-05T17:37:41.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.160+0000 7f03d9e62700 1 Processor -- start 2020-12-05T17:37:41.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.160+0000 7f03d9e62700 1 -- start start 2020-12-05T17:37:41.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.160+0000 7f03d9e62700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f03d40ad250 0x7f03d413d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:41.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.160+0000 7f03d9e62700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03d40b2100 0x7f03d412bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:41.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.160+0000 7f03d9e62700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03d412e360 0x7f03d412c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:41.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.160+0000 7f03d9e62700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d40b38b0 con 0x7f03d40b2100 2020-12-05T17:37:41.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.160+0000 7f03d9e62700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d41418d0 con 0x7f03d412e360 2020-12-05T17:37:41.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.160+0000 7f03d9e62700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d4141b70 con 0x7f03d40ad250 2020-12-05T17:37:41.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.160+0000 7f03cbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03d40b2100 0x7f03d412bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:41.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.160+0000 7f03cbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03d40b2100 0x7f03d412bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36464/0 (socket says 172.21.15.32:36464) 2020-12-05T17:37:41.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.160+0000 7f03cbfff700 1 -- 172.21.15.32:0/2909814226 learned_addr learned my addr 172.21.15.32:0/2909814226 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:41.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.160+0000 7f03d8e60700 1 --2- 172.21.15.32:0/2909814226 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f03d40ad250 0x7f03d413d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:41.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.160+0000 7f03d9661700 1 --2- 172.21.15.32:0/2909814226 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03d412e360 0x7f03d412c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:41.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.164+0000 7f03cbfff700 1 -- 172.21.15.32:0/2909814226 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f03d40ad250 msgr2=0x7f03d413d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:41.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.164+0000 7f03cbfff700 1 --2- 172.21.15.32:0/2909814226 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f03d40ad250 0x7f03d413d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:41.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.164+0000 7f03cbfff700 1 -- 172.21.15.32:0/2909814226 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03d412e360 msgr2=0x7f03d412c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:41.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.164+0000 7f03cbfff700 1 --2- 172.21.15.32:0/2909814226 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03d412e360 0x7f03d412c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:41.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.164+0000 7f03cbfff700 1 -- 172.21.15.32:0/2909814226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d413d7f0 con 0x7f03d40b2100 2020-12-05T17:37:41.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.164+0000 7f03d8e60700 1 --2- 172.21.15.32:0/2909814226 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f03d40ad250 0x7f03d413d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:37:41.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.164+0000 7f03cbfff700 1 --2- 172.21.15.32:0/2909814226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03d40b2100 0x7f03d412bdc0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f03c400c000 tx=0x7f03c4006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:41.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.164+0000 7f03c9ffb700 1 -- 172.21.15.32:0/2909814226 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03c400ca10 con 0x7f03d40b2100 2020-12-05T17:37:41.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.164+0000 7f03c9ffb700 1 -- 172.21.15.32:0/2909814226 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c400be90 con 0x7f03d40b2100 2020-12-05T17:37:41.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.164+0000 7f03d9e62700 1 -- 172.21.15.32:0/2909814226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d413ca10 con 0x7f03d40b2100 2020-12-05T17:37:41.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.164+0000 7f03c9ffb700 1 -- 172.21.15.32:0/2909814226 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03c400cd60 con 0x7f03d40b2100 2020-12-05T17:37:41.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.164+0000 7f03d9e62700 1 -- 172.21.15.32:0/2909814226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d4141220 con 0x7f03d40b2100 2020-12-05T17:37:41.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.164+0000 7f03d9e62700 1 -- 172.21.15.32:0/2909814226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d413dbb0 con 0x7f03d40b2100 2020-12-05T17:37:41.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.164+0000 7f03c9ffb700 1 -- 172.21.15.32:0/2909814226 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f03c401c460 con 0x7f03d40b2100 2020-12-05T17:37:41.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.164+0000 7f03c9ffb700 1 --2- 172.21.15.32:0/2909814226 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f03b4055440 0x7f03b4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:41.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.164+0000 7f03c9ffb700 1 -- 172.21.15.32:0/2909814226 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f03c406f240 con 0x7f03d40b2100 2020-12-05T17:37:41.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.164+0000 7f03d8e60700 1 --2- 172.21.15.32:0/2909814226 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f03b4055440 0x7f03b4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:41.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.168+0000 7f03d8e60700 1 --2- 172.21.15.32:0/2909814226 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f03b4055440 0x7f03b4057900 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f03d001a000 tx=0x7f03d0016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:41.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.172+0000 7f03c9ffb700 1 -- 172.21.15.32:0/2909814226 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f03c40453d0 con 0x7f03d40b2100 2020-12-05T17:37:41.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:41.320+0000 7f03d9e62700 1 -- 172.21.15.32:0/2909814226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f03d413d980 con 0x7f03d40b2100 2020-12-05T17:37:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.012+0000 7f03c9ffb700 1 -- 172.21.15.32:0/2909814226 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v65) v1 ==== 173+0+0 (secure 0 0 0) 0x7f03c406d0d0 con 0x7f03d40b2100 2020-12-05T17:37:42.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.016+0000 7f03d9e62700 1 -- 172.21.15.32:0/2909814226 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f03b4055440 msgr2=0x7f03b4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.016+0000 7f03d9e62700 1 --2- 172.21.15.32:0/2909814226 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f03b4055440 0x7f03b4057900 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f03d001a000 tx=0x7f03d0016000).stop 2020-12-05T17:37:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.016+0000 7f03d9e62700 1 -- 172.21.15.32:0/2909814226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03d40b2100 msgr2=0x7f03d412bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.016+0000 7f03d9e62700 1 --2- 172.21.15.32:0/2909814226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03d40b2100 0x7f03d412bdc0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f03c400c000 tx=0x7f03c4006dc0).stop 2020-12-05T17:37:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.016+0000 7f03d9e62700 1 -- 172.21.15.32:0/2909814226 shutdown_connections 2020-12-05T17:37:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.016+0000 7f03d9e62700 1 --2- 172.21.15.32:0/2909814226 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03d412e360 0x7f03d412c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.016+0000 7f03d9e62700 1 --2- 172.21.15.32:0/2909814226 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f03b4055440 0x7f03b4057900 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.016+0000 7f03d9e62700 1 --2- 172.21.15.32:0/2909814226 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f03d40ad250 0x7f03d413d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.016+0000 7f03d9e62700 1 --2- 172.21.15.32:0/2909814226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03d40b2100 0x7f03d412bdc0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.016+0000 7f03d9e62700 1 -- 172.21.15.32:0/2909814226 >> 172.21.15.32:0/2909814226 conn(0x7f03d40620b0 msgr2=0x7f03d40b52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.016+0000 7f03d9e62700 1 -- 172.21.15.32:0/2909814226 shutdown_connections 2020-12-05T17:37:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.016+0000 7f03d9e62700 1 -- 172.21.15.32:0/2909814226 wait complete. 2020-12-05T17:37:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2020-12-05T17:37:42.037 INFO:tasks.ceph.mds.a:Restarting daemon 2020-12-05T17:37:42.037 INFO:tasks.ceph.mds.a:Stopping old one... 2020-12-05T17:37:42.038 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-12-05T17:37:42.038 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:37:42.038 INFO:tasks.ceph.mds.b:Restarting daemon 2020-12-05T17:37:42.038 INFO:tasks.ceph.mds.b:Stopping old one... 2020-12-05T17:37:42.039 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-12-05T17:37:42.039 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:37:42.049 INFO:tasks.ceph.mds.b:Stopped 2020-12-05T17:37:42.050 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-12-05T17:37:42.051 INFO:tasks.ceph.mds.a:Stopped 2020-12-05T17:37:42.051 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-12-05T17:37:42.092 INFO:tasks.ceph.mds.b:Started 2020-12-05T17:37:42.092 DEBUG:teuthology.parallel:result is None 2020-12-05T17:37:42.094 INFO:tasks.ceph.mds.a:Started 2020-12-05T17:37:42.095 DEBUG:teuthology.parallel:result is None 2020-12-05T17:37:42.095 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-12-05T17:37:42.165 INFO:tasks.ceph.mds.b.smithi061.stdout:starting mds.b at 2020-12-05T17:37:42.171 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:37:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdeb23700 1 Processor -- start 2020-12-05T17:37:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdeb23700 1 -- start start 2020-12-05T17:37:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdeb23700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cd80b52f0 0x7f5cd80b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdeb23700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5cd80b1870 0x7f5cd8138810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdeb23700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f5cd8138d00 0x7f5cd80ae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdeb23700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd80ad4f0 con 0x7f5cd80b1870 2020-12-05T17:37:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdeb23700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd81420f0 con 0x7f5cd8138d00 2020-12-05T17:37:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdeb23700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd80b0380 con 0x7f5cd80b52f0 2020-12-05T17:37:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdd320700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5cd80b1870 0x7f5cd8138810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdd320700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5cd80b1870 0x7f5cd8138810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36486/0 (socket says 172.21.15.32:36486) 2020-12-05T17:37:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdd320700 1 -- 172.21.15.32:0/3656399252 learned_addr learned my addr 172.21.15.32:0/3656399252 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cddb21700 1 --2- 172.21.15.32:0/3656399252 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cd80b52f0 0x7f5cd80b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cde322700 1 --2- 172.21.15.32:0/3656399252 >> v2:172.21.15.61:3300/0 conn(0x7f5cd8138d00 0x7f5cd80ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdd320700 1 -- 172.21.15.32:0/3656399252 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cd80b52f0 msgr2=0x7f5cd80b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdd320700 1 --2- 172.21.15.32:0/3656399252 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cd80b52f0 0x7f5cd80b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdd320700 1 -- 172.21.15.32:0/3656399252 >> v2:172.21.15.61:3300/0 conn(0x7f5cd8138d00 msgr2=0x7f5cd80ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdd320700 1 --2- 172.21.15.32:0/3656399252 >> v2:172.21.15.61:3300/0 conn(0x7f5cd8138d00 0x7f5cd80ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdd320700 1 -- 172.21.15.32:0/3656399252 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd80b0620 con 0x7f5cd80b1870 2020-12-05T17:37:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdd320700 1 --2- 172.21.15.32:0/3656399252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cd80b1870 0x7f5cd8138810 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f5cd80adc80 tx=0x7f5ccc006e80).ready entity=mon.0 client_cookie=eb079e72d64c6976 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdcb1f700 1 -- 172.21.15.32:0/3656399252 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ccc009600 con 0x7f5cd80b1870 2020-12-05T17:37:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdcb1f700 1 -- 172.21.15.32:0/3656399252 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ccc009760 con 0x7f5cd80b1870 2020-12-05T17:37:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdcb1f700 1 -- 172.21.15.32:0/3656399252 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ccc009ac0 con 0x7f5cd80b1870 2020-12-05T17:37:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdeb23700 1 -- 172.21.15.32:0/3656399252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cd80b1870 msgr2=0x7f5cd8138810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdeb23700 1 --2- 172.21.15.32:0/3656399252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cd80b1870 0x7f5cd8138810 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f5cd80adc80 tx=0x7f5ccc006e80).stop 2020-12-05T17:37:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdeb23700 1 -- 172.21.15.32:0/3656399252 shutdown_connections 2020-12-05T17:37:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdeb23700 1 --2- 172.21.15.32:0/3656399252 >> v2:172.21.15.61:3300/0 conn(0x7f5cd8138d00 0x7f5cd80ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:42.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdeb23700 1 --2- 172.21.15.32:0/3656399252 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cd80b52f0 0x7f5cd80b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:42.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdeb23700 1 --2- 172.21.15.32:0/3656399252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cd80b1870 0x7f5cd8138810 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:42.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdeb23700 1 -- 172.21.15.32:0/3656399252 >> 172.21.15.32:0/3656399252 conn(0x7f5cd805bbb0 msgr2=0x7f5cd8062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:42.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdeb23700 1 -- 172.21.15.32:0/3656399252 shutdown_connections 2020-12-05T17:37:42.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.208+0000 7f5cdeb23700 1 -- 172.21.15.32:0/3656399252 wait complete. 2020-12-05T17:37:42.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cdeb23700 1 Processor -- start 2020-12-05T17:37:42.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cdeb23700 1 -- start start 2020-12-05T17:37:42.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cdeb23700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cd80b1870 0x7f5cd813aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:42.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cdeb23700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cd80b52f0 0x7f5cd8140330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:42.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cdeb23700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cd8138d00 0x7f5cd8065130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:42.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cdeb23700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd8066580 con 0x7f5cd80b52f0 2020-12-05T17:37:42.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cdeb23700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd80667b0 con 0x7f5cd80b1870 2020-12-05T17:37:42.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cdeb23700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd8066a50 con 0x7f5cd8138d00 2020-12-05T17:37:42.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cdd320700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cd80b52f0 0x7f5cd8140330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:42.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cdd320700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cd80b52f0 0x7f5cd8140330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36492/0 (socket says 172.21.15.32:36492) 2020-12-05T17:37:42.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cdd320700 1 -- 172.21.15.32:0/1174799534 learned_addr learned my addr 172.21.15.32:0/1174799534 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:42.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cddb21700 1 --2- 172.21.15.32:0/1174799534 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cd80b1870 0x7f5cd813aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:42.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cde322700 1 --2- 172.21.15.32:0/1174799534 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cd8138d00 0x7f5cd8065130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:42.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cdd320700 1 -- 172.21.15.32:0/1174799534 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cd8138d00 msgr2=0x7f5cd8065130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:42.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cdd320700 1 --2- 172.21.15.32:0/1174799534 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cd8138d00 0x7f5cd8065130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:42.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cdd320700 1 -- 172.21.15.32:0/1174799534 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cd80b1870 msgr2=0x7f5cd813aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:42.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cdd320700 1 --2- 172.21.15.32:0/1174799534 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cd80b1870 0x7f5cd813aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:42.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cdd320700 1 -- 172.21.15.32:0/1174799534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd8151420 con 0x7f5cd80b52f0 2020-12-05T17:37:42.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cddb21700 1 --2- 172.21.15.32:0/1174799534 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cd80b1870 0x7f5cd813aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:42.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cdd320700 1 --2- 172.21.15.32:0/1174799534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cd80b52f0 0x7f5cd8140330 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f5cd80adc80 tx=0x7f5ccc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:42.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cd6ffd700 1 -- 172.21.15.32:0/1174799534 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ccc011ca0 con 0x7f5cd80b52f0 2020-12-05T17:37:42.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cd6ffd700 1 -- 172.21.15.32:0/1174799534 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ccc011e00 con 0x7f5cd80b52f0 2020-12-05T17:37:42.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cdeb23700 1 -- 172.21.15.32:0/1174799534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd8140a80 con 0x7f5cd80b52f0 2020-12-05T17:37:42.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cd6ffd700 1 -- 172.21.15.32:0/1174799534 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ccc01e650 con 0x7f5cd80b52f0 2020-12-05T17:37:42.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cdeb23700 1 -- 172.21.15.32:0/1174799534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd8139b80 con 0x7f5cd80b52f0 2020-12-05T17:37:42.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.212+0000 7f5cdeb23700 1 -- 172.21.15.32:0/1174799534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd8066c50 con 0x7f5cd80b52f0 2020-12-05T17:37:42.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.216+0000 7f5cd6ffd700 1 -- 172.21.15.32:0/1174799534 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5ccc025030 con 0x7f5cd80b52f0 2020-12-05T17:37:42.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.216+0000 7f5cd6ffd700 1 --2- 172.21.15.32:0/1174799534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5cb80554e0 0x7f5cb80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:42.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.216+0000 7f5cd6ffd700 1 -- 172.21.15.32:0/1174799534 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f5ccc06f0d0 con 0x7f5cd80b52f0 2020-12-05T17:37:42.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.216+0000 7f5cddb21700 1 --2- 172.21.15.32:0/1174799534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5cb80554e0 0x7f5cb80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:42.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.216+0000 7f5cddb21700 1 --2- 172.21.15.32:0/1174799534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5cb80554e0 0x7f5cb80579a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f5cc800b000 tx=0x7f5cc8008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:42.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.220+0000 7f5cd6ffd700 1 -- 172.21.15.32:0/1174799534 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5ccc045550 con 0x7f5cd80b52f0 2020-12-05T17:37:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:42.376+0000 7f5cdeb23700 1 -- 172.21.15.32:0/1174799534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f5cd8057050 con 0x7f5cd80b52f0 2020-12-05T17:37:43.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.183+0000 7f5cd6ffd700 1 -- 172.21.15.32:0/1174799534 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v67) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5ccc048b70 con 0x7f5cd80b52f0 2020-12-05T17:37:43.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.187+0000 7f5cdeb23700 1 -- 172.21.15.32:0/1174799534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5cb80554e0 msgr2=0x7f5cb80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:43.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.187+0000 7f5cdeb23700 1 --2- 172.21.15.32:0/1174799534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5cb80554e0 0x7f5cb80579a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f5cc800b000 tx=0x7f5cc8008040).stop 2020-12-05T17:37:43.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.187+0000 7f5cdeb23700 1 -- 172.21.15.32:0/1174799534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cd80b52f0 msgr2=0x7f5cd8140330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:43.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.187+0000 7f5cdeb23700 1 --2- 172.21.15.32:0/1174799534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cd80b52f0 0x7f5cd8140330 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f5cd80adc80 tx=0x7f5ccc015000).stop 2020-12-05T17:37:43.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.191+0000 7f5cdeb23700 1 -- 172.21.15.32:0/1174799534 shutdown_connections 2020-12-05T17:37:43.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.191+0000 7f5cdeb23700 1 --2- 172.21.15.32:0/1174799534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5cb80554e0 0x7f5cb80579a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.191+0000 7f5cdeb23700 1 --2- 172.21.15.32:0/1174799534 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cd8138d00 0x7f5cd8065130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.191+0000 7f5cdeb23700 1 --2- 172.21.15.32:0/1174799534 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cd80b1870 0x7f5cd813aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.191+0000 7f5cdeb23700 1 --2- 172.21.15.32:0/1174799534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cd80b52f0 0x7f5cd8140330 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.191+0000 7f5cdeb23700 1 -- 172.21.15.32:0/1174799534 >> 172.21.15.32:0/1174799534 conn(0x7f5cd805bbb0 msgr2=0x7f5cd8067150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:43.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.191+0000 7f5cdeb23700 1 -- 172.21.15.32:0/1174799534 shutdown_connections 2020-12-05T17:37:43.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.191+0000 7f5cdeb23700 1 -- 172.21.15.32:0/1174799534 wait complete. 2020-12-05T17:37:43.195 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2020-12-05T17:37:43.210 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2020-12-05T17:37:43.211 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-12-05T17:37:43.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.323+0000 7fa9da877700 1 Processor -- start 2020-12-05T17:37:43.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.323+0000 7fa9da877700 1 -- start start 2020-12-05T17:37:43.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.323+0000 7fa9da877700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa9d405f6f0 0x7fa9d40b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:43.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.323+0000 7fa9da877700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9d40afea0 0x7fa9d41409c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:43.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.323+0000 7fa9da877700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d405fad0 con 0x7fa9d40af920 2020-12-05T17:37:43.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.323+0000 7fa9da877700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d40b2cd0 con 0x7fa9d405f6f0 2020-12-05T17:37:43.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.323+0000 7fa9da877700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d40b3560 con 0x7fa9d40afea0 2020-12-05T17:37:43.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.323+0000 7fa9da076700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9d40afea0 0x7fa9d41409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:43.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.323+0000 7fa9d9875700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa9d405f6f0 0x7fa9d40b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:43.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.323+0000 7fa9da076700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9d40afea0 0x7fa9d41409c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:33290/0 (socket says 172.21.15.32:33290) 2020-12-05T17:37:43.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.323+0000 7fa9da076700 1 -- 172.21.15.32:0/1914828695 learned_addr learned my addr 172.21.15.32:0/1914828695 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:43.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.323+0000 7fa9da076700 1 -- 172.21.15.32:0/1914828695 >> v2:172.21.15.61:3300/0 conn(0x7fa9d405f6f0 msgr2=0x7fa9d40b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:43.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.323+0000 7fa9da076700 1 --2- 172.21.15.32:0/1914828695 >> v2:172.21.15.61:3300/0 conn(0x7fa9d405f6f0 0x7fa9d40b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.323+0000 7fa9da076700 1 -- 172.21.15.32:0/1914828695 >> v1:172.21.15.61:6789/0 conn(0x7fa9d40af920 legacy=0x7fa9d4063040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:43.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.323+0000 7fa9da076700 1 -- 172.21.15.32:0/1914828695 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d4144650 con 0x7fa9d40afea0 2020-12-05T17:37:43.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.323+0000 7fa9da076700 1 --2- 172.21.15.32:0/1914828695 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9d40afea0 0x7fa9d41409c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fa9cc002f30 tx=0x7fa9cc006e80).ready entity=mon.2 client_cookie=facd9a91358af5f5 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:43.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.327+0000 7fa9d8873700 1 -- 172.21.15.32:0/1914828695 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9cc0097f0 con 0x7fa9d40afea0 2020-12-05T17:37:43.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.327+0000 7fa9d8873700 1 -- 172.21.15.32:0/1914828695 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc009950 con 0x7fa9d40afea0 2020-12-05T17:37:43.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.327+0000 7fa9d8873700 1 -- 172.21.15.32:0/1914828695 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9cc009cb0 con 0x7fa9d40afea0 2020-12-05T17:37:43.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.327+0000 7fa9da877700 1 -- 172.21.15.32:0/1914828695 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9d40afea0 msgr2=0x7fa9d41409c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:43.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.327+0000 7fa9da877700 1 --2- 172.21.15.32:0/1914828695 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9d40afea0 0x7fa9d41409c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fa9cc002f30 tx=0x7fa9cc006e80).stop 2020-12-05T17:37:43.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.327+0000 7fa9da877700 1 -- 172.21.15.32:0/1914828695 shutdown_connections 2020-12-05T17:37:43.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.327+0000 7fa9da877700 1 --2- 172.21.15.32:0/1914828695 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9d40afea0 0x7fa9d41409c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.327+0000 7fa9da877700 1 --2- 172.21.15.32:0/1914828695 >> v2:172.21.15.61:3300/0 conn(0x7fa9d405f6f0 0x7fa9d40b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.327+0000 7fa9da877700 1 -- 172.21.15.32:0/1914828695 >> 172.21.15.32:0/1914828695 conn(0x7fa9d40604c0 msgr2=0x7fa9d405c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:43.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.327+0000 7fa9da877700 1 -- 172.21.15.32:0/1914828695 shutdown_connections 2020-12-05T17:37:43.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.327+0000 7fa9da877700 1 -- 172.21.15.32:0/1914828695 wait complete. 2020-12-05T17:37:43.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.327+0000 7fa9da877700 1 Processor -- start 2020-12-05T17:37:43.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.327+0000 7fa9da877700 1 -- start start 2020-12-05T17:37:43.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.331+0000 7fa9da877700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9d405f6f0 0x7fa9d4152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:43.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.331+0000 7fa9da877700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa9d40af920 0x7fa9d41471e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:43.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.331+0000 7fa9da877700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9d40afea0 0x7fa9d410ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:43.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.331+0000 7fa9da877700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d4066210 con 0x7fa9d40afea0 2020-12-05T17:37:43.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.331+0000 7fa9da877700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d4065380 con 0x7fa9d40af920 2020-12-05T17:37:43.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.331+0000 7fa9da877700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d4065f70 con 0x7fa9d405f6f0 2020-12-05T17:37:43.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.331+0000 7fa9da076700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9d40afea0 0x7fa9d410ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:43.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.331+0000 7fa9da076700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9d40afea0 0x7fa9d410ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36506/0 (socket says 172.21.15.32:36506) 2020-12-05T17:37:43.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.331+0000 7fa9da076700 1 -- 172.21.15.32:0/1598871920 learned_addr learned my addr 172.21.15.32:0/1598871920 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:43.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.331+0000 7fa9d9074700 1 --2- 172.21.15.32:0/1598871920 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa9d40af920 0x7fa9d41471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:43.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.331+0000 7fa9d9875700 1 --2- 172.21.15.32:0/1598871920 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9d405f6f0 0x7fa9d4152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:43.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.331+0000 7fa9da076700 1 -- 172.21.15.32:0/1598871920 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9d405f6f0 msgr2=0x7fa9d4152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:43.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.331+0000 7fa9da076700 1 --2- 172.21.15.32:0/1598871920 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9d405f6f0 0x7fa9d4152710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.331+0000 7fa9da076700 1 -- 172.21.15.32:0/1598871920 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa9d40af920 msgr2=0x7fa9d41471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:43.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.331+0000 7fa9da076700 1 --2- 172.21.15.32:0/1598871920 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa9d40af920 0x7fa9d41471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.331+0000 7fa9da076700 1 -- 172.21.15.32:0/1598871920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d40650b0 con 0x7fa9d40afea0 2020-12-05T17:37:43.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.331+0000 7fa9da076700 1 --2- 172.21.15.32:0/1598871920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9d40afea0 0x7fa9d410ee20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7fa9cc002f30 tx=0x7fa9cc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:43.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.331+0000 7fa9c2ffd700 1 -- 172.21.15.32:0/1598871920 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9cc011b80 con 0x7fa9d40afea0 2020-12-05T17:37:43.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.331+0000 7fa9da877700 1 -- 172.21.15.32:0/1598871920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d40641d0 con 0x7fa9d40afea0 2020-12-05T17:37:43.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.331+0000 7fa9c2ffd700 1 -- 172.21.15.32:0/1598871920 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc011ce0 con 0x7fa9d40afea0 2020-12-05T17:37:43.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.331+0000 7fa9c2ffd700 1 -- 172.21.15.32:0/1598871920 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9cc01e650 con 0x7fa9d40afea0 2020-12-05T17:37:43.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.331+0000 7fa9da877700 1 -- 172.21.15.32:0/1598871920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d4064cb0 con 0x7fa9d40afea0 2020-12-05T17:37:43.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.335+0000 7fa9da877700 1 -- 172.21.15.32:0/1598871920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d4057050 con 0x7fa9d40afea0 2020-12-05T17:37:43.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.335+0000 7fa9c2ffd700 1 -- 172.21.15.32:0/1598871920 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa9cc025030 con 0x7fa9d40afea0 2020-12-05T17:37:43.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.335+0000 7fa9c2ffd700 1 --2- 172.21.15.32:0/1598871920 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa9b4055440 0x7fa9b4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:43.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.335+0000 7fa9c2ffd700 1 -- 172.21.15.32:0/1598871920 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fa9cc06f070 con 0x7fa9d40afea0 2020-12-05T17:37:43.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.335+0000 7fa9d9875700 1 --2- 172.21.15.32:0/1598871920 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa9b4055440 0x7fa9b4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:43.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.335+0000 7fa9d9875700 1 --2- 172.21.15.32:0/1598871920 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa9b4055440 0x7fa9b4057900 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fa9d000b000 tx=0x7fa9d00091a0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:43.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.343+0000 7fa9c2ffd700 1 -- 172.21.15.32:0/1598871920 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa9cc0454b0 con 0x7fa9d40afea0 2020-12-05T17:37:43.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.539+0000 7fa9da877700 1 -- 172.21.15.32:0/1598871920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa9d4069690 con 0x7fa9d40afea0 2020-12-05T17:37:43.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.539+0000 7fa9c2ffd700 1 -- 172.21.15.32:0/1598871920 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+4242 (secure 0 0 0) 0x7fa9cc023020 con 0x7fa9d40afea0 2020-12-05T17:37:43.542 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:43.543 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:37:43.543 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2020-12-05T17:37:43.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:43.543 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2020-12-05T17:37:43.543 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfpb/aExAAdMKW4yIgC2zhkzyW6l8xug==", 2020-12-05T17:37:43.544 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:37:43.544 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:37:43.544 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:37:43.544 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:37:43.544 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:37:43.545 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:43.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:43.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:43.545 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2020-12-05T17:37:43.545 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfBKTYGRAAaPkezwmTiAvkUfTuDGDNZA==", 2020-12-05T17:37:43.546 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:37:43.546 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:37:43.546 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:37:43.546 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:37:43.546 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:37:43.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:43.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:43.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:43.547 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2020-12-05T17:37:43.547 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHw8tflo5EKhAAwV9D64fAoY3uvXCm+Aksag==", 2020-12-05T17:37:43.548 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:37:43.548 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:37:43.548 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:37:43.548 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:37:43.548 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:43.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:43.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:43.549 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2020-12-05T17:37:43.549 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDKw8tfTjEiERAAkW6mkgApoXwyF49kPjZBGg==", 2020-12-05T17:37:43.549 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:37:43.549 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:37:43.550 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:37:43.550 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:37:43.550 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:43.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:43.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:43.551 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2020-12-05T17:37:43.551 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDMw8tfLEzBMxAAiJ2NifsOLEkt1e3vmhwSTA==", 2020-12-05T17:37:43.551 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:37:43.551 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:37:43.551 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:37:43.552 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:37:43.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:43.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:43.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:43.552 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2020-12-05T17:37:43.552 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPw8tfV+vnGxAAQHvXJRW5X9Vhe/eYJBJ/uQ==", 2020-12-05T17:37:43.553 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:37:43.553 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:37:43.553 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:37:43.553 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:37:43.553 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:43.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:43.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:43.554 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2020-12-05T17:37:43.554 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUw8tfN8wMJxAApc5LY9HGDeIV/o0eil/g4A==", 2020-12-05T17:37:43.554 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:37:43.555 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:37:43.555 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:37:43.555 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:37:43.555 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:43.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:43.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:43.556 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2020-12-05T17:37:43.556 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXw8tfvZ5dDxAAtAfftExkljlBvn7ZSoD/Og==", 2020-12-05T17:37:43.556 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:37:43.556 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:37:43.557 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:37:43.557 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:37:43.557 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:43.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:43.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:43.558 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2020-12-05T17:37:43.558 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDZw8tfhXFzMhAAZXNwNGfKv360Tq4N/DX5Mw==", 2020-12-05T17:37:43.558 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:37:43.558 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:37:43.558 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:37:43.559 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:37:43.559 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:43.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:43.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:43.559 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2020-12-05T17:37:43.560 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDcw8tfROfSGhAA/Y2Y1b4k5zQju0MM7LASeQ==", 2020-12-05T17:37:43.560 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:37:43.560 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:37:43.560 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:37:43.560 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:37:43.561 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:43.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:43.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:43.561 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2020-12-05T17:37:43.561 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfWDp/HxAAmvLPj7C3WDgj0OC2o7lBJQ==", 2020-12-05T17:37:43.562 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:37:43.562 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:37:43.562 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:37:43.562 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:37:43.562 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:43.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:43.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:43.563 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2020-12-05T17:37:43.563 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfugnmIRAAW6+/QGMLkHw+Z97RHpV4kw==", 2020-12-05T17:37:43.563 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:37:43.563 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:37:43.564 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:37:43.564 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:37:43.564 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:43.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:43.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:43.565 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2020-12-05T17:37:43.565 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfojyqJBAAyta+dSy2yGnC1kKe3YOAYw==", 2020-12-05T17:37:43.565 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:37:43.565 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:37:43.565 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:37:43.565 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:37:43.566 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:43.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:43.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:43.566 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2020-12-05T17:37:43.566 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDFw8tfES0PLBAAA1cUV44snAUDF2TyAmXImw==", 2020-12-05T17:37:43.567 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:37:43.567 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:37:43.567 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:37:43.567 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:37:43.567 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:37:43.568 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:43.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:43.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:43.568 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2020-12-05T17:37:43.568 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfDr8mERAAS7bpAQXgW78W8GDq1v1zaA==", 2020-12-05T17:37:43.569 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:37:43.569 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:37:43.569 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:37:43.569 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:37:43.569 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:43.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:43.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:43.570 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2020-12-05T17:37:43.570 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tf3HmcDhAATNKHhH9zgasedaNEzgG0OQ==", 2020-12-05T17:37:43.570 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:37:43.571 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:37:43.571 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:37:43.571 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:37:43.571 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:43.571 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:43.571 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:37:43.572 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:37:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.539+0000 7fa9da877700 1 -- 172.21.15.32:0/1598871920 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa9b4055440 msgr2=0x7fa9b4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.539+0000 7fa9da877700 1 --2- 172.21.15.32:0/1598871920 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa9b4055440 0x7fa9b4057900 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fa9d000b000 tx=0x7fa9d00091a0).stop 2020-12-05T17:37:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.539+0000 7fa9da877700 1 -- 172.21.15.32:0/1598871920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9d40afea0 msgr2=0x7fa9d410ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.539+0000 7fa9da877700 1 --2- 172.21.15.32:0/1598871920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9d40afea0 0x7fa9d410ee20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7fa9cc002f30 tx=0x7fa9cc015000).stop 2020-12-05T17:37:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.539+0000 7fa9da877700 1 -- 172.21.15.32:0/1598871920 shutdown_connections 2020-12-05T17:37:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.539+0000 7fa9da877700 1 --2- 172.21.15.32:0/1598871920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9d40afea0 0x7fa9d410ee20 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.539+0000 7fa9da877700 1 --2- 172.21.15.32:0/1598871920 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa9b4055440 0x7fa9b4057900 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.539+0000 7fa9da877700 1 --2- 172.21.15.32:0/1598871920 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9d405f6f0 0x7fa9d4152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.539+0000 7fa9da877700 1 --2- 172.21.15.32:0/1598871920 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa9d40af920 0x7fa9d41471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.539+0000 7fa9da877700 1 -- 172.21.15.32:0/1598871920 >> 172.21.15.32:0/1598871920 conn(0x7fa9d40604c0 msgr2=0x7fa9d4143c30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.543+0000 7fa9da877700 1 -- 172.21.15.32:0/1598871920 shutdown_connections 2020-12-05T17:37:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.543+0000 7fa9da877700 1 -- 172.21.15.32:0/1598871920 wait complete. 2020-12-05T17:37:43.576 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:37:43.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.687+0000 7fd2b9cd5700 1 Processor -- start 2020-12-05T17:37:43.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.687+0000 7fd2b9cd5700 1 -- start start 2020-12-05T17:37:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.687+0000 7fd2b9cd5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2b405f3b0 0x7fd2b405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.687+0000 7fd2b9cd5700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd2b4062400 0x7fd2b40b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.687+0000 7fd2b9cd5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b4142450 con 0x7fd2b405feb0 2020-12-05T17:37:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.687+0000 7fd2b9cd5700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b40b42f0 con 0x7fd2b4062400 2020-12-05T17:37:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.687+0000 7fd2b9cd5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b4061780 con 0x7fd2b405f3b0 2020-12-05T17:37:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.687+0000 7fd2b94d4700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd2b405feb0 0x7fd2b40b3be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39162/0 (socket says 172.21.15.32:39162) 2020-12-05T17:37:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.687+0000 7fd2b94d4700 1 -- 172.21.15.32:0/2616586999 learned_addr learned my addr 172.21.15.32:0/2616586999 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.687+0000 7fd2b8cd3700 1 --2- 172.21.15.32:0/2616586999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2b405f3b0 0x7fd2b405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.687+0000 7fd2b3fff700 1 --2- 172.21.15.32:0/2616586999 >> v2:172.21.15.61:3300/0 conn(0x7fd2b4062400 0x7fd2b40b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:43.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.687+0000 7fd2b37fe700 1 -- 172.21.15.32:0/2616586999 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764074783 0 0) 0x7fd2b4142450 con 0x7fd2b405feb0 2020-12-05T17:37:43.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.687+0000 7fd2b37fe700 1 -- 172.21.15.32:0/2616586999 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2a0003230 con 0x7fd2b405feb0 2020-12-05T17:37:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.687+0000 7fd2b8cd3700 1 -- 172.21.15.32:0/2616586999 >> v2:172.21.15.61:3300/0 conn(0x7fd2b4062400 msgr2=0x7fd2b40b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.687+0000 7fd2b8cd3700 1 --2- 172.21.15.32:0/2616586999 >> v2:172.21.15.61:3300/0 conn(0x7fd2b4062400 0x7fd2b40b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.687+0000 7fd2b8cd3700 1 -- 172.21.15.32:0/2616586999 >> v1:172.21.15.32:6789/0 conn(0x7fd2b405feb0 legacy=0x7fd2b40b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.687+0000 7fd2b8cd3700 1 -- 172.21.15.32:0/2616586999 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b4061a20 con 0x7fd2b405f3b0 2020-12-05T17:37:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.687+0000 7fd2b8cd3700 1 --2- 172.21.15.32:0/2616586999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2b405f3b0 0x7fd2b405f790 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fd2a40032b0 tx=0x7fd2a40091a0).ready entity=mon.2 client_cookie=c13791bc49a6ab06 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:43.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.687+0000 7fd2b37fe700 1 -- 172.21.15.32:0/2616586999 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2a400b940 con 0x7fd2b405f3b0 2020-12-05T17:37:43.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.687+0000 7fd2b37fe700 1 -- 172.21.15.32:0/2616586999 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a400baa0 con 0x7fd2b405f3b0 2020-12-05T17:37:43.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.687+0000 7fd2b37fe700 1 -- 172.21.15.32:0/2616586999 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2a400bc10 con 0x7fd2b405f3b0 2020-12-05T17:37:43.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.691+0000 7fd2b9cd5700 1 -- 172.21.15.32:0/2616586999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2b405f3b0 msgr2=0x7fd2b405f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:43.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.691+0000 7fd2b9cd5700 1 --2- 172.21.15.32:0/2616586999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2b405f3b0 0x7fd2b405f790 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fd2a40032b0 tx=0x7fd2a40091a0).stop 2020-12-05T17:37:43.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.691+0000 7fd2b9cd5700 1 -- 172.21.15.32:0/2616586999 shutdown_connections 2020-12-05T17:37:43.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.691+0000 7fd2b9cd5700 1 --2- 172.21.15.32:0/2616586999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2b405f3b0 0x7fd2b405f790 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.691+0000 7fd2b9cd5700 1 --2- 172.21.15.32:0/2616586999 >> v2:172.21.15.61:3300/0 conn(0x7fd2b4062400 0x7fd2b40b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.691+0000 7fd2b9cd5700 1 -- 172.21.15.32:0/2616586999 >> 172.21.15.32:0/2616586999 conn(0x7fd2b40b8ee0 msgr2=0x7fd2b40b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:43.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.691+0000 7fd2b9cd5700 1 -- 172.21.15.32:0/2616586999 shutdown_connections 2020-12-05T17:37:43.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.691+0000 7fd2b9cd5700 1 -- 172.21.15.32:0/2616586999 wait complete. 2020-12-05T17:37:43.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.691+0000 7fd2b9cd5700 1 Processor -- start 2020-12-05T17:37:43.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.691+0000 7fd2b9cd5700 1 -- start start 2020-12-05T17:37:43.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b9cd5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2b405f3b0 0x7fd2b413f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:43.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b9cd5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2b405feb0 0x7fd2b4066490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:43.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b9cd5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd2b4062400 0x7fd2b4066980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:43.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b9cd5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b40b1f70 con 0x7fd2b405f3b0 2020-12-05T17:37:43.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b9cd5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b4066e70 con 0x7fd2b4062400 2020-12-05T17:37:43.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b9cd5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b40b1a30 con 0x7fd2b405feb0 2020-12-05T17:37:43.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b8cd3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2b405f3b0 0x7fd2b413f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:43.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b8cd3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2b405f3b0 0x7fd2b413f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36516/0 (socket says 172.21.15.32:36516) 2020-12-05T17:37:43.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b8cd3700 1 -- 172.21.15.32:0/1830204911 learned_addr learned my addr 172.21.15.32:0/1830204911 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b94d4700 1 --2- 172.21.15.32:0/1830204911 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd2b4062400 0x7fd2b4066980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b3fff700 1 --2- 172.21.15.32:0/1830204911 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2b405feb0 0x7fd2b4066490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b8cd3700 1 -- 172.21.15.32:0/1830204911 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2b405feb0 msgr2=0x7fd2b4066490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b8cd3700 1 --2- 172.21.15.32:0/1830204911 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2b405feb0 0x7fd2b4066490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b8cd3700 1 -- 172.21.15.32:0/1830204911 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd2b4062400 msgr2=0x7fd2b4066980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b8cd3700 1 --2- 172.21.15.32:0/1830204911 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd2b4062400 0x7fd2b4066980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b8cd3700 1 -- 172.21.15.32:0/1830204911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b40b1cd0 con 0x7fd2b405f3b0 2020-12-05T17:37:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b94d4700 1 --2- 172.21.15.32:0/1830204911 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd2b4062400 0x7fd2b4066980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b3fff700 1 --2- 172.21.15.32:0/1830204911 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2b405feb0 0x7fd2b4066490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b8cd3700 1 --2- 172.21.15.32:0/1830204911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2b405f3b0 0x7fd2b413f960 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7fd2a40032b0 tx=0x7fd2a4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b1ffb700 1 -- 172.21.15.32:0/1830204911 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2a400be80 con 0x7fd2b405f3b0 2020-12-05T17:37:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b1ffb700 1 -- 172.21.15.32:0/1830204911 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a401f3f0 con 0x7fd2b405f3b0 2020-12-05T17:37:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b9cd5700 1 -- 172.21.15.32:0/1830204911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b40b16e0 con 0x7fd2b405f3b0 2020-12-05T17:37:43.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b1ffb700 1 -- 172.21.15.32:0/1830204911 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2a401f750 con 0x7fd2b405f3b0 2020-12-05T17:37:43.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.695+0000 7fd2b9cd5700 1 -- 172.21.15.32:0/1830204911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b41446b0 con 0x7fd2b405f3b0 2020-12-05T17:37:43.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.699+0000 7fd2b9cd5700 1 -- 172.21.15.32:0/1830204911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b4154af0 con 0x7fd2b405f3b0 2020-12-05T17:37:43.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.699+0000 7fd2b1ffb700 1 -- 172.21.15.32:0/1830204911 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd2a400f070 con 0x7fd2b405f3b0 2020-12-05T17:37:43.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.699+0000 7fd2b1ffb700 1 --2- 172.21.15.32:0/1830204911 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd294055440 0x7fd294057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:43.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.699+0000 7fd2b3fff700 1 --2- 172.21.15.32:0/1830204911 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd294055440 0x7fd294057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:43.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.699+0000 7fd2b1ffb700 1 -- 172.21.15.32:0/1830204911 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fd2a406e7b0 con 0x7fd2b405f3b0 2020-12-05T17:37:43.707 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.699+0000 7fd2b3fff700 1 --2- 172.21.15.32:0/1830204911 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd294055440 0x7fd294057900 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fd2a800b000 tx=0x7fd2a8008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:43.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.703+0000 7fd2b1ffb700 1 -- 172.21.15.32:0/1830204911 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd2a4044c90 con 0x7fd2b405f3b0 2020-12-05T17:37:43.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.851+0000 7fd2b9cd5700 1 -- 172.21.15.32:0/1830204911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2b4057050 con 0x7fd2b405f3b0 2020-12-05T17:37:43.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.851+0000 7fd2b1ffb700 1 -- 172.21.15.32:0/1830204911 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+13415 (secure 0 0 0) 0x7fd2a4024090 con 0x7fd2b405f3b0 2020-12-05T17:37:43.855 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:43.855 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":67,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:37:43.181651+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:37:43.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.855+0000 7fd2b9cd5700 1 -- 172.21.15.32:0/1830204911 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd294055440 msgr2=0x7fd294057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:43.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.855+0000 7fd2b9cd5700 1 --2- 172.21.15.32:0/1830204911 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd294055440 0x7fd294057900 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fd2a800b000 tx=0x7fd2a8008040).stop 2020-12-05T17:37:43.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.855+0000 7fd2b9cd5700 1 -- 172.21.15.32:0/1830204911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2b405f3b0 msgr2=0x7fd2b413f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:43.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.855+0000 7fd2b9cd5700 1 --2- 172.21.15.32:0/1830204911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2b405f3b0 0x7fd2b413f960 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7fd2a40032b0 tx=0x7fd2a4016000).stop 2020-12-05T17:37:43.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.855+0000 7fd2b9cd5700 1 -- 172.21.15.32:0/1830204911 shutdown_connections 2020-12-05T17:37:43.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.855+0000 7fd2b9cd5700 1 --2- 172.21.15.32:0/1830204911 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd2b4062400 0x7fd2b4066980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.855+0000 7fd2b9cd5700 1 --2- 172.21.15.32:0/1830204911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2b405f3b0 0x7fd2b413f960 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.855+0000 7fd2b9cd5700 1 --2- 172.21.15.32:0/1830204911 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd294055440 0x7fd294057900 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.855+0000 7fd2b9cd5700 1 --2- 172.21.15.32:0/1830204911 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2b405feb0 0x7fd2b4066490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.855+0000 7fd2b9cd5700 1 -- 172.21.15.32:0/1830204911 >> 172.21.15.32:0/1830204911 conn(0x7fd2b40b8ee0 msgr2=0x7fd2b4142620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:43.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.855+0000 7fd2b9cd5700 1 -- 172.21.15.32:0/1830204911 shutdown_connections 2020-12-05T17:37:43.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.855+0000 7fd2b9cd5700 1 -- 172.21.15.32:0/1830204911 wait complete. 2020-12-05T17:37:43.869 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-12-05T17:37:43.870 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:37:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.983+0000 7f3a49f05700 1 Processor -- start 2020-12-05T17:37:43.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.983+0000 7f3a49f05700 1 -- start start 2020-12-05T17:37:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a49f05700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f3a440b5b20 0x7f3a440b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a49f05700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3a440b5240 0x7f3a44145f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a49f05700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a440b1920 con 0x7f3a440b6690 2020-12-05T17:37:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a49f05700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a440b1bc0 con 0x7f3a440b5240 2020-12-05T17:37:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a49f05700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a440b5680 con 0x7f3a440b5b20 2020-12-05T17:37:43.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a43fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3a440b5240 0x7f3a44145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:43.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a42ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3a440b6690 0x7f3a440b2060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39176/0 (socket says 172.21.15.32:39176) 2020-12-05T17:37:43.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a42ffd700 1 -- 172.21.15.32:0/3550730432 learned_addr learned my addr 172.21.15.32:0/3550730432 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:43.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a437fe700 1 --2- 172.21.15.32:0/3550730432 >> v2:172.21.15.61:3300/0 conn(0x7f3a440b5b20 0x7f3a440b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:43.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a43fff700 1 -- 172.21.15.32:0/3550730432 >> v2:172.21.15.61:3300/0 conn(0x7f3a440b5b20 msgr2=0x7f3a440b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:43.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a43fff700 1 --2- 172.21.15.32:0/3550730432 >> v2:172.21.15.61:3300/0 conn(0x7f3a440b5b20 0x7f3a440b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a43fff700 1 -- 172.21.15.32:0/3550730432 >> v1:172.21.15.32:6789/0 conn(0x7f3a440b6690 legacy=0x7f3a440b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:43.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a43fff700 1 -- 172.21.15.32:0/3550730432 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a44146400 con 0x7f3a440b5240 2020-12-05T17:37:43.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a437fe700 1 --2- 172.21.15.32:0/3550730432 >> v2:172.21.15.61:3300/0 conn(0x7f3a440b5b20 0x7f3a440b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:43.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a427fc700 1 -- 172.21.15.32:0/3550730432 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 354360108 0 0) 0x7f3a440b1920 con 0x7f3a440b6690 2020-12-05T17:37:43.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a43fff700 1 --2- 172.21.15.32:0/3550730432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a440b5240 0x7f3a44145f10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f3a3c002f30 tx=0x7f3a3c006dc0).ready entity=mon.0 client_cookie=32c487413e0654cf server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:43.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a427fc700 1 -- 172.21.15.32:0/3550730432 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a3c0097f0 con 0x7f3a440b5240 2020-12-05T17:37:43.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a427fc700 1 -- 172.21.15.32:0/3550730432 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3c009950 con 0x7f3a440b5240 2020-12-05T17:37:43.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a427fc700 1 -- 172.21.15.32:0/3550730432 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a3c009cb0 con 0x7f3a440b5240 2020-12-05T17:37:43.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a49f05700 1 -- 172.21.15.32:0/3550730432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a440b5240 msgr2=0x7f3a44145f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:43.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a49f05700 1 --2- 172.21.15.32:0/3550730432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a440b5240 0x7f3a44145f10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f3a3c002f30 tx=0x7f3a3c006dc0).stop 2020-12-05T17:37:43.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a49f05700 1 -- 172.21.15.32:0/3550730432 shutdown_connections 2020-12-05T17:37:43.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a49f05700 1 --2- 172.21.15.32:0/3550730432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a440b5240 0x7f3a44145f10 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a49f05700 1 --2- 172.21.15.32:0/3550730432 >> v2:172.21.15.61:3300/0 conn(0x7f3a440b5b20 0x7f3a440b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:43.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.987+0000 7f3a49f05700 1 -- 172.21.15.32:0/3550730432 >> 172.21.15.32:0/3550730432 conn(0x7f3a4405f3b0 msgr2=0x7f3a4405f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:43.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.991+0000 7f3a49f05700 1 -- 172.21.15.32:0/3550730432 shutdown_connections 2020-12-05T17:37:43.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.991+0000 7f3a49f05700 1 -- 172.21.15.32:0/3550730432 wait complete. 2020-12-05T17:37:43.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.991+0000 7f3a49f05700 1 Processor -- start 2020-12-05T17:37:43.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.991+0000 7f3a49f05700 1 -- start start 2020-12-05T17:37:43.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.991+0000 7f3a49f05700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a440b5240 0x7f3a440a34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:43.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.991+0000 7f3a49f05700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3a440b5b20 0x7f3a441517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:43.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.991+0000 7f3a49f05700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a440b6690 0x7f3a44151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:44.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.991+0000 7f3a49f05700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a440667a0 con 0x7f3a440b6690 2020-12-05T17:37:44.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.991+0000 7f3a49f05700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a44066e70 con 0x7f3a440b5b20 2020-12-05T17:37:44.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.991+0000 7f3a49f05700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a440a5d70 con 0x7f3a440b5240 2020-12-05T17:37:44.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.991+0000 7f3a43fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a440b6690 0x7f3a44151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:44.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.991+0000 7f3a43fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a440b6690 0x7f3a44151ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36534/0 (socket says 172.21.15.32:36534) 2020-12-05T17:37:44.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.991+0000 7f3a43fff700 1 -- 172.21.15.32:0/3838435846 learned_addr learned my addr 172.21.15.32:0/3838435846 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:44.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.991+0000 7f3a42ffd700 1 --2- 172.21.15.32:0/3838435846 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3a440b5b20 0x7f3a441517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:44.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.991+0000 7f3a437fe700 1 --2- 172.21.15.32:0/3838435846 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a440b5240 0x7f3a440a34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:44.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.995+0000 7f3a43fff700 1 -- 172.21.15.32:0/3838435846 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a440b5240 msgr2=0x7f3a440a34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:44.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.995+0000 7f3a43fff700 1 --2- 172.21.15.32:0/3838435846 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a440b5240 0x7f3a440a34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.995+0000 7f3a43fff700 1 -- 172.21.15.32:0/3838435846 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3a440b5b20 msgr2=0x7f3a441517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:44.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.995+0000 7f3a43fff700 1 --2- 172.21.15.32:0/3838435846 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3a440b5b20 0x7f3a441517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.995+0000 7f3a43fff700 1 -- 172.21.15.32:0/3838435846 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a440a6010 con 0x7f3a440b6690 2020-12-05T17:37:44.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.995+0000 7f3a43fff700 1 --2- 172.21.15.32:0/3838435846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a440b6690 0x7f3a44151ce0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f3a3c002f30 tx=0x7f3a3c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:44.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.995+0000 7f3a40ff9700 1 -- 172.21.15.32:0/3838435846 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a3c011b80 con 0x7f3a440b6690 2020-12-05T17:37:44.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.995+0000 7f3a40ff9700 1 -- 172.21.15.32:0/3838435846 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3c011ce0 con 0x7f3a440b6690 2020-12-05T17:37:44.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.995+0000 7f3a49f05700 1 -- 172.21.15.32:0/3838435846 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a440669d0 con 0x7f3a440b6690 2020-12-05T17:37:44.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.995+0000 7f3a40ff9700 1 -- 172.21.15.32:0/3838435846 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a3c01d640 con 0x7f3a440b6690 2020-12-05T17:37:44.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.995+0000 7f3a49f05700 1 -- 172.21.15.32:0/3838435846 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a440a3c80 con 0x7f3a440b6690 2020-12-05T17:37:44.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.995+0000 7f3a49f05700 1 -- 172.21.15.32:0/3838435846 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a440a2f10 con 0x7f3a440b6690 2020-12-05T17:37:44.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.995+0000 7f3a40ff9700 1 -- 172.21.15.32:0/3838435846 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3a3c024030 con 0x7f3a440b6690 2020-12-05T17:37:44.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.999+0000 7f3a40ff9700 1 --2- 172.21.15.32:0/3838435846 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3a240553b0 0x7f3a24057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:44.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.999+0000 7f3a40ff9700 1 -- 172.21.15.32:0/3838435846 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f3a3c06eea0 con 0x7f3a440b6690 2020-12-05T17:37:44.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.999+0000 7f3a437fe700 1 --2- 172.21.15.32:0/3838435846 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3a240553b0 0x7f3a24057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:44.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:43.999+0000 7f3a437fe700 1 --2- 172.21.15.32:0/3838435846 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3a240553b0 0x7f3a24057870 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f3a3400d000 tx=0x7f3a34008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:44.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.003+0000 7f3a40ff9700 1 -- 172.21.15.32:0/3838435846 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3a440a2f10 con 0x7f3a440b6690 2020-12-05T17:37:44.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.147+0000 7f3a49f05700 1 -- 172.21.15.32:0/3838435846 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f3a440a2f10 con 0x7f3a440b6690 2020-12-05T17:37:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.147+0000 7f3a40ff9700 1 -- 172.21.15.32:0/3838435846 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v67) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3a440a2f10 con 0x7f3a440b6690 2020-12-05T17:37:44.150 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:37:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.147+0000 7f3a49f05700 1 -- 172.21.15.32:0/3838435846 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3a240553b0 msgr2=0x7f3a24057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.147+0000 7f3a49f05700 1 --2- 172.21.15.32:0/3838435846 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3a240553b0 0x7f3a24057870 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f3a3400d000 tx=0x7f3a34008040).stop 2020-12-05T17:37:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.147+0000 7f3a49f05700 1 -- 172.21.15.32:0/3838435846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a440b6690 msgr2=0x7f3a44151ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.147+0000 7f3a49f05700 1 --2- 172.21.15.32:0/3838435846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a440b6690 0x7f3a44151ce0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f3a3c002f30 tx=0x7f3a3c015000).stop 2020-12-05T17:37:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.147+0000 7f3a49f05700 1 -- 172.21.15.32:0/3838435846 shutdown_connections 2020-12-05T17:37:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.147+0000 7f3a49f05700 1 --2- 172.21.15.32:0/3838435846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a440b6690 0x7f3a44151ce0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.147+0000 7f3a49f05700 1 --2- 172.21.15.32:0/3838435846 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3a240553b0 0x7f3a24057870 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.147+0000 7f3a49f05700 1 --2- 172.21.15.32:0/3838435846 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a440b5240 0x7f3a440a34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.147+0000 7f3a49f05700 1 --2- 172.21.15.32:0/3838435846 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3a440b5b20 0x7f3a441517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.147+0000 7f3a49f05700 1 -- 172.21.15.32:0/3838435846 >> 172.21.15.32:0/3838435846 conn(0x7f3a4405f3b0 msgr2=0x7f3a4405f790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.147+0000 7f3a49f05700 1 -- 172.21.15.32:0/3838435846 shutdown_connections 2020-12-05T17:37:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.147+0000 7f3a49f05700 1 -- 172.21.15.32:0/3838435846 wait complete. 2020-12-05T17:37:44.164 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:37:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96d0918700 1 Processor -- start 2020-12-05T17:37:44.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96d0918700 1 -- start start 2020-12-05T17:37:44.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96d0918700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96cc0b2000 0x7f96cc142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:44.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96d0918700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f96cc13fd20 0x7f96cc0af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:44.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96d0918700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96cc140190 con 0x7f96cc143410 2020-12-05T17:37:44.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96d0918700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96cc0aeba0 con 0x7f96cc0b2000 2020-12-05T17:37:44.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96d0918700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96cc0b23e0 con 0x7f96cc13fd20 2020-12-05T17:37:44.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96cad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96cc0b2000 0x7f96cc142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:44.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96cad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96cc0b2000 0x7f96cc142f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36538/0 (socket says 172.21.15.32:36538) 2020-12-05T17:37:44.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96cad9d700 1 -- 172.21.15.32:0/120381497 learned_addr learned my addr 172.21.15.32:0/120381497 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:44.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96ca59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f96cc143410 0x7f96cc13f610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39186/0 (socket says 172.21.15.32:39186) 2020-12-05T17:37:44.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96cb59e700 1 --2- 172.21.15.32:0/120381497 >> v2:172.21.15.61:3300/0 conn(0x7f96cc13fd20 0x7f96cc0af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96c9d9b700 1 -- 172.21.15.32:0/120381497 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1813138293 0 0) 0x7f96cc140190 con 0x7f96cc143410 2020-12-05T17:37:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96cad9d700 1 -- 172.21.15.32:0/120381497 >> v2:172.21.15.61:3300/0 conn(0x7f96cc13fd20 msgr2=0x7f96cc0af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96cad9d700 1 --2- 172.21.15.32:0/120381497 >> v2:172.21.15.61:3300/0 conn(0x7f96cc13fd20 0x7f96cc0af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96cad9d700 1 -- 172.21.15.32:0/120381497 >> v1:172.21.15.32:6789/0 conn(0x7f96cc143410 legacy=0x7f96cc13f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96cad9d700 1 -- 172.21.15.32:0/120381497 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96cc0af190 con 0x7f96cc0b2000 2020-12-05T17:37:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96cb59e700 1 --2- 172.21.15.32:0/120381497 >> v2:172.21.15.61:3300/0 conn(0x7f96cc13fd20 0x7f96cc0af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96cad9d700 1 --2- 172.21.15.32:0/120381497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96cc0b2000 0x7f96cc142f20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f96cc0acda0 tx=0x7f96bc009270).ready entity=mon.0 client_cookie=25d4dcf1dc981c69 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96c9d9b700 1 -- 172.21.15.32:0/120381497 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96bc00bbc0 con 0x7f96cc0b2000 2020-12-05T17:37:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96c9d9b700 1 -- 172.21.15.32:0/120381497 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96bc00bd20 con 0x7f96cc0b2000 2020-12-05T17:37:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96c9d9b700 1 -- 172.21.15.32:0/120381497 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96bc00be90 con 0x7f96cc0b2000 2020-12-05T17:37:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96d0918700 1 -- 172.21.15.32:0/120381497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96cc0b2000 msgr2=0x7f96cc142f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.283+0000 7f96d0918700 1 --2- 172.21.15.32:0/120381497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96cc0b2000 0x7f96cc142f20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f96cc0acda0 tx=0x7f96bc009270).stop 2020-12-05T17:37:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96d0918700 1 -- 172.21.15.32:0/120381497 shutdown_connections 2020-12-05T17:37:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96d0918700 1 --2- 172.21.15.32:0/120381497 >> v2:172.21.15.61:3300/0 conn(0x7f96cc13fd20 0x7f96cc0af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96d0918700 1 --2- 172.21.15.32:0/120381497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96cc0b2000 0x7f96cc142f20 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96d0918700 1 -- 172.21.15.32:0/120381497 >> 172.21.15.32:0/120381497 conn(0x7f96cc0b5620 msgr2=0x7f96cc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96d0918700 1 -- 172.21.15.32:0/120381497 shutdown_connections 2020-12-05T17:37:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96d0918700 1 -- 172.21.15.32:0/120381497 wait complete. 2020-12-05T17:37:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96d0918700 1 Processor -- start 2020-12-05T17:37:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96d0918700 1 -- start start 2020-12-05T17:37:44.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96d0918700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96cc0b2000 0x7f96cc145210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:44.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96d0918700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96cc13fd20 0x7f96cc0af650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:44.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96d0918700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96cc143410 0x7f96cc154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:44.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96d0918700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc145700 con 0x7f96cc143410 2020-12-05T17:37:44.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96d0918700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc13cad0 con 0x7f96cc13fd20 2020-12-05T17:37:44.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96d0918700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc151a40 con 0x7f96cc0b2000 2020-12-05T17:37:44.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96cb59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96cc143410 0x7f96cc154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:44.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96cb59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96cc143410 0x7f96cc154d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36548/0 (socket says 172.21.15.32:36548) 2020-12-05T17:37:44.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96cb59e700 1 -- 172.21.15.32:0/153330568 learned_addr learned my addr 172.21.15.32:0/153330568 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:44.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96ca59c700 1 --2- 172.21.15.32:0/153330568 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96cc13fd20 0x7f96cc0af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:44.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96cad9d700 1 --2- 172.21.15.32:0/153330568 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96cc0b2000 0x7f96cc145210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:44.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96cb59e700 1 -- 172.21.15.32:0/153330568 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96cc0b2000 msgr2=0x7f96cc145210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:44.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96cb59e700 1 --2- 172.21.15.32:0/153330568 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96cc0b2000 0x7f96cc145210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96cb59e700 1 -- 172.21.15.32:0/153330568 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96cc13fd20 msgr2=0x7f96cc0af650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:44.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96cb59e700 1 --2- 172.21.15.32:0/153330568 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96cc13fd20 0x7f96cc0af650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96cb59e700 1 -- 172.21.15.32:0/153330568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96cc151770 con 0x7f96cc143410 2020-12-05T17:37:44.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.287+0000 7f96cad9d700 1 --2- 172.21.15.32:0/153330568 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96cc0b2000 0x7f96cc145210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:44.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.291+0000 7f96cb59e700 1 --2- 172.21.15.32:0/153330568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96cc143410 0x7f96cc154d30 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f96c4006910 tx=0x7f96c4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:44.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.291+0000 7f96bbfff700 1 -- 172.21.15.32:0/153330568 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96c40127e0 con 0x7f96cc143410 2020-12-05T17:37:44.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.291+0000 7f96bbfff700 1 -- 172.21.15.32:0/153330568 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c4012940 con 0x7f96cc143410 2020-12-05T17:37:44.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.291+0000 7f96bbfff700 1 -- 172.21.15.32:0/153330568 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96c4012ca0 con 0x7f96cc143410 2020-12-05T17:37:44.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.291+0000 7f96d0918700 1 -- 172.21.15.32:0/153330568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96cc0afb40 con 0x7f96cc143410 2020-12-05T17:37:44.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.291+0000 7f96d0918700 1 -- 172.21.15.32:0/153330568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96cc13c550 con 0x7f96cc143410 2020-12-05T17:37:44.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.291+0000 7f96d0918700 1 -- 172.21.15.32:0/153330568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96cc13c940 con 0x7f96cc143410 2020-12-05T17:37:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.291+0000 7f96bbfff700 1 -- 172.21.15.32:0/153330568 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f96c4013480 con 0x7f96cc143410 2020-12-05T17:37:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.291+0000 7f96bbfff700 1 --2- 172.21.15.32:0/153330568 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f96a80553b0 0x7f96a8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.291+0000 7f96bbfff700 1 -- 172.21.15.32:0/153330568 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f96c406e5a0 con 0x7f96cc143410 2020-12-05T17:37:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.291+0000 7f96cad9d700 1 --2- 172.21.15.32:0/153330568 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f96a80553b0 0x7f96a8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.291+0000 7f96cad9d700 1 --2- 172.21.15.32:0/153330568 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f96a80553b0 0x7f96a8057870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f96bc018000 tx=0x7f96bc016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:44.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.299+0000 7f96bbfff700 1 -- 172.21.15.32:0/153330568 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f96c4044980 con 0x7f96cc143410 2020-12-05T17:37:44.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.443+0000 7f96d0918700 1 -- 172.21.15.32:0/153330568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f96cc057050 con 0x7f96cc143410 2020-12-05T17:37:44.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.443+0000 7f96bbfff700 1 -- 172.21.15.32:0/153330568 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v67) v1 ==== 92+0+10 (secure 0 0 0) 0x7f96c4047fa0 con 0x7f96cc143410 2020-12-05T17:37:44.449 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:37:44.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.443+0000 7f96d0918700 1 -- 172.21.15.32:0/153330568 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f96a80553b0 msgr2=0x7f96a8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:44.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.443+0000 7f96d0918700 1 --2- 172.21.15.32:0/153330568 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f96a80553b0 0x7f96a8057870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f96bc018000 tx=0x7f96bc016000).stop 2020-12-05T17:37:44.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.443+0000 7f96d0918700 1 -- 172.21.15.32:0/153330568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96cc143410 msgr2=0x7f96cc154d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:44.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.443+0000 7f96d0918700 1 --2- 172.21.15.32:0/153330568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96cc143410 0x7f96cc154d30 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f96c4006910 tx=0x7f96c4008040).stop 2020-12-05T17:37:44.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.443+0000 7f96d0918700 1 -- 172.21.15.32:0/153330568 shutdown_connections 2020-12-05T17:37:44.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.443+0000 7f96d0918700 1 --2- 172.21.15.32:0/153330568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96cc143410 0x7f96cc154d30 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.443+0000 7f96d0918700 1 --2- 172.21.15.32:0/153330568 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f96a80553b0 0x7f96a8057870 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.443+0000 7f96d0918700 1 --2- 172.21.15.32:0/153330568 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96cc0b2000 0x7f96cc145210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.443+0000 7f96d0918700 1 --2- 172.21.15.32:0/153330568 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96cc13fd20 0x7f96cc0af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.443+0000 7f96d0918700 1 -- 172.21.15.32:0/153330568 >> 172.21.15.32:0/153330568 conn(0x7f96cc0b5620 msgr2=0x7f96cc13e960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:44.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.447+0000 7f96d0918700 1 -- 172.21.15.32:0/153330568 shutdown_connections 2020-12-05T17:37:44.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.447+0000 7f96d0918700 1 -- 172.21.15.32:0/153330568 wait complete. 2020-12-05T17:37:44.461 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-12-05T17:37:44.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.571+0000 7fefc65f6700 1 Processor -- start 2020-12-05T17:37:44.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.571+0000 7fefc65f6700 1 -- start start 2020-12-05T17:37:44.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.571+0000 7fefc65f6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefc0145c00 0x7fefc0146050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:44.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.571+0000 7fefc65f6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefc00b7af0 con 0x7fefc0061780 2020-12-05T17:37:44.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.571+0000 7fefc65f6700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefc0062970 con 0x7fefc013c6f0 2020-12-05T17:37:44.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.571+0000 7fefc65f6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc00b2340 con 0x7fefc0145c00 2020-12-05T17:37:44.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.571+0000 7fefc4df3700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fefc0061780 0x7fefc00b73e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39204/0 (socket says 172.21.15.32:39204) 2020-12-05T17:37:44.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.571+0000 7fefc4df3700 1 -- 172.21.15.32:0/1500924487 learned_addr learned my addr 172.21.15.32:0/1500924487 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:44.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.571+0000 7fefc5df5700 1 --2- 172.21.15.32:0/1500924487 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefc0145c00 0x7fefc0146050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:44.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.571+0000 7fefbffff700 1 -- 172.21.15.32:0/1500924487 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1114132755 0 0) 0x7fefc00b7af0 con 0x7fefc0061780 2020-12-05T17:37:44.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.571+0000 7fefbffff700 1 -- 172.21.15.32:0/1500924487 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefac003230 con 0x7fefc0061780 2020-12-05T17:37:44.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.571+0000 7fefbffff700 1 -- 172.21.15.32:0/1500924487 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 567955449 0 0) 0x7fefc0062970 con 0x7fefc013c6f0 2020-12-05T17:37:44.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.571+0000 7fefc5df5700 1 -- 172.21.15.32:0/1500924487 >> v1:172.21.15.61:6789/0 conn(0x7fefc013c6f0 legacy=0x7fefc00ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:44.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.571+0000 7fefc5df5700 1 -- 172.21.15.32:0/1500924487 >> v1:172.21.15.32:6789/0 conn(0x7fefc0061780 legacy=0x7fefc00b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:44.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.571+0000 7fefc5df5700 1 -- 172.21.15.32:0/1500924487 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc00add70 con 0x7fefc0145c00 2020-12-05T17:37:44.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.571+0000 7fefc5df5700 1 --2- 172.21.15.32:0/1500924487 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefc0145c00 0x7fefc0146050 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fefb8002f30 tx=0x7fefb8006dc0).ready entity=mon.2 client_cookie=bc0b11a0053ac653 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:44.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.571+0000 7fefbffff700 1 -- 172.21.15.32:0/1500924487 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefb80097b0 con 0x7fefc0145c00 2020-12-05T17:37:44.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.571+0000 7fefbffff700 1 -- 172.21.15.32:0/1500924487 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb8009910 con 0x7fefc0145c00 2020-12-05T17:37:44.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.571+0000 7fefbffff700 1 -- 172.21.15.32:0/1500924487 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefb8009c70 con 0x7fefc0145c00 2020-12-05T17:37:44.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.571+0000 7fefc65f6700 1 -- 172.21.15.32:0/1500924487 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefc0145c00 msgr2=0x7fefc0146050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:44.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.571+0000 7fefc65f6700 1 --2- 172.21.15.32:0/1500924487 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefc0145c00 0x7fefc0146050 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fefb8002f30 tx=0x7fefb8006dc0).stop 2020-12-05T17:37:44.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.575+0000 7fefc65f6700 1 -- 172.21.15.32:0/1500924487 shutdown_connections 2020-12-05T17:37:44.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.575+0000 7fefc65f6700 1 --2- 172.21.15.32:0/1500924487 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefc0145c00 0x7fefc0146050 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.575+0000 7fefc65f6700 1 -- 172.21.15.32:0/1500924487 >> 172.21.15.32:0/1500924487 conn(0x7fefc005bc20 msgr2=0x7fefc00b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:44.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.575+0000 7fefc65f6700 1 -- 172.21.15.32:0/1500924487 shutdown_connections 2020-12-05T17:37:44.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.575+0000 7fefc65f6700 1 -- 172.21.15.32:0/1500924487 wait complete. 2020-12-05T17:37:44.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.575+0000 7fefc65f6700 1 Processor -- start 2020-12-05T17:37:44.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.575+0000 7fefc65f6700 1 -- start start 2020-12-05T17:37:44.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.575+0000 7fefc65f6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fefc0061780 0x7fefc0063f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:44.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.575+0000 7fefc65f6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefc013c6f0 0x7fefc013a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:44.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.575+0000 7fefc65f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefc0145c00 0x7fefc013acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:44.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.575+0000 7fefc65f6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc0149770 con 0x7fefc0145c00 2020-12-05T17:37:44.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.575+0000 7fefc65f6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc0063620 con 0x7fefc0061780 2020-12-05T17:37:44.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.575+0000 7fefc65f6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc0066b70 con 0x7fefc013c6f0 2020-12-05T17:37:44.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.575+0000 7fefc5df5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefc0145c00 0x7fefc013acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:44.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.575+0000 7fefc5df5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefc0145c00 0x7fefc013acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36560/0 (socket says 172.21.15.32:36560) 2020-12-05T17:37:44.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.575+0000 7fefc5df5700 1 -- 172.21.15.32:0/24858094 learned_addr learned my addr 172.21.15.32:0/24858094 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:44.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.575+0000 7fefc55f4700 1 --2- 172.21.15.32:0/24858094 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fefc0061780 0x7fefc0063f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:44.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.575+0000 7fefc4df3700 1 --2- 172.21.15.32:0/24858094 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefc013c6f0 0x7fefc013a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:44.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.579+0000 7fefc5df5700 1 -- 172.21.15.32:0/24858094 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefc013c6f0 msgr2=0x7fefc013a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:44.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.579+0000 7fefc5df5700 1 --2- 172.21.15.32:0/24858094 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefc013c6f0 0x7fefc013a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.579+0000 7fefc5df5700 1 -- 172.21.15.32:0/24858094 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fefc0061780 msgr2=0x7fefc0063f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.579+0000 7fefc5df5700 1 --2- 172.21.15.32:0/24858094 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fefc0061780 0x7fefc0063f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.579+0000 7fefc5df5700 1 -- 172.21.15.32:0/24858094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc0062fe0 con 0x7fefc0145c00 2020-12-05T17:37:44.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.579+0000 7fefc55f4700 1 --2- 172.21.15.32:0/24858094 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fefc0061780 0x7fefc0063f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:37:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.579+0000 7fefc5df5700 1 --2- 172.21.15.32:0/24858094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefc0145c00 0x7fefc013acf0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7fefb8002f30 tx=0x7fefb8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.579+0000 7fefbe7fc700 1 -- 172.21.15.32:0/24858094 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefb8011ae0 con 0x7fefc0145c00 2020-12-05T17:37:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.579+0000 7fefbe7fc700 1 -- 172.21.15.32:0/24858094 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb8011c40 con 0x7fefc0145c00 2020-12-05T17:37:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.579+0000 7fefc65f6700 1 -- 172.21.15.32:0/24858094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc01493f0 con 0x7fefc0145c00 2020-12-05T17:37:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.579+0000 7fefbe7fc700 1 -- 172.21.15.32:0/24858094 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefb801d450 con 0x7fefc0145c00 2020-12-05T17:37:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.579+0000 7fefc65f6700 1 -- 172.21.15.32:0/24858094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc01484f0 con 0x7fefc0145c00 2020-12-05T17:37:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.579+0000 7fefc65f6700 1 -- 172.21.15.32:0/24858094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefc00637e0 con 0x7fefc0145c00 2020-12-05T17:37:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.579+0000 7fefbe7fc700 1 -- 172.21.15.32:0/24858094 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fefb8024030 con 0x7fefc0145c00 2020-12-05T17:37:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.579+0000 7fefbe7fc700 1 --2- 172.21.15.32:0/24858094 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fefa0055330 0x7fefa00577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.579+0000 7fefbe7fc700 1 -- 172.21.15.32:0/24858094 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fefb806eed0 con 0x7fefc0145c00 2020-12-05T17:37:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.579+0000 7fefc55f4700 1 --2- 172.21.15.32:0/24858094 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fefa0055330 0x7fefa00577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.583+0000 7fefc55f4700 1 --2- 172.21.15.32:0/24858094 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fefa0055330 0x7fefa00577f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fefb000d000 tx=0x7fefb0009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.587+0000 7fefbe7fc700 1 -- 172.21.15.32:0/24858094 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fefb80453f0 con 0x7fefc0145c00 2020-12-05T17:37:44.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.727+0000 7fefc65f6700 1 -- 172.21.15.32:0/24858094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fefc0057050 con 0x7fefc0145c00 2020-12-05T17:37:44.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.731+0000 7fefbe7fc700 1 -- 172.21.15.32:0/24858094 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v67) v1 ==== 84+0+143 (secure 0 0 0) 0x7fefb8022070 con 0x7fefc0145c00 2020-12-05T17:37:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:44.733 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2020-12-05T17:37:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2020-12-05T17:37:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2020-12-05T17:37:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:37:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:37:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2020-12-05T17:37:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2020-12-05T17:37:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:37:44.735 INFO:teuthology.orchestra.run.smithi032.stdout:] 2020-12-05T17:37:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.731+0000 7fefc65f6700 1 -- 172.21.15.32:0/24858094 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fefa0055330 msgr2=0x7fefa00577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:44.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.731+0000 7fefc65f6700 1 --2- 172.21.15.32:0/24858094 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fefa0055330 0x7fefa00577f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fefb000d000 tx=0x7fefb0009040).stop 2020-12-05T17:37:44.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.731+0000 7fefc65f6700 1 -- 172.21.15.32:0/24858094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefc0145c00 msgr2=0x7fefc013acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:44.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.731+0000 7fefc65f6700 1 --2- 172.21.15.32:0/24858094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefc0145c00 0x7fefc013acf0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7fefb8002f30 tx=0x7fefb8015000).stop 2020-12-05T17:37:44.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.731+0000 7fefc65f6700 1 -- 172.21.15.32:0/24858094 shutdown_connections 2020-12-05T17:37:44.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.731+0000 7fefc65f6700 1 --2- 172.21.15.32:0/24858094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefc0145c00 0x7fefc013acf0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.731+0000 7fefc65f6700 1 --2- 172.21.15.32:0/24858094 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fefc0061780 0x7fefc0063f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.731+0000 7fefc65f6700 1 --2- 172.21.15.32:0/24858094 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fefa0055330 0x7fefa00577f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.731+0000 7fefc65f6700 1 --2- 172.21.15.32:0/24858094 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefc013c6f0 0x7fefc013a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.731+0000 7fefc65f6700 1 -- 172.21.15.32:0/24858094 >> 172.21.15.32:0/24858094 conn(0x7fefc005bc20 msgr2=0x7fefc0143e60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:44.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.731+0000 7fefc65f6700 1 -- 172.21.15.32:0/24858094 shutdown_connections 2020-12-05T17:37:44.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.731+0000 7fefc65f6700 1 -- 172.21.15.32:0/24858094 wait complete. 2020-12-05T17:37:44.747 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-12-05T17:37:44.748 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-12-05T17:37:44.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.863+0000 7f01f2ef8700 1 Processor -- start 2020-12-05T17:37:44.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.863+0000 7f01f2ef8700 1 -- start start 2020-12-05T17:37:44.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f2ef8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ec0af5f0 0x7f01ec0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:44.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f2ef8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ec1468e0 con 0x7f01ec0afd30 2020-12-05T17:37:44.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f2ef8700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ec146b10 con 0x7f01ec0b1dc0 2020-12-05T17:37:44.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f2ef8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec146db0 con 0x7f01ec0af5f0 2020-12-05T17:37:44.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f16f5700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f01ec0afd30 0x7f01ec141c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39216/0 (socket says 172.21.15.32:39216) 2020-12-05T17:37:44.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f16f5700 1 -- 172.21.15.32:0/1109526671 learned_addr learned my addr 172.21.15.32:0/1109526671 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:44.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f26f7700 1 --2- 172.21.15.32:0/1109526671 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ec0af5f0 0x7f01ec0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:44.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f0ef4700 1 -- 172.21.15.32:0/1109526671 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2022706017 0 0) 0x7f01ec1468e0 con 0x7f01ec0afd30 2020-12-05T17:37:44.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f0ef4700 1 -- 172.21.15.32:0/1109526671 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01d4003230 con 0x7f01ec0afd30 2020-12-05T17:37:44.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f0ef4700 1 -- 172.21.15.32:0/1109526671 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3446772761 0 0) 0x7f01ec146b10 con 0x7f01ec0b1dc0 2020-12-05T17:37:44.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f0ef4700 1 -- 172.21.15.32:0/1109526671 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01ec1468e0 con 0x7f01ec0b1dc0 2020-12-05T17:37:44.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f0ef4700 1 -- 172.21.15.32:0/1109526671 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 200055986 0 0) 0x7f01d4003230 con 0x7f01ec0afd30 2020-12-05T17:37:44.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f0ef4700 1 -- 172.21.15.32:0/1109526671 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ec0af5f0 msgr2=0x7f01ec0b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:44.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f0ef4700 1 --2- 172.21.15.32:0/1109526671 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ec0af5f0 0x7f01ec0b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f0ef4700 1 -- 172.21.15.32:0/1109526671 >> v1:172.21.15.61:6789/0 conn(0x7f01ec0b1dc0 legacy=0x7f01ec0ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:44.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f0ef4700 1 -- 172.21.15.32:0/1109526671 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec0ae160 con 0x7f01ec0afd30 2020-12-05T17:37:44.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f0ef4700 1 -- 172.21.15.32:0/1109526671 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f01e80016d0 con 0x7f01ec0afd30 2020-12-05T17:37:44.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f0ef4700 1 -- 172.21.15.32:0/1109526671 >> v1:172.21.15.32:6789/0 conn(0x7f01ec0afd30 legacy=0x7f01ec141c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:44.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f0ef4700 1 --2- 172.21.15.32:0/1109526671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01d40038e0 0x7f01d4003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:44.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f0ef4700 1 --2- 172.21.15.32:0/1109526671 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01d40044d0 0x7f01d4004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:44.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f0ef4700 1 --2- 172.21.15.32:0/1109526671 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ec0af5f0 0x7f01d4004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:44.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f0ef4700 1 -- 172.21.15.32:0/1109526671 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec146db0 con 0x7f01d40038e0 2020-12-05T17:37:44.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f0ef4700 1 -- 172.21.15.32:0/1109526671 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d4005360 con 0x7f01d40044d0 2020-12-05T17:37:44.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f0ef4700 1 -- 172.21.15.32:0/1109526671 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d400b770 con 0x7f01ec0af5f0 2020-12-05T17:37:44.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f1ef6700 1 --2- 172.21.15.32:0/1109526671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01d40038e0 0x7f01d4003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:44.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f16f5700 1 --2- 172.21.15.32:0/1109526671 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01d40044d0 0x7f01d4004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:44.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f26f7700 1 --2- 172.21.15.32:0/1109526671 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ec0af5f0 0x7f01d4004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:44.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f1ef6700 1 -- 172.21.15.32:0/1109526671 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ec0af5f0 msgr2=0x7f01d4004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:44.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f1ef6700 1 --2- 172.21.15.32:0/1109526671 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ec0af5f0 0x7f01d4004e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f1ef6700 1 -- 172.21.15.32:0/1109526671 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01d40044d0 msgr2=0x7f01d4004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:44.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f1ef6700 1 --2- 172.21.15.32:0/1109526671 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01d40044d0 0x7f01d4004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f1ef6700 1 -- 172.21.15.32:0/1109526671 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d4003230 con 0x7f01d40038e0 2020-12-05T17:37:44.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f1ef6700 1 --2- 172.21.15.32:0/1109526671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01d40038e0 0x7f01d4003ce0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f01d40054a0 tx=0x7f01e0007b60).ready entity=mon.0 client_cookie=98e7aa09befcad47 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:44.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f0ef4700 1 -- 172.21.15.32:0/1109526671 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01e000c7d0 con 0x7f01d40038e0 2020-12-05T17:37:44.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f0ef4700 1 -- 172.21.15.32:0/1109526671 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e000c930 con 0x7f01d40038e0 2020-12-05T17:37:44.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f0ef4700 1 -- 172.21.15.32:0/1109526671 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01e000cc90 con 0x7f01d40038e0 2020-12-05T17:37:44.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f2ef8700 1 -- 172.21.15.32:0/1109526671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01d40038e0 msgr2=0x7f01d4003ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:44.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f2ef8700 1 --2- 172.21.15.32:0/1109526671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01d40038e0 0x7f01d4003ce0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f01d40054a0 tx=0x7f01e0007b60).stop 2020-12-05T17:37:44.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f26f7700 1 -- 172.21.15.32:0/1109526671 reap_dead start 2020-12-05T17:37:44.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f2ef8700 1 -- 172.21.15.32:0/1109526671 shutdown_connections 2020-12-05T17:37:44.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f2ef8700 1 -- 172.21.15.32:0/1109526671 >> 172.21.15.32:0/1109526671 conn(0x7f01ec0620b0 msgr2=0x7f01ec062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:44.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f2ef8700 1 -- 172.21.15.32:0/1109526671 shutdown_connections 2020-12-05T17:37:44.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.867+0000 7f01f2ef8700 1 -- 172.21.15.32:0/1109526671 wait complete. 2020-12-05T17:37:44.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01f2ef8700 1 Processor -- start 2020-12-05T17:37:44.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01f2ef8700 1 -- start start 2020-12-05T17:37:44.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01f2ef8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ec0b13d0 0x7f01ec154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:44.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01f2ef8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01ec0640a0 0x7f01ec0649b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:44.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01f2ef8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01ec064ea0 0x7f01ec13f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:44.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01f2ef8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0630e0 con 0x7f01ec0640a0 2020-12-05T17:37:44.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01f2ef8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec065320 con 0x7f01ec064ea0 2020-12-05T17:37:44.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01f2ef8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0b17b0 con 0x7f01ec0b13d0 2020-12-05T17:37:44.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01f16f5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01ec0640a0 0x7f01ec0649b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:44.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01f1ef6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ec0b13d0 0x7f01ec154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:44.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01f16f5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01ec0640a0 0x7f01ec0649b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36580/0 (socket says 172.21.15.32:36580) 2020-12-05T17:37:44.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01f16f5700 1 -- 172.21.15.32:0/3034993735 learned_addr learned my addr 172.21.15.32:0/3034993735 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:44.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01f26f7700 1 --2- 172.21.15.32:0/3034993735 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01ec064ea0 0x7f01ec13f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:44.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01f16f5700 1 -- 172.21.15.32:0/3034993735 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ec0b13d0 msgr2=0x7f01ec154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:44.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01f16f5700 1 --2- 172.21.15.32:0/3034993735 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ec0b13d0 0x7f01ec154a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01f16f5700 1 -- 172.21.15.32:0/3034993735 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01ec064ea0 msgr2=0x7f01ec13f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:44.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01f16f5700 1 --2- 172.21.15.32:0/3034993735 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01ec064ea0 0x7f01ec13f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:44.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01f16f5700 1 -- 172.21.15.32:0/3034993735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec063310 con 0x7f01ec0640a0 2020-12-05T17:37:44.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01f26f7700 1 --2- 172.21.15.32:0/3034993735 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01ec064ea0 0x7f01ec13f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:44.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01f16f5700 1 --2- 172.21.15.32:0/3034993735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01ec0640a0 0x7f01ec0649b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f01ec065490 tx=0x7f01ec0ae2f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:44.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01deffd700 1 -- 172.21.15.32:0/3034993735 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01e800c9a0 con 0x7f01ec0640a0 2020-12-05T17:37:44.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01deffd700 1 -- 172.21.15.32:0/3034993735 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e800cb00 con 0x7f01ec0640a0 2020-12-05T17:37:44.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01f2ef8700 1 -- 172.21.15.32:0/3034993735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ec0635b0 con 0x7f01ec0640a0 2020-12-05T17:37:44.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01deffd700 1 -- 172.21.15.32:0/3034993735 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01e800ce60 con 0x7f01ec0640a0 2020-12-05T17:37:44.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.871+0000 7f01f2ef8700 1 -- 172.21.15.32:0/3034993735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ec112990 con 0x7f01ec0640a0 2020-12-05T17:37:44.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.875+0000 7f01f2ef8700 1 -- 172.21.15.32:0/3034993735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ec057050 con 0x7f01ec0640a0 2020-12-05T17:37:44.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.875+0000 7f01deffd700 1 -- 172.21.15.32:0/3034993735 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f01e8014430 con 0x7f01ec0640a0 2020-12-05T17:37:44.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.875+0000 7f01deffd700 1 --2- 172.21.15.32:0/3034993735 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f01cc0554e0 0x7f01cc0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:44.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.875+0000 7f01deffd700 1 -- 172.21.15.32:0/3034993735 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f01e806dba0 con 0x7f01ec0640a0 2020-12-05T17:37:44.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.875+0000 7f01f1ef6700 1 --2- 172.21.15.32:0/3034993735 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f01cc0554e0 0x7f01cc0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:44.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.875+0000 7f01f1ef6700 1 --2- 172.21.15.32:0/3034993735 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f01cc0554e0 0x7f01cc0579a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f01e001b000 tx=0x7f01e0017000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:44.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:44.883+0000 7f01deffd700 1 -- 172.21.15.32:0/3034993735 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f01e8043f50 con 0x7f01ec0640a0 2020-12-05T17:37:45.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.027+0000 7f01f2ef8700 1 -- 172.21.15.32:0/3034993735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f01ec060c10 con 0x7f01ec0640a0 2020-12-05T17:37:45.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.139+0000 7f01deffd700 1 -- 172.21.15.32:0/3034993735 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v68) v1 ==== 133+0+0 (secure 0 0 0) 0x7f01e8047530 con 0x7f01ec0640a0 2020-12-05T17:37:45.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.143+0000 7f01f2ef8700 1 -- 172.21.15.32:0/3034993735 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f01cc0554e0 msgr2=0x7f01cc0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:45.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.143+0000 7f01f2ef8700 1 --2- 172.21.15.32:0/3034993735 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f01cc0554e0 0x7f01cc0579a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f01e001b000 tx=0x7f01e0017000).stop 2020-12-05T17:37:45.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.143+0000 7f01f2ef8700 1 -- 172.21.15.32:0/3034993735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01ec0640a0 msgr2=0x7f01ec0649b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:45.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.143+0000 7f01f2ef8700 1 --2- 172.21.15.32:0/3034993735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01ec0640a0 0x7f01ec0649b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f01ec065490 tx=0x7f01ec0ae2f0).stop 2020-12-05T17:37:45.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.147+0000 7f01f2ef8700 1 -- 172.21.15.32:0/3034993735 shutdown_connections 2020-12-05T17:37:45.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.147+0000 7f01f2ef8700 1 --2- 172.21.15.32:0/3034993735 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01ec064ea0 0x7f01ec13f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:45.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.147+0000 7f01f2ef8700 1 --2- 172.21.15.32:0/3034993735 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f01cc0554e0 0x7f01cc0579a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:45.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.147+0000 7f01f2ef8700 1 --2- 172.21.15.32:0/3034993735 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01ec0b13d0 0x7f01ec154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:45.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.147+0000 7f01f2ef8700 1 --2- 172.21.15.32:0/3034993735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01ec0640a0 0x7f01ec0649b0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:45.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.147+0000 7f01f2ef8700 1 -- 172.21.15.32:0/3034993735 >> 172.21.15.32:0/3034993735 conn(0x7f01ec0620b0 msgr2=0x7f01ec0ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:45.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.147+0000 7f01f2ef8700 1 -- 172.21.15.32:0/3034993735 shutdown_connections 2020-12-05T17:37:45.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.147+0000 7f01f2ef8700 1 -- 172.21.15.32:0/3034993735 wait complete. 2020-12-05T17:37:45.152 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2020-12-05T17:37:45.165 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-12-05T17:37:45.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce759e700 1 Processor -- start 2020-12-05T17:37:45.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce759e700 1 -- start start 2020-12-05T17:37:45.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ce80b2960 0x7f0ce8140400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:45.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce80ad9b0 con 0x7f0ce80667f0 2020-12-05T17:37:45.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce759e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce813f4f0 con 0x7f0ce80ad280 2020-12-05T17:37:45.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce759e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce80b32c0 con 0x7f0ce80b2960 2020-12-05T17:37:45.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ce80b2960 0x7f0ce8140400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:45.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce6d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0ce80667f0 0x7f0ce80ae830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39236/0 (socket says 172.21.15.32:39236) 2020-12-05T17:37:45.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce6d9d700 1 -- 172.21.15.32:0/1862454635 learned_addr learned my addr 172.21.15.32:0/1862454635 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:45.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce559a700 1 -- 172.21.15.32:0/1862454635 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3770726293 0 0) 0x7f0ce80ad9b0 con 0x7f0ce80667f0 2020-12-05T17:37:45.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce559a700 1 -- 172.21.15.32:0/1862454635 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ccc003230 con 0x7f0ce80667f0 2020-12-05T17:37:45.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce659c700 1 -- 172.21.15.32:0/1862454635 >> v1:172.21.15.61:6789/0 conn(0x7f0ce80ad280 legacy=0x7f0ce80660e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:45.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce659c700 1 -- 172.21.15.32:0/1862454635 >> v1:172.21.15.32:6789/0 conn(0x7f0ce80667f0 legacy=0x7f0ce80ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:45.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce559a700 1 -- 172.21.15.32:0/1862454635 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 75335780 0 0) 0x7f0ce813f4f0 con 0x7f0ce80ad280 2020-12-05T17:37:45.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce659c700 1 -- 172.21.15.32:0/1862454635 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce80b3020 con 0x7f0ce80b2960 2020-12-05T17:37:45.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce659c700 1 --2- 172.21.15.32:0/1862454635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce80b2960 0x7f0ce8140400 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f0cd8002f30 tx=0x7f0cd8006d60).ready entity=mon.0 client_cookie=54ae7b55c38e066 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:45.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce559a700 1 -- 172.21.15.32:0/1862454635 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cd8009600 con 0x7f0ce80b2960 2020-12-05T17:37:45.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce559a700 1 -- 172.21.15.32:0/1862454635 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd8009760 con 0x7f0ce80b2960 2020-12-05T17:37:45.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce559a700 1 -- 172.21.15.32:0/1862454635 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cd80098d0 con 0x7f0ce80b2960 2020-12-05T17:37:45.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce759e700 1 -- 172.21.15.32:0/1862454635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce80b2960 msgr2=0x7f0ce8140400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:45.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce759e700 1 --2- 172.21.15.32:0/1862454635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce80b2960 0x7f0ce8140400 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f0cd8002f30 tx=0x7f0cd8006d60).stop 2020-12-05T17:37:45.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce759e700 1 -- 172.21.15.32:0/1862454635 shutdown_connections 2020-12-05T17:37:45.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce759e700 1 --2- 172.21.15.32:0/1862454635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce80b2960 0x7f0ce8140400 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:45.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce759e700 1 -- 172.21.15.32:0/1862454635 >> 172.21.15.32:0/1862454635 conn(0x7f0ce8061010 msgr2=0x7f0ce8061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:45.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce759e700 1 -- 172.21.15.32:0/1862454635 shutdown_connections 2020-12-05T17:37:45.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.283+0000 7f0ce759e700 1 -- 172.21.15.32:0/1862454635 wait complete. 2020-12-05T17:37:45.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0ce759e700 1 Processor -- start 2020-12-05T17:37:45.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0ce759e700 1 -- start start 2020-12-05T17:37:45.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0ce759e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ce80667f0 0x7f0ce813cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:45.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0ce759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce80ad280 0x7f0ce8065600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:45.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0ce759e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ce80b2960 0x7f0ce805f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:45.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0ce759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce813aa30 con 0x7f0ce80ad280 2020-12-05T17:37:45.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0ce759e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce81557b0 con 0x7f0ce80667f0 2020-12-05T17:37:45.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0ce759e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce8155380 con 0x7f0ce80b2960 2020-12-05T17:37:45.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0ce5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce80ad280 0x7f0ce8065600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:45.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0ce5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce80ad280 0x7f0ce8065600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36596/0 (socket says 172.21.15.32:36596) 2020-12-05T17:37:45.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0ce5d9b700 1 -- 172.21.15.32:0/2072470565 learned_addr learned my addr 172.21.15.32:0/2072470565 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:45.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0ce659c700 1 --2- 172.21.15.32:0/2072470565 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ce80667f0 0x7f0ce813cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:45.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0ce6d9d700 1 --2- 172.21.15.32:0/2072470565 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ce80b2960 0x7f0ce805f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:45.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0ce5d9b700 1 -- 172.21.15.32:0/2072470565 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ce80b2960 msgr2=0x7f0ce805f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:45.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0ce5d9b700 1 --2- 172.21.15.32:0/2072470565 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ce80b2960 0x7f0ce805f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:45.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0ce5d9b700 1 -- 172.21.15.32:0/2072470565 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ce80667f0 msgr2=0x7f0ce813cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:45.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0ce5d9b700 1 --2- 172.21.15.32:0/2072470565 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ce80667f0 0x7f0ce813cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:45.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0ce5d9b700 1 -- 172.21.15.32:0/2072470565 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce8140a90 con 0x7f0ce80ad280 2020-12-05T17:37:45.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0ce5d9b700 1 --2- 172.21.15.32:0/2072470565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce80ad280 0x7f0ce8065600 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f0ce81558f0 tx=0x7f0cdc007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:45.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0cd77fe700 1 -- 172.21.15.32:0/2072470565 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cdc00e4a0 con 0x7f0ce80ad280 2020-12-05T17:37:45.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0cd77fe700 1 -- 172.21.15.32:0/2072470565 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc00e600 con 0x7f0ce80ad280 2020-12-05T17:37:45.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0cd77fe700 1 -- 172.21.15.32:0/2072470565 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cdc00e960 con 0x7f0ce80ad280 2020-12-05T17:37:45.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0ce759e700 1 -- 172.21.15.32:0/2072470565 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce813cfa0 con 0x7f0ce80ad280 2020-12-05T17:37:45.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.287+0000 7f0ce759e700 1 -- 172.21.15.32:0/2072470565 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce80b18e0 con 0x7f0ce80ad280 2020-12-05T17:37:45.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.291+0000 7f0ce759e700 1 -- 172.21.15.32:0/2072470565 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce8057050 con 0x7f0ce80ad280 2020-12-05T17:37:45.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.291+0000 7f0cd77fe700 1 -- 172.21.15.32:0/2072470565 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0cdc019030 con 0x7f0ce80ad280 2020-12-05T17:37:45.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.291+0000 7f0cd77fe700 1 --2- 172.21.15.32:0/2072470565 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0cc4055440 0x7f0cc4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:45.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.291+0000 7f0cd77fe700 1 -- 172.21.15.32:0/2072470565 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f0cdc06d710 con 0x7f0ce80ad280 2020-12-05T17:37:45.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.291+0000 7f0ce659c700 1 --2- 172.21.15.32:0/2072470565 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0cc4055440 0x7f0cc4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:45.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.291+0000 7f0ce659c700 1 --2- 172.21.15.32:0/2072470565 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0cc4055440 0x7f0cc4057900 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f0cd801b000 tx=0x7f0cd8016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:45.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.299+0000 7f0cd77fe700 1 -- 172.21.15.32:0/2072470565 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0cdc043b30 con 0x7f0ce80ad280 2020-12-05T17:37:45.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:45.443+0000 7f0ce759e700 1 -- 172.21.15.32:0/2072470565 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f0ce81554c0 con 0x7f0ce80ad280 2020-12-05T17:37:45.527 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:37:45.530 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:37:45.532 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:37:45.535 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:37:45.537 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:37:46.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.143+0000 7f0cd77fe700 1 -- 172.21.15.32:0/2072470565 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v69) v1 ==== 125+0+0 (secure 0 0 0) 0x7f0cdc021070 con 0x7f0ce80ad280 2020-12-05T17:37:46.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.151+0000 7f0ce759e700 1 -- 172.21.15.32:0/2072470565 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0cc4055440 msgr2=0x7f0cc4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:46.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.151+0000 7f0ce759e700 1 --2- 172.21.15.32:0/2072470565 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0cc4055440 0x7f0cc4057900 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f0cd801b000 tx=0x7f0cd8016000).stop 2020-12-05T17:37:46.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.151+0000 7f0ce759e700 1 -- 172.21.15.32:0/2072470565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce80ad280 msgr2=0x7f0ce8065600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:46.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.151+0000 7f0ce759e700 1 --2- 172.21.15.32:0/2072470565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce80ad280 0x7f0ce8065600 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f0ce81558f0 tx=0x7f0cdc007ac0).stop 2020-12-05T17:37:46.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.151+0000 7f0ce759e700 1 -- 172.21.15.32:0/2072470565 shutdown_connections 2020-12-05T17:37:46.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.151+0000 7f0ce759e700 1 --2- 172.21.15.32:0/2072470565 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0cc4055440 0x7f0cc4057900 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.151+0000 7f0ce759e700 1 --2- 172.21.15.32:0/2072470565 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ce80b2960 0x7f0ce805f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.151+0000 7f0ce759e700 1 --2- 172.21.15.32:0/2072470565 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ce80667f0 0x7f0ce813cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.151+0000 7f0ce759e700 1 --2- 172.21.15.32:0/2072470565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce80ad280 0x7f0ce8065600 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.151+0000 7f0ce759e700 1 -- 172.21.15.32:0/2072470565 >> 172.21.15.32:0/2072470565 conn(0x7f0ce8061010 msgr2=0x7f0ce813e030 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:46.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.155+0000 7f0ce759e700 1 -- 172.21.15.32:0/2072470565 shutdown_connections 2020-12-05T17:37:46.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.155+0000 7f0ce759e700 1 -- 172.21.15.32:0/2072470565 wait complete. 2020-12-05T17:37:46.160 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2020-12-05T17:37:46.174 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-12-05T17:37:46.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.279+0000 7fbbfbfff700 1 Processor -- start 2020-12-05T17:37:46.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.279+0000 7fbbfbfff700 1 -- start start 2020-12-05T17:37:46.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfbfff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fbbfc13ed60 0x7fbbfc0af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:46.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfbfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbbfc0ad250 0x7fbbfc142600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:46.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfbfff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbfc0b24e0 con 0x7fbbfc0b2100 2020-12-05T17:37:46.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfbfff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbfc13fd10 con 0x7fbbfc13ed60 2020-12-05T17:37:46.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfbfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc0b0780 con 0x7fbbfc0ad250 2020-12-05T17:37:46.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfb7fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbbfc0ad250 0x7fbbfc142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:46.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfaffd700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fbbfc0b2100 0x7fbbfc138820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:51888/0 (socket says 172.21.15.32:51888) 2020-12-05T17:37:46.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfa7fc700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fbbfc13ed60 0x7fbbfc0af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:46.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfaffd700 1 -- 172.21.15.32:0/1247545852 learned_addr learned my addr 172.21.15.32:0/1247545852 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:46.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbf9ffb700 1 -- 172.21.15.32:0/1247545852 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 854485071 0 0) 0x7fbbfc0b24e0 con 0x7fbbfc0b2100 2020-12-05T17:37:46.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbf9ffb700 1 -- 172.21.15.32:0/1247545852 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbe0003230 con 0x7fbbfc0b2100 2020-12-05T17:37:46.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfb7fe700 1 -- 172.21.15.32:0/1247545852 >> v2:172.21.15.61:3300/0 conn(0x7fbbfc13ed60 msgr2=0x7fbbfc0af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:46.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfb7fe700 1 --2- 172.21.15.32:0/1247545852 >> v2:172.21.15.61:3300/0 conn(0x7fbbfc13ed60 0x7fbbfc0af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfb7fe700 1 -- 172.21.15.32:0/1247545852 >> v1:172.21.15.61:6789/0 conn(0x7fbbfc0b2100 legacy=0x7fbbfc138820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:46.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfb7fe700 1 -- 172.21.15.32:0/1247545852 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbfc0b0240 con 0x7fbbfc0ad250 2020-12-05T17:37:46.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfb7fe700 1 --2- 172.21.15.32:0/1247545852 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbbfc0ad250 0x7fbbfc142600 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fbbf4002f30 tx=0x7fbbf4006dc0).ready entity=mon.2 client_cookie=b373fef83b056e3 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:46.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbf9ffb700 1 -- 172.21.15.32:0/1247545852 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbf40096a0 con 0x7fbbfc0ad250 2020-12-05T17:37:46.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbf9ffb700 1 -- 172.21.15.32:0/1247545852 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf4009800 con 0x7fbbfc0ad250 2020-12-05T17:37:46.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbf9ffb700 1 -- 172.21.15.32:0/1247545852 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbf4009b60 con 0x7fbbfc0ad250 2020-12-05T17:37:46.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfbfff700 1 -- 172.21.15.32:0/1247545852 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbbfc0ad250 msgr2=0x7fbbfc142600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:46.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfbfff700 1 --2- 172.21.15.32:0/1247545852 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbbfc0ad250 0x7fbbfc142600 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fbbf4002f30 tx=0x7fbbf4006dc0).stop 2020-12-05T17:37:46.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfbfff700 1 -- 172.21.15.32:0/1247545852 shutdown_connections 2020-12-05T17:37:46.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfbfff700 1 --2- 172.21.15.32:0/1247545852 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbbfc0ad250 0x7fbbfc142600 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfbfff700 1 --2- 172.21.15.32:0/1247545852 >> v2:172.21.15.61:3300/0 conn(0x7fbbfc13ed60 0x7fbbfc0af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfbfff700 1 -- 172.21.15.32:0/1247545852 >> 172.21.15.32:0/1247545852 conn(0x7fbbfc0620b0 msgr2=0x7fbbfc062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:46.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfbfff700 1 -- 172.21.15.32:0/1247545852 shutdown_connections 2020-12-05T17:37:46.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfbfff700 1 -- 172.21.15.32:0/1247545852 wait complete. 2020-12-05T17:37:46.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfbfff700 1 Processor -- start 2020-12-05T17:37:46.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.283+0000 7fbbfbfff700 1 -- start start 2020-12-05T17:37:46.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.287+0000 7fbbfbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbfc0ad250 0x7fbbfc1370e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:46.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.287+0000 7fbbfbfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbbfc0b2100 0x7fbbfc135690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:46.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.287+0000 7fbbfbfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbbfc066dd0 0x7fbbfc152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:46.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.287+0000 7fbbfbfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc135b80 con 0x7fbbfc0ad250 2020-12-05T17:37:46.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.287+0000 7fbbfbfff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc0640f0 con 0x7fbbfc0b2100 2020-12-05T17:37:46.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.287+0000 7fbbfbfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc064380 con 0x7fbbfc066dd0 2020-12-05T17:37:46.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.287+0000 7fbbfaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbfc0ad250 0x7fbbfc1370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:46.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.287+0000 7fbbfaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbfc0ad250 0x7fbbfc1370e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36610/0 (socket says 172.21.15.32:36610) 2020-12-05T17:37:46.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.287+0000 7fbbfaffd700 1 -- 172.21.15.32:0/1450648026 learned_addr learned my addr 172.21.15.32:0/1450648026 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:46.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.287+0000 7fbbfb7fe700 1 --2- 172.21.15.32:0/1450648026 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbbfc066dd0 0x7fbbfc152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:46.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.287+0000 7fbbfa7fc700 1 --2- 172.21.15.32:0/1450648026 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbbfc0b2100 0x7fbbfc135690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:46.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.287+0000 7fbbfaffd700 1 -- 172.21.15.32:0/1450648026 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbbfc066dd0 msgr2=0x7fbbfc152910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:46.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.287+0000 7fbbfaffd700 1 --2- 172.21.15.32:0/1450648026 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbbfc066dd0 0x7fbbfc152910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.287+0000 7fbbfaffd700 1 -- 172.21.15.32:0/1450648026 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbbfc0b2100 msgr2=0x7fbbfc135690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:46.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.287+0000 7fbbfaffd700 1 --2- 172.21.15.32:0/1450648026 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbbfc0b2100 0x7fbbfc135690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.287+0000 7fbbfaffd700 1 -- 172.21.15.32:0/1450648026 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbfc133530 con 0x7fbbfc0ad250 2020-12-05T17:37:46.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.287+0000 7fbbfaffd700 1 --2- 172.21.15.32:0/1450648026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbfc0ad250 0x7fbbfc1370e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7fbbec001450 tx=0x7fbbec007ed0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:46.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.287+0000 7fbbebfff700 1 -- 172.21.15.32:0/1450648026 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbec005b70 con 0x7fbbfc0ad250 2020-12-05T17:37:46.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.287+0000 7fbbebfff700 1 -- 172.21.15.32:0/1450648026 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbec005cd0 con 0x7fbbfc0ad250 2020-12-05T17:37:46.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.287+0000 7fbbfbfff700 1 -- 172.21.15.32:0/1450648026 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbfc1339d0 con 0x7fbbfc0ad250 2020-12-05T17:37:46.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.287+0000 7fbbebfff700 1 -- 172.21.15.32:0/1450648026 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbec007300 con 0x7fbbfc0ad250 2020-12-05T17:37:46.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.287+0000 7fbbfbfff700 1 -- 172.21.15.32:0/1450648026 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbfc153020 con 0x7fbbfc0ad250 2020-12-05T17:37:46.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.287+0000 7fbbfbfff700 1 -- 172.21.15.32:0/1450648026 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbfc153670 con 0x7fbbfc0ad250 2020-12-05T17:37:46.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.291+0000 7fbbebfff700 1 -- 172.21.15.32:0/1450648026 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbbec010030 con 0x7fbbfc0ad250 2020-12-05T17:37:46.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.291+0000 7fbbebfff700 1 --2- 172.21.15.32:0/1450648026 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbbd80553b0 0x7fbbd8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:46.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.291+0000 7fbbebfff700 1 -- 172.21.15.32:0/1450648026 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fbbec06e1c0 con 0x7fbbfc0ad250 2020-12-05T17:37:46.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.291+0000 7fbbfa7fc700 1 --2- 172.21.15.32:0/1450648026 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbbd80553b0 0x7fbbd8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:46.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.291+0000 7fbbfa7fc700 1 --2- 172.21.15.32:0/1450648026 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbbd80553b0 0x7fbbd8057870 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7fbbf000b000 tx=0x7fbbf0006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:46.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.295+0000 7fbbebfff700 1 -- 172.21.15.32:0/1450648026 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbbec044340 con 0x7fbbfc0ad250 2020-12-05T17:37:46.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.491+0000 7fbbfbfff700 1 -- 172.21.15.32:0/1450648026 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fbbfc057050 con 0x7fbbfc0ad250 2020-12-05T17:37:46.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.499+0000 7fbbebfff700 1 -- 172.21.15.32:0/1450648026 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 13 and data pool 14 v31) v1 ==== 189+0+0 (secure 0 0 0) 0x7fbbec047960 con 0x7fbbfc0ad250 2020-12-05T17:37:46.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.503+0000 7fbbfbfff700 1 -- 172.21.15.32:0/1450648026 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbbd80553b0 msgr2=0x7fbbd8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:46.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.503+0000 7fbbfbfff700 1 --2- 172.21.15.32:0/1450648026 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbbd80553b0 0x7fbbd8057870 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7fbbf000b000 tx=0x7fbbf0006dc0).stop 2020-12-05T17:37:46.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.503+0000 7fbbfbfff700 1 -- 172.21.15.32:0/1450648026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbfc0ad250 msgr2=0x7fbbfc1370e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:46.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.503+0000 7fbbfbfff700 1 --2- 172.21.15.32:0/1450648026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbfc0ad250 0x7fbbfc1370e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7fbbec001450 tx=0x7fbbec007ed0).stop 2020-12-05T17:37:46.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.503+0000 7fbbfbfff700 1 -- 172.21.15.32:0/1450648026 shutdown_connections 2020-12-05T17:37:46.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.503+0000 7fbbfbfff700 1 --2- 172.21.15.32:0/1450648026 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbbd80553b0 0x7fbbd8057870 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.503+0000 7fbbfbfff700 1 --2- 172.21.15.32:0/1450648026 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbbfc066dd0 0x7fbbfc152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.503+0000 7fbbfbfff700 1 --2- 172.21.15.32:0/1450648026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbfc0ad250 0x7fbbfc1370e0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.503+0000 7fbbfbfff700 1 --2- 172.21.15.32:0/1450648026 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbbfc0b2100 0x7fbbfc135690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.503+0000 7fbbfbfff700 1 -- 172.21.15.32:0/1450648026 >> 172.21.15.32:0/1450648026 conn(0x7fbbfc0620b0 msgr2=0x7fbbfc0b4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:46.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.503+0000 7fbbfbfff700 1 -- 172.21.15.32:0/1450648026 shutdown_connections 2020-12-05T17:37:46.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.503+0000 7fbbfbfff700 1 -- 172.21.15.32:0/1450648026 wait complete. 2020-12-05T17:37:46.509 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 13 and data pool 14 2020-12-05T17:37:46.522 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:37:46.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.619+0000 7fdc6562c700 1 Processor -- start 2020-12-05T17:37:46.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.619+0000 7fdc6562c700 1 -- start start 2020-12-05T17:37:46.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.619+0000 7fdc6562c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fdc600b7720 0x7fdc60061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:46.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.619+0000 7fdc6562c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc601403f0 0x7fdc60143e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:46.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.619+0000 7fdc6562c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc60144320 0x7fdc6013d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:46.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.619+0000 7fdc6562c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc60143a00 con 0x7fdc601403f0 2020-12-05T17:37:46.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.619+0000 7fdc6562c700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc60140850 con 0x7fdc600b7720 2020-12-05T17:37:46.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.619+0000 7fdc6562c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc600b7b00 con 0x7fdc60144320 2020-12-05T17:37:46.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.619+0000 7fdc577fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc601403f0 0x7fdc60143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:46.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc577fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc601403f0 0x7fdc60143e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36616/0 (socket says 172.21.15.32:36616) 2020-12-05T17:37:46.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc577fe700 1 -- 172.21.15.32:0/698870639 learned_addr learned my addr 172.21.15.32:0/698870639 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:46.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc5ffff700 1 --2- 172.21.15.32:0/698870639 >> v2:172.21.15.61:3300/0 conn(0x7fdc600b7720 0x7fdc60061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:46.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc64e2b700 1 --2- 172.21.15.32:0/698870639 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc60144320 0x7fdc6013d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:46.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc577fe700 1 -- 172.21.15.32:0/698870639 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc60144320 msgr2=0x7fdc6013d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:46.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc577fe700 1 --2- 172.21.15.32:0/698870639 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc60144320 0x7fdc6013d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc577fe700 1 -- 172.21.15.32:0/698870639 >> v2:172.21.15.61:3300/0 conn(0x7fdc600b7720 msgr2=0x7fdc60061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:46.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc577fe700 1 --2- 172.21.15.32:0/698870639 >> v2:172.21.15.61:3300/0 conn(0x7fdc600b7720 0x7fdc60061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc577fe700 1 -- 172.21.15.32:0/698870639 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6005fe60 con 0x7fdc601403f0 2020-12-05T17:37:46.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc5ffff700 1 --2- 172.21.15.32:0/698870639 >> v2:172.21.15.61:3300/0 conn(0x7fdc600b7720 0x7fdc60061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:46.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc577fe700 1 --2- 172.21.15.32:0/698870639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc601403f0 0x7fdc60143e30 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7fdc48002f30 tx=0x7fdc48006e80).ready entity=mon.0 client_cookie=a710a977518b99f4 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:46.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc5f7fe700 1 -- 172.21.15.32:0/698870639 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc480097f0 con 0x7fdc601403f0 2020-12-05T17:37:46.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc5f7fe700 1 -- 172.21.15.32:0/698870639 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc48009950 con 0x7fdc601403f0 2020-12-05T17:37:46.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc6562c700 1 -- 172.21.15.32:0/698870639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc601403f0 msgr2=0x7fdc60143e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:46.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc6562c700 1 --2- 172.21.15.32:0/698870639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc601403f0 0x7fdc60143e30 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7fdc48002f30 tx=0x7fdc48006e80).stop 2020-12-05T17:37:46.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc5f7fe700 1 -- 172.21.15.32:0/698870639 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc48009cb0 con 0x7fdc601403f0 2020-12-05T17:37:46.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc6562c700 1 -- 172.21.15.32:0/698870639 shutdown_connections 2020-12-05T17:37:46.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc6562c700 1 --2- 172.21.15.32:0/698870639 >> v2:172.21.15.61:3300/0 conn(0x7fdc600b7720 0x7fdc60061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc6562c700 1 --2- 172.21.15.32:0/698870639 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc60144320 0x7fdc6013d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc6562c700 1 --2- 172.21.15.32:0/698870639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc601403f0 0x7fdc60143e30 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc6562c700 1 -- 172.21.15.32:0/698870639 >> 172.21.15.32:0/698870639 conn(0x7fdc6005bc20 msgr2=0x7fdc600b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:46.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc6562c700 1 -- 172.21.15.32:0/698870639 shutdown_connections 2020-12-05T17:37:46.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc6562c700 1 -- 172.21.15.32:0/698870639 wait complete. 2020-12-05T17:37:46.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc6562c700 1 Processor -- start 2020-12-05T17:37:46.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc6562c700 1 -- start start 2020-12-05T17:37:46.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc6562c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc600b7720 0x7fdc600b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:46.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc6562c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc601403f0 0x7fdc600b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:46.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc6562c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc60144320 0x7fdc60154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:46.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc6562c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc601351d0 con 0x7fdc601403f0 2020-12-05T17:37:46.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc6562c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc600afaa0 con 0x7fdc600b7720 2020-12-05T17:37:46.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc6562c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc600af800 con 0x7fdc60144320 2020-12-05T17:37:46.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc577fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc601403f0 0x7fdc600b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:46.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc577fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc601403f0 0x7fdc600b38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36624/0 (socket says 172.21.15.32:36624) 2020-12-05T17:37:46.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc577fe700 1 -- 172.21.15.32:0/1714101118 learned_addr learned my addr 172.21.15.32:0/1714101118 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:46.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc64e2b700 1 --2- 172.21.15.32:0/1714101118 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc60144320 0x7fdc60154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:46.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.623+0000 7fdc5ffff700 1 --2- 172.21.15.32:0/1714101118 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc600b7720 0x7fdc600b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:46.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.627+0000 7fdc577fe700 1 -- 172.21.15.32:0/1714101118 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc60144320 msgr2=0x7fdc60154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:46.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.627+0000 7fdc577fe700 1 --2- 172.21.15.32:0/1714101118 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc60144320 0x7fdc60154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.627+0000 7fdc577fe700 1 -- 172.21.15.32:0/1714101118 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc600b7720 msgr2=0x7fdc600b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:46.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.627+0000 7fdc577fe700 1 --2- 172.21.15.32:0/1714101118 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc600b7720 0x7fdc600b33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.627+0000 7fdc577fe700 1 -- 172.21.15.32:0/1714101118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc600b2190 con 0x7fdc601403f0 2020-12-05T17:37:46.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.627+0000 7fdc577fe700 1 --2- 172.21.15.32:0/1714101118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc601403f0 0x7fdc600b38c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7fdc48002f30 tx=0x7fdc48015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.627+0000 7fdc5dffb700 1 -- 172.21.15.32:0/1714101118 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc48011ca0 con 0x7fdc601403f0 2020-12-05T17:37:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.627+0000 7fdc6562c700 1 -- 172.21.15.32:0/1714101118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc600ae760 con 0x7fdc601403f0 2020-12-05T17:37:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.627+0000 7fdc5dffb700 1 -- 172.21.15.32:0/1714101118 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc48011e00 con 0x7fdc601403f0 2020-12-05T17:37:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.627+0000 7fdc5dffb700 1 -- 172.21.15.32:0/1714101118 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc4801e650 con 0x7fdc601403f0 2020-12-05T17:37:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.627+0000 7fdc6562c700 1 -- 172.21.15.32:0/1714101118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc600aebf0 con 0x7fdc601403f0 2020-12-05T17:37:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.627+0000 7fdc6562c700 1 -- 172.21.15.32:0/1714101118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc600b2320 con 0x7fdc601403f0 2020-12-05T17:37:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.627+0000 7fdc5dffb700 1 -- 172.21.15.32:0/1714101118 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fdc4801a040 con 0x7fdc601403f0 2020-12-05T17:37:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.627+0000 7fdc5dffb700 1 --2- 172.21.15.32:0/1714101118 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdc44055440 0x7fdc44057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.627+0000 7fdc5dffb700 1 -- 172.21.15.32:0/1714101118 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdc4806f1a0 con 0x7fdc601403f0 2020-12-05T17:37:46.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.627+0000 7fdc5ffff700 1 --2- 172.21.15.32:0/1714101118 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdc44055440 0x7fdc44057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:46.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.627+0000 7fdc5ffff700 1 --2- 172.21.15.32:0/1714101118 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdc44055440 0x7fdc44057900 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7fdc5000c000 tx=0x7fdc50008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:46.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.635+0000 7fdc5dffb700 1 -- 172.21.15.32:0/1714101118 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fdc480452a0 con 0x7fdc601403f0 2020-12-05T17:37:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.779+0000 7fdc6562c700 1 -- 172.21.15.32:0/1714101118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc60057050 con 0x7fdc601403f0 2020-12-05T17:37:46.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.783+0000 7fdc5dffb700 1 -- 172.21.15.32:0/1714101118 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16287 (secure 0 0 0) 0x7fdc48023090 con 0x7fdc601403f0 2020-12-05T17:37:46.787 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:46.787 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":71,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:37:46.495540+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:37:45.088140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-12-05T17:37:45.501261+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:37:46.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.783+0000 7fdc6562c700 1 -- 172.21.15.32:0/1714101118 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdc44055440 msgr2=0x7fdc44057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:46.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.783+0000 7fdc6562c700 1 --2- 172.21.15.32:0/1714101118 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdc44055440 0x7fdc44057900 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7fdc5000c000 tx=0x7fdc50008040).stop 2020-12-05T17:37:46.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.783+0000 7fdc6562c700 1 -- 172.21.15.32:0/1714101118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc601403f0 msgr2=0x7fdc600b38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:46.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.783+0000 7fdc6562c700 1 --2- 172.21.15.32:0/1714101118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc601403f0 0x7fdc600b38c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7fdc48002f30 tx=0x7fdc48015000).stop 2020-12-05T17:37:46.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.783+0000 7fdc6562c700 1 -- 172.21.15.32:0/1714101118 shutdown_connections 2020-12-05T17:37:46.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.787+0000 7fdc6562c700 1 --2- 172.21.15.32:0/1714101118 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdc44055440 0x7fdc44057900 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.787+0000 7fdc6562c700 1 --2- 172.21.15.32:0/1714101118 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc60144320 0x7fdc60154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.787+0000 7fdc6562c700 1 --2- 172.21.15.32:0/1714101118 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc600b7720 0x7fdc600b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.787+0000 7fdc6562c700 1 --2- 172.21.15.32:0/1714101118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc601403f0 0x7fdc600b38c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.787+0000 7fdc6562c700 1 -- 172.21.15.32:0/1714101118 >> 172.21.15.32:0/1714101118 conn(0x7fdc6005bc20 msgr2=0x7fdc601353b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:46.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.787+0000 7fdc6562c700 1 -- 172.21.15.32:0/1714101118 shutdown_connections 2020-12-05T17:37:46.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.787+0000 7fdc6562c700 1 -- 172.21.15.32:0/1714101118 wait complete. 2020-12-05T17:37:46.805 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:37:46.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.915+0000 7f535bfff700 1 Processor -- start 2020-12-05T17:37:46.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.915+0000 7f535bfff700 1 -- start start 2020-12-05T17:37:46.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.915+0000 7f535bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f535c0ae510 0x7f535c145e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:46.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.915+0000 7f535bfff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f535c0afd90 0x7f535c066140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:46.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.915+0000 7f535bfff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f535c0ad9d0 con 0x7f535c05f6f0 2020-12-05T17:37:46.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.915+0000 7f535bfff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f535c0b17a0 con 0x7f535c0afd90 2020-12-05T17:37:46.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.915+0000 7f535bfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535c05fad0 con 0x7f535c0ae510 2020-12-05T17:37:46.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.915+0000 7f535affd700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f535c05f6f0 0x7f535c0ade00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:51918/0 (socket says 172.21.15.32:51918) 2020-12-05T17:37:46.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.915+0000 7f535a7fc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f535c0ae510 0x7f535c145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:46.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.915+0000 7f535affd700 1 -- 172.21.15.32:0/1632125998 learned_addr learned my addr 172.21.15.32:0/1632125998 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:46.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.915+0000 7f535b7fe700 1 --2- 172.21.15.32:0/1632125998 >> v2:172.21.15.61:3300/0 conn(0x7f535c0afd90 0x7f535c066140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:46.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.915+0000 7f535a7fc700 1 -- 172.21.15.32:0/1632125998 >> v2:172.21.15.61:3300/0 conn(0x7f535c0afd90 msgr2=0x7f535c066140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:46.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.915+0000 7f535a7fc700 1 --2- 172.21.15.32:0/1632125998 >> v2:172.21.15.61:3300/0 conn(0x7f535c0afd90 0x7f535c066140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.915+0000 7f535a7fc700 1 -- 172.21.15.32:0/1632125998 >> v1:172.21.15.61:6789/0 conn(0x7f535c05f6f0 legacy=0x7f535c0ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:46.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.915+0000 7f535a7fc700 1 -- 172.21.15.32:0/1632125998 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c0b2c30 con 0x7f535c0ae510 2020-12-05T17:37:46.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.915+0000 7f535a7fc700 1 --2- 172.21.15.32:0/1632125998 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f535c0ae510 0x7f535c145e40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f5350002f30 tx=0x7f5350006dc0).ready entity=mon.2 client_cookie=89cd5c4eeb9b74bb server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:46.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.915+0000 7f5359ffb700 1 -- 172.21.15.32:0/1632125998 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5350009760 con 0x7f535c0ae510 2020-12-05T17:37:46.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.915+0000 7f5359ffb700 1 -- 172.21.15.32:0/1632125998 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53500098c0 con 0x7f535c0ae510 2020-12-05T17:37:46.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f535bfff700 1 -- 172.21.15.32:0/1632125998 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f535c0ae510 msgr2=0x7f535c145e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:46.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f5359ffb700 1 -- 172.21.15.32:0/1632125998 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5350009c20 con 0x7f535c0ae510 2020-12-05T17:37:46.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f535bfff700 1 --2- 172.21.15.32:0/1632125998 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f535c0ae510 0x7f535c145e40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f5350002f30 tx=0x7f5350006dc0).stop 2020-12-05T17:37:46.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f535bfff700 1 -- 172.21.15.32:0/1632125998 shutdown_connections 2020-12-05T17:37:46.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f535bfff700 1 --2- 172.21.15.32:0/1632125998 >> v2:172.21.15.61:3300/0 conn(0x7f535c0afd90 0x7f535c066140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f535bfff700 1 --2- 172.21.15.32:0/1632125998 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f535c0ae510 0x7f535c145e40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f535bfff700 1 -- 172.21.15.32:0/1632125998 >> 172.21.15.32:0/1632125998 conn(0x7f535c0604c0 msgr2=0x7f535c05c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:46.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f535bfff700 1 -- 172.21.15.32:0/1632125998 shutdown_connections 2020-12-05T17:37:46.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f535bfff700 1 -- 172.21.15.32:0/1632125998 wait complete. 2020-12-05T17:37:46.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f535bfff700 1 Processor -- start 2020-12-05T17:37:46.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f535bfff700 1 -- start start 2020-12-05T17:37:46.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f535bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f535c05f6f0 0x7f535c1470e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:46.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f535bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f535c0ae510 0x7f535c142da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:46.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f535bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f535c0afd90 0x7f535c152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:46.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f535bfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c0b5040 con 0x7f535c0ae510 2020-12-05T17:37:46.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f535bfff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c0b60c0 con 0x7f535c0afd90 2020-12-05T17:37:46.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f535bfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535c0b5b80 con 0x7f535c05f6f0 2020-12-05T17:37:46.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f535affd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f535c05f6f0 0x7f535c1470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:46.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f535a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f535c0ae510 0x7f535c142da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:46.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f535b7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f535c0afd90 0x7f535c152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:46.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f535affd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f535c05f6f0 0x7f535c1470e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:33426/0 (socket says 172.21.15.32:33426) 2020-12-05T17:37:46.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f535b7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f535c0afd90 0x7f535c152010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:56530/0 (socket says 172.21.15.32:56530) 2020-12-05T17:37:46.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f535affd700 1 -- 172.21.15.32:0/2679201297 learned_addr learned my addr 172.21.15.32:0/2679201297 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:46.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.919+0000 7f535a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f535c0ae510 0x7f535c142da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36640/0 (socket says 172.21.15.32:36640) 2020-12-05T17:37:46.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.923+0000 7f535a7fc700 1 -- 172.21.15.32:0/2679201297 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f535c05f6f0 msgr2=0x7f535c1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:46.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.923+0000 7f535a7fc700 1 --2- 172.21.15.32:0/2679201297 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f535c05f6f0 0x7f535c1470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.923+0000 7f535a7fc700 1 -- 172.21.15.32:0/2679201297 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f535c0afd90 msgr2=0x7f535c152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:46.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.923+0000 7f535a7fc700 1 --2- 172.21.15.32:0/2679201297 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f535c0afd90 0x7f535c152010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:46.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.923+0000 7f535a7fc700 1 -- 172.21.15.32:0/2679201297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c0b5e20 con 0x7f535c0ae510 2020-12-05T17:37:46.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.923+0000 7f535b7fe700 1 --2- 172.21.15.32:0/2679201297 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f535c0afd90 0x7f535c152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:37:46.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.923+0000 7f535affd700 1 --2- 172.21.15.32:0/2679201297 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f535c05f6f0 0x7f535c1470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:37:46.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.923+0000 7f535a7fc700 1 --2- 172.21.15.32:0/2679201297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f535c0ae510 0x7f535c142da0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f5350002f30 tx=0x7f5350015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:46.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.923+0000 7f534bfff700 1 -- 172.21.15.32:0/2679201297 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53500119a0 con 0x7f535c0ae510 2020-12-05T17:37:46.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.923+0000 7f534bfff700 1 -- 172.21.15.32:0/2679201297 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5350011b00 con 0x7f535c0ae510 2020-12-05T17:37:46.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.923+0000 7f535bfff700 1 -- 172.21.15.32:0/2679201297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535c143290 con 0x7f535c0ae510 2020-12-05T17:37:46.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.923+0000 7f534bfff700 1 -- 172.21.15.32:0/2679201297 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5350011e60 con 0x7f535c0ae510 2020-12-05T17:37:46.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.923+0000 7f535bfff700 1 -- 172.21.15.32:0/2679201297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535c147860 con 0x7f535c0ae510 2020-12-05T17:37:46.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.923+0000 7f534bfff700 1 -- 172.21.15.32:0/2679201297 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5350024030 con 0x7f535c0ae510 2020-12-05T17:37:46.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.923+0000 7f535bfff700 1 -- 172.21.15.32:0/2679201297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535c057050 con 0x7f535c0ae510 2020-12-05T17:37:46.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.923+0000 7f534bfff700 1 --2- 172.21.15.32:0/2679201297 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f53380553b0 0x7f5338057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:46.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.923+0000 7f535affd700 1 --2- 172.21.15.32:0/2679201297 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f53380553b0 0x7f5338057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:46.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.923+0000 7f534bfff700 1 -- 172.21.15.32:0/2679201297 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f535006ee60 con 0x7f535c0ae510 2020-12-05T17:37:46.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.927+0000 7f535affd700 1 --2- 172.21.15.32:0/2679201297 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f53380553b0 0x7f5338057870 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f534c00d000 tx=0x7f534c007c10).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:46.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:46.931+0000 7f534bfff700 1 -- 172.21.15.32:0/2679201297 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5350044ef0 con 0x7f535c0ae510 2020-12-05T17:37:47.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.071+0000 7f535bfff700 1 -- 172.21.15.32:0/2679201297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f535c0b02a0 con 0x7f535c0ae510 2020-12-05T17:37:47.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.075+0000 7f534bfff700 1 -- 172.21.15.32:0/2679201297 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f5350022020 con 0x7f535c0ae510 2020-12-05T17:37:47.079 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:47.080 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":71,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:37:46.495540+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:37:45.088140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-12-05T17:37:45.501261+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:37:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.075+0000 7f535bfff700 1 -- 172.21.15.32:0/2679201297 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f53380553b0 msgr2=0x7f5338057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.075+0000 7f535bfff700 1 --2- 172.21.15.32:0/2679201297 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f53380553b0 0x7f5338057870 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f534c00d000 tx=0x7f534c007c10).stop 2020-12-05T17:37:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.075+0000 7f535bfff700 1 -- 172.21.15.32:0/2679201297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f535c0ae510 msgr2=0x7f535c142da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.075+0000 7f535bfff700 1 --2- 172.21.15.32:0/2679201297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f535c0ae510 0x7f535c142da0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f5350002f30 tx=0x7f5350015000).stop 2020-12-05T17:37:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.075+0000 7f535bfff700 1 -- 172.21.15.32:0/2679201297 shutdown_connections 2020-12-05T17:37:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.075+0000 7f535bfff700 1 --2- 172.21.15.32:0/2679201297 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f535c0afd90 0x7f535c152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.075+0000 7f535bfff700 1 --2- 172.21.15.32:0/2679201297 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f53380553b0 0x7f5338057870 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.075+0000 7f535bfff700 1 --2- 172.21.15.32:0/2679201297 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f535c05f6f0 0x7f535c1470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.075+0000 7f535bfff700 1 --2- 172.21.15.32:0/2679201297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f535c0ae510 0x7f535c142da0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.075+0000 7f535bfff700 1 -- 172.21.15.32:0/2679201297 >> 172.21.15.32:0/2679201297 conn(0x7f535c0604c0 msgr2=0x7f535c148720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:47.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.075+0000 7f535bfff700 1 -- 172.21.15.32:0/2679201297 shutdown_connections 2020-12-05T17:37:47.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.079+0000 7f535bfff700 1 -- 172.21.15.32:0/2679201297 wait complete. 2020-12-05T17:37:47.093 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-12-05T17:37:47.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.211+0000 7fe697fff700 1 Processor -- start 2020-12-05T17:37:47.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.211+0000 7fe697fff700 1 -- start start 2020-12-05T17:37:47.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.215+0000 7fe697fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe698144610 0x7fe69813fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:47.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.215+0000 7fe697fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6981407e0 con 0x7fe6980af5c0 2020-12-05T17:37:47.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.215+0000 7fe697fff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe698144a40 con 0x7fe6980af9a0 2020-12-05T17:37:47.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.215+0000 7fe697fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6981405f0 con 0x7fe698144610 2020-12-05T17:37:47.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.215+0000 7fe6977fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe698144610 0x7fe69813fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:47.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.215+0000 7fe6977fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe698144610 0x7fe69813fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36648/0 (socket says 172.21.15.32:36648) 2020-12-05T17:37:47.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.215+0000 7fe6977fe700 1 -- 172.21.15.32:0/2101352540 learned_addr learned my addr 172.21.15.32:0/2101352540 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:47.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.215+0000 7fe696ffd700 1 --1- 172.21.15.32:0/2101352540 >> v1:172.21.15.32:6789/0 conn(0x7fe6980af5c0 0x7fe698143f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39292/0 (socket says 172.21.15.32:39292) 2020-12-05T17:37:47.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.215+0000 7fe6977fe700 1 -- 172.21.15.32:0/2101352540 >> v1:172.21.15.61:6789/0 conn(0x7fe6980af9a0 legacy=0x7fe69813f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.215+0000 7fe6977fe700 1 -- 172.21.15.32:0/2101352540 >> v1:172.21.15.32:6789/0 conn(0x7fe6980af5c0 legacy=0x7fe698143f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.215+0000 7fe695ffb700 1 -- 172.21.15.32:0/2101352540 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4150044382 0 0) 0x7fe6981407e0 con 0x7fe6980af5c0 2020-12-05T17:37:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.215+0000 7fe6977fe700 1 -- 172.21.15.32:0/2101352540 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe698061b30 con 0x7fe698144610 2020-12-05T17:37:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.215+0000 7fe6977fe700 1 --2- 172.21.15.32:0/2101352540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe698144610 0x7fe69813fcc0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7fe690002f30 tx=0x7fe690006e80).ready entity=mon.0 client_cookie=e9f8a49f5e20f6bd server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.215+0000 7fe695ffb700 1 -- 172.21.15.32:0/2101352540 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6900097f0 con 0x7fe698144610 2020-12-05T17:37:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.215+0000 7fe695ffb700 1 -- 172.21.15.32:0/2101352540 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe690009950 con 0x7fe698144610 2020-12-05T17:37:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.215+0000 7fe695ffb700 1 -- 172.21.15.32:0/2101352540 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe690009cb0 con 0x7fe698144610 2020-12-05T17:37:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.215+0000 7fe697fff700 1 -- 172.21.15.32:0/2101352540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe698144610 msgr2=0x7fe69813fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.215+0000 7fe697fff700 1 --2- 172.21.15.32:0/2101352540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe698144610 0x7fe69813fcc0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7fe690002f30 tx=0x7fe690006e80).stop 2020-12-05T17:37:47.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.215+0000 7fe697fff700 1 -- 172.21.15.32:0/2101352540 shutdown_connections 2020-12-05T17:37:47.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.215+0000 7fe697fff700 1 --2- 172.21.15.32:0/2101352540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe698144610 0x7fe69813fcc0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.215+0000 7fe697fff700 1 -- 172.21.15.32:0/2101352540 >> 172.21.15.32:0/2101352540 conn(0x7fe698061010 msgr2=0x7fe698061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:47.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.215+0000 7fe697fff700 1 -- 172.21.15.32:0/2101352540 shutdown_connections 2020-12-05T17:37:47.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.215+0000 7fe697fff700 1 -- 172.21.15.32:0/2101352540 wait complete. 2020-12-05T17:37:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.219+0000 7fe697fff700 1 Processor -- start 2020-12-05T17:37:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.219+0000 7fe697fff700 1 -- start start 2020-12-05T17:37:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.219+0000 7fe697fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe6980af5c0 0x7fe6980ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.219+0000 7fe697fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6980af9a0 0x7fe698151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:47.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.219+0000 7fe697fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe698144610 0x7fe6981520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:47.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.219+0000 7fe697fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6980adc60 con 0x7fe6980af9a0 2020-12-05T17:37:47.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.219+0000 7fe697fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69813da40 con 0x7fe6980af5c0 2020-12-05T17:37:47.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.219+0000 7fe697fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6980b5730 con 0x7fe698144610 2020-12-05T17:37:47.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.219+0000 7fe6967fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6980af9a0 0x7fe698151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:47.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.219+0000 7fe6967fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6980af9a0 0x7fe698151680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36652/0 (socket says 172.21.15.32:36652) 2020-12-05T17:37:47.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.219+0000 7fe6967fc700 1 -- 172.21.15.32:0/3694515633 learned_addr learned my addr 172.21.15.32:0/3694515633 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:47.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.219+0000 7fe6977fe700 1 --2- 172.21.15.32:0/3694515633 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe698144610 0x7fe6981520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:47.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.219+0000 7fe696ffd700 1 --2- 172.21.15.32:0/3694515633 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe6980af5c0 0x7fe6980ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:47.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.219+0000 7fe6967fc700 1 -- 172.21.15.32:0/3694515633 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe698144610 msgr2=0x7fe6981520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:47.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.219+0000 7fe6967fc700 1 --2- 172.21.15.32:0/3694515633 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe698144610 0x7fe6981520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.219+0000 7fe6967fc700 1 -- 172.21.15.32:0/3694515633 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe6980af5c0 msgr2=0x7fe6980ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:47.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.219+0000 7fe6967fc700 1 --2- 172.21.15.32:0/3694515633 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe6980af5c0 0x7fe6980ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.219+0000 7fe6967fc700 1 -- 172.21.15.32:0/3694515633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6980ad9c0 con 0x7fe6980af9a0 2020-12-05T17:37:47.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.219+0000 7fe6967fc700 1 --2- 172.21.15.32:0/3694515633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6980af9a0 0x7fe698151680 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fe6980b6d30 tx=0x7fe680007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:47.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.219+0000 7fe68ffff700 1 -- 172.21.15.32:0/3694515633 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe680013410 con 0x7fe6980af9a0 2020-12-05T17:37:47.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.219+0000 7fe68ffff700 1 -- 172.21.15.32:0/3694515633 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe680013570 con 0x7fe6980af9a0 2020-12-05T17:37:47.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.223+0000 7fe697fff700 1 -- 172.21.15.32:0/3694515633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6980aeb80 con 0x7fe6980af9a0 2020-12-05T17:37:47.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.223+0000 7fe68ffff700 1 -- 172.21.15.32:0/3694515633 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6800138d0 con 0x7fe6980af9a0 2020-12-05T17:37:47.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.223+0000 7fe697fff700 1 -- 172.21.15.32:0/3694515633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6980ad4b0 con 0x7fe6980af9a0 2020-12-05T17:37:47.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.223+0000 7fe697fff700 1 -- 172.21.15.32:0/3694515633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe698151d10 con 0x7fe6980af9a0 2020-12-05T17:37:47.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.223+0000 7fe68ffff700 1 -- 172.21.15.32:0/3694515633 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe680010030 con 0x7fe6980af9a0 2020-12-05T17:37:47.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.223+0000 7fe68ffff700 1 --2- 172.21.15.32:0/3694515633 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe6700554e0 0x7fe6700579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:47.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.223+0000 7fe696ffd700 1 --2- 172.21.15.32:0/3694515633 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe6700554e0 0x7fe6700579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:47.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.223+0000 7fe68ffff700 1 -- 172.21.15.32:0/3694515633 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe68006df80 con 0x7fe6980af9a0 2020-12-05T17:37:47.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.223+0000 7fe696ffd700 1 --2- 172.21.15.32:0/3694515633 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe6700554e0 0x7fe6700579a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fe68800d000 tx=0x7fe688009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:47.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.231+0000 7fe68ffff700 1 -- 172.21.15.32:0/3694515633 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe680044050 con 0x7fe6980af9a0 2020-12-05T17:37:47.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.415+0000 7fe697fff700 1 -- 172.21.15.32:0/3694515633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe698057050 con 0x7fe6980af9a0 2020-12-05T17:37:47.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.503+0000 7fe68ffff700 1 -- 172.21.15.32:0/3694515633 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe680047670 con 0x7fe6980af9a0 2020-12-05T17:37:47.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.507+0000 7fe697fff700 1 -- 172.21.15.32:0/3694515633 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe6700554e0 msgr2=0x7fe6700579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:47.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.507+0000 7fe697fff700 1 --2- 172.21.15.32:0/3694515633 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe6700554e0 0x7fe6700579a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fe68800d000 tx=0x7fe688009040).stop 2020-12-05T17:37:47.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.507+0000 7fe697fff700 1 -- 172.21.15.32:0/3694515633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6980af9a0 msgr2=0x7fe698151680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:47.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.507+0000 7fe697fff700 1 --2- 172.21.15.32:0/3694515633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6980af9a0 0x7fe698151680 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fe6980b6d30 tx=0x7fe680007c20).stop 2020-12-05T17:37:47.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.507+0000 7fe697fff700 1 -- 172.21.15.32:0/3694515633 shutdown_connections 2020-12-05T17:37:47.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.507+0000 7fe697fff700 1 --2- 172.21.15.32:0/3694515633 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe6700554e0 0x7fe6700579a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.507+0000 7fe697fff700 1 --2- 172.21.15.32:0/3694515633 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe698144610 0x7fe6981520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.507+0000 7fe697fff700 1 --2- 172.21.15.32:0/3694515633 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe6980af5c0 0x7fe6980ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.507+0000 7fe697fff700 1 --2- 172.21.15.32:0/3694515633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6980af9a0 0x7fe698151680 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.507+0000 7fe697fff700 1 -- 172.21.15.32:0/3694515633 >> 172.21.15.32:0/3694515633 conn(0x7fe698061010 msgr2=0x7fe6981433a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:47.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.507+0000 7fe697fff700 1 -- 172.21.15.32:0/3694515633 shutdown_connections 2020-12-05T17:37:47.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.507+0000 7fe697fff700 1 -- 172.21.15.32:0/3694515633 wait complete. 2020-12-05T17:37:47.527 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:37:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.635+0000 7fad66004700 1 Processor -- start 2020-12-05T17:37:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.635+0000 7fad66004700 1 -- start start 2020-12-05T17:37:47.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.639+0000 7fad66004700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad600b5f00 0x7fad600b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:47.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.639+0000 7fad66004700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fad600ad510 0x7fad600af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:47.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.639+0000 7fad66004700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad60142ab0 con 0x7fad60066370 2020-12-05T17:37:47.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.639+0000 7fad66004700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad600b62e0 con 0x7fad600ad510 2020-12-05T17:37:47.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.639+0000 7fad66004700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad600afbf0 con 0x7fad600b5f00 2020-12-05T17:37:47.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.639+0000 7fad65002700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad600b5f00 0x7fad600b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:47.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.639+0000 7fad65803700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fad60066370 0x7fad60142ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:51950/0 (socket says 172.21.15.32:51950) 2020-12-05T17:37:47.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.639+0000 7fad65803700 1 -- 172.21.15.32:0/3110060381 learned_addr learned my addr 172.21.15.32:0/3110060381 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:47.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.639+0000 7fad64801700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fad600ad510 0x7fad600af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:47.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.639+0000 7fad4ffff700 1 -- 172.21.15.32:0/3110060381 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3786707915 0 0) 0x7fad60142ab0 con 0x7fad60066370 2020-12-05T17:37:47.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.639+0000 7fad4ffff700 1 -- 172.21.15.32:0/3110060381 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad48003230 con 0x7fad60066370 2020-12-05T17:37:47.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.639+0000 7fad65002700 1 -- 172.21.15.32:0/3110060381 >> v2:172.21.15.61:3300/0 conn(0x7fad600ad510 msgr2=0x7fad600af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:47.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.639+0000 7fad65002700 1 --2- 172.21.15.32:0/3110060381 >> v2:172.21.15.61:3300/0 conn(0x7fad600ad510 0x7fad600af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.639+0000 7fad65002700 1 -- 172.21.15.32:0/3110060381 >> v1:172.21.15.61:6789/0 conn(0x7fad60066370 legacy=0x7fad60142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:47.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.639+0000 7fad65002700 1 -- 172.21.15.32:0/3110060381 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad600afe90 con 0x7fad600b5f00 2020-12-05T17:37:47.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.639+0000 7fad65002700 1 --2- 172.21.15.32:0/3110060381 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad600b5f00 0x7fad600b14e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fad5c002f30 tx=0x7fad5c006dc0).ready entity=mon.2 client_cookie=190fb572c864a79a server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:47.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.639+0000 7fad4ffff700 1 -- 172.21.15.32:0/3110060381 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad5c0096a0 con 0x7fad600b5f00 2020-12-05T17:37:47.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.639+0000 7fad4ffff700 1 -- 172.21.15.32:0/3110060381 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5c009800 con 0x7fad600b5f00 2020-12-05T17:37:47.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.639+0000 7fad4ffff700 1 -- 172.21.15.32:0/3110060381 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad5c009b60 con 0x7fad600b5f00 2020-12-05T17:37:47.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.639+0000 7fad66004700 1 -- 172.21.15.32:0/3110060381 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad600b5f00 msgr2=0x7fad600b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:47.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.639+0000 7fad66004700 1 --2- 172.21.15.32:0/3110060381 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad600b5f00 0x7fad600b14e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fad5c002f30 tx=0x7fad5c006dc0).stop 2020-12-05T17:37:47.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.643+0000 7fad66004700 1 -- 172.21.15.32:0/3110060381 shutdown_connections 2020-12-05T17:37:47.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.643+0000 7fad66004700 1 --2- 172.21.15.32:0/3110060381 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad600b5f00 0x7fad600b14e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.643+0000 7fad66004700 1 --2- 172.21.15.32:0/3110060381 >> v2:172.21.15.61:3300/0 conn(0x7fad600ad510 0x7fad600af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.643+0000 7fad66004700 1 -- 172.21.15.32:0/3110060381 >> 172.21.15.32:0/3110060381 conn(0x7fad600604c0 msgr2=0x7fad6005c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:47.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.643+0000 7fad66004700 1 -- 172.21.15.32:0/3110060381 shutdown_connections 2020-12-05T17:37:47.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.643+0000 7fad66004700 1 -- 172.21.15.32:0/3110060381 wait complete. 2020-12-05T17:37:47.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.643+0000 7fad66004700 1 Processor -- start 2020-12-05T17:37:47.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.643+0000 7fad66004700 1 -- start start 2020-12-05T17:37:47.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.647+0000 7fad66004700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fad60066370 0x7fad600632e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:47.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.647+0000 7fad66004700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad600ad510 0x7fad60152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:47.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.647+0000 7fad66004700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad600b5f00 0x7fad60134690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:47.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.647+0000 7fad66004700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad60152e30 con 0x7fad600b5f00 2020-12-05T17:37:47.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.647+0000 7fad66004700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad60153080 con 0x7fad60066370 2020-12-05T17:37:47.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.647+0000 7fad66004700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad60153320 con 0x7fad600ad510 2020-12-05T17:37:47.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.647+0000 7fad65803700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad600b5f00 0x7fad60134690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:47.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.647+0000 7fad65803700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad600b5f00 0x7fad60134690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36668/0 (socket says 172.21.15.32:36668) 2020-12-05T17:37:47.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.647+0000 7fad65803700 1 -- 172.21.15.32:0/1975371963 learned_addr learned my addr 172.21.15.32:0/1975371963 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:47.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.647+0000 7fad65002700 1 --2- 172.21.15.32:0/1975371963 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fad60066370 0x7fad600632e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:47.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.647+0000 7fad64801700 1 --2- 172.21.15.32:0/1975371963 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad600ad510 0x7fad60152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:47.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.647+0000 7fad65803700 1 -- 172.21.15.32:0/1975371963 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad600ad510 msgr2=0x7fad60152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:47.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.647+0000 7fad65803700 1 --2- 172.21.15.32:0/1975371963 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad600ad510 0x7fad60152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.647+0000 7fad65803700 1 -- 172.21.15.32:0/1975371963 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fad60066370 msgr2=0x7fad600632e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:47.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.647+0000 7fad65803700 1 --2- 172.21.15.32:0/1975371963 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fad60066370 0x7fad600632e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.647+0000 7fad65803700 1 -- 172.21.15.32:0/1975371963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad601535c0 con 0x7fad600b5f00 2020-12-05T17:37:47.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.647+0000 7fad65803700 1 --2- 172.21.15.32:0/1975371963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad600b5f00 0x7fad60134690 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7fad58001450 tx=0x7fad58007e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:47.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.647+0000 7fad4e7fc700 1 -- 172.21.15.32:0/1975371963 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad58005b70 con 0x7fad600b5f00 2020-12-05T17:37:47.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.647+0000 7fad4e7fc700 1 -- 172.21.15.32:0/1975371963 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad58005cd0 con 0x7fad600b5f00 2020-12-05T17:37:47.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.647+0000 7fad4e7fc700 1 -- 172.21.15.32:0/1975371963 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad58007300 con 0x7fad600b5f00 2020-12-05T17:37:47.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.647+0000 7fad66004700 1 -- 172.21.15.32:0/1975371963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6014f280 con 0x7fad600b5f00 2020-12-05T17:37:47.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.647+0000 7fad66004700 1 -- 172.21.15.32:0/1975371963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad60145280 con 0x7fad600b5f00 2020-12-05T17:37:47.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.651+0000 7fad66004700 1 -- 172.21.15.32:0/1975371963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad60057050 con 0x7fad600b5f00 2020-12-05T17:37:47.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.651+0000 7fad4e7fc700 1 -- 172.21.15.32:0/1975371963 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fad58010030 con 0x7fad600b5f00 2020-12-05T17:37:47.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.651+0000 7fad4e7fc700 1 --2- 172.21.15.32:0/1975371963 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fad40055500 0x7fad400579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:47.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.651+0000 7fad4e7fc700 1 -- 172.21.15.32:0/1975371963 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fad5806e1c0 con 0x7fad600b5f00 2020-12-05T17:37:47.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.651+0000 7fad65002700 1 --2- 172.21.15.32:0/1975371963 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fad40055500 0x7fad400579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:47.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.651+0000 7fad65002700 1 --2- 172.21.15.32:0/1975371963 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fad40055500 0x7fad400579c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fad5c01b000 tx=0x7fad5c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:47.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.659+0000 7fad4e7fc700 1 -- 172.21.15.32:0/1975371963 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fad580441f0 con 0x7fad600b5f00 2020-12-05T17:37:47.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.843+0000 7fad66004700 1 -- 172.21.15.32:0/1975371963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad600b12c0 con 0x7fad600b5f00 2020-12-05T17:37:47.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.843+0000 7fad4e7fc700 1 -- 172.21.15.32:0/1975371963 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+2137 (secure 0 0 0) 0x7fad58047810 con 0x7fad600b5f00 2020-12-05T17:37:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:47.849 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":32,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5645,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/992428276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":992428276},{"type":"v1","addr":"172.21.15.32:6835","nonce":992428276}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:37:46.495876+0000","modified":"2020-12-05T17:37:47.498743+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6160":{"gid":6160,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.61:6833/4214217205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4214217205},{"type":"v1","addr":"172.21.15.61:6833","nonce":4214217205}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-12-05T17:37:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.847+0000 7fad66004700 1 -- 172.21.15.32:0/1975371963 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fad40055500 msgr2=0x7fad400579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.847+0000 7fad66004700 1 --2- 172.21.15.32:0/1975371963 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fad40055500 0x7fad400579c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fad5c01b000 tx=0x7fad5c015000).stop 2020-12-05T17:37:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.847+0000 7fad66004700 1 -- 172.21.15.32:0/1975371963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad600b5f00 msgr2=0x7fad60134690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.847+0000 7fad66004700 1 --2- 172.21.15.32:0/1975371963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad600b5f00 0x7fad60134690 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7fad58001450 tx=0x7fad58007e40).stop 2020-12-05T17:37:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.847+0000 7fad66004700 1 -- 172.21.15.32:0/1975371963 shutdown_connections 2020-12-05T17:37:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.847+0000 7fad66004700 1 --2- 172.21.15.32:0/1975371963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad600b5f00 0x7fad60134690 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.847+0000 7fad66004700 1 --2- 172.21.15.32:0/1975371963 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fad60066370 0x7fad600632e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.847+0000 7fad66004700 1 --2- 172.21.15.32:0/1975371963 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fad40055500 0x7fad400579c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.847+0000 7fad66004700 1 --2- 172.21.15.32:0/1975371963 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad600ad510 0x7fad60152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.847+0000 7fad66004700 1 -- 172.21.15.32:0/1975371963 >> 172.21.15.32:0/1975371963 conn(0x7fad600604c0 msgr2=0x7fad6013dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.847+0000 7fad66004700 1 -- 172.21.15.32:0/1975371963 shutdown_connections 2020-12-05T17:37:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.847+0000 7fad66004700 1 -- 172.21.15.32:0/1975371963 wait complete. 2020-12-05T17:37:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 32 2020-12-05T17:37:47.866 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:37:47.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.975+0000 7f9e018e9700 1 Processor -- start 2020-12-05T17:37:47.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.975+0000 7f9e018e9700 1 -- start start 2020-12-05T17:37:47.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.975+0000 7f9e018e9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9dfc0b7720 0x7f9dfc061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:47.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.975+0000 7f9e018e9700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9dfc1403f0 0x7f9dfc143e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:47.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.975+0000 7f9e018e9700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dfc143a00 con 0x7f9dfc144320 2020-12-05T17:37:47.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.975+0000 7f9e018e9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dfc0b7b00 con 0x7f9dfc0b7720 2020-12-05T17:37:47.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.975+0000 7f9e018e9700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dfc05fe60 con 0x7f9dfc1403f0 2020-12-05T17:37:47.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.975+0000 7f9e008e7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9dfc0b7720 0x7f9dfc061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:47.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.975+0000 7f9e008e7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9dfc0b7720 0x7f9dfc061780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36674/0 (socket says 172.21.15.32:36674) 2020-12-05T17:37:47.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.975+0000 7f9e008e7700 1 -- 172.21.15.32:0/2098889452 learned_addr learned my addr 172.21.15.32:0/2098889452 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.975+0000 7f9dfbfff700 1 --2- 172.21.15.32:0/2098889452 >> v2:172.21.15.61:3300/0 conn(0x7f9dfc1403f0 0x7f9dfc143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.975+0000 7f9e010e8700 1 --1- 172.21.15.32:0/2098889452 >> v1:172.21.15.61:6789/0 conn(0x7f9dfc144320 0x7f9dfc13d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:51960/0 (socket says 172.21.15.32:51960) 2020-12-05T17:37:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.979+0000 7f9e008e7700 1 -- 172.21.15.32:0/2098889452 >> v2:172.21.15.61:3300/0 conn(0x7f9dfc1403f0 msgr2=0x7f9dfc143e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.979+0000 7f9e008e7700 1 --2- 172.21.15.32:0/2098889452 >> v2:172.21.15.61:3300/0 conn(0x7f9dfc1403f0 0x7f9dfc143e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.979+0000 7f9e008e7700 1 -- 172.21.15.32:0/2098889452 >> v1:172.21.15.61:6789/0 conn(0x7f9dfc144320 legacy=0x7f9dfc13d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.979+0000 7f9e008e7700 1 -- 172.21.15.32:0/2098889452 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc140850 con 0x7f9dfc0b7720 2020-12-05T17:37:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.979+0000 7f9dfbfff700 1 --2- 172.21.15.32:0/2098889452 >> v2:172.21.15.61:3300/0 conn(0x7f9dfc1403f0 0x7f9dfc143e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:37:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.979+0000 7f9e008e7700 1 --2- 172.21.15.32:0/2098889452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dfc0b7720 0x7f9dfc061780 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f9dec002f30 tx=0x7f9dec006e80).ready entity=mon.0 client_cookie=5a0e1b25bff9adf5 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.979+0000 7f9dfb7fe700 1 -- 172.21.15.32:0/2098889452 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dec0097f0 con 0x7f9dfc0b7720 2020-12-05T17:37:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.979+0000 7f9dfb7fe700 1 -- 172.21.15.32:0/2098889452 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dec009950 con 0x7f9dfc0b7720 2020-12-05T17:37:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.979+0000 7f9dfb7fe700 1 -- 172.21.15.32:0/2098889452 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dec009cb0 con 0x7f9dfc0b7720 2020-12-05T17:37:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.979+0000 7f9e018e9700 1 -- 172.21.15.32:0/2098889452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dfc0b7720 msgr2=0x7f9dfc061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.979+0000 7f9e018e9700 1 --2- 172.21.15.32:0/2098889452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dfc0b7720 0x7f9dfc061780 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f9dec002f30 tx=0x7f9dec006e80).stop 2020-12-05T17:37:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.979+0000 7f9e018e9700 1 -- 172.21.15.32:0/2098889452 shutdown_connections 2020-12-05T17:37:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.979+0000 7f9e018e9700 1 --2- 172.21.15.32:0/2098889452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dfc0b7720 0x7f9dfc061780 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.979+0000 7f9e018e9700 1 --2- 172.21.15.32:0/2098889452 >> v2:172.21.15.61:3300/0 conn(0x7f9dfc1403f0 0x7f9dfc143e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.979+0000 7f9e018e9700 1 -- 172.21.15.32:0/2098889452 >> 172.21.15.32:0/2098889452 conn(0x7f9dfc05bc20 msgr2=0x7f9dfc0b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.979+0000 7f9e018e9700 1 -- 172.21.15.32:0/2098889452 shutdown_connections 2020-12-05T17:37:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.979+0000 7f9e018e9700 1 -- 172.21.15.32:0/2098889452 wait complete. 2020-12-05T17:37:47.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.979+0000 7f9e018e9700 1 Processor -- start 2020-12-05T17:37:47.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.979+0000 7f9e018e9700 1 -- start start 2020-12-05T17:37:47.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9e018e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dfc0b7720 0x7f9dfc146760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:47.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9e018e9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9dfc1403f0 0x7f9dfc063380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:47.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9e018e9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9dfc144320 0x7f9dfc13b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:47.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9e018e9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc145df0 con 0x7f9dfc0b7720 2020-12-05T17:37:47.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9e018e9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc146020 con 0x7f9dfc144320 2020-12-05T17:37:47.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9e018e9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc1462c0 con 0x7f9dfc1403f0 2020-12-05T17:37:47.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9e008e7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dfc0b7720 0x7f9dfc146760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:47.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9e008e7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dfc0b7720 0x7f9dfc146760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36682/0 (socket says 172.21.15.32:36682) 2020-12-05T17:37:47.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9e008e7700 1 -- 172.21.15.32:0/684098492 learned_addr learned my addr 172.21.15.32:0/684098492 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:47.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9e010e8700 1 --2- 172.21.15.32:0/684098492 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9dfc144320 0x7f9dfc13b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:47.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9dfbfff700 1 --2- 172.21.15.32:0/684098492 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9dfc1403f0 0x7f9dfc063380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:47.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9e008e7700 1 -- 172.21.15.32:0/684098492 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9dfc1403f0 msgr2=0x7f9dfc063380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:47.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9e008e7700 1 --2- 172.21.15.32:0/684098492 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9dfc1403f0 0x7f9dfc063380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9e008e7700 1 -- 172.21.15.32:0/684098492 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9dfc144320 msgr2=0x7f9dfc13b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:47.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9e008e7700 1 --2- 172.21.15.32:0/684098492 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9dfc144320 0x7f9dfc13b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:47.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9e008e7700 1 -- 172.21.15.32:0/684098492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc0b3cc0 con 0x7f9dfc0b7720 2020-12-05T17:37:47.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9e008e7700 1 --2- 172.21.15.32:0/684098492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dfc0b7720 0x7f9dfc146760 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f9dec002f30 tx=0x7f9dec015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:47.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9df9ffb700 1 -- 172.21.15.32:0/684098492 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dec011b80 con 0x7f9dfc0b7720 2020-12-05T17:37:47.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9df9ffb700 1 -- 172.21.15.32:0/684098492 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dec011ce0 con 0x7f9dfc0b7720 2020-12-05T17:37:47.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9e018e9700 1 -- 172.21.15.32:0/684098492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dfc13b6b0 con 0x7f9dfc0b7720 2020-12-05T17:37:47.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9df9ffb700 1 -- 172.21.15.32:0/684098492 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dec01e650 con 0x7f9dfc0b7720 2020-12-05T17:37:47.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9e018e9700 1 -- 172.21.15.32:0/684098492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dfc1395b0 con 0x7f9dfc0b7720 2020-12-05T17:37:47.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9e018e9700 1 -- 172.21.15.32:0/684098492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dfc146480 con 0x7f9dfc0b7720 2020-12-05T17:37:47.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.983+0000 7f9df9ffb700 1 -- 172.21.15.32:0/684098492 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9dec025030 con 0x7f9dfc0b7720 2020-12-05T17:37:47.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.987+0000 7f9df9ffb700 1 --2- 172.21.15.32:0/684098492 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9ddc055440 0x7f9ddc057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:47.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.987+0000 7f9df9ffb700 1 -- 172.21.15.32:0/684098492 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9dec06f0b0 con 0x7f9dfc0b7720 2020-12-05T17:37:47.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.987+0000 7f9dfbfff700 1 --2- 172.21.15.32:0/684098492 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9ddc055440 0x7f9ddc057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:47.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.987+0000 7f9dfbfff700 1 --2- 172.21.15.32:0/684098492 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9ddc055440 0x7f9ddc057900 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f9df000b000 tx=0x7f9df0006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:47.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:47.991+0000 7f9df9ffb700 1 -- 172.21.15.32:0/684098492 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9dec045250 con 0x7f9dfc0b7720 2020-12-05T17:37:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.135+0000 7f9e018e9700 1 -- 172.21.15.32:0/684098492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9dfc057050 con 0x7f9dfc0b7720 2020-12-05T17:37:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.135+0000 7f9df9ffb700 1 -- 172.21.15.32:0/684098492 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f9dec023070 con 0x7f9dfc0b7720 2020-12-05T17:37:48.140 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:48.140 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":72,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:37:47.499387+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:37:45.088140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-12-05T17:37:45.501261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:37:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.135+0000 7f9e018e9700 1 -- 172.21.15.32:0/684098492 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9ddc055440 msgr2=0x7f9ddc057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.135+0000 7f9e018e9700 1 --2- 172.21.15.32:0/684098492 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9ddc055440 0x7f9ddc057900 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f9df000b000 tx=0x7f9df0006dc0).stop 2020-12-05T17:37:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.135+0000 7f9e018e9700 1 -- 172.21.15.32:0/684098492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dfc0b7720 msgr2=0x7f9dfc146760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:48.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.135+0000 7f9e018e9700 1 --2- 172.21.15.32:0/684098492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dfc0b7720 0x7f9dfc146760 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f9dec002f30 tx=0x7f9dec015000).stop 2020-12-05T17:37:48.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.135+0000 7f9e018e9700 1 -- 172.21.15.32:0/684098492 shutdown_connections 2020-12-05T17:37:48.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.135+0000 7f9e018e9700 1 --2- 172.21.15.32:0/684098492 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9dfc144320 0x7f9dfc13b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:48.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.135+0000 7f9e018e9700 1 --2- 172.21.15.32:0/684098492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dfc0b7720 0x7f9dfc146760 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.135+0000 7f9e018e9700 1 --2- 172.21.15.32:0/684098492 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9ddc055440 0x7f9ddc057900 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.135+0000 7f9e018e9700 1 --2- 172.21.15.32:0/684098492 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9dfc1403f0 0x7f9dfc063380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.135+0000 7f9e018e9700 1 -- 172.21.15.32:0/684098492 >> 172.21.15.32:0/684098492 conn(0x7f9dfc05bc20 msgr2=0x7f9dfc061780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.139+0000 7f9e018e9700 1 -- 172.21.15.32:0/684098492 shutdown_connections 2020-12-05T17:37:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.139+0000 7f9e018e9700 1 -- 172.21.15.32:0/684098492 wait complete. 2020-12-05T17:37:48.160 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:37:48.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.279+0000 7f05aea74700 1 Processor -- start 2020-12-05T17:37:48.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.279+0000 7f05aea74700 1 -- start start 2020-12-05T17:37:48.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.279+0000 7f05aea74700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f05a805feb0 0x7f05a80ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:48.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.279+0000 7f05aea74700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a80b0380 con 0x7f05a805f6f0 2020-12-05T17:37:48.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.279+0000 7f05aea74700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a805fad0 con 0x7f05a8061780 2020-12-05T17:37:48.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.279+0000 7f05aea74700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80aeaf0 con 0x7f05a805feb0 2020-12-05T17:37:48.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.279+0000 7f05ad011700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f05a805f6f0 0x7f05a80adef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39334/0 (socket says 172.21.15.32:39334) 2020-12-05T17:37:48.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.279+0000 7f05ad011700 1 -- 172.21.15.32:0/1713014764 learned_addr learned my addr 172.21.15.32:0/1713014764 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:48.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.279+0000 7f05ac810700 1 --2- 172.21.15.32:0/1713014764 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f05a805feb0 0x7f05a80ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:48.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.279+0000 7f05a7fff700 1 -- 172.21.15.32:0/1713014764 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2395100369 0 0) 0x7f05a80b0380 con 0x7f05a805f6f0 2020-12-05T17:37:48.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.279+0000 7f05a7fff700 1 -- 172.21.15.32:0/1713014764 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0590003230 con 0x7f05a805f6f0 2020-12-05T17:37:48.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.279+0000 7f05a7fff700 1 -- 172.21.15.32:0/1713014764 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2550961489 0 0) 0x7f05a805fad0 con 0x7f05a8061780 2020-12-05T17:37:48.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.279+0000 7f05ac810700 1 -- 172.21.15.32:0/1713014764 >> v1:172.21.15.61:6789/0 conn(0x7f05a8061780 legacy=0x7f05a814e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:48.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.279+0000 7f05ac810700 1 -- 172.21.15.32:0/1713014764 >> v1:172.21.15.32:6789/0 conn(0x7f05a805f6f0 legacy=0x7f05a80adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:48.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.279+0000 7f05ac810700 1 -- 172.21.15.32:0/1713014764 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a80aecd0 con 0x7f05a805feb0 2020-12-05T17:37:48.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.279+0000 7f05ac810700 1 --2- 172.21.15.32:0/1713014764 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f05a805feb0 0x7f05a80ae600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f059c002f30 tx=0x7f059c006dc0).ready entity=mon.2 client_cookie=e36262df130c6cd6 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:48.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.279+0000 7f05a7fff700 1 -- 172.21.15.32:0/1713014764 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f059c0097b0 con 0x7f05a805feb0 2020-12-05T17:37:48.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.283+0000 7f05a7fff700 1 -- 172.21.15.32:0/1713014764 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059c009910 con 0x7f05a805feb0 2020-12-05T17:37:48.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.283+0000 7f05a7fff700 1 -- 172.21.15.32:0/1713014764 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f059c009c70 con 0x7f05a805feb0 2020-12-05T17:37:48.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.283+0000 7f05aea74700 1 -- 172.21.15.32:0/1713014764 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f05a805feb0 msgr2=0x7f05a80ae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:48.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.283+0000 7f05aea74700 1 --2- 172.21.15.32:0/1713014764 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f05a805feb0 0x7f05a80ae600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f059c002f30 tx=0x7f059c006dc0).stop 2020-12-05T17:37:48.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.283+0000 7f05aea74700 1 -- 172.21.15.32:0/1713014764 shutdown_connections 2020-12-05T17:37:48.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.283+0000 7f05aea74700 1 --2- 172.21.15.32:0/1713014764 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f05a805feb0 0x7f05a80ae600 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:48.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.283+0000 7f05aea74700 1 -- 172.21.15.32:0/1713014764 >> 172.21.15.32:0/1713014764 conn(0x7f05a80b4b10 msgr2=0x7f05a80b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:48.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.283+0000 7f05aea74700 1 -- 172.21.15.32:0/1713014764 shutdown_connections 2020-12-05T17:37:48.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.283+0000 7f05aea74700 1 -- 172.21.15.32:0/1713014764 wait complete. 2020-12-05T17:37:48.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.283+0000 7f05aea74700 1 Processor -- start 2020-12-05T17:37:48.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.283+0000 7f05aea74700 1 -- start start 2020-12-05T17:37:48.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05aea74700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f05a805f6f0 0x7f05a8065250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:48.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05aea74700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f05a805feb0 0x7f05a814e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:48.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05aea74700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a8061780 0x7f05a8073170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:48.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05aea74700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a8072d40 con 0x7f05a8061780 2020-12-05T17:37:48.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05aea74700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80669e0 con 0x7f05a805feb0 2020-12-05T17:37:48.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05aea74700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a8066c80 con 0x7f05a805f6f0 2020-12-05T17:37:48.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05ad812700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a8061780 0x7f05a8073170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:48.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05ad812700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a8061780 0x7f05a8073170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36696/0 (socket says 172.21.15.32:36696) 2020-12-05T17:37:48.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05ad812700 1 -- 172.21.15.32:0/2884046872 learned_addr learned my addr 172.21.15.32:0/2884046872 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:48.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05ad011700 1 --2- 172.21.15.32:0/2884046872 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f05a805f6f0 0x7f05a8065250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:48.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05ac810700 1 --2- 172.21.15.32:0/2884046872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f05a805feb0 0x7f05a814e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:48.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05ad812700 1 -- 172.21.15.32:0/2884046872 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f05a805f6f0 msgr2=0x7f05a8065250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:48.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05ad812700 1 --2- 172.21.15.32:0/2884046872 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f05a805f6f0 0x7f05a8065250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:48.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05ad812700 1 -- 172.21.15.32:0/2884046872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f05a805feb0 msgr2=0x7f05a814e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:48.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05ad812700 1 --2- 172.21.15.32:0/2884046872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f05a805feb0 0x7f05a814e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:48.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05ad812700 1 -- 172.21.15.32:0/2884046872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a814eb30 con 0x7f05a8061780 2020-12-05T17:37:48.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05ad812700 1 --2- 172.21.15.32:0/2884046872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a8061780 0x7f05a8073170 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7f05a0001320 tx=0x7f05a0007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:48.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05a5ffb700 1 -- 172.21.15.32:0/2884046872 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05a000a7b0 con 0x7f05a8061780 2020-12-05T17:37:48.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05a5ffb700 1 -- 172.21.15.32:0/2884046872 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a000a910 con 0x7f05a8061780 2020-12-05T17:37:48.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05aea74700 1 -- 172.21.15.32:0/2884046872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a8066660 con 0x7f05a8061780 2020-12-05T17:37:48.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05a5ffb700 1 -- 172.21.15.32:0/2884046872 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05a000ac70 con 0x7f05a8061780 2020-12-05T17:37:48.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05aea74700 1 -- 172.21.15.32:0/2884046872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a814e8c0 con 0x7f05a8061780 2020-12-05T17:37:48.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05aea74700 1 -- 172.21.15.32:0/2884046872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a8065740 con 0x7f05a8061780 2020-12-05T17:37:48.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.287+0000 7f05a5ffb700 1 -- 172.21.15.32:0/2884046872 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f05a0019030 con 0x7f05a8061780 2020-12-05T17:37:48.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.291+0000 7f05a5ffb700 1 --2- 172.21.15.32:0/2884046872 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0588055320 0x7f05880577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:48.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.291+0000 7f05ad011700 1 --2- 172.21.15.32:0/2884046872 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0588055320 0x7f05880577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:48.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.291+0000 7f05a5ffb700 1 -- 172.21.15.32:0/2884046872 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f05a006da20 con 0x7f05a8061780 2020-12-05T17:37:48.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.291+0000 7f05ad011700 1 --2- 172.21.15.32:0/2884046872 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0588055320 0x7f05880577e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f059800b000 tx=0x7f0598009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:48.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.295+0000 7f05a5ffb700 1 -- 172.21.15.32:0/2884046872 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f05a0041df0 con 0x7f05a8061780 2020-12-05T17:37:48.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.483+0000 7f05aea74700 1 -- 172.21.15.32:0/2884046872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f05a8057050 con 0x7f05a8061780 2020-12-05T17:37:48.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.491+0000 7f05a5ffb700 1 -- 172.21.15.32:0/2884046872 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v23) v1 ==== 184+0+0 (secure 0 0 0) 0x7f05a0043b60 con 0x7f05a8061780 2020-12-05T17:37:48.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.495+0000 7f05aea74700 1 -- 172.21.15.32:0/2884046872 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0588055320 msgr2=0x7f05880577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.495+0000 7f05aea74700 1 --2- 172.21.15.32:0/2884046872 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0588055320 0x7f05880577e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f059800b000 tx=0x7f0598009040).stop 2020-12-05T17:37:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.495+0000 7f05aea74700 1 -- 172.21.15.32:0/2884046872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a8061780 msgr2=0x7f05a8073170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.495+0000 7f05aea74700 1 --2- 172.21.15.32:0/2884046872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a8061780 0x7f05a8073170 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7f05a0001320 tx=0x7f05a0007f10).stop 2020-12-05T17:37:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.495+0000 7f05aea74700 1 -- 172.21.15.32:0/2884046872 shutdown_connections 2020-12-05T17:37:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.495+0000 7f05aea74700 1 --2- 172.21.15.32:0/2884046872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a8061780 0x7f05a8073170 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:48.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.495+0000 7f05aea74700 1 --2- 172.21.15.32:0/2884046872 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0588055320 0x7f05880577e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:48.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.495+0000 7f05aea74700 1 --2- 172.21.15.32:0/2884046872 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f05a805f6f0 0x7f05a8065250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:48.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.495+0000 7f05aea74700 1 --2- 172.21.15.32:0/2884046872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f05a805feb0 0x7f05a814e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:48.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.495+0000 7f05aea74700 1 -- 172.21.15.32:0/2884046872 >> 172.21.15.32:0/2884046872 conn(0x7f05a80b4b10 msgr2=0x7f05a80b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:48.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.495+0000 7f05aea74700 1 -- 172.21.15.32:0/2884046872 shutdown_connections 2020-12-05T17:37:48.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.495+0000 7f05aea74700 1 -- 172.21.15.32:0/2884046872 wait complete. 2020-12-05T17:37:48.501 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2020-12-05T17:37:48.515 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:37:48.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.631+0000 7fa983fff700 1 Processor -- start 2020-12-05T17:37:48.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.631+0000 7fa983fff700 1 -- start start 2020-12-05T17:37:48.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.631+0000 7fa983fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9840b0240 0x7fa984153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:48.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.631+0000 7fa983fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9840ae860 0x7fa98414f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:48.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.631+0000 7fa983fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa984080a90 con 0x7fa9840611b0 2020-12-05T17:37:48.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.631+0000 7fa983fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9840aecb0 con 0x7fa9840ae860 2020-12-05T17:37:48.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.631+0000 7fa983fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa984152860 con 0x7fa9840b0240 2020-12-05T17:37:48.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.631+0000 7fa989732700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9840ae860 0x7fa98414f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:48.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.631+0000 7fa9837fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa9840611b0 0x7fa98414c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39348/0 (socket says 172.21.15.32:39348) 2020-12-05T17:37:48.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.631+0000 7fa9837fe700 1 -- 172.21.15.32:0/1441657524 learned_addr learned my addr 172.21.15.32:0/1441657524 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:48.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.631+0000 7fa982ffd700 1 --2- 172.21.15.32:0/1441657524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9840b0240 0x7fa984153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:48.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.635+0000 7fa989732700 1 -- 172.21.15.32:0/1441657524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9840b0240 msgr2=0x7fa984153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.635+0000 7fa989732700 1 --2- 172.21.15.32:0/1441657524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9840b0240 0x7fa984153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.635+0000 7fa989732700 1 -- 172.21.15.32:0/1441657524 >> v1:172.21.15.32:6789/0 conn(0x7fa9840611b0 legacy=0x7fa98414c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.635+0000 7fa989732700 1 -- 172.21.15.32:0/1441657524 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa984152b00 con 0x7fa9840ae860 2020-12-05T17:37:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.635+0000 7fa989732700 1 --2- 172.21.15.32:0/1441657524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9840ae860 0x7fa98414f0d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fa97c0032b0 tx=0x7fa97c0091a0).ready entity=mon.0 client_cookie=bfd129a57ac88c49 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.635+0000 7fa9827fc700 1 -- 172.21.15.32:0/1441657524 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa97c00ba50 con 0x7fa9840ae860 2020-12-05T17:37:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.635+0000 7fa9827fc700 1 -- 172.21.15.32:0/1441657524 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97c00bbb0 con 0x7fa9840ae860 2020-12-05T17:37:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.635+0000 7fa9827fc700 1 -- 172.21.15.32:0/1441657524 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa97c00bd20 con 0x7fa9840ae860 2020-12-05T17:37:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.635+0000 7fa983fff700 1 -- 172.21.15.32:0/1441657524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9840ae860 msgr2=0x7fa98414f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.635+0000 7fa983fff700 1 --2- 172.21.15.32:0/1441657524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9840ae860 0x7fa98414f0d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fa97c0032b0 tx=0x7fa97c0091a0).stop 2020-12-05T17:37:48.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.635+0000 7fa983fff700 1 -- 172.21.15.32:0/1441657524 shutdown_connections 2020-12-05T17:37:48.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.635+0000 7fa983fff700 1 --2- 172.21.15.32:0/1441657524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9840ae860 0x7fa98414f0d0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:48.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.635+0000 7fa983fff700 1 --2- 172.21.15.32:0/1441657524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9840b0240 0x7fa984153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:48.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.635+0000 7fa983fff700 1 -- 172.21.15.32:0/1441657524 >> 172.21.15.32:0/1441657524 conn(0x7fa9840b5f90 msgr2=0x7fa9840b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:48.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.635+0000 7fa983fff700 1 -- 172.21.15.32:0/1441657524 shutdown_connections 2020-12-05T17:37:48.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.635+0000 7fa983fff700 1 -- 172.21.15.32:0/1441657524 wait complete. 2020-12-05T17:37:48.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa983fff700 1 Processor -- start 2020-12-05T17:37:48.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa983fff700 1 -- start start 2020-12-05T17:37:48.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa983fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9840611b0 0x7fa9840b4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:48.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa983fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa9840ae860 0x7fa984064850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:48.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa983fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9840b0240 0x7fa984064d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:48.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa983fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98412c4c0 con 0x7fa9840611b0 2020-12-05T17:37:48.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa983fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98412bae0 con 0x7fa9840ae860 2020-12-05T17:37:48.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa983fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98412bd80 con 0x7fa9840b0240 2020-12-05T17:37:48.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa9837fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9840611b0 0x7fa9840b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:48.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa9837fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9840611b0 0x7fa9840b4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36706/0 (socket says 172.21.15.32:36706) 2020-12-05T17:37:48.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa9837fe700 1 -- 172.21.15.32:0/2970575293 learned_addr learned my addr 172.21.15.32:0/2970575293 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:48.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa982ffd700 1 --2- 172.21.15.32:0/2970575293 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa9840ae860 0x7fa984064850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:48.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa989732700 1 --2- 172.21.15.32:0/2970575293 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9840b0240 0x7fa984064d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:48.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa9837fe700 1 -- 172.21.15.32:0/2970575293 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9840b0240 msgr2=0x7fa984064d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:48.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa9837fe700 1 --2- 172.21.15.32:0/2970575293 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9840b0240 0x7fa984064d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:48.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa9837fe700 1 -- 172.21.15.32:0/2970575293 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa9840ae860 msgr2=0x7fa984064850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:48.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa9837fe700 1 --2- 172.21.15.32:0/2970575293 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa9840ae860 0x7fa984064850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:48.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa9837fe700 1 -- 172.21.15.32:0/2970575293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98412c020 con 0x7fa9840611b0 2020-12-05T17:37:48.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa9837fe700 1 --2- 172.21.15.32:0/2970575293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9840611b0 0x7fa9840b4890 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7fa984060810 tx=0x7fa974007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:48.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa96bfff700 1 -- 172.21.15.32:0/2970575293 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9740072e0 con 0x7fa9840611b0 2020-12-05T17:37:48.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa96bfff700 1 -- 172.21.15.32:0/2970575293 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa974007440 con 0x7fa9840611b0 2020-12-05T17:37:48.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa983fff700 1 -- 172.21.15.32:0/2970575293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa98412b760 con 0x7fa9840611b0 2020-12-05T17:37:48.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa96bfff700 1 -- 172.21.15.32:0/2970575293 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9740057d0 con 0x7fa9840611b0 2020-12-05T17:37:48.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.639+0000 7fa983fff700 1 -- 172.21.15.32:0/2970575293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9840b4d80 con 0x7fa9840611b0 2020-12-05T17:37:48.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.643+0000 7fa983fff700 1 -- 172.21.15.32:0/2970575293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa984063800 con 0x7fa9840611b0 2020-12-05T17:37:48.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.643+0000 7fa96bfff700 1 -- 172.21.15.32:0/2970575293 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa97401a070 con 0x7fa9840611b0 2020-12-05T17:37:48.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.643+0000 7fa96bfff700 1 --2- 172.21.15.32:0/2970575293 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa9640554d0 0x7fa964057990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:48.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.643+0000 7fa96bfff700 1 -- 172.21.15.32:0/2970575293 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa97406d140 con 0x7fa9840611b0 2020-12-05T17:37:48.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.643+0000 7fa982ffd700 1 --2- 172.21.15.32:0/2970575293 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa9640554d0 0x7fa964057990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:48.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.643+0000 7fa982ffd700 1 --2- 172.21.15.32:0/2970575293 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa9640554d0 0x7fa964057990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fa97800c000 tx=0x7fa978008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:48.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.651+0000 7fa96bfff700 1 -- 172.21.15.32:0/2970575293 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa984063800 con 0x7fa9840611b0 2020-12-05T17:37:48.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.843+0000 7fa983fff700 1 -- 172.21.15.32:0/2970575293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fa984063800 con 0x7fa9840611b0 2020-12-05T17:37:48.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.847+0000 7fa96bfff700 1 -- 172.21.15.32:0/2970575293 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v24) v1 ==== 184+0+0 (secure 0 0 0) 0x7fa984063800 con 0x7fa9840611b0 2020-12-05T17:37:48.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.851+0000 7fa983fff700 1 -- 172.21.15.32:0/2970575293 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa9640554d0 msgr2=0x7fa964057990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:48.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.851+0000 7fa983fff700 1 --2- 172.21.15.32:0/2970575293 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa9640554d0 0x7fa964057990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fa97800c000 tx=0x7fa978008040).stop 2020-12-05T17:37:48.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.851+0000 7fa983fff700 1 -- 172.21.15.32:0/2970575293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9840611b0 msgr2=0x7fa9840b4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:48.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.851+0000 7fa983fff700 1 --2- 172.21.15.32:0/2970575293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9840611b0 0x7fa9840b4890 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7fa984060810 tx=0x7fa974007c10).stop 2020-12-05T17:37:48.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.851+0000 7fa983fff700 1 -- 172.21.15.32:0/2970575293 shutdown_connections 2020-12-05T17:37:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.851+0000 7fa983fff700 1 --2- 172.21.15.32:0/2970575293 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa9640554d0 0x7fa964057990 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.851+0000 7fa983fff700 1 --2- 172.21.15.32:0/2970575293 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9840b0240 0x7fa984064d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.851+0000 7fa983fff700 1 --2- 172.21.15.32:0/2970575293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9840611b0 0x7fa9840b4890 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.851+0000 7fa983fff700 1 --2- 172.21.15.32:0/2970575293 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa9840ae860 0x7fa984064850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:48.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.851+0000 7fa983fff700 1 -- 172.21.15.32:0/2970575293 >> 172.21.15.32:0/2970575293 conn(0x7fa9840b5f90 msgr2=0x7fa9840adda0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:48.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.851+0000 7fa983fff700 1 -- 172.21.15.32:0/2970575293 shutdown_connections 2020-12-05T17:37:48.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.851+0000 7fa983fff700 1 -- 172.21.15.32:0/2970575293 wait complete. 2020-12-05T17:37:48.859 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2020-12-05T17:37:48.870 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:37:48.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.991+0000 7f0209526700 1 Processor -- start 2020-12-05T17:37:48.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.991+0000 7f0209526700 1 -- start start 2020-12-05T17:37:48.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.991+0000 7f0209526700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f020405f6f0 0x7f02040adef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:48.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.991+0000 7f0209526700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f020405feb0 0x7f02040ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:48.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.991+0000 7f0209526700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02040b0380 con 0x7f0204061780 2020-12-05T17:37:48.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.991+0000 7f0209526700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02040aef70 con 0x7f020405feb0 2020-12-05T17:37:49.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.991+0000 7f0209526700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020405fad0 con 0x7f020405f6f0 2020-12-05T17:37:49.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.991+0000 7f0202ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f020405feb0 0x7f02040ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:49.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.991+0000 7f0202ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f020405feb0 0x7f02040ae3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36716/0 (socket says 172.21.15.32:36716) 2020-12-05T17:37:49.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.991+0000 7f0202ffd700 1 -- 172.21.15.32:0/4112803410 learned_addr learned my addr 172.21.15.32:0/4112803410 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:49.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.991+0000 7f02037fe700 1 --2- 172.21.15.32:0/4112803410 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f020405f6f0 0x7f02040adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:49.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.991+0000 7f0202ffd700 1 -- 172.21.15.32:0/4112803410 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f020405f6f0 msgr2=0x7f02040adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:49.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.991+0000 7f0202ffd700 1 --2- 172.21.15.32:0/4112803410 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f020405f6f0 0x7f02040adef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:49.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.991+0000 7f0202ffd700 1 -- 172.21.15.32:0/4112803410 >> v1:172.21.15.61:6789/0 conn(0x7f0204061780 legacy=0x7f0204152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:49.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.991+0000 7f0202ffd700 1 -- 172.21.15.32:0/4112803410 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02040aecd0 con 0x7f020405feb0 2020-12-05T17:37:49.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.991+0000 7f0202ffd700 1 --2- 172.21.15.32:0/4112803410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020405feb0 0x7f02040ae3e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f01ec002f30 tx=0x7f01ec006e80).ready entity=mon.0 client_cookie=97fd84a1aa29a447 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:49.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.991+0000 7f02027fc700 1 -- 172.21.15.32:0/4112803410 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01ec0097f0 con 0x7f020405feb0 2020-12-05T17:37:49.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.991+0000 7f02027fc700 1 -- 172.21.15.32:0/4112803410 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec009950 con 0x7f020405feb0 2020-12-05T17:37:49.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.991+0000 7f02027fc700 1 -- 172.21.15.32:0/4112803410 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01ec009cb0 con 0x7f020405feb0 2020-12-05T17:37:49.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.991+0000 7f0209526700 1 -- 172.21.15.32:0/4112803410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020405feb0 msgr2=0x7f02040ae3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:49.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.991+0000 7f0209526700 1 --2- 172.21.15.32:0/4112803410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020405feb0 0x7f02040ae3e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f01ec002f30 tx=0x7f01ec006e80).stop 2020-12-05T17:37:49.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.995+0000 7f0209526700 1 -- 172.21.15.32:0/4112803410 shutdown_connections 2020-12-05T17:37:49.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.995+0000 7f0209526700 1 --2- 172.21.15.32:0/4112803410 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f020405f6f0 0x7f02040adef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:49.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.995+0000 7f0209526700 1 --2- 172.21.15.32:0/4112803410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020405feb0 0x7f02040ae3e0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:49.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.995+0000 7f0209526700 1 -- 172.21.15.32:0/4112803410 >> 172.21.15.32:0/4112803410 conn(0x7f02040b4b10 msgr2=0x7f02040b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:49.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.995+0000 7f0209526700 1 -- 172.21.15.32:0/4112803410 shutdown_connections 2020-12-05T17:37:49.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.995+0000 7f0209526700 1 -- 172.21.15.32:0/4112803410 wait complete. 2020-12-05T17:37:49.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.995+0000 7f0209526700 1 Processor -- start 2020-12-05T17:37:49.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.995+0000 7f0209526700 1 -- start start 2020-12-05T17:37:49.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.995+0000 7f0209526700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f020405f6f0 0x7f020414e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:49.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.999+0000 7f0209526700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020405feb0 0x7f0204063d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:49.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.999+0000 7f0209526700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0204061780 0x7f0204064270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:49.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.999+0000 7f0209526700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0204066bc0 con 0x7f020405feb0 2020-12-05T17:37:49.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.999+0000 7f0209526700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0204066920 con 0x7f020405f6f0 2020-12-05T17:37:49.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.999+0000 7f0209526700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020414ee20 con 0x7f0204061780 2020-12-05T17:37:49.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.999+0000 7f0202ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020405feb0 0x7f0204063d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:49.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.999+0000 7f0202ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020405feb0 0x7f0204063d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36724/0 (socket says 172.21.15.32:36724) 2020-12-05T17:37:49.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.999+0000 7f0202ffd700 1 -- 172.21.15.32:0/1079993692 learned_addr learned my addr 172.21.15.32:0/1079993692 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:49.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.999+0000 7f0203fff700 1 --2- 172.21.15.32:0/1079993692 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0204061780 0x7f0204064270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:49.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.999+0000 7f02037fe700 1 --2- 172.21.15.32:0/1079993692 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f020405f6f0 0x7f020414e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:49.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.999+0000 7f0202ffd700 1 -- 172.21.15.32:0/1079993692 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0204061780 msgr2=0x7f0204064270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:49.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.999+0000 7f0202ffd700 1 --2- 172.21.15.32:0/1079993692 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0204061780 0x7f0204064270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:49.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.999+0000 7f0202ffd700 1 -- 172.21.15.32:0/1079993692 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f020405f6f0 msgr2=0x7f020414e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:49.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.999+0000 7f0202ffd700 1 --2- 172.21.15.32:0/1079993692 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f020405f6f0 0x7f020414e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:49.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.999+0000 7f0202ffd700 1 -- 172.21.15.32:0/1079993692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0204064760 con 0x7f020405feb0 2020-12-05T17:37:49.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.999+0000 7f0202ffd700 1 --2- 172.21.15.32:0/1079993692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020405feb0 0x7f0204063d80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7f01ec002f30 tx=0x7f01ec015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:49.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.999+0000 7f01fbfff700 1 -- 172.21.15.32:0/1079993692 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01ec011bf0 con 0x7f020405feb0 2020-12-05T17:37:49.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.999+0000 7f01fbfff700 1 -- 172.21.15.32:0/1079993692 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec011d50 con 0x7f020405feb0 2020-12-05T17:37:49.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.999+0000 7f0209526700 1 -- 172.21.15.32:0/1079993692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0204062540 con 0x7f020405feb0 2020-12-05T17:37:49.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.999+0000 7f01fbfff700 1 -- 172.21.15.32:0/1079993692 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01ec01e650 con 0x7f020405feb0 2020-12-05T17:37:49.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.999+0000 7f0209526700 1 -- 172.21.15.32:0/1079993692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0204138400 con 0x7f020405feb0 2020-12-05T17:37:49.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.999+0000 7f01fbfff700 1 -- 172.21.15.32:0/1079993692 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f01ec025030 con 0x7f020405feb0 2020-12-05T17:37:49.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:48.999+0000 7f0209526700 1 -- 172.21.15.32:0/1079993692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0204066770 con 0x7f020405feb0 2020-12-05T17:37:49.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.003+0000 7f01fbfff700 1 --2- 172.21.15.32:0/1079993692 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f01dc055320 0x7f01dc0577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:49.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.003+0000 7f01fbfff700 1 -- 172.21.15.32:0/1079993692 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f01ec06f400 con 0x7f020405feb0 2020-12-05T17:37:49.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.003+0000 7f02037fe700 1 --2- 172.21.15.32:0/1079993692 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f01dc055320 0x7f01dc0577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:49.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.003+0000 7f02037fe700 1 --2- 172.21.15.32:0/1079993692 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f01dc055320 0x7f01dc0577e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f01fc00b000 tx=0x7f01fc006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:49.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.007+0000 7f01fbfff700 1 -- 172.21.15.32:0/1079993692 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f01ec0455a0 con 0x7f020405feb0 2020-12-05T17:37:49.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.207+0000 7f0209526700 1 -- 172.21.15.32:0/1079993692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f02040648f0 con 0x7f020405feb0 2020-12-05T17:37:49.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.215+0000 7f01fbfff700 1 -- 172.21.15.32:0/1079993692 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v25) v1 ==== 184+0+0 (secure 0 0 0) 0x7f01ec0230e0 con 0x7f020405feb0 2020-12-05T17:37:49.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.219+0000 7f0209526700 1 -- 172.21.15.32:0/1079993692 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f01dc055320 msgr2=0x7f01dc0577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:49.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.219+0000 7f0209526700 1 --2- 172.21.15.32:0/1079993692 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f01dc055320 0x7f01dc0577e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f01fc00b000 tx=0x7f01fc006dc0).stop 2020-12-05T17:37:49.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.219+0000 7f0209526700 1 -- 172.21.15.32:0/1079993692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020405feb0 msgr2=0x7f0204063d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:49.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.219+0000 7f0209526700 1 --2- 172.21.15.32:0/1079993692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020405feb0 0x7f0204063d80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7f01ec002f30 tx=0x7f01ec015000).stop 2020-12-05T17:37:49.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.219+0000 7f0209526700 1 -- 172.21.15.32:0/1079993692 shutdown_connections 2020-12-05T17:37:49.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.219+0000 7f0209526700 1 --2- 172.21.15.32:0/1079993692 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f01dc055320 0x7f01dc0577e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:49.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.219+0000 7f0209526700 1 --2- 172.21.15.32:0/1079993692 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0204061780 0x7f0204064270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:49.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.219+0000 7f0209526700 1 --2- 172.21.15.32:0/1079993692 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f020405f6f0 0x7f020414e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:49.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.219+0000 7f0209526700 1 --2- 172.21.15.32:0/1079993692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020405feb0 0x7f0204063d80 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:49.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.219+0000 7f0209526700 1 -- 172.21.15.32:0/1079993692 >> 172.21.15.32:0/1079993692 conn(0x7f02040b4b10 msgr2=0x7f02041237a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:49.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.219+0000 7f0209526700 1 -- 172.21.15.32:0/1079993692 shutdown_connections 2020-12-05T17:37:49.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.219+0000 7f0209526700 1 -- 172.21.15.32:0/1079993692 wait complete. 2020-12-05T17:37:49.225 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.2 2020-12-05T17:37:49.239 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:37:49.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.347+0000 7f34f1802700 1 Processor -- start 2020-12-05T17:37:49.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.347+0000 7f34f1802700 1 -- start start 2020-12-05T17:37:49.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.347+0000 7f34f1802700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34ec05f3b0 0x7f34ec05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:49.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.347+0000 7f34f1802700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ec142450 con 0x7f34ec062400 2020-12-05T17:37:49.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.347+0000 7f34f1802700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ec061780 con 0x7f34ec05feb0 2020-12-05T17:37:49.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.347+0000 7f34f1802700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34ec061a20 con 0x7f34ec05f3b0 2020-12-05T17:37:49.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.351+0000 7f34f0800700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34ec05f3b0 0x7f34ec05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:49.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.351+0000 7f34f0800700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34ec05f3b0 0x7f34ec05f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36728/0 (socket says 172.21.15.32:36728) 2020-12-05T17:37:49.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.351+0000 7f34f0800700 1 -- 172.21.15.32:0/122999341 learned_addr learned my addr 172.21.15.32:0/122999341 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:49.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.351+0000 7f34ebfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f34ec062400 0x7f34ec0b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39380/0 (socket says 172.21.15.32:39380) 2020-12-05T17:37:49.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.351+0000 7f34f0800700 1 -- 172.21.15.32:0/122999341 >> v1:172.21.15.61:6789/0 conn(0x7f34ec05feb0 legacy=0x7f34ec0b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:49.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.351+0000 7f34f0800700 1 -- 172.21.15.32:0/122999341 >> v1:172.21.15.32:6789/0 conn(0x7f34ec062400 legacy=0x7f34ec0b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:49.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.351+0000 7f34f0800700 1 -- 172.21.15.32:0/122999341 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec061cc0 con 0x7f34ec05f3b0 2020-12-05T17:37:49.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.351+0000 7f34f0800700 1 --2- 172.21.15.32:0/122999341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34ec05f3b0 0x7f34ec05f790 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7f34dc002f30 tx=0x7f34dc006e80).ready entity=mon.0 client_cookie=5a7524591777b1c2 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:49.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.351+0000 7f34eb7fe700 1 -- 172.21.15.32:0/122999341 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34dc0097f0 con 0x7f34ec05f3b0 2020-12-05T17:37:49.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.351+0000 7f34eb7fe700 1 -- 172.21.15.32:0/122999341 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34dc009950 con 0x7f34ec05f3b0 2020-12-05T17:37:49.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.351+0000 7f34eb7fe700 1 -- 172.21.15.32:0/122999341 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34dc009cb0 con 0x7f34ec05f3b0 2020-12-05T17:37:49.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.351+0000 7f34f1802700 1 -- 172.21.15.32:0/122999341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34ec05f3b0 msgr2=0x7f34ec05f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:49.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.351+0000 7f34f1802700 1 --2- 172.21.15.32:0/122999341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34ec05f3b0 0x7f34ec05f790 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7f34dc002f30 tx=0x7f34dc006e80).stop 2020-12-05T17:37:49.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.351+0000 7f34f1802700 1 -- 172.21.15.32:0/122999341 shutdown_connections 2020-12-05T17:37:49.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.351+0000 7f34f1802700 1 --2- 172.21.15.32:0/122999341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34ec05f3b0 0x7f34ec05f790 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:49.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.351+0000 7f34f1802700 1 -- 172.21.15.32:0/122999341 >> 172.21.15.32:0/122999341 conn(0x7f34ec0b8ee0 msgr2=0x7f34ec0b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:49.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.351+0000 7f34f1802700 1 -- 172.21.15.32:0/122999341 shutdown_connections 2020-12-05T17:37:49.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.351+0000 7f34f1802700 1 -- 172.21.15.32:0/122999341 wait complete. 2020-12-05T17:37:49.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.351+0000 7f34f1802700 1 Processor -- start 2020-12-05T17:37:49.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.351+0000 7f34f1802700 1 -- start start 2020-12-05T17:37:49.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.355+0000 7f34f1802700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34ec05f3b0 0x7f34ec0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:49.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.355+0000 7f34f1802700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34ec05feb0 0x7f34ec143a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:49.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.355+0000 7f34f1802700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34ec062400 0x7f34ec154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:49.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.355+0000 7f34f1802700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec146250 con 0x7f34ec062400 2020-12-05T17:37:49.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.355+0000 7f34f1802700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec0b2610 con 0x7f34ec05f3b0 2020-12-05T17:37:49.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.355+0000 7f34f1802700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec0b2350 con 0x7f34ec05feb0 2020-12-05T17:37:49.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.355+0000 7f34f1001700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34ec062400 0x7f34ec154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:49.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.355+0000 7f34f1001700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34ec062400 0x7f34ec154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36736/0 (socket says 172.21.15.32:36736) 2020-12-05T17:37:49.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.355+0000 7f34f1001700 1 -- 172.21.15.32:0/2426717811 learned_addr learned my addr 172.21.15.32:0/2426717811 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:49.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.355+0000 7f34ebfff700 1 --2- 172.21.15.32:0/2426717811 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34ec05feb0 0x7f34ec143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:49.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.355+0000 7f34f0800700 1 --2- 172.21.15.32:0/2426717811 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34ec05f3b0 0x7f34ec0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:49.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.355+0000 7f34f1001700 1 -- 172.21.15.32:0/2426717811 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34ec05feb0 msgr2=0x7f34ec143a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:49.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.355+0000 7f34f1001700 1 --2- 172.21.15.32:0/2426717811 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34ec05feb0 0x7f34ec143a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:49.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.355+0000 7f34f1001700 1 -- 172.21.15.32:0/2426717811 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34ec05f3b0 msgr2=0x7f34ec0b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:49.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.355+0000 7f34f1001700 1 --2- 172.21.15.32:0/2426717811 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34ec05f3b0 0x7f34ec0b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:49.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.355+0000 7f34f1001700 1 -- 172.21.15.32:0/2426717811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec0b31d0 con 0x7f34ec062400 2020-12-05T17:37:49.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.355+0000 7f34f1001700 1 --2- 172.21.15.32:0/2426717811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34ec062400 0x7f34ec154ab0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f34ec05c5c0 tx=0x7f34e4007ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:49.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.355+0000 7f34e9ffb700 1 -- 172.21.15.32:0/2426717811 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34e40030f0 con 0x7f34ec062400 2020-12-05T17:37:49.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.355+0000 7f34e9ffb700 1 -- 172.21.15.32:0/2426717811 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e400a7b0 con 0x7f34ec062400 2020-12-05T17:37:49.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.355+0000 7f34e9ffb700 1 -- 172.21.15.32:0/2426717811 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34e400ab10 con 0x7f34ec062400 2020-12-05T17:37:49.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.355+0000 7f34f1802700 1 -- 172.21.15.32:0/2426717811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ec145e60 con 0x7f34ec062400 2020-12-05T17:37:49.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.355+0000 7f34f1802700 1 -- 172.21.15.32:0/2426717811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ec1467c0 con 0x7f34ec062400 2020-12-05T17:37:49.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.355+0000 7f34f1802700 1 -- 172.21.15.32:0/2426717811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ec1442b0 con 0x7f34ec062400 2020-12-05T17:37:49.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.359+0000 7f34e9ffb700 1 -- 172.21.15.32:0/2426717811 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f34e4019030 con 0x7f34ec062400 2020-12-05T17:37:49.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.359+0000 7f34e9ffb700 1 --2- 172.21.15.32:0/2426717811 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f34d00553b0 0x7f34d0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:49.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.359+0000 7f34e9ffb700 1 -- 172.21.15.32:0/2426717811 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f34e406d890 con 0x7f34ec062400 2020-12-05T17:37:49.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.359+0000 7f34f0800700 1 --2- 172.21.15.32:0/2426717811 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f34d00553b0 0x7f34d0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:49.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.359+0000 7f34f0800700 1 --2- 172.21.15.32:0/2426717811 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f34d00553b0 0x7f34d0057870 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f34dc01a000 tx=0x7f34dc015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:49.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.363+0000 7f34e9ffb700 1 -- 172.21.15.32:0/2426717811 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f34ec1442b0 con 0x7f34ec062400 2020-12-05T17:37:49.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.547+0000 7f34f1802700 1 -- 172.21.15.32:0/2426717811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34ec1442b0 con 0x7f34ec062400 2020-12-05T17:37:49.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.547+0000 7f34e9ffb700 1 -- 172.21.15.32:0/2426717811 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f34ec1442b0 con 0x7f34ec062400 2020-12-05T17:37:49.553 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:49.553 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":32,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5645,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/992428276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":992428276},{"type":"v1","addr":"172.21.15.32:6835","nonce":992428276}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:37:46.495876+0000","modified":"2020-12-05T17:37:47.498743+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6160":{"gid":6160,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.61:6833/4214217205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4214217205},{"type":"v1","addr":"172.21.15.61:6833","nonce":4214217205}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-12-05T17:37:49.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.551+0000 7f34f1802700 1 -- 172.21.15.32:0/2426717811 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f34d00553b0 msgr2=0x7f34d0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:49.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.551+0000 7f34f1802700 1 --2- 172.21.15.32:0/2426717811 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f34d00553b0 0x7f34d0057870 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f34dc01a000 tx=0x7f34dc015000).stop 2020-12-05T17:37:49.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.551+0000 7f34f1802700 1 -- 172.21.15.32:0/2426717811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34ec062400 msgr2=0x7f34ec154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:49.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.551+0000 7f34f1802700 1 --2- 172.21.15.32:0/2426717811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34ec062400 0x7f34ec154ab0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f34ec05c5c0 tx=0x7f34e4007ee0).stop 2020-12-05T17:37:49.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.551+0000 7f34f1802700 1 -- 172.21.15.32:0/2426717811 shutdown_connections 2020-12-05T17:37:49.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.551+0000 7f34f1802700 1 --2- 172.21.15.32:0/2426717811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34ec062400 0x7f34ec154ab0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:49.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.551+0000 7f34f1802700 1 --2- 172.21.15.32:0/2426717811 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34ec05f3b0 0x7f34ec0b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:49.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.551+0000 7f34f1802700 1 --2- 172.21.15.32:0/2426717811 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f34d00553b0 0x7f34d0057870 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:49.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.551+0000 7f34f1802700 1 --2- 172.21.15.32:0/2426717811 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34ec05feb0 0x7f34ec143a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:49.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.551+0000 7f34f1802700 1 -- 172.21.15.32:0/2426717811 >> 172.21.15.32:0/2426717811 conn(0x7f34ec0b8ee0 msgr2=0x7f34ec0b80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:49.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.551+0000 7f34f1802700 1 -- 172.21.15.32:0/2426717811 shutdown_connections 2020-12-05T17:37:49.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:49.551+0000 7f34f1802700 1 -- 172.21.15.32:0/2426717811 wait complete. 2020-12-05T17:37:49.558 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 32 2020-12-05T17:37:49.769 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:37:46.495876+0000', 'modified': '2020-12-05T17:37:47.498743+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6160}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6160': {'gid': 6160, 'name': 'b', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.61:6833/4214217205', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6832', 'nonce': 4214217205}, {'type': 'v1', 'addr': '172.21.15.61:6833', 'nonce': 4214217205}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:37:49.769 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:37:49.771 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-12-05T17:37:49.909 INFO:teuthology.orchestra.run.smithi061.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":6160,"want_state":"up:active","state":"up:active","rank_uptime":3.398987521,"mdsmap_epoch":32,"osdmap_epoch":72,"osdmap_epoch_barrier":71,"uptime":7.7316573200000001} 2020-12-05T17:37:49.910 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 6160, "mdsmap_epoch": 32, "osdmap_epoch": 72, "osdmap_epoch_barrier": 71, "rank_uptime": 3.398987521, "state": "up:active", "uptime": 7.73165732, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:37:49.911 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:37:50.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.055+0000 7fa9089f4700 1 Processor -- start 2020-12-05T17:37:50.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.055+0000 7fa9089f4700 1 -- start start 2020-12-05T17:37:50.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.055+0000 7fa9089f4700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa9040b54b0 0x7fa9040b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.055+0000 7fa9089f4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9040afd20 0x7fa9040b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.055+0000 7fa9089f4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9040adfe0 con 0x7fa904145c90 2020-12-05T17:37:50.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.055+0000 7fa9089f4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9040b0630 con 0x7fa9040afd20 2020-12-05T17:37:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.055+0000 7fa9089f4700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9040ae210 con 0x7fa9040b54b0 2020-12-05T17:37:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.059+0000 7fa902ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9040afd20 0x7fa9040b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.059+0000 7fa902ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9040afd20 0x7fa9040b0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36742/0 (socket says 172.21.15.32:36742) 2020-12-05T17:37:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.059+0000 7fa902ffd700 1 -- 172.21.15.32:0/2770474983 learned_addr learned my addr 172.21.15.32:0/2770474983 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:50.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.059+0000 7fa903fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa904145c90 0x7fa9041460d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39394/0 (socket says 172.21.15.32:39394) 2020-12-05T17:37:50.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.059+0000 7fa9037fe700 1 --2- 172.21.15.32:0/2770474983 >> v2:172.21.15.61:3300/0 conn(0x7fa9040b54b0 0x7fa9040b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.059+0000 7fa902ffd700 1 -- 172.21.15.32:0/2770474983 >> v2:172.21.15.61:3300/0 conn(0x7fa9040b54b0 msgr2=0x7fa9040b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:50.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.059+0000 7fa902ffd700 1 --2- 172.21.15.32:0/2770474983 >> v2:172.21.15.61:3300/0 conn(0x7fa9040b54b0 0x7fa9040b0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.059+0000 7fa902ffd700 1 -- 172.21.15.32:0/2770474983 >> v1:172.21.15.32:6789/0 conn(0x7fa904145c90 legacy=0x7fa9041460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:50.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.059+0000 7fa902ffd700 1 -- 172.21.15.32:0/2770474983 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9040677f0 con 0x7fa9040afd20 2020-12-05T17:37:50.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.059+0000 7fa902ffd700 1 --2- 172.21.15.32:0/2770474983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9040afd20 0x7fa9040b0f80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7fa8f4002f30 tx=0x7fa8f4006e80).ready entity=mon.0 client_cookie=9890f29246097de3 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:50.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.059+0000 7fa9027fc700 1 -- 172.21.15.32:0/2770474983 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8f40097f0 con 0x7fa9040afd20 2020-12-05T17:37:50.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.059+0000 7fa9027fc700 1 -- 172.21.15.32:0/2770474983 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f4009950 con 0x7fa9040afd20 2020-12-05T17:37:50.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.059+0000 7fa9027fc700 1 -- 172.21.15.32:0/2770474983 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8f4009ac0 con 0x7fa9040afd20 2020-12-05T17:37:50.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.059+0000 7fa9089f4700 1 -- 172.21.15.32:0/2770474983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9040afd20 msgr2=0x7fa9040b0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:50.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.059+0000 7fa9089f4700 1 --2- 172.21.15.32:0/2770474983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9040afd20 0x7fa9040b0f80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7fa8f4002f30 tx=0x7fa8f4006e80).stop 2020-12-05T17:37:50.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.059+0000 7fa9089f4700 1 -- 172.21.15.32:0/2770474983 shutdown_connections 2020-12-05T17:37:50.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.059+0000 7fa9089f4700 1 --2- 172.21.15.32:0/2770474983 >> v2:172.21.15.61:3300/0 conn(0x7fa9040b54b0 0x7fa9040b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.059+0000 7fa9089f4700 1 --2- 172.21.15.32:0/2770474983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9040afd20 0x7fa9040b0f80 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.059+0000 7fa9089f4700 1 -- 172.21.15.32:0/2770474983 >> 172.21.15.32:0/2770474983 conn(0x7fa90405bbb0 msgr2=0x7fa904062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:50.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.059+0000 7fa9089f4700 1 -- 172.21.15.32:0/2770474983 shutdown_connections 2020-12-05T17:37:50.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.059+0000 7fa9089f4700 1 -- 172.21.15.32:0/2770474983 wait complete. 2020-12-05T17:37:50.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa9089f4700 1 Processor -- start 2020-12-05T17:37:50.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa9089f4700 1 -- start start 2020-12-05T17:37:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa9089f4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa9040afd20 0x7fa90413db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa9089f4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9040b54b0 0x7fa904145550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa9089f4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa904145c90 0x7fa90413c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa9089f4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa904143ea0 con 0x7fa904145c90 2020-12-05T17:37:50.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa9089f4700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa904144140 con 0x7fa9040afd20 2020-12-05T17:37:50.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa9089f4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa904145a40 con 0x7fa9040b54b0 2020-12-05T17:37:50.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa903fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa904145c90 0x7fa90413c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa903fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa904145c90 0x7fa90413c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36752/0 (socket says 172.21.15.32:36752) 2020-12-05T17:37:50.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa903fff700 1 -- 172.21.15.32:0/3036831307 learned_addr learned my addr 172.21.15.32:0/3036831307 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:50.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa902ffd700 1 --2- 172.21.15.32:0/3036831307 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9040b54b0 0x7fa904145550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa9037fe700 1 --2- 172.21.15.32:0/3036831307 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa9040afd20 0x7fa90413db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa903fff700 1 -- 172.21.15.32:0/3036831307 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9040b54b0 msgr2=0x7fa904145550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:50.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa903fff700 1 --2- 172.21.15.32:0/3036831307 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9040b54b0 0x7fa904145550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa903fff700 1 -- 172.21.15.32:0/3036831307 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa9040afd20 msgr2=0x7fa90413db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:50.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa903fff700 1 --2- 172.21.15.32:0/3036831307 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa9040afd20 0x7fa90413db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa903fff700 1 -- 172.21.15.32:0/3036831307 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa904151770 con 0x7fa904145c90 2020-12-05T17:37:50.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa903fff700 1 --2- 172.21.15.32:0/3036831307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa904145c90 0x7fa90413c270 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7fa904062be0 tx=0x7fa8fc007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:50.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa900ff9700 1 -- 172.21.15.32:0/3036831307 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8fc00ce20 con 0x7fa904145c90 2020-12-05T17:37:50.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa900ff9700 1 -- 172.21.15.32:0/3036831307 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8fc005ab0 con 0x7fa904145c90 2020-12-05T17:37:50.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa9089f4700 1 -- 172.21.15.32:0/3036831307 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa90410ea40 con 0x7fa904145c90 2020-12-05T17:37:50.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa900ff9700 1 -- 172.21.15.32:0/3036831307 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8fc005e10 con 0x7fa904145c90 2020-12-05T17:37:50.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.063+0000 7fa9089f4700 1 -- 172.21.15.32:0/3036831307 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa90413c760 con 0x7fa904145c90 2020-12-05T17:37:50.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.067+0000 7fa9089f4700 1 -- 172.21.15.32:0/3036831307 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa90410eee0 con 0x7fa904145c90 2020-12-05T17:37:50.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.067+0000 7fa900ff9700 1 -- 172.21.15.32:0/3036831307 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa8fc010030 con 0x7fa904145c90 2020-12-05T17:37:50.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.067+0000 7fa900ff9700 1 --2- 172.21.15.32:0/3036831307 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa8e4055440 0x7fa8e4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.067+0000 7fa900ff9700 1 -- 172.21.15.32:0/3036831307 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa8fc06df10 con 0x7fa904145c90 2020-12-05T17:37:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.067+0000 7fa9037fe700 1 --2- 172.21.15.32:0/3036831307 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa8e4055440 0x7fa8e4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.067+0000 7fa9037fe700 1 --2- 172.21.15.32:0/3036831307 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa8e4055440 0x7fa8e4057900 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7fa8f800b000 tx=0x7fa8f8006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.071+0000 7fa900ff9700 1 -- 172.21.15.32:0/3036831307 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa8fc044050 con 0x7fa904145c90 2020-12-05T17:37:50.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.215+0000 7fa9089f4700 1 -- 172.21.15.32:0/3036831307 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa904057050 con 0x7fa904145c90 2020-12-05T17:37:50.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.215+0000 7fa900ff9700 1 -- 172.21.15.32:0/3036831307 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16274 (secure 0 0 0) 0x7fa8fc047670 con 0x7fa904145c90 2020-12-05T17:37:50.222 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:50.222 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":72,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:37:47.499387+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:37:45.088140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-12-05T17:37:45.501261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:37:50.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.219+0000 7fa9089f4700 1 -- 172.21.15.32:0/3036831307 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa8e4055440 msgr2=0x7fa8e4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:50.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.219+0000 7fa9089f4700 1 --2- 172.21.15.32:0/3036831307 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa8e4055440 0x7fa8e4057900 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7fa8f800b000 tx=0x7fa8f8006dc0).stop 2020-12-05T17:37:50.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.219+0000 7fa9089f4700 1 -- 172.21.15.32:0/3036831307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa904145c90 msgr2=0x7fa90413c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:50.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.219+0000 7fa9089f4700 1 --2- 172.21.15.32:0/3036831307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa904145c90 0x7fa90413c270 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7fa904062be0 tx=0x7fa8fc007f70).stop 2020-12-05T17:37:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.219+0000 7fa9089f4700 1 -- 172.21.15.32:0/3036831307 shutdown_connections 2020-12-05T17:37:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.219+0000 7fa9089f4700 1 --2- 172.21.15.32:0/3036831307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa904145c90 0x7fa90413c270 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.219+0000 7fa9089f4700 1 --2- 172.21.15.32:0/3036831307 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa9040afd20 0x7fa90413db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.219+0000 7fa9089f4700 1 --2- 172.21.15.32:0/3036831307 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa8e4055440 0x7fa8e4057900 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.219+0000 7fa9089f4700 1 --2- 172.21.15.32:0/3036831307 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa9040b54b0 0x7fa904145550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.219+0000 7fa9089f4700 1 -- 172.21.15.32:0/3036831307 >> 172.21.15.32:0/3036831307 conn(0x7fa90405bbb0 msgr2=0x7fa9040b3490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:50.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.219+0000 7fa9089f4700 1 -- 172.21.15.32:0/3036831307 shutdown_connections 2020-12-05T17:37:50.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.219+0000 7fa9089f4700 1 -- 172.21.15.32:0/3036831307 wait complete. 2020-12-05T17:37:50.235 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:37:45.088140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:37:45.501261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:37:50.237 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:37:50.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.351+0000 7f30711f7700 1 Processor -- start 2020-12-05T17:37:50.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.351+0000 7f30711f7700 1 -- start start 2020-12-05T17:37:50.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.351+0000 7f30711f7700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f306c0b12d0 0x7f306c1387e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f30711f7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f306c0b1b50 0x7f306c138cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f30711f7700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306c0b1ff0 con 0x7f306c0afef0 2020-12-05T17:37:50.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f30711f7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f306c13fba0 con 0x7f306c0b1b50 2020-12-05T17:37:50.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f30711f7700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f306c0ad720 con 0x7f306c0b12d0 2020-12-05T17:37:50.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f306b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f306c0b1b50 0x7f306c138cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f306b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f306c0b1b50 0x7f306c138cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36758/0 (socket says 172.21.15.32:36758) 2020-12-05T17:37:50.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f306b7fe700 1 -- 172.21.15.32:0/3874531335 learned_addr learned my addr 172.21.15.32:0/3874531335 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:50.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f306bfff700 1 --2- 172.21.15.32:0/3874531335 >> v2:172.21.15.61:3300/0 conn(0x7f306c0b12d0 0x7f306c1387e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f306b7fe700 1 -- 172.21.15.32:0/3874531335 >> v2:172.21.15.61:3300/0 conn(0x7f306c0b12d0 msgr2=0x7f306c1387e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f306b7fe700 1 --2- 172.21.15.32:0/3874531335 >> v2:172.21.15.61:3300/0 conn(0x7f306c0b12d0 0x7f306c1387e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f306b7fe700 1 -- 172.21.15.32:0/3874531335 >> v1:172.21.15.61:6789/0 conn(0x7f306c0afef0 legacy=0x7f306c1423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:50.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f306b7fe700 1 -- 172.21.15.32:0/3874531335 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c0b2290 con 0x7f306c0b1b50 2020-12-05T17:37:50.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f306b7fe700 1 --2- 172.21.15.32:0/3874531335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306c0b1b50 0x7f306c138cd0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f306c0ad860 tx=0x7f3054006e80).ready entity=mon.0 client_cookie=ebb84dccf31a282f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:50.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f306affd700 1 -- 172.21.15.32:0/3874531335 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30540097b0 con 0x7f306c0b1b50 2020-12-05T17:37:50.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f306affd700 1 -- 172.21.15.32:0/3874531335 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3054009910 con 0x7f306c0b1b50 2020-12-05T17:37:50.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f306affd700 1 -- 172.21.15.32:0/3874531335 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3054009c70 con 0x7f306c0b1b50 2020-12-05T17:37:50.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f30711f7700 1 -- 172.21.15.32:0/3874531335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306c0b1b50 msgr2=0x7f306c138cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:50.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f30711f7700 1 --2- 172.21.15.32:0/3874531335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306c0b1b50 0x7f306c138cd0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f306c0ad860 tx=0x7f3054006e80).stop 2020-12-05T17:37:50.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f30711f7700 1 -- 172.21.15.32:0/3874531335 shutdown_connections 2020-12-05T17:37:50.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f30711f7700 1 --2- 172.21.15.32:0/3874531335 >> v2:172.21.15.61:3300/0 conn(0x7f306c0b12d0 0x7f306c1387e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f30711f7700 1 --2- 172.21.15.32:0/3874531335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306c0b1b50 0x7f306c138cd0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f30711f7700 1 -- 172.21.15.32:0/3874531335 >> 172.21.15.32:0/3874531335 conn(0x7f306c05bf80 msgr2=0x7f306c062190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:50.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f30711f7700 1 -- 172.21.15.32:0/3874531335 shutdown_connections 2020-12-05T17:37:50.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.355+0000 7f30711f7700 1 -- 172.21.15.32:0/3874531335 wait complete. 2020-12-05T17:37:50.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f30711f7700 1 Processor -- start 2020-12-05T17:37:50.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f30711f7700 1 -- start start 2020-12-05T17:37:50.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f30711f7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f306c0afef0 0x7f306c13a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f30711f7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f306c0b12d0 0x7f306c144e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f30711f7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306c0b1b50 0x7f306c13c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f30711f7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c066a90 con 0x7f306c0b1b50 2020-12-05T17:37:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f30711f7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c066d10 con 0x7f306c0b12d0 2020-12-05T17:37:50.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f30711f7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306c13ba20 con 0x7f306c0afef0 2020-12-05T17:37:50.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f30709f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306c0b1b50 0x7f306c13c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f306b7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f306c0b12d0 0x7f306c144e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f306bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f306c0afef0 0x7f306c13a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f30709f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306c0b1b50 0x7f306c13c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36766/0 (socket says 172.21.15.32:36766) 2020-12-05T17:37:50.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f30709f6700 1 -- 172.21.15.32:0/1010629974 learned_addr learned my addr 172.21.15.32:0/1010629974 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:50.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f30709f6700 1 -- 172.21.15.32:0/1010629974 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f306c0afef0 msgr2=0x7f306c13a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:50.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f30709f6700 1 --2- 172.21.15.32:0/1010629974 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f306c0afef0 0x7f306c13a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f30709f6700 1 -- 172.21.15.32:0/1010629974 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f306c0b12d0 msgr2=0x7f306c144e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:50.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f30709f6700 1 --2- 172.21.15.32:0/1010629974 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f306c0b12d0 0x7f306c144e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.381 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f30709f6700 1 -- 172.21.15.32:0/1010629974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c13bcc0 con 0x7f306c0b1b50 2020-12-05T17:37:50.381 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f30709f6700 1 --2- 172.21.15.32:0/1010629974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306c0b1b50 0x7f306c13c160 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f306c062f80 tx=0x7f3064007ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:50.381 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f30697fa700 1 -- 172.21.15.32:0/1010629974 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f306400ce20 con 0x7f306c0b1b50 2020-12-05T17:37:50.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f30697fa700 1 -- 172.21.15.32:0/1010629974 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306400a7b0 con 0x7f306c0b1b50 2020-12-05T17:37:50.383 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f30711f7700 1 -- 172.21.15.32:0/1010629974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f306c13c650 con 0x7f306c0b1b50 2020-12-05T17:37:50.383 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f30697fa700 1 -- 172.21.15.32:0/1010629974 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f306400a920 con 0x7f306c0b1b50 2020-12-05T17:37:50.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.359+0000 7f30711f7700 1 -- 172.21.15.32:0/1010629974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306c1395d0 con 0x7f306c0b1b50 2020-12-05T17:37:50.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.363+0000 7f30711f7700 1 -- 172.21.15.32:0/1010629974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306c057050 con 0x7f306c0b1b50 2020-12-05T17:37:50.385 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.363+0000 7f30697fa700 1 -- 172.21.15.32:0/1010629974 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3064019030 con 0x7f306c0b1b50 2020-12-05T17:37:50.385 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.363+0000 7f30697fa700 1 --2- 172.21.15.32:0/1010629974 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3050055440 0x7f3050057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.363+0000 7f30697fa700 1 -- 172.21.15.32:0/1010629974 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f306406d890 con 0x7f306c0b1b50 2020-12-05T17:37:50.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.363+0000 7f306bfff700 1 --2- 172.21.15.32:0/1010629974 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3050055440 0x7f3050057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.363+0000 7f306bfff700 1 --2- 172.21.15.32:0/1010629974 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3050055440 0x7f3050057900 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f305c00b000 tx=0x7f305c006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:50.388 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.367+0000 7f30697fa700 1 -- 172.21.15.32:0/1010629974 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3064043920 con 0x7f306c0b1b50 2020-12-05T17:37:50.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.507+0000 7f30711f7700 1 -- 172.21.15.32:0/1010629974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f306c05aab0 con 0x7f306c0b1b50 2020-12-05T17:37:50.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.507+0000 7f30697fa700 1 -- 172.21.15.32:0/1010629974 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v72) v1 ==== 110+0+10 (secure 0 0 0) 0x7f306406be60 con 0x7f306c0b1b50 2020-12-05T17:37:50.512 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:37:50.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.507+0000 7f30711f7700 1 -- 172.21.15.32:0/1010629974 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3050055440 msgr2=0x7f3050057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:50.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.507+0000 7f30711f7700 1 --2- 172.21.15.32:0/1010629974 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3050055440 0x7f3050057900 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f305c00b000 tx=0x7f305c006dc0).stop 2020-12-05T17:37:50.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.507+0000 7f30711f7700 1 -- 172.21.15.32:0/1010629974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306c0b1b50 msgr2=0x7f306c13c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:50.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.507+0000 7f30711f7700 1 --2- 172.21.15.32:0/1010629974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306c0b1b50 0x7f306c13c160 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f306c062f80 tx=0x7f3064007ee0).stop 2020-12-05T17:37:50.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.507+0000 7f30711f7700 1 -- 172.21.15.32:0/1010629974 shutdown_connections 2020-12-05T17:37:50.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.507+0000 7f30711f7700 1 --2- 172.21.15.32:0/1010629974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306c0b1b50 0x7f306c13c160 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.507+0000 7f30711f7700 1 --2- 172.21.15.32:0/1010629974 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3050055440 0x7f3050057900 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.507+0000 7f30711f7700 1 --2- 172.21.15.32:0/1010629974 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f306c0afef0 0x7f306c13a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.507+0000 7f30711f7700 1 --2- 172.21.15.32:0/1010629974 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f306c0b12d0 0x7f306c144e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.507+0000 7f30711f7700 1 -- 172.21.15.32:0/1010629974 >> 172.21.15.32:0/1010629974 conn(0x7f306c05bf80 msgr2=0x7f306c05fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:50.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.507+0000 7f30711f7700 1 -- 172.21.15.32:0/1010629974 shutdown_connections 2020-12-05T17:37:50.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.507+0000 7f30711f7700 1 -- 172.21.15.32:0/1010629974 wait complete. 2020-12-05T17:37:50.527 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:37:50.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.643+0000 7f5882a65700 1 Processor -- start 2020-12-05T17:37:50.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882a65700 1 -- start start 2020-12-05T17:37:50.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882a65700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f587c061ba0 0x7f587c066e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882a65700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f587c0b3020 0x7f587c143c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882a65700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f587c0625c0 0x7f587c144160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882a65700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f587c0b3540 con 0x7f587c0625c0 2020-12-05T17:37:50.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882a65700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f587c061f80 con 0x7f587c0b3020 2020-12-05T17:37:50.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882a65700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587c144650 con 0x7f587c061ba0 2020-12-05T17:37:50.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882264700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f587c0625c0 0x7f587c144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882264700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f587c0625c0 0x7f587c144160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36774/0 (socket says 172.21.15.32:36774) 2020-12-05T17:37:50.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882264700 1 -- 172.21.15.32:0/695785043 learned_addr learned my addr 172.21.15.32:0/695785043 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:50.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5881262700 1 --2- 172.21.15.32:0/695785043 >> v2:172.21.15.61:3300/0 conn(0x7f587c0b3020 0x7f587c143c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5881a63700 1 --2- 172.21.15.32:0/695785043 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f587c061ba0 0x7f587c066e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882264700 1 -- 172.21.15.32:0/695785043 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f587c061ba0 msgr2=0x7f587c066e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:50.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882264700 1 --2- 172.21.15.32:0/695785043 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f587c061ba0 0x7f587c066e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882264700 1 -- 172.21.15.32:0/695785043 >> v2:172.21.15.61:3300/0 conn(0x7f587c0b3020 msgr2=0x7f587c143c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:50.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882264700 1 --2- 172.21.15.32:0/695785043 >> v2:172.21.15.61:3300/0 conn(0x7f587c0b3020 0x7f587c143c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882264700 1 -- 172.21.15.32:0/695785043 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c066750 con 0x7f587c0625c0 2020-12-05T17:37:50.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5881a63700 1 --2- 172.21.15.32:0/695785043 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f587c061ba0 0x7f587c066e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:50.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882264700 1 --2- 172.21.15.32:0/695785043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f587c0625c0 0x7f587c144160 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7f58740032b0 tx=0x7f5874009270).ready entity=mon.0 client_cookie=2c245e3bdebca1ec server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:50.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5880a61700 1 -- 172.21.15.32:0/695785043 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5874009e60 con 0x7f587c0625c0 2020-12-05T17:37:50.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5880a61700 1 -- 172.21.15.32:0/695785043 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587400bc00 con 0x7f587c0625c0 2020-12-05T17:37:50.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5880a61700 1 -- 172.21.15.32:0/695785043 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f587400bd70 con 0x7f587c0625c0 2020-12-05T17:37:50.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882a65700 1 -- 172.21.15.32:0/695785043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f587c0625c0 msgr2=0x7f587c144160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:50.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882a65700 1 --2- 172.21.15.32:0/695785043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f587c0625c0 0x7f587c144160 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7f58740032b0 tx=0x7f5874009270).stop 2020-12-05T17:37:50.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882a65700 1 -- 172.21.15.32:0/695785043 shutdown_connections 2020-12-05T17:37:50.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882a65700 1 --2- 172.21.15.32:0/695785043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f587c0625c0 0x7f587c144160 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882a65700 1 --2- 172.21.15.32:0/695785043 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f587c061ba0 0x7f587c066e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882a65700 1 --2- 172.21.15.32:0/695785043 >> v2:172.21.15.61:3300/0 conn(0x7f587c0b3020 0x7f587c143c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882a65700 1 -- 172.21.15.32:0/695785043 >> 172.21.15.32:0/695785043 conn(0x7f587c0b6d10 msgr2=0x7f587c0b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:50.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882a65700 1 -- 172.21.15.32:0/695785043 shutdown_connections 2020-12-05T17:37:50.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.647+0000 7f5882a65700 1 -- 172.21.15.32:0/695785043 wait complete. 2020-12-05T17:37:50.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f5882a65700 1 Processor -- start 2020-12-05T17:37:50.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f5882a65700 1 -- start start 2020-12-05T17:37:50.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f5882a65700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f587c061ba0 0x7f587c13bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f5882a65700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f587c0625c0 0x7f587c152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f5882a65700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f587c0b3020 0x7f587c0b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f5882a65700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587c13a360 con 0x7f587c0625c0 2020-12-05T17:37:50.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f5882a65700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587c0ad790 con 0x7f587c061ba0 2020-12-05T17:37:50.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f5882a65700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587c0ada30 con 0x7f587c0b3020 2020-12-05T17:37:50.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f5881262700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f587c0625c0 0x7f587c152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f5881262700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f587c0625c0 0x7f587c152800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36778/0 (socket says 172.21.15.32:36778) 2020-12-05T17:37:50.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f5881262700 1 -- 172.21.15.32:0/3900847006 learned_addr learned my addr 172.21.15.32:0/3900847006 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:50.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f5881a63700 1 --2- 172.21.15.32:0/3900847006 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f587c061ba0 0x7f587c13bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f5882264700 1 --2- 172.21.15.32:0/3900847006 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f587c0b3020 0x7f587c0b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f5881262700 1 -- 172.21.15.32:0/3900847006 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f587c0b3020 msgr2=0x7f587c0b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:50.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f5881262700 1 --2- 172.21.15.32:0/3900847006 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f587c0b3020 0x7f587c0b0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f5881262700 1 -- 172.21.15.32:0/3900847006 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f587c061ba0 msgr2=0x7f587c13bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:50.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f5881262700 1 --2- 172.21.15.32:0/3900847006 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f587c061ba0 0x7f587c13bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f5881262700 1 -- 172.21.15.32:0/3900847006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c0ad410 con 0x7f587c0625c0 2020-12-05T17:37:50.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f5881262700 1 --2- 172.21.15.32:0/3900847006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f587c0625c0 0x7f587c152800 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7f587800c000 tx=0x7f5878006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:50.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f586affd700 1 -- 172.21.15.32:0/3900847006 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f587800ca10 con 0x7f587c0625c0 2020-12-05T17:37:50.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f586affd700 1 -- 172.21.15.32:0/3900847006 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587800cb70 con 0x7f587c0625c0 2020-12-05T17:37:50.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f586affd700 1 -- 172.21.15.32:0/3900847006 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f587800be50 con 0x7f587c0625c0 2020-12-05T17:37:50.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f5882a65700 1 -- 172.21.15.32:0/3900847006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f587c0b0a70 con 0x7f587c0625c0 2020-12-05T17:37:50.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.651+0000 7f5882a65700 1 -- 172.21.15.32:0/3900847006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f587c140bd0 con 0x7f587c0625c0 2020-12-05T17:37:50.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.655+0000 7f5882a65700 1 -- 172.21.15.32:0/3900847006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f587c0ad600 con 0x7f587c0625c0 2020-12-05T17:37:50.675 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.655+0000 7f586affd700 1 -- 172.21.15.32:0/3900847006 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f587801b410 con 0x7f587c0625c0 2020-12-05T17:37:50.675 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.655+0000 7f586affd700 1 --2- 172.21.15.32:0/3900847006 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f585c055440 0x7f585c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.675 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.655+0000 7f5881a63700 1 --2- 172.21.15.32:0/3900847006 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f585c055440 0x7f585c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.675 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.655+0000 7f586affd700 1 -- 172.21.15.32:0/3900847006 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f587806e130 con 0x7f587c0625c0 2020-12-05T17:37:50.676 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.655+0000 7f5881a63700 1 --2- 172.21.15.32:0/3900847006 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f585c055440 0x7f585c057900 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f586c00c000 tx=0x7f586c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:50.676 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.663+0000 7f586affd700 1 -- 172.21.15.32:0/3900847006 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5878044270 con 0x7f587c0625c0 2020-12-05T17:37:50.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.803+0000 7f5882a65700 1 -- 172.21.15.32:0/3900847006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f587c057050 con 0x7f587c0625c0 2020-12-05T17:37:50.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.807+0000 7f586affd700 1 -- 172.21.15.32:0/3900847006 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v72) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5878047890 con 0x7f587c0625c0 2020-12-05T17:37:50.810 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:37:50.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.807+0000 7f5882a65700 1 -- 172.21.15.32:0/3900847006 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f585c055440 msgr2=0x7f585c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:50.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.807+0000 7f5882a65700 1 --2- 172.21.15.32:0/3900847006 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f585c055440 0x7f585c057900 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f586c00c000 tx=0x7f586c008040).stop 2020-12-05T17:37:50.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.807+0000 7f5882a65700 1 -- 172.21.15.32:0/3900847006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f587c0625c0 msgr2=0x7f587c152800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:50.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.807+0000 7f5882a65700 1 --2- 172.21.15.32:0/3900847006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f587c0625c0 0x7f587c152800 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7f587800c000 tx=0x7f5878006dc0).stop 2020-12-05T17:37:50.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.807+0000 7f5882a65700 1 -- 172.21.15.32:0/3900847006 shutdown_connections 2020-12-05T17:37:50.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.807+0000 7f5882a65700 1 --2- 172.21.15.32:0/3900847006 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f585c055440 0x7f585c057900 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.807+0000 7f5882a65700 1 --2- 172.21.15.32:0/3900847006 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f587c0b3020 0x7f587c0b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.807+0000 7f5882a65700 1 --2- 172.21.15.32:0/3900847006 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f587c061ba0 0x7f587c13bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.807+0000 7f5882a65700 1 --2- 172.21.15.32:0/3900847006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f587c0625c0 0x7f587c152800 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.807+0000 7f5882a65700 1 -- 172.21.15.32:0/3900847006 >> 172.21.15.32:0/3900847006 conn(0x7f587c0b6d10 msgr2=0x7f587c0b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:50.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.807+0000 7f5882a65700 1 -- 172.21.15.32:0/3900847006 shutdown_connections 2020-12-05T17:37:50.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.807+0000 7f5882a65700 1 -- 172.21.15.32:0/3900847006 wait complete. 2020-12-05T17:37:50.824 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:37:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.939+0000 7fb6d7143700 1 Processor -- start 2020-12-05T17:37:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.939+0000 7fb6d7143700 1 -- start start 2020-12-05T17:37:50.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d7143700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb6d005ce60 0x7fb6d013ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d7143700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb6d00ad220 0x7fb6d013ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d7143700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6d0142cc0 0x7fb6d00b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d7143700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d0143c20 con 0x7fb6d005ce60 2020-12-05T17:37:50.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d7143700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d00b0ca0 con 0x7fb6d00ad220 2020-12-05T17:37:50.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d7143700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d005d240 con 0x7fb6d0142cc0 2020-12-05T17:37:50.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d4edf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb6d005ce60 0x7fb6d013ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d4edf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb6d005ce60 0x7fb6d013ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36784/0 (socket says 172.21.15.32:36784) 2020-12-05T17:37:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d4edf700 1 -- 172.21.15.32:0/3619792535 learned_addr learned my addr 172.21.15.32:0/3619792535 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d56e0700 1 --2- 172.21.15.32:0/3619792535 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6d0142cc0 0x7fb6d00b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6cffff700 1 --2- 172.21.15.32:0/3619792535 >> v2:172.21.15.61:3300/0 conn(0x7fb6d00ad220 0x7fb6d013ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d4edf700 1 -- 172.21.15.32:0/3619792535 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6d0142cc0 msgr2=0x7fb6d00b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d4edf700 1 --2- 172.21.15.32:0/3619792535 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6d0142cc0 0x7fb6d00b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d4edf700 1 -- 172.21.15.32:0/3619792535 >> v2:172.21.15.61:3300/0 conn(0x7fb6d00ad220 msgr2=0x7fb6d013ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d4edf700 1 --2- 172.21.15.32:0/3619792535 >> v2:172.21.15.61:3300/0 conn(0x7fb6d00ad220 0x7fb6d013ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d4edf700 1 -- 172.21.15.32:0/3619792535 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d0143240 con 0x7fb6d005ce60 2020-12-05T17:37:50.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d4edf700 1 --2- 172.21.15.32:0/3619792535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6d005ce60 0x7fb6d013ea70 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7fb6c00032b0 tx=0x7fb6c0009270).ready entity=mon.0 client_cookie=81a610d3f49f682a server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:50.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6cf7fe700 1 -- 172.21.15.32:0/3619792535 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6c0009e60 con 0x7fb6d005ce60 2020-12-05T17:37:50.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6cf7fe700 1 -- 172.21.15.32:0/3619792535 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c000bb50 con 0x7fb6d005ce60 2020-12-05T17:37:50.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6cf7fe700 1 -- 172.21.15.32:0/3619792535 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6c000bcc0 con 0x7fb6d005ce60 2020-12-05T17:37:50.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d7143700 1 -- 172.21.15.32:0/3619792535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6d005ce60 msgr2=0x7fb6d013ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:50.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d7143700 1 --2- 172.21.15.32:0/3619792535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6d005ce60 0x7fb6d013ea70 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7fb6c00032b0 tx=0x7fb6c0009270).stop 2020-12-05T17:37:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d7143700 1 -- 172.21.15.32:0/3619792535 shutdown_connections 2020-12-05T17:37:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d7143700 1 --2- 172.21.15.32:0/3619792535 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6d0142cc0 0x7fb6d00b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d7143700 1 --2- 172.21.15.32:0/3619792535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6d005ce60 0x7fb6d013ea70 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d7143700 1 --2- 172.21.15.32:0/3619792535 >> v2:172.21.15.61:3300/0 conn(0x7fb6d00ad220 0x7fb6d013ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d7143700 1 -- 172.21.15.32:0/3619792535 >> 172.21.15.32:0/3619792535 conn(0x7fb6d0069cb0 msgr2=0x7fb6d00604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d7143700 1 -- 172.21.15.32:0/3619792535 shutdown_connections 2020-12-05T17:37:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.943+0000 7fb6d7143700 1 -- 172.21.15.32:0/3619792535 wait complete. 2020-12-05T17:37:50.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.947+0000 7fb6d7143700 1 Processor -- start 2020-12-05T17:37:50.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.947+0000 7fb6d7143700 1 -- start start 2020-12-05T17:37:50.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.947+0000 7fb6d7143700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6d005ce60 0x7fb6d013eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.947+0000 7fb6d7143700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6d00ad220 0x7fb6d00b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.947+0000 7fb6d7143700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6d0142cc0 0x7fb6d00b5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.947+0000 7fb6d7143700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d00b3c00 con 0x7fb6d0142cc0 2020-12-05T17:37:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.947+0000 7fb6d7143700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d00b3ea0 con 0x7fb6d005ce60 2020-12-05T17:37:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.947+0000 7fb6d7143700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d00b4140 con 0x7fb6d00ad220 2020-12-05T17:37:50.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.947+0000 7fb6d56e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6d0142cc0 0x7fb6d00b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.947+0000 7fb6d4edf700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6d005ce60 0x7fb6d013eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.947+0000 7fb6d56e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6d0142cc0 0x7fb6d00b5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36792/0 (socket says 172.21.15.32:36792) 2020-12-05T17:37:50.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.947+0000 7fb6d56e0700 1 -- 172.21.15.32:0/3003500352 learned_addr learned my addr 172.21.15.32:0/3003500352 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:50.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.947+0000 7fb6cffff700 1 --2- 172.21.15.32:0/3003500352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6d00ad220 0x7fb6d00b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.951+0000 7fb6d56e0700 1 -- 172.21.15.32:0/3003500352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6d00ad220 msgr2=0x7fb6d00b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:50.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.951+0000 7fb6d56e0700 1 --2- 172.21.15.32:0/3003500352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6d00ad220 0x7fb6d00b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.951+0000 7fb6d56e0700 1 -- 172.21.15.32:0/3003500352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6d005ce60 msgr2=0x7fb6d013eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:50.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.951+0000 7fb6d56e0700 1 --2- 172.21.15.32:0/3003500352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6d005ce60 0x7fb6d013eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:50.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.951+0000 7fb6d56e0700 1 -- 172.21.15.32:0/3003500352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d00b45e0 con 0x7fb6d0142cc0 2020-12-05T17:37:50.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.951+0000 7fb6d56e0700 1 --2- 172.21.15.32:0/3003500352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6d0142cc0 0x7fb6d00b5820 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7fb6c8006a50 tx=0x7fb6c8006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:50.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.951+0000 7fb6cdffb700 1 -- 172.21.15.32:0/3003500352 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6c800c7e0 con 0x7fb6d0142cc0 2020-12-05T17:37:50.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.951+0000 7fb6cdffb700 1 -- 172.21.15.32:0/3003500352 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c800c940 con 0x7fb6d0142cc0 2020-12-05T17:37:50.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.951+0000 7fb6d7143700 1 -- 172.21.15.32:0/3003500352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d013f9b0 con 0x7fb6d0142cc0 2020-12-05T17:37:50.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.951+0000 7fb6cdffb700 1 -- 172.21.15.32:0/3003500352 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6c800cca0 con 0x7fb6d0142cc0 2020-12-05T17:37:50.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.951+0000 7fb6d7143700 1 -- 172.21.15.32:0/3003500352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d01418b0 con 0x7fb6d0142cc0 2020-12-05T17:37:50.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.951+0000 7fb6cdffb700 1 -- 172.21.15.32:0/3003500352 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb6c801c460 con 0x7fb6d0142cc0 2020-12-05T17:37:50.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.951+0000 7fb6d7143700 1 -- 172.21.15.32:0/3003500352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d00b4300 con 0x7fb6d0142cc0 2020-12-05T17:37:50.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.951+0000 7fb6cdffb700 1 --2- 172.21.15.32:0/3003500352 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb6b00553b0 0x7fb6b0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:50.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.951+0000 7fb6cdffb700 1 -- 172.21.15.32:0/3003500352 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb6c806e3a0 con 0x7fb6d0142cc0 2020-12-05T17:37:50.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.951+0000 7fb6d4edf700 1 --2- 172.21.15.32:0/3003500352 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb6b00553b0 0x7fb6b0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:50.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.951+0000 7fb6d4edf700 1 --2- 172.21.15.32:0/3003500352 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb6b00553b0 0x7fb6b0057870 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7fb6c001a000 tx=0x7fb6c0016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:50.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:50.959+0000 7fb6cdffb700 1 -- 172.21.15.32:0/3003500352 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb6c80444e0 con 0x7fb6d0142cc0 2020-12-05T17:37:51.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.099+0000 7fb6d7143700 1 -- 172.21.15.32:0/3003500352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb6d00b6af0 con 0x7fb6d0142cc0 2020-12-05T17:37:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.099+0000 7fb6cdffb700 1 -- 172.21.15.32:0/3003500352 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) v1 ==== 104+0+10 (secure 0 0 0) 0x7fb6c8047b00 con 0x7fb6d0142cc0 2020-12-05T17:37:51.105 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:37:51.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.103+0000 7fb6d7143700 1 -- 172.21.15.32:0/3003500352 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb6b00553b0 msgr2=0x7fb6b0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:51.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.103+0000 7fb6d7143700 1 --2- 172.21.15.32:0/3003500352 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb6b00553b0 0x7fb6b0057870 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7fb6c001a000 tx=0x7fb6c0016000).stop 2020-12-05T17:37:51.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.103+0000 7fb6d7143700 1 -- 172.21.15.32:0/3003500352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6d0142cc0 msgr2=0x7fb6d00b5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:51.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.103+0000 7fb6d7143700 1 --2- 172.21.15.32:0/3003500352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6d0142cc0 0x7fb6d00b5820 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7fb6c8006a50 tx=0x7fb6c8006dc0).stop 2020-12-05T17:37:51.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.103+0000 7fb6d7143700 1 -- 172.21.15.32:0/3003500352 shutdown_connections 2020-12-05T17:37:51.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.103+0000 7fb6d7143700 1 --2- 172.21.15.32:0/3003500352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6d0142cc0 0x7fb6d00b5820 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:51.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.103+0000 7fb6d7143700 1 --2- 172.21.15.32:0/3003500352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6d005ce60 0x7fb6d013eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:51.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.103+0000 7fb6d7143700 1 --2- 172.21.15.32:0/3003500352 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb6b00553b0 0x7fb6b0057870 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.103+0000 7fb6d7143700 1 --2- 172.21.15.32:0/3003500352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6d00ad220 0x7fb6d00b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.103+0000 7fb6d7143700 1 -- 172.21.15.32:0/3003500352 >> 172.21.15.32:0/3003500352 conn(0x7fb6d0069cb0 msgr2=0x7fb6d0142120 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.103+0000 7fb6d7143700 1 -- 172.21.15.32:0/3003500352 shutdown_connections 2020-12-05T17:37:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.103+0000 7fb6d7143700 1 -- 172.21.15.32:0/3003500352 wait complete. 2020-12-05T17:37:51.120 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:37:51.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.231+0000 7fd603232700 1 Processor -- start 2020-12-05T17:37:51.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.231+0000 7fd603232700 1 -- start start 2020-12-05T17:37:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.235+0000 7fd603232700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5fc0af0d0 0x7fd5fc13daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.235+0000 7fd603232700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd5fc13df90 0x7fd5fc0637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.235+0000 7fd603232700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5fc05f5c0 con 0x7fd5fc0b0d70 2020-12-05T17:37:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.235+0000 7fd603232700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5fc0af4b0 con 0x7fd5fc0af0d0 2020-12-05T17:37:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.235+0000 7fd603232700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5fc05f7f0 con 0x7fd5fc13df90 2020-12-05T17:37:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.235+0000 7fd602230700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5fc0af0d0 0x7fd5fc13daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.235+0000 7fd602230700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5fc0af0d0 0x7fd5fc13daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36798/0 (socket says 172.21.15.32:36798) 2020-12-05T17:37:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.235+0000 7fd602230700 1 -- 172.21.15.32:0/4227448743 learned_addr learned my addr 172.21.15.32:0/4227448743 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:51.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.235+0000 7fd601a2f700 1 --2- 172.21.15.32:0/4227448743 >> v2:172.21.15.61:3300/0 conn(0x7fd5fc13df90 0x7fd5fc0637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:51.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.235+0000 7fd602230700 1 -- 172.21.15.32:0/4227448743 >> v2:172.21.15.61:3300/0 conn(0x7fd5fc13df90 msgr2=0x7fd5fc0637f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:51.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.235+0000 7fd602230700 1 --2- 172.21.15.32:0/4227448743 >> v2:172.21.15.61:3300/0 conn(0x7fd5fc13df90 0x7fd5fc0637f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:51.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.235+0000 7fd602230700 1 -- 172.21.15.32:0/4227448743 >> v1:172.21.15.61:6789/0 conn(0x7fd5fc0b0d70 legacy=0x7fd5fc063ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:51.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.235+0000 7fd602230700 1 -- 172.21.15.32:0/4227448743 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc066ae0 con 0x7fd5fc0af0d0 2020-12-05T17:37:51.244 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.235+0000 7fd602230700 1 --2- 172.21.15.32:0/4227448743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc0af0d0 0x7fd5fc13daa0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7fd5f8002f30 tx=0x7fd5f8006e80).ready entity=mon.0 client_cookie=2bcd7ff151e0408a server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:51.244 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.235+0000 7fd60122e700 1 -- 172.21.15.32:0/4227448743 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5f80097f0 con 0x7fd5fc0af0d0 2020-12-05T17:37:51.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.235+0000 7fd60122e700 1 -- 172.21.15.32:0/4227448743 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f8009950 con 0x7fd5fc0af0d0 2020-12-05T17:37:51.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.235+0000 7fd60122e700 1 -- 172.21.15.32:0/4227448743 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5f8009cb0 con 0x7fd5fc0af0d0 2020-12-05T17:37:51.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.235+0000 7fd603232700 1 -- 172.21.15.32:0/4227448743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc0af0d0 msgr2=0x7fd5fc13daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:51.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.235+0000 7fd603232700 1 --2- 172.21.15.32:0/4227448743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc0af0d0 0x7fd5fc13daa0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7fd5f8002f30 tx=0x7fd5f8006e80).stop 2020-12-05T17:37:51.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.239+0000 7fd603232700 1 -- 172.21.15.32:0/4227448743 shutdown_connections 2020-12-05T17:37:51.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.239+0000 7fd603232700 1 --2- 172.21.15.32:0/4227448743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc0af0d0 0x7fd5fc13daa0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:51.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.239+0000 7fd603232700 1 --2- 172.21.15.32:0/4227448743 >> v2:172.21.15.61:3300/0 conn(0x7fd5fc13df90 0x7fd5fc0637f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:51.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.239+0000 7fd603232700 1 -- 172.21.15.32:0/4227448743 >> 172.21.15.32:0/4227448743 conn(0x7fd5fc0620b0 msgr2=0x7fd5fc062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.239+0000 7fd603232700 1 -- 172.21.15.32:0/4227448743 shutdown_connections 2020-12-05T17:37:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.239+0000 7fd603232700 1 -- 172.21.15.32:0/4227448743 wait complete. 2020-12-05T17:37:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.239+0000 7fd603232700 1 Processor -- start 2020-12-05T17:37:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.239+0000 7fd603232700 1 -- start start 2020-12-05T17:37:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.239+0000 7fd603232700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc0af0d0 0x7fd5fc0b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.239+0000 7fd603232700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd5fc0b0d70 0x7fd5fc064e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:51.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.239+0000 7fd603232700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd5fc13df90 0x7fd5fc13ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:51.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.239+0000 7fd603232700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc13ead0 con 0x7fd5fc0af0d0 2020-12-05T17:37:51.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.239+0000 7fd603232700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc142ec0 con 0x7fd5fc0b0d70 2020-12-05T17:37:51.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.239+0000 7fd603232700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc143160 con 0x7fd5fc13df90 2020-12-05T17:37:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.239+0000 7fd602230700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc0af0d0 0x7fd5fc0b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.239+0000 7fd602230700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc0af0d0 0x7fd5fc0b4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36804/0 (socket says 172.21.15.32:36804) 2020-12-05T17:37:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.239+0000 7fd602230700 1 -- 172.21.15.32:0/442048075 learned_addr learned my addr 172.21.15.32:0/442048075 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.243+0000 7fd601a2f700 1 --2- 172.21.15.32:0/442048075 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd5fc0b0d70 0x7fd5fc064e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:51.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.243+0000 7fd602a31700 1 --2- 172.21.15.32:0/442048075 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd5fc13df90 0x7fd5fc13ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:51.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.243+0000 7fd602230700 1 -- 172.21.15.32:0/442048075 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd5fc13df90 msgr2=0x7fd5fc13ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:51.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.243+0000 7fd602230700 1 --2- 172.21.15.32:0/442048075 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd5fc13df90 0x7fd5fc13ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:51.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.243+0000 7fd602230700 1 -- 172.21.15.32:0/442048075 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd5fc0b0d70 msgr2=0x7fd5fc064e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:51.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.243+0000 7fd602230700 1 --2- 172.21.15.32:0/442048075 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd5fc0b0d70 0x7fd5fc064e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:51.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.243+0000 7fd602230700 1 -- 172.21.15.32:0/442048075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc1394c0 con 0x7fd5fc0af0d0 2020-12-05T17:37:51.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.243+0000 7fd602230700 1 --2- 172.21.15.32:0/442048075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc0af0d0 0x7fd5fc0b4930 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7fd5f8002f30 tx=0x7fd5f8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:51.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.243+0000 7fd5eb7fe700 1 -- 172.21.15.32:0/442048075 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5f8011c30 con 0x7fd5fc0af0d0 2020-12-05T17:37:51.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.243+0000 7fd5eb7fe700 1 -- 172.21.15.32:0/442048075 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f8011d90 con 0x7fd5fc0af0d0 2020-12-05T17:37:51.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.243+0000 7fd603232700 1 -- 172.21.15.32:0/442048075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5fc065300 con 0x7fd5fc0af0d0 2020-12-05T17:37:51.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.243+0000 7fd5eb7fe700 1 -- 172.21.15.32:0/442048075 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5f801e650 con 0x7fd5fc0af0d0 2020-12-05T17:37:51.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.243+0000 7fd603232700 1 -- 172.21.15.32:0/442048075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5fc143b20 con 0x7fd5fc0af0d0 2020-12-05T17:37:51.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.243+0000 7fd5eb7fe700 1 -- 172.21.15.32:0/442048075 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd5f8025030 con 0x7fd5fc0af0d0 2020-12-05T17:37:51.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.243+0000 7fd603232700 1 -- 172.21.15.32:0/442048075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5fc13e6d0 con 0x7fd5fc0af0d0 2020-12-05T17:37:51.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.243+0000 7fd5eb7fe700 1 --2- 172.21.15.32:0/442048075 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd5dc0554e0 0x7fd5dc0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:51.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.243+0000 7fd5eb7fe700 1 -- 172.21.15.32:0/442048075 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd5f806f330 con 0x7fd5fc0af0d0 2020-12-05T17:37:51.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.243+0000 7fd601a2f700 1 --2- 172.21.15.32:0/442048075 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd5dc0554e0 0x7fd5dc0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:51.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.247+0000 7fd601a2f700 1 --2- 172.21.15.32:0/442048075 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd5dc0554e0 0x7fd5dc0579a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fd5ec00b000 tx=0x7fd5ec006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:51.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.251+0000 7fd5eb7fe700 1 -- 172.21.15.32:0/442048075 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd5fc13e6d0 con 0x7fd5fc0af0d0 2020-12-05T17:37:51.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.391+0000 7fd603232700 1 -- 172.21.15.32:0/442048075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd5fc143b20 con 0x7fd5fc0af0d0 2020-12-05T17:37:51.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.391+0000 7fd5eb7fe700 1 -- 172.21.15.32:0/442048075 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) v1 ==== 100+0+10 (secure 0 0 0) 0x7fd5fc143b20 con 0x7fd5fc0af0d0 2020-12-05T17:37:51.397 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:37:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.395+0000 7fd603232700 1 -- 172.21.15.32:0/442048075 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd5dc0554e0 msgr2=0x7fd5dc0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.395+0000 7fd603232700 1 --2- 172.21.15.32:0/442048075 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd5dc0554e0 0x7fd5dc0579a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fd5ec00b000 tx=0x7fd5ec006dc0).stop 2020-12-05T17:37:51.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.395+0000 7fd603232700 1 -- 172.21.15.32:0/442048075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc0af0d0 msgr2=0x7fd5fc0b4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:51.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.395+0000 7fd603232700 1 --2- 172.21.15.32:0/442048075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc0af0d0 0x7fd5fc0b4930 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7fd5f8002f30 tx=0x7fd5f8015000).stop 2020-12-05T17:37:51.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.395+0000 7fd603232700 1 -- 172.21.15.32:0/442048075 shutdown_connections 2020-12-05T17:37:51.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.395+0000 7fd603232700 1 --2- 172.21.15.32:0/442048075 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd5dc0554e0 0x7fd5dc0579a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:51.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.395+0000 7fd603232700 1 --2- 172.21.15.32:0/442048075 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd5fc13df90 0x7fd5fc13ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:51.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.395+0000 7fd603232700 1 --2- 172.21.15.32:0/442048075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc0af0d0 0x7fd5fc0b4930 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:51.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.395+0000 7fd603232700 1 --2- 172.21.15.32:0/442048075 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd5fc0b0d70 0x7fd5fc064e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:51.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.395+0000 7fd603232700 1 -- 172.21.15.32:0/442048075 >> 172.21.15.32:0/442048075 conn(0x7fd5fc0620b0 msgr2=0x7fd5fc0b5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:51.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.395+0000 7fd603232700 1 -- 172.21.15.32:0/442048075 shutdown_connections 2020-12-05T17:37:51.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.395+0000 7fd603232700 1 -- 172.21.15.32:0/442048075 wait complete. 2020-12-05T17:37:51.411 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:37:51.411 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:37:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.515+0000 7f1a9b166700 1 Processor -- start 2020-12-05T17:37:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.515+0000 7f1a9b166700 1 -- start start 2020-12-05T17:37:51.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.515+0000 7f1a9b166700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a9405f6f0 0x7f1a941428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:51.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.515+0000 7f1a9b166700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1a940b0880 0x7f1a94066070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:51.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.515+0000 7f1a9b166700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a9405fad0 con 0x7f1a940b1980 2020-12-05T17:37:51.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.515+0000 7f1a9b166700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a940b2f80 con 0x7f1a940b0880 2020-12-05T17:37:51.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.515+0000 7f1a9b166700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a94145e40 con 0x7f1a9405f6f0 2020-12-05T17:37:51.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.515+0000 7f1a99963700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1a940b1980 0x7f1a94146580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39462/0 (socket says 172.21.15.32:39462) 2020-12-05T17:37:51.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.515+0000 7f1a9a965700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1a940b0880 0x7f1a94066070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:51.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.515+0000 7f1a99963700 1 -- 172.21.15.32:0/1836370391 learned_addr learned my addr 172.21.15.32:0/1836370391 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:51.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.515+0000 7f1a9a164700 1 --2- 172.21.15.32:0/1836370391 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a9405f6f0 0x7f1a941428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:51.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.515+0000 7f1a9a965700 1 -- 172.21.15.32:0/1836370391 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a9405f6f0 msgr2=0x7f1a941428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:51.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.515+0000 7f1a9a965700 1 --2- 172.21.15.32:0/1836370391 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a9405f6f0 0x7f1a941428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:51.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.515+0000 7f1a9a965700 1 -- 172.21.15.32:0/1836370391 >> v1:172.21.15.32:6789/0 conn(0x7f1a940b1980 legacy=0x7f1a94146580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:51.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.515+0000 7f1a9a965700 1 -- 172.21.15.32:0/1836370391 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a941460e0 con 0x7f1a940b0880 2020-12-05T17:37:51.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.515+0000 7f1a99162700 1 -- 172.21.15.32:0/1836370391 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491364545 0 0) 0x7f1a9405fad0 con 0x7f1a940b1980 2020-12-05T17:37:51.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.515+0000 7f1a9a965700 1 --2- 172.21.15.32:0/1836370391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a940b0880 0x7f1a94066070 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7f1a8c0032b0 tx=0x7f1a8c0091a0).ready entity=mon.0 client_cookie=e139ed713f432ed3 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:51.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.515+0000 7f1a99162700 1 -- 172.21.15.32:0/1836370391 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a8c00b9a0 con 0x7f1a940b0880 2020-12-05T17:37:51.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.515+0000 7f1a99162700 1 -- 172.21.15.32:0/1836370391 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c00bb00 con 0x7f1a940b0880 2020-12-05T17:37:51.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.515+0000 7f1a99162700 1 -- 172.21.15.32:0/1836370391 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a8c00bc70 con 0x7f1a940b0880 2020-12-05T17:37:51.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.515+0000 7f1a9b166700 1 -- 172.21.15.32:0/1836370391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a940b0880 msgr2=0x7f1a94066070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:51.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.515+0000 7f1a9b166700 1 --2- 172.21.15.32:0/1836370391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a940b0880 0x7f1a94066070 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7f1a8c0032b0 tx=0x7f1a8c0091a0).stop 2020-12-05T17:37:51.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.519+0000 7f1a9b166700 1 -- 172.21.15.32:0/1836370391 shutdown_connections 2020-12-05T17:37:51.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.519+0000 7f1a9b166700 1 --2- 172.21.15.32:0/1836370391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a940b0880 0x7f1a94066070 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:51.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.519+0000 7f1a9b166700 1 --2- 172.21.15.32:0/1836370391 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a9405f6f0 0x7f1a941428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:51.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.519+0000 7f1a9b166700 1 -- 172.21.15.32:0/1836370391 >> 172.21.15.32:0/1836370391 conn(0x7f1a940604c0 msgr2=0x7f1a9405c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:51.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.519+0000 7f1a9b166700 1 -- 172.21.15.32:0/1836370391 shutdown_connections 2020-12-05T17:37:51.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.519+0000 7f1a9b166700 1 -- 172.21.15.32:0/1836370391 wait complete. 2020-12-05T17:37:51.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.519+0000 7f1a9b166700 1 Processor -- start 2020-12-05T17:37:51.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.519+0000 7f1a9b166700 1 -- start start 2020-12-05T17:37:51.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.519+0000 7f1a9b166700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a9405f6f0 0x7f1a94113590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:51.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.519+0000 7f1a9b166700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a940b0880 0x7f1a941549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:51.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.519+0000 7f1a9b166700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a940b1980 0x7f1a94064380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:51.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.519+0000 7f1a9b166700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a94114690 con 0x7f1a940b1980 2020-12-05T17:37:51.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.519+0000 7f1a9b166700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a941148c0 con 0x7f1a9405f6f0 2020-12-05T17:37:51.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.519+0000 7f1a9b166700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a94114b60 con 0x7f1a940b0880 2020-12-05T17:37:51.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.519+0000 7f1a9a965700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a940b1980 0x7f1a94064380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:51.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.519+0000 7f1a9a965700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a940b1980 0x7f1a94064380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36822/0 (socket says 172.21.15.32:36822) 2020-12-05T17:37:51.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.519+0000 7f1a9a965700 1 -- 172.21.15.32:0/869561076 learned_addr learned my addr 172.21.15.32:0/869561076 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:51.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.519+0000 7f1a99963700 1 --2- 172.21.15.32:0/869561076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a940b0880 0x7f1a941549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:51.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.519+0000 7f1a9a164700 1 --2- 172.21.15.32:0/869561076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a9405f6f0 0x7f1a94113590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:51.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.523+0000 7f1a9a965700 1 -- 172.21.15.32:0/869561076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a940b0880 msgr2=0x7f1a941549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:51.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.523+0000 7f1a9a965700 1 --2- 172.21.15.32:0/869561076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a940b0880 0x7f1a941549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:51.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.523+0000 7f1a9a965700 1 -- 172.21.15.32:0/869561076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a9405f6f0 msgr2=0x7f1a94113590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:51.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.523+0000 7f1a9a965700 1 --2- 172.21.15.32:0/869561076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a9405f6f0 0x7f1a94113590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:51.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.523+0000 7f1a9a965700 1 -- 172.21.15.32:0/869561076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a94116cd0 con 0x7f1a940b1980 2020-12-05T17:37:51.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.523+0000 7f1a99963700 1 --2- 172.21.15.32:0/869561076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a940b0880 0x7f1a941549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:51.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.523+0000 7f1a9a164700 1 --2- 172.21.15.32:0/869561076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a9405f6f0 0x7f1a94113590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:51.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.523+0000 7f1a9a965700 1 --2- 172.21.15.32:0/869561076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a940b1980 0x7f1a94064380 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f1a8c0032b0 tx=0x7f1a8c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:51.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.523+0000 7f1a837fe700 1 -- 172.21.15.32:0/869561076 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a8c01f3f0 con 0x7f1a940b1980 2020-12-05T17:37:51.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.523+0000 7f1a837fe700 1 -- 172.21.15.32:0/869561076 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c01f550 con 0x7f1a940b1980 2020-12-05T17:37:51.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.523+0000 7f1a9b166700 1 -- 172.21.15.32:0/869561076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a941158d0 con 0x7f1a940b1980 2020-12-05T17:37:51.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.523+0000 7f1a837fe700 1 -- 172.21.15.32:0/869561076 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a8c01f8b0 con 0x7f1a940b1980 2020-12-05T17:37:51.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.523+0000 7f1a9b166700 1 -- 172.21.15.32:0/869561076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a94113b80 con 0x7f1a940b1980 2020-12-05T17:37:51.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.523+0000 7f1a9b166700 1 -- 172.21.15.32:0/869561076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a94116e60 con 0x7f1a940b1980 2020-12-05T17:37:51.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.523+0000 7f1a837fe700 1 -- 172.21.15.32:0/869561076 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1a8c026030 con 0x7f1a940b1980 2020-12-05T17:37:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.523+0000 7f1a837fe700 1 --2- 172.21.15.32:0/869561076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1a740553b0 0x7f1a74057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.523+0000 7f1a837fe700 1 -- 172.21.15.32:0/869561076 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1a8c06ed70 con 0x7f1a940b1980 2020-12-05T17:37:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.523+0000 7f1a9a164700 1 --2- 172.21.15.32:0/869561076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1a740553b0 0x7f1a74057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.523+0000 7f1a9a164700 1 --2- 172.21.15.32:0/869561076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1a740553b0 0x7f1a74057870 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f1a9000b000 tx=0x7f1a90008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.531+0000 7f1a837fe700 1 -- 172.21.15.32:0/869561076 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1a8c044e00 con 0x7f1a940b1980 2020-12-05T17:37:51.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.718+0000 7f1a9b166700 1 -- 172.21.15.32:0/869561076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a94057050 con 0x7f1a940b1980 2020-12-05T17:37:51.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.718+0000 7f1a837fe700 1 -- 172.21.15.32:0/869561076 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f1a8c024090 con 0x7f1a940b1980 2020-12-05T17:37:51.723 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:51.723 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":32,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5645,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/992428276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":992428276},{"type":"v1","addr":"172.21.15.32:6835","nonce":992428276}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:37:46.495876+0000","modified":"2020-12-05T17:37:47.498743+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6160":{"gid":6160,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.61:6833/4214217205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4214217205},{"type":"v1","addr":"172.21.15.61:6833","nonce":4214217205}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-12-05T17:37:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.718+0000 7f1a9b166700 1 -- 172.21.15.32:0/869561076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1a740553b0 msgr2=0x7f1a74057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.718+0000 7f1a9b166700 1 --2- 172.21.15.32:0/869561076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1a740553b0 0x7f1a74057870 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f1a9000b000 tx=0x7f1a90008040).stop 2020-12-05T17:37:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.718+0000 7f1a9b166700 1 -- 172.21.15.32:0/869561076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a940b1980 msgr2=0x7f1a94064380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.718+0000 7f1a9b166700 1 --2- 172.21.15.32:0/869561076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a940b1980 0x7f1a94064380 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f1a8c0032b0 tx=0x7f1a8c016000).stop 2020-12-05T17:37:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.718+0000 7f1a9b166700 1 -- 172.21.15.32:0/869561076 shutdown_connections 2020-12-05T17:37:51.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.718+0000 7f1a9b166700 1 --2- 172.21.15.32:0/869561076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a940b1980 0x7f1a94064380 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:51.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.718+0000 7f1a9b166700 1 --2- 172.21.15.32:0/869561076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a9405f6f0 0x7f1a94113590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:51.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.718+0000 7f1a9b166700 1 --2- 172.21.15.32:0/869561076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1a740553b0 0x7f1a74057870 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:51.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.718+0000 7f1a9b166700 1 --2- 172.21.15.32:0/869561076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a940b0880 0x7f1a941549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:51.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.718+0000 7f1a9b166700 1 -- 172.21.15.32:0/869561076 >> 172.21.15.32:0/869561076 conn(0x7f1a940604c0 msgr2=0x7f1a940b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:51.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.718+0000 7f1a9b166700 1 -- 172.21.15.32:0/869561076 shutdown_connections 2020-12-05T17:37:51.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:51.718+0000 7f1a9b166700 1 -- 172.21.15.32:0/869561076 wait complete. 2020-12-05T17:37:51.727 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 32 2020-12-05T17:37:51.939 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:37:46.495876+0000', 'modified': '2020-12-05T17:37:47.498743+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6160}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6160': {'gid': 6160, 'name': 'b', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.61:6833/4214217205', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6832', 'nonce': 4214217205}, {'type': 'v1', 'addr': '172.21.15.61:6833', 'nonce': 4214217205}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:37:51.940 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:37:51.940 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-12-05T17:37:52.069 INFO:teuthology.orchestra.run.smithi061.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":6160,"want_state":"up:active","state":"up:active","rank_uptime":5.5600963979999998,"mdsmap_epoch":32,"osdmap_epoch":72,"osdmap_epoch_barrier":71,"uptime":9.8927666399999996} 2020-12-05T17:37:52.070 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 6160, "mdsmap_epoch": 32, "osdmap_epoch": 72, "osdmap_epoch_barrier": 71, "rank_uptime": 5.560096398, "state": "up:active", "uptime": 9.89276664, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:37:52.071 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:37:52.071 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-12-05T17:37:52.071 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-12-05T17:37:52.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:52.072 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:37:52.079 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:37:52.138 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:37:52.138 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:37:52.193 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:37:52.195 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:37:52.195 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:52.196 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:37:52.244 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:37:52.245 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:37:52.288 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:37:52.304 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:37:52.305 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:37:52.360 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:37:52.361 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:37:52.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:52.362 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:37:52.366 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[10995]: starting ceph client 2020-12-05T17:37:52.366 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:2020-12-05T17:37:52.361+0000 7fb9dbe810c0 -1 init, newargv = 0x556a3ca37400 newargc=9 2020-12-05T17:37:52.376 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[10995]: starting fuse 2020-12-05T17:37:53.471 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:37:53.487 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:37:53.488 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:37:53.545 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:37:53.547 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:37:53.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:53.548 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:37:53.596 INFO:teuthology.orchestra.run.smithi036.stdout:49 2020-12-05T17:37:53.597 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-12-05T17:37:53.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:53.598 INFO:teuthology.orchestra.run.smithi036:> sudo python3 -c ' 2020-12-05T17:37:53.598 INFO:teuthology.orchestra.run.smithi036:> import glob 2020-12-05T17:37:53.599 INFO:teuthology.orchestra.run.smithi036:> import re 2020-12-05T17:37:53.599 INFO:teuthology.orchestra.run.smithi036:> import os 2020-12-05T17:37:53.600 INFO:teuthology.orchestra.run.smithi036:> import subprocess 2020-12-05T17:37:53.600 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:37:53.600 INFO:teuthology.orchestra.run.smithi036:> def find_socket(client_name): 2020-12-05T17:37:53.601 INFO:teuthology.orchestra.run.smithi036:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-12-05T17:37:53.601 INFO:teuthology.orchestra.run.smithi036:> files = glob.glob(asok_path) 2020-12-05T17:37:53.602 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:37:53.602 INFO:teuthology.orchestra.run.smithi036:> # Given a non-glob path, it better be there 2020-12-05T17:37:53.602 INFO:teuthology.orchestra.run.smithi036:> if "*" not in asok_path: 2020-12-05T17:37:53.603 INFO:teuthology.orchestra.run.smithi036:> assert(len(files) == 1) 2020-12-05T17:37:53.603 INFO:teuthology.orchestra.run.smithi036:> return files[0] 2020-12-05T17:37:53.604 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:37:53.604 INFO:teuthology.orchestra.run.smithi036:> for f in files: 2020-12-05T17:37:53.604 INFO:teuthology.orchestra.run.smithi036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:37:53.605 INFO:teuthology.orchestra.run.smithi036:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:37:53.605 INFO:teuthology.orchestra.run.smithi036:> return f 2020-12-05T17:37:53.606 INFO:teuthology.orchestra.run.smithi036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:37:53.606 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:37:53.607 INFO:teuthology.orchestra.run.smithi036:> print(find_socket("client.0")) 2020-12-05T17:37:53.607 INFO:teuthology.orchestra.run.smithi036:> ' 2020-12-05T17:37:53.687 INFO:teuthology.orchestra.run.smithi036.stdout:/var/run/ceph/ceph-client.0.10995.asok 2020-12-05T17:37:53.693 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.10995.asok 2020-12-05T17:37:53.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:53.694 INFO:teuthology.orchestra.run.smithi036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.10995.asok status 2020-12-05T17:37:53.799 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2020-12-05T17:37:53.799 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": { 2020-12-05T17:37:53.799 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:37:53.799 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:37:53.800 INFO:teuthology.orchestra.run.smithi036.stdout: "entity_id": "0", 2020-12-05T17:37:53.800 INFO:teuthology.orchestra.run.smithi036.stdout: "hostname": "smithi036", 2020-12-05T17:37:53.800 INFO:teuthology.orchestra.run.smithi036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-12-05T17:37:53.800 INFO:teuthology.orchestra.run.smithi036.stdout: "pid": "10995", 2020-12-05T17:37:53.800 INFO:teuthology.orchestra.run.smithi036.stdout: "root": "/" 2020-12-05T17:37:53.801 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:37:53.801 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_count": 0, 2020-12-05T17:37:53.801 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_pinned_count": 0, 2020-12-05T17:37:53.801 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 6411, 2020-12-05T17:37:53.801 INFO:teuthology.orchestra.run.smithi036.stdout: "inst": { 2020-12-05T17:37:53.802 INFO:teuthology.orchestra.run.smithi036.stdout: "name": { 2020-12-05T17:37:53.802 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "client", 2020-12-05T17:37:53.802 INFO:teuthology.orchestra.run.smithi036.stdout: "num": 6411 2020-12-05T17:37:53.802 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:37:53.802 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:37:53.803 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:37:53.803 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:37:53.803 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 892894983 2020-12-05T17:37:53.803 INFO:teuthology.orchestra.run.smithi036.stdout: } 2020-12-05T17:37:53.803 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:37:53.804 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:37:53.804 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:37:53.804 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:37:53.804 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 892894983 2020-12-05T17:37:53.804 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:37:53.805 INFO:teuthology.orchestra.run.smithi036.stdout: "inst_str": "client.6411 172.21.15.36:0/892894983", 2020-12-05T17:37:53.805 INFO:teuthology.orchestra.run.smithi036.stdout: "addr_str": "172.21.15.36:0/892894983", 2020-12-05T17:37:53.805 INFO:teuthology.orchestra.run.smithi036.stdout: "inode_count": 1, 2020-12-05T17:37:53.805 INFO:teuthology.orchestra.run.smithi036.stdout: "mds_epoch": 32, 2020-12-05T17:37:53.805 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch": 72, 2020-12-05T17:37:53.806 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:37:53.806 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklisted": false 2020-12-05T17:37:53.806 INFO:teuthology.orchestra.run.smithi036.stdout:} 2020-12-05T17:37:53.808 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:53.809 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:37:53.809 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:37:53.816 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:37:53.817 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:37:53.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:53.818 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:37:53.875 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:37:53.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.990+0000 7f41ce3a9700 1 Processor -- start 2020-12-05T17:37:53.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.990+0000 7f41ce3a9700 1 -- start start 2020-12-05T17:37:53.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.990+0000 7f41ce3a9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f41c8144540 0x7f41c8061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:53.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.990+0000 7f41ce3a9700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f41c81403f0 0x7f41c813d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:53.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.990+0000 7f41ce3a9700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41c8143a00 con 0x7f41c80b7720 2020-12-05T17:37:53.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.990+0000 7f41ce3a9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41c8140830 con 0x7f41c8144540 2020-12-05T17:37:53.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.990+0000 7f41ce3a9700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41c80b7b00 con 0x7f41c81403f0 2020-12-05T17:37:54.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.994+0000 7f41ccba6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f41c8144540 0x7f41c8061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.994+0000 7f41ccba6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f41c8144540 0x7f41c8061780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36826/0 (socket says 172.21.15.32:36826) 2020-12-05T17:37:54.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.994+0000 7f41ccba6700 1 -- 172.21.15.32:0/3991930798 learned_addr learned my addr 172.21.15.32:0/3991930798 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:54.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.994+0000 7f41cdba8700 1 --2- 172.21.15.32:0/3991930798 >> v2:172.21.15.61:3300/0 conn(0x7f41c81403f0 0x7f41c813d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.994+0000 7f41ccba6700 1 -- 172.21.15.32:0/3991930798 >> v2:172.21.15.61:3300/0 conn(0x7f41c81403f0 msgr2=0x7f41c813d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:54.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.994+0000 7f41ccba6700 1 --2- 172.21.15.32:0/3991930798 >> v2:172.21.15.61:3300/0 conn(0x7f41c81403f0 0x7f41c813d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.994+0000 7f41ccba6700 1 -- 172.21.15.32:0/3991930798 >> v1:172.21.15.61:6789/0 conn(0x7f41c80b7720 legacy=0x7f41c8143e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:54.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.994+0000 7f41ccba6700 1 -- 172.21.15.32:0/3991930798 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c805fe60 con 0x7f41c8144540 2020-12-05T17:37:54.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.994+0000 7f41ccba6700 1 --2- 172.21.15.32:0/3991930798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41c8144540 0x7f41c8061780 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f41bc0032b0 tx=0x7f41bc009270).ready entity=mon.0 client_cookie=f61f2f4d1a3be1da server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:54.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.994+0000 7f41c7fff700 1 -- 172.21.15.32:0/3991930798 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41bc00bc00 con 0x7f41c8144540 2020-12-05T17:37:54.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.994+0000 7f41c7fff700 1 -- 172.21.15.32:0/3991930798 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41bc00bd60 con 0x7f41c8144540 2020-12-05T17:37:54.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.994+0000 7f41c7fff700 1 -- 172.21.15.32:0/3991930798 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41bc011400 con 0x7f41c8144540 2020-12-05T17:37:54.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.994+0000 7f41ce3a9700 1 -- 172.21.15.32:0/3991930798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41c8144540 msgr2=0x7f41c8061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:54.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.994+0000 7f41ce3a9700 1 --2- 172.21.15.32:0/3991930798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41c8144540 0x7f41c8061780 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f41bc0032b0 tx=0x7f41bc009270).stop 2020-12-05T17:37:54.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.994+0000 7f41ce3a9700 1 -- 172.21.15.32:0/3991930798 shutdown_connections 2020-12-05T17:37:54.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.994+0000 7f41ce3a9700 1 --2- 172.21.15.32:0/3991930798 >> v2:172.21.15.61:3300/0 conn(0x7f41c81403f0 0x7f41c813d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.994+0000 7f41ce3a9700 1 --2- 172.21.15.32:0/3991930798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41c8144540 0x7f41c8061780 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.994+0000 7f41ce3a9700 1 -- 172.21.15.32:0/3991930798 >> 172.21.15.32:0/3991930798 conn(0x7f41c805bc20 msgr2=0x7f41c80b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:54.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.994+0000 7f41ce3a9700 1 -- 172.21.15.32:0/3991930798 shutdown_connections 2020-12-05T17:37:54.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.994+0000 7f41ce3a9700 1 -- 172.21.15.32:0/3991930798 wait complete. 2020-12-05T17:37:54.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.994+0000 7f41ce3a9700 1 Processor -- start 2020-12-05T17:37:54.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.994+0000 7f41ce3a9700 1 -- start start 2020-12-05T17:37:54.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.998+0000 7f41ce3a9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f41c80b7720 0x7f41c8146360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:54.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.998+0000 7f41ce3a9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41c81403f0 0x7f41c8152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:54.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.998+0000 7f41ce3a9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f41c8144540 0x7f41c8153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:54.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.998+0000 7f41ce3a9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c813c320 con 0x7f41c81403f0 2020-12-05T17:37:54.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.998+0000 7f41ce3a9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c813b310 con 0x7f41c80b7720 2020-12-05T17:37:54.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.998+0000 7f41ce3a9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41c8152a10 con 0x7f41c8144540 2020-12-05T17:37:54.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.998+0000 7f41ccba6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41c81403f0 0x7f41c8152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.998+0000 7f41ccba6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41c81403f0 0x7f41c8152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36834/0 (socket says 172.21.15.32:36834) 2020-12-05T17:37:54.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.998+0000 7f41ccba6700 1 -- 172.21.15.32:0/3775756281 learned_addr learned my addr 172.21.15.32:0/3775756281 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:54.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.998+0000 7f41cdba8700 1 --2- 172.21.15.32:0/3775756281 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f41c8144540 0x7f41c8153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.998+0000 7f41cd3a7700 1 --2- 172.21.15.32:0/3775756281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f41c80b7720 0x7f41c8146360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.998+0000 7f41ccba6700 1 -- 172.21.15.32:0/3775756281 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f41c8144540 msgr2=0x7f41c8153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:54.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.998+0000 7f41ccba6700 1 --2- 172.21.15.32:0/3775756281 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f41c8144540 0x7f41c8153330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.998+0000 7f41ccba6700 1 -- 172.21.15.32:0/3775756281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f41c80b7720 msgr2=0x7f41c8146360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:54.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.998+0000 7f41ccba6700 1 --2- 172.21.15.32:0/3775756281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f41c80b7720 0x7f41c8146360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.998+0000 7f41ccba6700 1 -- 172.21.15.32:0/3775756281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c813c080 con 0x7f41c81403f0 2020-12-05T17:37:54.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.998+0000 7f41ccba6700 1 --2- 172.21.15.32:0/3775756281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41c81403f0 0x7f41c8152e40 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f41bc0032b0 tx=0x7f41bc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:54.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.998+0000 7f41c67fc700 1 -- 172.21.15.32:0/3775756281 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41bc01f5e0 con 0x7f41c81403f0 2020-12-05T17:37:54.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.998+0000 7f41c67fc700 1 -- 172.21.15.32:0/3775756281 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41bc01f740 con 0x7f41c81403f0 2020-12-05T17:37:54.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.998+0000 7f41ce3a9700 1 -- 172.21.15.32:0/3775756281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c8146850 con 0x7f41c81403f0 2020-12-05T17:37:54.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.998+0000 7f41c67fc700 1 -- 172.21.15.32:0/3775756281 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41bc01faa0 con 0x7f41c81403f0 2020-12-05T17:37:54.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.998+0000 7f41ce3a9700 1 -- 172.21.15.32:0/3775756281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c813b510 con 0x7f41c81403f0 2020-12-05T17:37:54.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:53.998+0000 7f41ce3a9700 1 -- 172.21.15.32:0/3775756281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41c80ae930 con 0x7f41c81403f0 2020-12-05T17:37:54.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.002+0000 7f41c67fc700 1 -- 172.21.15.32:0/3775756281 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f41bc026030 con 0x7f41c81403f0 2020-12-05T17:37:54.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.002+0000 7f41c67fc700 1 --2- 172.21.15.32:0/3775756281 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f41a8055440 0x7f41a8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:54.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.002+0000 7f41c67fc700 1 -- 172.21.15.32:0/3775756281 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f41bc06f090 con 0x7f41c81403f0 2020-12-05T17:37:54.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.002+0000 7f41cd3a7700 1 --2- 172.21.15.32:0/3775756281 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f41a8055440 0x7f41a8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.002+0000 7f41cd3a7700 1 --2- 172.21.15.32:0/3775756281 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f41a8055440 0x7f41a8057900 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f41b800b000 tx=0x7f41b8009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:54.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.010+0000 7f41c67fc700 1 -- 172.21.15.32:0/3775756281 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f41bc045230 con 0x7f41c81403f0 2020-12-05T17:37:54.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.146+0000 7f41ce3a9700 1 -- 172.21.15.32:0/3775756281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41c8057050 con 0x7f41c81403f0 2020-12-05T17:37:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.146+0000 7f41c67fc700 1 -- 172.21.15.32:0/3775756281 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f41bc024090 con 0x7f41c81403f0 2020-12-05T17:37:54.153 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:54.154 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":72,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:37:47.499387+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:37:45.088140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-12-05T17:37:45.501261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:37:54.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.150+0000 7f41ce3a9700 1 -- 172.21.15.32:0/3775756281 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f41a8055440 msgr2=0x7f41a8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:54.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.150+0000 7f41ce3a9700 1 --2- 172.21.15.32:0/3775756281 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f41a8055440 0x7f41a8057900 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f41b800b000 tx=0x7f41b8009040).stop 2020-12-05T17:37:54.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.150+0000 7f41ce3a9700 1 -- 172.21.15.32:0/3775756281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41c81403f0 msgr2=0x7f41c8152e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:54.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.150+0000 7f41ce3a9700 1 --2- 172.21.15.32:0/3775756281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41c81403f0 0x7f41c8152e40 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f41bc0032b0 tx=0x7f41bc016000).stop 2020-12-05T17:37:54.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.150+0000 7f41ce3a9700 1 -- 172.21.15.32:0/3775756281 shutdown_connections 2020-12-05T17:37:54.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.150+0000 7f41ce3a9700 1 --2- 172.21.15.32:0/3775756281 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f41a8055440 0x7f41a8057900 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.150+0000 7f41ce3a9700 1 --2- 172.21.15.32:0/3775756281 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f41c8144540 0x7f41c8153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.150+0000 7f41ce3a9700 1 --2- 172.21.15.32:0/3775756281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f41c80b7720 0x7f41c8146360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.150+0000 7f41ce3a9700 1 --2- 172.21.15.32:0/3775756281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41c81403f0 0x7f41c8152e40 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.150+0000 7f41ce3a9700 1 -- 172.21.15.32:0/3775756281 >> 172.21.15.32:0/3775756281 conn(0x7f41c805bc20 msgr2=0x7f41c8061780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:54.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.150+0000 7f41ce3a9700 1 -- 172.21.15.32:0/3775756281 shutdown_connections 2020-12-05T17:37:54.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.150+0000 7f41ce3a9700 1 -- 172.21.15.32:0/3775756281 wait complete. 2020-12-05T17:37:54.168 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:37:45.088140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:37:45.501261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:37:54.169 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:37:54.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.282+0000 7f9047330700 1 Processor -- start 2020-12-05T17:37:54.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.282+0000 7f9047330700 1 -- start start 2020-12-05T17:37:54.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.282+0000 7f9047330700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f904005cd70 0x7f90400b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:54.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.282+0000 7f9047330700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f90400aff50 0x7f90400b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:54.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.282+0000 7f9047330700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f904005d150 con 0x7f904005fc20 2020-12-05T17:37:54.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.282+0000 7f9047330700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9040146c40 con 0x7f90400aff50 2020-12-05T17:37:54.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.282+0000 7f9047330700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f904005f780 con 0x7f904005cd70 2020-12-05T17:37:54.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.282+0000 7f90448cb700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f904005fc20 0x7f904013fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39492/0 (socket says 172.21.15.32:39492) 2020-12-05T17:37:54.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.282+0000 7f90448cb700 1 -- 172.21.15.32:0/3691644939 learned_addr learned my addr 172.21.15.32:0/3691644939 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:54.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.282+0000 7f90450cc700 1 --2- 172.21.15.32:0/3691644939 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f904005cd70 0x7f90400b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.282+0000 7f90458cd700 1 --2- 172.21.15.32:0/3691644939 >> v2:172.21.15.61:3300/0 conn(0x7f90400aff50 0x7f90400b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.282+0000 7f9037fff700 1 -- 172.21.15.32:0/3691644939 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1544468450 0 0) 0x7f904005d150 con 0x7f904005fc20 2020-12-05T17:37:54.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.282+0000 7f9037fff700 1 -- 172.21.15.32:0/3691644939 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9024003230 con 0x7f904005fc20 2020-12-05T17:37:54.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.286+0000 7f90450cc700 1 -- 172.21.15.32:0/3691644939 >> v2:172.21.15.61:3300/0 conn(0x7f90400aff50 msgr2=0x7f90400b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:54.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.286+0000 7f90450cc700 1 --2- 172.21.15.32:0/3691644939 >> v2:172.21.15.61:3300/0 conn(0x7f90400aff50 0x7f90400b5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.286+0000 7f90450cc700 1 -- 172.21.15.32:0/3691644939 >> v1:172.21.15.32:6789/0 conn(0x7f904005fc20 legacy=0x7f904013fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:54.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.286+0000 7f90450cc700 1 -- 172.21.15.32:0/3691644939 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9040146930 con 0x7f904005cd70 2020-12-05T17:37:54.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.286+0000 7f90450cc700 1 --2- 172.21.15.32:0/3691644939 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f904005cd70 0x7f90400b1990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f903c002f30 tx=0x7f903c006dc0).ready entity=mon.2 client_cookie=1e9d1a3b3dd668c1 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:54.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.286+0000 7f9037fff700 1 -- 172.21.15.32:0/3691644939 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f903c0096a0 con 0x7f904005cd70 2020-12-05T17:37:54.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.286+0000 7f9037fff700 1 -- 172.21.15.32:0/3691644939 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903c009800 con 0x7f904005cd70 2020-12-05T17:37:54.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.286+0000 7f9037fff700 1 -- 172.21.15.32:0/3691644939 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f903c009b60 con 0x7f904005cd70 2020-12-05T17:37:54.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.286+0000 7f9047330700 1 -- 172.21.15.32:0/3691644939 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f904005cd70 msgr2=0x7f90400b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:54.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.286+0000 7f9047330700 1 --2- 172.21.15.32:0/3691644939 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f904005cd70 0x7f90400b1990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f903c002f30 tx=0x7f903c006dc0).stop 2020-12-05T17:37:54.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.286+0000 7f9047330700 1 -- 172.21.15.32:0/3691644939 shutdown_connections 2020-12-05T17:37:54.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.286+0000 7f9047330700 1 --2- 172.21.15.32:0/3691644939 >> v2:172.21.15.61:3300/0 conn(0x7f90400aff50 0x7f90400b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.286+0000 7f9047330700 1 --2- 172.21.15.32:0/3691644939 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f904005cd70 0x7f90400b1990 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.286+0000 7f9047330700 1 -- 172.21.15.32:0/3691644939 >> 172.21.15.32:0/3691644939 conn(0x7f9040069cb0 msgr2=0x7f90400604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:54.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.286+0000 7f9047330700 1 -- 172.21.15.32:0/3691644939 shutdown_connections 2020-12-05T17:37:54.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.286+0000 7f9047330700 1 -- 172.21.15.32:0/3691644939 wait complete. 2020-12-05T17:37:54.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.286+0000 7f9047330700 1 Processor -- start 2020-12-05T17:37:54.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.286+0000 7f9047330700 1 -- start start 2020-12-05T17:37:54.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.290+0000 7f9047330700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f904005cd70 0x7f90400634b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:54.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.290+0000 7f9047330700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f904005fc20 0x7f9040155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:54.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.290+0000 7f9047330700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90400aff50 0x7f9040155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:54.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.290+0000 7f9047330700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9040140d80 con 0x7f90400aff50 2020-12-05T17:37:54.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.290+0000 7f9047330700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9040141020 con 0x7f904005fc20 2020-12-05T17:37:54.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.290+0000 7f9047330700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9040140a00 con 0x7f904005cd70 2020-12-05T17:37:54.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.290+0000 7f90458cd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90400aff50 0x7f9040155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.290+0000 7f90458cd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90400aff50 0x7f9040155520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36850/0 (socket says 172.21.15.32:36850) 2020-12-05T17:37:54.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.290+0000 7f90450cc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f904005cd70 0x7f90400634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.290+0000 7f90448cb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f904005fc20 0x7f9040155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.290+0000 7f90458cd700 1 -- 172.21.15.32:0/3307849590 learned_addr learned my addr 172.21.15.32:0/3307849590 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:54.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.290+0000 7f90458cd700 1 -- 172.21.15.32:0/3307849590 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f904005cd70 msgr2=0x7f90400634b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:54.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.290+0000 7f90458cd700 1 --2- 172.21.15.32:0/3307849590 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f904005cd70 0x7f90400634b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.290+0000 7f90458cd700 1 -- 172.21.15.32:0/3307849590 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f904005fc20 msgr2=0x7f9040155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:54.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.290+0000 7f90458cd700 1 --2- 172.21.15.32:0/3307849590 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f904005fc20 0x7f9040155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.290+0000 7f90458cd700 1 -- 172.21.15.32:0/3307849590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9040155a10 con 0x7f90400aff50 2020-12-05T17:37:54.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.290+0000 7f90458cd700 1 --2- 172.21.15.32:0/3307849590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90400aff50 0x7f9040155520 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7f9038006dd0 tx=0x7f90380091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:54.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.290+0000 7f90367fc700 1 -- 172.21.15.32:0/3307849590 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f903800c8f0 con 0x7f90400aff50 2020-12-05T17:37:54.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.290+0000 7f9047330700 1 -- 172.21.15.32:0/3307849590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9040065400 con 0x7f90400aff50 2020-12-05T17:37:54.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.290+0000 7f90367fc700 1 -- 172.21.15.32:0/3307849590 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903800ca50 con 0x7f90400aff50 2020-12-05T17:37:54.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.290+0000 7f90367fc700 1 -- 172.21.15.32:0/3307849590 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f903800cdb0 con 0x7f90400aff50 2020-12-05T17:37:54.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.290+0000 7f9047330700 1 -- 172.21.15.32:0/3307849590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9040065ff0 con 0x7f90400aff50 2020-12-05T17:37:54.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.290+0000 7f9047330700 1 -- 172.21.15.32:0/3307849590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9040140bc0 con 0x7f90400aff50 2020-12-05T17:37:54.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.294+0000 7f90367fc700 1 -- 172.21.15.32:0/3307849590 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9038022070 con 0x7f90400aff50 2020-12-05T17:37:54.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.294+0000 7f90367fc700 1 --2- 172.21.15.32:0/3307849590 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9028055440 0x7f9028057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:54.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.294+0000 7f90367fc700 1 -- 172.21.15.32:0/3307849590 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f903806e6e0 con 0x7f90400aff50 2020-12-05T17:37:54.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.294+0000 7f90450cc700 1 --2- 172.21.15.32:0/3307849590 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9028055440 0x7f9028057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.294+0000 7f90450cc700 1 --2- 172.21.15.32:0/3307849590 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9028055440 0x7f9028057900 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f903c01b000 tx=0x7f903c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:54.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.298+0000 7f90367fc700 1 -- 172.21.15.32:0/3307849590 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f90380447e0 con 0x7f90400aff50 2020-12-05T17:37:54.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.446+0000 7f9047330700 1 -- 172.21.15.32:0/3307849590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f90401411e0 con 0x7f90400aff50 2020-12-05T17:37:54.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.446+0000 7f90367fc700 1 -- 172.21.15.32:0/3307849590 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v72) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9038016020 con 0x7f90400aff50 2020-12-05T17:37:54.452 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:37:54.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.446+0000 7f9047330700 1 -- 172.21.15.32:0/3307849590 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9028055440 msgr2=0x7f9028057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:54.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.446+0000 7f9047330700 1 --2- 172.21.15.32:0/3307849590 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9028055440 0x7f9028057900 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f903c01b000 tx=0x7f903c015000).stop 2020-12-05T17:37:54.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.446+0000 7f9047330700 1 -- 172.21.15.32:0/3307849590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90400aff50 msgr2=0x7f9040155520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:54.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.446+0000 7f9047330700 1 --2- 172.21.15.32:0/3307849590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90400aff50 0x7f9040155520 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7f9038006dd0 tx=0x7f90380091a0).stop 2020-12-05T17:37:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.446+0000 7f9047330700 1 -- 172.21.15.32:0/3307849590 shutdown_connections 2020-12-05T17:37:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.446+0000 7f9047330700 1 --2- 172.21.15.32:0/3307849590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90400aff50 0x7f9040155520 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.446+0000 7f9047330700 1 --2- 172.21.15.32:0/3307849590 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9028055440 0x7f9028057900 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.446+0000 7f9047330700 1 --2- 172.21.15.32:0/3307849590 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f904005cd70 0x7f90400634b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.446+0000 7f9047330700 1 --2- 172.21.15.32:0/3307849590 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f904005fc20 0x7f9040155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.446+0000 7f9047330700 1 -- 172.21.15.32:0/3307849590 >> 172.21.15.32:0/3307849590 conn(0x7f9040069cb0 msgr2=0x7f904013ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.446+0000 7f9047330700 1 -- 172.21.15.32:0/3307849590 shutdown_connections 2020-12-05T17:37:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.446+0000 7f9047330700 1 -- 172.21.15.32:0/3307849590 wait complete. 2020-12-05T17:37:54.464 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:37:54.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.578+0000 7fc528fea700 1 Processor -- start 2020-12-05T17:37:54.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.578+0000 7fc528fea700 1 -- start start 2020-12-05T17:37:54.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.578+0000 7fc528fea700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc52405feb0 0x7fc52405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:54.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.578+0000 7fc528fea700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5240b2fb0 0x7fc524061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:54.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.578+0000 7fc528fea700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5240b2df0 con 0x7fc52405f3b0 2020-12-05T17:37:54.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.578+0000 7fc528fea700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5240b3c90 con 0x7fc5240b2fb0 2020-12-05T17:37:54.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.578+0000 7fc528fea700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5240b2b50 con 0x7fc52405feb0 2020-12-05T17:37:54.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.578+0000 7fc523fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5240b2fb0 0x7fc524061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.578+0000 7fc523fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5240b2fb0 0x7fc524061780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36858/0 (socket says 172.21.15.32:36858) 2020-12-05T17:37:54.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.578+0000 7fc523fff700 1 -- 172.21.15.32:0/521213915 learned_addr learned my addr 172.21.15.32:0/521213915 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:54.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.578+0000 7fc51affd700 1 --2- 172.21.15.32:0/521213915 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc52405feb0 0x7fc52405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc523fff700 1 -- 172.21.15.32:0/521213915 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc52405feb0 msgr2=0x7fc52405f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:54.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc523fff700 1 --2- 172.21.15.32:0/521213915 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc52405feb0 0x7fc52405f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc523fff700 1 -- 172.21.15.32:0/521213915 >> v1:172.21.15.61:6789/0 conn(0x7fc52405f3b0 legacy=0x7fc52413c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:54.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc523fff700 1 -- 172.21.15.32:0/521213915 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc524142820 con 0x7fc5240b2fb0 2020-12-05T17:37:54.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc51affd700 1 --2- 172.21.15.32:0/521213915 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc52405feb0 0x7fc52405f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:54.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc523fff700 1 --2- 172.21.15.32:0/521213915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5240b2fb0 0x7fc524061780 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7fc51c0032b0 tx=0x7fc51c009270).ready entity=mon.0 client_cookie=9149faf0dace3d3d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:54.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc522ffd700 1 -- 172.21.15.32:0/521213915 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc51c00bc00 con 0x7fc5240b2fb0 2020-12-05T17:37:54.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc522ffd700 1 -- 172.21.15.32:0/521213915 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51c00bd60 con 0x7fc5240b2fb0 2020-12-05T17:37:54.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc522ffd700 1 -- 172.21.15.32:0/521213915 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc51c011400 con 0x7fc5240b2fb0 2020-12-05T17:37:54.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc528fea700 1 -- 172.21.15.32:0/521213915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5240b2fb0 msgr2=0x7fc524061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:54.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc528fea700 1 --2- 172.21.15.32:0/521213915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5240b2fb0 0x7fc524061780 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7fc51c0032b0 tx=0x7fc51c009270).stop 2020-12-05T17:37:54.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc528fea700 1 -- 172.21.15.32:0/521213915 shutdown_connections 2020-12-05T17:37:54.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc528fea700 1 --2- 172.21.15.32:0/521213915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5240b2fb0 0x7fc524061780 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc528fea700 1 --2- 172.21.15.32:0/521213915 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc52405feb0 0x7fc52405f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc528fea700 1 -- 172.21.15.32:0/521213915 >> 172.21.15.32:0/521213915 conn(0x7fc5240b8f20 msgr2=0x7fc5240b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:54.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc528fea700 1 -- 172.21.15.32:0/521213915 shutdown_connections 2020-12-05T17:37:54.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc528fea700 1 -- 172.21.15.32:0/521213915 wait complete. 2020-12-05T17:37:54.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc528fea700 1 Processor -- start 2020-12-05T17:37:54.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc528fea700 1 -- start start 2020-12-05T17:37:54.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc528fea700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52405f3b0 0x7fc524152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:54.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc528fea700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc52405feb0 0x7fc524154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:54.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc528fea700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc5240b2fb0 0x7fc52413ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:54.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc528fea700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5240642a0 con 0x7fc52405f3b0 2020-12-05T17:37:54.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc528fea700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc524063c40 con 0x7fc5240b2fb0 2020-12-05T17:37:54.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.582+0000 7fc528fea700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc524155780 con 0x7fc52405feb0 2020-12-05T17:37:54.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.586+0000 7fc5237fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52405f3b0 0x7fc524152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.586+0000 7fc5237fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52405f3b0 0x7fc524152350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36860/0 (socket says 172.21.15.32:36860) 2020-12-05T17:37:54.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.586+0000 7fc5237fe700 1 -- 172.21.15.32:0/1427859769 learned_addr learned my addr 172.21.15.32:0/1427859769 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:54.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.586+0000 7fc51affd700 1 --2- 172.21.15.32:0/1427859769 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc52405feb0 0x7fc524154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.586+0000 7fc523fff700 1 --2- 172.21.15.32:0/1427859769 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc5240b2fb0 0x7fc52413ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.586+0000 7fc5237fe700 1 -- 172.21.15.32:0/1427859769 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc52405feb0 msgr2=0x7fc524154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:54.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.586+0000 7fc5237fe700 1 --2- 172.21.15.32:0/1427859769 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc52405feb0 0x7fc524154e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.586+0000 7fc5237fe700 1 -- 172.21.15.32:0/1427859769 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc5240b2fb0 msgr2=0x7fc52413ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:54.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.586+0000 7fc5237fe700 1 --2- 172.21.15.32:0/1427859769 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc5240b2fb0 0x7fc52413ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.586+0000 7fc5237fe700 1 -- 172.21.15.32:0/1427859769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc524063e70 con 0x7fc52405f3b0 2020-12-05T17:37:54.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.586+0000 7fc5237fe700 1 --2- 172.21.15.32:0/1427859769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52405f3b0 0x7fc524152350 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7fc5241520b0 tx=0x7fc514007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:54.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.586+0000 7fc5217fa700 1 -- 172.21.15.32:0/1427859769 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc514007240 con 0x7fc52405f3b0 2020-12-05T17:37:54.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.586+0000 7fc5217fa700 1 -- 172.21.15.32:0/1427859769 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5140073a0 con 0x7fc52405f3b0 2020-12-05T17:37:54.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.586+0000 7fc528fea700 1 -- 172.21.15.32:0/1427859769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5241554e0 con 0x7fc52405f3b0 2020-12-05T17:37:54.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.586+0000 7fc5217fa700 1 -- 172.21.15.32:0/1427859769 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5140066d0 con 0x7fc52405f3b0 2020-12-05T17:37:54.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.586+0000 7fc528fea700 1 -- 172.21.15.32:0/1427859769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc524063840 con 0x7fc52405f3b0 2020-12-05T17:37:54.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.586+0000 7fc528fea700 1 -- 172.21.15.32:0/1427859769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc524057050 con 0x7fc52405f3b0 2020-12-05T17:37:54.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.586+0000 7fc5217fa700 1 -- 172.21.15.32:0/1427859769 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc514010070 con 0x7fc52405f3b0 2020-12-05T17:37:54.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.590+0000 7fc5217fa700 1 --2- 172.21.15.32:0/1427859769 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc5080553b0 0x7fc508057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:54.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.590+0000 7fc5217fa700 1 -- 172.21.15.32:0/1427859769 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc51406d770 con 0x7fc52405f3b0 2020-12-05T17:37:54.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.590+0000 7fc51affd700 1 --2- 172.21.15.32:0/1427859769 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc5080553b0 0x7fc508057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.590+0000 7fc51affd700 1 --2- 172.21.15.32:0/1427859769 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc5080553b0 0x7fc508057870 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fc50c00c000 tx=0x7fc50c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:54.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.594+0000 7fc5217fa700 1 -- 172.21.15.32:0/1427859769 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc514043910 con 0x7fc52405f3b0 2020-12-05T17:37:54.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.738+0000 7fc528fea700 1 -- 172.21.15.32:0/1427859769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc524066e60 con 0x7fc52405f3b0 2020-12-05T17:37:54.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.738+0000 7fc5217fa700 1 -- 172.21.15.32:0/1427859769 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v72) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc514046f30 con 0x7fc52405f3b0 2020-12-05T17:37:54.745 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:37:54.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.742+0000 7fc528fea700 1 -- 172.21.15.32:0/1427859769 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc5080553b0 msgr2=0x7fc508057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:54.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.742+0000 7fc528fea700 1 --2- 172.21.15.32:0/1427859769 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc5080553b0 0x7fc508057870 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fc50c00c000 tx=0x7fc50c008040).stop 2020-12-05T17:37:54.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.742+0000 7fc528fea700 1 -- 172.21.15.32:0/1427859769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52405f3b0 msgr2=0x7fc524152350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:54.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.742+0000 7fc528fea700 1 --2- 172.21.15.32:0/1427859769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52405f3b0 0x7fc524152350 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7fc5241520b0 tx=0x7fc514007be0).stop 2020-12-05T17:37:54.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.742+0000 7fc528fea700 1 -- 172.21.15.32:0/1427859769 shutdown_connections 2020-12-05T17:37:54.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.742+0000 7fc528fea700 1 --2- 172.21.15.32:0/1427859769 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc5240b2fb0 0x7fc52413ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.742+0000 7fc528fea700 1 --2- 172.21.15.32:0/1427859769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52405f3b0 0x7fc524152350 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.742+0000 7fc528fea700 1 --2- 172.21.15.32:0/1427859769 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc5080553b0 0x7fc508057870 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.742+0000 7fc528fea700 1 --2- 172.21.15.32:0/1427859769 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc52405feb0 0x7fc524154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.742+0000 7fc528fea700 1 -- 172.21.15.32:0/1427859769 >> 172.21.15.32:0/1427859769 conn(0x7fc5240b8f20 msgr2=0x7fc5240b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:54.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.742+0000 7fc528fea700 1 -- 172.21.15.32:0/1427859769 shutdown_connections 2020-12-05T17:37:54.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.742+0000 7fc528fea700 1 -- 172.21.15.32:0/1427859769 wait complete. 2020-12-05T17:37:54.759 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:37:54.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.874+0000 7f045d3d1700 1 Processor -- start 2020-12-05T17:37:54.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.874+0000 7f045d3d1700 1 -- start start 2020-12-05T17:37:54.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.874+0000 7f045d3d1700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f045805feb0 0x7f045805f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:54.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.874+0000 7f045d3d1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04580b2fb0 0x7f0458061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:54.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.874+0000 7f045d3d1700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04580b2df0 con 0x7f045805f3b0 2020-12-05T17:37:54.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.874+0000 7f045d3d1700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04580b3c90 con 0x7f045805feb0 2020-12-05T17:37:54.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.874+0000 7f045d3d1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04580b2b50 con 0x7f04580b2fb0 2020-12-05T17:37:54.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.874+0000 7f045cbd0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04580b2fb0 0x7f0458061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.874+0000 7f04577fe700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f045805feb0 0x7f045805f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.874+0000 7f045cbd0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04580b2fb0 0x7f0458061780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:33662/0 (socket says 172.21.15.32:33662) 2020-12-05T17:37:54.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.874+0000 7f0457fff700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f045805f3b0 0x7f045813c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:52156/0 (socket says 172.21.15.32:52156) 2020-12-05T17:37:54.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.874+0000 7f045cbd0700 1 -- 172.21.15.32:0/2034947038 learned_addr learned my addr 172.21.15.32:0/2034947038 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:54.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.874+0000 7f045cbd0700 1 -- 172.21.15.32:0/2034947038 >> v2:172.21.15.61:3300/0 conn(0x7f045805feb0 msgr2=0x7f045805f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:54.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.874+0000 7f045cbd0700 1 --2- 172.21.15.32:0/2034947038 >> v2:172.21.15.61:3300/0 conn(0x7f045805feb0 0x7f045805f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.874+0000 7f045cbd0700 1 -- 172.21.15.32:0/2034947038 >> v1:172.21.15.61:6789/0 conn(0x7f045805f3b0 legacy=0x7f045813c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:54.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.874+0000 7f045cbd0700 1 -- 172.21.15.32:0/2034947038 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0458142820 con 0x7f04580b2fb0 2020-12-05T17:37:54.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.874+0000 7f045cbd0700 1 --2- 172.21.15.32:0/2034947038 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04580b2fb0 0x7f0458061780 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f04500032b0 tx=0x7f0450009270).ready entity=mon.2 client_cookie=2fef6013595c749b server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:54.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.878+0000 7f0456ffd700 1 -- 172.21.15.32:0/2034947038 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0450009e60 con 0x7f04580b2fb0 2020-12-05T17:37:54.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.878+0000 7f0456ffd700 1 -- 172.21.15.32:0/2034947038 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045000bc00 con 0x7f04580b2fb0 2020-12-05T17:37:54.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.878+0000 7f0456ffd700 1 -- 172.21.15.32:0/2034947038 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f045000bd70 con 0x7f04580b2fb0 2020-12-05T17:37:54.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.878+0000 7f045d3d1700 1 -- 172.21.15.32:0/2034947038 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04580b2fb0 msgr2=0x7f0458061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:54.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.878+0000 7f045d3d1700 1 --2- 172.21.15.32:0/2034947038 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04580b2fb0 0x7f0458061780 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f04500032b0 tx=0x7f0450009270).stop 2020-12-05T17:37:54.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.878+0000 7f045d3d1700 1 -- 172.21.15.32:0/2034947038 shutdown_connections 2020-12-05T17:37:54.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.878+0000 7f045d3d1700 1 --2- 172.21.15.32:0/2034947038 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04580b2fb0 0x7f0458061780 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.878+0000 7f045d3d1700 1 --2- 172.21.15.32:0/2034947038 >> v2:172.21.15.61:3300/0 conn(0x7f045805feb0 0x7f045805f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.878+0000 7f045d3d1700 1 -- 172.21.15.32:0/2034947038 >> 172.21.15.32:0/2034947038 conn(0x7f04580b8f20 msgr2=0x7f04580b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:54.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.878+0000 7f045d3d1700 1 -- 172.21.15.32:0/2034947038 shutdown_connections 2020-12-05T17:37:54.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.878+0000 7f045d3d1700 1 -- 172.21.15.32:0/2034947038 wait complete. 2020-12-05T17:37:54.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.878+0000 7f045d3d1700 1 Processor -- start 2020-12-05T17:37:54.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.878+0000 7f045d3d1700 1 -- start start 2020-12-05T17:37:54.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.882+0000 7f045d3d1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f045805f3b0 0x7f0458147aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:54.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.882+0000 7f045d3d1700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f045805feb0 0x7f0458140bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:54.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.882+0000 7f045d3d1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04580b2fb0 0x7f04581410e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:54.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.882+0000 7f045d3d1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0458112970 con 0x7f04580b2fb0 2020-12-05T17:37:54.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.882+0000 7f045d3d1700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0458149a50 con 0x7f045805feb0 2020-12-05T17:37:54.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.882+0000 7f045d3d1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0458113b90 con 0x7f045805f3b0 2020-12-05T17:37:54.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.882+0000 7f045cbd0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04580b2fb0 0x7f04581410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.882+0000 7f0457fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f045805f3b0 0x7f0458147aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.882+0000 7f045cbd0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04580b2fb0 0x7f04581410e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36876/0 (socket says 172.21.15.32:36876) 2020-12-05T17:37:54.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.882+0000 7f045cbd0700 1 -- 172.21.15.32:0/4265246092 learned_addr learned my addr 172.21.15.32:0/4265246092 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:54.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.882+0000 7f04577fe700 1 --2- 172.21.15.32:0/4265246092 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f045805feb0 0x7f0458140bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.882+0000 7f045cbd0700 1 -- 172.21.15.32:0/4265246092 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f045805f3b0 msgr2=0x7f0458147aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:54.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.882+0000 7f045cbd0700 1 --2- 172.21.15.32:0/4265246092 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f045805f3b0 0x7f0458147aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.882+0000 7f045cbd0700 1 -- 172.21.15.32:0/4265246092 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f045805feb0 msgr2=0x7f0458140bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:54.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.882+0000 7f045cbd0700 1 --2- 172.21.15.32:0/4265246092 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f045805feb0 0x7f0458140bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:54.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.882+0000 7f045cbd0700 1 -- 172.21.15.32:0/4265246092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0458113e30 con 0x7f04580b2fb0 2020-12-05T17:37:54.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.882+0000 7f045cbd0700 1 --2- 172.21.15.32:0/4265246092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04580b2fb0 0x7f04581410e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f04500032b0 tx=0x7f0450016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:54.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.882+0000 7f04557fa700 1 -- 172.21.15.32:0/4265246092 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f045001f3f0 con 0x7f04580b2fb0 2020-12-05T17:37:54.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.882+0000 7f04557fa700 1 -- 172.21.15.32:0/4265246092 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045001f550 con 0x7f04580b2fb0 2020-12-05T17:37:54.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.882+0000 7f04557fa700 1 -- 172.21.15.32:0/4265246092 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f045001f8b0 con 0x7f04580b2fb0 2020-12-05T17:37:54.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.882+0000 7f045d3d1700 1 -- 172.21.15.32:0/4265246092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04581140d0 con 0x7f04580b2fb0 2020-12-05T17:37:54.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.882+0000 7f045d3d1700 1 -- 172.21.15.32:0/4265246092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0458146d00 con 0x7f04580b2fb0 2020-12-05T17:37:54.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.886+0000 7f045d3d1700 1 -- 172.21.15.32:0/4265246092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0458057050 con 0x7f04580b2fb0 2020-12-05T17:37:54.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.886+0000 7f04557fa700 1 -- 172.21.15.32:0/4265246092 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0450026030 con 0x7f04580b2fb0 2020-12-05T17:37:54.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.886+0000 7f04557fa700 1 --2- 172.21.15.32:0/4265246092 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f04380553b0 0x7f0438057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:54.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.886+0000 7f04557fa700 1 -- 172.21.15.32:0/4265246092 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f045006ea80 con 0x7f04580b2fb0 2020-12-05T17:37:54.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.886+0000 7f0457fff700 1 --2- 172.21.15.32:0/4265246092 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f04380553b0 0x7f0438057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:54.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.886+0000 7f0457fff700 1 --2- 172.21.15.32:0/4265246092 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f04380553b0 0x7f0438057870 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f044c00d000 tx=0x7f044c007b50).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:54.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:54.894+0000 7f04557fa700 1 -- 172.21.15.32:0/4265246092 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0450044c20 con 0x7f04580b2fb0 2020-12-05T17:37:55.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.038+0000 7f045d3d1700 1 -- 172.21.15.32:0/4265246092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f045805aab0 con 0x7f04580b2fb0 2020-12-05T17:37:55.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.038+0000 7f04557fa700 1 -- 172.21.15.32:0/4265246092 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0450048240 con 0x7f04580b2fb0 2020-12-05T17:37:55.044 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:37:55.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.038+0000 7f045d3d1700 1 -- 172.21.15.32:0/4265246092 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f04380553b0 msgr2=0x7f0438057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:55.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.038+0000 7f045d3d1700 1 --2- 172.21.15.32:0/4265246092 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f04380553b0 0x7f0438057870 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f044c00d000 tx=0x7f044c007b50).stop 2020-12-05T17:37:55.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.038+0000 7f045d3d1700 1 -- 172.21.15.32:0/4265246092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04580b2fb0 msgr2=0x7f04581410e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:55.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.038+0000 7f045d3d1700 1 --2- 172.21.15.32:0/4265246092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04580b2fb0 0x7f04581410e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f04500032b0 tx=0x7f0450016000).stop 2020-12-05T17:37:55.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.038+0000 7f045d3d1700 1 -- 172.21.15.32:0/4265246092 shutdown_connections 2020-12-05T17:37:55.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.038+0000 7f045d3d1700 1 --2- 172.21.15.32:0/4265246092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04580b2fb0 0x7f04581410e0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:55.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.038+0000 7f045d3d1700 1 --2- 172.21.15.32:0/4265246092 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f04380553b0 0x7f0438057870 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:55.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.038+0000 7f045d3d1700 1 --2- 172.21.15.32:0/4265246092 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f045805f3b0 0x7f0458147aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:55.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.038+0000 7f045d3d1700 1 --2- 172.21.15.32:0/4265246092 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f045805feb0 0x7f0458140bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:55.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.038+0000 7f045d3d1700 1 -- 172.21.15.32:0/4265246092 >> 172.21.15.32:0/4265246092 conn(0x7f04580b8f20 msgr2=0x7f0458061780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:55.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.038+0000 7f045d3d1700 1 -- 172.21.15.32:0/4265246092 shutdown_connections 2020-12-05T17:37:55.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.042+0000 7f045d3d1700 1 -- 172.21.15.32:0/4265246092 wait complete. 2020-12-05T17:37:55.057 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:37:55.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.170+0000 7f842c99b700 1 Processor -- start 2020-12-05T17:37:55.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.170+0000 7f842c99b700 1 -- start start 2020-12-05T17:37:55.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.174+0000 7f842c99b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f842805f6f0 0x7f8428146370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:55.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.174+0000 7f842c99b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f84280afdf0 0x7f8428146860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:55.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.174+0000 7f842c99b700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84280b0960 con 0x7f84280b3490 2020-12-05T17:37:55.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.174+0000 7f842c99b700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84280ae410 con 0x7f842805f6f0 2020-12-05T17:37:55.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.174+0000 7f842c99b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842805fad0 con 0x7f84280afdf0 2020-12-05T17:37:55.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.174+0000 7f842659c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f842805f6f0 0x7f8428146370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:55.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.174+0000 7f8426d9d700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f84280b3490 0x7f84280b5df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:52174/0 (socket says 172.21.15.32:52174) 2020-12-05T17:37:55.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.174+0000 7f8426d9d700 1 -- 172.21.15.32:0/3258681587 learned_addr learned my addr 172.21.15.32:0/3258681587 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:55.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.174+0000 7f8425d9b700 1 --2- 172.21.15.32:0/3258681587 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f84280afdf0 0x7f8428146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:55.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.174+0000 7f842659c700 1 -- 172.21.15.32:0/3258681587 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f84280afdf0 msgr2=0x7f8428146860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:55.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.174+0000 7f842659c700 1 --2- 172.21.15.32:0/3258681587 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f84280afdf0 0x7f8428146860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:55.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.174+0000 7f842659c700 1 -- 172.21.15.32:0/3258681587 >> v1:172.21.15.61:6789/0 conn(0x7f84280b3490 legacy=0x7f84280b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:55.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.174+0000 7f842659c700 1 -- 172.21.15.32:0/3258681587 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8428142860 con 0x7f842805f6f0 2020-12-05T17:37:55.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.174+0000 7f842659c700 1 --2- 172.21.15.32:0/3258681587 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f842805f6f0 0x7f8428146370 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f8418002f30 tx=0x7f8418006dc0).ready entity=mon.1 client_cookie=d2a1b843240cf4e9 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:55.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.174+0000 7f842559a700 1 -- 172.21.15.32:0/3258681587 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8418009760 con 0x7f842805f6f0 2020-12-05T17:37:55.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.174+0000 7f842559a700 1 -- 172.21.15.32:0/3258681587 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84180098c0 con 0x7f842805f6f0 2020-12-05T17:37:55.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.174+0000 7f842559a700 1 -- 172.21.15.32:0/3258681587 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8418009c20 con 0x7f842805f6f0 2020-12-05T17:37:55.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.174+0000 7f842c99b700 1 -- 172.21.15.32:0/3258681587 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f842805f6f0 msgr2=0x7f8428146370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:55.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.174+0000 7f842c99b700 1 --2- 172.21.15.32:0/3258681587 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f842805f6f0 0x7f8428146370 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f8418002f30 tx=0x7f8418006dc0).stop 2020-12-05T17:37:55.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.178+0000 7f842c99b700 1 -- 172.21.15.32:0/3258681587 shutdown_connections 2020-12-05T17:37:55.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.178+0000 7f842c99b700 1 --2- 172.21.15.32:0/3258681587 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f842805f6f0 0x7f8428146370 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:55.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.178+0000 7f842c99b700 1 --2- 172.21.15.32:0/3258681587 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f84280afdf0 0x7f8428146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:55.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.178+0000 7f842c99b700 1 -- 172.21.15.32:0/3258681587 >> 172.21.15.32:0/3258681587 conn(0x7f842805d070 msgr2=0x7f84280604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:55.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.178+0000 7f842c99b700 1 -- 172.21.15.32:0/3258681587 shutdown_connections 2020-12-05T17:37:55.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.178+0000 7f842c99b700 1 -- 172.21.15.32:0/3258681587 wait complete. 2020-12-05T17:37:55.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.178+0000 7f842c99b700 1 Processor -- start 2020-12-05T17:37:55.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.178+0000 7f842c99b700 1 -- start start 2020-12-05T17:37:55.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.182+0000 7f842c99b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f842805f6f0 0x7f84280b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:55.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.182+0000 7f842c99b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f84280afdf0 0x7f8428151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:55.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.182+0000 7f842c99b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84280b3490 0x7f8428113ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:55.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.182+0000 7f842c99b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8428148620 con 0x7f84280b3490 2020-12-05T17:37:55.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.182+0000 7f842c99b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8428146120 con 0x7f84280afdf0 2020-12-05T17:37:55.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.182+0000 7f842c99b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8428113840 con 0x7f842805f6f0 2020-12-05T17:37:55.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.182+0000 7f8426d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84280b3490 0x7f8428113ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:55.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.182+0000 7f8426d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84280b3490 0x7f8428113ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36892/0 (socket says 172.21.15.32:36892) 2020-12-05T17:37:55.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.182+0000 7f8426d9d700 1 -- 172.21.15.32:0/305108832 learned_addr learned my addr 172.21.15.32:0/305108832 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:55.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.182+0000 7f8425d9b700 1 --2- 172.21.15.32:0/305108832 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f84280afdf0 0x7f8428151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:55.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.182+0000 7f842659c700 1 --2- 172.21.15.32:0/305108832 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f842805f6f0 0x7f84280b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:55.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.182+0000 7f8426d9d700 1 -- 172.21.15.32:0/305108832 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f842805f6f0 msgr2=0x7f84280b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:55.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.182+0000 7f8426d9d700 1 --2- 172.21.15.32:0/305108832 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f842805f6f0 0x7f84280b5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:55.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.182+0000 7f8426d9d700 1 -- 172.21.15.32:0/305108832 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f84280afdf0 msgr2=0x7f8428151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:55.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.182+0000 7f8426d9d700 1 --2- 172.21.15.32:0/305108832 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f84280afdf0 0x7f8428151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:55.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.182+0000 7f8426d9d700 1 -- 172.21.15.32:0/305108832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8428145da0 con 0x7f84280b3490 2020-12-05T17:37:55.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.182+0000 7f8426d9d700 1 --2- 172.21.15.32:0/305108832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84280b3490 0x7f8428113ce0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f84280b2e50 tx=0x7f842000cb40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:55.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.182+0000 7f84177fe700 1 -- 172.21.15.32:0/305108832 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8420005b70 con 0x7f84280b3490 2020-12-05T17:37:55.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.182+0000 7f84177fe700 1 -- 172.21.15.32:0/305108832 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8420005cd0 con 0x7f84280b3490 2020-12-05T17:37:55.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.182+0000 7f842c99b700 1 -- 172.21.15.32:0/305108832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8428151d10 con 0x7f84280b3490 2020-12-05T17:37:55.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.182+0000 7f84177fe700 1 -- 172.21.15.32:0/305108832 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8420007300 con 0x7f84280b3490 2020-12-05T17:37:55.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.182+0000 7f842c99b700 1 -- 172.21.15.32:0/305108832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8428147fb0 con 0x7f84280b3490 2020-12-05T17:37:55.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.186+0000 7f84177fe700 1 -- 172.21.15.32:0/305108832 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8420010030 con 0x7f84280b3490 2020-12-05T17:37:55.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.186+0000 7f842c99b700 1 -- 172.21.15.32:0/305108832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8428146fd0 con 0x7f84280b3490 2020-12-05T17:37:55.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.186+0000 7f84177fe700 1 --2- 172.21.15.32:0/305108832 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f84040554e0 0x7f84040579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:55.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.186+0000 7f84177fe700 1 -- 172.21.15.32:0/305108832 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f842006dfe0 con 0x7f84280b3490 2020-12-05T17:37:55.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.186+0000 7f842659c700 1 --2- 172.21.15.32:0/305108832 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f84040554e0 0x7f84040579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:55.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.186+0000 7f842659c700 1 --2- 172.21.15.32:0/305108832 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f84040554e0 0x7f84040579a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f841801a000 tx=0x7f8418015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:55.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.190+0000 7f84177fe700 1 -- 172.21.15.32:0/305108832 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8428146fd0 con 0x7f84280b3490 2020-12-05T17:37:55.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.330+0000 7f842c99b700 1 -- 172.21.15.32:0/305108832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8428146fd0 con 0x7f84280b3490 2020-12-05T17:37:55.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.330+0000 7f84177fe700 1 -- 172.21.15.32:0/305108832 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) v1 ==== 100+0+10 (secure 0 0 0) 0x7f8428146fd0 con 0x7f84280b3490 2020-12-05T17:37:55.337 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:37:55.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.334+0000 7f842c99b700 1 -- 172.21.15.32:0/305108832 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f84040554e0 msgr2=0x7f84040579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:55.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.334+0000 7f842c99b700 1 --2- 172.21.15.32:0/305108832 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f84040554e0 0x7f84040579a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f841801a000 tx=0x7f8418015000).stop 2020-12-05T17:37:55.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.334+0000 7f842c99b700 1 -- 172.21.15.32:0/305108832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84280b3490 msgr2=0x7f8428113ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:55.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.334+0000 7f842c99b700 1 --2- 172.21.15.32:0/305108832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84280b3490 0x7f8428113ce0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f84280b2e50 tx=0x7f842000cb40).stop 2020-12-05T17:37:55.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.334+0000 7f842c99b700 1 -- 172.21.15.32:0/305108832 shutdown_connections 2020-12-05T17:37:55.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.334+0000 7f842c99b700 1 --2- 172.21.15.32:0/305108832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84280b3490 0x7f8428113ce0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:55.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.334+0000 7f842c99b700 1 --2- 172.21.15.32:0/305108832 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f84040554e0 0x7f84040579a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:55.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.334+0000 7f842c99b700 1 --2- 172.21.15.32:0/305108832 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f842805f6f0 0x7f84280b5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:55.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.334+0000 7f842c99b700 1 --2- 172.21.15.32:0/305108832 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f84280afdf0 0x7f8428151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:55.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.334+0000 7f842c99b700 1 -- 172.21.15.32:0/305108832 >> 172.21.15.32:0/305108832 conn(0x7f842805d070 msgr2=0x7f84280aead0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:55.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.334+0000 7f842c99b700 1 -- 172.21.15.32:0/305108832 shutdown_connections 2020-12-05T17:37:55.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.334+0000 7f842c99b700 1 -- 172.21.15.32:0/305108832 wait complete. 2020-12-05T17:37:55.351 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:37:55.351 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:37:55.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.458+0000 7f8814f1a700 1 Processor -- start 2020-12-05T17:37:55.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.458+0000 7f8814f1a700 1 -- start start 2020-12-05T17:37:55.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.458+0000 7f8814f1a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f88100b1fc0 0x7f88100af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:55.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.458+0000 7f8814f1a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f88100ab710 0x7f8810061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:55.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.458+0000 7f8814f1a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8810146a90 con 0x7f8810147400 2020-12-05T17:37:55.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.458+0000 7f8814f1a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88100b0110 con 0x7f88100ab710 2020-12-05T17:37:55.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.458+0000 7f8814f1a700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88100b23a0 con 0x7f88100b1fc0 2020-12-05T17:37:55.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.458+0000 7f880effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f88100ab710 0x7f8810061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:55.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.462+0000 7f880effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f88100ab710 0x7f8810061780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36896/0 (socket says 172.21.15.32:36896) 2020-12-05T17:37:55.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.462+0000 7f880effd700 1 -- 172.21.15.32:0/3018425670 learned_addr learned my addr 172.21.15.32:0/3018425670 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:55.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.462+0000 7f880f7fe700 1 --2- 172.21.15.32:0/3018425670 >> v2:172.21.15.61:3300/0 conn(0x7f88100b1fc0 0x7f88100af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:55.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.462+0000 7f880effd700 1 -- 172.21.15.32:0/3018425670 >> v2:172.21.15.61:3300/0 conn(0x7f88100b1fc0 msgr2=0x7f88100af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:55.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.462+0000 7f880effd700 1 --2- 172.21.15.32:0/3018425670 >> v2:172.21.15.61:3300/0 conn(0x7f88100b1fc0 0x7f88100af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:55.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.462+0000 7f880effd700 1 -- 172.21.15.32:0/3018425670 >> v1:172.21.15.32:6789/0 conn(0x7f8810147400 legacy=0x7f8810145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:55.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.462+0000 7f880effd700 1 -- 172.21.15.32:0/3018425670 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8810146cc0 con 0x7f88100ab710 2020-12-05T17:37:55.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.462+0000 7f880f7fe700 1 --2- 172.21.15.32:0/3018425670 >> v2:172.21.15.61:3300/0 conn(0x7f88100b1fc0 0x7f88100af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:37:55.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.462+0000 7f880effd700 1 --2- 172.21.15.32:0/3018425670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88100ab710 0x7f8810061780 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7f87f8002f30 tx=0x7f87f8006e80).ready entity=mon.0 client_cookie=ab2efbf08683a1cd server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:55.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.462+0000 7f880e7fc700 1 -- 172.21.15.32:0/3018425670 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87f80097f0 con 0x7f88100ab710 2020-12-05T17:37:55.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.462+0000 7f880e7fc700 1 -- 172.21.15.32:0/3018425670 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f8009950 con 0x7f88100ab710 2020-12-05T17:37:55.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.462+0000 7f880e7fc700 1 -- 172.21.15.32:0/3018425670 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87f8009cb0 con 0x7f88100ab710 2020-12-05T17:37:55.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.462+0000 7f8814f1a700 1 -- 172.21.15.32:0/3018425670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88100ab710 msgr2=0x7f8810061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:55.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.462+0000 7f8814f1a700 1 --2- 172.21.15.32:0/3018425670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88100ab710 0x7f8810061780 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7f87f8002f30 tx=0x7f87f8006e80).stop 2020-12-05T17:37:55.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.462+0000 7f8814f1a700 1 -- 172.21.15.32:0/3018425670 shutdown_connections 2020-12-05T17:37:55.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.462+0000 7f8814f1a700 1 --2- 172.21.15.32:0/3018425670 >> v2:172.21.15.61:3300/0 conn(0x7f88100b1fc0 0x7f88100af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:55.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.462+0000 7f8814f1a700 1 --2- 172.21.15.32:0/3018425670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88100ab710 0x7f8810061780 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:55.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.462+0000 7f8814f1a700 1 -- 172.21.15.32:0/3018425670 >> 172.21.15.32:0/3018425670 conn(0x7f88100b5530 msgr2=0x7f88100b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:55.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.462+0000 7f8814f1a700 1 -- 172.21.15.32:0/3018425670 shutdown_connections 2020-12-05T17:37:55.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.462+0000 7f8814f1a700 1 -- 172.21.15.32:0/3018425670 wait complete. 2020-12-05T17:37:55.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.466+0000 7f8814f1a700 1 Processor -- start 2020-12-05T17:37:55.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.466+0000 7f8814f1a700 1 -- start start 2020-12-05T17:37:55.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.466+0000 7f8814f1a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88100ab710 0x7f88100acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:55.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.466+0000 7f8814f1a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88100b1fc0 0x7f88100a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:55.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.466+0000 7f8814f1a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8810147400 0x7f88100a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:55.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.466+0000 7f8814f1a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88100a1fc0 con 0x7f88100ab710 2020-12-05T17:37:55.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.466+0000 7f8814f1a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881009f1b0 con 0x7f8810147400 2020-12-05T17:37:55.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.466+0000 7f8814f1a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f881009ef10 con 0x7f88100b1fc0 2020-12-05T17:37:55.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.466+0000 7f880f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88100ab710 0x7f88100acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:55.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.466+0000 7f880f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88100ab710 0x7f88100acd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36902/0 (socket says 172.21.15.32:36902) 2020-12-05T17:37:55.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.466+0000 7f880f7fe700 1 -- 172.21.15.32:0/1267982789 learned_addr learned my addr 172.21.15.32:0/1267982789 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:55.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.466+0000 7f880effd700 1 --2- 172.21.15.32:0/1267982789 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88100b1fc0 0x7f88100a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:55.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.466+0000 7f880ffff700 1 --2- 172.21.15.32:0/1267982789 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8810147400 0x7f88100a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:55.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.466+0000 7f880f7fe700 1 -- 172.21.15.32:0/1267982789 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88100b1fc0 msgr2=0x7f88100a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:55.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.466+0000 7f880f7fe700 1 --2- 172.21.15.32:0/1267982789 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88100b1fc0 0x7f88100a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:55.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.466+0000 7f880f7fe700 1 -- 172.21.15.32:0/1267982789 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8810147400 msgr2=0x7f88100a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:55.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.466+0000 7f880f7fe700 1 --2- 172.21.15.32:0/1267982789 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8810147400 0x7f88100a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:55.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.466+0000 7f880f7fe700 1 -- 172.21.15.32:0/1267982789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f881014da90 con 0x7f88100ab710 2020-12-05T17:37:55.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.466+0000 7f880ffff700 1 --2- 172.21.15.32:0/1267982789 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8810147400 0x7f88100a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:55.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.466+0000 7f880effd700 1 --2- 172.21.15.32:0/1267982789 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88100b1fc0 0x7f88100a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:55.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.466+0000 7f880f7fe700 1 --2- 172.21.15.32:0/1267982789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88100ab710 0x7f88100acd40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7f881014d1e0 tx=0x7f8800008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:55.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.466+0000 7f880cff9700 1 -- 172.21.15.32:0/1267982789 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8800007710 con 0x7f88100ab710 2020-12-05T17:37:55.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.470+0000 7f880cff9700 1 -- 172.21.15.32:0/1267982789 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8800007870 con 0x7f88100ab710 2020-12-05T17:37:55.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.470+0000 7f8814f1a700 1 -- 172.21.15.32:0/1267982789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88100ad230 con 0x7f88100ab710 2020-12-05T17:37:55.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.470+0000 7f880cff9700 1 -- 172.21.15.32:0/1267982789 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8800007bd0 con 0x7f88100ab710 2020-12-05T17:37:55.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.470+0000 7f8814f1a700 1 -- 172.21.15.32:0/1267982789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88100ad9c0 con 0x7f88100ab710 2020-12-05T17:37:55.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.470+0000 7f8814f1a700 1 -- 172.21.15.32:0/1267982789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8810057050 con 0x7f88100ab710 2020-12-05T17:37:55.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.470+0000 7f880cff9700 1 -- 172.21.15.32:0/1267982789 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8800007d30 con 0x7f88100ab710 2020-12-05T17:37:55.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.470+0000 7f880cff9700 1 --2- 172.21.15.32:0/1267982789 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f87e80554e0 0x7f87e80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:55.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.470+0000 7f880cff9700 1 -- 172.21.15.32:0/1267982789 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f880006e2d0 con 0x7f88100ab710 2020-12-05T17:37:55.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.470+0000 7f880effd700 1 --2- 172.21.15.32:0/1267982789 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f87e80554e0 0x7f87e80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:55.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.470+0000 7f880effd700 1 --2- 172.21.15.32:0/1267982789 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f87e80554e0 0x7f87e80579a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f87f801a000 tx=0x7f87f8015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:55.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.474+0000 7f880cff9700 1 -- 172.21.15.32:0/1267982789 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8800044470 con 0x7f88100ab710 2020-12-05T17:37:55.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.662+0000 7f8814f1a700 1 -- 172.21.15.32:0/1267982789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88100abe90 con 0x7f88100ab710 2020-12-05T17:37:55.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.662+0000 7f880cff9700 1 -- 172.21.15.32:0/1267982789 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f8800047a90 con 0x7f88100ab710 2020-12-05T17:37:55.669 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:55.669 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":32,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5645,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/992428276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":992428276},{"type":"v1","addr":"172.21.15.32:6835","nonce":992428276}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:37:46.495876+0000","modified":"2020-12-05T17:37:47.498743+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6160":{"gid":6160,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.61:6833/4214217205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4214217205},{"type":"v1","addr":"172.21.15.61:6833","nonce":4214217205}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-12-05T17:37:55.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.666+0000 7f8814f1a700 1 -- 172.21.15.32:0/1267982789 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f87e80554e0 msgr2=0x7f87e80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:55.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.666+0000 7f8814f1a700 1 --2- 172.21.15.32:0/1267982789 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f87e80554e0 0x7f87e80579a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f87f801a000 tx=0x7f87f8015000).stop 2020-12-05T17:37:55.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.666+0000 7f8814f1a700 1 -- 172.21.15.32:0/1267982789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88100ab710 msgr2=0x7f88100acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:55.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.666+0000 7f8814f1a700 1 --2- 172.21.15.32:0/1267982789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88100ab710 0x7f88100acd40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7f881014d1e0 tx=0x7f8800008040).stop 2020-12-05T17:37:55.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.666+0000 7f8814f1a700 1 -- 172.21.15.32:0/1267982789 shutdown_connections 2020-12-05T17:37:55.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.666+0000 7f8814f1a700 1 --2- 172.21.15.32:0/1267982789 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8810147400 0x7f88100a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:55.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.666+0000 7f8814f1a700 1 --2- 172.21.15.32:0/1267982789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88100ab710 0x7f88100acd40 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:55.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.666+0000 7f8814f1a700 1 --2- 172.21.15.32:0/1267982789 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f87e80554e0 0x7f87e80579a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:55.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.666+0000 7f8814f1a700 1 --2- 172.21.15.32:0/1267982789 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88100b1fc0 0x7f88100a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:55.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.666+0000 7f8814f1a700 1 -- 172.21.15.32:0/1267982789 >> 172.21.15.32:0/1267982789 conn(0x7f88100b5530 msgr2=0x7f8810144ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:55.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.666+0000 7f8814f1a700 1 -- 172.21.15.32:0/1267982789 shutdown_connections 2020-12-05T17:37:55.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:55.666+0000 7f8814f1a700 1 -- 172.21.15.32:0/1267982789 wait complete. 2020-12-05T17:37:55.673 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 32 2020-12-05T17:37:55.885 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:37:46.495876+0000', 'modified': '2020-12-05T17:37:47.498743+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6160}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6160': {'gid': 6160, 'name': 'b', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.61:6833/4214217205', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6832', 'nonce': 4214217205}, {'type': 'v1', 'addr': '172.21.15.61:6833', 'nonce': 4214217205}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:37:55.885 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:37:55.886 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-12-05T17:37:56.016 INFO:teuthology.orchestra.run.smithi061.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":6160,"want_state":"up:active","state":"up:active","rank_uptime":9.5059864760000004,"mdsmap_epoch":32,"osdmap_epoch":72,"osdmap_epoch_barrier":71,"uptime":13.838650565} 2020-12-05T17:37:56.016 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 6160, "mdsmap_epoch": 32, "osdmap_epoch": 72, "osdmap_epoch_barrier": 71, "rank_uptime": 9.505986476, "state": "up:active", "uptime": 13.838650565, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:37:56.017 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:37:56.017 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-12-05T17:37:56.018 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi002.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-12-05T17:37:56.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:56.018 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:37:56.027 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:37:56.085 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:37:56.085 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:37:56.139 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:37:56.140 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:37:56.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:56.141 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:37:56.187 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:37:56.187 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:37:56.235 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:37:56.252 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:37:56.253 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:37:56.310 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:37:56.312 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:37:56.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:56.313 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:37:56.314 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:2020-12-05T17:37:56.307+0000 7f475d2b10c0 -1 init, newargv = 0x556e8b3f0b10 newargc=9 2020-12-05T17:37:56.314 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[10556]: starting ceph client 2020-12-05T17:37:56.324 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[10556]: starting fuse 2020-12-05T17:37:57.421 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:37:57.437 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:37:57.437 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:37:57.495 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:37:57.497 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:37:57.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:57.498 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:37:57.547 INFO:teuthology.orchestra.run.smithi002.stdout:50 2020-12-05T17:37:57.547 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [50] 2020-12-05T17:37:57.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:57.548 INFO:teuthology.orchestra.run.smithi002:> sudo python3 -c ' 2020-12-05T17:37:57.549 INFO:teuthology.orchestra.run.smithi002:> import glob 2020-12-05T17:37:57.549 INFO:teuthology.orchestra.run.smithi002:> import re 2020-12-05T17:37:57.549 INFO:teuthology.orchestra.run.smithi002:> import os 2020-12-05T17:37:57.550 INFO:teuthology.orchestra.run.smithi002:> import subprocess 2020-12-05T17:37:57.550 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:37:57.550 INFO:teuthology.orchestra.run.smithi002:> def find_socket(client_name): 2020-12-05T17:37:57.551 INFO:teuthology.orchestra.run.smithi002:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-12-05T17:37:57.551 INFO:teuthology.orchestra.run.smithi002:> files = glob.glob(asok_path) 2020-12-05T17:37:57.552 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:37:57.552 INFO:teuthology.orchestra.run.smithi002:> # Given a non-glob path, it better be there 2020-12-05T17:37:57.552 INFO:teuthology.orchestra.run.smithi002:> if "*" not in asok_path: 2020-12-05T17:37:57.553 INFO:teuthology.orchestra.run.smithi002:> assert(len(files) == 1) 2020-12-05T17:37:57.553 INFO:teuthology.orchestra.run.smithi002:> return files[0] 2020-12-05T17:37:57.554 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:37:57.554 INFO:teuthology.orchestra.run.smithi002:> for f in files: 2020-12-05T17:37:57.555 INFO:teuthology.orchestra.run.smithi002:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:37:57.555 INFO:teuthology.orchestra.run.smithi002:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:37:57.555 INFO:teuthology.orchestra.run.smithi002:> return f 2020-12-05T17:37:57.556 INFO:teuthology.orchestra.run.smithi002:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:37:57.556 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:37:57.557 INFO:teuthology.orchestra.run.smithi002:> print(find_socket("client.1")) 2020-12-05T17:37:57.557 INFO:teuthology.orchestra.run.smithi002:> ' 2020-12-05T17:37:57.637 INFO:teuthology.orchestra.run.smithi002.stdout:/var/run/ceph/ceph-client.1.10556.asok 2020-12-05T17:37:57.643 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.10556.asok 2020-12-05T17:37:57.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:57.644 INFO:teuthology.orchestra.run.smithi002:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.10556.asok status 2020-12-05T17:37:57.789 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2020-12-05T17:37:57.790 INFO:teuthology.orchestra.run.smithi002.stdout: "metadata": { 2020-12-05T17:37:57.790 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:37:57.790 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:37:57.791 INFO:teuthology.orchestra.run.smithi002.stdout: "entity_id": "1", 2020-12-05T17:37:57.791 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": "smithi002", 2020-12-05T17:37:57.791 INFO:teuthology.orchestra.run.smithi002.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-12-05T17:37:57.791 INFO:teuthology.orchestra.run.smithi002.stdout: "pid": "10556", 2020-12-05T17:37:57.792 INFO:teuthology.orchestra.run.smithi002.stdout: "root": "/" 2020-12-05T17:37:57.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:37:57.792 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_count": 0, 2020-12-05T17:37:57.792 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_pinned_count": 0, 2020-12-05T17:37:57.792 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 6453, 2020-12-05T17:37:57.793 INFO:teuthology.orchestra.run.smithi002.stdout: "inst": { 2020-12-05T17:37:57.793 INFO:teuthology.orchestra.run.smithi002.stdout: "name": { 2020-12-05T17:37:57.793 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "client", 2020-12-05T17:37:57.793 INFO:teuthology.orchestra.run.smithi002.stdout: "num": 6453 2020-12-05T17:37:57.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:37:57.794 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2020-12-05T17:37:57.794 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2020-12-05T17:37:57.794 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2020-12-05T17:37:57.794 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 596534474 2020-12-05T17:37:57.794 INFO:teuthology.orchestra.run.smithi002.stdout: } 2020-12-05T17:37:57.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:37:57.795 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2020-12-05T17:37:57.795 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2020-12-05T17:37:57.795 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2020-12-05T17:37:57.795 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 596534474 2020-12-05T17:37:57.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:37:57.796 INFO:teuthology.orchestra.run.smithi002.stdout: "inst_str": "client.6453 172.21.15.2:0/596534474", 2020-12-05T17:37:57.796 INFO:teuthology.orchestra.run.smithi002.stdout: "addr_str": "172.21.15.2:0/596534474", 2020-12-05T17:37:57.796 INFO:teuthology.orchestra.run.smithi002.stdout: "inode_count": 1, 2020-12-05T17:37:57.796 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_epoch": 32, 2020-12-05T17:37:57.797 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch": 72, 2020-12-05T17:37:57.797 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:37:57.797 INFO:teuthology.orchestra.run.smithi002.stdout: "blacklisted": false 2020-12-05T17:37:57.797 INFO:teuthology.orchestra.run.smithi002.stdout:} 2020-12-05T17:37:57.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:57.800 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:37:57.800 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:37:57.807 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:37:57.808 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:37:57.808 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:57.809 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:37:57.866 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:37:57.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f06b3517700 1 Processor -- start 2020-12-05T17:37:57.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f06b3517700 1 -- start start 2020-12-05T17:37:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f06b3517700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f06ac0afe50 0x7f06ac146a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f06b3517700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06ac0b0230 con 0x7f06ac0b3aa0 2020-12-05T17:37:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f06b3517700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06ac0b6a50 con 0x7f06ac0b1880 2020-12-05T17:37:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f06b3517700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06ac146080 con 0x7f06ac0afe50 2020-12-05T17:37:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f06b0ab2700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f06ac0b3aa0 0x7f06ac1429e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39558/0 (socket says 172.21.15.32:39558) 2020-12-05T17:37:57.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f06b0ab2700 1 -- 172.21.15.32:0/2599540536 learned_addr learned my addr 172.21.15.32:0/2599540536 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:37:57.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f06b12b3700 1 --2- 172.21.15.32:0/2599540536 >> v2:172.21.15.61:3300/0 conn(0x7f06ac0afe50 0x7f06ac146a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:57.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f069ffff700 1 -- 172.21.15.32:0/2599540536 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 33345313 0 0) 0x7f06ac0b0230 con 0x7f06ac0b3aa0 2020-12-05T17:37:57.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f069ffff700 1 -- 172.21.15.32:0/2599540536 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0690003230 con 0x7f06ac0b3aa0 2020-12-05T17:37:57.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f069ffff700 1 -- 172.21.15.32:0/2599540536 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749883709 0 0) 0x7f06ac0b6a50 con 0x7f06ac0b1880 2020-12-05T17:37:57.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f069ffff700 1 -- 172.21.15.32:0/2599540536 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06ac0b0230 con 0x7f06ac0b1880 2020-12-05T17:37:57.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f069ffff700 1 -- 172.21.15.32:0/2599540536 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3462141267 0 0) 0x7f0690003230 con 0x7f06ac0b3aa0 2020-12-05T17:37:57.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f069ffff700 1 -- 172.21.15.32:0/2599540536 >> v2:172.21.15.61:3300/0 conn(0x7f06ac0afe50 msgr2=0x7f06ac146a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:57.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f069ffff700 1 --2- 172.21.15.32:0/2599540536 >> v2:172.21.15.61:3300/0 conn(0x7f06ac0afe50 0x7f06ac146a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f069ffff700 1 -- 172.21.15.32:0/2599540536 >> v1:172.21.15.61:6789/0 conn(0x7f06ac0b1880 legacy=0x7f06ac13f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f069ffff700 1 -- 172.21.15.32:0/2599540536 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ac146320 con 0x7f06ac0b3aa0 2020-12-05T17:37:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f069ffff700 1 -- 172.21.15.32:0/2599540536 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f06a80016d0 con 0x7f06ac0b3aa0 2020-12-05T17:37:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f069ffff700 1 -- 172.21.15.32:0/2599540536 >> v1:172.21.15.32:6789/0 conn(0x7f06ac0b3aa0 legacy=0x7f06ac1429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f069ffff700 1 --2- 172.21.15.32:0/2599540536 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06900038b0 0x7f0690003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f069ffff700 1 --2- 172.21.15.32:0/2599540536 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06900044d0 0x7f0690004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f069ffff700 1 --2- 172.21.15.32:0/2599540536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0690004e40 0x7f069000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f069ffff700 1 -- 172.21.15.32:0/2599540536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac146080 con 0x7f0690004e40 2020-12-05T17:37:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f069ffff700 1 -- 172.21.15.32:0/2599540536 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06900053a0 con 0x7f06900038b0 2020-12-05T17:37:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.982+0000 7f069ffff700 1 -- 172.21.15.32:0/2599540536 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069000bc90 con 0x7f06900044d0 2020-12-05T17:37:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.986+0000 7f06b1ab4700 1 --2- 172.21.15.32:0/2599540536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0690004e40 0x7f069000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.986+0000 7f06b12b3700 1 --2- 172.21.15.32:0/2599540536 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06900038b0 0x7f0690003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.986+0000 7f06b0ab2700 1 --2- 172.21.15.32:0/2599540536 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06900044d0 0x7f0690004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.986+0000 7f06b1ab4700 1 -- 172.21.15.32:0/2599540536 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06900044d0 msgr2=0x7f0690004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.986+0000 7f06b1ab4700 1 --2- 172.21.15.32:0/2599540536 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06900044d0 0x7f0690004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:57.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.986+0000 7f06b1ab4700 1 -- 172.21.15.32:0/2599540536 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06900038b0 msgr2=0x7f0690003cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:57.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.986+0000 7f06b1ab4700 1 --2- 172.21.15.32:0/2599540536 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06900038b0 0x7f0690003cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:57.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.986+0000 7f06b1ab4700 1 -- 172.21.15.32:0/2599540536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0690003230 con 0x7f0690004e40 2020-12-05T17:37:57.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.986+0000 7f06b1ab4700 1 -- 172.21.15.32:0/2599540536 reap_dead start 2020-12-05T17:37:57.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.986+0000 7f06b1ab4700 1 --2- 172.21.15.32:0/2599540536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0690004e40 0x7f069000b720 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7f06a4001450 tx=0x7f06a4007ee0).ready entity=mon.0 client_cookie=2b1a19bb50b6d090 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:57.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.986+0000 7f069ffff700 1 -- 172.21.15.32:0/2599540536 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06a400cc00 con 0x7f0690004e40 2020-12-05T17:37:57.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.986+0000 7f069ffff700 1 -- 172.21.15.32:0/2599540536 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a400cd60 con 0x7f0690004e40 2020-12-05T17:37:57.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.986+0000 7f069ffff700 1 -- 172.21.15.32:0/2599540536 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06a40125e0 con 0x7f0690004e40 2020-12-05T17:37:57.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.986+0000 7f06b3517700 1 -- 172.21.15.32:0/2599540536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0690004e40 msgr2=0x7f069000b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:57.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.986+0000 7f06b3517700 1 --2- 172.21.15.32:0/2599540536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0690004e40 0x7f069000b720 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7f06a4001450 tx=0x7f06a4007ee0).stop 2020-12-05T17:37:58.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.986+0000 7f06b3517700 1 -- 172.21.15.32:0/2599540536 shutdown_connections 2020-12-05T17:37:58.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.986+0000 7f06b3517700 1 --2- 172.21.15.32:0/2599540536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0690004e40 0x7f069000b720 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:58.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.986+0000 7f06b3517700 1 -- 172.21.15.32:0/2599540536 >> 172.21.15.32:0/2599540536 conn(0x7f06ac0b9100 msgr2=0x7f06ac0b9510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:58.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.986+0000 7f06b3517700 1 -- 172.21.15.32:0/2599540536 shutdown_connections 2020-12-05T17:37:58.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.986+0000 7f06b3517700 1 -- 172.21.15.32:0/2599540536 wait complete. 2020-12-05T17:37:58.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.986+0000 7f06b3517700 1 Processor -- start 2020-12-05T17:37:58.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.986+0000 7f06b3517700 1 -- start start 2020-12-05T17:37:58.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f06b3517700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0690004e40 0x7f06ac0621c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:58.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f06b3517700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06ac0b85b0 0x7f06ac154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:58.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f06b3517700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06ac155420 0x7f06ac1426f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:58.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f06b3517700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac0b8e60 con 0x7f06ac0b85b0 2020-12-05T17:37:58.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f06b3517700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac154ad0 con 0x7f06ac155420 2020-12-05T17:37:58.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f06b3517700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac13be20 con 0x7f0690004e40 2020-12-05T17:37:58.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f06b0ab2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06ac0b85b0 0x7f06ac154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:58.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f06b0ab2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06ac0b85b0 0x7f06ac154f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36922/0 (socket says 172.21.15.32:36922) 2020-12-05T17:37:58.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f06b0ab2700 1 -- 172.21.15.32:0/1995548196 learned_addr learned my addr 172.21.15.32:0/1995548196 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:58.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f06b1ab4700 1 --2- 172.21.15.32:0/1995548196 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06ac155420 0x7f06ac1426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:58.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f06b12b3700 1 --2- 172.21.15.32:0/1995548196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0690004e40 0x7f06ac0621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:58.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f06b0ab2700 1 -- 172.21.15.32:0/1995548196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0690004e40 msgr2=0x7f06ac0621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:58.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f06b0ab2700 1 --2- 172.21.15.32:0/1995548196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0690004e40 0x7f06ac0621c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:58.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f06b0ab2700 1 -- 172.21.15.32:0/1995548196 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06ac155420 msgr2=0x7f06ac1426f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:58.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f06b0ab2700 1 --2- 172.21.15.32:0/1995548196 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06ac155420 0x7f06ac1426f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:58.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f06b0ab2700 1 -- 172.21.15.32:0/1995548196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ac13c0c0 con 0x7f06ac0b85b0 2020-12-05T17:37:58.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f06b1ab4700 1 --2- 172.21.15.32:0/1995548196 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06ac155420 0x7f06ac1426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:37:58.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f06b0ab2700 1 --2- 172.21.15.32:0/1995548196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06ac0b85b0 0x7f06ac154f30 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7f06a800d000 tx=0x7f06ac1464b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:58.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f069effd700 1 -- 172.21.15.32:0/1995548196 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06a800fa30 con 0x7f06ac0b85b0 2020-12-05T17:37:58.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f069effd700 1 -- 172.21.15.32:0/1995548196 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a800fb90 con 0x7f06ac0b85b0 2020-12-05T17:37:58.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f06b3517700 1 -- 172.21.15.32:0/1995548196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06ac13bb10 con 0x7f06ac0b85b0 2020-12-05T17:37:58.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f069effd700 1 -- 172.21.15.32:0/1995548196 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06a8012350 con 0x7f06ac0b85b0 2020-12-05T17:37:58.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f06b3517700 1 -- 172.21.15.32:0/1995548196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ac13c420 con 0x7f06ac0b85b0 2020-12-05T17:37:58.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.990+0000 7f06b3517700 1 -- 172.21.15.32:0/1995548196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06ac057050 con 0x7f06ac0b85b0 2020-12-05T17:37:58.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.994+0000 7f069effd700 1 -- 172.21.15.32:0/1995548196 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f06a801b030 con 0x7f06ac0b85b0 2020-12-05T17:37:58.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.994+0000 7f069effd700 1 --2- 172.21.15.32:0/1995548196 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0688055500 0x7f06880579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:58.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.994+0000 7f069effd700 1 -- 172.21.15.32:0/1995548196 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f06a806d930 con 0x7f06ac0b85b0 2020-12-05T17:37:58.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.994+0000 7f06b12b3700 1 --2- 172.21.15.32:0/1995548196 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0688055500 0x7f06880579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:58.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:57.994+0000 7f06b12b3700 1 --2- 172.21.15.32:0/1995548196 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0688055500 0x7f06880579c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f06a0008000 tx=0x7f06a0008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:58.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.002+0000 7f069effd700 1 -- 172.21.15.32:0/1995548196 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f06a8043ad0 con 0x7f06ac0b85b0 2020-12-05T17:37:58.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.190+0000 7f06b3517700 1 -- 172.21.15.32:0/1995548196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06ac069690 con 0x7f06ac0b85b0 2020-12-05T17:37:58.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.190+0000 7f069effd700 1 -- 172.21.15.32:0/1995548196 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f06a806be60 con 0x7f06ac0b85b0 2020-12-05T17:37:58.198 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:58.198 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":32,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5645,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/992428276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":992428276},{"type":"v1","addr":"172.21.15.32:6835","nonce":992428276}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:37:46.495876+0000","modified":"2020-12-05T17:37:47.498743+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6160":{"gid":6160,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.61:6833/4214217205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4214217205},{"type":"v1","addr":"172.21.15.61:6833","nonce":4214217205}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-12-05T17:37:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.194+0000 7f06b3517700 1 -- 172.21.15.32:0/1995548196 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0688055500 msgr2=0x7f06880579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.194+0000 7f06b3517700 1 --2- 172.21.15.32:0/1995548196 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0688055500 0x7f06880579c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f06a0008000 tx=0x7f06a0008040).stop 2020-12-05T17:37:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.194+0000 7f06b3517700 1 -- 172.21.15.32:0/1995548196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06ac0b85b0 msgr2=0x7f06ac154f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.194+0000 7f06b3517700 1 --2- 172.21.15.32:0/1995548196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06ac0b85b0 0x7f06ac154f30 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7f06a800d000 tx=0x7f06ac1464b0).stop 2020-12-05T17:37:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.194+0000 7f06b3517700 1 -- 172.21.15.32:0/1995548196 shutdown_connections 2020-12-05T17:37:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.194+0000 7f06b3517700 1 --2- 172.21.15.32:0/1995548196 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06ac155420 0x7f06ac1426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.194+0000 7f06b3517700 1 --2- 172.21.15.32:0/1995548196 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0688055500 0x7f06880579c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.194+0000 7f06b3517700 1 --2- 172.21.15.32:0/1995548196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0690004e40 0x7f06ac0621c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:58.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.194+0000 7f06b3517700 1 --2- 172.21.15.32:0/1995548196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06ac0b85b0 0x7f06ac154f30 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:58.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.194+0000 7f06b3517700 1 -- 172.21.15.32:0/1995548196 >> 172.21.15.32:0/1995548196 conn(0x7f06ac0b9100 msgr2=0x7f06ac0b6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:58.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.194+0000 7f06b3517700 1 -- 172.21.15.32:0/1995548196 shutdown_connections 2020-12-05T17:37:58.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.194+0000 7f06b3517700 1 -- 172.21.15.32:0/1995548196 wait complete. 2020-12-05T17:37:58.203 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 32 2020-12-05T17:37:58.212 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:37:58.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.322+0000 7f2e32fcb700 1 Processor -- start 2020-12-05T17:37:58.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.322+0000 7f2e32fcb700 1 -- start start 2020-12-05T17:37:58.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.322+0000 7f2e32fcb700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f2e2c0b7720 0x7f2e2c061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:58.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.322+0000 7f2e32fcb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e2c1466b0 0x7f2e2c14a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:58.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e32fcb700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2c14a090 con 0x7f2e2c0b12d0 2020-12-05T17:37:58.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e32fcb700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e2c0b1730 con 0x7f2e2c0b7720 2020-12-05T17:37:58.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e32fcb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c0b7b00 con 0x7f2e2c1466b0 2020-12-05T17:37:58.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e327ca700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e2c1466b0 0x7f2e2c14a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:58.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e327ca700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e2c1466b0 0x7f2e2c14a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:33722/0 (socket says 172.21.15.32:33722) 2020-12-05T17:37:58.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e317c8700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f2e2c0b12d0 0x7f2e2c145fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:52218/0 (socket says 172.21.15.32:52218) 2020-12-05T17:37:58.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e327ca700 1 -- 172.21.15.32:0/1345031630 learned_addr learned my addr 172.21.15.32:0/1345031630 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:58.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e31fc9700 1 --2- 172.21.15.32:0/1345031630 >> v2:172.21.15.61:3300/0 conn(0x7f2e2c0b7720 0x7f2e2c061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:58.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e327ca700 1 -- 172.21.15.32:0/1345031630 >> v2:172.21.15.61:3300/0 conn(0x7f2e2c0b7720 msgr2=0x7f2e2c061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:37:58.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e327ca700 1 --2- 172.21.15.32:0/1345031630 >> v2:172.21.15.61:3300/0 conn(0x7f2e2c0b7720 0x7f2e2c061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:58.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e327ca700 1 -- 172.21.15.32:0/1345031630 >> v1:172.21.15.61:6789/0 conn(0x7f2e2c0b12d0 legacy=0x7f2e2c145fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:58.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e327ca700 1 -- 172.21.15.32:0/1345031630 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c0626f0 con 0x7f2e2c1466b0 2020-12-05T17:37:58.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e30fc7700 1 -- 172.21.15.32:0/1345031630 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3079148482 0 0) 0x7f2e2c14a090 con 0x7f2e2c0b12d0 2020-12-05T17:37:58.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e327ca700 1 --2- 172.21.15.32:0/1345031630 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e2c1466b0 0x7f2e2c14a4f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f2e24002f30 tx=0x7f2e24006e80).ready entity=mon.2 client_cookie=2e393c7ea680eac7 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:58.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e30fc7700 1 -- 172.21.15.32:0/1345031630 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e240097f0 con 0x7f2e2c1466b0 2020-12-05T17:37:58.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e30fc7700 1 -- 172.21.15.32:0/1345031630 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e24009950 con 0x7f2e2c1466b0 2020-12-05T17:37:58.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e30fc7700 1 -- 172.21.15.32:0/1345031630 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e24009cb0 con 0x7f2e2c1466b0 2020-12-05T17:37:58.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e32fcb700 1 -- 172.21.15.32:0/1345031630 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e2c1466b0 msgr2=0x7f2e2c14a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:58.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e32fcb700 1 --2- 172.21.15.32:0/1345031630 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e2c1466b0 0x7f2e2c14a4f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f2e24002f30 tx=0x7f2e24006e80).stop 2020-12-05T17:37:58.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e32fcb700 1 -- 172.21.15.32:0/1345031630 shutdown_connections 2020-12-05T17:37:58.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e32fcb700 1 --2- 172.21.15.32:0/1345031630 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e2c1466b0 0x7f2e2c14a4f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:58.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e32fcb700 1 --2- 172.21.15.32:0/1345031630 >> v2:172.21.15.61:3300/0 conn(0x7f2e2c0b7720 0x7f2e2c061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:58.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e32fcb700 1 -- 172.21.15.32:0/1345031630 >> 172.21.15.32:0/1345031630 conn(0x7f2e2c05bc20 msgr2=0x7f2e2c0b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:58.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e32fcb700 1 -- 172.21.15.32:0/1345031630 shutdown_connections 2020-12-05T17:37:58.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.326+0000 7f2e32fcb700 1 -- 172.21.15.32:0/1345031630 wait complete. 2020-12-05T17:37:58.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e32fcb700 1 Processor -- start 2020-12-05T17:37:58.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e32fcb700 1 -- start start 2020-12-05T17:37:58.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e32fcb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e2c0b12d0 0x7f2e2c0b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:58.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e32fcb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e2c0b7720 0x7f2e2c14b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:58.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e32fcb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e2c1466b0 0x7f2e2c14b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:58.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e32fcb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c0664a0 con 0x7f2e2c0b7720 2020-12-05T17:37:58.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e32fcb700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c0667c0 con 0x7f2e2c1466b0 2020-12-05T17:37:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e32fcb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c066a60 con 0x7f2e2c0b12d0 2020-12-05T17:37:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e317c8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e2c0b7720 0x7f2e2c14b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e317c8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e2c0b7720 0x7f2e2c14b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36938/0 (socket says 172.21.15.32:36938) 2020-12-05T17:37:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e317c8700 1 -- 172.21.15.32:0/1368913619 learned_addr learned my addr 172.21.15.32:0/1368913619 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:37:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e31fc9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e2c0b12d0 0x7f2e2c0b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e327ca700 1 --2- 172.21.15.32:0/1368913619 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e2c1466b0 0x7f2e2c14b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e317c8700 1 -- 172.21.15.32:0/1368913619 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e2c0b12d0 msgr2=0x7f2e2c0b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e317c8700 1 --2- 172.21.15.32:0/1368913619 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e2c0b12d0 0x7f2e2c0b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e317c8700 1 -- 172.21.15.32:0/1368913619 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e2c1466b0 msgr2=0x7f2e2c14b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e317c8700 1 --2- 172.21.15.32:0/1368913619 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e2c1466b0 0x7f2e2c14b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e317c8700 1 -- 172.21.15.32:0/1368913619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c066120 con 0x7f2e2c0b7720 2020-12-05T17:37:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e317c8700 1 --2- 172.21.15.32:0/1368913619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e2c0b7720 0x7f2e2c14b100 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7f2e28001320 tx=0x7f2e28007e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e22ffd700 1 -- 172.21.15.32:0/1368913619 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e28005ab0 con 0x7f2e2c0b7720 2020-12-05T17:37:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e22ffd700 1 -- 172.21.15.32:0/1368913619 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e28005c10 con 0x7f2e2c0b7720 2020-12-05T17:37:58.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e32fcb700 1 -- 172.21.15.32:0/1368913619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e2c14cdd0 con 0x7f2e2c0b7720 2020-12-05T17:37:58.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e22ffd700 1 -- 172.21.15.32:0/1368913619 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e28007110 con 0x7f2e2c0b7720 2020-12-05T17:37:58.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e32fcb700 1 -- 172.21.15.32:0/1368913619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2c14d3d0 con 0x7f2e2c0b7720 2020-12-05T17:37:58.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.330+0000 7f2e32fcb700 1 -- 172.21.15.32:0/1368913619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e2c066c20 con 0x7f2e2c0b7720 2020-12-05T17:37:58.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.334+0000 7f2e22ffd700 1 -- 172.21.15.32:0/1368913619 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2e28010030 con 0x7f2e2c0b7720 2020-12-05T17:37:58.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.334+0000 7f2e22ffd700 1 --2- 172.21.15.32:0/1368913619 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2e0c055440 0x7f2e0c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:37:58.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.334+0000 7f2e22ffd700 1 -- 172.21.15.32:0/1368913619 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2e2806e230 con 0x7f2e2c0b7720 2020-12-05T17:37:58.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.334+0000 7f2e31fc9700 1 --2- 172.21.15.32:0/1368913619 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2e0c055440 0x7f2e0c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:37:58.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.334+0000 7f2e31fc9700 1 --2- 172.21.15.32:0/1368913619 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2e0c055440 0x7f2e0c057900 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f2e1c00b000 tx=0x7f2e1c006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:37:58.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.342+0000 7f2e22ffd700 1 -- 172.21.15.32:0/1368913619 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2e280442c0 con 0x7f2e2c0b7720 2020-12-05T17:37:58.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.530+0000 7f2e32fcb700 1 -- 172.21.15.32:0/1368913619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e2c066e20 con 0x7f2e2c0b7720 2020-12-05T17:37:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.530+0000 7f2e22ffd700 1 -- 172.21.15.32:0/1368913619 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f2e280478e0 con 0x7f2e2c0b7720 2020-12-05T17:37:58.535 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:37:58.535 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":32,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5645,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/992428276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":992428276},{"type":"v1","addr":"172.21.15.32:6835","nonce":992428276}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:37:46.495876+0000","modified":"2020-12-05T17:37:47.498743+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6160":{"gid":6160,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.61:6833/4214217205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4214217205},{"type":"v1","addr":"172.21.15.61:6833","nonce":4214217205}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-12-05T17:37:58.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.530+0000 7f2e32fcb700 1 -- 172.21.15.32:0/1368913619 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2e0c055440 msgr2=0x7f2e0c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:58.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.530+0000 7f2e32fcb700 1 --2- 172.21.15.32:0/1368913619 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2e0c055440 0x7f2e0c057900 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f2e1c00b000 tx=0x7f2e1c006dc0).stop 2020-12-05T17:37:58.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.530+0000 7f2e32fcb700 1 -- 172.21.15.32:0/1368913619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e2c0b7720 msgr2=0x7f2e2c14b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:37:58.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.530+0000 7f2e32fcb700 1 --2- 172.21.15.32:0/1368913619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e2c0b7720 0x7f2e2c14b100 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7f2e28001320 tx=0x7f2e28007e90).stop 2020-12-05T17:37:58.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.530+0000 7f2e32fcb700 1 -- 172.21.15.32:0/1368913619 shutdown_connections 2020-12-05T17:37:58.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.530+0000 7f2e32fcb700 1 --2- 172.21.15.32:0/1368913619 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e2c1466b0 0x7f2e2c14b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:58.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.530+0000 7f2e32fcb700 1 --2- 172.21.15.32:0/1368913619 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2e0c055440 0x7f2e0c057900 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:58.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.530+0000 7f2e32fcb700 1 --2- 172.21.15.32:0/1368913619 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e2c0b12d0 0x7f2e2c0b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:58.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.530+0000 7f2e32fcb700 1 --2- 172.21.15.32:0/1368913619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e2c0b7720 0x7f2e2c14b100 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:37:58.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.530+0000 7f2e32fcb700 1 -- 172.21.15.32:0/1368913619 >> 172.21.15.32:0/1368913619 conn(0x7f2e2c05bc20 msgr2=0x7f2e2c0b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:37:58.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.530+0000 7f2e32fcb700 1 -- 172.21.15.32:0/1368913619 shutdown_connections 2020-12-05T17:37:58.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:37:58.530+0000 7f2e32fcb700 1 -- 172.21.15.32:0/1368913619 wait complete. 2020-12-05T17:37:58.540 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 32 2020-12-05T17:37:58.548 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_cap_revoke_eviction_timeout 30.0 2020-12-05T17:37:58.683 INFO:teuthology.orchestra.run.smithi061.stdout:{"success":"mds_cap_revoke_eviction_timeout = '30.000000' "} 2020-12-05T17:37:58.683 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_cap_revoke_eviction_timeout = '30.000000' " } 2020-12-05T17:37:58.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:58.684 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:37:58.685 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:37:58.692 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:37:58.693 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:37:58.694 INFO:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-12-05T17:37:58.694 INFO:teuthology.orchestra.run.smithi036:> import time 2020-12-05T17:37:58.695 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:37:58.695 INFO:teuthology.orchestra.run.smithi036:> with open("/home/ubuntu/cephtest/mnt.0/background_file", '"'"'w'"'"') as f: 2020-12-05T17:37:58.695 INFO:teuthology.orchestra.run.smithi036:> f.write('"'"'content'"'"') 2020-12-05T17:37:58.696 INFO:teuthology.orchestra.run.smithi036:> f.flush() 2020-12-05T17:37:58.696 INFO:teuthology.orchestra.run.smithi036:> f.write('"'"'content2'"'"') 2020-12-05T17:37:58.696 INFO:teuthology.orchestra.run.smithi036:> while True: 2020-12-05T17:37:58.697 INFO:teuthology.orchestra.run.smithi036:> time.sleep(1) 2020-12-05T17:37:58.697 INFO:teuthology.orchestra.run.smithi036:> ' 2020-12-05T17:37:58.737 INFO:teuthology.orchestra.run.smithi036:> sudo ls /home/ubuntu/cephtest/mnt.0/background_file 2020-12-05T17:37:58.759 INFO:teuthology.orchestra.run.smithi036.stderr:ls: cannot access '/home/ubuntu/cephtest/mnt.0/background_file': No such file or directory 2020-12-05T17:37:58.760 DEBUG:teuthology.orchestra.run:got remote process result: 2 2020-12-05T17:37:59.761 INFO:teuthology.orchestra.run.smithi036:> sudo ls /home/ubuntu/cephtest/mnt.0/background_file 2020-12-05T17:37:59.777 INFO:teuthology.orchestra.run.smithi036.stdout:/home/ubuntu/cephtest/mnt.0/background_file 2020-12-05T17:37:59.778 DEBUG:tasks.cephfs.mount:File background_file became visible from 0 after 1s 2020-12-05T17:37:59.779 INFO:teuthology.orchestra.run.smithi002:> sudo ls /home/ubuntu/cephtest/mnt.1/background_file 2020-12-05T17:37:59.819 INFO:teuthology.orchestra.run.smithi002.stdout:/home/ubuntu/cephtest/mnt.1/background_file 2020-12-05T17:37:59.821 DEBUG:tasks.cephfs.mount:File background_file became visible from 1 after 0s 2020-12-05T17:37:59.821 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@smithi036.front.sepia.ceph.com... 2020-12-05T17:37:59.837 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:37:59.838 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:37:59.838 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:37:59.868 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:37:59.869 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:37:59.869 INFO:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-12-05T17:37:59.870 INFO:teuthology.orchestra.run.smithi002:> import os 2020-12-05T17:37:59.870 INFO:teuthology.orchestra.run.smithi002:> import time 2020-12-05T17:37:59.870 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:37:59.871 INFO:teuthology.orchestra.run.smithi002:> fd = os.open("/home/ubuntu/cephtest/mnt.1/background_file", os.O_RDWR | os.O_CREAT, 0o644) 2020-12-05T17:37:59.871 INFO:teuthology.orchestra.run.smithi002:> try: 2020-12-05T17:37:59.871 INFO:teuthology.orchestra.run.smithi002:> while True: 2020-12-05T17:37:59.872 INFO:teuthology.orchestra.run.smithi002:> os.write(fd, b'"'"'content'"'"') 2020-12-05T17:37:59.872 INFO:teuthology.orchestra.run.smithi002:> time.sleep(1) 2020-12-05T17:37:59.872 INFO:teuthology.orchestra.run.smithi002:> if not False: 2020-12-05T17:37:59.873 INFO:teuthology.orchestra.run.smithi002:> break 2020-12-05T17:37:59.873 INFO:teuthology.orchestra.run.smithi002:> except IOError as e: 2020-12-05T17:37:59.873 INFO:teuthology.orchestra.run.smithi002:> pass 2020-12-05T17:37:59.874 INFO:teuthology.orchestra.run.smithi002:> os.close(fd) 2020-12-05T17:37:59.874 INFO:teuthology.orchestra.run.smithi002:> ' 2020-12-05T17:38:15.592 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:38:15.595 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:38:15.598 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:38:15.600 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:38:15.603 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:38:38.235 INFO:tasks.cephfs.test_misc:cap_waiter waited 38.32442092895508s 2020-12-05T17:38:38.236 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist ls --format=json 2020-12-05T17:38:38.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.356+0000 7f0c30d3d700 1 Processor -- start 2020-12-05T17:38:38.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.356+0000 7f0c30d3d700 1 -- start start 2020-12-05T17:38:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.356+0000 7f0c30d3d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c2c0b1fc0 0x7f0c2c0af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.356+0000 7f0c30d3d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0c2c147400 0x7f0c2c145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:38.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.356+0000 7f0c30d3d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2c146a90 con 0x7f0c2c0ab710 2020-12-05T17:38:38.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.356+0000 7f0c30d3d700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2c061e90 con 0x7f0c2c147400 2020-12-05T17:38:38.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.356+0000 7f0c30d3d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c0b0110 con 0x7f0c2c0b1fc0 2020-12-05T17:38:38.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.356+0000 7f0c22ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0c2c0ab710 0x7f0c2c061780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39592/0 (socket says 172.21.15.32:39592) 2020-12-05T17:38:38.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.356+0000 7f0c22ffd700 1 -- 172.21.15.32:0/322020850 learned_addr learned my addr 172.21.15.32:0/322020850 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:38:38.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.356+0000 7f0c2bfff700 1 --2- 172.21.15.32:0/322020850 >> v2:172.21.15.61:3300/0 conn(0x7f0c2c147400 0x7f0c2c145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:38.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.356+0000 7f0c2b7fe700 1 --2- 172.21.15.32:0/322020850 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c2c0b1fc0 0x7f0c2c0af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:38.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c2affd700 1 -- 172.21.15.32:0/322020850 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402446812 0 0) 0x7f0c2c146a90 con 0x7f0c2c0ab710 2020-12-05T17:38:38.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c2b7fe700 1 -- 172.21.15.32:0/322020850 >> v2:172.21.15.61:3300/0 conn(0x7f0c2c147400 msgr2=0x7f0c2c145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:38.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c2b7fe700 1 --2- 172.21.15.32:0/322020850 >> v2:172.21.15.61:3300/0 conn(0x7f0c2c147400 0x7f0c2c145d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:38.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c2b7fe700 1 -- 172.21.15.32:0/322020850 >> v1:172.21.15.32:6789/0 conn(0x7f0c2c0ab710 legacy=0x7f0c2c061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:38.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c2b7fe700 1 -- 172.21.15.32:0/322020850 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c0b23a0 con 0x7f0c2c0b1fc0 2020-12-05T17:38:38.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c2b7fe700 1 --2- 172.21.15.32:0/322020850 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c2c0b1fc0 0x7f0c2c0af230 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f0c1c002f30 tx=0x7f0c1c006dc0).ready entity=mon.2 client_cookie=baafe1f277793448 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:38.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c2affd700 1 -- 172.21.15.32:0/322020850 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c1c009760 con 0x7f0c2c0b1fc0 2020-12-05T17:38:38.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c2affd700 1 -- 172.21.15.32:0/322020850 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1c0098c0 con 0x7f0c2c0b1fc0 2020-12-05T17:38:38.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c2affd700 1 -- 172.21.15.32:0/322020850 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c1c009c20 con 0x7f0c2c0b1fc0 2020-12-05T17:38:38.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c30d3d700 1 -- 172.21.15.32:0/322020850 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c2c0b1fc0 msgr2=0x7f0c2c0af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:38.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c30d3d700 1 --2- 172.21.15.32:0/322020850 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c2c0b1fc0 0x7f0c2c0af230 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f0c1c002f30 tx=0x7f0c1c006dc0).stop 2020-12-05T17:38:38.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c30d3d700 1 -- 172.21.15.32:0/322020850 shutdown_connections 2020-12-05T17:38:38.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c30d3d700 1 --2- 172.21.15.32:0/322020850 >> v2:172.21.15.61:3300/0 conn(0x7f0c2c147400 0x7f0c2c145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:38.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c30d3d700 1 --2- 172.21.15.32:0/322020850 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c2c0b1fc0 0x7f0c2c0af230 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:38.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c30d3d700 1 -- 172.21.15.32:0/322020850 >> 172.21.15.32:0/322020850 conn(0x7f0c2c0b5530 msgr2=0x7f0c2c0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:38.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c30d3d700 1 -- 172.21.15.32:0/322020850 shutdown_connections 2020-12-05T17:38:38.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c30d3d700 1 -- 172.21.15.32:0/322020850 wait complete. 2020-12-05T17:38:38.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c30d3d700 1 Processor -- start 2020-12-05T17:38:38.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c30d3d700 1 -- start start 2020-12-05T17:38:38.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c30d3d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c2c0ab710 0x7f0c2c144b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:38.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c30d3d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c2c0b1fc0 0x7f0c2c1559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:38.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c30d3d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c2c147400 0x7f0c2c155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:38.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c30d3d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c066ef0 con 0x7f0c2c0b1fc0 2020-12-05T17:38:38.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c30d3d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c1555e0 con 0x7f0c2c0ab710 2020-12-05T17:38:38.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c30d3d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c0abb10 con 0x7f0c2c147400 2020-12-05T17:38:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c22ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c2c0b1fc0 0x7f0c2c1559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c22ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c2c0b1fc0 0x7f0c2c1559d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36950/0 (socket says 172.21.15.32:36950) 2020-12-05T17:38:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c22ffd700 1 -- 172.21.15.32:0/2925223078 learned_addr learned my addr 172.21.15.32:0/2925223078 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.360+0000 7f0c2bfff700 1 --2- 172.21.15.32:0/2925223078 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c2c147400 0x7f0c2c155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.364+0000 7f0c2b7fe700 1 --2- 172.21.15.32:0/2925223078 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c2c0ab710 0x7f0c2c144b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.364+0000 7f0c22ffd700 1 -- 172.21.15.32:0/2925223078 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c2c147400 msgr2=0x7f0c2c155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.364+0000 7f0c22ffd700 1 --2- 172.21.15.32:0/2925223078 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c2c147400 0x7f0c2c155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.364+0000 7f0c22ffd700 1 -- 172.21.15.32:0/2925223078 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c2c0ab710 msgr2=0x7f0c2c144b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.364+0000 7f0c22ffd700 1 --2- 172.21.15.32:0/2925223078 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c2c0ab710 0x7f0c2c144b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.364+0000 7f0c22ffd700 1 -- 172.21.15.32:0/2925223078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c14cbd0 con 0x7f0c2c0b1fc0 2020-12-05T17:38:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.364+0000 7f0c2b7fe700 1 --2- 172.21.15.32:0/2925223078 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c2c0ab710 0x7f0c2c144b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:38:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.364+0000 7f0c22ffd700 1 --2- 172.21.15.32:0/2925223078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c2c0b1fc0 0x7f0c2c1559d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f0c14000b80 tx=0x7f0c14007a30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.364+0000 7f0c297fa700 1 -- 172.21.15.32:0/2925223078 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c14007320 con 0x7f0c2c0b1fc0 2020-12-05T17:38:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.364+0000 7f0c297fa700 1 -- 172.21.15.32:0/2925223078 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c14007480 con 0x7f0c2c0b1fc0 2020-12-05T17:38:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.364+0000 7f0c297fa700 1 -- 172.21.15.32:0/2925223078 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c140057d0 con 0x7f0c2c0b1fc0 2020-12-05T17:38:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.364+0000 7f0c30d3d700 1 -- 172.21.15.32:0/2925223078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2c14ce70 con 0x7f0c2c0b1fc0 2020-12-05T17:38:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.364+0000 7f0c30d3d700 1 -- 172.21.15.32:0/2925223078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c2c14dce0 con 0x7f0c2c0b1fc0 2020-12-05T17:38:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.364+0000 7f0c30d3d700 1 -- 172.21.15.32:0/2925223078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c2c057050 con 0x7f0c2c0b1fc0 2020-12-05T17:38:38.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.364+0000 7f0c297fa700 1 -- 172.21.15.32:0/2925223078 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0c14010030 con 0x7f0c2c0b1fc0 2020-12-05T17:38:38.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.364+0000 7f0c297fa700 1 --2- 172.21.15.32:0/2925223078 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c040553f0 0x7f0c040578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:38.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.364+0000 7f0c2b7fe700 1 --2- 172.21.15.32:0/2925223078 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c040553f0 0x7f0c040578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:38.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.364+0000 7f0c297fa700 1 -- 172.21.15.32:0/2925223078 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f0c1406dd80 con 0x7f0c2c0b1fc0 2020-12-05T17:38:38.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.368+0000 7f0c2b7fe700 1 --2- 172.21.15.32:0/2925223078 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c040553f0 0x7f0c040578b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f0c1c018000 tx=0x7f0c1c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:38.381 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.376+0000 7f0c297fa700 1 -- 172.21.15.32:0/2925223078 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0c14043f00 con 0x7f0c2c0b1fc0 2020-12-05T17:38:38.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.536+0000 7f0c30d3d700 1 -- 172.21.15.32:0/2925223078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist ls", "format": "json"} v 0) v1 -- 0x7f0c2c05aab0 con 0x7f0c2c0b1fc0 2020-12-05T17:38:38.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.536+0000 7f0c297fa700 1 -- 172.21.15.32:0/2925223078 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist ls", "format": "json"}]=0 listed 1 entries v73) v1 ==== 98+0+79 (secure 0 0 0) 0x7f0c1406f090 con 0x7f0c2c0b1fc0 2020-12-05T17:38:38.541 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:38:38.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.536+0000 7f0c30d3d700 1 -- 172.21.15.32:0/2925223078 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c040553f0 msgr2=0x7f0c040578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:38.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.536+0000 7f0c30d3d700 1 --2- 172.21.15.32:0/2925223078 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c040553f0 0x7f0c040578b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f0c1c018000 tx=0x7f0c1c015000).stop 2020-12-05T17:38:38.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.536+0000 7f0c30d3d700 1 -- 172.21.15.32:0/2925223078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c2c0b1fc0 msgr2=0x7f0c2c1559d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:38.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.536+0000 7f0c30d3d700 1 --2- 172.21.15.32:0/2925223078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c2c0b1fc0 0x7f0c2c1559d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f0c14000b80 tx=0x7f0c14007a30).stop 2020-12-05T17:38:38.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.536+0000 7f0c30d3d700 1 -- 172.21.15.32:0/2925223078 shutdown_connections 2020-12-05T17:38:38.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.536+0000 7f0c30d3d700 1 --2- 172.21.15.32:0/2925223078 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c040553f0 0x7f0c040578b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:38.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.536+0000 7f0c30d3d700 1 --2- 172.21.15.32:0/2925223078 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c2c147400 0x7f0c2c155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:38.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.536+0000 7f0c30d3d700 1 --2- 172.21.15.32:0/2925223078 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c2c0ab710 0x7f0c2c144b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:38.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.536+0000 7f0c30d3d700 1 --2- 172.21.15.32:0/2925223078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c2c0b1fc0 0x7f0c2c1559d0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:38.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.536+0000 7f0c30d3d700 1 -- 172.21.15.32:0/2925223078 >> 172.21.15.32:0/2925223078 conn(0x7f0c2c0b5530 msgr2=0x7f0c2c149fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:38.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.536+0000 7f0c30d3d700 1 -- 172.21.15.32:0/2925223078 shutdown_connections 2020-12-05T17:38:38.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.536+0000 7f0c30d3d700 1 -- 172.21.15.32:0/2925223078 wait complete. 2020-12-05T17:38:38.546 INFO:teuthology.orchestra.run.smithi032.stderr:listed 1 entries 2020-12-05T17:38:38.554 INFO:teuthology.orchestra.run.smithi032.stdout:[{"addr":"172.21.15.36:0/892894983","until":"2020-12-05T18:38:32.172754+0000"}] 2020-12-05T17:38:38.569 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2020-12-05T17:38:38.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:38:38.570 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:38:38.571 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:38:38.618 INFO:teuthology.orchestra.run.smithi036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2020-12-05T17:38:38.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:38:38.620 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2020-12-05T17:38:38.620 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi036.front.sepia.ceph.com... 2020-12-05T17:38:38.621 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:38:38.621 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:38:38.698 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:38:38.699 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:38:38.699 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:38:38.746 INFO:teuthology.orchestra.run.smithi036.stdout:ext2/ext3 2020-12-05T17:38:38.747 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:38:38.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:38:38.748 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:38:38.794 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:38:38.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.911+0000 7fe13b26b700 1 Processor -- start 2020-12-05T17:38:38.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe13b26b700 1 -- start start 2020-12-05T17:38:38.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe13b26b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe13405f3b0 0x7fe13405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:38.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe13b26b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fe13405feb0 0x7fe1340b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:38.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe13b26b700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe134142450 con 0x7fe134062400 2020-12-05T17:38:38.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe13b26b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1340b42f0 con 0x7fe13405f3b0 2020-12-05T17:38:38.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe13b26b700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe134061780 con 0x7fe13405feb0 2020-12-05T17:38:38.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe13a269700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe13405f3b0 0x7fe13405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:38.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe13aa6a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fe13405feb0 0x7fe1340b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:38.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe13a269700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe13405f3b0 0x7fe13405f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36958/0 (socket says 172.21.15.32:36958) 2020-12-05T17:38:38.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe13a269700 1 -- 172.21.15.32:0/461769227 learned_addr learned my addr 172.21.15.32:0/461769227 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:38.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe139267700 1 -- 172.21.15.32:0/461769227 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3013341658 0 0) 0x7fe134142450 con 0x7fe134062400 2020-12-05T17:38:38.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe13a269700 1 -- 172.21.15.32:0/461769227 >> v2:172.21.15.61:3300/0 conn(0x7fe13405feb0 msgr2=0x7fe1340b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:38.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe13a269700 1 --2- 172.21.15.32:0/461769227 >> v2:172.21.15.61:3300/0 conn(0x7fe13405feb0 0x7fe1340b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:38.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe13a269700 1 -- 172.21.15.32:0/461769227 >> v1:172.21.15.61:6789/0 conn(0x7fe134062400 legacy=0x7fe1340b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:38.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe13a269700 1 -- 172.21.15.32:0/461769227 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe134061a20 con 0x7fe13405f3b0 2020-12-05T17:38:38.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe13a269700 1 --2- 172.21.15.32:0/461769227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13405f3b0 0x7fe13405f790 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7fe124002f30 tx=0x7fe124006e80).ready entity=mon.0 client_cookie=f3e25724d208ebff server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:38.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe139267700 1 -- 172.21.15.32:0/461769227 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1240097f0 con 0x7fe13405f3b0 2020-12-05T17:38:38.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe139267700 1 -- 172.21.15.32:0/461769227 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe124009950 con 0x7fe13405f3b0 2020-12-05T17:38:38.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe139267700 1 -- 172.21.15.32:0/461769227 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe124009cb0 con 0x7fe13405f3b0 2020-12-05T17:38:38.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe13b26b700 1 -- 172.21.15.32:0/461769227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13405f3b0 msgr2=0x7fe13405f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:38.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe13b26b700 1 --2- 172.21.15.32:0/461769227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13405f3b0 0x7fe13405f790 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7fe124002f30 tx=0x7fe124006e80).stop 2020-12-05T17:38:38.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe13b26b700 1 -- 172.21.15.32:0/461769227 shutdown_connections 2020-12-05T17:38:38.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe13b26b700 1 --2- 172.21.15.32:0/461769227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13405f3b0 0x7fe13405f790 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:38.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe13b26b700 1 --2- 172.21.15.32:0/461769227 >> v2:172.21.15.61:3300/0 conn(0x7fe13405feb0 0x7fe1340b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:38.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe13b26b700 1 -- 172.21.15.32:0/461769227 >> 172.21.15.32:0/461769227 conn(0x7fe1340b8ee0 msgr2=0x7fe1340b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:38.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe13b26b700 1 -- 172.21.15.32:0/461769227 shutdown_connections 2020-12-05T17:38:38.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.915+0000 7fe13b26b700 1 -- 172.21.15.32:0/461769227 wait complete. 2020-12-05T17:38:38.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe13b26b700 1 Processor -- start 2020-12-05T17:38:38.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe13b26b700 1 -- start start 2020-12-05T17:38:38.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe13b26b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe13405f3b0 0x7fe134064cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:38.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe13b26b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe13405feb0 0x7fe134154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:38.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe13b26b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe134062400 0x7fe1340b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:38.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe13b26b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe134144730 con 0x7fe134062400 2020-12-05T17:38:38.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe13b26b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe134148d00 con 0x7fe13405feb0 2020-12-05T17:38:38.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe13b26b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe134144960 con 0x7fe13405f3b0 2020-12-05T17:38:38.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe13aa6a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe134062400 0x7fe1340b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:38.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe13aa6a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe134062400 0x7fe1340b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36968/0 (socket says 172.21.15.32:36968) 2020-12-05T17:38:38.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe13aa6a700 1 -- 172.21.15.32:0/2653061199 learned_addr learned my addr 172.21.15.32:0/2653061199 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:38.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe13a269700 1 --2- 172.21.15.32:0/2653061199 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe13405f3b0 0x7fe134064cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:38.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe139a68700 1 --2- 172.21.15.32:0/2653061199 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe13405feb0 0x7fe134154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:38.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe13aa6a700 1 -- 172.21.15.32:0/2653061199 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe13405f3b0 msgr2=0x7fe134064cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:38.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe13aa6a700 1 --2- 172.21.15.32:0/2653061199 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe13405f3b0 0x7fe134064cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:38.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe13aa6a700 1 -- 172.21.15.32:0/2653061199 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe13405feb0 msgr2=0x7fe134154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:38.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe13aa6a700 1 --2- 172.21.15.32:0/2653061199 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe13405feb0 0x7fe134154820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:38.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe13aa6a700 1 -- 172.21.15.32:0/2653061199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1340b2760 con 0x7fe134062400 2020-12-05T17:38:38.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe13aa6a700 1 --2- 172.21.15.32:0/2653061199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe134062400 0x7fe1340b2270 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7fe12c00c000 tx=0x7fe12c006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:38.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe12b7fe700 1 -- 172.21.15.32:0/2653061199 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe12c00ca10 con 0x7fe134062400 2020-12-05T17:38:38.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe12b7fe700 1 -- 172.21.15.32:0/2653061199 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12c00cb70 con 0x7fe134062400 2020-12-05T17:38:38.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe13b26b700 1 -- 172.21.15.32:0/2653061199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe134144c00 con 0x7fe134062400 2020-12-05T17:38:38.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe12b7fe700 1 -- 172.21.15.32:0/2653061199 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe12c00be30 con 0x7fe134062400 2020-12-05T17:38:38.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe13b26b700 1 -- 172.21.15.32:0/2653061199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe134145800 con 0x7fe134062400 2020-12-05T17:38:38.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.919+0000 7fe13b26b700 1 -- 172.21.15.32:0/2653061199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe134057050 con 0x7fe134062400 2020-12-05T17:38:38.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.923+0000 7fe12b7fe700 1 -- 172.21.15.32:0/2653061199 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe12c01c460 con 0x7fe134062400 2020-12-05T17:38:38.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.923+0000 7fe12b7fe700 1 --2- 172.21.15.32:0/2653061199 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe114055330 0x7fe1140577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:38.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.923+0000 7fe12b7fe700 1 -- 172.21.15.32:0/2653061199 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6463+0+0 (secure 0 0 0) 0x7fe12c06e480 con 0x7fe134062400 2020-12-05T17:38:38.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.923+0000 7fe13a269700 1 --2- 172.21.15.32:0/2653061199 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe114055330 0x7fe1140577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:38.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.923+0000 7fe13a269700 1 --2- 172.21.15.32:0/2653061199 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe114055330 0x7fe1140577f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7fe12401c000 tx=0x7fe124015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:38.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:38.927+0000 7fe12b7fe700 1 -- 172.21.15.32:0/2653061199 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe12c0445a0 con 0x7fe134062400 2020-12-05T17:38:39.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.071+0000 7fe13b26b700 1 -- 172.21.15.32:0/2653061199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe13405aab0 con 0x7fe134062400 2020-12-05T17:38:39.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.071+0000 7fe12b7fe700 1 -- 172.21.15.32:0/2653061199 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16334 (secure 0 0 0) 0x7fe12c047bc0 con 0x7fe134062400 2020-12-05T17:38:39.078 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:38:39.079 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":73,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:38:32.844799+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:37:45.088140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-12-05T17:37:45.501261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.36:0/892894983":"2020-12-05T18:38:32.172754+0000"},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:38:39.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.075+0000 7fe13b26b700 1 -- 172.21.15.32:0/2653061199 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe114055330 msgr2=0x7fe1140577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:39.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.075+0000 7fe13b26b700 1 --2- 172.21.15.32:0/2653061199 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe114055330 0x7fe1140577f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7fe12401c000 tx=0x7fe124015000).stop 2020-12-05T17:38:39.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.075+0000 7fe13b26b700 1 -- 172.21.15.32:0/2653061199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe134062400 msgr2=0x7fe1340b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:39.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.075+0000 7fe13b26b700 1 --2- 172.21.15.32:0/2653061199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe134062400 0x7fe1340b2270 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7fe12c00c000 tx=0x7fe12c006dc0).stop 2020-12-05T17:38:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.075+0000 7fe13b26b700 1 -- 172.21.15.32:0/2653061199 shutdown_connections 2020-12-05T17:38:39.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.075+0000 7fe13b26b700 1 --2- 172.21.15.32:0/2653061199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe134062400 0x7fe1340b2270 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.075+0000 7fe13b26b700 1 --2- 172.21.15.32:0/2653061199 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe114055330 0x7fe1140577f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.075+0000 7fe13b26b700 1 --2- 172.21.15.32:0/2653061199 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe13405f3b0 0x7fe134064cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.075+0000 7fe13b26b700 1 --2- 172.21.15.32:0/2653061199 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe13405feb0 0x7fe134154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.075+0000 7fe13b26b700 1 -- 172.21.15.32:0/2653061199 >> 172.21.15.32:0/2653061199 conn(0x7fe1340b8ee0 msgr2=0x7fe134142450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:39.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.075+0000 7fe13b26b700 1 -- 172.21.15.32:0/2653061199 shutdown_connections 2020-12-05T17:38:39.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.075+0000 7fe13b26b700 1 -- 172.21.15.32:0/2653061199 wait complete. 2020-12-05T17:38:39.092 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:37:45.088140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:37:45.501261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:38:39.093 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:38:39.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.203+0000 7ff3ed4bb700 1 Processor -- start 2020-12-05T17:38:39.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.203+0000 7ff3ed4bb700 1 -- start start 2020-12-05T17:38:39.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.203+0000 7ff3ed4bb700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff3e80b5730 0x7ff3e8146fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:39.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.203+0000 7ff3ed4bb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e8061910 con 0x7ff3e8062b20 2020-12-05T17:38:39.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.203+0000 7ff3ed4bb700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e8061f00 con 0x7ff3e80b0060 2020-12-05T17:38:39.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.203+0000 7ff3ed4bb700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3e80614f0 con 0x7ff3e80b5730 2020-12-05T17:38:39.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.203+0000 7ff3e77fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff3e8062b20 0x7ff3e8145d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39624/0 (socket says 172.21.15.32:39624) 2020-12-05T17:38:39.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.203+0000 7ff3e77fe700 1 -- 172.21.15.32:0/434037411 learned_addr learned my addr 172.21.15.32:0/434037411 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:38:39.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.203+0000 7ff3e6ffd700 1 --2- 172.21.15.32:0/434037411 >> v2:172.21.15.61:3300/0 conn(0x7ff3e80b5730 0x7ff3e8146fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:39.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.203+0000 7ff3e5ffb700 1 -- 172.21.15.32:0/434037411 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3782444175 0 0) 0x7ff3e8061910 con 0x7ff3e8062b20 2020-12-05T17:38:39.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.203+0000 7ff3e5ffb700 1 -- 172.21.15.32:0/434037411 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3d0003230 con 0x7ff3e8062b20 2020-12-05T17:38:39.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.203+0000 7ff3e6ffd700 1 -- 172.21.15.32:0/434037411 >> v1:172.21.15.61:6789/0 conn(0x7ff3e80b0060 legacy=0x7ff3e8147490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:39.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.203+0000 7ff3e6ffd700 1 -- 172.21.15.32:0/434037411 >> v1:172.21.15.32:6789/0 conn(0x7ff3e8062b20 legacy=0x7ff3e8145d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:39.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.203+0000 7ff3e6ffd700 1 -- 172.21.15.32:0/434037411 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e80b5b10 con 0x7ff3e80b5730 2020-12-05T17:38:39.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.203+0000 7ff3e6ffd700 1 --2- 172.21.15.32:0/434037411 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3e80b5730 0x7ff3e8146fa0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7ff3d4002f30 tx=0x7ff3d4006dc0).ready entity=mon.1 client_cookie=412a0db8db2eaa45 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:39.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.203+0000 7ff3e5ffb700 1 -- 172.21.15.32:0/434037411 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3d4009800 con 0x7ff3e80b5730 2020-12-05T17:38:39.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.207+0000 7ff3e5ffb700 1 -- 172.21.15.32:0/434037411 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d4009960 con 0x7ff3e80b5730 2020-12-05T17:38:39.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.207+0000 7ff3ed4bb700 1 -- 172.21.15.32:0/434037411 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3e80b5730 msgr2=0x7ff3e8146fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:39.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.207+0000 7ff3e5ffb700 1 -- 172.21.15.32:0/434037411 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3d4009cc0 con 0x7ff3e80b5730 2020-12-05T17:38:39.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.207+0000 7ff3ed4bb700 1 --2- 172.21.15.32:0/434037411 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3e80b5730 0x7ff3e8146fa0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7ff3d4002f30 tx=0x7ff3d4006dc0).stop 2020-12-05T17:38:39.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.207+0000 7ff3ed4bb700 1 -- 172.21.15.32:0/434037411 shutdown_connections 2020-12-05T17:38:39.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.207+0000 7ff3ed4bb700 1 --2- 172.21.15.32:0/434037411 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3e80b5730 0x7ff3e8146fa0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.207+0000 7ff3ed4bb700 1 -- 172.21.15.32:0/434037411 >> 172.21.15.32:0/434037411 conn(0x7ff3e805fe80 msgr2=0x7ff3e80b4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:39.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.207+0000 7ff3ed4bb700 1 -- 172.21.15.32:0/434037411 shutdown_connections 2020-12-05T17:38:39.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.207+0000 7ff3ed4bb700 1 -- 172.21.15.32:0/434037411 wait complete. 2020-12-05T17:38:39.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.207+0000 7ff3ed4bb700 1 Processor -- start 2020-12-05T17:38:39.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.207+0000 7ff3ed4bb700 1 -- start start 2020-12-05T17:38:39.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3ed4bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3e8062b20 0x7ff3e80ab130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:39.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3ed4bb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3e80b0060 0x7ff3e809e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:39.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3ed4bb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff3e80b5730 0x7ff3e80a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:39.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3ed4bb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80ae670 con 0x7ff3e80b0060 2020-12-05T17:38:39.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3ed4bb700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e815a050 con 0x7ff3e8062b20 2020-12-05T17:38:39.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3ed4bb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e8066a50 con 0x7ff3e80b5730 2020-12-05T17:38:39.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3e67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3e80b0060 0x7ff3e809e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:39.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3e6ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3e8062b20 0x7ff3e80ab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:39.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3e67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3e80b0060 0x7ff3e809e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36980/0 (socket says 172.21.15.32:36980) 2020-12-05T17:38:39.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3e6ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3e8062b20 0x7ff3e80ab130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:56870/0 (socket says 172.21.15.32:56870) 2020-12-05T17:38:39.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3e67fc700 1 -- 172.21.15.32:0/584421398 learned_addr learned my addr 172.21.15.32:0/584421398 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:39.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3e77fe700 1 --2- 172.21.15.32:0/584421398 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff3e80b5730 0x7ff3e80a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:39.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3e67fc700 1 -- 172.21.15.32:0/584421398 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff3e80b5730 msgr2=0x7ff3e80a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:39.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3e67fc700 1 --2- 172.21.15.32:0/584421398 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff3e80b5730 0x7ff3e80a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3e67fc700 1 -- 172.21.15.32:0/584421398 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3e8062b20 msgr2=0x7ff3e80ab130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:39.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3e67fc700 1 --2- 172.21.15.32:0/584421398 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3e8062b20 0x7ff3e80ab130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3e67fc700 1 -- 172.21.15.32:0/584421398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e8159d70 con 0x7ff3e80b0060 2020-12-05T17:38:39.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3e67fc700 1 --2- 172.21.15.32:0/584421398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3e80b0060 0x7ff3e809e120 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7ff3e806a370 tx=0x7ff3d8007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:39.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3dffff700 1 -- 172.21.15.32:0/584421398 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3d8013410 con 0x7ff3e80b0060 2020-12-05T17:38:39.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3ed4bb700 1 -- 172.21.15.32:0/584421398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e8063710 con 0x7ff3e80b0060 2020-12-05T17:38:39.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3dffff700 1 -- 172.21.15.32:0/584421398 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d8013570 con 0x7ff3e80b0060 2020-12-05T17:38:39.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3ed4bb700 1 -- 172.21.15.32:0/584421398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e80a21b0 con 0x7ff3e80b0060 2020-12-05T17:38:39.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3dffff700 1 -- 172.21.15.32:0/584421398 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3d80138d0 con 0x7ff3e80b0060 2020-12-05T17:38:39.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3ed4bb700 1 -- 172.21.15.32:0/584421398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e8063380 con 0x7ff3e80b0060 2020-12-05T17:38:39.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.211+0000 7ff3dffff700 1 -- 172.21.15.32:0/584421398 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff3d8010030 con 0x7ff3e80b0060 2020-12-05T17:38:39.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.215+0000 7ff3dffff700 1 --2- 172.21.15.32:0/584421398 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff3c40554e0 0x7ff3c40579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:39.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.215+0000 7ff3dffff700 1 -- 172.21.15.32:0/584421398 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6463+0+0 (secure 0 0 0) 0x7ff3d8001e40 con 0x7ff3e80b0060 2020-12-05T17:38:39.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.215+0000 7ff3e6ffd700 1 --2- 172.21.15.32:0/584421398 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff3c40554e0 0x7ff3c40579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:39.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.215+0000 7ff3e6ffd700 1 --2- 172.21.15.32:0/584421398 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff3c40554e0 0x7ff3c40579a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7ff3d401a000 tx=0x7ff3d4015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:39.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.219+0000 7ff3dffff700 1 -- 172.21.15.32:0/584421398 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff3e8063380 con 0x7ff3e80b0060 2020-12-05T17:38:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.359+0000 7ff3ed4bb700 1 -- 172.21.15.32:0/584421398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff3e814b2d0 con 0x7ff3e80b0060 2020-12-05T17:38:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.363+0000 7ff3dffff700 1 -- 172.21.15.32:0/584421398 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v73) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff3e814b2d0 con 0x7ff3e80b0060 2020-12-05T17:38:39.368 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:38:39.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.363+0000 7ff3ed4bb700 1 -- 172.21.15.32:0/584421398 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff3c40554e0 msgr2=0x7ff3c40579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.363+0000 7ff3ed4bb700 1 --2- 172.21.15.32:0/584421398 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff3c40554e0 0x7ff3c40579a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7ff3d401a000 tx=0x7ff3d4015000).stop 2020-12-05T17:38:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.363+0000 7ff3ed4bb700 1 -- 172.21.15.32:0/584421398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3e80b0060 msgr2=0x7ff3e809e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.363+0000 7ff3ed4bb700 1 --2- 172.21.15.32:0/584421398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3e80b0060 0x7ff3e809e120 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7ff3e806a370 tx=0x7ff3d8007bc0).stop 2020-12-05T17:38:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.363+0000 7ff3ed4bb700 1 -- 172.21.15.32:0/584421398 shutdown_connections 2020-12-05T17:38:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.363+0000 7ff3ed4bb700 1 --2- 172.21.15.32:0/584421398 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff3c40554e0 0x7ff3c40579a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.363+0000 7ff3ed4bb700 1 --2- 172.21.15.32:0/584421398 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff3e80b5730 0x7ff3e80a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.363+0000 7ff3ed4bb700 1 --2- 172.21.15.32:0/584421398 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3e8062b20 0x7ff3e80ab130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.363+0000 7ff3ed4bb700 1 --2- 172.21.15.32:0/584421398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3e80b0060 0x7ff3e809e120 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.363+0000 7ff3ed4bb700 1 -- 172.21.15.32:0/584421398 >> 172.21.15.32:0/584421398 conn(0x7ff3e805fe80 msgr2=0x7ff3e80af680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.363+0000 7ff3ed4bb700 1 -- 172.21.15.32:0/584421398 shutdown_connections 2020-12-05T17:38:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.363+0000 7ff3ed4bb700 1 -- 172.21.15.32:0/584421398 wait complete. 2020-12-05T17:38:39.381 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:38:39.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.495+0000 7f2a36290700 1 Processor -- start 2020-12-05T17:38:39.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.495+0000 7f2a36290700 1 -- start start 2020-12-05T17:38:39.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a36290700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a3005feb0 0x7f2a3005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:39.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a36290700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f2a300b2fb0 0x7f2a30061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:39.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a36290700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a300b2df0 con 0x7f2a3005f3b0 2020-12-05T17:38:39.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a36290700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a300b3c90 con 0x7f2a300b2fb0 2020-12-05T17:38:39.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a36290700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a300b2b50 con 0x7f2a3005feb0 2020-12-05T17:38:39.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a35a8f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f2a300b2fb0 0x7f2a30061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:39.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a34a8d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a3005feb0 0x7f2a3005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:39.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a3528e700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f2a3005f3b0 0x7f2a3013c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:52274/0 (socket says 172.21.15.32:52274) 2020-12-05T17:38:39.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a3528e700 1 -- 172.21.15.32:0/1578170774 learned_addr learned my addr 172.21.15.32:0/1578170774 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:38:39.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a34a8d700 1 -- 172.21.15.32:0/1578170774 >> v2:172.21.15.61:3300/0 conn(0x7f2a300b2fb0 msgr2=0x7f2a30061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:39.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a34a8d700 1 --2- 172.21.15.32:0/1578170774 >> v2:172.21.15.61:3300/0 conn(0x7f2a300b2fb0 0x7f2a30061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a34a8d700 1 -- 172.21.15.32:0/1578170774 >> v1:172.21.15.61:6789/0 conn(0x7f2a3005f3b0 legacy=0x7f2a3013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:39.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a34a8d700 1 -- 172.21.15.32:0/1578170774 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a30142820 con 0x7f2a3005feb0 2020-12-05T17:38:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a34a8d700 1 --2- 172.21.15.32:0/1578170774 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a3005feb0 0x7f2a3005f790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f2a20002f30 tx=0x7f2a20006dc0).ready entity=mon.2 client_cookie=efc0065c10c6b2a7 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a27fff700 1 -- 172.21.15.32:0/1578170774 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a20009760 con 0x7f2a3005feb0 2020-12-05T17:38:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a27fff700 1 -- 172.21.15.32:0/1578170774 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a200098c0 con 0x7f2a3005feb0 2020-12-05T17:38:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a27fff700 1 -- 172.21.15.32:0/1578170774 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a20009c20 con 0x7f2a3005feb0 2020-12-05T17:38:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a36290700 1 -- 172.21.15.32:0/1578170774 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a3005feb0 msgr2=0x7f2a3005f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a36290700 1 --2- 172.21.15.32:0/1578170774 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a3005feb0 0x7f2a3005f790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f2a20002f30 tx=0x7f2a20006dc0).stop 2020-12-05T17:38:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a36290700 1 -- 172.21.15.32:0/1578170774 shutdown_connections 2020-12-05T17:38:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a36290700 1 --2- 172.21.15.32:0/1578170774 >> v2:172.21.15.61:3300/0 conn(0x7f2a300b2fb0 0x7f2a30061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a36290700 1 --2- 172.21.15.32:0/1578170774 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a3005feb0 0x7f2a3005f790 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a36290700 1 -- 172.21.15.32:0/1578170774 >> 172.21.15.32:0/1578170774 conn(0x7f2a300b8f20 msgr2=0x7f2a300b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:39.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a36290700 1 -- 172.21.15.32:0/1578170774 shutdown_connections 2020-12-05T17:38:39.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.499+0000 7f2a36290700 1 -- 172.21.15.32:0/1578170774 wait complete. 2020-12-05T17:38:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a36290700 1 Processor -- start 2020-12-05T17:38:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a36290700 1 -- start start 2020-12-05T17:38:39.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a36290700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a3005f3b0 0x7f2a301513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:39.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a36290700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a3005feb0 0x7f2a30065270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:39.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a36290700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2a300b2fb0 0x7f2a30145fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:39.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a36290700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a30066610 con 0x7f2a3005f3b0 2020-12-05T17:38:39.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a36290700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a300668b0 con 0x7f2a300b2fb0 2020-12-05T17:38:39.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a36290700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a30064cf0 con 0x7f2a3005feb0 2020-12-05T17:38:39.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a3528e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a3005f3b0 0x7f2a301513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:39.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a3528e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a3005f3b0 0x7f2a301513f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36992/0 (socket says 172.21.15.32:36992) 2020-12-05T17:38:39.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a3528e700 1 -- 172.21.15.32:0/3086140799 learned_addr learned my addr 172.21.15.32:0/3086140799 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:39.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a34a8d700 1 --2- 172.21.15.32:0/3086140799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a3005feb0 0x7f2a30065270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:39.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a35a8f700 1 --2- 172.21.15.32:0/3086140799 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2a300b2fb0 0x7f2a30145fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:39.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a3528e700 1 -- 172.21.15.32:0/3086140799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a3005feb0 msgr2=0x7f2a30065270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:39.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a3528e700 1 --2- 172.21.15.32:0/3086140799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a3005feb0 0x7f2a30065270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a3528e700 1 -- 172.21.15.32:0/3086140799 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2a300b2fb0 msgr2=0x7f2a30145fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:39.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a3528e700 1 --2- 172.21.15.32:0/3086140799 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2a300b2fb0 0x7f2a30145fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a3528e700 1 -- 172.21.15.32:0/3086140799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3013d240 con 0x7f2a3005f3b0 2020-12-05T17:38:39.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a3528e700 1 --2- 172.21.15.32:0/3086140799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a3005f3b0 0x7f2a301513f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f2a30061550 tx=0x7f2a2c007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:39.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a267fc700 1 -- 172.21.15.32:0/3086140799 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a2c0072e0 con 0x7f2a3005f3b0 2020-12-05T17:38:39.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a36290700 1 -- 172.21.15.32:0/3086140799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3013d4f0 con 0x7f2a3005f3b0 2020-12-05T17:38:39.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a267fc700 1 -- 172.21.15.32:0/3086140799 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2c007440 con 0x7f2a3005f3b0 2020-12-05T17:38:39.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a267fc700 1 -- 172.21.15.32:0/3086140799 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a2c0057d0 con 0x7f2a3005f3b0 2020-12-05T17:38:39.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.503+0000 7f2a36290700 1 -- 172.21.15.32:0/3086140799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a30063960 con 0x7f2a3005f3b0 2020-12-05T17:38:39.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.507+0000 7f2a36290700 1 -- 172.21.15.32:0/3086140799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a30066a70 con 0x7f2a3005f3b0 2020-12-05T17:38:39.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.507+0000 7f2a267fc700 1 -- 172.21.15.32:0/3086140799 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2a2c01a070 con 0x7f2a3005f3b0 2020-12-05T17:38:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.507+0000 7f2a267fc700 1 --2- 172.21.15.32:0/3086140799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2a18055350 0x7f2a18057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.507+0000 7f2a267fc700 1 -- 172.21.15.32:0/3086140799 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f2a2c06d2d0 con 0x7f2a3005f3b0 2020-12-05T17:38:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.507+0000 7f2a34a8d700 1 --2- 172.21.15.32:0/3086140799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2a18055350 0x7f2a18057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.507+0000 7f2a34a8d700 1 --2- 172.21.15.32:0/3086140799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2a18055350 0x7f2a18057810 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f2a2001a000 tx=0x7f2a20015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:39.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.515+0000 7f2a267fc700 1 -- 172.21.15.32:0/3086140799 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2a2c0433b0 con 0x7f2a3005f3b0 2020-12-05T17:38:39.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.655+0000 7f2a36290700 1 -- 172.21.15.32:0/3086140799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2a30057050 con 0x7f2a3005f3b0 2020-12-05T17:38:39.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.655+0000 7f2a267fc700 1 -- 172.21.15.32:0/3086140799 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v73) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2a2c0469d0 con 0x7f2a3005f3b0 2020-12-05T17:38:39.662 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:38:39.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.659+0000 7f2a36290700 1 -- 172.21.15.32:0/3086140799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2a18055350 msgr2=0x7f2a18057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:39.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.659+0000 7f2a36290700 1 --2- 172.21.15.32:0/3086140799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2a18055350 0x7f2a18057810 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f2a2001a000 tx=0x7f2a20015000).stop 2020-12-05T17:38:39.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.659+0000 7f2a36290700 1 -- 172.21.15.32:0/3086140799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a3005f3b0 msgr2=0x7f2a301513f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:39.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.659+0000 7f2a36290700 1 --2- 172.21.15.32:0/3086140799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a3005f3b0 0x7f2a301513f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f2a30061550 tx=0x7f2a2c007c10).stop 2020-12-05T17:38:39.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.659+0000 7f2a36290700 1 -- 172.21.15.32:0/3086140799 shutdown_connections 2020-12-05T17:38:39.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.659+0000 7f2a36290700 1 --2- 172.21.15.32:0/3086140799 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2a300b2fb0 0x7f2a30145fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.659+0000 7f2a36290700 1 --2- 172.21.15.32:0/3086140799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a3005f3b0 0x7f2a301513f0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.659+0000 7f2a36290700 1 --2- 172.21.15.32:0/3086140799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2a18055350 0x7f2a18057810 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.659+0000 7f2a36290700 1 --2- 172.21.15.32:0/3086140799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a3005feb0 0x7f2a30065270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.659+0000 7f2a36290700 1 -- 172.21.15.32:0/3086140799 >> 172.21.15.32:0/3086140799 conn(0x7f2a300b8f20 msgr2=0x7f2a300b6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:39.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.659+0000 7f2a36290700 1 -- 172.21.15.32:0/3086140799 shutdown_connections 2020-12-05T17:38:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.659+0000 7f2a36290700 1 -- 172.21.15.32:0/3086140799 wait complete. 2020-12-05T17:38:39.678 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:38:39.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d2bf0700 1 Processor -- start 2020-12-05T17:38:39.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d2bf0700 1 -- start start 2020-12-05T17:38:39.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d2bf0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f55cc0b2000 0x7f55cc145e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:39.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d2bf0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55cc061780 0x7f55cc0afe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:39.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d2bf0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55cc0b23e0 con 0x7f55cc0af4b0 2020-12-05T17:38:39.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d2bf0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55cc146a00 con 0x7f55cc0b2000 2020-12-05T17:38:39.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d2bf0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc0ae7b0 con 0x7f55cc061780 2020-12-05T17:38:39.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d1bee700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f55cc0b2000 0x7f55cc145e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:39.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d1bee700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f55cc0b2000 0x7f55cc145e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37000/0 (socket says 172.21.15.32:37000) 2020-12-05T17:38:39.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d1bee700 1 -- 172.21.15.32:0/1989408400 learned_addr learned my addr 172.21.15.32:0/1989408400 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:39.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d13ed700 1 --1- 172.21.15.32:0/1989408400 >> v1:172.21.15.32:6789/0 conn(0x7f55cc0af4b0 0x7f55cc1462f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39650/0 (socket says 172.21.15.32:39650) 2020-12-05T17:38:39.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d23ef700 1 --2- 172.21.15.32:0/1989408400 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55cc061780 0x7f55cc0afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:39.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d1bee700 1 -- 172.21.15.32:0/1989408400 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55cc061780 msgr2=0x7f55cc0afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:39.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d1bee700 1 --2- 172.21.15.32:0/1989408400 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55cc061780 0x7f55cc0afe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d1bee700 1 -- 172.21.15.32:0/1989408400 >> v1:172.21.15.32:6789/0 conn(0x7f55cc0af4b0 legacy=0x7f55cc1462f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:39.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d1bee700 1 -- 172.21.15.32:0/1989408400 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc0aea50 con 0x7f55cc0b2000 2020-12-05T17:38:39.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d1bee700 1 --2- 172.21.15.32:0/1989408400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55cc0b2000 0x7f55cc145e00 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f55c0002f30 tx=0x7f55c0006e80).ready entity=mon.0 client_cookie=a3e02c0015536d49 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:39.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d0bec700 1 -- 172.21.15.32:0/1989408400 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55c00097f0 con 0x7f55cc0b2000 2020-12-05T17:38:39.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d0bec700 1 -- 172.21.15.32:0/1989408400 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c0009950 con 0x7f55cc0b2000 2020-12-05T17:38:39.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d0bec700 1 -- 172.21.15.32:0/1989408400 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55c0009ac0 con 0x7f55cc0b2000 2020-12-05T17:38:39.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d2bf0700 1 -- 172.21.15.32:0/1989408400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55cc0b2000 msgr2=0x7f55cc145e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:39.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d2bf0700 1 --2- 172.21.15.32:0/1989408400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55cc0b2000 0x7f55cc145e00 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f55c0002f30 tx=0x7f55c0006e80).stop 2020-12-05T17:38:39.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d2bf0700 1 -- 172.21.15.32:0/1989408400 shutdown_connections 2020-12-05T17:38:39.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d2bf0700 1 --2- 172.21.15.32:0/1989408400 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55cc061780 0x7f55cc0afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d2bf0700 1 --2- 172.21.15.32:0/1989408400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55cc0b2000 0x7f55cc145e00 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d2bf0700 1 -- 172.21.15.32:0/1989408400 >> 172.21.15.32:0/1989408400 conn(0x7f55cc0b5620 msgr2=0x7f55cc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:39.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d2bf0700 1 -- 172.21.15.32:0/1989408400 shutdown_connections 2020-12-05T17:38:39.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.787+0000 7f55d2bf0700 1 -- 172.21.15.32:0/1989408400 wait complete. 2020-12-05T17:38:39.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55d2bf0700 1 Processor -- start 2020-12-05T17:38:39.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55d2bf0700 1 -- start start 2020-12-05T17:38:39.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55d2bf0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f55cc061780 0x7f55cc1448f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:39.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55d2bf0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55cc0af4b0 0x7f55cc0651b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:39.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55d2bf0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55cc0b2000 0x7f55cc0634e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:39.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55d2bf0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc1499b0 con 0x7f55cc0b2000 2020-12-05T17:38:39.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55d2bf0700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc14aa90 con 0x7f55cc061780 2020-12-05T17:38:39.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55d2bf0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc0656a0 con 0x7f55cc0af4b0 2020-12-05T17:38:39.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55d23ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55cc0b2000 0x7f55cc0634e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:39.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55d23ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55cc0b2000 0x7f55cc0634e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37008/0 (socket says 172.21.15.32:37008) 2020-12-05T17:38:39.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55d23ef700 1 -- 172.21.15.32:0/3324937981 learned_addr learned my addr 172.21.15.32:0/3324937981 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:39.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55d13ed700 1 --2- 172.21.15.32:0/3324937981 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55cc0af4b0 0x7f55cc0651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:39.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55d1bee700 1 --2- 172.21.15.32:0/3324937981 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f55cc061780 0x7f55cc1448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:39.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55d23ef700 1 -- 172.21.15.32:0/3324937981 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55cc0af4b0 msgr2=0x7f55cc0651b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:39.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55d23ef700 1 --2- 172.21.15.32:0/3324937981 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55cc0af4b0 0x7f55cc0651b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55d23ef700 1 -- 172.21.15.32:0/3324937981 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f55cc061780 msgr2=0x7f55cc1448f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:39.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55d23ef700 1 --2- 172.21.15.32:0/3324937981 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f55cc061780 0x7f55cc1448f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55d23ef700 1 -- 172.21.15.32:0/3324937981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc149bf0 con 0x7f55cc0b2000 2020-12-05T17:38:39.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55d23ef700 1 --2- 172.21.15.32:0/3324937981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55cc0b2000 0x7f55cc0634e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f55c400a000 tx=0x7f55c4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:39.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55bf7fe700 1 -- 172.21.15.32:0/3324937981 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55c400b9e0 con 0x7f55cc0b2000 2020-12-05T17:38:39.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55bf7fe700 1 -- 172.21.15.32:0/3324937981 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c400bb40 con 0x7f55cc0b2000 2020-12-05T17:38:39.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55d2bf0700 1 -- 172.21.15.32:0/3324937981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55cc145350 con 0x7f55cc0b2000 2020-12-05T17:38:39.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55bf7fe700 1 -- 172.21.15.32:0/3324937981 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55c400bea0 con 0x7f55cc0b2000 2020-12-05T17:38:39.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.791+0000 7f55d2bf0700 1 -- 172.21.15.32:0/3324937981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55cc066d50 con 0x7f55cc0b2000 2020-12-05T17:38:39.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.795+0000 7f55d2bf0700 1 -- 172.21.15.32:0/3324937981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55cc063d10 con 0x7f55cc0b2000 2020-12-05T17:38:39.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.795+0000 7f55bf7fe700 1 -- 172.21.15.32:0/3324937981 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f55c401c460 con 0x7f55cc0b2000 2020-12-05T17:38:39.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.795+0000 7f55bf7fe700 1 --2- 172.21.15.32:0/3324937981 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f55a80554e0 0x7f55a80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:39.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.795+0000 7f55bf7fe700 1 -- 172.21.15.32:0/3324937981 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f55c406e6e0 con 0x7f55cc0b2000 2020-12-05T17:38:39.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.795+0000 7f55d1bee700 1 --2- 172.21.15.32:0/3324937981 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f55a80554e0 0x7f55a80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:39.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.795+0000 7f55d1bee700 1 --2- 172.21.15.32:0/3324937981 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f55a80554e0 0x7f55a80579a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f55c001c000 tx=0x7f55c0016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:39.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.799+0000 7f55bf7fe700 1 -- 172.21.15.32:0/3324937981 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f55c4044800 con 0x7f55cc0b2000 2020-12-05T17:38:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.943+0000 7f55d2bf0700 1 -- 172.21.15.32:0/3324937981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f55cc057050 con 0x7f55cc0b2000 2020-12-05T17:38:39.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.947+0000 7f55bf7fe700 1 -- 172.21.15.32:0/3324937981 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v73) v1 ==== 104+0+10 (secure 0 0 0) 0x7f55c4047e20 con 0x7f55cc0b2000 2020-12-05T17:38:39.953 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:38:39.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.947+0000 7f55d2bf0700 1 -- 172.21.15.32:0/3324937981 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f55a80554e0 msgr2=0x7f55a80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.947+0000 7f55d2bf0700 1 --2- 172.21.15.32:0/3324937981 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f55a80554e0 0x7f55a80579a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f55c001c000 tx=0x7f55c0016000).stop 2020-12-05T17:38:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.947+0000 7f55d2bf0700 1 -- 172.21.15.32:0/3324937981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55cc0b2000 msgr2=0x7f55cc0634e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.947+0000 7f55d2bf0700 1 --2- 172.21.15.32:0/3324937981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55cc0b2000 0x7f55cc0634e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f55c400a000 tx=0x7f55c4008040).stop 2020-12-05T17:38:39.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.947+0000 7f55d2bf0700 1 -- 172.21.15.32:0/3324937981 shutdown_connections 2020-12-05T17:38:39.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.947+0000 7f55d2bf0700 1 --2- 172.21.15.32:0/3324937981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55cc0b2000 0x7f55cc0634e0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.947+0000 7f55d2bf0700 1 --2- 172.21.15.32:0/3324937981 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f55cc061780 0x7f55cc1448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.947+0000 7f55d2bf0700 1 --2- 172.21.15.32:0/3324937981 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f55a80554e0 0x7f55a80579a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.947+0000 7f55d2bf0700 1 --2- 172.21.15.32:0/3324937981 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55cc0af4b0 0x7f55cc0651b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:39.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.947+0000 7f55d2bf0700 1 -- 172.21.15.32:0/3324937981 >> 172.21.15.32:0/3324937981 conn(0x7f55cc0b5620 msgr2=0x7f55cc143b10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:39.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.947+0000 7f55d2bf0700 1 -- 172.21.15.32:0/3324937981 shutdown_connections 2020-12-05T17:38:39.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:39.947+0000 7f55d2bf0700 1 -- 172.21.15.32:0/3324937981 wait complete. 2020-12-05T17:38:39.973 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:38:40.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.091+0000 7f2a4f59e700 1 Processor -- start 2020-12-05T17:38:40.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.091+0000 7f2a4f59e700 1 -- start start 2020-12-05T17:38:40.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a50061920 0x7f2a50064a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:40.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4f59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a50145e80 con 0x7f2a500b16b0 2020-12-05T17:38:40.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4f59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a50146120 con 0x7f2a50142fc0 2020-12-05T17:38:40.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a501463c0 con 0x7f2a50061920 2020-12-05T17:38:40.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4dd9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2a500b16b0 0x7f2a501428b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39664/0 (socket says 172.21.15.32:39664) 2020-12-05T17:38:40.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4dd9b700 1 -- 172.21.15.32:0/1257009925 learned_addr learned my addr 172.21.15.32:0/1257009925 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:38:40.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4e59c700 1 --2- 172.21.15.32:0/1257009925 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a50061920 0x7f2a50064a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:40.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4d59a700 1 -- 172.21.15.32:0/1257009925 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7234019 0 0) 0x7f2a50145e80 con 0x7f2a500b16b0 2020-12-05T17:38:40.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4d59a700 1 -- 172.21.15.32:0/1257009925 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a34003230 con 0x7f2a500b16b0 2020-12-05T17:38:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4d59a700 1 -- 172.21.15.32:0/1257009925 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 300969039 0 0) 0x7f2a50146120 con 0x7f2a50142fc0 2020-12-05T17:38:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4d59a700 1 -- 172.21.15.32:0/1257009925 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a50145e80 con 0x7f2a50142fc0 2020-12-05T17:38:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4e59c700 1 -- 172.21.15.32:0/1257009925 >> v1:172.21.15.61:6789/0 conn(0x7f2a50142fc0 legacy=0x7f2a5013c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4e59c700 1 -- 172.21.15.32:0/1257009925 >> v1:172.21.15.32:6789/0 conn(0x7f2a500b16b0 legacy=0x7f2a501428b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4e59c700 1 -- 172.21.15.32:0/1257009925 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a500626b0 con 0x7f2a50061920 2020-12-05T17:38:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4e59c700 1 --2- 172.21.15.32:0/1257009925 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a50061920 0x7f2a50064a10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f2a40002f30 tx=0x7f2a40006dc0).ready entity=mon.2 client_cookie=7d719b9b6579565e server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4d59a700 1 -- 172.21.15.32:0/1257009925 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a40009730 con 0x7f2a50061920 2020-12-05T17:38:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4d59a700 1 -- 172.21.15.32:0/1257009925 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a40009890 con 0x7f2a50061920 2020-12-05T17:38:40.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4d59a700 1 -- 172.21.15.32:0/1257009925 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a40009bf0 con 0x7f2a50061920 2020-12-05T17:38:40.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4f59e700 1 -- 172.21.15.32:0/1257009925 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a50061920 msgr2=0x7f2a50064a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:40.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4f59e700 1 --2- 172.21.15.32:0/1257009925 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a50061920 0x7f2a50064a10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f2a40002f30 tx=0x7f2a40006dc0).stop 2020-12-05T17:38:40.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4f59e700 1 -- 172.21.15.32:0/1257009925 shutdown_connections 2020-12-05T17:38:40.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4f59e700 1 --2- 172.21.15.32:0/1257009925 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a50061920 0x7f2a50064a10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:40.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4f59e700 1 -- 172.21.15.32:0/1257009925 >> 172.21.15.32:0/1257009925 conn(0x7f2a500b6d10 msgr2=0x7f2a500b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:40.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4f59e700 1 -- 172.21.15.32:0/1257009925 shutdown_connections 2020-12-05T17:38:40.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.095+0000 7f2a4f59e700 1 -- 172.21.15.32:0/1257009925 wait complete. 2020-12-05T17:38:40.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.099+0000 7f2a4f59e700 1 Processor -- start 2020-12-05T17:38:40.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.099+0000 7f2a4f59e700 1 -- start start 2020-12-05T17:38:40.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.099+0000 7f2a4f59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2a50061920 0x7f2a500b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:40.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.099+0000 7f2a4f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a500b16b0 0x7f2a50066310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:40.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.103+0000 7f2a4f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a50142fc0 0x7f2a5013fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:40.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.103+0000 7f2a4f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5013b970 con 0x7f2a50142fc0 2020-12-05T17:38:40.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.103+0000 7f2a4f59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a500b40e0 con 0x7f2a50061920 2020-12-05T17:38:40.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.103+0000 7f2a4f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5013de90 con 0x7f2a500b16b0 2020-12-05T17:38:40.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.103+0000 7f2a4ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a50142fc0 0x7f2a5013fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:40.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.103+0000 7f2a4ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a50142fc0 0x7f2a5013fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37024/0 (socket says 172.21.15.32:37024) 2020-12-05T17:38:40.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.103+0000 7f2a4ed9d700 1 -- 172.21.15.32:0/466341734 learned_addr learned my addr 172.21.15.32:0/466341734 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:40.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.103+0000 7f2a4e59c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2a50061920 0x7f2a500b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:40.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.103+0000 7f2a4dd9b700 1 --2- 172.21.15.32:0/466341734 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a500b16b0 0x7f2a50066310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:40.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.103+0000 7f2a4ed9d700 1 -- 172.21.15.32:0/466341734 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a500b16b0 msgr2=0x7f2a50066310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:40.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.103+0000 7f2a4ed9d700 1 --2- 172.21.15.32:0/466341734 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a500b16b0 0x7f2a50066310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:40.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.103+0000 7f2a4ed9d700 1 -- 172.21.15.32:0/466341734 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2a50061920 msgr2=0x7f2a500b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:40.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.103+0000 7f2a4ed9d700 1 --2- 172.21.15.32:0/466341734 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2a50061920 0x7f2a500b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:40.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.103+0000 7f2a4ed9d700 1 -- 172.21.15.32:0/466341734 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5013e130 con 0x7f2a50142fc0 2020-12-05T17:38:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.103+0000 7f2a4ed9d700 1 --2- 172.21.15.32:0/466341734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a50142fc0 0x7f2a5013fc80 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f2a48001450 tx=0x7f2a48007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.103+0000 7f2a3f7fe700 1 -- 172.21.15.32:0/466341734 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a4800ce40 con 0x7f2a50142fc0 2020-12-05T17:38:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.103+0000 7f2a3f7fe700 1 -- 172.21.15.32:0/466341734 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a48002240 con 0x7f2a50142fc0 2020-12-05T17:38:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.103+0000 7f2a4f59e700 1 -- 172.21.15.32:0/466341734 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a50066a70 con 0x7f2a50142fc0 2020-12-05T17:38:40.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.103+0000 7f2a3f7fe700 1 -- 172.21.15.32:0/466341734 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a4800a5a0 con 0x7f2a50142fc0 2020-12-05T17:38:40.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.103+0000 7f2a4f59e700 1 -- 172.21.15.32:0/466341734 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a500b4560 con 0x7f2a50142fc0 2020-12-05T17:38:40.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.103+0000 7f2a4f59e700 1 -- 172.21.15.32:0/466341734 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a5013e2c0 con 0x7f2a50142fc0 2020-12-05T17:38:40.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.103+0000 7f2a3f7fe700 1 -- 172.21.15.32:0/466341734 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2a48019030 con 0x7f2a50142fc0 2020-12-05T17:38:40.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.107+0000 7f2a3f7fe700 1 --2- 172.21.15.32:0/466341734 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2a2c055470 0x7f2a2c057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:40.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.107+0000 7f2a3f7fe700 1 -- 172.21.15.32:0/466341734 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f2a4806d920 con 0x7f2a50142fc0 2020-12-05T17:38:40.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.107+0000 7f2a4e59c700 1 --2- 172.21.15.32:0/466341734 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2a2c055470 0x7f2a2c057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:40.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.107+0000 7f2a4e59c700 1 --2- 172.21.15.32:0/466341734 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2a2c055470 0x7f2a2c057930 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f2a40019000 tx=0x7f2a40015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:40.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.111+0000 7f2a3f7fe700 1 -- 172.21.15.32:0/466341734 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2a48043a00 con 0x7f2a50142fc0 2020-12-05T17:38:40.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.259+0000 7f2a4f59e700 1 -- 172.21.15.32:0/466341734 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2a50057050 con 0x7f2a50142fc0 2020-12-05T17:38:40.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.259+0000 7f2a3f7fe700 1 -- 172.21.15.32:0/466341734 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v73) v1 ==== 100+0+10 (secure 0 0 0) 0x7f2a48047020 con 0x7f2a50142fc0 2020-12-05T17:38:40.267 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:38:40.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.263+0000 7f2a4f59e700 1 -- 172.21.15.32:0/466341734 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2a2c055470 msgr2=0x7f2a2c057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:40.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.263+0000 7f2a4f59e700 1 --2- 172.21.15.32:0/466341734 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2a2c055470 0x7f2a2c057930 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f2a40019000 tx=0x7f2a40015000).stop 2020-12-05T17:38:40.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.263+0000 7f2a4f59e700 1 -- 172.21.15.32:0/466341734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a50142fc0 msgr2=0x7f2a5013fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:40.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.263+0000 7f2a4f59e700 1 --2- 172.21.15.32:0/466341734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a50142fc0 0x7f2a5013fc80 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f2a48001450 tx=0x7f2a48007e00).stop 2020-12-05T17:38:40.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.263+0000 7f2a4f59e700 1 -- 172.21.15.32:0/466341734 shutdown_connections 2020-12-05T17:38:40.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.263+0000 7f2a4f59e700 1 --2- 172.21.15.32:0/466341734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a50142fc0 0x7f2a5013fc80 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:40.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.263+0000 7f2a4f59e700 1 --2- 172.21.15.32:0/466341734 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2a50061920 0x7f2a500b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:40.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.263+0000 7f2a4f59e700 1 --2- 172.21.15.32:0/466341734 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2a2c055470 0x7f2a2c057930 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:40.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.263+0000 7f2a4f59e700 1 --2- 172.21.15.32:0/466341734 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a500b16b0 0x7f2a50066310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:40.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.263+0000 7f2a4f59e700 1 -- 172.21.15.32:0/466341734 >> 172.21.15.32:0/466341734 conn(0x7f2a500b6d10 msgr2=0x7f2a50143d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:40.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.263+0000 7f2a4f59e700 1 -- 172.21.15.32:0/466341734 shutdown_connections 2020-12-05T17:38:40.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.263+0000 7f2a4f59e700 1 -- 172.21.15.32:0/466341734 wait complete. 2020-12-05T17:38:40.285 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:38:40.286 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:38:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.387+0000 7fd4f337d700 1 Processor -- start 2020-12-05T17:38:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.387+0000 7fd4f337d700 1 -- start start 2020-12-05T17:38:40.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.387+0000 7fd4f337d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd4ec0af920 0x7fd4ec063040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:40.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.387+0000 7fd4f337d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4ec05fad0 con 0x7fd4ec0afea0 2020-12-05T17:38:40.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.387+0000 7fd4f337d700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4ec144650 con 0x7fd4ec05f6f0 2020-12-05T17:38:40.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.387+0000 7fd4f337d700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ec0b2cd0 con 0x7fd4ec0af920 2020-12-05T17:38:40.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.387+0000 7fd4f2b7c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd4ec0afea0 0x7fd4ec1409c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39680/0 (socket says 172.21.15.32:39680) 2020-12-05T17:38:40.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.387+0000 7fd4f2b7c700 1 -- 172.21.15.32:0/1300755206 learned_addr learned my addr 172.21.15.32:0/1300755206 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:38:40.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.387+0000 7fd4f1b7a700 1 --2- 172.21.15.32:0/1300755206 >> v2:172.21.15.61:3300/0 conn(0x7fd4ec0af920 0x7fd4ec063040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.391+0000 7fd4f1379700 1 -- 172.21.15.32:0/1300755206 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3601675906 0 0) 0x7fd4ec05fad0 con 0x7fd4ec0afea0 2020-12-05T17:38:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.391+0000 7fd4f1379700 1 -- 172.21.15.32:0/1300755206 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4d4003230 con 0x7fd4ec0afea0 2020-12-05T17:38:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.391+0000 7fd4f1b7a700 1 -- 172.21.15.32:0/1300755206 >> v1:172.21.15.61:6789/0 conn(0x7fd4ec05f6f0 legacy=0x7fd4ec0b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.391+0000 7fd4f1b7a700 1 -- 172.21.15.32:0/1300755206 >> v1:172.21.15.32:6789/0 conn(0x7fd4ec0afea0 legacy=0x7fd4ec1409c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.391+0000 7fd4f1b7a700 1 -- 172.21.15.32:0/1300755206 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec1448f0 con 0x7fd4ec0af920 2020-12-05T17:38:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.391+0000 7fd4f1b7a700 1 --2- 172.21.15.32:0/1300755206 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4ec0af920 0x7fd4ec063040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fd4e0002f30 tx=0x7fd4e0006dc0).ready entity=mon.1 client_cookie=eaeb41914a91ea04 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:40.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.391+0000 7fd4f1379700 1 -- 172.21.15.32:0/1300755206 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4e0009800 con 0x7fd4ec0af920 2020-12-05T17:38:40.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.391+0000 7fd4f1379700 1 -- 172.21.15.32:0/1300755206 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e0009960 con 0x7fd4ec0af920 2020-12-05T17:38:40.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.391+0000 7fd4f1379700 1 -- 172.21.15.32:0/1300755206 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4e0009cc0 con 0x7fd4ec0af920 2020-12-05T17:38:40.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.391+0000 7fd4f337d700 1 -- 172.21.15.32:0/1300755206 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4ec0af920 msgr2=0x7fd4ec063040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:40.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.391+0000 7fd4f337d700 1 --2- 172.21.15.32:0/1300755206 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4ec0af920 0x7fd4ec063040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fd4e0002f30 tx=0x7fd4e0006dc0).stop 2020-12-05T17:38:40.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.391+0000 7fd4f337d700 1 -- 172.21.15.32:0/1300755206 shutdown_connections 2020-12-05T17:38:40.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.391+0000 7fd4f337d700 1 --2- 172.21.15.32:0/1300755206 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4ec0af920 0x7fd4ec063040 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:40.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.391+0000 7fd4f337d700 1 -- 172.21.15.32:0/1300755206 >> 172.21.15.32:0/1300755206 conn(0x7fd4ec0604c0 msgr2=0x7fd4ec05c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:40.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.391+0000 7fd4f337d700 1 -- 172.21.15.32:0/1300755206 shutdown_connections 2020-12-05T17:38:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.391+0000 7fd4f337d700 1 -- 172.21.15.32:0/1300755206 wait complete. 2020-12-05T17:38:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.391+0000 7fd4f337d700 1 Processor -- start 2020-12-05T17:38:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.391+0000 7fd4f337d700 1 -- start start 2020-12-05T17:38:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4f337d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4ec05f6f0 0x7fd4ec0632e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4f337d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4ec0af920 0x7fd4ec154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4f337d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ec0afea0 0x7fd4ec064870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4f337d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec147250 con 0x7fd4ec0afea0 2020-12-05T17:38:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4f337d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec1474f0 con 0x7fd4ec0af920 2020-12-05T17:38:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4f337d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec0643d0 con 0x7fd4ec05f6f0 2020-12-05T17:38:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4f2b7c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ec0afea0 0x7fd4ec064870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4f2b7c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ec0afea0 0x7fd4ec064870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37038/0 (socket says 172.21.15.32:37038) 2020-12-05T17:38:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4f2b7c700 1 -- 172.21.15.32:0/3754811186 learned_addr learned my addr 172.21.15.32:0/3754811186 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4f1b7a700 1 --2- 172.21.15.32:0/3754811186 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4ec0af920 0x7fd4ec154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4f237b700 1 --2- 172.21.15.32:0/3754811186 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4ec05f6f0 0x7fd4ec0632e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4f1b7a700 1 -- 172.21.15.32:0/3754811186 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4ec05f6f0 msgr2=0x7fd4ec0632e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4f1b7a700 1 --2- 172.21.15.32:0/3754811186 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4ec05f6f0 0x7fd4ec0632e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4f1b7a700 1 -- 172.21.15.32:0/3754811186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ec0afea0 msgr2=0x7fd4ec064870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4f1b7a700 1 --2- 172.21.15.32:0/3754811186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ec0afea0 0x7fd4ec064870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4f1b7a700 1 -- 172.21.15.32:0/3754811186 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec154e60 con 0x7fd4ec0af920 2020-12-05T17:38:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4f2b7c700 1 --2- 172.21.15.32:0/3754811186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ec0afea0 0x7fd4ec064870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:38:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4f1b7a700 1 --2- 172.21.15.32:0/3754811186 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4ec0af920 0x7fd4ec154970 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fd4e0002f30 tx=0x7fd4e0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4df7fe700 1 -- 172.21.15.32:0/3754811186 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4e0011a20 con 0x7fd4ec0af920 2020-12-05T17:38:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4df7fe700 1 -- 172.21.15.32:0/3754811186 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e0011b80 con 0x7fd4ec0af920 2020-12-05T17:38:40.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4f337d700 1 -- 172.21.15.32:0/3754811186 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ec066600 con 0x7fd4ec0af920 2020-12-05T17:38:40.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4df7fe700 1 -- 172.21.15.32:0/3754811186 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4e001d440 con 0x7fd4ec0af920 2020-12-05T17:38:40.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4f337d700 1 -- 172.21.15.32:0/3754811186 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ec146fc0 con 0x7fd4ec0af920 2020-12-05T17:38:40.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.395+0000 7fd4f337d700 1 -- 172.21.15.32:0/3754811186 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4ec064110 con 0x7fd4ec0af920 2020-12-05T17:38:40.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.399+0000 7fd4df7fe700 1 -- 172.21.15.32:0/3754811186 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd4e0024030 con 0x7fd4ec0af920 2020-12-05T17:38:40.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.399+0000 7fd4df7fe700 1 --2- 172.21.15.32:0/3754811186 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd4cc055440 0x7fd4cc057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:40.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.399+0000 7fd4df7fe700 1 -- 172.21.15.32:0/3754811186 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6463+0+0 (secure 0 0 0) 0x7fd4e006ef30 con 0x7fd4ec0af920 2020-12-05T17:38:40.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.399+0000 7fd4f237b700 1 --2- 172.21.15.32:0/3754811186 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd4cc055440 0x7fd4cc057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:40.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.399+0000 7fd4f237b700 1 --2- 172.21.15.32:0/3754811186 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd4cc055440 0x7fd4cc057900 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fd4e400d000 tx=0x7fd4e4007ee0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:40.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.407+0000 7fd4df7fe700 1 -- 172.21.15.32:0/3754811186 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd4e00450b0 con 0x7fd4ec0af920 2020-12-05T17:38:40.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.595+0000 7fd4f337d700 1 -- 172.21.15.32:0/3754811186 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4ec064110 con 0x7fd4ec0af920 2020-12-05T17:38:40.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.595+0000 7fd4df7fe700 1 -- 172.21.15.32:0/3754811186 <== mon.1 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+2137 (secure 0 0 0) 0x7fd4ec064110 con 0x7fd4ec0af920 2020-12-05T17:38:40.601 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:38:40.601 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":32,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5645,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/992428276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":992428276},{"type":"v1","addr":"172.21.15.32:6835","nonce":992428276}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:37:46.495876+0000","modified":"2020-12-05T17:37:47.498743+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6160":{"gid":6160,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.61:6833/4214217205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4214217205},{"type":"v1","addr":"172.21.15.61:6833","nonce":4214217205}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-12-05T17:38:40.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.595+0000 7fd4f337d700 1 -- 172.21.15.32:0/3754811186 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd4cc055440 msgr2=0x7fd4cc057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:40.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.595+0000 7fd4f337d700 1 --2- 172.21.15.32:0/3754811186 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd4cc055440 0x7fd4cc057900 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fd4e400d000 tx=0x7fd4e4007ee0).stop 2020-12-05T17:38:40.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.595+0000 7fd4f337d700 1 -- 172.21.15.32:0/3754811186 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4ec0af920 msgr2=0x7fd4ec154970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:40.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.595+0000 7fd4f337d700 1 --2- 172.21.15.32:0/3754811186 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4ec0af920 0x7fd4ec154970 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fd4e0002f30 tx=0x7fd4e0015000).stop 2020-12-05T17:38:40.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.599+0000 7fd4f337d700 1 -- 172.21.15.32:0/3754811186 shutdown_connections 2020-12-05T17:38:40.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.599+0000 7fd4f337d700 1 --2- 172.21.15.32:0/3754811186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ec0afea0 0x7fd4ec064870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:40.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.599+0000 7fd4f337d700 1 --2- 172.21.15.32:0/3754811186 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd4cc055440 0x7fd4cc057900 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:40.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.599+0000 7fd4f337d700 1 --2- 172.21.15.32:0/3754811186 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4ec05f6f0 0x7fd4ec0632e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:40.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.599+0000 7fd4f337d700 1 --2- 172.21.15.32:0/3754811186 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4ec0af920 0x7fd4ec154970 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:40.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.599+0000 7fd4f337d700 1 -- 172.21.15.32:0/3754811186 >> 172.21.15.32:0/3754811186 conn(0x7fd4ec0604c0 msgr2=0x7fd4ec141040 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:40.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.599+0000 7fd4f337d700 1 -- 172.21.15.32:0/3754811186 shutdown_connections 2020-12-05T17:38:40.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:40.599+0000 7fd4f337d700 1 -- 172.21.15.32:0/3754811186 wait complete. 2020-12-05T17:38:40.606 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 32 2020-12-05T17:38:40.820 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:37:46.495876+0000', 'modified': '2020-12-05T17:37:47.498743+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6160}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6160': {'gid': 6160, 'name': 'b', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.61:6833/4214217205', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6832', 'nonce': 4214217205}, {'type': 'v1', 'addr': '172.21.15.61:6833', 'nonce': 4214217205}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:38:40.821 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:38:40.821 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-12-05T17:38:40.948 INFO:teuthology.orchestra.run.smithi061.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":6160,"want_state":"up:active","state":"up:active","rank_uptime":54.437200498000003,"mdsmap_epoch":32,"osdmap_epoch":73,"osdmap_epoch_barrier":73,"uptime":58.769868056} 2020-12-05T17:38:40.948 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 6160, "mdsmap_epoch": 32, "osdmap_epoch": 73, "osdmap_epoch_barrier": 73, "rank_uptime": 54.437200498, "state": "up:active", "uptime": 58.769868056, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:38:40.949 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:38:40.949 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-12-05T17:38:40.950 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-12-05T17:38:40.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:38:40.950 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:38:40.958 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:38:41.015 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:38:41.015 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:38:41.070 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:38:41.071 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:38:41.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:38:41.072 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:38:41.118 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:38:41.119 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:38:41.161 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:38:41.176 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:38:41.177 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:38:41.235 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:38:41.236 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:38:41.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:38:41.237 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:38:41.239 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:2020-12-05T17:38:41.235+0000 7f0a59f160c0 -1 init, newargv = 0x55c0a1909610 newargc=9ceph-fuse[11080]: starting ceph client 2020-12-05T17:38:41.239 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr: 2020-12-05T17:38:41.250 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[11080]: starting fuse 2020-12-05T17:38:42.347 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:38:42.363 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:38:42.363 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:38:42.415 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:38:42.416 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:38:42.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:38:42.417 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:38:42.466 INFO:teuthology.orchestra.run.smithi036.stdout:49 2020-12-05T17:38:42.466 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-12-05T17:38:42.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:38:42.467 INFO:teuthology.orchestra.run.smithi036:> sudo python3 -c ' 2020-12-05T17:38:42.467 INFO:teuthology.orchestra.run.smithi036:> import glob 2020-12-05T17:38:42.468 INFO:teuthology.orchestra.run.smithi036:> import re 2020-12-05T17:38:42.468 INFO:teuthology.orchestra.run.smithi036:> import os 2020-12-05T17:38:42.469 INFO:teuthology.orchestra.run.smithi036:> import subprocess 2020-12-05T17:38:42.469 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:38:42.469 INFO:teuthology.orchestra.run.smithi036:> def find_socket(client_name): 2020-12-05T17:38:42.470 INFO:teuthology.orchestra.run.smithi036:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-12-05T17:38:42.470 INFO:teuthology.orchestra.run.smithi036:> files = glob.glob(asok_path) 2020-12-05T17:38:42.471 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:38:42.471 INFO:teuthology.orchestra.run.smithi036:> # Given a non-glob path, it better be there 2020-12-05T17:38:42.471 INFO:teuthology.orchestra.run.smithi036:> if "*" not in asok_path: 2020-12-05T17:38:42.472 INFO:teuthology.orchestra.run.smithi036:> assert(len(files) == 1) 2020-12-05T17:38:42.472 INFO:teuthology.orchestra.run.smithi036:> return files[0] 2020-12-05T17:38:42.472 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:38:42.473 INFO:teuthology.orchestra.run.smithi036:> for f in files: 2020-12-05T17:38:42.473 INFO:teuthology.orchestra.run.smithi036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:38:42.474 INFO:teuthology.orchestra.run.smithi036:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:38:42.474 INFO:teuthology.orchestra.run.smithi036:> return f 2020-12-05T17:38:42.474 INFO:teuthology.orchestra.run.smithi036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:38:42.475 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:38:42.475 INFO:teuthology.orchestra.run.smithi036:> print(find_socket("client.0")) 2020-12-05T17:38:42.475 INFO:teuthology.orchestra.run.smithi036:> ' 2020-12-05T17:38:42.559 INFO:teuthology.orchestra.run.smithi036.stdout:/var/run/ceph/ceph-client.0.11080.asok 2020-12-05T17:38:42.564 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.11080.asok 2020-12-05T17:38:42.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:38:42.565 INFO:teuthology.orchestra.run.smithi036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.11080.asok status 2020-12-05T17:38:42.667 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2020-12-05T17:38:42.667 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": { 2020-12-05T17:38:42.668 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:38:42.668 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:38:42.668 INFO:teuthology.orchestra.run.smithi036.stdout: "entity_id": "0", 2020-12-05T17:38:42.668 INFO:teuthology.orchestra.run.smithi036.stdout: "hostname": "smithi036", 2020-12-05T17:38:42.669 INFO:teuthology.orchestra.run.smithi036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-12-05T17:38:42.669 INFO:teuthology.orchestra.run.smithi036.stdout: "pid": "11080", 2020-12-05T17:38:42.669 INFO:teuthology.orchestra.run.smithi036.stdout: "root": "/" 2020-12-05T17:38:42.669 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:38:42.670 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_count": 0, 2020-12-05T17:38:42.670 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_pinned_count": 0, 2020-12-05T17:38:42.670 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 6513, 2020-12-05T17:38:42.670 INFO:teuthology.orchestra.run.smithi036.stdout: "inst": { 2020-12-05T17:38:42.671 INFO:teuthology.orchestra.run.smithi036.stdout: "name": { 2020-12-05T17:38:42.671 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "client", 2020-12-05T17:38:42.671 INFO:teuthology.orchestra.run.smithi036.stdout: "num": 6513 2020-12-05T17:38:42.671 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:38:42.671 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:38:42.672 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:38:42.672 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:38:42.672 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 521305946 2020-12-05T17:38:42.672 INFO:teuthology.orchestra.run.smithi036.stdout: } 2020-12-05T17:38:42.673 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:38:42.673 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:38:42.673 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:38:42.673 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:38:42.674 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 521305946 2020-12-05T17:38:42.674 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:38:42.674 INFO:teuthology.orchestra.run.smithi036.stdout: "inst_str": "client.6513 172.21.15.36:0/521305946", 2020-12-05T17:38:42.674 INFO:teuthology.orchestra.run.smithi036.stdout: "addr_str": "172.21.15.36:0/521305946", 2020-12-05T17:38:42.675 INFO:teuthology.orchestra.run.smithi036.stdout: "inode_count": 1, 2020-12-05T17:38:42.675 INFO:teuthology.orchestra.run.smithi036.stdout: "mds_epoch": 32, 2020-12-05T17:38:42.675 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch": 73, 2020-12-05T17:38:42.675 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:38:42.675 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklisted": false 2020-12-05T17:38:42.676 INFO:teuthology.orchestra.run.smithi036.stdout:} 2020-12-05T17:38:42.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:38:42.679 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:38:42.679 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:38:42.686 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:38:42.687 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:38:42.688 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:38:42.688 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:38:42.746 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:38:42.747 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:38:42.764 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:38:42.781 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:38:42.804 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:38:42.821 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:38:42.837 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:38:42.838 INFO:tasks.cephfs.mount:Terminating background process 2020-12-05T17:38:42.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:38:42.839 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:38:42.839 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:38:42.855 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:38:42.856 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:38:42.856 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi036.front.sepia.ceph.com... 2020-12-05T17:38:42.857 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:38:42.857 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:38:42.940 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[11080]: fuse finished with error 0 and tester_r 0 2020-12-05T17:38:42.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:38:42.943 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:38:42.943 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:38:42.990 INFO:teuthology.orchestra.run.smithi036.stdout:ext2/ext3 2020-12-05T17:38:42.991 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:38:43.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:38:43.035 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:38:43.043 INFO:tasks.cephfs.mount:Terminating background process 2020-12-05T17:38:43.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:38:43.044 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:38:43.044 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:38:43.052 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:38:43.053 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:38:43.053 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi002.front.sepia.ceph.com... 2020-12-05T17:38:43.054 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:38:43.054 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:38:43.122 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[10556]: fuse finished with error 0 and tester_r 0 2020-12-05T17:38:43.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:38:43.125 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:38:43.125 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:38:43.172 INFO:teuthology.orchestra.run.smithi002.stdout:ext2/ext3 2020-12-05T17:38:43.174 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:38:43.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:38:43.293 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:38:43.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:38:43.341 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:38:43.341 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:38:43.348 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:38:43.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:38:43.349 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2020-12-05T17:38:43.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:38:43.350 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:38:43.395 INFO:teuthology.orchestra.run.smithi053.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:38:43.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:38:43.397 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2020-12-05T17:38:43.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.519+0000 7f9d89174700 1 Processor -- start 2020-12-05T17:38:43.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.519+0000 7f9d89174700 1 -- start start 2020-12-05T17:38:43.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d89174700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9d840b1fc0 0x7f9d840af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:43.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d89174700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d840ab710 0x7f9d84061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:43.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d89174700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9d84147400 0x7f9d84145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:43.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d89174700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d840b0110 con 0x7f9d840ab710 2020-12-05T17:38:43.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d89174700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d84146a90 con 0x7f9d840b1fc0 2020-12-05T17:38:43.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d89174700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d840b23a0 con 0x7f9d84147400 2020-12-05T17:38:43.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d837fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d840ab710 0x7f9d84061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:43.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d837fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d840ab710 0x7f9d84061780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37042/0 (socket says 172.21.15.32:37042) 2020-12-05T17:38:43.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d837fe700 1 -- 172.21.15.32:0/2074861724 learned_addr learned my addr 172.21.15.32:0/2074861724 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:43.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d88973700 1 --2- 172.21.15.32:0/2074861724 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9d84147400 0x7f9d84145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:43.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d83fff700 1 --2- 172.21.15.32:0/2074861724 >> v2:172.21.15.61:3300/0 conn(0x7f9d840b1fc0 0x7f9d840af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:43.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d837fe700 1 -- 172.21.15.32:0/2074861724 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9d84147400 msgr2=0x7f9d84145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:43.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d837fe700 1 --2- 172.21.15.32:0/2074861724 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9d84147400 0x7f9d84145d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:43.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d837fe700 1 -- 172.21.15.32:0/2074861724 >> v2:172.21.15.61:3300/0 conn(0x7f9d840b1fc0 msgr2=0x7f9d840af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:43.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d837fe700 1 --2- 172.21.15.32:0/2074861724 >> v2:172.21.15.61:3300/0 conn(0x7f9d840b1fc0 0x7f9d840af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:43.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d837fe700 1 -- 172.21.15.32:0/2074861724 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d84146cc0 con 0x7f9d840ab710 2020-12-05T17:38:43.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d83fff700 1 --2- 172.21.15.32:0/2074861724 >> v2:172.21.15.61:3300/0 conn(0x7f9d840b1fc0 0x7f9d840af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:38:43.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d88973700 1 --2- 172.21.15.32:0/2074861724 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9d84147400 0x7f9d84145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:38:43.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d837fe700 1 --2- 172.21.15.32:0/2074861724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d840ab710 0x7f9d84061780 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f9d6c002f30 tx=0x7f9d6c006e80).ready entity=mon.0 client_cookie=c13e61c4c8132702 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:43.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d82ffd700 1 -- 172.21.15.32:0/2074861724 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d6c0097f0 con 0x7f9d840ab710 2020-12-05T17:38:43.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d82ffd700 1 -- 172.21.15.32:0/2074861724 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6c009950 con 0x7f9d840ab710 2020-12-05T17:38:43.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d82ffd700 1 -- 172.21.15.32:0/2074861724 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d6c009ac0 con 0x7f9d840ab710 2020-12-05T17:38:43.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d89174700 1 -- 172.21.15.32:0/2074861724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d840ab710 msgr2=0x7f9d84061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:43.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d89174700 1 --2- 172.21.15.32:0/2074861724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d840ab710 0x7f9d84061780 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f9d6c002f30 tx=0x7f9d6c006e80).stop 2020-12-05T17:38:43.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d89174700 1 -- 172.21.15.32:0/2074861724 shutdown_connections 2020-12-05T17:38:43.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d89174700 1 --2- 172.21.15.32:0/2074861724 >> v2:172.21.15.61:3300/0 conn(0x7f9d840b1fc0 0x7f9d840af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:43.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d89174700 1 --2- 172.21.15.32:0/2074861724 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9d84147400 0x7f9d84145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:43.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d89174700 1 --2- 172.21.15.32:0/2074861724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d840ab710 0x7f9d84061780 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:43.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d89174700 1 -- 172.21.15.32:0/2074861724 >> 172.21.15.32:0/2074861724 conn(0x7f9d840b5530 msgr2=0x7f9d840b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:43.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d89174700 1 -- 172.21.15.32:0/2074861724 shutdown_connections 2020-12-05T17:38:43.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d89174700 1 -- 172.21.15.32:0/2074861724 wait complete. 2020-12-05T17:38:43.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d89174700 1 Processor -- start 2020-12-05T17:38:43.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.523+0000 7f9d89174700 1 -- start start 2020-12-05T17:38:43.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d89174700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d840ab710 0x7f9d84148190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:43.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d89174700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9d840b1fc0 0x7f9d84158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:43.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d89174700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9d84147400 0x7f9d840635c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:43.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d89174700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d840abf80 con 0x7f9d840ab710 2020-12-05T17:38:43.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d89174700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d840ac220 con 0x7f9d840b1fc0 2020-12-05T17:38:43.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d89174700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d84063120 con 0x7f9d84147400 2020-12-05T17:38:43.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d83fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d840ab710 0x7f9d84148190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:43.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d83fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d840ab710 0x7f9d84148190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37048/0 (socket says 172.21.15.32:37048) 2020-12-05T17:38:43.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d83fff700 1 -- 172.21.15.32:0/394975618 learned_addr learned my addr 172.21.15.32:0/394975618 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:43.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d88973700 1 --2- 172.21.15.32:0/394975618 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9d84147400 0x7f9d840635c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:43.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d837fe700 1 --2- 172.21.15.32:0/394975618 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9d840b1fc0 0x7f9d84158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:43.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d83fff700 1 -- 172.21.15.32:0/394975618 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9d84147400 msgr2=0x7f9d840635c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:43.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d83fff700 1 --2- 172.21.15.32:0/394975618 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9d84147400 0x7f9d840635c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:43.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d83fff700 1 -- 172.21.15.32:0/394975618 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9d840b1fc0 msgr2=0x7f9d84158c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:43.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d83fff700 1 --2- 172.21.15.32:0/394975618 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9d840b1fc0 0x7f9d84158c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:43.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d83fff700 1 -- 172.21.15.32:0/394975618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d840ad240 con 0x7f9d840ab710 2020-12-05T17:38:43.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d837fe700 1 --2- 172.21.15.32:0/394975618 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9d840b1fc0 0x7f9d84158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:38:43.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d83fff700 1 --2- 172.21.15.32:0/394975618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d840ab710 0x7f9d84148190 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f9d74007000 tx=0x7f9d74008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:43.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d817fa700 1 -- 172.21.15.32:0/394975618 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d74010580 con 0x7f9d840ab710 2020-12-05T17:38:43.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d817fa700 1 -- 172.21.15.32:0/394975618 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d740106e0 con 0x7f9d840ab710 2020-12-05T17:38:43.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d89174700 1 -- 172.21.15.32:0/394975618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d840abb50 con 0x7f9d840ab710 2020-12-05T17:38:43.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d817fa700 1 -- 172.21.15.32:0/394975618 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d74010a40 con 0x7f9d840ab710 2020-12-05T17:38:43.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d89174700 1 -- 172.21.15.32:0/394975618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d840ad7b0 con 0x7f9d840ab710 2020-12-05T17:38:43.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.527+0000 7f9d89174700 1 -- 172.21.15.32:0/394975618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d840632e0 con 0x7f9d840ab710 2020-12-05T17:38:43.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.531+0000 7f9d817fa700 1 -- 172.21.15.32:0/394975618 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9d74010ba0 con 0x7f9d840ab710 2020-12-05T17:38:43.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.531+0000 7f9d817fa700 1 --2- 172.21.15.32:0/394975618 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9d68055330 0x7f9d680577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:43.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.531+0000 7f9d817fa700 1 -- 172.21.15.32:0/394975618 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f9d7406e360 con 0x7f9d840ab710 2020-12-05T17:38:43.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.531+0000 7f9d837fe700 1 --2- 172.21.15.32:0/394975618 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9d68055330 0x7f9d680577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:43.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.531+0000 7f9d837fe700 1 --2- 172.21.15.32:0/394975618 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9d68055330 0x7f9d680577f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f9d6c019000 tx=0x7f9d6c016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:43.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.539+0000 7f9d817fa700 1 -- 172.21.15.32:0/394975618 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9d740443d0 con 0x7f9d840ab710 2020-12-05T17:38:43.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.675+0000 7f9d89174700 1 -- 172.21.15.32:0/394975618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9d84057050 con 0x7f9d840ab710 2020-12-05T17:38:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.679+0000 7f9d817fa700 1 -- 172.21.15.32:0/394975618 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9d740479f0 con 0x7f9d840ab710 2020-12-05T17:38:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.683+0000 7f9d89174700 1 -- 172.21.15.32:0/394975618 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9d68055330 msgr2=0x7f9d680577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.683+0000 7f9d89174700 1 --2- 172.21.15.32:0/394975618 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9d68055330 0x7f9d680577f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f9d6c019000 tx=0x7f9d6c016000).stop 2020-12-05T17:38:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.683+0000 7f9d89174700 1 -- 172.21.15.32:0/394975618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d840ab710 msgr2=0x7f9d84148190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.683+0000 7f9d89174700 1 --2- 172.21.15.32:0/394975618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d840ab710 0x7f9d84148190 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f9d74007000 tx=0x7f9d74008040).stop 2020-12-05T17:38:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.683+0000 7f9d89174700 1 -- 172.21.15.32:0/394975618 shutdown_connections 2020-12-05T17:38:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.683+0000 7f9d89174700 1 --2- 172.21.15.32:0/394975618 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9d68055330 0x7f9d680577f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.683+0000 7f9d89174700 1 --2- 172.21.15.32:0/394975618 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9d84147400 0x7f9d840635c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.683+0000 7f9d89174700 1 --2- 172.21.15.32:0/394975618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d840ab710 0x7f9d84148190 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.683+0000 7f9d89174700 1 --2- 172.21.15.32:0/394975618 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9d840b1fc0 0x7f9d84158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:43.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.683+0000 7f9d89174700 1 -- 172.21.15.32:0/394975618 >> 172.21.15.32:0/394975618 conn(0x7f9d840b5530 msgr2=0x7f9d840af090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:43.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.683+0000 7f9d89174700 1 -- 172.21.15.32:0/394975618 shutdown_connections 2020-12-05T17:38:43.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.683+0000 7f9d89174700 1 -- 172.21.15.32:0/394975618 wait complete. 2020-12-05T17:38:43.708 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder' 2020-12-05T17:38:43.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f979195e700 1 Processor -- start 2020-12-05T17:38:43.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f979195e700 1 -- start start 2020-12-05T17:38:43.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f979195e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f978c05cd70 0x7f978c143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:43.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f979195e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f978c05f7f0 0x7f978c0af160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:43.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f979195e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f978c0af650 0x7f978c0b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:43.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f979195e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f978c0633c0 con 0x7f978c05cd70 2020-12-05T17:38:43.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f979195e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f978c05d150 con 0x7f978c0af650 2020-12-05T17:38:43.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f979195e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f978c05fe10 con 0x7f978c05f7f0 2020-12-05T17:38:43.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f978affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f978c05cd70 0x7f978c143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:43.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f978affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f978c05cd70 0x7f978c143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37056/0 (socket says 172.21.15.32:37056) 2020-12-05T17:38:43.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f978affd700 1 -- 172.21.15.32:0/856774062 learned_addr learned my addr 172.21.15.32:0/856774062 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:43.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f978a7fc700 1 --2- 172.21.15.32:0/856774062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f978c05f7f0 0x7f978c0af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:43.826 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f978b7fe700 1 --2- 172.21.15.32:0/856774062 >> v2:172.21.15.61:3300/0 conn(0x7f978c0af650 0x7f978c0b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:43.826 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f978affd700 1 -- 172.21.15.32:0/856774062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f978c05f7f0 msgr2=0x7f978c0af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:43.826 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f978affd700 1 --2- 172.21.15.32:0/856774062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f978c05f7f0 0x7f978c0af160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:43.826 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f978affd700 1 -- 172.21.15.32:0/856774062 >> v2:172.21.15.61:3300/0 conn(0x7f978c0af650 msgr2=0x7f978c0b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:43.826 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f978affd700 1 --2- 172.21.15.32:0/856774062 >> v2:172.21.15.61:3300/0 conn(0x7f978c0af650 0x7f978c0b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:43.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f978affd700 1 -- 172.21.15.32:0/856774062 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978c0600b0 con 0x7f978c05cd70 2020-12-05T17:38:43.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f978a7fc700 1 --2- 172.21.15.32:0/856774062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f978c05f7f0 0x7f978c0af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:38:43.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f978affd700 1 --2- 172.21.15.32:0/856774062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f978c05cd70 0x7f978c143d30 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7f977c0032b0 tx=0x7f977c009270).ready entity=mon.0 client_cookie=a1e564c2aee15afa server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:43.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f9789ffb700 1 -- 172.21.15.32:0/856774062 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f977c00bc00 con 0x7f978c05cd70 2020-12-05T17:38:43.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f9789ffb700 1 -- 172.21.15.32:0/856774062 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977c00bd60 con 0x7f978c05cd70 2020-12-05T17:38:43.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f9789ffb700 1 -- 172.21.15.32:0/856774062 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f977c011400 con 0x7f978c05cd70 2020-12-05T17:38:43.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f979195e700 1 -- 172.21.15.32:0/856774062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f978c05cd70 msgr2=0x7f978c143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:43.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.815+0000 7f979195e700 1 --2- 172.21.15.32:0/856774062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f978c05cd70 0x7f978c143d30 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7f977c0032b0 tx=0x7f977c009270).stop 2020-12-05T17:38:43.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.819+0000 7f979195e700 1 -- 172.21.15.32:0/856774062 shutdown_connections 2020-12-05T17:38:43.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.819+0000 7f979195e700 1 --2- 172.21.15.32:0/856774062 >> v2:172.21.15.61:3300/0 conn(0x7f978c0af650 0x7f978c0b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:43.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.819+0000 7f979195e700 1 --2- 172.21.15.32:0/856774062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f978c05cd70 0x7f978c143d30 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:43.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.819+0000 7f979195e700 1 --2- 172.21.15.32:0/856774062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f978c05f7f0 0x7f978c0af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:43.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.819+0000 7f979195e700 1 -- 172.21.15.32:0/856774062 >> 172.21.15.32:0/856774062 conn(0x7f978c069cb0 msgr2=0x7f978c0604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:43.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.819+0000 7f979195e700 1 -- 172.21.15.32:0/856774062 shutdown_connections 2020-12-05T17:38:43.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.819+0000 7f979195e700 1 -- 172.21.15.32:0/856774062 wait complete. 2020-12-05T17:38:43.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.819+0000 7f979195e700 1 Processor -- start 2020-12-05T17:38:43.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.819+0000 7f979195e700 1 -- start start 2020-12-05T17:38:43.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.819+0000 7f979195e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f978c05cd70 0x7f978c0b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:43.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.819+0000 7f979195e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f978c05f7f0 0x7f978c066000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:43.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.819+0000 7f979195e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f978c0af650 0x7f978c064b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:43.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.819+0000 7f979195e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c0ae0e0 con 0x7f978c0af650 2020-12-05T17:38:43.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.819+0000 7f979195e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c0ae380 con 0x7f978c05f7f0 2020-12-05T17:38:43.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.819+0000 7f979195e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f978c065b60 con 0x7f978c05cd70 2020-12-05T17:38:43.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.819+0000 7f978b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f978c0af650 0x7f978c064b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:43.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.819+0000 7f978b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f978c0af650 0x7f978c064b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37066/0 (socket says 172.21.15.32:37066) 2020-12-05T17:38:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.819+0000 7f978b7fe700 1 -- 172.21.15.32:0/3505177451 learned_addr learned my addr 172.21.15.32:0/3505177451 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.819+0000 7f978affd700 1 --2- 172.21.15.32:0/3505177451 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f978c05cd70 0x7f978c0b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.819+0000 7f978a7fc700 1 --2- 172.21.15.32:0/3505177451 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f978c05f7f0 0x7f978c066000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.823+0000 7f978b7fe700 1 -- 172.21.15.32:0/3505177451 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f978c05cd70 msgr2=0x7f978c0b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.823+0000 7f978b7fe700 1 --2- 172.21.15.32:0/3505177451 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f978c05cd70 0x7f978c0b49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.823+0000 7f978b7fe700 1 -- 172.21.15.32:0/3505177451 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f978c05f7f0 msgr2=0x7f978c066000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.823+0000 7f978b7fe700 1 --2- 172.21.15.32:0/3505177451 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f978c05f7f0 0x7f978c066000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.823+0000 7f978b7fe700 1 -- 172.21.15.32:0/3505177451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978c0664f0 con 0x7f978c0af650 2020-12-05T17:38:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.823+0000 7f978b7fe700 1 --2- 172.21.15.32:0/3505177451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f978c0af650 0x7f978c064b50 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7f978400a000 tx=0x7f9784008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.823+0000 7f9783fff700 1 -- 172.21.15.32:0/3505177451 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f978400f7e0 con 0x7f978c0af650 2020-12-05T17:38:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.823+0000 7f9783fff700 1 -- 172.21.15.32:0/3505177451 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978400f940 con 0x7f978c0af650 2020-12-05T17:38:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.823+0000 7f979195e700 1 -- 172.21.15.32:0/3505177451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f978c0addd0 con 0x7f978c0af650 2020-12-05T17:38:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.823+0000 7f9783fff700 1 -- 172.21.15.32:0/3505177451 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f978400fca0 con 0x7f978c0af650 2020-12-05T17:38:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.823+0000 7f979195e700 1 -- 172.21.15.32:0/3505177451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f978c144590 con 0x7f978c0af650 2020-12-05T17:38:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.823+0000 7f979195e700 1 -- 172.21.15.32:0/3505177451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f978c065d20 con 0x7f978c0af650 2020-12-05T17:38:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.823+0000 7f9783fff700 1 -- 172.21.15.32:0/3505177451 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f978401a030 con 0x7f978c0af650 2020-12-05T17:38:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.823+0000 7f9783fff700 1 --2- 172.21.15.32:0/3505177451 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f97700553b0 0x7f9770057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.823+0000 7f978affd700 1 --2- 172.21.15.32:0/3505177451 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f97700553b0 0x7f9770057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.823+0000 7f9783fff700 1 -- 172.21.15.32:0/3505177451 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f978406de60 con 0x7f978c0af650 2020-12-05T17:38:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.827+0000 7f978affd700 1 --2- 172.21.15.32:0/3505177451 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f97700553b0 0x7f9770057870 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f977c01a000 tx=0x7f977c016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:43.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:43.831+0000 7f9783fff700 1 -- 172.21.15.32:0/3505177451 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9784043ed0 con 0x7f978c0af650 2020-12-05T17:38:44.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:44.019+0000 7f979195e700 1 -- 172.21.15.32:0/3505177451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]} v 0) v1 -- 0x7f978c057050 con 0x7f978c0af650 2020-12-05T17:38:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.027+0000 7f9783fff700 1 -- 172.21.15.32:0/3505177451 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]}]=0 v305) v1 ==== 139+0+0 (secure 0 0 0) 0x7f97840474f0 con 0x7f978c0af650 2020-12-05T17:38:45.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.031+0000 7f979195e700 1 -- 172.21.15.32:0/3505177451 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f97700553b0 msgr2=0x7f9770057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:45.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.031+0000 7f979195e700 1 --2- 172.21.15.32:0/3505177451 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f97700553b0 0x7f9770057870 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f977c01a000 tx=0x7f977c016000).stop 2020-12-05T17:38:45.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.031+0000 7f979195e700 1 -- 172.21.15.32:0/3505177451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f978c0af650 msgr2=0x7f978c064b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:45.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.031+0000 7f979195e700 1 --2- 172.21.15.32:0/3505177451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f978c0af650 0x7f978c064b50 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7f978400a000 tx=0x7f9784008040).stop 2020-12-05T17:38:45.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.031+0000 7f979195e700 1 -- 172.21.15.32:0/3505177451 shutdown_connections 2020-12-05T17:38:45.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.031+0000 7f979195e700 1 --2- 172.21.15.32:0/3505177451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f978c0af650 0x7f978c064b50 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:45.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.031+0000 7f979195e700 1 --2- 172.21.15.32:0/3505177451 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f97700553b0 0x7f9770057870 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:45.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.031+0000 7f979195e700 1 --2- 172.21.15.32:0/3505177451 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f978c05cd70 0x7f978c0b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.031+0000 7f979195e700 1 --2- 172.21.15.32:0/3505177451 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f978c05f7f0 0x7f978c066000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.031+0000 7f979195e700 1 -- 172.21.15.32:0/3505177451 >> 172.21.15.32:0/3505177451 conn(0x7f978c069cb0 msgr2=0x7f978c0b5450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.031+0000 7f979195e700 1 -- 172.21.15.32:0/3505177451 shutdown_connections 2020-12-05T17:38:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.031+0000 7f979195e700 1 -- 172.21.15.32:0/3505177451 wait complete. 2020-12-05T17:38:45.058 INFO:tasks.cephfs_test_runner:test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc) ... ok 2020-12-05T17:38:45.058 INFO:tasks.cephfs_test_runner:Starting test: test_dump_inode (tasks.cephfs.test_misc.TestMisc) 2020-12-05T17:38:45.059 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode' 2020-12-05T17:38:45.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc830f96700 1 Processor -- start 2020-12-05T17:38:45.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc830f96700 1 -- start start 2020-12-05T17:38:45.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc830f96700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc82c062400 0x7fc82c05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:45.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc830f96700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc82c05feb0 0x7fc82c0b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:45.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc830f96700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc82c142450 con 0x7fc82c05f3b0 2020-12-05T17:38:45.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc830f96700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc82c0b42f0 con 0x7fc82c062400 2020-12-05T17:38:45.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc830f96700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc82c061780 con 0x7fc82c05feb0 2020-12-05T17:38:45.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc82affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc82c062400 0x7fc82c05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:45.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc82b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc82c05f3b0 0x7fc82c0b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39720/0 (socket says 172.21.15.32:39720) 2020-12-05T17:38:45.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc82b7fe700 1 -- 172.21.15.32:0/152066002 learned_addr learned my addr 172.21.15.32:0/152066002 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:38:45.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc82bfff700 1 --2- 172.21.15.32:0/152066002 >> v2:172.21.15.61:3300/0 conn(0x7fc82c05feb0 0x7fc82c0b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:45.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc82a7fc700 1 -- 172.21.15.32:0/152066002 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2015704634 0 0) 0x7fc82c142450 con 0x7fc82c05f3b0 2020-12-05T17:38:45.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc82a7fc700 1 -- 172.21.15.32:0/152066002 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc80c003230 con 0x7fc82c05f3b0 2020-12-05T17:38:45.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc82affd700 1 -- 172.21.15.32:0/152066002 >> v2:172.21.15.61:3300/0 conn(0x7fc82c05feb0 msgr2=0x7fc82c0b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:45.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc82affd700 1 --2- 172.21.15.32:0/152066002 >> v2:172.21.15.61:3300/0 conn(0x7fc82c05feb0 0x7fc82c0b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:45.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc82a7fc700 1 -- 172.21.15.32:0/152066002 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3667298592 0 0) 0x7fc80c003230 con 0x7fc82c05f3b0 2020-12-05T17:38:45.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc82affd700 1 -- 172.21.15.32:0/152066002 >> v1:172.21.15.32:6789/0 conn(0x7fc82c05f3b0 legacy=0x7fc82c0b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:45.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc82affd700 1 -- 172.21.15.32:0/152066002 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82c061a20 con 0x7fc82c062400 2020-12-05T17:38:45.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc82affd700 1 --2- 172.21.15.32:0/152066002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc82c062400 0x7fc82c05f790 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7fc81c001870 tx=0x7fc814006dc0).ready entity=mon.0 client_cookie=51ab0c75a2d314d5 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:45.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc82a7fc700 1 -- 172.21.15.32:0/152066002 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc81c0016d0 con 0x7fc82c062400 2020-12-05T17:38:45.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc82a7fc700 1 -- 172.21.15.32:0/152066002 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81400d5c0 con 0x7fc82c062400 2020-12-05T17:38:45.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc82a7fc700 1 -- 172.21.15.32:0/152066002 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc81400d730 con 0x7fc82c062400 2020-12-05T17:38:45.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc830f96700 1 -- 172.21.15.32:0/152066002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc82c062400 msgr2=0x7fc82c05f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:45.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.175+0000 7fc830f96700 1 --2- 172.21.15.32:0/152066002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc82c062400 0x7fc82c05f790 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7fc81c001870 tx=0x7fc814006dc0).stop 2020-12-05T17:38:45.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc830f96700 1 -- 172.21.15.32:0/152066002 shutdown_connections 2020-12-05T17:38:45.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc830f96700 1 --2- 172.21.15.32:0/152066002 >> v2:172.21.15.61:3300/0 conn(0x7fc82c05feb0 0x7fc82c0b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:45.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc830f96700 1 --2- 172.21.15.32:0/152066002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc82c062400 0x7fc82c05f790 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:45.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc830f96700 1 -- 172.21.15.32:0/152066002 >> 172.21.15.32:0/152066002 conn(0x7fc82c0b8ee0 msgr2=0x7fc82c0b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:45.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc830f96700 1 -- 172.21.15.32:0/152066002 shutdown_connections 2020-12-05T17:38:45.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc830f96700 1 -- 172.21.15.32:0/152066002 wait complete. 2020-12-05T17:38:45.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc830f96700 1 Processor -- start 2020-12-05T17:38:45.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc830f96700 1 -- start start 2020-12-05T17:38:45.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc830f96700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc82c05f3b0 0x7fc82c063340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:45.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc830f96700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc82c05feb0 0x7fc82c147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:45.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc830f96700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc82c062400 0x7fc82c148140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:45.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc830f96700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82c146aa0 con 0x7fc82c05feb0 2020-12-05T17:38:45.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc830f96700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82c144610 con 0x7fc82c062400 2020-12-05T17:38:45.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc830f96700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82c144870 con 0x7fc82c05f3b0 2020-12-05T17:38:45.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc82affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc82c05feb0 0x7fc82c147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:45.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc82affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc82c05feb0 0x7fc82c147c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37080/0 (socket says 172.21.15.32:37080) 2020-12-05T17:38:45.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc82affd700 1 -- 172.21.15.32:0/1889930234 learned_addr learned my addr 172.21.15.32:0/1889930234 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:45.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc82b7fe700 1 --2- 172.21.15.32:0/1889930234 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc82c05f3b0 0x7fc82c063340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:45.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc82bfff700 1 --2- 172.21.15.32:0/1889930234 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc82c062400 0x7fc82c148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:45.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc82affd700 1 -- 172.21.15.32:0/1889930234 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc82c05f3b0 msgr2=0x7fc82c063340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:45.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc82affd700 1 --2- 172.21.15.32:0/1889930234 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc82c05f3b0 0x7fc82c063340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:45.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc82affd700 1 -- 172.21.15.32:0/1889930234 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc82c062400 msgr2=0x7fc82c148140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc82affd700 1 --2- 172.21.15.32:0/1889930234 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc82c062400 0x7fc82c148140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc82affd700 1 -- 172.21.15.32:0/1889930234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82c148ed0 con 0x7fc82c05feb0 2020-12-05T17:38:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc82affd700 1 --2- 172.21.15.32:0/1889930234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc82c05feb0 0x7fc82c147c50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7fc81c001870 tx=0x7fc814015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:45.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc828ff9700 1 -- 172.21.15.32:0/1889930234 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc814011850 con 0x7fc82c05feb0 2020-12-05T17:38:45.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc828ff9700 1 -- 172.21.15.32:0/1889930234 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8140119b0 con 0x7fc82c05feb0 2020-12-05T17:38:45.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc830f96700 1 -- 172.21.15.32:0/1889930234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc82c146620 con 0x7fc82c05feb0 2020-12-05T17:38:45.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc828ff9700 1 -- 172.21.15.32:0/1889930234 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc814011d10 con 0x7fc82c05feb0 2020-12-05T17:38:45.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.179+0000 7fc830f96700 1 -- 172.21.15.32:0/1889930234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc82c145170 con 0x7fc82c05feb0 2020-12-05T17:38:45.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.183+0000 7fc830f96700 1 -- 172.21.15.32:0/1889930234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc82c149510 con 0x7fc82c05feb0 2020-12-05T17:38:45.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.183+0000 7fc828ff9700 1 -- 172.21.15.32:0/1889930234 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc814024030 con 0x7fc82c05feb0 2020-12-05T17:38:45.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.183+0000 7fc828ff9700 1 --2- 172.21.15.32:0/1889930234 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc810055330 0x7fc8100577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:45.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.183+0000 7fc828ff9700 1 -- 172.21.15.32:0/1889930234 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6463+0+0 (secure 0 0 0) 0x7fc81406eee0 con 0x7fc82c05feb0 2020-12-05T17:38:45.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.183+0000 7fc82b7fe700 1 --2- 172.21.15.32:0/1889930234 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc810055330 0x7fc8100577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:45.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.183+0000 7fc82b7fe700 1 --2- 172.21.15.32:0/1889930234 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc810055330 0x7fc8100577f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fc81c00d000 tx=0x7fc81c000b80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:45.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.187+0000 7fc828ff9700 1 -- 172.21.15.32:0/1889930234 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc82c149510 con 0x7fc82c05feb0 2020-12-05T17:38:45.385 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:45.379+0000 7fc830f96700 1 -- 172.21.15.32:0/1889930234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]} v 0) v1 -- 0x7fc82c149510 con 0x7fc82c05feb0 2020-12-05T17:38:45.647 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:38:45.650 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:38:45.652 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:38:45.654 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:38:45.656 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:38:46.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.031+0000 7fc828ff9700 1 -- 172.21.15.32:0/1889930234 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]}]=0 v306) v1 ==== 129+0+0 (secure 0 0 0) 0x7fc82c149510 con 0x7fc82c05feb0 2020-12-05T17:38:46.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.035+0000 7fc830f96700 1 -- 172.21.15.32:0/1889930234 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc810055330 msgr2=0x7fc8100577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:46.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.035+0000 7fc830f96700 1 --2- 172.21.15.32:0/1889930234 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc810055330 0x7fc8100577f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fc81c00d000 tx=0x7fc81c000b80).stop 2020-12-05T17:38:46.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.035+0000 7fc830f96700 1 -- 172.21.15.32:0/1889930234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc82c05feb0 msgr2=0x7fc82c147c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:46.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.035+0000 7fc830f96700 1 --2- 172.21.15.32:0/1889930234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc82c05feb0 0x7fc82c147c50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7fc81c001870 tx=0x7fc814015000).stop 2020-12-05T17:38:46.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.035+0000 7fc830f96700 1 -- 172.21.15.32:0/1889930234 shutdown_connections 2020-12-05T17:38:46.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.035+0000 7fc830f96700 1 --2- 172.21.15.32:0/1889930234 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc82c062400 0x7fc82c148140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:46.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.035+0000 7fc830f96700 1 --2- 172.21.15.32:0/1889930234 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc810055330 0x7fc8100577f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:46.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.035+0000 7fc830f96700 1 --2- 172.21.15.32:0/1889930234 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc82c05f3b0 0x7fc82c063340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:46.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.035+0000 7fc830f96700 1 --2- 172.21.15.32:0/1889930234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc82c05feb0 0x7fc82c147c50 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:46.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.035+0000 7fc830f96700 1 -- 172.21.15.32:0/1889930234 >> 172.21.15.32:0/1889930234 conn(0x7fc82c0b8ee0 msgr2=0x7fc82c0af730 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:46.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.035+0000 7fc830f96700 1 -- 172.21.15.32:0/1889930234 shutdown_connections 2020-12-05T17:38:46.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.035+0000 7fc830f96700 1 -- 172.21.15.32:0/1889930234 wait complete. 2020-12-05T17:38:46.060 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-12-05T17:38:46.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.183+0000 7f5c1b431700 1 Processor -- start 2020-12-05T17:38:46.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.183+0000 7f5c1b431700 1 -- start start 2020-12-05T17:38:46.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.183+0000 7f5c1b431700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5c141462e0 0x7f5c141528b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:46.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.183+0000 7f5c1b431700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c140affd0 con 0x7f5c140b2100 2020-12-05T17:38:46.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.183+0000 7f5c1b431700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c140b24e0 con 0x7f5c1414aa00 2020-12-05T17:38:46.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.183+0000 7f5c1b431700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c1414ae70 con 0x7f5c141462e0 2020-12-05T17:38:46.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.183+0000 7f5c1a42f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5c140b2100 0x7f5c1414a2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39732/0 (socket says 172.21.15.32:39732) 2020-12-05T17:38:46.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.183+0000 7f5c1a42f700 1 -- 172.21.15.32:0/3926301484 learned_addr learned my addr 172.21.15.32:0/3926301484 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:38:46.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.183+0000 7f5c1ac30700 1 --2- 172.21.15.32:0/3926301484 >> v2:172.21.15.32:3300/0 conn(0x7f5c141462e0 0x7f5c141528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:46.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.183+0000 7f5c1942d700 1 -- 172.21.15.32:0/3926301484 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2073146714 0 0) 0x7f5c140affd0 con 0x7f5c140b2100 2020-12-05T17:38:46.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.183+0000 7f5c1942d700 1 -- 172.21.15.32:0/3926301484 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bfc003230 con 0x7f5c140b2100 2020-12-05T17:38:46.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.183+0000 7f5c1942d700 1 -- 172.21.15.32:0/3926301484 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1526763555 0 0) 0x7f5bfc003230 con 0x7f5c140b2100 2020-12-05T17:38:46.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.183+0000 7f5c1ac30700 1 -- 172.21.15.32:0/3926301484 >> v1:172.21.15.61:6789/0 conn(0x7f5c1414aa00 legacy=0x7f5c14145bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:46.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.183+0000 7f5c1ac30700 1 -- 172.21.15.32:0/3926301484 >> v1:172.21.15.32:6789/0 conn(0x7f5c140b2100 legacy=0x7f5c1414a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:46.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.183+0000 7f5c1ac30700 1 -- 172.21.15.32:0/3926301484 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c140afd30 con 0x7f5c141462e0 2020-12-05T17:38:46.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.187+0000 7f5c1ac30700 1 --2- 172.21.15.32:0/3926301484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c141462e0 0x7f5c141528b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f5c10001870 tx=0x7f5c0c0091a0).ready entity=mon.0 client_cookie=86517b028583323 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:46.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.187+0000 7f5c1942d700 1 -- 172.21.15.32:0/3926301484 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c100016d0 con 0x7f5c141462e0 2020-12-05T17:38:46.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.187+0000 7f5c1942d700 1 -- 172.21.15.32:0/3926301484 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0c00b860 con 0x7f5c141462e0 2020-12-05T17:38:46.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.187+0000 7f5c1942d700 1 -- 172.21.15.32:0/3926301484 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c0c00b9d0 con 0x7f5c141462e0 2020-12-05T17:38:46.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.187+0000 7f5c1b431700 1 -- 172.21.15.32:0/3926301484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c141462e0 msgr2=0x7f5c141528b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:46.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.187+0000 7f5c1b431700 1 --2- 172.21.15.32:0/3926301484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c141462e0 0x7f5c141528b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f5c10001870 tx=0x7f5c0c0091a0).stop 2020-12-05T17:38:46.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.187+0000 7f5c1b431700 1 -- 172.21.15.32:0/3926301484 shutdown_connections 2020-12-05T17:38:46.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.187+0000 7f5c1b431700 1 --2- 172.21.15.32:0/3926301484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c141462e0 0x7f5c141528b0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:46.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.187+0000 7f5c1b431700 1 -- 172.21.15.32:0/3926301484 >> 172.21.15.32:0/3926301484 conn(0x7f5c140620b0 msgr2=0x7f5c14062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:46.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.187+0000 7f5c1b431700 1 -- 172.21.15.32:0/3926301484 shutdown_connections 2020-12-05T17:38:46.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.187+0000 7f5c1b431700 1 -- 172.21.15.32:0/3926301484 wait complete. 2020-12-05T17:38:46.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.187+0000 7f5c1b431700 1 Processor -- start 2020-12-05T17:38:46.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.187+0000 7f5c1b431700 1 -- start start 2020-12-05T17:38:46.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c1b431700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c140b2100 0x7f5c140652f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:46.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c1b431700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c141462e0 0x7f5c14066a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:46.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c1b431700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c1414aa00 0x7f5c1410e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:46.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c1b431700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c141450d0 con 0x7f5c140b2100 2020-12-05T17:38:46.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c1b431700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c14063040 con 0x7f5c1414aa00 2020-12-05T17:38:46.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c1b431700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c140632e0 con 0x7f5c141462e0 2020-12-05T17:38:46.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c1a42f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c140b2100 0x7f5c140652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:46.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c1a42f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c140b2100 0x7f5c140652f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37090/0 (socket says 172.21.15.32:37090) 2020-12-05T17:38:46.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c1a42f700 1 -- 172.21.15.32:0/3930401477 learned_addr learned my addr 172.21.15.32:0/3930401477 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:46.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c1ac30700 1 --2- 172.21.15.32:0/3930401477 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c1414aa00 0x7f5c1410e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:46.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c19c2e700 1 --2- 172.21.15.32:0/3930401477 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c141462e0 0x7f5c14066a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:46.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c1a42f700 1 -- 172.21.15.32:0/3930401477 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c141462e0 msgr2=0x7f5c14066a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:46.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c1a42f700 1 --2- 172.21.15.32:0/3930401477 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c141462e0 0x7f5c14066a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:46.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c1a42f700 1 -- 172.21.15.32:0/3930401477 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c1414aa00 msgr2=0x7f5c1410e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:46.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c1a42f700 1 --2- 172.21.15.32:0/3930401477 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c1414aa00 0x7f5c1410e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:46.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c1a42f700 1 -- 172.21.15.32:0/3930401477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c14063580 con 0x7f5c140b2100 2020-12-05T17:38:46.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c1ac30700 1 --2- 172.21.15.32:0/3930401477 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c1414aa00 0x7f5c1410e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:38:46.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c19c2e700 1 --2- 172.21.15.32:0/3930401477 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c141462e0 0x7f5c14066a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:38:46.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c1a42f700 1 --2- 172.21.15.32:0/3930401477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c140b2100 0x7f5c140652f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7f5c10001930 tx=0x7f5c10001330).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:46.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c037fe700 1 -- 172.21.15.32:0/3930401477 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c10007170 con 0x7f5c140b2100 2020-12-05T17:38:46.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c037fe700 1 -- 172.21.15.32:0/3930401477 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c100072d0 con 0x7f5c140b2100 2020-12-05T17:38:46.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c1b431700 1 -- 172.21.15.32:0/3930401477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c14064dc0 con 0x7f5c140b2100 2020-12-05T17:38:46.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c037fe700 1 -- 172.21.15.32:0/3930401477 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c100048f0 con 0x7f5c140b2100 2020-12-05T17:38:46.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c1b431700 1 -- 172.21.15.32:0/3930401477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c14159380 con 0x7f5c140b2100 2020-12-05T17:38:46.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.191+0000 7f5c1b431700 1 -- 172.21.15.32:0/3930401477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c14057050 con 0x7f5c140b2100 2020-12-05T17:38:46.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.195+0000 7f5c037fe700 1 -- 172.21.15.32:0/3930401477 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5c10010070 con 0x7f5c140b2100 2020-12-05T17:38:46.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.195+0000 7f5c037fe700 1 --2- 172.21.15.32:0/3930401477 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5bf4055350 0x7f5bf4057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:46.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.195+0000 7f5c037fe700 1 -- 172.21.15.32:0/3930401477 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f5c1006e450 con 0x7f5c140b2100 2020-12-05T17:38:46.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.195+0000 7f5c19c2e700 1 --2- 172.21.15.32:0/3930401477 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5bf4055350 0x7f5bf4057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:46.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.195+0000 7f5c19c2e700 1 --2- 172.21.15.32:0/3930401477 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5bf4055350 0x7f5bf4057810 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f5c04008000 tx=0x7f5c04009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:46.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.203+0000 7f5c037fe700 1 -- 172.21.15.32:0/3930401477 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5c100445d0 con 0x7f5c140b2100 2020-12-05T17:38:46.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.347+0000 7f5c1b431700 1 -- 172.21.15.32:0/3930401477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f5c1405f8f0 con 0x7f5c140b2100 2020-12-05T17:38:46.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.351+0000 7f5c037fe700 1 -- 172.21.15.32:0/3930401477 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5c10047bf0 con 0x7f5c140b2100 2020-12-05T17:38:46.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.351+0000 7f5c1b431700 1 -- 172.21.15.32:0/3930401477 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5bf4055350 msgr2=0x7f5bf4057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:46.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.351+0000 7f5c1b431700 1 --2- 172.21.15.32:0/3930401477 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5bf4055350 0x7f5bf4057810 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f5c04008000 tx=0x7f5c04009040).stop 2020-12-05T17:38:46.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.351+0000 7f5c1b431700 1 -- 172.21.15.32:0/3930401477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c140b2100 msgr2=0x7f5c140652f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:46.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.351+0000 7f5c1b431700 1 --2- 172.21.15.32:0/3930401477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c140b2100 0x7f5c140652f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7f5c10001930 tx=0x7f5c10001330).stop 2020-12-05T17:38:46.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.351+0000 7f5c1b431700 1 -- 172.21.15.32:0/3930401477 shutdown_connections 2020-12-05T17:38:46.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.351+0000 7f5c1b431700 1 --2- 172.21.15.32:0/3930401477 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c1414aa00 0x7f5c1410e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:46.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.351+0000 7f5c1b431700 1 --2- 172.21.15.32:0/3930401477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c140b2100 0x7f5c140652f0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:46.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.351+0000 7f5c1b431700 1 --2- 172.21.15.32:0/3930401477 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5bf4055350 0x7f5bf4057810 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.351+0000 7f5c1b431700 1 --2- 172.21.15.32:0/3930401477 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c141462e0 0x7f5c14066a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.351+0000 7f5c1b431700 1 -- 172.21.15.32:0/3930401477 >> 172.21.15.32:0/3930401477 conn(0x7f5c140620b0 msgr2=0x7f5c14146c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.351+0000 7f5c1b431700 1 -- 172.21.15.32:0/3930401477 shutdown_connections 2020-12-05T17:38:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.351+0000 7f5c1b431700 1 -- 172.21.15.32:0/3930401477 wait complete. 2020-12-05T17:38:46.371 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:38:46.372 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:38:46.388 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:38:46.404 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:38:46.421 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:38:46.434 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:38:46.452 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:38:46.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:38:46.452 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:38:46.453 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:38:46.470 INFO:teuthology.orchestra.run.smithi036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-12-05T17:38:46.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:38:46.472 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-12-05T17:38:46.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:38:46.473 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:38:46.473 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:38:46.480 INFO:teuthology.orchestra.run.smithi002.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-12-05T17:38:46.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:38:46.482 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-12-05T17:38:46.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:38:46.483 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:38:46.483 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:38:46.490 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:38:46.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:38:46.491 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2020-12-05T17:38:46.492 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-12-05T17:38:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.603+0000 7fe41b9cc700 1 Processor -- start 2020-12-05T17:38:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.603+0000 7fe41b9cc700 1 -- start start 2020-12-05T17:38:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.603+0000 7fe41b9cc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe414061910 0x7fe41405fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.603+0000 7fe41b9cc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4140628e0 con 0x7fe414062500 2020-12-05T17:38:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.603+0000 7fe41b9cc700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4141440f0 con 0x7fe414141e70 2020-12-05T17:38:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.603+0000 7fe41b9cc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe414060260 con 0x7fe414061910 2020-12-05T17:38:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.603+0000 7fe418f67700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe414061910 0x7fe41405fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.603+0000 7fe418f67700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe414061910 0x7fe41405fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37100/0 (socket says 172.21.15.32:37100) 2020-12-05T17:38:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.603+0000 7fe419768700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe414062500 0x7fe414144830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39746/0 (socket says 172.21.15.32:39746) 2020-12-05T17:38:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.607+0000 7fe418f67700 1 -- 172.21.15.32:0/1029026453 learned_addr learned my addr 172.21.15.32:0/1029026453 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:46.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.607+0000 7fe418f67700 1 -- 172.21.15.32:0/1029026453 >> v1:172.21.15.61:6789/0 conn(0x7fe414141e70 legacy=0x7fe414140b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.607+0000 7fe418f67700 1 -- 172.21.15.32:0/1029026453 >> v1:172.21.15.32:6789/0 conn(0x7fe414062500 legacy=0x7fe414144830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.607+0000 7fe418f67700 1 -- 172.21.15.32:0/1029026453 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe414144390 con 0x7fe414061910 2020-12-05T17:38:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.607+0000 7fe418f67700 1 --2- 172.21.15.32:0/1029026453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe414061910 0x7fe41405fd70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7fe408002f30 tx=0x7fe408006e80).ready entity=mon.0 client_cookie=e9a3863fb6fb0ea5 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.607+0000 7fe407fff700 1 -- 172.21.15.32:0/1029026453 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4080097f0 con 0x7fe414061910 2020-12-05T17:38:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.607+0000 7fe407fff700 1 -- 172.21.15.32:0/1029026453 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe408009950 con 0x7fe414061910 2020-12-05T17:38:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.607+0000 7fe407fff700 1 -- 172.21.15.32:0/1029026453 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe408009cb0 con 0x7fe414061910 2020-12-05T17:38:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.607+0000 7fe41b9cc700 1 -- 172.21.15.32:0/1029026453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe414061910 msgr2=0x7fe41405fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.607+0000 7fe41b9cc700 1 --2- 172.21.15.32:0/1029026453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe414061910 0x7fe41405fd70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7fe408002f30 tx=0x7fe408006e80).stop 2020-12-05T17:38:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.607+0000 7fe41b9cc700 1 -- 172.21.15.32:0/1029026453 shutdown_connections 2020-12-05T17:38:46.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.607+0000 7fe41b9cc700 1 --2- 172.21.15.32:0/1029026453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe414061910 0x7fe41405fd70 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:46.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.607+0000 7fe41b9cc700 1 -- 172.21.15.32:0/1029026453 >> 172.21.15.32:0/1029026453 conn(0x7fe4140b57c0 msgr2=0x7fe41405f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:46.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.607+0000 7fe41b9cc700 1 -- 172.21.15.32:0/1029026453 shutdown_connections 2020-12-05T17:38:46.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.607+0000 7fe41b9cc700 1 -- 172.21.15.32:0/1029026453 wait complete. 2020-12-05T17:38:46.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.611+0000 7fe41b9cc700 1 Processor -- start 2020-12-05T17:38:46.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.611+0000 7fe41b9cc700 1 -- start start 2020-12-05T17:38:46.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.611+0000 7fe41b9cc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe414061910 0x7fe4140655c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:46.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.611+0000 7fe41b9cc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe414062500 0x7fe414065d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:46.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.611+0000 7fe41b9cc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe414141e70 0x7fe414143810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:46.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.611+0000 7fe41b9cc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4140650e0 con 0x7fe414141e70 2020-12-05T17:38:46.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.611+0000 7fe41b9cc700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe414153730 con 0x7fe414061910 2020-12-05T17:38:46.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.611+0000 7fe41b9cc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4141539d0 con 0x7fe414062500 2020-12-05T17:38:46.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.611+0000 7fe419f69700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe414141e70 0x7fe414143810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:46.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.611+0000 7fe419f69700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe414141e70 0x7fe414143810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37108/0 (socket says 172.21.15.32:37108) 2020-12-05T17:38:46.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.611+0000 7fe419f69700 1 -- 172.21.15.32:0/560881968 learned_addr learned my addr 172.21.15.32:0/560881968 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:46.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.611+0000 7fe419768700 1 --2- 172.21.15.32:0/560881968 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe414061910 0x7fe4140655c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:46.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.611+0000 7fe418f67700 1 --2- 172.21.15.32:0/560881968 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe414062500 0x7fe414065d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:46.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.611+0000 7fe419f69700 1 -- 172.21.15.32:0/560881968 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe414062500 msgr2=0x7fe414065d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:46.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.611+0000 7fe419f69700 1 --2- 172.21.15.32:0/560881968 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe414062500 0x7fe414065d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:46.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.611+0000 7fe419f69700 1 -- 172.21.15.32:0/560881968 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe414061910 msgr2=0x7fe4140655c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:46.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.611+0000 7fe419f69700 1 --2- 172.21.15.32:0/560881968 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe414061910 0x7fe4140655c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:46.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.611+0000 7fe419f69700 1 -- 172.21.15.32:0/560881968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe41414f690 con 0x7fe414141e70 2020-12-05T17:38:46.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.611+0000 7fe419f69700 1 --2- 172.21.15.32:0/560881968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe414141e70 0x7fe414143810 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7fe414060b40 tx=0x7fe40c007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:46.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.615+0000 7fe4067fc700 1 -- 172.21.15.32:0/560881968 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe40c00a5a0 con 0x7fe414141e70 2020-12-05T17:38:46.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.615+0000 7fe41b9cc700 1 -- 172.21.15.32:0/560881968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe41409e5b0 con 0x7fe414141e70 2020-12-05T17:38:46.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.615+0000 7fe4067fc700 1 -- 172.21.15.32:0/560881968 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40c00a700 con 0x7fe414141e70 2020-12-05T17:38:46.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.615+0000 7fe4067fc700 1 -- 172.21.15.32:0/560881968 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe40c00aa60 con 0x7fe414141e70 2020-12-05T17:38:46.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.615+0000 7fe41b9cc700 1 -- 172.21.15.32:0/560881968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe41414fb90 con 0x7fe414141e70 2020-12-05T17:38:46.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.615+0000 7fe41b9cc700 1 -- 172.21.15.32:0/560881968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe414057050 con 0x7fe414141e70 2020-12-05T17:38:46.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.615+0000 7fe4067fc700 1 -- 172.21.15.32:0/560881968 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe40c019030 con 0x7fe414141e70 2020-12-05T17:38:46.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.615+0000 7fe4067fc700 1 --2- 172.21.15.32:0/560881968 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe3fc055440 0x7fe3fc057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:46.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.615+0000 7fe4067fc700 1 -- 172.21.15.32:0/560881968 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6463+0+0 (secure 0 0 0) 0x7fe40c06d980 con 0x7fe414141e70 2020-12-05T17:38:46.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.615+0000 7fe419768700 1 --2- 172.21.15.32:0/560881968 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe3fc055440 0x7fe3fc057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:46.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.615+0000 7fe419768700 1 --2- 172.21.15.32:0/560881968 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe3fc055440 0x7fe3fc057900 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fe41000d000 tx=0x7fe410007b50).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:46.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.623+0000 7fe4067fc700 1 -- 172.21.15.32:0/560881968 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe40c043970 con 0x7fe414141e70 2020-12-05T17:38:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.763+0000 7fe41b9cc700 1 -- 172.21.15.32:0/560881968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fe41414f880 con 0x7fe414141e70 2020-12-05T17:38:46.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.767+0000 7fe4067fc700 1 -- 172.21.15.32:0/560881968 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v73) v1 ==== 81+0+34115 (secure 0 0 0) 0x7fe40c046f90 con 0x7fe414141e70 2020-12-05T17:38:46.774 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:38:46.774 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:38:46.774 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 73, 2020-12-05T17:38:46.775 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", 2020-12-05T17:38:46.775 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2020-12-05T17:31:20.975238+0000", 2020-12-05T17:38:46.775 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2020-12-05T17:38:32.844799+0000", 2020-12-05T17:38:46.775 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:38:46.776 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:38:46.776 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-12-05T17:38:46.776 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2020-12-05T17:38:46.776 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2020-12-05T17:38:46.776 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2020-12-05T17:38:46.777 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2020-12-05T17:38:46.777 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2020-12-05T17:38:46.777 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2020-12-05T17:38:46.777 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:38:46.778 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2020-12-05T17:38:46.778 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2020-12-05T17:38:46.778 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-12-05T17:38:46.778 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2020-12-05T17:38:46.778 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2020-12-05T17:38:46.779 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 14, 2020-12-05T17:38:46.779 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2020-12-05T17:38:46.779 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2020-12-05T17:38:46.779 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2020-12-05T17:38:46.780 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2020-12-05T17:38:46.780 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2020-12-05T17:38:46.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.780 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2020-12-05T17:38:46.781 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2020-12-05T17:38:46.781 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:24.858732+0000", 2020-12-05T17:38:46.781 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:38:46.781 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:38:46.781 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:38:46.782 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:38:46.782 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:38:46.782 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:38:46.782 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:38:46.783 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:38:46.783 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2020-12-05T17:38:46.783 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2020-12-05T17:38:46.783 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2020-12-05T17:38:46.783 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2020-12-05T17:38:46.784 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2020-12-05T17:38:46.784 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:38:46.784 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:38:46.784 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:38:46.785 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:38:46.785 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:38:46.785 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:38:46.785 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:38:46.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.786 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "16", 2020-12-05T17:38:46.786 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:38:46.786 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:38:46.786 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:38:46.787 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:38:46.787 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:38:46.787 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:38:46.787 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:38:46.787 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:38:46.788 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:38:46.788 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:38:46.788 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:38:46.788 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:38:46.788 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:38:46.789 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:38:46.789 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:38:46.789 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:38:46.789 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:38:46.789 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:38:46.790 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:38:46.790 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:38:46.790 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:38:46.790 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:38:46.791 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:38:46.791 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:38:46.791 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:38:46.791 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:38:46.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.792 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:38:46.792 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:38:46.792 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:38:46.792 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:38:46.793 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:38:46.793 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:38:46.793 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:38:46.793 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:38:46.793 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:38:46.794 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:38:46.794 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:38:46.794 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:38:46.794 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2020-12-05T17:38:46.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.795 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:38:46.795 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2020-12-05T17:38:46.795 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.796 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2020-12-05T17:38:46.796 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2020-12-05T17:38:46.796 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:33.305595+0000", 2020-12-05T17:38:46.796 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:38:46.797 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:38:46.797 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:38:46.797 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:38:46.797 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:38:46.797 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:38:46.798 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:38:46.798 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:38:46.798 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:38:46.798 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:38:46.798 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:38:46.799 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:38:46.799 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:38:46.799 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:38:46.799 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:38:46.800 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:38:46.800 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:38:46.800 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:38:46.800 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:38:46.800 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:38:46.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.801 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "19", 2020-12-05T17:38:46.801 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:38:46.801 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:38:46.801 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:38:46.802 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:38:46.802 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:38:46.802 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:38:46.802 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:38:46.802 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:38:46.803 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:38:46.803 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:38:46.803 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:38:46.803 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:38:46.803 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:38:46.804 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:38:46.804 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:38:46.804 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:38:46.804 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:38:46.805 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:38:46.805 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:38:46.805 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:38:46.805 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:38:46.805 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:38:46.806 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:38:46.806 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:38:46.806 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:38:46.806 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:38:46.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.807 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:38:46.807 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:38:46.807 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:38:46.807 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:38:46.808 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:38:46.808 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:38:46.808 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:38:46.808 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:38:46.808 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:38:46.809 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:38:46.809 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:38:46.809 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:38:46.809 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:38:46.809 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2020-12-05T17:38:46.810 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.810 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 13, 2020-12-05T17:38:46.810 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2020-12-05T17:38:46.811 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:37:45.088140+0000", 2020-12-05T17:38:46.811 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:38:46.811 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:38:46.811 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:38:46.812 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:38:46.812 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:38:46.812 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:38:46.812 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:38:46.813 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:38:46.813 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:38:46.813 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:38:46.813 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:38:46.814 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:38:46.814 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:38:46.814 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:38:46.814 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:38:46.814 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:38:46.815 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:38:46.815 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:38:46.815 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:38:46.815 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:38:46.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.816 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "71", 2020-12-05T17:38:46.816 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:38:46.816 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:38:46.817 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:38:46.817 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:38:46.817 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:38:46.817 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:38:46.817 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:38:46.818 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:38:46.818 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:38:46.818 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:38:46.818 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:38:46.819 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:38:46.819 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:38:46.819 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:38:46.819 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:38:46.819 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:38:46.820 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:38:46.820 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:38:46.820 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:38:46.820 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:38:46.820 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:38:46.821 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:38:46.821 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:38:46.821 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:38:46.821 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:38:46.822 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:38:46.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.822 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:38:46.822 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:38:46.823 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:38:46.823 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:38:46.823 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:38:46.823 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:38:46.823 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:38:46.824 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:38:46.824 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:38:46.824 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:38:46.824 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:38:46.825 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:38:46.825 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2020-12-05T17:38:46.825 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2020-12-05T17:38:46.825 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2020-12-05T17:38:46.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.826 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:38:46.826 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:38:46.826 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2020-12-05T17:38:46.826 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.827 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 14, 2020-12-05T17:38:46.828 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2020-12-05T17:38:46.828 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:37:45.501261+0000", 2020-12-05T17:38:46.828 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:38:46.828 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:38:46.828 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:38:46.829 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:38:46.829 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:38:46.829 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:38:46.829 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:38:46.830 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:38:46.830 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:38:46.830 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:38:46.830 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:38:46.831 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:38:46.831 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:38:46.831 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:38:46.831 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:38:46.832 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:38:46.832 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:38:46.832 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:38:46.832 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:38:46.832 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:38:46.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.833 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "72", 2020-12-05T17:38:46.833 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:38:46.833 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:38:46.833 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:38:46.834 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:38:46.834 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:38:46.834 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:38:46.834 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:38:46.834 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:38:46.835 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:38:46.835 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:38:46.835 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:38:46.835 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:38:46.836 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:38:46.836 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:38:46.836 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:38:46.836 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:38:46.836 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:38:46.837 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:38:46.837 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:38:46.837 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:38:46.837 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:38:46.837 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:38:46.838 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:38:46.838 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:38:46.838 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:38:46.838 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:38:46.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.839 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:38:46.839 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:38:46.839 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:38:46.839 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:38:46.840 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:38:46.840 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:38:46.840 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:38:46.840 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:38:46.841 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:38:46.841 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:38:46.841 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:38:46.841 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:38:46.841 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:38:46.842 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:38:46.842 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2020-12-05T17:38:46.842 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.842 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.843 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.843 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:38:46.843 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2020-12-05T17:38:46.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.843 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:38:46.844 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "b941127b-0a49-4e06-8d11-17794ff8b013", 2020-12-05T17:38:46.844 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:38:46.844 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:38:46.844 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:38:46.845 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:38:46.845 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:38:46.845 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:38:46.845 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:38:46.845 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 69, 2020-12-05T17:38:46.846 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:38:46.846 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:38:46.846 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:38:46.846 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.847 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.847 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2020-12-05T17:38:46.847 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:38:46.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.848 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.848 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2020-12-05T17:38:46.848 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:38:46.849 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.849 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.849 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:38:46.849 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.850 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.850 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2020-12-05T17:38:46.850 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:38:46.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.851 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.851 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2020-12-05T17:38:46.851 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:38:46.852 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.852 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.852 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:38:46.853 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.853 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.853 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2020-12-05T17:38:46.854 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:38:46.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.854 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.854 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2020-12-05T17:38:46.855 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:38:46.855 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.855 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.856 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:38:46.856 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.856 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.857 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2020-12-05T17:38:46.857 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:38:46.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.857 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.858 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2020-12-05T17:38:46.858 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:38:46.858 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.858 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.859 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6803/11526", 2020-12-05T17:38:46.859 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6805/11526", 2020-12-05T17:38:46.859 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6809/11526", 2020-12-05T17:38:46.859 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6807/11526", 2020-12-05T17:38:46.859 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:38:46.860 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:38:46.860 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:38:46.860 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.861 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:38:46.861 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "01bd7ff5-a86b-4e11-ba1e-484c309c0fba", 2020-12-05T17:38:46.861 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:38:46.861 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:38:46.862 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:38:46.862 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:38:46.862 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:38:46.862 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:38:46.863 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:38:46.863 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 60, 2020-12-05T17:38:46.863 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:38:46.863 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:38:46.864 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:38:46.864 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.864 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.865 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2020-12-05T17:38:46.865 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:38:46.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.865 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.866 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2020-12-05T17:38:46.866 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:38:46.866 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.866 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.867 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:38:46.867 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.867 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.868 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2020-12-05T17:38:46.868 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:38:46.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.868 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.869 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2020-12-05T17:38:46.869 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:38:46.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.869 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.870 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:38:46.870 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.870 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.871 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2020-12-05T17:38:46.871 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:38:46.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.871 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.872 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2020-12-05T17:38:46.872 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:38:46.872 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.872 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.873 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:38:46.873 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.873 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.874 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2020-12-05T17:38:46.874 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:38:46.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.874 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.875 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2020-12-05T17:38:46.875 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:38:46.875 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.875 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.876 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6819/11538", 2020-12-05T17:38:46.876 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6821/11538", 2020-12-05T17:38:46.876 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6825/11538", 2020-12-05T17:38:46.876 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6823/11538", 2020-12-05T17:38:46.877 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:38:46.877 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:38:46.877 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:38:46.877 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.878 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:38:46.878 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "94736ef5-64c2-4a97-b963-bd38cddd9c52", 2020-12-05T17:38:46.878 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:38:46.879 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:38:46.879 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:38:46.879 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:38:46.879 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:38:46.880 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:38:46.880 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:38:46.880 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 69, 2020-12-05T17:38:46.880 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:38:46.880 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:38:46.881 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:38:46.881 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.881 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.881 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2020-12-05T17:38:46.882 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:38:46.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.882 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.882 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2020-12-05T17:38:46.883 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:38:46.883 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.883 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.884 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:38:46.884 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.884 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.884 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2020-12-05T17:38:46.885 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:38:46.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.885 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.885 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2020-12-05T17:38:46.886 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:38:46.886 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.886 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.886 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:38:46.887 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.887 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.887 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2020-12-05T17:38:46.888 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:38:46.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.888 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.888 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2020-12-05T17:38:46.889 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:38:46.889 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.889 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.889 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:38:46.890 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.890 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.890 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2020-12-05T17:38:46.890 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:38:46.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.891 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.891 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2020-12-05T17:38:46.891 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:38:46.892 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.892 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.892 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6811/11521", 2020-12-05T17:38:46.893 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6813/11521", 2020-12-05T17:38:46.893 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6817/11521", 2020-12-05T17:38:46.893 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6815/11521", 2020-12-05T17:38:46.893 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:38:46.893 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:38:46.894 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:38:46.894 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.894 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:38:46.895 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "69a1f3ff-fd25-43df-b60c-e32a89c35234", 2020-12-05T17:38:46.895 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:38:46.895 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:38:46.895 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:38:46.896 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:38:46.896 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:38:46.896 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:38:46.896 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:38:46.896 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 69, 2020-12-05T17:38:46.897 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:38:46.897 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:38:46.897 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:38:46.897 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.898 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.898 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2020-12-05T17:38:46.898 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:38:46.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.899 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.899 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2020-12-05T17:38:46.899 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:38:46.899 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.900 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.900 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:38:46.900 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.901 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.901 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2020-12-05T17:38:46.901 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:38:46.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.902 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.902 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2020-12-05T17:38:46.902 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:38:46.902 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.903 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.903 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:38:46.903 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.904 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.904 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6832", 2020-12-05T17:38:46.904 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:38:46.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.905 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.905 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6833", 2020-12-05T17:38:46.905 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:38:46.905 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.906 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.906 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:38:46.906 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.907 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.907 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2020-12-05T17:38:46.907 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:38:46.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.908 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.908 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2020-12-05T17:38:46.908 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:38:46.908 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.908 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.909 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6827/11535", 2020-12-05T17:38:46.909 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6829/11535", 2020-12-05T17:38:46.909 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6833/11535", 2020-12-05T17:38:46.909 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6831/11535", 2020-12-05T17:38:46.910 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:38:46.910 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:38:46.910 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:38:46.910 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.911 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:38:46.911 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c301fe4e-5eab-402c-ae6f-db3e6a7181af", 2020-12-05T17:38:46.911 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:38:46.912 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:38:46.912 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:38:46.912 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:38:46.912 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:38:46.913 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:38:46.913 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:38:46.913 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 68, 2020-12-05T17:38:46.913 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:38:46.913 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:38:46.914 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:38:46.914 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.914 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.915 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6808", 2020-12-05T17:38:46.915 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:38:46.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.915 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.916 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6809", 2020-12-05T17:38:46.916 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:38:46.916 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.916 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.917 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:38:46.917 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.918 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.918 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6810", 2020-12-05T17:38:46.918 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:38:46.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.919 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.919 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6811", 2020-12-05T17:38:46.919 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:38:46.920 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.920 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.920 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:38:46.920 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.921 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.921 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6817", 2020-12-05T17:38:46.921 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:38:46.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.922 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.922 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6819", 2020-12-05T17:38:46.922 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:38:46.923 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.923 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.923 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:38:46.923 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.924 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.924 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6812", 2020-12-05T17:38:46.924 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:38:46.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.925 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.925 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6814", 2020-12-05T17:38:46.925 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:38:46.925 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.926 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.926 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6809/11541", 2020-12-05T17:38:46.926 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6811/11541", 2020-12-05T17:38:46.927 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6819/11541", 2020-12-05T17:38:46.927 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6814/11541", 2020-12-05T17:38:46.927 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:38:46.927 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:38:46.928 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:38:46.928 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.928 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:38:46.929 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "9476eff4-345b-46c5-a473-1d9a89a6d327", 2020-12-05T17:38:46.929 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:38:46.929 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:38:46.929 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:38:46.929 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:38:46.930 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:38:46.930 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:38:46.930 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:38:46.930 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 69, 2020-12-05T17:38:46.931 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:38:46.931 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:38:46.931 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:38:46.931 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.932 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.932 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6800", 2020-12-05T17:38:46.932 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:38:46.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.933 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.933 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6801", 2020-12-05T17:38:46.933 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:38:46.933 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.934 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.934 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:38:46.934 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.935 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.935 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6802", 2020-12-05T17:38:46.935 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:38:46.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.936 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.936 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6803", 2020-12-05T17:38:46.936 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:38:46.936 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.936 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.937 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:38:46.937 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.938 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.938 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6806", 2020-12-05T17:38:46.938 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:38:46.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.939 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.939 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6807", 2020-12-05T17:38:46.939 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:38:46.939 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.940 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.940 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:38:46.940 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.941 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.941 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6804", 2020-12-05T17:38:46.941 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:38:46.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.942 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.942 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6805", 2020-12-05T17:38:46.945 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:38:46.945 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.945 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.946 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6801/11539", 2020-12-05T17:38:46.946 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6803/11539", 2020-12-05T17:38:46.946 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6807/11539", 2020-12-05T17:38:46.946 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6805/11539", 2020-12-05T17:38:46.947 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:38:46.947 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:38:46.947 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:38:46.947 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.948 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:38:46.948 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8dfd8567-9b1c-4d13-9c96-071183e469c2", 2020-12-05T17:38:46.948 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:38:46.948 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:38:46.949 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:38:46.949 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:38:46.949 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:38:46.949 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:38:46.950 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:38:46.950 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 69, 2020-12-05T17:38:46.950 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:38:46.950 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:38:46.950 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:38:46.951 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.951 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.951 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6820", 2020-12-05T17:38:46.951 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:38:46.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.952 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.952 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6822", 2020-12-05T17:38:46.952 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:38:46.953 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.953 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.953 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:38:46.954 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.954 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6823", 2020-12-05T17:38:46.954 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:38:46.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.955 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.955 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6825", 2020-12-05T17:38:46.955 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:38:46.956 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.956 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.956 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:38:46.956 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.957 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.957 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6829", 2020-12-05T17:38:46.957 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:38:46.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.958 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.958 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6830", 2020-12-05T17:38:46.958 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:38:46.959 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.959 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.959 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:38:46.959 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.960 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.960 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6826", 2020-12-05T17:38:46.960 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:38:46.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.961 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.961 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6827", 2020-12-05T17:38:46.961 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:38:46.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.962 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.962 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6822/11542", 2020-12-05T17:38:46.962 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6825/11542", 2020-12-05T17:38:46.963 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6830/11542", 2020-12-05T17:38:46.963 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6827/11542", 2020-12-05T17:38:46.963 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:38:46.963 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:38:46.963 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:38:46.964 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.964 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:38:46.965 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "24d466b3-01dc-4ba2-8b81-85250d163bd2", 2020-12-05T17:38:46.965 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:38:46.965 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:38:46.965 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:38:46.965 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:38:46.966 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:38:46.966 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:38:46.966 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:38:46.966 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 68, 2020-12-05T17:38:46.967 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:38:46.967 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:38:46.967 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:38:46.967 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.968 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.968 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6813", 2020-12-05T17:38:46.968 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:38:46.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.969 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.969 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6815", 2020-12-05T17:38:46.969 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:38:46.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.970 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.970 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:38:46.970 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.971 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.971 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6816", 2020-12-05T17:38:46.971 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:38:46.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.972 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.972 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6818", 2020-12-05T17:38:46.972 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:38:46.972 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.973 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.973 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:38:46.973 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.974 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.974 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6828", 2020-12-05T17:38:46.974 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:38:46.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.975 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.975 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6831", 2020-12-05T17:38:46.975 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:38:46.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.976 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.976 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:38:46.976 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:38:46.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.977 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:38:46.977 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6821", 2020-12-05T17:38:46.977 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:38:46.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.978 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:38:46.978 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6824", 2020-12-05T17:38:46.978 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:38:46.978 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.978 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.979 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6815/11540", 2020-12-05T17:38:46.979 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6818/11540", 2020-12-05T17:38:46.979 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6831/11540", 2020-12-05T17:38:46.980 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6824/11540", 2020-12-05T17:38:46.980 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:38:46.980 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:38:46.980 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:38:46.980 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:46.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.981 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:38:46.981 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2020-12-05T17:38:46.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.981 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:38:46.982 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:38:46.982 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:38:46.982 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:38:46.982 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:38:46.982 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:38:46.983 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.398868+0000", 2020-12-05T17:38:46.983 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:38:46.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.984 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:38:46.984 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:38:46.984 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:38:46.984 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:38:46.984 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:38:46.985 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:38:46.985 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.675748+0000", 2020-12-05T17:38:46.985 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:38:46.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.986 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:38:46.986 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:38:46.986 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:38:46.986 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:38:46.987 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:38:46.987 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:38:46.987 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.620888+0000", 2020-12-05T17:38:46.987 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:38:46.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.988 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:38:46.988 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:38:46.988 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:38:46.989 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:38:46.989 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:38:46.989 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:38:46.989 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.681600+0000", 2020-12-05T17:38:46.989 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:38:46.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.990 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:38:46.990 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:38:46.990 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:38:46.991 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:38:46.991 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:38:46.991 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:38:46.991 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.658769+0000", 2020-12-05T17:38:46.991 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:38:46.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.992 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:38:46.992 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:38:46.992 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:38:46.993 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:38:46.993 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:38:46.993 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:38:46.993 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.630708+0000", 2020-12-05T17:38:46.994 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:38:46.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.994 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:38:46.994 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:38:46.995 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:38:46.995 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:38:46.995 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:38:46.995 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:38:46.995 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.648352+0000", 2020-12-05T17:38:46.996 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:38:46.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:46.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:46.996 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:38:46.997 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:38:46.997 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:38:46.997 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:38:46.997 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:38:46.997 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:38:46.998 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.629399+0000", 2020-12-05T17:38:46.998 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:38:46.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:46.998 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:38:46.999 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2020-12-05T17:38:46.999 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2020-12-05T17:38:46.999 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2020-12-05T17:38:46.999 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2020-12-05T17:38:46.999 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": { 2020-12-05T17:38:47.000 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.36:0/892894983": "2020-12-05T18:38:32.172754+0000" 2020-12-05T17:38:47.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:47.000 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2020-12-05T17:38:47.000 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2020-12-05T17:38:47.000 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2020-12-05T17:38:47.001 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:38:47.001 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2020-12-05T17:38:47.001 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2020-12-05T17:38:47.001 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:38:47.002 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2020-12-05T17:38:47.002 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:38:47.002 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:38:47.002 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2020-12-05T17:38:47.002 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2020-12-05T17:38:47.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:47.003 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2020-12-05T17:38:47.003 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:38:47.003 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:38:47.004 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2020-12-05T17:38:47.004 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:38:47.004 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:38:47.004 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2020-12-05T17:38:47.004 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:47.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:47.005 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2020-12-05T17:38:47.005 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2020-12-05T17:38:47.005 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2020-12-05T17:38:47.005 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2020-12-05T17:38:47.006 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2020-12-05T17:38:47.006 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:38:47.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.771+0000 7fe41b9cc700 1 -- 172.21.15.32:0/560881968 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe3fc055440 msgr2=0x7fe3fc057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:47.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.771+0000 7fe41b9cc700 1 --2- 172.21.15.32:0/560881968 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe3fc055440 0x7fe3fc057900 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fe41000d000 tx=0x7fe410007b50).stop 2020-12-05T17:38:47.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.771+0000 7fe41b9cc700 1 -- 172.21.15.32:0/560881968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe414141e70 msgr2=0x7fe414143810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:47.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.771+0000 7fe41b9cc700 1 --2- 172.21.15.32:0/560881968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe414141e70 0x7fe414143810 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7fe414060b40 tx=0x7fe40c007f70).stop 2020-12-05T17:38:47.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.771+0000 7fe41b9cc700 1 -- 172.21.15.32:0/560881968 shutdown_connections 2020-12-05T17:38:47.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.771+0000 7fe41b9cc700 1 --2- 172.21.15.32:0/560881968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe414141e70 0x7fe414143810 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:47.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.771+0000 7fe41b9cc700 1 --2- 172.21.15.32:0/560881968 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe414061910 0x7fe4140655c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:47.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.771+0000 7fe41b9cc700 1 --2- 172.21.15.32:0/560881968 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe3fc055440 0x7fe3fc057900 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:47.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.771+0000 7fe41b9cc700 1 --2- 172.21.15.32:0/560881968 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe414062500 0x7fe414065d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:47.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.771+0000 7fe41b9cc700 1 -- 172.21.15.32:0/560881968 >> 172.21.15.32:0/560881968 conn(0x7fe4140b57c0 msgr2=0x7fe414147080 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:47.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.771+0000 7fe41b9cc700 1 -- 172.21.15.32:0/560881968 shutdown_connections 2020-12-05T17:38:47.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:46.771+0000 7fe41b9cc700 1 -- 172.21.15.32:0/560881968 wait complete. 2020-12-05T17:38:47.011 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:38:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.131+0000 7ffa689de700 1 Processor -- start 2020-12-05T17:38:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.131+0000 7ffa689de700 1 -- start start 2020-12-05T17:38:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.131+0000 7ffa689de700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ffa640aeeb0 0x7ffa6405fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.131+0000 7ffa689de700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa6413fb70 con 0x7ffa640ad1b0 2020-12-05T17:38:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.131+0000 7ffa689de700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa640af460 con 0x7ffa640aead0 2020-12-05T17:38:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.131+0000 7ffa689de700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa640af700 con 0x7ffa640aeeb0 2020-12-05T17:38:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.131+0000 7ffa61d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ffa640ad1b0 0x7ffa6405f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39764/0 (socket says 172.21.15.32:39764) 2020-12-05T17:38:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.131+0000 7ffa61d9b700 1 -- 172.21.15.32:0/2651180954 learned_addr learned my addr 172.21.15.32:0/2651180954 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:38:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.131+0000 7ffa62d9d700 1 --2- 172.21.15.32:0/2651180954 >> v2:172.21.15.61:3300/0 conn(0x7ffa640aeeb0 0x7ffa6405fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.131+0000 7ffa6159a700 1 -- 172.21.15.32:0/2651180954 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204903812 0 0) 0x7ffa6413fb70 con 0x7ffa640ad1b0 2020-12-05T17:38:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.131+0000 7ffa6159a700 1 -- 172.21.15.32:0/2651180954 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa48003230 con 0x7ffa640ad1b0 2020-12-05T17:38:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.131+0000 7ffa62d9d700 1 -- 172.21.15.32:0/2651180954 >> v1:172.21.15.61:6789/0 conn(0x7ffa640aead0 legacy=0x7ffa641389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.131+0000 7ffa6159a700 1 -- 172.21.15.32:0/2651180954 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2995187496 0 0) 0x7ffa640af460 con 0x7ffa640aead0 2020-12-05T17:38:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.131+0000 7ffa62d9d700 1 -- 172.21.15.32:0/2651180954 >> v1:172.21.15.32:6789/0 conn(0x7ffa640ad1b0 legacy=0x7ffa6405f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.131+0000 7ffa62d9d700 1 -- 172.21.15.32:0/2651180954 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa64139100 con 0x7ffa640aeeb0 2020-12-05T17:38:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.131+0000 7ffa62d9d700 1 --2- 172.21.15.32:0/2651180954 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa640aeeb0 0x7ffa6405fba0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7ffa5c0032b0 tx=0x7ffa5c00a360).ready entity=mon.1 client_cookie=7a9c979d12daf065 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.131+0000 7ffa6159a700 1 -- 172.21.15.32:0/2651180954 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa5c010410 con 0x7ffa640aeeb0 2020-12-05T17:38:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.131+0000 7ffa6159a700 1 -- 172.21.15.32:0/2651180954 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5c010570 con 0x7ffa640aeeb0 2020-12-05T17:38:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.131+0000 7ffa6159a700 1 -- 172.21.15.32:0/2651180954 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa5c0108d0 con 0x7ffa640aeeb0 2020-12-05T17:38:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.131+0000 7ffa689de700 1 -- 172.21.15.32:0/2651180954 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa640aeeb0 msgr2=0x7ffa6405fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.131+0000 7ffa689de700 1 --2- 172.21.15.32:0/2651180954 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa640aeeb0 0x7ffa6405fba0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7ffa5c0032b0 tx=0x7ffa5c00a360).stop 2020-12-05T17:38:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.135+0000 7ffa689de700 1 -- 172.21.15.32:0/2651180954 shutdown_connections 2020-12-05T17:38:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.135+0000 7ffa689de700 1 --2- 172.21.15.32:0/2651180954 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa640aeeb0 0x7ffa6405fba0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.135+0000 7ffa689de700 1 -- 172.21.15.32:0/2651180954 >> 172.21.15.32:0/2651180954 conn(0x7ffa640b5580 msgr2=0x7ffa640b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:47.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.135+0000 7ffa689de700 1 -- 172.21.15.32:0/2651180954 shutdown_connections 2020-12-05T17:38:47.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.135+0000 7ffa689de700 1 -- 172.21.15.32:0/2651180954 wait complete. 2020-12-05T17:38:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.135+0000 7ffa689de700 1 Processor -- start 2020-12-05T17:38:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.135+0000 7ffa689de700 1 -- start start 2020-12-05T17:38:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa689de700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa640ad1b0 0x7ffa640630b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa689de700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa640aead0 0x7ffa64135d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa689de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa640aeeb0 0x7ffa64136280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa689de700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6413ac30 con 0x7ffa640aeeb0 2020-12-05T17:38:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa689de700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa640b2330 con 0x7ffa640aead0 2020-12-05T17:38:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa689de700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6413a8b0 con 0x7ffa640ad1b0 2020-12-05T17:38:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa62d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa640aeeb0 0x7ffa64136280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa62d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa640aeeb0 0x7ffa64136280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37122/0 (socket says 172.21.15.32:37122) 2020-12-05T17:38:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa62d9d700 1 -- 172.21.15.32:0/882344412 learned_addr learned my addr 172.21.15.32:0/882344412 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa6259c700 1 --2- 172.21.15.32:0/882344412 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa640ad1b0 0x7ffa640630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa61d9b700 1 --2- 172.21.15.32:0/882344412 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa640aead0 0x7ffa64135d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa62d9d700 1 -- 172.21.15.32:0/882344412 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa640ad1b0 msgr2=0x7ffa640630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa62d9d700 1 --2- 172.21.15.32:0/882344412 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa640ad1b0 0x7ffa640630b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa62d9d700 1 -- 172.21.15.32:0/882344412 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa640aead0 msgr2=0x7ffa64135d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa62d9d700 1 --2- 172.21.15.32:0/882344412 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa640aead0 0x7ffa64135d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa62d9d700 1 -- 172.21.15.32:0/882344412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa64136770 con 0x7ffa640aeeb0 2020-12-05T17:38:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa6259c700 1 --2- 172.21.15.32:0/882344412 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa640ad1b0 0x7ffa640630b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:38:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa61d9b700 1 --2- 172.21.15.32:0/882344412 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa640aead0 0x7ffa64135d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:38:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa62d9d700 1 --2- 172.21.15.32:0/882344412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa640aeeb0 0x7ffa64136280 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7ffa5c0032b0 tx=0x7ffa6413fd00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa537fe700 1 -- 172.21.15.32:0/882344412 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa5c01a850 con 0x7ffa640aeeb0 2020-12-05T17:38:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa537fe700 1 -- 172.21.15.32:0/882344412 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5c01a9b0 con 0x7ffa640aeeb0 2020-12-05T17:38:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa689de700 1 -- 172.21.15.32:0/882344412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa6413a4b0 con 0x7ffa640aeeb0 2020-12-05T17:38:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa537fe700 1 -- 172.21.15.32:0/882344412 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa5c01ad10 con 0x7ffa640aeeb0 2020-12-05T17:38:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa689de700 1 -- 172.21.15.32:0/882344412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa640b2970 con 0x7ffa640aeeb0 2020-12-05T17:38:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.139+0000 7ffa689de700 1 -- 172.21.15.32:0/882344412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa64136900 con 0x7ffa640aeeb0 2020-12-05T17:38:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.143+0000 7ffa537fe700 1 -- 172.21.15.32:0/882344412 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ffa5c02a070 con 0x7ffa640aeeb0 2020-12-05T17:38:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.143+0000 7ffa537fe700 1 --2- 172.21.15.32:0/882344412 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffa40055330 0x7ffa400577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.143+0000 7ffa537fe700 1 -- 172.21.15.32:0/882344412 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6463+0+0 (secure 0 0 0) 0x7ffa5c06fe50 con 0x7ffa640aeeb0 2020-12-05T17:38:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.143+0000 7ffa6259c700 1 --2- 172.21.15.32:0/882344412 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffa40055330 0x7ffa400577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.143+0000 7ffa6259c700 1 --2- 172.21.15.32:0/882344412 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffa40055330 0x7ffa400577f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7ffa5400b000 tx=0x7ffa54009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.151+0000 7ffa537fe700 1 -- 172.21.15.32:0/882344412 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ffa5c045fd0 con 0x7ffa640aeeb0 2020-12-05T17:38:47.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.339+0000 7ffa689de700 1 -- 172.21.15.32:0/882344412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa6413aa70 con 0x7ffa640aeeb0 2020-12-05T17:38:47.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.339+0000 7ffa537fe700 1 -- 172.21.15.32:0/882344412 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+2137 (secure 0 0 0) 0x7ffa5c0495f0 con 0x7ffa640aeeb0 2020-12-05T17:38:47.347 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:38:47.347 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":32,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5645,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/992428276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":992428276},{"type":"v1","addr":"172.21.15.32:6835","nonce":992428276}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:37:46.495876+0000","modified":"2020-12-05T17:37:47.498743+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6160":{"gid":6160,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.61:6833/4214217205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4214217205},{"type":"v1","addr":"172.21.15.61:6833","nonce":4214217205}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-12-05T17:38:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.343+0000 7ffa689de700 1 -- 172.21.15.32:0/882344412 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffa40055330 msgr2=0x7ffa400577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:47.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.343+0000 7ffa689de700 1 --2- 172.21.15.32:0/882344412 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffa40055330 0x7ffa400577f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7ffa5400b000 tx=0x7ffa54009040).stop 2020-12-05T17:38:47.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.343+0000 7ffa689de700 1 -- 172.21.15.32:0/882344412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa640aeeb0 msgr2=0x7ffa64136280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:47.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.343+0000 7ffa689de700 1 --2- 172.21.15.32:0/882344412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa640aeeb0 0x7ffa64136280 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7ffa5c0032b0 tx=0x7ffa6413fd00).stop 2020-12-05T17:38:47.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.343+0000 7ffa689de700 1 -- 172.21.15.32:0/882344412 shutdown_connections 2020-12-05T17:38:47.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.343+0000 7ffa689de700 1 --2- 172.21.15.32:0/882344412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa640aeeb0 0x7ffa64136280 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:47.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.343+0000 7ffa689de700 1 --2- 172.21.15.32:0/882344412 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffa40055330 0x7ffa400577f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:47.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.343+0000 7ffa689de700 1 --2- 172.21.15.32:0/882344412 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa640ad1b0 0x7ffa640630b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:47.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.343+0000 7ffa689de700 1 --2- 172.21.15.32:0/882344412 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa640aead0 0x7ffa64135d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:47.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.343+0000 7ffa689de700 1 -- 172.21.15.32:0/882344412 >> 172.21.15.32:0/882344412 conn(0x7ffa640b5580 msgr2=0x7ffa64067340 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:47.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.343+0000 7ffa689de700 1 -- 172.21.15.32:0/882344412 shutdown_connections 2020-12-05T17:38:47.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.343+0000 7ffa689de700 1 -- 172.21.15.32:0/882344412 wait complete. 2020-12-05T17:38:47.352 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 32 2020-12-05T17:38:47.361 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-12-05T17:38:47.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7bb4b8d700 1 Processor -- start 2020-12-05T17:38:47.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7bb4b8d700 1 -- start start 2020-12-05T17:38:47.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7bb4b8d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f7bb00af0d0 0x7f7bb0064040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:47.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7bb4b8d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7bb00b2650 0x7f7bb0143b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:47.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7bb4b8d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7bb00b2030 0x7f7bb0144060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:47.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7bb4b8d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb00b1110 con 0x7f7bb00b2030 2020-12-05T17:38:47.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7bb4b8d700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb00b3020 con 0x7f7bb00af0d0 2020-12-05T17:38:47.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7bb4b8d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb00af4b0 con 0x7f7bb00b2650 2020-12-05T17:38:47.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7baffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7bb00b2030 0x7f7bb0144060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:47.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7baffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7bb00b2030 0x7f7bb0144060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37126/0 (socket says 172.21.15.32:37126) 2020-12-05T17:38:47.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7baffff700 1 -- 172.21.15.32:0/40667217 learned_addr learned my addr 172.21.15.32:0/40667217 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:47.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7baf7fe700 1 --2- 172.21.15.32:0/40667217 >> v2:172.21.15.61:3300/0 conn(0x7f7bb00af0d0 0x7f7bb0064040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:47.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7ba6ffd700 1 --2- 172.21.15.32:0/40667217 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7bb00b2650 0x7f7bb0143b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:47.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7baffff700 1 -- 172.21.15.32:0/40667217 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7bb00b2650 msgr2=0x7f7bb0143b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:47.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7baffff700 1 --2- 172.21.15.32:0/40667217 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7bb00b2650 0x7f7bb0143b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:47.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7baffff700 1 -- 172.21.15.32:0/40667217 >> v2:172.21.15.61:3300/0 conn(0x7f7bb00af0d0 msgr2=0x7f7bb0064040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:47.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7baffff700 1 --2- 172.21.15.32:0/40667217 >> v2:172.21.15.61:3300/0 conn(0x7f7bb00af0d0 0x7f7bb0064040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:47.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7baffff700 1 -- 172.21.15.32:0/40667217 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb0144550 con 0x7f7bb00b2030 2020-12-05T17:38:47.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7baffff700 1 --2- 172.21.15.32:0/40667217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb00b2030 0x7f7bb0144060 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f7ba80032b0 tx=0x7f7ba8009270).ready entity=mon.0 client_cookie=d6e1f507bdb75938 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:47.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7baeffd700 1 -- 172.21.15.32:0/40667217 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ba800bc00 con 0x7f7bb00b2030 2020-12-05T17:38:47.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7baeffd700 1 -- 172.21.15.32:0/40667217 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba800bd60 con 0x7f7bb00b2030 2020-12-05T17:38:47.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7baeffd700 1 -- 172.21.15.32:0/40667217 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ba8011400 con 0x7f7bb00b2030 2020-12-05T17:38:47.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7bb4b8d700 1 -- 172.21.15.32:0/40667217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb00b2030 msgr2=0x7f7bb0144060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:47.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7bb4b8d700 1 --2- 172.21.15.32:0/40667217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb00b2030 0x7f7bb0144060 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f7ba80032b0 tx=0x7f7ba8009270).stop 2020-12-05T17:38:47.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7bb4b8d700 1 -- 172.21.15.32:0/40667217 shutdown_connections 2020-12-05T17:38:47.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7bb4b8d700 1 --2- 172.21.15.32:0/40667217 >> v2:172.21.15.61:3300/0 conn(0x7f7bb00af0d0 0x7f7bb0064040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:47.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7bb4b8d700 1 --2- 172.21.15.32:0/40667217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb00b2030 0x7f7bb0144060 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7bb4b8d700 1 --2- 172.21.15.32:0/40667217 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7bb00b2650 0x7f7bb0143b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7bb4b8d700 1 -- 172.21.15.32:0/40667217 >> 172.21.15.32:0/40667217 conn(0x7f7bb005bbb0 msgr2=0x7f7bb0062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7bb4b8d700 1 -- 172.21.15.32:0/40667217 shutdown_connections 2020-12-05T17:38:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.467+0000 7f7bb4b8d700 1 -- 172.21.15.32:0/40667217 wait complete. 2020-12-05T17:38:47.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7bb4b8d700 1 Processor -- start 2020-12-05T17:38:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7bb4b8d700 1 -- start start 2020-12-05T17:38:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7bb4b8d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7bb00af0d0 0x7f7bb013dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7bb4b8d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb00b2030 0x7f7bb0152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7bb4b8d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7bb00b2650 0x7f7bb00b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7bb4b8d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb01519b0 con 0x7f7bb00b2030 2020-12-05T17:38:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7bb4b8d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb0151c50 con 0x7f7bb00b2650 2020-12-05T17:38:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7bb4b8d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb0151ef0 con 0x7f7bb00af0d0 2020-12-05T17:38:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7ba6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb00b2030 0x7f7bb0152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7baf7fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7bb00af0d0 0x7f7bb013dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7ba6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb00b2030 0x7f7bb0152390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37134/0 (socket says 172.21.15.32:37134) 2020-12-05T17:38:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7ba6ffd700 1 -- 172.21.15.32:0/2202428791 learned_addr learned my addr 172.21.15.32:0/2202428791 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7baffff700 1 --2- 172.21.15.32:0/2202428791 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7bb00b2650 0x7f7bb00b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7baffff700 1 -- 172.21.15.32:0/2202428791 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7bb00af0d0 msgr2=0x7f7bb013dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7baffff700 1 --2- 172.21.15.32:0/2202428791 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7bb00af0d0 0x7f7bb013dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7baffff700 1 -- 172.21.15.32:0/2202428791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb00b2030 msgr2=0x7f7bb0152390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7baffff700 1 --2- 172.21.15.32:0/2202428791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb00b2030 0x7f7bb0152390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7baffff700 1 -- 172.21.15.32:0/2202428791 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb0141380 con 0x7f7bb00b2650 2020-12-05T17:38:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7ba6ffd700 1 --2- 172.21.15.32:0/2202428791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb00b2030 0x7f7bb0152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:38:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7baffff700 1 --2- 172.21.15.32:0/2202428791 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7bb00b2650 0x7f7bb00b58f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f7ba80032b0 tx=0x7f7ba8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7bad7fa700 1 -- 172.21.15.32:0/2202428791 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ba801f670 con 0x7f7bb00b2650 2020-12-05T17:38:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7bad7fa700 1 -- 172.21.15.32:0/2202428791 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba801f7d0 con 0x7f7bb00b2650 2020-12-05T17:38:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7bb4b8d700 1 -- 172.21.15.32:0/2202428791 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb0151590 con 0x7f7bb00b2650 2020-12-05T17:38:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7bad7fa700 1 -- 172.21.15.32:0/2202428791 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ba801fb30 con 0x7f7bb00b2650 2020-12-05T17:38:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.471+0000 7f7bb4b8d700 1 -- 172.21.15.32:0/2202428791 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb0154d30 con 0x7f7bb00b2650 2020-12-05T17:38:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.475+0000 7f7bb4b8d700 1 -- 172.21.15.32:0/2202428791 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb0151810 con 0x7f7bb00b2650 2020-12-05T17:38:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.475+0000 7f7bad7fa700 1 -- 172.21.15.32:0/2202428791 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7ba800f070 con 0x7f7bb00b2650 2020-12-05T17:38:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.475+0000 7f7bad7fa700 1 --2- 172.21.15.32:0/2202428791 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7b94055330 0x7f7b940577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.475+0000 7f7baf7fe700 1 --2- 172.21.15.32:0/2202428791 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7b94055330 0x7f7b940577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.475+0000 7f7bad7fa700 1 -- 172.21.15.32:0/2202428791 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f7ba806f200 con 0x7f7bb00b2650 2020-12-05T17:38:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.475+0000 7f7baf7fe700 1 --2- 172.21.15.32:0/2202428791 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7b94055330 0x7f7b940577f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f7ba000c000 tx=0x7f7ba0008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.487+0000 7f7bad7fa700 1 -- 172.21.15.32:0/2202428791 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7bb0151810 con 0x7f7bb00b2650 2020-12-05T17:38:47.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:47.675+0000 7f7bb4b8d700 1 -- 172.21.15.32:0/2202428791 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7bb0151810 con 0x7f7bb00b2650 2020-12-05T17:38:48.357 INFO:tasks.ceph.mon.a.smithi032.stderr:2020-12-05T17:38:48.351+0000 7fdef8136700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-12-05T17:38:48.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.359+0000 7f7bad7fa700 1 -- 172.21.15.32:0/2202428791 <== mon.1 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7bb0151810 con 0x7f7bb00b2650 2020-12-05T17:38:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.363+0000 7f7bb4b8d700 1 -- 172.21.15.32:0/2202428791 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7b94055330 msgr2=0x7f7b940577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.363+0000 7f7bb4b8d700 1 --2- 172.21.15.32:0/2202428791 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7b94055330 0x7f7b940577f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f7ba000c000 tx=0x7f7ba0008040).stop 2020-12-05T17:38:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.363+0000 7f7bb4b8d700 1 -- 172.21.15.32:0/2202428791 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7bb00b2650 msgr2=0x7f7bb00b58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:48.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.363+0000 7f7bb4b8d700 1 --2- 172.21.15.32:0/2202428791 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7bb00b2650 0x7f7bb00b58f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f7ba80032b0 tx=0x7f7ba8016000).stop 2020-12-05T17:38:48.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.363+0000 7f7bb4b8d700 1 -- 172.21.15.32:0/2202428791 shutdown_connections 2020-12-05T17:38:48.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.363+0000 7f7bb4b8d700 1 --2- 172.21.15.32:0/2202428791 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7bb00b2650 0x7f7bb00b58f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:48.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.363+0000 7f7bb4b8d700 1 --2- 172.21.15.32:0/2202428791 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7b94055330 0x7f7b940577f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:48.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.363+0000 7f7bb4b8d700 1 --2- 172.21.15.32:0/2202428791 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7bb00af0d0 0x7f7bb013dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:48.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.363+0000 7f7bb4b8d700 1 --2- 172.21.15.32:0/2202428791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb00b2030 0x7f7bb0152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:48.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.363+0000 7f7bb4b8d700 1 -- 172.21.15.32:0/2202428791 >> 172.21.15.32:0/2202428791 conn(0x7f7bb005bbb0 msgr2=0x7f7bb0141a70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:48.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.363+0000 7f7bb4b8d700 1 -- 172.21.15.32:0/2202428791 shutdown_connections 2020-12-05T17:38:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.363+0000 7f7bb4b8d700 1 -- 172.21.15.32:0/2202428791 wait complete. 2020-12-05T17:38:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-12-05T17:38:48.388 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:38:48.394 INFO:tasks.ceph.mds.b.smithi061.stdout:starting mds.b at 2020-12-05T17:38:48.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb1485dc700 1 Processor -- start 2020-12-05T17:38:48.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb1485dc700 1 -- start start 2020-12-05T17:38:48.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb1485dc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1400af680 0x7fb1400b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:48.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb1485dc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb1400afa60 0x7fb1400b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:48.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb1485dc700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb140066ba0 0x7fb140142b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:48.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb1485dc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1400ae340 con 0x7fb1400af680 2020-12-05T17:38:48.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb1485dc700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb140140070 con 0x7fb140066ba0 2020-12-05T17:38:48.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb1485dc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb140140310 con 0x7fb1400afa60 2020-12-05T17:38:48.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb146378700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1400af680 0x7fb1400b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb146378700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1400af680 0x7fb1400b14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37140/0 (socket says 172.21.15.32:37140) 2020-12-05T17:38:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb146378700 1 -- 172.21.15.32:0/2310791699 learned_addr learned my addr 172.21.15.32:0/2310791699 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb145b77700 1 --2- 172.21.15.32:0/2310791699 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb1400afa60 0x7fb1400b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb146b79700 1 --2- 172.21.15.32:0/2310791699 >> v2:172.21.15.61:3300/0 conn(0x7fb140066ba0 0x7fb140142b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb146378700 1 -- 172.21.15.32:0/2310791699 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb1400afa60 msgr2=0x7fb1400b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:48.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb146378700 1 --2- 172.21.15.32:0/2310791699 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb1400afa60 0x7fb1400b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:48.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb146378700 1 -- 172.21.15.32:0/2310791699 >> v2:172.21.15.61:3300/0 conn(0x7fb140066ba0 msgr2=0x7fb140142b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:48.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb146378700 1 --2- 172.21.15.32:0/2310791699 >> v2:172.21.15.61:3300/0 conn(0x7fb140066ba0 0x7fb140142b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:48.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb146378700 1 -- 172.21.15.32:0/2310791699 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1400678c0 con 0x7fb1400af680 2020-12-05T17:38:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb146378700 1 --2- 172.21.15.32:0/2310791699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1400af680 0x7fb1400b14e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7fb1300032b0 tx=0x7fb130009270).ready entity=mon.0 client_cookie=6b8a3cd4d3184b20 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb145376700 1 -- 172.21.15.32:0/2310791699 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb13000bc00 con 0x7fb1400af680 2020-12-05T17:38:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb145376700 1 -- 172.21.15.32:0/2310791699 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13000bd60 con 0x7fb1400af680 2020-12-05T17:38:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb145376700 1 -- 172.21.15.32:0/2310791699 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb130011400 con 0x7fb1400af680 2020-12-05T17:38:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb1485dc700 1 -- 172.21.15.32:0/2310791699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1400af680 msgr2=0x7fb1400b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:48.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.499+0000 7fb1485dc700 1 --2- 172.21.15.32:0/2310791699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1400af680 0x7fb1400b14e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7fb1300032b0 tx=0x7fb130009270).stop 2020-12-05T17:38:48.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.503+0000 7fb1485dc700 1 -- 172.21.15.32:0/2310791699 shutdown_connections 2020-12-05T17:38:48.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.503+0000 7fb1485dc700 1 --2- 172.21.15.32:0/2310791699 >> v2:172.21.15.61:3300/0 conn(0x7fb140066ba0 0x7fb140142b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:48.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.503+0000 7fb1485dc700 1 --2- 172.21.15.32:0/2310791699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1400af680 0x7fb1400b14e0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:48.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.503+0000 7fb1485dc700 1 --2- 172.21.15.32:0/2310791699 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb1400afa60 0x7fb1400b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:48.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.503+0000 7fb1485dc700 1 -- 172.21.15.32:0/2310791699 >> 172.21.15.32:0/2310791699 conn(0x7fb1400b5580 msgr2=0x7fb1400b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:48.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.503+0000 7fb1485dc700 1 -- 172.21.15.32:0/2310791699 shutdown_connections 2020-12-05T17:38:48.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.503+0000 7fb1485dc700 1 -- 172.21.15.32:0/2310791699 wait complete. 2020-12-05T17:38:48.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.503+0000 7fb1485dc700 1 Processor -- start 2020-12-05T17:38:48.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.503+0000 7fb1485dc700 1 -- start start 2020-12-05T17:38:48.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.503+0000 7fb1485dc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb140066ba0 0x7fb14013f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:48.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.503+0000 7fb1485dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1400af680 0x7fb1400650a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:48.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.503+0000 7fb1485dc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1400afa60 0x7fb140065590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:48.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.503+0000 7fb1485dc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14013a9c0 con 0x7fb1400af680 2020-12-05T17:38:48.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.503+0000 7fb1485dc700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb140064430 con 0x7fb1400afa60 2020-12-05T17:38:48.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.503+0000 7fb1485dc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1400646d0 con 0x7fb140066ba0 2020-12-05T17:38:48.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.503+0000 7fb145b77700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1400af680 0x7fb1400650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:48.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.503+0000 7fb146b79700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1400afa60 0x7fb140065590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:48.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.503+0000 7fb145b77700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1400af680 0x7fb1400650a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37150/0 (socket says 172.21.15.32:37150) 2020-12-05T17:38:48.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.503+0000 7fb145b77700 1 -- 172.21.15.32:0/2011858668 learned_addr learned my addr 172.21.15.32:0/2011858668 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:48.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.503+0000 7fb146378700 1 --2- 172.21.15.32:0/2011858668 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb140066ba0 0x7fb14013f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:48.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.507+0000 7fb145b77700 1 -- 172.21.15.32:0/2011858668 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb140066ba0 msgr2=0x7fb14013f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:48.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.507+0000 7fb145b77700 1 --2- 172.21.15.32:0/2011858668 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb140066ba0 0x7fb14013f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:48.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.507+0000 7fb145b77700 1 -- 172.21.15.32:0/2011858668 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1400afa60 msgr2=0x7fb140065590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:48.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.507+0000 7fb145b77700 1 --2- 172.21.15.32:0/2011858668 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1400afa60 0x7fb140065590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:48.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.507+0000 7fb145b77700 1 -- 172.21.15.32:0/2011858668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb140064970 con 0x7fb1400af680 2020-12-05T17:38:48.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.507+0000 7fb146b79700 1 --2- 172.21.15.32:0/2011858668 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1400afa60 0x7fb140065590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:38:48.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.507+0000 7fb146378700 1 --2- 172.21.15.32:0/2011858668 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb140066ba0 0x7fb14013f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:38:48.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.507+0000 7fb145b77700 1 --2- 172.21.15.32:0/2011858668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1400af680 0x7fb1400650a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7fb13c00a000 tx=0x7fb13c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:48.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.507+0000 7fb1377fe700 1 -- 172.21.15.32:0/2011858668 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb13c00f7e0 con 0x7fb1400af680 2020-12-05T17:38:48.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.507+0000 7fb1377fe700 1 -- 172.21.15.32:0/2011858668 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13c00f940 con 0x7fb1400af680 2020-12-05T17:38:48.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.507+0000 7fb1485dc700 1 -- 172.21.15.32:0/2011858668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb14013f940 con 0x7fb1400af680 2020-12-05T17:38:48.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.507+0000 7fb1377fe700 1 -- 172.21.15.32:0/2011858668 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb13c00fca0 con 0x7fb1400af680 2020-12-05T17:38:48.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.507+0000 7fb1485dc700 1 -- 172.21.15.32:0/2011858668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14013ccb0 con 0x7fb1400af680 2020-12-05T17:38:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.507+0000 7fb1485dc700 1 -- 172.21.15.32:0/2011858668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb140057050 con 0x7fb1400af680 2020-12-05T17:38:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.507+0000 7fb1377fe700 1 -- 172.21.15.32:0/2011858668 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb13c01a030 con 0x7fb1400af680 2020-12-05T17:38:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.507+0000 7fb1377fe700 1 --2- 172.21.15.32:0/2011858668 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb120055440 0x7fb120057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.507+0000 7fb1377fe700 1 -- 172.21.15.32:0/2011858668 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fb13c06dce0 con 0x7fb1400af680 2020-12-05T17:38:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.511+0000 7fb146378700 1 --2- 172.21.15.32:0/2011858668 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb120055440 0x7fb120057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:48.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.511+0000 7fb146378700 1 --2- 172.21.15.32:0/2011858668 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb120055440 0x7fb120057900 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7fb13001a000 tx=0x7fb130016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:48.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.515+0000 7fb1377fe700 1 -- 172.21.15.32:0/2011858668 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb13c042df0 con 0x7fb1400af680 2020-12-05T17:38:48.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.699+0000 7fb1485dc700 1 -- 172.21.15.32:0/2011858668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb140067240 con 0x7fb1400af680 2020-12-05T17:38:48.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.699+0000 7fb1377fe700 1 -- 172.21.15.32:0/2011858668 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+1776 (secure 0 0 0) 0x7fb13c043eb0 con 0x7fb1400af680 2020-12-05T17:38:48.706 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:38:48.706 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5645,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/992428276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":992428276},{"type":"v1","addr":"172.21.15.32:6835","nonce":992428276}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:37:46.495876+0000","modified":"2020-12-05T17:38:48.355529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-12-05T17:38:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.703+0000 7fb1485dc700 1 -- 172.21.15.32:0/2011858668 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb120055440 msgr2=0x7fb120057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:48.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.703+0000 7fb1485dc700 1 --2- 172.21.15.32:0/2011858668 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb120055440 0x7fb120057900 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7fb13001a000 tx=0x7fb130016000).stop 2020-12-05T17:38:48.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.703+0000 7fb1485dc700 1 -- 172.21.15.32:0/2011858668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1400af680 msgr2=0x7fb1400650a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:48.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.703+0000 7fb1485dc700 1 --2- 172.21.15.32:0/2011858668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1400af680 0x7fb1400650a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7fb13c00a000 tx=0x7fb13c008040).stop 2020-12-05T17:38:48.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.703+0000 7fb1485dc700 1 -- 172.21.15.32:0/2011858668 shutdown_connections 2020-12-05T17:38:48.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.703+0000 7fb1485dc700 1 --2- 172.21.15.32:0/2011858668 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1400afa60 0x7fb140065590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:48.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.703+0000 7fb1485dc700 1 --2- 172.21.15.32:0/2011858668 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb120055440 0x7fb120057900 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:48.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.703+0000 7fb1485dc700 1 --2- 172.21.15.32:0/2011858668 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb140066ba0 0x7fb14013f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:48.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.703+0000 7fb1485dc700 1 --2- 172.21.15.32:0/2011858668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1400af680 0x7fb1400650a0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:48.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.703+0000 7fb1485dc700 1 -- 172.21.15.32:0/2011858668 >> 172.21.15.32:0/2011858668 conn(0x7fb1400b5580 msgr2=0x7fb1400b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:48.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.703+0000 7fb1485dc700 1 -- 172.21.15.32:0/2011858668 shutdown_connections 2020-12-05T17:38:48.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.703+0000 7fb1485dc700 1 -- 172.21.15.32:0/2011858668 wait complete. 2020-12-05T17:38:48.713 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 33 2020-12-05T17:38:48.726 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-12-05T17:38:48.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb762b64700 1 Processor -- start 2020-12-05T17:38:48.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb762b64700 1 -- start start 2020-12-05T17:38:48.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb762b64700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb75c0ab710 0x7fb75c0af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:48.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb762b64700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb75c147400 0x7fb75c145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:48.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb762b64700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb75c146a90 con 0x7fb75c0b1fc0 2020-12-05T17:38:48.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb762b64700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb75c061e90 con 0x7fb75c147400 2020-12-05T17:38:48.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb762b64700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c0b0110 con 0x7fb75c0ab710 2020-12-05T17:38:48.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb762363700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb75c147400 0x7fb75c145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:48.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb761b62700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb75c0b1fc0 0x7fb75c061780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39804/0 (socket says 172.21.15.32:39804) 2020-12-05T17:38:48.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb762363700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb75c147400 0x7fb75c145d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37158/0 (socket says 172.21.15.32:37158) 2020-12-05T17:38:48.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb761b62700 1 -- 172.21.15.32:0/2494070180 learned_addr learned my addr 172.21.15.32:0/2494070180 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:38:48.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb761361700 1 --2- 172.21.15.32:0/2494070180 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb75c0ab710 0x7fb75c0af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:48.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb762363700 1 -- 172.21.15.32:0/2494070180 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb75c0ab710 msgr2=0x7fb75c0af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb762363700 1 --2- 172.21.15.32:0/2494070180 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb75c0ab710 0x7fb75c0af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb762363700 1 -- 172.21.15.32:0/2494070180 >> v1:172.21.15.32:6789/0 conn(0x7fb75c0b1fc0 legacy=0x7fb75c061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:48.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb760b60700 1 -- 172.21.15.32:0/2494070180 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4262451796 0 0) 0x7fb75c146a90 con 0x7fb75c0b1fc0 2020-12-05T17:38:48.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb762363700 1 -- 172.21.15.32:0/2494070180 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb75c0b23a0 con 0x7fb75c147400 2020-12-05T17:38:48.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb761361700 1 --2- 172.21.15.32:0/2494070180 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb75c0ab710 0x7fb75c0af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:38:48.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb762363700 1 --2- 172.21.15.32:0/2494070180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb75c147400 0x7fb75c145d10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7fb7540032b0 tx=0x7fb7540091a0).ready entity=mon.0 client_cookie=7b291d6923f14393 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:48.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb760b60700 1 -- 172.21.15.32:0/2494070180 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb75400ba50 con 0x7fb75c147400 2020-12-05T17:38:48.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb760b60700 1 -- 172.21.15.32:0/2494070180 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb75400bbb0 con 0x7fb75c147400 2020-12-05T17:38:48.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb760b60700 1 -- 172.21.15.32:0/2494070180 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb75400bd20 con 0x7fb75c147400 2020-12-05T17:38:48.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb762b64700 1 -- 172.21.15.32:0/2494070180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb75c147400 msgr2=0x7fb75c145d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:48.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb762b64700 1 --2- 172.21.15.32:0/2494070180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb75c147400 0x7fb75c145d10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7fb7540032b0 tx=0x7fb7540091a0).stop 2020-12-05T17:38:48.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb762b64700 1 -- 172.21.15.32:0/2494070180 shutdown_connections 2020-12-05T17:38:48.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb762b64700 1 --2- 172.21.15.32:0/2494070180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb75c147400 0x7fb75c145d10 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:48.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb762b64700 1 --2- 172.21.15.32:0/2494070180 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb75c0ab710 0x7fb75c0af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:48.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb762b64700 1 -- 172.21.15.32:0/2494070180 >> 172.21.15.32:0/2494070180 conn(0x7fb75c0b5530 msgr2=0x7fb75c0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:48.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb762b64700 1 -- 172.21.15.32:0/2494070180 shutdown_connections 2020-12-05T17:38:48.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.835+0000 7fb762b64700 1 -- 172.21.15.32:0/2494070180 wait complete. 2020-12-05T17:38:48.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb762b64700 1 Processor -- start 2020-12-05T17:38:48.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb762b64700 1 -- start start 2020-12-05T17:38:48.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb762b64700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb75c0ab710 0x7fb75c09e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:48.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb762b64700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb75c0b1fc0 0x7fb75c066970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:48.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb762b64700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb75c147400 0x7fb75c0abf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:48.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb762b64700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c0aeba0 con 0x7fb75c147400 2020-12-05T17:38:48.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb762b64700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c0ac470 con 0x7fb75c0ab710 2020-12-05T17:38:48.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb762b64700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c0acf60 con 0x7fb75c0b1fc0 2020-12-05T17:38:48.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb762363700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb75c147400 0x7fb75c0abf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:48.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb762363700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb75c147400 0x7fb75c0abf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37162/0 (socket says 172.21.15.32:37162) 2020-12-05T17:38:48.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb762363700 1 -- 172.21.15.32:0/2262883843 learned_addr learned my addr 172.21.15.32:0/2262883843 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:48.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb761361700 1 --2- 172.21.15.32:0/2262883843 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb75c0b1fc0 0x7fb75c066970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:48.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb761b62700 1 --2- 172.21.15.32:0/2262883843 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb75c0ab710 0x7fb75c09e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:48.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb762363700 1 -- 172.21.15.32:0/2262883843 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb75c0b1fc0 msgr2=0x7fb75c066970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:48.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb762363700 1 --2- 172.21.15.32:0/2262883843 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb75c0b1fc0 0x7fb75c066970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:48.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb762363700 1 -- 172.21.15.32:0/2262883843 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb75c0ab710 msgr2=0x7fb75c09e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:48.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb762363700 1 --2- 172.21.15.32:0/2262883843 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb75c0ab710 0x7fb75c09e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:48.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb762363700 1 -- 172.21.15.32:0/2262883843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb75c0aedd0 con 0x7fb75c147400 2020-12-05T17:38:48.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb761361700 1 --2- 172.21.15.32:0/2262883843 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb75c0b1fc0 0x7fb75c066970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:38:48.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb761b62700 1 --2- 172.21.15.32:0/2262883843 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb75c0ab710 0x7fb75c09e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:38:48.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb762363700 1 --2- 172.21.15.32:0/2262883843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb75c147400 0x7fb75c0abf80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7fb7540032b0 tx=0x7fb754016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:48.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb74affd700 1 -- 172.21.15.32:0/2262883843 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb754010820 con 0x7fb75c147400 2020-12-05T17:38:48.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb74affd700 1 -- 172.21.15.32:0/2262883843 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb754010980 con 0x7fb75c147400 2020-12-05T17:38:48.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb762b64700 1 -- 172.21.15.32:0/2262883843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb75c063d10 con 0x7fb75c147400 2020-12-05T17:38:48.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb74affd700 1 -- 172.21.15.32:0/2262883843 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb754010ce0 con 0x7fb75c147400 2020-12-05T17:38:48.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.839+0000 7fb762b64700 1 -- 172.21.15.32:0/2262883843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb75c0a1e30 con 0x7fb75c147400 2020-12-05T17:38:48.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.843+0000 7fb762b64700 1 -- 172.21.15.32:0/2262883843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb75c057050 con 0x7fb75c147400 2020-12-05T17:38:48.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.843+0000 7fb74affd700 1 -- 172.21.15.32:0/2262883843 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb754027030 con 0x7fb75c147400 2020-12-05T17:38:48.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.843+0000 7fb74affd700 1 --2- 172.21.15.32:0/2262883843 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb73c055330 0x7fb73c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:48.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.843+0000 7fb74affd700 1 -- 172.21.15.32:0/2262883843 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fb75406f4c0 con 0x7fb75c147400 2020-12-05T17:38:48.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.843+0000 7fb761b62700 1 --2- 172.21.15.32:0/2262883843 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb73c055330 0x7fb73c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:48.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.843+0000 7fb761b62700 1 --2- 172.21.15.32:0/2262883843 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb73c055330 0x7fb73c0577f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7fb74c00b000 tx=0x7fb74c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:48.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:48.847+0000 7fb74affd700 1 -- 172.21.15.32:0/2262883843 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb7540455e0 con 0x7fb75c147400 2020-12-05T17:38:49.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.035+0000 7fb762b64700 1 -- 172.21.15.32:0/2262883843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb75c064280 con 0x7fb75c147400 2020-12-05T17:38:49.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.359+0000 7fb74affd700 1 -- 172.21.15.32:0/2262883843 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb754048c00 con 0x7fb75c147400 2020-12-05T17:38:49.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.363+0000 7fb762b64700 1 -- 172.21.15.32:0/2262883843 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb73c055330 msgr2=0x7fb73c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:49.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.363+0000 7fb762b64700 1 --2- 172.21.15.32:0/2262883843 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb73c055330 0x7fb73c0577f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7fb74c00b000 tx=0x7fb74c009040).stop 2020-12-05T17:38:49.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.363+0000 7fb762b64700 1 -- 172.21.15.32:0/2262883843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb75c147400 msgr2=0x7fb75c0abf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:49.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.363+0000 7fb762b64700 1 --2- 172.21.15.32:0/2262883843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb75c147400 0x7fb75c0abf80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7fb7540032b0 tx=0x7fb754016000).stop 2020-12-05T17:38:49.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.363+0000 7fb762b64700 1 -- 172.21.15.32:0/2262883843 shutdown_connections 2020-12-05T17:38:49.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.363+0000 7fb762b64700 1 --2- 172.21.15.32:0/2262883843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb75c147400 0x7fb75c0abf80 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:49.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.363+0000 7fb762b64700 1 --2- 172.21.15.32:0/2262883843 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb75c0ab710 0x7fb75c09e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:49.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.363+0000 7fb762b64700 1 --2- 172.21.15.32:0/2262883843 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb73c055330 0x7fb73c0577f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:49.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.363+0000 7fb762b64700 1 --2- 172.21.15.32:0/2262883843 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb75c0b1fc0 0x7fb75c066970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:49.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.363+0000 7fb762b64700 1 -- 172.21.15.32:0/2262883843 >> 172.21.15.32:0/2262883843 conn(0x7fb75c0b5530 msgr2=0x7fb75c061780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:49.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.363+0000 7fb762b64700 1 -- 172.21.15.32:0/2262883843 shutdown_connections 2020-12-05T17:38:49.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.363+0000 7fb762b64700 1 -- 172.21.15.32:0/2262883843 wait complete. 2020-12-05T17:38:49.389 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-12-05T17:38:49.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.511+0000 7fc34359e700 1 Processor -- start 2020-12-05T17:38:49.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.511+0000 7fc34359e700 1 -- start start 2020-12-05T17:38:49.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc34359e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc3440afd30 0x7fc3440b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:49.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc34359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc3440af5f0 0x7fc3440aed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:49.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc34359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3441468e0 con 0x7fc3440b1dc0 2020-12-05T17:38:49.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc34359e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc344142c90 con 0x7fc3440af5f0 2020-12-05T17:38:49.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc34359e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc344146b10 con 0x7fc3440afd30 2020-12-05T17:38:49.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc342d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc3440af5f0 0x7fc3440aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:49.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc34259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc3440b1dc0 0x7fc3440ae600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39816/0 (socket says 172.21.15.32:39816) 2020-12-05T17:38:49.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc34259c700 1 -- 172.21.15.32:0/2340682975 learned_addr learned my addr 172.21.15.32:0/2340682975 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:38:49.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc341d9b700 1 --2- 172.21.15.32:0/2340682975 >> v2:172.21.15.61:3300/0 conn(0x7fc3440afd30 0x7fc3440b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:49.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc342d9d700 1 -- 172.21.15.32:0/2340682975 >> v2:172.21.15.61:3300/0 conn(0x7fc3440afd30 msgr2=0x7fc3440b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:49.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc342d9d700 1 --2- 172.21.15.32:0/2340682975 >> v2:172.21.15.61:3300/0 conn(0x7fc3440afd30 0x7fc3440b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:49.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc342d9d700 1 -- 172.21.15.32:0/2340682975 >> v1:172.21.15.32:6789/0 conn(0x7fc3440b1dc0 legacy=0x7fc3440ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:49.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc342d9d700 1 -- 172.21.15.32:0/2340682975 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc344146db0 con 0x7fc3440af5f0 2020-12-05T17:38:49.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc342d9d700 1 --2- 172.21.15.32:0/2340682975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3440af5f0 0x7fc3440aed10 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7fc33c0032b0 tx=0x7fc33c0091a0).ready entity=mon.0 client_cookie=e027d6beae6b46a1 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:49.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc34159a700 1 -- 172.21.15.32:0/2340682975 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc33c00b9a0 con 0x7fc3440af5f0 2020-12-05T17:38:49.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc34159a700 1 -- 172.21.15.32:0/2340682975 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33c00bb00 con 0x7fc3440af5f0 2020-12-05T17:38:49.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc34159a700 1 -- 172.21.15.32:0/2340682975 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc33c00bc70 con 0x7fc3440af5f0 2020-12-05T17:38:49.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc34359e700 1 -- 172.21.15.32:0/2340682975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3440af5f0 msgr2=0x7fc3440aed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:49.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc34359e700 1 --2- 172.21.15.32:0/2340682975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3440af5f0 0x7fc3440aed10 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7fc33c0032b0 tx=0x7fc33c0091a0).stop 2020-12-05T17:38:49.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc34359e700 1 -- 172.21.15.32:0/2340682975 shutdown_connections 2020-12-05T17:38:49.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc34359e700 1 --2- 172.21.15.32:0/2340682975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3440af5f0 0x7fc3440aed10 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:49.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc34359e700 1 --2- 172.21.15.32:0/2340682975 >> v2:172.21.15.61:3300/0 conn(0x7fc3440afd30 0x7fc3440b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:49.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc34359e700 1 -- 172.21.15.32:0/2340682975 >> 172.21.15.32:0/2340682975 conn(0x7fc3440620b0 msgr2=0x7fc344062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:49.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc34359e700 1 -- 172.21.15.32:0/2340682975 shutdown_connections 2020-12-05T17:38:49.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.515+0000 7fc34359e700 1 -- 172.21.15.32:0/2340682975 wait complete. 2020-12-05T17:38:49.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc34359e700 1 Processor -- start 2020-12-05T17:38:49.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc34359e700 1 -- start start 2020-12-05T17:38:49.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc34359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3440af5f0 0x7fc344155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:49.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc34359e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc3440afd30 0x7fc344144480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:49.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc34359e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc3440b1dc0 0x7fc34413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:49.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc34359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3441496c0 con 0x7fc3440af5f0 2020-12-05T17:38:49.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc34359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc344154b10 con 0x7fc3440b1dc0 2020-12-05T17:38:49.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc34359e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc344154d40 con 0x7fc3440afd30 2020-12-05T17:38:49.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc34259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3440af5f0 0x7fc344155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:49.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc34259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3440af5f0 0x7fc344155480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37174/0 (socket says 172.21.15.32:37174) 2020-12-05T17:38:49.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc34259c700 1 -- 172.21.15.32:0/2035110064 learned_addr learned my addr 172.21.15.32:0/2035110064 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:49.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc341d9b700 1 --2- 172.21.15.32:0/2035110064 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc3440afd30 0x7fc344144480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:49.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc342d9d700 1 --2- 172.21.15.32:0/2035110064 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc3440b1dc0 0x7fc34413bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:49.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc34259c700 1 -- 172.21.15.32:0/2035110064 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc3440afd30 msgr2=0x7fc344144480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:49.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc34259c700 1 --2- 172.21.15.32:0/2035110064 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc3440afd30 0x7fc344144480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:49.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc34259c700 1 -- 172.21.15.32:0/2035110064 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc3440b1dc0 msgr2=0x7fc34413bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:49.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc34259c700 1 --2- 172.21.15.32:0/2035110064 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc3440b1dc0 0x7fc34413bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:49.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc34259c700 1 -- 172.21.15.32:0/2035110064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc344154fe0 con 0x7fc3440af5f0 2020-12-05T17:38:49.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc34259c700 1 --2- 172.21.15.32:0/2035110064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3440af5f0 0x7fc344155480 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7fc3440b5a80 tx=0x7fc334007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:49.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc33b7fe700 1 -- 172.21.15.32:0/2035110064 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc334007320 con 0x7fc3440af5f0 2020-12-05T17:38:49.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc33b7fe700 1 -- 172.21.15.32:0/2035110064 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc334007480 con 0x7fc3440af5f0 2020-12-05T17:38:49.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc33b7fe700 1 -- 172.21.15.32:0/2035110064 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc334005220 con 0x7fc3440af5f0 2020-12-05T17:38:49.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc34359e700 1 -- 172.21.15.32:0/2035110064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc344155970 con 0x7fc3440af5f0 2020-12-05T17:38:49.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.519+0000 7fc34359e700 1 -- 172.21.15.32:0/2035110064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc344066d00 con 0x7fc3440af5f0 2020-12-05T17:38:49.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.523+0000 7fc33b7fe700 1 -- 172.21.15.32:0/2035110064 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc33401b070 con 0x7fc3440af5f0 2020-12-05T17:38:49.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.523+0000 7fc34359e700 1 -- 172.21.15.32:0/2035110064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc344057050 con 0x7fc3440af5f0 2020-12-05T17:38:49.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.523+0000 7fc33b7fe700 1 --2- 172.21.15.32:0/2035110064 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc3280554e0 0x7fc3280579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:49.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.523+0000 7fc33b7fe700 1 -- 172.21.15.32:0/2035110064 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fc33406d6d0 con 0x7fc3440af5f0 2020-12-05T17:38:49.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.523+0000 7fc341d9b700 1 --2- 172.21.15.32:0/2035110064 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc3280554e0 0x7fc3280579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:49.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.523+0000 7fc341d9b700 1 --2- 172.21.15.32:0/2035110064 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc3280554e0 0x7fc3280579a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fc32c008000 tx=0x7fc32c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:49.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.527+0000 7fc33b7fe700 1 -- 172.21.15.32:0/2035110064 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc3340436e0 con 0x7fc3440af5f0 2020-12-05T17:38:49.677 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:49.671+0000 7fc34359e700 1 -- 172.21.15.32:0/2035110064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc344060c10 con 0x7fc3440af5f0 2020-12-05T17:38:50.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.407+0000 7fc33b7fe700 1 -- 172.21.15.32:0/2035110064 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v75) v1 ==== 185+0+0 (secure 0 0 0) 0x7fc334046d00 con 0x7fc3440af5f0 2020-12-05T17:38:50.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.411+0000 7fc34359e700 1 -- 172.21.15.32:0/2035110064 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc3280554e0 msgr2=0x7fc3280579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:50.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.411+0000 7fc34359e700 1 --2- 172.21.15.32:0/2035110064 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc3280554e0 0x7fc3280579a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fc32c008000 tx=0x7fc32c008040).stop 2020-12-05T17:38:50.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.411+0000 7fc34359e700 1 -- 172.21.15.32:0/2035110064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3440af5f0 msgr2=0x7fc344155480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:50.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.411+0000 7fc34359e700 1 --2- 172.21.15.32:0/2035110064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3440af5f0 0x7fc344155480 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7fc3440b5a80 tx=0x7fc334007c30).stop 2020-12-05T17:38:50.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.411+0000 7fc34359e700 1 -- 172.21.15.32:0/2035110064 shutdown_connections 2020-12-05T17:38:50.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.411+0000 7fc34359e700 1 --2- 172.21.15.32:0/2035110064 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc3440b1dc0 0x7fc34413bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:50.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.411+0000 7fc34359e700 1 --2- 172.21.15.32:0/2035110064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3440af5f0 0x7fc344155480 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:50.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.411+0000 7fc34359e700 1 --2- 172.21.15.32:0/2035110064 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc3280554e0 0x7fc3280579a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:50.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.411+0000 7fc34359e700 1 --2- 172.21.15.32:0/2035110064 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc3440afd30 0x7fc344144480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:50.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.411+0000 7fc34359e700 1 -- 172.21.15.32:0/2035110064 >> 172.21.15.32:0/2035110064 conn(0x7fc3440620b0 msgr2=0x7fc344144fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:50.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.411+0000 7fc34359e700 1 -- 172.21.15.32:0/2035110064 shutdown_connections 2020-12-05T17:38:50.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.411+0000 7fc34359e700 1 -- 172.21.15.32:0/2035110064 wait complete. 2020-12-05T17:38:50.422 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2020-12-05T17:38:50.434 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-12-05T17:38:50.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.555+0000 7f68b759e700 1 Processor -- start 2020-12-05T17:38:50.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.555+0000 7f68b759e700 1 -- start start 2020-12-05T17:38:50.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.555+0000 7f68b759e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f68b80afd30 0x7f68b80b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:50.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.555+0000 7f68b759e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68b80af5f0 0x7f68b80aed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:50.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.555+0000 7f68b759e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b81468e0 con 0x7f68b80b1dc0 2020-12-05T17:38:50.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.555+0000 7f68b759e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b8142c90 con 0x7f68b80afd30 2020-12-05T17:38:50.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.555+0000 7f68b759e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b8146b10 con 0x7f68b80af5f0 2020-12-05T17:38:50.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.555+0000 7f68b6d9d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68b80af5f0 0x7f68b80aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:50.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.555+0000 7f68b5d9b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f68b80afd30 0x7f68b80b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:50.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.555+0000 7f68b6d9d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68b80af5f0 0x7f68b80aed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:33976/0 (socket says 172.21.15.32:33976) 2020-12-05T17:38:50.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.555+0000 7f68b6d9d700 1 -- 172.21.15.32:0/4176784771 learned_addr learned my addr 172.21.15.32:0/4176784771 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:50.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.555+0000 7f68b659c700 1 --1- 172.21.15.32:0/4176784771 >> v1:172.21.15.61:6789/0 conn(0x7f68b80b1dc0 0x7f68b80ae600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:52470/0 (socket says 172.21.15.32:52470) 2020-12-05T17:38:50.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.559+0000 7f68b6d9d700 1 -- 172.21.15.32:0/4176784771 >> v2:172.21.15.61:3300/0 conn(0x7f68b80afd30 msgr2=0x7f68b80b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:50.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.559+0000 7f68b6d9d700 1 --2- 172.21.15.32:0/4176784771 >> v2:172.21.15.61:3300/0 conn(0x7f68b80afd30 0x7f68b80b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:50.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.559+0000 7f68b6d9d700 1 -- 172.21.15.32:0/4176784771 >> v1:172.21.15.61:6789/0 conn(0x7f68b80b1dc0 legacy=0x7f68b80ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:50.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.559+0000 7f68b6d9d700 1 -- 172.21.15.32:0/4176784771 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b8146db0 con 0x7f68b80af5f0 2020-12-05T17:38:50.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.559+0000 7f68b6d9d700 1 --2- 172.21.15.32:0/4176784771 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68b80af5f0 0x7f68b80aed10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f68b0002f30 tx=0x7f68b0006e80).ready entity=mon.2 client_cookie=961866145f4c64ee server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:50.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.559+0000 7f68b559a700 1 -- 172.21.15.32:0/4176784771 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68b00097f0 con 0x7f68b80af5f0 2020-12-05T17:38:50.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.559+0000 7f68b559a700 1 -- 172.21.15.32:0/4176784771 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b0009950 con 0x7f68b80af5f0 2020-12-05T17:38:50.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.559+0000 7f68b559a700 1 -- 172.21.15.32:0/4176784771 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68b0009cb0 con 0x7f68b80af5f0 2020-12-05T17:38:50.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.559+0000 7f68b759e700 1 -- 172.21.15.32:0/4176784771 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68b80af5f0 msgr2=0x7f68b80aed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:50.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.559+0000 7f68b759e700 1 --2- 172.21.15.32:0/4176784771 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68b80af5f0 0x7f68b80aed10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f68b0002f30 tx=0x7f68b0006e80).stop 2020-12-05T17:38:50.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.559+0000 7f68b759e700 1 -- 172.21.15.32:0/4176784771 shutdown_connections 2020-12-05T17:38:50.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.559+0000 7f68b759e700 1 --2- 172.21.15.32:0/4176784771 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68b80af5f0 0x7f68b80aed10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:50.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.559+0000 7f68b759e700 1 --2- 172.21.15.32:0/4176784771 >> v2:172.21.15.61:3300/0 conn(0x7f68b80afd30 0x7f68b80b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:50.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.559+0000 7f68b759e700 1 -- 172.21.15.32:0/4176784771 >> 172.21.15.32:0/4176784771 conn(0x7f68b80620b0 msgr2=0x7f68b8062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:50.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.559+0000 7f68b759e700 1 -- 172.21.15.32:0/4176784771 shutdown_connections 2020-12-05T17:38:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.559+0000 7f68b759e700 1 -- 172.21.15.32:0/4176784771 wait complete. 2020-12-05T17:38:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.563+0000 7f68b759e700 1 Processor -- start 2020-12-05T17:38:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.563+0000 7f68b759e700 1 -- start start 2020-12-05T17:38:50.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.563+0000 7f68b759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68b80af5f0 0x7f68b8063880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:50.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.563+0000 7f68b759e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68b80afd30 0x7f68b80647a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:50.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.563+0000 7f68b759e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68b80b1dc0 0x7f68b8144110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:50.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.563+0000 7f68b759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b8149ac0 con 0x7f68b80af5f0 2020-12-05T17:38:50.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.563+0000 7f68b759e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b8149580 con 0x7f68b80b1dc0 2020-12-05T17:38:50.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.563+0000 7f68b759e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b8149820 con 0x7f68b80afd30 2020-12-05T17:38:50.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.563+0000 7f68b659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68b80af5f0 0x7f68b8063880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:50.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.563+0000 7f68b6d9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68b80b1dc0 0x7f68b8144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:50.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.563+0000 7f68b6d9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68b80b1dc0 0x7f68b8144110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:57084/0 (socket says 172.21.15.32:57084) 2020-12-05T17:38:50.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.563+0000 7f68b6d9d700 1 -- 172.21.15.32:0/3405738342 learned_addr learned my addr 172.21.15.32:0/3405738342 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:50.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.563+0000 7f68b5d9b700 1 --2- 172.21.15.32:0/3405738342 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68b80afd30 0x7f68b80647a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:50.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.563+0000 7f68b659c700 1 -- 172.21.15.32:0/3405738342 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68b80afd30 msgr2=0x7f68b80647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:50.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.563+0000 7f68b659c700 1 --2- 172.21.15.32:0/3405738342 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68b80afd30 0x7f68b80647a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:50.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.563+0000 7f68b659c700 1 -- 172.21.15.32:0/3405738342 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68b80b1dc0 msgr2=0x7f68b8144110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:50.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.563+0000 7f68b659c700 1 --2- 172.21.15.32:0/3405738342 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68b80b1dc0 0x7f68b8144110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:50.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.563+0000 7f68b659c700 1 -- 172.21.15.32:0/3405738342 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b8147ed0 con 0x7f68b80af5f0 2020-12-05T17:38:50.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.563+0000 7f68b659c700 1 --2- 172.21.15.32:0/3405738342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68b80af5f0 0x7f68b8063880 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f68b805c820 tx=0x7f68a8007ed0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:50.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.563+0000 7f68a77fe700 1 -- 172.21.15.32:0/3405738342 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68a800ce20 con 0x7f68b80af5f0 2020-12-05T17:38:50.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.567+0000 7f68a77fe700 1 -- 172.21.15.32:0/3405738342 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a8005ab0 con 0x7f68b80af5f0 2020-12-05T17:38:50.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.567+0000 7f68a77fe700 1 -- 172.21.15.32:0/3405738342 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68a8005e10 con 0x7f68b80af5f0 2020-12-05T17:38:50.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.567+0000 7f68b759e700 1 -- 172.21.15.32:0/3405738342 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b81480c0 con 0x7f68b80af5f0 2020-12-05T17:38:50.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.567+0000 7f68b759e700 1 -- 172.21.15.32:0/3405738342 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b813f0a0 con 0x7f68b80af5f0 2020-12-05T17:38:50.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.567+0000 7f68b759e700 1 -- 172.21.15.32:0/3405738342 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b8057050 con 0x7f68b80af5f0 2020-12-05T17:38:50.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.567+0000 7f68a77fe700 1 -- 172.21.15.32:0/3405738342 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f68a8010030 con 0x7f68b80af5f0 2020-12-05T17:38:50.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.567+0000 7f68a77fe700 1 --2- 172.21.15.32:0/3405738342 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6894055440 0x7f6894057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:50.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.567+0000 7f68a77fe700 1 -- 172.21.15.32:0/3405738342 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6111+0+0 (secure 0 0 0) 0x7f68a806df00 con 0x7f68b80af5f0 2020-12-05T17:38:50.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.567+0000 7f68b5d9b700 1 --2- 172.21.15.32:0/3405738342 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6894055440 0x7f6894057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:50.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.571+0000 7f68b5d9b700 1 --2- 172.21.15.32:0/3405738342 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6894055440 0x7f6894057900 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f68ac008000 tx=0x7f68ac008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:50.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.575+0000 7f68a77fe700 1 -- 172.21.15.32:0/3405738342 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f68a80440d0 con 0x7f68b80af5f0 2020-12-05T17:38:50.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:50.719+0000 7f68b759e700 1 -- 172.21.15.32:0/3405738342 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f68b8060c10 con 0x7f68b80af5f0 2020-12-05T17:38:51.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.411+0000 7f68a77fe700 1 -- 172.21.15.32:0/3405738342 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v76) v1 ==== 173+0+0 (secure 0 0 0) 0x7f68a80476f0 con 0x7f68b80af5f0 2020-12-05T17:38:51.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.415+0000 7f68b759e700 1 -- 172.21.15.32:0/3405738342 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6894055440 msgr2=0x7f6894057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:51.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.415+0000 7f68b759e700 1 --2- 172.21.15.32:0/3405738342 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6894055440 0x7f6894057900 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f68ac008000 tx=0x7f68ac008040).stop 2020-12-05T17:38:51.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.415+0000 7f68b759e700 1 -- 172.21.15.32:0/3405738342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68b80af5f0 msgr2=0x7f68b8063880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:51.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.415+0000 7f68b759e700 1 --2- 172.21.15.32:0/3405738342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68b80af5f0 0x7f68b8063880 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f68b805c820 tx=0x7f68a8007ed0).stop 2020-12-05T17:38:51.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.415+0000 7f68b759e700 1 -- 172.21.15.32:0/3405738342 shutdown_connections 2020-12-05T17:38:51.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.415+0000 7f68b759e700 1 --2- 172.21.15.32:0/3405738342 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68b80b1dc0 0x7f68b8144110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:51.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.415+0000 7f68b759e700 1 --2- 172.21.15.32:0/3405738342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68b80af5f0 0x7f68b8063880 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:51.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.415+0000 7f68b759e700 1 --2- 172.21.15.32:0/3405738342 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6894055440 0x7f6894057900 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:51.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.415+0000 7f68b759e700 1 --2- 172.21.15.32:0/3405738342 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68b80afd30 0x7f68b80647a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:51.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.415+0000 7f68b759e700 1 -- 172.21.15.32:0/3405738342 >> 172.21.15.32:0/3405738342 conn(0x7f68b80620b0 msgr2=0x7f68b80b52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:51.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.415+0000 7f68b759e700 1 -- 172.21.15.32:0/3405738342 shutdown_connections 2020-12-05T17:38:51.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.415+0000 7f68b759e700 1 -- 172.21.15.32:0/3405738342 wait complete. 2020-12-05T17:38:51.424 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2020-12-05T17:38:51.439 INFO:tasks.ceph.mds.a:Restarting daemon 2020-12-05T17:38:51.439 INFO:tasks.ceph.mds.a:Stopping old one... 2020-12-05T17:38:51.440 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-12-05T17:38:51.440 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:38:51.440 INFO:tasks.ceph.mds.b:Restarting daemon 2020-12-05T17:38:51.441 INFO:tasks.ceph.mds.b:Stopping old one... 2020-12-05T17:38:51.441 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-12-05T17:38:51.441 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:38:51.452 INFO:tasks.ceph.mds.a:Stopped 2020-12-05T17:38:51.453 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-12-05T17:38:51.454 INFO:tasks.ceph.mds.b:Stopped 2020-12-05T17:38:51.454 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-12-05T17:38:51.496 INFO:tasks.ceph.mds.a:Started 2020-12-05T17:38:51.497 DEBUG:teuthology.parallel:result is None 2020-12-05T17:38:51.498 INFO:tasks.ceph.mds.b:Started 2020-12-05T17:38:51.498 DEBUG:teuthology.parallel:result is None 2020-12-05T17:38:51.499 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-12-05T17:38:51.574 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:38:51.574 INFO:tasks.ceph.mds.b.smithi061.stdout:starting mds.b at 2020-12-05T17:38:51.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.603+0000 7fef25863700 1 Processor -- start 2020-12-05T17:38:51.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.603+0000 7fef25863700 1 -- start start 2020-12-05T17:38:51.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef25863700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fef200b21c0 0x7fef20061910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:51.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef25863700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fef200aea30 0x7fef20143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:51.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef25863700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef200ab320 con 0x7fef200afe10 2020-12-05T17:38:51.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef25863700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef200afb20 con 0x7fef200b21c0 2020-12-05T17:38:51.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef25863700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef20141ca0 con 0x7fef200aea30 2020-12-05T17:38:51.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef1effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fef200b21c0 0x7fef20061910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:51.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef1effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fef200b21c0 0x7fef20061910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37208/0 (socket says 172.21.15.32:37208) 2020-12-05T17:38:51.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef1effd700 1 -- 172.21.15.32:0/2104207390 learned_addr learned my addr 172.21.15.32:0/2104207390 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:51.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef1f7fe700 1 --2- 172.21.15.32:0/2104207390 >> v2:172.21.15.61:3300/0 conn(0x7fef200aea30 0x7fef20143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:51.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef1effd700 1 -- 172.21.15.32:0/2104207390 >> v2:172.21.15.61:3300/0 conn(0x7fef200aea30 msgr2=0x7fef20143fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:51.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef1effd700 1 --2- 172.21.15.32:0/2104207390 >> v2:172.21.15.61:3300/0 conn(0x7fef200aea30 0x7fef20143fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:51.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef1effd700 1 -- 172.21.15.32:0/2104207390 >> v1:172.21.15.61:6789/0 conn(0x7fef200afe10 legacy=0x7fef200ae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:51.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef1effd700 1 -- 172.21.15.32:0/2104207390 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef200b25a0 con 0x7fef200b21c0 2020-12-05T17:38:51.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef1effd700 1 --2- 172.21.15.32:0/2104207390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef200b21c0 0x7fef20061910 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7fef10002f30 tx=0x7fef10006e80).ready entity=mon.0 client_cookie=fa7355f16a59de server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:51.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef1dffb700 1 -- 172.21.15.32:0/2104207390 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef100097f0 con 0x7fef200b21c0 2020-12-05T17:38:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef1dffb700 1 -- 172.21.15.32:0/2104207390 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef10009950 con 0x7fef200b21c0 2020-12-05T17:38:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef1dffb700 1 -- 172.21.15.32:0/2104207390 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef10009cb0 con 0x7fef200b21c0 2020-12-05T17:38:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef25863700 1 -- 172.21.15.32:0/2104207390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef200b21c0 msgr2=0x7fef20061910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef25863700 1 --2- 172.21.15.32:0/2104207390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef200b21c0 0x7fef20061910 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7fef10002f30 tx=0x7fef10006e80).stop 2020-12-05T17:38:51.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef25863700 1 -- 172.21.15.32:0/2104207390 shutdown_connections 2020-12-05T17:38:51.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef25863700 1 --2- 172.21.15.32:0/2104207390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef200b21c0 0x7fef20061910 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:51.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef25863700 1 --2- 172.21.15.32:0/2104207390 >> v2:172.21.15.61:3300/0 conn(0x7fef200aea30 0x7fef20143fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:51.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef25863700 1 -- 172.21.15.32:0/2104207390 >> 172.21.15.32:0/2104207390 conn(0x7fef200b57a0 msgr2=0x7fef200b1770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:51.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef25863700 1 -- 172.21.15.32:0/2104207390 shutdown_connections 2020-12-05T17:38:51.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef25863700 1 -- 172.21.15.32:0/2104207390 wait complete. 2020-12-05T17:38:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef25863700 1 Processor -- start 2020-12-05T17:38:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.607+0000 7fef25863700 1 -- start start 2020-12-05T17:38:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef25863700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef200aea30 0x7fef2013eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef25863700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef200afe10 0x7fef20144a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef25863700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef200b21c0 0x7fef201426b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef25863700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2009e690 con 0x7fef200b21c0 2020-12-05T17:38:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef25863700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef20141f70 con 0x7fef200afe10 2020-12-05T17:38:51.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef25863700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef20142210 con 0x7fef200aea30 2020-12-05T17:38:51.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef1f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef200b21c0 0x7fef201426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:51.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef1effd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef200aea30 0x7fef2013eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:51.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef1f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef200b21c0 0x7fef201426b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37218/0 (socket says 172.21.15.32:37218) 2020-12-05T17:38:51.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef1f7fe700 1 -- 172.21.15.32:0/708812877 learned_addr learned my addr 172.21.15.32:0/708812877 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:51.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef1e7fc700 1 --2- 172.21.15.32:0/708812877 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef200afe10 0x7fef20144a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:51.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef1f7fe700 1 -- 172.21.15.32:0/708812877 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef200aea30 msgr2=0x7fef2013eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:51.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef1f7fe700 1 --2- 172.21.15.32:0/708812877 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef200aea30 0x7fef2013eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:51.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef1f7fe700 1 -- 172.21.15.32:0/708812877 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef200afe10 msgr2=0x7fef20144a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:51.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef1f7fe700 1 --2- 172.21.15.32:0/708812877 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef200afe10 0x7fef20144a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:51.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef1f7fe700 1 -- 172.21.15.32:0/708812877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2009e330 con 0x7fef200b21c0 2020-12-05T17:38:51.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef1effd700 1 --2- 172.21.15.32:0/708812877 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef200aea30 0x7fef2013eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:38:51.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef1f7fe700 1 --2- 172.21.15.32:0/708812877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef200b21c0 0x7fef201426b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7fef1800a000 tx=0x7fef18008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:51.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef0ffff700 1 -- 172.21.15.32:0/708812877 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef1800b9a0 con 0x7fef200b21c0 2020-12-05T17:38:51.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef0ffff700 1 -- 172.21.15.32:0/708812877 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1800bb00 con 0x7fef200b21c0 2020-12-05T17:38:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef25863700 1 -- 172.21.15.32:0/708812877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef20143720 con 0x7fef200b21c0 2020-12-05T17:38:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef0ffff700 1 -- 172.21.15.32:0/708812877 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef1800be60 con 0x7fef200b21c0 2020-12-05T17:38:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef25863700 1 -- 172.21.15.32:0/708812877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef201450e0 con 0x7fef200b21c0 2020-12-05T17:38:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef25863700 1 -- 172.21.15.32:0/708812877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef201423d0 con 0x7fef200b21c0 2020-12-05T17:38:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef0ffff700 1 -- 172.21.15.32:0/708812877 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fef1801c460 con 0x7fef200b21c0 2020-12-05T17:38:51.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef0ffff700 1 --2- 172.21.15.32:0/708812877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feefc0554e0 0x7feefc0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:51.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef0ffff700 1 -- 172.21.15.32:0/708812877 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5729+0+0 (secure 0 0 0) 0x7fef1806e4e0 con 0x7fef200b21c0 2020-12-05T17:38:51.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.611+0000 7fef1effd700 1 --2- 172.21.15.32:0/708812877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feefc0554e0 0x7feefc0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:51.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.615+0000 7fef1effd700 1 --2- 172.21.15.32:0/708812877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feefc0554e0 0x7feefc0579a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7fef1001c000 tx=0x7fef10015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:51.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.619+0000 7fef0ffff700 1 -- 172.21.15.32:0/708812877 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fef18044830 con 0x7fef200b21c0 2020-12-05T17:38:51.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:51.787+0000 7fef25863700 1 -- 172.21.15.32:0/708812877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fef20057050 con 0x7fef200b21c0 2020-12-05T17:38:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.607+0000 7fef0ffff700 1 -- 172.21.15.32:0/708812877 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v79) v1 ==== 97+0+0 (secure 0 0 0) 0x7fef18047e50 con 0x7fef200b21c0 2020-12-05T17:38:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.611+0000 7fef25863700 1 -- 172.21.15.32:0/708812877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feefc0554e0 msgr2=0x7feefc0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.611+0000 7fef25863700 1 --2- 172.21.15.32:0/708812877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feefc0554e0 0x7feefc0579a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7fef1001c000 tx=0x7fef10015000).stop 2020-12-05T17:38:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.611+0000 7fef25863700 1 -- 172.21.15.32:0/708812877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef200b21c0 msgr2=0x7fef201426b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.611+0000 7fef25863700 1 --2- 172.21.15.32:0/708812877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef200b21c0 0x7fef201426b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7fef1800a000 tx=0x7fef18008040).stop 2020-12-05T17:38:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.611+0000 7fef25863700 1 -- 172.21.15.32:0/708812877 shutdown_connections 2020-12-05T17:38:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.611+0000 7fef25863700 1 --2- 172.21.15.32:0/708812877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef200b21c0 0x7fef201426b0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.611+0000 7fef25863700 1 --2- 172.21.15.32:0/708812877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feefc0554e0 0x7feefc0579a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.611+0000 7fef25863700 1 --2- 172.21.15.32:0/708812877 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef200aea30 0x7fef2013eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.611+0000 7fef25863700 1 --2- 172.21.15.32:0/708812877 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef200afe10 0x7fef20144a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:52.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.611+0000 7fef25863700 1 -- 172.21.15.32:0/708812877 >> 172.21.15.32:0/708812877 conn(0x7fef200b57a0 msgr2=0x7fef200b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:52.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.611+0000 7fef25863700 1 -- 172.21.15.32:0/708812877 shutdown_connections 2020-12-05T17:38:52.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.611+0000 7fef25863700 1 -- 172.21.15.32:0/708812877 wait complete. 2020-12-05T17:38:52.620 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2020-12-05T17:38:52.634 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2020-12-05T17:38:52.634 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-12-05T17:38:52.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f884a669700 1 Processor -- start 2020-12-05T17:38:52.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f884a669700 1 -- start start 2020-12-05T17:38:52.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f884a669700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f884405f3b0 0x7f884405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:52.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f884a669700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f884405feb0 0x7f88440b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:52.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f884a669700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8844142450 con 0x7f8844062400 2020-12-05T17:38:52.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f884a669700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88440b42f0 con 0x7f884405f3b0 2020-12-05T17:38:52.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f884a669700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8844061780 con 0x7f884405feb0 2020-12-05T17:38:52.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f8849667700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f884405f3b0 0x7f884405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:52.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f8849667700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f884405f3b0 0x7f884405f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37224/0 (socket says 172.21.15.32:37224) 2020-12-05T17:38:52.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f8849667700 1 -- 172.21.15.32:0/1561082212 learned_addr learned my addr 172.21.15.32:0/1561082212 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:52.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f8849e68700 1 --2- 172.21.15.32:0/1561082212 >> v2:172.21.15.61:3300/0 conn(0x7f884405feb0 0x7f88440b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:52.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f8849667700 1 -- 172.21.15.32:0/1561082212 >> v2:172.21.15.61:3300/0 conn(0x7f884405feb0 msgr2=0x7f88440b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:52.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f8849667700 1 --2- 172.21.15.32:0/1561082212 >> v2:172.21.15.61:3300/0 conn(0x7f884405feb0 0x7f88440b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:52.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f8849667700 1 -- 172.21.15.32:0/1561082212 >> v1:172.21.15.32:6789/0 conn(0x7f8844062400 legacy=0x7f88440b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:52.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f8849667700 1 -- 172.21.15.32:0/1561082212 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8844061a20 con 0x7f884405f3b0 2020-12-05T17:38:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f8849667700 1 --2- 172.21.15.32:0/1561082212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f884405f3b0 0x7f884405f790 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7f8834002f30 tx=0x7f8834006e80).ready entity=mon.0 client_cookie=72de87a413bddf7a server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f8843fff700 1 -- 172.21.15.32:0/1561082212 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88340097f0 con 0x7f884405f3b0 2020-12-05T17:38:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f8843fff700 1 -- 172.21.15.32:0/1561082212 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8834009950 con 0x7f884405f3b0 2020-12-05T17:38:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f8843fff700 1 -- 172.21.15.32:0/1561082212 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8834009cb0 con 0x7f884405f3b0 2020-12-05T17:38:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f884a669700 1 -- 172.21.15.32:0/1561082212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f884405f3b0 msgr2=0x7f884405f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:52.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f884a669700 1 --2- 172.21.15.32:0/1561082212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f884405f3b0 0x7f884405f790 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7f8834002f30 tx=0x7f8834006e80).stop 2020-12-05T17:38:52.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f884a669700 1 -- 172.21.15.32:0/1561082212 shutdown_connections 2020-12-05T17:38:52.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f884a669700 1 --2- 172.21.15.32:0/1561082212 >> v2:172.21.15.61:3300/0 conn(0x7f884405feb0 0x7f88440b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:52.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f884a669700 1 --2- 172.21.15.32:0/1561082212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f884405f3b0 0x7f884405f790 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:52.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f884a669700 1 -- 172.21.15.32:0/1561082212 >> 172.21.15.32:0/1561082212 conn(0x7f88440b8ee0 msgr2=0x7f88440b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:52.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f884a669700 1 -- 172.21.15.32:0/1561082212 shutdown_connections 2020-12-05T17:38:52.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.747+0000 7f884a669700 1 -- 172.21.15.32:0/1561082212 wait complete. 2020-12-05T17:38:52.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f884a669700 1 Processor -- start 2020-12-05T17:38:52.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f884a669700 1 -- start start 2020-12-05T17:38:52.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f884a669700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f884405f3b0 0x7f8844146060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:52.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f884a669700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f884405feb0 0x7f8844147490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:52.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f884a669700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8844062400 0x7f8844064810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:52.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f884a669700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8844066f00 con 0x7f884405feb0 2020-12-05T17:38:52.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f884a669700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88440630a0 con 0x7f884405f3b0 2020-12-05T17:38:52.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f884a669700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8844063340 con 0x7f8844062400 2020-12-05T17:38:52.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f8848e66700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f884405feb0 0x7f8844147490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:52.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f8849667700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f884405f3b0 0x7f8844146060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:52.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f8849667700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f884405f3b0 0x7f8844146060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:57122/0 (socket says 172.21.15.32:57122) 2020-12-05T17:38:52.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f8849667700 1 -- 172.21.15.32:0/1288659143 learned_addr learned my addr 172.21.15.32:0/1288659143 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:52.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f8848e66700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f884405feb0 0x7f8844147490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37232/0 (socket says 172.21.15.32:37232) 2020-12-05T17:38:52.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f8849e68700 1 --2- 172.21.15.32:0/1288659143 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8844062400 0x7f8844064810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:52.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f8849667700 1 -- 172.21.15.32:0/1288659143 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8844062400 msgr2=0x7f8844064810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:52.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f8849667700 1 --2- 172.21.15.32:0/1288659143 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8844062400 0x7f8844064810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:52.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f8849667700 1 -- 172.21.15.32:0/1288659143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f884405feb0 msgr2=0x7f8844147490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:52.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f8849667700 1 --2- 172.21.15.32:0/1288659143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f884405feb0 0x7f8844147490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:52.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f8849667700 1 -- 172.21.15.32:0/1288659143 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88440635e0 con 0x7f884405f3b0 2020-12-05T17:38:52.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f8849667700 1 --2- 172.21.15.32:0/1288659143 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f884405f3b0 0x7f8844146060 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f8834002f30 tx=0x7f8834015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:52.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f88427fc700 1 -- 172.21.15.32:0/1288659143 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8834011c10 con 0x7f884405f3b0 2020-12-05T17:38:52.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f88427fc700 1 -- 172.21.15.32:0/1288659143 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8834011d70 con 0x7f884405f3b0 2020-12-05T17:38:52.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f884a669700 1 -- 172.21.15.32:0/1288659143 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8844145130 con 0x7f884405f3b0 2020-12-05T17:38:52.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f88427fc700 1 -- 172.21.15.32:0/1288659143 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f883401e650 con 0x7f884405f3b0 2020-12-05T17:38:52.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f884a669700 1 -- 172.21.15.32:0/1288659143 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8844064d00 con 0x7f884405f3b0 2020-12-05T17:38:52.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.751+0000 7f884a669700 1 -- 172.21.15.32:0/1288659143 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8844057050 con 0x7f884405f3b0 2020-12-05T17:38:52.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.755+0000 7f88427fc700 1 -- 172.21.15.32:0/1288659143 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8834025030 con 0x7f884405f3b0 2020-12-05T17:38:52.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.755+0000 7f88427fc700 1 --2- 172.21.15.32:0/1288659143 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f88240553b0 0x7f8824057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:52.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.755+0000 7f88427fc700 1 -- 172.21.15.32:0/1288659143 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f883406ef60 con 0x7f884405f3b0 2020-12-05T17:38:52.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.755+0000 7f8848e66700 1 --2- 172.21.15.32:0/1288659143 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f88240553b0 0x7f8824057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:52.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.755+0000 7f8848e66700 1 --2- 172.21.15.32:0/1288659143 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f88240553b0 0x7f8824057870 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f883800d000 tx=0x7f8838007b40).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:52.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.763+0000 7f88427fc700 1 -- 172.21.15.32:0/1288659143 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8834045440 con 0x7f884405f3b0 2020-12-05T17:38:52.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.959+0000 7f884a669700 1 -- 172.21.15.32:0/1288659143 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8844063770 con 0x7f884405f3b0 2020-12-05T17:38:52.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.959+0000 7f88427fc700 1 -- 172.21.15.32:0/1288659143 <== mon.1 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f8834023020 con 0x7f884405f3b0 2020-12-05T17:38:52.966 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:38:52.967 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:38:52.967 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2020-12-05T17:38:52.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:52.967 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2020-12-05T17:38:52.968 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfpb/aExAAdMKW4yIgC2zhkzyW6l8xug==", 2020-12-05T17:38:52.968 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:38:52.968 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:38:52.968 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:38:52.969 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:38:52.969 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:38:52.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:52.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:52.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:52.970 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2020-12-05T17:38:52.970 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfBKTYGRAAaPkezwmTiAvkUfTuDGDNZA==", 2020-12-05T17:38:52.970 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:38:52.970 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:38:52.970 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:38:52.971 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:38:52.971 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:38:52.971 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:52.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:52.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:52.972 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2020-12-05T17:38:52.972 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHw8tflo5EKhAAwV9D64fAoY3uvXCm+Aksag==", 2020-12-05T17:38:52.972 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:38:52.972 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:38:52.973 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:38:52.973 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:38:52.973 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:52.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:52.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:52.974 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2020-12-05T17:38:52.974 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDKw8tfTjEiERAAkW6mkgApoXwyF49kPjZBGg==", 2020-12-05T17:38:52.974 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:38:52.974 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:38:52.974 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:38:52.975 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:38:52.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:52.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:52.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:52.975 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2020-12-05T17:38:52.976 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDMw8tfLEzBMxAAiJ2NifsOLEkt1e3vmhwSTA==", 2020-12-05T17:38:52.976 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:38:52.976 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:38:52.976 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:38:52.976 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:38:52.977 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:52.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:52.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:52.977 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2020-12-05T17:38:52.977 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPw8tfV+vnGxAAQHvXJRW5X9Vhe/eYJBJ/uQ==", 2020-12-05T17:38:52.978 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:38:52.978 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:38:52.978 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:38:52.978 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:38:52.979 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:52.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:52.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:52.979 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2020-12-05T17:38:52.979 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUw8tfN8wMJxAApc5LY9HGDeIV/o0eil/g4A==", 2020-12-05T17:38:52.980 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:38:52.980 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:38:52.980 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:38:52.980 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:38:52.980 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:52.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:52.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:52.981 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2020-12-05T17:38:52.981 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXw8tfvZ5dDxAAtAfftExkljlBvn7ZSoD/Og==", 2020-12-05T17:38:52.981 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:38:52.982 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:38:52.982 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:38:52.982 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:38:52.982 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:52.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:52.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:52.983 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2020-12-05T17:38:52.983 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDZw8tfhXFzMhAAZXNwNGfKv360Tq4N/DX5Mw==", 2020-12-05T17:38:52.983 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:38:52.984 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:38:52.984 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:38:52.984 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:38:52.984 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:52.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:52.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:52.985 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2020-12-05T17:38:52.985 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDcw8tfROfSGhAA/Y2Y1b4k5zQju0MM7LASeQ==", 2020-12-05T17:38:52.985 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:38:52.985 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:38:52.986 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:38:52.986 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:38:52.986 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:52.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:52.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:52.987 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2020-12-05T17:38:52.987 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfWDp/HxAAmvLPj7C3WDgj0OC2o7lBJQ==", 2020-12-05T17:38:52.987 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:38:52.987 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:38:52.987 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:38:52.988 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:38:52.988 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:52.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:52.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:52.988 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2020-12-05T17:38:52.989 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfugnmIRAAW6+/QGMLkHw+Z97RHpV4kw==", 2020-12-05T17:38:52.989 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:38:52.989 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:38:52.989 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:38:52.989 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:38:52.990 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:52.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:52.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:52.990 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2020-12-05T17:38:52.991 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfojyqJBAAyta+dSy2yGnC1kKe3YOAYw==", 2020-12-05T17:38:52.991 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:38:52.991 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:38:52.991 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:38:52.991 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:38:52.992 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:52.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:52.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:52.995 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2020-12-05T17:38:52.995 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDFw8tfES0PLBAAA1cUV44snAUDF2TyAmXImw==", 2020-12-05T17:38:52.996 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:38:52.996 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:38:52.996 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:38:52.996 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:38:52.996 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:38:52.997 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:52.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:52.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:52.998 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2020-12-05T17:38:52.998 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfDr8mERAAS7bpAQXgW78W8GDq1v1zaA==", 2020-12-05T17:38:52.998 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:38:52.998 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:38:52.998 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:38:52.999 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:38:52.999 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:52.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:52.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:53.000 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2020-12-05T17:38:53.000 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tf3HmcDhAATNKHhH9zgasedaNEzgG0OQ==", 2020-12-05T17:38:53.000 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:38:53.000 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:38:53.001 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:38:53.001 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:38:53.001 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:53.001 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:53.001 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:38:53.002 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:38:53.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.963+0000 7f884a669700 1 -- 172.21.15.32:0/1288659143 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f88240553b0 msgr2=0x7f8824057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:53.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.963+0000 7f884a669700 1 --2- 172.21.15.32:0/1288659143 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f88240553b0 0x7f8824057870 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f883800d000 tx=0x7f8838007b40).stop 2020-12-05T17:38:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.963+0000 7f884a669700 1 -- 172.21.15.32:0/1288659143 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f884405f3b0 msgr2=0x7f8844146060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.963+0000 7f884a669700 1 --2- 172.21.15.32:0/1288659143 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f884405f3b0 0x7f8844146060 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f8834002f30 tx=0x7f8834015000).stop 2020-12-05T17:38:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.963+0000 7f884a669700 1 -- 172.21.15.32:0/1288659143 shutdown_connections 2020-12-05T17:38:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.963+0000 7f884a669700 1 --2- 172.21.15.32:0/1288659143 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f88240553b0 0x7f8824057870 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.963+0000 7f884a669700 1 --2- 172.21.15.32:0/1288659143 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8844062400 0x7f8844064810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.963+0000 7f884a669700 1 --2- 172.21.15.32:0/1288659143 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f884405f3b0 0x7f8844146060 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.963+0000 7f884a669700 1 --2- 172.21.15.32:0/1288659143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f884405feb0 0x7f8844147490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.963+0000 7f884a669700 1 -- 172.21.15.32:0/1288659143 >> 172.21.15.32:0/1288659143 conn(0x7f88440b8ee0 msgr2=0x7f884413f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:53.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.963+0000 7f884a669700 1 -- 172.21.15.32:0/1288659143 shutdown_connections 2020-12-05T17:38:53.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:52.963+0000 7f884a669700 1 -- 172.21.15.32:0/1288659143 wait complete. 2020-12-05T17:38:53.006 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:38:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.115+0000 7ff05759e700 1 Processor -- start 2020-12-05T17:38:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.115+0000 7ff05759e700 1 -- start start 2020-12-05T17:38:53.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.119+0000 7ff05759e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff0580b52f0 0x7ff0580b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:53.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.119+0000 7ff05759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0580ae860 0x7ff0580aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:53.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.119+0000 7ff05759e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0581420f0 con 0x7ff0580b1870 2020-12-05T17:38:53.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.119+0000 7ff05759e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0580ad4f0 con 0x7ff0580ae860 2020-12-05T17:38:53.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.119+0000 7ff05759e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0580b0380 con 0x7ff0580b52f0 2020-12-05T17:38:53.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.119+0000 7ff056d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0580ae860 0x7ff0580aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:53.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.119+0000 7ff05659c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff0580b52f0 0x7ff0580b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:53.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.119+0000 7ff056d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0580ae860 0x7ff0580aecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37242/0 (socket says 172.21.15.32:37242) 2020-12-05T17:38:53.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.119+0000 7ff05659c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff0580b52f0 0x7ff0580b56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:34028/0 (socket says 172.21.15.32:34028) 2020-12-05T17:38:53.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.119+0000 7ff056d9d700 1 -- 172.21.15.32:0/4100757058 learned_addr learned my addr 172.21.15.32:0/4100757058 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:53.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.119+0000 7ff056d9d700 1 -- 172.21.15.32:0/4100757058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff0580b52f0 msgr2=0x7ff0580b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:53.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.119+0000 7ff056d9d700 1 --2- 172.21.15.32:0/4100757058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff0580b52f0 0x7ff0580b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.119+0000 7ff056d9d700 1 -- 172.21.15.32:0/4100757058 >> v1:172.21.15.61:6789/0 conn(0x7ff0580b1870 legacy=0x7ff058138810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:53.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.119+0000 7ff056d9d700 1 -- 172.21.15.32:0/4100757058 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0580b0620 con 0x7ff0580ae860 2020-12-05T17:38:53.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.119+0000 7ff056d9d700 1 --2- 172.21.15.32:0/4100757058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0580ae860 0x7ff0580aecf0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7ff0500032b0 tx=0x7ff050009270).ready entity=mon.0 client_cookie=f778f582ee9cd0d9 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:53.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.119+0000 7ff055d9b700 1 -- 172.21.15.32:0/4100757058 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff050009e60 con 0x7ff0580ae860 2020-12-05T17:38:53.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.119+0000 7ff055d9b700 1 -- 172.21.15.32:0/4100757058 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05000bc00 con 0x7ff0580ae860 2020-12-05T17:38:53.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.119+0000 7ff055d9b700 1 -- 172.21.15.32:0/4100757058 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff05000bd70 con 0x7ff0580ae860 2020-12-05T17:38:53.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.119+0000 7ff05759e700 1 -- 172.21.15.32:0/4100757058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0580ae860 msgr2=0x7ff0580aecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:53.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.119+0000 7ff05759e700 1 --2- 172.21.15.32:0/4100757058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0580ae860 0x7ff0580aecf0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7ff0500032b0 tx=0x7ff050009270).stop 2020-12-05T17:38:53.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.123+0000 7ff05759e700 1 -- 172.21.15.32:0/4100757058 shutdown_connections 2020-12-05T17:38:53.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.123+0000 7ff05759e700 1 --2- 172.21.15.32:0/4100757058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0580ae860 0x7ff0580aecf0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.123+0000 7ff05759e700 1 --2- 172.21.15.32:0/4100757058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff0580b52f0 0x7ff0580b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.123+0000 7ff05759e700 1 -- 172.21.15.32:0/4100757058 >> 172.21.15.32:0/4100757058 conn(0x7ff05805bbb0 msgr2=0x7ff058062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:53.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.123+0000 7ff05759e700 1 -- 172.21.15.32:0/4100757058 shutdown_connections 2020-12-05T17:38:53.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.123+0000 7ff05759e700 1 -- 172.21.15.32:0/4100757058 wait complete. 2020-12-05T17:38:53.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.123+0000 7ff05759e700 1 Processor -- start 2020-12-05T17:38:53.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.123+0000 7ff05759e700 1 -- start start 2020-12-05T17:38:53.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.127+0000 7ff05759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0580ae860 0x7ff058065530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:53.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.127+0000 7ff05759e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff0580b1870 0x7ff058064640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:53.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.127+0000 7ff05759e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff0580b52f0 0x7ff058112060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:53.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.127+0000 7ff05759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05813b010 con 0x7ff0580ae860 2020-12-05T17:38:53.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.127+0000 7ff05759e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05810e880 con 0x7ff0580b1870 2020-12-05T17:38:53.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.127+0000 7ff05759e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff05810eb20 con 0x7ff0580b52f0 2020-12-05T17:38:53.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.127+0000 7ff05659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0580ae860 0x7ff058065530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:53.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.127+0000 7ff05659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0580ae860 0x7ff058065530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37244/0 (socket says 172.21.15.32:37244) 2020-12-05T17:38:53.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.127+0000 7ff05659c700 1 -- 172.21.15.32:0/3789040142 learned_addr learned my addr 172.21.15.32:0/3789040142 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:53.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.127+0000 7ff056d9d700 1 --2- 172.21.15.32:0/3789040142 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff0580b52f0 0x7ff058112060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:53.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.127+0000 7ff04dd9b700 1 --2- 172.21.15.32:0/3789040142 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff0580b1870 0x7ff058064640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:53.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.127+0000 7ff05659c700 1 -- 172.21.15.32:0/3789040142 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff0580b52f0 msgr2=0x7ff058112060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:53.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.127+0000 7ff05659c700 1 --2- 172.21.15.32:0/3789040142 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff0580b52f0 0x7ff058112060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.127+0000 7ff05659c700 1 -- 172.21.15.32:0/3789040142 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff0580b1870 msgr2=0x7ff058064640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:53.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.127+0000 7ff05659c700 1 --2- 172.21.15.32:0/3789040142 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff0580b1870 0x7ff058064640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.127+0000 7ff05659c700 1 -- 172.21.15.32:0/3789040142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05810e570 con 0x7ff0580ae860 2020-12-05T17:38:53.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.127+0000 7ff05659c700 1 --2- 172.21.15.32:0/3789040142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0580ae860 0x7ff058065530 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7ff04800c000 tx=0x7ff048006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:53.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.127+0000 7ff04ffff700 1 -- 172.21.15.32:0/3789040142 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff04800ca10 con 0x7ff0580ae860 2020-12-05T17:38:53.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.127+0000 7ff05759e700 1 -- 172.21.15.32:0/3789040142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff058066df0 con 0x7ff0580ae860 2020-12-05T17:38:53.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.127+0000 7ff04ffff700 1 -- 172.21.15.32:0/3789040142 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04800cb70 con 0x7ff0580ae860 2020-12-05T17:38:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.127+0000 7ff04ffff700 1 -- 172.21.15.32:0/3789040142 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff04800be50 con 0x7ff0580ae860 2020-12-05T17:38:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.127+0000 7ff05759e700 1 -- 172.21.15.32:0/3789040142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05810f810 con 0x7ff0580ae860 2020-12-05T17:38:53.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.127+0000 7ff05759e700 1 -- 172.21.15.32:0/3789040142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff058057050 con 0x7ff0580ae860 2020-12-05T17:38:53.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.131+0000 7ff04ffff700 1 -- 172.21.15.32:0/3789040142 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff04801b400 con 0x7ff0580ae860 2020-12-05T17:38:53.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.131+0000 7ff04ffff700 1 --2- 172.21.15.32:0/3789040142 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff03c0553f0 0x7ff03c0578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:53.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.131+0000 7ff04ffff700 1 -- 172.21.15.32:0/3789040142 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5600+0+0 (secure 0 0 0) 0x7ff04806de70 con 0x7ff0580ae860 2020-12-05T17:38:53.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.131+0000 7ff04dd9b700 1 --2- 172.21.15.32:0/3789040142 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff03c0553f0 0x7ff03c0578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:53.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.131+0000 7ff04dd9b700 1 --2- 172.21.15.32:0/3789040142 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff03c0553f0 0x7ff03c0578b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7ff04000d000 tx=0x7ff040007b60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.135+0000 7ff04ffff700 1 -- 172.21.15.32:0/3789040142 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff048044250 con 0x7ff0580ae860 2020-12-05T17:38:53.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.275+0000 7ff05759e700 1 -- 172.21.15.32:0/3789040142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff058111ce0 con 0x7ff0580ae860 2020-12-05T17:38:53.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.275+0000 7ff04ffff700 1 -- 172.21.15.32:0/3789040142 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+13415 (secure 0 0 0) 0x7ff048047870 con 0x7ff0580ae860 2020-12-05T17:38:53.282 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:38:53.282 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":79,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:38:52.603608+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:38:53.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.279+0000 7ff05759e700 1 -- 172.21.15.32:0/3789040142 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff03c0553f0 msgr2=0x7ff03c0578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:53.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.279+0000 7ff05759e700 1 --2- 172.21.15.32:0/3789040142 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff03c0553f0 0x7ff03c0578b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7ff04000d000 tx=0x7ff040007b60).stop 2020-12-05T17:38:53.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.279+0000 7ff05759e700 1 -- 172.21.15.32:0/3789040142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0580ae860 msgr2=0x7ff058065530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:53.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.279+0000 7ff05759e700 1 --2- 172.21.15.32:0/3789040142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0580ae860 0x7ff058065530 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7ff04800c000 tx=0x7ff048006dc0).stop 2020-12-05T17:38:53.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.279+0000 7ff05759e700 1 -- 172.21.15.32:0/3789040142 shutdown_connections 2020-12-05T17:38:53.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.279+0000 7ff05759e700 1 --2- 172.21.15.32:0/3789040142 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff03c0553f0 0x7ff03c0578b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.279+0000 7ff05759e700 1 --2- 172.21.15.32:0/3789040142 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff0580b52f0 0x7ff058112060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.279+0000 7ff05759e700 1 --2- 172.21.15.32:0/3789040142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0580ae860 0x7ff058065530 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.279+0000 7ff05759e700 1 --2- 172.21.15.32:0/3789040142 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff0580b1870 0x7ff058064640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.279+0000 7ff05759e700 1 -- 172.21.15.32:0/3789040142 >> 172.21.15.32:0/3789040142 conn(0x7ff05805bbb0 msgr2=0x7ff0580aec40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:53.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.279+0000 7ff05759e700 1 -- 172.21.15.32:0/3789040142 shutdown_connections 2020-12-05T17:38:53.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.279+0000 7ff05759e700 1 -- 172.21.15.32:0/3789040142 wait complete. 2020-12-05T17:38:53.296 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-12-05T17:38:53.297 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:38:53.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.415+0000 7fc3454e1700 1 Processor -- start 2020-12-05T17:38:53.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.415+0000 7fc3454e1700 1 -- start start 2020-12-05T17:38:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.415+0000 7fc3454e1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc3400ad280 0x7fc340140400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.415+0000 7fc3454e1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3400ad9b0 con 0x7fc3400b2960 2020-12-05T17:38:53.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.415+0000 7fc3454e1700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc34013f4f0 con 0x7fc3400667f0 2020-12-05T17:38:53.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.415+0000 7fc3454e1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3400b32c0 con 0x7fc3400ad280 2020-12-05T17:38:53.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.419+0000 7fc33f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc3400ad280 0x7fc340140400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:53.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.419+0000 7fc33ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc3400b2960 0x7fc3400660e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39900/0 (socket says 172.21.15.32:39900) 2020-12-05T17:38:53.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.419+0000 7fc33ffff700 1 -- 172.21.15.32:0/532788597 learned_addr learned my addr 172.21.15.32:0/532788597 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:38:53.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.419+0000 7fc33f7fe700 1 -- 172.21.15.32:0/532788597 >> v1:172.21.15.61:6789/0 conn(0x7fc3400667f0 legacy=0x7fc3400ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:53.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.419+0000 7fc33f7fe700 1 -- 172.21.15.32:0/532788597 >> v1:172.21.15.32:6789/0 conn(0x7fc3400b2960 legacy=0x7fc3400660e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:53.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.419+0000 7fc33f7fe700 1 -- 172.21.15.32:0/532788597 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3400b3020 con 0x7fc3400ad280 2020-12-05T17:38:53.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.419+0000 7fc33f7fe700 1 --2- 172.21.15.32:0/532788597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3400ad280 0x7fc340140400 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7fc328002f30 tx=0x7fc328006dc0).ready entity=mon.0 client_cookie=2bec76ca332464be server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:53.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.419+0000 7fc33effd700 1 -- 172.21.15.32:0/532788597 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3280097f0 con 0x7fc3400ad280 2020-12-05T17:38:53.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.419+0000 7fc33effd700 1 -- 172.21.15.32:0/532788597 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc328009950 con 0x7fc3400ad280 2020-12-05T17:38:53.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.419+0000 7fc33effd700 1 -- 172.21.15.32:0/532788597 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc328009cb0 con 0x7fc3400ad280 2020-12-05T17:38:53.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.419+0000 7fc3454e1700 1 -- 172.21.15.32:0/532788597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3400ad280 msgr2=0x7fc340140400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:53.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.419+0000 7fc3454e1700 1 --2- 172.21.15.32:0/532788597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3400ad280 0x7fc340140400 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7fc328002f30 tx=0x7fc328006dc0).stop 2020-12-05T17:38:53.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.419+0000 7fc3454e1700 1 -- 172.21.15.32:0/532788597 shutdown_connections 2020-12-05T17:38:53.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.419+0000 7fc3454e1700 1 --2- 172.21.15.32:0/532788597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3400ad280 0x7fc340140400 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.419+0000 7fc3454e1700 1 -- 172.21.15.32:0/532788597 >> 172.21.15.32:0/532788597 conn(0x7fc340061010 msgr2=0x7fc340061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:53.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.419+0000 7fc3454e1700 1 -- 172.21.15.32:0/532788597 shutdown_connections 2020-12-05T17:38:53.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.419+0000 7fc3454e1700 1 -- 172.21.15.32:0/532788597 wait complete. 2020-12-05T17:38:53.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.419+0000 7fc3454e1700 1 Processor -- start 2020-12-05T17:38:53.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.419+0000 7fc3454e1700 1 -- start start 2020-12-05T17:38:53.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc3454e1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc3400667f0 0x7fc34013b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:53.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc3454e1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3400ad280 0x7fc340151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:53.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc3454e1700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc3400b2960 0x7fc340144540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc3454e1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc340144de0 con 0x7fc3400ad280 2020-12-05T17:38:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc3454e1700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc340141f10 con 0x7fc3400b2960 2020-12-05T17:38:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc3454e1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc340152050 con 0x7fc3400667f0 2020-12-05T17:38:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc33f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3400ad280 0x7fc340151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc33f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3400ad280 0x7fc340151b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37258/0 (socket says 172.21.15.32:37258) 2020-12-05T17:38:53.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc33f7fe700 1 -- 172.21.15.32:0/3741817165 learned_addr learned my addr 172.21.15.32:0/3741817165 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:53.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc344ce0700 1 --2- 172.21.15.32:0/3741817165 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc3400b2960 0x7fc340144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:53.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc33ffff700 1 --2- 172.21.15.32:0/3741817165 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc3400667f0 0x7fc34013b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:53.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc344ce0700 1 -- 172.21.15.32:0/3741817165 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc3400667f0 msgr2=0x7fc34013b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:53.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc344ce0700 1 --2- 172.21.15.32:0/3741817165 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc3400667f0 0x7fc34013b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc344ce0700 1 -- 172.21.15.32:0/3741817165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3400ad280 msgr2=0x7fc340151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:53.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc344ce0700 1 --2- 172.21.15.32:0/3741817165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3400ad280 0x7fc340151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc344ce0700 1 -- 172.21.15.32:0/3741817165 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc340144a30 con 0x7fc3400b2960 2020-12-05T17:38:53.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc33f7fe700 1 --2- 172.21.15.32:0/3741817165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3400ad280 0x7fc340151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:38:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc344ce0700 1 --2- 172.21.15.32:0/3741817165 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc3400b2960 0x7fc340144540 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fc3400621a0 tx=0x7fc338007e80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc33d7fa700 1 -- 172.21.15.32:0/3741817165 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc33800a5a0 con 0x7fc3400b2960 2020-12-05T17:38:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc33d7fa700 1 -- 172.21.15.32:0/3741817165 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33800a700 con 0x7fc3400b2960 2020-12-05T17:38:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc3454e1700 1 -- 172.21.15.32:0/3741817165 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc340143850 con 0x7fc3400b2960 2020-12-05T17:38:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc33d7fa700 1 -- 172.21.15.32:0/3741817165 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc33800aa60 con 0x7fc3400b2960 2020-12-05T17:38:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc3454e1700 1 -- 172.21.15.32:0/3741817165 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3401425d0 con 0x7fc3400b2960 2020-12-05T17:38:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.423+0000 7fc3454e1700 1 -- 172.21.15.32:0/3741817165 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc340141d60 con 0x7fc3400b2960 2020-12-05T17:38:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.427+0000 7fc33d7fa700 1 -- 172.21.15.32:0/3741817165 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc338019030 con 0x7fc3400b2960 2020-12-05T17:38:53.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.427+0000 7fc33d7fa700 1 --2- 172.21.15.32:0/3741817165 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc3240554e0 0x7fc3240579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:53.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.427+0000 7fc33d7fa700 1 -- 172.21.15.32:0/3741817165 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fc33806d470 con 0x7fc3400b2960 2020-12-05T17:38:53.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.427+0000 7fc33ffff700 1 --2- 172.21.15.32:0/3741817165 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc3240554e0 0x7fc3240579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.427+0000 7fc33ffff700 1 --2- 172.21.15.32:0/3741817165 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc3240554e0 0x7fc3240579a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7fc33000d000 tx=0x7fc330009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:53.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.435+0000 7fc33d7fa700 1 -- 172.21.15.32:0/3741817165 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc338043950 con 0x7fc3400b2960 2020-12-05T17:38:53.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.575+0000 7fc3454e1700 1 -- 172.21.15.32:0/3741817165 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc340142b90 con 0x7fc3400b2960 2020-12-05T17:38:53.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.579+0000 7fc33d7fa700 1 -- 172.21.15.32:0/3741817165 <== mon.1 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v79) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc338046f70 con 0x7fc3400b2960 2020-12-05T17:38:53.584 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:38:53.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.579+0000 7fc3454e1700 1 -- 172.21.15.32:0/3741817165 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc3240554e0 msgr2=0x7fc3240579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:53.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.579+0000 7fc3454e1700 1 --2- 172.21.15.32:0/3741817165 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc3240554e0 0x7fc3240579a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7fc33000d000 tx=0x7fc330009040).stop 2020-12-05T17:38:53.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.579+0000 7fc3454e1700 1 -- 172.21.15.32:0/3741817165 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc3400b2960 msgr2=0x7fc340144540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:53.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.579+0000 7fc3454e1700 1 --2- 172.21.15.32:0/3741817165 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc3400b2960 0x7fc340144540 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fc3400621a0 tx=0x7fc338007e80).stop 2020-12-05T17:38:53.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.579+0000 7fc3454e1700 1 -- 172.21.15.32:0/3741817165 shutdown_connections 2020-12-05T17:38:53.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.579+0000 7fc3454e1700 1 --2- 172.21.15.32:0/3741817165 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc3400b2960 0x7fc340144540 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.579+0000 7fc3454e1700 1 --2- 172.21.15.32:0/3741817165 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc3240554e0 0x7fc3240579a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.579+0000 7fc3454e1700 1 --2- 172.21.15.32:0/3741817165 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc3400667f0 0x7fc34013b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.579+0000 7fc3454e1700 1 --2- 172.21.15.32:0/3741817165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3400ad280 0x7fc340151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.579+0000 7fc3454e1700 1 -- 172.21.15.32:0/3741817165 >> 172.21.15.32:0/3741817165 conn(0x7fc340061010 msgr2=0x7fc3400b4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:53.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.579+0000 7fc3454e1700 1 -- 172.21.15.32:0/3741817165 shutdown_connections 2020-12-05T17:38:53.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.579+0000 7fc3454e1700 1 -- 172.21.15.32:0/3741817165 wait complete. 2020-12-05T17:38:53.597 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:38:53.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.711+0000 7f675f472700 1 Processor -- start 2020-12-05T17:38:53.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.711+0000 7f675f472700 1 -- start start 2020-12-05T17:38:53.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.711+0000 7f675f472700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f675805f3b0 0x7f675805f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:53.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.711+0000 7f675f472700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f675805feb0 0x7f6758061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:53.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.711+0000 7f675f472700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f67580b2fb0 0x7f675813c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:53.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.711+0000 7f675f472700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67580b2df0 con 0x7f675805f3b0 2020-12-05T17:38:53.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.711+0000 7f675f472700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67580b3c90 con 0x7f67580b2fb0 2020-12-05T17:38:53.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.711+0000 7f675f472700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67580b2b50 con 0x7f675805feb0 2020-12-05T17:38:53.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.711+0000 7f675e470700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f675805f3b0 0x7f675805f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:53.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.711+0000 7f675e470700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f675805f3b0 0x7f675805f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37264/0 (socket says 172.21.15.32:37264) 2020-12-05T17:38:53.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.711+0000 7f675e470700 1 -- 172.21.15.32:0/3173522538 learned_addr learned my addr 172.21.15.32:0/3173522538 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:53.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.711+0000 7f675dc6f700 1 --2- 172.21.15.32:0/3173522538 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f675805feb0 0x7f6758061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:53.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.711+0000 7f675ec71700 1 --2- 172.21.15.32:0/3173522538 >> v2:172.21.15.61:3300/0 conn(0x7f67580b2fb0 0x7f675813c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:53.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.711+0000 7f675e470700 1 -- 172.21.15.32:0/3173522538 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f675805feb0 msgr2=0x7f6758061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:53.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.711+0000 7f675e470700 1 --2- 172.21.15.32:0/3173522538 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f675805feb0 0x7f6758061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.711+0000 7f675e470700 1 -- 172.21.15.32:0/3173522538 >> v2:172.21.15.61:3300/0 conn(0x7f67580b2fb0 msgr2=0x7f675813c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:53.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.711+0000 7f675e470700 1 --2- 172.21.15.32:0/3173522538 >> v2:172.21.15.61:3300/0 conn(0x7f67580b2fb0 0x7f675813c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.711+0000 7f675e470700 1 -- 172.21.15.32:0/3173522538 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6758142820 con 0x7f675805f3b0 2020-12-05T17:38:53.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.711+0000 7f675e470700 1 --2- 172.21.15.32:0/3173522538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f675805f3b0 0x7f675805f790 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7f674c002f30 tx=0x7f674c006e80).ready entity=mon.0 client_cookie=d0aec2c5c035965 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:53.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.715+0000 7f675d46e700 1 -- 172.21.15.32:0/3173522538 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f674c009640 con 0x7f675805f3b0 2020-12-05T17:38:53.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.715+0000 7f675d46e700 1 -- 172.21.15.32:0/3173522538 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f674c0097a0 con 0x7f675805f3b0 2020-12-05T17:38:53.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.715+0000 7f675d46e700 1 -- 172.21.15.32:0/3173522538 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f674c009b00 con 0x7f675805f3b0 2020-12-05T17:38:53.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.715+0000 7f675f472700 1 -- 172.21.15.32:0/3173522538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f675805f3b0 msgr2=0x7f675805f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:53.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.715+0000 7f675f472700 1 --2- 172.21.15.32:0/3173522538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f675805f3b0 0x7f675805f790 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7f674c002f30 tx=0x7f674c006e80).stop 2020-12-05T17:38:53.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.715+0000 7f675f472700 1 -- 172.21.15.32:0/3173522538 shutdown_connections 2020-12-05T17:38:53.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.715+0000 7f675f472700 1 --2- 172.21.15.32:0/3173522538 >> v2:172.21.15.61:3300/0 conn(0x7f67580b2fb0 0x7f675813c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.728 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.715+0000 7f675f472700 1 --2- 172.21.15.32:0/3173522538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f675805f3b0 0x7f675805f790 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.728 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.715+0000 7f675f472700 1 --2- 172.21.15.32:0/3173522538 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f675805feb0 0x7f6758061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.728 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.715+0000 7f675f472700 1 -- 172.21.15.32:0/3173522538 >> 172.21.15.32:0/3173522538 conn(0x7f67580b8f20 msgr2=0x7f67580b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:53.728 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.715+0000 7f675f472700 1 -- 172.21.15.32:0/3173522538 shutdown_connections 2020-12-05T17:38:53.729 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.715+0000 7f675f472700 1 -- 172.21.15.32:0/3173522538 wait complete. 2020-12-05T17:38:53.729 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.715+0000 7f675f472700 1 Processor -- start 2020-12-05T17:38:53.729 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.715+0000 7f675f472700 1 -- start start 2020-12-05T17:38:53.729 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f675f472700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f675805f3b0 0x7f6758143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:53.730 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f675f472700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f675805feb0 0x7f67581443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:53.731 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f675f472700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f67580b2fb0 0x7f6758148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:53.731 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f675f472700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6758141120 con 0x7f675805f3b0 2020-12-05T17:38:53.731 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f675f472700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6758146800 con 0x7f675805feb0 2020-12-05T17:38:53.731 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f675f472700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6758146aa0 con 0x7f67580b2fb0 2020-12-05T17:38:53.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f675e470700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f675805f3b0 0x7f6758143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:53.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f675e470700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f675805f3b0 0x7f6758143eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37270/0 (socket says 172.21.15.32:37270) 2020-12-05T17:38:53.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f675e470700 1 -- 172.21.15.32:0/3284473858 learned_addr learned my addr 172.21.15.32:0/3284473858 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:53.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f675dc6f700 1 --2- 172.21.15.32:0/3284473858 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f675805feb0 0x7f67581443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:53.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f675ec71700 1 --2- 172.21.15.32:0/3284473858 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f67580b2fb0 0x7f6758148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:53.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f675e470700 1 -- 172.21.15.32:0/3284473858 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f67580b2fb0 msgr2=0x7f6758148d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:53.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f675e470700 1 --2- 172.21.15.32:0/3284473858 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f67580b2fb0 0x7f6758148d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f675e470700 1 -- 172.21.15.32:0/3284473858 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f675805feb0 msgr2=0x7f67581443a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:53.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f675e470700 1 --2- 172.21.15.32:0/3284473858 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f675805feb0 0x7f67581443a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f675e470700 1 -- 172.21.15.32:0/3284473858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6758141390 con 0x7f675805f3b0 2020-12-05T17:38:53.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f675ec71700 1 --2- 172.21.15.32:0/3284473858 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f67580b2fb0 0x7f6758148d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:38:53.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f675e470700 1 --2- 172.21.15.32:0/3284473858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f675805f3b0 0x7f6758143eb0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7f674c002f30 tx=0x7f674c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:53.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f674b7fe700 1 -- 172.21.15.32:0/3284473858 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f674c011ca0 con 0x7f675805f3b0 2020-12-05T17:38:53.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f674b7fe700 1 -- 172.21.15.32:0/3284473858 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f674c011e00 con 0x7f675805f3b0 2020-12-05T17:38:53.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f674b7fe700 1 -- 172.21.15.32:0/3284473858 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f674c01d640 con 0x7f675805f3b0 2020-12-05T17:38:53.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f675f472700 1 -- 172.21.15.32:0/3284473858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6758141830 con 0x7f675805f3b0 2020-12-05T17:38:53.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f675f472700 1 -- 172.21.15.32:0/3284473858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6758142500 con 0x7f675805f3b0 2020-12-05T17:38:53.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f675f472700 1 -- 172.21.15.32:0/3284473858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6758057050 con 0x7f675805f3b0 2020-12-05T17:38:53.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f674b7fe700 1 -- 172.21.15.32:0/3284473858 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f674c00f070 con 0x7f675805f3b0 2020-12-05T17:38:53.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.719+0000 7f674b7fe700 1 --2- 172.21.15.32:0/3284473858 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f67380553b0 0x7f6738057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:53.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.723+0000 7f674b7fe700 1 -- 172.21.15.32:0/3284473858 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f674c06f7b0 con 0x7f675805f3b0 2020-12-05T17:38:53.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.723+0000 7f675dc6f700 1 --2- 172.21.15.32:0/3284473858 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f67380553b0 0x7f6738057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:53.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.723+0000 7f675dc6f700 1 --2- 172.21.15.32:0/3284473858 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f67380553b0 0x7f6738057870 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f675400b000 tx=0x7f6754006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:53.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.727+0000 7f674b7fe700 1 -- 172.21.15.32:0/3284473858 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f674c045d10 con 0x7f675805f3b0 2020-12-05T17:38:53.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.867+0000 7f675f472700 1 -- 172.21.15.32:0/3284473858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f675805aab0 con 0x7f675805f3b0 2020-12-05T17:38:53.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.871+0000 7f674b7fe700 1 -- 172.21.15.32:0/3284473858 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v79) v1 ==== 92+0+10 (secure 0 0 0) 0x7f674c0492b0 con 0x7f675805f3b0 2020-12-05T17:38:53.876 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:38:53.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.871+0000 7f675f472700 1 -- 172.21.15.32:0/3284473858 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f67380553b0 msgr2=0x7f6738057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:53.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.871+0000 7f675f472700 1 --2- 172.21.15.32:0/3284473858 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f67380553b0 0x7f6738057870 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f675400b000 tx=0x7f6754006dc0).stop 2020-12-05T17:38:53.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.871+0000 7f675f472700 1 -- 172.21.15.32:0/3284473858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f675805f3b0 msgr2=0x7f6758143eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:53.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.871+0000 7f675f472700 1 --2- 172.21.15.32:0/3284473858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f675805f3b0 0x7f6758143eb0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7f674c002f30 tx=0x7f674c015000).stop 2020-12-05T17:38:53.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.871+0000 7f675f472700 1 -- 172.21.15.32:0/3284473858 shutdown_connections 2020-12-05T17:38:53.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.871+0000 7f675f472700 1 --2- 172.21.15.32:0/3284473858 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f67380553b0 0x7f6738057870 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.871+0000 7f675f472700 1 --2- 172.21.15.32:0/3284473858 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f67580b2fb0 0x7f6758148d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.871+0000 7f675f472700 1 --2- 172.21.15.32:0/3284473858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f675805f3b0 0x7f6758143eb0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.871+0000 7f675f472700 1 --2- 172.21.15.32:0/3284473858 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f675805feb0 0x7f67581443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:53.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.871+0000 7f675f472700 1 -- 172.21.15.32:0/3284473858 >> 172.21.15.32:0/3284473858 conn(0x7f67580b8f20 msgr2=0x7f67580b9520 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:53.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.871+0000 7f675f472700 1 -- 172.21.15.32:0/3284473858 shutdown_connections 2020-12-05T17:38:53.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.871+0000 7f675f472700 1 -- 172.21.15.32:0/3284473858 wait complete. 2020-12-05T17:38:53.890 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-12-05T17:38:54.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.995+0000 7fa378f6b700 1 Processor -- start 2020-12-05T17:38:54.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.995+0000 7fa378f6b700 1 -- start start 2020-12-05T17:38:54.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa378f6b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3740b00a0 0x7fa374061910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:54.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa378f6b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3740601e0 con 0x7fa3740acca0 2020-12-05T17:38:54.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa378f6b700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa374061fa0 con 0x7fa37405fe00 2020-12-05T17:38:54.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa378f6b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3740ac1a0 con 0x7fa3740b00a0 2020-12-05T17:38:54.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa372d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa3740acca0 0x7fa374145d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39930/0 (socket says 172.21.15.32:39930) 2020-12-05T17:38:54.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa372d9d700 1 -- 172.21.15.32:0/1273075053 learned_addr learned my addr 172.21.15.32:0/1273075053 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:38:54.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa369d9b700 1 --2- 172.21.15.32:0/1273075053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3740b00a0 0x7fa374061910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:54.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa371d9b700 1 -- 172.21.15.32:0/1273075053 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1067393684 0 0) 0x7fa3740601e0 con 0x7fa3740acca0 2020-12-05T17:38:54.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa371d9b700 1 -- 172.21.15.32:0/1273075053 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa354003230 con 0x7fa3740acca0 2020-12-05T17:38:54.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa371d9b700 1 -- 172.21.15.32:0/1273075053 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3694240429 0 0) 0x7fa354003230 con 0x7fa3740acca0 2020-12-05T17:38:54.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa371d9b700 1 -- 172.21.15.32:0/1273075053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3740b00a0 msgr2=0x7fa374061910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:54.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa371d9b700 1 --2- 172.21.15.32:0/1273075053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3740b00a0 0x7fa374061910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:54.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa371d9b700 1 -- 172.21.15.32:0/1273075053 >> v1:172.21.15.61:6789/0 conn(0x7fa37405fe00 legacy=0x7fa3740ac590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:54.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa371d9b700 1 -- 172.21.15.32:0/1273075053 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa374062240 con 0x7fa3740acca0 2020-12-05T17:38:54.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa371d9b700 1 -- 172.21.15.32:0/1273075053 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fa36c001a50 con 0x7fa3740acca0 2020-12-05T17:38:54.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa371d9b700 1 -- 172.21.15.32:0/1273075053 >> v1:172.21.15.32:6789/0 conn(0x7fa3740acca0 legacy=0x7fa374145d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:54.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa371d9b700 1 --2- 172.21.15.32:0/1273075053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3740b00a0 0x7fa354003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:54.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa371d9b700 1 --2- 172.21.15.32:0/1273075053 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa354003c60 0x7fa3540044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:54.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa371d9b700 1 --2- 172.21.15.32:0/1273075053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3540049c0 0x7fa354004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:54.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa371d9b700 1 -- 172.21.15.32:0/1273075053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3540041f0 con 0x7fa3540049c0 2020-12-05T17:38:54.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa371d9b700 1 -- 172.21.15.32:0/1273075053 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35400b720 con 0x7fa354003c60 2020-12-05T17:38:54.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa371d9b700 1 -- 172.21.15.32:0/1273075053 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35400b860 con 0x7fa3740b00a0 2020-12-05T17:38:54.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa372d9d700 1 --2- 172.21.15.32:0/1273075053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3540049c0 0x7fa354004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:54.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa369d9b700 1 --2- 172.21.15.32:0/1273075053 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa354003c60 0x7fa3540044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:54.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa37259c700 1 --2- 172.21.15.32:0/1273075053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3740b00a0 0x7fa354003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:54.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa372d9d700 1 -- 172.21.15.32:0/1273075053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3740b00a0 msgr2=0x7fa354003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:54.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa372d9d700 1 --2- 172.21.15.32:0/1273075053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3740b00a0 0x7fa354003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:54.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa372d9d700 1 -- 172.21.15.32:0/1273075053 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa354003c60 msgr2=0x7fa3540044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:54.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa372d9d700 1 --2- 172.21.15.32:0/1273075053 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa354003c60 0x7fa3540044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:54.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa372d9d700 1 -- 172.21.15.32:0/1273075053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa354003230 con 0x7fa3540049c0 2020-12-05T17:38:54.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa37259c700 1 --2- 172.21.15.32:0/1273075053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3740b00a0 0x7fa354003750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:38:54.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa372d9d700 1 --2- 172.21.15.32:0/1273075053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3540049c0 0x7fa354004e40 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7fa3540054a0 tx=0x7fa374146e20).ready entity=mon.0 client_cookie=cc969bf6b55b9b19 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:54.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa371d9b700 1 -- 172.21.15.32:0/1273075053 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa36c005b70 con 0x7fa3540049c0 2020-12-05T17:38:54.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa371d9b700 1 -- 172.21.15.32:0/1273075053 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36c005cd0 con 0x7fa3540049c0 2020-12-05T17:38:54.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa371d9b700 1 -- 172.21.15.32:0/1273075053 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa36c0072c0 con 0x7fa3540049c0 2020-12-05T17:38:54.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa378f6b700 1 -- 172.21.15.32:0/1273075053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3540049c0 msgr2=0x7fa354004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:54.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa378f6b700 1 --2- 172.21.15.32:0/1273075053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3540049c0 0x7fa354004e40 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7fa3540054a0 tx=0x7fa374146e20).stop 2020-12-05T17:38:54.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:53.999+0000 7fa372d9d700 1 -- 172.21.15.32:0/1273075053 reap_dead start 2020-12-05T17:38:54.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.003+0000 7fa378f6b700 1 -- 172.21.15.32:0/1273075053 shutdown_connections 2020-12-05T17:38:54.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.003+0000 7fa378f6b700 1 -- 172.21.15.32:0/1273075053 >> 172.21.15.32:0/1273075053 conn(0x7fa3740b57c0 msgr2=0x7fa37405f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:54.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.003+0000 7fa378f6b700 1 -- 172.21.15.32:0/1273075053 shutdown_connections 2020-12-05T17:38:54.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.003+0000 7fa378f6b700 1 -- 172.21.15.32:0/1273075053 wait complete. 2020-12-05T17:38:54.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.003+0000 7fa378f6b700 1 Processor -- start 2020-12-05T17:38:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.003+0000 7fa378f6b700 1 -- start start 2020-12-05T17:38:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.007+0000 7fa378f6b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa374066820 0x7fa374156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.007+0000 7fa378f6b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa374062c80 0x7fa374156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.007+0000 7fa378f6b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa374156a30 0x7fa374159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:54.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.007+0000 7fa378f6b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3740654e0 con 0x7fa374062c80 2020-12-05T17:38:54.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.007+0000 7fa378f6b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3740af4a0 con 0x7fa374156a30 2020-12-05T17:38:54.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.007+0000 7fa378f6b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa374155bf0 con 0x7fa374066820 2020-12-05T17:38:54.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.007+0000 7fa369d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa374062c80 0x7fa374156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:54.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.007+0000 7fa369d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa374062c80 0x7fa374156540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37292/0 (socket says 172.21.15.32:37292) 2020-12-05T17:38:54.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.007+0000 7fa369d9b700 1 -- 172.21.15.32:0/47350928 learned_addr learned my addr 172.21.15.32:0/47350928 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:54.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.007+0000 7fa37259c700 1 --2- 172.21.15.32:0/47350928 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa374066820 0x7fa374156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:54.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.007+0000 7fa372d9d700 1 --2- 172.21.15.32:0/47350928 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa374156a30 0x7fa374159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:54.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.007+0000 7fa369d9b700 1 -- 172.21.15.32:0/47350928 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa374066820 msgr2=0x7fa374156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:54.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.007+0000 7fa369d9b700 1 --2- 172.21.15.32:0/47350928 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa374066820 0x7fa374156050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:54.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.007+0000 7fa369d9b700 1 -- 172.21.15.32:0/47350928 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa374156a30 msgr2=0x7fa374159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:54.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.007+0000 7fa369d9b700 1 --2- 172.21.15.32:0/47350928 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa374156a30 0x7fa374159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:54.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.007+0000 7fa369d9b700 1 -- 172.21.15.32:0/47350928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa374066c00 con 0x7fa374062c80 2020-12-05T17:38:54.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.007+0000 7fa369d9b700 1 --2- 172.21.15.32:0/47350928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa374062c80 0x7fa374156540 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7fa35c006750 tx=0x7fa35c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:54.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.007+0000 7fa36bfff700 1 -- 172.21.15.32:0/47350928 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa35c00f8b0 con 0x7fa374062c80 2020-12-05T17:38:54.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.007+0000 7fa36bfff700 1 -- 172.21.15.32:0/47350928 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35c00fa10 con 0x7fa374062c80 2020-12-05T17:38:54.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.007+0000 7fa378f6b700 1 -- 172.21.15.32:0/47350928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa374063e30 con 0x7fa374062c80 2020-12-05T17:38:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.007+0000 7fa36bfff700 1 -- 172.21.15.32:0/47350928 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa35c00fb80 con 0x7fa374062c80 2020-12-05T17:38:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.007+0000 7fa378f6b700 1 -- 172.21.15.32:0/47350928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa374064370 con 0x7fa374062c80 2020-12-05T17:38:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.007+0000 7fa378f6b700 1 -- 172.21.15.32:0/47350928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa374064530 con 0x7fa374062c80 2020-12-05T17:38:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.011+0000 7fa36bfff700 1 -- 172.21.15.32:0/47350928 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa35c024070 con 0x7fa374062c80 2020-12-05T17:38:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.011+0000 7fa36bfff700 1 --2- 172.21.15.32:0/47350928 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa358055330 0x7fa3580577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.011+0000 7fa36bfff700 1 -- 172.21.15.32:0/47350928 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fa35c06e570 con 0x7fa374062c80 2020-12-05T17:38:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.011+0000 7fa37259c700 1 --2- 172.21.15.32:0/47350928 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa358055330 0x7fa3580577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.011+0000 7fa37259c700 1 --2- 172.21.15.32:0/47350928 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa358055330 0x7fa3580577f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fa36400d000 tx=0x7fa36400c040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.015+0000 7fa36bfff700 1 -- 172.21.15.32:0/47350928 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa35c044950 con 0x7fa374062c80 2020-12-05T17:38:54.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.159+0000 7fa378f6b700 1 -- 172.21.15.32:0/47350928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa374057050 con 0x7fa374062c80 2020-12-05T17:38:54.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.159+0000 7fa36bfff700 1 -- 172.21.15.32:0/47350928 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+143 (secure 0 0 0) 0x7fa35c047f70 con 0x7fa374062c80 2020-12-05T17:38:54.166 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:38:54.166 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2020-12-05T17:38:54.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:54.167 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2020-12-05T17:38:54.167 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2020-12-05T17:38:54.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:38:54.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:38:54.168 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2020-12-05T17:38:54.168 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2020-12-05T17:38:54.168 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:38:54.168 INFO:teuthology.orchestra.run.smithi032.stdout:] 2020-12-05T17:38:54.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.163+0000 7fa378f6b700 1 -- 172.21.15.32:0/47350928 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa358055330 msgr2=0x7fa3580577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:54.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.163+0000 7fa378f6b700 1 --2- 172.21.15.32:0/47350928 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa358055330 0x7fa3580577f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fa36400d000 tx=0x7fa36400c040).stop 2020-12-05T17:38:54.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.163+0000 7fa378f6b700 1 -- 172.21.15.32:0/47350928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa374062c80 msgr2=0x7fa374156540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:54.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.163+0000 7fa378f6b700 1 --2- 172.21.15.32:0/47350928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa374062c80 0x7fa374156540 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7fa35c006750 tx=0x7fa35c008040).stop 2020-12-05T17:38:54.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.163+0000 7fa378f6b700 1 -- 172.21.15.32:0/47350928 shutdown_connections 2020-12-05T17:38:54.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.163+0000 7fa378f6b700 1 --2- 172.21.15.32:0/47350928 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa374156a30 0x7fa374159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:54.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.163+0000 7fa378f6b700 1 --2- 172.21.15.32:0/47350928 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa358055330 0x7fa3580577f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:54.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.163+0000 7fa378f6b700 1 --2- 172.21.15.32:0/47350928 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa374066820 0x7fa374156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:54.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.163+0000 7fa378f6b700 1 --2- 172.21.15.32:0/47350928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa374062c80 0x7fa374156540 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:54.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.163+0000 7fa378f6b700 1 -- 172.21.15.32:0/47350928 >> 172.21.15.32:0/47350928 conn(0x7fa3740b57c0 msgr2=0x7fa37414a420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:54.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.163+0000 7fa378f6b700 1 -- 172.21.15.32:0/47350928 shutdown_connections 2020-12-05T17:38:54.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.163+0000 7fa378f6b700 1 -- 172.21.15.32:0/47350928 wait complete. 2020-12-05T17:38:54.180 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-12-05T17:38:54.180 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-12-05T17:38:54.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.291+0000 7f19b900c700 1 Processor -- start 2020-12-05T17:38:54.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.291+0000 7f19b900c700 1 -- start start 2020-12-05T17:38:54.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b900c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19b405f6f0 0x7f19b4141700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:54.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b900c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b41439c0 con 0x7f19b4142300 2020-12-05T17:38:54.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b900c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b405fe20 con 0x7f19b4061b30 2020-12-05T17:38:54.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b900c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19b4146910 con 0x7f19b405f6f0 2020-12-05T17:38:54.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19b405f6f0 0x7f19b4141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:54.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b880b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f19b4142300 0x7f19b4142790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39950/0 (socket says 172.21.15.32:39950) 2020-12-05T17:38:54.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b880b700 1 -- 172.21.15.32:0/2036436153 learned_addr learned my addr 172.21.15.32:0/2036436153 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:38:54.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b3fff700 1 -- 172.21.15.32:0/2036436153 >> v1:172.21.15.61:6789/0 conn(0x7f19b4061b30 legacy=0x7f19b4141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:54.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b3fff700 1 -- 172.21.15.32:0/2036436153 >> v1:172.21.15.32:6789/0 conn(0x7f19b4142300 legacy=0x7f19b4142790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:54.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b3fff700 1 -- 172.21.15.32:0/2036436153 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b405fad0 con 0x7f19b405f6f0 2020-12-05T17:38:54.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b2ffd700 1 -- 172.21.15.32:0/2036436153 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2879832191 0 0) 0x7f19b41439c0 con 0x7f19b4142300 2020-12-05T17:38:54.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b3fff700 1 --2- 172.21.15.32:0/2036436153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19b405f6f0 0x7f19b4141700 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7f19a4002f30 tx=0x7f19a4006d60).ready entity=mon.0 client_cookie=818ac04af2840574 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:54.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b2ffd700 1 -- 172.21.15.32:0/2036436153 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19a40097f0 con 0x7f19b405f6f0 2020-12-05T17:38:54.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b2ffd700 1 -- 172.21.15.32:0/2036436153 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a4009950 con 0x7f19b405f6f0 2020-12-05T17:38:54.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b2ffd700 1 -- 172.21.15.32:0/2036436153 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19a4009ac0 con 0x7f19b405f6f0 2020-12-05T17:38:54.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b900c700 1 -- 172.21.15.32:0/2036436153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19b405f6f0 msgr2=0x7f19b4141700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:54.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b900c700 1 --2- 172.21.15.32:0/2036436153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19b405f6f0 0x7f19b4141700 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7f19a4002f30 tx=0x7f19a4006d60).stop 2020-12-05T17:38:54.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b900c700 1 -- 172.21.15.32:0/2036436153 shutdown_connections 2020-12-05T17:38:54.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b900c700 1 --2- 172.21.15.32:0/2036436153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19b405f6f0 0x7f19b4141700 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:54.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b900c700 1 -- 172.21.15.32:0/2036436153 >> 172.21.15.32:0/2036436153 conn(0x7f19b4061010 msgr2=0x7f19b4061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:54.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b900c700 1 -- 172.21.15.32:0/2036436153 shutdown_connections 2020-12-05T17:38:54.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b900c700 1 -- 172.21.15.32:0/2036436153 wait complete. 2020-12-05T17:38:54.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b900c700 1 Processor -- start 2020-12-05T17:38:54.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.295+0000 7f19b900c700 1 -- start start 2020-12-05T17:38:54.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.299+0000 7f19b900c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19b405f6f0 0x7f19b40b6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:54.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.299+0000 7f19b900c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19b4061b30 0x7f19b4151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:54.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.299+0000 7f19b900c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19b4142300 0x7f19b4152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:54.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.299+0000 7f19b900c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b4151810 con 0x7f19b4061b30 2020-12-05T17:38:54.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.299+0000 7f19b900c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b40b4060 con 0x7f19b4142300 2020-12-05T17:38:54.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.299+0000 7f19b900c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b40b4300 con 0x7f19b405f6f0 2020-12-05T17:38:54.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.299+0000 7f19b37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19b4061b30 0x7f19b4151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:54.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.299+0000 7f19b37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19b4061b30 0x7f19b4151c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37306/0 (socket says 172.21.15.32:37306) 2020-12-05T17:38:54.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.299+0000 7f19b37fe700 1 -- 172.21.15.32:0/2112151204 learned_addr learned my addr 172.21.15.32:0/2112151204 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:54.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.299+0000 7f19b880b700 1 --2- 172.21.15.32:0/2112151204 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19b4142300 0x7f19b4152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:54.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.299+0000 7f19b3fff700 1 --2- 172.21.15.32:0/2112151204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19b405f6f0 0x7f19b40b6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:54.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.299+0000 7f19b37fe700 1 -- 172.21.15.32:0/2112151204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19b405f6f0 msgr2=0x7f19b40b6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:54.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.299+0000 7f19b37fe700 1 --2- 172.21.15.32:0/2112151204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19b405f6f0 0x7f19b40b6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:54.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.299+0000 7f19b37fe700 1 -- 172.21.15.32:0/2112151204 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19b4142300 msgr2=0x7f19b4152130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:54.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.299+0000 7f19b37fe700 1 --2- 172.21.15.32:0/2112151204 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19b4142300 0x7f19b4152130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:54.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.299+0000 7f19b37fe700 1 -- 172.21.15.32:0/2112151204 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b4065370 con 0x7f19b4061b30 2020-12-05T17:38:54.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.299+0000 7f19b37fe700 1 --2- 172.21.15.32:0/2112151204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19b4061b30 0x7f19b4151c40 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7f19b40b05d0 tx=0x7f19a8007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:54.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.299+0000 7f19b17fa700 1 -- 172.21.15.32:0/2112151204 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19a8007240 con 0x7f19b4061b30 2020-12-05T17:38:54.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.299+0000 7f19b17fa700 1 -- 172.21.15.32:0/2112151204 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a80073a0 con 0x7f19b4061b30 2020-12-05T17:38:54.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.299+0000 7f19b900c700 1 -- 172.21.15.32:0/2112151204 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b4063040 con 0x7f19b4061b30 2020-12-05T17:38:54.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.299+0000 7f19b17fa700 1 -- 172.21.15.32:0/2112151204 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19a80066d0 con 0x7f19b4061b30 2020-12-05T17:38:54.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.299+0000 7f19b900c700 1 -- 172.21.15.32:0/2112151204 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b40b3bf0 con 0x7f19b4061b30 2020-12-05T17:38:54.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.299+0000 7f19b900c700 1 -- 172.21.15.32:0/2112151204 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19b4151980 con 0x7f19b4061b30 2020-12-05T17:38:54.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.303+0000 7f19b17fa700 1 -- 172.21.15.32:0/2112151204 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f19a8010070 con 0x7f19b4061b30 2020-12-05T17:38:54.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.303+0000 7f19b17fa700 1 --2- 172.21.15.32:0/2112151204 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f19940553b0 0x7f1994057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:54.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.303+0000 7f19b17fa700 1 -- 172.21.15.32:0/2112151204 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f19a806d3c0 con 0x7f19b4061b30 2020-12-05T17:38:54.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.303+0000 7f19b3fff700 1 --2- 172.21.15.32:0/2112151204 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f19940553b0 0x7f1994057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:54.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.303+0000 7f19b3fff700 1 --2- 172.21.15.32:0/2112151204 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f19940553b0 0x7f1994057870 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f19a401b000 tx=0x7f19a4016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:54.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.311+0000 7f19b17fa700 1 -- 172.21.15.32:0/2112151204 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f19b4151980 con 0x7f19b4061b30 2020-12-05T17:38:54.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:54.451+0000 7f19b900c700 1 -- 172.21.15.32:0/2112151204 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f19b4151950 con 0x7f19b4061b30 2020-12-05T17:38:55.463 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.455+0000 7f19b17fa700 1 -- 172.21.15.32:0/2112151204 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v80) v1 ==== 133+0+0 (secure 0 0 0) 0x7f19a80438a0 con 0x7f19b4061b30 2020-12-05T17:38:55.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.459+0000 7f19b900c700 1 -- 172.21.15.32:0/2112151204 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f19940553b0 msgr2=0x7f1994057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:55.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.459+0000 7f19b900c700 1 --2- 172.21.15.32:0/2112151204 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f19940553b0 0x7f1994057870 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f19a401b000 tx=0x7f19a4016000).stop 2020-12-05T17:38:55.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.459+0000 7f19b900c700 1 -- 172.21.15.32:0/2112151204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19b4061b30 msgr2=0x7f19b4151c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:55.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.459+0000 7f19b900c700 1 --2- 172.21.15.32:0/2112151204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19b4061b30 0x7f19b4151c40 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7f19b40b05d0 tx=0x7f19a8007be0).stop 2020-12-05T17:38:55.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.459+0000 7f19b900c700 1 -- 172.21.15.32:0/2112151204 shutdown_connections 2020-12-05T17:38:55.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.459+0000 7f19b900c700 1 --2- 172.21.15.32:0/2112151204 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19b4142300 0x7f19b4152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:55.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.459+0000 7f19b900c700 1 --2- 172.21.15.32:0/2112151204 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f19940553b0 0x7f1994057870 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:55.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.459+0000 7f19b900c700 1 --2- 172.21.15.32:0/2112151204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19b405f6f0 0x7f19b40b6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:55.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.459+0000 7f19b900c700 1 --2- 172.21.15.32:0/2112151204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19b4061b30 0x7f19b4151c40 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:55.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.459+0000 7f19b900c700 1 -- 172.21.15.32:0/2112151204 >> 172.21.15.32:0/2112151204 conn(0x7f19b4061010 msgr2=0x7f19b4142db0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:55.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.459+0000 7f19b900c700 1 -- 172.21.15.32:0/2112151204 shutdown_connections 2020-12-05T17:38:55.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.459+0000 7f19b900c700 1 -- 172.21.15.32:0/2112151204 wait complete. 2020-12-05T17:38:55.473 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2020-12-05T17:38:55.482 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-12-05T17:38:55.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.591+0000 7fe9116b5700 1 Processor -- start 2020-12-05T17:38:55.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.591+0000 7fe9116b5700 1 -- start start 2020-12-05T17:38:55.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.591+0000 7fe9116b5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe90c0b1dc0 0x7fe90c0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:55.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.591+0000 7fe9116b5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe90c0afd30 0x7fe90c0ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:55.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe9116b5700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe90c1468e0 con 0x7fe90c0af5f0 2020-12-05T17:38:55.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe9116b5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe90c142c90 con 0x7fe90c0afd30 2020-12-05T17:38:55.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe9116b5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c146b10 con 0x7fe90c0b1dc0 2020-12-05T17:38:55.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe90b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe90c0afd30 0x7fe90c0ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:55.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe90b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe90c0afd30 0x7fe90c0ae600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37312/0 (socket says 172.21.15.32:37312) 2020-12-05T17:38:55.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe90b7fe700 1 -- 172.21.15.32:0/645350804 learned_addr learned my addr 172.21.15.32:0/645350804 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe90bfff700 1 --2- 172.21.15.32:0/645350804 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe90c0b1dc0 0x7fe90c0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe90affd700 1 -- 172.21.15.32:0/645350804 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2414098841 0 0) 0x7fe90c1468e0 con 0x7fe90c0af5f0 2020-12-05T17:38:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe90affd700 1 -- 172.21.15.32:0/645350804 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8f8003230 con 0x7fe90c0af5f0 2020-12-05T17:38:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe90b7fe700 1 -- 172.21.15.32:0/645350804 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe90c0b1dc0 msgr2=0x7fe90c0b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:55.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe90b7fe700 1 --2- 172.21.15.32:0/645350804 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe90c0b1dc0 0x7fe90c0b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:55.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe90b7fe700 1 -- 172.21.15.32:0/645350804 >> v1:172.21.15.61:6789/0 conn(0x7fe90c0af5f0 legacy=0x7fe90c0aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:55.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe90b7fe700 1 -- 172.21.15.32:0/645350804 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c146db0 con 0x7fe90c0afd30 2020-12-05T17:38:55.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe90b7fe700 1 --2- 172.21.15.32:0/645350804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe90c0afd30 0x7fe90c0ae600 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7fe904001450 tx=0x7fe8fc006e80).ready entity=mon.0 client_cookie=eb7be3d6776a6330 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe90affd700 1 -- 172.21.15.32:0/645350804 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8fc009600 con 0x7fe90c0afd30 2020-12-05T17:38:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe90affd700 1 -- 172.21.15.32:0/645350804 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc009760 con 0x7fe90c0afd30 2020-12-05T17:38:55.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe90affd700 1 -- 172.21.15.32:0/645350804 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8fc009910 con 0x7fe90c0afd30 2020-12-05T17:38:55.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe9116b5700 1 -- 172.21.15.32:0/645350804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe90c0afd30 msgr2=0x7fe90c0ae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:55.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe9116b5700 1 --2- 172.21.15.32:0/645350804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe90c0afd30 0x7fe90c0ae600 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7fe904001450 tx=0x7fe8fc006e80).stop 2020-12-05T17:38:55.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe9116b5700 1 -- 172.21.15.32:0/645350804 shutdown_connections 2020-12-05T17:38:55.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe9116b5700 1 --2- 172.21.15.32:0/645350804 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe90c0b1dc0 0x7fe90c0b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:55.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe9116b5700 1 --2- 172.21.15.32:0/645350804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe90c0afd30 0x7fe90c0ae600 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:55.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe9116b5700 1 -- 172.21.15.32:0/645350804 >> 172.21.15.32:0/645350804 conn(0x7fe90c0620b0 msgr2=0x7fe90c062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:55.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe9116b5700 1 -- 172.21.15.32:0/645350804 shutdown_connections 2020-12-05T17:38:55.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.595+0000 7fe9116b5700 1 -- 172.21.15.32:0/645350804 wait complete. 2020-12-05T17:38:55.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe9116b5700 1 Processor -- start 2020-12-05T17:38:55.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe9116b5700 1 -- start start 2020-12-05T17:38:55.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe9116b5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe90c0af5f0 0x7fe90c140420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:55.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe9116b5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe90c0afd30 0x7fe90c154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:55.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe9116b5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe90c0b1dc0 0x7fe90c063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:55.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe9116b5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c145c20 con 0x7fe90c0afd30 2020-12-05T17:38:55.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe9116b5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c0639c0 con 0x7fe90c0af5f0 2020-12-05T17:38:55.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe9116b5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c0651b0 con 0x7fe90c0b1dc0 2020-12-05T17:38:55.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe90b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe90c0afd30 0x7fe90c154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:55.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe90b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe90c0afd30 0x7fe90c154e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37318/0 (socket says 172.21.15.32:37318) 2020-12-05T17:38:55.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe90b7fe700 1 -- 172.21.15.32:0/3070125134 learned_addr learned my addr 172.21.15.32:0/3070125134 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:55.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe90bfff700 1 --2- 172.21.15.32:0/3070125134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe90c0af5f0 0x7fe90c140420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:55.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe910eb4700 1 --2- 172.21.15.32:0/3070125134 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe90c0b1dc0 0x7fe90c063040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:55.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe90b7fe700 1 -- 172.21.15.32:0/3070125134 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe90c0b1dc0 msgr2=0x7fe90c063040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:55.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe90b7fe700 1 --2- 172.21.15.32:0/3070125134 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe90c0b1dc0 0x7fe90c063040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:55.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe90b7fe700 1 -- 172.21.15.32:0/3070125134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe90c0af5f0 msgr2=0x7fe90c140420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:55.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe90b7fe700 1 --2- 172.21.15.32:0/3070125134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe90c0af5f0 0x7fe90c140420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:55.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe90b7fe700 1 -- 172.21.15.32:0/3070125134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c155330 con 0x7fe90c0afd30 2020-12-05T17:38:55.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe90b7fe700 1 --2- 172.21.15.32:0/3070125134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe90c0afd30 0x7fe90c154e40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7fe904001450 tx=0x7fe8fc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:55.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe9097fa700 1 -- 172.21.15.32:0/3070125134 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8fc011cf0 con 0x7fe90c0afd30 2020-12-05T17:38:55.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe9097fa700 1 -- 172.21.15.32:0/3070125134 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc011e50 con 0x7fe90c0afd30 2020-12-05T17:38:55.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe9116b5700 1 -- 172.21.15.32:0/3070125134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe90c063530 con 0x7fe90c0afd30 2020-12-05T17:38:55.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe9097fa700 1 -- 172.21.15.32:0/3070125134 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8fc021630 con 0x7fe90c0afd30 2020-12-05T17:38:55.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe9116b5700 1 -- 172.21.15.32:0/3070125134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe90c13e140 con 0x7fe90c0afd30 2020-12-05T17:38:55.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.599+0000 7fe9116b5700 1 -- 172.21.15.32:0/3070125134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe90c065000 con 0x7fe90c0afd30 2020-12-05T17:38:55.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.603+0000 7fe9097fa700 1 -- 172.21.15.32:0/3070125134 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe8fc029070 con 0x7fe90c0afd30 2020-12-05T17:38:55.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.603+0000 7fe9097fa700 1 --2- 172.21.15.32:0/3070125134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe8ec055440 0x7fe8ec057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:55.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.603+0000 7fe9097fa700 1 -- 172.21.15.32:0/3070125134 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5954+0+0 (secure 0 0 0) 0x7fe8fc06eb30 con 0x7fe90c0afd30 2020-12-05T17:38:55.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.603+0000 7fe90bfff700 1 --2- 172.21.15.32:0/3070125134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe8ec055440 0x7fe8ec057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:55.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.603+0000 7fe90bfff700 1 --2- 172.21.15.32:0/3070125134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe8ec055440 0x7fe8ec057900 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7fe90000b000 tx=0x7fe900006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:55.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.607+0000 7fe9097fa700 1 -- 172.21.15.32:0/3070125134 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe8fc044e40 con 0x7fe90c0afd30 2020-12-05T17:38:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:55.747+0000 7fe9116b5700 1 -- 172.21.15.32:0/3070125134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fe90c065370 con 0x7fe90c0afd30 2020-12-05T17:38:56.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.463+0000 7fe9097fa700 1 -- 172.21.15.32:0/3070125134 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v81) v1 ==== 125+0+0 (secure 0 0 0) 0x7fe8fc015030 con 0x7fe90c0afd30 2020-12-05T17:38:56.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.467+0000 7fe9116b5700 1 -- 172.21.15.32:0/3070125134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe8ec055440 msgr2=0x7fe8ec057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:56.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.467+0000 7fe9116b5700 1 --2- 172.21.15.32:0/3070125134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe8ec055440 0x7fe8ec057900 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7fe90000b000 tx=0x7fe900006dc0).stop 2020-12-05T17:38:56.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.467+0000 7fe9116b5700 1 -- 172.21.15.32:0/3070125134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe90c0afd30 msgr2=0x7fe90c154e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:56.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.467+0000 7fe9116b5700 1 --2- 172.21.15.32:0/3070125134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe90c0afd30 0x7fe90c154e40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7fe904001450 tx=0x7fe8fc016000).stop 2020-12-05T17:38:56.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.467+0000 7fe9116b5700 1 -- 172.21.15.32:0/3070125134 shutdown_connections 2020-12-05T17:38:56.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.467+0000 7fe9116b5700 1 --2- 172.21.15.32:0/3070125134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe8ec055440 0x7fe8ec057900 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:56.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.467+0000 7fe9116b5700 1 --2- 172.21.15.32:0/3070125134 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe90c0b1dc0 0x7fe90c063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:56.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.467+0000 7fe9116b5700 1 --2- 172.21.15.32:0/3070125134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe90c0af5f0 0x7fe90c140420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:56.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.467+0000 7fe9116b5700 1 --2- 172.21.15.32:0/3070125134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe90c0afd30 0x7fe90c154e40 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:56.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.467+0000 7fe9116b5700 1 -- 172.21.15.32:0/3070125134 >> 172.21.15.32:0/3070125134 conn(0x7fe90c0620b0 msgr2=0x7fe90c05fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:56.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.467+0000 7fe9116b5700 1 -- 172.21.15.32:0/3070125134 shutdown_connections 2020-12-05T17:38:56.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.471+0000 7fe9116b5700 1 -- 172.21.15.32:0/3070125134 wait complete. 2020-12-05T17:38:56.478 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2020-12-05T17:38:56.490 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-12-05T17:38:56.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.611+0000 7f004359e700 1 Processor -- start 2020-12-05T17:38:56.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.611+0000 7f004359e700 1 -- start start 2020-12-05T17:38:56.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.611+0000 7f004359e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f004405f6f0 0x7f0044141700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:56.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.611+0000 7f004359e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0044061b30 0x7f0044141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:56.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.611+0000 7f004359e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00441439c0 con 0x7f00441420e0 2020-12-05T17:38:56.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.611+0000 7f004359e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0044142550 con 0x7f004405f6f0 2020-12-05T17:38:56.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.611+0000 7f004359e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004405fe20 con 0x7f0044061b30 2020-12-05T17:38:56.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.611+0000 7f0041d9b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0044061b30 0x7f0044141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:56.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.611+0000 7f004259c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f004405f6f0 0x7f0044141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:56.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.611+0000 7f0042d9d700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f00441420e0 0x7f00440b2660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:52618/0 (socket says 172.21.15.32:52618) 2020-12-05T17:38:56.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.611+0000 7f0042d9d700 1 -- 172.21.15.32:0/2334374734 learned_addr learned my addr 172.21.15.32:0/2334374734 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:38:56.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.611+0000 7f004159a700 1 -- 172.21.15.32:0/2334374734 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294158008 0 0) 0x7f00441439c0 con 0x7f00441420e0 2020-12-05T17:38:56.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.611+0000 7f004159a700 1 -- 172.21.15.32:0/2334374734 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0028003230 con 0x7f00441420e0 2020-12-05T17:38:56.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.611+0000 7f0041d9b700 1 -- 172.21.15.32:0/2334374734 >> v2:172.21.15.61:3300/0 conn(0x7f004405f6f0 msgr2=0x7f0044141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:56.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.611+0000 7f0041d9b700 1 --2- 172.21.15.32:0/2334374734 >> v2:172.21.15.61:3300/0 conn(0x7f004405f6f0 0x7f0044141700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:56.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.611+0000 7f0041d9b700 1 -- 172.21.15.32:0/2334374734 >> v1:172.21.15.61:6789/0 conn(0x7f00441420e0 legacy=0x7f00440b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:56.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.611+0000 7f0041d9b700 1 -- 172.21.15.32:0/2334374734 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0044146910 con 0x7f0044061b30 2020-12-05T17:38:56.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.611+0000 7f004259c700 1 --2- 172.21.15.32:0/2334374734 >> v2:172.21.15.61:3300/0 conn(0x7f004405f6f0 0x7f0044141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:38:56.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.615+0000 7f0041d9b700 1 --2- 172.21.15.32:0/2334374734 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0044061b30 0x7f0044141bf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f0038002f30 tx=0x7f0038006dc0).ready entity=mon.2 client_cookie=2c97ff11ab766099 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:56.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.615+0000 7f004159a700 1 -- 172.21.15.32:0/2334374734 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00380096a0 con 0x7f0044061b30 2020-12-05T17:38:56.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.615+0000 7f004159a700 1 -- 172.21.15.32:0/2334374734 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0038009800 con 0x7f0044061b30 2020-12-05T17:38:56.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.615+0000 7f004159a700 1 -- 172.21.15.32:0/2334374734 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0038009b60 con 0x7f0044061b30 2020-12-05T17:38:56.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.615+0000 7f004359e700 1 -- 172.21.15.32:0/2334374734 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0044061b30 msgr2=0x7f0044141bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:56.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.615+0000 7f004359e700 1 --2- 172.21.15.32:0/2334374734 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0044061b30 0x7f0044141bf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f0038002f30 tx=0x7f0038006dc0).stop 2020-12-05T17:38:56.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.615+0000 7f004359e700 1 -- 172.21.15.32:0/2334374734 shutdown_connections 2020-12-05T17:38:56.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.615+0000 7f004359e700 1 --2- 172.21.15.32:0/2334374734 >> v2:172.21.15.61:3300/0 conn(0x7f004405f6f0 0x7f0044141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:56.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.615+0000 7f004359e700 1 --2- 172.21.15.32:0/2334374734 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0044061b30 0x7f0044141bf0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:56.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.615+0000 7f004359e700 1 -- 172.21.15.32:0/2334374734 >> 172.21.15.32:0/2334374734 conn(0x7f0044061010 msgr2=0x7f0044061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:56.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.615+0000 7f004359e700 1 -- 172.21.15.32:0/2334374734 shutdown_connections 2020-12-05T17:38:56.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.615+0000 7f004359e700 1 -- 172.21.15.32:0/2334374734 wait complete. 2020-12-05T17:38:56.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f004359e700 1 Processor -- start 2020-12-05T17:38:56.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f004359e700 1 -- start start 2020-12-05T17:38:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f004359e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f004405f6f0 0x7f0044064150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f004359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0044061b30 0x7f00440b4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f004359e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f00441420e0 0x7f0044154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f004359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0044113230 con 0x7f0044061b30 2020-12-05T17:38:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f004359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0044063150 con 0x7f004405f6f0 2020-12-05T17:38:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f004359e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00440633f0 con 0x7f00441420e0 2020-12-05T17:38:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f0041d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0044061b30 0x7f00440b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f004259c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f004405f6f0 0x7f0044064150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f004259c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f004405f6f0 0x7f0044064150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:57224/0 (socket says 172.21.15.32:57224) 2020-12-05T17:38:56.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f004259c700 1 -- 172.21.15.32:0/1018729857 learned_addr learned my addr 172.21.15.32:0/1018729857 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:56.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f0042d9d700 1 --2- 172.21.15.32:0/1018729857 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f00441420e0 0x7f0044154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:56.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f0041d9b700 1 -- 172.21.15.32:0/1018729857 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f00441420e0 msgr2=0x7f0044154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:56.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f0041d9b700 1 --2- 172.21.15.32:0/1018729857 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f00441420e0 0x7f0044154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:56.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f0041d9b700 1 -- 172.21.15.32:0/1018729857 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f004405f6f0 msgr2=0x7f0044064150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:56.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f0041d9b700 1 --2- 172.21.15.32:0/1018729857 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f004405f6f0 0x7f0044064150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:56.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f0041d9b700 1 -- 172.21.15.32:0/1018729857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0044063750 con 0x7f0044061b30 2020-12-05T17:38:56.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f004259c700 1 --2- 172.21.15.32:0/1018729857 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f004405f6f0 0x7f0044064150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:38:56.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f0041d9b700 1 --2- 172.21.15.32:0/1018729857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0044061b30 0x7f00440b4170 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7f0038002f30 tx=0x7f0038015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:56.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f00337fe700 1 -- 172.21.15.32:0/1018729857 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0038011a20 con 0x7f0044061b30 2020-12-05T17:38:56.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f00337fe700 1 -- 172.21.15.32:0/1018729857 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0038011b80 con 0x7f0044061b30 2020-12-05T17:38:56.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f004359e700 1 -- 172.21.15.32:0/1018729857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00440650f0 con 0x7f0044061b30 2020-12-05T17:38:56.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f00337fe700 1 -- 172.21.15.32:0/1018729857 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f003801c3b0 con 0x7f0044061b30 2020-12-05T17:38:56.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.619+0000 7f004359e700 1 -- 172.21.15.32:0/1018729857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0044113830 con 0x7f0044061b30 2020-12-05T17:38:56.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.623+0000 7f004359e700 1 -- 172.21.15.32:0/1018729857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0044057050 con 0x7f0044061b30 2020-12-05T17:38:56.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.623+0000 7f00337fe700 1 -- 172.21.15.32:0/1018729857 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f003800f070 con 0x7f0044061b30 2020-12-05T17:38:56.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.623+0000 7f00337fe700 1 --2- 172.21.15.32:0/1018729857 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0020055330 0x7f00200577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:56.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.623+0000 7f00337fe700 1 -- 172.21.15.32:0/1018729857 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f003806ead0 con 0x7f0044061b30 2020-12-05T17:38:56.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.623+0000 7f004259c700 1 --2- 172.21.15.32:0/1018729857 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0020055330 0x7f00200577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:56.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.623+0000 7f004259c700 1 --2- 172.21.15.32:0/1018729857 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0020055330 0x7f00200577f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f0034008000 tx=0x7f0034008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:56.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.631+0000 7f00337fe700 1 -- 172.21.15.32:0/1018729857 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0038044cf0 con 0x7f0044061b30 2020-12-05T17:38:56.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.815+0000 7f004359e700 1 -- 172.21.15.32:0/1018729857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f0044069690 con 0x7f0044061b30 2020-12-05T17:38:56.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.827+0000 7f00337fe700 1 -- 172.21.15.32:0/1018729857 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 15 and data pool 16 v36) v1 ==== 189+0+0 (secure 0 0 0) 0x7f0038023070 con 0x7f0044061b30 2020-12-05T17:38:56.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.843+0000 7f004359e700 1 -- 172.21.15.32:0/1018729857 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0020055330 msgr2=0x7f00200577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:56.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.843+0000 7f004359e700 1 --2- 172.21.15.32:0/1018729857 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0020055330 0x7f00200577f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f0034008000 tx=0x7f0034008040).stop 2020-12-05T17:38:56.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.843+0000 7f004359e700 1 -- 172.21.15.32:0/1018729857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0044061b30 msgr2=0x7f00440b4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:56.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.843+0000 7f004359e700 1 --2- 172.21.15.32:0/1018729857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0044061b30 0x7f00440b4170 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7f0038002f30 tx=0x7f0038015000).stop 2020-12-05T17:38:56.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.843+0000 7f004359e700 1 -- 172.21.15.32:0/1018729857 shutdown_connections 2020-12-05T17:38:56.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.843+0000 7f004359e700 1 --2- 172.21.15.32:0/1018729857 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0020055330 0x7f00200577f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:56.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.843+0000 7f004359e700 1 --2- 172.21.15.32:0/1018729857 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f00441420e0 0x7f0044154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:56.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.843+0000 7f004359e700 1 --2- 172.21.15.32:0/1018729857 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f004405f6f0 0x7f0044064150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:56.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.843+0000 7f004359e700 1 --2- 172.21.15.32:0/1018729857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0044061b30 0x7f00440b4170 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:56.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.843+0000 7f004359e700 1 -- 172.21.15.32:0/1018729857 >> 172.21.15.32:0/1018729857 conn(0x7f0044061010 msgr2=0x7f0044142690 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:56.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.843+0000 7f004359e700 1 -- 172.21.15.32:0/1018729857 shutdown_connections 2020-12-05T17:38:56.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.843+0000 7f004359e700 1 -- 172.21.15.32:0/1018729857 wait complete. 2020-12-05T17:38:56.852 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 15 and data pool 16 2020-12-05T17:38:56.862 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:38:56.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.959+0000 7fc5a8a0c700 1 Processor -- start 2020-12-05T17:38:56.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.959+0000 7fc5a8a0c700 1 -- start start 2020-12-05T17:38:56.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.963+0000 7fc5a8a0c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5a4142570 0x7fc5a4142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:56.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.963+0000 7fc5a8a0c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a40b2390 con 0x7fc5a40af0d0 2020-12-05T17:38:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.963+0000 7fc5a8a0c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a4062430 con 0x7fc5a4061cf0 2020-12-05T17:38:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.963+0000 7fc5a8a0c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a413ca50 con 0x7fc5a4142570 2020-12-05T17:38:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.963+0000 7fc5a2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5a4142570 0x7fc5a4142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.963+0000 7fc5a3fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc5a40af0d0 0x7fc5a40af4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40002/0 (socket says 172.21.15.32:40002) 2020-12-05T17:38:56.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.963+0000 7fc5a3fff700 1 -- 172.21.15.32:0/1104850981 learned_addr learned my addr 172.21.15.32:0/1104850981 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:38:56.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.963+0000 7fc5a2ffd700 1 -- 172.21.15.32:0/1104850981 >> v1:172.21.15.61:6789/0 conn(0x7fc5a4061cf0 legacy=0x7fc5a40b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:56.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.963+0000 7fc5a2ffd700 1 -- 172.21.15.32:0/1104850981 >> v1:172.21.15.32:6789/0 conn(0x7fc5a40af0d0 legacy=0x7fc5a40af4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:56.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.963+0000 7fc5a2ffd700 1 -- 172.21.15.32:0/1104850981 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a40626d0 con 0x7fc5a4142570 2020-12-05T17:38:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.963+0000 7fc5a2ffd700 1 --2- 172.21.15.32:0/1104850981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a4142570 0x7fc5a4142f20 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7fc58c002f30 tx=0x7fc58c006dc0).ready entity=mon.0 client_cookie=1a30d52b8f333d97 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.963+0000 7fc5a27fc700 1 -- 172.21.15.32:0/1104850981 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc58c0097f0 con 0x7fc5a4142570 2020-12-05T17:38:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.963+0000 7fc5a27fc700 1 -- 172.21.15.32:0/1104850981 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58c009950 con 0x7fc5a4142570 2020-12-05T17:38:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.963+0000 7fc5a27fc700 1 -- 172.21.15.32:0/1104850981 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc58c009cf0 con 0x7fc5a4142570 2020-12-05T17:38:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.963+0000 7fc5a8a0c700 1 -- 172.21.15.32:0/1104850981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a4142570 msgr2=0x7fc5a4142f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.963+0000 7fc5a8a0c700 1 --2- 172.21.15.32:0/1104850981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a4142570 0x7fc5a4142f20 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7fc58c002f30 tx=0x7fc58c006dc0).stop 2020-12-05T17:38:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.963+0000 7fc5a8a0c700 1 -- 172.21.15.32:0/1104850981 shutdown_connections 2020-12-05T17:38:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.963+0000 7fc5a8a0c700 1 --2- 172.21.15.32:0/1104850981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a4142570 0x7fc5a4142f20 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.963+0000 7fc5a8a0c700 1 -- 172.21.15.32:0/1104850981 >> 172.21.15.32:0/1104850981 conn(0x7fc5a40b5530 msgr2=0x7fc5a40b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.963+0000 7fc5a8a0c700 1 -- 172.21.15.32:0/1104850981 shutdown_connections 2020-12-05T17:38:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.963+0000 7fc5a8a0c700 1 -- 172.21.15.32:0/1104850981 wait complete. 2020-12-05T17:38:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.967+0000 7fc5a8a0c700 1 Processor -- start 2020-12-05T17:38:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.967+0000 7fc5a8a0c700 1 -- start start 2020-12-05T17:38:56.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a8a0c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc5a4061cf0 0x7fc5a4063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:56.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a8a0c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5a40af0d0 0x7fc5a40ac710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:56.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a8a0c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a4142570 0x7fc5a4146c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:56.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a8a0c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a41517b0 con 0x7fc5a4142570 2020-12-05T17:38:56.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a8a0c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a4063630 con 0x7fc5a4061cf0 2020-12-05T17:38:56.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a8a0c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a41519e0 con 0x7fc5a40af0d0 2020-12-05T17:38:56.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a4142570 0x7fc5a4146c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:56.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a4142570 0x7fc5a4146c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37360/0 (socket says 172.21.15.32:37360) 2020-12-05T17:38:56.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a3fff700 1 -- 172.21.15.32:0/3550078981 learned_addr learned my addr 172.21.15.32:0/3550078981 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:56.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a37fe700 1 --2- 172.21.15.32:0/3550078981 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc5a4061cf0 0x7fc5a4063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:56.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a2ffd700 1 --2- 172.21.15.32:0/3550078981 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5a40af0d0 0x7fc5a40ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:56.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a3fff700 1 -- 172.21.15.32:0/3550078981 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5a40af0d0 msgr2=0x7fc5a40ac710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:56.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a3fff700 1 --2- 172.21.15.32:0/3550078981 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5a40af0d0 0x7fc5a40ac710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:56.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a3fff700 1 -- 172.21.15.32:0/3550078981 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc5a4061cf0 msgr2=0x7fc5a4063000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:56.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a3fff700 1 --2- 172.21.15.32:0/3550078981 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc5a4061cf0 0x7fc5a4063000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:56.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a3fff700 1 -- 172.21.15.32:0/3550078981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a40acc00 con 0x7fc5a4142570 2020-12-05T17:38:56.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a37fe700 1 --2- 172.21.15.32:0/3550078981 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc5a4061cf0 0x7fc5a4063000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:38:56.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a3fff700 1 --2- 172.21.15.32:0/3550078981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a4142570 0x7fc5a4146c60 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7fc5a40608b0 tx=0x7fc59c007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:56.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a0ff9700 1 -- 172.21.15.32:0/3550078981 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc59c0020e0 con 0x7fc5a4142570 2020-12-05T17:38:56.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a0ff9700 1 -- 172.21.15.32:0/3550078981 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59c005b70 con 0x7fc5a4142570 2020-12-05T17:38:56.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a8a0c700 1 -- 172.21.15.32:0/3550078981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a4147150 con 0x7fc5a4142570 2020-12-05T17:38:56.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a0ff9700 1 -- 172.21.15.32:0/3550078981 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc59c007110 con 0x7fc5a4142570 2020-12-05T17:38:56.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a8a0c700 1 -- 172.21.15.32:0/3550078981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a413fc70 con 0x7fc5a4142570 2020-12-05T17:38:56.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a8a0c700 1 -- 172.21.15.32:0/3550078981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a4057050 con 0x7fc5a4142570 2020-12-05T17:38:56.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a0ff9700 1 -- 172.21.15.32:0/3550078981 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc59c010030 con 0x7fc5a4142570 2020-12-05T17:38:56.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a0ff9700 1 --2- 172.21.15.32:0/3550078981 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc588055440 0x7fc588057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:56.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a0ff9700 1 -- 172.21.15.32:0/3550078981 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc59c06e020 con 0x7fc5a4142570 2020-12-05T17:38:56.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.971+0000 7fc5a37fe700 1 --2- 172.21.15.32:0/3550078981 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc588055440 0x7fc588057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:56.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.975+0000 7fc5a37fe700 1 --2- 172.21.15.32:0/3550078981 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc588055440 0x7fc588057900 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7fc59400b000 tx=0x7fc594009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:56.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:56.979+0000 7fc5a0ff9700 1 -- 172.21.15.32:0/3550078981 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc59c0441c0 con 0x7fc5a4142570 2020-12-05T17:38:57.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.123+0000 7fc5a8a0c700 1 -- 172.21.15.32:0/3550078981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5a4063800 con 0x7fc5a4142570 2020-12-05T17:38:57.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.127+0000 7fc5a0ff9700 1 -- 172.21.15.32:0/3550078981 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16287 (secure 0 0 0) 0x7fc59c0477e0 con 0x7fc5a4142570 2020-12-05T17:38:57.132 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:38:57.132 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":82,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:38:56.820563+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:38:54.508932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-12-05T17:38:55.805478+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:38:57.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.127+0000 7fc5a8a0c700 1 -- 172.21.15.32:0/3550078981 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc588055440 msgr2=0x7fc588057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:57.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.127+0000 7fc5a8a0c700 1 --2- 172.21.15.32:0/3550078981 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc588055440 0x7fc588057900 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7fc59400b000 tx=0x7fc594009040).stop 2020-12-05T17:38:57.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.127+0000 7fc5a8a0c700 1 -- 172.21.15.32:0/3550078981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a4142570 msgr2=0x7fc5a4146c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:57.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.127+0000 7fc5a8a0c700 1 --2- 172.21.15.32:0/3550078981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a4142570 0x7fc5a4146c60 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7fc5a40608b0 tx=0x7fc59c007f10).stop 2020-12-05T17:38:57.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.127+0000 7fc5a8a0c700 1 -- 172.21.15.32:0/3550078981 shutdown_connections 2020-12-05T17:38:57.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.127+0000 7fc5a8a0c700 1 --2- 172.21.15.32:0/3550078981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a4142570 0x7fc5a4146c60 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.127+0000 7fc5a8a0c700 1 --2- 172.21.15.32:0/3550078981 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc5a4061cf0 0x7fc5a4063000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.127+0000 7fc5a8a0c700 1 --2- 172.21.15.32:0/3550078981 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc588055440 0x7fc588057900 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.127+0000 7fc5a8a0c700 1 --2- 172.21.15.32:0/3550078981 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5a40af0d0 0x7fc5a40ac710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.127+0000 7fc5a8a0c700 1 -- 172.21.15.32:0/3550078981 >> 172.21.15.32:0/3550078981 conn(0x7fc5a40b5530 msgr2=0x7fc5a413bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:57.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.127+0000 7fc5a8a0c700 1 -- 172.21.15.32:0/3550078981 shutdown_connections 2020-12-05T17:38:57.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.127+0000 7fc5a8a0c700 1 -- 172.21.15.32:0/3550078981 wait complete. 2020-12-05T17:38:57.150 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:38:57.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a23fff700 1 Processor -- start 2020-12-05T17:38:57.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a23fff700 1 -- start start 2020-12-05T17:38:57.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a23fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5a240b1fc0 0x7f5a240af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:57.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a23fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5a24147400 0x7f5a24145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:57.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a23fff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a24146a90 con 0x7f5a240ab710 2020-12-05T17:38:57.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a23fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a24061e90 con 0x7f5a240b1fc0 2020-12-05T17:38:57.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a23fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a240b0110 con 0x7f5a24147400 2020-12-05T17:38:57.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a22ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5a240b1fc0 0x7f5a240af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:57.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a22ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5a240b1fc0 0x7f5a240af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37364/0 (socket says 172.21.15.32:37364) 2020-12-05T17:38:57.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a22ffd700 1 -- 172.21.15.32:0/3167673044 learned_addr learned my addr 172.21.15.32:0/3167673044 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:57.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a237fe700 1 --2- 172.21.15.32:0/3167673044 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5a24147400 0x7f5a24145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:57.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a22ffd700 1 -- 172.21.15.32:0/3167673044 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5a24147400 msgr2=0x7f5a24145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:57.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a22ffd700 1 --2- 172.21.15.32:0/3167673044 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5a24147400 0x7f5a24145d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a22ffd700 1 -- 172.21.15.32:0/3167673044 >> v1:172.21.15.61:6789/0 conn(0x7f5a240ab710 legacy=0x7f5a24061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:57.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a22ffd700 1 -- 172.21.15.32:0/3167673044 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a240b23a0 con 0x7f5a240b1fc0 2020-12-05T17:38:57.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a22ffd700 1 --2- 172.21.15.32:0/3167673044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a240b1fc0 0x7f5a240af230 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7f5a1c002f30 tx=0x7f5a1c006e80).ready entity=mon.0 client_cookie=d1d196e4e6d034e8 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:57.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a21ffb700 1 -- 172.21.15.32:0/3167673044 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a1c0097f0 con 0x7f5a240b1fc0 2020-12-05T17:38:57.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a21ffb700 1 -- 172.21.15.32:0/3167673044 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1c009950 con 0x7f5a240b1fc0 2020-12-05T17:38:57.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a23fff700 1 -- 172.21.15.32:0/3167673044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a240b1fc0 msgr2=0x7f5a240af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:57.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a21ffb700 1 -- 172.21.15.32:0/3167673044 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a1c009cb0 con 0x7f5a240b1fc0 2020-12-05T17:38:57.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a23fff700 1 --2- 172.21.15.32:0/3167673044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a240b1fc0 0x7f5a240af230 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7f5a1c002f30 tx=0x7f5a1c006e80).stop 2020-12-05T17:38:57.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a23fff700 1 -- 172.21.15.32:0/3167673044 shutdown_connections 2020-12-05T17:38:57.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a23fff700 1 --2- 172.21.15.32:0/3167673044 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5a24147400 0x7f5a24145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a23fff700 1 --2- 172.21.15.32:0/3167673044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a240b1fc0 0x7f5a240af230 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a23fff700 1 -- 172.21.15.32:0/3167673044 >> 172.21.15.32:0/3167673044 conn(0x7f5a240b5530 msgr2=0x7f5a240b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:57.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.255+0000 7f5a23fff700 1 -- 172.21.15.32:0/3167673044 shutdown_connections 2020-12-05T17:38:57.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a23fff700 1 -- 172.21.15.32:0/3167673044 wait complete. 2020-12-05T17:38:57.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a23fff700 1 Processor -- start 2020-12-05T17:38:57.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a23fff700 1 -- start start 2020-12-05T17:38:57.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a23fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a240ab710 0x7f5a2414aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:57.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a23fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5a240b1fc0 0x7f5a240b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:57.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a23fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5a24147400 0x7f5a2414b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:57.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a23fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a24147e50 con 0x7f5a240ab710 2020-12-05T17:38:57.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a23fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2414ce80 con 0x7f5a240b1fc0 2020-12-05T17:38:57.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a23fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2414d120 con 0x7f5a24147400 2020-12-05T17:38:57.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a22ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a240ab710 0x7f5a2414aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:57.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a22ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a240ab710 0x7f5a2414aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37372/0 (socket says 172.21.15.32:37372) 2020-12-05T17:38:57.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a22ffd700 1 -- 172.21.15.32:0/3620790222 learned_addr learned my addr 172.21.15.32:0/3620790222 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:57.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a237fe700 1 --2- 172.21.15.32:0/3620790222 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5a24147400 0x7f5a2414b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:57.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a227fc700 1 --2- 172.21.15.32:0/3620790222 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5a240b1fc0 0x7f5a240b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:57.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a22ffd700 1 -- 172.21.15.32:0/3620790222 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5a24147400 msgr2=0x7f5a2414b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:57.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a22ffd700 1 --2- 172.21.15.32:0/3620790222 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5a24147400 0x7f5a2414b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a22ffd700 1 -- 172.21.15.32:0/3620790222 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5a240b1fc0 msgr2=0x7f5a240b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:57.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a22ffd700 1 --2- 172.21.15.32:0/3620790222 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5a240b1fc0 0x7f5a240b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a22ffd700 1 -- 172.21.15.32:0/3620790222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2414dc80 con 0x7f5a240ab710 2020-12-05T17:38:57.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a237fe700 1 --2- 172.21.15.32:0/3620790222 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5a24147400 0x7f5a2414b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:38:57.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a22ffd700 1 --2- 172.21.15.32:0/3620790222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a240ab710 0x7f5a2414aa00 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7f5a1c002f30 tx=0x7f5a1c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:57.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a13fff700 1 -- 172.21.15.32:0/3620790222 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a1c011d30 con 0x7f5a240ab710 2020-12-05T17:38:57.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a13fff700 1 -- 172.21.15.32:0/3620790222 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1c011e90 con 0x7f5a240ab710 2020-12-05T17:38:57.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a23fff700 1 -- 172.21.15.32:0/3620790222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2414de70 con 0x7f5a240ab710 2020-12-05T17:38:57.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a13fff700 1 -- 172.21.15.32:0/3620790222 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a1c01e650 con 0x7f5a240ab710 2020-12-05T17:38:57.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.259+0000 7f5a23fff700 1 -- 172.21.15.32:0/3620790222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a24148460 con 0x7f5a240ab710 2020-12-05T17:38:57.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.263+0000 7f5a23fff700 1 -- 172.21.15.32:0/3620790222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a2414d2e0 con 0x7f5a240ab710 2020-12-05T17:38:57.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.263+0000 7f5a13fff700 1 -- 172.21.15.32:0/3620790222 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5a1c025030 con 0x7f5a240ab710 2020-12-05T17:38:57.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.263+0000 7f5a13fff700 1 --2- 172.21.15.32:0/3620790222 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5a000554e0 0x7f5a000579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:57.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.263+0000 7f5a13fff700 1 -- 172.21.15.32:0/3620790222 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5a1c06f690 con 0x7f5a240ab710 2020-12-05T17:38:57.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.263+0000 7f5a227fc700 1 --2- 172.21.15.32:0/3620790222 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5a000554e0 0x7f5a000579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:57.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.263+0000 7f5a227fc700 1 --2- 172.21.15.32:0/3620790222 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5a000554e0 0x7f5a000579a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f5a1800b000 tx=0x7f5a18009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:57.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.271+0000 7f5a13fff700 1 -- 172.21.15.32:0/3620790222 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5a1c0456a0 con 0x7f5a240ab710 2020-12-05T17:38:57.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.411+0000 7f5a23fff700 1 -- 172.21.15.32:0/3620790222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a24057050 con 0x7f5a240ab710 2020-12-05T17:38:57.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.411+0000 7f5a13fff700 1 -- 172.21.15.32:0/3620790222 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f5a1c023090 con 0x7f5a240ab710 2020-12-05T17:38:57.419 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:38:57.419 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":82,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:38:56.820563+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:38:54.508932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-12-05T17:38:55.805478+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:38:57.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.415+0000 7f5a23fff700 1 -- 172.21.15.32:0/3620790222 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5a000554e0 msgr2=0x7f5a000579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:57.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.415+0000 7f5a23fff700 1 --2- 172.21.15.32:0/3620790222 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5a000554e0 0x7f5a000579a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f5a1800b000 tx=0x7f5a18009040).stop 2020-12-05T17:38:57.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.415+0000 7f5a23fff700 1 -- 172.21.15.32:0/3620790222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a240ab710 msgr2=0x7f5a2414aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:57.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.415+0000 7f5a23fff700 1 --2- 172.21.15.32:0/3620790222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a240ab710 0x7f5a2414aa00 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7f5a1c002f30 tx=0x7f5a1c015000).stop 2020-12-05T17:38:57.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.415+0000 7f5a23fff700 1 -- 172.21.15.32:0/3620790222 shutdown_connections 2020-12-05T17:38:57.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.415+0000 7f5a23fff700 1 --2- 172.21.15.32:0/3620790222 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5a000554e0 0x7f5a000579a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.415+0000 7f5a23fff700 1 --2- 172.21.15.32:0/3620790222 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5a24147400 0x7f5a2414b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.415+0000 7f5a23fff700 1 --2- 172.21.15.32:0/3620790222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a240ab710 0x7f5a2414aa00 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.415+0000 7f5a23fff700 1 --2- 172.21.15.32:0/3620790222 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5a240b1fc0 0x7f5a240b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.415+0000 7f5a23fff700 1 -- 172.21.15.32:0/3620790222 >> 172.21.15.32:0/3620790222 conn(0x7f5a240b5530 msgr2=0x7f5a240af740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:57.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.415+0000 7f5a23fff700 1 -- 172.21.15.32:0/3620790222 shutdown_connections 2020-12-05T17:38:57.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.415+0000 7f5a23fff700 1 -- 172.21.15.32:0/3620790222 wait complete. 2020-12-05T17:38:57.438 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-12-05T17:38:57.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.551+0000 7fb4ff59e700 1 Processor -- start 2020-12-05T17:38:57.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.551+0000 7fb4ff59e700 1 -- start start 2020-12-05T17:38:57.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.551+0000 7fb4ff59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb500061920 0x7fb500064a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:57.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.551+0000 7fb4ff59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb500145e80 con 0x7fb5000b16b0 2020-12-05T17:38:57.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.551+0000 7fb4ff59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb500146120 con 0x7fb500142fc0 2020-12-05T17:38:57.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.551+0000 7fb4ff59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5001463c0 con 0x7fb500061920 2020-12-05T17:38:57.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.551+0000 7fb4f5d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb5000b16b0 0x7fb5001428b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40030/0 (socket says 172.21.15.32:40030) 2020-12-05T17:38:57.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.551+0000 7fb4f5d9b700 1 -- 172.21.15.32:0/3739128807 learned_addr learned my addr 172.21.15.32:0/3739128807 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:38:57.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.551+0000 7fb4fe59c700 1 --2- 172.21.15.32:0/3739128807 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb500061920 0x7fb500064a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.551+0000 7fb4fdd9b700 1 -- 172.21.15.32:0/3739128807 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383283326 0 0) 0x7fb500145e80 con 0x7fb5000b16b0 2020-12-05T17:38:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.551+0000 7fb4fdd9b700 1 -- 172.21.15.32:0/3739128807 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4e0003230 con 0x7fb5000b16b0 2020-12-05T17:38:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.551+0000 7fb4fdd9b700 1 -- 172.21.15.32:0/3739128807 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3410029897 0 0) 0x7fb500146120 con 0x7fb500142fc0 2020-12-05T17:38:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.551+0000 7fb4fdd9b700 1 -- 172.21.15.32:0/3739128807 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb500145e80 con 0x7fb500142fc0 2020-12-05T17:38:57.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.551+0000 7fb4fe59c700 1 -- 172.21.15.32:0/3739128807 >> v1:172.21.15.61:6789/0 conn(0x7fb500142fc0 legacy=0x7fb50013c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:57.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.551+0000 7fb4fe59c700 1 -- 172.21.15.32:0/3739128807 >> v1:172.21.15.32:6789/0 conn(0x7fb5000b16b0 legacy=0x7fb5001428b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:57.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.551+0000 7fb4fe59c700 1 -- 172.21.15.32:0/3739128807 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5000626b0 con 0x7fb500061920 2020-12-05T17:38:57.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.551+0000 7fb4fe59c700 1 --2- 172.21.15.32:0/3739128807 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb500061920 0x7fb500064a10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fb4f0002f30 tx=0x7fb4f0006dc0).ready entity=mon.2 client_cookie=aa5a9c669c3c650f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:57.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.551+0000 7fb4fdd9b700 1 -- 172.21.15.32:0/3739128807 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4f0009730 con 0x7fb500061920 2020-12-05T17:38:57.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.551+0000 7fb4fdd9b700 1 -- 172.21.15.32:0/3739128807 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f0009890 con 0x7fb500061920 2020-12-05T17:38:57.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.551+0000 7fb4fdd9b700 1 -- 172.21.15.32:0/3739128807 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4f0009bf0 con 0x7fb500061920 2020-12-05T17:38:57.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.551+0000 7fb4ff59e700 1 -- 172.21.15.32:0/3739128807 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb500061920 msgr2=0x7fb500064a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:57.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.551+0000 7fb4ff59e700 1 --2- 172.21.15.32:0/3739128807 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb500061920 0x7fb500064a10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fb4f0002f30 tx=0x7fb4f0006dc0).stop 2020-12-05T17:38:57.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.555+0000 7fb4ff59e700 1 -- 172.21.15.32:0/3739128807 shutdown_connections 2020-12-05T17:38:57.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.555+0000 7fb4ff59e700 1 --2- 172.21.15.32:0/3739128807 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb500061920 0x7fb500064a10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.555+0000 7fb4ff59e700 1 -- 172.21.15.32:0/3739128807 >> 172.21.15.32:0/3739128807 conn(0x7fb5000b6d10 msgr2=0x7fb5000b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:57.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.555+0000 7fb4ff59e700 1 -- 172.21.15.32:0/3739128807 shutdown_connections 2020-12-05T17:38:57.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.555+0000 7fb4ff59e700 1 -- 172.21.15.32:0/3739128807 wait complete. 2020-12-05T17:38:57.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.555+0000 7fb4ff59e700 1 Processor -- start 2020-12-05T17:38:57.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.555+0000 7fb4ff59e700 1 -- start start 2020-12-05T17:38:57.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.555+0000 7fb4ff59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb500061920 0x7fb500066310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:57.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.555+0000 7fb4ff59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5000b16b0 0x7fb500154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:57.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.555+0000 7fb4ff59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb500142fc0 0x7fb500140140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:57.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.555+0000 7fb4ff59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50013e310 con 0x7fb5000b16b0 2020-12-05T17:38:57.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.555+0000 7fb4ff59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50013fca0 con 0x7fb500061920 2020-12-05T17:38:57.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.555+0000 7fb4ff59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5001552a0 con 0x7fb500142fc0 2020-12-05T17:38:57.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.559+0000 7fb4f5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5000b16b0 0x7fb500154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:57.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.559+0000 7fb4fed9d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb500142fc0 0x7fb500140140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:57.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.559+0000 7fb4f5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5000b16b0 0x7fb500154db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37388/0 (socket says 172.21.15.32:37388) 2020-12-05T17:38:57.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.559+0000 7fb4fe59c700 1 --2- 172.21.15.32:0/2976223346 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb500061920 0x7fb500066310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:57.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.559+0000 7fb4f5d9b700 1 -- 172.21.15.32:0/2976223346 learned_addr learned my addr 172.21.15.32:0/2976223346 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:57.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.559+0000 7fb4f5d9b700 1 -- 172.21.15.32:0/2976223346 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb500142fc0 msgr2=0x7fb500140140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:57.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.559+0000 7fb4f5d9b700 1 --2- 172.21.15.32:0/2976223346 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb500142fc0 0x7fb500140140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.559+0000 7fb4f5d9b700 1 -- 172.21.15.32:0/2976223346 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb500061920 msgr2=0x7fb500066310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:57.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.559+0000 7fb4f5d9b700 1 --2- 172.21.15.32:0/2976223346 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb500061920 0x7fb500066310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.559+0000 7fb4f5d9b700 1 -- 172.21.15.32:0/2976223346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50013b4f0 con 0x7fb5000b16b0 2020-12-05T17:38:57.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.559+0000 7fb4fed9d700 1 --2- 172.21.15.32:0/2976223346 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb500142fc0 0x7fb500140140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:38:57.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.559+0000 7fb4fe59c700 1 --2- 172.21.15.32:0/2976223346 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb500061920 0x7fb500066310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:38:57.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.559+0000 7fb4f5d9b700 1 --2- 172.21.15.32:0/2976223346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5000b16b0 0x7fb500154db0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7fb4e8001190 tx=0x7fb4e8007b20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:57.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.559+0000 7fb4f7fff700 1 -- 172.21.15.32:0/2976223346 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4e80072e0 con 0x7fb5000b16b0 2020-12-05T17:38:57.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.559+0000 7fb4ff59e700 1 -- 172.21.15.32:0/2976223346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb50013b880 con 0x7fb5000b16b0 2020-12-05T17:38:57.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.559+0000 7fb4f7fff700 1 -- 172.21.15.32:0/2976223346 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e8007440 con 0x7fb5000b16b0 2020-12-05T17:38:57.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.559+0000 7fb4ff59e700 1 -- 172.21.15.32:0/2976223346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5000b44a0 con 0x7fb5000b16b0 2020-12-05T17:38:57.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.559+0000 7fb4f7fff700 1 -- 172.21.15.32:0/2976223346 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4e80055c0 con 0x7fb5000b16b0 2020-12-05T17:38:57.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.559+0000 7fb4ff59e700 1 -- 172.21.15.32:0/2976223346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb50013e170 con 0x7fb5000b16b0 2020-12-05T17:38:57.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.563+0000 7fb4f7fff700 1 -- 172.21.15.32:0/2976223346 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb4e8010070 con 0x7fb5000b16b0 2020-12-05T17:38:57.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.563+0000 7fb4f7fff700 1 --2- 172.21.15.32:0/2976223346 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb4e40554e0 0x7fb4e40579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:57.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.563+0000 7fb4f7fff700 1 -- 172.21.15.32:0/2976223346 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb4e806d640 con 0x7fb5000b16b0 2020-12-05T17:38:57.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.563+0000 7fb4fe59c700 1 --2- 172.21.15.32:0/2976223346 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb4e40554e0 0x7fb4e40579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:57.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.563+0000 7fb4fe59c700 1 --2- 172.21.15.32:0/2976223346 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb4e40554e0 0x7fb4e40579a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fb4f001b000 tx=0x7fb4f0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:57.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.571+0000 7fb4f7fff700 1 -- 172.21.15.32:0/2976223346 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb50013e170 con 0x7fb5000b16b0 2020-12-05T17:38:57.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.755+0000 7fb4ff59e700 1 -- 172.21.15.32:0/2976223346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb50013e170 con 0x7fb5000b16b0 2020-12-05T17:38:57.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.835+0000 7fb4f7fff700 1 -- 172.21.15.32:0/2976223346 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v37) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb50013e170 con 0x7fb5000b16b0 2020-12-05T17:38:57.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.839+0000 7fb4ff59e700 1 -- 172.21.15.32:0/2976223346 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb4e40554e0 msgr2=0x7fb4e40579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:57.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.839+0000 7fb4ff59e700 1 --2- 172.21.15.32:0/2976223346 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb4e40554e0 0x7fb4e40579a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fb4f001b000 tx=0x7fb4f0015000).stop 2020-12-05T17:38:57.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.839+0000 7fb4ff59e700 1 -- 172.21.15.32:0/2976223346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5000b16b0 msgr2=0x7fb500154db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:57.845 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.839+0000 7fb4ff59e700 1 --2- 172.21.15.32:0/2976223346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5000b16b0 0x7fb500154db0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7fb4e8001190 tx=0x7fb4e8007b20).stop 2020-12-05T17:38:57.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.839+0000 7fb4ff59e700 1 -- 172.21.15.32:0/2976223346 shutdown_connections 2020-12-05T17:38:57.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.839+0000 7fb4ff59e700 1 --2- 172.21.15.32:0/2976223346 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb4e40554e0 0x7fb4e40579a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.839+0000 7fb4ff59e700 1 --2- 172.21.15.32:0/2976223346 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb500142fc0 0x7fb500140140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.839+0000 7fb4ff59e700 1 --2- 172.21.15.32:0/2976223346 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb500061920 0x7fb500066310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.839+0000 7fb4ff59e700 1 --2- 172.21.15.32:0/2976223346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5000b16b0 0x7fb500154db0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.839+0000 7fb4ff59e700 1 -- 172.21.15.32:0/2976223346 >> 172.21.15.32:0/2976223346 conn(0x7fb5000b6d10 msgr2=0x7fb500143d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:57.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.839+0000 7fb4ff59e700 1 -- 172.21.15.32:0/2976223346 shutdown_connections 2020-12-05T17:38:57.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.839+0000 7fb4ff59e700 1 -- 172.21.15.32:0/2976223346 wait complete. 2020-12-05T17:38:57.859 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:38:57.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.971+0000 7fa0b35e1700 1 Processor -- start 2020-12-05T17:38:57.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.971+0000 7fa0b35e1700 1 -- start start 2020-12-05T17:38:57.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.971+0000 7fa0b35e1700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa0ac0b7180 0x7fa0ac0ae400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:57.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.971+0000 7fa0b35e1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0ac143f40 0x7fa0ac1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:57.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.971+0000 7fa0b35e1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ac0b1850 con 0x7fa0ac0b3430 2020-12-05T17:38:57.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.971+0000 7fa0b35e1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0ac0b7560 con 0x7fa0ac143f40 2020-12-05T17:38:57.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.971+0000 7fa0b35e1700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0ac0b1ab0 con 0x7fa0ac0b7180 2020-12-05T17:38:57.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.971+0000 7fa0b0b7c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa0ac0b3430 0x7fa0ac0669f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40040/0 (socket says 172.21.15.32:40040) 2020-12-05T17:38:57.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.971+0000 7fa0b0b7c700 1 -- 172.21.15.32:0/80629039 learned_addr learned my addr 172.21.15.32:0/80629039 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:38:57.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.971+0000 7fa0b1b7e700 1 --2- 172.21.15.32:0/80629039 >> v2:172.21.15.32:3300/0 conn(0x7fa0ac143f40 0x7fa0ac1443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:57.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.971+0000 7fa0b137d700 1 --2- 172.21.15.32:0/80629039 >> v2:172.21.15.61:3300/0 conn(0x7fa0ac0b7180 0x7fa0ac0ae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:57.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.971+0000 7fa0a3fff700 1 -- 172.21.15.32:0/80629039 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750120019 0 0) 0x7fa0ac0b1850 con 0x7fa0ac0b3430 2020-12-05T17:38:57.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.971+0000 7fa0a3fff700 1 -- 172.21.15.32:0/80629039 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa090003230 con 0x7fa0ac0b3430 2020-12-05T17:38:57.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.971+0000 7fa0b1b7e700 1 -- 172.21.15.32:0/80629039 >> v2:172.21.15.61:3300/0 conn(0x7fa0ac0b7180 msgr2=0x7fa0ac0ae400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:57.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.971+0000 7fa0b1b7e700 1 --2- 172.21.15.32:0/80629039 >> v2:172.21.15.61:3300/0 conn(0x7fa0ac0b7180 0x7fa0ac0ae400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.971+0000 7fa0b1b7e700 1 -- 172.21.15.32:0/80629039 >> v1:172.21.15.32:6789/0 conn(0x7fa0ac0b3430 legacy=0x7fa0ac0669f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:57.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.971+0000 7fa0b1b7e700 1 -- 172.21.15.32:0/80629039 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ac066550 con 0x7fa0ac143f40 2020-12-05T17:38:57.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.975+0000 7fa0b1b7e700 1 --2- 172.21.15.32:0/80629039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0ac143f40 0x7fa0ac1443f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7fa0a40032b0 tx=0x7fa0a40091a0).ready entity=mon.0 client_cookie=742910311729808f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:57.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.975+0000 7fa0a3fff700 1 -- 172.21.15.32:0/80629039 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0a400b940 con 0x7fa0ac143f40 2020-12-05T17:38:57.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.975+0000 7fa0a3fff700 1 -- 172.21.15.32:0/80629039 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a400baa0 con 0x7fa0ac143f40 2020-12-05T17:38:57.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.975+0000 7fa0a3fff700 1 -- 172.21.15.32:0/80629039 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0a400bc10 con 0x7fa0ac143f40 2020-12-05T17:38:57.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.975+0000 7fa0b35e1700 1 -- 172.21.15.32:0/80629039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0ac143f40 msgr2=0x7fa0ac1443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:57.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.975+0000 7fa0b35e1700 1 --2- 172.21.15.32:0/80629039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0ac143f40 0x7fa0ac1443f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7fa0a40032b0 tx=0x7fa0a40091a0).stop 2020-12-05T17:38:57.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.975+0000 7fa0b35e1700 1 -- 172.21.15.32:0/80629039 shutdown_connections 2020-12-05T17:38:57.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.975+0000 7fa0b35e1700 1 --2- 172.21.15.32:0/80629039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0ac143f40 0x7fa0ac1443f0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.975+0000 7fa0b35e1700 1 --2- 172.21.15.32:0/80629039 >> v2:172.21.15.61:3300/0 conn(0x7fa0ac0b7180 0x7fa0ac0ae400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.975+0000 7fa0b35e1700 1 -- 172.21.15.32:0/80629039 >> 172.21.15.32:0/80629039 conn(0x7fa0ac067820 msgr2=0x7fa0ac05c960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:57.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.975+0000 7fa0b35e1700 1 -- 172.21.15.32:0/80629039 shutdown_connections 2020-12-05T17:38:57.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.975+0000 7fa0b35e1700 1 -- 172.21.15.32:0/80629039 wait complete. 2020-12-05T17:38:57.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.975+0000 7fa0b35e1700 1 Processor -- start 2020-12-05T17:38:57.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.975+0000 7fa0b35e1700 1 -- start start 2020-12-05T17:38:57.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.979+0000 7fa0b35e1700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0ac0b3430 0x7fa0ac142dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:57.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.979+0000 7fa0b35e1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0ac0b7180 0x7fa0ac146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:57.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.979+0000 7fa0b35e1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0ac143f40 0x7fa0ac146a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:57.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.979+0000 7fa0b35e1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac146f10 con 0x7fa0ac143f40 2020-12-05T17:38:57.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.979+0000 7fa0b35e1700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac0a9f10 con 0x7fa0ac0b3430 2020-12-05T17:38:57.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.979+0000 7fa0b35e1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac146090 con 0x7fa0ac0b7180 2020-12-05T17:38:57.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.979+0000 7fa0b1b7e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0ac143f40 0x7fa0ac146a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:57.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.979+0000 7fa0b1b7e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0ac143f40 0x7fa0ac146a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37402/0 (socket says 172.21.15.32:37402) 2020-12-05T17:38:57.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.979+0000 7fa0b1b7e700 1 -- 172.21.15.32:0/3083394990 learned_addr learned my addr 172.21.15.32:0/3083394990 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:57.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.979+0000 7fa0b137d700 1 --2- 172.21.15.32:0/3083394990 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0ac0b3430 0x7fa0ac142dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.979+0000 7fa0b0b7c700 1 --2- 172.21.15.32:0/3083394990 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0ac0b7180 0x7fa0ac146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.979+0000 7fa0b1b7e700 1 -- 172.21.15.32:0/3083394990 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0ac0b7180 msgr2=0x7fa0ac146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.979+0000 7fa0b1b7e700 1 --2- 172.21.15.32:0/3083394990 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0ac0b7180 0x7fa0ac146530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.979+0000 7fa0b1b7e700 1 -- 172.21.15.32:0/3083394990 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0ac0b3430 msgr2=0x7fa0ac142dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.979+0000 7fa0b1b7e700 1 --2- 172.21.15.32:0/3083394990 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0ac0b3430 0x7fa0ac142dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.979+0000 7fa0b1b7e700 1 -- 172.21.15.32:0/3083394990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ac0aa1b0 con 0x7fa0ac143f40 2020-12-05T17:38:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.979+0000 7fa0b1b7e700 1 --2- 172.21.15.32:0/3083394990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0ac143f40 0x7fa0ac146a20 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7fa0a40032b0 tx=0x7fa0a4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.979+0000 7fa0a27fc700 1 -- 172.21.15.32:0/3083394990 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0a4011ea0 con 0x7fa0ac143f40 2020-12-05T17:38:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.979+0000 7fa0a27fc700 1 -- 172.21.15.32:0/3083394990 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a4020400 con 0x7fa0ac143f40 2020-12-05T17:38:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.979+0000 7fa0b35e1700 1 -- 172.21.15.32:0/3083394990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0ac0a9b90 con 0x7fa0ac143f40 2020-12-05T17:38:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.979+0000 7fa0a27fc700 1 -- 172.21.15.32:0/3083394990 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0a4020760 con 0x7fa0ac143f40 2020-12-05T17:38:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.979+0000 7fa0b35e1700 1 -- 172.21.15.32:0/3083394990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ac09f220 con 0x7fa0ac143f40 2020-12-05T17:38:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.979+0000 7fa0b35e1700 1 -- 172.21.15.32:0/3083394990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0ac146250 con 0x7fa0ac143f40 2020-12-05T17:38:57.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.983+0000 7fa0a27fc700 1 -- 172.21.15.32:0/3083394990 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa0a4027030 con 0x7fa0ac143f40 2020-12-05T17:38:57.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.983+0000 7fa0a27fc700 1 --2- 172.21.15.32:0/3083394990 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa0940553b0 0x7fa094057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:57.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.983+0000 7fa0a27fc700 1 -- 172.21.15.32:0/3083394990 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa0a406ee60 con 0x7fa0ac143f40 2020-12-05T17:38:57.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.983+0000 7fa0b137d700 1 --2- 172.21.15.32:0/3083394990 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa0940553b0 0x7fa094057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:57.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.983+0000 7fa0b137d700 1 --2- 172.21.15.32:0/3083394990 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa0940553b0 0x7fa094057870 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7fa09c006000 tx=0x7fa09c006d60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:57.991+0000 7fa0a27fc700 1 -- 172.21.15.32:0/3083394990 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa0a4045000 con 0x7fa0ac143f40 2020-12-05T17:38:58.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.187+0000 7fa0b35e1700 1 -- 172.21.15.32:0/3083394990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0ac057050 con 0x7fa0ac143f40 2020-12-05T17:38:58.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.187+0000 7fa0a27fc700 1 -- 172.21.15.32:0/3083394990 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+2134 (secure 0 0 0) 0x7fa0a4025090 con 0x7fa0ac143f40 2020-12-05T17:38:58.194 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:38:58.194 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":37,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6535,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2494768218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2494768218},{"type":"v1","addr":"172.21.15.61:6833","nonce":2494768218}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":35}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:38:56.821107+0000","modified":"2020-12-05T17:38:57.824754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/81037173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":81037173},{"type":"v1","addr":"172.21.15.32:6835","nonce":81037173}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-12-05T17:38:58.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.191+0000 7fa0b35e1700 1 -- 172.21.15.32:0/3083394990 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa0940553b0 msgr2=0x7fa094057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:58.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.191+0000 7fa0b35e1700 1 --2- 172.21.15.32:0/3083394990 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa0940553b0 0x7fa094057870 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7fa09c006000 tx=0x7fa09c006d60).stop 2020-12-05T17:38:58.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.191+0000 7fa0b35e1700 1 -- 172.21.15.32:0/3083394990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0ac143f40 msgr2=0x7fa0ac146a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:58.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.191+0000 7fa0b35e1700 1 --2- 172.21.15.32:0/3083394990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0ac143f40 0x7fa0ac146a20 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7fa0a40032b0 tx=0x7fa0a4016000).stop 2020-12-05T17:38:58.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.191+0000 7fa0b35e1700 1 -- 172.21.15.32:0/3083394990 shutdown_connections 2020-12-05T17:38:58.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.191+0000 7fa0b35e1700 1 --2- 172.21.15.32:0/3083394990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0ac143f40 0x7fa0ac146a20 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.191+0000 7fa0b35e1700 1 --2- 172.21.15.32:0/3083394990 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0ac0b3430 0x7fa0ac142dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.191+0000 7fa0b35e1700 1 --2- 172.21.15.32:0/3083394990 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa0940553b0 0x7fa094057870 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.191+0000 7fa0b35e1700 1 --2- 172.21.15.32:0/3083394990 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0ac0b7180 0x7fa0ac146530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.191+0000 7fa0b35e1700 1 -- 172.21.15.32:0/3083394990 >> 172.21.15.32:0/3083394990 conn(0x7fa0ac067820 msgr2=0x7fa0ac0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:58.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.191+0000 7fa0b35e1700 1 -- 172.21.15.32:0/3083394990 shutdown_connections 2020-12-05T17:38:58.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.191+0000 7fa0b35e1700 1 -- 172.21.15.32:0/3083394990 wait complete. 2020-12-05T17:38:58.199 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 37 2020-12-05T17:38:58.208 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:38:58.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f847b206700 1 Processor -- start 2020-12-05T17:38:58.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f847b206700 1 -- start start 2020-12-05T17:38:58.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f847b206700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f84740b2140 0x7f84740af170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:58.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f847b206700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f84741441e0 0x7f8474144640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:58.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f847b206700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8474143b10 con 0x7f84740aed90 2020-12-05T17:38:58.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f847b206700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f847413df90 con 0x7f84740b2140 2020-12-05T17:38:58.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f847b206700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84740b1c30 con 0x7f84741441e0 2020-12-05T17:38:58.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f847a204700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f84740aed90 0x7f84741404a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40054/0 (socket says 172.21.15.32:40054) 2020-12-05T17:38:58.323 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f847a204700 1 -- 172.21.15.32:0/3186694418 learned_addr learned my addr 172.21.15.32:0/3186694418 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:38:58.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f847aa05700 1 --2- 172.21.15.32:0/3186694418 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f84741441e0 0x7f8474144640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:58.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f8479a03700 1 --2- 172.21.15.32:0/3186694418 >> v2:172.21.15.61:3300/0 conn(0x7f84740b2140 0x7f84740af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:58.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f8479202700 1 -- 172.21.15.32:0/3186694418 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2253908269 0 0) 0x7f8474143b10 con 0x7f84740aed90 2020-12-05T17:38:58.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f8479202700 1 -- 172.21.15.32:0/3186694418 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f845c003230 con 0x7f84740aed90 2020-12-05T17:38:58.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f8479202700 1 -- 172.21.15.32:0/3186694418 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1810690382 0 0) 0x7f845c003230 con 0x7f84740aed90 2020-12-05T17:38:58.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f8479202700 1 -- 172.21.15.32:0/3186694418 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f84741441e0 msgr2=0x7f8474144640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:58.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f8479202700 1 --2- 172.21.15.32:0/3186694418 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f84741441e0 0x7f8474144640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f8479202700 1 -- 172.21.15.32:0/3186694418 >> v2:172.21.15.61:3300/0 conn(0x7f84740b2140 msgr2=0x7f84740af170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:58.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f8479202700 1 --2- 172.21.15.32:0/3186694418 >> v2:172.21.15.61:3300/0 conn(0x7f84740b2140 0x7f84740af170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f8479202700 1 -- 172.21.15.32:0/3186694418 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84740b2650 con 0x7f84740aed90 2020-12-05T17:38:58.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f8479202700 1 -- 172.21.15.32:0/3186694418 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f8470001a50 con 0x7f84740aed90 2020-12-05T17:38:58.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f8479202700 1 -- 172.21.15.32:0/3186694418 >> v1:172.21.15.32:6789/0 conn(0x7f84740aed90 legacy=0x7f84741404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:58.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f8479202700 1 --2- 172.21.15.32:0/3186694418 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f845c003720 0x7f845c003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:58.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f8479202700 1 --2- 172.21.15.32:0/3186694418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f845c0040a0 0x7f845c004570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:58.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f8479202700 1 --2- 172.21.15.32:0/3186694418 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f845c004b70 0x7f845c004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:58.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f8479202700 1 -- 172.21.15.32:0/3186694418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847413df90 con 0x7f845c0040a0 2020-12-05T17:38:58.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f8479202700 1 -- 172.21.15.32:0/3186694418 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84740b1c30 con 0x7f845c004b70 2020-12-05T17:38:58.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f8479202700 1 -- 172.21.15.32:0/3186694418 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845c00b8f0 con 0x7f845c003720 2020-12-05T17:38:58.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f8479a03700 1 --2- 172.21.15.32:0/3186694418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f845c0040a0 0x7f845c004570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:58.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f847aa05700 1 --2- 172.21.15.32:0/3186694418 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f84741441e0 0x7f8474144640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:38:58.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.315+0000 7f847a204700 1 --2- 172.21.15.32:0/3186694418 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f845c003720 0x7f845c003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:58.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.319+0000 7f847aa05700 1 --2- 172.21.15.32:0/3186694418 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f845c004b70 0x7f845c004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:58.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.319+0000 7f8479a03700 1 -- 172.21.15.32:0/3186694418 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f845c003720 msgr2=0x7f845c003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:58.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.319+0000 7f8479a03700 1 --2- 172.21.15.32:0/3186694418 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f845c003720 0x7f845c003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.319+0000 7f8479a03700 1 -- 172.21.15.32:0/3186694418 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f845c004b70 msgr2=0x7f845c004ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:58.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.319+0000 7f8479a03700 1 --2- 172.21.15.32:0/3186694418 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f845c004b70 0x7f845c004ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.319+0000 7f8479a03700 1 -- 172.21.15.32:0/3186694418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845c003230 con 0x7f845c0040a0 2020-12-05T17:38:58.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.319+0000 7f8479a03700 1 --2- 172.21.15.32:0/3186694418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f845c0040a0 0x7f845c004570 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f846400c000 tx=0x7f8464008040).ready entity=mon.0 client_cookie=fb8fdf1a56c4e156 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:58.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.319+0000 7f8479202700 1 -- 172.21.15.32:0/3186694418 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f846400e970 con 0x7f845c0040a0 2020-12-05T17:38:58.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.319+0000 7f8479202700 1 -- 172.21.15.32:0/3186694418 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846400ead0 con 0x7f845c0040a0 2020-12-05T17:38:58.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.319+0000 7f8479202700 1 -- 172.21.15.32:0/3186694418 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f846400ee30 con 0x7f845c0040a0 2020-12-05T17:38:58.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.319+0000 7f847b206700 1 -- 172.21.15.32:0/3186694418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f845c0040a0 msgr2=0x7f845c004570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:58.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.319+0000 7f847b206700 1 --2- 172.21.15.32:0/3186694418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f845c0040a0 0x7f845c004570 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f846400c000 tx=0x7f8464008040).stop 2020-12-05T17:38:58.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.319+0000 7f847aa05700 1 -- 172.21.15.32:0/3186694418 reap_dead start 2020-12-05T17:38:58.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.319+0000 7f847b206700 1 -- 172.21.15.32:0/3186694418 shutdown_connections 2020-12-05T17:38:58.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.319+0000 7f847b206700 1 -- 172.21.15.32:0/3186694418 >> 172.21.15.32:0/3186694418 conn(0x7f847405bbb0 msgr2=0x7f8474062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:58.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.319+0000 7f847b206700 1 -- 172.21.15.32:0/3186694418 shutdown_connections 2020-12-05T17:38:58.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.319+0000 7f847b206700 1 -- 172.21.15.32:0/3186694418 wait complete. 2020-12-05T17:38:58.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.319+0000 7f847b206700 1 Processor -- start 2020-12-05T17:38:58.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.319+0000 7f847b206700 1 -- start start 2020-12-05T17:38:58.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.323+0000 7f847b206700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8474142ce0 0x7f84741526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:58.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.323+0000 7f847b206700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8474066000 0x7f84741407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:58.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.323+0000 7f847b206700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8474140cb0 0x7f8474139820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:58.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.323+0000 7f847b206700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84741403d0 con 0x7f8474140cb0 2020-12-05T17:38:58.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.323+0000 7f847b206700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84741430c0 con 0x7f8474142ce0 2020-12-05T17:38:58.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.323+0000 7f847b206700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847413e180 con 0x7f8474066000 2020-12-05T17:38:58.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.323+0000 7f847aa05700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8474140cb0 0x7f8474139820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:58.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.323+0000 7f847aa05700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8474140cb0 0x7f8474139820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37422/0 (socket says 172.21.15.32:37422) 2020-12-05T17:38:58.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.323+0000 7f847aa05700 1 -- 172.21.15.32:0/4067228108 learned_addr learned my addr 172.21.15.32:0/4067228108 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:58.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.323+0000 7f8479a03700 1 --2- 172.21.15.32:0/4067228108 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8474066000 0x7f84741407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:58.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.323+0000 7f847a204700 1 --2- 172.21.15.32:0/4067228108 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8474142ce0 0x7f84741526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:58.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.323+0000 7f847aa05700 1 -- 172.21.15.32:0/4067228108 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8474066000 msgr2=0x7f84741407c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:58.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.323+0000 7f847aa05700 1 --2- 172.21.15.32:0/4067228108 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8474066000 0x7f84741407c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.323+0000 7f847aa05700 1 -- 172.21.15.32:0/4067228108 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8474142ce0 msgr2=0x7f84741526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:58.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.323+0000 7f847aa05700 1 --2- 172.21.15.32:0/4067228108 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8474142ce0 0x7f84741526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.323+0000 7f847aa05700 1 -- 172.21.15.32:0/4067228108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8474064d20 con 0x7f8474140cb0 2020-12-05T17:38:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.323+0000 7f847aa05700 1 --2- 172.21.15.32:0/4067228108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8474140cb0 0x7f8474139820 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f846c00b000 tx=0x7f846c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.323+0000 7f84637fe700 1 -- 172.21.15.32:0/4067228108 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f846c00e8a0 con 0x7f8474140cb0 2020-12-05T17:38:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.323+0000 7f84637fe700 1 -- 172.21.15.32:0/4067228108 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846c00ea00 con 0x7f8474140cb0 2020-12-05T17:38:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.323+0000 7f847b206700 1 -- 172.21.15.32:0/4067228108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8474064fc0 con 0x7f8474140cb0 2020-12-05T17:38:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.323+0000 7f84637fe700 1 -- 172.21.15.32:0/4067228108 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f846c00eda0 con 0x7f8474140cb0 2020-12-05T17:38:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.323+0000 7f847b206700 1 -- 172.21.15.32:0/4067228108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8474141140 con 0x7f8474140cb0 2020-12-05T17:38:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.323+0000 7f847b206700 1 -- 172.21.15.32:0/4067228108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8474057050 con 0x7f8474140cb0 2020-12-05T17:38:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.327+0000 7f84637fe700 1 -- 172.21.15.32:0/4067228108 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f846c014470 con 0x7f8474140cb0 2020-12-05T17:38:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.327+0000 7f84637fe700 1 --2- 172.21.15.32:0/4067228108 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f84540553b0 0x7f8454057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.327+0000 7f84637fe700 1 -- 172.21.15.32:0/4067228108 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f846c06e9a0 con 0x7f8474140cb0 2020-12-05T17:38:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.327+0000 7f847a204700 1 --2- 172.21.15.32:0/4067228108 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f84540553b0 0x7f8454057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.327+0000 7f847a204700 1 --2- 172.21.15.32:0/4067228108 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f84540553b0 0x7f8454057870 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f8470005000 tx=0x7f84740b27e0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:58.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.331+0000 7f84637fe700 1 -- 172.21.15.32:0/4067228108 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f846c044ae0 con 0x7f8474140cb0 2020-12-05T17:38:58.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.471+0000 7f847b206700 1 -- 172.21.15.32:0/4067228108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f847405c300 con 0x7f8474140cb0 2020-12-05T17:38:58.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.471+0000 7f84637fe700 1 -- 172.21.15.32:0/4067228108 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f846c048100 con 0x7f8474140cb0 2020-12-05T17:38:58.479 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:38:58.479 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":83,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:38:57.823795+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:38:54.508932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-12-05T17:38:55.805478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:38:58.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.475+0000 7f847b206700 1 -- 172.21.15.32:0/4067228108 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f84540553b0 msgr2=0x7f8454057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:58.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.475+0000 7f847b206700 1 --2- 172.21.15.32:0/4067228108 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f84540553b0 0x7f8454057870 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f8470005000 tx=0x7f84740b27e0).stop 2020-12-05T17:38:58.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.475+0000 7f847b206700 1 -- 172.21.15.32:0/4067228108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8474140cb0 msgr2=0x7f8474139820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:58.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.475+0000 7f847b206700 1 --2- 172.21.15.32:0/4067228108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8474140cb0 0x7f8474139820 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f846c00b000 tx=0x7f846c008040).stop 2020-12-05T17:38:58.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.475+0000 7f847b206700 1 -- 172.21.15.32:0/4067228108 shutdown_connections 2020-12-05T17:38:58.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.475+0000 7f847b206700 1 --2- 172.21.15.32:0/4067228108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8474140cb0 0x7f8474139820 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.475+0000 7f847b206700 1 --2- 172.21.15.32:0/4067228108 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8474142ce0 0x7f84741526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.475+0000 7f847b206700 1 --2- 172.21.15.32:0/4067228108 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f84540553b0 0x7f8454057870 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.475+0000 7f847b206700 1 --2- 172.21.15.32:0/4067228108 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8474066000 0x7f84741407c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.475+0000 7f847b206700 1 -- 172.21.15.32:0/4067228108 >> 172.21.15.32:0/4067228108 conn(0x7f847405bbb0 msgr2=0x7f84740669e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:58.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.475+0000 7f847b206700 1 -- 172.21.15.32:0/4067228108 shutdown_connections 2020-12-05T17:38:58.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.475+0000 7f847b206700 1 -- 172.21.15.32:0/4067228108 wait complete. 2020-12-05T17:38:58.498 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:38:58.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.611+0000 7f151f59e700 1 Processor -- start 2020-12-05T17:38:58.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.611+0000 7f151f59e700 1 -- start start 2020-12-05T17:38:58.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.611+0000 7f151f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15200611b0 0x7f1520153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:58.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.611+0000 7f151f59e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f15200b0240 0x7f15200ae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:58.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.611+0000 7f151f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f152014c340 0x7f152014c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:58.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.611+0000 7f151f59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1520080a90 con 0x7f152014c340 2020-12-05T17:38:58.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.611+0000 7f151f59e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1520152860 con 0x7f15200b0240 2020-12-05T17:38:58.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.611+0000 7f151f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1520152b00 con 0x7f15200611b0 2020-12-05T17:38:58.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.611+0000 7f1524b98700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f152014c340 0x7f152014c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:58.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.611+0000 7f1524b98700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f152014c340 0x7f152014c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37430/0 (socket says 172.21.15.32:37430) 2020-12-05T17:38:58.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.611+0000 7f1524b98700 1 -- 172.21.15.32:0/1448584358 learned_addr learned my addr 172.21.15.32:0/1448584358 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:58.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.611+0000 7f151ed9d700 1 --2- 172.21.15.32:0/1448584358 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15200611b0 0x7f1520153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:58.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.611+0000 7f151e59c700 1 --2- 172.21.15.32:0/1448584358 >> v2:172.21.15.61:3300/0 conn(0x7f15200b0240 0x7f15200ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:58.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.611+0000 7f1524b98700 1 -- 172.21.15.32:0/1448584358 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15200611b0 msgr2=0x7f1520153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:58.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.611+0000 7f1524b98700 1 --2- 172.21.15.32:0/1448584358 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15200611b0 0x7f1520153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.611+0000 7f1524b98700 1 -- 172.21.15.32:0/1448584358 >> v2:172.21.15.61:3300/0 conn(0x7f15200b0240 msgr2=0x7f15200ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:58.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.611+0000 7f1524b98700 1 --2- 172.21.15.32:0/1448584358 >> v2:172.21.15.61:3300/0 conn(0x7f15200b0240 0x7f15200ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.611+0000 7f1524b98700 1 -- 172.21.15.32:0/1448584358 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1520152da0 con 0x7f152014c340 2020-12-05T17:38:58.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.615+0000 7f1524b98700 1 --2- 172.21.15.32:0/1448584358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f152014c340 0x7f152014c790 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7f15180032b0 tx=0x7f1518009270).ready entity=mon.0 client_cookie=1eeb8f1d7a53d865 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:58.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.615+0000 7f151dd9b700 1 -- 172.21.15.32:0/1448584358 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1518009e60 con 0x7f152014c340 2020-12-05T17:38:58.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.615+0000 7f151dd9b700 1 -- 172.21.15.32:0/1448584358 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151800bb50 con 0x7f152014c340 2020-12-05T17:38:58.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.615+0000 7f151dd9b700 1 -- 172.21.15.32:0/1448584358 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f151800bcc0 con 0x7f152014c340 2020-12-05T17:38:58.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.615+0000 7f151f59e700 1 -- 172.21.15.32:0/1448584358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f152014c340 msgr2=0x7f152014c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:58.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.615+0000 7f151f59e700 1 --2- 172.21.15.32:0/1448584358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f152014c340 0x7f152014c790 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7f15180032b0 tx=0x7f1518009270).stop 2020-12-05T17:38:58.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.615+0000 7f151f59e700 1 -- 172.21.15.32:0/1448584358 shutdown_connections 2020-12-05T17:38:58.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.615+0000 7f151f59e700 1 --2- 172.21.15.32:0/1448584358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f152014c340 0x7f152014c790 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.615+0000 7f151f59e700 1 --2- 172.21.15.32:0/1448584358 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15200611b0 0x7f1520153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.615+0000 7f151f59e700 1 --2- 172.21.15.32:0/1448584358 >> v2:172.21.15.61:3300/0 conn(0x7f15200b0240 0x7f15200ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.615+0000 7f151f59e700 1 -- 172.21.15.32:0/1448584358 >> 172.21.15.32:0/1448584358 conn(0x7f15200b5f90 msgr2=0x7f15200b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:58.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.615+0000 7f151f59e700 1 -- 172.21.15.32:0/1448584358 shutdown_connections 2020-12-05T17:38:58.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.615+0000 7f151f59e700 1 -- 172.21.15.32:0/1448584358 wait complete. 2020-12-05T17:38:58.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.615+0000 7f151f59e700 1 Processor -- start 2020-12-05T17:38:58.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.615+0000 7f151f59e700 1 -- start start 2020-12-05T17:38:58.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.619+0000 7f151f59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15200611b0 0x7f1520066e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:58.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.619+0000 7f151f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15200b0240 0x7f152011baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:58.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.619+0000 7f151f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f152014c340 0x7f152011e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:58.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.619+0000 7f151f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152011f220 con 0x7f152014c340 2020-12-05T17:38:58.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.619+0000 7f151f59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152011f4c0 con 0x7f15200611b0 2020-12-05T17:38:58.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.619+0000 7f151f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15200b0930 con 0x7f15200b0240 2020-12-05T17:38:58.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.619+0000 7f1524b98700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f152014c340 0x7f152011e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:58.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.619+0000 7f1524b98700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f152014c340 0x7f152011e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37434/0 (socket says 172.21.15.32:37434) 2020-12-05T17:38:58.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.619+0000 7f151ed9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15200611b0 0x7f1520066e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:58.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.619+0000 7f1524b98700 1 -- 172.21.15.32:0/2141518237 learned_addr learned my addr 172.21.15.32:0/2141518237 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:58.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.619+0000 7f151e59c700 1 --2- 172.21.15.32:0/2141518237 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15200b0240 0x7f152011baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:58.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.619+0000 7f1524b98700 1 -- 172.21.15.32:0/2141518237 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15200b0240 msgr2=0x7f152011baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:58.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.619+0000 7f1524b98700 1 --2- 172.21.15.32:0/2141518237 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15200b0240 0x7f152011baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.619+0000 7f1524b98700 1 -- 172.21.15.32:0/2141518237 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15200611b0 msgr2=0x7f1520066e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:58.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.619+0000 7f1524b98700 1 --2- 172.21.15.32:0/2141518237 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15200611b0 0x7f1520066e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.619+0000 7f1524b98700 1 -- 172.21.15.32:0/2141518237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152007f9c0 con 0x7f152014c340 2020-12-05T17:38:58.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.619+0000 7f1524b98700 1 --2- 172.21.15.32:0/2141518237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f152014c340 0x7f152011e690 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7f15180032b0 tx=0x7f1518016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:58.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.619+0000 7f150f7fe700 1 -- 172.21.15.32:0/2141518237 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f151801f3f0 con 0x7f152014c340 2020-12-05T17:38:58.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.619+0000 7f150f7fe700 1 -- 172.21.15.32:0/2141518237 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151801f550 con 0x7f152014c340 2020-12-05T17:38:58.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.619+0000 7f151f59e700 1 -- 172.21.15.32:0/2141518237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1520063160 con 0x7f152014c340 2020-12-05T17:38:58.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.619+0000 7f150f7fe700 1 -- 172.21.15.32:0/2141518237 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f151801f8b0 con 0x7f152014c340 2020-12-05T17:38:58.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.619+0000 7f151f59e700 1 -- 172.21.15.32:0/2141518237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152011bfe0 con 0x7f152014c340 2020-12-05T17:38:58.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.623+0000 7f151f59e700 1 -- 172.21.15.32:0/2141518237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1520057050 con 0x7f152014c340 2020-12-05T17:38:58.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.623+0000 7f150f7fe700 1 -- 172.21.15.32:0/2141518237 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1518026030 con 0x7f152014c340 2020-12-05T17:38:58.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.623+0000 7f150f7fe700 1 --2- 172.21.15.32:0/2141518237 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15040553b0 0x7f1504057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:58.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.623+0000 7f151ed9d700 1 --2- 172.21.15.32:0/2141518237 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15040553b0 0x7f1504057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:58.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.623+0000 7f150f7fe700 1 -- 172.21.15.32:0/2141518237 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f151806ecc0 con 0x7f152014c340 2020-12-05T17:38:58.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.623+0000 7f151ed9d700 1 --2- 172.21.15.32:0/2141518237 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15040553b0 0x7f1504057870 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f151000b000 tx=0x7f1510006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:58.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.631+0000 7f150f7fe700 1 -- 172.21.15.32:0/2141518237 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1518044d90 con 0x7f152014c340 2020-12-05T17:38:58.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.815+0000 7f151f59e700 1 -- 172.21.15.32:0/2141518237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f15200b6da0 con 0x7f152014c340 2020-12-05T17:38:58.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.823+0000 7f150f7fe700 1 -- 172.21.15.32:0/2141518237 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v26) v1 ==== 184+0+0 (secure 0 0 0) 0x7f15180150e0 con 0x7f152014c340 2020-12-05T17:38:58.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.827+0000 7f151f59e700 1 -- 172.21.15.32:0/2141518237 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15040553b0 msgr2=0x7f1504057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:58.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.827+0000 7f151f59e700 1 --2- 172.21.15.32:0/2141518237 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15040553b0 0x7f1504057870 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f151000b000 tx=0x7f1510006dc0).stop 2020-12-05T17:38:58.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.827+0000 7f151f59e700 1 -- 172.21.15.32:0/2141518237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f152014c340 msgr2=0x7f152011e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:58.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.827+0000 7f151f59e700 1 --2- 172.21.15.32:0/2141518237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f152014c340 0x7f152011e690 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7f15180032b0 tx=0x7f1518016000).stop 2020-12-05T17:38:58.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.827+0000 7f151f59e700 1 -- 172.21.15.32:0/2141518237 shutdown_connections 2020-12-05T17:38:58.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.827+0000 7f151f59e700 1 --2- 172.21.15.32:0/2141518237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f152014c340 0x7f152011e690 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.827+0000 7f151f59e700 1 --2- 172.21.15.32:0/2141518237 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15200611b0 0x7f1520066e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.827+0000 7f151f59e700 1 --2- 172.21.15.32:0/2141518237 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15040553b0 0x7f1504057870 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.827+0000 7f151f59e700 1 --2- 172.21.15.32:0/2141518237 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15200b0240 0x7f152011baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.827+0000 7f151f59e700 1 -- 172.21.15.32:0/2141518237 >> 172.21.15.32:0/2141518237 conn(0x7f15200b5f90 msgr2=0x7f152014bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:58.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.827+0000 7f151f59e700 1 -- 172.21.15.32:0/2141518237 shutdown_connections 2020-12-05T17:38:58.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.827+0000 7f151f59e700 1 -- 172.21.15.32:0/2141518237 wait complete. 2020-12-05T17:38:58.835 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2020-12-05T17:38:58.845 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:38:58.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.963+0000 7f0e9e004700 1 Processor -- start 2020-12-05T17:38:58.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.963+0000 7f0e9e004700 1 -- start start 2020-12-05T17:38:58.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.963+0000 7f0e9e004700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e980611b0 0x7f0e98153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:58.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.963+0000 7f0e9e004700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0e980ae860 0x7f0e9814f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:58.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.963+0000 7f0e9e004700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e98080a90 con 0x7f0e980b0240 2020-12-05T17:38:58.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.963+0000 7f0e9e004700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e980aecb0 con 0x7f0e980ae860 2020-12-05T17:38:58.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.963+0000 7f0e9e004700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e98152860 con 0x7f0e980611b0 2020-12-05T17:38:58.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.963+0000 7f0e9d002700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0e980b0240 0x7f0e9814c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40088/0 (socket says 172.21.15.32:40088) 2020-12-05T17:38:58.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.963+0000 7f0e9d002700 1 -- 172.21.15.32:0/1342770516 learned_addr learned my addr 172.21.15.32:0/1342770516 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:38:58.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.963+0000 7f0e9d803700 1 --2- 172.21.15.32:0/1342770516 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e980611b0 0x7f0e98153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:58.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.963+0000 7f0e9f266700 1 --2- 172.21.15.32:0/1342770516 >> v2:172.21.15.61:3300/0 conn(0x7f0e980ae860 0x7f0e9814f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:58.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.963+0000 7f0e9c801700 1 -- 172.21.15.32:0/1342770516 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1974258226 0 0) 0x7f0e98080a90 con 0x7f0e980b0240 2020-12-05T17:38:58.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.963+0000 7f0e9c801700 1 -- 172.21.15.32:0/1342770516 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e80003230 con 0x7f0e980b0240 2020-12-05T17:38:58.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.963+0000 7f0e9d803700 1 -- 172.21.15.32:0/1342770516 >> v2:172.21.15.61:3300/0 conn(0x7f0e980ae860 msgr2=0x7f0e9814f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:58.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.963+0000 7f0e9d803700 1 --2- 172.21.15.32:0/1342770516 >> v2:172.21.15.61:3300/0 conn(0x7f0e980ae860 0x7f0e9814f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.963+0000 7f0e9d803700 1 -- 172.21.15.32:0/1342770516 >> v1:172.21.15.32:6789/0 conn(0x7f0e980b0240 legacy=0x7f0e9814c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:58.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.963+0000 7f0e9d803700 1 -- 172.21.15.32:0/1342770516 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e98152b00 con 0x7f0e980611b0 2020-12-05T17:38:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.963+0000 7f0e9d803700 1 --2- 172.21.15.32:0/1342770516 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e980611b0 0x7f0e98153240 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f0e880032b0 tx=0x7f0e880091a0).ready entity=mon.2 client_cookie=3306604a031e6b07 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.963+0000 7f0e9c801700 1 -- 172.21.15.32:0/1342770516 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e8800b940 con 0x7f0e980611b0 2020-12-05T17:38:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.963+0000 7f0e9c801700 1 -- 172.21.15.32:0/1342770516 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8800baa0 con 0x7f0e980611b0 2020-12-05T17:38:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.963+0000 7f0e9c801700 1 -- 172.21.15.32:0/1342770516 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e8800bc10 con 0x7f0e980611b0 2020-12-05T17:38:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.963+0000 7f0e9e004700 1 -- 172.21.15.32:0/1342770516 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e980611b0 msgr2=0x7f0e98153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.963+0000 7f0e9e004700 1 --2- 172.21.15.32:0/1342770516 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e980611b0 0x7f0e98153240 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f0e880032b0 tx=0x7f0e880091a0).stop 2020-12-05T17:38:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.967+0000 7f0e9e004700 1 -- 172.21.15.32:0/1342770516 shutdown_connections 2020-12-05T17:38:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.967+0000 7f0e9e004700 1 --2- 172.21.15.32:0/1342770516 >> v2:172.21.15.61:3300/0 conn(0x7f0e980ae860 0x7f0e9814f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.967+0000 7f0e9e004700 1 --2- 172.21.15.32:0/1342770516 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e980611b0 0x7f0e98153240 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.967+0000 7f0e9e004700 1 -- 172.21.15.32:0/1342770516 >> 172.21.15.32:0/1342770516 conn(0x7f0e980b5f90 msgr2=0x7f0e980b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:58.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.967+0000 7f0e9e004700 1 -- 172.21.15.32:0/1342770516 shutdown_connections 2020-12-05T17:38:58.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.967+0000 7f0e9e004700 1 -- 172.21.15.32:0/1342770516 wait complete. 2020-12-05T17:38:58.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.967+0000 7f0e9e004700 1 Processor -- start 2020-12-05T17:38:58.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.967+0000 7f0e9e004700 1 -- start start 2020-12-05T17:38:58.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e9e004700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e980611b0 0x7f0e980632d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:58.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e9e004700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e980ae860 0x7f0e980b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:58.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e9e004700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e980b0240 0x7f0e980b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:58.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e9e004700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e98063cc0 con 0x7f0e980ae860 2020-12-05T17:38:58.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e9e004700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e98065220 con 0x7f0e980611b0 2020-12-05T17:38:58.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e9e004700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e980b1b30 con 0x7f0e980b0240 2020-12-05T17:38:58.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e9d002700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e980ae860 0x7f0e980b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:58.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e9f266700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e980b0240 0x7f0e980b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:58.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e9d803700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e980611b0 0x7f0e980632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:58.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e9d002700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e980ae860 0x7f0e980b28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37448/0 (socket says 172.21.15.32:37448) 2020-12-05T17:38:58.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e9d002700 1 -- 172.21.15.32:0/90390135 learned_addr learned my addr 172.21.15.32:0/90390135 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:58.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e9d002700 1 -- 172.21.15.32:0/90390135 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e980b0240 msgr2=0x7f0e980b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:58.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e9d002700 1 --2- 172.21.15.32:0/90390135 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e980b0240 0x7f0e980b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e9d002700 1 -- 172.21.15.32:0/90390135 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e980611b0 msgr2=0x7f0e980632d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:58.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e9d002700 1 --2- 172.21.15.32:0/90390135 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e980611b0 0x7f0e980632d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:58.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e9d002700 1 -- 172.21.15.32:0/90390135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e98065450 con 0x7f0e980ae860 2020-12-05T17:38:58.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e9f266700 1 --2- 172.21.15.32:0/90390135 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e980b0240 0x7f0e980b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:38:58.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e9d803700 1 --2- 172.21.15.32:0/90390135 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e980611b0 0x7f0e980632d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:38:58.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e9d002700 1 --2- 172.21.15.32:0/90390135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e980ae860 0x7f0e980b28b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7f0e8c0010d0 tx=0x7f0e8c007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:58.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e967fc700 1 -- 172.21.15.32:0/90390135 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e8c0072e0 con 0x7f0e980ae860 2020-12-05T17:38:58.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e967fc700 1 -- 172.21.15.32:0/90390135 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8c007440 con 0x7f0e980ae860 2020-12-05T17:38:58.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e9e004700 1 -- 172.21.15.32:0/90390135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e980637c0 con 0x7f0e980ae860 2020-12-05T17:38:58.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e967fc700 1 -- 172.21.15.32:0/90390135 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e8c0057d0 con 0x7f0e980ae860 2020-12-05T17:38:58.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e9e004700 1 -- 172.21.15.32:0/90390135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e980b1fe0 con 0x7f0e980ae860 2020-12-05T17:38:58.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e967fc700 1 -- 172.21.15.32:0/90390135 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0e8c01a060 con 0x7f0e980ae860 2020-12-05T17:38:58.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.971+0000 7f0e9e004700 1 -- 172.21.15.32:0/90390135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e980b1c70 con 0x7f0e980ae860 2020-12-05T17:38:58.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.975+0000 7f0e967fc700 1 --2- 172.21.15.32:0/90390135 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0e78055440 0x7f0e78057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:58.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.975+0000 7f0e967fc700 1 -- 172.21.15.32:0/90390135 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0e8c06d880 con 0x7f0e980ae860 2020-12-05T17:38:58.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.975+0000 7f0e9d803700 1 --2- 172.21.15.32:0/90390135 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0e78055440 0x7f0e78057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:58.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.975+0000 7f0e9d803700 1 --2- 172.21.15.32:0/90390135 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0e78055440 0x7f0e78057900 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f0e8801d000 tx=0x7f0e88016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:58.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:58.979+0000 7f0e967fc700 1 -- 172.21.15.32:0/90390135 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0e8c043a20 con 0x7f0e980ae860 2020-12-05T17:38:59.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.175+0000 7f0e9e004700 1 -- 172.21.15.32:0/90390135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f0e98057050 con 0x7f0e980ae860 2020-12-05T17:38:59.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.183+0000 7f0e967fc700 1 -- 172.21.15.32:0/90390135 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v27) v1 ==== 184+0+0 (secure 0 0 0) 0x7f0e8c018020 con 0x7f0e980ae860 2020-12-05T17:38:59.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.183+0000 7f0e9e004700 1 -- 172.21.15.32:0/90390135 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0e78055440 msgr2=0x7f0e78057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:59.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.183+0000 7f0e9e004700 1 --2- 172.21.15.32:0/90390135 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0e78055440 0x7f0e78057900 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f0e8801d000 tx=0x7f0e88016000).stop 2020-12-05T17:38:59.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.183+0000 7f0e9e004700 1 -- 172.21.15.32:0/90390135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e980ae860 msgr2=0x7f0e980b28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:59.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.183+0000 7f0e9e004700 1 --2- 172.21.15.32:0/90390135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e980ae860 0x7f0e980b28b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7f0e8c0010d0 tx=0x7f0e8c007b50).stop 2020-12-05T17:38:59.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.183+0000 7f0e9e004700 1 -- 172.21.15.32:0/90390135 shutdown_connections 2020-12-05T17:38:59.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.183+0000 7f0e9e004700 1 --2- 172.21.15.32:0/90390135 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0e78055440 0x7f0e78057900 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:59.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.183+0000 7f0e9e004700 1 --2- 172.21.15.32:0/90390135 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e980b0240 0x7f0e980b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:59.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.183+0000 7f0e9e004700 1 --2- 172.21.15.32:0/90390135 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e980611b0 0x7f0e980632d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:59.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.183+0000 7f0e9e004700 1 --2- 172.21.15.32:0/90390135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e980ae860 0x7f0e980b28b0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:59.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.183+0000 7f0e9e004700 1 -- 172.21.15.32:0/90390135 >> 172.21.15.32:0/90390135 conn(0x7f0e980b5f90 msgr2=0x7f0e98061590 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:59.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.183+0000 7f0e9e004700 1 -- 172.21.15.32:0/90390135 shutdown_connections 2020-12-05T17:38:59.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.183+0000 7f0e9e004700 1 -- 172.21.15.32:0/90390135 wait complete. 2020-12-05T17:38:59.194 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2020-12-05T17:38:59.203 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:38:59.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccdd5b9700 1 Processor -- start 2020-12-05T17:38:59.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccdd5b9700 1 -- start start 2020-12-05T17:38:59.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccdd5b9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fccd8061780 0x7fccd80ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:59.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccdd5b9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd80b0380 con 0x7fccd805feb0 2020-12-05T17:38:59.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccdd5b9700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd805fad0 con 0x7fccd805f6f0 2020-12-05T17:38:59.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccdd5b9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd80aeaf0 con 0x7fccd8061780 2020-12-05T17:38:59.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd6ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fccd805feb0 0x7fccd814e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40104/0 (socket says 172.21.15.32:40104) 2020-12-05T17:38:59.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd6ffd700 1 -- 172.21.15.32:0/3343301352 learned_addr learned my addr 172.21.15.32:0/3343301352 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:38:59.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd7fff700 1 --2- 172.21.15.32:0/3343301352 >> v2:172.21.15.32:3300/0 conn(0x7fccd8061780 0x7fccd80ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:59.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd67fc700 1 -- 172.21.15.32:0/3343301352 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608707539 0 0) 0x7fccd80b0380 con 0x7fccd805feb0 2020-12-05T17:38:59.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd67fc700 1 -- 172.21.15.32:0/3343301352 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccc4003230 con 0x7fccd805feb0 2020-12-05T17:38:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd67fc700 1 -- 172.21.15.32:0/3343301352 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2828197499 0 0) 0x7fccc4003230 con 0x7fccd805feb0 2020-12-05T17:38:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd67fc700 1 -- 172.21.15.32:0/3343301352 >> v2:172.21.15.32:3300/0 conn(0x7fccd8061780 msgr2=0x7fccd80ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd67fc700 1 --2- 172.21.15.32:0/3343301352 >> v2:172.21.15.32:3300/0 conn(0x7fccd8061780 0x7fccd80ae600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd67fc700 1 -- 172.21.15.32:0/3343301352 >> v1:172.21.15.61:6789/0 conn(0x7fccd805f6f0 legacy=0x7fccd80adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd67fc700 1 -- 172.21.15.32:0/3343301352 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd80aecd0 con 0x7fccd805feb0 2020-12-05T17:38:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd67fc700 1 -- 172.21.15.32:0/3343301352 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fccc0001a50 con 0x7fccd805feb0 2020-12-05T17:38:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd67fc700 1 -- 172.21.15.32:0/3343301352 >> v1:172.21.15.32:6789/0 conn(0x7fccd805feb0 legacy=0x7fccd814e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd67fc700 1 --2- 172.21.15.32:0/3343301352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fccc4003720 0x7fccc4003b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd67fc700 1 --2- 172.21.15.32:0/3343301352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccc40044d0 0x7fccc4004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd67fc700 1 --2- 172.21.15.32:0/3343301352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fccc4004e40 0x7fccc400b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd67fc700 1 -- 172.21.15.32:0/3343301352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd80aeaf0 con 0x7fccc40044d0 2020-12-05T17:38:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd67fc700 1 -- 172.21.15.32:0/3343301352 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc40041c0 con 0x7fccc4004e40 2020-12-05T17:38:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd67fc700 1 -- 172.21.15.32:0/3343301352 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc404fc20 con 0x7fccc4003720 2020-12-05T17:38:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd6ffd700 1 --2- 172.21.15.32:0/3343301352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccc40044d0 0x7fccc4004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd77fe700 1 --2- 172.21.15.32:0/3343301352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fccc4003720 0x7fccc4003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd7fff700 1 --2- 172.21.15.32:0/3343301352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fccc4004e40 0x7fccc400b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd77fe700 1 -- 172.21.15.32:0/3343301352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fccc4004e40 msgr2=0x7fccc400b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd77fe700 1 --2- 172.21.15.32:0/3343301352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fccc4004e40 0x7fccc400b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd77fe700 1 -- 172.21.15.32:0/3343301352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccc40044d0 msgr2=0x7fccc4004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:38:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd77fe700 1 --2- 172.21.15.32:0/3343301352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccc40044d0 0x7fccc4004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:59.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd77fe700 1 -- 172.21.15.32:0/3343301352 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc4003230 con 0x7fccc4003720 2020-12-05T17:38:59.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd6ffd700 1 --2- 172.21.15.32:0/3343301352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccc40044d0 0x7fccc4004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:38:59.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd7fff700 1 -- 172.21.15.32:0/3343301352 reap_dead start 2020-12-05T17:38:59.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd77fe700 1 --2- 172.21.15.32:0/3343301352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fccc4003720 0x7fccc4003b00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fccc80010d0 tx=0x7fccc8007b60).ready entity=mon.2 client_cookie=e450d75e5d971780 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:59.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd67fc700 1 -- 172.21.15.32:0/3343301352 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccc800c7f0 con 0x7fccc4003720 2020-12-05T17:38:59.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd67fc700 1 -- 172.21.15.32:0/3343301352 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc800c950 con 0x7fccc4003720 2020-12-05T17:38:59.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccd67fc700 1 -- 172.21.15.32:0/3343301352 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccc800ccb0 con 0x7fccc4003720 2020-12-05T17:38:59.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccdd5b9700 1 -- 172.21.15.32:0/3343301352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fccc4003720 msgr2=0x7fccc4003b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:59.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.327+0000 7fccdd5b9700 1 --2- 172.21.15.32:0/3343301352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fccc4003720 0x7fccc4003b00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fccc80010d0 tx=0x7fccc8007b60).stop 2020-12-05T17:38:59.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.331+0000 7fccdd5b9700 1 -- 172.21.15.32:0/3343301352 shutdown_connections 2020-12-05T17:38:59.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.331+0000 7fccdd5b9700 1 --2- 172.21.15.32:0/3343301352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fccc4003720 0x7fccc4003b00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:59.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.331+0000 7fccdd5b9700 1 -- 172.21.15.32:0/3343301352 >> 172.21.15.32:0/3343301352 conn(0x7fccd80b4b10 msgr2=0x7fccd80b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:59.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.331+0000 7fccdd5b9700 1 -- 172.21.15.32:0/3343301352 shutdown_connections 2020-12-05T17:38:59.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.331+0000 7fccdd5b9700 1 -- 172.21.15.32:0/3343301352 wait complete. 2020-12-05T17:38:59.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.331+0000 7fccdd5b9700 1 Processor -- start 2020-12-05T17:38:59.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.331+0000 7fccdd5b9700 1 -- start start 2020-12-05T17:38:59.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fccdd5b9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccc4003720 0x7fccd812d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:59.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fccdd5b9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fccd8110370 0x7fccd813e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:59.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fccdd5b9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fccd8110cf0 0x7fccd813e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:59.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fccdd5b9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd81504a0 con 0x7fccc4003720 2020-12-05T17:38:59.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fccdd5b9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd8066a40 con 0x7fccd8110370 2020-12-05T17:38:59.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fccdd5b9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd813eb30 con 0x7fccd8110cf0 2020-12-05T17:38:59.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fccd77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccc4003720 0x7fccd812d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:59.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fccd77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccc4003720 0x7fccd812d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37466/0 (socket says 172.21.15.32:37466) 2020-12-05T17:38:59.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fccd77fe700 1 -- 172.21.15.32:0/321911625 learned_addr learned my addr 172.21.15.32:0/321911625 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:59.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fccd6ffd700 1 --2- 172.21.15.32:0/321911625 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fccd8110370 0x7fccd813e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:59.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fccd7fff700 1 --2- 172.21.15.32:0/321911625 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fccd8110cf0 0x7fccd813e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:59.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fccd77fe700 1 -- 172.21.15.32:0/321911625 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fccd8110cf0 msgr2=0x7fccd813e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:59.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fccd77fe700 1 --2- 172.21.15.32:0/321911625 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fccd8110cf0 0x7fccd813e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:59.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fccd77fe700 1 -- 172.21.15.32:0/321911625 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fccd8110370 msgr2=0x7fccd813e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:59.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fccd77fe700 1 --2- 172.21.15.32:0/321911625 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fccd8110370 0x7fccd813e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:59.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fccd77fe700 1 -- 172.21.15.32:0/321911625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd813ecf0 con 0x7fccc4003720 2020-12-05T17:38:59.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fccd7fff700 1 --2- 172.21.15.32:0/321911625 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fccd8110cf0 0x7fccd813e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:38:59.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fccd77fe700 1 --2- 172.21.15.32:0/321911625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccc4003720 0x7fccd812d840 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7fccc80010d0 tx=0x7fccc8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:59.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fcccffff700 1 -- 172.21.15.32:0/321911625 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccc8000e50 con 0x7fccc4003720 2020-12-05T17:38:59.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fcccffff700 1 -- 172.21.15.32:0/321911625 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc8012a10 con 0x7fccc4003720 2020-12-05T17:38:59.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fcccffff700 1 -- 172.21.15.32:0/321911625 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccc8012d70 con 0x7fccc4003720 2020-12-05T17:38:59.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fccdd5b9700 1 -- 172.21.15.32:0/321911625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd813ef40 con 0x7fccc4003720 2020-12-05T17:38:59.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fccdd5b9700 1 -- 172.21.15.32:0/321911625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd811d770 con 0x7fccc4003720 2020-12-05T17:38:59.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fccdd5b9700 1 -- 172.21.15.32:0/321911625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd8066c00 con 0x7fccc4003720 2020-12-05T17:38:59.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.335+0000 7fcccffff700 1 -- 172.21.15.32:0/321911625 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fccc8010070 con 0x7fccc4003720 2020-12-05T17:38:59.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.339+0000 7fcccffff700 1 --2- 172.21.15.32:0/321911625 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fccbc055320 0x7fccbc0577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:59.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.339+0000 7fcccffff700 1 -- 172.21.15.32:0/321911625 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fccc806e7f0 con 0x7fccc4003720 2020-12-05T17:38:59.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.339+0000 7fccd6ffd700 1 --2- 172.21.15.32:0/321911625 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fccbc055320 0x7fccbc0577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:59.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.339+0000 7fccd6ffd700 1 --2- 172.21.15.32:0/321911625 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fccbc055320 0x7fccbc0577e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fccc0005000 tx=0x7fccd80aee60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:59.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.347+0000 7fcccffff700 1 -- 172.21.15.32:0/321911625 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fccc8044990 con 0x7fccc4003720 2020-12-05T17:38:59.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.539+0000 7fccdd5b9700 1 -- 172.21.15.32:0/321911625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fccd8057050 con 0x7fccc4003720 2020-12-05T17:38:59.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.547+0000 7fcccffff700 1 -- 172.21.15.32:0/321911625 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v28) v1 ==== 184+0+0 (secure 0 0 0) 0x7fccc8021030 con 0x7fccc4003720 2020-12-05T17:38:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.547+0000 7fccdd5b9700 1 -- 172.21.15.32:0/321911625 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fccbc055320 msgr2=0x7fccbc0577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.547+0000 7fccdd5b9700 1 --2- 172.21.15.32:0/321911625 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fccbc055320 0x7fccbc0577e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fccc0005000 tx=0x7fccd80aee60).stop 2020-12-05T17:38:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.547+0000 7fccdd5b9700 1 -- 172.21.15.32:0/321911625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccc4003720 msgr2=0x7fccd812d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.547+0000 7fccdd5b9700 1 --2- 172.21.15.32:0/321911625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccc4003720 0x7fccd812d840 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7fccc80010d0 tx=0x7fccc8017000).stop 2020-12-05T17:38:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.547+0000 7fccdd5b9700 1 -- 172.21.15.32:0/321911625 shutdown_connections 2020-12-05T17:38:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.547+0000 7fccdd5b9700 1 --2- 172.21.15.32:0/321911625 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fccbc055320 0x7fccbc0577e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.547+0000 7fccdd5b9700 1 --2- 172.21.15.32:0/321911625 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fccd8110cf0 0x7fccd813e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.547+0000 7fccdd5b9700 1 --2- 172.21.15.32:0/321911625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccc4003720 0x7fccd812d840 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.547+0000 7fccdd5b9700 1 --2- 172.21.15.32:0/321911625 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fccd8110370 0x7fccd813e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.547+0000 7fccdd5b9700 1 -- 172.21.15.32:0/321911625 >> 172.21.15.32:0/321911625 conn(0x7fccd80b4b10 msgr2=0x7fccd8131340 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.547+0000 7fccdd5b9700 1 -- 172.21.15.32:0/321911625 shutdown_connections 2020-12-05T17:38:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.547+0000 7fccdd5b9700 1 -- 172.21.15.32:0/321911625 wait complete. 2020-12-05T17:38:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.2 2020-12-05T17:38:59.567 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:38:59.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.675+0000 7f0b5edb9700 1 Processor -- start 2020-12-05T17:38:59.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.675+0000 7f0b5edb9700 1 -- start start 2020-12-05T17:38:59.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.675+0000 7f0b5edb9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b580ad1b0 0x7f0b580aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:59.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.675+0000 7f0b5edb9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b5813fb70 con 0x7f0b5805f490 2020-12-05T17:38:59.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.675+0000 7f0b5edb9700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b580af460 con 0x7f0b580aead0 2020-12-05T17:38:59.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.675+0000 7f0b5edb9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b580af700 con 0x7f0b580ad1b0 2020-12-05T17:38:59.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.675+0000 7f0b5cb55700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f0b580aead0 0x7f0b581389f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:52762/0 (socket says 172.21.15.32:52762) 2020-12-05T17:38:59.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.675+0000 7f0b5d356700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0b5805f490 0x7f0b5805f940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40126/0 (socket says 172.21.15.32:40126) 2020-12-05T17:38:59.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.675+0000 7f0b5cb55700 1 -- 172.21.15.32:0/243783034 learned_addr learned my addr 172.21.15.32:0/243783034 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:38:59.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.679+0000 7f0b57fff700 1 --2- 172.21.15.32:0/243783034 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b580ad1b0 0x7f0b580aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:59.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.679+0000 7f0b577fe700 1 -- 172.21.15.32:0/243783034 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252742099 0 0) 0x7f0b580af460 con 0x7f0b580aead0 2020-12-05T17:38:59.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.679+0000 7f0b577fe700 1 -- 172.21.15.32:0/243783034 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b38003230 con 0x7f0b580aead0 2020-12-05T17:38:59.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.679+0000 7f0b577fe700 1 -- 172.21.15.32:0/243783034 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992682967 0 0) 0x7f0b5813fb70 con 0x7f0b5805f490 2020-12-05T17:38:59.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.679+0000 7f0b577fe700 1 -- 172.21.15.32:0/243783034 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b580af460 con 0x7f0b5805f490 2020-12-05T17:38:59.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.679+0000 7f0b57fff700 1 -- 172.21.15.32:0/243783034 >> v1:172.21.15.61:6789/0 conn(0x7f0b580aead0 legacy=0x7f0b581389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:59.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.679+0000 7f0b57fff700 1 -- 172.21.15.32:0/243783034 >> v1:172.21.15.32:6789/0 conn(0x7f0b5805f490 legacy=0x7f0b5805f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:59.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.679+0000 7f0b57fff700 1 -- 172.21.15.32:0/243783034 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b58139100 con 0x7f0b580ad1b0 2020-12-05T17:38:59.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.679+0000 7f0b57fff700 1 --2- 172.21.15.32:0/243783034 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b580ad1b0 0x7f0b580aeeb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f0b40002f30 tx=0x7f0b40006dc0).ready entity=mon.2 client_cookie=61b4ef22d4ad73f0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:59.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.679+0000 7f0b577fe700 1 -- 172.21.15.32:0/243783034 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b40009730 con 0x7f0b580ad1b0 2020-12-05T17:38:59.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.679+0000 7f0b577fe700 1 -- 172.21.15.32:0/243783034 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b40009890 con 0x7f0b580ad1b0 2020-12-05T17:38:59.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.679+0000 7f0b577fe700 1 -- 172.21.15.32:0/243783034 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b40009bf0 con 0x7f0b580ad1b0 2020-12-05T17:38:59.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.679+0000 7f0b5edb9700 1 -- 172.21.15.32:0/243783034 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b580ad1b0 msgr2=0x7f0b580aeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:59.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.679+0000 7f0b5edb9700 1 --2- 172.21.15.32:0/243783034 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b580ad1b0 0x7f0b580aeeb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f0b40002f30 tx=0x7f0b40006dc0).stop 2020-12-05T17:38:59.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.679+0000 7f0b5edb9700 1 -- 172.21.15.32:0/243783034 shutdown_connections 2020-12-05T17:38:59.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.679+0000 7f0b5edb9700 1 --2- 172.21.15.32:0/243783034 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b580ad1b0 0x7f0b580aeeb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:59.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.679+0000 7f0b5edb9700 1 -- 172.21.15.32:0/243783034 >> 172.21.15.32:0/243783034 conn(0x7f0b580b5580 msgr2=0x7f0b580b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:59.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.679+0000 7f0b5edb9700 1 -- 172.21.15.32:0/243783034 shutdown_connections 2020-12-05T17:38:59.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.679+0000 7f0b5edb9700 1 -- 172.21.15.32:0/243783034 wait complete. 2020-12-05T17:38:59.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b5edb9700 1 Processor -- start 2020-12-05T17:38:59.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b5edb9700 1 -- start start 2020-12-05T17:38:59.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b5edb9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b5805f490 0x7f0b580b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:59.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b5edb9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b580ad1b0 0x7f0b58152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:59.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b5edb9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b580aead0 0x7f0b581532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:59.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b5edb9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b58152980 con 0x7f0b580aead0 2020-12-05T17:38:59.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b5edb9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b58064200 con 0x7f0b5805f490 2020-12-05T17:38:59.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b5edb9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b580b2840 con 0x7f0b580ad1b0 2020-12-05T17:38:59.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b5d356700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b580aead0 0x7f0b581532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:59.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b5d356700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b580aead0 0x7f0b581532a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37484/0 (socket says 172.21.15.32:37484) 2020-12-05T17:38:59.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b5d356700 1 -- 172.21.15.32:0/420149636 learned_addr learned my addr 172.21.15.32:0/420149636 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:38:59.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b5cb55700 1 --2- 172.21.15.32:0/420149636 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b5805f490 0x7f0b580b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:59.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b57fff700 1 --2- 172.21.15.32:0/420149636 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b580ad1b0 0x7f0b58152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:59.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b5d356700 1 -- 172.21.15.32:0/420149636 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b580ad1b0 msgr2=0x7f0b58152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:59.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b5d356700 1 --2- 172.21.15.32:0/420149636 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b580ad1b0 0x7f0b58152db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:59.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b5d356700 1 -- 172.21.15.32:0/420149636 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b5805f490 msgr2=0x7f0b580b2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:59.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b5d356700 1 --2- 172.21.15.32:0/420149636 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b5805f490 0x7f0b580b2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:59.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b5d356700 1 -- 172.21.15.32:0/420149636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b58139bb0 con 0x7f0b580aead0 2020-12-05T17:38:59.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b5d356700 1 --2- 172.21.15.32:0/420149636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b580aead0 0x7f0b581532a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7f0b500010d0 tx=0x7f0b50007ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:59.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b55ffb700 1 -- 172.21.15.32:0/420149636 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b500072e0 con 0x7f0b580aead0 2020-12-05T17:38:59.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b55ffb700 1 -- 172.21.15.32:0/420149636 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b50005520 con 0x7f0b580aead0 2020-12-05T17:38:59.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b5edb9700 1 -- 172.21.15.32:0/420149636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b58063e80 con 0x7f0b580aead0 2020-12-05T17:38:59.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b55ffb700 1 -- 172.21.15.32:0/420149636 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b500058c0 con 0x7f0b580aead0 2020-12-05T17:38:59.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.683+0000 7f0b5edb9700 1 -- 172.21.15.32:0/420149636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b581395c0 con 0x7f0b580aead0 2020-12-05T17:38:59.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.687+0000 7f0b5edb9700 1 -- 172.21.15.32:0/420149636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b58152af0 con 0x7f0b580aead0 2020-12-05T17:38:59.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.687+0000 7f0b55ffb700 1 -- 172.21.15.32:0/420149636 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0b50010070 con 0x7f0b580aead0 2020-12-05T17:38:59.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.687+0000 7f0b55ffb700 1 --2- 172.21.15.32:0/420149636 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0b3c055330 0x7f0b3c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:38:59.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.687+0000 7f0b55ffb700 1 -- 172.21.15.32:0/420149636 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0b5006d7d0 con 0x7f0b580aead0 2020-12-05T17:38:59.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.687+0000 7f0b5cb55700 1 --2- 172.21.15.32:0/420149636 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0b3c055330 0x7f0b3c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:38:59.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.687+0000 7f0b5cb55700 1 --2- 172.21.15.32:0/420149636 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0b3c055330 0x7f0b3c0577f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f0b4800d000 tx=0x7f0b48009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:38:59.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.695+0000 7f0b55ffb700 1 -- 172.21.15.32:0/420149636 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0b500438d0 con 0x7f0b580aead0 2020-12-05T17:38:59.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.883+0000 7f0b5edb9700 1 -- 172.21.15.32:0/420149636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b580b30b0 con 0x7f0b580aead0 2020-12-05T17:38:59.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.883+0000 7f0b55ffb700 1 -- 172.21.15.32:0/420149636 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f0b580b30b0 con 0x7f0b580aead0 2020-12-05T17:38:59.891 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:38:59.891 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":37,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6535,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2494768218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2494768218},{"type":"v1","addr":"172.21.15.61:6833","nonce":2494768218}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":35}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:38:56.821107+0000","modified":"2020-12-05T17:38:57.824754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/81037173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":81037173},{"type":"v1","addr":"172.21.15.32:6835","nonce":81037173}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-12-05T17:38:59.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.887+0000 7f0b5edb9700 1 -- 172.21.15.32:0/420149636 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0b3c055330 msgr2=0x7f0b3c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:59.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.887+0000 7f0b5edb9700 1 --2- 172.21.15.32:0/420149636 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0b3c055330 0x7f0b3c0577f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f0b4800d000 tx=0x7f0b48009040).stop 2020-12-05T17:38:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.887+0000 7f0b5edb9700 1 -- 172.21.15.32:0/420149636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b580aead0 msgr2=0x7f0b581532a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:38:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.887+0000 7f0b5edb9700 1 --2- 172.21.15.32:0/420149636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b580aead0 0x7f0b581532a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7f0b500010d0 tx=0x7f0b50007ab0).stop 2020-12-05T17:38:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.887+0000 7f0b5edb9700 1 -- 172.21.15.32:0/420149636 shutdown_connections 2020-12-05T17:38:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.887+0000 7f0b5edb9700 1 --2- 172.21.15.32:0/420149636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b580aead0 0x7f0b581532a0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.887+0000 7f0b5edb9700 1 --2- 172.21.15.32:0/420149636 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b5805f490 0x7f0b580b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.887+0000 7f0b5edb9700 1 --2- 172.21.15.32:0/420149636 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0b3c055330 0x7f0b3c0577f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.887+0000 7f0b5edb9700 1 --2- 172.21.15.32:0/420149636 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b580ad1b0 0x7f0b58152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:38:59.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.887+0000 7f0b5edb9700 1 -- 172.21.15.32:0/420149636 >> 172.21.15.32:0/420149636 conn(0x7f0b580b5580 msgr2=0x7f0b58143350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:38:59.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.887+0000 7f0b5edb9700 1 -- 172.21.15.32:0/420149636 shutdown_connections 2020-12-05T17:38:59.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:38:59.887+0000 7f0b5edb9700 1 -- 172.21.15.32:0/420149636 wait complete. 2020-12-05T17:38:59.895 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 37 2020-12-05T17:39:00.108 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:38:56.821107+0000', 'modified': '2020-12-05T17:38:57.824754+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6609}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6609': {'gid': 6609, 'name': 'a', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/81037173', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 81037173}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 81037173}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:39:00.109 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:39:00.109 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:39:00.241 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":6609,"want_state":"up:active","state":"up:active","rank_uptime":3.4053795249999999,"mdsmap_epoch":37,"osdmap_epoch":83,"osdmap_epoch_barrier":83,"uptime":8.6548958840000001} 2020-12-05T17:39:00.242 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 6609, "mdsmap_epoch": 37, "osdmap_epoch": 83, "osdmap_epoch_barrier": 83, "rank_uptime": 3.405379525, "state": "up:active", "uptime": 8.654895884, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:39:00.243 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:39:00.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.355+0000 7fbd83fff700 1 Processor -- start 2020-12-05T17:39:00.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.355+0000 7fbd83fff700 1 -- start start 2020-12-05T17:39:00.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.355+0000 7fbd83fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbd840b2ca0 0x7fbd8413fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.355+0000 7fbd83fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd840b2450 con 0x7fbd84139540 2020-12-05T17:39:00.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.355+0000 7fbd83fff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd84066d40 con 0x7fbd840b2070 2020-12-05T17:39:00.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.355+0000 7fbd83fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840acb80 con 0x7fbd840b2ca0 2020-12-05T17:39:00.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.355+0000 7fbd827fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbd84139540 0x7fbd8413f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40138/0 (socket says 172.21.15.32:40138) 2020-12-05T17:39:00.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.355+0000 7fbd827fc700 1 -- 172.21.15.32:0/1016715781 learned_addr learned my addr 172.21.15.32:0/1016715781 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:00.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd837fe700 1 --2- 172.21.15.32:0/1016715781 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbd840b2ca0 0x7fbd8413fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd81ffb700 1 -- 172.21.15.32:0/1016715781 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069302965 0 0) 0x7fbd840b2450 con 0x7fbd84139540 2020-12-05T17:39:00.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd81ffb700 1 -- 172.21.15.32:0/1016715781 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd68003230 con 0x7fbd84139540 2020-12-05T17:39:00.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd81ffb700 1 -- 172.21.15.32:0/1016715781 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 305552252 0 0) 0x7fbd84066d40 con 0x7fbd840b2070 2020-12-05T17:39:00.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd81ffb700 1 -- 172.21.15.32:0/1016715781 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd840b2450 con 0x7fbd840b2070 2020-12-05T17:39:00.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd81ffb700 1 -- 172.21.15.32:0/1016715781 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2237842871 0 0) 0x7fbd68003230 con 0x7fbd84139540 2020-12-05T17:39:00.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd81ffb700 1 -- 172.21.15.32:0/1016715781 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbd840b2ca0 msgr2=0x7fbd8413fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:00.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd81ffb700 1 --2- 172.21.15.32:0/1016715781 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbd840b2ca0 0x7fbd8413fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd81ffb700 1 -- 172.21.15.32:0/1016715781 >> v1:172.21.15.61:6789/0 conn(0x7fbd840b2070 legacy=0x7fbd84138e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:00.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd81ffb700 1 -- 172.21.15.32:0/1016715781 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd84066fe0 con 0x7fbd84139540 2020-12-05T17:39:00.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd81ffb700 1 -- 172.21.15.32:0/1016715781 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fbd780016d0 con 0x7fbd84139540 2020-12-05T17:39:00.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd81ffb700 1 -- 172.21.15.32:0/1016715781 >> v1:172.21.15.32:6789/0 conn(0x7fbd84139540 legacy=0x7fbd8413f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:00.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd81ffb700 1 --2- 172.21.15.32:0/1016715781 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbd680038e0 0x7fbd68003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd81ffb700 1 --2- 172.21.15.32:0/1016715781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd680044d0 0x7fbd68004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd81ffb700 1 --2- 172.21.15.32:0/1016715781 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbd840b2ca0 0x7fbd68004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd81ffb700 1 -- 172.21.15.32:0/1016715781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd68005360 con 0x7fbd680044d0 2020-12-05T17:39:00.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd81ffb700 1 -- 172.21.15.32:0/1016715781 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd68004280 con 0x7fbd680038e0 2020-12-05T17:39:00.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd81ffb700 1 -- 172.21.15.32:0/1016715781 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6800fd30 con 0x7fbd840b2ca0 2020-12-05T17:39:00.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd827fc700 1 --2- 172.21.15.32:0/1016715781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd680044d0 0x7fbd68004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd82ffd700 1 --2- 172.21.15.32:0/1016715781 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbd680038e0 0x7fbd68003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd837fe700 1 --2- 172.21.15.32:0/1016715781 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbd840b2ca0 0x7fbd68004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd827fc700 1 -- 172.21.15.32:0/1016715781 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbd840b2ca0 msgr2=0x7fbd68004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:00.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd827fc700 1 --2- 172.21.15.32:0/1016715781 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbd840b2ca0 0x7fbd68004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd827fc700 1 -- 172.21.15.32:0/1016715781 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbd680038e0 msgr2=0x7fbd68003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:00.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd827fc700 1 --2- 172.21.15.32:0/1016715781 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbd680038e0 0x7fbd68003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd827fc700 1 -- 172.21.15.32:0/1016715781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd68003230 con 0x7fbd680044d0 2020-12-05T17:39:00.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd827fc700 1 --2- 172.21.15.32:0/1016715781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd680044d0 0x7fbd68004950 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7fbd68004440 tx=0x7fbd78007690).ready entity=mon.0 client_cookie=d911fc8f7373df21 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:00.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd81ffb700 1 -- 172.21.15.32:0/1016715781 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd7800cde0 con 0x7fbd680044d0 2020-12-05T17:39:00.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd81ffb700 1 -- 172.21.15.32:0/1016715781 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd780070d0 con 0x7fbd680044d0 2020-12-05T17:39:00.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd81ffb700 1 -- 172.21.15.32:0/1016715781 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd78007430 con 0x7fbd680044d0 2020-12-05T17:39:00.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd83fff700 1 -- 172.21.15.32:0/1016715781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd680044d0 msgr2=0x7fbd68004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:00.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd83fff700 1 --2- 172.21.15.32:0/1016715781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd680044d0 0x7fbd68004950 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7fbd68004440 tx=0x7fbd78007690).stop 2020-12-05T17:39:00.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd837fe700 1 -- 172.21.15.32:0/1016715781 reap_dead start 2020-12-05T17:39:00.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd83fff700 1 -- 172.21.15.32:0/1016715781 shutdown_connections 2020-12-05T17:39:00.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd83fff700 1 -- 172.21.15.32:0/1016715781 >> 172.21.15.32:0/1016715781 conn(0x7fbd8405bbb0 msgr2=0x7fbd84062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:00.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd83fff700 1 -- 172.21.15.32:0/1016715781 shutdown_connections 2020-12-05T17:39:00.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.359+0000 7fbd83fff700 1 -- 172.21.15.32:0/1016715781 wait complete. 2020-12-05T17:39:00.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.363+0000 7fbd83fff700 1 Processor -- start 2020-12-05T17:39:00.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.363+0000 7fbd83fff700 1 -- start start 2020-12-05T17:39:00.381 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.363+0000 7fbd83fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbd840b3ef0 0x7fbd84149b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.363+0000 7fbd83fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd8413e2c0 0x7fbd8414a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.363+0000 7fbd83fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbd8410eba0 0x7fbd8410f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.363+0000 7fbd83fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840b42d0 con 0x7fbd8413e2c0 2020-12-05T17:39:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.363+0000 7fbd83fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8413d100 con 0x7fbd8410eba0 2020-12-05T17:39:00.383 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.363+0000 7fbd83fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8413f340 con 0x7fbd840b3ef0 2020-12-05T17:39:00.383 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.363+0000 7fbd827fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd8413e2c0 0x7fbd8414a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.383 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.363+0000 7fbd827fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd8413e2c0 0x7fbd8414a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37502/0 (socket says 172.21.15.32:37502) 2020-12-05T17:39:00.383 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.363+0000 7fbd827fc700 1 -- 172.21.15.32:0/3661041578 learned_addr learned my addr 172.21.15.32:0/3661041578 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:00.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.363+0000 7fbd82ffd700 1 --2- 172.21.15.32:0/3661041578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbd840b3ef0 0x7fbd84149b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.363+0000 7fbd837fe700 1 --2- 172.21.15.32:0/3661041578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbd8410eba0 0x7fbd8410f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.367+0000 7fbd82ffd700 1 -- 172.21.15.32:0/3661041578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbd8410eba0 msgr2=0x7fbd8410f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:00.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.367+0000 7fbd82ffd700 1 --2- 172.21.15.32:0/3661041578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbd8410eba0 0x7fbd8410f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.385 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.367+0000 7fbd82ffd700 1 -- 172.21.15.32:0/3661041578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd8413e2c0 msgr2=0x7fbd8414a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:00.385 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.367+0000 7fbd82ffd700 1 --2- 172.21.15.32:0/3661041578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd8413e2c0 0x7fbd8414a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.385 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.367+0000 7fbd82ffd700 1 -- 172.21.15.32:0/3661041578 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8413d500 con 0x7fbd840b3ef0 2020-12-05T17:39:00.385 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.367+0000 7fbd827fc700 1 --2- 172.21.15.32:0/3661041578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd8413e2c0 0x7fbd8414a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:00.385 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.367+0000 7fbd82ffd700 1 --2- 172.21.15.32:0/3661041578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbd840b3ef0 0x7fbd84149b40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fbd7400b000 tx=0x7fbd74009040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:00.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.367+0000 7fbd73fff700 1 -- 172.21.15.32:0/3661041578 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd74003510 con 0x7fbd840b3ef0 2020-12-05T17:39:00.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.367+0000 7fbd83fff700 1 -- 172.21.15.32:0/3661041578 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd8414a520 con 0x7fbd840b3ef0 2020-12-05T17:39:00.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.367+0000 7fbd83fff700 1 -- 172.21.15.32:0/3661041578 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd8413d950 con 0x7fbd840b3ef0 2020-12-05T17:39:00.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.367+0000 7fbd73fff700 1 -- 172.21.15.32:0/3661041578 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd74001640 con 0x7fbd840b3ef0 2020-12-05T17:39:00.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.367+0000 7fbd73fff700 1 -- 172.21.15.32:0/3661041578 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd740017b0 con 0x7fbd840b3ef0 2020-12-05T17:39:00.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.367+0000 7fbd83fff700 1 -- 172.21.15.32:0/3661041578 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd8413f180 con 0x7fbd840b3ef0 2020-12-05T17:39:00.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.367+0000 7fbd73fff700 1 -- 172.21.15.32:0/3661041578 <== mon.2 v2:172.21.15.61:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbd74001910 con 0x7fbd840b3ef0 2020-12-05T17:39:00.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.367+0000 7fbd73fff700 1 --2- 172.21.15.32:0/3661041578 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbd600553b0 0x7fbd60057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.388 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.367+0000 7fbd73fff700 1 -- 172.21.15.32:0/3661041578 <== mon.2 v2:172.21.15.61:3301/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbd7406e340 con 0x7fbd840b3ef0 2020-12-05T17:39:00.388 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.367+0000 7fbd827fc700 1 --2- 172.21.15.32:0/3661041578 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbd600553b0 0x7fbd60057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.388 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.367+0000 7fbd827fc700 1 --2- 172.21.15.32:0/3661041578 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbd600553b0 0x7fbd60057870 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7fbd78005000 tx=0x7fbd78015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:00.389 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.375+0000 7fbd73fff700 1 -- 172.21.15.32:0/3661041578 <== mon.2 v2:172.21.15.61:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbd740443d0 con 0x7fbd840b3ef0 2020-12-05T17:39:00.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.523+0000 7fbd83fff700 1 -- 172.21.15.32:0/3661041578 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd8413f480 con 0x7fbd840b3ef0 2020-12-05T17:39:00.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.523+0000 7fbd73fff700 1 -- 172.21.15.32:0/3661041578 <== mon.2 v2:172.21.15.61:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16274 (secure 0 0 0) 0x7fbd74039fb0 con 0x7fbd840b3ef0 2020-12-05T17:39:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:00.531 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":83,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:38:57.823795+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:38:54.508932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-12-05T17:38:55.805478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:39:00.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.527+0000 7fbd83fff700 1 -- 172.21.15.32:0/3661041578 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbd600553b0 msgr2=0x7fbd60057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:00.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.527+0000 7fbd83fff700 1 --2- 172.21.15.32:0/3661041578 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbd600553b0 0x7fbd60057870 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7fbd78005000 tx=0x7fbd78015000).stop 2020-12-05T17:39:00.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.527+0000 7fbd83fff700 1 -- 172.21.15.32:0/3661041578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbd840b3ef0 msgr2=0x7fbd84149b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:00.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.527+0000 7fbd83fff700 1 --2- 172.21.15.32:0/3661041578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbd840b3ef0 0x7fbd84149b40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fbd7400b000 tx=0x7fbd74009040).stop 2020-12-05T17:39:00.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.527+0000 7fbd83fff700 1 -- 172.21.15.32:0/3661041578 shutdown_connections 2020-12-05T17:39:00.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.527+0000 7fbd83fff700 1 --2- 172.21.15.32:0/3661041578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbd8410eba0 0x7fbd8410f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.527+0000 7fbd83fff700 1 --2- 172.21.15.32:0/3661041578 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbd600553b0 0x7fbd60057870 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.527+0000 7fbd83fff700 1 --2- 172.21.15.32:0/3661041578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbd840b3ef0 0x7fbd84149b40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.527+0000 7fbd83fff700 1 --2- 172.21.15.32:0/3661041578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd8413e2c0 0x7fbd8414a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.527+0000 7fbd83fff700 1 -- 172.21.15.32:0/3661041578 >> 172.21.15.32:0/3661041578 conn(0x7fbd8405bbb0 msgr2=0x7fbd840ad460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:00.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.527+0000 7fbd83fff700 1 -- 172.21.15.32:0/3661041578 shutdown_connections 2020-12-05T17:39:00.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.527+0000 7fbd83fff700 1 -- 172.21.15.32:0/3661041578 wait complete. 2020-12-05T17:39:00.550 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:38:54.508932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:38:55.805478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:39:00.551 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:39:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ae9dc700 1 Processor -- start 2020-12-05T17:39:00.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ae9dc700 1 -- start start 2020-12-05T17:39:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ae9dc700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd4a8144610 0x7fd4a813fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ae9dc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4a81407e0 con 0x7fd4a80af5c0 2020-12-05T17:39:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ae9dc700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4a8144a40 con 0x7fd4a80af9a0 2020-12-05T17:39:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ae9dc700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4a81405f0 con 0x7fd4a8144610 2020-12-05T17:39:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ad9da700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd4a80af5c0 0x7fd4a8143f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40156/0 (socket says 172.21.15.32:40156) 2020-12-05T17:39:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ad9da700 1 -- 172.21.15.32:0/2508526388 learned_addr learned my addr 172.21.15.32:0/2508526388 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:00.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ae1db700 1 --2- 172.21.15.32:0/2508526388 >> v2:172.21.15.61:3300/0 conn(0x7fd4a8144610 0x7fd4a813fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ac9d8700 1 -- 172.21.15.32:0/2508526388 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814355527 0 0) 0x7fd4a81407e0 con 0x7fd4a80af5c0 2020-12-05T17:39:00.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ac9d8700 1 -- 172.21.15.32:0/2508526388 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd490003230 con 0x7fd4a80af5c0 2020-12-05T17:39:00.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ac9d8700 1 -- 172.21.15.32:0/2508526388 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3482383784 0 0) 0x7fd490003230 con 0x7fd4a80af5c0 2020-12-05T17:39:00.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ac9d8700 1 -- 172.21.15.32:0/2508526388 >> v2:172.21.15.61:3300/0 conn(0x7fd4a8144610 msgr2=0x7fd4a813fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:00.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ac9d8700 1 --2- 172.21.15.32:0/2508526388 >> v2:172.21.15.61:3300/0 conn(0x7fd4a8144610 0x7fd4a813fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ac9d8700 1 -- 172.21.15.32:0/2508526388 >> v1:172.21.15.61:6789/0 conn(0x7fd4a80af9a0 legacy=0x7fd4a813f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:00.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ac9d8700 1 -- 172.21.15.32:0/2508526388 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a8061b30 con 0x7fd4a80af5c0 2020-12-05T17:39:00.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ac9d8700 1 -- 172.21.15.32:0/2508526388 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fd4a4001bd0 con 0x7fd4a80af5c0 2020-12-05T17:39:00.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ac9d8700 1 -- 172.21.15.32:0/2508526388 >> v1:172.21.15.32:6789/0 conn(0x7fd4a80af5c0 legacy=0x7fd4a8143f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:00.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ac9d8700 1 --2- 172.21.15.32:0/2508526388 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd490003720 0x7fd490003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ac9d8700 1 --2- 172.21.15.32:0/2508526388 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4900044d0 0x7fd490004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ac9d8700 1 --2- 172.21.15.32:0/2508526388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd490004e40 0x7fd49000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ac9d8700 1 -- 172.21.15.32:0/2508526388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd490004250 con 0x7fd490004e40 2020-12-05T17:39:00.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ac9d8700 1 -- 172.21.15.32:0/2508526388 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49000bc10 con 0x7fd490003720 2020-12-05T17:39:00.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.659+0000 7fd4ac9d8700 1 -- 172.21.15.32:0/2508526388 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49000bd50 con 0x7fd4900044d0 2020-12-05T17:39:00.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ae1db700 1 --2- 172.21.15.32:0/2508526388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd490004e40 0x7fd49000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ad9da700 1 --2- 172.21.15.32:0/2508526388 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd490003720 0x7fd490003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ad1d9700 1 --2- 172.21.15.32:0/2508526388 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4900044d0 0x7fd490004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ae1db700 1 -- 172.21.15.32:0/2508526388 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4900044d0 msgr2=0x7fd490004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:00.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ae1db700 1 --2- 172.21.15.32:0/2508526388 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4900044d0 0x7fd490004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.675 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ae1db700 1 -- 172.21.15.32:0/2508526388 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd490003720 msgr2=0x7fd490003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:00.675 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ae1db700 1 --2- 172.21.15.32:0/2508526388 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd490003720 0x7fd490003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.675 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ae1db700 1 -- 172.21.15.32:0/2508526388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd490003230 con 0x7fd490004e40 2020-12-05T17:39:00.675 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ad9da700 1 --2- 172.21.15.32:0/2508526388 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd490003720 0x7fd490003b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:00.676 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ad1d9700 1 --2- 172.21.15.32:0/2508526388 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4900044d0 0x7fd490004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:39:00.676 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ae1db700 1 -- 172.21.15.32:0/2508526388 reap_dead start 2020-12-05T17:39:00.676 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ae1db700 1 --2- 172.21.15.32:0/2508526388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd490004e40 0x7fd49000b720 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7fd4a000c000 tx=0x7fd4a0008040).ready entity=mon.0 client_cookie=506cdfe5f60cc394 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:00.676 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ac9d8700 1 -- 172.21.15.32:0/2508526388 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4a000e9d0 con 0x7fd490004e40 2020-12-05T17:39:00.677 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ac9d8700 1 -- 172.21.15.32:0/2508526388 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a000eb30 con 0x7fd490004e40 2020-12-05T17:39:00.677 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ac9d8700 1 -- 172.21.15.32:0/2508526388 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4a000ee90 con 0x7fd490004e40 2020-12-05T17:39:00.677 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ae9dc700 1 -- 172.21.15.32:0/2508526388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd490004e40 msgr2=0x7fd49000b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:00.677 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ae9dc700 1 --2- 172.21.15.32:0/2508526388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd490004e40 0x7fd49000b720 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7fd4a000c000 tx=0x7fd4a0008040).stop 2020-12-05T17:39:00.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ae9dc700 1 -- 172.21.15.32:0/2508526388 shutdown_connections 2020-12-05T17:39:00.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ae9dc700 1 --2- 172.21.15.32:0/2508526388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd490004e40 0x7fd49000b720 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ae9dc700 1 -- 172.21.15.32:0/2508526388 >> 172.21.15.32:0/2508526388 conn(0x7fd4a8061010 msgr2=0x7fd4a8061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:00.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ae9dc700 1 -- 172.21.15.32:0/2508526388 shutdown_connections 2020-12-05T17:39:00.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ae9dc700 1 -- 172.21.15.32:0/2508526388 wait complete. 2020-12-05T17:39:00.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ae9dc700 1 Processor -- start 2020-12-05T17:39:00.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.663+0000 7fd4ae9dc700 1 -- start start 2020-12-05T17:39:00.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd4ae9dc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd490004e40 0x7fd4a813c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd4ae9dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a80b3a00 0x7fd4a80656c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd4ae9dc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4a8065bb0 0x7fd4a81524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd4ae9dc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a80b5810 con 0x7fd4a80b3a00 2020-12-05T17:39:00.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd4ae9dc700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a80b3e80 con 0x7fd490004e40 2020-12-05T17:39:00.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd4ae9dc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a8065230 con 0x7fd4a8065bb0 2020-12-05T17:39:00.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd4ad1d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a80b3a00 0x7fd4a80656c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd4ad9da700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd490004e40 0x7fd4a813c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd4ae1db700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4a8065bb0 0x7fd4a81524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd4ad1d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a80b3a00 0x7fd4a80656c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37524/0 (socket says 172.21.15.32:37524) 2020-12-05T17:39:00.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd4ad9da700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd490004e40 0x7fd4a813c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:57412/0 (socket says 172.21.15.32:57412) 2020-12-05T17:39:00.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd4ad1d9700 1 -- 172.21.15.32:0/2326647597 learned_addr learned my addr 172.21.15.32:0/2326647597 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:00.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd4ad1d9700 1 -- 172.21.15.32:0/2326647597 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4a8065bb0 msgr2=0x7fd4a81524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:00.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd4ad1d9700 1 --2- 172.21.15.32:0/2326647597 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4a8065bb0 0x7fd4a81524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd4ad1d9700 1 -- 172.21.15.32:0/2326647597 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd490004e40 msgr2=0x7fd4a813c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:00.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd4ad1d9700 1 --2- 172.21.15.32:0/2326647597 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd490004e40 0x7fd4a813c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd4ad1d9700 1 -- 172.21.15.32:0/2326647597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a8066030 con 0x7fd4a80b3a00 2020-12-05T17:39:00.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd4ad1d9700 1 --2- 172.21.15.32:0/2326647597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a80b3a00 0x7fd4a80656c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7fd498008000 tx=0x7fd498009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:00.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd496ffd700 1 -- 172.21.15.32:0/2326647597 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4980104e0 con 0x7fd4a80b3a00 2020-12-05T17:39:00.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd496ffd700 1 -- 172.21.15.32:0/2326647597 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd498010640 con 0x7fd4a80b3a00 2020-12-05T17:39:00.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd4ae9dc700 1 -- 172.21.15.32:0/2326647597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a80b5a40 con 0x7fd4a80b3a00 2020-12-05T17:39:00.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd496ffd700 1 -- 172.21.15.32:0/2326647597 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4980109a0 con 0x7fd4a80b3a00 2020-12-05T17:39:00.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd4ae9dc700 1 -- 172.21.15.32:0/2326647597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a8063a80 con 0x7fd4a80b3a00 2020-12-05T17:39:00.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd4ae9dc700 1 -- 172.21.15.32:0/2326647597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a8057050 con 0x7fd4a80b3a00 2020-12-05T17:39:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.667+0000 7fd496ffd700 1 -- 172.21.15.32:0/2326647597 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd498010b00 con 0x7fd4a80b3a00 2020-12-05T17:39:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.671+0000 7fd496ffd700 1 --2- 172.21.15.32:0/2326647597 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd488055440 0x7fd488057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.671+0000 7fd496ffd700 1 -- 172.21.15.32:0/2326647597 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd49806dcf0 con 0x7fd4a80b3a00 2020-12-05T17:39:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.671+0000 7fd4ad9da700 1 --2- 172.21.15.32:0/2326647597 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd488055440 0x7fd488057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.671+0000 7fd4ad9da700 1 --2- 172.21.15.32:0/2326647597 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd488055440 0x7fd488057900 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7fd4a400d000 tx=0x7fd4a400c040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.675+0000 7fd496ffd700 1 -- 172.21.15.32:0/2326647597 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd498043e90 con 0x7fd4a80b3a00 2020-12-05T17:39:00.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.819+0000 7fd4ae9dc700 1 -- 172.21.15.32:0/2326647597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd4a8069690 con 0x7fd4a80b3a00 2020-12-05T17:39:00.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.823+0000 7fd496ffd700 1 -- 172.21.15.32:0/2326647597 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v83) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd4980474b0 con 0x7fd4a80b3a00 2020-12-05T17:39:00.828 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:39:00.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.823+0000 7fd4ae9dc700 1 -- 172.21.15.32:0/2326647597 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd488055440 msgr2=0x7fd488057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:00.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.823+0000 7fd4ae9dc700 1 --2- 172.21.15.32:0/2326647597 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd488055440 0x7fd488057900 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7fd4a400d000 tx=0x7fd4a400c040).stop 2020-12-05T17:39:00.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.823+0000 7fd4ae9dc700 1 -- 172.21.15.32:0/2326647597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a80b3a00 msgr2=0x7fd4a80656c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:00.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.823+0000 7fd4ae9dc700 1 --2- 172.21.15.32:0/2326647597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a80b3a00 0x7fd4a80656c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7fd498008000 tx=0x7fd498009040).stop 2020-12-05T17:39:00.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.823+0000 7fd4ae9dc700 1 -- 172.21.15.32:0/2326647597 shutdown_connections 2020-12-05T17:39:00.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.823+0000 7fd4ae9dc700 1 --2- 172.21.15.32:0/2326647597 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd488055440 0x7fd488057900 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.823+0000 7fd4ae9dc700 1 --2- 172.21.15.32:0/2326647597 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4a8065bb0 0x7fd4a81524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.823+0000 7fd4ae9dc700 1 --2- 172.21.15.32:0/2326647597 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd490004e40 0x7fd4a813c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.823+0000 7fd4ae9dc700 1 --2- 172.21.15.32:0/2326647597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a80b3a00 0x7fd4a80656c0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.823+0000 7fd4ae9dc700 1 -- 172.21.15.32:0/2326647597 >> 172.21.15.32:0/2326647597 conn(0x7fd4a8061010 msgr2=0x7fd4a80b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:00.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.823+0000 7fd4ae9dc700 1 -- 172.21.15.32:0/2326647597 shutdown_connections 2020-12-05T17:39:00.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.823+0000 7fd4ae9dc700 1 -- 172.21.15.32:0/2326647597 wait complete. 2020-12-05T17:39:00.842 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:39:00.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.955+0000 7fd4ca7b5700 1 Processor -- start 2020-12-05T17:39:00.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.955+0000 7fd4ca7b5700 1 -- start start 2020-12-05T17:39:00.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4ca7b5700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd4c4061b30 0x7fd4c4141e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4ca7b5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c41439c0 con 0x7fd4c405f6f0 2020-12-05T17:39:00.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4ca7b5700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c405fe20 con 0x7fd4c4142300 2020-12-05T17:39:00.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4ca7b5700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4c4146910 con 0x7fd4c4061b30 2020-12-05T17:39:00.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4c97b3700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd4c405f6f0 0x7fd4c4141700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40176/0 (socket says 172.21.15.32:40176) 2020-12-05T17:39:00.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4c97b3700 1 -- 172.21.15.32:0/1398161735 learned_addr learned my addr 172.21.15.32:0/1398161735 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:00.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4c8fb2700 1 --2- 172.21.15.32:0/1398161735 >> v2:172.21.15.61:3300/0 conn(0x7fd4c4061b30 0x7fd4c4141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4b3fff700 1 -- 172.21.15.32:0/1398161735 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2373974639 0 0) 0x7fd4c41439c0 con 0x7fd4c405f6f0 2020-12-05T17:39:00.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4b3fff700 1 -- 172.21.15.32:0/1398161735 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4ac003230 con 0x7fd4c405f6f0 2020-12-05T17:39:00.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4b3fff700 1 -- 172.21.15.32:0/1398161735 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1416573664 0 0) 0x7fd4c405fe20 con 0x7fd4c4142300 2020-12-05T17:39:00.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4b3fff700 1 -- 172.21.15.32:0/1398161735 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4c41439c0 con 0x7fd4c4142300 2020-12-05T17:39:00.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4b3fff700 1 -- 172.21.15.32:0/1398161735 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 897645319 0 0) 0x7fd4ac003230 con 0x7fd4c405f6f0 2020-12-05T17:39:00.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4b3fff700 1 -- 172.21.15.32:0/1398161735 >> v2:172.21.15.61:3300/0 conn(0x7fd4c4061b30 msgr2=0x7fd4c4141e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:00.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4b3fff700 1 --2- 172.21.15.32:0/1398161735 >> v2:172.21.15.61:3300/0 conn(0x7fd4c4061b30 0x7fd4c4141e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4b3fff700 1 -- 172.21.15.32:0/1398161735 >> v1:172.21.15.61:6789/0 conn(0x7fd4c4142300 legacy=0x7fd4c4142790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:00.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4b3fff700 1 -- 172.21.15.32:0/1398161735 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c405fad0 con 0x7fd4c405f6f0 2020-12-05T17:39:00.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4b3fff700 1 -- 172.21.15.32:0/1398161735 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fd4c0001850 con 0x7fd4c405f6f0 2020-12-05T17:39:00.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4b3fff700 1 -- 172.21.15.32:0/1398161735 >> v1:172.21.15.32:6789/0 conn(0x7fd4c405f6f0 legacy=0x7fd4c4141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:00.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4b3fff700 1 --2- 172.21.15.32:0/1398161735 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4ac003900 0x7fd4ac003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4b3fff700 1 --2- 172.21.15.32:0/1398161735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ac0044d0 0x7fd4ac004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4b3fff700 1 --2- 172.21.15.32:0/1398161735 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4ac004e40 0x7fd4ac00b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4b3fff700 1 -- 172.21.15.32:0/1398161735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c4146910 con 0x7fd4ac0044d0 2020-12-05T17:39:00.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4b3fff700 1 -- 172.21.15.32:0/1398161735 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac00bc10 con 0x7fd4ac003900 2020-12-05T17:39:00.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4b3fff700 1 -- 172.21.15.32:0/1398161735 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac00bd50 con 0x7fd4ac004e40 2020-12-05T17:39:00.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4c8fb2700 1 --2- 172.21.15.32:0/1398161735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ac0044d0 0x7fd4ac004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4c97b3700 1 --2- 172.21.15.32:0/1398161735 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4ac003900 0x7fd4ac003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4c9fb4700 1 --2- 172.21.15.32:0/1398161735 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4ac004e40 0x7fd4ac00b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4c8fb2700 1 -- 172.21.15.32:0/1398161735 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4ac004e40 msgr2=0x7fd4ac00b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:00.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4c8fb2700 1 --2- 172.21.15.32:0/1398161735 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4ac004e40 0x7fd4ac00b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4c8fb2700 1 -- 172.21.15.32:0/1398161735 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4ac003900 msgr2=0x7fd4ac003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4c8fb2700 1 --2- 172.21.15.32:0/1398161735 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4ac003900 0x7fd4ac003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4c8fb2700 1 -- 172.21.15.32:0/1398161735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac003230 con 0x7fd4ac0044d0 2020-12-05T17:39:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.959+0000 7fd4c9fb4700 1 -- 172.21.15.32:0/1398161735 reap_dead start 2020-12-05T17:39:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.963+0000 7fd4c8fb2700 1 --2- 172.21.15.32:0/1398161735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ac0044d0 0x7fd4ac004950 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7fd4b400c000 tx=0x7fd4b4008040).ready entity=mon.0 client_cookie=5389f1d4bca142dc server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.963+0000 7fd4b3fff700 1 -- 172.21.15.32:0/1398161735 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4b400e8a0 con 0x7fd4ac0044d0 2020-12-05T17:39:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.963+0000 7fd4b3fff700 1 -- 172.21.15.32:0/1398161735 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b400ea00 con 0x7fd4ac0044d0 2020-12-05T17:39:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.963+0000 7fd4b3fff700 1 -- 172.21.15.32:0/1398161735 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4b400ed60 con 0x7fd4ac0044d0 2020-12-05T17:39:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.963+0000 7fd4ca7b5700 1 -- 172.21.15.32:0/1398161735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ac0044d0 msgr2=0x7fd4ac004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.963+0000 7fd4ca7b5700 1 --2- 172.21.15.32:0/1398161735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ac0044d0 0x7fd4ac004950 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7fd4b400c000 tx=0x7fd4b4008040).stop 2020-12-05T17:39:00.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.963+0000 7fd4ca7b5700 1 -- 172.21.15.32:0/1398161735 shutdown_connections 2020-12-05T17:39:00.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.963+0000 7fd4ca7b5700 1 --2- 172.21.15.32:0/1398161735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ac0044d0 0x7fd4ac004950 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.963+0000 7fd4ca7b5700 1 -- 172.21.15.32:0/1398161735 >> 172.21.15.32:0/1398161735 conn(0x7fd4c4061010 msgr2=0x7fd4c4061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:00.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.963+0000 7fd4ca7b5700 1 -- 172.21.15.32:0/1398161735 shutdown_connections 2020-12-05T17:39:00.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.963+0000 7fd4ca7b5700 1 -- 172.21.15.32:0/1398161735 wait complete. 2020-12-05T17:39:00.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.963+0000 7fd4ca7b5700 1 Processor -- start 2020-12-05T17:39:00.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.963+0000 7fd4ca7b5700 1 -- start start 2020-12-05T17:39:00.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4ca7b5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ac0044d0 0x7fd4c40b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4ca7b5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4c4063040 0x7fd4c40b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4ca7b5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4c413dba0 0x7fd4c413e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4ca7b5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c40b4e50 con 0x7fd4ac0044d0 2020-12-05T17:39:00.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4ca7b5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c4065000 con 0x7fd4c4063040 2020-12-05T17:39:00.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4ca7b5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c40652a0 con 0x7fd4c413dba0 2020-12-05T17:39:00.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4c97b3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ac0044d0 0x7fd4c40b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4c97b3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ac0044d0 0x7fd4c40b5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37540/0 (socket says 172.21.15.32:37540) 2020-12-05T17:39:00.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4c97b3700 1 -- 172.21.15.32:0/3649408907 learned_addr learned my addr 172.21.15.32:0/3649408907 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:00.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4c8fb2700 1 --2- 172.21.15.32:0/3649408907 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4c4063040 0x7fd4c40b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4c9fb4700 1 --2- 172.21.15.32:0/3649408907 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4c413dba0 0x7fd4c413e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4c97b3700 1 -- 172.21.15.32:0/3649408907 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4c413dba0 msgr2=0x7fd4c413e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:00.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4c97b3700 1 --2- 172.21.15.32:0/3649408907 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4c413dba0 0x7fd4c413e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4c97b3700 1 -- 172.21.15.32:0/3649408907 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4c4063040 msgr2=0x7fd4c40b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:00.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4c97b3700 1 --2- 172.21.15.32:0/3649408907 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4c4063040 0x7fd4c40b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:00.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4c97b3700 1 -- 172.21.15.32:0/3649408907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c40b4500 con 0x7fd4ac0044d0 2020-12-05T17:39:00.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4c97b3700 1 --2- 172.21.15.32:0/3649408907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ac0044d0 0x7fd4c40b5160 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7fd4c0002c40 tx=0x7fd4c0009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:00.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4b27fc700 1 -- 172.21.15.32:0/3649408907 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4c00072c0 con 0x7fd4ac0044d0 2020-12-05T17:39:00.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4b27fc700 1 -- 172.21.15.32:0/3649408907 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c000d8f0 con 0x7fd4ac0044d0 2020-12-05T17:39:00.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4ca7b5700 1 -- 172.21.15.32:0/3649408907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4c413e510 con 0x7fd4ac0044d0 2020-12-05T17:39:00.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4b27fc700 1 -- 172.21.15.32:0/3649408907 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4c000dc50 con 0x7fd4ac0044d0 2020-12-05T17:39:00.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4ca7b5700 1 -- 172.21.15.32:0/3649408907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c4064d40 con 0x7fd4ac0044d0 2020-12-05T17:39:00.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4b27fc700 1 -- 172.21.15.32:0/3649408907 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd4c001a030 con 0x7fd4ac0044d0 2020-12-05T17:39:00.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4ca7b5700 1 -- 172.21.15.32:0/3649408907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4c4065460 con 0x7fd4ac0044d0 2020-12-05T17:39:00.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4b27fc700 1 --2- 172.21.15.32:0/3649408907 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd4a4055330 0x7fd4a40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:00.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.967+0000 7fd4b27fc700 1 -- 172.21.15.32:0/3649408907 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd4c006db90 con 0x7fd4ac0044d0 2020-12-05T17:39:00.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.971+0000 7fd4c8fb2700 1 --2- 172.21.15.32:0/3649408907 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd4a4055330 0x7fd4a40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:00.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.971+0000 7fd4c8fb2700 1 --2- 172.21.15.32:0/3649408907 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd4a4055330 0x7fd4a40577f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fd4b4008000 tx=0x7fd4b401a000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:00.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:00.975+0000 7fd4b27fc700 1 -- 172.21.15.32:0/3649408907 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd4c0043d30 con 0x7fd4ac0044d0 2020-12-05T17:39:01.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.115+0000 7fd4ca7b5700 1 -- 172.21.15.32:0/3649408907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd4c4057050 con 0x7fd4ac0044d0 2020-12-05T17:39:01.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.115+0000 7fd4b27fc700 1 -- 172.21.15.32:0/3649408907 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v83) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd4c0047350 con 0x7fd4ac0044d0 2020-12-05T17:39:01.120 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:39:01.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.115+0000 7fd4ca7b5700 1 -- 172.21.15.32:0/3649408907 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd4a4055330 msgr2=0x7fd4a40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:01.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.115+0000 7fd4ca7b5700 1 --2- 172.21.15.32:0/3649408907 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd4a4055330 0x7fd4a40577f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fd4b4008000 tx=0x7fd4b401a000).stop 2020-12-05T17:39:01.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.115+0000 7fd4ca7b5700 1 -- 172.21.15.32:0/3649408907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ac0044d0 msgr2=0x7fd4c40b5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:01.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.115+0000 7fd4ca7b5700 1 --2- 172.21.15.32:0/3649408907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ac0044d0 0x7fd4c40b5160 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7fd4c0002c40 tx=0x7fd4c0009040).stop 2020-12-05T17:39:01.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.115+0000 7fd4ca7b5700 1 -- 172.21.15.32:0/3649408907 shutdown_connections 2020-12-05T17:39:01.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.115+0000 7fd4ca7b5700 1 --2- 172.21.15.32:0/3649408907 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd4a4055330 0x7fd4a40577f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:01.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.115+0000 7fd4ca7b5700 1 --2- 172.21.15.32:0/3649408907 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4c413dba0 0x7fd4c413e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:01.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.115+0000 7fd4ca7b5700 1 --2- 172.21.15.32:0/3649408907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ac0044d0 0x7fd4c40b5160 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:01.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.115+0000 7fd4ca7b5700 1 --2- 172.21.15.32:0/3649408907 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4c4063040 0x7fd4c40b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:01.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.115+0000 7fd4ca7b5700 1 -- 172.21.15.32:0/3649408907 >> 172.21.15.32:0/3649408907 conn(0x7fd4c4061010 msgr2=0x7fd4c4145c90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:01.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.115+0000 7fd4ca7b5700 1 -- 172.21.15.32:0/3649408907 shutdown_connections 2020-12-05T17:39:01.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.115+0000 7fd4ca7b5700 1 -- 172.21.15.32:0/3649408907 wait complete. 2020-12-05T17:39:01.135 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:39:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.251+0000 7fd1ebfff700 1 Processor -- start 2020-12-05T17:39:01.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.251+0000 7fd1ebfff700 1 -- start start 2020-12-05T17:39:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.251+0000 7fd1ebfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1ec13f610 0x7fd1ec13fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.251+0000 7fd1ebfff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd1ec0af4b0 0x7fd1ec0af920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.251+0000 7fd1ebfff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ec0b23e0 con 0x7fd1ec0b2000 2020-12-05T17:39:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.251+0000 7fd1ebfff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ec0aeba0 con 0x7fd1ec0af4b0 2020-12-05T17:39:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.251+0000 7fd1ebfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec0af190 con 0x7fd1ec13f610 2020-12-05T17:39:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.251+0000 7fd1eaffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd1ec0b2000 0x7fd1ec142f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40196/0 (socket says 172.21.15.32:40196) 2020-12-05T17:39:01.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.251+0000 7fd1eaffd700 1 -- 172.21.15.32:0/2705311604 learned_addr learned my addr 172.21.15.32:0/2705311604 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:01.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.251+0000 7fd1ea7fc700 1 --2- 172.21.15.32:0/2705311604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1ec13f610 0x7fd1ec13fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:01.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.251+0000 7fd1eb7fe700 1 --2- 172.21.15.32:0/2705311604 >> v2:172.21.15.61:3300/0 conn(0x7fd1ec0af4b0 0x7fd1ec0af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:01.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.251+0000 7fd1e9ffb700 1 -- 172.21.15.32:0/2705311604 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371058262 0 0) 0x7fd1ec0b23e0 con 0x7fd1ec0b2000 2020-12-05T17:39:01.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.251+0000 7fd1e9ffb700 1 -- 172.21.15.32:0/2705311604 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1cc003230 con 0x7fd1ec0b2000 2020-12-05T17:39:01.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.251+0000 7fd1e9ffb700 1 -- 172.21.15.32:0/2705311604 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1719170027 0 0) 0x7fd1cc003230 con 0x7fd1ec0b2000 2020-12-05T17:39:01.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.251+0000 7fd1e9ffb700 1 -- 172.21.15.32:0/2705311604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1ec13f610 msgr2=0x7fd1ec13fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:01.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.251+0000 7fd1e9ffb700 1 --2- 172.21.15.32:0/2705311604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1ec13f610 0x7fd1ec13fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:01.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.251+0000 7fd1e9ffb700 1 -- 172.21.15.32:0/2705311604 >> v2:172.21.15.61:3300/0 conn(0x7fd1ec0af4b0 msgr2=0x7fd1ec0af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:01.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.251+0000 7fd1e9ffb700 1 --2- 172.21.15.32:0/2705311604 >> v2:172.21.15.61:3300/0 conn(0x7fd1ec0af4b0 0x7fd1ec0af920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:01.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.251+0000 7fd1e9ffb700 1 -- 172.21.15.32:0/2705311604 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec0ae010 con 0x7fd1ec0b2000 2020-12-05T17:39:01.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.251+0000 7fd1e9ffb700 1 -- 172.21.15.32:0/2705311604 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fd1dc0016d0 con 0x7fd1ec0b2000 2020-12-05T17:39:01.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1e9ffb700 1 -- 172.21.15.32:0/2705311604 >> v1:172.21.15.32:6789/0 conn(0x7fd1ec0b2000 legacy=0x7fd1ec142f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:01.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1e9ffb700 1 --2- 172.21.15.32:0/2705311604 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1cc003720 0x7fd1cc003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:01.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1e9ffb700 1 --2- 172.21.15.32:0/2705311604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1cc0044d0 0x7fd1cc004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:01.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1e9ffb700 1 --2- 172.21.15.32:0/2705311604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1ec13f610 0x7fd1cc004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:01.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1e9ffb700 1 -- 172.21.15.32:0/2705311604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec0aeba0 con 0x7fd1cc0044d0 2020-12-05T17:39:01.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1e9ffb700 1 -- 172.21.15.32:0/2705311604 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec0af190 con 0x7fd1cc003720 2020-12-05T17:39:01.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1e9ffb700 1 -- 172.21.15.32:0/2705311604 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc004260 con 0x7fd1ec13f610 2020-12-05T17:39:01.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1ea7fc700 1 --2- 172.21.15.32:0/2705311604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1cc0044d0 0x7fd1cc004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:01.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1eb7fe700 1 --2- 172.21.15.32:0/2705311604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1ec13f610 0x7fd1cc004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:01.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1ea7fc700 1 -- 172.21.15.32:0/2705311604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1ec13f610 msgr2=0x7fd1cc004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:01.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1ea7fc700 1 --2- 172.21.15.32:0/2705311604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1ec13f610 0x7fd1cc004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:01.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1ea7fc700 1 -- 172.21.15.32:0/2705311604 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1cc003720 msgr2=0x7fd1cc003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:01.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1ea7fc700 1 --2- 172.21.15.32:0/2705311604 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1cc003720 0x7fd1cc003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-12-05T17:39:01.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1ea7fc700 1 -- 172.21.15.32:0/2705311604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc003230 con 0x7fd1cc0044d0 2020-12-05T17:39:01.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1ea7fc700 1 --2- 172.21.15.32:0/2705311604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1cc0044d0 0x7fd1cc004950 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7fd1d400c000 tx=0x7fd1d4008040).ready entity=mon.0 client_cookie=876ffd34f4f3a6ca server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:01.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1e9ffb700 1 -- 172.21.15.32:0/2705311604 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1d400e9d0 con 0x7fd1cc0044d0 2020-12-05T17:39:01.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1e9ffb700 1 -- 172.21.15.32:0/2705311604 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d400eb30 con 0x7fd1cc0044d0 2020-12-05T17:39:01.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1e9ffb700 1 -- 172.21.15.32:0/2705311604 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1d400ee90 con 0x7fd1cc0044d0 2020-12-05T17:39:01.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1ebfff700 1 -- 172.21.15.32:0/2705311604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1cc0044d0 msgr2=0x7fd1cc004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:01.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1ebfff700 1 --2- 172.21.15.32:0/2705311604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1cc0044d0 0x7fd1cc004950 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7fd1d400c000 tx=0x7fd1d4008040).stop 2020-12-05T17:39:01.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1eb7fe700 1 -- 172.21.15.32:0/2705311604 reap_dead start 2020-12-05T17:39:01.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1ebfff700 1 -- 172.21.15.32:0/2705311604 shutdown_connections 2020-12-05T17:39:01.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1ebfff700 1 -- 172.21.15.32:0/2705311604 >> 172.21.15.32:0/2705311604 conn(0x7fd1ec0b5620 msgr2=0x7fd1ec0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:01.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1ebfff700 1 -- 172.21.15.32:0/2705311604 shutdown_connections 2020-12-05T17:39:01.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1ebfff700 1 -- 172.21.15.32:0/2705311604 wait complete. 2020-12-05T17:39:01.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1ebfff700 1 Processor -- start 2020-12-05T17:39:01.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1ebfff700 1 -- start start 2020-12-05T17:39:01.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1ebfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1ec0afc60 0x7fd1ec154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:01.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1ebfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1ec063550 0x7fd1ec13b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:01.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1ebfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec112110 0x7fd1ec13bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:01.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1ebfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec064460 con 0x7fd1ec112110 2020-12-05T17:39:01.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1ebfff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec13c270 con 0x7fd1ec063550 2020-12-05T17:39:01.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1ebfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec0632f0 con 0x7fd1ec0afc60 2020-12-05T17:39:01.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1eb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec112110 0x7fd1ec13bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:01.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.255+0000 7fd1eaffd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1ec0afc60 0x7fd1ec154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:01.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.259+0000 7fd1eb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec112110 0x7fd1ec13bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37564/0 (socket says 172.21.15.32:37564) 2020-12-05T17:39:01.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.259+0000 7fd1eaffd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1ec0afc60 0x7fd1ec154cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:34350/0 (socket says 172.21.15.32:34350) 2020-12-05T17:39:01.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.259+0000 7fd1eb7fe700 1 -- 172.21.15.32:0/3532074186 learned_addr learned my addr 172.21.15.32:0/3532074186 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:01.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.259+0000 7fd1ea7fc700 1 --2- 172.21.15.32:0/3532074186 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1ec063550 0x7fd1ec13b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:01.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.259+0000 7fd1eb7fe700 1 -- 172.21.15.32:0/3532074186 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1ec0afc60 msgr2=0x7fd1ec154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:01.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.259+0000 7fd1eb7fe700 1 --2- 172.21.15.32:0/3532074186 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1ec0afc60 0x7fd1ec154cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:01.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.259+0000 7fd1eb7fe700 1 -- 172.21.15.32:0/3532074186 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1ec063550 msgr2=0x7fd1ec13b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:01.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.259+0000 7fd1eb7fe700 1 --2- 172.21.15.32:0/3532074186 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1ec063550 0x7fd1ec13b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:01.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.259+0000 7fd1eb7fe700 1 -- 172.21.15.32:0/3532074186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec13dbf0 con 0x7fd1ec112110 2020-12-05T17:39:01.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.259+0000 7fd1eb7fe700 1 --2- 172.21.15.32:0/3532074186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec112110 0x7fd1ec13bd80 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7fd1e400b000 tx=0x7fd1e4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:01.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.259+0000 7fd1e3fff700 1 -- 172.21.15.32:0/3532074186 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1e400ea90 con 0x7fd1ec112110 2020-12-05T17:39:01.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.259+0000 7fd1e3fff700 1 -- 172.21.15.32:0/3532074186 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e400ebf0 con 0x7fd1ec112110 2020-12-05T17:39:01.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.259+0000 7fd1ebfff700 1 -- 172.21.15.32:0/3532074186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ec0b0040 con 0x7fd1ec112110 2020-12-05T17:39:01.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.259+0000 7fd1e3fff700 1 -- 172.21.15.32:0/3532074186 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1e4014470 con 0x7fd1ec112110 2020-12-05T17:39:01.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.259+0000 7fd1ebfff700 1 -- 172.21.15.32:0/3532074186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec0664e0 con 0x7fd1ec112110 2020-12-05T17:39:01.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.259+0000 7fd1ebfff700 1 -- 172.21.15.32:0/3532074186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ec057050 con 0x7fd1ec112110 2020-12-05T17:39:01.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.259+0000 7fd1e3fff700 1 -- 172.21.15.32:0/3532074186 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd1e40145d0 con 0x7fd1ec112110 2020-12-05T17:39:01.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.263+0000 7fd1e3fff700 1 --2- 172.21.15.32:0/3532074186 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1d0055500 0x7fd1d00579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:01.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.263+0000 7fd1eaffd700 1 --2- 172.21.15.32:0/3532074186 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1d0055500 0x7fd1d00579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:01.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.263+0000 7fd1e3fff700 1 -- 172.21.15.32:0/3532074186 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd1e406ec20 con 0x7fd1ec112110 2020-12-05T17:39:01.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.263+0000 7fd1eaffd700 1 --2- 172.21.15.32:0/3532074186 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1d0055500 0x7fd1d00579c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7fd1dc00d000 tx=0x7fd1ec0ae1a0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:01.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.267+0000 7fd1e3fff700 1 -- 172.21.15.32:0/3532074186 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd1e4044bd0 con 0x7fd1ec112110 2020-12-05T17:39:01.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.411+0000 7fd1ebfff700 1 -- 172.21.15.32:0/3532074186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd1ec0008d0 con 0x7fd1ec112110 2020-12-05T17:39:01.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.411+0000 7fd1e3fff700 1 -- 172.21.15.32:0/3532074186 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+10 (secure 0 0 0) 0x7fd1e40481f0 con 0x7fd1ec112110 2020-12-05T17:39:01.420 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:39:01.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.415+0000 7fd1ebfff700 1 -- 172.21.15.32:0/3532074186 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1d0055500 msgr2=0x7fd1d00579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:01.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.415+0000 7fd1ebfff700 1 --2- 172.21.15.32:0/3532074186 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1d0055500 0x7fd1d00579c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7fd1dc00d000 tx=0x7fd1ec0ae1a0).stop 2020-12-05T17:39:01.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.415+0000 7fd1ebfff700 1 -- 172.21.15.32:0/3532074186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec112110 msgr2=0x7fd1ec13bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:01.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.415+0000 7fd1ebfff700 1 --2- 172.21.15.32:0/3532074186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec112110 0x7fd1ec13bd80 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7fd1e400b000 tx=0x7fd1e4008040).stop 2020-12-05T17:39:01.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.415+0000 7fd1ebfff700 1 -- 172.21.15.32:0/3532074186 shutdown_connections 2020-12-05T17:39:01.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.415+0000 7fd1ebfff700 1 --2- 172.21.15.32:0/3532074186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec112110 0x7fd1ec13bd80 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:01.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.415+0000 7fd1ebfff700 1 --2- 172.21.15.32:0/3532074186 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1d0055500 0x7fd1d00579c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:01.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.415+0000 7fd1ebfff700 1 --2- 172.21.15.32:0/3532074186 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1ec0afc60 0x7fd1ec154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:01.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.415+0000 7fd1ebfff700 1 --2- 172.21.15.32:0/3532074186 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1ec063550 0x7fd1ec13b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:01.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.415+0000 7fd1ebfff700 1 -- 172.21.15.32:0/3532074186 >> 172.21.15.32:0/3532074186 conn(0x7fd1ec0b5620 msgr2=0x7fd1ec0b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:01.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.415+0000 7fd1ebfff700 1 -- 172.21.15.32:0/3532074186 shutdown_connections 2020-12-05T17:39:01.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.415+0000 7fd1ebfff700 1 -- 172.21.15.32:0/3532074186 wait complete. 2020-12-05T17:39:01.433 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:39:01.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.543+0000 7fee566e7700 1 Processor -- start 2020-12-05T17:39:01.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.543+0000 7fee566e7700 1 -- start start 2020-12-05T17:39:01.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.543+0000 7fee566e7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee500b5f00 0x7fee500b5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:01.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.543+0000 7fee566e7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee500b1920 con 0x7fee500b6690 2020-12-05T17:39:01.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.543+0000 7fee566e7700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee500b1bc0 con 0x7fee500b5b20 2020-12-05T17:39:01.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.543+0000 7fee566e7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee500b5730 con 0x7fee500b5f00 2020-12-05T17:39:01.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.543+0000 7fee54c84700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee500b5f00 0x7fee500b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:01.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.543+0000 7fee4f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fee500b6690 0x7fee50145f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40218/0 (socket says 172.21.15.32:40218) 2020-12-05T17:39:01.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.543+0000 7fee54c84700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee500b5f00 0x7fee500b5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37572/0 (socket says 172.21.15.32:37572) 2020-12-05T17:39:01.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.543+0000 7fee4f7fe700 1 -- 172.21.15.32:0/3580104046 learned_addr learned my addr 172.21.15.32:0/3580104046 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:01.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.543+0000 7fee54c84700 1 -- 172.21.15.32:0/3580104046 >> v1:172.21.15.61:6789/0 conn(0x7fee500b5b20 legacy=0x7fee500b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:01.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.543+0000 7fee54c84700 1 -- 172.21.15.32:0/3580104046 >> v1:172.21.15.32:6789/0 conn(0x7fee500b6690 legacy=0x7fee50145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:01.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.543+0000 7fee54c84700 1 -- 172.21.15.32:0/3580104046 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee50146620 con 0x7fee500b5f00 2020-12-05T17:39:01.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.543+0000 7fee4effd700 1 -- 172.21.15.32:0/3580104046 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3321762618 0 0) 0x7fee500b1920 con 0x7fee500b6690 2020-12-05T17:39:01.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.543+0000 7fee54c84700 1 --2- 172.21.15.32:0/3580104046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee500b5f00 0x7fee500b5240 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7fee480032b0 tx=0x7fee480091a0).ready entity=mon.0 client_cookie=f7b749ed65d3b762 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:01.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.543+0000 7fee4effd700 1 -- 172.21.15.32:0/3580104046 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee4800ba50 con 0x7fee500b5f00 2020-12-05T17:39:01.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.543+0000 7fee4effd700 1 -- 172.21.15.32:0/3580104046 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4800bbb0 con 0x7fee500b5f00 2020-12-05T17:39:01.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.543+0000 7fee4effd700 1 -- 172.21.15.32:0/3580104046 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee4800bd20 con 0x7fee500b5f00 2020-12-05T17:39:01.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.543+0000 7fee566e7700 1 -- 172.21.15.32:0/3580104046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee500b5f00 msgr2=0x7fee500b5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:01.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.543+0000 7fee566e7700 1 --2- 172.21.15.32:0/3580104046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee500b5f00 0x7fee500b5240 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7fee480032b0 tx=0x7fee480091a0).stop 2020-12-05T17:39:01.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee566e7700 1 -- 172.21.15.32:0/3580104046 shutdown_connections 2020-12-05T17:39:01.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee566e7700 1 --2- 172.21.15.32:0/3580104046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee500b5f00 0x7fee500b5240 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:01.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee566e7700 1 -- 172.21.15.32:0/3580104046 >> 172.21.15.32:0/3580104046 conn(0x7fee5005f3b0 msgr2=0x7fee5005f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:01.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee566e7700 1 -- 172.21.15.32:0/3580104046 shutdown_connections 2020-12-05T17:39:01.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee566e7700 1 -- 172.21.15.32:0/3580104046 wait complete. 2020-12-05T17:39:01.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee566e7700 1 Processor -- start 2020-12-05T17:39:01.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee566e7700 1 -- start start 2020-12-05T17:39:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee566e7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee500b5b20 0x7fee500a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee566e7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee500b5f00 0x7fee501405c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee566e7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fee500b6690 0x7fee50140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee566e7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee50065eb0 con 0x7fee500b5b20 2020-12-05T17:39:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee566e7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee50066b80 con 0x7fee500b5f00 2020-12-05T17:39:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee566e7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee500a3110 con 0x7fee500b6690 2020-12-05T17:39:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee4ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee500b5b20 0x7fee500a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee4ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee500b5b20 0x7fee500a3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37574/0 (socket says 172.21.15.32:37574) 2020-12-05T17:39:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee4ffff700 1 -- 172.21.15.32:0/3941689541 learned_addr learned my addr 172.21.15.32:0/3941689541 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee4f7fe700 1 --2- 172.21.15.32:0/3941689541 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee500b5f00 0x7fee501405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:01.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee54c84700 1 --2- 172.21.15.32:0/3941689541 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fee500b6690 0x7fee50140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:01.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee4ffff700 1 -- 172.21.15.32:0/3941689541 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fee500b6690 msgr2=0x7fee50140ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:01.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee4ffff700 1 --2- 172.21.15.32:0/3941689541 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fee500b6690 0x7fee50140ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:01.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee4ffff700 1 -- 172.21.15.32:0/3941689541 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee500b5f00 msgr2=0x7fee501405c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:01.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee4ffff700 1 --2- 172.21.15.32:0/3941689541 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee500b5f00 0x7fee501405c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:01.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee4ffff700 1 -- 172.21.15.32:0/3941689541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee500a33b0 con 0x7fee500b5b20 2020-12-05T17:39:01.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.547+0000 7fee4ffff700 1 --2- 172.21.15.32:0/3941689541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee500b5b20 0x7fee500a3bd0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7fee50061150 tx=0x7fee40007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:01.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.551+0000 7fee4d7fa700 1 -- 172.21.15.32:0/3941689541 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee400072a0 con 0x7fee500b5b20 2020-12-05T17:39:01.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.551+0000 7fee4d7fa700 1 -- 172.21.15.32:0/3941689541 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee400064e0 con 0x7fee500b5b20 2020-12-05T17:39:01.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.551+0000 7fee566e7700 1 -- 172.21.15.32:0/3941689541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee500668e0 con 0x7fee500b5b20 2020-12-05T17:39:01.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.551+0000 7fee4d7fa700 1 -- 172.21.15.32:0/3941689541 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee40006840 con 0x7fee500b5b20 2020-12-05T17:39:01.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.551+0000 7fee566e7700 1 -- 172.21.15.32:0/3941689541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee500a64a0 con 0x7fee500b5b20 2020-12-05T17:39:01.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.551+0000 7fee566e7700 1 -- 172.21.15.32:0/3941689541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee50057050 con 0x7fee500b5b20 2020-12-05T17:39:01.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.551+0000 7fee4d7fa700 1 -- 172.21.15.32:0/3941689541 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fee40019020 con 0x7fee500b5b20 2020-12-05T17:39:01.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.551+0000 7fee4d7fa700 1 --2- 172.21.15.32:0/3941689541 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fee30055440 0x7fee30057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:01.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.551+0000 7fee4d7fa700 1 -- 172.21.15.32:0/3941689541 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fee4006d9b0 con 0x7fee500b5b20 2020-12-05T17:39:01.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.551+0000 7fee4f7fe700 1 --2- 172.21.15.32:0/3941689541 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fee30055440 0x7fee30057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:01.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.551+0000 7fee4f7fe700 1 --2- 172.21.15.32:0/3941689541 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fee30055440 0x7fee30057900 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7fee4400b000 tx=0x7fee44009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.559+0000 7fee4d7fa700 1 -- 172.21.15.32:0/3941689541 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fee40043ab0 con 0x7fee500b5b20 2020-12-05T17:39:01.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.699+0000 7fee566e7700 1 -- 172.21.15.32:0/3941689541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fee5005bae0 con 0x7fee500b5b20 2020-12-05T17:39:01.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.699+0000 7fee4d7fa700 1 -- 172.21.15.32:0/3941689541 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+10 (secure 0 0 0) 0x7fee400470d0 con 0x7fee500b5b20 2020-12-05T17:39:01.705 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:39:01.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.699+0000 7fee566e7700 1 -- 172.21.15.32:0/3941689541 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fee30055440 msgr2=0x7fee30057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:01.707 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.699+0000 7fee566e7700 1 --2- 172.21.15.32:0/3941689541 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fee30055440 0x7fee30057900 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7fee4400b000 tx=0x7fee44009040).stop 2020-12-05T17:39:01.707 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.699+0000 7fee566e7700 1 -- 172.21.15.32:0/3941689541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee500b5b20 msgr2=0x7fee500a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:01.707 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.699+0000 7fee566e7700 1 --2- 172.21.15.32:0/3941689541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee500b5b20 0x7fee500a3bd0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7fee50061150 tx=0x7fee40007be0).stop 2020-12-05T17:39:01.707 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.699+0000 7fee566e7700 1 -- 172.21.15.32:0/3941689541 shutdown_connections 2020-12-05T17:39:01.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.699+0000 7fee566e7700 1 --2- 172.21.15.32:0/3941689541 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fee30055440 0x7fee30057900 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:01.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.699+0000 7fee566e7700 1 --2- 172.21.15.32:0/3941689541 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fee500b6690 0x7fee50140ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:01.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.699+0000 7fee566e7700 1 --2- 172.21.15.32:0/3941689541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee500b5b20 0x7fee500a3bd0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:01.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.699+0000 7fee566e7700 1 --2- 172.21.15.32:0/3941689541 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee500b5f00 0x7fee501405c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:01.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.699+0000 7fee566e7700 1 -- 172.21.15.32:0/3941689541 >> 172.21.15.32:0/3941689541 conn(0x7fee5005f3b0 msgr2=0x7fee500b9500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:01.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.699+0000 7fee566e7700 1 -- 172.21.15.32:0/3941689541 shutdown_connections 2020-12-05T17:39:01.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.699+0000 7fee566e7700 1 -- 172.21.15.32:0/3941689541 wait complete. 2020-12-05T17:39:01.718 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:39:01.719 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:39:01.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.839+0000 7f4a9255c700 1 Processor -- start 2020-12-05T17:39:01.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.839+0000 7f4a9255c700 1 -- start start 2020-12-05T17:39:01.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.839+0000 7f4a9255c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4a8c0af230 0x7f4a8c145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:01.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.839+0000 7f4a9255c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a8c146a90 con 0x7f4a8c0ab710 2020-12-05T17:39:01.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.839+0000 7f4a9255c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a8c0b23a0 con 0x7f4a8c0b1fc0 2020-12-05T17:39:01.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.839+0000 7f4a9255c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a8c061e90 con 0x7f4a8c0af230 2020-12-05T17:39:01.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.839+0000 7f4a90d59700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4a8c0ab710 0x7f4a8c147400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40230/0 (socket says 172.21.15.32:40230) 2020-12-05T17:39:01.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.839+0000 7f4a90d59700 1 -- 172.21.15.32:0/2718005278 learned_addr learned my addr 172.21.15.32:0/2718005278 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:01.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.839+0000 7f4a91d5b700 1 --2- 172.21.15.32:0/2718005278 >> v2:172.21.15.32:3300/0 conn(0x7f4a8c0af230 0x7f4a8c145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:01.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a7ffff700 1 -- 172.21.15.32:0/2718005278 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4040478907 0 0) 0x7f4a8c146a90 con 0x7f4a8c0ab710 2020-12-05T17:39:01.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a91d5b700 1 -- 172.21.15.32:0/2718005278 >> v1:172.21.15.61:6789/0 conn(0x7f4a8c0b1fc0 legacy=0x7f4a8c061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:01.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a91d5b700 1 -- 172.21.15.32:0/2718005278 >> v1:172.21.15.32:6789/0 conn(0x7f4a8c0ab710 legacy=0x7f4a8c147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:01.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a91d5b700 1 -- 172.21.15.32:0/2718005278 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c146cc0 con 0x7f4a8c0af230 2020-12-05T17:39:01.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a91d5b700 1 --2- 172.21.15.32:0/2718005278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8c0af230 0x7f4a8c145d10 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7f4a80001110 tx=0x7f4a84006dc0).ready entity=mon.0 client_cookie=c3881ab42b052d16 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:01.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a7ffff700 1 -- 172.21.15.32:0/2718005278 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a840096f0 con 0x7f4a8c0af230 2020-12-05T17:39:01.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a7ffff700 1 -- 172.21.15.32:0/2718005278 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a84009850 con 0x7f4a8c0af230 2020-12-05T17:39:01.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a7ffff700 1 -- 172.21.15.32:0/2718005278 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a84009bb0 con 0x7f4a8c0af230 2020-12-05T17:39:01.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a9255c700 1 -- 172.21.15.32:0/2718005278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8c0af230 msgr2=0x7f4a8c145d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:01.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a9255c700 1 --2- 172.21.15.32:0/2718005278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8c0af230 0x7f4a8c145d10 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7f4a80001110 tx=0x7f4a84006dc0).stop 2020-12-05T17:39:01.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a9255c700 1 -- 172.21.15.32:0/2718005278 shutdown_connections 2020-12-05T17:39:01.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a9255c700 1 --2- 172.21.15.32:0/2718005278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8c0af230 0x7f4a8c145d10 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:01.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a9255c700 1 -- 172.21.15.32:0/2718005278 >> 172.21.15.32:0/2718005278 conn(0x7f4a8c0b5530 msgr2=0x7f4a8c0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:01.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a9255c700 1 -- 172.21.15.32:0/2718005278 shutdown_connections 2020-12-05T17:39:01.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a9255c700 1 -- 172.21.15.32:0/2718005278 wait complete. 2020-12-05T17:39:01.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a9255c700 1 Processor -- start 2020-12-05T17:39:01.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a9255c700 1 -- start start 2020-12-05T17:39:01.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a9255c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a8c0ab710 0x7f4a8c143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:01.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a9255c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4a8c0af230 0x7f4a8c09e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:01.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a9255c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8c0b1fc0 0x7f4a8c158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:01.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a9255c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c09eb40 con 0x7f4a8c0b1fc0 2020-12-05T17:39:01.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a9255c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c09ede0 con 0x7f4a8c0af230 2020-12-05T17:39:01.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a9255c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c09f080 con 0x7f4a8c0ab710 2020-12-05T17:39:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.843+0000 7f4a91d5b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8c0b1fc0 0x7f4a8c158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.847+0000 7f4a90d59700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4a8c0af230 0x7f4a8c09e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.847+0000 7f4a9155a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a8c0ab710 0x7f4a8c143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.847+0000 7f4a91d5b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8c0b1fc0 0x7f4a8c158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37592/0 (socket says 172.21.15.32:37592) 2020-12-05T17:39:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.847+0000 7f4a91d5b700 1 -- 172.21.15.32:0/2743107406 learned_addr learned my addr 172.21.15.32:0/2743107406 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:01.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.847+0000 7f4a91d5b700 1 -- 172.21.15.32:0/2743107406 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a8c0ab710 msgr2=0x7f4a8c143650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:01.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.847+0000 7f4a91d5b700 1 --2- 172.21.15.32:0/2743107406 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a8c0ab710 0x7f4a8c143650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:01.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.847+0000 7f4a91d5b700 1 -- 172.21.15.32:0/2743107406 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4a8c0af230 msgr2=0x7f4a8c09e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:01.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.847+0000 7f4a91d5b700 1 --2- 172.21.15.32:0/2743107406 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4a8c0af230 0x7f4a8c09e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:01.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.847+0000 7f4a91d5b700 1 -- 172.21.15.32:0/2743107406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c155900 con 0x7f4a8c0b1fc0 2020-12-05T17:39:01.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.847+0000 7f4a91d5b700 1 --2- 172.21.15.32:0/2743107406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8c0b1fc0 0x7f4a8c158f80 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7f4a80001110 tx=0x7f4a84015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:01.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.847+0000 7f4a7effd700 1 -- 172.21.15.32:0/2743107406 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a84011980 con 0x7f4a8c0b1fc0 2020-12-05T17:39:01.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.847+0000 7f4a7effd700 1 -- 172.21.15.32:0/2743107406 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a84011ae0 con 0x7f4a8c0b1fc0 2020-12-05T17:39:01.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.847+0000 7f4a9255c700 1 -- 172.21.15.32:0/2743107406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a8c0a1c30 con 0x7f4a8c0b1fc0 2020-12-05T17:39:01.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.847+0000 7f4a7effd700 1 -- 172.21.15.32:0/2743107406 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a84011e40 con 0x7f4a8c0b1fc0 2020-12-05T17:39:01.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.847+0000 7f4a9255c700 1 -- 172.21.15.32:0/2743107406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a8c159470 con 0x7f4a8c0b1fc0 2020-12-05T17:39:01.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.847+0000 7f4a9255c700 1 -- 172.21.15.32:0/2743107406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a8c09e990 con 0x7f4a8c0b1fc0 2020-12-05T17:39:01.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.847+0000 7f4a7effd700 1 -- 172.21.15.32:0/2743107406 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4a84024030 con 0x7f4a8c0b1fc0 2020-12-05T17:39:01.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.847+0000 7f4a7effd700 1 --2- 172.21.15.32:0/2743107406 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4a680553b0 0x7f4a68057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:01.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.847+0000 7f4a9155a700 1 --2- 172.21.15.32:0/2743107406 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4a680553b0 0x7f4a68057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:01.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.847+0000 7f4a7effd700 1 -- 172.21.15.32:0/2743107406 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4a8406f280 con 0x7f4a8c0b1fc0 2020-12-05T17:39:01.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.851+0000 7f4a9155a700 1 --2- 172.21.15.32:0/2743107406 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4a680553b0 0x7f4a68057870 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f4a8000d000 tx=0x7f4a80007ac0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:01.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:01.855+0000 7f4a7effd700 1 -- 172.21.15.32:0/2743107406 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4a84045310 con 0x7f4a8c0b1fc0 2020-12-05T17:39:02.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:02.039+0000 7f4a9255c700 1 -- 172.21.15.32:0/2743107406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a8c09f240 con 0x7f4a8c0b1fc0 2020-12-05T17:39:02.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:02.039+0000 7f4a7effd700 1 -- 172.21.15.32:0/2743107406 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f4a84022020 con 0x7f4a8c0b1fc0 2020-12-05T17:39:02.047 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:02.048 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":37,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6535,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2494768218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2494768218},{"type":"v1","addr":"172.21.15.61:6833","nonce":2494768218}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":35}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:38:56.821107+0000","modified":"2020-12-05T17:38:57.824754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/81037173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":81037173},{"type":"v1","addr":"172.21.15.32:6835","nonce":81037173}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-12-05T17:39:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:02.039+0000 7f4a9255c700 1 -- 172.21.15.32:0/2743107406 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4a680553b0 msgr2=0x7f4a68057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:02.039+0000 7f4a9255c700 1 --2- 172.21.15.32:0/2743107406 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4a680553b0 0x7f4a68057870 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f4a8000d000 tx=0x7f4a80007ac0).stop 2020-12-05T17:39:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:02.039+0000 7f4a9255c700 1 -- 172.21.15.32:0/2743107406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8c0b1fc0 msgr2=0x7f4a8c158f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:02.039+0000 7f4a9255c700 1 --2- 172.21.15.32:0/2743107406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8c0b1fc0 0x7f4a8c158f80 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7f4a80001110 tx=0x7f4a84015000).stop 2020-12-05T17:39:02.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:02.043+0000 7f4a9255c700 1 -- 172.21.15.32:0/2743107406 shutdown_connections 2020-12-05T17:39:02.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:02.043+0000 7f4a9255c700 1 --2- 172.21.15.32:0/2743107406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8c0b1fc0 0x7f4a8c158f80 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:02.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:02.043+0000 7f4a9255c700 1 --2- 172.21.15.32:0/2743107406 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4a680553b0 0x7f4a68057870 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:02.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:02.043+0000 7f4a9255c700 1 --2- 172.21.15.32:0/2743107406 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a8c0ab710 0x7f4a8c143650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:02.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:02.043+0000 7f4a9255c700 1 --2- 172.21.15.32:0/2743107406 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4a8c0af230 0x7f4a8c09e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:02.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:02.043+0000 7f4a9255c700 1 -- 172.21.15.32:0/2743107406 >> 172.21.15.32:0/2743107406 conn(0x7f4a8c0b5530 msgr2=0x7f4a8c0b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:02.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:02.043+0000 7f4a9255c700 1 -- 172.21.15.32:0/2743107406 shutdown_connections 2020-12-05T17:39:02.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:02.043+0000 7f4a9255c700 1 -- 172.21.15.32:0/2743107406 wait complete. 2020-12-05T17:39:02.059 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 37 2020-12-05T17:39:02.264 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:38:56.821107+0000', 'modified': '2020-12-05T17:38:57.824754+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6609}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6609': {'gid': 6609, 'name': 'a', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/81037173', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 81037173}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 81037173}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:39:02.265 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:39:02.266 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:39:02.401 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":6609,"want_state":"up:active","state":"up:active","rank_uptime":5.5643150629999996,"mdsmap_epoch":37,"osdmap_epoch":83,"osdmap_epoch_barrier":83,"uptime":10.813820099000001} 2020-12-05T17:39:02.401 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 6609, "mdsmap_epoch": 37, "osdmap_epoch": 83, "osdmap_epoch_barrier": 83, "rank_uptime": 5.564315063, "state": "up:active", "uptime": 10.813820099, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:39:02.402 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:39:02.403 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-12-05T17:39:02.403 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-12-05T17:39:02.404 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:02.405 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:02.416 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:39:02.479 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:39:02.480 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:39:02.538 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:39:02.539 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:39:02.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:02.541 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:39:02.585 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:39:02.586 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:02.630 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:39:02.644 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:39:02.645 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:39:02.703 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:39:02.705 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:39:02.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:02.706 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:39:02.707 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:2020-12-05T17:39:02.703+0000 7f39216400c0 -1 init, newargv = 0x556665d741c0 newargc=9 2020-12-05T17:39:02.707 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[11166]: starting ceph client 2020-12-05T17:39:02.717 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[11166]: starting fuse 2020-12-05T17:39:03.813 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:39:03.830 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:39:03.831 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:39:03.886 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:39:03.887 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:39:03.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:03.888 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:39:03.934 INFO:teuthology.orchestra.run.smithi036.stdout:49 2020-12-05T17:39:03.934 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-12-05T17:39:03.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:03.935 INFO:teuthology.orchestra.run.smithi036:> sudo python3 -c ' 2020-12-05T17:39:03.936 INFO:teuthology.orchestra.run.smithi036:> import glob 2020-12-05T17:39:03.936 INFO:teuthology.orchestra.run.smithi036:> import re 2020-12-05T17:39:03.936 INFO:teuthology.orchestra.run.smithi036:> import os 2020-12-05T17:39:03.937 INFO:teuthology.orchestra.run.smithi036:> import subprocess 2020-12-05T17:39:03.937 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:39:03.938 INFO:teuthology.orchestra.run.smithi036:> def find_socket(client_name): 2020-12-05T17:39:03.938 INFO:teuthology.orchestra.run.smithi036:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-12-05T17:39:03.939 INFO:teuthology.orchestra.run.smithi036:> files = glob.glob(asok_path) 2020-12-05T17:39:03.939 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:39:03.939 INFO:teuthology.orchestra.run.smithi036:> # Given a non-glob path, it better be there 2020-12-05T17:39:03.940 INFO:teuthology.orchestra.run.smithi036:> if "*" not in asok_path: 2020-12-05T17:39:03.940 INFO:teuthology.orchestra.run.smithi036:> assert(len(files) == 1) 2020-12-05T17:39:03.941 INFO:teuthology.orchestra.run.smithi036:> return files[0] 2020-12-05T17:39:03.941 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:39:03.941 INFO:teuthology.orchestra.run.smithi036:> for f in files: 2020-12-05T17:39:03.942 INFO:teuthology.orchestra.run.smithi036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:39:03.943 INFO:teuthology.orchestra.run.smithi036:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:39:03.943 INFO:teuthology.orchestra.run.smithi036:> return f 2020-12-05T17:39:03.943 INFO:teuthology.orchestra.run.smithi036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:39:03.944 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:39:03.944 INFO:teuthology.orchestra.run.smithi036:> print(find_socket("client.0")) 2020-12-05T17:39:03.944 INFO:teuthology.orchestra.run.smithi036:> ' 2020-12-05T17:39:04.027 INFO:teuthology.orchestra.run.smithi036.stdout:/var/run/ceph/ceph-client.0.11166.asok 2020-12-05T17:39:04.032 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.11166.asok 2020-12-05T17:39:04.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:04.033 INFO:teuthology.orchestra.run.smithi036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.11166.asok status 2020-12-05T17:39:04.134 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2020-12-05T17:39:04.134 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": { 2020-12-05T17:39:04.134 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:39:04.135 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:39:04.135 INFO:teuthology.orchestra.run.smithi036.stdout: "entity_id": "0", 2020-12-05T17:39:04.135 INFO:teuthology.orchestra.run.smithi036.stdout: "hostname": "smithi036", 2020-12-05T17:39:04.135 INFO:teuthology.orchestra.run.smithi036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-12-05T17:39:04.135 INFO:teuthology.orchestra.run.smithi036.stdout: "pid": "11166", 2020-12-05T17:39:04.136 INFO:teuthology.orchestra.run.smithi036.stdout: "root": "/" 2020-12-05T17:39:04.136 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:39:04.136 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_count": 0, 2020-12-05T17:39:04.136 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_pinned_count": 0, 2020-12-05T17:39:04.137 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 6786, 2020-12-05T17:39:04.137 INFO:teuthology.orchestra.run.smithi036.stdout: "inst": { 2020-12-05T17:39:04.137 INFO:teuthology.orchestra.run.smithi036.stdout: "name": { 2020-12-05T17:39:04.138 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "client", 2020-12-05T17:39:04.138 INFO:teuthology.orchestra.run.smithi036.stdout: "num": 6786 2020-12-05T17:39:04.138 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:39:04.138 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:39:04.138 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:39:04.139 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:39:04.139 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 550017400 2020-12-05T17:39:04.139 INFO:teuthology.orchestra.run.smithi036.stdout: } 2020-12-05T17:39:04.139 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:39:04.140 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:39:04.140 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:39:04.140 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:39:04.140 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 550017400 2020-12-05T17:39:04.140 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:39:04.141 INFO:teuthology.orchestra.run.smithi036.stdout: "inst_str": "client.6786 172.21.15.36:0/550017400", 2020-12-05T17:39:04.141 INFO:teuthology.orchestra.run.smithi036.stdout: "addr_str": "172.21.15.36:0/550017400", 2020-12-05T17:39:04.141 INFO:teuthology.orchestra.run.smithi036.stdout: "inode_count": 1, 2020-12-05T17:39:04.141 INFO:teuthology.orchestra.run.smithi036.stdout: "mds_epoch": 37, 2020-12-05T17:39:04.141 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch": 83, 2020-12-05T17:39:04.142 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:39:04.142 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklisted": false 2020-12-05T17:39:04.142 INFO:teuthology.orchestra.run.smithi036.stdout:} 2020-12-05T17:39:04.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:04.144 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:04.144 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:04.151 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:39:04.152 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:39:04.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:04.153 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:04.213 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:39:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.331+0000 7f4c62981700 1 Processor -- start 2020-12-05T17:39:04.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.331+0000 7f4c62981700 1 -- start start 2020-12-05T17:39:04.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.331+0000 7f4c62981700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4c5c062400 0x7f4c5c05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:04.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.331+0000 7f4c62981700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4c5c05feb0 0x7f4c5c0b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:04.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.331+0000 7f4c62981700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c5c142450 con 0x7f4c5c05f3b0 2020-12-05T17:39:04.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.331+0000 7f4c62981700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c5c0b42f0 con 0x7f4c5c062400 2020-12-05T17:39:04.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.331+0000 7f4c62981700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c061780 con 0x7f4c5c05feb0 2020-12-05T17:39:04.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.331+0000 7f4c6117e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4c5c062400 0x7f4c5c05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:04.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.331+0000 7f4c6117e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4c5c062400 0x7f4c5c05f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37598/0 (socket says 172.21.15.32:37598) 2020-12-05T17:39:04.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.331+0000 7f4c6117e700 1 -- 172.21.15.32:0/2426212247 learned_addr learned my addr 172.21.15.32:0/2426212247 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:04.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.331+0000 7f4c62180700 1 --2- 172.21.15.32:0/2426212247 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4c5c05feb0 0x7f4c5c0b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.335+0000 7f4c6117e700 1 -- 172.21.15.32:0/2426212247 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4c5c05feb0 msgr2=0x7f4c5c0b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.335+0000 7f4c6117e700 1 --2- 172.21.15.32:0/2426212247 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4c5c05feb0 0x7f4c5c0b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.335+0000 7f4c6117e700 1 -- 172.21.15.32:0/2426212247 >> v1:172.21.15.61:6789/0 conn(0x7f4c5c05f3b0 legacy=0x7f4c5c0b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.335+0000 7f4c6117e700 1 -- 172.21.15.32:0/2426212247 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5c061a20 con 0x7f4c5c062400 2020-12-05T17:39:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.335+0000 7f4c6117e700 1 --2- 172.21.15.32:0/2426212247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c5c062400 0x7f4c5c05f790 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7f4c58002f30 tx=0x7f4c58006e80).ready entity=mon.0 client_cookie=b829ce16a08e21bf server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:04.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.335+0000 7f4c6097d700 1 -- 172.21.15.32:0/2426212247 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c580097f0 con 0x7f4c5c062400 2020-12-05T17:39:04.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.335+0000 7f4c6097d700 1 -- 172.21.15.32:0/2426212247 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c58009950 con 0x7f4c5c062400 2020-12-05T17:39:04.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.335+0000 7f4c6097d700 1 -- 172.21.15.32:0/2426212247 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c58009cb0 con 0x7f4c5c062400 2020-12-05T17:39:04.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.335+0000 7f4c62981700 1 -- 172.21.15.32:0/2426212247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c5c062400 msgr2=0x7f4c5c05f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:04.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.335+0000 7f4c62981700 1 --2- 172.21.15.32:0/2426212247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c5c062400 0x7f4c5c05f790 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7f4c58002f30 tx=0x7f4c58006e80).stop 2020-12-05T17:39:04.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.335+0000 7f4c62981700 1 -- 172.21.15.32:0/2426212247 shutdown_connections 2020-12-05T17:39:04.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.335+0000 7f4c62981700 1 --2- 172.21.15.32:0/2426212247 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4c5c05feb0 0x7f4c5c0b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.335+0000 7f4c62981700 1 --2- 172.21.15.32:0/2426212247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c5c062400 0x7f4c5c05f790 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.335+0000 7f4c62981700 1 -- 172.21.15.32:0/2426212247 >> 172.21.15.32:0/2426212247 conn(0x7f4c5c0b8ee0 msgr2=0x7f4c5c0b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:04.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.335+0000 7f4c62981700 1 -- 172.21.15.32:0/2426212247 shutdown_connections 2020-12-05T17:39:04.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.335+0000 7f4c62981700 1 -- 172.21.15.32:0/2426212247 wait complete. 2020-12-05T17:39:04.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c62981700 1 Processor -- start 2020-12-05T17:39:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c62981700 1 -- start start 2020-12-05T17:39:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c62981700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c5c05f3b0 0x7f4c5c148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c62981700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4c5c05feb0 0x7f4c5c1451e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c62981700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4c5c062400 0x7f4c5c064d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:04.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c62981700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c145c60 con 0x7f4c5c05f3b0 2020-12-05T17:39:04.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c62981700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c145f00 con 0x7f4c5c05feb0 2020-12-05T17:39:04.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c62981700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c1461a0 con 0x7f4c5c062400 2020-12-05T17:39:04.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c6197f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c5c05f3b0 0x7f4c5c148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:04.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c6197f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c5c05f3b0 0x7f4c5c148780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37602/0 (socket says 172.21.15.32:37602) 2020-12-05T17:39:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c6197f700 1 -- 172.21.15.32:0/2783808812 learned_addr learned my addr 172.21.15.32:0/2783808812 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c6117e700 1 --2- 172.21.15.32:0/2783808812 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4c5c05feb0 0x7f4c5c1451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c62180700 1 --2- 172.21.15.32:0/2783808812 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4c5c062400 0x7f4c5c064d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c6197f700 1 -- 172.21.15.32:0/2783808812 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4c5c062400 msgr2=0x7f4c5c064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:04.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c6197f700 1 --2- 172.21.15.32:0/2783808812 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4c5c062400 0x7f4c5c064d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c6197f700 1 -- 172.21.15.32:0/2783808812 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4c5c05feb0 msgr2=0x7f4c5c1451e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:04.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c6197f700 1 --2- 172.21.15.32:0/2783808812 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4c5c05feb0 0x7f4c5c1451e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c6197f700 1 -- 172.21.15.32:0/2783808812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5c146640 con 0x7f4c5c05f3b0 2020-12-05T17:39:04.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c6197f700 1 --2- 172.21.15.32:0/2783808812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c5c05f3b0 0x7f4c5c148780 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f4c5c05c5c0 tx=0x7f4c50007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:04.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c4effd700 1 -- 172.21.15.32:0/2783808812 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c50007240 con 0x7f4c5c05f3b0 2020-12-05T17:39:04.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c4effd700 1 -- 172.21.15.32:0/2783808812 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c500073a0 con 0x7f4c5c05f3b0 2020-12-05T17:39:04.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c62981700 1 -- 172.21.15.32:0/2783808812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c5c1468f0 con 0x7f4c5c05f3b0 2020-12-05T17:39:04.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c4effd700 1 -- 172.21.15.32:0/2783808812 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c500066d0 con 0x7f4c5c05f3b0 2020-12-05T17:39:04.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.339+0000 7f4c62981700 1 -- 172.21.15.32:0/2783808812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c5c0afad0 con 0x7f4c5c05f3b0 2020-12-05T17:39:04.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.343+0000 7f4c62981700 1 -- 172.21.15.32:0/2783808812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c5c146360 con 0x7f4c5c05f3b0 2020-12-05T17:39:04.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.343+0000 7f4c4effd700 1 -- 172.21.15.32:0/2783808812 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4c50010070 con 0x7f4c5c05f3b0 2020-12-05T17:39:04.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.343+0000 7f4c4effd700 1 --2- 172.21.15.32:0/2783808812 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4c3c0553b0 0x7f4c3c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:04.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.343+0000 7f4c6117e700 1 --2- 172.21.15.32:0/2783808812 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4c3c0553b0 0x7f4c3c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:04.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.343+0000 7f4c4effd700 1 -- 172.21.15.32:0/2783808812 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4c5006d8c0 con 0x7f4c5c05f3b0 2020-12-05T17:39:04.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.343+0000 7f4c6117e700 1 --2- 172.21.15.32:0/2783808812 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4c3c0553b0 0x7f4c3c057870 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f4c5801c000 tx=0x7f4c58015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.351+0000 7f4c4effd700 1 -- 172.21.15.32:0/2783808812 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4c500438d0 con 0x7f4c5c05f3b0 2020-12-05T17:39:04.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.495+0000 7f4c62981700 1 -- 172.21.15.32:0/2783808812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c5c057050 con 0x7f4c5c05f3b0 2020-12-05T17:39:04.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.499+0000 7f4c4effd700 1 -- 172.21.15.32:0/2783808812 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f4c50018030 con 0x7f4c5c05f3b0 2020-12-05T17:39:04.505 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:04.505 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":83,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:38:57.823795+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:38:54.508932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-12-05T17:38:55.805478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:39:04.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.499+0000 7f4c62981700 1 -- 172.21.15.32:0/2783808812 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4c3c0553b0 msgr2=0x7f4c3c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:04.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.499+0000 7f4c62981700 1 --2- 172.21.15.32:0/2783808812 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4c3c0553b0 0x7f4c3c057870 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f4c5801c000 tx=0x7f4c58015000).stop 2020-12-05T17:39:04.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.499+0000 7f4c62981700 1 -- 172.21.15.32:0/2783808812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c5c05f3b0 msgr2=0x7f4c5c148780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:04.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.499+0000 7f4c62981700 1 --2- 172.21.15.32:0/2783808812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c5c05f3b0 0x7f4c5c148780 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f4c5c05c5c0 tx=0x7f4c50007be0).stop 2020-12-05T17:39:04.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.499+0000 7f4c62981700 1 -- 172.21.15.32:0/2783808812 shutdown_connections 2020-12-05T17:39:04.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.499+0000 7f4c62981700 1 --2- 172.21.15.32:0/2783808812 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4c3c0553b0 0x7f4c3c057870 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.499+0000 7f4c62981700 1 --2- 172.21.15.32:0/2783808812 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4c5c062400 0x7f4c5c064d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.499+0000 7f4c62981700 1 --2- 172.21.15.32:0/2783808812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c5c05f3b0 0x7f4c5c148780 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.499+0000 7f4c62981700 1 --2- 172.21.15.32:0/2783808812 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4c5c05feb0 0x7f4c5c1451e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.499+0000 7f4c62981700 1 -- 172.21.15.32:0/2783808812 >> 172.21.15.32:0/2783808812 conn(0x7f4c5c0b8ee0 msgr2=0x7f4c5c140760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:04.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.499+0000 7f4c62981700 1 -- 172.21.15.32:0/2783808812 shutdown_connections 2020-12-05T17:39:04.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.499+0000 7f4c62981700 1 -- 172.21.15.32:0/2783808812 wait complete. 2020-12-05T17:39:04.519 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:38:54.508932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:38:55.805478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:39:04.520 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:39:04.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.643+0000 7fead39fa700 1 Processor -- start 2020-12-05T17:39:04.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.643+0000 7fead39fa700 1 -- start start 2020-12-05T17:39:04.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.643+0000 7fead39fa700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feacc0b29d0 0x7feacc05f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:04.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.643+0000 7fead39fa700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feacc05ffb0 0x7feacc145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:04.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.643+0000 7fead39fa700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feacc0b2db0 con 0x7feacc0acd10 2020-12-05T17:39:04.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.643+0000 7fead39fa700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feacc0b0390 con 0x7feacc0b29d0 2020-12-05T17:39:04.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.643+0000 7fead39fa700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feacc0b1260 con 0x7feacc05ffb0 2020-12-05T17:39:04.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.643+0000 7fead1796700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feacc0b29d0 0x7feacc05f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:04.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.643+0000 7fead1796700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feacc0b29d0 0x7feacc05f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37610/0 (socket says 172.21.15.32:37610) 2020-12-05T17:39:04.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.643+0000 7fead1796700 1 -- 172.21.15.32:0/36922683 learned_addr learned my addr 172.21.15.32:0/36922683 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:04.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.643+0000 7fead1f97700 1 --2- 172.21.15.32:0/36922683 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feacc05ffb0 0x7feacc145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:04.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.643+0000 7fead1796700 1 -- 172.21.15.32:0/36922683 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feacc05ffb0 msgr2=0x7feacc145ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:04.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.643+0000 7fead1796700 1 --2- 172.21.15.32:0/36922683 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feacc05ffb0 0x7feacc145ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.643+0000 7fead1796700 1 -- 172.21.15.32:0/36922683 >> v1:172.21.15.32:6789/0 conn(0x7feacc0acd10 legacy=0x7feacc05f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:04.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.643+0000 7fead1796700 1 -- 172.21.15.32:0/36922683 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feacc0b1500 con 0x7feacc0b29d0 2020-12-05T17:39:04.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.643+0000 7fead1796700 1 --2- 172.21.15.32:0/36922683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feacc0b29d0 0x7feacc05f3b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0x7feac8002f30 tx=0x7feac8006e80).ready entity=mon.0 client_cookie=cae5b97eb456baad server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:04.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.643+0000 7feabffff700 1 -- 172.21.15.32:0/36922683 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feac80097f0 con 0x7feacc0b29d0 2020-12-05T17:39:04.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.643+0000 7feabffff700 1 -- 172.21.15.32:0/36922683 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac8009950 con 0x7feacc0b29d0 2020-12-05T17:39:04.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.643+0000 7feabffff700 1 -- 172.21.15.32:0/36922683 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feac8009cb0 con 0x7feacc0b29d0 2020-12-05T17:39:04.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.647+0000 7fead39fa700 1 -- 172.21.15.32:0/36922683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feacc0b29d0 msgr2=0x7feacc05f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:04.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.647+0000 7fead39fa700 1 --2- 172.21.15.32:0/36922683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feacc0b29d0 0x7feacc05f3b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0x7feac8002f30 tx=0x7feac8006e80).stop 2020-12-05T17:39:04.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.647+0000 7fead39fa700 1 -- 172.21.15.32:0/36922683 shutdown_connections 2020-12-05T17:39:04.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.647+0000 7fead39fa700 1 --2- 172.21.15.32:0/36922683 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feacc05ffb0 0x7feacc145ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.647+0000 7fead39fa700 1 --2- 172.21.15.32:0/36922683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feacc0b29d0 0x7feacc05f3b0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.647+0000 7fead39fa700 1 -- 172.21.15.32:0/36922683 >> 172.21.15.32:0/36922683 conn(0x7feacc062310 msgr2=0x7feacc0626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:04.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.647+0000 7fead39fa700 1 -- 172.21.15.32:0/36922683 shutdown_connections 2020-12-05T17:39:04.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.647+0000 7fead39fa700 1 -- 172.21.15.32:0/36922683 wait complete. 2020-12-05T17:39:04.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.647+0000 7fead39fa700 1 Processor -- start 2020-12-05T17:39:04.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.647+0000 7fead39fa700 1 -- start start 2020-12-05T17:39:04.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.651+0000 7fead39fa700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feacc05ffb0 0x7feacc142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:04.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.651+0000 7fead39fa700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feacc0b29d0 0x7feacc147590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:04.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.651+0000 7fead39fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feacc145110 0x7feacc139c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:04.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.651+0000 7fead39fa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc146bb0 con 0x7feacc145110 2020-12-05T17:39:04.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.651+0000 7fead39fa700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc146e50 con 0x7feacc0b29d0 2020-12-05T17:39:04.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.651+0000 7fead39fa700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feacc1470f0 con 0x7feacc05ffb0 2020-12-05T17:39:04.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.651+0000 7fead1f97700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feacc145110 0x7feacc139c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:04.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.651+0000 7fead1f97700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feacc145110 0x7feacc139c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37620/0 (socket says 172.21.15.32:37620) 2020-12-05T17:39:04.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.651+0000 7fead1f97700 1 -- 172.21.15.32:0/2161428263 learned_addr learned my addr 172.21.15.32:0/2161428263 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:04.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.651+0000 7fead1796700 1 --2- 172.21.15.32:0/2161428263 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feacc05ffb0 0x7feacc142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:04.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.651+0000 7fead0f95700 1 --2- 172.21.15.32:0/2161428263 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feacc0b29d0 0x7feacc147590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:04.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.651+0000 7fead1f97700 1 -- 172.21.15.32:0/2161428263 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feacc05ffb0 msgr2=0x7feacc142050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:04.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.651+0000 7fead1f97700 1 --2- 172.21.15.32:0/2161428263 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feacc05ffb0 0x7feacc142050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.651+0000 7fead1f97700 1 -- 172.21.15.32:0/2161428263 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feacc0b29d0 msgr2=0x7feacc147590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:04.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.651+0000 7fead1f97700 1 --2- 172.21.15.32:0/2161428263 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feacc0b29d0 0x7feacc147590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.651+0000 7fead1f97700 1 -- 172.21.15.32:0/2161428263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feacc143d80 con 0x7feacc145110 2020-12-05T17:39:04.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.651+0000 7fead1f97700 1 --2- 172.21.15.32:0/2161428263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feacc145110 0x7feacc139c60 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7feac400a000 tx=0x7feac4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:04.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.651+0000 7feabe7fc700 1 -- 172.21.15.32:0/2161428263 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feac400b980 con 0x7feacc145110 2020-12-05T17:39:04.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.651+0000 7feabe7fc700 1 -- 172.21.15.32:0/2161428263 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac400bae0 con 0x7feacc145110 2020-12-05T17:39:04.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.651+0000 7fead39fa700 1 -- 172.21.15.32:0/2161428263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feacc144d40 con 0x7feacc145110 2020-12-05T17:39:04.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.651+0000 7feabe7fc700 1 -- 172.21.15.32:0/2161428263 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feac400be40 con 0x7feacc145110 2020-12-05T17:39:04.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.651+0000 7fead39fa700 1 -- 172.21.15.32:0/2161428263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feacc13a370 con 0x7feacc145110 2020-12-05T17:39:04.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.655+0000 7feabe7fc700 1 -- 172.21.15.32:0/2161428263 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7feac401c460 con 0x7feacc145110 2020-12-05T17:39:04.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.655+0000 7fead39fa700 1 -- 172.21.15.32:0/2161428263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feacc143f10 con 0x7feacc145110 2020-12-05T17:39:04.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.655+0000 7feabe7fc700 1 --2- 172.21.15.32:0/2161428263 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feab4055440 0x7feab4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:04.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.655+0000 7fead1796700 1 --2- 172.21.15.32:0/2161428263 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feab4055440 0x7feab4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:04.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.655+0000 7feabe7fc700 1 -- 172.21.15.32:0/2161428263 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7feac406e7f0 con 0x7feacc145110 2020-12-05T17:39:04.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.655+0000 7fead1796700 1 --2- 172.21.15.32:0/2161428263 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feab4055440 0x7feab4057900 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7feac801c000 tx=0x7feac8015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:04.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.663+0000 7feabe7fc700 1 -- 172.21.15.32:0/2161428263 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7feac4044820 con 0x7feacc145110 2020-12-05T17:39:04.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.803+0000 7fead39fa700 1 -- 172.21.15.32:0/2161428263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7feacc1472b0 con 0x7feacc145110 2020-12-05T17:39:04.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.803+0000 7feabe7fc700 1 -- 172.21.15.32:0/2161428263 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v83) v1 ==== 110+0+10 (secure 0 0 0) 0x7feac4047e40 con 0x7feacc145110 2020-12-05T17:39:04.809 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:39:04.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.803+0000 7fead39fa700 1 -- 172.21.15.32:0/2161428263 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feab4055440 msgr2=0x7feab4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:04.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.803+0000 7fead39fa700 1 --2- 172.21.15.32:0/2161428263 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feab4055440 0x7feab4057900 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7feac801c000 tx=0x7feac8015000).stop 2020-12-05T17:39:04.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.803+0000 7fead39fa700 1 -- 172.21.15.32:0/2161428263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feacc145110 msgr2=0x7feacc139c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:04.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.803+0000 7fead39fa700 1 --2- 172.21.15.32:0/2161428263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feacc145110 0x7feacc139c60 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7feac400a000 tx=0x7feac4008040).stop 2020-12-05T17:39:04.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.807+0000 7fead39fa700 1 -- 172.21.15.32:0/2161428263 shutdown_connections 2020-12-05T17:39:04.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.807+0000 7fead39fa700 1 --2- 172.21.15.32:0/2161428263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feacc145110 0x7feacc139c60 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.807+0000 7fead39fa700 1 --2- 172.21.15.32:0/2161428263 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feab4055440 0x7feab4057900 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.807+0000 7fead39fa700 1 --2- 172.21.15.32:0/2161428263 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feacc05ffb0 0x7feacc142050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.807+0000 7fead39fa700 1 --2- 172.21.15.32:0/2161428263 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feacc0b29d0 0x7feacc147590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.807+0000 7fead39fa700 1 -- 172.21.15.32:0/2161428263 >> 172.21.15.32:0/2161428263 conn(0x7feacc062310 msgr2=0x7feacc1429a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:04.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.807+0000 7fead39fa700 1 -- 172.21.15.32:0/2161428263 shutdown_connections 2020-12-05T17:39:04.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.807+0000 7fead39fa700 1 -- 172.21.15.32:0/2161428263 wait complete. 2020-12-05T17:39:04.829 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:39:04.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.943+0000 7f4a91c1b700 1 Processor -- start 2020-12-05T17:39:04.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.943+0000 7f4a91c1b700 1 -- start start 2020-12-05T17:39:04.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.943+0000 7f4a91c1b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f4a8c0b5e20 0x7f4a8c0ae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:04.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.943+0000 7f4a91c1b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4a8c0b66f0 0x7f4a8c0619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:04.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.943+0000 7f4a91c1b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a8c061ee0 0x7f4a8c062330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:04.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.943+0000 7f4a91c1b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a8c0ae740 con 0x7f4a8c0b66f0 2020-12-05T17:39:04.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.943+0000 7f4a91c1b700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a8c0af260 con 0x7f4a8c0b5e20 2020-12-05T17:39:04.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.943+0000 7f4a91c1b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c062820 con 0x7f4a8c061ee0 2020-12-05T17:39:04.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.943+0000 7f4a8affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4a8c0b66f0 0x7f4a8c0619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:04.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.943+0000 7f4a8affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4a8c0b66f0 0x7f4a8c0619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37626/0 (socket says 172.21.15.32:37626) 2020-12-05T17:39:04.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.943+0000 7f4a8affd700 1 -- 172.21.15.32:0/1988374769 learned_addr learned my addr 172.21.15.32:0/1988374769 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:04.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.943+0000 7f4a8b7fe700 1 --2- 172.21.15.32:0/1988374769 >> v2:172.21.15.61:3300/0 conn(0x7f4a8c0b5e20 0x7f4a8c0ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:04.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.943+0000 7f4a8bfff700 1 --2- 172.21.15.32:0/1988374769 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a8c061ee0 0x7f4a8c062330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:04.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.943+0000 7f4a8b7fe700 1 -- 172.21.15.32:0/1988374769 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a8c061ee0 msgr2=0x7f4a8c062330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:04.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.943+0000 7f4a8b7fe700 1 --2- 172.21.15.32:0/1988374769 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a8c061ee0 0x7f4a8c062330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.943+0000 7f4a8b7fe700 1 -- 172.21.15.32:0/1988374769 >> v2:172.21.15.32:3300/0 conn(0x7f4a8c0b66f0 msgr2=0x7f4a8c0619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:04.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.943+0000 7f4a8b7fe700 1 --2- 172.21.15.32:0/1988374769 >> v2:172.21.15.32:3300/0 conn(0x7f4a8c0b66f0 0x7f4a8c0619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.943+0000 7f4a8b7fe700 1 -- 172.21.15.32:0/1988374769 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c0b6200 con 0x7f4a8c0b5e20 2020-12-05T17:39:04.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.943+0000 7f4a8bfff700 1 --2- 172.21.15.32:0/1988374769 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a8c061ee0 0x7f4a8c062330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:04.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.943+0000 7f4a8b7fe700 1 --2- 172.21.15.32:0/1988374769 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4a8c0b5e20 0x7f4a8c0ae250 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f4a7c002f30 tx=0x7f4a7c006dc0).ready entity=mon.1 client_cookie=49c6d335918ec03d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:04.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.947+0000 7f4a8a7fc700 1 -- 172.21.15.32:0/1988374769 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a7c009650 con 0x7f4a8c0b5e20 2020-12-05T17:39:04.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.947+0000 7f4a8a7fc700 1 -- 172.21.15.32:0/1988374769 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7c0097b0 con 0x7f4a8c0b5e20 2020-12-05T17:39:04.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.947+0000 7f4a8a7fc700 1 -- 172.21.15.32:0/1988374769 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a7c009b10 con 0x7f4a8c0b5e20 2020-12-05T17:39:04.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.947+0000 7f4a91c1b700 1 -- 172.21.15.32:0/1988374769 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4a8c0b5e20 msgr2=0x7f4a8c0ae250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:04.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.947+0000 7f4a91c1b700 1 --2- 172.21.15.32:0/1988374769 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4a8c0b5e20 0x7f4a8c0ae250 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f4a7c002f30 tx=0x7f4a7c006dc0).stop 2020-12-05T17:39:04.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.947+0000 7f4a91c1b700 1 -- 172.21.15.32:0/1988374769 shutdown_connections 2020-12-05T17:39:04.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.947+0000 7f4a91c1b700 1 --2- 172.21.15.32:0/1988374769 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4a8c0b5e20 0x7f4a8c0ae250 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.947+0000 7f4a91c1b700 1 --2- 172.21.15.32:0/1988374769 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a8c061ee0 0x7f4a8c062330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.947+0000 7f4a91c1b700 1 --2- 172.21.15.32:0/1988374769 >> v2:172.21.15.32:3300/0 conn(0x7f4a8c0b66f0 0x7f4a8c0619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.947+0000 7f4a91c1b700 1 -- 172.21.15.32:0/1988374769 >> 172.21.15.32:0/1988374769 conn(0x7f4a8c05f3b0 msgr2=0x7f4a8c05f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:04.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.947+0000 7f4a91c1b700 1 -- 172.21.15.32:0/1988374769 shutdown_connections 2020-12-05T17:39:04.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.947+0000 7f4a91c1b700 1 -- 172.21.15.32:0/1988374769 wait complete. 2020-12-05T17:39:04.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.947+0000 7f4a91c1b700 1 Processor -- start 2020-12-05T17:39:04.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.947+0000 7f4a91c1b700 1 -- start start 2020-12-05T17:39:04.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.947+0000 7f4a91c1b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a8c061ee0 0x7f4a8c0b7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:04.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.947+0000 7f4a91c1b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4a8c0b5e20 0x7f4a8c13bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:04.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.947+0000 7f4a91c1b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8c0b66f0 0x7f4a8c13a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a91c1b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c151870 con 0x7f4a8c0b66f0 2020-12-05T17:39:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a91c1b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c1558a0 con 0x7f4a8c0b5e20 2020-12-05T17:39:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a91c1b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c155b40 con 0x7f4a8c061ee0 2020-12-05T17:39:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a8bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8c0b66f0 0x7f4a8c13a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a8bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8c0b66f0 0x7f4a8c13a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37634/0 (socket says 172.21.15.32:37634) 2020-12-05T17:39:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a8bfff700 1 -- 172.21.15.32:0/3667695484 learned_addr learned my addr 172.21.15.32:0/3667695484 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a8b7fe700 1 --2- 172.21.15.32:0/3667695484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a8c061ee0 0x7f4a8c0b7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a8affd700 1 --2- 172.21.15.32:0/3667695484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4a8c0b5e20 0x7f4a8c13bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a8bfff700 1 -- 172.21.15.32:0/3667695484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a8c061ee0 msgr2=0x7f4a8c0b7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a8bfff700 1 --2- 172.21.15.32:0/3667695484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a8c061ee0 0x7f4a8c0b7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a8bfff700 1 -- 172.21.15.32:0/3667695484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4a8c0b5e20 msgr2=0x7f4a8c13bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:04.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a8bfff700 1 --2- 172.21.15.32:0/3667695484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4a8c0b5e20 0x7f4a8c13bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:04.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a8bfff700 1 -- 172.21.15.32:0/3667695484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c146310 con 0x7f4a8c0b66f0 2020-12-05T17:39:04.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a8bfff700 1 --2- 172.21.15.32:0/3667695484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8c0b66f0 0x7f4a8c13a490 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7f4a8400c000 tx=0x7f4a840091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:04.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a88ff9700 1 -- 172.21.15.32:0/3667695484 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a8400cbd0 con 0x7f4a8c0b66f0 2020-12-05T17:39:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a88ff9700 1 -- 172.21.15.32:0/3667695484 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8400cd30 con 0x7f4a8c0b66f0 2020-12-05T17:39:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a91c1b700 1 -- 172.21.15.32:0/3667695484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a8c0b6ee0 con 0x7f4a8c0b66f0 2020-12-05T17:39:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a88ff9700 1 -- 172.21.15.32:0/3667695484 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a8400cea0 con 0x7f4a8c0b66f0 2020-12-05T17:39:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a91c1b700 1 -- 172.21.15.32:0/3667695484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a8c13a980 con 0x7f4a8c0b66f0 2020-12-05T17:39:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a91c1b700 1 -- 172.21.15.32:0/3667695484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a8c155d00 con 0x7f4a8c0b66f0 2020-12-05T17:39:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a88ff9700 1 -- 172.21.15.32:0/3667695484 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4a84022070 con 0x7f4a8c0b66f0 2020-12-05T17:39:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a88ff9700 1 --2- 172.21.15.32:0/3667695484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4a6c0553b0 0x7f4a6c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a8b7fe700 1 --2- 172.21.15.32:0/3667695484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4a6c0553b0 0x7f4a6c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:04.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.951+0000 7f4a88ff9700 1 -- 172.21.15.32:0/3667695484 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4a8406e8c0 con 0x7f4a8c0b66f0 2020-12-05T17:39:04.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.955+0000 7f4a8b7fe700 1 --2- 172.21.15.32:0/3667695484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4a6c0553b0 0x7f4a6c057870 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f4a7c01b000 tx=0x7f4a7c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:04.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:04.959+0000 7f4a88ff9700 1 -- 172.21.15.32:0/3667695484 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4a84044a60 con 0x7f4a8c0b66f0 2020-12-05T17:39:05.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.099+0000 7f4a91c1b700 1 -- 172.21.15.32:0/3667695484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4a8c057050 con 0x7f4a8c0b66f0 2020-12-05T17:39:05.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.103+0000 7f4a88ff9700 1 -- 172.21.15.32:0/3667695484 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v83) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4a84016020 con 0x7f4a8c0b66f0 2020-12-05T17:39:05.108 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:39:05.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.103+0000 7f4a91c1b700 1 -- 172.21.15.32:0/3667695484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4a6c0553b0 msgr2=0x7f4a6c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:05.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.103+0000 7f4a91c1b700 1 --2- 172.21.15.32:0/3667695484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4a6c0553b0 0x7f4a6c057870 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f4a7c01b000 tx=0x7f4a7c015000).stop 2020-12-05T17:39:05.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.103+0000 7f4a91c1b700 1 -- 172.21.15.32:0/3667695484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8c0b66f0 msgr2=0x7f4a8c13a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:05.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.103+0000 7f4a91c1b700 1 --2- 172.21.15.32:0/3667695484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8c0b66f0 0x7f4a8c13a490 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7f4a8400c000 tx=0x7f4a840091a0).stop 2020-12-05T17:39:05.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.103+0000 7f4a91c1b700 1 -- 172.21.15.32:0/3667695484 shutdown_connections 2020-12-05T17:39:05.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.103+0000 7f4a91c1b700 1 --2- 172.21.15.32:0/3667695484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8c0b66f0 0x7f4a8c13a490 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.103+0000 7f4a91c1b700 1 --2- 172.21.15.32:0/3667695484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4a6c0553b0 0x7f4a6c057870 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.103+0000 7f4a91c1b700 1 --2- 172.21.15.32:0/3667695484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a8c061ee0 0x7f4a8c0b7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.103+0000 7f4a91c1b700 1 --2- 172.21.15.32:0/3667695484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4a8c0b5e20 0x7f4a8c13bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.103+0000 7f4a91c1b700 1 -- 172.21.15.32:0/3667695484 >> 172.21.15.32:0/3667695484 conn(0x7f4a8c05f3b0 msgr2=0x7f4a8c05f790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:05.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.103+0000 7f4a91c1b700 1 -- 172.21.15.32:0/3667695484 shutdown_connections 2020-12-05T17:39:05.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.103+0000 7f4a91c1b700 1 -- 172.21.15.32:0/3667695484 wait complete. 2020-12-05T17:39:05.121 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:39:05.244 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.239+0000 7f7cc5f1f700 1 Processor -- start 2020-12-05T17:39:05.244 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.239+0000 7f7cc5f1f700 1 -- start start 2020-12-05T17:39:05.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.239+0000 7f7cc5f1f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f7cc00b5b20 0x7f7cc00b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:05.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.239+0000 7f7cc5f1f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cc00b1920 con 0x7f7cc00b5240 2020-12-05T17:39:05.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.239+0000 7f7cc5f1f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cc00b1bc0 con 0x7f7cc00b6690 2020-12-05T17:39:05.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.239+0000 7f7cc5f1f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cc00b5680 con 0x7f7cc00b5b20 2020-12-05T17:39:05.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.239+0000 7f7cbffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7cc00b5240 0x7f7cc0145f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40286/0 (socket says 172.21.15.32:40286) 2020-12-05T17:39:05.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.239+0000 7f7cbffff700 1 -- 172.21.15.32:0/1052538910 learned_addr learned my addr 172.21.15.32:0/1052538910 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:05.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.239+0000 7f7cbf7fe700 1 --2- 172.21.15.32:0/1052538910 >> v2:172.21.15.61:3300/0 conn(0x7f7cc00b5b20 0x7f7cc00b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:05.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.239+0000 7f7cbe7fc700 1 -- 172.21.15.32:0/1052538910 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252341625 0 0) 0x7f7cc00b1920 con 0x7f7cc00b5240 2020-12-05T17:39:05.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.239+0000 7f7cbe7fc700 1 -- 172.21.15.32:0/1052538910 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ca0003230 con 0x7f7cc00b5240 2020-12-05T17:39:05.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.239+0000 7f7cbf7fe700 1 -- 172.21.15.32:0/1052538910 >> v1:172.21.15.61:6789/0 conn(0x7f7cc00b6690 legacy=0x7f7cc00b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:05.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.239+0000 7f7cbf7fe700 1 -- 172.21.15.32:0/1052538910 >> v1:172.21.15.32:6789/0 conn(0x7f7cc00b5240 legacy=0x7f7cc0145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:05.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.239+0000 7f7cbe7fc700 1 -- 172.21.15.32:0/1052538910 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4144812590 0 0) 0x7f7ca0003230 con 0x7f7cc00b5240 2020-12-05T17:39:05.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.239+0000 7f7cbf7fe700 1 -- 172.21.15.32:0/1052538910 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc0146620 con 0x7f7cc00b5b20 2020-12-05T17:39:05.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.239+0000 7f7cbf7fe700 1 --2- 172.21.15.32:0/1052538910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7cc00b5b20 0x7f7cc00b5f00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f7cb8001bf0 tx=0x7f7ca8006dc0).ready entity=mon.1 client_cookie=b8827a289a3cd3a8 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:05.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.239+0000 7f7cbe7fc700 1 -- 172.21.15.32:0/1052538910 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cb8001a50 con 0x7f7cc00b5b20 2020-12-05T17:39:05.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.239+0000 7f7cbe7fc700 1 -- 172.21.15.32:0/1052538910 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca8009670 con 0x7f7cc00b5b20 2020-12-05T17:39:05.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.239+0000 7f7cbe7fc700 1 -- 172.21.15.32:0/1052538910 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ca80097e0 con 0x7f7cc00b5b20 2020-12-05T17:39:05.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.239+0000 7f7cc5f1f700 1 -- 172.21.15.32:0/1052538910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7cc00b5b20 msgr2=0x7f7cc00b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:05.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.239+0000 7f7cc5f1f700 1 --2- 172.21.15.32:0/1052538910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7cc00b5b20 0x7f7cc00b5f00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f7cb8001bf0 tx=0x7f7ca8006dc0).stop 2020-12-05T17:39:05.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.243+0000 7f7cc5f1f700 1 -- 172.21.15.32:0/1052538910 shutdown_connections 2020-12-05T17:39:05.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.243+0000 7f7cc5f1f700 1 --2- 172.21.15.32:0/1052538910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7cc00b5b20 0x7f7cc00b5f00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.243+0000 7f7cc5f1f700 1 -- 172.21.15.32:0/1052538910 >> 172.21.15.32:0/1052538910 conn(0x7f7cc005f3b0 msgr2=0x7f7cc005f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:05.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.243+0000 7f7cc5f1f700 1 -- 172.21.15.32:0/1052538910 shutdown_connections 2020-12-05T17:39:05.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.243+0000 7f7cc5f1f700 1 -- 172.21.15.32:0/1052538910 wait complete. 2020-12-05T17:39:05.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.243+0000 7f7cc5f1f700 1 Processor -- start 2020-12-05T17:39:05.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.243+0000 7f7cc5f1f700 1 -- start start 2020-12-05T17:39:05.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cc5f1f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7cc00b5240 0x7f7cc00a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:05.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cc5f1f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7cc00b5b20 0x7f7cc0151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:05.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cc5f1f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cc00b6690 0x7f7cc0151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:05.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cc5f1f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc0152470 con 0x7f7cc00b6690 2020-12-05T17:39:05.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cc5f1f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc01528a0 con 0x7f7cc00b5240 2020-12-05T17:39:05.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cc5f1f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc00659d0 con 0x7f7cc00b5b20 2020-12-05T17:39:05.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cbffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cc00b6690 0x7f7cc0151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:05.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cbffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cc00b6690 0x7f7cc0151e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37646/0 (socket says 172.21.15.32:37646) 2020-12-05T17:39:05.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cbffff700 1 -- 172.21.15.32:0/1459702943 learned_addr learned my addr 172.21.15.32:0/1459702943 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:05.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cbf7fe700 1 --2- 172.21.15.32:0/1459702943 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7cc00b5240 0x7f7cc00a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:05.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cbeffd700 1 --2- 172.21.15.32:0/1459702943 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7cc00b5b20 0x7f7cc0151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:05.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cbffff700 1 -- 172.21.15.32:0/1459702943 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7cc00b5b20 msgr2=0x7f7cc0151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:05.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cbffff700 1 --2- 172.21.15.32:0/1459702943 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7cc00b5b20 0x7f7cc0151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cbffff700 1 -- 172.21.15.32:0/1459702943 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7cc00b5240 msgr2=0x7f7cc00a5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:05.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cbffff700 1 --2- 172.21.15.32:0/1459702943 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7cc00b5240 0x7f7cc00a5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cbffff700 1 -- 172.21.15.32:0/1459702943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc00a5fa0 con 0x7f7cc00b6690 2020-12-05T17:39:05.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cbffff700 1 --2- 172.21.15.32:0/1459702943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cc00b6690 0x7f7cc0151e40 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7f7cb8001cb0 tx=0x7f7cb8001630).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:05.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cbcff9700 1 -- 172.21.15.32:0/1459702943 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cb8004a80 con 0x7f7cc00b6690 2020-12-05T17:39:05.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cbcff9700 1 -- 172.21.15.32:0/1459702943 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb8004be0 con 0x7f7cc00b6690 2020-12-05T17:39:05.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cc5f1f700 1 -- 172.21.15.32:0/1459702943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc00a5c40 con 0x7f7cc00b6690 2020-12-05T17:39:05.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cbcff9700 1 -- 172.21.15.32:0/1459702943 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cb8007170 con 0x7f7cc00b6690 2020-12-05T17:39:05.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cc5f1f700 1 -- 172.21.15.32:0/1459702943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc00669e0 con 0x7f7cc00b6690 2020-12-05T17:39:05.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cc5f1f700 1 -- 172.21.15.32:0/1459702943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cc0057050 con 0x7f7cc00b6690 2020-12-05T17:39:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cbcff9700 1 -- 172.21.15.32:0/1459702943 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7cb801b060 con 0x7f7cc00b6690 2020-12-05T17:39:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cbcff9700 1 --2- 172.21.15.32:0/1459702943 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7ca40553b0 0x7f7ca4057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cbcff9700 1 -- 172.21.15.32:0/1459702943 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7cb806df50 con 0x7f7cc00b6690 2020-12-05T17:39:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.247+0000 7f7cbf7fe700 1 --2- 172.21.15.32:0/1459702943 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7ca40553b0 0x7f7ca4057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:05.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.251+0000 7f7cbf7fe700 1 --2- 172.21.15.32:0/1459702943 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7ca40553b0 0x7f7ca4057870 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f7ca8017000 tx=0x7f7ca8015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:05.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.255+0000 7f7cbcff9700 1 -- 172.21.15.32:0/1459702943 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7cb80440f0 con 0x7f7cc00b6690 2020-12-05T17:39:05.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.395+0000 7f7cc5f1f700 1 -- 172.21.15.32:0/1459702943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7cc005bac0 con 0x7f7cc00b6690 2020-12-05T17:39:05.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.395+0000 7f7cbcff9700 1 -- 172.21.15.32:0/1459702943 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+10 (secure 0 0 0) 0x7f7cb8019020 con 0x7f7cc00b6690 2020-12-05T17:39:05.401 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:39:05.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.395+0000 7f7cc5f1f700 1 -- 172.21.15.32:0/1459702943 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7ca40553b0 msgr2=0x7f7ca4057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:05.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.395+0000 7f7cc5f1f700 1 --2- 172.21.15.32:0/1459702943 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7ca40553b0 0x7f7ca4057870 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f7ca8017000 tx=0x7f7ca8015000).stop 2020-12-05T17:39:05.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.395+0000 7f7cc5f1f700 1 -- 172.21.15.32:0/1459702943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cc00b6690 msgr2=0x7f7cc0151e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:05.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.395+0000 7f7cc5f1f700 1 --2- 172.21.15.32:0/1459702943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cc00b6690 0x7f7cc0151e40 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7f7cb8001cb0 tx=0x7f7cb8001630).stop 2020-12-05T17:39:05.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.395+0000 7f7cc5f1f700 1 -- 172.21.15.32:0/1459702943 shutdown_connections 2020-12-05T17:39:05.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.395+0000 7f7cc5f1f700 1 --2- 172.21.15.32:0/1459702943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cc00b6690 0x7f7cc0151e40 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.395+0000 7f7cc5f1f700 1 --2- 172.21.15.32:0/1459702943 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7cc00b5240 0x7f7cc00a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.395+0000 7f7cc5f1f700 1 --2- 172.21.15.32:0/1459702943 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7ca40553b0 0x7f7ca4057870 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.395+0000 7f7cc5f1f700 1 --2- 172.21.15.32:0/1459702943 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7cc00b5b20 0x7f7cc0151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.395+0000 7f7cc5f1f700 1 -- 172.21.15.32:0/1459702943 >> 172.21.15.32:0/1459702943 conn(0x7f7cc005f3b0 msgr2=0x7f7cc005f790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:05.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.395+0000 7f7cc5f1f700 1 -- 172.21.15.32:0/1459702943 shutdown_connections 2020-12-05T17:39:05.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.395+0000 7f7cc5f1f700 1 -- 172.21.15.32:0/1459702943 wait complete. 2020-12-05T17:39:05.415 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:39:05.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.527+0000 7fc3dc97e700 1 Processor -- start 2020-12-05T17:39:05.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.527+0000 7fc3dc97e700 1 -- start start 2020-12-05T17:39:05.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.527+0000 7fc3dc97e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc3d805f3b0 0x7fc3d805f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:05.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.527+0000 7fc3dc97e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc3d805feb0 0x7fc3d8061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:05.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.527+0000 7fc3dc97e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d80b2df0 con 0x7fc3d80b2fb0 2020-12-05T17:39:05.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.527+0000 7fc3dc97e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d80b3c90 con 0x7fc3d805f3b0 2020-12-05T17:39:05.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.527+0000 7fc3dc97e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d80b2b50 con 0x7fc3d805feb0 2020-12-05T17:39:05.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.527+0000 7fc3d6d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc3d805f3b0 0x7fc3d805f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:05.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.527+0000 7fc3d6d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc3d805f3b0 0x7fc3d805f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37652/0 (socket says 172.21.15.32:37652) 2020-12-05T17:39:05.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.527+0000 7fc3d759e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc3d80b2fb0 0x7fc3d813c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40304/0 (socket says 172.21.15.32:40304) 2020-12-05T17:39:05.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.527+0000 7fc3d6d9d700 1 -- 172.21.15.32:0/1507643573 learned_addr learned my addr 172.21.15.32:0/1507643573 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:05.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.527+0000 7fc3d659c700 1 --2- 172.21.15.32:0/1507643573 >> v2:172.21.15.61:3300/0 conn(0x7fc3d805feb0 0x7fc3d8061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:05.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.527+0000 7fc3d6d9d700 1 -- 172.21.15.32:0/1507643573 >> v2:172.21.15.61:3300/0 conn(0x7fc3d805feb0 msgr2=0x7fc3d8061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:05.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.527+0000 7fc3d6d9d700 1 --2- 172.21.15.32:0/1507643573 >> v2:172.21.15.61:3300/0 conn(0x7fc3d805feb0 0x7fc3d8061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.527+0000 7fc3d6d9d700 1 -- 172.21.15.32:0/1507643573 >> v1:172.21.15.32:6789/0 conn(0x7fc3d80b2fb0 legacy=0x7fc3d813c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:05.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.527+0000 7fc3d6d9d700 1 -- 172.21.15.32:0/1507643573 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d8142820 con 0x7fc3d805f3b0 2020-12-05T17:39:05.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.527+0000 7fc3d6d9d700 1 --2- 172.21.15.32:0/1507643573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3d805f3b0 0x7fc3d805f790 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7fc3c8002f30 tx=0x7fc3c8006e80).ready entity=mon.0 client_cookie=431b4cc95e305888 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:05.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.527+0000 7fc3d5d9b700 1 -- 172.21.15.32:0/1507643573 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3c80097f0 con 0x7fc3d805f3b0 2020-12-05T17:39:05.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.527+0000 7fc3d5d9b700 1 -- 172.21.15.32:0/1507643573 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c8009950 con 0x7fc3d805f3b0 2020-12-05T17:39:05.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.527+0000 7fc3d5d9b700 1 -- 172.21.15.32:0/1507643573 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3c8009cb0 con 0x7fc3d805f3b0 2020-12-05T17:39:05.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.527+0000 7fc3dc97e700 1 -- 172.21.15.32:0/1507643573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3d805f3b0 msgr2=0x7fc3d805f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:05.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.527+0000 7fc3dc97e700 1 --2- 172.21.15.32:0/1507643573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3d805f3b0 0x7fc3d805f790 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7fc3c8002f30 tx=0x7fc3c8006e80).stop 2020-12-05T17:39:05.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.531+0000 7fc3dc97e700 1 -- 172.21.15.32:0/1507643573 shutdown_connections 2020-12-05T17:39:05.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.531+0000 7fc3dc97e700 1 --2- 172.21.15.32:0/1507643573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3d805f3b0 0x7fc3d805f790 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.531+0000 7fc3dc97e700 1 --2- 172.21.15.32:0/1507643573 >> v2:172.21.15.61:3300/0 conn(0x7fc3d805feb0 0x7fc3d8061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.531+0000 7fc3dc97e700 1 -- 172.21.15.32:0/1507643573 >> 172.21.15.32:0/1507643573 conn(0x7fc3d80b8f20 msgr2=0x7fc3d80b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:05.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.531+0000 7fc3dc97e700 1 -- 172.21.15.32:0/1507643573 shutdown_connections 2020-12-05T17:39:05.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.531+0000 7fc3dc97e700 1 -- 172.21.15.32:0/1507643573 wait complete. 2020-12-05T17:39:05.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.531+0000 7fc3dc97e700 1 Processor -- start 2020-12-05T17:39:05.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.531+0000 7fc3dc97e700 1 -- start start 2020-12-05T17:39:05.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.535+0000 7fc3dc97e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc3d805f3b0 0x7fc3d8144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:05.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.535+0000 7fc3dc97e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc3d805feb0 0x7fc3d8151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:05.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.535+0000 7fc3dc97e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3d80b2fb0 0x7fc3d8145770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:05.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.535+0000 7fc3dc97e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d813f230 con 0x7fc3d80b2fb0 2020-12-05T17:39:05.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.535+0000 7fc3dc97e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d813f4d0 con 0x7fc3d805f3b0 2020-12-05T17:39:05.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.535+0000 7fc3dc97e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d81452d0 con 0x7fc3d805feb0 2020-12-05T17:39:05.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.535+0000 7fc3d759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3d80b2fb0 0x7fc3d8145770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:05.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.535+0000 7fc3d759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3d80b2fb0 0x7fc3d8145770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37662/0 (socket says 172.21.15.32:37662) 2020-12-05T17:39:05.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.535+0000 7fc3d759e700 1 -- 172.21.15.32:0/998837877 learned_addr learned my addr 172.21.15.32:0/998837877 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:05.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.535+0000 7fc3d6d9d700 1 --2- 172.21.15.32:0/998837877 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc3d805f3b0 0x7fc3d8144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:05.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.535+0000 7fc3d659c700 1 --2- 172.21.15.32:0/998837877 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc3d805feb0 0x7fc3d8151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:05.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.535+0000 7fc3d759e700 1 -- 172.21.15.32:0/998837877 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc3d805feb0 msgr2=0x7fc3d8151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:05.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.535+0000 7fc3d759e700 1 --2- 172.21.15.32:0/998837877 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc3d805feb0 0x7fc3d8151320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.535+0000 7fc3d759e700 1 -- 172.21.15.32:0/998837877 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc3d805f3b0 msgr2=0x7fc3d8144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:05.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.535+0000 7fc3d759e700 1 --2- 172.21.15.32:0/998837877 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc3d805f3b0 0x7fc3d8144320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.535+0000 7fc3d759e700 1 -- 172.21.15.32:0/998837877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d8144ed0 con 0x7fc3d80b2fb0 2020-12-05T17:39:05.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.535+0000 7fc3d759e700 1 --2- 172.21.15.32:0/998837877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3d80b2fb0 0x7fc3d8145770 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7fc3d8061550 tx=0x7fc3d0007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:05.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.535+0000 7fc3c7fff700 1 -- 172.21.15.32:0/998837877 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3d00058a0 con 0x7fc3d80b2fb0 2020-12-05T17:39:05.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.535+0000 7fc3c7fff700 1 -- 172.21.15.32:0/998837877 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d0005a00 con 0x7fc3d80b2fb0 2020-12-05T17:39:05.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.535+0000 7fc3c7fff700 1 -- 172.21.15.32:0/998837877 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3d0005d60 con 0x7fc3d80b2fb0 2020-12-05T17:39:05.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.535+0000 7fc3dc97e700 1 -- 172.21.15.32:0/998837877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d8146ae0 con 0x7fc3d80b2fb0 2020-12-05T17:39:05.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.535+0000 7fc3dc97e700 1 -- 172.21.15.32:0/998837877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d813d380 con 0x7fc3d80b2fb0 2020-12-05T17:39:05.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.539+0000 7fc3dc97e700 1 -- 172.21.15.32:0/998837877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d8145490 con 0x7fc3d80b2fb0 2020-12-05T17:39:05.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.539+0000 7fc3c7fff700 1 -- 172.21.15.32:0/998837877 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc3d0010030 con 0x7fc3d80b2fb0 2020-12-05T17:39:05.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.539+0000 7fc3c7fff700 1 --2- 172.21.15.32:0/998837877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc3b4055330 0x7fc3b40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:05.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.539+0000 7fc3c7fff700 1 -- 172.21.15.32:0/998837877 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc3d006de80 con 0x7fc3d80b2fb0 2020-12-05T17:39:05.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.539+0000 7fc3d6d9d700 1 --2- 172.21.15.32:0/998837877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc3b4055330 0x7fc3b40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:05.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.539+0000 7fc3d6d9d700 1 --2- 172.21.15.32:0/998837877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc3b4055330 0x7fc3b40577f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fc3c801c000 tx=0x7fc3c8015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:05.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.543+0000 7fc3c7fff700 1 -- 172.21.15.32:0/998837877 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc3d0043fa0 con 0x7fc3d80b2fb0 2020-12-05T17:39:05.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.683+0000 7fc3dc97e700 1 -- 172.21.15.32:0/998837877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc3d8057050 con 0x7fc3d80b2fb0 2020-12-05T17:39:05.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.683+0000 7fc3c7fff700 1 -- 172.21.15.32:0/998837877 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc3d00475c0 con 0x7fc3d80b2fb0 2020-12-05T17:39:05.691 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:39:05.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.687+0000 7fc3dc97e700 1 -- 172.21.15.32:0/998837877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc3b4055330 msgr2=0x7fc3b40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:05.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.687+0000 7fc3dc97e700 1 --2- 172.21.15.32:0/998837877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc3b4055330 0x7fc3b40577f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fc3c801c000 tx=0x7fc3c8015000).stop 2020-12-05T17:39:05.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.687+0000 7fc3dc97e700 1 -- 172.21.15.32:0/998837877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3d80b2fb0 msgr2=0x7fc3d8145770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:05.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.687+0000 7fc3dc97e700 1 --2- 172.21.15.32:0/998837877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3d80b2fb0 0x7fc3d8145770 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7fc3d8061550 tx=0x7fc3d0007f70).stop 2020-12-05T17:39:05.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.687+0000 7fc3dc97e700 1 -- 172.21.15.32:0/998837877 shutdown_connections 2020-12-05T17:39:05.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.687+0000 7fc3dc97e700 1 --2- 172.21.15.32:0/998837877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3d80b2fb0 0x7fc3d8145770 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.687+0000 7fc3dc97e700 1 --2- 172.21.15.32:0/998837877 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc3d805f3b0 0x7fc3d8144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.687+0000 7fc3dc97e700 1 --2- 172.21.15.32:0/998837877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc3b4055330 0x7fc3b40577f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.687+0000 7fc3dc97e700 1 --2- 172.21.15.32:0/998837877 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc3d805feb0 0x7fc3d8151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.687+0000 7fc3dc97e700 1 -- 172.21.15.32:0/998837877 >> 172.21.15.32:0/998837877 conn(0x7fc3d80b8f20 msgr2=0x7fc3d80b55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:05.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.687+0000 7fc3dc97e700 1 -- 172.21.15.32:0/998837877 shutdown_connections 2020-12-05T17:39:05.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.687+0000 7fc3dc97e700 1 -- 172.21.15.32:0/998837877 wait complete. 2020-12-05T17:39:05.704 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:39:05.705 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:39:05.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e9d0e700 1 Processor -- start 2020-12-05T17:39:05.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e9d0e700 1 -- start start 2020-12-05T17:39:05.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e9d0e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20e405f3b0 0x7f20e405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:05.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e9d0e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f20e4062400 0x7f20e40b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:05.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e9d0e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20e405feb0 0x7f20e40b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:05.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e9d0e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e4142450 con 0x7f20e405feb0 2020-12-05T17:39:05.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e9d0e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e4061780 con 0x7f20e4062400 2020-12-05T17:39:05.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e9d0e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e4061a20 con 0x7f20e405f3b0 2020-12-05T17:39:05.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e950d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20e405feb0 0x7f20e40b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:05.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e950d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20e405feb0 0x7f20e40b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37670/0 (socket says 172.21.15.32:37670) 2020-12-05T17:39:05.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e950d700 1 -- 172.21.15.32:0/882088467 learned_addr learned my addr 172.21.15.32:0/882088467 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:05.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e3fff700 1 --2- 172.21.15.32:0/882088467 >> v2:172.21.15.61:3300/0 conn(0x7f20e4062400 0x7f20e40b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:05.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e8d0c700 1 --2- 172.21.15.32:0/882088467 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20e405f3b0 0x7f20e405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:05.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e950d700 1 -- 172.21.15.32:0/882088467 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20e405f3b0 msgr2=0x7f20e405f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e950d700 1 --2- 172.21.15.32:0/882088467 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20e405f3b0 0x7f20e405f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e950d700 1 -- 172.21.15.32:0/882088467 >> v2:172.21.15.61:3300/0 conn(0x7f20e4062400 msgr2=0x7f20e40b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e950d700 1 --2- 172.21.15.32:0/882088467 >> v2:172.21.15.61:3300/0 conn(0x7f20e4062400 0x7f20e40b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e950d700 1 -- 172.21.15.32:0/882088467 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e4061cc0 con 0x7f20e405feb0 2020-12-05T17:39:05.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e3fff700 1 --2- 172.21.15.32:0/882088467 >> v2:172.21.15.61:3300/0 conn(0x7f20e4062400 0x7f20e40b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:05.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e950d700 1 --2- 172.21.15.32:0/882088467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e405feb0 0x7f20e40b3be0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x7f20cc002f30 tx=0x7f20cc006e80).ready entity=mon.0 client_cookie=52697aa6cc424468 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:05.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e37fe700 1 -- 172.21.15.32:0/882088467 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20cc0097f0 con 0x7f20e405feb0 2020-12-05T17:39:05.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e37fe700 1 -- 172.21.15.32:0/882088467 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20cc009950 con 0x7f20e405feb0 2020-12-05T17:39:05.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e37fe700 1 -- 172.21.15.32:0/882088467 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20cc009cb0 con 0x7f20e405feb0 2020-12-05T17:39:05.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e9d0e700 1 -- 172.21.15.32:0/882088467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e405feb0 msgr2=0x7f20e40b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:05.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e9d0e700 1 --2- 172.21.15.32:0/882088467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e405feb0 0x7f20e40b3be0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x7f20cc002f30 tx=0x7f20cc006e80).stop 2020-12-05T17:39:05.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e9d0e700 1 -- 172.21.15.32:0/882088467 shutdown_connections 2020-12-05T17:39:05.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e9d0e700 1 --2- 172.21.15.32:0/882088467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e405feb0 0x7f20e40b3be0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e9d0e700 1 --2- 172.21.15.32:0/882088467 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20e405f3b0 0x7f20e405f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e9d0e700 1 --2- 172.21.15.32:0/882088467 >> v2:172.21.15.61:3300/0 conn(0x7f20e4062400 0x7f20e40b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e9d0e700 1 -- 172.21.15.32:0/882088467 >> 172.21.15.32:0/882088467 conn(0x7f20e40b8ee0 msgr2=0x7f20e40b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e9d0e700 1 -- 172.21.15.32:0/882088467 shutdown_connections 2020-12-05T17:39:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.811+0000 7f20e9d0e700 1 -- 172.21.15.32:0/882088467 wait complete. 2020-12-05T17:39:05.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e9d0e700 1 Processor -- start 2020-12-05T17:39:05.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e9d0e700 1 -- start start 2020-12-05T17:39:05.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e9d0e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e405f3b0 0x7f20e40632f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:05.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e9d0e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f20e405feb0 0x7f20e4143a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:05.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e9d0e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20e4062400 0x7f20e4143f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:05.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e9d0e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e4063f70 con 0x7f20e405f3b0 2020-12-05T17:39:05.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e9d0e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e4145fb0 con 0x7f20e405feb0 2020-12-05T17:39:05.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e9d0e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e4146250 con 0x7f20e4062400 2020-12-05T17:39:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e8d0c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e405f3b0 0x7f20e40632f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e8d0c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e405f3b0 0x7f20e40632f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37672/0 (socket says 172.21.15.32:37672) 2020-12-05T17:39:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e8d0c700 1 -- 172.21.15.32:0/2196490696 learned_addr learned my addr 172.21.15.32:0/2196490696 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e950d700 1 --2- 172.21.15.32:0/2196490696 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20e4062400 0x7f20e4143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:05.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e3fff700 1 --2- 172.21.15.32:0/2196490696 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f20e405feb0 0x7f20e4143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:05.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e8d0c700 1 -- 172.21.15.32:0/2196490696 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20e4062400 msgr2=0x7f20e4143f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:05.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e8d0c700 1 --2- 172.21.15.32:0/2196490696 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20e4062400 0x7f20e4143f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e8d0c700 1 -- 172.21.15.32:0/2196490696 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f20e405feb0 msgr2=0x7f20e4143a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:05.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e8d0c700 1 --2- 172.21.15.32:0/2196490696 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f20e405feb0 0x7f20e4143a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:05.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e8d0c700 1 -- 172.21.15.32:0/2196490696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e41464f0 con 0x7f20e405f3b0 2020-12-05T17:39:05.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e8d0c700 1 --2- 172.21.15.32:0/2196490696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e405f3b0 0x7f20e40632f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7f20dc0033f0 tx=0x7f20dc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:05.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e1ffb700 1 -- 172.21.15.32:0/2196490696 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20dc00ba70 con 0x7f20e405f3b0 2020-12-05T17:39:05.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e1ffb700 1 -- 172.21.15.32:0/2196490696 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20dc00bbd0 con 0x7f20e405f3b0 2020-12-05T17:39:05.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e9d0e700 1 -- 172.21.15.32:0/2196490696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e40afc10 con 0x7f20e405f3b0 2020-12-05T17:39:05.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e1ffb700 1 -- 172.21.15.32:0/2196490696 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20dc012420 con 0x7f20e405f3b0 2020-12-05T17:39:05.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.815+0000 7f20e9d0e700 1 -- 172.21.15.32:0/2196490696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e4145aa0 con 0x7f20e405f3b0 2020-12-05T17:39:05.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.819+0000 7f20e9d0e700 1 -- 172.21.15.32:0/2196490696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e4144f60 con 0x7f20e405f3b0 2020-12-05T17:39:05.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.819+0000 7f20e1ffb700 1 -- 172.21.15.32:0/2196490696 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f20dc012580 con 0x7f20e405f3b0 2020-12-05T17:39:05.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.819+0000 7f20e1ffb700 1 --2- 172.21.15.32:0/2196490696 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f20c80553b0 0x7f20c8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:05.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.819+0000 7f20e1ffb700 1 -- 172.21.15.32:0/2196490696 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f20dc06e6c0 con 0x7f20e405f3b0 2020-12-05T17:39:05.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.819+0000 7f20e3fff700 1 --2- 172.21.15.32:0/2196490696 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f20c80553b0 0x7f20c8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:05.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.819+0000 7f20e3fff700 1 --2- 172.21.15.32:0/2196490696 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f20c80553b0 0x7f20c8057870 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f20d400c000 tx=0x7f20d4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:05.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:05.827+0000 7f20e1ffb700 1 -- 172.21.15.32:0/2196490696 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f20e4144f60 con 0x7f20e405f3b0 2020-12-05T17:39:06.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:06.007+0000 7f20e9d0e700 1 -- 172.21.15.32:0/2196490696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20e4144f60 con 0x7f20e405f3b0 2020-12-05T17:39:06.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:06.011+0000 7f20e1ffb700 1 -- 172.21.15.32:0/2196490696 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f20e4144f60 con 0x7f20e405f3b0 2020-12-05T17:39:06.016 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:06.016 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":37,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6535,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2494768218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2494768218},{"type":"v1","addr":"172.21.15.61:6833","nonce":2494768218}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":35}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:38:56.821107+0000","modified":"2020-12-05T17:38:57.824754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/81037173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":81037173},{"type":"v1","addr":"172.21.15.32:6835","nonce":81037173}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-12-05T17:39:06.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:06.011+0000 7f20e9d0e700 1 -- 172.21.15.32:0/2196490696 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f20c80553b0 msgr2=0x7f20c8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:06.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:06.011+0000 7f20e9d0e700 1 --2- 172.21.15.32:0/2196490696 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f20c80553b0 0x7f20c8057870 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f20d400c000 tx=0x7f20d4008040).stop 2020-12-05T17:39:06.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:06.011+0000 7f20e9d0e700 1 -- 172.21.15.32:0/2196490696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e405f3b0 msgr2=0x7f20e40632f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:06.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:06.011+0000 7f20e9d0e700 1 --2- 172.21.15.32:0/2196490696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e405f3b0 0x7f20e40632f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7f20dc0033f0 tx=0x7f20dc008040).stop 2020-12-05T17:39:06.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:06.011+0000 7f20e9d0e700 1 -- 172.21.15.32:0/2196490696 shutdown_connections 2020-12-05T17:39:06.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:06.011+0000 7f20e9d0e700 1 --2- 172.21.15.32:0/2196490696 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f20c80553b0 0x7f20c8057870 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:06.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:06.011+0000 7f20e9d0e700 1 --2- 172.21.15.32:0/2196490696 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20e4062400 0x7f20e4143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:06.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:06.011+0000 7f20e9d0e700 1 --2- 172.21.15.32:0/2196490696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e405f3b0 0x7f20e40632f0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:06.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:06.011+0000 7f20e9d0e700 1 --2- 172.21.15.32:0/2196490696 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f20e405feb0 0x7f20e4143a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:06.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:06.011+0000 7f20e9d0e700 1 -- 172.21.15.32:0/2196490696 >> 172.21.15.32:0/2196490696 conn(0x7f20e40b8ee0 msgr2=0x7f20e413f180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:06.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:06.011+0000 7f20e9d0e700 1 -- 172.21.15.32:0/2196490696 shutdown_connections 2020-12-05T17:39:06.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:06.011+0000 7f20e9d0e700 1 -- 172.21.15.32:0/2196490696 wait complete. 2020-12-05T17:39:06.021 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 37 2020-12-05T17:39:06.231 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:38:56.821107+0000', 'modified': '2020-12-05T17:38:57.824754+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6609}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6609': {'gid': 6609, 'name': 'a', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/81037173', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 81037173}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 81037173}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:39:06.231 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:39:06.232 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:39:06.365 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":6609,"want_state":"up:active","state":"up:active","rank_uptime":9.5283834360000004,"mdsmap_epoch":37,"osdmap_epoch":83,"osdmap_epoch_barrier":83,"uptime":14.777891207} 2020-12-05T17:39:06.365 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 6609, "mdsmap_epoch": 37, "osdmap_epoch": 83, "osdmap_epoch_barrier": 83, "rank_uptime": 9.528383436, "state": "up:active", "uptime": 14.777891207, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:39:06.366 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:39:06.366 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-12-05T17:39:06.366 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi002.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-12-05T17:39:06.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:06.367 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:06.375 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:39:06.433 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:39:06.434 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:39:06.489 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:39:06.490 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:39:06.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:06.491 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:39:06.535 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:39:06.536 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:06.580 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:39:06.596 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:39:06.596 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:39:06.647 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:39:06.648 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:39:06.649 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:06.649 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:39:06.653 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:2020-12-05T17:39:06.648+0000 7fabb6e510c0 -1 init, newargv = 0x558ac4b92c00 newargc=9 2020-12-05T17:39:06.654 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[10653]: starting ceph client 2020-12-05T17:39:06.664 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[10653]: starting fuse 2020-12-05T17:39:07.760 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:39:07.776 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:39:07.777 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:39:07.832 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:39:07.834 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:39:07.835 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:07.835 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:39:07.885 INFO:teuthology.orchestra.run.smithi002.stdout:50 2020-12-05T17:39:07.885 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [50] 2020-12-05T17:39:07.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:07.887 INFO:teuthology.orchestra.run.smithi002:> sudo python3 -c ' 2020-12-05T17:39:07.887 INFO:teuthology.orchestra.run.smithi002:> import glob 2020-12-05T17:39:07.887 INFO:teuthology.orchestra.run.smithi002:> import re 2020-12-05T17:39:07.888 INFO:teuthology.orchestra.run.smithi002:> import os 2020-12-05T17:39:07.888 INFO:teuthology.orchestra.run.smithi002:> import subprocess 2020-12-05T17:39:07.889 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:39:07.889 INFO:teuthology.orchestra.run.smithi002:> def find_socket(client_name): 2020-12-05T17:39:07.889 INFO:teuthology.orchestra.run.smithi002:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-12-05T17:39:07.890 INFO:teuthology.orchestra.run.smithi002:> files = glob.glob(asok_path) 2020-12-05T17:39:07.890 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:39:07.891 INFO:teuthology.orchestra.run.smithi002:> # Given a non-glob path, it better be there 2020-12-05T17:39:07.891 INFO:teuthology.orchestra.run.smithi002:> if "*" not in asok_path: 2020-12-05T17:39:07.891 INFO:teuthology.orchestra.run.smithi002:> assert(len(files) == 1) 2020-12-05T17:39:07.892 INFO:teuthology.orchestra.run.smithi002:> return files[0] 2020-12-05T17:39:07.892 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:39:07.892 INFO:teuthology.orchestra.run.smithi002:> for f in files: 2020-12-05T17:39:07.893 INFO:teuthology.orchestra.run.smithi002:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:39:07.893 INFO:teuthology.orchestra.run.smithi002:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:39:07.894 INFO:teuthology.orchestra.run.smithi002:> return f 2020-12-05T17:39:07.894 INFO:teuthology.orchestra.run.smithi002:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:39:07.895 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:39:07.895 INFO:teuthology.orchestra.run.smithi002:> print(find_socket("client.1")) 2020-12-05T17:39:07.895 INFO:teuthology.orchestra.run.smithi002:> ' 2020-12-05T17:39:07.977 INFO:teuthology.orchestra.run.smithi002.stdout:/var/run/ceph/ceph-client.1.10653.asok 2020-12-05T17:39:07.982 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.10653.asok 2020-12-05T17:39:07.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:07.983 INFO:teuthology.orchestra.run.smithi002:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.10653.asok status 2020-12-05T17:39:08.082 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2020-12-05T17:39:08.083 INFO:teuthology.orchestra.run.smithi002.stdout: "metadata": { 2020-12-05T17:39:08.083 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:39:08.083 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:39:08.083 INFO:teuthology.orchestra.run.smithi002.stdout: "entity_id": "1", 2020-12-05T17:39:08.084 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": "smithi002", 2020-12-05T17:39:08.084 INFO:teuthology.orchestra.run.smithi002.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-12-05T17:39:08.084 INFO:teuthology.orchestra.run.smithi002.stdout: "pid": "10653", 2020-12-05T17:39:08.084 INFO:teuthology.orchestra.run.smithi002.stdout: "root": "/" 2020-12-05T17:39:08.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:39:08.085 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_count": 0, 2020-12-05T17:39:08.085 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_pinned_count": 0, 2020-12-05T17:39:08.085 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 6742, 2020-12-05T17:39:08.085 INFO:teuthology.orchestra.run.smithi002.stdout: "inst": { 2020-12-05T17:39:08.085 INFO:teuthology.orchestra.run.smithi002.stdout: "name": { 2020-12-05T17:39:08.086 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "client", 2020-12-05T17:39:08.086 INFO:teuthology.orchestra.run.smithi002.stdout: "num": 6742 2020-12-05T17:39:08.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:39:08.086 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2020-12-05T17:39:08.086 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2020-12-05T17:39:08.087 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2020-12-05T17:39:08.087 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 102461427 2020-12-05T17:39:08.087 INFO:teuthology.orchestra.run.smithi002.stdout: } 2020-12-05T17:39:08.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:39:08.088 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2020-12-05T17:39:08.088 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2020-12-05T17:39:08.088 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2020-12-05T17:39:08.088 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 102461427 2020-12-05T17:39:08.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:39:08.089 INFO:teuthology.orchestra.run.smithi002.stdout: "inst_str": "client.6742 172.21.15.2:0/102461427", 2020-12-05T17:39:08.089 INFO:teuthology.orchestra.run.smithi002.stdout: "addr_str": "172.21.15.2:0/102461427", 2020-12-05T17:39:08.089 INFO:teuthology.orchestra.run.smithi002.stdout: "inode_count": 1, 2020-12-05T17:39:08.090 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_epoch": 37, 2020-12-05T17:39:08.090 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch": 83, 2020-12-05T17:39:08.090 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:39:08.090 INFO:teuthology.orchestra.run.smithi002.stdout: "blacklisted": false 2020-12-05T17:39:08.091 INFO:teuthology.orchestra.run.smithi002.stdout:} 2020-12-05T17:39:08.092 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:08.092 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:08.093 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:08.099 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:39:08.100 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:39:08.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:08.101 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:08.159 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:39:08.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.275+0000 7f9a12acc700 1 Processor -- start 2020-12-05T17:39:08.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.275+0000 7f9a12acc700 1 -- start start 2020-12-05T17:39:08.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.275+0000 7f9a12acc700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9a0c0b1fc0 0x7f9a0c0af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:08.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.275+0000 7f9a12acc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a0c0ab710 0x7f9a0c061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:08.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.275+0000 7f9a12acc700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0c146a90 con 0x7f9a0c147400 2020-12-05T17:39:08.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.275+0000 7f9a12acc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0c0b0110 con 0x7f9a0c0ab710 2020-12-05T17:39:08.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.275+0000 7f9a12acc700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0c0b23a0 con 0x7f9a0c0b1fc0 2020-12-05T17:39:08.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.275+0000 7f9a112c9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a0c0ab710 0x7f9a0c061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:08.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.275+0000 7f9a11aca700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9a0c0b1fc0 0x7f9a0c0af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:08.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.275+0000 7f9a112c9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a0c0ab710 0x7f9a0c061780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37684/0 (socket says 172.21.15.32:37684) 2020-12-05T17:39:08.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.275+0000 7f9a112c9700 1 -- 172.21.15.32:0/2496838882 learned_addr learned my addr 172.21.15.32:0/2496838882 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:08.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.275+0000 7f9a112c9700 1 -- 172.21.15.32:0/2496838882 >> v2:172.21.15.61:3300/0 conn(0x7f9a0c0b1fc0 msgr2=0x7f9a0c0af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:08.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.275+0000 7f9a112c9700 1 --2- 172.21.15.32:0/2496838882 >> v2:172.21.15.61:3300/0 conn(0x7f9a0c0b1fc0 0x7f9a0c0af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:08.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.275+0000 7f9a112c9700 1 -- 172.21.15.32:0/2496838882 >> v1:172.21.15.61:6789/0 conn(0x7f9a0c147400 legacy=0x7f9a0c145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:08.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.275+0000 7f9a112c9700 1 -- 172.21.15.32:0/2496838882 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c146cc0 con 0x7f9a0c0ab710 2020-12-05T17:39:08.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.275+0000 7f9a112c9700 1 --2- 172.21.15.32:0/2496838882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0c0ab710 0x7f9a0c061780 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 rx=0x7f9a00002f30 tx=0x7f9a00006e80).ready entity=mon.0 client_cookie=2b6ce9b7c9152562 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:08.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.279+0000 7f9a10ac8700 1 -- 172.21.15.32:0/2496838882 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a000097f0 con 0x7f9a0c0ab710 2020-12-05T17:39:08.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.279+0000 7f9a10ac8700 1 -- 172.21.15.32:0/2496838882 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a00009950 con 0x7f9a0c0ab710 2020-12-05T17:39:08.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.279+0000 7f9a12acc700 1 -- 172.21.15.32:0/2496838882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0c0ab710 msgr2=0x7f9a0c061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:08.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.279+0000 7f9a10ac8700 1 -- 172.21.15.32:0/2496838882 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a00009cb0 con 0x7f9a0c0ab710 2020-12-05T17:39:08.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.279+0000 7f9a12acc700 1 --2- 172.21.15.32:0/2496838882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0c0ab710 0x7f9a0c061780 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 rx=0x7f9a00002f30 tx=0x7f9a00006e80).stop 2020-12-05T17:39:08.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.279+0000 7f9a12acc700 1 -- 172.21.15.32:0/2496838882 shutdown_connections 2020-12-05T17:39:08.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.279+0000 7f9a12acc700 1 --2- 172.21.15.32:0/2496838882 >> v2:172.21.15.61:3300/0 conn(0x7f9a0c0b1fc0 0x7f9a0c0af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:08.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.279+0000 7f9a12acc700 1 --2- 172.21.15.32:0/2496838882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0c0ab710 0x7f9a0c061780 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.279+0000 7f9a12acc700 1 -- 172.21.15.32:0/2496838882 >> 172.21.15.32:0/2496838882 conn(0x7f9a0c0b5530 msgr2=0x7f9a0c0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.279+0000 7f9a12acc700 1 -- 172.21.15.32:0/2496838882 shutdown_connections 2020-12-05T17:39:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.279+0000 7f9a12acc700 1 -- 172.21.15.32:0/2496838882 wait complete. 2020-12-05T17:39:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.279+0000 7f9a12acc700 1 Processor -- start 2020-12-05T17:39:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.279+0000 7f9a12acc700 1 -- start start 2020-12-05T17:39:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.283+0000 7f9a12acc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0c0ab710 0x7f9a0c14d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.283+0000 7f9a12acc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a0c0b1fc0 0x7f9a0c140b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.283+0000 7f9a12acc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a0c147400 0x7f9a0c141080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.283+0000 7f9a12acc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0ae860 con 0x7f9a0c0ab710 2020-12-05T17:39:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.283+0000 7f9a12acc700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0ad840 con 0x7f9a0c147400 2020-12-05T17:39:08.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.283+0000 7f9a12acc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0625e0 con 0x7f9a0c0b1fc0 2020-12-05T17:39:08.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.283+0000 7f9a11aca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0c0ab710 0x7f9a0c14d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:08.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.283+0000 7f9a11aca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0c0ab710 0x7f9a0c14d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37686/0 (socket says 172.21.15.32:37686) 2020-12-05T17:39:08.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.283+0000 7f9a11aca700 1 -- 172.21.15.32:0/2891894534 learned_addr learned my addr 172.21.15.32:0/2891894534 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:08.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.283+0000 7f9a112c9700 1 --2- 172.21.15.32:0/2891894534 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a0c0b1fc0 0x7f9a0c140b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:08.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.283+0000 7f9a122cb700 1 --2- 172.21.15.32:0/2891894534 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a0c147400 0x7f9a0c141080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:08.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.283+0000 7f9a11aca700 1 -- 172.21.15.32:0/2891894534 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a0c0b1fc0 msgr2=0x7f9a0c140b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:08.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.283+0000 7f9a11aca700 1 --2- 172.21.15.32:0/2891894534 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a0c0b1fc0 0x7f9a0c140b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:08.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.283+0000 7f9a11aca700 1 -- 172.21.15.32:0/2891894534 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a0c147400 msgr2=0x7f9a0c141080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:08.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.283+0000 7f9a11aca700 1 --2- 172.21.15.32:0/2891894534 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a0c147400 0x7f9a0c141080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:08.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.283+0000 7f9a11aca700 1 -- 172.21.15.32:0/2891894534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c0ae5c0 con 0x7f9a0c0ab710 2020-12-05T17:39:08.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.283+0000 7f9a11aca700 1 --2- 172.21.15.32:0/2891894534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0c0ab710 0x7f9a0c14d770 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7f9a0800c000 tx=0x7f9a08006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:08.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.283+0000 7f99feffd700 1 -- 172.21.15.32:0/2891894534 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a0800ca10 con 0x7f9a0c0ab710 2020-12-05T17:39:08.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.283+0000 7f99feffd700 1 -- 172.21.15.32:0/2891894534 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0800cb70 con 0x7f9a0c0ab710 2020-12-05T17:39:08.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.283+0000 7f9a12acc700 1 -- 172.21.15.32:0/2891894534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0c14dc60 con 0x7f9a0c0ab710 2020-12-05T17:39:08.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.283+0000 7f99feffd700 1 -- 172.21.15.32:0/2891894534 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a080123f0 con 0x7f9a0c0ab710 2020-12-05T17:39:08.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.283+0000 7f9a12acc700 1 -- 172.21.15.32:0/2891894534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0c141570 con 0x7f9a0c0ab710 2020-12-05T17:39:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.283+0000 7f99feffd700 1 -- 172.21.15.32:0/2891894534 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9a08012550 con 0x7f9a0c0ab710 2020-12-05T17:39:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.287+0000 7f9a12acc700 1 -- 172.21.15.32:0/2891894534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0c057050 con 0x7f9a0c0ab710 2020-12-05T17:39:08.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.287+0000 7f99feffd700 1 --2- 172.21.15.32:0/2891894534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f99ec0553b0 0x7f99ec057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:08.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.287+0000 7f99feffd700 1 -- 172.21.15.32:0/2891894534 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9a0806e4a0 con 0x7f9a0c0ab710 2020-12-05T17:39:08.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.287+0000 7f9a112c9700 1 --2- 172.21.15.32:0/2891894534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f99ec0553b0 0x7f99ec057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:08.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.287+0000 7f9a112c9700 1 --2- 172.21.15.32:0/2891894534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f99ec0553b0 0x7f99ec057870 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f9a0001c000 tx=0x7f9a00015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:08.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.291+0000 7f99feffd700 1 -- 172.21.15.32:0/2891894534 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9a080445e0 con 0x7f9a0c0ab710 2020-12-05T17:39:08.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.479+0000 7f9a12acc700 1 -- 172.21.15.32:0/2891894534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a0c05aab0 con 0x7f9a0c0ab710 2020-12-05T17:39:08.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.479+0000 7f99feffd700 1 -- 172.21.15.32:0/2891894534 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f9a08047c00 con 0x7f9a0c0ab710 2020-12-05T17:39:08.485 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:08.486 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":37,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6535,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2494768218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2494768218},{"type":"v1","addr":"172.21.15.61:6833","nonce":2494768218}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":35}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:38:56.821107+0000","modified":"2020-12-05T17:38:57.824754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/81037173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":81037173},{"type":"v1","addr":"172.21.15.32:6835","nonce":81037173}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-12-05T17:39:08.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.479+0000 7f9a12acc700 1 -- 172.21.15.32:0/2891894534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f99ec0553b0 msgr2=0x7f99ec057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:08.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.479+0000 7f9a12acc700 1 --2- 172.21.15.32:0/2891894534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f99ec0553b0 0x7f99ec057870 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f9a0001c000 tx=0x7f9a00015000).stop 2020-12-05T17:39:08.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.479+0000 7f9a12acc700 1 -- 172.21.15.32:0/2891894534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0c0ab710 msgr2=0x7f9a0c14d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:08.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.479+0000 7f9a12acc700 1 --2- 172.21.15.32:0/2891894534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0c0ab710 0x7f9a0c14d770 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7f9a0800c000 tx=0x7f9a08006dc0).stop 2020-12-05T17:39:08.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.479+0000 7f9a12acc700 1 -- 172.21.15.32:0/2891894534 shutdown_connections 2020-12-05T17:39:08.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.479+0000 7f9a12acc700 1 --2- 172.21.15.32:0/2891894534 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a0c147400 0x7f9a0c141080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:08.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.479+0000 7f9a12acc700 1 --2- 172.21.15.32:0/2891894534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0c0ab710 0x7f9a0c14d770 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:08.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.479+0000 7f9a12acc700 1 --2- 172.21.15.32:0/2891894534 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f99ec0553b0 0x7f99ec057870 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:08.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.479+0000 7f9a12acc700 1 --2- 172.21.15.32:0/2891894534 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a0c0b1fc0 0x7f9a0c140b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:08.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.479+0000 7f9a12acc700 1 -- 172.21.15.32:0/2891894534 >> 172.21.15.32:0/2891894534 conn(0x7f9a0c0b5530 msgr2=0x7f9a0c0b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:08.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.479+0000 7f9a12acc700 1 -- 172.21.15.32:0/2891894534 shutdown_connections 2020-12-05T17:39:08.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:08.479+0000 7f9a12acc700 1 -- 172.21.15.32:0/2891894534 wait complete. 2020-12-05T17:39:08.491 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 37 2020-12-05T17:39:08.498 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump inode 1 2020-12-05T17:39:08.614 INFO:teuthology.orchestra.run.smithi032.stdout:{"path":"/","ino":1,"rdev":0,"ctime":"2020-12-05T17:39:04.209809+0000","btime":"2020-12-05T17:38:56.824931+0000","mode":17407,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":4194304,"stripe_count":1,"object_size":4194304,"pool_id":16,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":0,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2020-12-05T17:38:56.824931+0000","atime":"0.000000","time_warp_seq":0,"change_attr":1,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"2020-12-05T17:39:04.209809+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"2020-12-05T17:38:56.824931+0000"},"version":3,"file_data_version":0,"xattr_version":1,"backtrace_version":3,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"request":1,"lock":3,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1,"authpin":1},"nref":9,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":1,"num_xlocks":0,"xlock_by":{}},"authlock":{"gather_set":[],"state":"xlockdone","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":1,"xlock_by":{}},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{"gather_set":[],"state":"sync","is_leased":false,"num_rdlocks":1,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"nestlock":{},"flocklock":{},"policylock":{},"states":["auth","dirty","dirtyparent"],"client_caps":[{"client_id":6742,"pending":"pLsXsFs","issued":"pLsXsFs","wanted":"-","last_sent":2},{"client_id":6786,"pending":"pLsXsFs","issued":"pLsXsFs","wanted":"-","last_sent":2}],"loner":-1,"want_loner":-1,"mds_caps_wanted":[]} 2020-12-05T17:39:08.616 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 0, "rctime": "2020-12-05T17:38:56.824931+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "0.000000", "auth_pins": 2, "auth_state": { "replicas": {} }, "authlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "state": "xlockdone", "xlock_by": {} }, "backtrace_version": 3, "btime": "2020-12-05T17:38:56.824931+0000", "change_attr": 1, "client_caps": [ { "client_id": 6742, "issued": "pLsXsFs", "last_sent": 2, "pending": "pLsXsFs", "wanted": "-" }, { "client_id": 6786, "issued": "pLsXsFs", "last_sent": 2, "pending": "pLsXsFs", "wanted": "-" } ], "client_ranges": [], "ctime": "2020-12-05T17:39:04.209809+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 0, "ino": 1, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 4194304, "pool_id": 16, "pool_ns": "", "stripe_count": 1, "stripe_unit": 4194304 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 17407, "mtime": "2020-12-05T17:38:56.824931+0000", "nestlock": {}, "nlink": 1, "nref": 9, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/", "pins": { "authpin": 1, "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "lock": 3, "request": 1 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2020-12-05T17:39:04.209809+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": { "gather_set": [], "is_leased": false, "num_rdlocks": 1, "num_wrlocks": 0, "num_xlocks": 0, "state": "sync", "xlock_by": {} }, "states": [ "auth", "dirty", "dirtyparent" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 0, "truncate_size": 18446744073709551615, "uid": 0, "version": 3, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } 2020-12-05T17:39:08.616 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:39:08.617 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:08.633 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:08.648 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:08.665 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:08.685 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:08.703 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:39:08.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:08.704 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:08.704 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:08.715 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:39:08.716 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:39:08.716 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi036.front.sepia.ceph.com... 2020-12-05T17:39:08.717 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:39:08.717 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:08.793 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[11166]: fuse finished with error 0 and tester_r 0 2020-12-05T17:39:08.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:08.795 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:08.795 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:08.842 INFO:teuthology.orchestra.run.smithi036.stdout:ext2/ext3 2020-12-05T17:39:08.843 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:39:08.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:39:08.911 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:08.919 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:08.919 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:08.920 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:08.928 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:39:08.929 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:39:08.930 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi002.front.sepia.ceph.com... 2020-12-05T17:39:08.930 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:39:08.931 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:09.003 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[10653]: fuse finished with error 0 and tester_r 0 2020-12-05T17:39:09.008 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:09.008 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:09.009 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:09.096 INFO:teuthology.orchestra.run.smithi002.stdout:ext2/ext3 2020-12-05T17:39:09.097 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:39:09.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:39:09.098 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:09.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:09.145 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:09.146 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:39:09.153 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:39:09.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:39:09.155 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2020-12-05T17:39:09.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:39:09.156 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:39:09.204 INFO:teuthology.orchestra.run.smithi053.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:39:09.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:39:09.206 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2020-12-05T17:39:09.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.327+0000 7f9ad3c9f700 1 Processor -- start 2020-12-05T17:39:09.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.327+0000 7f9ad3c9f700 1 -- start start 2020-12-05T17:39:09.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad3c9f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9acc0ad080 0x7f9acc142660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:09.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad3c9f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9acc0b6460 con 0x7f9acc0b0980 2020-12-05T17:39:09.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad3c9f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9acc0af390 con 0x7f9acc0b6080 2020-12-05T17:39:09.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad3c9f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0afc80 con 0x7f9acc0ad080 2020-12-05T17:39:09.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad223c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9acc0b0980 0x7f9acc13fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40346/0 (socket says 172.21.15.32:40346) 2020-12-05T17:39:09.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad223c700 1 -- 172.21.15.32:0/1205835358 learned_addr learned my addr 172.21.15.32:0/1205835358 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:09.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad123a700 1 --2- 172.21.15.32:0/1205835358 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9acc0ad080 0x7f9acc142660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:09.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad0a39700 1 -- 172.21.15.32:0/1205835358 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 501658363 0 0) 0x7f9acc0b6460 con 0x7f9acc0b0980 2020-12-05T17:39:09.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad0a39700 1 -- 172.21.15.32:0/1205835358 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ab4003230 con 0x7f9acc0b0980 2020-12-05T17:39:09.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad0a39700 1 -- 172.21.15.32:0/1205835358 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1649891961 0 0) 0x7f9acc0af390 con 0x7f9acc0b6080 2020-12-05T17:39:09.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad0a39700 1 -- 172.21.15.32:0/1205835358 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9acc0b6460 con 0x7f9acc0b6080 2020-12-05T17:39:09.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad123a700 1 -- 172.21.15.32:0/1205835358 >> v1:172.21.15.61:6789/0 conn(0x7f9acc0b6080 legacy=0x7f9acc141f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:09.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad123a700 1 -- 172.21.15.32:0/1205835358 >> v1:172.21.15.32:6789/0 conn(0x7f9acc0b0980 legacy=0x7f9acc13fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:09.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad123a700 1 -- 172.21.15.32:0/1205835358 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc0af630 con 0x7f9acc0ad080 2020-12-05T17:39:09.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad123a700 1 --2- 172.21.15.32:0/1205835358 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9acc0ad080 0x7f9acc142660 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f9abc002f30 tx=0x7f9abc006dc0).ready entity=mon.2 client_cookie=26aa0e38326f6f25 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:09.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad0a39700 1 -- 172.21.15.32:0/1205835358 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9abc009730 con 0x7f9acc0ad080 2020-12-05T17:39:09.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad0a39700 1 -- 172.21.15.32:0/1205835358 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9abc009890 con 0x7f9acc0ad080 2020-12-05T17:39:09.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad0a39700 1 -- 172.21.15.32:0/1205835358 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9abc009bf0 con 0x7f9acc0ad080 2020-12-05T17:39:09.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad3c9f700 1 -- 172.21.15.32:0/1205835358 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9acc0ad080 msgr2=0x7f9acc142660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:09.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad3c9f700 1 --2- 172.21.15.32:0/1205835358 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9acc0ad080 0x7f9acc142660 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f9abc002f30 tx=0x7f9abc006dc0).stop 2020-12-05T17:39:09.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad3c9f700 1 -- 172.21.15.32:0/1205835358 shutdown_connections 2020-12-05T17:39:09.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad3c9f700 1 --2- 172.21.15.32:0/1205835358 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9acc0ad080 0x7f9acc142660 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:09.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad3c9f700 1 -- 172.21.15.32:0/1205835358 >> 172.21.15.32:0/1205835358 conn(0x7f9acc069cb0 msgr2=0x7f9acc0604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:09.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad3c9f700 1 -- 172.21.15.32:0/1205835358 shutdown_connections 2020-12-05T17:39:09.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.331+0000 7f9ad3c9f700 1 -- 172.21.15.32:0/1205835358 wait complete. 2020-12-05T17:39:09.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9ad3c9f700 1 Processor -- start 2020-12-05T17:39:09.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9ad3c9f700 1 -- start start 2020-12-05T17:39:09.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9ad3c9f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9acc0ad080 0x7f9acc13d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:09.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9ad3c9f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9acc0b0980 0x7f9acc152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:09.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9ad3c9f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc0b6080 0x7f9acc14fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:09.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9ad3c9f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc10f150 con 0x7f9acc0b6080 2020-12-05T17:39:09.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9ad3c9f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc10f380 con 0x7f9acc0b0980 2020-12-05T17:39:09.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9ad3c9f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc14f7c0 con 0x7f9acc0ad080 2020-12-05T17:39:09.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9ad223c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc0b6080 0x7f9acc14fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:09.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9ad123a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9acc0b0980 0x7f9acc152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:09.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9ad223c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc0b6080 0x7f9acc14fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37704/0 (socket says 172.21.15.32:37704) 2020-12-05T17:39:09.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9ad223c700 1 -- 172.21.15.32:0/3997913195 learned_addr learned my addr 172.21.15.32:0/3997913195 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:09.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9ad1a3b700 1 --2- 172.21.15.32:0/3997913195 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9acc0ad080 0x7f9acc13d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:09.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9ad223c700 1 -- 172.21.15.32:0/3997913195 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9acc0ad080 msgr2=0x7f9acc13d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:09.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9ad223c700 1 --2- 172.21.15.32:0/3997913195 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9acc0ad080 0x7f9acc13d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:09.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9ad223c700 1 -- 172.21.15.32:0/3997913195 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9acc0b0980 msgr2=0x7f9acc152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:09.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9ad223c700 1 --2- 172.21.15.32:0/3997913195 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9acc0b0980 0x7f9acc152aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:09.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9ad223c700 1 -- 172.21.15.32:0/3997913195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc14f4f0 con 0x7f9acc0b6080 2020-12-05T17:39:09.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9ad223c700 1 --2- 172.21.15.32:0/3997913195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc0b6080 0x7f9acc14fc60 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0x7f9ac4001450 tx=0x7f9ac4007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:09.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9abaffd700 1 -- 172.21.15.32:0/3997913195 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ac4005b70 con 0x7f9acc0b6080 2020-12-05T17:39:09.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9abaffd700 1 -- 172.21.15.32:0/3997913195 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac4005cd0 con 0x7f9acc0b6080 2020-12-05T17:39:09.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9abaffd700 1 -- 172.21.15.32:0/3997913195 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ac4007300 con 0x7f9acc0b6080 2020-12-05T17:39:09.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9ad3c9f700 1 -- 172.21.15.32:0/3997913195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9acc063040 con 0x7f9acc0b6080 2020-12-05T17:39:09.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.335+0000 7f9ad3c9f700 1 -- 172.21.15.32:0/3997913195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9acc152f90 con 0x7f9acc0b6080 2020-12-05T17:39:09.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.339+0000 7f9abaffd700 1 -- 172.21.15.32:0/3997913195 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9ac4010030 con 0x7f9acc0b6080 2020-12-05T17:39:09.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.339+0000 7f9ad3c9f700 1 -- 172.21.15.32:0/3997913195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9acc14f980 con 0x7f9acc0b6080 2020-12-05T17:39:09.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.339+0000 7f9abaffd700 1 --2- 172.21.15.32:0/3997913195 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9aac0553b0 0x7f9aac057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:09.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.339+0000 7f9abaffd700 1 -- 172.21.15.32:0/3997913195 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9ac406e150 con 0x7f9acc0b6080 2020-12-05T17:39:09.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.339+0000 7f9ad1a3b700 1 --2- 172.21.15.32:0/3997913195 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9aac0553b0 0x7f9aac057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:09.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.339+0000 7f9ad1a3b700 1 --2- 172.21.15.32:0/3997913195 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9aac0553b0 0x7f9aac057870 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f9ac800d000 tx=0x7f9ac8007aa0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:09.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.343+0000 7f9abaffd700 1 -- 172.21.15.32:0/3997913195 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9ac4044160 con 0x7f9acc0b6080 2020-12-05T17:39:09.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.491+0000 7f9ad3c9f700 1 -- 172.21.15.32:0/3997913195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9acc057050 con 0x7f9acc0b6080 2020-12-05T17:39:09.503 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.499+0000 7f9abaffd700 1 -- 172.21.15.32:0/3997913195 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9ac4047780 con 0x7f9acc0b6080 2020-12-05T17:39:09.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.499+0000 7f9ad3c9f700 1 -- 172.21.15.32:0/3997913195 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9aac0553b0 msgr2=0x7f9aac057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:09.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.499+0000 7f9ad3c9f700 1 --2- 172.21.15.32:0/3997913195 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9aac0553b0 0x7f9aac057870 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f9ac800d000 tx=0x7f9ac8007aa0).stop 2020-12-05T17:39:09.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.499+0000 7f9ad3c9f700 1 -- 172.21.15.32:0/3997913195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc0b6080 msgr2=0x7f9acc14fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:09.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.499+0000 7f9ad3c9f700 1 --2- 172.21.15.32:0/3997913195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc0b6080 0x7f9acc14fc60 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0x7f9ac4001450 tx=0x7f9ac4007dd0).stop 2020-12-05T17:39:09.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.499+0000 7f9ad3c9f700 1 -- 172.21.15.32:0/3997913195 shutdown_connections 2020-12-05T17:39:09.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.499+0000 7f9ad3c9f700 1 --2- 172.21.15.32:0/3997913195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc0b6080 0x7f9acc14fc60 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:09.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.499+0000 7f9ad3c9f700 1 --2- 172.21.15.32:0/3997913195 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9aac0553b0 0x7f9aac057870 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.499+0000 7f9ad3c9f700 1 --2- 172.21.15.32:0/3997913195 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9acc0ad080 0x7f9acc13d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.499+0000 7f9ad3c9f700 1 --2- 172.21.15.32:0/3997913195 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9acc0b0980 0x7f9acc152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.499+0000 7f9ad3c9f700 1 -- 172.21.15.32:0/3997913195 >> 172.21.15.32:0/3997913195 conn(0x7f9acc069cb0 msgr2=0x7f9acc0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.499+0000 7f9ad3c9f700 1 -- 172.21.15.32:0/3997913195 shutdown_connections 2020-12-05T17:39:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.499+0000 7f9ad3c9f700 1 -- 172.21.15.32:0/3997913195 wait complete. 2020-12-05T17:39:09.525 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode' 2020-12-05T17:39:09.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e7fb0700 1 Processor -- start 2020-12-05T17:39:09.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e7fb0700 1 -- start start 2020-12-05T17:39:09.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e7fb0700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f01e00ad1b0 0x7f01e0143350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:09.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e7fb0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01e0143840 0x7f01e00aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:09.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e7fb0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e00ae8b0 con 0x7f01e00b6080 2020-12-05T17:39:09.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e7fb0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e00b0da0 con 0x7f01e0143840 2020-12-05T17:39:09.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e7fb0700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e00b6460 con 0x7f01e00ad1b0 2020-12-05T17:39:09.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e5d4c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f01e00b6080 0x7f01e0142c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40356/0 (socket says 172.21.15.32:40356) 2020-12-05T17:39:09.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e5d4c700 1 -- 172.21.15.32:0/3192846003 learned_addr learned my addr 172.21.15.32:0/3192846003 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:09.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e654d700 1 --2- 172.21.15.32:0/3192846003 >> v2:172.21.15.32:3300/0 conn(0x7f01e0143840 0x7f01e00aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:09.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e554b700 1 --2- 172.21.15.32:0/3192846003 >> v2:172.21.15.61:3300/0 conn(0x7f01e00ad1b0 0x7f01e0143350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:09.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e4d4a700 1 -- 172.21.15.32:0/3192846003 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 191043748 0 0) 0x7f01e00ae8b0 con 0x7f01e00b6080 2020-12-05T17:39:09.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e4d4a700 1 -- 172.21.15.32:0/3192846003 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01c0003230 con 0x7f01e00b6080 2020-12-05T17:39:09.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e654d700 1 -- 172.21.15.32:0/3192846003 >> v2:172.21.15.61:3300/0 conn(0x7f01e00ad1b0 msgr2=0x7f01e0143350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:09.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e654d700 1 --2- 172.21.15.32:0/3192846003 >> v2:172.21.15.61:3300/0 conn(0x7f01e00ad1b0 0x7f01e0143350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:09.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e654d700 1 -- 172.21.15.32:0/3192846003 >> v1:172.21.15.32:6789/0 conn(0x7f01e00b6080 legacy=0x7f01e0142c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:09.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e654d700 1 -- 172.21.15.32:0/3192846003 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e0140490 con 0x7f01e0143840 2020-12-05T17:39:09.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e654d700 1 --2- 172.21.15.32:0/3192846003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e0143840 0x7f01e00aece0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7f01d0002f30 tx=0x7f01d0006dc0).ready entity=mon.0 client_cookie=46297b710229d03b server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:09.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e4d4a700 1 -- 172.21.15.32:0/3192846003 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01d00096f0 con 0x7f01e0143840 2020-12-05T17:39:09.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e4d4a700 1 -- 172.21.15.32:0/3192846003 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d0009850 con 0x7f01e0143840 2020-12-05T17:39:09.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e4d4a700 1 -- 172.21.15.32:0/3192846003 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01d0009bb0 con 0x7f01e0143840 2020-12-05T17:39:09.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e7fb0700 1 -- 172.21.15.32:0/3192846003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e0143840 msgr2=0x7f01e00aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:09.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e7fb0700 1 --2- 172.21.15.32:0/3192846003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e0143840 0x7f01e00aece0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7f01d0002f30 tx=0x7f01d0006dc0).stop 2020-12-05T17:39:09.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e7fb0700 1 -- 172.21.15.32:0/3192846003 shutdown_connections 2020-12-05T17:39:09.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e7fb0700 1 --2- 172.21.15.32:0/3192846003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e0143840 0x7f01e00aece0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:09.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e7fb0700 1 --2- 172.21.15.32:0/3192846003 >> v2:172.21.15.61:3300/0 conn(0x7f01e00ad1b0 0x7f01e0143350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:09.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e7fb0700 1 -- 172.21.15.32:0/3192846003 >> 172.21.15.32:0/3192846003 conn(0x7f01e0069cb0 msgr2=0x7f01e00604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:09.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e7fb0700 1 -- 172.21.15.32:0/3192846003 shutdown_connections 2020-12-05T17:39:09.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.635+0000 7f01e7fb0700 1 -- 172.21.15.32:0/3192846003 wait complete. 2020-12-05T17:39:09.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.639+0000 7f01e7fb0700 1 Processor -- start 2020-12-05T17:39:09.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.639+0000 7f01e7fb0700 1 -- start start 2020-12-05T17:39:09.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.639+0000 7f01e7fb0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e00ad1b0 0x7f01e00b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:09.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.639+0000 7f01e7fb0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01e00b6080 0x7f01e01471a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:09.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.639+0000 7f01e7fb0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01e0143840 0x7f01e0147690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:09.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.639+0000 7f01e7fb0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e0154c00 con 0x7f01e00ad1b0 2020-12-05T17:39:09.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.639+0000 7f01e7fb0700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e0143dd0 con 0x7f01e0143840 2020-12-05T17:39:09.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.639+0000 7f01e7fb0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e0144070 con 0x7f01e00b6080 2020-12-05T17:39:09.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.639+0000 7f01e5d4c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e00ad1b0 0x7f01e00b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:09.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.639+0000 7f01e5d4c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e00ad1b0 0x7f01e00b16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37714/0 (socket says 172.21.15.32:37714) 2020-12-05T17:39:09.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.639+0000 7f01e5d4c700 1 -- 172.21.15.32:0/3803400663 learned_addr learned my addr 172.21.15.32:0/3803400663 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:09.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.639+0000 7f01e554b700 1 --2- 172.21.15.32:0/3803400663 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01e00b6080 0x7f01e01471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:09.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.639+0000 7f01e654d700 1 --2- 172.21.15.32:0/3803400663 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01e0143840 0x7f01e0147690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:09.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.639+0000 7f01e5d4c700 1 -- 172.21.15.32:0/3803400663 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01e00b6080 msgr2=0x7f01e01471a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:09.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.639+0000 7f01e5d4c700 1 --2- 172.21.15.32:0/3803400663 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01e00b6080 0x7f01e01471a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:09.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.639+0000 7f01e5d4c700 1 -- 172.21.15.32:0/3803400663 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01e0143840 msgr2=0x7f01e0147690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:09.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.639+0000 7f01e5d4c700 1 --2- 172.21.15.32:0/3803400663 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01e0143840 0x7f01e0147690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:09.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.639+0000 7f01e5d4c700 1 -- 172.21.15.32:0/3803400663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e00652a0 con 0x7f01e00ad1b0 2020-12-05T17:39:09.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.639+0000 7f01e5d4c700 1 --2- 172.21.15.32:0/3803400663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e00ad1b0 0x7f01e00b16f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7f01d8001450 tx=0x7f01d8007ed0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:09.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.643+0000 7f01deffd700 1 -- 172.21.15.32:0/3803400663 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01d8005b70 con 0x7f01e00ad1b0 2020-12-05T17:39:09.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.643+0000 7f01deffd700 1 -- 172.21.15.32:0/3803400663 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d8005cd0 con 0x7f01e00ad1b0 2020-12-05T17:39:09.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.643+0000 7f01e7fb0700 1 -- 172.21.15.32:0/3803400663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e0145b30 con 0x7f01e00ad1b0 2020-12-05T17:39:09.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.643+0000 7f01deffd700 1 -- 172.21.15.32:0/3803400663 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01d8007300 con 0x7f01e00ad1b0 2020-12-05T17:39:09.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.643+0000 7f01e7fb0700 1 -- 172.21.15.32:0/3803400663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e0147b80 con 0x7f01e00ad1b0 2020-12-05T17:39:09.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.643+0000 7f01e7fb0700 1 -- 172.21.15.32:0/3803400663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e0144230 con 0x7f01e00ad1b0 2020-12-05T17:39:09.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.643+0000 7f01deffd700 1 -- 172.21.15.32:0/3803400663 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f01d8010030 con 0x7f01e00ad1b0 2020-12-05T17:39:09.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.643+0000 7f01deffd700 1 --2- 172.21.15.32:0/3803400663 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f01c40553f0 0x7f01c40578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:09.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.643+0000 7f01e554b700 1 --2- 172.21.15.32:0/3803400663 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f01c40553f0 0x7f01c40578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:09.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.643+0000 7f01deffd700 1 -- 172.21.15.32:0/3803400663 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f01d806e290 con 0x7f01e00ad1b0 2020-12-05T17:39:09.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.647+0000 7f01e554b700 1 --2- 172.21.15.32:0/3803400663 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f01c40553f0 0x7f01c40578b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f01c800b000 tx=0x7f01c8008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:09.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.651+0000 7f01deffd700 1 -- 172.21.15.32:0/3803400663 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f01d80442c0 con 0x7f01e00ad1b0 2020-12-05T17:39:09.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:09.843+0000 7f01e7fb0700 1 -- 172.21.15.32:0/3803400663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]} v 0) v1 -- 0x7f01e0057050 con 0x7f01e00ad1b0 2020-12-05T17:39:10.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.595+0000 7f01deffd700 1 -- 172.21.15.32:0/3803400663 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]}]=0 v330) v1 ==== 126+0+0 (secure 0 0 0) 0x7f01d80478e0 con 0x7f01e00ad1b0 2020-12-05T17:39:10.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.599+0000 7f01e7fb0700 1 -- 172.21.15.32:0/3803400663 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f01c40553f0 msgr2=0x7f01c40578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:10.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.599+0000 7f01e7fb0700 1 --2- 172.21.15.32:0/3803400663 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f01c40553f0 0x7f01c40578b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f01c800b000 tx=0x7f01c8008040).stop 2020-12-05T17:39:10.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.599+0000 7f01e7fb0700 1 -- 172.21.15.32:0/3803400663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e00ad1b0 msgr2=0x7f01e00b16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:10.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.599+0000 7f01e7fb0700 1 --2- 172.21.15.32:0/3803400663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e00ad1b0 0x7f01e00b16f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7f01d8001450 tx=0x7f01d8007ed0).stop 2020-12-05T17:39:10.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.599+0000 7f01e7fb0700 1 -- 172.21.15.32:0/3803400663 shutdown_connections 2020-12-05T17:39:10.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.599+0000 7f01e7fb0700 1 --2- 172.21.15.32:0/3803400663 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01e0143840 0x7f01e0147690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:10.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.599+0000 7f01e7fb0700 1 --2- 172.21.15.32:0/3803400663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e00ad1b0 0x7f01e00b16f0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:10.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.599+0000 7f01e7fb0700 1 --2- 172.21.15.32:0/3803400663 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f01c40553f0 0x7f01c40578b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:10.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.599+0000 7f01e7fb0700 1 --2- 172.21.15.32:0/3803400663 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01e00b6080 0x7f01e01471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:10.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.599+0000 7f01e7fb0700 1 -- 172.21.15.32:0/3803400663 >> 172.21.15.32:0/3803400663 conn(0x7f01e0069cb0 msgr2=0x7f01e0142b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:10.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.599+0000 7f01e7fb0700 1 -- 172.21.15.32:0/3803400663 shutdown_connections 2020-12-05T17:39:10.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.599+0000 7f01e7fb0700 1 -- 172.21.15.32:0/3803400663 wait complete. 2020-12-05T17:39:10.624 INFO:tasks.cephfs_test_runner:test_dump_inode (tasks.cephfs.test_misc.TestMisc) ... ok 2020-12-05T17:39:10.625 INFO:tasks.cephfs_test_runner:Starting test: test_dump_inode_hexademical (tasks.cephfs.test_misc.TestMisc) 2020-12-05T17:39:10.625 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical' 2020-12-05T17:39:10.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.739+0000 7f32aaddb700 1 Processor -- start 2020-12-05T17:39:10.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.739+0000 7f32aaddb700 1 -- start start 2020-12-05T17:39:10.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32aaddb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32a40b7720 0x7f32a4061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:10.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32aaddb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f32a41403f0 0x7f32a4143e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:10.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32aaddb700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a4143a00 con 0x7f32a4144320 2020-12-05T17:39:10.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32aaddb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32a4140870 con 0x7f32a41403f0 2020-12-05T17:39:10.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32aaddb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a40b7b00 con 0x7f32a40b7720 2020-12-05T17:39:10.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32a95d8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f32a41403f0 0x7f32a4143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:10.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32a95d8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f32a41403f0 0x7f32a4143e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37722/0 (socket says 172.21.15.32:37722) 2020-12-05T17:39:10.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32a95d8700 1 -- 172.21.15.32:0/4200558803 learned_addr learned my addr 172.21.15.32:0/4200558803 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:10.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32a9dd9700 1 --2- 172.21.15.32:0/4200558803 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32a40b7720 0x7f32a4061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:10.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32a95d8700 1 -- 172.21.15.32:0/4200558803 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32a40b7720 msgr2=0x7f32a4061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:10.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32a95d8700 1 --2- 172.21.15.32:0/4200558803 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32a40b7720 0x7f32a4061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:10.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32a95d8700 1 -- 172.21.15.32:0/4200558803 >> v1:172.21.15.61:6789/0 conn(0x7f32a4144320 legacy=0x7f32a413d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:10.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32a95d8700 1 -- 172.21.15.32:0/4200558803 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a405fe60 con 0x7f32a41403f0 2020-12-05T17:39:10.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32a95d8700 1 --2- 172.21.15.32:0/4200558803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a41403f0 0x7f32a4143e30 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f32a0002f30 tx=0x7f32a0006e80).ready entity=mon.0 client_cookie=2fa1771d64a13b2f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:10.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32a8dd7700 1 -- 172.21.15.32:0/4200558803 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32a00097f0 con 0x7f32a41403f0 2020-12-05T17:39:10.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32a8dd7700 1 -- 172.21.15.32:0/4200558803 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a0009950 con 0x7f32a41403f0 2020-12-05T17:39:10.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32aaddb700 1 -- 172.21.15.32:0/4200558803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a41403f0 msgr2=0x7f32a4143e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:10.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32a8dd7700 1 -- 172.21.15.32:0/4200558803 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32a0009cb0 con 0x7f32a41403f0 2020-12-05T17:39:10.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32aaddb700 1 --2- 172.21.15.32:0/4200558803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a41403f0 0x7f32a4143e30 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f32a0002f30 tx=0x7f32a0006e80).stop 2020-12-05T17:39:10.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32aaddb700 1 -- 172.21.15.32:0/4200558803 shutdown_connections 2020-12-05T17:39:10.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32aaddb700 1 --2- 172.21.15.32:0/4200558803 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32a40b7720 0x7f32a4061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:10.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32aaddb700 1 --2- 172.21.15.32:0/4200558803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a41403f0 0x7f32a4143e30 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:10.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32aaddb700 1 -- 172.21.15.32:0/4200558803 >> 172.21.15.32:0/4200558803 conn(0x7f32a405bc20 msgr2=0x7f32a40b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:10.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32aaddb700 1 -- 172.21.15.32:0/4200558803 shutdown_connections 2020-12-05T17:39:10.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.743+0000 7f32aaddb700 1 -- 172.21.15.32:0/4200558803 wait complete. 2020-12-05T17:39:10.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f32aaddb700 1 Processor -- start 2020-12-05T17:39:10.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f32aaddb700 1 -- start start 2020-12-05T17:39:10.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f32aaddb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a40b7720 0x7f32a40b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:10.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f32aaddb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32a41403f0 0x7f32a4063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:10.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f32aaddb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32a4144320 0x7f32a4063530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:10.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f32aaddb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a4151830 con 0x7f32a40b7720 2020-12-05T17:39:10.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f32aaddb700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a4151ad0 con 0x7f32a4144320 2020-12-05T17:39:10.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f32aaddb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a4151d70 con 0x7f32a41403f0 2020-12-05T17:39:10.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f32a9dd9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a40b7720 0x7f32a40b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:10.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f32a9dd9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a40b7720 0x7f32a40b26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37728/0 (socket says 172.21.15.32:37728) 2020-12-05T17:39:10.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f32a9dd9700 1 -- 172.21.15.32:0/3640318787 learned_addr learned my addr 172.21.15.32:0/3640318787 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:10.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f32aa5da700 1 --2- 172.21.15.32:0/3640318787 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32a4144320 0x7f32a4063530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:10.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f32a95d8700 1 --2- 172.21.15.32:0/3640318787 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32a41403f0 0x7f32a4063040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:10.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f32a9dd9700 1 -- 172.21.15.32:0/3640318787 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32a41403f0 msgr2=0x7f32a4063040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:10.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f32a9dd9700 1 --2- 172.21.15.32:0/3640318787 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32a41403f0 0x7f32a4063040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:10.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f32a9dd9700 1 -- 172.21.15.32:0/3640318787 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32a4144320 msgr2=0x7f32a4063530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:10.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f32a9dd9700 1 --2- 172.21.15.32:0/3640318787 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32a4144320 0x7f32a4063530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:10.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f32a9dd9700 1 -- 172.21.15.32:0/3640318787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a41514c0 con 0x7f32a40b7720 2020-12-05T17:39:10.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f32aa5da700 1 --2- 172.21.15.32:0/3640318787 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32a4144320 0x7f32a4063530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:10.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f32a9dd9700 1 --2- 172.21.15.32:0/3640318787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a40b7720 0x7f32a40b26a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7f3298003070 tx=0x7f3298008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:10.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f3296ffd700 1 -- 172.21.15.32:0/3640318787 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3298009980 con 0x7f32a40b7720 2020-12-05T17:39:10.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f3296ffd700 1 -- 172.21.15.32:0/3640318787 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3298009ae0 con 0x7f32a40b7720 2020-12-05T17:39:10.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f3296ffd700 1 -- 172.21.15.32:0/3640318787 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3298009e40 con 0x7f32a40b7720 2020-12-05T17:39:10.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f32aaddb700 1 -- 172.21.15.32:0/3640318787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a4063a20 con 0x7f32a40b7720 2020-12-05T17:39:10.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f32aaddb700 1 -- 172.21.15.32:0/3640318787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a4152430 con 0x7f32a40b7720 2020-12-05T17:39:10.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.747+0000 7f32aaddb700 1 -- 172.21.15.32:0/3640318787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a4151f30 con 0x7f32a40b7720 2020-12-05T17:39:10.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.751+0000 7f3296ffd700 1 -- 172.21.15.32:0/3640318787 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3298013480 con 0x7f32a40b7720 2020-12-05T17:39:10.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.751+0000 7f3296ffd700 1 --2- 172.21.15.32:0/3640318787 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3284055330 0x7f32840577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:10.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.751+0000 7f3296ffd700 1 -- 172.21.15.32:0/3640318787 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f329806e650 con 0x7f32a40b7720 2020-12-05T17:39:10.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.751+0000 7f32a95d8700 1 --2- 172.21.15.32:0/3640318787 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3284055330 0x7f32840577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:10.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.751+0000 7f32a95d8700 1 --2- 172.21.15.32:0/3640318787 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3284055330 0x7f32840577f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f32a001a000 tx=0x7f32a0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:10.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.759+0000 7f3296ffd700 1 -- 172.21.15.32:0/3640318787 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3298044790 con 0x7f32a40b7720 2020-12-05T17:39:10.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:10.951+0000 7f32aaddb700 1 -- 172.21.15.32:0/3640318787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]} v 0) v1 -- 0x7f32a4057050 con 0x7f32a40b7720 2020-12-05T17:39:11.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.599+0000 7f3296ffd700 1 -- 172.21.15.32:0/3640318787 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]}]=0 v331) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3298047db0 con 0x7f32a40b7720 2020-12-05T17:39:11.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.603+0000 7f32aaddb700 1 -- 172.21.15.32:0/3640318787 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3284055330 msgr2=0x7f32840577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:11.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.603+0000 7f32aaddb700 1 --2- 172.21.15.32:0/3640318787 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3284055330 0x7f32840577f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f32a001a000 tx=0x7f32a0015000).stop 2020-12-05T17:39:11.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.603+0000 7f32aaddb700 1 -- 172.21.15.32:0/3640318787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a40b7720 msgr2=0x7f32a40b26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:11.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.603+0000 7f32aaddb700 1 --2- 172.21.15.32:0/3640318787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a40b7720 0x7f32a40b26a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7f3298003070 tx=0x7f3298008040).stop 2020-12-05T17:39:11.609 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.603+0000 7f32aaddb700 1 -- 172.21.15.32:0/3640318787 shutdown_connections 2020-12-05T17:39:11.609 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.603+0000 7f32aaddb700 1 --2- 172.21.15.32:0/3640318787 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32a4144320 0x7f32a4063530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:11.609 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.603+0000 7f32aaddb700 1 --2- 172.21.15.32:0/3640318787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a40b7720 0x7f32a40b26a0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:11.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.603+0000 7f32aaddb700 1 --2- 172.21.15.32:0/3640318787 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3284055330 0x7f32840577f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:11.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.603+0000 7f32aaddb700 1 --2- 172.21.15.32:0/3640318787 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32a41403f0 0x7f32a4063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:11.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.603+0000 7f32aaddb700 1 -- 172.21.15.32:0/3640318787 >> 172.21.15.32:0/3640318787 conn(0x7f32a405bc20 msgr2=0x7f32a413e360 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:11.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.603+0000 7f32aaddb700 1 -- 172.21.15.32:0/3640318787 shutdown_connections 2020-12-05T17:39:11.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.603+0000 7f32aaddb700 1 -- 172.21.15.32:0/3640318787 wait complete. 2020-12-05T17:39:11.628 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-12-05T17:39:11.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.751+0000 7f8adffff700 1 Processor -- start 2020-12-05T17:39:11.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.751+0000 7f8adffff700 1 -- start start 2020-12-05T17:39:11.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.751+0000 7f8adffff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f8ae00afd30 0x7f8ae00b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:11.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.751+0000 7f8adffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8ae00af5f0 0x7f8ae00aed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:11.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.751+0000 7f8adffff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae01468e0 con 0x7f8ae00b1dc0 2020-12-05T17:39:11.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.751+0000 7f8adffff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae0142c90 con 0x7f8ae00afd30 2020-12-05T17:39:11.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.751+0000 7f8adffff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae0146b10 con 0x7f8ae00af5f0 2020-12-05T17:39:11.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.755+0000 7f8ade7fc700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f8ae00afd30 0x7f8ae00b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:11.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.755+0000 7f8adf7fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8ae00af5f0 0x7f8ae00aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:11.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.755+0000 7f8ade7fc700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f8ae00afd30 0x7f8ae00b21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:57630/0 (socket says 172.21.15.32:57630) 2020-12-05T17:39:11.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.755+0000 7f8ade7fc700 1 -- 172.21.15.32:0/696768471 learned_addr learned my addr 172.21.15.32:0/696768471 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:11.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.755+0000 7f8ade7fc700 1 -- 172.21.15.32:0/696768471 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8ae00af5f0 msgr2=0x7f8ae00aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:11.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.755+0000 7f8ade7fc700 1 --2- 172.21.15.32:0/696768471 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8ae00af5f0 0x7f8ae00aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:11.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.755+0000 7f8ade7fc700 1 -- 172.21.15.32:0/696768471 >> v1:172.21.15.61:6789/0 conn(0x7f8ae00b1dc0 legacy=0x7f8ae00ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:11.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.755+0000 7f8ade7fc700 1 -- 172.21.15.32:0/696768471 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae0146db0 con 0x7f8ae00afd30 2020-12-05T17:39:11.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.755+0000 7f8ade7fc700 1 --2- 172.21.15.32:0/696768471 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ae00afd30 0x7f8ae00b21a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f8ac8002f30 tx=0x7f8ac8006e80).ready entity=mon.1 client_cookie=81dc1503a8612e42 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:11.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.755+0000 7f8addffb700 1 -- 172.21.15.32:0/696768471 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ac80097f0 con 0x7f8ae00afd30 2020-12-05T17:39:11.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.755+0000 7f8addffb700 1 -- 172.21.15.32:0/696768471 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac8009950 con 0x7f8ae00afd30 2020-12-05T17:39:11.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.755+0000 7f8addffb700 1 -- 172.21.15.32:0/696768471 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ac8009cb0 con 0x7f8ae00afd30 2020-12-05T17:39:11.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.755+0000 7f8adffff700 1 -- 172.21.15.32:0/696768471 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ae00afd30 msgr2=0x7f8ae00b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:11.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.755+0000 7f8adffff700 1 --2- 172.21.15.32:0/696768471 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ae00afd30 0x7f8ae00b21a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f8ac8002f30 tx=0x7f8ac8006e80).stop 2020-12-05T17:39:11.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.755+0000 7f8adffff700 1 -- 172.21.15.32:0/696768471 shutdown_connections 2020-12-05T17:39:11.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.755+0000 7f8adffff700 1 --2- 172.21.15.32:0/696768471 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8ae00af5f0 0x7f8ae00aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:11.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.755+0000 7f8adffff700 1 --2- 172.21.15.32:0/696768471 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ae00afd30 0x7f8ae00b21a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:11.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.755+0000 7f8adffff700 1 -- 172.21.15.32:0/696768471 >> 172.21.15.32:0/696768471 conn(0x7f8ae00620b0 msgr2=0x7f8ae0062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:11.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.755+0000 7f8adffff700 1 -- 172.21.15.32:0/696768471 shutdown_connections 2020-12-05T17:39:11.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.755+0000 7f8adffff700 1 -- 172.21.15.32:0/696768471 wait complete. 2020-12-05T17:39:11.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.759+0000 7f8adffff700 1 Processor -- start 2020-12-05T17:39:11.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.759+0000 7f8adffff700 1 -- start start 2020-12-05T17:39:11.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.759+0000 7f8adffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ae00af5f0 0x7f8ae00b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:11.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.759+0000 7f8adffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8ae00afd30 0x7f8ae013f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:11.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.759+0000 7f8adffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae00b1dc0 0x7f8ae013f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:11.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.759+0000 7f8adffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae013fc50 con 0x7f8ae00b1dc0 2020-12-05T17:39:11.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.759+0000 7f8adffff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae0145920 con 0x7f8ae00af5f0 2020-12-05T17:39:11.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.759+0000 7f8adffff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae0145bc0 con 0x7f8ae00afd30 2020-12-05T17:39:11.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.759+0000 7f8adf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae00b1dc0 0x7f8ae013f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:11.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.759+0000 7f8adeffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ae00af5f0 0x7f8ae00b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:11.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.759+0000 7f8adf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae00b1dc0 0x7f8ae013f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37746/0 (socket says 172.21.15.32:37746) 2020-12-05T17:39:11.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.759+0000 7f8ade7fc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8ae00afd30 0x7f8ae013f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:11.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.759+0000 7f8adf7fe700 1 -- 172.21.15.32:0/579155612 learned_addr learned my addr 172.21.15.32:0/579155612 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:11.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.759+0000 7f8ade7fc700 1 -- 172.21.15.32:0/579155612 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ae00af5f0 msgr2=0x7f8ae00b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:11.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.759+0000 7f8ade7fc700 1 --2- 172.21.15.32:0/579155612 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ae00af5f0 0x7f8ae00b04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:11.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.759+0000 7f8ade7fc700 1 -- 172.21.15.32:0/579155612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae00b1dc0 msgr2=0x7f8ae013f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:11.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.759+0000 7f8ade7fc700 1 --2- 172.21.15.32:0/579155612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae00b1dc0 0x7f8ae013f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:11.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.759+0000 7f8ade7fc700 1 -- 172.21.15.32:0/579155612 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae0145e60 con 0x7f8ae00afd30 2020-12-05T17:39:11.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.759+0000 7f8adeffd700 1 --2- 172.21.15.32:0/579155612 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ae00af5f0 0x7f8ae00b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:11.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.759+0000 7f8ade7fc700 1 --2- 172.21.15.32:0/579155612 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8ae00afd30 0x7f8ae013f270 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f8ac8002f30 tx=0x7f8ac8015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:11.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.763+0000 7f8ad7fff700 1 -- 172.21.15.32:0/579155612 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ac8011b40 con 0x7f8ae00afd30 2020-12-05T17:39:11.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.763+0000 7f8ad7fff700 1 -- 172.21.15.32:0/579155612 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac8011ca0 con 0x7f8ae00afd30 2020-12-05T17:39:11.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.763+0000 7f8adffff700 1 -- 172.21.15.32:0/579155612 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae0063a10 con 0x7f8ae00afd30 2020-12-05T17:39:11.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.763+0000 7f8ad7fff700 1 -- 172.21.15.32:0/579155612 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ac801e460 con 0x7f8ae00afd30 2020-12-05T17:39:11.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.763+0000 7f8adffff700 1 -- 172.21.15.32:0/579155612 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae0144520 con 0x7f8ae00afd30 2020-12-05T17:39:11.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.763+0000 7f8adffff700 1 -- 172.21.15.32:0/579155612 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae0144c40 con 0x7f8ae00afd30 2020-12-05T17:39:11.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.763+0000 7f8ad7fff700 1 -- 172.21.15.32:0/579155612 <== mon.2 v2:172.21.15.61:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8ac8025030 con 0x7f8ae00afd30 2020-12-05T17:39:11.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.763+0000 7f8ad7fff700 1 --2- 172.21.15.32:0/579155612 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8ac40554e0 0x7f8ac40579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.763+0000 7f8adeffd700 1 --2- 172.21.15.32:0/579155612 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8ac40554e0 0x7f8ac40579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.763+0000 7f8ad7fff700 1 -- 172.21.15.32:0/579155612 <== mon.2 v2:172.21.15.61:3301/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8ac806f430 con 0x7f8ae00afd30 2020-12-05T17:39:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.767+0000 7f8adeffd700 1 --2- 172.21.15.32:0/579155612 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8ac40554e0 0x7f8ac40579a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f8ad800d000 tx=0x7f8ad8009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:11.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.771+0000 7f8ad7fff700 1 -- 172.21.15.32:0/579155612 <== mon.2 v2:172.21.15.61:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8ae0144c40 con 0x7f8ae00afd30 2020-12-05T17:39:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.915+0000 7f8adffff700 1 -- 172.21.15.32:0/579155612 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f8ae0144c40 con 0x7f8ae00afd30 2020-12-05T17:39:11.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.927+0000 7f8ad7fff700 1 -- 172.21.15.32:0/579155612 <== mon.2 v2:172.21.15.61:3301/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8ae0144c40 con 0x7f8ae00afd30 2020-12-05T17:39:11.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.931+0000 7f8adffff700 1 -- 172.21.15.32:0/579155612 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8ac40554e0 msgr2=0x7f8ac40579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:11.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.931+0000 7f8adffff700 1 --2- 172.21.15.32:0/579155612 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8ac40554e0 0x7f8ac40579a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f8ad800d000 tx=0x7f8ad8009040).stop 2020-12-05T17:39:11.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.931+0000 7f8adffff700 1 -- 172.21.15.32:0/579155612 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8ae00afd30 msgr2=0x7f8ae013f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:11.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.931+0000 7f8adffff700 1 --2- 172.21.15.32:0/579155612 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8ae00afd30 0x7f8ae013f270 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f8ac8002f30 tx=0x7f8ac8015000).stop 2020-12-05T17:39:11.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.931+0000 7f8adffff700 1 -- 172.21.15.32:0/579155612 shutdown_connections 2020-12-05T17:39:11.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.931+0000 7f8adffff700 1 --2- 172.21.15.32:0/579155612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae00b1dc0 0x7f8ae013f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:11.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.931+0000 7f8adffff700 1 --2- 172.21.15.32:0/579155612 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ae00af5f0 0x7f8ae00b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:11.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.931+0000 7f8adffff700 1 --2- 172.21.15.32:0/579155612 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8ac40554e0 0x7f8ac40579a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:11.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.931+0000 7f8adffff700 1 --2- 172.21.15.32:0/579155612 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8ae00afd30 0x7f8ae013f270 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:11.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.931+0000 7f8adffff700 1 -- 172.21.15.32:0/579155612 >> 172.21.15.32:0/579155612 conn(0x7f8ae00620b0 msgr2=0x7f8ae00b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:11.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.931+0000 7f8adffff700 1 -- 172.21.15.32:0/579155612 shutdown_connections 2020-12-05T17:39:11.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:11.931+0000 7f8adffff700 1 -- 172.21.15.32:0/579155612 wait complete. 2020-12-05T17:39:11.952 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:39:11.953 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:11.970 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:11.984 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:12.002 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:12.019 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:12.037 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:39:12.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:12.038 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:12.039 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:12.054 INFO:teuthology.orchestra.run.smithi036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-12-05T17:39:12.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:39:12.055 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-12-05T17:39:12.056 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:12.056 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:12.057 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:12.063 INFO:teuthology.orchestra.run.smithi002.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-12-05T17:39:12.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:39:12.065 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-12-05T17:39:12.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:12.066 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:12.251 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:39:12.263 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:39:12.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:39:12.264 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2020-12-05T17:39:12.264 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-12-05T17:39:12.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.407+0000 7fc04d7f6700 1 Processor -- start 2020-12-05T17:39:12.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.407+0000 7fc04d7f6700 1 -- start start 2020-12-05T17:39:12.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.407+0000 7fc04d7f6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0480b56d0 0x7fc0480aef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:12.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.407+0000 7fc04d7f6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0481420f0 con 0x7fc0480b1870 2020-12-05T17:39:12.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.407+0000 7fc04d7f6700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0480b0380 con 0x7fc0480b52f0 2020-12-05T17:39:12.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.407+0000 7fc04d7f6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0480ad4f0 con 0x7fc0480b56d0 2020-12-05T17:39:12.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.407+0000 7fc04cff5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0480b56d0 0x7fc0480aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:12.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.407+0000 7fc0477fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc0480b1870 0x7fc0480ae860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40402/0 (socket says 172.21.15.32:40402) 2020-12-05T17:39:12.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.407+0000 7fc04cff5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0480b56d0 0x7fc0480aef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37752/0 (socket says 172.21.15.32:37752) 2020-12-05T17:39:12.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.407+0000 7fc0477fe700 1 -- 172.21.15.32:0/4125993801 learned_addr learned my addr 172.21.15.32:0/4125993801 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:12.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.407+0000 7fc04cff5700 1 -- 172.21.15.32:0/4125993801 >> v1:172.21.15.61:6789/0 conn(0x7fc0480b52f0 legacy=0x7fc048138810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:12.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.407+0000 7fc046ffd700 1 -- 172.21.15.32:0/4125993801 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1658321048 0 0) 0x7fc0481420f0 con 0x7fc0480b1870 2020-12-05T17:39:12.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.407+0000 7fc04cff5700 1 -- 172.21.15.32:0/4125993801 >> v1:172.21.15.32:6789/0 conn(0x7fc0480b1870 legacy=0x7fc0480ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:12.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.407+0000 7fc04cff5700 1 -- 172.21.15.32:0/4125993801 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0480b0620 con 0x7fc0480b56d0 2020-12-05T17:39:12.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.407+0000 7fc04cff5700 1 --2- 172.21.15.32:0/4125993801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0480b56d0 0x7fc0480aef70 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7fc038001110 tx=0x7fc04000a360).ready entity=mon.0 client_cookie=f8c236bb4a2912c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:12.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.407+0000 7fc046ffd700 1 -- 172.21.15.32:0/4125993801 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc04000ce40 con 0x7fc0480b56d0 2020-12-05T17:39:12.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.407+0000 7fc046ffd700 1 -- 172.21.15.32:0/4125993801 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04000f470 con 0x7fc0480b56d0 2020-12-05T17:39:12.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.407+0000 7fc046ffd700 1 -- 172.21.15.32:0/4125993801 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc04000f7d0 con 0x7fc0480b56d0 2020-12-05T17:39:12.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.407+0000 7fc04d7f6700 1 -- 172.21.15.32:0/4125993801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0480b56d0 msgr2=0x7fc0480aef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:12.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.407+0000 7fc04d7f6700 1 --2- 172.21.15.32:0/4125993801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0480b56d0 0x7fc0480aef70 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7fc038001110 tx=0x7fc04000a360).stop 2020-12-05T17:39:12.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc04d7f6700 1 -- 172.21.15.32:0/4125993801 shutdown_connections 2020-12-05T17:39:12.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc04d7f6700 1 --2- 172.21.15.32:0/4125993801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0480b56d0 0x7fc0480aef70 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:12.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc04d7f6700 1 -- 172.21.15.32:0/4125993801 >> 172.21.15.32:0/4125993801 conn(0x7fc04805bbb0 msgr2=0x7fc048062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:12.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc04d7f6700 1 -- 172.21.15.32:0/4125993801 shutdown_connections 2020-12-05T17:39:12.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc04d7f6700 1 -- 172.21.15.32:0/4125993801 wait complete. 2020-12-05T17:39:12.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc04d7f6700 1 Processor -- start 2020-12-05T17:39:12.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc04d7f6700 1 -- start start 2020-12-05T17:39:12.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc04d7f6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc0480b1870 0x7fc0480aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:12.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc04d7f6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc0480b52f0 0x7fc04813e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:12.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc04d7f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0480b56d0 0x7fc04813ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:12.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc04d7f6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc048063490 con 0x7fc0480b56d0 2020-12-05T17:39:12.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc04d7f6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0480636c0 con 0x7fc0480b52f0 2020-12-05T17:39:12.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc04d7f6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc048065110 con 0x7fc0480b1870 2020-12-05T17:39:12.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc04cff5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0480b56d0 0x7fc04813ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:12.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc04cff5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0480b56d0 0x7fc04813ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37760/0 (socket says 172.21.15.32:37760) 2020-12-05T17:39:12.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc04cff5700 1 -- 172.21.15.32:0/2769458594 learned_addr learned my addr 172.21.15.32:0/2769458594 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:12.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc0477fe700 1 --2- 172.21.15.32:0/2769458594 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc0480b52f0 0x7fc04813e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:12.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc047fff700 1 --2- 172.21.15.32:0/2769458594 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc0480b1870 0x7fc0480aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:12.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc04cff5700 1 -- 172.21.15.32:0/2769458594 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc0480b1870 msgr2=0x7fc0480aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:12.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc04cff5700 1 --2- 172.21.15.32:0/2769458594 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc0480b1870 0x7fc0480aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:12.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc04cff5700 1 -- 172.21.15.32:0/2769458594 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc0480b52f0 msgr2=0x7fc04813e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:12.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc04cff5700 1 --2- 172.21.15.32:0/2769458594 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc0480b52f0 0x7fc04813e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:12.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc04cff5700 1 -- 172.21.15.32:0/2769458594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0480af4d0 con 0x7fc0480b56d0 2020-12-05T17:39:12.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.411+0000 7fc0477fe700 1 --2- 172.21.15.32:0/2769458594 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc0480b52f0 0x7fc04813e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:12.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.415+0000 7fc04cff5700 1 --2- 172.21.15.32:0/2769458594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0480b56d0 0x7fc04813ecf0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7fc038001110 tx=0x7fc040018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:12.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.415+0000 7fc0457fa700 1 -- 172.21.15.32:0/2769458594 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc040021610 con 0x7fc0480b56d0 2020-12-05T17:39:12.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.415+0000 7fc0457fa700 1 -- 172.21.15.32:0/2769458594 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc040021770 con 0x7fc0480b56d0 2020-12-05T17:39:12.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.415+0000 7fc04d7f6700 1 -- 172.21.15.32:0/2769458594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc048064c70 con 0x7fc0480b56d0 2020-12-05T17:39:12.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.415+0000 7fc0457fa700 1 -- 172.21.15.32:0/2769458594 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc040021ad0 con 0x7fc0480b56d0 2020-12-05T17:39:12.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.415+0000 7fc04d7f6700 1 -- 172.21.15.32:0/2769458594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc048063ea0 con 0x7fc0480b56d0 2020-12-05T17:39:12.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.415+0000 7fc04d7f6700 1 -- 172.21.15.32:0/2769458594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc048064f50 con 0x7fc0480b56d0 2020-12-05T17:39:12.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.415+0000 7fc0457fa700 1 -- 172.21.15.32:0/2769458594 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc040028030 con 0x7fc0480b56d0 2020-12-05T17:39:12.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.415+0000 7fc0457fa700 1 --2- 172.21.15.32:0/2769458594 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc02c055330 0x7fc02c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:12.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.415+0000 7fc0457fa700 1 -- 172.21.15.32:0/2769458594 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc0400718d0 con 0x7fc0480b56d0 2020-12-05T17:39:12.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.415+0000 7fc047fff700 1 --2- 172.21.15.32:0/2769458594 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc02c055330 0x7fc02c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:12.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.415+0000 7fc047fff700 1 --2- 172.21.15.32:0/2769458594 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc02c055330 0x7fc02c0577f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fc03800b000 tx=0x7fc038009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:12.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.423+0000 7fc0457fa700 1 -- 172.21.15.32:0/2769458594 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc040047ab0 con 0x7fc0480b56d0 2020-12-05T17:39:12.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.567+0000 7fc04d7f6700 1 -- 172.21.15.32:0/2769458594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fc048057050 con 0x7fc0480b56d0 2020-12-05T17:39:12.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.571+0000 7fc0457fa700 1 -- 172.21.15.32:0/2769458594 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v83) v1 ==== 81+0+34040 (secure 0 0 0) 0x7fc04004b090 con 0x7fc0480b56d0 2020-12-05T17:39:12.576 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:12.577 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:39:12.577 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 83, 2020-12-05T17:39:12.578 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", 2020-12-05T17:39:12.578 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2020-12-05T17:31:20.975238+0000", 2020-12-05T17:39:12.579 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2020-12-05T17:38:57.823795+0000", 2020-12-05T17:39:12.579 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:39:12.580 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:39:12.580 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-12-05T17:39:12.581 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2020-12-05T17:39:12.581 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2020-12-05T17:39:12.582 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2020-12-05T17:39:12.582 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2020-12-05T17:39:12.582 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2020-12-05T17:39:12.582 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2020-12-05T17:39:12.583 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:39:12.583 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2020-12-05T17:39:12.583 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2020-12-05T17:39:12.583 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-12-05T17:39:12.584 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2020-12-05T17:39:12.584 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2020-12-05T17:39:12.584 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 16, 2020-12-05T17:39:12.584 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2020-12-05T17:39:12.584 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2020-12-05T17:39:12.585 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2020-12-05T17:39:12.585 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2020-12-05T17:39:12.585 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2020-12-05T17:39:12.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.586 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2020-12-05T17:39:12.586 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2020-12-05T17:39:12.586 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:24.858732+0000", 2020-12-05T17:39:12.586 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:39:12.587 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:39:12.587 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:39:12.587 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:39:12.588 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:39:12.588 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:39:12.589 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:39:12.589 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:39:12.589 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2020-12-05T17:39:12.589 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2020-12-05T17:39:12.590 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2020-12-05T17:39:12.591 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2020-12-05T17:39:12.591 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2020-12-05T17:39:12.592 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:39:12.592 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:39:12.592 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:39:12.593 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:39:12.593 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:39:12.593 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:39:12.593 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:39:12.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.594 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "16", 2020-12-05T17:39:12.594 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:39:12.594 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:39:12.594 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:39:12.595 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:39:12.595 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:39:12.595 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:39:12.595 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:39:12.595 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:39:12.596 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:39:12.596 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:39:12.596 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:39:12.596 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:39:12.597 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:39:12.597 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:39:12.597 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:39:12.597 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:39:12.597 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:39:12.598 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:39:12.598 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:39:12.598 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:39:12.598 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:39:12.598 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:39:12.599 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:39:12.599 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:39:12.599 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:39:12.599 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:39:12.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.600 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:39:12.600 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:39:12.600 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:39:12.600 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:39:12.601 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:39:12.601 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:39:12.601 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:39:12.601 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:39:12.602 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:39:12.602 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:39:12.602 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:39:12.602 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:39:12.602 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2020-12-05T17:39:12.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.603 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:39:12.603 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2020-12-05T17:39:12.603 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.604 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2020-12-05T17:39:12.604 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2020-12-05T17:39:12.604 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:33.305595+0000", 2020-12-05T17:39:12.605 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:39:12.605 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:39:12.605 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:39:12.606 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:39:12.606 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:39:12.606 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:39:12.606 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:39:12.606 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:39:12.607 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:39:12.607 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:39:12.607 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:39:12.607 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:39:12.608 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:39:12.608 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:39:12.609 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:39:12.609 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:39:12.609 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:39:12.609 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:39:12.610 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:39:12.610 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:39:12.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.610 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "19", 2020-12-05T17:39:12.610 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:39:12.611 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:39:12.611 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:39:12.611 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:39:12.611 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:39:12.612 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:39:12.612 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:39:12.612 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:39:12.612 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:39:12.613 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:39:12.613 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:39:12.613 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:39:12.613 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:39:12.613 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:39:12.614 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:39:12.614 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:39:12.614 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:39:12.615 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:39:12.615 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:39:12.615 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:39:12.615 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:39:12.616 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:39:12.616 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:39:12.616 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:39:12.616 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:39:12.616 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:39:12.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.617 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:39:12.617 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:39:12.617 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:39:12.618 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:39:12.618 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:39:12.618 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:39:12.618 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:39:12.618 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:39:12.619 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:39:12.619 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:39:12.619 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:39:12.619 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:39:12.619 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:39:12.620 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2020-12-05T17:39:12.620 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.621 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 15, 2020-12-05T17:39:12.621 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2020-12-05T17:39:12.621 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:38:54.508932+0000", 2020-12-05T17:39:12.622 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:39:12.622 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:39:12.622 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:39:12.622 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:39:12.623 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:39:12.623 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:39:12.623 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:39:12.623 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:39:12.623 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:39:12.624 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:39:12.624 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:39:12.624 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:39:12.624 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:39:12.625 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:39:12.625 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:39:12.625 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:39:12.625 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:39:12.625 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:39:12.626 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:39:12.626 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:39:12.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.626 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "82", 2020-12-05T17:39:12.627 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:39:12.627 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:39:12.627 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:39:12.627 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:39:12.627 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:39:12.628 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:39:12.628 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:39:12.628 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:39:12.628 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:39:12.629 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:39:12.629 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:39:12.629 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:39:12.629 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:39:12.629 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:39:12.630 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:39:12.630 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:39:12.630 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:39:12.630 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:39:12.630 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:39:12.631 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:39:12.631 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:39:12.631 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:39:12.631 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:39:12.632 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:39:12.632 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:39:12.632 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:39:12.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.632 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:39:12.633 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:39:12.633 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:39:12.633 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:39:12.633 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:39:12.633 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:39:12.634 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:39:12.634 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:39:12.634 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:39:12.634 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:39:12.635 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:39:12.635 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:39:12.635 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2020-12-05T17:39:12.635 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2020-12-05T17:39:12.635 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2020-12-05T17:39:12.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.636 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:39:12.636 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:39:12.636 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2020-12-05T17:39:12.636 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.637 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.637 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 16, 2020-12-05T17:39:12.638 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2020-12-05T17:39:12.638 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:38:55.805478+0000", 2020-12-05T17:39:12.638 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:39:12.638 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:39:12.638 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:39:12.639 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:39:12.639 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:39:12.639 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:39:12.639 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:39:12.640 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:39:12.640 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:39:12.640 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:39:12.640 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:39:12.640 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:39:12.641 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:39:12.641 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:39:12.641 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:39:12.641 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:39:12.642 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:39:12.642 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:39:12.642 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:39:12.642 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:39:12.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.643 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "83", 2020-12-05T17:39:12.643 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:39:12.643 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:39:12.643 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:39:12.643 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:39:12.644 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:39:12.644 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:39:12.644 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:39:12.644 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:39:12.644 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:39:12.645 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:39:12.645 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:39:12.645 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:39:12.645 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:39:12.646 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:39:12.646 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:39:12.646 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:39:12.646 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:39:12.646 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:39:12.647 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:39:12.647 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:39:12.647 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:39:12.647 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:39:12.648 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:39:12.648 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:39:12.648 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:39:12.648 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:39:12.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.649 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:39:12.649 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:39:12.649 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:39:12.649 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:39:12.650 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:39:12.650 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:39:12.650 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:39:12.650 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:39:12.650 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:39:12.651 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:39:12.651 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:39:12.651 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:39:12.651 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:39:12.652 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:39:12.652 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2020-12-05T17:39:12.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.653 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:39:12.653 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2020-12-05T17:39:12.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.653 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:39:12.654 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "b941127b-0a49-4e06-8d11-17794ff8b013", 2020-12-05T17:39:12.654 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:39:12.654 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:39:12.654 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:39:12.654 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:39:12.655 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:39:12.655 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:39:12.655 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:39:12.655 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 81, 2020-12-05T17:39:12.656 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:39:12.656 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:39:12.656 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:39:12.656 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.657 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.657 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2020-12-05T17:39:12.657 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:39:12.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.658 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.658 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2020-12-05T17:39:12.658 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:39:12.658 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.659 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.659 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:39:12.659 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.660 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.660 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2020-12-05T17:39:12.660 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:39:12.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.661 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.661 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2020-12-05T17:39:12.662 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:39:12.662 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.662 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.663 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:39:12.663 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.663 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.663 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2020-12-05T17:39:12.664 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:39:12.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.664 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.664 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2020-12-05T17:39:12.665 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:39:12.665 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.665 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.666 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:39:12.666 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.666 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.666 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2020-12-05T17:39:12.667 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:39:12.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.667 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.668 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2020-12-05T17:39:12.668 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:39:12.668 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.668 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.669 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6803/11526", 2020-12-05T17:39:12.669 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6805/11526", 2020-12-05T17:39:12.669 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6809/11526", 2020-12-05T17:39:12.669 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6807/11526", 2020-12-05T17:39:12.670 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:39:12.670 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:39:12.670 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:39:12.670 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.671 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:39:12.671 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "01bd7ff5-a86b-4e11-ba1e-484c309c0fba", 2020-12-05T17:39:12.671 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:39:12.672 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:39:12.672 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:39:12.672 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:39:12.672 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:39:12.672 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:39:12.673 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:39:12.673 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 80, 2020-12-05T17:39:12.673 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:39:12.673 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:39:12.673 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:39:12.674 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.674 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.674 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2020-12-05T17:39:12.675 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:39:12.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.675 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.676 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2020-12-05T17:39:12.676 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:39:12.676 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.676 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.677 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:39:12.677 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.677 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.677 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2020-12-05T17:39:12.678 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:39:12.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.678 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.679 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2020-12-05T17:39:12.679 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:39:12.679 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.679 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.680 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:39:12.680 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.680 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.681 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2020-12-05T17:39:12.681 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:39:12.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.681 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.682 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2020-12-05T17:39:12.682 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:39:12.682 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.682 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.683 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:39:12.684 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.684 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.684 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2020-12-05T17:39:12.685 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:39:12.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.685 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.686 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2020-12-05T17:39:12.686 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:39:12.686 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.686 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.687 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6819/11538", 2020-12-05T17:39:12.687 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6821/11538", 2020-12-05T17:39:12.687 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6825/11538", 2020-12-05T17:39:12.687 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6823/11538", 2020-12-05T17:39:12.688 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:39:12.688 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:39:12.688 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:39:12.688 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.689 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:39:12.689 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "94736ef5-64c2-4a97-b963-bd38cddd9c52", 2020-12-05T17:39:12.689 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:39:12.690 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:39:12.690 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:39:12.690 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:39:12.690 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:39:12.691 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:39:12.691 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:39:12.691 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 81, 2020-12-05T17:39:12.691 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:39:12.691 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:39:12.692 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:39:12.692 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.692 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.692 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2020-12-05T17:39:12.693 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:39:12.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.693 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.694 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2020-12-05T17:39:12.694 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:39:12.694 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.695 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.695 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:39:12.696 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.696 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.696 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2020-12-05T17:39:12.697 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:39:12.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.697 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.697 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2020-12-05T17:39:12.698 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:39:12.698 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.698 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.699 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:39:12.699 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.699 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.699 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2020-12-05T17:39:12.700 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:39:12.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.700 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.701 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2020-12-05T17:39:12.701 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:39:12.701 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.701 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.702 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:39:12.702 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.702 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.703 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2020-12-05T17:39:12.703 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:39:12.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.704 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.704 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2020-12-05T17:39:12.704 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:39:12.704 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.704 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.705 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6811/11521", 2020-12-05T17:39:12.705 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6813/11521", 2020-12-05T17:39:12.706 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6817/11521", 2020-12-05T17:39:12.706 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6815/11521", 2020-12-05T17:39:12.706 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:39:12.706 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:39:12.707 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:39:12.707 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.707 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:39:12.708 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "69a1f3ff-fd25-43df-b60c-e32a89c35234", 2020-12-05T17:39:12.708 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:39:12.708 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:39:12.708 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:39:12.709 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:39:12.709 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:39:12.709 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:39:12.709 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:39:12.709 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 81, 2020-12-05T17:39:12.710 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:39:12.710 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:39:12.710 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:39:12.710 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.711 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.711 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2020-12-05T17:39:12.711 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:39:12.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.712 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.712 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2020-12-05T17:39:12.712 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:39:12.712 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.713 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.713 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:39:12.713 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.714 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.714 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2020-12-05T17:39:12.714 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:39:12.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.715 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.716 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2020-12-05T17:39:12.716 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:39:12.716 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.716 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.717 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:39:12.717 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.717 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.717 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6832", 2020-12-05T17:39:12.718 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:39:12.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.718 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.719 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6833", 2020-12-05T17:39:12.719 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:39:12.719 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.719 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.720 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:39:12.720 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.720 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.721 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2020-12-05T17:39:12.721 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:39:12.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.722 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.722 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2020-12-05T17:39:12.722 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:39:12.722 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.723 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.723 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6827/11535", 2020-12-05T17:39:12.724 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6829/11535", 2020-12-05T17:39:12.724 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6833/11535", 2020-12-05T17:39:12.725 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6831/11535", 2020-12-05T17:39:12.725 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:39:12.725 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:39:12.725 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:39:12.725 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.726 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:39:12.726 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c301fe4e-5eab-402c-ae6f-db3e6a7181af", 2020-12-05T17:39:12.727 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:39:12.727 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:39:12.727 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:39:12.727 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:39:12.727 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:39:12.728 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:39:12.728 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:39:12.728 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 81, 2020-12-05T17:39:12.728 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:39:12.729 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:39:12.729 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:39:12.729 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.730 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.730 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6808", 2020-12-05T17:39:12.730 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:39:12.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.731 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.731 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6809", 2020-12-05T17:39:12.731 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:39:12.731 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.732 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.732 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:39:12.732 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.733 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.733 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6810", 2020-12-05T17:39:12.733 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:39:12.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.734 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.734 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6811", 2020-12-05T17:39:12.734 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:39:12.735 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.735 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.735 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:39:12.735 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.736 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.736 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6817", 2020-12-05T17:39:12.736 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:39:12.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.737 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.737 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6819", 2020-12-05T17:39:12.737 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:39:12.738 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.738 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.738 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:39:12.739 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.739 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.739 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6812", 2020-12-05T17:39:12.739 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:39:12.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.740 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.740 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6814", 2020-12-05T17:39:12.741 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:39:12.741 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.741 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.741 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6809/11541", 2020-12-05T17:39:12.742 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6811/11541", 2020-12-05T17:39:12.742 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6819/11541", 2020-12-05T17:39:12.742 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6814/11541", 2020-12-05T17:39:12.742 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:39:12.743 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:39:12.743 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:39:12.743 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.744 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:39:12.744 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "9476eff4-345b-46c5-a473-1d9a89a6d327", 2020-12-05T17:39:12.744 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:39:12.744 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:39:12.744 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:39:12.745 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:39:12.745 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:39:12.745 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:39:12.745 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:39:12.746 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 81, 2020-12-05T17:39:12.746 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:39:12.746 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:39:12.746 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:39:12.746 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.747 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.747 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6800", 2020-12-05T17:39:12.747 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:39:12.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.748 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.748 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6801", 2020-12-05T17:39:12.748 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:39:12.749 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.749 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.749 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:39:12.749 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.750 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6802", 2020-12-05T17:39:12.750 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:39:12.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.751 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.751 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6803", 2020-12-05T17:39:12.751 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:39:12.752 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.752 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.752 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:39:12.752 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.753 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.753 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6806", 2020-12-05T17:39:12.753 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:39:12.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.754 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.754 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6807", 2020-12-05T17:39:12.754 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:39:12.755 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.755 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.755 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:39:12.756 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.756 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.756 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6804", 2020-12-05T17:39:12.756 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:39:12.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.757 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.757 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6805", 2020-12-05T17:39:12.757 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:39:12.758 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.758 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.758 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6801/11539", 2020-12-05T17:39:12.759 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6803/11539", 2020-12-05T17:39:12.759 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6807/11539", 2020-12-05T17:39:12.759 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6805/11539", 2020-12-05T17:39:12.760 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:39:12.760 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:39:12.760 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:39:12.760 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.761 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:39:12.761 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8dfd8567-9b1c-4d13-9c96-071183e469c2", 2020-12-05T17:39:12.761 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:39:12.762 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:39:12.762 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:39:12.762 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:39:12.762 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:39:12.762 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:39:12.763 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:39:12.763 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 81, 2020-12-05T17:39:12.763 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:39:12.763 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:39:12.764 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:39:12.764 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.764 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.765 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6820", 2020-12-05T17:39:12.765 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:39:12.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.765 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.766 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6822", 2020-12-05T17:39:12.766 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:39:12.766 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.766 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.767 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:39:12.767 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.767 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.768 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6823", 2020-12-05T17:39:12.768 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:39:12.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.769 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.769 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6825", 2020-12-05T17:39:12.769 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:39:12.769 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.769 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.770 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:39:12.770 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.771 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.771 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6829", 2020-12-05T17:39:12.771 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:39:12.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.772 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.772 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6830", 2020-12-05T17:39:12.772 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:39:12.772 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.772 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.773 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:39:12.774 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.774 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.775 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6826", 2020-12-05T17:39:12.775 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:39:12.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.776 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.776 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6827", 2020-12-05T17:39:12.776 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:39:12.776 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.776 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.777 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6822/11542", 2020-12-05T17:39:12.777 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6825/11542", 2020-12-05T17:39:12.777 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6830/11542", 2020-12-05T17:39:12.777 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6827/11542", 2020-12-05T17:39:12.778 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:39:12.778 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:39:12.778 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:39:12.778 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.779 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:39:12.779 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "24d466b3-01dc-4ba2-8b81-85250d163bd2", 2020-12-05T17:39:12.779 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:39:12.780 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:39:12.780 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:39:12.780 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:39:12.780 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:39:12.780 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:39:12.781 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:39:12.781 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 68, 2020-12-05T17:39:12.781 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:39:12.781 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:39:12.782 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:39:12.782 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.782 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.783 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6813", 2020-12-05T17:39:12.783 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:39:12.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.783 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.784 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6815", 2020-12-05T17:39:12.784 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:39:12.784 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.784 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.785 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:39:12.785 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.785 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.786 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6816", 2020-12-05T17:39:12.786 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:39:12.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.786 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.787 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6818", 2020-12-05T17:39:12.787 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:39:12.787 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.787 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.788 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:39:12.788 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.788 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.789 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6828", 2020-12-05T17:39:12.789 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:39:12.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.790 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.790 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6831", 2020-12-05T17:39:12.790 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:39:12.790 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.791 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.791 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:39:12.791 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:12.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.792 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:12.792 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6821", 2020-12-05T17:39:12.792 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:39:12.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.793 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:12.793 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6824", 2020-12-05T17:39:12.794 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:39:12.794 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.794 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.794 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6815/11540", 2020-12-05T17:39:12.795 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6818/11540", 2020-12-05T17:39:12.795 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6831/11540", 2020-12-05T17:39:12.795 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6824/11540", 2020-12-05T17:39:12.795 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:39:12.795 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:39:12.796 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:39:12.796 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:12.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.796 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:39:12.797 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2020-12-05T17:39:12.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.797 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:39:12.797 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:39:12.797 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:39:12.798 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:39:12.798 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:39:12.798 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:39:12.798 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.398868+0000", 2020-12-05T17:39:12.799 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:39:12.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.799 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:39:12.799 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:39:12.800 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:39:12.800 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:39:12.800 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:39:12.800 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:39:12.801 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.675748+0000", 2020-12-05T17:39:12.801 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:39:12.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.801 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:39:12.802 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:39:12.802 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:39:12.802 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:39:12.802 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:39:12.802 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:39:12.803 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.620888+0000", 2020-12-05T17:39:12.803 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:39:12.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.803 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:39:12.804 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:39:12.804 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:39:12.804 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:39:12.804 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:39:12.805 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:39:12.805 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.681600+0000", 2020-12-05T17:39:12.805 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:39:12.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.806 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:39:12.806 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:39:12.806 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:39:12.806 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:39:12.807 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:39:12.807 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:39:12.807 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.658769+0000", 2020-12-05T17:39:12.807 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:39:12.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.808 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:39:12.808 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:39:12.808 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:39:12.809 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:39:12.809 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:39:12.809 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:39:12.809 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.630708+0000", 2020-12-05T17:39:12.809 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:39:12.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.810 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:39:12.810 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:39:12.811 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:39:12.811 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:39:12.811 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:39:12.811 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:39:12.811 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.648352+0000", 2020-12-05T17:39:12.812 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:39:12.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:12.812 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:39:12.813 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:39:12.813 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:39:12.813 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:39:12.813 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:39:12.813 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:39:12.815 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.629399+0000", 2020-12-05T17:39:12.815 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:39:12.816 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.816 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:39:12.816 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2020-12-05T17:39:12.817 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2020-12-05T17:39:12.817 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2020-12-05T17:39:12.817 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2020-12-05T17:39:12.817 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": {}, 2020-12-05T17:39:12.817 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2020-12-05T17:39:12.818 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2020-12-05T17:39:12.818 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2020-12-05T17:39:12.819 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:39:12.819 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2020-12-05T17:39:12.819 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2020-12-05T17:39:12.819 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:39:12.820 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2020-12-05T17:39:12.820 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:39:12.820 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:39:12.820 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2020-12-05T17:39:12.821 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2020-12-05T17:39:12.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.821 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2020-12-05T17:39:12.822 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:39:12.822 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:39:12.822 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2020-12-05T17:39:12.822 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:39:12.823 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:39:12.823 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2020-12-05T17:39:12.823 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:12.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:12.824 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2020-12-05T17:39:12.824 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2020-12-05T17:39:12.824 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2020-12-05T17:39:12.824 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2020-12-05T17:39:12.824 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2020-12-05T17:39:12.825 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:39:12.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.571+0000 7fc04d7f6700 1 -- 172.21.15.32:0/2769458594 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc02c055330 msgr2=0x7fc02c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:12.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.571+0000 7fc04d7f6700 1 --2- 172.21.15.32:0/2769458594 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc02c055330 0x7fc02c0577f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fc03800b000 tx=0x7fc038009040).stop 2020-12-05T17:39:12.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.571+0000 7fc04d7f6700 1 -- 172.21.15.32:0/2769458594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0480b56d0 msgr2=0x7fc04813ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:12.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.571+0000 7fc04d7f6700 1 --2- 172.21.15.32:0/2769458594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0480b56d0 0x7fc04813ecf0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7fc038001110 tx=0x7fc040018000).stop 2020-12-05T17:39:12.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.571+0000 7fc04d7f6700 1 -- 172.21.15.32:0/2769458594 shutdown_connections 2020-12-05T17:39:12.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.571+0000 7fc04d7f6700 1 --2- 172.21.15.32:0/2769458594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0480b56d0 0x7fc04813ecf0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:12.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.571+0000 7fc04d7f6700 1 --2- 172.21.15.32:0/2769458594 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc02c055330 0x7fc02c0577f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:12.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.571+0000 7fc04d7f6700 1 --2- 172.21.15.32:0/2769458594 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc0480b1870 0x7fc0480aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:12.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.571+0000 7fc04d7f6700 1 --2- 172.21.15.32:0/2769458594 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc0480b52f0 0x7fc04813e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:12.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.571+0000 7fc04d7f6700 1 -- 172.21.15.32:0/2769458594 >> 172.21.15.32:0/2769458594 conn(0x7fc04805bbb0 msgr2=0x7fc0480b3280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:12.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.571+0000 7fc04d7f6700 1 -- 172.21.15.32:0/2769458594 shutdown_connections 2020-12-05T17:39:12.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.571+0000 7fc04d7f6700 1 -- 172.21.15.32:0/2769458594 wait complete. 2020-12-05T17:39:12.830 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:39:12.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.947+0000 7f7a60a1f700 1 Processor -- start 2020-12-05T17:39:12.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.947+0000 7f7a60a1f700 1 -- start start 2020-12-05T17:39:12.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.947+0000 7f7a60a1f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a5c0b14e0 0x7f7a5c0ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:12.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.947+0000 7f7a60a1f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a5c0af480 0x7f7a5c142ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:12.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.947+0000 7f7a60a1f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a5c142ab0 con 0x7f7a5c0b5f00 2020-12-05T17:39:12.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.947+0000 7f7a60a1f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a5c0af8f0 con 0x7f7a5c0af480 2020-12-05T17:39:12.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.947+0000 7f7a60a1f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c0b62e0 con 0x7f7a5c0b14e0 2020-12-05T17:39:12.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.947+0000 7f7a5b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7a5c0b5f00 0x7f7a5c066370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40412/0 (socket says 172.21.15.32:40412) 2020-12-05T17:39:12.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.947+0000 7f7a5b7fe700 1 -- 172.21.15.32:0/1400046337 learned_addr learned my addr 172.21.15.32:0/1400046337 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:12.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.947+0000 7f7a5bfff700 1 --2- 172.21.15.32:0/1400046337 >> v2:172.21.15.32:3300/0 conn(0x7f7a5c0af480 0x7f7a5c142ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:12.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.947+0000 7f7a5affd700 1 --2- 172.21.15.32:0/1400046337 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a5c0b14e0 0x7f7a5c0ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:12.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.947+0000 7f7a5a7fc700 1 -- 172.21.15.32:0/1400046337 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2298692765 0 0) 0x7f7a5c142ab0 con 0x7f7a5c0b5f00 2020-12-05T17:39:12.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.947+0000 7f7a5a7fc700 1 -- 172.21.15.32:0/1400046337 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a3c003230 con 0x7f7a5c0b5f00 2020-12-05T17:39:12.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.947+0000 7f7a5bfff700 1 -- 172.21.15.32:0/1400046337 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a5c0b14e0 msgr2=0x7f7a5c0ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.947+0000 7f7a5bfff700 1 --2- 172.21.15.32:0/1400046337 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a5c0b14e0 0x7f7a5c0ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.947+0000 7f7a5bfff700 1 -- 172.21.15.32:0/1400046337 >> v1:172.21.15.32:6789/0 conn(0x7f7a5c0b5f00 legacy=0x7f7a5c066370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.947+0000 7f7a5bfff700 1 -- 172.21.15.32:0/1400046337 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c0afbf0 con 0x7f7a5c0af480 2020-12-05T17:39:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.947+0000 7f7a5bfff700 1 --2- 172.21.15.32:0/1400046337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5c0af480 0x7f7a5c142ee0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7f7a54002f30 tx=0x7f7a54006dc0).ready entity=mon.0 client_cookie=93c244e19f69b0ca server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:12.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.947+0000 7f7a5a7fc700 1 -- 172.21.15.32:0/1400046337 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a540096a0 con 0x7f7a5c0af480 2020-12-05T17:39:12.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.947+0000 7f7a5a7fc700 1 -- 172.21.15.32:0/1400046337 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a54009800 con 0x7f7a5c0af480 2020-12-05T17:39:12.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.947+0000 7f7a5a7fc700 1 -- 172.21.15.32:0/1400046337 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a54009b60 con 0x7f7a5c0af480 2020-12-05T17:39:12.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.947+0000 7f7a60a1f700 1 -- 172.21.15.32:0/1400046337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5c0af480 msgr2=0x7f7a5c142ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:12.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.947+0000 7f7a60a1f700 1 --2- 172.21.15.32:0/1400046337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5c0af480 0x7f7a5c142ee0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7f7a54002f30 tx=0x7f7a54006dc0).stop 2020-12-05T17:39:12.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.951+0000 7f7a60a1f700 1 -- 172.21.15.32:0/1400046337 shutdown_connections 2020-12-05T17:39:12.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.951+0000 7f7a60a1f700 1 --2- 172.21.15.32:0/1400046337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5c0af480 0x7f7a5c142ee0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:12.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.951+0000 7f7a60a1f700 1 --2- 172.21.15.32:0/1400046337 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a5c0b14e0 0x7f7a5c0ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:12.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.951+0000 7f7a60a1f700 1 -- 172.21.15.32:0/1400046337 >> 172.21.15.32:0/1400046337 conn(0x7f7a5c0604c0 msgr2=0x7f7a5c05c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:12.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.951+0000 7f7a60a1f700 1 -- 172.21.15.32:0/1400046337 shutdown_connections 2020-12-05T17:39:12.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.951+0000 7f7a60a1f700 1 -- 172.21.15.32:0/1400046337 wait complete. 2020-12-05T17:39:12.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.951+0000 7f7a60a1f700 1 Processor -- start 2020-12-05T17:39:12.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.951+0000 7f7a60a1f700 1 -- start start 2020-12-05T17:39:12.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.951+0000 7f7a60a1f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a5c0af480 0x7f7a5c05f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:12.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.951+0000 7f7a60a1f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a5c0b14e0 0x7f7a5c1519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:12.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.951+0000 7f7a60a1f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5c0b5f00 0x7f7a5c151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:12.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.951+0000 7f7a60a1f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c063140 con 0x7f7a5c0b5f00 2020-12-05T17:39:12.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.951+0000 7f7a60a1f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c1515a0 con 0x7f7a5c0af480 2020-12-05T17:39:12.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.951+0000 7f7a60a1f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c154d40 con 0x7f7a5c0b14e0 2020-12-05T17:39:12.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.951+0000 7f7a5bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5c0b5f00 0x7f7a5c151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:12.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.951+0000 7f7a5bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5c0b5f00 0x7f7a5c151ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37772/0 (socket says 172.21.15.32:37772) 2020-12-05T17:39:12.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.951+0000 7f7a5bfff700 1 -- 172.21.15.32:0/574717777 learned_addr learned my addr 172.21.15.32:0/574717777 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:12.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.951+0000 7f7a5b7fe700 1 --2- 172.21.15.32:0/574717777 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a5c0af480 0x7f7a5c05f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:12.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.951+0000 7f7a5affd700 1 --2- 172.21.15.32:0/574717777 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a5c0b14e0 0x7f7a5c1519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:12.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.955+0000 7f7a5bfff700 1 -- 172.21.15.32:0/574717777 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a5c0b14e0 msgr2=0x7f7a5c1519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:12.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.955+0000 7f7a5bfff700 1 --2- 172.21.15.32:0/574717777 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a5c0b14e0 0x7f7a5c1519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:12.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.955+0000 7f7a5bfff700 1 -- 172.21.15.32:0/574717777 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a5c0af480 msgr2=0x7f7a5c05f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:12.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.955+0000 7f7a5bfff700 1 --2- 172.21.15.32:0/574717777 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a5c0af480 0x7f7a5c05f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:12.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.955+0000 7f7a5bfff700 1 -- 172.21.15.32:0/574717777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c144780 con 0x7f7a5c0b5f00 2020-12-05T17:39:12.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.955+0000 7f7a5bfff700 1 --2- 172.21.15.32:0/574717777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5c0b5f00 0x7f7a5c151ed0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7f7a54002f30 tx=0x7f7a54015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:12.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.955+0000 7f7a58ff9700 1 -- 172.21.15.32:0/574717777 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a540118e0 con 0x7f7a5c0b5f00 2020-12-05T17:39:12.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.955+0000 7f7a58ff9700 1 -- 172.21.15.32:0/574717777 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a54011a40 con 0x7f7a5c0b5f00 2020-12-05T17:39:12.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.955+0000 7f7a60a1f700 1 -- 172.21.15.32:0/574717777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a5c144270 con 0x7f7a5c0b5f00 2020-12-05T17:39:12.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.955+0000 7f7a58ff9700 1 -- 172.21.15.32:0/574717777 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a54011da0 con 0x7f7a5c0b5f00 2020-12-05T17:39:12.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.955+0000 7f7a60a1f700 1 -- 172.21.15.32:0/574717777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5c05fbe0 con 0x7f7a5c0b5f00 2020-12-05T17:39:12.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.955+0000 7f7a60a1f700 1 -- 172.21.15.32:0/574717777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a5c154b50 con 0x7f7a5c0b5f00 2020-12-05T17:39:12.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.955+0000 7f7a58ff9700 1 -- 172.21.15.32:0/574717777 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7a54024030 con 0x7f7a5c0b5f00 2020-12-05T17:39:12.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.959+0000 7f7a58ff9700 1 --2- 172.21.15.32:0/574717777 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7a40055440 0x7f7a40057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:12.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.959+0000 7f7a58ff9700 1 -- 172.21.15.32:0/574717777 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7a5406f1f0 con 0x7f7a5c0b5f00 2020-12-05T17:39:12.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.959+0000 7f7a5b7fe700 1 --2- 172.21.15.32:0/574717777 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7a40055440 0x7f7a40057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:12.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.959+0000 7f7a5b7fe700 1 --2- 172.21.15.32:0/574717777 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7a40055440 0x7f7a40057900 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f7a4400d000 tx=0x7f7a44009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:12.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:12.963+0000 7f7a58ff9700 1 -- 172.21.15.32:0/574717777 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7a54045280 con 0x7f7a5c0b5f00 2020-12-05T17:39:13.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.147+0000 7f7a60a1f700 1 -- 172.21.15.32:0/574717777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a5c057050 con 0x7f7a5c0b5f00 2020-12-05T17:39:13.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.147+0000 7f7a58ff9700 1 -- 172.21.15.32:0/574717777 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f7a54022020 con 0x7f7a5c0b5f00 2020-12-05T17:39:13.154 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:13.154 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":37,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6535,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2494768218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2494768218},{"type":"v1","addr":"172.21.15.61:6833","nonce":2494768218}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":35}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:38:56.821107+0000","modified":"2020-12-05T17:38:57.824754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/81037173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":81037173},{"type":"v1","addr":"172.21.15.32:6835","nonce":81037173}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-12-05T17:39:13.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.147+0000 7f7a60a1f700 1 -- 172.21.15.32:0/574717777 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7a40055440 msgr2=0x7f7a40057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:13.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.147+0000 7f7a60a1f700 1 --2- 172.21.15.32:0/574717777 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7a40055440 0x7f7a40057900 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f7a4400d000 tx=0x7f7a44009040).stop 2020-12-05T17:39:13.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.147+0000 7f7a60a1f700 1 -- 172.21.15.32:0/574717777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5c0b5f00 msgr2=0x7f7a5c151ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:13.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.147+0000 7f7a60a1f700 1 --2- 172.21.15.32:0/574717777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5c0b5f00 0x7f7a5c151ed0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7f7a54002f30 tx=0x7f7a54015000).stop 2020-12-05T17:39:13.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.147+0000 7f7a60a1f700 1 -- 172.21.15.32:0/574717777 shutdown_connections 2020-12-05T17:39:13.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.147+0000 7f7a60a1f700 1 --2- 172.21.15.32:0/574717777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5c0b5f00 0x7f7a5c151ed0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:13.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.147+0000 7f7a60a1f700 1 --2- 172.21.15.32:0/574717777 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a5c0af480 0x7f7a5c05f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:13.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.147+0000 7f7a60a1f700 1 --2- 172.21.15.32:0/574717777 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7a40055440 0x7f7a40057900 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:13.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.147+0000 7f7a60a1f700 1 --2- 172.21.15.32:0/574717777 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a5c0b14e0 0x7f7a5c1519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:13.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.147+0000 7f7a60a1f700 1 -- 172.21.15.32:0/574717777 >> 172.21.15.32:0/574717777 conn(0x7f7a5c0604c0 msgr2=0x7f7a5c13f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:13.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.147+0000 7f7a60a1f700 1 -- 172.21.15.32:0/574717777 shutdown_connections 2020-12-05T17:39:13.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.147+0000 7f7a60a1f700 1 -- 172.21.15.32:0/574717777 wait complete. 2020-12-05T17:39:13.158 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 37 2020-12-05T17:39:13.166 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-12-05T17:39:13.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.279+0000 7f44df39a700 1 Processor -- start 2020-12-05T17:39:13.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.279+0000 7f44df39a700 1 -- start start 2020-12-05T17:39:13.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44df39a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f44d80b1fc0 0x7f44d80af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:13.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44df39a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d8146a90 con 0x7f44d8147400 2020-12-05T17:39:13.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44df39a700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d80b23a0 con 0x7f44d80ab710 2020-12-05T17:39:13.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44df39a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d8061e90 con 0x7f44d80b1fc0 2020-12-05T17:39:13.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44deb99700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f44d8147400 0x7f44d8145d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40430/0 (socket says 172.21.15.32:40430) 2020-12-05T17:39:13.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44deb99700 1 -- 172.21.15.32:0/2837827143 learned_addr learned my addr 172.21.15.32:0/2837827143 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:13.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44de398700 1 --2- 172.21.15.32:0/2837827143 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f44d80b1fc0 0x7f44d80af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:13.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44dd396700 1 -- 172.21.15.32:0/2837827143 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2260034768 0 0) 0x7f44d8146a90 con 0x7f44d8147400 2020-12-05T17:39:13.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44dd396700 1 -- 172.21.15.32:0/2837827143 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44c0003230 con 0x7f44d8147400 2020-12-05T17:39:13.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44dd396700 1 -- 172.21.15.32:0/2837827143 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3598709776 0 0) 0x7f44c0003230 con 0x7f44d8147400 2020-12-05T17:39:13.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44dd396700 1 -- 172.21.15.32:0/2837827143 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f44d80b1fc0 msgr2=0x7f44d80af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:13.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44dd396700 1 --2- 172.21.15.32:0/2837827143 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f44d80b1fc0 0x7f44d80af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:13.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44dd396700 1 -- 172.21.15.32:0/2837827143 >> v1:172.21.15.61:6789/0 conn(0x7f44d80ab710 legacy=0x7f44d8061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44dd396700 1 -- 172.21.15.32:0/2837827143 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d8146cc0 con 0x7f44d8147400 2020-12-05T17:39:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44dd396700 1 -- 172.21.15.32:0/2837827143 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f44d0001a50 con 0x7f44d8147400 2020-12-05T17:39:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44dd396700 1 -- 172.21.15.32:0/2837827143 >> v1:172.21.15.32:6789/0 conn(0x7f44d8147400 legacy=0x7f44d8145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44dd396700 1 --2- 172.21.15.32:0/2837827143 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f44d80b1fc0 0x7f44c0003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44dd396700 1 --2- 172.21.15.32:0/2837827143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44c0003c60 0x7f44c00044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44dd396700 1 --2- 172.21.15.32:0/2837827143 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f44c00049c0 0x7f44c0004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44dd396700 1 -- 172.21.15.32:0/2837827143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d8061e90 con 0x7f44c0003c60 2020-12-05T17:39:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44dd396700 1 -- 172.21.15.32:0/2837827143 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c00041f0 con 0x7f44c00049c0 2020-12-05T17:39:13.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44dd396700 1 -- 172.21.15.32:0/2837827143 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c000b720 con 0x7f44d80b1fc0 2020-12-05T17:39:13.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44ddb97700 1 --2- 172.21.15.32:0/2837827143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44c0003c60 0x7f44c00044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:13.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44de398700 1 --2- 172.21.15.32:0/2837827143 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f44d80b1fc0 0x7f44c0003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:13.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44deb99700 1 --2- 172.21.15.32:0/2837827143 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f44c00049c0 0x7f44c0004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:13.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44ddb97700 1 -- 172.21.15.32:0/2837827143 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f44d80b1fc0 msgr2=0x7f44c0003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44ddb97700 1 --2- 172.21.15.32:0/2837827143 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f44d80b1fc0 0x7f44c0003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44ddb97700 1 -- 172.21.15.32:0/2837827143 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f44c00049c0 msgr2=0x7f44c0004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44ddb97700 1 --2- 172.21.15.32:0/2837827143 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f44c00049c0 0x7f44c0004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44ddb97700 1 -- 172.21.15.32:0/2837827143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c0003230 con 0x7f44c0003c60 2020-12-05T17:39:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44ddb97700 1 --2- 172.21.15.32:0/2837827143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44c0003c60 0x7f44c00044d0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7f44d80608f0 tx=0x7f44c8007b60).ready entity=mon.0 client_cookie=d1f4b61fe9f26d1a server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44dd396700 1 -- 172.21.15.32:0/2837827143 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44c800c600 con 0x7f44c0003c60 2020-12-05T17:39:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44dd396700 1 -- 172.21.15.32:0/2837827143 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c800c760 con 0x7f44c0003c60 2020-12-05T17:39:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44dd396700 1 -- 172.21.15.32:0/2837827143 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44c800cac0 con 0x7f44c0003c60 2020-12-05T17:39:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44df39a700 1 -- 172.21.15.32:0/2837827143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44c0003c60 msgr2=0x7f44c00044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:13.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44df39a700 1 --2- 172.21.15.32:0/2837827143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44c0003c60 0x7f44c00044d0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7f44d80608f0 tx=0x7f44c8007b60).stop 2020-12-05T17:39:13.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44deb99700 1 -- 172.21.15.32:0/2837827143 reap_dead start 2020-12-05T17:39:13.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44df39a700 1 -- 172.21.15.32:0/2837827143 shutdown_connections 2020-12-05T17:39:13.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44df39a700 1 -- 172.21.15.32:0/2837827143 >> 172.21.15.32:0/2837827143 conn(0x7f44d80b5530 msgr2=0x7f44d80b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:13.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44df39a700 1 -- 172.21.15.32:0/2837827143 shutdown_connections 2020-12-05T17:39:13.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.283+0000 7f44df39a700 1 -- 172.21.15.32:0/2837827143 wait complete. 2020-12-05T17:39:13.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.287+0000 7f44df39a700 1 Processor -- start 2020-12-05T17:39:13.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.287+0000 7f44df39a700 1 -- start start 2020-12-05T17:39:13.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.287+0000 7f44df39a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f44d814d350 0x7f44d8063660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:13.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.287+0000 7f44df39a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f44d814c180 0x7f44d8158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:13.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.287+0000 7f44df39a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44d81593d0 0x7f44d80645f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:13.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.287+0000 7f44df39a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d814ce10 con 0x7f44d81593d0 2020-12-05T17:39:13.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.287+0000 7f44df39a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d8063b50 con 0x7f44d814c180 2020-12-05T17:39:13.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.287+0000 7f44df39a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d814d730 con 0x7f44d814d350 2020-12-05T17:39:13.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.287+0000 7f44deb99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44d81593d0 0x7f44d80645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:13.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.287+0000 7f44ddb97700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f44d814c180 0x7f44d8158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:13.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.287+0000 7f44de398700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f44d814d350 0x7f44d8063660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:13.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.287+0000 7f44deb99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44d81593d0 0x7f44d80645f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37794/0 (socket says 172.21.15.32:37794) 2020-12-05T17:39:13.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.287+0000 7f44deb99700 1 -- 172.21.15.32:0/2452106617 learned_addr learned my addr 172.21.15.32:0/2452106617 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:13.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.291+0000 7f44deb99700 1 -- 172.21.15.32:0/2452106617 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f44d814d350 msgr2=0x7f44d8063660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:13.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.291+0000 7f44deb99700 1 --2- 172.21.15.32:0/2452106617 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f44d814d350 0x7f44d8063660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:13.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.291+0000 7f44deb99700 1 -- 172.21.15.32:0/2452106617 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f44d814c180 msgr2=0x7f44d8158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:13.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.291+0000 7f44deb99700 1 --2- 172.21.15.32:0/2452106617 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f44d814c180 0x7f44d8158ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:13.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.291+0000 7f44deb99700 1 -- 172.21.15.32:0/2452106617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d8065340 con 0x7f44d81593d0 2020-12-05T17:39:13.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.291+0000 7f44ddb97700 1 --2- 172.21.15.32:0/2452106617 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f44d814c180 0x7f44d8158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:13.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.291+0000 7f44deb99700 1 --2- 172.21.15.32:0/2452106617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44d81593d0 0x7f44d80645f0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f44d8063540 tx=0x7f44d80b1220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:13.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.291+0000 7f44c77fe700 1 -- 172.21.15.32:0/2452106617 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44d000fb00 con 0x7f44d81593d0 2020-12-05T17:39:13.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.291+0000 7f44c77fe700 1 -- 172.21.15.32:0/2452106617 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d000fc60 con 0x7f44d81593d0 2020-12-05T17:39:13.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.291+0000 7f44df39a700 1 -- 172.21.15.32:0/2452106617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d80631c0 con 0x7f44d81593d0 2020-12-05T17:39:13.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.291+0000 7f44c77fe700 1 -- 172.21.15.32:0/2452106617 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44d0015410 con 0x7f44d81593d0 2020-12-05T17:39:13.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.291+0000 7f44df39a700 1 -- 172.21.15.32:0/2452106617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d814bda0 con 0x7f44d81593d0 2020-12-05T17:39:13.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.291+0000 7f44c77fe700 1 -- 172.21.15.32:0/2452106617 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f44d0015570 con 0x7f44d81593d0 2020-12-05T17:39:13.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.291+0000 7f44df39a700 1 -- 172.21.15.32:0/2452106617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d8063020 con 0x7f44d81593d0 2020-12-05T17:39:13.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.291+0000 7f44c77fe700 1 --2- 172.21.15.32:0/2452106617 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f44b80554e0 0x7f44b80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:13.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.291+0000 7f44de398700 1 --2- 172.21.15.32:0/2452106617 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f44b80554e0 0x7f44b80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:13.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.291+0000 7f44c77fe700 1 -- 172.21.15.32:0/2452106617 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f44d006e190 con 0x7f44d81593d0 2020-12-05T17:39:13.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.291+0000 7f44de398700 1 --2- 172.21.15.32:0/2452106617 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f44b80554e0 0x7f44b80579a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f44d4008000 tx=0x7f44d4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:13.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.299+0000 7f44c77fe700 1 -- 172.21.15.32:0/2452106617 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f44d8063020 con 0x7f44d81593d0 2020-12-05T17:39:13.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.487+0000 7f44df39a700 1 -- 172.21.15.32:0/2452106617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f44d8063020 con 0x7f44d81593d0 2020-12-05T17:39:13.602 INFO:tasks.ceph.mon.a.smithi032.stderr:2020-12-05T17:39:13.599+0000 7fdef8136700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-12-05T17:39:13.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.611+0000 7f44c77fe700 1 -- 172.21.15.32:0/2452106617 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) v1 ==== 161+0+0 (secure 0 0 0) 0x7f44d8063020 con 0x7f44d81593d0 2020-12-05T17:39:13.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.615+0000 7f44df39a700 1 -- 172.21.15.32:0/2452106617 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f44b80554e0 msgr2=0x7f44b80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:13.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.615+0000 7f44df39a700 1 --2- 172.21.15.32:0/2452106617 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f44b80554e0 0x7f44b80579a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f44d4008000 tx=0x7f44d4008040).stop 2020-12-05T17:39:13.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.615+0000 7f44df39a700 1 -- 172.21.15.32:0/2452106617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44d81593d0 msgr2=0x7f44d80645f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:13.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.615+0000 7f44df39a700 1 --2- 172.21.15.32:0/2452106617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44d81593d0 0x7f44d80645f0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f44d8063540 tx=0x7f44d80b1220).stop 2020-12-05T17:39:13.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.615+0000 7f44df39a700 1 -- 172.21.15.32:0/2452106617 shutdown_connections 2020-12-05T17:39:13.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.615+0000 7f44df39a700 1 --2- 172.21.15.32:0/2452106617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44d81593d0 0x7f44d80645f0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:13.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.615+0000 7f44df39a700 1 --2- 172.21.15.32:0/2452106617 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f44b80554e0 0x7f44b80579a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:13.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.615+0000 7f44df39a700 1 --2- 172.21.15.32:0/2452106617 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f44d814d350 0x7f44d8063660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:13.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.615+0000 7f44df39a700 1 --2- 172.21.15.32:0/2452106617 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f44d814c180 0x7f44d8158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:13.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.615+0000 7f44df39a700 1 -- 172.21.15.32:0/2452106617 >> 172.21.15.32:0/2452106617 conn(0x7f44d80b5530 msgr2=0x7f44d80b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:13.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.615+0000 7f44df39a700 1 -- 172.21.15.32:0/2452106617 shutdown_connections 2020-12-05T17:39:13.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.615+0000 7f44df39a700 1 -- 172.21.15.32:0/2452106617 wait complete. 2020-12-05T17:39:13.623 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-12-05T17:39:13.634 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:39:13.641 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:39:13.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.743+0000 7f51d311c700 1 Processor -- start 2020-12-05T17:39:13.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.743+0000 7f51d311c700 1 -- start start 2020-12-05T17:39:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.743+0000 7f51d311c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f51cc0b56d0 0x7f51cc0aef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.743+0000 7f51d311c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51cc1420f0 con 0x7f51cc0b1870 2020-12-05T17:39:13.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d311c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51cc0b0380 con 0x7f51cc0b52f0 2020-12-05T17:39:13.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d311c700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51cc0ad4f0 con 0x7f51cc0b56d0 2020-12-05T17:39:13.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d1919700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f51cc0b1870 0x7f51cc0ae860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40460/0 (socket says 172.21.15.32:40460) 2020-12-05T17:39:13.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d1919700 1 -- 172.21.15.32:0/2156324659 learned_addr learned my addr 172.21.15.32:0/2156324659 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:13.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d291b700 1 --2- 172.21.15.32:0/2156324659 >> v2:172.21.15.61:3300/0 conn(0x7f51cc0b56d0 0x7f51cc0aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:13.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d1118700 1 -- 172.21.15.32:0/2156324659 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2794131843 0 0) 0x7f51cc1420f0 con 0x7f51cc0b1870 2020-12-05T17:39:13.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d1118700 1 -- 172.21.15.32:0/2156324659 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51b0004340 con 0x7f51cc0b1870 2020-12-05T17:39:13.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d1118700 1 -- 172.21.15.32:0/2156324659 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2037124254 0 0) 0x7f51cc0b0380 con 0x7f51cc0b52f0 2020-12-05T17:39:13.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d1118700 1 -- 172.21.15.32:0/2156324659 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51cc1420f0 con 0x7f51cc0b52f0 2020-12-05T17:39:13.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d1118700 1 -- 172.21.15.32:0/2156324659 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2314471509 0 0) 0x7f51b0004340 con 0x7f51cc0b1870 2020-12-05T17:39:13.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d1118700 1 -- 172.21.15.32:0/2156324659 >> v2:172.21.15.61:3300/0 conn(0x7f51cc0b56d0 msgr2=0x7f51cc0aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:13.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d1118700 1 --2- 172.21.15.32:0/2156324659 >> v2:172.21.15.61:3300/0 conn(0x7f51cc0b56d0 0x7f51cc0aef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:13.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d1118700 1 -- 172.21.15.32:0/2156324659 >> v1:172.21.15.61:6789/0 conn(0x7f51cc0b52f0 legacy=0x7f51cc138810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:13.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d1118700 1 -- 172.21.15.32:0/2156324659 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51cc0b0620 con 0x7f51cc0b1870 2020-12-05T17:39:13.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d1118700 1 -- 172.21.15.32:0/2156324659 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f51b80016d0 con 0x7f51cc0b1870 2020-12-05T17:39:13.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d1118700 1 -- 172.21.15.32:0/2156324659 >> v1:172.21.15.32:6789/0 conn(0x7f51cc0b1870 legacy=0x7f51cc0ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:13.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d1118700 1 --2- 172.21.15.32:0/2156324659 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f51b00049c0 0x7f51b0004dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:13.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d1118700 1 --2- 172.21.15.32:0/2156324659 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f51b00055e0 0x7f51b0005a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:13.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d1118700 1 --2- 172.21.15.32:0/2156324659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b0005f50 0x7f51b000c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:13.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d1118700 1 -- 172.21.15.32:0/2156324659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc0ad4f0 con 0x7f51b0005f50 2020-12-05T17:39:13.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d1118700 1 -- 172.21.15.32:0/2156324659 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b00064b0 con 0x7f51b00055e0 2020-12-05T17:39:13.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d1118700 1 -- 172.21.15.32:0/2156324659 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b0003230 con 0x7f51b00049c0 2020-12-05T17:39:13.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d291b700 1 --2- 172.21.15.32:0/2156324659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b0005f50 0x7f51b000c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:13.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d1919700 1 --2- 172.21.15.32:0/2156324659 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f51b00055e0 0x7f51b0005a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:13.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d211a700 1 --2- 172.21.15.32:0/2156324659 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f51b00049c0 0x7f51b0004dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:13.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d291b700 1 -- 172.21.15.32:0/2156324659 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f51b00049c0 msgr2=0x7f51b0004dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:13.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d291b700 1 --2- 172.21.15.32:0/2156324659 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f51b00049c0 0x7f51b0004dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:13.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d291b700 1 -- 172.21.15.32:0/2156324659 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f51b00055e0 msgr2=0x7f51b0005a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:13.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d291b700 1 --2- 172.21.15.32:0/2156324659 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f51b00055e0 0x7f51b0005a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:13.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d291b700 1 -- 172.21.15.32:0/2156324659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b0004340 con 0x7f51b0005f50 2020-12-05T17:39:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d211a700 1 --2- 172.21.15.32:0/2156324659 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f51b00049c0 0x7f51b0004dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d291b700 1 -- 172.21.15.32:0/2156324659 reap_dead start 2020-12-05T17:39:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d291b700 1 --2- 172.21.15.32:0/2156324659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b0005f50 0x7f51b000c830 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7f51c400c000 tx=0x7f51c4008040).ready entity=mon.0 client_cookie=6614adf116fe4b9e server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d1118700 1 -- 172.21.15.32:0/2156324659 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51c400e970 con 0x7f51b0005f50 2020-12-05T17:39:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d1118700 1 -- 172.21.15.32:0/2156324659 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c400ead0 con 0x7f51b0005f50 2020-12-05T17:39:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d1118700 1 -- 172.21.15.32:0/2156324659 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51c400ee30 con 0x7f51b0005f50 2020-12-05T17:39:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d311c700 1 -- 172.21.15.32:0/2156324659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b0005f50 msgr2=0x7f51b000c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d311c700 1 --2- 172.21.15.32:0/2156324659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b0005f50 0x7f51b000c830 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7f51c400c000 tx=0x7f51c4008040).stop 2020-12-05T17:39:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d311c700 1 -- 172.21.15.32:0/2156324659 shutdown_connections 2020-12-05T17:39:13.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d311c700 1 --2- 172.21.15.32:0/2156324659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b0005f50 0x7f51b000c830 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:13.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d311c700 1 -- 172.21.15.32:0/2156324659 >> 172.21.15.32:0/2156324659 conn(0x7f51cc05bbb0 msgr2=0x7f51cc062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:13.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d311c700 1 -- 172.21.15.32:0/2156324659 shutdown_connections 2020-12-05T17:39:13.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.747+0000 7f51d311c700 1 -- 172.21.15.32:0/2156324659 wait complete. 2020-12-05T17:39:13.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.751+0000 7f51d311c700 1 Processor -- start 2020-12-05T17:39:13.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.751+0000 7f51d311c700 1 -- start start 2020-12-05T17:39:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.751+0000 7f51d311c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f51b0005f50 0x7f51cc14d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:13.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.751+0000 7f51d311c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51cc066780 0x7f51cc13f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:13.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.751+0000 7f51d311c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f51cc065740 0x7f51cc0634d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:13.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.751+0000 7f51d311c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc0647a0 con 0x7f51cc066780 2020-12-05T17:39:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.751+0000 7f51d311c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc064a40 con 0x7f51cc065740 2020-12-05T17:39:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.751+0000 7f51d311c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc064ce0 con 0x7f51b0005f50 2020-12-05T17:39:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.751+0000 7f51d1919700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51cc066780 0x7f51cc13f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.751+0000 7f51d1919700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51cc066780 0x7f51cc13f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37824/0 (socket says 172.21.15.32:37824) 2020-12-05T17:39:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.751+0000 7f51d1919700 1 -- 172.21.15.32:0/3122385799 learned_addr learned my addr 172.21.15.32:0/3122385799 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:13.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.751+0000 7f51d211a700 1 --2- 172.21.15.32:0/3122385799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f51b0005f50 0x7f51cc14d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:13.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.751+0000 7f51d291b700 1 --2- 172.21.15.32:0/3122385799 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f51cc065740 0x7f51cc0634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:13.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.751+0000 7f51d1919700 1 -- 172.21.15.32:0/3122385799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f51b0005f50 msgr2=0x7f51cc14d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:13.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.751+0000 7f51d1919700 1 --2- 172.21.15.32:0/3122385799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f51b0005f50 0x7f51cc14d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:13.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.751+0000 7f51d1919700 1 -- 172.21.15.32:0/3122385799 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f51cc065740 msgr2=0x7f51cc0634d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:13.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.751+0000 7f51d1919700 1 --2- 172.21.15.32:0/3122385799 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f51cc065740 0x7f51cc0634d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:13.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.751+0000 7f51d1919700 1 -- 172.21.15.32:0/3122385799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51cc063030 con 0x7f51cc066780 2020-12-05T17:39:13.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.751+0000 7f51d1919700 1 --2- 172.21.15.32:0/3122385799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51cc066780 0x7f51cc13f6e0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7f51b8001000 tx=0x7f51cc0b0510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:13.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.751+0000 7f51cb7fe700 1 -- 172.21.15.32:0/3122385799 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51b800fa90 con 0x7f51cc066780 2020-12-05T17:39:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.751+0000 7f51d311c700 1 -- 172.21.15.32:0/3122385799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51cc0639c0 con 0x7f51cc066780 2020-12-05T17:39:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.751+0000 7f51cb7fe700 1 -- 172.21.15.32:0/3122385799 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b800fbf0 con 0x7f51cc066780 2020-12-05T17:39:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.751+0000 7f51cb7fe700 1 -- 172.21.15.32:0/3122385799 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51b8012350 con 0x7f51cc066780 2020-12-05T17:39:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.755+0000 7f51d311c700 1 -- 172.21.15.32:0/3122385799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51cc063ee0 con 0x7f51cc066780 2020-12-05T17:39:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.755+0000 7f51d311c700 1 -- 172.21.15.32:0/3122385799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51cc0631c0 con 0x7f51cc066780 2020-12-05T17:39:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.755+0000 7f51cb7fe700 1 -- 172.21.15.32:0/3122385799 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f51b80124b0 con 0x7f51cc066780 2020-12-05T17:39:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.755+0000 7f51cb7fe700 1 --2- 172.21.15.32:0/3122385799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f51b4055330 0x7f51b40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.755+0000 7f51cb7fe700 1 -- 172.21.15.32:0/3122385799 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f51b806da80 con 0x7f51cc066780 2020-12-05T17:39:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.755+0000 7f51d211a700 1 --2- 172.21.15.32:0/3122385799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f51b4055330 0x7f51b40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.755+0000 7f51d211a700 1 --2- 172.21.15.32:0/3122385799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f51b4055330 0x7f51b40577f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f51c000d000 tx=0x7f51c000c040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.763+0000 7f51cb7fe700 1 -- 172.21.15.32:0/3122385799 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f51b8043ad0 con 0x7f51cc066780 2020-12-05T17:39:13.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.947+0000 7f51d311c700 1 -- 172.21.15.32:0/3122385799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51cc057050 con 0x7f51cc066780 2020-12-05T17:39:13.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.951+0000 7f51cb7fe700 1 -- 172.21.15.32:0/3122385799 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+1779 (secure 0 0 0) 0x7f51b8019020 con 0x7f51cc066780 2020-12-05T17:39:13.955 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:13.956 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":38,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6535,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2494768218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2494768218},{"type":"v1","addr":"172.21.15.61:6833","nonce":2494768218}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:38:56.821107+0000","modified":"2020-12-05T17:39:13.601331+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-12-05T17:39:13.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.951+0000 7f51d311c700 1 -- 172.21.15.32:0/3122385799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f51b4055330 msgr2=0x7f51b40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:13.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.951+0000 7f51d311c700 1 --2- 172.21.15.32:0/3122385799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f51b4055330 0x7f51b40577f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f51c000d000 tx=0x7f51c000c040).stop 2020-12-05T17:39:13.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.951+0000 7f51d311c700 1 -- 172.21.15.32:0/3122385799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51cc066780 msgr2=0x7f51cc13f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:13.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.951+0000 7f51d311c700 1 --2- 172.21.15.32:0/3122385799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51cc066780 0x7f51cc13f6e0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7f51b8001000 tx=0x7f51cc0b0510).stop 2020-12-05T17:39:13.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.951+0000 7f51d311c700 1 -- 172.21.15.32:0/3122385799 shutdown_connections 2020-12-05T17:39:13.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.951+0000 7f51d311c700 1 --2- 172.21.15.32:0/3122385799 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f51cc065740 0x7f51cc0634d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:13.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.951+0000 7f51d311c700 1 --2- 172.21.15.32:0/3122385799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f51b4055330 0x7f51b40577f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:13.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.951+0000 7f51d311c700 1 --2- 172.21.15.32:0/3122385799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f51b0005f50 0x7f51cc14d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:13.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.951+0000 7f51d311c700 1 --2- 172.21.15.32:0/3122385799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51cc066780 0x7f51cc13f6e0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:13.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.951+0000 7f51d311c700 1 -- 172.21.15.32:0/3122385799 >> 172.21.15.32:0/3122385799 conn(0x7f51cc05bbb0 msgr2=0x7f51cc13e810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:13.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.951+0000 7f51d311c700 1 -- 172.21.15.32:0/3122385799 shutdown_connections 2020-12-05T17:39:13.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:13.951+0000 7f51d311c700 1 -- 172.21.15.32:0/3122385799 wait complete. 2020-12-05T17:39:13.960 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 38 2020-12-05T17:39:13.969 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-12-05T17:39:14.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.123+0000 7f7f3f180700 1 Processor -- start 2020-12-05T17:39:14.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.123+0000 7f7f3f180700 1 -- start start 2020-12-05T17:39:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.123+0000 7f7f3f180700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7f380b7720 0x7f7f38061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.123+0000 7f7f3f180700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f7f381403f0 0x7f7f38143e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.123+0000 7f7f3f180700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f38144320 0x7f7f3813d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.123+0000 7f7f3f180700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f38140870 con 0x7f7f38144320 2020-12-05T17:39:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.123+0000 7f7f3f180700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f38143a00 con 0x7f7f381403f0 2020-12-05T17:39:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.123+0000 7f7f3f180700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f380b7b00 con 0x7f7f380b7720 2020-12-05T17:39:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.123+0000 7f7f3e97f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f38144320 0x7f7f3813d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:14.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.123+0000 7f7f3e97f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f38144320 0x7f7f3813d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37834/0 (socket says 172.21.15.32:37834) 2020-12-05T17:39:14.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.123+0000 7f7f3e97f700 1 -- 172.21.15.32:0/559301204 learned_addr learned my addr 172.21.15.32:0/559301204 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:14.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.123+0000 7f7f3d97d700 1 --2- 172.21.15.32:0/559301204 >> v2:172.21.15.61:3300/0 conn(0x7f7f381403f0 0x7f7f38143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:14.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.123+0000 7f7f3e17e700 1 --2- 172.21.15.32:0/559301204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7f380b7720 0x7f7f38061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:14.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.127+0000 7f7f3e97f700 1 -- 172.21.15.32:0/559301204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7f380b7720 msgr2=0x7f7f38061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:14.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.127+0000 7f7f3e97f700 1 --2- 172.21.15.32:0/559301204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7f380b7720 0x7f7f38061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:14.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.127+0000 7f7f3e97f700 1 -- 172.21.15.32:0/559301204 >> v2:172.21.15.61:3300/0 conn(0x7f7f381403f0 msgr2=0x7f7f38143e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:14.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.127+0000 7f7f3e97f700 1 --2- 172.21.15.32:0/559301204 >> v2:172.21.15.61:3300/0 conn(0x7f7f381403f0 0x7f7f38143e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:14.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.127+0000 7f7f3e97f700 1 -- 172.21.15.32:0/559301204 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3805fe60 con 0x7f7f38144320 2020-12-05T17:39:14.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.127+0000 7f7f3e17e700 1 --2- 172.21.15.32:0/559301204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7f380b7720 0x7f7f38061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:14.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.127+0000 7f7f3e97f700 1 --2- 172.21.15.32:0/559301204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f38144320 0x7f7f3813d3c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7f7f300032b0 tx=0x7f7f30009270).ready entity=mon.0 client_cookie=f8206c5f50d802c2 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:14.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.127+0000 7f7f3d17c700 1 -- 172.21.15.32:0/559301204 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f30009e60 con 0x7f7f38144320 2020-12-05T17:39:14.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.127+0000 7f7f3d17c700 1 -- 172.21.15.32:0/559301204 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3000bc00 con 0x7f7f38144320 2020-12-05T17:39:14.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.127+0000 7f7f3d17c700 1 -- 172.21.15.32:0/559301204 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f3000bd70 con 0x7f7f38144320 2020-12-05T17:39:14.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.127+0000 7f7f3f180700 1 -- 172.21.15.32:0/559301204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f38144320 msgr2=0x7f7f3813d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:14.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.127+0000 7f7f3f180700 1 --2- 172.21.15.32:0/559301204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f38144320 0x7f7f3813d3c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7f7f300032b0 tx=0x7f7f30009270).stop 2020-12-05T17:39:14.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.127+0000 7f7f3f180700 1 -- 172.21.15.32:0/559301204 shutdown_connections 2020-12-05T17:39:14.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.127+0000 7f7f3f180700 1 --2- 172.21.15.32:0/559301204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f38144320 0x7f7f3813d3c0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:14.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.127+0000 7f7f3f180700 1 --2- 172.21.15.32:0/559301204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7f380b7720 0x7f7f38061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:14.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.127+0000 7f7f3f180700 1 --2- 172.21.15.32:0/559301204 >> v2:172.21.15.61:3300/0 conn(0x7f7f381403f0 0x7f7f38143e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:14.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.127+0000 7f7f3f180700 1 -- 172.21.15.32:0/559301204 >> 172.21.15.32:0/559301204 conn(0x7f7f3805bc20 msgr2=0x7f7f380b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:14.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.127+0000 7f7f3f180700 1 -- 172.21.15.32:0/559301204 shutdown_connections 2020-12-05T17:39:14.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.127+0000 7f7f3f180700 1 -- 172.21.15.32:0/559301204 wait complete. 2020-12-05T17:39:14.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.127+0000 7f7f3f180700 1 Processor -- start 2020-12-05T17:39:14.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.127+0000 7f7f3f180700 1 -- start start 2020-12-05T17:39:14.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f3f180700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7f380b7720 0x7f7f380b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:14.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f3f180700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7f381403f0 0x7f7f380b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:14.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f3f180700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f38144320 0x7f7f3813b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:14.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f3f180700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f380b3820 con 0x7f7f38144320 2020-12-05T17:39:14.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f3f180700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f380b3ac0 con 0x7f7f380b7720 2020-12-05T17:39:14.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f3f180700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f380b3d60 con 0x7f7f381403f0 2020-12-05T17:39:14.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f3e97f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f38144320 0x7f7f3813b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:14.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f3e97f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f38144320 0x7f7f3813b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37840/0 (socket says 172.21.15.32:37840) 2020-12-05T17:39:14.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f3e97f700 1 -- 172.21.15.32:0/4168865688 learned_addr learned my addr 172.21.15.32:0/4168865688 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:14.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f3e17e700 1 --2- 172.21.15.32:0/4168865688 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7f380b7720 0x7f7f380b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:14.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f3d97d700 1 --2- 172.21.15.32:0/4168865688 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7f381403f0 0x7f7f380b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:14.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f3e97f700 1 -- 172.21.15.32:0/4168865688 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7f381403f0 msgr2=0x7f7f380b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f3e97f700 1 --2- 172.21.15.32:0/4168865688 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7f381403f0 0x7f7f380b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f3e97f700 1 -- 172.21.15.32:0/4168865688 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7f380b7720 msgr2=0x7f7f380b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f3e97f700 1 --2- 172.21.15.32:0/4168865688 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7f380b7720 0x7f7f380b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f3e97f700 1 -- 172.21.15.32:0/4168865688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f380b1230 con 0x7f7f38144320 2020-12-05T17:39:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f3e17e700 1 --2- 172.21.15.32:0/4168865688 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7f380b7720 0x7f7f380b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f3e97f700 1 --2- 172.21.15.32:0/4168865688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f38144320 0x7f7f3813b700 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f7f300032b0 tx=0x7f7f30016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f277fe700 1 -- 172.21.15.32:0/4168865688 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f3001f5e0 con 0x7f7f38144320 2020-12-05T17:39:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f277fe700 1 -- 172.21.15.32:0/4168865688 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3000bed0 con 0x7f7f38144320 2020-12-05T17:39:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f3f180700 1 -- 172.21.15.32:0/4168865688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f380b4200 con 0x7f7f38144320 2020-12-05T17:39:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f277fe700 1 -- 172.21.15.32:0/4168865688 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f3001f930 con 0x7f7f38144320 2020-12-05T17:39:14.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f3f180700 1 -- 172.21.15.32:0/4168865688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f380af820 con 0x7f7f38144320 2020-12-05T17:39:14.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f3f180700 1 -- 172.21.15.32:0/4168865688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f380b3680 con 0x7f7f38144320 2020-12-05T17:39:14.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.131+0000 7f7f277fe700 1 -- 172.21.15.32:0/4168865688 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7f30026030 con 0x7f7f38144320 2020-12-05T17:39:14.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.135+0000 7f7f277fe700 1 --2- 172.21.15.32:0/4168865688 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7f180554e0 0x7f7f180579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:14.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.135+0000 7f7f277fe700 1 -- 172.21.15.32:0/4168865688 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f7f3006ea00 con 0x7f7f38144320 2020-12-05T17:39:14.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.135+0000 7f7f3e17e700 1 --2- 172.21.15.32:0/4168865688 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7f180554e0 0x7f7f180579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:14.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.135+0000 7f7f3e17e700 1 --2- 172.21.15.32:0/4168865688 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7f180554e0 0x7f7f180579a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f7f3400c000 tx=0x7f7f34008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:14.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.139+0000 7f7f277fe700 1 -- 172.21.15.32:0/4168865688 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7f380b3680 con 0x7f7f38144320 2020-12-05T17:39:14.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.323+0000 7f7f3f180700 1 -- 172.21.15.32:0/4168865688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7f380af820 con 0x7f7f38144320 2020-12-05T17:39:14.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.619+0000 7f7f277fe700 1 -- 172.21.15.32:0/4168865688 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7f380af820 con 0x7f7f38144320 2020-12-05T17:39:14.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.619+0000 7f7f3f180700 1 -- 172.21.15.32:0/4168865688 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7f180554e0 msgr2=0x7f7f180579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:14.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.619+0000 7f7f3f180700 1 --2- 172.21.15.32:0/4168865688 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7f180554e0 0x7f7f180579a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f7f3400c000 tx=0x7f7f34008040).stop 2020-12-05T17:39:14.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.619+0000 7f7f3f180700 1 -- 172.21.15.32:0/4168865688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f38144320 msgr2=0x7f7f3813b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:14.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.619+0000 7f7f3f180700 1 --2- 172.21.15.32:0/4168865688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f38144320 0x7f7f3813b700 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f7f300032b0 tx=0x7f7f30016000).stop 2020-12-05T17:39:14.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.623+0000 7f7f3f180700 1 -- 172.21.15.32:0/4168865688 shutdown_connections 2020-12-05T17:39:14.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.623+0000 7f7f3f180700 1 --2- 172.21.15.32:0/4168865688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f38144320 0x7f7f3813b700 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:14.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.623+0000 7f7f3f180700 1 --2- 172.21.15.32:0/4168865688 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7f380b7720 0x7f7f380b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:14.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.623+0000 7f7f3f180700 1 --2- 172.21.15.32:0/4168865688 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7f180554e0 0x7f7f180579a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:14.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.623+0000 7f7f3f180700 1 --2- 172.21.15.32:0/4168865688 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7f381403f0 0x7f7f380b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:14.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.623+0000 7f7f3f180700 1 -- 172.21.15.32:0/4168865688 >> 172.21.15.32:0/4168865688 conn(0x7f7f3805bc20 msgr2=0x7f7f38141900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:14.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.623+0000 7f7f3f180700 1 -- 172.21.15.32:0/4168865688 shutdown_connections 2020-12-05T17:39:14.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.623+0000 7f7f3f180700 1 -- 172.21.15.32:0/4168865688 wait complete. 2020-12-05T17:39:14.641 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-12-05T17:39:14.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.759+0000 7f8457fff700 1 Processor -- start 2020-12-05T17:39:14.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.759+0000 7f8457fff700 1 -- start start 2020-12-05T17:39:14.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f8457fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84580afd30 0x7f84580b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:14.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f8457fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f84580af5f0 0x7f84580aed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:14.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f8457fff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84581468e0 con 0x7f84580b1dc0 2020-12-05T17:39:14.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f8457fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8458142c90 con 0x7f84580afd30 2020-12-05T17:39:14.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f8457fff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8458146b10 con 0x7f84580af5f0 2020-12-05T17:39:14.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f84567fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84580afd30 0x7f84580b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:14.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f84567fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84580afd30 0x7f84580b21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37848/0 (socket says 172.21.15.32:37848) 2020-12-05T17:39:14.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f84567fc700 1 -- 172.21.15.32:0/1684076091 learned_addr learned my addr 172.21.15.32:0/1684076091 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f84577fe700 1 --2- 172.21.15.32:0/1684076091 >> v2:172.21.15.61:3300/0 conn(0x7f84580af5f0 0x7f84580aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f84567fc700 1 -- 172.21.15.32:0/1684076091 >> v2:172.21.15.61:3300/0 conn(0x7f84580af5f0 msgr2=0x7f84580aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f84567fc700 1 --2- 172.21.15.32:0/1684076091 >> v2:172.21.15.61:3300/0 conn(0x7f84580af5f0 0x7f84580aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:14.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f84567fc700 1 -- 172.21.15.32:0/1684076091 >> v1:172.21.15.61:6789/0 conn(0x7f84580b1dc0 legacy=0x7f84580ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:14.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f84567fc700 1 -- 172.21.15.32:0/1684076091 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8458146db0 con 0x7f84580afd30 2020-12-05T17:39:14.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f84567fc700 1 --2- 172.21.15.32:0/1684076091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84580afd30 0x7f84580b21a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7f8448002f30 tx=0x7f8448006e80).ready entity=mon.0 client_cookie=a20837af8ae19aa0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:14.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f8455ffb700 1 -- 172.21.15.32:0/1684076091 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84480097f0 con 0x7f84580afd30 2020-12-05T17:39:14.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f8455ffb700 1 -- 172.21.15.32:0/1684076091 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8448009950 con 0x7f84580afd30 2020-12-05T17:39:14.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f8455ffb700 1 -- 172.21.15.32:0/1684076091 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8448009cb0 con 0x7f84580afd30 2020-12-05T17:39:14.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f8457fff700 1 -- 172.21.15.32:0/1684076091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84580afd30 msgr2=0x7f84580b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:14.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f8457fff700 1 --2- 172.21.15.32:0/1684076091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84580afd30 0x7f84580b21a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7f8448002f30 tx=0x7f8448006e80).stop 2020-12-05T17:39:14.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f8457fff700 1 -- 172.21.15.32:0/1684076091 shutdown_connections 2020-12-05T17:39:14.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f8457fff700 1 --2- 172.21.15.32:0/1684076091 >> v2:172.21.15.61:3300/0 conn(0x7f84580af5f0 0x7f84580aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:14.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f8457fff700 1 --2- 172.21.15.32:0/1684076091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84580afd30 0x7f84580b21a0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:14.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f8457fff700 1 -- 172.21.15.32:0/1684076091 >> 172.21.15.32:0/1684076091 conn(0x7f84580620b0 msgr2=0x7f8458062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:14.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f8457fff700 1 -- 172.21.15.32:0/1684076091 shutdown_connections 2020-12-05T17:39:14.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f8457fff700 1 -- 172.21.15.32:0/1684076091 wait complete. 2020-12-05T17:39:14.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.763+0000 7f8457fff700 1 Processor -- start 2020-12-05T17:39:14.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f8457fff700 1 -- start start 2020-12-05T17:39:14.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f8457fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f84580af5f0 0x7f8458065040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:14.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f8457fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84580afd30 0x7f84580b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:14.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f8457fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f84580b1dc0 0x7f84580631a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:14.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f8457fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8458064090 con 0x7f84580afd30 2020-12-05T17:39:14.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f8457fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8458144e90 con 0x7f84580b1dc0 2020-12-05T17:39:14.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f8457fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8458145130 con 0x7f84580af5f0 2020-12-05T17:39:14.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f84567fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84580afd30 0x7f84580b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f84567fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84580afd30 0x7f84580b0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37852/0 (socket says 172.21.15.32:37852) 2020-12-05T17:39:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f84567fc700 1 -- 172.21.15.32:0/1434477877 learned_addr learned my addr 172.21.15.32:0/1434477877 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f84577fe700 1 --2- 172.21.15.32:0/1434477877 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f84580b1dc0 0x7f84580631a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f8456ffd700 1 --2- 172.21.15.32:0/1434477877 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f84580af5f0 0x7f8458065040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:14.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f84567fc700 1 -- 172.21.15.32:0/1434477877 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f84580af5f0 msgr2=0x7f8458065040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:14.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f84567fc700 1 --2- 172.21.15.32:0/1434477877 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f84580af5f0 0x7f8458065040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f84567fc700 1 -- 172.21.15.32:0/1434477877 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f84580b1dc0 msgr2=0x7f84580631a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f84567fc700 1 --2- 172.21.15.32:0/1434477877 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f84580b1dc0 0x7f84580631a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f84567fc700 1 -- 172.21.15.32:0/1434477877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84581453d0 con 0x7f84580afd30 2020-12-05T17:39:14.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f84577fe700 1 --2- 172.21.15.32:0/1434477877 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f84580b1dc0 0x7f84580631a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:39:14.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f8456ffd700 1 --2- 172.21.15.32:0/1434477877 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f84580af5f0 0x7f8458065040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:39:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f84567fc700 1 --2- 172.21.15.32:0/1434477877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84580afd30 0x7f84580b0630 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7f8448002f30 tx=0x7f8448015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f844ffff700 1 -- 172.21.15.32:0/1434477877 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8448011d30 con 0x7f84580afd30 2020-12-05T17:39:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f844ffff700 1 -- 172.21.15.32:0/1434477877 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8448011e90 con 0x7f84580afd30 2020-12-05T17:39:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f8457fff700 1 -- 172.21.15.32:0/1434477877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8458145670 con 0x7f84580afd30 2020-12-05T17:39:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f844ffff700 1 -- 172.21.15.32:0/1434477877 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f844801e650 con 0x7f84580afd30 2020-12-05T17:39:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f8457fff700 1 -- 172.21.15.32:0/1434477877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8458066dd0 con 0x7f84580afd30 2020-12-05T17:39:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.767+0000 7f8457fff700 1 -- 172.21.15.32:0/1434477877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8458146550 con 0x7f84580afd30 2020-12-05T17:39:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.771+0000 7f844ffff700 1 -- 172.21.15.32:0/1434477877 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f844801a040 con 0x7f84580afd30 2020-12-05T17:39:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.771+0000 7f844ffff700 1 --2- 172.21.15.32:0/1434477877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f843c055500 0x7f843c0579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.771+0000 7f844ffff700 1 -- 172.21.15.32:0/1434477877 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f844806f280 con 0x7f84580afd30 2020-12-05T17:39:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.771+0000 7f8456ffd700 1 --2- 172.21.15.32:0/1434477877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f843c055500 0x7f843c0579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:14.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.771+0000 7f8456ffd700 1 --2- 172.21.15.32:0/1434477877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f843c055500 0x7f843c0579c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f8440008000 tx=0x7f8440009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:14.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.779+0000 7f844ffff700 1 -- 172.21.15.32:0/1434477877 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8458146550 con 0x7f84580afd30 2020-12-05T17:39:14.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:14.927+0000 7f8457fff700 1 -- 172.21.15.32:0/1434477877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8458146550 con 0x7f84580afd30 2020-12-05T17:39:15.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.627+0000 7f844ffff700 1 -- 172.21.15.32:0/1434477877 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v85) v1 ==== 185+0+0 (secure 0 0 0) 0x7f8458146550 con 0x7f84580afd30 2020-12-05T17:39:15.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.631+0000 7f8457fff700 1 -- 172.21.15.32:0/1434477877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f843c055500 msgr2=0x7f843c0579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:15.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.631+0000 7f8457fff700 1 --2- 172.21.15.32:0/1434477877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f843c055500 0x7f843c0579c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f8440008000 tx=0x7f8440009040).stop 2020-12-05T17:39:15.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.631+0000 7f8457fff700 1 -- 172.21.15.32:0/1434477877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84580afd30 msgr2=0x7f84580b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:15.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.631+0000 7f8457fff700 1 --2- 172.21.15.32:0/1434477877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84580afd30 0x7f84580b0630 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7f8448002f30 tx=0x7f8448015000).stop 2020-12-05T17:39:15.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.631+0000 7f8457fff700 1 -- 172.21.15.32:0/1434477877 shutdown_connections 2020-12-05T17:39:15.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.631+0000 7f8457fff700 1 --2- 172.21.15.32:0/1434477877 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f84580b1dc0 0x7f84580631a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:15.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.631+0000 7f8457fff700 1 --2- 172.21.15.32:0/1434477877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f843c055500 0x7f843c0579c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:15.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.631+0000 7f8457fff700 1 --2- 172.21.15.32:0/1434477877 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f84580af5f0 0x7f8458065040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:15.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.631+0000 7f8457fff700 1 --2- 172.21.15.32:0/1434477877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84580afd30 0x7f84580b0630 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:15.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.631+0000 7f8457fff700 1 -- 172.21.15.32:0/1434477877 >> 172.21.15.32:0/1434477877 conn(0x7f84580620b0 msgr2=0x7f845805f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:15.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.631+0000 7f8457fff700 1 -- 172.21.15.32:0/1434477877 shutdown_connections 2020-12-05T17:39:15.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.631+0000 7f8457fff700 1 -- 172.21.15.32:0/1434477877 wait complete. 2020-12-05T17:39:15.640 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2020-12-05T17:39:15.654 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-12-05T17:39:15.712 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:39:15.715 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:39:15.717 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:39:15.719 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:39:15.722 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:39:15.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.771+0000 7f09b2f04700 1 Processor -- start 2020-12-05T17:39:15.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.771+0000 7f09b2f04700 1 -- start start 2020-12-05T17:39:15.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.771+0000 7f09b2f04700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09ac14b320 0x7f09ac14d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:15.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.771+0000 7f09b2f04700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09ac1439c0 con 0x7f09ac05f6f0 2020-12-05T17:39:15.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.771+0000 7f09b2f04700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09ac142520 con 0x7f09ac061b30 2020-12-05T17:39:15.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.771+0000 7f09b2f04700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09ac05fe20 con 0x7f09ac14b320 2020-12-05T17:39:15.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.771+0000 7f09b2703700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09ac14b320 0x7f09ac14d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:15.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.771+0000 7f09b1f02700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f09ac05f6f0 0x7f09ac141700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40506/0 (socket says 172.21.15.32:40506) 2020-12-05T17:39:15.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.771+0000 7f09b1f02700 1 -- 172.21.15.32:0/2323648274 learned_addr learned my addr 172.21.15.32:0/2323648274 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:15.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.771+0000 7f09b2703700 1 -- 172.21.15.32:0/2323648274 >> v1:172.21.15.61:6789/0 conn(0x7f09ac061b30 legacy=0x7f09ac141e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:15.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.771+0000 7f09b2703700 1 -- 172.21.15.32:0/2323648274 >> v1:172.21.15.32:6789/0 conn(0x7f09ac05f6f0 legacy=0x7f09ac141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:15.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.771+0000 7f09b0f00700 1 -- 172.21.15.32:0/2323648274 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1527822700 0 0) 0x7f09ac1439c0 con 0x7f09ac05f6f0 2020-12-05T17:39:15.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.771+0000 7f09b2703700 1 -- 172.21.15.32:0/2323648274 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac146910 con 0x7f09ac14b320 2020-12-05T17:39:15.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.771+0000 7f09b2703700 1 --2- 172.21.15.32:0/2323648274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09ac14b320 0x7f09ac14d710 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f09a40032b0 tx=0x7f09a40091a0).ready entity=mon.0 client_cookie=61087a46e30717a7 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:15.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.771+0000 7f09b0f00700 1 -- 172.21.15.32:0/2323648274 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09a400ba50 con 0x7f09ac14b320 2020-12-05T17:39:15.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.771+0000 7f09b0f00700 1 -- 172.21.15.32:0/2323648274 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a400bbb0 con 0x7f09ac14b320 2020-12-05T17:39:15.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.771+0000 7f09b0f00700 1 -- 172.21.15.32:0/2323648274 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09a400bd20 con 0x7f09ac14b320 2020-12-05T17:39:15.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.771+0000 7f09b2f04700 1 -- 172.21.15.32:0/2323648274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09ac14b320 msgr2=0x7f09ac14d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:15.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.771+0000 7f09b2f04700 1 --2- 172.21.15.32:0/2323648274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09ac14b320 0x7f09ac14d710 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f09a40032b0 tx=0x7f09a40091a0).stop 2020-12-05T17:39:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b2f04700 1 -- 172.21.15.32:0/2323648274 shutdown_connections 2020-12-05T17:39:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b2f04700 1 --2- 172.21.15.32:0/2323648274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09ac14b320 0x7f09ac14d710 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b2f04700 1 -- 172.21.15.32:0/2323648274 >> 172.21.15.32:0/2323648274 conn(0x7f09ac061010 msgr2=0x7f09ac061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b2f04700 1 -- 172.21.15.32:0/2323648274 shutdown_connections 2020-12-05T17:39:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b2f04700 1 -- 172.21.15.32:0/2323648274 wait complete. 2020-12-05T17:39:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b2f04700 1 Processor -- start 2020-12-05T17:39:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b2f04700 1 -- start start 2020-12-05T17:39:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b2f04700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09ac05f6f0 0x7f09ac0b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b2f04700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09ac061b30 0x7f09ac13cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b2f04700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f09ac14b320 0x7f09ac13d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b2f04700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac0646e0 con 0x7f09ac061b30 2020-12-05T17:39:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b2f04700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac13d530 con 0x7f09ac14b320 2020-12-05T17:39:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b2f04700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac063d00 con 0x7f09ac05f6f0 2020-12-05T17:39:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b1701700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09ac061b30 0x7f09ac13cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b1701700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09ac061b30 0x7f09ac13cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37866/0 (socket says 172.21.15.32:37866) 2020-12-05T17:39:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b1701700 1 -- 172.21.15.32:0/3171590444 learned_addr learned my addr 172.21.15.32:0/3171590444 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b1f02700 1 --2- 172.21.15.32:0/3171590444 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09ac05f6f0 0x7f09ac0b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b2703700 1 --2- 172.21.15.32:0/3171590444 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f09ac14b320 0x7f09ac13d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b1701700 1 -- 172.21.15.32:0/3171590444 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09ac05f6f0 msgr2=0x7f09ac0b5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b1701700 1 --2- 172.21.15.32:0/3171590444 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09ac05f6f0 0x7f09ac0b5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b1701700 1 -- 172.21.15.32:0/3171590444 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f09ac14b320 msgr2=0x7f09ac13d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b1701700 1 --2- 172.21.15.32:0/3171590444 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f09ac14b320 0x7f09ac13d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b1701700 1 -- 172.21.15.32:0/3171590444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac063fa0 con 0x7f09ac061b30 2020-12-05T17:39:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b1701700 1 --2- 172.21.15.32:0/3171590444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09ac061b30 0x7f09ac13cb50 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7f09ac13d6c0 tx=0x7f099c007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f099affd700 1 -- 172.21.15.32:0/3171590444 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f099c007240 con 0x7f09ac061b30 2020-12-05T17:39:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f099affd700 1 -- 172.21.15.32:0/3171590444 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099c0073a0 con 0x7f09ac061b30 2020-12-05T17:39:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b2f04700 1 -- 172.21.15.32:0/3171590444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09ac064240 con 0x7f09ac061b30 2020-12-05T17:39:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f099affd700 1 -- 172.21.15.32:0/3171590444 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f099c0066d0 con 0x7f09ac061b30 2020-12-05T17:39:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.775+0000 7f09b2f04700 1 -- 172.21.15.32:0/3171590444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09ac0664c0 con 0x7f09ac061b30 2020-12-05T17:39:15.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.779+0000 7f09b2f04700 1 -- 172.21.15.32:0/3171590444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09ac063750 con 0x7f09ac061b30 2020-12-05T17:39:15.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.779+0000 7f099affd700 1 -- 172.21.15.32:0/3171590444 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f099c010030 con 0x7f09ac061b30 2020-12-05T17:39:15.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.779+0000 7f099affd700 1 --2- 172.21.15.32:0/3171590444 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f098c0553b0 0x7f098c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:15.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.779+0000 7f099affd700 1 -- 172.21.15.32:0/3171590444 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f099c06dc80 con 0x7f09ac061b30 2020-12-05T17:39:15.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.779+0000 7f09b1f02700 1 --2- 172.21.15.32:0/3171590444 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f098c0553b0 0x7f098c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:15.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.779+0000 7f09b1f02700 1 --2- 172.21.15.32:0/3171590444 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f098c0553b0 0x7f098c057870 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f09a800b000 tx=0x7f09a8009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:15.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.787+0000 7f099affd700 1 -- 172.21.15.32:0/3171590444 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f09ac063750 con 0x7f09ac061b30 2020-12-05T17:39:15.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:15.931+0000 7f09b2f04700 1 -- 172.21.15.32:0/3171590444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f09ac063750 con 0x7f09ac061b30 2020-12-05T17:39:16.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:16.855+0000 7f099affd700 1 -- 172.21.15.32:0/3171590444 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v86) v1 ==== 173+0+0 (secure 0 0 0) 0x7f09ac063750 con 0x7f09ac061b30 2020-12-05T17:39:16.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:16.859+0000 7f09b2f04700 1 -- 172.21.15.32:0/3171590444 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f098c0553b0 msgr2=0x7f098c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:16.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:16.859+0000 7f09b2f04700 1 --2- 172.21.15.32:0/3171590444 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f098c0553b0 0x7f098c057870 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f09a800b000 tx=0x7f09a8009040).stop 2020-12-05T17:39:16.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:16.859+0000 7f09b2f04700 1 -- 172.21.15.32:0/3171590444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09ac061b30 msgr2=0x7f09ac13cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:16.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:16.859+0000 7f09b2f04700 1 --2- 172.21.15.32:0/3171590444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09ac061b30 0x7f09ac13cb50 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7f09ac13d6c0 tx=0x7f099c007be0).stop 2020-12-05T17:39:16.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:16.859+0000 7f09b2f04700 1 -- 172.21.15.32:0/3171590444 shutdown_connections 2020-12-05T17:39:16.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:16.859+0000 7f09b2f04700 1 --2- 172.21.15.32:0/3171590444 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f09ac14b320 0x7f09ac13d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:16.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:16.859+0000 7f09b2f04700 1 --2- 172.21.15.32:0/3171590444 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f098c0553b0 0x7f098c057870 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:16.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:16.859+0000 7f09b2f04700 1 --2- 172.21.15.32:0/3171590444 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09ac05f6f0 0x7f09ac0b5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:16.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:16.859+0000 7f09b2f04700 1 --2- 172.21.15.32:0/3171590444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09ac061b30 0x7f09ac13cb50 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:16.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:16.859+0000 7f09b2f04700 1 -- 172.21.15.32:0/3171590444 >> 172.21.15.32:0/3171590444 conn(0x7f09ac061010 msgr2=0x7f09ac0b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:16.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:16.859+0000 7f09b2f04700 1 -- 172.21.15.32:0/3171590444 shutdown_connections 2020-12-05T17:39:16.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:16.859+0000 7f09b2f04700 1 -- 172.21.15.32:0/3171590444 wait complete. 2020-12-05T17:39:16.868 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2020-12-05T17:39:16.881 INFO:tasks.ceph.mds.a:Restarting daemon 2020-12-05T17:39:16.881 INFO:tasks.ceph.mds.a:Stopping old one... 2020-12-05T17:39:16.882 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-12-05T17:39:16.882 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:39:16.882 INFO:tasks.ceph.mds.b:Restarting daemon 2020-12-05T17:39:16.883 INFO:tasks.ceph.mds.b:Stopping old one... 2020-12-05T17:39:16.883 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-12-05T17:39:16.883 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:39:16.898 INFO:tasks.ceph.mds.b:Stopped 2020-12-05T17:39:16.899 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-12-05T17:39:16.937 INFO:tasks.ceph.mds.a:Stopped 2020-12-05T17:39:16.938 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-12-05T17:39:16.941 INFO:tasks.ceph.mds.b:Started 2020-12-05T17:39:16.942 DEBUG:teuthology.parallel:result is None 2020-12-05T17:39:16.980 INFO:tasks.ceph.mds.a:Started 2020-12-05T17:39:16.981 DEBUG:teuthology.parallel:result is None 2020-12-05T17:39:16.981 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-12-05T17:39:17.018 INFO:tasks.ceph.mds.b.smithi061.stdout:starting mds.b at 2020-12-05T17:39:17.061 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:39:17.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.103+0000 7f95961f0700 1 Processor -- start 2020-12-05T17:39:17.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.103+0000 7f95961f0700 1 -- start start 2020-12-05T17:39:17.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.103+0000 7f95961f0700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9590062400 0x7f959005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.103+0000 7f95961f0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f959005feb0 0x7f95900b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.103+0000 7f95961f0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9590142450 con 0x7f959005f3b0 2020-12-05T17:39:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.103+0000 7f95961f0700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95900b42f0 con 0x7f9590062400 2020-12-05T17:39:17.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.103+0000 7f95961f0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9590061780 con 0x7f959005feb0 2020-12-05T17:39:17.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.103+0000 7f95951ee700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f959005f3b0 0x7f95900b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40534/0 (socket says 172.21.15.32:40534) 2020-12-05T17:39:17.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.103+0000 7f95951ee700 1 -- 172.21.15.32:0/2402293745 learned_addr learned my addr 172.21.15.32:0/2402293745 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:17.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.103+0000 7f95949ed700 1 --2- 172.21.15.32:0/2402293745 >> v2:172.21.15.61:3300/0 conn(0x7f9590062400 0x7f959005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:17.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.103+0000 7f95959ef700 1 --2- 172.21.15.32:0/2402293745 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f959005feb0 0x7f95900b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:17.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.103+0000 7f9587fff700 1 -- 172.21.15.32:0/2402293745 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3186060498 0 0) 0x7f9590142450 con 0x7f959005f3b0 2020-12-05T17:39:17.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.103+0000 7f9587fff700 1 -- 172.21.15.32:0/2402293745 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9574003230 con 0x7f959005f3b0 2020-12-05T17:39:17.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.103+0000 7f95949ed700 1 -- 172.21.15.32:0/2402293745 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f959005feb0 msgr2=0x7f95900b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:17.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.103+0000 7f95949ed700 1 --2- 172.21.15.32:0/2402293745 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f959005feb0 0x7f95900b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:17.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.103+0000 7f95949ed700 1 -- 172.21.15.32:0/2402293745 >> v1:172.21.15.32:6789/0 conn(0x7f959005f3b0 legacy=0x7f95900b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.103+0000 7f95949ed700 1 -- 172.21.15.32:0/2402293745 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9590061a20 con 0x7f9590062400 2020-12-05T17:39:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.103+0000 7f95949ed700 1 --2- 172.21.15.32:0/2402293745 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9590062400 0x7f959005f790 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f9580002f30 tx=0x7f9580006dc0).ready entity=mon.1 client_cookie=838eb43d8d7de4c4 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.103+0000 7f9587fff700 1 -- 172.21.15.32:0/2402293745 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95800096a0 con 0x7f9590062400 2020-12-05T17:39:17.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.103+0000 7f9587fff700 1 -- 172.21.15.32:0/2402293745 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9580009800 con 0x7f9590062400 2020-12-05T17:39:17.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.103+0000 7f9587fff700 1 -- 172.21.15.32:0/2402293745 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9580009b60 con 0x7f9590062400 2020-12-05T17:39:17.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.103+0000 7f95961f0700 1 -- 172.21.15.32:0/2402293745 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9590062400 msgr2=0x7f959005f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:17.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.103+0000 7f95961f0700 1 --2- 172.21.15.32:0/2402293745 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9590062400 0x7f959005f790 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f9580002f30 tx=0x7f9580006dc0).stop 2020-12-05T17:39:17.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.107+0000 7f95961f0700 1 -- 172.21.15.32:0/2402293745 shutdown_connections 2020-12-05T17:39:17.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.107+0000 7f95961f0700 1 --2- 172.21.15.32:0/2402293745 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f959005feb0 0x7f95900b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:17.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.107+0000 7f95961f0700 1 --2- 172.21.15.32:0/2402293745 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9590062400 0x7f959005f790 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:17.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.107+0000 7f95961f0700 1 -- 172.21.15.32:0/2402293745 >> 172.21.15.32:0/2402293745 conn(0x7f95900b8ee0 msgr2=0x7f95900b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:17.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.107+0000 7f95961f0700 1 -- 172.21.15.32:0/2402293745 shutdown_connections 2020-12-05T17:39:17.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.107+0000 7f95961f0700 1 -- 172.21.15.32:0/2402293745 wait complete. 2020-12-05T17:39:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.107+0000 7f95961f0700 1 Processor -- start 2020-12-05T17:39:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.107+0000 7f95961f0700 1 -- start start 2020-12-05T17:39:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.111+0000 7f95961f0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f959005f3b0 0x7f95901513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.111+0000 7f95961f0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f959005feb0 0x7f9590145540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.111+0000 7f95961f0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9590062400 0x7f95900af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.111+0000 7f95961f0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95900646f0 con 0x7f959005f3b0 2020-12-05T17:39:17.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.111+0000 7f95961f0700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9590064990 con 0x7f9590062400 2020-12-05T17:39:17.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.111+0000 7f95961f0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95900b9c10 con 0x7f959005feb0 2020-12-05T17:39:17.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.111+0000 7f95951ee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f959005f3b0 0x7f95901513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:17.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.111+0000 7f95951ee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f959005f3b0 0x7f95901513a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37890/0 (socket says 172.21.15.32:37890) 2020-12-05T17:39:17.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.111+0000 7f95951ee700 1 -- 172.21.15.32:0/3573855298 learned_addr learned my addr 172.21.15.32:0/3573855298 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:17.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.111+0000 7f95959ef700 1 --2- 172.21.15.32:0/3573855298 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9590062400 0x7f95900af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:17.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.111+0000 7f95949ed700 1 --2- 172.21.15.32:0/3573855298 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f959005feb0 0x7f9590145540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:17.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.111+0000 7f95951ee700 1 -- 172.21.15.32:0/3573855298 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f959005feb0 msgr2=0x7f9590145540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:17.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.111+0000 7f95951ee700 1 --2- 172.21.15.32:0/3573855298 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f959005feb0 0x7f9590145540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:17.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.111+0000 7f95951ee700 1 -- 172.21.15.32:0/3573855298 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9590062400 msgr2=0x7f95900af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:17.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.111+0000 7f95951ee700 1 --2- 172.21.15.32:0/3573855298 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9590062400 0x7f95900af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:17.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.111+0000 7f95951ee700 1 -- 172.21.15.32:0/3573855298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9590064170 con 0x7f959005f3b0 2020-12-05T17:39:17.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.111+0000 7f95951ee700 1 --2- 172.21.15.32:0/3573855298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f959005f3b0 0x7f95901513a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7f958c0010d0 tx=0x7f958c007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:17.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.111+0000 7f95867fc700 1 -- 172.21.15.32:0/3573855298 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f958c0072e0 con 0x7f959005f3b0 2020-12-05T17:39:17.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.111+0000 7f95867fc700 1 -- 172.21.15.32:0/3573855298 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958c007440 con 0x7f959005f3b0 2020-12-05T17:39:17.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.111+0000 7f95867fc700 1 -- 172.21.15.32:0/3573855298 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f958c0057d0 con 0x7f959005f3b0 2020-12-05T17:39:17.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.111+0000 7f95961f0700 1 -- 172.21.15.32:0/3573855298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95900654f0 con 0x7f959005f3b0 2020-12-05T17:39:17.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.111+0000 7f95961f0700 1 -- 172.21.15.32:0/3573855298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95900af7d0 con 0x7f959005f3b0 2020-12-05T17:39:17.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.115+0000 7f95961f0700 1 -- 172.21.15.32:0/3573855298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9590064b50 con 0x7f959005f3b0 2020-12-05T17:39:17.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.115+0000 7f95867fc700 1 -- 172.21.15.32:0/3573855298 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f958c010070 con 0x7f959005f3b0 2020-12-05T17:39:17.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.115+0000 7f95867fc700 1 --2- 172.21.15.32:0/3573855298 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9578055440 0x7f9578057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:17.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.115+0000 7f95867fc700 1 -- 172.21.15.32:0/3573855298 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f958c06d490 con 0x7f959005f3b0 2020-12-05T17:39:17.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.115+0000 7f95949ed700 1 --2- 172.21.15.32:0/3573855298 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9578055440 0x7f9578057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:17.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.115+0000 7f95949ed700 1 --2- 172.21.15.32:0/3573855298 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9578055440 0x7f9578057900 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f958001b000 tx=0x7f9580015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:17.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.119+0000 7f95867fc700 1 -- 172.21.15.32:0/3573855298 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f958c043910 con 0x7f959005f3b0 2020-12-05T17:39:17.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:17.275+0000 7f95961f0700 1 -- 172.21.15.32:0/3573855298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f9590057050 con 0x7f959005f3b0 2020-12-05T17:39:18.094 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.087+0000 7f95867fc700 1 -- 172.21.15.32:0/3573855298 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v88) v1 ==== 97+0+0 (secure 0 0 0) 0x7f958c046f30 con 0x7f959005f3b0 2020-12-05T17:39:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.091+0000 7f95961f0700 1 -- 172.21.15.32:0/3573855298 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9578055440 msgr2=0x7f9578057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.091+0000 7f95961f0700 1 --2- 172.21.15.32:0/3573855298 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9578055440 0x7f9578057900 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f958001b000 tx=0x7f9580015000).stop 2020-12-05T17:39:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.091+0000 7f95961f0700 1 -- 172.21.15.32:0/3573855298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f959005f3b0 msgr2=0x7f95901513a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.091+0000 7f95961f0700 1 --2- 172.21.15.32:0/3573855298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f959005f3b0 0x7f95901513a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7f958c0010d0 tx=0x7f958c007aa0).stop 2020-12-05T17:39:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.091+0000 7f95961f0700 1 -- 172.21.15.32:0/3573855298 shutdown_connections 2020-12-05T17:39:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.091+0000 7f95961f0700 1 --2- 172.21.15.32:0/3573855298 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9590062400 0x7f95900af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.091+0000 7f95961f0700 1 --2- 172.21.15.32:0/3573855298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f959005f3b0 0x7f95901513a0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.091+0000 7f95961f0700 1 --2- 172.21.15.32:0/3573855298 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9578055440 0x7f9578057900 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.091+0000 7f95961f0700 1 --2- 172.21.15.32:0/3573855298 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f959005feb0 0x7f9590145540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.091+0000 7f95961f0700 1 -- 172.21.15.32:0/3573855298 >> 172.21.15.32:0/3573855298 conn(0x7f95900b8ee0 msgr2=0x7f9590140600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:18.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.091+0000 7f95961f0700 1 -- 172.21.15.32:0/3573855298 shutdown_connections 2020-12-05T17:39:18.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.091+0000 7f95961f0700 1 -- 172.21.15.32:0/3573855298 wait complete. 2020-12-05T17:39:18.101 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2020-12-05T17:39:18.113 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2020-12-05T17:39:18.114 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-12-05T17:39:18.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.227+0000 7f49d14f7700 1 Processor -- start 2020-12-05T17:39:18.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.227+0000 7f49d14f7700 1 -- start start 2020-12-05T17:39:18.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.227+0000 7f49d14f7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49cc05f3b0 0x7f49cc05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:18.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.227+0000 7f49d14f7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49cc142450 con 0x7f49cc05feb0 2020-12-05T17:39:18.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.227+0000 7f49d14f7700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49cc061780 con 0x7f49cc062400 2020-12-05T17:39:18.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.227+0000 7f49d14f7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc061a20 con 0x7f49cc05f3b0 2020-12-05T17:39:18.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.227+0000 7f49d0cf6700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f49cc05feb0 0x7f49cc0b0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40548/0 (socket says 172.21.15.32:40548) 2020-12-05T17:39:18.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.227+0000 7f49d0cf6700 1 -- 172.21.15.32:0/534804324 learned_addr learned my addr 172.21.15.32:0/534804324 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:18.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.227+0000 7f49cbfff700 1 --2- 172.21.15.32:0/534804324 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49cc05f3b0 0x7f49cc05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:18.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.227+0000 7f49caffd700 1 -- 172.21.15.32:0/534804324 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1308339228 0 0) 0x7f49cc142450 con 0x7f49cc05feb0 2020-12-05T17:39:18.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.227+0000 7f49caffd700 1 -- 172.21.15.32:0/534804324 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49ac003230 con 0x7f49cc05feb0 2020-12-05T17:39:18.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.227+0000 7f49caffd700 1 -- 172.21.15.32:0/534804324 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3831756215 0 0) 0x7f49cc061780 con 0x7f49cc062400 2020-12-05T17:39:18.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.227+0000 7f49caffd700 1 -- 172.21.15.32:0/534804324 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49cc142450 con 0x7f49cc062400 2020-12-05T17:39:18.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.227+0000 7f49caffd700 1 -- 172.21.15.32:0/534804324 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 699491607 0 0) 0x7f49ac003230 con 0x7f49cc05feb0 2020-12-05T17:39:18.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49caffd700 1 -- 172.21.15.32:0/534804324 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49cc05f3b0 msgr2=0x7f49cc05f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:18.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49caffd700 1 --2- 172.21.15.32:0/534804324 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49cc05f3b0 0x7f49cc05f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49caffd700 1 -- 172.21.15.32:0/534804324 >> v1:172.21.15.61:6789/0 conn(0x7f49cc062400 legacy=0x7f49cc0b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:18.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49caffd700 1 -- 172.21.15.32:0/534804324 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc061cc0 con 0x7f49cc05feb0 2020-12-05T17:39:18.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49caffd700 1 -- 172.21.15.32:0/534804324 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f49c4001a50 con 0x7f49cc05feb0 2020-12-05T17:39:18.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49caffd700 1 -- 172.21.15.32:0/534804324 >> v1:172.21.15.32:6789/0 conn(0x7f49cc05feb0 legacy=0x7f49cc0b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:18.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49caffd700 1 --2- 172.21.15.32:0/534804324 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49ac0038e0 0x7f49ac003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:18.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49caffd700 1 --2- 172.21.15.32:0/534804324 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49cc05f3b0 0x7f49ac0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:18.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49caffd700 1 --2- 172.21.15.32:0/534804324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49ac0049c0 0x7f49ac004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:18.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49caffd700 1 -- 172.21.15.32:0/534804324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc061a20 con 0x7f49ac0049c0 2020-12-05T17:39:18.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49caffd700 1 -- 172.21.15.32:0/534804324 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac005360 con 0x7f49ac0038e0 2020-12-05T17:39:18.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49caffd700 1 -- 172.21.15.32:0/534804324 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac00b770 con 0x7f49cc05f3b0 2020-12-05T17:39:18.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49cbfff700 1 --2- 172.21.15.32:0/534804324 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49ac0038e0 0x7f49ac003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:18.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49d0cf6700 1 --2- 172.21.15.32:0/534804324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49ac0049c0 0x7f49ac004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:18.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49cb7fe700 1 --2- 172.21.15.32:0/534804324 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49cc05f3b0 0x7f49ac0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:18.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49d0cf6700 1 -- 172.21.15.32:0/534804324 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49cc05f3b0 msgr2=0x7f49ac0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:18.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49d0cf6700 1 --2- 172.21.15.32:0/534804324 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49cc05f3b0 0x7f49ac0044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49d0cf6700 1 -- 172.21.15.32:0/534804324 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49ac0038e0 msgr2=0x7f49ac003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:18.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49d0cf6700 1 --2- 172.21.15.32:0/534804324 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49ac0038e0 0x7f49ac003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.244 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49d0cf6700 1 -- 172.21.15.32:0/534804324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac003230 con 0x7f49ac0049c0 2020-12-05T17:39:18.244 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49cbfff700 1 --2- 172.21.15.32:0/534804324 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49ac0038e0 0x7f49ac003ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:18.244 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49d0cf6700 1 --2- 172.21.15.32:0/534804324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49ac0049c0 0x7f49ac004e40 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7f49ac0054a0 tx=0x7f49cc0b0a70).ready entity=mon.0 client_cookie=be2c3157960b28f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:18.244 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49caffd700 1 -- 172.21.15.32:0/534804324 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49c4005b70 con 0x7f49ac0049c0 2020-12-05T17:39:18.244 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49caffd700 1 -- 172.21.15.32:0/534804324 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c4005cd0 con 0x7f49ac0049c0 2020-12-05T17:39:18.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49d14f7700 1 -- 172.21.15.32:0/534804324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49ac0049c0 msgr2=0x7f49ac004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:18.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49d14f7700 1 --2- 172.21.15.32:0/534804324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49ac0049c0 0x7f49ac004e40 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7f49ac0054a0 tx=0x7f49cc0b0a70).stop 2020-12-05T17:39:18.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49caffd700 1 -- 172.21.15.32:0/534804324 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49c40072c0 con 0x7f49ac0049c0 2020-12-05T17:39:18.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49d0cf6700 1 -- 172.21.15.32:0/534804324 reap_dead start 2020-12-05T17:39:18.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49d14f7700 1 -- 172.21.15.32:0/534804324 shutdown_connections 2020-12-05T17:39:18.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49d14f7700 1 -- 172.21.15.32:0/534804324 >> 172.21.15.32:0/534804324 conn(0x7f49cc0b8ee0 msgr2=0x7f49cc0b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:18.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49d14f7700 1 -- 172.21.15.32:0/534804324 shutdown_connections 2020-12-05T17:39:18.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.231+0000 7f49d14f7700 1 -- 172.21.15.32:0/534804324 wait complete. 2020-12-05T17:39:18.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49d14f7700 1 Processor -- start 2020-12-05T17:39:18.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49d14f7700 1 -- start start 2020-12-05T17:39:18.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49d14f7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49cc145e40 0x7f49cc13e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:18.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49d14f7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49cc151da0 0x7f49cc13e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:18.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49d14f7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49cc145590 0x7f49cc147bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:18.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49d14f7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc144bc0 con 0x7f49cc151da0 2020-12-05T17:39:18.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49d14f7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc146220 con 0x7f49cc145e40 2020-12-05T17:39:18.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49d14f7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc1441d0 con 0x7f49cc145590 2020-12-05T17:39:18.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49cb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49cc151da0 0x7f49cc13e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:18.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49cb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49cc151da0 0x7f49cc13e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37914/0 (socket says 172.21.15.32:37914) 2020-12-05T17:39:18.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49cb7fe700 1 -- 172.21.15.32:0/4276206281 learned_addr learned my addr 172.21.15.32:0/4276206281 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:18.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49cbfff700 1 --2- 172.21.15.32:0/4276206281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49cc145e40 0x7f49cc13e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:18.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49d0cf6700 1 --2- 172.21.15.32:0/4276206281 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49cc145590 0x7f49cc147bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:18.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49cb7fe700 1 -- 172.21.15.32:0/4276206281 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49cc145590 msgr2=0x7f49cc147bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:18.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49cb7fe700 1 --2- 172.21.15.32:0/4276206281 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49cc145590 0x7f49cc147bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49cb7fe700 1 -- 172.21.15.32:0/4276206281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49cc145e40 msgr2=0x7f49cc13e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:18.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49cb7fe700 1 --2- 172.21.15.32:0/4276206281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49cc145e40 0x7f49cc13e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49cb7fe700 1 -- 172.21.15.32:0/4276206281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc144470 con 0x7f49cc151da0 2020-12-05T17:39:18.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49cb7fe700 1 --2- 172.21.15.32:0/4276206281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49cc151da0 0x7f49cc13e950 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7f49bc00d000 tx=0x7f49bc007b60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:18.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49c97fa700 1 -- 172.21.15.32:0/4276206281 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49bc00f690 con 0x7f49cc151da0 2020-12-05T17:39:18.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49c97fa700 1 -- 172.21.15.32:0/4276206281 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49bc00f7f0 con 0x7f49cc151da0 2020-12-05T17:39:18.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49d14f7700 1 -- 172.21.15.32:0/4276206281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49cc146810 con 0x7f49cc151da0 2020-12-05T17:39:18.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49c97fa700 1 -- 172.21.15.32:0/4276206281 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49bc00fb50 con 0x7f49cc151da0 2020-12-05T17:39:18.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.235+0000 7f49d14f7700 1 -- 172.21.15.32:0/4276206281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49cc144980 con 0x7f49cc151da0 2020-12-05T17:39:18.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.239+0000 7f49d14f7700 1 -- 172.21.15.32:0/4276206281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49cc057050 con 0x7f49cc151da0 2020-12-05T17:39:18.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.239+0000 7f49c97fa700 1 -- 172.21.15.32:0/4276206281 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f49bc01a030 con 0x7f49cc151da0 2020-12-05T17:39:18.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.239+0000 7f49c97fa700 1 --2- 172.21.15.32:0/4276206281 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49b0055500 0x7f49b00579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:18.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.239+0000 7f49cbfff700 1 --2- 172.21.15.32:0/4276206281 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49b0055500 0x7f49b00579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:18.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.239+0000 7f49c97fa700 1 -- 172.21.15.32:0/4276206281 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f49bc06d880 con 0x7f49cc151da0 2020-12-05T17:39:18.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.239+0000 7f49cbfff700 1 --2- 172.21.15.32:0/4276206281 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49b0055500 0x7f49b00579c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f49b4008000 tx=0x7f49b4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:18.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.243+0000 7f49c97fa700 1 -- 172.21.15.32:0/4276206281 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f49bc043d60 con 0x7f49cc151da0 2020-12-05T17:39:18.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.435+0000 7f49d14f7700 1 -- 172.21.15.32:0/4276206281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f49cc151ba0 con 0x7f49cc151da0 2020-12-05T17:39:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.439+0000 7f49c97fa700 1 -- 172.21.15.32:0/4276206281 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v28) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f49bc018070 con 0x7f49cc151da0 2020-12-05T17:39:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:18.446 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:39:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2020-12-05T17:39:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2020-12-05T17:39:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfpb/aExAAdMKW4yIgC2zhkzyW6l8xug==", 2020-12-05T17:39:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:39:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:39:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:39:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2020-12-05T17:39:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfBKTYGRAAaPkezwmTiAvkUfTuDGDNZA==", 2020-12-05T17:39:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:39:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:39:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:39:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2020-12-05T17:39:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHw8tflo5EKhAAwV9D64fAoY3uvXCm+Aksag==", 2020-12-05T17:39:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:39:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:39:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2020-12-05T17:39:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDKw8tfTjEiERAAkW6mkgApoXwyF49kPjZBGg==", 2020-12-05T17:39:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:39:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:39:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2020-12-05T17:39:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDMw8tfLEzBMxAAiJ2NifsOLEkt1e3vmhwSTA==", 2020-12-05T17:39:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:39:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:39:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2020-12-05T17:39:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPw8tfV+vnGxAAQHvXJRW5X9Vhe/eYJBJ/uQ==", 2020-12-05T17:39:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:39:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:39:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2020-12-05T17:39:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUw8tfN8wMJxAApc5LY9HGDeIV/o0eil/g4A==", 2020-12-05T17:39:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:39:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:39:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2020-12-05T17:39:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXw8tfvZ5dDxAAtAfftExkljlBvn7ZSoD/Og==", 2020-12-05T17:39:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:39:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:39:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2020-12-05T17:39:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDZw8tfhXFzMhAAZXNwNGfKv360Tq4N/DX5Mw==", 2020-12-05T17:39:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:39:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:39:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2020-12-05T17:39:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDcw8tfROfSGhAA/Y2Y1b4k5zQju0MM7LASeQ==", 2020-12-05T17:39:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:39:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:39:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2020-12-05T17:39:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfWDp/HxAAmvLPj7C3WDgj0OC2o7lBJQ==", 2020-12-05T17:39:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:39:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:39:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:39:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2020-12-05T17:39:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfugnmIRAAW6+/QGMLkHw+Z97RHpV4kw==", 2020-12-05T17:39:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:39:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:39:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:39:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2020-12-05T17:39:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfojyqJBAAyta+dSy2yGnC1kKe3YOAYw==", 2020-12-05T17:39:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:39:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:39:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:39:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2020-12-05T17:39:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDFw8tfES0PLBAAA1cUV44snAUDF2TyAmXImw==", 2020-12-05T17:39:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:39:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:39:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:39:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2020-12-05T17:39:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfDr8mERAAS7bpAQXgW78W8GDq1v1zaA==", 2020-12-05T17:39:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:39:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:39:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2020-12-05T17:39:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tf3HmcDhAATNKHhH9zgasedaNEzgG0OQ==", 2020-12-05T17:39:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:39:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:39:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:18.479 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:39:18.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.439+0000 7f49d14f7700 1 -- 172.21.15.32:0/4276206281 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49b0055500 msgr2=0x7f49b00579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:18.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.439+0000 7f49d14f7700 1 --2- 172.21.15.32:0/4276206281 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49b0055500 0x7f49b00579c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f49b4008000 tx=0x7f49b4008040).stop 2020-12-05T17:39:18.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.439+0000 7f49d14f7700 1 -- 172.21.15.32:0/4276206281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49cc151da0 msgr2=0x7f49cc13e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:18.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.439+0000 7f49d14f7700 1 --2- 172.21.15.32:0/4276206281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49cc151da0 0x7f49cc13e950 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7f49bc00d000 tx=0x7f49bc007b60).stop 2020-12-05T17:39:18.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.439+0000 7f49d14f7700 1 -- 172.21.15.32:0/4276206281 shutdown_connections 2020-12-05T17:39:18.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.439+0000 7f49d14f7700 1 --2- 172.21.15.32:0/4276206281 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f49b0055500 0x7f49b00579c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.439+0000 7f49d14f7700 1 --2- 172.21.15.32:0/4276206281 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f49cc145590 0x7f49cc147bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.439+0000 7f49d14f7700 1 --2- 172.21.15.32:0/4276206281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f49cc145e40 0x7f49cc13e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.439+0000 7f49d14f7700 1 --2- 172.21.15.32:0/4276206281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49cc151da0 0x7f49cc13e950 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.439+0000 7f49d14f7700 1 -- 172.21.15.32:0/4276206281 >> 172.21.15.32:0/4276206281 conn(0x7f49cc0b8ee0 msgr2=0x7f49cc140570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:18.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.439+0000 7f49d14f7700 1 -- 172.21.15.32:0/4276206281 shutdown_connections 2020-12-05T17:39:18.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.439+0000 7f49d14f7700 1 -- 172.21.15.32:0/4276206281 wait complete. 2020-12-05T17:39:18.484 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:39:18.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.595+0000 7fb3f6fcd700 1 Processor -- start 2020-12-05T17:39:18.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.595+0000 7fb3f6fcd700 1 -- start start 2020-12-05T17:39:18.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.595+0000 7fb3f6fcd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3f00ad1b0 0x7fb3f0143350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:18.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.595+0000 7fb3f6fcd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f00ae8b0 con 0x7fb3f0143840 2020-12-05T17:39:18.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.595+0000 7fb3f6fcd700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f00b6460 con 0x7fb3f00b6080 2020-12-05T17:39:18.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.595+0000 7fb3f6fcd700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f00b0da0 con 0x7fb3f00ad1b0 2020-12-05T17:39:18.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.595+0000 7fb3f556a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb3f0143840 0x7fb3f00aece0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40572/0 (socket says 172.21.15.32:40572) 2020-12-05T17:39:18.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.595+0000 7fb3f556a700 1 -- 172.21.15.32:0/1786390852 learned_addr learned my addr 172.21.15.32:0/1786390852 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:18.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.595+0000 7fb3effff700 1 --2- 172.21.15.32:0/1786390852 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3f00ad1b0 0x7fb3f0143350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:18.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.595+0000 7fb3ef7fe700 1 -- 172.21.15.32:0/1786390852 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1250695355 0 0) 0x7fb3f00ae8b0 con 0x7fb3f0143840 2020-12-05T17:39:18.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.595+0000 7fb3ef7fe700 1 -- 172.21.15.32:0/1786390852 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3dc003230 con 0x7fb3f0143840 2020-12-05T17:39:18.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.595+0000 7fb3ef7fe700 1 -- 172.21.15.32:0/1786390852 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2288624030 0 0) 0x7fb3f00b6460 con 0x7fb3f00b6080 2020-12-05T17:39:18.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.595+0000 7fb3ef7fe700 1 -- 172.21.15.32:0/1786390852 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3f00ae8b0 con 0x7fb3f00b6080 2020-12-05T17:39:18.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.595+0000 7fb3effff700 1 -- 172.21.15.32:0/1786390852 >> v1:172.21.15.61:6789/0 conn(0x7fb3f00b6080 legacy=0x7fb3f0142c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:18.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.595+0000 7fb3effff700 1 -- 172.21.15.32:0/1786390852 >> v1:172.21.15.32:6789/0 conn(0x7fb3f0143840 legacy=0x7fb3f00aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:18.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.595+0000 7fb3effff700 1 -- 172.21.15.32:0/1786390852 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f0140490 con 0x7fb3f00ad1b0 2020-12-05T17:39:18.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.599+0000 7fb3effff700 1 --2- 172.21.15.32:0/1786390852 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3f00ad1b0 0x7fb3f0143350 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fb3e4002f30 tx=0x7fb3e4006dc0).ready entity=mon.2 client_cookie=580cdf6d0a0ebd6c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:18.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.599+0000 7fb3ef7fe700 1 -- 172.21.15.32:0/1786390852 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3e4009730 con 0x7fb3f00ad1b0 2020-12-05T17:39:18.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.599+0000 7fb3ef7fe700 1 -- 172.21.15.32:0/1786390852 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e4009890 con 0x7fb3f00ad1b0 2020-12-05T17:39:18.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.599+0000 7fb3ef7fe700 1 -- 172.21.15.32:0/1786390852 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3e4009bf0 con 0x7fb3f00ad1b0 2020-12-05T17:39:18.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.599+0000 7fb3f6fcd700 1 -- 172.21.15.32:0/1786390852 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3f00ad1b0 msgr2=0x7fb3f0143350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:18.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.599+0000 7fb3f6fcd700 1 --2- 172.21.15.32:0/1786390852 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3f00ad1b0 0x7fb3f0143350 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fb3e4002f30 tx=0x7fb3e4006dc0).stop 2020-12-05T17:39:18.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.599+0000 7fb3f6fcd700 1 -- 172.21.15.32:0/1786390852 shutdown_connections 2020-12-05T17:39:18.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.599+0000 7fb3f6fcd700 1 --2- 172.21.15.32:0/1786390852 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3f00ad1b0 0x7fb3f0143350 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.599+0000 7fb3f6fcd700 1 -- 172.21.15.32:0/1786390852 >> 172.21.15.32:0/1786390852 conn(0x7fb3f0069cb0 msgr2=0x7fb3f00604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:18.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.599+0000 7fb3f6fcd700 1 -- 172.21.15.32:0/1786390852 shutdown_connections 2020-12-05T17:39:18.609 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.599+0000 7fb3f6fcd700 1 -- 172.21.15.32:0/1786390852 wait complete. 2020-12-05T17:39:18.609 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3f6fcd700 1 Processor -- start 2020-12-05T17:39:18.609 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3f6fcd700 1 -- start start 2020-12-05T17:39:18.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3f6fcd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3f00ad1b0 0x7fb3f00650e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:18.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3f6fcd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3f00b6080 0x7fb3f0141e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:18.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3f6fcd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3f0143840 0x7fb3f013d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:18.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3f6fcd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f01104f0 con 0x7fb3f00ad1b0 2020-12-05T17:39:18.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3f6fcd700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f0110790 con 0x7fb3f00b6080 2020-12-05T17:39:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3f6fcd700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f0110a30 con 0x7fb3f0143840 2020-12-05T17:39:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3f4d69700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3f00ad1b0 0x7fb3f00650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3f4d69700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3f00ad1b0 0x7fb3f00650e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37926/0 (socket says 172.21.15.32:37926) 2020-12-05T17:39:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3f4d69700 1 -- 172.21.15.32:0/3727315852 learned_addr learned my addr 172.21.15.32:0/3727315852 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3effff700 1 --2- 172.21.15.32:0/3727315852 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3f00b6080 0x7fb3f0141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3f556a700 1 --2- 172.21.15.32:0/3727315852 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3f0143840 0x7fb3f013d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3f4d69700 1 -- 172.21.15.32:0/3727315852 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3f0143840 msgr2=0x7fb3f013d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3f4d69700 1 --2- 172.21.15.32:0/3727315852 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3f0143840 0x7fb3f013d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3f4d69700 1 -- 172.21.15.32:0/3727315852 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3f00b6080 msgr2=0x7fb3f0141e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3f4d69700 1 --2- 172.21.15.32:0/3727315852 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3f00b6080 0x7fb3f0141e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3f4d69700 1 -- 172.21.15.32:0/3727315852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f01101e0 con 0x7fb3f00ad1b0 2020-12-05T17:39:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3effff700 1 --2- 172.21.15.32:0/3727315852 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3f00b6080 0x7fb3f0141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3f556a700 1 --2- 172.21.15.32:0/3727315852 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3f0143840 0x7fb3f013d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3f4d69700 1 --2- 172.21.15.32:0/3727315852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3f00ad1b0 0x7fb3f00650e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7fb3e00010d0 tx=0x7fb3e0007b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3edffb700 1 -- 172.21.15.32:0/3727315852 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3e000e4a0 con 0x7fb3f00ad1b0 2020-12-05T17:39:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3f6fcd700 1 -- 172.21.15.32:0/3727315852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f0110ed0 con 0x7fb3f00ad1b0 2020-12-05T17:39:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3edffb700 1 -- 172.21.15.32:0/3727315852 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e000e600 con 0x7fb3f00ad1b0 2020-12-05T17:39:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3f6fcd700 1 -- 172.21.15.32:0/3727315852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f013d9a0 con 0x7fb3f00ad1b0 2020-12-05T17:39:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.603+0000 7fb3edffb700 1 -- 172.21.15.32:0/3727315852 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3e000e960 con 0x7fb3f00ad1b0 2020-12-05T17:39:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.607+0000 7fb3f6fcd700 1 -- 172.21.15.32:0/3727315852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f0110bf0 con 0x7fb3f00ad1b0 2020-12-05T17:39:18.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.607+0000 7fb3edffb700 1 -- 172.21.15.32:0/3727315852 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb3e0019030 con 0x7fb3f00ad1b0 2020-12-05T17:39:18.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.607+0000 7fb3edffb700 1 --2- 172.21.15.32:0/3727315852 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3d0055330 0x7fb3d00577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:18.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.607+0000 7fb3edffb700 1 -- 172.21.15.32:0/3727315852 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fb3e006d360 con 0x7fb3f00ad1b0 2020-12-05T17:39:18.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.607+0000 7fb3effff700 1 --2- 172.21.15.32:0/3727315852 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3d0055330 0x7fb3d00577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:18.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.607+0000 7fb3effff700 1 --2- 172.21.15.32:0/3727315852 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3d0055330 0x7fb3d00577f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7fb3e4019000 tx=0x7fb3e4015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:18.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.615+0000 7fb3edffb700 1 -- 172.21.15.32:0/3727315852 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb3e0043650 con 0x7fb3f00ad1b0 2020-12-05T17:39:18.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.759+0000 7fb3f6fcd700 1 -- 172.21.15.32:0/3727315852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3f0057050 con 0x7fb3f00ad1b0 2020-12-05T17:39:18.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.763+0000 7fb3edffb700 1 -- 172.21.15.32:0/3727315852 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+13415 (secure 0 0 0) 0x7fb3e0046c70 con 0x7fb3f00ad1b0 2020-12-05T17:39:18.768 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:18.768 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":88,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:39:18.084948+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:39:18.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.763+0000 7fb3f6fcd700 1 -- 172.21.15.32:0/3727315852 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3d0055330 msgr2=0x7fb3d00577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:18.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.763+0000 7fb3f6fcd700 1 --2- 172.21.15.32:0/3727315852 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3d0055330 0x7fb3d00577f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7fb3e4019000 tx=0x7fb3e4015000).stop 2020-12-05T17:39:18.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.763+0000 7fb3f6fcd700 1 -- 172.21.15.32:0/3727315852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3f00ad1b0 msgr2=0x7fb3f00650e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:18.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.763+0000 7fb3f6fcd700 1 --2- 172.21.15.32:0/3727315852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3f00ad1b0 0x7fb3f00650e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7fb3e00010d0 tx=0x7fb3e0007b30).stop 2020-12-05T17:39:18.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.763+0000 7fb3f6fcd700 1 -- 172.21.15.32:0/3727315852 shutdown_connections 2020-12-05T17:39:18.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.763+0000 7fb3f6fcd700 1 --2- 172.21.15.32:0/3727315852 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3d0055330 0x7fb3d00577f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.763+0000 7fb3f6fcd700 1 --2- 172.21.15.32:0/3727315852 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3f0143840 0x7fb3f013d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.763+0000 7fb3f6fcd700 1 --2- 172.21.15.32:0/3727315852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3f00ad1b0 0x7fb3f00650e0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.763+0000 7fb3f6fcd700 1 --2- 172.21.15.32:0/3727315852 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3f00b6080 0x7fb3f0141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.763+0000 7fb3f6fcd700 1 -- 172.21.15.32:0/3727315852 >> 172.21.15.32:0/3727315852 conn(0x7fb3f0069cb0 msgr2=0x7fb3f00aec00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.763+0000 7fb3f6fcd700 1 -- 172.21.15.32:0/3727315852 shutdown_connections 2020-12-05T17:39:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.763+0000 7fb3f6fcd700 1 -- 172.21.15.32:0/3727315852 wait complete. 2020-12-05T17:39:18.781 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-12-05T17:39:18.782 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:39:18.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969e85c700 1 Processor -- start 2020-12-05T17:39:18.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969e85c700 1 -- start start 2020-12-05T17:39:18.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969e85c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9698061ba0 0x7f96980b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:18.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969e85c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96980b1720 0x7f969814a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:18.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969e85c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9698061f80 con 0x7f969814aeb0 2020-12-05T17:39:18.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969e85c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9698147ad0 con 0x7f96980b1720 2020-12-05T17:39:18.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969e85c700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f969814a280 con 0x7f9698061ba0 2020-12-05T17:39:18.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969d059700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96980b1720 0x7f969814a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:18.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969d059700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96980b1720 0x7f969814a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37936/0 (socket says 172.21.15.32:37936) 2020-12-05T17:39:18.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969d059700 1 -- 172.21.15.32:0/4286703496 learned_addr learned my addr 172.21.15.32:0/4286703496 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:18.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969d85a700 1 --2- 172.21.15.32:0/4286703496 >> v2:172.21.15.61:3300/0 conn(0x7f9698061ba0 0x7f96980b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:18.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969d059700 1 -- 172.21.15.32:0/4286703496 >> v2:172.21.15.61:3300/0 conn(0x7f9698061ba0 msgr2=0x7f96980b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:18.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969d059700 1 --2- 172.21.15.32:0/4286703496 >> v2:172.21.15.61:3300/0 conn(0x7f9698061ba0 0x7f96980b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969d059700 1 -- 172.21.15.32:0/4286703496 >> v1:172.21.15.61:6789/0 conn(0x7f969814aeb0 legacy=0x7f96981460b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:18.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969d059700 1 -- 172.21.15.32:0/4286703496 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969814a520 con 0x7f96980b1720 2020-12-05T17:39:18.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969d059700 1 --2- 172.21.15.32:0/4286703496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96980b1720 0x7f969814a9c0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f9688002f30 tx=0x7f9688006e80).ready entity=mon.0 client_cookie=6a1c9cf7d099bb21 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:18.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969c858700 1 -- 172.21.15.32:0/4286703496 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96880097f0 con 0x7f96980b1720 2020-12-05T17:39:18.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969c858700 1 -- 172.21.15.32:0/4286703496 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9688009950 con 0x7f96980b1720 2020-12-05T17:39:18.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969c858700 1 -- 172.21.15.32:0/4286703496 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9688009cb0 con 0x7f96980b1720 2020-12-05T17:39:18.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969e85c700 1 -- 172.21.15.32:0/4286703496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96980b1720 msgr2=0x7f969814a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:18.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969e85c700 1 --2- 172.21.15.32:0/4286703496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96980b1720 0x7f969814a9c0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f9688002f30 tx=0x7f9688006e80).stop 2020-12-05T17:39:18.904 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969e85c700 1 -- 172.21.15.32:0/4286703496 shutdown_connections 2020-12-05T17:39:18.904 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969e85c700 1 --2- 172.21.15.32:0/4286703496 >> v2:172.21.15.61:3300/0 conn(0x7f9698061ba0 0x7f96980b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969e85c700 1 --2- 172.21.15.32:0/4286703496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96980b1720 0x7f969814a9c0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969e85c700 1 -- 172.21.15.32:0/4286703496 >> 172.21.15.32:0/4286703496 conn(0x7f96980b6d10 msgr2=0x7f96980b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:18.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969e85c700 1 -- 172.21.15.32:0/4286703496 shutdown_connections 2020-12-05T17:39:18.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.891+0000 7f969e85c700 1 -- 172.21.15.32:0/4286703496 wait complete. 2020-12-05T17:39:18.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f969e85c700 1 Processor -- start 2020-12-05T17:39:18.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f969e85c700 1 -- start start 2020-12-05T17:39:18.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f969e85c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9698061ba0 0x7f9698065060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:18.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f969e85c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96980b1720 0x7f9698064690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:18.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f969e85c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f969814aeb0 0x7f96980aced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:18.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f969e85c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969814e040 con 0x7f96980b1720 2020-12-05T17:39:18.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f969e85c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969814b940 con 0x7f969814aeb0 2020-12-05T17:39:18.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f969e85c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9698065730 con 0x7f9698061ba0 2020-12-05T17:39:18.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f969d059700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96980b1720 0x7f9698064690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:18.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f969d059700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96980b1720 0x7f9698064690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37942/0 (socket says 172.21.15.32:37942) 2020-12-05T17:39:18.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f969d059700 1 -- 172.21.15.32:0/3373065625 learned_addr learned my addr 172.21.15.32:0/3373065625 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:18.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f969d85a700 1 --2- 172.21.15.32:0/3373065625 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9698061ba0 0x7f9698065060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:18.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f969e05b700 1 --2- 172.21.15.32:0/3373065625 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f969814aeb0 0x7f96980aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:18.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f969d059700 1 -- 172.21.15.32:0/3373065625 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9698061ba0 msgr2=0x7f9698065060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:18.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f969d059700 1 --2- 172.21.15.32:0/3373065625 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9698061ba0 0x7f9698065060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f969d059700 1 -- 172.21.15.32:0/3373065625 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f969814aeb0 msgr2=0x7f96980aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:18.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f969d059700 1 --2- 172.21.15.32:0/3373065625 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f969814aeb0 0x7f96980aced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:18.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f969d059700 1 -- 172.21.15.32:0/3373065625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96980ad3c0 con 0x7f96980b1720 2020-12-05T17:39:18.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f969e05b700 1 --2- 172.21.15.32:0/3373065625 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f969814aeb0 0x7f96980aced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:39:18.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f969d059700 1 --2- 172.21.15.32:0/3373065625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96980b1720 0x7f9698064690 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f9688002f30 tx=0x7f9688015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:18.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f9686ffd700 1 -- 172.21.15.32:0/3373065625 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9688011d30 con 0x7f96980b1720 2020-12-05T17:39:18.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f9686ffd700 1 -- 172.21.15.32:0/3373065625 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9688011e90 con 0x7f96980b1720 2020-12-05T17:39:18.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f9686ffd700 1 -- 172.21.15.32:0/3373065625 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f968801e650 con 0x7f96980b1720 2020-12-05T17:39:18.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f969e85c700 1 -- 172.21.15.32:0/3373065625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969814b3c0 con 0x7f96980b1720 2020-12-05T17:39:18.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.895+0000 7f969e85c700 1 -- 172.21.15.32:0/3373065625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9698065bd0 con 0x7f96980b1720 2020-12-05T17:39:18.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.899+0000 7f9686ffd700 1 -- 172.21.15.32:0/3373065625 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f968801a040 con 0x7f96980b1720 2020-12-05T17:39:18.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.899+0000 7f969e85c700 1 -- 172.21.15.32:0/3373065625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96980658f0 con 0x7f96980b1720 2020-12-05T17:39:18.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.899+0000 7f9686ffd700 1 --2- 172.21.15.32:0/3373065625 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9678055440 0x7f9678057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:18.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.899+0000 7f9686ffd700 1 -- 172.21.15.32:0/3373065625 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f968806edc0 con 0x7f96980b1720 2020-12-05T17:39:18.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.899+0000 7f969d85a700 1 --2- 172.21.15.32:0/3373065625 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9678055440 0x7f9678057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:18.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.899+0000 7f969d85a700 1 --2- 172.21.15.32:0/3373065625 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9678055440 0x7f9678057900 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f969400c000 tx=0x7f9694008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:18.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:18.903+0000 7f9686ffd700 1 -- 172.21.15.32:0/3373065625 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9688045370 con 0x7f96980b1720 2020-12-05T17:39:19.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.047+0000 7f969e85c700 1 -- 172.21.15.32:0/3373065625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f969814bab0 con 0x7f96980b1720 2020-12-05T17:39:19.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.047+0000 7f9686ffd700 1 -- 172.21.15.32:0/3373065625 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v88) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9688023020 con 0x7f96980b1720 2020-12-05T17:39:19.054 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:39:19.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.051+0000 7f969e85c700 1 -- 172.21.15.32:0/3373065625 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9678055440 msgr2=0x7f9678057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:19.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.051+0000 7f969e85c700 1 --2- 172.21.15.32:0/3373065625 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9678055440 0x7f9678057900 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f969400c000 tx=0x7f9694008040).stop 2020-12-05T17:39:19.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.051+0000 7f969e85c700 1 -- 172.21.15.32:0/3373065625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96980b1720 msgr2=0x7f9698064690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:19.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.051+0000 7f969e85c700 1 --2- 172.21.15.32:0/3373065625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96980b1720 0x7f9698064690 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f9688002f30 tx=0x7f9688015000).stop 2020-12-05T17:39:19.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.051+0000 7f969e85c700 1 -- 172.21.15.32:0/3373065625 shutdown_connections 2020-12-05T17:39:19.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.051+0000 7f969e85c700 1 --2- 172.21.15.32:0/3373065625 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f969814aeb0 0x7f96980aced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.051+0000 7f969e85c700 1 --2- 172.21.15.32:0/3373065625 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9678055440 0x7f9678057900 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.051+0000 7f969e85c700 1 --2- 172.21.15.32:0/3373065625 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9698061ba0 0x7f9698065060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.051+0000 7f969e85c700 1 --2- 172.21.15.32:0/3373065625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96980b1720 0x7f9698064690 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.051+0000 7f969e85c700 1 -- 172.21.15.32:0/3373065625 >> 172.21.15.32:0/3373065625 conn(0x7f96980b6d10 msgr2=0x7f96980b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:19.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.051+0000 7f969e85c700 1 -- 172.21.15.32:0/3373065625 shutdown_connections 2020-12-05T17:39:19.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.051+0000 7f969e85c700 1 -- 172.21.15.32:0/3373065625 wait complete. 2020-12-05T17:39:19.068 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:39:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.183+0000 7fd5a325a700 1 Processor -- start 2020-12-05T17:39:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.183+0000 7fd5a325a700 1 -- start start 2020-12-05T17:39:19.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.183+0000 7fd5a325a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd59c05f3b0 0x7fd59c05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:19.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.183+0000 7fd5a325a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd59c05feb0 0x7fd59c061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:19.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.183+0000 7fd5a325a700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd59c0b2df0 con 0x7fd59c0b2fb0 2020-12-05T17:39:19.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.183+0000 7fd5a325a700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd59c0b3c90 con 0x7fd59c05f3b0 2020-12-05T17:39:19.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.183+0000 7fd5a325a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c0b2b50 con 0x7fd59c05feb0 2020-12-05T17:39:19.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a2a59700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fd59c0b2fb0 0x7fd59c13c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:53240/0 (socket says 172.21.15.32:53240) 2020-12-05T17:39:19.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a2a59700 1 -- 172.21.15.32:0/3156342585 learned_addr learned my addr 172.21.15.32:0/3156342585 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:19.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a1a57700 1 --2- 172.21.15.32:0/3156342585 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd59c05feb0 0x7fd59c061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:19.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a2258700 1 --2- 172.21.15.32:0/3156342585 >> v2:172.21.15.61:3300/0 conn(0x7fd59c05f3b0 0x7fd59c05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:19.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a1256700 1 -- 172.21.15.32:0/3156342585 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967396676 0 0) 0x7fd59c0b2df0 con 0x7fd59c0b2fb0 2020-12-05T17:39:19.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a1256700 1 -- 172.21.15.32:0/3156342585 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd584003230 con 0x7fd59c0b2fb0 2020-12-05T17:39:19.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a1a57700 1 -- 172.21.15.32:0/3156342585 >> v2:172.21.15.61:3300/0 conn(0x7fd59c05f3b0 msgr2=0x7fd59c05f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:19.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a1a57700 1 --2- 172.21.15.32:0/3156342585 >> v2:172.21.15.61:3300/0 conn(0x7fd59c05f3b0 0x7fd59c05f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a1a57700 1 -- 172.21.15.32:0/3156342585 >> v1:172.21.15.61:6789/0 conn(0x7fd59c0b2fb0 legacy=0x7fd59c13c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:19.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a1a57700 1 -- 172.21.15.32:0/3156342585 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59c142820 con 0x7fd59c05feb0 2020-12-05T17:39:19.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a1a57700 1 --2- 172.21.15.32:0/3156342585 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd59c05feb0 0x7fd59c061780 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fd590002f30 tx=0x7fd590006dc0).ready entity=mon.2 client_cookie=bb5282dd5ed3f9dd server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:19.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a1256700 1 -- 172.21.15.32:0/3156342585 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5900096a0 con 0x7fd59c05feb0 2020-12-05T17:39:19.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a1256700 1 -- 172.21.15.32:0/3156342585 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd590009800 con 0x7fd59c05feb0 2020-12-05T17:39:19.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a325a700 1 -- 172.21.15.32:0/3156342585 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd59c05feb0 msgr2=0x7fd59c061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:19.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a1256700 1 -- 172.21.15.32:0/3156342585 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd590009b60 con 0x7fd59c05feb0 2020-12-05T17:39:19.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a325a700 1 --2- 172.21.15.32:0/3156342585 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd59c05feb0 0x7fd59c061780 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fd590002f30 tx=0x7fd590006dc0).stop 2020-12-05T17:39:19.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a325a700 1 -- 172.21.15.32:0/3156342585 shutdown_connections 2020-12-05T17:39:19.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a325a700 1 --2- 172.21.15.32:0/3156342585 >> v2:172.21.15.61:3300/0 conn(0x7fd59c05f3b0 0x7fd59c05f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a325a700 1 --2- 172.21.15.32:0/3156342585 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd59c05feb0 0x7fd59c061780 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a325a700 1 -- 172.21.15.32:0/3156342585 >> 172.21.15.32:0/3156342585 conn(0x7fd59c0b8f20 msgr2=0x7fd59c0b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:19.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a325a700 1 -- 172.21.15.32:0/3156342585 shutdown_connections 2020-12-05T17:39:19.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a325a700 1 -- 172.21.15.32:0/3156342585 wait complete. 2020-12-05T17:39:19.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a325a700 1 Processor -- start 2020-12-05T17:39:19.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.187+0000 7fd5a325a700 1 -- start start 2020-12-05T17:39:19.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd5a325a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd59c05f3b0 0x7fd59c1422b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:19.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd5a325a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd59c05feb0 0x7fd59c146890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:19.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd5a325a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd59c0b2fb0 0x7fd59c146190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:19.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd5a325a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c155410 con 0x7fd59c05feb0 2020-12-05T17:39:19.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd5a325a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c1556b0 con 0x7fd59c05f3b0 2020-12-05T17:39:19.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd5a325a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c141720 con 0x7fd59c0b2fb0 2020-12-05T17:39:19.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd5a1a57700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd59c05feb0 0x7fd59c146890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:19.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd5a1a57700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd59c05feb0 0x7fd59c146890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37956/0 (socket says 172.21.15.32:37956) 2020-12-05T17:39:19.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd5a1a57700 1 -- 172.21.15.32:0/683661182 learned_addr learned my addr 172.21.15.32:0/683661182 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:19.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd5a2258700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd59c05f3b0 0x7fd59c1422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:19.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd5a2a59700 1 --2- 172.21.15.32:0/683661182 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd59c0b2fb0 0x7fd59c146190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:19.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd5a1a57700 1 -- 172.21.15.32:0/683661182 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd59c0b2fb0 msgr2=0x7fd59c146190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:19.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd5a1a57700 1 --2- 172.21.15.32:0/683661182 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd59c0b2fb0 0x7fd59c146190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd5a1a57700 1 -- 172.21.15.32:0/683661182 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd59c05f3b0 msgr2=0x7fd59c1422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:19.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd5a1a57700 1 --2- 172.21.15.32:0/683661182 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd59c05f3b0 0x7fd59c1422b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd5a1a57700 1 -- 172.21.15.32:0/683661182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59c1418e0 con 0x7fd59c05feb0 2020-12-05T17:39:19.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd5a2a59700 1 --2- 172.21.15.32:0/683661182 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd59c0b2fb0 0x7fd59c146190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd5a1a57700 1 --2- 172.21.15.32:0/683661182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd59c05feb0 0x7fd59c146890 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7fd590002f30 tx=0x7fd590015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd58f7fe700 1 -- 172.21.15.32:0/683661182 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd590011b80 con 0x7fd59c05feb0 2020-12-05T17:39:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd58f7fe700 1 -- 172.21.15.32:0/683661182 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd590011ce0 con 0x7fd59c05feb0 2020-12-05T17:39:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd5a325a700 1 -- 172.21.15.32:0/683661182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd59c148e80 con 0x7fd59c05feb0 2020-12-05T17:39:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd58f7fe700 1 -- 172.21.15.32:0/683661182 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd59001c5a0 con 0x7fd59c05feb0 2020-12-05T17:39:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd5a325a700 1 -- 172.21.15.32:0/683661182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd59c1493b0 con 0x7fd59c05feb0 2020-12-05T17:39:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd5a325a700 1 -- 172.21.15.32:0/683661182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd59c155280 con 0x7fd59c05feb0 2020-12-05T17:39:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd58f7fe700 1 -- 172.21.15.32:0/683661182 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd590025030 con 0x7fd59c05feb0 2020-12-05T17:39:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd58f7fe700 1 --2- 172.21.15.32:0/683661182 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd57c0553b0 0x7fd57c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:19.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.191+0000 7fd58f7fe700 1 -- 172.21.15.32:0/683661182 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fd59006ef80 con 0x7fd59c05feb0 2020-12-05T17:39:19.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.195+0000 7fd5a2258700 1 --2- 172.21.15.32:0/683661182 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd57c0553b0 0x7fd57c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:19.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.195+0000 7fd5a2258700 1 --2- 172.21.15.32:0/683661182 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd57c0553b0 0x7fd57c057870 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7fd59800b000 tx=0x7fd598006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:19.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.199+0000 7fd58f7fe700 1 -- 172.21.15.32:0/683661182 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd590045460 con 0x7fd59c05feb0 2020-12-05T17:39:19.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.347+0000 7fd5a325a700 1 -- 172.21.15.32:0/683661182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd59c155870 con 0x7fd59c05feb0 2020-12-05T17:39:19.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.347+0000 7fd58f7fe700 1 -- 172.21.15.32:0/683661182 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v88) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd590023030 con 0x7fd59c05feb0 2020-12-05T17:39:19.352 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:39:19.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.347+0000 7fd5a325a700 1 -- 172.21.15.32:0/683661182 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd57c0553b0 msgr2=0x7fd57c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:19.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.347+0000 7fd5a325a700 1 --2- 172.21.15.32:0/683661182 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd57c0553b0 0x7fd57c057870 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7fd59800b000 tx=0x7fd598006dc0).stop 2020-12-05T17:39:19.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.347+0000 7fd5a325a700 1 -- 172.21.15.32:0/683661182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd59c05feb0 msgr2=0x7fd59c146890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:19.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.347+0000 7fd5a325a700 1 --2- 172.21.15.32:0/683661182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd59c05feb0 0x7fd59c146890 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7fd590002f30 tx=0x7fd590015000).stop 2020-12-05T17:39:19.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.347+0000 7fd5a325a700 1 -- 172.21.15.32:0/683661182 shutdown_connections 2020-12-05T17:39:19.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.347+0000 7fd5a325a700 1 --2- 172.21.15.32:0/683661182 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd57c0553b0 0x7fd57c057870 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.347+0000 7fd5a325a700 1 --2- 172.21.15.32:0/683661182 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd59c0b2fb0 0x7fd59c146190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.347+0000 7fd5a325a700 1 --2- 172.21.15.32:0/683661182 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd59c05f3b0 0x7fd59c1422b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.347+0000 7fd5a325a700 1 --2- 172.21.15.32:0/683661182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd59c05feb0 0x7fd59c146890 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.347+0000 7fd5a325a700 1 -- 172.21.15.32:0/683661182 >> 172.21.15.32:0/683661182 conn(0x7fd59c0b8f20 msgr2=0x7fd59c0b7910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:19.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.347+0000 7fd5a325a700 1 -- 172.21.15.32:0/683661182 shutdown_connections 2020-12-05T17:39:19.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.347+0000 7fd5a325a700 1 -- 172.21.15.32:0/683661182 wait complete. 2020-12-05T17:39:19.366 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-12-05T17:39:19.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.475+0000 7f276e121700 1 Processor -- start 2020-12-05T17:39:19.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.475+0000 7f276e121700 1 -- start start 2020-12-05T17:39:19.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.475+0000 7f276e121700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27680b5f00 0x7f27680b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:19.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.475+0000 7f276e121700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27680ad510 0x7f27680af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.475+0000 7f276e121700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2768142ab0 con 0x7f2768066370 2020-12-05T17:39:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.475+0000 7f276e121700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27680b62e0 con 0x7f27680ad510 2020-12-05T17:39:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.475+0000 7f276e121700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27680afbf0 con 0x7f27680b5f00 2020-12-05T17:39:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.475+0000 7f276c91e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27680ad510 0x7f27680af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.475+0000 7f276c91e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27680ad510 0x7f27680af480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37962/0 (socket says 172.21.15.32:37962) 2020-12-05T17:39:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.475+0000 7f276c91e700 1 -- 172.21.15.32:0/697797026 learned_addr learned my addr 172.21.15.32:0/697797026 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.475+0000 7f276d11f700 1 --2- 172.21.15.32:0/697797026 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27680b5f00 0x7f27680b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.475+0000 7f276c91e700 1 -- 172.21.15.32:0/697797026 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27680b5f00 msgr2=0x7f27680b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.475+0000 7f276c91e700 1 --2- 172.21.15.32:0/697797026 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27680b5f00 0x7f27680b14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.475+0000 7f276c91e700 1 -- 172.21.15.32:0/697797026 >> v1:172.21.15.61:6789/0 conn(0x7f2768066370 legacy=0x7f2768142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.475+0000 7f276c91e700 1 -- 172.21.15.32:0/697797026 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27680afe90 con 0x7f27680ad510 2020-12-05T17:39:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.475+0000 7f276c91e700 1 --2- 172.21.15.32:0/697797026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27680ad510 0x7f27680af480 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7f2750002f30 tx=0x7f2750006e80).ready entity=mon.0 client_cookie=75078518ad4a0484 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.475+0000 7f2767fff700 1 -- 172.21.15.32:0/697797026 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27500097f0 con 0x7f27680ad510 2020-12-05T17:39:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.475+0000 7f2767fff700 1 -- 172.21.15.32:0/697797026 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2750009950 con 0x7f27680ad510 2020-12-05T17:39:19.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.475+0000 7f2767fff700 1 -- 172.21.15.32:0/697797026 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2750009ac0 con 0x7f27680ad510 2020-12-05T17:39:19.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.475+0000 7f276e121700 1 -- 172.21.15.32:0/697797026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27680ad510 msgr2=0x7f27680af480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:19.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.475+0000 7f276e121700 1 --2- 172.21.15.32:0/697797026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27680ad510 0x7f27680af480 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7f2750002f30 tx=0x7f2750006e80).stop 2020-12-05T17:39:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276e121700 1 -- 172.21.15.32:0/697797026 shutdown_connections 2020-12-05T17:39:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276e121700 1 --2- 172.21.15.32:0/697797026 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27680b5f00 0x7f27680b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276e121700 1 --2- 172.21.15.32:0/697797026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27680ad510 0x7f27680af480 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276e121700 1 -- 172.21.15.32:0/697797026 >> 172.21.15.32:0/697797026 conn(0x7f27680604c0 msgr2=0x7f276805c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276e121700 1 -- 172.21.15.32:0/697797026 shutdown_connections 2020-12-05T17:39:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276e121700 1 -- 172.21.15.32:0/697797026 wait complete. 2020-12-05T17:39:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276e121700 1 Processor -- start 2020-12-05T17:39:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276e121700 1 -- start start 2020-12-05T17:39:19.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276e121700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2768066370 0x7f27680b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:19.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276e121700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27680ad510 0x7f2768127950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:19.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276e121700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27680b5f00 0x7f2768151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276e121700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2768142620 con 0x7f2768066370 2020-12-05T17:39:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276e121700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27680b53c0 con 0x7f27680b5f00 2020-12-05T17:39:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276e121700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27680b5660 con 0x7f27680ad510 2020-12-05T17:39:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276d11f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2768066370 0x7f27680b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276c91e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27680ad510 0x7f2768127950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276d11f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2768066370 0x7f27680b4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37968/0 (socket says 172.21.15.32:37968) 2020-12-05T17:39:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276d11f700 1 -- 172.21.15.32:0/2251758406 learned_addr learned my addr 172.21.15.32:0/2251758406 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276d920700 1 --2- 172.21.15.32:0/2251758406 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27680b5f00 0x7f2768151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276d11f700 1 -- 172.21.15.32:0/2251758406 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27680ad510 msgr2=0x7f2768127950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:19.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276d11f700 1 --2- 172.21.15.32:0/2251758406 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27680ad510 0x7f2768127950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276d11f700 1 -- 172.21.15.32:0/2251758406 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27680b5f00 msgr2=0x7f2768151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:19.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276d11f700 1 --2- 172.21.15.32:0/2251758406 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27680b5f00 0x7f2768151c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276d11f700 1 -- 172.21.15.32:0/2251758406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27680b5900 con 0x7f2768066370 2020-12-05T17:39:19.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276c91e700 1 --2- 172.21.15.32:0/2251758406 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27680ad510 0x7f2768127950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:19.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276d11f700 1 --2- 172.21.15.32:0/2251758406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2768066370 0x7f27680b4d90 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7f276000a000 tx=0x7f2760008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:19.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f27667fc700 1 -- 172.21.15.32:0/2251758406 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2760011780 con 0x7f2768066370 2020-12-05T17:39:19.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f27667fc700 1 -- 172.21.15.32:0/2251758406 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27600118e0 con 0x7f2768066370 2020-12-05T17:39:19.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276e121700 1 -- 172.21.15.32:0/2251758406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2768151650 con 0x7f2768066370 2020-12-05T17:39:19.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f27667fc700 1 -- 172.21.15.32:0/2251758406 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2760011c40 con 0x7f2768066370 2020-12-05T17:39:19.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.479+0000 7f276e121700 1 -- 172.21.15.32:0/2251758406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2768142150 con 0x7f2768066370 2020-12-05T17:39:19.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.483+0000 7f276e121700 1 -- 172.21.15.32:0/2251758406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2768057050 con 0x7f2768066370 2020-12-05T17:39:19.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.483+0000 7f27667fc700 1 -- 172.21.15.32:0/2251758406 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2760012420 con 0x7f2768066370 2020-12-05T17:39:19.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.483+0000 7f27667fc700 1 --2- 172.21.15.32:0/2251758406 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f274c0553f0 0x7f274c0578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:19.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.483+0000 7f27667fc700 1 -- 172.21.15.32:0/2251758406 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f276006de10 con 0x7f2768066370 2020-12-05T17:39:19.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.483+0000 7f276c91e700 1 --2- 172.21.15.32:0/2251758406 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f274c0553f0 0x7f274c0578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:19.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.483+0000 7f276c91e700 1 --2- 172.21.15.32:0/2251758406 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f274c0553f0 0x7f274c0578b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f275001b000 tx=0x7f2750016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:19.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.491+0000 7f27667fc700 1 -- 172.21.15.32:0/2251758406 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f27600441f0 con 0x7f2768066370 2020-12-05T17:39:19.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.631+0000 7f276e121700 1 -- 172.21.15.32:0/2251758406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f276813ef70 con 0x7f2768066370 2020-12-05T17:39:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.631+0000 7f27667fc700 1 -- 172.21.15.32:0/2251758406 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v88) v1 ==== 84+0+143 (secure 0 0 0) 0x7f2760047810 con 0x7f2768066370 2020-12-05T17:39:19.640 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:19.640 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2020-12-05T17:39:19.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:19.640 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2020-12-05T17:39:19.640 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2020-12-05T17:39:19.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:19.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:19.641 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2020-12-05T17:39:19.641 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2020-12-05T17:39:19.642 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:19.642 INFO:teuthology.orchestra.run.smithi032.stdout:] 2020-12-05T17:39:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.635+0000 7f276e121700 1 -- 172.21.15.32:0/2251758406 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f274c0553f0 msgr2=0x7f274c0578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.635+0000 7f276e121700 1 --2- 172.21.15.32:0/2251758406 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f274c0553f0 0x7f274c0578b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f275001b000 tx=0x7f2750016000).stop 2020-12-05T17:39:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.635+0000 7f276e121700 1 -- 172.21.15.32:0/2251758406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2768066370 msgr2=0x7f27680b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.635+0000 7f276e121700 1 --2- 172.21.15.32:0/2251758406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2768066370 0x7f27680b4d90 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7f276000a000 tx=0x7f2760008040).stop 2020-12-05T17:39:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.635+0000 7f276e121700 1 -- 172.21.15.32:0/2251758406 shutdown_connections 2020-12-05T17:39:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.635+0000 7f276e121700 1 --2- 172.21.15.32:0/2251758406 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27680b5f00 0x7f2768151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.635+0000 7f276e121700 1 --2- 172.21.15.32:0/2251758406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2768066370 0x7f27680b4d90 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.635+0000 7f276e121700 1 --2- 172.21.15.32:0/2251758406 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f274c0553f0 0x7f274c0578b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.635+0000 7f276e121700 1 --2- 172.21.15.32:0/2251758406 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27680ad510 0x7f2768127950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.635+0000 7f276e121700 1 -- 172.21.15.32:0/2251758406 >> 172.21.15.32:0/2251758406 conn(0x7f27680604c0 msgr2=0x7f276813f430 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:19.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.635+0000 7f276e121700 1 -- 172.21.15.32:0/2251758406 shutdown_connections 2020-12-05T17:39:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.635+0000 7f276e121700 1 -- 172.21.15.32:0/2251758406 wait complete. 2020-12-05T17:39:19.653 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-12-05T17:39:19.653 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-12-05T17:39:19.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.775+0000 7fcc6c9f2700 1 Processor -- start 2020-12-05T17:39:19.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.775+0000 7fcc6c9f2700 1 -- start start 2020-12-05T17:39:19.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.775+0000 7fcc6c9f2700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fcc6805f550 0x7fcc680acb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:19.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.775+0000 7fcc6c9f2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc68063040 0x7fcc6813fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:19.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.775+0000 7fcc6c9f2700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc68067110 con 0x7fcc680af090 2020-12-05T17:39:19.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.775+0000 7fcc6c9f2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc6813f760 con 0x7fcc68063040 2020-12-05T17:39:19.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.775+0000 7fcc6c9f2700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc6813fa00 con 0x7fcc6805f550 2020-12-05T17:39:19.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.775+0000 7fcc5ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc68063040 0x7fcc6813fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:19.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.775+0000 7fcc5ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc68063040 0x7fcc6813fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37978/0 (socket says 172.21.15.32:37978) 2020-12-05T17:39:19.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.775+0000 7fcc5ffff700 1 -- 172.21.15.32:0/3486976430 learned_addr learned my addr 172.21.15.32:0/3486976430 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:19.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.775+0000 7fcc66d9d700 1 --2- 172.21.15.32:0/3486976430 >> v2:172.21.15.61:3300/0 conn(0x7fcc6805f550 0x7fcc680acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:19.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.775+0000 7fcc5ffff700 1 -- 172.21.15.32:0/3486976430 >> v2:172.21.15.61:3300/0 conn(0x7fcc6805f550 msgr2=0x7fcc680acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:19.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.775+0000 7fcc5ffff700 1 --2- 172.21.15.32:0/3486976430 >> v2:172.21.15.61:3300/0 conn(0x7fcc6805f550 0x7fcc680acb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.775+0000 7fcc5ffff700 1 -- 172.21.15.32:0/3486976430 >> v1:172.21.15.61:6789/0 conn(0x7fcc680af090 legacy=0x7fcc680af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:19.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.775+0000 7fcc5ffff700 1 -- 172.21.15.32:0/3486976430 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc68066d90 con 0x7fcc68063040 2020-12-05T17:39:19.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.775+0000 7fcc5ffff700 1 --2- 172.21.15.32:0/3486976430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc68063040 0x7fcc6813fea0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7fcc50002f30 tx=0x7fcc50006e80).ready entity=mon.0 client_cookie=996da954f75bda7e server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:19.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.775+0000 7fcc6659c700 1 -- 172.21.15.32:0/3486976430 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc500097f0 con 0x7fcc68063040 2020-12-05T17:39:19.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.775+0000 7fcc6659c700 1 -- 172.21.15.32:0/3486976430 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc50009950 con 0x7fcc68063040 2020-12-05T17:39:19.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.775+0000 7fcc6659c700 1 -- 172.21.15.32:0/3486976430 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc50009cb0 con 0x7fcc68063040 2020-12-05T17:39:19.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.775+0000 7fcc6c9f2700 1 -- 172.21.15.32:0/3486976430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc68063040 msgr2=0x7fcc6813fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:19.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.775+0000 7fcc6c9f2700 1 --2- 172.21.15.32:0/3486976430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc68063040 0x7fcc6813fea0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7fcc50002f30 tx=0x7fcc50006e80).stop 2020-12-05T17:39:19.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.775+0000 7fcc6c9f2700 1 -- 172.21.15.32:0/3486976430 shutdown_connections 2020-12-05T17:39:19.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.779+0000 7fcc6c9f2700 1 --2- 172.21.15.32:0/3486976430 >> v2:172.21.15.61:3300/0 conn(0x7fcc6805f550 0x7fcc680acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.779+0000 7fcc6c9f2700 1 --2- 172.21.15.32:0/3486976430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc68063040 0x7fcc6813fea0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.779+0000 7fcc6c9f2700 1 -- 172.21.15.32:0/3486976430 >> 172.21.15.32:0/3486976430 conn(0x7fcc680620b0 msgr2=0x7fcc68062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:19.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.779+0000 7fcc6c9f2700 1 -- 172.21.15.32:0/3486976430 shutdown_connections 2020-12-05T17:39:19.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.779+0000 7fcc6c9f2700 1 -- 172.21.15.32:0/3486976430 wait complete. 2020-12-05T17:39:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.779+0000 7fcc6c9f2700 1 Processor -- start 2020-12-05T17:39:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.779+0000 7fcc6c9f2700 1 -- start start 2020-12-05T17:39:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.779+0000 7fcc6c9f2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc6805f550 0x7fcc6812bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.779+0000 7fcc6c9f2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcc68063040 0x7fcc6812cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.779+0000 7fcc6c9f2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc680af090 0x7fcc6812d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.779+0000 7fcc6c9f2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc680638e0 con 0x7fcc6805f550 2020-12-05T17:39:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.779+0000 7fcc6c9f2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc68063b80 con 0x7fcc680af090 2020-12-05T17:39:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.779+0000 7fcc6c9f2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc68063e20 con 0x7fcc68063040 2020-12-05T17:39:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.779+0000 7fcc66d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc6805f550 0x7fcc6812bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.779+0000 7fcc66d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc6805f550 0x7fcc6812bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37982/0 (socket says 172.21.15.32:37982) 2020-12-05T17:39:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.779+0000 7fcc66d9d700 1 -- 172.21.15.32:0/158284457 learned_addr learned my addr 172.21.15.32:0/158284457 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.779+0000 7fcc5ffff700 1 --2- 172.21.15.32:0/158284457 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcc68063040 0x7fcc6812cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.779+0000 7fcc6759e700 1 --2- 172.21.15.32:0/158284457 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc680af090 0x7fcc6812d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.783+0000 7fcc66d9d700 1 -- 172.21.15.32:0/158284457 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcc68063040 msgr2=0x7fcc6812cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.783+0000 7fcc66d9d700 1 --2- 172.21.15.32:0/158284457 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcc68063040 0x7fcc6812cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.783+0000 7fcc66d9d700 1 -- 172.21.15.32:0/158284457 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc680af090 msgr2=0x7fcc6812d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:19.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.783+0000 7fcc66d9d700 1 --2- 172.21.15.32:0/158284457 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc680af090 0x7fcc6812d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:19.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.783+0000 7fcc66d9d700 1 -- 172.21.15.32:0/158284457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc68063560 con 0x7fcc6805f550 2020-12-05T17:39:19.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.783+0000 7fcc66d9d700 1 --2- 172.21.15.32:0/158284457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc6805f550 0x7fcc6812bdd0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7fcc5800c000 tx=0x7fcc58006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:19.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.783+0000 7fcc64d99700 1 -- 172.21.15.32:0/158284457 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc5800ca10 con 0x7fcc6805f550 2020-12-05T17:39:19.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.783+0000 7fcc64d99700 1 -- 172.21.15.32:0/158284457 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5800cb70 con 0x7fcc6805f550 2020-12-05T17:39:19.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.783+0000 7fcc6c9f2700 1 -- 172.21.15.32:0/158284457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc68064930 con 0x7fcc6805f550 2020-12-05T17:39:19.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.783+0000 7fcc64d99700 1 -- 172.21.15.32:0/158284457 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc580123f0 con 0x7fcc6805f550 2020-12-05T17:39:19.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.783+0000 7fcc6c9f2700 1 -- 172.21.15.32:0/158284457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6812d700 con 0x7fcc6805f550 2020-12-05T17:39:19.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.783+0000 7fcc6c9f2700 1 -- 172.21.15.32:0/158284457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc68063fe0 con 0x7fcc6805f550 2020-12-05T17:39:19.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.783+0000 7fcc64d99700 1 -- 172.21.15.32:0/158284457 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fcc58012550 con 0x7fcc6805f550 2020-12-05T17:39:19.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.783+0000 7fcc64d99700 1 --2- 172.21.15.32:0/158284457 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcc4c055470 0x7fcc4c057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:19.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.783+0000 7fcc64d99700 1 -- 172.21.15.32:0/158284457 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fcc5806e020 con 0x7fcc6805f550 2020-12-05T17:39:19.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.783+0000 7fcc5ffff700 1 --2- 172.21.15.32:0/158284457 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcc4c055470 0x7fcc4c057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:19.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.787+0000 7fcc5ffff700 1 --2- 172.21.15.32:0/158284457 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcc4c055470 0x7fcc4c057930 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7fcc5001c000 tx=0x7fcc50015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:19.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.791+0000 7fcc64d99700 1 -- 172.21.15.32:0/158284457 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fcc580444a0 con 0x7fcc6805f550 2020-12-05T17:39:19.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:19.935+0000 7fcc6c9f2700 1 -- 172.21.15.32:0/158284457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fcc68057050 con 0x7fcc6805f550 2020-12-05T17:39:20.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:20.887+0000 7fcc64d99700 1 -- 172.21.15.32:0/158284457 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v89) v1 ==== 133+0+0 (secure 0 0 0) 0x7fcc58047ac0 con 0x7fcc6805f550 2020-12-05T17:39:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:20.887+0000 7fcc6c9f2700 1 -- 172.21.15.32:0/158284457 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcc4c055470 msgr2=0x7fcc4c057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:20.887+0000 7fcc6c9f2700 1 --2- 172.21.15.32:0/158284457 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcc4c055470 0x7fcc4c057930 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7fcc5001c000 tx=0x7fcc50015000).stop 2020-12-05T17:39:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:20.887+0000 7fcc6c9f2700 1 -- 172.21.15.32:0/158284457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc6805f550 msgr2=0x7fcc6812bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:20.887+0000 7fcc6c9f2700 1 --2- 172.21.15.32:0/158284457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc6805f550 0x7fcc6812bdd0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7fcc5800c000 tx=0x7fcc58006dc0).stop 2020-12-05T17:39:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:20.887+0000 7fcc6c9f2700 1 -- 172.21.15.32:0/158284457 shutdown_connections 2020-12-05T17:39:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:20.887+0000 7fcc6c9f2700 1 --2- 172.21.15.32:0/158284457 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc680af090 0x7fcc6812d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:20.887+0000 7fcc6c9f2700 1 --2- 172.21.15.32:0/158284457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc6805f550 0x7fcc6812bdd0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:20.887+0000 7fcc6c9f2700 1 --2- 172.21.15.32:0/158284457 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcc4c055470 0x7fcc4c057930 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:20.887+0000 7fcc6c9f2700 1 --2- 172.21.15.32:0/158284457 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcc68063040 0x7fcc6812cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:20.887+0000 7fcc6c9f2700 1 -- 172.21.15.32:0/158284457 >> 172.21.15.32:0/158284457 conn(0x7fcc680620b0 msgr2=0x7fcc680acb80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:20.887+0000 7fcc6c9f2700 1 -- 172.21.15.32:0/158284457 shutdown_connections 2020-12-05T17:39:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:20.887+0000 7fcc6c9f2700 1 -- 172.21.15.32:0/158284457 wait complete. 2020-12-05T17:39:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2020-12-05T17:39:20.912 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-12-05T17:39:21.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.031+0000 7fe486737700 1 Processor -- start 2020-12-05T17:39:21.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.031+0000 7fe486737700 1 -- start start 2020-12-05T17:39:21.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.031+0000 7fe486737700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe4801433d0 0x7fe48005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:21.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.031+0000 7fe486737700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4800b2df0 con 0x7fe48005feb0 2020-12-05T17:39:21.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.031+0000 7fe486737700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4800b3c90 con 0x7fe48005f3b0 2020-12-05T17:39:21.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.031+0000 7fe486737700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4800b2b50 con 0x7fe4801433d0 2020-12-05T17:39:21.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.031+0000 7fe485f36700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe4801433d0 0x7fe48005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:21.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.031+0000 7fe485f36700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe4801433d0 0x7fe48005f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37994/0 (socket says 172.21.15.32:37994) 2020-12-05T17:39:21.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.031+0000 7fe485f36700 1 -- 172.21.15.32:0/2192351438 learned_addr learned my addr 172.21.15.32:0/2192351438 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:21.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.031+0000 7fe484f34700 1 --1- 172.21.15.32:0/2192351438 >> v1:172.21.15.32:6789/0 conn(0x7fe48005feb0 0x7fe480142cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40640/0 (socket says 172.21.15.32:40640) 2020-12-05T17:39:21.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.031+0000 7fe485f36700 1 -- 172.21.15.32:0/2192351438 >> v1:172.21.15.61:6789/0 conn(0x7fe48005f3b0 legacy=0x7fe48013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:21.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.031+0000 7fe485f36700 1 -- 172.21.15.32:0/2192351438 >> v1:172.21.15.32:6789/0 conn(0x7fe48005feb0 legacy=0x7fe480142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:21.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.031+0000 7fe485f36700 1 -- 172.21.15.32:0/2192351438 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe480142820 con 0x7fe4801433d0 2020-12-05T17:39:21.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.031+0000 7fe485f36700 1 --2- 172.21.15.32:0/2192351438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4801433d0 0x7fe48005f790 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7fe4780032b0 tx=0x7fe478009270).ready entity=mon.0 client_cookie=d4ea22a5db6c60df server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:21.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe473fff700 1 -- 172.21.15.32:0/2192351438 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe47800bc00 con 0x7fe4801433d0 2020-12-05T17:39:21.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe473fff700 1 -- 172.21.15.32:0/2192351438 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47800bd60 con 0x7fe4801433d0 2020-12-05T17:39:21.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe473fff700 1 -- 172.21.15.32:0/2192351438 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe478011400 con 0x7fe4801433d0 2020-12-05T17:39:21.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe486737700 1 -- 172.21.15.32:0/2192351438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4801433d0 msgr2=0x7fe48005f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:21.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe486737700 1 --2- 172.21.15.32:0/2192351438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4801433d0 0x7fe48005f790 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7fe4780032b0 tx=0x7fe478009270).stop 2020-12-05T17:39:21.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe486737700 1 -- 172.21.15.32:0/2192351438 shutdown_connections 2020-12-05T17:39:21.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe486737700 1 --2- 172.21.15.32:0/2192351438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4801433d0 0x7fe48005f790 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:21.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe486737700 1 -- 172.21.15.32:0/2192351438 >> 172.21.15.32:0/2192351438 conn(0x7fe4800b8f20 msgr2=0x7fe4800b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:21.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe486737700 1 -- 172.21.15.32:0/2192351438 shutdown_connections 2020-12-05T17:39:21.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe486737700 1 -- 172.21.15.32:0/2192351438 wait complete. 2020-12-05T17:39:21.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe486737700 1 Processor -- start 2020-12-05T17:39:21.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe486737700 1 -- start start 2020-12-05T17:39:21.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe486737700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe48005f3b0 0x7fe4801127f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:21.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe486737700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe48005feb0 0x7fe480112110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:21.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe486737700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe4801433d0 0x7fe480149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:21.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe486737700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe480147c60 con 0x7fe48005feb0 2020-12-05T17:39:21.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe486737700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe480147f00 con 0x7fe48005f3b0 2020-12-05T17:39:21.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe486737700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe480114180 con 0x7fe4801433d0 2020-12-05T17:39:21.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe484f34700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe48005feb0 0x7fe480112110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:21.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe484f34700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe48005feb0 0x7fe480112110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37998/0 (socket says 172.21.15.32:37998) 2020-12-05T17:39:21.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe484f34700 1 -- 172.21.15.32:0/3023481844 learned_addr learned my addr 172.21.15.32:0/3023481844 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:21.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe485735700 1 --2- 172.21.15.32:0/3023481844 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe48005f3b0 0x7fe4801127f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:21.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe485f36700 1 --2- 172.21.15.32:0/3023481844 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe4801433d0 0x7fe480149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:21.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe484f34700 1 -- 172.21.15.32:0/3023481844 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe4801433d0 msgr2=0x7fe480149700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:21.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe484f34700 1 --2- 172.21.15.32:0/3023481844 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe4801433d0 0x7fe480149700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:21.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe484f34700 1 -- 172.21.15.32:0/3023481844 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe48005f3b0 msgr2=0x7fe4801127f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:21.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe484f34700 1 --2- 172.21.15.32:0/3023481844 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe48005f3b0 0x7fe4801127f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:21.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.035+0000 7fe484f34700 1 -- 172.21.15.32:0/3023481844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe480151530 con 0x7fe48005feb0 2020-12-05T17:39:21.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.039+0000 7fe484f34700 1 --2- 172.21.15.32:0/3023481844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe48005feb0 0x7fe480112110 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7fe480061550 tx=0x7fe474007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:21.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.039+0000 7fe4727fc700 1 -- 172.21.15.32:0/3023481844 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4740072e0 con 0x7fe48005feb0 2020-12-05T17:39:21.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.039+0000 7fe4727fc700 1 -- 172.21.15.32:0/3023481844 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe474005520 con 0x7fe48005feb0 2020-12-05T17:39:21.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.039+0000 7fe486737700 1 -- 172.21.15.32:0/3023481844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4801517e0 con 0x7fe48005feb0 2020-12-05T17:39:21.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.039+0000 7fe4727fc700 1 -- 172.21.15.32:0/3023481844 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe474005880 con 0x7fe48005feb0 2020-12-05T17:39:21.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.039+0000 7fe486737700 1 -- 172.21.15.32:0/3023481844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe480148370 con 0x7fe48005feb0 2020-12-05T17:39:21.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.039+0000 7fe486737700 1 -- 172.21.15.32:0/3023481844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe480057050 con 0x7fe48005feb0 2020-12-05T17:39:21.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.039+0000 7fe4727fc700 1 -- 172.21.15.32:0/3023481844 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe474010070 con 0x7fe48005feb0 2020-12-05T17:39:21.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.043+0000 7fe4727fc700 1 --2- 172.21.15.32:0/3023481844 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe468055330 0x7fe4680577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:21.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.043+0000 7fe4727fc700 1 -- 172.21.15.32:0/3023481844 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5954+0+0 (secure 0 0 0) 0x7fe47406d510 con 0x7fe48005feb0 2020-12-05T17:39:21.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.043+0000 7fe485735700 1 --2- 172.21.15.32:0/3023481844 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe468055330 0x7fe4680577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:21.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.043+0000 7fe485735700 1 --2- 172.21.15.32:0/3023481844 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe468055330 0x7fe4680577f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fe47c00d000 tx=0x7fe47c007b60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:21.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.047+0000 7fe4727fc700 1 -- 172.21.15.32:0/3023481844 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe474043880 con 0x7fe48005feb0 2020-12-05T17:39:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.191+0000 7fe486737700 1 -- 172.21.15.32:0/3023481844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fe48005aab0 con 0x7fe48005feb0 2020-12-05T17:39:21.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.899+0000 7fe4727fc700 1 -- 172.21.15.32:0/3023481844 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v90) v1 ==== 125+0+0 (secure 0 0 0) 0x7fe474046ea0 con 0x7fe48005feb0 2020-12-05T17:39:21.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.903+0000 7fe486737700 1 -- 172.21.15.32:0/3023481844 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe468055330 msgr2=0x7fe4680577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:21.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.903+0000 7fe486737700 1 --2- 172.21.15.32:0/3023481844 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe468055330 0x7fe4680577f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fe47c00d000 tx=0x7fe47c007b60).stop 2020-12-05T17:39:21.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.903+0000 7fe486737700 1 -- 172.21.15.32:0/3023481844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe48005feb0 msgr2=0x7fe480112110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:21.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.903+0000 7fe486737700 1 --2- 172.21.15.32:0/3023481844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe48005feb0 0x7fe480112110 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7fe480061550 tx=0x7fe474007bd0).stop 2020-12-05T17:39:21.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.907+0000 7fe486737700 1 -- 172.21.15.32:0/3023481844 shutdown_connections 2020-12-05T17:39:21.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.907+0000 7fe486737700 1 --2- 172.21.15.32:0/3023481844 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe468055330 0x7fe4680577f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:21.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.907+0000 7fe486737700 1 --2- 172.21.15.32:0/3023481844 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe4801433d0 0x7fe480149700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:21.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.907+0000 7fe486737700 1 --2- 172.21.15.32:0/3023481844 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe48005f3b0 0x7fe4801127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:21.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.907+0000 7fe486737700 1 --2- 172.21.15.32:0/3023481844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe48005feb0 0x7fe480112110 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:21.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.907+0000 7fe486737700 1 -- 172.21.15.32:0/3023481844 >> 172.21.15.32:0/3023481844 conn(0x7fe4800b8f20 msgr2=0x7fe4800b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:21.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.907+0000 7fe486737700 1 -- 172.21.15.32:0/3023481844 shutdown_connections 2020-12-05T17:39:21.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:21.907+0000 7fe486737700 1 -- 172.21.15.32:0/3023481844 wait complete. 2020-12-05T17:39:21.914 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2020-12-05T17:39:21.928 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-12-05T17:39:22.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.039+0000 7f664b0f1700 1 Processor -- start 2020-12-05T17:39:22.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.039+0000 7f664b0f1700 1 -- start start 2020-12-05T17:39:22.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f664b0f1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6644061b30 0x7f6644141e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f664b0f1700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f6644142300 0x7f6644142790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f664b0f1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66441439c0 con 0x7f664405f6f0 2020-12-05T17:39:22.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f664b0f1700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664405fe20 con 0x7f6644142300 2020-12-05T17:39:22.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f664b0f1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6644146910 con 0x7f6644061b30 2020-12-05T17:39:22.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f664a0ef700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f664405f6f0 0x7f6644141700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40652/0 (socket says 172.21.15.32:40652) 2020-12-05T17:39:22.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f664a0ef700 1 -- 172.21.15.32:0/972446023 learned_addr learned my addr 172.21.15.32:0/972446023 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:22.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f66498ee700 1 --2- 172.21.15.32:0/972446023 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6644061b30 0x7f6644141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f664a8f0700 1 --2- 172.21.15.32:0/972446023 >> v2:172.21.15.61:3300/0 conn(0x7f6644142300 0x7f6644142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f66490ed700 1 -- 172.21.15.32:0/972446023 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3384089308 0 0) 0x7f66441439c0 con 0x7f664405f6f0 2020-12-05T17:39:22.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f66490ed700 1 -- 172.21.15.32:0/972446023 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f662c003230 con 0x7f664405f6f0 2020-12-05T17:39:22.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f66498ee700 1 -- 172.21.15.32:0/972446023 >> v2:172.21.15.61:3300/0 conn(0x7f6644142300 msgr2=0x7f6644142790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:22.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f66498ee700 1 --2- 172.21.15.32:0/972446023 >> v2:172.21.15.61:3300/0 conn(0x7f6644142300 0x7f6644142790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f66498ee700 1 -- 172.21.15.32:0/972446023 >> v1:172.21.15.32:6789/0 conn(0x7f664405f6f0 legacy=0x7f6644141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:22.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f66498ee700 1 -- 172.21.15.32:0/972446023 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664405fad0 con 0x7f6644061b30 2020-12-05T17:39:22.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f66498ee700 1 --2- 172.21.15.32:0/972446023 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6644061b30 0x7f6644141e10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f6634002f30 tx=0x7f6634006dc0).ready entity=mon.2 client_cookie=6016fbf80996687f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:22.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f66490ed700 1 -- 172.21.15.32:0/972446023 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66340096a0 con 0x7f6644061b30 2020-12-05T17:39:22.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f66490ed700 1 -- 172.21.15.32:0/972446023 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6634009800 con 0x7f6644061b30 2020-12-05T17:39:22.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f66490ed700 1 -- 172.21.15.32:0/972446023 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6634009b60 con 0x7f6644061b30 2020-12-05T17:39:22.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f664b0f1700 1 -- 172.21.15.32:0/972446023 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6644061b30 msgr2=0x7f6644141e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:22.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f664b0f1700 1 --2- 172.21.15.32:0/972446023 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6644061b30 0x7f6644141e10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f6634002f30 tx=0x7f6634006dc0).stop 2020-12-05T17:39:22.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f664b0f1700 1 -- 172.21.15.32:0/972446023 shutdown_connections 2020-12-05T17:39:22.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f664b0f1700 1 --2- 172.21.15.32:0/972446023 >> v2:172.21.15.61:3300/0 conn(0x7f6644142300 0x7f6644142790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f664b0f1700 1 --2- 172.21.15.32:0/972446023 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6644061b30 0x7f6644141e10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f664b0f1700 1 -- 172.21.15.32:0/972446023 >> 172.21.15.32:0/972446023 conn(0x7f6644061010 msgr2=0x7f6644061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:22.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f664b0f1700 1 -- 172.21.15.32:0/972446023 shutdown_connections 2020-12-05T17:39:22.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.043+0000 7f664b0f1700 1 -- 172.21.15.32:0/972446023 wait complete. 2020-12-05T17:39:22.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.047+0000 7f664b0f1700 1 Processor -- start 2020-12-05T17:39:22.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.047+0000 7f664b0f1700 1 -- start start 2020-12-05T17:39:22.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.051+0000 7f664b0f1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f664405f6f0 0x7f66440644e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.051+0000 7f664b0f1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6644061b30 0x7f66440b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.051+0000 7f664b0f1700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6644142300 0x7f6644064ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.051+0000 7f664b0f1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66440656c0 con 0x7f6644061b30 2020-12-05T17:39:22.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.051+0000 7f664b0f1700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6644113230 con 0x7f6644142300 2020-12-05T17:39:22.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.051+0000 7f664b0f1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6644065180 con 0x7f664405f6f0 2020-12-05T17:39:22.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.051+0000 7f66498ee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6644061b30 0x7f66440b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.051+0000 7f66498ee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6644061b30 0x7f66440b4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38012/0 (socket says 172.21.15.32:38012) 2020-12-05T17:39:22.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.051+0000 7f66498ee700 1 -- 172.21.15.32:0/1654825924 learned_addr learned my addr 172.21.15.32:0/1654825924 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:22.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.051+0000 7f664a0ef700 1 --2- 172.21.15.32:0/1654825924 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f664405f6f0 0x7f66440644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.051+0000 7f664a8f0700 1 --2- 172.21.15.32:0/1654825924 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6644142300 0x7f6644064ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.051+0000 7f66498ee700 1 -- 172.21.15.32:0/1654825924 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f664405f6f0 msgr2=0x7f66440644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:22.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.051+0000 7f66498ee700 1 --2- 172.21.15.32:0/1654825924 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f664405f6f0 0x7f66440644e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.051+0000 7f66498ee700 1 -- 172.21.15.32:0/1654825924 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6644142300 msgr2=0x7f6644064ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:22.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.051+0000 7f66498ee700 1 --2- 172.21.15.32:0/1654825924 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6644142300 0x7f6644064ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.051+0000 7f66498ee700 1 -- 172.21.15.32:0/1654825924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6644065420 con 0x7f6644061b30 2020-12-05T17:39:22.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.051+0000 7f66498ee700 1 --2- 172.21.15.32:0/1654825924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6644061b30 0x7f66440b4420 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7f6634002f30 tx=0x7f6634015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:22.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.051+0000 7f66337fe700 1 -- 172.21.15.32:0/1654825924 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66340119e0 con 0x7f6644061b30 2020-12-05T17:39:22.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.051+0000 7f66337fe700 1 -- 172.21.15.32:0/1654825924 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6634011b40 con 0x7f6644061b30 2020-12-05T17:39:22.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.051+0000 7f664b0f1700 1 -- 172.21.15.32:0/1654825924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66441549a0 con 0x7f6644061b30 2020-12-05T17:39:22.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.051+0000 7f66337fe700 1 -- 172.21.15.32:0/1654825924 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6634011ea0 con 0x7f6644061b30 2020-12-05T17:39:22.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.051+0000 7f664b0f1700 1 -- 172.21.15.32:0/1654825924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6644113620 con 0x7f6644061b30 2020-12-05T17:39:22.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.051+0000 7f664b0f1700 1 -- 172.21.15.32:0/1654825924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6644064fd0 con 0x7f6644061b30 2020-12-05T17:39:22.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.055+0000 7f66337fe700 1 -- 172.21.15.32:0/1654825924 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6634024030 con 0x7f6644061b30 2020-12-05T17:39:22.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.055+0000 7f66337fe700 1 --2- 172.21.15.32:0/1654825924 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6624055440 0x7f6624057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.055+0000 7f66337fe700 1 -- 172.21.15.32:0/1654825924 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f663406f0a0 con 0x7f6644061b30 2020-12-05T17:39:22.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.055+0000 7f664a0ef700 1 --2- 172.21.15.32:0/1654825924 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6624055440 0x7f6624057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.055+0000 7f664a0ef700 1 --2- 172.21.15.32:0/1654825924 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6624055440 0x7f6624057900 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f664000d000 tx=0x7f6640002850).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.063+0000 7f66337fe700 1 -- 172.21.15.32:0/1654825924 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f66340451b0 con 0x7f6644061b30 2020-12-05T17:39:22.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.255+0000 7f664b0f1700 1 -- 172.21.15.32:0/1654825924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f6644057050 con 0x7f6644061b30 2020-12-05T17:39:22.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.267+0000 7f66337fe700 1 -- 172.21.15.32:0/1654825924 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 17 and data pool 18 v41) v1 ==== 189+0+0 (secure 0 0 0) 0x7f6634022070 con 0x7f6644061b30 2020-12-05T17:39:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.275+0000 7f664b0f1700 1 -- 172.21.15.32:0/1654825924 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6624055440 msgr2=0x7f6624057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.275+0000 7f664b0f1700 1 --2- 172.21.15.32:0/1654825924 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6624055440 0x7f6624057900 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f664000d000 tx=0x7f6640002850).stop 2020-12-05T17:39:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.275+0000 7f664b0f1700 1 -- 172.21.15.32:0/1654825924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6644061b30 msgr2=0x7f66440b4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.275+0000 7f664b0f1700 1 --2- 172.21.15.32:0/1654825924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6644061b30 0x7f66440b4420 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7f6634002f30 tx=0x7f6634015000).stop 2020-12-05T17:39:22.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.275+0000 7f664b0f1700 1 -- 172.21.15.32:0/1654825924 shutdown_connections 2020-12-05T17:39:22.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.275+0000 7f664b0f1700 1 --2- 172.21.15.32:0/1654825924 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6644142300 0x7f6644064ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.275+0000 7f664b0f1700 1 --2- 172.21.15.32:0/1654825924 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6624055440 0x7f6624057900 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.275+0000 7f664b0f1700 1 --2- 172.21.15.32:0/1654825924 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f664405f6f0 0x7f66440644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.275+0000 7f664b0f1700 1 --2- 172.21.15.32:0/1654825924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6644061b30 0x7f66440b4420 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.275+0000 7f664b0f1700 1 -- 172.21.15.32:0/1654825924 >> 172.21.15.32:0/1654825924 conn(0x7f6644061010 msgr2=0x7f66440ade50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.275+0000 7f664b0f1700 1 -- 172.21.15.32:0/1654825924 shutdown_connections 2020-12-05T17:39:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.275+0000 7f664b0f1700 1 -- 172.21.15.32:0/1654825924 wait complete. 2020-12-05T17:39:22.284 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 17 and data pool 18 2020-12-05T17:39:22.298 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:39:22.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.395+0000 7fa0d3448700 1 Processor -- start 2020-12-05T17:39:22.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.395+0000 7fa0d3448700 1 -- start start 2020-12-05T17:39:22.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d3448700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0cc0af170 0x7fa0cc066a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d3448700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0cc143b10 con 0x7fa0cc0aed90 2020-12-05T17:39:22.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d2446700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa0cc0aed90 0x7fa0cc1404a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40674/0 (socket says 172.21.15.32:40674) 2020-12-05T17:39:22.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d3448700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0cc0b1c30 con 0x7fa0cc0b2140 2020-12-05T17:39:22.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d2446700 1 -- 172.21.15.32:0/4018414212 learned_addr learned my addr 172.21.15.32:0/4018414212 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d3448700 1 -- 172.21.15.32:0/4018414212 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc13df90 con 0x7fa0cc0af170 2020-12-05T17:39:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d2c47700 1 --2- 172.21.15.32:0/4018414212 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0cc0af170 0x7fa0cc066a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1444700 1 -- 172.21.15.32:0/4018414212 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3562260556 0 0) 0x7fa0cc143b10 con 0x7fa0cc0aed90 2020-12-05T17:39:22.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1444700 1 -- 172.21.15.32:0/4018414212 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0b0003230 con 0x7fa0cc0aed90 2020-12-05T17:39:22.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1444700 1 -- 172.21.15.32:0/4018414212 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3889854565 0 0) 0x7fa0b0003230 con 0x7fa0cc0aed90 2020-12-05T17:39:22.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1444700 1 -- 172.21.15.32:0/4018414212 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0cc0af170 msgr2=0x7fa0cc066a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:22.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1444700 1 --2- 172.21.15.32:0/4018414212 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0cc0af170 0x7fa0cc066a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1444700 1 -- 172.21.15.32:0/4018414212 >> v1:172.21.15.61:6789/0 conn(0x7fa0cc0b2140 legacy=0x7fa0cc1441e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:22.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1444700 1 -- 172.21.15.32:0/4018414212 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc0b2650 con 0x7fa0cc0aed90 2020-12-05T17:39:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1444700 1 -- 172.21.15.32:0/4018414212 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fa0c0001a50 con 0x7fa0cc0aed90 2020-12-05T17:39:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1444700 1 -- 172.21.15.32:0/4018414212 >> v1:172.21.15.32:6789/0 conn(0x7fa0cc0aed90 legacy=0x7fa0cc1404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1444700 1 --2- 172.21.15.32:0/4018414212 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0b0003750 0x7fa0b0003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1444700 1 --2- 172.21.15.32:0/4018414212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0b0004090 0x7fa0b0004570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1444700 1 --2- 172.21.15.32:0/4018414212 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0cc0af170 0x7fa0b0004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1444700 1 -- 172.21.15.32:0/4018414212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc13df90 con 0x7fa0b0004090 2020-12-05T17:39:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1444700 1 -- 172.21.15.32:0/4018414212 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b0005110 con 0x7fa0b0003750 2020-12-05T17:39:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1444700 1 -- 172.21.15.32:0/4018414212 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b00052d0 con 0x7fa0cc0af170 2020-12-05T17:39:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d2446700 1 --2- 172.21.15.32:0/4018414212 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0b0003750 0x7fa0b0003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1c45700 1 --2- 172.21.15.32:0/4018414212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0b0004090 0x7fa0b0004570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d2c47700 1 --2- 172.21.15.32:0/4018414212 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0cc0af170 0x7fa0b0004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1c45700 1 -- 172.21.15.32:0/4018414212 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0cc0af170 msgr2=0x7fa0b0004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:22.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1c45700 1 --2- 172.21.15.32:0/4018414212 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0cc0af170 0x7fa0b0004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1c45700 1 -- 172.21.15.32:0/4018414212 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0b0003750 msgr2=0x7fa0b0003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:22.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1c45700 1 --2- 172.21.15.32:0/4018414212 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0b0003750 0x7fa0b0003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1c45700 1 -- 172.21.15.32:0/4018414212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b0003230 con 0x7fa0b0004090 2020-12-05T17:39:22.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d2c47700 1 --2- 172.21.15.32:0/4018414212 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0cc0af170 0x7fa0b0004bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:39:22.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1c45700 1 --2- 172.21.15.32:0/4018414212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0b0004090 0x7fa0b0004570 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7fa0b80010d0 tx=0x7fa0b8007b60).ready entity=mon.0 client_cookie=b2b3a456950888ff server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:22.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1444700 1 -- 172.21.15.32:0/4018414212 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0b800c7f0 con 0x7fa0b0004090 2020-12-05T17:39:22.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1444700 1 -- 172.21.15.32:0/4018414212 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b800c950 con 0x7fa0b0004090 2020-12-05T17:39:22.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d1444700 1 -- 172.21.15.32:0/4018414212 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0b800ccb0 con 0x7fa0b0004090 2020-12-05T17:39:22.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d3448700 1 -- 172.21.15.32:0/4018414212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0b0004090 msgr2=0x7fa0b0004570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:22.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d3448700 1 --2- 172.21.15.32:0/4018414212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0b0004090 0x7fa0b0004570 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7fa0b80010d0 tx=0x7fa0b8007b60).stop 2020-12-05T17:39:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d2c47700 1 -- 172.21.15.32:0/4018414212 reap_dead start 2020-12-05T17:39:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d3448700 1 -- 172.21.15.32:0/4018414212 shutdown_connections 2020-12-05T17:39:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d3448700 1 -- 172.21.15.32:0/4018414212 >> 172.21.15.32:0/4018414212 conn(0x7fa0cc05bbb0 msgr2=0x7fa0cc062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d3448700 1 -- 172.21.15.32:0/4018414212 shutdown_connections 2020-12-05T17:39:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.399+0000 7fa0d3448700 1 -- 172.21.15.32:0/4018414212 wait complete. 2020-12-05T17:39:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d3448700 1 Processor -- start 2020-12-05T17:39:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d3448700 1 -- start start 2020-12-05T17:39:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d3448700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0cc0ad060 0x7fa0cc064e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d3448700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0cc066160 0x7fa0cc13ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d3448700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0cc140410 0x7fa0cc1548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d3448700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc0ad440 con 0x7fa0cc140410 2020-12-05T17:39:22.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d3448700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc0ae800 con 0x7fa0cc0ad060 2020-12-05T17:39:22.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d3448700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc0aeaa0 con 0x7fa0cc066160 2020-12-05T17:39:22.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d2c47700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0cc140410 0x7fa0cc1548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d1c45700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0cc066160 0x7fa0cc13ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d2446700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0cc0ad060 0x7fa0cc064e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d2c47700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0cc140410 0x7fa0cc1548c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38042/0 (socket says 172.21.15.32:38042) 2020-12-05T17:39:22.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d1c45700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0cc066160 0x7fa0cc13ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:34828/0 (socket says 172.21.15.32:34828) 2020-12-05T17:39:22.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d2446700 1 --2- 172.21.15.32:0/2848675674 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0cc0ad060 0x7fa0cc064e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:57932/0 (socket says 172.21.15.32:57932) 2020-12-05T17:39:22.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d2c47700 1 -- 172.21.15.32:0/2848675674 learned_addr learned my addr 172.21.15.32:0/2848675674 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:22.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d2c47700 1 -- 172.21.15.32:0/2848675674 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0cc066160 msgr2=0x7fa0cc13ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:22.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d2c47700 1 --2- 172.21.15.32:0/2848675674 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0cc066160 0x7fa0cc13ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d2c47700 1 -- 172.21.15.32:0/2848675674 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0cc0ad060 msgr2=0x7fa0cc064e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:22.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d2c47700 1 --2- 172.21.15.32:0/2848675674 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0cc0ad060 0x7fa0cc064e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d2c47700 1 -- 172.21.15.32:0/2848675674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc140830 con 0x7fa0cc140410 2020-12-05T17:39:22.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d2446700 1 --2- 172.21.15.32:0/2848675674 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0cc0ad060 0x7fa0cc064e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:22.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d2c47700 1 --2- 172.21.15.32:0/2848675674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0cc140410 0x7fa0cc1548c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7fa0c4008000 tx=0x7fa0c4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:22.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0cb7fe700 1 -- 172.21.15.32:0/2848675674 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0c40116c0 con 0x7fa0cc140410 2020-12-05T17:39:22.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0cb7fe700 1 -- 172.21.15.32:0/2848675674 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c4011820 con 0x7fa0cc140410 2020-12-05T17:39:22.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d3448700 1 -- 172.21.15.32:0/2848675674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0cc140ad0 con 0x7fa0cc140410 2020-12-05T17:39:22.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0cb7fe700 1 -- 172.21.15.32:0/2848675674 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0c4011bc0 con 0x7fa0cc140410 2020-12-05T17:39:22.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.403+0000 7fa0d3448700 1 -- 172.21.15.32:0/2848675674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0cc065540 con 0x7fa0cc140410 2020-12-05T17:39:22.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.407+0000 7fa0d3448700 1 -- 172.21.15.32:0/2848675674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0cc140c60 con 0x7fa0cc140410 2020-12-05T17:39:22.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.407+0000 7fa0cb7fe700 1 -- 172.21.15.32:0/2848675674 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa0c4011d20 con 0x7fa0cc140410 2020-12-05T17:39:22.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.407+0000 7fa0cb7fe700 1 --2- 172.21.15.32:0/2848675674 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa0b4055350 0x7fa0b4057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.407+0000 7fa0cb7fe700 1 -- 172.21.15.32:0/2848675674 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa0c406dcb0 con 0x7fa0cc140410 2020-12-05T17:39:22.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.407+0000 7fa0d2446700 1 --2- 172.21.15.32:0/2848675674 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa0b4055350 0x7fa0b4057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.407+0000 7fa0d2446700 1 --2- 172.21.15.32:0/2848675674 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa0b4055350 0x7fa0b4057810 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fa0c0005000 tx=0x7fa0cc0b27e0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:22.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.411+0000 7fa0cb7fe700 1 -- 172.21.15.32:0/2848675674 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa0c4043e50 con 0x7fa0cc140410 2020-12-05T17:39:22.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.551+0000 7fa0d3448700 1 -- 172.21.15.32:0/2848675674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0cc057050 con 0x7fa0cc140410 2020-12-05T17:39:22.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.555+0000 7fa0cb7fe700 1 -- 172.21.15.32:0/2848675674 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16287 (secure 0 0 0) 0x7fa0c4047470 con 0x7fa0cc140410 2020-12-05T17:39:22.559 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:22.559 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":91,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:39:22.261286+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:39:19.993416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2020-12-05T17:39:21.252765+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:39:22.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.555+0000 7fa0d3448700 1 -- 172.21.15.32:0/2848675674 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa0b4055350 msgr2=0x7fa0b4057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:22.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.555+0000 7fa0d3448700 1 --2- 172.21.15.32:0/2848675674 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa0b4055350 0x7fa0b4057810 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fa0c0005000 tx=0x7fa0cc0b27e0).stop 2020-12-05T17:39:22.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.555+0000 7fa0d3448700 1 -- 172.21.15.32:0/2848675674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0cc140410 msgr2=0x7fa0cc1548c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:22.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.555+0000 7fa0d3448700 1 --2- 172.21.15.32:0/2848675674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0cc140410 0x7fa0cc1548c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7fa0c4008000 tx=0x7fa0c4008040).stop 2020-12-05T17:39:22.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.555+0000 7fa0d3448700 1 -- 172.21.15.32:0/2848675674 shutdown_connections 2020-12-05T17:39:22.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.555+0000 7fa0d3448700 1 --2- 172.21.15.32:0/2848675674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0cc140410 0x7fa0cc1548c0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.555+0000 7fa0d3448700 1 --2- 172.21.15.32:0/2848675674 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa0cc0ad060 0x7fa0cc064e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.555+0000 7fa0d3448700 1 --2- 172.21.15.32:0/2848675674 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa0b4055350 0x7fa0b4057810 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.555+0000 7fa0d3448700 1 --2- 172.21.15.32:0/2848675674 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa0cc066160 0x7fa0cc13ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.555+0000 7fa0d3448700 1 -- 172.21.15.32:0/2848675674 >> 172.21.15.32:0/2848675674 conn(0x7fa0cc05bbb0 msgr2=0x7fa0cc143230 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:22.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.555+0000 7fa0d3448700 1 -- 172.21.15.32:0/2848675674 shutdown_connections 2020-12-05T17:39:22.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.555+0000 7fa0d3448700 1 -- 172.21.15.32:0/2848675674 wait complete. 2020-12-05T17:39:22.573 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:39:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83c4afc700 1 Processor -- start 2020-12-05T17:39:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83c4afc700 1 -- start start 2020-12-05T17:39:22.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83c4afc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83c005fe00 0x7f83c0061910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83c4afc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f83c00b00a0 0x7f83c014a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83c4afc700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c00601e0 con 0x7f83c00ac590 2020-12-05T17:39:22.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83c4afc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83c00ac1a0 con 0x7f83c005fe00 2020-12-05T17:39:22.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83c4afc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c0061fa0 con 0x7f83c00b00a0 2020-12-05T17:39:22.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83be59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83c005fe00 0x7f83c0061910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83be59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83c005fe00 0x7f83c0061910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38048/0 (socket says 172.21.15.32:38048) 2020-12-05T17:39:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83be59c700 1 -- 172.21.15.32:0/3744712505 learned_addr learned my addr 172.21.15.32:0/3744712505 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83b5d9b700 1 --2- 172.21.15.32:0/3744712505 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f83c00b00a0 0x7f83c014a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83be59c700 1 -- 172.21.15.32:0/3744712505 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f83c00b00a0 msgr2=0x7f83c014a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83be59c700 1 --2- 172.21.15.32:0/3744712505 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f83c00b00a0 0x7f83c014a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83be59c700 1 -- 172.21.15.32:0/3744712505 >> v1:172.21.15.61:6789/0 conn(0x7f83c00ac590 legacy=0x7f83c00aca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83be59c700 1 -- 172.21.15.32:0/3744712505 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c0062240 con 0x7f83c005fe00 2020-12-05T17:39:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83be59c700 1 --2- 172.21.15.32:0/3744712505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c005fe00 0x7f83c0061910 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7f83b0002f30 tx=0x7f83b0006e80).ready entity=mon.0 client_cookie=6cf8545012ef0725 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83bdd9b700 1 -- 172.21.15.32:0/3744712505 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83b00097f0 con 0x7f83c005fe00 2020-12-05T17:39:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83bdd9b700 1 -- 172.21.15.32:0/3744712505 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b0009950 con 0x7f83c005fe00 2020-12-05T17:39:22.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83bdd9b700 1 -- 172.21.15.32:0/3744712505 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83b0009cb0 con 0x7f83c005fe00 2020-12-05T17:39:22.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83c4afc700 1 -- 172.21.15.32:0/3744712505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c005fe00 msgr2=0x7f83c0061910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:22.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83c4afc700 1 --2- 172.21.15.32:0/3744712505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c005fe00 0x7f83c0061910 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7f83b0002f30 tx=0x7f83b0006e80).stop 2020-12-05T17:39:22.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83c4afc700 1 -- 172.21.15.32:0/3744712505 shutdown_connections 2020-12-05T17:39:22.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83c4afc700 1 --2- 172.21.15.32:0/3744712505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c005fe00 0x7f83c0061910 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83c4afc700 1 --2- 172.21.15.32:0/3744712505 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f83c00b00a0 0x7f83c014a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83c4afc700 1 -- 172.21.15.32:0/3744712505 >> 172.21.15.32:0/3744712505 conn(0x7f83c00b57c0 msgr2=0x7f83c005f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:22.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83c4afc700 1 -- 172.21.15.32:0/3744712505 shutdown_connections 2020-12-05T17:39:22.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.683+0000 7f83c4afc700 1 -- 172.21.15.32:0/3744712505 wait complete. 2020-12-05T17:39:22.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83c4afc700 1 Processor -- start 2020-12-05T17:39:22.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83c4afc700 1 -- start start 2020-12-05T17:39:22.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83c4afc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c005fe00 0x7f83c01566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83c4afc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f83c00ac590 0x7f83c0066540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83c4afc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f83c00b00a0 0x7f83c0063e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83c4afc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c0062e70 con 0x7f83c005fe00 2020-12-05T17:39:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83c4afc700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c0066a30 con 0x7f83c00b00a0 2020-12-05T17:39:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83c4afc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c00b09c0 con 0x7f83c00ac590 2020-12-05T17:39:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83be59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c005fe00 0x7f83c01566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83be59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c005fe00 0x7f83c01566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38052/0 (socket says 172.21.15.32:38052) 2020-12-05T17:39:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83be59c700 1 -- 172.21.15.32:0/3391000633 learned_addr learned my addr 172.21.15.32:0/3391000633 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83b5d9b700 1 --2- 172.21.15.32:0/3391000633 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f83c00ac590 0x7f83c0066540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83bed9d700 1 --2- 172.21.15.32:0/3391000633 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f83c00b00a0 0x7f83c0063e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83be59c700 1 -- 172.21.15.32:0/3391000633 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f83c00ac590 msgr2=0x7f83c0066540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:22.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83be59c700 1 --2- 172.21.15.32:0/3391000633 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f83c00ac590 0x7f83c0066540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83be59c700 1 -- 172.21.15.32:0/3391000633 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f83c00b00a0 msgr2=0x7f83c0063e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:22.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83be59c700 1 --2- 172.21.15.32:0/3391000633 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f83c00b00a0 0x7f83c0063e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83be59c700 1 -- 172.21.15.32:0/3391000633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c0147c50 con 0x7f83c005fe00 2020-12-05T17:39:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83be59c700 1 --2- 172.21.15.32:0/3391000633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c005fe00 0x7f83c01566c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7f83b0002f30 tx=0x7f83b0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83b7fff700 1 -- 172.21.15.32:0/3391000633 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83b0011b80 con 0x7f83c005fe00 2020-12-05T17:39:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83b7fff700 1 -- 172.21.15.32:0/3391000633 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b0011ce0 con 0x7f83c005fe00 2020-12-05T17:39:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83c4afc700 1 -- 172.21.15.32:0/3391000633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c0063270 con 0x7f83c005fe00 2020-12-05T17:39:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83b7fff700 1 -- 172.21.15.32:0/3391000633 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83b001e650 con 0x7f83c005fe00 2020-12-05T17:39:22.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.687+0000 7f83c4afc700 1 -- 172.21.15.32:0/3391000633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c0064330 con 0x7f83c005fe00 2020-12-05T17:39:22.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.691+0000 7f83c4afc700 1 -- 172.21.15.32:0/3391000633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c0147de0 con 0x7f83c005fe00 2020-12-05T17:39:22.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.691+0000 7f83b7fff700 1 -- 172.21.15.32:0/3391000633 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f83b0025030 con 0x7f83c005fe00 2020-12-05T17:39:22.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.691+0000 7f83b7fff700 1 --2- 172.21.15.32:0/3391000633 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f83a40553b0 0x7f83a4057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.691+0000 7f83b7fff700 1 -- 172.21.15.32:0/3391000633 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f83b006f4c0 con 0x7f83c005fe00 2020-12-05T17:39:22.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.691+0000 7f83b5d9b700 1 --2- 172.21.15.32:0/3391000633 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f83a40553b0 0x7f83a4057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.691+0000 7f83b5d9b700 1 --2- 172.21.15.32:0/3391000633 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f83a40553b0 0x7f83a4057870 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f83a800c000 tx=0x7f83a8008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:22.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.699+0000 7f83b7fff700 1 -- 172.21.15.32:0/3391000633 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f83b0045550 con 0x7f83c005fe00 2020-12-05T17:39:22.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.839+0000 7f83c4afc700 1 -- 172.21.15.32:0/3391000633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83c0057050 con 0x7f83c005fe00 2020-12-05T17:39:22.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.843+0000 7f83b7fff700 1 -- 172.21.15.32:0/3391000633 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f83b0023020 con 0x7f83c005fe00 2020-12-05T17:39:22.849 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:22.849 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":91,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:39:22.261286+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:39:19.993416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2020-12-05T17:39:21.252765+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:39:22.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.843+0000 7f83c4afc700 1 -- 172.21.15.32:0/3391000633 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f83a40553b0 msgr2=0x7f83a4057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:22.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.843+0000 7f83c4afc700 1 --2- 172.21.15.32:0/3391000633 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f83a40553b0 0x7f83a4057870 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f83a800c000 tx=0x7f83a8008040).stop 2020-12-05T17:39:22.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.843+0000 7f83c4afc700 1 -- 172.21.15.32:0/3391000633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c005fe00 msgr2=0x7f83c01566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:22.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.843+0000 7f83c4afc700 1 --2- 172.21.15.32:0/3391000633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c005fe00 0x7f83c01566c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7f83b0002f30 tx=0x7f83b0015000).stop 2020-12-05T17:39:22.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.843+0000 7f83c4afc700 1 -- 172.21.15.32:0/3391000633 shutdown_connections 2020-12-05T17:39:22.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.843+0000 7f83c4afc700 1 --2- 172.21.15.32:0/3391000633 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f83c00b00a0 0x7f83c0063e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.843+0000 7f83c4afc700 1 --2- 172.21.15.32:0/3391000633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c005fe00 0x7f83c01566c0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.843+0000 7f83c4afc700 1 --2- 172.21.15.32:0/3391000633 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f83a40553b0 0x7f83a4057870 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.843+0000 7f83c4afc700 1 --2- 172.21.15.32:0/3391000633 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f83c00ac590 0x7f83c0066540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.843+0000 7f83c4afc700 1 -- 172.21.15.32:0/3391000633 >> 172.21.15.32:0/3391000633 conn(0x7f83c00b57c0 msgr2=0x7f83c00af690 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:22.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.843+0000 7f83c4afc700 1 -- 172.21.15.32:0/3391000633 shutdown_connections 2020-12-05T17:39:22.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.843+0000 7f83c4afc700 1 -- 172.21.15.32:0/3391000633 wait complete. 2020-12-05T17:39:22.863 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-12-05T17:39:22.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.979+0000 7f36856e8700 1 Processor -- start 2020-12-05T17:39:22.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.979+0000 7f36856e8700 1 -- start start 2020-12-05T17:39:22.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.979+0000 7f36856e8700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f368005f3b0 0x7f368005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.979+0000 7f36856e8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f368005feb0 0x7f3680061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.979+0000 7f36856e8700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36800b2df0 con 0x7f36800b2fb0 2020-12-05T17:39:22.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.979+0000 7f36856e8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36800b3c90 con 0x7f368005feb0 2020-12-05T17:39:22.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.979+0000 7f36856e8700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36800b2b50 con 0x7f368005f3b0 2020-12-05T17:39:22.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.979+0000 7f367f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f368005feb0 0x7f3680061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.979+0000 7f367f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f368005feb0 0x7f3680061780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38064/0 (socket says 172.21.15.32:38064) 2020-12-05T17:39:22.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.979+0000 7f367f7fe700 1 -- 172.21.15.32:0/927666562 learned_addr learned my addr 172.21.15.32:0/927666562 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:22.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.979+0000 7f367ffff700 1 --2- 172.21.15.32:0/927666562 >> v2:172.21.15.61:3300/0 conn(0x7f368005f3b0 0x7f368005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.979+0000 7f367f7fe700 1 -- 172.21.15.32:0/927666562 >> v2:172.21.15.61:3300/0 conn(0x7f368005f3b0 msgr2=0x7f368005f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:22.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.979+0000 7f367f7fe700 1 --2- 172.21.15.32:0/927666562 >> v2:172.21.15.61:3300/0 conn(0x7f368005f3b0 0x7f368005f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.979+0000 7f367f7fe700 1 -- 172.21.15.32:0/927666562 >> v1:172.21.15.61:6789/0 conn(0x7f36800b2fb0 legacy=0x7f368013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:22.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.979+0000 7f367f7fe700 1 -- 172.21.15.32:0/927666562 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3680142820 con 0x7f368005feb0 2020-12-05T17:39:22.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.979+0000 7f367ffff700 1 --2- 172.21.15.32:0/927666562 >> v2:172.21.15.61:3300/0 conn(0x7f368005f3b0 0x7f368005f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:22.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.979+0000 7f367f7fe700 1 --2- 172.21.15.32:0/927666562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368005feb0 0x7f3680061780 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7f3670002f30 tx=0x7f3670006e80).ready entity=mon.0 client_cookie=f2f83e58c3ad5c17 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:22.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.983+0000 7f367effd700 1 -- 172.21.15.32:0/927666562 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36700097f0 con 0x7f368005feb0 2020-12-05T17:39:22.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.983+0000 7f367effd700 1 -- 172.21.15.32:0/927666562 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3670009950 con 0x7f368005feb0 2020-12-05T17:39:22.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.983+0000 7f367effd700 1 -- 172.21.15.32:0/927666562 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3670009cb0 con 0x7f368005feb0 2020-12-05T17:39:22.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.983+0000 7f36856e8700 1 -- 172.21.15.32:0/927666562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368005feb0 msgr2=0x7f3680061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:22.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.983+0000 7f36856e8700 1 --2- 172.21.15.32:0/927666562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368005feb0 0x7f3680061780 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7f3670002f30 tx=0x7f3670006e80).stop 2020-12-05T17:39:22.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.983+0000 7f36856e8700 1 -- 172.21.15.32:0/927666562 shutdown_connections 2020-12-05T17:39:22.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.983+0000 7f36856e8700 1 --2- 172.21.15.32:0/927666562 >> v2:172.21.15.61:3300/0 conn(0x7f368005f3b0 0x7f368005f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.983+0000 7f36856e8700 1 --2- 172.21.15.32:0/927666562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368005feb0 0x7f3680061780 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:22.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.983+0000 7f36856e8700 1 -- 172.21.15.32:0/927666562 >> 172.21.15.32:0/927666562 conn(0x7f36800b8f20 msgr2=0x7f36800b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:22.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.983+0000 7f36856e8700 1 -- 172.21.15.32:0/927666562 shutdown_connections 2020-12-05T17:39:22.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.983+0000 7f36856e8700 1 -- 172.21.15.32:0/927666562 wait complete. 2020-12-05T17:39:22.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.983+0000 7f36856e8700 1 Processor -- start 2020-12-05T17:39:22.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.983+0000 7f36856e8700 1 -- start start 2020-12-05T17:39:22.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f36856e8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368005f3b0 0x7f36801445c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f36856e8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f368005feb0 0x7f3680141c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f36856e8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f36800b2fb0 0x7f3680142170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:22.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f36856e8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3680147180 con 0x7f368005f3b0 2020-12-05T17:39:22.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f36856e8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3680142660 con 0x7f368005feb0 2020-12-05T17:39:22.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f36856e8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36801464d0 con 0x7f36800b2fb0 2020-12-05T17:39:22.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f367ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368005f3b0 0x7f36801445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f367ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368005f3b0 0x7f36801445c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38066/0 (socket says 172.21.15.32:38066) 2020-12-05T17:39:22.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f367f7fe700 1 --2- 172.21.15.32:0/3584409784 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f368005feb0 0x7f3680141c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:22.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f367ffff700 1 -- 172.21.15.32:0/3584409784 learned_addr learned my addr 172.21.15.32:0/3584409784 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:23.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f3684ee7700 1 --2- 172.21.15.32:0/3584409784 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f36800b2fb0 0x7f3680142170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:23.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f367ffff700 1 -- 172.21.15.32:0/3584409784 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f36800b2fb0 msgr2=0x7f3680142170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:23.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f367ffff700 1 --2- 172.21.15.32:0/3584409784 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f36800b2fb0 0x7f3680142170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f367ffff700 1 -- 172.21.15.32:0/3584409784 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f368005feb0 msgr2=0x7f3680141c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:23.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f367ffff700 1 --2- 172.21.15.32:0/3584409784 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f368005feb0 0x7f3680141c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f367ffff700 1 -- 172.21.15.32:0/3584409784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36801473b0 con 0x7f368005f3b0 2020-12-05T17:39:23.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f367f7fe700 1 --2- 172.21.15.32:0/3584409784 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f368005feb0 0x7f3680141c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:23.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f367ffff700 1 --2- 172.21.15.32:0/3584409784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368005f3b0 0x7f36801445c0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7f3674006910 tx=0x7f3674008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:23.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f367d7fa700 1 -- 172.21.15.32:0/3584409784 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3674009980 con 0x7f368005f3b0 2020-12-05T17:39:23.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f367d7fa700 1 -- 172.21.15.32:0/3584409784 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3674009ae0 con 0x7f368005f3b0 2020-12-05T17:39:23.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f36856e8700 1 -- 172.21.15.32:0/3584409784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3680147650 con 0x7f368005f3b0 2020-12-05T17:39:23.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f367d7fa700 1 -- 172.21.15.32:0/3584409784 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3674009e40 con 0x7f368005f3b0 2020-12-05T17:39:23.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f36856e8700 1 -- 172.21.15.32:0/3584409784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3680145570 con 0x7f368005f3b0 2020-12-05T17:39:23.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f367d7fa700 1 -- 172.21.15.32:0/3584409784 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3674013480 con 0x7f368005f3b0 2020-12-05T17:39:23.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.987+0000 7f36856e8700 1 -- 172.21.15.32:0/3584409784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3680057050 con 0x7f368005f3b0 2020-12-05T17:39:23.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.991+0000 7f367d7fa700 1 --2- 172.21.15.32:0/3584409784 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f36600553b0 0x7f3660057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:23.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.991+0000 7f367f7fe700 1 --2- 172.21.15.32:0/3584409784 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f36600553b0 0x7f3660057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:23.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.991+0000 7f367d7fa700 1 -- 172.21.15.32:0/3584409784 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f367406e870 con 0x7f368005f3b0 2020-12-05T17:39:23.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.991+0000 7f367f7fe700 1 --2- 172.21.15.32:0/3584409784 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f36600553b0 0x7f3660057870 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f367001c000 tx=0x7f3670015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:23.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:22.995+0000 7f367d7fa700 1 -- 172.21.15.32:0/3584409784 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f36740448a0 con 0x7f368005f3b0 2020-12-05T17:39:23.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.179+0000 7f36856e8700 1 -- 172.21.15.32:0/3584409784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f368005aab0 con 0x7f368005f3b0 2020-12-05T17:39:23.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.279+0000 7f367d7fa700 1 -- 172.21.15.32:0/3584409784 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v42) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3674047ec0 con 0x7f368005f3b0 2020-12-05T17:39:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.283+0000 7f36856e8700 1 -- 172.21.15.32:0/3584409784 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f36600553b0 msgr2=0x7f3660057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.283+0000 7f36856e8700 1 --2- 172.21.15.32:0/3584409784 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f36600553b0 0x7f3660057870 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f367001c000 tx=0x7f3670015000).stop 2020-12-05T17:39:23.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.283+0000 7f36856e8700 1 -- 172.21.15.32:0/3584409784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368005f3b0 msgr2=0x7f36801445c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:23.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.283+0000 7f36856e8700 1 --2- 172.21.15.32:0/3584409784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368005f3b0 0x7f36801445c0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7f3674006910 tx=0x7f3674008040).stop 2020-12-05T17:39:23.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.283+0000 7f36856e8700 1 -- 172.21.15.32:0/3584409784 shutdown_connections 2020-12-05T17:39:23.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.283+0000 7f36856e8700 1 --2- 172.21.15.32:0/3584409784 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f36600553b0 0x7f3660057870 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.283+0000 7f36856e8700 1 --2- 172.21.15.32:0/3584409784 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f36800b2fb0 0x7f3680142170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.283+0000 7f36856e8700 1 --2- 172.21.15.32:0/3584409784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368005f3b0 0x7f36801445c0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.283+0000 7f36856e8700 1 --2- 172.21.15.32:0/3584409784 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f368005feb0 0x7f3680141c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.283+0000 7f36856e8700 1 -- 172.21.15.32:0/3584409784 >> 172.21.15.32:0/3584409784 conn(0x7f36800b8f20 msgr2=0x7f36800b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:23.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.283+0000 7f36856e8700 1 -- 172.21.15.32:0/3584409784 shutdown_connections 2020-12-05T17:39:23.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.283+0000 7f36856e8700 1 -- 172.21.15.32:0/3584409784 wait complete. 2020-12-05T17:39:23.302 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:39:23.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.415+0000 7fb7fd9e5700 1 Processor -- start 2020-12-05T17:39:23.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.415+0000 7fb7fd9e5700 1 -- start start 2020-12-05T17:39:23.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.415+0000 7fb7fd9e5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7f80b1fc0 0x7fb7f80b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:23.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.415+0000 7fb7fd9e5700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb7f813f670 0x7fb7f813fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:23.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.415+0000 7fb7fd9e5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f80b23a0 con 0x7fb7f80af0d0 2020-12-05T17:39:23.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.415+0000 7fb7fd9e5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f813f280 con 0x7fb7f80b1fc0 2020-12-05T17:39:23.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.415+0000 7fb7fd9e5700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f80abfa0 con 0x7fb7f813f670 2020-12-05T17:39:23.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.419+0000 7fb7fc9e3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7f80b1fc0 0x7fb7f80b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:23.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.419+0000 7fb7f7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb7f80af0d0 0x7fb7f813c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40726/0 (socket says 172.21.15.32:40726) 2020-12-05T17:39:23.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.419+0000 7fb7fc9e3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7f80b1fc0 0x7fb7f80b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38074/0 (socket says 172.21.15.32:38074) 2020-12-05T17:39:23.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.419+0000 7fb7f7fff700 1 -- 172.21.15.32:0/170529839 learned_addr learned my addr 172.21.15.32:0/170529839 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:23.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.419+0000 7fb7fd1e4700 1 --2- 172.21.15.32:0/170529839 >> v2:172.21.15.61:3300/0 conn(0x7fb7f813f670 0x7fb7f813fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:23.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.419+0000 7fb7f77fe700 1 -- 172.21.15.32:0/170529839 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125979817 0 0) 0x7fb7f80b23a0 con 0x7fb7f80af0d0 2020-12-05T17:39:23.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.419+0000 7fb7fc9e3700 1 -- 172.21.15.32:0/170529839 >> v2:172.21.15.61:3300/0 conn(0x7fb7f813f670 msgr2=0x7fb7f813fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:23.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.419+0000 7fb7fc9e3700 1 --2- 172.21.15.32:0/170529839 >> v2:172.21.15.61:3300/0 conn(0x7fb7f813f670 0x7fb7f813fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.419+0000 7fb7fc9e3700 1 -- 172.21.15.32:0/170529839 >> v1:172.21.15.32:6789/0 conn(0x7fb7f80af0d0 legacy=0x7fb7f813c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:23.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.419+0000 7fb7fc9e3700 1 -- 172.21.15.32:0/170529839 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f80ac240 con 0x7fb7f80b1fc0 2020-12-05T17:39:23.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.419+0000 7fb7fc9e3700 1 --2- 172.21.15.32:0/170529839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f80b1fc0 0x7fb7f80b0fa0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7fb7e8002f30 tx=0x7fb7e8006dc0).ready entity=mon.0 client_cookie=f710ae1df3ef7e3c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:23.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.419+0000 7fb7f77fe700 1 -- 172.21.15.32:0/170529839 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7e80097b0 con 0x7fb7f80b1fc0 2020-12-05T17:39:23.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.419+0000 7fb7f77fe700 1 -- 172.21.15.32:0/170529839 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e8009910 con 0x7fb7f80b1fc0 2020-12-05T17:39:23.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.419+0000 7fb7fd9e5700 1 -- 172.21.15.32:0/170529839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f80b1fc0 msgr2=0x7fb7f80b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:23.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.419+0000 7fb7fd9e5700 1 --2- 172.21.15.32:0/170529839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f80b1fc0 0x7fb7f80b0fa0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7fb7e8002f30 tx=0x7fb7e8006dc0).stop 2020-12-05T17:39:23.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.419+0000 7fb7fd9e5700 1 -- 172.21.15.32:0/170529839 shutdown_connections 2020-12-05T17:39:23.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.419+0000 7fb7fd9e5700 1 --2- 172.21.15.32:0/170529839 >> v2:172.21.15.61:3300/0 conn(0x7fb7f813f670 0x7fb7f813fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.419+0000 7fb7fd9e5700 1 --2- 172.21.15.32:0/170529839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f80b1fc0 0x7fb7f80b0fa0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.419+0000 7fb7fd9e5700 1 -- 172.21.15.32:0/170529839 >> 172.21.15.32:0/170529839 conn(0x7fb7f80b5530 msgr2=0x7fb7f80b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:23.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.419+0000 7fb7fd9e5700 1 -- 172.21.15.32:0/170529839 shutdown_connections 2020-12-05T17:39:23.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.419+0000 7fb7fd9e5700 1 -- 172.21.15.32:0/170529839 wait complete. 2020-12-05T17:39:23.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.419+0000 7fb7fd9e5700 1 Processor -- start 2020-12-05T17:39:23.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.419+0000 7fb7fd9e5700 1 -- start start 2020-12-05T17:39:23.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7fd9e5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f80af0d0 0x7fb7f813c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:23.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7fd9e5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7f80b1fc0 0x7fb7f813cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:23.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7fd9e5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb7f813f670 0x7fb7f813d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:23.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7fd9e5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f8142b60 con 0x7fb7f80af0d0 2020-12-05T17:39:23.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7fd9e5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f8142e00 con 0x7fb7f813f670 2020-12-05T17:39:23.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7fd9e5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f81430a0 con 0x7fb7f80b1fc0 2020-12-05T17:39:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7fc9e3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f80af0d0 0x7fb7f813c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7fc9e3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f80af0d0 0x7fb7f813c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38080/0 (socket says 172.21.15.32:38080) 2020-12-05T17:39:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7fc9e3700 1 -- 172.21.15.32:0/254676436 learned_addr learned my addr 172.21.15.32:0/254676436 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7f7fff700 1 --2- 172.21.15.32:0/254676436 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7f80b1fc0 0x7fb7f813cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7fd1e4700 1 --2- 172.21.15.32:0/254676436 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb7f813f670 0x7fb7f813d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7fc9e3700 1 -- 172.21.15.32:0/254676436 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7f80b1fc0 msgr2=0x7fb7f813cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7fc9e3700 1 --2- 172.21.15.32:0/254676436 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7f80b1fc0 0x7fb7f813cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7fc9e3700 1 -- 172.21.15.32:0/254676436 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb7f813f670 msgr2=0x7fb7f813d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7fc9e3700 1 --2- 172.21.15.32:0/254676436 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb7f813f670 0x7fb7f813d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7fc9e3700 1 -- 172.21.15.32:0/254676436 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f813ef40 con 0x7fb7f80af0d0 2020-12-05T17:39:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7f7fff700 1 --2- 172.21.15.32:0/254676436 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7f80b1fc0 0x7fb7f813cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7fc9e3700 1 --2- 172.21.15.32:0/254676436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f80af0d0 0x7fb7f813c2d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7fb7e8002f30 tx=0x7fb7e8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7f5ffb700 1 -- 172.21.15.32:0/254676436 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7e8011ce0 con 0x7fb7f80af0d0 2020-12-05T17:39:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7f5ffb700 1 -- 172.21.15.32:0/254676436 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e8011e40 con 0x7fb7f80af0d0 2020-12-05T17:39:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7fd9e5700 1 -- 172.21.15.32:0/254676436 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f813cfd0 con 0x7fb7f80af0d0 2020-12-05T17:39:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7f5ffb700 1 -- 172.21.15.32:0/254676436 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7e801d630 con 0x7fb7f80af0d0 2020-12-05T17:39:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7fd9e5700 1 -- 172.21.15.32:0/254676436 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f80aa980 con 0x7fb7f80af0d0 2020-12-05T17:39:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.423+0000 7fb7fd9e5700 1 -- 172.21.15.32:0/254676436 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f8057050 con 0x7fb7f80af0d0 2020-12-05T17:39:23.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.427+0000 7fb7f5ffb700 1 -- 172.21.15.32:0/254676436 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb7e8017070 con 0x7fb7f80af0d0 2020-12-05T17:39:23.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.427+0000 7fb7f5ffb700 1 --2- 172.21.15.32:0/254676436 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb7d80553b0 0x7fb7d8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:23.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.427+0000 7fb7f5ffb700 1 -- 172.21.15.32:0/254676436 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb7e806eda0 con 0x7fb7f80af0d0 2020-12-05T17:39:23.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.427+0000 7fb7f7fff700 1 --2- 172.21.15.32:0/254676436 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb7d80553b0 0x7fb7d8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:23.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.427+0000 7fb7f7fff700 1 --2- 172.21.15.32:0/254676436 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb7d80553b0 0x7fb7d8057870 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7fb7ec00b000 tx=0x7fb7ec009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:23.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.431+0000 7fb7f5ffb700 1 -- 172.21.15.32:0/254676436 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb7e8044ea0 con 0x7fb7f80af0d0 2020-12-05T17:39:23.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.631+0000 7fb7fd9e5700 1 -- 172.21.15.32:0/254676436 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7f8069ee0 con 0x7fb7f80af0d0 2020-12-05T17:39:23.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.631+0000 7fb7f5ffb700 1 -- 172.21.15.32:0/254676436 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+2140 (secure 0 0 0) 0x7fb7e8022020 con 0x7fb7f80af0d0 2020-12-05T17:39:23.636 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:23.636 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":42,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6823,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2664846066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2664846066},{"type":"v1","addr":"172.21.15.61:6833","nonce":2664846066}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:39:22.261725+0000","modified":"2020-12-05T17:39:23.265348+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6930":{"gid":6930,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4087622496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4087622496},{"type":"v1","addr":"172.21.15.32:6835","nonce":4087622496}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2020-12-05T17:39:23.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.631+0000 7fb7fd9e5700 1 -- 172.21.15.32:0/254676436 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb7d80553b0 msgr2=0x7fb7d8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:23.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.631+0000 7fb7fd9e5700 1 --2- 172.21.15.32:0/254676436 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb7d80553b0 0x7fb7d8057870 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7fb7ec00b000 tx=0x7fb7ec009040).stop 2020-12-05T17:39:23.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.631+0000 7fb7fd9e5700 1 -- 172.21.15.32:0/254676436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f80af0d0 msgr2=0x7fb7f813c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:23.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.631+0000 7fb7fd9e5700 1 --2- 172.21.15.32:0/254676436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f80af0d0 0x7fb7f813c2d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7fb7e8002f30 tx=0x7fb7e8015000).stop 2020-12-05T17:39:23.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.631+0000 7fb7fd9e5700 1 -- 172.21.15.32:0/254676436 shutdown_connections 2020-12-05T17:39:23.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.631+0000 7fb7fd9e5700 1 --2- 172.21.15.32:0/254676436 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb7f813f670 0x7fb7f813d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.631+0000 7fb7fd9e5700 1 --2- 172.21.15.32:0/254676436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f80af0d0 0x7fb7f813c2d0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.631+0000 7fb7fd9e5700 1 --2- 172.21.15.32:0/254676436 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb7d80553b0 0x7fb7d8057870 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.631+0000 7fb7fd9e5700 1 --2- 172.21.15.32:0/254676436 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7f80b1fc0 0x7fb7f813cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.631+0000 7fb7fd9e5700 1 -- 172.21.15.32:0/254676436 >> 172.21.15.32:0/254676436 conn(0x7fb7f80b5530 msgr2=0x7fb7f80ad7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:23.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.631+0000 7fb7fd9e5700 1 -- 172.21.15.32:0/254676436 shutdown_connections 2020-12-05T17:39:23.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.631+0000 7fb7fd9e5700 1 -- 172.21.15.32:0/254676436 wait complete. 2020-12-05T17:39:23.641 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 42 2020-12-05T17:39:23.649 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:39:23.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.755+0000 7eff3f59e700 1 Processor -- start 2020-12-05T17:39:23.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.755+0000 7eff3f59e700 1 -- start start 2020-12-05T17:39:23.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.755+0000 7eff3f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff400b1fc0 0x7eff400af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:23.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.755+0000 7eff3f59e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7eff40147400 0x7eff40145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:23.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.755+0000 7eff3f59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff40146a90 con 0x7eff400ab710 2020-12-05T17:39:23.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.755+0000 7eff3f59e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff40061e90 con 0x7eff40147400 2020-12-05T17:39:23.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.755+0000 7eff3f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff400b0110 con 0x7eff400b1fc0 2020-12-05T17:39:23.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.755+0000 7eff3dd9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7eff400ab710 0x7eff40061780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40740/0 (socket says 172.21.15.32:40740) 2020-12-05T17:39:23.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.755+0000 7eff3dd9b700 1 -- 172.21.15.32:0/1563611155 learned_addr learned my addr 172.21.15.32:0/1563611155 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:23.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.755+0000 7eff3ed9d700 1 --2- 172.21.15.32:0/1563611155 >> v2:172.21.15.61:3300/0 conn(0x7eff40147400 0x7eff40145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:23.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.755+0000 7eff3e59c700 1 --2- 172.21.15.32:0/1563611155 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff400b1fc0 0x7eff400af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:23.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.755+0000 7eff3d59a700 1 -- 172.21.15.32:0/1563611155 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3359069438 0 0) 0x7eff40146a90 con 0x7eff400ab710 2020-12-05T17:39:23.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.755+0000 7eff3d59a700 1 -- 172.21.15.32:0/1563611155 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff20003230 con 0x7eff400ab710 2020-12-05T17:39:23.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.755+0000 7eff3d59a700 1 -- 172.21.15.32:0/1563611155 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1435377641 0 0) 0x7eff20003230 con 0x7eff400ab710 2020-12-05T17:39:23.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.755+0000 7eff3ed9d700 1 -- 172.21.15.32:0/1563611155 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff400b1fc0 msgr2=0x7eff400af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:23.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.755+0000 7eff3ed9d700 1 --2- 172.21.15.32:0/1563611155 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff400b1fc0 0x7eff400af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.755+0000 7eff3ed9d700 1 -- 172.21.15.32:0/1563611155 >> v1:172.21.15.32:6789/0 conn(0x7eff400ab710 legacy=0x7eff40061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:23.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.755+0000 7eff3ed9d700 1 -- 172.21.15.32:0/1563611155 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff400b23a0 con 0x7eff40147400 2020-12-05T17:39:23.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.755+0000 7eff3ed9d700 1 --2- 172.21.15.32:0/1563611155 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff40147400 0x7eff40145d10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7eff30001870 tx=0x7eff380091a0).ready entity=mon.1 client_cookie=d79c602f586f89e2 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:23.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.759+0000 7eff3d59a700 1 -- 172.21.15.32:0/1563611155 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff300016d0 con 0x7eff40147400 2020-12-05T17:39:23.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.759+0000 7eff3d59a700 1 -- 172.21.15.32:0/1563611155 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3800b800 con 0x7eff40147400 2020-12-05T17:39:23.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.759+0000 7eff3d59a700 1 -- 172.21.15.32:0/1563611155 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff3800b970 con 0x7eff40147400 2020-12-05T17:39:23.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.759+0000 7eff3f59e700 1 -- 172.21.15.32:0/1563611155 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff40147400 msgr2=0x7eff40145d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:23.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.759+0000 7eff3f59e700 1 --2- 172.21.15.32:0/1563611155 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff40147400 0x7eff40145d10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7eff30001870 tx=0x7eff380091a0).stop 2020-12-05T17:39:23.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.759+0000 7eff3f59e700 1 -- 172.21.15.32:0/1563611155 shutdown_connections 2020-12-05T17:39:23.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.759+0000 7eff3f59e700 1 --2- 172.21.15.32:0/1563611155 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff40147400 0x7eff40145d10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.759+0000 7eff3f59e700 1 --2- 172.21.15.32:0/1563611155 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff400b1fc0 0x7eff400af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.759+0000 7eff3f59e700 1 -- 172.21.15.32:0/1563611155 >> 172.21.15.32:0/1563611155 conn(0x7eff400b5530 msgr2=0x7eff400b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:23.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.759+0000 7eff3f59e700 1 -- 172.21.15.32:0/1563611155 shutdown_connections 2020-12-05T17:39:23.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.759+0000 7eff3f59e700 1 -- 172.21.15.32:0/1563611155 wait complete. 2020-12-05T17:39:23.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff3f59e700 1 Processor -- start 2020-12-05T17:39:23.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff3f59e700 1 -- start start 2020-12-05T17:39:23.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff3f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff400ab710 0x7eff40143030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:23.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff3f59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff400b1fc0 0x7eff40158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:23.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff3f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff40147400 0x7eff40159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:23.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff3f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff400ae4c0 con 0x7eff400ab710 2020-12-05T17:39:23.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff3f59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff40066cf0 con 0x7eff400b1fc0 2020-12-05T17:39:23.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff3f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff400a1e70 con 0x7eff40147400 2020-12-05T17:39:23.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff3e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff400ab710 0x7eff40143030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:23.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff3e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff400ab710 0x7eff40143030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38094/0 (socket says 172.21.15.32:38094) 2020-12-05T17:39:23.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff3e59c700 1 -- 172.21.15.32:0/536579928 learned_addr learned my addr 172.21.15.32:0/536579928 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:23.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff3ed9d700 1 --2- 172.21.15.32:0/536579928 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff40147400 0x7eff40159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:23.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff3dd9b700 1 --2- 172.21.15.32:0/536579928 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff400b1fc0 0x7eff40158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:23.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff3e59c700 1 -- 172.21.15.32:0/536579928 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff40147400 msgr2=0x7eff40159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:23.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff3e59c700 1 --2- 172.21.15.32:0/536579928 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff40147400 0x7eff40159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff3e59c700 1 -- 172.21.15.32:0/536579928 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff400b1fc0 msgr2=0x7eff40158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:23.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff3e59c700 1 --2- 172.21.15.32:0/536579928 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff400b1fc0 0x7eff40158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff3e59c700 1 -- 172.21.15.32:0/536579928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff40143ad0 con 0x7eff400ab710 2020-12-05T17:39:23.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff3e59c700 1 --2- 172.21.15.32:0/536579928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff400ab710 0x7eff40143030 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7eff2800c000 tx=0x7eff28006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:23.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff377fe700 1 -- 172.21.15.32:0/536579928 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff2800ca10 con 0x7eff400ab710 2020-12-05T17:39:23.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff377fe700 1 -- 172.21.15.32:0/536579928 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2800cb70 con 0x7eff400ab710 2020-12-05T17:39:23.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff3f59e700 1 -- 172.21.15.32:0/536579928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff4009e120 con 0x7eff400ab710 2020-12-05T17:39:23.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff377fe700 1 -- 172.21.15.32:0/536579928 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff280123f0 con 0x7eff400ab710 2020-12-05T17:39:23.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.763+0000 7eff3f59e700 1 -- 172.21.15.32:0/536579928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff40159530 con 0x7eff400ab710 2020-12-05T17:39:23.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.767+0000 7eff3f59e700 1 -- 172.21.15.32:0/536579928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff40057050 con 0x7eff400ab710 2020-12-05T17:39:23.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.767+0000 7eff377fe700 1 -- 172.21.15.32:0/536579928 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7eff28012550 con 0x7eff400ab710 2020-12-05T17:39:23.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.767+0000 7eff377fe700 1 --2- 172.21.15.32:0/536579928 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7eff24055350 0x7eff24057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:23.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.767+0000 7eff377fe700 1 -- 172.21.15.32:0/536579928 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7eff2806e480 con 0x7eff400ab710 2020-12-05T17:39:23.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.767+0000 7eff3dd9b700 1 --2- 172.21.15.32:0/536579928 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7eff24055350 0x7eff24057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:23.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.767+0000 7eff3dd9b700 1 --2- 172.21.15.32:0/536579928 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7eff24055350 0x7eff24057810 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7eff3000d000 tx=0x7eff30001ad0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:23.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.775+0000 7eff377fe700 1 -- 172.21.15.32:0/536579928 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7eff280445c0 con 0x7eff400ab710 2020-12-05T17:39:23.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.919+0000 7eff3f59e700 1 -- 172.21.15.32:0/536579928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff4009e310 con 0x7eff400ab710 2020-12-05T17:39:23.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.919+0000 7eff377fe700 1 -- 172.21.15.32:0/536579928 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16274 (secure 0 0 0) 0x7eff28047be0 con 0x7eff400ab710 2020-12-05T17:39:23.926 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:23.926 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":92,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:39:23.264356+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:39:19.993416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2020-12-05T17:39:21.252765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:39:23.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.923+0000 7eff3f59e700 1 -- 172.21.15.32:0/536579928 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7eff24055350 msgr2=0x7eff24057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:23.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.923+0000 7eff3f59e700 1 --2- 172.21.15.32:0/536579928 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7eff24055350 0x7eff24057810 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7eff3000d000 tx=0x7eff30001ad0).stop 2020-12-05T17:39:23.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.923+0000 7eff3f59e700 1 -- 172.21.15.32:0/536579928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff400ab710 msgr2=0x7eff40143030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:23.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.923+0000 7eff3f59e700 1 --2- 172.21.15.32:0/536579928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff400ab710 0x7eff40143030 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7eff2800c000 tx=0x7eff28006dc0).stop 2020-12-05T17:39:23.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.923+0000 7eff3f59e700 1 -- 172.21.15.32:0/536579928 shutdown_connections 2020-12-05T17:39:23.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.923+0000 7eff3f59e700 1 --2- 172.21.15.32:0/536579928 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7eff24055350 0x7eff24057810 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.923+0000 7eff3f59e700 1 --2- 172.21.15.32:0/536579928 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff40147400 0x7eff40159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.923+0000 7eff3f59e700 1 --2- 172.21.15.32:0/536579928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff400ab710 0x7eff40143030 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.923+0000 7eff3f59e700 1 --2- 172.21.15.32:0/536579928 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff400b1fc0 0x7eff40158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:23.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.923+0000 7eff3f59e700 1 -- 172.21.15.32:0/536579928 >> 172.21.15.32:0/536579928 conn(0x7eff400b5530 msgr2=0x7eff400b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:23.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.923+0000 7eff3f59e700 1 -- 172.21.15.32:0/536579928 shutdown_connections 2020-12-05T17:39:23.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:23.923+0000 7eff3f59e700 1 -- 172.21.15.32:0/536579928 wait complete. 2020-12-05T17:39:23.940 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:39:24.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.051+0000 7fc8c1343700 1 Processor -- start 2020-12-05T17:39:24.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.051+0000 7fc8c1343700 1 -- start start 2020-12-05T17:39:24.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.055+0000 7fc8c1343700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8bc0611b0 0x7fc8bc153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:24.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.055+0000 7fc8c1343700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8bc080a90 con 0x7fc8bc0b0240 2020-12-05T17:39:24.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.055+0000 7fc8c1343700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8bc0aec40 con 0x7fc8bc0ae860 2020-12-05T17:39:24.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.055+0000 7fc8c1343700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8bc152860 con 0x7fc8bc0611b0 2020-12-05T17:39:24.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.055+0000 7fc8bbfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc8bc0b0240 0x7fc8bc14c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40750/0 (socket says 172.21.15.32:40750) 2020-12-05T17:39:24.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.055+0000 7fc8c0b42700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8bc0611b0 0x7fc8bc153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:24.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.055+0000 7fc8bbfff700 1 -- 172.21.15.32:0/1659563747 learned_addr learned my addr 172.21.15.32:0/1659563747 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:24.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.055+0000 7fc8bb7fe700 1 -- 172.21.15.32:0/1659563747 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 667459709 0 0) 0x7fc8bc080a90 con 0x7fc8bc0b0240 2020-12-05T17:39:24.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.055+0000 7fc8bb7fe700 1 -- 172.21.15.32:0/1659563747 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8ac003230 con 0x7fc8bc0b0240 2020-12-05T17:39:24.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.055+0000 7fc8c0b42700 1 -- 172.21.15.32:0/1659563747 >> v1:172.21.15.61:6789/0 conn(0x7fc8bc0ae860 legacy=0x7fc8bc14f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:24.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.055+0000 7fc8c0b42700 1 -- 172.21.15.32:0/1659563747 >> v1:172.21.15.32:6789/0 conn(0x7fc8bc0b0240 legacy=0x7fc8bc14c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:24.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.055+0000 7fc8c0b42700 1 -- 172.21.15.32:0/1659563747 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8bc152b00 con 0x7fc8bc0611b0 2020-12-05T17:39:24.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.055+0000 7fc8c0b42700 1 --2- 172.21.15.32:0/1659563747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8bc0611b0 0x7fc8bc153240 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7fc8b0002f30 tx=0x7fc8b0006dc0).ready entity=mon.0 client_cookie=47816b9ce57bc254 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:24.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.055+0000 7fc8bb7fe700 1 -- 172.21.15.32:0/1659563747 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8b0009800 con 0x7fc8bc0611b0 2020-12-05T17:39:24.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.055+0000 7fc8bb7fe700 1 -- 172.21.15.32:0/1659563747 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b0009960 con 0x7fc8bc0611b0 2020-12-05T17:39:24.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.055+0000 7fc8bb7fe700 1 -- 172.21.15.32:0/1659563747 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8b0009cc0 con 0x7fc8bc0611b0 2020-12-05T17:39:24.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.055+0000 7fc8c1343700 1 -- 172.21.15.32:0/1659563747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8bc0611b0 msgr2=0x7fc8bc153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:24.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.055+0000 7fc8c1343700 1 --2- 172.21.15.32:0/1659563747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8bc0611b0 0x7fc8bc153240 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7fc8b0002f30 tx=0x7fc8b0006dc0).stop 2020-12-05T17:39:24.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.055+0000 7fc8c1343700 1 -- 172.21.15.32:0/1659563747 shutdown_connections 2020-12-05T17:39:24.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.055+0000 7fc8c1343700 1 --2- 172.21.15.32:0/1659563747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8bc0611b0 0x7fc8bc153240 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:24.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.055+0000 7fc8c1343700 1 -- 172.21.15.32:0/1659563747 >> 172.21.15.32:0/1659563747 conn(0x7fc8bc0b5f90 msgr2=0x7fc8bc0b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:24.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.055+0000 7fc8c1343700 1 -- 172.21.15.32:0/1659563747 shutdown_connections 2020-12-05T17:39:24.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.055+0000 7fc8c1343700 1 -- 172.21.15.32:0/1659563747 wait complete. 2020-12-05T17:39:24.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8c1343700 1 Processor -- start 2020-12-05T17:39:24.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8c1343700 1 -- start start 2020-12-05T17:39:24.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8c1343700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8bc0611b0 0x7fc8bc0b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:24.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8c1343700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc8bc0ae860 0x7fc8bc127e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:24.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8c1343700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8bc0b0240 0x7fc8bc128310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:24.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8c1343700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc120450 con 0x7fc8bc0611b0 2020-12-05T17:39:24.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8c1343700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc1279f0 con 0x7fc8bc0b0240 2020-12-05T17:39:24.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8c1343700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc126bb0 con 0x7fc8bc0ae860 2020-12-05T17:39:24.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8c0b42700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8bc0611b0 0x7fc8bc0b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:24.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8c0b42700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8bc0611b0 0x7fc8bc0b3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38108/0 (socket says 172.21.15.32:38108) 2020-12-05T17:39:24.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8c0b42700 1 -- 172.21.15.32:0/2623150589 learned_addr learned my addr 172.21.15.32:0/2623150589 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:24.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8bbfff700 1 --2- 172.21.15.32:0/2623150589 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc8bc0ae860 0x7fc8bc127e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:24.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8c25a5700 1 --2- 172.21.15.32:0/2623150589 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8bc0b0240 0x7fc8bc128310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:24.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8c0b42700 1 -- 172.21.15.32:0/2623150589 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc8bc0ae860 msgr2=0x7fc8bc127e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:24.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8c0b42700 1 --2- 172.21.15.32:0/2623150589 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc8bc0ae860 0x7fc8bc127e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:24.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8c0b42700 1 -- 172.21.15.32:0/2623150589 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8bc0b0240 msgr2=0x7fc8bc128310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:24.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8c0b42700 1 --2- 172.21.15.32:0/2623150589 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8bc0b0240 0x7fc8bc128310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:24.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8c0b42700 1 -- 172.21.15.32:0/2623150589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8bc064af0 con 0x7fc8bc0611b0 2020-12-05T17:39:24.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8c25a5700 1 --2- 172.21.15.32:0/2623150589 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8bc0b0240 0x7fc8bc128310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:39:24.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8c0b42700 1 --2- 172.21.15.32:0/2623150589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8bc0611b0 0x7fc8bc0b3010 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7fc8b0002f30 tx=0x7fc8b0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:24.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8b97fa700 1 -- 172.21.15.32:0/2623150589 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8b0011b00 con 0x7fc8bc0611b0 2020-12-05T17:39:24.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8b97fa700 1 -- 172.21.15.32:0/2623150589 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b0011c60 con 0x7fc8bc0611b0 2020-12-05T17:39:24.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8b97fa700 1 -- 172.21.15.32:0/2623150589 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8b001d440 con 0x7fc8bc0611b0 2020-12-05T17:39:24.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8c1343700 1 -- 172.21.15.32:0/2623150589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8bc064760 con 0x7fc8bc0611b0 2020-12-05T17:39:24.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.059+0000 7fc8c1343700 1 -- 172.21.15.32:0/2623150589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8bc1289a0 con 0x7fc8bc0611b0 2020-12-05T17:39:24.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.063+0000 7fc8c1343700 1 -- 172.21.15.32:0/2623150589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8bc127b60 con 0x7fc8bc0611b0 2020-12-05T17:39:24.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.063+0000 7fc8b97fa700 1 -- 172.21.15.32:0/2623150589 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc8b0024030 con 0x7fc8bc0611b0 2020-12-05T17:39:24.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.063+0000 7fc8b97fa700 1 --2- 172.21.15.32:0/2623150589 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc89c055320 0x7fc89c0577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:24.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.063+0000 7fc8b97fa700 1 -- 172.21.15.32:0/2623150589 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc8b006efe0 con 0x7fc8bc0611b0 2020-12-05T17:39:24.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.063+0000 7fc8bbfff700 1 --2- 172.21.15.32:0/2623150589 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc89c055320 0x7fc89c0577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:24.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.063+0000 7fc8bbfff700 1 --2- 172.21.15.32:0/2623150589 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc89c055320 0x7fc89c0577e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7fc8a800b000 tx=0x7fc8a8009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:24.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.067+0000 7fc8b97fa700 1 -- 172.21.15.32:0/2623150589 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc8b00450d0 con 0x7fc8bc0611b0 2020-12-05T17:39:24.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.259+0000 7fc8c1343700 1 -- 172.21.15.32:0/2623150589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fc8bc1289a0 con 0x7fc8bc0611b0 2020-12-05T17:39:24.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.267+0000 7fc8b97fa700 1 -- 172.21.15.32:0/2623150589 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v29) v1 ==== 184+0+0 (secure 0 0 0) 0x7fc8bc1289a0 con 0x7fc8bc0611b0 2020-12-05T17:39:24.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.271+0000 7fc8c1343700 1 -- 172.21.15.32:0/2623150589 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc89c055320 msgr2=0x7fc89c0577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:24.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.271+0000 7fc8c1343700 1 --2- 172.21.15.32:0/2623150589 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc89c055320 0x7fc89c0577e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7fc8a800b000 tx=0x7fc8a8009040).stop 2020-12-05T17:39:24.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.271+0000 7fc8c1343700 1 -- 172.21.15.32:0/2623150589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8bc0611b0 msgr2=0x7fc8bc0b3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:24.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.271+0000 7fc8c1343700 1 --2- 172.21.15.32:0/2623150589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8bc0611b0 0x7fc8bc0b3010 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7fc8b0002f30 tx=0x7fc8b0015000).stop 2020-12-05T17:39:24.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.271+0000 7fc8c1343700 1 -- 172.21.15.32:0/2623150589 shutdown_connections 2020-12-05T17:39:24.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.271+0000 7fc8c1343700 1 --2- 172.21.15.32:0/2623150589 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8bc0b0240 0x7fc8bc128310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:24.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.271+0000 7fc8c1343700 1 --2- 172.21.15.32:0/2623150589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8bc0611b0 0x7fc8bc0b3010 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:24.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.271+0000 7fc8c1343700 1 --2- 172.21.15.32:0/2623150589 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc89c055320 0x7fc89c0577e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:24.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.271+0000 7fc8c1343700 1 --2- 172.21.15.32:0/2623150589 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc8bc0ae860 0x7fc8bc127e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:24.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.271+0000 7fc8c1343700 1 -- 172.21.15.32:0/2623150589 >> 172.21.15.32:0/2623150589 conn(0x7fc8bc0b5f90 msgr2=0x7fc8bc122e50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:24.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.271+0000 7fc8c1343700 1 -- 172.21.15.32:0/2623150589 shutdown_connections 2020-12-05T17:39:24.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.271+0000 7fc8c1343700 1 -- 172.21.15.32:0/2623150589 wait complete. 2020-12-05T17:39:24.281 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2020-12-05T17:39:24.293 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:39:24.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.415+0000 7f6edd199700 1 Processor -- start 2020-12-05T17:39:24.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.415+0000 7f6edd199700 1 -- start start 2020-12-05T17:39:24.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.415+0000 7f6edd199700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f6ed80b0240 0x7f6ed8153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:24.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.415+0000 7f6edd199700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ed80ae860 0x7f6ed814f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:24.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.415+0000 7f6edd199700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ed8080a90 con 0x7f6ed80611b0 2020-12-05T17:39:24.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.415+0000 7f6edd199700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ed8152860 con 0x7f6ed80b0240 2020-12-05T17:39:24.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.415+0000 7f6edd199700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed8152b00 con 0x7f6ed80ae860 2020-12-05T17:39:24.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.415+0000 7f6ede3fb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ed80ae860 0x7f6ed814f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:24.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.415+0000 7f6ed7fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f6ed80b0240 0x7f6ed8153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:24.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.415+0000 7f6ede3fb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ed80ae860 0x7f6ed814f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:34910/0 (socket says 172.21.15.32:34910) 2020-12-05T17:39:24.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.415+0000 7f6ede3fb700 1 -- 172.21.15.32:0/1423367207 learned_addr learned my addr 172.21.15.32:0/1423367207 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:24.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.415+0000 7f6edc998700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f6ed80611b0 0x7f6ed814c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:53404/0 (socket says 172.21.15.32:53404) 2020-12-05T17:39:24.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.419+0000 7f6ed77fe700 1 -- 172.21.15.32:0/1423367207 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1928031330 0 0) 0x7f6ed8080a90 con 0x7f6ed80611b0 2020-12-05T17:39:24.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.419+0000 7f6ed77fe700 1 -- 172.21.15.32:0/1423367207 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ec0003230 con 0x7f6ed80611b0 2020-12-05T17:39:24.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.419+0000 7f6ede3fb700 1 -- 172.21.15.32:0/1423367207 >> v2:172.21.15.61:3300/0 conn(0x7f6ed80b0240 msgr2=0x7f6ed8153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:24.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.419+0000 7f6ede3fb700 1 --2- 172.21.15.32:0/1423367207 >> v2:172.21.15.61:3300/0 conn(0x7f6ed80b0240 0x7f6ed8153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:24.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.419+0000 7f6ede3fb700 1 -- 172.21.15.32:0/1423367207 >> v1:172.21.15.61:6789/0 conn(0x7f6ed80611b0 legacy=0x7f6ed814c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:24.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.419+0000 7f6ede3fb700 1 -- 172.21.15.32:0/1423367207 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed80aec90 con 0x7f6ed80ae860 2020-12-05T17:39:24.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.419+0000 7f6ede3fb700 1 --2- 172.21.15.32:0/1423367207 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ed80ae860 0x7f6ed814f0d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f6ed0002f30 tx=0x7f6ed0006e80).ready entity=mon.2 client_cookie=33edd348b5de26ab server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:24.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.419+0000 7f6ed77fe700 1 -- 172.21.15.32:0/1423367207 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ed0009640 con 0x7f6ed80ae860 2020-12-05T17:39:24.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.419+0000 7f6ed77fe700 1 -- 172.21.15.32:0/1423367207 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed00097a0 con 0x7f6ed80ae860 2020-12-05T17:39:24.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.419+0000 7f6ed77fe700 1 -- 172.21.15.32:0/1423367207 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ed0009b00 con 0x7f6ed80ae860 2020-12-05T17:39:24.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.419+0000 7f6edd199700 1 -- 172.21.15.32:0/1423367207 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ed80ae860 msgr2=0x7f6ed814f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:24.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.419+0000 7f6edd199700 1 --2- 172.21.15.32:0/1423367207 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ed80ae860 0x7f6ed814f0d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f6ed0002f30 tx=0x7f6ed0006e80).stop 2020-12-05T17:39:24.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.419+0000 7f6edd199700 1 -- 172.21.15.32:0/1423367207 shutdown_connections 2020-12-05T17:39:24.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.419+0000 7f6edd199700 1 --2- 172.21.15.32:0/1423367207 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ed80ae860 0x7f6ed814f0d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:24.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.419+0000 7f6edd199700 1 --2- 172.21.15.32:0/1423367207 >> v2:172.21.15.61:3300/0 conn(0x7f6ed80b0240 0x7f6ed8153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:24.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.419+0000 7f6edd199700 1 -- 172.21.15.32:0/1423367207 >> 172.21.15.32:0/1423367207 conn(0x7f6ed80b5f90 msgr2=0x7f6ed80b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:24.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.419+0000 7f6edd199700 1 -- 172.21.15.32:0/1423367207 shutdown_connections 2020-12-05T17:39:24.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.419+0000 7f6edd199700 1 -- 172.21.15.32:0/1423367207 wait complete. 2020-12-05T17:39:24.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6edd199700 1 Processor -- start 2020-12-05T17:39:24.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6edd199700 1 -- start start 2020-12-05T17:39:24.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6edd199700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ed80611b0 0x7f6ed8063fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:24.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6edd199700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ed80ae860 0x7f6ed8077350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:24.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6edd199700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6ed80b0240 0x7f6ed8077840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:24.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6edd199700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed814f960 con 0x7f6ed80ae860 2020-12-05T17:39:24.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6edd199700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed8075050 con 0x7f6ed80b0240 2020-12-05T17:39:24.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6edd199700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed8074db0 con 0x7f6ed80611b0 2020-12-05T17:39:24.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6ed7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ed80ae860 0x7f6ed8077350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:24.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6ed7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ed80ae860 0x7f6ed8077350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38124/0 (socket says 172.21.15.32:38124) 2020-12-05T17:39:24.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6ed7fff700 1 -- 172.21.15.32:0/529637458 learned_addr learned my addr 172.21.15.32:0/529637458 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:24.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6edc998700 1 --2- 172.21.15.32:0/529637458 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ed80611b0 0x7f6ed8063fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:24.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6ede3fb700 1 --2- 172.21.15.32:0/529637458 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6ed80b0240 0x7f6ed8077840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:24.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6ed7fff700 1 -- 172.21.15.32:0/529637458 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ed80611b0 msgr2=0x7f6ed8063fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:24.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6ed7fff700 1 --2- 172.21.15.32:0/529637458 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ed80611b0 0x7f6ed8063fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:24.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6ed7fff700 1 -- 172.21.15.32:0/529637458 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6ed80b0240 msgr2=0x7f6ed8077840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:24.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6ed7fff700 1 --2- 172.21.15.32:0/529637458 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6ed80b0240 0x7f6ed8077840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:24.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6ed7fff700 1 -- 172.21.15.32:0/529637458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed8074550 con 0x7f6ed80ae860 2020-12-05T17:39:24.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6ed7fff700 1 --2- 172.21.15.32:0/529637458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ed80ae860 0x7f6ed8077350 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7f6ecc00c000 tx=0x7f6ecc006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:24.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6ed57fa700 1 -- 172.21.15.32:0/529637458 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ecc00ca10 con 0x7f6ed80ae860 2020-12-05T17:39:24.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6ed57fa700 1 -- 172.21.15.32:0/529637458 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ecc00cb70 con 0x7f6ed80ae860 2020-12-05T17:39:24.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6edd199700 1 -- 172.21.15.32:0/529637458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ed814fdc0 con 0x7f6ed80ae860 2020-12-05T17:39:24.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6ed57fa700 1 -- 172.21.15.32:0/529637458 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ecc00be50 con 0x7f6ed80ae860 2020-12-05T17:39:24.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.423+0000 7f6edd199700 1 -- 172.21.15.32:0/529637458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed807eec0 con 0x7f6ed80ae860 2020-12-05T17:39:24.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.427+0000 7f6edd199700 1 -- 172.21.15.32:0/529637458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ed8057050 con 0x7f6ed80ae860 2020-12-05T17:39:24.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.427+0000 7f6ed57fa700 1 -- 172.21.15.32:0/529637458 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6ecc01b410 con 0x7f6ed80ae860 2020-12-05T17:39:24.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.427+0000 7f6ed57fa700 1 --2- 172.21.15.32:0/529637458 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6eb8055320 0x7f6eb80577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:24.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.427+0000 7f6ed57fa700 1 -- 172.21.15.32:0/529637458 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6ecc06e130 con 0x7f6ed80ae860 2020-12-05T17:39:24.443 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.427+0000 7f6edc998700 1 --2- 172.21.15.32:0/529637458 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6eb8055320 0x7f6eb80577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:24.443 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.427+0000 7f6edc998700 1 --2- 172.21.15.32:0/529637458 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6eb8055320 0x7f6eb80577e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f6ec800d000 tx=0x7f6ec8007e00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:24.443 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.435+0000 7f6ed57fa700 1 -- 172.21.15.32:0/529637458 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6ecc044270 con 0x7f6ed80ae860 2020-12-05T17:39:24.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.627+0000 7f6edd199700 1 -- 172.21.15.32:0/529637458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f6ed80b6da0 con 0x7f6ed80ae860 2020-12-05T17:39:24.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.635+0000 7f6ed57fa700 1 -- 172.21.15.32:0/529637458 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v30) v1 ==== 184+0+0 (secure 0 0 0) 0x7f6ecc047890 con 0x7f6ed80ae860 2020-12-05T17:39:24.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.635+0000 7f6edd199700 1 -- 172.21.15.32:0/529637458 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6eb8055320 msgr2=0x7f6eb80577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:24.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.635+0000 7f6edd199700 1 --2- 172.21.15.32:0/529637458 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6eb8055320 0x7f6eb80577e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f6ec800d000 tx=0x7f6ec8007e00).stop 2020-12-05T17:39:24.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.635+0000 7f6edd199700 1 -- 172.21.15.32:0/529637458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ed80ae860 msgr2=0x7f6ed8077350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:24.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.635+0000 7f6edd199700 1 --2- 172.21.15.32:0/529637458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ed80ae860 0x7f6ed8077350 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7f6ecc00c000 tx=0x7f6ecc006dc0).stop 2020-12-05T17:39:24.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.635+0000 7f6edd199700 1 -- 172.21.15.32:0/529637458 shutdown_connections 2020-12-05T17:39:24.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.635+0000 7f6edd199700 1 --2- 172.21.15.32:0/529637458 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6ed80b0240 0x7f6ed8077840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:24.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.635+0000 7f6edd199700 1 --2- 172.21.15.32:0/529637458 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6eb8055320 0x7f6eb80577e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:24.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.635+0000 7f6edd199700 1 --2- 172.21.15.32:0/529637458 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ed80611b0 0x7f6ed8063fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:24.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.635+0000 7f6edd199700 1 --2- 172.21.15.32:0/529637458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ed80ae860 0x7f6ed8077350 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:24.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.635+0000 7f6edd199700 1 -- 172.21.15.32:0/529637458 >> 172.21.15.32:0/529637458 conn(0x7f6ed80b5f90 msgr2=0x7f6ed80b2940 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.639+0000 7f6edd199700 1 -- 172.21.15.32:0/529637458 shutdown_connections 2020-12-05T17:39:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.639+0000 7f6edd199700 1 -- 172.21.15.32:0/529637458 wait complete. 2020-12-05T17:39:24.647 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2020-12-05T17:39:24.656 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:39:24.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.775+0000 7f7c0df3d700 1 Processor -- start 2020-12-05T17:39:24.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.775+0000 7f7c0df3d700 1 -- start start 2020-12-05T17:39:24.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.775+0000 7f7c0df3d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7c080611b0 0x7f7c08153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:24.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.775+0000 7f7c0df3d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c08080a90 con 0x7f7c080b0240 2020-12-05T17:39:24.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.775+0000 7f7c0df3d700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c08152860 con 0x7f7c080ae860 2020-12-05T17:39:24.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.775+0000 7f7c0df3d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c080aecb0 con 0x7f7c080611b0 2020-12-05T17:39:24.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.775+0000 7f7c0cf3b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7c080b0240 0x7f7c0814c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40780/0 (socket says 172.21.15.32:40780) 2020-12-05T17:39:24.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.775+0000 7f7c0cf3b700 1 -- 172.21.15.32:0/1308957294 learned_addr learned my addr 172.21.15.32:0/1308957294 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:24.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.775+0000 7f7c0d73c700 1 --2- 172.21.15.32:0/1308957294 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7c080611b0 0x7f7c08153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:24.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.775+0000 7f7bfffff700 1 -- 172.21.15.32:0/1308957294 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 803301491 0 0) 0x7f7c08080a90 con 0x7f7c080b0240 2020-12-05T17:39:24.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.775+0000 7f7bfffff700 1 -- 172.21.15.32:0/1308957294 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bf4003230 con 0x7f7c080b0240 2020-12-05T17:39:24.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.779+0000 7f7bfffff700 1 -- 172.21.15.32:0/1308957294 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3741015451 0 0) 0x7f7c08152860 con 0x7f7c080ae860 2020-12-05T17:39:24.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.779+0000 7f7bfffff700 1 -- 172.21.15.32:0/1308957294 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c08080a90 con 0x7f7c080ae860 2020-12-05T17:39:24.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.779+0000 7f7c0d73c700 1 -- 172.21.15.32:0/1308957294 >> v1:172.21.15.61:6789/0 conn(0x7f7c080ae860 legacy=0x7f7c0814f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:24.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.779+0000 7f7c0d73c700 1 -- 172.21.15.32:0/1308957294 >> v1:172.21.15.32:6789/0 conn(0x7f7c080b0240 legacy=0x7f7c0814c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:24.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.779+0000 7f7bfffff700 1 -- 172.21.15.32:0/1308957294 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3186009380 0 0) 0x7f7bf4003230 con 0x7f7c080b0240 2020-12-05T17:39:24.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.779+0000 7f7c0d73c700 1 -- 172.21.15.32:0/1308957294 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c08152b00 con 0x7f7c080611b0 2020-12-05T17:39:24.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.779+0000 7f7c0d73c700 1 --2- 172.21.15.32:0/1308957294 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7c080611b0 0x7f7c08153240 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f7bf8001870 tx=0x7f7c040091a0).ready entity=mon.2 client_cookie=71748c766f92fa60 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:24.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.779+0000 7f7bfffff700 1 -- 172.21.15.32:0/1308957294 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7bf80016d0 con 0x7f7c080611b0 2020-12-05T17:39:24.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.779+0000 7f7bfffff700 1 -- 172.21.15.32:0/1308957294 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0400b800 con 0x7f7c080611b0 2020-12-05T17:39:24.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.779+0000 7f7bfffff700 1 -- 172.21.15.32:0/1308957294 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c0400b970 con 0x7f7c080611b0 2020-12-05T17:39:24.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.779+0000 7f7c0df3d700 1 -- 172.21.15.32:0/1308957294 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7c080611b0 msgr2=0x7f7c08153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:24.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.779+0000 7f7c0df3d700 1 --2- 172.21.15.32:0/1308957294 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7c080611b0 0x7f7c08153240 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f7bf8001870 tx=0x7f7c040091a0).stop 2020-12-05T17:39:24.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.779+0000 7f7c0df3d700 1 -- 172.21.15.32:0/1308957294 shutdown_connections 2020-12-05T17:39:24.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.779+0000 7f7c0df3d700 1 --2- 172.21.15.32:0/1308957294 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7c080611b0 0x7f7c08153240 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:24.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.779+0000 7f7c0df3d700 1 -- 172.21.15.32:0/1308957294 >> 172.21.15.32:0/1308957294 conn(0x7f7c080b5f90 msgr2=0x7f7c080b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:24.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.779+0000 7f7c0df3d700 1 -- 172.21.15.32:0/1308957294 shutdown_connections 2020-12-05T17:39:24.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.779+0000 7f7c0df3d700 1 -- 172.21.15.32:0/1308957294 wait complete. 2020-12-05T17:39:24.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.779+0000 7f7c0df3d700 1 Processor -- start 2020-12-05T17:39:24.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.779+0000 7f7c0df3d700 1 -- start start 2020-12-05T17:39:24.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7c0df3d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7c080611b0 0x7f7c080b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:24.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7c0df3d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c080ae860 0x7f7c080b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:24.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7c0df3d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7c080b0240 0x7f7c08159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:24.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7c0df3d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c080b15a0 con 0x7f7c080ae860 2020-12-05T17:39:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7c0df3d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c080b1840 con 0x7f7c080b0240 2020-12-05T17:39:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7c0df3d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c080b08d0 con 0x7f7c080611b0 2020-12-05T17:39:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7c0cf3b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c080ae860 0x7f7c080b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7c0cf3b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c080ae860 0x7f7c080b1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38138/0 (socket says 172.21.15.32:38138) 2020-12-05T17:39:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7c0cf3b700 1 -- 172.21.15.32:0/2875001514 learned_addr learned my addr 172.21.15.32:0/2875001514 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7c0d73c700 1 --2- 172.21.15.32:0/2875001514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7c080611b0 0x7f7c080b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7c0f19f700 1 --2- 172.21.15.32:0/2875001514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7c080b0240 0x7f7c08159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:24.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7c0cf3b700 1 -- 172.21.15.32:0/2875001514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7c080611b0 msgr2=0x7f7c080b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:24.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7c0cf3b700 1 --2- 172.21.15.32:0/2875001514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7c080611b0 0x7f7c080b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:24.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7c0cf3b700 1 -- 172.21.15.32:0/2875001514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7c080b0240 msgr2=0x7f7c08159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:24.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7c0cf3b700 1 --2- 172.21.15.32:0/2875001514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7c080b0240 0x7f7c08159440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:24.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7c0cf3b700 1 -- 172.21.15.32:0/2875001514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c080b0a90 con 0x7f7c080ae860 2020-12-05T17:39:24.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7c0cf3b700 1 --2- 172.21.15.32:0/2875001514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c080ae860 0x7f7c080b1ce0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7f7bf8000fa0 tx=0x7f7bf8001a70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:24.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7bfdffb700 1 -- 172.21.15.32:0/2875001514 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7bf8007e30 con 0x7f7c080ae860 2020-12-05T17:39:24.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7bfdffb700 1 -- 172.21.15.32:0/2875001514 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf8007170 con 0x7f7c080ae860 2020-12-05T17:39:24.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7c0df3d700 1 -- 172.21.15.32:0/2875001514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c080b0c40 con 0x7f7c080ae860 2020-12-05T17:39:24.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7bfdffb700 1 -- 172.21.15.32:0/2875001514 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7bf800a830 con 0x7f7c080ae860 2020-12-05T17:39:24.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7c0df3d700 1 -- 172.21.15.32:0/2875001514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c080b23f0 con 0x7f7c080ae860 2020-12-05T17:39:24.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7c0df3d700 1 -- 172.21.15.32:0/2875001514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c080b13e0 con 0x7f7c080ae860 2020-12-05T17:39:24.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.783+0000 7f7bfdffb700 1 -- 172.21.15.32:0/2875001514 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7bf8010070 con 0x7f7c080ae860 2020-12-05T17:39:24.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.787+0000 7f7bfdffb700 1 --2- 172.21.15.32:0/2875001514 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7be8055320 0x7f7be80577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:24.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.787+0000 7f7bfdffb700 1 -- 172.21.15.32:0/2875001514 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7bf806dc80 con 0x7f7c080ae860 2020-12-05T17:39:24.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.787+0000 7f7c0d73c700 1 --2- 172.21.15.32:0/2875001514 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7be8055320 0x7f7be80577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:24.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.787+0000 7f7c0d73c700 1 --2- 172.21.15.32:0/2875001514 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7be8055320 0x7f7be80577e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f7c04019000 tx=0x7f7c04016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:24.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.791+0000 7f7bfdffb700 1 -- 172.21.15.32:0/2875001514 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7bf8043d80 con 0x7f7c080ae860 2020-12-05T17:39:24.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.987+0000 7f7c0df3d700 1 -- 172.21.15.32:0/2875001514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f7c08057050 con 0x7f7c080ae860 2020-12-05T17:39:24.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.991+0000 7f7bfdffb700 1 -- 172.21.15.32:0/2875001514 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v31) v1 ==== 184+0+0 (secure 0 0 0) 0x7f7bf80473a0 con 0x7f7c080ae860 2020-12-05T17:39:25.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.995+0000 7f7c0df3d700 1 -- 172.21.15.32:0/2875001514 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7be8055320 msgr2=0x7f7be80577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:25.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.995+0000 7f7c0df3d700 1 --2- 172.21.15.32:0/2875001514 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7be8055320 0x7f7be80577e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f7c04019000 tx=0x7f7c04016000).stop 2020-12-05T17:39:25.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.995+0000 7f7c0df3d700 1 -- 172.21.15.32:0/2875001514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c080ae860 msgr2=0x7f7c080b1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:25.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.995+0000 7f7c0df3d700 1 --2- 172.21.15.32:0/2875001514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c080ae860 0x7f7c080b1ce0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7f7bf8000fa0 tx=0x7f7bf8001a70).stop 2020-12-05T17:39:25.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.995+0000 7f7c0df3d700 1 -- 172.21.15.32:0/2875001514 shutdown_connections 2020-12-05T17:39:25.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.995+0000 7f7c0df3d700 1 --2- 172.21.15.32:0/2875001514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7c080b0240 0x7f7c08159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:25.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.995+0000 7f7c0df3d700 1 --2- 172.21.15.32:0/2875001514 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7be8055320 0x7f7be80577e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:25.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.995+0000 7f7c0df3d700 1 --2- 172.21.15.32:0/2875001514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7c080611b0 0x7f7c080b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:25.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.995+0000 7f7c0df3d700 1 --2- 172.21.15.32:0/2875001514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c080ae860 0x7f7c080b1ce0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:25.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.995+0000 7f7c0df3d700 1 -- 172.21.15.32:0/2875001514 >> 172.21.15.32:0/2875001514 conn(0x7f7c080b5f90 msgr2=0x7f7c081520d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:25.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.995+0000 7f7c0df3d700 1 -- 172.21.15.32:0/2875001514 shutdown_connections 2020-12-05T17:39:25.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:24.995+0000 7f7c0df3d700 1 -- 172.21.15.32:0/2875001514 wait complete. 2020-12-05T17:39:25.004 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.2 2020-12-05T17:39:25.016 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:39:25.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.123+0000 7f348eaa0700 1 Processor -- start 2020-12-05T17:39:25.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.123+0000 7f348eaa0700 1 -- start start 2020-12-05T17:39:25.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348eaa0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34880b1fc0 0x7f34880af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348eaa0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3488146a90 con 0x7f3488147400 2020-12-05T17:39:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348eaa0700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34880b23a0 con 0x7f34880ab710 2020-12-05T17:39:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348eaa0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3488061e90 con 0x7f34880b1fc0 2020-12-05T17:39:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348e29f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3488147400 0x7f3488145d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40794/0 (socket says 172.21.15.32:40794) 2020-12-05T17:39:25.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348e29f700 1 -- 172.21.15.32:0/1371546056 learned_addr learned my addr 172.21.15.32:0/1371546056 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:25.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348da9e700 1 --2- 172.21.15.32:0/1371546056 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34880b1fc0 0x7f34880af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:25.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348ca9c700 1 -- 172.21.15.32:0/1371546056 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2918271550 0 0) 0x7f3488146a90 con 0x7f3488147400 2020-12-05T17:39:25.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348ca9c700 1 -- 172.21.15.32:0/1371546056 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3470003230 con 0x7f3488147400 2020-12-05T17:39:25.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348ca9c700 1 -- 172.21.15.32:0/1371546056 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 714137070 0 0) 0x7f34880b23a0 con 0x7f34880ab710 2020-12-05T17:39:25.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348ca9c700 1 -- 172.21.15.32:0/1371546056 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3488146a90 con 0x7f34880ab710 2020-12-05T17:39:25.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348ca9c700 1 -- 172.21.15.32:0/1371546056 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2699702501 0 0) 0x7f3470003230 con 0x7f3488147400 2020-12-05T17:39:25.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348ca9c700 1 -- 172.21.15.32:0/1371546056 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34880b1fc0 msgr2=0x7f34880af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:25.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348ca9c700 1 --2- 172.21.15.32:0/1371546056 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34880b1fc0 0x7f34880af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:25.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348ca9c700 1 -- 172.21.15.32:0/1371546056 >> v1:172.21.15.61:6789/0 conn(0x7f34880ab710 legacy=0x7f3488061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:25.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348ca9c700 1 -- 172.21.15.32:0/1371546056 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3488146cc0 con 0x7f3488147400 2020-12-05T17:39:25.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348ca9c700 1 -- 172.21.15.32:0/1371546056 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f3480001a50 con 0x7f3488147400 2020-12-05T17:39:25.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348ca9c700 1 -- 172.21.15.32:0/1371546056 >> v1:172.21.15.32:6789/0 conn(0x7f3488147400 legacy=0x7f3488145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:25.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348ca9c700 1 --2- 172.21.15.32:0/1371546056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34700038e0 0x7f3470003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:25.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348ca9c700 1 --2- 172.21.15.32:0/1371546056 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34880b1fc0 0x7f34700044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348ca9c700 1 --2- 172.21.15.32:0/1371546056 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34700049c0 0x7f3470004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348ca9c700 1 -- 172.21.15.32:0/1371546056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3488061e90 con 0x7f34700038e0 2020-12-05T17:39:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348da9e700 1 --2- 172.21.15.32:0/1371546056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34700038e0 0x7f3470003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348ca9c700 1 -- 172.21.15.32:0/1371546056 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3470005360 con 0x7f34700049c0 2020-12-05T17:39:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348ca9c700 1 -- 172.21.15.32:0/1371546056 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f347000b770 con 0x7f34880b1fc0 2020-12-05T17:39:25.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348e29f700 1 --2- 172.21.15.32:0/1371546056 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34700049c0 0x7f3470004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:25.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348d29d700 1 --2- 172.21.15.32:0/1371546056 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34880b1fc0 0x7f34700044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:25.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348da9e700 1 -- 172.21.15.32:0/1371546056 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34880b1fc0 msgr2=0x7f34700044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:25.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348da9e700 1 --2- 172.21.15.32:0/1371546056 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34880b1fc0 0x7f34700044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:25.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348da9e700 1 -- 172.21.15.32:0/1371546056 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34700049c0 msgr2=0x7f3470004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:25.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348da9e700 1 --2- 172.21.15.32:0/1371546056 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34700049c0 0x7f3470004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:25.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348da9e700 1 -- 172.21.15.32:0/1371546056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3470003230 con 0x7f34700038e0 2020-12-05T17:39:25.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348e29f700 1 --2- 172.21.15.32:0/1371546056 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34700049c0 0x7f3470004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:39:25.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348da9e700 1 --2- 172.21.15.32:0/1371546056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34700038e0 0x7f3470003ce0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7f34700054a0 tx=0x7f3478008040).ready entity=mon.0 client_cookie=bf6416d30a9fa526 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:25.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348ca9c700 1 -- 172.21.15.32:0/1371546056 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f347800e9d0 con 0x7f34700038e0 2020-12-05T17:39:25.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348ca9c700 1 -- 172.21.15.32:0/1371546056 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347800eb30 con 0x7f34700038e0 2020-12-05T17:39:25.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348ca9c700 1 -- 172.21.15.32:0/1371546056 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f347800ee90 con 0x7f34700038e0 2020-12-05T17:39:25.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348eaa0700 1 -- 172.21.15.32:0/1371546056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34700038e0 msgr2=0x7f3470003ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:25.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348eaa0700 1 --2- 172.21.15.32:0/1371546056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34700038e0 0x7f3470003ce0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7f34700054a0 tx=0x7f3478008040).stop 2020-12-05T17:39:25.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348e29f700 1 -- 172.21.15.32:0/1371546056 reap_dead start 2020-12-05T17:39:25.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348eaa0700 1 -- 172.21.15.32:0/1371546056 shutdown_connections 2020-12-05T17:39:25.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348eaa0700 1 -- 172.21.15.32:0/1371546056 >> 172.21.15.32:0/1371546056 conn(0x7f34880b5530 msgr2=0x7f34880b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:25.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348eaa0700 1 -- 172.21.15.32:0/1371546056 shutdown_connections 2020-12-05T17:39:25.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.127+0000 7f348eaa0700 1 -- 172.21.15.32:0/1371546056 wait complete. 2020-12-05T17:39:25.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f348eaa0700 1 Processor -- start 2020-12-05T17:39:25.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f348eaa0700 1 -- start start 2020-12-05T17:39:25.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f348eaa0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34880adf00 0x7f34880ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:25.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f348eaa0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34880ac950 0x7f3488066790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:25.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f348eaa0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34880643d0 0x7f3488064860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:25.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f348eaa0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3488143b50 con 0x7f34880643d0 2020-12-05T17:39:25.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f348eaa0700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34880ae2e0 con 0x7f34880ac950 2020-12-05T17:39:25.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f348eaa0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3488065250 con 0x7f34880adf00 2020-12-05T17:39:25.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f348e29f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34880643d0 0x7f3488064860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:25.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f348e29f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34880643d0 0x7f3488064860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38160/0 (socket says 172.21.15.32:38160) 2020-12-05T17:39:25.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f348da9e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34880adf00 0x7f34880ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:25.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f348e29f700 1 -- 172.21.15.32:0/3100475067 learned_addr learned my addr 172.21.15.32:0/3100475067 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:25.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f348d29d700 1 --2- 172.21.15.32:0/3100475067 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34880ac950 0x7f3488066790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:25.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f348e29f700 1 -- 172.21.15.32:0/3100475067 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34880adf00 msgr2=0x7f34880ad1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:25.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f348e29f700 1 --2- 172.21.15.32:0/3100475067 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34880adf00 0x7f34880ad1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:25.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f348e29f700 1 -- 172.21.15.32:0/3100475067 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34880ac950 msgr2=0x7f3488066790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:25.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f348e29f700 1 --2- 172.21.15.32:0/3100475067 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34880ac950 0x7f3488066790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:25.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f348e29f700 1 -- 172.21.15.32:0/3100475067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3488064d50 con 0x7f34880643d0 2020-12-05T17:39:25.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f348d29d700 1 --2- 172.21.15.32:0/3100475067 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34880ac950 0x7f3488066790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:25.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f348e29f700 1 --2- 172.21.15.32:0/3100475067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34880643d0 0x7f3488064860 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7f3480001c30 tx=0x7f3480009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:25.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f3476ffd700 1 -- 172.21.15.32:0/3100475067 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3480004b40 con 0x7f34880643d0 2020-12-05T17:39:25.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f3476ffd700 1 -- 172.21.15.32:0/3100475067 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3480004ca0 con 0x7f34880643d0 2020-12-05T17:39:25.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f3476ffd700 1 -- 172.21.15.32:0/3100475067 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3480007030 con 0x7f34880643d0 2020-12-05T17:39:25.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f348eaa0700 1 -- 172.21.15.32:0/3100475067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34880ad6e0 con 0x7f34880643d0 2020-12-05T17:39:25.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.131+0000 7f348eaa0700 1 -- 172.21.15.32:0/3100475067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3488065f50 con 0x7f34880643d0 2020-12-05T17:39:25.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.135+0000 7f348eaa0700 1 -- 172.21.15.32:0/3100475067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34880ac6f0 con 0x7f34880643d0 2020-12-05T17:39:25.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.135+0000 7f3476ffd700 1 -- 172.21.15.32:0/3100475067 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f348001e070 con 0x7f34880643d0 2020-12-05T17:39:25.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.135+0000 7f3476ffd700 1 --2- 172.21.15.32:0/3100475067 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3468055330 0x7f34680577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:25.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.135+0000 7f3476ffd700 1 -- 172.21.15.32:0/3100475067 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f348006d360 con 0x7f34880643d0 2020-12-05T17:39:25.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.135+0000 7f348da9e700 1 --2- 172.21.15.32:0/3100475067 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3468055330 0x7f34680577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:25.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.135+0000 7f348da9e700 1 --2- 172.21.15.32:0/3100475067 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3468055330 0x7f34680577f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f3478007000 tx=0x7f347801a000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:25.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.143+0000 7f3476ffd700 1 -- 172.21.15.32:0/3100475067 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3480043500 con 0x7f34880643d0 2020-12-05T17:39:25.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.331+0000 7f348eaa0700 1 -- 172.21.15.32:0/3100475067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3488057050 con 0x7f34880643d0 2020-12-05T17:39:25.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.331+0000 7f3476ffd700 1 -- 172.21.15.32:0/3100475067 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f3480046b20 con 0x7f34880643d0 2020-12-05T17:39:25.337 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:25.337 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":42,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6823,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2664846066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2664846066},{"type":"v1","addr":"172.21.15.61:6833","nonce":2664846066}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:39:22.261725+0000","modified":"2020-12-05T17:39:23.265348+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6930":{"gid":6930,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4087622496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4087622496},{"type":"v1","addr":"172.21.15.32:6835","nonce":4087622496}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2020-12-05T17:39:25.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.331+0000 7f348eaa0700 1 -- 172.21.15.32:0/3100475067 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3468055330 msgr2=0x7f34680577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:25.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.331+0000 7f348eaa0700 1 --2- 172.21.15.32:0/3100475067 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3468055330 0x7f34680577f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f3478007000 tx=0x7f347801a000).stop 2020-12-05T17:39:25.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.331+0000 7f348eaa0700 1 -- 172.21.15.32:0/3100475067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34880643d0 msgr2=0x7f3488064860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:25.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.331+0000 7f348eaa0700 1 --2- 172.21.15.32:0/3100475067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34880643d0 0x7f3488064860 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7f3480001c30 tx=0x7f3480009040).stop 2020-12-05T17:39:25.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.331+0000 7f348eaa0700 1 -- 172.21.15.32:0/3100475067 shutdown_connections 2020-12-05T17:39:25.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.331+0000 7f348eaa0700 1 --2- 172.21.15.32:0/3100475067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34880643d0 0x7f3488064860 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:25.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.331+0000 7f348eaa0700 1 --2- 172.21.15.32:0/3100475067 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3468055330 0x7f34680577f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:25.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.331+0000 7f348eaa0700 1 --2- 172.21.15.32:0/3100475067 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34880adf00 0x7f34880ad1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:25.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.331+0000 7f348eaa0700 1 --2- 172.21.15.32:0/3100475067 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34880ac950 0x7f3488066790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:25.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.331+0000 7f348eaa0700 1 -- 172.21.15.32:0/3100475067 >> 172.21.15.32:0/3100475067 conn(0x7f34880b5530 msgr2=0x7f34880b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:25.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.331+0000 7f348eaa0700 1 -- 172.21.15.32:0/3100475067 shutdown_connections 2020-12-05T17:39:25.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.331+0000 7f348eaa0700 1 -- 172.21.15.32:0/3100475067 wait complete. 2020-12-05T17:39:25.341 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 42 2020-12-05T17:39:25.552 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:39:22.261725+0000', 'modified': '2020-12-05T17:39:23.265348+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6930}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6930': {'gid': 6930, 'name': 'a', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/4087622496', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 4087622496}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 4087622496}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:39:25.553 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:39:25.553 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:39:25.686 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":6930,"want_state":"up:active","state":"up:active","rank_uptime":3.4099855419999998,"mdsmap_epoch":42,"osdmap_epoch":92,"osdmap_epoch_barrier":92,"uptime":8.6135238849999993} 2020-12-05T17:39:25.687 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 6930, "mdsmap_epoch": 42, "osdmap_epoch": 92, "osdmap_epoch_barrier": 92, "rank_uptime": 3.409985542, "state": "up:active", "uptime": 8.613523885, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:39:25.688 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:39:25.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.799+0000 7f3fc4dfc700 1 Processor -- start 2020-12-05T17:39:25.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.799+0000 7f3fc4dfc700 1 -- start start 2020-12-05T17:39:25.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.799+0000 7f3fc4dfc700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f3fc0062400 0x7f3fc005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:25.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.799+0000 7f3fc4dfc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc0142450 con 0x7f3fc005f3b0 2020-12-05T17:39:25.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.799+0000 7f3fc4dfc700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc0061780 con 0x7f3fc005feb0 2020-12-05T17:39:25.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.799+0000 7f3fc4dfc700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fc0061a20 con 0x7f3fc0062400 2020-12-05T17:39:25.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.799+0000 7f3fbf7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3fc005f3b0 0x7f3fc00b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40812/0 (socket says 172.21.15.32:40812) 2020-12-05T17:39:25.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.799+0000 7f3fbf7fe700 1 -- 172.21.15.32:0/2765690388 learned_addr learned my addr 172.21.15.32:0/2765690388 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:25.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.799+0000 7f3fbeffd700 1 --2- 172.21.15.32:0/2765690388 >> v2:172.21.15.61:3300/0 conn(0x7f3fc0062400 0x7f3fc005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:25.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.799+0000 7f3fbe7fc700 1 -- 172.21.15.32:0/2765690388 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1227899875 0 0) 0x7f3fc0142450 con 0x7f3fc005f3b0 2020-12-05T17:39:25.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.799+0000 7f3fbe7fc700 1 -- 172.21.15.32:0/2765690388 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fac003230 con 0x7f3fc005f3b0 2020-12-05T17:39:25.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.799+0000 7f3fbe7fc700 1 -- 172.21.15.32:0/2765690388 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302135957 0 0) 0x7f3fc0061780 con 0x7f3fc005feb0 2020-12-05T17:39:25.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.799+0000 7f3fbe7fc700 1 -- 172.21.15.32:0/2765690388 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fc0142450 con 0x7f3fc005feb0 2020-12-05T17:39:25.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.799+0000 7f3fbeffd700 1 -- 172.21.15.32:0/2765690388 >> v1:172.21.15.61:6789/0 conn(0x7f3fc005feb0 legacy=0x7f3fc00b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:25.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.799+0000 7f3fbe7fc700 1 -- 172.21.15.32:0/2765690388 <== mon.1 v1:172.21.15.61:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1326140462 0 0) 0x7f3fc0142450 con 0x7f3fc005feb0 2020-12-05T17:39:25.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.799+0000 7f3fbeffd700 1 -- 172.21.15.32:0/2765690388 >> v1:172.21.15.32:6789/0 conn(0x7f3fc005f3b0 legacy=0x7f3fc00b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:25.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.799+0000 7f3fbeffd700 1 -- 172.21.15.32:0/2765690388 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc0061cc0 con 0x7f3fc0062400 2020-12-05T17:39:25.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.799+0000 7f3fbeffd700 1 --2- 172.21.15.32:0/2765690388 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3fc0062400 0x7f3fc005f790 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f3fb00010d0 tx=0x7f3fb4006dc0).ready entity=mon.1 client_cookie=40cd9f3d73ed87b3 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:25.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.799+0000 7f3fbe7fc700 1 -- 172.21.15.32:0/2765690388 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fb0001650 con 0x7f3fc0062400 2020-12-05T17:39:25.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.799+0000 7f3fbe7fc700 1 -- 172.21.15.32:0/2765690388 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb400d5c0 con 0x7f3fc0062400 2020-12-05T17:39:25.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.799+0000 7f3fbe7fc700 1 -- 172.21.15.32:0/2765690388 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fb8001a50 con 0x7f3fc0062400 2020-12-05T17:39:25.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.799+0000 7f3fc4dfc700 1 -- 172.21.15.32:0/2765690388 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3fc0062400 msgr2=0x7f3fc005f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:25.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.799+0000 7f3fc4dfc700 1 --2- 172.21.15.32:0/2765690388 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3fc0062400 0x7f3fc005f790 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f3fb00010d0 tx=0x7f3fb4006dc0).stop 2020-12-05T17:39:25.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.803+0000 7f3fc4dfc700 1 -- 172.21.15.32:0/2765690388 shutdown_connections 2020-12-05T17:39:25.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.803+0000 7f3fc4dfc700 1 --2- 172.21.15.32:0/2765690388 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3fc0062400 0x7f3fc005f790 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:25.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.803+0000 7f3fc4dfc700 1 -- 172.21.15.32:0/2765690388 >> 172.21.15.32:0/2765690388 conn(0x7f3fc00b8ee0 msgr2=0x7f3fc00b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:25.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.803+0000 7f3fc4dfc700 1 -- 172.21.15.32:0/2765690388 shutdown_connections 2020-12-05T17:39:25.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.803+0000 7f3fc4dfc700 1 -- 172.21.15.32:0/2765690388 wait complete. 2020-12-05T17:39:25.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.803+0000 7f3fc4dfc700 1 Processor -- start 2020-12-05T17:39:25.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.803+0000 7f3fc4dfc700 1 -- start start 2020-12-05T17:39:25.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.803+0000 7f3fc4dfc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3fc005f3b0 0x7f3fc0065e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.803+0000 7f3fc4dfc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3fc005feb0 0x7f3fc0154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.803+0000 7f3fc4dfc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fc0062400 0x7f3fc01550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.803+0000 7f3fc4dfc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc0066310 con 0x7f3fc0062400 2020-12-05T17:39:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.803+0000 7f3fc4dfc700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc00642f0 con 0x7f3fc005feb0 2020-12-05T17:39:25.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.803+0000 7f3fc4dfc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc0064590 con 0x7f3fc005f3b0 2020-12-05T17:39:25.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.803+0000 7f3fbffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fc0062400 0x7f3fc01550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:25.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.803+0000 7f3fbffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fc0062400 0x7f3fc01550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38174/0 (socket says 172.21.15.32:38174) 2020-12-05T17:39:25.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.803+0000 7f3fbffff700 1 -- 172.21.15.32:0/3064908431 learned_addr learned my addr 172.21.15.32:0/3064908431 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:25.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.803+0000 7f3fbeffd700 1 --2- 172.21.15.32:0/3064908431 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3fc005feb0 0x7f3fc0154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:25.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.803+0000 7f3fbf7fe700 1 --2- 172.21.15.32:0/3064908431 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3fc005f3b0 0x7f3fc0065e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:25.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.807+0000 7f3fbffff700 1 -- 172.21.15.32:0/3064908431 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3fc005f3b0 msgr2=0x7f3fc0065e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:25.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.807+0000 7f3fbffff700 1 --2- 172.21.15.32:0/3064908431 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3fc005f3b0 0x7f3fc0065e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:25.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.807+0000 7f3fbffff700 1 -- 172.21.15.32:0/3064908431 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3fc005feb0 msgr2=0x7f3fc0154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:25.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.807+0000 7f3fbffff700 1 --2- 172.21.15.32:0/3064908431 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3fc005feb0 0x7f3fc0154bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:25.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.807+0000 7f3fbffff700 1 -- 172.21.15.32:0/3064908431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc0064830 con 0x7f3fc0062400 2020-12-05T17:39:25.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.807+0000 7f3fbffff700 1 --2- 172.21.15.32:0/3064908431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fc0062400 0x7f3fc01550b0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f3fb8001c30 tx=0x7f3fb800a570).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:25.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.807+0000 7f3fbcff9700 1 -- 172.21.15.32:0/3064908431 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fb800f070 con 0x7f3fc0062400 2020-12-05T17:39:25.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.807+0000 7f3fbcff9700 1 -- 172.21.15.32:0/3064908431 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb800cd40 con 0x7f3fc0062400 2020-12-05T17:39:25.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.807+0000 7f3fc4dfc700 1 -- 172.21.15.32:0/3064908431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc0063340 con 0x7f3fc0062400 2020-12-05T17:39:25.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.807+0000 7f3fbcff9700 1 -- 172.21.15.32:0/3064908431 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fb8014610 con 0x7f3fc0062400 2020-12-05T17:39:25.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.807+0000 7f3fc4dfc700 1 -- 172.21.15.32:0/3064908431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc0144f20 con 0x7f3fc0062400 2020-12-05T17:39:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.807+0000 7f3fc4dfc700 1 -- 172.21.15.32:0/3064908431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc0057050 con 0x7f3fc0062400 2020-12-05T17:39:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.807+0000 7f3fbcff9700 1 -- 172.21.15.32:0/3064908431 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3fb801b060 con 0x7f3fc0062400 2020-12-05T17:39:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.807+0000 7f3fbcff9700 1 --2- 172.21.15.32:0/3064908431 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3fa00553b0 0x7f3fa0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:25.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.807+0000 7f3fbcff9700 1 -- 172.21.15.32:0/3064908431 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3fb806e830 con 0x7f3fc0062400 2020-12-05T17:39:25.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.807+0000 7f3fbf7fe700 1 --2- 172.21.15.32:0/3064908431 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3fa00553b0 0x7f3fa0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:25.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.811+0000 7f3fbf7fe700 1 --2- 172.21.15.32:0/3064908431 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3fa00553b0 0x7f3fa0057870 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f3fb000d000 tx=0x7f3fb0001800).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:25.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.815+0000 7f3fbcff9700 1 -- 172.21.15.32:0/3064908431 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3fb80448c0 con 0x7f3fc0062400 2020-12-05T17:39:25.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.955+0000 7f3fc4dfc700 1 -- 172.21.15.32:0/3064908431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3fc005aab0 con 0x7f3fc0062400 2020-12-05T17:39:25.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.955+0000 7f3fbcff9700 1 -- 172.21.15.32:0/3064908431 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f3fb8019070 con 0x7f3fc0062400 2020-12-05T17:39:25.963 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:25.964 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":92,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:39:23.264356+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:39:19.993416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2020-12-05T17:39:21.252765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:39:25.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.959+0000 7f3fc4dfc700 1 -- 172.21.15.32:0/3064908431 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3fa00553b0 msgr2=0x7f3fa0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:25.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.959+0000 7f3fc4dfc700 1 --2- 172.21.15.32:0/3064908431 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3fa00553b0 0x7f3fa0057870 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f3fb000d000 tx=0x7f3fb0001800).stop 2020-12-05T17:39:25.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.959+0000 7f3fc4dfc700 1 -- 172.21.15.32:0/3064908431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fc0062400 msgr2=0x7f3fc01550b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:25.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.959+0000 7f3fc4dfc700 1 --2- 172.21.15.32:0/3064908431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fc0062400 0x7f3fc01550b0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f3fb8001c30 tx=0x7f3fb800a570).stop 2020-12-05T17:39:25.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.959+0000 7f3fc4dfc700 1 -- 172.21.15.32:0/3064908431 shutdown_connections 2020-12-05T17:39:25.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.959+0000 7f3fc4dfc700 1 --2- 172.21.15.32:0/3064908431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fc0062400 0x7f3fc01550b0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:25.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.959+0000 7f3fc4dfc700 1 --2- 172.21.15.32:0/3064908431 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3fa00553b0 0x7f3fa0057870 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:25.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.959+0000 7f3fc4dfc700 1 --2- 172.21.15.32:0/3064908431 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3fc005f3b0 0x7f3fc0065e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:25.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.959+0000 7f3fc4dfc700 1 --2- 172.21.15.32:0/3064908431 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3fc005feb0 0x7f3fc0154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:25.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.959+0000 7f3fc4dfc700 1 -- 172.21.15.32:0/3064908431 >> 172.21.15.32:0/3064908431 conn(0x7f3fc00b8ee0 msgr2=0x7f3fc013fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:25.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.959+0000 7f3fc4dfc700 1 -- 172.21.15.32:0/3064908431 shutdown_connections 2020-12-05T17:39:25.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:25.959+0000 7f3fc4dfc700 1 -- 172.21.15.32:0/3064908431 wait complete. 2020-12-05T17:39:25.978 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:39:19.993416+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:39:21.252765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:39:25.979 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:39:26.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.099+0000 7f33b9143700 1 Processor -- start 2020-12-05T17:39:26.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.099+0000 7f33b9143700 1 -- start start 2020-12-05T17:39:26.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.099+0000 7f33b9143700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f33b40b5b20 0x7f33b40b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:26.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.099+0000 7f33b9143700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f33b40b5240 0x7f33b4145f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:26.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.099+0000 7f33b9143700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b40b1920 con 0x7f33b40b6690 2020-12-05T17:39:26.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.099+0000 7f33b9143700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b40b1bc0 con 0x7f33b40b5b20 2020-12-05T17:39:26.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.099+0000 7f33b9143700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b40b5680 con 0x7f33b40b5240 2020-12-05T17:39:26.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.099+0000 7f33b2d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f33b40b5b20 0x7f33b40b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:26.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.099+0000 7f33b2d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f33b40b5b20 0x7f33b40b5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38178/0 (socket says 172.21.15.32:38178) 2020-12-05T17:39:26.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.099+0000 7f33b2d9d700 1 -- 172.21.15.32:0/1798397597 learned_addr learned my addr 172.21.15.32:0/1798397597 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:26.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.099+0000 7f33b359e700 1 --2- 172.21.15.32:0/1798397597 >> v2:172.21.15.61:3300/0 conn(0x7f33b40b5240 0x7f33b4145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:26.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.099+0000 7f33b2d9d700 1 -- 172.21.15.32:0/1798397597 >> v2:172.21.15.61:3300/0 conn(0x7f33b40b5240 msgr2=0x7f33b4145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:26.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.099+0000 7f33b2d9d700 1 --2- 172.21.15.32:0/1798397597 >> v2:172.21.15.61:3300/0 conn(0x7f33b40b5240 0x7f33b4145f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.099+0000 7f33b2d9d700 1 -- 172.21.15.32:0/1798397597 >> v1:172.21.15.61:6789/0 conn(0x7f33b40b6690 legacy=0x7f33b40b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:26.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.099+0000 7f33b2d9d700 1 -- 172.21.15.32:0/1798397597 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b4146400 con 0x7f33b40b5b20 2020-12-05T17:39:26.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.099+0000 7f33b359e700 1 --2- 172.21.15.32:0/1798397597 >> v2:172.21.15.61:3300/0 conn(0x7f33b40b5240 0x7f33b4145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:39:26.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.099+0000 7f33b2d9d700 1 --2- 172.21.15.32:0/1798397597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b40b5b20 0x7f33b40b5f00 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7f33a4002f30 tx=0x7f33a4006e80).ready entity=mon.0 client_cookie=4b6acb06f2c498fb server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:26.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.099+0000 7f33b259c700 1 -- 172.21.15.32:0/1798397597 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33a40097f0 con 0x7f33b40b5b20 2020-12-05T17:39:26.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.099+0000 7f33b259c700 1 -- 172.21.15.32:0/1798397597 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a4009950 con 0x7f33b40b5b20 2020-12-05T17:39:26.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.099+0000 7f33b9143700 1 -- 172.21.15.32:0/1798397597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b40b5b20 msgr2=0x7f33b40b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:26.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.099+0000 7f33b259c700 1 -- 172.21.15.32:0/1798397597 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33a4009cb0 con 0x7f33b40b5b20 2020-12-05T17:39:26.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.099+0000 7f33b9143700 1 --2- 172.21.15.32:0/1798397597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b40b5b20 0x7f33b40b5f00 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7f33a4002f30 tx=0x7f33a4006e80).stop 2020-12-05T17:39:26.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.103+0000 7f33b9143700 1 -- 172.21.15.32:0/1798397597 shutdown_connections 2020-12-05T17:39:26.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.103+0000 7f33b9143700 1 --2- 172.21.15.32:0/1798397597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b40b5b20 0x7f33b40b5f00 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.103+0000 7f33b9143700 1 --2- 172.21.15.32:0/1798397597 >> v2:172.21.15.61:3300/0 conn(0x7f33b40b5240 0x7f33b4145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.103+0000 7f33b9143700 1 -- 172.21.15.32:0/1798397597 >> 172.21.15.32:0/1798397597 conn(0x7f33b405f3b0 msgr2=0x7f33b405f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:26.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.103+0000 7f33b9143700 1 -- 172.21.15.32:0/1798397597 shutdown_connections 2020-12-05T17:39:26.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.103+0000 7f33b9143700 1 -- 172.21.15.32:0/1798397597 wait complete. 2020-12-05T17:39:26.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.103+0000 7f33b9143700 1 Processor -- start 2020-12-05T17:39:26.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.103+0000 7f33b9143700 1 -- start start 2020-12-05T17:39:26.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.103+0000 7f33b9143700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b40b5240 0x7f33b40b95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:26.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.103+0000 7f33b9143700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f33b40b5b20 0x7f33b4063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:26.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.103+0000 7f33b9143700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f33b40b6690 0x7f33b40a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:26.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33b9143700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b4063530 con 0x7f33b40b5240 2020-12-05T17:39:26.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33b9143700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b40a37f0 con 0x7f33b40b6690 2020-12-05T17:39:26.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33b9143700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b4064f40 con 0x7f33b40b5b20 2020-12-05T17:39:26.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33b2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b40b5240 0x7f33b40b95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:26.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33b2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b40b5240 0x7f33b40b95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38188/0 (socket says 172.21.15.32:38188) 2020-12-05T17:39:26.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33b2d9d700 1 -- 172.21.15.32:0/699586193 learned_addr learned my addr 172.21.15.32:0/699586193 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:26.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33aa59c700 1 --2- 172.21.15.32:0/699586193 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f33b40b5b20 0x7f33b4063040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:26.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33b359e700 1 --2- 172.21.15.32:0/699586193 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f33b40b6690 0x7f33b40a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:26.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33b2d9d700 1 -- 172.21.15.32:0/699586193 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f33b40b5b20 msgr2=0x7f33b4063040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:26.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33b2d9d700 1 --2- 172.21.15.32:0/699586193 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f33b40b5b20 0x7f33b4063040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33b2d9d700 1 -- 172.21.15.32:0/699586193 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f33b40b6690 msgr2=0x7f33b40a3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:26.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33b2d9d700 1 --2- 172.21.15.32:0/699586193 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f33b40b6690 0x7f33b40a3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33b2d9d700 1 -- 172.21.15.32:0/699586193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b40651e0 con 0x7f33b40b5240 2020-12-05T17:39:26.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33b2d9d700 1 --2- 172.21.15.32:0/699586193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b40b5240 0x7f33b40b95c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7f33a4002f30 tx=0x7f33a4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:26.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33b0d99700 1 -- 172.21.15.32:0/699586193 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33a4011e50 con 0x7f33b40b5240 2020-12-05T17:39:26.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33b0d99700 1 -- 172.21.15.32:0/699586193 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a401e460 con 0x7f33b40b5240 2020-12-05T17:39:26.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33b9143700 1 -- 172.21.15.32:0/699586193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b4064bc0 con 0x7f33b40b5240 2020-12-05T17:39:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33b0d99700 1 -- 172.21.15.32:0/699586193 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33a401e7c0 con 0x7f33b40b5240 2020-12-05T17:39:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33b9143700 1 -- 172.21.15.32:0/699586193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b4065b60 con 0x7f33b40b5240 2020-12-05T17:39:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33b9143700 1 -- 172.21.15.32:0/699586193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b4057050 con 0x7f33b40b5240 2020-12-05T17:39:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33b0d99700 1 -- 172.21.15.32:0/699586193 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f33a4025030 con 0x7f33b40b5240 2020-12-05T17:39:26.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33b0d99700 1 --2- 172.21.15.32:0/699586193 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f33980554e0 0x7f33980579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33b0d99700 1 -- 172.21.15.32:0/699586193 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f33a406f780 con 0x7f33b40b5240 2020-12-05T17:39:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.107+0000 7f33aa59c700 1 --2- 172.21.15.32:0/699586193 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f33980554e0 0x7f33980579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.111+0000 7f33aa59c700 1 --2- 172.21.15.32:0/699586193 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f33980554e0 0x7f33980579a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f339c00d000 tx=0x7f339c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.115+0000 7f33b0d99700 1 -- 172.21.15.32:0/699586193 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f33a4045880 con 0x7f33b40b5240 2020-12-05T17:39:26.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.259+0000 7f33b9143700 1 -- 172.21.15.32:0/699586193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f33b405bae0 con 0x7f33b40b5240 2020-12-05T17:39:26.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.259+0000 7f33b0d99700 1 -- 172.21.15.32:0/699586193 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v92) v1 ==== 110+0+10 (secure 0 0 0) 0x7f33a40230b0 con 0x7f33b40b5240 2020-12-05T17:39:26.266 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:39:26.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.263+0000 7f33b9143700 1 -- 172.21.15.32:0/699586193 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f33980554e0 msgr2=0x7f33980579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:26.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.263+0000 7f33b9143700 1 --2- 172.21.15.32:0/699586193 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f33980554e0 0x7f33980579a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f339c00d000 tx=0x7f339c009040).stop 2020-12-05T17:39:26.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.263+0000 7f33b9143700 1 -- 172.21.15.32:0/699586193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b40b5240 msgr2=0x7f33b40b95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:26.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.263+0000 7f33b9143700 1 --2- 172.21.15.32:0/699586193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b40b5240 0x7f33b40b95c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7f33a4002f30 tx=0x7f33a4015000).stop 2020-12-05T17:39:26.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.263+0000 7f33b9143700 1 -- 172.21.15.32:0/699586193 shutdown_connections 2020-12-05T17:39:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.263+0000 7f33b9143700 1 --2- 172.21.15.32:0/699586193 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f33b40b6690 0x7f33b40a3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.263+0000 7f33b9143700 1 --2- 172.21.15.32:0/699586193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b40b5240 0x7f33b40b95c0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.263+0000 7f33b9143700 1 --2- 172.21.15.32:0/699586193 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f33980554e0 0x7f33980579a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.263+0000 7f33b9143700 1 --2- 172.21.15.32:0/699586193 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f33b40b5b20 0x7f33b4063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.263+0000 7f33b9143700 1 -- 172.21.15.32:0/699586193 >> 172.21.15.32:0/699586193 conn(0x7f33b405f3b0 msgr2=0x7f33b4143e10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.263+0000 7f33b9143700 1 -- 172.21.15.32:0/699586193 shutdown_connections 2020-12-05T17:39:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.263+0000 7f33b9143700 1 -- 172.21.15.32:0/699586193 wait complete. 2020-12-05T17:39:26.279 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:39:26.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1be93b2700 1 Processor -- start 2020-12-05T17:39:26.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1be93b2700 1 -- start start 2020-12-05T17:39:26.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1be93b2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1be40af0d0 0x7f1be413daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:26.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1be93b2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1be413df90 0x7f1be40637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:26.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1be93b2700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1be40b0d70 0x7f1be4063ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:26.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1be93b2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1be405f5c0 con 0x7f1be413df90 2020-12-05T17:39:26.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1be93b2700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1be40af4b0 con 0x7f1be40b0d70 2020-12-05T17:39:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1be93b2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be405f7f0 con 0x7f1be40af0d0 2020-12-05T17:39:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1bdb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1be413df90 0x7f1be40637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1be8bb1700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1be40b0d70 0x7f1be4063ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1bdb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1be413df90 0x7f1be40637f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38196/0 (socket says 172.21.15.32:38196) 2020-12-05T17:39:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1be8bb1700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1be40b0d70 0x7f1be4063ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:58086/0 (socket says 172.21.15.32:58086) 2020-12-05T17:39:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1bdb7fe700 1 -- 172.21.15.32:0/2474533657 learned_addr learned my addr 172.21.15.32:0/2474533657 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1be3fff700 1 --2- 172.21.15.32:0/2474533657 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1be40af0d0 0x7f1be413daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1bdb7fe700 1 -- 172.21.15.32:0/2474533657 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1be40af0d0 msgr2=0x7f1be413daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1bdb7fe700 1 --2- 172.21.15.32:0/2474533657 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1be40af0d0 0x7f1be413daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1bdb7fe700 1 -- 172.21.15.32:0/2474533657 >> v2:172.21.15.61:3300/0 conn(0x7f1be40b0d70 msgr2=0x7f1be4063ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1bdb7fe700 1 --2- 172.21.15.32:0/2474533657 >> v2:172.21.15.61:3300/0 conn(0x7f1be40b0d70 0x7f1be4063ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1bdb7fe700 1 -- 172.21.15.32:0/2474533657 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be4066ae0 con 0x7f1be413df90 2020-12-05T17:39:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1be8bb1700 1 --2- 172.21.15.32:0/2474533657 >> v2:172.21.15.61:3300/0 conn(0x7f1be40b0d70 0x7f1be4063ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:26.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1bdb7fe700 1 --2- 172.21.15.32:0/2474533657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1be413df90 0x7f1be40637f0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7f1bcc002f30 tx=0x7f1bcc006e80).ready entity=mon.0 client_cookie=335d93893b8cba3c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:26.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1be37fe700 1 -- 172.21.15.32:0/2474533657 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bcc009640 con 0x7f1be413df90 2020-12-05T17:39:26.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1be37fe700 1 -- 172.21.15.32:0/2474533657 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bcc0097a0 con 0x7f1be413df90 2020-12-05T17:39:26.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1be37fe700 1 -- 172.21.15.32:0/2474533657 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bcc009b00 con 0x7f1be413df90 2020-12-05T17:39:26.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1be93b2700 1 -- 172.21.15.32:0/2474533657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1be413df90 msgr2=0x7f1be40637f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:26.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.391+0000 7f1be93b2700 1 --2- 172.21.15.32:0/2474533657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1be413df90 0x7f1be40637f0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7f1bcc002f30 tx=0x7f1bcc006e80).stop 2020-12-05T17:39:26.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.395+0000 7f1be93b2700 1 -- 172.21.15.32:0/2474533657 shutdown_connections 2020-12-05T17:39:26.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.395+0000 7f1be93b2700 1 --2- 172.21.15.32:0/2474533657 >> v2:172.21.15.61:3300/0 conn(0x7f1be40b0d70 0x7f1be4063ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.395+0000 7f1be93b2700 1 --2- 172.21.15.32:0/2474533657 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1be40af0d0 0x7f1be413daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.395+0000 7f1be93b2700 1 --2- 172.21.15.32:0/2474533657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1be413df90 0x7f1be40637f0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.395+0000 7f1be93b2700 1 -- 172.21.15.32:0/2474533657 >> 172.21.15.32:0/2474533657 conn(0x7f1be40620b0 msgr2=0x7f1be4062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.395+0000 7f1be93b2700 1 -- 172.21.15.32:0/2474533657 shutdown_connections 2020-12-05T17:39:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.395+0000 7f1be93b2700 1 -- 172.21.15.32:0/2474533657 wait complete. 2020-12-05T17:39:26.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.395+0000 7f1be93b2700 1 Processor -- start 2020-12-05T17:39:26.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.395+0000 7f1be93b2700 1 -- start start 2020-12-05T17:39:26.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.395+0000 7f1be93b2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1be40af0d0 0x7f1be4141f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:26.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.395+0000 7f1be93b2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1be40b0d70 0x7f1be4139b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:26.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.395+0000 7f1be93b2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1be413df90 0x7f1be413a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:26.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.395+0000 7f1be93b2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be4140050 con 0x7f1be40b0d70 2020-12-05T17:39:26.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.395+0000 7f1be93b2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be413eb10 con 0x7f1be40af0d0 2020-12-05T17:39:26.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.395+0000 7f1be93b2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be4150880 con 0x7f1be413df90 2020-12-05T17:39:26.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.399+0000 7f1bdb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1be40b0d70 0x7f1be4139b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.399+0000 7f1bdb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1be40b0d70 0x7f1be4139b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38200/0 (socket says 172.21.15.32:38200) 2020-12-05T17:39:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.399+0000 7f1bdb7fe700 1 -- 172.21.15.32:0/1387933513 learned_addr learned my addr 172.21.15.32:0/1387933513 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.399+0000 7f1be8bb1700 1 --2- 172.21.15.32:0/1387933513 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1be413df90 0x7f1be413a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.399+0000 7f1be3fff700 1 --2- 172.21.15.32:0/1387933513 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1be40af0d0 0x7f1be4141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.399+0000 7f1bdb7fe700 1 -- 172.21.15.32:0/1387933513 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1be413df90 msgr2=0x7f1be413a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:26.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.399+0000 7f1bdb7fe700 1 --2- 172.21.15.32:0/1387933513 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1be413df90 0x7f1be413a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.399+0000 7f1bdb7fe700 1 -- 172.21.15.32:0/1387933513 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1be40af0d0 msgr2=0x7f1be4141f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:26.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.399+0000 7f1bdb7fe700 1 --2- 172.21.15.32:0/1387933513 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1be40af0d0 0x7f1be4141f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.399+0000 7f1bdb7fe700 1 -- 172.21.15.32:0/1387933513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be413a570 con 0x7f1be40b0d70 2020-12-05T17:39:26.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.399+0000 7f1be3fff700 1 --2- 172.21.15.32:0/1387933513 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1be40af0d0 0x7f1be4141f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:26.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.399+0000 7f1bdb7fe700 1 --2- 172.21.15.32:0/1387933513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1be40b0d70 0x7f1be4139b90 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7f1bcc002f30 tx=0x7f1bcc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:26.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.399+0000 7f1be1ffb700 1 -- 172.21.15.32:0/1387933513 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bcc011a20 con 0x7f1be40b0d70 2020-12-05T17:39:26.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.399+0000 7f1be1ffb700 1 -- 172.21.15.32:0/1387933513 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bcc011b80 con 0x7f1be40b0d70 2020-12-05T17:39:26.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.399+0000 7f1be93b2700 1 -- 172.21.15.32:0/1387933513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be413e670 con 0x7f1be40b0d70 2020-12-05T17:39:26.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.399+0000 7f1be1ffb700 1 -- 172.21.15.32:0/1387933513 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bcc01d450 con 0x7f1be40b0d70 2020-12-05T17:39:26.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.399+0000 7f1be93b2700 1 -- 172.21.15.32:0/1387933513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be4152d20 con 0x7f1be40b0d70 2020-12-05T17:39:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.399+0000 7f1be93b2700 1 -- 172.21.15.32:0/1387933513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be4057050 con 0x7f1be40b0d70 2020-12-05T17:39:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.399+0000 7f1be1ffb700 1 -- 172.21.15.32:0/1387933513 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1bcc00f070 con 0x7f1be40b0d70 2020-12-05T17:39:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.399+0000 7f1be1ffb700 1 --2- 172.21.15.32:0/1387933513 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1bc80554e0 0x7f1bc80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.403+0000 7f1be1ffb700 1 -- 172.21.15.32:0/1387933513 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1bcc06f760 con 0x7f1be40b0d70 2020-12-05T17:39:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.403+0000 7f1be3fff700 1 --2- 172.21.15.32:0/1387933513 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1bc80554e0 0x7f1bc80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.403+0000 7f1be3fff700 1 --2- 172.21.15.32:0/1387933513 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1bc80554e0 0x7f1bc80579a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f1bd400b000 tx=0x7f1bd40091a0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.407+0000 7f1be1ffb700 1 -- 172.21.15.32:0/1387933513 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1bcc0459c0 con 0x7f1be40b0d70 2020-12-05T17:39:26.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.547+0000 7f1be93b2700 1 -- 172.21.15.32:0/1387933513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1be413fe60 con 0x7f1be40b0d70 2020-12-05T17:39:26.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.551+0000 7f1be1ffb700 1 -- 172.21.15.32:0/1387933513 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v92) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1bcc048f20 con 0x7f1be40b0d70 2020-12-05T17:39:26.555 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:39:26.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.551+0000 7f1be93b2700 1 -- 172.21.15.32:0/1387933513 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1bc80554e0 msgr2=0x7f1bc80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:26.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.551+0000 7f1be93b2700 1 --2- 172.21.15.32:0/1387933513 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1bc80554e0 0x7f1bc80579a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f1bd400b000 tx=0x7f1bd40091a0).stop 2020-12-05T17:39:26.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.551+0000 7f1be93b2700 1 -- 172.21.15.32:0/1387933513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1be40b0d70 msgr2=0x7f1be4139b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:26.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.551+0000 7f1be93b2700 1 --2- 172.21.15.32:0/1387933513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1be40b0d70 0x7f1be4139b90 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7f1bcc002f30 tx=0x7f1bcc015000).stop 2020-12-05T17:39:26.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.551+0000 7f1be93b2700 1 -- 172.21.15.32:0/1387933513 shutdown_connections 2020-12-05T17:39:26.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.551+0000 7f1be93b2700 1 --2- 172.21.15.32:0/1387933513 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1bc80554e0 0x7f1bc80579a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.551+0000 7f1be93b2700 1 --2- 172.21.15.32:0/1387933513 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1be413df90 0x7f1be413a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.551+0000 7f1be93b2700 1 --2- 172.21.15.32:0/1387933513 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1be40af0d0 0x7f1be4141f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.551+0000 7f1be93b2700 1 --2- 172.21.15.32:0/1387933513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1be40b0d70 0x7f1be4139b90 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.551+0000 7f1be93b2700 1 -- 172.21.15.32:0/1387933513 >> 172.21.15.32:0/1387933513 conn(0x7f1be40620b0 msgr2=0x7f1be40b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:26.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.551+0000 7f1be93b2700 1 -- 172.21.15.32:0/1387933513 shutdown_connections 2020-12-05T17:39:26.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.551+0000 7f1be93b2700 1 -- 172.21.15.32:0/1387933513 wait complete. 2020-12-05T17:39:26.570 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:39:26.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa18c819700 1 Processor -- start 2020-12-05T17:39:26.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa18c819700 1 -- start start 2020-12-05T17:39:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa18c819700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa1880b2000 0x7fa188145e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa18c819700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1880af4b0 0x7fa1881462f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa18c819700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1880b23e0 con 0x7fa188061780 2020-12-05T17:39:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa18c819700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1880ae7b0 con 0x7fa1880af4b0 2020-12-05T17:39:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa18c819700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1880aea50 con 0x7fa1880b2000 2020-12-05T17:39:26.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa186ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1880af4b0 0x7fa1881462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:26.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa186ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1880af4b0 0x7fa1881462f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38206/0 (socket says 172.21.15.32:38206) 2020-12-05T17:39:26.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa186ffd700 1 -- 172.21.15.32:0/506184998 learned_addr learned my addr 172.21.15.32:0/506184998 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:26.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa1877fe700 1 --2- 172.21.15.32:0/506184998 >> v2:172.21.15.61:3300/0 conn(0x7fa1880b2000 0x7fa188145e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:26.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa186ffd700 1 -- 172.21.15.32:0/506184998 >> v2:172.21.15.61:3300/0 conn(0x7fa1880b2000 msgr2=0x7fa188145e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:26.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa186ffd700 1 --2- 172.21.15.32:0/506184998 >> v2:172.21.15.61:3300/0 conn(0x7fa1880b2000 0x7fa188145e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa186ffd700 1 -- 172.21.15.32:0/506184998 >> v1:172.21.15.61:6789/0 conn(0x7fa188061780 legacy=0x7fa1880afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:26.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa186ffd700 1 -- 172.21.15.32:0/506184998 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1880b0f60 con 0x7fa1880af4b0 2020-12-05T17:39:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa186ffd700 1 --2- 172.21.15.32:0/506184998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1880af4b0 0x7fa1881462f0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7fa170002f30 tx=0x7fa170006e80).ready entity=mon.0 client_cookie=466edcd8170fad48 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa1867fc700 1 -- 172.21.15.32:0/506184998 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1700097f0 con 0x7fa1880af4b0 2020-12-05T17:39:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa1867fc700 1 -- 172.21.15.32:0/506184998 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa170009950 con 0x7fa1880af4b0 2020-12-05T17:39:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa1867fc700 1 -- 172.21.15.32:0/506184998 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa170009cb0 con 0x7fa1880af4b0 2020-12-05T17:39:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa18c819700 1 -- 172.21.15.32:0/506184998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1880af4b0 msgr2=0x7fa1881462f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa18c819700 1 --2- 172.21.15.32:0/506184998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1880af4b0 0x7fa1881462f0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7fa170002f30 tx=0x7fa170006e80).stop 2020-12-05T17:39:26.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa18c819700 1 -- 172.21.15.32:0/506184998 shutdown_connections 2020-12-05T17:39:26.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa18c819700 1 --2- 172.21.15.32:0/506184998 >> v2:172.21.15.61:3300/0 conn(0x7fa1880b2000 0x7fa188145e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa18c819700 1 --2- 172.21.15.32:0/506184998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1880af4b0 0x7fa1881462f0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa18c819700 1 -- 172.21.15.32:0/506184998 >> 172.21.15.32:0/506184998 conn(0x7fa1880b5620 msgr2=0x7fa1880b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:26.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa18c819700 1 -- 172.21.15.32:0/506184998 shutdown_connections 2020-12-05T17:39:26.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.687+0000 7fa18c819700 1 -- 172.21.15.32:0/506184998 wait complete. 2020-12-05T17:39:26.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa18c819700 1 Processor -- start 2020-12-05T17:39:26.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa18c819700 1 -- start start 2020-12-05T17:39:26.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa18c819700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa188061780 0x7fa188066b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:26.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa18c819700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1880af4b0 0x7fa188159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:26.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa18c819700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1880b2000 0x7fa18814c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:26.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa18c819700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1880632f0 con 0x7fa1880af4b0 2020-12-05T17:39:26.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa18c819700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1881591d0 con 0x7fa188061780 2020-12-05T17:39:26.707 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa18c819700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa188159470 con 0x7fa1880b2000 2020-12-05T17:39:26.707 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa186ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1880af4b0 0x7fa188159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:26.707 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa186ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1880af4b0 0x7fa188159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38214/0 (socket says 172.21.15.32:38214) 2020-12-05T17:39:26.707 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa186ffd700 1 -- 172.21.15.32:0/1462893914 learned_addr learned my addr 172.21.15.32:0/1462893914 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:26.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa187fff700 1 --2- 172.21.15.32:0/1462893914 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1880b2000 0x7fa18814c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:26.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa1877fe700 1 --2- 172.21.15.32:0/1462893914 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa188061780 0x7fa188066b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:26.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa186ffd700 1 -- 172.21.15.32:0/1462893914 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1880b2000 msgr2=0x7fa18814c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:26.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa186ffd700 1 --2- 172.21.15.32:0/1462893914 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1880b2000 0x7fa18814c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa186ffd700 1 -- 172.21.15.32:0/1462893914 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa188061780 msgr2=0x7fa188066b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:26.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa186ffd700 1 --2- 172.21.15.32:0/1462893914 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa188061780 0x7fa188066b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa186ffd700 1 -- 172.21.15.32:0/1462893914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa188159710 con 0x7fa1880af4b0 2020-12-05T17:39:26.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa1877fe700 1 --2- 172.21.15.32:0/1462893914 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa188061780 0x7fa188066b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:39:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa186ffd700 1 --2- 172.21.15.32:0/1462893914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1880af4b0 0x7fa188159bb0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7fa170002f30 tx=0x7fa170015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa184ff9700 1 -- 172.21.15.32:0/1462893914 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa170011ce0 con 0x7fa1880af4b0 2020-12-05T17:39:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa184ff9700 1 -- 172.21.15.32:0/1462893914 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa170011e40 con 0x7fa1880af4b0 2020-12-05T17:39:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa18c819700 1 -- 172.21.15.32:0/1462893914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18814ccd0 con 0x7fa1880af4b0 2020-12-05T17:39:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa184ff9700 1 -- 172.21.15.32:0/1462893914 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa17001e650 con 0x7fa1880af4b0 2020-12-05T17:39:26.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.691+0000 7fa18c819700 1 -- 172.21.15.32:0/1462893914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1880636d0 con 0x7fa1880af4b0 2020-12-05T17:39:26.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.695+0000 7fa18c819700 1 -- 172.21.15.32:0/1462893914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1881598a0 con 0x7fa1880af4b0 2020-12-05T17:39:26.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.695+0000 7fa184ff9700 1 -- 172.21.15.32:0/1462893914 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa170025030 con 0x7fa1880af4b0 2020-12-05T17:39:26.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.695+0000 7fa184ff9700 1 --2- 172.21.15.32:0/1462893914 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa16c0554e0 0x7fa16c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:26.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.695+0000 7fa184ff9700 1 -- 172.21.15.32:0/1462893914 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa17006f400 con 0x7fa1880af4b0 2020-12-05T17:39:26.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.695+0000 7fa1877fe700 1 --2- 172.21.15.32:0/1462893914 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa16c0554e0 0x7fa16c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:26.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.695+0000 7fa1877fe700 1 --2- 172.21.15.32:0/1462893914 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa16c0554e0 0x7fa16c0579a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fa178008000 tx=0x7fa178008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:26.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.703+0000 7fa184ff9700 1 -- 172.21.15.32:0/1462893914 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa1700455a0 con 0x7fa1880af4b0 2020-12-05T17:39:26.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.847+0000 7fa18c819700 1 -- 172.21.15.32:0/1462893914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa188057050 con 0x7fa1880af4b0 2020-12-05T17:39:26.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.847+0000 7fa184ff9700 1 -- 172.21.15.32:0/1462893914 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+10 (secure 0 0 0) 0x7fa1700230a0 con 0x7fa1880af4b0 2020-12-05T17:39:26.853 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:39:26.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.847+0000 7fa18c819700 1 -- 172.21.15.32:0/1462893914 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa16c0554e0 msgr2=0x7fa16c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:26.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.847+0000 7fa18c819700 1 --2- 172.21.15.32:0/1462893914 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa16c0554e0 0x7fa16c0579a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fa178008000 tx=0x7fa178008040).stop 2020-12-05T17:39:26.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.847+0000 7fa18c819700 1 -- 172.21.15.32:0/1462893914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1880af4b0 msgr2=0x7fa188159bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:26.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.851+0000 7fa18c819700 1 --2- 172.21.15.32:0/1462893914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1880af4b0 0x7fa188159bb0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7fa170002f30 tx=0x7fa170015000).stop 2020-12-05T17:39:26.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.851+0000 7fa18c819700 1 -- 172.21.15.32:0/1462893914 shutdown_connections 2020-12-05T17:39:26.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.851+0000 7fa18c819700 1 --2- 172.21.15.32:0/1462893914 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa16c0554e0 0x7fa16c0579a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.851+0000 7fa18c819700 1 --2- 172.21.15.32:0/1462893914 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1880b2000 0x7fa18814c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.851+0000 7fa18c819700 1 --2- 172.21.15.32:0/1462893914 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa188061780 0x7fa188066b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.851+0000 7fa18c819700 1 --2- 172.21.15.32:0/1462893914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1880af4b0 0x7fa188159bb0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.851+0000 7fa18c819700 1 -- 172.21.15.32:0/1462893914 >> 172.21.15.32:0/1462893914 conn(0x7fa1880b5620 msgr2=0x7fa1880adc20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:26.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.851+0000 7fa18c819700 1 -- 172.21.15.32:0/1462893914 shutdown_connections 2020-12-05T17:39:26.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.851+0000 7fa18c819700 1 -- 172.21.15.32:0/1462893914 wait complete. 2020-12-05T17:39:26.867 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:39:26.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.983+0000 7f8c555ea700 1 Processor -- start 2020-12-05T17:39:26.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.983+0000 7f8c555ea700 1 -- start start 2020-12-05T17:39:26.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.983+0000 7f8c555ea700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f8c500b2960 0x7f8c50140400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:26.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.983+0000 7f8c555ea700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c500ad280 0x7f8c500660e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:26.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.983+0000 7f8c555ea700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c500ad9b0 con 0x7f8c50061b30 2020-12-05T17:39:26.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.983+0000 7f8c555ea700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c500b32c0 con 0x7f8c500b2960 2020-12-05T17:39:26.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.983+0000 7f8c555ea700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5013f4f0 con 0x7f8c500ad280 2020-12-05T17:39:26.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.983+0000 7f8c54de9700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8c50061b30 0x7f8c500ae830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40872/0 (socket says 172.21.15.32:40872) 2020-12-05T17:39:26.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.983+0000 7f8c54de9700 1 -- 172.21.15.32:0/2120149540 learned_addr learned my addr 172.21.15.32:0/2120149540 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:26.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.983+0000 7f8c4f7fe700 1 --2- 172.21.15.32:0/2120149540 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c500ad280 0x7f8c500660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:26.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.983+0000 7f8c4ffff700 1 --2- 172.21.15.32:0/2120149540 >> v2:172.21.15.61:3300/0 conn(0x7f8c500b2960 0x7f8c50140400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:26.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.983+0000 7f8c4effd700 1 -- 172.21.15.32:0/2120149540 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2838377018 0 0) 0x7f8c500ad9b0 con 0x7f8c50061b30 2020-12-05T17:39:26.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.983+0000 7f8c4effd700 1 -- 172.21.15.32:0/2120149540 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c3c003230 con 0x7f8c50061b30 2020-12-05T17:39:26.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.983+0000 7f8c4ffff700 1 -- 172.21.15.32:0/2120149540 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c500ad280 msgr2=0x7f8c500660e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:26.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.983+0000 7f8c4ffff700 1 --2- 172.21.15.32:0/2120149540 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c500ad280 0x7f8c500660e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.983+0000 7f8c4ffff700 1 -- 172.21.15.32:0/2120149540 >> v1:172.21.15.32:6789/0 conn(0x7f8c50061b30 legacy=0x7f8c500ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:26.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.983+0000 7f8c4ffff700 1 -- 172.21.15.32:0/2120149540 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c500b3020 con 0x7f8c500b2960 2020-12-05T17:39:26.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.983+0000 7f8c4f7fe700 1 --2- 172.21.15.32:0/2120149540 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c500ad280 0x7f8c500660e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:26.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.983+0000 7f8c4ffff700 1 --2- 172.21.15.32:0/2120149540 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c500b2960 0x7f8c50140400 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f8c400032b0 tx=0x7f8c400091a0).ready entity=mon.1 client_cookie=c1a567e29dfc4618 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:26.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.983+0000 7f8c4effd700 1 -- 172.21.15.32:0/2120149540 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c4000b940 con 0x7f8c500b2960 2020-12-05T17:39:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c4effd700 1 -- 172.21.15.32:0/2120149540 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4000baa0 con 0x7f8c500b2960 2020-12-05T17:39:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c555ea700 1 -- 172.21.15.32:0/2120149540 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c500b2960 msgr2=0x7f8c50140400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c4effd700 1 -- 172.21.15.32:0/2120149540 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c4000bc10 con 0x7f8c500b2960 2020-12-05T17:39:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c555ea700 1 --2- 172.21.15.32:0/2120149540 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c500b2960 0x7f8c50140400 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f8c400032b0 tx=0x7f8c400091a0).stop 2020-12-05T17:39:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c555ea700 1 -- 172.21.15.32:0/2120149540 shutdown_connections 2020-12-05T17:39:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c555ea700 1 --2- 172.21.15.32:0/2120149540 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c500b2960 0x7f8c50140400 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c555ea700 1 --2- 172.21.15.32:0/2120149540 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c500ad280 0x7f8c500660e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c555ea700 1 -- 172.21.15.32:0/2120149540 >> 172.21.15.32:0/2120149540 conn(0x7f8c50061010 msgr2=0x7f8c50061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:27.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c555ea700 1 -- 172.21.15.32:0/2120149540 shutdown_connections 2020-12-05T17:39:27.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c555ea700 1 -- 172.21.15.32:0/2120149540 wait complete. 2020-12-05T17:39:27.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c555ea700 1 Processor -- start 2020-12-05T17:39:27.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c555ea700 1 -- start start 2020-12-05T17:39:27.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c555ea700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c50061b30 0x7f8c500b4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:27.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c555ea700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c500ad280 0x7f8c50152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:27.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c555ea700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c500b2960 0x7f8c50147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:27.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c555ea700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c50147bc0 con 0x7f8c500b2960 2020-12-05T17:39:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c555ea700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c501459d0 con 0x7f8c500ad280 2020-12-05T17:39:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c555ea700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c50145380 con 0x7f8c50061b30 2020-12-05T17:39:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c54de9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c500b2960 0x7f8c50147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:27.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c54de9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c500b2960 0x7f8c50147ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38230/0 (socket says 172.21.15.32:38230) 2020-12-05T17:39:27.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c54de9700 1 -- 172.21.15.32:0/3687869956 learned_addr learned my addr 172.21.15.32:0/3687869956 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:27.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c4ffff700 1 --2- 172.21.15.32:0/3687869956 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c50061b30 0x7f8c500b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:27.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c4f7fe700 1 --2- 172.21.15.32:0/3687869956 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c500ad280 0x7f8c50152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:27.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c54de9700 1 -- 172.21.15.32:0/3687869956 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c50061b30 msgr2=0x7f8c500b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:27.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c54de9700 1 --2- 172.21.15.32:0/3687869956 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c50061b30 0x7f8c500b4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:27.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c54de9700 1 -- 172.21.15.32:0/3687869956 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c500ad280 msgr2=0x7f8c50152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:27.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c54de9700 1 --2- 172.21.15.32:0/3687869956 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c500ad280 0x7f8c50152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:27.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c54de9700 1 -- 172.21.15.32:0/3687869956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c50145620 con 0x7f8c500b2960 2020-12-05T17:39:27.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c54de9700 1 --2- 172.21.15.32:0/3687869956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c500b2960 0x7f8c50147ff0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7f8c480010d0 tx=0x7f8c48007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:27.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c4d7fa700 1 -- 172.21.15.32:0/3687869956 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c48007320 con 0x7f8c500b2960 2020-12-05T17:39:27.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c4d7fa700 1 -- 172.21.15.32:0/3687869956 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c48007480 con 0x7f8c500b2960 2020-12-05T17:39:27.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.987+0000 7f8c555ea700 1 -- 172.21.15.32:0/3687869956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c50066420 con 0x7f8c500b2960 2020-12-05T17:39:27.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.991+0000 7f8c4d7fa700 1 -- 172.21.15.32:0/3687869956 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c480057d0 con 0x7f8c500b2960 2020-12-05T17:39:27.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.991+0000 7f8c555ea700 1 -- 172.21.15.32:0/3687869956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c50144450 con 0x7f8c500b2960 2020-12-05T17:39:27.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.991+0000 7f8c555ea700 1 -- 172.21.15.32:0/3687869956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c50057050 con 0x7f8c500b2960 2020-12-05T17:39:27.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.991+0000 7f8c4d7fa700 1 -- 172.21.15.32:0/3687869956 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8c48010030 con 0x7f8c500b2960 2020-12-05T17:39:27.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.991+0000 7f8c4d7fa700 1 --2- 172.21.15.32:0/3687869956 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8c30055440 0x7f8c30057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:27.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.991+0000 7f8c4d7fa700 1 -- 172.21.15.32:0/3687869956 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8c4806df70 con 0x7f8c500b2960 2020-12-05T17:39:27.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.991+0000 7f8c4ffff700 1 --2- 172.21.15.32:0/3687869956 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8c30055440 0x7f8c30057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:27.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.991+0000 7f8c4ffff700 1 --2- 172.21.15.32:0/3687869956 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8c30055440 0x7f8c30057900 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f8c4001a000 tx=0x7f8c40016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:27.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:26.999+0000 7f8c4d7fa700 1 -- 172.21.15.32:0/3687869956 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8c48043f80 con 0x7f8c500b2960 2020-12-05T17:39:27.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.139+0000 7f8c555ea700 1 -- 172.21.15.32:0/3687869956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8c50069690 con 0x7f8c500b2960 2020-12-05T17:39:27.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.139+0000 7f8c4d7fa700 1 -- 172.21.15.32:0/3687869956 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+10 (secure 0 0 0) 0x7f8c480475a0 con 0x7f8c500b2960 2020-12-05T17:39:27.146 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:39:27.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.143+0000 7f8c555ea700 1 -- 172.21.15.32:0/3687869956 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8c30055440 msgr2=0x7f8c30057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:27.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.143+0000 7f8c555ea700 1 --2- 172.21.15.32:0/3687869956 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8c30055440 0x7f8c30057900 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f8c4001a000 tx=0x7f8c40016000).stop 2020-12-05T17:39:27.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.143+0000 7f8c555ea700 1 -- 172.21.15.32:0/3687869956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c500b2960 msgr2=0x7f8c50147ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:27.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.143+0000 7f8c555ea700 1 --2- 172.21.15.32:0/3687869956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c500b2960 0x7f8c50147ff0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7f8c480010d0 tx=0x7f8c48007ac0).stop 2020-12-05T17:39:27.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.143+0000 7f8c555ea700 1 -- 172.21.15.32:0/3687869956 shutdown_connections 2020-12-05T17:39:27.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.143+0000 7f8c555ea700 1 --2- 172.21.15.32:0/3687869956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c500b2960 0x7f8c50147ff0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:27.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.143+0000 7f8c555ea700 1 --2- 172.21.15.32:0/3687869956 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8c30055440 0x7f8c30057900 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:27.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.143+0000 7f8c555ea700 1 --2- 172.21.15.32:0/3687869956 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c50061b30 0x7f8c500b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:27.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.143+0000 7f8c555ea700 1 --2- 172.21.15.32:0/3687869956 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c500ad280 0x7f8c50152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.143+0000 7f8c555ea700 1 -- 172.21.15.32:0/3687869956 >> 172.21.15.32:0/3687869956 conn(0x7f8c50061010 msgr2=0x7f8c500b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.143+0000 7f8c555ea700 1 -- 172.21.15.32:0/3687869956 shutdown_connections 2020-12-05T17:39:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.143+0000 7f8c555ea700 1 -- 172.21.15.32:0/3687869956 wait complete. 2020-12-05T17:39:27.160 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:39:27.160 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:39:27.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c8c38a700 1 Processor -- start 2020-12-05T17:39:27.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c8c38a700 1 -- start start 2020-12-05T17:39:27.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c8c38a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0c840acb80 0x7f0c840acf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:27.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c8c38a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0c8405fdc0 0x7f0c8413fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:27.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c8c38a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c840affa0 con 0x7f0c84067480 2020-12-05T17:39:27.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c8c38a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c840ad6a0 con 0x7f0c8405fdc0 2020-12-05T17:39:27.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c8c38a700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c84067a00 con 0x7f0c840acb80 2020-12-05T17:39:27.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c89925700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0c84067480 0x7f0c84142800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40884/0 (socket says 172.21.15.32:40884) 2020-12-05T17:39:27.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c89925700 1 -- 172.21.15.32:0/164092045 learned_addr learned my addr 172.21.15.32:0/164092045 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:27.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c8a927700 1 --2- 172.21.15.32:0/164092045 >> v2:172.21.15.32:3300/0 conn(0x7f0c8405fdc0 0x7f0c8413fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:27.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c8a126700 1 --2- 172.21.15.32:0/164092045 >> v2:172.21.15.61:3300/0 conn(0x7f0c840acb80 0x7f0c840acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:27.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c89124700 1 -- 172.21.15.32:0/164092045 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130161529 0 0) 0x7f0c840affa0 con 0x7f0c84067480 2020-12-05T17:39:27.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c89124700 1 -- 172.21.15.32:0/164092045 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c70003230 con 0x7f0c84067480 2020-12-05T17:39:27.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c89124700 1 -- 172.21.15.32:0/164092045 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 228421835 0 0) 0x7f0c70003230 con 0x7f0c84067480 2020-12-05T17:39:27.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c89124700 1 -- 172.21.15.32:0/164092045 >> v2:172.21.15.61:3300/0 conn(0x7f0c840acb80 msgr2=0x7f0c840acf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:27.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c89124700 1 --2- 172.21.15.32:0/164092045 >> v2:172.21.15.61:3300/0 conn(0x7f0c840acb80 0x7f0c840acf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:27.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c89124700 1 -- 172.21.15.32:0/164092045 >> v2:172.21.15.32:3300/0 conn(0x7f0c8405fdc0 msgr2=0x7f0c8413fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:27.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c89124700 1 --2- 172.21.15.32:0/164092045 >> v2:172.21.15.32:3300/0 conn(0x7f0c8405fdc0 0x7f0c8413fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:27.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c89124700 1 -- 172.21.15.32:0/164092045 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c840b01d0 con 0x7f0c84067480 2020-12-05T17:39:27.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c89124700 1 -- 172.21.15.32:0/164092045 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f0c780016d0 con 0x7f0c84067480 2020-12-05T17:39:27.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c89124700 1 -- 172.21.15.32:0/164092045 >> v1:172.21.15.32:6789/0 conn(0x7f0c84067480 legacy=0x7f0c84142800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:27.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c89124700 1 --2- 172.21.15.32:0/164092045 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c70003720 0x7f0c70003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:27.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c89124700 1 --2- 172.21.15.32:0/164092045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c700044d0 0x7f0c70004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:27.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c89124700 1 --2- 172.21.15.32:0/164092045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c70004e40 0x7f0c7000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:27.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c89124700 1 -- 172.21.15.32:0/164092045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c840ad6a0 con 0x7f0c70004e40 2020-12-05T17:39:27.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c89124700 1 -- 172.21.15.32:0/164092045 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c84067a00 con 0x7f0c700044d0 2020-12-05T17:39:27.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c89124700 1 -- 172.21.15.32:0/164092045 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c70004250 con 0x7f0c70003720 2020-12-05T17:39:27.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c8a927700 1 --2- 172.21.15.32:0/164092045 >> v2:172.21.15.32:3300/0 conn(0x7f0c8405fdc0 0x7f0c8413fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:27.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c8a126700 1 --2- 172.21.15.32:0/164092045 >> v2:172.21.15.61:3300/0 conn(0x7f0c840acb80 0x7f0c840acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:27.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c8a927700 1 --2- 172.21.15.32:0/164092045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c70004e40 0x7f0c7000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:27.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c89925700 1 --2- 172.21.15.32:0/164092045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c700044d0 0x7f0c70004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:27.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c8a126700 1 --2- 172.21.15.32:0/164092045 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c70003720 0x7f0c70003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:27.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c8a927700 1 -- 172.21.15.32:0/164092045 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c70003720 msgr2=0x7f0c70003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:27.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c8a927700 1 --2- 172.21.15.32:0/164092045 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c70003720 0x7f0c70003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:27.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c8a927700 1 -- 172.21.15.32:0/164092045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c700044d0 msgr2=0x7f0c70004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:27.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.267+0000 7f0c8a927700 1 --2- 172.21.15.32:0/164092045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c700044d0 0x7f0c70004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:27.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.271+0000 7f0c8a927700 1 -- 172.21.15.32:0/164092045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c70003230 con 0x7f0c70004e40 2020-12-05T17:39:27.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.271+0000 7f0c8a927700 1 -- 172.21.15.32:0/164092045 reap_dead start 2020-12-05T17:39:27.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.271+0000 7f0c8a927700 1 --2- 172.21.15.32:0/164092045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c70004e40 0x7f0c7000b720 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7f0c7400c000 tx=0x7f0c74008040).ready entity=mon.0 client_cookie=60eb016045a43168 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:27.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.271+0000 7f0c89124700 1 -- 172.21.15.32:0/164092045 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c7400b4d0 con 0x7f0c70004e40 2020-12-05T17:39:27.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.271+0000 7f0c89124700 1 -- 172.21.15.32:0/164092045 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7400eb80 con 0x7f0c70004e40 2020-12-05T17:39:27.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.271+0000 7f0c89124700 1 -- 172.21.15.32:0/164092045 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c740123f0 con 0x7f0c70004e40 2020-12-05T17:39:27.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.271+0000 7f0c8c38a700 1 -- 172.21.15.32:0/164092045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c70004e40 msgr2=0x7f0c7000b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:27.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.271+0000 7f0c8c38a700 1 --2- 172.21.15.32:0/164092045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c70004e40 0x7f0c7000b720 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7f0c7400c000 tx=0x7f0c74008040).stop 2020-12-05T17:39:27.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.271+0000 7f0c8c38a700 1 -- 172.21.15.32:0/164092045 shutdown_connections 2020-12-05T17:39:27.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.271+0000 7f0c8c38a700 1 --2- 172.21.15.32:0/164092045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c70004e40 0x7f0c7000b720 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:27.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.271+0000 7f0c8c38a700 1 -- 172.21.15.32:0/164092045 >> 172.21.15.32:0/164092045 conn(0x7f0c840b5580 msgr2=0x7f0c840b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:27.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.271+0000 7f0c8c38a700 1 -- 172.21.15.32:0/164092045 shutdown_connections 2020-12-05T17:39:27.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.271+0000 7f0c8c38a700 1 -- 172.21.15.32:0/164092045 wait complete. 2020-12-05T17:39:27.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.271+0000 7f0c8c38a700 1 Processor -- start 2020-12-05T17:39:27.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.271+0000 7f0c8c38a700 1 -- start start 2020-12-05T17:39:27.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c8c38a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c70004e40 0x7f0c84110b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:27.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c8c38a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c8410ff40 0x7f0c8413ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:27.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c8c38a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c84140440 0x7f0c84151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:27.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c8c38a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c84113420 con 0x7f0c8410ff40 2020-12-05T17:39:27.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c8c38a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c84113180 con 0x7f0c84140440 2020-12-05T17:39:27.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c8c38a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8410eca0 con 0x7f0c70004e40 2020-12-05T17:39:27.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c89925700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c8410ff40 0x7f0c8413ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:27.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c89925700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c8410ff40 0x7f0c8413ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38248/0 (socket says 172.21.15.32:38248) 2020-12-05T17:39:27.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c89925700 1 -- 172.21.15.32:0/3071822636 learned_addr learned my addr 172.21.15.32:0/3071822636 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:27.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c8a126700 1 --2- 172.21.15.32:0/3071822636 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c70004e40 0x7f0c84110b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:27.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c8a927700 1 --2- 172.21.15.32:0/3071822636 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c84140440 0x7f0c84151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:27.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c89925700 1 -- 172.21.15.32:0/3071822636 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c70004e40 msgr2=0x7f0c84110b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:27.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c89925700 1 --2- 172.21.15.32:0/3071822636 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c70004e40 0x7f0c84110b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:27.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c89925700 1 -- 172.21.15.32:0/3071822636 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c84140440 msgr2=0x7f0c84151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:27.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c89925700 1 --2- 172.21.15.32:0/3071822636 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c84140440 0x7f0c84151490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:27.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c89925700 1 -- 172.21.15.32:0/3071822636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c841408c0 con 0x7f0c8410ff40 2020-12-05T17:39:27.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c89925700 1 --2- 172.21.15.32:0/3071822636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c8410ff40 0x7f0c8413ff50 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7f0c7800d000 tx=0x7f0c840b0360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:27.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c837fe700 1 -- 172.21.15.32:0/3071822636 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c78004ea0 con 0x7f0c8410ff40 2020-12-05T17:39:27.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c837fe700 1 -- 172.21.15.32:0/3071822636 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7800fa30 con 0x7f0c8410ff40 2020-12-05T17:39:27.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c8c38a700 1 -- 172.21.15.32:0/3071822636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8410f4a0 con 0x7f0c8410ff40 2020-12-05T17:39:27.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c837fe700 1 -- 172.21.15.32:0/3071822636 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c7800fd90 con 0x7f0c8410ff40 2020-12-05T17:39:27.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c8c38a700 1 -- 172.21.15.32:0/3071822636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c84149860 con 0x7f0c8410ff40 2020-12-05T17:39:27.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c8c38a700 1 -- 172.21.15.32:0/3071822636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c84112fd0 con 0x7f0c8410ff40 2020-12-05T17:39:27.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c837fe700 1 -- 172.21.15.32:0/3071822636 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0c7801b030 con 0x7f0c8410ff40 2020-12-05T17:39:27.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c837fe700 1 --2- 172.21.15.32:0/3071822636 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c640553b0 0x7f0c64057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:27.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c837fe700 1 -- 172.21.15.32:0/3071822636 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0c7806da80 con 0x7f0c8410ff40 2020-12-05T17:39:27.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.275+0000 7f0c8a126700 1 --2- 172.21.15.32:0/3071822636 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c640553b0 0x7f0c64057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:27.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.279+0000 7f0c8a126700 1 --2- 172.21.15.32:0/3071822636 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c640553b0 0x7f0c64057870 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f0c7c00b000 tx=0x7f0c7c0091a0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:27.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.283+0000 7f0c837fe700 1 -- 172.21.15.32:0/3071822636 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0c78043c20 con 0x7f0c8410ff40 2020-12-05T17:39:27.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.467+0000 7f0c8c38a700 1 -- 172.21.15.32:0/3071822636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c84057050 con 0x7f0c8410ff40 2020-12-05T17:39:27.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.467+0000 7f0c837fe700 1 -- 172.21.15.32:0/3071822636 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f0c78019030 con 0x7f0c8410ff40 2020-12-05T17:39:27.473 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:27.473 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":42,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6823,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2664846066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2664846066},{"type":"v1","addr":"172.21.15.61:6833","nonce":2664846066}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:39:22.261725+0000","modified":"2020-12-05T17:39:23.265348+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6930":{"gid":6930,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4087622496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4087622496},{"type":"v1","addr":"172.21.15.32:6835","nonce":4087622496}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2020-12-05T17:39:27.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.467+0000 7f0c8c38a700 1 -- 172.21.15.32:0/3071822636 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c640553b0 msgr2=0x7f0c64057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:27.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.467+0000 7f0c8c38a700 1 --2- 172.21.15.32:0/3071822636 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c640553b0 0x7f0c64057870 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f0c7c00b000 tx=0x7f0c7c0091a0).stop 2020-12-05T17:39:27.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.467+0000 7f0c8c38a700 1 -- 172.21.15.32:0/3071822636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c8410ff40 msgr2=0x7f0c8413ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:27.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.467+0000 7f0c8c38a700 1 --2- 172.21.15.32:0/3071822636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c8410ff40 0x7f0c8413ff50 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7f0c7800d000 tx=0x7f0c840b0360).stop 2020-12-05T17:39:27.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.467+0000 7f0c8c38a700 1 -- 172.21.15.32:0/3071822636 shutdown_connections 2020-12-05T17:39:27.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.467+0000 7f0c8c38a700 1 --2- 172.21.15.32:0/3071822636 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c84140440 0x7f0c84151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:27.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.467+0000 7f0c8c38a700 1 --2- 172.21.15.32:0/3071822636 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0c640553b0 0x7f0c64057870 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:27.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.467+0000 7f0c8c38a700 1 --2- 172.21.15.32:0/3071822636 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c70004e40 0x7f0c84110b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:27.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.467+0000 7f0c8c38a700 1 --2- 172.21.15.32:0/3071822636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c8410ff40 0x7f0c8413ff50 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:27.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.467+0000 7f0c8c38a700 1 -- 172.21.15.32:0/3071822636 >> 172.21.15.32:0/3071822636 conn(0x7f0c840b5580 msgr2=0x7f0c841420f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:27.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.467+0000 7f0c8c38a700 1 -- 172.21.15.32:0/3071822636 shutdown_connections 2020-12-05T17:39:27.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:27.471+0000 7f0c8c38a700 1 -- 172.21.15.32:0/3071822636 wait complete. 2020-12-05T17:39:27.477 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 42 2020-12-05T17:39:27.689 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:39:22.261725+0000', 'modified': '2020-12-05T17:39:23.265348+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6930}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6930': {'gid': 6930, 'name': 'a', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/4087622496', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 4087622496}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 4087622496}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:39:27.689 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:39:27.690 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:39:27.821 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":6930,"want_state":"up:active","state":"up:active","rank_uptime":5.5444511040000002,"mdsmap_epoch":42,"osdmap_epoch":92,"osdmap_epoch_barrier":92,"uptime":10.747995619999999} 2020-12-05T17:39:27.821 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 6930, "mdsmap_epoch": 42, "osdmap_epoch": 92, "osdmap_epoch_barrier": 92, "rank_uptime": 5.544451104, "state": "up:active", "uptime": 10.74799562, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:39:27.822 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:39:27.823 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-12-05T17:39:27.823 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-12-05T17:39:27.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:27.824 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:27.833 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:39:27.891 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:39:27.892 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:39:27.946 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:39:27.948 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:39:27.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:27.949 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:39:27.998 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:39:27.998 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:28.042 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:39:28.060 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:39:28.060 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:39:28.115 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:39:28.117 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:39:28.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:28.118 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:39:28.120 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:2020-12-05T17:39:28.115+0000 7f8298b9f0c0 -1 init, newargv = 0x5558c14e2200 newargc=9ceph-fuse[ 2020-12-05T17:39:28.121 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:11252]: starting ceph client 2020-12-05T17:39:28.130 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[11252]: starting fuse 2020-12-05T17:39:29.228 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:39:29.244 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:39:29.245 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:39:29.302 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:39:29.303 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:39:29.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:29.305 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:39:29.350 INFO:teuthology.orchestra.run.smithi036.stdout:49 2020-12-05T17:39:29.350 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-12-05T17:39:29.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:29.351 INFO:teuthology.orchestra.run.smithi036:> sudo python3 -c ' 2020-12-05T17:39:29.352 INFO:teuthology.orchestra.run.smithi036:> import glob 2020-12-05T17:39:29.352 INFO:teuthology.orchestra.run.smithi036:> import re 2020-12-05T17:39:29.352 INFO:teuthology.orchestra.run.smithi036:> import os 2020-12-05T17:39:29.353 INFO:teuthology.orchestra.run.smithi036:> import subprocess 2020-12-05T17:39:29.353 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:39:29.354 INFO:teuthology.orchestra.run.smithi036:> def find_socket(client_name): 2020-12-05T17:39:29.354 INFO:teuthology.orchestra.run.smithi036:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-12-05T17:39:29.355 INFO:teuthology.orchestra.run.smithi036:> files = glob.glob(asok_path) 2020-12-05T17:39:29.355 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:39:29.355 INFO:teuthology.orchestra.run.smithi036:> # Given a non-glob path, it better be there 2020-12-05T17:39:29.356 INFO:teuthology.orchestra.run.smithi036:> if "*" not in asok_path: 2020-12-05T17:39:29.356 INFO:teuthology.orchestra.run.smithi036:> assert(len(files) == 1) 2020-12-05T17:39:29.357 INFO:teuthology.orchestra.run.smithi036:> return files[0] 2020-12-05T17:39:29.357 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:39:29.357 INFO:teuthology.orchestra.run.smithi036:> for f in files: 2020-12-05T17:39:29.358 INFO:teuthology.orchestra.run.smithi036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:39:29.358 INFO:teuthology.orchestra.run.smithi036:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:39:29.359 INFO:teuthology.orchestra.run.smithi036:> return f 2020-12-05T17:39:29.359 INFO:teuthology.orchestra.run.smithi036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:39:29.359 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:39:29.360 INFO:teuthology.orchestra.run.smithi036:> print(find_socket("client.0")) 2020-12-05T17:39:29.360 INFO:teuthology.orchestra.run.smithi036:> ' 2020-12-05T17:39:29.442 INFO:teuthology.orchestra.run.smithi036.stdout:/var/run/ceph/ceph-client.0.11252.asok 2020-12-05T17:39:29.448 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.11252.asok 2020-12-05T17:39:29.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:29.449 INFO:teuthology.orchestra.run.smithi036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.11252.asok status 2020-12-05T17:39:29.550 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2020-12-05T17:39:29.550 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": { 2020-12-05T17:39:29.550 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:39:29.551 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:39:29.551 INFO:teuthology.orchestra.run.smithi036.stdout: "entity_id": "0", 2020-12-05T17:39:29.551 INFO:teuthology.orchestra.run.smithi036.stdout: "hostname": "smithi036", 2020-12-05T17:39:29.551 INFO:teuthology.orchestra.run.smithi036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-12-05T17:39:29.551 INFO:teuthology.orchestra.run.smithi036.stdout: "pid": "11252", 2020-12-05T17:39:29.552 INFO:teuthology.orchestra.run.smithi036.stdout: "root": "/" 2020-12-05T17:39:29.552 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:39:29.552 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_count": 0, 2020-12-05T17:39:29.552 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_pinned_count": 0, 2020-12-05T17:39:29.552 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 6245, 2020-12-05T17:39:29.553 INFO:teuthology.orchestra.run.smithi036.stdout: "inst": { 2020-12-05T17:39:29.553 INFO:teuthology.orchestra.run.smithi036.stdout: "name": { 2020-12-05T17:39:29.553 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "client", 2020-12-05T17:39:29.553 INFO:teuthology.orchestra.run.smithi036.stdout: "num": 6245 2020-12-05T17:39:29.553 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:39:29.554 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:39:29.554 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:39:29.554 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:39:29.554 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 2984897014 2020-12-05T17:39:29.555 INFO:teuthology.orchestra.run.smithi036.stdout: } 2020-12-05T17:39:29.555 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:39:29.555 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:39:29.555 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:39:29.555 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:39:29.556 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 2984897014 2020-12-05T17:39:29.556 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:39:29.556 INFO:teuthology.orchestra.run.smithi036.stdout: "inst_str": "client.6245 172.21.15.36:0/2984897014", 2020-12-05T17:39:29.556 INFO:teuthology.orchestra.run.smithi036.stdout: "addr_str": "172.21.15.36:0/2984897014", 2020-12-05T17:39:29.556 INFO:teuthology.orchestra.run.smithi036.stdout: "inode_count": 1, 2020-12-05T17:39:29.557 INFO:teuthology.orchestra.run.smithi036.stdout: "mds_epoch": 42, 2020-12-05T17:39:29.557 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch": 92, 2020-12-05T17:39:29.557 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:39:29.557 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklisted": false 2020-12-05T17:39:29.558 INFO:teuthology.orchestra.run.smithi036.stdout:} 2020-12-05T17:39:29.560 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:29.561 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:29.561 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:29.568 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:39:29.569 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:39:29.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:29.570 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:29.628 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:39:29.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd49cc23700 1 Processor -- start 2020-12-05T17:39:29.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd49cc23700 1 -- start start 2020-12-05T17:39:29.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd49cc23700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd49805f6f0 0x7fd498145e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:29.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd49cc23700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd4980afd90 0x7fd4980ade00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:29.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd49cc23700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4980b34a0 0x7fd4980ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:29.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd49cc23700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4980b17a0 con 0x7fd49805f6f0 2020-12-05T17:39:29.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd49cc23700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4980ae7e0 con 0x7fd4980afd90 2020-12-05T17:39:29.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd49cc23700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4980ad9d0 con 0x7fd4980b34a0 2020-12-05T17:39:29.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd4977fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd49805f6f0 0x7fd498145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:29.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd4977fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd49805f6f0 0x7fd498145e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38254/0 (socket says 172.21.15.32:38254) 2020-12-05T17:39:29.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd4977fe700 1 -- 172.21.15.32:0/3602933618 learned_addr learned my addr 172.21.15.32:0/3602933618 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:29.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd497fff700 1 --2- 172.21.15.32:0/3602933618 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4980b34a0 0x7fd4980ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:29.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd496ffd700 1 --2- 172.21.15.32:0/3602933618 >> v2:172.21.15.61:3300/0 conn(0x7fd4980afd90 0x7fd4980ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:29.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd4977fe700 1 -- 172.21.15.32:0/3602933618 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4980b34a0 msgr2=0x7fd4980ae2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:29.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd4977fe700 1 --2- 172.21.15.32:0/3602933618 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4980b34a0 0x7fd4980ae2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:29.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd4977fe700 1 -- 172.21.15.32:0/3602933618 >> v2:172.21.15.61:3300/0 conn(0x7fd4980afd90 msgr2=0x7fd4980ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:29.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd4977fe700 1 --2- 172.21.15.32:0/3602933618 >> v2:172.21.15.61:3300/0 conn(0x7fd4980afd90 0x7fd4980ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:29.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd4977fe700 1 -- 172.21.15.32:0/3602933618 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49805fad0 con 0x7fd49805f6f0 2020-12-05T17:39:29.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd4977fe700 1 --2- 172.21.15.32:0/3602933618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd49805f6f0 0x7fd498145e40 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7fd498142920 tx=0x7fd490009270).ready entity=mon.0 client_cookie=c981f324beff553c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:29.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd4967fc700 1 -- 172.21.15.32:0/3602933618 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd49000bb40 con 0x7fd49805f6f0 2020-12-05T17:39:29.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd4967fc700 1 -- 172.21.15.32:0/3602933618 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49000bca0 con 0x7fd49805f6f0 2020-12-05T17:39:29.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd4967fc700 1 -- 172.21.15.32:0/3602933618 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd49000be10 con 0x7fd49805f6f0 2020-12-05T17:39:29.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd49cc23700 1 -- 172.21.15.32:0/3602933618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd49805f6f0 msgr2=0x7fd498145e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:29.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.743+0000 7fd49cc23700 1 --2- 172.21.15.32:0/3602933618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd49805f6f0 0x7fd498145e40 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7fd498142920 tx=0x7fd490009270).stop 2020-12-05T17:39:29.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd49cc23700 1 -- 172.21.15.32:0/3602933618 shutdown_connections 2020-12-05T17:39:29.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd49cc23700 1 --2- 172.21.15.32:0/3602933618 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4980b34a0 0x7fd4980ae2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:29.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd49cc23700 1 --2- 172.21.15.32:0/3602933618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd49805f6f0 0x7fd498145e40 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:29.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd49cc23700 1 --2- 172.21.15.32:0/3602933618 >> v2:172.21.15.61:3300/0 conn(0x7fd4980afd90 0x7fd4980ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:29.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd49cc23700 1 -- 172.21.15.32:0/3602933618 >> 172.21.15.32:0/3602933618 conn(0x7fd4980604c0 msgr2=0x7fd49805c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:29.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd49cc23700 1 -- 172.21.15.32:0/3602933618 shutdown_connections 2020-12-05T17:39:29.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd49cc23700 1 -- 172.21.15.32:0/3602933618 wait complete. 2020-12-05T17:39:29.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd49cc23700 1 Processor -- start 2020-12-05T17:39:29.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd49cc23700 1 -- start start 2020-12-05T17:39:29.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd49cc23700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd49805f6f0 0x7fd4981487f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:29.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd49cc23700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4980afd90 0x7fd498146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:29.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd49cc23700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4980b34a0 0x7fd498065570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:29.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd49cc23700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd498145690 con 0x7fd49805f6f0 2020-12-05T17:39:29.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd49cc23700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd498143d40 con 0x7fd4980afd90 2020-12-05T17:39:29.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd49cc23700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4981467a0 con 0x7fd4980b34a0 2020-12-05T17:39:29.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd4977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd49805f6f0 0x7fd4981487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:29.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd4977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd49805f6f0 0x7fd4981487f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38262/0 (socket says 172.21.15.32:38262) 2020-12-05T17:39:29.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd4977fe700 1 -- 172.21.15.32:0/400552122 learned_addr learned my addr 172.21.15.32:0/400552122 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:29.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd497fff700 1 --2- 172.21.15.32:0/400552122 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4980b34a0 0x7fd498065570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:29.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd496ffd700 1 --2- 172.21.15.32:0/400552122 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4980afd90 0x7fd498146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:29.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd4977fe700 1 -- 172.21.15.32:0/400552122 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4980b34a0 msgr2=0x7fd498065570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:29.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd4977fe700 1 --2- 172.21.15.32:0/400552122 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4980b34a0 0x7fd498065570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:29.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd4977fe700 1 -- 172.21.15.32:0/400552122 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4980afd90 msgr2=0x7fd498146170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:29.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd4977fe700 1 --2- 172.21.15.32:0/400552122 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4980afd90 0x7fd498146170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:29.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd4977fe700 1 -- 172.21.15.32:0/400552122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd498146a40 con 0x7fd49805f6f0 2020-12-05T17:39:29.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd496ffd700 1 --2- 172.21.15.32:0/400552122 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4980afd90 0x7fd498146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:29.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd4977fe700 1 --2- 172.21.15.32:0/400552122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd49805f6f0 0x7fd4981487f0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7fd498142920 tx=0x7fd490016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:29.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd494ff9700 1 -- 172.21.15.32:0/400552122 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd490022070 con 0x7fd49805f6f0 2020-12-05T17:39:29.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd494ff9700 1 -- 172.21.15.32:0/400552122 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49001e610 con 0x7fd49805f6f0 2020-12-05T17:39:29.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd49cc23700 1 -- 172.21.15.32:0/400552122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd498146ce0 con 0x7fd49805f6f0 2020-12-05T17:39:29.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd494ff9700 1 -- 172.21.15.32:0/400552122 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd49001e970 con 0x7fd49805f6f0 2020-12-05T17:39:29.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.747+0000 7fd49cc23700 1 -- 172.21.15.32:0/400552122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd49813de50 con 0x7fd49805f6f0 2020-12-05T17:39:29.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.751+0000 7fd49cc23700 1 -- 172.21.15.32:0/400552122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd498057050 con 0x7fd49805f6f0 2020-12-05T17:39:29.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.751+0000 7fd494ff9700 1 -- 172.21.15.32:0/400552122 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd49000f070 con 0x7fd49805f6f0 2020-12-05T17:39:29.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.751+0000 7fd494ff9700 1 --2- 172.21.15.32:0/400552122 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd47c055470 0x7fd47c057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:29.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.751+0000 7fd496ffd700 1 --2- 172.21.15.32:0/400552122 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd47c055470 0x7fd47c057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:29.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.751+0000 7fd494ff9700 1 -- 172.21.15.32:0/400552122 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd49006ee50 con 0x7fd49805f6f0 2020-12-05T17:39:29.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.751+0000 7fd496ffd700 1 --2- 172.21.15.32:0/400552122 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd47c055470 0x7fd47c057930 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7fd48000c000 tx=0x7fd480008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:29.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.755+0000 7fd494ff9700 1 -- 172.21.15.32:0/400552122 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd490044ff0 con 0x7fd49805f6f0 2020-12-05T17:39:29.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.903+0000 7fd49cc23700 1 -- 172.21.15.32:0/400552122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4980b02a0 con 0x7fd49805f6f0 2020-12-05T17:39:29.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.907+0000 7fd494ff9700 1 -- 172.21.15.32:0/400552122 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16274 (secure 0 0 0) 0x7fd490015070 con 0x7fd49805f6f0 2020-12-05T17:39:29.912 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:29.913 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":92,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:39:23.264356+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:39:19.993416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2020-12-05T17:39:21.252765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:39:29.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.907+0000 7fd49cc23700 1 -- 172.21.15.32:0/400552122 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd47c055470 msgr2=0x7fd47c057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:29.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.907+0000 7fd49cc23700 1 --2- 172.21.15.32:0/400552122 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd47c055470 0x7fd47c057930 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7fd48000c000 tx=0x7fd480008040).stop 2020-12-05T17:39:29.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.907+0000 7fd49cc23700 1 -- 172.21.15.32:0/400552122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd49805f6f0 msgr2=0x7fd4981487f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:29.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.907+0000 7fd49cc23700 1 --2- 172.21.15.32:0/400552122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd49805f6f0 0x7fd4981487f0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7fd498142920 tx=0x7fd490016000).stop 2020-12-05T17:39:29.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.907+0000 7fd49cc23700 1 -- 172.21.15.32:0/400552122 shutdown_connections 2020-12-05T17:39:29.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.907+0000 7fd49cc23700 1 --2- 172.21.15.32:0/400552122 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd47c055470 0x7fd47c057930 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:29.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.907+0000 7fd49cc23700 1 --2- 172.21.15.32:0/400552122 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4980b34a0 0x7fd498065570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:29.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.907+0000 7fd49cc23700 1 --2- 172.21.15.32:0/400552122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd49805f6f0 0x7fd4981487f0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:29.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.907+0000 7fd49cc23700 1 --2- 172.21.15.32:0/400552122 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4980afd90 0x7fd498146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:29.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.907+0000 7fd49cc23700 1 -- 172.21.15.32:0/400552122 >> 172.21.15.32:0/400552122 conn(0x7fd4980604c0 msgr2=0x7fd4980add20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:29.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.907+0000 7fd49cc23700 1 -- 172.21.15.32:0/400552122 shutdown_connections 2020-12-05T17:39:29.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:29.907+0000 7fd49cc23700 1 -- 172.21.15.32:0/400552122 wait complete. 2020-12-05T17:39:29.927 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:39:19.993416+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:39:21.252765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:39:29.928 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:39:30.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.035+0000 7f5ce4e73700 1 Processor -- start 2020-12-05T17:39:30.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.035+0000 7f5ce4e73700 1 -- start start 2020-12-05T17:39:30.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5ce4e73700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5ce0066c70 0x7f5ce00af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5ce4e73700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce00672a0 con 0x7f5ce00b29d0 2020-12-05T17:39:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5ce4e73700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce00b2db0 con 0x7f5ce00b1730 2020-12-05T17:39:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5ce4e73700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce013f6a0 con 0x7f5ce0066c70 2020-12-05T17:39:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cde59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5ce00b29d0 0x7f5ce00acb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40916/0 (socket says 172.21.15.32:40916) 2020-12-05T17:39:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cde59c700 1 -- 172.21.15.32:0/4276548053 learned_addr learned my addr 172.21.15.32:0/4276548053 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cded9d700 1 --2- 172.21.15.32:0/4276548053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5ce0066c70 0x7f5ce00af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cdd59a700 1 -- 172.21.15.32:0/4276548053 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992665135 0 0) 0x7f5ce00672a0 con 0x7f5ce00b29d0 2020-12-05T17:39:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cdd59a700 1 -- 172.21.15.32:0/4276548053 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cc8003230 con 0x7f5ce00b29d0 2020-12-05T17:39:30.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cdd59a700 1 -- 172.21.15.32:0/4276548053 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3932933140 0 0) 0x7f5ce00b2db0 con 0x7f5ce00b1730 2020-12-05T17:39:30.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cdd59a700 1 -- 172.21.15.32:0/4276548053 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ce00672a0 con 0x7f5ce00b1730 2020-12-05T17:39:30.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cdd59a700 1 -- 172.21.15.32:0/4276548053 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 999620517 0 0) 0x7f5cc8003230 con 0x7f5ce00b29d0 2020-12-05T17:39:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cdd59a700 1 -- 172.21.15.32:0/4276548053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5ce0066c70 msgr2=0x7f5ce00af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cdd59a700 1 --2- 172.21.15.32:0/4276548053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5ce0066c70 0x7f5ce00af9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cdd59a700 1 -- 172.21.15.32:0/4276548053 >> v1:172.21.15.61:6789/0 conn(0x7f5ce00b1730 legacy=0x7f5ce00af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cdd59a700 1 -- 172.21.15.32:0/4276548053 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce013f940 con 0x7f5ce00b29d0 2020-12-05T17:39:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cdd59a700 1 -- 172.21.15.32:0/4276548053 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f5cd00016d0 con 0x7f5ce00b29d0 2020-12-05T17:39:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cdd59a700 1 -- 172.21.15.32:0/4276548053 >> v1:172.21.15.32:6789/0 conn(0x7f5ce00b29d0 legacy=0x7f5ce00acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cdd59a700 1 --2- 172.21.15.32:0/4276548053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5ce0066c70 0x7f5cc80038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cdd59a700 1 --2- 172.21.15.32:0/4276548053 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cc8003dd0 0x7f5cc80044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cdd59a700 1 --2- 172.21.15.32:0/4276548053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cc80049c0 0x7f5cc8004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cdd59a700 1 -- 172.21.15.32:0/4276548053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce013f6a0 con 0x7f5cc80049c0 2020-12-05T17:39:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cdd59a700 1 -- 172.21.15.32:0/4276548053 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc8005360 con 0x7f5cc8003dd0 2020-12-05T17:39:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cdd59a700 1 -- 172.21.15.32:0/4276548053 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc800d980 con 0x7f5ce0066c70 2020-12-05T17:39:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cded9d700 1 --2- 172.21.15.32:0/4276548053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cc80049c0 0x7f5cc8004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cde59c700 1 --2- 172.21.15.32:0/4276548053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5ce0066c70 0x7f5cc80038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cddd9b700 1 --2- 172.21.15.32:0/4276548053 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cc8003dd0 0x7f5cc80044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cded9d700 1 -- 172.21.15.32:0/4276548053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5ce0066c70 msgr2=0x7f5cc80038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cded9d700 1 --2- 172.21.15.32:0/4276548053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5ce0066c70 0x7f5cc80038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cded9d700 1 -- 172.21.15.32:0/4276548053 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cc8003dd0 msgr2=0x7f5cc80044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cded9d700 1 --2- 172.21.15.32:0/4276548053 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cc8003dd0 0x7f5cc80044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cded9d700 1 -- 172.21.15.32:0/4276548053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc8003230 con 0x7f5cc80049c0 2020-12-05T17:39:30.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cded9d700 1 --2- 172.21.15.32:0/4276548053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cc80049c0 0x7f5cc8004e40 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7f5cc80054a0 tx=0x7f5cd80091a0).ready entity=mon.0 client_cookie=e3ce81ef063297e5 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:30.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cdd59a700 1 -- 172.21.15.32:0/4276548053 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cd800eb80 con 0x7f5cc80049c0 2020-12-05T17:39:30.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cdd59a700 1 -- 172.21.15.32:0/4276548053 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd800ece0 con 0x7f5cc80049c0 2020-12-05T17:39:30.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cdd59a700 1 -- 172.21.15.32:0/4276548053 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cd80123f0 con 0x7f5cc80049c0 2020-12-05T17:39:30.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5ce4e73700 1 -- 172.21.15.32:0/4276548053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cc80049c0 msgr2=0x7f5cc8004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:30.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5ce4e73700 1 --2- 172.21.15.32:0/4276548053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cc80049c0 0x7f5cc8004e40 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7f5cc80054a0 tx=0x7f5cd80091a0).stop 2020-12-05T17:39:30.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.039+0000 7f5cded9d700 1 -- 172.21.15.32:0/4276548053 reap_dead start 2020-12-05T17:39:30.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.043+0000 7f5ce4e73700 1 -- 172.21.15.32:0/4276548053 shutdown_connections 2020-12-05T17:39:30.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.043+0000 7f5ce4e73700 1 -- 172.21.15.32:0/4276548053 >> 172.21.15.32:0/4276548053 conn(0x7f5ce0062310 msgr2=0x7f5ce00626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:30.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.043+0000 7f5ce4e73700 1 -- 172.21.15.32:0/4276548053 shutdown_connections 2020-12-05T17:39:30.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.043+0000 7f5ce4e73700 1 -- 172.21.15.32:0/4276548053 wait complete. 2020-12-05T17:39:30.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.043+0000 7f5ce4e73700 1 Processor -- start 2020-12-05T17:39:30.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.043+0000 7f5ce4e73700 1 -- start start 2020-12-05T17:39:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.043+0000 7f5ce4e73700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce0063bd0 0x7f5ce0063fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.043+0000 7f5ce4e73700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5ce014df60 0x7f5ce0065ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.043+0000 7f5ce4e73700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5ce0065fd0 0x7f5ce0150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.043+0000 7f5ce4e73700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce013a3f0 con 0x7f5ce0063bd0 2020-12-05T17:39:30.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.043+0000 7f5ce4e73700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce00644a0 con 0x7f5ce0065fd0 2020-12-05T17:39:30.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.043+0000 7f5ce4e73700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce013a620 con 0x7f5ce014df60 2020-12-05T17:39:30.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.043+0000 7f5cde59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce0063bd0 0x7f5ce0063fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.043+0000 7f5cde59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce0063bd0 0x7f5ce0063fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38280/0 (socket says 172.21.15.32:38280) 2020-12-05T17:39:30.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.047+0000 7f5cde59c700 1 -- 172.21.15.32:0/960324084 learned_addr learned my addr 172.21.15.32:0/960324084 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:30.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.047+0000 7f5cded9d700 1 --2- 172.21.15.32:0/960324084 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5ce0065fd0 0x7f5ce0150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.047+0000 7f5cddd9b700 1 --2- 172.21.15.32:0/960324084 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5ce014df60 0x7f5ce0065ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.047+0000 7f5cde59c700 1 -- 172.21.15.32:0/960324084 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5ce014df60 msgr2=0x7f5ce0065ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:30.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.047+0000 7f5cde59c700 1 --2- 172.21.15.32:0/960324084 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5ce014df60 0x7f5ce0065ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.047+0000 7f5cde59c700 1 -- 172.21.15.32:0/960324084 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5ce0065fd0 msgr2=0x7f5ce0150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:30.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.047+0000 7f5cde59c700 1 --2- 172.21.15.32:0/960324084 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5ce0065fd0 0x7f5ce0150a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.047+0000 7f5cde59c700 1 -- 172.21.15.32:0/960324084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce013a0a0 con 0x7f5ce0063bd0 2020-12-05T17:39:30.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.047+0000 7f5cde59c700 1 --2- 172.21.15.32:0/960324084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce0063bd0 0x7f5ce0063fb0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7f5cd000d000 tx=0x7f5ce013f590).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:30.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.047+0000 7f5cd77fe700 1 -- 172.21.15.32:0/960324084 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cd000f9b0 con 0x7f5ce0063bd0 2020-12-05T17:39:30.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.047+0000 7f5cd77fe700 1 -- 172.21.15.32:0/960324084 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd000fb10 con 0x7f5ce0063bd0 2020-12-05T17:39:30.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.047+0000 7f5ce4e73700 1 -- 172.21.15.32:0/960324084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce014e400 con 0x7f5ce0063bd0 2020-12-05T17:39:30.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.047+0000 7f5cd77fe700 1 -- 172.21.15.32:0/960324084 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cd000fe70 con 0x7f5ce0063bd0 2020-12-05T17:39:30.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.047+0000 7f5ce4e73700 1 -- 172.21.15.32:0/960324084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce00666a0 con 0x7f5ce0063bd0 2020-12-05T17:39:30.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.047+0000 7f5cd77fe700 1 -- 172.21.15.32:0/960324084 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5cd001b030 con 0x7f5ce0063bd0 2020-12-05T17:39:30.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.047+0000 7f5ce4e73700 1 -- 172.21.15.32:0/960324084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce0057050 con 0x7f5ce0063bd0 2020-12-05T17:39:30.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.047+0000 7f5cd77fe700 1 --2- 172.21.15.32:0/960324084 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5cbc055440 0x7f5cbc057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.047+0000 7f5cd77fe700 1 -- 172.21.15.32:0/960324084 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5cd006db90 con 0x7f5ce0063bd0 2020-12-05T17:39:30.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.047+0000 7f5cddd9b700 1 --2- 172.21.15.32:0/960324084 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5cbc055440 0x7f5cbc057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.051+0000 7f5cddd9b700 1 --2- 172.21.15.32:0/960324084 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5cbc055440 0x7f5cbc057900 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f5ccc00c000 tx=0x7f5ccc007b40).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:30.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.055+0000 7f5cd77fe700 1 -- 172.21.15.32:0/960324084 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5cd0043c20 con 0x7f5ce0063bd0 2020-12-05T17:39:30.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.199+0000 7f5ce4e73700 1 -- 172.21.15.32:0/960324084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f5ce013a7e0 con 0x7f5ce0063bd0 2020-12-05T17:39:30.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.199+0000 7f5cd77fe700 1 -- 172.21.15.32:0/960324084 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v92) v1 ==== 110+0+10 (secure 0 0 0) 0x7f5cd0019030 con 0x7f5ce0063bd0 2020-12-05T17:39:30.204 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:39:30.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.199+0000 7f5ce4e73700 1 -- 172.21.15.32:0/960324084 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5cbc055440 msgr2=0x7f5cbc057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:30.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.199+0000 7f5ce4e73700 1 --2- 172.21.15.32:0/960324084 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5cbc055440 0x7f5cbc057900 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f5ccc00c000 tx=0x7f5ccc007b40).stop 2020-12-05T17:39:30.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.199+0000 7f5ce4e73700 1 -- 172.21.15.32:0/960324084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce0063bd0 msgr2=0x7f5ce0063fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.199+0000 7f5ce4e73700 1 --2- 172.21.15.32:0/960324084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce0063bd0 0x7f5ce0063fb0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7f5cd000d000 tx=0x7f5ce013f590).stop 2020-12-05T17:39:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.199+0000 7f5ce4e73700 1 -- 172.21.15.32:0/960324084 shutdown_connections 2020-12-05T17:39:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.199+0000 7f5ce4e73700 1 --2- 172.21.15.32:0/960324084 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5ce0065fd0 0x7f5ce0150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.199+0000 7f5ce4e73700 1 --2- 172.21.15.32:0/960324084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce0063bd0 0x7f5ce0063fb0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.199+0000 7f5ce4e73700 1 --2- 172.21.15.32:0/960324084 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5cbc055440 0x7f5cbc057900 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.199+0000 7f5ce4e73700 1 --2- 172.21.15.32:0/960324084 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5ce014df60 0x7f5ce0065ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.199+0000 7f5ce4e73700 1 -- 172.21.15.32:0/960324084 >> 172.21.15.32:0/960324084 conn(0x7f5ce0062310 msgr2=0x7f5ce00b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.199+0000 7f5ce4e73700 1 -- 172.21.15.32:0/960324084 shutdown_connections 2020-12-05T17:39:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.199+0000 7f5ce4e73700 1 -- 172.21.15.32:0/960324084 wait complete. 2020-12-05T17:39:30.218 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:39:30.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.331+0000 7f115ebfa700 1 Processor -- start 2020-12-05T17:39:30.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.331+0000 7f115ebfa700 1 -- start start 2020-12-05T17:39:30.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115ebfa700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f115805f3b0 0x7f115805f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115ebfa700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11580b2fb0 0x7f1158061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115ebfa700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11580b2df0 con 0x7f115805feb0 2020-12-05T17:39:30.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115ebfa700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11580b3c90 con 0x7f115805f3b0 2020-12-05T17:39:30.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115ebfa700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11580b2b50 con 0x7f11580b2fb0 2020-12-05T17:39:30.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115dbf8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f115805f3b0 0x7f115805f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115dbf8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f115805f3b0 0x7f115805f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38288/0 (socket says 172.21.15.32:38288) 2020-12-05T17:39:30.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115dbf8700 1 -- 172.21.15.32:0/1621748058 learned_addr learned my addr 172.21.15.32:0/1621748058 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:30.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115e3f9700 1 --2- 172.21.15.32:0/1621748058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11580b2fb0 0x7f1158061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115dbf8700 1 -- 172.21.15.32:0/1621748058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11580b2fb0 msgr2=0x7f1158061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:30.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115dbf8700 1 --2- 172.21.15.32:0/1621748058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11580b2fb0 0x7f1158061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115dbf8700 1 -- 172.21.15.32:0/1621748058 >> v1:172.21.15.61:6789/0 conn(0x7f115805feb0 legacy=0x7f115813c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:30.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115dbf8700 1 -- 172.21.15.32:0/1621748058 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1158142820 con 0x7f115805f3b0 2020-12-05T17:39:30.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115dbf8700 1 --2- 172.21.15.32:0/1621748058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f115805f3b0 0x7f115805f790 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0x7f1154002f30 tx=0x7f1154006e80).ready entity=mon.0 client_cookie=eb63e78daa621afd server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:30.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115cbf6700 1 -- 172.21.15.32:0/1621748058 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11540097f0 con 0x7f115805f3b0 2020-12-05T17:39:30.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115cbf6700 1 -- 172.21.15.32:0/1621748058 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1154009950 con 0x7f115805f3b0 2020-12-05T17:39:30.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115ebfa700 1 -- 172.21.15.32:0/1621748058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f115805f3b0 msgr2=0x7f115805f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:30.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115cbf6700 1 -- 172.21.15.32:0/1621748058 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1154009cb0 con 0x7f115805f3b0 2020-12-05T17:39:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115ebfa700 1 --2- 172.21.15.32:0/1621748058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f115805f3b0 0x7f115805f790 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0x7f1154002f30 tx=0x7f1154006e80).stop 2020-12-05T17:39:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115ebfa700 1 -- 172.21.15.32:0/1621748058 shutdown_connections 2020-12-05T17:39:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115ebfa700 1 --2- 172.21.15.32:0/1621748058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11580b2fb0 0x7f1158061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115ebfa700 1 --2- 172.21.15.32:0/1621748058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f115805f3b0 0x7f115805f790 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115ebfa700 1 -- 172.21.15.32:0/1621748058 >> 172.21.15.32:0/1621748058 conn(0x7f11580b8f20 msgr2=0x7f11580b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115ebfa700 1 -- 172.21.15.32:0/1621748058 shutdown_connections 2020-12-05T17:39:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.335+0000 7f115ebfa700 1 -- 172.21.15.32:0/1621748058 wait complete. 2020-12-05T17:39:30.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.339+0000 7f115ebfa700 1 Processor -- start 2020-12-05T17:39:30.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.339+0000 7f115ebfa700 1 -- start start 2020-12-05T17:39:30.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.339+0000 7f115ebfa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f115805f3b0 0x7f1158145fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.339+0000 7f115ebfa700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f115805feb0 0x7f1158154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.339+0000 7f115ebfa700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11580b2fb0 0x7f115813cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.339+0000 7f115ebfa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115813f440 con 0x7f115805f3b0 2020-12-05T17:39:30.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.339+0000 7f115ebfa700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1158064f50 con 0x7f115805feb0 2020-12-05T17:39:30.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.339+0000 7f115ebfa700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11580651f0 con 0x7f11580b2fb0 2020-12-05T17:39:30.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.339+0000 7f115dbf8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f115805f3b0 0x7f1158145fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.339+0000 7f115dbf8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f115805f3b0 0x7f1158145fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38294/0 (socket says 172.21.15.32:38294) 2020-12-05T17:39:30.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.339+0000 7f115dbf8700 1 -- 172.21.15.32:0/2900450472 learned_addr learned my addr 172.21.15.32:0/2900450472 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:30.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.339+0000 7f115d3f7700 1 --2- 172.21.15.32:0/2900450472 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f115805feb0 0x7f1158154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.339+0000 7f115e3f9700 1 --2- 172.21.15.32:0/2900450472 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11580b2fb0 0x7f115813cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.339+0000 7f115dbf8700 1 -- 172.21.15.32:0/2900450472 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11580b2fb0 msgr2=0x7f115813cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:30.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.339+0000 7f115dbf8700 1 --2- 172.21.15.32:0/2900450472 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11580b2fb0 0x7f115813cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.339+0000 7f115dbf8700 1 -- 172.21.15.32:0/2900450472 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f115805feb0 msgr2=0x7f1158154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:30.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.339+0000 7f115dbf8700 1 --2- 172.21.15.32:0/2900450472 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f115805feb0 0x7f1158154d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.339+0000 7f115dbf8700 1 -- 172.21.15.32:0/2900450472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1158155200 con 0x7f115805f3b0 2020-12-05T17:39:30.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.339+0000 7f115dbf8700 1 --2- 172.21.15.32:0/2900450472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f115805f3b0 0x7f1158145fc0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7f1154002f30 tx=0x7f1154015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:30.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.339+0000 7f1146ffd700 1 -- 172.21.15.32:0/2900450472 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1154011b00 con 0x7f115805f3b0 2020-12-05T17:39:30.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.339+0000 7f1146ffd700 1 -- 172.21.15.32:0/2900450472 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1154011c60 con 0x7f115805f3b0 2020-12-05T17:39:30.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.339+0000 7f1146ffd700 1 -- 172.21.15.32:0/2900450472 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f115401e460 con 0x7f115805f3b0 2020-12-05T17:39:30.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.343+0000 7f115ebfa700 1 -- 172.21.15.32:0/2900450472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1158066710 con 0x7f115805f3b0 2020-12-05T17:39:30.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.343+0000 7f115ebfa700 1 -- 172.21.15.32:0/2900450472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115813f130 con 0x7f115805f3b0 2020-12-05T17:39:30.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.343+0000 7f115ebfa700 1 -- 172.21.15.32:0/2900450472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f115813d5e0 con 0x7f115805f3b0 2020-12-05T17:39:30.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.343+0000 7f1146ffd700 1 -- 172.21.15.32:0/2900450472 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1154025030 con 0x7f115805f3b0 2020-12-05T17:39:30.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.343+0000 7f1146ffd700 1 --2- 172.21.15.32:0/2900450472 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1138055380 0x7f1138057840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.343+0000 7f1146ffd700 1 -- 172.21.15.32:0/2900450472 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f115406f330 con 0x7f115805f3b0 2020-12-05T17:39:30.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.343+0000 7f115d3f7700 1 --2- 172.21.15.32:0/2900450472 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1138055380 0x7f1138057840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.343+0000 7f115d3f7700 1 --2- 172.21.15.32:0/2900450472 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1138055380 0x7f1138057840 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f114800d000 tx=0x7f1148000b80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:30.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.351+0000 7f1146ffd700 1 -- 172.21.15.32:0/2900450472 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f115813d5e0 con 0x7f115805f3b0 2020-12-05T17:39:30.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.495+0000 7f115ebfa700 1 -- 172.21.15.32:0/2900450472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f115813d5e0 con 0x7f115805f3b0 2020-12-05T17:39:30.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.495+0000 7f1146ffd700 1 -- 172.21.15.32:0/2900450472 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v92) v1 ==== 92+0+10 (secure 0 0 0) 0x7f115813d5e0 con 0x7f115805f3b0 2020-12-05T17:39:30.500 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:39:30.503 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.495+0000 7f115ebfa700 1 -- 172.21.15.32:0/2900450472 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1138055380 msgr2=0x7f1138057840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:30.503 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.495+0000 7f115ebfa700 1 --2- 172.21.15.32:0/2900450472 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1138055380 0x7f1138057840 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f114800d000 tx=0x7f1148000b80).stop 2020-12-05T17:39:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.495+0000 7f115ebfa700 1 -- 172.21.15.32:0/2900450472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f115805f3b0 msgr2=0x7f1158145fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.495+0000 7f115ebfa700 1 --2- 172.21.15.32:0/2900450472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f115805f3b0 0x7f1158145fc0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7f1154002f30 tx=0x7f1154015000).stop 2020-12-05T17:39:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.499+0000 7f115ebfa700 1 -- 172.21.15.32:0/2900450472 shutdown_connections 2020-12-05T17:39:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.499+0000 7f115ebfa700 1 --2- 172.21.15.32:0/2900450472 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1138055380 0x7f1138057840 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.499+0000 7f115ebfa700 1 --2- 172.21.15.32:0/2900450472 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11580b2fb0 0x7f115813cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.499+0000 7f115ebfa700 1 --2- 172.21.15.32:0/2900450472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f115805f3b0 0x7f1158145fc0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.499+0000 7f115ebfa700 1 --2- 172.21.15.32:0/2900450472 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f115805feb0 0x7f1158154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.499+0000 7f115ebfa700 1 -- 172.21.15.32:0/2900450472 >> 172.21.15.32:0/2900450472 conn(0x7f11580b8f20 msgr2=0x7f11580b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:30.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.499+0000 7f115ebfa700 1 -- 172.21.15.32:0/2900450472 shutdown_connections 2020-12-05T17:39:30.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.499+0000 7f115ebfa700 1 -- 172.21.15.32:0/2900450472 wait complete. 2020-12-05T17:39:30.519 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:39:30.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.631+0000 7f25863b8700 1 Processor -- start 2020-12-05T17:39:30.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.631+0000 7f25863b8700 1 -- start start 2020-12-05T17:39:30.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f25863b8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f258005feb0 0x7f258005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f25863b8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f25800b2fb0 0x7f2580061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f25863b8700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25800b2df0 con 0x7f258005f3b0 2020-12-05T17:39:30.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f25863b8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25800b3c90 con 0x7f25800b2fb0 2020-12-05T17:39:30.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f25863b8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25800b2b50 con 0x7f258005feb0 2020-12-05T17:39:30.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f2585bb7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f25800b2fb0 0x7f2580061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f2585bb7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f25800b2fb0 0x7f2580061780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38306/0 (socket says 172.21.15.32:38306) 2020-12-05T17:39:30.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f2585bb7700 1 -- 172.21.15.32:0/3463129669 learned_addr learned my addr 172.21.15.32:0/3463129669 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:30.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f2584bb5700 1 --2- 172.21.15.32:0/3463129669 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f258005feb0 0x7f258005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f2585bb7700 1 -- 172.21.15.32:0/3463129669 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f258005feb0 msgr2=0x7f258005f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:30.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f2585bb7700 1 --2- 172.21.15.32:0/3463129669 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f258005feb0 0x7f258005f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f2585bb7700 1 -- 172.21.15.32:0/3463129669 >> v1:172.21.15.61:6789/0 conn(0x7f258005f3b0 legacy=0x7f258013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:30.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f2585bb7700 1 -- 172.21.15.32:0/3463129669 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2580142820 con 0x7f25800b2fb0 2020-12-05T17:39:30.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f2585bb7700 1 --2- 172.21.15.32:0/3463129669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25800b2fb0 0x7f2580061780 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f25780032b0 tx=0x7f2578009270).ready entity=mon.0 client_cookie=96c1ac0e2f098de0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:30.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f2573fff700 1 -- 172.21.15.32:0/3463129669 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2578009e60 con 0x7f25800b2fb0 2020-12-05T17:39:30.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f2573fff700 1 -- 172.21.15.32:0/3463129669 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257800bc00 con 0x7f25800b2fb0 2020-12-05T17:39:30.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f2573fff700 1 -- 172.21.15.32:0/3463129669 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f257800bd70 con 0x7f25800b2fb0 2020-12-05T17:39:30.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f25863b8700 1 -- 172.21.15.32:0/3463129669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25800b2fb0 msgr2=0x7f2580061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:30.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f25863b8700 1 --2- 172.21.15.32:0/3463129669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25800b2fb0 0x7f2580061780 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f25780032b0 tx=0x7f2578009270).stop 2020-12-05T17:39:30.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f25863b8700 1 -- 172.21.15.32:0/3463129669 shutdown_connections 2020-12-05T17:39:30.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f25863b8700 1 --2- 172.21.15.32:0/3463129669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25800b2fb0 0x7f2580061780 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f25863b8700 1 --2- 172.21.15.32:0/3463129669 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f258005feb0 0x7f258005f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f25863b8700 1 -- 172.21.15.32:0/3463129669 >> 172.21.15.32:0/3463129669 conn(0x7f25800b8f20 msgr2=0x7f25800b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:30.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f25863b8700 1 -- 172.21.15.32:0/3463129669 shutdown_connections 2020-12-05T17:39:30.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.635+0000 7f25863b8700 1 -- 172.21.15.32:0/3463129669 wait complete. 2020-12-05T17:39:30.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f25863b8700 1 Processor -- start 2020-12-05T17:39:30.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f25863b8700 1 -- start start 2020-12-05T17:39:30.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f25863b8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f258005f3b0 0x7f2580145e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f25863b8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f258005feb0 0x7f25800663b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f25863b8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25800b2fb0 0x7f258013ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f25863b8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25800668a0 con 0x7f25800b2fb0 2020-12-05T17:39:30.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f25863b8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2580066af0 con 0x7f258005feb0 2020-12-05T17:39:30.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f25863b8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2580151550 con 0x7f258005f3b0 2020-12-05T17:39:30.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f2585bb7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25800b2fb0 0x7f258013ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f2585bb7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25800b2fb0 0x7f258013ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38312/0 (socket says 172.21.15.32:38312) 2020-12-05T17:39:30.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f2585bb7700 1 -- 172.21.15.32:0/2750196210 learned_addr learned my addr 172.21.15.32:0/2750196210 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:30.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f25853b6700 1 --2- 172.21.15.32:0/2750196210 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f258005f3b0 0x7f2580145e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f2584bb5700 1 --2- 172.21.15.32:0/2750196210 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f258005feb0 0x7f25800663b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f2585bb7700 1 -- 172.21.15.32:0/2750196210 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f258005f3b0 msgr2=0x7f2580145e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:30.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f2585bb7700 1 --2- 172.21.15.32:0/2750196210 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f258005f3b0 0x7f2580145e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f2585bb7700 1 -- 172.21.15.32:0/2750196210 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f258005feb0 msgr2=0x7f25800663b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:30.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f2585bb7700 1 --2- 172.21.15.32:0/2750196210 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f258005feb0 0x7f25800663b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f2585bb7700 1 -- 172.21.15.32:0/2750196210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2580063580 con 0x7f25800b2fb0 2020-12-05T17:39:30.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f2585bb7700 1 --2- 172.21.15.32:0/2750196210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25800b2fb0 0x7f258013ba90 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0x7f25780032b0 tx=0x7f2578016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:30.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f25727fc700 1 -- 172.21.15.32:0/2750196210 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f257801f5e0 con 0x7f25800b2fb0 2020-12-05T17:39:30.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f25727fc700 1 -- 172.21.15.32:0/2750196210 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257801f740 con 0x7f25800b2fb0 2020-12-05T17:39:30.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f25863b8700 1 -- 172.21.15.32:0/2750196210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25800654e0 con 0x7f25800b2fb0 2020-12-05T17:39:30.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f25727fc700 1 -- 172.21.15.32:0/2750196210 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f257801faa0 con 0x7f25800b2fb0 2020-12-05T17:39:30.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.639+0000 7f25863b8700 1 -- 172.21.15.32:0/2750196210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258013bf80 con 0x7f25800b2fb0 2020-12-05T17:39:30.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.643+0000 7f25727fc700 1 -- 172.21.15.32:0/2750196210 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2578026030 con 0x7f25800b2fb0 2020-12-05T17:39:30.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.643+0000 7f25863b8700 1 -- 172.21.15.32:0/2750196210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2580151710 con 0x7f25800b2fb0 2020-12-05T17:39:30.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.643+0000 7f25727fc700 1 --2- 172.21.15.32:0/2750196210 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2568055330 0x7f25680577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.643+0000 7f25727fc700 1 -- 172.21.15.32:0/2750196210 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f257806f000 con 0x7f25800b2fb0 2020-12-05T17:39:30.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.643+0000 7f25853b6700 1 --2- 172.21.15.32:0/2750196210 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2568055330 0x7f25680577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.643+0000 7f25853b6700 1 --2- 172.21.15.32:0/2750196210 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2568055330 0x7f25680577f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f257c00d000 tx=0x7f257c007b60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:30.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.651+0000 7f25727fc700 1 -- 172.21.15.32:0/2750196210 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f25780451a0 con 0x7f25800b2fb0 2020-12-05T17:39:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.795+0000 7f25863b8700 1 -- 172.21.15.32:0/2750196210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2580057050 con 0x7f25800b2fb0 2020-12-05T17:39:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.795+0000 7f25727fc700 1 -- 172.21.15.32:0/2750196210 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+10 (secure 0 0 0) 0x7f25780487c0 con 0x7f25800b2fb0 2020-12-05T17:39:30.800 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:39:30.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.795+0000 7f25863b8700 1 -- 172.21.15.32:0/2750196210 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2568055330 msgr2=0x7f25680577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:30.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.795+0000 7f25863b8700 1 --2- 172.21.15.32:0/2750196210 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2568055330 0x7f25680577f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f257c00d000 tx=0x7f257c007b60).stop 2020-12-05T17:39:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.795+0000 7f25863b8700 1 -- 172.21.15.32:0/2750196210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25800b2fb0 msgr2=0x7f258013ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.795+0000 7f25863b8700 1 --2- 172.21.15.32:0/2750196210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25800b2fb0 0x7f258013ba90 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0x7f25780032b0 tx=0x7f2578016000).stop 2020-12-05T17:39:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.795+0000 7f25863b8700 1 -- 172.21.15.32:0/2750196210 shutdown_connections 2020-12-05T17:39:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.795+0000 7f25863b8700 1 --2- 172.21.15.32:0/2750196210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25800b2fb0 0x7f258013ba90 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.795+0000 7f25863b8700 1 --2- 172.21.15.32:0/2750196210 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2568055330 0x7f25680577f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.795+0000 7f25863b8700 1 --2- 172.21.15.32:0/2750196210 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f258005f3b0 0x7f2580145e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.795+0000 7f25863b8700 1 --2- 172.21.15.32:0/2750196210 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f258005feb0 0x7f25800663b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.795+0000 7f25863b8700 1 -- 172.21.15.32:0/2750196210 >> 172.21.15.32:0/2750196210 conn(0x7f25800b8f20 msgr2=0x7f25800b6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:30.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.795+0000 7f25863b8700 1 -- 172.21.15.32:0/2750196210 shutdown_connections 2020-12-05T17:39:30.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.795+0000 7f25863b8700 1 -- 172.21.15.32:0/2750196210 wait complete. 2020-12-05T17:39:30.817 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:39:30.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.931+0000 7f821759e700 1 Processor -- start 2020-12-05T17:39:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.931+0000 7f821759e700 1 -- start start 2020-12-05T17:39:30.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.931+0000 7f821759e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82181396c0 0x7f82180af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.931+0000 7f821759e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f8218062ac0 0x7f821813f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.931+0000 7f821759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8218143390 con 0x7f82180b1e60 2020-12-05T17:39:30.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.931+0000 7f821759e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82180ae0a0 con 0x7f8218062ac0 2020-12-05T17:39:30.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.931+0000 7f821759e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8218143610 con 0x7f82181396c0 2020-12-05T17:39:30.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.931+0000 7f821659c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f82180b1e60 0x7f8218138fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40964/0 (socket says 172.21.15.32:40964) 2020-12-05T17:39:30.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.931+0000 7f821659c700 1 -- 172.21.15.32:0/3944263247 learned_addr learned my addr 172.21.15.32:0/3944263247 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:30.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.935+0000 7f8215d9b700 1 --2- 172.21.15.32:0/3944263247 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82181396c0 0x7f82180af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.935+0000 7f8216d9d700 1 --2- 172.21.15.32:0/3944263247 >> v2:172.21.15.61:3300/0 conn(0x7f8218062ac0 0x7f821813f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.935+0000 7f821559a700 1 -- 172.21.15.32:0/3944263247 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1010608490 0 0) 0x7f8218143390 con 0x7f82180b1e60 2020-12-05T17:39:30.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.935+0000 7f821559a700 1 -- 172.21.15.32:0/3944263247 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81fc003230 con 0x7f82180b1e60 2020-12-05T17:39:30.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.935+0000 7f8215d9b700 1 -- 172.21.15.32:0/3944263247 >> v2:172.21.15.61:3300/0 conn(0x7f8218062ac0 msgr2=0x7f821813f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:30.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.935+0000 7f8215d9b700 1 --2- 172.21.15.32:0/3944263247 >> v2:172.21.15.61:3300/0 conn(0x7f8218062ac0 0x7f821813f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.935+0000 7f8215d9b700 1 -- 172.21.15.32:0/3944263247 >> v1:172.21.15.32:6789/0 conn(0x7f82180b1e60 legacy=0x7f8218138fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:30.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.935+0000 7f8215d9b700 1 -- 172.21.15.32:0/3944263247 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82181438b0 con 0x7f82181396c0 2020-12-05T17:39:30.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.935+0000 7f8216d9d700 1 --2- 172.21.15.32:0/3944263247 >> v2:172.21.15.61:3300/0 conn(0x7f8218062ac0 0x7f821813f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:30.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.935+0000 7f8215d9b700 1 --2- 172.21.15.32:0/3944263247 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82181396c0 0x7f82180af4b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f820c002f30 tx=0x7f820c006dc0).ready entity=mon.2 client_cookie=d27fa0c2282b267c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:30.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.935+0000 7f821559a700 1 -- 172.21.15.32:0/3944263247 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f820c0096a0 con 0x7f82181396c0 2020-12-05T17:39:30.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.935+0000 7f821559a700 1 -- 172.21.15.32:0/3944263247 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820c009800 con 0x7f82181396c0 2020-12-05T17:39:30.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.935+0000 7f821559a700 1 -- 172.21.15.32:0/3944263247 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f820c009b60 con 0x7f82181396c0 2020-12-05T17:39:30.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.935+0000 7f821759e700 1 -- 172.21.15.32:0/3944263247 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82181396c0 msgr2=0x7f82180af4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:30.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.935+0000 7f821759e700 1 --2- 172.21.15.32:0/3944263247 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82181396c0 0x7f82180af4b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f820c002f30 tx=0x7f820c006dc0).stop 2020-12-05T17:39:30.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.935+0000 7f821759e700 1 -- 172.21.15.32:0/3944263247 shutdown_connections 2020-12-05T17:39:30.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.935+0000 7f821759e700 1 --2- 172.21.15.32:0/3944263247 >> v2:172.21.15.61:3300/0 conn(0x7f8218062ac0 0x7f821813f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.935+0000 7f821759e700 1 --2- 172.21.15.32:0/3944263247 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82181396c0 0x7f82180af4b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.935+0000 7f821759e700 1 -- 172.21.15.32:0/3944263247 >> 172.21.15.32:0/3944263247 conn(0x7f82180b5620 msgr2=0x7f82180b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:30.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.935+0000 7f821759e700 1 -- 172.21.15.32:0/3944263247 shutdown_connections 2020-12-05T17:39:30.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.935+0000 7f821759e700 1 -- 172.21.15.32:0/3944263247 wait complete. 2020-12-05T17:39:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f821759e700 1 Processor -- start 2020-12-05T17:39:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f821759e700 1 -- start start 2020-12-05T17:39:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f821759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8218062ac0 0x7f821813c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f821759e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f82180b1e60 0x7f821813dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f821759e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82181396c0 0x7f821814f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f821759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821812e630 con 0x7f8218062ac0 2020-12-05T17:39:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f821759e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821813a9a0 con 0x7f82180b1e60 2020-12-05T17:39:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f821759e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821813bea0 con 0x7f82181396c0 2020-12-05T17:39:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f821659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8218062ac0 0x7f821813c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f821659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8218062ac0 0x7f821813c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38322/0 (socket says 172.21.15.32:38322) 2020-12-05T17:39:30.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f821659c700 1 -- 172.21.15.32:0/2228725084 learned_addr learned my addr 172.21.15.32:0/2228725084 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:30.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f8215d9b700 1 --2- 172.21.15.32:0/2228725084 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f82180b1e60 0x7f821813dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f8216d9d700 1 --2- 172.21.15.32:0/2228725084 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82181396c0 0x7f821814f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f821659c700 1 -- 172.21.15.32:0/2228725084 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82181396c0 msgr2=0x7f821814f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:30.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f821659c700 1 --2- 172.21.15.32:0/2228725084 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82181396c0 0x7f821814f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f821659c700 1 -- 172.21.15.32:0/2228725084 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f82180b1e60 msgr2=0x7f821813dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:30.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f821659c700 1 --2- 172.21.15.32:0/2228725084 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f82180b1e60 0x7f821813dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:30.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f821659c700 1 -- 172.21.15.32:0/2228725084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821813c140 con 0x7f8218062ac0 2020-12-05T17:39:30.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f821659c700 1 --2- 172.21.15.32:0/2228725084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8218062ac0 0x7f821813c5e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f82080010d0 tx=0x7f8208007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:30.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f82077fe700 1 -- 172.21.15.32:0/2228725084 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82080072e0 con 0x7f8218062ac0 2020-12-05T17:39:30.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f82077fe700 1 -- 172.21.15.32:0/2228725084 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8208007440 con 0x7f8218062ac0 2020-12-05T17:39:30.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f82077fe700 1 -- 172.21.15.32:0/2228725084 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82080057d0 con 0x7f8218062ac0 2020-12-05T17:39:30.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f821759e700 1 -- 172.21.15.32:0/2228725084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f821813cad0 con 0x7f8218062ac0 2020-12-05T17:39:30.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.939+0000 7f821759e700 1 -- 172.21.15.32:0/2228725084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f821813e120 con 0x7f8218062ac0 2020-12-05T17:39:30.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.943+0000 7f821759e700 1 -- 172.21.15.32:0/2228725084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f821813ede0 con 0x7f8218062ac0 2020-12-05T17:39:30.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.943+0000 7f82077fe700 1 -- 172.21.15.32:0/2228725084 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8208010070 con 0x7f8218062ac0 2020-12-05T17:39:30.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.943+0000 7f82077fe700 1 --2- 172.21.15.32:0/2228725084 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f81f4055440 0x7f81f4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:30.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.943+0000 7f8215d9b700 1 --2- 172.21.15.32:0/2228725084 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f81f4055440 0x7f81f4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:30.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.943+0000 7f82077fe700 1 -- 172.21.15.32:0/2228725084 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f820806da70 con 0x7f8218062ac0 2020-12-05T17:39:30.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.947+0000 7f8215d9b700 1 --2- 172.21.15.32:0/2228725084 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f81f4055440 0x7f81f4057900 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f820c01b000 tx=0x7f820c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:30.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:30.951+0000 7f82077fe700 1 -- 172.21.15.32:0/2228725084 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8208043c10 con 0x7f8218062ac0 2020-12-05T17:39:31.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.091+0000 7f821759e700 1 -- 172.21.15.32:0/2228725084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f821813ede0 con 0x7f8218062ac0 2020-12-05T17:39:31.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.091+0000 7f82077fe700 1 -- 172.21.15.32:0/2228725084 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+10 (secure 0 0 0) 0x7f821813ede0 con 0x7f8218062ac0 2020-12-05T17:39:31.097 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:39:31.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.091+0000 7f821759e700 1 -- 172.21.15.32:0/2228725084 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f81f4055440 msgr2=0x7f81f4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:31.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.091+0000 7f821759e700 1 --2- 172.21.15.32:0/2228725084 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f81f4055440 0x7f81f4057900 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f820c01b000 tx=0x7f820c015000).stop 2020-12-05T17:39:31.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.091+0000 7f821759e700 1 -- 172.21.15.32:0/2228725084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8218062ac0 msgr2=0x7f821813c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:31.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.091+0000 7f821759e700 1 --2- 172.21.15.32:0/2228725084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8218062ac0 0x7f821813c5e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f82080010d0 tx=0x7f8208007b50).stop 2020-12-05T17:39:31.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.091+0000 7f821759e700 1 -- 172.21.15.32:0/2228725084 shutdown_connections 2020-12-05T17:39:31.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.091+0000 7f821759e700 1 --2- 172.21.15.32:0/2228725084 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f81f4055440 0x7f81f4057900 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:31.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.091+0000 7f821759e700 1 --2- 172.21.15.32:0/2228725084 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82181396c0 0x7f821814f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.091+0000 7f821759e700 1 --2- 172.21.15.32:0/2228725084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8218062ac0 0x7f821813c5e0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.095+0000 7f821759e700 1 --2- 172.21.15.32:0/2228725084 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f82180b1e60 0x7f821813dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.095+0000 7f821759e700 1 -- 172.21.15.32:0/2228725084 >> 172.21.15.32:0/2228725084 conn(0x7f82180b5620 msgr2=0x7f82180af4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.095+0000 7f821759e700 1 -- 172.21.15.32:0/2228725084 shutdown_connections 2020-12-05T17:39:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.095+0000 7f821759e700 1 -- 172.21.15.32:0/2228725084 wait complete. 2020-12-05T17:39:31.110 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:39:31.111 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:39:31.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.215+0000 7fc7ec5b0700 1 Processor -- start 2020-12-05T17:39:31.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.215+0000 7fc7ec5b0700 1 -- start start 2020-12-05T17:39:31.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.215+0000 7fc7ec5b0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7e405cd70 0x7fc7e4143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:31.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.215+0000 7fc7ec5b0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7e40633c0 con 0x7fc7e40af870 2020-12-05T17:39:31.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.215+0000 7fc7ec5b0700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7e405d150 con 0x7fc7e405f7f0 2020-12-05T17:39:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.215+0000 7fc7ec5b0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7e405fe10 con 0x7fc7e405cd70 2020-12-05T17:39:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.215+0000 7fc7ea34c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7e405cd70 0x7fc7e4143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.215+0000 7fc7ea34c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7e405cd70 0x7fc7e4143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38330/0 (socket says 172.21.15.32:38330) 2020-12-05T17:39:31.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.215+0000 7fc7ea34c700 1 -- 172.21.15.32:0/3422275878 learned_addr learned my addr 172.21.15.32:0/3422275878 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:31.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.215+0000 7fc7eab4d700 1 --1- 172.21.15.32:0/3422275878 >> v1:172.21.15.32:6789/0 conn(0x7fc7e40af870 0x7fc7e413d330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40982/0 (socket says 172.21.15.32:40982) 2020-12-05T17:39:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.215+0000 7fc7ea34c700 1 -- 172.21.15.32:0/3422275878 >> v1:172.21.15.61:6789/0 conn(0x7fc7e405f7f0 legacy=0x7fc7e40af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.215+0000 7fc7ea34c700 1 -- 172.21.15.32:0/3422275878 >> v1:172.21.15.32:6789/0 conn(0x7fc7e40af870 legacy=0x7fc7e413d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.215+0000 7fc7ea34c700 1 -- 172.21.15.32:0/3422275878 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e40600b0 con 0x7fc7e405cd70 2020-12-05T17:39:31.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.215+0000 7fc7ea34c700 1 --2- 172.21.15.32:0/3422275878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7e405cd70 0x7fc7e4143d30 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7fc7e00032b0 tx=0x7fc7e0009270).ready entity=mon.0 client_cookie=8e7e2ffec30b78c5 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:31.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.215+0000 7fc7e934a700 1 -- 172.21.15.32:0/3422275878 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7e000bc00 con 0x7fc7e405cd70 2020-12-05T17:39:31.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.215+0000 7fc7e934a700 1 -- 172.21.15.32:0/3422275878 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e000bd60 con 0x7fc7e405cd70 2020-12-05T17:39:31.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.215+0000 7fc7e934a700 1 -- 172.21.15.32:0/3422275878 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7e0011400 con 0x7fc7e405cd70 2020-12-05T17:39:31.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.215+0000 7fc7ec5b0700 1 -- 172.21.15.32:0/3422275878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7e405cd70 msgr2=0x7fc7e4143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:31.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.215+0000 7fc7ec5b0700 1 --2- 172.21.15.32:0/3422275878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7e405cd70 0x7fc7e4143d30 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7fc7e00032b0 tx=0x7fc7e0009270).stop 2020-12-05T17:39:31.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7ec5b0700 1 -- 172.21.15.32:0/3422275878 shutdown_connections 2020-12-05T17:39:31.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7ec5b0700 1 --2- 172.21.15.32:0/3422275878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7e405cd70 0x7fc7e4143d30 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:31.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7ec5b0700 1 -- 172.21.15.32:0/3422275878 >> 172.21.15.32:0/3422275878 conn(0x7fc7e4069cb0 msgr2=0x7fc7e40604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:31.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7ec5b0700 1 -- 172.21.15.32:0/3422275878 shutdown_connections 2020-12-05T17:39:31.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7ec5b0700 1 -- 172.21.15.32:0/3422275878 wait complete. 2020-12-05T17:39:31.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7ec5b0700 1 Processor -- start 2020-12-05T17:39:31.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7ec5b0700 1 -- start start 2020-12-05T17:39:31.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7ec5b0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc7e405cd70 0x7fc7e40b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:31.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7ec5b0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7e405f7f0 0x7fc7e40ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:31.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7ec5b0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc7e40af870 0x7fc7e40ad790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:31.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7ec5b0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e40ace70 con 0x7fc7e405f7f0 2020-12-05T17:39:31.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7ec5b0700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e4063610 con 0x7fc7e40af870 2020-12-05T17:39:31.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7ec5b0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e40638b0 con 0x7fc7e405cd70 2020-12-05T17:39:31.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7e9b4b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7e405f7f0 0x7fc7e40ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:31.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7e9b4b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7e405f7f0 0x7fc7e40ad2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38336/0 (socket says 172.21.15.32:38336) 2020-12-05T17:39:31.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7e9b4b700 1 -- 172.21.15.32:0/316201883 learned_addr learned my addr 172.21.15.32:0/316201883 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:31.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7ea34c700 1 --2- 172.21.15.32:0/316201883 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc7e405cd70 0x7fc7e40b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:31.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7eab4d700 1 --2- 172.21.15.32:0/316201883 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc7e40af870 0x7fc7e40ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:31.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7e9b4b700 1 -- 172.21.15.32:0/316201883 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc7e405cd70 msgr2=0x7fc7e40b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:31.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7e9b4b700 1 --2- 172.21.15.32:0/316201883 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc7e405cd70 0x7fc7e40b5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:31.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7e9b4b700 1 -- 172.21.15.32:0/316201883 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc7e40af870 msgr2=0x7fc7e40ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:31.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7e9b4b700 1 --2- 172.21.15.32:0/316201883 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc7e40af870 0x7fc7e40ad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:31.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7e9b4b700 1 -- 172.21.15.32:0/316201883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e4063b50 con 0x7fc7e405f7f0 2020-12-05T17:39:31.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7ea34c700 1 --2- 172.21.15.32:0/316201883 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc7e405cd70 0x7fc7e40b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:31.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7eab4d700 1 --2- 172.21.15.32:0/316201883 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc7e40af870 0x7fc7e40ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:31.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.219+0000 7fc7e9b4b700 1 --2- 172.21.15.32:0/316201883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7e405f7f0 0x7fc7e40ad2a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7fc7e405d410 tx=0x7fc7d4007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:31.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.223+0000 7fc7d37fe700 1 -- 172.21.15.32:0/316201883 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7d400e4a0 con 0x7fc7e405f7f0 2020-12-05T17:39:31.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.223+0000 7fc7ec5b0700 1 -- 172.21.15.32:0/316201883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7e4151920 con 0x7fc7e405f7f0 2020-12-05T17:39:31.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.223+0000 7fc7d37fe700 1 -- 172.21.15.32:0/316201883 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d400e600 con 0x7fc7e405f7f0 2020-12-05T17:39:31.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.223+0000 7fc7d37fe700 1 -- 172.21.15.32:0/316201883 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7d400e960 con 0x7fc7e405f7f0 2020-12-05T17:39:31.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.223+0000 7fc7ec5b0700 1 -- 172.21.15.32:0/316201883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e40ade20 con 0x7fc7e405f7f0 2020-12-05T17:39:31.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.223+0000 7fc7ec5b0700 1 -- 172.21.15.32:0/316201883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7e40649f0 con 0x7fc7e405f7f0 2020-12-05T17:39:31.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.223+0000 7fc7d37fe700 1 -- 172.21.15.32:0/316201883 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc7d4019030 con 0x7fc7e405f7f0 2020-12-05T17:39:31.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.223+0000 7fc7d37fe700 1 --2- 172.21.15.32:0/316201883 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc7c4055440 0x7fc7c4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:31.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.223+0000 7fc7d37fe700 1 -- 172.21.15.32:0/316201883 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc7d406d760 con 0x7fc7e405f7f0 2020-12-05T17:39:31.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.223+0000 7fc7ea34c700 1 --2- 172.21.15.32:0/316201883 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc7c4055440 0x7fc7c4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:31.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.223+0000 7fc7ea34c700 1 --2- 172.21.15.32:0/316201883 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc7c4055440 0x7fc7c4057900 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7fc7e0018000 tx=0x7fc7e0016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:31.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.227+0000 7fc7d37fe700 1 -- 172.21.15.32:0/316201883 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc7d4043860 con 0x7fc7e405f7f0 2020-12-05T17:39:31.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.411+0000 7fc7ec5b0700 1 -- 172.21.15.32:0/316201883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7e4063ce0 con 0x7fc7e405f7f0 2020-12-05T17:39:31.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.415+0000 7fc7d37fe700 1 -- 172.21.15.32:0/316201883 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+2140 (secure 0 0 0) 0x7fc7d4046e80 con 0x7fc7e405f7f0 2020-12-05T17:39:31.420 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:31.420 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":42,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6823,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2664846066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2664846066},{"type":"v1","addr":"172.21.15.61:6833","nonce":2664846066}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:39:22.261725+0000","modified":"2020-12-05T17:39:23.265348+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6930":{"gid":6930,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4087622496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4087622496},{"type":"v1","addr":"172.21.15.32:6835","nonce":4087622496}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2020-12-05T17:39:31.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.415+0000 7fc7ec5b0700 1 -- 172.21.15.32:0/316201883 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc7c4055440 msgr2=0x7fc7c4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:31.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.415+0000 7fc7ec5b0700 1 --2- 172.21.15.32:0/316201883 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc7c4055440 0x7fc7c4057900 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7fc7e0018000 tx=0x7fc7e0016000).stop 2020-12-05T17:39:31.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.415+0000 7fc7ec5b0700 1 -- 172.21.15.32:0/316201883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7e405f7f0 msgr2=0x7fc7e40ad2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:31.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.415+0000 7fc7ec5b0700 1 --2- 172.21.15.32:0/316201883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7e405f7f0 0x7fc7e40ad2a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7fc7e405d410 tx=0x7fc7d4007c20).stop 2020-12-05T17:39:31.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.415+0000 7fc7ec5b0700 1 -- 172.21.15.32:0/316201883 shutdown_connections 2020-12-05T17:39:31.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.415+0000 7fc7ec5b0700 1 --2- 172.21.15.32:0/316201883 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc7e40af870 0x7fc7e40ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:31.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.415+0000 7fc7ec5b0700 1 --2- 172.21.15.32:0/316201883 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc7c4055440 0x7fc7c4057900 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:31.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.415+0000 7fc7ec5b0700 1 --2- 172.21.15.32:0/316201883 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc7e405cd70 0x7fc7e40b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:31.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.415+0000 7fc7ec5b0700 1 --2- 172.21.15.32:0/316201883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7e405f7f0 0x7fc7e40ad2a0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:31.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.415+0000 7fc7ec5b0700 1 -- 172.21.15.32:0/316201883 >> 172.21.15.32:0/316201883 conn(0x7fc7e4069cb0 msgr2=0x7fc7e40b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:31.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.415+0000 7fc7ec5b0700 1 -- 172.21.15.32:0/316201883 shutdown_connections 2020-12-05T17:39:31.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:31.415+0000 7fc7ec5b0700 1 -- 172.21.15.32:0/316201883 wait complete. 2020-12-05T17:39:31.424 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 42 2020-12-05T17:39:31.635 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:39:22.261725+0000', 'modified': '2020-12-05T17:39:23.265348+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6930}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6930': {'gid': 6930, 'name': 'a', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/4087622496', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 4087622496}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 4087622496}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:39:31.636 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:39:31.637 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:39:31.766 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":6930,"want_state":"up:active","state":"up:active","rank_uptime":9.4901629649999997,"mdsmap_epoch":42,"osdmap_epoch":92,"osdmap_epoch_barrier":92,"uptime":14.693710574000001} 2020-12-05T17:39:31.767 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 6930, "mdsmap_epoch": 42, "osdmap_epoch": 92, "osdmap_epoch_barrier": 92, "rank_uptime": 9.490162965, "state": "up:active", "uptime": 14.693710574, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:39:31.767 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:39:31.768 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-12-05T17:39:31.768 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi002.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-12-05T17:39:31.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:31.769 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:31.777 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:39:31.833 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:39:31.834 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:39:31.889 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:39:31.890 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:39:31.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:31.891 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:39:31.940 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:39:31.941 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:31.984 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:39:31.999 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:39:31.999 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:39:32.057 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:39:32.058 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:39:32.059 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:32.060 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:39:32.065 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:2020-12-05T17:39:32.059+0000 7f0d1b8b90c0 -1 init, newargv = 0x561c8f356530 newargc=9 2020-12-05T17:39:32.065 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[10740]: starting ceph client 2020-12-05T17:39:32.076 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[10740]: starting fuse 2020-12-05T17:39:33.171 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:39:33.195 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:39:33.196 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:39:33.262 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:39:33.263 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:39:33.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:33.264 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:39:33.325 INFO:teuthology.orchestra.run.smithi002.stdout:50 2020-12-05T17:39:33.327 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [50] 2020-12-05T17:39:33.328 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:33.328 INFO:teuthology.orchestra.run.smithi002:> sudo python3 -c ' 2020-12-05T17:39:33.329 INFO:teuthology.orchestra.run.smithi002:> import glob 2020-12-05T17:39:33.329 INFO:teuthology.orchestra.run.smithi002:> import re 2020-12-05T17:39:33.330 INFO:teuthology.orchestra.run.smithi002:> import os 2020-12-05T17:39:33.330 INFO:teuthology.orchestra.run.smithi002:> import subprocess 2020-12-05T17:39:33.330 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:39:33.331 INFO:teuthology.orchestra.run.smithi002:> def find_socket(client_name): 2020-12-05T17:39:33.331 INFO:teuthology.orchestra.run.smithi002:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-12-05T17:39:33.331 INFO:teuthology.orchestra.run.smithi002:> files = glob.glob(asok_path) 2020-12-05T17:39:33.332 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:39:33.332 INFO:teuthology.orchestra.run.smithi002:> # Given a non-glob path, it better be there 2020-12-05T17:39:33.333 INFO:teuthology.orchestra.run.smithi002:> if "*" not in asok_path: 2020-12-05T17:39:33.333 INFO:teuthology.orchestra.run.smithi002:> assert(len(files) == 1) 2020-12-05T17:39:33.333 INFO:teuthology.orchestra.run.smithi002:> return files[0] 2020-12-05T17:39:33.334 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:39:33.334 INFO:teuthology.orchestra.run.smithi002:> for f in files: 2020-12-05T17:39:33.335 INFO:teuthology.orchestra.run.smithi002:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:39:33.335 INFO:teuthology.orchestra.run.smithi002:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:39:33.335 INFO:teuthology.orchestra.run.smithi002:> return f 2020-12-05T17:39:33.336 INFO:teuthology.orchestra.run.smithi002:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:39:33.336 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:39:33.337 INFO:teuthology.orchestra.run.smithi002:> print(find_socket("client.1")) 2020-12-05T17:39:33.337 INFO:teuthology.orchestra.run.smithi002:> ' 2020-12-05T17:39:33.421 INFO:teuthology.orchestra.run.smithi002.stdout:/var/run/ceph/ceph-client.1.10740.asok 2020-12-05T17:39:33.427 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.10740.asok 2020-12-05T17:39:33.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:33.428 INFO:teuthology.orchestra.run.smithi002:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.10740.asok status 2020-12-05T17:39:33.580 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2020-12-05T17:39:33.581 INFO:teuthology.orchestra.run.smithi002.stdout: "metadata": { 2020-12-05T17:39:33.581 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:39:33.582 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:39:33.582 INFO:teuthology.orchestra.run.smithi002.stdout: "entity_id": "1", 2020-12-05T17:39:33.583 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": "smithi002", 2020-12-05T17:39:33.583 INFO:teuthology.orchestra.run.smithi002.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-12-05T17:39:33.584 INFO:teuthology.orchestra.run.smithi002.stdout: "pid": "10740", 2020-12-05T17:39:33.585 INFO:teuthology.orchestra.run.smithi002.stdout: "root": "/" 2020-12-05T17:39:33.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:39:33.585 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_count": 0, 2020-12-05T17:39:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_pinned_count": 0, 2020-12-05T17:39:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 7131, 2020-12-05T17:39:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: "inst": { 2020-12-05T17:39:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: "name": { 2020-12-05T17:39:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "client", 2020-12-05T17:39:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: "num": 7131 2020-12-05T17:39:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:39:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2020-12-05T17:39:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2020-12-05T17:39:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2020-12-05T17:39:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1671039485 2020-12-05T17:39:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: } 2020-12-05T17:39:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:39:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2020-12-05T17:39:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2020-12-05T17:39:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2020-12-05T17:39:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1671039485 2020-12-05T17:39:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:39:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: "inst_str": "client.7131 172.21.15.2:0/1671039485", 2020-12-05T17:39:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: "addr_str": "172.21.15.2:0/1671039485", 2020-12-05T17:39:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: "inode_count": 1, 2020-12-05T17:39:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_epoch": 42, 2020-12-05T17:39:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch": 92, 2020-12-05T17:39:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:39:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: "blacklisted": false 2020-12-05T17:39:33.591 INFO:teuthology.orchestra.run.smithi002.stdout:} 2020-12-05T17:39:33.592 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:33.592 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:33.593 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:33.603 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:39:33.604 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:39:33.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:33.605 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:33.664 INFO:teuthology.orchestra.run.smithi036:> cd /home/ubuntu/cephtest/mnt.0 && sudo mkdir -p foo 2020-12-05T17:39:33.697 INFO:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-12-05T17:39:33.697 INFO:teuthology.orchestra.run.smithi036:> import os 2020-12-05T17:39:33.698 INFO:teuthology.orchestra.run.smithi036:> import stat 2020-12-05T17:39:33.698 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:39:33.698 INFO:teuthology.orchestra.run.smithi036:> print(os.stat("/home/ubuntu/cephtest/mnt.0/foo").st_ino) 2020-12-05T17:39:33.699 INFO:teuthology.orchestra.run.smithi036:> ' 2020-12-05T17:39:33.827 INFO:teuthology.orchestra.run.smithi036.stdout:1099511627776 2020-12-05T17:39:34.014 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:39:34.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.127+0000 7f4364d39700 1 Processor -- start 2020-12-05T17:39:34.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.127+0000 7f4364d39700 1 -- start start 2020-12-05T17:39:34.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.127+0000 7f4364d39700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f43600acb80 0x7f43600acf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:34.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.127+0000 7f4364d39700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f436005fdc0 0x7f436013fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:34.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.127+0000 7f4364d39700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43600affa0 con 0x7f4360067480 2020-12-05T17:39:34.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.127+0000 7f4364d39700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43600ad6a0 con 0x7f43600acb80 2020-12-05T17:39:34.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.127+0000 7f4364d39700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4360067a00 con 0x7f436005fdc0 2020-12-05T17:39:34.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.127+0000 7f435e59c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f43600acb80 0x7f43600acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:34.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.127+0000 7f435dd9b700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f4360067480 0x7f4360142800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:53632/0 (socket says 172.21.15.32:53632) 2020-12-05T17:39:34.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.127+0000 7f435dd9b700 1 -- 172.21.15.32:0/2131161412 learned_addr learned my addr 172.21.15.32:0/2131161412 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:34.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.127+0000 7f435ed9d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f436005fdc0 0x7f436013fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:34.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.127+0000 7f435e59c700 1 -- 172.21.15.32:0/2131161412 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f436005fdc0 msgr2=0x7f436013fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:34.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.127+0000 7f435e59c700 1 --2- 172.21.15.32:0/2131161412 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f436005fdc0 0x7f436013fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:34.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.127+0000 7f435e59c700 1 -- 172.21.15.32:0/2131161412 >> v1:172.21.15.61:6789/0 conn(0x7f4360067480 legacy=0x7f4360142800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:34.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.127+0000 7f435e59c700 1 -- 172.21.15.32:0/2131161412 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43600b01d0 con 0x7f43600acb80 2020-12-05T17:39:34.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.127+0000 7f435e59c700 1 --2- 172.21.15.32:0/2131161412 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f43600acb80 0x7f43600acf60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f4348002f30 tx=0x7f4348006dc0).ready entity=mon.1 client_cookie=32a3e38580cdcc55 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:34.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.127+0000 7f435d59a700 1 -- 172.21.15.32:0/2131161412 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4348009760 con 0x7f43600acb80 2020-12-05T17:39:34.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.127+0000 7f435d59a700 1 -- 172.21.15.32:0/2131161412 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43480098c0 con 0x7f43600acb80 2020-12-05T17:39:34.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.127+0000 7f435d59a700 1 -- 172.21.15.32:0/2131161412 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4348009c20 con 0x7f43600acb80 2020-12-05T17:39:34.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.127+0000 7f4364d39700 1 -- 172.21.15.32:0/2131161412 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f43600acb80 msgr2=0x7f43600acf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:34.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.127+0000 7f4364d39700 1 --2- 172.21.15.32:0/2131161412 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f43600acb80 0x7f43600acf60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f4348002f30 tx=0x7f4348006dc0).stop 2020-12-05T17:39:34.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.131+0000 7f4364d39700 1 -- 172.21.15.32:0/2131161412 shutdown_connections 2020-12-05T17:39:34.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.131+0000 7f4364d39700 1 --2- 172.21.15.32:0/2131161412 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f436005fdc0 0x7f436013fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:34.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.131+0000 7f4364d39700 1 --2- 172.21.15.32:0/2131161412 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f43600acb80 0x7f43600acf60 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:34.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.131+0000 7f4364d39700 1 -- 172.21.15.32:0/2131161412 >> 172.21.15.32:0/2131161412 conn(0x7f43600b5580 msgr2=0x7f43600b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:34.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.131+0000 7f4364d39700 1 -- 172.21.15.32:0/2131161412 shutdown_connections 2020-12-05T17:39:34.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.131+0000 7f4364d39700 1 -- 172.21.15.32:0/2131161412 wait complete. 2020-12-05T17:39:34.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.131+0000 7f4364d39700 1 Processor -- start 2020-12-05T17:39:34.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.131+0000 7f4364d39700 1 -- start start 2020-12-05T17:39:34.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.131+0000 7f4364d39700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f436005fdc0 0x7f436010efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:34.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.131+0000 7f4364d39700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4360067480 0x7f436013ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:34.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.131+0000 7f4364d39700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43600acb80 0x7f43601403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:34.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.131+0000 7f4364d39700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4360112700 con 0x7f43600acb80 2020-12-05T17:39:34.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.131+0000 7f4364d39700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43601129a0 con 0x7f436005fdc0 2020-12-05T17:39:34.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.131+0000 7f4364d39700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4360112c40 con 0x7f4360067480 2020-12-05T17:39:34.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.131+0000 7f435ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43600acb80 0x7f43601403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:34.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.131+0000 7f435ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43600acb80 0x7f43601403f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38354/0 (socket says 172.21.15.32:38354) 2020-12-05T17:39:34.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.131+0000 7f435ed9d700 1 -- 172.21.15.32:0/112148927 learned_addr learned my addr 172.21.15.32:0/112148927 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:34.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.131+0000 7f435e59c700 1 --2- 172.21.15.32:0/112148927 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f436005fdc0 0x7f436010efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:34.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.131+0000 7f435dd9b700 1 --2- 172.21.15.32:0/112148927 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4360067480 0x7f436013ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:34.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.135+0000 7f435ed9d700 1 -- 172.21.15.32:0/112148927 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4360067480 msgr2=0x7f436013ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:34.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.135+0000 7f435ed9d700 1 --2- 172.21.15.32:0/112148927 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4360067480 0x7f436013ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:34.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.135+0000 7f435ed9d700 1 -- 172.21.15.32:0/112148927 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f436005fdc0 msgr2=0x7f436010efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:34.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.135+0000 7f435ed9d700 1 --2- 172.21.15.32:0/112148927 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f436005fdc0 0x7f436010efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:34.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.135+0000 7f435ed9d700 1 -- 172.21.15.32:0/112148927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4360112ee0 con 0x7f43600acb80 2020-12-05T17:39:34.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.135+0000 7f435dd9b700 1 --2- 172.21.15.32:0/112148927 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4360067480 0x7f436013ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:34.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.135+0000 7f435e59c700 1 --2- 172.21.15.32:0/112148927 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f436005fdc0 0x7f436010efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:34.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.135+0000 7f435ed9d700 1 --2- 172.21.15.32:0/112148927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43600acb80 0x7f43601403f0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f4358006dd0 tx=0x7f43580091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:34.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.135+0000 7f43577fe700 1 -- 172.21.15.32:0/112148927 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f435800eee0 con 0x7f43600acb80 2020-12-05T17:39:34.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.135+0000 7f43577fe700 1 -- 172.21.15.32:0/112148927 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435800cbb0 con 0x7f43600acb80 2020-12-05T17:39:34.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.135+0000 7f4364d39700 1 -- 172.21.15.32:0/112148927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4360113380 con 0x7f43600acb80 2020-12-05T17:39:34.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.135+0000 7f43577fe700 1 -- 172.21.15.32:0/112148927 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43580123f0 con 0x7f43600acb80 2020-12-05T17:39:34.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.135+0000 7f4364d39700 1 -- 172.21.15.32:0/112148927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4360140b00 con 0x7f43600acb80 2020-12-05T17:39:34.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.135+0000 7f4364d39700 1 -- 172.21.15.32:0/112148927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4360113070 con 0x7f43600acb80 2020-12-05T17:39:34.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.135+0000 7f43577fe700 1 -- 172.21.15.32:0/112148927 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4358022070 con 0x7f43600acb80 2020-12-05T17:39:34.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.135+0000 7f43577fe700 1 --2- 172.21.15.32:0/112148927 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4344055470 0x7f4344057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:34.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.135+0000 7f43577fe700 1 -- 172.21.15.32:0/112148927 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f435806e660 con 0x7f43600acb80 2020-12-05T17:39:34.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.135+0000 7f435e59c700 1 --2- 172.21.15.32:0/112148927 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4344055470 0x7f4344057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:34.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.139+0000 7f435e59c700 1 --2- 172.21.15.32:0/112148927 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4344055470 0x7f4344057930 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f4348018000 tx=0x7f4348015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:34.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.143+0000 7f43577fe700 1 -- 172.21.15.32:0/112148927 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4358044800 con 0x7f43600acb80 2020-12-05T17:39:34.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.331+0000 7f4364d39700 1 -- 172.21.15.32:0/112148927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4360140b00 con 0x7f43600acb80 2020-12-05T17:39:34.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.331+0000 7f43577fe700 1 -- 172.21.15.32:0/112148927 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f4360140b00 con 0x7f43600acb80 2020-12-05T17:39:34.339 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:34.339 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":42,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6823,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2664846066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2664846066},{"type":"v1","addr":"172.21.15.61:6833","nonce":2664846066}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:39:22.261725+0000","modified":"2020-12-05T17:39:23.265348+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6930":{"gid":6930,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4087622496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4087622496},{"type":"v1","addr":"172.21.15.32:6835","nonce":4087622496}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2020-12-05T17:39:34.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.335+0000 7f4364d39700 1 -- 172.21.15.32:0/112148927 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4344055470 msgr2=0x7f4344057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:34.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.335+0000 7f4364d39700 1 --2- 172.21.15.32:0/112148927 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4344055470 0x7f4344057930 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f4348018000 tx=0x7f4348015000).stop 2020-12-05T17:39:34.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.335+0000 7f4364d39700 1 -- 172.21.15.32:0/112148927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43600acb80 msgr2=0x7f43601403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:34.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.335+0000 7f4364d39700 1 --2- 172.21.15.32:0/112148927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43600acb80 0x7f43601403f0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f4358006dd0 tx=0x7f43580091a0).stop 2020-12-05T17:39:34.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.335+0000 7f4364d39700 1 -- 172.21.15.32:0/112148927 shutdown_connections 2020-12-05T17:39:34.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.335+0000 7f4364d39700 1 --2- 172.21.15.32:0/112148927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43600acb80 0x7f43601403f0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:34.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.335+0000 7f4364d39700 1 --2- 172.21.15.32:0/112148927 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f436005fdc0 0x7f436010efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:34.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.335+0000 7f4364d39700 1 --2- 172.21.15.32:0/112148927 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4344055470 0x7f4344057930 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:34.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.335+0000 7f4364d39700 1 --2- 172.21.15.32:0/112148927 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4360067480 0x7f436013ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:34.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.335+0000 7f4364d39700 1 -- 172.21.15.32:0/112148927 >> 172.21.15.32:0/112148927 conn(0x7f43600b5580 msgr2=0x7f4360141a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.335+0000 7f4364d39700 1 -- 172.21.15.32:0/112148927 shutdown_connections 2020-12-05T17:39:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:34.335+0000 7f4364d39700 1 -- 172.21.15.32:0/112148927 wait complete. 2020-12-05T17:39:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 42 2020-12-05T17:39:34.353 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump inode 0x10000000000 2020-12-05T17:39:34.473 INFO:teuthology.orchestra.run.smithi032.stdout:{"path":"/foo","ino":1099511627776,"rdev":0,"ctime":"2020-12-05T17:39:33.685998+0000","btime":"2020-12-05T17:39:33.685998+0000","mode":16877,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2020-12-05T17:39:33.685998+0000","atime":"2020-12-05T17:39:33.685998+0000","time_warp_seq":0,"change_attr":0,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"version":2,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1},"nref":4,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"linklock":{},"dirfragtreelock":{},"filelock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"xattrlock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":6245,"pending":"pAsxLsXsxFsx","issued":"pAsxLsXsxFsx","wanted":"-","last_sent":1}],"loner":6245,"want_loner":6245,"mds_caps_wanted":[]} 2020-12-05T17:39:34.474 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2020-12-05T17:39:33.685998+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "backtrace_version": 2, "btime": "2020-12-05T17:39:33.685998+0000", "change_attr": 0, "client_caps": [ { "client_id": 6245, "issued": "pAsxLsXsxFsx", "last_sent": 1, "pending": "pAsxLsXsxFsx", "wanted": "-" } ], "client_ranges": [], "ctime": "2020-12-05T17:39:33.685998+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "flocklock": {}, "gid": 0, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": 6245, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2020-12-05T17:39:33.685998+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "nlink": 1, "nref": 4, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/foo", "pins": { "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "request": 0 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 0, "version": 2, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": 6245, "xattr_version": 1, "xattrlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "xattrs": [] } 2020-12-05T17:39:34.474 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:39:34.475 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:34.492 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:34.507 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:34.525 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:34.542 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:34.560 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:39:34.561 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:34.561 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:34.561 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:34.575 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:39:34.576 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:39:34.577 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi036.front.sepia.ceph.com... 2020-12-05T17:39:34.577 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:39:34.578 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:34.664 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[11252]: fuse finished with error 0 and tester_r 0 2020-12-05T17:39:34.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:34.667 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:34.667 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:34.713 INFO:teuthology.orchestra.run.smithi036.stdout:ext2/ext3 2020-12-05T17:39:34.714 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:39:34.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:39:34.721 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:34.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:34.771 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:34.771 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:34.779 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:39:34.780 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:39:34.780 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi002.front.sepia.ceph.com... 2020-12-05T17:39:34.781 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:39:34.782 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:34.855 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[10740]: fuse finished with error 0 and tester_r 0 2020-12-05T17:39:34.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:34.857 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:34.857 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:34.905 INFO:teuthology.orchestra.run.smithi002.stdout:ext2/ext3 2020-12-05T17:39:34.906 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:39:35.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:39:35.020 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:35.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:35.069 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:35.070 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:39:35.077 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:39:35.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:39:35.078 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2020-12-05T17:39:35.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:39:35.079 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:39:35.128 INFO:teuthology.orchestra.run.smithi053.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:39:35.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:39:35.130 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2020-12-05T17:39:35.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.247+0000 7faaad34c700 1 Processor -- start 2020-12-05T17:39:35.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.247+0000 7faaad34c700 1 -- start start 2020-12-05T17:39:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.247+0000 7faaad34c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7faaa80af230 0x7faaa8145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.247+0000 7faaad34c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa8146a90 con 0x7faaa80b1fc0 2020-12-05T17:39:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.247+0000 7faaad34c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa80b23a0 con 0x7faaa80ab710 2020-12-05T17:39:35.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.247+0000 7faaad34c700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaa8061e90 con 0x7faaa80af230 2020-12-05T17:39:35.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.247+0000 7faaa7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7faaa80b1fc0 0x7faaa8061780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41006/0 (socket says 172.21.15.32:41006) 2020-12-05T17:39:35.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.247+0000 7faaa7fff700 1 -- 172.21.15.32:0/2780875836 learned_addr learned my addr 172.21.15.32:0/2780875836 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:35.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.247+0000 7faaacb4b700 1 --2- 172.21.15.32:0/2780875836 >> v2:172.21.15.61:3300/0 conn(0x7faaa80af230 0x7faaa8145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:35.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.247+0000 7faaa6ffd700 1 -- 172.21.15.32:0/2780875836 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22550297 0 0) 0x7faaa8146a90 con 0x7faaa80b1fc0 2020-12-05T17:39:35.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.247+0000 7faaa6ffd700 1 -- 172.21.15.32:0/2780875836 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa88003230 con 0x7faaa80b1fc0 2020-12-05T17:39:35.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.247+0000 7faaacb4b700 1 -- 172.21.15.32:0/2780875836 >> v1:172.21.15.61:6789/0 conn(0x7faaa80ab710 legacy=0x7faaa8147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:35.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.247+0000 7faaacb4b700 1 -- 172.21.15.32:0/2780875836 >> v1:172.21.15.32:6789/0 conn(0x7faaa80b1fc0 legacy=0x7faaa8061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:35.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.247+0000 7faaacb4b700 1 -- 172.21.15.32:0/2780875836 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa8146cc0 con 0x7faaa80af230 2020-12-05T17:39:35.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.247+0000 7faaacb4b700 1 --2- 172.21.15.32:0/2780875836 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faaa80af230 0x7faaa8145d10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7faaa0002f30 tx=0x7faaa0006dc0).ready entity=mon.1 client_cookie=80c4225616463e7c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:35.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.247+0000 7faaa6ffd700 1 -- 172.21.15.32:0/2780875836 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faaa0009800 con 0x7faaa80af230 2020-12-05T17:39:35.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.247+0000 7faaa6ffd700 1 -- 172.21.15.32:0/2780875836 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa0009960 con 0x7faaa80af230 2020-12-05T17:39:35.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.247+0000 7faaa6ffd700 1 -- 172.21.15.32:0/2780875836 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faaa0009cc0 con 0x7faaa80af230 2020-12-05T17:39:35.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.247+0000 7faaad34c700 1 -- 172.21.15.32:0/2780875836 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faaa80af230 msgr2=0x7faaa8145d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:35.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.247+0000 7faaad34c700 1 --2- 172.21.15.32:0/2780875836 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faaa80af230 0x7faaa8145d10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7faaa0002f30 tx=0x7faaa0006dc0).stop 2020-12-05T17:39:35.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaad34c700 1 -- 172.21.15.32:0/2780875836 shutdown_connections 2020-12-05T17:39:35.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaad34c700 1 --2- 172.21.15.32:0/2780875836 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faaa80af230 0x7faaa8145d10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:35.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaad34c700 1 -- 172.21.15.32:0/2780875836 >> 172.21.15.32:0/2780875836 conn(0x7faaa80b5530 msgr2=0x7faaa80b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:35.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaad34c700 1 -- 172.21.15.32:0/2780875836 shutdown_connections 2020-12-05T17:39:35.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaad34c700 1 -- 172.21.15.32:0/2780875836 wait complete. 2020-12-05T17:39:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaad34c700 1 Processor -- start 2020-12-05T17:39:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaad34c700 1 -- start start 2020-12-05T17:39:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaad34c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa80ab710 0x7faaa8143260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:35.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaad34c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faaa80af230 0x7faaa80adc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:35.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaad34c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faaa80b1fc0 0x7faaa80ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:35.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaad34c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa814ac40 con 0x7faaa80ab710 2020-12-05T17:39:35.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaad34c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa8149a90 con 0x7faaa80b1fc0 2020-12-05T17:39:35.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaad34c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa8149ef0 con 0x7faaa80af230 2020-12-05T17:39:35.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaa7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa80ab710 0x7faaa8143260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:35.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaa7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa80ab710 0x7faaa8143260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38366/0 (socket says 172.21.15.32:38366) 2020-12-05T17:39:35.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaa7fff700 1 -- 172.21.15.32:0/1671486210 learned_addr learned my addr 172.21.15.32:0/1671486210 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:35.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaacb4b700 1 --2- 172.21.15.32:0/1671486210 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faaa80b1fc0 0x7faaa80ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:35.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaa77fe700 1 --2- 172.21.15.32:0/1671486210 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faaa80af230 0x7faaa80adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:35.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaa7fff700 1 -- 172.21.15.32:0/1671486210 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faaa80af230 msgr2=0x7faaa80adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:35.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaa7fff700 1 --2- 172.21.15.32:0/1671486210 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faaa80af230 0x7faaa80adc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:35.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaa7fff700 1 -- 172.21.15.32:0/1671486210 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faaa80b1fc0 msgr2=0x7faaa80ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:35.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaa7fff700 1 --2- 172.21.15.32:0/1671486210 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faaa80b1fc0 0x7faaa80ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:35.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaa7fff700 1 -- 172.21.15.32:0/1671486210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa80ac860 con 0x7faaa80ab710 2020-12-05T17:39:35.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaa77fe700 1 --2- 172.21.15.32:0/1671486210 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faaa80af230 0x7faaa80adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:35.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaacb4b700 1 --2- 172.21.15.32:0/1671486210 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faaa80b1fc0 0x7faaa80ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:35.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.251+0000 7faaa7fff700 1 --2- 172.21.15.32:0/1671486210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa80ab710 0x7faaa8143260 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0x7faa98001450 tx=0x7faa98007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:35.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.255+0000 7faaa57fa700 1 -- 172.21.15.32:0/1671486210 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa98005b70 con 0x7faaa80ab710 2020-12-05T17:39:35.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.255+0000 7faaad34c700 1 -- 172.21.15.32:0/1671486210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa814a9a0 con 0x7faaa80ab710 2020-12-05T17:39:35.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.255+0000 7faaa57fa700 1 -- 172.21.15.32:0/1671486210 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa98005cd0 con 0x7faaa80ab710 2020-12-05T17:39:35.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.255+0000 7faaa57fa700 1 -- 172.21.15.32:0/1671486210 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa98007300 con 0x7faaa80ab710 2020-12-05T17:39:35.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.255+0000 7faaad34c700 1 -- 172.21.15.32:0/1671486210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa8149510 con 0x7faaa80ab710 2020-12-05T17:39:35.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.255+0000 7faaad34c700 1 -- 172.21.15.32:0/1671486210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa8057050 con 0x7faaa80ab710 2020-12-05T17:39:35.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.255+0000 7faaa57fa700 1 -- 172.21.15.32:0/1671486210 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7faa98010030 con 0x7faaa80ab710 2020-12-05T17:39:35.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.255+0000 7faaa57fa700 1 --2- 172.21.15.32:0/1671486210 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faa8c0553b0 0x7faa8c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:35.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.255+0000 7faaa57fa700 1 -- 172.21.15.32:0/1671486210 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7faa9806e180 con 0x7faaa80ab710 2020-12-05T17:39:35.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.255+0000 7faaa77fe700 1 --2- 172.21.15.32:0/1671486210 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faa8c0553b0 0x7faa8c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:35.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.259+0000 7faaa77fe700 1 --2- 172.21.15.32:0/1671486210 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faa8c0553b0 0x7faa8c057870 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7faa9000b000 tx=0x7faa90009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:35.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.263+0000 7faaa57fa700 1 -- 172.21.15.32:0/1671486210 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7faa98044280 con 0x7faaa80ab710 2020-12-05T17:39:35.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.407+0000 7faaad34c700 1 -- 172.21.15.32:0/1671486210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7faaa805aab0 con 0x7faaa80ab710 2020-12-05T17:39:35.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.411+0000 7faaa57fa700 1 -- 172.21.15.32:0/1671486210 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7faa980478a0 con 0x7faaa80ab710 2020-12-05T17:39:35.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.415+0000 7faaad34c700 1 -- 172.21.15.32:0/1671486210 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faa8c0553b0 msgr2=0x7faa8c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:35.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.415+0000 7faaad34c700 1 --2- 172.21.15.32:0/1671486210 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faa8c0553b0 0x7faa8c057870 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7faa9000b000 tx=0x7faa90009040).stop 2020-12-05T17:39:35.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.415+0000 7faaad34c700 1 -- 172.21.15.32:0/1671486210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa80ab710 msgr2=0x7faaa8143260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:35.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.415+0000 7faaad34c700 1 --2- 172.21.15.32:0/1671486210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa80ab710 0x7faaa8143260 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0x7faa98001450 tx=0x7faa98007e20).stop 2020-12-05T17:39:35.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.415+0000 7faaad34c700 1 -- 172.21.15.32:0/1671486210 shutdown_connections 2020-12-05T17:39:35.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.415+0000 7faaad34c700 1 --2- 172.21.15.32:0/1671486210 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faaa80b1fc0 0x7faaa80ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:35.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.415+0000 7faaad34c700 1 --2- 172.21.15.32:0/1671486210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa80ab710 0x7faaa8143260 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:35.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.415+0000 7faaad34c700 1 --2- 172.21.15.32:0/1671486210 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faa8c0553b0 0x7faa8c057870 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:35.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.415+0000 7faaad34c700 1 --2- 172.21.15.32:0/1671486210 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faaa80af230 0x7faaa80adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:35.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.415+0000 7faaad34c700 1 -- 172.21.15.32:0/1671486210 >> 172.21.15.32:0/1671486210 conn(0x7faaa80b5530 msgr2=0x7faaa8144380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:35.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.415+0000 7faaad34c700 1 -- 172.21.15.32:0/1671486210 shutdown_connections 2020-12-05T17:39:35.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.415+0000 7faaad34c700 1 -- 172.21.15.32:0/1671486210 wait complete. 2020-12-05T17:39:35.437 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical' 2020-12-05T17:39:35.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.547+0000 7f7730907700 1 Processor -- start 2020-12-05T17:39:35.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.547+0000 7f7730907700 1 -- start start 2020-12-05T17:39:35.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f7730907700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f772c0b1fc0 0x7f772c0af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:35.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f7730907700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f772c147400 0x7f772c145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:35.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f7730907700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772c146a90 con 0x7f772c0ab710 2020-12-05T17:39:35.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f7730907700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f772c061e90 con 0x7f772c0b1fc0 2020-12-05T17:39:35.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f7730907700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f772c0b0110 con 0x7f772c147400 2020-12-05T17:39:35.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f772b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f772c0b1fc0 0x7f772c0af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:35.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f772b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f772c0b1fc0 0x7f772c0af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38374/0 (socket says 172.21.15.32:38374) 2020-12-05T17:39:35.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f772bfff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f772c147400 0x7f772c145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:35.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f772b7fe700 1 -- 172.21.15.32:0/3599012496 learned_addr learned my addr 172.21.15.32:0/3599012496 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:35.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f772b7fe700 1 -- 172.21.15.32:0/3599012496 >> v2:172.21.15.61:3300/0 conn(0x7f772c147400 msgr2=0x7f772c145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:35.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f772b7fe700 1 --2- 172.21.15.32:0/3599012496 >> v2:172.21.15.61:3300/0 conn(0x7f772c147400 0x7f772c145d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:35.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f772b7fe700 1 -- 172.21.15.32:0/3599012496 >> v1:172.21.15.61:6789/0 conn(0x7f772c0ab710 legacy=0x7f772c061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:35.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f772b7fe700 1 -- 172.21.15.32:0/3599012496 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772c0b23a0 con 0x7f772c0b1fc0 2020-12-05T17:39:35.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f772bfff700 1 --2- 172.21.15.32:0/3599012496 >> v2:172.21.15.61:3300/0 conn(0x7f772c147400 0x7f772c145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:35.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f772b7fe700 1 --2- 172.21.15.32:0/3599012496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f772c0b1fc0 0x7f772c0af230 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f771c002f30 tx=0x7f771c006e80).ready entity=mon.0 client_cookie=5374249e0e50cbdf server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:35.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f772a7fc700 1 -- 172.21.15.32:0/3599012496 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f771c0097f0 con 0x7f772c0b1fc0 2020-12-05T17:39:35.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f772a7fc700 1 -- 172.21.15.32:0/3599012496 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771c009950 con 0x7f772c0b1fc0 2020-12-05T17:39:35.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f7730907700 1 -- 172.21.15.32:0/3599012496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f772c0b1fc0 msgr2=0x7f772c0af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:35.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f772a7fc700 1 -- 172.21.15.32:0/3599012496 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f771c009cb0 con 0x7f772c0b1fc0 2020-12-05T17:39:35.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f7730907700 1 --2- 172.21.15.32:0/3599012496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f772c0b1fc0 0x7f772c0af230 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f771c002f30 tx=0x7f771c006e80).stop 2020-12-05T17:39:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f7730907700 1 -- 172.21.15.32:0/3599012496 shutdown_connections 2020-12-05T17:39:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f7730907700 1 --2- 172.21.15.32:0/3599012496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f772c0b1fc0 0x7f772c0af230 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f7730907700 1 --2- 172.21.15.32:0/3599012496 >> v2:172.21.15.61:3300/0 conn(0x7f772c147400 0x7f772c145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f7730907700 1 -- 172.21.15.32:0/3599012496 >> 172.21.15.32:0/3599012496 conn(0x7f772c0b5530 msgr2=0x7f772c0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f7730907700 1 -- 172.21.15.32:0/3599012496 shutdown_connections 2020-12-05T17:39:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f7730907700 1 -- 172.21.15.32:0/3599012496 wait complete. 2020-12-05T17:39:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f7730907700 1 Processor -- start 2020-12-05T17:39:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.551+0000 7f7730907700 1 -- start start 2020-12-05T17:39:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.555+0000 7f7730907700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f772c0ab710 0x7f772c14d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.555+0000 7f7730907700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f772c0b1fc0 0x7f772c140e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:35.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.555+0000 7f7730907700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f772c147400 0x7f772c141380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:35.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.555+0000 7f7730907700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772c149a10 con 0x7f772c0b1fc0 2020-12-05T17:39:35.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.555+0000 7f7730907700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772c149cb0 con 0x7f772c147400 2020-12-05T17:39:35.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.555+0000 7f7730907700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772c149f50 con 0x7f772c0ab710 2020-12-05T17:39:35.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.555+0000 7f772affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f772c0b1fc0 0x7f772c140e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:35.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.555+0000 7f772affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f772c0b1fc0 0x7f772c140e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38378/0 (socket says 172.21.15.32:38378) 2020-12-05T17:39:35.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.555+0000 7f772affd700 1 -- 172.21.15.32:0/2526353604 learned_addr learned my addr 172.21.15.32:0/2526353604 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:35.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.555+0000 7f772bfff700 1 --2- 172.21.15.32:0/2526353604 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f772c147400 0x7f772c141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:35.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.555+0000 7f772b7fe700 1 --2- 172.21.15.32:0/2526353604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f772c0ab710 0x7f772c14d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:35.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.555+0000 7f772affd700 1 -- 172.21.15.32:0/2526353604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f772c0ab710 msgr2=0x7f772c14d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:35.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.555+0000 7f772affd700 1 --2- 172.21.15.32:0/2526353604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f772c0ab710 0x7f772c14d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:35.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.555+0000 7f772affd700 1 -- 172.21.15.32:0/2526353604 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f772c147400 msgr2=0x7f772c141380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:35.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.555+0000 7f772affd700 1 --2- 172.21.15.32:0/2526353604 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f772c147400 0x7f772c141380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:35.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.555+0000 7f772affd700 1 -- 172.21.15.32:0/2526353604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772c141870 con 0x7f772c0b1fc0 2020-12-05T17:39:35.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.555+0000 7f772affd700 1 --2- 172.21.15.32:0/2526353604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f772c0b1fc0 0x7f772c140e90 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7f772c0608f0 tx=0x7f7720007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:35.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.555+0000 7f7728ff9700 1 -- 172.21.15.32:0/2526353604 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f772000e4a0 con 0x7f772c0b1fc0 2020-12-05T17:39:35.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.555+0000 7f7728ff9700 1 -- 172.21.15.32:0/2526353604 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772000e600 con 0x7f772c0b1fc0 2020-12-05T17:39:35.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.555+0000 7f7728ff9700 1 -- 172.21.15.32:0/2526353604 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f772000e960 con 0x7f772c0b1fc0 2020-12-05T17:39:35.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.555+0000 7f7730907700 1 -- 172.21.15.32:0/2526353604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f772c158cf0 con 0x7f772c0b1fc0 2020-12-05T17:39:35.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.555+0000 7f7730907700 1 -- 172.21.15.32:0/2526353604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f772c0ac890 con 0x7f772c0b1fc0 2020-12-05T17:39:35.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.559+0000 7f7730907700 1 -- 172.21.15.32:0/2526353604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f772c057050 con 0x7f772c0b1fc0 2020-12-05T17:39:35.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.559+0000 7f7728ff9700 1 -- 172.21.15.32:0/2526353604 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7720010030 con 0x7f772c0b1fc0 2020-12-05T17:39:35.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.559+0000 7f7728ff9700 1 --2- 172.21.15.32:0/2526353604 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f770c055440 0x7f770c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:35.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.559+0000 7f772b7fe700 1 --2- 172.21.15.32:0/2526353604 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f770c055440 0x7f770c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:35.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.559+0000 7f7728ff9700 1 -- 172.21.15.32:0/2526353604 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f772006dee0 con 0x7f772c0b1fc0 2020-12-05T17:39:35.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.559+0000 7f772b7fe700 1 --2- 172.21.15.32:0/2526353604 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f770c055440 0x7f770c057900 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f771c01c000 tx=0x7f771c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:35.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.567+0000 7f7728ff9700 1 -- 172.21.15.32:0/2526353604 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7720044080 con 0x7f772c0b1fc0 2020-12-05T17:39:35.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:35.751+0000 7f7730907700 1 -- 172.21.15.32:0/2526353604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]} v 0) v1 -- 0x7f772c05aab0 con 0x7f772c0b1fc0 2020-12-05T17:39:36.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.027+0000 7f7728ff9700 1 -- 172.21.15.32:0/2526353604 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]}]=0 v353) v1 ==== 138+0+0 (secure 0 0 0) 0x7f77200476a0 con 0x7f772c0b1fc0 2020-12-05T17:39:36.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.031+0000 7f7730907700 1 -- 172.21.15.32:0/2526353604 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f770c055440 msgr2=0x7f770c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:36.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.031+0000 7f7730907700 1 --2- 172.21.15.32:0/2526353604 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f770c055440 0x7f770c057900 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f771c01c000 tx=0x7f771c015000).stop 2020-12-05T17:39:36.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.031+0000 7f7730907700 1 -- 172.21.15.32:0/2526353604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f772c0b1fc0 msgr2=0x7f772c140e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:36.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.031+0000 7f7730907700 1 --2- 172.21.15.32:0/2526353604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f772c0b1fc0 0x7f772c140e90 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7f772c0608f0 tx=0x7f7720007c20).stop 2020-12-05T17:39:36.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.031+0000 7f7730907700 1 -- 172.21.15.32:0/2526353604 shutdown_connections 2020-12-05T17:39:36.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.031+0000 7f7730907700 1 --2- 172.21.15.32:0/2526353604 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f772c147400 0x7f772c141380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:36.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.031+0000 7f7730907700 1 --2- 172.21.15.32:0/2526353604 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f770c055440 0x7f770c057900 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:36.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.031+0000 7f7730907700 1 --2- 172.21.15.32:0/2526353604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f772c0ab710 0x7f772c14d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:36.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.031+0000 7f7730907700 1 --2- 172.21.15.32:0/2526353604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f772c0b1fc0 0x7f772c140e90 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:36.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.031+0000 7f7730907700 1 -- 172.21.15.32:0/2526353604 >> 172.21.15.32:0/2526353604 conn(0x7f772c0b5530 msgr2=0x7f772c0af740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:36.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.031+0000 7f7730907700 1 -- 172.21.15.32:0/2526353604 shutdown_connections 2020-12-05T17:39:36.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.031+0000 7f7730907700 1 -- 172.21.15.32:0/2526353604 wait complete. 2020-12-05T17:39:36.053 INFO:tasks.cephfs_test_runner:test_dump_inode_hexademical (tasks.cephfs.test_misc.TestMisc) ... ok 2020-12-05T17:39:36.053 INFO:tasks.cephfs_test_runner:Starting test: test_filtered_df (tasks.cephfs.test_misc.TestMisc) 2020-12-05T17:39:36.054 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df' 2020-12-05T17:39:36.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.163+0000 7efdf1ff3700 1 Processor -- start 2020-12-05T17:39:36.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.163+0000 7efdf1ff3700 1 -- start start 2020-12-05T17:39:36.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdf1ff3700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7efdec0afe50 0x7efdec146a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:36.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdf1ff3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efdec0b3aa0 0x7efdec1429e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:36.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdf1ff3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdec0b0230 con 0x7efdec0b1880 2020-12-05T17:39:36.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdf1ff3700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdec0b6a50 con 0x7efdec0afe50 2020-12-05T17:39:36.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdf1ff3700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdec146080 con 0x7efdec0b3aa0 2020-12-05T17:39:36.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdebfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efdec0b1880 0x7efdec142ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41038/0 (socket says 172.21.15.32:41038) 2020-12-05T17:39:36.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdebfff700 1 -- 172.21.15.32:0/2907767569 learned_addr learned my addr 172.21.15.32:0/2907767569 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:36.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdeaffd700 1 --2- 172.21.15.32:0/2907767569 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efdec0b3aa0 0x7efdec1429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:36.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdeb7fe700 1 --2- 172.21.15.32:0/2907767569 >> v2:172.21.15.61:3300/0 conn(0x7efdec0afe50 0x7efdec146a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:36.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdea7fc700 1 -- 172.21.15.32:0/2907767569 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1513058203 0 0) 0x7efdec0b0230 con 0x7efdec0b1880 2020-12-05T17:39:36.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdea7fc700 1 -- 172.21.15.32:0/2907767569 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdd8003230 con 0x7efdec0b1880 2020-12-05T17:39:36.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdeb7fe700 1 -- 172.21.15.32:0/2907767569 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efdec0b3aa0 msgr2=0x7efdec1429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:36.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdeb7fe700 1 --2- 172.21.15.32:0/2907767569 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efdec0b3aa0 0x7efdec1429e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:36.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdeb7fe700 1 -- 172.21.15.32:0/2907767569 >> v1:172.21.15.32:6789/0 conn(0x7efdec0b1880 legacy=0x7efdec142ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:36.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdeb7fe700 1 -- 172.21.15.32:0/2907767569 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec146320 con 0x7efdec0afe50 2020-12-05T17:39:36.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdeaffd700 1 --2- 172.21.15.32:0/2907767569 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efdec0b3aa0 0x7efdec1429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:36.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdeb7fe700 1 --2- 172.21.15.32:0/2907767569 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efdec0afe50 0x7efdec146a60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7efddc002f30 tx=0x7efddc006dc0).ready entity=mon.1 client_cookie=4ea82025450d90f9 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:36.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdea7fc700 1 -- 172.21.15.32:0/2907767569 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efddc0096a0 con 0x7efdec0afe50 2020-12-05T17:39:36.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdea7fc700 1 -- 172.21.15.32:0/2907767569 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efddc009800 con 0x7efdec0afe50 2020-12-05T17:39:36.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdea7fc700 1 -- 172.21.15.32:0/2907767569 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efddc009b60 con 0x7efdec0afe50 2020-12-05T17:39:36.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdf1ff3700 1 -- 172.21.15.32:0/2907767569 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efdec0afe50 msgr2=0x7efdec146a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:36.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdf1ff3700 1 --2- 172.21.15.32:0/2907767569 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efdec0afe50 0x7efdec146a60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7efddc002f30 tx=0x7efddc006dc0).stop 2020-12-05T17:39:36.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdf1ff3700 1 -- 172.21.15.32:0/2907767569 shutdown_connections 2020-12-05T17:39:36.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdf1ff3700 1 --2- 172.21.15.32:0/2907767569 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efdec0afe50 0x7efdec146a60 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:36.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdf1ff3700 1 --2- 172.21.15.32:0/2907767569 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efdec0b3aa0 0x7efdec1429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:36.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdf1ff3700 1 -- 172.21.15.32:0/2907767569 >> 172.21.15.32:0/2907767569 conn(0x7efdec0b9100 msgr2=0x7efdec0b9510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:36.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdf1ff3700 1 -- 172.21.15.32:0/2907767569 shutdown_connections 2020-12-05T17:39:36.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.167+0000 7efdf1ff3700 1 -- 172.21.15.32:0/2907767569 wait complete. 2020-12-05T17:39:36.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.171+0000 7efdf1ff3700 1 Processor -- start 2020-12-05T17:39:36.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.171+0000 7efdf1ff3700 1 -- start start 2020-12-05T17:39:36.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.171+0000 7efdf1ff3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efdec0afe50 0x7efdec1405e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:36.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.171+0000 7efdf1ff3700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efdec0b1880 0x7efdec0b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:36.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.171+0000 7efdf1ff3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdec0b3aa0 0x7efdec155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:36.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.171+0000 7efdf1ff3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec154f10 con 0x7efdec0b3aa0 2020-12-05T17:39:36.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.171+0000 7efdf1ff3700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec142b90 con 0x7efdec0b1880 2020-12-05T17:39:36.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.171+0000 7efdf1ff3700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdec0b2d40 con 0x7efdec0afe50 2020-12-05T17:39:36.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.171+0000 7efdebfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdec0b3aa0 0x7efdec155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:36.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.171+0000 7efdebfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdec0b3aa0 0x7efdec155340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38396/0 (socket says 172.21.15.32:38396) 2020-12-05T17:39:36.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.171+0000 7efdebfff700 1 -- 172.21.15.32:0/3383784724 learned_addr learned my addr 172.21.15.32:0/3383784724 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:36.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.171+0000 7efdeaffd700 1 --2- 172.21.15.32:0/3383784724 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efdec0b1880 0x7efdec0b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:36.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.171+0000 7efdeb7fe700 1 --2- 172.21.15.32:0/3383784724 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efdec0afe50 0x7efdec1405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:36.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.171+0000 7efdebfff700 1 -- 172.21.15.32:0/3383784724 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efdec0afe50 msgr2=0x7efdec1405e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:36.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.171+0000 7efdebfff700 1 --2- 172.21.15.32:0/3383784724 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efdec0afe50 0x7efdec1405e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:36.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.171+0000 7efdebfff700 1 -- 172.21.15.32:0/3383784724 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efdec0b1880 msgr2=0x7efdec0b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:36.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.171+0000 7efdebfff700 1 --2- 172.21.15.32:0/3383784724 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efdec0b1880 0x7efdec0b5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:36.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.171+0000 7efdebfff700 1 -- 172.21.15.32:0/3383784724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec142e30 con 0x7efdec0b3aa0 2020-12-05T17:39:36.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.171+0000 7efdebfff700 1 --2- 172.21.15.32:0/3383784724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdec0b3aa0 0x7efdec155340 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7efde4001450 tx=0x7efde4007e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:36.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.175+0000 7efde8ff9700 1 -- 172.21.15.32:0/3383784724 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efde400ce70 con 0x7efdec0b3aa0 2020-12-05T17:39:36.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.175+0000 7efde8ff9700 1 -- 172.21.15.32:0/3383784724 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde4005b70 con 0x7efdec0b3aa0 2020-12-05T17:39:36.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.175+0000 7efdf1ff3700 1 -- 172.21.15.32:0/3383784724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdec066ba0 con 0x7efdec0b3aa0 2020-12-05T17:39:36.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.175+0000 7efde8ff9700 1 -- 172.21.15.32:0/3383784724 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efde4007110 con 0x7efdec0b3aa0 2020-12-05T17:39:36.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.175+0000 7efdf1ff3700 1 -- 172.21.15.32:0/3383784724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdec142860 con 0x7efdec0b3aa0 2020-12-05T17:39:36.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.175+0000 7efdf1ff3700 1 -- 172.21.15.32:0/3383784724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdec066eb0 con 0x7efdec0b3aa0 2020-12-05T17:39:36.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.175+0000 7efde8ff9700 1 -- 172.21.15.32:0/3383784724 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7efde4010030 con 0x7efdec0b3aa0 2020-12-05T17:39:36.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.175+0000 7efde8ff9700 1 --2- 172.21.15.32:0/3383784724 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efdcc055440 0x7efdcc057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:36.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.175+0000 7efde8ff9700 1 -- 172.21.15.32:0/3383784724 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7efde406e050 con 0x7efdec0b3aa0 2020-12-05T17:39:36.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.175+0000 7efdeb7fe700 1 --2- 172.21.15.32:0/3383784724 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efdcc055440 0x7efdcc057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:36.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.175+0000 7efdeb7fe700 1 --2- 172.21.15.32:0/3383784724 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efdcc055440 0x7efdcc057900 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7efddc01b000 tx=0x7efddc015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:36.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.183+0000 7efde8ff9700 1 -- 172.21.15.32:0/3383784724 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7efde40441f0 con 0x7efdec0b3aa0 2020-12-05T17:39:36.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:36.375+0000 7efdf1ff3700 1 -- 172.21.15.32:0/3383784724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]} v 0) v1 -- 0x7efdec057050 con 0x7efdec0b3aa0 2020-12-05T17:39:37.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.035+0000 7efde8ff9700 1 -- 172.21.15.32:0/3383784724 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]}]=0 v354) v1 ==== 130+0+0 (secure 0 0 0) 0x7efde4047810 con 0x7efdec0b3aa0 2020-12-05T17:39:37.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.039+0000 7efdf1ff3700 1 -- 172.21.15.32:0/3383784724 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efdcc055440 msgr2=0x7efdcc057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:37.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.039+0000 7efdf1ff3700 1 --2- 172.21.15.32:0/3383784724 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efdcc055440 0x7efdcc057900 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7efddc01b000 tx=0x7efddc015000).stop 2020-12-05T17:39:37.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.039+0000 7efdf1ff3700 1 -- 172.21.15.32:0/3383784724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdec0b3aa0 msgr2=0x7efdec155340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:37.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.039+0000 7efdf1ff3700 1 --2- 172.21.15.32:0/3383784724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdec0b3aa0 0x7efdec155340 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7efde4001450 tx=0x7efde4007e40).stop 2020-12-05T17:39:37.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.039+0000 7efdf1ff3700 1 -- 172.21.15.32:0/3383784724 shutdown_connections 2020-12-05T17:39:37.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.039+0000 7efdf1ff3700 1 --2- 172.21.15.32:0/3383784724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdec0b3aa0 0x7efdec155340 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:37.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.039+0000 7efdf1ff3700 1 --2- 172.21.15.32:0/3383784724 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efdcc055440 0x7efdcc057900 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:37.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.039+0000 7efdf1ff3700 1 --2- 172.21.15.32:0/3383784724 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efdec0afe50 0x7efdec1405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:37.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.039+0000 7efdf1ff3700 1 --2- 172.21.15.32:0/3383784724 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efdec0b1880 0x7efdec0b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.039+0000 7efdf1ff3700 1 -- 172.21.15.32:0/3383784724 >> 172.21.15.32:0/3383784724 conn(0x7efdec0b9100 msgr2=0x7efdec13f040 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.039+0000 7efdf1ff3700 1 -- 172.21.15.32:0/3383784724 shutdown_connections 2020-12-05T17:39:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.039+0000 7efdf1ff3700 1 -- 172.21.15.32:0/3383784724 wait complete. 2020-12-05T17:39:37.063 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-12-05T17:39:37.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.187+0000 7f1c89dd6700 1 Processor -- start 2020-12-05T17:39:37.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.187+0000 7f1c89dd6700 1 -- start start 2020-12-05T17:39:37.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.187+0000 7f1c89dd6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c840b6690 0x7f1c840b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:37.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.187+0000 7f1c89dd6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c840b56f0 con 0x7f1c840b5240 2020-12-05T17:39:37.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.187+0000 7f1c89dd6700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c840b1920 con 0x7f1c840b5b20 2020-12-05T17:39:37.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.187+0000 7f1c89dd6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c840b1bc0 con 0x7f1c840b6690 2020-12-05T17:39:37.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.187+0000 7f1c83fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1c840b5240 0x7f1c84145f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41052/0 (socket says 172.21.15.32:41052) 2020-12-05T17:39:37.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.187+0000 7f1c83fff700 1 -- 172.21.15.32:0/3234563689 learned_addr learned my addr 172.21.15.32:0/3234563689 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:37.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.187+0000 7f1c82ffd700 1 --2- 172.21.15.32:0/3234563689 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c840b6690 0x7f1c840b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:37.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.187+0000 7f1c827fc700 1 -- 172.21.15.32:0/3234563689 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2045546365 0 0) 0x7f1c840b56f0 con 0x7f1c840b5240 2020-12-05T17:39:37.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.187+0000 7f1c827fc700 1 -- 172.21.15.32:0/3234563689 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c64003230 con 0x7f1c840b5240 2020-12-05T17:39:37.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.187+0000 7f1c827fc700 1 -- 172.21.15.32:0/3234563689 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1001475575 0 0) 0x7f1c840b1920 con 0x7f1c840b5b20 2020-12-05T17:39:37.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.187+0000 7f1c827fc700 1 -- 172.21.15.32:0/3234563689 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c840b56f0 con 0x7f1c840b5b20 2020-12-05T17:39:37.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.187+0000 7f1c82ffd700 1 -- 172.21.15.32:0/3234563689 >> v1:172.21.15.61:6789/0 conn(0x7f1c840b5b20 legacy=0x7f1c840b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:37.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.187+0000 7f1c82ffd700 1 -- 172.21.15.32:0/3234563689 >> v1:172.21.15.32:6789/0 conn(0x7f1c840b5240 legacy=0x7f1c84145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:37.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.187+0000 7f1c82ffd700 1 -- 172.21.15.32:0/3234563689 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c84146620 con 0x7f1c840b6690 2020-12-05T17:39:37.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.187+0000 7f1c82ffd700 1 --2- 172.21.15.32:0/3234563689 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c840b6690 0x7f1c840b5f00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f1c6c002f30 tx=0x7f1c6c006dc0).ready entity=mon.2 client_cookie=c6936157516d0453 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:37.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.191+0000 7f1c827fc700 1 -- 172.21.15.32:0/3234563689 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c6c009730 con 0x7f1c840b6690 2020-12-05T17:39:37.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.191+0000 7f1c827fc700 1 -- 172.21.15.32:0/3234563689 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6c009890 con 0x7f1c840b6690 2020-12-05T17:39:37.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.191+0000 7f1c827fc700 1 -- 172.21.15.32:0/3234563689 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c6c009bf0 con 0x7f1c840b6690 2020-12-05T17:39:37.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.191+0000 7f1c89dd6700 1 -- 172.21.15.32:0/3234563689 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c840b6690 msgr2=0x7f1c840b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:37.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.191+0000 7f1c89dd6700 1 --2- 172.21.15.32:0/3234563689 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c840b6690 0x7f1c840b5f00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f1c6c002f30 tx=0x7f1c6c006dc0).stop 2020-12-05T17:39:37.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.191+0000 7f1c89dd6700 1 -- 172.21.15.32:0/3234563689 shutdown_connections 2020-12-05T17:39:37.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.191+0000 7f1c89dd6700 1 --2- 172.21.15.32:0/3234563689 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c840b6690 0x7f1c840b5f00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:37.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.191+0000 7f1c89dd6700 1 -- 172.21.15.32:0/3234563689 >> 172.21.15.32:0/3234563689 conn(0x7f1c8405f3b0 msgr2=0x7f1c8405f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:37.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.191+0000 7f1c89dd6700 1 -- 172.21.15.32:0/3234563689 shutdown_connections 2020-12-05T17:39:37.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.191+0000 7f1c89dd6700 1 -- 172.21.15.32:0/3234563689 wait complete. 2020-12-05T17:39:37.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.191+0000 7f1c89dd6700 1 Processor -- start 2020-12-05T17:39:37.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.191+0000 7f1c89dd6700 1 -- start start 2020-12-05T17:39:37.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c89dd6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c840b5240 0x7f1c840b9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:37.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c89dd6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c840b5b20 0x7f1c840a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:37.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c89dd6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c840b6690 0x7f1c840a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:37.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c89dd6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c840663b0 con 0x7f1c840b6690 2020-12-05T17:39:37.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c89dd6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c840a2530 con 0x7f1c840b5b20 2020-12-05T17:39:37.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c89dd6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c840665e0 con 0x7f1c840b5240 2020-12-05T17:39:37.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c83fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c840b6690 0x7f1c840a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:37.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c82ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c840b5b20 0x7f1c840a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:37.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c83fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c840b6690 0x7f1c840a53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38410/0 (socket says 172.21.15.32:38410) 2020-12-05T17:39:37.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c82ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c840b5b20 0x7f1c840a4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:58300/0 (socket says 172.21.15.32:58300) 2020-12-05T17:39:37.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c83fff700 1 -- 172.21.15.32:0/2882691166 learned_addr learned my addr 172.21.15.32:0/2882691166 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:37.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c837fe700 1 --2- 172.21.15.32:0/2882691166 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c840b5240 0x7f1c840b9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:37.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c837fe700 1 -- 172.21.15.32:0/2882691166 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c840b5b20 msgr2=0x7f1c840a4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:37.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c837fe700 1 --2- 172.21.15.32:0/2882691166 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c840b5b20 0x7f1c840a4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:37.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c837fe700 1 -- 172.21.15.32:0/2882691166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c840b6690 msgr2=0x7f1c840a53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:37.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c837fe700 1 --2- 172.21.15.32:0/2882691166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c840b6690 0x7f1c840a53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:37.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c837fe700 1 -- 172.21.15.32:0/2882691166 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c840a2790 con 0x7f1c840b5240 2020-12-05T17:39:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c82ffd700 1 --2- 172.21.15.32:0/2882691166 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c840b5b20 0x7f1c840a4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c83fff700 1 --2- 172.21.15.32:0/2882691166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c840b6690 0x7f1c840a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c837fe700 1 --2- 172.21.15.32:0/2882691166 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c840b5240 0x7f1c840b9780 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f1c74001450 tx=0x7f1c74007e20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c80ff9700 1 -- 172.21.15.32:0/2882691166 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c74002240 con 0x7f1c840b5240 2020-12-05T17:39:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c80ff9700 1 -- 172.21.15.32:0/2882691166 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c740023a0 con 0x7f1c840b5240 2020-12-05T17:39:37.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c89dd6700 1 -- 172.21.15.32:0/2882691166 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c840a2c30 con 0x7f1c840b5240 2020-12-05T17:39:37.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c80ff9700 1 -- 172.21.15.32:0/2882691166 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c74001610 con 0x7f1c840b5240 2020-12-05T17:39:37.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c89dd6700 1 -- 172.21.15.32:0/2882691166 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c84065460 con 0x7f1c840b5240 2020-12-05T17:39:37.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c89dd6700 1 -- 172.21.15.32:0/2882691166 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c840a2920 con 0x7f1c840b5240 2020-12-05T17:39:37.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.195+0000 7f1c80ff9700 1 -- 172.21.15.32:0/2882691166 <== mon.2 v2:172.21.15.61:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1c74010070 con 0x7f1c840b5240 2020-12-05T17:39:37.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.199+0000 7f1c80ff9700 1 --2- 172.21.15.32:0/2882691166 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1c680554e0 0x7f1c680579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:37.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.199+0000 7f1c80ff9700 1 -- 172.21.15.32:0/2882691166 <== mon.2 v2:172.21.15.61:3301/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1c7406d660 con 0x7f1c840b5240 2020-12-05T17:39:37.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.199+0000 7f1c82ffd700 1 --2- 172.21.15.32:0/2882691166 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1c680554e0 0x7f1c680579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:37.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.199+0000 7f1c82ffd700 1 --2- 172.21.15.32:0/2882691166 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1c680554e0 0x7f1c680579a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f1c6c019000 tx=0x7f1c6c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.207+0000 7f1c80ff9700 1 -- 172.21.15.32:0/2882691166 <== mon.2 v2:172.21.15.61:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1c74043800 con 0x7f1c840b5240 2020-12-05T17:39:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.355+0000 7f1c89dd6700 1 -- 172.21.15.32:0/2882691166 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f1c84057050 con 0x7f1c840b5240 2020-12-05T17:39:37.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.363+0000 7f1c80ff9700 1 -- 172.21.15.32:0/2882691166 <== mon.2 v2:172.21.15.61:3301/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1c74046e20 con 0x7f1c840b5240 2020-12-05T17:39:37.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.367+0000 7f1c89dd6700 1 -- 172.21.15.32:0/2882691166 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1c680554e0 msgr2=0x7f1c680579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:37.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.367+0000 7f1c89dd6700 1 --2- 172.21.15.32:0/2882691166 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1c680554e0 0x7f1c680579a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f1c6c019000 tx=0x7f1c6c015000).stop 2020-12-05T17:39:37.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.367+0000 7f1c89dd6700 1 -- 172.21.15.32:0/2882691166 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c840b5240 msgr2=0x7f1c840b9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:37.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.367+0000 7f1c89dd6700 1 --2- 172.21.15.32:0/2882691166 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c840b5240 0x7f1c840b9780 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f1c74001450 tx=0x7f1c74007e20).stop 2020-12-05T17:39:37.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.367+0000 7f1c89dd6700 1 -- 172.21.15.32:0/2882691166 shutdown_connections 2020-12-05T17:39:37.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.367+0000 7f1c89dd6700 1 --2- 172.21.15.32:0/2882691166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c840b6690 0x7f1c840a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:37.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.367+0000 7f1c89dd6700 1 --2- 172.21.15.32:0/2882691166 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1c680554e0 0x7f1c680579a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:37.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.367+0000 7f1c89dd6700 1 --2- 172.21.15.32:0/2882691166 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c840b5240 0x7f1c840b9780 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:37.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.367+0000 7f1c89dd6700 1 --2- 172.21.15.32:0/2882691166 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c840b5b20 0x7f1c840a4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:37.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.367+0000 7f1c89dd6700 1 -- 172.21.15.32:0/2882691166 >> 172.21.15.32:0/2882691166 conn(0x7f1c8405f3b0 msgr2=0x7f1c8405f790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:37.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.367+0000 7f1c89dd6700 1 -- 172.21.15.32:0/2882691166 shutdown_connections 2020-12-05T17:39:37.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.367+0000 7f1c89dd6700 1 -- 172.21.15.32:0/2882691166 wait complete. 2020-12-05T17:39:37.387 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:39:37.388 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:37.406 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:37.423 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:37.440 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:37.457 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:37.475 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:39:37.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:37.476 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:37.476 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:37.490 INFO:teuthology.orchestra.run.smithi036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-12-05T17:39:37.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:39:37.491 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-12-05T17:39:37.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:37.492 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:37.493 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:37.499 INFO:teuthology.orchestra.run.smithi002.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-12-05T17:39:37.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:39:37.501 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-12-05T17:39:37.501 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:37.502 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:37.502 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:39:37.509 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:39:37.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:39:37.510 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2020-12-05T17:39:37.511 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-12-05T17:39:37.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6b07f700 1 Processor -- start 2020-12-05T17:39:37.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6b07f700 1 -- start start 2020-12-05T17:39:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6b07f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b64144540 0x7f9b64061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6b07f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b64143a00 con 0x7f9b641403f0 2020-12-05T17:39:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6b07f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b64140830 con 0x7f9b640b7720 2020-12-05T17:39:37.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6b07f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b640b7b00 con 0x7f9b64144540 2020-12-05T17:39:37.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6987c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b64144540 0x7f9b64061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:37.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6a87e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9b641403f0 0x7f9b6413d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41064/0 (socket says 172.21.15.32:41064) 2020-12-05T17:39:37.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6a87e700 1 -- 172.21.15.32:0/4215744377 learned_addr learned my addr 172.21.15.32:0/4215744377 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:37.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6907b700 1 -- 172.21.15.32:0/4215744377 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3700431632 0 0) 0x7f9b64143a00 con 0x7f9b641403f0 2020-12-05T17:39:37.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6907b700 1 -- 172.21.15.32:0/4215744377 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b4c003230 con 0x7f9b641403f0 2020-12-05T17:39:37.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6987c700 1 -- 172.21.15.32:0/4215744377 >> v1:172.21.15.61:6789/0 conn(0x7f9b640b7720 legacy=0x7f9b64143e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:37.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6907b700 1 -- 172.21.15.32:0/4215744377 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2787914368 0 0) 0x7f9b64140830 con 0x7f9b640b7720 2020-12-05T17:39:37.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6987c700 1 -- 172.21.15.32:0/4215744377 >> v1:172.21.15.32:6789/0 conn(0x7f9b641403f0 legacy=0x7f9b6413d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:37.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6987c700 1 -- 172.21.15.32:0/4215744377 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6405fe60 con 0x7f9b64144540 2020-12-05T17:39:37.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6987c700 1 --2- 172.21.15.32:0/4215744377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b64144540 0x7f9b64061780 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f9b58002f30 tx=0x7f9b58006dc0).ready entity=mon.0 client_cookie=227caa68c33d6261 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:37.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6907b700 1 -- 172.21.15.32:0/4215744377 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b58009800 con 0x7f9b64144540 2020-12-05T17:39:37.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6907b700 1 -- 172.21.15.32:0/4215744377 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b58009960 con 0x7f9b64144540 2020-12-05T17:39:37.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6907b700 1 -- 172.21.15.32:0/4215744377 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b58009cc0 con 0x7f9b64144540 2020-12-05T17:39:37.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6b07f700 1 -- 172.21.15.32:0/4215744377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b64144540 msgr2=0x7f9b64061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:37.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6b07f700 1 --2- 172.21.15.32:0/4215744377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b64144540 0x7f9b64061780 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f9b58002f30 tx=0x7f9b58006dc0).stop 2020-12-05T17:39:37.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6b07f700 1 -- 172.21.15.32:0/4215744377 shutdown_connections 2020-12-05T17:39:37.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6b07f700 1 --2- 172.21.15.32:0/4215744377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b64144540 0x7f9b64061780 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:37.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6b07f700 1 -- 172.21.15.32:0/4215744377 >> 172.21.15.32:0/4215744377 conn(0x7f9b6405bc20 msgr2=0x7f9b640b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:37.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6b07f700 1 -- 172.21.15.32:0/4215744377 shutdown_connections 2020-12-05T17:39:37.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.627+0000 7f9b6b07f700 1 -- 172.21.15.32:0/4215744377 wait complete. 2020-12-05T17:39:37.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b6b07f700 1 Processor -- start 2020-12-05T17:39:37.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b6b07f700 1 -- start start 2020-12-05T17:39:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b6b07f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b640b7720 0x7f9b64110900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b6b07f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b641403f0 0x7f9b64110200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b6b07f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b64144540 0x7f9b640affc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b6b07f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b640b3f70 con 0x7f9b640b7720 2020-12-05T17:39:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b6b07f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6414fa70 con 0x7f9b64144540 2020-12-05T17:39:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b6b07f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6414fd10 con 0x7f9b641403f0 2020-12-05T17:39:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b6a07d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b640b7720 0x7f9b64110900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b6a07d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b640b7720 0x7f9b64110900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38420/0 (socket says 172.21.15.32:38420) 2020-12-05T17:39:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b6a07d700 1 -- 172.21.15.32:0/813183047 learned_addr learned my addr 172.21.15.32:0/813183047 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:37.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b6987c700 1 --2- 172.21.15.32:0/813183047 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b641403f0 0x7f9b64110200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:37.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b6a87e700 1 --2- 172.21.15.32:0/813183047 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b64144540 0x7f9b640affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:37.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b6a07d700 1 -- 172.21.15.32:0/813183047 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b641403f0 msgr2=0x7f9b64110200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:37.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b6a07d700 1 --2- 172.21.15.32:0/813183047 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b641403f0 0x7f9b64110200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:37.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b6a07d700 1 -- 172.21.15.32:0/813183047 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b64144540 msgr2=0x7f9b640affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:37.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b6a07d700 1 --2- 172.21.15.32:0/813183047 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b64144540 0x7f9b640affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:37.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b6a07d700 1 -- 172.21.15.32:0/813183047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b640b3bf0 con 0x7f9b640b7720 2020-12-05T17:39:37.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b6a87e700 1 --2- 172.21.15.32:0/813183047 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b64144540 0x7f9b640affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:37.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b6987c700 1 --2- 172.21.15.32:0/813183047 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b641403f0 0x7f9b64110200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:37.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b6a07d700 1 --2- 172.21.15.32:0/813183047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b640b7720 0x7f9b64110900 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7f9b5c000f00 tx=0x7f9b5c007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:37.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b577fe700 1 -- 172.21.15.32:0/813183047 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b5c00a770 con 0x7f9b640b7720 2020-12-05T17:39:37.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b577fe700 1 -- 172.21.15.32:0/813183047 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5c00a8d0 con 0x7f9b640b7720 2020-12-05T17:39:37.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b6b07f700 1 -- 172.21.15.32:0/813183047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b640b29a0 con 0x7f9b640b7720 2020-12-05T17:39:37.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b577fe700 1 -- 172.21.15.32:0/813183047 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b5c00aa40 con 0x7f9b640b7720 2020-12-05T17:39:37.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.631+0000 7f9b6b07f700 1 -- 172.21.15.32:0/813183047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b640b2f00 con 0x7f9b640b7720 2020-12-05T17:39:37.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.635+0000 7f9b577fe700 1 -- 172.21.15.32:0/813183047 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9b5c010070 con 0x7f9b640b7720 2020-12-05T17:39:37.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.635+0000 7f9b6b07f700 1 -- 172.21.15.32:0/813183047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b6414fed0 con 0x7f9b640b7720 2020-12-05T17:39:37.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.635+0000 7f9b577fe700 1 --2- 172.21.15.32:0/813183047 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9b440554e0 0x7f9b440579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:37.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.635+0000 7f9b6987c700 1 --2- 172.21.15.32:0/813183047 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9b440554e0 0x7f9b440579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:37.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.635+0000 7f9b577fe700 1 -- 172.21.15.32:0/813183047 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9b5c06de90 con 0x7f9b640b7720 2020-12-05T17:39:37.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.635+0000 7f9b6987c700 1 --2- 172.21.15.32:0/813183047 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9b440554e0 0x7f9b440579a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f9b58019000 tx=0x7f9b58015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:37.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.639+0000 7f9b577fe700 1 -- 172.21.15.32:0/813183047 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9b5c043f60 con 0x7f9b640b7720 2020-12-05T17:39:37.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.779+0000 7f9b6b07f700 1 -- 172.21.15.32:0/813183047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f9b64057050 con 0x7f9b640b7720 2020-12-05T17:39:37.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.783+0000 7f9b577fe700 1 -- 172.21.15.32:0/813183047 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v92) v1 ==== 81+0+34040 (secure 0 0 0) 0x7f9b5c047580 con 0x7f9b640b7720 2020-12-05T17:39:37.790 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:37.790 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:39:37.790 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 92, 2020-12-05T17:39:37.791 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", 2020-12-05T17:39:37.791 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2020-12-05T17:31:20.975238+0000", 2020-12-05T17:39:37.792 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2020-12-05T17:39:23.264356+0000", 2020-12-05T17:39:37.792 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:39:37.792 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:39:37.792 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-12-05T17:39:37.793 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2020-12-05T17:39:37.793 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2020-12-05T17:39:37.793 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2020-12-05T17:39:37.794 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2020-12-05T17:39:37.794 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2020-12-05T17:39:37.794 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2020-12-05T17:39:37.795 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:39:37.795 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2020-12-05T17:39:37.795 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2020-12-05T17:39:37.796 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-12-05T17:39:37.796 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2020-12-05T17:39:37.796 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2020-12-05T17:39:37.796 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 18, 2020-12-05T17:39:37.797 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2020-12-05T17:39:37.797 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2020-12-05T17:39:37.797 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2020-12-05T17:39:37.797 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2020-12-05T17:39:37.798 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2020-12-05T17:39:37.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.798 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2020-12-05T17:39:37.798 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2020-12-05T17:39:37.798 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:24.858732+0000", 2020-12-05T17:39:37.799 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:39:37.799 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:39:37.799 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:39:37.799 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:39:37.799 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:39:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:39:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:39:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:39:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2020-12-05T17:39:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2020-12-05T17:39:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2020-12-05T17:39:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2020-12-05T17:39:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2020-12-05T17:39:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:39:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:39:37.802 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:39:37.802 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:39:37.802 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:39:37.802 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:39:37.802 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:39:37.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.803 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "16", 2020-12-05T17:39:37.803 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:39:37.803 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:39:37.803 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:39:37.804 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:39:37.804 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:39:37.804 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:39:37.804 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:39:37.804 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:39:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:39:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:39:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:39:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:39:37.806 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:39:37.806 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:39:37.806 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:39:37.806 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:39:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:39:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:39:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:39:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:39:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:39:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:39:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:39:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:39:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:39:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:39:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:39:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:39:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:39:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:39:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:39:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:39:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:39:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:39:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:39:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:39:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:39:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:39:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2020-12-05T17:39:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:39:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2020-12-05T17:39:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2020-12-05T17:39:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2020-12-05T17:39:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:33.305595+0000", 2020-12-05T17:39:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:39:37.814 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:39:37.814 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:39:37.814 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:39:37.814 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:39:37.814 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:39:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:39:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:39:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:39:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:39:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:39:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:39:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:39:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:39:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:39:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:39:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:39:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:39:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:39:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:39:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "19", 2020-12-05T17:39:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:39:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:39:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:39:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:39:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:39:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:39:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:39:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:39:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:39:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:39:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:39:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:39:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:39:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:39:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:39:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:39:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:39:37.822 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:39:37.822 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:39:37.822 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:39:37.822 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:39:37.822 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:39:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:39:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:39:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:39:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:39:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:39:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:39:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:39:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:39:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:39:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:39:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:39:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:39:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:39:37.826 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:39:37.826 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:39:37.826 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:39:37.826 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:39:37.827 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2020-12-05T17:39:37.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.828 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 17, 2020-12-05T17:39:37.828 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2020-12-05T17:39:37.828 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:39:19.993416+0000", 2020-12-05T17:39:37.828 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:39:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:39:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:39:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:39:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:39:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:39:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:39:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:39:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:39:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:39:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:39:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:39:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:39:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:39:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:39:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:39:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:39:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:39:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:39:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:39:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "91", 2020-12-05T17:39:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:39:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:39:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:39:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:39:37.834 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:39:37.834 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:39:37.834 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:39:37.834 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:39:37.834 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:39:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:39:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:39:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:39:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:39:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:39:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:39:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:39:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:39:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:39:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:39:37.837 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:39:37.837 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:39:37.837 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:39:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:39:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:39:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:39:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:39:37.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.839 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:39:37.839 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:39:37.839 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:39:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:39:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:39:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:39:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:39:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:39:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:39:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:39:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:39:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:39:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2020-12-05T17:39:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2020-12-05T17:39:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2020-12-05T17:39:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:39:37.843 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:39:37.843 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2020-12-05T17:39:37.843 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.843 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.844 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 18, 2020-12-05T17:39:37.844 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2020-12-05T17:39:37.844 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:39:21.252765+0000", 2020-12-05T17:39:37.844 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:39:37.845 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:39:37.845 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:39:37.845 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:39:37.845 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:39:37.845 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:39:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:39:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:39:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:39:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:39:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:39:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:39:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:39:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:39:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:39:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:39:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:39:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:39:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:39:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:39:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "92", 2020-12-05T17:39:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:39:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:39:37.850 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:39:37.850 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:39:37.850 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:39:37.850 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:39:37.850 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:39:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:39:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:39:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:39:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:39:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:39:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:39:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:39:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:39:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:39:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:39:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:39:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:39:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:39:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:39:37.854 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:39:37.854 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:39:37.854 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:39:37.855 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:39:37.855 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:39:37.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.855 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:39:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:39:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:39:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:39:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:39:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:39:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:39:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:39:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:39:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:39:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:39:37.858 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:39:37.858 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:39:37.858 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:39:37.859 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2020-12-05T17:39:37.859 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.859 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.859 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:39:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2020-12-05T17:39:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:39:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "b941127b-0a49-4e06-8d11-17794ff8b013", 2020-12-05T17:39:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:39:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:39:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:39:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:39:37.862 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:39:37.862 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:39:37.862 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:39:37.862 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 90, 2020-12-05T17:39:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:39:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:39:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:39:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2020-12-05T17:39:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:39:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2020-12-05T17:39:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:39:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.866 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.866 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:39:37.866 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2020-12-05T17:39:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:39:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2020-12-05T17:39:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:39:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:39:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2020-12-05T17:39:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:39:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2020-12-05T17:39:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:39:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:39:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2020-12-05T17:39:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:39:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2020-12-05T17:39:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:39:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6803/11526", 2020-12-05T17:39:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6805/11526", 2020-12-05T17:39:37.876 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6809/11526", 2020-12-05T17:39:37.876 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6807/11526", 2020-12-05T17:39:37.876 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:39:37.876 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:39:37.877 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:39:37.877 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.878 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:39:37.878 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "01bd7ff5-a86b-4e11-ba1e-484c309c0fba", 2020-12-05T17:39:37.878 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:39:37.879 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:39:37.879 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:39:37.879 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:39:37.880 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:39:37.880 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:39:37.880 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:39:37.880 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 90, 2020-12-05T17:39:37.880 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:39:37.881 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:39:37.881 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:39:37.881 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.882 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.882 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2020-12-05T17:39:37.882 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:39:37.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.883 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.883 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2020-12-05T17:39:37.883 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:39:37.884 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.884 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.884 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:39:37.885 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.885 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.885 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2020-12-05T17:39:37.885 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:39:37.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.886 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.886 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2020-12-05T17:39:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:39:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:39:37.888 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.888 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.888 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2020-12-05T17:39:37.889 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:39:37.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.889 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.890 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2020-12-05T17:39:37.890 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:39:37.890 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:39:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.892 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.892 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2020-12-05T17:39:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:39:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2020-12-05T17:39:37.894 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:39:37.894 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.894 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.894 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6819/11538", 2020-12-05T17:39:37.894 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6821/11538", 2020-12-05T17:39:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6825/11538", 2020-12-05T17:39:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6823/11538", 2020-12-05T17:39:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:39:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:39:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:39:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:39:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "94736ef5-64c2-4a97-b963-bd38cddd9c52", 2020-12-05T17:39:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:39:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:39:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:39:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:39:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:39:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:39:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:39:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 90, 2020-12-05T17:39:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:39:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:39:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:39:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2020-12-05T17:39:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:39:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2020-12-05T17:39:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:39:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:39:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2020-12-05T17:39:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:39:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2020-12-05T17:39:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:39:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:39:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2020-12-05T17:39:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:39:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2020-12-05T17:39:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:39:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:39:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2020-12-05T17:39:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:39:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2020-12-05T17:39:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:39:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6811/11521", 2020-12-05T17:39:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6813/11521", 2020-12-05T17:39:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6817/11521", 2020-12-05T17:39:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6815/11521", 2020-12-05T17:39:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:39:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:39:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:39:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:39:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "69a1f3ff-fd25-43df-b60c-e32a89c35234", 2020-12-05T17:39:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:39:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:39:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:39:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:39:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:39:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:39:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:39:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 90, 2020-12-05T17:39:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:39:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:39:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:39:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2020-12-05T17:39:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:39:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2020-12-05T17:39:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:39:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:39:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2020-12-05T17:39:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:39:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2020-12-05T17:39:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:39:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:39:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6832", 2020-12-05T17:39:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:39:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6833", 2020-12-05T17:39:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:39:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:39:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2020-12-05T17:39:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:39:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2020-12-05T17:39:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:39:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6827/11535", 2020-12-05T17:39:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6829/11535", 2020-12-05T17:39:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6833/11535", 2020-12-05T17:39:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6831/11535", 2020-12-05T17:39:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:39:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:39:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:39:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:39:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c301fe4e-5eab-402c-ae6f-db3e6a7181af", 2020-12-05T17:39:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:39:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:39:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:39:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:39:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:39:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:39:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:39:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 89, 2020-12-05T17:39:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:39:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:39:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:39:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6808", 2020-12-05T17:39:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:39:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6809", 2020-12-05T17:39:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:39:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:39:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6810", 2020-12-05T17:39:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:39:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6811", 2020-12-05T17:39:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:39:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:39:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6817", 2020-12-05T17:39:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:39:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6819", 2020-12-05T17:39:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:39:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:39:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6812", 2020-12-05T17:39:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:39:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6814", 2020-12-05T17:39:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:39:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6809/11541", 2020-12-05T17:39:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6811/11541", 2020-12-05T17:39:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6819/11541", 2020-12-05T17:39:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6814/11541", 2020-12-05T17:39:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:39:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:39:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:39:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:39:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "9476eff4-345b-46c5-a473-1d9a89a6d327", 2020-12-05T17:39:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:39:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:39:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:39:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:39:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:39:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:39:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:39:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 81, 2020-12-05T17:39:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:39:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:39:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:39:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6800", 2020-12-05T17:39:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:39:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6801", 2020-12-05T17:39:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:39:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:39:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6802", 2020-12-05T17:39:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:39:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6803", 2020-12-05T17:39:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:39:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:39:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6806", 2020-12-05T17:39:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:39:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6807", 2020-12-05T17:39:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:39:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:39:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6804", 2020-12-05T17:39:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:39:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6805", 2020-12-05T17:39:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:39:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6801/11539", 2020-12-05T17:39:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6803/11539", 2020-12-05T17:39:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6807/11539", 2020-12-05T17:39:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6805/11539", 2020-12-05T17:39:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:39:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:39:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:39:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:39:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8dfd8567-9b1c-4d13-9c96-071183e469c2", 2020-12-05T17:39:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:39:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:39:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:39:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:39:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:39:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:39:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:39:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 89, 2020-12-05T17:39:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:39:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:39:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:39:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6820", 2020-12-05T17:39:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:39:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6822", 2020-12-05T17:39:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:39:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:39:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6823", 2020-12-05T17:39:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:39:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6825", 2020-12-05T17:39:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:39:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:39:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6829", 2020-12-05T17:39:37.981 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:39:37.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.982 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.982 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6830", 2020-12-05T17:39:37.982 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:39:37.982 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.983 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.983 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:39:37.983 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.984 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.984 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6826", 2020-12-05T17:39:37.985 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:39:37.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.985 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.985 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6827", 2020-12-05T17:39:37.986 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:39:37.986 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.986 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.987 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6822/11542", 2020-12-05T17:39:37.987 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6825/11542", 2020-12-05T17:39:37.987 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6830/11542", 2020-12-05T17:39:37.987 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6827/11542", 2020-12-05T17:39:37.987 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:39:37.988 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:39:37.988 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:39:37.988 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.988 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:39:37.989 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "24d466b3-01dc-4ba2-8b81-85250d163bd2", 2020-12-05T17:39:37.989 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:39:37.989 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:39:37.989 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:39:37.989 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:39:37.990 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:39:37.990 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:39:37.990 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:39:37.990 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 90, 2020-12-05T17:39:37.990 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:39:37.991 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:39:37.991 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:39:37.991 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.991 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.992 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6813", 2020-12-05T17:39:37.992 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:39:37.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.992 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.993 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6815", 2020-12-05T17:39:37.993 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:39:37.993 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.993 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.994 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:39:37.994 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.994 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.994 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6816", 2020-12-05T17:39:37.995 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:39:37.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.995 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.995 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6818", 2020-12-05T17:39:37.996 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:39:37.996 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.996 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.996 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:39:37.996 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.997 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:37.997 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6828", 2020-12-05T17:39:37.997 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:39:37.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:37.998 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:37.998 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6831", 2020-12-05T17:39:37.998 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:39:37.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:37.999 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:37.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:37.999 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:39:37.999 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:39:37.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:38.000 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:39:38.000 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6821", 2020-12-05T17:39:38.000 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:39:38.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:38.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:38.001 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:39:38.001 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6824", 2020-12-05T17:39:38.001 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:39:38.001 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:38.002 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:38.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:38.002 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6815/11540", 2020-12-05T17:39:38.002 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6818/11540", 2020-12-05T17:39:38.002 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6831/11540", 2020-12-05T17:39:38.003 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6824/11540", 2020-12-05T17:39:38.003 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:39:38.003 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:39:38.003 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:39:38.003 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:38.004 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:38.004 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:39:38.004 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2020-12-05T17:39:38.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:38.004 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:39:38.005 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:39:38.005 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:39:38.006 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:39:38.006 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:39:38.006 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:39:38.006 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.398868+0000", 2020-12-05T17:39:38.007 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:39:38.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:38.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:38.007 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:39:38.007 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:39:38.008 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:39:38.008 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:39:38.008 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:39:38.008 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:39:38.008 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.675748+0000", 2020-12-05T17:39:38.009 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:39:38.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:38.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:38.009 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:39:38.009 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:39:38.010 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:39:38.010 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:39:38.010 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:39:38.010 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:39:38.010 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.620888+0000", 2020-12-05T17:39:38.011 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:39:38.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:38.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:38.011 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:39:38.012 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:39:38.012 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:39:38.012 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:39:38.012 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:39:38.012 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:39:38.013 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.681600+0000", 2020-12-05T17:39:38.013 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:39:38.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:38.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:38.013 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:39:38.014 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:39:38.014 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:39:38.014 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:39:38.014 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:39:38.014 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:39:38.015 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.658769+0000", 2020-12-05T17:39:38.015 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:39:38.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:38.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:38.016 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:39:38.016 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:39:38.016 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:39:38.016 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:39:38.016 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:39:38.017 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:39:38.017 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.630708+0000", 2020-12-05T17:39:38.017 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:39:38.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:38.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:38.018 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:39:38.018 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:39:38.018 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:39:38.018 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:39:38.018 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:39:38.019 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:39:38.019 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.648352+0000", 2020-12-05T17:39:38.019 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:39:38.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:38.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:38.020 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:39:38.020 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:39:38.020 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:39:38.020 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:39:38.021 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:39:38.021 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:39:38.021 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.629399+0000", 2020-12-05T17:39:38.021 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:39:38.021 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:38.022 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:39:38.022 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2020-12-05T17:39:38.022 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2020-12-05T17:39:38.022 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2020-12-05T17:39:38.023 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2020-12-05T17:39:38.023 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": {}, 2020-12-05T17:39:38.023 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2020-12-05T17:39:38.023 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2020-12-05T17:39:38.023 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2020-12-05T17:39:38.024 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:39:38.024 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2020-12-05T17:39:38.024 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2020-12-05T17:39:38.024 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:39:38.024 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2020-12-05T17:39:38.025 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:39:38.025 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:39:38.025 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2020-12-05T17:39:38.025 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2020-12-05T17:39:38.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:38.026 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2020-12-05T17:39:38.026 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:39:38.026 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:39:38.027 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2020-12-05T17:39:38.027 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:39:38.027 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:39:38.027 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2020-12-05T17:39:38.027 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:38.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:38.028 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2020-12-05T17:39:38.028 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2020-12-05T17:39:38.028 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2020-12-05T17:39:38.028 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2020-12-05T17:39:38.029 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2020-12-05T17:39:38.029 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:39:38.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.783+0000 7f9b6b07f700 1 -- 172.21.15.32:0/813183047 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9b440554e0 msgr2=0x7f9b440579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:38.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.783+0000 7f9b6b07f700 1 --2- 172.21.15.32:0/813183047 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9b440554e0 0x7f9b440579a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f9b58019000 tx=0x7f9b58015000).stop 2020-12-05T17:39:38.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.783+0000 7f9b6b07f700 1 -- 172.21.15.32:0/813183047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b640b7720 msgr2=0x7f9b64110900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:38.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.783+0000 7f9b6b07f700 1 --2- 172.21.15.32:0/813183047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b640b7720 0x7f9b64110900 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7f9b5c000f00 tx=0x7f9b5c007e10).stop 2020-12-05T17:39:38.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.783+0000 7f9b6b07f700 1 -- 172.21.15.32:0/813183047 shutdown_connections 2020-12-05T17:39:38.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.783+0000 7f9b6b07f700 1 --2- 172.21.15.32:0/813183047 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b64144540 0x7f9b640affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:38.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.783+0000 7f9b6b07f700 1 --2- 172.21.15.32:0/813183047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b640b7720 0x7f9b64110900 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:38.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.783+0000 7f9b6b07f700 1 --2- 172.21.15.32:0/813183047 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9b440554e0 0x7f9b440579a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:38.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.783+0000 7f9b6b07f700 1 --2- 172.21.15.32:0/813183047 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b641403f0 0x7f9b64110200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:38.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.783+0000 7f9b6b07f700 1 -- 172.21.15.32:0/813183047 >> 172.21.15.32:0/813183047 conn(0x7f9b6405bc20 msgr2=0x7f9b6405f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:38.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.783+0000 7f9b6b07f700 1 -- 172.21.15.32:0/813183047 shutdown_connections 2020-12-05T17:39:38.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:37.787+0000 7f9b6b07f700 1 -- 172.21.15.32:0/813183047 wait complete. 2020-12-05T17:39:38.033 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:39:38.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.151+0000 7f0d320f8700 1 Processor -- start 2020-12-05T17:39:38.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.151+0000 7f0d320f8700 1 -- start start 2020-12-05T17:39:38.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d320f8700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0d2c0b5f00 0x7f0d2c0b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:38.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d320f8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d2c0ad510 0x7f0d2c0af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:38.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d320f8700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d2c142ab0 con 0x7f0d2c066370 2020-12-05T17:39:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d320f8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d2c0b62e0 con 0x7f0d2c0ad510 2020-12-05T17:39:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d320f8700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d2c0afbf0 con 0x7f0d2c0b5f00 2020-12-05T17:39:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d308f5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d2c0ad510 0x7f0d2c0af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d308f5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d2c0ad510 0x7f0d2c0af480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38430/0 (socket says 172.21.15.32:38430) 2020-12-05T17:39:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d308f5700 1 -- 172.21.15.32:0/2370202014 learned_addr learned my addr 172.21.15.32:0/2370202014 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d310f6700 1 --2- 172.21.15.32:0/2370202014 >> v2:172.21.15.61:3300/0 conn(0x7f0d2c0b5f00 0x7f0d2c0b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d308f5700 1 -- 172.21.15.32:0/2370202014 >> v2:172.21.15.61:3300/0 conn(0x7f0d2c0b5f00 msgr2=0x7f0d2c0b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:38.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d308f5700 1 --2- 172.21.15.32:0/2370202014 >> v2:172.21.15.61:3300/0 conn(0x7f0d2c0b5f00 0x7f0d2c0b14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:38.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d308f5700 1 -- 172.21.15.32:0/2370202014 >> v1:172.21.15.61:6789/0 conn(0x7f0d2c066370 legacy=0x7f0d2c142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:38.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d308f5700 1 -- 172.21.15.32:0/2370202014 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d2c0afe90 con 0x7f0d2c0ad510 2020-12-05T17:39:38.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d308f5700 1 --2- 172.21.15.32:0/2370202014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d2c0ad510 0x7f0d2c0af480 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x7f0d1c002f30 tx=0x7f0d1c006e80).ready entity=mon.0 client_cookie=ebe113995c1afaf6 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:38.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d23fff700 1 -- 172.21.15.32:0/2370202014 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d1c0097f0 con 0x7f0d2c0ad510 2020-12-05T17:39:38.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d23fff700 1 -- 172.21.15.32:0/2370202014 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1c009950 con 0x7f0d2c0ad510 2020-12-05T17:39:38.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d23fff700 1 -- 172.21.15.32:0/2370202014 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d1c009cb0 con 0x7f0d2c0ad510 2020-12-05T17:39:38.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d320f8700 1 -- 172.21.15.32:0/2370202014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d2c0ad510 msgr2=0x7f0d2c0af480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:38.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d320f8700 1 --2- 172.21.15.32:0/2370202014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d2c0ad510 0x7f0d2c0af480 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x7f0d1c002f30 tx=0x7f0d1c006e80).stop 2020-12-05T17:39:38.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d320f8700 1 -- 172.21.15.32:0/2370202014 shutdown_connections 2020-12-05T17:39:38.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d320f8700 1 --2- 172.21.15.32:0/2370202014 >> v2:172.21.15.61:3300/0 conn(0x7f0d2c0b5f00 0x7f0d2c0b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:38.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d320f8700 1 --2- 172.21.15.32:0/2370202014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d2c0ad510 0x7f0d2c0af480 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:38.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d320f8700 1 -- 172.21.15.32:0/2370202014 >> 172.21.15.32:0/2370202014 conn(0x7f0d2c0604c0 msgr2=0x7f0d2c05c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:38.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d320f8700 1 -- 172.21.15.32:0/2370202014 shutdown_connections 2020-12-05T17:39:38.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.155+0000 7f0d320f8700 1 -- 172.21.15.32:0/2370202014 wait complete. 2020-12-05T17:39:38.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.159+0000 7f0d320f8700 1 Processor -- start 2020-12-05T17:39:38.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.159+0000 7f0d320f8700 1 -- start start 2020-12-05T17:39:38.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.159+0000 7f0d320f8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d2c066370 0x7f0d2c1409b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:38.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.159+0000 7f0d320f8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0d2c0ad510 0x7f0d2c147150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:38.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.159+0000 7f0d320f8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0d2c0b5f00 0x7f0d2c155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:38.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.159+0000 7f0d320f8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c0b5a00 con 0x7f0d2c066370 2020-12-05T17:39:38.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.159+0000 7f0d320f8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c0b4440 con 0x7f0d2c0b5f00 2020-12-05T17:39:38.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.159+0000 7f0d320f8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c0b3f00 con 0x7f0d2c0ad510 2020-12-05T17:39:38.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.159+0000 7f0d310f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d2c066370 0x7f0d2c1409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:38.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.159+0000 7f0d310f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d2c066370 0x7f0d2c1409b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38434/0 (socket says 172.21.15.32:38434) 2020-12-05T17:39:38.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.159+0000 7f0d310f6700 1 -- 172.21.15.32:0/2505145776 learned_addr learned my addr 172.21.15.32:0/2505145776 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:38.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.159+0000 7f0d308f5700 1 --2- 172.21.15.32:0/2505145776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0d2c0ad510 0x7f0d2c147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:38.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.159+0000 7f0d318f7700 1 --2- 172.21.15.32:0/2505145776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0d2c0b5f00 0x7f0d2c155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:38.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.159+0000 7f0d310f6700 1 -- 172.21.15.32:0/2505145776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0d2c0ad510 msgr2=0x7f0d2c147150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:38.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.159+0000 7f0d310f6700 1 --2- 172.21.15.32:0/2505145776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0d2c0ad510 0x7f0d2c147150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:38.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.159+0000 7f0d310f6700 1 -- 172.21.15.32:0/2505145776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0d2c0b5f00 msgr2=0x7f0d2c155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:38.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.159+0000 7f0d310f6700 1 --2- 172.21.15.32:0/2505145776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0d2c0b5f00 0x7f0d2c155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:38.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.159+0000 7f0d310f6700 1 -- 172.21.15.32:0/2505145776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d2c0b41a0 con 0x7f0d2c066370 2020-12-05T17:39:38.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.159+0000 7f0d310f6700 1 --2- 172.21.15.32:0/2505145776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d2c066370 0x7f0d2c1409b0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7f0d2800c000 tx=0x7f0d28006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:38.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.159+0000 7f0d227fc700 1 -- 172.21.15.32:0/2505145776 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d2800ca10 con 0x7f0d2c066370 2020-12-05T17:39:38.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.159+0000 7f0d227fc700 1 -- 172.21.15.32:0/2505145776 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2800cb70 con 0x7f0d2c066370 2020-12-05T17:39:38.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.159+0000 7f0d320f8700 1 -- 172.21.15.32:0/2505145776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d2c0b2c50 con 0x7f0d2c066370 2020-12-05T17:39:38.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.159+0000 7f0d227fc700 1 -- 172.21.15.32:0/2505145776 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d2800be50 con 0x7f0d2c066370 2020-12-05T17:39:38.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.163+0000 7f0d320f8700 1 -- 172.21.15.32:0/2505145776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d2c0b3c80 con 0x7f0d2c066370 2020-12-05T17:39:38.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.163+0000 7f0d227fc700 1 -- 172.21.15.32:0/2505145776 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0d2801c460 con 0x7f0d2c066370 2020-12-05T17:39:38.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.163+0000 7f0d320f8700 1 -- 172.21.15.32:0/2505145776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d2c154ff0 con 0x7f0d2c066370 2020-12-05T17:39:38.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.163+0000 7f0d227fc700 1 --2- 172.21.15.32:0/2505145776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0d140553b0 0x7f0d14057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:38.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.163+0000 7f0d227fc700 1 -- 172.21.15.32:0/2505145776 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0d2806e450 con 0x7f0d2c066370 2020-12-05T17:39:38.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.163+0000 7f0d308f5700 1 --2- 172.21.15.32:0/2505145776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0d140553b0 0x7f0d14057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:38.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.163+0000 7f0d308f5700 1 --2- 172.21.15.32:0/2505145776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0d140553b0 0x7f0d14057870 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f0d1c01c000 tx=0x7f0d1c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:38.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.171+0000 7f0d227fc700 1 -- 172.21.15.32:0/2505145776 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0d28044590 con 0x7f0d2c066370 2020-12-05T17:39:38.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.363+0000 7f0d320f8700 1 -- 172.21.15.32:0/2505145776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d2c057050 con 0x7f0d2c066370 2020-12-05T17:39:38.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.363+0000 7f0d227fc700 1 -- 172.21.15.32:0/2505145776 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f0d28047bb0 con 0x7f0d2c066370 2020-12-05T17:39:38.369 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:38.369 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":42,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6823,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2664846066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2664846066},{"type":"v1","addr":"172.21.15.61:6833","nonce":2664846066}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:39:22.261725+0000","modified":"2020-12-05T17:39:23.265348+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6930":{"gid":6930,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4087622496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4087622496},{"type":"v1","addr":"172.21.15.32:6835","nonce":4087622496}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2020-12-05T17:39:38.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.363+0000 7f0d320f8700 1 -- 172.21.15.32:0/2505145776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0d140553b0 msgr2=0x7f0d14057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:38.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.363+0000 7f0d320f8700 1 --2- 172.21.15.32:0/2505145776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0d140553b0 0x7f0d14057870 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f0d1c01c000 tx=0x7f0d1c015000).stop 2020-12-05T17:39:38.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.363+0000 7f0d320f8700 1 -- 172.21.15.32:0/2505145776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d2c066370 msgr2=0x7f0d2c1409b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:38.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.363+0000 7f0d320f8700 1 --2- 172.21.15.32:0/2505145776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d2c066370 0x7f0d2c1409b0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7f0d2800c000 tx=0x7f0d28006dc0).stop 2020-12-05T17:39:38.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.363+0000 7f0d320f8700 1 -- 172.21.15.32:0/2505145776 shutdown_connections 2020-12-05T17:39:38.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.363+0000 7f0d320f8700 1 --2- 172.21.15.32:0/2505145776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0d2c0b5f00 0x7f0d2c155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:38.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.363+0000 7f0d320f8700 1 --2- 172.21.15.32:0/2505145776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d2c066370 0x7f0d2c1409b0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:38.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.363+0000 7f0d320f8700 1 --2- 172.21.15.32:0/2505145776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0d140553b0 0x7f0d14057870 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:38.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.363+0000 7f0d320f8700 1 --2- 172.21.15.32:0/2505145776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0d2c0ad510 0x7f0d2c147150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:38.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.363+0000 7f0d320f8700 1 -- 172.21.15.32:0/2505145776 >> 172.21.15.32:0/2505145776 conn(0x7f0d2c0604c0 msgr2=0x7f0d2c13f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:38.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.367+0000 7f0d320f8700 1 -- 172.21.15.32:0/2505145776 shutdown_connections 2020-12-05T17:39:38.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.367+0000 7f0d320f8700 1 -- 172.21.15.32:0/2505145776 wait complete. 2020-12-05T17:39:38.373 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 42 2020-12-05T17:39:38.382 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-12-05T17:39:38.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.495+0000 7f47c1b23700 1 Processor -- start 2020-12-05T17:39:38.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.495+0000 7f47c1b23700 1 -- start start 2020-12-05T17:39:38.503 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.495+0000 7f47c1b23700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f47bc0b1fc0 0x7f47bc0af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:38.503 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.495+0000 7f47c1b23700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f47bc147400 0x7f47bc145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:38.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.495+0000 7f47c1b23700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47bc146a90 con 0x7f47bc0ab710 2020-12-05T17:39:38.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.495+0000 7f47c1b23700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47bc061e90 con 0x7f47bc0b1fc0 2020-12-05T17:39:38.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.495+0000 7f47c1b23700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc0b0110 con 0x7f47bc147400 2020-12-05T17:39:38.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.495+0000 7f47c0b21700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f47bc0b1fc0 0x7f47bc0af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:38.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.495+0000 7f47c0b21700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f47bc0b1fc0 0x7f47bc0af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38444/0 (socket says 172.21.15.32:38444) 2020-12-05T17:39:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.495+0000 7f47c0b21700 1 -- 172.21.15.32:0/2742606903 learned_addr learned my addr 172.21.15.32:0/2742606903 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.495+0000 7f47c1322700 1 --2- 172.21.15.32:0/2742606903 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f47bc147400 0x7f47bc145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.495+0000 7f47c0b21700 1 -- 172.21.15.32:0/2742606903 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f47bc147400 msgr2=0x7f47bc145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.495+0000 7f47c0b21700 1 --2- 172.21.15.32:0/2742606903 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f47bc147400 0x7f47bc145d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.495+0000 7f47c0b21700 1 -- 172.21.15.32:0/2742606903 >> v1:172.21.15.61:6789/0 conn(0x7f47bc0ab710 legacy=0x7f47bc061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.495+0000 7f47c0b21700 1 -- 172.21.15.32:0/2742606903 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47bc0b23a0 con 0x7f47bc0b1fc0 2020-12-05T17:39:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.495+0000 7f47c0b21700 1 --2- 172.21.15.32:0/2742606903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47bc0b1fc0 0x7f47bc0af230 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7f47ac002f30 tx=0x7f47ac006e80).ready entity=mon.0 client_cookie=c07743852a09e95b server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.495+0000 7f47bb7fe700 1 -- 172.21.15.32:0/2742606903 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47ac0097f0 con 0x7f47bc0b1fc0 2020-12-05T17:39:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.495+0000 7f47bb7fe700 1 -- 172.21.15.32:0/2742606903 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ac009950 con 0x7f47bc0b1fc0 2020-12-05T17:39:38.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.495+0000 7f47bb7fe700 1 -- 172.21.15.32:0/2742606903 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47ac009cb0 con 0x7f47bc0b1fc0 2020-12-05T17:39:38.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.495+0000 7f47c1b23700 1 -- 172.21.15.32:0/2742606903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47bc0b1fc0 msgr2=0x7f47bc0af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:38.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.495+0000 7f47c1b23700 1 --2- 172.21.15.32:0/2742606903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47bc0b1fc0 0x7f47bc0af230 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7f47ac002f30 tx=0x7f47ac006e80).stop 2020-12-05T17:39:38.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47c1b23700 1 -- 172.21.15.32:0/2742606903 shutdown_connections 2020-12-05T17:39:38.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47c1b23700 1 --2- 172.21.15.32:0/2742606903 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f47bc147400 0x7f47bc145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:38.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47c1b23700 1 --2- 172.21.15.32:0/2742606903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47bc0b1fc0 0x7f47bc0af230 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:38.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47c1b23700 1 -- 172.21.15.32:0/2742606903 >> 172.21.15.32:0/2742606903 conn(0x7f47bc0b5530 msgr2=0x7f47bc0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:38.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47c1b23700 1 -- 172.21.15.32:0/2742606903 shutdown_connections 2020-12-05T17:39:38.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47c1b23700 1 -- 172.21.15.32:0/2742606903 wait complete. 2020-12-05T17:39:38.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47c1b23700 1 Processor -- start 2020-12-05T17:39:38.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47c1b23700 1 -- start start 2020-12-05T17:39:38.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47c1b23700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f47bc0ab710 0x7f47bc0b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:38.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47c1b23700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47bc0b1fc0 0x7f47bc158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:38.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47c1b23700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f47bc147400 0x7f47bc14a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:38.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47c1b23700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc14a0a0 con 0x7f47bc0b1fc0 2020-12-05T17:39:38.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47c1b23700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc0ad670 con 0x7f47bc147400 2020-12-05T17:39:38.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47c1b23700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc0ad910 con 0x7f47bc0ab710 2020-12-05T17:39:38.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47bbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47bc0b1fc0 0x7f47bc158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:38.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47bbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47bc0b1fc0 0x7f47bc158d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38448/0 (socket says 172.21.15.32:38448) 2020-12-05T17:39:38.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47bbfff700 1 -- 172.21.15.32:0/2853621963 learned_addr learned my addr 172.21.15.32:0/2853621963 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:38.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47c1322700 1 --2- 172.21.15.32:0/2853621963 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f47bc147400 0x7f47bc14a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:38.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47c0b21700 1 --2- 172.21.15.32:0/2853621963 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f47bc0ab710 0x7f47bc0b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:38.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47bbfff700 1 -- 172.21.15.32:0/2853621963 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f47bc0ab710 msgr2=0x7f47bc0b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:38.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47bbfff700 1 --2- 172.21.15.32:0/2853621963 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f47bc0ab710 0x7f47bc0b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:38.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47bbfff700 1 -- 172.21.15.32:0/2853621963 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f47bc147400 msgr2=0x7f47bc14a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:38.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47bbfff700 1 --2- 172.21.15.32:0/2853621963 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f47bc147400 0x7f47bc14a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:38.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.499+0000 7f47bbfff700 1 -- 172.21.15.32:0/2853621963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47bc0ad2f0 con 0x7f47bc0b1fc0 2020-12-05T17:39:38.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.503+0000 7f47bbfff700 1 --2- 172.21.15.32:0/2853621963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47bc0b1fc0 0x7f47bc158d80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 rx=0x7f47bc0608f0 tx=0x7f47b0007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:38.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.503+0000 7f47b9ffb700 1 -- 172.21.15.32:0/2853621963 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47b0013410 con 0x7f47bc0b1fc0 2020-12-05T17:39:38.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.503+0000 7f47b9ffb700 1 -- 172.21.15.32:0/2853621963 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b0013570 con 0x7f47bc0b1fc0 2020-12-05T17:39:38.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.503+0000 7f47c1b23700 1 -- 172.21.15.32:0/2853621963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47bc147e50 con 0x7f47bc0b1fc0 2020-12-05T17:39:38.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.503+0000 7f47b9ffb700 1 -- 172.21.15.32:0/2853621963 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47b00138d0 con 0x7f47bc0b1fc0 2020-12-05T17:39:38.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.503+0000 7f47c1b23700 1 -- 172.21.15.32:0/2853621963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47bc0ae840 con 0x7f47bc0b1fc0 2020-12-05T17:39:38.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.503+0000 7f47c1b23700 1 -- 172.21.15.32:0/2853621963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47bc0ad4e0 con 0x7f47bc0b1fc0 2020-12-05T17:39:38.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.503+0000 7f47b9ffb700 1 -- 172.21.15.32:0/2853621963 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f47b0010030 con 0x7f47bc0b1fc0 2020-12-05T17:39:38.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.503+0000 7f47b9ffb700 1 --2- 172.21.15.32:0/2853621963 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f479c055330 0x7f479c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:38.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.503+0000 7f47b9ffb700 1 -- 172.21.15.32:0/2853621963 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f47b006de50 con 0x7f47bc0b1fc0 2020-12-05T17:39:38.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.503+0000 7f47c0b21700 1 --2- 172.21.15.32:0/2853621963 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f479c055330 0x7f479c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:38.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.503+0000 7f47c0b21700 1 --2- 172.21.15.32:0/2853621963 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f479c055330 0x7f479c0577f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f47ac01c000 tx=0x7f47ac015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:38.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.511+0000 7f47b9ffb700 1 -- 172.21.15.32:0/2853621963 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f47b0043ff0 con 0x7f47bc0b1fc0 2020-12-05T17:39:38.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:38.699+0000 7f47c1b23700 1 -- 172.21.15.32:0/2853621963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f47bc057050 con 0x7f47bc0b1fc0 2020-12-05T17:39:39.042 INFO:tasks.ceph.mon.a.smithi032.stderr:2020-12-05T17:39:39.039+0000 7fdef8136700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-12-05T17:39:39.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.043+0000 7f47b9ffb700 1 -- 172.21.15.32:0/2853621963 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) v1 ==== 161+0+0 (secure 0 0 0) 0x7f47b0047610 con 0x7f47bc0b1fc0 2020-12-05T17:39:39.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.047+0000 7f47c1b23700 1 -- 172.21.15.32:0/2853621963 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f479c055330 msgr2=0x7f479c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:39.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.047+0000 7f47c1b23700 1 --2- 172.21.15.32:0/2853621963 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f479c055330 0x7f479c0577f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f47ac01c000 tx=0x7f47ac015000).stop 2020-12-05T17:39:39.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.047+0000 7f47c1b23700 1 -- 172.21.15.32:0/2853621963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47bc0b1fc0 msgr2=0x7f47bc158d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:39.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.047+0000 7f47c1b23700 1 --2- 172.21.15.32:0/2853621963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47bc0b1fc0 0x7f47bc158d80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 rx=0x7f47bc0608f0 tx=0x7f47b0007c20).stop 2020-12-05T17:39:39.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.047+0000 7f47c1b23700 1 -- 172.21.15.32:0/2853621963 shutdown_connections 2020-12-05T17:39:39.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.047+0000 7f47c1b23700 1 --2- 172.21.15.32:0/2853621963 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f47bc147400 0x7f47bc14a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:39.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.047+0000 7f47c1b23700 1 --2- 172.21.15.32:0/2853621963 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f479c055330 0x7f479c0577f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:39.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.047+0000 7f47c1b23700 1 --2- 172.21.15.32:0/2853621963 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f47bc0ab710 0x7f47bc0b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:39.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.047+0000 7f47c1b23700 1 --2- 172.21.15.32:0/2853621963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47bc0b1fc0 0x7f47bc158d80 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:39.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.047+0000 7f47c1b23700 1 -- 172.21.15.32:0/2853621963 >> 172.21.15.32:0/2853621963 conn(0x7f47bc0b5530 msgr2=0x7f47bc061780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:39.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.051+0000 7f47c1b23700 1 -- 172.21.15.32:0/2853621963 shutdown_connections 2020-12-05T17:39:39.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.051+0000 7f47c1b23700 1 -- 172.21.15.32:0/2853621963 wait complete. 2020-12-05T17:39:39.058 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-12-05T17:39:39.074 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:39:39.077 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:39:39.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.175+0000 7fba1cc1c700 1 Processor -- start 2020-12-05T17:39:39.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.175+0000 7fba1cc1c700 1 -- start start 2020-12-05T17:39:39.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.175+0000 7fba1cc1c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fba180ab710 0x7fba180af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:39.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.175+0000 7fba1cc1c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba18146a90 con 0x7fba180b1fc0 2020-12-05T17:39:39.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.175+0000 7fba1cc1c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba180b23a0 con 0x7fba18147400 2020-12-05T17:39:39.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.175+0000 7fba1cc1c700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba18061e90 con 0x7fba180ab710 2020-12-05T17:39:39.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.175+0000 7fba177fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fba180b1fc0 0x7fba18061780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41116/0 (socket says 172.21.15.32:41116) 2020-12-05T17:39:39.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.175+0000 7fba177fe700 1 -- 172.21.15.32:0/8249557 learned_addr learned my addr 172.21.15.32:0/8249557 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:39.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.175+0000 7fba16ffd700 1 --2- 172.21.15.32:0/8249557 >> v2:172.21.15.61:3300/0 conn(0x7fba180ab710 0x7fba180af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:39.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.175+0000 7fba167fc700 1 -- 172.21.15.32:0/8249557 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 580152407 0 0) 0x7fba18146a90 con 0x7fba180b1fc0 2020-12-05T17:39:39.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.175+0000 7fba167fc700 1 -- 172.21.15.32:0/8249557 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9f8003230 con 0x7fba180b1fc0 2020-12-05T17:39:39.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.175+0000 7fba167fc700 1 -- 172.21.15.32:0/8249557 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251321158 0 0) 0x7fba180b23a0 con 0x7fba18147400 2020-12-05T17:39:39.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.175+0000 7fba167fc700 1 -- 172.21.15.32:0/8249557 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba18146a90 con 0x7fba18147400 2020-12-05T17:39:39.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.175+0000 7fba16ffd700 1 -- 172.21.15.32:0/8249557 >> v1:172.21.15.61:6789/0 conn(0x7fba18147400 legacy=0x7fba18145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:39.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.175+0000 7fba16ffd700 1 -- 172.21.15.32:0/8249557 >> v1:172.21.15.32:6789/0 conn(0x7fba180b1fc0 legacy=0x7fba18061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:39.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.175+0000 7fba16ffd700 1 -- 172.21.15.32:0/8249557 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba18146cc0 con 0x7fba180ab710 2020-12-05T17:39:39.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.175+0000 7fba16ffd700 1 --2- 172.21.15.32:0/8249557 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba180ab710 0x7fba180af230 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fba080032b0 tx=0x7fba080091a0).ready entity=mon.1 client_cookie=6bf0f4e180f7e4a6 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:39.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.175+0000 7fba167fc700 1 -- 172.21.15.32:0/8249557 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba0800b940 con 0x7fba180ab710 2020-12-05T17:39:39.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.175+0000 7fba167fc700 1 -- 172.21.15.32:0/8249557 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0800baa0 con 0x7fba180ab710 2020-12-05T17:39:39.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.175+0000 7fba167fc700 1 -- 172.21.15.32:0/8249557 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba0800bc10 con 0x7fba180ab710 2020-12-05T17:39:39.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.175+0000 7fba1cc1c700 1 -- 172.21.15.32:0/8249557 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba180ab710 msgr2=0x7fba180af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:39.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.175+0000 7fba1cc1c700 1 --2- 172.21.15.32:0/8249557 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba180ab710 0x7fba180af230 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fba080032b0 tx=0x7fba080091a0).stop 2020-12-05T17:39:39.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba1cc1c700 1 -- 172.21.15.32:0/8249557 shutdown_connections 2020-12-05T17:39:39.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba1cc1c700 1 --2- 172.21.15.32:0/8249557 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba180ab710 0x7fba180af230 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:39.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba1cc1c700 1 -- 172.21.15.32:0/8249557 >> 172.21.15.32:0/8249557 conn(0x7fba180b5530 msgr2=0x7fba180b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:39.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba1cc1c700 1 -- 172.21.15.32:0/8249557 shutdown_connections 2020-12-05T17:39:39.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba1cc1c700 1 -- 172.21.15.32:0/8249557 wait complete. 2020-12-05T17:39:39.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba1cc1c700 1 Processor -- start 2020-12-05T17:39:39.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba1cc1c700 1 -- start start 2020-12-05T17:39:39.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba1cc1c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba180ab710 0x7fba18144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:39.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba1cc1c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fba180b1fc0 0x7fba180664c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:39.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba1cc1c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba18147400 0x7fba180669b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:39.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba1cc1c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba180ae7f0 con 0x7fba180ab710 2020-12-05T17:39:39.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba1cc1c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba18066ea0 con 0x7fba18147400 2020-12-05T17:39:39.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba1cc1c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1809e4a0 con 0x7fba180b1fc0 2020-12-05T17:39:39.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba180ab710 0x7fba18144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:39.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba180ab710 0x7fba18144bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38474/0 (socket says 172.21.15.32:38474) 2020-12-05T17:39:39.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba177fe700 1 -- 172.21.15.32:0/1280924136 learned_addr learned my addr 172.21.15.32:0/1280924136 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:39.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba17fff700 1 --2- 172.21.15.32:0/1280924136 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba18147400 0x7fba180669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:39.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba16ffd700 1 --2- 172.21.15.32:0/1280924136 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fba180b1fc0 0x7fba180664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:39.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba177fe700 1 -- 172.21.15.32:0/1280924136 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fba180b1fc0 msgr2=0x7fba180664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:39.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba177fe700 1 --2- 172.21.15.32:0/1280924136 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fba180b1fc0 0x7fba180664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:39.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba177fe700 1 -- 172.21.15.32:0/1280924136 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba18147400 msgr2=0x7fba180669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:39.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba177fe700 1 --2- 172.21.15.32:0/1280924136 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba18147400 0x7fba180669b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:39.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba177fe700 1 -- 172.21.15.32:0/1280924136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba180aea20 con 0x7fba180ab710 2020-12-05T17:39:39.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba17fff700 1 --2- 172.21.15.32:0/1280924136 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba18147400 0x7fba180669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:39.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba177fe700 1 --2- 172.21.15.32:0/1280924136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba180ab710 0x7fba18144bd0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7fba10001090 tx=0x7fba10007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:39.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba14ff9700 1 -- 172.21.15.32:0/1280924136 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba10007320 con 0x7fba180ab710 2020-12-05T17:39:39.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba14ff9700 1 -- 172.21.15.32:0/1280924136 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba10007480 con 0x7fba180ab710 2020-12-05T17:39:39.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba1cc1c700 1 -- 172.21.15.32:0/1280924136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1809e120 con 0x7fba180ab710 2020-12-05T17:39:39.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.179+0000 7fba14ff9700 1 -- 172.21.15.32:0/1280924136 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba100057d0 con 0x7fba180ab710 2020-12-05T17:39:39.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.183+0000 7fba1cc1c700 1 -- 172.21.15.32:0/1280924136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba18158e00 con 0x7fba180ab710 2020-12-05T17:39:39.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.183+0000 7fba14ff9700 1 -- 172.21.15.32:0/1280924136 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fba10010030 con 0x7fba180ab710 2020-12-05T17:39:39.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.183+0000 7fba1cc1c700 1 -- 172.21.15.32:0/1280924136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba18057050 con 0x7fba180ab710 2020-12-05T17:39:39.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.183+0000 7fba14ff9700 1 --2- 172.21.15.32:0/1280924136 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb9fc055330 0x7fb9fc0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:39.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.183+0000 7fba14ff9700 1 -- 172.21.15.32:0/1280924136 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fba1006dcc0 con 0x7fba180ab710 2020-12-05T17:39:39.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.183+0000 7fba16ffd700 1 --2- 172.21.15.32:0/1280924136 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb9fc055330 0x7fb9fc0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:39.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.183+0000 7fba16ffd700 1 --2- 172.21.15.32:0/1280924136 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb9fc055330 0x7fb9fc0577f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7fba08019000 tx=0x7fba08016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:39.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.191+0000 7fba14ff9700 1 -- 172.21.15.32:0/1280924136 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fba10043e10 con 0x7fba180ab710 2020-12-05T17:39:39.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.375+0000 7fba1cc1c700 1 -- 172.21.15.32:0/1280924136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba1809e310 con 0x7fba180ab710 2020-12-05T17:39:39.381 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.375+0000 7fba14ff9700 1 -- 172.21.15.32:0/1280924136 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+1779 (secure 0 0 0) 0x7fba10047430 con 0x7fba180ab710 2020-12-05T17:39:39.382 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:39.382 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":43,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6823,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2664846066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2664846066},{"type":"v1","addr":"172.21.15.61:6833","nonce":2664846066}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40}],"filesystems":[{"mdsmap":{"epoch":43,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:39:22.261725+0000","modified":"2020-12-05T17:39:39.041540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2020-12-05T17:39:39.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.375+0000 7fba1cc1c700 1 -- 172.21.15.32:0/1280924136 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb9fc055330 msgr2=0x7fb9fc0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:39.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.375+0000 7fba1cc1c700 1 --2- 172.21.15.32:0/1280924136 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb9fc055330 0x7fb9fc0577f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7fba08019000 tx=0x7fba08016000).stop 2020-12-05T17:39:39.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.375+0000 7fba1cc1c700 1 -- 172.21.15.32:0/1280924136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba180ab710 msgr2=0x7fba18144bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:39.385 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.375+0000 7fba1cc1c700 1 --2- 172.21.15.32:0/1280924136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba180ab710 0x7fba18144bd0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7fba10001090 tx=0x7fba10007ad0).stop 2020-12-05T17:39:39.385 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.375+0000 7fba1cc1c700 1 -- 172.21.15.32:0/1280924136 shutdown_connections 2020-12-05T17:39:39.385 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.375+0000 7fba1cc1c700 1 --2- 172.21.15.32:0/1280924136 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba18147400 0x7fba180669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:39.385 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.379+0000 7fba1cc1c700 1 --2- 172.21.15.32:0/1280924136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba180ab710 0x7fba18144bd0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:39.385 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.379+0000 7fba1cc1c700 1 --2- 172.21.15.32:0/1280924136 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb9fc055330 0x7fb9fc0577f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:39.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.379+0000 7fba1cc1c700 1 --2- 172.21.15.32:0/1280924136 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fba180b1fc0 0x7fba180664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:39.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.379+0000 7fba1cc1c700 1 -- 172.21.15.32:0/1280924136 >> 172.21.15.32:0/1280924136 conn(0x7fba180b5530 msgr2=0x7fba180af4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:39.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.379+0000 7fba1cc1c700 1 -- 172.21.15.32:0/1280924136 shutdown_connections 2020-12-05T17:39:39.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.379+0000 7fba1cc1c700 1 -- 172.21.15.32:0/1280924136 wait complete. 2020-12-05T17:39:39.387 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 43 2020-12-05T17:39:39.395 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-12-05T17:39:39.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.551+0000 7f32267d5700 1 Processor -- start 2020-12-05T17:39:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.551+0000 7f32267d5700 1 -- start start 2020-12-05T17:39:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.551+0000 7f32267d5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f32200b6690 0x7f32200b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:39.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.551+0000 7f32267d5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32200b56f0 con 0x7f32200b5240 2020-12-05T17:39:39.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.551+0000 7f32267d5700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32200b1920 con 0x7f32200b5b20 2020-12-05T17:39:39.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.551+0000 7f32267d5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32200b1bc0 con 0x7f32200b6690 2020-12-05T17:39:39.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.555+0000 7f3224d72700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f32200b5240 0x7f3220145f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41134/0 (socket says 172.21.15.32:41134) 2020-12-05T17:39:39.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.555+0000 7f321f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f32200b6690 0x7f32200b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:39.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.555+0000 7f3224d72700 1 -- 172.21.15.32:0/2120612916 learned_addr learned my addr 172.21.15.32:0/2120612916 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:39.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.555+0000 7f321effd700 1 -- 172.21.15.32:0/2120612916 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441763399 0 0) 0x7f32200b1920 con 0x7f32200b5b20 2020-12-05T17:39:39.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.555+0000 7f321effd700 1 -- 172.21.15.32:0/2120612916 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3200003230 con 0x7f32200b5b20 2020-12-05T17:39:39.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.555+0000 7f321f7fe700 1 -- 172.21.15.32:0/2120612916 >> v1:172.21.15.61:6789/0 conn(0x7f32200b5b20 legacy=0x7f32200b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:39.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.555+0000 7f321f7fe700 1 -- 172.21.15.32:0/2120612916 >> v1:172.21.15.32:6789/0 conn(0x7f32200b5240 legacy=0x7f3220145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:39.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.555+0000 7f321f7fe700 1 -- 172.21.15.32:0/2120612916 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3220146620 con 0x7f32200b6690 2020-12-05T17:39:39.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.555+0000 7f321f7fe700 1 --2- 172.21.15.32:0/2120612916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32200b6690 0x7f32200b5f00 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 rx=0x7f3208002f30 tx=0x7f3208006dc0).ready entity=mon.0 client_cookie=adc48da24f4dd7e8 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:39.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.555+0000 7f321effd700 1 -- 172.21.15.32:0/2120612916 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3208009800 con 0x7f32200b6690 2020-12-05T17:39:39.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.555+0000 7f321effd700 1 -- 172.21.15.32:0/2120612916 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3208009960 con 0x7f32200b6690 2020-12-05T17:39:39.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.555+0000 7f321effd700 1 -- 172.21.15.32:0/2120612916 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3208009cc0 con 0x7f32200b6690 2020-12-05T17:39:39.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.555+0000 7f32267d5700 1 -- 172.21.15.32:0/2120612916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32200b6690 msgr2=0x7f32200b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:39.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.555+0000 7f32267d5700 1 --2- 172.21.15.32:0/2120612916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32200b6690 0x7f32200b5f00 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 rx=0x7f3208002f30 tx=0x7f3208006dc0).stop 2020-12-05T17:39:39.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.555+0000 7f32267d5700 1 -- 172.21.15.32:0/2120612916 shutdown_connections 2020-12-05T17:39:39.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.555+0000 7f32267d5700 1 --2- 172.21.15.32:0/2120612916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32200b6690 0x7f32200b5f00 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:39.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.555+0000 7f32267d5700 1 -- 172.21.15.32:0/2120612916 >> 172.21.15.32:0/2120612916 conn(0x7f322005f3b0 msgr2=0x7f322005f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:39.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.555+0000 7f32267d5700 1 -- 172.21.15.32:0/2120612916 shutdown_connections 2020-12-05T17:39:39.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.555+0000 7f32267d5700 1 -- 172.21.15.32:0/2120612916 wait complete. 2020-12-05T17:39:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.555+0000 7f32267d5700 1 Processor -- start 2020-12-05T17:39:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.555+0000 7f32267d5700 1 -- start start 2020-12-05T17:39:39.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f32267d5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32200b5240 0x7f32200645b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:39.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f32267d5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32200b5b20 0x7f3220141010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:39.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f32267d5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32200b6690 0x7f322013f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:39.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f32267d5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322013e8d0 con 0x7f32200b5b20 2020-12-05T17:39:39.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f32267d5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322013c450 con 0x7f32200b6690 2020-12-05T17:39:39.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f32267d5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32201408d0 con 0x7f32200b5240 2020-12-05T17:39:39.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f321f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32200b5b20 0x7f3220141010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:39.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f321f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32200b5b20 0x7f3220141010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38490/0 (socket says 172.21.15.32:38490) 2020-12-05T17:39:39.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f321f7fe700 1 -- 172.21.15.32:0/812420909 learned_addr learned my addr 172.21.15.32:0/812420909 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:39.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f3224d72700 1 --2- 172.21.15.32:0/812420909 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32200b6690 0x7f322013f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:39.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f321ffff700 1 --2- 172.21.15.32:0/812420909 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32200b5240 0x7f32200645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:39.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f321f7fe700 1 -- 172.21.15.32:0/812420909 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32200b5240 msgr2=0x7f32200645b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:39.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f321f7fe700 1 --2- 172.21.15.32:0/812420909 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32200b5240 0x7f32200645b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:39.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f321f7fe700 1 -- 172.21.15.32:0/812420909 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32200b6690 msgr2=0x7f322013f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:39.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f321f7fe700 1 --2- 172.21.15.32:0/812420909 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32200b6690 0x7f322013f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:39.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f321f7fe700 1 -- 172.21.15.32:0/812420909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3220140b70 con 0x7f32200b5b20 2020-12-05T17:39:39.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f3224d72700 1 --2- 172.21.15.32:0/812420909 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32200b6690 0x7f322013f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:39.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f321ffff700 1 --2- 172.21.15.32:0/812420909 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32200b5240 0x7f32200645b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:39.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f321f7fe700 1 --2- 172.21.15.32:0/812420909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32200b5b20 0x7f3220141010 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7f3208002f30 tx=0x7f3208015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:39.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f321d7fa700 1 -- 172.21.15.32:0/812420909 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3208011ca0 con 0x7f32200b5b20 2020-12-05T17:39:39.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f321d7fa700 1 -- 172.21.15.32:0/812420909 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3208011e00 con 0x7f32200b5b20 2020-12-05T17:39:39.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f321d7fa700 1 -- 172.21.15.32:0/812420909 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f320801d630 con 0x7f32200b5b20 2020-12-05T17:39:39.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f32267d5700 1 -- 172.21.15.32:0/812420909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322013dff0 con 0x7f32200b5b20 2020-12-05T17:39:39.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f32267d5700 1 -- 172.21.15.32:0/812420909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32201416a0 con 0x7f32200b5b20 2020-12-05T17:39:39.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f321d7fa700 1 -- 172.21.15.32:0/812420909 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3208017070 con 0x7f32200b5b20 2020-12-05T17:39:39.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.559+0000 7f32267d5700 1 -- 172.21.15.32:0/812420909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3220140d00 con 0x7f32200b5b20 2020-12-05T17:39:39.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.563+0000 7f321d7fa700 1 --2- 172.21.15.32:0/812420909 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3204055500 0x7f32040579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:39.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.563+0000 7f321ffff700 1 --2- 172.21.15.32:0/812420909 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3204055500 0x7f32040579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:39.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.563+0000 7f321d7fa700 1 -- 172.21.15.32:0/812420909 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f320806ee00 con 0x7f32200b5b20 2020-12-05T17:39:39.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.563+0000 7f321ffff700 1 --2- 172.21.15.32:0/812420909 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3204055500 0x7f32040579c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f321000b000 tx=0x7f3210009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:39.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.571+0000 7f321d7fa700 1 -- 172.21.15.32:0/812420909 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3208044e40 con 0x7f32200b5b20 2020-12-05T17:39:39.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:39.759+0000 7f32267d5700 1 -- 172.21.15.32:0/812420909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3220057050 con 0x7f32200b5b20 2020-12-05T17:39:40.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.063+0000 7f321d7fa700 1 -- 172.21.15.32:0/812420909 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3208022020 con 0x7f32200b5b20 2020-12-05T17:39:40.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.067+0000 7f32267d5700 1 -- 172.21.15.32:0/812420909 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3204055500 msgr2=0x7f32040579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:40.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.067+0000 7f32267d5700 1 --2- 172.21.15.32:0/812420909 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3204055500 0x7f32040579c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f321000b000 tx=0x7f3210009040).stop 2020-12-05T17:39:40.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.067+0000 7f32267d5700 1 -- 172.21.15.32:0/812420909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32200b5b20 msgr2=0x7f3220141010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:40.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.067+0000 7f32267d5700 1 --2- 172.21.15.32:0/812420909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32200b5b20 0x7f3220141010 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7f3208002f30 tx=0x7f3208015000).stop 2020-12-05T17:39:40.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.067+0000 7f32267d5700 1 -- 172.21.15.32:0/812420909 shutdown_connections 2020-12-05T17:39:40.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.067+0000 7f32267d5700 1 --2- 172.21.15.32:0/812420909 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32200b6690 0x7f322013f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:40.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.067+0000 7f32267d5700 1 --2- 172.21.15.32:0/812420909 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3204055500 0x7f32040579c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:40.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.067+0000 7f32267d5700 1 --2- 172.21.15.32:0/812420909 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32200b5240 0x7f32200645b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:40.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.067+0000 7f32267d5700 1 --2- 172.21.15.32:0/812420909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32200b5b20 0x7f3220141010 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:40.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.067+0000 7f32267d5700 1 -- 172.21.15.32:0/812420909 >> 172.21.15.32:0/812420909 conn(0x7f322005f3b0 msgr2=0x7f3220145e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:40.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.067+0000 7f32267d5700 1 -- 172.21.15.32:0/812420909 shutdown_connections 2020-12-05T17:39:40.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.067+0000 7f32267d5700 1 -- 172.21.15.32:0/812420909 wait complete. 2020-12-05T17:39:40.087 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-12-05T17:39:40.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.207+0000 7f0dabefa700 1 Processor -- start 2020-12-05T17:39:40.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.207+0000 7f0dabefa700 1 -- start start 2020-12-05T17:39:40.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0dabefa700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0da40b6690 0x7f0da40b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:40.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0dabefa700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0da40b5240 0x7f0da4145f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:40.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0dabefa700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0da40b56f0 con 0x7f0da40b5b20 2020-12-05T17:39:40.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0dabefa700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da40b1920 con 0x7f0da40b6690 2020-12-05T17:39:40.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0dabefa700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da40b1bc0 con 0x7f0da40b5240 2020-12-05T17:39:40.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0da9495700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0da40b6690 0x7f0da40b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:40.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0da9c96700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0da40b5b20 0x7f0da40b2060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41146/0 (socket says 172.21.15.32:41146) 2020-12-05T17:39:40.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0da9c96700 1 -- 172.21.15.32:0/694213637 learned_addr learned my addr 172.21.15.32:0/694213637 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:40.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0daa497700 1 --2- 172.21.15.32:0/694213637 >> v2:172.21.15.61:3300/0 conn(0x7f0da40b5240 0x7f0da4145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:40.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0da9495700 1 -- 172.21.15.32:0/694213637 >> v2:172.21.15.61:3300/0 conn(0x7f0da40b5240 msgr2=0x7f0da4145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:40.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0da9495700 1 --2- 172.21.15.32:0/694213637 >> v2:172.21.15.61:3300/0 conn(0x7f0da40b5240 0x7f0da4145f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:40.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0da8c94700 1 -- 172.21.15.32:0/694213637 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2680734423 0 0) 0x7f0da40b56f0 con 0x7f0da40b5b20 2020-12-05T17:39:40.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0da9495700 1 -- 172.21.15.32:0/694213637 >> v1:172.21.15.32:6789/0 conn(0x7f0da40b5b20 legacy=0x7f0da40b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:40.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0da9495700 1 -- 172.21.15.32:0/694213637 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da4146400 con 0x7f0da40b6690 2020-12-05T17:39:40.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0da9495700 1 --2- 172.21.15.32:0/694213637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da40b6690 0x7f0da40b5f00 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7f0d98002f30 tx=0x7f0d98006dc0).ready entity=mon.0 client_cookie=489689eb8418b1a1 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:40.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0da8c94700 1 -- 172.21.15.32:0/694213637 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d980097b0 con 0x7f0da40b6690 2020-12-05T17:39:40.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0da8c94700 1 -- 172.21.15.32:0/694213637 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d98009910 con 0x7f0da40b6690 2020-12-05T17:39:40.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0da8c94700 1 -- 172.21.15.32:0/694213637 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d98009c70 con 0x7f0da40b6690 2020-12-05T17:39:40.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0dabefa700 1 -- 172.21.15.32:0/694213637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da40b6690 msgr2=0x7f0da40b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:40.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0dabefa700 1 --2- 172.21.15.32:0/694213637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da40b6690 0x7f0da40b5f00 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7f0d98002f30 tx=0x7f0d98006dc0).stop 2020-12-05T17:39:40.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0dabefa700 1 -- 172.21.15.32:0/694213637 shutdown_connections 2020-12-05T17:39:40.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0dabefa700 1 --2- 172.21.15.32:0/694213637 >> v2:172.21.15.61:3300/0 conn(0x7f0da40b5240 0x7f0da4145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:40.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0dabefa700 1 --2- 172.21.15.32:0/694213637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da40b6690 0x7f0da40b5f00 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:40.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0dabefa700 1 -- 172.21.15.32:0/694213637 >> 172.21.15.32:0/694213637 conn(0x7f0da405f3b0 msgr2=0x7f0da405f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:40.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0dabefa700 1 -- 172.21.15.32:0/694213637 shutdown_connections 2020-12-05T17:39:40.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0dabefa700 1 -- 172.21.15.32:0/694213637 wait complete. 2020-12-05T17:39:40.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0dabefa700 1 Processor -- start 2020-12-05T17:39:40.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.211+0000 7f0dabefa700 1 -- start start 2020-12-05T17:39:40.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0dabefa700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0da40b5240 0x7f0da40a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:40.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0dabefa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da40b5b20 0x7f0da40aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:40.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0dabefa700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0da40b6690 0x7f0da40aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:40.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0dabefa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da40a3d20 con 0x7f0da40b5b20 2020-12-05T17:39:40.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0dabefa700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da40a3f60 con 0x7f0da40b5240 2020-12-05T17:39:40.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0dabefa700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da40a6410 con 0x7f0da40b6690 2020-12-05T17:39:40.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0da9495700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da40b5b20 0x7f0da40aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:40.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0da9495700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da40b5b20 0x7f0da40aea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38506/0 (socket says 172.21.15.32:38506) 2020-12-05T17:39:40.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0da9495700 1 -- 172.21.15.32:0/1220487089 learned_addr learned my addr 172.21.15.32:0/1220487089 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:40.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0daa497700 1 --2- 172.21.15.32:0/1220487089 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0da40b6690 0x7f0da40aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:40.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0da9c96700 1 --2- 172.21.15.32:0/1220487089 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0da40b5240 0x7f0da40a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:40.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0da9495700 1 -- 172.21.15.32:0/1220487089 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0da40b6690 msgr2=0x7f0da40aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:40.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0da9495700 1 --2- 172.21.15.32:0/1220487089 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0da40b6690 0x7f0da40aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:40.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0da9495700 1 -- 172.21.15.32:0/1220487089 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0da40b5240 msgr2=0x7f0da40a3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:40.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0da9495700 1 --2- 172.21.15.32:0/1220487089 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0da40b5240 0x7f0da40a3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:40.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0da9495700 1 -- 172.21.15.32:0/1220487089 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da4066af0 con 0x7f0da40b5b20 2020-12-05T17:39:40.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0daa497700 1 --2- 172.21.15.32:0/1220487089 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0da40b6690 0x7f0da40aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:40.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0da9c96700 1 --2- 172.21.15.32:0/1220487089 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0da40b5240 0x7f0da40a3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:40.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0da9495700 1 --2- 172.21.15.32:0/1220487089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da40b5b20 0x7f0da40aea50 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x7f0d98002f30 tx=0x7f0d98015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:40.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0d96ffd700 1 -- 172.21.15.32:0/1220487089 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d98011a30 con 0x7f0da40b5b20 2020-12-05T17:39:40.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0d96ffd700 1 -- 172.21.15.32:0/1220487089 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d98011b90 con 0x7f0da40b5b20 2020-12-05T17:39:40.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0dabefa700 1 -- 172.21.15.32:0/1220487089 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da40af430 con 0x7f0da40b5b20 2020-12-05T17:39:40.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0d96ffd700 1 -- 172.21.15.32:0/1220487089 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d9801d440 con 0x7f0da40b5b20 2020-12-05T17:39:40.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0dabefa700 1 -- 172.21.15.32:0/1220487089 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da40a32a0 con 0x7f0da40b5b20 2020-12-05T17:39:40.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.215+0000 7f0dabefa700 1 -- 172.21.15.32:0/1220487089 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da4066d40 con 0x7f0da40b5b20 2020-12-05T17:39:40.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.219+0000 7f0d96ffd700 1 -- 172.21.15.32:0/1220487089 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0d98024030 con 0x7f0da40b5b20 2020-12-05T17:39:40.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.219+0000 7f0d96ffd700 1 --2- 172.21.15.32:0/1220487089 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0d840554e0 0x7f0d840579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:40.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.219+0000 7f0da9c96700 1 --2- 172.21.15.32:0/1220487089 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0d840554e0 0x7f0d840579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:40.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.219+0000 7f0d96ffd700 1 -- 172.21.15.32:0/1220487089 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f0d9806f0e0 con 0x7f0da40b5b20 2020-12-05T17:39:40.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.219+0000 7f0da9c96700 1 --2- 172.21.15.32:0/1220487089 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0d840554e0 0x7f0d840579a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f0da000b000 tx=0x7f0da0009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:40.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.223+0000 7f0d96ffd700 1 -- 172.21.15.32:0/1220487089 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0d98045120 con 0x7f0da40b5b20 2020-12-05T17:39:40.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:40.363+0000 7f0dabefa700 1 -- 172.21.15.32:0/1220487089 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0da4057050 con 0x7f0da40b5b20 2020-12-05T17:39:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.067+0000 7f0d96ffd700 1 -- 172.21.15.32:0/1220487089 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) v1 ==== 185+0+0 (secure 0 0 0) 0x7f0d980220e0 con 0x7f0da40b5b20 2020-12-05T17:39:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.071+0000 7f0dabefa700 1 -- 172.21.15.32:0/1220487089 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0d840554e0 msgr2=0x7f0d840579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.071+0000 7f0dabefa700 1 --2- 172.21.15.32:0/1220487089 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0d840554e0 0x7f0d840579a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f0da000b000 tx=0x7f0da0009040).stop 2020-12-05T17:39:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.071+0000 7f0dabefa700 1 -- 172.21.15.32:0/1220487089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da40b5b20 msgr2=0x7f0da40aea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.071+0000 7f0dabefa700 1 --2- 172.21.15.32:0/1220487089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da40b5b20 0x7f0da40aea50 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x7f0d98002f30 tx=0x7f0d98015000).stop 2020-12-05T17:39:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.071+0000 7f0dabefa700 1 -- 172.21.15.32:0/1220487089 shutdown_connections 2020-12-05T17:39:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.071+0000 7f0dabefa700 1 --2- 172.21.15.32:0/1220487089 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0d840554e0 0x7f0d840579a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.071+0000 7f0dabefa700 1 --2- 172.21.15.32:0/1220487089 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0da40b6690 0x7f0da40aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.071+0000 7f0dabefa700 1 --2- 172.21.15.32:0/1220487089 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0da40b5240 0x7f0da40a3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.071+0000 7f0dabefa700 1 --2- 172.21.15.32:0/1220487089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da40b5b20 0x7f0da40aea50 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.071+0000 7f0dabefa700 1 -- 172.21.15.32:0/1220487089 >> 172.21.15.32:0/1220487089 conn(0x7f0da405f3b0 msgr2=0x7f0da4142570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.071+0000 7f0dabefa700 1 -- 172.21.15.32:0/1220487089 shutdown_connections 2020-12-05T17:39:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.071+0000 7f0dabefa700 1 -- 172.21.15.32:0/1220487089 wait complete. 2020-12-05T17:39:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2020-12-05T17:39:41.093 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-12-05T17:39:41.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.199+0000 7f191d21c700 1 Processor -- start 2020-12-05T17:39:41.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.199+0000 7f191d21c700 1 -- start start 2020-12-05T17:39:41.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f191d21c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f19180afd30 0x7f19180b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:41.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f191d21c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19180af5f0 0x7f19180aed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:41.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f191d21c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19181468e0 con 0x7f19180b1dc0 2020-12-05T17:39:41.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f191d21c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1918142c90 con 0x7f19180af5f0 2020-12-05T17:39:41.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f191d21c700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1918146b10 con 0x7f19180afd30 2020-12-05T17:39:41.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f191ca1b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19180af5f0 0x7f19180aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:41.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f191ca1b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19180af5f0 0x7f19180aed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38516/0 (socket says 172.21.15.32:38516) 2020-12-05T17:39:41.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f191ca1b700 1 -- 172.21.15.32:0/833964545 learned_addr learned my addr 172.21.15.32:0/833964545 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:41.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f1917fff700 1 --1- 172.21.15.32:0/833964545 >> v1:172.21.15.32:6789/0 conn(0x7f19180b1dc0 0x7f19180ae600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41160/0 (socket says 172.21.15.32:41160) 2020-12-05T17:39:41.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f19177fe700 1 --2- 172.21.15.32:0/833964545 >> v2:172.21.15.61:3300/0 conn(0x7f19180afd30 0x7f19180b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:41.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f191ca1b700 1 -- 172.21.15.32:0/833964545 >> v2:172.21.15.61:3300/0 conn(0x7f19180afd30 msgr2=0x7f19180b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:41.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f191ca1b700 1 --2- 172.21.15.32:0/833964545 >> v2:172.21.15.61:3300/0 conn(0x7f19180afd30 0x7f19180b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:41.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f191ca1b700 1 -- 172.21.15.32:0/833964545 >> v1:172.21.15.32:6789/0 conn(0x7f19180b1dc0 legacy=0x7f19180ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:41.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f191ca1b700 1 -- 172.21.15.32:0/833964545 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1918146db0 con 0x7f19180af5f0 2020-12-05T17:39:41.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f1916ffd700 1 -- 172.21.15.32:0/833964545 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2782889894 0 0) 0x7f19181468e0 con 0x7f19180b1dc0 2020-12-05T17:39:41.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f191ca1b700 1 --2- 172.21.15.32:0/833964545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19180af5f0 0x7f19180aed10 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7f19100032b0 tx=0x7f1910009270).ready entity=mon.0 client_cookie=3dcb06bec12472e5 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:41.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f1916ffd700 1 -- 172.21.15.32:0/833964545 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1910009e60 con 0x7f19180af5f0 2020-12-05T17:39:41.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f1916ffd700 1 -- 172.21.15.32:0/833964545 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191000bc00 con 0x7f19180af5f0 2020-12-05T17:39:41.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f1916ffd700 1 -- 172.21.15.32:0/833964545 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f191000bd70 con 0x7f19180af5f0 2020-12-05T17:39:41.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f191d21c700 1 -- 172.21.15.32:0/833964545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19180af5f0 msgr2=0x7f19180aed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:41.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f191d21c700 1 --2- 172.21.15.32:0/833964545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19180af5f0 0x7f19180aed10 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7f19100032b0 tx=0x7f1910009270).stop 2020-12-05T17:39:41.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f191d21c700 1 -- 172.21.15.32:0/833964545 shutdown_connections 2020-12-05T17:39:41.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f191d21c700 1 --2- 172.21.15.32:0/833964545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19180af5f0 0x7f19180aed10 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:41.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f191d21c700 1 --2- 172.21.15.32:0/833964545 >> v2:172.21.15.61:3300/0 conn(0x7f19180afd30 0x7f19180b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:41.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f191d21c700 1 -- 172.21.15.32:0/833964545 >> 172.21.15.32:0/833964545 conn(0x7f19180620b0 msgr2=0x7f1918062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:41.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f191d21c700 1 -- 172.21.15.32:0/833964545 shutdown_connections 2020-12-05T17:39:41.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.203+0000 7f191d21c700 1 -- 172.21.15.32:0/833964545 wait complete. 2020-12-05T17:39:41.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f191d21c700 1 Processor -- start 2020-12-05T17:39:41.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f191d21c700 1 -- start start 2020-12-05T17:39:41.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f191d21c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19180af5f0 0x7f19180642b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:41.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f191d21c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19180afd30 0x7f1918151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:41.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f191d21c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19180b1dc0 0x7f1918151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:41.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f191d21c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1918063d60 con 0x7f19180afd30 2020-12-05T17:39:41.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f191d21c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1918066c40 con 0x7f19180b1dc0 2020-12-05T17:39:41.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f191d21c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19180639e0 con 0x7f19180af5f0 2020-12-05T17:39:41.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f19177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19180afd30 0x7f1918151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:41.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f19177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19180afd30 0x7f1918151490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38518/0 (socket says 172.21.15.32:38518) 2020-12-05T17:39:41.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f19177fe700 1 -- 172.21.15.32:0/2508059181 learned_addr learned my addr 172.21.15.32:0/2508059181 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:41.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f1917fff700 1 --2- 172.21.15.32:0/2508059181 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19180af5f0 0x7f19180642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:41.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f191ca1b700 1 --2- 172.21.15.32:0/2508059181 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19180b1dc0 0x7f1918151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:41.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f19177fe700 1 -- 172.21.15.32:0/2508059181 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19180af5f0 msgr2=0x7f19180642b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:41.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f19177fe700 1 --2- 172.21.15.32:0/2508059181 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19180af5f0 0x7f19180642b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:41.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f19177fe700 1 -- 172.21.15.32:0/2508059181 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19180b1dc0 msgr2=0x7f1918151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:41.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f19177fe700 1 --2- 172.21.15.32:0/2508059181 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19180b1dc0 0x7f1918151980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:41.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f19177fe700 1 -- 172.21.15.32:0/2508059181 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1918149590 con 0x7f19180afd30 2020-12-05T17:39:41.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f191ca1b700 1 --2- 172.21.15.32:0/2508059181 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19180b1dc0 0x7f1918151980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:39:41.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f19177fe700 1 --2- 172.21.15.32:0/2508059181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19180afd30 0x7f1918151490 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7f190c00c000 tx=0x7f190c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:41.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f19157fa700 1 -- 172.21.15.32:0/2508059181 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f190c00c870 con 0x7f19180afd30 2020-12-05T17:39:41.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f19157fa700 1 -- 172.21.15.32:0/2508059181 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190c00c9d0 con 0x7f19180afd30 2020-12-05T17:39:41.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f191d21c700 1 -- 172.21.15.32:0/2508059181 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1918064b70 con 0x7f19180afd30 2020-12-05T17:39:41.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f19157fa700 1 -- 172.21.15.32:0/2508059181 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f190c00cd30 con 0x7f19180afd30 2020-12-05T17:39:41.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f191d21c700 1 -- 172.21.15.32:0/2508059181 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1918149b30 con 0x7f19180afd30 2020-12-05T17:39:41.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.207+0000 7f191d21c700 1 -- 172.21.15.32:0/2508059181 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19181521b0 con 0x7f19180afd30 2020-12-05T17:39:41.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.211+0000 7f19157fa700 1 -- 172.21.15.32:0/2508059181 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f190c022070 con 0x7f19180afd30 2020-12-05T17:39:41.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.211+0000 7f19157fa700 1 --2- 172.21.15.32:0/2508059181 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f18f8055440 0x7f18f8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:41.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.211+0000 7f19157fa700 1 -- 172.21.15.32:0/2508059181 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f190c06e470 con 0x7f19180afd30 2020-12-05T17:39:41.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.211+0000 7f1917fff700 1 --2- 172.21.15.32:0/2508059181 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f18f8055440 0x7f18f8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:41.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.211+0000 7f1917fff700 1 --2- 172.21.15.32:0/2508059181 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f18f8055440 0x7f18f8057900 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f190800b000 tx=0x7f1908007160).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:41.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.215+0000 7f19157fa700 1 -- 172.21.15.32:0/2508059181 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f190c044660 con 0x7f19180afd30 2020-12-05T17:39:41.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:41.359+0000 7f191d21c700 1 -- 172.21.15.32:0/2508059181 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1918063ba0 con 0x7f19180afd30 2020-12-05T17:39:42.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.079+0000 7f19157fa700 1 -- 172.21.15.32:0/2508059181 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) v1 ==== 173+0+0 (secure 0 0 0) 0x7f190c047c80 con 0x7f19180afd30 2020-12-05T17:39:42.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.083+0000 7f191d21c700 1 -- 172.21.15.32:0/2508059181 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f18f8055440 msgr2=0x7f18f8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:42.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.083+0000 7f191d21c700 1 --2- 172.21.15.32:0/2508059181 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f18f8055440 0x7f18f8057900 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f190800b000 tx=0x7f1908007160).stop 2020-12-05T17:39:42.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.083+0000 7f191d21c700 1 -- 172.21.15.32:0/2508059181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19180afd30 msgr2=0x7f1918151490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:42.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.083+0000 7f191d21c700 1 --2- 172.21.15.32:0/2508059181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19180afd30 0x7f1918151490 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7f190c00c000 tx=0x7f190c008040).stop 2020-12-05T17:39:42.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.083+0000 7f191d21c700 1 -- 172.21.15.32:0/2508059181 shutdown_connections 2020-12-05T17:39:42.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.083+0000 7f191d21c700 1 --2- 172.21.15.32:0/2508059181 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19180b1dc0 0x7f1918151980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:42.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.083+0000 7f191d21c700 1 --2- 172.21.15.32:0/2508059181 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f18f8055440 0x7f18f8057900 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:42.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.083+0000 7f191d21c700 1 --2- 172.21.15.32:0/2508059181 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19180af5f0 0x7f19180642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:42.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.083+0000 7f191d21c700 1 --2- 172.21.15.32:0/2508059181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19180afd30 0x7f1918151490 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:42.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.083+0000 7f191d21c700 1 -- 172.21.15.32:0/2508059181 >> 172.21.15.32:0/2508059181 conn(0x7f19180620b0 msgr2=0x7f1918145de0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:42.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.083+0000 7f191d21c700 1 -- 172.21.15.32:0/2508059181 shutdown_connections 2020-12-05T17:39:42.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.083+0000 7f191d21c700 1 -- 172.21.15.32:0/2508059181 wait complete. 2020-12-05T17:39:42.091 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2020-12-05T17:39:42.104 INFO:tasks.ceph.mds.a:Restarting daemon 2020-12-05T17:39:42.104 INFO:tasks.ceph.mds.a:Stopping old one... 2020-12-05T17:39:42.104 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-12-05T17:39:42.105 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:39:42.105 INFO:tasks.ceph.mds.b:Restarting daemon 2020-12-05T17:39:42.105 INFO:tasks.ceph.mds.b:Stopping old one... 2020-12-05T17:39:42.106 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-12-05T17:39:42.106 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:39:42.115 INFO:tasks.ceph.mds.a:Stopped 2020-12-05T17:39:42.116 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-12-05T17:39:42.118 INFO:tasks.ceph.mds.b:Stopped 2020-12-05T17:39:42.118 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-12-05T17:39:42.160 INFO:tasks.ceph.mds.a:Started 2020-12-05T17:39:42.160 DEBUG:teuthology.parallel:result is None 2020-12-05T17:39:42.163 INFO:tasks.ceph.mds.b:Started 2020-12-05T17:39:42.163 DEBUG:teuthology.parallel:result is None 2020-12-05T17:39:42.164 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-12-05T17:39:42.235 INFO:tasks.ceph.mds.b.smithi061.stdout:starting mds.b at 2020-12-05T17:39:42.237 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:39:42.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.279+0000 7fd187fff700 1 Processor -- start 2020-12-05T17:39:42.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.279+0000 7fd187fff700 1 -- start start 2020-12-05T17:39:42.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.279+0000 7fd187fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1880b1980 0x7fd188146580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:42.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.279+0000 7fd187fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd188146a70 0x7fd188066070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.279+0000 7fd187fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd188143000 con 0x7fd18805f6f0 2020-12-05T17:39:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.279+0000 7fd187fff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd18805fad0 con 0x7fd188146a70 2020-12-05T17:39:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.279+0000 7fd187fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880b2f80 con 0x7fd1880b1980 2020-12-05T17:39:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.279+0000 7fd186ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd18805f6f0 0x7fd1881428f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41186/0 (socket says 172.21.15.32:41186) 2020-12-05T17:39:42.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.279+0000 7fd186ffd700 1 -- 172.21.15.32:0/3075161717 learned_addr learned my addr 172.21.15.32:0/3075161717 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:42.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.279+0000 7fd1867fc700 1 --2- 172.21.15.32:0/3075161717 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1880b1980 0x7fd188146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:42.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.279+0000 7fd1877fe700 1 --2- 172.21.15.32:0/3075161717 >> v2:172.21.15.61:3300/0 conn(0x7fd188146a70 0x7fd188066070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:42.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.279+0000 7fd185ffb700 1 -- 172.21.15.32:0/3075161717 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 521566413 0 0) 0x7fd188143000 con 0x7fd18805f6f0 2020-12-05T17:39:42.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.279+0000 7fd185ffb700 1 -- 172.21.15.32:0/3075161717 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd16c003230 con 0x7fd18805f6f0 2020-12-05T17:39:42.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.279+0000 7fd1867fc700 1 -- 172.21.15.32:0/3075161717 >> v2:172.21.15.61:3300/0 conn(0x7fd188146a70 msgr2=0x7fd188066070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:42.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.279+0000 7fd1867fc700 1 --2- 172.21.15.32:0/3075161717 >> v2:172.21.15.61:3300/0 conn(0x7fd188146a70 0x7fd188066070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:42.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.279+0000 7fd1867fc700 1 -- 172.21.15.32:0/3075161717 >> v1:172.21.15.32:6789/0 conn(0x7fd18805f6f0 legacy=0x7fd1881428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:42.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.279+0000 7fd1867fc700 1 -- 172.21.15.32:0/3075161717 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd188145e40 con 0x7fd1880b1980 2020-12-05T17:39:42.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.283+0000 7fd1867fc700 1 --2- 172.21.15.32:0/3075161717 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1880b1980 0x7fd188146580 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fd178002f30 tx=0x7fd178006dc0).ready entity=mon.2 client_cookie=1ec92ff2d9b19679 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:42.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.283+0000 7fd185ffb700 1 -- 172.21.15.32:0/3075161717 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1780096a0 con 0x7fd1880b1980 2020-12-05T17:39:42.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.283+0000 7fd185ffb700 1 -- 172.21.15.32:0/3075161717 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd178009800 con 0x7fd1880b1980 2020-12-05T17:39:42.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.283+0000 7fd185ffb700 1 -- 172.21.15.32:0/3075161717 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd178009b60 con 0x7fd1880b1980 2020-12-05T17:39:42.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.283+0000 7fd187fff700 1 -- 172.21.15.32:0/3075161717 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1880b1980 msgr2=0x7fd188146580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:42.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.283+0000 7fd187fff700 1 --2- 172.21.15.32:0/3075161717 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1880b1980 0x7fd188146580 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fd178002f30 tx=0x7fd178006dc0).stop 2020-12-05T17:39:42.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.283+0000 7fd187fff700 1 -- 172.21.15.32:0/3075161717 shutdown_connections 2020-12-05T17:39:42.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.283+0000 7fd187fff700 1 --2- 172.21.15.32:0/3075161717 >> v2:172.21.15.61:3300/0 conn(0x7fd188146a70 0x7fd188066070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:42.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.283+0000 7fd187fff700 1 --2- 172.21.15.32:0/3075161717 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1880b1980 0x7fd188146580 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:42.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.283+0000 7fd187fff700 1 -- 172.21.15.32:0/3075161717 >> 172.21.15.32:0/3075161717 conn(0x7fd1880604c0 msgr2=0x7fd18805c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:42.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.283+0000 7fd187fff700 1 -- 172.21.15.32:0/3075161717 shutdown_connections 2020-12-05T17:39:42.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.283+0000 7fd187fff700 1 -- 172.21.15.32:0/3075161717 wait complete. 2020-12-05T17:39:42.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.283+0000 7fd187fff700 1 Processor -- start 2020-12-05T17:39:42.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.283+0000 7fd187fff700 1 -- start start 2020-12-05T17:39:42.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.283+0000 7fd187fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd18805f6f0 0x7fd1881416e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:42.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.283+0000 7fd187fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1880b1980 0x7fd1881514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd187fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd188146a70 0x7fd188155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd187fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd188065b50 con 0x7fd1880b1980 2020-12-05T17:39:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd187fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880b6ac0 con 0x7fd18805f6f0 2020-12-05T17:39:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd187fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd188154a30 con 0x7fd188146a70 2020-12-05T17:39:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd1867fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1880b1980 0x7fd1881514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:42.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd1867fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1880b1980 0x7fd1881514b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38546/0 (socket says 172.21.15.32:38546) 2020-12-05T17:39:42.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd1867fc700 1 -- 172.21.15.32:0/3573322698 learned_addr learned my addr 172.21.15.32:0/3573322698 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:42.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd1877fe700 1 --2- 172.21.15.32:0/3573322698 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd188146a70 0x7fd188155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:42.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd186ffd700 1 --2- 172.21.15.32:0/3573322698 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd18805f6f0 0x7fd1881416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd1867fc700 1 -- 172.21.15.32:0/3573322698 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd188146a70 msgr2=0x7fd188155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd1867fc700 1 --2- 172.21.15.32:0/3573322698 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd188146a70 0x7fd188155100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd1867fc700 1 -- 172.21.15.32:0/3573322698 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd18805f6f0 msgr2=0x7fd1881416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd1867fc700 1 --2- 172.21.15.32:0/3573322698 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd18805f6f0 0x7fd1881416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd1867fc700 1 -- 172.21.15.32:0/3573322698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd188114180 con 0x7fd1880b1980 2020-12-05T17:39:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd186ffd700 1 --2- 172.21.15.32:0/3573322698 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd18805f6f0 0x7fd1881416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd1867fc700 1 --2- 172.21.15.32:0/3573322698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1880b1980 0x7fd1881514b0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 rx=0x7fd188113610 tx=0x7fd178015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd177fff700 1 -- 172.21.15.32:0/3573322698 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd178011bf0 con 0x7fd1880b1980 2020-12-05T17:39:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd187fff700 1 -- 172.21.15.32:0/3573322698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd188114420 con 0x7fd1880b1980 2020-12-05T17:39:42.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd177fff700 1 -- 172.21.15.32:0/3573322698 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd178011d50 con 0x7fd1880b1980 2020-12-05T17:39:42.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd177fff700 1 -- 172.21.15.32:0/3573322698 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd17801d640 con 0x7fd1880b1980 2020-12-05T17:39:42.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd187fff700 1 -- 172.21.15.32:0/3573322698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1880b5e20 con 0x7fd1880b1980 2020-12-05T17:39:42.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd187fff700 1 -- 172.21.15.32:0/3573322698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd188057050 con 0x7fd1880b1980 2020-12-05T17:39:42.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd177fff700 1 -- 172.21.15.32:0/3573322698 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd17800f070 con 0x7fd1880b1980 2020-12-05T17:39:42.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd177fff700 1 --2- 172.21.15.32:0/3573322698 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1640554e0 0x7fd1640579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:42.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.287+0000 7fd177fff700 1 -- 172.21.15.32:0/3573322698 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fd17806f720 con 0x7fd1880b1980 2020-12-05T17:39:42.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.291+0000 7fd186ffd700 1 --2- 172.21.15.32:0/3573322698 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1640554e0 0x7fd1640579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:42.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.291+0000 7fd186ffd700 1 --2- 172.21.15.32:0/3573322698 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1640554e0 0x7fd1640579a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7fd17c00d000 tx=0x7fd17c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:42.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.295+0000 7fd177fff700 1 -- 172.21.15.32:0/3573322698 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd178045b00 con 0x7fd1880b1980 2020-12-05T17:39:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:42.455+0000 7fd187fff700 1 -- 172.21.15.32:0/3573322698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fd188069690 con 0x7fd1880b1980 2020-12-05T17:39:43.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.279+0000 7fd177fff700 1 -- 172.21.15.32:0/3573322698 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v98) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd178049120 con 0x7fd1880b1980 2020-12-05T17:39:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.283+0000 7fd187fff700 1 -- 172.21.15.32:0/3573322698 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1640554e0 msgr2=0x7fd1640579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.283+0000 7fd187fff700 1 --2- 172.21.15.32:0/3573322698 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1640554e0 0x7fd1640579a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7fd17c00d000 tx=0x7fd17c009040).stop 2020-12-05T17:39:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.283+0000 7fd187fff700 1 -- 172.21.15.32:0/3573322698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1880b1980 msgr2=0x7fd1881514b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:43.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.283+0000 7fd187fff700 1 --2- 172.21.15.32:0/3573322698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1880b1980 0x7fd1881514b0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 rx=0x7fd188113610 tx=0x7fd178015000).stop 2020-12-05T17:39:43.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.283+0000 7fd187fff700 1 -- 172.21.15.32:0/3573322698 shutdown_connections 2020-12-05T17:39:43.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.283+0000 7fd187fff700 1 --2- 172.21.15.32:0/3573322698 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1640554e0 0x7fd1640579a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:43.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.283+0000 7fd187fff700 1 --2- 172.21.15.32:0/3573322698 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd188146a70 0x7fd188155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:43.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.283+0000 7fd187fff700 1 --2- 172.21.15.32:0/3573322698 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd18805f6f0 0x7fd1881416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:43.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.283+0000 7fd187fff700 1 --2- 172.21.15.32:0/3573322698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1880b1980 0x7fd1881514b0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:43.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.283+0000 7fd187fff700 1 -- 172.21.15.32:0/3573322698 >> 172.21.15.32:0/3573322698 conn(0x7fd1880604c0 msgr2=0x7fd188140c60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:43.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.283+0000 7fd187fff700 1 -- 172.21.15.32:0/3573322698 shutdown_connections 2020-12-05T17:39:43.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.283+0000 7fd187fff700 1 -- 172.21.15.32:0/3573322698 wait complete. 2020-12-05T17:39:43.291 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2020-12-05T17:39:43.303 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2020-12-05T17:39:43.304 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-12-05T17:39:43.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.415+0000 7f865bfff700 1 Processor -- start 2020-12-05T17:39:43.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.415+0000 7f865bfff700 1 -- start start 2020-12-05T17:39:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f865c062400 0x7f865c05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f865c05feb0 0x7f865c0b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865bfff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f865c142450 con 0x7f865c05f3b0 2020-12-05T17:39:43.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865bfff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f865c0b42f0 con 0x7f865c05feb0 2020-12-05T17:39:43.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865bfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c061780 con 0x7f865c062400 2020-12-05T17:39:43.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f865c05feb0 0x7f865c0b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:43.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f865c05feb0 0x7f865c0b3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38558/0 (socket says 172.21.15.32:38558) 2020-12-05T17:39:43.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865b7fe700 1 -- 172.21.15.32:0/3929797713 learned_addr learned my addr 172.21.15.32:0/3929797713 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:43.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f86527fc700 1 --2- 172.21.15.32:0/3929797713 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f865c062400 0x7f865c05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:43.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865b7fe700 1 -- 172.21.15.32:0/3929797713 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f865c062400 msgr2=0x7f865c05f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:43.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865b7fe700 1 --2- 172.21.15.32:0/3929797713 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f865c062400 0x7f865c05f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:43.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865b7fe700 1 -- 172.21.15.32:0/3929797713 >> v1:172.21.15.61:6789/0 conn(0x7f865c05f3b0 legacy=0x7f865c0b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:43.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865b7fe700 1 -- 172.21.15.32:0/3929797713 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c061a20 con 0x7f865c05feb0 2020-12-05T17:39:43.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865b7fe700 1 --2- 172.21.15.32:0/3929797713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f865c05feb0 0x7f865c0b3e00 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7f86540032b0 tx=0x7f8654009270).ready entity=mon.0 client_cookie=e82e2f3ebf8a63dc server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:43.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865a7fc700 1 -- 172.21.15.32:0/3929797713 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f865400bc00 con 0x7f865c05feb0 2020-12-05T17:39:43.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865a7fc700 1 -- 172.21.15.32:0/3929797713 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865400bd60 con 0x7f865c05feb0 2020-12-05T17:39:43.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865a7fc700 1 -- 172.21.15.32:0/3929797713 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8654011400 con 0x7f865c05feb0 2020-12-05T17:39:43.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865bfff700 1 -- 172.21.15.32:0/3929797713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f865c05feb0 msgr2=0x7f865c0b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:43.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865bfff700 1 --2- 172.21.15.32:0/3929797713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f865c05feb0 0x7f865c0b3e00 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7f86540032b0 tx=0x7f8654009270).stop 2020-12-05T17:39:43.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865bfff700 1 -- 172.21.15.32:0/3929797713 shutdown_connections 2020-12-05T17:39:43.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865bfff700 1 --2- 172.21.15.32:0/3929797713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f865c05feb0 0x7f865c0b3e00 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:43.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865bfff700 1 --2- 172.21.15.32:0/3929797713 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f865c062400 0x7f865c05f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:43.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865bfff700 1 -- 172.21.15.32:0/3929797713 >> 172.21.15.32:0/3929797713 conn(0x7f865c0b8ee0 msgr2=0x7f865c0b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:43.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865bfff700 1 -- 172.21.15.32:0/3929797713 shutdown_connections 2020-12-05T17:39:43.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.419+0000 7f865bfff700 1 -- 172.21.15.32:0/3929797713 wait complete. 2020-12-05T17:39:43.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.423+0000 7f865bfff700 1 Processor -- start 2020-12-05T17:39:43.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.423+0000 7f865bfff700 1 -- start start 2020-12-05T17:39:43.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.423+0000 7f865bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f865c05f3b0 0x7f865c0b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:43.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.423+0000 7f865bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f865c05feb0 0x7f865c154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:43.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.423+0000 7f865bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f865c062400 0x7f865c154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:43.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.423+0000 7f865bfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c1522d0 con 0x7f865c05f3b0 2020-12-05T17:39:43.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.423+0000 7f865bfff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c1516c0 con 0x7f865c062400 2020-12-05T17:39:43.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.423+0000 7f865bfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c063110 con 0x7f865c05feb0 2020-12-05T17:39:43.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.423+0000 7f865affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f865c05f3b0 0x7f865c0b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:43.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.423+0000 7f865affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f865c05f3b0 0x7f865c0b73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38560/0 (socket says 172.21.15.32:38560) 2020-12-05T17:39:43.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.423+0000 7f865affd700 1 -- 172.21.15.32:0/1677208730 learned_addr learned my addr 172.21.15.32:0/1677208730 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:43.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.423+0000 7f865b7fe700 1 --2- 172.21.15.32:0/1677208730 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f865c062400 0x7f865c154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:43.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.423+0000 7f86527fc700 1 --2- 172.21.15.32:0/1677208730 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f865c05feb0 0x7f865c154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:43.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.423+0000 7f865affd700 1 -- 172.21.15.32:0/1677208730 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f865c05feb0 msgr2=0x7f865c154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:43.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.423+0000 7f865affd700 1 --2- 172.21.15.32:0/1677208730 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f865c05feb0 0x7f865c154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:43.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.423+0000 7f865affd700 1 -- 172.21.15.32:0/1677208730 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f865c062400 msgr2=0x7f865c154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:43.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.423+0000 7f865affd700 1 --2- 172.21.15.32:0/1677208730 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f865c062400 0x7f865c154d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:43.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.423+0000 7f865affd700 1 -- 172.21.15.32:0/1677208730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c0633b0 con 0x7f865c05f3b0 2020-12-05T17:39:43.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.423+0000 7f865b7fe700 1 --2- 172.21.15.32:0/1677208730 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f865c062400 0x7f865c154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:43.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.427+0000 7f865affd700 1 --2- 172.21.15.32:0/1677208730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f865c05f3b0 0x7f865c0b73c0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7f865c0a29f0 tx=0x7f864c007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:43.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.427+0000 7f8658ff9700 1 -- 172.21.15.32:0/1677208730 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f864c007240 con 0x7f865c05f3b0 2020-12-05T17:39:43.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.427+0000 7f8658ff9700 1 -- 172.21.15.32:0/1677208730 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864c0073a0 con 0x7f865c05f3b0 2020-12-05T17:39:43.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.427+0000 7f865bfff700 1 -- 172.21.15.32:0/1677208730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865c0b84e0 con 0x7f865c05f3b0 2020-12-05T17:39:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.427+0000 7f8658ff9700 1 -- 172.21.15.32:0/1677208730 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f864c0066d0 con 0x7f865c05f3b0 2020-12-05T17:39:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.427+0000 7f865bfff700 1 -- 172.21.15.32:0/1677208730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865c151490 con 0x7f865c05f3b0 2020-12-05T17:39:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.427+0000 7f865bfff700 1 -- 172.21.15.32:0/1677208730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865c057050 con 0x7f865c05f3b0 2020-12-05T17:39:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.427+0000 7f8658ff9700 1 -- 172.21.15.32:0/1677208730 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f864c010070 con 0x7f865c05f3b0 2020-12-05T17:39:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.427+0000 7f8658ff9700 1 --2- 172.21.15.32:0/1677208730 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f86400553b0 0x7f8640057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:43.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.427+0000 7f8658ff9700 1 -- 172.21.15.32:0/1677208730 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f864c06d450 con 0x7f865c05f3b0 2020-12-05T17:39:43.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.427+0000 7f86527fc700 1 --2- 172.21.15.32:0/1677208730 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f86400553b0 0x7f8640057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:43.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.431+0000 7f86527fc700 1 --2- 172.21.15.32:0/1677208730 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f86400553b0 0x7f8640057870 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f864400c000 tx=0x7f8644008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:43.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.439+0000 7f8658ff9700 1 -- 172.21.15.32:0/1677208730 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f864c0438d0 con 0x7f865c05f3b0 2020-12-05T17:39:43.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.631+0000 7f865bfff700 1 -- 172.21.15.32:0/1677208730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f865c05aab0 con 0x7f865c05f3b0 2020-12-05T17:39:43.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.631+0000 7f8658ff9700 1 -- 172.21.15.32:0/1677208730 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v31) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f864c046ef0 con 0x7f865c05f3b0 2020-12-05T17:39:43.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.635+0000 7f865bfff700 1 -- 172.21.15.32:0/1677208730 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f86400553b0 msgr2=0x7f8640057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:43.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.635+0000 7f865bfff700 1 --2- 172.21.15.32:0/1677208730 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f86400553b0 0x7f8640057870 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f864400c000 tx=0x7f8644008040).stop 2020-12-05T17:39:43.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.635+0000 7f865bfff700 1 -- 172.21.15.32:0/1677208730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f865c05f3b0 msgr2=0x7f865c0b73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:43.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.635+0000 7f865bfff700 1 --2- 172.21.15.32:0/1677208730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f865c05f3b0 0x7f865c0b73c0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7f865c0a29f0 tx=0x7f864c007be0).stop 2020-12-05T17:39:43.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.635+0000 7f865bfff700 1 -- 172.21.15.32:0/1677208730 shutdown_connections 2020-12-05T17:39:43.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.635+0000 7f865bfff700 1 --2- 172.21.15.32:0/1677208730 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f865c062400 0x7f865c154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:43.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.635+0000 7f865bfff700 1 --2- 172.21.15.32:0/1677208730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f865c05f3b0 0x7f865c0b73c0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:43.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.635+0000 7f865bfff700 1 --2- 172.21.15.32:0/1677208730 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f86400553b0 0x7f8640057870 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:43.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.635+0000 7f865bfff700 1 --2- 172.21.15.32:0/1677208730 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f865c05feb0 0x7f865c154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:43.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.635+0000 7f865bfff700 1 -- 172.21.15.32:0/1677208730 >> 172.21.15.32:0/1677208730 conn(0x7f865c0b8ee0 msgr2=0x7f865c146090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:43.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.635+0000 7f865bfff700 1 -- 172.21.15.32:0/1677208730 shutdown_connections 2020-12-05T17:39:43.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.635+0000 7f865bfff700 1 -- 172.21.15.32:0/1677208730 wait complete. 2020-12-05T17:39:43.649 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:43.649 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:39:43.649 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2020-12-05T17:39:43.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:43.650 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2020-12-05T17:39:43.650 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfpb/aExAAdMKW4yIgC2zhkzyW6l8xug==", 2020-12-05T17:39:43.650 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:43.650 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:39:43.651 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:39:43.651 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:39:43.651 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:43.651 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:43.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:43.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:43.652 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2020-12-05T17:39:43.652 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfBKTYGRAAaPkezwmTiAvkUfTuDGDNZA==", 2020-12-05T17:39:43.653 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:43.653 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:39:43.653 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:39:43.653 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:39:43.653 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:43.654 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:43.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:43.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:43.654 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2020-12-05T17:39:43.654 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHw8tflo5EKhAAwV9D64fAoY3uvXCm+Aksag==", 2020-12-05T17:39:43.655 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:43.655 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:39:43.655 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:39:43.655 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:43.656 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:43.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:43.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:43.656 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2020-12-05T17:39:43.656 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDKw8tfTjEiERAAkW6mkgApoXwyF49kPjZBGg==", 2020-12-05T17:39:43.657 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:43.657 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:39:43.657 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:39:43.657 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:43.657 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:43.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:43.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:43.658 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2020-12-05T17:39:43.658 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDMw8tfLEzBMxAAiJ2NifsOLEkt1e3vmhwSTA==", 2020-12-05T17:39:43.658 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:43.659 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:39:43.659 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:39:43.659 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:43.659 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:43.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:43.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:43.660 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2020-12-05T17:39:43.660 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPw8tfV+vnGxAAQHvXJRW5X9Vhe/eYJBJ/uQ==", 2020-12-05T17:39:43.660 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:43.661 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:39:43.661 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:39:43.661 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:43.661 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:43.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:43.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:43.662 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2020-12-05T17:39:43.662 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUw8tfN8wMJxAApc5LY9HGDeIV/o0eil/g4A==", 2020-12-05T17:39:43.662 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:43.662 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:39:43.663 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:39:43.663 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:43.663 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:43.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:43.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:43.664 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2020-12-05T17:39:43.664 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXw8tfvZ5dDxAAtAfftExkljlBvn7ZSoD/Og==", 2020-12-05T17:39:43.664 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:43.664 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:39:43.664 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:39:43.665 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:43.665 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:43.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:43.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:43.666 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2020-12-05T17:39:43.666 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDZw8tfhXFzMhAAZXNwNGfKv360Tq4N/DX5Mw==", 2020-12-05T17:39:43.666 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:43.666 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:39:43.667 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:39:43.667 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:43.667 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:43.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:43.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:43.668 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2020-12-05T17:39:43.668 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDcw8tfROfSGhAA/Y2Y1b4k5zQju0MM7LASeQ==", 2020-12-05T17:39:43.668 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:43.668 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:39:43.668 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:39:43.669 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:43.669 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:43.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:43.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:43.669 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2020-12-05T17:39:43.670 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfWDp/HxAAmvLPj7C3WDgj0OC2o7lBJQ==", 2020-12-05T17:39:43.670 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:43.670 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:39:43.670 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:39:43.670 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:39:43.671 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:43.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:43.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:43.673 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2020-12-05T17:39:43.673 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfugnmIRAAW6+/QGMLkHw+Z97RHpV4kw==", 2020-12-05T17:39:43.673 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:43.673 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:39:43.674 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:39:43.674 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:39:43.675 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:43.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:43.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:43.676 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2020-12-05T17:39:43.676 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfojyqJBAAyta+dSy2yGnC1kKe3YOAYw==", 2020-12-05T17:39:43.676 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:43.676 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:39:43.677 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:39:43.677 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:39:43.677 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:43.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:43.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:43.678 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2020-12-05T17:39:43.678 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDFw8tfES0PLBAAA1cUV44snAUDF2TyAmXImw==", 2020-12-05T17:39:43.678 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:43.679 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:39:43.679 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:39:43.679 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:39:43.679 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:43.679 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:43.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:43.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:43.680 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2020-12-05T17:39:43.680 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfDr8mERAAS7bpAQXgW78W8GDq1v1zaA==", 2020-12-05T17:39:43.680 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:43.681 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:39:43.681 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:39:43.681 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:43.681 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:43.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:43.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:43.682 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2020-12-05T17:39:43.682 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tf3HmcDhAATNKHhH9zgasedaNEzgG0OQ==", 2020-12-05T17:39:43.682 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:39:43.683 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:39:43.683 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:39:43.683 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:39:43.683 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:43.683 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:43.684 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:43.684 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:39:43.685 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:39:43.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.795+0000 7f5b51e61700 1 Processor -- start 2020-12-05T17:39:43.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.799+0000 7f5b51e61700 1 -- start start 2020-12-05T17:39:43.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.799+0000 7f5b51e61700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b4c0afd20 0x7f5b4c1421e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:43.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.799+0000 7f5b51e61700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b4c142a30 0x7f5b4c142ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:43.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.799+0000 7f5b51e61700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b4c0af300 con 0x7f5b4c0b5650 2020-12-05T17:39:43.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.799+0000 7f5b51e61700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b4c0b5a30 con 0x7f5b4c0afd20 2020-12-05T17:39:43.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.799+0000 7f5b51e61700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c0aea90 con 0x7f5b4c142a30 2020-12-05T17:39:43.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.799+0000 7f5b4bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b4c0afd20 0x7f5b4c1421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:43.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.799+0000 7f5b50e5f700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f5b4c0b5650 0x7f5b4c141ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:53856/0 (socket says 172.21.15.32:53856) 2020-12-05T17:39:43.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.799+0000 7f5b50e5f700 1 -- 172.21.15.32:0/1024003490 learned_addr learned my addr 172.21.15.32:0/1024003490 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:43.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.799+0000 7f5b51660700 1 --2- 172.21.15.32:0/1024003490 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b4c142a30 0x7f5b4c142ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:43.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.799+0000 7f5b4b7fe700 1 -- 172.21.15.32:0/1024003490 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 736444291 0 0) 0x7f5b4c0af300 con 0x7f5b4c0b5650 2020-12-05T17:39:43.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.799+0000 7f5b4b7fe700 1 -- 172.21.15.32:0/1024003490 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b30003230 con 0x7f5b4c0b5650 2020-12-05T17:39:43.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.799+0000 7f5b4bfff700 1 -- 172.21.15.32:0/1024003490 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b4c142a30 msgr2=0x7f5b4c142ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:43.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.799+0000 7f5b4bfff700 1 --2- 172.21.15.32:0/1024003490 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b4c142a30 0x7f5b4c142ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:43.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.799+0000 7f5b4bfff700 1 -- 172.21.15.32:0/1024003490 >> v1:172.21.15.61:6789/0 conn(0x7f5b4c0b5650 legacy=0x7f5b4c141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:43.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.799+0000 7f5b4bfff700 1 -- 172.21.15.32:0/1024003490 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c0aed30 con 0x7f5b4c0afd20 2020-12-05T17:39:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.799+0000 7f5b4bfff700 1 --2- 172.21.15.32:0/1024003490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b4c0afd20 0x7f5b4c1421e0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 rx=0x7f5b38002f30 tx=0x7f5b38006dc0).ready entity=mon.0 client_cookie=2324f6bf92f3f934 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.799+0000 7f5b4b7fe700 1 -- 172.21.15.32:0/1024003490 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b380096a0 con 0x7f5b4c0afd20 2020-12-05T17:39:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.799+0000 7f5b4b7fe700 1 -- 172.21.15.32:0/1024003490 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b38009800 con 0x7f5b4c0afd20 2020-12-05T17:39:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.799+0000 7f5b4b7fe700 1 -- 172.21.15.32:0/1024003490 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b38009b60 con 0x7f5b4c0afd20 2020-12-05T17:39:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.799+0000 7f5b51e61700 1 -- 172.21.15.32:0/1024003490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b4c0afd20 msgr2=0x7f5b4c1421e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.799+0000 7f5b51e61700 1 --2- 172.21.15.32:0/1024003490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b4c0afd20 0x7f5b4c1421e0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 rx=0x7f5b38002f30 tx=0x7f5b38006dc0).stop 2020-12-05T17:39:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.803+0000 7f5b51e61700 1 -- 172.21.15.32:0/1024003490 shutdown_connections 2020-12-05T17:39:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.803+0000 7f5b51e61700 1 --2- 172.21.15.32:0/1024003490 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b4c142a30 0x7f5b4c142ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.803+0000 7f5b51e61700 1 --2- 172.21.15.32:0/1024003490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b4c0afd20 0x7f5b4c1421e0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:43.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.803+0000 7f5b51e61700 1 -- 172.21.15.32:0/1024003490 >> 172.21.15.32:0/1024003490 conn(0x7f5b4c05bbb0 msgr2=0x7f5b4c062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:43.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.803+0000 7f5b51e61700 1 -- 172.21.15.32:0/1024003490 shutdown_connections 2020-12-05T17:39:43.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.803+0000 7f5b51e61700 1 -- 172.21.15.32:0/1024003490 wait complete. 2020-12-05T17:39:43.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.803+0000 7f5b51e61700 1 Processor -- start 2020-12-05T17:39:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.803+0000 7f5b51e61700 1 -- start start 2020-12-05T17:39:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.807+0000 7f5b51e61700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5b4c0afd20 0x7f5b4c13ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.807+0000 7f5b51e61700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b4c0b5650 0x7f5b4c1517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.807+0000 7f5b51e61700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b4c142a30 0x7f5b4c151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.807+0000 7f5b51e61700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c110b90 con 0x7f5b4c142a30 2020-12-05T17:39:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.807+0000 7f5b51e61700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c1408a0 con 0x7f5b4c0afd20 2020-12-05T17:39:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.807+0000 7f5b51e61700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c1101c0 con 0x7f5b4c0b5650 2020-12-05T17:39:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.807+0000 7f5b51660700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b4c142a30 0x7f5b4c151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.807+0000 7f5b50e5f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5b4c0afd20 0x7f5b4c13ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.807+0000 7f5b4bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b4c0b5650 0x7f5b4c1517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:43.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.807+0000 7f5b51660700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b4c142a30 0x7f5b4c151c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38578/0 (socket says 172.21.15.32:38578) 2020-12-05T17:39:43.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.807+0000 7f5b51660700 1 -- 172.21.15.32:0/19281253 learned_addr learned my addr 172.21.15.32:0/19281253 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:43.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.807+0000 7f5b51660700 1 -- 172.21.15.32:0/19281253 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b4c0b5650 msgr2=0x7f5b4c1517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:43.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.807+0000 7f5b51660700 1 --2- 172.21.15.32:0/19281253 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b4c0b5650 0x7f5b4c1517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.807+0000 7f5b51660700 1 -- 172.21.15.32:0/19281253 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5b4c0afd20 msgr2=0x7f5b4c13ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.807+0000 7f5b51660700 1 --2- 172.21.15.32:0/19281253 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5b4c0afd20 0x7f5b4c13ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.807+0000 7f5b51660700 1 -- 172.21.15.32:0/19281253 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c10efb0 con 0x7f5b4c142a30 2020-12-05T17:39:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.807+0000 7f5b51660700 1 --2- 172.21.15.32:0/19281253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b4c142a30 0x7f5b4c151c90 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f5b44006dd0 tx=0x7f5b440091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:43.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.807+0000 7f5b49ffb700 1 -- 172.21.15.32:0/19281253 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b4400c8f0 con 0x7f5b4c142a30 2020-12-05T17:39:43.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.807+0000 7f5b49ffb700 1 -- 172.21.15.32:0/19281253 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4400ca50 con 0x7f5b4c142a30 2020-12-05T17:39:43.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.807+0000 7f5b49ffb700 1 -- 172.21.15.32:0/19281253 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b4400cdb0 con 0x7f5b4c142a30 2020-12-05T17:39:43.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.807+0000 7f5b51e61700 1 -- 172.21.15.32:0/19281253 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b4c10f250 con 0x7f5b4c142a30 2020-12-05T17:39:43.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.807+0000 7f5b51e61700 1 -- 172.21.15.32:0/19281253 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b4c149400 con 0x7f5b4c142a30 2020-12-05T17:39:43.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.811+0000 7f5b51e61700 1 -- 172.21.15.32:0/19281253 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b4c057050 con 0x7f5b4c142a30 2020-12-05T17:39:43.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.811+0000 7f5b49ffb700 1 -- 172.21.15.32:0/19281253 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5b44022070 con 0x7f5b4c142a30 2020-12-05T17:39:43.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.811+0000 7f5b49ffb700 1 --2- 172.21.15.32:0/19281253 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b340553b0 0x7f5b34057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:43.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.811+0000 7f5b49ffb700 1 -- 172.21.15.32:0/19281253 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f5b4406e9a0 con 0x7f5b4c142a30 2020-12-05T17:39:43.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.811+0000 7f5b50e5f700 1 --2- 172.21.15.32:0/19281253 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b340553b0 0x7f5b34057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:43.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.811+0000 7f5b50e5f700 1 --2- 172.21.15.32:0/19281253 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b340553b0 0x7f5b34057870 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f5b4000d000 tx=0x7f5b40009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:43.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.815+0000 7f5b49ffb700 1 -- 172.21.15.32:0/19281253 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5b44044e80 con 0x7f5b4c142a30 2020-12-05T17:39:43.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.959+0000 7f5b51e61700 1 -- 172.21.15.32:0/19281253 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b4c152320 con 0x7f5b4c142a30 2020-12-05T17:39:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.959+0000 7f5b49ffb700 1 -- 172.21.15.32:0/19281253 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+13415 (secure 0 0 0) 0x7f5b44029090 con 0x7f5b4c142a30 2020-12-05T17:39:43.965 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:43.966 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":98,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:39:43.268285+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:39:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.959+0000 7f5b51e61700 1 -- 172.21.15.32:0/19281253 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b340553b0 msgr2=0x7f5b34057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.963+0000 7f5b51e61700 1 --2- 172.21.15.32:0/19281253 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b340553b0 0x7f5b34057870 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f5b4000d000 tx=0x7f5b40009040).stop 2020-12-05T17:39:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.963+0000 7f5b51e61700 1 -- 172.21.15.32:0/19281253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b4c142a30 msgr2=0x7f5b4c151c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.963+0000 7f5b51e61700 1 --2- 172.21.15.32:0/19281253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b4c142a30 0x7f5b4c151c90 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f5b44006dd0 tx=0x7f5b440091a0).stop 2020-12-05T17:39:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.963+0000 7f5b51e61700 1 -- 172.21.15.32:0/19281253 shutdown_connections 2020-12-05T17:39:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.963+0000 7f5b51e61700 1 --2- 172.21.15.32:0/19281253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b4c142a30 0x7f5b4c151c90 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.963+0000 7f5b51e61700 1 --2- 172.21.15.32:0/19281253 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5b4c0afd20 0x7f5b4c13ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:43.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.963+0000 7f5b51e61700 1 --2- 172.21.15.32:0/19281253 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b340553b0 0x7f5b34057870 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:43.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.963+0000 7f5b51e61700 1 --2- 172.21.15.32:0/19281253 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b4c0b5650 0x7f5b4c1517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:43.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.963+0000 7f5b51e61700 1 -- 172.21.15.32:0/19281253 >> 172.21.15.32:0/19281253 conn(0x7f5b4c05bbb0 msgr2=0x7f5b4c142ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:43.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.963+0000 7f5b51e61700 1 -- 172.21.15.32:0/19281253 shutdown_connections 2020-12-05T17:39:43.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:43.963+0000 7f5b51e61700 1 -- 172.21.15.32:0/19281253 wait complete. 2020-12-05T17:39:43.980 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-12-05T17:39:43.981 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:39:44.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.091+0000 7f0ff5599700 1 Processor -- start 2020-12-05T17:39:44.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.091+0000 7f0ff5599700 1 -- start start 2020-12-05T17:39:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.091+0000 7f0ff5599700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0ff00ad250 0x7f0ff00af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.091+0000 7f0ff5599700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff00b24e0 con 0x7f0ff013ed60 2020-12-05T17:39:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.091+0000 7f0ff5599700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff00b0780 con 0x7f0ff00b2100 2020-12-05T17:39:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.091+0000 7f0ff5599700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff013fd10 con 0x7f0ff00ad250 2020-12-05T17:39:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.091+0000 7f0fef7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0ff013ed60 0x7f0ff0142600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41232/0 (socket says 172.21.15.32:41232) 2020-12-05T17:39:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.091+0000 7f0fef7fe700 1 -- 172.21.15.32:0/1116146387 learned_addr learned my addr 172.21.15.32:0/1116146387 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.091+0000 7f0ff4d98700 1 --2- 172.21.15.32:0/1116146387 >> v2:172.21.15.61:3300/0 conn(0x7f0ff00ad250 0x7f0ff00af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.091+0000 7f0feeffd700 1 -- 172.21.15.32:0/1116146387 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 627752881 0 0) 0x7f0ff00b24e0 con 0x7f0ff013ed60 2020-12-05T17:39:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.091+0000 7f0feeffd700 1 -- 172.21.15.32:0/1116146387 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fdc003230 con 0x7f0ff013ed60 2020-12-05T17:39:44.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.091+0000 7f0ff4d98700 1 -- 172.21.15.32:0/1116146387 >> v1:172.21.15.61:6789/0 conn(0x7f0ff00b2100 legacy=0x7f0ff0138820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.091+0000 7f0ff4d98700 1 -- 172.21.15.32:0/1116146387 >> v1:172.21.15.32:6789/0 conn(0x7f0ff013ed60 legacy=0x7f0ff0142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.091+0000 7f0ff4d98700 1 -- 172.21.15.32:0/1116146387 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff00b0240 con 0x7f0ff00ad250 2020-12-05T17:39:44.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.091+0000 7f0ff4d98700 1 --2- 172.21.15.32:0/1116146387 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ff00ad250 0x7f0ff00af360 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f0fe80032b0 tx=0x7f0fe80091a0).ready entity=mon.1 client_cookie=3d670d95fbbe71a4 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.095+0000 7f0feeffd700 1 -- 172.21.15.32:0/1116146387 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fe800b9a0 con 0x7f0ff00ad250 2020-12-05T17:39:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.095+0000 7f0feeffd700 1 -- 172.21.15.32:0/1116146387 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe800bb00 con 0x7f0ff00ad250 2020-12-05T17:39:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.095+0000 7f0ff5599700 1 -- 172.21.15.32:0/1116146387 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ff00ad250 msgr2=0x7f0ff00af360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.095+0000 7f0feeffd700 1 -- 172.21.15.32:0/1116146387 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fe800bc70 con 0x7f0ff00ad250 2020-12-05T17:39:44.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.095+0000 7f0ff5599700 1 --2- 172.21.15.32:0/1116146387 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ff00ad250 0x7f0ff00af360 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f0fe80032b0 tx=0x7f0fe80091a0).stop 2020-12-05T17:39:44.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.095+0000 7f0ff5599700 1 -- 172.21.15.32:0/1116146387 shutdown_connections 2020-12-05T17:39:44.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.095+0000 7f0ff5599700 1 --2- 172.21.15.32:0/1116146387 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ff00ad250 0x7f0ff00af360 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.095+0000 7f0ff5599700 1 -- 172.21.15.32:0/1116146387 >> 172.21.15.32:0/1116146387 conn(0x7f0ff00620b0 msgr2=0x7f0ff0062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:44.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.095+0000 7f0ff5599700 1 -- 172.21.15.32:0/1116146387 shutdown_connections 2020-12-05T17:39:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.095+0000 7f0ff5599700 1 -- 172.21.15.32:0/1116146387 wait complete. 2020-12-05T17:39:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0ff5599700 1 Processor -- start 2020-12-05T17:39:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0ff5599700 1 -- start start 2020-12-05T17:39:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0ff5599700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff00ad250 0x7f0ff00b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0ff5599700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ff00b2100 0x7f0ff01335f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0ff5599700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ff0065320 0x7f0ff0133ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0ff5599700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff0066930 con 0x7f0ff00ad250 2020-12-05T17:39:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0ff5599700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff0134dc0 con 0x7f0ff0065320 2020-12-05T17:39:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0ff5599700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff0065000 con 0x7f0ff00b2100 2020-12-05T17:39:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0feffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff00ad250 0x7f0ff00b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0feffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff00ad250 0x7f0ff00b49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38588/0 (socket says 172.21.15.32:38588) 2020-12-05T17:39:44.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0feffff700 1 -- 172.21.15.32:0/1691106298 learned_addr learned my addr 172.21.15.32:0/1691106298 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:44.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0fef7fe700 1 --2- 172.21.15.32:0/1691106298 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ff00b2100 0x7f0ff01335f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0ff4d98700 1 --2- 172.21.15.32:0/1691106298 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ff0065320 0x7f0ff0133ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0feffff700 1 -- 172.21.15.32:0/1691106298 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ff00b2100 msgr2=0x7f0ff01335f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0feffff700 1 --2- 172.21.15.32:0/1691106298 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ff00b2100 0x7f0ff01335f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0feffff700 1 -- 172.21.15.32:0/1691106298 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ff0065320 msgr2=0x7f0ff0133ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0feffff700 1 --2- 172.21.15.32:0/1691106298 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ff0065320 0x7f0ff0133ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0feffff700 1 -- 172.21.15.32:0/1691106298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff0135260 con 0x7f0ff00ad250 2020-12-05T17:39:44.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0feffff700 1 --2- 172.21.15.32:0/1691106298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff00ad250 0x7f0ff00b49e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0x7f0ff0134f80 tx=0x7f0fe0007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:44.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0fed7fa700 1 -- 172.21.15.32:0/1691106298 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fe000e4a0 con 0x7f0ff00ad250 2020-12-05T17:39:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0fed7fa700 1 -- 172.21.15.32:0/1691106298 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe000e600 con 0x7f0ff00ad250 2020-12-05T17:39:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0ff5599700 1 -- 172.21.15.32:0/1691106298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff0133fd0 con 0x7f0ff00ad250 2020-12-05T17:39:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0fed7fa700 1 -- 172.21.15.32:0/1691106298 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fe000e960 con 0x7f0ff00ad250 2020-12-05T17:39:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.099+0000 7f0ff5599700 1 -- 172.21.15.32:0/1691106298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff013b7a0 con 0x7f0ff00ad250 2020-12-05T17:39:44.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.103+0000 7f0ff5599700 1 -- 172.21.15.32:0/1691106298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff0057050 con 0x7f0ff00ad250 2020-12-05T17:39:44.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.103+0000 7f0fed7fa700 1 -- 172.21.15.32:0/1691106298 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0fe0010030 con 0x7f0ff00ad250 2020-12-05T17:39:44.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.103+0000 7f0fed7fa700 1 --2- 172.21.15.32:0/1691106298 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0fd00553b0 0x7f0fd0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.103+0000 7f0fed7fa700 1 -- 172.21.15.32:0/1691106298 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f0fe006dc00 con 0x7f0ff00ad250 2020-12-05T17:39:44.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.103+0000 7f0fef7fe700 1 --2- 172.21.15.32:0/1691106298 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0fd00553b0 0x7f0fd0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.103+0000 7f0fef7fe700 1 --2- 172.21.15.32:0/1691106298 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0fd00553b0 0x7f0fd0057870 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f0fe400d000 tx=0x7f0fe4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.111+0000 7f0fed7fa700 1 -- 172.21.15.32:0/1691106298 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0fe0073020 con 0x7f0ff00ad250 2020-12-05T17:39:44.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.251+0000 7f0ff5599700 1 -- 172.21.15.32:0/1691106298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0ff0060c10 con 0x7f0ff00ad250 2020-12-05T17:39:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.251+0000 7f0fed7fa700 1 -- 172.21.15.32:0/1691106298 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v98) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0fe00440e0 con 0x7f0ff00ad250 2020-12-05T17:39:44.258 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:39:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.255+0000 7f0ff5599700 1 -- 172.21.15.32:0/1691106298 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0fd00553b0 msgr2=0x7f0fd0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.255+0000 7f0ff5599700 1 --2- 172.21.15.32:0/1691106298 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0fd00553b0 0x7f0fd0057870 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f0fe400d000 tx=0x7f0fe4009040).stop 2020-12-05T17:39:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.255+0000 7f0ff5599700 1 -- 172.21.15.32:0/1691106298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff00ad250 msgr2=0x7f0ff00b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.255+0000 7f0ff5599700 1 --2- 172.21.15.32:0/1691106298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff00ad250 0x7f0ff00b49e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0x7f0ff0134f80 tx=0x7f0fe0007c20).stop 2020-12-05T17:39:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.255+0000 7f0ff5599700 1 -- 172.21.15.32:0/1691106298 shutdown_connections 2020-12-05T17:39:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.255+0000 7f0ff5599700 1 --2- 172.21.15.32:0/1691106298 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ff0065320 0x7f0ff0133ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.255+0000 7f0ff5599700 1 --2- 172.21.15.32:0/1691106298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff00ad250 0x7f0ff00b49e0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.255+0000 7f0ff5599700 1 --2- 172.21.15.32:0/1691106298 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0fd00553b0 0x7f0fd0057870 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.255+0000 7f0ff5599700 1 --2- 172.21.15.32:0/1691106298 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ff00b2100 0x7f0ff01335f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.255+0000 7f0ff5599700 1 -- 172.21.15.32:0/1691106298 >> 172.21.15.32:0/1691106298 conn(0x7f0ff00620b0 msgr2=0x7f0ff00b24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.255+0000 7f0ff5599700 1 -- 172.21.15.32:0/1691106298 shutdown_connections 2020-12-05T17:39:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.255+0000 7f0ff5599700 1 -- 172.21.15.32:0/1691106298 wait complete. 2020-12-05T17:39:44.277 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:39:44.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.387+0000 7fac0b0b3700 1 Processor -- start 2020-12-05T17:39:44.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.387+0000 7fac0b0b3700 1 -- start start 2020-12-05T17:39:44.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.387+0000 7fac0b0b3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac040b2b00 0x7fac0413bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac0b0b3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac040b2ee0 con 0x7fac040aea80 2020-12-05T17:39:44.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac0b0b3700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac040b0c30 con 0x7fac040ad230 2020-12-05T17:39:44.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac0b0b3700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac0413ed30 con 0x7fac040b2b00 2020-12-05T17:39:44.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac03fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fac040aea80 0x7fac0413c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41246/0 (socket says 172.21.15.32:41246) 2020-12-05T17:39:44.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac03fff700 1 -- 172.21.15.32:0/2472297953 learned_addr learned my addr 172.21.15.32:0/2472297953 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:44.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac08e4f700 1 --2- 172.21.15.32:0/2472297953 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac040b2b00 0x7fac0413bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac037fe700 1 -- 172.21.15.32:0/2472297953 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2558918483 0 0) 0x7fac040b2ee0 con 0x7fac040aea80 2020-12-05T17:39:44.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac037fe700 1 -- 172.21.15.32:0/2472297953 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabf0004340 con 0x7fac040aea80 2020-12-05T17:39:44.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac037fe700 1 -- 172.21.15.32:0/2472297953 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 29302986 0 0) 0x7fac040b0c30 con 0x7fac040ad230 2020-12-05T17:39:44.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac037fe700 1 -- 172.21.15.32:0/2472297953 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac040b2ee0 con 0x7fac040ad230 2020-12-05T17:39:44.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac037fe700 1 -- 172.21.15.32:0/2472297953 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3643876445 0 0) 0x7fabf0004340 con 0x7fac040aea80 2020-12-05T17:39:44.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac037fe700 1 -- 172.21.15.32:0/2472297953 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac040b2b00 msgr2=0x7fac0413bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:44.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac037fe700 1 --2- 172.21.15.32:0/2472297953 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac040b2b00 0x7fac0413bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac037fe700 1 -- 172.21.15.32:0/2472297953 >> v1:172.21.15.61:6789/0 conn(0x7fac040ad230 legacy=0x7fac0413f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac037fe700 1 -- 172.21.15.32:0/2472297953 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac04142de0 con 0x7fac040aea80 2020-12-05T17:39:44.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac037fe700 1 -- 172.21.15.32:0/2472297953 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fabf80016d0 con 0x7fac040aea80 2020-12-05T17:39:44.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac037fe700 1 -- 172.21.15.32:0/2472297953 >> v1:172.21.15.32:6789/0 conn(0x7fac040aea80 legacy=0x7fac0413c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac037fe700 1 --2- 172.21.15.32:0/2472297953 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fabf00049f0 0x7fabf0004df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac037fe700 1 --2- 172.21.15.32:0/2472297953 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac040b2b00 0x7fabf00055e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac037fe700 1 --2- 172.21.15.32:0/2472297953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabf0005ad0 0x7fabf0005f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac037fe700 1 -- 172.21.15.32:0/2472297953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0413ed30 con 0x7fabf0005ad0 2020-12-05T17:39:44.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac037fe700 1 -- 172.21.15.32:0/2472297953 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf0006470 con 0x7fabf00049f0 2020-12-05T17:39:44.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac037fe700 1 -- 172.21.15.32:0/2472297953 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf000c880 con 0x7fac040b2b00 2020-12-05T17:39:44.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac09650700 1 --2- 172.21.15.32:0/2472297953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabf0005ad0 0x7fabf0005f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac03fff700 1 --2- 172.21.15.32:0/2472297953 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac040b2b00 0x7fabf00055e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac08e4f700 1 --2- 172.21.15.32:0/2472297953 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fabf00049f0 0x7fabf0004df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac09650700 1 -- 172.21.15.32:0/2472297953 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac040b2b00 msgr2=0x7fabf00055e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:44.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac09650700 1 --2- 172.21.15.32:0/2472297953 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac040b2b00 0x7fabf00055e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac09650700 1 -- 172.21.15.32:0/2472297953 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fabf00049f0 msgr2=0x7fabf0004df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:44.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac09650700 1 --2- 172.21.15.32:0/2472297953 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fabf00049f0 0x7fabf0004df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac09650700 1 -- 172.21.15.32:0/2472297953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf0004340 con 0x7fabf0005ad0 2020-12-05T17:39:44.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac09650700 1 --2- 172.21.15.32:0/2472297953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabf0005ad0 0x7fabf0005f50 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7fabf00065b0 tx=0x7fabfc007ee0).ready entity=mon.0 client_cookie=d514cd20bcfe9999 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:44.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac037fe700 1 -- 172.21.15.32:0/2472297953 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabfc00cbc0 con 0x7fabf0005ad0 2020-12-05T17:39:44.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac037fe700 1 -- 172.21.15.32:0/2472297953 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabfc00cd20 con 0x7fabf0005ad0 2020-12-05T17:39:44.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac037fe700 1 -- 172.21.15.32:0/2472297953 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabfc0125f0 con 0x7fabf0005ad0 2020-12-05T17:39:44.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac0b0b3700 1 -- 172.21.15.32:0/2472297953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabf0005ad0 msgr2=0x7fabf0005f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac0b0b3700 1 --2- 172.21.15.32:0/2472297953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabf0005ad0 0x7fabf0005f50 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7fabf00065b0 tx=0x7fabfc007ee0).stop 2020-12-05T17:39:44.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac09650700 1 -- 172.21.15.32:0/2472297953 reap_dead start 2020-12-05T17:39:44.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac0b0b3700 1 -- 172.21.15.32:0/2472297953 shutdown_connections 2020-12-05T17:39:44.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac0b0b3700 1 -- 172.21.15.32:0/2472297953 >> 172.21.15.32:0/2472297953 conn(0x7fac04062310 msgr2=0x7fac040626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:44.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac0b0b3700 1 -- 172.21.15.32:0/2472297953 shutdown_connections 2020-12-05T17:39:44.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.391+0000 7fac0b0b3700 1 -- 172.21.15.32:0/2472297953 wait complete. 2020-12-05T17:39:44.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac0b0b3700 1 Processor -- start 2020-12-05T17:39:44.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac0b0b3700 1 -- start start 2020-12-05T17:39:44.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac0b0b3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac0413d350 0x7fac04111980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac0b0b3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac04067480 0x7fac04111e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac0b0b3700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac04112360 0x7fac04065250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac0b0b3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0410f0b0 con 0x7fac04067480 2020-12-05T17:39:44.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac0b0b3700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac04066740 con 0x7fac04112360 2020-12-05T17:39:44.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac0b0b3700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac0413d730 con 0x7fac0413d350 2020-12-05T17:39:44.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac03fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac04067480 0x7fac04111e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac03fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac04067480 0x7fac04111e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38610/0 (socket says 172.21.15.32:38610) 2020-12-05T17:39:44.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac03fff700 1 -- 172.21.15.32:0/2192215002 learned_addr learned my addr 172.21.15.32:0/2192215002 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:44.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac08e4f700 1 --2- 172.21.15.32:0/2192215002 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac0413d350 0x7fac04111980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac09650700 1 --2- 172.21.15.32:0/2192215002 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac04112360 0x7fac04065250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac03fff700 1 -- 172.21.15.32:0/2192215002 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac0413d350 msgr2=0x7fac04111980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac03fff700 1 --2- 172.21.15.32:0/2192215002 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac0413d350 0x7fac04111980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac03fff700 1 -- 172.21.15.32:0/2192215002 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac04112360 msgr2=0x7fac04065250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac03fff700 1 --2- 172.21.15.32:0/2192215002 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac04112360 0x7fac04065250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac03fff700 1 -- 172.21.15.32:0/2192215002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac04110000 con 0x7fac04067480 2020-12-05T17:39:44.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac03fff700 1 --2- 172.21.15.32:0/2192215002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac04067480 0x7fac04111e70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7fabf800d000 tx=0x7fabf8001e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:44.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac01ffb700 1 -- 172.21.15.32:0/2192215002 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabf800fa30 con 0x7fac04067480 2020-12-05T17:39:44.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac01ffb700 1 -- 172.21.15.32:0/2192215002 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf800fb90 con 0x7fac04067480 2020-12-05T17:39:44.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac0b0b3700 1 -- 172.21.15.32:0/2192215002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac04110280 con 0x7fac04067480 2020-12-05T17:39:44.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac01ffb700 1 -- 172.21.15.32:0/2192215002 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabf80016a0 con 0x7fac04067480 2020-12-05T17:39:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.395+0000 7fac0b0b3700 1 -- 172.21.15.32:0/2192215002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac04067140 con 0x7fac04067480 2020-12-05T17:39:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.399+0000 7fac0b0b3700 1 -- 172.21.15.32:0/2192215002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac04057050 con 0x7fac04067480 2020-12-05T17:39:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.399+0000 7fac01ffb700 1 -- 172.21.15.32:0/2192215002 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fabf801b030 con 0x7fac04067480 2020-12-05T17:39:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.399+0000 7fac01ffb700 1 --2- 172.21.15.32:0/2192215002 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fabe4055330 0x7fabe40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.399+0000 7fac01ffb700 1 -- 172.21.15.32:0/2192215002 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fabf806d720 con 0x7fac04067480 2020-12-05T17:39:44.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.399+0000 7fac08e4f700 1 --2- 172.21.15.32:0/2192215002 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fabe4055330 0x7fabe40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.399+0000 7fac08e4f700 1 --2- 172.21.15.32:0/2192215002 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fabe4055330 0x7fabe40577f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7fabf4008000 tx=0x7fabf4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:44.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.407+0000 7fac01ffb700 1 -- 172.21.15.32:0/2192215002 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fabf8043c00 con 0x7fac04067480 2020-12-05T17:39:44.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.551+0000 7fac0b0b3700 1 -- 172.21.15.32:0/2192215002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fac041117f0 con 0x7fac04067480 2020-12-05T17:39:44.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.551+0000 7fac01ffb700 1 -- 172.21.15.32:0/2192215002 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v98) v1 ==== 92+0+10 (secure 0 0 0) 0x7fabf8019030 con 0x7fac04067480 2020-12-05T17:39:44.556 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:39:44.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.551+0000 7fac0b0b3700 1 -- 172.21.15.32:0/2192215002 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fabe4055330 msgr2=0x7fabe40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.551+0000 7fac0b0b3700 1 --2- 172.21.15.32:0/2192215002 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fabe4055330 0x7fabe40577f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7fabf4008000 tx=0x7fabf4008040).stop 2020-12-05T17:39:44.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.551+0000 7fac0b0b3700 1 -- 172.21.15.32:0/2192215002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac04067480 msgr2=0x7fac04111e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.551+0000 7fac0b0b3700 1 --2- 172.21.15.32:0/2192215002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac04067480 0x7fac04111e70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7fabf800d000 tx=0x7fabf8001e50).stop 2020-12-05T17:39:44.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.551+0000 7fac0b0b3700 1 -- 172.21.15.32:0/2192215002 shutdown_connections 2020-12-05T17:39:44.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.551+0000 7fac0b0b3700 1 --2- 172.21.15.32:0/2192215002 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac04112360 0x7fac04065250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.551+0000 7fac0b0b3700 1 --2- 172.21.15.32:0/2192215002 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fabe4055330 0x7fabe40577f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.551+0000 7fac0b0b3700 1 --2- 172.21.15.32:0/2192215002 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac0413d350 0x7fac04111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.551+0000 7fac0b0b3700 1 --2- 172.21.15.32:0/2192215002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac04067480 0x7fac04111e70 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.551+0000 7fac0b0b3700 1 -- 172.21.15.32:0/2192215002 >> 172.21.15.32:0/2192215002 conn(0x7fac04062310 msgr2=0x7fac0413c080 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.551+0000 7fac0b0b3700 1 -- 172.21.15.32:0/2192215002 shutdown_connections 2020-12-05T17:39:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.551+0000 7fac0b0b3700 1 -- 172.21.15.32:0/2192215002 wait complete. 2020-12-05T17:39:44.570 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-12-05T17:39:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.675+0000 7f1cd64d2700 1 Processor -- start 2020-12-05T17:39:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.675+0000 7f1cd64d2700 1 -- start start 2020-12-05T17:39:44.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.675+0000 7f1cd64d2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1cd00b1980 0x7f1cd0146580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.675+0000 7f1cd64d2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1cd0146a70 0x7f1cd0066070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.675+0000 7f1cd64d2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd0143000 con 0x7f1cd005f6f0 2020-12-05T17:39:44.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.675+0000 7f1cd64d2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd005fad0 con 0x7f1cd00b1980 2020-12-05T17:39:44.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.675+0000 7f1cd64d2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd00b2f80 con 0x7f1cd0146a70 2020-12-05T17:39:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.675+0000 7f1cd54d0700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1cd005f6f0 0x7f1cd01428f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41264/0 (socket says 172.21.15.32:41264) 2020-12-05T17:39:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.675+0000 7f1cd54d0700 1 -- 172.21.15.32:0/3260080810 learned_addr learned my addr 172.21.15.32:0/3260080810 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.675+0000 7f1cd4ccf700 1 --2- 172.21.15.32:0/3260080810 >> v2:172.21.15.32:3300/0 conn(0x7f1cd00b1980 0x7f1cd0146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.675+0000 7f1cd5cd1700 1 --2- 172.21.15.32:0/3260080810 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1cd0146a70 0x7f1cd0066070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.675+0000 7f1ccffff700 1 -- 172.21.15.32:0/3260080810 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764646599 0 0) 0x7f1cd0143000 con 0x7f1cd005f6f0 2020-12-05T17:39:44.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.675+0000 7f1ccffff700 1 -- 172.21.15.32:0/3260080810 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cbc003230 con 0x7f1cd005f6f0 2020-12-05T17:39:44.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.675+0000 7f1cd4ccf700 1 -- 172.21.15.32:0/3260080810 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1cd0146a70 msgr2=0x7f1cd0066070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:44.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.675+0000 7f1cd4ccf700 1 --2- 172.21.15.32:0/3260080810 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1cd0146a70 0x7f1cd0066070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.675+0000 7f1cd4ccf700 1 -- 172.21.15.32:0/3260080810 >> v1:172.21.15.32:6789/0 conn(0x7f1cd005f6f0 legacy=0x7f1cd01428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.675+0000 7f1cd4ccf700 1 -- 172.21.15.32:0/3260080810 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd0145e40 con 0x7f1cd00b1980 2020-12-05T17:39:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.675+0000 7f1cd4ccf700 1 --2- 172.21.15.32:0/3260080810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cd00b1980 0x7f1cd0146580 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7f1cc4002f30 tx=0x7f1cc4006dc0).ready entity=mon.0 client_cookie=44efadd9933fdaae server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.679+0000 7f1ccffff700 1 -- 172.21.15.32:0/3260080810 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cc40096a0 con 0x7f1cd00b1980 2020-12-05T17:39:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.679+0000 7f1ccffff700 1 -- 172.21.15.32:0/3260080810 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc4009800 con 0x7f1cd00b1980 2020-12-05T17:39:44.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.679+0000 7f1ccffff700 1 -- 172.21.15.32:0/3260080810 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cc4009b60 con 0x7f1cd00b1980 2020-12-05T17:39:44.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.679+0000 7f1cd64d2700 1 -- 172.21.15.32:0/3260080810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cd00b1980 msgr2=0x7f1cd0146580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.679+0000 7f1cd64d2700 1 --2- 172.21.15.32:0/3260080810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cd00b1980 0x7f1cd0146580 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7f1cc4002f30 tx=0x7f1cc4006dc0).stop 2020-12-05T17:39:44.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.679+0000 7f1cd64d2700 1 -- 172.21.15.32:0/3260080810 shutdown_connections 2020-12-05T17:39:44.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.679+0000 7f1cd64d2700 1 --2- 172.21.15.32:0/3260080810 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1cd0146a70 0x7f1cd0066070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.679+0000 7f1cd64d2700 1 --2- 172.21.15.32:0/3260080810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cd00b1980 0x7f1cd0146580 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.679+0000 7f1cd64d2700 1 -- 172.21.15.32:0/3260080810 >> 172.21.15.32:0/3260080810 conn(0x7f1cd00604c0 msgr2=0x7f1cd005c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:44.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.679+0000 7f1cd64d2700 1 -- 172.21.15.32:0/3260080810 shutdown_connections 2020-12-05T17:39:44.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.679+0000 7f1cd64d2700 1 -- 172.21.15.32:0/3260080810 wait complete. 2020-12-05T17:39:44.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.679+0000 7f1cd64d2700 1 Processor -- start 2020-12-05T17:39:44.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.679+0000 7f1cd64d2700 1 -- start start 2020-12-05T17:39:44.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.679+0000 7f1cd64d2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1cd005f6f0 0x7f1cd00b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cd64d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cd00b1980 0x7f1cd00658d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cd64d2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1cd0146a70 0x7f1cd00648d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cd64d2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd0155790 con 0x7f1cd00b1980 2020-12-05T17:39:44.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cd64d2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd01514a0 con 0x7f1cd0146a70 2020-12-05T17:39:44.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cd64d2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd0151740 con 0x7f1cd005f6f0 2020-12-05T17:39:44.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cd4ccf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cd00b1980 0x7f1cd00658d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cd4ccf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cd00b1980 0x7f1cd00658d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38624/0 (socket says 172.21.15.32:38624) 2020-12-05T17:39:44.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cd4ccf700 1 -- 172.21.15.32:0/1143873595 learned_addr learned my addr 172.21.15.32:0/1143873595 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:44.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cd5cd1700 1 --2- 172.21.15.32:0/1143873595 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1cd0146a70 0x7f1cd00648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cd54d0700 1 --2- 172.21.15.32:0/1143873595 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1cd005f6f0 0x7f1cd00b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cd4ccf700 1 -- 172.21.15.32:0/1143873595 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1cd005f6f0 msgr2=0x7f1cd00b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cd4ccf700 1 --2- 172.21.15.32:0/1143873595 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1cd005f6f0 0x7f1cd00b5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cd4ccf700 1 -- 172.21.15.32:0/1143873595 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1cd0146a70 msgr2=0x7f1cd00648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cd4ccf700 1 --2- 172.21.15.32:0/1143873595 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1cd0146a70 0x7f1cd00648d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cd4ccf700 1 -- 172.21.15.32:0/1143873595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd0064dc0 con 0x7f1cd00b1980 2020-12-05T17:39:44.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cd4ccf700 1 --2- 172.21.15.32:0/1143873595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cd00b1980 0x7f1cd00658d0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7f1cc4002f30 tx=0x7f1cc4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:44.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cce7fc700 1 -- 172.21.15.32:0/1143873595 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cc4011c20 con 0x7f1cd00b1980 2020-12-05T17:39:44.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cce7fc700 1 -- 172.21.15.32:0/1143873595 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc4011d80 con 0x7f1cd00b1980 2020-12-05T17:39:44.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cd64d2700 1 -- 172.21.15.32:0/1143873595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd01552b0 con 0x7f1cd00b1980 2020-12-05T17:39:44.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cce7fc700 1 -- 172.21.15.32:0/1143873595 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cc401d640 con 0x7f1cd00b1980 2020-12-05T17:39:44.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cd64d2700 1 -- 172.21.15.32:0/1143873595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd0155560 con 0x7f1cd00b1980 2020-12-05T17:39:44.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cd64d2700 1 -- 172.21.15.32:0/1143873595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd0057050 con 0x7f1cd00b1980 2020-12-05T17:39:44.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cce7fc700 1 -- 172.21.15.32:0/1143873595 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1cc4024030 con 0x7f1cd00b1980 2020-12-05T17:39:44.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cce7fc700 1 --2- 172.21.15.32:0/1143873595 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1cb0055440 0x7f1cb0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cce7fc700 1 -- 172.21.15.32:0/1143873595 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f1cc406ef10 con 0x7f1cd00b1980 2020-12-05T17:39:44.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.683+0000 7f1cd54d0700 1 --2- 172.21.15.32:0/1143873595 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1cb0055440 0x7f1cb0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.687+0000 7f1cd54d0700 1 --2- 172.21.15.32:0/1143873595 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1cb0055440 0x7f1cb0057900 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f1cc000b000 tx=0x7f1cc0009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:44.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.695+0000 7f1cce7fc700 1 -- 172.21.15.32:0/1143873595 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1cc40453f0 con 0x7f1cd00b1980 2020-12-05T17:39:44.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.835+0000 7f1cd64d2700 1 -- 172.21.15.32:0/1143873595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1cd01518b0 con 0x7f1cd00b1980 2020-12-05T17:39:44.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.839+0000 7f1cce7fc700 1 -- 172.21.15.32:0/1143873595 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v98) v1 ==== 84+0+143 (secure 0 0 0) 0x7f1cc4022020 con 0x7f1cd00b1980 2020-12-05T17:39:44.843 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:44.843 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2020-12-05T17:39:44.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:44.844 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2020-12-05T17:39:44.844 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2020-12-05T17:39:44.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:44.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:44.845 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2020-12-05T17:39:44.845 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2020-12-05T17:39:44.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:44.845 INFO:teuthology.orchestra.run.smithi032.stdout:] 2020-12-05T17:39:44.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.839+0000 7f1cd64d2700 1 -- 172.21.15.32:0/1143873595 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1cb0055440 msgr2=0x7f1cb0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.839+0000 7f1cd64d2700 1 --2- 172.21.15.32:0/1143873595 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1cb0055440 0x7f1cb0057900 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f1cc000b000 tx=0x7f1cc0009040).stop 2020-12-05T17:39:44.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.839+0000 7f1cd64d2700 1 -- 172.21.15.32:0/1143873595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cd00b1980 msgr2=0x7f1cd00658d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.839+0000 7f1cd64d2700 1 --2- 172.21.15.32:0/1143873595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cd00b1980 0x7f1cd00658d0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7f1cc4002f30 tx=0x7f1cc4015000).stop 2020-12-05T17:39:44.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.839+0000 7f1cd64d2700 1 -- 172.21.15.32:0/1143873595 shutdown_connections 2020-12-05T17:39:44.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.839+0000 7f1cd64d2700 1 --2- 172.21.15.32:0/1143873595 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1cd0146a70 0x7f1cd00648d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.839+0000 7f1cd64d2700 1 --2- 172.21.15.32:0/1143873595 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1cb0055440 0x7f1cb0057900 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.839+0000 7f1cd64d2700 1 --2- 172.21.15.32:0/1143873595 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1cd005f6f0 0x7f1cd00b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.839+0000 7f1cd64d2700 1 --2- 172.21.15.32:0/1143873595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cd00b1980 0x7f1cd00658d0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.839+0000 7f1cd64d2700 1 -- 172.21.15.32:0/1143873595 >> 172.21.15.32:0/1143873595 conn(0x7f1cd00604c0 msgr2=0x7f1cd0145200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:44.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.839+0000 7f1cd64d2700 1 -- 172.21.15.32:0/1143873595 shutdown_connections 2020-12-05T17:39:44.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.839+0000 7f1cd64d2700 1 -- 172.21.15.32:0/1143873595 wait complete. 2020-12-05T17:39:44.857 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-12-05T17:39:44.858 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-12-05T17:39:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.971+0000 7f5fd88a1700 1 Processor -- start 2020-12-05T17:39:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.971+0000 7f5fd88a1700 1 -- start start 2020-12-05T17:39:44.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.971+0000 7f5fd88a1700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f5fd4061ba0 0x7f5fd4066e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.971+0000 7f5fd88a1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5fd40b3020 0x7f5fd4143c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.971+0000 7f5fd88a1700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd40b3540 con 0x7f5fd40625c0 2020-12-05T17:39:44.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.971+0000 7f5fd88a1700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd4061f80 con 0x7f5fd4061ba0 2020-12-05T17:39:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.971+0000 7f5fd88a1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd4066750 con 0x7f5fd40b3020 2020-12-05T17:39:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.971+0000 7f5fd2d9d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f5fd4061ba0 0x7f5fd4066e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.971+0000 7f5fd2d9d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f5fd4061ba0 0x7f5fd4066e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:58522/0 (socket says 172.21.15.32:58522) 2020-12-05T17:39:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.971+0000 7f5fd2d9d700 1 -- 172.21.15.32:0/3044628131 learned_addr learned my addr 172.21.15.32:0/3044628131 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.971+0000 7f5fd359e700 1 --1- 172.21.15.32:0/3044628131 >> v1:172.21.15.61:6789/0 conn(0x7f5fd40625c0 0x7f5fd413a1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:53922/0 (socket says 172.21.15.32:53922) 2020-12-05T17:39:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.971+0000 7f5fd259c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5fd40b3020 0x7f5fd4143c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.971+0000 7f5fd2d9d700 1 -- 172.21.15.32:0/3044628131 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5fd40b3020 msgr2=0x7f5fd4143c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:44.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.971+0000 7f5fd2d9d700 1 --2- 172.21.15.32:0/3044628131 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5fd40b3020 0x7f5fd4143c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.971+0000 7f5fd2d9d700 1 -- 172.21.15.32:0/3044628131 >> v1:172.21.15.61:6789/0 conn(0x7f5fd40625c0 legacy=0x7f5fd413a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.971+0000 7f5fd2d9d700 1 -- 172.21.15.32:0/3044628131 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd40669f0 con 0x7f5fd4061ba0 2020-12-05T17:39:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.971+0000 7f5fd2d9d700 1 --2- 172.21.15.32:0/3044628131 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fd4061ba0 0x7f5fd4066e90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f5fc4002f30 tx=0x7f5fc4006e80).ready entity=mon.1 client_cookie=8ed92e526d7f001 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.971+0000 7f5fd1d9b700 1 -- 172.21.15.32:0/3044628131 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fc40097f0 con 0x7f5fd4061ba0 2020-12-05T17:39:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.971+0000 7f5fd1d9b700 1 -- 172.21.15.32:0/3044628131 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc4009950 con 0x7f5fd4061ba0 2020-12-05T17:39:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.971+0000 7f5fd1d9b700 1 -- 172.21.15.32:0/3044628131 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fc4009cb0 con 0x7f5fd4061ba0 2020-12-05T17:39:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.971+0000 7f5fd88a1700 1 -- 172.21.15.32:0/3044628131 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fd4061ba0 msgr2=0x7f5fd4066e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.971+0000 7f5fd88a1700 1 --2- 172.21.15.32:0/3044628131 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fd4061ba0 0x7f5fd4066e90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f5fc4002f30 tx=0x7f5fc4006e80).stop 2020-12-05T17:39:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.975+0000 7f5fd88a1700 1 -- 172.21.15.32:0/3044628131 shutdown_connections 2020-12-05T17:39:44.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.975+0000 7f5fd88a1700 1 --2- 172.21.15.32:0/3044628131 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fd4061ba0 0x7f5fd4066e90 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.975+0000 7f5fd88a1700 1 --2- 172.21.15.32:0/3044628131 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5fd40b3020 0x7f5fd4143c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.975+0000 7f5fd88a1700 1 -- 172.21.15.32:0/3044628131 >> 172.21.15.32:0/3044628131 conn(0x7f5fd40b6d10 msgr2=0x7f5fd40b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:44.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.975+0000 7f5fd88a1700 1 -- 172.21.15.32:0/3044628131 shutdown_connections 2020-12-05T17:39:44.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.975+0000 7f5fd88a1700 1 -- 172.21.15.32:0/3044628131 wait complete. 2020-12-05T17:39:44.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.975+0000 7f5fd88a1700 1 Processor -- start 2020-12-05T17:39:44.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.975+0000 7f5fd88a1700 1 -- start start 2020-12-05T17:39:44.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.979+0000 7f5fd88a1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5fd4061ba0 0x7f5fd40b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.979+0000 7f5fd88a1700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fd40625c0 0x7f5fd413b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.979+0000 7f5fd88a1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd40b3020 0x7f5fd40b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.979+0000 7f5fd88a1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd414f7b0 con 0x7f5fd40b3020 2020-12-05T17:39:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.979+0000 7f5fd88a1700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd413c780 con 0x7f5fd40625c0 2020-12-05T17:39:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.979+0000 7f5fd88a1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd4150840 con 0x7f5fd4061ba0 2020-12-05T17:39:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.979+0000 7f5fd359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd40b3020 0x7f5fd40b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.979+0000 7f5fd359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd40b3020 0x7f5fd40b07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38640/0 (socket says 172.21.15.32:38640) 2020-12-05T17:39:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.979+0000 7f5fd2d9d700 1 --2- 172.21.15.32:0/2957313032 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5fd4061ba0 0x7f5fd40b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.979+0000 7f5fd359e700 1 -- 172.21.15.32:0/2957313032 learned_addr learned my addr 172.21.15.32:0/2957313032 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.979+0000 7f5fd259c700 1 --2- 172.21.15.32:0/2957313032 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fd40625c0 0x7f5fd413b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.979+0000 7f5fd359e700 1 -- 172.21.15.32:0/2957313032 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5fd4061ba0 msgr2=0x7f5fd40b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.979+0000 7f5fd359e700 1 --2- 172.21.15.32:0/2957313032 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5fd4061ba0 0x7f5fd40b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.979+0000 7f5fd359e700 1 -- 172.21.15.32:0/2957313032 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fd40625c0 msgr2=0x7f5fd413b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:44.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.979+0000 7f5fd359e700 1 --2- 172.21.15.32:0/2957313032 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fd40625c0 0x7f5fd413b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:44.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.979+0000 7f5fd359e700 1 -- 172.21.15.32:0/2957313032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd413ca20 con 0x7f5fd40b3020 2020-12-05T17:39:44.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.979+0000 7f5fd359e700 1 --2- 172.21.15.32:0/2957313032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd40b3020 0x7f5fd40b07f0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7f5fd4144d90 tx=0x7f5fcc007ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:44.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.979+0000 7f5fc3fff700 1 -- 172.21.15.32:0/2957313032 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fcc00ce20 con 0x7f5fd40b3020 2020-12-05T17:39:44.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.979+0000 7f5fc3fff700 1 -- 172.21.15.32:0/2957313032 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fcc005ab0 con 0x7f5fd40b3020 2020-12-05T17:39:44.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.979+0000 7f5fd88a1700 1 -- 172.21.15.32:0/2957313032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd40ae7b0 con 0x7f5fd40b3020 2020-12-05T17:39:44.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.979+0000 7f5fc3fff700 1 -- 172.21.15.32:0/2957313032 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fcc005e10 con 0x7f5fd40b3020 2020-12-05T17:39:44.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.979+0000 7f5fd88a1700 1 -- 172.21.15.32:0/2957313032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd4140de0 con 0x7f5fd40b3020 2020-12-05T17:39:44.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.979+0000 7f5fd88a1700 1 -- 172.21.15.32:0/2957313032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd4057050 con 0x7f5fd40b3020 2020-12-05T17:39:44.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.983+0000 7f5fc3fff700 1 -- 172.21.15.32:0/2957313032 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5fcc010030 con 0x7f5fd40b3020 2020-12-05T17:39:44.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.983+0000 7f5fc3fff700 1 --2- 172.21.15.32:0/2957313032 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5fb0055440 0x7f5fb0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:44.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.983+0000 7f5fc3fff700 1 -- 172.21.15.32:0/2957313032 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f5fcc06dc70 con 0x7f5fd40b3020 2020-12-05T17:39:44.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.983+0000 7f5fd2d9d700 1 --2- 172.21.15.32:0/2957313032 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5fb0055440 0x7f5fb0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:44.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.983+0000 7f5fd2d9d700 1 --2- 172.21.15.32:0/2957313032 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5fb0055440 0x7f5fb0057900 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f5fc401c000 tx=0x7f5fc4015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:44.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:44.987+0000 7f5fc3fff700 1 -- 172.21.15.32:0/2957313032 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5fcc044150 con 0x7f5fd40b3020 2020-12-05T17:39:45.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:45.135+0000 7f5fd88a1700 1 -- 172.21.15.32:0/2957313032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f5fd4142360 con 0x7f5fd40b3020 2020-12-05T17:39:45.767 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:39:45.769 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:39:45.772 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:39:45.775 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:39:45.777 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:39:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.107+0000 7f5fc3fff700 1 -- 172.21.15.32:0/2957313032 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v99) v1 ==== 133+0+0 (secure 0 0 0) 0x7f5fcc047770 con 0x7f5fd40b3020 2020-12-05T17:39:46.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.111+0000 7f5fd88a1700 1 -- 172.21.15.32:0/2957313032 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5fb0055440 msgr2=0x7f5fb0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.111+0000 7f5fd88a1700 1 --2- 172.21.15.32:0/2957313032 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5fb0055440 0x7f5fb0057900 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f5fc401c000 tx=0x7f5fc4015000).stop 2020-12-05T17:39:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.111+0000 7f5fd88a1700 1 -- 172.21.15.32:0/2957313032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd40b3020 msgr2=0x7f5fd40b07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.111+0000 7f5fd88a1700 1 --2- 172.21.15.32:0/2957313032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd40b3020 0x7f5fd40b07f0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7f5fd4144d90 tx=0x7f5fcc007ee0).stop 2020-12-05T17:39:46.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.111+0000 7f5fd88a1700 1 -- 172.21.15.32:0/2957313032 shutdown_connections 2020-12-05T17:39:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.111+0000 7f5fd88a1700 1 --2- 172.21.15.32:0/2957313032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd40b3020 0x7f5fd40b07f0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.111+0000 7f5fd88a1700 1 --2- 172.21.15.32:0/2957313032 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5fb0055440 0x7f5fb0057900 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.111+0000 7f5fd88a1700 1 --2- 172.21.15.32:0/2957313032 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5fd4061ba0 0x7f5fd40b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:46.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.111+0000 7f5fd88a1700 1 --2- 172.21.15.32:0/2957313032 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fd40625c0 0x7f5fd413b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:46.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.111+0000 7f5fd88a1700 1 -- 172.21.15.32:0/2957313032 >> 172.21.15.32:0/2957313032 conn(0x7f5fd40b6d10 msgr2=0x7f5fd40b6190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:46.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.111+0000 7f5fd88a1700 1 -- 172.21.15.32:0/2957313032 shutdown_connections 2020-12-05T17:39:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.111+0000 7f5fd88a1700 1 -- 172.21.15.32:0/2957313032 wait complete. 2020-12-05T17:39:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2020-12-05T17:39:46.134 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-12-05T17:39:46.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.251+0000 7f7abecf8700 1 Processor -- start 2020-12-05T17:39:46.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.251+0000 7f7abecf8700 1 -- start start 2020-12-05T17:39:46.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.255+0000 7f7abecf8700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f7ab805f3b0 0x7f7ab805f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:46.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.255+0000 7f7abecf8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7ab80b2fb0 0x7f7ab8061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:46.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.255+0000 7f7abecf8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab80b2df0 con 0x7f7ab805feb0 2020-12-05T17:39:46.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.255+0000 7f7abecf8700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ab80b3c90 con 0x7f7ab805f3b0 2020-12-05T17:39:46.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.255+0000 7f7abecf8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab80b2b50 con 0x7f7ab80b2fb0 2020-12-05T17:39:46.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.255+0000 7f7abd4f5700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7ab805feb0 0x7f7ab813c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41292/0 (socket says 172.21.15.32:41292) 2020-12-05T17:39:46.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.255+0000 7f7abd4f5700 1 -- 172.21.15.32:0/1657189961 learned_addr learned my addr 172.21.15.32:0/1657189961 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:46.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.255+0000 7f7abdcf6700 1 --2- 172.21.15.32:0/1657189961 >> v2:172.21.15.61:3300/0 conn(0x7f7ab805f3b0 0x7f7ab805f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:46.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.255+0000 7f7abe4f7700 1 --2- 172.21.15.32:0/1657189961 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7ab80b2fb0 0x7f7ab8061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:46.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.255+0000 7f7abccf4700 1 -- 172.21.15.32:0/1657189961 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3451469017 0 0) 0x7f7ab80b2df0 con 0x7f7ab805feb0 2020-12-05T17:39:46.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.255+0000 7f7abccf4700 1 -- 172.21.15.32:0/1657189961 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7aa0003230 con 0x7f7ab805feb0 2020-12-05T17:39:46.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.255+0000 7f7abe4f7700 1 -- 172.21.15.32:0/1657189961 >> v2:172.21.15.61:3300/0 conn(0x7f7ab805f3b0 msgr2=0x7f7ab805f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:46.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.255+0000 7f7abe4f7700 1 --2- 172.21.15.32:0/1657189961 >> v2:172.21.15.61:3300/0 conn(0x7f7ab805f3b0 0x7f7ab805f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:46.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.255+0000 7f7abccf4700 1 -- 172.21.15.32:0/1657189961 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3726025951 0 0) 0x7f7aa0003230 con 0x7f7ab805feb0 2020-12-05T17:39:46.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.255+0000 7f7abe4f7700 1 -- 172.21.15.32:0/1657189961 >> v1:172.21.15.32:6789/0 conn(0x7f7ab805feb0 legacy=0x7f7ab813c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:46.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.255+0000 7f7abe4f7700 1 -- 172.21.15.32:0/1657189961 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab8142820 con 0x7f7ab80b2fb0 2020-12-05T17:39:46.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.255+0000 7f7abdcf6700 1 --2- 172.21.15.32:0/1657189961 >> v2:172.21.15.61:3300/0 conn(0x7f7ab805f3b0 0x7f7ab805f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:46.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.255+0000 7f7abe4f7700 1 --2- 172.21.15.32:0/1657189961 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7ab80b2fb0 0x7f7ab8061780 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f7ab4001870 tx=0x7f7ab00091a0).ready entity=mon.2 client_cookie=8fcb94ad3defe00c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:46.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.255+0000 7f7abccf4700 1 -- 172.21.15.32:0/1657189961 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ab40016d0 con 0x7f7ab80b2fb0 2020-12-05T17:39:46.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.255+0000 7f7abccf4700 1 -- 172.21.15.32:0/1657189961 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab000b800 con 0x7f7ab80b2fb0 2020-12-05T17:39:46.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.255+0000 7f7abccf4700 1 -- 172.21.15.32:0/1657189961 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ab000b970 con 0x7f7ab80b2fb0 2020-12-05T17:39:46.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.255+0000 7f7abecf8700 1 -- 172.21.15.32:0/1657189961 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7ab80b2fb0 msgr2=0x7f7ab8061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:46.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.255+0000 7f7abecf8700 1 --2- 172.21.15.32:0/1657189961 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7ab80b2fb0 0x7f7ab8061780 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f7ab4001870 tx=0x7f7ab00091a0).stop 2020-12-05T17:39:46.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.259+0000 7f7abecf8700 1 -- 172.21.15.32:0/1657189961 shutdown_connections 2020-12-05T17:39:46.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.259+0000 7f7abecf8700 1 --2- 172.21.15.32:0/1657189961 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7ab80b2fb0 0x7f7ab8061780 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:46.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.259+0000 7f7abecf8700 1 --2- 172.21.15.32:0/1657189961 >> v2:172.21.15.61:3300/0 conn(0x7f7ab805f3b0 0x7f7ab805f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:46.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.259+0000 7f7abecf8700 1 -- 172.21.15.32:0/1657189961 >> 172.21.15.32:0/1657189961 conn(0x7f7ab80b8f20 msgr2=0x7f7ab80b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:46.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.259+0000 7f7abecf8700 1 -- 172.21.15.32:0/1657189961 shutdown_connections 2020-12-05T17:39:46.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.259+0000 7f7abecf8700 1 -- 172.21.15.32:0/1657189961 wait complete. 2020-12-05T17:39:46.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.259+0000 7f7abecf8700 1 Processor -- start 2020-12-05T17:39:46.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.259+0000 7f7abecf8700 1 -- start start 2020-12-05T17:39:46.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.263+0000 7f7abecf8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7ab805f3b0 0x7f7ab8141a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:46.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.263+0000 7f7abecf8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ab805feb0 0x7f7ab8143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:46.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.263+0000 7f7abecf8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7ab80b2fb0 0x7f7ab8149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:46.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.263+0000 7f7abecf8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab81452d0 con 0x7f7ab805feb0 2020-12-05T17:39:46.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.263+0000 7f7abecf8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab8146860 con 0x7f7ab805f3b0 2020-12-05T17:39:46.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.263+0000 7f7abecf8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab813fb70 con 0x7f7ab80b2fb0 2020-12-05T17:39:46.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.263+0000 7f7abd4f5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ab805feb0 0x7f7ab8143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.263+0000 7f7abd4f5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ab805feb0 0x7f7ab8143eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38652/0 (socket says 172.21.15.32:38652) 2020-12-05T17:39:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.263+0000 7f7abdcf6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7ab805f3b0 0x7f7ab8141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.263+0000 7f7abd4f5700 1 -- 172.21.15.32:0/4098620514 learned_addr learned my addr 172.21.15.32:0/4098620514 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.263+0000 7f7abe4f7700 1 --2- 172.21.15.32:0/4098620514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7ab80b2fb0 0x7f7ab8149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.263+0000 7f7abd4f5700 1 -- 172.21.15.32:0/4098620514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7ab80b2fb0 msgr2=0x7f7ab8149080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:46.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.263+0000 7f7abd4f5700 1 --2- 172.21.15.32:0/4098620514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7ab80b2fb0 0x7f7ab8149080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:46.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.263+0000 7f7abd4f5700 1 -- 172.21.15.32:0/4098620514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7ab805f3b0 msgr2=0x7f7ab8141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:46.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.263+0000 7f7abd4f5700 1 --2- 172.21.15.32:0/4098620514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7ab805f3b0 0x7f7ab8141a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:46.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.263+0000 7f7abd4f5700 1 -- 172.21.15.32:0/4098620514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab8146a90 con 0x7f7ab805feb0 2020-12-05T17:39:46.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.263+0000 7f7abd4f5700 1 --2- 172.21.15.32:0/4098620514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ab805feb0 0x7f7ab8143eb0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 rx=0x7f7ab4001930 tx=0x7f7ab4001b20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.263+0000 7f7aaaffd700 1 -- 172.21.15.32:0/4098620514 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ab400ce90 con 0x7f7ab805feb0 2020-12-05T17:39:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.263+0000 7f7aaaffd700 1 -- 172.21.15.32:0/4098620514 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab4007170 con 0x7f7ab805feb0 2020-12-05T17:39:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.263+0000 7f7aaaffd700 1 -- 172.21.15.32:0/4098620514 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ab400a7b0 con 0x7f7ab805feb0 2020-12-05T17:39:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.263+0000 7f7abecf8700 1 -- 172.21.15.32:0/4098620514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab8144700 con 0x7f7ab805feb0 2020-12-05T17:39:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.263+0000 7f7abecf8700 1 -- 172.21.15.32:0/4098620514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab8140570 con 0x7f7ab805feb0 2020-12-05T17:39:46.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.263+0000 7f7abecf8700 1 -- 172.21.15.32:0/4098620514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab81418a0 con 0x7f7ab805feb0 2020-12-05T17:39:46.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.267+0000 7f7aaaffd700 1 -- 172.21.15.32:0/4098620514 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7ab4010070 con 0x7f7ab805feb0 2020-12-05T17:39:46.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.267+0000 7f7aaaffd700 1 --2- 172.21.15.32:0/4098620514 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7a98055330 0x7f7a980577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:46.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.267+0000 7f7aaaffd700 1 -- 172.21.15.32:0/4098620514 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f7ab406daa0 con 0x7f7ab805feb0 2020-12-05T17:39:46.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.267+0000 7f7abdcf6700 1 --2- 172.21.15.32:0/4098620514 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7a98055330 0x7f7a980577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:46.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.267+0000 7f7abdcf6700 1 --2- 172.21.15.32:0/4098620514 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7a98055330 0x7f7a980577f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f7aac00b000 tx=0x7f7aac006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:46.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.271+0000 7f7aaaffd700 1 -- 172.21.15.32:0/4098620514 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7ab4043d00 con 0x7f7ab805feb0 2020-12-05T17:39:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:46.415+0000 7f7abecf8700 1 -- 172.21.15.32:0/4098620514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f7ab8057050 con 0x7f7ab805feb0 2020-12-05T17:39:47.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.111+0000 7f7aaaffd700 1 -- 172.21.15.32:0/4098620514 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v100) v1 ==== 125+0+0 (secure 0 0 0) 0x7f7ab4019030 con 0x7f7ab805feb0 2020-12-05T17:39:47.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.119+0000 7f7abecf8700 1 -- 172.21.15.32:0/4098620514 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7a98055330 msgr2=0x7f7a980577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.119+0000 7f7abecf8700 1 --2- 172.21.15.32:0/4098620514 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7a98055330 0x7f7a980577f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f7aac00b000 tx=0x7f7aac006dc0).stop 2020-12-05T17:39:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.119+0000 7f7abecf8700 1 -- 172.21.15.32:0/4098620514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ab805feb0 msgr2=0x7f7ab8143eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.119+0000 7f7abecf8700 1 --2- 172.21.15.32:0/4098620514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ab805feb0 0x7f7ab8143eb0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 rx=0x7f7ab4001930 tx=0x7f7ab4001b20).stop 2020-12-05T17:39:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.123+0000 7f7abecf8700 1 -- 172.21.15.32:0/4098620514 shutdown_connections 2020-12-05T17:39:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.123+0000 7f7abecf8700 1 --2- 172.21.15.32:0/4098620514 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7a98055330 0x7f7a980577f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.123+0000 7f7abecf8700 1 --2- 172.21.15.32:0/4098620514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7ab80b2fb0 0x7f7ab8149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.123+0000 7f7abecf8700 1 --2- 172.21.15.32:0/4098620514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7ab805f3b0 0x7f7ab8141a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.123+0000 7f7abecf8700 1 --2- 172.21.15.32:0/4098620514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ab805feb0 0x7f7ab8143eb0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.123+0000 7f7abecf8700 1 -- 172.21.15.32:0/4098620514 >> 172.21.15.32:0/4098620514 conn(0x7f7ab80b8f20 msgr2=0x7f7ab8142010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:47.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.127+0000 7f7abecf8700 1 -- 172.21.15.32:0/4098620514 shutdown_connections 2020-12-05T17:39:47.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.127+0000 7f7abecf8700 1 -- 172.21.15.32:0/4098620514 wait complete. 2020-12-05T17:39:47.133 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2020-12-05T17:39:47.145 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-12-05T17:39:47.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.259+0000 7fabf1ccc700 1 Processor -- start 2020-12-05T17:39:47.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.259+0000 7fabf1ccc700 1 -- start start 2020-12-05T17:39:47.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.263+0000 7fabf1ccc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fabec05f3b0 0x7fabec05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:47.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.263+0000 7fabf1ccc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabec0b2df0 con 0x7fabec0b2fb0 2020-12-05T17:39:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.263+0000 7fabf1ccc700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabec0b3c90 con 0x7fabec05feb0 2020-12-05T17:39:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.263+0000 7fabf1ccc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabec0b2b50 con 0x7fabec05f3b0 2020-12-05T17:39:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.263+0000 7fabf0cca700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fabec05f3b0 0x7fabec05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.263+0000 7fabf14cb700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fabec0b2fb0 0x7fabec142cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41310/0 (socket says 172.21.15.32:41310) 2020-12-05T17:39:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.263+0000 7fabf14cb700 1 -- 172.21.15.32:0/268601633 learned_addr learned my addr 172.21.15.32:0/268601633 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:47.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.263+0000 7fabf0cca700 1 -- 172.21.15.32:0/268601633 >> v1:172.21.15.61:6789/0 conn(0x7fabec05feb0 legacy=0x7fabec13c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:47.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.263+0000 7fabf0cca700 1 -- 172.21.15.32:0/268601633 >> v1:172.21.15.32:6789/0 conn(0x7fabec0b2fb0 legacy=0x7fabec142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:47.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.263+0000 7fabf0cca700 1 -- 172.21.15.32:0/268601633 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec142820 con 0x7fabec05f3b0 2020-12-05T17:39:47.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.263+0000 7fabeb7fe700 1 -- 172.21.15.32:0/268601633 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825540071 0 0) 0x7fabec0b2df0 con 0x7fabec0b2fb0 2020-12-05T17:39:47.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.263+0000 7fabf0cca700 1 --2- 172.21.15.32:0/268601633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabec05f3b0 0x7fabec05f790 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0x7fabe0002f30 tx=0x7fabe0006dc0).ready entity=mon.0 client_cookie=8f49a5f49b3874b2 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:47.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.263+0000 7fabeb7fe700 1 -- 172.21.15.32:0/268601633 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabe00097f0 con 0x7fabec05f3b0 2020-12-05T17:39:47.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.263+0000 7fabeb7fe700 1 -- 172.21.15.32:0/268601633 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe0009950 con 0x7fabec05f3b0 2020-12-05T17:39:47.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.263+0000 7fabeb7fe700 1 -- 172.21.15.32:0/268601633 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabe0009cb0 con 0x7fabec05f3b0 2020-12-05T17:39:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.263+0000 7fabf1ccc700 1 -- 172.21.15.32:0/268601633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabec05f3b0 msgr2=0x7fabec05f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.263+0000 7fabf1ccc700 1 --2- 172.21.15.32:0/268601633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabec05f3b0 0x7fabec05f790 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0x7fabe0002f30 tx=0x7fabe0006dc0).stop 2020-12-05T17:39:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.263+0000 7fabf1ccc700 1 -- 172.21.15.32:0/268601633 shutdown_connections 2020-12-05T17:39:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.263+0000 7fabf1ccc700 1 --2- 172.21.15.32:0/268601633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabec05f3b0 0x7fabec05f790 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.263+0000 7fabf1ccc700 1 -- 172.21.15.32:0/268601633 >> 172.21.15.32:0/268601633 conn(0x7fabec0b8f20 msgr2=0x7fabec0b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:47.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.263+0000 7fabf1ccc700 1 -- 172.21.15.32:0/268601633 shutdown_connections 2020-12-05T17:39:47.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.263+0000 7fabf1ccc700 1 -- 172.21.15.32:0/268601633 wait complete. 2020-12-05T17:39:47.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.267+0000 7fabf1ccc700 1 Processor -- start 2020-12-05T17:39:47.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.267+0000 7fabf1ccc700 1 -- start start 2020-12-05T17:39:47.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.267+0000 7fabf1ccc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabec05f3b0 0x7fabec1480f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:47.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.267+0000 7fabf1ccc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fabec05feb0 0x7fabec1451a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:47.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.267+0000 7fabf1ccc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fabec0b2fb0 0x7fabec143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:47.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.267+0000 7fabf1ccc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec145690 con 0x7fabec05f3b0 2020-12-05T17:39:47.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.267+0000 7fabf1ccc700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec13f430 con 0x7fabec05feb0 2020-12-05T17:39:47.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.267+0000 7fabf1ccc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabec13cef0 con 0x7fabec0b2fb0 2020-12-05T17:39:47.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.267+0000 7fabf0cca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabec05f3b0 0x7fabec1480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:47.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.267+0000 7fabf0cca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabec05f3b0 0x7fabec1480f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38664/0 (socket says 172.21.15.32:38664) 2020-12-05T17:39:47.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.267+0000 7fabf0cca700 1 -- 172.21.15.32:0/2433997755 learned_addr learned my addr 172.21.15.32:0/2433997755 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:47.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.267+0000 7fabebfff700 1 --2- 172.21.15.32:0/2433997755 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fabec05feb0 0x7fabec1451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:47.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.267+0000 7fabf14cb700 1 --2- 172.21.15.32:0/2433997755 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fabec0b2fb0 0x7fabec143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:47.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.267+0000 7fabf0cca700 1 -- 172.21.15.32:0/2433997755 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fabec0b2fb0 msgr2=0x7fabec143eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:47.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.267+0000 7fabf0cca700 1 --2- 172.21.15.32:0/2433997755 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fabec0b2fb0 0x7fabec143eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.267+0000 7fabf0cca700 1 -- 172.21.15.32:0/2433997755 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fabec05feb0 msgr2=0x7fabec1451a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:47.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.267+0000 7fabf0cca700 1 --2- 172.21.15.32:0/2433997755 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fabec05feb0 0x7fabec1451a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.267+0000 7fabf0cca700 1 -- 172.21.15.32:0/2433997755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec13d190 con 0x7fabec05f3b0 2020-12-05T17:39:47.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.267+0000 7fabf0cca700 1 --2- 172.21.15.32:0/2433997755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabec05f3b0 0x7fabec1480f0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0x7fabe0002f30 tx=0x7fabe0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:47.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.267+0000 7fabe9ffb700 1 -- 172.21.15.32:0/2433997755 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabe0021070 con 0x7fabec05f3b0 2020-12-05T17:39:47.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.267+0000 7fabe9ffb700 1 -- 172.21.15.32:0/2433997755 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe001d450 con 0x7fabec05f3b0 2020-12-05T17:39:47.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.267+0000 7fabf1ccc700 1 -- 172.21.15.32:0/2433997755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabec13f0b0 con 0x7fabec05f3b0 2020-12-05T17:39:47.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.271+0000 7fabe9ffb700 1 -- 172.21.15.32:0/2433997755 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabe001d7b0 con 0x7fabec05f3b0 2020-12-05T17:39:47.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.271+0000 7fabf1ccc700 1 -- 172.21.15.32:0/2433997755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabec146ad0 con 0x7fabec05f3b0 2020-12-05T17:39:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.271+0000 7fabf1ccc700 1 -- 172.21.15.32:0/2433997755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabec13d320 con 0x7fabec05f3b0 2020-12-05T17:39:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.271+0000 7fabe9ffb700 1 -- 172.21.15.32:0/2433997755 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fabe000f070 con 0x7fabec05f3b0 2020-12-05T17:39:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.271+0000 7fabe9ffb700 1 --2- 172.21.15.32:0/2433997755 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fabd4055440 0x7fabd4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.271+0000 7fabe9ffb700 1 -- 172.21.15.32:0/2433997755 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fabe006fa80 con 0x7fabec05f3b0 2020-12-05T17:39:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.271+0000 7fabebfff700 1 --2- 172.21.15.32:0/2433997755 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fabd4055440 0x7fabd4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.275+0000 7fabebfff700 1 --2- 172.21.15.32:0/2433997755 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fabd4055440 0x7fabd4057900 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7fabd800d000 tx=0x7fabd8007b60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.279+0000 7fabe9ffb700 1 -- 172.21.15.32:0/2433997755 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fabe0045b90 con 0x7fabec05f3b0 2020-12-05T17:39:47.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.463+0000 7fabf1ccc700 1 -- 172.21.15.32:0/2433997755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fabec13f240 con 0x7fabec05f3b0 2020-12-05T17:39:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.479+0000 7fabe9ffb700 1 -- 172.21.15.32:0/2433997755 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 19 and data pool 20 v46) v1 ==== 189+0+0 (secure 0 0 0) 0x7fabe00491b0 con 0x7fabec05f3b0 2020-12-05T17:39:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.479+0000 7fabf1ccc700 1 -- 172.21.15.32:0/2433997755 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fabd4055440 msgr2=0x7fabd4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.483+0000 7fabf1ccc700 1 --2- 172.21.15.32:0/2433997755 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fabd4055440 0x7fabd4057900 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7fabd800d000 tx=0x7fabd8007b60).stop 2020-12-05T17:39:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.483+0000 7fabf1ccc700 1 -- 172.21.15.32:0/2433997755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabec05f3b0 msgr2=0x7fabec1480f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.483+0000 7fabf1ccc700 1 --2- 172.21.15.32:0/2433997755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabec05f3b0 0x7fabec1480f0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0x7fabe0002f30 tx=0x7fabe0015000).stop 2020-12-05T17:39:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.483+0000 7fabf1ccc700 1 -- 172.21.15.32:0/2433997755 shutdown_connections 2020-12-05T17:39:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.483+0000 7fabf1ccc700 1 --2- 172.21.15.32:0/2433997755 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fabd4055440 0x7fabd4057900 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.483+0000 7fabf1ccc700 1 --2- 172.21.15.32:0/2433997755 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fabec0b2fb0 0x7fabec143eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.483+0000 7fabf1ccc700 1 --2- 172.21.15.32:0/2433997755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabec05f3b0 0x7fabec1480f0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.483+0000 7fabf1ccc700 1 --2- 172.21.15.32:0/2433997755 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fabec05feb0 0x7fabec1451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.483+0000 7fabf1ccc700 1 -- 172.21.15.32:0/2433997755 >> 172.21.15.32:0/2433997755 conn(0x7fabec0b8f20 msgr2=0x7fabec0b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.483+0000 7fabf1ccc700 1 -- 172.21.15.32:0/2433997755 shutdown_connections 2020-12-05T17:39:47.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.483+0000 7fabf1ccc700 1 -- 172.21.15.32:0/2433997755 wait complete. 2020-12-05T17:39:47.490 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 19 and data pool 20 2020-12-05T17:39:47.507 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:39:47.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.607+0000 7f4ce3460700 1 Processor -- start 2020-12-05T17:39:47.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.607+0000 7f4ce3460700 1 -- start start 2020-12-05T17:39:47.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.607+0000 7f4ce3460700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4cdc0af680 0x7f4cdc0b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:47.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.607+0000 7f4ce3460700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4cdc066ba0 0x7f4cdc0b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:47.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.607+0000 7f4ce3460700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cdc140070 con 0x7f4cdc0afa60 2020-12-05T17:39:47.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.607+0000 7f4ce3460700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cdc0ae340 con 0x7f4cdc0af680 2020-12-05T17:39:47.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.607+0000 7f4ce3460700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc0adf10 con 0x7f4cdc066ba0 2020-12-05T17:39:47.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.607+0000 7f4ce11fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4cdc0af680 0x7f4cdc0b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:47.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.607+0000 7f4ce11fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4cdc0af680 0x7f4cdc0b14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38684/0 (socket says 172.21.15.32:38684) 2020-12-05T17:39:47.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.607+0000 7f4ce11fc700 1 -- 172.21.15.32:0/1082058288 learned_addr learned my addr 172.21.15.32:0/1082058288 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:47.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.607+0000 7f4ce19fd700 1 --2- 172.21.15.32:0/1082058288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4cdc066ba0 0x7f4cdc0b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:47.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.607+0000 7f4ce11fc700 1 -- 172.21.15.32:0/1082058288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4cdc066ba0 msgr2=0x7f4cdc0b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:47.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.607+0000 7f4ce11fc700 1 --2- 172.21.15.32:0/1082058288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4cdc066ba0 0x7f4cdc0b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.607+0000 7f4ce11fc700 1 -- 172.21.15.32:0/1082058288 >> v1:172.21.15.61:6789/0 conn(0x7f4cdc0afa60 legacy=0x7f4cdc142b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:47.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.607+0000 7f4ce11fc700 1 -- 172.21.15.32:0/1082058288 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cdc140310 con 0x7f4cdc0af680 2020-12-05T17:39:47.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.607+0000 7f4ce19fd700 1 --2- 172.21.15.32:0/1082058288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4cdc066ba0 0x7f4cdc0b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:39:47.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.607+0000 7f4ce11fc700 1 --2- 172.21.15.32:0/1082058288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cdc0af680 0x7f4cdc0b14e0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 rx=0x7f4cdc061c70 tx=0x7f4cd8006e80).ready entity=mon.0 client_cookie=a3d093071589ca3f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:47.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.607+0000 7f4ccbfff700 1 -- 172.21.15.32:0/1082058288 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4cd80097b0 con 0x7f4cdc0af680 2020-12-05T17:39:47.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.607+0000 7f4ccbfff700 1 -- 172.21.15.32:0/1082058288 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd8009910 con 0x7f4cdc0af680 2020-12-05T17:39:47.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.607+0000 7f4ccbfff700 1 -- 172.21.15.32:0/1082058288 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4cd8009ac0 con 0x7f4cdc0af680 2020-12-05T17:39:47.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.607+0000 7f4ce3460700 1 -- 172.21.15.32:0/1082058288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cdc0af680 msgr2=0x7f4cdc0b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:47.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.607+0000 7f4ce3460700 1 --2- 172.21.15.32:0/1082058288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cdc0af680 0x7f4cdc0b14e0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 rx=0x7f4cdc061c70 tx=0x7f4cd8006e80).stop 2020-12-05T17:39:47.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.611+0000 7f4ce3460700 1 -- 172.21.15.32:0/1082058288 shutdown_connections 2020-12-05T17:39:47.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.611+0000 7f4ce3460700 1 --2- 172.21.15.32:0/1082058288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4cdc066ba0 0x7f4cdc0b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.611+0000 7f4ce3460700 1 --2- 172.21.15.32:0/1082058288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cdc0af680 0x7f4cdc0b14e0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.611+0000 7f4ce3460700 1 -- 172.21.15.32:0/1082058288 >> 172.21.15.32:0/1082058288 conn(0x7f4cdc0b5580 msgr2=0x7f4cdc0b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:47.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.611+0000 7f4ce3460700 1 -- 172.21.15.32:0/1082058288 shutdown_connections 2020-12-05T17:39:47.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.611+0000 7f4ce3460700 1 -- 172.21.15.32:0/1082058288 wait complete. 2020-12-05T17:39:47.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.611+0000 7f4ce3460700 1 Processor -- start 2020-12-05T17:39:47.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.611+0000 7f4ce3460700 1 -- start start 2020-12-05T17:39:47.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.611+0000 7f4ce3460700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cdc066ba0 0x7f4cdc13c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:47.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.611+0000 7f4ce3460700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4cdc0af680 0x7f4cdc13e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:47.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.611+0000 7f4ce3460700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4cdc0afa60 0x7f4cdc13d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:47.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.611+0000 7f4ce3460700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc134dd0 con 0x7f4cdc066ba0 2020-12-05T17:39:47.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.611+0000 7f4ce3460700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc135070 con 0x7f4cdc0af680 2020-12-05T17:39:47.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.611+0000 7f4ce3460700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc135310 con 0x7f4cdc0afa60 2020-12-05T17:39:47.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.611+0000 7f4ce11fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cdc066ba0 0x7f4cdc13c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:47.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.611+0000 7f4ce11fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cdc066ba0 0x7f4cdc13c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38690/0 (socket says 172.21.15.32:38690) 2020-12-05T17:39:47.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.611+0000 7f4ce11fc700 1 -- 172.21.15.32:0/2916197020 learned_addr learned my addr 172.21.15.32:0/2916197020 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:47.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.611+0000 7f4ce19fd700 1 --2- 172.21.15.32:0/2916197020 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4cdc0afa60 0x7f4cdc13d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:47.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.611+0000 7f4ce09fb700 1 --2- 172.21.15.32:0/2916197020 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4cdc0af680 0x7f4cdc13e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:47.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.615+0000 7f4ce11fc700 1 -- 172.21.15.32:0/2916197020 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4cdc0afa60 msgr2=0x7f4cdc13d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:47.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.615+0000 7f4ce11fc700 1 --2- 172.21.15.32:0/2916197020 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4cdc0afa60 0x7f4cdc13d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.615+0000 7f4ce11fc700 1 -- 172.21.15.32:0/2916197020 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4cdc0af680 msgr2=0x7f4cdc13e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:47.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.615+0000 7f4ce11fc700 1 --2- 172.21.15.32:0/2916197020 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4cdc0af680 0x7f4cdc13e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.615+0000 7f4ce11fc700 1 -- 172.21.15.32:0/2916197020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cdc134530 con 0x7f4cdc066ba0 2020-12-05T17:39:47.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.615+0000 7f4ce11fc700 1 --2- 172.21.15.32:0/2916197020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cdc066ba0 0x7f4cdc13c6d0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7f4cdc061c70 tx=0x7f4cd8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:47.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.615+0000 7f4cca7fc700 1 -- 172.21.15.32:0/2916197020 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4cd8021440 con 0x7f4cdc066ba0 2020-12-05T17:39:47.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.615+0000 7f4cca7fc700 1 -- 172.21.15.32:0/2916197020 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd80215a0 con 0x7f4cdc066ba0 2020-12-05T17:39:47.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.615+0000 7f4cca7fc700 1 -- 172.21.15.32:0/2916197020 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4cd8021900 con 0x7f4cdc066ba0 2020-12-05T17:39:47.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.615+0000 7f4ce3460700 1 -- 172.21.15.32:0/2916197020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cdc1349a0 con 0x7f4cdc066ba0 2020-12-05T17:39:47.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.615+0000 7f4ce3460700 1 -- 172.21.15.32:0/2916197020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cdc13d830 con 0x7f4cdc066ba0 2020-12-05T17:39:47.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.615+0000 7f4ce3460700 1 -- 172.21.15.32:0/2916197020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cdc057050 con 0x7f4cdc066ba0 2020-12-05T17:39:47.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.615+0000 7f4cca7fc700 1 -- 172.21.15.32:0/2916197020 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4cd8027020 con 0x7f4cdc066ba0 2020-12-05T17:39:47.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.615+0000 7f4cca7fc700 1 --2- 172.21.15.32:0/2916197020 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4cbc0553b0 0x7f4cbc057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:47.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.615+0000 7f4cca7fc700 1 -- 172.21.15.32:0/2916197020 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4cd806f4f0 con 0x7f4cdc066ba0 2020-12-05T17:39:47.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.615+0000 7f4ce09fb700 1 --2- 172.21.15.32:0/2916197020 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4cbc0553b0 0x7f4cbc057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:47.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.619+0000 7f4ce09fb700 1 --2- 172.21.15.32:0/2916197020 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4cbc0553b0 0x7f4cbc057870 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f4ccc00c000 tx=0x7f4ccc006ca0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:47.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.623+0000 7f4cca7fc700 1 -- 172.21.15.32:0/2916197020 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4cd80456d0 con 0x7f4cdc066ba0 2020-12-05T17:39:47.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.763+0000 7f4ce3460700 1 -- 172.21.15.32:0/2916197020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4cdc13cf40 con 0x7f4cdc066ba0 2020-12-05T17:39:47.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.767+0000 7f4cca7fc700 1 -- 172.21.15.32:0/2916197020 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+16295 (secure 0 0 0) 0x7f4cd8015090 con 0x7f4cdc066ba0 2020-12-05T17:39:47.773 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:47.774 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":101,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:39:47.470782+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:39:45.190638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2020-12-05T17:39:46.472081+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:39:47.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.767+0000 7f4ce3460700 1 -- 172.21.15.32:0/2916197020 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4cbc0553b0 msgr2=0x7f4cbc057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:47.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.767+0000 7f4ce3460700 1 --2- 172.21.15.32:0/2916197020 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4cbc0553b0 0x7f4cbc057870 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f4ccc00c000 tx=0x7f4ccc006ca0).stop 2020-12-05T17:39:47.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.767+0000 7f4ce3460700 1 -- 172.21.15.32:0/2916197020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cdc066ba0 msgr2=0x7f4cdc13c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.767+0000 7f4ce3460700 1 --2- 172.21.15.32:0/2916197020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cdc066ba0 0x7f4cdc13c6d0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7f4cdc061c70 tx=0x7f4cd8016000).stop 2020-12-05T17:39:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.767+0000 7f4ce3460700 1 -- 172.21.15.32:0/2916197020 shutdown_connections 2020-12-05T17:39:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.767+0000 7f4ce3460700 1 --2- 172.21.15.32:0/2916197020 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4cbc0553b0 0x7f4cbc057870 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.767+0000 7f4ce3460700 1 --2- 172.21.15.32:0/2916197020 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4cdc0afa60 0x7f4cdc13d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.767+0000 7f4ce3460700 1 --2- 172.21.15.32:0/2916197020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cdc066ba0 0x7f4cdc13c6d0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.767+0000 7f4ce3460700 1 --2- 172.21.15.32:0/2916197020 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4cdc0af680 0x7f4cdc13e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.767+0000 7f4ce3460700 1 -- 172.21.15.32:0/2916197020 >> 172.21.15.32:0/2916197020 conn(0x7f4cdc0b5580 msgr2=0x7f4cdc05f860 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:47.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.771+0000 7f4ce3460700 1 -- 172.21.15.32:0/2916197020 shutdown_connections 2020-12-05T17:39:47.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.771+0000 7f4ce3460700 1 -- 172.21.15.32:0/2916197020 wait complete. 2020-12-05T17:39:47.787 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:39:47.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.895+0000 7f086e703700 1 Processor -- start 2020-12-05T17:39:47.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.895+0000 7f086e703700 1 -- start start 2020-12-05T17:39:47.902 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.895+0000 7f086e703700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f08680b7720 0x7f0868061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:47.902 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.895+0000 7f086e703700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08681403f0 0x7f0868143e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:47.902 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.895+0000 7f086e703700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0868143a00 con 0x7f0868144320 2020-12-05T17:39:47.903 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.895+0000 7f086e703700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08680b7b00 con 0x7f08680b7720 2020-12-05T17:39:47.903 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.895+0000 7f086e703700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086805fe60 con 0x7f08681403f0 2020-12-05T17:39:47.903 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.895+0000 7f086df02700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0868144320 0x7f086813d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41350/0 (socket says 172.21.15.32:41350) 2020-12-05T17:39:47.903 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.895+0000 7f086df02700 1 -- 172.21.15.32:0/3421480842 learned_addr learned my addr 172.21.15.32:0/3421480842 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:47.904 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.895+0000 7f086d701700 1 --2- 172.21.15.32:0/3421480842 >> v2:172.21.15.61:3300/0 conn(0x7f08680b7720 0x7f0868061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:47.904 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.895+0000 7f086cf00700 1 --2- 172.21.15.32:0/3421480842 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08681403f0 0x7f0868143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:47.904 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.895+0000 7f085ffff700 1 -- 172.21.15.32:0/3421480842 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113048145 0 0) 0x7f0868143a00 con 0x7f0868144320 2020-12-05T17:39:47.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.895+0000 7f085ffff700 1 -- 172.21.15.32:0/3421480842 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f084c003230 con 0x7f0868144320 2020-12-05T17:39:47.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.895+0000 7f086cf00700 1 -- 172.21.15.32:0/3421480842 >> v2:172.21.15.61:3300/0 conn(0x7f08680b7720 msgr2=0x7f0868061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:47.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.895+0000 7f086cf00700 1 --2- 172.21.15.32:0/3421480842 >> v2:172.21.15.61:3300/0 conn(0x7f08680b7720 0x7f0868061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.895+0000 7f086cf00700 1 -- 172.21.15.32:0/3421480842 >> v1:172.21.15.32:6789/0 conn(0x7f0868144320 legacy=0x7f086813d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:47.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.895+0000 7f086cf00700 1 -- 172.21.15.32:0/3421480842 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0868140850 con 0x7f08681403f0 2020-12-05T17:39:47.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.895+0000 7f086cf00700 1 --2- 172.21.15.32:0/3421480842 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08681403f0 0x7f0868143e30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f0858002f30 tx=0x7f0858006dc0).ready entity=mon.2 client_cookie=635ed0363df53e3 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:47.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.895+0000 7f085ffff700 1 -- 172.21.15.32:0/3421480842 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08580096a0 con 0x7f08681403f0 2020-12-05T17:39:47.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.895+0000 7f085ffff700 1 -- 172.21.15.32:0/3421480842 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0858009800 con 0x7f08681403f0 2020-12-05T17:39:47.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.895+0000 7f085ffff700 1 -- 172.21.15.32:0/3421480842 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0858009b60 con 0x7f08681403f0 2020-12-05T17:39:47.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.895+0000 7f086e703700 1 -- 172.21.15.32:0/3421480842 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08681403f0 msgr2=0x7f0868143e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:47.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.895+0000 7f086e703700 1 --2- 172.21.15.32:0/3421480842 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08681403f0 0x7f0868143e30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f0858002f30 tx=0x7f0858006dc0).stop 2020-12-05T17:39:47.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086e703700 1 -- 172.21.15.32:0/3421480842 shutdown_connections 2020-12-05T17:39:47.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086e703700 1 --2- 172.21.15.32:0/3421480842 >> v2:172.21.15.61:3300/0 conn(0x7f08680b7720 0x7f0868061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086e703700 1 --2- 172.21.15.32:0/3421480842 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08681403f0 0x7f0868143e30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086e703700 1 -- 172.21.15.32:0/3421480842 >> 172.21.15.32:0/3421480842 conn(0x7f086805bc20 msgr2=0x7f08680b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:47.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086e703700 1 -- 172.21.15.32:0/3421480842 shutdown_connections 2020-12-05T17:39:47.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086e703700 1 -- 172.21.15.32:0/3421480842 wait complete. 2020-12-05T17:39:47.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086e703700 1 Processor -- start 2020-12-05T17:39:47.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086e703700 1 -- start start 2020-12-05T17:39:47.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086e703700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08680b7720 0x7f0868110b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:47.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086e703700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08681403f0 0x7f0868145bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:47.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086e703700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0868144320 0x7f086813b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:47.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086e703700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08680633d0 con 0x7f08680b7720 2020-12-05T17:39:47.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086e703700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0868063670 con 0x7f0868144320 2020-12-05T17:39:47.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086e703700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0868063b10 con 0x7f08681403f0 2020-12-05T17:39:47.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086d701700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08680b7720 0x7f0868110b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:47.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086d701700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08680b7720 0x7f0868110b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38706/0 (socket says 172.21.15.32:38706) 2020-12-05T17:39:47.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086cf00700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08681403f0 0x7f0868145bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:47.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086d701700 1 -- 172.21.15.32:0/1582110491 learned_addr learned my addr 172.21.15.32:0/1582110491 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:47.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086df02700 1 --2- 172.21.15.32:0/1582110491 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0868144320 0x7f086813b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:47.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086d701700 1 -- 172.21.15.32:0/1582110491 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08681403f0 msgr2=0x7f0868145bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:47.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086d701700 1 --2- 172.21.15.32:0/1582110491 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08681403f0 0x7f0868145bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086d701700 1 -- 172.21.15.32:0/1582110491 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0868144320 msgr2=0x7f086813b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:47.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086d701700 1 --2- 172.21.15.32:0/1582110491 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0868144320 0x7f086813b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:47.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.899+0000 7f086d701700 1 -- 172.21.15.32:0/1582110491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08681473d0 con 0x7f08680b7720 2020-12-05T17:39:47.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.903+0000 7f086d701700 1 --2- 172.21.15.32:0/1582110491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08680b7720 0x7f0868110b90 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7f086400c000 tx=0x7f0864006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:47.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.903+0000 7f085e7fc700 1 -- 172.21.15.32:0/1582110491 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f086400ca10 con 0x7f08680b7720 2020-12-05T17:39:47.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.903+0000 7f086e703700 1 -- 172.21.15.32:0/1582110491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0868063d70 con 0x7f08680b7720 2020-12-05T17:39:47.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.903+0000 7f085e7fc700 1 -- 172.21.15.32:0/1582110491 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086400cb70 con 0x7f08680b7720 2020-12-05T17:39:47.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.903+0000 7f086e703700 1 -- 172.21.15.32:0/1582110491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f086813bbc0 con 0x7f08680b7720 2020-12-05T17:39:47.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.903+0000 7f085e7fc700 1 -- 172.21.15.32:0/1582110491 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f086401c460 con 0x7f08680b7720 2020-12-05T17:39:47.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.903+0000 7f086e703700 1 -- 172.21.15.32:0/1582110491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0868057050 con 0x7f08680b7720 2020-12-05T17:39:47.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.903+0000 7f085e7fc700 1 -- 172.21.15.32:0/1582110491 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f08640123f0 con 0x7f08680b7720 2020-12-05T17:39:47.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.903+0000 7f085e7fc700 1 --2- 172.21.15.32:0/1582110491 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f08500553b0 0x7f0850057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:47.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.903+0000 7f085e7fc700 1 -- 172.21.15.32:0/1582110491 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f086406e580 con 0x7f08680b7720 2020-12-05T17:39:47.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.903+0000 7f086cf00700 1 --2- 172.21.15.32:0/1582110491 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f08500553b0 0x7f0850057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:47.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.903+0000 7f086cf00700 1 --2- 172.21.15.32:0/1582110491 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f08500553b0 0x7f0850057870 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f085801b000 tx=0x7f0858015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:47.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:47.911+0000 7f085e7fc700 1 -- 172.21.15.32:0/1582110491 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0864044620 con 0x7f08680b7720 2020-12-05T17:39:48.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.051+0000 7f086e703700 1 -- 172.21.15.32:0/1582110491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f086805aab0 con 0x7f08680b7720 2020-12-05T17:39:48.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.051+0000 7f085e7fc700 1 -- 172.21.15.32:0/1582110491 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+16295 (secure 0 0 0) 0x7f0864047c40 con 0x7f08680b7720 2020-12-05T17:39:48.058 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:48.058 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":101,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:39:47.470782+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:39:45.190638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2020-12-05T17:39:46.472081+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:39:48.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.055+0000 7f086e703700 1 -- 172.21.15.32:0/1582110491 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f08500553b0 msgr2=0x7f0850057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:48.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.055+0000 7f086e703700 1 --2- 172.21.15.32:0/1582110491 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f08500553b0 0x7f0850057870 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f085801b000 tx=0x7f0858015000).stop 2020-12-05T17:39:48.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.055+0000 7f086e703700 1 -- 172.21.15.32:0/1582110491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08680b7720 msgr2=0x7f0868110b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:48.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.055+0000 7f086e703700 1 --2- 172.21.15.32:0/1582110491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08680b7720 0x7f0868110b90 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7f086400c000 tx=0x7f0864006dc0).stop 2020-12-05T17:39:48.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.055+0000 7f086e703700 1 -- 172.21.15.32:0/1582110491 shutdown_connections 2020-12-05T17:39:48.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.055+0000 7f086e703700 1 --2- 172.21.15.32:0/1582110491 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0868144320 0x7f086813b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.055+0000 7f086e703700 1 --2- 172.21.15.32:0/1582110491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08680b7720 0x7f0868110b90 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.055+0000 7f086e703700 1 --2- 172.21.15.32:0/1582110491 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f08500553b0 0x7f0850057870 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.055+0000 7f086e703700 1 --2- 172.21.15.32:0/1582110491 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08681403f0 0x7f0868145bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.055+0000 7f086e703700 1 -- 172.21.15.32:0/1582110491 >> 172.21.15.32:0/1582110491 conn(0x7f086805bc20 msgr2=0x7f0868141900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:48.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.055+0000 7f086e703700 1 -- 172.21.15.32:0/1582110491 shutdown_connections 2020-12-05T17:39:48.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.055+0000 7f086e703700 1 -- 172.21.15.32:0/1582110491 wait complete. 2020-12-05T17:39:48.073 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-12-05T17:39:48.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.187+0000 7fb31892c700 1 Processor -- start 2020-12-05T17:39:48.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.187+0000 7fb31892c700 1 -- start start 2020-12-05T17:39:48.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.187+0000 7fb31892c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb31413f610 0x7fb31413fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:48.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.187+0000 7fb31892c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3140b0030 con 0x7fb3140af4b0 2020-12-05T17:39:48.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.187+0000 7fb31892c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3140b23e0 con 0x7fb3140b2000 2020-12-05T17:39:48.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.187+0000 7fb31892c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3140aeba0 con 0x7fb31413f610 2020-12-05T17:39:48.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.187+0000 7fb31359e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb3140af4b0 0x7fb3140af920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41364/0 (socket says 172.21.15.32:41364) 2020-12-05T17:39:48.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.187+0000 7fb31359e700 1 -- 172.21.15.32:0/1158685851 learned_addr learned my addr 172.21.15.32:0/1158685851 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.187+0000 7fb31259c700 1 --2- 172.21.15.32:0/1158685851 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb31413f610 0x7fb31413fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.191+0000 7fb311d9b700 1 -- 172.21.15.32:0/1158685851 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507025544 0 0) 0x7fb30c0010d0 con 0x7fb3140af4b0 2020-12-05T17:39:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.191+0000 7fb311d9b700 1 -- 172.21.15.32:0/1158685851 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2f4003230 con 0x7fb3140af4b0 2020-12-05T17:39:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.191+0000 7fb311d9b700 1 -- 172.21.15.32:0/1158685851 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 260816567 0 0) 0x7fb3140b23e0 con 0x7fb3140b2000 2020-12-05T17:39:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.191+0000 7fb311d9b700 1 -- 172.21.15.32:0/1158685851 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb30c0010d0 con 0x7fb3140b2000 2020-12-05T17:39:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.191+0000 7fb31259c700 1 -- 172.21.15.32:0/1158685851 >> v1:172.21.15.61:6789/0 conn(0x7fb3140b2000 legacy=0x7fb314142f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.191+0000 7fb31259c700 1 -- 172.21.15.32:0/1158685851 >> v1:172.21.15.32:6789/0 conn(0x7fb3140af4b0 legacy=0x7fb3140af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.191+0000 7fb31259c700 1 -- 172.21.15.32:0/1158685851 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3140af190 con 0x7fb31413f610 2020-12-05T17:39:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.191+0000 7fb31259c700 1 --2- 172.21.15.32:0/1158685851 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb31413f610 0x7fb31413fa90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fb2fc002f30 tx=0x7fb2fc006dc0).ready entity=mon.2 client_cookie=65869a89b9feb8a1 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.191+0000 7fb311d9b700 1 -- 172.21.15.32:0/1158685851 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2fc009730 con 0x7fb31413f610 2020-12-05T17:39:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.191+0000 7fb311d9b700 1 -- 172.21.15.32:0/1158685851 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2fc009890 con 0x7fb31413f610 2020-12-05T17:39:48.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.191+0000 7fb31892c700 1 -- 172.21.15.32:0/1158685851 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb31413f610 msgr2=0x7fb31413fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:48.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.191+0000 7fb311d9b700 1 -- 172.21.15.32:0/1158685851 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2fc009bf0 con 0x7fb31413f610 2020-12-05T17:39:48.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.191+0000 7fb31892c700 1 --2- 172.21.15.32:0/1158685851 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb31413f610 0x7fb31413fa90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fb2fc002f30 tx=0x7fb2fc006dc0).stop 2020-12-05T17:39:48.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.191+0000 7fb31892c700 1 -- 172.21.15.32:0/1158685851 shutdown_connections 2020-12-05T17:39:48.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.191+0000 7fb31892c700 1 --2- 172.21.15.32:0/1158685851 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb31413f610 0x7fb31413fa90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.191+0000 7fb31892c700 1 -- 172.21.15.32:0/1158685851 >> 172.21.15.32:0/1158685851 conn(0x7fb3140b5620 msgr2=0x7fb3140b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:48.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.191+0000 7fb31892c700 1 -- 172.21.15.32:0/1158685851 shutdown_connections 2020-12-05T17:39:48.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.191+0000 7fb31892c700 1 -- 172.21.15.32:0/1158685851 wait complete. 2020-12-05T17:39:48.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.191+0000 7fb31892c700 1 Processor -- start 2020-12-05T17:39:48.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.191+0000 7fb31892c700 1 -- start start 2020-12-05T17:39:48.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.191+0000 7fb31892c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3140af4b0 0x7fb314062a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.191+0000 7fb31892c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3140b2000 0x7fb3141461d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.195+0000 7fb31892c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb31413f610 0x7fb3141466c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.195+0000 7fb31892c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb314139540 con 0x7fb3140b2000 2020-12-05T17:39:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.195+0000 7fb31892c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb314146bb0 con 0x7fb3140af4b0 2020-12-05T17:39:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.195+0000 7fb31892c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31413c680 con 0x7fb31413f610 2020-12-05T17:39:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.195+0000 7fb31259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3140b2000 0x7fb3141461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.195+0000 7fb31259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3140b2000 0x7fb3141461d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38720/0 (socket says 172.21.15.32:38720) 2020-12-05T17:39:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.195+0000 7fb31259c700 1 -- 172.21.15.32:0/4176452137 learned_addr learned my addr 172.21.15.32:0/4176452137 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.195+0000 7fb312d9d700 1 --2- 172.21.15.32:0/4176452137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3140af4b0 0x7fb314062a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.195+0000 7fb31359e700 1 --2- 172.21.15.32:0/4176452137 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb31413f610 0x7fb3141466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:48.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.195+0000 7fb31259c700 1 -- 172.21.15.32:0/4176452137 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb31413f610 msgr2=0x7fb3141466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:48.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.195+0000 7fb31259c700 1 --2- 172.21.15.32:0/4176452137 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb31413f610 0x7fb3141466c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.195+0000 7fb31259c700 1 -- 172.21.15.32:0/4176452137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3140af4b0 msgr2=0x7fb314062a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:48.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.195+0000 7fb31259c700 1 --2- 172.21.15.32:0/4176452137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3140af4b0 0x7fb314062a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.195+0000 7fb31259c700 1 -- 172.21.15.32:0/4176452137 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb314154da0 con 0x7fb3140b2000 2020-12-05T17:39:48.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.195+0000 7fb31259c700 1 --2- 172.21.15.32:0/4176452137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3140b2000 0x7fb3141461d0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7fb2fc002f30 tx=0x7fb2fc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:48.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.195+0000 7fb30bfff700 1 -- 172.21.15.32:0/4176452137 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2fc0119a0 con 0x7fb3140b2000 2020-12-05T17:39:48.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.195+0000 7fb31892c700 1 -- 172.21.15.32:0/4176452137 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31413fa30 con 0x7fb3140b2000 2020-12-05T17:39:48.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.195+0000 7fb30bfff700 1 -- 172.21.15.32:0/4176452137 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2fc011b00 con 0x7fb3140b2000 2020-12-05T17:39:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.195+0000 7fb30bfff700 1 -- 172.21.15.32:0/4176452137 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2fc011e60 con 0x7fb3140b2000 2020-12-05T17:39:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.195+0000 7fb31892c700 1 -- 172.21.15.32:0/4176452137 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31413bf70 con 0x7fb3140b2000 2020-12-05T17:39:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.195+0000 7fb31892c700 1 -- 172.21.15.32:0/4176452137 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb314057050 con 0x7fb3140b2000 2020-12-05T17:39:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.199+0000 7fb30bfff700 1 -- 172.21.15.32:0/4176452137 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb2fc00f070 con 0x7fb3140b2000 2020-12-05T17:39:48.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.199+0000 7fb30bfff700 1 --2- 172.21.15.32:0/4176452137 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb2f8055470 0x7fb2f8057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:48.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.199+0000 7fb30bfff700 1 -- 172.21.15.32:0/4176452137 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb2fc06f290 con 0x7fb3140b2000 2020-12-05T17:39:48.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.199+0000 7fb312d9d700 1 --2- 172.21.15.32:0/4176452137 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb2f8055470 0x7fb2f8057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:48.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.199+0000 7fb312d9d700 1 --2- 172.21.15.32:0/4176452137 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb2f8055470 0x7fb2f8057930 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7fb30400d000 tx=0x7fb304007e20).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:48.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.203+0000 7fb30bfff700 1 -- 172.21.15.32:0/4176452137 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb2fc045430 con 0x7fb3140b2000 2020-12-05T17:39:48.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.391+0000 7fb31892c700 1 -- 172.21.15.32:0/4176452137 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb31413fd20 con 0x7fb3140b2000 2020-12-05T17:39:48.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.475+0000 7fb30bfff700 1 -- 172.21.15.32:0/4176452137 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v47) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb2fc022070 con 0x7fb3140b2000 2020-12-05T17:39:48.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.479+0000 7fb31892c700 1 -- 172.21.15.32:0/4176452137 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb2f8055470 msgr2=0x7fb2f8057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:48.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.479+0000 7fb31892c700 1 --2- 172.21.15.32:0/4176452137 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb2f8055470 0x7fb2f8057930 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7fb30400d000 tx=0x7fb304007e20).stop 2020-12-05T17:39:48.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.479+0000 7fb31892c700 1 -- 172.21.15.32:0/4176452137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3140b2000 msgr2=0x7fb3141461d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:48.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.479+0000 7fb31892c700 1 --2- 172.21.15.32:0/4176452137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3140b2000 0x7fb3141461d0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7fb2fc002f30 tx=0x7fb2fc015000).stop 2020-12-05T17:39:48.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.479+0000 7fb31892c700 1 -- 172.21.15.32:0/4176452137 shutdown_connections 2020-12-05T17:39:48.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.479+0000 7fb31892c700 1 --2- 172.21.15.32:0/4176452137 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb2f8055470 0x7fb2f8057930 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.479+0000 7fb31892c700 1 --2- 172.21.15.32:0/4176452137 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb31413f610 0x7fb3141466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.479+0000 7fb31892c700 1 --2- 172.21.15.32:0/4176452137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3140af4b0 0x7fb314062a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.479+0000 7fb31892c700 1 --2- 172.21.15.32:0/4176452137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3140b2000 0x7fb3141461d0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.479+0000 7fb31892c700 1 -- 172.21.15.32:0/4176452137 >> 172.21.15.32:0/4176452137 conn(0x7fb3140b5620 msgr2=0x7fb3140b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:48.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.479+0000 7fb31892c700 1 -- 172.21.15.32:0/4176452137 shutdown_connections 2020-12-05T17:39:48.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.479+0000 7fb31892c700 1 -- 172.21.15.32:0/4176452137 wait complete. 2020-12-05T17:39:48.504 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:39:48.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe4d05d700 1 Processor -- start 2020-12-05T17:39:48.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe4d05d700 1 -- start start 2020-12-05T17:39:48.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe4d05d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe480afe50 0x7fbe48146a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:48.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe4d05d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fbe480b1880 0x7fbe481430f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:48.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe4d05d700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe480b0230 con 0x7fbe480b3aa0 2020-12-05T17:39:48.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe4d05d700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe480b6a50 con 0x7fbe480b1880 2020-12-05T17:39:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe4d05d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe48146080 con 0x7fbe480afe50 2020-12-05T17:39:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe46d9d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe480afe50 0x7fbe48146a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe4759e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fbe480b1880 0x7fbe481430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe4659c700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fbe480b3aa0 0x7fbe481429e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:54016/0 (socket says 172.21.15.32:54016) 2020-12-05T17:39:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe4659c700 1 -- 172.21.15.32:0/1354439025 learned_addr learned my addr 172.21.15.32:0/1354439025 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe45d9b700 1 -- 172.21.15.32:0/1354439025 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306569110 0 0) 0x7fbe480b0230 con 0x7fbe480b3aa0 2020-12-05T17:39:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe45d9b700 1 -- 172.21.15.32:0/1354439025 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe28003230 con 0x7fbe480b3aa0 2020-12-05T17:39:48.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe46d9d700 1 -- 172.21.15.32:0/1354439025 >> v2:172.21.15.61:3300/0 conn(0x7fbe480b1880 msgr2=0x7fbe481430f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:48.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe46d9d700 1 --2- 172.21.15.32:0/1354439025 >> v2:172.21.15.61:3300/0 conn(0x7fbe480b1880 0x7fbe481430f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe46d9d700 1 -- 172.21.15.32:0/1354439025 >> v1:172.21.15.61:6789/0 conn(0x7fbe480b3aa0 legacy=0x7fbe481429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:48.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe46d9d700 1 -- 172.21.15.32:0/1354439025 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe48146320 con 0x7fbe480afe50 2020-12-05T17:39:48.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe4759e700 1 --2- 172.21.15.32:0/1354439025 >> v2:172.21.15.61:3300/0 conn(0x7fbe480b1880 0x7fbe481430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:48.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe46d9d700 1 --2- 172.21.15.32:0/1354439025 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe480afe50 0x7fbe48146a60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fbe38002f30 tx=0x7fbe38006dc0).ready entity=mon.2 client_cookie=59f6a933b9e3e7e9 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:48.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe45d9b700 1 -- 172.21.15.32:0/1354439025 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe380096a0 con 0x7fbe480afe50 2020-12-05T17:39:48.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe45d9b700 1 -- 172.21.15.32:0/1354439025 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe38009800 con 0x7fbe480afe50 2020-12-05T17:39:48.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe45d9b700 1 -- 172.21.15.32:0/1354439025 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe38009b60 con 0x7fbe480afe50 2020-12-05T17:39:48.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe4d05d700 1 -- 172.21.15.32:0/1354439025 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe480afe50 msgr2=0x7fbe48146a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:48.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.631+0000 7fbe4d05d700 1 --2- 172.21.15.32:0/1354439025 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe480afe50 0x7fbe48146a60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fbe38002f30 tx=0x7fbe38006dc0).stop 2020-12-05T17:39:48.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.635+0000 7fbe4d05d700 1 -- 172.21.15.32:0/1354439025 shutdown_connections 2020-12-05T17:39:48.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.635+0000 7fbe4d05d700 1 --2- 172.21.15.32:0/1354439025 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe480afe50 0x7fbe48146a60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.635+0000 7fbe4d05d700 1 --2- 172.21.15.32:0/1354439025 >> v2:172.21.15.61:3300/0 conn(0x7fbe480b1880 0x7fbe481430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.635+0000 7fbe4d05d700 1 -- 172.21.15.32:0/1354439025 >> 172.21.15.32:0/1354439025 conn(0x7fbe480b9100 msgr2=0x7fbe480b9510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:48.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.635+0000 7fbe4d05d700 1 -- 172.21.15.32:0/1354439025 shutdown_connections 2020-12-05T17:39:48.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.635+0000 7fbe4d05d700 1 -- 172.21.15.32:0/1354439025 wait complete. 2020-12-05T17:39:48.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.635+0000 7fbe4d05d700 1 Processor -- start 2020-12-05T17:39:48.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.635+0000 7fbe4d05d700 1 -- start start 2020-12-05T17:39:48.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.635+0000 7fbe4d05d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe480afe50 0x7fbe48142b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:48.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.635+0000 7fbe4d05d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbe480b1880 0x7fbe481418f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:48.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.635+0000 7fbe4d05d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe480b3aa0 0x7fbe481551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:48.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.635+0000 7fbe4d05d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe48062a00 con 0x7fbe480afe50 2020-12-05T17:39:48.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.635+0000 7fbe4d05d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe48062c30 con 0x7fbe480b1880 2020-12-05T17:39:48.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.635+0000 7fbe4d05d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe48062ed0 con 0x7fbe480b3aa0 2020-12-05T17:39:48.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.635+0000 7fbe46d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe480afe50 0x7fbe48142b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:48.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.635+0000 7fbe46d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe480afe50 0x7fbe48142b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38732/0 (socket says 172.21.15.32:38732) 2020-12-05T17:39:48.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.635+0000 7fbe46d9d700 1 -- 172.21.15.32:0/2888407889 learned_addr learned my addr 172.21.15.32:0/2888407889 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:48.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.635+0000 7fbe4659c700 1 --2- 172.21.15.32:0/2888407889 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbe480b1880 0x7fbe481418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:48.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.635+0000 7fbe4759e700 1 --2- 172.21.15.32:0/2888407889 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe480b3aa0 0x7fbe481551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:48.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.639+0000 7fbe46d9d700 1 -- 172.21.15.32:0/2888407889 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe480b3aa0 msgr2=0x7fbe481551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:48.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.639+0000 7fbe46d9d700 1 --2- 172.21.15.32:0/2888407889 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe480b3aa0 0x7fbe481551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.639+0000 7fbe46d9d700 1 -- 172.21.15.32:0/2888407889 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbe480b1880 msgr2=0x7fbe481418f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:48.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.639+0000 7fbe46d9d700 1 --2- 172.21.15.32:0/2888407889 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbe480b1880 0x7fbe481418f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.639+0000 7fbe46d9d700 1 -- 172.21.15.32:0/2888407889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe48063170 con 0x7fbe480afe50 2020-12-05T17:39:48.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.639+0000 7fbe4659c700 1 --2- 172.21.15.32:0/2888407889 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbe480b1880 0x7fbe481418f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:48.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.639+0000 7fbe46d9d700 1 --2- 172.21.15.32:0/2888407889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe480afe50 0x7fbe48142b20 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7fbe38002f30 tx=0x7fbe38015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:48.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.639+0000 7fbe3ffff700 1 -- 172.21.15.32:0/2888407889 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe38011bb0 con 0x7fbe480afe50 2020-12-05T17:39:48.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.639+0000 7fbe3ffff700 1 -- 172.21.15.32:0/2888407889 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe38011d10 con 0x7fbe480afe50 2020-12-05T17:39:48.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.639+0000 7fbe4d05d700 1 -- 172.21.15.32:0/2888407889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe480663c0 con 0x7fbe480afe50 2020-12-05T17:39:48.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.639+0000 7fbe3ffff700 1 -- 172.21.15.32:0/2888407889 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe3801d640 con 0x7fbe480afe50 2020-12-05T17:39:48.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.639+0000 7fbe4d05d700 1 -- 172.21.15.32:0/2888407889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe481556b0 con 0x7fbe480afe50 2020-12-05T17:39:48.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.639+0000 7fbe4d05d700 1 -- 172.21.15.32:0/2888407889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe48063300 con 0x7fbe480afe50 2020-12-05T17:39:48.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.639+0000 7fbe3ffff700 1 -- 172.21.15.32:0/2888407889 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbe38024030 con 0x7fbe480afe50 2020-12-05T17:39:48.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.639+0000 7fbe3ffff700 1 --2- 172.21.15.32:0/2888407889 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbe2c055350 0x7fbe2c057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:48.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.639+0000 7fbe3ffff700 1 -- 172.21.15.32:0/2888407889 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbe3806efd0 con 0x7fbe480afe50 2020-12-05T17:39:48.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.639+0000 7fbe4659c700 1 --2- 172.21.15.32:0/2888407889 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbe2c055350 0x7fbe2c057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:48.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.639+0000 7fbe4659c700 1 --2- 172.21.15.32:0/2888407889 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbe2c055350 0x7fbe2c057810 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fbe3000d000 tx=0x7fbe30009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:48.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.647+0000 7fbe3ffff700 1 -- 172.21.15.32:0/2888407889 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbe38045170 con 0x7fbe480afe50 2020-12-05T17:39:48.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.831+0000 7fbe4d05d700 1 -- 172.21.15.32:0/2888407889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe48057050 con 0x7fbe480afe50 2020-12-05T17:39:48.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.831+0000 7fbe3ffff700 1 -- 172.21.15.32:0/2888407889 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+2140 (secure 0 0 0) 0x7fbe38022070 con 0x7fbe480afe50 2020-12-05T17:39:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:48.837 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":47,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/3759190037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3759190037},{"type":"v1","addr":"172.21.15.61:6833","nonce":3759190037}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":45}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:39:47.471195+0000","modified":"2020-12-05T17:39:48.473895+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7233":{"gid":7233,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4174090156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4174090156},{"type":"v1","addr":"172.21.15.32:6835","nonce":4174090156}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2020-12-05T17:39:48.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.831+0000 7fbe4d05d700 1 -- 172.21.15.32:0/2888407889 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbe2c055350 msgr2=0x7fbe2c057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:48.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.831+0000 7fbe4d05d700 1 --2- 172.21.15.32:0/2888407889 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbe2c055350 0x7fbe2c057810 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fbe3000d000 tx=0x7fbe30009040).stop 2020-12-05T17:39:48.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.831+0000 7fbe4d05d700 1 -- 172.21.15.32:0/2888407889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe480afe50 msgr2=0x7fbe48142b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:48.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.831+0000 7fbe4d05d700 1 --2- 172.21.15.32:0/2888407889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe480afe50 0x7fbe48142b20 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7fbe38002f30 tx=0x7fbe38015000).stop 2020-12-05T17:39:48.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.835+0000 7fbe4d05d700 1 -- 172.21.15.32:0/2888407889 shutdown_connections 2020-12-05T17:39:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.835+0000 7fbe4d05d700 1 --2- 172.21.15.32:0/2888407889 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbe2c055350 0x7fbe2c057810 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.835+0000 7fbe4d05d700 1 --2- 172.21.15.32:0/2888407889 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe480b3aa0 0x7fbe481551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.835+0000 7fbe4d05d700 1 --2- 172.21.15.32:0/2888407889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe480afe50 0x7fbe48142b20 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.835+0000 7fbe4d05d700 1 --2- 172.21.15.32:0/2888407889 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbe480b1880 0x7fbe481418f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.835+0000 7fbe4d05d700 1 -- 172.21.15.32:0/2888407889 >> 172.21.15.32:0/2888407889 conn(0x7fbe480b9100 msgr2=0x7fbe4813f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.835+0000 7fbe4d05d700 1 -- 172.21.15.32:0/2888407889 shutdown_connections 2020-12-05T17:39:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.835+0000 7fbe4d05d700 1 -- 172.21.15.32:0/2888407889 wait complete. 2020-12-05T17:39:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 47 2020-12-05T17:39:48.852 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:39:48.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.959+0000 7fd95a323700 1 Processor -- start 2020-12-05T17:39:48.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.959+0000 7fd95a323700 1 -- start start 2020-12-05T17:39:48.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.959+0000 7fd95a323700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd954062400 0x7fd95405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:48.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.959+0000 7fd95a323700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd95405feb0 0x7fd9540b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:48.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.959+0000 7fd95a323700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd954142450 con 0x7fd95405f3b0 2020-12-05T17:39:48.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.959+0000 7fd95a323700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9540b42f0 con 0x7fd95405feb0 2020-12-05T17:39:48.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.959+0000 7fd95a323700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd954061780 con 0x7fd954062400 2020-12-05T17:39:48.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.959+0000 7fd959321700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd95405f3b0 0x7fd9540b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41388/0 (socket says 172.21.15.32:41388) 2020-12-05T17:39:48.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.959+0000 7fd959321700 1 -- 172.21.15.32:0/3306503279 learned_addr learned my addr 172.21.15.32:0/3306503279 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:48.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.959+0000 7fd958b20700 1 --2- 172.21.15.32:0/3306503279 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd954062400 0x7fd95405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:48.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.959+0000 7fd959b22700 1 --2- 172.21.15.32:0/3306503279 >> v2:172.21.15.61:3300/0 conn(0x7fd95405feb0 0x7fd9540b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:48.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.959+0000 7fd94bfff700 1 -- 172.21.15.32:0/3306503279 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890479639 0 0) 0x7fd954142450 con 0x7fd95405f3b0 2020-12-05T17:39:48.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.959+0000 7fd94bfff700 1 -- 172.21.15.32:0/3306503279 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd938003230 con 0x7fd95405f3b0 2020-12-05T17:39:48.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.959+0000 7fd959b22700 1 -- 172.21.15.32:0/3306503279 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd954062400 msgr2=0x7fd95405f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:48.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.959+0000 7fd959b22700 1 --2- 172.21.15.32:0/3306503279 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd954062400 0x7fd95405f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.959+0000 7fd959b22700 1 -- 172.21.15.32:0/3306503279 >> v1:172.21.15.32:6789/0 conn(0x7fd95405f3b0 legacy=0x7fd9540b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:48.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.959+0000 7fd959b22700 1 -- 172.21.15.32:0/3306503279 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd954061a20 con 0x7fd95405feb0 2020-12-05T17:39:48.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.959+0000 7fd959b22700 1 --2- 172.21.15.32:0/3306503279 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd95405feb0 0x7fd9540b3e00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7fd94c0032b0 tx=0x7fd94c0091a0).ready entity=mon.1 client_cookie=b5a7779db4a13532 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:48.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.963+0000 7fd94bfff700 1 -- 172.21.15.32:0/3306503279 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd94c00b940 con 0x7fd95405feb0 2020-12-05T17:39:48.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.963+0000 7fd94bfff700 1 -- 172.21.15.32:0/3306503279 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd94c00baa0 con 0x7fd95405feb0 2020-12-05T17:39:48.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.963+0000 7fd94bfff700 1 -- 172.21.15.32:0/3306503279 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd94c00bc10 con 0x7fd95405feb0 2020-12-05T17:39:48.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.963+0000 7fd95a323700 1 -- 172.21.15.32:0/3306503279 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd95405feb0 msgr2=0x7fd9540b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:48.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.963+0000 7fd95a323700 1 --2- 172.21.15.32:0/3306503279 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd95405feb0 0x7fd9540b3e00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7fd94c0032b0 tx=0x7fd94c0091a0).stop 2020-12-05T17:39:48.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.963+0000 7fd95a323700 1 -- 172.21.15.32:0/3306503279 shutdown_connections 2020-12-05T17:39:48.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.963+0000 7fd95a323700 1 --2- 172.21.15.32:0/3306503279 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd95405feb0 0x7fd9540b3e00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.963+0000 7fd95a323700 1 --2- 172.21.15.32:0/3306503279 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd954062400 0x7fd95405f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.963+0000 7fd95a323700 1 -- 172.21.15.32:0/3306503279 >> 172.21.15.32:0/3306503279 conn(0x7fd9540b8ee0 msgr2=0x7fd9540b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:48.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.963+0000 7fd95a323700 1 -- 172.21.15.32:0/3306503279 shutdown_connections 2020-12-05T17:39:48.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.963+0000 7fd95a323700 1 -- 172.21.15.32:0/3306503279 wait complete. 2020-12-05T17:39:48.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.963+0000 7fd95a323700 1 Processor -- start 2020-12-05T17:39:48.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.963+0000 7fd95a323700 1 -- start start 2020-12-05T17:39:48.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.963+0000 7fd95a323700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd95405f3b0 0x7fd9540b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:48.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.963+0000 7fd95a323700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd95405feb0 0x7fd954065640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:48.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.963+0000 7fd95a323700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd954062400 0x7fd9541518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:48.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.963+0000 7fd95a323700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9540642c0 con 0x7fd95405feb0 2020-12-05T17:39:48.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.963+0000 7fd95a323700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd954065b30 con 0x7fd954062400 2020-12-05T17:39:48.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.963+0000 7fd95a323700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9540644b0 con 0x7fd95405f3b0 2020-12-05T17:39:48.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.967+0000 7fd958b20700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd95405feb0 0x7fd954065640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:48.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.967+0000 7fd958b20700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd95405feb0 0x7fd954065640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38748/0 (socket says 172.21.15.32:38748) 2020-12-05T17:39:48.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.967+0000 7fd958b20700 1 -- 172.21.15.32:0/397222429 learned_addr learned my addr 172.21.15.32:0/397222429 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:48.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.967+0000 7fd959b22700 1 --2- 172.21.15.32:0/397222429 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd954062400 0x7fd9541518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:48.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.967+0000 7fd959321700 1 --2- 172.21.15.32:0/397222429 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd95405f3b0 0x7fd9540b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:48.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.967+0000 7fd958b20700 1 -- 172.21.15.32:0/397222429 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd95405f3b0 msgr2=0x7fd9540b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:48.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.967+0000 7fd958b20700 1 --2- 172.21.15.32:0/397222429 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd95405f3b0 0x7fd9540b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.967+0000 7fd958b20700 1 -- 172.21.15.32:0/397222429 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd954062400 msgr2=0x7fd9541518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:48.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.967+0000 7fd958b20700 1 --2- 172.21.15.32:0/397222429 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd954062400 0x7fd9541518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:48.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.967+0000 7fd958b20700 1 -- 172.21.15.32:0/397222429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd954064750 con 0x7fd95405feb0 2020-12-05T17:39:48.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.967+0000 7fd959321700 1 --2- 172.21.15.32:0/397222429 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd95405f3b0 0x7fd9540b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:39:48.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.967+0000 7fd958b20700 1 --2- 172.21.15.32:0/397222429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd95405feb0 0x7fd954065640 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 rx=0x7fd95000c000 tx=0x7fd950006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:48.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.967+0000 7fd94a7fc700 1 -- 172.21.15.32:0/397222429 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd95000ca10 con 0x7fd95405feb0 2020-12-05T17:39:48.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.967+0000 7fd94a7fc700 1 -- 172.21.15.32:0/397222429 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95000cb70 con 0x7fd95405feb0 2020-12-05T17:39:48.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.967+0000 7fd95a323700 1 -- 172.21.15.32:0/397222429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd954151460 con 0x7fd95405feb0 2020-12-05T17:39:48.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.967+0000 7fd94a7fc700 1 -- 172.21.15.32:0/397222429 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9500123f0 con 0x7fd95405feb0 2020-12-05T17:39:48.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.967+0000 7fd95a323700 1 -- 172.21.15.32:0/397222429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd954064d70 con 0x7fd95405feb0 2020-12-05T17:39:48.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.967+0000 7fd95a323700 1 -- 172.21.15.32:0/397222429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9540648e0 con 0x7fd95405feb0 2020-12-05T17:39:48.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.967+0000 7fd94a7fc700 1 -- 172.21.15.32:0/397222429 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd950012550 con 0x7fd95405feb0 2020-12-05T17:39:48.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.971+0000 7fd94a7fc700 1 --2- 172.21.15.32:0/397222429 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd93c0553b0 0x7fd93c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:48.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.971+0000 7fd94a7fc700 1 -- 172.21.15.32:0/397222429 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd95006e660 con 0x7fd95405feb0 2020-12-05T17:39:48.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.971+0000 7fd959321700 1 --2- 172.21.15.32:0/397222429 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd93c0553b0 0x7fd93c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:48.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.971+0000 7fd959321700 1 --2- 172.21.15.32:0/397222429 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd93c0553b0 0x7fd93c057870 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7fd944008000 tx=0x7fd944009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:48.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:48.975+0000 7fd94a7fc700 1 -- 172.21.15.32:0/397222429 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd950044690 con 0x7fd95405feb0 2020-12-05T17:39:49.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.119+0000 7fd95a323700 1 -- 172.21.15.32:0/397222429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd954057050 con 0x7fd95405feb0 2020-12-05T17:39:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.119+0000 7fd94a7fc700 1 -- 172.21.15.32:0/397222429 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16282 (secure 0 0 0) 0x7fd950047cb0 con 0x7fd95405feb0 2020-12-05T17:39:49.127 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:49.127 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":102,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:39:48.474575+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:39:45.190638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2020-12-05T17:39:46.472081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:39:49.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.123+0000 7fd95a323700 1 -- 172.21.15.32:0/397222429 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd93c0553b0 msgr2=0x7fd93c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:49.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.123+0000 7fd95a323700 1 --2- 172.21.15.32:0/397222429 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd93c0553b0 0x7fd93c057870 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7fd944008000 tx=0x7fd944009040).stop 2020-12-05T17:39:49.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.123+0000 7fd95a323700 1 -- 172.21.15.32:0/397222429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd95405feb0 msgr2=0x7fd954065640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:49.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.123+0000 7fd95a323700 1 --2- 172.21.15.32:0/397222429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd95405feb0 0x7fd954065640 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 rx=0x7fd95000c000 tx=0x7fd950006dc0).stop 2020-12-05T17:39:49.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.123+0000 7fd95a323700 1 -- 172.21.15.32:0/397222429 shutdown_connections 2020-12-05T17:39:49.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.123+0000 7fd95a323700 1 --2- 172.21.15.32:0/397222429 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd954062400 0x7fd9541518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.123+0000 7fd95a323700 1 --2- 172.21.15.32:0/397222429 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd93c0553b0 0x7fd93c057870 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.123+0000 7fd95a323700 1 --2- 172.21.15.32:0/397222429 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd95405f3b0 0x7fd9540b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.123+0000 7fd95a323700 1 --2- 172.21.15.32:0/397222429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd95405feb0 0x7fd954065640 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.123+0000 7fd95a323700 1 -- 172.21.15.32:0/397222429 >> 172.21.15.32:0/397222429 conn(0x7fd9540b8ee0 msgr2=0x7fd9540b92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:49.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.123+0000 7fd95a323700 1 -- 172.21.15.32:0/397222429 shutdown_connections 2020-12-05T17:39:49.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.123+0000 7fd95a323700 1 -- 172.21.15.32:0/397222429 wait complete. 2020-12-05T17:39:49.148 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:39:49.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.267+0000 7f6db90ed700 1 Processor -- start 2020-12-05T17:39:49.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.267+0000 7f6db90ed700 1 -- start start 2020-12-05T17:39:49.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.267+0000 7f6db90ed700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6db40b0240 0x7f6db4153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:49.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.267+0000 7f6db90ed700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db4080a90 con 0x7f6db40ae860 2020-12-05T17:39:49.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.267+0000 7f6db90ed700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db40aec40 con 0x7f6db40611b0 2020-12-05T17:39:49.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.267+0000 7f6db90ed700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db4152860 con 0x7f6db40b0240 2020-12-05T17:39:49.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.267+0000 7f6db3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6db40b0240 0x7f6db4153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:49.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.267+0000 7f6dba34f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6db40ae860 0x7f6db414f0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41406/0 (socket says 172.21.15.32:41406) 2020-12-05T17:39:49.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.267+0000 7f6dba34f700 1 -- 172.21.15.32:0/2605685472 learned_addr learned my addr 172.21.15.32:0/2605685472 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:49.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.267+0000 7f6db3fff700 1 -- 172.21.15.32:0/2605685472 >> v1:172.21.15.61:6789/0 conn(0x7f6db40611b0 legacy=0x7f6db414c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:49.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.267+0000 7f6db3fff700 1 -- 172.21.15.32:0/2605685472 >> v1:172.21.15.32:6789/0 conn(0x7f6db40ae860 legacy=0x7f6db414f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.267+0000 7f6db37fe700 1 -- 172.21.15.32:0/2605685472 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 68382455 0 0) 0x7f6db4080a90 con 0x7f6db40ae860 2020-12-05T17:39:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.267+0000 7f6db3fff700 1 -- 172.21.15.32:0/2605685472 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db4152b00 con 0x7f6db40b0240 2020-12-05T17:39:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.267+0000 7f6db3fff700 1 --2- 172.21.15.32:0/2605685472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db40b0240 0x7f6db4153240 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7f6d9c002f30 tx=0x7f6d9c006dc0).ready entity=mon.0 client_cookie=2c70223039e922c1 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.267+0000 7f6db37fe700 1 -- 172.21.15.32:0/2605685472 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d9c0097f0 con 0x7f6db40b0240 2020-12-05T17:39:49.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.267+0000 7f6db37fe700 1 -- 172.21.15.32:0/2605685472 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9c009950 con 0x7f6db40b0240 2020-12-05T17:39:49.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.267+0000 7f6db37fe700 1 -- 172.21.15.32:0/2605685472 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d9c009cb0 con 0x7f6db40b0240 2020-12-05T17:39:49.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.267+0000 7f6db90ed700 1 -- 172.21.15.32:0/2605685472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db40b0240 msgr2=0x7f6db4153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:49.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.267+0000 7f6db90ed700 1 --2- 172.21.15.32:0/2605685472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db40b0240 0x7f6db4153240 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7f6d9c002f30 tx=0x7f6d9c006dc0).stop 2020-12-05T17:39:49.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.271+0000 7f6db90ed700 1 -- 172.21.15.32:0/2605685472 shutdown_connections 2020-12-05T17:39:49.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.271+0000 7f6db90ed700 1 --2- 172.21.15.32:0/2605685472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db40b0240 0x7f6db4153240 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.271+0000 7f6db90ed700 1 -- 172.21.15.32:0/2605685472 >> 172.21.15.32:0/2605685472 conn(0x7f6db40b5f90 msgr2=0x7f6db40b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:49.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.271+0000 7f6db90ed700 1 -- 172.21.15.32:0/2605685472 shutdown_connections 2020-12-05T17:39:49.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.271+0000 7f6db90ed700 1 -- 172.21.15.32:0/2605685472 wait complete. 2020-12-05T17:39:49.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.271+0000 7f6db90ed700 1 Processor -- start 2020-12-05T17:39:49.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.271+0000 7f6db90ed700 1 -- start start 2020-12-05T17:39:49.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.271+0000 7f6db90ed700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6db40611b0 0x7f6db414ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:49.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.271+0000 7f6db90ed700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db40ae860 0x7f6db413ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:49.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.271+0000 7f6db90ed700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6db40b0240 0x7f6db40b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:49.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.271+0000 7f6db90ed700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db413e700 con 0x7f6db40ae860 2020-12-05T17:39:49.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.271+0000 7f6db90ed700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db413e9a0 con 0x7f6db40611b0 2020-12-05T17:39:49.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.271+0000 7f6db90ed700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db413f330 con 0x7f6db40b0240 2020-12-05T17:39:49.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.275+0000 7f6db3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db40ae860 0x7f6db413ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:49.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.275+0000 7f6db3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db40ae860 0x7f6db413ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38762/0 (socket says 172.21.15.32:38762) 2020-12-05T17:39:49.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.275+0000 7f6db3fff700 1 -- 172.21.15.32:0/66577837 learned_addr learned my addr 172.21.15.32:0/66577837 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:49.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.275+0000 7f6db88ec700 1 --2- 172.21.15.32:0/66577837 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6db40611b0 0x7f6db414ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:49.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.275+0000 7f6dba34f700 1 --2- 172.21.15.32:0/66577837 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6db40b0240 0x7f6db40b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:49.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.275+0000 7f6db3fff700 1 -- 172.21.15.32:0/66577837 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6db40b0240 msgr2=0x7f6db40b2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:49.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.275+0000 7f6db3fff700 1 --2- 172.21.15.32:0/66577837 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6db40b0240 0x7f6db40b2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.275+0000 7f6db3fff700 1 -- 172.21.15.32:0/66577837 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6db40611b0 msgr2=0x7f6db414ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:49.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.275+0000 7f6db3fff700 1 --2- 172.21.15.32:0/66577837 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6db40611b0 0x7f6db414ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.275+0000 7f6db3fff700 1 -- 172.21.15.32:0/66577837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db4066d00 con 0x7f6db40ae860 2020-12-05T17:39:49.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.275+0000 7f6dba34f700 1 --2- 172.21.15.32:0/66577837 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6db40b0240 0x7f6db40b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:39:49.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.275+0000 7f6db3fff700 1 --2- 172.21.15.32:0/66577837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db40ae860 0x7f6db413ee40 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7f6d9c002f30 tx=0x7f6d9c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:49.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.275+0000 7f6db17fa700 1 -- 172.21.15.32:0/66577837 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d9c021070 con 0x7f6db40ae860 2020-12-05T17:39:49.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.275+0000 7f6db17fa700 1 -- 172.21.15.32:0/66577837 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9c01d450 con 0x7f6db40ae860 2020-12-05T17:39:49.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.275+0000 7f6db90ed700 1 -- 172.21.15.32:0/66577837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db4065d40 con 0x7f6db40ae860 2020-12-05T17:39:49.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.275+0000 7f6db17fa700 1 -- 172.21.15.32:0/66577837 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d9c01d7b0 con 0x7f6db40ae860 2020-12-05T17:39:49.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.275+0000 7f6db90ed700 1 -- 172.21.15.32:0/66577837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db40b31d0 con 0x7f6db40ae860 2020-12-05T17:39:49.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.275+0000 7f6db90ed700 1 -- 172.21.15.32:0/66577837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db4066ef0 con 0x7f6db40ae860 2020-12-05T17:39:49.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.275+0000 7f6db17fa700 1 -- 172.21.15.32:0/66577837 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6d9c00f070 con 0x7f6db40ae860 2020-12-05T17:39:49.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.275+0000 7f6db17fa700 1 --2- 172.21.15.32:0/66577837 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6d980553b0 0x7f6d98057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:49.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.275+0000 7f6db88ec700 1 --2- 172.21.15.32:0/66577837 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6d980553b0 0x7f6d98057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:49.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.275+0000 7f6db17fa700 1 -- 172.21.15.32:0/66577837 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6d9c06f3c0 con 0x7f6db40ae860 2020-12-05T17:39:49.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.279+0000 7f6db88ec700 1 --2- 172.21.15.32:0/66577837 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6d980553b0 0x7f6d98057870 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f6da400d000 tx=0x7f6da4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:49.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.287+0000 7f6db17fa700 1 -- 172.21.15.32:0/66577837 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6d9c045560 con 0x7f6db40ae860 2020-12-05T17:39:49.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.475+0000 7f6db90ed700 1 -- 172.21.15.32:0/66577837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f6db4130b00 con 0x7f6db40ae860 2020-12-05T17:39:49.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.483+0000 7f6db17fa700 1 -- 172.21.15.32:0/66577837 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v32) v1 ==== 184+0+0 (secure 0 0 0) 0x7f6db4130b00 con 0x7f6db40ae860 2020-12-05T17:39:49.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.487+0000 7f6db90ed700 1 -- 172.21.15.32:0/66577837 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6d980553b0 msgr2=0x7f6d98057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:49.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.487+0000 7f6db90ed700 1 --2- 172.21.15.32:0/66577837 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6d980553b0 0x7f6d98057870 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f6da400d000 tx=0x7f6da4009040).stop 2020-12-05T17:39:49.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.487+0000 7f6db90ed700 1 -- 172.21.15.32:0/66577837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db40ae860 msgr2=0x7f6db413ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:49.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.487+0000 7f6db90ed700 1 --2- 172.21.15.32:0/66577837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db40ae860 0x7f6db413ee40 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7f6d9c002f30 tx=0x7f6d9c015000).stop 2020-12-05T17:39:49.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.487+0000 7f6db90ed700 1 -- 172.21.15.32:0/66577837 shutdown_connections 2020-12-05T17:39:49.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.487+0000 7f6db90ed700 1 --2- 172.21.15.32:0/66577837 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6d980553b0 0x7f6d98057870 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.487+0000 7f6db90ed700 1 --2- 172.21.15.32:0/66577837 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6db40b0240 0x7f6db40b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.487+0000 7f6db90ed700 1 --2- 172.21.15.32:0/66577837 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6db40611b0 0x7f6db414ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.487+0000 7f6db90ed700 1 --2- 172.21.15.32:0/66577837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db40ae860 0x7f6db413ee40 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.487+0000 7f6db90ed700 1 -- 172.21.15.32:0/66577837 >> 172.21.15.32:0/66577837 conn(0x7f6db40b5f90 msgr2=0x7f6db41505e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:49.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.487+0000 7f6db90ed700 1 -- 172.21.15.32:0/66577837 shutdown_connections 2020-12-05T17:39:49.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.487+0000 7f6db90ed700 1 -- 172.21.15.32:0/66577837 wait complete. 2020-12-05T17:39:49.496 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2020-12-05T17:39:49.506 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:39:49.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.619+0000 7fddd9bf5700 1 Processor -- start 2020-12-05T17:39:49.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.619+0000 7fddd9bf5700 1 -- start start 2020-12-05T17:39:49.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fddd9bf5700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fddd40611b0 0x7fddd4153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:49.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fddd9bf5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fddd40b0240 0x7fddd40ae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:49.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fddd9bf5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fddd414c340 0x7fddd414c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:49.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fddd9bf5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd4080a90 con 0x7fddd414c340 2020-12-05T17:39:49.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fddd9bf5700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd4152860 con 0x7fddd40611b0 2020-12-05T17:39:49.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fddd9bf5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd4152b00 con 0x7fddd40b0240 2020-12-05T17:39:49.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fdddae57700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fddd414c340 0x7fddd414c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:49.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fdddae57700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fddd414c340 0x7fddd414c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38772/0 (socket says 172.21.15.32:38772) 2020-12-05T17:39:49.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fdddae57700 1 -- 172.21.15.32:0/1961201336 learned_addr learned my addr 172.21.15.32:0/1961201336 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:49.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fddd93f4700 1 --2- 172.21.15.32:0/1961201336 >> v2:172.21.15.61:3300/0 conn(0x7fddd40611b0 0x7fddd4153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:49.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fddd8bf3700 1 --2- 172.21.15.32:0/1961201336 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fddd40b0240 0x7fddd40ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:49.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fdddae57700 1 -- 172.21.15.32:0/1961201336 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fddd40b0240 msgr2=0x7fddd40ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:49.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fdddae57700 1 --2- 172.21.15.32:0/1961201336 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fddd40b0240 0x7fddd40ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fdddae57700 1 -- 172.21.15.32:0/1961201336 >> v2:172.21.15.61:3300/0 conn(0x7fddd40611b0 msgr2=0x7fddd4153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:49.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fdddae57700 1 --2- 172.21.15.32:0/1961201336 >> v2:172.21.15.61:3300/0 conn(0x7fddd40611b0 0x7fddd4153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fdddae57700 1 -- 172.21.15.32:0/1961201336 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd4152da0 con 0x7fddd414c340 2020-12-05T17:39:49.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fddd8bf3700 1 --2- 172.21.15.32:0/1961201336 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fddd40b0240 0x7fddd40ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fdddae57700 1 --2- 172.21.15.32:0/1961201336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddd414c340 0x7fddd414c790 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7fddcc002f30 tx=0x7fddcc006e80).ready entity=mon.0 client_cookie=b7708d0d9eb977c1 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fddcbfff700 1 -- 172.21.15.32:0/1961201336 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddcc0097f0 con 0x7fddd414c340 2020-12-05T17:39:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fddcbfff700 1 -- 172.21.15.32:0/1961201336 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddcc009950 con 0x7fddd414c340 2020-12-05T17:39:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fddcbfff700 1 -- 172.21.15.32:0/1961201336 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddcc009cb0 con 0x7fddd414c340 2020-12-05T17:39:49.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fddd9bf5700 1 -- 172.21.15.32:0/1961201336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddd414c340 msgr2=0x7fddd414c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:49.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fddd9bf5700 1 --2- 172.21.15.32:0/1961201336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddd414c340 0x7fddd414c790 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7fddcc002f30 tx=0x7fddcc006e80).stop 2020-12-05T17:39:49.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fddd9bf5700 1 -- 172.21.15.32:0/1961201336 shutdown_connections 2020-12-05T17:39:49.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fddd9bf5700 1 --2- 172.21.15.32:0/1961201336 >> v2:172.21.15.61:3300/0 conn(0x7fddd40611b0 0x7fddd4153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fddd9bf5700 1 --2- 172.21.15.32:0/1961201336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddd414c340 0x7fddd414c790 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fddd9bf5700 1 --2- 172.21.15.32:0/1961201336 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fddd40b0240 0x7fddd40ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fddd9bf5700 1 -- 172.21.15.32:0/1961201336 >> 172.21.15.32:0/1961201336 conn(0x7fddd40b5f90 msgr2=0x7fddd40b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:49.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fddd9bf5700 1 -- 172.21.15.32:0/1961201336 shutdown_connections 2020-12-05T17:39:49.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.623+0000 7fddd9bf5700 1 -- 172.21.15.32:0/1961201336 wait complete. 2020-12-05T17:39:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.627+0000 7fddd9bf5700 1 Processor -- start 2020-12-05T17:39:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.627+0000 7fddd9bf5700 1 -- start start 2020-12-05T17:39:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.627+0000 7fddd9bf5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fddd40611b0 0x7fddd4074a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.627+0000 7fddd9bf5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddd40b0240 0x7fddd40af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.627+0000 7fddd9bf5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddd414c340 0x7fddd413c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.627+0000 7fddd9bf5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd40665f0 con 0x7fddd414c340 2020-12-05T17:39:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.627+0000 7fddd9bf5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd4063990 con 0x7fddd40b0240 2020-12-05T17:39:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.627+0000 7fddd9bf5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd4063c30 con 0x7fddd40611b0 2020-12-05T17:39:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.627+0000 7fdddae57700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddd414c340 0x7fddd413c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.627+0000 7fdddae57700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddd414c340 0x7fddd413c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38778/0 (socket says 172.21.15.32:38778) 2020-12-05T17:39:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.627+0000 7fddd93f4700 1 --2- 172.21.15.32:0/1716129576 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fddd40611b0 0x7fddd4074a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.627+0000 7fdddae57700 1 -- 172.21.15.32:0/1716129576 learned_addr learned my addr 172.21.15.32:0/1716129576 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.627+0000 7fddd8bf3700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddd40b0240 0x7fddd40af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:49.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.627+0000 7fdddae57700 1 -- 172.21.15.32:0/1716129576 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fddd40611b0 msgr2=0x7fddd4074a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:49.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.627+0000 7fdddae57700 1 --2- 172.21.15.32:0/1716129576 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fddd40611b0 0x7fddd4074a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.627+0000 7fdddae57700 1 -- 172.21.15.32:0/1716129576 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddd40b0240 msgr2=0x7fddd40af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:49.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.627+0000 7fdddae57700 1 --2- 172.21.15.32:0/1716129576 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddd40b0240 0x7fddd40af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.627+0000 7fdddae57700 1 -- 172.21.15.32:0/1716129576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd4063ed0 con 0x7fddd414c340 2020-12-05T17:39:49.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.627+0000 7fdddae57700 1 --2- 172.21.15.32:0/1716129576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddd414c340 0x7fddd413c6c0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7fddcc002f30 tx=0x7fddcc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:49.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.631+0000 7fddc9ffb700 1 -- 172.21.15.32:0/1716129576 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddcc011b00 con 0x7fddd414c340 2020-12-05T17:39:49.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.631+0000 7fddc9ffb700 1 -- 172.21.15.32:0/1716129576 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddcc011c60 con 0x7fddd414c340 2020-12-05T17:39:49.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.631+0000 7fddc9ffb700 1 -- 172.21.15.32:0/1716129576 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddcc01e460 con 0x7fddd414c340 2020-12-05T17:39:49.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.631+0000 7fddd9bf5700 1 -- 172.21.15.32:0/1716129576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd4063610 con 0x7fddd414c340 2020-12-05T17:39:49.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.631+0000 7fddd9bf5700 1 -- 172.21.15.32:0/1716129576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd413cbb0 con 0x7fddd414c340 2020-12-05T17:39:49.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.631+0000 7fddc9ffb700 1 -- 172.21.15.32:0/1716129576 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fddcc025030 con 0x7fddd414c340 2020-12-05T17:39:49.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.631+0000 7fddd9bf5700 1 -- 172.21.15.32:0/1716129576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd4064060 con 0x7fddd414c340 2020-12-05T17:39:49.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.631+0000 7fddc9ffb700 1 --2- 172.21.15.32:0/1716129576 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fddb40553b0 0x7fddb4057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:49.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.631+0000 7fddd93f4700 1 --2- 172.21.15.32:0/1716129576 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fddb40553b0 0x7fddb4057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:49.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.631+0000 7fddc9ffb700 1 -- 172.21.15.32:0/1716129576 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fddcc06f2e0 con 0x7fddd414c340 2020-12-05T17:39:49.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.631+0000 7fddd93f4700 1 --2- 172.21.15.32:0/1716129576 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fddb40553b0 0x7fddb4057870 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7fddd000b000 tx=0x7fddd00091a0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:49.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.639+0000 7fddc9ffb700 1 -- 172.21.15.32:0/1716129576 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fddcc045500 con 0x7fddd414c340 2020-12-05T17:39:49.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.831+0000 7fddd9bf5700 1 -- 172.21.15.32:0/1716129576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fddd4057050 con 0x7fddd414c340 2020-12-05T17:39:49.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.835+0000 7fddc9ffb700 1 -- 172.21.15.32:0/1716129576 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v33) v1 ==== 184+0+0 (secure 0 0 0) 0x7fddcc0230e0 con 0x7fddd414c340 2020-12-05T17:39:49.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.839+0000 7fddd9bf5700 1 -- 172.21.15.32:0/1716129576 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fddb40553b0 msgr2=0x7fddb4057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:49.845 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.839+0000 7fddd9bf5700 1 --2- 172.21.15.32:0/1716129576 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fddb40553b0 0x7fddb4057870 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7fddd000b000 tx=0x7fddd00091a0).stop 2020-12-05T17:39:49.845 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.839+0000 7fddd9bf5700 1 -- 172.21.15.32:0/1716129576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddd414c340 msgr2=0x7fddd413c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.839+0000 7fddd9bf5700 1 --2- 172.21.15.32:0/1716129576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddd414c340 0x7fddd413c6c0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7fddcc002f30 tx=0x7fddcc015000).stop 2020-12-05T17:39:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.839+0000 7fddd9bf5700 1 -- 172.21.15.32:0/1716129576 shutdown_connections 2020-12-05T17:39:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.839+0000 7fddd9bf5700 1 --2- 172.21.15.32:0/1716129576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddd414c340 0x7fddd413c6c0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.839+0000 7fddd9bf5700 1 --2- 172.21.15.32:0/1716129576 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fddb40553b0 0x7fddb4057870 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.839+0000 7fddd9bf5700 1 --2- 172.21.15.32:0/1716129576 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fddd40611b0 0x7fddd4074a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.839+0000 7fddd9bf5700 1 --2- 172.21.15.32:0/1716129576 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddd40b0240 0x7fddd40af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.839+0000 7fddd9bf5700 1 -- 172.21.15.32:0/1716129576 >> 172.21.15.32:0/1716129576 conn(0x7fddd40b5f90 msgr2=0x7fddd41505e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:49.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.839+0000 7fddd9bf5700 1 -- 172.21.15.32:0/1716129576 shutdown_connections 2020-12-05T17:39:49.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.839+0000 7fddd9bf5700 1 -- 172.21.15.32:0/1716129576 wait complete. 2020-12-05T17:39:49.848 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2020-12-05T17:39:49.857 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:39:49.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.979+0000 7fbe1359e700 1 Processor -- start 2020-12-05T17:39:49.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.979+0000 7fbe1359e700 1 -- start start 2020-12-05T17:39:49.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.979+0000 7fbe1359e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fbe140611b0 0x7fbe14153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:49.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.979+0000 7fbe1359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbe140b0240 0x7fbe140ae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:49.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.979+0000 7fbe1359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe14080a90 con 0x7fbe1414c340 2020-12-05T17:39:49.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.979+0000 7fbe1359e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe14152860 con 0x7fbe140b0240 2020-12-05T17:39:49.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.979+0000 7fbe1359e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe14152b00 con 0x7fbe140611b0 2020-12-05T17:39:49.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.979+0000 7fbe1259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbe140b0240 0x7fbe140ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:49.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.979+0000 7fbe1259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbe140b0240 0x7fbe140ae860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38784/0 (socket says 172.21.15.32:38784) 2020-12-05T17:39:49.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.979+0000 7fbe1259c700 1 -- 172.21.15.32:0/1722566653 learned_addr learned my addr 172.21.15.32:0/1722566653 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:49.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.979+0000 7fbe12d9d700 1 --2- 172.21.15.32:0/1722566653 >> v2:172.21.15.61:3300/0 conn(0x7fbe140611b0 0x7fbe14153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:49.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.979+0000 7fbe1259c700 1 -- 172.21.15.32:0/1722566653 >> v2:172.21.15.61:3300/0 conn(0x7fbe140611b0 msgr2=0x7fbe14153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:49.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.983+0000 7fbe1259c700 1 --2- 172.21.15.32:0/1722566653 >> v2:172.21.15.61:3300/0 conn(0x7fbe140611b0 0x7fbe14153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.983+0000 7fbe1259c700 1 -- 172.21.15.32:0/1722566653 >> v1:172.21.15.32:6789/0 conn(0x7fbe1414c340 legacy=0x7fbe1414f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:49.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.983+0000 7fbe1259c700 1 -- 172.21.15.32:0/1722566653 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe14152da0 con 0x7fbe140b0240 2020-12-05T17:39:49.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.983+0000 7fbe1259c700 1 --2- 172.21.15.32:0/1722566653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe140b0240 0x7fbe140ae860 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7fbdfc002f30 tx=0x7fbdfc006e80).ready entity=mon.0 client_cookie=ff66024886abdc37 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:49.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.983+0000 7fbe11d9b700 1 -- 172.21.15.32:0/1722566653 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdfc0097f0 con 0x7fbe140b0240 2020-12-05T17:39:49.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.983+0000 7fbe11d9b700 1 -- 172.21.15.32:0/1722566653 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdfc009950 con 0x7fbe140b0240 2020-12-05T17:39:49.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.983+0000 7fbe11d9b700 1 -- 172.21.15.32:0/1722566653 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdfc009ac0 con 0x7fbe140b0240 2020-12-05T17:39:49.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.983+0000 7fbe1359e700 1 -- 172.21.15.32:0/1722566653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe140b0240 msgr2=0x7fbe140ae860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:49.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.983+0000 7fbe1359e700 1 --2- 172.21.15.32:0/1722566653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe140b0240 0x7fbe140ae860 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7fbdfc002f30 tx=0x7fbdfc006e80).stop 2020-12-05T17:39:49.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.983+0000 7fbe1359e700 1 -- 172.21.15.32:0/1722566653 shutdown_connections 2020-12-05T17:39:49.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.983+0000 7fbe1359e700 1 --2- 172.21.15.32:0/1722566653 >> v2:172.21.15.61:3300/0 conn(0x7fbe140611b0 0x7fbe14153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.983+0000 7fbe1359e700 1 --2- 172.21.15.32:0/1722566653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe140b0240 0x7fbe140ae860 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:49.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.983+0000 7fbe1359e700 1 -- 172.21.15.32:0/1722566653 >> 172.21.15.32:0/1722566653 conn(0x7fbe140b5f90 msgr2=0x7fbe140b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:49.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.983+0000 7fbe1359e700 1 -- 172.21.15.32:0/1722566653 shutdown_connections 2020-12-05T17:39:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.983+0000 7fbe1359e700 1 -- 172.21.15.32:0/1722566653 wait complete. 2020-12-05T17:39:49.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.983+0000 7fbe1359e700 1 Processor -- start 2020-12-05T17:39:49.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.983+0000 7fbe1359e700 1 -- start start 2020-12-05T17:39:49.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.983+0000 7fbe1359e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbe140611b0 0x7fbe14066330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:49.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe1359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe140b0240 0x7fbe1406fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:49.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe1359e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe1414c340 0x7fbe1411c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:50.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe1359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe14066f80 con 0x7fbe140b0240 2020-12-05T17:39:50.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe1359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe140722f0 con 0x7fbe140611b0 2020-12-05T17:39:50.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe1359e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe14072580 con 0x7fbe1414c340 2020-12-05T17:39:50.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe1259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe140b0240 0x7fbe1406fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:50.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe1259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe140b0240 0x7fbe1406fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38788/0 (socket says 172.21.15.32:38788) 2020-12-05T17:39:50.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe1259c700 1 -- 172.21.15.32:0/2285906989 learned_addr learned my addr 172.21.15.32:0/2285906989 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:50.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe12d9d700 1 --2- 172.21.15.32:0/2285906989 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbe140611b0 0x7fbe14066330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:50.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe18d8d700 1 --2- 172.21.15.32:0/2285906989 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe1414c340 0x7fbe1411c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:50.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe1259c700 1 -- 172.21.15.32:0/2285906989 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe1414c340 msgr2=0x7fbe1411c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:50.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe1259c700 1 --2- 172.21.15.32:0/2285906989 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe1414c340 0x7fbe1411c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:50.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe1259c700 1 -- 172.21.15.32:0/2285906989 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbe140611b0 msgr2=0x7fbe14066330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:50.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe1259c700 1 --2- 172.21.15.32:0/2285906989 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbe140611b0 0x7fbe14066330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:50.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe1259c700 1 -- 172.21.15.32:0/2285906989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe14066ae0 con 0x7fbe140b0240 2020-12-05T17:39:50.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe12d9d700 1 --2- 172.21.15.32:0/2285906989 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbe140611b0 0x7fbe14066330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:50.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe1259c700 1 --2- 172.21.15.32:0/2285906989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe140b0240 0x7fbe1406fc30 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7fbdfc002f30 tx=0x7fbdfc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:50.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe0b7fe700 1 -- 172.21.15.32:0/2285906989 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdfc021440 con 0x7fbe140b0240 2020-12-05T17:39:50.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe0b7fe700 1 -- 172.21.15.32:0/2285906989 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdfc0215a0 con 0x7fbe140b0240 2020-12-05T17:39:50.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe0b7fe700 1 -- 172.21.15.32:0/2285906989 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdfc021900 con 0x7fbe140b0240 2020-12-05T17:39:50.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe1359e700 1 -- 172.21.15.32:0/2285906989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe14070120 con 0x7fbe140b0240 2020-12-05T17:39:50.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe1359e700 1 -- 172.21.15.32:0/2285906989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe14065960 con 0x7fbe140b0240 2020-12-05T17:39:50.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe0b7fe700 1 -- 172.21.15.32:0/2285906989 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbdfc027020 con 0x7fbe140b0240 2020-12-05T17:39:50.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe1359e700 1 -- 172.21.15.32:0/2285906989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe14072740 con 0x7fbe140b0240 2020-12-05T17:39:50.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe0b7fe700 1 --2- 172.21.15.32:0/2285906989 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbdf8055320 0x7fbdf80577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:50.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe12d9d700 1 --2- 172.21.15.32:0/2285906989 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbdf8055320 0x7fbdf80577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:50.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.987+0000 7fbe0b7fe700 1 -- 172.21.15.32:0/2285906989 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbdfc06f2e0 con 0x7fbe140b0240 2020-12-05T17:39:50.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.991+0000 7fbe12d9d700 1 --2- 172.21.15.32:0/2285906989 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbdf8055320 0x7fbdf80577e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7fbe0400c000 tx=0x7fbe04008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:50.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:49.995+0000 7fbe0b7fe700 1 -- 172.21.15.32:0/2285906989 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbdfc0453b0 con 0x7fbe140b0240 2020-12-05T17:39:50.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.219+0000 7fbe1359e700 1 -- 172.21.15.32:0/2285906989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fbe14066c70 con 0x7fbe140b0240 2020-12-05T17:39:50.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.223+0000 7fbe0b7fe700 1 -- 172.21.15.32:0/2285906989 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v34) v1 ==== 184+0+0 (secure 0 0 0) 0x7fbdfc015020 con 0x7fbe140b0240 2020-12-05T17:39:50.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.227+0000 7fbe1359e700 1 -- 172.21.15.32:0/2285906989 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbdf8055320 msgr2=0x7fbdf80577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:50.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.227+0000 7fbe1359e700 1 --2- 172.21.15.32:0/2285906989 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbdf8055320 0x7fbdf80577e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7fbe0400c000 tx=0x7fbe04008040).stop 2020-12-05T17:39:50.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.227+0000 7fbe1359e700 1 -- 172.21.15.32:0/2285906989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe140b0240 msgr2=0x7fbe1406fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:50.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.227+0000 7fbe1359e700 1 --2- 172.21.15.32:0/2285906989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe140b0240 0x7fbe1406fc30 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7fbdfc002f30 tx=0x7fbdfc016000).stop 2020-12-05T17:39:50.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.227+0000 7fbe1359e700 1 -- 172.21.15.32:0/2285906989 shutdown_connections 2020-12-05T17:39:50.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.227+0000 7fbe1359e700 1 --2- 172.21.15.32:0/2285906989 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbdf8055320 0x7fbdf80577e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:50.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.227+0000 7fbe1359e700 1 --2- 172.21.15.32:0/2285906989 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbe1414c340 0x7fbe1411c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:50.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.227+0000 7fbe1359e700 1 --2- 172.21.15.32:0/2285906989 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbe140611b0 0x7fbe14066330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:50.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.227+0000 7fbe1359e700 1 --2- 172.21.15.32:0/2285906989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe140b0240 0x7fbe1406fc30 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:50.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.227+0000 7fbe1359e700 1 -- 172.21.15.32:0/2285906989 >> 172.21.15.32:0/2285906989 conn(0x7fbe140b5f90 msgr2=0x7fbe140b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:50.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.227+0000 7fbe1359e700 1 -- 172.21.15.32:0/2285906989 shutdown_connections 2020-12-05T17:39:50.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.227+0000 7fbe1359e700 1 -- 172.21.15.32:0/2285906989 wait complete. 2020-12-05T17:39:50.236 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.2 2020-12-05T17:39:50.246 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:39:50.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.351+0000 7fa8f694f700 1 Processor -- start 2020-12-05T17:39:50.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.351+0000 7fa8f694f700 1 -- start start 2020-12-05T17:39:50.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f694f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa8f005f6f0 0x7fa8f01428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:50.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f694f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa8f00b1980 0x7fa8f0146580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:50.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f694f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f005fad0 con 0x7fa8f0146a70 2020-12-05T17:39:50.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f694f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8f00b2f80 con 0x7fa8f00b1980 2020-12-05T17:39:50.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f694f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0145e40 con 0x7fa8f005f6f0 2020-12-05T17:39:50.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f514c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa8f00b1980 0x7fa8f0146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:50.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f594d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa8f005f6f0 0x7fa8f01428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f514c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa8f00b1980 0x7fa8f0146580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:58688/0 (socket says 172.21.15.32:58688) 2020-12-05T17:39:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f614e700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fa8f0146a70 0x7fa8f0066070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:54088/0 (socket says 172.21.15.32:54088) 2020-12-05T17:39:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f514c700 1 -- 172.21.15.32:0/3825900076 learned_addr learned my addr 172.21.15.32:0/3825900076 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f514c700 1 -- 172.21.15.32:0/3825900076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa8f005f6f0 msgr2=0x7fa8f01428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f514c700 1 --2- 172.21.15.32:0/3825900076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa8f005f6f0 0x7fa8f01428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:50.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f514c700 1 -- 172.21.15.32:0/3825900076 >> v1:172.21.15.61:6789/0 conn(0x7fa8f0146a70 legacy=0x7fa8f0066070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:50.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f494b700 1 -- 172.21.15.32:0/3825900076 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1252031041 0 0) 0x7fa8f005fad0 con 0x7fa8f0146a70 2020-12-05T17:39:50.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f514c700 1 -- 172.21.15.32:0/3825900076 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f01460e0 con 0x7fa8f00b1980 2020-12-05T17:39:50.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f514c700 1 --2- 172.21.15.32:0/3825900076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa8f00b1980 0x7fa8f0146580 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fa8e0002f30 tx=0x7fa8e0006e80).ready entity=mon.1 client_cookie=bf206af5561d5cb4 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:50.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f494b700 1 -- 172.21.15.32:0/3825900076 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8e00097f0 con 0x7fa8f00b1980 2020-12-05T17:39:50.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f494b700 1 -- 172.21.15.32:0/3825900076 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e0009950 con 0x7fa8f00b1980 2020-12-05T17:39:50.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f494b700 1 -- 172.21.15.32:0/3825900076 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8e0009cb0 con 0x7fa8f00b1980 2020-12-05T17:39:50.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f694f700 1 -- 172.21.15.32:0/3825900076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa8f00b1980 msgr2=0x7fa8f0146580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:50.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f694f700 1 --2- 172.21.15.32:0/3825900076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa8f00b1980 0x7fa8f0146580 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fa8e0002f30 tx=0x7fa8e0006e80).stop 2020-12-05T17:39:50.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f694f700 1 -- 172.21.15.32:0/3825900076 shutdown_connections 2020-12-05T17:39:50.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f694f700 1 --2- 172.21.15.32:0/3825900076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa8f005f6f0 0x7fa8f01428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:50.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f694f700 1 --2- 172.21.15.32:0/3825900076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa8f00b1980 0x7fa8f0146580 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:50.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f694f700 1 -- 172.21.15.32:0/3825900076 >> 172.21.15.32:0/3825900076 conn(0x7fa8f00604c0 msgr2=0x7fa8f005c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:50.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f694f700 1 -- 172.21.15.32:0/3825900076 shutdown_connections 2020-12-05T17:39:50.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.355+0000 7fa8f694f700 1 -- 172.21.15.32:0/3825900076 wait complete. 2020-12-05T17:39:50.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8f694f700 1 Processor -- start 2020-12-05T17:39:50.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8f694f700 1 -- start start 2020-12-05T17:39:50.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8f694f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa8f005f6f0 0x7fa8f0066910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:50.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8f694f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa8f00b1980 0x7fa8f0144770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:50.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8f694f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f0146a70 0x7fa8f013e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:50.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8f694f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0152650 con 0x7fa8f0146a70 2020-12-05T17:39:50.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8f694f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0065b60 con 0x7fa8f005f6f0 2020-12-05T17:39:50.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8f694f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0145210 con 0x7fa8f00b1980 2020-12-05T17:39:50.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8f614e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f0146a70 0x7fa8f013e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:50.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8f614e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f0146a70 0x7fa8f013e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38806/0 (socket says 172.21.15.32:38806) 2020-12-05T17:39:50.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8f614e700 1 -- 172.21.15.32:0/3269107824 learned_addr learned my addr 172.21.15.32:0/3269107824 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:50.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8f594d700 1 --2- 172.21.15.32:0/3269107824 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa8f005f6f0 0x7fa8f0066910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:50.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8f514c700 1 --2- 172.21.15.32:0/3269107824 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa8f00b1980 0x7fa8f0144770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:50.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8f614e700 1 -- 172.21.15.32:0/3269107824 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa8f00b1980 msgr2=0x7fa8f0144770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:50.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8f614e700 1 --2- 172.21.15.32:0/3269107824 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa8f00b1980 0x7fa8f0144770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:50.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8f614e700 1 -- 172.21.15.32:0/3269107824 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa8f005f6f0 msgr2=0x7fa8f0066910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:50.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8f614e700 1 --2- 172.21.15.32:0/3269107824 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa8f005f6f0 0x7fa8f0066910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8f614e700 1 -- 172.21.15.32:0/3269107824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f01515f0 con 0x7fa8f0146a70 2020-12-05T17:39:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8f614e700 1 --2- 172.21.15.32:0/3269107824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f0146a70 0x7fa8f013e920 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7fa8e8000f00 tx=0x7fa8e8007ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8deffd700 1 -- 172.21.15.32:0/3269107824 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8e8002080 con 0x7fa8f0146a70 2020-12-05T17:39:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8deffd700 1 -- 172.21.15.32:0/3269107824 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e8005ab0 con 0x7fa8f0146a70 2020-12-05T17:39:50.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8f694f700 1 -- 172.21.15.32:0/3269107824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f0064660 con 0x7fa8f0146a70 2020-12-05T17:39:50.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8deffd700 1 -- 172.21.15.32:0/3269107824 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8e8005e10 con 0x7fa8f0146a70 2020-12-05T17:39:50.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.359+0000 7fa8f694f700 1 -- 172.21.15.32:0/3269107824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f013efb0 con 0x7fa8f0146a70 2020-12-05T17:39:50.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.363+0000 7fa8f694f700 1 -- 172.21.15.32:0/3269107824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f0065d20 con 0x7fa8f0146a70 2020-12-05T17:39:50.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.363+0000 7fa8deffd700 1 -- 172.21.15.32:0/3269107824 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa8e8010030 con 0x7fa8f0146a70 2020-12-05T17:39:50.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.363+0000 7fa8deffd700 1 --2- 172.21.15.32:0/3269107824 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa8d0055440 0x7fa8d0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:50.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.363+0000 7fa8f594d700 1 --2- 172.21.15.32:0/3269107824 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa8d0055440 0x7fa8d0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:50.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.363+0000 7fa8deffd700 1 -- 172.21.15.32:0/3269107824 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa8e806e040 con 0x7fa8f0146a70 2020-12-05T17:39:50.380 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.363+0000 7fa8f594d700 1 --2- 172.21.15.32:0/3269107824 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa8d0055440 0x7fa8d0057900 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fa8ec00b000 tx=0x7fa8ec006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:50.381 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.367+0000 7fa8deffd700 1 -- 172.21.15.32:0/3269107824 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa8e80441e0 con 0x7fa8f0146a70 2020-12-05T17:39:50.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.559+0000 7fa8f694f700 1 -- 172.21.15.32:0/3269107824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8f00647f0 con 0x7fa8f0146a70 2020-12-05T17:39:50.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.559+0000 7fa8deffd700 1 -- 172.21.15.32:0/3269107824 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+2140 (secure 0 0 0) 0x7fa8e8047800 con 0x7fa8f0146a70 2020-12-05T17:39:50.565 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:50.565 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":47,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/3759190037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3759190037},{"type":"v1","addr":"172.21.15.61:6833","nonce":3759190037}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":45}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:39:47.471195+0000","modified":"2020-12-05T17:39:48.473895+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7233":{"gid":7233,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4174090156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4174090156},{"type":"v1","addr":"172.21.15.32:6835","nonce":4174090156}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2020-12-05T17:39:50.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.559+0000 7fa8f694f700 1 -- 172.21.15.32:0/3269107824 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa8d0055440 msgr2=0x7fa8d0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:50.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.559+0000 7fa8f694f700 1 --2- 172.21.15.32:0/3269107824 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa8d0055440 0x7fa8d0057900 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fa8ec00b000 tx=0x7fa8ec006dc0).stop 2020-12-05T17:39:50.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.559+0000 7fa8f694f700 1 -- 172.21.15.32:0/3269107824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f0146a70 msgr2=0x7fa8f013e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:50.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.559+0000 7fa8f694f700 1 --2- 172.21.15.32:0/3269107824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f0146a70 0x7fa8f013e920 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7fa8e8000f00 tx=0x7fa8e8007ec0).stop 2020-12-05T17:39:50.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.559+0000 7fa8f694f700 1 -- 172.21.15.32:0/3269107824 shutdown_connections 2020-12-05T17:39:50.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.559+0000 7fa8f694f700 1 --2- 172.21.15.32:0/3269107824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f0146a70 0x7fa8f013e920 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:50.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.559+0000 7fa8f694f700 1 --2- 172.21.15.32:0/3269107824 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa8f005f6f0 0x7fa8f0066910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:50.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.559+0000 7fa8f694f700 1 --2- 172.21.15.32:0/3269107824 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa8d0055440 0x7fa8d0057900 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:50.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.559+0000 7fa8f694f700 1 --2- 172.21.15.32:0/3269107824 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa8f00b1980 0x7fa8f0144770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:50.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.559+0000 7fa8f694f700 1 -- 172.21.15.32:0/3269107824 >> 172.21.15.32:0/3269107824 conn(0x7fa8f00604c0 msgr2=0x7fa8f006a840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:50.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.563+0000 7fa8f694f700 1 -- 172.21.15.32:0/3269107824 shutdown_connections 2020-12-05T17:39:50.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:50.563+0000 7fa8f694f700 1 -- 172.21.15.32:0/3269107824 wait complete. 2020-12-05T17:39:50.569 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 47 2020-12-05T17:39:50.781 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:39:47.471195+0000', 'modified': '2020-12-05T17:39:48.473895+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7233}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7233': {'gid': 7233, 'name': 'a', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/4174090156', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 4174090156}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 4174090156}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:39:50.781 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:39:50.782 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:39:50.914 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":7233,"want_state":"up:active","state":"up:active","rank_uptime":3.4284970270000001,"mdsmap_epoch":47,"osdmap_epoch":102,"osdmap_epoch_barrier":101,"uptime":8.6644698160000004} 2020-12-05T17:39:50.914 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 7233, "mdsmap_epoch": 47, "osdmap_epoch": 102, "osdmap_epoch_barrier": 101, "rank_uptime": 3.428497027, "state": "up:active", "uptime": 8.664469816, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:39:50.915 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:39:51.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.023+0000 7f944268d700 1 Processor -- start 2020-12-05T17:39:51.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.023+0000 7f944268d700 1 -- start start 2020-12-05T17:39:51.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.027+0000 7f944268d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f943c0b1fc0 0x7f943c0af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.027+0000 7f944268d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943c146a90 con 0x7f943c147400 2020-12-05T17:39:51.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.027+0000 7f944268d700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943c0b23a0 con 0x7f943c0ab710 2020-12-05T17:39:51.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.027+0000 7f944268d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f943c061e90 con 0x7f943c0b1fc0 2020-12-05T17:39:51.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.027+0000 7f944168b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f943c0b1fc0 0x7f943c0af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.027+0000 7f9441e8c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f943c147400 0x7f943c145d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41462/0 (socket says 172.21.15.32:41462) 2020-12-05T17:39:51.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.027+0000 7f9441e8c700 1 -- 172.21.15.32:0/1725035178 learned_addr learned my addr 172.21.15.32:0/1725035178 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:51.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.027+0000 7f944168b700 1 -- 172.21.15.32:0/1725035178 >> v1:172.21.15.61:6789/0 conn(0x7f943c0ab710 legacy=0x7f943c061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:51.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.027+0000 7f944168b700 1 -- 172.21.15.32:0/1725035178 >> v1:172.21.15.32:6789/0 conn(0x7f943c147400 legacy=0x7f943c145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:51.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.027+0000 7f944168b700 1 -- 172.21.15.32:0/1725035178 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943c146cc0 con 0x7f943c0b1fc0 2020-12-05T17:39:51.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.027+0000 7f9433fff700 1 -- 172.21.15.32:0/1725035178 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3225183920 0 0) 0x7f943c146a90 con 0x7f943c147400 2020-12-05T17:39:51.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.027+0000 7f944168b700 1 --2- 172.21.15.32:0/1725035178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f943c0b1fc0 0x7f943c0af230 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f942c002f30 tx=0x7f942c006dc0).ready entity=mon.0 client_cookie=180263c2a9bdf9a6 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:51.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.027+0000 7f9433fff700 1 -- 172.21.15.32:0/1725035178 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f942c0097f0 con 0x7f943c0b1fc0 2020-12-05T17:39:51.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.027+0000 7f9433fff700 1 -- 172.21.15.32:0/1725035178 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942c009950 con 0x7f943c0b1fc0 2020-12-05T17:39:51.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.027+0000 7f9433fff700 1 -- 172.21.15.32:0/1725035178 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f942c009cb0 con 0x7f943c0b1fc0 2020-12-05T17:39:51.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.027+0000 7f944268d700 1 -- 172.21.15.32:0/1725035178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f943c0b1fc0 msgr2=0x7f943c0af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:51.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.027+0000 7f944268d700 1 --2- 172.21.15.32:0/1725035178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f943c0b1fc0 0x7f943c0af230 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f942c002f30 tx=0x7f942c006dc0).stop 2020-12-05T17:39:51.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.027+0000 7f944268d700 1 -- 172.21.15.32:0/1725035178 shutdown_connections 2020-12-05T17:39:51.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.027+0000 7f944268d700 1 --2- 172.21.15.32:0/1725035178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f943c0b1fc0 0x7f943c0af230 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.027+0000 7f944268d700 1 -- 172.21.15.32:0/1725035178 >> 172.21.15.32:0/1725035178 conn(0x7f943c0b5530 msgr2=0x7f943c0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:51.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.027+0000 7f944268d700 1 -- 172.21.15.32:0/1725035178 shutdown_connections 2020-12-05T17:39:51.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.027+0000 7f944268d700 1 -- 172.21.15.32:0/1725035178 wait complete. 2020-12-05T17:39:51.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.031+0000 7f944268d700 1 Processor -- start 2020-12-05T17:39:51.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.031+0000 7f944268d700 1 -- start start 2020-12-05T17:39:51.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.031+0000 7f944268d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f943c0ab710 0x7f943c158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.031+0000 7f944268d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f943c0b1fc0 0x7f943c144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.031+0000 7f944268d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f943c147400 0x7f943c09e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.031+0000 7f944268d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c0adab0 con 0x7f943c147400 2020-12-05T17:39:51.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.031+0000 7f944268d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c0ac7f0 con 0x7f943c0b1fc0 2020-12-05T17:39:51.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.031+0000 7f944268d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943c0abb50 con 0x7f943c0ab710 2020-12-05T17:39:51.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.031+0000 7f9441e8c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f943c147400 0x7f943c09e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.031+0000 7f9441e8c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f943c147400 0x7f943c09e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38820/0 (socket says 172.21.15.32:38820) 2020-12-05T17:39:51.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.031+0000 7f9441e8c700 1 -- 172.21.15.32:0/437314717 learned_addr learned my addr 172.21.15.32:0/437314717 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:51.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.031+0000 7f944168b700 1 --2- 172.21.15.32:0/437314717 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f943c0ab710 0x7f943c158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.031+0000 7f9440e8a700 1 --2- 172.21.15.32:0/437314717 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f943c0b1fc0 0x7f943c144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.031+0000 7f9441e8c700 1 -- 172.21.15.32:0/437314717 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f943c0ab710 msgr2=0x7f943c158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:51.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.031+0000 7f9441e8c700 1 --2- 172.21.15.32:0/437314717 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f943c0ab710 0x7f943c158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.031+0000 7f9441e8c700 1 -- 172.21.15.32:0/437314717 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f943c0b1fc0 msgr2=0x7f943c144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:51.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.031+0000 7f9441e8c700 1 --2- 172.21.15.32:0/437314717 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f943c0b1fc0 0x7f943c144bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.031+0000 7f9441e8c700 1 -- 172.21.15.32:0/437314717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943c0ac060 con 0x7f943c147400 2020-12-05T17:39:51.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.031+0000 7f9440e8a700 1 --2- 172.21.15.32:0/437314717 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f943c0b1fc0 0x7f943c144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:51.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.031+0000 7f9441e8c700 1 --2- 172.21.15.32:0/437314717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f943c147400 0x7f943c09e740 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f9434000f00 tx=0x7f9434007ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:51.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.035+0000 7f94327fc700 1 -- 172.21.15.32:0/437314717 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9434002080 con 0x7f943c147400 2020-12-05T17:39:51.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.035+0000 7f944268d700 1 -- 172.21.15.32:0/437314717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f943c0ac2b0 con 0x7f943c147400 2020-12-05T17:39:51.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.035+0000 7f94327fc700 1 -- 172.21.15.32:0/437314717 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9434005b70 con 0x7f943c147400 2020-12-05T17:39:51.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.035+0000 7f94327fc700 1 -- 172.21.15.32:0/437314717 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9434007110 con 0x7f943c147400 2020-12-05T17:39:51.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.035+0000 7f944268d700 1 -- 172.21.15.32:0/437314717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943c14daa0 con 0x7f943c147400 2020-12-05T17:39:51.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.035+0000 7f944268d700 1 -- 172.21.15.32:0/437314717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f943c057050 con 0x7f943c147400 2020-12-05T17:39:51.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.035+0000 7f94327fc700 1 -- 172.21.15.32:0/437314717 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9434010030 con 0x7f943c147400 2020-12-05T17:39:51.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.035+0000 7f94327fc700 1 --2- 172.21.15.32:0/437314717 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9424055330 0x7f94240577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.035+0000 7f94327fc700 1 -- 172.21.15.32:0/437314717 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f943406e060 con 0x7f943c147400 2020-12-05T17:39:51.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.035+0000 7f944168b700 1 --2- 172.21.15.32:0/437314717 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9424055330 0x7f94240577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.035+0000 7f944168b700 1 --2- 172.21.15.32:0/437314717 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9424055330 0x7f94240577f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f942c01b000 tx=0x7f942c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:51.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.039+0000 7f94327fc700 1 -- 172.21.15.32:0/437314717 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9434044160 con 0x7f943c147400 2020-12-05T17:39:51.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.187+0000 7f944268d700 1 -- 172.21.15.32:0/437314717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f943c0abd10 con 0x7f943c147400 2020-12-05T17:39:51.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.187+0000 7f94327fc700 1 -- 172.21.15.32:0/437314717 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16282 (secure 0 0 0) 0x7f9434047780 con 0x7f943c147400 2020-12-05T17:39:51.194 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:51.194 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":102,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:39:48.474575+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:39:45.190638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2020-12-05T17:39:46.472081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:39:51.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.191+0000 7f944268d700 1 -- 172.21.15.32:0/437314717 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9424055330 msgr2=0x7f94240577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:51.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.191+0000 7f944268d700 1 --2- 172.21.15.32:0/437314717 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9424055330 0x7f94240577f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f942c01b000 tx=0x7f942c015000).stop 2020-12-05T17:39:51.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.191+0000 7f944268d700 1 -- 172.21.15.32:0/437314717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f943c147400 msgr2=0x7f943c09e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.191+0000 7f944268d700 1 --2- 172.21.15.32:0/437314717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f943c147400 0x7f943c09e740 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f9434000f00 tx=0x7f9434007ec0).stop 2020-12-05T17:39:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.191+0000 7f944268d700 1 -- 172.21.15.32:0/437314717 shutdown_connections 2020-12-05T17:39:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.191+0000 7f944268d700 1 --2- 172.21.15.32:0/437314717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f943c147400 0x7f943c09e740 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.191+0000 7f944268d700 1 --2- 172.21.15.32:0/437314717 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9424055330 0x7f94240577f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.191+0000 7f944268d700 1 --2- 172.21.15.32:0/437314717 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f943c0ab710 0x7f943c158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.191+0000 7f944268d700 1 --2- 172.21.15.32:0/437314717 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f943c0b1fc0 0x7f943c144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.191+0000 7f944268d700 1 -- 172.21.15.32:0/437314717 >> 172.21.15.32:0/437314717 conn(0x7f943c0b5530 msgr2=0x7f943c14a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:51.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.191+0000 7f944268d700 1 -- 172.21.15.32:0/437314717 shutdown_connections 2020-12-05T17:39:51.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.191+0000 7f944268d700 1 -- 172.21.15.32:0/437314717 wait complete. 2020-12-05T17:39:51.208 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:39:45.190638+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:39:46.472081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:39:51.209 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:39:51.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68ea249700 1 Processor -- start 2020-12-05T17:39:51.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68ea249700 1 -- start start 2020-12-05T17:39:51.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68ea249700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68e4061920 0x7f68e40aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68ea249700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68e4061d00 0x7f68e4142200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68ea249700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f68e4062180 0x7f68e41426f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68ea249700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e4062ba0 con 0x7f68e4061920 2020-12-05T17:39:51.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68ea249700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e4142be0 con 0x7f68e4062180 2020-12-05T17:39:51.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68ea249700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e40b1380 con 0x7f68e4061d00 2020-12-05T17:39:51.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68e9247700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68e4061920 0x7f68e40aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68e9247700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68e4061920 0x7f68e40aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38824/0 (socket says 172.21.15.32:38824) 2020-12-05T17:39:51.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68e9247700 1 -- 172.21.15.32:0/1395940777 learned_addr learned my addr 172.21.15.32:0/1395940777 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:51.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68e8a46700 1 --2- 172.21.15.32:0/1395940777 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68e4061d00 0x7f68e4142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68e9a48700 1 --2- 172.21.15.32:0/1395940777 >> v2:172.21.15.61:3300/0 conn(0x7f68e4062180 0x7f68e41426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68e9247700 1 -- 172.21.15.32:0/1395940777 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68e4061d00 msgr2=0x7f68e4142200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:51.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68e9247700 1 --2- 172.21.15.32:0/1395940777 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68e4061d00 0x7f68e4142200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68e9247700 1 -- 172.21.15.32:0/1395940777 >> v2:172.21.15.61:3300/0 conn(0x7f68e4062180 msgr2=0x7f68e41426f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:51.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68e9247700 1 --2- 172.21.15.32:0/1395940777 >> v2:172.21.15.61:3300/0 conn(0x7f68e4062180 0x7f68e41426f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68e9247700 1 -- 172.21.15.32:0/1395940777 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e40ae620 con 0x7f68e4061920 2020-12-05T17:39:51.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68e9247700 1 --2- 172.21.15.32:0/1395940777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68e4061920 0x7f68e40aed60 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7f68e4062ce0 tx=0x7f68e0006e80).ready entity=mon.0 client_cookie=ae9b9ee61b4e60c7 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68dbfff700 1 -- 172.21.15.32:0/1395940777 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68e00097b0 con 0x7f68e4061920 2020-12-05T17:39:51.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68dbfff700 1 -- 172.21.15.32:0/1395940777 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e0009910 con 0x7f68e4061920 2020-12-05T17:39:51.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68dbfff700 1 -- 172.21.15.32:0/1395940777 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68e0009c70 con 0x7f68e4061920 2020-12-05T17:39:51.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68ea249700 1 -- 172.21.15.32:0/1395940777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68e4061920 msgr2=0x7f68e40aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:51.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.323+0000 7f68ea249700 1 --2- 172.21.15.32:0/1395940777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68e4061920 0x7f68e40aed60 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7f68e4062ce0 tx=0x7f68e0006e80).stop 2020-12-05T17:39:51.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.327+0000 7f68ea249700 1 -- 172.21.15.32:0/1395940777 shutdown_connections 2020-12-05T17:39:51.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.327+0000 7f68ea249700 1 --2- 172.21.15.32:0/1395940777 >> v2:172.21.15.61:3300/0 conn(0x7f68e4062180 0x7f68e41426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.327+0000 7f68ea249700 1 --2- 172.21.15.32:0/1395940777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68e4061920 0x7f68e40aed60 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.327+0000 7f68ea249700 1 --2- 172.21.15.32:0/1395940777 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68e4061d00 0x7f68e4142200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.327+0000 7f68ea249700 1 -- 172.21.15.32:0/1395940777 >> 172.21.15.32:0/1395940777 conn(0x7f68e40b5620 msgr2=0x7f68e40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:51.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.327+0000 7f68ea249700 1 -- 172.21.15.32:0/1395940777 shutdown_connections 2020-12-05T17:39:51.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.327+0000 7f68ea249700 1 -- 172.21.15.32:0/1395940777 wait complete. 2020-12-05T17:39:51.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.327+0000 7f68ea249700 1 Processor -- start 2020-12-05T17:39:51.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.327+0000 7f68ea249700 1 -- start start 2020-12-05T17:39:51.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68ea249700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68e4061920 0x7f68e4065490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68ea249700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68e4061d00 0x7f68e41135a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68ea249700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68e4062180 0x7f68e4154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68ea249700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e413c6a0 con 0x7f68e4061920 2020-12-05T17:39:51.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68ea249700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e413c940 con 0x7f68e4061d00 2020-12-05T17:39:51.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68ea249700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e413cd50 con 0x7f68e4062180 2020-12-05T17:39:51.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68e9247700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68e4061920 0x7f68e4065490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68e9247700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68e4061920 0x7f68e4065490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38830/0 (socket says 172.21.15.32:38830) 2020-12-05T17:39:51.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68e9247700 1 -- 172.21.15.32:0/507214589 learned_addr learned my addr 172.21.15.32:0/507214589 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:51.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68e9a48700 1 --2- 172.21.15.32:0/507214589 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68e4062180 0x7f68e4154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68e8a46700 1 --2- 172.21.15.32:0/507214589 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68e4061d00 0x7f68e41135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68e9247700 1 -- 172.21.15.32:0/507214589 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68e4062180 msgr2=0x7f68e4154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:51.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68e9247700 1 --2- 172.21.15.32:0/507214589 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68e4062180 0x7f68e4154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68e9247700 1 -- 172.21.15.32:0/507214589 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68e4061d00 msgr2=0x7f68e41135a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:51.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68e9247700 1 --2- 172.21.15.32:0/507214589 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68e4061d00 0x7f68e41135a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68e9247700 1 -- 172.21.15.32:0/507214589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e413cf10 con 0x7f68e4061920 2020-12-05T17:39:51.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68e8a46700 1 --2- 172.21.15.32:0/507214589 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68e4061d00 0x7f68e41135a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:51.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68e9247700 1 --2- 172.21.15.32:0/507214589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68e4061920 0x7f68e4065490 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7f68e4062ce0 tx=0x7f68e0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:51.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68da7fc700 1 -- 172.21.15.32:0/507214589 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68e0011c80 con 0x7f68e4061920 2020-12-05T17:39:51.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68da7fc700 1 -- 172.21.15.32:0/507214589 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e0011de0 con 0x7f68e4061920 2020-12-05T17:39:51.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68ea249700 1 -- 172.21.15.32:0/507214589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e4155400 con 0x7f68e4061920 2020-12-05T17:39:51.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68da7fc700 1 -- 172.21.15.32:0/507214589 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68e001e650 con 0x7f68e4061920 2020-12-05T17:39:51.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68ea249700 1 -- 172.21.15.32:0/507214589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e41557f0 con 0x7f68e4061920 2020-12-05T17:39:51.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.331+0000 7f68ea249700 1 -- 172.21.15.32:0/507214589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e4057050 con 0x7f68e4061920 2020-12-05T17:39:51.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.335+0000 7f68da7fc700 1 -- 172.21.15.32:0/507214589 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f68e0025030 con 0x7f68e4061920 2020-12-05T17:39:51.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.335+0000 7f68da7fc700 1 --2- 172.21.15.32:0/507214589 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f68cc0553b0 0x7f68cc057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.335+0000 7f68e8a46700 1 --2- 172.21.15.32:0/507214589 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f68cc0553b0 0x7f68cc057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.335+0000 7f68da7fc700 1 -- 172.21.15.32:0/507214589 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f68e006f440 con 0x7f68e4061920 2020-12-05T17:39:51.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.335+0000 7f68e8a46700 1 --2- 172.21.15.32:0/507214589 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f68cc0553b0 0x7f68cc057870 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f68d400c000 tx=0x7f68d4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:51.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.343+0000 7f68da7fc700 1 -- 172.21.15.32:0/507214589 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f68e00455e0 con 0x7f68e4061920 2020-12-05T17:39:51.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.487+0000 7f68ea249700 1 -- 172.21.15.32:0/507214589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f68e405aab0 con 0x7f68e4061920 2020-12-05T17:39:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.487+0000 7f68da7fc700 1 -- 172.21.15.32:0/507214589 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v102) v1 ==== 110+0+10 (secure 0 0 0) 0x7f68e0023020 con 0x7f68e4061920 2020-12-05T17:39:51.492 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:39:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.487+0000 7f68ea249700 1 -- 172.21.15.32:0/507214589 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f68cc0553b0 msgr2=0x7f68cc057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.487+0000 7f68ea249700 1 --2- 172.21.15.32:0/507214589 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f68cc0553b0 0x7f68cc057870 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f68d400c000 tx=0x7f68d4008040).stop 2020-12-05T17:39:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.487+0000 7f68ea249700 1 -- 172.21.15.32:0/507214589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68e4061920 msgr2=0x7f68e4065490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.487+0000 7f68ea249700 1 --2- 172.21.15.32:0/507214589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68e4061920 0x7f68e4065490 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7f68e4062ce0 tx=0x7f68e0015000).stop 2020-12-05T17:39:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.487+0000 7f68ea249700 1 -- 172.21.15.32:0/507214589 shutdown_connections 2020-12-05T17:39:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.487+0000 7f68ea249700 1 --2- 172.21.15.32:0/507214589 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f68cc0553b0 0x7f68cc057870 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.487+0000 7f68ea249700 1 --2- 172.21.15.32:0/507214589 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68e4062180 0x7f68e4154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.487+0000 7f68ea249700 1 --2- 172.21.15.32:0/507214589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68e4061920 0x7f68e4065490 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.487+0000 7f68ea249700 1 --2- 172.21.15.32:0/507214589 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68e4061d00 0x7f68e41135a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.487+0000 7f68ea249700 1 -- 172.21.15.32:0/507214589 >> 172.21.15.32:0/507214589 conn(0x7f68e40b5620 msgr2=0x7f68e40af4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.487+0000 7f68ea249700 1 -- 172.21.15.32:0/507214589 shutdown_connections 2020-12-05T17:39:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.487+0000 7f68ea249700 1 -- 172.21.15.32:0/507214589 wait complete. 2020-12-05T17:39:51.507 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:39:51.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.619+0000 7f34ac9b5700 1 Processor -- start 2020-12-05T17:39:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.619+0000 7f34ac9b5700 1 -- start start 2020-12-05T17:39:51.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34ac9b5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34a813f610 0x7f34a813fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34ac9b5700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f34a80af4b0 0x7f34a80af920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34ac9b5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34a80b23e0 con 0x7f34a80b2000 2020-12-05T17:39:51.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34ac9b5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34a80aeba0 con 0x7f34a813f610 2020-12-05T17:39:51.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34ac9b5700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34a80af190 con 0x7f34a80af4b0 2020-12-05T17:39:51.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34a659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34a813f610 0x7f34a813fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34a6d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f34a80b2000 0x7f34a8142f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41486/0 (socket says 172.21.15.32:41486) 2020-12-05T17:39:51.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34a659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34a813f610 0x7f34a813fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38840/0 (socket says 172.21.15.32:38840) 2020-12-05T17:39:51.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34a659c700 1 -- 172.21.15.32:0/1942434930 learned_addr learned my addr 172.21.15.32:0/1942434930 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:51.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34a759e700 1 --2- 172.21.15.32:0/1942434930 >> v2:172.21.15.61:3300/0 conn(0x7f34a80af4b0 0x7f34a80af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34a659c700 1 -- 172.21.15.32:0/1942434930 >> v2:172.21.15.61:3300/0 conn(0x7f34a80af4b0 msgr2=0x7f34a80af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:51.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34a659c700 1 --2- 172.21.15.32:0/1942434930 >> v2:172.21.15.61:3300/0 conn(0x7f34a80af4b0 0x7f34a80af920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34a659c700 1 -- 172.21.15.32:0/1942434930 >> v1:172.21.15.32:6789/0 conn(0x7f34a80b2000 legacy=0x7f34a8142f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:51.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34a659c700 1 -- 172.21.15.32:0/1942434930 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a80ae010 con 0x7f34a813f610 2020-12-05T17:39:51.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34a759e700 1 --2- 172.21.15.32:0/1942434930 >> v2:172.21.15.61:3300/0 conn(0x7f34a80af4b0 0x7f34a80af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:51.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34a659c700 1 --2- 172.21.15.32:0/1942434930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34a813f610 0x7f34a813fac0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 rx=0x7f3490002f30 tx=0x7f3490006e80).ready entity=mon.0 client_cookie=70d03553277be74 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:51.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34a5d9b700 1 -- 172.21.15.32:0/1942434930 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34900097f0 con 0x7f34a813f610 2020-12-05T17:39:51.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34a5d9b700 1 -- 172.21.15.32:0/1942434930 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3490009950 con 0x7f34a813f610 2020-12-05T17:39:51.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34a5d9b700 1 -- 172.21.15.32:0/1942434930 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3490009cb0 con 0x7f34a813f610 2020-12-05T17:39:51.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34ac9b5700 1 -- 172.21.15.32:0/1942434930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34a813f610 msgr2=0x7f34a813fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:51.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34ac9b5700 1 --2- 172.21.15.32:0/1942434930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34a813f610 0x7f34a813fac0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 rx=0x7f3490002f30 tx=0x7f3490006e80).stop 2020-12-05T17:39:51.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34ac9b5700 1 -- 172.21.15.32:0/1942434930 shutdown_connections 2020-12-05T17:39:51.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34ac9b5700 1 --2- 172.21.15.32:0/1942434930 >> v2:172.21.15.61:3300/0 conn(0x7f34a80af4b0 0x7f34a80af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34ac9b5700 1 --2- 172.21.15.32:0/1942434930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34a813f610 0x7f34a813fac0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34ac9b5700 1 -- 172.21.15.32:0/1942434930 >> 172.21.15.32:0/1942434930 conn(0x7f34a80b5620 msgr2=0x7f34a80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:51.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34ac9b5700 1 -- 172.21.15.32:0/1942434930 shutdown_connections 2020-12-05T17:39:51.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34ac9b5700 1 -- 172.21.15.32:0/1942434930 wait complete. 2020-12-05T17:39:51.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34ac9b5700 1 Processor -- start 2020-12-05T17:39:51.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.623+0000 7f34ac9b5700 1 -- start start 2020-12-05T17:39:51.650 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f34ac9b5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34a80af4b0 0x7f34a813c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f34ac9b5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34a80b2000 0x7f34a8143ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f34ac9b5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34a813f610 0x7f34a80640e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f34ac9b5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a80645d0 con 0x7f34a80b2000 2020-12-05T17:39:51.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f34ac9b5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a8151860 con 0x7f34a813f610 2020-12-05T17:39:51.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f34ac9b5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a8151b00 con 0x7f34a80af4b0 2020-12-05T17:39:51.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f34a659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34a80b2000 0x7f34a8143ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f34a659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34a80b2000 0x7f34a8143ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38844/0 (socket says 172.21.15.32:38844) 2020-12-05T17:39:51.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f34a659c700 1 -- 172.21.15.32:0/3895273928 learned_addr learned my addr 172.21.15.32:0/3895273928 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:51.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f34a759e700 1 --2- 172.21.15.32:0/3895273928 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34a813f610 0x7f34a80640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f34a6d9d700 1 --2- 172.21.15.32:0/3895273928 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34a80af4b0 0x7f34a813c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f34a659c700 1 -- 172.21.15.32:0/3895273928 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34a80af4b0 msgr2=0x7f34a813c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:51.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f34a659c700 1 --2- 172.21.15.32:0/3895273928 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34a80af4b0 0x7f34a813c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f34a659c700 1 -- 172.21.15.32:0/3895273928 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34a813f610 msgr2=0x7f34a80640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:51.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f34a659c700 1 --2- 172.21.15.32:0/3895273928 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34a813f610 0x7f34a80640e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f34a659c700 1 -- 172.21.15.32:0/3895273928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a8151da0 con 0x7f34a80b2000 2020-12-05T17:39:51.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f34a659c700 1 --2- 172.21.15.32:0/3895273928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34a80b2000 0x7f34a8143ee0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f3490002f30 tx=0x7f3490015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:51.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f349ffff700 1 -- 172.21.15.32:0/3895273928 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3490011c10 con 0x7f34a80b2000 2020-12-05T17:39:51.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f349ffff700 1 -- 172.21.15.32:0/3895273928 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3490011d70 con 0x7f34a80b2000 2020-12-05T17:39:51.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f34ac9b5700 1 -- 172.21.15.32:0/3895273928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a8152240 con 0x7f34a80b2000 2020-12-05T17:39:51.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f349ffff700 1 -- 172.21.15.32:0/3895273928 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f349001e650 con 0x7f34a80b2000 2020-12-05T17:39:51.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f34ac9b5700 1 -- 172.21.15.32:0/3895273928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a81526f0 con 0x7f34a80b2000 2020-12-05T17:39:51.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f34ac9b5700 1 -- 172.21.15.32:0/3895273928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a8151f30 con 0x7f34a80b2000 2020-12-05T17:39:51.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f349ffff700 1 -- 172.21.15.32:0/3895273928 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3490025030 con 0x7f34a80b2000 2020-12-05T17:39:51.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.627+0000 7f349ffff700 1 --2- 172.21.15.32:0/3895273928 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f348c055470 0x7f348c057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.631+0000 7f349ffff700 1 -- 172.21.15.32:0/3895273928 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f349006f3b0 con 0x7f34a80b2000 2020-12-05T17:39:51.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.631+0000 7f34a6d9d700 1 --2- 172.21.15.32:0/3895273928 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f348c055470 0x7f348c057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.631+0000 7f34a6d9d700 1 --2- 172.21.15.32:0/3895273928 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f348c055470 0x7f348c057930 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f349800d000 tx=0x7f3498007ed0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:51.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.635+0000 7f349ffff700 1 -- 172.21.15.32:0/3895273928 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f34900455d0 con 0x7f34a80b2000 2020-12-05T17:39:51.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.779+0000 7f34ac9b5700 1 -- 172.21.15.32:0/3895273928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f34a8057050 con 0x7f34a80b2000 2020-12-05T17:39:51.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.779+0000 7f349ffff700 1 -- 172.21.15.32:0/3895273928 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v102) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3490023020 con 0x7f34a80b2000 2020-12-05T17:39:51.787 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:39:51.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.783+0000 7f34ac9b5700 1 -- 172.21.15.32:0/3895273928 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f348c055470 msgr2=0x7f348c057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:51.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.783+0000 7f34ac9b5700 1 --2- 172.21.15.32:0/3895273928 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f348c055470 0x7f348c057930 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f349800d000 tx=0x7f3498007ed0).stop 2020-12-05T17:39:51.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.783+0000 7f34ac9b5700 1 -- 172.21.15.32:0/3895273928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34a80b2000 msgr2=0x7f34a8143ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:51.790 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.783+0000 7f34ac9b5700 1 --2- 172.21.15.32:0/3895273928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34a80b2000 0x7f34a8143ee0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f3490002f30 tx=0x7f3490015000).stop 2020-12-05T17:39:51.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.783+0000 7f34ac9b5700 1 -- 172.21.15.32:0/3895273928 shutdown_connections 2020-12-05T17:39:51.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.783+0000 7f34ac9b5700 1 --2- 172.21.15.32:0/3895273928 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34a813f610 0x7f34a80640e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.783+0000 7f34ac9b5700 1 --2- 172.21.15.32:0/3895273928 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f348c055470 0x7f348c057930 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.791 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.783+0000 7f34ac9b5700 1 --2- 172.21.15.32:0/3895273928 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34a80af4b0 0x7f34a813c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.783+0000 7f34ac9b5700 1 --2- 172.21.15.32:0/3895273928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34a80b2000 0x7f34a8143ee0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.783+0000 7f34ac9b5700 1 -- 172.21.15.32:0/3895273928 >> 172.21.15.32:0/3895273928 conn(0x7f34a80b5620 msgr2=0x7f34a80b12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:51.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.783+0000 7f34ac9b5700 1 -- 172.21.15.32:0/3895273928 shutdown_connections 2020-12-05T17:39:51.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.783+0000 7f34ac9b5700 1 -- 172.21.15.32:0/3895273928 wait complete. 2020-12-05T17:39:51.801 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:39:51.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.911+0000 7f592eeee700 1 Processor -- start 2020-12-05T17:39:51.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.911+0000 7f592eeee700 1 -- start start 2020-12-05T17:39:51.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592eeee700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5928063040 0x7f592813fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592eeee700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f59280af090 0x7f59280af520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592eeee700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5928067110 con 0x7f592805f550 2020-12-05T17:39:51.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592eeee700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59280ad290 con 0x7f5928063040 2020-12-05T17:39:51.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592eeee700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592813f760 con 0x7f59280af090 2020-12-05T17:39:51.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592d6eb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5928063040 0x7f592813fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592deec700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f592805f550 0x7f59280acb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41500/0 (socket says 172.21.15.32:41500) 2020-12-05T17:39:51.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592deec700 1 -- 172.21.15.32:0/2247244391 learned_addr learned my addr 172.21.15.32:0/2247244391 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592e6ed700 1 --2- 172.21.15.32:0/2247244391 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f59280af090 0x7f59280af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592d6eb700 1 -- 172.21.15.32:0/2247244391 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f59280af090 msgr2=0x7f59280af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592d6eb700 1 --2- 172.21.15.32:0/2247244391 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f59280af090 0x7f59280af520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592ceea700 1 -- 172.21.15.32:0/2247244391 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771653189 0 0) 0x7f5928067110 con 0x7f592805f550 2020-12-05T17:39:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592d6eb700 1 -- 172.21.15.32:0/2247244391 >> v1:172.21.15.32:6789/0 conn(0x7f592805f550 legacy=0x7f59280acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:51.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592d6eb700 1 -- 172.21.15.32:0/2247244391 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592813fa00 con 0x7f5928063040 2020-12-05T17:39:51.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592d6eb700 1 --2- 172.21.15.32:0/2247244391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5928063040 0x7f592813fea0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7f5918002f30 tx=0x7f5918006dc0).ready entity=mon.0 client_cookie=83aaec9b067e0868 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:51.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592ceea700 1 -- 172.21.15.32:0/2247244391 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5918009760 con 0x7f5928063040 2020-12-05T17:39:51.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592ceea700 1 -- 172.21.15.32:0/2247244391 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59180098c0 con 0x7f5928063040 2020-12-05T17:39:51.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592ceea700 1 -- 172.21.15.32:0/2247244391 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5918009c20 con 0x7f5928063040 2020-12-05T17:39:51.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592eeee700 1 -- 172.21.15.32:0/2247244391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5928063040 msgr2=0x7f592813fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:51.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592eeee700 1 --2- 172.21.15.32:0/2247244391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5928063040 0x7f592813fea0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7f5918002f30 tx=0x7f5918006dc0).stop 2020-12-05T17:39:51.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592eeee700 1 -- 172.21.15.32:0/2247244391 shutdown_connections 2020-12-05T17:39:51.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592eeee700 1 --2- 172.21.15.32:0/2247244391 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f59280af090 0x7f59280af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592eeee700 1 --2- 172.21.15.32:0/2247244391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5928063040 0x7f592813fea0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592eeee700 1 -- 172.21.15.32:0/2247244391 >> 172.21.15.32:0/2247244391 conn(0x7f59280620b0 msgr2=0x7f5928062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:51.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.915+0000 7f592eeee700 1 -- 172.21.15.32:0/2247244391 shutdown_connections 2020-12-05T17:39:51.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.919+0000 7f592eeee700 1 -- 172.21.15.32:0/2247244391 wait complete. 2020-12-05T17:39:51.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.919+0000 7f592eeee700 1 Processor -- start 2020-12-05T17:39:51.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.919+0000 7f592eeee700 1 -- start start 2020-12-05T17:39:51.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.919+0000 7f592eeee700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f592805f550 0x7f5928139cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.919+0000 7f592eeee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5928063040 0x7f59280b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.919+0000 7f592eeee700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59280af090 0x7f5928153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.923+0000 7f592eeee700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592813e4e0 con 0x7f5928063040 2020-12-05T17:39:51.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.923+0000 7f592eeee700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5928152a20 con 0x7f59280af090 2020-12-05T17:39:51.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.923+0000 7f592eeee700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5928152cc0 con 0x7f592805f550 2020-12-05T17:39:51.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.923+0000 7f592d6eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5928063040 0x7f59280b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.923+0000 7f592d6eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5928063040 0x7f59280b21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38860/0 (socket says 172.21.15.32:38860) 2020-12-05T17:39:51.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.923+0000 7f592d6eb700 1 -- 172.21.15.32:0/954138675 learned_addr learned my addr 172.21.15.32:0/954138675 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:51.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.923+0000 7f592deec700 1 --2- 172.21.15.32:0/954138675 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f592805f550 0x7f5928139cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.923+0000 7f592e6ed700 1 --2- 172.21.15.32:0/954138675 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59280af090 0x7f5928153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.923+0000 7f592d6eb700 1 -- 172.21.15.32:0/954138675 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f592805f550 msgr2=0x7f5928139cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:51.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.923+0000 7f592d6eb700 1 --2- 172.21.15.32:0/954138675 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f592805f550 0x7f5928139cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.923+0000 7f592d6eb700 1 -- 172.21.15.32:0/954138675 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59280af090 msgr2=0x7f5928153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:51.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.923+0000 7f592d6eb700 1 --2- 172.21.15.32:0/954138675 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59280af090 0x7f5928153160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:51.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.923+0000 7f592d6eb700 1 -- 172.21.15.32:0/954138675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592813a5c0 con 0x7f5928063040 2020-12-05T17:39:51.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.923+0000 7f592e6ed700 1 --2- 172.21.15.32:0/954138675 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59280af090 0x7f5928153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:51.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.923+0000 7f592d6eb700 1 --2- 172.21.15.32:0/954138675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5928063040 0x7f59280b21a0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7f5918002f30 tx=0x7f5918015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:51.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.923+0000 7f5916ffd700 1 -- 172.21.15.32:0/954138675 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59180119a0 con 0x7f5928063040 2020-12-05T17:39:51.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.923+0000 7f5916ffd700 1 -- 172.21.15.32:0/954138675 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5918011b00 con 0x7f5928063040 2020-12-05T17:39:51.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.923+0000 7f592eeee700 1 -- 172.21.15.32:0/954138675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592813e0f0 con 0x7f5928063040 2020-12-05T17:39:51.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.923+0000 7f5916ffd700 1 -- 172.21.15.32:0/954138675 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5918011e60 con 0x7f5928063040 2020-12-05T17:39:51.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.923+0000 7f592eeee700 1 -- 172.21.15.32:0/954138675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5928153870 con 0x7f5928063040 2020-12-05T17:39:51.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.923+0000 7f592eeee700 1 -- 172.21.15.32:0/954138675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5928152e80 con 0x7f5928063040 2020-12-05T17:39:51.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.923+0000 7f5916ffd700 1 -- 172.21.15.32:0/954138675 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5918024030 con 0x7f5928063040 2020-12-05T17:39:51.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.927+0000 7f5916ffd700 1 --2- 172.21.15.32:0/954138675 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f59080554e0 0x7f59080579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:51.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.927+0000 7f5916ffd700 1 -- 172.21.15.32:0/954138675 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f591806ef20 con 0x7f5928063040 2020-12-05T17:39:51.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.927+0000 7f592deec700 1 --2- 172.21.15.32:0/954138675 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f59080554e0 0x7f59080579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:51.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.927+0000 7f592deec700 1 --2- 172.21.15.32:0/954138675 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f59080554e0 0x7f59080579a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f592400d000 tx=0x7f5924009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:51.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:51.935+0000 7f5916ffd700 1 -- 172.21.15.32:0/954138675 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f59180450c0 con 0x7f5928063040 2020-12-05T17:39:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.075+0000 7f592eeee700 1 -- 172.21.15.32:0/954138675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5928057050 con 0x7f5928063040 2020-12-05T17:39:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.075+0000 7f5916ffd700 1 -- 172.21.15.32:0/954138675 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) v1 ==== 104+0+10 (secure 0 0 0) 0x7f59180220a0 con 0x7f5928063040 2020-12-05T17:39:52.082 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:39:52.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.079+0000 7f592eeee700 1 -- 172.21.15.32:0/954138675 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f59080554e0 msgr2=0x7f59080579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:52.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.079+0000 7f592eeee700 1 --2- 172.21.15.32:0/954138675 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f59080554e0 0x7f59080579a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f592400d000 tx=0x7f5924009040).stop 2020-12-05T17:39:52.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.079+0000 7f592eeee700 1 -- 172.21.15.32:0/954138675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5928063040 msgr2=0x7f59280b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.079+0000 7f592eeee700 1 --2- 172.21.15.32:0/954138675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5928063040 0x7f59280b21a0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7f5918002f30 tx=0x7f5918015000).stop 2020-12-05T17:39:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.079+0000 7f592eeee700 1 -- 172.21.15.32:0/954138675 shutdown_connections 2020-12-05T17:39:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.079+0000 7f592eeee700 1 --2- 172.21.15.32:0/954138675 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59280af090 0x7f5928153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.079+0000 7f592eeee700 1 --2- 172.21.15.32:0/954138675 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f59080554e0 0x7f59080579a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.079+0000 7f592eeee700 1 --2- 172.21.15.32:0/954138675 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f592805f550 0x7f5928139cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.079+0000 7f592eeee700 1 --2- 172.21.15.32:0/954138675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5928063040 0x7f59280b21a0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.079+0000 7f592eeee700 1 -- 172.21.15.32:0/954138675 >> 172.21.15.32:0/954138675 conn(0x7f59280620b0 msgr2=0x7f5928062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.079+0000 7f592eeee700 1 -- 172.21.15.32:0/954138675 shutdown_connections 2020-12-05T17:39:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.079+0000 7f592eeee700 1 -- 172.21.15.32:0/954138675 wait complete. 2020-12-05T17:39:52.096 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:39:52.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.211+0000 7fb416194700 1 Processor -- start 2020-12-05T17:39:52.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.211+0000 7fb416194700 1 -- start start 2020-12-05T17:39:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.211+0000 7fb416194700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb41005feb0 0x7fb41005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.211+0000 7fb416194700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4100b2fb0 0x7fb410061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.211+0000 7fb416194700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4100b2df0 con 0x7fb41005f3b0 2020-12-05T17:39:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.211+0000 7fb416194700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4100b3c90 con 0x7fb4100b2fb0 2020-12-05T17:39:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.211+0000 7fb416194700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4100b2b50 con 0x7fb41005feb0 2020-12-05T17:39:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.211+0000 7fb415192700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb41005f3b0 0x7fb41013c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41514/0 (socket says 172.21.15.32:41514) 2020-12-05T17:39:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.211+0000 7fb415192700 1 -- 172.21.15.32:0/242788668 learned_addr learned my addr 172.21.15.32:0/242788668 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.211+0000 7fb415993700 1 --2- 172.21.15.32:0/242788668 >> v2:172.21.15.32:3300/0 conn(0x7fb4100b2fb0 0x7fb410061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.211+0000 7fb414991700 1 --2- 172.21.15.32:0/242788668 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb41005feb0 0x7fb41005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.211+0000 7fb40ffff700 1 -- 172.21.15.32:0/242788668 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1345926607 0 0) 0x7fb4100b2df0 con 0x7fb41005f3b0 2020-12-05T17:39:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.211+0000 7fb40ffff700 1 -- 172.21.15.32:0/242788668 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3fc003230 con 0x7fb41005f3b0 2020-12-05T17:39:52.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.211+0000 7fb40ffff700 1 -- 172.21.15.32:0/242788668 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2930690295 0 0) 0x7fb3fc003230 con 0x7fb41005f3b0 2020-12-05T17:39:52.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.211+0000 7fb415993700 1 -- 172.21.15.32:0/242788668 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb41005feb0 msgr2=0x7fb41005f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:52.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.211+0000 7fb415993700 1 --2- 172.21.15.32:0/242788668 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb41005feb0 0x7fb41005f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:52.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.211+0000 7fb415993700 1 -- 172.21.15.32:0/242788668 >> v1:172.21.15.32:6789/0 conn(0x7fb41005f3b0 legacy=0x7fb41013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:52.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.211+0000 7fb415993700 1 -- 172.21.15.32:0/242788668 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb410142820 con 0x7fb4100b2fb0 2020-12-05T17:39:52.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.211+0000 7fb415993700 1 --2- 172.21.15.32:0/242788668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4100b2fb0 0x7fb410061780 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7fb400001bf0 tx=0x7fb408006dc0).ready entity=mon.0 client_cookie=6936679d40c1404 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:52.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.211+0000 7fb40ffff700 1 -- 172.21.15.32:0/242788668 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb400001a50 con 0x7fb4100b2fb0 2020-12-05T17:39:52.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.211+0000 7fb40ffff700 1 -- 172.21.15.32:0/242788668 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40800d570 con 0x7fb4100b2fb0 2020-12-05T17:39:52.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.215+0000 7fb40ffff700 1 -- 172.21.15.32:0/242788668 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb40800d6e0 con 0x7fb4100b2fb0 2020-12-05T17:39:52.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.215+0000 7fb416194700 1 -- 172.21.15.32:0/242788668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4100b2fb0 msgr2=0x7fb410061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:52.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.215+0000 7fb416194700 1 --2- 172.21.15.32:0/242788668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4100b2fb0 0x7fb410061780 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7fb400001bf0 tx=0x7fb408006dc0).stop 2020-12-05T17:39:52.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.215+0000 7fb416194700 1 -- 172.21.15.32:0/242788668 shutdown_connections 2020-12-05T17:39:52.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.215+0000 7fb416194700 1 --2- 172.21.15.32:0/242788668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4100b2fb0 0x7fb410061780 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:52.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.215+0000 7fb416194700 1 --2- 172.21.15.32:0/242788668 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb41005feb0 0x7fb41005f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:52.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.215+0000 7fb416194700 1 -- 172.21.15.32:0/242788668 >> 172.21.15.32:0/242788668 conn(0x7fb4100b8f20 msgr2=0x7fb4100b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:52.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.215+0000 7fb416194700 1 -- 172.21.15.32:0/242788668 shutdown_connections 2020-12-05T17:39:52.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.215+0000 7fb416194700 1 -- 172.21.15.32:0/242788668 wait complete. 2020-12-05T17:39:52.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.215+0000 7fb416194700 1 Processor -- start 2020-12-05T17:39:52.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.215+0000 7fb416194700 1 -- start start 2020-12-05T17:39:52.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.219+0000 7fb416194700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb41005f3b0 0x7fb4101422c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:52.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.219+0000 7fb416194700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb41005feb0 0x7fb41013e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:52.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.219+0000 7fb416194700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4100b2fb0 0x7fb4101519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:52.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.219+0000 7fb416194700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4101515e0 con 0x7fb41005feb0 2020-12-05T17:39:52.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.219+0000 7fb416194700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4101527f0 con 0x7fb4100b2fb0 2020-12-05T17:39:52.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.219+0000 7fb416194700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb410141320 con 0x7fb41005f3b0 2020-12-05T17:39:52.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.219+0000 7fb414991700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb41005feb0 0x7fb41013e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:52.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.219+0000 7fb414991700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb41005feb0 0x7fb41013e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38874/0 (socket says 172.21.15.32:38874) 2020-12-05T17:39:52.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.219+0000 7fb414991700 1 -- 172.21.15.32:0/2000201375 learned_addr learned my addr 172.21.15.32:0/2000201375 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:52.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.219+0000 7fb415192700 1 --2- 172.21.15.32:0/2000201375 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb41005f3b0 0x7fb4101422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:52.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.219+0000 7fb415993700 1 --2- 172.21.15.32:0/2000201375 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4100b2fb0 0x7fb4101519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:52.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.219+0000 7fb414991700 1 -- 172.21.15.32:0/2000201375 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb41005f3b0 msgr2=0x7fb4101422c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:52.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.219+0000 7fb414991700 1 --2- 172.21.15.32:0/2000201375 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb41005f3b0 0x7fb4101422c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:52.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.219+0000 7fb414991700 1 -- 172.21.15.32:0/2000201375 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4100b2fb0 msgr2=0x7fb4101519a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:52.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.219+0000 7fb414991700 1 --2- 172.21.15.32:0/2000201375 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4100b2fb0 0x7fb4101519a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:52.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.219+0000 7fb414991700 1 -- 172.21.15.32:0/2000201375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4101469c0 con 0x7fb41005feb0 2020-12-05T17:39:52.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.219+0000 7fb414991700 1 --2- 172.21.15.32:0/2000201375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb41005feb0 0x7fb41013e900 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 rx=0x7fb40400c000 tx=0x7fb404006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:52.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.219+0000 7fb40e7fc700 1 -- 172.21.15.32:0/2000201375 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb40400ca10 con 0x7fb41005feb0 2020-12-05T17:39:52.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.219+0000 7fb40e7fc700 1 -- 172.21.15.32:0/2000201375 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40400cb70 con 0x7fb41005feb0 2020-12-05T17:39:52.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.219+0000 7fb416194700 1 -- 172.21.15.32:0/2000201375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb410151e90 con 0x7fb41005feb0 2020-12-05T17:39:52.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.219+0000 7fb40e7fc700 1 -- 172.21.15.32:0/2000201375 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4040123f0 con 0x7fb41005feb0 2020-12-05T17:39:52.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.219+0000 7fb416194700 1 -- 172.21.15.32:0/2000201375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb410152390 con 0x7fb41005feb0 2020-12-05T17:39:52.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.223+0000 7fb416194700 1 -- 172.21.15.32:0/2000201375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb410057050 con 0x7fb41005feb0 2020-12-05T17:39:52.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.223+0000 7fb40e7fc700 1 -- 172.21.15.32:0/2000201375 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb404012550 con 0x7fb41005feb0 2020-12-05T17:39:52.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.223+0000 7fb40e7fc700 1 --2- 172.21.15.32:0/2000201375 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3f0055440 0x7fb3f0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:52.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.223+0000 7fb40e7fc700 1 -- 172.21.15.32:0/2000201375 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb40406e4a0 con 0x7fb41005feb0 2020-12-05T17:39:52.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.223+0000 7fb415192700 1 --2- 172.21.15.32:0/2000201375 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3f0055440 0x7fb3f0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:52.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.223+0000 7fb415192700 1 --2- 172.21.15.32:0/2000201375 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3f0055440 0x7fb3f0057900 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7fb40000d000 tx=0x7fb400001600).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:52.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.231+0000 7fb40e7fc700 1 -- 172.21.15.32:0/2000201375 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb4040445e0 con 0x7fb41005feb0 2020-12-05T17:39:52.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.379+0000 7fb416194700 1 -- 172.21.15.32:0/2000201375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb4100b8d00 con 0x7fb41005feb0 2020-12-05T17:39:52.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.379+0000 7fb40e7fc700 1 -- 172.21.15.32:0/2000201375 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) v1 ==== 100+0+10 (secure 0 0 0) 0x7fb404047c00 con 0x7fb41005feb0 2020-12-05T17:39:52.387 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:39:52.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.383+0000 7fb416194700 1 -- 172.21.15.32:0/2000201375 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3f0055440 msgr2=0x7fb3f0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:52.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.383+0000 7fb416194700 1 --2- 172.21.15.32:0/2000201375 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3f0055440 0x7fb3f0057900 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7fb40000d000 tx=0x7fb400001600).stop 2020-12-05T17:39:52.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.383+0000 7fb416194700 1 -- 172.21.15.32:0/2000201375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb41005feb0 msgr2=0x7fb41013e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:52.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.383+0000 7fb416194700 1 --2- 172.21.15.32:0/2000201375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb41005feb0 0x7fb41013e900 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 rx=0x7fb40400c000 tx=0x7fb404006dc0).stop 2020-12-05T17:39:52.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.383+0000 7fb416194700 1 -- 172.21.15.32:0/2000201375 shutdown_connections 2020-12-05T17:39:52.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.383+0000 7fb416194700 1 --2- 172.21.15.32:0/2000201375 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4100b2fb0 0x7fb4101519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:52.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.383+0000 7fb416194700 1 --2- 172.21.15.32:0/2000201375 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3f0055440 0x7fb3f0057900 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:52.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.383+0000 7fb416194700 1 --2- 172.21.15.32:0/2000201375 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb41005f3b0 0x7fb4101422c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:52.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.383+0000 7fb416194700 1 --2- 172.21.15.32:0/2000201375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb41005feb0 0x7fb41013e900 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:52.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.383+0000 7fb416194700 1 -- 172.21.15.32:0/2000201375 >> 172.21.15.32:0/2000201375 conn(0x7fb4100b8f20 msgr2=0x7fb4100b7830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:52.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.383+0000 7fb416194700 1 -- 172.21.15.32:0/2000201375 shutdown_connections 2020-12-05T17:39:52.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.383+0000 7fb416194700 1 -- 172.21.15.32:0/2000201375 wait complete. 2020-12-05T17:39:52.402 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:39:52.403 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:39:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.511+0000 7ff0b759e700 1 Processor -- start 2020-12-05T17:39:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.511+0000 7ff0b759e700 1 -- start start 2020-12-05T17:39:52.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.511+0000 7ff0b759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0b805feb0 0x7ff0b805f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:52.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.511+0000 7ff0b759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b8142450 con 0x7ff0b805f3b0 2020-12-05T17:39:52.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.511+0000 7ff0b759e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b8061780 con 0x7ff0b8062400 2020-12-05T17:39:52.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.511+0000 7ff0b759e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b8061a20 con 0x7ff0b805feb0 2020-12-05T17:39:52.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.515+0000 7ff0b659c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff0b805f3b0 0x7ff0b80b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41528/0 (socket says 172.21.15.32:41528) 2020-12-05T17:39:52.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.515+0000 7ff0b659c700 1 -- 172.21.15.32:0/1029477958 learned_addr learned my addr 172.21.15.32:0/1029477958 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:52.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.515+0000 7ff0b6d9d700 1 --2- 172.21.15.32:0/1029477958 >> v2:172.21.15.32:3300/0 conn(0x7ff0b805feb0 0x7ff0b805f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.515+0000 7ff0b6d9d700 1 -- 172.21.15.32:0/1029477958 >> v1:172.21.15.61:6789/0 conn(0x7ff0b8062400 legacy=0x7ff0b80b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.515+0000 7ff0b6d9d700 1 -- 172.21.15.32:0/1029477958 >> v1:172.21.15.32:6789/0 conn(0x7ff0b805f3b0 legacy=0x7ff0b80b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.515+0000 7ff0b6d9d700 1 -- 172.21.15.32:0/1029477958 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b8061cc0 con 0x7ff0b805feb0 2020-12-05T17:39:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.515+0000 7ff0b6d9d700 1 --2- 172.21.15.32:0/1029477958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0b805feb0 0x7ff0b805f790 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7ff0b00032b0 tx=0x7ff0b00091a0).ready entity=mon.0 client_cookie=7d2ff21fb8c425e8 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.515+0000 7ff0b5d9b700 1 -- 172.21.15.32:0/1029477958 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0b000ba50 con 0x7ff0b805feb0 2020-12-05T17:39:52.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.515+0000 7ff0b5d9b700 1 -- 172.21.15.32:0/1029477958 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b000bbb0 con 0x7ff0b805feb0 2020-12-05T17:39:52.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.515+0000 7ff0b759e700 1 -- 172.21.15.32:0/1029477958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0b805feb0 msgr2=0x7ff0b805f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:52.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.515+0000 7ff0b5d9b700 1 -- 172.21.15.32:0/1029477958 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0b000bd20 con 0x7ff0b805feb0 2020-12-05T17:39:52.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.515+0000 7ff0b759e700 1 --2- 172.21.15.32:0/1029477958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0b805feb0 0x7ff0b805f790 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7ff0b00032b0 tx=0x7ff0b00091a0).stop 2020-12-05T17:39:52.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.515+0000 7ff0b759e700 1 -- 172.21.15.32:0/1029477958 shutdown_connections 2020-12-05T17:39:52.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.515+0000 7ff0b759e700 1 --2- 172.21.15.32:0/1029477958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0b805feb0 0x7ff0b805f790 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:52.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.515+0000 7ff0b759e700 1 -- 172.21.15.32:0/1029477958 >> 172.21.15.32:0/1029477958 conn(0x7ff0b80b8ee0 msgr2=0x7ff0b80b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:52.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.515+0000 7ff0b759e700 1 -- 172.21.15.32:0/1029477958 shutdown_connections 2020-12-05T17:39:52.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.515+0000 7ff0b759e700 1 -- 172.21.15.32:0/1029477958 wait complete. 2020-12-05T17:39:52.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.515+0000 7ff0b759e700 1 Processor -- start 2020-12-05T17:39:52.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0b759e700 1 -- start start 2020-12-05T17:39:52.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0b759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0b805f3b0 0x7ff0b8064210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:52.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0b759e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff0b805feb0 0x7ff0b813e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0b759e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff0b8062400 0x7ff0b8155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0b759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b8144540 con 0x7ff0b805f3b0 2020-12-05T17:39:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0b759e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b8064700 con 0x7ff0b805feb0 2020-12-05T17:39:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0b759e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b8143bd0 con 0x7ff0b8062400 2020-12-05T17:39:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0b659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0b805f3b0 0x7ff0b8064210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0b659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0b805f3b0 0x7ff0b8064210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38886/0 (socket says 172.21.15.32:38886) 2020-12-05T17:39:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0b659c700 1 -- 172.21.15.32:0/2140064073 learned_addr learned my addr 172.21.15.32:0/2140064073 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0b6d9d700 1 --2- 172.21.15.32:0/2140064073 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff0b8062400 0x7ff0b8155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0affff700 1 --2- 172.21.15.32:0/2140064073 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff0b805feb0 0x7ff0b813e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0b659c700 1 -- 172.21.15.32:0/2140064073 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff0b8062400 msgr2=0x7ff0b8155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0b659c700 1 --2- 172.21.15.32:0/2140064073 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff0b8062400 0x7ff0b8155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0b659c700 1 -- 172.21.15.32:0/2140064073 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff0b805feb0 msgr2=0x7ff0b813e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0b659c700 1 --2- 172.21.15.32:0/2140064073 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff0b805feb0 0x7ff0b813e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0b659c700 1 -- 172.21.15.32:0/2140064073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b8154dd0 con 0x7ff0b805f3b0 2020-12-05T17:39:52.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0affff700 1 --2- 172.21.15.32:0/2140064073 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff0b805feb0 0x7ff0b813e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:52.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0b659c700 1 --2- 172.21.15.32:0/2140064073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0b805f3b0 0x7ff0b8064210 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0x7ff0b80622e0 tx=0x7ff0a8007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:52.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0af7fe700 1 -- 172.21.15.32:0/2140064073 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0a80072e0 con 0x7ff0b805f3b0 2020-12-05T17:39:52.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0af7fe700 1 -- 172.21.15.32:0/2140064073 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a8007440 con 0x7ff0b805f3b0 2020-12-05T17:39:52.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0b759e700 1 -- 172.21.15.32:0/2140064073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b81442a0 con 0x7ff0b805f3b0 2020-12-05T17:39:52.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0af7fe700 1 -- 172.21.15.32:0/2140064073 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0a80057d0 con 0x7ff0b805f3b0 2020-12-05T17:39:52.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0b759e700 1 -- 172.21.15.32:0/2140064073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b8143e80 con 0x7ff0b805f3b0 2020-12-05T17:39:52.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0b759e700 1 -- 172.21.15.32:0/2140064073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b81440d0 con 0x7ff0b805f3b0 2020-12-05T17:39:52.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.519+0000 7ff0af7fe700 1 -- 172.21.15.32:0/2140064073 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff0a801a060 con 0x7ff0b805f3b0 2020-12-05T17:39:52.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.523+0000 7ff0af7fe700 1 --2- 172.21.15.32:0/2140064073 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff09c055440 0x7ff09c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:52.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.523+0000 7ff0af7fe700 1 -- 172.21.15.32:0/2140064073 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff0a806d770 con 0x7ff0b805f3b0 2020-12-05T17:39:52.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.523+0000 7ff0affff700 1 --2- 172.21.15.32:0/2140064073 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff09c055440 0x7ff09c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:52.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.523+0000 7ff0affff700 1 --2- 172.21.15.32:0/2140064073 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff09c055440 0x7ff09c057900 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7ff0a000b000 tx=0x7ff0a0009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:52.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.531+0000 7ff0af7fe700 1 -- 172.21.15.32:0/2140064073 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff0a8043910 con 0x7ff0b805f3b0 2020-12-05T17:39:52.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.719+0000 7ff0b759e700 1 -- 172.21.15.32:0/2140064073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0b8155a60 con 0x7ff0b805f3b0 2020-12-05T17:39:52.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.719+0000 7ff0af7fe700 1 -- 172.21.15.32:0/2140064073 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+2140 (secure 0 0 0) 0x7ff0a8018020 con 0x7ff0b805f3b0 2020-12-05T17:39:52.724 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:52.725 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":47,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/3759190037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3759190037},{"type":"v1","addr":"172.21.15.61:6833","nonce":3759190037}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":45}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:39:47.471195+0000","modified":"2020-12-05T17:39:48.473895+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7233":{"gid":7233,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4174090156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4174090156},{"type":"v1","addr":"172.21.15.32:6835","nonce":4174090156}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2020-12-05T17:39:52.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.719+0000 7ff0b759e700 1 -- 172.21.15.32:0/2140064073 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff09c055440 msgr2=0x7ff09c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:52.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.719+0000 7ff0b759e700 1 --2- 172.21.15.32:0/2140064073 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff09c055440 0x7ff09c057900 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7ff0a000b000 tx=0x7ff0a0009040).stop 2020-12-05T17:39:52.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.719+0000 7ff0b759e700 1 -- 172.21.15.32:0/2140064073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0b805f3b0 msgr2=0x7ff0b8064210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:52.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.719+0000 7ff0b759e700 1 --2- 172.21.15.32:0/2140064073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0b805f3b0 0x7ff0b8064210 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0x7ff0b80622e0 tx=0x7ff0a8007bd0).stop 2020-12-05T17:39:52.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.719+0000 7ff0b759e700 1 -- 172.21.15.32:0/2140064073 shutdown_connections 2020-12-05T17:39:52.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.719+0000 7ff0b759e700 1 --2- 172.21.15.32:0/2140064073 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff09c055440 0x7ff09c057900 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:52.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.719+0000 7ff0b759e700 1 --2- 172.21.15.32:0/2140064073 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff0b8062400 0x7ff0b8155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:52.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.719+0000 7ff0b759e700 1 --2- 172.21.15.32:0/2140064073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0b805f3b0 0x7ff0b8064210 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:52.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.719+0000 7ff0b759e700 1 --2- 172.21.15.32:0/2140064073 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff0b805feb0 0x7ff0b813e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:52.728 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.719+0000 7ff0b759e700 1 -- 172.21.15.32:0/2140064073 >> 172.21.15.32:0/2140064073 conn(0x7ff0b80b8ee0 msgr2=0x7ff0b80b6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:52.728 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.719+0000 7ff0b759e700 1 -- 172.21.15.32:0/2140064073 shutdown_connections 2020-12-05T17:39:52.728 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:52.719+0000 7ff0b759e700 1 -- 172.21.15.32:0/2140064073 wait complete. 2020-12-05T17:39:52.728 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 47 2020-12-05T17:39:52.940 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:39:47.471195+0000', 'modified': '2020-12-05T17:39:48.473895+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7233}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7233': {'gid': 7233, 'name': 'a', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/4174090156', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 4174090156}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 4174090156}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:39:52.940 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:39:52.941 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:39:53.070 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":7233,"want_state":"up:active","state":"up:active","rank_uptime":5.5850378230000004,"mdsmap_epoch":47,"osdmap_epoch":102,"osdmap_epoch_barrier":101,"uptime":10.821011686} 2020-12-05T17:39:53.071 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 7233, "mdsmap_epoch": 47, "osdmap_epoch": 102, "osdmap_epoch_barrier": 101, "rank_uptime": 5.585037823, "state": "up:active", "uptime": 10.821011686, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:39:53.071 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:39:53.072 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-12-05T17:39:53.072 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-12-05T17:39:53.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:53.073 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:53.081 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:39:53.139 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:39:53.140 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:39:53.193 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:39:53.194 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:39:53.195 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:53.195 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:39:53.242 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:39:53.242 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:53.286 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:39:53.302 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:39:53.303 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:39:53.359 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:39:53.360 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:39:53.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:53.361 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:39:53.364 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:2020-12-05T17:39:53.362+0000 7efdb28e40c0 -1 init, newargv = 0x55961eac0250 newargc=9ceph-fuse[ 2020-12-05T17:39:53.365 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:11344]: starting ceph client 2020-12-05T17:39:53.375 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[11344]: starting fuse 2020-12-05T17:39:54.472 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:39:54.488 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:39:54.489 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:39:54.545 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:39:54.545 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:39:54.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:54.547 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:39:54.594 INFO:teuthology.orchestra.run.smithi036.stdout:49 2020-12-05T17:39:54.594 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-12-05T17:39:54.595 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:54.595 INFO:teuthology.orchestra.run.smithi036:> sudo python3 -c ' 2020-12-05T17:39:54.595 INFO:teuthology.orchestra.run.smithi036:> import glob 2020-12-05T17:39:54.596 INFO:teuthology.orchestra.run.smithi036:> import re 2020-12-05T17:39:54.596 INFO:teuthology.orchestra.run.smithi036:> import os 2020-12-05T17:39:54.597 INFO:teuthology.orchestra.run.smithi036:> import subprocess 2020-12-05T17:39:54.597 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:39:54.597 INFO:teuthology.orchestra.run.smithi036:> def find_socket(client_name): 2020-12-05T17:39:54.598 INFO:teuthology.orchestra.run.smithi036:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-12-05T17:39:54.598 INFO:teuthology.orchestra.run.smithi036:> files = glob.glob(asok_path) 2020-12-05T17:39:54.598 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:39:54.599 INFO:teuthology.orchestra.run.smithi036:> # Given a non-glob path, it better be there 2020-12-05T17:39:54.599 INFO:teuthology.orchestra.run.smithi036:> if "*" not in asok_path: 2020-12-05T17:39:54.600 INFO:teuthology.orchestra.run.smithi036:> assert(len(files) == 1) 2020-12-05T17:39:54.600 INFO:teuthology.orchestra.run.smithi036:> return files[0] 2020-12-05T17:39:54.600 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:39:54.601 INFO:teuthology.orchestra.run.smithi036:> for f in files: 2020-12-05T17:39:54.601 INFO:teuthology.orchestra.run.smithi036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:39:54.601 INFO:teuthology.orchestra.run.smithi036:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:39:54.602 INFO:teuthology.orchestra.run.smithi036:> return f 2020-12-05T17:39:54.602 INFO:teuthology.orchestra.run.smithi036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:39:54.602 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:39:54.603 INFO:teuthology.orchestra.run.smithi036:> print(find_socket("client.0")) 2020-12-05T17:39:54.603 INFO:teuthology.orchestra.run.smithi036:> ' 2020-12-05T17:39:54.684 INFO:teuthology.orchestra.run.smithi036.stdout:/var/run/ceph/ceph-client.0.11344.asok 2020-12-05T17:39:54.690 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.11344.asok 2020-12-05T17:39:54.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:54.691 INFO:teuthology.orchestra.run.smithi036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.11344.asok status 2020-12-05T17:39:54.789 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2020-12-05T17:39:54.790 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": { 2020-12-05T17:39:54.790 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:39:54.791 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:39:54.791 INFO:teuthology.orchestra.run.smithi036.stdout: "entity_id": "0", 2020-12-05T17:39:54.791 INFO:teuthology.orchestra.run.smithi036.stdout: "hostname": "smithi036", 2020-12-05T17:39:54.792 INFO:teuthology.orchestra.run.smithi036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-12-05T17:39:54.792 INFO:teuthology.orchestra.run.smithi036.stdout: "pid": "11344", 2020-12-05T17:39:54.793 INFO:teuthology.orchestra.run.smithi036.stdout: "root": "/" 2020-12-05T17:39:54.793 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:39:54.793 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_count": 0, 2020-12-05T17:39:54.794 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_pinned_count": 0, 2020-12-05T17:39:54.794 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 7404, 2020-12-05T17:39:54.795 INFO:teuthology.orchestra.run.smithi036.stdout: "inst": { 2020-12-05T17:39:54.795 INFO:teuthology.orchestra.run.smithi036.stdout: "name": { 2020-12-05T17:39:54.795 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "client", 2020-12-05T17:39:54.796 INFO:teuthology.orchestra.run.smithi036.stdout: "num": 7404 2020-12-05T17:39:54.796 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:39:54.797 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:39:54.797 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:39:54.797 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:39:54.798 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 1901607223 2020-12-05T17:39:54.798 INFO:teuthology.orchestra.run.smithi036.stdout: } 2020-12-05T17:39:54.799 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:39:54.799 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:39:54.799 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:39:54.800 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:39:54.800 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 1901607223 2020-12-05T17:39:54.801 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:39:54.801 INFO:teuthology.orchestra.run.smithi036.stdout: "inst_str": "client.7404 172.21.15.36:0/1901607223", 2020-12-05T17:39:54.801 INFO:teuthology.orchestra.run.smithi036.stdout: "addr_str": "172.21.15.36:0/1901607223", 2020-12-05T17:39:54.802 INFO:teuthology.orchestra.run.smithi036.stdout: "inode_count": 1, 2020-12-05T17:39:54.802 INFO:teuthology.orchestra.run.smithi036.stdout: "mds_epoch": 47, 2020-12-05T17:39:54.803 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch": 102, 2020-12-05T17:39:54.803 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:39:54.803 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklisted": false 2020-12-05T17:39:54.804 INFO:teuthology.orchestra.run.smithi036.stdout:} 2020-12-05T17:39:54.805 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:54.806 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:54.807 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:54.815 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:39:54.816 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:39:54.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:54.817 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:54.877 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:39:55.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.995+0000 7f5889543700 1 Processor -- start 2020-12-05T17:39:55.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.995+0000 7f5889543700 1 -- start start 2020-12-05T17:39:55.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.995+0000 7f5889543700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f588405f3b0 0x7f588405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.995+0000 7f5889543700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f588405feb0 0x7f58840b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.995+0000 7f5889543700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5884142450 con 0x7f5884062400 2020-12-05T17:39:55.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.995+0000 7f5889543700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58840b42f0 con 0x7f588405feb0 2020-12-05T17:39:55.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.995+0000 7f5889543700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5884061780 con 0x7f588405f3b0 2020-12-05T17:39:55.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.995+0000 7f5888d42700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f588405feb0 0x7f58840b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.995+0000 7f5888d42700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f588405feb0 0x7f58840b3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38898/0 (socket says 172.21.15.32:38898) 2020-12-05T17:39:55.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.995+0000 7f5888d42700 1 -- 172.21.15.32:0/1769103129 learned_addr learned my addr 172.21.15.32:0/1769103129 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:55.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.995+0000 7f5883fff700 1 --2- 172.21.15.32:0/1769103129 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f588405f3b0 0x7f588405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.999+0000 7f5888d42700 1 -- 172.21.15.32:0/1769103129 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f588405f3b0 msgr2=0x7f588405f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:55.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.999+0000 7f5888d42700 1 --2- 172.21.15.32:0/1769103129 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f588405f3b0 0x7f588405f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.999+0000 7f5888d42700 1 -- 172.21.15.32:0/1769103129 >> v1:172.21.15.61:6789/0 conn(0x7f5884062400 legacy=0x7f58840b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.999+0000 7f5888d42700 1 -- 172.21.15.32:0/1769103129 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5884061a20 con 0x7f588405feb0 2020-12-05T17:39:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.999+0000 7f5888d42700 1 --2- 172.21.15.32:0/1769103129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f588405feb0 0x7f58840b3e00 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7f587c0032b0 tx=0x7f587c009270).ready entity=mon.0 client_cookie=2adc571c2445315b server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.999+0000 7f58837fe700 1 -- 172.21.15.32:0/1769103129 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f587c009e60 con 0x7f588405feb0 2020-12-05T17:39:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.999+0000 7f58837fe700 1 -- 172.21.15.32:0/1769103129 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587c00bc00 con 0x7f588405feb0 2020-12-05T17:39:55.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.999+0000 7f5889543700 1 -- 172.21.15.32:0/1769103129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f588405feb0 msgr2=0x7f58840b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:55.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.999+0000 7f5889543700 1 --2- 172.21.15.32:0/1769103129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f588405feb0 0x7f58840b3e00 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7f587c0032b0 tx=0x7f587c009270).stop 2020-12-05T17:39:55.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.999+0000 7f58837fe700 1 -- 172.21.15.32:0/1769103129 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f587c00bd70 con 0x7f588405feb0 2020-12-05T17:39:55.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.999+0000 7f5889543700 1 -- 172.21.15.32:0/1769103129 shutdown_connections 2020-12-05T17:39:55.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.999+0000 7f5889543700 1 --2- 172.21.15.32:0/1769103129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f588405feb0 0x7f58840b3e00 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.999+0000 7f5889543700 1 --2- 172.21.15.32:0/1769103129 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f588405f3b0 0x7f588405f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.999+0000 7f5889543700 1 -- 172.21.15.32:0/1769103129 >> 172.21.15.32:0/1769103129 conn(0x7f58840b8ee0 msgr2=0x7f58840b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:55.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.999+0000 7f5889543700 1 -- 172.21.15.32:0/1769103129 shutdown_connections 2020-12-05T17:39:55.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.999+0000 7f5889543700 1 -- 172.21.15.32:0/1769103129 wait complete. 2020-12-05T17:39:55.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.999+0000 7f5889543700 1 Processor -- start 2020-12-05T17:39:55.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:54.999+0000 7f5889543700 1 -- start start 2020-12-05T17:39:55.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f5889543700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f588405f3b0 0x7f588413f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f5889543700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f588405feb0 0x7f58840b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f5889543700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5884062400 0x7f58841512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f5889543700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58840b2010 con 0x7f5884062400 2020-12-05T17:39:55.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f5889543700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58840b1140 con 0x7f588405f3b0 2020-12-05T17:39:55.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f5889543700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58840b13e0 con 0x7f588405feb0 2020-12-05T17:39:55.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f5888d42700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5884062400 0x7f58841512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f5888d42700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5884062400 0x7f58841512f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38904/0 (socket says 172.21.15.32:38904) 2020-12-05T17:39:55.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f5888d42700 1 -- 172.21.15.32:0/586425786 learned_addr learned my addr 172.21.15.32:0/586425786 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:55.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f587b7fe700 1 --2- 172.21.15.32:0/586425786 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f588405feb0 0x7f58840b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f5883fff700 1 --2- 172.21.15.32:0/586425786 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f588405f3b0 0x7f588413f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f587b7fe700 1 -- 172.21.15.32:0/586425786 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f588405f3b0 msgr2=0x7f588413f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:55.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f587b7fe700 1 --2- 172.21.15.32:0/586425786 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f588405f3b0 0x7f588413f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f587b7fe700 1 -- 172.21.15.32:0/586425786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5884062400 msgr2=0x7f58841512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:55.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f587b7fe700 1 --2- 172.21.15.32:0/586425786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5884062400 0x7f58841512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f587b7fe700 1 -- 172.21.15.32:0/586425786 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58840b1680 con 0x7f588405feb0 2020-12-05T17:39:55.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f5888d42700 1 --2- 172.21.15.32:0/586425786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5884062400 0x7f58841512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:55.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f587b7fe700 1 --2- 172.21.15.32:0/586425786 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f588405feb0 0x7f58840b1b20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f588405c5c0 tx=0x7f586c007b60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:55.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f5881ffb700 1 -- 172.21.15.32:0/586425786 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f586c013410 con 0x7f588405feb0 2020-12-05T17:39:55.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f5889543700 1 -- 172.21.15.32:0/586425786 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5884140190 con 0x7f588405feb0 2020-12-05T17:39:55.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f5889543700 1 -- 172.21.15.32:0/586425786 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5884063bf0 con 0x7f588405feb0 2020-12-05T17:39:55.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f5881ffb700 1 -- 172.21.15.32:0/586425786 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586c013570 con 0x7f588405feb0 2020-12-05T17:39:55.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f5881ffb700 1 -- 172.21.15.32:0/586425786 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f586c013720 con 0x7f588405feb0 2020-12-05T17:39:55.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.003+0000 7f5889543700 1 -- 172.21.15.32:0/586425786 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58840b2ed0 con 0x7f588405feb0 2020-12-05T17:39:55.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.007+0000 7f5881ffb700 1 -- 172.21.15.32:0/586425786 <== mon.2 v2:172.21.15.61:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f586c019020 con 0x7f588405feb0 2020-12-05T17:39:55.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.007+0000 7f5881ffb700 1 --2- 172.21.15.32:0/586425786 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f58680554e0 0x7f58680579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.007+0000 7f5881ffb700 1 -- 172.21.15.32:0/586425786 <== mon.2 v2:172.21.15.61:3301/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f586c06e040 con 0x7f588405feb0 2020-12-05T17:39:55.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.007+0000 7f5883fff700 1 --2- 172.21.15.32:0/586425786 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f58680554e0 0x7f58680579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.007+0000 7f5883fff700 1 --2- 172.21.15.32:0/586425786 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f58680554e0 0x7f58680579a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f587400b000 tx=0x7f5874006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:55.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.019+0000 7f5881ffb700 1 -- 172.21.15.32:0/586425786 <== mon.2 v2:172.21.15.61:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f586c0440d0 con 0x7f588405feb0 2020-12-05T17:39:55.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.167+0000 7f5889543700 1 -- 172.21.15.32:0/586425786 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58840b1810 con 0x7f588405feb0 2020-12-05T17:39:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.167+0000 7f5881ffb700 1 -- 172.21.15.32:0/586425786 <== mon.2 v2:172.21.15.61:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16282 (secure 0 0 0) 0x7f586c0476f0 con 0x7f588405feb0 2020-12-05T17:39:55.174 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:55.174 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":102,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:39:48.474575+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:39:45.190638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2020-12-05T17:39:46.472081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:39:55.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.171+0000 7f5889543700 1 -- 172.21.15.32:0/586425786 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f58680554e0 msgr2=0x7f58680579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:55.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.171+0000 7f5889543700 1 --2- 172.21.15.32:0/586425786 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f58680554e0 0x7f58680579a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f587400b000 tx=0x7f5874006dc0).stop 2020-12-05T17:39:55.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.171+0000 7f5889543700 1 -- 172.21.15.32:0/586425786 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f588405feb0 msgr2=0x7f58840b1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:55.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.171+0000 7f5889543700 1 --2- 172.21.15.32:0/586425786 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f588405feb0 0x7f58840b1b20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f588405c5c0 tx=0x7f586c007b60).stop 2020-12-05T17:39:55.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.171+0000 7f5889543700 1 -- 172.21.15.32:0/586425786 shutdown_connections 2020-12-05T17:39:55.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.171+0000 7f5889543700 1 --2- 172.21.15.32:0/586425786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5884062400 0x7f58841512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.171+0000 7f5889543700 1 --2- 172.21.15.32:0/586425786 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f588405f3b0 0x7f588413f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.171+0000 7f5889543700 1 --2- 172.21.15.32:0/586425786 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f58680554e0 0x7f58680579a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.171+0000 7f5889543700 1 --2- 172.21.15.32:0/586425786 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f588405feb0 0x7f58840b1b20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.171+0000 7f5889543700 1 -- 172.21.15.32:0/586425786 >> 172.21.15.32:0/586425786 conn(0x7f58840b8ee0 msgr2=0x7f5884142e00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.171+0000 7f5889543700 1 -- 172.21.15.32:0/586425786 shutdown_connections 2020-12-05T17:39:55.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.171+0000 7f5889543700 1 -- 172.21.15.32:0/586425786 wait complete. 2020-12-05T17:39:55.188 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:39:45.190638+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:39:46.472081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:39:55.189 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:39:55.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce3fff700 1 Processor -- start 2020-12-05T17:39:55.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce3fff700 1 -- start start 2020-12-05T17:39:55.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ce40b2000 0x7f0ce4145e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce3fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0ce4061780 0x7f0ce40afe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce3fff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce40b23e0 con 0x7f0ce40af4b0 2020-12-05T17:39:55.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce3fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce4146a00 con 0x7f0ce40b2000 2020-12-05T17:39:55.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce3fff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce40ae7b0 con 0x7f0ce4061780 2020-12-05T17:39:55.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ce40b2000 0x7f0ce4145e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ce40b2000 0x7f0ce4145e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38908/0 (socket says 172.21.15.32:38908) 2020-12-05T17:39:55.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce2ffd700 1 -- 172.21.15.32:0/1676651995 learned_addr learned my addr 172.21.15.32:0/1676651995 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:55.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce37fe700 1 --2- 172.21.15.32:0/1676651995 >> v2:172.21.15.61:3300/0 conn(0x7f0ce4061780 0x7f0ce40afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce2ffd700 1 -- 172.21.15.32:0/1676651995 >> v2:172.21.15.61:3300/0 conn(0x7f0ce4061780 msgr2=0x7f0ce40afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:55.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce2ffd700 1 --2- 172.21.15.32:0/1676651995 >> v2:172.21.15.61:3300/0 conn(0x7f0ce4061780 0x7f0ce40afe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce2ffd700 1 -- 172.21.15.32:0/1676651995 >> v1:172.21.15.61:6789/0 conn(0x7f0ce40af4b0 legacy=0x7f0ce41462f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:55.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce2ffd700 1 -- 172.21.15.32:0/1676651995 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce40aea50 con 0x7f0ce40b2000 2020-12-05T17:39:55.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce2ffd700 1 --2- 172.21.15.32:0/1676651995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce40b2000 0x7f0ce4145e00 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0x7f0cd8002f30 tx=0x7f0cd8006e80).ready entity=mon.0 client_cookie=76d653eb86e786b7 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:55.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce1ffb700 1 -- 172.21.15.32:0/1676651995 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cd80097f0 con 0x7f0ce40b2000 2020-12-05T17:39:55.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce1ffb700 1 -- 172.21.15.32:0/1676651995 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd8009950 con 0x7f0ce40b2000 2020-12-05T17:39:55.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce1ffb700 1 -- 172.21.15.32:0/1676651995 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cd8009cb0 con 0x7f0ce40b2000 2020-12-05T17:39:55.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce3fff700 1 -- 172.21.15.32:0/1676651995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce40b2000 msgr2=0x7f0ce4145e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:55.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce3fff700 1 --2- 172.21.15.32:0/1676651995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce40b2000 0x7f0ce4145e00 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0x7f0cd8002f30 tx=0x7f0cd8006e80).stop 2020-12-05T17:39:55.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce3fff700 1 -- 172.21.15.32:0/1676651995 shutdown_connections 2020-12-05T17:39:55.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce3fff700 1 --2- 172.21.15.32:0/1676651995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce40b2000 0x7f0ce4145e00 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce3fff700 1 --2- 172.21.15.32:0/1676651995 >> v2:172.21.15.61:3300/0 conn(0x7f0ce4061780 0x7f0ce40afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce3fff700 1 -- 172.21.15.32:0/1676651995 >> 172.21.15.32:0/1676651995 conn(0x7f0ce40b5620 msgr2=0x7f0ce40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce3fff700 1 -- 172.21.15.32:0/1676651995 shutdown_connections 2020-12-05T17:39:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.303+0000 7f0ce3fff700 1 -- 172.21.15.32:0/1676651995 wait complete. 2020-12-05T17:39:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0ce3fff700 1 Processor -- start 2020-12-05T17:39:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0ce3fff700 1 -- start start 2020-12-05T17:39:55.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0ce3fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ce4061780 0x7f0ce40658d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0ce3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce40af4b0 0x7f0ce4065dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0ce3fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ce40b2000 0x7f0ce4111400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0ce3fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce410ec50 con 0x7f0ce40af4b0 2020-12-05T17:39:55.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0ce3fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce4063e50 con 0x7f0ce4061780 2020-12-05T17:39:55.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0ce3fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce410ee80 con 0x7f0ce40b2000 2020-12-05T17:39:55.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0ce27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce40af4b0 0x7f0ce4065dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0ce27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce40af4b0 0x7f0ce4065dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38916/0 (socket says 172.21.15.32:38916) 2020-12-05T17:39:55.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0ce27fc700 1 -- 172.21.15.32:0/3754620681 learned_addr learned my addr 172.21.15.32:0/3754620681 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:55.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0ce2ffd700 1 --2- 172.21.15.32:0/3754620681 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ce4061780 0x7f0ce40658d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0ce37fe700 1 --2- 172.21.15.32:0/3754620681 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ce40b2000 0x7f0ce4111400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0ce27fc700 1 -- 172.21.15.32:0/3754620681 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ce40b2000 msgr2=0x7f0ce4111400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:55.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0ce27fc700 1 --2- 172.21.15.32:0/3754620681 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ce40b2000 0x7f0ce4111400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0ce27fc700 1 -- 172.21.15.32:0/3754620681 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ce4061780 msgr2=0x7f0ce40658d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:55.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0ce27fc700 1 --2- 172.21.15.32:0/3754620681 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ce4061780 0x7f0ce40658d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0ce27fc700 1 -- 172.21.15.32:0/3754620681 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce410f120 con 0x7f0ce40af4b0 2020-12-05T17:39:55.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0ce37fe700 1 --2- 172.21.15.32:0/3754620681 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ce40b2000 0x7f0ce4111400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:55.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0ce27fc700 1 --2- 172.21.15.32:0/3754620681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce40af4b0 0x7f0ce4065dc0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7f0ce40adba0 tx=0x7f0cd4007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:55.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0cd3fff700 1 -- 172.21.15.32:0/3754620681 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cd400e4a0 con 0x7f0ce40af4b0 2020-12-05T17:39:55.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0cd3fff700 1 -- 172.21.15.32:0/3754620681 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd400e600 con 0x7f0ce40af4b0 2020-12-05T17:39:55.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0cd3fff700 1 -- 172.21.15.32:0/3754620681 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cd400e960 con 0x7f0ce40af4b0 2020-12-05T17:39:55.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0ce3fff700 1 -- 172.21.15.32:0/3754620681 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce4064080 con 0x7f0ce40af4b0 2020-12-05T17:39:55.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.307+0000 7f0ce3fff700 1 -- 172.21.15.32:0/3754620681 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce4065070 con 0x7f0ce40af4b0 2020-12-05T17:39:55.323 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.311+0000 7f0ce3fff700 1 -- 172.21.15.32:0/3754620681 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce4064210 con 0x7f0ce40af4b0 2020-12-05T17:39:55.324 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.311+0000 7f0cd3fff700 1 -- 172.21.15.32:0/3754620681 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0cd4010030 con 0x7f0ce40af4b0 2020-12-05T17:39:55.324 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.311+0000 7f0cd3fff700 1 --2- 172.21.15.32:0/3754620681 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0cc0055470 0x7f0cc0057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.324 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.311+0000 7f0cd3fff700 1 -- 172.21.15.32:0/3754620681 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0cd406dec0 con 0x7f0ce40af4b0 2020-12-05T17:39:55.324 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.311+0000 7f0ce2ffd700 1 --2- 172.21.15.32:0/3754620681 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0cc0055470 0x7f0cc0057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.311+0000 7f0ce2ffd700 1 --2- 172.21.15.32:0/3754620681 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0cc0055470 0x7f0cc0057930 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f0cd801c000 tx=0x7f0cd8015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:55.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.319+0000 7f0cd3fff700 1 -- 172.21.15.32:0/3754620681 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0cd4044060 con 0x7f0ce40af4b0 2020-12-05T17:39:55.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.467+0000 7f0ce3fff700 1 -- 172.21.15.32:0/3754620681 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0ce4057050 con 0x7f0ce40af4b0 2020-12-05T17:39:55.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.467+0000 7f0cd3fff700 1 -- 172.21.15.32:0/3754620681 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v102) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0cd4047680 con 0x7f0ce40af4b0 2020-12-05T17:39:55.472 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:39:55.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.467+0000 7f0ce3fff700 1 -- 172.21.15.32:0/3754620681 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0cc0055470 msgr2=0x7f0cc0057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:55.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.467+0000 7f0ce3fff700 1 --2- 172.21.15.32:0/3754620681 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0cc0055470 0x7f0cc0057930 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f0cd801c000 tx=0x7f0cd8015000).stop 2020-12-05T17:39:55.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.467+0000 7f0ce3fff700 1 -- 172.21.15.32:0/3754620681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce40af4b0 msgr2=0x7f0ce4065dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:55.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.467+0000 7f0ce3fff700 1 --2- 172.21.15.32:0/3754620681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce40af4b0 0x7f0ce4065dc0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7f0ce40adba0 tx=0x7f0cd4007c20).stop 2020-12-05T17:39:55.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.467+0000 7f0ce3fff700 1 -- 172.21.15.32:0/3754620681 shutdown_connections 2020-12-05T17:39:55.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.467+0000 7f0ce3fff700 1 --2- 172.21.15.32:0/3754620681 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0cc0055470 0x7f0cc0057930 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.467+0000 7f0ce3fff700 1 --2- 172.21.15.32:0/3754620681 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ce40b2000 0x7f0ce4111400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.467+0000 7f0ce3fff700 1 --2- 172.21.15.32:0/3754620681 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ce4061780 0x7f0ce40658d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.467+0000 7f0ce3fff700 1 --2- 172.21.15.32:0/3754620681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce40af4b0 0x7f0ce4065dc0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.467+0000 7f0ce3fff700 1 -- 172.21.15.32:0/3754620681 >> 172.21.15.32:0/3754620681 conn(0x7f0ce40b5620 msgr2=0x7f0ce40afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:55.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.467+0000 7f0ce3fff700 1 -- 172.21.15.32:0/3754620681 shutdown_connections 2020-12-05T17:39:55.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.467+0000 7f0ce3fff700 1 -- 172.21.15.32:0/3754620681 wait complete. 2020-12-05T17:39:55.486 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:39:55.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f1594e1d700 1 Processor -- start 2020-12-05T17:39:55.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f1594e1d700 1 -- start start 2020-12-05T17:39:55.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f1594e1d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f159005f6f0 0x7f159005fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f1594e1d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1590061b30 0x7f15900b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f1594e1d700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15900afe80 con 0x7f15900b2b50 2020-12-05T17:39:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f1594e1d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15900b1a70 con 0x7f1590061b30 2020-12-05T17:39:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f1594e1d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15900b16c0 con 0x7f159005f6f0 2020-12-05T17:39:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f158effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1590061b30 0x7f15900b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f158effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1590061b30 0x7f15900b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38924/0 (socket says 172.21.15.32:38924) 2020-12-05T17:39:55.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f158effd700 1 -- 172.21.15.32:0/1864020312 learned_addr learned my addr 172.21.15.32:0/1864020312 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:55.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f158f7fe700 1 --2- 172.21.15.32:0/1864020312 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f159005f6f0 0x7f159005fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f158effd700 1 -- 172.21.15.32:0/1864020312 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f159005f6f0 msgr2=0x7f159005fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:55.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f158effd700 1 --2- 172.21.15.32:0/1864020312 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f159005f6f0 0x7f159005fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f158effd700 1 -- 172.21.15.32:0/1864020312 >> v1:172.21.15.61:6789/0 conn(0x7f15900b2b50 legacy=0x7f1590145ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f158effd700 1 -- 172.21.15.32:0/1864020312 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159014ad00 con 0x7f1590061b30 2020-12-05T17:39:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f158f7fe700 1 --2- 172.21.15.32:0/1864020312 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f159005f6f0 0x7f159005fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f158effd700 1 --2- 172.21.15.32:0/1864020312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1590061b30 0x7f15900b2660 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7f15900b0270 tx=0x7f1584006e80).ready entity=mon.0 client_cookie=b4becf2e632f7588 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f158e7fc700 1 -- 172.21.15.32:0/1864020312 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15840097b0 con 0x7f1590061b30 2020-12-05T17:39:55.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f158e7fc700 1 -- 172.21.15.32:0/1864020312 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1584009910 con 0x7f1590061b30 2020-12-05T17:39:55.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f158e7fc700 1 -- 172.21.15.32:0/1864020312 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1584009c70 con 0x7f1590061b30 2020-12-05T17:39:55.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f1594e1d700 1 -- 172.21.15.32:0/1864020312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1590061b30 msgr2=0x7f15900b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:55.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f1594e1d700 1 --2- 172.21.15.32:0/1864020312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1590061b30 0x7f15900b2660 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7f15900b0270 tx=0x7f1584006e80).stop 2020-12-05T17:39:55.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f1594e1d700 1 -- 172.21.15.32:0/1864020312 shutdown_connections 2020-12-05T17:39:55.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f1594e1d700 1 --2- 172.21.15.32:0/1864020312 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f159005f6f0 0x7f159005fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f1594e1d700 1 --2- 172.21.15.32:0/1864020312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1590061b30 0x7f15900b2660 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f1594e1d700 1 -- 172.21.15.32:0/1864020312 >> 172.21.15.32:0/1864020312 conn(0x7f1590061010 msgr2=0x7f1590061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:55.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f1594e1d700 1 -- 172.21.15.32:0/1864020312 shutdown_connections 2020-12-05T17:39:55.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.595+0000 7f1594e1d700 1 -- 172.21.15.32:0/1864020312 wait complete. 2020-12-05T17:39:55.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f1594e1d700 1 Processor -- start 2020-12-05T17:39:55.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f1594e1d700 1 -- start start 2020-12-05T17:39:55.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f1594e1d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f159005f6f0 0x7f15900b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f1594e1d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1590061b30 0x7f1590111060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f1594e1d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15900b2b50 0x7f1590112340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f1594e1d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1590065110 con 0x7f1590061b30 2020-12-05T17:39:55.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f1594e1d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1590066300 con 0x7f15900b2b50 2020-12-05T17:39:55.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f1594e1d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1590065340 con 0x7f159005f6f0 2020-12-05T17:39:55.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f158effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1590061b30 0x7f1590111060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f158effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1590061b30 0x7f1590111060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38930/0 (socket says 172.21.15.32:38930) 2020-12-05T17:39:55.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f158effd700 1 -- 172.21.15.32:0/2671405525 learned_addr learned my addr 172.21.15.32:0/2671405525 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:55.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f158f7fe700 1 --2- 172.21.15.32:0/2671405525 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f159005f6f0 0x7f15900b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f158ffff700 1 --2- 172.21.15.32:0/2671405525 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15900b2b50 0x7f1590112340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f158effd700 1 -- 172.21.15.32:0/2671405525 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f159005f6f0 msgr2=0x7f15900b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:55.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f158effd700 1 --2- 172.21.15.32:0/2671405525 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f159005f6f0 0x7f15900b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f158effd700 1 -- 172.21.15.32:0/2671405525 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15900b2b50 msgr2=0x7f1590112340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:55.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f158effd700 1 --2- 172.21.15.32:0/2671405525 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15900b2b50 0x7f1590112340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f158effd700 1 -- 172.21.15.32:0/2671405525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1590066060 con 0x7f1590061b30 2020-12-05T17:39:55.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f158ffff700 1 --2- 172.21.15.32:0/2671405525 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15900b2b50 0x7f1590112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:55.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f158effd700 1 --2- 172.21.15.32:0/2671405525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1590061b30 0x7f1590111060 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7f1590066490 tx=0x7f1584015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:55.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f158cff9700 1 -- 172.21.15.32:0/2671405525 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1584011c20 con 0x7f1590061b30 2020-12-05T17:39:55.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f158cff9700 1 -- 172.21.15.32:0/2671405525 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1584011d80 con 0x7f1590061b30 2020-12-05T17:39:55.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f1594e1d700 1 -- 172.21.15.32:0/2671405525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1590064230 con 0x7f1590061b30 2020-12-05T17:39:55.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f158cff9700 1 -- 172.21.15.32:0/2671405525 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f158401e650 con 0x7f1590061b30 2020-12-05T17:39:55.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f1594e1d700 1 -- 172.21.15.32:0/2671405525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159010f7f0 con 0x7f1590061b30 2020-12-05T17:39:55.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.599+0000 7f1594e1d700 1 -- 172.21.15.32:0/2671405525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1590057050 con 0x7f1590061b30 2020-12-05T17:39:55.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.603+0000 7f158cff9700 1 -- 172.21.15.32:0/2671405525 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1584025030 con 0x7f1590061b30 2020-12-05T17:39:55.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.603+0000 7f158cff9700 1 --2- 172.21.15.32:0/2671405525 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15700554e0 0x7f15700579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.603+0000 7f158cff9700 1 -- 172.21.15.32:0/2671405525 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f158406f610 con 0x7f1590061b30 2020-12-05T17:39:55.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.603+0000 7f158f7fe700 1 --2- 172.21.15.32:0/2671405525 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15700554e0 0x7f15700579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.603+0000 7f158f7fe700 1 --2- 172.21.15.32:0/2671405525 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15700554e0 0x7f15700579a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f158000c000 tx=0x7f1580008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:55.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.607+0000 7f158cff9700 1 -- 172.21.15.32:0/2671405525 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f15840456a0 con 0x7f1590061b30 2020-12-05T17:39:55.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.747+0000 7f1594e1d700 1 -- 172.21.15.32:0/2671405525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1590069690 con 0x7f1590061b30 2020-12-05T17:39:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.747+0000 7f158cff9700 1 -- 172.21.15.32:0/2671405525 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v102) v1 ==== 92+0+10 (secure 0 0 0) 0x7f15840230a0 con 0x7f1590061b30 2020-12-05T17:39:55.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.747+0000 7f1594e1d700 1 -- 172.21.15.32:0/2671405525 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15700554e0 msgr2=0x7f15700579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:55.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.747+0000 7f1594e1d700 1 --2- 172.21.15.32:0/2671405525 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15700554e0 0x7f15700579a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f158000c000 tx=0x7f1580008040).stop 2020-12-05T17:39:55.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.747+0000 7f1594e1d700 1 -- 172.21.15.32:0/2671405525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1590061b30 msgr2=0x7f1590111060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:55.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.747+0000 7f1594e1d700 1 --2- 172.21.15.32:0/2671405525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1590061b30 0x7f1590111060 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7f1590066490 tx=0x7f1584015000).stop 2020-12-05T17:39:55.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.747+0000 7f1594e1d700 1 -- 172.21.15.32:0/2671405525 shutdown_connections 2020-12-05T17:39:55.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.747+0000 7f1594e1d700 1 --2- 172.21.15.32:0/2671405525 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15900b2b50 0x7f1590112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.747+0000 7f1594e1d700 1 --2- 172.21.15.32:0/2671405525 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15700554e0 0x7f15700579a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.747+0000 7f1594e1d700 1 --2- 172.21.15.32:0/2671405525 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f159005f6f0 0x7f15900b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.747+0000 7f1594e1d700 1 --2- 172.21.15.32:0/2671405525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1590061b30 0x7f1590111060 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.747+0000 7f1594e1d700 1 -- 172.21.15.32:0/2671405525 >> 172.21.15.32:0/2671405525 conn(0x7f1590061010 msgr2=0x7f15900af200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:55.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.747+0000 7f1594e1d700 1 -- 172.21.15.32:0/2671405525 shutdown_connections 2020-12-05T17:39:55.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.747+0000 7f1594e1d700 1 -- 172.21.15.32:0/2671405525 wait complete. 2020-12-05T17:39:55.760 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:39:55.766 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:39:55.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74bf110700 1 Processor -- start 2020-12-05T17:39:55.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74bf110700 1 -- start start 2020-12-05T17:39:55.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74bf110700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b80b5b20 0x7f74b80b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74bf110700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f74b80b5240 0x7f74b8145f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74bf110700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b80b56d0 con 0x7f74b80b6690 2020-12-05T17:39:55.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74bf110700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74b80b1920 con 0x7f74b80b5240 2020-12-05T17:39:55.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74bf110700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b80b1bc0 con 0x7f74b80b5b20 2020-12-05T17:39:55.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74b7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f74b80b6690 0x7f74b80b2060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41586/0 (socket says 172.21.15.32:41586) 2020-12-05T17:39:55.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74b7fff700 1 -- 172.21.15.32:0/335998450 learned_addr learned my addr 172.21.15.32:0/335998450 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:55.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74bceac700 1 --2- 172.21.15.32:0/335998450 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b80b5b20 0x7f74b80b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74bd6ad700 1 --2- 172.21.15.32:0/335998450 >> v2:172.21.15.61:3300/0 conn(0x7f74b80b5240 0x7f74b8145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74b77fe700 1 -- 172.21.15.32:0/335998450 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3847839938 0 0) 0x7f74b80b56d0 con 0x7f74b80b6690 2020-12-05T17:39:55.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74b77fe700 1 -- 172.21.15.32:0/335998450 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7498003230 con 0x7f74b80b6690 2020-12-05T17:39:55.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74b77fe700 1 -- 172.21.15.32:0/335998450 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3750375680 0 0) 0x7f7498003230 con 0x7f74b80b6690 2020-12-05T17:39:55.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74b77fe700 1 -- 172.21.15.32:0/335998450 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b80b5b20 msgr2=0x7f74b80b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:55.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74b77fe700 1 --2- 172.21.15.32:0/335998450 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b80b5b20 0x7f74b80b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74b77fe700 1 -- 172.21.15.32:0/335998450 >> v2:172.21.15.61:3300/0 conn(0x7f74b80b5240 msgr2=0x7f74b8145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:55.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74b77fe700 1 --2- 172.21.15.32:0/335998450 >> v2:172.21.15.61:3300/0 conn(0x7f74b80b5240 0x7f74b8145f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74b77fe700 1 -- 172.21.15.32:0/335998450 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b8146400 con 0x7f74b80b6690 2020-12-05T17:39:55.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74b77fe700 1 -- 172.21.15.32:0/335998450 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f74a00016d0 con 0x7f74b80b6690 2020-12-05T17:39:55.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74b77fe700 1 -- 172.21.15.32:0/335998450 >> v1:172.21.15.32:6789/0 conn(0x7f74b80b6690 legacy=0x7f74b80b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:55.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74b77fe700 1 --2- 172.21.15.32:0/335998450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7498003720 0x7f7498003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74b77fe700 1 --2- 172.21.15.32:0/335998450 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b80b5b20 0x7f74980044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74b77fe700 1 --2- 172.21.15.32:0/335998450 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74980049c0 0x7f7498004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74b77fe700 1 -- 172.21.15.32:0/335998450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b80b1920 con 0x7f7498003720 2020-12-05T17:39:55.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74b77fe700 1 -- 172.21.15.32:0/335998450 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b80b1bc0 con 0x7f74980049c0 2020-12-05T17:39:55.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74b77fe700 1 -- 172.21.15.32:0/335998450 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74980041c0 con 0x7f74b80b5b20 2020-12-05T17:39:55.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74bceac700 1 --2- 172.21.15.32:0/335998450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7498003720 0x7f7498003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74bd6ad700 1 --2- 172.21.15.32:0/335998450 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74980049c0 0x7f7498004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.879+0000 7f74b7fff700 1 --2- 172.21.15.32:0/335998450 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b80b5b20 0x7f74980044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bceac700 1 -- 172.21.15.32:0/335998450 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b80b5b20 msgr2=0x7f74980044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:55.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bceac700 1 --2- 172.21.15.32:0/335998450 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b80b5b20 0x7f74980044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bceac700 1 -- 172.21.15.32:0/335998450 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74980049c0 msgr2=0x7f7498004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:55.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bceac700 1 --2- 172.21.15.32:0/335998450 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74980049c0 0x7f7498004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bceac700 1 -- 172.21.15.32:0/335998450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7498003230 con 0x7f7498003720 2020-12-05T17:39:55.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74b7fff700 1 --2- 172.21.15.32:0/335998450 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b80b5b20 0x7f74980044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:55.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bceac700 1 --2- 172.21.15.32:0/335998450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7498003720 0x7f7498003b40 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7f74a800c000 tx=0x7f74a80091a0).ready entity=mon.0 client_cookie=9f75b0fdd1999288 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:55.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74b77fe700 1 -- 172.21.15.32:0/335998450 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74a800ed00 con 0x7f7498003720 2020-12-05T17:39:55.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74b77fe700 1 -- 172.21.15.32:0/335998450 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a800ee60 con 0x7f7498003720 2020-12-05T17:39:55.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74b77fe700 1 -- 172.21.15.32:0/335998450 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74a80125e0 con 0x7f7498003720 2020-12-05T17:39:55.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bf110700 1 -- 172.21.15.32:0/335998450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7498003720 msgr2=0x7f7498003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:55.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bf110700 1 --2- 172.21.15.32:0/335998450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7498003720 0x7f7498003b40 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7f74a800c000 tx=0x7f74a80091a0).stop 2020-12-05T17:39:55.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bd6ad700 1 -- 172.21.15.32:0/335998450 reap_dead start 2020-12-05T17:39:55.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bf110700 1 -- 172.21.15.32:0/335998450 shutdown_connections 2020-12-05T17:39:55.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bf110700 1 -- 172.21.15.32:0/335998450 >> 172.21.15.32:0/335998450 conn(0x7f74b805f3b0 msgr2=0x7f74b805f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:55.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bf110700 1 -- 172.21.15.32:0/335998450 shutdown_connections 2020-12-05T17:39:55.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bf110700 1 -- 172.21.15.32:0/335998450 wait complete. 2020-12-05T17:39:55.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bf110700 1 Processor -- start 2020-12-05T17:39:55.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bf110700 1 -- start start 2020-12-05T17:39:55.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bf110700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74b80b70c0 0x7f74b80a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bf110700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b80a2ec0 0x7f74b8154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bf110700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74b80a6c10 0x7f74b80a7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bf110700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b80a3340 con 0x7f74b80b70c0 2020-12-05T17:39:55.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bf110700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b80a67e0 con 0x7f74b80a6c10 2020-12-05T17:39:55.902 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bf110700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b80a7580 con 0x7f74b80a2ec0 2020-12-05T17:39:55.902 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bceac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74b80b70c0 0x7f74b80a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.902 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bceac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74b80b70c0 0x7f74b80a42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38948/0 (socket says 172.21.15.32:38948) 2020-12-05T17:39:55.902 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.883+0000 7f74bceac700 1 -- 172.21.15.32:0/1403592091 learned_addr learned my addr 172.21.15.32:0/1403592091 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:55.902 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.887+0000 7f74bd6ad700 1 --2- 172.21.15.32:0/1403592091 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74b80a6c10 0x7f74b80a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.903 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.887+0000 7f74b7fff700 1 --2- 172.21.15.32:0/1403592091 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b80a2ec0 0x7f74b8154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.903 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.887+0000 7f74bceac700 1 -- 172.21.15.32:0/1403592091 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b80a2ec0 msgr2=0x7f74b8154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:55.903 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.887+0000 7f74bceac700 1 --2- 172.21.15.32:0/1403592091 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b80a2ec0 0x7f74b8154d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.903 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.887+0000 7f74bceac700 1 -- 172.21.15.32:0/1403592091 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74b80a6c10 msgr2=0x7f74b80a7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:55.903 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.887+0000 7f74bceac700 1 --2- 172.21.15.32:0/1403592091 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74b80a6c10 0x7f74b80a7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:55.904 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.887+0000 7f74bceac700 1 -- 172.21.15.32:0/1403592091 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b80a3560 con 0x7f74b80b70c0 2020-12-05T17:39:55.904 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.887+0000 7f74bceac700 1 --2- 172.21.15.32:0/1403592091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74b80b70c0 0x7f74b80a42f0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7f74b80a6af0 tx=0x7f74a801a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:55.904 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.887+0000 7f74b5ffb700 1 -- 172.21.15.32:0/1403592091 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74a80245e0 con 0x7f74b80b70c0 2020-12-05T17:39:55.904 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.887+0000 7f74b5ffb700 1 -- 172.21.15.32:0/1403592091 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a80095c0 con 0x7f74b80b70c0 2020-12-05T17:39:55.904 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.887+0000 7f74b5ffb700 1 -- 172.21.15.32:0/1403592091 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74a8024930 con 0x7f74b80b70c0 2020-12-05T17:39:55.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.887+0000 7f74bf110700 1 -- 172.21.15.32:0/1403592091 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b80a3800 con 0x7f74b80b70c0 2020-12-05T17:39:55.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.887+0000 7f74bf110700 1 -- 172.21.15.32:0/1403592091 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b80aeb00 con 0x7f74b80b70c0 2020-12-05T17:39:55.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.887+0000 7f74bf110700 1 -- 172.21.15.32:0/1403592091 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b8057050 con 0x7f74b80b70c0 2020-12-05T17:39:55.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.887+0000 7f74b5ffb700 1 -- 172.21.15.32:0/1403592091 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f74a8024a90 con 0x7f74b80b70c0 2020-12-05T17:39:55.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.887+0000 7f74b5ffb700 1 --2- 172.21.15.32:0/1403592091 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f749c055350 0x7f749c057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:55.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.891+0000 7f74b7fff700 1 --2- 172.21.15.32:0/1403592091 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f749c055350 0x7f749c057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:55.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.891+0000 7f74b5ffb700 1 -- 172.21.15.32:0/1403592091 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f74a806f950 con 0x7f74b80b70c0 2020-12-05T17:39:55.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.891+0000 7f74b7fff700 1 --2- 172.21.15.32:0/1403592091 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f749c055350 0x7f749c057810 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f74a000d000 tx=0x7f74a00014b0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:55.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:55.895+0000 7f74b5ffb700 1 -- 172.21.15.32:0/1403592091 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f74a8045af0 con 0x7f74b80b70c0 2020-12-05T17:39:56.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.035+0000 7f74bf110700 1 -- 172.21.15.32:0/1403592091 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f74b805bae0 con 0x7f74b80b70c0 2020-12-05T17:39:56.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.039+0000 7f74b5ffb700 1 -- 172.21.15.32:0/1403592091 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) v1 ==== 104+0+10 (secure 0 0 0) 0x7f74a8016020 con 0x7f74b80b70c0 2020-12-05T17:39:56.043 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:39:56.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.039+0000 7f74bf110700 1 -- 172.21.15.32:0/1403592091 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f749c055350 msgr2=0x7f749c057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:56.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.039+0000 7f74bf110700 1 --2- 172.21.15.32:0/1403592091 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f749c055350 0x7f749c057810 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f74a000d000 tx=0x7f74a00014b0).stop 2020-12-05T17:39:56.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.039+0000 7f74bf110700 1 -- 172.21.15.32:0/1403592091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74b80b70c0 msgr2=0x7f74b80a42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:56.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.039+0000 7f74bf110700 1 --2- 172.21.15.32:0/1403592091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74b80b70c0 0x7f74b80a42f0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7f74b80a6af0 tx=0x7f74a801a000).stop 2020-12-05T17:39:56.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.039+0000 7f74bf110700 1 -- 172.21.15.32:0/1403592091 shutdown_connections 2020-12-05T17:39:56.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.039+0000 7f74bf110700 1 --2- 172.21.15.32:0/1403592091 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74b80a6c10 0x7f74b80a7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:56.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.039+0000 7f74bf110700 1 --2- 172.21.15.32:0/1403592091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74b80b70c0 0x7f74b80a42f0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:56.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.039+0000 7f74bf110700 1 --2- 172.21.15.32:0/1403592091 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f749c055350 0x7f749c057810 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:56.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.039+0000 7f74bf110700 1 --2- 172.21.15.32:0/1403592091 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b80a2ec0 0x7f74b8154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:56.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.039+0000 7f74bf110700 1 -- 172.21.15.32:0/1403592091 >> 172.21.15.32:0/1403592091 conn(0x7f74b805f3b0 msgr2=0x7f74b80b84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:56.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.039+0000 7f74bf110700 1 -- 172.21.15.32:0/1403592091 shutdown_connections 2020-12-05T17:39:56.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.039+0000 7f74bf110700 1 -- 172.21.15.32:0/1403592091 wait complete. 2020-12-05T17:39:56.056 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:39:56.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.171+0000 7fe25f816700 1 Processor -- start 2020-12-05T17:39:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.171+0000 7fe25f816700 1 -- start start 2020-12-05T17:39:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.171+0000 7fe25f816700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fe2580aea80 0x7fe25813c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.171+0000 7fe25f816700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2580ad230 0x7fe25813f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.171+0000 7fe25f816700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2580b2ee0 con 0x7fe2580b2b00 2020-12-05T17:39:56.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.171+0000 7fe25f816700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe25813ed30 con 0x7fe2580ad230 2020-12-05T17:39:56.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.171+0000 7fe25f816700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2580b0c30 con 0x7fe2580aea80 2020-12-05T17:39:56.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.171+0000 7fe25ddb3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2580ad230 0x7fe25813f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:56.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.171+0000 7fe25d5b2700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe2580b2b00 0x7fe25813bc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41606/0 (socket says 172.21.15.32:41606) 2020-12-05T17:39:56.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.171+0000 7fe25ddb3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2580ad230 0x7fe25813f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38960/0 (socket says 172.21.15.32:38960) 2020-12-05T17:39:56.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.171+0000 7fe25d5b2700 1 -- 172.21.15.32:0/2749554533 learned_addr learned my addr 172.21.15.32:0/2749554533 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:56.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.171+0000 7fe25cdb1700 1 --2- 172.21.15.32:0/2749554533 >> v2:172.21.15.61:3300/0 conn(0x7fe2580aea80 0x7fe25813c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:56.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.171+0000 7fe25ddb3700 1 -- 172.21.15.32:0/2749554533 >> v2:172.21.15.61:3300/0 conn(0x7fe2580aea80 msgr2=0x7fe25813c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:56.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.171+0000 7fe25ddb3700 1 --2- 172.21.15.32:0/2749554533 >> v2:172.21.15.61:3300/0 conn(0x7fe2580aea80 0x7fe25813c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:56.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.171+0000 7fe25ddb3700 1 -- 172.21.15.32:0/2749554533 >> v1:172.21.15.32:6789/0 conn(0x7fe2580b2b00 legacy=0x7fe25813bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:56.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.171+0000 7fe257fff700 1 -- 172.21.15.32:0/2749554533 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 497215677 0 0) 0x7fe2580b2ee0 con 0x7fe2580b2b00 2020-12-05T17:39:56.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.171+0000 7fe25ddb3700 1 -- 172.21.15.32:0/2749554533 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe258142de0 con 0x7fe2580ad230 2020-12-05T17:39:56.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.171+0000 7fe25cdb1700 1 --2- 172.21.15.32:0/2749554533 >> v2:172.21.15.61:3300/0 conn(0x7fe2580aea80 0x7fe25813c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:56.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.175+0000 7fe25ddb3700 1 --2- 172.21.15.32:0/2749554533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2580ad230 0x7fe25813f150 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7fe250002f30 tx=0x7fe250006dc0).ready entity=mon.0 client_cookie=c6d20bb2f4b7572c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:56.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.175+0000 7fe257fff700 1 -- 172.21.15.32:0/2749554533 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2500097f0 con 0x7fe2580ad230 2020-12-05T17:39:56.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.175+0000 7fe257fff700 1 -- 172.21.15.32:0/2749554533 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe250009950 con 0x7fe2580ad230 2020-12-05T17:39:56.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.175+0000 7fe257fff700 1 -- 172.21.15.32:0/2749554533 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe250009cb0 con 0x7fe2580ad230 2020-12-05T17:39:56.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.175+0000 7fe25f816700 1 -- 172.21.15.32:0/2749554533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2580ad230 msgr2=0x7fe25813f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:56.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.175+0000 7fe25f816700 1 --2- 172.21.15.32:0/2749554533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2580ad230 0x7fe25813f150 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7fe250002f30 tx=0x7fe250006dc0).stop 2020-12-05T17:39:56.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.175+0000 7fe25f816700 1 -- 172.21.15.32:0/2749554533 shutdown_connections 2020-12-05T17:39:56.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.175+0000 7fe25f816700 1 --2- 172.21.15.32:0/2749554533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2580ad230 0x7fe25813f150 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:56.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.175+0000 7fe25f816700 1 --2- 172.21.15.32:0/2749554533 >> v2:172.21.15.61:3300/0 conn(0x7fe2580aea80 0x7fe25813c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:56.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.175+0000 7fe25f816700 1 -- 172.21.15.32:0/2749554533 >> 172.21.15.32:0/2749554533 conn(0x7fe258062310 msgr2=0x7fe2580626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:56.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.175+0000 7fe25f816700 1 -- 172.21.15.32:0/2749554533 shutdown_connections 2020-12-05T17:39:56.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.175+0000 7fe25f816700 1 -- 172.21.15.32:0/2749554533 wait complete. 2020-12-05T17:39:56.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.175+0000 7fe25f816700 1 Processor -- start 2020-12-05T17:39:56.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.175+0000 7fe25f816700 1 -- start start 2020-12-05T17:39:56.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.175+0000 7fe25f816700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2580ad230 0x7fe258142580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:56.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.175+0000 7fe25f816700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2580aea80 0x7fe2581518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:56.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.175+0000 7fe25f816700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2580b2b00 0x7fe258151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:56.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.175+0000 7fe25f816700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe258114780 con 0x7fe2580ad230 2020-12-05T17:39:56.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.175+0000 7fe25f816700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe258115890 con 0x7fe2580aea80 2020-12-05T17:39:56.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.175+0000 7fe25f816700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe258066830 con 0x7fe2580b2b00 2020-12-05T17:39:56.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.179+0000 7fe25d5b2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2580ad230 0x7fe258142580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:56.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.179+0000 7fe25d5b2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2580ad230 0x7fe258142580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38962/0 (socket says 172.21.15.32:38962) 2020-12-05T17:39:56.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.179+0000 7fe25d5b2700 1 -- 172.21.15.32:0/2671954776 learned_addr learned my addr 172.21.15.32:0/2671954776 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:56.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.179+0000 7fe25cdb1700 1 --2- 172.21.15.32:0/2671954776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2580aea80 0x7fe2581518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:56.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.179+0000 7fe25d5b2700 1 -- 172.21.15.32:0/2671954776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2580b2b00 msgr2=0x7fe258151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.179+0000 7fe25ddb3700 1 --2- 172.21.15.32:0/2671954776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2580b2b00 0x7fe258151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.179+0000 7fe25d5b2700 1 --2- 172.21.15.32:0/2671954776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2580b2b00 0x7fe258151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.179+0000 7fe25d5b2700 1 -- 172.21.15.32:0/2671954776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2580aea80 msgr2=0x7fe2581518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.179+0000 7fe25d5b2700 1 --2- 172.21.15.32:0/2671954776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2580aea80 0x7fe2581518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.179+0000 7fe25d5b2700 1 -- 172.21.15.32:0/2671954776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe258066590 con 0x7fe2580ad230 2020-12-05T17:39:56.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.179+0000 7fe25cdb1700 1 --2- 172.21.15.32:0/2671954776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2580aea80 0x7fe2581518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:39:56.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.179+0000 7fe25ddb3700 1 --2- 172.21.15.32:0/2671954776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2580b2b00 0x7fe258151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:39:56.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.179+0000 7fe25d5b2700 1 --2- 172.21.15.32:0/2671954776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2580ad230 0x7fe258142580 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7fe24c000b80 tx=0x7fe24c007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:56.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.179+0000 7fe2567fc700 1 -- 172.21.15.32:0/2671954776 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe24c0072e0 con 0x7fe2580ad230 2020-12-05T17:39:56.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.179+0000 7fe2567fc700 1 -- 172.21.15.32:0/2671954776 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24c007440 con 0x7fe2580ad230 2020-12-05T17:39:56.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.179+0000 7fe25f816700 1 -- 172.21.15.32:0/2671954776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2581155f0 con 0x7fe2580ad230 2020-12-05T17:39:56.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.179+0000 7fe2567fc700 1 -- 172.21.15.32:0/2671954776 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe24c0057d0 con 0x7fe2580ad230 2020-12-05T17:39:56.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.179+0000 7fe25f816700 1 -- 172.21.15.32:0/2671954776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe258065720 con 0x7fe2580ad230 2020-12-05T17:39:56.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.179+0000 7fe25f816700 1 -- 172.21.15.32:0/2671954776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe258066330 con 0x7fe2580ad230 2020-12-05T17:39:56.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.179+0000 7fe2567fc700 1 -- 172.21.15.32:0/2671954776 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe24c01a060 con 0x7fe2580ad230 2020-12-05T17:39:56.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.183+0000 7fe2567fc700 1 --2- 172.21.15.32:0/2671954776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe238055450 0x7fe238057910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:56.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.183+0000 7fe2567fc700 1 -- 172.21.15.32:0/2671954776 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe24c06d560 con 0x7fe2580ad230 2020-12-05T17:39:56.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.183+0000 7fe25cdb1700 1 --2- 172.21.15.32:0/2671954776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe238055450 0x7fe238057910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:56.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.183+0000 7fe25cdb1700 1 --2- 172.21.15.32:0/2671954776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe238055450 0x7fe238057910 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7fe24800d000 tx=0x7fe248008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:56.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.187+0000 7fe2567fc700 1 -- 172.21.15.32:0/2671954776 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe24c043700 con 0x7fe2580ad230 2020-12-05T17:39:56.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.327+0000 7fe25f816700 1 -- 172.21.15.32:0/2671954776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe258067120 con 0x7fe2580ad230 2020-12-05T17:39:56.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.327+0000 7fe2567fc700 1 -- 172.21.15.32:0/2671954776 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) v1 ==== 100+0+10 (secure 0 0 0) 0x7fe24c018020 con 0x7fe2580ad230 2020-12-05T17:39:56.335 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:39:56.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.331+0000 7fe25f816700 1 -- 172.21.15.32:0/2671954776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe238055450 msgr2=0x7fe238057910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:56.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.331+0000 7fe25f816700 1 --2- 172.21.15.32:0/2671954776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe238055450 0x7fe238057910 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7fe24800d000 tx=0x7fe248008040).stop 2020-12-05T17:39:56.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.331+0000 7fe25f816700 1 -- 172.21.15.32:0/2671954776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2580ad230 msgr2=0x7fe258142580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:56.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.331+0000 7fe25f816700 1 --2- 172.21.15.32:0/2671954776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2580ad230 0x7fe258142580 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7fe24c000b80 tx=0x7fe24c007bc0).stop 2020-12-05T17:39:56.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.331+0000 7fe25f816700 1 -- 172.21.15.32:0/2671954776 shutdown_connections 2020-12-05T17:39:56.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.331+0000 7fe25f816700 1 --2- 172.21.15.32:0/2671954776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe238055450 0x7fe238057910 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:56.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.331+0000 7fe25f816700 1 --2- 172.21.15.32:0/2671954776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2580b2b00 0x7fe258151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:56.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.331+0000 7fe25f816700 1 --2- 172.21.15.32:0/2671954776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2580ad230 0x7fe258142580 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:56.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.331+0000 7fe25f816700 1 --2- 172.21.15.32:0/2671954776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2580aea80 0x7fe2581518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:56.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.331+0000 7fe25f816700 1 -- 172.21.15.32:0/2671954776 >> 172.21.15.32:0/2671954776 conn(0x7fe258062310 msgr2=0x7fe25813f050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:56.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.331+0000 7fe25f816700 1 -- 172.21.15.32:0/2671954776 shutdown_connections 2020-12-05T17:39:56.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.331+0000 7fe25f816700 1 -- 172.21.15.32:0/2671954776 wait complete. 2020-12-05T17:39:56.349 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:39:56.350 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:39:56.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.455+0000 7f77c959f700 1 Processor -- start 2020-12-05T17:39:56.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.455+0000 7f77c959f700 1 -- start start 2020-12-05T17:39:56.462 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.455+0000 7f77c959f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f77c40b5710 0x7f77c40afe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:56.463 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.455+0000 7f77c959f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f77c40b0340 0x7f77c4147780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:56.463 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.455+0000 7f77c959f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77c40aa490 con 0x7f77c4069cb0 2020-12-05T17:39:56.463 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.455+0000 7f77c959f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77c40b5af0 con 0x7f77c40b0340 2020-12-05T17:39:56.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.455+0000 7f77c959f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77c40a83d0 con 0x7f77c40b5710 2020-12-05T17:39:56.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.455+0000 7f77c37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f77c40b0340 0x7f77c4147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:56.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.455+0000 7f77c37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f77c40b0340 0x7f77c4147780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38972/0 (socket says 172.21.15.32:38972) 2020-12-05T17:39:56.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.455+0000 7f77c37fe700 1 -- 172.21.15.32:0/1355393194 learned_addr learned my addr 172.21.15.32:0/1355393194 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:56.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.455+0000 7f77c2ffd700 1 --2- 172.21.15.32:0/1355393194 >> v2:172.21.15.61:3300/0 conn(0x7f77c40b5710 0x7f77c40afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:56.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.455+0000 7f77c37fe700 1 -- 172.21.15.32:0/1355393194 >> v2:172.21.15.61:3300/0 conn(0x7f77c40b5710 msgr2=0x7f77c40afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:56.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.455+0000 7f77c37fe700 1 --2- 172.21.15.32:0/1355393194 >> v2:172.21.15.61:3300/0 conn(0x7f77c40b5710 0x7f77c40afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:56.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.455+0000 7f77c37fe700 1 -- 172.21.15.32:0/1355393194 >> v1:172.21.15.61:6789/0 conn(0x7f77c4069cb0 legacy=0x7f77c4147070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:56.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.455+0000 7f77c37fe700 1 -- 172.21.15.32:0/1355393194 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c40aa6c0 con 0x7f77c40b0340 2020-12-05T17:39:56.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.455+0000 7f77c37fe700 1 --2- 172.21.15.32:0/1355393194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77c40b0340 0x7f77c4147780 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7f77bc0032b0 tx=0x7f77bc009270).ready entity=mon.0 client_cookie=88836a8e3585ce9e server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:56.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.455+0000 7f77c1ffb700 1 -- 172.21.15.32:0/1355393194 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77bc00bc00 con 0x7f77c40b0340 2020-12-05T17:39:56.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.455+0000 7f77c1ffb700 1 -- 172.21.15.32:0/1355393194 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77bc00bd60 con 0x7f77c40b0340 2020-12-05T17:39:56.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.455+0000 7f77c1ffb700 1 -- 172.21.15.32:0/1355393194 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77bc011400 con 0x7f77c40b0340 2020-12-05T17:39:56.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.455+0000 7f77c959f700 1 -- 172.21.15.32:0/1355393194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77c40b0340 msgr2=0x7f77c4147780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:56.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.455+0000 7f77c959f700 1 --2- 172.21.15.32:0/1355393194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77c40b0340 0x7f77c4147780 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7f77bc0032b0 tx=0x7f77bc009270).stop 2020-12-05T17:39:56.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.459+0000 7f77c959f700 1 -- 172.21.15.32:0/1355393194 shutdown_connections 2020-12-05T17:39:56.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.459+0000 7f77c959f700 1 --2- 172.21.15.32:0/1355393194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77c40b0340 0x7f77c4147780 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:56.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.459+0000 7f77c959f700 1 --2- 172.21.15.32:0/1355393194 >> v2:172.21.15.61:3300/0 conn(0x7f77c40b5710 0x7f77c40afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:56.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.459+0000 7f77c959f700 1 -- 172.21.15.32:0/1355393194 >> 172.21.15.32:0/1355393194 conn(0x7f77c405fe40 msgr2=0x7f77c40b4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:56.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.459+0000 7f77c959f700 1 -- 172.21.15.32:0/1355393194 shutdown_connections 2020-12-05T17:39:56.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.459+0000 7f77c959f700 1 -- 172.21.15.32:0/1355393194 wait complete. 2020-12-05T17:39:56.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.459+0000 7f77c959f700 1 Processor -- start 2020-12-05T17:39:56.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.459+0000 7f77c959f700 1 -- start start 2020-12-05T17:39:56.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.459+0000 7f77c959f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f77c4069cb0 0x7f77c4159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:56.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.459+0000 7f77c959f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77c40b0340 0x7f77c414aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:56.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.459+0000 7f77c959f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f77c40b5710 0x7f77c4063020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:56.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.459+0000 7f77c959f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c414d590 con 0x7f77c40b0340 2020-12-05T17:39:56.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.459+0000 7f77c959f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c40659b0 con 0x7f77c4069cb0 2020-12-05T17:39:56.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.459+0000 7f77c959f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c414ce50 con 0x7f77c40b5710 2020-12-05T17:39:56.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.459+0000 7f77c27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77c40b0340 0x7f77c414aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:56.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.459+0000 7f77c27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77c40b0340 0x7f77c414aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38976/0 (socket says 172.21.15.32:38976) 2020-12-05T17:39:56.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.459+0000 7f77c27fc700 1 -- 172.21.15.32:0/2396366291 learned_addr learned my addr 172.21.15.32:0/2396366291 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:56.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.463+0000 7f77c2ffd700 1 --2- 172.21.15.32:0/2396366291 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f77c4069cb0 0x7f77c4159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:56.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.463+0000 7f77c37fe700 1 --2- 172.21.15.32:0/2396366291 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f77c40b5710 0x7f77c4063020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:56.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.463+0000 7f77c2ffd700 1 -- 172.21.15.32:0/2396366291 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f77c40b5710 msgr2=0x7f77c4063020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:56.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.463+0000 7f77c2ffd700 1 --2- 172.21.15.32:0/2396366291 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f77c40b5710 0x7f77c4063020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:56.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.463+0000 7f77c2ffd700 1 -- 172.21.15.32:0/2396366291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77c40b0340 msgr2=0x7f77c414aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:56.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.463+0000 7f77c2ffd700 1 --2- 172.21.15.32:0/2396366291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77c40b0340 0x7f77c414aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:56.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.463+0000 7f77c2ffd700 1 -- 172.21.15.32:0/2396366291 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c414d0f0 con 0x7f77c4069cb0 2020-12-05T17:39:56.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.463+0000 7f77c27fc700 1 --2- 172.21.15.32:0/2396366291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77c40b0340 0x7f77c414aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:39:56.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.463+0000 7f77c2ffd700 1 --2- 172.21.15.32:0/2396366291 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f77c4069cb0 0x7f77c4159390 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f77c414e190 tx=0x7f77b4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:56.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.463+0000 7f77b3fff700 1 -- 172.21.15.32:0/2396366291 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77b400f7e0 con 0x7f77c4069cb0 2020-12-05T17:39:56.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.463+0000 7f77b3fff700 1 -- 172.21.15.32:0/2396366291 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b400f940 con 0x7f77c4069cb0 2020-12-05T17:39:56.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.463+0000 7f77c959f700 1 -- 172.21.15.32:0/2396366291 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c414a790 con 0x7f77c4069cb0 2020-12-05T17:39:56.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.463+0000 7f77b3fff700 1 -- 172.21.15.32:0/2396366291 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77b400fca0 con 0x7f77c4069cb0 2020-12-05T17:39:56.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.463+0000 7f77c959f700 1 -- 172.21.15.32:0/2396366291 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c414b2b0 con 0x7f77c4069cb0 2020-12-05T17:39:56.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.463+0000 7f77c959f700 1 -- 172.21.15.32:0/2396366291 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77c4057050 con 0x7f77c4069cb0 2020-12-05T17:39:56.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.467+0000 7f77b3fff700 1 -- 172.21.15.32:0/2396366291 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f77b401a030 con 0x7f77c4069cb0 2020-12-05T17:39:56.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.467+0000 7f77b3fff700 1 --2- 172.21.15.32:0/2396366291 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f77a00554e0 0x7f77a00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:56.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.467+0000 7f77b3fff700 1 -- 172.21.15.32:0/2396366291 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f77b406dcf0 con 0x7f77c4069cb0 2020-12-05T17:39:56.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.467+0000 7f77c27fc700 1 --2- 172.21.15.32:0/2396366291 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f77a00554e0 0x7f77a00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:56.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.467+0000 7f77c27fc700 1 --2- 172.21.15.32:0/2396366291 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f77a00554e0 0x7f77a00579a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f77b800d000 tx=0x7f77b8009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:56.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.475+0000 7f77b3fff700 1 -- 172.21.15.32:0/2396366291 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f77b4043df0 con 0x7f77c4069cb0 2020-12-05T17:39:56.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.667+0000 7f77c959f700 1 -- 172.21.15.32:0/2396366291 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77c40b51b0 con 0x7f77c4069cb0 2020-12-05T17:39:56.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.667+0000 7f77b3fff700 1 -- 172.21.15.32:0/2396366291 <== mon.1 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f77b4047410 con 0x7f77c4069cb0 2020-12-05T17:39:56.674 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:56.674 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":47,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/3759190037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3759190037},{"type":"v1","addr":"172.21.15.61:6833","nonce":3759190037}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":45}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:39:47.471195+0000","modified":"2020-12-05T17:39:48.473895+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7233":{"gid":7233,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4174090156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4174090156},{"type":"v1","addr":"172.21.15.32:6835","nonce":4174090156}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2020-12-05T17:39:56.677 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.671+0000 7f77c959f700 1 -- 172.21.15.32:0/2396366291 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f77a00554e0 msgr2=0x7f77a00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:56.677 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.671+0000 7f77c959f700 1 --2- 172.21.15.32:0/2396366291 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f77a00554e0 0x7f77a00579a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f77b800d000 tx=0x7f77b8009040).stop 2020-12-05T17:39:56.677 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.671+0000 7f77c959f700 1 -- 172.21.15.32:0/2396366291 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f77c4069cb0 msgr2=0x7f77c4159390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:56.677 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.671+0000 7f77c959f700 1 --2- 172.21.15.32:0/2396366291 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f77c4069cb0 0x7f77c4159390 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f77c414e190 tx=0x7f77b4008040).stop 2020-12-05T17:39:56.678 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.671+0000 7f77c959f700 1 -- 172.21.15.32:0/2396366291 shutdown_connections 2020-12-05T17:39:56.678 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.671+0000 7f77c959f700 1 --2- 172.21.15.32:0/2396366291 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f77a00554e0 0x7f77a00579a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:56.678 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.671+0000 7f77c959f700 1 --2- 172.21.15.32:0/2396366291 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f77c40b5710 0x7f77c4063020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:56.678 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.671+0000 7f77c959f700 1 --2- 172.21.15.32:0/2396366291 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f77c4069cb0 0x7f77c4159390 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:56.679 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.671+0000 7f77c959f700 1 --2- 172.21.15.32:0/2396366291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77c40b0340 0x7f77c414aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:56.679 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.671+0000 7f77c959f700 1 -- 172.21.15.32:0/2396366291 >> 172.21.15.32:0/2396366291 conn(0x7f77c405fe40 msgr2=0x7f77c4062170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:56.679 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.671+0000 7f77c959f700 1 -- 172.21.15.32:0/2396366291 shutdown_connections 2020-12-05T17:39:56.679 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:56.671+0000 7f77c959f700 1 -- 172.21.15.32:0/2396366291 wait complete. 2020-12-05T17:39:56.680 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 47 2020-12-05T17:39:56.895 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:39:47.471195+0000', 'modified': '2020-12-05T17:39:48.473895+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7233}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7233': {'gid': 7233, 'name': 'a', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/4174090156', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 4174090156}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 4174090156}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:39:56.895 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:39:56.896 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:39:57.024 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":7233,"want_state":"up:active","state":"up:active","rank_uptime":9.5383286829999996,"mdsmap_epoch":47,"osdmap_epoch":102,"osdmap_epoch_barrier":101,"uptime":14.774303159} 2020-12-05T17:39:57.025 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 7233, "mdsmap_epoch": 47, "osdmap_epoch": 102, "osdmap_epoch_barrier": 101, "rank_uptime": 9.538328683, "state": "up:active", "uptime": 14.774303159, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:39:57.025 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:39:57.026 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-12-05T17:39:57.026 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi002.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-12-05T17:39:57.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:57.027 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:57.036 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:39:57.093 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:39:57.094 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:39:57.153 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:39:57.154 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:39:57.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:57.155 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:39:57.204 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:39:57.204 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:57.248 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:39:57.265 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:39:57.266 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:39:57.322 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:39:57.323 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:39:57.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:57.324 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:39:57.329 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:2020-12-05T17:39:57.323+0000 7f3a0a71e0c0 -1 init, newargv = 0x564b9374a3b0 newargc=9 2020-12-05T17:39:57.329 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[10826]: starting ceph client 2020-12-05T17:39:57.340 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[10826]: starting fuse 2020-12-05T17:39:58.437 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:39:58.454 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:39:58.454 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:39:58.508 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:39:58.509 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:39:58.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:58.511 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:39:58.556 INFO:teuthology.orchestra.run.smithi002.stdout:50 2020-12-05T17:39:58.557 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [50] 2020-12-05T17:39:58.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:58.558 INFO:teuthology.orchestra.run.smithi002:> sudo python3 -c ' 2020-12-05T17:39:58.558 INFO:teuthology.orchestra.run.smithi002:> import glob 2020-12-05T17:39:58.558 INFO:teuthology.orchestra.run.smithi002:> import re 2020-12-05T17:39:58.559 INFO:teuthology.orchestra.run.smithi002:> import os 2020-12-05T17:39:58.559 INFO:teuthology.orchestra.run.smithi002:> import subprocess 2020-12-05T17:39:58.560 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:39:58.560 INFO:teuthology.orchestra.run.smithi002:> def find_socket(client_name): 2020-12-05T17:39:58.560 INFO:teuthology.orchestra.run.smithi002:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-12-05T17:39:58.561 INFO:teuthology.orchestra.run.smithi002:> files = glob.glob(asok_path) 2020-12-05T17:39:58.561 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:39:58.561 INFO:teuthology.orchestra.run.smithi002:> # Given a non-glob path, it better be there 2020-12-05T17:39:58.562 INFO:teuthology.orchestra.run.smithi002:> if "*" not in asok_path: 2020-12-05T17:39:58.562 INFO:teuthology.orchestra.run.smithi002:> assert(len(files) == 1) 2020-12-05T17:39:58.563 INFO:teuthology.orchestra.run.smithi002:> return files[0] 2020-12-05T17:39:58.563 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:39:58.563 INFO:teuthology.orchestra.run.smithi002:> for f in files: 2020-12-05T17:39:58.564 INFO:teuthology.orchestra.run.smithi002:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:39:58.564 INFO:teuthology.orchestra.run.smithi002:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:39:58.565 INFO:teuthology.orchestra.run.smithi002:> return f 2020-12-05T17:39:58.565 INFO:teuthology.orchestra.run.smithi002:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:39:58.566 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:39:58.566 INFO:teuthology.orchestra.run.smithi002:> print(find_socket("client.1")) 2020-12-05T17:39:58.566 INFO:teuthology.orchestra.run.smithi002:> ' 2020-12-05T17:39:58.649 INFO:teuthology.orchestra.run.smithi002.stdout:/var/run/ceph/ceph-client.1.10826.asok 2020-12-05T17:39:58.654 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.10826.asok 2020-12-05T17:39:58.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:58.655 INFO:teuthology.orchestra.run.smithi002:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.10826.asok status 2020-12-05T17:39:58.757 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2020-12-05T17:39:58.757 INFO:teuthology.orchestra.run.smithi002.stdout: "metadata": { 2020-12-05T17:39:58.758 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:39:58.758 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:39:58.758 INFO:teuthology.orchestra.run.smithi002.stdout: "entity_id": "1", 2020-12-05T17:39:58.759 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": "smithi002", 2020-12-05T17:39:58.759 INFO:teuthology.orchestra.run.smithi002.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-12-05T17:39:58.759 INFO:teuthology.orchestra.run.smithi002.stdout: "pid": "10826", 2020-12-05T17:39:58.760 INFO:teuthology.orchestra.run.smithi002.stdout: "root": "/" 2020-12-05T17:39:58.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:39:58.760 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_count": 0, 2020-12-05T17:39:58.761 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_pinned_count": 0, 2020-12-05T17:39:58.761 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 6488, 2020-12-05T17:39:58.761 INFO:teuthology.orchestra.run.smithi002.stdout: "inst": { 2020-12-05T17:39:58.762 INFO:teuthology.orchestra.run.smithi002.stdout: "name": { 2020-12-05T17:39:58.762 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "client", 2020-12-05T17:39:58.762 INFO:teuthology.orchestra.run.smithi002.stdout: "num": 6488 2020-12-05T17:39:58.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:39:58.763 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2020-12-05T17:39:58.763 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2020-12-05T17:39:58.763 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2020-12-05T17:39:58.764 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1580441764 2020-12-05T17:39:58.764 INFO:teuthology.orchestra.run.smithi002.stdout: } 2020-12-05T17:39:58.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:39:58.765 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2020-12-05T17:39:58.766 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2020-12-05T17:39:58.766 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2020-12-05T17:39:58.766 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1580441764 2020-12-05T17:39:58.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:39:58.767 INFO:teuthology.orchestra.run.smithi002.stdout: "inst_str": "client.6488 172.21.15.2:0/1580441764", 2020-12-05T17:39:58.767 INFO:teuthology.orchestra.run.smithi002.stdout: "addr_str": "172.21.15.2:0/1580441764", 2020-12-05T17:39:58.767 INFO:teuthology.orchestra.run.smithi002.stdout: "inode_count": 1, 2020-12-05T17:39:58.768 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_epoch": 47, 2020-12-05T17:39:58.768 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch": 102, 2020-12-05T17:39:58.768 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:39:58.769 INFO:teuthology.orchestra.run.smithi002.stdout: "blacklisted": false 2020-12-05T17:39:58.769 INFO:teuthology.orchestra.run.smithi002.stdout:} 2020-12-05T17:39:58.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:58.771 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:58.772 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:58.786 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:39:58.787 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:39:58.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:58.788 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:58.850 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early df --format=json-pretty 2020-12-05T17:39:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.971+0000 7f6192a30700 1 Processor -- start 2020-12-05T17:39:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.971+0000 7f6192a30700 1 -- start start 2020-12-05T17:39:58.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.971+0000 7f6192a30700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f618c0b54d0 0x7f618c0ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.971+0000 7f6192a30700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f618c14aed0 con 0x7f618c0afd40 2020-12-05T17:39:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.971+0000 7f6192a30700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f618c067a40 con 0x7f618c1470a0 2020-12-05T17:39:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.971+0000 7f6192a30700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f618c0ae080 con 0x7f618c0b54d0 2020-12-05T17:39:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.971+0000 7f6191a2e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f618c0b54d0 0x7f618c0ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:58.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.971+0000 7f619122d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f618c0afd40 0x7f618c14a0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41632/0 (socket says 172.21.15.32:41632) 2020-12-05T17:39:58.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.971+0000 7f6191a2e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f618c0b54d0 0x7f618c0ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38984/0 (socket says 172.21.15.32:38984) 2020-12-05T17:39:58.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.971+0000 7f619122d700 1 -- 172.21.15.32:0/4170298063 learned_addr learned my addr 172.21.15.32:0/4170298063 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:58.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.975+0000 7f6191a2e700 1 -- 172.21.15.32:0/4170298063 >> v1:172.21.15.61:6789/0 conn(0x7f618c1470a0 legacy=0x7f618c14a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:58.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.975+0000 7f6191a2e700 1 -- 172.21.15.32:0/4170298063 >> v1:172.21.15.32:6789/0 conn(0x7f618c0afd40 legacy=0x7f618c14a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:58.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.975+0000 7f6190a2c700 1 -- 172.21.15.32:0/4170298063 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3281400454 0 0) 0x7f618c14aed0 con 0x7f618c0afd40 2020-12-05T17:39:58.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.975+0000 7f6191a2e700 1 -- 172.21.15.32:0/4170298063 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c0ae320 con 0x7f618c0b54d0 2020-12-05T17:39:58.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.975+0000 7f6191a2e700 1 --2- 172.21.15.32:0/4170298063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f618c0b54d0 0x7f618c0ae7c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7f6188002f30 tx=0x7f6188006dc0).ready entity=mon.0 client_cookie=6e81437f16e9394e server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:58.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.975+0000 7f6190a2c700 1 -- 172.21.15.32:0/4170298063 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61880097f0 con 0x7f618c0b54d0 2020-12-05T17:39:58.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.975+0000 7f6190a2c700 1 -- 172.21.15.32:0/4170298063 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6188009950 con 0x7f618c0b54d0 2020-12-05T17:39:58.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.975+0000 7f6192a30700 1 -- 172.21.15.32:0/4170298063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f618c0b54d0 msgr2=0x7f618c0ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:58.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.975+0000 7f6190a2c700 1 -- 172.21.15.32:0/4170298063 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6188009cb0 con 0x7f618c0b54d0 2020-12-05T17:39:58.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.975+0000 7f6192a30700 1 --2- 172.21.15.32:0/4170298063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f618c0b54d0 0x7f618c0ae7c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7f6188002f30 tx=0x7f6188006dc0).stop 2020-12-05T17:39:58.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.975+0000 7f6192a30700 1 -- 172.21.15.32:0/4170298063 shutdown_connections 2020-12-05T17:39:58.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.975+0000 7f6192a30700 1 --2- 172.21.15.32:0/4170298063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f618c0b54d0 0x7f618c0ae7c0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:58.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.975+0000 7f6192a30700 1 -- 172.21.15.32:0/4170298063 >> 172.21.15.32:0/4170298063 conn(0x7f618c062070 msgr2=0x7f618c062450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:58.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.975+0000 7f6192a30700 1 -- 172.21.15.32:0/4170298063 shutdown_connections 2020-12-05T17:39:58.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.975+0000 7f6192a30700 1 -- 172.21.15.32:0/4170298063 wait complete. 2020-12-05T17:39:58.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.975+0000 7f6192a30700 1 Processor -- start 2020-12-05T17:39:58.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.975+0000 7f6192a30700 1 -- start start 2020-12-05T17:39:58.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.979+0000 7f6192a30700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f618c0afd40 0x7f618c14bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:58.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.979+0000 7f6192a30700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f618c0b54d0 0x7f618c140030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:58.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.979+0000 7f6192a30700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f618c1470a0 0x7f618c140520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:58.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.979+0000 7f6192a30700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c14aa20 con 0x7f618c0b54d0 2020-12-05T17:39:58.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.979+0000 7f6192a30700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c14adf0 con 0x7f618c1470a0 2020-12-05T17:39:58.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.979+0000 7f6192a30700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618c14a7c0 con 0x7f618c0afd40 2020-12-05T17:39:58.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.979+0000 7f619122d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f618c0b54d0 0x7f618c140030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:58.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.979+0000 7f6191a2e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f618c0afd40 0x7f618c14bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:58.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.979+0000 7f619122d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f618c0b54d0 0x7f618c140030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38994/0 (socket says 172.21.15.32:38994) 2020-12-05T17:39:58.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.979+0000 7f619122d700 1 -- 172.21.15.32:0/2643078705 learned_addr learned my addr 172.21.15.32:0/2643078705 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:58.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.979+0000 7f619222f700 1 --2- 172.21.15.32:0/2643078705 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f618c1470a0 0x7f618c140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:58.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.979+0000 7f619122d700 1 -- 172.21.15.32:0/2643078705 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f618c0afd40 msgr2=0x7f618c14bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:58.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.979+0000 7f619122d700 1 --2- 172.21.15.32:0/2643078705 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f618c0afd40 0x7f618c14bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:58.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.979+0000 7f619122d700 1 -- 172.21.15.32:0/2643078705 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f618c1470a0 msgr2=0x7f618c140520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:58.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.979+0000 7f619122d700 1 --2- 172.21.15.32:0/2643078705 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f618c1470a0 0x7f618c140520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:58.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.979+0000 7f619122d700 1 -- 172.21.15.32:0/2643078705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c14c680 con 0x7f618c0b54d0 2020-12-05T17:39:58.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.979+0000 7f619122d700 1 --2- 172.21.15.32:0/2643078705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f618c0b54d0 0x7f618c140030 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f617c000b80 tx=0x7f617c007a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:58.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.979+0000 7f617affd700 1 -- 172.21.15.32:0/2643078705 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f617c0072e0 con 0x7f618c0b54d0 2020-12-05T17:39:58.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.979+0000 7f617affd700 1 -- 172.21.15.32:0/2643078705 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617c007440 con 0x7f618c0b54d0 2020-12-05T17:39:58.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.979+0000 7f6192a30700 1 -- 172.21.15.32:0/2643078705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f618c14c920 con 0x7f618c0b54d0 2020-12-05T17:39:58.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.979+0000 7f617affd700 1 -- 172.21.15.32:0/2643078705 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f617c0057d0 con 0x7f618c0b54d0 2020-12-05T17:39:58.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.979+0000 7f6192a30700 1 -- 172.21.15.32:0/2643078705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618c14a4c0 con 0x7f618c0b54d0 2020-12-05T17:39:58.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.979+0000 7f6192a30700 1 -- 172.21.15.32:0/2643078705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618c057050 con 0x7f618c0b54d0 2020-12-05T17:39:58.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.983+0000 7f617affd700 1 -- 172.21.15.32:0/2643078705 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f617c010070 con 0x7f618c0b54d0 2020-12-05T17:39:58.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.983+0000 7f617affd700 1 --2- 172.21.15.32:0/2643078705 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f616c0553b0 0x7f616c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:58.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.983+0000 7f617affd700 1 -- 172.21.15.32:0/2643078705 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f617c06db20 con 0x7f618c0b54d0 2020-12-05T17:39:58.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.983+0000 7f6191a2e700 1 --2- 172.21.15.32:0/2643078705 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f616c0553b0 0x7f616c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:58.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.983+0000 7f6191a2e700 1 --2- 172.21.15.32:0/2643078705 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f616c0553b0 0x7f616c057870 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f618801b000 tx=0x7f6188015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:58.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:58.987+0000 7f617affd700 1 -- 172.21.15.32:0/2643078705 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f617c043cc0 con 0x7f618c0b54d0 2020-12-05T17:39:59.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.179+0000 7f6192a30700 1 -- 172.21.15.32:0/2643078705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f618c05c300 con 0x7f618c0b54d0 2020-12-05T17:39:59.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.179+0000 7f617affd700 1 -- 172.21.15.32:0/2643078705 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1936 (secure 0 0 0) 0x7f617c0472e0 con 0x7f618c0b54d0 2020-12-05T17:39:59.186 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:59.186 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:39:59.186 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2020-12-05T17:39:59.187 INFO:teuthology.orchestra.run.smithi032.stdout: "total_bytes": 773094113280, 2020-12-05T17:39:59.187 INFO:teuthology.orchestra.run.smithi032.stdout: "total_avail_bytes": 764493856768, 2020-12-05T17:39:59.187 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_bytes": 10321920, 2020-12-05T17:39:59.187 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_bytes": 8600256512, 2020-12-05T17:39:59.188 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_ratio": 0.011124462820589542, 2020-12-05T17:39:59.188 INFO:teuthology.orchestra.run.smithi032.stdout: "num_osds": 8, 2020-12-05T17:39:59.188 INFO:teuthology.orchestra.run.smithi032.stdout: "num_per_pool_osds": 8, 2020-12-05T17:39:59.188 INFO:teuthology.orchestra.run.smithi032.stdout: "num_per_pool_omap_osds": 8 2020-12-05T17:39:59.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:59.189 INFO:teuthology.orchestra.run.smithi032.stdout: "stats_by_class": { 2020-12-05T17:39:59.189 INFO:teuthology.orchestra.run.smithi032.stdout: "ssd": { 2020-12-05T17:39:59.189 INFO:teuthology.orchestra.run.smithi032.stdout: "total_bytes": 773094113280, 2020-12-05T17:39:59.189 INFO:teuthology.orchestra.run.smithi032.stdout: "total_avail_bytes": 764493856768, 2020-12-05T17:39:59.189 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_bytes": 10321920, 2020-12-05T17:39:59.190 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_bytes": 8600256512, 2020-12-05T17:39:59.190 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_ratio": 0.011124462820589542 2020-12-05T17:39:59.190 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:59.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:59.190 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2020-12-05T17:39:59.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:59.191 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_health_metrics", 2020-12-05T17:39:59.191 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 1, 2020-12-05T17:39:59.191 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2020-12-05T17:39:59.191 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2020-12-05T17:39:59.192 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 2, 2020-12-05T17:39:59.192 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2020-12-05T17:39:59.192 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2020-12-05T17:39:59.192 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2020-12-05T17:39:59.193 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362919526400 2020-12-05T17:39:59.193 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:59.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:59.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:59.194 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd", 2020-12-05T17:39:59.194 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 2, 2020-12-05T17:39:59.194 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2020-12-05T17:39:59.194 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2020-12-05T17:39:59.195 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 0, 2020-12-05T17:39:59.195 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2020-12-05T17:39:59.195 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2020-12-05T17:39:59.195 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2020-12-05T17:39:59.195 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362919526400 2020-12-05T17:39:59.196 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:59.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:59.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:59.196 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs_metadata", 2020-12-05T17:39:59.196 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 19, 2020-12-05T17:39:59.197 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2020-12-05T17:39:59.197 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 3350, 2020-12-05T17:39:59.197 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 22, 2020-12-05T17:39:59.197 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 64, 2020-12-05T17:39:59.197 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 65536, 2020-12-05T17:39:59.198 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 9.0289979937097087e-08, 2020-12-05T17:39:59.198 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362919526400 2020-12-05T17:39:59.198 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:59.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:39:59.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:39:59.199 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs_data", 2020-12-05T17:39:59.199 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 20, 2020-12-05T17:39:59.199 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2020-12-05T17:39:59.199 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2020-12-05T17:39:59.199 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 0, 2020-12-05T17:39:59.200 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2020-12-05T17:39:59.200 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2020-12-05T17:39:59.200 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2020-12-05T17:39:59.200 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362919526400 2020-12-05T17:39:59.201 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:59.201 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:39:59.201 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:39:59.202 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:39:59.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.183+0000 7f6192a30700 1 -- 172.21.15.32:0/2643078705 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f616c0553b0 msgr2=0x7f616c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:59.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.183+0000 7f6192a30700 1 --2- 172.21.15.32:0/2643078705 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f616c0553b0 0x7f616c057870 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f618801b000 tx=0x7f6188015000).stop 2020-12-05T17:39:59.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.183+0000 7f6192a30700 1 -- 172.21.15.32:0/2643078705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f618c0b54d0 msgr2=0x7f618c140030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:59.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.183+0000 7f6192a30700 1 --2- 172.21.15.32:0/2643078705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f618c0b54d0 0x7f618c140030 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f617c000b80 tx=0x7f617c007a90).stop 2020-12-05T17:39:59.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.183+0000 7f6192a30700 1 -- 172.21.15.32:0/2643078705 shutdown_connections 2020-12-05T17:39:59.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.183+0000 7f6192a30700 1 --2- 172.21.15.32:0/2643078705 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f618c1470a0 0x7f618c140520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:59.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.183+0000 7f6192a30700 1 --2- 172.21.15.32:0/2643078705 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f616c0553b0 0x7f616c057870 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:59.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.183+0000 7f6192a30700 1 --2- 172.21.15.32:0/2643078705 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f618c0afd40 0x7f618c14bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:59.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.183+0000 7f6192a30700 1 --2- 172.21.15.32:0/2643078705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f618c0b54d0 0x7f618c140030 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:59.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.183+0000 7f6192a30700 1 -- 172.21.15.32:0/2643078705 >> 172.21.15.32:0/2643078705 conn(0x7f618c062070 msgr2=0x7f618c1461f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:59.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.183+0000 7f6192a30700 1 -- 172.21.15.32:0/2643078705 shutdown_connections 2020-12-05T17:39:59.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.183+0000 7f6192a30700 1 -- 172.21.15.32:0/2643078705 wait complete. 2020-12-05T17:39:59.207 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data size -f json-pretty 2020-12-05T17:39:59.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f0545252700 1 Processor -- start 2020-12-05T17:39:59.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f0545252700 1 -- start start 2020-12-05T17:39:59.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f0545252700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f05400b2000 0x7f0540142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:59.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f0545252700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0540140190 con 0x7f0540143410 2020-12-05T17:39:59.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f0545252700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05400b23e0 con 0x7f054013fd20 2020-12-05T17:39:59.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f0545252700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05400aeba0 con 0x7f05400b2000 2020-12-05T17:39:59.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0540143410 0x7f054013f610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41650/0 (socket says 172.21.15.32:41650) 2020-12-05T17:39:59.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053f7fe700 1 -- 172.21.15.32:0/1399442569 learned_addr learned my addr 172.21.15.32:0/1399442569 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:39:59.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053ffff700 1 --2- 172.21.15.32:0/1399442569 >> v2:172.21.15.61:3300/0 conn(0x7f05400b2000 0x7f0540142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:59.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053effd700 1 -- 172.21.15.32:0/1399442569 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 493829865 0 0) 0x7f0540140190 con 0x7f0540143410 2020-12-05T17:39:59.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053effd700 1 -- 172.21.15.32:0/1399442569 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0524003230 con 0x7f0540143410 2020-12-05T17:39:59.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053effd700 1 -- 172.21.15.32:0/1399442569 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2036728968 0 0) 0x7f05400b23e0 con 0x7f054013fd20 2020-12-05T17:39:59.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053effd700 1 -- 172.21.15.32:0/1399442569 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0540140190 con 0x7f054013fd20 2020-12-05T17:39:59.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053effd700 1 -- 172.21.15.32:0/1399442569 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4264502125 0 0) 0x7f0524003230 con 0x7f0540143410 2020-12-05T17:39:59.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053effd700 1 -- 172.21.15.32:0/1399442569 >> v2:172.21.15.61:3300/0 conn(0x7f05400b2000 msgr2=0x7f0540142f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:59.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053effd700 1 --2- 172.21.15.32:0/1399442569 >> v2:172.21.15.61:3300/0 conn(0x7f05400b2000 0x7f0540142f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:59.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053effd700 1 -- 172.21.15.32:0/1399442569 >> v1:172.21.15.61:6789/0 conn(0x7f054013fd20 legacy=0x7f05400af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:59.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053effd700 1 -- 172.21.15.32:0/1399442569 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05400af190 con 0x7f0540143410 2020-12-05T17:39:59.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053effd700 1 -- 172.21.15.32:0/1399442569 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f05280016d0 con 0x7f0540143410 2020-12-05T17:39:59.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053effd700 1 -- 172.21.15.32:0/1399442569 >> v1:172.21.15.32:6789/0 conn(0x7f0540143410 legacy=0x7f054013f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:59.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053effd700 1 --2- 172.21.15.32:0/1399442569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05240038b0 0x7f0524003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:59.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053effd700 1 --2- 172.21.15.32:0/1399442569 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f05240044d0 0x7f0524004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:59.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053effd700 1 --2- 172.21.15.32:0/1399442569 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0524004e40 0x7f052400b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:59.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053effd700 1 -- 172.21.15.32:0/1399442569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05400aeba0 con 0x7f05240038b0 2020-12-05T17:39:59.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053effd700 1 -- 172.21.15.32:0/1399442569 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05240053a0 con 0x7f0524004e40 2020-12-05T17:39:59.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053effd700 1 -- 172.21.15.32:0/1399442569 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052400bc90 con 0x7f05240044d0 2020-12-05T17:39:59.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053ffff700 1 --2- 172.21.15.32:0/1399442569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05240038b0 0x7f0524003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:59.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053f7fe700 1 --2- 172.21.15.32:0/1399442569 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f05240044d0 0x7f0524004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:59.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f0544a51700 1 --2- 172.21.15.32:0/1399442569 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0524004e40 0x7f052400b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:59.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053ffff700 1 -- 172.21.15.32:0/1399442569 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f05240044d0 msgr2=0x7f0524004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:59.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053ffff700 1 --2- 172.21.15.32:0/1399442569 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f05240044d0 0x7f0524004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:59.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053ffff700 1 -- 172.21.15.32:0/1399442569 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0524004e40 msgr2=0x7f052400b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:39:59.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053ffff700 1 --2- 172.21.15.32:0/1399442569 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0524004e40 0x7f052400b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:59.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f0544a51700 1 -- 172.21.15.32:0/1399442569 reap_dead start 2020-12-05T17:39:59.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053ffff700 1 -- 172.21.15.32:0/1399442569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0524003230 con 0x7f05240038b0 2020-12-05T17:39:59.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053ffff700 1 --2- 172.21.15.32:0/1399442569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05240038b0 0x7f0524003cb0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7f053000c000 tx=0x7f0530008040).ready entity=mon.0 client_cookie=f8ded10ef566ecc0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:59.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.323+0000 7f053effd700 1 -- 172.21.15.32:0/1399442569 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f053000e8a0 con 0x7f05240038b0 2020-12-05T17:39:59.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.327+0000 7f053effd700 1 -- 172.21.15.32:0/1399442569 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053000ea00 con 0x7f05240038b0 2020-12-05T17:39:59.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.327+0000 7f053effd700 1 -- 172.21.15.32:0/1399442569 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f053000ed60 con 0x7f05240038b0 2020-12-05T17:39:59.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.327+0000 7f0545252700 1 -- 172.21.15.32:0/1399442569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05240038b0 msgr2=0x7f0524003cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:59.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.327+0000 7f0545252700 1 --2- 172.21.15.32:0/1399442569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05240038b0 0x7f0524003cb0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7f053000c000 tx=0x7f0530008040).stop 2020-12-05T17:39:59.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.327+0000 7f0545252700 1 -- 172.21.15.32:0/1399442569 shutdown_connections 2020-12-05T17:39:59.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.327+0000 7f0545252700 1 --2- 172.21.15.32:0/1399442569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05240038b0 0x7f0524003cb0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:59.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.327+0000 7f0545252700 1 -- 172.21.15.32:0/1399442569 >> 172.21.15.32:0/1399442569 conn(0x7f05400b5620 msgr2=0x7f05400b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:59.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.327+0000 7f0545252700 1 -- 172.21.15.32:0/1399442569 shutdown_connections 2020-12-05T17:39:59.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.327+0000 7f0545252700 1 -- 172.21.15.32:0/1399442569 wait complete. 2020-12-05T17:39:59.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.327+0000 7f0545252700 1 Processor -- start 2020-12-05T17:39:59.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.327+0000 7f0545252700 1 -- start start 2020-12-05T17:39:59.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.327+0000 7f0545252700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05240038b0 0x7f0540154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:59.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.327+0000 7f0545252700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0540143f40 0x7f05400af920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:59.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.327+0000 7f0545252700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0540063040 0x7f05400634c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:59.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.327+0000 7f0545252700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05400639b0 con 0x7f05240038b0 2020-12-05T17:39:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.327+0000 7f0545252700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0540141da0 con 0x7f0540143f40 2020-12-05T17:39:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.327+0000 7f0545252700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0540152280 con 0x7f0540063040 2020-12-05T17:39:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.327+0000 7f053ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05240038b0 0x7f0540154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.327+0000 7f053ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05240038b0 0x7f0540154c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39014/0 (socket says 172.21.15.32:39014) 2020-12-05T17:39:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.327+0000 7f053ffff700 1 -- 172.21.15.32:0/3193490069 learned_addr learned my addr 172.21.15.32:0/3193490069 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:39:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.327+0000 7f053f7fe700 1 --2- 172.21.15.32:0/3193490069 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0540143f40 0x7f05400af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.327+0000 7f0544a51700 1 --2- 172.21.15.32:0/3193490069 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0540063040 0x7f05400634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.331+0000 7f053ffff700 1 -- 172.21.15.32:0/3193490069 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0540063040 msgr2=0x7f05400634c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.331+0000 7f053ffff700 1 --2- 172.21.15.32:0/3193490069 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0540063040 0x7f05400634c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.331+0000 7f053ffff700 1 -- 172.21.15.32:0/3193490069 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0540143f40 msgr2=0x7f05400af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.331+0000 7f053ffff700 1 --2- 172.21.15.32:0/3193490069 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0540143f40 0x7f05400af920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.331+0000 7f053ffff700 1 -- 172.21.15.32:0/3193490069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05400b0640 con 0x7f05240038b0 2020-12-05T17:39:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.331+0000 7f053ffff700 1 --2- 172.21.15.32:0/3193490069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05240038b0 0x7f0540154c30 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0x7f053000c000 tx=0x7f053001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.331+0000 7f053d7fa700 1 -- 172.21.15.32:0/3193490069 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f053001c430 con 0x7f05240038b0 2020-12-05T17:39:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.331+0000 7f053d7fa700 1 -- 172.21.15.32:0/3193490069 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053001c590 con 0x7f05240038b0 2020-12-05T17:39:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.331+0000 7f0545252700 1 -- 172.21.15.32:0/3193490069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0540146880 con 0x7f05240038b0 2020-12-05T17:39:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.331+0000 7f053d7fa700 1 -- 172.21.15.32:0/3193490069 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f053001c8f0 con 0x7f05240038b0 2020-12-05T17:39:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.331+0000 7f0545252700 1 -- 172.21.15.32:0/3193490069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054013b540 con 0x7f05240038b0 2020-12-05T17:39:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.331+0000 7f053d7fa700 1 -- 172.21.15.32:0/3193490069 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f053001ca50 con 0x7f05240038b0 2020-12-05T17:39:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.331+0000 7f0545252700 1 -- 172.21.15.32:0/3193490069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05401520c0 con 0x7f05240038b0 2020-12-05T17:39:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.331+0000 7f053d7fa700 1 --2- 172.21.15.32:0/3193490069 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0518055500 0x7f05180579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:39:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.331+0000 7f053d7fa700 1 -- 172.21.15.32:0/3193490069 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f053006fa80 con 0x7f05240038b0 2020-12-05T17:39:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.331+0000 7f053f7fe700 1 --2- 172.21.15.32:0/3193490069 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0518055500 0x7f05180579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:39:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.335+0000 7f053f7fe700 1 --2- 172.21.15.32:0/3193490069 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0518055500 0x7f05180579c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f0528005000 tx=0x7f05400af320).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:39:59.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.339+0000 7f053d7fa700 1 -- 172.21.15.32:0/3193490069 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0530045b10 con 0x7f05240038b0 2020-12-05T17:39:59.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.483+0000 7f0545252700 1 -- 172.21.15.32:0/3193490069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "size", "format": "json-pretty"} v 0) v1 -- 0x7f0540057050 con 0x7f05240038b0 2020-12-05T17:39:59.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.483+0000 7f053d7fa700 1 -- 172.21.15.32:0/3193490069 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "size", "format": "json-pretty"}]=0 v102) v1 ==== 123+0+64 (secure 0 0 0) 0x7f0530016020 con 0x7f05240038b0 2020-12-05T17:39:59.489 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:39:59.489 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:39:59.490 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": "cephfs_data", 2020-12-05T17:39:59.490 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_id": 20, 2020-12-05T17:39:59.490 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2 2020-12-05T17:39:59.491 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:39:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.487+0000 7f0545252700 1 -- 172.21.15.32:0/3193490069 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0518055500 msgr2=0x7f05180579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.487+0000 7f0545252700 1 --2- 172.21.15.32:0/3193490069 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0518055500 0x7f05180579c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f0528005000 tx=0x7f05400af320).stop 2020-12-05T17:39:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.487+0000 7f0545252700 1 -- 172.21.15.32:0/3193490069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05240038b0 msgr2=0x7f0540154c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:39:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.487+0000 7f0545252700 1 --2- 172.21.15.32:0/3193490069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05240038b0 0x7f0540154c30 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0x7f053000c000 tx=0x7f053001a000).stop 2020-12-05T17:39:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.487+0000 7f0545252700 1 -- 172.21.15.32:0/3193490069 shutdown_connections 2020-12-05T17:39:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.487+0000 7f0545252700 1 --2- 172.21.15.32:0/3193490069 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0518055500 0x7f05180579c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.487+0000 7f0545252700 1 --2- 172.21.15.32:0/3193490069 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0540063040 0x7f05400634c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.487+0000 7f0545252700 1 --2- 172.21.15.32:0/3193490069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05240038b0 0x7f0540154c30 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.487+0000 7f0545252700 1 --2- 172.21.15.32:0/3193490069 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0540143f40 0x7f05400af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:39:59.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.487+0000 7f0545252700 1 -- 172.21.15.32:0/3193490069 >> 172.21.15.32:0/3193490069 conn(0x7f05400b5620 msgr2=0x7f05400b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:39:59.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.487+0000 7f0545252700 1 -- 172.21.15.32:0/3193490069 shutdown_connections 2020-12-05T17:39:59.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:39:59.487+0000 7f0545252700 1 -- 172.21.15.32:0/3193490069 wait complete. 2020-12-05T17:39:59.503 INFO:teuthology.orchestra.run.smithi036:> cd /home/ubuntu/cephtest/mnt.0 && sudo df . 2020-12-05T17:39:59.528 INFO:teuthology.orchestra.run.smithi036.stdout:Filesystem 1K-blocks Used Available Use% Mounted on 2020-12-05T17:39:59.529 INFO:teuthology.orchestra.run.smithi036.stdout:ceph-fuse 354410496 0 354410496 0% /home/ubuntu/cephtest/mnt.0 2020-12-05T17:39:59.530 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:39:59.531 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:59.548 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:59.562 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:59.587 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:59.603 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:39:59.630 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:39:59.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:59.631 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:59.632 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:59.639 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:39:59.640 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:39:59.640 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi036.front.sepia.ceph.com... 2020-12-05T17:39:59.641 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:39:59.641 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:59.712 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[11344]: fuse finished with error 0 and tester_r 0 2020-12-05T17:39:59.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:59.716 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:59.716 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:59.762 INFO:teuthology.orchestra.run.smithi036.stdout:ext2/ext3 2020-12-05T17:39:59.764 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:39:59.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:39:59.764 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:39:59.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:59.811 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:59.811 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:59.819 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:39:59.820 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:39:59.820 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi002.front.sepia.ceph.com... 2020-12-05T17:39:59.821 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:39:59.821 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:59.902 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[10826]: fuse finished with error 0 and tester_r 0 2020-12-05T17:39:59.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:39:59.905 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:39:59.905 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:39:59.952 INFO:teuthology.orchestra.run.smithi002.stdout:ext2/ext3 2020-12-05T17:39:59.953 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:40:00.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:40:00.067 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:40:00.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:00.117 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:40:00.118 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:40:00.125 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:40:00.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:40:00.126 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2020-12-05T17:40:00.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:40:00.127 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:40:00.172 INFO:teuthology.orchestra.run.smithi053.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:40:00.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:40:00.174 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2020-12-05T17:40:00.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.291+0000 7f48fee36700 1 Processor -- start 2020-12-05T17:40:00.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.291+0000 7f48fee36700 1 -- start start 2020-12-05T17:40:00.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fee36700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48f80b1fc0 0x7f48f80af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:00.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fee36700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f48f80ab710 0x7f48f8061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:00.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fee36700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f8146a90 con 0x7f48f8147400 2020-12-05T17:40:00.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fee36700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f80b0110 con 0x7f48f80ab710 2020-12-05T17:40:00.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fee36700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f80b23a0 con 0x7f48f80b1fc0 2020-12-05T17:40:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fd633700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f48f80ab710 0x7f48f8061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fd633700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f48f80ab710 0x7f48f8061780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39018/0 (socket says 172.21.15.32:39018) 2020-12-05T17:40:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fd633700 1 -- 172.21.15.32:0/3591111711 learned_addr learned my addr 172.21.15.32:0/3591111711 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fde34700 1 --2- 172.21.15.32:0/3591111711 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48f80b1fc0 0x7f48f80af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fd633700 1 -- 172.21.15.32:0/3591111711 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48f80b1fc0 msgr2=0x7f48f80af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fd633700 1 --2- 172.21.15.32:0/3591111711 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48f80b1fc0 0x7f48f80af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fd633700 1 -- 172.21.15.32:0/3591111711 >> v1:172.21.15.61:6789/0 conn(0x7f48f8147400 legacy=0x7f48f8145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fd633700 1 -- 172.21.15.32:0/3591111711 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f8146cc0 con 0x7f48f80ab710 2020-12-05T17:40:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fd633700 1 --2- 172.21.15.32:0/3591111711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48f80ab710 0x7f48f8061780 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7f48ec002f30 tx=0x7f48ec006e80).ready entity=mon.0 client_cookie=47d404814198a090 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fce32700 1 -- 172.21.15.32:0/3591111711 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48ec0097f0 con 0x7f48f80ab710 2020-12-05T17:40:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fce32700 1 -- 172.21.15.32:0/3591111711 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ec009950 con 0x7f48f80ab710 2020-12-05T17:40:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fce32700 1 -- 172.21.15.32:0/3591111711 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48ec009cb0 con 0x7f48f80ab710 2020-12-05T17:40:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fee36700 1 -- 172.21.15.32:0/3591111711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48f80ab710 msgr2=0x7f48f8061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fee36700 1 --2- 172.21.15.32:0/3591111711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48f80ab710 0x7f48f8061780 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7f48ec002f30 tx=0x7f48ec006e80).stop 2020-12-05T17:40:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fee36700 1 -- 172.21.15.32:0/3591111711 shutdown_connections 2020-12-05T17:40:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fee36700 1 --2- 172.21.15.32:0/3591111711 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48f80b1fc0 0x7f48f80af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:00.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fee36700 1 --2- 172.21.15.32:0/3591111711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48f80ab710 0x7f48f8061780 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:00.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fee36700 1 -- 172.21.15.32:0/3591111711 >> 172.21.15.32:0/3591111711 conn(0x7f48f80b5530 msgr2=0x7f48f80b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:00.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fee36700 1 -- 172.21.15.32:0/3591111711 shutdown_connections 2020-12-05T17:40:00.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.295+0000 7f48fee36700 1 -- 172.21.15.32:0/3591111711 wait complete. 2020-12-05T17:40:00.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48fee36700 1 Processor -- start 2020-12-05T17:40:00.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48fee36700 1 -- start start 2020-12-05T17:40:00.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48fee36700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48f80ab710 0x7f48f80abcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:00.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48fee36700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48f80b1fc0 0x7f48f80663f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:00.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48fee36700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48f8147400 0x7f48f80668e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:00.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48fee36700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f814adc0 con 0x7f48f8147400 2020-12-05T17:40:00.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48fee36700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f80ad250 con 0x7f48f80ab710 2020-12-05T17:40:00.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48fee36700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f80ad4f0 con 0x7f48f80b1fc0 2020-12-05T17:40:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48fe635700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48f8147400 0x7f48f80668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48fe635700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48f8147400 0x7f48f80668e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39028/0 (socket says 172.21.15.32:39028) 2020-12-05T17:40:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48fe635700 1 -- 172.21.15.32:0/299090345 learned_addr learned my addr 172.21.15.32:0/299090345 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48fde34700 1 --2- 172.21.15.32:0/299090345 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48f80ab710 0x7f48f80abcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48fd633700 1 --2- 172.21.15.32:0/299090345 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48f80b1fc0 0x7f48f80663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:00.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48fe635700 1 -- 172.21.15.32:0/299090345 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48f80b1fc0 msgr2=0x7f48f80663f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:00.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48fe635700 1 --2- 172.21.15.32:0/299090345 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48f80b1fc0 0x7f48f80663f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:00.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48fe635700 1 -- 172.21.15.32:0/299090345 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48f80ab710 msgr2=0x7f48f80abcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:00.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48fe635700 1 --2- 172.21.15.32:0/299090345 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48f80ab710 0x7f48f80abcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:00.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48fe635700 1 -- 172.21.15.32:0/299090345 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f80ad790 con 0x7f48f8147400 2020-12-05T17:40:00.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48fd633700 1 --2- 172.21.15.32:0/299090345 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48f80b1fc0 0x7f48f80663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:40:00.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48fe635700 1 --2- 172.21.15.32:0/299090345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48f8147400 0x7f48f80668e0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0x7f48f81472e0 tx=0x7f48f0007f20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:00.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48eaffd700 1 -- 172.21.15.32:0/299090345 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48f000a5a0 con 0x7f48f8147400 2020-12-05T17:40:00.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48eaffd700 1 -- 172.21.15.32:0/299090345 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f000a700 con 0x7f48f8147400 2020-12-05T17:40:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48fee36700 1 -- 172.21.15.32:0/299090345 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f80ac1a0 con 0x7f48f8147400 2020-12-05T17:40:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48eaffd700 1 -- 172.21.15.32:0/299090345 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48f000aa60 con 0x7f48f8147400 2020-12-05T17:40:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.299+0000 7f48fee36700 1 -- 172.21.15.32:0/299090345 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f80adc30 con 0x7f48f8147400 2020-12-05T17:40:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.303+0000 7f48fee36700 1 -- 172.21.15.32:0/299090345 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f8057050 con 0x7f48f8147400 2020-12-05T17:40:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.303+0000 7f48eaffd700 1 -- 172.21.15.32:0/299090345 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f48f0019030 con 0x7f48f8147400 2020-12-05T17:40:00.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.303+0000 7f48eaffd700 1 --2- 172.21.15.32:0/299090345 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f48d8055330 0x7f48d80577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:00.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.303+0000 7f48eaffd700 1 -- 172.21.15.32:0/299090345 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f48f006d910 con 0x7f48f8147400 2020-12-05T17:40:00.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.303+0000 7f48fde34700 1 --2- 172.21.15.32:0/299090345 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f48d8055330 0x7f48d80577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:00.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.303+0000 7f48fde34700 1 --2- 172.21.15.32:0/299090345 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f48d8055330 0x7f48d80577f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f48f400c000 tx=0x7f48f4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:00.323 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.311+0000 7f48eaffd700 1 -- 172.21.15.32:0/299090345 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f48f00439a0 con 0x7f48f8147400 2020-12-05T17:40:00.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.447+0000 7f48fee36700 1 -- 172.21.15.32:0/299090345 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f48f80ad010 con 0x7f48f8147400 2020-12-05T17:40:00.461 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.455+0000 7f48eaffd700 1 -- 172.21.15.32:0/299090345 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f48f0046fc0 con 0x7f48f8147400 2020-12-05T17:40:00.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.459+0000 7f48fee36700 1 -- 172.21.15.32:0/299090345 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f48d8055330 msgr2=0x7f48d80577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:00.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.459+0000 7f48fee36700 1 --2- 172.21.15.32:0/299090345 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f48d8055330 0x7f48d80577f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f48f400c000 tx=0x7f48f4008040).stop 2020-12-05T17:40:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.459+0000 7f48fee36700 1 -- 172.21.15.32:0/299090345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48f8147400 msgr2=0x7f48f80668e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.459+0000 7f48fee36700 1 --2- 172.21.15.32:0/299090345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48f8147400 0x7f48f80668e0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0x7f48f81472e0 tx=0x7f48f0007f20).stop 2020-12-05T17:40:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.459+0000 7f48fee36700 1 -- 172.21.15.32:0/299090345 shutdown_connections 2020-12-05T17:40:00.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.459+0000 7f48fee36700 1 --2- 172.21.15.32:0/299090345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48f8147400 0x7f48f80668e0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:00.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.459+0000 7f48fee36700 1 --2- 172.21.15.32:0/299090345 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48f80ab710 0x7f48f80abcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:00.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.459+0000 7f48fee36700 1 --2- 172.21.15.32:0/299090345 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f48d8055330 0x7f48d80577f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:00.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.459+0000 7f48fee36700 1 --2- 172.21.15.32:0/299090345 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48f80b1fc0 0x7f48f80663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:00.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.459+0000 7f48fee36700 1 -- 172.21.15.32:0/299090345 >> 172.21.15.32:0/299090345 conn(0x7f48f80b5530 msgr2=0x7f48f8144c30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:00.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.459+0000 7f48fee36700 1 -- 172.21.15.32:0/299090345 shutdown_connections 2020-12-05T17:40:00.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.459+0000 7f48fee36700 1 -- 172.21.15.32:0/299090345 wait complete. 2020-12-05T17:40:00.477 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df' 2020-12-05T17:40:00.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.583+0000 7f961ffff700 1 Processor -- start 2020-12-05T17:40:00.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.583+0000 7f961ffff700 1 -- start start 2020-12-05T17:40:00.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.583+0000 7f961ffff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f96200af920 0x7f9620063040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:00.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.583+0000 7f961ffff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f962005fad0 con 0x7f962005f6f0 2020-12-05T17:40:00.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.583+0000 7f961ffff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9620144650 con 0x7f96200afea0 2020-12-05T17:40:00.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.583+0000 7f961ffff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96200b2cd0 con 0x7f96200af920 2020-12-05T17:40:00.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.583+0000 7f961effd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f962005f6f0 0x7f96200b1d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41680/0 (socket says 172.21.15.32:41680) 2020-12-05T17:40:00.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.583+0000 7f961effd700 1 -- 172.21.15.32:0/3486801199 learned_addr learned my addr 172.21.15.32:0/3486801199 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:00.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.583+0000 7f961e7fc700 1 --2- 172.21.15.32:0/3486801199 >> v2:172.21.15.61:3300/0 conn(0x7f96200af920 0x7f9620063040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:00.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.583+0000 7f961dffb700 1 -- 172.21.15.32:0/3486801199 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2130887526 0 0) 0x7f962005fad0 con 0x7f962005f6f0 2020-12-05T17:40:00.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.583+0000 7f961dffb700 1 -- 172.21.15.32:0/3486801199 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9604003230 con 0x7f962005f6f0 2020-12-05T17:40:00.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.583+0000 7f961dffb700 1 -- 172.21.15.32:0/3486801199 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 113101547 0 0) 0x7f9620144650 con 0x7f96200afea0 2020-12-05T17:40:00.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.583+0000 7f961dffb700 1 -- 172.21.15.32:0/3486801199 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f962005fad0 con 0x7f96200afea0 2020-12-05T17:40:00.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.583+0000 7f961e7fc700 1 -- 172.21.15.32:0/3486801199 >> v1:172.21.15.61:6789/0 conn(0x7f96200afea0 legacy=0x7f96201409c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:00.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.583+0000 7f961e7fc700 1 -- 172.21.15.32:0/3486801199 >> v1:172.21.15.32:6789/0 conn(0x7f962005f6f0 legacy=0x7f96200b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:00.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.583+0000 7f961e7fc700 1 -- 172.21.15.32:0/3486801199 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96201448f0 con 0x7f96200af920 2020-12-05T17:40:00.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.587+0000 7f961e7fc700 1 --2- 172.21.15.32:0/3486801199 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96200af920 0x7f9620063040 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f9614002f30 tx=0x7f9614006dc0).ready entity=mon.1 client_cookie=49c90c53b56d9ceb server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:00.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.587+0000 7f961dffb700 1 -- 172.21.15.32:0/3486801199 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9614009740 con 0x7f96200af920 2020-12-05T17:40:00.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.587+0000 7f961dffb700 1 -- 172.21.15.32:0/3486801199 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96140098a0 con 0x7f96200af920 2020-12-05T17:40:00.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.587+0000 7f961dffb700 1 -- 172.21.15.32:0/3486801199 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9614009c00 con 0x7f96200af920 2020-12-05T17:40:00.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.587+0000 7f961ffff700 1 -- 172.21.15.32:0/3486801199 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96200af920 msgr2=0x7f9620063040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:00.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.587+0000 7f961ffff700 1 --2- 172.21.15.32:0/3486801199 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96200af920 0x7f9620063040 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f9614002f30 tx=0x7f9614006dc0).stop 2020-12-05T17:40:00.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.587+0000 7f961ffff700 1 -- 172.21.15.32:0/3486801199 shutdown_connections 2020-12-05T17:40:00.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.587+0000 7f961ffff700 1 --2- 172.21.15.32:0/3486801199 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96200af920 0x7f9620063040 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:00.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.587+0000 7f961ffff700 1 -- 172.21.15.32:0/3486801199 >> 172.21.15.32:0/3486801199 conn(0x7f96200604c0 msgr2=0x7f962005c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:00.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.587+0000 7f961ffff700 1 -- 172.21.15.32:0/3486801199 shutdown_connections 2020-12-05T17:40:00.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.587+0000 7f961ffff700 1 -- 172.21.15.32:0/3486801199 wait complete. 2020-12-05T17:40:00.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f961ffff700 1 Processor -- start 2020-12-05T17:40:00.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f961ffff700 1 -- start start 2020-12-05T17:40:00.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f961ffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f962005f6f0 0x7f96200634d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:00.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f961ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96200af920 0x7f962010f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:00.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f961ffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96200afea0 0x7f9620150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:00.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f961ffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962010efb0 con 0x7f96200af920 2020-12-05T17:40:00.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f961ffff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9620111c70 con 0x7f962005f6f0 2020-12-05T17:40:00.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f961ffff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9620150590 con 0x7f96200afea0 2020-12-05T17:40:00.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f961e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96200af920 0x7f962010f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:00.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f961e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96200af920 0x7f962010f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39042/0 (socket says 172.21.15.32:39042) 2020-12-05T17:40:00.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f961e7fc700 1 -- 172.21.15.32:0/1813270659 learned_addr learned my addr 172.21.15.32:0/1813270659 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:00.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f961effd700 1 --2- 172.21.15.32:0/1813270659 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f962005f6f0 0x7f96200634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:00.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f961f7fe700 1 --2- 172.21.15.32:0/1813270659 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96200afea0 0x7f9620150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:00.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f961e7fc700 1 -- 172.21.15.32:0/1813270659 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96200afea0 msgr2=0x7f9620150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:00.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f961e7fc700 1 --2- 172.21.15.32:0/1813270659 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96200afea0 0x7f9620150c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:00.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f961e7fc700 1 -- 172.21.15.32:0/1813270659 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f962005f6f0 msgr2=0x7f96200634d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:00.609 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f961e7fc700 1 --2- 172.21.15.32:0/1813270659 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f962005f6f0 0x7f96200634d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:00.609 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f961e7fc700 1 -- 172.21.15.32:0/1813270659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9620111ea0 con 0x7f96200af920 2020-12-05T17:40:00.609 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f961e7fc700 1 --2- 172.21.15.32:0/1813270659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96200af920 0x7f962010f740 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x7f9614002f30 tx=0x7f9614015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:00.609 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f960ffff700 1 -- 172.21.15.32:0/1813270659 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9614011ad0 con 0x7f96200af920 2020-12-05T17:40:00.609 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f960ffff700 1 -- 172.21.15.32:0/1813270659 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9614011c30 con 0x7f96200af920 2020-12-05T17:40:00.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f961ffff700 1 -- 172.21.15.32:0/1813270659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96201507f0 con 0x7f96200af920 2020-12-05T17:40:00.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f960ffff700 1 -- 172.21.15.32:0/1813270659 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f961401d450 con 0x7f96200af920 2020-12-05T17:40:00.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.591+0000 7f961ffff700 1 -- 172.21.15.32:0/1813270659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9620066520 con 0x7f96200af920 2020-12-05T17:40:00.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.595+0000 7f961ffff700 1 -- 172.21.15.32:0/1813270659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96201514c0 con 0x7f96200af920 2020-12-05T17:40:00.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.595+0000 7f960ffff700 1 -- 172.21.15.32:0/1813270659 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f961400f070 con 0x7f96200af920 2020-12-05T17:40:00.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.595+0000 7f960ffff700 1 --2- 172.21.15.32:0/1813270659 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f95fc055330 0x7f95fc0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:00.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.595+0000 7f960ffff700 1 -- 172.21.15.32:0/1813270659 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f961406f910 con 0x7f96200af920 2020-12-05T17:40:00.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.595+0000 7f961effd700 1 --2- 172.21.15.32:0/1813270659 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f95fc055330 0x7f95fc0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:00.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.595+0000 7f961effd700 1 --2- 172.21.15.32:0/1813270659 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f95fc055330 0x7f95fc0577f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f961000d000 tx=0x7f9610009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:00.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.603+0000 7f960ffff700 1 -- 172.21.15.32:0/1813270659 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f96201514c0 con 0x7f96200af920 2020-12-05T17:40:00.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:00.791+0000 7f961ffff700 1 -- 172.21.15.32:0/1813270659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]} v 0) v1 -- 0x7f96201514c0 con 0x7f96200af920 2020-12-05T17:40:01.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.031+0000 7f960ffff700 1 -- 172.21.15.32:0/1813270659 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]}]=0 v375) v1 ==== 127+0+0 (secure 0 0 0) 0x7f96201514c0 con 0x7f96200af920 2020-12-05T17:40:01.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.035+0000 7f961ffff700 1 -- 172.21.15.32:0/1813270659 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f95fc055330 msgr2=0x7f95fc0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:01.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.035+0000 7f961ffff700 1 --2- 172.21.15.32:0/1813270659 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f95fc055330 0x7f95fc0577f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f961000d000 tx=0x7f9610009040).stop 2020-12-05T17:40:01.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.035+0000 7f961ffff700 1 -- 172.21.15.32:0/1813270659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96200af920 msgr2=0x7f962010f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:01.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.035+0000 7f961ffff700 1 --2- 172.21.15.32:0/1813270659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96200af920 0x7f962010f740 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x7f9614002f30 tx=0x7f9614015000).stop 2020-12-05T17:40:01.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.035+0000 7f961ffff700 1 -- 172.21.15.32:0/1813270659 shutdown_connections 2020-12-05T17:40:01.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.035+0000 7f961ffff700 1 --2- 172.21.15.32:0/1813270659 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f95fc055330 0x7f95fc0577f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:01.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.035+0000 7f961ffff700 1 --2- 172.21.15.32:0/1813270659 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96200afea0 0x7f9620150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:01.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.035+0000 7f961ffff700 1 --2- 172.21.15.32:0/1813270659 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f962005f6f0 0x7f96200634d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:01.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.035+0000 7f961ffff700 1 --2- 172.21.15.32:0/1813270659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96200af920 0x7f962010f740 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:01.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.035+0000 7f961ffff700 1 -- 172.21.15.32:0/1813270659 >> 172.21.15.32:0/1813270659 conn(0x7f96200604c0 msgr2=0x7f96200b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:01.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.035+0000 7f961ffff700 1 -- 172.21.15.32:0/1813270659 shutdown_connections 2020-12-05T17:40:01.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.035+0000 7f961ffff700 1 -- 172.21.15.32:0/1813270659 wait complete. 2020-12-05T17:40:01.059 INFO:tasks.cephfs_test_runner:test_filtered_df (tasks.cephfs.test_misc.TestMisc) ... ok 2020-12-05T17:40:01.060 INFO:tasks.cephfs_test_runner:Starting test: test_fs_new (tasks.cephfs.test_misc.TestMisc) 2020-12-05T17:40:01.061 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new' 2020-12-05T17:40:01.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.179+0000 7f98c6ac7700 1 Processor -- start 2020-12-05T17:40:01.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.179+0000 7f98c6ac7700 1 -- start start 2020-12-05T17:40:01.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.179+0000 7f98c6ac7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f98c00b1fc0 0x7f98c00af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.179+0000 7f98c6ac7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c0146a90 con 0x7f98c0147400 2020-12-05T17:40:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.179+0000 7f98c6ac7700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c00b23a0 con 0x7f98c00ab710 2020-12-05T17:40:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.179+0000 7f98c6ac7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c0061e90 con 0x7f98c00b1fc0 2020-12-05T17:40:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.179+0000 7f98c62c6700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f98c0147400 0x7f98c0145d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41694/0 (socket says 172.21.15.32:41694) 2020-12-05T17:40:01.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.179+0000 7f98c62c6700 1 -- 172.21.15.32:0/4042346550 learned_addr learned my addr 172.21.15.32:0/4042346550 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:01.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.179+0000 7f98c5ac5700 1 --2- 172.21.15.32:0/4042346550 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f98c00b1fc0 0x7f98c00af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:01.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.179+0000 7f98c4ac3700 1 -- 172.21.15.32:0/4042346550 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090029702 0 0) 0x7f98c0146a90 con 0x7f98c0147400 2020-12-05T17:40:01.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.179+0000 7f98c4ac3700 1 -- 172.21.15.32:0/4042346550 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98a8003230 con 0x7f98c0147400 2020-12-05T17:40:01.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.179+0000 7f98c4ac3700 1 -- 172.21.15.32:0/4042346550 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2910054378 0 0) 0x7f98c00b23a0 con 0x7f98c00ab710 2020-12-05T17:40:01.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.179+0000 7f98c4ac3700 1 -- 172.21.15.32:0/4042346550 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98c0146a90 con 0x7f98c00ab710 2020-12-05T17:40:01.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.179+0000 7f98c5ac5700 1 -- 172.21.15.32:0/4042346550 >> v1:172.21.15.61:6789/0 conn(0x7f98c00ab710 legacy=0x7f98c0061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:01.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.179+0000 7f98c5ac5700 1 -- 172.21.15.32:0/4042346550 >> v1:172.21.15.32:6789/0 conn(0x7f98c0147400 legacy=0x7f98c0145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:01.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.179+0000 7f98c5ac5700 1 -- 172.21.15.32:0/4042346550 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c0146cc0 con 0x7f98c00b1fc0 2020-12-05T17:40:01.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.183+0000 7f98c5ac5700 1 --2- 172.21.15.32:0/4042346550 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f98c00b1fc0 0x7f98c00af230 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f98bc002f30 tx=0x7f98bc006dc0).ready entity=mon.2 client_cookie=f6ac124e5b1f455a server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:01.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.183+0000 7f98c4ac3700 1 -- 172.21.15.32:0/4042346550 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98bc009730 con 0x7f98c00b1fc0 2020-12-05T17:40:01.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.183+0000 7f98c4ac3700 1 -- 172.21.15.32:0/4042346550 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98bc009890 con 0x7f98c00b1fc0 2020-12-05T17:40:01.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.183+0000 7f98c4ac3700 1 -- 172.21.15.32:0/4042346550 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98bc009bf0 con 0x7f98c00b1fc0 2020-12-05T17:40:01.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.183+0000 7f98c6ac7700 1 -- 172.21.15.32:0/4042346550 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f98c00b1fc0 msgr2=0x7f98c00af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:01.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.183+0000 7f98c6ac7700 1 --2- 172.21.15.32:0/4042346550 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f98c00b1fc0 0x7f98c00af230 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f98bc002f30 tx=0x7f98bc006dc0).stop 2020-12-05T17:40:01.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.183+0000 7f98c6ac7700 1 -- 172.21.15.32:0/4042346550 shutdown_connections 2020-12-05T17:40:01.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.183+0000 7f98c6ac7700 1 --2- 172.21.15.32:0/4042346550 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f98c00b1fc0 0x7f98c00af230 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:01.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.183+0000 7f98c6ac7700 1 -- 172.21.15.32:0/4042346550 >> 172.21.15.32:0/4042346550 conn(0x7f98c00b5530 msgr2=0x7f98c00b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:01.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.183+0000 7f98c6ac7700 1 -- 172.21.15.32:0/4042346550 shutdown_connections 2020-12-05T17:40:01.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.183+0000 7f98c6ac7700 1 -- 172.21.15.32:0/4042346550 wait complete. 2020-12-05T17:40:01.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.183+0000 7f98c6ac7700 1 Processor -- start 2020-12-05T17:40:01.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.183+0000 7f98c6ac7700 1 -- start start 2020-12-05T17:40:01.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.187+0000 7f98c6ac7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98c00ab710 0x7f98c00ad680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:01.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.187+0000 7f98c6ac7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f98c00b1fc0 0x7f98c0155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:01.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.187+0000 7f98c6ac7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f98c0147400 0x7f98c0147e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:01.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.187+0000 7f98c6ac7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c00ae680 con 0x7f98c00ab710 2020-12-05T17:40:01.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.187+0000 7f98c6ac7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c00acdf0 con 0x7f98c0147400 2020-12-05T17:40:01.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.187+0000 7f98c6ac7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c014a5a0 con 0x7f98c00b1fc0 2020-12-05T17:40:01.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.187+0000 7f98c5ac5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98c00ab710 0x7f98c00ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:01.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.187+0000 7f98c5ac5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98c00ab710 0x7f98c00ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39052/0 (socket says 172.21.15.32:39052) 2020-12-05T17:40:01.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.187+0000 7f98c5ac5700 1 -- 172.21.15.32:0/2249475674 learned_addr learned my addr 172.21.15.32:0/2249475674 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:01.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.187+0000 7f98c62c6700 1 --2- 172.21.15.32:0/2249475674 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f98c0147400 0x7f98c0147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:01.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.187+0000 7f98c52c4700 1 --2- 172.21.15.32:0/2249475674 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f98c00b1fc0 0x7f98c0155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:01.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.187+0000 7f98c5ac5700 1 -- 172.21.15.32:0/2249475674 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f98c00b1fc0 msgr2=0x7f98c0155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:01.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.187+0000 7f98c5ac5700 1 --2- 172.21.15.32:0/2249475674 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f98c00b1fc0 0x7f98c0155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:01.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.187+0000 7f98c5ac5700 1 -- 172.21.15.32:0/2249475674 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f98c0147400 msgr2=0x7f98c0147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:01.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.187+0000 7f98c5ac5700 1 --2- 172.21.15.32:0/2249475674 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f98c0147400 0x7f98c0147e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:01.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.187+0000 7f98c5ac5700 1 -- 172.21.15.32:0/2249475674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c014a7d0 con 0x7f98c00ab710 2020-12-05T17:40:01.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.187+0000 7f98c5ac5700 1 --2- 172.21.15.32:0/2249475674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98c00ab710 0x7f98c00ad680 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7f98c00ae1e0 tx=0x7f98bc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:01.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.187+0000 7f98b2ffd700 1 -- 172.21.15.32:0/2249475674 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98bc011aa0 con 0x7f98c00ab710 2020-12-05T17:40:01.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.187+0000 7f98b2ffd700 1 -- 172.21.15.32:0/2249475674 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98bc011c00 con 0x7f98c00ab710 2020-12-05T17:40:01.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.187+0000 7f98c6ac7700 1 -- 172.21.15.32:0/2249475674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c00abbf0 con 0x7f98c00ab710 2020-12-05T17:40:01.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.187+0000 7f98b2ffd700 1 -- 172.21.15.32:0/2249475674 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98bc01d450 con 0x7f98c00ab710 2020-12-05T17:40:01.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.187+0000 7f98c6ac7700 1 -- 172.21.15.32:0/2249475674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c00acf90 con 0x7f98c00ab710 2020-12-05T17:40:01.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.187+0000 7f98c6ac7700 1 -- 172.21.15.32:0/2249475674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c00abdf0 con 0x7f98c00ab710 2020-12-05T17:40:01.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.191+0000 7f98b2ffd700 1 -- 172.21.15.32:0/2249475674 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f98bc024030 con 0x7f98c00ab710 2020-12-05T17:40:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.191+0000 7f98b2ffd700 1 --2- 172.21.15.32:0/2249475674 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f98a00553b0 0x7f98a0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.191+0000 7f98c52c4700 1 --2- 172.21.15.32:0/2249475674 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f98a00553b0 0x7f98a0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.191+0000 7f98b2ffd700 1 -- 172.21.15.32:0/2249475674 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f98bc06f0f0 con 0x7f98c00ab710 2020-12-05T17:40:01.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.191+0000 7f98c52c4700 1 --2- 172.21.15.32:0/2249475674 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f98a00553b0 0x7f98a0057870 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f98b400d000 tx=0x7f98b4007aa0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:01.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.195+0000 7f98b2ffd700 1 -- 172.21.15.32:0/2249475674 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f98bc045290 con 0x7f98c00ab710 2020-12-05T17:40:01.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:01.387+0000 7f98c6ac7700 1 -- 172.21.15.32:0/2249475674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new"]} v 0) v1 -- 0x7f98c0067cd0 con 0x7f98c00ab710 2020-12-05T17:40:02.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.035+0000 7f98b2ffd700 1 -- 172.21.15.32:0/2249475674 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new"]}]=0 v376) v1 ==== 125+0+0 (secure 0 0 0) 0x7f98bc022020 con 0x7f98c00ab710 2020-12-05T17:40:02.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.039+0000 7f98c6ac7700 1 -- 172.21.15.32:0/2249475674 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f98a00553b0 msgr2=0x7f98a0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:02.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.039+0000 7f98c6ac7700 1 --2- 172.21.15.32:0/2249475674 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f98a00553b0 0x7f98a0057870 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f98b400d000 tx=0x7f98b4007aa0).stop 2020-12-05T17:40:02.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.039+0000 7f98c6ac7700 1 -- 172.21.15.32:0/2249475674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98c00ab710 msgr2=0x7f98c00ad680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:02.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.039+0000 7f98c6ac7700 1 --2- 172.21.15.32:0/2249475674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98c00ab710 0x7f98c00ad680 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7f98c00ae1e0 tx=0x7f98bc015000).stop 2020-12-05T17:40:02.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.039+0000 7f98c6ac7700 1 -- 172.21.15.32:0/2249475674 shutdown_connections 2020-12-05T17:40:02.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.039+0000 7f98c6ac7700 1 --2- 172.21.15.32:0/2249475674 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f98c0147400 0x7f98c0147e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:02.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.039+0000 7f98c6ac7700 1 --2- 172.21.15.32:0/2249475674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98c00ab710 0x7f98c00ad680 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:02.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.039+0000 7f98c6ac7700 1 --2- 172.21.15.32:0/2249475674 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f98a00553b0 0x7f98a0057870 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:02.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.039+0000 7f98c6ac7700 1 --2- 172.21.15.32:0/2249475674 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f98c00b1fc0 0x7f98c0155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:02.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.039+0000 7f98c6ac7700 1 -- 172.21.15.32:0/2249475674 >> 172.21.15.32:0/2249475674 conn(0x7f98c00b5530 msgr2=0x7f98c00afea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:02.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.039+0000 7f98c6ac7700 1 -- 172.21.15.32:0/2249475674 shutdown_connections 2020-12-05T17:40:02.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.039+0000 7f98c6ac7700 1 -- 172.21.15.32:0/2249475674 wait complete. 2020-12-05T17:40:02.064 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-12-05T17:40:02.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3f1ae8700 1 Processor -- start 2020-12-05T17:40:02.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3f1ae8700 1 -- start start 2020-12-05T17:40:02.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3f1ae8700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fe3ec1402f0 0x7fe3ec05ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:02.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3f1ae8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3ec05fef0 con 0x7fe3ec0b0de0 2020-12-05T17:40:02.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3f1ae8700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3ec05d060 con 0x7fe3ec05fb10 2020-12-05T17:40:02.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3f1ae8700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3ec0af080 con 0x7fe3ec1402f0 2020-12-05T17:40:02.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3ebfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe3ec0b0de0 0x7fe3ec143ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41712/0 (socket says 172.21.15.32:41712) 2020-12-05T17:40:02.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3ebfff700 1 -- 172.21.15.32:0/2830781310 learned_addr learned my addr 172.21.15.32:0/2830781310 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:02.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3e2ffd700 1 --2- 172.21.15.32:0/2830781310 >> v2:172.21.15.61:3300/0 conn(0x7fe3ec1402f0 0x7fe3ec05ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:02.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3eaffd700 1 -- 172.21.15.32:0/2830781310 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3086812893 0 0) 0x7fe3ec05fef0 con 0x7fe3ec0b0de0 2020-12-05T17:40:02.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3eaffd700 1 -- 172.21.15.32:0/2830781310 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3cc003230 con 0x7fe3ec0b0de0 2020-12-05T17:40:02.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3eaffd700 1 -- 172.21.15.32:0/2830781310 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1437993867 0 0) 0x7fe3ec05d060 con 0x7fe3ec05fb10 2020-12-05T17:40:02.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3eaffd700 1 -- 172.21.15.32:0/2830781310 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3ec05fef0 con 0x7fe3ec05fb10 2020-12-05T17:40:02.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3eaffd700 1 -- 172.21.15.32:0/2830781310 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1492178350 0 0) 0x7fe3cc003230 con 0x7fe3ec0b0de0 2020-12-05T17:40:02.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3eaffd700 1 -- 172.21.15.32:0/2830781310 >> v2:172.21.15.61:3300/0 conn(0x7fe3ec1402f0 msgr2=0x7fe3ec05ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:02.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3eaffd700 1 --2- 172.21.15.32:0/2830781310 >> v2:172.21.15.61:3300/0 conn(0x7fe3ec1402f0 0x7fe3ec05ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:02.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3eaffd700 1 -- 172.21.15.32:0/2830781310 >> v1:172.21.15.61:6789/0 conn(0x7fe3ec05fb10 legacy=0x7fe3ec13fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:02.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3eaffd700 1 -- 172.21.15.32:0/2830781310 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ec0af320 con 0x7fe3ec0b0de0 2020-12-05T17:40:02.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3eaffd700 1 -- 172.21.15.32:0/2830781310 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fe3e4001a50 con 0x7fe3ec0b0de0 2020-12-05T17:40:02.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3eaffd700 1 -- 172.21.15.32:0/2830781310 >> v1:172.21.15.32:6789/0 conn(0x7fe3ec0b0de0 legacy=0x7fe3ec143ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:02.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3eaffd700 1 --2- 172.21.15.32:0/2830781310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3cc003900 0x7fe3cc003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:02.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3eaffd700 1 --2- 172.21.15.32:0/2830781310 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3cc0044d0 0x7fe3cc004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:02.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3eaffd700 1 --2- 172.21.15.32:0/2830781310 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3cc004e40 0x7fe3cc00b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:02.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3eaffd700 1 -- 172.21.15.32:0/2830781310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec0af080 con 0x7fe3cc003900 2020-12-05T17:40:02.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3eaffd700 1 -- 172.21.15.32:0/2830781310 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc00bc10 con 0x7fe3cc004e40 2020-12-05T17:40:02.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3eaffd700 1 -- 172.21.15.32:0/2830781310 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc00bd50 con 0x7fe3cc0044d0 2020-12-05T17:40:02.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3eb7fe700 1 --2- 172.21.15.32:0/2830781310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3cc003900 0x7fe3cc003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:02.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3ebfff700 1 --2- 172.21.15.32:0/2830781310 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3cc004e40 0x7fe3cc00b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:02.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.183+0000 7fe3e2ffd700 1 --2- 172.21.15.32:0/2830781310 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3cc0044d0 0x7fe3cc004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:02.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3eb7fe700 1 -- 172.21.15.32:0/2830781310 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3cc0044d0 msgr2=0x7fe3cc004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:02.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3eb7fe700 1 --2- 172.21.15.32:0/2830781310 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3cc0044d0 0x7fe3cc004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:02.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3eb7fe700 1 -- 172.21.15.32:0/2830781310 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3cc004e40 msgr2=0x7fe3cc00b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:02.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3eb7fe700 1 --2- 172.21.15.32:0/2830781310 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3cc004e40 0x7fe3cc00b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:02.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3eb7fe700 1 -- 172.21.15.32:0/2830781310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc003230 con 0x7fe3cc003900 2020-12-05T17:40:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3ebfff700 1 -- 172.21.15.32:0/2830781310 reap_dead start 2020-12-05T17:40:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3eb7fe700 1 --2- 172.21.15.32:0/2830781310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3cc003900 0x7fe3cc003d00 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7fe3dc0010d0 tx=0x7fe3dc007b60).ready entity=mon.0 client_cookie=1e3ee1d26bbc5f06 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3eaffd700 1 -- 172.21.15.32:0/2830781310 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3dc00c660 con 0x7fe3cc003900 2020-12-05T17:40:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3eaffd700 1 -- 172.21.15.32:0/2830781310 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3dc00c7c0 con 0x7fe3cc003900 2020-12-05T17:40:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3eaffd700 1 -- 172.21.15.32:0/2830781310 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3dc00cb20 con 0x7fe3cc003900 2020-12-05T17:40:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3f1ae8700 1 -- 172.21.15.32:0/2830781310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3cc003900 msgr2=0x7fe3cc003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3f1ae8700 1 --2- 172.21.15.32:0/2830781310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3cc003900 0x7fe3cc003d00 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7fe3dc0010d0 tx=0x7fe3dc007b60).stop 2020-12-05T17:40:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3f1ae8700 1 -- 172.21.15.32:0/2830781310 shutdown_connections 2020-12-05T17:40:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3f1ae8700 1 --2- 172.21.15.32:0/2830781310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3cc003900 0x7fe3cc003d00 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3f1ae8700 1 -- 172.21.15.32:0/2830781310 >> 172.21.15.32:0/2830781310 conn(0x7fe3ec069cb0 msgr2=0x7fe3ec0604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3f1ae8700 1 -- 172.21.15.32:0/2830781310 shutdown_connections 2020-12-05T17:40:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3f1ae8700 1 -- 172.21.15.32:0/2830781310 wait complete. 2020-12-05T17:40:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3f1ae8700 1 Processor -- start 2020-12-05T17:40:02.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3f1ae8700 1 -- start start 2020-12-05T17:40:02.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3f1ae8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3cc003900 0x7fe3ec143f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:02.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3f1ae8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3ec1555a0 0x7fe3ec0645c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:02.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3f1ae8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3ec13d7a0 0x7fe3ec064ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:02.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3f1ae8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec0ace70 con 0x7fe3ec13d7a0 2020-12-05T17:40:02.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3f1ae8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec065e00 con 0x7fe3ec1555a0 2020-12-05T17:40:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.187+0000 7fe3f1ae8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec0ad0a0 con 0x7fe3cc003900 2020-12-05T17:40:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.191+0000 7fe3ebfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3ec13d7a0 0x7fe3ec064ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.191+0000 7fe3ebfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3ec13d7a0 0x7fe3ec064ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39076/0 (socket says 172.21.15.32:39076) 2020-12-05T17:40:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.191+0000 7fe3ebfff700 1 -- 172.21.15.32:0/3782546897 learned_addr learned my addr 172.21.15.32:0/3782546897 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.191+0000 7fe3eb7fe700 1 --2- 172.21.15.32:0/3782546897 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3cc003900 0x7fe3ec143f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:02.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.191+0000 7fe3e2ffd700 1 --2- 172.21.15.32:0/3782546897 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3ec1555a0 0x7fe3ec0645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:02.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.191+0000 7fe3ebfff700 1 -- 172.21.15.32:0/3782546897 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3cc003900 msgr2=0x7fe3ec143f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:02.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.191+0000 7fe3ebfff700 1 --2- 172.21.15.32:0/3782546897 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3cc003900 0x7fe3ec143f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:02.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.191+0000 7fe3ebfff700 1 -- 172.21.15.32:0/3782546897 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3ec1555a0 msgr2=0x7fe3ec0645c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:02.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.191+0000 7fe3ebfff700 1 --2- 172.21.15.32:0/3782546897 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3ec1555a0 0x7fe3ec0645c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:02.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.191+0000 7fe3ebfff700 1 -- 172.21.15.32:0/3782546897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ec063120 con 0x7fe3ec13d7a0 2020-12-05T17:40:02.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.191+0000 7fe3eb7fe700 1 --2- 172.21.15.32:0/3782546897 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3cc003900 0x7fe3ec143f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:40:02.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.191+0000 7fe3ebfff700 1 --2- 172.21.15.32:0/3782546897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3ec13d7a0 0x7fe3ec064ab0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0x7fe3ec0b2a40 tx=0x7fe3e40021d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:02.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.191+0000 7fe3e97fa700 1 -- 172.21.15.32:0/3782546897 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3e400fc10 con 0x7fe3ec13d7a0 2020-12-05T17:40:02.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.191+0000 7fe3e97fa700 1 -- 172.21.15.32:0/3782546897 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e400fd70 con 0x7fe3ec13d7a0 2020-12-05T17:40:02.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.191+0000 7fe3f1ae8700 1 -- 172.21.15.32:0/3782546897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3ec064fa0 con 0x7fe3ec13d7a0 2020-12-05T17:40:02.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.191+0000 7fe3e97fa700 1 -- 172.21.15.32:0/3782546897 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3e400fee0 con 0x7fe3ec13d7a0 2020-12-05T17:40:02.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.191+0000 7fe3f1ae8700 1 -- 172.21.15.32:0/3782546897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3ec145d80 con 0x7fe3ec13d7a0 2020-12-05T17:40:02.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.191+0000 7fe3e97fa700 1 -- 172.21.15.32:0/3782546897 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe3e401b030 con 0x7fe3ec13d7a0 2020-12-05T17:40:02.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.191+0000 7fe3f1ae8700 1 -- 172.21.15.32:0/3782546897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3ec1553c0 con 0x7fe3ec13d7a0 2020-12-05T17:40:02.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.191+0000 7fe3e97fa700 1 --2- 172.21.15.32:0/3782546897 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe3d0055440 0x7fe3d0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:02.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.191+0000 7fe3e97fa700 1 -- 172.21.15.32:0/3782546897 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe3e406d920 con 0x7fe3ec13d7a0 2020-12-05T17:40:02.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.191+0000 7fe3eb7fe700 1 --2- 172.21.15.32:0/3782546897 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe3d0055440 0x7fe3d0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:02.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.195+0000 7fe3eb7fe700 1 --2- 172.21.15.32:0/3782546897 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe3d0055440 0x7fe3d0057900 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fe3dc019000 tx=0x7fe3dc017000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:02.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.199+0000 7fe3e97fa700 1 -- 172.21.15.32:0/3782546897 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe3e4043ac0 con 0x7fe3ec13d7a0 2020-12-05T17:40:02.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.339+0000 7fe3f1ae8700 1 -- 172.21.15.32:0/3782546897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7fe3ec057050 con 0x7fe3ec13d7a0 2020-12-05T17:40:02.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.347+0000 7fe3e97fa700 1 -- 172.21.15.32:0/3782546897 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe3e4019020 con 0x7fe3ec13d7a0 2020-12-05T17:40:02.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.351+0000 7fe3f1ae8700 1 -- 172.21.15.32:0/3782546897 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe3d0055440 msgr2=0x7fe3d0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:02.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.351+0000 7fe3f1ae8700 1 --2- 172.21.15.32:0/3782546897 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe3d0055440 0x7fe3d0057900 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fe3dc019000 tx=0x7fe3dc017000).stop 2020-12-05T17:40:02.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.351+0000 7fe3f1ae8700 1 -- 172.21.15.32:0/3782546897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3ec13d7a0 msgr2=0x7fe3ec064ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:02.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.351+0000 7fe3f1ae8700 1 --2- 172.21.15.32:0/3782546897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3ec13d7a0 0x7fe3ec064ab0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0x7fe3ec0b2a40 tx=0x7fe3e40021d0).stop 2020-12-05T17:40:02.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.351+0000 7fe3f1ae8700 1 -- 172.21.15.32:0/3782546897 shutdown_connections 2020-12-05T17:40:02.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.351+0000 7fe3f1ae8700 1 --2- 172.21.15.32:0/3782546897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3ec13d7a0 0x7fe3ec064ab0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:02.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.351+0000 7fe3f1ae8700 1 --2- 172.21.15.32:0/3782546897 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe3d0055440 0x7fe3d0057900 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:02.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.351+0000 7fe3f1ae8700 1 --2- 172.21.15.32:0/3782546897 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3cc003900 0x7fe3ec143f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:02.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.351+0000 7fe3f1ae8700 1 --2- 172.21.15.32:0/3782546897 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3ec1555a0 0x7fe3ec0645c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:02.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.351+0000 7fe3f1ae8700 1 -- 172.21.15.32:0/3782546897 >> 172.21.15.32:0/3782546897 conn(0x7fe3ec069cb0 msgr2=0x7fe3ec0b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:02.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.351+0000 7fe3f1ae8700 1 -- 172.21.15.32:0/3782546897 shutdown_connections 2020-12-05T17:40:02.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.351+0000 7fe3f1ae8700 1 -- 172.21.15.32:0/3782546897 wait complete. 2020-12-05T17:40:02.370 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:40:02.370 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:40:02.389 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:40:02.407 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:40:02.424 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:40:02.438 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:40:02.457 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:40:02.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:02.458 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:40:02.459 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:40:02.478 INFO:teuthology.orchestra.run.smithi036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-12-05T17:40:02.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:40:02.480 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-12-05T17:40:02.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:02.481 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:40:02.481 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:40:02.488 INFO:teuthology.orchestra.run.smithi002.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-12-05T17:40:02.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:40:02.490 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-12-05T17:40:02.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:02.492 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:40:02.493 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:40:02.501 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:40:02.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:40:02.502 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2020-12-05T17:40:02.503 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-12-05T17:40:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.615+0000 7fcc8a014700 1 Processor -- start 2020-12-05T17:40:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc8a014700 1 -- start start 2020-12-05T17:40:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc8a014700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcc8405f6f0 0x7fcc840b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc8a014700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc840afea0 0x7fcc841409c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:02.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc8a014700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc8405fad0 con 0x7fcc840af920 2020-12-05T17:40:02.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc8a014700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc840b2cd0 con 0x7fcc840afea0 2020-12-05T17:40:02.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc8a014700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc840b3560 con 0x7fcc8405f6f0 2020-12-05T17:40:02.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc89813700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc840afea0 0x7fcc841409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:02.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc89813700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc840afea0 0x7fcc841409c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39084/0 (socket says 172.21.15.32:39084) 2020-12-05T17:40:02.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc89813700 1 -- 172.21.15.32:0/2795871855 learned_addr learned my addr 172.21.15.32:0/2795871855 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:02.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc89012700 1 --2- 172.21.15.32:0/2795871855 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcc8405f6f0 0x7fcc840b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:02.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc89813700 1 -- 172.21.15.32:0/2795871855 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcc8405f6f0 msgr2=0x7fcc840b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:02.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc89813700 1 --2- 172.21.15.32:0/2795871855 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcc8405f6f0 0x7fcc840b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:02.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc89813700 1 -- 172.21.15.32:0/2795871855 >> v1:172.21.15.61:6789/0 conn(0x7fcc840af920 legacy=0x7fcc84063040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:02.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc89813700 1 -- 172.21.15.32:0/2795871855 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc84144650 con 0x7fcc840afea0 2020-12-05T17:40:02.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc89012700 1 --2- 172.21.15.32:0/2795871855 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcc8405f6f0 0x7fcc840b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:40:02.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc89813700 1 --2- 172.21.15.32:0/2795871855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc840afea0 0x7fcc841409c0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7fcc8406a370 tx=0x7fcc7c009270).ready entity=mon.0 client_cookie=24c622c38510c008 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc7bfff700 1 -- 172.21.15.32:0/2795871855 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc7c00bbc0 con 0x7fcc840afea0 2020-12-05T17:40:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc7bfff700 1 -- 172.21.15.32:0/2795871855 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7c00bd20 con 0x7fcc840afea0 2020-12-05T17:40:02.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc7bfff700 1 -- 172.21.15.32:0/2795871855 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc7c011400 con 0x7fcc840afea0 2020-12-05T17:40:02.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc8a014700 1 -- 172.21.15.32:0/2795871855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc840afea0 msgr2=0x7fcc841409c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:02.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc8a014700 1 --2- 172.21.15.32:0/2795871855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc840afea0 0x7fcc841409c0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7fcc8406a370 tx=0x7fcc7c009270).stop 2020-12-05T17:40:02.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc8a014700 1 -- 172.21.15.32:0/2795871855 shutdown_connections 2020-12-05T17:40:02.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc8a014700 1 --2- 172.21.15.32:0/2795871855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc840afea0 0x7fcc841409c0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:02.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc8a014700 1 --2- 172.21.15.32:0/2795871855 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcc8405f6f0 0x7fcc840b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:02.640 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc8a014700 1 -- 172.21.15.32:0/2795871855 >> 172.21.15.32:0/2795871855 conn(0x7fcc840604c0 msgr2=0x7fcc8405c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:02.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc8a014700 1 -- 172.21.15.32:0/2795871855 shutdown_connections 2020-12-05T17:40:02.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.619+0000 7fcc8a014700 1 -- 172.21.15.32:0/2795871855 wait complete. 2020-12-05T17:40:02.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.623+0000 7fcc8a014700 1 Processor -- start 2020-12-05T17:40:02.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.623+0000 7fcc8a014700 1 -- start start 2020-12-05T17:40:02.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.627+0000 7fcc8a014700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcc8405f6f0 0x7fcc84063410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:02.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.627+0000 7fcc8a014700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc840af920 0x7fcc840b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:02.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.627+0000 7fcc8a014700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc840afea0 0x7fcc840b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:02.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.627+0000 7fcc8a014700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc840649d0 con 0x7fcc840afea0 2020-12-05T17:40:02.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.627+0000 7fcc8a014700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8414fda0 con 0x7fcc840af920 2020-12-05T17:40:02.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.627+0000 7fcc8a014700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc84150040 con 0x7fcc8405f6f0 2020-12-05T17:40:02.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.627+0000 7fcc89813700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc840afea0 0x7fcc840b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:02.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.627+0000 7fcc89012700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcc8405f6f0 0x7fcc84063410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:02.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.627+0000 7fcc89813700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc840afea0 0x7fcc840b4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39090/0 (socket says 172.21.15.32:39090) 2020-12-05T17:40:02.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.627+0000 7fcc89813700 1 -- 172.21.15.32:0/2699917380 learned_addr learned my addr 172.21.15.32:0/2699917380 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:02.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.627+0000 7fcc88811700 1 --2- 172.21.15.32:0/2699917380 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc840af920 0x7fcc840b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:02.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.627+0000 7fcc89813700 1 -- 172.21.15.32:0/2699917380 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcc8405f6f0 msgr2=0x7fcc84063410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:02.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.627+0000 7fcc89813700 1 --2- 172.21.15.32:0/2699917380 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcc8405f6f0 0x7fcc84063410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:02.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.627+0000 7fcc89813700 1 -- 172.21.15.32:0/2699917380 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc840af920 msgr2=0x7fcc840b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:02.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.627+0000 7fcc89813700 1 --2- 172.21.15.32:0/2699917380 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc840af920 0x7fcc840b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:02.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.627+0000 7fcc89813700 1 -- 172.21.15.32:0/2699917380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc84063900 con 0x7fcc840afea0 2020-12-05T17:40:02.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.627+0000 7fcc89813700 1 --2- 172.21.15.32:0/2699917380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc840afea0 0x7fcc840b4ee0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 rx=0x7fcc8406a370 tx=0x7fcc7c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:02.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.627+0000 7fcc7a7fc700 1 -- 172.21.15.32:0/2699917380 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc7c011480 con 0x7fcc840afea0 2020-12-05T17:40:02.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.627+0000 7fcc7a7fc700 1 -- 172.21.15.32:0/2699917380 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7c01f5e0 con 0x7fcc840afea0 2020-12-05T17:40:02.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.627+0000 7fcc8a014700 1 -- 172.21.15.32:0/2699917380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc84064650 con 0x7fcc840afea0 2020-12-05T17:40:02.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.627+0000 7fcc7a7fc700 1 -- 172.21.15.32:0/2699917380 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc7c01f940 con 0x7fcc840afea0 2020-12-05T17:40:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.627+0000 7fcc8a014700 1 -- 172.21.15.32:0/2699917380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc84063d30 con 0x7fcc840afea0 2020-12-05T17:40:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.627+0000 7fcc7a7fc700 1 -- 172.21.15.32:0/2699917380 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fcc7c026030 con 0x7fcc840afea0 2020-12-05T17:40:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.631+0000 7fcc8a014700 1 -- 172.21.15.32:0/2699917380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc84057050 con 0x7fcc840afea0 2020-12-05T17:40:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.631+0000 7fcc7a7fc700 1 --2- 172.21.15.32:0/2699917380 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcc6c0553b0 0x7fcc6c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.631+0000 7fcc7a7fc700 1 -- 172.21.15.32:0/2699917380 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcc7c06ecf0 con 0x7fcc840afea0 2020-12-05T17:40:02.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.631+0000 7fcc89012700 1 --2- 172.21.15.32:0/2699917380 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcc6c0553b0 0x7fcc6c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:02.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.631+0000 7fcc89012700 1 --2- 172.21.15.32:0/2699917380 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcc6c0553b0 0x7fcc6c057870 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7fcc8000d000 tx=0x7fcc80008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:02.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.635+0000 7fcc7a7fc700 1 -- 172.21.15.32:0/2699917380 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fcc7c044ed0 con 0x7fcc840afea0 2020-12-05T17:40:02.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.779+0000 7fcc8a014700 1 -- 172.21.15.32:0/2699917380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fcc84057050 con 0x7fcc840afea0 2020-12-05T17:40:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.783+0000 7fcc7a7fc700 1 -- 172.21.15.32:0/2699917380 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v102) v1 ==== 81+0+34048 (secure 0 0 0) 0x7fcc7c015090 con 0x7fcc840afea0 2020-12-05T17:40:02.792 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:40:02.793 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:40:02.793 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 102, 2020-12-05T17:40:02.793 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", 2020-12-05T17:40:02.794 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2020-12-05T17:31:20.975238+0000", 2020-12-05T17:40:02.794 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2020-12-05T17:39:48.474575+0000", 2020-12-05T17:40:02.794 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:40:02.795 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:40:02.795 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-12-05T17:40:02.795 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2020-12-05T17:40:02.796 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2020-12-05T17:40:02.796 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2020-12-05T17:40:02.796 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2020-12-05T17:40:02.797 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2020-12-05T17:40:02.797 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2020-12-05T17:40:02.798 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:40:02.798 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2020-12-05T17:40:02.798 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2020-12-05T17:40:02.799 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-12-05T17:40:02.799 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2020-12-05T17:40:02.799 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2020-12-05T17:40:02.800 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 20, 2020-12-05T17:40:02.800 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2020-12-05T17:40:02.800 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2020-12-05T17:40:02.801 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2020-12-05T17:40:02.801 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2020-12-05T17:40:02.802 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2020-12-05T17:40:02.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.803 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2020-12-05T17:40:02.803 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2020-12-05T17:40:02.804 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:24.858732+0000", 2020-12-05T17:40:02.804 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:40:02.804 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:40:02.805 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:40:02.806 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:40:02.806 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:40:02.807 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:40:02.807 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:40:02.808 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:40:02.809 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2020-12-05T17:40:02.809 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2020-12-05T17:40:02.810 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2020-12-05T17:40:02.810 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2020-12-05T17:40:02.810 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2020-12-05T17:40:02.811 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:40:02.811 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:40:02.811 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:40:02.811 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:40:02.811 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:40:02.812 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:40:02.812 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:40:02.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.812 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "16", 2020-12-05T17:40:02.812 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:40:02.813 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:40:02.813 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:40:02.813 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:40:02.813 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:40:02.814 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:40:02.814 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:40:02.814 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:40:02.814 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:40:02.814 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:40:02.815 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:40:02.815 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:40:02.815 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:40:02.815 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:40:02.815 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:40:02.816 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:40:02.816 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:40:02.816 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:40:02.816 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:40:02.817 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:40:02.817 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:40:02.817 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:40:02.817 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:40:02.818 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:40:02.818 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:40:02.818 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:40:02.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.819 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:40:02.819 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:40:02.820 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:40:02.820 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:40:02.820 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:40:02.820 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:40:02.821 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:40:02.821 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:40:02.821 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:40:02.821 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:40:02.821 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:40:02.822 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:40:02.822 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2020-12-05T17:40:02.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.822 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:40:02.822 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2020-12-05T17:40:02.823 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.823 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2020-12-05T17:40:02.823 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2020-12-05T17:40:02.824 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:33.305595+0000", 2020-12-05T17:40:02.824 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:40:02.824 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:40:02.824 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:40:02.825 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:40:02.825 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:40:02.825 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:40:02.825 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:40:02.825 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:40:02.826 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:40:02.826 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:40:02.826 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:40:02.826 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:40:02.827 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:40:02.827 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:40:02.827 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:40:02.827 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:40:02.827 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:40:02.828 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:40:02.828 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:40:02.828 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:40:02.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.828 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "19", 2020-12-05T17:40:02.829 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:40:02.829 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:40:02.829 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:40:02.830 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:40:02.830 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:40:02.830 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:40:02.831 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:40:02.831 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:40:02.831 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:40:02.831 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:40:02.831 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:40:02.832 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:40:02.832 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:40:02.832 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:40:02.833 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:40:02.833 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:40:02.833 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:40:02.833 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:40:02.834 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:40:02.834 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:40:02.834 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:40:02.834 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:40:02.834 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:40:02.835 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:40:02.835 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:40:02.835 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:40:02.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.835 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:40:02.836 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:40:02.836 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:40:02.836 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:40:02.836 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:40:02.837 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:40:02.837 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:40:02.837 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:40:02.837 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:40:02.837 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:40:02.838 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:40:02.838 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:40:02.838 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:40:02.838 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2020-12-05T17:40:02.838 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.839 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 19, 2020-12-05T17:40:02.839 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2020-12-05T17:40:02.840 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:39:45.190638+0000", 2020-12-05T17:40:02.840 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:40:02.840 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:40:02.840 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:40:02.841 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:40:02.841 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:40:02.841 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:40:02.841 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:40:02.841 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:40:02.842 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:40:02.842 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:40:02.842 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:40:02.842 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:40:02.843 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:40:02.843 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:40:02.843 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:40:02.843 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:40:02.843 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:40:02.844 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:40:02.844 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:40:02.844 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:40:02.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.845 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "101", 2020-12-05T17:40:02.845 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:40:02.845 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:40:02.845 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:40:02.845 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:40:02.846 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:40:02.846 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:40:02.846 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:40:02.846 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:40:02.847 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:40:02.847 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:40:02.848 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:40:02.848 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:40:02.848 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:40:02.848 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:40:02.849 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:40:02.849 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:40:02.849 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:40:02.849 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:40:02.849 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:40:02.850 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:40:02.850 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:40:02.850 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:40:02.850 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:40:02.851 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:40:02.851 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:40:02.851 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:40:02.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.851 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:40:02.852 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:40:02.852 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:40:02.852 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:40:02.852 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:40:02.853 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:40:02.853 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:40:02.853 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:40:02.853 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:40:02.854 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:40:02.854 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:40:02.854 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:40:02.855 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2020-12-05T17:40:02.855 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2020-12-05T17:40:02.855 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2020-12-05T17:40:02.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.856 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:40:02.856 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:40:02.856 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2020-12-05T17:40:02.856 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.857 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.857 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 20, 2020-12-05T17:40:02.857 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2020-12-05T17:40:02.858 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:39:46.472081+0000", 2020-12-05T17:40:02.858 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:40:02.858 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:40:02.858 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:40:02.858 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:40:02.859 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:40:02.859 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:40:02.860 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:40:02.860 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:40:02.860 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:40:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:40:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:40:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:40:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:40:02.862 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:40:02.862 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:40:02.862 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:40:02.862 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:40:02.862 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:40:02.863 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:40:02.863 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:40:02.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.863 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "102", 2020-12-05T17:40:02.864 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:40:02.865 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:40:02.865 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:40:02.865 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:40:02.865 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:40:02.866 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:40:02.866 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:40:02.866 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:40:02.866 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:40:02.867 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:40:02.867 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:40:02.867 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:40:02.867 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:40:02.868 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:40:02.868 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:40:02.868 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:40:02.868 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:40:02.868 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:40:02.869 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:40:02.869 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:40:02.869 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:40:02.870 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:40:02.870 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:40:02.870 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:40:02.870 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:40:02.871 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:40:02.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.872 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:40:02.872 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:40:02.872 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:40:02.873 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:40:02.873 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:40:02.874 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:40:02.874 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:40:02.874 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:40:02.874 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:40:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:40:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:40:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:40:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:40:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:40:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2020-12-05T17:40:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:40:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2020-12-05T17:40:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:40:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "b941127b-0a49-4e06-8d11-17794ff8b013", 2020-12-05T17:40:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:40:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:40:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:40:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:40:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:40:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:40:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:40:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 100, 2020-12-05T17:40:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:40:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:40:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:40:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2020-12-05T17:40:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:40:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.883 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.883 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2020-12-05T17:40:02.883 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:40:02.883 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.884 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.884 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:40:02.885 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.886 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.886 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2020-12-05T17:40:02.887 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:40:02.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.889 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.889 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2020-12-05T17:40:02.890 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:40:02.890 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.890 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.891 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:40:02.891 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.893 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.893 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2020-12-05T17:40:02.893 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:40:02.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.895 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.896 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2020-12-05T17:40:02.896 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:40:02.896 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.897 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.897 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:40:02.898 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.898 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.899 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2020-12-05T17:40:02.899 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:40:02.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.900 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.900 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2020-12-05T17:40:02.901 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:40:02.901 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.901 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.903 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6803/11526", 2020-12-05T17:40:02.903 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6805/11526", 2020-12-05T17:40:02.903 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6809/11526", 2020-12-05T17:40:02.904 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6807/11526", 2020-12-05T17:40:02.904 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:40:02.905 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:40:02.905 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:40:02.905 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.906 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:40:02.907 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "01bd7ff5-a86b-4e11-ba1e-484c309c0fba", 2020-12-05T17:40:02.907 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:40:02.907 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:40:02.908 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:40:02.908 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:40:02.908 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:40:02.909 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:40:02.909 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:40:02.909 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 100, 2020-12-05T17:40:02.909 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:40:02.910 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:40:02.910 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:40:02.910 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.911 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.911 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2020-12-05T17:40:02.911 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:40:02.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.912 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.912 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2020-12-05T17:40:02.912 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:40:02.913 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.913 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.913 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:40:02.914 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.914 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.915 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2020-12-05T17:40:02.915 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:40:02.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.915 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.916 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2020-12-05T17:40:02.916 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:40:02.916 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.916 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.917 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:40:02.917 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.917 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.918 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2020-12-05T17:40:02.918 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:40:02.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.919 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.919 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2020-12-05T17:40:02.919 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:40:02.920 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.920 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.920 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:40:02.921 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.921 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.921 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2020-12-05T17:40:02.921 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:40:02.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.922 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.923 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2020-12-05T17:40:02.923 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:40:02.923 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.924 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.924 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6819/11538", 2020-12-05T17:40:02.924 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6821/11538", 2020-12-05T17:40:02.925 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6825/11538", 2020-12-05T17:40:02.925 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6823/11538", 2020-12-05T17:40:02.925 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:40:02.925 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:40:02.926 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:40:02.926 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.927 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:40:02.927 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "94736ef5-64c2-4a97-b963-bd38cddd9c52", 2020-12-05T17:40:02.927 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:40:02.927 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:40:02.928 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:40:02.928 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:40:02.928 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:40:02.929 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:40:02.929 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:40:02.929 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 99, 2020-12-05T17:40:02.930 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:40:02.930 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:40:02.930 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:40:02.930 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.931 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.931 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2020-12-05T17:40:02.931 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:40:02.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.932 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.932 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2020-12-05T17:40:02.932 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:40:02.933 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.933 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.934 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:40:02.934 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.935 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.935 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2020-12-05T17:40:02.936 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:40:02.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.937 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.937 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2020-12-05T17:40:02.937 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:40:02.938 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.938 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.939 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:40:02.939 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.940 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.940 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2020-12-05T17:40:02.940 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:40:02.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.941 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.942 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2020-12-05T17:40:02.942 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:40:02.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.942 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.943 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:40:02.943 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.944 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.944 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2020-12-05T17:40:02.944 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:40:02.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.945 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.945 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2020-12-05T17:40:02.945 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:40:02.945 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.945 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.946 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6811/11521", 2020-12-05T17:40:02.946 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6813/11521", 2020-12-05T17:40:02.946 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6817/11521", 2020-12-05T17:40:02.946 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6815/11521", 2020-12-05T17:40:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:40:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:40:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:40:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:40:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "69a1f3ff-fd25-43df-b60c-e32a89c35234", 2020-12-05T17:40:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:40:02.949 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:40:02.949 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:40:02.949 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:40:02.949 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:40:02.949 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:40:02.950 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:40:02.950 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 100, 2020-12-05T17:40:02.950 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:40:02.950 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:40:02.950 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:40:02.951 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.951 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.951 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2020-12-05T17:40:02.951 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:40:02.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.952 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.952 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2020-12-05T17:40:02.952 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:40:02.953 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.953 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.953 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:40:02.953 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.954 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2020-12-05T17:40:02.954 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:40:02.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.955 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.955 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2020-12-05T17:40:02.955 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:40:02.956 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.956 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.956 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:40:02.957 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.957 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.957 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6832", 2020-12-05T17:40:02.957 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:40:02.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.958 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.958 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6833", 2020-12-05T17:40:02.958 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:40:02.959 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.959 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.959 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:40:02.959 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.960 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.960 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2020-12-05T17:40:02.960 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:40:02.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.961 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.961 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2020-12-05T17:40:02.961 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:40:02.961 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.962 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.962 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6827/11535", 2020-12-05T17:40:02.962 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6829/11535", 2020-12-05T17:40:02.963 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6833/11535", 2020-12-05T17:40:02.963 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6831/11535", 2020-12-05T17:40:02.963 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:40:02.963 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:40:02.963 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:40:02.964 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.964 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:40:02.964 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c301fe4e-5eab-402c-ae6f-db3e6a7181af", 2020-12-05T17:40:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:40:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:40:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:40:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:40:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:40:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:40:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:40:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 99, 2020-12-05T17:40:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:40:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:40:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:40:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6808", 2020-12-05T17:40:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:40:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6809", 2020-12-05T17:40:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:40:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:40:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6810", 2020-12-05T17:40:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:40:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6811", 2020-12-05T17:40:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:40:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:40:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6817", 2020-12-05T17:40:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:40:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6819", 2020-12-05T17:40:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:40:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:40:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.976 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6812", 2020-12-05T17:40:02.976 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:40:02.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6814", 2020-12-05T17:40:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:40:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6809/11541", 2020-12-05T17:40:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6811/11541", 2020-12-05T17:40:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6819/11541", 2020-12-05T17:40:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6814/11541", 2020-12-05T17:40:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:40:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:40:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:40:02.980 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.980 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:40:02.980 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "9476eff4-345b-46c5-a473-1d9a89a6d327", 2020-12-05T17:40:02.981 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:40:02.981 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:40:02.981 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:40:02.981 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:40:02.981 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:40:02.982 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:40:02.982 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:40:02.982 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 99, 2020-12-05T17:40:02.982 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:40:02.983 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:40:02.983 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:40:02.983 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.983 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.984 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6800", 2020-12-05T17:40:02.984 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:40:02.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.984 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.985 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6801", 2020-12-05T17:40:02.985 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:40:02.985 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.985 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.986 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:40:02.986 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.986 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.987 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6802", 2020-12-05T17:40:02.987 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:40:02.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.987 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.988 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6803", 2020-12-05T17:40:02.988 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:40:02.988 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.988 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.989 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:40:02.989 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.989 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.989 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6806", 2020-12-05T17:40:02.990 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:40:02.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.990 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.990 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6807", 2020-12-05T17:40:02.991 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:40:02.991 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.991 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.991 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:40:02.992 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.992 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:02.992 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6804", 2020-12-05T17:40:02.993 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:40:02.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.993 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:02.993 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6805", 2020-12-05T17:40:02.994 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:40:02.994 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:02.994 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.994 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6801/11539", 2020-12-05T17:40:02.995 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6803/11539", 2020-12-05T17:40:02.995 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6807/11539", 2020-12-05T17:40:02.995 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6805/11539", 2020-12-05T17:40:02.995 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:40:02.995 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:40:02.996 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:40:02.996 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:02.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:02.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.996 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:40:02.997 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8dfd8567-9b1c-4d13-9c96-071183e469c2", 2020-12-05T17:40:02.997 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:40:02.997 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:40:02.997 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:40:02.997 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:40:02.998 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:40:02.998 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:40:02.998 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:40:02.998 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 100, 2020-12-05T17:40:02.998 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:40:02.999 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:40:02.999 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:40:02.999 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:02.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:02.999 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:03.000 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6820", 2020-12-05T17:40:03.000 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:40:03.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.000 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:03.001 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6822", 2020-12-05T17:40:03.001 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:40:03.001 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:03.001 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:03.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.002 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:40:03.002 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:03.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.002 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:03.003 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6823", 2020-12-05T17:40:03.003 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:40:03.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.003 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:03.004 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6825", 2020-12-05T17:40:03.004 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:40:03.004 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:03.004 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:03.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.005 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:40:03.005 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:03.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.005 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:03.005 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6829", 2020-12-05T17:40:03.006 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:40:03.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.006 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:03.006 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6830", 2020-12-05T17:40:03.007 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:40:03.007 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:03.007 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:03.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.007 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:40:03.008 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:03.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.008 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:03.008 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6826", 2020-12-05T17:40:03.008 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:40:03.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.009 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:03.009 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6827", 2020-12-05T17:40:03.010 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:40:03.010 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:03.010 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:03.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.010 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6822/11542", 2020-12-05T17:40:03.011 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6825/11542", 2020-12-05T17:40:03.011 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6830/11542", 2020-12-05T17:40:03.011 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6827/11542", 2020-12-05T17:40:03.011 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:40:03.011 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:40:03.012 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:40:03.012 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:03.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.012 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:40:03.013 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "24d466b3-01dc-4ba2-8b81-85250d163bd2", 2020-12-05T17:40:03.013 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:40:03.013 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:40:03.013 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:40:03.013 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:40:03.014 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:40:03.014 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:40:03.014 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:40:03.014 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 100, 2020-12-05T17:40:03.014 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:40:03.015 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:40:03.015 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:40:03.015 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:03.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.015 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:03.016 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6813", 2020-12-05T17:40:03.016 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:40:03.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.017 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:03.018 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6815", 2020-12-05T17:40:03.018 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:40:03.018 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:03.019 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:03.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.020 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:40:03.020 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:03.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.021 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:03.021 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6816", 2020-12-05T17:40:03.022 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:40:03.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.023 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:03.023 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6818", 2020-12-05T17:40:03.024 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:40:03.024 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:03.025 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:03.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.025 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:40:03.026 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:03.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.027 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:03.027 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6828", 2020-12-05T17:40:03.027 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:40:03.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.029 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:03.029 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6831", 2020-12-05T17:40:03.030 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:40:03.030 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:03.030 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:03.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.031 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:40:03.032 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:03.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.032 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:03.033 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6821", 2020-12-05T17:40:03.033 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:40:03.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.035 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:03.035 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6824", 2020-12-05T17:40:03.035 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:40:03.036 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:03.036 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:03.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.037 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6815/11540", 2020-12-05T17:40:03.037 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6818/11540", 2020-12-05T17:40:03.038 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6831/11540", 2020-12-05T17:40:03.038 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6824/11540", 2020-12-05T17:40:03.039 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:40:03.039 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:40:03.039 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:40:03.040 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:03.040 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:03.041 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:40:03.041 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2020-12-05T17:40:03.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.042 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:40:03.042 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:40:03.043 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:40:03.043 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:40:03.044 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:40:03.044 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:40:03.044 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.398868+0000", 2020-12-05T17:40:03.045 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:40:03.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:40:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:40:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:40:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:40:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:40:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:40:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.675748+0000", 2020-12-05T17:40:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:40:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:40:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:40:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:40:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:40:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:40:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:40:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.620888+0000", 2020-12-05T17:40:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:40:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:40:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:40:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:40:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:40:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:40:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:40:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.681600+0000", 2020-12-05T17:40:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:40:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:40:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:40:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:40:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:40:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:40:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:40:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.658769+0000", 2020-12-05T17:40:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:40:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:40:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:40:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:40:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:40:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:40:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:40:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.630708+0000", 2020-12-05T17:40:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:40:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:40:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:40:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:40:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:40:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:40:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:40:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.648352+0000", 2020-12-05T17:40:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:40:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:40:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:40:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:40:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:40:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:40:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:40:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.629399+0000", 2020-12-05T17:40:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:40:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:40:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2020-12-05T17:40:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2020-12-05T17:40:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2020-12-05T17:40:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2020-12-05T17:40:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": {}, 2020-12-05T17:40:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2020-12-05T17:40:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2020-12-05T17:40:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2020-12-05T17:40:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:40:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2020-12-05T17:40:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2020-12-05T17:40:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:40:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2020-12-05T17:40:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:40:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:40:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2020-12-05T17:40:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2020-12-05T17:40:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2020-12-05T17:40:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:40:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:40:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2020-12-05T17:40:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:40:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:40:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2020-12-05T17:40:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2020-12-05T17:40:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2020-12-05T17:40:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2020-12-05T17:40:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2020-12-05T17:40:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2020-12-05T17:40:03.069 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:40:03.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.783+0000 7fcc8a014700 1 -- 172.21.15.32:0/2699917380 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcc6c0553b0 msgr2=0x7fcc6c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:03.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.783+0000 7fcc8a014700 1 --2- 172.21.15.32:0/2699917380 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcc6c0553b0 0x7fcc6c057870 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7fcc8000d000 tx=0x7fcc80008040).stop 2020-12-05T17:40:03.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.783+0000 7fcc8a014700 1 -- 172.21.15.32:0/2699917380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc840afea0 msgr2=0x7fcc840b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:03.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.783+0000 7fcc8a014700 1 --2- 172.21.15.32:0/2699917380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc840afea0 0x7fcc840b4ee0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 rx=0x7fcc8406a370 tx=0x7fcc7c016000).stop 2020-12-05T17:40:03.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.783+0000 7fcc8a014700 1 -- 172.21.15.32:0/2699917380 shutdown_connections 2020-12-05T17:40:03.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.783+0000 7fcc8a014700 1 --2- 172.21.15.32:0/2699917380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc840afea0 0x7fcc840b4ee0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:03.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.783+0000 7fcc8a014700 1 --2- 172.21.15.32:0/2699917380 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcc6c0553b0 0x7fcc6c057870 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:03.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.783+0000 7fcc8a014700 1 --2- 172.21.15.32:0/2699917380 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcc8405f6f0 0x7fcc84063410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:03.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.783+0000 7fcc8a014700 1 --2- 172.21.15.32:0/2699917380 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc840af920 0x7fcc840b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:03.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.783+0000 7fcc8a014700 1 -- 172.21.15.32:0/2699917380 >> 172.21.15.32:0/2699917380 conn(0x7fcc840604c0 msgr2=0x7fcc84143a30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:03.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.783+0000 7fcc8a014700 1 -- 172.21.15.32:0/2699917380 shutdown_connections 2020-12-05T17:40:03.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:02.783+0000 7fcc8a014700 1 -- 172.21.15.32:0/2699917380 wait complete. 2020-12-05T17:40:03.074 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:40:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.195+0000 7f071f231700 1 Processor -- start 2020-12-05T17:40:03.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.195+0000 7f071f231700 1 -- start start 2020-12-05T17:40:03.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.199+0000 7f071f231700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07180af680 0x7f07180b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:03.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.199+0000 7f071f231700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0718143940 con 0x7f07180afa60 2020-12-05T17:40:03.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.199+0000 7f071f231700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0718140070 con 0x7f0718066ba0 2020-12-05T17:40:03.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.199+0000 7f071f231700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07180ae340 con 0x7f07180af680 2020-12-05T17:40:03.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.199+0000 7f071cfcd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07180af680 0x7f07180b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:03.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.199+0000 7f0717fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f07180afa60 0x7f0718142b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41744/0 (socket says 172.21.15.32:41744) 2020-12-05T17:40:03.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.199+0000 7f0717fff700 1 -- 172.21.15.32:0/927640581 learned_addr learned my addr 172.21.15.32:0/927640581 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:03.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.199+0000 7f071cfcd700 1 -- 172.21.15.32:0/927640581 >> v1:172.21.15.61:6789/0 conn(0x7f0718066ba0 legacy=0x7f0718143230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:03.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.199+0000 7f071cfcd700 1 -- 172.21.15.32:0/927640581 >> v1:172.21.15.32:6789/0 conn(0x7f07180afa60 legacy=0x7f0718142b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:03.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.199+0000 7f07177fe700 1 -- 172.21.15.32:0/927640581 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1667290809 0 0) 0x7f0718143940 con 0x7f07180afa60 2020-12-05T17:40:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.199+0000 7f071cfcd700 1 -- 172.21.15.32:0/927640581 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0718140310 con 0x7f07180af680 2020-12-05T17:40:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.199+0000 7f071cfcd700 1 --2- 172.21.15.32:0/927640581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07180af680 0x7f07180b14e0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7f0708002f30 tx=0x7f0708006dc0).ready entity=mon.0 client_cookie=471cfbade32bd831 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.199+0000 7f07177fe700 1 -- 172.21.15.32:0/927640581 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07080097f0 con 0x7f07180af680 2020-12-05T17:40:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.199+0000 7f07177fe700 1 -- 172.21.15.32:0/927640581 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0708009950 con 0x7f07180af680 2020-12-05T17:40:03.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.199+0000 7f07177fe700 1 -- 172.21.15.32:0/927640581 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0708009cb0 con 0x7f07180af680 2020-12-05T17:40:03.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.199+0000 7f071f231700 1 -- 172.21.15.32:0/927640581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07180af680 msgr2=0x7f07180b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:03.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.199+0000 7f071f231700 1 --2- 172.21.15.32:0/927640581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07180af680 0x7f07180b14e0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7f0708002f30 tx=0x7f0708006dc0).stop 2020-12-05T17:40:03.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.199+0000 7f071f231700 1 -- 172.21.15.32:0/927640581 shutdown_connections 2020-12-05T17:40:03.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.199+0000 7f071f231700 1 --2- 172.21.15.32:0/927640581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07180af680 0x7f07180b14e0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:03.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.199+0000 7f071f231700 1 -- 172.21.15.32:0/927640581 >> 172.21.15.32:0/927640581 conn(0x7f07180b5580 msgr2=0x7f07180b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:03.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.199+0000 7f071f231700 1 -- 172.21.15.32:0/927640581 shutdown_connections 2020-12-05T17:40:03.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.199+0000 7f071f231700 1 -- 172.21.15.32:0/927640581 wait complete. 2020-12-05T17:40:03.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f071f231700 1 Processor -- start 2020-12-05T17:40:03.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f071f231700 1 -- start start 2020-12-05T17:40:03.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f071f231700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0718066ba0 0x7f0718151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:03.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f071f231700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07180af680 0x7f071813a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:03.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f071f231700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07180afa60 0x7f0718063b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:03.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f071f231700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07180636d0 con 0x7f07180af680 2020-12-05T17:40:03.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f071f231700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0718144100 con 0x7f0718066ba0 2020-12-05T17:40:03.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f071f231700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07181443a0 con 0x7f07180afa60 2020-12-05T17:40:03.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f0717fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07180af680 0x7f071813a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:03.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f0717fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07180af680 0x7f071813a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39102/0 (socket says 172.21.15.32:39102) 2020-12-05T17:40:03.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f0717fff700 1 -- 172.21.15.32:0/3337320348 learned_addr learned my addr 172.21.15.32:0/3337320348 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:03.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f071cfcd700 1 --2- 172.21.15.32:0/3337320348 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0718066ba0 0x7f0718151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:03.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f071d7ce700 1 --2- 172.21.15.32:0/3337320348 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07180afa60 0x7f0718063b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:03.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f0717fff700 1 -- 172.21.15.32:0/3337320348 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07180afa60 msgr2=0x7f0718063b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:03.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f0717fff700 1 --2- 172.21.15.32:0/3337320348 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07180afa60 0x7f0718063b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:03.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f0717fff700 1 -- 172.21.15.32:0/3337320348 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0718066ba0 msgr2=0x7f0718151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:03.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f0717fff700 1 --2- 172.21.15.32:0/3337320348 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0718066ba0 0x7f0718151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:03.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f0717fff700 1 -- 172.21.15.32:0/3337320348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0718143d80 con 0x7f07180af680 2020-12-05T17:40:03.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f0717fff700 1 --2- 172.21.15.32:0/3337320348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07180af680 0x7f071813a4d0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 rx=0x7f0700000b80 tx=0x7f0700007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:03.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f0715ffb700 1 -- 172.21.15.32:0/3337320348 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07000072e0 con 0x7f07180af680 2020-12-05T17:40:03.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f0715ffb700 1 -- 172.21.15.32:0/3337320348 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0700007440 con 0x7f07180af680 2020-12-05T17:40:03.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f071f231700 1 -- 172.21.15.32:0/3337320348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071813a9c0 con 0x7f07180af680 2020-12-05T17:40:03.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f0715ffb700 1 -- 172.21.15.32:0/3337320348 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07000057d0 con 0x7f07180af680 2020-12-05T17:40:03.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.203+0000 7f071f231700 1 -- 172.21.15.32:0/3337320348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0718145350 con 0x7f07180af680 2020-12-05T17:40:03.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.207+0000 7f071f231700 1 -- 172.21.15.32:0/3337320348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0718063810 con 0x7f07180af680 2020-12-05T17:40:03.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.207+0000 7f0715ffb700 1 -- 172.21.15.32:0/3337320348 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0700005930 con 0x7f07180af680 2020-12-05T17:40:03.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.207+0000 7f0715ffb700 1 --2- 172.21.15.32:0/3337320348 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f06fc0553b0 0x7f06fc057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:03.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.207+0000 7f0715ffb700 1 -- 172.21.15.32:0/3337320348 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f070006faa0 con 0x7f07180af680 2020-12-05T17:40:03.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.207+0000 7f071cfcd700 1 --2- 172.21.15.32:0/3337320348 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f06fc0553b0 0x7f06fc057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:03.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.207+0000 7f071cfcd700 1 --2- 172.21.15.32:0/3337320348 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f06fc0553b0 0x7f06fc057870 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f070801b000 tx=0x7f0708015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:03.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.211+0000 7f0715ffb700 1 -- 172.21.15.32:0/3337320348 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0700045c40 con 0x7f07180af680 2020-12-05T17:40:03.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.403+0000 7f071f231700 1 -- 172.21.15.32:0/3337320348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0718057050 con 0x7f07180af680 2020-12-05T17:40:03.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.403+0000 7f0715ffb700 1 -- 172.21.15.32:0/3337320348 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f0700049260 con 0x7f07180af680 2020-12-05T17:40:03.409 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:40:03.409 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":47,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/3759190037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3759190037},{"type":"v1","addr":"172.21.15.61:6833","nonce":3759190037}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":45}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:39:47.471195+0000","modified":"2020-12-05T17:39:48.473895+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7233":{"gid":7233,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4174090156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4174090156},{"type":"v1","addr":"172.21.15.32:6835","nonce":4174090156}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2020-12-05T17:40:03.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.403+0000 7f071f231700 1 -- 172.21.15.32:0/3337320348 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f06fc0553b0 msgr2=0x7f06fc057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:03.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.403+0000 7f071f231700 1 --2- 172.21.15.32:0/3337320348 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f06fc0553b0 0x7f06fc057870 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f070801b000 tx=0x7f0708015000).stop 2020-12-05T17:40:03.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.403+0000 7f071f231700 1 -- 172.21.15.32:0/3337320348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07180af680 msgr2=0x7f071813a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:03.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.403+0000 7f071f231700 1 --2- 172.21.15.32:0/3337320348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07180af680 0x7f071813a4d0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 rx=0x7f0700000b80 tx=0x7f0700007b90).stop 2020-12-05T17:40:03.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.407+0000 7f071f231700 1 -- 172.21.15.32:0/3337320348 shutdown_connections 2020-12-05T17:40:03.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.407+0000 7f071f231700 1 --2- 172.21.15.32:0/3337320348 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f06fc0553b0 0x7f06fc057870 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:03.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.407+0000 7f071f231700 1 --2- 172.21.15.32:0/3337320348 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07180afa60 0x7f0718063b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:03.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.407+0000 7f071f231700 1 --2- 172.21.15.32:0/3337320348 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0718066ba0 0x7f0718151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:03.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.407+0000 7f071f231700 1 --2- 172.21.15.32:0/3337320348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07180af680 0x7f071813a4d0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:03.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.407+0000 7f071f231700 1 -- 172.21.15.32:0/3337320348 >> 172.21.15.32:0/3337320348 conn(0x7f07180b5580 msgr2=0x7f07180b1660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:03.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.407+0000 7f071f231700 1 -- 172.21.15.32:0/3337320348 shutdown_connections 2020-12-05T17:40:03.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.407+0000 7f071f231700 1 -- 172.21.15.32:0/3337320348 wait complete. 2020-12-05T17:40:03.413 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 47 2020-12-05T17:40:03.425 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-12-05T17:40:03.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.535+0000 7faf85f62700 1 Processor -- start 2020-12-05T17:40:03.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.535+0000 7faf85f62700 1 -- start start 2020-12-05T17:40:03.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.539+0000 7faf85f62700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faf800af230 0x7faf80145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:03.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.539+0000 7faf85f62700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf80146a90 con 0x7faf800ab710 2020-12-05T17:40:03.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.539+0000 7faf85f62700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf800b23a0 con 0x7faf800b1fc0 2020-12-05T17:40:03.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.539+0000 7faf85f62700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf80061e90 con 0x7faf800af230 2020-12-05T17:40:03.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.539+0000 7faf85761700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faf800af230 0x7faf80145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:03.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.539+0000 7faf85761700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faf800af230 0x7faf80145d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39110/0 (socket says 172.21.15.32:39110) 2020-12-05T17:40:03.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.539+0000 7faf85761700 1 -- 172.21.15.32:0/663830553 learned_addr learned my addr 172.21.15.32:0/663830553 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:03.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.539+0000 7faf85761700 1 -- 172.21.15.32:0/663830553 >> v1:172.21.15.61:6789/0 conn(0x7faf800b1fc0 legacy=0x7faf80061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:03.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.539+0000 7faf85761700 1 -- 172.21.15.32:0/663830553 >> v1:172.21.15.32:6789/0 conn(0x7faf800ab710 legacy=0x7faf80147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:03.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.539+0000 7faf85761700 1 -- 172.21.15.32:0/663830553 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf80146cc0 con 0x7faf800af230 2020-12-05T17:40:03.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.539+0000 7faf85761700 1 --2- 172.21.15.32:0/663830553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf800af230 0x7faf80145d10 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7faf780032b0 tx=0x7faf78009270).ready entity=mon.0 client_cookie=7ea7f0cc8cc68e2 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:03.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.539+0000 7faf7f7fe700 1 -- 172.21.15.32:0/663830553 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf7800bc00 con 0x7faf800af230 2020-12-05T17:40:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.539+0000 7faf7f7fe700 1 -- 172.21.15.32:0/663830553 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7800bd60 con 0x7faf800af230 2020-12-05T17:40:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.539+0000 7faf7f7fe700 1 -- 172.21.15.32:0/663830553 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf78011400 con 0x7faf800af230 2020-12-05T17:40:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.539+0000 7faf85f62700 1 -- 172.21.15.32:0/663830553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf800af230 msgr2=0x7faf80145d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.539+0000 7faf85f62700 1 --2- 172.21.15.32:0/663830553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf800af230 0x7faf80145d10 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7faf780032b0 tx=0x7faf78009270).stop 2020-12-05T17:40:03.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.539+0000 7faf85f62700 1 -- 172.21.15.32:0/663830553 shutdown_connections 2020-12-05T17:40:03.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.539+0000 7faf85f62700 1 --2- 172.21.15.32:0/663830553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf800af230 0x7faf80145d10 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:03.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.539+0000 7faf85f62700 1 -- 172.21.15.32:0/663830553 >> 172.21.15.32:0/663830553 conn(0x7faf800b5530 msgr2=0x7faf800b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:03.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.539+0000 7faf85f62700 1 -- 172.21.15.32:0/663830553 shutdown_connections 2020-12-05T17:40:03.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.539+0000 7faf85f62700 1 -- 172.21.15.32:0/663830553 wait complete. 2020-12-05T17:40:03.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.543+0000 7faf85f62700 1 Processor -- start 2020-12-05T17:40:03.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.543+0000 7faf85f62700 1 -- start start 2020-12-05T17:40:03.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.543+0000 7faf85f62700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faf800ab710 0x7faf80142220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:03.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.543+0000 7faf85f62700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf800af230 0x7faf80155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:03.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.543+0000 7faf85f62700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faf800b1fc0 0x7faf80159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:03.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.543+0000 7faf85f62700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8014a330 con 0x7faf800af230 2020-12-05T17:40:03.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.543+0000 7faf85f62700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8014a090 con 0x7faf800b1fc0 2020-12-05T17:40:03.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.543+0000 7faf85f62700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf80159170 con 0x7faf800ab710 2020-12-05T17:40:03.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.543+0000 7faf7ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf800af230 0x7faf80155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:03.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.543+0000 7faf7ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf800af230 0x7faf80155a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39116/0 (socket says 172.21.15.32:39116) 2020-12-05T17:40:03.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.543+0000 7faf7ffff700 1 -- 172.21.15.32:0/797277878 learned_addr learned my addr 172.21.15.32:0/797277878 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:03.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.543+0000 7faf84f60700 1 --2- 172.21.15.32:0/797277878 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faf800ab710 0x7faf80142220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:03.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.543+0000 7faf85761700 1 --2- 172.21.15.32:0/797277878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faf800b1fc0 0x7faf80159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:03.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.543+0000 7faf7ffff700 1 -- 172.21.15.32:0/797277878 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faf800ab710 msgr2=0x7faf80142220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:03.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.543+0000 7faf7ffff700 1 --2- 172.21.15.32:0/797277878 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faf800ab710 0x7faf80142220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:03.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.543+0000 7faf7ffff700 1 -- 172.21.15.32:0/797277878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faf800b1fc0 msgr2=0x7faf80159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:03.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.543+0000 7faf7ffff700 1 --2- 172.21.15.32:0/797277878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faf800b1fc0 0x7faf80159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:03.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.543+0000 7faf7ffff700 1 -- 172.21.15.32:0/797277878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf80155f50 con 0x7faf800af230 2020-12-05T17:40:03.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.543+0000 7faf7ffff700 1 --2- 172.21.15.32:0/797277878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf800af230 0x7faf80155a60 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 rx=0x7faf800608f0 tx=0x7faf70007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:03.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.547+0000 7faf7dffb700 1 -- 172.21.15.32:0/797277878 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf70013410 con 0x7faf800af230 2020-12-05T17:40:03.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.547+0000 7faf7dffb700 1 -- 172.21.15.32:0/797277878 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf70013570 con 0x7faf800af230 2020-12-05T17:40:03.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.547+0000 7faf85f62700 1 -- 172.21.15.32:0/797277878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf800aed60 con 0x7faf800af230 2020-12-05T17:40:03.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.547+0000 7faf7dffb700 1 -- 172.21.15.32:0/797277878 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf700138d0 con 0x7faf800af230 2020-12-05T17:40:03.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.547+0000 7faf85f62700 1 -- 172.21.15.32:0/797277878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf80149e50 con 0x7faf800af230 2020-12-05T17:40:03.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.547+0000 7faf85f62700 1 -- 172.21.15.32:0/797277878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf80057050 con 0x7faf800af230 2020-12-05T17:40:03.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.547+0000 7faf7dffb700 1 -- 172.21.15.32:0/797277878 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7faf70010030 con 0x7faf800af230 2020-12-05T17:40:03.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.547+0000 7faf7dffb700 1 --2- 172.21.15.32:0/797277878 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faf64055330 0x7faf640577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:03.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.547+0000 7faf7dffb700 1 -- 172.21.15.32:0/797277878 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6420+0+0 (secure 0 0 0) 0x7faf7006ddf0 con 0x7faf800af230 2020-12-05T17:40:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.547+0000 7faf84f60700 1 --2- 172.21.15.32:0/797277878 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faf64055330 0x7faf640577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.547+0000 7faf84f60700 1 --2- 172.21.15.32:0/797277878 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faf64055330 0x7faf640577f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7faf6800c000 tx=0x7faf68008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.551+0000 7faf7dffb700 1 -- 172.21.15.32:0/797277878 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7faf70043e80 con 0x7faf800af230 2020-12-05T17:40:03.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:03.735+0000 7faf85f62700 1 -- 172.21.15.32:0/797277878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7faf801458a0 con 0x7faf800af230 2020-12-05T17:40:04.353 INFO:tasks.ceph.mon.a.smithi032.stderr:2020-12-05T17:40:04.347+0000 7fdef8136700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-12-05T17:40:04.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.351+0000 7faf7dffb700 1 -- 172.21.15.32:0/797277878 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) v1 ==== 161+0+0 (secure 0 0 0) 0x7faf700474a0 con 0x7faf800af230 2020-12-05T17:40:04.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.355+0000 7faf85f62700 1 -- 172.21.15.32:0/797277878 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faf64055330 msgr2=0x7faf640577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:04.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.355+0000 7faf85f62700 1 --2- 172.21.15.32:0/797277878 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faf64055330 0x7faf640577f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7faf6800c000 tx=0x7faf68008040).stop 2020-12-05T17:40:04.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.355+0000 7faf85f62700 1 -- 172.21.15.32:0/797277878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf800af230 msgr2=0x7faf80155a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:04.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.355+0000 7faf85f62700 1 --2- 172.21.15.32:0/797277878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf800af230 0x7faf80155a60 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 rx=0x7faf800608f0 tx=0x7faf70007c20).stop 2020-12-05T17:40:04.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.355+0000 7faf85f62700 1 -- 172.21.15.32:0/797277878 shutdown_connections 2020-12-05T17:40:04.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.355+0000 7faf85f62700 1 --2- 172.21.15.32:0/797277878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faf800b1fc0 0x7faf80159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:04.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.355+0000 7faf85f62700 1 --2- 172.21.15.32:0/797277878 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faf64055330 0x7faf640577f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:04.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.355+0000 7faf85f62700 1 --2- 172.21.15.32:0/797277878 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faf800ab710 0x7faf80142220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:04.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.355+0000 7faf85f62700 1 --2- 172.21.15.32:0/797277878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf800af230 0x7faf80155a60 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:04.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.355+0000 7faf85f62700 1 -- 172.21.15.32:0/797277878 >> 172.21.15.32:0/797277878 conn(0x7faf800b5530 msgr2=0x7faf80144d10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:04.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.359+0000 7faf85f62700 1 -- 172.21.15.32:0/797277878 shutdown_connections 2020-12-05T17:40:04.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.359+0000 7faf85f62700 1 -- 172.21.15.32:0/797277878 wait complete. 2020-12-05T17:40:04.365 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-12-05T17:40:04.379 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:40:04.382 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:40:04.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.487+0000 7fac3e400700 1 Processor -- start 2020-12-05T17:40:04.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.487+0000 7fac3e400700 1 -- start start 2020-12-05T17:40:04.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.487+0000 7fac3e400700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fac380aead0 0x7fac380aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:04.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.487+0000 7fac3e400700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac380ad1b0 0x7fac381389f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:04.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.487+0000 7fac3e400700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fac38138ee0 0x7fac38142290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:04.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.487+0000 7fac3e400700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac3813fb70 con 0x7fac38138ee0 2020-12-05T17:40:04.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.487+0000 7fac3e400700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac380af460 con 0x7fac380aead0 2020-12-05T17:40:04.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.487+0000 7fac3e400700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac380af700 con 0x7fac380ad1b0 2020-12-05T17:40:04.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac3c99d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fac38138ee0 0x7fac38142290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:04.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac3c99d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fac38138ee0 0x7fac38142290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39138/0 (socket says 172.21.15.32:39138) 2020-12-05T17:40:04.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac3c99d700 1 -- 172.21.15.32:0/348798541 learned_addr learned my addr 172.21.15.32:0/348798541 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:04.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac37fff700 1 --2- 172.21.15.32:0/348798541 >> v2:172.21.15.61:3300/0 conn(0x7fac380aead0 0x7fac380aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:04.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac377fe700 1 --2- 172.21.15.32:0/348798541 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac380ad1b0 0x7fac381389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:04.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac3c99d700 1 -- 172.21.15.32:0/348798541 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac380ad1b0 msgr2=0x7fac381389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:04.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac3c99d700 1 --2- 172.21.15.32:0/348798541 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac380ad1b0 0x7fac381389f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:04.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac3c99d700 1 -- 172.21.15.32:0/348798541 >> v2:172.21.15.61:3300/0 conn(0x7fac380aead0 msgr2=0x7fac380aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:04.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac3c99d700 1 --2- 172.21.15.32:0/348798541 >> v2:172.21.15.61:3300/0 conn(0x7fac380aead0 0x7fac380aeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:04.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac3c99d700 1 -- 172.21.15.32:0/348798541 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac3813fdf0 con 0x7fac38138ee0 2020-12-05T17:40:04.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac3c99d700 1 --2- 172.21.15.32:0/348798541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac38138ee0 0x7fac38142290 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7fac300032b0 tx=0x7fac30009270).ready entity=mon.0 client_cookie=fd98e81e8d0097d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:04.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac36ffd700 1 -- 172.21.15.32:0/348798541 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac30009e60 con 0x7fac38138ee0 2020-12-05T17:40:04.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac36ffd700 1 -- 172.21.15.32:0/348798541 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3000bb50 con 0x7fac38138ee0 2020-12-05T17:40:04.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac36ffd700 1 -- 172.21.15.32:0/348798541 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac3000bcc0 con 0x7fac38138ee0 2020-12-05T17:40:04.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac3e400700 1 -- 172.21.15.32:0/348798541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac38138ee0 msgr2=0x7fac38142290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:04.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac3e400700 1 --2- 172.21.15.32:0/348798541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac38138ee0 0x7fac38142290 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7fac300032b0 tx=0x7fac30009270).stop 2020-12-05T17:40:04.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac3e400700 1 -- 172.21.15.32:0/348798541 shutdown_connections 2020-12-05T17:40:04.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac3e400700 1 --2- 172.21.15.32:0/348798541 >> v2:172.21.15.61:3300/0 conn(0x7fac380aead0 0x7fac380aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:04.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac3e400700 1 --2- 172.21.15.32:0/348798541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac38138ee0 0x7fac38142290 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:04.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac3e400700 1 --2- 172.21.15.32:0/348798541 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac380ad1b0 0x7fac381389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:04.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac3e400700 1 -- 172.21.15.32:0/348798541 >> 172.21.15.32:0/348798541 conn(0x7fac380b5580 msgr2=0x7fac380b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:04.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac3e400700 1 -- 172.21.15.32:0/348798541 shutdown_connections 2020-12-05T17:40:04.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac3e400700 1 -- 172.21.15.32:0/348798541 wait complete. 2020-12-05T17:40:04.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac3e400700 1 Processor -- start 2020-12-05T17:40:04.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.491+0000 7fac3e400700 1 -- start start 2020-12-05T17:40:04.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.495+0000 7fac3e400700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac380ad1b0 0x7fac3813be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:04.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.495+0000 7fac3e400700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac380aead0 0x7fac38063270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:04.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.495+0000 7fac3e400700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac38138ee0 0x7fac38063760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:04.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.495+0000 7fac3e400700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3814e6a0 con 0x7fac380ad1b0 2020-12-05T17:40:04.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.495+0000 7fac3e400700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3814e940 con 0x7fac38138ee0 2020-12-05T17:40:04.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.495+0000 7fac3e400700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac3814ebe0 con 0x7fac380aead0 2020-12-05T17:40:04.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.495+0000 7fac37fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac380ad1b0 0x7fac3813be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:04.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.495+0000 7fac37fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac380ad1b0 0x7fac3813be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39140/0 (socket says 172.21.15.32:39140) 2020-12-05T17:40:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.495+0000 7fac37fff700 1 -- 172.21.15.32:0/805280593 learned_addr learned my addr 172.21.15.32:0/805280593 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.495+0000 7fac377fe700 1 --2- 172.21.15.32:0/805280593 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac380aead0 0x7fac38063270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.495+0000 7fac3c99d700 1 --2- 172.21.15.32:0/805280593 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac38138ee0 0x7fac38063760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:04.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.495+0000 7fac37fff700 1 -- 172.21.15.32:0/805280593 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac380aead0 msgr2=0x7fac38063270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:04.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.495+0000 7fac37fff700 1 --2- 172.21.15.32:0/805280593 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac380aead0 0x7fac38063270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:04.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.495+0000 7fac37fff700 1 -- 172.21.15.32:0/805280593 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac38138ee0 msgr2=0x7fac38063760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:04.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.495+0000 7fac37fff700 1 --2- 172.21.15.32:0/805280593 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac38138ee0 0x7fac38063760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:04.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.495+0000 7fac37fff700 1 -- 172.21.15.32:0/805280593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac3814e2b0 con 0x7fac380ad1b0 2020-12-05T17:40:04.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.495+0000 7fac37fff700 1 --2- 172.21.15.32:0/805280593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac380ad1b0 0x7fac3813be20 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7fac2800c000 tx=0x7fac28006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:04.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.495+0000 7fac357fa700 1 -- 172.21.15.32:0/805280593 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac2800ca10 con 0x7fac380ad1b0 2020-12-05T17:40:04.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.495+0000 7fac357fa700 1 -- 172.21.15.32:0/805280593 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2800cb70 con 0x7fac380ad1b0 2020-12-05T17:40:04.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.495+0000 7fac3e400700 1 -- 172.21.15.32:0/805280593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac3814f080 con 0x7fac380ad1b0 2020-12-05T17:40:04.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.495+0000 7fac357fa700 1 -- 172.21.15.32:0/805280593 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac2800b370 con 0x7fac380ad1b0 2020-12-05T17:40:04.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.495+0000 7fac3e400700 1 -- 172.21.15.32:0/805280593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac38063e70 con 0x7fac380ad1b0 2020-12-05T17:40:04.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.499+0000 7fac3e400700 1 -- 172.21.15.32:0/805280593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac3814f270 con 0x7fac380ad1b0 2020-12-05T17:40:04.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.499+0000 7fac357fa700 1 -- 172.21.15.32:0/805280593 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fac28021070 con 0x7fac380ad1b0 2020-12-05T17:40:04.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.499+0000 7fac357fa700 1 --2- 172.21.15.32:0/805280593 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fac1c0554e0 0x7fac1c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:04.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.499+0000 7fac357fa700 1 -- 172.21.15.32:0/805280593 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fac2806fe30 con 0x7fac380ad1b0 2020-12-05T17:40:04.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.499+0000 7fac377fe700 1 --2- 172.21.15.32:0/805280593 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fac1c0554e0 0x7fac1c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:04.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.499+0000 7fac377fe700 1 --2- 172.21.15.32:0/805280593 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fac1c0554e0 0x7fac1c0579a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7fac2000b000 tx=0x7fac20006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:04.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.503+0000 7fac357fa700 1 -- 172.21.15.32:0/805280593 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fac28045f80 con 0x7fac380ad1b0 2020-12-05T17:40:04.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.687+0000 7fac3e400700 1 -- 172.21.15.32:0/805280593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac3814e4a0 con 0x7fac380ad1b0 2020-12-05T17:40:04.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.691+0000 7fac357fa700 1 -- 172.21.15.32:0/805280593 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+1780 (secure 0 0 0) 0x7fac280495a0 con 0x7fac380ad1b0 2020-12-05T17:40:04.696 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:40:04.697 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":48,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/3759190037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3759190037},{"type":"v1","addr":"172.21.15.61:6833","nonce":3759190037}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":45}],"filesystems":[{"mdsmap":{"epoch":48,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:39:47.471195+0000","modified":"2020-12-05T17:40:04.350413+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2020-12-05T17:40:04.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.691+0000 7fac3e400700 1 -- 172.21.15.32:0/805280593 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fac1c0554e0 msgr2=0x7fac1c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:04.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.691+0000 7fac3e400700 1 --2- 172.21.15.32:0/805280593 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fac1c0554e0 0x7fac1c0579a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7fac2000b000 tx=0x7fac20006dc0).stop 2020-12-05T17:40:04.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.691+0000 7fac3e400700 1 -- 172.21.15.32:0/805280593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac380ad1b0 msgr2=0x7fac3813be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:04.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.691+0000 7fac3e400700 1 --2- 172.21.15.32:0/805280593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac380ad1b0 0x7fac3813be20 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7fac2800c000 tx=0x7fac28006dc0).stop 2020-12-05T17:40:04.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.691+0000 7fac3e400700 1 -- 172.21.15.32:0/805280593 shutdown_connections 2020-12-05T17:40:04.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.691+0000 7fac3e400700 1 --2- 172.21.15.32:0/805280593 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac38138ee0 0x7fac38063760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:04.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.691+0000 7fac3e400700 1 --2- 172.21.15.32:0/805280593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac380ad1b0 0x7fac3813be20 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:04.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.691+0000 7fac3e400700 1 --2- 172.21.15.32:0/805280593 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fac1c0554e0 0x7fac1c0579a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:04.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.691+0000 7fac3e400700 1 --2- 172.21.15.32:0/805280593 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac380aead0 0x7fac38063270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:04.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.691+0000 7fac3e400700 1 -- 172.21.15.32:0/805280593 >> 172.21.15.32:0/805280593 conn(0x7fac380b5580 msgr2=0x7fac38067150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:04.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.691+0000 7fac3e400700 1 -- 172.21.15.32:0/805280593 shutdown_connections 2020-12-05T17:40:04.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.691+0000 7fac3e400700 1 -- 172.21.15.32:0/805280593 wait complete. 2020-12-05T17:40:04.703 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 48 2020-12-05T17:40:04.709 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-12-05T17:40:04.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.827+0000 7fbb6a798700 1 Processor -- start 2020-12-05T17:40:04.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.827+0000 7fbb6a798700 1 -- start start 2020-12-05T17:40:04.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb6a798700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbb640b5f00 0x7fbb640b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:04.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb6a798700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fbb640ad510 0x7fbb640af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:04.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb6a798700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb64142ab0 con 0x7fbb64066370 2020-12-05T17:40:04.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb6a798700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb640b62e0 con 0x7fbb640b5f00 2020-12-05T17:40:04.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb6a798700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb640afbf0 con 0x7fbb640ad510 2020-12-05T17:40:04.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb69796700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbb640b5f00 0x7fbb640b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:04.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb69f97700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbb64066370 0x7fbb64142ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41800/0 (socket says 172.21.15.32:41800) 2020-12-05T17:40:04.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb69f97700 1 -- 172.21.15.32:0/2191230560 learned_addr learned my addr 172.21.15.32:0/2191230560 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:04.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb68f95700 1 --2- 172.21.15.32:0/2191230560 >> v2:172.21.15.61:3300/0 conn(0x7fbb640ad510 0x7fbb640af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:04.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb69796700 1 -- 172.21.15.32:0/2191230560 >> v2:172.21.15.61:3300/0 conn(0x7fbb640ad510 msgr2=0x7fbb640af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb69796700 1 --2- 172.21.15.32:0/2191230560 >> v2:172.21.15.61:3300/0 conn(0x7fbb640ad510 0x7fbb640af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb69796700 1 -- 172.21.15.32:0/2191230560 >> v1:172.21.15.32:6789/0 conn(0x7fbb64066370 legacy=0x7fbb64142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb69796700 1 -- 172.21.15.32:0/2191230560 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb640afe90 con 0x7fbb640b5f00 2020-12-05T17:40:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb53fff700 1 -- 172.21.15.32:0/2191230560 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1910517961 0 0) 0x7fbb64142ab0 con 0x7fbb64066370 2020-12-05T17:40:04.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb69796700 1 --2- 172.21.15.32:0/2191230560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb640b5f00 0x7fbb640b14e0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7fbb60002f30 tx=0x7fbb60006dc0).ready entity=mon.0 client_cookie=74025171385a5e8d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:04.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb53fff700 1 -- 172.21.15.32:0/2191230560 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb600097b0 con 0x7fbb640b5f00 2020-12-05T17:40:04.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb53fff700 1 -- 172.21.15.32:0/2191230560 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb60009910 con 0x7fbb640b5f00 2020-12-05T17:40:04.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb53fff700 1 -- 172.21.15.32:0/2191230560 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb60009c70 con 0x7fbb640b5f00 2020-12-05T17:40:04.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb6a798700 1 -- 172.21.15.32:0/2191230560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb640b5f00 msgr2=0x7fbb640b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb6a798700 1 --2- 172.21.15.32:0/2191230560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb640b5f00 0x7fbb640b14e0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7fbb60002f30 tx=0x7fbb60006dc0).stop 2020-12-05T17:40:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb6a798700 1 -- 172.21.15.32:0/2191230560 shutdown_connections 2020-12-05T17:40:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb6a798700 1 --2- 172.21.15.32:0/2191230560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb640b5f00 0x7fbb640b14e0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb6a798700 1 --2- 172.21.15.32:0/2191230560 >> v2:172.21.15.61:3300/0 conn(0x7fbb640ad510 0x7fbb640af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb6a798700 1 -- 172.21.15.32:0/2191230560 >> 172.21.15.32:0/2191230560 conn(0x7fbb640604c0 msgr2=0x7fbb6405c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb6a798700 1 -- 172.21.15.32:0/2191230560 shutdown_connections 2020-12-05T17:40:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.831+0000 7fbb6a798700 1 -- 172.21.15.32:0/2191230560 wait complete. 2020-12-05T17:40:04.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb6a798700 1 Processor -- start 2020-12-05T17:40:04.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb6a798700 1 -- start start 2020-12-05T17:40:04.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb6a798700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb64066370 0x7fbb640b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:04.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb6a798700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb640ad510 0x7fbb64065430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:04.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb6a798700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb640b5f00 0x7fbb64065920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:04.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb6a798700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb64063320 con 0x7fbb640ad510 2020-12-05T17:40:04.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb6a798700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb64063ef0 con 0x7fbb640b5f00 2020-12-05T17:40:04.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb6a798700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb64063580 con 0x7fbb64066370 2020-12-05T17:40:04.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb68f95700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb640ad510 0x7fbb64065430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:04.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb68f95700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb640ad510 0x7fbb64065430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39154/0 (socket says 172.21.15.32:39154) 2020-12-05T17:40:04.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb68f95700 1 -- 172.21.15.32:0/4120561644 learned_addr learned my addr 172.21.15.32:0/4120561644 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:04.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb69f97700 1 --2- 172.21.15.32:0/4120561644 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb640b5f00 0x7fbb64065920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:04.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb69796700 1 --2- 172.21.15.32:0/4120561644 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb64066370 0x7fbb640b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:04.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb68f95700 1 -- 172.21.15.32:0/4120561644 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb64066370 msgr2=0x7fbb640b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:04.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb68f95700 1 --2- 172.21.15.32:0/4120561644 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb64066370 0x7fbb640b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:04.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb68f95700 1 -- 172.21.15.32:0/4120561644 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb640b5f00 msgr2=0x7fbb64065920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:04.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb68f95700 1 --2- 172.21.15.32:0/4120561644 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb640b5f00 0x7fbb64065920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:04.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb68f95700 1 -- 172.21.15.32:0/4120561644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb64063040 con 0x7fbb640ad510 2020-12-05T17:40:04.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb68f95700 1 --2- 172.21.15.32:0/4120561644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb640ad510 0x7fbb64065430 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7fbb5400c000 tx=0x7fbb54006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:04.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb527fc700 1 -- 172.21.15.32:0/4120561644 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb5400ca10 con 0x7fbb640ad510 2020-12-05T17:40:04.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb527fc700 1 -- 172.21.15.32:0/4120561644 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5400cb70 con 0x7fbb640ad510 2020-12-05T17:40:04.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb6a798700 1 -- 172.21.15.32:0/4120561644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6414e2d0 con 0x7fbb640ad510 2020-12-05T17:40:04.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb527fc700 1 -- 172.21.15.32:0/4120561644 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb540123f0 con 0x7fbb640ad510 2020-12-05T17:40:04.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.835+0000 7fbb6a798700 1 -- 172.21.15.32:0/4120561644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb64063c60 con 0x7fbb640ad510 2020-12-05T17:40:04.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.839+0000 7fbb6a798700 1 -- 172.21.15.32:0/4120561644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb64063740 con 0x7fbb640ad510 2020-12-05T17:40:04.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.839+0000 7fbb527fc700 1 -- 172.21.15.32:0/4120561644 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbb54012550 con 0x7fbb640ad510 2020-12-05T17:40:04.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.839+0000 7fbb527fc700 1 --2- 172.21.15.32:0/4120561644 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbb440553b0 0x7fbb44057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:04.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.839+0000 7fbb527fc700 1 -- 172.21.15.32:0/4120561644 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fbb5406e690 con 0x7fbb640ad510 2020-12-05T17:40:04.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.839+0000 7fbb69796700 1 --2- 172.21.15.32:0/4120561644 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbb440553b0 0x7fbb44057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:04.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.839+0000 7fbb69796700 1 --2- 172.21.15.32:0/4120561644 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbb440553b0 0x7fbb44057870 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fbb6001a000 tx=0x7fbb60015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:04.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:04.847+0000 7fbb527fc700 1 -- 172.21.15.32:0/4120561644 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbb540445f0 con 0x7fbb640ad510 2020-12-05T17:40:05.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.031+0000 7fbb6a798700 1 -- 172.21.15.32:0/4120561644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbb64064060 con 0x7fbb640ad510 2020-12-05T17:40:05.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.355+0000 7fbb527fc700 1 -- 172.21.15.32:0/4120561644 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbb54047c10 con 0x7fbb640ad510 2020-12-05T17:40:05.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.359+0000 7fbb6a798700 1 -- 172.21.15.32:0/4120561644 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbb440553b0 msgr2=0x7fbb44057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:05.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.359+0000 7fbb6a798700 1 --2- 172.21.15.32:0/4120561644 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbb440553b0 0x7fbb44057870 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fbb6001a000 tx=0x7fbb60015000).stop 2020-12-05T17:40:05.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.359+0000 7fbb6a798700 1 -- 172.21.15.32:0/4120561644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb640ad510 msgr2=0x7fbb64065430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:05.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.359+0000 7fbb6a798700 1 --2- 172.21.15.32:0/4120561644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb640ad510 0x7fbb64065430 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7fbb5400c000 tx=0x7fbb54006dc0).stop 2020-12-05T17:40:05.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.359+0000 7fbb6a798700 1 -- 172.21.15.32:0/4120561644 shutdown_connections 2020-12-05T17:40:05.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.359+0000 7fbb6a798700 1 --2- 172.21.15.32:0/4120561644 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb640b5f00 0x7fbb64065920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:05.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.359+0000 7fbb6a798700 1 --2- 172.21.15.32:0/4120561644 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbb440553b0 0x7fbb44057870 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:05.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.359+0000 7fbb6a798700 1 --2- 172.21.15.32:0/4120561644 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb64066370 0x7fbb640b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:05.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.359+0000 7fbb6a798700 1 --2- 172.21.15.32:0/4120561644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb640ad510 0x7fbb64065430 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:05.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.359+0000 7fbb6a798700 1 -- 172.21.15.32:0/4120561644 >> 172.21.15.32:0/4120561644 conn(0x7fbb640604c0 msgr2=0x7fbb640ae9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:05.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.359+0000 7fbb6a798700 1 -- 172.21.15.32:0/4120561644 shutdown_connections 2020-12-05T17:40:05.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.359+0000 7fbb6a798700 1 -- 172.21.15.32:0/4120561644 wait complete. 2020-12-05T17:40:05.382 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-12-05T17:40:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.507+0000 7f1d22c37700 1 Processor -- start 2020-12-05T17:40:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.507+0000 7f1d22c37700 1 -- start start 2020-12-05T17:40:05.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d22c37700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1d1c05feb0 0x7f1d1c05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:05.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d22c37700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1d1c0b2fb0 0x7f1d1c061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:05.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d22c37700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d1c0b2df0 con 0x7f1d1c05f3b0 2020-12-05T17:40:05.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d22c37700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d1c0b3c90 con 0x7f1d1c05feb0 2020-12-05T17:40:05.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d22c37700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d1c0b2b50 con 0x7f1d1c0b2fb0 2020-12-05T17:40:05.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d21434700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1d1c05feb0 0x7f1d1c05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:05.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d21c35700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1d1c05f3b0 0x7f1d1c13c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41810/0 (socket says 172.21.15.32:41810) 2020-12-05T17:40:05.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d21c35700 1 -- 172.21.15.32:0/3592433738 learned_addr learned my addr 172.21.15.32:0/3592433738 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d22436700 1 --2- 172.21.15.32:0/3592433738 >> v2:172.21.15.61:3300/0 conn(0x7f1d1c0b2fb0 0x7f1d1c061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d21434700 1 -- 172.21.15.32:0/3592433738 >> v2:172.21.15.61:3300/0 conn(0x7f1d1c0b2fb0 msgr2=0x7f1d1c061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d21434700 1 --2- 172.21.15.32:0/3592433738 >> v2:172.21.15.61:3300/0 conn(0x7f1d1c0b2fb0 0x7f1d1c061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d21434700 1 -- 172.21.15.32:0/3592433738 >> v1:172.21.15.32:6789/0 conn(0x7f1d1c05f3b0 legacy=0x7f1d1c13c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d20c33700 1 -- 172.21.15.32:0/3592433738 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 227341410 0 0) 0x7f1d1c0b2df0 con 0x7f1d1c05f3b0 2020-12-05T17:40:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d21434700 1 -- 172.21.15.32:0/3592433738 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1c142820 con 0x7f1d1c05feb0 2020-12-05T17:40:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d22436700 1 --2- 172.21.15.32:0/3592433738 >> v2:172.21.15.61:3300/0 conn(0x7f1d1c0b2fb0 0x7f1d1c061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:40:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d21434700 1 --2- 172.21.15.32:0/3592433738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d1c05feb0 0x7f1d1c05f790 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7f1d0c002f30 tx=0x7f1d0c006dc0).ready entity=mon.0 client_cookie=5834b55678090f94 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d20c33700 1 -- 172.21.15.32:0/3592433738 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d0c0097f0 con 0x7f1d1c05feb0 2020-12-05T17:40:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d20c33700 1 -- 172.21.15.32:0/3592433738 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0c009950 con 0x7f1d1c05feb0 2020-12-05T17:40:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d20c33700 1 -- 172.21.15.32:0/3592433738 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d0c009cb0 con 0x7f1d1c05feb0 2020-12-05T17:40:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d22c37700 1 -- 172.21.15.32:0/3592433738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d1c05feb0 msgr2=0x7f1d1c05f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d22c37700 1 --2- 172.21.15.32:0/3592433738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d1c05feb0 0x7f1d1c05f790 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7f1d0c002f30 tx=0x7f1d0c006dc0).stop 2020-12-05T17:40:05.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d22c37700 1 -- 172.21.15.32:0/3592433738 shutdown_connections 2020-12-05T17:40:05.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d22c37700 1 --2- 172.21.15.32:0/3592433738 >> v2:172.21.15.61:3300/0 conn(0x7f1d1c0b2fb0 0x7f1d1c061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:05.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d22c37700 1 --2- 172.21.15.32:0/3592433738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d1c05feb0 0x7f1d1c05f790 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:05.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d22c37700 1 -- 172.21.15.32:0/3592433738 >> 172.21.15.32:0/3592433738 conn(0x7f1d1c0b8f20 msgr2=0x7f1d1c0b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:05.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d22c37700 1 -- 172.21.15.32:0/3592433738 shutdown_connections 2020-12-05T17:40:05.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.511+0000 7f1d22c37700 1 -- 172.21.15.32:0/3592433738 wait complete. 2020-12-05T17:40:05.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d22c37700 1 Processor -- start 2020-12-05T17:40:05.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d22c37700 1 -- start start 2020-12-05T17:40:05.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d22c37700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d1c05f3b0 0x7f1d1c143b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:05.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d22c37700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d1c05feb0 0x7f1d1c13e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:05.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d22c37700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1d1c0b2fb0 0x7f1d1c148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:05.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d22c37700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c1441a0 con 0x7f1d1c05f3b0 2020-12-05T17:40:05.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d22c37700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c144440 con 0x7f1d1c05feb0 2020-12-05T17:40:05.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d22c37700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c1446e0 con 0x7f1d1c0b2fb0 2020-12-05T17:40:05.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d21c35700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d1c05f3b0 0x7f1d1c143b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:05.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d21c35700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d1c05f3b0 0x7f1d1c143b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39168/0 (socket says 172.21.15.32:39168) 2020-12-05T17:40:05.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d21c35700 1 -- 172.21.15.32:0/968862717 learned_addr learned my addr 172.21.15.32:0/968862717 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:05.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d21434700 1 --2- 172.21.15.32:0/968862717 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d1c05feb0 0x7f1d1c13e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:05.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d22436700 1 --2- 172.21.15.32:0/968862717 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1d1c0b2fb0 0x7f1d1c148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:05.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d21c35700 1 -- 172.21.15.32:0/968862717 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1d1c0b2fb0 msgr2=0x7f1d1c148f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:05.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d21c35700 1 --2- 172.21.15.32:0/968862717 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1d1c0b2fb0 0x7f1d1c148f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:05.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d21c35700 1 -- 172.21.15.32:0/968862717 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d1c05feb0 msgr2=0x7f1d1c13e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:05.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d21c35700 1 --2- 172.21.15.32:0/968862717 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d1c05feb0 0x7f1d1c13e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:05.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d21c35700 1 -- 172.21.15.32:0/968862717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1c1510f0 con 0x7f1d1c05f3b0 2020-12-05T17:40:05.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d21c35700 1 --2- 172.21.15.32:0/968862717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d1c05f3b0 0x7f1d1c143b70 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7f1d18000fa0 tx=0x7f1d18007b20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:05.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d0affd700 1 -- 172.21.15.32:0/968862717 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d180072e0 con 0x7f1d1c05f3b0 2020-12-05T17:40:05.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d0affd700 1 -- 172.21.15.32:0/968862717 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d18007440 con 0x7f1d1c05f3b0 2020-12-05T17:40:05.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d22c37700 1 -- 172.21.15.32:0/968862717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1c151340 con 0x7f1d1c05f3b0 2020-12-05T17:40:05.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d0affd700 1 -- 172.21.15.32:0/968862717 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d180057d0 con 0x7f1d1c05f3b0 2020-12-05T17:40:05.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.515+0000 7f1d22c37700 1 -- 172.21.15.32:0/968862717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d1c1423e0 con 0x7f1d1c05f3b0 2020-12-05T17:40:05.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.519+0000 7f1d22c37700 1 -- 172.21.15.32:0/968862717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d1c1448a0 con 0x7f1d1c05f3b0 2020-12-05T17:40:05.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.519+0000 7f1d0affd700 1 -- 172.21.15.32:0/968862717 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1d18010070 con 0x7f1d1c05f3b0 2020-12-05T17:40:05.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.519+0000 7f1d0affd700 1 --2- 172.21.15.32:0/968862717 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1cfc0553b0 0x7f1cfc057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:05.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.519+0000 7f1d0affd700 1 -- 172.21.15.32:0/968862717 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f1d1806d8b0 con 0x7f1d1c05f3b0 2020-12-05T17:40:05.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.519+0000 7f1d21434700 1 --2- 172.21.15.32:0/968862717 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1cfc0553b0 0x7f1cfc057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:05.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.519+0000 7f1d21434700 1 --2- 172.21.15.32:0/968862717 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1cfc0553b0 0x7f1cfc057870 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f1d0c01b000 tx=0x7f1d0c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:05.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.527+0000 7f1d0affd700 1 -- 172.21.15.32:0/968862717 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1d18043960 con 0x7f1d1c05f3b0 2020-12-05T17:40:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:05.671+0000 7f1d22c37700 1 -- 172.21.15.32:0/968862717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1d1c057050 con 0x7f1d1c05f3b0 2020-12-05T17:40:06.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.367+0000 7f1d0affd700 1 -- 172.21.15.32:0/968862717 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v104) v1 ==== 185+0+0 (secure 0 0 0) 0x7f1d18046f80 con 0x7f1d1c05f3b0 2020-12-05T17:40:06.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.371+0000 7f1d22c37700 1 -- 172.21.15.32:0/968862717 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1cfc0553b0 msgr2=0x7f1cfc057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:06.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.371+0000 7f1d22c37700 1 --2- 172.21.15.32:0/968862717 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1cfc0553b0 0x7f1cfc057870 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f1d0c01b000 tx=0x7f1d0c015000).stop 2020-12-05T17:40:06.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.371+0000 7f1d22c37700 1 -- 172.21.15.32:0/968862717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d1c05f3b0 msgr2=0x7f1d1c143b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:06.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.371+0000 7f1d22c37700 1 --2- 172.21.15.32:0/968862717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d1c05f3b0 0x7f1d1c143b70 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7f1d18000fa0 tx=0x7f1d18007b20).stop 2020-12-05T17:40:06.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.371+0000 7f1d22c37700 1 -- 172.21.15.32:0/968862717 shutdown_connections 2020-12-05T17:40:06.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.371+0000 7f1d22c37700 1 --2- 172.21.15.32:0/968862717 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1cfc0553b0 0x7f1cfc057870 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:06.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.371+0000 7f1d22c37700 1 --2- 172.21.15.32:0/968862717 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1d1c0b2fb0 0x7f1d1c148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:06.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.371+0000 7f1d22c37700 1 --2- 172.21.15.32:0/968862717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d1c05f3b0 0x7f1d1c143b70 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:06.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.371+0000 7f1d22c37700 1 --2- 172.21.15.32:0/968862717 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d1c05feb0 0x7f1d1c13e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:06.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.371+0000 7f1d22c37700 1 -- 172.21.15.32:0/968862717 >> 172.21.15.32:0/968862717 conn(0x7f1d1c0b8f20 msgr2=0x7f1d1c0b6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:06.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.371+0000 7f1d22c37700 1 -- 172.21.15.32:0/968862717 shutdown_connections 2020-12-05T17:40:06.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.371+0000 7f1d22c37700 1 -- 172.21.15.32:0/968862717 wait complete. 2020-12-05T17:40:06.378 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2020-12-05T17:40:06.392 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-12-05T17:40:06.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.515+0000 7f9620f49700 1 Processor -- start 2020-12-05T17:40:06.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.515+0000 7f9620f49700 1 -- start start 2020-12-05T17:40:06.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.515+0000 7f9620f49700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f961c0b1dc0 0x7f961c0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:06.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.515+0000 7f9620f49700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f961c0af5f0 0x7f961c0aed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:06.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.515+0000 7f9620f49700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f961c1468e0 con 0x7f961c0afd30 2020-12-05T17:40:06.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.515+0000 7f9620f49700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f961c142c90 con 0x7f961c0b1dc0 2020-12-05T17:40:06.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.515+0000 7f9620f49700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f961c146b10 con 0x7f961c0af5f0 2020-12-05T17:40:06.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.515+0000 7f961b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f961c0b1dc0 0x7f961c0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:06.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f961b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f961c0b1dc0 0x7f961c0b21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39176/0 (socket says 172.21.15.32:39176) 2020-12-05T17:40:06.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f961b7fe700 1 -- 172.21.15.32:0/2870093074 learned_addr learned my addr 172.21.15.32:0/2870093074 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:06.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f961bfff700 1 --2- 172.21.15.32:0/2870093074 >> v2:172.21.15.61:3300/0 conn(0x7f961c0af5f0 0x7f961c0aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:06.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f961b7fe700 1 -- 172.21.15.32:0/2870093074 >> v2:172.21.15.61:3300/0 conn(0x7f961c0af5f0 msgr2=0x7f961c0aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:06.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f961b7fe700 1 --2- 172.21.15.32:0/2870093074 >> v2:172.21.15.61:3300/0 conn(0x7f961c0af5f0 0x7f961c0aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:06.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f961b7fe700 1 -- 172.21.15.32:0/2870093074 >> v1:172.21.15.61:6789/0 conn(0x7f961c0afd30 legacy=0x7f961c0ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:06.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f961b7fe700 1 -- 172.21.15.32:0/2870093074 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961c146db0 con 0x7f961c0b1dc0 2020-12-05T17:40:06.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f961b7fe700 1 --2- 172.21.15.32:0/2870093074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f961c0b1dc0 0x7f961c0b21a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 rx=0x7f960c002f30 tx=0x7f960c006e80).ready entity=mon.0 client_cookie=11b62c39d74aca96 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:06.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f961a7fc700 1 -- 172.21.15.32:0/2870093074 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f960c0097f0 con 0x7f961c0b1dc0 2020-12-05T17:40:06.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f961a7fc700 1 -- 172.21.15.32:0/2870093074 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960c009950 con 0x7f961c0b1dc0 2020-12-05T17:40:06.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f961a7fc700 1 -- 172.21.15.32:0/2870093074 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f960c009cb0 con 0x7f961c0b1dc0 2020-12-05T17:40:06.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f9620f49700 1 -- 172.21.15.32:0/2870093074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f961c0b1dc0 msgr2=0x7f961c0b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:06.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f9620f49700 1 --2- 172.21.15.32:0/2870093074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f961c0b1dc0 0x7f961c0b21a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 rx=0x7f960c002f30 tx=0x7f960c006e80).stop 2020-12-05T17:40:06.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f9620f49700 1 -- 172.21.15.32:0/2870093074 shutdown_connections 2020-12-05T17:40:06.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f9620f49700 1 --2- 172.21.15.32:0/2870093074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f961c0b1dc0 0x7f961c0b21a0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:06.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f9620f49700 1 --2- 172.21.15.32:0/2870093074 >> v2:172.21.15.61:3300/0 conn(0x7f961c0af5f0 0x7f961c0aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:06.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f9620f49700 1 -- 172.21.15.32:0/2870093074 >> 172.21.15.32:0/2870093074 conn(0x7f961c0620b0 msgr2=0x7f961c062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:06.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f9620f49700 1 -- 172.21.15.32:0/2870093074 shutdown_connections 2020-12-05T17:40:06.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f9620f49700 1 -- 172.21.15.32:0/2870093074 wait complete. 2020-12-05T17:40:06.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f9620f49700 1 Processor -- start 2020-12-05T17:40:06.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f9620f49700 1 -- start start 2020-12-05T17:40:06.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f9620f49700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f961c0af5f0 0x7f961c066650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:06.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f9620f49700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f961c0afd30 0x7f961c154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:06.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f9620f49700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f961c0b1dc0 0x7f961c155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:06.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f9620f49700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961c0630e0 con 0x7f961c0af5f0 2020-12-05T17:40:06.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f9620f49700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961c1498f0 con 0x7f961c0b1dc0 2020-12-05T17:40:06.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f9620f49700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961c155630 con 0x7f961c0afd30 2020-12-05T17:40:06.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f961b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f961c0af5f0 0x7f961c066650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:06.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f961b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f961c0af5f0 0x7f961c066650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39182/0 (socket says 172.21.15.32:39182) 2020-12-05T17:40:06.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f961b7fe700 1 -- 172.21.15.32:0/1235712587 learned_addr learned my addr 172.21.15.32:0/1235712587 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:06.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f961bfff700 1 --2- 172.21.15.32:0/1235712587 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f961c0b1dc0 0x7f961c155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:06.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f961affd700 1 --2- 172.21.15.32:0/1235712587 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f961c0afd30 0x7f961c154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:06.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f961b7fe700 1 -- 172.21.15.32:0/1235712587 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f961c0afd30 msgr2=0x7f961c154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:06.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f961b7fe700 1 --2- 172.21.15.32:0/1235712587 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f961c0afd30 0x7f961c154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:06.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f961b7fe700 1 -- 172.21.15.32:0/1235712587 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f961c0b1dc0 msgr2=0x7f961c155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:06.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f961b7fe700 1 --2- 172.21.15.32:0/1235712587 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f961c0b1dc0 0x7f961c155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:06.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f961b7fe700 1 -- 172.21.15.32:0/1235712587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961c1557f0 con 0x7f961c0af5f0 2020-12-05T17:40:06.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.519+0000 7f961b7fe700 1 --2- 172.21.15.32:0/1235712587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f961c0af5f0 0x7f961c066650 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 rx=0x7f960c002f30 tx=0x7f960c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:06.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.523+0000 7f9618ff9700 1 -- 172.21.15.32:0/1235712587 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f960c011c30 con 0x7f961c0af5f0 2020-12-05T17:40:06.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.523+0000 7f9618ff9700 1 -- 172.21.15.32:0/1235712587 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960c011d90 con 0x7f961c0af5f0 2020-12-05T17:40:06.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.523+0000 7f9620f49700 1 -- 172.21.15.32:0/1235712587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961c064610 con 0x7f961c0af5f0 2020-12-05T17:40:06.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.523+0000 7f9618ff9700 1 -- 172.21.15.32:0/1235712587 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f960c01e650 con 0x7f961c0af5f0 2020-12-05T17:40:06.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.523+0000 7f9620f49700 1 -- 172.21.15.32:0/1235712587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961c064c50 con 0x7f961c0af5f0 2020-12-05T17:40:06.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.523+0000 7f9620f49700 1 -- 172.21.15.32:0/1235712587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f961c0632e0 con 0x7f961c0af5f0 2020-12-05T17:40:06.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.523+0000 7f9618ff9700 1 -- 172.21.15.32:0/1235712587 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f960c025030 con 0x7f961c0af5f0 2020-12-05T17:40:06.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.523+0000 7f9618ff9700 1 --2- 172.21.15.32:0/1235712587 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f96000554e0 0x7f96000579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:06.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.523+0000 7f9618ff9700 1 -- 172.21.15.32:0/1235712587 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f960c06f2d0 con 0x7f961c0af5f0 2020-12-05T17:40:06.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.523+0000 7f961affd700 1 --2- 172.21.15.32:0/1235712587 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f96000554e0 0x7f96000579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:06.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.523+0000 7f961affd700 1 --2- 172.21.15.32:0/1235712587 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f96000554e0 0x7f96000579a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f960400b000 tx=0x7f9604009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:06.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.527+0000 7f9618ff9700 1 -- 172.21.15.32:0/1235712587 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f960c0455d0 con 0x7f961c0af5f0 2020-12-05T17:40:06.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:06.671+0000 7f9620f49700 1 -- 172.21.15.32:0/1235712587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f961c057050 con 0x7f961c0af5f0 2020-12-05T17:40:07.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.375+0000 7f9618ff9700 1 -- 172.21.15.32:0/1235712587 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v105) v1 ==== 173+0+0 (secure 0 0 0) 0x7f960c023020 con 0x7f961c0af5f0 2020-12-05T17:40:07.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.379+0000 7f9620f49700 1 -- 172.21.15.32:0/1235712587 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f96000554e0 msgr2=0x7f96000579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:07.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.379+0000 7f9620f49700 1 --2- 172.21.15.32:0/1235712587 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f96000554e0 0x7f96000579a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f960400b000 tx=0x7f9604009040).stop 2020-12-05T17:40:07.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.379+0000 7f9620f49700 1 -- 172.21.15.32:0/1235712587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f961c0af5f0 msgr2=0x7f961c066650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:07.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.379+0000 7f9620f49700 1 --2- 172.21.15.32:0/1235712587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f961c0af5f0 0x7f961c066650 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 rx=0x7f960c002f30 tx=0x7f960c015000).stop 2020-12-05T17:40:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.379+0000 7f9620f49700 1 -- 172.21.15.32:0/1235712587 shutdown_connections 2020-12-05T17:40:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.379+0000 7f9620f49700 1 --2- 172.21.15.32:0/1235712587 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f961c0b1dc0 0x7f961c155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.379+0000 7f9620f49700 1 --2- 172.21.15.32:0/1235712587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f961c0af5f0 0x7f961c066650 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.379+0000 7f9620f49700 1 --2- 172.21.15.32:0/1235712587 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f96000554e0 0x7f96000579a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.379+0000 7f9620f49700 1 --2- 172.21.15.32:0/1235712587 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f961c0afd30 0x7f961c154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.379+0000 7f9620f49700 1 -- 172.21.15.32:0/1235712587 >> 172.21.15.32:0/1235712587 conn(0x7f961c0620b0 msgr2=0x7f961c145de0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.379+0000 7f9620f49700 1 -- 172.21.15.32:0/1235712587 shutdown_connections 2020-12-05T17:40:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.379+0000 7f9620f49700 1 -- 172.21.15.32:0/1235712587 wait complete. 2020-12-05T17:40:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2020-12-05T17:40:07.400 INFO:tasks.ceph.mds.a:Restarting daemon 2020-12-05T17:40:07.401 INFO:tasks.ceph.mds.a:Stopping old one... 2020-12-05T17:40:07.402 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-12-05T17:40:07.402 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:40:07.402 INFO:tasks.ceph.mds.b:Restarting daemon 2020-12-05T17:40:07.403 INFO:tasks.ceph.mds.b:Stopping old one... 2020-12-05T17:40:07.403 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-12-05T17:40:07.403 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:40:07.415 INFO:tasks.ceph.mds.a:Stopped 2020-12-05T17:40:07.416 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-12-05T17:40:07.417 INFO:tasks.ceph.mds.b:Stopped 2020-12-05T17:40:07.417 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-12-05T17:40:07.460 INFO:tasks.ceph.mds.a:Started 2020-12-05T17:40:07.461 DEBUG:teuthology.parallel:result is None 2020-12-05T17:40:07.464 INFO:tasks.ceph.mds.b:Started 2020-12-05T17:40:07.464 DEBUG:teuthology.parallel:result is None 2020-12-05T17:40:07.465 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-12-05T17:40:07.541 INFO:tasks.ceph.mds.b.smithi061.stdout:starting mds.b at 2020-12-05T17:40:07.559 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:40:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.583+0000 7f4fba1bd700 1 Processor -- start 2020-12-05T17:40:07.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.583+0000 7f4fba1bd700 1 -- start start 2020-12-05T17:40:07.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.583+0000 7f4fba1bd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4fb40b5f00 0x7f4fb40b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:07.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.583+0000 7f4fba1bd700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f4fb40ad510 0x7f4fb40af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:07.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.583+0000 7f4fba1bd700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb4142ab0 con 0x7f4fb4066370 2020-12-05T17:40:07.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.583+0000 7f4fba1bd700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fb40b62e0 con 0x7f4fb40ad510 2020-12-05T17:40:07.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.583+0000 7f4fba1bd700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb40afbf0 con 0x7f4fb40b5f00 2020-12-05T17:40:07.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.583+0000 7f4fb89ba700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f4fb40ad510 0x7f4fb40af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:07.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.583+0000 7f4fb91bb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4fb40b5f00 0x7f4fb40b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:07.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.583+0000 7f4fb89ba700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f4fb40ad510 0x7f4fb40af480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:59096/0 (socket says 172.21.15.32:59096) 2020-12-05T17:40:07.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.583+0000 7f4fb91bb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4fb40b5f00 0x7f4fb40b14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:35992/0 (socket says 172.21.15.32:35992) 2020-12-05T17:40:07.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.583+0000 7f4fb89ba700 1 -- 172.21.15.32:0/3522486077 learned_addr learned my addr 172.21.15.32:0/3522486077 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:07.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.583+0000 7f4fb91bb700 1 -- 172.21.15.32:0/3522486077 >> v2:172.21.15.61:3300/0 conn(0x7f4fb40ad510 msgr2=0x7f4fb40af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:07.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.583+0000 7f4fb91bb700 1 --2- 172.21.15.32:0/3522486077 >> v2:172.21.15.61:3300/0 conn(0x7f4fb40ad510 0x7f4fb40af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:07.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.583+0000 7f4fb91bb700 1 -- 172.21.15.32:0/3522486077 >> v1:172.21.15.61:6789/0 conn(0x7f4fb4066370 legacy=0x7f4fb4142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:07.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.583+0000 7f4fb91bb700 1 -- 172.21.15.32:0/3522486077 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb40afe90 con 0x7f4fb40b5f00 2020-12-05T17:40:07.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.583+0000 7f4fb91bb700 1 --2- 172.21.15.32:0/3522486077 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4fb40b5f00 0x7f4fb40b14e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f4fb0002f30 tx=0x7f4fb0006dc0).ready entity=mon.2 client_cookie=7087ceba060cab70 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:07.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.583+0000 7f4fa3fff700 1 -- 172.21.15.32:0/3522486077 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fb0009760 con 0x7f4fb40b5f00 2020-12-05T17:40:07.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.583+0000 7f4fa3fff700 1 -- 172.21.15.32:0/3522486077 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb00098c0 con 0x7f4fb40b5f00 2020-12-05T17:40:07.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.583+0000 7f4fa3fff700 1 -- 172.21.15.32:0/3522486077 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fb0009c20 con 0x7f4fb40b5f00 2020-12-05T17:40:07.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.583+0000 7f4fba1bd700 1 -- 172.21.15.32:0/3522486077 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4fb40b5f00 msgr2=0x7f4fb40b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:07.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.583+0000 7f4fba1bd700 1 --2- 172.21.15.32:0/3522486077 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4fb40b5f00 0x7f4fb40b14e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f4fb0002f30 tx=0x7f4fb0006dc0).stop 2020-12-05T17:40:07.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fba1bd700 1 -- 172.21.15.32:0/3522486077 shutdown_connections 2020-12-05T17:40:07.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fba1bd700 1 --2- 172.21.15.32:0/3522486077 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4fb40b5f00 0x7f4fb40b14e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:07.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fba1bd700 1 --2- 172.21.15.32:0/3522486077 >> v2:172.21.15.61:3300/0 conn(0x7f4fb40ad510 0x7f4fb40af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:07.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fba1bd700 1 -- 172.21.15.32:0/3522486077 >> 172.21.15.32:0/3522486077 conn(0x7f4fb40604c0 msgr2=0x7f4fb405c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:07.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fba1bd700 1 -- 172.21.15.32:0/3522486077 shutdown_connections 2020-12-05T17:40:07.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fba1bd700 1 -- 172.21.15.32:0/3522486077 wait complete. 2020-12-05T17:40:07.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fba1bd700 1 Processor -- start 2020-12-05T17:40:07.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fba1bd700 1 -- start start 2020-12-05T17:40:07.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fba1bd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4fb4066370 0x7f4fb40b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:07.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fba1bd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fb40ad510 0x7f4fb413df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:07.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fba1bd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4fb40b5f00 0x7f4fb4064380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:07.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fba1bd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb40b3d70 con 0x7f4fb40ad510 2020-12-05T17:40:07.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fba1bd700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb413a7f0 con 0x7f4fb40b5f00 2020-12-05T17:40:07.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fba1bd700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb40b5890 con 0x7f4fb4066370 2020-12-05T17:40:07.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fb89ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fb40ad510 0x7f4fb413df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:07.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fb89ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fb40ad510 0x7f4fb413df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39208/0 (socket says 172.21.15.32:39208) 2020-12-05T17:40:07.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fb89ba700 1 -- 172.21.15.32:0/441694391 learned_addr learned my addr 172.21.15.32:0/441694391 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:07.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fb91bb700 1 --2- 172.21.15.32:0/441694391 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4fb4066370 0x7f4fb40b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:07.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fb99bc700 1 --2- 172.21.15.32:0/441694391 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4fb40b5f00 0x7f4fb4064380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:07.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fb89ba700 1 -- 172.21.15.32:0/441694391 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4fb4066370 msgr2=0x7f4fb40b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:07.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fb89ba700 1 --2- 172.21.15.32:0/441694391 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4fb4066370 0x7f4fb40b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:07.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fb89ba700 1 -- 172.21.15.32:0/441694391 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4fb40b5f00 msgr2=0x7f4fb4064380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:07.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fb89ba700 1 --2- 172.21.15.32:0/441694391 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4fb40b5f00 0x7f4fb4064380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:07.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fb89ba700 1 -- 172.21.15.32:0/441694391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb413a530 con 0x7f4fb40ad510 2020-12-05T17:40:07.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fb89ba700 1 --2- 172.21.15.32:0/441694391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fb40ad510 0x7f4fb413df50 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 rx=0x7f4fa4006b10 tx=0x7f4fa4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:07.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fa27fc700 1 -- 172.21.15.32:0/441694391 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fa400c820 con 0x7f4fb40ad510 2020-12-05T17:40:07.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fa27fc700 1 -- 172.21.15.32:0/441694391 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa400c980 con 0x7f4fb40ad510 2020-12-05T17:40:07.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fba1bd700 1 -- 172.21.15.32:0/441694391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb413d1c0 con 0x7f4fb40ad510 2020-12-05T17:40:07.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.587+0000 7f4fa27fc700 1 -- 172.21.15.32:0/441694391 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fa400cce0 con 0x7f4fb40ad510 2020-12-05T17:40:07.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.591+0000 7f4fba1bd700 1 -- 172.21.15.32:0/441694391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb413a9d0 con 0x7f4fb40ad510 2020-12-05T17:40:07.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.591+0000 7f4fa27fc700 1 -- 172.21.15.32:0/441694391 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4fa401b400 con 0x7f4fb40ad510 2020-12-05T17:40:07.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.591+0000 7f4fa27fc700 1 --2- 172.21.15.32:0/441694391 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4f94055500 0x7f4f940579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:07.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.591+0000 7f4fba1bd700 1 -- 172.21.15.32:0/441694391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb40b3ab0 con 0x7f4fb40ad510 2020-12-05T17:40:07.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.591+0000 7f4fb91bb700 1 --2- 172.21.15.32:0/441694391 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4f94055500 0x7f4f940579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:07.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.591+0000 7f4fa27fc700 1 -- 172.21.15.32:0/441694391 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f4fa406e110 con 0x7f4fb40ad510 2020-12-05T17:40:07.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.591+0000 7f4fb91bb700 1 --2- 172.21.15.32:0/441694391 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4f94055500 0x7f4f940579c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f4fb001a000 tx=0x7f4fb0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:07.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.599+0000 7f4fa27fc700 1 -- 172.21.15.32:0/441694391 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4fb40b3ab0 con 0x7f4fb40ad510 2020-12-05T17:40:07.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:07.755+0000 7f4fba1bd700 1 -- 172.21.15.32:0/441694391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f4fb40b3ab0 con 0x7f4fb40ad510 2020-12-05T17:40:08.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.587+0000 7f4fa27fc700 1 -- 172.21.15.32:0/441694391 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v107) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4fb40b3ab0 con 0x7f4fb40ad510 2020-12-05T17:40:08.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.591+0000 7f4fba1bd700 1 -- 172.21.15.32:0/441694391 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4f94055500 msgr2=0x7f4f940579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:08.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.591+0000 7f4fba1bd700 1 --2- 172.21.15.32:0/441694391 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4f94055500 0x7f4f940579c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f4fb001a000 tx=0x7f4fb0015000).stop 2020-12-05T17:40:08.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.591+0000 7f4fba1bd700 1 -- 172.21.15.32:0/441694391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fb40ad510 msgr2=0x7f4fb413df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:08.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.591+0000 7f4fba1bd700 1 --2- 172.21.15.32:0/441694391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fb40ad510 0x7f4fb413df50 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 rx=0x7f4fa4006b10 tx=0x7f4fa4008040).stop 2020-12-05T17:40:08.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.591+0000 7f4fba1bd700 1 -- 172.21.15.32:0/441694391 shutdown_connections 2020-12-05T17:40:08.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.591+0000 7f4fba1bd700 1 --2- 172.21.15.32:0/441694391 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4fb40b5f00 0x7f4fb4064380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:08.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.591+0000 7f4fba1bd700 1 --2- 172.21.15.32:0/441694391 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4f94055500 0x7f4f940579c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:08.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.591+0000 7f4fba1bd700 1 --2- 172.21.15.32:0/441694391 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4fb4066370 0x7f4fb40b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:08.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.591+0000 7f4fba1bd700 1 --2- 172.21.15.32:0/441694391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fb40ad510 0x7f4fb413df50 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.591+0000 7f4fba1bd700 1 -- 172.21.15.32:0/441694391 >> 172.21.15.32:0/441694391 conn(0x7f4fb40604c0 msgr2=0x7f4fb405c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.591+0000 7f4fba1bd700 1 -- 172.21.15.32:0/441694391 shutdown_connections 2020-12-05T17:40:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.591+0000 7f4fba1bd700 1 -- 172.21.15.32:0/441694391 wait complete. 2020-12-05T17:40:08.600 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2020-12-05T17:40:08.612 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2020-12-05T17:40:08.613 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-12-05T17:40:08.729 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.723+0000 7f2c64d72700 1 Processor -- start 2020-12-05T17:40:08.729 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.723+0000 7f2c64d72700 1 -- start start 2020-12-05T17:40:08.731 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c64d72700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2c600af170 0x7f2c60066a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:08.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c64d72700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c60143b10 con 0x7f2c600aed90 2020-12-05T17:40:08.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c64d72700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c600b1c30 con 0x7f2c600b2140 2020-12-05T17:40:08.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c5f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2c600aed90 0x7f2c601404a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41866/0 (socket says 172.21.15.32:41866) 2020-12-05T17:40:08.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c64d72700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c6013df90 con 0x7f2c600af170 2020-12-05T17:40:08.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c5f7fe700 1 -- 172.21.15.32:0/3983365107 learned_addr learned my addr 172.21.15.32:0/3983365107 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:08.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c5ffff700 1 --2- 172.21.15.32:0/3983365107 >> v2:172.21.15.32:3300/0 conn(0x7f2c600af170 0x7f2c60066a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:08.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c5effd700 1 -- 172.21.15.32:0/3983365107 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3205162118 0 0) 0x7f2c60143b10 con 0x7f2c600aed90 2020-12-05T17:40:08.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c5effd700 1 -- 172.21.15.32:0/3983365107 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c40003230 con 0x7f2c600aed90 2020-12-05T17:40:08.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c5effd700 1 -- 172.21.15.32:0/3983365107 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 250747044 0 0) 0x7f2c40003230 con 0x7f2c600aed90 2020-12-05T17:40:08.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c5ffff700 1 -- 172.21.15.32:0/3983365107 >> v1:172.21.15.61:6789/0 conn(0x7f2c600b2140 legacy=0x7f2c601441e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:08.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c5ffff700 1 -- 172.21.15.32:0/3983365107 >> v1:172.21.15.32:6789/0 conn(0x7f2c600aed90 legacy=0x7f2c601404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:08.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c5ffff700 1 -- 172.21.15.32:0/3983365107 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c600b2650 con 0x7f2c600af170 2020-12-05T17:40:08.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c5ffff700 1 --2- 172.21.15.32:0/3983365107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c600af170 0x7f2c60066a50 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7f2c50001bf0 tx=0x7f2c58006dc0).ready entity=mon.0 client_cookie=82745169bb4ad65a server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:08.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c5effd700 1 -- 172.21.15.32:0/3983365107 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c50001a50 con 0x7f2c600af170 2020-12-05T17:40:08.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c5effd700 1 -- 172.21.15.32:0/3983365107 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c58009670 con 0x7f2c600af170 2020-12-05T17:40:08.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c5effd700 1 -- 172.21.15.32:0/3983365107 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c580097e0 con 0x7f2c600af170 2020-12-05T17:40:08.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c64d72700 1 -- 172.21.15.32:0/3983365107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c600af170 msgr2=0x7f2c60066a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:08.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c64d72700 1 --2- 172.21.15.32:0/3983365107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c600af170 0x7f2c60066a50 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7f2c50001bf0 tx=0x7f2c58006dc0).stop 2020-12-05T17:40:08.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c64d72700 1 -- 172.21.15.32:0/3983365107 shutdown_connections 2020-12-05T17:40:08.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c64d72700 1 --2- 172.21.15.32:0/3983365107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c600af170 0x7f2c60066a50 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:08.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c64d72700 1 -- 172.21.15.32:0/3983365107 >> 172.21.15.32:0/3983365107 conn(0x7f2c6005bbb0 msgr2=0x7f2c60062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:08.740 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c64d72700 1 -- 172.21.15.32:0/3983365107 shutdown_connections 2020-12-05T17:40:08.740 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.727+0000 7f2c64d72700 1 -- 172.21.15.32:0/3983365107 wait complete. 2020-12-05T17:40:08.740 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.731+0000 7f2c64d72700 1 Processor -- start 2020-12-05T17:40:08.740 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.731+0000 7f2c64d72700 1 -- start start 2020-12-05T17:40:08.741 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.731+0000 7f2c64d72700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c600aed90 0x7f2c60154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:08.741 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.731+0000 7f2c64d72700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2c600af170 0x7f2c6013bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:08.741 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.731+0000 7f2c64d72700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c600b2140 0x7f2c60112d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:08.741 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.731+0000 7f2c64d72700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6013e450 con 0x7f2c600b2140 2020-12-05T17:40:08.741 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.731+0000 7f2c64d72700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6013e6f0 con 0x7f2c600af170 2020-12-05T17:40:08.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.731+0000 7f2c64d72700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c600adcc0 con 0x7f2c600aed90 2020-12-05T17:40:08.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.731+0000 7f2c5ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c600b2140 0x7f2c60112d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:08.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.731+0000 7f2c5ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c600b2140 0x7f2c60112d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39228/0 (socket says 172.21.15.32:39228) 2020-12-05T17:40:08.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.731+0000 7f2c5f7fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c600aed90 0x7f2c60154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:08.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.731+0000 7f2c5ffff700 1 -- 172.21.15.32:0/133397695 learned_addr learned my addr 172.21.15.32:0/133397695 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:08.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.731+0000 7f2c56ffd700 1 --2- 172.21.15.32:0/133397695 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2c600af170 0x7f2c6013bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:08.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.731+0000 7f2c5ffff700 1 -- 172.21.15.32:0/133397695 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c600aed90 msgr2=0x7f2c60154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:08.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.731+0000 7f2c5ffff700 1 --2- 172.21.15.32:0/133397695 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c600aed90 0x7f2c60154b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:08.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.731+0000 7f2c5ffff700 1 -- 172.21.15.32:0/133397695 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2c600af170 msgr2=0x7f2c6013bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:08.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.731+0000 7f2c5ffff700 1 --2- 172.21.15.32:0/133397695 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2c600af170 0x7f2c6013bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:08.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.731+0000 7f2c5ffff700 1 -- 172.21.15.32:0/133397695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c6013e0d0 con 0x7f2c600b2140 2020-12-05T17:40:08.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.731+0000 7f2c5ffff700 1 --2- 172.21.15.32:0/133397695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c600b2140 0x7f2c60112d00 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 rx=0x7f2c50001bf0 tx=0x7f2c58015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:08.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.731+0000 7f2c5d7fa700 1 -- 172.21.15.32:0/133397695 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c58011b40 con 0x7f2c600b2140 2020-12-05T17:40:08.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.731+0000 7f2c5d7fa700 1 -- 172.21.15.32:0/133397695 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c58011ca0 con 0x7f2c600b2140 2020-12-05T17:40:08.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.731+0000 7f2c64d72700 1 -- 172.21.15.32:0/133397695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c6013ef10 con 0x7f2c600b2140 2020-12-05T17:40:08.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.731+0000 7f2c5d7fa700 1 -- 172.21.15.32:0/133397695 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c5801d450 con 0x7f2c600b2140 2020-12-05T17:40:08.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.735+0000 7f2c64d72700 1 -- 172.21.15.32:0/133397695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c60113390 con 0x7f2c600b2140 2020-12-05T17:40:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.735+0000 7f2c64d72700 1 -- 172.21.15.32:0/133397695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6013e8b0 con 0x7f2c600b2140 2020-12-05T17:40:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.735+0000 7f2c5d7fa700 1 -- 172.21.15.32:0/133397695 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2c58024030 con 0x7f2c600b2140 2020-12-05T17:40:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.735+0000 7f2c5d7fa700 1 --2- 172.21.15.32:0/133397695 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2c44055330 0x7f2c440577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.735+0000 7f2c5d7fa700 1 -- 172.21.15.32:0/133397695 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f2c5806edb0 con 0x7f2c600b2140 2020-12-05T17:40:08.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.735+0000 7f2c5f7fe700 1 --2- 172.21.15.32:0/133397695 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2c44055330 0x7f2c440577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:08.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.735+0000 7f2c5f7fe700 1 --2- 172.21.15.32:0/133397695 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2c44055330 0x7f2c440577f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f2c5000d000 tx=0x7f2c50001600).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:08.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.743+0000 7f2c5d7fa700 1 -- 172.21.15.32:0/133397695 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2c58045290 con 0x7f2c600b2140 2020-12-05T17:40:08.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.931+0000 7f2c64d72700 1 -- 172.21.15.32:0/133397695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2c60057050 con 0x7f2c600b2140 2020-12-05T17:40:08.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.931+0000 7f2c5d7fa700 1 -- 172.21.15.32:0/133397695 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v34) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f2c58022070 con 0x7f2c600b2140 2020-12-05T17:40:08.937 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:40:08.937 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:40:08.937 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2020-12-05T17:40:08.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:08.938 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2020-12-05T17:40:08.938 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfpb/aExAAdMKW4yIgC2zhkzyW6l8xug==", 2020-12-05T17:40:08.938 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:08.939 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:40:08.939 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:40:08.939 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:40:08.939 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:08.940 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:08.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:08.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:08.940 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2020-12-05T17:40:08.940 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfBKTYGRAAaPkezwmTiAvkUfTuDGDNZA==", 2020-12-05T17:40:08.941 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:08.941 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:40:08.941 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:40:08.941 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:40:08.942 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:08.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:08.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:08.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:08.943 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2020-12-05T17:40:08.943 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHw8tflo5EKhAAwV9D64fAoY3uvXCm+Aksag==", 2020-12-05T17:40:08.943 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:08.943 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:40:08.943 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:40:08.944 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:08.944 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:08.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:08.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:08.945 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2020-12-05T17:40:08.945 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDKw8tfTjEiERAAkW6mkgApoXwyF49kPjZBGg==", 2020-12-05T17:40:08.945 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:08.945 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:40:08.945 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:40:08.946 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:08.946 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:08.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:08.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:08.947 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2020-12-05T17:40:08.947 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDMw8tfLEzBMxAAiJ2NifsOLEkt1e3vmhwSTA==", 2020-12-05T17:40:08.947 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:08.947 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:40:08.947 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:40:08.948 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:08.948 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:08.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:08.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:08.949 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2020-12-05T17:40:08.949 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPw8tfV+vnGxAAQHvXJRW5X9Vhe/eYJBJ/uQ==", 2020-12-05T17:40:08.949 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:08.949 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:40:08.949 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:40:08.950 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:08.950 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:08.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:08.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:08.951 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2020-12-05T17:40:08.951 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUw8tfN8wMJxAApc5LY9HGDeIV/o0eil/g4A==", 2020-12-05T17:40:08.951 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:08.952 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:40:08.952 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:40:08.952 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:08.952 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:08.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:08.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:08.953 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2020-12-05T17:40:08.953 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXw8tfvZ5dDxAAtAfftExkljlBvn7ZSoD/Og==", 2020-12-05T17:40:08.953 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:08.954 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:40:08.954 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:40:08.954 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:08.954 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:08.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:08.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:08.955 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2020-12-05T17:40:08.955 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDZw8tfhXFzMhAAZXNwNGfKv360Tq4N/DX5Mw==", 2020-12-05T17:40:08.955 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:08.956 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:40:08.956 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:40:08.956 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:08.956 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:08.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:08.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:08.957 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2020-12-05T17:40:08.957 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDcw8tfROfSGhAA/Y2Y1b4k5zQju0MM7LASeQ==", 2020-12-05T17:40:08.957 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:08.958 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:40:08.958 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:40:08.958 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:08.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:08.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:08.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:08.959 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2020-12-05T17:40:08.959 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfWDp/HxAAmvLPj7C3WDgj0OC2o7lBJQ==", 2020-12-05T17:40:08.959 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:08.960 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:40:08.960 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:40:08.960 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:40:08.960 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:08.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:08.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:08.961 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2020-12-05T17:40:08.961 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfugnmIRAAW6+/QGMLkHw+Z97RHpV4kw==", 2020-12-05T17:40:08.962 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:08.962 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:40:08.962 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:40:08.962 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:40:08.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:08.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:08.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:08.963 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2020-12-05T17:40:08.963 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfojyqJBAAyta+dSy2yGnC1kKe3YOAYw==", 2020-12-05T17:40:08.964 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:08.964 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:40:08.964 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:40:08.965 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:40:08.965 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:08.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:08.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:08.965 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2020-12-05T17:40:08.966 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDFw8tfES0PLBAAA1cUV44snAUDF2TyAmXImw==", 2020-12-05T17:40:08.966 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:08.966 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:40:08.967 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:40:08.967 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:40:08.967 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:08.967 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:08.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:08.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:08.968 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2020-12-05T17:40:08.968 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfDr8mERAAS7bpAQXgW78W8GDq1v1zaA==", 2020-12-05T17:40:08.969 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:08.969 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:40:08.969 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:40:08.969 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:08.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:08.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:08.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:08.970 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2020-12-05T17:40:08.970 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tf3HmcDhAATNKHhH9zgasedaNEzgG0OQ==", 2020-12-05T17:40:08.971 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:08.971 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:40:08.971 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:40:08.971 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:08.971 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:08.972 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:08.972 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:08.972 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:40:08.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.935+0000 7f2c64d72700 1 -- 172.21.15.32:0/133397695 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2c44055330 msgr2=0x7f2c440577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:08.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.935+0000 7f2c64d72700 1 --2- 172.21.15.32:0/133397695 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2c44055330 0x7f2c440577f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f2c5000d000 tx=0x7f2c50001600).stop 2020-12-05T17:40:08.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.935+0000 7f2c64d72700 1 -- 172.21.15.32:0/133397695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c600b2140 msgr2=0x7f2c60112d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:08.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.935+0000 7f2c64d72700 1 --2- 172.21.15.32:0/133397695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c600b2140 0x7f2c60112d00 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 rx=0x7f2c50001bf0 tx=0x7f2c58015000).stop 2020-12-05T17:40:08.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.935+0000 7f2c64d72700 1 -- 172.21.15.32:0/133397695 shutdown_connections 2020-12-05T17:40:08.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.935+0000 7f2c64d72700 1 --2- 172.21.15.32:0/133397695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c600b2140 0x7f2c60112d00 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:08.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.935+0000 7f2c64d72700 1 --2- 172.21.15.32:0/133397695 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2c44055330 0x7f2c440577f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:08.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.935+0000 7f2c64d72700 1 --2- 172.21.15.32:0/133397695 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c600aed90 0x7f2c60154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:08.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.935+0000 7f2c64d72700 1 --2- 172.21.15.32:0/133397695 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2c600af170 0x7f2c6013bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:08.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.935+0000 7f2c64d72700 1 -- 172.21.15.32:0/133397695 >> 172.21.15.32:0/133397695 conn(0x7f2c6005bbb0 msgr2=0x7f2c6005f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:08.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.935+0000 7f2c64d72700 1 -- 172.21.15.32:0/133397695 shutdown_connections 2020-12-05T17:40:08.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:08.935+0000 7f2c64d72700 1 -- 172.21.15.32:0/133397695 wait complete. 2020-12-05T17:40:08.978 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:40:09.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.091+0000 7fb54359e700 1 Processor -- start 2020-12-05T17:40:09.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.091+0000 7fb54359e700 1 -- start start 2020-12-05T17:40:09.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb54359e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5440b1fc0 0x7fb5440b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:09.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb54359e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb5440af0d0 0x7fb54413c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:09.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb54359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5440b23a0 con 0x7fb54413c860 2020-12-05T17:40:09.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb54359e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb54413f280 con 0x7fb5440af0d0 2020-12-05T17:40:09.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb54359e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54413ccd0 con 0x7fb5440b1fc0 2020-12-05T17:40:09.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb542d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb54413c860 0x7fb54413f670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41882/0 (socket says 172.21.15.32:41882) 2020-12-05T17:40:09.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb542d9d700 1 -- 172.21.15.32:0/1887210316 learned_addr learned my addr 172.21.15.32:0/1887210316 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:09.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb539d9b700 1 --2- 172.21.15.32:0/1887210316 >> v2:172.21.15.61:3300/0 conn(0x7fb5440af0d0 0x7fb54413c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:09.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb54259c700 1 --2- 172.21.15.32:0/1887210316 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5440b1fc0 0x7fb5440b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:09.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb541d9b700 1 -- 172.21.15.32:0/1887210316 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091582286 0 0) 0x7fb5440b23a0 con 0x7fb54413c860 2020-12-05T17:40:09.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb541d9b700 1 -- 172.21.15.32:0/1887210316 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb524003230 con 0x7fb54413c860 2020-12-05T17:40:09.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb541d9b700 1 -- 172.21.15.32:0/1887210316 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 167903613 0 0) 0x7fb524003230 con 0x7fb54413c860 2020-12-05T17:40:09.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb541d9b700 1 -- 172.21.15.32:0/1887210316 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5440b1fc0 msgr2=0x7fb5440b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:09.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb541d9b700 1 --2- 172.21.15.32:0/1887210316 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5440b1fc0 0x7fb5440b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb541d9b700 1 -- 172.21.15.32:0/1887210316 >> v2:172.21.15.61:3300/0 conn(0x7fb5440af0d0 msgr2=0x7fb54413c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:09.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb541d9b700 1 --2- 172.21.15.32:0/1887210316 >> v2:172.21.15.61:3300/0 conn(0x7fb5440af0d0 0x7fb54413c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb541d9b700 1 -- 172.21.15.32:0/1887210316 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5440abfa0 con 0x7fb54413c860 2020-12-05T17:40:09.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb541d9b700 1 -- 172.21.15.32:0/1887210316 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fb53c001a50 con 0x7fb54413c860 2020-12-05T17:40:09.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb541d9b700 1 -- 172.21.15.32:0/1887210316 >> v1:172.21.15.32:6789/0 conn(0x7fb54413c860 legacy=0x7fb54413f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:09.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb541d9b700 1 --2- 172.21.15.32:0/1887210316 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb524003720 0x7fb524003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:09.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb541d9b700 1 --2- 172.21.15.32:0/1887210316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5240044d0 0x7fb524004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:09.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb541d9b700 1 --2- 172.21.15.32:0/1887210316 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5440b1fc0 0x7fb524004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:09.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb541d9b700 1 -- 172.21.15.32:0/1887210316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54413ccd0 con 0x7fb5240044d0 2020-12-05T17:40:09.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb541d9b700 1 -- 172.21.15.32:0/1887210316 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb524005380 con 0x7fb524003720 2020-12-05T17:40:09.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb541d9b700 1 -- 172.21.15.32:0/1887210316 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5240042e0 con 0x7fb5440b1fc0 2020-12-05T17:40:09.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb539d9b700 1 --2- 172.21.15.32:0/1887210316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5240044d0 0x7fb524004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:09.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb54259c700 1 --2- 172.21.15.32:0/1887210316 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb524003720 0x7fb524003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:09.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb542d9d700 1 --2- 172.21.15.32:0/1887210316 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5440b1fc0 0x7fb524004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:09.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb539d9b700 1 -- 172.21.15.32:0/1887210316 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5440b1fc0 msgr2=0x7fb524004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:09.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb539d9b700 1 --2- 172.21.15.32:0/1887210316 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5440b1fc0 0x7fb524004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb539d9b700 1 -- 172.21.15.32:0/1887210316 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb524003720 msgr2=0x7fb524003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb539d9b700 1 --2- 172.21.15.32:0/1887210316 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb524003720 0x7fb524003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb539d9b700 1 -- 172.21.15.32:0/1887210316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb524003230 con 0x7fb5240044d0 2020-12-05T17:40:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb542d9d700 1 --2- 172.21.15.32:0/1887210316 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5440b1fc0 0x7fb524004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:40:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb539d9b700 1 --2- 172.21.15.32:0/1887210316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5240044d0 0x7fb524004950 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7fb52c00c000 tx=0x7fb52c008040).ready entity=mon.0 client_cookie=fb3c1cf4e23591a9 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:09.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb541d9b700 1 -- 172.21.15.32:0/1887210316 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb52c00e970 con 0x7fb5240044d0 2020-12-05T17:40:09.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb541d9b700 1 -- 172.21.15.32:0/1887210316 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52c00ead0 con 0x7fb5240044d0 2020-12-05T17:40:09.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb541d9b700 1 -- 172.21.15.32:0/1887210316 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb52c00ee30 con 0x7fb5240044d0 2020-12-05T17:40:09.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb54359e700 1 -- 172.21.15.32:0/1887210316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5240044d0 msgr2=0x7fb524004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:09.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb54359e700 1 --2- 172.21.15.32:0/1887210316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5240044d0 0x7fb524004950 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7fb52c00c000 tx=0x7fb52c008040).stop 2020-12-05T17:40:09.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb542d9d700 1 -- 172.21.15.32:0/1887210316 reap_dead start 2020-12-05T17:40:09.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb54359e700 1 -- 172.21.15.32:0/1887210316 shutdown_connections 2020-12-05T17:40:09.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb54359e700 1 -- 172.21.15.32:0/1887210316 >> 172.21.15.32:0/1887210316 conn(0x7fb5440b5530 msgr2=0x7fb5440b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:09.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb54359e700 1 -- 172.21.15.32:0/1887210316 shutdown_connections 2020-12-05T17:40:09.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.095+0000 7fb54359e700 1 -- 172.21.15.32:0/1887210316 wait complete. 2020-12-05T17:40:09.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.099+0000 7fb54359e700 1 Processor -- start 2020-12-05T17:40:09.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.099+0000 7fb54359e700 1 -- start start 2020-12-05T17:40:09.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.099+0000 7fb54359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5440adae0 0x7fb54413ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:09.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.099+0000 7fb54359e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb54413e330 0x7fb544140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:09.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.099+0000 7fb54359e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb544137b00 0x7fb544137f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:09.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.099+0000 7fb54359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54413f180 con 0x7fb5440adae0 2020-12-05T17:40:09.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.099+0000 7fb54359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5440adec0 con 0x7fb54413e330 2020-12-05T17:40:09.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.099+0000 7fb54359e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54413e970 con 0x7fb544137b00 2020-12-05T17:40:09.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.103+0000 7fb54259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5440adae0 0x7fb54413ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:09.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.103+0000 7fb54259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5440adae0 0x7fb54413ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39244/0 (socket says 172.21.15.32:39244) 2020-12-05T17:40:09.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.103+0000 7fb54259c700 1 -- 172.21.15.32:0/2871912641 learned_addr learned my addr 172.21.15.32:0/2871912641 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:09.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.103+0000 7fb539d9b700 1 --2- 172.21.15.32:0/2871912641 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb54413e330 0x7fb544140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:09.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.103+0000 7fb542d9d700 1 --2- 172.21.15.32:0/2871912641 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb544137b00 0x7fb544137f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:09.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.103+0000 7fb54259c700 1 -- 172.21.15.32:0/2871912641 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb544137b00 msgr2=0x7fb544137f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:09.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.103+0000 7fb54259c700 1 --2- 172.21.15.32:0/2871912641 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb544137b00 0x7fb544137f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.103+0000 7fb54259c700 1 -- 172.21.15.32:0/2871912641 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb54413e330 msgr2=0x7fb544140ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:09.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.103+0000 7fb54259c700 1 --2- 172.21.15.32:0/2871912641 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb54413e330 0x7fb544140ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.103+0000 7fb54259c700 1 -- 172.21.15.32:0/2871912641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54413eee0 con 0x7fb5440adae0 2020-12-05T17:40:09.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.103+0000 7fb54259c700 1 --2- 172.21.15.32:0/2871912641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5440adae0 0x7fb54413ce10 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0x7fb5440ae050 tx=0x7fb534008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:09.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.103+0000 7fb53bfff700 1 -- 172.21.15.32:0/2871912641 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb53400e8a0 con 0x7fb5440adae0 2020-12-05T17:40:09.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.103+0000 7fb53bfff700 1 -- 172.21.15.32:0/2871912641 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53400ea00 con 0x7fb5440adae0 2020-12-05T17:40:09.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.103+0000 7fb54359e700 1 -- 172.21.15.32:0/2871912641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb544143560 con 0x7fb5440adae0 2020-12-05T17:40:09.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.103+0000 7fb53bfff700 1 -- 172.21.15.32:0/2871912641 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb53400ed60 con 0x7fb5440adae0 2020-12-05T17:40:09.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.103+0000 7fb54359e700 1 -- 172.21.15.32:0/2871912641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb544143230 con 0x7fb5440adae0 2020-12-05T17:40:09.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.103+0000 7fb54359e700 1 -- 172.21.15.32:0/2871912641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb54413eb70 con 0x7fb5440adae0 2020-12-05T17:40:09.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.103+0000 7fb53bfff700 1 -- 172.21.15.32:0/2871912641 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb534024070 con 0x7fb5440adae0 2020-12-05T17:40:09.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.107+0000 7fb53bfff700 1 --2- 172.21.15.32:0/2871912641 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb528055440 0x7fb528057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:09.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.107+0000 7fb53bfff700 1 -- 172.21.15.32:0/2871912641 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fb53406e5b0 con 0x7fb5440adae0 2020-12-05T17:40:09.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.107+0000 7fb539d9b700 1 --2- 172.21.15.32:0/2871912641 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb528055440 0x7fb528057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:09.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.107+0000 7fb539d9b700 1 --2- 172.21.15.32:0/2871912641 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb528055440 0x7fb528057900 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7fb52c00b000 tx=0x7fb52c01a000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:09.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.115+0000 7fb53bfff700 1 -- 172.21.15.32:0/2871912641 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb534044a90 con 0x7fb5440adae0 2020-12-05T17:40:09.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.255+0000 7fb54359e700 1 -- 172.21.15.32:0/2871912641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb544057050 con 0x7fb5440adae0 2020-12-05T17:40:09.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.259+0000 7fb53bfff700 1 -- 172.21.15.32:0/2871912641 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+13421 (secure 0 0 0) 0x7fb5340480b0 con 0x7fb5440adae0 2020-12-05T17:40:09.264 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:40:09.264 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":107,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:40:08.584113+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:40:09.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.259+0000 7fb54359e700 1 -- 172.21.15.32:0/2871912641 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb528055440 msgr2=0x7fb528057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:09.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.259+0000 7fb54359e700 1 --2- 172.21.15.32:0/2871912641 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb528055440 0x7fb528057900 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7fb52c00b000 tx=0x7fb52c01a000).stop 2020-12-05T17:40:09.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.259+0000 7fb54359e700 1 -- 172.21.15.32:0/2871912641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5440adae0 msgr2=0x7fb54413ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:09.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.259+0000 7fb54359e700 1 --2- 172.21.15.32:0/2871912641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5440adae0 0x7fb54413ce10 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0x7fb5440ae050 tx=0x7fb534008040).stop 2020-12-05T17:40:09.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.259+0000 7fb54359e700 1 -- 172.21.15.32:0/2871912641 shutdown_connections 2020-12-05T17:40:09.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.259+0000 7fb54359e700 1 --2- 172.21.15.32:0/2871912641 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb528055440 0x7fb528057900 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.259+0000 7fb54359e700 1 --2- 172.21.15.32:0/2871912641 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb544137b00 0x7fb544137f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.259+0000 7fb54359e700 1 --2- 172.21.15.32:0/2871912641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5440adae0 0x7fb54413ce10 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.259+0000 7fb54359e700 1 --2- 172.21.15.32:0/2871912641 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb54413e330 0x7fb544140ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.259+0000 7fb54359e700 1 -- 172.21.15.32:0/2871912641 >> 172.21.15.32:0/2871912641 conn(0x7fb5440b5530 msgr2=0x7fb544138e90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:09.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.259+0000 7fb54359e700 1 -- 172.21.15.32:0/2871912641 shutdown_connections 2020-12-05T17:40:09.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.259+0000 7fb54359e700 1 -- 172.21.15.32:0/2871912641 wait complete. 2020-12-05T17:40:09.278 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-12-05T17:40:09.279 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:40:09.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.395+0000 7fddfb59e700 1 Processor -- start 2020-12-05T17:40:09.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.395+0000 7fddfb59e700 1 -- start start 2020-12-05T17:40:09.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.395+0000 7fddfb59e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fddfc0b1dc0 0x7fddfc0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:09.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.395+0000 7fddfb59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddfc1468e0 con 0x7fddfc0afd30 2020-12-05T17:40:09.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.395+0000 7fddfb59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddfc146b10 con 0x7fddfc0af5f0 2020-12-05T17:40:09.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.395+0000 7fddfb59e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddfc146db0 con 0x7fddfc0b1dc0 2020-12-05T17:40:09.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddf1d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fddfc0afd30 0x7fddfc0ae600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41902/0 (socket says 172.21.15.32:41902) 2020-12-05T17:40:09.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddf1d9b700 1 -- 172.21.15.32:0/1550817710 learned_addr learned my addr 172.21.15.32:0/1550817710 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:09.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddfa59c700 1 --2- 172.21.15.32:0/1550817710 >> v2:172.21.15.61:3300/0 conn(0x7fddfc0b1dc0 0x7fddfc0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:09.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddf9d9b700 1 -- 172.21.15.32:0/1550817710 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 950305235 0 0) 0x7fddfc1468e0 con 0x7fddfc0afd30 2020-12-05T17:40:09.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddf9d9b700 1 -- 172.21.15.32:0/1550817710 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdddc003230 con 0x7fddfc0afd30 2020-12-05T17:40:09.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddf9d9b700 1 -- 172.21.15.32:0/1550817710 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 353486315 0 0) 0x7fddfc146b10 con 0x7fddfc0af5f0 2020-12-05T17:40:09.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddf9d9b700 1 -- 172.21.15.32:0/1550817710 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddfc1468e0 con 0x7fddfc0af5f0 2020-12-05T17:40:09.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddf9d9b700 1 -- 172.21.15.32:0/1550817710 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 412931428 0 0) 0x7fdddc003230 con 0x7fddfc0afd30 2020-12-05T17:40:09.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddf9d9b700 1 -- 172.21.15.32:0/1550817710 >> v2:172.21.15.61:3300/0 conn(0x7fddfc0b1dc0 msgr2=0x7fddfc0b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:09.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddf9d9b700 1 --2- 172.21.15.32:0/1550817710 >> v2:172.21.15.61:3300/0 conn(0x7fddfc0b1dc0 0x7fddfc0b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddf9d9b700 1 -- 172.21.15.32:0/1550817710 >> v1:172.21.15.61:6789/0 conn(0x7fddfc0af5f0 legacy=0x7fddfc141c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:09.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddf9d9b700 1 -- 172.21.15.32:0/1550817710 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddfc0ae160 con 0x7fddfc0afd30 2020-12-05T17:40:09.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddf9d9b700 1 -- 172.21.15.32:0/1550817710 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fdde40016d0 con 0x7fddfc0afd30 2020-12-05T17:40:09.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddf9d9b700 1 -- 172.21.15.32:0/1550817710 >> v1:172.21.15.32:6789/0 conn(0x7fddfc0afd30 legacy=0x7fddfc0ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:09.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddf9d9b700 1 --2- 172.21.15.32:0/1550817710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdddc0038b0 0x7fdddc003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:09.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddf9d9b700 1 --2- 172.21.15.32:0/1550817710 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdddc0044d0 0x7fdddc004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:09.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddf9d9b700 1 --2- 172.21.15.32:0/1550817710 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdddc004e40 0x7fdddc00b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:09.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddf9d9b700 1 -- 172.21.15.32:0/1550817710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc146db0 con 0x7fdddc0038b0 2020-12-05T17:40:09.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddf9d9b700 1 -- 172.21.15.32:0/1550817710 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc0053a0 con 0x7fdddc0044d0 2020-12-05T17:40:09.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddf9d9b700 1 -- 172.21.15.32:0/1550817710 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc00bc90 con 0x7fdddc004e40 2020-12-05T17:40:09.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddfa59c700 1 --2- 172.21.15.32:0/1550817710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdddc0038b0 0x7fdddc003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:09.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddf1d9b700 1 --2- 172.21.15.32:0/1550817710 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdddc0044d0 0x7fdddc004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:09.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddfad9d700 1 --2- 172.21.15.32:0/1550817710 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdddc004e40 0x7fdddc00b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:09.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddfa59c700 1 -- 172.21.15.32:0/1550817710 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdddc004e40 msgr2=0x7fdddc00b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:09.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddfa59c700 1 --2- 172.21.15.32:0/1550817710 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdddc004e40 0x7fdddc00b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddfa59c700 1 -- 172.21.15.32:0/1550817710 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdddc0044d0 msgr2=0x7fdddc004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:09.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddfa59c700 1 --2- 172.21.15.32:0/1550817710 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdddc0044d0 0x7fdddc004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddfa59c700 1 -- 172.21.15.32:0/1550817710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdddc003230 con 0x7fdddc0038b0 2020-12-05T17:40:09.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddfad9d700 1 -- 172.21.15.32:0/1550817710 reap_dead start 2020-12-05T17:40:09.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddfa59c700 1 --2- 172.21.15.32:0/1550817710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdddc0038b0 0x7fdddc003cb0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7fddec00c000 tx=0x7fddec008040).ready entity=mon.0 client_cookie=5879e16571937931 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:09.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddf9d9b700 1 -- 172.21.15.32:0/1550817710 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddec00e9d0 con 0x7fdddc0038b0 2020-12-05T17:40:09.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddf9d9b700 1 -- 172.21.15.32:0/1550817710 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddec00eb30 con 0x7fdddc0038b0 2020-12-05T17:40:09.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddf9d9b700 1 -- 172.21.15.32:0/1550817710 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddec00ee90 con 0x7fdddc0038b0 2020-12-05T17:40:09.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddfb59e700 1 -- 172.21.15.32:0/1550817710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdddc0038b0 msgr2=0x7fdddc003cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:09.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddfb59e700 1 --2- 172.21.15.32:0/1550817710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdddc0038b0 0x7fdddc003cb0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7fddec00c000 tx=0x7fddec008040).stop 2020-12-05T17:40:09.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddfb59e700 1 -- 172.21.15.32:0/1550817710 shutdown_connections 2020-12-05T17:40:09.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddfb59e700 1 --2- 172.21.15.32:0/1550817710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdddc0038b0 0x7fdddc003cb0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddfb59e700 1 -- 172.21.15.32:0/1550817710 >> 172.21.15.32:0/1550817710 conn(0x7fddfc0620b0 msgr2=0x7fddfc062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:09.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddfb59e700 1 -- 172.21.15.32:0/1550817710 shutdown_connections 2020-12-05T17:40:09.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddfb59e700 1 -- 172.21.15.32:0/1550817710 wait complete. 2020-12-05T17:40:09.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddfb59e700 1 Processor -- start 2020-12-05T17:40:09.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.399+0000 7fddfb59e700 1 -- start start 2020-12-05T17:40:09.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddfb59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdddc0038b0 0x7fddfc1514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:09.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddfb59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddfc148820 0x7fddfc0b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:09.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddfb59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddfc063040 0x7fddfc0634f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:09.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddfb59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc0644e0 con 0x7fddfc148820 2020-12-05T17:40:09.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddfb59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc147900 con 0x7fddfc063040 2020-12-05T17:40:09.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddfb59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc149940 con 0x7fdddc0038b0 2020-12-05T17:40:09.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddf1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddfc148820 0x7fddfc0b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:09.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddf1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddfc148820 0x7fddfc0b0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39264/0 (socket says 172.21.15.32:39264) 2020-12-05T17:40:09.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddf1d9b700 1 -- 172.21.15.32:0/4024943526 learned_addr learned my addr 172.21.15.32:0/4024943526 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:09.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddfad9d700 1 --2- 172.21.15.32:0/4024943526 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddfc063040 0x7fddfc0634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:09.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddfa59c700 1 --2- 172.21.15.32:0/4024943526 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdddc0038b0 0x7fddfc1514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:09.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddf1d9b700 1 -- 172.21.15.32:0/4024943526 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdddc0038b0 msgr2=0x7fddfc1514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:09.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddf1d9b700 1 --2- 172.21.15.32:0/4024943526 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdddc0038b0 0x7fddfc1514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddf1d9b700 1 -- 172.21.15.32:0/4024943526 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddfc063040 msgr2=0x7fddfc0634f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:09.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddf1d9b700 1 --2- 172.21.15.32:0/4024943526 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddfc063040 0x7fddfc0634f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddf1d9b700 1 -- 172.21.15.32:0/4024943526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddfc147af0 con 0x7fddfc148820 2020-12-05T17:40:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddf1d9b700 1 --2- 172.21.15.32:0/4024943526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddfc148820 0x7fddfc0b0740 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0x7fdde400b000 tx=0x7fddfc0ae2f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddf3fff700 1 -- 172.21.15.32:0/4024943526 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdde400c9e0 con 0x7fddfc148820 2020-12-05T17:40:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddf3fff700 1 -- 172.21.15.32:0/4024943526 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde400ee90 con 0x7fddfc148820 2020-12-05T17:40:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddfb59e700 1 -- 172.21.15.32:0/4024943526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddfc0b13d0 con 0x7fddfc148820 2020-12-05T17:40:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddf3fff700 1 -- 172.21.15.32:0/4024943526 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdde400cd30 con 0x7fddfc148820 2020-12-05T17:40:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddfb59e700 1 -- 172.21.15.32:0/4024943526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddfc063b80 con 0x7fddfc148820 2020-12-05T17:40:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddf3fff700 1 -- 172.21.15.32:0/4024943526 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fdde4014430 con 0x7fddfc148820 2020-12-05T17:40:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.403+0000 7fddfb59e700 1 -- 172.21.15.32:0/4024943526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddfc147c80 con 0x7fddfc148820 2020-12-05T17:40:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.407+0000 7fddf3fff700 1 --2- 172.21.15.32:0/4024943526 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdde00553b0 0x7fdde0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.407+0000 7fddf3fff700 1 -- 172.21.15.32:0/4024943526 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fdde406d7f0 con 0x7fddfc148820 2020-12-05T17:40:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.407+0000 7fddfa59c700 1 --2- 172.21.15.32:0/4024943526 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdde00553b0 0x7fdde0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.407+0000 7fddfa59c700 1 --2- 172.21.15.32:0/4024943526 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdde00553b0 0x7fdde0057870 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7fddec008000 tx=0x7fddec01a000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:09.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.411+0000 7fddf3fff700 1 -- 172.21.15.32:0/4024943526 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fdde4043c70 con 0x7fddfc148820 2020-12-05T17:40:09.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.555+0000 7fddfb59e700 1 -- 172.21.15.32:0/4024943526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fddfc149b00 con 0x7fddfc148820 2020-12-05T17:40:09.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.555+0000 7fddf3fff700 1 -- 172.21.15.32:0/4024943526 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v107) v1 ==== 110+0+10 (secure 0 0 0) 0x7fdde4047290 con 0x7fddfc148820 2020-12-05T17:40:09.561 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:40:09.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.555+0000 7fddfb59e700 1 -- 172.21.15.32:0/4024943526 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdde00553b0 msgr2=0x7fdde0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:09.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.555+0000 7fddfb59e700 1 --2- 172.21.15.32:0/4024943526 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdde00553b0 0x7fdde0057870 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7fddec008000 tx=0x7fddec01a000).stop 2020-12-05T17:40:09.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.555+0000 7fddfb59e700 1 -- 172.21.15.32:0/4024943526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddfc148820 msgr2=0x7fddfc0b0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:09.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.555+0000 7fddfb59e700 1 --2- 172.21.15.32:0/4024943526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddfc148820 0x7fddfc0b0740 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0x7fdde400b000 tx=0x7fddfc0ae2f0).stop 2020-12-05T17:40:09.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.559+0000 7fddfb59e700 1 -- 172.21.15.32:0/4024943526 shutdown_connections 2020-12-05T17:40:09.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.559+0000 7fddfb59e700 1 --2- 172.21.15.32:0/4024943526 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddfc063040 0x7fddfc0634f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.559+0000 7fddfb59e700 1 --2- 172.21.15.32:0/4024943526 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdde00553b0 0x7fdde0057870 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.559+0000 7fddfb59e700 1 --2- 172.21.15.32:0/4024943526 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdddc0038b0 0x7fddfc1514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.559+0000 7fddfb59e700 1 --2- 172.21.15.32:0/4024943526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddfc148820 0x7fddfc0b0740 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.559+0000 7fddfb59e700 1 -- 172.21.15.32:0/4024943526 >> 172.21.15.32:0/4024943526 conn(0x7fddfc0620b0 msgr2=0x7fddfc062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.559+0000 7fddfb59e700 1 -- 172.21.15.32:0/4024943526 shutdown_connections 2020-12-05T17:40:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.559+0000 7fddfb59e700 1 -- 172.21.15.32:0/4024943526 wait complete. 2020-12-05T17:40:09.576 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:40:09.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.695+0000 7fa5a359e700 1 Processor -- start 2020-12-05T17:40:09.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.695+0000 7fa5a359e700 1 -- start start 2020-12-05T17:40:09.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a359e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa5a413ed60 0x7fa5a40af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:09.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a40b24e0 con 0x7fa5a40b2100 2020-12-05T17:40:09.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a359e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a40b0780 con 0x7fa5a40ad250 2020-12-05T17:40:09.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a359e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5a413fd10 con 0x7fa5a413ed60 2020-12-05T17:40:09.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa5a40b2100 0x7fa5a4138820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41920/0 (socket says 172.21.15.32:41920) 2020-12-05T17:40:09.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a259c700 1 -- 172.21.15.32:0/493126786 learned_addr learned my addr 172.21.15.32:0/493126786 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:09.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a1d9b700 1 --2- 172.21.15.32:0/493126786 >> v2:172.21.15.61:3300/0 conn(0x7fa5a413ed60 0x7fa5a40af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:09.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a159a700 1 -- 172.21.15.32:0/493126786 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592296436 0 0) 0x7fa5a40b24e0 con 0x7fa5a40b2100 2020-12-05T17:40:09.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a159a700 1 -- 172.21.15.32:0/493126786 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa588003230 con 0x7fa5a40b2100 2020-12-05T17:40:09.707 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a159a700 1 -- 172.21.15.32:0/493126786 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 730798250 0 0) 0x7fa588003230 con 0x7fa5a40b2100 2020-12-05T17:40:09.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a1d9b700 1 -- 172.21.15.32:0/493126786 >> v1:172.21.15.61:6789/0 conn(0x7fa5a40ad250 legacy=0x7fa5a4142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:09.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a1d9b700 1 -- 172.21.15.32:0/493126786 >> v1:172.21.15.32:6789/0 conn(0x7fa5a40b2100 legacy=0x7fa5a4138820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:09.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a1d9b700 1 -- 172.21.15.32:0/493126786 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a40b0240 con 0x7fa5a413ed60 2020-12-05T17:40:09.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a1d9b700 1 --2- 172.21.15.32:0/493126786 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5a413ed60 0x7fa5a40af360 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fa594001870 tx=0x7fa598006e50).ready entity=mon.1 client_cookie=fcd873b859ccb253 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:09.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a159a700 1 -- 172.21.15.32:0/493126786 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa598014070 con 0x7fa5a413ed60 2020-12-05T17:40:09.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a159a700 1 -- 172.21.15.32:0/493126786 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59800a7f0 con 0x7fa5a413ed60 2020-12-05T17:40:09.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a359e700 1 -- 172.21.15.32:0/493126786 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5a413ed60 msgr2=0x7fa5a40af360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:09.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a159a700 1 -- 172.21.15.32:0/493126786 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5940016d0 con 0x7fa5a413ed60 2020-12-05T17:40:09.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a359e700 1 --2- 172.21.15.32:0/493126786 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5a413ed60 0x7fa5a40af360 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fa594001870 tx=0x7fa598006e50).stop 2020-12-05T17:40:09.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a359e700 1 -- 172.21.15.32:0/493126786 shutdown_connections 2020-12-05T17:40:09.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a359e700 1 --2- 172.21.15.32:0/493126786 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5a413ed60 0x7fa5a40af360 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a359e700 1 -- 172.21.15.32:0/493126786 >> 172.21.15.32:0/493126786 conn(0x7fa5a40620b0 msgr2=0x7fa5a4062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:09.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a359e700 1 -- 172.21.15.32:0/493126786 shutdown_connections 2020-12-05T17:40:09.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.699+0000 7fa5a359e700 1 -- 172.21.15.32:0/493126786 wait complete. 2020-12-05T17:40:09.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.703+0000 7fa5a359e700 1 Processor -- start 2020-12-05T17:40:09.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.703+0000 7fa5a359e700 1 -- start start 2020-12-05T17:40:09.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.703+0000 7fa5a359e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5a40ad250 0x7fa5a410e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:09.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.703+0000 7fa5a359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a40b2100 0x7fa5a413a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:09.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.703+0000 7fa5a359e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5a4112570 0x7fa5a413abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:09.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.703+0000 7fa5a359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a413b770 con 0x7fa5a40b2100 2020-12-05T17:40:09.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.703+0000 7fa5a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a413b4d0 con 0x7fa5a40ad250 2020-12-05T17:40:09.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.703+0000 7fa5a359e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a413ce80 con 0x7fa5a4112570 2020-12-05T17:40:09.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.703+0000 7fa5a1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a40b2100 0x7fa5a413a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:09.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.703+0000 7fa5a1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a40b2100 0x7fa5a413a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39280/0 (socket says 172.21.15.32:39280) 2020-12-05T17:40:09.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.703+0000 7fa5a1d9b700 1 -- 172.21.15.32:0/1041313256 learned_addr learned my addr 172.21.15.32:0/1041313256 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:09.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.703+0000 7fa5a259c700 1 --2- 172.21.15.32:0/1041313256 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5a40ad250 0x7fa5a410e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:09.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.703+0000 7fa5a2d9d700 1 --2- 172.21.15.32:0/1041313256 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5a4112570 0x7fa5a413abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:09.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.703+0000 7fa5a1d9b700 1 -- 172.21.15.32:0/1041313256 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5a4112570 msgr2=0x7fa5a413abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:09.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.703+0000 7fa5a1d9b700 1 --2- 172.21.15.32:0/1041313256 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5a4112570 0x7fa5a413abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.703+0000 7fa5a1d9b700 1 -- 172.21.15.32:0/1041313256 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5a40ad250 msgr2=0x7fa5a410e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:09.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.703+0000 7fa5a1d9b700 1 --2- 172.21.15.32:0/1041313256 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5a40ad250 0x7fa5a410e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.703+0000 7fa5a1d9b700 1 -- 172.21.15.32:0/1041313256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a413b0d0 con 0x7fa5a40b2100 2020-12-05T17:40:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.707+0000 7fa5a1d9b700 1 --2- 172.21.15.32:0/1041313256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a40b2100 0x7fa5a413a6f0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7fa594001870 tx=0x7fa5a40b03d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.707+0000 7fa5937fe700 1 -- 172.21.15.32:0/1041313256 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa598010d40 con 0x7fa5a40b2100 2020-12-05T17:40:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.707+0000 7fa5937fe700 1 -- 172.21.15.32:0/1041313256 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa598020070 con 0x7fa5a40b2100 2020-12-05T17:40:09.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.707+0000 7fa5a359e700 1 -- 172.21.15.32:0/1041313256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a413e5a0 con 0x7fa5a40b2100 2020-12-05T17:40:09.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.707+0000 7fa5937fe700 1 -- 172.21.15.32:0/1041313256 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa598010ea0 con 0x7fa5a40b2100 2020-12-05T17:40:09.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.707+0000 7fa5a359e700 1 -- 172.21.15.32:0/1041313256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a413d5f0 con 0x7fa5a40b2100 2020-12-05T17:40:09.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.707+0000 7fa5a359e700 1 -- 172.21.15.32:0/1041313256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a413d040 con 0x7fa5a40b2100 2020-12-05T17:40:09.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.707+0000 7fa5937fe700 1 -- 172.21.15.32:0/1041313256 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa59800e040 con 0x7fa5a40b2100 2020-12-05T17:40:09.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.707+0000 7fa5937fe700 1 --2- 172.21.15.32:0/1041313256 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa5800554e0 0x7fa5800579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:09.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.707+0000 7fa5a259c700 1 --2- 172.21.15.32:0/1041313256 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa5800554e0 0x7fa5800579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:09.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.707+0000 7fa5937fe700 1 -- 172.21.15.32:0/1041313256 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fa59806f5e0 con 0x7fa5a40b2100 2020-12-05T17:40:09.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.711+0000 7fa5a259c700 1 --2- 172.21.15.32:0/1041313256 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa5800554e0 0x7fa5800579a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fa59400d000 tx=0x7fa594001280).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:09.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.715+0000 7fa5937fe700 1 -- 172.21.15.32:0/1041313256 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa598045b70 con 0x7fa5a40b2100 2020-12-05T17:40:09.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.859+0000 7fa5a359e700 1 -- 172.21.15.32:0/1041313256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa5a413b2c0 con 0x7fa5a40b2100 2020-12-05T17:40:09.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.859+0000 7fa5937fe700 1 -- 172.21.15.32:0/1041313256 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v107) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa598049190 con 0x7fa5a40b2100 2020-12-05T17:40:09.866 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:40:09.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.863+0000 7fa5a359e700 1 -- 172.21.15.32:0/1041313256 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa5800554e0 msgr2=0x7fa5800579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:09.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.863+0000 7fa5a359e700 1 --2- 172.21.15.32:0/1041313256 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa5800554e0 0x7fa5800579a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fa59400d000 tx=0x7fa594001280).stop 2020-12-05T17:40:09.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.863+0000 7fa5a359e700 1 -- 172.21.15.32:0/1041313256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a40b2100 msgr2=0x7fa5a413a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:09.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.863+0000 7fa5a359e700 1 --2- 172.21.15.32:0/1041313256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a40b2100 0x7fa5a413a6f0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7fa594001870 tx=0x7fa5a40b03d0).stop 2020-12-05T17:40:09.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.863+0000 7fa5a359e700 1 -- 172.21.15.32:0/1041313256 shutdown_connections 2020-12-05T17:40:09.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.863+0000 7fa5a359e700 1 --2- 172.21.15.32:0/1041313256 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa5800554e0 0x7fa5800579a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.863+0000 7fa5a359e700 1 --2- 172.21.15.32:0/1041313256 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5a4112570 0x7fa5a413abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.863+0000 7fa5a359e700 1 --2- 172.21.15.32:0/1041313256 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5a40ad250 0x7fa5a410e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.863+0000 7fa5a359e700 1 --2- 172.21.15.32:0/1041313256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a40b2100 0x7fa5a413a6f0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:09.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.863+0000 7fa5a359e700 1 -- 172.21.15.32:0/1041313256 >> 172.21.15.32:0/1041313256 conn(0x7fa5a40620b0 msgr2=0x7fa5a413c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:09.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.863+0000 7fa5a359e700 1 -- 172.21.15.32:0/1041313256 shutdown_connections 2020-12-05T17:40:09.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.863+0000 7fa5a359e700 1 -- 172.21.15.32:0/1041313256 wait complete. 2020-12-05T17:40:09.880 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-12-05T17:40:09.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.987+0000 7f9037fff700 1 Processor -- start 2020-12-05T17:40:09.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.987+0000 7f9037fff700 1 -- start start 2020-12-05T17:40:09.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f9037fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f903805f6f0 0x7f90381428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:09.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f9037fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f903805fad0 con 0x7f90380b1980 2020-12-05T17:40:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f9037fff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90380b2f80 con 0x7f90380b0880 2020-12-05T17:40:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f9037fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9038145e40 con 0x7f903805f6f0 2020-12-05T17:40:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f902e7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f90380b1980 0x7f9038146580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41934/0 (socket says 172.21.15.32:41934) 2020-12-05T17:40:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f902e7fc700 1 -- 172.21.15.32:0/2466443189 learned_addr learned my addr 172.21.15.32:0/2466443189 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f9036ffd700 1 --2- 172.21.15.32:0/2466443189 >> v2:172.21.15.32:3300/0 conn(0x7f903805f6f0 0x7f90381428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:09.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f90367fc700 1 -- 172.21.15.32:0/2466443189 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2471922963 0 0) 0x7f903805fad0 con 0x7f90380b1980 2020-12-05T17:40:09.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f90367fc700 1 -- 172.21.15.32:0/2466443189 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9018003230 con 0x7f90380b1980 2020-12-05T17:40:09.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f90367fc700 1 -- 172.21.15.32:0/2466443189 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1596111953 0 0) 0x7f90380b2f80 con 0x7f90380b0880 2020-12-05T17:40:09.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f90367fc700 1 -- 172.21.15.32:0/2466443189 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f903805fad0 con 0x7f90380b0880 2020-12-05T17:40:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f9036ffd700 1 -- 172.21.15.32:0/2466443189 >> v1:172.21.15.61:6789/0 conn(0x7f90380b0880 legacy=0x7f9038066070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f9036ffd700 1 -- 172.21.15.32:0/2466443189 >> v1:172.21.15.32:6789/0 conn(0x7f90380b1980 legacy=0x7f9038146580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f90367fc700 1 -- 172.21.15.32:0/2466443189 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2481254539 0 0) 0x7f9018003230 con 0x7f90380b1980 2020-12-05T17:40:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f9036ffd700 1 -- 172.21.15.32:0/2466443189 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90381460e0 con 0x7f903805f6f0 2020-12-05T17:40:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f9036ffd700 1 --2- 172.21.15.32:0/2466443189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f903805f6f0 0x7f90381428f0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f90200010d0 tx=0x7f9028006dc0).ready entity=mon.0 client_cookie=d15b5bfe4481893 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f90367fc700 1 -- 172.21.15.32:0/2466443189 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90200016d0 con 0x7f903805f6f0 2020-12-05T17:40:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f90367fc700 1 -- 172.21.15.32:0/2466443189 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902800d610 con 0x7f903805f6f0 2020-12-05T17:40:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f90367fc700 1 -- 172.21.15.32:0/2466443189 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f902800d780 con 0x7f903805f6f0 2020-12-05T17:40:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f9037fff700 1 -- 172.21.15.32:0/2466443189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f903805f6f0 msgr2=0x7f90381428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f9037fff700 1 --2- 172.21.15.32:0/2466443189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f903805f6f0 0x7f90381428f0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f90200010d0 tx=0x7f9028006dc0).stop 2020-12-05T17:40:10.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f9037fff700 1 -- 172.21.15.32:0/2466443189 shutdown_connections 2020-12-05T17:40:10.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f9037fff700 1 --2- 172.21.15.32:0/2466443189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f903805f6f0 0x7f90381428f0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:10.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f9037fff700 1 -- 172.21.15.32:0/2466443189 >> 172.21.15.32:0/2466443189 conn(0x7f90380604c0 msgr2=0x7f903805c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f9037fff700 1 -- 172.21.15.32:0/2466443189 shutdown_connections 2020-12-05T17:40:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f9037fff700 1 -- 172.21.15.32:0/2466443189 wait complete. 2020-12-05T17:40:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f9037fff700 1 Processor -- start 2020-12-05T17:40:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.991+0000 7f9037fff700 1 -- start start 2020-12-05T17:40:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.995+0000 7f9037fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f903805f6f0 0x7f90380646c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.995+0000 7f9037fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90380b0880 0x7f9038151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.995+0000 7f9037fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90380b1980 0x7f90381153f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.995+0000 7f9037fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9038114220 con 0x7f90380b1980 2020-12-05T17:40:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.995+0000 7f9037fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9038151e80 con 0x7f903805f6f0 2020-12-05T17:40:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.995+0000 7f9037fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9038113b10 con 0x7f90380b0880 2020-12-05T17:40:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.995+0000 7f90377fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90380b1980 0x7f90381153f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.995+0000 7f90377fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90380b1980 0x7f90381153f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39296/0 (socket says 172.21.15.32:39296) 2020-12-05T17:40:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.995+0000 7f902e7fc700 1 --2- 172.21.15.32:0/1979154706 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90380b0880 0x7f9038151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.995+0000 7f90377fe700 1 -- 172.21.15.32:0/1979154706 learned_addr learned my addr 172.21.15.32:0/1979154706 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:10.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.995+0000 7f9036ffd700 1 --2- 172.21.15.32:0/1979154706 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f903805f6f0 0x7f90380646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:10.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.995+0000 7f90377fe700 1 -- 172.21.15.32:0/1979154706 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90380b0880 msgr2=0x7f9038151990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:10.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.995+0000 7f90377fe700 1 --2- 172.21.15.32:0/1979154706 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90380b0880 0x7f9038151990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:10.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.995+0000 7f90377fe700 1 -- 172.21.15.32:0/1979154706 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f903805f6f0 msgr2=0x7f90380646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:10.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.995+0000 7f90377fe700 1 --2- 172.21.15.32:0/1979154706 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f903805f6f0 0x7f90380646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:10.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.995+0000 7f90377fe700 1 -- 172.21.15.32:0/1979154706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9038113db0 con 0x7f90380b1980 2020-12-05T17:40:10.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.995+0000 7f90377fe700 1 --2- 172.21.15.32:0/1979154706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90380b1980 0x7f90381153f0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 rx=0x7f9030001450 tx=0x7f9030007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:10.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.995+0000 7f9034ff9700 1 -- 172.21.15.32:0/1979154706 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9030002240 con 0x7f90380b1980 2020-12-05T17:40:10.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.995+0000 7f9034ff9700 1 -- 172.21.15.32:0/1979154706 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90300023a0 con 0x7f90380b1980 2020-12-05T17:40:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.995+0000 7f9037fff700 1 -- 172.21.15.32:0/1979154706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9038155990 con 0x7f90380b1980 2020-12-05T17:40:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.995+0000 7f9034ff9700 1 -- 172.21.15.32:0/1979154706 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9030001800 con 0x7f90380b1980 2020-12-05T17:40:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.995+0000 7f9037fff700 1 -- 172.21.15.32:0/1979154706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9038113750 con 0x7f90380b1980 2020-12-05T17:40:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.999+0000 7f9037fff700 1 -- 172.21.15.32:0/1979154706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9038057050 con 0x7f90380b1980 2020-12-05T17:40:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.999+0000 7f9034ff9700 1 -- 172.21.15.32:0/1979154706 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9030010070 con 0x7f90380b1980 2020-12-05T17:40:10.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.999+0000 7f9034ff9700 1 --2- 172.21.15.32:0/1979154706 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f901c0553b0 0x7f901c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:10.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.999+0000 7f9034ff9700 1 -- 172.21.15.32:0/1979154706 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f903006d310 con 0x7f90380b1980 2020-12-05T17:40:10.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.999+0000 7f9036ffd700 1 --2- 172.21.15.32:0/1979154706 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f901c0553b0 0x7f901c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:09.999+0000 7f9036ffd700 1 --2- 172.21.15.32:0/1979154706 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f901c0553b0 0x7f901c057870 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f902801a000 tx=0x7f9028015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.007+0000 7f9034ff9700 1 -- 172.21.15.32:0/1979154706 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9030043750 con 0x7f90380b1980 2020-12-05T17:40:10.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.147+0000 7f9037fff700 1 -- 172.21.15.32:0/1979154706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9038069690 con 0x7f90380b1980 2020-12-05T17:40:10.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.147+0000 7f9034ff9700 1 -- 172.21.15.32:0/1979154706 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v107) v1 ==== 84+0+143 (secure 0 0 0) 0x7f9030046d70 con 0x7f90380b1980 2020-12-05T17:40:10.154 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:40:10.156 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2020-12-05T17:40:10.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:10.156 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2020-12-05T17:40:10.156 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2020-12-05T17:40:10.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:10.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:10.157 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2020-12-05T17:40:10.157 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2020-12-05T17:40:10.157 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:10.157 INFO:teuthology.orchestra.run.smithi032.stdout:] 2020-12-05T17:40:10.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.151+0000 7f9037fff700 1 -- 172.21.15.32:0/1979154706 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f901c0553b0 msgr2=0x7f901c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:10.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.151+0000 7f9037fff700 1 --2- 172.21.15.32:0/1979154706 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f901c0553b0 0x7f901c057870 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f902801a000 tx=0x7f9028015000).stop 2020-12-05T17:40:10.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.151+0000 7f9037fff700 1 -- 172.21.15.32:0/1979154706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90380b1980 msgr2=0x7f90381153f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:10.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.151+0000 7f9037fff700 1 --2- 172.21.15.32:0/1979154706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90380b1980 0x7f90381153f0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 rx=0x7f9030001450 tx=0x7f9030007de0).stop 2020-12-05T17:40:10.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.151+0000 7f9037fff700 1 -- 172.21.15.32:0/1979154706 shutdown_connections 2020-12-05T17:40:10.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.151+0000 7f9037fff700 1 --2- 172.21.15.32:0/1979154706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90380b1980 0x7f90381153f0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:10.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.151+0000 7f9037fff700 1 --2- 172.21.15.32:0/1979154706 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f903805f6f0 0x7f90380646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:10.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.151+0000 7f9037fff700 1 --2- 172.21.15.32:0/1979154706 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f901c0553b0 0x7f901c057870 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:10.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.151+0000 7f9037fff700 1 --2- 172.21.15.32:0/1979154706 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90380b0880 0x7f9038151990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:10.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.151+0000 7f9037fff700 1 -- 172.21.15.32:0/1979154706 >> 172.21.15.32:0/1979154706 conn(0x7f90380604c0 msgr2=0x7f90380aef80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:10.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.151+0000 7f9037fff700 1 -- 172.21.15.32:0/1979154706 shutdown_connections 2020-12-05T17:40:10.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.151+0000 7f9037fff700 1 -- 172.21.15.32:0/1979154706 wait complete. 2020-12-05T17:40:10.168 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-12-05T17:40:10.169 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-12-05T17:40:10.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.283+0000 7f56cb59e700 1 Processor -- start 2020-12-05T17:40:10.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.283+0000 7f56cb59e700 1 -- start start 2020-12-05T17:40:10.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56cb59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56cc0af0d0 0x7f56cc13daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:10.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56cb59e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f56cc13df90 0x7f56cc0637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:10.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56cb59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56cc0b0d70 0x7f56cc063ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:10.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56cb59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56cc05f5c0 con 0x7f56cc0af0d0 2020-12-05T17:40:10.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56cb59e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56cc0af4b0 con 0x7f56cc13df90 2020-12-05T17:40:10.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56cb59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc05f7f0 con 0x7f56cc0b0d70 2020-12-05T17:40:10.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56ca59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56cc0af0d0 0x7f56cc13daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:10.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56ca59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56cc0af0d0 0x7f56cc13daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39300/0 (socket says 172.21.15.32:39300) 2020-12-05T17:40:10.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56ca59c700 1 -- 172.21.15.32:0/1057794630 learned_addr learned my addr 172.21.15.32:0/1057794630 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:10.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56c9d9b700 1 --2- 172.21.15.32:0/1057794630 >> v2:172.21.15.61:3300/0 conn(0x7f56cc13df90 0x7f56cc0637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:10.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56cad9d700 1 --2- 172.21.15.32:0/1057794630 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56cc0b0d70 0x7f56cc063ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:10.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56ca59c700 1 -- 172.21.15.32:0/1057794630 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56cc0b0d70 msgr2=0x7f56cc063ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:10.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56ca59c700 1 --2- 172.21.15.32:0/1057794630 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56cc0b0d70 0x7f56cc063ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:10.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56ca59c700 1 -- 172.21.15.32:0/1057794630 >> v2:172.21.15.61:3300/0 conn(0x7f56cc13df90 msgr2=0x7f56cc0637f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:10.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56ca59c700 1 --2- 172.21.15.32:0/1057794630 >> v2:172.21.15.61:3300/0 conn(0x7f56cc13df90 0x7f56cc0637f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:10.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56ca59c700 1 -- 172.21.15.32:0/1057794630 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc066ae0 con 0x7f56cc0af0d0 2020-12-05T17:40:10.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56c9d9b700 1 --2- 172.21.15.32:0/1057794630 >> v2:172.21.15.61:3300/0 conn(0x7f56cc13df90 0x7f56cc0637f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:40:10.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56ca59c700 1 --2- 172.21.15.32:0/1057794630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56cc0af0d0 0x7f56cc13daa0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7f56bc002f30 tx=0x7f56bc006e80).ready entity=mon.0 client_cookie=39102bcf45189650 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:10.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56c959a700 1 -- 172.21.15.32:0/1057794630 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56bc0097f0 con 0x7f56cc0af0d0 2020-12-05T17:40:10.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56c959a700 1 -- 172.21.15.32:0/1057794630 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc009950 con 0x7f56cc0af0d0 2020-12-05T17:40:10.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56c959a700 1 -- 172.21.15.32:0/1057794630 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56bc009cb0 con 0x7f56cc0af0d0 2020-12-05T17:40:10.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56cb59e700 1 -- 172.21.15.32:0/1057794630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56cc0af0d0 msgr2=0x7f56cc13daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:10.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56cb59e700 1 --2- 172.21.15.32:0/1057794630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56cc0af0d0 0x7f56cc13daa0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7f56bc002f30 tx=0x7f56bc006e80).stop 2020-12-05T17:40:10.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56cb59e700 1 -- 172.21.15.32:0/1057794630 shutdown_connections 2020-12-05T17:40:10.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56cb59e700 1 --2- 172.21.15.32:0/1057794630 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56cc0b0d70 0x7f56cc063ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:10.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56cb59e700 1 --2- 172.21.15.32:0/1057794630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56cc0af0d0 0x7f56cc13daa0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:10.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56cb59e700 1 --2- 172.21.15.32:0/1057794630 >> v2:172.21.15.61:3300/0 conn(0x7f56cc13df90 0x7f56cc0637f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:10.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56cb59e700 1 -- 172.21.15.32:0/1057794630 >> 172.21.15.32:0/1057794630 conn(0x7f56cc0620b0 msgr2=0x7f56cc062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:10.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56cb59e700 1 -- 172.21.15.32:0/1057794630 shutdown_connections 2020-12-05T17:40:10.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.287+0000 7f56cb59e700 1 -- 172.21.15.32:0/1057794630 wait complete. 2020-12-05T17:40:10.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.291+0000 7f56cb59e700 1 Processor -- start 2020-12-05T17:40:10.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.291+0000 7f56cb59e700 1 -- start start 2020-12-05T17:40:10.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.291+0000 7f56cb59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56cc0af0d0 0x7f56cc140160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:10.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.291+0000 7f56cb59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f56cc0b0d70 0x7f56cc0b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:10.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.291+0000 7f56cb59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56cc13df90 0x7f56cc13f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.291+0000 7f56cb59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc0ae870 con 0x7f56cc13df90 2020-12-05T17:40:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.291+0000 7f56cb59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc1441e0 con 0x7f56cc0b0d70 2020-12-05T17:40:10.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.291+0000 7f56cb59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc144480 con 0x7f56cc0af0d0 2020-12-05T17:40:10.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.291+0000 7f56cad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56cc13df90 0x7f56cc13f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:10.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.295+0000 7f56cad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56cc13df90 0x7f56cc13f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39310/0 (socket says 172.21.15.32:39310) 2020-12-05T17:40:10.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.295+0000 7f56cad9d700 1 -- 172.21.15.32:0/1516371104 learned_addr learned my addr 172.21.15.32:0/1516371104 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:10.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.295+0000 7f56ca59c700 1 --2- 172.21.15.32:0/1516371104 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56cc0af0d0 0x7f56cc140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:10.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.295+0000 7f56c9d9b700 1 --2- 172.21.15.32:0/1516371104 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f56cc0b0d70 0x7f56cc0b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:10.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.295+0000 7f56cad9d700 1 -- 172.21.15.32:0/1516371104 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56cc0af0d0 msgr2=0x7f56cc140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:10.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.295+0000 7f56cad9d700 1 --2- 172.21.15.32:0/1516371104 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56cc0af0d0 0x7f56cc140160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:10.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.295+0000 7f56cad9d700 1 -- 172.21.15.32:0/1516371104 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f56cc0b0d70 msgr2=0x7f56cc0b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:10.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.295+0000 7f56cad9d700 1 --2- 172.21.15.32:0/1516371104 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f56cc0b0d70 0x7f56cc0b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:10.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.295+0000 7f56cad9d700 1 -- 172.21.15.32:0/1516371104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc13fe60 con 0x7f56cc13df90 2020-12-05T17:40:10.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.295+0000 7f56cad9d700 1 --2- 172.21.15.32:0/1516371104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56cc13df90 0x7f56cc13f970 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7f56c400a000 tx=0x7f56c4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:10.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.295+0000 7f56bb7fe700 1 -- 172.21.15.32:0/1516371104 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56c400b9a0 con 0x7f56cc13df90 2020-12-05T17:40:10.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.295+0000 7f56bb7fe700 1 -- 172.21.15.32:0/1516371104 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c400bb00 con 0x7f56cc13df90 2020-12-05T17:40:10.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.295+0000 7f56cb59e700 1 -- 172.21.15.32:0/1516371104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56cc0660d0 con 0x7f56cc13df90 2020-12-05T17:40:10.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.295+0000 7f56bb7fe700 1 -- 172.21.15.32:0/1516371104 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56c400be60 con 0x7f56cc13df90 2020-12-05T17:40:10.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.295+0000 7f56cb59e700 1 -- 172.21.15.32:0/1516371104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56cc0ae5d0 con 0x7f56cc13df90 2020-12-05T17:40:10.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.295+0000 7f56bb7fe700 1 -- 172.21.15.32:0/1516371104 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f56c401c460 con 0x7f56cc13df90 2020-12-05T17:40:10.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.295+0000 7f56cb59e700 1 -- 172.21.15.32:0/1516371104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56cc144640 con 0x7f56cc13df90 2020-12-05T17:40:10.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.295+0000 7f56bb7fe700 1 --2- 172.21.15.32:0/1516371104 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f56a8055470 0x7f56a8057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.295+0000 7f56bb7fe700 1 -- 172.21.15.32:0/1516371104 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f56c406e2f0 con 0x7f56cc13df90 2020-12-05T17:40:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.295+0000 7f56ca59c700 1 --2- 172.21.15.32:0/1516371104 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f56a8055470 0x7f56a8057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.299+0000 7f56ca59c700 1 --2- 172.21.15.32:0/1516371104 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f56a8055470 0x7f56a8057930 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f56bc01c000 tx=0x7f56bc015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.303+0000 7f56bb7fe700 1 -- 172.21.15.32:0/1516371104 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f56c4044770 con 0x7f56cc13df90 2020-12-05T17:40:10.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:10.451+0000 7f56cb59e700 1 -- 172.21.15.32:0/1516371104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f56cc057050 con 0x7f56cc13df90 2020-12-05T17:40:11.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.399+0000 7f56bb7fe700 1 -- 172.21.15.32:0/1516371104 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v108) v1 ==== 133+0+0 (secure 0 0 0) 0x7f56c4047d90 con 0x7f56cc13df90 2020-12-05T17:40:11.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.403+0000 7f56cb59e700 1 -- 172.21.15.32:0/1516371104 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f56a8055470 msgr2=0x7f56a8057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:11.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.403+0000 7f56cb59e700 1 --2- 172.21.15.32:0/1516371104 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f56a8055470 0x7f56a8057930 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f56bc01c000 tx=0x7f56bc015000).stop 2020-12-05T17:40:11.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.403+0000 7f56cb59e700 1 -- 172.21.15.32:0/1516371104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56cc13df90 msgr2=0x7f56cc13f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:11.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.403+0000 7f56cb59e700 1 --2- 172.21.15.32:0/1516371104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56cc13df90 0x7f56cc13f970 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7f56c400a000 tx=0x7f56c4008040).stop 2020-12-05T17:40:11.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.407+0000 7f56cb59e700 1 -- 172.21.15.32:0/1516371104 shutdown_connections 2020-12-05T17:40:11.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.407+0000 7f56cb59e700 1 --2- 172.21.15.32:0/1516371104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56cc13df90 0x7f56cc13f970 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:11.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.407+0000 7f56cb59e700 1 --2- 172.21.15.32:0/1516371104 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f56a8055470 0x7f56a8057930 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:11.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.407+0000 7f56cb59e700 1 --2- 172.21.15.32:0/1516371104 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f56cc0af0d0 0x7f56cc140160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:11.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.407+0000 7f56cb59e700 1 --2- 172.21.15.32:0/1516371104 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f56cc0b0d70 0x7f56cc0b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:11.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.407+0000 7f56cb59e700 1 -- 172.21.15.32:0/1516371104 >> 172.21.15.32:0/1516371104 conn(0x7f56cc0620b0 msgr2=0x7f56cc13c150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:11.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.407+0000 7f56cb59e700 1 -- 172.21.15.32:0/1516371104 shutdown_connections 2020-12-05T17:40:11.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.407+0000 7f56cb59e700 1 -- 172.21.15.32:0/1516371104 wait complete. 2020-12-05T17:40:11.413 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2020-12-05T17:40:11.427 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-12-05T17:40:11.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2f0e103700 1 Processor -- start 2020-12-05T17:40:11.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2f0e103700 1 -- start start 2020-12-05T17:40:11.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2f0e103700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f2f080af4b0 0x7f2f08146510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:11.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2f0e103700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f08061780 0x7f2f080afe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:11.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2f0e103700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f080b23e0 con 0x7f2f080b2000 2020-12-05T17:40:11.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2f0e103700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f08146a00 con 0x7f2f080af4b0 2020-12-05T17:40:11.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2f0e103700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f080ae7b0 con 0x7f2f08061780 2020-12-05T17:40:11.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2f0d101700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2f080b2000 0x7f2f08145e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41962/0 (socket says 172.21.15.32:41962) 2020-12-05T17:40:11.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2f0d101700 1 -- 172.21.15.32:0/2213972080 learned_addr learned my addr 172.21.15.32:0/2213972080 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:11.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2f0d902700 1 --2- 172.21.15.32:0/2213972080 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f08061780 0x7f2f080afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:11.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2f0c900700 1 --2- 172.21.15.32:0/2213972080 >> v2:172.21.15.61:3300/0 conn(0x7f2f080af4b0 0x7f2f08146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:11.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2efffff700 1 -- 172.21.15.32:0/2213972080 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 264523378 0 0) 0x7f2f080b23e0 con 0x7f2f080b2000 2020-12-05T17:40:11.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2efffff700 1 -- 172.21.15.32:0/2213972080 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ef4003230 con 0x7f2f080b2000 2020-12-05T17:40:11.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2f0c900700 1 -- 172.21.15.32:0/2213972080 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f08061780 msgr2=0x7f2f080afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:11.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2f0c900700 1 --2- 172.21.15.32:0/2213972080 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f08061780 0x7f2f080afe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:11.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2f0c900700 1 -- 172.21.15.32:0/2213972080 >> v1:172.21.15.32:6789/0 conn(0x7f2f080b2000 legacy=0x7f2f08145e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:11.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2efffff700 1 -- 172.21.15.32:0/2213972080 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1713649056 0 0) 0x7f2ef4003230 con 0x7f2f080b2000 2020-12-05T17:40:11.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2f0c900700 1 -- 172.21.15.32:0/2213972080 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f080aea50 con 0x7f2f080af4b0 2020-12-05T17:40:11.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2f0d902700 1 --2- 172.21.15.32:0/2213972080 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f08061780 0x7f2f080afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:40:11.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2f0c900700 1 --2- 172.21.15.32:0/2213972080 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2f080af4b0 0x7f2f08146510 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f2ef8001870 tx=0x7f2f040091a0).ready entity=mon.1 client_cookie=467246ac5733c093 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:11.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2efffff700 1 -- 172.21.15.32:0/2213972080 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ef80016d0 con 0x7f2f080af4b0 2020-12-05T17:40:11.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2efffff700 1 -- 172.21.15.32:0/2213972080 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0400b800 con 0x7f2f080af4b0 2020-12-05T17:40:11.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2efffff700 1 -- 172.21.15.32:0/2213972080 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f0400b970 con 0x7f2f080af4b0 2020-12-05T17:40:11.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2f0e103700 1 -- 172.21.15.32:0/2213972080 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2f080af4b0 msgr2=0x7f2f08146510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:11.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2f0e103700 1 --2- 172.21.15.32:0/2213972080 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2f080af4b0 0x7f2f08146510 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f2ef8001870 tx=0x7f2f040091a0).stop 2020-12-05T17:40:11.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2f0e103700 1 -- 172.21.15.32:0/2213972080 shutdown_connections 2020-12-05T17:40:11.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2f0e103700 1 --2- 172.21.15.32:0/2213972080 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f08061780 0x7f2f080afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:11.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2f0e103700 1 --2- 172.21.15.32:0/2213972080 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2f080af4b0 0x7f2f08146510 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:11.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.547+0000 7f2f0e103700 1 -- 172.21.15.32:0/2213972080 >> 172.21.15.32:0/2213972080 conn(0x7f2f080b5620 msgr2=0x7f2f080b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:11.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.551+0000 7f2f0e103700 1 -- 172.21.15.32:0/2213972080 shutdown_connections 2020-12-05T17:40:11.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.551+0000 7f2f0e103700 1 -- 172.21.15.32:0/2213972080 wait complete. 2020-12-05T17:40:11.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.551+0000 7f2f0e103700 1 Processor -- start 2020-12-05T17:40:11.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.551+0000 7f2f0e103700 1 -- start start 2020-12-05T17:40:11.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.551+0000 7f2f0e103700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f08061780 0x7f2f0814b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:11.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.551+0000 7f2f0e103700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2f080af4b0 0x7f2f08064c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:11.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.551+0000 7f2f0e103700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f080b2000 0x7f2f08155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:11.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.551+0000 7f2f0e103700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f08063ab0 con 0x7f2f080b2000 2020-12-05T17:40:11.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.551+0000 7f2f0e103700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0814a400 con 0x7f2f080af4b0 2020-12-05T17:40:11.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.551+0000 7f2f0e103700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f08064530 con 0x7f2f08061780 2020-12-05T17:40:11.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.551+0000 7f2f0d902700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f080b2000 0x7f2f08155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:11.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.551+0000 7f2f0d902700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f080b2000 0x7f2f08155ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39324/0 (socket says 172.21.15.32:39324) 2020-12-05T17:40:11.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.551+0000 7f2f0d902700 1 -- 172.21.15.32:0/1229202287 learned_addr learned my addr 172.21.15.32:0/1229202287 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:11.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.551+0000 7f2f0c900700 1 --2- 172.21.15.32:0/1229202287 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2f080af4b0 0x7f2f08064c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:11.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.551+0000 7f2f0d101700 1 --2- 172.21.15.32:0/1229202287 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f08061780 0x7f2f0814b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:11.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.551+0000 7f2f0d902700 1 -- 172.21.15.32:0/1229202287 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f08061780 msgr2=0x7f2f0814b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:11.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.551+0000 7f2f0d902700 1 --2- 172.21.15.32:0/1229202287 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f08061780 0x7f2f0814b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:11.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.551+0000 7f2f0d902700 1 -- 172.21.15.32:0/1229202287 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2f080af4b0 msgr2=0x7f2f08064c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:11.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.551+0000 7f2f0d902700 1 --2- 172.21.15.32:0/1229202287 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2f080af4b0 0x7f2f08064c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:11.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.551+0000 7f2f0d902700 1 -- 172.21.15.32:0/1229202287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0814a670 con 0x7f2f080b2000 2020-12-05T17:40:11.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.551+0000 7f2f0d101700 1 --2- 172.21.15.32:0/1229202287 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f08061780 0x7f2f0814b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:40:11.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.551+0000 7f2f0c900700 1 --2- 172.21.15.32:0/1229202287 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2f080af4b0 0x7f2f08064c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:40:11.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.551+0000 7f2f0d902700 1 --2- 172.21.15.32:0/1229202287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f080b2000 0x7f2f08155ab0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 rx=0x7f2f0814a590 tx=0x7f2f00006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:11.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.555+0000 7f2efe7fc700 1 -- 172.21.15.32:0/1229202287 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f0000cbf0 con 0x7f2f080b2000 2020-12-05T17:40:11.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.555+0000 7f2efe7fc700 1 -- 172.21.15.32:0/1229202287 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0000cd50 con 0x7f2f080b2000 2020-12-05T17:40:11.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.555+0000 7f2f0e103700 1 -- 172.21.15.32:0/1229202287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f080647b0 con 0x7f2f080b2000 2020-12-05T17:40:11.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.555+0000 7f2efe7fc700 1 -- 172.21.15.32:0/1229202287 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f000125e0 con 0x7f2f080b2000 2020-12-05T17:40:11.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.555+0000 7f2f0e103700 1 -- 172.21.15.32:0/1229202287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f08061390 con 0x7f2f080b2000 2020-12-05T17:40:11.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.555+0000 7f2f0e103700 1 -- 172.21.15.32:0/1229202287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f08064940 con 0x7f2f080b2000 2020-12-05T17:40:11.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.555+0000 7f2efe7fc700 1 -- 172.21.15.32:0/1229202287 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2f00012740 con 0x7f2f080b2000 2020-12-05T17:40:11.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.555+0000 7f2efe7fc700 1 --2- 172.21.15.32:0/1229202287 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2ee8055440 0x7f2ee8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:11.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.555+0000 7f2efe7fc700 1 -- 172.21.15.32:0/1229202287 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f2f0006e6a0 con 0x7f2f080b2000 2020-12-05T17:40:11.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.555+0000 7f2f0d101700 1 --2- 172.21.15.32:0/1229202287 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2ee8055440 0x7f2ee8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:11.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.555+0000 7f2f0d101700 1 --2- 172.21.15.32:0/1229202287 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2ee8055440 0x7f2ee8057900 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f2ef800b000 tx=0x7f2ef8001400).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:11.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.563+0000 7f2efe7fc700 1 -- 172.21.15.32:0/1229202287 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2f00044910 con 0x7f2f080b2000 2020-12-05T17:40:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:11.707+0000 7f2f0e103700 1 -- 172.21.15.32:0/1229202287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f2f08057050 con 0x7f2f080b2000 2020-12-05T17:40:12.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.403+0000 7f2efe7fc700 1 -- 172.21.15.32:0/1229202287 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v109) v1 ==== 125+0+0 (secure 0 0 0) 0x7f2f00047f30 con 0x7f2f080b2000 2020-12-05T17:40:12.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.411+0000 7f2f0e103700 1 -- 172.21.15.32:0/1229202287 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2ee8055440 msgr2=0x7f2ee8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:12.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.411+0000 7f2f0e103700 1 --2- 172.21.15.32:0/1229202287 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2ee8055440 0x7f2ee8057900 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f2ef800b000 tx=0x7f2ef8001400).stop 2020-12-05T17:40:12.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.411+0000 7f2f0e103700 1 -- 172.21.15.32:0/1229202287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f080b2000 msgr2=0x7f2f08155ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:12.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.411+0000 7f2f0e103700 1 --2- 172.21.15.32:0/1229202287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f080b2000 0x7f2f08155ab0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 rx=0x7f2f0814a590 tx=0x7f2f00006dc0).stop 2020-12-05T17:40:12.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.415+0000 7f2f0e103700 1 -- 172.21.15.32:0/1229202287 shutdown_connections 2020-12-05T17:40:12.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.415+0000 7f2f0e103700 1 --2- 172.21.15.32:0/1229202287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f080b2000 0x7f2f08155ab0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:12.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.415+0000 7f2f0e103700 1 --2- 172.21.15.32:0/1229202287 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2ee8055440 0x7f2ee8057900 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:12.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.415+0000 7f2f0e103700 1 --2- 172.21.15.32:0/1229202287 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2f08061780 0x7f2f0814b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:12.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.415+0000 7f2f0e103700 1 --2- 172.21.15.32:0/1229202287 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2f080af4b0 0x7f2f08064c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:12.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.415+0000 7f2f0e103700 1 -- 172.21.15.32:0/1229202287 >> 172.21.15.32:0/1229202287 conn(0x7f2f080b5620 msgr2=0x7f2f080b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:12.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.415+0000 7f2f0e103700 1 -- 172.21.15.32:0/1229202287 shutdown_connections 2020-12-05T17:40:12.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.415+0000 7f2f0e103700 1 -- 172.21.15.32:0/1229202287 wait complete. 2020-12-05T17:40:12.421 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2020-12-05T17:40:12.433 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-12-05T17:40:12.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff1258a9700 1 Processor -- start 2020-12-05T17:40:12.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff1258a9700 1 -- start start 2020-12-05T17:40:12.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff1258a9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff120142fc0 0x7ff120064a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:12.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff1258a9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff120145e80 con 0x7ff120061920 2020-12-05T17:40:12.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff1258a9700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff120146120 con 0x7ff1200b16b0 2020-12-05T17:40:12.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff1258a9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1201463c0 con 0x7ff120142fc0 2020-12-05T17:40:12.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff1248a7700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff120061920 0x7ff1201428b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41976/0 (socket says 172.21.15.32:41976) 2020-12-05T17:40:12.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff1248a7700 1 -- 172.21.15.32:0/401702364 learned_addr learned my addr 172.21.15.32:0/401702364 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:12.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff11ffff700 1 --2- 172.21.15.32:0/401702364 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff120142fc0 0x7ff120064a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:12.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff11f7fe700 1 -- 172.21.15.32:0/401702364 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2808922178 0 0) 0x7ff120145e80 con 0x7ff120061920 2020-12-05T17:40:12.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff11f7fe700 1 -- 172.21.15.32:0/401702364 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff104003230 con 0x7ff120061920 2020-12-05T17:40:12.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff11f7fe700 1 -- 172.21.15.32:0/401702364 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3128112528 0 0) 0x7ff120146120 con 0x7ff1200b16b0 2020-12-05T17:40:12.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff11f7fe700 1 -- 172.21.15.32:0/401702364 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff120145e80 con 0x7ff1200b16b0 2020-12-05T17:40:12.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff11ffff700 1 -- 172.21.15.32:0/401702364 >> v1:172.21.15.61:6789/0 conn(0x7ff1200b16b0 legacy=0x7ff12013c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:12.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff11f7fe700 1 -- 172.21.15.32:0/401702364 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1923274425 0 0) 0x7ff104003230 con 0x7ff120061920 2020-12-05T17:40:12.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff11ffff700 1 -- 172.21.15.32:0/401702364 >> v1:172.21.15.32:6789/0 conn(0x7ff120061920 legacy=0x7ff1201428b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:12.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff11ffff700 1 -- 172.21.15.32:0/401702364 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1200626b0 con 0x7ff120142fc0 2020-12-05T17:40:12.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff11ffff700 1 --2- 172.21.15.32:0/401702364 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff120142fc0 0x7ff120064a10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7ff1140010d0 tx=0x7ff10c006dc0).ready entity=mon.2 client_cookie=261153175e8773fe server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:12.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff11f7fe700 1 -- 172.21.15.32:0/401702364 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1140016d0 con 0x7ff120142fc0 2020-12-05T17:40:12.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff11f7fe700 1 -- 172.21.15.32:0/401702364 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10c00d5c0 con 0x7ff120142fc0 2020-12-05T17:40:12.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff11f7fe700 1 -- 172.21.15.32:0/401702364 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff10c00d730 con 0x7ff120142fc0 2020-12-05T17:40:12.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff1258a9700 1 -- 172.21.15.32:0/401702364 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff120142fc0 msgr2=0x7ff120064a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:12.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff1258a9700 1 --2- 172.21.15.32:0/401702364 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff120142fc0 0x7ff120064a10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7ff1140010d0 tx=0x7ff10c006dc0).stop 2020-12-05T17:40:12.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff1258a9700 1 -- 172.21.15.32:0/401702364 shutdown_connections 2020-12-05T17:40:12.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff1258a9700 1 --2- 172.21.15.32:0/401702364 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff120142fc0 0x7ff120064a10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:12.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.543+0000 7ff1258a9700 1 -- 172.21.15.32:0/401702364 >> 172.21.15.32:0/401702364 conn(0x7ff1200b6d10 msgr2=0x7ff1200b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:12.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.547+0000 7ff1258a9700 1 -- 172.21.15.32:0/401702364 shutdown_connections 2020-12-05T17:40:12.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.547+0000 7ff1258a9700 1 -- 172.21.15.32:0/401702364 wait complete. 2020-12-05T17:40:12.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.547+0000 7ff1258a9700 1 Processor -- start 2020-12-05T17:40:12.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.547+0000 7ff1258a9700 1 -- start start 2020-12-05T17:40:12.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.547+0000 7ff1258a9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff120061920 0x7ff1201405c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:12.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.547+0000 7ff1258a9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff1200b16b0 0x7ff1200b3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:12.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.547+0000 7ff1258a9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff120142fc0 0x7ff1200b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:12.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.547+0000 7ff1258a9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1200b3e10 con 0x7ff120142fc0 2020-12-05T17:40:12.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.547+0000 7ff1258a9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1200b4030 con 0x7ff120061920 2020-12-05T17:40:12.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.547+0000 7ff1258a9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1200b42d0 con 0x7ff1200b16b0 2020-12-05T17:40:12.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.547+0000 7ff1250a8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff120142fc0 0x7ff1200b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:12.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.547+0000 7ff1250a8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff120142fc0 0x7ff1200b5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39336/0 (socket says 172.21.15.32:39336) 2020-12-05T17:40:12.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.547+0000 7ff1250a8700 1 -- 172.21.15.32:0/3874583484 learned_addr learned my addr 172.21.15.32:0/3874583484 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:12.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.547+0000 7ff1248a7700 1 --2- 172.21.15.32:0/3874583484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff120061920 0x7ff1201405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:12.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.547+0000 7ff11ffff700 1 --2- 172.21.15.32:0/3874583484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff1200b16b0 0x7ff1200b3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:12.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.547+0000 7ff1250a8700 1 -- 172.21.15.32:0/3874583484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff1200b16b0 msgr2=0x7ff1200b3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:12.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.547+0000 7ff1250a8700 1 --2- 172.21.15.32:0/3874583484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff1200b16b0 0x7ff1200b3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:12.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.547+0000 7ff1250a8700 1 -- 172.21.15.32:0/3874583484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff120061920 msgr2=0x7ff1201405c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:12.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.547+0000 7ff1250a8700 1 --2- 172.21.15.32:0/3874583484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff120061920 0x7ff1201405c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:12.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.547+0000 7ff1250a8700 1 -- 172.21.15.32:0/3874583484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1200b6090 con 0x7ff120142fc0 2020-12-05T17:40:12.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.547+0000 7ff1250a8700 1 --2- 172.21.15.32:0/3874583484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff120142fc0 0x7ff1200b5ba0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7ff118001450 tx=0x7ff118007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:12.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.551+0000 7ff11dffb700 1 -- 172.21.15.32:0/3874583484 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff118002240 con 0x7ff120142fc0 2020-12-05T17:40:12.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.551+0000 7ff11dffb700 1 -- 172.21.15.32:0/3874583484 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1180023a0 con 0x7ff120142fc0 2020-12-05T17:40:12.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.551+0000 7ff1258a9700 1 -- 172.21.15.32:0/3874583484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff120140af0 con 0x7ff120142fc0 2020-12-05T17:40:12.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.551+0000 7ff11dffb700 1 -- 172.21.15.32:0/3874583484 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff118001800 con 0x7ff120142fc0 2020-12-05T17:40:12.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.551+0000 7ff1258a9700 1 -- 172.21.15.32:0/3874583484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12013b1b0 con 0x7ff120142fc0 2020-12-05T17:40:12.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.551+0000 7ff1258a9700 1 -- 172.21.15.32:0/3874583484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1200b4490 con 0x7ff120142fc0 2020-12-05T17:40:12.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.551+0000 7ff11dffb700 1 -- 172.21.15.32:0/3874583484 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff118010070 con 0x7ff120142fc0 2020-12-05T17:40:12.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.551+0000 7ff11dffb700 1 --2- 172.21.15.32:0/3874583484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff1080554e0 0x7ff1080579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:12.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.551+0000 7ff1248a7700 1 --2- 172.21.15.32:0/3874583484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff1080554e0 0x7ff1080579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:12.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.551+0000 7ff11dffb700 1 -- 172.21.15.32:0/3874583484 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6304+0+0 (secure 0 0 0) 0x7ff11806d520 con 0x7ff120142fc0 2020-12-05T17:40:12.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.555+0000 7ff1248a7700 1 --2- 172.21.15.32:0/3874583484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff1080554e0 0x7ff1080579a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7ff11400d000 tx=0x7ff114000b80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:12.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.559+0000 7ff11dffb700 1 -- 172.21.15.32:0/3874583484 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff118043740 con 0x7ff120142fc0 2020-12-05T17:40:12.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.743+0000 7ff1258a9700 1 -- 172.21.15.32:0/3874583484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7ff120057050 con 0x7ff120142fc0 2020-12-05T17:40:12.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.755+0000 7ff11dffb700 1 -- 172.21.15.32:0/3874583484 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 21 and data pool 22 v51) v1 ==== 189+0+0 (secure 0 0 0) 0x7ff118046d60 con 0x7ff120142fc0 2020-12-05T17:40:12.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.763+0000 7ff1258a9700 1 -- 172.21.15.32:0/3874583484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff1080554e0 msgr2=0x7ff1080579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:12.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.763+0000 7ff1258a9700 1 --2- 172.21.15.32:0/3874583484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff1080554e0 0x7ff1080579a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7ff11400d000 tx=0x7ff114000b80).stop 2020-12-05T17:40:12.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.763+0000 7ff1258a9700 1 -- 172.21.15.32:0/3874583484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff120142fc0 msgr2=0x7ff1200b5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:12.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.763+0000 7ff1258a9700 1 --2- 172.21.15.32:0/3874583484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff120142fc0 0x7ff1200b5ba0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7ff118001450 tx=0x7ff118007de0).stop 2020-12-05T17:40:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.763+0000 7ff1258a9700 1 -- 172.21.15.32:0/3874583484 shutdown_connections 2020-12-05T17:40:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.763+0000 7ff1258a9700 1 --2- 172.21.15.32:0/3874583484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff120142fc0 0x7ff1200b5ba0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.763+0000 7ff1258a9700 1 --2- 172.21.15.32:0/3874583484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff120061920 0x7ff1201405c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.763+0000 7ff1258a9700 1 --2- 172.21.15.32:0/3874583484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff1080554e0 0x7ff1080579a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.763+0000 7ff1258a9700 1 --2- 172.21.15.32:0/3874583484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff1200b16b0 0x7ff1200b3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.763+0000 7ff1258a9700 1 -- 172.21.15.32:0/3874583484 >> 172.21.15.32:0/3874583484 conn(0x7ff1200b6d10 msgr2=0x7ff120143d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.763+0000 7ff1258a9700 1 -- 172.21.15.32:0/3874583484 shutdown_connections 2020-12-05T17:40:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.763+0000 7ff1258a9700 1 -- 172.21.15.32:0/3874583484 wait complete. 2020-12-05T17:40:12.770 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 21 and data pool 22 2020-12-05T17:40:12.782 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:40:12.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d21861700 1 Processor -- start 2020-12-05T17:40:12.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d21861700 1 -- start start 2020-12-05T17:40:12.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d21861700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f6d1c067480 0x7f6d1c0ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:12.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d21861700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6d1c0b13b0 0x7f6d1c14ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:12.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d21861700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d1c14a1e0 con 0x7f6d1c0acd60 2020-12-05T17:40:12.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d21861700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d1c060550 con 0x7f6d1c067480 2020-12-05T17:40:12.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d21861700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c05fdc0 con 0x7f6d1c0b13b0 2020-12-05T17:40:12.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d1affd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6d1c0acd60 0x7f6d1c14a640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42002/0 (socket says 172.21.15.32:42002) 2020-12-05T17:40:12.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d1affd700 1 -- 172.21.15.32:0/3105107513 learned_addr learned my addr 172.21.15.32:0/3105107513 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:12.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d1a7fc700 1 --2- 172.21.15.32:0/3105107513 >> v2:172.21.15.61:3300/0 conn(0x7f6d1c067480 0x7f6d1c0ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:12.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d19ffb700 1 -- 172.21.15.32:0/3105107513 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 399240257 0 0) 0x7f6d1c14a1e0 con 0x7f6d1c0acd60 2020-12-05T17:40:12.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d19ffb700 1 -- 172.21.15.32:0/3105107513 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cfc003230 con 0x7f6d1c0acd60 2020-12-05T17:40:12.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d1b7fe700 1 --2- 172.21.15.32:0/3105107513 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6d1c0b13b0 0x7f6d1c14ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:12.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d19ffb700 1 -- 172.21.15.32:0/3105107513 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1070099191 0 0) 0x7f6cfc003230 con 0x7f6d1c0acd60 2020-12-05T17:40:12.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d19ffb700 1 -- 172.21.15.32:0/3105107513 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6d1c0b13b0 msgr2=0x7f6d1c14ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:12.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d19ffb700 1 --2- 172.21.15.32:0/3105107513 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6d1c0b13b0 0x7f6d1c14ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:12.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d19ffb700 1 -- 172.21.15.32:0/3105107513 >> v2:172.21.15.61:3300/0 conn(0x7f6d1c067480 msgr2=0x7f6d1c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:12.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d19ffb700 1 --2- 172.21.15.32:0/3105107513 >> v2:172.21.15.61:3300/0 conn(0x7f6d1c067480 0x7f6d1c0ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d19ffb700 1 -- 172.21.15.32:0/3105107513 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c060060 con 0x7f6d1c0acd60 2020-12-05T17:40:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d19ffb700 1 -- 172.21.15.32:0/3105107513 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f6d0c0016d0 con 0x7f6d1c0acd60 2020-12-05T17:40:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d19ffb700 1 -- 172.21.15.32:0/3105107513 >> v1:172.21.15.32:6789/0 conn(0x7f6d1c0acd60 legacy=0x7f6d1c14a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d19ffb700 1 --2- 172.21.15.32:0/3105107513 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6cfc003720 0x7f6cfc003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:12.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d19ffb700 1 --2- 172.21.15.32:0/3105107513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cfc0044d0 0x7f6cfc004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:12.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d19ffb700 1 --2- 172.21.15.32:0/3105107513 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6d1c0b13b0 0x7f6cfc004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:12.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d19ffb700 1 -- 172.21.15.32:0/3105107513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c060550 con 0x7f6cfc0044d0 2020-12-05T17:40:12.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d19ffb700 1 -- 172.21.15.32:0/3105107513 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c05fdc0 con 0x7f6cfc003720 2020-12-05T17:40:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d19ffb700 1 -- 172.21.15.32:0/3105107513 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc004260 con 0x7f6d1c0b13b0 2020-12-05T17:40:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d1a7fc700 1 --2- 172.21.15.32:0/3105107513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cfc0044d0 0x7f6cfc004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d1b7fe700 1 --2- 172.21.15.32:0/3105107513 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6d1c0b13b0 0x7f6cfc004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.879+0000 7f6d1affd700 1 --2- 172.21.15.32:0/3105107513 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6cfc003720 0x7f6cfc003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.883+0000 7f6d1a7fc700 1 -- 172.21.15.32:0/3105107513 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6d1c0b13b0 msgr2=0x7f6cfc004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.883+0000 7f6d1a7fc700 1 --2- 172.21.15.32:0/3105107513 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6d1c0b13b0 0x7f6cfc004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.883+0000 7f6d1a7fc700 1 -- 172.21.15.32:0/3105107513 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6cfc003720 msgr2=0x7f6cfc003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.883+0000 7f6d1a7fc700 1 --2- 172.21.15.32:0/3105107513 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6cfc003720 0x7f6cfc003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:12.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.883+0000 7f6d1a7fc700 1 -- 172.21.15.32:0/3105107513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cfc003230 con 0x7f6cfc0044d0 2020-12-05T17:40:12.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.883+0000 7f6d1a7fc700 1 --2- 172.21.15.32:0/3105107513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cfc0044d0 0x7f6cfc004950 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7f6d0400c000 tx=0x7f6d040091a0).ready entity=mon.0 client_cookie=fd9cb8f081f45d4c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:12.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.883+0000 7f6d19ffb700 1 -- 172.21.15.32:0/3105107513 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d0400ec30 con 0x7f6cfc0044d0 2020-12-05T17:40:12.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.883+0000 7f6d19ffb700 1 -- 172.21.15.32:0/3105107513 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0400ed90 con 0x7f6cfc0044d0 2020-12-05T17:40:12.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.883+0000 7f6d19ffb700 1 -- 172.21.15.32:0/3105107513 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d040125e0 con 0x7f6cfc0044d0 2020-12-05T17:40:12.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.883+0000 7f6d21861700 1 -- 172.21.15.32:0/3105107513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cfc0044d0 msgr2=0x7f6cfc004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:12.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.883+0000 7f6d21861700 1 --2- 172.21.15.32:0/3105107513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cfc0044d0 0x7f6cfc004950 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7f6d0400c000 tx=0x7f6d040091a0).stop 2020-12-05T17:40:12.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.883+0000 7f6d1b7fe700 1 -- 172.21.15.32:0/3105107513 reap_dead start 2020-12-05T17:40:12.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.883+0000 7f6d21861700 1 -- 172.21.15.32:0/3105107513 shutdown_connections 2020-12-05T17:40:12.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.883+0000 7f6d21861700 1 -- 172.21.15.32:0/3105107513 >> 172.21.15.32:0/3105107513 conn(0x7f6d1c0b5580 msgr2=0x7f6d1c0b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:12.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.883+0000 7f6d21861700 1 -- 172.21.15.32:0/3105107513 shutdown_connections 2020-12-05T17:40:12.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.883+0000 7f6d21861700 1 -- 172.21.15.32:0/3105107513 wait complete. 2020-12-05T17:40:12.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.883+0000 7f6d21861700 1 Processor -- start 2020-12-05T17:40:12.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.883+0000 7f6d21861700 1 -- start start 2020-12-05T17:40:12.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d21861700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6d1c0643a0 0x7f6d1c0b2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:12.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d21861700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6d1c117b70 0x7f6d1c0651e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:12.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d21861700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d1c159910 0x7f6d1c159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:12.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d21861700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c159280 con 0x7f6d1c159910 2020-12-05T17:40:12.902 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d21861700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c064780 con 0x7f6d1c117b70 2020-12-05T17:40:12.902 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d21861700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0b29a0 con 0x7f6d1c0643a0 2020-12-05T17:40:12.902 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d1b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d1c159910 0x7f6d1c159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:12.902 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d1b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d1c159910 0x7f6d1c159d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39370/0 (socket says 172.21.15.32:39370) 2020-12-05T17:40:12.902 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d1b7fe700 1 -- 172.21.15.32:0/3315215172 learned_addr learned my addr 172.21.15.32:0/3315215172 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:12.903 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d1a7fc700 1 --2- 172.21.15.32:0/3315215172 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6d1c117b70 0x7f6d1c0651e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:12.903 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d1affd700 1 --2- 172.21.15.32:0/3315215172 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6d1c0643a0 0x7f6d1c0b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:12.903 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d1b7fe700 1 -- 172.21.15.32:0/3315215172 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6d1c0643a0 msgr2=0x7f6d1c0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:12.904 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d1b7fe700 1 --2- 172.21.15.32:0/3315215172 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6d1c0643a0 0x7f6d1c0b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:12.904 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d1b7fe700 1 -- 172.21.15.32:0/3315215172 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6d1c117b70 msgr2=0x7f6d1c0651e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:12.904 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d1b7fe700 1 --2- 172.21.15.32:0/3315215172 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6d1c117b70 0x7f6d1c0651e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:12.904 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d1b7fe700 1 -- 172.21.15.32:0/3315215172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c159470 con 0x7f6d1c159910 2020-12-05T17:40:12.904 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d1a7fc700 1 --2- 172.21.15.32:0/3315215172 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6d1c117b70 0x7f6d1c0651e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:40:12.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d1affd700 1 --2- 172.21.15.32:0/3315215172 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6d1c0643a0 0x7f6d1c0b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:40:12.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d1b7fe700 1 --2- 172.21.15.32:0/3315215172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d1c159910 0x7f6d1c159d90 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7f6d14008000 tx=0x7f6d14008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:12.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d13fff700 1 -- 172.21.15.32:0/3315215172 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d1400f8b0 con 0x7f6d1c159910 2020-12-05T17:40:12.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d13fff700 1 -- 172.21.15.32:0/3315215172 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1400fa10 con 0x7f6d1c159910 2020-12-05T17:40:12.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d21861700 1 -- 172.21.15.32:0/3315215172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1c117ff0 con 0x7f6d1c159910 2020-12-05T17:40:12.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d13fff700 1 -- 172.21.15.32:0/3315215172 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d1400fd70 con 0x7f6d1c159910 2020-12-05T17:40:12.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d21861700 1 -- 172.21.15.32:0/3315215172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1c1447e0 con 0x7f6d1c159910 2020-12-05T17:40:12.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d21861700 1 -- 172.21.15.32:0/3315215172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1c0b2b60 con 0x7f6d1c159910 2020-12-05T17:40:12.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.887+0000 7f6d13fff700 1 -- 172.21.15.32:0/3315215172 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6d1401e460 con 0x7f6d1c159910 2020-12-05T17:40:12.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.891+0000 7f6d13fff700 1 --2- 172.21.15.32:0/3315215172 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6d000553f0 0x7f6d000578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:12.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.891+0000 7f6d13fff700 1 -- 172.21.15.32:0/3315215172 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6d1406e990 con 0x7f6d1c159910 2020-12-05T17:40:12.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.891+0000 7f6d1affd700 1 --2- 172.21.15.32:0/3315215172 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6d000553f0 0x7f6d000578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:12.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.891+0000 7f6d1affd700 1 --2- 172.21.15.32:0/3315215172 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6d000553f0 0x7f6d000578b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f6d0c005000 tx=0x7f6d0c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:12.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:12.895+0000 7f6d13fff700 1 -- 172.21.15.32:0/3315215172 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6d14044a30 con 0x7f6d1c159910 2020-12-05T17:40:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.039+0000 7f6d21861700 1 -- 172.21.15.32:0/3315215172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d1c159600 con 0x7f6d1c159910 2020-12-05T17:40:13.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.039+0000 7f6d13fff700 1 -- 172.21.15.32:0/3315215172 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f6d14048050 con 0x7f6d1c159910 2020-12-05T17:40:13.044 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:40:13.045 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":110,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:40:12.749016+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:40:10.513416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2020-12-05T17:40:11.768907+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:40:13.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.043+0000 7f6d21861700 1 -- 172.21.15.32:0/3315215172 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6d000553f0 msgr2=0x7f6d000578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:13.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.043+0000 7f6d21861700 1 --2- 172.21.15.32:0/3315215172 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6d000553f0 0x7f6d000578b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f6d0c005000 tx=0x7f6d0c009040).stop 2020-12-05T17:40:13.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.043+0000 7f6d21861700 1 -- 172.21.15.32:0/3315215172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d1c159910 msgr2=0x7f6d1c159d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:13.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.043+0000 7f6d21861700 1 --2- 172.21.15.32:0/3315215172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d1c159910 0x7f6d1c159d90 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7f6d14008000 tx=0x7f6d14008040).stop 2020-12-05T17:40:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.043+0000 7f6d21861700 1 -- 172.21.15.32:0/3315215172 shutdown_connections 2020-12-05T17:40:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.043+0000 7f6d21861700 1 --2- 172.21.15.32:0/3315215172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d1c159910 0x7f6d1c159d90 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.043+0000 7f6d21861700 1 --2- 172.21.15.32:0/3315215172 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6d000553f0 0x7f6d000578b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.043+0000 7f6d21861700 1 --2- 172.21.15.32:0/3315215172 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6d1c0643a0 0x7f6d1c0b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.043+0000 7f6d21861700 1 --2- 172.21.15.32:0/3315215172 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6d1c117b70 0x7f6d1c0651e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.043+0000 7f6d21861700 1 -- 172.21.15.32:0/3315215172 >> 172.21.15.32:0/3315215172 conn(0x7f6d1c0b5580 msgr2=0x7f6d1c0b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.043+0000 7f6d21861700 1 -- 172.21.15.32:0/3315215172 shutdown_connections 2020-12-05T17:40:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.043+0000 7f6d21861700 1 -- 172.21.15.32:0/3315215172 wait complete. 2020-12-05T17:40:13.065 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:40:13.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.163+0000 7f718ee2e700 1 Processor -- start 2020-12-05T17:40:13.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.167+0000 7f718ee2e700 1 -- start start 2020-12-05T17:40:13.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.167+0000 7f718ee2e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71880b7580 0x7f7188145b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:13.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.167+0000 7f718ee2e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71880b1880 0x7f71880b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:13.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.167+0000 7f718ee2e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7188061b30 con 0x7f71880ae750 2020-12-05T17:40:13.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.167+0000 7f718ee2e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71880b21b0 con 0x7f71880b7580 2020-12-05T17:40:13.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.167+0000 7f718ee2e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7188146050 con 0x7f71880b1880 2020-12-05T17:40:13.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.167+0000 7f718de2c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71880b7580 0x7f7188145b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.167+0000 7f718de2c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71880b7580 0x7f7188145b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39374/0 (socket says 172.21.15.32:39374) 2020-12-05T17:40:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.167+0000 7f718de2c700 1 -- 172.21.15.32:0/845104051 learned_addr learned my addr 172.21.15.32:0/845104051 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.167+0000 7f718e62d700 1 --2- 172.21.15.32:0/845104051 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71880b1880 0x7f71880b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.167+0000 7f718de2c700 1 -- 172.21.15.32:0/845104051 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71880b1880 msgr2=0x7f71880b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.167+0000 7f718de2c700 1 --2- 172.21.15.32:0/845104051 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71880b1880 0x7f71880b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.167+0000 7f718de2c700 1 -- 172.21.15.32:0/845104051 >> v1:172.21.15.61:6789/0 conn(0x7f71880ae750 legacy=0x7f718813f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.167+0000 7f718de2c700 1 -- 172.21.15.32:0/845104051 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7188061d70 con 0x7f71880b7580 2020-12-05T17:40:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.167+0000 7f718de2c700 1 --2- 172.21.15.32:0/845104051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71880b7580 0x7f7188145b60 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 rx=0x7f7184002f30 tx=0x7f7184006e80).ready entity=mon.0 client_cookie=de0af807cc1b3e90 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.167+0000 7f718ce2a700 1 -- 172.21.15.32:0/845104051 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71840097f0 con 0x7f71880b7580 2020-12-05T17:40:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.167+0000 7f718ce2a700 1 -- 172.21.15.32:0/845104051 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7184009950 con 0x7f71880b7580 2020-12-05T17:40:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.167+0000 7f718ce2a700 1 -- 172.21.15.32:0/845104051 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7184009cb0 con 0x7f71880b7580 2020-12-05T17:40:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.167+0000 7f718ee2e700 1 -- 172.21.15.32:0/845104051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71880b7580 msgr2=0x7f7188145b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:13.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.167+0000 7f718ee2e700 1 --2- 172.21.15.32:0/845104051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71880b7580 0x7f7188145b60 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 rx=0x7f7184002f30 tx=0x7f7184006e80).stop 2020-12-05T17:40:13.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.171+0000 7f718ee2e700 1 -- 172.21.15.32:0/845104051 shutdown_connections 2020-12-05T17:40:13.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.171+0000 7f718ee2e700 1 --2- 172.21.15.32:0/845104051 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71880b1880 0x7f71880b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.171+0000 7f718ee2e700 1 --2- 172.21.15.32:0/845104051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71880b7580 0x7f7188145b60 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.171+0000 7f718ee2e700 1 -- 172.21.15.32:0/845104051 >> 172.21.15.32:0/845104051 conn(0x7f718805bc20 msgr2=0x7f71880b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:13.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.171+0000 7f718ee2e700 1 -- 172.21.15.32:0/845104051 shutdown_connections 2020-12-05T17:40:13.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.171+0000 7f718ee2e700 1 -- 172.21.15.32:0/845104051 wait complete. 2020-12-05T17:40:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.171+0000 7f718ee2e700 1 Processor -- start 2020-12-05T17:40:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.171+0000 7f718ee2e700 1 -- start start 2020-12-05T17:40:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.175+0000 7f718ee2e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71880ae750 0x7f7188151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.175+0000 7f718ee2e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71880b1880 0x7f7188145400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.175+0000 7f718ee2e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71880b7580 0x7f7188066280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.175+0000 7f718ee2e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71880630f0 con 0x7f71880ae750 2020-12-05T17:40:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.175+0000 7f718ee2e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7188066770 con 0x7f71880b1880 2020-12-05T17:40:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.175+0000 7f718ee2e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7188065610 con 0x7f71880b7580 2020-12-05T17:40:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.175+0000 7f718de2c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71880ae750 0x7f7188151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.175+0000 7f718de2c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71880ae750 0x7f7188151330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39384/0 (socket says 172.21.15.32:39384) 2020-12-05T17:40:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.175+0000 7f718e62d700 1 --2- 172.21.15.32:0/2122953954 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71880b7580 0x7f7188066280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.175+0000 7f718de2c700 1 -- 172.21.15.32:0/2122953954 learned_addr learned my addr 172.21.15.32:0/2122953954 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.175+0000 7f718d62b700 1 --2- 172.21.15.32:0/2122953954 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71880b1880 0x7f7188145400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.175+0000 7f718de2c700 1 -- 172.21.15.32:0/2122953954 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71880b7580 msgr2=0x7f7188066280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.175+0000 7f718de2c700 1 --2- 172.21.15.32:0/2122953954 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71880b7580 0x7f7188066280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.175+0000 7f718de2c700 1 -- 172.21.15.32:0/2122953954 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71880b1880 msgr2=0x7f7188145400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:13.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.175+0000 7f718de2c700 1 --2- 172.21.15.32:0/2122953954 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71880b1880 0x7f7188145400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.175+0000 7f718de2c700 1 -- 172.21.15.32:0/2122953954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7188065370 con 0x7f71880ae750 2020-12-05T17:40:13.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.175+0000 7f718de2c700 1 --2- 172.21.15.32:0/2122953954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71880ae750 0x7f7188151330 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 rx=0x7f7184002f30 tx=0x7f7184015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:13.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.175+0000 7f7176ffd700 1 -- 172.21.15.32:0/2122953954 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7184011910 con 0x7f71880ae750 2020-12-05T17:40:13.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.175+0000 7f7176ffd700 1 -- 172.21.15.32:0/2122953954 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7184011a70 con 0x7f71880ae750 2020-12-05T17:40:13.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.175+0000 7f718ee2e700 1 -- 172.21.15.32:0/2122953954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7188064e60 con 0x7f71880ae750 2020-12-05T17:40:13.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.175+0000 7f7176ffd700 1 -- 172.21.15.32:0/2122953954 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7184011dd0 con 0x7f71880ae750 2020-12-05T17:40:13.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.175+0000 7f718ee2e700 1 -- 172.21.15.32:0/2122953954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7188063770 con 0x7f71880ae750 2020-12-05T17:40:13.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.175+0000 7f718ee2e700 1 -- 172.21.15.32:0/2122953954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71880632b0 con 0x7f71880ae750 2020-12-05T17:40:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.179+0000 7f7176ffd700 1 -- 172.21.15.32:0/2122953954 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7184025030 con 0x7f71880ae750 2020-12-05T17:40:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.179+0000 7f7176ffd700 1 --2- 172.21.15.32:0/2122953954 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7168055470 0x7f7168057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.179+0000 7f7176ffd700 1 -- 172.21.15.32:0/2122953954 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f718406f410 con 0x7f71880ae750 2020-12-05T17:40:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.179+0000 7f718d62b700 1 --2- 172.21.15.32:0/2122953954 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7168055470 0x7f7168057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.179+0000 7f718d62b700 1 --2- 172.21.15.32:0/2122953954 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7168055470 0x7f7168057930 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f717800d000 tx=0x7f7178007b60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:13.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.187+0000 7f7176ffd700 1 -- 172.21.15.32:0/2122953954 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f71880632b0 con 0x7f71880ae750 2020-12-05T17:40:13.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.323+0000 7f718ee2e700 1 -- 172.21.15.32:0/2122953954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71880632b0 con 0x7f71880ae750 2020-12-05T17:40:13.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.323+0000 7f7176ffd700 1 -- 172.21.15.32:0/2122953954 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f71880632b0 con 0x7f71880ae750 2020-12-05T17:40:13.330 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:40:13.330 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":110,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:40:12.749016+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:40:10.513416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2020-12-05T17:40:11.768907+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:40:13.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.327+0000 7f718ee2e700 1 -- 172.21.15.32:0/2122953954 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7168055470 msgr2=0x7f7168057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:13.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.327+0000 7f718ee2e700 1 --2- 172.21.15.32:0/2122953954 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7168055470 0x7f7168057930 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f717800d000 tx=0x7f7178007b60).stop 2020-12-05T17:40:13.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.327+0000 7f718ee2e700 1 -- 172.21.15.32:0/2122953954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71880ae750 msgr2=0x7f7188151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:13.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.327+0000 7f718ee2e700 1 --2- 172.21.15.32:0/2122953954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71880ae750 0x7f7188151330 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 rx=0x7f7184002f30 tx=0x7f7184015000).stop 2020-12-05T17:40:13.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.327+0000 7f718ee2e700 1 -- 172.21.15.32:0/2122953954 shutdown_connections 2020-12-05T17:40:13.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.327+0000 7f718ee2e700 1 --2- 172.21.15.32:0/2122953954 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7168055470 0x7f7168057930 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.327+0000 7f718ee2e700 1 --2- 172.21.15.32:0/2122953954 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71880b7580 0x7f7188066280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.327+0000 7f718ee2e700 1 --2- 172.21.15.32:0/2122953954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71880ae750 0x7f7188151330 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.327+0000 7f718ee2e700 1 --2- 172.21.15.32:0/2122953954 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71880b1880 0x7f7188145400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.327+0000 7f718ee2e700 1 -- 172.21.15.32:0/2122953954 >> 172.21.15.32:0/2122953954 conn(0x7f718805bc20 msgr2=0x7f71880b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:13.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.327+0000 7f718ee2e700 1 -- 172.21.15.32:0/2122953954 shutdown_connections 2020-12-05T17:40:13.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.327+0000 7f718ee2e700 1 -- 172.21.15.32:0/2122953954 wait complete. 2020-12-05T17:40:13.344 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-12-05T17:40:13.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.455+0000 7f819df28700 1 Processor -- start 2020-12-05T17:40:13.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.455+0000 7f819df28700 1 -- start start 2020-12-05T17:40:13.463 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.455+0000 7f819df28700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f819805feb0 0x7f819805f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:13.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.455+0000 7f819df28700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f81980b2fb0 0x7f8198061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:13.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.455+0000 7f819df28700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81980b2df0 con 0x7f819805f3b0 2020-12-05T17:40:13.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.455+0000 7f819df28700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81980b3c90 con 0x7f819805feb0 2020-12-05T17:40:13.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.455+0000 7f819df28700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81980b2b50 con 0x7f81980b2fb0 2020-12-05T17:40:13.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.455+0000 7f8197fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f819805feb0 0x7f819805f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:13.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.455+0000 7f8197fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f819805feb0 0x7f819805f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39390/0 (socket says 172.21.15.32:39390) 2020-12-05T17:40:13.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.455+0000 7f8197fff700 1 -- 172.21.15.32:0/3203946537 learned_addr learned my addr 172.21.15.32:0/3203946537 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:13.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.455+0000 7f819d727700 1 --2- 172.21.15.32:0/3203946537 >> v2:172.21.15.61:3300/0 conn(0x7f81980b2fb0 0x7f8198061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:13.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.455+0000 7f81977fe700 1 -- 172.21.15.32:0/3203946537 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285691444 0 0) 0x7f81980b2df0 con 0x7f819805f3b0 2020-12-05T17:40:13.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.455+0000 7f81977fe700 1 -- 172.21.15.32:0/3203946537 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8178003230 con 0x7f819805f3b0 2020-12-05T17:40:13.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.455+0000 7f8197fff700 1 -- 172.21.15.32:0/3203946537 >> v2:172.21.15.61:3300/0 conn(0x7f81980b2fb0 msgr2=0x7f8198061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:13.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.455+0000 7f8197fff700 1 --2- 172.21.15.32:0/3203946537 >> v2:172.21.15.61:3300/0 conn(0x7f81980b2fb0 0x7f8198061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.455+0000 7f8197fff700 1 -- 172.21.15.32:0/3203946537 >> v1:172.21.15.61:6789/0 conn(0x7f819805f3b0 legacy=0x7f819813c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.455+0000 7f8197fff700 1 -- 172.21.15.32:0/3203946537 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8198142820 con 0x7f819805feb0 2020-12-05T17:40:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.459+0000 7f8197fff700 1 --2- 172.21.15.32:0/3203946537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819805feb0 0x7f819805f790 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7f8180002f30 tx=0x7f8180006e80).ready entity=mon.0 client_cookie=f7bd1838e011fbbb server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.459+0000 7f81977fe700 1 -- 172.21.15.32:0/3203946537 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8180009640 con 0x7f819805feb0 2020-12-05T17:40:13.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.459+0000 7f81977fe700 1 -- 172.21.15.32:0/3203946537 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81800097a0 con 0x7f819805feb0 2020-12-05T17:40:13.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.459+0000 7f81977fe700 1 -- 172.21.15.32:0/3203946537 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8180009b00 con 0x7f819805feb0 2020-12-05T17:40:13.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.459+0000 7f819df28700 1 -- 172.21.15.32:0/3203946537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819805feb0 msgr2=0x7f819805f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:13.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.459+0000 7f819df28700 1 --2- 172.21.15.32:0/3203946537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819805feb0 0x7f819805f790 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7f8180002f30 tx=0x7f8180006e80).stop 2020-12-05T17:40:13.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.459+0000 7f819df28700 1 -- 172.21.15.32:0/3203946537 shutdown_connections 2020-12-05T17:40:13.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.459+0000 7f819df28700 1 --2- 172.21.15.32:0/3203946537 >> v2:172.21.15.61:3300/0 conn(0x7f81980b2fb0 0x7f8198061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.459+0000 7f819df28700 1 --2- 172.21.15.32:0/3203946537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819805feb0 0x7f819805f790 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.459+0000 7f819df28700 1 -- 172.21.15.32:0/3203946537 >> 172.21.15.32:0/3203946537 conn(0x7f81980b8f20 msgr2=0x7f81980b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:13.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.459+0000 7f819df28700 1 -- 172.21.15.32:0/3203946537 shutdown_connections 2020-12-05T17:40:13.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.459+0000 7f819df28700 1 -- 172.21.15.32:0/3203946537 wait complete. 2020-12-05T17:40:13.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.459+0000 7f819df28700 1 Processor -- start 2020-12-05T17:40:13.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.459+0000 7f819df28700 1 -- start start 2020-12-05T17:40:13.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.463+0000 7f819df28700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f819805f3b0 0x7f8198144e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:13.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.463+0000 7f819df28700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819805feb0 0x7f8198140a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:13.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.463+0000 7f819df28700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f81980b2fb0 0x7f8198140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:13.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.463+0000 7f819df28700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8198112720 con 0x7f819805feb0 2020-12-05T17:40:13.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.463+0000 7f819df28700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8198145af0 con 0x7f819805f3b0 2020-12-05T17:40:13.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.463+0000 7f819df28700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81981496e0 con 0x7f81980b2fb0 2020-12-05T17:40:13.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.463+0000 7f8197fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819805feb0 0x7f8198140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:13.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.463+0000 7f8197fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819805feb0 0x7f8198140a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39396/0 (socket says 172.21.15.32:39396) 2020-12-05T17:40:13.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.463+0000 7f8197fff700 1 -- 172.21.15.32:0/2354304990 learned_addr learned my addr 172.21.15.32:0/2354304990 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:13.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.463+0000 7f819cf26700 1 --2- 172.21.15.32:0/2354304990 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f819805f3b0 0x7f8198144e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:13.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.463+0000 7f819d727700 1 --2- 172.21.15.32:0/2354304990 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f81980b2fb0 0x7f8198140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:13.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.463+0000 7f8197fff700 1 -- 172.21.15.32:0/2354304990 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f81980b2fb0 msgr2=0x7f8198140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:13.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.463+0000 7f8197fff700 1 --2- 172.21.15.32:0/2354304990 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f81980b2fb0 0x7f8198140ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.463+0000 7f8197fff700 1 -- 172.21.15.32:0/2354304990 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f819805f3b0 msgr2=0x7f8198144e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:13.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.463+0000 7f8197fff700 1 --2- 172.21.15.32:0/2354304990 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f819805f3b0 0x7f8198144e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.463+0000 7f8197fff700 1 -- 172.21.15.32:0/2354304990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8198147b40 con 0x7f819805feb0 2020-12-05T17:40:13.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.463+0000 7f8197fff700 1 --2- 172.21.15.32:0/2354304990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819805feb0 0x7f8198140a00 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 rx=0x7f8198146ed0 tx=0x7f8180015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:13.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.463+0000 7f8195ffb700 1 -- 172.21.15.32:0/2354304990 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8180011a40 con 0x7f819805feb0 2020-12-05T17:40:13.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.463+0000 7f8195ffb700 1 -- 172.21.15.32:0/2354304990 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8180011ba0 con 0x7f819805feb0 2020-12-05T17:40:13.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.463+0000 7f819df28700 1 -- 172.21.15.32:0/2354304990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8198149910 con 0x7f819805feb0 2020-12-05T17:40:13.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.463+0000 7f8195ffb700 1 -- 172.21.15.32:0/2354304990 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f818001d450 con 0x7f819805feb0 2020-12-05T17:40:13.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.463+0000 7f819df28700 1 -- 172.21.15.32:0/2354304990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81981413e0 con 0x7f819805feb0 2020-12-05T17:40:13.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.463+0000 7f819df28700 1 -- 172.21.15.32:0/2354304990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8198057050 con 0x7f819805feb0 2020-12-05T17:40:13.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.467+0000 7f8195ffb700 1 -- 172.21.15.32:0/2354304990 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8180024030 con 0x7f819805feb0 2020-12-05T17:40:13.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.467+0000 7f8195ffb700 1 --2- 172.21.15.32:0/2354304990 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f817c0553b0 0x7f817c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:13.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.467+0000 7f8195ffb700 1 -- 172.21.15.32:0/2354304990 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f818006f2e0 con 0x7f819805feb0 2020-12-05T17:40:13.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.467+0000 7f819cf26700 1 --2- 172.21.15.32:0/2354304990 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f817c0553b0 0x7f817c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:13.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.467+0000 7f819cf26700 1 --2- 172.21.15.32:0/2354304990 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f817c0553b0 0x7f817c057870 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f818800d000 tx=0x7f8188007aa0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:13.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.475+0000 7f8195ffb700 1 -- 172.21.15.32:0/2354304990 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8180045370 con 0x7f819805feb0 2020-12-05T17:40:13.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.659+0000 7f819df28700 1 -- 172.21.15.32:0/2354304990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f819805aab0 con 0x7f819805feb0 2020-12-05T17:40:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.763+0000 7f8195ffb700 1 -- 172.21.15.32:0/2354304990 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v52) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8180022020 con 0x7f819805feb0 2020-12-05T17:40:13.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.767+0000 7f819df28700 1 -- 172.21.15.32:0/2354304990 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f817c0553b0 msgr2=0x7f817c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:13.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.767+0000 7f819df28700 1 --2- 172.21.15.32:0/2354304990 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f817c0553b0 0x7f817c057870 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f818800d000 tx=0x7f8188007aa0).stop 2020-12-05T17:40:13.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.767+0000 7f819df28700 1 -- 172.21.15.32:0/2354304990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819805feb0 msgr2=0x7f8198140a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:13.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.767+0000 7f819df28700 1 --2- 172.21.15.32:0/2354304990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819805feb0 0x7f8198140a00 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 rx=0x7f8198146ed0 tx=0x7f8180015000).stop 2020-12-05T17:40:13.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.767+0000 7f819df28700 1 -- 172.21.15.32:0/2354304990 shutdown_connections 2020-12-05T17:40:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.767+0000 7f819df28700 1 --2- 172.21.15.32:0/2354304990 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f817c0553b0 0x7f817c057870 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.767+0000 7f819df28700 1 --2- 172.21.15.32:0/2354304990 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f81980b2fb0 0x7f8198140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.767+0000 7f819df28700 1 --2- 172.21.15.32:0/2354304990 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f819805f3b0 0x7f8198144e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.767+0000 7f819df28700 1 --2- 172.21.15.32:0/2354304990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819805feb0 0x7f8198140a00 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.767+0000 7f819df28700 1 -- 172.21.15.32:0/2354304990 >> 172.21.15.32:0/2354304990 conn(0x7f81980b8f20 msgr2=0x7f81980b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.767+0000 7f819df28700 1 -- 172.21.15.32:0/2354304990 shutdown_connections 2020-12-05T17:40:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.767+0000 7f819df28700 1 -- 172.21.15.32:0/2354304990 wait complete. 2020-12-05T17:40:13.785 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:40:13.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.891+0000 7fe9e1b9d700 1 Processor -- start 2020-12-05T17:40:13.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.891+0000 7fe9e1b9d700 1 -- start start 2020-12-05T17:40:13.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.895+0000 7fe9e1b9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe9dc05f3b0 0x7fe9dc05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:13.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.895+0000 7fe9e1b9d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fe9dc062400 0x7fe9dc0b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:13.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.895+0000 7fe9e1b9d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe9dc05feb0 0x7fe9dc0b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:13.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.895+0000 7fe9e1b9d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9dc142450 con 0x7fe9dc05f3b0 2020-12-05T17:40:13.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.895+0000 7fe9e1b9d700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9dc061780 con 0x7fe9dc062400 2020-12-05T17:40:13.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.895+0000 7fe9e1b9d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc061a20 con 0x7fe9dc05feb0 2020-12-05T17:40:13.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.895+0000 7fe9e0b9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe9dc05f3b0 0x7fe9dc05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:13.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.895+0000 7fe9e0b9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe9dc05f3b0 0x7fe9dc05f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39402/0 (socket says 172.21.15.32:39402) 2020-12-05T17:40:13.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.895+0000 7fe9e0b9b700 1 -- 172.21.15.32:0/1586567643 learned_addr learned my addr 172.21.15.32:0/1586567643 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:13.902 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.895+0000 7fe9dbfff700 1 --2- 172.21.15.32:0/1586567643 >> v2:172.21.15.61:3300/0 conn(0x7fe9dc062400 0x7fe9dc0b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:13.903 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.895+0000 7fe9e139c700 1 --2- 172.21.15.32:0/1586567643 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe9dc05feb0 0x7fe9dc0b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:13.904 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.895+0000 7fe9e0b9b700 1 -- 172.21.15.32:0/1586567643 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe9dc05feb0 msgr2=0x7fe9dc0b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:13.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.895+0000 7fe9e0b9b700 1 --2- 172.21.15.32:0/1586567643 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe9dc05feb0 0x7fe9dc0b3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.895+0000 7fe9e0b9b700 1 -- 172.21.15.32:0/1586567643 >> v2:172.21.15.61:3300/0 conn(0x7fe9dc062400 msgr2=0x7fe9dc0b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:13.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.895+0000 7fe9e0b9b700 1 --2- 172.21.15.32:0/1586567643 >> v2:172.21.15.61:3300/0 conn(0x7fe9dc062400 0x7fe9dc0b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.895+0000 7fe9e0b9b700 1 -- 172.21.15.32:0/1586567643 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9dc061cc0 con 0x7fe9dc05f3b0 2020-12-05T17:40:13.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.895+0000 7fe9e0b9b700 1 --2- 172.21.15.32:0/1586567643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9dc05f3b0 0x7fe9dc05f790 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7fe9c4002f30 tx=0x7fe9c4006e80).ready entity=mon.0 client_cookie=64049324c09fe4e3 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:13.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.895+0000 7fe9db7fe700 1 -- 172.21.15.32:0/1586567643 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9c4009640 con 0x7fe9dc05f3b0 2020-12-05T17:40:13.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.895+0000 7fe9db7fe700 1 -- 172.21.15.32:0/1586567643 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c40097a0 con 0x7fe9dc05f3b0 2020-12-05T17:40:13.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.895+0000 7fe9db7fe700 1 -- 172.21.15.32:0/1586567643 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9c4009b00 con 0x7fe9dc05f3b0 2020-12-05T17:40:13.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.895+0000 7fe9e1b9d700 1 -- 172.21.15.32:0/1586567643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9dc05f3b0 msgr2=0x7fe9dc05f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:13.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.895+0000 7fe9e1b9d700 1 --2- 172.21.15.32:0/1586567643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9dc05f3b0 0x7fe9dc05f790 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7fe9c4002f30 tx=0x7fe9c4006e80).stop 2020-12-05T17:40:13.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e1b9d700 1 -- 172.21.15.32:0/1586567643 shutdown_connections 2020-12-05T17:40:13.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e1b9d700 1 --2- 172.21.15.32:0/1586567643 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe9dc05feb0 0x7fe9dc0b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e1b9d700 1 --2- 172.21.15.32:0/1586567643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9dc05f3b0 0x7fe9dc05f790 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e1b9d700 1 --2- 172.21.15.32:0/1586567643 >> v2:172.21.15.61:3300/0 conn(0x7fe9dc062400 0x7fe9dc0b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e1b9d700 1 -- 172.21.15.32:0/1586567643 >> 172.21.15.32:0/1586567643 conn(0x7fe9dc0b8ee0 msgr2=0x7fe9dc0b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:13.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e1b9d700 1 -- 172.21.15.32:0/1586567643 shutdown_connections 2020-12-05T17:40:13.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e1b9d700 1 -- 172.21.15.32:0/1586567643 wait complete. 2020-12-05T17:40:13.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e1b9d700 1 Processor -- start 2020-12-05T17:40:13.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e1b9d700 1 -- start start 2020-12-05T17:40:13.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e1b9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe9dc05f3b0 0x7fe9dc063150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:13.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e1b9d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe9dc05feb0 0x7fe9dc145d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:13.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e1b9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9dc062400 0x7fe9dc151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:13.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e1b9d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc1525f0 con 0x7fe9dc062400 2020-12-05T17:40:13.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e1b9d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc065470 con 0x7fe9dc05f3b0 2020-12-05T17:40:13.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e1b9d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc145640 con 0x7fe9dc05feb0 2020-12-05T17:40:13.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e139c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9dc062400 0x7fe9dc151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:13.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e139c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9dc062400 0x7fe9dc151370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39412/0 (socket says 172.21.15.32:39412) 2020-12-05T17:40:13.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e139c700 1 -- 172.21.15.32:0/3879243435 learned_addr learned my addr 172.21.15.32:0/3879243435 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:13.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e0b9b700 1 --2- 172.21.15.32:0/3879243435 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe9dc05f3b0 0x7fe9dc063150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:13.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9dbfff700 1 --2- 172.21.15.32:0/3879243435 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe9dc05feb0 0x7fe9dc145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:13.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e139c700 1 -- 172.21.15.32:0/3879243435 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe9dc05feb0 msgr2=0x7fe9dc145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:13.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e139c700 1 --2- 172.21.15.32:0/3879243435 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe9dc05feb0 0x7fe9dc145d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e139c700 1 -- 172.21.15.32:0/3879243435 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe9dc05f3b0 msgr2=0x7fe9dc063150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:13.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e139c700 1 --2- 172.21.15.32:0/3879243435 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe9dc05f3b0 0x7fe9dc063150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:13.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e139c700 1 -- 172.21.15.32:0/3879243435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9dc1458e0 con 0x7fe9dc062400 2020-12-05T17:40:13.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.899+0000 7fe9e139c700 1 --2- 172.21.15.32:0/3879243435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9dc062400 0x7fe9dc151370 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 rx=0x7fe9d4006dd0 tx=0x7fe9d40091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:13.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.903+0000 7fe9d9ffb700 1 -- 172.21.15.32:0/3879243435 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9d400eee0 con 0x7fe9dc062400 2020-12-05T17:40:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.903+0000 7fe9d9ffb700 1 -- 172.21.15.32:0/3879243435 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d400cbb0 con 0x7fe9dc062400 2020-12-05T17:40:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.903+0000 7fe9d9ffb700 1 -- 172.21.15.32:0/3879243435 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9d40123f0 con 0x7fe9dc062400 2020-12-05T17:40:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.903+0000 7fe9e1b9d700 1 -- 172.21.15.32:0/3879243435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9dc063640 con 0x7fe9dc062400 2020-12-05T17:40:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.903+0000 7fe9e1b9d700 1 -- 172.21.15.32:0/3879243435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9dc145070 con 0x7fe9dc062400 2020-12-05T17:40:13.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.903+0000 7fe9e1b9d700 1 -- 172.21.15.32:0/3879243435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9dc057050 con 0x7fe9dc062400 2020-12-05T17:40:13.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.903+0000 7fe9d9ffb700 1 -- 172.21.15.32:0/3879243435 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe9d4022070 con 0x7fe9dc062400 2020-12-05T17:40:13.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.903+0000 7fe9d9ffb700 1 --2- 172.21.15.32:0/3879243435 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe9c0055330 0x7fe9c00577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:13.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.903+0000 7fe9d9ffb700 1 -- 172.21.15.32:0/3879243435 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe9d406e830 con 0x7fe9dc062400 2020-12-05T17:40:13.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.903+0000 7fe9e0b9b700 1 --2- 172.21.15.32:0/3879243435 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe9c0055330 0x7fe9c00577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:13.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.903+0000 7fe9e0b9b700 1 --2- 172.21.15.32:0/3879243435 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe9c0055330 0x7fe9c00577f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fe9c4019000 tx=0x7fe9c4015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:13.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:13.911+0000 7fe9d9ffb700 1 -- 172.21.15.32:0/3879243435 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe9d40449d0 con 0x7fe9dc062400 2020-12-05T17:40:14.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.103+0000 7fe9e1b9d700 1 -- 172.21.15.32:0/3879243435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9dc0b8c30 con 0x7fe9dc062400 2020-12-05T17:40:14.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.103+0000 7fe9d9ffb700 1 -- 172.21.15.32:0/3879243435 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+2142 (secure 0 0 0) 0x7fe9d4016070 con 0x7fe9dc062400 2020-12-05T17:40:14.109 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:40:14.109 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":52,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6548,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2853534360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2853534360},{"type":"v1","addr":"172.21.15.61:6833","nonce":2853534360}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":50}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:40:12.749385+0000","modified":"2020-12-05T17:40:13.752777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7563":{"gid":7563,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/2641840537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2641840537},{"type":"v1","addr":"172.21.15.32:6835","nonce":2641840537}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2020-12-05T17:40:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.107+0000 7fe9e1b9d700 1 -- 172.21.15.32:0/3879243435 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe9c0055330 msgr2=0x7fe9c00577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.107+0000 7fe9e1b9d700 1 --2- 172.21.15.32:0/3879243435 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe9c0055330 0x7fe9c00577f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fe9c4019000 tx=0x7fe9c4015000).stop 2020-12-05T17:40:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.107+0000 7fe9e1b9d700 1 -- 172.21.15.32:0/3879243435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9dc062400 msgr2=0x7fe9dc151370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:14.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.107+0000 7fe9e1b9d700 1 --2- 172.21.15.32:0/3879243435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9dc062400 0x7fe9dc151370 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 rx=0x7fe9d4006dd0 tx=0x7fe9d40091a0).stop 2020-12-05T17:40:14.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.107+0000 7fe9e1b9d700 1 -- 172.21.15.32:0/3879243435 shutdown_connections 2020-12-05T17:40:14.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.107+0000 7fe9e1b9d700 1 --2- 172.21.15.32:0/3879243435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9dc062400 0x7fe9dc151370 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:14.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.107+0000 7fe9e1b9d700 1 --2- 172.21.15.32:0/3879243435 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe9dc05f3b0 0x7fe9dc063150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:14.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.107+0000 7fe9e1b9d700 1 --2- 172.21.15.32:0/3879243435 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe9c0055330 0x7fe9c00577f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:14.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.107+0000 7fe9e1b9d700 1 --2- 172.21.15.32:0/3879243435 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe9dc05feb0 0x7fe9dc145d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:14.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.107+0000 7fe9e1b9d700 1 -- 172.21.15.32:0/3879243435 >> 172.21.15.32:0/3879243435 conn(0x7fe9dc0b8ee0 msgr2=0x7fe9dc0b6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:14.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.107+0000 7fe9e1b9d700 1 -- 172.21.15.32:0/3879243435 shutdown_connections 2020-12-05T17:40:14.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.107+0000 7fe9e1b9d700 1 -- 172.21.15.32:0/3879243435 wait complete. 2020-12-05T17:40:14.114 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 52 2020-12-05T17:40:14.123 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:40:14.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.227+0000 7f65f456f700 1 Processor -- start 2020-12-05T17:40:14.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.227+0000 7f65f456f700 1 -- start start 2020-12-05T17:40:14.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.227+0000 7f65f456f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f65ec0aead0 0x7f65ec0aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:14.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.227+0000 7f65f456f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f65ec0ad1b0 0x7f65ec1389f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:14.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.227+0000 7f65f456f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65ec13fb70 con 0x7f65ec138ee0 2020-12-05T17:40:14.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.227+0000 7f65f456f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65ec0af460 con 0x7f65ec0aead0 2020-12-05T17:40:14.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.227+0000 7f65f456f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65ec0af700 con 0x7f65ec0ad1b0 2020-12-05T17:40:14.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.227+0000 7f65f230b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f65ec0aead0 0x7f65ec0aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:14.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.227+0000 7f65f2b0c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f65ec138ee0 0x7f65ec142290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42064/0 (socket says 172.21.15.32:42064) 2020-12-05T17:40:14.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.227+0000 7f65f2b0c700 1 -- 172.21.15.32:0/3393728780 learned_addr learned my addr 172.21.15.32:0/3393728780 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:14.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.227+0000 7f65f1b0a700 1 --2- 172.21.15.32:0/3393728780 >> v2:172.21.15.61:3300/0 conn(0x7f65ec0ad1b0 0x7f65ec1389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:14.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.231+0000 7f65f230b700 1 -- 172.21.15.32:0/3393728780 >> v2:172.21.15.61:3300/0 conn(0x7f65ec0ad1b0 msgr2=0x7f65ec1389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:14.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.231+0000 7f65f230b700 1 --2- 172.21.15.32:0/3393728780 >> v2:172.21.15.61:3300/0 conn(0x7f65ec0ad1b0 0x7f65ec1389f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:14.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.231+0000 7f65f230b700 1 -- 172.21.15.32:0/3393728780 >> v1:172.21.15.32:6789/0 conn(0x7f65ec138ee0 legacy=0x7f65ec142290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:14.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.231+0000 7f65f1309700 1 -- 172.21.15.32:0/3393728780 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3093745557 0 0) 0x7f65ec13fb70 con 0x7f65ec138ee0 2020-12-05T17:40:14.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.231+0000 7f65f230b700 1 -- 172.21.15.32:0/3393728780 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ec13fdf0 con 0x7f65ec0aead0 2020-12-05T17:40:14.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.231+0000 7f65f230b700 1 --2- 172.21.15.32:0/3393728780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65ec0aead0 0x7f65ec0aeeb0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7f65e8002f30 tx=0x7f65e8006dc0).ready entity=mon.0 client_cookie=1fa4b542ee5e107a server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:14.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.231+0000 7f65f1309700 1 -- 172.21.15.32:0/3393728780 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65e80097b0 con 0x7f65ec0aead0 2020-12-05T17:40:14.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.231+0000 7f65f1309700 1 -- 172.21.15.32:0/3393728780 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e8009910 con 0x7f65ec0aead0 2020-12-05T17:40:14.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.231+0000 7f65f1309700 1 -- 172.21.15.32:0/3393728780 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65e8009c70 con 0x7f65ec0aead0 2020-12-05T17:40:14.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.231+0000 7f65f456f700 1 -- 172.21.15.32:0/3393728780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65ec0aead0 msgr2=0x7f65ec0aeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:14.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.231+0000 7f65f456f700 1 --2- 172.21.15.32:0/3393728780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65ec0aead0 0x7f65ec0aeeb0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7f65e8002f30 tx=0x7f65e8006dc0).stop 2020-12-05T17:40:14.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.231+0000 7f65f456f700 1 -- 172.21.15.32:0/3393728780 shutdown_connections 2020-12-05T17:40:14.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.231+0000 7f65f456f700 1 --2- 172.21.15.32:0/3393728780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65ec0aead0 0x7f65ec0aeeb0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:14.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.231+0000 7f65f456f700 1 --2- 172.21.15.32:0/3393728780 >> v2:172.21.15.61:3300/0 conn(0x7f65ec0ad1b0 0x7f65ec1389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:14.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.231+0000 7f65f456f700 1 -- 172.21.15.32:0/3393728780 >> 172.21.15.32:0/3393728780 conn(0x7f65ec0b5580 msgr2=0x7f65ec0b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:14.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.231+0000 7f65f456f700 1 -- 172.21.15.32:0/3393728780 shutdown_connections 2020-12-05T17:40:14.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.231+0000 7f65f456f700 1 -- 172.21.15.32:0/3393728780 wait complete. 2020-12-05T17:40:14.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.231+0000 7f65f456f700 1 Processor -- start 2020-12-05T17:40:14.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.231+0000 7f65f456f700 1 -- start start 2020-12-05T17:40:14.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.235+0000 7f65f456f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65ec0ad1b0 0x7f65ec142450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:14.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.235+0000 7f65f456f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f65ec0aead0 0x7f65ec151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:14.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.235+0000 7f65f456f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65ec138ee0 0x7f65ec151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:14.244 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.235+0000 7f65f456f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec154f00 con 0x7f65ec0ad1b0 2020-12-05T17:40:14.244 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.235+0000 7f65f456f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec139a60 con 0x7f65ec138ee0 2020-12-05T17:40:14.244 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.235+0000 7f65f456f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec139d00 con 0x7f65ec0aead0 2020-12-05T17:40:14.244 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.235+0000 7f65f230b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65ec0ad1b0 0x7f65ec142450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:14.244 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.235+0000 7f65f230b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65ec0ad1b0 0x7f65ec142450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39422/0 (socket says 172.21.15.32:39422) 2020-12-05T17:40:14.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.235+0000 7f65f230b700 1 -- 172.21.15.32:0/1596109483 learned_addr learned my addr 172.21.15.32:0/1596109483 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:14.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.235+0000 7f65f1b0a700 1 --2- 172.21.15.32:0/1596109483 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f65ec0aead0 0x7f65ec151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:14.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.235+0000 7f65f2b0c700 1 --2- 172.21.15.32:0/1596109483 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65ec138ee0 0x7f65ec151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:14.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.235+0000 7f65f230b700 1 -- 172.21.15.32:0/1596109483 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f65ec0aead0 msgr2=0x7f65ec151930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:14.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.235+0000 7f65f230b700 1 --2- 172.21.15.32:0/1596109483 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f65ec0aead0 0x7f65ec151930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:14.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.235+0000 7f65f230b700 1 -- 172.21.15.32:0/1596109483 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65ec138ee0 msgr2=0x7f65ec151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:14.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.235+0000 7f65f230b700 1 --2- 172.21.15.32:0/1596109483 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65ec138ee0 0x7f65ec151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:14.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.235+0000 7f65f230b700 1 -- 172.21.15.32:0/1596109483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ec154c60 con 0x7f65ec0ad1b0 2020-12-05T17:40:14.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.235+0000 7f65f230b700 1 --2- 172.21.15.32:0/1596109483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65ec0ad1b0 0x7f65ec142450 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0x7f65e8002f30 tx=0x7f65e8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:14.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.235+0000 7f65df7fe700 1 -- 172.21.15.32:0/1596109483 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65e8011a10 con 0x7f65ec0ad1b0 2020-12-05T17:40:14.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.235+0000 7f65df7fe700 1 -- 172.21.15.32:0/1596109483 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e8011b70 con 0x7f65ec0ad1b0 2020-12-05T17:40:14.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.235+0000 7f65df7fe700 1 -- 172.21.15.32:0/1596109483 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65e801d440 con 0x7f65ec0ad1b0 2020-12-05T17:40:14.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.235+0000 7f65f456f700 1 -- 172.21.15.32:0/1596109483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65ec142a70 con 0x7f65ec0ad1b0 2020-12-05T17:40:14.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.235+0000 7f65f456f700 1 -- 172.21.15.32:0/1596109483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65ec152310 con 0x7f65ec0ad1b0 2020-12-05T17:40:14.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.235+0000 7f65f456f700 1 -- 172.21.15.32:0/1596109483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65ec139840 con 0x7f65ec0ad1b0 2020-12-05T17:40:14.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.239+0000 7f65df7fe700 1 -- 172.21.15.32:0/1596109483 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f65e8017070 con 0x7f65ec0ad1b0 2020-12-05T17:40:14.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.239+0000 7f65df7fe700 1 --2- 172.21.15.32:0/1596109483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f65cc055330 0x7f65cc0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:14.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.239+0000 7f65df7fe700 1 -- 172.21.15.32:0/1596109483 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f65e806eb10 con 0x7f65ec0ad1b0 2020-12-05T17:40:14.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.239+0000 7f65f1b0a700 1 --2- 172.21.15.32:0/1596109483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f65cc055330 0x7f65cc0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:14.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.239+0000 7f65f1b0a700 1 --2- 172.21.15.32:0/1596109483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f65cc055330 0x7f65cc0577f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f65e000b000 tx=0x7f65e0006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:14.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.243+0000 7f65df7fe700 1 -- 172.21.15.32:0/1596109483 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f65e8044cb0 con 0x7f65ec0ad1b0 2020-12-05T17:40:14.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.387+0000 7f65f456f700 1 -- 172.21.15.32:0/1596109483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65ec057050 con 0x7f65ec0ad1b0 2020-12-05T17:40:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.391+0000 7f65df7fe700 1 -- 172.21.15.32:0/1596109483 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f65e80482d0 con 0x7f65ec0ad1b0 2020-12-05T17:40:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.391+0000 7f65f456f700 1 -- 172.21.15.32:0/1596109483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f65cc055330 msgr2=0x7f65cc0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.391+0000 7f65f456f700 1 --2- 172.21.15.32:0/1596109483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f65cc055330 0x7f65cc0577f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f65e000b000 tx=0x7f65e0006dc0).stop 2020-12-05T17:40:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.391+0000 7f65f456f700 1 -- 172.21.15.32:0/1596109483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65ec0ad1b0 msgr2=0x7f65ec142450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:14.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.391+0000 7f65f456f700 1 --2- 172.21.15.32:0/1596109483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65ec0ad1b0 0x7f65ec142450 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0x7f65e8002f30 tx=0x7f65e8015000).stop 2020-12-05T17:40:14.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.391+0000 7f65f456f700 1 -- 172.21.15.32:0/1596109483 shutdown_connections 2020-12-05T17:40:14.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.391+0000 7f65f456f700 1 --2- 172.21.15.32:0/1596109483 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65ec138ee0 0x7f65ec151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:14.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.391+0000 7f65f456f700 1 --2- 172.21.15.32:0/1596109483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65ec0ad1b0 0x7f65ec142450 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:14.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.391+0000 7f65f456f700 1 --2- 172.21.15.32:0/1596109483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f65cc055330 0x7f65cc0577f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:14.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.391+0000 7f65f456f700 1 --2- 172.21.15.32:0/1596109483 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f65ec0aead0 0x7f65ec151930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:14.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.391+0000 7f65f456f700 1 -- 172.21.15.32:0/1596109483 >> 172.21.15.32:0/1596109483 conn(0x7f65ec0b5580 msgr2=0x7f65ec0b39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:14.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.391+0000 7f65f456f700 1 -- 172.21.15.32:0/1596109483 shutdown_connections 2020-12-05T17:40:14.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.391+0000 7f65f456f700 1 -- 172.21.15.32:0/1596109483 wait complete. 2020-12-05T17:40:14.795 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:40:14.795 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":111,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:40:13.752899+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:40:10.513416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2020-12-05T17:40:11.768907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:40:14.796 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:40:14.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.919+0000 7f7a0ffff700 1 Processor -- start 2020-12-05T17:40:14.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.919+0000 7f7a0ffff700 1 -- start start 2020-12-05T17:40:14.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.923+0000 7f7a0ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a100b0240 0x7f7a10153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:14.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.923+0000 7f7a0ffff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a10080a90 con 0x7f7a100611b0 2020-12-05T17:40:14.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.923+0000 7f7a0ffff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a100aec40 con 0x7f7a100ae860 2020-12-05T17:40:14.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.923+0000 7f7a0ffff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a10152860 con 0x7f7a100b0240 2020-12-05T17:40:14.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.923+0000 7f7a06ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a100b0240 0x7f7a10153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:14.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.923+0000 7f7a06ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a100b0240 0x7f7a10153240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39430/0 (socket says 172.21.15.32:39430) 2020-12-05T17:40:14.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.923+0000 7f7a06ffd700 1 -- 172.21.15.32:0/2386127450 learned_addr learned my addr 172.21.15.32:0/2386127450 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:14.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.923+0000 7f7a0f7fe700 1 --1- 172.21.15.32:0/2386127450 >> v1:172.21.15.32:6789/0 conn(0x7f7a100611b0 0x7f7a1014c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42078/0 (socket says 172.21.15.32:42078) 2020-12-05T17:40:14.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.923+0000 7f7a06ffd700 1 -- 172.21.15.32:0/2386127450 >> v1:172.21.15.61:6789/0 conn(0x7f7a100ae860 legacy=0x7f7a1014f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:14.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.923+0000 7f7a06ffd700 1 -- 172.21.15.32:0/2386127450 >> v1:172.21.15.32:6789/0 conn(0x7f7a100611b0 legacy=0x7f7a1014c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:14.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.923+0000 7f7a06ffd700 1 -- 172.21.15.32:0/2386127450 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a10152b00 con 0x7f7a100b0240 2020-12-05T17:40:14.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.923+0000 7f7a06ffd700 1 --2- 172.21.15.32:0/2386127450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a100b0240 0x7f7a10153240 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 rx=0x7f79f8002f30 tx=0x7f79f8006e80).ready entity=mon.0 client_cookie=a95e69dd45796bcd server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:14.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.923+0000 7f7a0effd700 1 -- 172.21.15.32:0/2386127450 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79f80097f0 con 0x7f7a100b0240 2020-12-05T17:40:14.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.923+0000 7f7a0effd700 1 -- 172.21.15.32:0/2386127450 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f8009950 con 0x7f7a100b0240 2020-12-05T17:40:14.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.923+0000 7f7a0effd700 1 -- 172.21.15.32:0/2386127450 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79f8009ac0 con 0x7f7a100b0240 2020-12-05T17:40:14.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.923+0000 7f7a0ffff700 1 -- 172.21.15.32:0/2386127450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a100b0240 msgr2=0x7f7a10153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:14.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.923+0000 7f7a0ffff700 1 --2- 172.21.15.32:0/2386127450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a100b0240 0x7f7a10153240 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 rx=0x7f79f8002f30 tx=0x7f79f8006e80).stop 2020-12-05T17:40:14.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.923+0000 7f7a0ffff700 1 -- 172.21.15.32:0/2386127450 shutdown_connections 2020-12-05T17:40:14.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.923+0000 7f7a0ffff700 1 --2- 172.21.15.32:0/2386127450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a100b0240 0x7f7a10153240 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:14.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.923+0000 7f7a0ffff700 1 -- 172.21.15.32:0/2386127450 >> 172.21.15.32:0/2386127450 conn(0x7f7a100b5f90 msgr2=0x7f7a100b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:14.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.923+0000 7f7a0ffff700 1 -- 172.21.15.32:0/2386127450 shutdown_connections 2020-12-05T17:40:14.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.923+0000 7f7a0ffff700 1 -- 172.21.15.32:0/2386127450 wait complete. 2020-12-05T17:40:14.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.927+0000 7f7a0ffff700 1 Processor -- start 2020-12-05T17:40:14.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.927+0000 7f7a0ffff700 1 -- start start 2020-12-05T17:40:14.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.927+0000 7f7a0ffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a100611b0 0x7f7a1012b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:14.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.927+0000 7f7a0ffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a100ae860 0x7f7a10070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:14.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.927+0000 7f7a0ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a100b0240 0x7f7a10070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:14.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.927+0000 7f7a0ffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1012be70 con 0x7f7a100b0240 2020-12-05T17:40:14.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.927+0000 7f7a0ffff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a100b19d0 con 0x7f7a100611b0 2020-12-05T17:40:14.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.927+0000 7f7a0ffff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a10065250 con 0x7f7a100ae860 2020-12-05T17:40:14.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.927+0000 7f7a15524700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a100b0240 0x7f7a10070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:14.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.927+0000 7f7a15524700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a100b0240 0x7f7a10070e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39438/0 (socket says 172.21.15.32:39438) 2020-12-05T17:40:14.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.927+0000 7f7a0f7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a100611b0 0x7f7a1012b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:14.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.927+0000 7f7a15524700 1 -- 172.21.15.32:0/213910459 learned_addr learned my addr 172.21.15.32:0/213910459 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:14.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.927+0000 7f7a06ffd700 1 --2- 172.21.15.32:0/213910459 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a100ae860 0x7f7a10070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:14.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.931+0000 7f7a15524700 1 -- 172.21.15.32:0/213910459 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a100ae860 msgr2=0x7f7a10070920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:14.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.931+0000 7f7a15524700 1 --2- 172.21.15.32:0/213910459 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a100ae860 0x7f7a10070920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:14.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.931+0000 7f7a15524700 1 -- 172.21.15.32:0/213910459 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a100611b0 msgr2=0x7f7a1012b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:14.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.931+0000 7f7a15524700 1 --2- 172.21.15.32:0/213910459 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a100611b0 0x7f7a1012b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:14.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.931+0000 7f7a15524700 1 -- 172.21.15.32:0/213910459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a100b1530 con 0x7f7a100b0240 2020-12-05T17:40:14.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.931+0000 7f7a15524700 1 --2- 172.21.15.32:0/213910459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a100b0240 0x7f7a10070e10 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7f7a080022e0 tx=0x7f7a0800cb40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:14.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.931+0000 7f7a0cff9700 1 -- 172.21.15.32:0/213910459 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a08010510 con 0x7f7a100b0240 2020-12-05T17:40:14.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.931+0000 7f7a0cff9700 1 -- 172.21.15.32:0/213910459 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a08010670 con 0x7f7a100b0240 2020-12-05T17:40:14.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.931+0000 7f7a0ffff700 1 -- 172.21.15.32:0/213910459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a100b2780 con 0x7f7a100b0240 2020-12-05T17:40:14.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.931+0000 7f7a0cff9700 1 -- 172.21.15.32:0/213910459 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a080107e0 con 0x7f7a100b0240 2020-12-05T17:40:14.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.931+0000 7f7a0ffff700 1 -- 172.21.15.32:0/213910459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a101502c0 con 0x7f7a100b0240 2020-12-05T17:40:14.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.931+0000 7f7a0ffff700 1 -- 172.21.15.32:0/213910459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a100b1830 con 0x7f7a100b0240 2020-12-05T17:40:14.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.931+0000 7f7a0cff9700 1 -- 172.21.15.32:0/213910459 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7a08010070 con 0x7f7a100b0240 2020-12-05T17:40:14.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.931+0000 7f7a0cff9700 1 --2- 172.21.15.32:0/213910459 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79f40553b0 0x7f79f4057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:14.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.935+0000 7f7a0f7fe700 1 --2- 172.21.15.32:0/213910459 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79f40553b0 0x7f79f4057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:14.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.935+0000 7f7a0cff9700 1 -- 172.21.15.32:0/213910459 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7a0806dd40 con 0x7f7a100b0240 2020-12-05T17:40:14.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.935+0000 7f7a0f7fe700 1 --2- 172.21.15.32:0/213910459 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79f40553b0 0x7f79f4057870 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f7a0000d000 tx=0x7f7a00007b10).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:14.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:14.939+0000 7f7a0cff9700 1 -- 172.21.15.32:0/213910459 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7a100b1830 con 0x7f7a100b0240 2020-12-05T17:40:15.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.135+0000 7f7a0ffff700 1 -- 172.21.15.32:0/213910459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f7a100b1830 con 0x7f7a100b0240 2020-12-05T17:40:15.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.143+0000 7f7a0cff9700 1 -- 172.21.15.32:0/213910459 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v35) v1 ==== 184+0+0 (secure 0 0 0) 0x7f7a100b1830 con 0x7f7a100b0240 2020-12-05T17:40:15.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.147+0000 7f7a0ffff700 1 -- 172.21.15.32:0/213910459 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79f40553b0 msgr2=0x7f79f4057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:15.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.147+0000 7f7a0ffff700 1 --2- 172.21.15.32:0/213910459 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79f40553b0 0x7f79f4057870 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f7a0000d000 tx=0x7f7a00007b10).stop 2020-12-05T17:40:15.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.147+0000 7f7a0ffff700 1 -- 172.21.15.32:0/213910459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a100b0240 msgr2=0x7f7a10070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:15.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.147+0000 7f7a0ffff700 1 --2- 172.21.15.32:0/213910459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a100b0240 0x7f7a10070e10 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7f7a080022e0 tx=0x7f7a0800cb40).stop 2020-12-05T17:40:15.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.147+0000 7f7a0ffff700 1 -- 172.21.15.32:0/213910459 shutdown_connections 2020-12-05T17:40:15.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.147+0000 7f7a0ffff700 1 --2- 172.21.15.32:0/213910459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a100b0240 0x7f7a10070e10 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:15.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.147+0000 7f7a0ffff700 1 --2- 172.21.15.32:0/213910459 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a100611b0 0x7f7a1012b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:15.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.147+0000 7f7a0ffff700 1 --2- 172.21.15.32:0/213910459 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79f40553b0 0x7f79f4057870 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:15.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.147+0000 7f7a0ffff700 1 --2- 172.21.15.32:0/213910459 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a100ae860 0x7f7a10070920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:15.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.147+0000 7f7a0ffff700 1 -- 172.21.15.32:0/213910459 >> 172.21.15.32:0/213910459 conn(0x7f7a100b5f90 msgr2=0x7f7a1012cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:15.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.147+0000 7f7a0ffff700 1 -- 172.21.15.32:0/213910459 shutdown_connections 2020-12-05T17:40:15.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.147+0000 7f7a0ffff700 1 -- 172.21.15.32:0/213910459 wait complete. 2020-12-05T17:40:15.156 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2020-12-05T17:40:15.164 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:40:15.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.279+0000 7f409a367700 1 Processor -- start 2020-12-05T17:40:15.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.279+0000 7f409a367700 1 -- start start 2020-12-05T17:40:15.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.279+0000 7f409a367700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40940b0240 0x7f4094153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:15.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.279+0000 7f409a367700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f40940ae860 0x7f409414f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:15.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.279+0000 7f409a367700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4094080a90 con 0x7f40940611b0 2020-12-05T17:40:15.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.279+0000 7f409a367700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40940aec40 con 0x7f40940b0240 2020-12-05T17:40:15.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.279+0000 7f409a367700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4094152860 con 0x7f40940ae860 2020-12-05T17:40:15.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.279+0000 7f4099b66700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f40940611b0 0x7f409414c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42092/0 (socket says 172.21.15.32:42092) 2020-12-05T17:40:15.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.279+0000 7f4099b66700 1 -- 172.21.15.32:0/1394156833 learned_addr learned my addr 172.21.15.32:0/1394156833 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:15.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.283+0000 7f4099365700 1 --2- 172.21.15.32:0/1394156833 >> v2:172.21.15.32:3300/0 conn(0x7f40940b0240 0x7f4094153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:15.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.283+0000 7f409b5c9700 1 --2- 172.21.15.32:0/1394156833 >> v2:172.21.15.61:3300/0 conn(0x7f40940ae860 0x7f409414f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:15.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.283+0000 7f4098b64700 1 -- 172.21.15.32:0/1394156833 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701252422 0 0) 0x7f4094080a90 con 0x7f40940611b0 2020-12-05T17:40:15.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.283+0000 7f4098b64700 1 -- 172.21.15.32:0/1394156833 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4080003230 con 0x7f40940611b0 2020-12-05T17:40:15.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.283+0000 7f4099365700 1 -- 172.21.15.32:0/1394156833 >> v2:172.21.15.61:3300/0 conn(0x7f40940ae860 msgr2=0x7f409414f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:15.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.283+0000 7f4099365700 1 --2- 172.21.15.32:0/1394156833 >> v2:172.21.15.61:3300/0 conn(0x7f40940ae860 0x7f409414f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.283+0000 7f4099365700 1 -- 172.21.15.32:0/1394156833 >> v1:172.21.15.32:6789/0 conn(0x7f40940611b0 legacy=0x7f409414c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.283+0000 7f4099365700 1 -- 172.21.15.32:0/1394156833 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4094152b00 con 0x7f40940b0240 2020-12-05T17:40:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.283+0000 7f4099365700 1 --2- 172.21.15.32:0/1394156833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40940b0240 0x7f4094153240 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7f4084002f30 tx=0x7f4084006dc0).ready entity=mon.0 client_cookie=684cb0fce4be535d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.283+0000 7f4098b64700 1 -- 172.21.15.32:0/1394156833 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40840096f0 con 0x7f40940b0240 2020-12-05T17:40:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.283+0000 7f4098b64700 1 -- 172.21.15.32:0/1394156833 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4084009850 con 0x7f40940b0240 2020-12-05T17:40:15.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.283+0000 7f4098b64700 1 -- 172.21.15.32:0/1394156833 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4084009bb0 con 0x7f40940b0240 2020-12-05T17:40:15.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.283+0000 7f409a367700 1 -- 172.21.15.32:0/1394156833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40940b0240 msgr2=0x7f4094153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:15.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.283+0000 7f409a367700 1 --2- 172.21.15.32:0/1394156833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40940b0240 0x7f4094153240 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7f4084002f30 tx=0x7f4084006dc0).stop 2020-12-05T17:40:15.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.283+0000 7f409a367700 1 -- 172.21.15.32:0/1394156833 shutdown_connections 2020-12-05T17:40:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.283+0000 7f409a367700 1 --2- 172.21.15.32:0/1394156833 >> v2:172.21.15.61:3300/0 conn(0x7f40940ae860 0x7f409414f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.283+0000 7f409a367700 1 --2- 172.21.15.32:0/1394156833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40940b0240 0x7f4094153240 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.283+0000 7f409a367700 1 -- 172.21.15.32:0/1394156833 >> 172.21.15.32:0/1394156833 conn(0x7f40940b5f90 msgr2=0x7f40940b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.283+0000 7f409a367700 1 -- 172.21.15.32:0/1394156833 shutdown_connections 2020-12-05T17:40:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.283+0000 7f409a367700 1 -- 172.21.15.32:0/1394156833 wait complete. 2020-12-05T17:40:15.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f409a367700 1 Processor -- start 2020-12-05T17:40:15.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f409a367700 1 -- start start 2020-12-05T17:40:15.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f409a367700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40940611b0 0x7f4094159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:15.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f409a367700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40940ae860 0x7f409413a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:15.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f409a367700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40940b0240 0x7f40940b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:15.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f409a367700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4094150330 con 0x7f40940ae860 2020-12-05T17:40:15.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f409a367700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4094158820 con 0x7f40940b0240 2020-12-05T17:40:15.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f409a367700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40940652f0 con 0x7f40940611b0 2020-12-05T17:40:15.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f4099365700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40940ae860 0x7f409413a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:15.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f4099365700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40940ae860 0x7f409413a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39454/0 (socket says 172.21.15.32:39454) 2020-12-05T17:40:15.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f4099365700 1 -- 172.21.15.32:0/930545277 learned_addr learned my addr 172.21.15.32:0/930545277 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:15.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f4099b66700 1 --2- 172.21.15.32:0/930545277 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40940611b0 0x7f4094159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:15.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f409b5c9700 1 --2- 172.21.15.32:0/930545277 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40940b0240 0x7f40940b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:15.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f4099365700 1 -- 172.21.15.32:0/930545277 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40940611b0 msgr2=0x7f4094159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:15.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f4099365700 1 --2- 172.21.15.32:0/930545277 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40940611b0 0x7f4094159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:15.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f4099365700 1 -- 172.21.15.32:0/930545277 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40940b0240 msgr2=0x7f40940b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:15.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f4099365700 1 --2- 172.21.15.32:0/930545277 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40940b0240 0x7f40940b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:15.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f4099365700 1 -- 172.21.15.32:0/930545277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409413a9c0 con 0x7f40940ae860 2020-12-05T17:40:15.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f4099365700 1 --2- 172.21.15.32:0/930545277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40940ae860 0x7f409413a4d0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7f4084002f30 tx=0x7f4084015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:15.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f408a7fc700 1 -- 172.21.15.32:0/930545277 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4084011b00 con 0x7f40940ae860 2020-12-05T17:40:15.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f408a7fc700 1 -- 172.21.15.32:0/930545277 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4084011c60 con 0x7f40940ae860 2020-12-05T17:40:15.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f409a367700 1 -- 172.21.15.32:0/930545277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40940b42b0 con 0x7f40940ae860 2020-12-05T17:40:15.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f408a7fc700 1 -- 172.21.15.32:0/930545277 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f408401d450 con 0x7f40940ae860 2020-12-05T17:40:15.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.287+0000 7f409a367700 1 -- 172.21.15.32:0/930545277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40941584c0 con 0x7f40940ae860 2020-12-05T17:40:15.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.291+0000 7f409a367700 1 -- 172.21.15.32:0/930545277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40940654b0 con 0x7f40940ae860 2020-12-05T17:40:15.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.291+0000 7f408a7fc700 1 -- 172.21.15.32:0/930545277 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4084024030 con 0x7f40940ae860 2020-12-05T17:40:15.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.291+0000 7f408a7fc700 1 --2- 172.21.15.32:0/930545277 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f407c055320 0x7f407c0577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:15.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.291+0000 7f408a7fc700 1 -- 172.21.15.32:0/930545277 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f408406f180 con 0x7f40940ae860 2020-12-05T17:40:15.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.291+0000 7f4099b66700 1 --2- 172.21.15.32:0/930545277 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f407c055320 0x7f407c0577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:15.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.291+0000 7f4099b66700 1 --2- 172.21.15.32:0/930545277 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f407c055320 0x7f407c0577e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f409000d000 tx=0x7f4090009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:15.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.299+0000 7f408a7fc700 1 -- 172.21.15.32:0/930545277 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4084045320 con 0x7f40940ae860 2020-12-05T17:40:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.491+0000 7f409a367700 1 -- 172.21.15.32:0/930545277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f4094057050 con 0x7f40940ae860 2020-12-05T17:40:15.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.499+0000 7f408a7fc700 1 -- 172.21.15.32:0/930545277 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v36) v1 ==== 184+0+0 (secure 0 0 0) 0x7f4084022020 con 0x7f40940ae860 2020-12-05T17:40:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.503+0000 7f409a367700 1 -- 172.21.15.32:0/930545277 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f407c055320 msgr2=0x7f407c0577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.503+0000 7f409a367700 1 --2- 172.21.15.32:0/930545277 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f407c055320 0x7f407c0577e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f409000d000 tx=0x7f4090009040).stop 2020-12-05T17:40:15.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.503+0000 7f409a367700 1 -- 172.21.15.32:0/930545277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40940ae860 msgr2=0x7f409413a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:15.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.503+0000 7f409a367700 1 --2- 172.21.15.32:0/930545277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40940ae860 0x7f409413a4d0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7f4084002f30 tx=0x7f4084015000).stop 2020-12-05T17:40:15.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.503+0000 7f409a367700 1 -- 172.21.15.32:0/930545277 shutdown_connections 2020-12-05T17:40:15.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.503+0000 7f409a367700 1 --2- 172.21.15.32:0/930545277 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40940b0240 0x7f40940b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:15.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.503+0000 7f409a367700 1 --2- 172.21.15.32:0/930545277 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f407c055320 0x7f407c0577e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:15.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.503+0000 7f409a367700 1 --2- 172.21.15.32:0/930545277 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40940611b0 0x7f4094159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:15.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.503+0000 7f409a367700 1 --2- 172.21.15.32:0/930545277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40940ae860 0x7f409413a4d0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:15.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.503+0000 7f409a367700 1 -- 172.21.15.32:0/930545277 >> 172.21.15.32:0/930545277 conn(0x7f40940b5f90 msgr2=0x7f40940aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:15.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.503+0000 7f409a367700 1 -- 172.21.15.32:0/930545277 shutdown_connections 2020-12-05T17:40:15.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.503+0000 7f409a367700 1 -- 172.21.15.32:0/930545277 wait complete. 2020-12-05T17:40:15.510 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2020-12-05T17:40:15.521 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:40:15.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.647+0000 7f9d7a572700 1 Processor -- start 2020-12-05T17:40:15.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.647+0000 7f9d7a572700 1 -- start start 2020-12-05T17:40:15.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.647+0000 7f9d7a572700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9d740b0240 0x7f9d74153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:15.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.647+0000 7f9d7a572700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9d740ae860 0x7f9d7414f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:15.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.647+0000 7f9d7a572700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d74080a90 con 0x7f9d740611b0 2020-12-05T17:40:15.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.647+0000 7f9d7a572700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d740aecb0 con 0x7f9d740ae860 2020-12-05T17:40:15.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.647+0000 7f9d7a572700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d74152860 con 0x7f9d740b0240 2020-12-05T17:40:15.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.647+0000 7f9d7b7d4700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9d740ae860 0x7f9d7414f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:15.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.647+0000 7f9d79570700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9d740b0240 0x7f9d74153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:15.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.647+0000 7f9d7b7d4700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9d740ae860 0x7f9d7414f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:59354/0 (socket says 172.21.15.32:59354) 2020-12-05T17:40:15.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.647+0000 7f9d7b7d4700 1 -- 172.21.15.32:0/2262521867 learned_addr learned my addr 172.21.15.32:0/2262521867 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:15.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.647+0000 7f9d79570700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9d740b0240 0x7f9d74153240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:36250/0 (socket says 172.21.15.32:36250) 2020-12-05T17:40:15.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.647+0000 7f9d7b7d4700 1 -- 172.21.15.32:0/2262521867 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9d740b0240 msgr2=0x7f9d74153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:15.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.647+0000 7f9d7b7d4700 1 --2- 172.21.15.32:0/2262521867 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9d740b0240 0x7f9d74153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:15.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.647+0000 7f9d7b7d4700 1 -- 172.21.15.32:0/2262521867 >> v1:172.21.15.61:6789/0 conn(0x7f9d740611b0 legacy=0x7f9d7414c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:15.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.647+0000 7f9d78d6f700 1 -- 172.21.15.32:0/2262521867 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2360738693 0 0) 0x7f9d74080a90 con 0x7f9d740611b0 2020-12-05T17:40:15.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.647+0000 7f9d7b7d4700 1 -- 172.21.15.32:0/2262521867 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d74152b00 con 0x7f9d740ae860 2020-12-05T17:40:15.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.647+0000 7f9d7b7d4700 1 --2- 172.21.15.32:0/2262521867 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9d740ae860 0x7f9d7414f0d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f9d64002f30 tx=0x7f9d64006e80).ready entity=mon.1 client_cookie=ee459256fa0298b1 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:15.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.647+0000 7f9d78d6f700 1 -- 172.21.15.32:0/2262521867 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d640097f0 con 0x7f9d740ae860 2020-12-05T17:40:15.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.647+0000 7f9d78d6f700 1 -- 172.21.15.32:0/2262521867 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d64009950 con 0x7f9d740ae860 2020-12-05T17:40:15.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.647+0000 7f9d78d6f700 1 -- 172.21.15.32:0/2262521867 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d64009cb0 con 0x7f9d740ae860 2020-12-05T17:40:15.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.651+0000 7f9d7a572700 1 -- 172.21.15.32:0/2262521867 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9d740ae860 msgr2=0x7f9d7414f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:15.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.651+0000 7f9d7a572700 1 --2- 172.21.15.32:0/2262521867 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9d740ae860 0x7f9d7414f0d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f9d64002f30 tx=0x7f9d64006e80).stop 2020-12-05T17:40:15.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.651+0000 7f9d7a572700 1 -- 172.21.15.32:0/2262521867 shutdown_connections 2020-12-05T17:40:15.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.651+0000 7f9d7a572700 1 --2- 172.21.15.32:0/2262521867 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9d740ae860 0x7f9d7414f0d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:15.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.651+0000 7f9d7a572700 1 --2- 172.21.15.32:0/2262521867 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9d740b0240 0x7f9d74153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:15.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.651+0000 7f9d7a572700 1 -- 172.21.15.32:0/2262521867 >> 172.21.15.32:0/2262521867 conn(0x7f9d740b5f90 msgr2=0x7f9d740b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:15.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.651+0000 7f9d7a572700 1 -- 172.21.15.32:0/2262521867 shutdown_connections 2020-12-05T17:40:15.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.651+0000 7f9d7a572700 1 -- 172.21.15.32:0/2262521867 wait complete. 2020-12-05T17:40:15.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.651+0000 7f9d7a572700 1 Processor -- start 2020-12-05T17:40:15.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.651+0000 7f9d7a572700 1 -- start start 2020-12-05T17:40:15.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d7a572700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d740611b0 0x7f9d741560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:15.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d7a572700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9d740ae860 0x7f9d74153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:15.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d7a572700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9d740b0240 0x7f9d74155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:15.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d7a572700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d741565c0 con 0x7f9d740611b0 2020-12-05T17:40:15.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d7a572700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d74154c30 con 0x7f9d740b0240 2020-12-05T17:40:15.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d7a572700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d740672d0 con 0x7f9d740ae860 2020-12-05T17:40:15.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d79d71700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d740611b0 0x7f9d741560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:15.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d79d71700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d740611b0 0x7f9d741560d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39466/0 (socket says 172.21.15.32:39466) 2020-12-05T17:40:15.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d79d71700 1 -- 172.21.15.32:0/4195674711 learned_addr learned my addr 172.21.15.32:0/4195674711 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:15.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d7b7d4700 1 --2- 172.21.15.32:0/4195674711 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9d740b0240 0x7f9d74155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:15.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d79570700 1 --2- 172.21.15.32:0/4195674711 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9d740ae860 0x7f9d74153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:15.675 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d79d71700 1 -- 172.21.15.32:0/4195674711 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9d740ae860 msgr2=0x7f9d74153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:15.675 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d79d71700 1 --2- 172.21.15.32:0/4195674711 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9d740ae860 0x7f9d74153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:15.675 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d79d71700 1 -- 172.21.15.32:0/4195674711 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9d740b0240 msgr2=0x7f9d74155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:15.676 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d79d71700 1 --2- 172.21.15.32:0/4195674711 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9d740b0240 0x7f9d74155700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:15.676 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d79d71700 1 -- 172.21.15.32:0/4195674711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d74066ba0 con 0x7f9d740611b0 2020-12-05T17:40:15.676 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d79570700 1 --2- 172.21.15.32:0/4195674711 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9d740ae860 0x7f9d74153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:40:15.677 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d7b7d4700 1 --2- 172.21.15.32:0/4195674711 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9d740b0240 0x7f9d74155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:40:15.677 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d79d71700 1 --2- 172.21.15.32:0/4195674711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d740611b0 0x7f9d741560d0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7f9d6c000f00 tx=0x7f9d6c007e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:15.678 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d727fc700 1 -- 172.21.15.32:0/4195674711 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d6c00a7b0 con 0x7f9d740611b0 2020-12-05T17:40:15.678 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d7a572700 1 -- 172.21.15.32:0/4195674711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d74067090 con 0x7f9d740611b0 2020-12-05T17:40:15.679 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d7a572700 1 -- 172.21.15.32:0/4195674711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d74155bf0 con 0x7f9d740611b0 2020-12-05T17:40:15.679 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d727fc700 1 -- 172.21.15.32:0/4195674711 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6c00a910 con 0x7f9d740611b0 2020-12-05T17:40:15.679 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.655+0000 7f9d727fc700 1 -- 172.21.15.32:0/4195674711 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d6c00ac70 con 0x7f9d740611b0 2020-12-05T17:40:15.679 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.659+0000 7f9d727fc700 1 -- 172.21.15.32:0/4195674711 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9d6c010070 con 0x7f9d740611b0 2020-12-05T17:40:15.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.659+0000 7f9d7a572700 1 -- 172.21.15.32:0/4195674711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d74155f00 con 0x7f9d740611b0 2020-12-05T17:40:15.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.659+0000 7f9d727fc700 1 --2- 172.21.15.32:0/4195674711 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9d540554d0 0x7f9d54057990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:15.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.659+0000 7f9d727fc700 1 -- 172.21.15.32:0/4195674711 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9d6c06d670 con 0x7f9d740611b0 2020-12-05T17:40:15.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.659+0000 7f9d79570700 1 --2- 172.21.15.32:0/4195674711 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9d540554d0 0x7f9d54057990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:15.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.659+0000 7f9d79570700 1 --2- 172.21.15.32:0/4195674711 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9d540554d0 0x7f9d54057990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f9d68008000 tx=0x7f9d68008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:15.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.667+0000 7f9d727fc700 1 -- 172.21.15.32:0/4195674711 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9d6c043710 con 0x7f9d740611b0 2020-12-05T17:40:15.819 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:40:15.821 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:40:15.824 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:40:15.826 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:40:15.828 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:40:15.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.859+0000 7f9d7a572700 1 -- 172.21.15.32:0/4195674711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f9d74066d30 con 0x7f9d740611b0 2020-12-05T17:40:15.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.863+0000 7f9d727fc700 1 -- 172.21.15.32:0/4195674711 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v37) v1 ==== 184+0+0 (secure 0 0 0) 0x7f9d6c046d30 con 0x7f9d740611b0 2020-12-05T17:40:15.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.863+0000 7f9d7a572700 1 -- 172.21.15.32:0/4195674711 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9d540554d0 msgr2=0x7f9d54057990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:15.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.863+0000 7f9d7a572700 1 --2- 172.21.15.32:0/4195674711 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9d540554d0 0x7f9d54057990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f9d68008000 tx=0x7f9d68008040).stop 2020-12-05T17:40:15.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.863+0000 7f9d7a572700 1 -- 172.21.15.32:0/4195674711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d740611b0 msgr2=0x7f9d741560d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:15.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.863+0000 7f9d7a572700 1 --2- 172.21.15.32:0/4195674711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d740611b0 0x7f9d741560d0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7f9d6c000f00 tx=0x7f9d6c007e40).stop 2020-12-05T17:40:15.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.863+0000 7f9d7a572700 1 -- 172.21.15.32:0/4195674711 shutdown_connections 2020-12-05T17:40:15.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.863+0000 7f9d7a572700 1 --2- 172.21.15.32:0/4195674711 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9d740b0240 0x7f9d74155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:15.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.863+0000 7f9d7a572700 1 --2- 172.21.15.32:0/4195674711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d740611b0 0x7f9d741560d0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:15.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.863+0000 7f9d7a572700 1 --2- 172.21.15.32:0/4195674711 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9d540554d0 0x7f9d54057990 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:15.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.863+0000 7f9d7a572700 1 --2- 172.21.15.32:0/4195674711 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9d740ae860 0x7f9d74153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:15.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.863+0000 7f9d7a572700 1 -- 172.21.15.32:0/4195674711 >> 172.21.15.32:0/4195674711 conn(0x7f9d740b5f90 msgr2=0x7f9d740adda0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:15.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.863+0000 7f9d7a572700 1 -- 172.21.15.32:0/4195674711 shutdown_connections 2020-12-05T17:40:15.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:15.863+0000 7f9d7a572700 1 -- 172.21.15.32:0/4195674711 wait complete. 2020-12-05T17:40:15.872 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.2 2020-12-05T17:40:15.881 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:40:16.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.035+0000 7fbcd605e700 1 Processor -- start 2020-12-05T17:40:16.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.035+0000 7fbcd605e700 1 -- start start 2020-12-05T17:40:16.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.035+0000 7fbcd605e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbcd0062400 0x7fbcd005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:16.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.035+0000 7fbcd605e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fbcd005feb0 0x7fbcd00b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:16.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.035+0000 7fbcd605e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd0142450 con 0x7fbcd005f3b0 2020-12-05T17:40:16.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.035+0000 7fbcd605e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd00b42f0 con 0x7fbcd005feb0 2020-12-05T17:40:16.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.035+0000 7fbcd605e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd0061780 con 0x7fbcd0062400 2020-12-05T17:40:16.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.035+0000 7fbcd485b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbcd0062400 0x7fbcd005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:16.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.035+0000 7fbcd585d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fbcd005feb0 0x7fbcd00b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:16.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.035+0000 7fbcd485b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbcd0062400 0x7fbcd005f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:36264/0 (socket says 172.21.15.32:36264) 2020-12-05T17:40:16.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.035+0000 7fbcd505c700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fbcd005f3b0 0x7fbcd00b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:54760/0 (socket says 172.21.15.32:54760) 2020-12-05T17:40:16.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.035+0000 7fbcd585d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fbcd005feb0 0x7fbcd00b3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:59368/0 (socket says 172.21.15.32:59368) 2020-12-05T17:40:16.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.035+0000 7fbcd485b700 1 -- 172.21.15.32:0/61341455 learned_addr learned my addr 172.21.15.32:0/61341455 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:16.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.035+0000 7fbcd585d700 1 -- 172.21.15.32:0/61341455 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbcd0062400 msgr2=0x7fbcd005f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:16.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.035+0000 7fbcd585d700 1 --2- 172.21.15.32:0/61341455 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbcd0062400 0x7fbcd005f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:16.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.035+0000 7fbcd585d700 1 -- 172.21.15.32:0/61341455 >> v1:172.21.15.61:6789/0 conn(0x7fbcd005f3b0 legacy=0x7fbcd00b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:16.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.035+0000 7fbcd585d700 1 -- 172.21.15.32:0/61341455 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd0061a20 con 0x7fbcd005feb0 2020-12-05T17:40:16.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.035+0000 7fbcd485b700 1 --2- 172.21.15.32:0/61341455 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbcd0062400 0x7fbcd005f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:40:16.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.039+0000 7fbcd585d700 1 --2- 172.21.15.32:0/61341455 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbcd005feb0 0x7fbcd00b3e00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7fbcc80032b0 tx=0x7fbcc80091a0).ready entity=mon.1 client_cookie=85fccf3e1922f6e7 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:16.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.039+0000 7fbcc3fff700 1 -- 172.21.15.32:0/61341455 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcc800b9a0 con 0x7fbcd005feb0 2020-12-05T17:40:16.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.039+0000 7fbcc3fff700 1 -- 172.21.15.32:0/61341455 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc800bb00 con 0x7fbcd005feb0 2020-12-05T17:40:16.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.039+0000 7fbcc3fff700 1 -- 172.21.15.32:0/61341455 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcc800bc70 con 0x7fbcd005feb0 2020-12-05T17:40:16.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.039+0000 7fbcd605e700 1 -- 172.21.15.32:0/61341455 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbcd005feb0 msgr2=0x7fbcd00b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:16.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.039+0000 7fbcd605e700 1 --2- 172.21.15.32:0/61341455 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbcd005feb0 0x7fbcd00b3e00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7fbcc80032b0 tx=0x7fbcc80091a0).stop 2020-12-05T17:40:16.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.039+0000 7fbcd605e700 1 -- 172.21.15.32:0/61341455 shutdown_connections 2020-12-05T17:40:16.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.039+0000 7fbcd605e700 1 --2- 172.21.15.32:0/61341455 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbcd005feb0 0x7fbcd00b3e00 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:16.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.039+0000 7fbcd605e700 1 --2- 172.21.15.32:0/61341455 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbcd0062400 0x7fbcd005f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:16.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.039+0000 7fbcd605e700 1 -- 172.21.15.32:0/61341455 >> 172.21.15.32:0/61341455 conn(0x7fbcd00b8ee0 msgr2=0x7fbcd00b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:16.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.039+0000 7fbcd605e700 1 -- 172.21.15.32:0/61341455 shutdown_connections 2020-12-05T17:40:16.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.039+0000 7fbcd605e700 1 -- 172.21.15.32:0/61341455 wait complete. 2020-12-05T17:40:16.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.039+0000 7fbcd605e700 1 Processor -- start 2020-12-05T17:40:16.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.039+0000 7fbcd605e700 1 -- start start 2020-12-05T17:40:16.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcd605e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd005f3b0 0x7fbcd00a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:16.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcd605e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbcd005feb0 0x7fbcd0065c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:16.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcd605e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbcd0062400 0x7fbcd00af070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:16.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcd605e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd00a3e10 con 0x7fbcd005f3b0 2020-12-05T17:40:16.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcd605e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd0064500 con 0x7fbcd0062400 2020-12-05T17:40:16.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcd605e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd0066140 con 0x7fbcd005feb0 2020-12-05T17:40:16.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcd505c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd005f3b0 0x7fbcd00a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:16.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcd505c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd005f3b0 0x7fbcd00a3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39478/0 (socket says 172.21.15.32:39478) 2020-12-05T17:40:16.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcd505c700 1 -- 172.21.15.32:0/2391398736 learned_addr learned my addr 172.21.15.32:0/2391398736 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:16.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcd585d700 1 --2- 172.21.15.32:0/2391398736 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbcd0062400 0x7fbcd00af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:16.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcd485b700 1 --2- 172.21.15.32:0/2391398736 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbcd005feb0 0x7fbcd0065c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:16.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcd505c700 1 -- 172.21.15.32:0/2391398736 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbcd005feb0 msgr2=0x7fbcd0065c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:16.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcd505c700 1 --2- 172.21.15.32:0/2391398736 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbcd005feb0 0x7fbcd0065c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:16.054 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcd505c700 1 -- 172.21.15.32:0/2391398736 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbcd0062400 msgr2=0x7fbcd00af070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:16.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcd505c700 1 --2- 172.21.15.32:0/2391398736 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbcd0062400 0x7fbcd00af070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:16.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcd505c700 1 -- 172.21.15.32:0/2391398736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd0066380 con 0x7fbcd005f3b0 2020-12-05T17:40:16.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcd485b700 1 --2- 172.21.15.32:0/2391398736 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbcd005feb0 0x7fbcd0065c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:40:16.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcd505c700 1 --2- 172.21.15.32:0/2391398736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd005f3b0 0x7fbcd00a3920 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7fbcd005c5c0 tx=0x7fbccc007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:16.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcc27fc700 1 -- 172.21.15.32:0/2391398736 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbccc0072e0 con 0x7fbcd005f3b0 2020-12-05T17:40:16.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcc27fc700 1 -- 172.21.15.32:0/2391398736 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbccc005520 con 0x7fbcd005f3b0 2020-12-05T17:40:16.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcd605e700 1 -- 172.21.15.32:0/2391398736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd0064990 con 0x7fbcd005f3b0 2020-12-05T17:40:16.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcc27fc700 1 -- 172.21.15.32:0/2391398736 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbccc005880 con 0x7fbcd005f3b0 2020-12-05T17:40:16.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcd605e700 1 -- 172.21.15.32:0/2391398736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd00653e0 con 0x7fbcd005f3b0 2020-12-05T17:40:16.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcc27fc700 1 -- 172.21.15.32:0/2391398736 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbccc010070 con 0x7fbcd005f3b0 2020-12-05T17:40:16.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.043+0000 7fbcd605e700 1 -- 172.21.15.32:0/2391398736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd0064690 con 0x7fbcd005f3b0 2020-12-05T17:40:16.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.047+0000 7fbcc27fc700 1 --2- 172.21.15.32:0/2391398736 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbcb0055440 0x7fbcb0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:16.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.047+0000 7fbcc27fc700 1 -- 172.21.15.32:0/2391398736 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbccc06d6e0 con 0x7fbcd005f3b0 2020-12-05T17:40:16.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.047+0000 7fbcd485b700 1 --2- 172.21.15.32:0/2391398736 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbcb0055440 0x7fbcb0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:16.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.047+0000 7fbcd485b700 1 --2- 172.21.15.32:0/2391398736 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbcb0055440 0x7fbcb0057900 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fbcc4008000 tx=0x7fbcc4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:16.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.051+0000 7fbcc27fc700 1 -- 172.21.15.32:0/2391398736 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbcd0064690 con 0x7fbcd005f3b0 2020-12-05T17:40:16.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.239+0000 7fbcd605e700 1 -- 172.21.15.32:0/2391398736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbcd0064690 con 0x7fbcd005f3b0 2020-12-05T17:40:16.244 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.239+0000 7fbcc27fc700 1 -- 172.21.15.32:0/2391398736 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+2142 (secure 0 0 0) 0x7fbcd0064690 con 0x7fbcd005f3b0 2020-12-05T17:40:16.245 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:40:16.245 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":52,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6548,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2853534360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2853534360},{"type":"v1","addr":"172.21.15.61:6833","nonce":2853534360}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":50}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:40:12.749385+0000","modified":"2020-12-05T17:40:13.752777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7563":{"gid":7563,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/2641840537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2641840537},{"type":"v1","addr":"172.21.15.32:6835","nonce":2641840537}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2020-12-05T17:40:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.243+0000 7fbcd605e700 1 -- 172.21.15.32:0/2391398736 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbcb0055440 msgr2=0x7fbcb0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.243+0000 7fbcd605e700 1 --2- 172.21.15.32:0/2391398736 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbcb0055440 0x7fbcb0057900 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fbcc4008000 tx=0x7fbcc4008040).stop 2020-12-05T17:40:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.243+0000 7fbcd605e700 1 -- 172.21.15.32:0/2391398736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd005f3b0 msgr2=0x7fbcd00a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.243+0000 7fbcd605e700 1 --2- 172.21.15.32:0/2391398736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd005f3b0 0x7fbcd00a3920 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7fbcd005c5c0 tx=0x7fbccc007c10).stop 2020-12-05T17:40:16.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.243+0000 7fbcd605e700 1 -- 172.21.15.32:0/2391398736 shutdown_connections 2020-12-05T17:40:16.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.243+0000 7fbcd605e700 1 --2- 172.21.15.32:0/2391398736 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbcd0062400 0x7fbcd00af070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:16.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.243+0000 7fbcd605e700 1 --2- 172.21.15.32:0/2391398736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd005f3b0 0x7fbcd00a3920 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:16.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.243+0000 7fbcd605e700 1 --2- 172.21.15.32:0/2391398736 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbcb0055440 0x7fbcb0057900 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:16.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.243+0000 7fbcd605e700 1 --2- 172.21.15.32:0/2391398736 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbcd005feb0 0x7fbcd0065c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:16.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.243+0000 7fbcd605e700 1 -- 172.21.15.32:0/2391398736 >> 172.21.15.32:0/2391398736 conn(0x7fbcd00b8ee0 msgr2=0x7fbcd01425e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:16.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.243+0000 7fbcd605e700 1 -- 172.21.15.32:0/2391398736 shutdown_connections 2020-12-05T17:40:16.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.243+0000 7fbcd605e700 1 -- 172.21.15.32:0/2391398736 wait complete. 2020-12-05T17:40:16.250 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 52 2020-12-05T17:40:16.461 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:40:12.749385+0000', 'modified': '2020-12-05T17:40:13.752777+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7563}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7563': {'gid': 7563, 'name': 'a', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/2641840537', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 2641840537}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 2641840537}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:40:16.462 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:40:16.462 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:40:16.593 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":7563,"want_state":"up:active","state":"up:active","rank_uptime":3.8299427119999998,"mdsmap_epoch":52,"osdmap_epoch":111,"osdmap_epoch_barrier":110,"uptime":9.022384486} 2020-12-05T17:40:16.593 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 7563, "mdsmap_epoch": 52, "osdmap_epoch": 111, "osdmap_epoch_barrier": 110, "rank_uptime": 3.829942712, "state": "up:active", "uptime": 9.022384486, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:40:16.594 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:40:16.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1bfff700 1 Processor -- start 2020-12-05T17:40:16.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1bfff700 1 -- start start 2020-12-05T17:40:16.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1bfff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ffb1c0b1fc0 0x7ffb1c0af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:16.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb1c0ab710 0x7ffb1c061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:16.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1bfff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb1c146a90 con 0x7ffb1c147400 2020-12-05T17:40:16.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1bfff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb1c0b0110 con 0x7ffb1c0b1fc0 2020-12-05T17:40:16.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1bfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c0b23a0 con 0x7ffb1c0ab710 2020-12-05T17:40:16.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ffb1c147400 0x7ffb1c145d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42138/0 (socket says 172.21.15.32:42138) 2020-12-05T17:40:16.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1b7fe700 1 -- 172.21.15.32:0/1065897194 learned_addr learned my addr 172.21.15.32:0/1065897194 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:16.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1affd700 1 --2- 172.21.15.32:0/1065897194 >> v2:172.21.15.61:3300/0 conn(0x7ffb1c0b1fc0 0x7ffb1c0af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:16.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1a7fc700 1 --2- 172.21.15.32:0/1065897194 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb1c0ab710 0x7ffb1c061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:16.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb19ffb700 1 -- 172.21.15.32:0/1065897194 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 501909871 0 0) 0x7ffb1c146a90 con 0x7ffb1c147400 2020-12-05T17:40:16.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb19ffb700 1 -- 172.21.15.32:0/1065897194 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb00003230 con 0x7ffb1c147400 2020-12-05T17:40:16.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1affd700 1 -- 172.21.15.32:0/1065897194 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb1c0ab710 msgr2=0x7ffb1c061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:16.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1affd700 1 --2- 172.21.15.32:0/1065897194 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb1c0ab710 0x7ffb1c061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:16.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1affd700 1 -- 172.21.15.32:0/1065897194 >> v1:172.21.15.32:6789/0 conn(0x7ffb1c147400 legacy=0x7ffb1c145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:16.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1affd700 1 -- 172.21.15.32:0/1065897194 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1c146cc0 con 0x7ffb1c0b1fc0 2020-12-05T17:40:16.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1affd700 1 --2- 172.21.15.32:0/1065897194 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffb1c0b1fc0 0x7ffb1c0af230 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7ffb0c002f30 tx=0x7ffb0c006dc0).ready entity=mon.1 client_cookie=ac99ce71991c7f8b server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:16.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb19ffb700 1 -- 172.21.15.32:0/1065897194 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb0c0096a0 con 0x7ffb1c0b1fc0 2020-12-05T17:40:16.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb19ffb700 1 -- 172.21.15.32:0/1065897194 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0c009800 con 0x7ffb1c0b1fc0 2020-12-05T17:40:16.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb19ffb700 1 -- 172.21.15.32:0/1065897194 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb0c009b60 con 0x7ffb1c0b1fc0 2020-12-05T17:40:16.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1bfff700 1 -- 172.21.15.32:0/1065897194 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffb1c0b1fc0 msgr2=0x7ffb1c0af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:16.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1bfff700 1 --2- 172.21.15.32:0/1065897194 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffb1c0b1fc0 0x7ffb1c0af230 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7ffb0c002f30 tx=0x7ffb0c006dc0).stop 2020-12-05T17:40:16.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1bfff700 1 -- 172.21.15.32:0/1065897194 shutdown_connections 2020-12-05T17:40:16.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1bfff700 1 --2- 172.21.15.32:0/1065897194 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffb1c0b1fc0 0x7ffb1c0af230 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:16.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1bfff700 1 --2- 172.21.15.32:0/1065897194 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb1c0ab710 0x7ffb1c061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:16.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1bfff700 1 -- 172.21.15.32:0/1065897194 >> 172.21.15.32:0/1065897194 conn(0x7ffb1c0b5530 msgr2=0x7ffb1c0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:16.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1bfff700 1 -- 172.21.15.32:0/1065897194 shutdown_connections 2020-12-05T17:40:16.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.707+0000 7ffb1bfff700 1 -- 172.21.15.32:0/1065897194 wait complete. 2020-12-05T17:40:16.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb1bfff700 1 Processor -- start 2020-12-05T17:40:16.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb1bfff700 1 -- start start 2020-12-05T17:40:16.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb1bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffb1c0ab710 0x7ffb1c155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:16.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb1bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb1c0b1fc0 0x7ffb1c14b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:16.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb1bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb1c147400 0x7ffb1c158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:16.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb1bfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c0aca60 con 0x7ffb1c147400 2020-12-05T17:40:16.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb1bfff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c0acd00 con 0x7ffb1c0ab710 2020-12-05T17:40:16.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb1bfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c158ac0 con 0x7ffb1c0b1fc0 2020-12-05T17:40:16.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb1b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb1c147400 0x7ffb1c158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:16.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb1b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb1c147400 0x7ffb1c158f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39494/0 (socket says 172.21.15.32:39494) 2020-12-05T17:40:16.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb1b7fe700 1 -- 172.21.15.32:0/2406636249 learned_addr learned my addr 172.21.15.32:0/2406636249 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:16.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb1affd700 1 --2- 172.21.15.32:0/2406636249 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffb1c0ab710 0x7ffb1c155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:16.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb1a7fc700 1 --2- 172.21.15.32:0/2406636249 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb1c0b1fc0 0x7ffb1c14b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:16.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb1b7fe700 1 -- 172.21.15.32:0/2406636249 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb1c0b1fc0 msgr2=0x7ffb1c14b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:16.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb1b7fe700 1 --2- 172.21.15.32:0/2406636249 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb1c0b1fc0 0x7ffb1c14b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:16.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb1b7fe700 1 -- 172.21.15.32:0/2406636249 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffb1c0ab710 msgr2=0x7ffb1c155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:16.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb1b7fe700 1 --2- 172.21.15.32:0/2406636249 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffb1c0ab710 0x7ffb1c155570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:16.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb1b7fe700 1 -- 172.21.15.32:0/2406636249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1c0ac070 con 0x7ffb1c147400 2020-12-05T17:40:16.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb1b7fe700 1 --2- 172.21.15.32:0/2406636249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb1c147400 0x7ffb1c158f60 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7ffb14001450 tx=0x7ffb14007e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:16.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb0bfff700 1 -- 172.21.15.32:0/2406636249 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb14005b70 con 0x7ffb1c147400 2020-12-05T17:40:16.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb1bfff700 1 -- 172.21.15.32:0/2406636249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb1c14dc10 con 0x7ffb1c147400 2020-12-05T17:40:16.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb0bfff700 1 -- 172.21.15.32:0/2406636249 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb14005cd0 con 0x7ffb1c147400 2020-12-05T17:40:16.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb1bfff700 1 -- 172.21.15.32:0/2406636249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb1c0abd10 con 0x7ffb1c147400 2020-12-05T17:40:16.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb0bfff700 1 -- 172.21.15.32:0/2406636249 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb14007300 con 0x7ffb1c147400 2020-12-05T17:40:16.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb1bfff700 1 -- 172.21.15.32:0/2406636249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb1c14def0 con 0x7ffb1c147400 2020-12-05T17:40:16.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.711+0000 7ffb0bfff700 1 -- 172.21.15.32:0/2406636249 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ffb14010030 con 0x7ffb1c147400 2020-12-05T17:40:16.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.715+0000 7ffb0bfff700 1 --2- 172.21.15.32:0/2406636249 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffaf8055440 0x7ffaf8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:16.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.715+0000 7ffb0bfff700 1 -- 172.21.15.32:0/2406636249 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ffb1406e150 con 0x7ffb1c147400 2020-12-05T17:40:16.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.715+0000 7ffb1affd700 1 --2- 172.21.15.32:0/2406636249 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffaf8055440 0x7ffaf8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:16.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.715+0000 7ffb1affd700 1 --2- 172.21.15.32:0/2406636249 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffaf8055440 0x7ffaf8057900 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7ffb0c019000 tx=0x7ffb0c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:16.728 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.719+0000 7ffb0bfff700 1 -- 172.21.15.32:0/2406636249 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ffb14044250 con 0x7ffb1c147400 2020-12-05T17:40:16.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.859+0000 7ffb1bfff700 1 -- 172.21.15.32:0/2406636249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb1c158c80 con 0x7ffb1c147400 2020-12-05T17:40:16.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.863+0000 7ffb0bfff700 1 -- 172.21.15.32:0/2406636249 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16285 (secure 0 0 0) 0x7ffb14047870 con 0x7ffb1c147400 2020-12-05T17:40:16.867 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:40:16.868 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":111,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:40:13.752899+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:40:10.513416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2020-12-05T17:40:11.768907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:40:16.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.863+0000 7ffb1bfff700 1 -- 172.21.15.32:0/2406636249 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffaf8055440 msgr2=0x7ffaf8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:16.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.863+0000 7ffb1bfff700 1 --2- 172.21.15.32:0/2406636249 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffaf8055440 0x7ffaf8057900 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7ffb0c019000 tx=0x7ffb0c015000).stop 2020-12-05T17:40:16.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.863+0000 7ffb1bfff700 1 -- 172.21.15.32:0/2406636249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb1c147400 msgr2=0x7ffb1c158f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:16.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.863+0000 7ffb1bfff700 1 --2- 172.21.15.32:0/2406636249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb1c147400 0x7ffb1c158f60 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7ffb14001450 tx=0x7ffb14007e40).stop 2020-12-05T17:40:16.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.863+0000 7ffb1bfff700 1 -- 172.21.15.32:0/2406636249 shutdown_connections 2020-12-05T17:40:16.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.863+0000 7ffb1bfff700 1 --2- 172.21.15.32:0/2406636249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb1c147400 0x7ffb1c158f60 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:16.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.863+0000 7ffb1bfff700 1 --2- 172.21.15.32:0/2406636249 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffb1c0ab710 0x7ffb1c155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:16.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.863+0000 7ffb1bfff700 1 --2- 172.21.15.32:0/2406636249 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffaf8055440 0x7ffaf8057900 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:16.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.863+0000 7ffb1bfff700 1 --2- 172.21.15.32:0/2406636249 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb1c0b1fc0 0x7ffb1c14b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:16.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.863+0000 7ffb1bfff700 1 -- 172.21.15.32:0/2406636249 >> 172.21.15.32:0/2406636249 conn(0x7ffb1c0b5530 msgr2=0x7ffb1c144260 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:16.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.863+0000 7ffb1bfff700 1 -- 172.21.15.32:0/2406636249 shutdown_connections 2020-12-05T17:40:16.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.863+0000 7ffb1bfff700 1 -- 172.21.15.32:0/2406636249 wait complete. 2020-12-05T17:40:16.881 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:40:10.513416+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:40:11.768907+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:40:16.882 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:40:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.999+0000 7fd285dcf700 1 Processor -- start 2020-12-05T17:40:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.999+0000 7fd285dcf700 1 -- start start 2020-12-05T17:40:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.999+0000 7fd285dcf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd2800b5b20 0x7fd2800b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.999+0000 7fd285dcf700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2800b5240 0x7fd280145f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.999+0000 7fd285dcf700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2800b56d0 con 0x7fd2800b6690 2020-12-05T17:40:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.999+0000 7fd285dcf700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2800b1920 con 0x7fd2800b5b20 2020-12-05T17:40:17.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.999+0000 7fd285dcf700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2800b1bc0 con 0x7fd2800b5240 2020-12-05T17:40:17.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.999+0000 7fd27f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd2800b5b20 0x7fd2800b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.999+0000 7fd27f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd2800b5b20 0x7fd2800b5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39500/0 (socket says 172.21.15.32:39500) 2020-12-05T17:40:17.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.999+0000 7fd27f7fe700 1 -- 172.21.15.32:0/3616060206 learned_addr learned my addr 172.21.15.32:0/3616060206 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:17.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.999+0000 7fd27effd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd2800b6690 0x7fd2800b2060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42152/0 (socket says 172.21.15.32:42152) 2020-12-05T17:40:17.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.999+0000 7fd27ffff700 1 --2- 172.21.15.32:0/3616060206 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2800b5240 0x7fd280145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.999+0000 7fd27f7fe700 1 -- 172.21.15.32:0/3616060206 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2800b5240 msgr2=0x7fd280145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:17.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.999+0000 7fd27f7fe700 1 --2- 172.21.15.32:0/3616060206 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2800b5240 0x7fd280145f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.999+0000 7fd27f7fe700 1 -- 172.21.15.32:0/3616060206 >> v1:172.21.15.32:6789/0 conn(0x7fd2800b6690 legacy=0x7fd2800b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:17.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.999+0000 7fd27f7fe700 1 -- 172.21.15.32:0/3616060206 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd280146400 con 0x7fd2800b5b20 2020-12-05T17:40:17.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.999+0000 7fd27ffff700 1 --2- 172.21.15.32:0/3616060206 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2800b5240 0x7fd280145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:40:17.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.999+0000 7fd27f7fe700 1 --2- 172.21.15.32:0/3616060206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2800b5b20 0x7fd2800b5f00 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7fd270002f30 tx=0x7fd270006e80).ready entity=mon.0 client_cookie=546135daa2a9bdcf server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.999+0000 7fd27e7fc700 1 -- 172.21.15.32:0/3616060206 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2700097f0 con 0x7fd2800b5b20 2020-12-05T17:40:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.999+0000 7fd27e7fc700 1 -- 172.21.15.32:0/3616060206 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd270009950 con 0x7fd2800b5b20 2020-12-05T17:40:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.999+0000 7fd27e7fc700 1 -- 172.21.15.32:0/3616060206 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd270009cb0 con 0x7fd2800b5b20 2020-12-05T17:40:17.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.999+0000 7fd285dcf700 1 -- 172.21.15.32:0/3616060206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2800b5b20 msgr2=0x7fd2800b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:17.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:16.999+0000 7fd285dcf700 1 --2- 172.21.15.32:0/3616060206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2800b5b20 0x7fd2800b5f00 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7fd270002f30 tx=0x7fd270006e80).stop 2020-12-05T17:40:17.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd285dcf700 1 -- 172.21.15.32:0/3616060206 shutdown_connections 2020-12-05T17:40:17.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd285dcf700 1 --2- 172.21.15.32:0/3616060206 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2800b5240 0x7fd280145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd285dcf700 1 --2- 172.21.15.32:0/3616060206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2800b5b20 0x7fd2800b5f00 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd285dcf700 1 -- 172.21.15.32:0/3616060206 >> 172.21.15.32:0/3616060206 conn(0x7fd28005f3b0 msgr2=0x7fd28005f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:17.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd285dcf700 1 -- 172.21.15.32:0/3616060206 shutdown_connections 2020-12-05T17:40:17.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd285dcf700 1 -- 172.21.15.32:0/3616060206 wait complete. 2020-12-05T17:40:17.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd285dcf700 1 Processor -- start 2020-12-05T17:40:17.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd285dcf700 1 -- start start 2020-12-05T17:40:17.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd285dcf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2800b5240 0x7fd280066920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd285dcf700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd2800b5b20 0x7fd2801519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd285dcf700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2800b6690 0x7fd280151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd285dcf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2800a58a0 con 0x7fd2800b5240 2020-12-05T17:40:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd285dcf700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2800a5b40 con 0x7fd2800b5b20 2020-12-05T17:40:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd285dcf700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2800a5de0 con 0x7fd2800b6690 2020-12-05T17:40:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd27f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2800b5240 0x7fd280066920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd27f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2800b5240 0x7fd280066920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39506/0 (socket says 172.21.15.32:39506) 2020-12-05T17:40:17.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd27f7fe700 1 -- 172.21.15.32:0/3740983033 learned_addr learned my addr 172.21.15.32:0/3740983033 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:17.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd27ffff700 1 --2- 172.21.15.32:0/3740983033 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2800b6690 0x7fd280151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd27effd700 1 --2- 172.21.15.32:0/3740983033 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd2800b5b20 0x7fd2801519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd27f7fe700 1 -- 172.21.15.32:0/3740983033 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2800b6690 msgr2=0x7fd280151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:17.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd27f7fe700 1 --2- 172.21.15.32:0/3740983033 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2800b6690 0x7fd280151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd27f7fe700 1 -- 172.21.15.32:0/3740983033 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd2800b5b20 msgr2=0x7fd2801519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:17.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd27f7fe700 1 --2- 172.21.15.32:0/3740983033 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd2800b5b20 0x7fd2801519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd27f7fe700 1 -- 172.21.15.32:0/3740983033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2800a6280 con 0x7fd2800b5240 2020-12-05T17:40:17.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd27effd700 1 --2- 172.21.15.32:0/3740983033 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd2800b5b20 0x7fd2801519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:40:17.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd27ffff700 1 --2- 172.21.15.32:0/3740983033 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2800b6690 0x7fd280151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:40:17.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd27f7fe700 1 --2- 172.21.15.32:0/3740983033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2800b5240 0x7fd280066920 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7fd270002f30 tx=0x7fd270015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:17.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd27cff9700 1 -- 172.21.15.32:0/3740983033 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd270011ce0 con 0x7fd2800b5240 2020-12-05T17:40:17.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd27cff9700 1 -- 172.21.15.32:0/3740983033 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd270011e40 con 0x7fd2800b5240 2020-12-05T17:40:17.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd285dcf700 1 -- 172.21.15.32:0/3740983033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2800a5520 con 0x7fd2800b5240 2020-12-05T17:40:17.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd27cff9700 1 -- 172.21.15.32:0/3740983033 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd27001e650 con 0x7fd2800b5240 2020-12-05T17:40:17.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.003+0000 7fd285dcf700 1 -- 172.21.15.32:0/3740983033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2800a30f0 con 0x7fd2800b5240 2020-12-05T17:40:17.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.007+0000 7fd285dcf700 1 -- 172.21.15.32:0/3740983033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd280057050 con 0x7fd2800b5240 2020-12-05T17:40:17.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.007+0000 7fd27cff9700 1 -- 172.21.15.32:0/3740983033 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd270025030 con 0x7fd2800b5240 2020-12-05T17:40:17.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.007+0000 7fd27cff9700 1 --2- 172.21.15.32:0/3740983033 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd260055440 0x7fd260057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.007+0000 7fd27effd700 1 --2- 172.21.15.32:0/3740983033 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd260055440 0x7fd260057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.007+0000 7fd27cff9700 1 -- 172.21.15.32:0/3740983033 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd27006f4f0 con 0x7fd2800b5240 2020-12-05T17:40:17.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.007+0000 7fd27effd700 1 --2- 172.21.15.32:0/3740983033 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd260055440 0x7fd260057900 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fd274008000 tx=0x7fd274009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:17.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.015+0000 7fd27cff9700 1 -- 172.21.15.32:0/3740983033 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd270045580 con 0x7fd2800b5240 2020-12-05T17:40:17.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.159+0000 7fd285dcf700 1 -- 172.21.15.32:0/3740983033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd28005bae0 con 0x7fd2800b5240 2020-12-05T17:40:17.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.159+0000 7fd27cff9700 1 -- 172.21.15.32:0/3740983033 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v111) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd270023020 con 0x7fd2800b5240 2020-12-05T17:40:17.164 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:40:17.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.159+0000 7fd285dcf700 1 -- 172.21.15.32:0/3740983033 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd260055440 msgr2=0x7fd260057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:17.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.159+0000 7fd285dcf700 1 --2- 172.21.15.32:0/3740983033 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd260055440 0x7fd260057900 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fd274008000 tx=0x7fd274009040).stop 2020-12-05T17:40:17.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.159+0000 7fd285dcf700 1 -- 172.21.15.32:0/3740983033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2800b5240 msgr2=0x7fd280066920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:17.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.159+0000 7fd285dcf700 1 --2- 172.21.15.32:0/3740983033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2800b5240 0x7fd280066920 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7fd270002f30 tx=0x7fd270015000).stop 2020-12-05T17:40:17.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.159+0000 7fd285dcf700 1 -- 172.21.15.32:0/3740983033 shutdown_connections 2020-12-05T17:40:17.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.159+0000 7fd285dcf700 1 --2- 172.21.15.32:0/3740983033 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd260055440 0x7fd260057900 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.159+0000 7fd285dcf700 1 --2- 172.21.15.32:0/3740983033 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd2800b6690 0x7fd280151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.159+0000 7fd285dcf700 1 --2- 172.21.15.32:0/3740983033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2800b5240 0x7fd280066920 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.159+0000 7fd285dcf700 1 --2- 172.21.15.32:0/3740983033 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd2800b5b20 0x7fd2801519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.159+0000 7fd285dcf700 1 -- 172.21.15.32:0/3740983033 >> 172.21.15.32:0/3740983033 conn(0x7fd28005f3b0 msgr2=0x7fd2800b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:17.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.159+0000 7fd285dcf700 1 -- 172.21.15.32:0/3740983033 shutdown_connections 2020-12-05T17:40:17.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.159+0000 7fd285dcf700 1 -- 172.21.15.32:0/3740983033 wait complete. 2020-12-05T17:40:17.178 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:40:17.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.291+0000 7f71c359e700 1 Processor -- start 2020-12-05T17:40:17.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.291+0000 7f71c359e700 1 -- start start 2020-12-05T17:40:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.291+0000 7f71c359e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f71c40b2000 0x7f71c4145e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.291+0000 7f71c359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71c40af4b0 0x7f71c41462f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.291+0000 7f71c359e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c40b23e0 con 0x7f71c4061780 2020-12-05T17:40:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.291+0000 7f71c359e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c40ae7b0 con 0x7f71c40af4b0 2020-12-05T17:40:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.291+0000 7f71c359e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c40aea50 con 0x7f71c40b2000 2020-12-05T17:40:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.291+0000 7f71c1d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71c40af4b0 0x7f71c41462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.291+0000 7f71c1d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71c40af4b0 0x7f71c41462f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39514/0 (socket says 172.21.15.32:39514) 2020-12-05T17:40:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.291+0000 7f71c1d9b700 1 -- 172.21.15.32:0/1728302058 learned_addr learned my addr 172.21.15.32:0/1728302058 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.291+0000 7f71c259c700 1 --2- 172.21.15.32:0/1728302058 >> v2:172.21.15.61:3300/0 conn(0x7f71c40b2000 0x7f71c4145e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.291+0000 7f71c1d9b700 1 -- 172.21.15.32:0/1728302058 >> v2:172.21.15.61:3300/0 conn(0x7f71c40b2000 msgr2=0x7f71c4145e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.291+0000 7f71c1d9b700 1 --2- 172.21.15.32:0/1728302058 >> v2:172.21.15.61:3300/0 conn(0x7f71c40b2000 0x7f71c4145e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.291+0000 7f71c1d9b700 1 -- 172.21.15.32:0/1728302058 >> v1:172.21.15.61:6789/0 conn(0x7f71c4061780 legacy=0x7f71c40afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:17.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.291+0000 7f71c1d9b700 1 -- 172.21.15.32:0/1728302058 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c40b0f60 con 0x7f71c40af4b0 2020-12-05T17:40:17.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.291+0000 7f71c1d9b700 1 --2- 172.21.15.32:0/1728302058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c40af4b0 0x7f71c41462f0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7f71ac002f30 tx=0x7f71ac006e80).ready entity=mon.0 client_cookie=f7527095ad751d30 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:17.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.291+0000 7f71c159a700 1 -- 172.21.15.32:0/1728302058 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71ac0097f0 con 0x7f71c40af4b0 2020-12-05T17:40:17.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.291+0000 7f71c159a700 1 -- 172.21.15.32:0/1728302058 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ac009950 con 0x7f71c40af4b0 2020-12-05T17:40:17.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.291+0000 7f71c159a700 1 -- 172.21.15.32:0/1728302058 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71ac009cb0 con 0x7f71c40af4b0 2020-12-05T17:40:17.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.291+0000 7f71c359e700 1 -- 172.21.15.32:0/1728302058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c40af4b0 msgr2=0x7f71c41462f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:17.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.291+0000 7f71c359e700 1 --2- 172.21.15.32:0/1728302058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c40af4b0 0x7f71c41462f0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7f71ac002f30 tx=0x7f71ac006e80).stop 2020-12-05T17:40:17.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c359e700 1 -- 172.21.15.32:0/1728302058 shutdown_connections 2020-12-05T17:40:17.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c359e700 1 --2- 172.21.15.32:0/1728302058 >> v2:172.21.15.61:3300/0 conn(0x7f71c40b2000 0x7f71c4145e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c359e700 1 --2- 172.21.15.32:0/1728302058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c40af4b0 0x7f71c41462f0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c359e700 1 -- 172.21.15.32:0/1728302058 >> 172.21.15.32:0/1728302058 conn(0x7f71c40b5620 msgr2=0x7f71c40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:17.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c359e700 1 -- 172.21.15.32:0/1728302058 shutdown_connections 2020-12-05T17:40:17.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c359e700 1 -- 172.21.15.32:0/1728302058 wait complete. 2020-12-05T17:40:17.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c359e700 1 Processor -- start 2020-12-05T17:40:17.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c359e700 1 -- start start 2020-12-05T17:40:17.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c359e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71c4061780 0x7f71c414c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c40af4b0 0x7f71c410ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c359e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71c40b2000 0x7f71c4066680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c40647a0 con 0x7f71c40af4b0 2020-12-05T17:40:17.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c4066220 con 0x7f71c4061780 2020-12-05T17:40:17.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c359e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c40649d0 con 0x7f71c40b2000 2020-12-05T17:40:17.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c40af4b0 0x7f71c410ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c40af4b0 0x7f71c410ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39522/0 (socket says 172.21.15.32:39522) 2020-12-05T17:40:17.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c1d9b700 1 -- 172.21.15.32:0/1412664652 learned_addr learned my addr 172.21.15.32:0/1412664652 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:17.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c259c700 1 --2- 172.21.15.32:0/1412664652 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71c4061780 0x7f71c414c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c2d9d700 1 --2- 172.21.15.32:0/1412664652 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71c40b2000 0x7f71c4066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c1d9b700 1 -- 172.21.15.32:0/1412664652 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71c40b2000 msgr2=0x7f71c4066680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:17.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c1d9b700 1 --2- 172.21.15.32:0/1412664652 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71c40b2000 0x7f71c4066680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c1d9b700 1 -- 172.21.15.32:0/1412664652 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71c4061780 msgr2=0x7f71c414c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:17.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c1d9b700 1 --2- 172.21.15.32:0/1412664652 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71c4061780 0x7f71c414c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.295+0000 7f71c1d9b700 1 -- 172.21.15.32:0/1412664652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c410f340 con 0x7f71c40af4b0 2020-12-05T17:40:17.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.299+0000 7f71c1d9b700 1 --2- 172.21.15.32:0/1412664652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c40af4b0 0x7f71c410ee50 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7f71ac002f30 tx=0x7f71ac015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:17.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.299+0000 7f71bb7fe700 1 -- 172.21.15.32:0/1412664652 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71ac011c30 con 0x7f71c40af4b0 2020-12-05T17:40:17.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.299+0000 7f71bb7fe700 1 -- 172.21.15.32:0/1412664652 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ac011d90 con 0x7f71c40af4b0 2020-12-05T17:40:17.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.299+0000 7f71c359e700 1 -- 172.21.15.32:0/1412664652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c4065010 con 0x7f71c40af4b0 2020-12-05T17:40:17.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.299+0000 7f71bb7fe700 1 -- 172.21.15.32:0/1412664652 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71ac01e650 con 0x7f71c40af4b0 2020-12-05T17:40:17.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.299+0000 7f71c359e700 1 -- 172.21.15.32:0/1412664652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c4111840 con 0x7f71c40af4b0 2020-12-05T17:40:17.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.299+0000 7f71c359e700 1 -- 172.21.15.32:0/1412664652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c4066eb0 con 0x7f71c40af4b0 2020-12-05T17:40:17.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.299+0000 7f71bb7fe700 1 -- 172.21.15.32:0/1412664652 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f71ac025030 con 0x7f71c40af4b0 2020-12-05T17:40:17.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.299+0000 7f71bb7fe700 1 --2- 172.21.15.32:0/1412664652 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f719c0553b0 0x7f719c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.299+0000 7f71bb7fe700 1 -- 172.21.15.32:0/1412664652 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f71ac06f460 con 0x7f71c40af4b0 2020-12-05T17:40:17.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.299+0000 7f71c259c700 1 --2- 172.21.15.32:0/1412664652 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f719c0553b0 0x7f719c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.303+0000 7f71c259c700 1 --2- 172.21.15.32:0/1412664652 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f719c0553b0 0x7f719c057870 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f71bc00b000 tx=0x7f71bc006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:17.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.307+0000 7f71bb7fe700 1 -- 172.21.15.32:0/1412664652 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f71c4066eb0 con 0x7f71c40af4b0 2020-12-05T17:40:17.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.447+0000 7f71c359e700 1 -- 172.21.15.32:0/1412664652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f71c40663c0 con 0x7f71c40af4b0 2020-12-05T17:40:17.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.447+0000 7f71bb7fe700 1 -- 172.21.15.32:0/1412664652 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v111) v1 ==== 92+0+10 (secure 0 0 0) 0x7f71c40663c0 con 0x7f71c40af4b0 2020-12-05T17:40:17.454 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:40:17.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.451+0000 7f71c359e700 1 -- 172.21.15.32:0/1412664652 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f719c0553b0 msgr2=0x7f719c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:17.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.451+0000 7f71c359e700 1 --2- 172.21.15.32:0/1412664652 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f719c0553b0 0x7f719c057870 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f71bc00b000 tx=0x7f71bc006dc0).stop 2020-12-05T17:40:17.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.451+0000 7f71c359e700 1 -- 172.21.15.32:0/1412664652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c40af4b0 msgr2=0x7f71c410ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:17.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.451+0000 7f71c359e700 1 --2- 172.21.15.32:0/1412664652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c40af4b0 0x7f71c410ee50 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7f71ac002f30 tx=0x7f71ac015000).stop 2020-12-05T17:40:17.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.451+0000 7f71c359e700 1 -- 172.21.15.32:0/1412664652 shutdown_connections 2020-12-05T17:40:17.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.451+0000 7f71c359e700 1 --2- 172.21.15.32:0/1412664652 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f719c0553b0 0x7f719c057870 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.451+0000 7f71c359e700 1 --2- 172.21.15.32:0/1412664652 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71c40b2000 0x7f71c4066680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.451+0000 7f71c359e700 1 --2- 172.21.15.32:0/1412664652 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71c4061780 0x7f71c414c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.451+0000 7f71c359e700 1 --2- 172.21.15.32:0/1412664652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c40af4b0 0x7f71c410ee50 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.451+0000 7f71c359e700 1 -- 172.21.15.32:0/1412664652 >> 172.21.15.32:0/1412664652 conn(0x7f71c40b5620 msgr2=0x7f71c4148050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:17.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.451+0000 7f71c359e700 1 -- 172.21.15.32:0/1412664652 shutdown_connections 2020-12-05T17:40:17.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.451+0000 7f71c359e700 1 -- 172.21.15.32:0/1412664652 wait complete. 2020-12-05T17:40:17.468 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:40:17.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.571+0000 7f2032eef700 1 Processor -- start 2020-12-05T17:40:17.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.571+0000 7f2032eef700 1 -- start start 2020-12-05T17:40:17.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f2032eef700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f202c0b3490 0x7f202c0b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f2032eef700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f202c063040 0x7f202c0b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f2032eef700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f202c05fad0 con 0x7f202c05f6f0 2020-12-05T17:40:17.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f2032eef700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f202c0b1d50 con 0x7f202c0b3490 2020-12-05T17:40:17.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f2032eef700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f202c0b1fd0 con 0x7f202c063040 2020-12-05T17:40:17.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f202bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f202c0b3490 0x7f202c0b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f202bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f202c0b3490 0x7f202c0b08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39530/0 (socket says 172.21.15.32:39530) 2020-12-05T17:40:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f202bfff700 1 -- 172.21.15.32:0/2042790152 learned_addr learned my addr 172.21.15.32:0/2042790152 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f203148c700 1 --2- 172.21.15.32:0/2042790152 >> v2:172.21.15.61:3300/0 conn(0x7f202c063040 0x7f202c0b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f202bfff700 1 -- 172.21.15.32:0/2042790152 >> v2:172.21.15.61:3300/0 conn(0x7f202c063040 msgr2=0x7f202c0b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:17.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f202bfff700 1 --2- 172.21.15.32:0/2042790152 >> v2:172.21.15.61:3300/0 conn(0x7f202c063040 0x7f202c0b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f202bfff700 1 -- 172.21.15.32:0/2042790152 >> v1:172.21.15.61:6789/0 conn(0x7f202c05f6f0 legacy=0x7f202c143ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:17.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f202bfff700 1 -- 172.21.15.32:0/2042790152 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c0b2270 con 0x7f202c0b3490 2020-12-05T17:40:17.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f202bfff700 1 --2- 172.21.15.32:0/2042790152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f202c0b3490 0x7f202c0b08e0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 rx=0x7f2018002f30 tx=0x7f2018006e80).ready entity=mon.0 client_cookie=6ae938de3f85c014 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:17.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f202b7fe700 1 -- 172.21.15.32:0/2042790152 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20180097f0 con 0x7f202c0b3490 2020-12-05T17:40:17.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f202b7fe700 1 -- 172.21.15.32:0/2042790152 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2018009950 con 0x7f202c0b3490 2020-12-05T17:40:17.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f202b7fe700 1 -- 172.21.15.32:0/2042790152 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2018009cb0 con 0x7f202c0b3490 2020-12-05T17:40:17.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f2032eef700 1 -- 172.21.15.32:0/2042790152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f202c0b3490 msgr2=0x7f202c0b08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:17.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f2032eef700 1 --2- 172.21.15.32:0/2042790152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f202c0b3490 0x7f202c0b08e0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 rx=0x7f2018002f30 tx=0x7f2018006e80).stop 2020-12-05T17:40:17.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f2032eef700 1 -- 172.21.15.32:0/2042790152 shutdown_connections 2020-12-05T17:40:17.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f2032eef700 1 --2- 172.21.15.32:0/2042790152 >> v2:172.21.15.61:3300/0 conn(0x7f202c063040 0x7f202c0b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f2032eef700 1 --2- 172.21.15.32:0/2042790152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f202c0b3490 0x7f202c0b08e0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f2032eef700 1 -- 172.21.15.32:0/2042790152 >> 172.21.15.32:0/2042790152 conn(0x7f202c05d070 msgr2=0x7f202c0604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:17.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.575+0000 7f2032eef700 1 -- 172.21.15.32:0/2042790152 shutdown_connections 2020-12-05T17:40:17.587 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.579+0000 7f2032eef700 1 -- 172.21.15.32:0/2042790152 wait complete. 2020-12-05T17:40:17.589 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.579+0000 7f2032eef700 1 Processor -- start 2020-12-05T17:40:17.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.579+0000 7f2032eef700 1 -- start start 2020-12-05T17:40:17.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.579+0000 7f2032eef700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f202c05f6f0 0x7f202c065460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.579+0000 7f2032eef700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f202c063040 0x7f202c146430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.590 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.579+0000 7f2032eef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f202c0b3490 0x7f202c13f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.579+0000 7f2032eef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c065f70 con 0x7f202c0b3490 2020-12-05T17:40:17.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.579+0000 7f2032eef700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c140a20 con 0x7f202c05f6f0 2020-12-05T17:40:17.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.579+0000 7f2032eef700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202c140cc0 con 0x7f202c063040 2020-12-05T17:40:17.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.579+0000 7f203148c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f202c0b3490 0x7f202c13f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.591 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.579+0000 7f203148c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f202c0b3490 0x7f202c13f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39538/0 (socket says 172.21.15.32:39538) 2020-12-05T17:40:17.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.579+0000 7f203148c700 1 -- 172.21.15.32:0/3532011605 learned_addr learned my addr 172.21.15.32:0/3532011605 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:17.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.579+0000 7f2030c8b700 1 --2- 172.21.15.32:0/3532011605 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f202c05f6f0 0x7f202c065460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.579+0000 7f202bfff700 1 --2- 172.21.15.32:0/3532011605 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f202c063040 0x7f202c146430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.592 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.583+0000 7f203148c700 1 -- 172.21.15.32:0/3532011605 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f202c063040 msgr2=0x7f202c146430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:17.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.583+0000 7f203148c700 1 --2- 172.21.15.32:0/3532011605 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f202c063040 0x7f202c146430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.583+0000 7f203148c700 1 -- 172.21.15.32:0/3532011605 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f202c05f6f0 msgr2=0x7f202c065460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:17.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.583+0000 7f203148c700 1 --2- 172.21.15.32:0/3532011605 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f202c05f6f0 0x7f202c065460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.583+0000 7f203148c700 1 -- 172.21.15.32:0/3532011605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c141160 con 0x7f202c0b3490 2020-12-05T17:40:17.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.583+0000 7f203148c700 1 --2- 172.21.15.32:0/3532011605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f202c0b3490 0x7f202c13f900 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7f2024006dd0 tx=0x7f20240091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:17.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.583+0000 7f2029ffb700 1 -- 172.21.15.32:0/3532011605 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f202400c950 con 0x7f202c0b3490 2020-12-05T17:40:17.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.583+0000 7f2029ffb700 1 -- 172.21.15.32:0/3532011605 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202400cab0 con 0x7f202c0b3490 2020-12-05T17:40:17.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.583+0000 7f2032eef700 1 -- 172.21.15.32:0/3532011605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202c1413b0 con 0x7f202c0b3490 2020-12-05T17:40:17.594 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.583+0000 7f2029ffb700 1 -- 172.21.15.32:0/3532011605 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f202400ce10 con 0x7f202c0b3490 2020-12-05T17:40:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.583+0000 7f2032eef700 1 -- 172.21.15.32:0/3532011605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202c065b90 con 0x7f202c0b3490 2020-12-05T17:40:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.583+0000 7f2029ffb700 1 -- 172.21.15.32:0/3532011605 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2024022070 con 0x7f202c0b3490 2020-12-05T17:40:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.583+0000 7f2029ffb700 1 --2- 172.21.15.32:0/3532011605 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2014055440 0x7f2014057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.583+0000 7f2029ffb700 1 -- 172.21.15.32:0/3532011605 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f202406e690 con 0x7f202c0b3490 2020-12-05T17:40:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.583+0000 7f2030c8b700 1 --2- 172.21.15.32:0/3532011605 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2014055440 0x7f2014057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.583+0000 7f2032eef700 1 -- 172.21.15.32:0/3532011605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f202c065de0 con 0x7f202c0b3490 2020-12-05T17:40:17.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.583+0000 7f2030c8b700 1 --2- 172.21.15.32:0/3532011605 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2014055440 0x7f2014057900 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f202000d000 tx=0x7f2020007b60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:17.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.591+0000 7f2029ffb700 1 -- 172.21.15.32:0/3532011605 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2024016080 con 0x7f202c0b3490 2020-12-05T17:40:17.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.731+0000 7f2032eef700 1 -- 172.21.15.32:0/3532011605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f202c140e80 con 0x7f202c0b3490 2020-12-05T17:40:17.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.735+0000 7f2029ffb700 1 -- 172.21.15.32:0/3532011605 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v111) v1 ==== 104+0+10 (secure 0 0 0) 0x7f2024044830 con 0x7f202c0b3490 2020-12-05T17:40:17.739 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:40:17.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.735+0000 7f2032eef700 1 -- 172.21.15.32:0/3532011605 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2014055440 msgr2=0x7f2014057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:17.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.735+0000 7f2032eef700 1 --2- 172.21.15.32:0/3532011605 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2014055440 0x7f2014057900 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f202000d000 tx=0x7f2020007b60).stop 2020-12-05T17:40:17.740 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.735+0000 7f2032eef700 1 -- 172.21.15.32:0/3532011605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f202c0b3490 msgr2=0x7f202c13f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:17.740 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.735+0000 7f2032eef700 1 --2- 172.21.15.32:0/3532011605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f202c0b3490 0x7f202c13f900 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7f2024006dd0 tx=0x7f20240091a0).stop 2020-12-05T17:40:17.741 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.735+0000 7f2032eef700 1 -- 172.21.15.32:0/3532011605 shutdown_connections 2020-12-05T17:40:17.741 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.735+0000 7f2032eef700 1 --2- 172.21.15.32:0/3532011605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f202c0b3490 0x7f202c13f900 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.741 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.735+0000 7f2032eef700 1 --2- 172.21.15.32:0/3532011605 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f202c05f6f0 0x7f202c065460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.735+0000 7f2032eef700 1 --2- 172.21.15.32:0/3532011605 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2014055440 0x7f2014057900 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.735+0000 7f2032eef700 1 --2- 172.21.15.32:0/3532011605 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f202c063040 0x7f202c146430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.735+0000 7f2032eef700 1 -- 172.21.15.32:0/3532011605 >> 172.21.15.32:0/3532011605 conn(0x7f202c05d070 msgr2=0x7f202c0b2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:17.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.735+0000 7f2032eef700 1 -- 172.21.15.32:0/3532011605 shutdown_connections 2020-12-05T17:40:17.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.735+0000 7f2032eef700 1 -- 172.21.15.32:0/3532011605 wait complete. 2020-12-05T17:40:17.753 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:40:17.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.867+0000 7fcb6913b700 1 Processor -- start 2020-12-05T17:40:17.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.867+0000 7fcb6913b700 1 -- start start 2020-12-05T17:40:17.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.867+0000 7fcb6913b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb640b2960 0x7fcb64140400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.867+0000 7fcb6913b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb640ad280 0x7fcb640660e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.867+0000 7fcb6913b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fcb64061b30 0x7fcb640665d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.867+0000 7fcb6913b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb640b32c0 con 0x7fcb640ad280 2020-12-05T17:40:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.867+0000 7fcb6913b700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb64066ac0 con 0x7fcb64061b30 2020-12-05T17:40:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.867+0000 7fcb6913b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb640ad9b0 con 0x7fcb640b2960 2020-12-05T17:40:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.867+0000 7fcb5b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb640ad280 0x7fcb640660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.867+0000 7fcb5b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb640ad280 0x7fcb640660e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39546/0 (socket says 172.21.15.32:39546) 2020-12-05T17:40:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.867+0000 7fcb5b7fe700 1 -- 172.21.15.32:0/4147919828 learned_addr learned my addr 172.21.15.32:0/4147919828 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.867+0000 7fcb63fff700 1 --2- 172.21.15.32:0/4147919828 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb640b2960 0x7fcb64140400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.867+0000 7fcb6893a700 1 --2- 172.21.15.32:0/4147919828 >> v2:172.21.15.61:3300/0 conn(0x7fcb64061b30 0x7fcb640665d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.867+0000 7fcb5b7fe700 1 -- 172.21.15.32:0/4147919828 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb640b2960 msgr2=0x7fcb64140400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.867+0000 7fcb5b7fe700 1 --2- 172.21.15.32:0/4147919828 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb640b2960 0x7fcb64140400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.867+0000 7fcb5b7fe700 1 -- 172.21.15.32:0/4147919828 >> v2:172.21.15.61:3300/0 conn(0x7fcb64061b30 msgr2=0x7fcb640665d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:17.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.867+0000 7fcb5b7fe700 1 --2- 172.21.15.32:0/4147919828 >> v2:172.21.15.61:3300/0 conn(0x7fcb64061b30 0x7fcb640665d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.867+0000 7fcb5b7fe700 1 -- 172.21.15.32:0/4147919828 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6413f4f0 con 0x7fcb640ad280 2020-12-05T17:40:17.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.867+0000 7fcb63fff700 1 --2- 172.21.15.32:0/4147919828 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb640b2960 0x7fcb64140400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:40:17.877 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.867+0000 7fcb5b7fe700 1 --2- 172.21.15.32:0/4147919828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb640ad280 0x7fcb640660e0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7fcb64066c00 tx=0x7fcb4c006e80).ready entity=mon.0 client_cookie=d410ed17763af94f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.871+0000 7fcb637fe700 1 -- 172.21.15.32:0/4147919828 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb4c00be60 con 0x7fcb640ad280 2020-12-05T17:40:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.871+0000 7fcb637fe700 1 -- 172.21.15.32:0/4147919828 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c0097b0 con 0x7fcb640ad280 2020-12-05T17:40:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.871+0000 7fcb637fe700 1 -- 172.21.15.32:0/4147919828 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb4c009b10 con 0x7fcb640ad280 2020-12-05T17:40:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.871+0000 7fcb6913b700 1 -- 172.21.15.32:0/4147919828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb640ad280 msgr2=0x7fcb640660e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.871+0000 7fcb6913b700 1 --2- 172.21.15.32:0/4147919828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb640ad280 0x7fcb640660e0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7fcb64066c00 tx=0x7fcb4c006e80).stop 2020-12-05T17:40:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.871+0000 7fcb6913b700 1 -- 172.21.15.32:0/4147919828 shutdown_connections 2020-12-05T17:40:17.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.871+0000 7fcb6913b700 1 --2- 172.21.15.32:0/4147919828 >> v2:172.21.15.61:3300/0 conn(0x7fcb64061b30 0x7fcb640665d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.871+0000 7fcb6913b700 1 --2- 172.21.15.32:0/4147919828 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb640b2960 0x7fcb64140400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.871+0000 7fcb6913b700 1 --2- 172.21.15.32:0/4147919828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb640ad280 0x7fcb640660e0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.871+0000 7fcb6913b700 1 -- 172.21.15.32:0/4147919828 >> 172.21.15.32:0/4147919828 conn(0x7fcb64061010 msgr2=0x7fcb64061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:17.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.871+0000 7fcb6913b700 1 -- 172.21.15.32:0/4147919828 shutdown_connections 2020-12-05T17:40:17.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.871+0000 7fcb6913b700 1 -- 172.21.15.32:0/4147919828 wait complete. 2020-12-05T17:40:17.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.871+0000 7fcb6913b700 1 Processor -- start 2020-12-05T17:40:17.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.875+0000 7fcb6913b700 1 -- start start 2020-12-05T17:40:17.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.875+0000 7fcb6913b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb64061b30 0x7fcb64141d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.875+0000 7fcb6913b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb640ad280 0x7fcb64063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.875+0000 7fcb6913b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb640b2960 0x7fcb6414eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.875+0000 7fcb6913b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb64110170 con 0x7fcb640b2960 2020-12-05T17:40:17.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.875+0000 7fcb6913b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb64064d90 con 0x7fcb64061b30 2020-12-05T17:40:17.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.875+0000 7fcb6913b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb640645a0 con 0x7fcb640ad280 2020-12-05T17:40:17.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.875+0000 7fcb6893a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb640b2960 0x7fcb6414eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.875+0000 7fcb6893a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb640b2960 0x7fcb6414eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39552/0 (socket says 172.21.15.32:39552) 2020-12-05T17:40:17.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.875+0000 7fcb5b7fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb640ad280 0x7fcb64063040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.875+0000 7fcb6893a700 1 -- 172.21.15.32:0/2958187234 learned_addr learned my addr 172.21.15.32:0/2958187234 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:17.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.875+0000 7fcb63fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb64061b30 0x7fcb64141d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.875+0000 7fcb6893a700 1 -- 172.21.15.32:0/2958187234 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb640ad280 msgr2=0x7fcb64063040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:17.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.875+0000 7fcb6893a700 1 --2- 172.21.15.32:0/2958187234 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb640ad280 0x7fcb64063040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.875+0000 7fcb6893a700 1 -- 172.21.15.32:0/2958187234 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb64061b30 msgr2=0x7fcb64141d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:17.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.875+0000 7fcb6893a700 1 --2- 172.21.15.32:0/2958187234 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb64061b30 0x7fcb64141d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:17.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.875+0000 7fcb6893a700 1 -- 172.21.15.32:0/2958187234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb64063530 con 0x7fcb640b2960 2020-12-05T17:40:17.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.875+0000 7fcb6893a700 1 --2- 172.21.15.32:0/2958187234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb640b2960 0x7fcb6414eba0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0x7fcb5c006dd0 tx=0x7fcb5c0091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:17.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.875+0000 7fcb61ffb700 1 -- 172.21.15.32:0/2958187234 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb5c00c950 con 0x7fcb640b2960 2020-12-05T17:40:17.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.875+0000 7fcb61ffb700 1 -- 172.21.15.32:0/2958187234 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5c00cab0 con 0x7fcb640b2960 2020-12-05T17:40:17.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.875+0000 7fcb6913b700 1 -- 172.21.15.32:0/2958187234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb6414e460 con 0x7fcb640b2960 2020-12-05T17:40:17.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.875+0000 7fcb61ffb700 1 -- 172.21.15.32:0/2958187234 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb5c00ce10 con 0x7fcb640b2960 2020-12-05T17:40:17.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.875+0000 7fcb6913b700 1 -- 172.21.15.32:0/2958187234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6410f260 con 0x7fcb640b2960 2020-12-05T17:40:17.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.879+0000 7fcb6913b700 1 -- 172.21.15.32:0/2958187234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb64057050 con 0x7fcb640b2960 2020-12-05T17:40:17.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.879+0000 7fcb61ffb700 1 -- 172.21.15.32:0/2958187234 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fcb5c022070 con 0x7fcb640b2960 2020-12-05T17:40:17.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.879+0000 7fcb61ffb700 1 --2- 172.21.15.32:0/2958187234 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcb480553b0 0x7fcb48057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:17.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.879+0000 7fcb61ffb700 1 -- 172.21.15.32:0/2958187234 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcb5c06e690 con 0x7fcb640b2960 2020-12-05T17:40:17.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.879+0000 7fcb63fff700 1 --2- 172.21.15.32:0/2958187234 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcb480553b0 0x7fcb48057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:17.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.879+0000 7fcb63fff700 1 --2- 172.21.15.32:0/2958187234 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcb480553b0 0x7fcb48057870 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fcb54008000 tx=0x7fcb54008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:17.883+0000 7fcb61ffb700 1 -- 172.21.15.32:0/2958187234 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fcb5c044830 con 0x7fcb640b2960 2020-12-05T17:40:18.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.027+0000 7fcb6913b700 1 -- 172.21.15.32:0/2958187234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcb64069690 con 0x7fcb640b2960 2020-12-05T17:40:18.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.027+0000 7fcb61ffb700 1 -- 172.21.15.32:0/2958187234 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v111) v1 ==== 100+0+10 (secure 0 0 0) 0x7fcb5c016020 con 0x7fcb640b2960 2020-12-05T17:40:18.033 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:40:18.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.031+0000 7fcb6913b700 1 -- 172.21.15.32:0/2958187234 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcb480553b0 msgr2=0x7fcb48057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:18.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.031+0000 7fcb6913b700 1 --2- 172.21.15.32:0/2958187234 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcb480553b0 0x7fcb48057870 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fcb54008000 tx=0x7fcb54008040).stop 2020-12-05T17:40:18.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.031+0000 7fcb6913b700 1 -- 172.21.15.32:0/2958187234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb640b2960 msgr2=0x7fcb6414eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.031+0000 7fcb6913b700 1 --2- 172.21.15.32:0/2958187234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb640b2960 0x7fcb6414eba0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0x7fcb5c006dd0 tx=0x7fcb5c0091a0).stop 2020-12-05T17:40:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.031+0000 7fcb6913b700 1 -- 172.21.15.32:0/2958187234 shutdown_connections 2020-12-05T17:40:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.031+0000 7fcb6913b700 1 --2- 172.21.15.32:0/2958187234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb640b2960 0x7fcb6414eba0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.031+0000 7fcb6913b700 1 --2- 172.21.15.32:0/2958187234 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb64061b30 0x7fcb64141d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.031+0000 7fcb6913b700 1 --2- 172.21.15.32:0/2958187234 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcb480553b0 0x7fcb48057870 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:18.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.031+0000 7fcb6913b700 1 --2- 172.21.15.32:0/2958187234 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb640ad280 0x7fcb64063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:18.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.031+0000 7fcb6913b700 1 -- 172.21.15.32:0/2958187234 >> 172.21.15.32:0/2958187234 conn(0x7fcb64061010 msgr2=0x7fcb640613f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:18.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.031+0000 7fcb6913b700 1 -- 172.21.15.32:0/2958187234 shutdown_connections 2020-12-05T17:40:18.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.031+0000 7fcb6913b700 1 -- 172.21.15.32:0/2958187234 wait complete. 2020-12-05T17:40:18.052 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:40:18.053 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:40:18.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.163+0000 7f5d4fe18700 1 Processor -- start 2020-12-05T17:40:18.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.163+0000 7f5d4fe18700 1 -- start start 2020-12-05T17:40:18.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.163+0000 7f5d4fe18700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f5d4805f6f0 0x7f5d48143d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:18.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.163+0000 7f5d4fe18700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5d48065d90 0x7f5d4813a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:18.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.163+0000 7f5d4fe18700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d480acb80 con 0x7f5d4813a380 2020-12-05T17:40:18.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.163+0000 7f5d4fe18700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d4813ad20 con 0x7f5d48065d90 2020-12-05T17:40:18.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.163+0000 7f5d4fe18700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d480b2860 con 0x7f5d4805f6f0 2020-12-05T17:40:18.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.163+0000 7f5d4e3b5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5d48065d90 0x7f5d4813a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:18.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.163+0000 7f5d4d3b3700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5d4813a380 0x7f5d48065680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42204/0 (socket says 172.21.15.32:42204) 2020-12-05T17:40:18.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.163+0000 7f5d4d3b3700 1 -- 172.21.15.32:0/2844226291 learned_addr learned my addr 172.21.15.32:0/2844226291 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:18.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.163+0000 7f5d4dbb4700 1 --2- 172.21.15.32:0/2844226291 >> v2:172.21.15.61:3300/0 conn(0x7f5d4805f6f0 0x7f5d48143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:18.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.163+0000 7f5d4e3b5700 1 -- 172.21.15.32:0/2844226291 >> v2:172.21.15.61:3300/0 conn(0x7f5d4805f6f0 msgr2=0x7f5d48143d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:18.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.163+0000 7f5d4e3b5700 1 --2- 172.21.15.32:0/2844226291 >> v2:172.21.15.61:3300/0 conn(0x7f5d4805f6f0 0x7f5d48143d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:18.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.163+0000 7f5d4e3b5700 1 -- 172.21.15.32:0/2844226291 >> v1:172.21.15.32:6789/0 conn(0x7f5d4813a380 legacy=0x7f5d48065680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:18.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.163+0000 7f5d4e3b5700 1 -- 172.21.15.32:0/2844226291 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d48065250 con 0x7f5d48065d90 2020-12-05T17:40:18.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.163+0000 7f5d4e3b5700 1 --2- 172.21.15.32:0/2844226291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d48065d90 0x7f5d4813a830 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7f5d400032b0 tx=0x7f5d400091a0).ready entity=mon.0 client_cookie=361593a02cc888ac server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:18.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.163+0000 7f5d4cbb2700 1 -- 172.21.15.32:0/2844226291 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d4000b9a0 con 0x7f5d48065d90 2020-12-05T17:40:18.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.163+0000 7f5d4cbb2700 1 -- 172.21.15.32:0/2844226291 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4000bb00 con 0x7f5d48065d90 2020-12-05T17:40:18.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.163+0000 7f5d4cbb2700 1 -- 172.21.15.32:0/2844226291 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d4000bc70 con 0x7f5d48065d90 2020-12-05T17:40:18.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.163+0000 7f5d4fe18700 1 -- 172.21.15.32:0/2844226291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d48065d90 msgr2=0x7f5d4813a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:18.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.163+0000 7f5d4fe18700 1 --2- 172.21.15.32:0/2844226291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d48065d90 0x7f5d4813a830 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7f5d400032b0 tx=0x7f5d400091a0).stop 2020-12-05T17:40:18.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4fe18700 1 -- 172.21.15.32:0/2844226291 shutdown_connections 2020-12-05T17:40:18.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4fe18700 1 --2- 172.21.15.32:0/2844226291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d48065d90 0x7f5d4813a830 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:18.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4fe18700 1 --2- 172.21.15.32:0/2844226291 >> v2:172.21.15.61:3300/0 conn(0x7f5d4805f6f0 0x7f5d48143d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:18.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4fe18700 1 -- 172.21.15.32:0/2844226291 >> 172.21.15.32:0/2844226291 conn(0x7f5d480b6ef0 msgr2=0x7f5d480b72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:18.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4fe18700 1 -- 172.21.15.32:0/2844226291 shutdown_connections 2020-12-05T17:40:18.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4fe18700 1 -- 172.21.15.32:0/2844226291 wait complete. 2020-12-05T17:40:18.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4fe18700 1 Processor -- start 2020-12-05T17:40:18.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4fe18700 1 -- start start 2020-12-05T17:40:18.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4fe18700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d4805f6f0 0x7f5d48143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:18.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4fe18700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5d48065d90 0x7f5d480afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:18.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4fe18700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d4813a380 0x7f5d480b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:18.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4fe18700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d48147440 con 0x7f5d4805f6f0 2020-12-05T17:40:18.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4fe18700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480645c0 con 0x7f5d4813a380 2020-12-05T17:40:18.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4fe18700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d48145020 con 0x7f5d48065d90 2020-12-05T17:40:18.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4dbb4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d4805f6f0 0x7f5d48143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:18.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4dbb4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d4805f6f0 0x7f5d48143ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39564/0 (socket says 172.21.15.32:39564) 2020-12-05T17:40:18.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4dbb4700 1 -- 172.21.15.32:0/2017293467 learned_addr learned my addr 172.21.15.32:0/2017293467 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:18.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4d3b3700 1 --2- 172.21.15.32:0/2017293467 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5d48065d90 0x7f5d480afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:18.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4e3b5700 1 --2- 172.21.15.32:0/2017293467 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d4813a380 0x7f5d480b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:18.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4dbb4700 1 -- 172.21.15.32:0/2017293467 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5d48065d90 msgr2=0x7f5d480afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:18.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4dbb4700 1 --2- 172.21.15.32:0/2017293467 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5d48065d90 0x7f5d480afb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:18.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4dbb4700 1 -- 172.21.15.32:0/2017293467 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d4813a380 msgr2=0x7f5d480b0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:18.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4dbb4700 1 --2- 172.21.15.32:0/2017293467 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d4813a380 0x7f5d480b0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:18.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4dbb4700 1 -- 172.21.15.32:0/2017293467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d480af8d0 con 0x7f5d4805f6f0 2020-12-05T17:40:18.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4e3b5700 1 --2- 172.21.15.32:0/2017293467 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d4813a380 0x7f5d480b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:40:18.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.167+0000 7f5d4dbb4700 1 --2- 172.21.15.32:0/2017293467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d4805f6f0 0x7f5d48143ab0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7f5d3800c000 tx=0x7f5d38006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:18.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.171+0000 7f5d46ffd700 1 -- 172.21.15.32:0/2017293467 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d3800ca10 con 0x7f5d4805f6f0 2020-12-05T17:40:18.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.171+0000 7f5d46ffd700 1 -- 172.21.15.32:0/2017293467 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3800cb70 con 0x7f5d4805f6f0 2020-12-05T17:40:18.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.171+0000 7f5d4fe18700 1 -- 172.21.15.32:0/2017293467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d480b0f00 con 0x7f5d4805f6f0 2020-12-05T17:40:18.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.171+0000 7f5d46ffd700 1 -- 172.21.15.32:0/2017293467 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d380123f0 con 0x7f5d4805f6f0 2020-12-05T17:40:18.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.171+0000 7f5d4fe18700 1 -- 172.21.15.32:0/2017293467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d48139a40 con 0x7f5d4805f6f0 2020-12-05T17:40:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.171+0000 7f5d4fe18700 1 -- 172.21.15.32:0/2017293467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d48057050 con 0x7f5d4805f6f0 2020-12-05T17:40:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.171+0000 7f5d46ffd700 1 -- 172.21.15.32:0/2017293467 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5d38012550 con 0x7f5d4805f6f0 2020-12-05T17:40:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.171+0000 7f5d46ffd700 1 --2- 172.21.15.32:0/2017293467 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5d280553b0 0x7f5d28057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.171+0000 7f5d46ffd700 1 -- 172.21.15.32:0/2017293467 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5d3806e500 con 0x7f5d4805f6f0 2020-12-05T17:40:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.171+0000 7f5d4d3b3700 1 --2- 172.21.15.32:0/2017293467 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5d280553b0 0x7f5d28057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:18.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.171+0000 7f5d4d3b3700 1 --2- 172.21.15.32:0/2017293467 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5d280553b0 0x7f5d28057870 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f5d3c00d000 tx=0x7f5d3c000b80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:18.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.179+0000 7f5d46ffd700 1 -- 172.21.15.32:0/2017293467 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5d380445a0 con 0x7f5d4805f6f0 2020-12-05T17:40:18.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.359+0000 7f5d4fe18700 1 -- 172.21.15.32:0/2017293467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d48069690 con 0x7f5d4805f6f0 2020-12-05T17:40:18.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.363+0000 7f5d46ffd700 1 -- 172.21.15.32:0/2017293467 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f5d38047bc0 con 0x7f5d4805f6f0 2020-12-05T17:40:18.367 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:40:18.367 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":52,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6548,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2853534360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2853534360},{"type":"v1","addr":"172.21.15.61:6833","nonce":2853534360}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":50}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:40:12.749385+0000","modified":"2020-12-05T17:40:13.752777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7563":{"gid":7563,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/2641840537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2641840537},{"type":"v1","addr":"172.21.15.32:6835","nonce":2641840537}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2020-12-05T17:40:18.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.363+0000 7f5d4fe18700 1 -- 172.21.15.32:0/2017293467 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5d280553b0 msgr2=0x7f5d28057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:18.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.363+0000 7f5d4fe18700 1 --2- 172.21.15.32:0/2017293467 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5d280553b0 0x7f5d28057870 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f5d3c00d000 tx=0x7f5d3c000b80).stop 2020-12-05T17:40:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.363+0000 7f5d4fe18700 1 -- 172.21.15.32:0/2017293467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d4805f6f0 msgr2=0x7f5d48143ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.363+0000 7f5d4fe18700 1 --2- 172.21.15.32:0/2017293467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d4805f6f0 0x7f5d48143ab0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7f5d3800c000 tx=0x7f5d38006dc0).stop 2020-12-05T17:40:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.363+0000 7f5d4fe18700 1 -- 172.21.15.32:0/2017293467 shutdown_connections 2020-12-05T17:40:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.363+0000 7f5d4fe18700 1 --2- 172.21.15.32:0/2017293467 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d4813a380 0x7f5d480b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:18.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.363+0000 7f5d4fe18700 1 --2- 172.21.15.32:0/2017293467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d4805f6f0 0x7f5d48143ab0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:18.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.363+0000 7f5d4fe18700 1 --2- 172.21.15.32:0/2017293467 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5d280553b0 0x7f5d28057870 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:18.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.363+0000 7f5d4fe18700 1 --2- 172.21.15.32:0/2017293467 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5d48065d90 0x7f5d480afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:18.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.363+0000 7f5d4fe18700 1 -- 172.21.15.32:0/2017293467 >> 172.21.15.32:0/2017293467 conn(0x7f5d480b6ef0 msgr2=0x7f5d480ad520 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:18.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.363+0000 7f5d4fe18700 1 -- 172.21.15.32:0/2017293467 shutdown_connections 2020-12-05T17:40:18.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:18.363+0000 7f5d4fe18700 1 -- 172.21.15.32:0/2017293467 wait complete. 2020-12-05T17:40:18.372 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 52 2020-12-05T17:40:18.582 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:40:12.749385+0000', 'modified': '2020-12-05T17:40:13.752777+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7563}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7563': {'gid': 7563, 'name': 'a', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/2641840537', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 2641840537}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 2641840537}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:40:18.582 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:40:18.584 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:40:18.717 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":7563,"want_state":"up:active","state":"up:active","rank_uptime":5.9541650370000001,"mdsmap_epoch":52,"osdmap_epoch":111,"osdmap_epoch_barrier":110,"uptime":11.146616551999999} 2020-12-05T17:40:18.718 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 7563, "mdsmap_epoch": 52, "osdmap_epoch": 111, "osdmap_epoch_barrier": 110, "rank_uptime": 5.954165037, "state": "up:active", "uptime": 11.146616552, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:40:18.718 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:40:18.719 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-12-05T17:40:18.719 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-12-05T17:40:18.720 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:18.720 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:40:18.728 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:40:18.784 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:40:18.785 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:40:18.842 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:40:18.843 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:40:18.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:18.844 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:40:18.890 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:40:18.890 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:40:18.937 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:40:18.953 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:40:18.954 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:40:19.010 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:40:19.011 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:40:19.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:19.012 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:40:19.016 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:2020-12-05T17:40:19.014+0000 7f0282f060c0 -1 init, newargv = 0x555615650620 newargc=9 2020-12-05T17:40:19.016 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[11451]: starting ceph client 2020-12-05T17:40:19.027 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[11451]: starting fuse 2020-12-05T17:40:20.122 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:40:20.139 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:40:20.139 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:40:20.198 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:40:20.199 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:40:20.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:20.201 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:40:20.246 INFO:teuthology.orchestra.run.smithi036.stdout:49 2020-12-05T17:40:20.247 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-12-05T17:40:20.247 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:20.248 INFO:teuthology.orchestra.run.smithi036:> sudo python3 -c ' 2020-12-05T17:40:20.248 INFO:teuthology.orchestra.run.smithi036:> import glob 2020-12-05T17:40:20.249 INFO:teuthology.orchestra.run.smithi036:> import re 2020-12-05T17:40:20.249 INFO:teuthology.orchestra.run.smithi036:> import os 2020-12-05T17:40:20.250 INFO:teuthology.orchestra.run.smithi036:> import subprocess 2020-12-05T17:40:20.250 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:40:20.251 INFO:teuthology.orchestra.run.smithi036:> def find_socket(client_name): 2020-12-05T17:40:20.251 INFO:teuthology.orchestra.run.smithi036:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-12-05T17:40:20.252 INFO:teuthology.orchestra.run.smithi036:> files = glob.glob(asok_path) 2020-12-05T17:40:20.252 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:40:20.253 INFO:teuthology.orchestra.run.smithi036:> # Given a non-glob path, it better be there 2020-12-05T17:40:20.253 INFO:teuthology.orchestra.run.smithi036:> if "*" not in asok_path: 2020-12-05T17:40:20.255 INFO:teuthology.orchestra.run.smithi036:> assert(len(files) == 1) 2020-12-05T17:40:20.255 INFO:teuthology.orchestra.run.smithi036:> return files[0] 2020-12-05T17:40:20.256 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:40:20.256 INFO:teuthology.orchestra.run.smithi036:> for f in files: 2020-12-05T17:40:20.257 INFO:teuthology.orchestra.run.smithi036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:40:20.257 INFO:teuthology.orchestra.run.smithi036:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:40:20.258 INFO:teuthology.orchestra.run.smithi036:> return f 2020-12-05T17:40:20.258 INFO:teuthology.orchestra.run.smithi036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:40:20.259 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:40:20.259 INFO:teuthology.orchestra.run.smithi036:> print(find_socket("client.0")) 2020-12-05T17:40:20.259 INFO:teuthology.orchestra.run.smithi036:> ' 2020-12-05T17:40:20.336 INFO:teuthology.orchestra.run.smithi036.stdout:/var/run/ceph/ceph-client.0.11451.asok 2020-12-05T17:40:20.341 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.11451.asok 2020-12-05T17:40:20.341 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:20.342 INFO:teuthology.orchestra.run.smithi036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.11451.asok status 2020-12-05T17:40:20.441 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2020-12-05T17:40:20.442 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": { 2020-12-05T17:40:20.442 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:40:20.442 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:40:20.442 INFO:teuthology.orchestra.run.smithi036.stdout: "entity_id": "0", 2020-12-05T17:40:20.443 INFO:teuthology.orchestra.run.smithi036.stdout: "hostname": "smithi036", 2020-12-05T17:40:20.443 INFO:teuthology.orchestra.run.smithi036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-12-05T17:40:20.443 INFO:teuthology.orchestra.run.smithi036.stdout: "pid": "11451", 2020-12-05T17:40:20.443 INFO:teuthology.orchestra.run.smithi036.stdout: "root": "/" 2020-12-05T17:40:20.444 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:40:20.444 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_count": 0, 2020-12-05T17:40:20.444 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_pinned_count": 0, 2020-12-05T17:40:20.444 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 7725, 2020-12-05T17:40:20.444 INFO:teuthology.orchestra.run.smithi036.stdout: "inst": { 2020-12-05T17:40:20.445 INFO:teuthology.orchestra.run.smithi036.stdout: "name": { 2020-12-05T17:40:20.445 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "client", 2020-12-05T17:40:20.445 INFO:teuthology.orchestra.run.smithi036.stdout: "num": 7725 2020-12-05T17:40:20.445 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:40:20.445 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:40:20.446 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:40:20.446 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:40:20.446 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 33795449 2020-12-05T17:40:20.446 INFO:teuthology.orchestra.run.smithi036.stdout: } 2020-12-05T17:40:20.447 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:40:20.447 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:40:20.447 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:40:20.447 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:40:20.447 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 33795449 2020-12-05T17:40:20.448 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:40:20.448 INFO:teuthology.orchestra.run.smithi036.stdout: "inst_str": "client.7725 172.21.15.36:0/33795449", 2020-12-05T17:40:20.448 INFO:teuthology.orchestra.run.smithi036.stdout: "addr_str": "172.21.15.36:0/33795449", 2020-12-05T17:40:20.448 INFO:teuthology.orchestra.run.smithi036.stdout: "inode_count": 1, 2020-12-05T17:40:20.448 INFO:teuthology.orchestra.run.smithi036.stdout: "mds_epoch": 52, 2020-12-05T17:40:20.449 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch": 111, 2020-12-05T17:40:20.449 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:40:20.449 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklisted": false 2020-12-05T17:40:20.450 INFO:teuthology.orchestra.run.smithi036.stdout:} 2020-12-05T17:40:20.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:20.453 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:40:20.453 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:40:20.460 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:40:20.461 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:40:20.461 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:20.462 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:40:20.521 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:40:20.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.639+0000 7f2151785700 1 Processor -- start 2020-12-05T17:40:20.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.639+0000 7f2151785700 1 -- start start 2020-12-05T17:40:20.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.639+0000 7f2151785700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f214c0b1fc0 0x7f214c0af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:20.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.639+0000 7f2151785700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f214c147400 0x7f214c145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:20.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.639+0000 7f2151785700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f214c146a90 con 0x7f214c0ab710 2020-12-05T17:40:20.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.639+0000 7f2151785700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f214c061e90 con 0x7f214c147400 2020-12-05T17:40:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.639+0000 7f2151785700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f214c0b0110 con 0x7f214c0b1fc0 2020-12-05T17:40:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.639+0000 7f2150f84700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f214c147400 0x7f214c145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.639+0000 7f2150f84700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f214c147400 0x7f214c145d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39572/0 (socket says 172.21.15.32:39572) 2020-12-05T17:40:20.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.639+0000 7f2150f84700 1 -- 172.21.15.32:0/1560564613 learned_addr learned my addr 172.21.15.32:0/1560564613 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:20.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.639+0000 7f214bfff700 1 --2- 172.21.15.32:0/1560564613 >> v2:172.21.15.61:3300/0 conn(0x7f214c0b1fc0 0x7f214c0af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:20.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.639+0000 7f2150f84700 1 -- 172.21.15.32:0/1560564613 >> v2:172.21.15.61:3300/0 conn(0x7f214c0b1fc0 msgr2=0x7f214c0af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:20.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.639+0000 7f2150f84700 1 --2- 172.21.15.32:0/1560564613 >> v2:172.21.15.61:3300/0 conn(0x7f214c0b1fc0 0x7f214c0af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:20.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.639+0000 7f2150f84700 1 -- 172.21.15.32:0/1560564613 >> v1:172.21.15.61:6789/0 conn(0x7f214c0ab710 legacy=0x7f214c061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:20.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.639+0000 7f2150f84700 1 -- 172.21.15.32:0/1560564613 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214c0b23a0 con 0x7f214c147400 2020-12-05T17:40:20.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.639+0000 7f214bfff700 1 --2- 172.21.15.32:0/1560564613 >> v2:172.21.15.61:3300/0 conn(0x7f214c0b1fc0 0x7f214c0af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:40:20.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.639+0000 7f2150f84700 1 --2- 172.21.15.32:0/1560564613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f214c147400 0x7f214c145d10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7f2144002f30 tx=0x7f2144006e80).ready entity=mon.0 client_cookie=f9ba77cd1b389c33 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:20.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.639+0000 7f214affd700 1 -- 172.21.15.32:0/1560564613 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21440097f0 con 0x7f214c147400 2020-12-05T17:40:20.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.639+0000 7f214affd700 1 -- 172.21.15.32:0/1560564613 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2144009950 con 0x7f214c147400 2020-12-05T17:40:20.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.639+0000 7f214affd700 1 -- 172.21.15.32:0/1560564613 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2144009cb0 con 0x7f214c147400 2020-12-05T17:40:20.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.639+0000 7f2151785700 1 -- 172.21.15.32:0/1560564613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f214c147400 msgr2=0x7f214c145d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:20.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.639+0000 7f2151785700 1 --2- 172.21.15.32:0/1560564613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f214c147400 0x7f214c145d10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7f2144002f30 tx=0x7f2144006e80).stop 2020-12-05T17:40:20.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2151785700 1 -- 172.21.15.32:0/1560564613 shutdown_connections 2020-12-05T17:40:20.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2151785700 1 --2- 172.21.15.32:0/1560564613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f214c147400 0x7f214c145d10 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:20.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2151785700 1 --2- 172.21.15.32:0/1560564613 >> v2:172.21.15.61:3300/0 conn(0x7f214c0b1fc0 0x7f214c0af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:20.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2151785700 1 -- 172.21.15.32:0/1560564613 >> 172.21.15.32:0/1560564613 conn(0x7f214c0b5530 msgr2=0x7f214c0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:20.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2151785700 1 -- 172.21.15.32:0/1560564613 shutdown_connections 2020-12-05T17:40:20.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2151785700 1 -- 172.21.15.32:0/1560564613 wait complete. 2020-12-05T17:40:20.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2151785700 1 Processor -- start 2020-12-05T17:40:20.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2151785700 1 -- start start 2020-12-05T17:40:20.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2151785700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f214c0ab710 0x7f214c0acf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:20.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2151785700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f214c0b1fc0 0x7f214c0a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:20.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2151785700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f214c147400 0x7f214c0a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:20.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2151785700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214c0ad860 con 0x7f214c147400 2020-12-05T17:40:20.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2151785700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214c0a1fc0 con 0x7f214c0ab710 2020-12-05T17:40:20.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2151785700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f214c0ac790 con 0x7f214c0b1fc0 2020-12-05T17:40:20.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2150f84700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f214c147400 0x7f214c0a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:20.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2150f84700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f214c147400 0x7f214c0a1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39578/0 (socket says 172.21.15.32:39578) 2020-12-05T17:40:20.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2150f84700 1 -- 172.21.15.32:0/4129171002 learned_addr learned my addr 172.21.15.32:0/4129171002 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:20.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f214bfff700 1 --2- 172.21.15.32:0/4129171002 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f214c0ab710 0x7f214c0acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:20.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f214b7fe700 1 --2- 172.21.15.32:0/4129171002 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f214c0b1fc0 0x7f214c0a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:20.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2150f84700 1 -- 172.21.15.32:0/4129171002 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f214c0b1fc0 msgr2=0x7f214c0a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:20.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2150f84700 1 --2- 172.21.15.32:0/4129171002 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f214c0b1fc0 0x7f214c0a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:20.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2150f84700 1 -- 172.21.15.32:0/4129171002 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f214c0ab710 msgr2=0x7f214c0acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:20.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2150f84700 1 --2- 172.21.15.32:0/4129171002 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f214c0ab710 0x7f214c0acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:20.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2150f84700 1 -- 172.21.15.32:0/4129171002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214c09efd0 con 0x7f214c147400 2020-12-05T17:40:20.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2150f84700 1 --2- 172.21.15.32:0/4129171002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f214c147400 0x7f214c0a1ad0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7f2144002f30 tx=0x7f2144015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:20.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f21497fa700 1 -- 172.21.15.32:0/4129171002 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2144011bf0 con 0x7f214c147400 2020-12-05T17:40:20.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f21497fa700 1 -- 172.21.15.32:0/4129171002 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2144011d50 con 0x7f214c147400 2020-12-05T17:40:20.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2151785700 1 -- 172.21.15.32:0/4129171002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f214c0ad420 con 0x7f214c147400 2020-12-05T17:40:20.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f21497fa700 1 -- 172.21.15.32:0/4129171002 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f214401e650 con 0x7f214c147400 2020-12-05T17:40:20.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.643+0000 7f2151785700 1 -- 172.21.15.32:0/4129171002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f214c14daf0 con 0x7f214c147400 2020-12-05T17:40:20.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.647+0000 7f2151785700 1 -- 172.21.15.32:0/4129171002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f214c0ad6d0 con 0x7f214c147400 2020-12-05T17:40:20.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.647+0000 7f21497fa700 1 -- 172.21.15.32:0/4129171002 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2144025030 con 0x7f214c147400 2020-12-05T17:40:20.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.647+0000 7f21497fa700 1 --2- 172.21.15.32:0/4129171002 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f21300553b0 0x7f2130057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:20.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.647+0000 7f21497fa700 1 -- 172.21.15.32:0/4129171002 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f214406f310 con 0x7f214c147400 2020-12-05T17:40:20.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.647+0000 7f214bfff700 1 --2- 172.21.15.32:0/4129171002 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f21300553b0 0x7f2130057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:20.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.647+0000 7f214bfff700 1 --2- 172.21.15.32:0/4129171002 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f21300553b0 0x7f2130057870 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f213400c000 tx=0x7f2134008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:20.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.655+0000 7f21497fa700 1 -- 172.21.15.32:0/4129171002 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f21440454f0 con 0x7f214c147400 2020-12-05T17:40:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.795+0000 7f2151785700 1 -- 172.21.15.32:0/4129171002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f214c057050 con 0x7f214c147400 2020-12-05T17:40:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.799+0000 7f21497fa700 1 -- 172.21.15.32:0/4129171002 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f2144023090 con 0x7f214c147400 2020-12-05T17:40:20.804 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:40:20.804 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":111,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:40:13.752899+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:40:10.513416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2020-12-05T17:40:11.768907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:40:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.799+0000 7f2151785700 1 -- 172.21.15.32:0/4129171002 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f21300553b0 msgr2=0x7f2130057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:20.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.799+0000 7f2151785700 1 --2- 172.21.15.32:0/4129171002 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f21300553b0 0x7f2130057870 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f213400c000 tx=0x7f2134008040).stop 2020-12-05T17:40:20.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.799+0000 7f2151785700 1 -- 172.21.15.32:0/4129171002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f214c147400 msgr2=0x7f214c0a1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:20.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.799+0000 7f2151785700 1 --2- 172.21.15.32:0/4129171002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f214c147400 0x7f214c0a1ad0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7f2144002f30 tx=0x7f2144015000).stop 2020-12-05T17:40:20.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.799+0000 7f2151785700 1 -- 172.21.15.32:0/4129171002 shutdown_connections 2020-12-05T17:40:20.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.799+0000 7f2151785700 1 --2- 172.21.15.32:0/4129171002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f214c147400 0x7f214c0a1ad0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:20.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.799+0000 7f2151785700 1 --2- 172.21.15.32:0/4129171002 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f214c0ab710 0x7f214c0acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:20.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.799+0000 7f2151785700 1 --2- 172.21.15.32:0/4129171002 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f21300553b0 0x7f2130057870 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:20.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.799+0000 7f2151785700 1 --2- 172.21.15.32:0/4129171002 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f214c0b1fc0 0x7f214c0a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:20.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.799+0000 7f2151785700 1 -- 172.21.15.32:0/4129171002 >> 172.21.15.32:0/4129171002 conn(0x7f214c0b5530 msgr2=0x7f214c0af740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:20.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.799+0000 7f2151785700 1 -- 172.21.15.32:0/4129171002 shutdown_connections 2020-12-05T17:40:20.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.799+0000 7f2151785700 1 -- 172.21.15.32:0/4129171002 wait complete. 2020-12-05T17:40:20.826 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:40:10.513416+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:40:11.768907+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:40:20.827 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:40:20.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.943+0000 7f312cfe9700 1 Processor -- start 2020-12-05T17:40:20.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.943+0000 7f312cfe9700 1 -- start start 2020-12-05T17:40:20.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.943+0000 7f312cfe9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31280b5b20 0x7f31280b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:20.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.943+0000 7f312cfe9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f31280b6690 0x7f31280b5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:20.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.943+0000 7f312cfe9700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31280b1920 con 0x7f31280b2060 2020-12-05T17:40:20.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.943+0000 7f312cfe9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31280b5730 con 0x7f31280b6690 2020-12-05T17:40:20.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.943+0000 7f312cfe9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31280b1bc0 con 0x7f31280b5b20 2020-12-05T17:40:20.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.943+0000 7f3125d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f31280b6690 0x7f31280b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:20.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.943+0000 7f3125d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f31280b6690 0x7f31280b5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39588/0 (socket says 172.21.15.32:39588) 2020-12-05T17:40:20.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.943+0000 7f3125d9b700 1 -- 172.21.15.32:0/729223239 learned_addr learned my addr 172.21.15.32:0/729223239 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:20.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.943+0000 7f3126d9d700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f31280b2060 0x7f3128145f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:54874/0 (socket says 172.21.15.32:54874) 2020-12-05T17:40:20.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.943+0000 7f312659c700 1 --2- 172.21.15.32:0/729223239 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31280b5b20 0x7f31280b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:20.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.943+0000 7f3125d9b700 1 -- 172.21.15.32:0/729223239 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31280b5b20 msgr2=0x7f31280b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:20.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.943+0000 7f3125d9b700 1 --2- 172.21.15.32:0/729223239 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31280b5b20 0x7f31280b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:20.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.943+0000 7f3125d9b700 1 -- 172.21.15.32:0/729223239 >> v1:172.21.15.61:6789/0 conn(0x7f31280b2060 legacy=0x7f3128145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:20.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.943+0000 7f3125d9b700 1 -- 172.21.15.32:0/729223239 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31280b24c0 con 0x7f31280b6690 2020-12-05T17:40:20.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.943+0000 7f3125d9b700 1 --2- 172.21.15.32:0/729223239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31280b6690 0x7f31280b5240 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0x7f311c002f30 tx=0x7f311c006e80).ready entity=mon.0 client_cookie=6cab254f7d922f31 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:20.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.947+0000 7f312559a700 1 -- 172.21.15.32:0/729223239 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f311c0097f0 con 0x7f31280b6690 2020-12-05T17:40:20.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.947+0000 7f312559a700 1 -- 172.21.15.32:0/729223239 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c009950 con 0x7f31280b6690 2020-12-05T17:40:20.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.947+0000 7f312559a700 1 -- 172.21.15.32:0/729223239 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f311c009cb0 con 0x7f31280b6690 2020-12-05T17:40:20.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.947+0000 7f312cfe9700 1 -- 172.21.15.32:0/729223239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31280b6690 msgr2=0x7f31280b5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:20.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.947+0000 7f312cfe9700 1 --2- 172.21.15.32:0/729223239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31280b6690 0x7f31280b5240 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0x7f311c002f30 tx=0x7f311c006e80).stop 2020-12-05T17:40:20.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.947+0000 7f312cfe9700 1 -- 172.21.15.32:0/729223239 shutdown_connections 2020-12-05T17:40:20.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.947+0000 7f312cfe9700 1 --2- 172.21.15.32:0/729223239 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31280b5b20 0x7f31280b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:20.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.947+0000 7f312cfe9700 1 --2- 172.21.15.32:0/729223239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31280b6690 0x7f31280b5240 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:20.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.947+0000 7f312cfe9700 1 -- 172.21.15.32:0/729223239 >> 172.21.15.32:0/729223239 conn(0x7f312805f3b0 msgr2=0x7f312805f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:20.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.947+0000 7f312cfe9700 1 -- 172.21.15.32:0/729223239 shutdown_connections 2020-12-05T17:40:20.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.947+0000 7f312cfe9700 1 -- 172.21.15.32:0/729223239 wait complete. 2020-12-05T17:40:20.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.947+0000 7f312cfe9700 1 Processor -- start 2020-12-05T17:40:20.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.947+0000 7f312cfe9700 1 -- start start 2020-12-05T17:40:20.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.951+0000 7f312cfe9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31280b2060 0x7f31281448f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:20.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.951+0000 7f312cfe9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f31280b5b20 0x7f31281464a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:20.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.951+0000 7f312cfe9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31280b6690 0x7f31280ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:20.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.951+0000 7f312cfe9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3128144de0 con 0x7f31280b6690 2020-12-05T17:40:20.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.951+0000 7f312cfe9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3128145150 con 0x7f31280b5b20 2020-12-05T17:40:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.951+0000 7f312cfe9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3128145c40 con 0x7f31280b2060 2020-12-05T17:40:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.951+0000 7f312659c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31280b2060 0x7f31281448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.951+0000 7f3126d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31280b6690 0x7f31280ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.951+0000 7f312659c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31280b2060 0x7f31281448f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:36380/0 (socket says 172.21.15.32:36380) 2020-12-05T17:40:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.951+0000 7f312659c700 1 -- 172.21.15.32:0/1934125776 learned_addr learned my addr 172.21.15.32:0/1934125776 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.951+0000 7f3125d9b700 1 --2- 172.21.15.32:0/1934125776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f31280b5b20 0x7f31281464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.951+0000 7f312659c700 1 -- 172.21.15.32:0/1934125776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f31280b5b20 msgr2=0x7f31281464a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.951+0000 7f312659c700 1 --2- 172.21.15.32:0/1934125776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f31280b5b20 0x7f31281464a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.951+0000 7f312659c700 1 -- 172.21.15.32:0/1934125776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31280b6690 msgr2=0x7f31280ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.951+0000 7f312659c700 1 --2- 172.21.15.32:0/1934125776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31280b6690 0x7f31280ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.951+0000 7f312659c700 1 -- 172.21.15.32:0/1934125776 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3128145ee0 con 0x7f31280b2060 2020-12-05T17:40:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.951+0000 7f3126d9d700 1 --2- 172.21.15.32:0/1934125776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31280b6690 0x7f31280ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:40:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.951+0000 7f312659c700 1 --2- 172.21.15.32:0/1934125776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31280b2060 0x7f31281448f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f311800c000 tx=0x7f3118006dc0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.951+0000 7f31177fe700 1 -- 172.21.15.32:0/1934125776 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f311800ca10 con 0x7f31280b2060 2020-12-05T17:40:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.951+0000 7f312cfe9700 1 -- 172.21.15.32:0/1934125776 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3128146990 con 0x7f31280b2060 2020-12-05T17:40:20.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.951+0000 7f31177fe700 1 -- 172.21.15.32:0/1934125776 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311800cb70 con 0x7f31280b2060 2020-12-05T17:40:20.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.951+0000 7f31177fe700 1 -- 172.21.15.32:0/1934125776 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f311801c460 con 0x7f31280b2060 2020-12-05T17:40:20.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.951+0000 7f312cfe9700 1 -- 172.21.15.32:0/1934125776 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3128066ce0 con 0x7f31280b2060 2020-12-05T17:40:20.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.955+0000 7f312cfe9700 1 -- 172.21.15.32:0/1934125776 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31280af5e0 con 0x7f31280b2060 2020-12-05T17:40:20.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.955+0000 7f31177fe700 1 -- 172.21.15.32:0/1934125776 <== mon.2 v2:172.21.15.61:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f31180123f0 con 0x7f31280b2060 2020-12-05T17:40:20.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.955+0000 7f31177fe700 1 --2- 172.21.15.32:0/1934125776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3104055330 0x7f31040577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:20.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.955+0000 7f31177fe700 1 -- 172.21.15.32:0/1934125776 <== mon.2 v2:172.21.15.61:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f311806e5a0 con 0x7f31280b2060 2020-12-05T17:40:20.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.955+0000 7f3125d9b700 1 --2- 172.21.15.32:0/1934125776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3104055330 0x7f31040577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:20.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.955+0000 7f3125d9b700 1 --2- 172.21.15.32:0/1934125776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3104055330 0x7f31040577f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f311c01c000 tx=0x7f311c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:20.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:20.963+0000 7f31177fe700 1 -- 172.21.15.32:0/1934125776 <== mon.2 v2:172.21.15.61:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f31180446f0 con 0x7f31280b2060 2020-12-05T17:40:21.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.103+0000 7f312cfe9700 1 -- 172.21.15.32:0/1934125776 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f3128057050 con 0x7f31280b2060 2020-12-05T17:40:21.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.107+0000 7f31177fe700 1 -- 172.21.15.32:0/1934125776 <== mon.2 v2:172.21.15.61:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v111) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3118047d10 con 0x7f31280b2060 2020-12-05T17:40:21.111 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:40:21.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.107+0000 7f312cfe9700 1 -- 172.21.15.32:0/1934125776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3104055330 msgr2=0x7f31040577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:21.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.107+0000 7f312cfe9700 1 --2- 172.21.15.32:0/1934125776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3104055330 0x7f31040577f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f311c01c000 tx=0x7f311c015000).stop 2020-12-05T17:40:21.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.107+0000 7f312cfe9700 1 -- 172.21.15.32:0/1934125776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31280b2060 msgr2=0x7f31281448f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:21.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.107+0000 7f312cfe9700 1 --2- 172.21.15.32:0/1934125776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31280b2060 0x7f31281448f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f311800c000 tx=0x7f3118006dc0).stop 2020-12-05T17:40:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.107+0000 7f312cfe9700 1 -- 172.21.15.32:0/1934125776 shutdown_connections 2020-12-05T17:40:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.107+0000 7f312cfe9700 1 --2- 172.21.15.32:0/1934125776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31280b6690 0x7f31280ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.107+0000 7f312cfe9700 1 --2- 172.21.15.32:0/1934125776 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3104055330 0x7f31040577f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.107+0000 7f312cfe9700 1 --2- 172.21.15.32:0/1934125776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31280b2060 0x7f31281448f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.107+0000 7f312cfe9700 1 --2- 172.21.15.32:0/1934125776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f31280b5b20 0x7f31281464a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.107+0000 7f312cfe9700 1 -- 172.21.15.32:0/1934125776 >> 172.21.15.32:0/1934125776 conn(0x7f312805f3b0 msgr2=0x7f31280b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:21.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.107+0000 7f312cfe9700 1 -- 172.21.15.32:0/1934125776 shutdown_connections 2020-12-05T17:40:21.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.107+0000 7f312cfe9700 1 -- 172.21.15.32:0/1934125776 wait complete. 2020-12-05T17:40:21.130 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:40:21.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.251+0000 7f5397823700 1 Processor -- start 2020-12-05T17:40:21.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.251+0000 7f5397823700 1 -- start start 2020-12-05T17:40:21.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.251+0000 7f5397823700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f53900b5b20 0x7f53900b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:21.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.251+0000 7f5397823700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53900b6690 0x7f53900b5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:21.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.251+0000 7f5397823700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53900b1920 con 0x7f53900b2060 2020-12-05T17:40:21.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.251+0000 7f5397823700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53900b5730 con 0x7f53900b5b20 2020-12-05T17:40:21.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.251+0000 7f5397823700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53900b1bc0 con 0x7f53900b6690 2020-12-05T17:40:21.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.251+0000 7f53955bf700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f53900b5b20 0x7f53900b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:21.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.251+0000 7f5395dc0700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f53900b2060 0x7f5390145f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:54888/0 (socket says 172.21.15.32:54888) 2020-12-05T17:40:21.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.251+0000 7f5395dc0700 1 -- 172.21.15.32:0/1172075045 learned_addr learned my addr 172.21.15.32:0/1172075045 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:21.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.251+0000 7f5394dbe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53900b6690 0x7f53900b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:21.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.251+0000 7f53955bf700 1 -- 172.21.15.32:0/1172075045 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53900b6690 msgr2=0x7f53900b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:21.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.251+0000 7f53955bf700 1 --2- 172.21.15.32:0/1172075045 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53900b6690 0x7f53900b5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.251+0000 7f53955bf700 1 -- 172.21.15.32:0/1172075045 >> v1:172.21.15.61:6789/0 conn(0x7f53900b2060 legacy=0x7f5390145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:21.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.251+0000 7f5387fff700 1 -- 172.21.15.32:0/1172075045 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157342556 0 0) 0x7f53900b1920 con 0x7f53900b2060 2020-12-05T17:40:21.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.251+0000 7f53955bf700 1 -- 172.21.15.32:0/1172075045 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53900b24c0 con 0x7f53900b5b20 2020-12-05T17:40:21.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.251+0000 7f53955bf700 1 --2- 172.21.15.32:0/1172075045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53900b5b20 0x7f53900b5f00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f538c002f30 tx=0x7f538c006dc0).ready entity=mon.1 client_cookie=e45a8a9d571e24c0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:21.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.251+0000 7f5387fff700 1 -- 172.21.15.32:0/1172075045 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f538c009760 con 0x7f53900b5b20 2020-12-05T17:40:21.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.251+0000 7f5387fff700 1 -- 172.21.15.32:0/1172075045 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538c0098c0 con 0x7f53900b5b20 2020-12-05T17:40:21.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.251+0000 7f5387fff700 1 -- 172.21.15.32:0/1172075045 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f538c009c20 con 0x7f53900b5b20 2020-12-05T17:40:21.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.251+0000 7f5397823700 1 -- 172.21.15.32:0/1172075045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53900b5b20 msgr2=0x7f53900b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:21.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.251+0000 7f5397823700 1 --2- 172.21.15.32:0/1172075045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53900b5b20 0x7f53900b5f00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f538c002f30 tx=0x7f538c006dc0).stop 2020-12-05T17:40:21.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.255+0000 7f5397823700 1 -- 172.21.15.32:0/1172075045 shutdown_connections 2020-12-05T17:40:21.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.255+0000 7f5397823700 1 --2- 172.21.15.32:0/1172075045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53900b5b20 0x7f53900b5f00 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.255+0000 7f5397823700 1 --2- 172.21.15.32:0/1172075045 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53900b6690 0x7f53900b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.255+0000 7f5397823700 1 -- 172.21.15.32:0/1172075045 >> 172.21.15.32:0/1172075045 conn(0x7f539005f3b0 msgr2=0x7f539005f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:21.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.255+0000 7f5397823700 1 -- 172.21.15.32:0/1172075045 shutdown_connections 2020-12-05T17:40:21.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.255+0000 7f5397823700 1 -- 172.21.15.32:0/1172075045 wait complete. 2020-12-05T17:40:21.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.255+0000 7f5397823700 1 Processor -- start 2020-12-05T17:40:21.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.255+0000 7f5397823700 1 -- start start 2020-12-05T17:40:21.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.255+0000 7f5397823700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53900b2060 0x7f53901445c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:21.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.255+0000 7f5397823700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53900b5b20 0x7f5390142550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:21.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.255+0000 7f5397823700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53900b6690 0x7f5390142a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:21.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.255+0000 7f5397823700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53900a4d30 con 0x7f53900b5b20 2020-12-05T17:40:21.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.255+0000 7f5397823700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53900a4a90 con 0x7f53900b2060 2020-12-05T17:40:21.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.255+0000 7f5397823700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53900a27a0 con 0x7f53900b6690 2020-12-05T17:40:21.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.255+0000 7f5394dbe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53900b5b20 0x7f5390142550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:21.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.255+0000 7f5395dc0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53900b6690 0x7f5390142a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:21.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.255+0000 7f5394dbe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53900b5b20 0x7f5390142550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39606/0 (socket says 172.21.15.32:39606) 2020-12-05T17:40:21.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.255+0000 7f5394dbe700 1 -- 172.21.15.32:0/3082731161 learned_addr learned my addr 172.21.15.32:0/3082731161 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:21.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.255+0000 7f53955bf700 1 --2- 172.21.15.32:0/3082731161 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53900b2060 0x7f53901445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:21.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.259+0000 7f5394dbe700 1 -- 172.21.15.32:0/3082731161 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53900b6690 msgr2=0x7f5390142a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:21.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.259+0000 7f5394dbe700 1 --2- 172.21.15.32:0/3082731161 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53900b6690 0x7f5390142a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.259+0000 7f5394dbe700 1 -- 172.21.15.32:0/3082731161 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53900b2060 msgr2=0x7f53901445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:21.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.259+0000 7f5394dbe700 1 --2- 172.21.15.32:0/3082731161 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53900b2060 0x7f53901445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.259+0000 7f5394dbe700 1 -- 172.21.15.32:0/3082731161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5390145170 con 0x7f53900b5b20 2020-12-05T17:40:21.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.259+0000 7f5394dbe700 1 --2- 172.21.15.32:0/3082731161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53900b5b20 0x7f5390142550 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7f538000c000 tx=0x7f5380006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:21.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.259+0000 7f53867fc700 1 -- 172.21.15.32:0/3082731161 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f538000ca10 con 0x7f53900b5b20 2020-12-05T17:40:21.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.259+0000 7f53867fc700 1 -- 172.21.15.32:0/3082731161 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538000cb70 con 0x7f53900b5b20 2020-12-05T17:40:21.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.259+0000 7f5397823700 1 -- 172.21.15.32:0/3082731161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5390142f30 con 0x7f53900b5b20 2020-12-05T17:40:21.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.259+0000 7f53867fc700 1 -- 172.21.15.32:0/3082731161 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f538000b2d0 con 0x7f53900b5b20 2020-12-05T17:40:21.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.259+0000 7f5397823700 1 -- 172.21.15.32:0/3082731161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53900a2c40 con 0x7f53900b5b20 2020-12-05T17:40:21.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.259+0000 7f5397823700 1 -- 172.21.15.32:0/3082731161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53900a48f0 con 0x7f53900b5b20 2020-12-05T17:40:21.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.259+0000 7f53867fc700 1 -- 172.21.15.32:0/3082731161 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f538001b400 con 0x7f53900b5b20 2020-12-05T17:40:21.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.259+0000 7f53867fc700 1 --2- 172.21.15.32:0/3082731161 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f53780554e0 0x7f53780579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:21.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.259+0000 7f53867fc700 1 -- 172.21.15.32:0/3082731161 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f538006e0d0 con 0x7f53900b5b20 2020-12-05T17:40:21.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.259+0000 7f53955bf700 1 --2- 172.21.15.32:0/3082731161 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f53780554e0 0x7f53780579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:21.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.263+0000 7f53955bf700 1 --2- 172.21.15.32:0/3082731161 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f53780554e0 0x7f53780579a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f538c01a000 tx=0x7f538c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.267+0000 7f53867fc700 1 -- 172.21.15.32:0/3082731161 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f53900a48f0 con 0x7f53900b5b20 2020-12-05T17:40:21.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.411+0000 7f5397823700 1 -- 172.21.15.32:0/3082731161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f53900a48f0 con 0x7f53900b5b20 2020-12-05T17:40:21.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.411+0000 7f53867fc700 1 -- 172.21.15.32:0/3082731161 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v111) v1 ==== 92+0+10 (secure 0 0 0) 0x7f53900a48f0 con 0x7f53900b5b20 2020-12-05T17:40:21.418 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:40:21.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.415+0000 7f5397823700 1 -- 172.21.15.32:0/3082731161 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f53780554e0 msgr2=0x7f53780579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:21.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.415+0000 7f5397823700 1 --2- 172.21.15.32:0/3082731161 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f53780554e0 0x7f53780579a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f538c01a000 tx=0x7f538c015000).stop 2020-12-05T17:40:21.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.415+0000 7f5397823700 1 -- 172.21.15.32:0/3082731161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53900b5b20 msgr2=0x7f5390142550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:21.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.415+0000 7f5397823700 1 --2- 172.21.15.32:0/3082731161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53900b5b20 0x7f5390142550 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7f538000c000 tx=0x7f5380006dc0).stop 2020-12-05T17:40:21.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.415+0000 7f5397823700 1 -- 172.21.15.32:0/3082731161 shutdown_connections 2020-12-05T17:40:21.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.415+0000 7f5397823700 1 --2- 172.21.15.32:0/3082731161 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f53780554e0 0x7f53780579a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.415+0000 7f5397823700 1 --2- 172.21.15.32:0/3082731161 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53900b6690 0x7f5390142a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.415+0000 7f5397823700 1 --2- 172.21.15.32:0/3082731161 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53900b2060 0x7f53901445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.415+0000 7f5397823700 1 --2- 172.21.15.32:0/3082731161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53900b5b20 0x7f5390142550 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.415+0000 7f5397823700 1 -- 172.21.15.32:0/3082731161 >> 172.21.15.32:0/3082731161 conn(0x7f539005f3b0 msgr2=0x7f539005f790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:21.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.415+0000 7f5397823700 1 -- 172.21.15.32:0/3082731161 shutdown_connections 2020-12-05T17:40:21.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.415+0000 7f5397823700 1 -- 172.21.15.32:0/3082731161 wait complete. 2020-12-05T17:40:21.436 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:40:21.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.551+0000 7fa38b46f700 1 Processor -- start 2020-12-05T17:40:21.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.551+0000 7fa38b46f700 1 -- start start 2020-12-05T17:40:21.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.551+0000 7fa38b46f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa3840b2000 0x7fa384142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:21.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.551+0000 7fa38b46f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa384143410 0x7fa38413f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:21.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.551+0000 7fa38b46f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa38413fb00 0x7fa3840af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:21.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.551+0000 7fa38b46f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3840aeba0 con 0x7fa38413fb00 2020-12-05T17:40:21.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.551+0000 7fa38b46f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3840b23e0 con 0x7fa3840b2000 2020-12-05T17:40:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.551+0000 7fa38b46f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3840af190 con 0x7fa384143410 2020-12-05T17:40:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.551+0000 7fa38ac6e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa38413fb00 0x7fa3840af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.551+0000 7fa38ac6e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa38413fb00 0x7fa3840af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39616/0 (socket says 172.21.15.32:39616) 2020-12-05T17:40:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.551+0000 7fa38ac6e700 1 -- 172.21.15.32:0/351298309 learned_addr learned my addr 172.21.15.32:0/351298309 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.551+0000 7fa389c6c700 1 --2- 172.21.15.32:0/351298309 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa384143410 0x7fa38413f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.551+0000 7fa38a46d700 1 --2- 172.21.15.32:0/351298309 >> v2:172.21.15.61:3300/0 conn(0x7fa3840b2000 0x7fa384142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.551+0000 7fa38ac6e700 1 -- 172.21.15.32:0/351298309 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa384143410 msgr2=0x7fa38413f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.551+0000 7fa38ac6e700 1 --2- 172.21.15.32:0/351298309 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa384143410 0x7fa38413f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.551+0000 7fa38ac6e700 1 -- 172.21.15.32:0/351298309 >> v2:172.21.15.61:3300/0 conn(0x7fa3840b2000 msgr2=0x7fa384142f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.551+0000 7fa38ac6e700 1 --2- 172.21.15.32:0/351298309 >> v2:172.21.15.61:3300/0 conn(0x7fa3840b2000 0x7fa384142f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.551+0000 7fa38ac6e700 1 -- 172.21.15.32:0/351298309 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3840ae010 con 0x7fa38413fb00 2020-12-05T17:40:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.555+0000 7fa38ac6e700 1 --2- 172.21.15.32:0/351298309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa38413fb00 0x7fa3840af4b0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7fa37c002f30 tx=0x7fa37c006e80).ready entity=mon.0 client_cookie=474281c321a19b0f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.555+0000 7fa38946b700 1 -- 172.21.15.32:0/351298309 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa37c009640 con 0x7fa38413fb00 2020-12-05T17:40:21.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.555+0000 7fa38946b700 1 -- 172.21.15.32:0/351298309 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37c0097a0 con 0x7fa38413fb00 2020-12-05T17:40:21.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.555+0000 7fa38946b700 1 -- 172.21.15.32:0/351298309 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa37c009b00 con 0x7fa38413fb00 2020-12-05T17:40:21.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.555+0000 7fa38b46f700 1 -- 172.21.15.32:0/351298309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa38413fb00 msgr2=0x7fa3840af4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:21.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.555+0000 7fa38b46f700 1 --2- 172.21.15.32:0/351298309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa38413fb00 0x7fa3840af4b0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7fa37c002f30 tx=0x7fa37c006e80).stop 2020-12-05T17:40:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.555+0000 7fa38b46f700 1 -- 172.21.15.32:0/351298309 shutdown_connections 2020-12-05T17:40:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.555+0000 7fa38b46f700 1 --2- 172.21.15.32:0/351298309 >> v2:172.21.15.61:3300/0 conn(0x7fa3840b2000 0x7fa384142f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.555+0000 7fa38b46f700 1 --2- 172.21.15.32:0/351298309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa38413fb00 0x7fa3840af4b0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.555+0000 7fa38b46f700 1 --2- 172.21.15.32:0/351298309 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa384143410 0x7fa38413f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.555+0000 7fa38b46f700 1 -- 172.21.15.32:0/351298309 >> 172.21.15.32:0/351298309 conn(0x7fa3840b5620 msgr2=0x7fa3840b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:21.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.555+0000 7fa38b46f700 1 -- 172.21.15.32:0/351298309 shutdown_connections 2020-12-05T17:40:21.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.555+0000 7fa38b46f700 1 -- 172.21.15.32:0/351298309 wait complete. 2020-12-05T17:40:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.555+0000 7fa38b46f700 1 Processor -- start 2020-12-05T17:40:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.555+0000 7fa38b46f700 1 -- start start 2020-12-05T17:40:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa38b46f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa3840b2000 0x7fa384149000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa38b46f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa38413fb00 0x7fa38413c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:21.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa38b46f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa384143410 0x7fa38413cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:21.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa38b46f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3840640e0 con 0x7fa38413fb00 2020-12-05T17:40:21.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa38b46f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa384064380 con 0x7fa3840b2000 2020-12-05T17:40:21.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa38b46f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa384064620 con 0x7fa384143410 2020-12-05T17:40:21.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa389c6c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa38413fb00 0x7fa38413c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:21.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa389c6c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa38413fb00 0x7fa38413c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39620/0 (socket says 172.21.15.32:39620) 2020-12-05T17:40:21.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa389c6c700 1 -- 172.21.15.32:0/339447252 learned_addr learned my addr 172.21.15.32:0/339447252 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:21.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa38a46d700 1 --2- 172.21.15.32:0/339447252 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa3840b2000 0x7fa384149000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:21.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa38ac6e700 1 --2- 172.21.15.32:0/339447252 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa384143410 0x7fa38413cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:21.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa389c6c700 1 -- 172.21.15.32:0/339447252 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa384143410 msgr2=0x7fa38413cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:21.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa389c6c700 1 --2- 172.21.15.32:0/339447252 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa384143410 0x7fa38413cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa389c6c700 1 -- 172.21.15.32:0/339447252 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa3840b2000 msgr2=0x7fa384149000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:21.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa389c6c700 1 --2- 172.21.15.32:0/339447252 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa3840b2000 0x7fa384149000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa389c6c700 1 -- 172.21.15.32:0/339447252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa384064ac0 con 0x7fa38413fb00 2020-12-05T17:40:21.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa389c6c700 1 --2- 172.21.15.32:0/339447252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa38413fb00 0x7fa38413c5e0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7fa37400c000 tx=0x7fa374006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:21.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa3737fe700 1 -- 172.21.15.32:0/339447252 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa37400ca10 con 0x7fa38413fb00 2020-12-05T17:40:21.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa3737fe700 1 -- 172.21.15.32:0/339447252 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37400cb70 con 0x7fa38413fb00 2020-12-05T17:40:21.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa38b46f700 1 -- 172.21.15.32:0/339447252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa384147b90 con 0x7fa38413fb00 2020-12-05T17:40:21.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa3737fe700 1 -- 172.21.15.32:0/339447252 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa37400be50 con 0x7fa38413fb00 2020-12-05T17:40:21.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa38b46f700 1 -- 172.21.15.32:0/339447252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa384063d60 con 0x7fa38413fb00 2020-12-05T17:40:21.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa38b46f700 1 -- 172.21.15.32:0/339447252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa384063f50 con 0x7fa38413fb00 2020-12-05T17:40:21.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa3737fe700 1 -- 172.21.15.32:0/339447252 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa37401b410 con 0x7fa38413fb00 2020-12-05T17:40:21.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa3737fe700 1 --2- 172.21.15.32:0/339447252 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa364055440 0x7fa364057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:21.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa3737fe700 1 -- 172.21.15.32:0/339447252 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa37406e120 con 0x7fa38413fb00 2020-12-05T17:40:21.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.559+0000 7fa38a46d700 1 --2- 172.21.15.32:0/339447252 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa364055440 0x7fa364057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:21.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.563+0000 7fa38a46d700 1 --2- 172.21.15.32:0/339447252 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa364055440 0x7fa364057900 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7fa38000b000 tx=0x7fa3800091a0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:21.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.571+0000 7fa3737fe700 1 -- 172.21.15.32:0/339447252 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa374044260 con 0x7fa38413fb00 2020-12-05T17:40:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.711+0000 7fa38b46f700 1 -- 172.21.15.32:0/339447252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa384057050 con 0x7fa38413fb00 2020-12-05T17:40:21.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.711+0000 7fa3737fe700 1 -- 172.21.15.32:0/339447252 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v111) v1 ==== 104+0+10 (secure 0 0 0) 0x7fa374047880 con 0x7fa38413fb00 2020-12-05T17:40:21.716 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:40:21.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.711+0000 7fa38b46f700 1 -- 172.21.15.32:0/339447252 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa364055440 msgr2=0x7fa364057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:21.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.711+0000 7fa38b46f700 1 --2- 172.21.15.32:0/339447252 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa364055440 0x7fa364057900 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7fa38000b000 tx=0x7fa3800091a0).stop 2020-12-05T17:40:21.718 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.711+0000 7fa38b46f700 1 -- 172.21.15.32:0/339447252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa38413fb00 msgr2=0x7fa38413c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:21.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.711+0000 7fa38b46f700 1 --2- 172.21.15.32:0/339447252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa38413fb00 0x7fa38413c5e0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7fa37400c000 tx=0x7fa374006dc0).stop 2020-12-05T17:40:21.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.711+0000 7fa38b46f700 1 -- 172.21.15.32:0/339447252 shutdown_connections 2020-12-05T17:40:21.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.711+0000 7fa38b46f700 1 --2- 172.21.15.32:0/339447252 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa364055440 0x7fa364057900 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.711+0000 7fa38b46f700 1 --2- 172.21.15.32:0/339447252 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa384143410 0x7fa38413cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.711+0000 7fa38b46f700 1 --2- 172.21.15.32:0/339447252 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa3840b2000 0x7fa384149000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.711+0000 7fa38b46f700 1 --2- 172.21.15.32:0/339447252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa38413fb00 0x7fa38413c5e0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.711+0000 7fa38b46f700 1 -- 172.21.15.32:0/339447252 >> 172.21.15.32:0/339447252 conn(0x7fa3840b5620 msgr2=0x7fa38413e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:21.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.715+0000 7fa38b46f700 1 -- 172.21.15.32:0/339447252 shutdown_connections 2020-12-05T17:40:21.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.715+0000 7fa38b46f700 1 -- 172.21.15.32:0/339447252 wait complete. 2020-12-05T17:40:21.735 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:40:21.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.851+0000 7fe0af287700 1 Processor -- start 2020-12-05T17:40:21.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.851+0000 7fe0af287700 1 -- start start 2020-12-05T17:40:21.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.851+0000 7fe0af287700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0a805fad0 0x7fe0a81517a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:21.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.851+0000 7fe0af287700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0a80b3480 con 0x7fe0a8061b30 2020-12-05T17:40:21.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.851+0000 7fe0af287700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0a80afe80 con 0x7fe0a805f6f0 2020-12-05T17:40:21.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.851+0000 7fe0af287700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a80b1a70 con 0x7fe0a805fad0 2020-12-05T17:40:21.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.851+0000 7fe0ada84700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe0a8061b30 0x7fe0a80b2d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42274/0 (socket says 172.21.15.32:42274) 2020-12-05T17:40:21.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.851+0000 7fe0ada84700 1 -- 172.21.15.32:0/2181710845 learned_addr learned my addr 172.21.15.32:0/2181710845 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:21.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.851+0000 7fe0aea86700 1 --2- 172.21.15.32:0/2181710845 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0a805fad0 0x7fe0a81517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:21.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.851+0000 7fe0ad283700 1 -- 172.21.15.32:0/2181710845 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3388274114 0 0) 0x7fe0a80b3480 con 0x7fe0a8061b30 2020-12-05T17:40:21.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.851+0000 7fe0ad283700 1 -- 172.21.15.32:0/2181710845 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe090003230 con 0x7fe0a8061b30 2020-12-05T17:40:21.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.851+0000 7fe0ad283700 1 -- 172.21.15.32:0/2181710845 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3351587387 0 0) 0x7fe0a80afe80 con 0x7fe0a805f6f0 2020-12-05T17:40:21.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.851+0000 7fe0ad283700 1 -- 172.21.15.32:0/2181710845 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0a80b3480 con 0x7fe0a805f6f0 2020-12-05T17:40:21.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.851+0000 7fe0aea86700 1 -- 172.21.15.32:0/2181710845 >> v1:172.21.15.61:6789/0 conn(0x7fe0a805f6f0 legacy=0x7fe0a80b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:21.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.851+0000 7fe0aea86700 1 -- 172.21.15.32:0/2181710845 >> v1:172.21.15.32:6789/0 conn(0x7fe0a8061b30 legacy=0x7fe0a80b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:21.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.851+0000 7fe0aea86700 1 -- 172.21.15.32:0/2181710845 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a814ad00 con 0x7fe0a805fad0 2020-12-05T17:40:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.851+0000 7fe0aea86700 1 --2- 172.21.15.32:0/2181710845 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0a805fad0 0x7fe0a81517a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fe0a8061f60 tx=0x7fe0a00091a0).ready entity=mon.2 client_cookie=39aea1be093fb963 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.851+0000 7fe0ad283700 1 -- 172.21.15.32:0/2181710845 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0a000b900 con 0x7fe0a805fad0 2020-12-05T17:40:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.851+0000 7fe0ad283700 1 -- 172.21.15.32:0/2181710845 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a000ba60 con 0x7fe0a805fad0 2020-12-05T17:40:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.851+0000 7fe0ad283700 1 -- 172.21.15.32:0/2181710845 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0a000bbd0 con 0x7fe0a805fad0 2020-12-05T17:40:21.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.851+0000 7fe0af287700 1 -- 172.21.15.32:0/2181710845 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0a805fad0 msgr2=0x7fe0a81517a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:21.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.851+0000 7fe0af287700 1 --2- 172.21.15.32:0/2181710845 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0a805fad0 0x7fe0a81517a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fe0a8061f60 tx=0x7fe0a00091a0).stop 2020-12-05T17:40:21.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.855+0000 7fe0af287700 1 -- 172.21.15.32:0/2181710845 shutdown_connections 2020-12-05T17:40:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.855+0000 7fe0af287700 1 --2- 172.21.15.32:0/2181710845 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0a805fad0 0x7fe0a81517a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.855+0000 7fe0af287700 1 -- 172.21.15.32:0/2181710845 >> 172.21.15.32:0/2181710845 conn(0x7fe0a8061010 msgr2=0x7fe0a8061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.855+0000 7fe0af287700 1 -- 172.21.15.32:0/2181710845 shutdown_connections 2020-12-05T17:40:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.855+0000 7fe0af287700 1 -- 172.21.15.32:0/2181710845 wait complete. 2020-12-05T17:40:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.855+0000 7fe0af287700 1 Processor -- start 2020-12-05T17:40:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.855+0000 7fe0af287700 1 -- start start 2020-12-05T17:40:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.855+0000 7fe0af287700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a805f6f0 0x7fe0a8144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.855+0000 7fe0af287700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0a805fad0 0x7fe0a810ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.855+0000 7fe0af287700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0a8061b30 0x7fe0a810f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:21.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.855+0000 7fe0af287700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a8064b10 con 0x7fe0a805f6f0 2020-12-05T17:40:21.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.855+0000 7fe0af287700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a814c7f0 con 0x7fe0a8061b30 2020-12-05T17:40:21.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.855+0000 7fe0af287700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a80659c0 con 0x7fe0a805fad0 2020-12-05T17:40:21.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.855+0000 7fe0ae285700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a805f6f0 0x7fe0a8144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:21.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.855+0000 7fe0ae285700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a805f6f0 0x7fe0a8144e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39632/0 (socket says 172.21.15.32:39632) 2020-12-05T17:40:21.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.855+0000 7fe0ae285700 1 -- 172.21.15.32:0/3566894446 learned_addr learned my addr 172.21.15.32:0/3566894446 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:21.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.855+0000 7fe0aea86700 1 --2- 172.21.15.32:0/3566894446 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0a8061b30 0x7fe0a810f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:21.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.859+0000 7fe0ada84700 1 --2- 172.21.15.32:0/3566894446 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0a805fad0 0x7fe0a810ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:21.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.859+0000 7fe0ae285700 1 -- 172.21.15.32:0/3566894446 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0a805fad0 msgr2=0x7fe0a810ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:21.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.859+0000 7fe0ae285700 1 --2- 172.21.15.32:0/3566894446 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0a805fad0 0x7fe0a810ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.859+0000 7fe0ae285700 1 -- 172.21.15.32:0/3566894446 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0a8061b30 msgr2=0x7fe0a810f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:21.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.859+0000 7fe0ae285700 1 --2- 172.21.15.32:0/3566894446 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0a8061b30 0x7fe0a810f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:21.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.859+0000 7fe0ae285700 1 -- 172.21.15.32:0/3566894446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a814ca20 con 0x7fe0a805f6f0 2020-12-05T17:40:21.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.859+0000 7fe0ae285700 1 --2- 172.21.15.32:0/3566894446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a805f6f0 0x7fe0a8144e70 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7fe0a8064c50 tx=0x7fe09c007b40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:21.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.859+0000 7fe09b7fe700 1 -- 172.21.15.32:0/3566894446 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe09c00e4a0 con 0x7fe0a805f6f0 2020-12-05T17:40:21.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.859+0000 7fe09b7fe700 1 -- 172.21.15.32:0/3566894446 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09c00e600 con 0x7fe0a805f6f0 2020-12-05T17:40:21.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.859+0000 7fe0af287700 1 -- 172.21.15.32:0/3566894446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a8065720 con 0x7fe0a805f6f0 2020-12-05T17:40:21.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.859+0000 7fe09b7fe700 1 -- 172.21.15.32:0/3566894446 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe09c00e960 con 0x7fe0a805f6f0 2020-12-05T17:40:21.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.859+0000 7fe0af287700 1 -- 172.21.15.32:0/3566894446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a810f800 con 0x7fe0a805f6f0 2020-12-05T17:40:21.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.859+0000 7fe0af287700 1 -- 172.21.15.32:0/3566894446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a814e030 con 0x7fe0a805f6f0 2020-12-05T17:40:21.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.859+0000 7fe09b7fe700 1 -- 172.21.15.32:0/3566894446 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe09c019030 con 0x7fe0a805f6f0 2020-12-05T17:40:21.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.859+0000 7fe09b7fe700 1 --2- 172.21.15.32:0/3566894446 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe088055330 0x7fe0880577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:21.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.863+0000 7fe09b7fe700 1 -- 172.21.15.32:0/3566894446 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe09c06d530 con 0x7fe0a805f6f0 2020-12-05T17:40:21.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.863+0000 7fe0ada84700 1 --2- 172.21.15.32:0/3566894446 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe088055330 0x7fe0880577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:21.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.863+0000 7fe0ada84700 1 --2- 172.21.15.32:0/3566894446 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe088055330 0x7fe0880577f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fe0a400d000 tx=0x7fe0a4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:21.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:21.867+0000 7fe09b7fe700 1 -- 172.21.15.32:0/3566894446 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe09c0436d0 con 0x7fe0a805f6f0 2020-12-05T17:40:22.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.011+0000 7fe0af287700 1 -- 172.21.15.32:0/3566894446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe0a80af760 con 0x7fe0a805f6f0 2020-12-05T17:40:22.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.011+0000 7fe09b7fe700 1 -- 172.21.15.32:0/3566894446 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v111) v1 ==== 100+0+10 (secure 0 0 0) 0x7fe09c046cf0 con 0x7fe0a805f6f0 2020-12-05T17:40:22.016 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:40:22.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.011+0000 7fe0af287700 1 -- 172.21.15.32:0/3566894446 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe088055330 msgr2=0x7fe0880577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:22.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.011+0000 7fe0af287700 1 --2- 172.21.15.32:0/3566894446 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe088055330 0x7fe0880577f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fe0a400d000 tx=0x7fe0a4009040).stop 2020-12-05T17:40:22.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.011+0000 7fe0af287700 1 -- 172.21.15.32:0/3566894446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a805f6f0 msgr2=0x7fe0a8144e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:22.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.011+0000 7fe0af287700 1 --2- 172.21.15.32:0/3566894446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a805f6f0 0x7fe0a8144e70 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7fe0a8064c50 tx=0x7fe09c007b40).stop 2020-12-05T17:40:22.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.011+0000 7fe0af287700 1 -- 172.21.15.32:0/3566894446 shutdown_connections 2020-12-05T17:40:22.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.011+0000 7fe0af287700 1 --2- 172.21.15.32:0/3566894446 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0a8061b30 0x7fe0a810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:22.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.011+0000 7fe0af287700 1 --2- 172.21.15.32:0/3566894446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a805f6f0 0x7fe0a8144e70 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:22.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.011+0000 7fe0af287700 1 --2- 172.21.15.32:0/3566894446 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe088055330 0x7fe0880577f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:22.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.011+0000 7fe0af287700 1 --2- 172.21.15.32:0/3566894446 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0a805fad0 0x7fe0a810ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:22.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.011+0000 7fe0af287700 1 -- 172.21.15.32:0/3566894446 >> 172.21.15.32:0/3566894446 conn(0x7fe0a8061010 msgr2=0x7fe0a80afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:22.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.011+0000 7fe0af287700 1 -- 172.21.15.32:0/3566894446 shutdown_connections 2020-12-05T17:40:22.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.011+0000 7fe0af287700 1 -- 172.21.15.32:0/3566894446 wait complete. 2020-12-05T17:40:22.030 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:40:22.031 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:40:22.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.135+0000 7f5947fff700 1 Processor -- start 2020-12-05T17:40:22.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.135+0000 7f5947fff700 1 -- start start 2020-12-05T17:40:22.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5947fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59480b1fc0 0x7f59480af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:22.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5947fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5948147400 0x7f5948145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:22.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5947fff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5948146a90 con 0x7f59480ab710 2020-12-05T17:40:22.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5947fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5948061e90 con 0x7f59480b1fc0 2020-12-05T17:40:22.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5947fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59480b0110 con 0x7f5948147400 2020-12-05T17:40:22.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5946ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59480b1fc0 0x7f59480af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:22.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5946ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59480b1fc0 0x7f59480af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39644/0 (socket says 172.21.15.32:39644) 2020-12-05T17:40:22.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5946ffd700 1 -- 172.21.15.32:0/1878920135 learned_addr learned my addr 172.21.15.32:0/1878920135 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:22.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f59477fe700 1 --2- 172.21.15.32:0/1878920135 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5948147400 0x7f5948145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:22.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5946ffd700 1 -- 172.21.15.32:0/1878920135 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5948147400 msgr2=0x7f5948145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:22.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5946ffd700 1 --2- 172.21.15.32:0/1878920135 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5948147400 0x7f5948145d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:22.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5946ffd700 1 -- 172.21.15.32:0/1878920135 >> v1:172.21.15.61:6789/0 conn(0x7f59480ab710 legacy=0x7f5948061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:22.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5946ffd700 1 -- 172.21.15.32:0/1878920135 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59480b23a0 con 0x7f59480b1fc0 2020-12-05T17:40:22.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5946ffd700 1 --2- 172.21.15.32:0/1878920135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59480b1fc0 0x7f59480af230 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7f5930002f30 tx=0x7f5930006e80).ready entity=mon.0 client_cookie=72e282e11e74b620 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:22.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5945ffb700 1 -- 172.21.15.32:0/1878920135 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59300097f0 con 0x7f59480b1fc0 2020-12-05T17:40:22.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5945ffb700 1 -- 172.21.15.32:0/1878920135 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5930009950 con 0x7f59480b1fc0 2020-12-05T17:40:22.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5945ffb700 1 -- 172.21.15.32:0/1878920135 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5930009cb0 con 0x7f59480b1fc0 2020-12-05T17:40:22.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5947fff700 1 -- 172.21.15.32:0/1878920135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59480b1fc0 msgr2=0x7f59480af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:22.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5947fff700 1 --2- 172.21.15.32:0/1878920135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59480b1fc0 0x7f59480af230 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7f5930002f30 tx=0x7f5930006e80).stop 2020-12-05T17:40:22.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5947fff700 1 -- 172.21.15.32:0/1878920135 shutdown_connections 2020-12-05T17:40:22.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5947fff700 1 --2- 172.21.15.32:0/1878920135 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5948147400 0x7f5948145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:22.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5947fff700 1 --2- 172.21.15.32:0/1878920135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59480b1fc0 0x7f59480af230 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:22.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5947fff700 1 -- 172.21.15.32:0/1878920135 >> 172.21.15.32:0/1878920135 conn(0x7f59480b5530 msgr2=0x7f59480b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:22.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5947fff700 1 -- 172.21.15.32:0/1878920135 shutdown_connections 2020-12-05T17:40:22.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5947fff700 1 -- 172.21.15.32:0/1878920135 wait complete. 2020-12-05T17:40:22.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5947fff700 1 Processor -- start 2020-12-05T17:40:22.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.139+0000 7f5947fff700 1 -- start start 2020-12-05T17:40:22.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.143+0000 7f5947fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f59480ab710 0x7f59481432d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.143+0000 7f5947fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59480b1fc0 0x7f594814a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.143+0000 7f5947fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5948147400 0x7f594814a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.143+0000 7f5947fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594814cd50 con 0x7f5948147400 2020-12-05T17:40:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.143+0000 7f5947fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594814de60 con 0x7f59480b1fc0 2020-12-05T17:40:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.143+0000 7f5947fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594814dbc0 con 0x7f59480ab710 2020-12-05T17:40:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.143+0000 7f59477fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5948147400 0x7f594814a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.143+0000 7f59477fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5948147400 0x7f594814a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39650/0 (socket says 172.21.15.32:39650) 2020-12-05T17:40:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.143+0000 7f59477fe700 1 -- 172.21.15.32:0/2126268306 learned_addr learned my addr 172.21.15.32:0/2126268306 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.143+0000 7f5946ffd700 1 --2- 172.21.15.32:0/2126268306 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f59480ab710 0x7f59481432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:22.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.143+0000 7f59467fc700 1 --2- 172.21.15.32:0/2126268306 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59480b1fc0 0x7f594814a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:22.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.143+0000 7f59477fe700 1 -- 172.21.15.32:0/2126268306 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f59480ab710 msgr2=0x7f59481432d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:22.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.143+0000 7f59477fe700 1 --2- 172.21.15.32:0/2126268306 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f59480ab710 0x7f59481432d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:22.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.143+0000 7f59477fe700 1 -- 172.21.15.32:0/2126268306 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59480b1fc0 msgr2=0x7f594814a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:22.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.143+0000 7f59477fe700 1 --2- 172.21.15.32:0/2126268306 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59480b1fc0 0x7f594814a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:22.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.143+0000 7f59477fe700 1 -- 172.21.15.32:0/2126268306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594814d1b0 con 0x7f5948147400 2020-12-05T17:40:22.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.143+0000 7f59477fe700 1 --2- 172.21.15.32:0/2126268306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5948147400 0x7f594814a540 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7f594814dff0 tx=0x7f5940006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.143+0000 7f593ffff700 1 -- 172.21.15.32:0/2126268306 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f594000c860 con 0x7f5948147400 2020-12-05T17:40:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.143+0000 7f593ffff700 1 -- 172.21.15.32:0/2126268306 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594000c9c0 con 0x7f5948147400 2020-12-05T17:40:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.143+0000 7f5947fff700 1 -- 172.21.15.32:0/2126268306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5948143e80 con 0x7f5948147400 2020-12-05T17:40:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.143+0000 7f593ffff700 1 -- 172.21.15.32:0/2126268306 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f594000cd20 con 0x7f5948147400 2020-12-05T17:40:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.143+0000 7f5947fff700 1 -- 172.21.15.32:0/2126268306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f594814c900 con 0x7f5948147400 2020-12-05T17:40:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.143+0000 7f5947fff700 1 -- 172.21.15.32:0/2126268306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f594814cb50 con 0x7f5948147400 2020-12-05T17:40:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.147+0000 7f593ffff700 1 -- 172.21.15.32:0/2126268306 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f594001c460 con 0x7f5948147400 2020-12-05T17:40:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.147+0000 7f593ffff700 1 --2- 172.21.15.32:0/2126268306 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f592c055330 0x7f592c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.147+0000 7f593ffff700 1 -- 172.21.15.32:0/2126268306 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f594006e3a0 con 0x7f5948147400 2020-12-05T17:40:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.147+0000 7f5946ffd700 1 --2- 172.21.15.32:0/2126268306 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f592c055330 0x7f592c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.147+0000 7f5946ffd700 1 --2- 172.21.15.32:0/2126268306 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f592c055330 0x7f592c0577f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f593001c000 tx=0x7f5930015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.151+0000 7f593ffff700 1 -- 172.21.15.32:0/2126268306 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f59400444e0 con 0x7f5948147400 2020-12-05T17:40:22.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.343+0000 7f5947fff700 1 -- 172.21.15.32:0/2126268306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f594814ce90 con 0x7f5948147400 2020-12-05T17:40:22.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.343+0000 7f593ffff700 1 -- 172.21.15.32:0/2126268306 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f5940047b00 con 0x7f5948147400 2020-12-05T17:40:22.348 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:40:22.349 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":52,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6548,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2853534360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2853534360},{"type":"v1","addr":"172.21.15.61:6833","nonce":2853534360}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":50}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:40:12.749385+0000","modified":"2020-12-05T17:40:13.752777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7563":{"gid":7563,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/2641840537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2641840537},{"type":"v1","addr":"172.21.15.32:6835","nonce":2641840537}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2020-12-05T17:40:22.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.343+0000 7f5947fff700 1 -- 172.21.15.32:0/2126268306 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f592c055330 msgr2=0x7f592c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:22.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.343+0000 7f5947fff700 1 --2- 172.21.15.32:0/2126268306 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f592c055330 0x7f592c0577f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f593001c000 tx=0x7f5930015000).stop 2020-12-05T17:40:22.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.343+0000 7f5947fff700 1 -- 172.21.15.32:0/2126268306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5948147400 msgr2=0x7f594814a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:22.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.343+0000 7f5947fff700 1 --2- 172.21.15.32:0/2126268306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5948147400 0x7f594814a540 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7f594814dff0 tx=0x7f5940006dc0).stop 2020-12-05T17:40:22.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.343+0000 7f5947fff700 1 -- 172.21.15.32:0/2126268306 shutdown_connections 2020-12-05T17:40:22.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.343+0000 7f5947fff700 1 --2- 172.21.15.32:0/2126268306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5948147400 0x7f594814a540 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:22.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.343+0000 7f5947fff700 1 --2- 172.21.15.32:0/2126268306 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f592c055330 0x7f592c0577f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:22.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.343+0000 7f5947fff700 1 --2- 172.21.15.32:0/2126268306 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f59480ab710 0x7f59481432d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:22.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.343+0000 7f5947fff700 1 --2- 172.21.15.32:0/2126268306 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59480b1fc0 0x7f594814a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:22.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.343+0000 7f5947fff700 1 -- 172.21.15.32:0/2126268306 >> 172.21.15.32:0/2126268306 conn(0x7f59480b5530 msgr2=0x7f59480af740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:22.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.347+0000 7f5947fff700 1 -- 172.21.15.32:0/2126268306 shutdown_connections 2020-12-05T17:40:22.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:22.347+0000 7f5947fff700 1 -- 172.21.15.32:0/2126268306 wait complete. 2020-12-05T17:40:22.354 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 52 2020-12-05T17:40:22.565 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:40:12.749385+0000', 'modified': '2020-12-05T17:40:13.752777+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7563}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7563': {'gid': 7563, 'name': 'a', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/2641840537', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 2641840537}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 2641840537}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:40:22.566 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:40:22.567 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:40:22.701 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":7563,"want_state":"up:active","state":"up:active","rank_uptime":9.9379369450000006,"mdsmap_epoch":52,"osdmap_epoch":111,"osdmap_epoch_barrier":110,"uptime":15.130389306} 2020-12-05T17:40:22.701 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 7563, "mdsmap_epoch": 52, "osdmap_epoch": 111, "osdmap_epoch_barrier": 110, "rank_uptime": 9.937936945, "state": "up:active", "uptime": 15.130389306, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:40:22.702 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:40:22.702 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-12-05T17:40:22.703 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi002.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-12-05T17:40:22.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:22.703 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:40:22.711 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:40:22.770 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:40:22.771 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:40:22.827 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:40:22.827 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:40:22.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:22.830 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:40:22.877 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:40:22.878 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:40:22.920 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:40:22.934 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:40:22.935 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:40:22.995 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:2020-12-05T17:40:22.994+0000 7f51736780c0 -1 init, newargv = 0x564218365a40 newargc=9 2020-12-05T17:40:22.997 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[10926]: starting ceph client 2020-12-05T17:40:22.998 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:40:22.998 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:40:22.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:22.999 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:40:23.007 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[10926]: starting fuse 2020-12-05T17:40:23.012 INFO:teuthology.orchestra.run.smithi002.stdout:50 2020-12-05T17:40:23.012 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [50] 2020-12-05T17:40:23.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:23.014 INFO:teuthology.orchestra.run.smithi002:> sudo python3 -c ' 2020-12-05T17:40:23.015 INFO:teuthology.orchestra.run.smithi002:> import glob 2020-12-05T17:40:23.016 INFO:teuthology.orchestra.run.smithi002:> import re 2020-12-05T17:40:23.016 INFO:teuthology.orchestra.run.smithi002:> import os 2020-12-05T17:40:23.022 INFO:teuthology.orchestra.run.smithi002:> import subprocess 2020-12-05T17:40:23.025 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:40:23.026 INFO:teuthology.orchestra.run.smithi002:> def find_socket(client_name): 2020-12-05T17:40:23.026 INFO:teuthology.orchestra.run.smithi002:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-12-05T17:40:23.027 INFO:teuthology.orchestra.run.smithi002:> files = glob.glob(asok_path) 2020-12-05T17:40:23.028 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:40:23.029 INFO:teuthology.orchestra.run.smithi002:> # Given a non-glob path, it better be there 2020-12-05T17:40:23.030 INFO:teuthology.orchestra.run.smithi002:> if "*" not in asok_path: 2020-12-05T17:40:23.030 INFO:teuthology.orchestra.run.smithi002:> assert(len(files) == 1) 2020-12-05T17:40:23.031 INFO:teuthology.orchestra.run.smithi002:> return files[0] 2020-12-05T17:40:23.031 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:40:23.032 INFO:teuthology.orchestra.run.smithi002:> for f in files: 2020-12-05T17:40:23.032 INFO:teuthology.orchestra.run.smithi002:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:40:23.032 INFO:teuthology.orchestra.run.smithi002:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:40:23.033 INFO:teuthology.orchestra.run.smithi002:> return f 2020-12-05T17:40:23.033 INFO:teuthology.orchestra.run.smithi002:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:40:23.034 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:40:23.034 INFO:teuthology.orchestra.run.smithi002:> print(find_socket("client.1")) 2020-12-05T17:40:23.034 INFO:teuthology.orchestra.run.smithi002:> ' 2020-12-05T17:40:23.099 INFO:teuthology.orchestra.run.smithi002.stdout:/var/run/ceph/ceph-client.1.10926.asok 2020-12-05T17:40:23.104 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.10926.asok 2020-12-05T17:40:23.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:23.105 INFO:teuthology.orchestra.run.smithi002:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.10926.asok status 2020-12-05T17:40:23.205 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2020-12-05T17:40:23.205 INFO:teuthology.orchestra.run.smithi002.stdout: "metadata": { 2020-12-05T17:40:23.206 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:40:23.206 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:40:23.206 INFO:teuthology.orchestra.run.smithi002.stdout: "entity_id": "1", 2020-12-05T17:40:23.206 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": "smithi002", 2020-12-05T17:40:23.207 INFO:teuthology.orchestra.run.smithi002.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-12-05T17:40:23.207 INFO:teuthology.orchestra.run.smithi002.stdout: "pid": "10926", 2020-12-05T17:40:23.207 INFO:teuthology.orchestra.run.smithi002.stdout: "root": "/" 2020-12-05T17:40:23.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:40:23.208 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_count": 0, 2020-12-05T17:40:23.208 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_pinned_count": 0, 2020-12-05T17:40:23.208 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 7764, 2020-12-05T17:40:23.208 INFO:teuthology.orchestra.run.smithi002.stdout: "inst": { 2020-12-05T17:40:23.209 INFO:teuthology.orchestra.run.smithi002.stdout: "name": { 2020-12-05T17:40:23.209 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "client", 2020-12-05T17:40:23.209 INFO:teuthology.orchestra.run.smithi002.stdout: "num": 7764 2020-12-05T17:40:23.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:40:23.209 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2020-12-05T17:40:23.210 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2020-12-05T17:40:23.210 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2020-12-05T17:40:23.210 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2981709195 2020-12-05T17:40:23.211 INFO:teuthology.orchestra.run.smithi002.stdout: } 2020-12-05T17:40:23.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:40:23.211 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2020-12-05T17:40:23.212 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2020-12-05T17:40:23.212 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2020-12-05T17:40:23.212 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2981709195 2020-12-05T17:40:23.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:40:23.213 INFO:teuthology.orchestra.run.smithi002.stdout: "inst_str": "client.7764 172.21.15.2:0/2981709195", 2020-12-05T17:40:23.213 INFO:teuthology.orchestra.run.smithi002.stdout: "addr_str": "172.21.15.2:0/2981709195", 2020-12-05T17:40:23.214 INFO:teuthology.orchestra.run.smithi002.stdout: "inode_count": 1, 2020-12-05T17:40:23.214 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_epoch": 52, 2020-12-05T17:40:23.215 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch": 111, 2020-12-05T17:40:23.215 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:40:23.215 INFO:teuthology.orchestra.run.smithi002.stdout: "blacklisted": false 2020-12-05T17:40:23.216 INFO:teuthology.orchestra.run.smithi002.stdout:} 2020-12-05T17:40:23.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:23.222 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:40:23.222 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:40:23.230 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:40:23.232 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:40:23.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:23.233 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:40:23.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:23.294 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:40:23.294 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:40:23.302 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:40:23.303 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:40:23.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:23.304 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:40:23.304 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:40:23.351 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:40:23.352 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:40:23.353 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi036.front.sepia.ceph.com... 2020-12-05T17:40:23.354 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:40:23.354 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:40:23.444 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[11451]: fuse finished with error 0 and tester_r 0 2020-12-05T17:40:23.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:23.446 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:40:23.447 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:40:23.454 INFO:teuthology.orchestra.run.smithi036.stdout:ext2/ext3 2020-12-05T17:40:23.455 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:40:23.457 INFO:teuthology.orchestra.run:waiting for 900 2020-12-05T17:40:29.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:40:29.561 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:40:29.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:29.571 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:40:29.572 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:40:29.580 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:40:29.581 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:40:29.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:29.582 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:40:29.583 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:40:29.629 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:40:29.631 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:40:29.631 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi002.front.sepia.ceph.com... 2020-12-05T17:40:29.632 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:40:29.633 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:40:29.711 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[10926]: fuse finished with error 0 and tester_r 0 2020-12-05T17:40:29.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:29.713 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:40:29.714 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:40:29.761 INFO:teuthology.orchestra.run.smithi002.stdout:ext2/ext3 2020-12-05T17:40:29.762 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:40:29.762 INFO:teuthology.orchestra.run:waiting for 900 2020-12-05T17:40:29.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:40:29.794 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:40:29.841 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-12-05T17:40:29.842 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:40:29.842 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-12-05T17:40:29.843 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:40:29.857 INFO:tasks.ceph.mds.a:Stopped 2020-12-05T17:40:29.858 DEBUG:teuthology.parallel:result is None 2020-12-05T17:40:29.859 INFO:tasks.ceph.mds.b:Stopped 2020-12-05T17:40:29.859 DEBUG:teuthology.parallel:result is None 2020-12-05T17:40:29.860 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early mds fail a 2020-12-05T17:40:29.860 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early mds fail b 2020-12-05T17:40:30.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.007+0000 7fd56bfff700 1 Processor -- start 2020-12-05T17:40:30.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.007+0000 7fd56bfff700 1 -- start start 2020-12-05T17:40:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.007+0000 7fd56bfff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd56c13c6f0 0x7fd56c061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.007+0000 7fd56bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd56c0ae4b0 0x7fd56c0b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.007+0000 7fd56bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd56c0b78d0 0x7fd56c145c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.007+0000 7fd56bfff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd56c0b2340 con 0x7fd56c0b78d0 2020-12-05T17:40:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.007+0000 7fd56bfff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd56c062970 con 0x7fd56c13c6f0 2020-12-05T17:40:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.007+0000 7fd56bfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c0add70 con 0x7fd56c0ae4b0 2020-12-05T17:40:30.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd56b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd56c0b78d0 0x7fd56c145c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:30.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd56b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd56c0b78d0 0x7fd56c145c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39656/0 (socket says 172.21.15.32:39656) 2020-12-05T17:40:30.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd56b7fe700 1 -- 172.21.15.32:0/3734149260 learned_addr learned my addr 172.21.15.32:0/3734149260 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:30.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd56a7fc700 1 --2- 172.21.15.32:0/3734149260 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd56c0ae4b0 0x7fd56c0b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:30.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd56affd700 1 --2- 172.21.15.32:0/3734149260 >> v2:172.21.15.61:3300/0 conn(0x7fd56c13c6f0 0x7fd56c061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:30.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd56b7fe700 1 -- 172.21.15.32:0/3734149260 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd56c0ae4b0 msgr2=0x7fd56c0b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:30.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd56b7fe700 1 --2- 172.21.15.32:0/3734149260 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd56c0ae4b0 0x7fd56c0b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:30.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd56b7fe700 1 -- 172.21.15.32:0/3734149260 >> v2:172.21.15.61:3300/0 conn(0x7fd56c13c6f0 msgr2=0x7fd56c061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:30.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd56b7fe700 1 --2- 172.21.15.32:0/3734149260 >> v2:172.21.15.61:3300/0 conn(0x7fd56c13c6f0 0x7fd56c061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:30.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd56b7fe700 1 -- 172.21.15.32:0/3734149260 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56c0ae010 con 0x7fd56c0b78d0 2020-12-05T17:40:30.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd56affd700 1 --2- 172.21.15.32:0/3734149260 >> v2:172.21.15.61:3300/0 conn(0x7fd56c13c6f0 0x7fd56c061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:40:30.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd56b7fe700 1 --2- 172.21.15.32:0/3734149260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd56c0b78d0 0x7fd56c145c00 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7fd5640032b0 tx=0x7fd564009270).ready entity=mon.0 client_cookie=cd7f22ab7580e9c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:30.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd569ffb700 1 -- 172.21.15.32:0/3734149260 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd564009e60 con 0x7fd56c0b78d0 2020-12-05T17:40:30.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd569ffb700 1 -- 172.21.15.32:0/3734149260 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56400bc00 con 0x7fd56c0b78d0 2020-12-05T17:40:30.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd569ffb700 1 -- 172.21.15.32:0/3734149260 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd56400bd70 con 0x7fd56c0b78d0 2020-12-05T17:40:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd56bfff700 1 -- 172.21.15.32:0/3734149260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd56c0b78d0 msgr2=0x7fd56c145c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd56bfff700 1 --2- 172.21.15.32:0/3734149260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd56c0b78d0 0x7fd56c145c00 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7fd5640032b0 tx=0x7fd564009270).stop 2020-12-05T17:40:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd56bfff700 1 -- 172.21.15.32:0/3734149260 shutdown_connections 2020-12-05T17:40:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd56bfff700 1 --2- 172.21.15.32:0/3734149260 >> v2:172.21.15.61:3300/0 conn(0x7fd56c13c6f0 0x7fd56c061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd56bfff700 1 --2- 172.21.15.32:0/3734149260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd56c0b78d0 0x7fd56c145c00 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd56bfff700 1 --2- 172.21.15.32:0/3734149260 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd56c0ae4b0 0x7fd56c0b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd56bfff700 1 -- 172.21.15.32:0/3734149260 >> 172.21.15.32:0/3734149260 conn(0x7fd56c05bc20 msgr2=0x7fd56c0b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd56bfff700 1 -- 172.21.15.32:0/3734149260 shutdown_connections 2020-12-05T17:40:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd56bfff700 1 -- 172.21.15.32:0/3734149260 wait complete. 2020-12-05T17:40:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd56bfff700 1 Processor -- start 2020-12-05T17:40:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7fd56bfff700 1 -- start start 2020-12-05T17:40:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.007+0000 7feb8cf7f700 1 Processor -- start 2020-12-05T17:40:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.007+0000 7feb8cf7f700 1 -- start start 2020-12-05T17:40:30.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb8cf7f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feb880afd20 0x7feb880b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:30.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb8cf7f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7feb88141ad0 0x7feb88141f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:30.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb8cf7f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb880adfe0 con 0x7feb880b54b0 2020-12-05T17:40:30.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb8cf7f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb880b0630 con 0x7feb880afd20 2020-12-05T17:40:30.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb8cf7f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb880ae210 con 0x7feb88141ad0 2020-12-05T17:40:30.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb86ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feb880afd20 0x7feb880b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:30.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb86ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feb880afd20 0x7feb880b11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39660/0 (socket says 172.21.15.32:39660) 2020-12-05T17:40:30.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb86ffd700 1 -- 172.21.15.32:0/3235471591 learned_addr learned my addr 172.21.15.32:0/3235471591 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:30.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb87fff700 1 --2- 172.21.15.32:0/3235471591 >> v2:172.21.15.61:3300/0 conn(0x7feb88141ad0 0x7feb88141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:30.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb86ffd700 1 -- 172.21.15.32:0/3235471591 >> v2:172.21.15.61:3300/0 conn(0x7feb88141ad0 msgr2=0x7feb88141f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:30.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb86ffd700 1 --2- 172.21.15.32:0/3235471591 >> v2:172.21.15.61:3300/0 conn(0x7feb88141ad0 0x7feb88141f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:30.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb86ffd700 1 -- 172.21.15.32:0/3235471591 >> v1:172.21.15.61:6789/0 conn(0x7feb880b54b0 legacy=0x7feb880b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:30.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb86ffd700 1 -- 172.21.15.32:0/3235471591 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb880677f0 con 0x7feb880afd20 2020-12-05T17:40:30.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb87fff700 1 --2- 172.21.15.32:0/3235471591 >> v2:172.21.15.61:3300/0 conn(0x7feb88141ad0 0x7feb88141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:40:30.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb86ffd700 1 --2- 172.21.15.32:0/3235471591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb880afd20 0x7feb880b11a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 rx=0x7feb7c002f30 tx=0x7feb7c006e80).ready entity=mon.0 client_cookie=ed3d926215b83bd0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:30.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb867fc700 1 -- 172.21.15.32:0/3235471591 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb7c0097f0 con 0x7feb880afd20 2020-12-05T17:40:30.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb867fc700 1 -- 172.21.15.32:0/3235471591 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7c009950 con 0x7feb880afd20 2020-12-05T17:40:30.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb867fc700 1 -- 172.21.15.32:0/3235471591 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb7c009cb0 con 0x7feb880afd20 2020-12-05T17:40:30.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb8cf7f700 1 -- 172.21.15.32:0/3235471591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb880afd20 msgr2=0x7feb880b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:30.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb8cf7f700 1 --2- 172.21.15.32:0/3235471591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb880afd20 0x7feb880b11a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 rx=0x7feb7c002f30 tx=0x7feb7c006e80).stop 2020-12-05T17:40:30.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb8cf7f700 1 -- 172.21.15.32:0/3235471591 shutdown_connections 2020-12-05T17:40:30.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb8cf7f700 1 --2- 172.21.15.32:0/3235471591 >> v2:172.21.15.61:3300/0 conn(0x7feb88141ad0 0x7feb88141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:30.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb8cf7f700 1 --2- 172.21.15.32:0/3235471591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb880afd20 0x7feb880b11a0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:30.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb8cf7f700 1 -- 172.21.15.32:0/3235471591 >> 172.21.15.32:0/3235471591 conn(0x7feb8805bbb0 msgr2=0x7feb88062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:30.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb8cf7f700 1 -- 172.21.15.32:0/3235471591 shutdown_connections 2020-12-05T17:40:30.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.011+0000 7feb8cf7f700 1 -- 172.21.15.32:0/3235471591 wait complete. 2020-12-05T17:40:30.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb8cf7f700 1 Processor -- start 2020-12-05T17:40:30.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb8cf7f700 1 -- start start 2020-12-05T17:40:30.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb8cf7f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb880afd20 0x7feb880b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:30.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb8cf7f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feb880b54b0 0x7feb88110450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:30.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb8cf7f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feb88141ad0 0x7feb8813d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:30.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb8cf7f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8810fae0 con 0x7feb880afd20 2020-12-05T17:40:30.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb8cf7f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb88149fe0 con 0x7feb880b54b0 2020-12-05T17:40:30.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb8cf7f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8810fd10 con 0x7feb88141ad0 2020-12-05T17:40:30.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb877fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb880afd20 0x7feb880b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:30.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb877fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb880afd20 0x7feb880b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39672/0 (socket says 172.21.15.32:39672) 2020-12-05T17:40:30.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb87fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feb88141ad0 0x7feb8813d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:30.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb877fe700 1 -- 172.21.15.32:0/1867063134 learned_addr learned my addr 172.21.15.32:0/1867063134 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:30.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb86ffd700 1 --2- 172.21.15.32:0/1867063134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feb880b54b0 0x7feb88110450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb877fe700 1 -- 172.21.15.32:0/1867063134 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feb88141ad0 msgr2=0x7feb8813d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb877fe700 1 --2- 172.21.15.32:0/1867063134 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feb88141ad0 0x7feb8813d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb877fe700 1 -- 172.21.15.32:0/1867063134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feb880b54b0 msgr2=0x7feb88110450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb877fe700 1 --2- 172.21.15.32:0/1867063134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feb880b54b0 0x7feb88110450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb877fe700 1 -- 172.21.15.32:0/1867063134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb8810ffb0 con 0x7feb880afd20 2020-12-05T17:40:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb877fe700 1 --2- 172.21.15.32:0/1867063134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb880afd20 0x7feb880b1b30 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7feb88062be0 tx=0x7feb78007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb84ff9700 1 -- 172.21.15.32:0/1867063134 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb78013410 con 0x7feb880afd20 2020-12-05T17:40:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb84ff9700 1 -- 172.21.15.32:0/1867063134 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb78013570 con 0x7feb880afd20 2020-12-05T17:40:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb8cf7f700 1 -- 172.21.15.32:0/1867063134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb88146970 con 0x7feb880afd20 2020-12-05T17:40:30.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb84ff9700 1 -- 172.21.15.32:0/1867063134 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb780138d0 con 0x7feb880afd20 2020-12-05T17:40:30.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7feb8cf7f700 1 -- 172.21.15.32:0/1867063134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb881459f0 con 0x7feb880afd20 2020-12-05T17:40:30.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.019+0000 7feb8cf7f700 1 -- 172.21.15.32:0/1867063134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb88057050 con 0x7feb880afd20 2020-12-05T17:40:30.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.019+0000 7feb84ff9700 1 -- 172.21.15.32:0/1867063134 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7feb78010030 con 0x7feb880afd20 2020-12-05T17:40:30.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.019+0000 7feb84ff9700 1 --2- 172.21.15.32:0/1867063134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feb68055440 0x7feb68057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:30.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.019+0000 7feb84ff9700 1 -- 172.21.15.32:0/1867063134 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7feb7806deb0 con 0x7feb880afd20 2020-12-05T17:40:30.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.019+0000 7feb86ffd700 1 --2- 172.21.15.32:0/1867063134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feb68055440 0x7feb68057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:30.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.019+0000 7feb86ffd700 1 --2- 172.21.15.32:0/1867063134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feb68055440 0x7feb68057900 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7feb7c01c000 tx=0x7feb7c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:30.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.027+0000 7feb84ff9700 1 -- 172.21.15.32:0/1867063134 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7feb78044050 con 0x7feb880afd20 2020-12-05T17:40:30.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd56bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd56c0ae4b0 0x7fd56c065240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:30.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd56bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd56c0b78d0 0x7fd56c1440b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:30.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd56bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd56c13c6f0 0x7fd56c066a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:30.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd56bfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c063610 con 0x7fd56c0ae4b0 2020-12-05T17:40:30.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd56bfff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c0630b0 con 0x7fd56c0b78d0 2020-12-05T17:40:30.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd56bfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c063370 con 0x7fd56c13c6f0 2020-12-05T17:40:30.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd56affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd56c0ae4b0 0x7fd56c065240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:30.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd56affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd56c0ae4b0 0x7fd56c065240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39664/0 (socket says 172.21.15.32:39664) 2020-12-05T17:40:30.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd56affd700 1 -- 172.21.15.32:0/987475131 learned_addr learned my addr 172.21.15.32:0/987475131 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:30.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd56b7fe700 1 --2- 172.21.15.32:0/987475131 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd56c13c6f0 0x7fd56c066a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:30.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd56a7fc700 1 --2- 172.21.15.32:0/987475131 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd56c0b78d0 0x7fd56c1440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:30.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd56affd700 1 -- 172.21.15.32:0/987475131 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd56c13c6f0 msgr2=0x7fd56c066a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:30.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd56affd700 1 --2- 172.21.15.32:0/987475131 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd56c13c6f0 0x7fd56c066a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:30.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd56affd700 1 -- 172.21.15.32:0/987475131 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd56c0b78d0 msgr2=0x7fd56c1440b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:30.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd56affd700 1 --2- 172.21.15.32:0/987475131 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd56c0b78d0 0x7fd56c1440b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:30.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd56affd700 1 -- 172.21.15.32:0/987475131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56c063800 con 0x7fd56c0ae4b0 2020-12-05T17:40:30.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd56b7fe700 1 --2- 172.21.15.32:0/987475131 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd56c13c6f0 0x7fd56c066a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:40:30.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd56a7fc700 1 --2- 172.21.15.32:0/987475131 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd56c0b78d0 0x7fd56c1440b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:40:30.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd56affd700 1 --2- 172.21.15.32:0/987475131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd56c0ae4b0 0x7fd56c065240 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7fd55c006910 tx=0x7fd55c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd55bfff700 1 -- 172.21.15.32:0/987475131 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd55c009980 con 0x7fd56c0ae4b0 2020-12-05T17:40:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd55bfff700 1 -- 172.21.15.32:0/987475131 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55c009ae0 con 0x7fd56c0ae4b0 2020-12-05T17:40:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd56bfff700 1 -- 172.21.15.32:0/987475131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56c144d50 con 0x7fd56c0ae4b0 2020-12-05T17:40:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd55bfff700 1 -- 172.21.15.32:0/987475131 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd55c009e40 con 0x7fd56c0ae4b0 2020-12-05T17:40:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd56bfff700 1 -- 172.21.15.32:0/987475131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56c13cd40 con 0x7fd56c0ae4b0 2020-12-05T17:40:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd56bfff700 1 -- 172.21.15.32:0/987475131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd56c057050 con 0x7fd56c0ae4b0 2020-12-05T17:40:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.015+0000 7fd55bfff700 1 -- 172.21.15.32:0/987475131 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd55c013480 con 0x7fd56c0ae4b0 2020-12-05T17:40:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.019+0000 7fd55bfff700 1 --2- 172.21.15.32:0/987475131 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd548055440 0x7fd548057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.019+0000 7fd55bfff700 1 -- 172.21.15.32:0/987475131 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd55c06e810 con 0x7fd56c0ae4b0 2020-12-05T17:40:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.019+0000 7fd56a7fc700 1 --2- 172.21.15.32:0/987475131 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd548055440 0x7fd548057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.019+0000 7fd56a7fc700 1 --2- 172.21.15.32:0/987475131 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd548055440 0x7fd548057900 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7fd56000b000 tx=0x7fd560008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.023+0000 7fd55bfff700 1 -- 172.21.15.32:0/987475131 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd55c044950 con 0x7fd56c0ae4b0 2020-12-05T17:40:30.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.219+0000 7fd56bfff700 1 -- 172.21.15.32:0/987475131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7fd56c0b1fe0 con 0x7fd56c0ae4b0 2020-12-05T17:40:30.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:30.223+0000 7feb8cf7f700 1 -- 172.21.15.32:0/1867063134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7feb880b2bf0 con 0x7feb880afd20 2020-12-05T17:40:31.041 INFO:tasks.ceph.mon.a.smithi032.stderr:2020-12-05T17:40:31.035+0000 7fdef8136700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-12-05T17:40:31.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.051+0000 7fd55bfff700 1 -- 172.21.15.32:0/987475131 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 6548 v53) v1 ==== 95+0+0 (secure 0 0 0) 0x7fd55c047f70 con 0x7fd56c0ae4b0 2020-12-05T17:40:31.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.051+0000 7feb84ff9700 1 -- 172.21.15.32:0/1867063134 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 7563 v53) v1 ==== 95+0+0 (secure 0 0 0) 0x7feb78047670 con 0x7feb880afd20 2020-12-05T17:40:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7feb8cf7f700 1 -- 172.21.15.32:0/1867063134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feb68055440 msgr2=0x7feb68057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7feb8cf7f700 1 --2- 172.21.15.32:0/1867063134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feb68055440 0x7feb68057900 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7feb7c01c000 tx=0x7feb7c015000).stop 2020-12-05T17:40:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7feb8cf7f700 1 -- 172.21.15.32:0/1867063134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb880afd20 msgr2=0x7feb880b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7feb8cf7f700 1 --2- 172.21.15.32:0/1867063134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb880afd20 0x7feb880b1b30 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7feb88062be0 tx=0x7feb78007c20).stop 2020-12-05T17:40:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7feb8cf7f700 1 -- 172.21.15.32:0/1867063134 shutdown_connections 2020-12-05T17:40:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7feb8cf7f700 1 --2- 172.21.15.32:0/1867063134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7feb68055440 0x7feb68057900 secure :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0x7feb7c01c000 tx=0x7feb7c015000).stop 2020-12-05T17:40:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7feb8cf7f700 1 --2- 172.21.15.32:0/1867063134 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feb88141ad0 0x7feb8813d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7feb8cf7f700 1 --2- 172.21.15.32:0/1867063134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb880afd20 0x7feb880b1b30 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:31.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7feb8cf7f700 1 --2- 172.21.15.32:0/1867063134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feb880b54b0 0x7feb88110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:31.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7feb8cf7f700 1 -- 172.21.15.32:0/1867063134 >> 172.21.15.32:0/1867063134 conn(0x7feb8805bbb0 msgr2=0x7feb880b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:31.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7feb8cf7f700 1 -- 172.21.15.32:0/1867063134 shutdown_connections 2020-12-05T17:40:31.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7feb8cf7f700 1 -- 172.21.15.32:0/1867063134 wait complete. 2020-12-05T17:40:31.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7fd56bfff700 1 -- 172.21.15.32:0/987475131 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd548055440 msgr2=0x7fd548057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:31.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7fd56bfff700 1 --2- 172.21.15.32:0/987475131 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd548055440 0x7fd548057900 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7fd56000b000 tx=0x7fd560008040).stop 2020-12-05T17:40:31.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7fd56bfff700 1 -- 172.21.15.32:0/987475131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd56c0ae4b0 msgr2=0x7fd56c065240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:31.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7fd56bfff700 1 --2- 172.21.15.32:0/987475131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd56c0ae4b0 0x7fd56c065240 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7fd55c006910 tx=0x7fd55c008040).stop 2020-12-05T17:40:31.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7fd56bfff700 1 -- 172.21.15.32:0/987475131 shutdown_connections 2020-12-05T17:40:31.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7fd56bfff700 1 --2- 172.21.15.32:0/987475131 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd548055440 0x7fd548057900 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:31.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7fd56bfff700 1 --2- 172.21.15.32:0/987475131 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd56c13c6f0 0x7fd56c066a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:31.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7fd56bfff700 1 --2- 172.21.15.32:0/987475131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd56c0ae4b0 0x7fd56c065240 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:31.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7fd56bfff700 1 --2- 172.21.15.32:0/987475131 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd56c0b78d0 0x7fd56c1440b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:31.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7fd56bfff700 1 -- 172.21.15.32:0/987475131 >> 172.21.15.32:0/987475131 conn(0x7fd56c05bc20 msgr2=0x7fd56c0b5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:31.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7fd56bfff700 1 -- 172.21.15.32:0/987475131 shutdown_connections 2020-12-05T17:40:31.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.055+0000 7fd56bfff700 1 -- 172.21.15.32:0/987475131 wait complete. 2020-12-05T17:40:31.065 INFO:teuthology.orchestra.run.smithi032.stderr:failed mds gid 6548 2020-12-05T17:40:31.066 INFO:teuthology.orchestra.run.smithi032.stderr:failed mds gid 7563 2020-12-05T17:40:31.075 DEBUG:teuthology.parallel:result is '' 2020-12-05T17:40:31.077 DEBUG:teuthology.parallel:result is '' 2020-12-05T17:40:31.077 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-12-05T17:40:31.244 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.239+0000 7f8283102700 1 Processor -- start 2020-12-05T17:40:31.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.239+0000 7f8283102700 1 -- start start 2020-12-05T17:40:31.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.239+0000 7f8283102700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f827c0b6690 0x7f827c0b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:31.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.239+0000 7f8283102700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f827c0b56f0 con 0x7f827c0b5240 2020-12-05T17:40:31.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.239+0000 7f8283102700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f827c0b1920 con 0x7f827c0b5b20 2020-12-05T17:40:31.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.239+0000 7f8283102700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827c0b1bc0 con 0x7f827c0b6690 2020-12-05T17:40:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f828169f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f827c0b5240 0x7f827c145f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42332/0 (socket says 172.21.15.32:42332) 2020-12-05T17:40:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f828169f700 1 -- 172.21.15.32:0/83180005 learned_addr learned my addr 172.21.15.32:0/83180005 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f827bfff700 1 --2- 172.21.15.32:0/83180005 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f827c0b6690 0x7f827c0b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f827b7fe700 1 -- 172.21.15.32:0/83180005 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181711271 0 0) 0x7f827c0b56f0 con 0x7f827c0b5240 2020-12-05T17:40:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f827b7fe700 1 -- 172.21.15.32:0/83180005 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8268003230 con 0x7f827c0b5240 2020-12-05T17:40:31.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f827b7fe700 1 -- 172.21.15.32:0/83180005 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1751763163 0 0) 0x7f827c0b1920 con 0x7f827c0b5b20 2020-12-05T17:40:31.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f827b7fe700 1 -- 172.21.15.32:0/83180005 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f827c0b56f0 con 0x7f827c0b5b20 2020-12-05T17:40:31.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f827b7fe700 1 -- 172.21.15.32:0/83180005 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3324414005 0 0) 0x7f8268003230 con 0x7f827c0b5240 2020-12-05T17:40:31.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f827b7fe700 1 -- 172.21.15.32:0/83180005 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f827c0b6690 msgr2=0x7f827c0b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:31.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f827b7fe700 1 --2- 172.21.15.32:0/83180005 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f827c0b6690 0x7f827c0b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:31.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f827b7fe700 1 -- 172.21.15.32:0/83180005 >> v1:172.21.15.61:6789/0 conn(0x7f827c0b5b20 legacy=0x7f827c0b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:31.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f827b7fe700 1 -- 172.21.15.32:0/83180005 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c146620 con 0x7f827c0b5240 2020-12-05T17:40:31.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f827b7fe700 1 -- 172.21.15.32:0/83180005 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f8274001a50 con 0x7f827c0b5240 2020-12-05T17:40:31.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f827b7fe700 1 -- 172.21.15.32:0/83180005 >> v1:172.21.15.32:6789/0 conn(0x7f827c0b5240 legacy=0x7f827c145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:31.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f827b7fe700 1 --2- 172.21.15.32:0/83180005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82680038e0 0x7f8268003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:31.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f827b7fe700 1 --2- 172.21.15.32:0/83180005 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f82680044d0 0x7f8268004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:31.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f827b7fe700 1 --2- 172.21.15.32:0/83180005 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f827c0b6690 0x7f8268004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:31.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f827b7fe700 1 -- 172.21.15.32:0/83180005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c0b1bc0 con 0x7f82680038e0 2020-12-05T17:40:31.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f827b7fe700 1 -- 172.21.15.32:0/83180005 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8268005330 con 0x7f82680044d0 2020-12-05T17:40:31.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f827b7fe700 1 -- 172.21.15.32:0/83180005 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826800b720 con 0x7f827c0b6690 2020-12-05T17:40:31.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f8280e9e700 1 --2- 172.21.15.32:0/83180005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82680038e0 0x7f8268003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:31.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f827bfff700 1 --2- 172.21.15.32:0/83180005 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f82680044d0 0x7f8268004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:31.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f828169f700 1 --2- 172.21.15.32:0/83180005 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f827c0b6690 0x7f8268004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:31.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f8280e9e700 1 -- 172.21.15.32:0/83180005 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f827c0b6690 msgr2=0x7f8268004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:31.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f8280e9e700 1 --2- 172.21.15.32:0/83180005 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f827c0b6690 0x7f8268004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:31.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f8280e9e700 1 -- 172.21.15.32:0/83180005 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f82680044d0 msgr2=0x7f8268004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:31.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f8280e9e700 1 --2- 172.21.15.32:0/83180005 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f82680044d0 0x7f8268004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:31.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f8280e9e700 1 -- 172.21.15.32:0/83180005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8268003230 con 0x7f82680038e0 2020-12-05T17:40:31.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f827bfff700 1 --2- 172.21.15.32:0/83180005 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f82680044d0 0x7f8268004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:40:31.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f8280e9e700 1 --2- 172.21.15.32:0/83180005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82680038e0 0x7f8268003ce0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 rx=0x7f826c0010d0 tx=0x7f826c007b60).ready entity=mon.0 client_cookie=506915ec0499734c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:31.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f827b7fe700 1 -- 172.21.15.32:0/83180005 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f826c00c870 con 0x7f82680038e0 2020-12-05T17:40:31.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f827b7fe700 1 -- 172.21.15.32:0/83180005 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826c00c9d0 con 0x7f82680038e0 2020-12-05T17:40:31.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f827b7fe700 1 -- 172.21.15.32:0/83180005 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f826c00cd30 con 0x7f82680038e0 2020-12-05T17:40:31.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f8283102700 1 -- 172.21.15.32:0/83180005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82680038e0 msgr2=0x7f8268003ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:31.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f8283102700 1 --2- 172.21.15.32:0/83180005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82680038e0 0x7f8268003ce0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 rx=0x7f826c0010d0 tx=0x7f826c007b60).stop 2020-12-05T17:40:31.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f828169f700 1 -- 172.21.15.32:0/83180005 reap_dead start 2020-12-05T17:40:31.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f8283102700 1 -- 172.21.15.32:0/83180005 shutdown_connections 2020-12-05T17:40:31.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f8283102700 1 -- 172.21.15.32:0/83180005 >> 172.21.15.32:0/83180005 conn(0x7f827c05f3b0 msgr2=0x7f827c05f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:31.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f8283102700 1 -- 172.21.15.32:0/83180005 shutdown_connections 2020-12-05T17:40:31.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.243+0000 7f8283102700 1 -- 172.21.15.32:0/83180005 wait complete. 2020-12-05T17:40:31.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f8283102700 1 Processor -- start 2020-12-05T17:40:31.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f8283102700 1 -- start start 2020-12-05T17:40:31.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f8283102700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f827c066610 0x7f827c154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:31.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f8283102700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f827c0a28d0 0x7f827c1552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:31.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f8283102700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f827c1557a0 0x7f827c150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:31.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f8283102700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c155c20 con 0x7f827c1557a0 2020-12-05T17:40:31.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f8283102700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c0669f0 con 0x7f827c066610 2020-12-05T17:40:31.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f8283102700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827c0a5240 con 0x7f827c0a28d0 2020-12-05T17:40:31.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f828169f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f827c1557a0 0x7f827c150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:31.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f828169f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f827c1557a0 0x7f827c150b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39694/0 (socket says 172.21.15.32:39694) 2020-12-05T17:40:31.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f828169f700 1 -- 172.21.15.32:0/1455481913 learned_addr learned my addr 172.21.15.32:0/1455481913 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:31.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f8280e9e700 1 --2- 172.21.15.32:0/1455481913 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f827c066610 0x7f827c154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:31.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f827bfff700 1 --2- 172.21.15.32:0/1455481913 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f827c0a28d0 0x7f827c1552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:31.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f828169f700 1 -- 172.21.15.32:0/1455481913 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f827c0a28d0 msgr2=0x7f827c1552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:31.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f828169f700 1 --2- 172.21.15.32:0/1455481913 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f827c0a28d0 0x7f827c1552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:31.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f828169f700 1 -- 172.21.15.32:0/1455481913 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f827c066610 msgr2=0x7f827c154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:31.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f828169f700 1 --2- 172.21.15.32:0/1455481913 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f827c066610 0x7f827c154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:31.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f828169f700 1 -- 172.21.15.32:0/1455481913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c0a54e0 con 0x7f827c1557a0 2020-12-05T17:40:31.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f827bfff700 1 --2- 172.21.15.32:0/1455481913 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f827c0a28d0 0x7f827c1552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:40:31.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f8280e9e700 1 --2- 172.21.15.32:0/1455481913 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f827c066610 0x7f827c154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:40:31.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f828169f700 1 --2- 172.21.15.32:0/1455481913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f827c1557a0 0x7f827c150b60 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7f827c0a5da0 tx=0x7f8274001360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:31.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f8279ffb700 1 -- 172.21.15.32:0/1455481913 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f827400cb40 con 0x7f827c1557a0 2020-12-05T17:40:31.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f8279ffb700 1 -- 172.21.15.32:0/1455481913 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827400cca0 con 0x7f827c1557a0 2020-12-05T17:40:31.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f8283102700 1 -- 172.21.15.32:0/1455481913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f827c0a5780 con 0x7f827c1557a0 2020-12-05T17:40:31.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f8279ffb700 1 -- 172.21.15.32:0/1455481913 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8274014430 con 0x7f827c1557a0 2020-12-05T17:40:31.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.247+0000 7f8283102700 1 -- 172.21.15.32:0/1455481913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f827c0a2e50 con 0x7f827c1557a0 2020-12-05T17:40:31.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.251+0000 7f8283102700 1 -- 172.21.15.32:0/1455481913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f827c057050 con 0x7f827c1557a0 2020-12-05T17:40:31.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.251+0000 7f8279ffb700 1 -- 172.21.15.32:0/1455481913 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f82740133e0 con 0x7f827c1557a0 2020-12-05T17:40:31.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.251+0000 7f8279ffb700 1 --2- 172.21.15.32:0/1455481913 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f825c0553b0 0x7f825c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:31.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.251+0000 7f8280e9e700 1 --2- 172.21.15.32:0/1455481913 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f825c0553b0 0x7f825c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:31.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.251+0000 7f8279ffb700 1 -- 172.21.15.32:0/1455481913 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f827406df90 con 0x7f827c1557a0 2020-12-05T17:40:31.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.251+0000 7f8280e9e700 1 --2- 172.21.15.32:0/1455481913 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f825c0553b0 0x7f825c057870 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f826c019000 tx=0x7f826c017000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:31.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.259+0000 7f8279ffb700 1 -- 172.21.15.32:0/1455481913 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8274043fe0 con 0x7f827c1557a0 2020-12-05T17:40:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:31.447+0000 7f8283102700 1 -- 172.21.15.32:0/1455481913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f827c05bae0 con 0x7f827c1557a0 2020-12-05T17:40:32.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.055+0000 7f8279ffb700 1 -- 172.21.15.32:0/1455481913 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v54) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8274047600 con 0x7f827c1557a0 2020-12-05T17:40:32.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.059+0000 7f8283102700 1 -- 172.21.15.32:0/1455481913 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f825c0553b0 msgr2=0x7f825c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:32.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.059+0000 7f8283102700 1 --2- 172.21.15.32:0/1455481913 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f825c0553b0 0x7f825c057870 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f826c019000 tx=0x7f826c017000).stop 2020-12-05T17:40:32.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.059+0000 7f8283102700 1 -- 172.21.15.32:0/1455481913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f827c1557a0 msgr2=0x7f827c150b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:32.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.059+0000 7f8283102700 1 --2- 172.21.15.32:0/1455481913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f827c1557a0 0x7f827c150b60 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7f827c0a5da0 tx=0x7f8274001360).stop 2020-12-05T17:40:32.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.059+0000 7f8283102700 1 -- 172.21.15.32:0/1455481913 shutdown_connections 2020-12-05T17:40:32.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.059+0000 7f8283102700 1 --2- 172.21.15.32:0/1455481913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f827c1557a0 0x7f827c150b60 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:32.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.059+0000 7f8283102700 1 --2- 172.21.15.32:0/1455481913 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f827c066610 0x7f827c154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:32.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.059+0000 7f8283102700 1 --2- 172.21.15.32:0/1455481913 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f825c0553b0 0x7f825c057870 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:32.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.059+0000 7f8283102700 1 --2- 172.21.15.32:0/1455481913 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f827c0a28d0 0x7f827c1552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:32.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.059+0000 7f8283102700 1 -- 172.21.15.32:0/1455481913 >> 172.21.15.32:0/1455481913 conn(0x7f827c05f3b0 msgr2=0x7f827c143110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:32.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.059+0000 7f8283102700 1 -- 172.21.15.32:0/1455481913 shutdown_connections 2020-12-05T17:40:32.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.059+0000 7f8283102700 1 -- 172.21.15.32:0/1455481913 wait complete. 2020-12-05T17:40:32.083 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-12-05T17:40:32.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.207+0000 7f261de6a700 1 Processor -- start 2020-12-05T17:40:32.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.207+0000 7f261de6a700 1 -- start start 2020-12-05T17:40:32.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.207+0000 7f261de6a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f26180af0d0 0x7f261813daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:32.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.207+0000 7f261de6a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f261805f5c0 con 0x7f26180b0d70 2020-12-05T17:40:32.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.207+0000 7f261de6a700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26180af4b0 con 0x7f261813df90 2020-12-05T17:40:32.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.207+0000 7f261de6a700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f261805f7f0 con 0x7f26180af0d0 2020-12-05T17:40:32.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.207+0000 7f261d669700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f26180b0d70 0x7f2618063f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42352/0 (socket says 172.21.15.32:42352) 2020-12-05T17:40:32.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.207+0000 7f261d669700 1 -- 172.21.15.32:0/3827825193 learned_addr learned my addr 172.21.15.32:0/3827825193 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:32.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.207+0000 7f261ce68700 1 --2- 172.21.15.32:0/3827825193 >> v2:172.21.15.61:3300/0 conn(0x7f26180af0d0 0x7f261813daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:32.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.211+0000 7f260f7fe700 1 -- 172.21.15.32:0/3827825193 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045199810 0 0) 0x7f261805f5c0 con 0x7f26180b0d70 2020-12-05T17:40:32.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.211+0000 7f260f7fe700 1 -- 172.21.15.32:0/3827825193 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2604003230 con 0x7f26180b0d70 2020-12-05T17:40:32.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.211+0000 7f261ce68700 1 -- 172.21.15.32:0/3827825193 >> v1:172.21.15.61:6789/0 conn(0x7f261813df90 legacy=0x7f26180637f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:32.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.211+0000 7f260f7fe700 1 -- 172.21.15.32:0/3827825193 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3073589193 0 0) 0x7f2604003230 con 0x7f26180b0d70 2020-12-05T17:40:32.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.211+0000 7f261ce68700 1 -- 172.21.15.32:0/3827825193 >> v1:172.21.15.32:6789/0 conn(0x7f26180b0d70 legacy=0x7f2618063f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:32.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.211+0000 7f261ce68700 1 -- 172.21.15.32:0/3827825193 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2618066ae0 con 0x7f26180af0d0 2020-12-05T17:40:32.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.211+0000 7f261ce68700 1 --2- 172.21.15.32:0/3827825193 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26180af0d0 0x7f261813daa0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f2610001bf0 tx=0x7f2614006dc0).ready entity=mon.1 client_cookie=7c93a60910a6a347 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:32.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.211+0000 7f260f7fe700 1 -- 172.21.15.32:0/3827825193 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2610001a50 con 0x7f26180af0d0 2020-12-05T17:40:32.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.211+0000 7f260f7fe700 1 -- 172.21.15.32:0/3827825193 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2614009670 con 0x7f26180af0d0 2020-12-05T17:40:32.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.211+0000 7f261de6a700 1 -- 172.21.15.32:0/3827825193 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26180af0d0 msgr2=0x7f261813daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:32.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.211+0000 7f261de6a700 1 --2- 172.21.15.32:0/3827825193 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26180af0d0 0x7f261813daa0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f2610001bf0 tx=0x7f2614006dc0).stop 2020-12-05T17:40:32.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.211+0000 7f261de6a700 1 -- 172.21.15.32:0/3827825193 shutdown_connections 2020-12-05T17:40:32.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.211+0000 7f261de6a700 1 --2- 172.21.15.32:0/3827825193 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26180af0d0 0x7f261813daa0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:32.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.211+0000 7f261de6a700 1 -- 172.21.15.32:0/3827825193 >> 172.21.15.32:0/3827825193 conn(0x7f26180620b0 msgr2=0x7f2618062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:32.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.211+0000 7f261de6a700 1 -- 172.21.15.32:0/3827825193 shutdown_connections 2020-12-05T17:40:32.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.211+0000 7f261de6a700 1 -- 172.21.15.32:0/3827825193 wait complete. 2020-12-05T17:40:32.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.211+0000 7f261de6a700 1 Processor -- start 2020-12-05T17:40:32.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.211+0000 7f261de6a700 1 -- start start 2020-12-05T17:40:32.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.215+0000 7f261de6a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26180af0d0 0x7f2618143520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:32.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.215+0000 7f261de6a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26180b0d70 0x7f26181423f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:32.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.215+0000 7f261de6a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f261813df90 0x7f261813cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:32.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.215+0000 7f261de6a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2618143a10 con 0x7f26180b0d70 2020-12-05T17:40:32.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.215+0000 7f261de6a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2618143c30 con 0x7f261813df90 2020-12-05T17:40:32.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.215+0000 7f261de6a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2618143ed0 con 0x7f26180af0d0 2020-12-05T17:40:32.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.215+0000 7f260ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26180b0d70 0x7f26181423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:32.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.215+0000 7f260ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26180b0d70 0x7f26181423f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39706/0 (socket says 172.21.15.32:39706) 2020-12-05T17:40:32.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.215+0000 7f260ffff700 1 -- 172.21.15.32:0/2664187465 learned_addr learned my addr 172.21.15.32:0/2664187465 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:32.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.215+0000 7f261ce68700 1 --2- 172.21.15.32:0/2664187465 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26180af0d0 0x7f2618143520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:32.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.215+0000 7f261d669700 1 --2- 172.21.15.32:0/2664187465 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f261813df90 0x7f261813cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.215+0000 7f260ffff700 1 -- 172.21.15.32:0/2664187465 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26180af0d0 msgr2=0x7f2618143520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.215+0000 7f260ffff700 1 --2- 172.21.15.32:0/2664187465 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26180af0d0 0x7f2618143520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.215+0000 7f260ffff700 1 -- 172.21.15.32:0/2664187465 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f261813df90 msgr2=0x7f261813cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.215+0000 7f260ffff700 1 --2- 172.21.15.32:0/2664187465 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f261813df90 0x7f261813cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.215+0000 7f260ffff700 1 -- 172.21.15.32:0/2664187465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2618144170 con 0x7f26180b0d70 2020-12-05T17:40:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.215+0000 7f260ffff700 1 --2- 172.21.15.32:0/2664187465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26180b0d70 0x7f26181423f0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7f2608000e90 tx=0x7f2608007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.215+0000 7f260dffb700 1 -- 172.21.15.32:0/2664187465 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2608013410 con 0x7f26180b0d70 2020-12-05T17:40:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.215+0000 7f260dffb700 1 -- 172.21.15.32:0/2664187465 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2608013570 con 0x7f26180b0d70 2020-12-05T17:40:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.215+0000 7f261de6a700 1 -- 172.21.15.32:0/2664187465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26181428e0 con 0x7f26180b0d70 2020-12-05T17:40:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.215+0000 7f260dffb700 1 -- 172.21.15.32:0/2664187465 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26080138d0 con 0x7f26180b0d70 2020-12-05T17:40:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.215+0000 7f261de6a700 1 -- 172.21.15.32:0/2664187465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f261810f990 con 0x7f26180b0d70 2020-12-05T17:40:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.219+0000 7f261de6a700 1 -- 172.21.15.32:0/2664187465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2618144300 con 0x7f26180b0d70 2020-12-05T17:40:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.219+0000 7f260dffb700 1 -- 172.21.15.32:0/2664187465 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2608010030 con 0x7f26180b0d70 2020-12-05T17:40:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.219+0000 7f260dffb700 1 --2- 172.21.15.32:0/2664187465 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f25f80553b0 0x7f25f8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.219+0000 7f260dffb700 1 -- 172.21.15.32:0/2664187465 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f260806dea0 con 0x7f26180b0d70 2020-12-05T17:40:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.219+0000 7f261ce68700 1 --2- 172.21.15.32:0/2664187465 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f25f80553b0 0x7f25f8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.219+0000 7f261ce68700 1 --2- 172.21.15.32:0/2664187465 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f25f80553b0 0x7f25f8057870 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f261401b000 tx=0x7f2614015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.223+0000 7f260dffb700 1 -- 172.21.15.32:0/2664187465 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2608043ff0 con 0x7f26180b0d70 2020-12-05T17:40:32.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:32.367+0000 7f261de6a700 1 -- 172.21.15.32:0/2664187465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2618057050 con 0x7f26180b0d70 2020-12-05T17:40:33.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.063+0000 7f260dffb700 1 -- 172.21.15.32:0/2664187465 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v114) v1 ==== 185+0+0 (secure 0 0 0) 0x7f2608047610 con 0x7f26180b0d70 2020-12-05T17:40:33.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.067+0000 7f261de6a700 1 -- 172.21.15.32:0/2664187465 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f25f80553b0 msgr2=0x7f25f8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:33.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.067+0000 7f261de6a700 1 --2- 172.21.15.32:0/2664187465 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f25f80553b0 0x7f25f8057870 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f261401b000 tx=0x7f2614015000).stop 2020-12-05T17:40:33.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.067+0000 7f261de6a700 1 -- 172.21.15.32:0/2664187465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26180b0d70 msgr2=0x7f26181423f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:33.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.067+0000 7f261de6a700 1 --2- 172.21.15.32:0/2664187465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26180b0d70 0x7f26181423f0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7f2608000e90 tx=0x7f2608007c20).stop 2020-12-05T17:40:33.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.067+0000 7f261de6a700 1 -- 172.21.15.32:0/2664187465 shutdown_connections 2020-12-05T17:40:33.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.067+0000 7f261de6a700 1 --2- 172.21.15.32:0/2664187465 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f261813df90 0x7f261813cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:33.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.067+0000 7f261de6a700 1 --2- 172.21.15.32:0/2664187465 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f25f80553b0 0x7f25f8057870 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:33.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.067+0000 7f261de6a700 1 --2- 172.21.15.32:0/2664187465 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26180af0d0 0x7f2618143520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:33.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.067+0000 7f261de6a700 1 --2- 172.21.15.32:0/2664187465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26180b0d70 0x7f26181423f0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:33.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.067+0000 7f261de6a700 1 -- 172.21.15.32:0/2664187465 >> 172.21.15.32:0/2664187465 conn(0x7f26180620b0 msgr2=0x7f26180b4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:33.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.067+0000 7f261de6a700 1 -- 172.21.15.32:0/2664187465 shutdown_connections 2020-12-05T17:40:33.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.067+0000 7f261de6a700 1 -- 172.21.15.32:0/2664187465 wait complete. 2020-12-05T17:40:33.076 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2020-12-05T17:40:33.088 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-12-05T17:40:33.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.211+0000 7fb0914ae700 1 Processor -- start 2020-12-05T17:40:33.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.211+0000 7fb0914ae700 1 -- start start 2020-12-05T17:40:33.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.211+0000 7fb0914ae700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb08c05f3b0 0x7fb08c05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:33.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb0914ae700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb08c05feb0 0x7fb08c061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:33.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb0914ae700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb08c0b2df0 con 0x7fb08c0b2fb0 2020-12-05T17:40:33.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb0914ae700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb08c0b3c90 con 0x7fb08c05f3b0 2020-12-05T17:40:33.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb0914ae700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb08c0b2b50 con 0x7fb08c05feb0 2020-12-05T17:40:33.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb08bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb08c05f3b0 0x7fb08c05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:33.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb08bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb08c05f3b0 0x7fb08c05f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39714/0 (socket says 172.21.15.32:39714) 2020-12-05T17:40:33.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb08bfff700 1 -- 172.21.15.32:0/1301206144 learned_addr learned my addr 172.21.15.32:0/1301206144 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:33.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb08b7fe700 1 --2- 172.21.15.32:0/1301206144 >> v2:172.21.15.61:3300/0 conn(0x7fb08c05feb0 0x7fb08c061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:33.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb08bfff700 1 -- 172.21.15.32:0/1301206144 >> v2:172.21.15.61:3300/0 conn(0x7fb08c05feb0 msgr2=0x7fb08c061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:33.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb08bfff700 1 --2- 172.21.15.32:0/1301206144 >> v2:172.21.15.61:3300/0 conn(0x7fb08c05feb0 0x7fb08c061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:33.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb08bfff700 1 -- 172.21.15.32:0/1301206144 >> v1:172.21.15.61:6789/0 conn(0x7fb08c0b2fb0 legacy=0x7fb08c13c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:33.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb08bfff700 1 -- 172.21.15.32:0/1301206144 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08c142820 con 0x7fb08c05f3b0 2020-12-05T17:40:33.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb08bfff700 1 --2- 172.21.15.32:0/1301206144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb08c05f3b0 0x7fb08c05f790 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0x7fb07c002f30 tx=0x7fb07c006e80).ready entity=mon.0 client_cookie=f25f40004d4c8c65 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:33.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb08affd700 1 -- 172.21.15.32:0/1301206144 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb07c0097f0 con 0x7fb08c05f3b0 2020-12-05T17:40:33.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb08affd700 1 -- 172.21.15.32:0/1301206144 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb07c009950 con 0x7fb08c05f3b0 2020-12-05T17:40:33.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb08affd700 1 -- 172.21.15.32:0/1301206144 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb07c009cb0 con 0x7fb08c05f3b0 2020-12-05T17:40:33.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb0914ae700 1 -- 172.21.15.32:0/1301206144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb08c05f3b0 msgr2=0x7fb08c05f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:33.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb0914ae700 1 --2- 172.21.15.32:0/1301206144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb08c05f3b0 0x7fb08c05f790 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0x7fb07c002f30 tx=0x7fb07c006e80).stop 2020-12-05T17:40:33.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb0914ae700 1 -- 172.21.15.32:0/1301206144 shutdown_connections 2020-12-05T17:40:33.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb0914ae700 1 --2- 172.21.15.32:0/1301206144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb08c05f3b0 0x7fb08c05f790 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:33.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb0914ae700 1 --2- 172.21.15.32:0/1301206144 >> v2:172.21.15.61:3300/0 conn(0x7fb08c05feb0 0x7fb08c061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:33.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb0914ae700 1 -- 172.21.15.32:0/1301206144 >> 172.21.15.32:0/1301206144 conn(0x7fb08c0b8f20 msgr2=0x7fb08c0b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:33.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb0914ae700 1 -- 172.21.15.32:0/1301206144 shutdown_connections 2020-12-05T17:40:33.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.215+0000 7fb0914ae700 1 -- 172.21.15.32:0/1301206144 wait complete. 2020-12-05T17:40:33.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb0914ae700 1 Processor -- start 2020-12-05T17:40:33.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb0914ae700 1 -- start start 2020-12-05T17:40:33.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb0914ae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb08c05f3b0 0x7fb08c13e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:33.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb0914ae700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb08c05feb0 0x7fb08c146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:33.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb0914ae700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb08c0b2fb0 0x7fb08c146600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:33.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb0914ae700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c141750 con 0x7fb08c05f3b0 2020-12-05T17:40:33.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb0914ae700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c141210 con 0x7fb08c0b2fb0 2020-12-05T17:40:33.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb0914ae700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c1414b0 con 0x7fb08c05feb0 2020-12-05T17:40:33.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb08bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb08c05f3b0 0x7fb08c13e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:33.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb08bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb08c05f3b0 0x7fb08c13e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39720/0 (socket says 172.21.15.32:39720) 2020-12-05T17:40:33.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb08bfff700 1 -- 172.21.15.32:0/4249408580 learned_addr learned my addr 172.21.15.32:0/4249408580 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:33.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb090cad700 1 --2- 172.21.15.32:0/4249408580 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb08c0b2fb0 0x7fb08c146600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:33.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb08b7fe700 1 --2- 172.21.15.32:0/4249408580 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb08c05feb0 0x7fb08c146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:33.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb08bfff700 1 -- 172.21.15.32:0/4249408580 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb08c05feb0 msgr2=0x7fb08c146110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:33.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb08bfff700 1 --2- 172.21.15.32:0/4249408580 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb08c05feb0 0x7fb08c146110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:33.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb08bfff700 1 -- 172.21.15.32:0/4249408580 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb08c0b2fb0 msgr2=0x7fb08c146600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:33.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb08bfff700 1 --2- 172.21.15.32:0/4249408580 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb08c0b2fb0 0x7fb08c146600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:33.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb08bfff700 1 -- 172.21.15.32:0/4249408580 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08c144950 con 0x7fb08c05f3b0 2020-12-05T17:40:33.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb08bfff700 1 --2- 172.21.15.32:0/4249408580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb08c05f3b0 0x7fb08c13e770 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7fb07c002f30 tx=0x7fb07c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:33.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb0897fa700 1 -- 172.21.15.32:0/4249408580 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb07c011c30 con 0x7fb08c05f3b0 2020-12-05T17:40:33.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb0897fa700 1 -- 172.21.15.32:0/4249408580 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb07c011d90 con 0x7fb08c05f3b0 2020-12-05T17:40:33.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb0914ae700 1 -- 172.21.15.32:0/4249408580 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb08c143b70 con 0x7fb08c05f3b0 2020-12-05T17:40:33.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb0897fa700 1 -- 172.21.15.32:0/4249408580 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb07c01e650 con 0x7fb08c05f3b0 2020-12-05T17:40:33.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.219+0000 7fb0914ae700 1 -- 172.21.15.32:0/4249408580 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb08c142330 con 0x7fb08c05f3b0 2020-12-05T17:40:33.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.223+0000 7fb0914ae700 1 -- 172.21.15.32:0/4249408580 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb08c145e50 con 0x7fb08c05f3b0 2020-12-05T17:40:33.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.223+0000 7fb0897fa700 1 -- 172.21.15.32:0/4249408580 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb07c01a040 con 0x7fb08c05f3b0 2020-12-05T17:40:33.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.223+0000 7fb0897fa700 1 --2- 172.21.15.32:0/4249408580 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb06c055330 0x7fb06c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:33.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.223+0000 7fb0897fa700 1 -- 172.21.15.32:0/4249408580 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6068+0+0 (secure 0 0 0) 0x7fb07c06f070 con 0x7fb08c05f3b0 2020-12-05T17:40:33.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.223+0000 7fb08b7fe700 1 --2- 172.21.15.32:0/4249408580 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb06c055330 0x7fb06c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:33.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.223+0000 7fb08b7fe700 1 --2- 172.21.15.32:0/4249408580 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb06c055330 0x7fb06c0577f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7fb08000b000 tx=0x7fb080006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:33.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.227+0000 7fb0897fa700 1 -- 172.21.15.32:0/4249408580 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb08c145e50 con 0x7fb08c05f3b0 2020-12-05T17:40:33.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:33.371+0000 7fb0914ae700 1 -- 172.21.15.32:0/4249408580 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fb08c145e50 con 0x7fb08c05f3b0 2020-12-05T17:40:34.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:34.071+0000 7fb0897fa700 1 -- 172.21.15.32:0/4249408580 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v115) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb08c145e50 con 0x7fb08c05f3b0 2020-12-05T17:40:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:34.079+0000 7fb0914ae700 1 -- 172.21.15.32:0/4249408580 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb06c055330 msgr2=0x7fb06c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:34.079+0000 7fb0914ae700 1 --2- 172.21.15.32:0/4249408580 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb06c055330 0x7fb06c0577f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7fb08000b000 tx=0x7fb080006dc0).stop 2020-12-05T17:40:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:34.079+0000 7fb0914ae700 1 -- 172.21.15.32:0/4249408580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb08c05f3b0 msgr2=0x7fb08c13e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:34.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:34.079+0000 7fb0914ae700 1 --2- 172.21.15.32:0/4249408580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb08c05f3b0 0x7fb08c13e770 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7fb07c002f30 tx=0x7fb07c015000).stop 2020-12-05T17:40:34.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:34.079+0000 7fb0914ae700 1 -- 172.21.15.32:0/4249408580 shutdown_connections 2020-12-05T17:40:34.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:34.079+0000 7fb0914ae700 1 --2- 172.21.15.32:0/4249408580 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb08c0b2fb0 0x7fb08c146600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:34.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:34.079+0000 7fb0914ae700 1 --2- 172.21.15.32:0/4249408580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb08c05f3b0 0x7fb08c13e770 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:34.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:34.079+0000 7fb0914ae700 1 --2- 172.21.15.32:0/4249408580 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb06c055330 0x7fb06c0577f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:34.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:34.079+0000 7fb0914ae700 1 --2- 172.21.15.32:0/4249408580 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb08c05feb0 0x7fb08c146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:34.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:34.079+0000 7fb0914ae700 1 -- 172.21.15.32:0/4249408580 >> 172.21.15.32:0/4249408580 conn(0x7fb08c0b8f20 msgr2=0x7fb08c061780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:34.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:34.079+0000 7fb0914ae700 1 -- 172.21.15.32:0/4249408580 shutdown_connections 2020-12-05T17:40:34.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:34.079+0000 7fb0914ae700 1 -- 172.21.15.32:0/4249408580 wait complete. 2020-12-05T17:40:34.087 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2020-12-05T17:40:34.097 INFO:teuthology.orchestra.run.smithi036:> sudo rados -p cephfs_metadata put key /etc/fstab 2020-12-05T17:40:34.141 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69e0225c00 1 Processor -- start 2020-12-05T17:40:34.141 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69e0225c00 1 -- start start 2020-12-05T17:40:34.142 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69e0225c00 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x56209ff76a10 0x56209ff80830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:34.142 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69e0225c00 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209ff773b0 0x56209ff80d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:34.142 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69e0225c00 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56209ff7caa0 0x56209ff8b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:34.142 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69e0225c00 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56209fe77450 con 0x56209ff773b0 2020-12-05T17:40:34.143 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69e0225c00 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x56209fe68450 con 0x56209ff7caa0 2020-12-05T17:40:34.143 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69e0225c00 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x56209ff8bbe0 con 0x56209ff76a10 2020-12-05T17:40:34.143 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69d2119700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x56209ff76a10 0x56209ff80830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:34.143 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69d291a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56209ff7caa0 0x56209ff8b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:34.144 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69d2119700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x56209ff76a10 0x56209ff80830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.36:39080/0 (socket says 172.21.15.36:39080) 2020-12-05T17:40:34.144 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69d2119700 1 -- 172.21.15.36:0/3385120841 learned_addr learned my addr 172.21.15.36:0/3385120841 (peer_addr_for_me v2:172.21.15.36:0/0) 2020-12-05T17:40:34.144 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69d1918700 1 --2- 172.21.15.36:0/3385120841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209ff773b0 0x56209ff80d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:34.150 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69d291a700 1 -- 172.21.15.36:0/3385120841 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x56209ff76a10 msgr2=0x56209ff80830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:34.150 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69d291a700 1 --2- 172.21.15.36:0/3385120841 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x56209ff76a10 0x56209ff80830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:34.150 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69d291a700 1 -- 172.21.15.36:0/3385120841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209ff773b0 msgr2=0x56209ff80d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:34.151 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69d291a700 1 --2- 172.21.15.36:0/3385120841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209ff773b0 0x56209ff80d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:34.151 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69d291a700 1 -- 172.21.15.36:0/3385120841 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56209fe8b740 con 0x56209ff7caa0 2020-12-05T17:40:34.151 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69d2119700 1 --2- 172.21.15.36:0/3385120841 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x56209ff76a10 0x56209ff80830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:40:34.151 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69d291a700 1 --2- 172.21.15.36:0/3385120841 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56209ff7caa0 0x56209ff8b6f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f69c4001190 tx=0x7f69c40016c0).ready entity=mon.1 client_cookie=847d3b65619a2da3 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:34.151 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69c3fff700 1 -- 172.21.15.36:0/3385120841 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69c400e430 con 0x56209ff7caa0 2020-12-05T17:40:34.152 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69c3fff700 1 -- 172.21.15.36:0/3385120841 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c400e590 con 0x56209ff7caa0 2020-12-05T17:40:34.152 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69e0225c00 1 -- 172.21.15.36:0/3385120841 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56209ff7caa0 msgr2=0x56209ff8b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:34.152 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69e0225c00 1 --2- 172.21.15.36:0/3385120841 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56209ff7caa0 0x56209ff8b6f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f69c4001190 tx=0x7f69c40016c0).stop 2020-12-05T17:40:34.152 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69c3fff700 1 -- 172.21.15.36:0/3385120841 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69c400e8f0 con 0x56209ff7caa0 2020-12-05T17:40:34.152 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69e0225c00 1 -- 172.21.15.36:0/3385120841 shutdown_connections 2020-12-05T17:40:34.153 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69e0225c00 1 --2- 172.21.15.36:0/3385120841 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56209ff7caa0 0x56209ff8b6f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:34.153 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69e0225c00 1 --2- 172.21.15.36:0/3385120841 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x56209ff76a10 0x56209ff80830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:34.153 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69e0225c00 1 --2- 172.21.15.36:0/3385120841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209ff773b0 0x56209ff80d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:34.153 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69e0225c00 1 -- 172.21.15.36:0/3385120841 >> 172.21.15.36:0/3385120841 conn(0x56209feea5d0 msgr2=0x56209ff7a090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:34.153 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69e0225c00 1 -- 172.21.15.36:0/3385120841 shutdown_connections 2020-12-05T17:40:34.153 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69e0225c00 1 -- 172.21.15.36:0/3385120841 wait complete. 2020-12-05T17:40:34.154 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.138+0000 7f69e0225c00 1 Processor -- start 2020-12-05T17:40:34.154 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69e0225c00 1 -- start start 2020-12-05T17:40:34.154 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69e0225c00 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209ff76a10 0x56209ff8b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:34.154 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69e0225c00 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x56209ff773b0 0x56209ff864a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:34.155 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69e0225c00 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56209ff7caa0 0x56209ff86990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:34.155 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69e0225c00 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56209ff76070 con 0x56209ff76a10 2020-12-05T17:40:34.155 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69e0225c00 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x56209ff86f70 con 0x56209ff7caa0 2020-12-05T17:40:34.155 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69e0225c00 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x56209ff87130 con 0x56209ff773b0 2020-12-05T17:40:34.155 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69d1918700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x56209ff773b0 0x56209ff864a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:34.155 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69d2119700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209ff76a10 0x56209ff8b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:34.156 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69d291a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56209ff7caa0 0x56209ff86990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:34.156 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69d1918700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x56209ff773b0 0x56209ff864a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.36:39088/0 (socket says 172.21.15.36:39088) 2020-12-05T17:40:34.156 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69d2119700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209ff76a10 0x56209ff8b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.36:43442/0 (socket says 172.21.15.36:43442) 2020-12-05T17:40:34.156 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69d2119700 1 -- 172.21.15.36:0/1477649712 learned_addr learned my addr 172.21.15.36:0/1477649712 (peer_addr_for_me v2:172.21.15.36:0/0) 2020-12-05T17:40:34.157 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69d2119700 1 -- 172.21.15.36:0/1477649712 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x56209ff773b0 msgr2=0x56209ff864a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:34.157 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69d2119700 1 --2- 172.21.15.36:0/1477649712 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x56209ff773b0 0x56209ff864a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:34.157 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69d2119700 1 -- 172.21.15.36:0/1477649712 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56209ff7caa0 msgr2=0x56209ff86990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:34.157 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69d2119700 1 --2- 172.21.15.36:0/1477649712 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56209ff7caa0 0x56209ff86990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:34.157 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69d2119700 1 -- 172.21.15.36:0/1477649712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56209fe611e0 con 0x56209ff76a10 2020-12-05T17:40:34.157 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69d291a700 1 --2- 172.21.15.36:0/1477649712 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56209ff7caa0 0x56209ff86990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:40:34.158 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69d2119700 1 --2- 172.21.15.36:0/1477649712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209ff76a10 0x56209ff8b350 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7f69cc007000 tx=0x7f69cc007040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:34.158 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69c37fe700 1 -- 172.21.15.36:0/1477649712 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69cc00edd0 con 0x56209ff76a10 2020-12-05T17:40:34.158 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69c37fe700 1 -- 172.21.15.36:0/1477649712 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69cc013440 con 0x56209ff76a10 2020-12-05T17:40:34.158 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69e0225c00 1 -- 172.21.15.36:0/1477649712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56209ff92dc0 con 0x56209ff76a10 2020-12-05T17:40:34.159 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69c37fe700 1 -- 172.21.15.36:0/1477649712 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69cc0135b0 con 0x56209ff76a10 2020-12-05T17:40:34.159 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.142+0000 7f69e0225c00 1 -- 172.21.15.36:0/1477649712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56209ff93110 con 0x56209ff76a10 2020-12-05T17:40:34.159 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.146+0000 7f69c37fe700 1 -- 172.21.15.36:0/1477649712 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f69cc023070 con 0x56209ff76a10 2020-12-05T17:40:34.159 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.146+0000 7f69c37fe700 1 --2- 172.21.15.36:0/1477649712 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f69b4054030 0x7f69b40564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:34.160 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.146+0000 7f69c37fe700 1 -- 172.21.15.36:0/1477649712 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f69cc06de00 con 0x56209ff76a10 2020-12-05T17:40:34.160 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.146+0000 7f69d1918700 1 --2- 172.21.15.36:0/1477649712 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f69b4054030 0x7f69b40564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:34.161 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.146+0000 7f69e0225c00 1 --2- 172.21.15.36:0/1477649712 >> [v2:172.21.15.32:6802/11526,v1:172.21.15.32:6803/11526] conn(0x56209ff93f60 0x56209ff96350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:34.161 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.146+0000 7f69e0225c00 1 -- 172.21.15.36:0/1477649712 --> [v2:172.21.15.32:6802/11526,v1:172.21.15.32:6803/11526] -- osd_op(unknown.0.0:1 23.0 23:0620e6f9:::key:head [writefull 0~617 in=617b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x56209ff968e0 con 0x56209ff93f60 2020-12-05T17:40:34.162 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.146+0000 7f69d291a700 1 --2- 172.21.15.36:0/1477649712 >> [v2:172.21.15.32:6802/11526,v1:172.21.15.32:6803/11526] conn(0x56209ff93f60 0x56209ff96350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:34.162 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.146+0000 7f69d1918700 1 --2- 172.21.15.36:0/1477649712 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f69b4054030 0x7f69b40564f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f69c8002000 tx=0x7f69c8015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:34.162 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:34.146+0000 7f69d291a700 1 --2- 172.21.15.36:0/1477649712 >> [v2:172.21.15.32:6802/11526,v1:172.21.15.32:6803/11526] conn(0x56209ff93f60 0x56209ff96350 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:35.402 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:35.398+0000 7f69c37fe700 1 -- 172.21.15.36:0/1477649712 <== osd.0 v2:172.21.15.32:6802/11526 1 ==== osd_map(116..116 src has 1..116) v4 ==== 320+0+0 (crc 0 0 0) 0x7f69c400ea00 con 0x56209ff93f60 2020-12-05T17:40:35.409 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:35.402+0000 7f69d291a700 1 -- 172.21.15.36:0/1477649712 <== osd.0 v2:172.21.15.32:6802/11526 2 ==== osd_op_reply(1 key [writefull 0~617] v116'1 uv1 ondisk = 0) v8 ==== 147+0+0 (crc 0 0 0) 0x7f69c4006030 con 0x56209ff93f60 2020-12-05T17:40:35.409 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:35.402+0000 7f69e0225c00 1 -- 172.21.15.36:0/1477649712 >> [v2:172.21.15.32:6802/11526,v1:172.21.15.32:6803/11526] conn(0x56209ff93f60 msgr2=0x56209ff96350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:35.409 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:35.402+0000 7f69e0225c00 1 --2- 172.21.15.36:0/1477649712 >> [v2:172.21.15.32:6802/11526,v1:172.21.15.32:6803/11526] conn(0x56209ff93f60 0x56209ff96350 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:35.409 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:35.402+0000 7f69e0225c00 1 -- 172.21.15.36:0/1477649712 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f69b4054030 msgr2=0x7f69b40564f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:35.410 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:35.402+0000 7f69e0225c00 1 --2- 172.21.15.36:0/1477649712 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f69b4054030 0x7f69b40564f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f69c8002000 tx=0x7f69c8015000).stop 2020-12-05T17:40:35.410 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:35.402+0000 7f69e0225c00 1 -- 172.21.15.36:0/1477649712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209ff76a10 msgr2=0x56209ff8b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:35.410 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:35.402+0000 7f69e0225c00 1 --2- 172.21.15.36:0/1477649712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209ff76a10 0x56209ff8b350 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7f69cc007000 tx=0x7f69cc007040).stop 2020-12-05T17:40:35.410 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:35.402+0000 7f69d291a700 1 -- 172.21.15.36:0/1477649712 reap_dead start 2020-12-05T17:40:35.411 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:35.402+0000 7f69e0225c00 1 -- 172.21.15.36:0/1477649712 shutdown_connections 2020-12-05T17:40:35.411 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:35.402+0000 7f69e0225c00 1 -- 172.21.15.36:0/1477649712 >> 172.21.15.36:0/1477649712 conn(0x56209feea5d0 msgr2=0x56209ff7a090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:35.411 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:35.402+0000 7f69e0225c00 1 -- 172.21.15.36:0/1477649712 shutdown_connections 2020-12-05T17:40:35.412 INFO:teuthology.orchestra.run.smithi036.stderr:2020-12-05T17:40:35.402+0000 7f69e0225c00 1 -- 172.21.15.36:0/1477649712 wait complete. 2020-12-05T17:40:35.413 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early df --format=json-pretty 2020-12-05T17:40:35.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a9688700 1 Processor -- start 2020-12-05T17:40:35.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a9688700 1 -- start start 2020-12-05T17:40:35.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a9688700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f54a40afd40 0x7f54a40b5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:35.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a9688700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54a413f8a0 0x7f54a413fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:35.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a9688700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54a40b0460 con 0x7f54a40b5330 2020-12-05T17:40:35.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a9688700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54a41438f0 con 0x7f54a40afd40 2020-12-05T17:40:35.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a9688700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a40b0690 con 0x7f54a413f8a0 2020-12-05T17:40:35.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a37fe700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f54a40afd40 0x7f54a40b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:35.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a8e87700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54a413f8a0 0x7f54a413fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:35.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a37fe700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f54a40afd40 0x7f54a40b5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:59622/0 (socket says 172.21.15.32:59622) 2020-12-05T17:40:35.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a8e87700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54a413f8a0 0x7f54a413fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:36520/0 (socket says 172.21.15.32:36520) 2020-12-05T17:40:35.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a37fe700 1 -- 172.21.15.32:0/3568074960 learned_addr learned my addr 172.21.15.32:0/3568074960 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:35.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a2ffd700 1 -- 172.21.15.32:0/3568074960 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1813465309 0 0) 0x7f54a40b0460 con 0x7f54a40b5330 2020-12-05T17:40:35.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a2ffd700 1 -- 172.21.15.32:0/3568074960 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5484003230 con 0x7f54a40b5330 2020-12-05T17:40:35.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a8e87700 1 -- 172.21.15.32:0/3568074960 >> v2:172.21.15.61:3300/0 conn(0x7f54a40afd40 msgr2=0x7f54a40b5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:35.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a8e87700 1 --2- 172.21.15.32:0/3568074960 >> v2:172.21.15.61:3300/0 conn(0x7f54a40afd40 0x7f54a40b5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:35.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a8e87700 1 -- 172.21.15.32:0/3568074960 >> v1:172.21.15.61:6789/0 conn(0x7f54a40b5330 legacy=0x7f54a40af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:35.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a8e87700 1 -- 172.21.15.32:0/3568074960 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a40ae9f0 con 0x7f54a413f8a0 2020-12-05T17:40:35.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a8e87700 1 --2- 172.21.15.32:0/3568074960 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54a413f8a0 0x7f54a413fd50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f549c0032b0 tx=0x7f549c0091a0).ready entity=mon.2 client_cookie=e0045341f7de4d5a server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:35.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a2ffd700 1 -- 172.21.15.32:0/3568074960 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f549c00b940 con 0x7f54a413f8a0 2020-12-05T17:40:35.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a2ffd700 1 -- 172.21.15.32:0/3568074960 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c00baa0 con 0x7f54a413f8a0 2020-12-05T17:40:35.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a9688700 1 -- 172.21.15.32:0/3568074960 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54a413f8a0 msgr2=0x7f54a413fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:35.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a9688700 1 --2- 172.21.15.32:0/3568074960 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54a413f8a0 0x7f54a413fd50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f549c0032b0 tx=0x7f549c0091a0).stop 2020-12-05T17:40:35.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a9688700 1 -- 172.21.15.32:0/3568074960 shutdown_connections 2020-12-05T17:40:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a9688700 1 --2- 172.21.15.32:0/3568074960 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54a413f8a0 0x7f54a413fd50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a9688700 1 --2- 172.21.15.32:0/3568074960 >> v2:172.21.15.61:3300/0 conn(0x7f54a40afd40 0x7f54a40b5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.527+0000 7f54a9688700 1 -- 172.21.15.32:0/3568074960 >> 172.21.15.32:0/3568074960 conn(0x7f54a4062070 msgr2=0x7f54a4062450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a9688700 1 -- 172.21.15.32:0/3568074960 shutdown_connections 2020-12-05T17:40:35.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a9688700 1 -- 172.21.15.32:0/3568074960 wait complete. 2020-12-05T17:40:35.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a9688700 1 Processor -- start 2020-12-05T17:40:35.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a9688700 1 -- start start 2020-12-05T17:40:35.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a9688700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54a40afd40 0x7f54a4147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:35.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a9688700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f54a40b5330 0x7f54a4143db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:35.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a9688700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a413f8a0 0x7f54a41442a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:35.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a9688700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a413c9a0 con 0x7f54a413f8a0 2020-12-05T17:40:35.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a9688700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a4146af0 con 0x7f54a40b5330 2020-12-05T17:40:35.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a9688700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a410f0b0 con 0x7f54a40afd40 2020-12-05T17:40:35.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a8e87700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a413f8a0 0x7f54a41442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:35.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a8e87700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a413f8a0 0x7f54a41442a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39736/0 (socket says 172.21.15.32:39736) 2020-12-05T17:40:35.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a8e87700 1 -- 172.21.15.32:0/3124957539 learned_addr learned my addr 172.21.15.32:0/3124957539 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:35.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a37fe700 1 --2- 172.21.15.32:0/3124957539 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f54a40b5330 0x7f54a4143db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:35.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a3fff700 1 --2- 172.21.15.32:0/3124957539 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54a40afd40 0x7f54a4147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:35.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a8e87700 1 -- 172.21.15.32:0/3124957539 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54a40afd40 msgr2=0x7f54a4147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:35.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a8e87700 1 --2- 172.21.15.32:0/3124957539 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54a40afd40 0x7f54a4147b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a8e87700 1 -- 172.21.15.32:0/3124957539 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f54a40b5330 msgr2=0x7f54a4143db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a8e87700 1 --2- 172.21.15.32:0/3124957539 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f54a40b5330 0x7f54a4143db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a8e87700 1 -- 172.21.15.32:0/3124957539 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a410f350 con 0x7f54a413f8a0 2020-12-05T17:40:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a8e87700 1 --2- 172.21.15.32:0/3124957539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a413f8a0 0x7f54a41442a0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7f549c0032b0 tx=0x7f549c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a17fa700 1 -- 172.21.15.32:0/3124957539 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f549c011d00 con 0x7f54a413f8a0 2020-12-05T17:40:35.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a17fa700 1 -- 172.21.15.32:0/3124957539 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c011e60 con 0x7f54a413f8a0 2020-12-05T17:40:35.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a9688700 1 -- 172.21.15.32:0/3124957539 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a4146770 con 0x7f54a413f8a0 2020-12-05T17:40:35.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a17fa700 1 -- 172.21.15.32:0/3124957539 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f549c0205f0 con 0x7f54a413f8a0 2020-12-05T17:40:35.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.531+0000 7f54a9688700 1 -- 172.21.15.32:0/3124957539 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a4144790 con 0x7f54a413f8a0 2020-12-05T17:40:35.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.535+0000 7f54a9688700 1 -- 172.21.15.32:0/3124957539 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54a4148450 con 0x7f54a413f8a0 2020-12-05T17:40:35.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.535+0000 7f54a17fa700 1 -- 172.21.15.32:0/3124957539 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f549c027030 con 0x7f54a413f8a0 2020-12-05T17:40:35.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.535+0000 7f54a17fa700 1 --2- 172.21.15.32:0/3124957539 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5488055470 0x7f5488057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:35.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.535+0000 7f54a3fff700 1 --2- 172.21.15.32:0/3124957539 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5488055470 0x7f5488057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:35.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.535+0000 7f54a17fa700 1 -- 172.21.15.32:0/3124957539 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f549c06eff0 con 0x7f54a413f8a0 2020-12-05T17:40:35.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.535+0000 7f54a3fff700 1 --2- 172.21.15.32:0/3124957539 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5488055470 0x7f5488057930 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f549400d000 tx=0x7f5494009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:35.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.543+0000 7f54a17fa700 1 -- 172.21.15.32:0/3124957539 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f54a4148450 con 0x7f54a413f8a0 2020-12-05T17:40:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.739+0000 7f54a9688700 1 -- 172.21.15.32:0/3124957539 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f54a4148450 con 0x7f54a413f8a0 2020-12-05T17:40:35.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.739+0000 7f54a17fa700 1 -- 172.21.15.32:0/3124957539 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1902 (secure 0 0 0) 0x7f549c025780 con 0x7f54a413f8a0 2020-12-05T17:40:35.745 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:40:35.745 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:40:35.746 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2020-12-05T17:40:35.746 INFO:teuthology.orchestra.run.smithi032.stdout: "total_bytes": 773094113280, 2020-12-05T17:40:35.746 INFO:teuthology.orchestra.run.smithi032.stdout: "total_avail_bytes": 764492607488, 2020-12-05T17:40:35.747 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_bytes": 11571200, 2020-12-05T17:40:35.747 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_bytes": 8601505792, 2020-12-05T17:40:35.747 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_ratio": 0.0111260786652565, 2020-12-05T17:40:35.748 INFO:teuthology.orchestra.run.smithi032.stdout: "num_osds": 8, 2020-12-05T17:40:35.748 INFO:teuthology.orchestra.run.smithi032.stdout: "num_per_pool_osds": 8, 2020-12-05T17:40:35.748 INFO:teuthology.orchestra.run.smithi032.stdout: "num_per_pool_omap_osds": 8 2020-12-05T17:40:35.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:35.749 INFO:teuthology.orchestra.run.smithi032.stdout: "stats_by_class": { 2020-12-05T17:40:35.749 INFO:teuthology.orchestra.run.smithi032.stdout: "ssd": { 2020-12-05T17:40:35.750 INFO:teuthology.orchestra.run.smithi032.stdout: "total_bytes": 773094113280, 2020-12-05T17:40:35.750 INFO:teuthology.orchestra.run.smithi032.stdout: "total_avail_bytes": 764492607488, 2020-12-05T17:40:35.750 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_bytes": 11571200, 2020-12-05T17:40:35.751 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_bytes": 8601505792, 2020-12-05T17:40:35.751 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_ratio": 0.0111260786652565 2020-12-05T17:40:35.751 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:35.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:35.752 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2020-12-05T17:40:35.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:35.753 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_health_metrics", 2020-12-05T17:40:35.753 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 1, 2020-12-05T17:40:35.753 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2020-12-05T17:40:35.753 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2020-12-05T17:40:35.754 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 2, 2020-12-05T17:40:35.754 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2020-12-05T17:40:35.754 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2020-12-05T17:40:35.755 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2020-12-05T17:40:35.755 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362918903808 2020-12-05T17:40:35.755 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:35.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:35.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:35.757 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd", 2020-12-05T17:40:35.757 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 2, 2020-12-05T17:40:35.758 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2020-12-05T17:40:35.758 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2020-12-05T17:40:35.758 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 0, 2020-12-05T17:40:35.759 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2020-12-05T17:40:35.759 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2020-12-05T17:40:35.759 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2020-12-05T17:40:35.760 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362918903808 2020-12-05T17:40:35.760 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:35.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:35.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:35.761 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs_data", 2020-12-05T17:40:35.761 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 22, 2020-12-05T17:40:35.761 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2020-12-05T17:40:35.763 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2020-12-05T17:40:35.763 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 0, 2020-12-05T17:40:35.763 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2020-12-05T17:40:35.763 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2020-12-05T17:40:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2020-12-05T17:40:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362918903808 2020-12-05T17:40:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:35.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:35.765 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs_metadata", 2020-12-05T17:40:35.765 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 23, 2020-12-05T17:40:35.765 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2020-12-05T17:40:35.766 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2020-12-05T17:40:35.766 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 0, 2020-12-05T17:40:35.766 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2020-12-05T17:40:35.766 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2020-12-05T17:40:35.766 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2020-12-05T17:40:35.767 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362918903808 2020-12-05T17:40:35.767 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:35.767 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:35.768 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:35.768 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:40:35.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.743+0000 7f54a9688700 1 -- 172.21.15.32:0/3124957539 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5488055470 msgr2=0x7f5488057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:35.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.743+0000 7f54a9688700 1 --2- 172.21.15.32:0/3124957539 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5488055470 0x7f5488057930 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f549400d000 tx=0x7f5494009040).stop 2020-12-05T17:40:35.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.743+0000 7f54a9688700 1 -- 172.21.15.32:0/3124957539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a413f8a0 msgr2=0x7f54a41442a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:35.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.743+0000 7f54a9688700 1 --2- 172.21.15.32:0/3124957539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a413f8a0 0x7f54a41442a0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7f549c0032b0 tx=0x7f549c016000).stop 2020-12-05T17:40:35.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.743+0000 7f54a9688700 1 -- 172.21.15.32:0/3124957539 shutdown_connections 2020-12-05T17:40:35.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.743+0000 7f54a9688700 1 --2- 172.21.15.32:0/3124957539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a413f8a0 0x7f54a41442a0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:35.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.743+0000 7f54a9688700 1 --2- 172.21.15.32:0/3124957539 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5488055470 0x7f5488057930 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:35.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.743+0000 7f54a9688700 1 --2- 172.21.15.32:0/3124957539 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54a40afd40 0x7f54a4147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:35.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.743+0000 7f54a9688700 1 --2- 172.21.15.32:0/3124957539 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f54a40b5330 0x7f54a4143db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:35.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.743+0000 7f54a9688700 1 -- 172.21.15.32:0/3124957539 >> 172.21.15.32:0/3124957539 conn(0x7f54a4062070 msgr2=0x7f54a40adf00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:35.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.743+0000 7f54a9688700 1 -- 172.21.15.32:0/3124957539 shutdown_connections 2020-12-05T17:40:35.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:35.743+0000 7f54a9688700 1 -- 172.21.15.32:0/3124957539 wait complete. 2020-12-05T17:40:35.772 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2020-12-05T17:40:40.774 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early df --format=json-pretty 2020-12-05T17:40:40.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.891+0000 7f79128fc700 1 Processor -- start 2020-12-05T17:40:40.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.891+0000 7f79128fc700 1 -- start start 2020-12-05T17:40:40.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.891+0000 7f79128fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f790c0ad380 0x7f790c066140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:40.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.891+0000 7f79128fc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f790c0afba0 con 0x7f790c0af280 2020-12-05T17:40:40.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.891+0000 7f79128fc700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f790c0b62e0 con 0x7f790c0b5f00 2020-12-05T17:40:40.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.891+0000 7f79128fc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f790c0b2790 con 0x7f790c0ad380 2020-12-05T17:40:40.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.891+0000 7f79120fb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f790c0ad380 0x7f790c066140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:40.898 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.891+0000 7f79120fb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f790c0ad380 0x7f790c066140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39746/0 (socket says 172.21.15.32:39746) 2020-12-05T17:40:40.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.891+0000 7f79120fb700 1 -- 172.21.15.32:0/3790540409 learned_addr learned my addr 172.21.15.32:0/3790540409 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:40.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.891+0000 7f79110f9700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f790c0af280 0x7f790c13f510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42390/0 (socket says 172.21.15.32:42390) 2020-12-05T17:40:40.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.891+0000 7f79120fb700 1 -- 172.21.15.32:0/3790540409 >> v1:172.21.15.61:6789/0 conn(0x7f790c0b5f00 legacy=0x7f790c0b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:40.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.891+0000 7f79120fb700 1 -- 172.21.15.32:0/3790540409 >> v1:172.21.15.32:6789/0 conn(0x7f790c0af280 legacy=0x7f790c13f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:40.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.891+0000 7f79120fb700 1 -- 172.21.15.32:0/3790540409 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790c0b2a30 con 0x7f790c0ad380 2020-12-05T17:40:40.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.891+0000 7f79120fb700 1 --2- 172.21.15.32:0/3790540409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f790c0ad380 0x7f790c066140 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7f7908002f30 tx=0x7f7908006e80).ready entity=mon.0 client_cookie=813735691a4c7b70 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:40.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.895+0000 7f79108f8700 1 -- 172.21.15.32:0/3790540409 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79080097f0 con 0x7f790c0ad380 2020-12-05T17:40:40.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.895+0000 7f79108f8700 1 -- 172.21.15.32:0/3790540409 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7908009950 con 0x7f790c0ad380 2020-12-05T17:40:40.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.895+0000 7f79108f8700 1 -- 172.21.15.32:0/3790540409 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7908009cb0 con 0x7f790c0ad380 2020-12-05T17:40:40.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.895+0000 7f79128fc700 1 -- 172.21.15.32:0/3790540409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f790c0ad380 msgr2=0x7f790c066140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:40.902 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.895+0000 7f79128fc700 1 --2- 172.21.15.32:0/3790540409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f790c0ad380 0x7f790c066140 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7f7908002f30 tx=0x7f7908006e80).stop 2020-12-05T17:40:40.902 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.895+0000 7f79128fc700 1 -- 172.21.15.32:0/3790540409 shutdown_connections 2020-12-05T17:40:40.902 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.895+0000 7f79128fc700 1 --2- 172.21.15.32:0/3790540409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f790c0ad380 0x7f790c066140 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:40.902 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.895+0000 7f79128fc700 1 -- 172.21.15.32:0/3790540409 >> 172.21.15.32:0/3790540409 conn(0x7f790c069cb0 msgr2=0x7f790c0604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:40.903 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.895+0000 7f79128fc700 1 -- 172.21.15.32:0/3790540409 shutdown_connections 2020-12-05T17:40:40.903 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.895+0000 7f79128fc700 1 -- 172.21.15.32:0/3790540409 wait complete. 2020-12-05T17:40:40.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.895+0000 7f79128fc700 1 Processor -- start 2020-12-05T17:40:40.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f79128fc700 1 -- start start 2020-12-05T17:40:40.905 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f79128fc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f790c0ad380 0x7f790c064550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:40.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f79128fc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f790c0af280 0x7f790c05f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:40.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f79128fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f790c0b5f00 0x7f790c13cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:40.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f79128fc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790c064a40 con 0x7f790c0b5f00 2020-12-05T17:40:40.906 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f79128fc700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790c065a90 con 0x7f790c0ad380 2020-12-05T17:40:40.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f79128fc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f790c065d30 con 0x7f790c0af280 2020-12-05T17:40:40.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f79120fb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f790c0b5f00 0x7f790c13cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:40.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f79120fb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f790c0b5f00 0x7f790c13cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39752/0 (socket says 172.21.15.32:39752) 2020-12-05T17:40:40.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f79120fb700 1 -- 172.21.15.32:0/2733217024 learned_addr learned my addr 172.21.15.32:0/2733217024 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:40.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f79118fa700 1 --2- 172.21.15.32:0/2733217024 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f790c0ad380 0x7f790c064550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:40.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f79110f9700 1 --2- 172.21.15.32:0/2733217024 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f790c0af280 0x7f790c05f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:40.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f79120fb700 1 -- 172.21.15.32:0/2733217024 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f790c0af280 msgr2=0x7f790c05f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:40.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f79120fb700 1 --2- 172.21.15.32:0/2733217024 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f790c0af280 0x7f790c05f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:40.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f79120fb700 1 -- 172.21.15.32:0/2733217024 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f790c0ad380 msgr2=0x7f790c064550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:40.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f79120fb700 1 --2- 172.21.15.32:0/2733217024 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f790c0ad380 0x7f790c064550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:40.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f79120fb700 1 -- 172.21.15.32:0/2733217024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790c05fab0 con 0x7f790c0b5f00 2020-12-05T17:40:40.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f79120fb700 1 --2- 172.21.15.32:0/2733217024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f790c0b5f00 0x7f790c13cec0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 rx=0x7f7908002f30 tx=0x7f7908015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:40.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f78feffd700 1 -- 172.21.15.32:0/2733217024 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7908011e70 con 0x7f790c0b5f00 2020-12-05T17:40:40.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f78feffd700 1 -- 172.21.15.32:0/2733217024 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7908009c50 con 0x7f790c0b5f00 2020-12-05T17:40:40.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f78feffd700 1 -- 172.21.15.32:0/2733217024 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f790801e650 con 0x7f790c0b5f00 2020-12-05T17:40:40.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f79128fc700 1 -- 172.21.15.32:0/2733217024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f790c062fa0 con 0x7f790c0b5f00 2020-12-05T17:40:40.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f79128fc700 1 -- 172.21.15.32:0/2733217024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f790c0656b0 con 0x7f790c0b5f00 2020-12-05T17:40:40.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.899+0000 7f79128fc700 1 -- 172.21.15.32:0/2733217024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f790c065ef0 con 0x7f790c0b5f00 2020-12-05T17:40:40.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.903+0000 7f78feffd700 1 -- 172.21.15.32:0/2733217024 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f790801a040 con 0x7f790c0b5f00 2020-12-05T17:40:40.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.903+0000 7f78feffd700 1 --2- 172.21.15.32:0/2733217024 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f78ec0554e0 0x7f78ec0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:40.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.903+0000 7f78feffd700 1 -- 172.21.15.32:0/2733217024 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f790806f220 con 0x7f790c0b5f00 2020-12-05T17:40:40.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.903+0000 7f79118fa700 1 --2- 172.21.15.32:0/2733217024 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f78ec0554e0 0x7f78ec0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:40.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.903+0000 7f79118fa700 1 --2- 172.21.15.32:0/2733217024 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f78ec0554e0 0x7f78ec0579a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f790000d000 tx=0x7f7900007b40).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:40.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:40.907+0000 7f78feffd700 1 -- 172.21.15.32:0/2733217024 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7908045400 con 0x7f790c0b5f00 2020-12-05T17:40:41.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:41.103+0000 7f79128fc700 1 -- 172.21.15.32:0/2733217024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f790c057050 con 0x7f790c0b5f00 2020-12-05T17:40:41.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:41.103+0000 7f78feffd700 1 -- 172.21.15.32:0/2733217024 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1906 (secure 0 0 0) 0x7f7908023090 con 0x7f790c0b5f00 2020-12-05T17:40:41.107 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:40:41.107 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:40:41.107 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2020-12-05T17:40:41.107 INFO:teuthology.orchestra.run.smithi032.stdout: "total_bytes": 773094113280, 2020-12-05T17:40:41.108 INFO:teuthology.orchestra.run.smithi032.stdout: "total_avail_bytes": 764492521472, 2020-12-05T17:40:41.108 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_bytes": 11657216, 2020-12-05T17:40:41.108 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_bytes": 8601591808, 2020-12-05T17:40:41.108 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_ratio": 0.011126189492642879, 2020-12-05T17:40:41.108 INFO:teuthology.orchestra.run.smithi032.stdout: "num_osds": 8, 2020-12-05T17:40:41.109 INFO:teuthology.orchestra.run.smithi032.stdout: "num_per_pool_osds": 8, 2020-12-05T17:40:41.109 INFO:teuthology.orchestra.run.smithi032.stdout: "num_per_pool_omap_osds": 8 2020-12-05T17:40:41.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:41.109 INFO:teuthology.orchestra.run.smithi032.stdout: "stats_by_class": { 2020-12-05T17:40:41.109 INFO:teuthology.orchestra.run.smithi032.stdout: "ssd": { 2020-12-05T17:40:41.110 INFO:teuthology.orchestra.run.smithi032.stdout: "total_bytes": 773094113280, 2020-12-05T17:40:41.110 INFO:teuthology.orchestra.run.smithi032.stdout: "total_avail_bytes": 764492521472, 2020-12-05T17:40:41.110 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_bytes": 11657216, 2020-12-05T17:40:41.110 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_bytes": 8601591808, 2020-12-05T17:40:41.110 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_ratio": 0.011126189492642879 2020-12-05T17:40:41.111 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:41.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:41.111 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2020-12-05T17:40:41.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:41.111 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_health_metrics", 2020-12-05T17:40:41.112 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 1, 2020-12-05T17:40:41.112 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2020-12-05T17:40:41.112 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2020-12-05T17:40:41.112 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 2, 2020-12-05T17:40:41.113 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2020-12-05T17:40:41.113 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2020-12-05T17:40:41.113 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2020-12-05T17:40:41.113 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362918805504 2020-12-05T17:40:41.113 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:41.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:41.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:41.114 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd", 2020-12-05T17:40:41.114 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 2, 2020-12-05T17:40:41.114 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2020-12-05T17:40:41.115 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2020-12-05T17:40:41.115 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 0, 2020-12-05T17:40:41.115 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2020-12-05T17:40:41.115 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2020-12-05T17:40:41.115 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2020-12-05T17:40:41.116 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362918805504 2020-12-05T17:40:41.116 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:41.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:41.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:41.116 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs_data", 2020-12-05T17:40:41.117 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 22, 2020-12-05T17:40:41.117 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2020-12-05T17:40:41.117 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2020-12-05T17:40:41.117 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 0, 2020-12-05T17:40:41.117 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2020-12-05T17:40:41.118 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2020-12-05T17:40:41.118 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2020-12-05T17:40:41.118 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362918805504 2020-12-05T17:40:41.118 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:41.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:41.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:41.119 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs_metadata", 2020-12-05T17:40:41.119 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 23, 2020-12-05T17:40:41.119 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2020-12-05T17:40:41.119 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2020-12-05T17:40:41.120 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 0, 2020-12-05T17:40:41.120 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2020-12-05T17:40:41.120 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2020-12-05T17:40:41.120 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2020-12-05T17:40:41.120 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362918805504 2020-12-05T17:40:41.121 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:41.121 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:41.121 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:41.121 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:40:41.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:41.103+0000 7f79128fc700 1 -- 172.21.15.32:0/2733217024 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f78ec0554e0 msgr2=0x7f78ec0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:41.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:41.103+0000 7f79128fc700 1 --2- 172.21.15.32:0/2733217024 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f78ec0554e0 0x7f78ec0579a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f790000d000 tx=0x7f7900007b40).stop 2020-12-05T17:40:41.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:41.103+0000 7f79128fc700 1 -- 172.21.15.32:0/2733217024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f790c0b5f00 msgr2=0x7f790c13cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:41.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:41.103+0000 7f79128fc700 1 --2- 172.21.15.32:0/2733217024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f790c0b5f00 0x7f790c13cec0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 rx=0x7f7908002f30 tx=0x7f7908015000).stop 2020-12-05T17:40:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:41.103+0000 7f79128fc700 1 -- 172.21.15.32:0/2733217024 shutdown_connections 2020-12-05T17:40:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:41.103+0000 7f79128fc700 1 --2- 172.21.15.32:0/2733217024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f790c0b5f00 0x7f790c13cec0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:41.103+0000 7f79128fc700 1 --2- 172.21.15.32:0/2733217024 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f790c0ad380 0x7f790c064550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:41.103+0000 7f79128fc700 1 --2- 172.21.15.32:0/2733217024 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f78ec0554e0 0x7f78ec0579a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:41.103+0000 7f79128fc700 1 --2- 172.21.15.32:0/2733217024 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f790c0af280 0x7f790c05f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:41.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:41.103+0000 7f79128fc700 1 -- 172.21.15.32:0/2733217024 >> 172.21.15.32:0/2733217024 conn(0x7f790c069cb0 msgr2=0x7f790c0663b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:41.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:41.103+0000 7f79128fc700 1 -- 172.21.15.32:0/2733217024 shutdown_connections 2020-12-05T17:40:41.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:41.103+0000 7f79128fc700 1 -- 172.21.15.32:0/2733217024 wait complete. 2020-12-05T17:40:41.125 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2020-12-05T17:40:45.892 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:40:45.895 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:40:45.898 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:40:45.900 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:40:45.903 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:40:46.127 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early df --format=json-pretty 2020-12-05T17:40:46.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.243+0000 7ff90fd8f700 1 Processor -- start 2020-12-05T17:40:46.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.243+0000 7ff90fd8f700 1 -- start start 2020-12-05T17:40:46.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.243+0000 7ff90fd8f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9080b00a0 0x7ff908061910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:46.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.243+0000 7ff90fd8f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9080acca0 0x7ff90814a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.243+0000 7ff90fd8f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9080601e0 con 0x7ff90805fe00 2020-12-05T17:40:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.243+0000 7ff90fd8f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9080ac1a0 con 0x7ff9080acca0 2020-12-05T17:40:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.243+0000 7ff90fd8f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff908061fa0 con 0x7ff9080b00a0 2020-12-05T17:40:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.243+0000 7ff90e32c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9080acca0 0x7ff90814a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.243+0000 7ff90e32c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9080acca0 0x7ff90814a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39760/0 (socket says 172.21.15.32:39760) 2020-12-05T17:40:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.243+0000 7ff90e32c700 1 -- 172.21.15.32:0/2331661882 learned_addr learned my addr 172.21.15.32:0/2331661882 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.243+0000 7ff90d32a700 1 --2- 172.21.15.32:0/2331661882 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9080b00a0 0x7ff908061910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.243+0000 7ff90e32c700 1 -- 172.21.15.32:0/2331661882 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9080b00a0 msgr2=0x7ff908061910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.243+0000 7ff90e32c700 1 --2- 172.21.15.32:0/2331661882 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9080b00a0 0x7ff908061910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.243+0000 7ff90e32c700 1 -- 172.21.15.32:0/2331661882 >> v1:172.21.15.61:6789/0 conn(0x7ff90805fe00 legacy=0x7ff9080ac590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.243+0000 7ff90e32c700 1 -- 172.21.15.32:0/2331661882 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff908062240 con 0x7ff9080acca0 2020-12-05T17:40:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.243+0000 7ff90e32c700 1 --2- 172.21.15.32:0/2331661882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9080acca0 0x7ff90814a3d0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7ff9000032b0 tx=0x7ff900009270).ready entity=mon.0 client_cookie=3c32031efe0e8501 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.243+0000 7ff90cb29700 1 -- 172.21.15.32:0/2331661882 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff900009e60 con 0x7ff9080acca0 2020-12-05T17:40:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90cb29700 1 -- 172.21.15.32:0/2331661882 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90000bc00 con 0x7ff9080acca0 2020-12-05T17:40:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90cb29700 1 -- 172.21.15.32:0/2331661882 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff90000bd70 con 0x7ff9080acca0 2020-12-05T17:40:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90fd8f700 1 -- 172.21.15.32:0/2331661882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9080acca0 msgr2=0x7ff90814a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90fd8f700 1 --2- 172.21.15.32:0/2331661882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9080acca0 0x7ff90814a3d0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7ff9000032b0 tx=0x7ff900009270).stop 2020-12-05T17:40:46.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90fd8f700 1 -- 172.21.15.32:0/2331661882 shutdown_connections 2020-12-05T17:40:46.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90fd8f700 1 --2- 172.21.15.32:0/2331661882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9080acca0 0x7ff90814a3d0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:46.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90fd8f700 1 --2- 172.21.15.32:0/2331661882 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9080b00a0 0x7ff908061910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:46.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90fd8f700 1 -- 172.21.15.32:0/2331661882 >> 172.21.15.32:0/2331661882 conn(0x7ff9080b57c0 msgr2=0x7ff90805f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:46.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90fd8f700 1 -- 172.21.15.32:0/2331661882 shutdown_connections 2020-12-05T17:40:46.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90fd8f700 1 -- 172.21.15.32:0/2331661882 wait complete. 2020-12-05T17:40:46.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90fd8f700 1 Processor -- start 2020-12-05T17:40:46.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90fd8f700 1 -- start start 2020-12-05T17:40:46.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90fd8f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff90805fe00 0x7ff9081438f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:46.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90fd8f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9080acca0 0x7ff90809e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:46.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90fd8f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9080b00a0 0x7ff908062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:46.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90fd8f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9080a23c0 con 0x7ff9080b00a0 2020-12-05T17:40:46.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90fd8f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff908155e40 con 0x7ff9080acca0 2020-12-05T17:40:46.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90fd8f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff908066b90 con 0x7ff90805fe00 2020-12-05T17:40:46.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90e32c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9080b00a0 0x7ff908062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:46.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90e32c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9080b00a0 0x7ff908062c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39766/0 (socket says 172.21.15.32:39766) 2020-12-05T17:40:46.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90e32c700 1 -- 172.21.15.32:0/2978269531 learned_addr learned my addr 172.21.15.32:0/2978269531 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:46.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90db2b700 1 --2- 172.21.15.32:0/2978269531 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff90805fe00 0x7ff9081438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:46.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90d32a700 1 --2- 172.21.15.32:0/2978269531 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9080acca0 0x7ff90809e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:46.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90e32c700 1 -- 172.21.15.32:0/2978269531 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff90805fe00 msgr2=0x7ff9081438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:46.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90e32c700 1 --2- 172.21.15.32:0/2978269531 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff90805fe00 0x7ff9081438f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:46.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90e32c700 1 -- 172.21.15.32:0/2978269531 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9080acca0 msgr2=0x7ff90809e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:46.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90e32c700 1 --2- 172.21.15.32:0/2978269531 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9080acca0 0x7ff90809e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:46.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90e32c700 1 -- 172.21.15.32:0/2978269531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff908159f30 con 0x7ff9080b00a0 2020-12-05T17:40:46.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.247+0000 7ff90e32c700 1 --2- 172.21.15.32:0/2978269531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9080b00a0 0x7ff908062c80 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7ff908143e60 tx=0x7ff900016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:46.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.251+0000 7ff8faffd700 1 -- 172.21.15.32:0/2978269531 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9000107a0 con 0x7ff9080b00a0 2020-12-05T17:40:46.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.251+0000 7ff8faffd700 1 -- 172.21.15.32:0/2978269531 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff900010900 con 0x7ff9080b00a0 2020-12-05T17:40:46.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.251+0000 7ff90fd8f700 1 -- 172.21.15.32:0/2978269531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff90815a1d0 con 0x7ff9080b00a0 2020-12-05T17:40:46.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.251+0000 7ff8faffd700 1 -- 172.21.15.32:0/2978269531 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff900010c60 con 0x7ff9080b00a0 2020-12-05T17:40:46.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.251+0000 7ff90fd8f700 1 -- 172.21.15.32:0/2978269531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff908063170 con 0x7ff9080b00a0 2020-12-05T17:40:46.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.251+0000 7ff90fd8f700 1 -- 172.21.15.32:0/2978269531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff908057050 con 0x7ff9080b00a0 2020-12-05T17:40:46.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.251+0000 7ff8faffd700 1 -- 172.21.15.32:0/2978269531 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff900026030 con 0x7ff9080b00a0 2020-12-05T17:40:46.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.251+0000 7ff8faffd700 1 --2- 172.21.15.32:0/2978269531 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff8e80554e0 0x7ff8e80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:46.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.251+0000 7ff8faffd700 1 -- 172.21.15.32:0/2978269531 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6422+0+0 (secure 0 0 0) 0x7ff90006f430 con 0x7ff9080b00a0 2020-12-05T17:40:46.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.251+0000 7ff90db2b700 1 --2- 172.21.15.32:0/2978269531 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff8e80554e0 0x7ff8e80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:46.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.251+0000 7ff90db2b700 1 --2- 172.21.15.32:0/2978269531 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff8e80554e0 0x7ff8e80579a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7ff90400d000 tx=0x7ff904007b60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:46.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.259+0000 7ff8faffd700 1 -- 172.21.15.32:0/2978269531 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff900045440 con 0x7ff9080b00a0 2020-12-05T17:40:46.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.451+0000 7ff90fd8f700 1 -- 172.21.15.32:0/2978269531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7ff908066e60 con 0x7ff9080b00a0 2020-12-05T17:40:46.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.451+0000 7ff8faffd700 1 -- 172.21.15.32:0/2978269531 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1932 (secure 0 0 0) 0x7ff900024090 con 0x7ff9080b00a0 2020-12-05T17:40:46.455 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:40:46.456 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:40:46.456 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2020-12-05T17:40:46.456 INFO:teuthology.orchestra.run.smithi032.stdout: "total_bytes": 773094113280, 2020-12-05T17:40:46.456 INFO:teuthology.orchestra.run.smithi032.stdout: "total_avail_bytes": 764492341248, 2020-12-05T17:40:46.456 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_bytes": 11837440, 2020-12-05T17:40:46.457 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_bytes": 8601772032, 2020-12-05T17:40:46.457 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_ratio": 0.011126423254609108, 2020-12-05T17:40:46.457 INFO:teuthology.orchestra.run.smithi032.stdout: "num_osds": 8, 2020-12-05T17:40:46.457 INFO:teuthology.orchestra.run.smithi032.stdout: "num_per_pool_osds": 8, 2020-12-05T17:40:46.457 INFO:teuthology.orchestra.run.smithi032.stdout: "num_per_pool_omap_osds": 8 2020-12-05T17:40:46.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:46.458 INFO:teuthology.orchestra.run.smithi032.stdout: "stats_by_class": { 2020-12-05T17:40:46.458 INFO:teuthology.orchestra.run.smithi032.stdout: "ssd": { 2020-12-05T17:40:46.458 INFO:teuthology.orchestra.run.smithi032.stdout: "total_bytes": 773094113280, 2020-12-05T17:40:46.458 INFO:teuthology.orchestra.run.smithi032.stdout: "total_avail_bytes": 764492341248, 2020-12-05T17:40:46.459 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_bytes": 11837440, 2020-12-05T17:40:46.459 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_bytes": 8601772032, 2020-12-05T17:40:46.459 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_ratio": 0.011126423254609108 2020-12-05T17:40:46.459 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:46.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:46.460 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2020-12-05T17:40:46.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:46.460 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_health_metrics", 2020-12-05T17:40:46.461 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 1, 2020-12-05T17:40:46.461 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2020-12-05T17:40:46.461 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2020-12-05T17:40:46.461 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 2, 2020-12-05T17:40:46.462 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2020-12-05T17:40:46.462 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2020-12-05T17:40:46.462 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2020-12-05T17:40:46.462 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362918805504 2020-12-05T17:40:46.462 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:46.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:46.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:46.463 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd", 2020-12-05T17:40:46.463 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 2, 2020-12-05T17:40:46.463 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2020-12-05T17:40:46.464 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2020-12-05T17:40:46.464 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 0, 2020-12-05T17:40:46.464 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2020-12-05T17:40:46.464 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2020-12-05T17:40:46.465 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2020-12-05T17:40:46.465 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362918805504 2020-12-05T17:40:46.465 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:46.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:46.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:46.466 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs_data", 2020-12-05T17:40:46.466 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 22, 2020-12-05T17:40:46.466 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2020-12-05T17:40:46.466 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2020-12-05T17:40:46.467 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 0, 2020-12-05T17:40:46.467 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2020-12-05T17:40:46.467 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2020-12-05T17:40:46.467 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2020-12-05T17:40:46.467 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362918805504 2020-12-05T17:40:46.468 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:46.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:46.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:46.468 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs_metadata", 2020-12-05T17:40:46.468 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 23, 2020-12-05T17:40:46.469 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2020-12-05T17:40:46.469 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 617, 2020-12-05T17:40:46.469 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 1, 2020-12-05T17:40:46.469 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 8, 2020-12-05T17:40:46.469 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 8192, 2020-12-05T17:40:46.470 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 1.1286270584776048e-08, 2020-12-05T17:40:46.470 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362918805504 2020-12-05T17:40:46.470 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:46.470 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:46.470 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:46.471 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:40:46.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.451+0000 7ff90fd8f700 1 -- 172.21.15.32:0/2978269531 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff8e80554e0 msgr2=0x7ff8e80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:46.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.451+0000 7ff90fd8f700 1 --2- 172.21.15.32:0/2978269531 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff8e80554e0 0x7ff8e80579a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7ff90400d000 tx=0x7ff904007b60).stop 2020-12-05T17:40:46.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.451+0000 7ff90fd8f700 1 -- 172.21.15.32:0/2978269531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9080b00a0 msgr2=0x7ff908062c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.451+0000 7ff90fd8f700 1 --2- 172.21.15.32:0/2978269531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9080b00a0 0x7ff908062c80 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7ff908143e60 tx=0x7ff900016000).stop 2020-12-05T17:40:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.451+0000 7ff90fd8f700 1 -- 172.21.15.32:0/2978269531 shutdown_connections 2020-12-05T17:40:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.451+0000 7ff90fd8f700 1 --2- 172.21.15.32:0/2978269531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9080b00a0 0x7ff908062c80 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.451+0000 7ff90fd8f700 1 --2- 172.21.15.32:0/2978269531 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff8e80554e0 0x7ff8e80579a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.451+0000 7ff90fd8f700 1 --2- 172.21.15.32:0/2978269531 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff90805fe00 0x7ff9081438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:46.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.451+0000 7ff90fd8f700 1 --2- 172.21.15.32:0/2978269531 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9080acca0 0x7ff90809e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:46.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.451+0000 7ff90fd8f700 1 -- 172.21.15.32:0/2978269531 >> 172.21.15.32:0/2978269531 conn(0x7ff9080b57c0 msgr2=0x7ff908144610 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:46.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.451+0000 7ff90fd8f700 1 -- 172.21.15.32:0/2978269531 shutdown_connections 2020-12-05T17:40:46.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.451+0000 7ff90fd8f700 1 -- 172.21.15.32:0/2978269531 wait complete. 2020-12-05T17:40:46.475 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s 2020-12-05T17:40:46.476 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data 2020-12-05T17:40:46.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba8a6ab700 1 Processor -- start 2020-12-05T17:40:46.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba8a6ab700 1 -- start start 2020-12-05T17:40:46.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba8a6ab700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba8405f3b0 0x7fba8405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:46.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba8a6ab700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fba8405feb0 0x7fba84061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:46.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba8a6ab700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fba840b2fb0 0x7fba8413c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:46.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba8a6ab700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba840b2df0 con 0x7fba8405f3b0 2020-12-05T17:40:46.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba8a6ab700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba840b3c90 con 0x7fba8405feb0 2020-12-05T17:40:46.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba8a6ab700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba840b2b50 con 0x7fba840b2fb0 2020-12-05T17:40:46.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba896a9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba8405f3b0 0x7fba8405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:46.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba896a9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba8405f3b0 0x7fba8405f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39770/0 (socket says 172.21.15.32:39770) 2020-12-05T17:40:46.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba896a9700 1 -- 172.21.15.32:0/2845255287 learned_addr learned my addr 172.21.15.32:0/2845255287 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:46.599 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba89eaa700 1 --2- 172.21.15.32:0/2845255287 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fba840b2fb0 0x7fba8413c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:46.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba88ea8700 1 --2- 172.21.15.32:0/2845255287 >> v2:172.21.15.61:3300/0 conn(0x7fba8405feb0 0x7fba84061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:46.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba896a9700 1 -- 172.21.15.32:0/2845255287 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fba840b2fb0 msgr2=0x7fba8413c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:46.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba896a9700 1 --2- 172.21.15.32:0/2845255287 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fba840b2fb0 0x7fba8413c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:46.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba896a9700 1 -- 172.21.15.32:0/2845255287 >> v2:172.21.15.61:3300/0 conn(0x7fba8405feb0 msgr2=0x7fba84061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:46.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba896a9700 1 --2- 172.21.15.32:0/2845255287 >> v2:172.21.15.61:3300/0 conn(0x7fba8405feb0 0x7fba84061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:46.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba896a9700 1 -- 172.21.15.32:0/2845255287 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba84142820 con 0x7fba8405f3b0 2020-12-05T17:40:46.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba896a9700 1 --2- 172.21.15.32:0/2845255287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba8405f3b0 0x7fba8405f790 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x7fba80002f30 tx=0x7fba80006e80).ready entity=mon.0 client_cookie=4c345d33d2d5e1cc server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:46.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba7bfff700 1 -- 172.21.15.32:0/2845255287 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba80009640 con 0x7fba8405f3b0 2020-12-05T17:40:46.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba7bfff700 1 -- 172.21.15.32:0/2845255287 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba800097a0 con 0x7fba8405f3b0 2020-12-05T17:40:46.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba7bfff700 1 -- 172.21.15.32:0/2845255287 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba80009b00 con 0x7fba8405f3b0 2020-12-05T17:40:46.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba8a6ab700 1 -- 172.21.15.32:0/2845255287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba8405f3b0 msgr2=0x7fba8405f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:46.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.591+0000 7fba8a6ab700 1 --2- 172.21.15.32:0/2845255287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba8405f3b0 0x7fba8405f790 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x7fba80002f30 tx=0x7fba80006e80).stop 2020-12-05T17:40:46.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.595+0000 7fba8a6ab700 1 -- 172.21.15.32:0/2845255287 shutdown_connections 2020-12-05T17:40:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.595+0000 7fba8a6ab700 1 --2- 172.21.15.32:0/2845255287 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fba840b2fb0 0x7fba8413c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.595+0000 7fba8a6ab700 1 --2- 172.21.15.32:0/2845255287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba8405f3b0 0x7fba8405f790 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.595+0000 7fba8a6ab700 1 --2- 172.21.15.32:0/2845255287 >> v2:172.21.15.61:3300/0 conn(0x7fba8405feb0 0x7fba84061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.595+0000 7fba8a6ab700 1 -- 172.21.15.32:0/2845255287 >> 172.21.15.32:0/2845255287 conn(0x7fba840b8f20 msgr2=0x7fba840b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.595+0000 7fba8a6ab700 1 -- 172.21.15.32:0/2845255287 shutdown_connections 2020-12-05T17:40:46.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.595+0000 7fba8a6ab700 1 -- 172.21.15.32:0/2845255287 wait complete. 2020-12-05T17:40:46.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.595+0000 7fba8a6ab700 1 Processor -- start 2020-12-05T17:40:46.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.595+0000 7fba8a6ab700 1 -- start start 2020-12-05T17:40:46.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba8a6ab700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba8405f3b0 0x7fba841547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:46.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba8a6ab700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fba8405feb0 0x7fba841422c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:46.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba8a6ab700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba840b2fb0 0x7fba8413e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:46.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba8a6ab700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba841414f0 con 0x7fba8405f3b0 2020-12-05T17:40:46.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba8a6ab700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba84141750 con 0x7fba840b2fb0 2020-12-05T17:40:46.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba8a6ab700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba841448e0 con 0x7fba8405feb0 2020-12-05T17:40:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba896a9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba8405f3b0 0x7fba841547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba896a9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba8405f3b0 0x7fba841547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39776/0 (socket says 172.21.15.32:39776) 2020-12-05T17:40:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba896a9700 1 -- 172.21.15.32:0/3422961748 learned_addr learned my addr 172.21.15.32:0/3422961748 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba89eaa700 1 --2- 172.21.15.32:0/3422961748 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba840b2fb0 0x7fba8413e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba88ea8700 1 --2- 172.21.15.32:0/3422961748 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fba8405feb0 0x7fba841422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba896a9700 1 -- 172.21.15.32:0/3422961748 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fba8405feb0 msgr2=0x7fba841422c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba896a9700 1 --2- 172.21.15.32:0/3422961748 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fba8405feb0 0x7fba841422c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba896a9700 1 -- 172.21.15.32:0/3422961748 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba840b2fb0 msgr2=0x7fba8413e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba896a9700 1 --2- 172.21.15.32:0/3422961748 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba840b2fb0 0x7fba8413e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba896a9700 1 -- 172.21.15.32:0/3422961748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba84140fe0 con 0x7fba8405f3b0 2020-12-05T17:40:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba896a9700 1 --2- 172.21.15.32:0/3422961748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba8405f3b0 0x7fba841547f0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7fba80002f30 tx=0x7fba80015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba7a7fc700 1 -- 172.21.15.32:0/3422961748 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba80011c10 con 0x7fba8405f3b0 2020-12-05T17:40:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba7a7fc700 1 -- 172.21.15.32:0/3422961748 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba80011d70 con 0x7fba8405f3b0 2020-12-05T17:40:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba8a6ab700 1 -- 172.21.15.32:0/3422961748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba84146140 con 0x7fba8405f3b0 2020-12-05T17:40:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba7a7fc700 1 -- 172.21.15.32:0/3422961748 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba8001d640 con 0x7fba8405f3b0 2020-12-05T17:40:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba8a6ab700 1 -- 172.21.15.32:0/3422961748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba841465f0 con 0x7fba8405f3b0 2020-12-05T17:40:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba8a6ab700 1 -- 172.21.15.32:0/3422961748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba84141170 con 0x7fba8405f3b0 2020-12-05T17:40:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba7a7fc700 1 -- 172.21.15.32:0/3422961748 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fba8000f070 con 0x7fba8405f3b0 2020-12-05T17:40:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba7a7fc700 1 --2- 172.21.15.32:0/3422961748 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fba6c055330 0x7fba6c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba7a7fc700 1 -- 172.21.15.32:0/3422961748 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6422+0+0 (secure 0 0 0) 0x7fba8006fa10 con 0x7fba8405f3b0 2020-12-05T17:40:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.599+0000 7fba88ea8700 1 --2- 172.21.15.32:0/3422961748 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fba6c055330 0x7fba6c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.603+0000 7fba88ea8700 1 --2- 172.21.15.32:0/3422961748 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fba6c055330 0x7fba6c0577f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7fba7400b000 tx=0x7fba74006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.607+0000 7fba7a7fc700 1 -- 172.21.15.32:0/3422961748 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fba80045bb0 con 0x7fba8405f3b0 2020-12-05T17:40:46.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.795+0000 7fba8a6ab700 1 -- 172.21.15.32:0/3422961748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fba84057050 con 0x7fba8405f3b0 2020-12-05T17:40:46.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.795+0000 7fba7a7fc700 1 -- 172.21.15.32:0/3422961748 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=-22 pool 'cephfs_metadata' already contains some objects. Use an empty pool instead. v54) v1 ==== 209+0+0 (secure 0 0 0) 0x7fba80006a10 con 0x7fba8405f3b0 2020-12-05T17:40:46.800 INFO:teuthology.orchestra.run.smithi032.stderr:Error EINVAL: pool 'cephfs_metadata' already contains some objects. Use an empty pool instead. 2020-12-05T17:40:46.805 DEBUG:teuthology.orchestra.run:got remote process result: 22 2020-12-05T17:40:46.806 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-12-05T17:40:46.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.923+0000 7f875b395700 1 Processor -- start 2020-12-05T17:40:46.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.923+0000 7f875b395700 1 -- start start 2020-12-05T17:40:46.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.923+0000 7f875b395700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f875413f610 0x7f875413fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:46.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.923+0000 7f875b395700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87540af4b0 0x7f87540af920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:46.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.923+0000 7f875b395700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87540b23e0 con 0x7f87540b2000 2020-12-05T17:40:46.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.923+0000 7f875b395700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87540aeba0 con 0x7f875413f610 2020-12-05T17:40:46.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.923+0000 7f875b395700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87540af190 con 0x7f87540af4b0 2020-12-05T17:40:46.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.927+0000 7f8759b92700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f875413f610 0x7f875413fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:46.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.927+0000 7f875ab94700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87540af4b0 0x7f87540af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:46.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.927+0000 7f8759b92700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f875413f610 0x7f875413fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:59676/0 (socket says 172.21.15.32:59676) 2020-12-05T17:40:46.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.927+0000 7f8759b92700 1 -- 172.21.15.32:0/20635988 learned_addr learned my addr 172.21.15.32:0/20635988 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:46.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.927+0000 7f8759b92700 1 -- 172.21.15.32:0/20635988 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87540af4b0 msgr2=0x7f87540af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:46.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.927+0000 7f8759b92700 1 --2- 172.21.15.32:0/20635988 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87540af4b0 0x7f87540af920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:46.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.927+0000 7f8759b92700 1 -- 172.21.15.32:0/20635988 >> v1:172.21.15.61:6789/0 conn(0x7f87540b2000 legacy=0x7f8754142f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:46.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.927+0000 7f8759b92700 1 -- 172.21.15.32:0/20635988 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87540ae010 con 0x7f875413f610 2020-12-05T17:40:46.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.927+0000 7f8759b92700 1 --2- 172.21.15.32:0/20635988 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f875413f610 0x7f875413fac0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f8744002f30 tx=0x7f8744006e80).ready entity=mon.1 client_cookie=b503ef5856a03ff3 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:46.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.927+0000 7f8759391700 1 -- 172.21.15.32:0/20635988 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87440097f0 con 0x7f875413f610 2020-12-05T17:40:46.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.927+0000 7f8759391700 1 -- 172.21.15.32:0/20635988 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8744009950 con 0x7f875413f610 2020-12-05T17:40:46.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.927+0000 7f8759391700 1 -- 172.21.15.32:0/20635988 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8744009cb0 con 0x7f875413f610 2020-12-05T17:40:46.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.927+0000 7f875b395700 1 -- 172.21.15.32:0/20635988 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f875413f610 msgr2=0x7f875413fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:46.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.927+0000 7f875b395700 1 --2- 172.21.15.32:0/20635988 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f875413f610 0x7f875413fac0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f8744002f30 tx=0x7f8744006e80).stop 2020-12-05T17:40:46.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.927+0000 7f875b395700 1 -- 172.21.15.32:0/20635988 shutdown_connections 2020-12-05T17:40:46.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.927+0000 7f875b395700 1 --2- 172.21.15.32:0/20635988 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87540af4b0 0x7f87540af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:46.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.927+0000 7f875b395700 1 --2- 172.21.15.32:0/20635988 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f875413f610 0x7f875413fac0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:46.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.927+0000 7f875b395700 1 -- 172.21.15.32:0/20635988 >> 172.21.15.32:0/20635988 conn(0x7f87540b5620 msgr2=0x7f87540b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:46.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.927+0000 7f875b395700 1 -- 172.21.15.32:0/20635988 shutdown_connections 2020-12-05T17:40:46.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.927+0000 7f875b395700 1 -- 172.21.15.32:0/20635988 wait complete. 2020-12-05T17:40:46.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.931+0000 7f875b395700 1 Processor -- start 2020-12-05T17:40:46.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.931+0000 7f875b395700 1 -- start start 2020-12-05T17:40:46.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.931+0000 7f875b395700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87540af4b0 0x7f8754062660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:46.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.931+0000 7f875b395700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87540b2000 0x7f8754144110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:46.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.931+0000 7f875b395700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f875413f610 0x7f8754144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.931+0000 7f875b395700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8754063040 con 0x7f87540af4b0 2020-12-05T17:40:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.931+0000 7f875b395700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87540632e0 con 0x7f87540b2000 2020-12-05T17:40:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.931+0000 7f875b395700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8754063580 con 0x7f875413f610 2020-12-05T17:40:46.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.931+0000 7f875a393700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87540af4b0 0x7f8754062660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:46.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.931+0000 7f875a393700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87540af4b0 0x7f8754062660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39790/0 (socket says 172.21.15.32:39790) 2020-12-05T17:40:46.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.931+0000 7f875a393700 1 -- 172.21.15.32:0/3304297994 learned_addr learned my addr 172.21.15.32:0/3304297994 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:46.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.931+0000 7f875ab94700 1 --2- 172.21.15.32:0/3304297994 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f875413f610 0x7f8754144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:46.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.931+0000 7f8759b92700 1 --2- 172.21.15.32:0/3304297994 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87540b2000 0x7f8754144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:46.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.931+0000 7f875a393700 1 -- 172.21.15.32:0/3304297994 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f875413f610 msgr2=0x7f8754144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:46.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.931+0000 7f875a393700 1 --2- 172.21.15.32:0/3304297994 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f875413f610 0x7f8754144600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:46.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.931+0000 7f875a393700 1 -- 172.21.15.32:0/3304297994 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87540b2000 msgr2=0x7f8754144110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:46.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.931+0000 7f875a393700 1 --2- 172.21.15.32:0/3304297994 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87540b2000 0x7f8754144110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:46.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.931+0000 7f875a393700 1 -- 172.21.15.32:0/3304297994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8754063820 con 0x7f87540af4b0 2020-12-05T17:40:46.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.931+0000 7f875a393700 1 --2- 172.21.15.32:0/3304297994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87540af4b0 0x7f8754062660 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7f875405cef0 tx=0x7f8750007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:46.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.931+0000 7f87437fe700 1 -- 172.21.15.32:0/3304297994 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8750007240 con 0x7f87540af4b0 2020-12-05T17:40:46.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.931+0000 7f87437fe700 1 -- 172.21.15.32:0/3304297994 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87500073a0 con 0x7f87540af4b0 2020-12-05T17:40:46.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.931+0000 7f875b395700 1 -- 172.21.15.32:0/3304297994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8754144af0 con 0x7f87540af4b0 2020-12-05T17:40:46.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.931+0000 7f87437fe700 1 -- 172.21.15.32:0/3304297994 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87500066d0 con 0x7f87540af4b0 2020-12-05T17:40:46.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.935+0000 7f875b395700 1 -- 172.21.15.32:0/3304297994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8754146850 con 0x7f87540af4b0 2020-12-05T17:40:46.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.935+0000 7f87437fe700 1 -- 172.21.15.32:0/3304297994 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8750010070 con 0x7f87540af4b0 2020-12-05T17:40:46.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.935+0000 7f87437fe700 1 --2- 172.21.15.32:0/3304297994 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f87340553b0 0x7f8734057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:46.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.935+0000 7f875b395700 1 -- 172.21.15.32:0/3304297994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8754057050 con 0x7f87540af4b0 2020-12-05T17:40:46.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.935+0000 7f87437fe700 1 -- 172.21.15.32:0/3304297994 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f875006d920 con 0x7f87540af4b0 2020-12-05T17:40:46.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.935+0000 7f8759b92700 1 --2- 172.21.15.32:0/3304297994 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f87340553b0 0x7f8734057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:46.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.935+0000 7f8759b92700 1 --2- 172.21.15.32:0/3304297994 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f87340553b0 0x7f8734057870 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f874401c000 tx=0x7f8744015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:46.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:46.943+0000 7f87437fe700 1 -- 172.21.15.32:0/3304297994 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8750047360 con 0x7f87540af4b0 2020-12-05T17:40:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.135+0000 7f875b395700 1 -- 172.21.15.32:0/3304297994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f87540639b0 con 0x7f87540af4b0 2020-12-05T17:40:47.139 INFO:tasks.ceph.mon.a.smithi032.stderr:2020-12-05T17:40:47.135+0000 7fdef5931700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-12-05T17:40:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.143+0000 7f87437fe700 1 -- 172.21.15.32:0/3304297994 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 23 and data pool 22 v55) v1 ==== 189+0+0 (secure 0 0 0) 0x7f8750018070 con 0x7f87540af4b0 2020-12-05T17:40:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.147+0000 7f875b395700 1 -- 172.21.15.32:0/3304297994 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f87340553b0 msgr2=0x7f8734057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.147+0000 7f875b395700 1 --2- 172.21.15.32:0/3304297994 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f87340553b0 0x7f8734057870 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f874401c000 tx=0x7f8744015000).stop 2020-12-05T17:40:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.147+0000 7f875b395700 1 -- 172.21.15.32:0/3304297994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87540af4b0 msgr2=0x7f8754062660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.147+0000 7f875b395700 1 --2- 172.21.15.32:0/3304297994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87540af4b0 0x7f8754062660 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7f875405cef0 tx=0x7f8750007be0).stop 2020-12-05T17:40:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.147+0000 7f875b395700 1 -- 172.21.15.32:0/3304297994 shutdown_connections 2020-12-05T17:40:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.147+0000 7f875b395700 1 --2- 172.21.15.32:0/3304297994 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f87340553b0 0x7f8734057870 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.147+0000 7f875b395700 1 --2- 172.21.15.32:0/3304297994 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f875413f610 0x7f8754144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.147+0000 7f875b395700 1 --2- 172.21.15.32:0/3304297994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87540af4b0 0x7f8754062660 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.147+0000 7f875b395700 1 --2- 172.21.15.32:0/3304297994 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87540b2000 0x7f8754144110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.147+0000 7f875b395700 1 -- 172.21.15.32:0/3304297994 >> 172.21.15.32:0/3304297994 conn(0x7f87540b5620 msgr2=0x7f87540b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.147+0000 7f875b395700 1 -- 172.21.15.32:0/3304297994 shutdown_connections 2020-12-05T17:40:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.147+0000 7f875b395700 1 -- 172.21.15.32:0/3304297994 wait complete. 2020-12-05T17:40:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 23 and data pool 22 2020-12-05T17:40:47.164 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-12-05T17:40:47.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.275+0000 7f3814a91700 1 Processor -- start 2020-12-05T17:40:47.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.275+0000 7f3814a91700 1 -- start start 2020-12-05T17:40:47.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.275+0000 7f3814a91700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f38100b5b20 0x7f38100b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:47.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.275+0000 7f3814a91700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38100b5240 0x7f3810145f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:47.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.275+0000 7f3814a91700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38100b1920 con 0x7f38100b6690 2020-12-05T17:40:47.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.275+0000 7f3814a91700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38100b1bc0 con 0x7f38100b5240 2020-12-05T17:40:47.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.275+0000 7f3814a91700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38100b5680 con 0x7f38100b5b20 2020-12-05T17:40:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.275+0000 7f380dd9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f38100b6690 0x7f38100b2060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42446/0 (socket says 172.21.15.32:42446) 2020-12-05T17:40:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.275+0000 7f380dd9b700 1 -- 172.21.15.32:0/1062172591 learned_addr learned my addr 172.21.15.32:0/1062172591 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.275+0000 7f380ed9d700 1 --2- 172.21.15.32:0/1062172591 >> v2:172.21.15.32:3300/0 conn(0x7f38100b5240 0x7f3810145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:47.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.275+0000 7f380e59c700 1 --2- 172.21.15.32:0/1062172591 >> v2:172.21.15.61:3300/0 conn(0x7f38100b5b20 0x7f38100b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:47.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.275+0000 7f380d59a700 1 -- 172.21.15.32:0/1062172591 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2529559185 0 0) 0x7f38100b1920 con 0x7f38100b6690 2020-12-05T17:40:47.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.275+0000 7f380d59a700 1 -- 172.21.15.32:0/1062172591 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37f0003230 con 0x7f38100b6690 2020-12-05T17:40:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.275+0000 7f380ed9d700 1 -- 172.21.15.32:0/1062172591 >> v2:172.21.15.61:3300/0 conn(0x7f38100b5b20 msgr2=0x7f38100b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.275+0000 7f380ed9d700 1 --2- 172.21.15.32:0/1062172591 >> v2:172.21.15.61:3300/0 conn(0x7f38100b5b20 0x7f38100b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.275+0000 7f380ed9d700 1 -- 172.21.15.32:0/1062172591 >> v1:172.21.15.32:6789/0 conn(0x7f38100b6690 legacy=0x7f38100b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.275+0000 7f380ed9d700 1 -- 172.21.15.32:0/1062172591 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3810146400 con 0x7f38100b5240 2020-12-05T17:40:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.275+0000 7f380ed9d700 1 --2- 172.21.15.32:0/1062172591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38100b5240 0x7f3810145f10 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f38080032b0 tx=0x7f38080091a0).ready entity=mon.0 client_cookie=439305a7c49203 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:47.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.275+0000 7f380d59a700 1 -- 172.21.15.32:0/1062172591 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f380800b940 con 0x7f38100b5240 2020-12-05T17:40:47.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.275+0000 7f380d59a700 1 -- 172.21.15.32:0/1062172591 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380800baa0 con 0x7f38100b5240 2020-12-05T17:40:47.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.275+0000 7f380d59a700 1 -- 172.21.15.32:0/1062172591 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f380800bc10 con 0x7f38100b5240 2020-12-05T17:40:47.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.275+0000 7f3814a91700 1 -- 172.21.15.32:0/1062172591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38100b5240 msgr2=0x7f3810145f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:47.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.275+0000 7f3814a91700 1 --2- 172.21.15.32:0/1062172591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38100b5240 0x7f3810145f10 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f38080032b0 tx=0x7f38080091a0).stop 2020-12-05T17:40:47.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.279+0000 7f3814a91700 1 -- 172.21.15.32:0/1062172591 shutdown_connections 2020-12-05T17:40:47.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.279+0000 7f3814a91700 1 --2- 172.21.15.32:0/1062172591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38100b5240 0x7f3810145f10 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:47.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.279+0000 7f3814a91700 1 --2- 172.21.15.32:0/1062172591 >> v2:172.21.15.61:3300/0 conn(0x7f38100b5b20 0x7f38100b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:47.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.279+0000 7f3814a91700 1 -- 172.21.15.32:0/1062172591 >> 172.21.15.32:0/1062172591 conn(0x7f381005f3b0 msgr2=0x7f381005f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.279+0000 7f3814a91700 1 -- 172.21.15.32:0/1062172591 shutdown_connections 2020-12-05T17:40:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.279+0000 7f3814a91700 1 -- 172.21.15.32:0/1062172591 wait complete. 2020-12-05T17:40:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.279+0000 7f3814a91700 1 Processor -- start 2020-12-05T17:40:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.279+0000 7f3814a91700 1 -- start start 2020-12-05T17:40:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.283+0000 7f3814a91700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f38100b5240 0x7f381014a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.283+0000 7f3814a91700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38100b5b20 0x7f3810147660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.283+0000 7f3814a91700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f38100b6690 0x7f3810147b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:47.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.283+0000 7f3814a91700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3810064340 con 0x7f38100b5b20 2020-12-05T17:40:47.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.283+0000 7f3814a91700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38100645e0 con 0x7f38100b5240 2020-12-05T17:40:47.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.283+0000 7f3814a91700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3810064880 con 0x7f38100b6690 2020-12-05T17:40:47.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.283+0000 7f380dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38100b5b20 0x7f3810147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:47.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.283+0000 7f380dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38100b5b20 0x7f3810147660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39806/0 (socket says 172.21.15.32:39806) 2020-12-05T17:40:47.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.283+0000 7f380dd9b700 1 -- 172.21.15.32:0/677866792 learned_addr learned my addr 172.21.15.32:0/677866792 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:47.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.283+0000 7f380ed9d700 1 --2- 172.21.15.32:0/677866792 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f38100b6690 0x7f3810147b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:47.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.283+0000 7f380e59c700 1 --2- 172.21.15.32:0/677866792 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f38100b5240 0x7f381014a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:47.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.283+0000 7f380dd9b700 1 -- 172.21.15.32:0/677866792 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f38100b6690 msgr2=0x7f3810147b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.283+0000 7f380dd9b700 1 --2- 172.21.15.32:0/677866792 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f38100b6690 0x7f3810147b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.283+0000 7f380dd9b700 1 -- 172.21.15.32:0/677866792 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f38100b5240 msgr2=0x7f381014a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.283+0000 7f380dd9b700 1 --2- 172.21.15.32:0/677866792 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f38100b5240 0x7f381014a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.283+0000 7f380dd9b700 1 -- 172.21.15.32:0/677866792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3810064d20 con 0x7f38100b5b20 2020-12-05T17:40:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.283+0000 7f380dd9b700 1 --2- 172.21.15.32:0/677866792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38100b5b20 0x7f3810147660 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 rx=0x7f37f80010d0 tx=0x7f37f8007af0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.283+0000 7f38077fe700 1 -- 172.21.15.32:0/677866792 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37f80072e0 con 0x7f38100b5b20 2020-12-05T17:40:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.283+0000 7f38077fe700 1 -- 172.21.15.32:0/677866792 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f8007440 con 0x7f38100b5b20 2020-12-05T17:40:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.283+0000 7f3814a91700 1 -- 172.21.15.32:0/677866792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3810149210 con 0x7f38100b5b20 2020-12-05T17:40:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.283+0000 7f38077fe700 1 -- 172.21.15.32:0/677866792 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37f80057d0 con 0x7f38100b5b20 2020-12-05T17:40:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.283+0000 7f3814a91700 1 -- 172.21.15.32:0/677866792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3810063c80 con 0x7f38100b5b20 2020-12-05T17:40:47.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.287+0000 7f3814a91700 1 -- 172.21.15.32:0/677866792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3810064a40 con 0x7f38100b5b20 2020-12-05T17:40:47.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.287+0000 7f38077fe700 1 -- 172.21.15.32:0/677866792 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f37f8010070 con 0x7f38100b5b20 2020-12-05T17:40:47.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.287+0000 7f38077fe700 1 --2- 172.21.15.32:0/677866792 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f37f40554e0 0x7f37f40579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:47.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.287+0000 7f38077fe700 1 -- 172.21.15.32:0/677866792 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f37f806d6d0 con 0x7f38100b5b20 2020-12-05T17:40:47.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.287+0000 7f380e59c700 1 --2- 172.21.15.32:0/677866792 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f37f40554e0 0x7f37f40579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:47.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.287+0000 7f380e59c700 1 --2- 172.21.15.32:0/677866792 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f37f40554e0 0x7f37f40579a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f380000b000 tx=0x7f3800006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:47.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.295+0000 7f38077fe700 1 -- 172.21.15.32:0/677866792 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f37f80438d0 con 0x7f38100b5b20 2020-12-05T17:40:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:47.479+0000 7f3814a91700 1 -- 172.21.15.32:0/677866792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3810057050 con 0x7f38100b5b20 2020-12-05T17:40:48.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.143+0000 7f38077fe700 1 -- 172.21.15.32:0/677866792 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) v1 ==== 104+0+0 (secure 0 0 0) 0x7f37f8018020 con 0x7f38100b5b20 2020-12-05T17:40:48.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.147+0000 7f3814a91700 1 -- 172.21.15.32:0/677866792 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f37f40554e0 msgr2=0x7f37f40579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:48.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.147+0000 7f3814a91700 1 --2- 172.21.15.32:0/677866792 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f37f40554e0 0x7f37f40579a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f380000b000 tx=0x7f3800006dc0).stop 2020-12-05T17:40:48.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.147+0000 7f3814a91700 1 -- 172.21.15.32:0/677866792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38100b5b20 msgr2=0x7f3810147660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:48.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.147+0000 7f3814a91700 1 --2- 172.21.15.32:0/677866792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38100b5b20 0x7f3810147660 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 rx=0x7f37f80010d0 tx=0x7f37f8007af0).stop 2020-12-05T17:40:48.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.147+0000 7f3814a91700 1 -- 172.21.15.32:0/677866792 shutdown_connections 2020-12-05T17:40:48.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.147+0000 7f3814a91700 1 --2- 172.21.15.32:0/677866792 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f37f40554e0 0x7f37f40579a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:48.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.147+0000 7f3814a91700 1 --2- 172.21.15.32:0/677866792 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f38100b6690 0x7f3810147b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:48.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.147+0000 7f3814a91700 1 --2- 172.21.15.32:0/677866792 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f38100b5240 0x7f381014a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.147+0000 7f3814a91700 1 --2- 172.21.15.32:0/677866792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38100b5b20 0x7f3810147660 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.147+0000 7f3814a91700 1 -- 172.21.15.32:0/677866792 >> 172.21.15.32:0/677866792 conn(0x7f381005f3b0 msgr2=0x7f381005f790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.147+0000 7f3814a91700 1 -- 172.21.15.32:0/677866792 shutdown_connections 2020-12-05T17:40:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.147+0000 7f3814a91700 1 -- 172.21.15.32:0/677866792 wait complete. 2020-12-05T17:40:48.169 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-12-05T17:40:48.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.295+0000 7fe3a359e700 1 Processor -- start 2020-12-05T17:40:48.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.295+0000 7fe3a359e700 1 -- start start 2020-12-05T17:40:48.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a359e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fe3a40b2100 0x7fe3a40af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:48.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a359e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3a40ad250 0x7fe3a4142600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:48.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a359e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a40b24e0 con 0x7fe3a413ed60 2020-12-05T17:40:48.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a359e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a413fd10 con 0x7fe3a40b2100 2020-12-05T17:40:48.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a359e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a40b0780 con 0x7fe3a40ad250 2020-12-05T17:40:48.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a2d9d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3a40ad250 0x7fe3a4142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:48.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a259c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fe3a40b2100 0x7fe3a40af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:48.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a1d9b700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fe3a413ed60 0x7fe3a4138820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:55102/0 (socket says 172.21.15.32:55102) 2020-12-05T17:40:48.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a1d9b700 1 -- 172.21.15.32:0/865778758 learned_addr learned my addr 172.21.15.32:0/865778758 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:48.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a159a700 1 -- 172.21.15.32:0/865778758 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255778991 0 0) 0x7fe3a40b24e0 con 0x7fe3a413ed60 2020-12-05T17:40:48.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a159a700 1 -- 172.21.15.32:0/865778758 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe388003230 con 0x7fe3a413ed60 2020-12-05T17:40:48.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a2d9d700 1 -- 172.21.15.32:0/865778758 >> v2:172.21.15.61:3300/0 conn(0x7fe3a40b2100 msgr2=0x7fe3a40af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:48.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a2d9d700 1 --2- 172.21.15.32:0/865778758 >> v2:172.21.15.61:3300/0 conn(0x7fe3a40b2100 0x7fe3a40af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:48.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a2d9d700 1 -- 172.21.15.32:0/865778758 >> v1:172.21.15.61:6789/0 conn(0x7fe3a413ed60 legacy=0x7fe3a4138820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:48.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a2d9d700 1 -- 172.21.15.32:0/865778758 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a40b0240 con 0x7fe3a40ad250 2020-12-05T17:40:48.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a2d9d700 1 --2- 172.21.15.32:0/865778758 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3a40ad250 0x7fe3a4142600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fe39c0032b0 tx=0x7fe39c0091a0).ready entity=mon.2 client_cookie=539f287d1dbb0781 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:48.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a159a700 1 -- 172.21.15.32:0/865778758 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe39c00b940 con 0x7fe3a40ad250 2020-12-05T17:40:48.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a159a700 1 -- 172.21.15.32:0/865778758 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39c00baa0 con 0x7fe3a40ad250 2020-12-05T17:40:48.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a159a700 1 -- 172.21.15.32:0/865778758 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe39c00bc10 con 0x7fe3a40ad250 2020-12-05T17:40:48.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a359e700 1 -- 172.21.15.32:0/865778758 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3a40ad250 msgr2=0x7fe3a4142600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:48.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a359e700 1 --2- 172.21.15.32:0/865778758 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3a40ad250 0x7fe3a4142600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fe39c0032b0 tx=0x7fe39c0091a0).stop 2020-12-05T17:40:48.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a359e700 1 -- 172.21.15.32:0/865778758 shutdown_connections 2020-12-05T17:40:48.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a359e700 1 --2- 172.21.15.32:0/865778758 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3a40ad250 0x7fe3a4142600 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:48.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a359e700 1 --2- 172.21.15.32:0/865778758 >> v2:172.21.15.61:3300/0 conn(0x7fe3a40b2100 0x7fe3a40af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:48.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a359e700 1 -- 172.21.15.32:0/865778758 >> 172.21.15.32:0/865778758 conn(0x7fe3a40620b0 msgr2=0x7fe3a4062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:48.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a359e700 1 -- 172.21.15.32:0/865778758 shutdown_connections 2020-12-05T17:40:48.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.299+0000 7fe3a359e700 1 -- 172.21.15.32:0/865778758 wait complete. 2020-12-05T17:40:48.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.303+0000 7fe3a359e700 1 Processor -- start 2020-12-05T17:40:48.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.303+0000 7fe3a359e700 1 -- start start 2020-12-05T17:40:48.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.303+0000 7fe3a359e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3a40ad250 0x7fe3a4137e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:48.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.303+0000 7fe3a359e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3a40b2100 0x7fe3a4136b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:48.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.303+0000 7fe3a359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3a4063b90 0x7fe3a4137050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:48.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.303+0000 7fe3a359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a4065360 con 0x7fe3a4063b90 2020-12-05T17:40:48.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.303+0000 7fe3a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a4065590 con 0x7fe3a40b2100 2020-12-05T17:40:48.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.303+0000 7fe3a359e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a4138730 con 0x7fe3a40ad250 2020-12-05T17:40:48.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.303+0000 7fe3a2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3a4063b90 0x7fe3a4137050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:48.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.303+0000 7fe3a2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3a4063b90 0x7fe3a4137050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39822/0 (socket says 172.21.15.32:39822) 2020-12-05T17:40:48.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.303+0000 7fe3a2d9d700 1 -- 172.21.15.32:0/3545619380 learned_addr learned my addr 172.21.15.32:0/3545619380 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:48.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.303+0000 7fe3a259c700 1 --2- 172.21.15.32:0/3545619380 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3a40ad250 0x7fe3a4137e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:48.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.303+0000 7fe3a1d9b700 1 --2- 172.21.15.32:0/3545619380 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3a40b2100 0x7fe3a4136b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:48.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.303+0000 7fe3a2d9d700 1 -- 172.21.15.32:0/3545619380 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3a40ad250 msgr2=0x7fe3a4137e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:48.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.303+0000 7fe3a2d9d700 1 --2- 172.21.15.32:0/3545619380 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3a40ad250 0x7fe3a4137e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:48.314 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.303+0000 7fe3a2d9d700 1 -- 172.21.15.32:0/3545619380 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3a40b2100 msgr2=0x7fe3a4136b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:48.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.303+0000 7fe3a2d9d700 1 --2- 172.21.15.32:0/3545619380 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3a40b2100 0x7fe3a4136b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:48.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.303+0000 7fe3a2d9d700 1 -- 172.21.15.32:0/3545619380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a4063790 con 0x7fe3a4063b90 2020-12-05T17:40:48.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.303+0000 7fe3a2d9d700 1 --2- 172.21.15.32:0/3545619380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3a4063b90 0x7fe3a4137050 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0x7fe39c0032b0 tx=0x7fe39c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:48.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.307+0000 7fe3937fe700 1 -- 172.21.15.32:0/3545619380 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe39c00be80 con 0x7fe3a4063b90 2020-12-05T17:40:48.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.307+0000 7fe3937fe700 1 -- 172.21.15.32:0/3545619380 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39c01f3f0 con 0x7fe3a4063b90 2020-12-05T17:40:48.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.307+0000 7fe3a359e700 1 -- 172.21.15.32:0/3545619380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a4137540 con 0x7fe3a4063b90 2020-12-05T17:40:48.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.307+0000 7fe3937fe700 1 -- 172.21.15.32:0/3545619380 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe39c01f750 con 0x7fe3a4063b90 2020-12-05T17:40:48.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.307+0000 7fe3a359e700 1 -- 172.21.15.32:0/3545619380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a4138490 con 0x7fe3a4063b90 2020-12-05T17:40:48.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.307+0000 7fe3a359e700 1 -- 172.21.15.32:0/3545619380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a4057050 con 0x7fe3a4063b90 2020-12-05T17:40:48.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.307+0000 7fe3937fe700 1 -- 172.21.15.32:0/3545619380 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe39c026030 con 0x7fe3a4063b90 2020-12-05T17:40:48.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.307+0000 7fe3937fe700 1 --2- 172.21.15.32:0/3545619380 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe3800553b0 0x7fe380057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:48.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.307+0000 7fe3937fe700 1 -- 172.21.15.32:0/3545619380 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fe39c06ebc0 con 0x7fe3a4063b90 2020-12-05T17:40:48.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.307+0000 7fe3a259c700 1 --2- 172.21.15.32:0/3545619380 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe3800553b0 0x7fe380057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:48.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.311+0000 7fe3a259c700 1 --2- 172.21.15.32:0/3545619380 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe3800553b0 0x7fe380057870 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7fe394008000 tx=0x7fe394008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:48.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.315+0000 7fe3937fe700 1 -- 172.21.15.32:0/3545619380 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe39c044d10 con 0x7fe3a4063b90 2020-12-05T17:40:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:48.459+0000 7fe3a359e700 1 -- 172.21.15.32:0/3545619380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe3a40b5080 con 0x7fe3a4063b90 2020-12-05T17:40:49.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.155+0000 7fe3937fe700 1 -- 172.21.15.32:0/3545619380 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v119) v1 ==== 185+0+0 (secure 0 0 0) 0x7fe39c0240e0 con 0x7fe3a4063b90 2020-12-05T17:40:49.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.159+0000 7fe3a359e700 1 -- 172.21.15.32:0/3545619380 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe3800553b0 msgr2=0x7fe380057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:49.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.159+0000 7fe3a359e700 1 --2- 172.21.15.32:0/3545619380 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe3800553b0 0x7fe380057870 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7fe394008000 tx=0x7fe394008040).stop 2020-12-05T17:40:49.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.159+0000 7fe3a359e700 1 -- 172.21.15.32:0/3545619380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3a4063b90 msgr2=0x7fe3a4137050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:49.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.159+0000 7fe3a359e700 1 --2- 172.21.15.32:0/3545619380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3a4063b90 0x7fe3a4137050 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0x7fe39c0032b0 tx=0x7fe39c016000).stop 2020-12-05T17:40:49.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.159+0000 7fe3a359e700 1 -- 172.21.15.32:0/3545619380 shutdown_connections 2020-12-05T17:40:49.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.159+0000 7fe3a359e700 1 --2- 172.21.15.32:0/3545619380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3a4063b90 0x7fe3a4137050 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:49.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.159+0000 7fe3a359e700 1 --2- 172.21.15.32:0/3545619380 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe3800553b0 0x7fe380057870 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:49.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.159+0000 7fe3a359e700 1 --2- 172.21.15.32:0/3545619380 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3a40ad250 0x7fe3a4137e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:49.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.159+0000 7fe3a359e700 1 --2- 172.21.15.32:0/3545619380 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3a40b2100 0x7fe3a4136b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:49.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.159+0000 7fe3a359e700 1 -- 172.21.15.32:0/3545619380 >> 172.21.15.32:0/3545619380 conn(0x7fe3a40620b0 msgr2=0x7fe3a4062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:49.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.159+0000 7fe3a359e700 1 -- 172.21.15.32:0/3545619380 shutdown_connections 2020-12-05T17:40:49.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.159+0000 7fe3a359e700 1 -- 172.21.15.32:0/3545619380 wait complete. 2020-12-05T17:40:49.166 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2020-12-05T17:40:49.178 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-12-05T17:40:49.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.299+0000 7f0b07b44700 1 Processor -- start 2020-12-05T17:40:49.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.303+0000 7f0b07b44700 1 -- start start 2020-12-05T17:40:49.308 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.303+0000 7f0b07b44700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0b000b6690 0x7f0b000b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:49.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.303+0000 7f0b07b44700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b000b5240 0x7f0b00145f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:49.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.303+0000 7f0b07b44700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b000b56f0 con 0x7f0b000b5b20 2020-12-05T17:40:49.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.303+0000 7f0b07b44700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b000b1920 con 0x7f0b000b5240 2020-12-05T17:40:49.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.303+0000 7f0b07b44700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b000b1bc0 con 0x7f0b000b6690 2020-12-05T17:40:49.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.303+0000 7f0b060e1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b000b5240 0x7f0b00145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:49.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.303+0000 7f0b060e1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b000b5240 0x7f0b00145f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39830/0 (socket says 172.21.15.32:39830) 2020-12-05T17:40:49.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.303+0000 7f0b060e1700 1 -- 172.21.15.32:0/593216460 learned_addr learned my addr 172.21.15.32:0/593216460 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:49.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.303+0000 7f0b050df700 1 --2- 172.21.15.32:0/593216460 >> v2:172.21.15.61:3300/0 conn(0x7f0b000b6690 0x7f0b000b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:49.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.303+0000 7f0b060e1700 1 -- 172.21.15.32:0/593216460 >> v2:172.21.15.61:3300/0 conn(0x7f0b000b6690 msgr2=0x7f0b000b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:49.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.303+0000 7f0b060e1700 1 --2- 172.21.15.32:0/593216460 >> v2:172.21.15.61:3300/0 conn(0x7f0b000b6690 0x7f0b000b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:49.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.303+0000 7f0b060e1700 1 -- 172.21.15.32:0/593216460 >> v1:172.21.15.61:6789/0 conn(0x7f0b000b5b20 legacy=0x7f0b000b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:49.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.303+0000 7f0b060e1700 1 -- 172.21.15.32:0/593216460 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b00146400 con 0x7f0b000b5240 2020-12-05T17:40:49.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.303+0000 7f0b060e1700 1 --2- 172.21.15.32:0/593216460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b000b5240 0x7f0b00145f10 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7f0af80032b0 tx=0x7f0af8009270).ready entity=mon.0 client_cookie=88a8d5bae62ed0c7 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:49.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.303+0000 7f0b048de700 1 -- 172.21.15.32:0/593216460 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0af8009e60 con 0x7f0b000b5240 2020-12-05T17:40:49.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.303+0000 7f0b048de700 1 -- 172.21.15.32:0/593216460 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af800bc00 con 0x7f0b000b5240 2020-12-05T17:40:49.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.303+0000 7f0b048de700 1 -- 172.21.15.32:0/593216460 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0af800bd70 con 0x7f0b000b5240 2020-12-05T17:40:49.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.303+0000 7f0b07b44700 1 -- 172.21.15.32:0/593216460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b000b5240 msgr2=0x7f0b00145f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:49.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.303+0000 7f0b07b44700 1 --2- 172.21.15.32:0/593216460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b000b5240 0x7f0b00145f10 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7f0af80032b0 tx=0x7f0af8009270).stop 2020-12-05T17:40:49.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.303+0000 7f0b07b44700 1 -- 172.21.15.32:0/593216460 shutdown_connections 2020-12-05T17:40:49.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.307+0000 7f0b07b44700 1 --2- 172.21.15.32:0/593216460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b000b5240 0x7f0b00145f10 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:49.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.307+0000 7f0b07b44700 1 --2- 172.21.15.32:0/593216460 >> v2:172.21.15.61:3300/0 conn(0x7f0b000b6690 0x7f0b000b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:49.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.307+0000 7f0b07b44700 1 -- 172.21.15.32:0/593216460 >> 172.21.15.32:0/593216460 conn(0x7f0b0005f3b0 msgr2=0x7f0b0005f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:49.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.307+0000 7f0b07b44700 1 -- 172.21.15.32:0/593216460 shutdown_connections 2020-12-05T17:40:49.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.307+0000 7f0b07b44700 1 -- 172.21.15.32:0/593216460 wait complete. 2020-12-05T17:40:49.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.307+0000 7f0b07b44700 1 Processor -- start 2020-12-05T17:40:49.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.307+0000 7f0b07b44700 1 -- start start 2020-12-05T17:40:49.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.307+0000 7f0b07b44700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b000b5240 0x7f0b00066980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:49.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.307+0000 7f0b07b44700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b000b5b20 0x7f0b00151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:49.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.307+0000 7f0b07b44700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b000b6690 0x7f0b00151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:49.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.307+0000 7f0b07b44700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b00152490 con 0x7f0b000b6690 2020-12-05T17:40:49.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.307+0000 7f0b07b44700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b001404c0 con 0x7f0b000b5240 2020-12-05T17:40:49.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.307+0000 7f0b07b44700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b00066090 con 0x7f0b000b5b20 2020-12-05T17:40:49.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.307+0000 7f0b060e1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b000b6690 0x7f0b00151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:49.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.307+0000 7f0b060e1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b000b6690 0x7f0b00151fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39836/0 (socket says 172.21.15.32:39836) 2020-12-05T17:40:49.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.307+0000 7f0b050df700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b000b5b20 0x7f0b00151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:49.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.307+0000 7f0b060e1700 1 -- 172.21.15.32:0/4034018821 learned_addr learned my addr 172.21.15.32:0/4034018821 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:49.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.307+0000 7f0b058e0700 1 --2- 172.21.15.32:0/4034018821 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b000b5240 0x7f0b00066980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:49.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.307+0000 7f0b060e1700 1 -- 172.21.15.32:0/4034018821 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b000b5b20 msgr2=0x7f0b00151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:49.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.307+0000 7f0b060e1700 1 --2- 172.21.15.32:0/4034018821 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b000b5b20 0x7f0b00151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:49.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.307+0000 7f0b060e1700 1 -- 172.21.15.32:0/4034018821 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b000b5240 msgr2=0x7f0b00066980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:49.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.307+0000 7f0b060e1700 1 --2- 172.21.15.32:0/4034018821 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b000b5240 0x7f0b00066980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:49.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.307+0000 7f0b060e1700 1 -- 172.21.15.32:0/4034018821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b000662c0 con 0x7f0b000b6690 2020-12-05T17:40:49.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.311+0000 7f0b060e1700 1 --2- 172.21.15.32:0/4034018821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b000b6690 0x7f0b00151fa0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7f0b000a3ef0 tx=0x7f0af8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:49.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.311+0000 7f0aeeffd700 1 -- 172.21.15.32:0/4034018821 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0af80107a0 con 0x7f0b000b6690 2020-12-05T17:40:49.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.311+0000 7f0aeeffd700 1 -- 172.21.15.32:0/4034018821 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af8010900 con 0x7f0b000b6690 2020-12-05T17:40:49.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.311+0000 7f0aeeffd700 1 -- 172.21.15.32:0/4034018821 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0af8010c60 con 0x7f0b000b6690 2020-12-05T17:40:49.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.311+0000 7f0b07b44700 1 -- 172.21.15.32:0/4034018821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b000a5820 con 0x7f0b000b6690 2020-12-05T17:40:49.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.311+0000 7f0b07b44700 1 -- 172.21.15.32:0/4034018821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b000a27c0 con 0x7f0b000b6690 2020-12-05T17:40:49.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.311+0000 7f0b07b44700 1 -- 172.21.15.32:0/4034018821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b000a59e0 con 0x7f0b000b6690 2020-12-05T17:40:49.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.311+0000 7f0aeeffd700 1 -- 172.21.15.32:0/4034018821 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0af800f070 con 0x7f0b000b6690 2020-12-05T17:40:49.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.311+0000 7f0aeeffd700 1 --2- 172.21.15.32:0/4034018821 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0ae00554e0 0x7f0ae00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:49.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.311+0000 7f0aeeffd700 1 -- 172.21.15.32:0/4034018821 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f0af806f650 con 0x7f0b000b6690 2020-12-05T17:40:49.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.315+0000 7f0b058e0700 1 --2- 172.21.15.32:0/4034018821 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0ae00554e0 0x7f0ae00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:49.323 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.315+0000 7f0b058e0700 1 --2- 172.21.15.32:0/4034018821 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0ae00554e0 0x7f0ae00579a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f0af000d000 tx=0x7f0af0000b80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:49.323 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.319+0000 7f0aeeffd700 1 -- 172.21.15.32:0/4034018821 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0af8049380 con 0x7f0b000b6690 2020-12-05T17:40:49.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:49.467+0000 7f0b07b44700 1 -- 172.21.15.32:0/4034018821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f0b00057050 con 0x7f0b000b6690 2020-12-05T17:40:50.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.159+0000 7f0aeeffd700 1 -- 172.21.15.32:0/4034018821 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v120) v1 ==== 133+0+0 (secure 0 0 0) 0x7f0af80150a0 con 0x7f0b000b6690 2020-12-05T17:40:50.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.163+0000 7f0b07b44700 1 -- 172.21.15.32:0/4034018821 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0ae00554e0 msgr2=0x7f0ae00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:50.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.163+0000 7f0b07b44700 1 --2- 172.21.15.32:0/4034018821 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0ae00554e0 0x7f0ae00579a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f0af000d000 tx=0x7f0af0000b80).stop 2020-12-05T17:40:50.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.163+0000 7f0b07b44700 1 -- 172.21.15.32:0/4034018821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b000b6690 msgr2=0x7f0b00151fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:50.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.163+0000 7f0b07b44700 1 --2- 172.21.15.32:0/4034018821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b000b6690 0x7f0b00151fa0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7f0b000a3ef0 tx=0x7f0af8016000).stop 2020-12-05T17:40:50.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.167+0000 7f0b07b44700 1 -- 172.21.15.32:0/4034018821 shutdown_connections 2020-12-05T17:40:50.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.167+0000 7f0b07b44700 1 --2- 172.21.15.32:0/4034018821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b000b6690 0x7f0b00151fa0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:50.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.167+0000 7f0b07b44700 1 --2- 172.21.15.32:0/4034018821 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b000b5240 0x7f0b00066980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:50.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.167+0000 7f0b07b44700 1 --2- 172.21.15.32:0/4034018821 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0ae00554e0 0x7f0ae00579a0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:50.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.167+0000 7f0b07b44700 1 --2- 172.21.15.32:0/4034018821 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b000b5b20 0x7f0b00151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:50.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.167+0000 7f0b07b44700 1 -- 172.21.15.32:0/4034018821 >> 172.21.15.32:0/4034018821 conn(0x7f0b0005f3b0 msgr2=0x7f0b000b8680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:50.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.167+0000 7f0b07b44700 1 -- 172.21.15.32:0/4034018821 shutdown_connections 2020-12-05T17:40:50.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.167+0000 7f0b07b44700 1 -- 172.21.15.32:0/4034018821 wait complete. 2020-12-05T17:40:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2020-12-05T17:40:50.186 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data 2020-12-05T17:40:50.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.299+0000 7fbc41857700 1 Processor -- start 2020-12-05T17:40:50.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.299+0000 7fbc41857700 1 -- start start 2020-12-05T17:40:50.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.299+0000 7fbc41857700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc3c05f3b0 0x7fbc3c05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:50.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.299+0000 7fbc41857700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc3c0b2df0 con 0x7fbc3c0b2fb0 2020-12-05T17:40:50.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.299+0000 7fbc41857700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc3c0b3c90 con 0x7fbc3c05feb0 2020-12-05T17:40:50.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.299+0000 7fbc41857700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c0b2b50 con 0x7fbc3c05f3b0 2020-12-05T17:40:50.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.299+0000 7fbc41056700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbc3c0b2fb0 0x7fbc3c142cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42492/0 (socket says 172.21.15.32:42492) 2020-12-05T17:40:50.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.299+0000 7fbc41056700 1 -- 172.21.15.32:0/2508703862 learned_addr learned my addr 172.21.15.32:0/2508703862 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:50.307 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.299+0000 7fbc40855700 1 --2- 172.21.15.32:0/2508703862 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc3c05f3b0 0x7fbc3c05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:50.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.303+0000 7fbc3b7fe700 1 -- 172.21.15.32:0/2508703862 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 470394098 0 0) 0x7fbc3c0b2df0 con 0x7fbc3c0b2fb0 2020-12-05T17:40:50.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.303+0000 7fbc3b7fe700 1 -- 172.21.15.32:0/2508703862 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc28003230 con 0x7fbc3c0b2fb0 2020-12-05T17:40:50.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.303+0000 7fbc3b7fe700 1 -- 172.21.15.32:0/2508703862 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1210648119 0 0) 0x7fbc3c0b3c90 con 0x7fbc3c05feb0 2020-12-05T17:40:50.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.303+0000 7fbc3b7fe700 1 -- 172.21.15.32:0/2508703862 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc3c0b2df0 con 0x7fbc3c05feb0 2020-12-05T17:40:50.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.303+0000 7fbc3b7fe700 1 -- 172.21.15.32:0/2508703862 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1721645829 0 0) 0x7fbc28003230 con 0x7fbc3c0b2fb0 2020-12-05T17:40:50.310 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.303+0000 7fbc40855700 1 -- 172.21.15.32:0/2508703862 >> v1:172.21.15.61:6789/0 conn(0x7fbc3c05feb0 legacy=0x7fbc3c13c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:50.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.303+0000 7fbc40855700 1 -- 172.21.15.32:0/2508703862 >> v1:172.21.15.32:6789/0 conn(0x7fbc3c0b2fb0 legacy=0x7fbc3c142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:50.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.303+0000 7fbc40855700 1 -- 172.21.15.32:0/2508703862 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3c142820 con 0x7fbc3c05f3b0 2020-12-05T17:40:50.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.303+0000 7fbc40855700 1 --2- 172.21.15.32:0/2508703862 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc3c05f3b0 0x7fbc3c05f790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fbc34001450 tx=0x7fbc2c006dc0).ready entity=mon.2 client_cookie=e62531cb0f3f03e3 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:50.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.303+0000 7fbc3b7fe700 1 -- 172.21.15.32:0/2508703862 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc34001a50 con 0x7fbc3c05f3b0 2020-12-05T17:40:50.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.303+0000 7fbc3b7fe700 1 -- 172.21.15.32:0/2508703862 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2c00d5c0 con 0x7fbc3c05f3b0 2020-12-05T17:40:50.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.303+0000 7fbc3b7fe700 1 -- 172.21.15.32:0/2508703862 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc2c00d730 con 0x7fbc3c05f3b0 2020-12-05T17:40:50.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.303+0000 7fbc41857700 1 -- 172.21.15.32:0/2508703862 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc3c05f3b0 msgr2=0x7fbc3c05f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:50.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.303+0000 7fbc41857700 1 --2- 172.21.15.32:0/2508703862 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc3c05f3b0 0x7fbc3c05f790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fbc34001450 tx=0x7fbc2c006dc0).stop 2020-12-05T17:40:50.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.303+0000 7fbc41857700 1 -- 172.21.15.32:0/2508703862 shutdown_connections 2020-12-05T17:40:50.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.303+0000 7fbc41857700 1 --2- 172.21.15.32:0/2508703862 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc3c05f3b0 0x7fbc3c05f790 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:50.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.303+0000 7fbc41857700 1 -- 172.21.15.32:0/2508703862 >> 172.21.15.32:0/2508703862 conn(0x7fbc3c0b8f20 msgr2=0x7fbc3c0b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:50.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.303+0000 7fbc41857700 1 -- 172.21.15.32:0/2508703862 shutdown_connections 2020-12-05T17:40:50.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.303+0000 7fbc41857700 1 -- 172.21.15.32:0/2508703862 wait complete. 2020-12-05T17:40:50.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc41857700 1 Processor -- start 2020-12-05T17:40:50.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc41857700 1 -- start start 2020-12-05T17:40:50.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc41857700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc3c05f3b0 0x7fbc3c066500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:50.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc41857700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc3c05feb0 0x7fbc3c145fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:50.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc41857700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc3c0b2fb0 0x7fbc3c13e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:50.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc41857700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c141730 con 0x7fbc3c05feb0 2020-12-05T17:40:50.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc41857700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c13d1e0 con 0x7fbc3c05f3b0 2020-12-05T17:40:50.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc41857700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c13d480 con 0x7fbc3c0b2fb0 2020-12-05T17:40:50.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc3bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc3c05feb0 0x7fbc3c145fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:50.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc3bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc3c05feb0 0x7fbc3c145fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39848/0 (socket says 172.21.15.32:39848) 2020-12-05T17:40:50.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc3bfff700 1 -- 172.21.15.32:0/3704333152 learned_addr learned my addr 172.21.15.32:0/3704333152 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:50.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc40855700 1 --2- 172.21.15.32:0/3704333152 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc3c05f3b0 0x7fbc3c066500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:50.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc41056700 1 --2- 172.21.15.32:0/3704333152 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc3c0b2fb0 0x7fbc3c13e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:50.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc40855700 1 -- 172.21.15.32:0/3704333152 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc3c0b2fb0 msgr2=0x7fbc3c13e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:50.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc40855700 1 --2- 172.21.15.32:0/3704333152 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc3c0b2fb0 0x7fbc3c13e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:50.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc40855700 1 -- 172.21.15.32:0/3704333152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc3c05feb0 msgr2=0x7fbc3c145fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:50.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc40855700 1 --2- 172.21.15.32:0/3704333152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc3c05feb0 0x7fbc3c145fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:50.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc40855700 1 -- 172.21.15.32:0/3704333152 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3c066200 con 0x7fbc3c05f3b0 2020-12-05T17:40:50.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc40855700 1 --2- 172.21.15.32:0/3704333152 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc3c05f3b0 0x7fbc3c066500 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7fbc3c1514f0 tx=0x7fbc2c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:50.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc39ffb700 1 -- 172.21.15.32:0/3704333152 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc2c011b00 con 0x7fbc3c05f3b0 2020-12-05T17:40:50.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc39ffb700 1 -- 172.21.15.32:0/3704333152 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2c011c60 con 0x7fbc3c05f3b0 2020-12-05T17:40:50.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc41857700 1 -- 172.21.15.32:0/3704333152 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc3c13cdb0 con 0x7fbc3c05f3b0 2020-12-05T17:40:50.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc39ffb700 1 -- 172.21.15.32:0/3704333152 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc2c01d450 con 0x7fbc3c05f3b0 2020-12-05T17:40:50.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.307+0000 7fbc41857700 1 -- 172.21.15.32:0/3704333152 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc3c13ead0 con 0x7fbc3c05f3b0 2020-12-05T17:40:50.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.311+0000 7fbc41857700 1 -- 172.21.15.32:0/3704333152 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc3c13cff0 con 0x7fbc3c05f3b0 2020-12-05T17:40:50.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.311+0000 7fbc39ffb700 1 -- 172.21.15.32:0/3704333152 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbc2c024030 con 0x7fbc3c05f3b0 2020-12-05T17:40:50.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.311+0000 7fbc39ffb700 1 --2- 172.21.15.32:0/3704333152 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbc1c055330 0x7fbc1c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:50.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.311+0000 7fbc39ffb700 1 -- 172.21.15.32:0/3704333152 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6422+0+0 (secure 0 0 0) 0x7fbc2c06efc0 con 0x7fbc3c05f3b0 2020-12-05T17:40:50.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.311+0000 7fbc3bfff700 1 --2- 172.21.15.32:0/3704333152 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbc1c055330 0x7fbc1c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:50.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.311+0000 7fbc3bfff700 1 --2- 172.21.15.32:0/3704333152 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbc1c055330 0x7fbc1c0577f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7fbc3000d000 tx=0x7fbc30007b00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:50.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.319+0000 7fbc39ffb700 1 -- 172.21.15.32:0/3704333152 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbc2c045160 con 0x7fbc3c05f3b0 2020-12-05T17:40:50.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.507+0000 7fbc41857700 1 -- 172.21.15.32:0/3704333152 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbc3c057050 con 0x7fbc3c05f3b0 2020-12-05T17:40:50.514 INFO:tasks.ceph.mon.a.smithi032.stderr:2020-12-05T17:40:50.511+0000 7fdef5931700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-12-05T17:40:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.519+0000 7fbc39ffb700 1 -- 172.21.15.32:0/3704333152 <== mon.1 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 22 v57) v1 ==== 174+0+0 (secure 0 0 0) 0x7fbc2c022070 con 0x7fbc3c05f3b0 2020-12-05T17:40:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.523+0000 7fbc41857700 1 -- 172.21.15.32:0/3704333152 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbc1c055330 msgr2=0x7fbc1c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.523+0000 7fbc41857700 1 --2- 172.21.15.32:0/3704333152 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbc1c055330 0x7fbc1c0577f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7fbc3000d000 tx=0x7fbc30007b00).stop 2020-12-05T17:40:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.523+0000 7fbc41857700 1 -- 172.21.15.32:0/3704333152 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc3c05f3b0 msgr2=0x7fbc3c066500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.523+0000 7fbc41857700 1 --2- 172.21.15.32:0/3704333152 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc3c05f3b0 0x7fbc3c066500 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7fbc3c1514f0 tx=0x7fbc2c015000).stop 2020-12-05T17:40:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.523+0000 7fbc41857700 1 -- 172.21.15.32:0/3704333152 shutdown_connections 2020-12-05T17:40:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.523+0000 7fbc41857700 1 --2- 172.21.15.32:0/3704333152 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbc1c055330 0x7fbc1c0577f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.523+0000 7fbc41857700 1 --2- 172.21.15.32:0/3704333152 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc3c0b2fb0 0x7fbc3c13e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.523+0000 7fbc41857700 1 --2- 172.21.15.32:0/3704333152 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc3c05f3b0 0x7fbc3c066500 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.523+0000 7fbc41857700 1 --2- 172.21.15.32:0/3704333152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc3c05feb0 0x7fbc3c145fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:50.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.523+0000 7fbc41857700 1 -- 172.21.15.32:0/3704333152 >> 172.21.15.32:0/3704333152 conn(0x7fbc3c0b8f20 msgr2=0x7fbc3c061780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:50.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.523+0000 7fbc41857700 1 -- 172.21.15.32:0/3704333152 shutdown_connections 2020-12-05T17:40:50.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.523+0000 7fbc41857700 1 -- 172.21.15.32:0/3704333152 wait complete. 2020-12-05T17:40:50.532 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 24 and data pool 22 2020-12-05T17:40:50.543 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:40:50.544 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:40:50.561 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:40:50.575 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:40:50.592 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:40:50.609 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:40:50.627 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:40:50.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:50.628 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:40:50.628 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:40:50.641 INFO:teuthology.orchestra.run.smithi036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-12-05T17:40:50.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:40:50.643 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-12-05T17:40:50.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:40:50.644 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:40:50.689 INFO:teuthology.orchestra.run.smithi036.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-12-05T17:40:50.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:40:50.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:50.691 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:40:50.692 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:40:50.699 INFO:teuthology.orchestra.run.smithi002.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-12-05T17:40:50.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:40:50.700 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-12-05T17:40:50.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:40:50.701 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:40:50.747 INFO:teuthology.orchestra.run.smithi002.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-12-05T17:40:50.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:40:50.749 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:50.749 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:40:50.749 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:40:50.757 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:40:50.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:40:50.758 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2020-12-05T17:40:50.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:40:50.759 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:40:50.804 INFO:teuthology.orchestra.run.smithi053.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:40:50.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:40:50.806 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2020-12-05T17:40:50.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc61ee73700 1 Processor -- start 2020-12-05T17:40:50.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc61ee73700 1 -- start start 2020-12-05T17:40:50.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc61ee73700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc6180b6690 0x7fc6180b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:50.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc61ee73700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6180b56f0 con 0x7fc6180b5240 2020-12-05T17:40:50.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc61ee73700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6180b1920 con 0x7fc6180b5b20 2020-12-05T17:40:50.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc61ee73700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6180b1bc0 con 0x7fc6180b6690 2020-12-05T17:40:50.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc61d410700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc6180b5240 0x7fc618145f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42506/0 (socket says 172.21.15.32:42506) 2020-12-05T17:40:50.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc61d410700 1 -- 172.21.15.32:0/2567785374 learned_addr learned my addr 172.21.15.32:0/2567785374 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:50.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc617fff700 1 --2- 172.21.15.32:0/2567785374 >> v2:172.21.15.61:3300/0 conn(0x7fc6180b6690 0x7fc6180b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:50.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc6177fe700 1 -- 172.21.15.32:0/2567785374 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 905005669 0 0) 0x7fc6180b56f0 con 0x7fc6180b5240 2020-12-05T17:40:50.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc6177fe700 1 -- 172.21.15.32:0/2567785374 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5f8003230 con 0x7fc6180b5240 2020-12-05T17:40:50.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc617fff700 1 -- 172.21.15.32:0/2567785374 >> v1:172.21.15.61:6789/0 conn(0x7fc6180b5b20 legacy=0x7fc6180b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:50.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc617fff700 1 -- 172.21.15.32:0/2567785374 >> v1:172.21.15.32:6789/0 conn(0x7fc6180b5240 legacy=0x7fc618145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:50.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc617fff700 1 -- 172.21.15.32:0/2567785374 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc618146620 con 0x7fc6180b6690 2020-12-05T17:40:50.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc617fff700 1 --2- 172.21.15.32:0/2567785374 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc6180b6690 0x7fc6180b5f00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7fc600002f30 tx=0x7fc600006dc0).ready entity=mon.1 client_cookie=738068561883e994 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:50.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc6177fe700 1 -- 172.21.15.32:0/2567785374 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc600009800 con 0x7fc6180b6690 2020-12-05T17:40:50.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc6177fe700 1 -- 172.21.15.32:0/2567785374 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc600009960 con 0x7fc6180b6690 2020-12-05T17:40:50.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc6177fe700 1 -- 172.21.15.32:0/2567785374 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc600009cc0 con 0x7fc6180b6690 2020-12-05T17:40:50.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc61ee73700 1 -- 172.21.15.32:0/2567785374 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc6180b6690 msgr2=0x7fc6180b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:50.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc61ee73700 1 --2- 172.21.15.32:0/2567785374 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc6180b6690 0x7fc6180b5f00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7fc600002f30 tx=0x7fc600006dc0).stop 2020-12-05T17:40:50.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc61ee73700 1 -- 172.21.15.32:0/2567785374 shutdown_connections 2020-12-05T17:40:50.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc61ee73700 1 --2- 172.21.15.32:0/2567785374 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc6180b6690 0x7fc6180b5f00 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:50.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc61ee73700 1 -- 172.21.15.32:0/2567785374 >> 172.21.15.32:0/2567785374 conn(0x7fc61805f3b0 msgr2=0x7fc61805f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:50.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc61ee73700 1 -- 172.21.15.32:0/2567785374 shutdown_connections 2020-12-05T17:40:50.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.927+0000 7fc61ee73700 1 -- 172.21.15.32:0/2567785374 wait complete. 2020-12-05T17:40:50.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc61ee73700 1 Processor -- start 2020-12-05T17:40:50.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc61ee73700 1 -- start start 2020-12-05T17:40:50.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc61ee73700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc6180b5240 0x7fc618066b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:50.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc61ee73700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc6180b5b20 0x7fc618140160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:50.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc61ee73700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6180b6690 0x7fc618140650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:50.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc61ee73700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6180a3480 con 0x7fc6180b6690 2020-12-05T17:40:50.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc61ee73700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6180a3720 con 0x7fc6180b5240 2020-12-05T17:40:50.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc61ee73700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6180a39c0 con 0x7fc6180b5b20 2020-12-05T17:40:50.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc61d410700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6180b6690 0x7fc618140650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:50.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc61d410700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6180b6690 0x7fc618140650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39864/0 (socket says 172.21.15.32:39864) 2020-12-05T17:40:50.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc61d410700 1 -- 172.21.15.32:0/3674808026 learned_addr learned my addr 172.21.15.32:0/3674808026 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:50.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc61cc0f700 1 --2- 172.21.15.32:0/3674808026 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc6180b5240 0x7fc618066b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:50.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc617fff700 1 --2- 172.21.15.32:0/3674808026 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc6180b5b20 0x7fc618140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:50.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc61d410700 1 -- 172.21.15.32:0/3674808026 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc6180b5b20 msgr2=0x7fc618140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:50.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc61d410700 1 --2- 172.21.15.32:0/3674808026 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc6180b5b20 0x7fc618140160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:50.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc61d410700 1 -- 172.21.15.32:0/3674808026 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc6180b5240 msgr2=0x7fc618066b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:50.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc61d410700 1 --2- 172.21.15.32:0/3674808026 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc6180b5240 0x7fc618066b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:50.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc61d410700 1 -- 172.21.15.32:0/3674808026 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6180a30e0 con 0x7fc6180b6690 2020-12-05T17:40:50.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc61d410700 1 --2- 172.21.15.32:0/3674808026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6180b6690 0x7fc618140650 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 rx=0x7fc608001450 tx=0x7fc608007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc615ffb700 1 -- 172.21.15.32:0/3674808026 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc60800ce70 con 0x7fc6180b6690 2020-12-05T17:40:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc615ffb700 1 -- 172.21.15.32:0/3674808026 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc608005b70 con 0x7fc6180b6690 2020-12-05T17:40:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc61ee73700 1 -- 172.21.15.32:0/3674808026 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc618140b40 con 0x7fc6180b6690 2020-12-05T17:40:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc615ffb700 1 -- 172.21.15.32:0/3674808026 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc608007110 con 0x7fc6180b6690 2020-12-05T17:40:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.931+0000 7fc61ee73700 1 -- 172.21.15.32:0/3674808026 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6180a4080 con 0x7fc6180b6690 2020-12-05T17:40:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.935+0000 7fc61ee73700 1 -- 172.21.15.32:0/3674808026 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6180a3b80 con 0x7fc6180b6690 2020-12-05T17:40:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.935+0000 7fc615ffb700 1 -- 172.21.15.32:0/3674808026 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc608010030 con 0x7fc6180b6690 2020-12-05T17:40:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.935+0000 7fc615ffb700 1 --2- 172.21.15.32:0/3674808026 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc5fc0553b0 0x7fc5fc057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.935+0000 7fc61cc0f700 1 --2- 172.21.15.32:0/3674808026 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc5fc0553b0 0x7fc5fc057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.935+0000 7fc615ffb700 1 -- 172.21.15.32:0/3674808026 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fc60806e140 con 0x7fc6180b6690 2020-12-05T17:40:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.935+0000 7fc61cc0f700 1 --2- 172.21.15.32:0/3674808026 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc5fc0553b0 0x7fc5fc057870 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7fc61000d000 tx=0x7fc610007b60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:50.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:50.943+0000 7fc615ffb700 1 -- 172.21.15.32:0/3674808026 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc608044180 con 0x7fc6180b6690 2020-12-05T17:40:51.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.083+0000 7fc61ee73700 1 -- 172.21.15.32:0/3674808026 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc618057050 con 0x7fc6180b6690 2020-12-05T17:40:51.093 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.087+0000 7fc615ffb700 1 -- 172.21.15.32:0/3674808026 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc6080477a0 con 0x7fc6180b6690 2020-12-05T17:40:51.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.091+0000 7fc61ee73700 1 -- 172.21.15.32:0/3674808026 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc5fc0553b0 msgr2=0x7fc5fc057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:51.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.091+0000 7fc61ee73700 1 --2- 172.21.15.32:0/3674808026 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc5fc0553b0 0x7fc5fc057870 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7fc61000d000 tx=0x7fc610007b60).stop 2020-12-05T17:40:51.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.091+0000 7fc61ee73700 1 -- 172.21.15.32:0/3674808026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6180b6690 msgr2=0x7fc618140650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:51.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.091+0000 7fc61ee73700 1 --2- 172.21.15.32:0/3674808026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6180b6690 0x7fc618140650 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 rx=0x7fc608001450 tx=0x7fc608007e10).stop 2020-12-05T17:40:51.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.091+0000 7fc61ee73700 1 -- 172.21.15.32:0/3674808026 shutdown_connections 2020-12-05T17:40:51.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.091+0000 7fc61ee73700 1 --2- 172.21.15.32:0/3674808026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6180b6690 0x7fc618140650 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:51.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.091+0000 7fc61ee73700 1 --2- 172.21.15.32:0/3674808026 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc6180b5240 0x7fc618066b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:51.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.091+0000 7fc61ee73700 1 --2- 172.21.15.32:0/3674808026 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc5fc0553b0 0x7fc5fc057870 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:51.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.091+0000 7fc61ee73700 1 --2- 172.21.15.32:0/3674808026 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc6180b5b20 0x7fc618140160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:51.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.091+0000 7fc61ee73700 1 -- 172.21.15.32:0/3674808026 >> 172.21.15.32:0/3674808026 conn(0x7fc61805f3b0 msgr2=0x7fc618145e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:51.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.091+0000 7fc61ee73700 1 -- 172.21.15.32:0/3674808026 shutdown_connections 2020-12-05T17:40:51.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.091+0000 7fc61ee73700 1 -- 172.21.15.32:0/3674808026 wait complete. 2020-12-05T17:40:51.108 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new' 2020-12-05T17:40:51.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.219+0000 7f66fac50700 1 Processor -- start 2020-12-05T17:40:51.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.219+0000 7f66fac50700 1 -- start start 2020-12-05T17:40:51.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.219+0000 7f66fac50700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f66f4062400 0x7f66f405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:51.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.219+0000 7f66fac50700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f66f405feb0 0x7f66f40b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:51.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.219+0000 7f66fac50700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f4142450 con 0x7f66f405f3b0 2020-12-05T17:40:51.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.219+0000 7f66fac50700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f40b42f0 con 0x7f66f405feb0 2020-12-05T17:40:51.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.219+0000 7f66fac50700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f4061780 con 0x7f66f4062400 2020-12-05T17:40:51.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.219+0000 7f66f9c4e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f66f405f3b0 0x7f66f40b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42516/0 (socket says 172.21.15.32:42516) 2020-12-05T17:40:51.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.219+0000 7f66f9c4e700 1 -- 172.21.15.32:0/2982255352 learned_addr learned my addr 172.21.15.32:0/2982255352 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:51.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.219+0000 7f66fa44f700 1 --2- 172.21.15.32:0/2982255352 >> v2:172.21.15.61:3300/0 conn(0x7f66f405feb0 0x7f66f40b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:51.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.219+0000 7f66f944d700 1 --2- 172.21.15.32:0/2982255352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f66f4062400 0x7f66f405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:51.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.219+0000 7f66f8c4c700 1 -- 172.21.15.32:0/2982255352 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4102993110 0 0) 0x7f66f4142450 con 0x7f66f405f3b0 2020-12-05T17:40:51.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.219+0000 7f66f8c4c700 1 -- 172.21.15.32:0/2982255352 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66dc003230 con 0x7f66f405f3b0 2020-12-05T17:40:51.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.223+0000 7f66f944d700 1 -- 172.21.15.32:0/2982255352 >> v2:172.21.15.61:3300/0 conn(0x7f66f405feb0 msgr2=0x7f66f40b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:51.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.223+0000 7f66f944d700 1 --2- 172.21.15.32:0/2982255352 >> v2:172.21.15.61:3300/0 conn(0x7f66f405feb0 0x7f66f40b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:51.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.223+0000 7f66f944d700 1 -- 172.21.15.32:0/2982255352 >> v1:172.21.15.32:6789/0 conn(0x7f66f405f3b0 legacy=0x7f66f40b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:51.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.223+0000 7f66f944d700 1 -- 172.21.15.32:0/2982255352 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f4061a20 con 0x7f66f4062400 2020-12-05T17:40:51.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.223+0000 7f66f944d700 1 --2- 172.21.15.32:0/2982255352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f66f4062400 0x7f66f405f790 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f66e4002f30 tx=0x7f66e4006dc0).ready entity=mon.2 client_cookie=7866ce646956da4c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:51.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.223+0000 7f66f8c4c700 1 -- 172.21.15.32:0/2982255352 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66e40096a0 con 0x7f66f4062400 2020-12-05T17:40:51.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.223+0000 7f66f8c4c700 1 -- 172.21.15.32:0/2982255352 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e4009800 con 0x7f66f4062400 2020-12-05T17:40:51.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.223+0000 7f66f8c4c700 1 -- 172.21.15.32:0/2982255352 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66e4009b60 con 0x7f66f4062400 2020-12-05T17:40:51.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.223+0000 7f66fac50700 1 -- 172.21.15.32:0/2982255352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f66f4062400 msgr2=0x7f66f405f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:51.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.223+0000 7f66fac50700 1 --2- 172.21.15.32:0/2982255352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f66f4062400 0x7f66f405f790 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f66e4002f30 tx=0x7f66e4006dc0).stop 2020-12-05T17:40:51.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.223+0000 7f66fac50700 1 -- 172.21.15.32:0/2982255352 shutdown_connections 2020-12-05T17:40:51.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.223+0000 7f66fac50700 1 --2- 172.21.15.32:0/2982255352 >> v2:172.21.15.61:3300/0 conn(0x7f66f405feb0 0x7f66f40b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:51.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.223+0000 7f66fac50700 1 --2- 172.21.15.32:0/2982255352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f66f4062400 0x7f66f405f790 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:51.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.223+0000 7f66fac50700 1 -- 172.21.15.32:0/2982255352 >> 172.21.15.32:0/2982255352 conn(0x7f66f40b8ee0 msgr2=0x7f66f40b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:51.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.223+0000 7f66fac50700 1 -- 172.21.15.32:0/2982255352 shutdown_connections 2020-12-05T17:40:51.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.223+0000 7f66fac50700 1 -- 172.21.15.32:0/2982255352 wait complete. 2020-12-05T17:40:51.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.223+0000 7f66fac50700 1 Processor -- start 2020-12-05T17:40:51.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.223+0000 7f66fac50700 1 -- start start 2020-12-05T17:40:51.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66fac50700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f66f405f3b0 0x7f66f40b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:51.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66fac50700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f66f405feb0 0x7f66f4064e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:51.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66fac50700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66f4062400 0x7f66f40aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:51.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66fac50700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f4066800 con 0x7f66f4062400 2020-12-05T17:40:51.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66fac50700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f40a2a00 con 0x7f66f405f3b0 2020-12-05T17:40:51.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66fac50700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f40a2c60 con 0x7f66f405feb0 2020-12-05T17:40:51.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66fa44f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66f4062400 0x7f66f40aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:51.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66f9c4e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f66f405f3b0 0x7f66f40b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:51.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66f9c4e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f66f405f3b0 0x7f66f40b2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:59766/0 (socket says 172.21.15.32:59766) 2020-12-05T17:40:51.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66fa44f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66f4062400 0x7f66f40aecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39878/0 (socket says 172.21.15.32:39878) 2020-12-05T17:40:51.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66f9c4e700 1 -- 172.21.15.32:0/3922561369 learned_addr learned my addr 172.21.15.32:0/3922561369 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:51.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66f944d700 1 --2- 172.21.15.32:0/3922561369 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f66f405feb0 0x7f66f4064e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:51.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66fa44f700 1 -- 172.21.15.32:0/3922561369 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f66f405feb0 msgr2=0x7f66f4064e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66fa44f700 1 --2- 172.21.15.32:0/3922561369 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f66f405feb0 0x7f66f4064e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66fa44f700 1 -- 172.21.15.32:0/3922561369 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f66f405f3b0 msgr2=0x7f66f40b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66fa44f700 1 --2- 172.21.15.32:0/3922561369 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f66f405f3b0 0x7f66f40b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66fa44f700 1 -- 172.21.15.32:0/3922561369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f40a2f00 con 0x7f66f4062400 2020-12-05T17:40:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66fa44f700 1 --2- 172.21.15.32:0/3922561369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66f4062400 0x7f66f40aecf0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7f66ec00c000 tx=0x7f66ec0091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66e2ffd700 1 -- 172.21.15.32:0/3922561369 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66ec00c8f0 con 0x7f66f4062400 2020-12-05T17:40:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66e2ffd700 1 -- 172.21.15.32:0/3922561369 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ec00ca50 con 0x7f66f4062400 2020-12-05T17:40:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66fac50700 1 -- 172.21.15.32:0/3922561369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f40660f0 con 0x7f66f4062400 2020-12-05T17:40:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66e2ffd700 1 -- 172.21.15.32:0/3922561369 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66ec00cdb0 con 0x7f66f4062400 2020-12-05T17:40:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66fac50700 1 -- 172.21.15.32:0/3922561369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f40a3860 con 0x7f66f4062400 2020-12-05T17:40:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.227+0000 7f66fac50700 1 -- 172.21.15.32:0/3922561369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f4057050 con 0x7f66f4062400 2020-12-05T17:40:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.231+0000 7f66e2ffd700 1 -- 172.21.15.32:0/3922561369 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f66ec022070 con 0x7f66f4062400 2020-12-05T17:40:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.231+0000 7f66e2ffd700 1 --2- 172.21.15.32:0/3922561369 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f66d4055330 0x7f66d40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.231+0000 7f66f9c4e700 1 --2- 172.21.15.32:0/3922561369 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f66d4055330 0x7f66d40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.231+0000 7f66e2ffd700 1 -- 172.21.15.32:0/3922561369 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f66ec06ed40 con 0x7f66f4062400 2020-12-05T17:40:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.231+0000 7f66f9c4e700 1 --2- 172.21.15.32:0/3922561369 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f66d4055330 0x7f66d40577f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f66f000d000 tx=0x7f66f0009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.235+0000 7f66e2ffd700 1 -- 172.21.15.32:0/3922561369 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f66ec044e90 con 0x7f66f4062400 2020-12-05T17:40:51.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:51.427+0000 7f66fac50700 1 -- 172.21.15.32:0/3922561369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new"]} v 0) v1 -- 0x7f66f40a3380 con 0x7f66f4062400 2020-12-05T17:40:52.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.167+0000 7f66e2ffd700 1 -- 172.21.15.32:0/3922561369 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new"]}]=0 v419) v1 ==== 122+0+0 (secure 0 0 0) 0x7f66ec0290c0 con 0x7f66f4062400 2020-12-05T17:40:52.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.171+0000 7f66fac50700 1 -- 172.21.15.32:0/3922561369 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f66d4055330 msgr2=0x7f66d40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:52.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.171+0000 7f66fac50700 1 --2- 172.21.15.32:0/3922561369 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f66d4055330 0x7f66d40577f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f66f000d000 tx=0x7f66f0009040).stop 2020-12-05T17:40:52.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.171+0000 7f66fac50700 1 -- 172.21.15.32:0/3922561369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66f4062400 msgr2=0x7f66f40aecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:52.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.171+0000 7f66fac50700 1 --2- 172.21.15.32:0/3922561369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66f4062400 0x7f66f40aecf0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7f66ec00c000 tx=0x7f66ec0091a0).stop 2020-12-05T17:40:52.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.171+0000 7f66fac50700 1 -- 172.21.15.32:0/3922561369 shutdown_connections 2020-12-05T17:40:52.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.171+0000 7f66fac50700 1 --2- 172.21.15.32:0/3922561369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66f4062400 0x7f66f40aecf0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:52.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.171+0000 7f66fac50700 1 --2- 172.21.15.32:0/3922561369 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f66f405f3b0 0x7f66f40b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:52.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.171+0000 7f66fac50700 1 --2- 172.21.15.32:0/3922561369 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f66d4055330 0x7f66d40577f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:52.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.171+0000 7f66fac50700 1 --2- 172.21.15.32:0/3922561369 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f66f405feb0 0x7f66f4064e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:52.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.171+0000 7f66fac50700 1 -- 172.21.15.32:0/3922561369 >> 172.21.15.32:0/3922561369 conn(0x7f66f40b8ee0 msgr2=0x7f66f4142620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:52.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.171+0000 7f66fac50700 1 -- 172.21.15.32:0/3922561369 shutdown_connections 2020-12-05T17:40:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.171+0000 7f66fac50700 1 -- 172.21.15.32:0/3922561369 wait complete. 2020-12-05T17:40:52.191 INFO:tasks.cephfs_test_runner:test_fs_new (tasks.cephfs.test_misc.TestMisc) ... ok 2020-12-05T17:40:52.192 INFO:tasks.cephfs_test_runner:Starting test: test_getattr_caps (tasks.cephfs.test_misc.TestMisc) 2020-12-05T17:40:52.192 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps' 2020-12-05T17:40:52.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.303+0000 7f072dae4700 1 Processor -- start 2020-12-05T17:40:52.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.303+0000 7f072dae4700 1 -- start start 2020-12-05T17:40:52.311 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.307+0000 7f072dae4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07280b1870 0x7f07280b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:52.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.307+0000 7f072dae4700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f07280ae860 0x7f07280aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:52.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.307+0000 7f072dae4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07281420f0 con 0x7f07280b52f0 2020-12-05T17:40:52.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.307+0000 7f072dae4700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07280ad4f0 con 0x7f07280ae860 2020-12-05T17:40:52.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.307+0000 7f072dae4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07280b0380 con 0x7f07280b1870 2020-12-05T17:40:52.312 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.307+0000 7f072cae2700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f07280b52f0 0x7f0728138810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42530/0 (socket says 172.21.15.32:42530) 2020-12-05T17:40:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.307+0000 7f072cae2700 1 -- 172.21.15.32:0/306821737 learned_addr learned my addr 172.21.15.32:0/306821737 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.307+0000 7f072d2e3700 1 --2- 172.21.15.32:0/306821737 >> v2:172.21.15.61:3300/0 conn(0x7f07280ae860 0x7f07280aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.307+0000 7f0727fff700 1 --2- 172.21.15.32:0/306821737 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07280b1870 0x7f07280b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:52.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.307+0000 7f07277fe700 1 -- 172.21.15.32:0/306821737 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 429323534 0 0) 0x7f07281420f0 con 0x7f07280b52f0 2020-12-05T17:40:52.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.307+0000 7f07277fe700 1 -- 172.21.15.32:0/306821737 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f070c003230 con 0x7f07280b52f0 2020-12-05T17:40:52.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.307+0000 7f072d2e3700 1 -- 172.21.15.32:0/306821737 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07280b1870 msgr2=0x7f07280b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:52.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.307+0000 7f072d2e3700 1 --2- 172.21.15.32:0/306821737 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07280b1870 0x7f07280b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:52.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.307+0000 7f072d2e3700 1 -- 172.21.15.32:0/306821737 >> v1:172.21.15.32:6789/0 conn(0x7f07280b52f0 legacy=0x7f0728138810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:52.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.307+0000 7f072d2e3700 1 -- 172.21.15.32:0/306821737 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07280b0620 con 0x7f07280ae860 2020-12-05T17:40:52.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.307+0000 7f072d2e3700 1 --2- 172.21.15.32:0/306821737 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07280ae860 0x7f07280aecf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f07200032b0 tx=0x7f07200091a0).ready entity=mon.1 client_cookie=876747cdfd9b81fb server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:52.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.307+0000 7f07277fe700 1 -- 172.21.15.32:0/306821737 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f072000b940 con 0x7f07280ae860 2020-12-05T17:40:52.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.307+0000 7f07277fe700 1 -- 172.21.15.32:0/306821737 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072000baa0 con 0x7f07280ae860 2020-12-05T17:40:52.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.307+0000 7f07277fe700 1 -- 172.21.15.32:0/306821737 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f072000bc10 con 0x7f07280ae860 2020-12-05T17:40:52.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.307+0000 7f072dae4700 1 -- 172.21.15.32:0/306821737 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07280ae860 msgr2=0x7f07280aecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:52.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.307+0000 7f072dae4700 1 --2- 172.21.15.32:0/306821737 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07280ae860 0x7f07280aecf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f07200032b0 tx=0x7f07200091a0).stop 2020-12-05T17:40:52.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.311+0000 7f072dae4700 1 -- 172.21.15.32:0/306821737 shutdown_connections 2020-12-05T17:40:52.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.311+0000 7f072dae4700 1 --2- 172.21.15.32:0/306821737 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07280ae860 0x7f07280aecf0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:52.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.311+0000 7f072dae4700 1 --2- 172.21.15.32:0/306821737 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07280b1870 0x7f07280b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:52.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.311+0000 7f072dae4700 1 -- 172.21.15.32:0/306821737 >> 172.21.15.32:0/306821737 conn(0x7f072805bbb0 msgr2=0x7f0728062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:52.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.311+0000 7f072dae4700 1 -- 172.21.15.32:0/306821737 shutdown_connections 2020-12-05T17:40:52.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.311+0000 7f072dae4700 1 -- 172.21.15.32:0/306821737 wait complete. 2020-12-05T17:40:52.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.311+0000 7f072dae4700 1 Processor -- start 2020-12-05T17:40:52.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.311+0000 7f072dae4700 1 -- start start 2020-12-05T17:40:52.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.311+0000 7f072dae4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07280ae860 0x7f072813b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:52.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.311+0000 7f072dae4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07280b1870 0x7f0728144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:52.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.311+0000 7f072dae4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07280b52f0 0x7f0728144810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:52.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.311+0000 7f072dae4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07280644a0 con 0x7f07280b1870 2020-12-05T17:40:52.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.311+0000 7f072dae4700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0728143e80 con 0x7f07280b52f0 2020-12-05T17:40:52.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.311+0000 7f072dae4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0728064910 con 0x7f07280ae860 2020-12-05T17:40:52.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.311+0000 7f0727fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07280b1870 0x7f0728144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:52.323 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.311+0000 7f0727fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07280b1870 0x7f0728144320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39890/0 (socket says 172.21.15.32:39890) 2020-12-05T17:40:52.323 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.311+0000 7f0727fff700 1 -- 172.21.15.32:0/4050737259 learned_addr learned my addr 172.21.15.32:0/4050737259 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:52.323 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.311+0000 7f072cae2700 1 --2- 172.21.15.32:0/4050737259 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07280ae860 0x7f072813b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:52.323 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.311+0000 7f072d2e3700 1 --2- 172.21.15.32:0/4050737259 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07280b52f0 0x7f0728144810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:52.323 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.315+0000 7f0727fff700 1 -- 172.21.15.32:0/4050737259 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07280ae860 msgr2=0x7f072813b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:52.324 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.315+0000 7f0727fff700 1 --2- 172.21.15.32:0/4050737259 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07280ae860 0x7f072813b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:52.324 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.315+0000 7f0727fff700 1 -- 172.21.15.32:0/4050737259 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07280b52f0 msgr2=0x7f0728144810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:52.324 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.315+0000 7f0727fff700 1 --2- 172.21.15.32:0/4050737259 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07280b52f0 0x7f0728144810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:52.324 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.315+0000 7f0727fff700 1 -- 172.21.15.32:0/4050737259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07280667b0 con 0x7f07280b1870 2020-12-05T17:40:52.324 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.315+0000 7f072cae2700 1 --2- 172.21.15.32:0/4050737259 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07280ae860 0x7f072813b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:40:52.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.315+0000 7f0727fff700 1 --2- 172.21.15.32:0/4050737259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07280b1870 0x7f0728144320 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 rx=0x7f071400c000 tx=0x7f0714006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:52.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.315+0000 7f0725ffb700 1 -- 172.21.15.32:0/4050737259 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f071400ca10 con 0x7f07280b1870 2020-12-05T17:40:52.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.315+0000 7f0725ffb700 1 -- 172.21.15.32:0/4050737259 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071400cb70 con 0x7f07280b1870 2020-12-05T17:40:52.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.315+0000 7f072dae4700 1 -- 172.21.15.32:0/4050737259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0728066a60 con 0x7f07280b1870 2020-12-05T17:40:52.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.315+0000 7f0725ffb700 1 -- 172.21.15.32:0/4050737259 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f071400ee80 con 0x7f07280b1870 2020-12-05T17:40:52.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.315+0000 7f072dae4700 1 -- 172.21.15.32:0/4050737259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07280656f0 con 0x7f07280b1870 2020-12-05T17:40:52.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.315+0000 7f072dae4700 1 -- 172.21.15.32:0/4050737259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0728057050 con 0x7f07280b1870 2020-12-05T17:40:52.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.315+0000 7f0725ffb700 1 -- 172.21.15.32:0/4050737259 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f071401c460 con 0x7f07280b1870 2020-12-05T17:40:52.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.315+0000 7f0725ffb700 1 --2- 172.21.15.32:0/4050737259 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0710055440 0x7f0710057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:52.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.315+0000 7f072cae2700 1 --2- 172.21.15.32:0/4050737259 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0710055440 0x7f0710057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:52.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.319+0000 7f0725ffb700 1 -- 172.21.15.32:0/4050737259 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f071406e5b0 con 0x7f07280b1870 2020-12-05T17:40:52.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.319+0000 7f072cae2700 1 --2- 172.21.15.32:0/4050737259 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0710055440 0x7f0710057900 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f071c00d000 tx=0x7f071c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:52.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.323+0000 7f0725ffb700 1 -- 172.21.15.32:0/4050737259 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0714044590 con 0x7f07280b1870 2020-12-05T17:40:52.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:52.519+0000 7f072dae4700 1 -- 172.21.15.32:0/4050737259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]} v 0) v1 -- 0x7f072805c300 con 0x7f07280b1870 2020-12-05T17:40:53.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.175+0000 7f0725ffb700 1 -- 172.21.15.32:0/4050737259 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]}]=0 v420) v1 ==== 131+0+0 (secure 0 0 0) 0x7f0714047bb0 con 0x7f07280b1870 2020-12-05T17:40:53.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.179+0000 7f072dae4700 1 -- 172.21.15.32:0/4050737259 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0710055440 msgr2=0x7f0710057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.179+0000 7f072dae4700 1 --2- 172.21.15.32:0/4050737259 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0710055440 0x7f0710057900 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f071c00d000 tx=0x7f071c009040).stop 2020-12-05T17:40:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.179+0000 7f072dae4700 1 -- 172.21.15.32:0/4050737259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07280b1870 msgr2=0x7f0728144320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.179+0000 7f072dae4700 1 --2- 172.21.15.32:0/4050737259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07280b1870 0x7f0728144320 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 rx=0x7f071400c000 tx=0x7f0714006dc0).stop 2020-12-05T17:40:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.179+0000 7f072dae4700 1 -- 172.21.15.32:0/4050737259 shutdown_connections 2020-12-05T17:40:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.179+0000 7f072dae4700 1 --2- 172.21.15.32:0/4050737259 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07280b52f0 0x7f0728144810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:53.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.179+0000 7f072dae4700 1 --2- 172.21.15.32:0/4050737259 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0710055440 0x7f0710057900 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:53.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.179+0000 7f072dae4700 1 --2- 172.21.15.32:0/4050737259 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07280ae860 0x7f072813b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:53.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.179+0000 7f072dae4700 1 --2- 172.21.15.32:0/4050737259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07280b1870 0x7f0728144320 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:53.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.179+0000 7f072dae4700 1 -- 172.21.15.32:0/4050737259 >> 172.21.15.32:0/4050737259 conn(0x7f072805bbb0 msgr2=0x7f0728141d90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:53.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.179+0000 7f072dae4700 1 -- 172.21.15.32:0/4050737259 shutdown_connections 2020-12-05T17:40:53.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.179+0000 7f072dae4700 1 -- 172.21.15.32:0/4050737259 wait complete. 2020-12-05T17:40:53.199 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-12-05T17:40:53.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.311+0000 7fafa6c2f700 1 Processor -- start 2020-12-05T17:40:53.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.311+0000 7fafa6c2f700 1 -- start start 2020-12-05T17:40:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.311+0000 7fafa6c2f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fafa00b2000 0x7fafa0142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa6c2f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fafa0143410 0x7fafa013f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa6c2f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fafa013fb00 0x7fafa00af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:53.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa6c2f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafa00aeba0 con 0x7fafa0143410 2020-12-05T17:40:53.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa6c2f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafa00b23e0 con 0x7fafa013fb00 2020-12-05T17:40:53.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa6c2f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa00af190 con 0x7fafa00b2000 2020-12-05T17:40:53.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa542c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fafa0143410 0x7fafa013f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:53.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa542c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fafa0143410 0x7fafa013f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39898/0 (socket says 172.21.15.32:39898) 2020-12-05T17:40:53.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa542c700 1 -- 172.21.15.32:0/3703645377 learned_addr learned my addr 172.21.15.32:0/3703645377 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:53.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa5c2d700 1 --2- 172.21.15.32:0/3703645377 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fafa00b2000 0x7fafa0142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:53.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa642e700 1 --2- 172.21.15.32:0/3703645377 >> v2:172.21.15.61:3300/0 conn(0x7fafa013fb00 0x7fafa00af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:53.321 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa542c700 1 -- 172.21.15.32:0/3703645377 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fafa00b2000 msgr2=0x7fafa0142f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:53.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa542c700 1 --2- 172.21.15.32:0/3703645377 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fafa00b2000 0x7fafa0142f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:53.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa542c700 1 -- 172.21.15.32:0/3703645377 >> v2:172.21.15.61:3300/0 conn(0x7fafa013fb00 msgr2=0x7fafa00af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:53.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa542c700 1 --2- 172.21.15.32:0/3703645377 >> v2:172.21.15.61:3300/0 conn(0x7fafa013fb00 0x7fafa00af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:53.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa542c700 1 -- 172.21.15.32:0/3703645377 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa00ae010 con 0x7fafa0143410 2020-12-05T17:40:53.322 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa642e700 1 --2- 172.21.15.32:0/3703645377 >> v2:172.21.15.61:3300/0 conn(0x7fafa013fb00 0x7fafa00af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:40:53.323 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa542c700 1 --2- 172.21.15.32:0/3703645377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafa0143410 0x7fafa013f610 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7faf9c002f30 tx=0x7faf9c006e80).ready entity=mon.0 client_cookie=c172f71fbfb3e486 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:53.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa4c2b700 1 -- 172.21.15.32:0/3703645377 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf9c0097f0 con 0x7fafa0143410 2020-12-05T17:40:53.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa4c2b700 1 -- 172.21.15.32:0/3703645377 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9c009950 con 0x7fafa0143410 2020-12-05T17:40:53.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa4c2b700 1 -- 172.21.15.32:0/3703645377 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf9c009cb0 con 0x7fafa0143410 2020-12-05T17:40:53.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa6c2f700 1 -- 172.21.15.32:0/3703645377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafa0143410 msgr2=0x7fafa013f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:53.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa6c2f700 1 --2- 172.21.15.32:0/3703645377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafa0143410 0x7fafa013f610 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7faf9c002f30 tx=0x7faf9c006e80).stop 2020-12-05T17:40:53.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa6c2f700 1 -- 172.21.15.32:0/3703645377 shutdown_connections 2020-12-05T17:40:53.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa6c2f700 1 --2- 172.21.15.32:0/3703645377 >> v2:172.21.15.61:3300/0 conn(0x7fafa013fb00 0x7fafa00af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:53.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa6c2f700 1 --2- 172.21.15.32:0/3703645377 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fafa00b2000 0x7fafa0142f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:53.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa6c2f700 1 --2- 172.21.15.32:0/3703645377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafa0143410 0x7fafa013f610 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:53.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa6c2f700 1 -- 172.21.15.32:0/3703645377 >> 172.21.15.32:0/3703645377 conn(0x7fafa00b5620 msgr2=0x7fafa00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:53.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa6c2f700 1 -- 172.21.15.32:0/3703645377 shutdown_connections 2020-12-05T17:40:53.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.315+0000 7fafa6c2f700 1 -- 172.21.15.32:0/3703645377 wait complete. 2020-12-05T17:40:53.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7fafa6c2f700 1 Processor -- start 2020-12-05T17:40:53.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7fafa6c2f700 1 -- start start 2020-12-05T17:40:53.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7fafa6c2f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fafa00b2000 0x7fafa0148030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:53.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7fafa6c2f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafa013fb00 0x7fafa013ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:53.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7fafa6c2f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fafa0143410 0x7fafa01551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:53.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7fafa6c2f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa0141eb0 con 0x7fafa013fb00 2020-12-05T17:40:53.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7fafa6c2f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa0154ab0 con 0x7fafa00b2000 2020-12-05T17:40:53.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7fafa6c2f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa0154d50 con 0x7fafa0143410 2020-12-05T17:40:53.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7fafa542c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafa013fb00 0x7fafa013ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:53.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7fafa5c2d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fafa00b2000 0x7fafa0148030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:53.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7fafa642e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fafa0143410 0x7fafa01551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:53.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7fafa542c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafa013fb00 0x7fafa013ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39904/0 (socket says 172.21.15.32:39904) 2020-12-05T17:40:53.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7fafa542c700 1 -- 172.21.15.32:0/252145375 learned_addr learned my addr 172.21.15.32:0/252145375 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:53.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7fafa542c700 1 -- 172.21.15.32:0/252145375 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fafa0143410 msgr2=0x7fafa01551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:53.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7fafa542c700 1 --2- 172.21.15.32:0/252145375 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fafa0143410 0x7fafa01551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:53.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7fafa542c700 1 -- 172.21.15.32:0/252145375 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fafa00b2000 msgr2=0x7fafa0148030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:53.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7fafa542c700 1 --2- 172.21.15.32:0/252145375 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fafa00b2000 0x7fafa0148030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:53.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7fafa542c700 1 -- 172.21.15.32:0/252145375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa0141ba0 con 0x7fafa013fb00 2020-12-05T17:40:53.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7fafa542c700 1 --2- 172.21.15.32:0/252145375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafa013fb00 0x7fafa013ff70 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7faf9c002f30 tx=0x7faf9c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:53.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7faf96ffd700 1 -- 172.21.15.32:0/252145375 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf9c011b40 con 0x7fafa013fb00 2020-12-05T17:40:53.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7faf96ffd700 1 -- 172.21.15.32:0/252145375 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9c011ca0 con 0x7fafa013fb00 2020-12-05T17:40:53.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7fafa6c2f700 1 -- 172.21.15.32:0/252145375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa0152570 con 0x7fafa013fb00 2020-12-05T17:40:53.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7faf96ffd700 1 -- 172.21.15.32:0/252145375 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf9c01e460 con 0x7fafa013fb00 2020-12-05T17:40:53.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.319+0000 7fafa6c2f700 1 -- 172.21.15.32:0/252145375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa0146970 con 0x7fafa013fb00 2020-12-05T17:40:53.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.323+0000 7fafa6c2f700 1 -- 172.21.15.32:0/252145375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafa0154f10 con 0x7fafa013fb00 2020-12-05T17:40:53.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.323+0000 7faf96ffd700 1 -- 172.21.15.32:0/252145375 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7faf9c01a040 con 0x7fafa013fb00 2020-12-05T17:40:53.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.323+0000 7faf96ffd700 1 --2- 172.21.15.32:0/252145375 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faf800553b0 0x7faf80057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:53.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.323+0000 7faf96ffd700 1 -- 172.21.15.32:0/252145375 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6506+0+0 (secure 0 0 0) 0x7faf9c06f200 con 0x7fafa013fb00 2020-12-05T17:40:53.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.323+0000 7fafa5c2d700 1 --2- 172.21.15.32:0/252145375 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faf800553b0 0x7faf80057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:53.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.323+0000 7fafa5c2d700 1 --2- 172.21.15.32:0/252145375 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faf800553b0 0x7faf80057870 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7faf9000b000 tx=0x7faf900091a0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:53.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.331+0000 7faf96ffd700 1 -- 172.21.15.32:0/252145375 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7faf9c0451c0 con 0x7fafa013fb00 2020-12-05T17:40:53.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.475+0000 7fafa6c2f700 1 -- 172.21.15.32:0/252145375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7fafa0057050 con 0x7fafa013fb00 2020-12-05T17:40:53.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.483+0000 7faf96ffd700 1 -- 172.21.15.32:0/252145375 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7faf9c0230c0 con 0x7fafa013fb00 2020-12-05T17:40:53.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.483+0000 7fafa6c2f700 1 -- 172.21.15.32:0/252145375 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faf800553b0 msgr2=0x7faf80057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:53.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.483+0000 7fafa6c2f700 1 --2- 172.21.15.32:0/252145375 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faf800553b0 0x7faf80057870 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7faf9000b000 tx=0x7faf900091a0).stop 2020-12-05T17:40:53.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.483+0000 7fafa6c2f700 1 -- 172.21.15.32:0/252145375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafa013fb00 msgr2=0x7fafa013ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:53.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.483+0000 7fafa6c2f700 1 --2- 172.21.15.32:0/252145375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafa013fb00 0x7fafa013ff70 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7faf9c002f30 tx=0x7faf9c015000).stop 2020-12-05T17:40:53.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.483+0000 7fafa6c2f700 1 -- 172.21.15.32:0/252145375 shutdown_connections 2020-12-05T17:40:53.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.483+0000 7fafa6c2f700 1 --2- 172.21.15.32:0/252145375 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faf800553b0 0x7faf80057870 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:53.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.483+0000 7fafa6c2f700 1 --2- 172.21.15.32:0/252145375 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fafa0143410 0x7fafa01551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:53.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.483+0000 7fafa6c2f700 1 --2- 172.21.15.32:0/252145375 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fafa00b2000 0x7fafa0148030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:53.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.483+0000 7fafa6c2f700 1 --2- 172.21.15.32:0/252145375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafa013fb00 0x7fafa013ff70 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:53.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.483+0000 7fafa6c2f700 1 -- 172.21.15.32:0/252145375 >> 172.21.15.32:0/252145375 conn(0x7fafa00b5620 msgr2=0x7fafa0061780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:53.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.483+0000 7fafa6c2f700 1 -- 172.21.15.32:0/252145375 shutdown_connections 2020-12-05T17:40:53.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.483+0000 7fafa6c2f700 1 -- 172.21.15.32:0/252145375 wait complete. 2020-12-05T17:40:53.507 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:40:53.507 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:40:53.524 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:40:53.541 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:40:53.558 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:40:53.574 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:40:53.592 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:40:53.592 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:53.593 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:40:53.593 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:40:53.605 INFO:teuthology.orchestra.run.smithi036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-12-05T17:40:53.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:40:53.607 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-12-05T17:40:53.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:53.608 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:40:53.608 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:40:53.615 INFO:teuthology.orchestra.run.smithi002.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-12-05T17:40:53.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:40:53.616 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-12-05T17:40:53.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:40:53.617 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:40:53.617 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:40:53.624 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:40:53.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:40:53.625 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2020-12-05T17:40:53.626 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-12-05T17:40:53.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.743+0000 7fb8fcf64700 1 Processor -- start 2020-12-05T17:40:53.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.743+0000 7fb8fcf64700 1 -- start start 2020-12-05T17:40:53.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.743+0000 7fb8fcf64700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8f80b52f0 0x7fb8f80b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:53.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.743+0000 7fb8fcf64700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8f80b1870 0x7fb8f8138810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:53.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.743+0000 7fb8fcf64700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f81420f0 con 0x7fb8f8138d00 2020-12-05T17:40:53.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.743+0000 7fb8fcf64700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f80ad4f0 con 0x7fb8f80b1870 2020-12-05T17:40:53.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.743+0000 7fb8fcf64700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f80b0380 con 0x7fb8f80b52f0 2020-12-05T17:40:53.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.743+0000 7fb8f6ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8f80b1870 0x7fb8f8138810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:53.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.743+0000 7fb8f7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb8f8138d00 0x7fb8f80ae860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42562/0 (socket says 172.21.15.32:42562) 2020-12-05T17:40:53.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.743+0000 7fb8f7fff700 1 -- 172.21.15.32:0/868085016 learned_addr learned my addr 172.21.15.32:0/868085016 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:53.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.743+0000 7fb8f77fe700 1 --2- 172.21.15.32:0/868085016 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8f80b52f0 0x7fb8f80b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:53.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.743+0000 7fb8f6ffd700 1 -- 172.21.15.32:0/868085016 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8f80b52f0 msgr2=0x7fb8f80b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:53.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.743+0000 7fb8f6ffd700 1 --2- 172.21.15.32:0/868085016 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8f80b52f0 0x7fb8f80b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:53.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.743+0000 7fb8f6ffd700 1 -- 172.21.15.32:0/868085016 >> v1:172.21.15.32:6789/0 conn(0x7fb8f8138d00 legacy=0x7fb8f80ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:53.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.743+0000 7fb8f67fc700 1 -- 172.21.15.32:0/868085016 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 625826997 0 0) 0x7fb8f81420f0 con 0x7fb8f8138d00 2020-12-05T17:40:53.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.743+0000 7fb8f6ffd700 1 -- 172.21.15.32:0/868085016 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f80b0620 con 0x7fb8f80b1870 2020-12-05T17:40:53.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.743+0000 7fb8f6ffd700 1 --2- 172.21.15.32:0/868085016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8f80b1870 0x7fb8f8138810 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7fb8f80adc80 tx=0x7fb8ec006dc0).ready entity=mon.0 client_cookie=e42f5f08b3568e27 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:53.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.743+0000 7fb8f67fc700 1 -- 172.21.15.32:0/868085016 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8ec009720 con 0x7fb8f80b1870 2020-12-05T17:40:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.743+0000 7fb8f67fc700 1 -- 172.21.15.32:0/868085016 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ec009880 con 0x7fb8f80b1870 2020-12-05T17:40:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.743+0000 7fb8fcf64700 1 -- 172.21.15.32:0/868085016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8f80b1870 msgr2=0x7fb8f8138810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.743+0000 7fb8f67fc700 1 -- 172.21.15.32:0/868085016 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8ec009be0 con 0x7fb8f80b1870 2020-12-05T17:40:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.743+0000 7fb8fcf64700 1 --2- 172.21.15.32:0/868085016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8f80b1870 0x7fb8f8138810 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7fb8f80adc80 tx=0x7fb8ec006dc0).stop 2020-12-05T17:40:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8fcf64700 1 -- 172.21.15.32:0/868085016 shutdown_connections 2020-12-05T17:40:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8fcf64700 1 --2- 172.21.15.32:0/868085016 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8f80b52f0 0x7fb8f80b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8fcf64700 1 --2- 172.21.15.32:0/868085016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8f80b1870 0x7fb8f8138810 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8fcf64700 1 -- 172.21.15.32:0/868085016 >> 172.21.15.32:0/868085016 conn(0x7fb8f805bbb0 msgr2=0x7fb8f8062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8fcf64700 1 -- 172.21.15.32:0/868085016 shutdown_connections 2020-12-05T17:40:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8fcf64700 1 -- 172.21.15.32:0/868085016 wait complete. 2020-12-05T17:40:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8fcf64700 1 Processor -- start 2020-12-05T17:40:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8fcf64700 1 -- start start 2020-12-05T17:40:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8fcf64700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb8f80b1870 0x7fb8f813f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8fcf64700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8f80b52f0 0x7fb8f8151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:53.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8fcf64700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8f8138d00 0x7fb8f8151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:53.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8fcf64700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f8066210 con 0x7fb8f8138d00 2020-12-05T17:40:53.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8fcf64700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f80664b0 con 0x7fb8f80b1870 2020-12-05T17:40:53.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8fcf64700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f8066750 con 0x7fb8f80b52f0 2020-12-05T17:40:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8f7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8f8138d00 0x7fb8f8151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8f6ffd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8f80b52f0 0x7fb8f8151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8f77fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb8f80b1870 0x7fb8f813f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8f7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8f8138d00 0x7fb8f8151b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39920/0 (socket says 172.21.15.32:39920) 2020-12-05T17:40:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8f7fff700 1 -- 172.21.15.32:0/3753123337 learned_addr learned my addr 172.21.15.32:0/3753123337 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8f6ffd700 1 -- 172.21.15.32:0/3753123337 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb8f80b1870 msgr2=0x7fb8f813f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8f6ffd700 1 --2- 172.21.15.32:0/3753123337 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb8f80b1870 0x7fb8f813f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8f6ffd700 1 -- 172.21.15.32:0/3753123337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8f8138d00 msgr2=0x7fb8f8151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8f6ffd700 1 --2- 172.21.15.32:0/3753123337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8f8138d00 0x7fb8f8151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8f6ffd700 1 -- 172.21.15.32:0/3753123337 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f80ad630 con 0x7fb8f80b52f0 2020-12-05T17:40:53.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8f7fff700 1 --2- 172.21.15.32:0/3753123337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8f8138d00 0x7fb8f8151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:40:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8f6ffd700 1 --2- 172.21.15.32:0/3753123337 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8f80b52f0 0x7fb8f8151660 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fb8f80adc80 tx=0x7fb8ec015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8f4ff9700 1 -- 172.21.15.32:0/3753123337 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8ec011b80 con 0x7fb8f80b52f0 2020-12-05T17:40:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8f4ff9700 1 -- 172.21.15.32:0/3753123337 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ec011ce0 con 0x7fb8f80b52f0 2020-12-05T17:40:53.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8fcf64700 1 -- 172.21.15.32:0/3753123337 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f813e9a0 con 0x7fb8f80b52f0 2020-12-05T17:40:53.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8f4ff9700 1 -- 172.21.15.32:0/3753123337 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8ec01d640 con 0x7fb8f80b52f0 2020-12-05T17:40:53.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.747+0000 7fb8fcf64700 1 -- 172.21.15.32:0/3753123337 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f8152260 con 0x7fb8f80b52f0 2020-12-05T17:40:53.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.751+0000 7fb8fcf64700 1 -- 172.21.15.32:0/3753123337 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f8066050 con 0x7fb8f80b52f0 2020-12-05T17:40:53.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.751+0000 7fb8f4ff9700 1 -- 172.21.15.32:0/3753123337 <== mon.2 v2:172.21.15.61:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb8ec00f070 con 0x7fb8f80b52f0 2020-12-05T17:40:53.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.751+0000 7fb8f4ff9700 1 --2- 172.21.15.32:0/3753123337 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8d80553b0 0x7fb8d8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:53.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.751+0000 7fb8f4ff9700 1 -- 172.21.15.32:0/3753123337 <== mon.2 v2:172.21.15.61:3301/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fb8ec06edf0 con 0x7fb8f80b52f0 2020-12-05T17:40:53.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.751+0000 7fb8f77fe700 1 --2- 172.21.15.32:0/3753123337 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8d80553b0 0x7fb8d8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:53.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.751+0000 7fb8f77fe700 1 --2- 172.21.15.32:0/3753123337 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8d80553b0 0x7fb8d8057870 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7fb8e800b000 tx=0x7fb8e80091a0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:53.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.763+0000 7fb8f4ff9700 1 -- 172.21.15.32:0/3753123337 <== mon.2 v2:172.21.15.61:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb8ec044ea0 con 0x7fb8f80b52f0 2020-12-05T17:40:53.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.903+0000 7fb8fcf64700 1 -- 172.21.15.32:0/3753123337 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb8f8057050 con 0x7fb8f80b52f0 2020-12-05T17:40:53.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.907+0000 7fb8f4ff9700 1 -- 172.21.15.32:0/3753123337 <== mon.2 v2:172.21.15.61:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v122) v1 ==== 81+0+34205 (secure 0 0 0) 0x7fb8ec022020 con 0x7fb8f80b52f0 2020-12-05T17:40:53.913 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:40:53.913 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:40:53.914 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 122, 2020-12-05T17:40:53.914 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", 2020-12-05T17:40:53.914 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2020-12-05T17:31:20.975238+0000", 2020-12-05T17:40:53.915 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2020-12-05T17:40:51.517421+0000", 2020-12-05T17:40:53.915 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:40:53.915 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:40:53.915 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-12-05T17:40:53.915 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2020-12-05T17:40:53.916 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2020-12-05T17:40:53.916 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2020-12-05T17:40:53.916 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2020-12-05T17:40:53.916 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2020-12-05T17:40:53.916 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2020-12-05T17:40:53.917 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:40:53.917 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2020-12-05T17:40:53.917 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2020-12-05T17:40:53.917 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-12-05T17:40:53.918 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2020-12-05T17:40:53.918 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2020-12-05T17:40:53.918 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 24, 2020-12-05T17:40:53.918 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2020-12-05T17:40:53.918 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2020-12-05T17:40:53.919 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2020-12-05T17:40:53.919 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2020-12-05T17:40:53.919 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2020-12-05T17:40:53.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:53.920 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2020-12-05T17:40:53.920 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2020-12-05T17:40:53.920 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:24.858732+0000", 2020-12-05T17:40:53.920 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:40:53.920 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:40:53.921 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:40:53.921 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:40:53.921 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:40:53.921 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:40:53.921 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:40:53.922 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:40:53.922 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2020-12-05T17:40:53.922 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2020-12-05T17:40:53.922 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2020-12-05T17:40:53.922 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2020-12-05T17:40:53.923 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2020-12-05T17:40:53.923 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:40:53.923 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:40:53.923 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:40:53.923 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:40:53.924 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:40:53.924 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:40:53.924 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:40:53.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:53.924 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "16", 2020-12-05T17:40:53.925 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:40:53.925 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:40:53.925 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:40:53.925 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:40:53.925 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:40:53.926 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:40:53.926 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:40:53.926 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:40:53.926 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:40:53.927 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:40:53.927 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:40:53.927 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:40:53.927 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:40:53.927 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:40:53.928 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:40:53.928 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:40:53.928 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:40:53.928 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:40:53.928 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:40:53.929 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:40:53.929 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:40:53.929 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:40:53.929 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:40:53.929 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:40:53.930 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:40:53.930 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:40:53.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:53.930 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:40:53.931 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:40:53.931 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:40:53.931 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:40:53.931 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:40:53.931 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:40:53.932 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:40:53.932 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:40:53.932 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:40:53.932 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:40:53.932 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:40:53.933 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:40:53.933 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2020-12-05T17:40:53.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:53.933 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:40:53.933 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2020-12-05T17:40:53.934 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:53.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:53.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:53.934 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2020-12-05T17:40:53.934 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2020-12-05T17:40:53.935 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:33.305595+0000", 2020-12-05T17:40:53.935 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:40:53.935 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:40:53.935 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:40:53.936 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:40:53.936 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:40:53.936 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:40:53.936 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:40:53.936 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:40:53.937 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:40:53.937 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:40:53.937 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:40:53.937 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:40:53.937 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:40:53.937 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:40:53.938 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:40:53.938 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:40:53.938 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:40:53.938 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:40:53.938 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:40:53.939 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:40:53.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:53.939 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "19", 2020-12-05T17:40:53.939 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:40:53.940 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:40:53.940 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:40:53.940 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:40:53.940 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:40:53.940 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:40:53.941 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:40:53.941 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:40:53.941 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:40:53.941 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:40:53.941 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:40:53.942 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:40:53.942 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:40:53.942 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:40:53.942 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:40:53.942 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:40:53.943 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:40:53.943 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:40:53.943 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:40:53.943 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:40:53.943 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:40:53.944 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:40:53.944 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:40:53.944 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:40:53.944 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:40:53.944 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:40:53.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:53.945 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:40:53.945 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:40:53.945 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:40:53.945 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:40:53.946 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:40:53.946 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:40:53.946 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:40:53.946 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:40:53.947 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:40:53.947 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:40:53.947 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:40:53.947 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:40:53.947 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:40:53.948 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2020-12-05T17:40:53.948 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:53.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:53.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:53.948 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 22, 2020-12-05T17:40:53.949 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2020-12-05T17:40:53.949 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:40:11.768907+0000", 2020-12-05T17:40:53.949 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:40:53.949 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:40:53.950 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:40:53.950 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:40:53.950 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:40:53.950 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:40:53.953 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:40:53.954 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:40:53.954 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:40:53.955 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:40:53.955 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:40:53.955 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:40:53.955 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:40:53.956 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:40:53.956 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:40:53.956 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:40:53.956 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:40:53.957 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:40:53.957 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:40:53.957 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:40:53.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:53.957 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "121", 2020-12-05T17:40:53.958 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:40:53.958 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:40:53.958 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:40:53.958 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:40:53.958 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:40:53.959 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:40:53.959 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:40:53.959 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:40:53.959 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:40:53.959 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:40:53.960 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:40:53.960 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:40:53.960 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:40:53.960 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:40:53.961 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:40:53.961 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:40:53.961 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:40:53.961 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:40:53.961 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:40:53.962 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:40:53.962 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:40:53.962 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:40:53.962 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:40:53.962 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:40:53.963 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:40:53.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:40:53.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:53.963 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:40:53.963 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:40:53.964 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:40:53.964 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:40:53.964 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:40:53.964 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:40:53.965 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:40:53.965 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:40:53.965 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:40:53.965 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:40:53.965 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:40:53.966 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:40:53.966 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:40:53.966 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:40:53.966 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2020-12-05T17:40:53.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:53.967 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:53.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:53.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:53.967 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 24, 2020-12-05T17:40:53.967 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2020-12-05T17:40:53.968 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:40:49.524656+0000", 2020-12-05T17:40:53.968 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:40:53.968 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:40:53.968 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:40:53.969 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:40:53.969 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:40:53.969 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:40:53.969 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:40:53.969 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:40:53.970 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:40:53.970 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:40:53.970 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:40:53.970 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:40:53.970 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:40:53.971 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:40:53.971 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:40:53.971 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:40:53.971 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:40:53.972 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:40:53.972 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:40:53.972 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:40:53.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:53.972 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "122", 2020-12-05T17:40:53.973 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:40:53.973 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:40:53.973 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:40:53.973 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:40:53.973 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:40:53.974 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:40:53.974 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:40:53.974 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:40:53.974 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:40:53.974 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:40:53.975 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:40:53.975 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:40:53.975 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:40:53.975 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:40:53.975 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:40:53.976 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:40:53.976 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:40:53.976 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:40:53.976 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:40:53.976 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:40:53.977 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:40:53.977 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:40:53.977 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:40:53.977 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:40:53.977 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:40:53.978 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:40:53.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:53.978 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:40:53.978 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:40:53.979 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:40:53.979 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:40:53.979 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:40:53.979 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:40:53.979 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:40:53.980 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:40:53.980 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:40:53.980 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:40:53.980 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:40:53.980 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:40:53.981 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2020-12-05T17:40:53.981 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2020-12-05T17:40:53.981 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2020-12-05T17:40:53.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:53.981 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:40:53.982 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:40:53.982 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2020-12-05T17:40:53.982 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:53.982 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:53.982 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:53.983 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:40:53.983 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2020-12-05T17:40:53.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:53.983 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:40:53.983 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "b941127b-0a49-4e06-8d11-17794ff8b013", 2020-12-05T17:40:53.984 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:40:53.984 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:40:53.984 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:40:53.984 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:40:53.985 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:40:53.985 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:40:53.985 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:40:53.985 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 120, 2020-12-05T17:40:53.985 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:40:53.986 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:40:53.986 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:40:53.986 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:53.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:53.986 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:53.987 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2020-12-05T17:40:53.987 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:40:53.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:53.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:53.988 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:53.988 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2020-12-05T17:40:53.988 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:40:53.988 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:53.988 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:53.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:53.989 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:40:53.989 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:53.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:53.989 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:53.990 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2020-12-05T17:40:53.990 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:40:53.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:53.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:53.990 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:53.991 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2020-12-05T17:40:53.991 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:40:53.991 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:53.991 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:53.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:53.992 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:40:53.992 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:53.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:53.992 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:53.992 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2020-12-05T17:40:53.993 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:40:53.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:53.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:53.993 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:53.993 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2020-12-05T17:40:53.994 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:40:53.994 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:53.994 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:53.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:53.995 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:40:53.995 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:53.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:53.995 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:53.995 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2020-12-05T17:40:53.996 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:40:53.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:53.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:53.996 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:53.996 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2020-12-05T17:40:53.997 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:40:53.997 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:53.997 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:53.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:53.998 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6803/11526", 2020-12-05T17:40:53.998 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6805/11526", 2020-12-05T17:40:53.998 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6809/11526", 2020-12-05T17:40:53.998 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6807/11526", 2020-12-05T17:40:53.998 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:40:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:40:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:40:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.000 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:40:54.000 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "01bd7ff5-a86b-4e11-ba1e-484c309c0fba", 2020-12-05T17:40:54.000 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:40:54.000 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:40:54.000 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:40:54.001 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:40:54.001 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:40:54.001 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:40:54.001 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:40:54.002 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 120, 2020-12-05T17:40:54.002 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:40:54.002 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:40:54.002 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:40:54.002 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.003 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.003 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2020-12-05T17:40:54.003 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:40:54.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.004 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.004 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2020-12-05T17:40:54.004 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:40:54.005 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.005 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.005 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:40:54.005 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.006 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.006 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2020-12-05T17:40:54.006 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:40:54.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2020-12-05T17:40:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:40:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:40:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.009 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.009 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2020-12-05T17:40:54.009 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:40:54.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.010 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.010 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2020-12-05T17:40:54.010 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:40:54.010 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.011 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.011 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:40:54.011 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.012 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.012 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2020-12-05T17:40:54.012 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:40:54.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.013 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.013 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2020-12-05T17:40:54.013 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:40:54.013 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.014 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.014 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6819/11538", 2020-12-05T17:40:54.014 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6821/11538", 2020-12-05T17:40:54.014 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6825/11538", 2020-12-05T17:40:54.015 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6823/11538", 2020-12-05T17:40:54.015 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:40:54.015 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:40:54.015 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:40:54.015 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.016 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:40:54.016 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "94736ef5-64c2-4a97-b963-bd38cddd9c52", 2020-12-05T17:40:54.016 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:40:54.017 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:40:54.017 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:40:54.017 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:40:54.017 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:40:54.017 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:40:54.018 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:40:54.018 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 115, 2020-12-05T17:40:54.018 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:40:54.018 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:40:54.018 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:40:54.019 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.019 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.019 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2020-12-05T17:40:54.019 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:40:54.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.020 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.020 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2020-12-05T17:40:54.020 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:40:54.021 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.021 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.021 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:40:54.022 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.022 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.022 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2020-12-05T17:40:54.022 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:40:54.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.023 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.023 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2020-12-05T17:40:54.023 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:40:54.024 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.024 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.024 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:40:54.024 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.025 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.025 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2020-12-05T17:40:54.025 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:40:54.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.026 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.026 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2020-12-05T17:40:54.026 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:40:54.027 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.027 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.027 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:40:54.027 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.028 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.028 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2020-12-05T17:40:54.028 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:40:54.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.029 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.029 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2020-12-05T17:40:54.029 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:40:54.030 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.030 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.030 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6811/11521", 2020-12-05T17:40:54.031 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6813/11521", 2020-12-05T17:40:54.031 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6817/11521", 2020-12-05T17:40:54.031 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6815/11521", 2020-12-05T17:40:54.031 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:40:54.031 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:40:54.032 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:40:54.032 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.032 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:40:54.033 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "69a1f3ff-fd25-43df-b60c-e32a89c35234", 2020-12-05T17:40:54.033 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:40:54.033 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:40:54.033 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:40:54.033 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:40:54.034 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:40:54.034 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:40:54.034 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:40:54.034 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 120, 2020-12-05T17:40:54.035 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:40:54.035 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:40:54.035 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:40:54.035 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.036 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.036 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2020-12-05T17:40:54.036 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:40:54.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.037 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.037 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2020-12-05T17:40:54.037 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:40:54.037 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.037 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.038 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:40:54.038 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.039 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.039 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2020-12-05T17:40:54.039 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:40:54.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.040 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.040 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2020-12-05T17:40:54.040 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:40:54.040 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.040 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.041 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:40:54.041 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.042 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.042 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6832", 2020-12-05T17:40:54.042 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:40:54.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.043 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.043 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6833", 2020-12-05T17:40:54.043 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:40:54.043 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.043 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.044 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:40:54.044 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.044 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.045 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2020-12-05T17:40:54.045 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:40:54.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.045 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.046 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2020-12-05T17:40:54.046 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:40:54.046 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.046 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.047 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6827/11535", 2020-12-05T17:40:54.047 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6829/11535", 2020-12-05T17:40:54.047 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6833/11535", 2020-12-05T17:40:54.047 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6831/11535", 2020-12-05T17:40:54.047 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:40:54.048 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:40:54.048 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:40:54.048 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.049 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:40:54.049 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c301fe4e-5eab-402c-ae6f-db3e6a7181af", 2020-12-05T17:40:54.049 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:40:54.049 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:40:54.049 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:40:54.050 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:40:54.050 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:40:54.050 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:40:54.050 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:40:54.051 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 115, 2020-12-05T17:40:54.051 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:40:54.051 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:40:54.051 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:40:54.051 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.052 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.052 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6808", 2020-12-05T17:40:54.052 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:40:54.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.053 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.053 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6809", 2020-12-05T17:40:54.053 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:40:54.053 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.054 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.054 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:40:54.054 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.055 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.055 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6810", 2020-12-05T17:40:54.055 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:40:54.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.056 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.056 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6811", 2020-12-05T17:40:54.056 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:40:54.056 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.056 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.057 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:40:54.057 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.057 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.058 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6817", 2020-12-05T17:40:54.058 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:40:54.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.058 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.059 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6819", 2020-12-05T17:40:54.059 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:40:54.059 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.059 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.060 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:40:54.060 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.060 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.060 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6812", 2020-12-05T17:40:54.061 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:40:54.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.061 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.061 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6814", 2020-12-05T17:40:54.062 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:40:54.062 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.062 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.063 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6809/11541", 2020-12-05T17:40:54.063 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6811/11541", 2020-12-05T17:40:54.063 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6819/11541", 2020-12-05T17:40:54.063 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6814/11541", 2020-12-05T17:40:54.063 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:40:54.064 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:40:54.064 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:40:54.064 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.064 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:40:54.065 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "9476eff4-345b-46c5-a473-1d9a89a6d327", 2020-12-05T17:40:54.065 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:40:54.065 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:40:54.065 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:40:54.066 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:40:54.066 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:40:54.066 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:40:54.066 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:40:54.066 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 120, 2020-12-05T17:40:54.067 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:40:54.067 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:40:54.067 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:40:54.067 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.068 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.068 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6800", 2020-12-05T17:40:54.068 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:40:54.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.069 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.069 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6801", 2020-12-05T17:40:54.069 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:40:54.069 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.069 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.070 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:40:54.070 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.070 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.071 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6802", 2020-12-05T17:40:54.071 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:40:54.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.071 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.072 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6803", 2020-12-05T17:40:54.072 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:40:54.072 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.072 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.073 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:40:54.073 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.073 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.073 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6806", 2020-12-05T17:40:54.074 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:40:54.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.074 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.074 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6807", 2020-12-05T17:40:54.075 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:40:54.075 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.075 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.075 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:40:54.076 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.076 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.076 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6804", 2020-12-05T17:40:54.076 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:40:54.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.077 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.077 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6805", 2020-12-05T17:40:54.077 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:40:54.078 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.078 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.078 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6801/11539", 2020-12-05T17:40:54.078 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6803/11539", 2020-12-05T17:40:54.079 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6807/11539", 2020-12-05T17:40:54.079 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6805/11539", 2020-12-05T17:40:54.079 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:40:54.079 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:40:54.080 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:40:54.080 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.080 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:40:54.081 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8dfd8567-9b1c-4d13-9c96-071183e469c2", 2020-12-05T17:40:54.081 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:40:54.081 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:40:54.081 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:40:54.081 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:40:54.081 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:40:54.082 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:40:54.082 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:40:54.082 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 120, 2020-12-05T17:40:54.082 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:40:54.083 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:40:54.083 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:40:54.083 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.083 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.084 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6820", 2020-12-05T17:40:54.084 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:40:54.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.084 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.085 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6822", 2020-12-05T17:40:54.085 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:40:54.085 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.085 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.086 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:40:54.086 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.086 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.086 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6823", 2020-12-05T17:40:54.087 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:40:54.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.087 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.087 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6825", 2020-12-05T17:40:54.088 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:40:54.088 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.088 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.088 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:40:54.089 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.089 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.089 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6829", 2020-12-05T17:40:54.089 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:40:54.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.090 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.090 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6830", 2020-12-05T17:40:54.091 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:40:54.091 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.091 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.091 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:40:54.092 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.092 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.092 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6826", 2020-12-05T17:40:54.092 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:40:54.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.093 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.093 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6827", 2020-12-05T17:40:54.093 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:40:54.094 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.094 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.094 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6822/11542", 2020-12-05T17:40:54.094 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6825/11542", 2020-12-05T17:40:54.095 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6830/11542", 2020-12-05T17:40:54.095 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6827/11542", 2020-12-05T17:40:54.095 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:40:54.095 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:40:54.095 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:40:54.096 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.096 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:40:54.096 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "24d466b3-01dc-4ba2-8b81-85250d163bd2", 2020-12-05T17:40:54.097 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:40:54.097 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:40:54.097 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:40:54.097 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:40:54.097 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:40:54.098 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:40:54.098 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:40:54.098 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 120, 2020-12-05T17:40:54.098 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:40:54.098 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:40:54.099 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:40:54.099 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.099 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.099 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6813", 2020-12-05T17:40:54.100 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:40:54.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.100 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.100 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6815", 2020-12-05T17:40:54.101 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:40:54.101 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.101 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.101 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:40:54.102 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.102 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.102 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6816", 2020-12-05T17:40:54.102 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:40:54.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.103 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.103 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6818", 2020-12-05T17:40:54.103 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:40:54.104 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.104 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.104 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:40:54.104 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.105 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.105 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6828", 2020-12-05T17:40:54.105 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:40:54.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.106 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.106 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6831", 2020-12-05T17:40:54.106 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:40:54.106 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.107 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.107 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:40:54.107 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:40:54.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.108 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:40:54.108 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6821", 2020-12-05T17:40:54.108 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:40:54.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.109 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:40:54.109 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6824", 2020-12-05T17:40:54.109 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:40:54.109 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.109 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.110 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6815/11540", 2020-12-05T17:40:54.110 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6818/11540", 2020-12-05T17:40:54.110 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6831/11540", 2020-12-05T17:40:54.110 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6824/11540", 2020-12-05T17:40:54.111 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:40:54.111 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:40:54.111 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:40:54.111 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:54.111 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.112 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:40:54.112 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2020-12-05T17:40:54.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.112 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:40:54.112 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:40:54.113 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:40:54.113 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:40:54.113 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:40:54.113 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:40:54.113 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.398868+0000", 2020-12-05T17:40:54.114 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:40:54.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.114 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:40:54.114 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:40:54.115 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:40:54.115 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:40:54.115 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:40:54.115 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:40:54.115 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.675748+0000", 2020-12-05T17:40:54.116 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:40:54.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.116 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:40:54.116 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:40:54.117 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:40:54.117 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:40:54.117 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:40:54.117 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:40:54.117 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.620888+0000", 2020-12-05T17:40:54.118 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:40:54.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.118 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:40:54.118 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:40:54.119 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:40:54.119 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:40:54.119 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:40:54.119 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:40:54.119 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.681600+0000", 2020-12-05T17:40:54.120 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:40:54.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.120 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:40:54.120 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:40:54.121 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:40:54.121 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:40:54.121 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:40:54.121 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:40:54.122 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.658769+0000", 2020-12-05T17:40:54.122 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:40:54.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.122 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:40:54.123 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:40:54.123 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:40:54.123 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:40:54.123 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:40:54.123 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:40:54.124 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.630708+0000", 2020-12-05T17:40:54.124 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:40:54.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.124 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:40:54.125 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:40:54.125 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:40:54.125 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:40:54.125 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:40:54.125 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:40:54.126 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.648352+0000", 2020-12-05T17:40:54.126 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:40:54.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:54.126 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:40:54.127 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:40:54.127 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:40:54.127 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:40:54.127 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:40:54.127 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:40:54.128 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.629399+0000", 2020-12-05T17:40:54.128 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:40:54.128 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.128 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:40:54.128 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2020-12-05T17:40:54.129 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2020-12-05T17:40:54.129 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2020-12-05T17:40:54.129 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2020-12-05T17:40:54.129 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": { 2020-12-05T17:40:54.129 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6835/2641840537": "2020-12-06T17:40:30.230142+0000", 2020-12-05T17:40:54.130 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6834/2641840537": "2020-12-06T17:40:30.230142+0000" 2020-12-05T17:40:54.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.130 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2020-12-05T17:40:54.130 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2020-12-05T17:40:54.131 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2020-12-05T17:40:54.131 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:40:54.131 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2020-12-05T17:40:54.131 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2020-12-05T17:40:54.131 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:40:54.132 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2020-12-05T17:40:54.132 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:40:54.132 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:40:54.132 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2020-12-05T17:40:54.132 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2020-12-05T17:40:54.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.133 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2020-12-05T17:40:54.133 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:40:54.133 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:40:54.133 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2020-12-05T17:40:54.134 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:40:54.134 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:40:54.134 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2020-12-05T17:40:54.134 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:54.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:54.135 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2020-12-05T17:40:54.135 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2020-12-05T17:40:54.135 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2020-12-05T17:40:54.135 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2020-12-05T17:40:54.135 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2020-12-05T17:40:54.136 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:40:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.911+0000 7fb8fcf64700 1 -- 172.21.15.32:0/3753123337 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8d80553b0 msgr2=0x7fb8d8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.911+0000 7fb8fcf64700 1 --2- 172.21.15.32:0/3753123337 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8d80553b0 0x7fb8d8057870 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7fb8e800b000 tx=0x7fb8e80091a0).stop 2020-12-05T17:40:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.911+0000 7fb8fcf64700 1 -- 172.21.15.32:0/3753123337 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8f80b52f0 msgr2=0x7fb8f8151660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.911+0000 7fb8fcf64700 1 --2- 172.21.15.32:0/3753123337 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8f80b52f0 0x7fb8f8151660 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fb8f80adc80 tx=0x7fb8ec015000).stop 2020-12-05T17:40:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.911+0000 7fb8fcf64700 1 -- 172.21.15.32:0/3753123337 shutdown_connections 2020-12-05T17:40:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.911+0000 7fb8fcf64700 1 --2- 172.21.15.32:0/3753123337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8f8138d00 0x7fb8f8151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.911+0000 7fb8fcf64700 1 --2- 172.21.15.32:0/3753123337 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb8f80b1870 0x7fb8f813f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.911+0000 7fb8fcf64700 1 --2- 172.21.15.32:0/3753123337 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8d80553b0 0x7fb8d8057870 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.911+0000 7fb8fcf64700 1 --2- 172.21.15.32:0/3753123337 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8f80b52f0 0x7fb8f8151660 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.911+0000 7fb8fcf64700 1 -- 172.21.15.32:0/3753123337 >> 172.21.15.32:0/3753123337 conn(0x7fb8f805bbb0 msgr2=0x7fb8f80af000 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.911+0000 7fb8fcf64700 1 -- 172.21.15.32:0/3753123337 shutdown_connections 2020-12-05T17:40:54.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:53.911+0000 7fb8fcf64700 1 -- 172.21.15.32:0/3753123337 wait complete. 2020-12-05T17:40:54.141 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:40:54.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.263+0000 7fa643262700 1 Processor -- start 2020-12-05T17:40:54.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.263+0000 7fa643262700 1 -- start start 2020-12-05T17:40:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.263+0000 7fa643262700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa63c0b1870 0x7fa63c0b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.263+0000 7fa643262700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa63c0ae860 0x7fa63c0aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.263+0000 7fa643262700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa63c1420f0 con 0x7fa63c0b52f0 2020-12-05T17:40:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.263+0000 7fa643262700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63c0ad4f0 con 0x7fa63c0b1870 2020-12-05T17:40:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.263+0000 7fa643262700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63c0b0380 con 0x7fa63c0ae860 2020-12-05T17:40:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.263+0000 7fa641a5f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa63c0b1870 0x7fa63c0b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.263+0000 7fa641a5f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa63c0b1870 0x7fa63c0b56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39926/0 (socket says 172.21.15.32:39926) 2020-12-05T17:40:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.263+0000 7fa642260700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa63c0b52f0 0x7fa63c138810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42572/0 (socket says 172.21.15.32:42572) 2020-12-05T17:40:54.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.263+0000 7fa641a5f700 1 -- 172.21.15.32:0/3894002253 learned_addr learned my addr 172.21.15.32:0/3894002253 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:54.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.263+0000 7fa642a61700 1 --2- 172.21.15.32:0/3894002253 >> v2:172.21.15.61:3300/0 conn(0x7fa63c0ae860 0x7fa63c0aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:54.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.263+0000 7fa64125e700 1 -- 172.21.15.32:0/3894002253 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1860735430 0 0) 0x7fa63c1420f0 con 0x7fa63c0b52f0 2020-12-05T17:40:54.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.263+0000 7fa64125e700 1 -- 172.21.15.32:0/3894002253 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa628003230 con 0x7fa63c0b52f0 2020-12-05T17:40:54.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa641a5f700 1 -- 172.21.15.32:0/3894002253 >> v2:172.21.15.61:3300/0 conn(0x7fa63c0ae860 msgr2=0x7fa63c0aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:54.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa641a5f700 1 --2- 172.21.15.32:0/3894002253 >> v2:172.21.15.61:3300/0 conn(0x7fa63c0ae860 0x7fa63c0aecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:54.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa641a5f700 1 -- 172.21.15.32:0/3894002253 >> v1:172.21.15.32:6789/0 conn(0x7fa63c0b52f0 legacy=0x7fa63c138810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:54.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa64125e700 1 -- 172.21.15.32:0/3894002253 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 611880192 0 0) 0x7fa628003230 con 0x7fa63c0b52f0 2020-12-05T17:40:54.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa641a5f700 1 -- 172.21.15.32:0/3894002253 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c0b0620 con 0x7fa63c0b1870 2020-12-05T17:40:54.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa641a5f700 1 --2- 172.21.15.32:0/3894002253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0b1870 0x7fa63c0b56d0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7fa62c001870 tx=0x7fa630009270).ready entity=mon.0 client_cookie=e9d7bbf87d904155 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:54.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa64125e700 1 -- 172.21.15.32:0/3894002253 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa62c0016d0 con 0x7fa63c0b1870 2020-12-05T17:40:54.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa64125e700 1 -- 172.21.15.32:0/3894002253 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa630009e60 con 0x7fa63c0b1870 2020-12-05T17:40:54.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa64125e700 1 -- 172.21.15.32:0/3894002253 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa63000ba10 con 0x7fa63c0b1870 2020-12-05T17:40:54.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa643262700 1 -- 172.21.15.32:0/3894002253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0b1870 msgr2=0x7fa63c0b56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:54.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa643262700 1 --2- 172.21.15.32:0/3894002253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0b1870 0x7fa63c0b56d0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7fa62c001870 tx=0x7fa630009270).stop 2020-12-05T17:40:54.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa643262700 1 -- 172.21.15.32:0/3894002253 shutdown_connections 2020-12-05T17:40:54.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa643262700 1 --2- 172.21.15.32:0/3894002253 >> v2:172.21.15.61:3300/0 conn(0x7fa63c0ae860 0x7fa63c0aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:54.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa643262700 1 --2- 172.21.15.32:0/3894002253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0b1870 0x7fa63c0b56d0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:54.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa643262700 1 -- 172.21.15.32:0/3894002253 >> 172.21.15.32:0/3894002253 conn(0x7fa63c05bbb0 msgr2=0x7fa63c062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:54.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa643262700 1 -- 172.21.15.32:0/3894002253 shutdown_connections 2020-12-05T17:40:54.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa643262700 1 -- 172.21.15.32:0/3894002253 wait complete. 2020-12-05T17:40:54.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa643262700 1 Processor -- start 2020-12-05T17:40:54.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa643262700 1 -- start start 2020-12-05T17:40:54.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa643262700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa63c0ae860 0x7fa63c141c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:54.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa643262700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0b1870 0x7fa63c12ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:54.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa643262700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa63c0b52f0 0x7fa63c1351a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:54.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa643262700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c135690 con 0x7fa63c0b1870 2020-12-05T17:40:54.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa643262700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c133700 con 0x7fa63c0b52f0 2020-12-05T17:40:54.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.267+0000 7fa643262700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c1331c0 con 0x7fa63c0ae860 2020-12-05T17:40:54.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.271+0000 7fa641a5f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0b1870 0x7fa63c12ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:54.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.271+0000 7fa641a5f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0b1870 0x7fa63c12ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39932/0 (socket says 172.21.15.32:39932) 2020-12-05T17:40:54.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.271+0000 7fa641a5f700 1 -- 172.21.15.32:0/1587923136 learned_addr learned my addr 172.21.15.32:0/1587923136 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:54.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.271+0000 7fa642260700 1 --2- 172.21.15.32:0/1587923136 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa63c0ae860 0x7fa63c141c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:54.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.271+0000 7fa642a61700 1 --2- 172.21.15.32:0/1587923136 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa63c0b52f0 0x7fa63c1351a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:54.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.271+0000 7fa641a5f700 1 -- 172.21.15.32:0/1587923136 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa63c0ae860 msgr2=0x7fa63c141c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:54.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.271+0000 7fa641a5f700 1 --2- 172.21.15.32:0/1587923136 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa63c0ae860 0x7fa63c141c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:54.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.271+0000 7fa641a5f700 1 -- 172.21.15.32:0/1587923136 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa63c0b52f0 msgr2=0x7fa63c1351a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:54.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.271+0000 7fa641a5f700 1 --2- 172.21.15.32:0/1587923136 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa63c0b52f0 0x7fa63c1351a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:54.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.271+0000 7fa641a5f700 1 -- 172.21.15.32:0/1587923136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c133460 con 0x7fa63c0b1870 2020-12-05T17:40:54.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.271+0000 7fa642a61700 1 --2- 172.21.15.32:0/1587923136 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa63c0b52f0 0x7fa63c1351a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:40:54.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.271+0000 7fa641a5f700 1 --2- 172.21.15.32:0/1587923136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0b1870 0x7fa63c12ed20 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 rx=0x7fa63c1357d0 tx=0x7fa630016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:54.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.271+0000 7fa63b7fe700 1 -- 172.21.15.32:0/1587923136 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa63000bed0 con 0x7fa63c0b1870 2020-12-05T17:40:54.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.271+0000 7fa63b7fe700 1 -- 172.21.15.32:0/1587923136 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa630003270 con 0x7fa63c0b1870 2020-12-05T17:40:54.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.271+0000 7fa643262700 1 -- 172.21.15.32:0/1587923136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63c130a50 con 0x7fa63c0b1870 2020-12-05T17:40:54.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.271+0000 7fa63b7fe700 1 -- 172.21.15.32:0/1587923136 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6300107a0 con 0x7fa63c0b1870 2020-12-05T17:40:54.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.271+0000 7fa643262700 1 -- 172.21.15.32:0/1587923136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63c131020 con 0x7fa63c0b1870 2020-12-05T17:40:54.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.271+0000 7fa643262700 1 -- 172.21.15.32:0/1587923136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa63c1419a0 con 0x7fa63c0b1870 2020-12-05T17:40:54.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.271+0000 7fa63b7fe700 1 -- 172.21.15.32:0/1587923136 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa63000f070 con 0x7fa63c0b1870 2020-12-05T17:40:54.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.275+0000 7fa63b7fe700 1 --2- 172.21.15.32:0/1587923136 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa61c055440 0x7fa61c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:54.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.275+0000 7fa63b7fe700 1 -- 172.21.15.32:0/1587923136 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fa63006f4d0 con 0x7fa63c0b1870 2020-12-05T17:40:54.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.275+0000 7fa642260700 1 --2- 172.21.15.32:0/1587923136 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa61c055440 0x7fa61c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:54.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.275+0000 7fa642260700 1 --2- 172.21.15.32:0/1587923136 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa61c055440 0x7fa61c057900 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7fa62c00b000 tx=0x7fa62c001340).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:54.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.279+0000 7fa63b7fe700 1 -- 172.21.15.32:0/1587923136 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa63c1419a0 con 0x7fa63c0b1870 2020-12-05T17:40:54.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.471+0000 7fa643262700 1 -- 172.21.15.32:0/1587923136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa63c1419a0 con 0x7fa63c0b1870 2020-12-05T17:40:54.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.471+0000 7fa63b7fe700 1 -- 172.21.15.32:0/1587923136 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+1426 (secure 0 0 0) 0x7fa6300155b0 con 0x7fa63c0b1870 2020-12-05T17:40:54.475 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:40:54.475 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":57,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:40:50.514905+0000","modified":"2020-12-05T17:40:50.515018+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[22],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2020-12-05T17:40:54.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.471+0000 7fa643262700 1 -- 172.21.15.32:0/1587923136 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa61c055440 msgr2=0x7fa61c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:54.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.471+0000 7fa643262700 1 --2- 172.21.15.32:0/1587923136 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa61c055440 0x7fa61c057900 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7fa62c00b000 tx=0x7fa62c001340).stop 2020-12-05T17:40:54.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.471+0000 7fa643262700 1 -- 172.21.15.32:0/1587923136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0b1870 msgr2=0x7fa63c12ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:54.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.471+0000 7fa643262700 1 --2- 172.21.15.32:0/1587923136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0b1870 0x7fa63c12ed20 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 rx=0x7fa63c1357d0 tx=0x7fa630016000).stop 2020-12-05T17:40:54.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.471+0000 7fa643262700 1 -- 172.21.15.32:0/1587923136 shutdown_connections 2020-12-05T17:40:54.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.471+0000 7fa643262700 1 --2- 172.21.15.32:0/1587923136 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa63c0b52f0 0x7fa63c1351a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:54.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.471+0000 7fa643262700 1 --2- 172.21.15.32:0/1587923136 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa61c055440 0x7fa61c057900 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:54.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.471+0000 7fa643262700 1 --2- 172.21.15.32:0/1587923136 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa63c0ae860 0x7fa63c141c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:54.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.471+0000 7fa643262700 1 --2- 172.21.15.32:0/1587923136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0b1870 0x7fa63c12ed20 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:54.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.471+0000 7fa643262700 1 -- 172.21.15.32:0/1587923136 >> 172.21.15.32:0/1587923136 conn(0x7fa63c05bbb0 msgr2=0x7fa63c0b3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:54.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.471+0000 7fa643262700 1 -- 172.21.15.32:0/1587923136 shutdown_connections 2020-12-05T17:40:54.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.471+0000 7fa643262700 1 -- 172.21.15.32:0/1587923136 wait complete. 2020-12-05T17:40:54.479 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 57 2020-12-05T17:40:54.489 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-12-05T17:40:54.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.599+0000 7fd58f59e700 1 Processor -- start 2020-12-05T17:40:54.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.599+0000 7fd58f59e700 1 -- start start 2020-12-05T17:40:54.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.603+0000 7fd58f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd590062400 0x7fd59005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:54.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.603+0000 7fd58f59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd590142450 con 0x7fd59005feb0 2020-12-05T17:40:54.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.603+0000 7fd58f59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd590061780 con 0x7fd59005f3b0 2020-12-05T17:40:54.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.603+0000 7fd58f59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd590061a20 con 0x7fd590062400 2020-12-05T17:40:54.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.603+0000 7fd58dd9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd590062400 0x7fd59005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:54.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.603+0000 7fd58dd9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd590062400 0x7fd59005f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39940/0 (socket says 172.21.15.32:39940) 2020-12-05T17:40:54.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.603+0000 7fd58dd9b700 1 -- 172.21.15.32:0/2353845930 learned_addr learned my addr 172.21.15.32:0/2353845930 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:54.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.603+0000 7fd58ed9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd59005feb0 0x7fd5900b0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42590/0 (socket says 172.21.15.32:42590) 2020-12-05T17:40:54.609 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.603+0000 7fd58dd9b700 1 -- 172.21.15.32:0/2353845930 >> v1:172.21.15.61:6789/0 conn(0x7fd59005f3b0 legacy=0x7fd5900b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:54.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.603+0000 7fd58dd9b700 1 -- 172.21.15.32:0/2353845930 >> v1:172.21.15.32:6789/0 conn(0x7fd59005feb0 legacy=0x7fd5900b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:54.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.603+0000 7fd58dd9b700 1 -- 172.21.15.32:0/2353845930 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd590061cc0 con 0x7fd590062400 2020-12-05T17:40:54.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.603+0000 7fd58dd9b700 1 --2- 172.21.15.32:0/2353845930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd590062400 0x7fd59005f790 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7fd580002f30 tx=0x7fd580006e80).ready entity=mon.0 client_cookie=79d21c9ebb63682a server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:54.610 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.603+0000 7fd58d59a700 1 -- 172.21.15.32:0/2353845930 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5800097f0 con 0x7fd590062400 2020-12-05T17:40:54.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.603+0000 7fd58d59a700 1 -- 172.21.15.32:0/2353845930 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd580009950 con 0x7fd590062400 2020-12-05T17:40:54.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.603+0000 7fd58d59a700 1 -- 172.21.15.32:0/2353845930 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd580009cb0 con 0x7fd590062400 2020-12-05T17:40:54.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.603+0000 7fd58f59e700 1 -- 172.21.15.32:0/2353845930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd590062400 msgr2=0x7fd59005f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:54.611 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.603+0000 7fd58f59e700 1 --2- 172.21.15.32:0/2353845930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd590062400 0x7fd59005f790 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7fd580002f30 tx=0x7fd580006e80).stop 2020-12-05T17:40:54.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.603+0000 7fd58f59e700 1 -- 172.21.15.32:0/2353845930 shutdown_connections 2020-12-05T17:40:54.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.603+0000 7fd58f59e700 1 --2- 172.21.15.32:0/2353845930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd590062400 0x7fd59005f790 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:54.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.603+0000 7fd58f59e700 1 -- 172.21.15.32:0/2353845930 >> 172.21.15.32:0/2353845930 conn(0x7fd5900b8ee0 msgr2=0x7fd5900b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:54.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.603+0000 7fd58f59e700 1 -- 172.21.15.32:0/2353845930 shutdown_connections 2020-12-05T17:40:54.612 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.603+0000 7fd58f59e700 1 -- 172.21.15.32:0/2353845930 wait complete. 2020-12-05T17:40:54.614 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd58f59e700 1 Processor -- start 2020-12-05T17:40:54.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd58f59e700 1 -- start start 2020-12-05T17:40:54.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd58f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd59005f3b0 0x7fd590143dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:54.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd58f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd59005feb0 0x7fd59013df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:54.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd58f59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd590062400 0x7fd59013e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:54.615 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd58f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd590152590 con 0x7fd59005f3b0 2020-12-05T17:40:54.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd58f59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd590151bb0 con 0x7fd590062400 2020-12-05T17:40:54.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd58f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd590151e50 con 0x7fd59005feb0 2020-12-05T17:40:54.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd58e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd59005f3b0 0x7fd590143dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:54.616 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd58e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd59005f3b0 0x7fd590143dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39944/0 (socket says 172.21.15.32:39944) 2020-12-05T17:40:54.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd58e59c700 1 -- 172.21.15.32:0/146676711 learned_addr learned my addr 172.21.15.32:0/146676711 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:54.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd58ed9d700 1 --2- 172.21.15.32:0/146676711 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd590062400 0x7fd59013e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:54.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd58dd9b700 1 --2- 172.21.15.32:0/146676711 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd59005feb0 0x7fd59013df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:54.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd58e59c700 1 -- 172.21.15.32:0/146676711 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd59005feb0 msgr2=0x7fd59013df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:54.617 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd58e59c700 1 --2- 172.21.15.32:0/146676711 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd59005feb0 0x7fd59013df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:54.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd58e59c700 1 -- 172.21.15.32:0/146676711 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd590062400 msgr2=0x7fd59013e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:54.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd58e59c700 1 --2- 172.21.15.32:0/146676711 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd590062400 0x7fd59013e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:54.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd58e59c700 1 -- 172.21.15.32:0/146676711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5901520f0 con 0x7fd59005f3b0 2020-12-05T17:40:54.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd58ed9d700 1 --2- 172.21.15.32:0/146676711 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd590062400 0x7fd59013e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:40:54.618 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd58e59c700 1 --2- 172.21.15.32:0/146676711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd59005f3b0 0x7fd590143dc0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7fd59005c5c0 tx=0x7fd578007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:54.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd5877fe700 1 -- 172.21.15.32:0/146676711 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd57800e4a0 con 0x7fd59005f3b0 2020-12-05T17:40:54.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd5877fe700 1 -- 172.21.15.32:0/146676711 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57800e600 con 0x7fd59005f3b0 2020-12-05T17:40:54.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd58f59e700 1 -- 172.21.15.32:0/146676711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5900af8c0 con 0x7fd59005f3b0 2020-12-05T17:40:54.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd5877fe700 1 -- 172.21.15.32:0/146676711 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd57800e960 con 0x7fd59005f3b0 2020-12-05T17:40:54.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.607+0000 7fd58f59e700 1 -- 172.21.15.32:0/146676711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5901516a0 con 0x7fd59005f3b0 2020-12-05T17:40:54.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.611+0000 7fd58f59e700 1 -- 172.21.15.32:0/146676711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd590063d60 con 0x7fd59005f3b0 2020-12-05T17:40:54.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.611+0000 7fd5877fe700 1 -- 172.21.15.32:0/146676711 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd578019030 con 0x7fd59005f3b0 2020-12-05T17:40:54.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.611+0000 7fd5877fe700 1 --2- 172.21.15.32:0/146676711 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd5740554e0 0x7fd5740579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:54.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.611+0000 7fd5877fe700 1 -- 172.21.15.32:0/146676711 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fd57806d820 con 0x7fd59005f3b0 2020-12-05T17:40:54.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.611+0000 7fd58dd9b700 1 --2- 172.21.15.32:0/146676711 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd5740554e0 0x7fd5740579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:54.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.611+0000 7fd58dd9b700 1 --2- 172.21.15.32:0/146676711 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd5740554e0 0x7fd5740579a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7fd58001a000 tx=0x7fd580015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:54.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.615+0000 7fd5877fe700 1 -- 172.21.15.32:0/146676711 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd590063d60 con 0x7fd59005f3b0 2020-12-05T17:40:54.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:54.803+0000 7fd58f59e700 1 -- 172.21.15.32:0/146676711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd590063d60 con 0x7fd59005f3b0 2020-12-05T17:40:55.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.187+0000 7fd5877fe700 1 -- 172.21.15.32:0/146676711 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd590063d60 con 0x7fd59005f3b0 2020-12-05T17:40:55.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.191+0000 7fd58f59e700 1 -- 172.21.15.32:0/146676711 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd5740554e0 msgr2=0x7fd5740579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:55.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.191+0000 7fd58f59e700 1 --2- 172.21.15.32:0/146676711 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd5740554e0 0x7fd5740579a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7fd58001a000 tx=0x7fd580015000).stop 2020-12-05T17:40:55.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.191+0000 7fd58f59e700 1 -- 172.21.15.32:0/146676711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd59005f3b0 msgr2=0x7fd590143dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:55.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.191+0000 7fd58f59e700 1 --2- 172.21.15.32:0/146676711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd59005f3b0 0x7fd590143dc0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7fd59005c5c0 tx=0x7fd578007c20).stop 2020-12-05T17:40:55.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.191+0000 7fd58f59e700 1 -- 172.21.15.32:0/146676711 shutdown_connections 2020-12-05T17:40:55.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.191+0000 7fd58f59e700 1 --2- 172.21.15.32:0/146676711 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd590062400 0x7fd59013e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:55.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.191+0000 7fd58f59e700 1 --2- 172.21.15.32:0/146676711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd59005f3b0 0x7fd590143dc0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:55.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.191+0000 7fd58f59e700 1 --2- 172.21.15.32:0/146676711 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd5740554e0 0x7fd5740579a0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:55.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.191+0000 7fd58f59e700 1 --2- 172.21.15.32:0/146676711 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd59005feb0 0x7fd59013df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:55.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.191+0000 7fd58f59e700 1 -- 172.21.15.32:0/146676711 >> 172.21.15.32:0/146676711 conn(0x7fd5900b8ee0 msgr2=0x7fd5900b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:55.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.191+0000 7fd58f59e700 1 -- 172.21.15.32:0/146676711 shutdown_connections 2020-12-05T17:40:55.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.191+0000 7fd58f59e700 1 -- 172.21.15.32:0/146676711 wait complete. 2020-12-05T17:40:55.198 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-12-05T17:40:55.214 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:40:55.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.331+0000 7f201ad71700 1 Processor -- start 2020-12-05T17:40:55.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.331+0000 7f201ad71700 1 -- start start 2020-12-05T17:40:55.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.331+0000 7f201ad71700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f201405f3b0 0x7f201405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:55.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.331+0000 7f201ad71700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f2014062400 0x7f20140b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:55.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.331+0000 7f201ad71700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2014142450 con 0x7f201405feb0 2020-12-05T17:40:55.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.331+0000 7f201ad71700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20140b42f0 con 0x7f201405f3b0 2020-12-05T17:40:55.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.331+0000 7f201ad71700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2014061780 con 0x7f2014062400 2020-12-05T17:40:55.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.331+0000 7f201a570700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f201405feb0 0x7f20140b3be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42600/0 (socket says 172.21.15.32:42600) 2020-12-05T17:40:55.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.331+0000 7f201a570700 1 -- 172.21.15.32:0/389696676 learned_addr learned my addr 172.21.15.32:0/389696676 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:55.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.331+0000 7f2019d6f700 1 --2- 172.21.15.32:0/389696676 >> v2:172.21.15.32:3300/0 conn(0x7f201405f3b0 0x7f201405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:55.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.331+0000 7f201956e700 1 --2- 172.21.15.32:0/389696676 >> v2:172.21.15.61:3300/0 conn(0x7f2014062400 0x7f20140b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:55.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f2018d6d700 1 -- 172.21.15.32:0/389696676 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 447565748 0 0) 0x7f2014142450 con 0x7f201405feb0 2020-12-05T17:40:55.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f2018d6d700 1 -- 172.21.15.32:0/389696676 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2000003230 con 0x7f201405feb0 2020-12-05T17:40:55.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f2018d6d700 1 -- 172.21.15.32:0/389696676 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3187061641 0 0) 0x7f2000003230 con 0x7f201405feb0 2020-12-05T17:40:55.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f2018d6d700 1 -- 172.21.15.32:0/389696676 >> v2:172.21.15.61:3300/0 conn(0x7f2014062400 msgr2=0x7f20140b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:55.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f2018d6d700 1 --2- 172.21.15.32:0/389696676 >> v2:172.21.15.61:3300/0 conn(0x7f2014062400 0x7f20140b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:55.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f2018d6d700 1 -- 172.21.15.32:0/389696676 >> v2:172.21.15.32:3300/0 conn(0x7f201405f3b0 msgr2=0x7f201405f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:55.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f2018d6d700 1 --2- 172.21.15.32:0/389696676 >> v2:172.21.15.32:3300/0 conn(0x7f201405f3b0 0x7f201405f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:55.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f2018d6d700 1 -- 172.21.15.32:0/389696676 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2014061a20 con 0x7f201405feb0 2020-12-05T17:40:55.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f2018d6d700 1 -- 172.21.15.32:0/389696676 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f200c001a50 con 0x7f201405feb0 2020-12-05T17:40:55.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f2018d6d700 1 -- 172.21.15.32:0/389696676 >> v1:172.21.15.32:6789/0 conn(0x7f201405feb0 legacy=0x7f20140b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:55.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f2018d6d700 1 --2- 172.21.15.32:0/389696676 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2000003720 0x7f2000003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:55.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f2018d6d700 1 --2- 172.21.15.32:0/389696676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20000044d0 0x7f2000004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:55.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f2018d6d700 1 --2- 172.21.15.32:0/389696676 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2000004e40 0x7f200000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:55.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f2018d6d700 1 -- 172.21.15.32:0/389696676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20140b42f0 con 0x7f20000044d0 2020-12-05T17:40:55.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f2018d6d700 1 -- 172.21.15.32:0/389696676 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2014061780 con 0x7f2000004e40 2020-12-05T17:40:55.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f2018d6d700 1 -- 172.21.15.32:0/389696676 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2000004250 con 0x7f2000003720 2020-12-05T17:40:55.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f2019d6f700 1 --2- 172.21.15.32:0/389696676 >> v2:172.21.15.32:3300/0 conn(0x7f201405f3b0 0x7f201405f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:40:55.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f201956e700 1 --2- 172.21.15.32:0/389696676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20000044d0 0x7f2000004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:55.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f201a570700 1 --2- 172.21.15.32:0/389696676 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2000004e40 0x7f200000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:55.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f2019d6f700 1 --2- 172.21.15.32:0/389696676 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2000003720 0x7f2000003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:55.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f201956e700 1 -- 172.21.15.32:0/389696676 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2000003720 msgr2=0x7f2000003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:55.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f201956e700 1 --2- 172.21.15.32:0/389696676 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2000003720 0x7f2000003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:55.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f201956e700 1 -- 172.21.15.32:0/389696676 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2000004e40 msgr2=0x7f200000b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:55.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f201956e700 1 --2- 172.21.15.32:0/389696676 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2000004e40 0x7f200000b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:55.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f201956e700 1 -- 172.21.15.32:0/389696676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2000003230 con 0x7f20000044d0 2020-12-05T17:40:55.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f201a570700 1 --2- 172.21.15.32:0/389696676 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2000004e40 0x7f200000b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:40:55.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f201a570700 1 -- 172.21.15.32:0/389696676 reap_dead start 2020-12-05T17:40:55.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f201956e700 1 --2- 172.21.15.32:0/389696676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20000044d0 0x7f2000004950 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 rx=0x7f200800a000 tx=0x7f2008008040).ready entity=mon.0 client_cookie=e5975d80d29d8648 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:55.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f2018d6d700 1 -- 172.21.15.32:0/389696676 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f200800ba60 con 0x7f20000044d0 2020-12-05T17:40:55.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f2018d6d700 1 -- 172.21.15.32:0/389696676 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f200800bbc0 con 0x7f20000044d0 2020-12-05T17:40:55.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f2018d6d700 1 -- 172.21.15.32:0/389696676 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2008013480 con 0x7f20000044d0 2020-12-05T17:40:55.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f201ad71700 1 -- 172.21.15.32:0/389696676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20000044d0 msgr2=0x7f2000004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:55.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f201ad71700 1 --2- 172.21.15.32:0/389696676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20000044d0 0x7f2000004950 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 rx=0x7f200800a000 tx=0x7f2008008040).stop 2020-12-05T17:40:55.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f201ad71700 1 -- 172.21.15.32:0/389696676 shutdown_connections 2020-12-05T17:40:55.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f201ad71700 1 --2- 172.21.15.32:0/389696676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20000044d0 0x7f2000004950 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:55.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f201ad71700 1 -- 172.21.15.32:0/389696676 >> 172.21.15.32:0/389696676 conn(0x7f20140b8ee0 msgr2=0x7f20140b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:55.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f201ad71700 1 -- 172.21.15.32:0/389696676 shutdown_connections 2020-12-05T17:40:55.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.335+0000 7f201ad71700 1 -- 172.21.15.32:0/389696676 wait complete. 2020-12-05T17:40:55.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.339+0000 7f201ad71700 1 Processor -- start 2020-12-05T17:40:55.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.339+0000 7f201ad71700 1 -- start start 2020-12-05T17:40:55.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.339+0000 7f201ad71700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20000044d0 0x7f20141547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:55.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.339+0000 7f201ad71700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20140650d0 0x7f2014140ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:55.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.339+0000 7f201ad71700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2014063b50 0x7f20140a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:55.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.343+0000 7f201ad71700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2014066b10 con 0x7f20140650d0 2020-12-05T17:40:55.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.343+0000 7f201ad71700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20140662b0 con 0x7f2014063b50 2020-12-05T17:40:55.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.343+0000 7f201ad71700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2014063fd0 con 0x7f20000044d0 2020-12-05T17:40:55.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.343+0000 7f2019d6f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20000044d0 0x7f20141547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:55.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.343+0000 7f201956e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20140650d0 0x7f2014140ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:55.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.343+0000 7f2019d6f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20000044d0 0x7f20141547b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:36754/0 (socket says 172.21.15.32:36754) 2020-12-05T17:40:55.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.343+0000 7f2019d6f700 1 -- 172.21.15.32:0/3589965930 learned_addr learned my addr 172.21.15.32:0/3589965930 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:55.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.343+0000 7f201956e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20140650d0 0x7f2014140ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39966/0 (socket says 172.21.15.32:39966) 2020-12-05T17:40:55.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.343+0000 7f201a570700 1 --2- 172.21.15.32:0/3589965930 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2014063b50 0x7f20140a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:55.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.343+0000 7f201956e700 1 -- 172.21.15.32:0/3589965930 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20000044d0 msgr2=0x7f20141547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:55.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.343+0000 7f201956e700 1 --2- 172.21.15.32:0/3589965930 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20000044d0 0x7f20141547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:55.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.343+0000 7f201956e700 1 -- 172.21.15.32:0/3589965930 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2014063b50 msgr2=0x7f20140a2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:55.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.343+0000 7f201956e700 1 --2- 172.21.15.32:0/3589965930 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2014063b50 0x7f20140a2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:55.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.343+0000 7f201956e700 1 -- 172.21.15.32:0/3589965930 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2014064260 con 0x7f20140650d0 2020-12-05T17:40:55.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.343+0000 7f201956e700 1 --2- 172.21.15.32:0/3589965930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20140650d0 0x7f2014140ae0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f20140a2cd0 tx=0x7f2008008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:55.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.343+0000 7f2012ffd700 1 -- 172.21.15.32:0/3589965930 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2008014df0 con 0x7f20140650d0 2020-12-05T17:40:55.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.343+0000 7f2012ffd700 1 -- 172.21.15.32:0/3589965930 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f200801d410 con 0x7f20140650d0 2020-12-05T17:40:55.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.343+0000 7f201ad71700 1 -- 172.21.15.32:0/3589965930 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2014064500 con 0x7f20140650d0 2020-12-05T17:40:55.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.343+0000 7f2012ffd700 1 -- 172.21.15.32:0/3589965930 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f200801d770 con 0x7f20140650d0 2020-12-05T17:40:55.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.343+0000 7f201ad71700 1 -- 172.21.15.32:0/3589965930 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20140a3500 con 0x7f20140650d0 2020-12-05T17:40:55.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.343+0000 7f2012ffd700 1 -- 172.21.15.32:0/3589965930 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f200801d8d0 con 0x7f20140650d0 2020-12-05T17:40:55.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.343+0000 7f201ad71700 1 -- 172.21.15.32:0/3589965930 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2014057050 con 0x7f20140650d0 2020-12-05T17:40:55.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.343+0000 7f2012ffd700 1 --2- 172.21.15.32:0/3589965930 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ff4055330 0x7f1ff40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:55.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.347+0000 7f2012ffd700 1 -- 172.21.15.32:0/3589965930 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f200806f880 con 0x7f20140650d0 2020-12-05T17:40:55.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.347+0000 7f2019d6f700 1 --2- 172.21.15.32:0/3589965930 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ff4055330 0x7f1ff40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:55.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.347+0000 7f2019d6f700 1 --2- 172.21.15.32:0/3589965930 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ff4055330 0x7f1ff40577f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f2004008000 tx=0x7f2004008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:55.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.351+0000 7f2012ffd700 1 -- 172.21.15.32:0/3589965930 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2008045900 con 0x7f20140650d0 2020-12-05T17:40:55.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.539+0000 7f201ad71700 1 -- 172.21.15.32:0/3589965930 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20140b2e60 con 0x7f20140650d0 2020-12-05T17:40:55.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.543+0000 7f2012ffd700 1 -- 172.21.15.32:0/3589965930 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+1426 (secure 0 0 0) 0x7f2008048f20 con 0x7f20140650d0 2020-12-05T17:40:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:40:55.547 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":58,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":58,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:40:50.514905+0000","modified":"2020-12-05T17:40:55.178705+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[22],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2020-12-05T17:40:55.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.543+0000 7f201ad71700 1 -- 172.21.15.32:0/3589965930 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ff4055330 msgr2=0x7f1ff40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:55.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.543+0000 7f201ad71700 1 --2- 172.21.15.32:0/3589965930 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ff4055330 0x7f1ff40577f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f2004008000 tx=0x7f2004008040).stop 2020-12-05T17:40:55.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.543+0000 7f201ad71700 1 -- 172.21.15.32:0/3589965930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20140650d0 msgr2=0x7f2014140ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:55.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.543+0000 7f201ad71700 1 --2- 172.21.15.32:0/3589965930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20140650d0 0x7f2014140ae0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f20140a2cd0 tx=0x7f2008008040).stop 2020-12-05T17:40:55.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.543+0000 7f201ad71700 1 -- 172.21.15.32:0/3589965930 shutdown_connections 2020-12-05T17:40:55.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.543+0000 7f201ad71700 1 --2- 172.21.15.32:0/3589965930 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2014063b50 0x7f20140a2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:55.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.543+0000 7f201ad71700 1 --2- 172.21.15.32:0/3589965930 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ff4055330 0x7f1ff40577f0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:55.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.543+0000 7f201ad71700 1 --2- 172.21.15.32:0/3589965930 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20000044d0 0x7f20141547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:55.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.543+0000 7f201ad71700 1 --2- 172.21.15.32:0/3589965930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20140650d0 0x7f2014140ae0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:55.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.543+0000 7f201ad71700 1 -- 172.21.15.32:0/3589965930 >> 172.21.15.32:0/3589965930 conn(0x7f20140b8ee0 msgr2=0x7f20140b6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:55.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.543+0000 7f201ad71700 1 -- 172.21.15.32:0/3589965930 shutdown_connections 2020-12-05T17:40:55.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.543+0000 7f201ad71700 1 -- 172.21.15.32:0/3589965930 wait complete. 2020-12-05T17:40:55.551 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 58 2020-12-05T17:40:55.564 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-12-05T17:40:55.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.679+0000 7f1c5f59e700 1 Processor -- start 2020-12-05T17:40:55.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.679+0000 7f1c5f59e700 1 -- start start 2020-12-05T17:40:55.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.679+0000 7f1c5f59e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1c6005feb0 0x7f1c600b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:55.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.679+0000 7f1c5f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1c60062210 0x7f1c60146970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:55.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.679+0000 7f1c5f59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c6005fad0 con 0x7f1c6005f6f0 2020-12-05T17:40:55.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.679+0000 7f1c5f59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c600b3360 con 0x7f1c60062210 2020-12-05T17:40:55.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.679+0000 7f1c5f59e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c600b5200 con 0x7f1c6005feb0 2020-12-05T17:40:55.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.679+0000 7f1c5e59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1c6005f6f0 0x7f1c60061b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42620/0 (socket says 172.21.15.32:42620) 2020-12-05T17:40:55.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.679+0000 7f1c5e59c700 1 -- 172.21.15.32:0/179995244 learned_addr learned my addr 172.21.15.32:0/179995244 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:55.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.679+0000 7f1c5ed9d700 1 --2- 172.21.15.32:0/179995244 >> v2:172.21.15.32:3300/0 conn(0x7f1c60062210 0x7f1c60146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:55.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.679+0000 7f1c5dd9b700 1 --2- 172.21.15.32:0/179995244 >> v2:172.21.15.61:3300/0 conn(0x7f1c6005feb0 0x7f1c600b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:55.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.679+0000 7f1c5d59a700 1 -- 172.21.15.32:0/179995244 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 613356565 0 0) 0x7f1c6005fad0 con 0x7f1c6005f6f0 2020-12-05T17:40:55.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.679+0000 7f1c5d59a700 1 -- 172.21.15.32:0/179995244 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c44003230 con 0x7f1c6005f6f0 2020-12-05T17:40:55.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.679+0000 7f1c5ed9d700 1 -- 172.21.15.32:0/179995244 >> v2:172.21.15.61:3300/0 conn(0x7f1c6005feb0 msgr2=0x7f1c600b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:55.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.679+0000 7f1c5ed9d700 1 --2- 172.21.15.32:0/179995244 >> v2:172.21.15.61:3300/0 conn(0x7f1c6005feb0 0x7f1c600b15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:55.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.679+0000 7f1c5ed9d700 1 -- 172.21.15.32:0/179995244 >> v1:172.21.15.32:6789/0 conn(0x7f1c6005f6f0 legacy=0x7f1c60061b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:55.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.679+0000 7f1c5ed9d700 1 -- 172.21.15.32:0/179995244 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c60062670 con 0x7f1c60062210 2020-12-05T17:40:55.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.679+0000 7f1c5ed9d700 1 --2- 172.21.15.32:0/179995244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c60062210 0x7f1c60146970 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f1c6005fc60 tx=0x7f1c58006dc0).ready entity=mon.0 client_cookie=a4bb177a5b1aed6d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:55.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.679+0000 7f1c5d59a700 1 -- 172.21.15.32:0/179995244 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c580096f0 con 0x7f1c60062210 2020-12-05T17:40:55.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.679+0000 7f1c5d59a700 1 -- 172.21.15.32:0/179995244 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c58009850 con 0x7f1c60062210 2020-12-05T17:40:55.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.679+0000 7f1c5d59a700 1 -- 172.21.15.32:0/179995244 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c58009bb0 con 0x7f1c60062210 2020-12-05T17:40:55.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.679+0000 7f1c5f59e700 1 -- 172.21.15.32:0/179995244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c60062210 msgr2=0x7f1c60146970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:55.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.679+0000 7f1c5f59e700 1 --2- 172.21.15.32:0/179995244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c60062210 0x7f1c60146970 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f1c6005fc60 tx=0x7f1c58006dc0).stop 2020-12-05T17:40:55.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5f59e700 1 -- 172.21.15.32:0/179995244 shutdown_connections 2020-12-05T17:40:55.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5f59e700 1 --2- 172.21.15.32:0/179995244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c60062210 0x7f1c60146970 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:55.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5f59e700 1 --2- 172.21.15.32:0/179995244 >> v2:172.21.15.61:3300/0 conn(0x7f1c6005feb0 0x7f1c600b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:55.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5f59e700 1 -- 172.21.15.32:0/179995244 >> 172.21.15.32:0/179995244 conn(0x7f1c600b8ee0 msgr2=0x7f1c600b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:55.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5f59e700 1 -- 172.21.15.32:0/179995244 shutdown_connections 2020-12-05T17:40:55.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5f59e700 1 -- 172.21.15.32:0/179995244 wait complete. 2020-12-05T17:40:55.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5f59e700 1 Processor -- start 2020-12-05T17:40:55.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5f59e700 1 -- start start 2020-12-05T17:40:55.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5f59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c6005f6f0 0x7f1c6014da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:55.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c6005feb0 0x7f1c601410a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:55.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c60062210 0x7f1c60141590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:55.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c60143700 con 0x7f1c6005feb0 2020-12-05T17:40:55.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5f59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c60066880 con 0x7f1c6005f6f0 2020-12-05T17:40:55.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c60066b20 con 0x7f1c60062210 2020-12-05T17:40:55.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c6005feb0 0x7f1c601410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:55.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c6005feb0 0x7f1c601410a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39980/0 (socket says 172.21.15.32:39980) 2020-12-05T17:40:55.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5dd9b700 1 -- 172.21.15.32:0/273521076 learned_addr learned my addr 172.21.15.32:0/273521076 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:55.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5e59c700 1 --2- 172.21.15.32:0/273521076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c6005f6f0 0x7f1c6014da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:55.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5ed9d700 1 --2- 172.21.15.32:0/273521076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c60062210 0x7f1c60141590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:55.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5dd9b700 1 -- 172.21.15.32:0/273521076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c60062210 msgr2=0x7f1c60141590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:55.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5dd9b700 1 --2- 172.21.15.32:0/273521076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c60062210 0x7f1c60141590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:55.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5dd9b700 1 -- 172.21.15.32:0/273521076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c6005f6f0 msgr2=0x7f1c6014da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:55.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5dd9b700 1 --2- 172.21.15.32:0/273521076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c6005f6f0 0x7f1c6014da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:55.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5dd9b700 1 -- 172.21.15.32:0/273521076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c60143380 con 0x7f1c6005feb0 2020-12-05T17:40:55.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5e59c700 1 --2- 172.21.15.32:0/273521076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c6005f6f0 0x7f1c6014da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:40:55.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5dd9b700 1 --2- 172.21.15.32:0/273521076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c6005feb0 0x7f1c601410a0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f1c5400c000 tx=0x7f1c54006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:55.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c4f7fe700 1 -- 172.21.15.32:0/273521076 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c5400cae0 con 0x7f1c6005feb0 2020-12-05T17:40:55.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c4f7fe700 1 -- 172.21.15.32:0/273521076 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5400cc40 con 0x7f1c6005feb0 2020-12-05T17:40:55.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.683+0000 7f1c5f59e700 1 -- 172.21.15.32:0/273521076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c60142f80 con 0x7f1c6005feb0 2020-12-05T17:40:55.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.687+0000 7f1c4f7fe700 1 -- 172.21.15.32:0/273521076 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c54013450 con 0x7f1c6005feb0 2020-12-05T17:40:55.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.687+0000 7f1c5f59e700 1 -- 172.21.15.32:0/273521076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c60147900 con 0x7f1c6005feb0 2020-12-05T17:40:55.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.687+0000 7f1c5f59e700 1 -- 172.21.15.32:0/273521076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c60066ce0 con 0x7f1c6005feb0 2020-12-05T17:40:55.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.687+0000 7f1c4f7fe700 1 -- 172.21.15.32:0/273521076 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1c5401a030 con 0x7f1c6005feb0 2020-12-05T17:40:55.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.687+0000 7f1c4f7fe700 1 --2- 172.21.15.32:0/273521076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1c3c055330 0x7f1c3c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:55.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.687+0000 7f1c5e59c700 1 --2- 172.21.15.32:0/273521076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1c3c055330 0x7f1c3c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:55.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.687+0000 7f1c4f7fe700 1 -- 172.21.15.32:0/273521076 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f1c5406df40 con 0x7f1c6005feb0 2020-12-05T17:40:55.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.691+0000 7f1c5e59c700 1 --2- 172.21.15.32:0/273521076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1c3c055330 0x7f1c3c0577f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f1c5000b000 tx=0x7f1c50009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:55.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.695+0000 7f1c4f7fe700 1 -- 172.21.15.32:0/273521076 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1c54042df0 con 0x7f1c6005feb0 2020-12-05T17:40:55.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:55.883+0000 7f1c5f59e700 1 -- 172.21.15.32:0/273521076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1c60057050 con 0x7f1c6005feb0 2020-12-05T17:40:56.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.195+0000 7f1c4f7fe700 1 -- 172.21.15.32:0/273521076 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1c54044140 con 0x7f1c6005feb0 2020-12-05T17:40:56.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.199+0000 7f1c5f59e700 1 -- 172.21.15.32:0/273521076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1c3c055330 msgr2=0x7f1c3c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:56.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.199+0000 7f1c5f59e700 1 --2- 172.21.15.32:0/273521076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1c3c055330 0x7f1c3c0577f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f1c5000b000 tx=0x7f1c50009040).stop 2020-12-05T17:40:56.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.199+0000 7f1c5f59e700 1 -- 172.21.15.32:0/273521076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c6005feb0 msgr2=0x7f1c601410a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:56.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.199+0000 7f1c5f59e700 1 --2- 172.21.15.32:0/273521076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c6005feb0 0x7f1c601410a0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f1c5400c000 tx=0x7f1c54006dc0).stop 2020-12-05T17:40:56.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.199+0000 7f1c5f59e700 1 -- 172.21.15.32:0/273521076 shutdown_connections 2020-12-05T17:40:56.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.199+0000 7f1c5f59e700 1 --2- 172.21.15.32:0/273521076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1c3c055330 0x7f1c3c0577f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:56.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.199+0000 7f1c5f59e700 1 --2- 172.21.15.32:0/273521076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c60062210 0x7f1c60141590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:56.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.199+0000 7f1c5f59e700 1 --2- 172.21.15.32:0/273521076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c6005f6f0 0x7f1c6014da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:56.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.199+0000 7f1c5f59e700 1 --2- 172.21.15.32:0/273521076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c6005feb0 0x7f1c601410a0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:56.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.199+0000 7f1c5f59e700 1 -- 172.21.15.32:0/273521076 >> 172.21.15.32:0/273521076 conn(0x7f1c600b8ee0 msgr2=0x7f1c600b8510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:56.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.199+0000 7f1c5f59e700 1 -- 172.21.15.32:0/273521076 shutdown_connections 2020-12-05T17:40:56.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.199+0000 7f1c5f59e700 1 -- 172.21.15.32:0/273521076 wait complete. 2020-12-05T17:40:56.221 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-12-05T17:40:56.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.347+0000 7fd0ade8e700 1 Processor -- start 2020-12-05T17:40:56.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.347+0000 7fd0ade8e700 1 -- start start 2020-12-05T17:40:56.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.347+0000 7fd0ade8e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd0a805f550 0x7fd0a80acb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:56.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.347+0000 7fd0ade8e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a8067110 con 0x7fd0a8063040 2020-12-05T17:40:56.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.347+0000 7fd0ade8e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a81405b0 con 0x7fd0a80af090 2020-12-05T17:40:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.347+0000 7fd0ade8e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0a813f760 con 0x7fd0a805f550 2020-12-05T17:40:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.347+0000 7fd0a7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd0a8063040 0x7fd0a813fea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42634/0 (socket says 172.21.15.32:42634) 2020-12-05T17:40:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.347+0000 7fd0a7fff700 1 -- 172.21.15.32:0/1928575571 learned_addr learned my addr 172.21.15.32:0/1928575571 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.347+0000 7fd0ace8c700 1 --2- 172.21.15.32:0/1928575571 >> v2:172.21.15.61:3300/0 conn(0x7fd0a805f550 0x7fd0a80acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:56.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0a77fe700 1 -- 172.21.15.32:0/1928575571 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122876808 0 0) 0x7fd0a8067110 con 0x7fd0a8063040 2020-12-05T17:40:56.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0a77fe700 1 -- 172.21.15.32:0/1928575571 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd08c003230 con 0x7fd0a8063040 2020-12-05T17:40:56.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0ace8c700 1 -- 172.21.15.32:0/1928575571 >> v1:172.21.15.61:6789/0 conn(0x7fd0a80af090 legacy=0x7fd0a80af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:56.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0ace8c700 1 -- 172.21.15.32:0/1928575571 >> v1:172.21.15.32:6789/0 conn(0x7fd0a8063040 legacy=0x7fd0a813fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:56.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0ace8c700 1 -- 172.21.15.32:0/1928575571 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a813fa00 con 0x7fd0a805f550 2020-12-05T17:40:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0ace8c700 1 --2- 172.21.15.32:0/1928575571 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0a805f550 0x7fd0a80acb80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7fd09c002f30 tx=0x7fd09c006dc0).ready entity=mon.1 client_cookie=fdb125f77ba67667 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0a77fe700 1 -- 172.21.15.32:0/1928575571 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd09c00fa60 con 0x7fd0a805f550 2020-12-05T17:40:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0a77fe700 1 -- 172.21.15.32:0/1928575571 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09c00fbc0 con 0x7fd0a805f550 2020-12-05T17:40:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0a77fe700 1 -- 172.21.15.32:0/1928575571 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd09c010470 con 0x7fd0a805f550 2020-12-05T17:40:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0ade8e700 1 -- 172.21.15.32:0/1928575571 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0a805f550 msgr2=0x7fd0a80acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0ade8e700 1 --2- 172.21.15.32:0/1928575571 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0a805f550 0x7fd0a80acb80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7fd09c002f30 tx=0x7fd09c006dc0).stop 2020-12-05T17:40:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0ade8e700 1 -- 172.21.15.32:0/1928575571 shutdown_connections 2020-12-05T17:40:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0ade8e700 1 --2- 172.21.15.32:0/1928575571 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0a805f550 0x7fd0a80acb80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0ade8e700 1 -- 172.21.15.32:0/1928575571 >> 172.21.15.32:0/1928575571 conn(0x7fd0a80620b0 msgr2=0x7fd0a8062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0ade8e700 1 -- 172.21.15.32:0/1928575571 shutdown_connections 2020-12-05T17:40:56.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0ade8e700 1 -- 172.21.15.32:0/1928575571 wait complete. 2020-12-05T17:40:56.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0ade8e700 1 Processor -- start 2020-12-05T17:40:56.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0ade8e700 1 -- start start 2020-12-05T17:40:56.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0ade8e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0a805f550 0x7fd0a81491c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:56.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0ade8e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0a8063040 0x7fd0a8140c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:56.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0ade8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a80af090 0x7fd0a81433b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:56.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0ade8e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a8065930 con 0x7fd0a80af090 2020-12-05T17:40:56.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0ade8e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a8065bd0 con 0x7fd0a8063040 2020-12-05T17:40:56.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0ade8e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a8065e70 con 0x7fd0a805f550 2020-12-05T17:40:56.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.351+0000 7fd0ad68d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a80af090 0x7fd0a81433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:56.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.355+0000 7fd0ad68d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a80af090 0x7fd0a81433b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39994/0 (socket says 172.21.15.32:39994) 2020-12-05T17:40:56.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.355+0000 7fd0ad68d700 1 -- 172.21.15.32:0/792089351 learned_addr learned my addr 172.21.15.32:0/792089351 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:56.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.355+0000 7fd0a7fff700 1 --2- 172.21.15.32:0/792089351 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0a8063040 0x7fd0a8140c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:56.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.355+0000 7fd0ace8c700 1 --2- 172.21.15.32:0/792089351 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0a805f550 0x7fd0a81491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:56.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.355+0000 7fd0ad68d700 1 -- 172.21.15.32:0/792089351 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0a805f550 msgr2=0x7fd0a81491c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:56.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.355+0000 7fd0ad68d700 1 --2- 172.21.15.32:0/792089351 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0a805f550 0x7fd0a81491c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:56.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.355+0000 7fd0ad68d700 1 -- 172.21.15.32:0/792089351 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0a8063040 msgr2=0x7fd0a8140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:56.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.355+0000 7fd0ad68d700 1 --2- 172.21.15.32:0/792089351 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0a8063040 0x7fd0a8140c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:56.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.355+0000 7fd0ad68d700 1 -- 172.21.15.32:0/792089351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a80662b0 con 0x7fd0a80af090 2020-12-05T17:40:56.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.355+0000 7fd0ad68d700 1 --2- 172.21.15.32:0/792089351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a80af090 0x7fd0a81433b0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7fd0a80ae750 tx=0x7fd0a0007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:56.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.355+0000 7fd0a5ffb700 1 -- 172.21.15.32:0/792089351 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0a000a5a0 con 0x7fd0a80af090 2020-12-05T17:40:56.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.355+0000 7fd0ade8e700 1 -- 172.21.15.32:0/792089351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a81410f0 con 0x7fd0a80af090 2020-12-05T17:40:56.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.355+0000 7fd0ade8e700 1 -- 172.21.15.32:0/792089351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a8066710 con 0x7fd0a80af090 2020-12-05T17:40:56.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.355+0000 7fd0a5ffb700 1 -- 172.21.15.32:0/792089351 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a000a700 con 0x7fd0a80af090 2020-12-05T17:40:56.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.355+0000 7fd0a5ffb700 1 -- 172.21.15.32:0/792089351 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0a000aa60 con 0x7fd0a80af090 2020-12-05T17:40:56.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.355+0000 7fd0ade8e700 1 -- 172.21.15.32:0/792089351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a8057050 con 0x7fd0a80af090 2020-12-05T17:40:56.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.355+0000 7fd0a5ffb700 1 -- 172.21.15.32:0/792089351 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd0a0019030 con 0x7fd0a80af090 2020-12-05T17:40:56.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.359+0000 7fd0a5ffb700 1 --2- 172.21.15.32:0/792089351 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd090055440 0x7fd090057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:56.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.359+0000 7fd0ace8c700 1 --2- 172.21.15.32:0/792089351 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd090055440 0x7fd090057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:56.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.359+0000 7fd0a5ffb700 1 -- 172.21.15.32:0/792089351 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fd0a006d830 con 0x7fd0a80af090 2020-12-05T17:40:56.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.359+0000 7fd0ace8c700 1 --2- 172.21.15.32:0/792089351 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd090055440 0x7fd090057900 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7fd09c01a000 tx=0x7fd09c017000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:56.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.363+0000 7fd0a5ffb700 1 -- 172.21.15.32:0/792089351 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd0a00438e0 con 0x7fd0a80af090 2020-12-05T17:40:56.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:56.507+0000 7fd0ade8e700 1 -- 172.21.15.32:0/792089351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd0a8060c10 con 0x7fd0a80af090 2020-12-05T17:40:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.207+0000 7fd0a5ffb700 1 -- 172.21.15.32:0/792089351 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v123) v1 ==== 185+0+0 (secure 0 0 0) 0x7fd0a0046f00 con 0x7fd0a80af090 2020-12-05T17:40:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.211+0000 7fd0ade8e700 1 -- 172.21.15.32:0/792089351 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd090055440 msgr2=0x7fd090057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.211+0000 7fd0ade8e700 1 --2- 172.21.15.32:0/792089351 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd090055440 0x7fd090057900 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7fd09c01a000 tx=0x7fd09c017000).stop 2020-12-05T17:40:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.211+0000 7fd0ade8e700 1 -- 172.21.15.32:0/792089351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a80af090 msgr2=0x7fd0a81433b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.211+0000 7fd0ade8e700 1 --2- 172.21.15.32:0/792089351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a80af090 0x7fd0a81433b0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7fd0a80ae750 tx=0x7fd0a0007f70).stop 2020-12-05T17:40:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.211+0000 7fd0ade8e700 1 -- 172.21.15.32:0/792089351 shutdown_connections 2020-12-05T17:40:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.211+0000 7fd0ade8e700 1 --2- 172.21.15.32:0/792089351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a80af090 0x7fd0a81433b0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.211+0000 7fd0ade8e700 1 --2- 172.21.15.32:0/792089351 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd090055440 0x7fd090057900 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:57.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.211+0000 7fd0ade8e700 1 --2- 172.21.15.32:0/792089351 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0a805f550 0x7fd0a81491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:57.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.211+0000 7fd0ade8e700 1 --2- 172.21.15.32:0/792089351 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0a8063040 0x7fd0a8140c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:57.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.211+0000 7fd0ade8e700 1 -- 172.21.15.32:0/792089351 >> 172.21.15.32:0/792089351 conn(0x7fd0a80620b0 msgr2=0x7fd0a80b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:57.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.211+0000 7fd0ade8e700 1 -- 172.21.15.32:0/792089351 shutdown_connections 2020-12-05T17:40:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.211+0000 7fd0ade8e700 1 -- 172.21.15.32:0/792089351 wait complete. 2020-12-05T17:40:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2020-12-05T17:40:57.233 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-12-05T17:40:57.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14d5c3d700 1 Processor -- start 2020-12-05T17:40:57.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14d5c3d700 1 -- start start 2020-12-05T17:40:57.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14d5c3d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f14d0061ba0 0x7f14d0066e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:57.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14d5c3d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d00b3540 con 0x7f14d0144380 2020-12-05T17:40:57.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14d5c3d700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d0061f80 con 0x7f14d00b3020 2020-12-05T17:40:57.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14d5c3d700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d0066750 con 0x7f14d0061ba0 2020-12-05T17:40:57.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14d543c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f14d0144380 0x7f14d013a1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42652/0 (socket says 172.21.15.32:42652) 2020-12-05T17:40:57.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14d543c700 1 -- 172.21.15.32:0/2710498900 learned_addr learned my addr 172.21.15.32:0/2710498900 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:40:57.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14d4c3b700 1 --2- 172.21.15.32:0/2710498900 >> v2:172.21.15.61:3300/0 conn(0x7f14d0061ba0 0x7f14d0066e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:57.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14cf7fe700 1 -- 172.21.15.32:0/2710498900 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940059734 0 0) 0x7f14d00b3540 con 0x7f14d0144380 2020-12-05T17:40:57.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14cf7fe700 1 -- 172.21.15.32:0/2710498900 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14b0003230 con 0x7f14d0144380 2020-12-05T17:40:57.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14cf7fe700 1 -- 172.21.15.32:0/2710498900 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3777098887 0 0) 0x7f14b0003230 con 0x7f14d0144380 2020-12-05T17:40:57.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14cf7fe700 1 -- 172.21.15.32:0/2710498900 >> v2:172.21.15.61:3300/0 conn(0x7f14d0061ba0 msgr2=0x7f14d0066e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:57.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14cf7fe700 1 --2- 172.21.15.32:0/2710498900 >> v2:172.21.15.61:3300/0 conn(0x7f14d0061ba0 0x7f14d0066e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:57.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14cf7fe700 1 -- 172.21.15.32:0/2710498900 >> v1:172.21.15.61:6789/0 conn(0x7f14d00b3020 legacy=0x7f14d0143c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:57.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14cf7fe700 1 -- 172.21.15.32:0/2710498900 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d00669f0 con 0x7f14d0144380 2020-12-05T17:40:57.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14cf7fe700 1 -- 172.21.15.32:0/2710498900 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f14c80016d0 con 0x7f14d0144380 2020-12-05T17:40:57.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14cf7fe700 1 -- 172.21.15.32:0/2710498900 >> v1:172.21.15.32:6789/0 conn(0x7f14d0144380 legacy=0x7f14d013a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:57.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14cf7fe700 1 --2- 172.21.15.32:0/2710498900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b0003720 0x7f14b0003b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:57.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14cf7fe700 1 --2- 172.21.15.32:0/2710498900 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14b00044d0 0x7f14b0004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:57.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14cf7fe700 1 --2- 172.21.15.32:0/2710498900 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14b0004e40 0x7f14b000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:57.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14cf7fe700 1 -- 172.21.15.32:0/2710498900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d0066750 con 0x7f14b0003720 2020-12-05T17:40:57.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14cf7fe700 1 -- 172.21.15.32:0/2710498900 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b00041c0 con 0x7f14b00044d0 2020-12-05T17:40:57.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14cf7fe700 1 -- 172.21.15.32:0/2710498900 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b000bc10 con 0x7f14b0004e40 2020-12-05T17:40:57.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14d4c3b700 1 --2- 172.21.15.32:0/2710498900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b0003720 0x7f14b0003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:57.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14d543c700 1 --2- 172.21.15.32:0/2710498900 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14b0004e40 0x7f14b000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:57.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14cffff700 1 --2- 172.21.15.32:0/2710498900 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14b00044d0 0x7f14b0004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:57.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14d4c3b700 1 -- 172.21.15.32:0/2710498900 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14b0004e40 msgr2=0x7f14b000b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:57.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14d4c3b700 1 --2- 172.21.15.32:0/2710498900 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14b0004e40 0x7f14b000b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:57.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14d4c3b700 1 -- 172.21.15.32:0/2710498900 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14b00044d0 msgr2=0x7f14b0004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:57.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14d4c3b700 1 --2- 172.21.15.32:0/2710498900 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14b00044d0 0x7f14b0004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:57.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14d4c3b700 1 -- 172.21.15.32:0/2710498900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b0003230 con 0x7f14b0003720 2020-12-05T17:40:57.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14d543c700 1 --2- 172.21.15.32:0/2710498900 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14b0004e40 0x7f14b000b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:40:57.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14d543c700 1 -- 172.21.15.32:0/2710498900 reap_dead start 2020-12-05T17:40:57.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14d4c3b700 1 --2- 172.21.15.32:0/2710498900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b0003720 0x7f14b0003b00 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 rx=0x7f14c000c000 tx=0x7f14c00091a0).ready entity=mon.0 client_cookie=6bade2111af3c29c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:57.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14cf7fe700 1 -- 172.21.15.32:0/2710498900 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14c000ed00 con 0x7f14b0003720 2020-12-05T17:40:57.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14cf7fe700 1 -- 172.21.15.32:0/2710498900 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c000ee60 con 0x7f14b0003720 2020-12-05T17:40:57.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14cf7fe700 1 -- 172.21.15.32:0/2710498900 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14c00125e0 con 0x7f14b0003720 2020-12-05T17:40:57.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14d5c3d700 1 -- 172.21.15.32:0/2710498900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b0003720 msgr2=0x7f14b0003b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:57.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.343+0000 7f14d5c3d700 1 --2- 172.21.15.32:0/2710498900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b0003720 0x7f14b0003b00 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 rx=0x7f14c000c000 tx=0x7f14c00091a0).stop 2020-12-05T17:40:57.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d5c3d700 1 -- 172.21.15.32:0/2710498900 shutdown_connections 2020-12-05T17:40:57.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d5c3d700 1 --2- 172.21.15.32:0/2710498900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b0003720 0x7f14b0003b00 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:57.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d5c3d700 1 -- 172.21.15.32:0/2710498900 >> 172.21.15.32:0/2710498900 conn(0x7f14d00b6d10 msgr2=0x7f14d00b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:57.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d5c3d700 1 -- 172.21.15.32:0/2710498900 shutdown_connections 2020-12-05T17:40:57.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d5c3d700 1 -- 172.21.15.32:0/2710498900 wait complete. 2020-12-05T17:40:57.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d5c3d700 1 Processor -- start 2020-12-05T17:40:57.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d5c3d700 1 -- start start 2020-12-05T17:40:57.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d5c3d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b0003720 0x7f14d01104c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:57.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d5c3d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14d0111a20 0x7f14d0111e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:57.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d5c3d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14d0152d30 0x7f14d01531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d5c3d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d0112370 con 0x7f14b0003720 2020-12-05T17:40:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d5c3d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d00b09b0 con 0x7f14d0152d30 2020-12-05T17:40:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d5c3d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d0111590 con 0x7f14d0111a20 2020-12-05T17:40:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d4c3b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b0003720 0x7f14d01104c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d4c3b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b0003720 0x7f14d01104c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40012/0 (socket says 172.21.15.32:40012) 2020-12-05T17:40:57.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d4c3b700 1 -- 172.21.15.32:0/188398586 learned_addr learned my addr 172.21.15.32:0/188398586 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:57.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14cffff700 1 --2- 172.21.15.32:0/188398586 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14d0111a20 0x7f14d0111e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:57.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d543c700 1 --2- 172.21.15.32:0/188398586 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14d0152d30 0x7f14d01531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:57.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d4c3b700 1 -- 172.21.15.32:0/188398586 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14d0111a20 msgr2=0x7f14d0111e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:57.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d4c3b700 1 --2- 172.21.15.32:0/188398586 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14d0111a20 0x7f14d0111e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:57.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d4c3b700 1 -- 172.21.15.32:0/188398586 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14d0152d30 msgr2=0x7f14d01531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:57.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d4c3b700 1 --2- 172.21.15.32:0/188398586 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14d0152d30 0x7f14d01531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:57.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d4c3b700 1 -- 172.21.15.32:0/188398586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d0114440 con 0x7f14b0003720 2020-12-05T17:40:57.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d4c3b700 1 --2- 172.21.15.32:0/188398586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b0003720 0x7f14d01104c0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f14c000c000 tx=0x7f14c001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:57.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14cdffb700 1 -- 172.21.15.32:0/188398586 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14c00095c0 con 0x7f14b0003720 2020-12-05T17:40:57.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14cdffb700 1 -- 172.21.15.32:0/188398586 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c0012600 con 0x7f14b0003720 2020-12-05T17:40:57.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d5c3d700 1 -- 172.21.15.32:0/188398586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d01146e0 con 0x7f14b0003720 2020-12-05T17:40:57.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14cdffb700 1 -- 172.21.15.32:0/188398586 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14c0012960 con 0x7f14b0003720 2020-12-05T17:40:57.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.347+0000 7f14d5c3d700 1 -- 172.21.15.32:0/188398586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d010ff00 con 0x7f14b0003720 2020-12-05T17:40:57.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.351+0000 7f14d5c3d700 1 -- 172.21.15.32:0/188398586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d0057050 con 0x7f14b0003720 2020-12-05T17:40:57.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.351+0000 7f14cdffb700 1 -- 172.21.15.32:0/188398586 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f14c0012ac0 con 0x7f14b0003720 2020-12-05T17:40:57.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.351+0000 7f14cdffb700 1 --2- 172.21.15.32:0/188398586 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f14b40554e0 0x7f14b40579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:57.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.351+0000 7f14cdffb700 1 -- 172.21.15.32:0/188398586 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f14c006fcd0 con 0x7f14b0003720 2020-12-05T17:40:57.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.351+0000 7f14cffff700 1 --2- 172.21.15.32:0/188398586 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f14b40554e0 0x7f14b40579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:57.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.351+0000 7f14cffff700 1 --2- 172.21.15.32:0/188398586 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f14b40554e0 0x7f14b40579a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f14b800d000 tx=0x7f14b8009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:57.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.355+0000 7f14cdffb700 1 -- 172.21.15.32:0/188398586 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f14c0049a00 con 0x7f14b0003720 2020-12-05T17:40:57.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:57.499+0000 7f14d5c3d700 1 -- 172.21.15.32:0/188398586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f14d00b7ad0 con 0x7f14b0003720 2020-12-05T17:40:58.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.211+0000 7f14cdffb700 1 -- 172.21.15.32:0/188398586 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v124) v1 ==== 173+0+0 (secure 0 0 0) 0x7f14c00280d0 con 0x7f14b0003720 2020-12-05T17:40:58.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.215+0000 7f14d5c3d700 1 -- 172.21.15.32:0/188398586 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f14b40554e0 msgr2=0x7f14b40579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:58.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.215+0000 7f14d5c3d700 1 --2- 172.21.15.32:0/188398586 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f14b40554e0 0x7f14b40579a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f14b800d000 tx=0x7f14b8009040).stop 2020-12-05T17:40:58.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.215+0000 7f14d5c3d700 1 -- 172.21.15.32:0/188398586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b0003720 msgr2=0x7f14d01104c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:58.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.215+0000 7f14d5c3d700 1 --2- 172.21.15.32:0/188398586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b0003720 0x7f14d01104c0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f14c000c000 tx=0x7f14c001a000).stop 2020-12-05T17:40:58.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.215+0000 7f14d5c3d700 1 -- 172.21.15.32:0/188398586 shutdown_connections 2020-12-05T17:40:58.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.215+0000 7f14d5c3d700 1 --2- 172.21.15.32:0/188398586 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14d0152d30 0x7f14d01531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:58.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.215+0000 7f14d5c3d700 1 --2- 172.21.15.32:0/188398586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b0003720 0x7f14d01104c0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:58.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.215+0000 7f14d5c3d700 1 --2- 172.21.15.32:0/188398586 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f14b40554e0 0x7f14b40579a0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:58.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.215+0000 7f14d5c3d700 1 --2- 172.21.15.32:0/188398586 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14d0111a20 0x7f14d0111e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:58.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.215+0000 7f14d5c3d700 1 -- 172.21.15.32:0/188398586 >> 172.21.15.32:0/188398586 conn(0x7f14d00b6d10 msgr2=0x7f14d013e680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:58.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.219+0000 7f14d5c3d700 1 -- 172.21.15.32:0/188398586 shutdown_connections 2020-12-05T17:40:58.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.219+0000 7f14d5c3d700 1 -- 172.21.15.32:0/188398586 wait complete. 2020-12-05T17:40:58.224 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2020-12-05T17:40:58.236 INFO:tasks.ceph.mds.a:Restarting daemon 2020-12-05T17:40:58.237 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-12-05T17:40:58.237 INFO:tasks.ceph.mds.b:Restarting daemon 2020-12-05T17:40:58.238 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-12-05T17:40:58.240 INFO:tasks.ceph.mds.a:Started 2020-12-05T17:40:58.240 DEBUG:teuthology.parallel:result is None 2020-12-05T17:40:58.241 INFO:tasks.ceph.mds.b:Started 2020-12-05T17:40:58.241 DEBUG:teuthology.parallel:result is None 2020-12-05T17:40:58.242 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-12-05T17:40:58.311 INFO:tasks.ceph.mds.b.smithi061.stdout:starting mds.b at 2020-12-05T17:40:58.317 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:40:58.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.351+0000 7fc428f60700 1 Processor -- start 2020-12-05T17:40:58.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.351+0000 7fc428f60700 1 -- start start 2020-12-05T17:40:58.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.351+0000 7fc428f60700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4240b2070 0x7fc424138e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:58.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.351+0000 7fc428f60700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc4240b2ca0 0x7fc424139320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:58.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.351+0000 7fc428f60700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4240b2450 con 0x7fc424142980 2020-12-05T17:40:58.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.351+0000 7fc428f60700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4240acb80 con 0x7fc4240b2ca0 2020-12-05T17:40:58.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.351+0000 7fc428f60700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc424139810 con 0x7fc4240b2070 2020-12-05T17:40:58.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.351+0000 7fc422ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc4240b2ca0 0x7fc424139320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:58.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.351+0000 7fc422ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc4240b2ca0 0x7fc424139320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40034/0 (socket says 172.21.15.32:40034) 2020-12-05T17:40:58.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.351+0000 7fc422ffd700 1 -- 172.21.15.32:0/4181612550 learned_addr learned my addr 172.21.15.32:0/4181612550 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:58.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.351+0000 7fc4237fe700 1 --2- 172.21.15.32:0/4181612550 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4240b2070 0x7fc424138e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:58.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.355+0000 7fc4227fc700 1 -- 172.21.15.32:0/4181612550 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2780107678 0 0) 0x7fc4240b2450 con 0x7fc424142980 2020-12-05T17:40:58.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.355+0000 7fc4227fc700 1 -- 172.21.15.32:0/4181612550 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc410003230 con 0x7fc424142980 2020-12-05T17:40:58.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.355+0000 7fc422ffd700 1 -- 172.21.15.32:0/4181612550 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4240b2070 msgr2=0x7fc424138e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:58.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.355+0000 7fc422ffd700 1 --2- 172.21.15.32:0/4181612550 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4240b2070 0x7fc424138e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:58.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.355+0000 7fc422ffd700 1 -- 172.21.15.32:0/4181612550 >> v1:172.21.15.61:6789/0 conn(0x7fc424142980 legacy=0x7fc424142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:58.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.355+0000 7fc422ffd700 1 -- 172.21.15.32:0/4181612550 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc424066d40 con 0x7fc4240b2ca0 2020-12-05T17:40:58.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.355+0000 7fc422ffd700 1 --2- 172.21.15.32:0/4181612550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4240b2ca0 0x7fc424139320 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7fc418002f30 tx=0x7fc418006e80).ready entity=mon.0 client_cookie=87a6df161a67150c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:58.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.355+0000 7fc4227fc700 1 -- 172.21.15.32:0/4181612550 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc418009640 con 0x7fc4240b2ca0 2020-12-05T17:40:58.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.355+0000 7fc4227fc700 1 -- 172.21.15.32:0/4181612550 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4180097a0 con 0x7fc4240b2ca0 2020-12-05T17:40:58.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.355+0000 7fc4227fc700 1 -- 172.21.15.32:0/4181612550 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc418009b00 con 0x7fc4240b2ca0 2020-12-05T17:40:58.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.355+0000 7fc428f60700 1 -- 172.21.15.32:0/4181612550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4240b2ca0 msgr2=0x7fc424139320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:58.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.355+0000 7fc428f60700 1 --2- 172.21.15.32:0/4181612550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4240b2ca0 0x7fc424139320 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7fc418002f30 tx=0x7fc418006e80).stop 2020-12-05T17:40:58.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.355+0000 7fc428f60700 1 -- 172.21.15.32:0/4181612550 shutdown_connections 2020-12-05T17:40:58.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.355+0000 7fc428f60700 1 --2- 172.21.15.32:0/4181612550 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4240b2070 0x7fc424138e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:58.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.355+0000 7fc428f60700 1 --2- 172.21.15.32:0/4181612550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4240b2ca0 0x7fc424139320 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:58.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.355+0000 7fc428f60700 1 -- 172.21.15.32:0/4181612550 >> 172.21.15.32:0/4181612550 conn(0x7fc42405bbb0 msgr2=0x7fc424062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.355+0000 7fc428f60700 1 -- 172.21.15.32:0/4181612550 shutdown_connections 2020-12-05T17:40:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.355+0000 7fc428f60700 1 -- 172.21.15.32:0/4181612550 wait complete. 2020-12-05T17:40:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.355+0000 7fc428f60700 1 Processor -- start 2020-12-05T17:40:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.355+0000 7fc428f60700 1 -- start start 2020-12-05T17:40:58.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.359+0000 7fc428f60700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4240b2070 0x7fc424147610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:58.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.359+0000 7fc428f60700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4240b2ca0 0x7fc424152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:58.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.359+0000 7fc428f60700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc424142980 0x7fc42413a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:58.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.359+0000 7fc428f60700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42413ca20 con 0x7fc424142980 2020-12-05T17:40:58.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.359+0000 7fc428f60700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4241530a0 con 0x7fc4240b2070 2020-12-05T17:40:58.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.359+0000 7fc428f60700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc424141550 con 0x7fc4240b2ca0 2020-12-05T17:40:58.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.359+0000 7fc423fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc424142980 0x7fc42413a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:58.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.359+0000 7fc423fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc424142980 0x7fc42413a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40042/0 (socket says 172.21.15.32:40042) 2020-12-05T17:40:58.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.359+0000 7fc422ffd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4240b2ca0 0x7fc424152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:58.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.359+0000 7fc423fff700 1 -- 172.21.15.32:0/1760324166 learned_addr learned my addr 172.21.15.32:0/1760324166 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:58.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.359+0000 7fc4237fe700 1 --2- 172.21.15.32:0/1760324166 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4240b2070 0x7fc424147610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:58.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.359+0000 7fc423fff700 1 -- 172.21.15.32:0/1760324166 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4240b2ca0 msgr2=0x7fc424152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:58.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.359+0000 7fc423fff700 1 --2- 172.21.15.32:0/1760324166 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4240b2ca0 0x7fc424152bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:58.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.359+0000 7fc423fff700 1 -- 172.21.15.32:0/1760324166 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4240b2070 msgr2=0x7fc424147610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:58.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.359+0000 7fc423fff700 1 --2- 172.21.15.32:0/1760324166 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4240b2070 0x7fc424147610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:58.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.359+0000 7fc423fff700 1 -- 172.21.15.32:0/1760324166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4241417f0 con 0x7fc424142980 2020-12-05T17:40:58.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.359+0000 7fc423fff700 1 --2- 172.21.15.32:0/1760324166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc424142980 0x7fc42413a150 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7fc41c001450 tx=0x7fc41c007da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:58.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.359+0000 7fc420ff9700 1 -- 172.21.15.32:0/1760324166 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc41c002240 con 0x7fc424142980 2020-12-05T17:40:58.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.359+0000 7fc420ff9700 1 -- 172.21.15.32:0/1760324166 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41c0023a0 con 0x7fc424142980 2020-12-05T17:40:58.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.359+0000 7fc428f60700 1 -- 172.21.15.32:0/1760324166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc424141a90 con 0x7fc424142980 2020-12-05T17:40:58.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.359+0000 7fc420ff9700 1 -- 172.21.15.32:0/1760324166 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc41c00a790 con 0x7fc424142980 2020-12-05T17:40:58.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.359+0000 7fc428f60700 1 -- 172.21.15.32:0/1760324166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc424139ba0 con 0x7fc424142980 2020-12-05T17:40:58.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.359+0000 7fc428f60700 1 -- 172.21.15.32:0/1760324166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc424139eb0 con 0x7fc424142980 2020-12-05T17:40:58.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.363+0000 7fc420ff9700 1 -- 172.21.15.32:0/1760324166 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc41c019030 con 0x7fc424142980 2020-12-05T17:40:58.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.363+0000 7fc420ff9700 1 --2- 172.21.15.32:0/1760324166 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc4040553b0 0x7fc404057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:58.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.363+0000 7fc420ff9700 1 -- 172.21.15.32:0/1760324166 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fc41c06d5c0 con 0x7fc424142980 2020-12-05T17:40:58.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.363+0000 7fc4237fe700 1 --2- 172.21.15.32:0/1760324166 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc4040553b0 0x7fc404057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:58.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.363+0000 7fc4237fe700 1 --2- 172.21.15.32:0/1760324166 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc4040553b0 0x7fc404057870 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7fc41400b000 tx=0x7fc414006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:58.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.371+0000 7fc420ff9700 1 -- 172.21.15.32:0/1760324166 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc424139eb0 con 0x7fc424142980 2020-12-05T17:40:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:58.531+0000 7fc428f60700 1 -- 172.21.15.32:0/1760324166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fc424139eb0 con 0x7fc424142980 2020-12-05T17:40:59.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.338+0000 7fc420ff9700 1 -- 172.21.15.32:0/1760324166 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v125) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc424139eb0 con 0x7fc424142980 2020-12-05T17:40:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.338+0000 7fc428f60700 1 -- 172.21.15.32:0/1760324166 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc4040553b0 msgr2=0x7fc404057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.338+0000 7fc428f60700 1 --2- 172.21.15.32:0/1760324166 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc4040553b0 0x7fc404057870 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7fc41400b000 tx=0x7fc414006dc0).stop 2020-12-05T17:40:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.338+0000 7fc428f60700 1 -- 172.21.15.32:0/1760324166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc424142980 msgr2=0x7fc42413a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.338+0000 7fc428f60700 1 --2- 172.21.15.32:0/1760324166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc424142980 0x7fc42413a150 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7fc41c001450 tx=0x7fc41c007da0).stop 2020-12-05T17:40:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.338+0000 7fc428f60700 1 -- 172.21.15.32:0/1760324166 shutdown_connections 2020-12-05T17:40:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.338+0000 7fc428f60700 1 --2- 172.21.15.32:0/1760324166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc424142980 0x7fc42413a150 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.338+0000 7fc428f60700 1 --2- 172.21.15.32:0/1760324166 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4240b2070 0x7fc424147610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.338+0000 7fc428f60700 1 --2- 172.21.15.32:0/1760324166 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc4040553b0 0x7fc404057870 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.338+0000 7fc428f60700 1 --2- 172.21.15.32:0/1760324166 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4240b2ca0 0x7fc424152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.338+0000 7fc428f60700 1 -- 172.21.15.32:0/1760324166 >> 172.21.15.32:0/1760324166 conn(0x7fc42405bbb0 msgr2=0x7fc424142e10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.342+0000 7fc428f60700 1 -- 172.21.15.32:0/1760324166 shutdown_connections 2020-12-05T17:40:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.342+0000 7fc428f60700 1 -- 172.21.15.32:0/1760324166 wait complete. 2020-12-05T17:40:59.348 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2020-12-05T17:40:59.361 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2020-12-05T17:40:59.362 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-12-05T17:40:59.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.474+0000 7f53eca24700 1 Processor -- start 2020-12-05T17:40:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.474+0000 7f53eca24700 1 -- start start 2020-12-05T17:40:59.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.474+0000 7f53eca24700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53e80b1fc0 0x7f53e80af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:59.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.474+0000 7f53eca24700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53e8147400 0x7f53e8145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:59.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.474+0000 7f53eca24700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e8146a90 con 0x7f53e80ab710 2020-12-05T17:40:59.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.474+0000 7f53eca24700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e8061e90 con 0x7f53e8147400 2020-12-05T17:40:59.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.474+0000 7f53eca24700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e80b0110 con 0x7f53e80b1fc0 2020-12-05T17:40:59.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.474+0000 7f53e7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53e8147400 0x7f53e8145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:59.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.474+0000 7f53e7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53e8147400 0x7f53e8145d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40052/0 (socket says 172.21.15.32:40052) 2020-12-05T17:40:59.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.474+0000 7f53e7fff700 1 -- 172.21.15.32:0/510472096 learned_addr learned my addr 172.21.15.32:0/510472096 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:59.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.474+0000 7f53e77fe700 1 --2- 172.21.15.32:0/510472096 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53e80b1fc0 0x7f53e80af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:59.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.474+0000 7f53e7fff700 1 -- 172.21.15.32:0/510472096 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53e80b1fc0 msgr2=0x7f53e80af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:59.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.474+0000 7f53e7fff700 1 --2- 172.21.15.32:0/510472096 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53e80b1fc0 0x7f53e80af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:59.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.474+0000 7f53e7fff700 1 -- 172.21.15.32:0/510472096 >> v1:172.21.15.61:6789/0 conn(0x7f53e80ab710 legacy=0x7f53e8061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:59.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.474+0000 7f53e7fff700 1 -- 172.21.15.32:0/510472096 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e80b23a0 con 0x7f53e8147400 2020-12-05T17:40:59.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.474+0000 7f53e7fff700 1 --2- 172.21.15.32:0/510472096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e8147400 0x7f53e8145d10 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7f53e0002f30 tx=0x7f53e0006e80).ready entity=mon.0 client_cookie=9f73c65b55fa980d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:59.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.478+0000 7f53e67fc700 1 -- 172.21.15.32:0/510472096 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53e00097f0 con 0x7f53e8147400 2020-12-05T17:40:59.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.478+0000 7f53e67fc700 1 -- 172.21.15.32:0/510472096 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e0009950 con 0x7f53e8147400 2020-12-05T17:40:59.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.478+0000 7f53e67fc700 1 -- 172.21.15.32:0/510472096 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53e0009cb0 con 0x7f53e8147400 2020-12-05T17:40:59.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.478+0000 7f53eca24700 1 -- 172.21.15.32:0/510472096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e8147400 msgr2=0x7f53e8145d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:59.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.478+0000 7f53eca24700 1 --2- 172.21.15.32:0/510472096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e8147400 0x7f53e8145d10 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7f53e0002f30 tx=0x7f53e0006e80).stop 2020-12-05T17:40:59.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.478+0000 7f53eca24700 1 -- 172.21.15.32:0/510472096 shutdown_connections 2020-12-05T17:40:59.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.478+0000 7f53eca24700 1 --2- 172.21.15.32:0/510472096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e8147400 0x7f53e8145d10 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:59.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.478+0000 7f53eca24700 1 --2- 172.21.15.32:0/510472096 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53e80b1fc0 0x7f53e80af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:59.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.478+0000 7f53eca24700 1 -- 172.21.15.32:0/510472096 >> 172.21.15.32:0/510472096 conn(0x7f53e80b5530 msgr2=0x7f53e80b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:59.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.478+0000 7f53eca24700 1 -- 172.21.15.32:0/510472096 shutdown_connections 2020-12-05T17:40:59.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.478+0000 7f53eca24700 1 -- 172.21.15.32:0/510472096 wait complete. 2020-12-05T17:40:59.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.478+0000 7f53eca24700 1 Processor -- start 2020-12-05T17:40:59.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.478+0000 7f53eca24700 1 -- start start 2020-12-05T17:40:59.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.482+0000 7f53eca24700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53e80ab710 0x7f53e80ad530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:59.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.482+0000 7f53eca24700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53e80b1fc0 0x7f53e80a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:59.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.482+0000 7f53eca24700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e8147400 0x7f53e80a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:59.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.482+0000 7f53eca24700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e8142b20 con 0x7f53e8147400 2020-12-05T17:40:59.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.482+0000 7f53eca24700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e809ecb0 con 0x7f53e80b1fc0 2020-12-05T17:40:59.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.482+0000 7f53eca24700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e809ef50 con 0x7f53e80ab710 2020-12-05T17:40:59.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.482+0000 7f53e7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e8147400 0x7f53e80a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:59.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.482+0000 7f53e7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e8147400 0x7f53e80a1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40058/0 (socket says 172.21.15.32:40058) 2020-12-05T17:40:59.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.482+0000 7f53e77fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53e80ab710 0x7f53e80ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:59.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.482+0000 7f53e7fff700 1 -- 172.21.15.32:0/29207304 learned_addr learned my addr 172.21.15.32:0/29207304 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.482+0000 7f53e6ffd700 1 --2- 172.21.15.32:0/29207304 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53e80b1fc0 0x7f53e80a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.482+0000 7f53e7fff700 1 -- 172.21.15.32:0/29207304 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53e80ab710 msgr2=0x7f53e80ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.482+0000 7f53e7fff700 1 --2- 172.21.15.32:0/29207304 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53e80ab710 0x7f53e80ad530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.482+0000 7f53e7fff700 1 -- 172.21.15.32:0/29207304 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53e80b1fc0 msgr2=0x7f53e80a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.482+0000 7f53e7fff700 1 --2- 172.21.15.32:0/29207304 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53e80b1fc0 0x7f53e80a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.482+0000 7f53e7fff700 1 -- 172.21.15.32:0/29207304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e80aca50 con 0x7f53e8147400 2020-12-05T17:40:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.482+0000 7f53e7fff700 1 --2- 172.21.15.32:0/29207304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e8147400 0x7f53e80a1a20 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 rx=0x7f53e0002f30 tx=0x7f53e0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.482+0000 7f53e4ff9700 1 -- 172.21.15.32:0/29207304 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53e0011bd0 con 0x7f53e8147400 2020-12-05T17:40:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.482+0000 7f53e4ff9700 1 -- 172.21.15.32:0/29207304 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e0011d30 con 0x7f53e8147400 2020-12-05T17:40:59.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.482+0000 7f53eca24700 1 -- 172.21.15.32:0/29207304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e80a1f10 con 0x7f53e8147400 2020-12-05T17:40:59.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.482+0000 7f53e4ff9700 1 -- 172.21.15.32:0/29207304 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53e0011ea0 con 0x7f53e8147400 2020-12-05T17:40:59.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.482+0000 7f53eca24700 1 -- 172.21.15.32:0/29207304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e80acef0 con 0x7f53e8147400 2020-12-05T17:40:59.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.486+0000 7f53eca24700 1 -- 172.21.15.32:0/29207304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e80acbe0 con 0x7f53e8147400 2020-12-05T17:40:59.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.486+0000 7f53e4ff9700 1 -- 172.21.15.32:0/29207304 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f53e0025030 con 0x7f53e8147400 2020-12-05T17:40:59.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.486+0000 7f53e4ff9700 1 --2- 172.21.15.32:0/29207304 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f53c8055500 0x7f53c80579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:59.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.486+0000 7f53e4ff9700 1 -- 172.21.15.32:0/29207304 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f53e006f110 con 0x7f53e8147400 2020-12-05T17:40:59.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.486+0000 7f53e77fe700 1 --2- 172.21.15.32:0/29207304 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f53c8055500 0x7f53c80579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:59.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.486+0000 7f53e77fe700 1 --2- 172.21.15.32:0/29207304 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f53c8055500 0x7f53c80579c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f53dc00c000 tx=0x7f53dc008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:59.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.494+0000 7f53e4ff9700 1 -- 172.21.15.32:0/29207304 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f53e00455f0 con 0x7f53e8147400 2020-12-05T17:40:59.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.682+0000 7f53eca24700 1 -- 172.21.15.32:0/29207304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f53e8057050 con 0x7f53e8147400 2020-12-05T17:40:59.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.682+0000 7f53e4ff9700 1 -- 172.21.15.32:0/29207304 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v37) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f53e0023020 con 0x7f53e8147400 2020-12-05T17:40:59.688 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:40:59.688 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:40:59.688 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2020-12-05T17:40:59.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:59.689 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2020-12-05T17:40:59.689 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfpb/aExAAdMKW4yIgC2zhkzyW6l8xug==", 2020-12-05T17:40:59.689 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:59.690 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:40:59.690 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:40:59.690 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:40:59.690 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:59.690 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:59.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:59.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:59.691 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2020-12-05T17:40:59.691 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfBKTYGRAAaPkezwmTiAvkUfTuDGDNZA==", 2020-12-05T17:40:59.691 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:59.692 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:40:59.692 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:40:59.692 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:40:59.692 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:59.692 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:59.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:59.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:59.693 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2020-12-05T17:40:59.693 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHw8tflo5EKhAAwV9D64fAoY3uvXCm+Aksag==", 2020-12-05T17:40:59.693 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:59.694 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:40:59.694 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:40:59.694 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:59.694 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:59.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:59.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:59.696 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2020-12-05T17:40:59.696 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDKw8tfTjEiERAAkW6mkgApoXwyF49kPjZBGg==", 2020-12-05T17:40:59.696 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:59.696 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:40:59.696 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:40:59.697 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:59.697 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:59.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:59.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:59.697 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2020-12-05T17:40:59.698 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDMw8tfLEzBMxAAiJ2NifsOLEkt1e3vmhwSTA==", 2020-12-05T17:40:59.698 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:59.698 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:40:59.698 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:40:59.698 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:59.699 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:59.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:59.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:59.699 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2020-12-05T17:40:59.699 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPw8tfV+vnGxAAQHvXJRW5X9Vhe/eYJBJ/uQ==", 2020-12-05T17:40:59.700 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:59.700 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:40:59.700 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:40:59.700 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:59.700 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:59.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:59.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:59.701 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2020-12-05T17:40:59.701 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUw8tfN8wMJxAApc5LY9HGDeIV/o0eil/g4A==", 2020-12-05T17:40:59.701 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:59.702 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:40:59.702 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:40:59.702 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:59.702 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:59.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:59.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:59.703 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2020-12-05T17:40:59.703 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXw8tfvZ5dDxAAtAfftExkljlBvn7ZSoD/Og==", 2020-12-05T17:40:59.703 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:59.704 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:40:59.704 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:40:59.704 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:59.704 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:59.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:59.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:59.705 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2020-12-05T17:40:59.705 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDZw8tfhXFzMhAAZXNwNGfKv360Tq4N/DX5Mw==", 2020-12-05T17:40:59.705 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:59.706 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:40:59.706 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:40:59.706 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:59.706 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:59.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:59.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:59.707 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2020-12-05T17:40:59.707 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDcw8tfROfSGhAA/Y2Y1b4k5zQju0MM7LASeQ==", 2020-12-05T17:40:59.707 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:59.708 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:40:59.708 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:40:59.708 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:59.708 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:59.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:59.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:59.709 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2020-12-05T17:40:59.709 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfWDp/HxAAmvLPj7C3WDgj0OC2o7lBJQ==", 2020-12-05T17:40:59.709 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:59.709 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:40:59.710 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:40:59.710 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:40:59.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:59.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:59.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:59.711 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2020-12-05T17:40:59.711 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfugnmIRAAW6+/QGMLkHw+Z97RHpV4kw==", 2020-12-05T17:40:59.711 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:59.711 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:40:59.712 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:40:59.712 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:40:59.712 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:59.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:59.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:59.713 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2020-12-05T17:40:59.713 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfojyqJBAAyta+dSy2yGnC1kKe3YOAYw==", 2020-12-05T17:40:59.713 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:59.713 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:40:59.714 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:40:59.715 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:40:59.715 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:59.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:59.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:59.715 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2020-12-05T17:40:59.716 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDFw8tfES0PLBAAA1cUV44snAUDF2TyAmXImw==", 2020-12-05T17:40:59.716 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:59.716 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:40:59.716 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:40:59.716 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:40:59.717 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:59.717 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:59.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:59.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:59.717 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2020-12-05T17:40:59.718 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfDr8mERAAS7bpAQXgW78W8GDq1v1zaA==", 2020-12-05T17:40:59.718 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:59.718 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:40:59.718 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:40:59.719 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:59.719 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:59.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:40:59.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:40:59.719 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2020-12-05T17:40:59.720 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tf3HmcDhAATNKHhH9zgasedaNEzgG0OQ==", 2020-12-05T17:40:59.720 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:40:59.720 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:40:59.720 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:40:59.721 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:40:59.721 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:59.721 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:40:59.721 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:40:59.721 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:40:59.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.686+0000 7f53eca24700 1 -- 172.21.15.32:0/29207304 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f53c8055500 msgr2=0x7f53c80579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:59.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.686+0000 7f53eca24700 1 --2- 172.21.15.32:0/29207304 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f53c8055500 0x7f53c80579c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f53dc00c000 tx=0x7f53dc008040).stop 2020-12-05T17:40:59.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.686+0000 7f53eca24700 1 -- 172.21.15.32:0/29207304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e8147400 msgr2=0x7f53e80a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:59.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.686+0000 7f53eca24700 1 --2- 172.21.15.32:0/29207304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e8147400 0x7f53e80a1a20 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 rx=0x7f53e0002f30 tx=0x7f53e0015000).stop 2020-12-05T17:40:59.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.686+0000 7f53eca24700 1 -- 172.21.15.32:0/29207304 shutdown_connections 2020-12-05T17:40:59.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.686+0000 7f53eca24700 1 --2- 172.21.15.32:0/29207304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e8147400 0x7f53e80a1a20 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:59.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.686+0000 7f53eca24700 1 --2- 172.21.15.32:0/29207304 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f53c8055500 0x7f53c80579c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:59.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.686+0000 7f53eca24700 1 --2- 172.21.15.32:0/29207304 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53e80ab710 0x7f53e80ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:59.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.686+0000 7f53eca24700 1 --2- 172.21.15.32:0/29207304 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53e80b1fc0 0x7f53e80a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:59.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.686+0000 7f53eca24700 1 -- 172.21.15.32:0/29207304 >> 172.21.15.32:0/29207304 conn(0x7f53e80b5530 msgr2=0x7f53e80af740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:59.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.686+0000 7f53eca24700 1 -- 172.21.15.32:0/29207304 shutdown_connections 2020-12-05T17:40:59.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.686+0000 7f53eca24700 1 -- 172.21.15.32:0/29207304 wait complete. 2020-12-05T17:40:59.726 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:40:59.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.838+0000 7f2b4ffff700 1 Processor -- start 2020-12-05T17:40:59.845 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.838+0000 7f2b4ffff700 1 -- start start 2020-12-05T17:40:59.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.842+0000 7f2b4ffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2b500b52f0 0x7f2b500b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:59.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.842+0000 7f2b4ffff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f2b500b1870 0x7f2b50138810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:59.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.842+0000 7f2b4ffff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b501420f0 con 0x7f2b50138d00 2020-12-05T17:40:59.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.842+0000 7f2b4ffff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b500ad4f0 con 0x7f2b500b1870 2020-12-05T17:40:59.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.842+0000 7f2b4ffff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b500b0380 con 0x7f2b500b52f0 2020-12-05T17:40:59.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.842+0000 7f2b4effd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2b500b52f0 0x7f2b500b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:59.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.842+0000 7f2b4e7fc700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f2b500b1870 0x7f2b50138810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:59.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.842+0000 7f2b4effd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2b500b52f0 0x7f2b500b56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:36852/0 (socket says 172.21.15.32:36852) 2020-12-05T17:40:59.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.842+0000 7f2b4e7fc700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f2b500b1870 0x7f2b50138810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:59956/0 (socket says 172.21.15.32:59956) 2020-12-05T17:40:59.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.842+0000 7f2b4effd700 1 -- 172.21.15.32:0/2618202584 learned_addr learned my addr 172.21.15.32:0/2618202584 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:59.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.842+0000 7f2b4f7fe700 1 --1- 172.21.15.32:0/2618202584 >> v1:172.21.15.61:6789/0 conn(0x7f2b50138d00 0x7f2b500ae860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:55354/0 (socket says 172.21.15.32:55354) 2020-12-05T17:40:59.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.842+0000 7f2b4effd700 1 -- 172.21.15.32:0/2618202584 >> v2:172.21.15.61:3300/0 conn(0x7f2b500b1870 msgr2=0x7f2b50138810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:40:59.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.842+0000 7f2b4effd700 1 --2- 172.21.15.32:0/2618202584 >> v2:172.21.15.61:3300/0 conn(0x7f2b500b1870 0x7f2b50138810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:59.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.842+0000 7f2b4effd700 1 -- 172.21.15.32:0/2618202584 >> v1:172.21.15.61:6789/0 conn(0x7f2b50138d00 legacy=0x7f2b500ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:59.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.842+0000 7f2b4effd700 1 -- 172.21.15.32:0/2618202584 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b500b0620 con 0x7f2b500b52f0 2020-12-05T17:40:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.842+0000 7f2b4effd700 1 --2- 172.21.15.32:0/2618202584 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2b500b52f0 0x7f2b500b56d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f2b500adc80 tx=0x7f2b40009270).ready entity=mon.2 client_cookie=b890687c3f38b3ff server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.842+0000 7f2b4dffb700 1 -- 172.21.15.32:0/2618202584 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b4000bdb0 con 0x7f2b500b52f0 2020-12-05T17:40:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.842+0000 7f2b4dffb700 1 -- 172.21.15.32:0/2618202584 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b40009e90 con 0x7f2b500b52f0 2020-12-05T17:40:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.842+0000 7f2b4dffb700 1 -- 172.21.15.32:0/2618202584 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b40010600 con 0x7f2b500b52f0 2020-12-05T17:40:59.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.842+0000 7f2b4ffff700 1 -- 172.21.15.32:0/2618202584 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2b500b52f0 msgr2=0x7f2b500b56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:59.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.842+0000 7f2b4ffff700 1 --2- 172.21.15.32:0/2618202584 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2b500b52f0 0x7f2b500b56d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f2b500adc80 tx=0x7f2b40009270).stop 2020-12-05T17:40:59.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4ffff700 1 -- 172.21.15.32:0/2618202584 shutdown_connections 2020-12-05T17:40:59.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4ffff700 1 --2- 172.21.15.32:0/2618202584 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2b500b52f0 0x7f2b500b56d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:59.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4ffff700 1 --2- 172.21.15.32:0/2618202584 >> v2:172.21.15.61:3300/0 conn(0x7f2b500b1870 0x7f2b50138810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:59.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4ffff700 1 -- 172.21.15.32:0/2618202584 >> 172.21.15.32:0/2618202584 conn(0x7f2b5005bbb0 msgr2=0x7f2b50062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:40:59.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4ffff700 1 -- 172.21.15.32:0/2618202584 shutdown_connections 2020-12-05T17:40:59.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4ffff700 1 -- 172.21.15.32:0/2618202584 wait complete. 2020-12-05T17:40:59.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4ffff700 1 Processor -- start 2020-12-05T17:40:59.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4ffff700 1 -- start start 2020-12-05T17:40:59.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b500b1870 0x7f2b5014d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:59.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4ffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2b500b52f0 0x7f2b50140330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:59.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4ffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2b50138d00 0x7f2b5013da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:59.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4ffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5013cc70 con 0x7f2b500b1870 2020-12-05T17:40:59.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4ffff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5013cf10 con 0x7f2b50138d00 2020-12-05T17:40:59.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4ffff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5013d1b0 con 0x7f2b500b52f0 2020-12-05T17:40:59.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b500b1870 0x7f2b5014d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:59.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4e7fc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2b500b52f0 0x7f2b50140330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:59.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b500b1870 0x7f2b5014d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40070/0 (socket says 172.21.15.32:40070) 2020-12-05T17:40:59.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4effd700 1 -- 172.21.15.32:0/1378612877 learned_addr learned my addr 172.21.15.32:0/1378612877 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:40:59.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4f7fe700 1 --2- 172.21.15.32:0/1378612877 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2b50138d00 0x7f2b5013da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:59.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4effd700 1 -- 172.21.15.32:0/1378612877 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2b500b52f0 msgr2=0x7f2b50140330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4effd700 1 --2- 172.21.15.32:0/1378612877 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2b500b52f0 0x7f2b50140330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4effd700 1 -- 172.21.15.32:0/1378612877 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2b50138d00 msgr2=0x7f2b5013da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:40:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4effd700 1 --2- 172.21.15.32:0/1378612877 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2b50138d00 0x7f2b5013da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:40:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4effd700 1 -- 172.21.15.32:0/1378612877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5014e390 con 0x7f2b500b1870 2020-12-05T17:40:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4e7fc700 1 --2- 172.21.15.32:0/1378612877 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2b500b52f0 0x7f2b50140330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:40:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.846+0000 7f2b4effd700 1 --2- 172.21.15.32:0/1378612877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b500b1870 0x7f2b5014d250 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0x7f2b500adc80 tx=0x7f2b40017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.850+0000 7f2b47fff700 1 -- 172.21.15.32:0/1378612877 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b40019870 con 0x7f2b500b1870 2020-12-05T17:40:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.850+0000 7f2b47fff700 1 -- 172.21.15.32:0/1378612877 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b400106c0 con 0x7f2b500b1870 2020-12-05T17:40:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.850+0000 7f2b4ffff700 1 -- 172.21.15.32:0/1378612877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5013d650 con 0x7f2b500b1870 2020-12-05T17:40:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.850+0000 7f2b47fff700 1 -- 172.21.15.32:0/1378612877 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b40019bc0 con 0x7f2b500b1870 2020-12-05T17:40:59.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.850+0000 7f2b4ffff700 1 -- 172.21.15.32:0/1378612877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5013c910 con 0x7f2b500b1870 2020-12-05T17:40:59.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.850+0000 7f2b4ffff700 1 -- 172.21.15.32:0/1378612877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b50057050 con 0x7f2b500b1870 2020-12-05T17:40:59.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.850+0000 7f2b47fff700 1 -- 172.21.15.32:0/1378612877 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2b4002a070 con 0x7f2b500b1870 2020-12-05T17:40:59.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.850+0000 7f2b47fff700 1 --2- 172.21.15.32:0/1378612877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2b34055350 0x7f2b34057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:40:59.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.850+0000 7f2b47fff700 1 -- 172.21.15.32:0/1378612877 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f2b4006f8e0 con 0x7f2b500b1870 2020-12-05T17:40:59.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.850+0000 7f2b4e7fc700 1 --2- 172.21.15.32:0/1378612877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2b34055350 0x7f2b34057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:40:59.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.850+0000 7f2b4e7fc700 1 --2- 172.21.15.32:0/1378612877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2b34055350 0x7f2b34057810 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f2b3800b000 tx=0x7f2b38006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:40:59.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:40:59.858+0000 7f2b47fff700 1 -- 172.21.15.32:0/1378612877 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2b40045dc0 con 0x7f2b500b1870 2020-12-05T17:41:00.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.002+0000 7f2b4ffff700 1 -- 172.21.15.32:0/1378612877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b5005c300 con 0x7f2b500b1870 2020-12-05T17:41:00.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.006+0000 7f2b47fff700 1 -- 172.21.15.32:0/1378612877 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+13424 (secure 0 0 0) 0x7f2b400493e0 con 0x7f2b500b1870 2020-12-05T17:41:00.010 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:00.010 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":125,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:40:59.328582+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:41:00.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.006+0000 7f2b4ffff700 1 -- 172.21.15.32:0/1378612877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2b34055350 msgr2=0x7f2b34057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:00.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.006+0000 7f2b4ffff700 1 --2- 172.21.15.32:0/1378612877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2b34055350 0x7f2b34057810 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f2b3800b000 tx=0x7f2b38006dc0).stop 2020-12-05T17:41:00.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.006+0000 7f2b4ffff700 1 -- 172.21.15.32:0/1378612877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b500b1870 msgr2=0x7f2b5014d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:00.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.006+0000 7f2b4ffff700 1 --2- 172.21.15.32:0/1378612877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b500b1870 0x7f2b5014d250 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0x7f2b500adc80 tx=0x7f2b40017000).stop 2020-12-05T17:41:00.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.006+0000 7f2b4ffff700 1 -- 172.21.15.32:0/1378612877 shutdown_connections 2020-12-05T17:41:00.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.006+0000 7f2b4ffff700 1 --2- 172.21.15.32:0/1378612877 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2b50138d00 0x7f2b5013da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.006+0000 7f2b4ffff700 1 --2- 172.21.15.32:0/1378612877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b500b1870 0x7f2b5014d250 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.006+0000 7f2b4ffff700 1 --2- 172.21.15.32:0/1378612877 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2b34055350 0x7f2b34057810 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.006+0000 7f2b4ffff700 1 --2- 172.21.15.32:0/1378612877 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2b500b52f0 0x7f2b50140330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.006+0000 7f2b4ffff700 1 -- 172.21.15.32:0/1378612877 >> 172.21.15.32:0/1378612877 conn(0x7f2b5005bbb0 msgr2=0x7f2b500aef10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:00.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.006+0000 7f2b4ffff700 1 -- 172.21.15.32:0/1378612877 shutdown_connections 2020-12-05T17:41:00.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.006+0000 7f2b4ffff700 1 -- 172.21.15.32:0/1378612877 wait complete. 2020-12-05T17:41:00.026 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-12-05T17:41:00.027 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:41:00.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.134+0000 7fd640e92700 1 Processor -- start 2020-12-05T17:41:00.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.134+0000 7fd640e92700 1 -- start start 2020-12-05T17:41:00.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd640e92700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63c0af4b0 0x7fd63c146510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:00.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd640e92700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63c0b23e0 con 0x7fd63c0b2000 2020-12-05T17:41:00.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd640e92700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63c0ae7b0 con 0x7fd63c061780 2020-12-05T17:41:00.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd640e92700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c146a00 con 0x7fd63c0af4b0 2020-12-05T17:41:00.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd63c0b2000 0x7fd63c145e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42724/0 (socket says 172.21.15.32:42724) 2020-12-05T17:41:00.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63b7fe700 1 -- 172.21.15.32:0/1028156843 learned_addr learned my addr 172.21.15.32:0/1028156843 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:00.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63affd700 1 --2- 172.21.15.32:0/1028156843 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63c0af4b0 0x7fd63c146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:00.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63a7fc700 1 -- 172.21.15.32:0/1028156843 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3740163261 0 0) 0x7fd63c0b23e0 con 0x7fd63c0b2000 2020-12-05T17:41:00.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63a7fc700 1 -- 172.21.15.32:0/1028156843 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd61c003230 con 0x7fd63c0b2000 2020-12-05T17:41:00.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63a7fc700 1 -- 172.21.15.32:0/1028156843 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3060878343 0 0) 0x7fd63c0ae7b0 con 0x7fd63c061780 2020-12-05T17:41:00.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63a7fc700 1 -- 172.21.15.32:0/1028156843 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd63c0b23e0 con 0x7fd63c061780 2020-12-05T17:41:00.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63a7fc700 1 -- 172.21.15.32:0/1028156843 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3316431215 0 0) 0x7fd61c003230 con 0x7fd63c0b2000 2020-12-05T17:41:00.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63a7fc700 1 -- 172.21.15.32:0/1028156843 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63c0af4b0 msgr2=0x7fd63c146510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:00.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63a7fc700 1 --2- 172.21.15.32:0/1028156843 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63c0af4b0 0x7fd63c146510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63a7fc700 1 -- 172.21.15.32:0/1028156843 >> v1:172.21.15.61:6789/0 conn(0x7fd63c061780 legacy=0x7fd63c0afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:00.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63a7fc700 1 -- 172.21.15.32:0/1028156843 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c0aea50 con 0x7fd63c0b2000 2020-12-05T17:41:00.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63a7fc700 1 -- 172.21.15.32:0/1028156843 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fd62c0016d0 con 0x7fd63c0b2000 2020-12-05T17:41:00.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63a7fc700 1 -- 172.21.15.32:0/1028156843 >> v1:172.21.15.32:6789/0 conn(0x7fd63c0b2000 legacy=0x7fd63c145e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:00.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63a7fc700 1 --2- 172.21.15.32:0/1028156843 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63c0af4b0 0x7fd61c0038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:00.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63a7fc700 1 --2- 172.21.15.32:0/1028156843 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd61c003dd0 0x7fd61c0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:00.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63a7fc700 1 --2- 172.21.15.32:0/1028156843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd61c0049c0 0x7fd61c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:00.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63a7fc700 1 -- 172.21.15.32:0/1028156843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c146a00 con 0x7fd61c0049c0 2020-12-05T17:41:00.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63a7fc700 1 -- 172.21.15.32:0/1028156843 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c005360 con 0x7fd61c003dd0 2020-12-05T17:41:00.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63a7fc700 1 -- 172.21.15.32:0/1028156843 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c00b770 con 0x7fd63c0af4b0 2020-12-05T17:41:00.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63bfff700 1 --2- 172.21.15.32:0/1028156843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd61c0049c0 0x7fd61c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:00.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63b7fe700 1 --2- 172.21.15.32:0/1028156843 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63c0af4b0 0x7fd61c0038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:00.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63affd700 1 --2- 172.21.15.32:0/1028156843 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd61c003dd0 0x7fd61c0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:00.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63bfff700 1 -- 172.21.15.32:0/1028156843 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63c0af4b0 msgr2=0x7fd61c0038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:00.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63bfff700 1 --2- 172.21.15.32:0/1028156843 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63c0af4b0 0x7fd61c0038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63bfff700 1 -- 172.21.15.32:0/1028156843 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd61c003dd0 msgr2=0x7fd61c0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:00.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63bfff700 1 --2- 172.21.15.32:0/1028156843 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd61c003dd0 0x7fd61c0044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63bfff700 1 -- 172.21.15.32:0/1028156843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61c003230 con 0x7fd61c0049c0 2020-12-05T17:41:00.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63b7fe700 1 --2- 172.21.15.32:0/1028156843 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63c0af4b0 0x7fd61c0038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:00.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63affd700 1 --2- 172.21.15.32:0/1028156843 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd61c003dd0 0x7fd61c0044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:00.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63bfff700 1 --2- 172.21.15.32:0/1028156843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd61c0049c0 0x7fd61c004e40 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7fd61c0054a0 tx=0x7fd634007b60).ready entity=mon.0 client_cookie=c4b6c15961c7c8fe server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:00.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63a7fc700 1 -- 172.21.15.32:0/1028156843 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd63400c830 con 0x7fd61c0049c0 2020-12-05T17:41:00.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63a7fc700 1 -- 172.21.15.32:0/1028156843 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63400c990 con 0x7fd61c0049c0 2020-12-05T17:41:00.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63a7fc700 1 -- 172.21.15.32:0/1028156843 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd63400ccf0 con 0x7fd61c0049c0 2020-12-05T17:41:00.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd640e92700 1 -- 172.21.15.32:0/1028156843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd61c0049c0 msgr2=0x7fd61c004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:00.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd640e92700 1 --2- 172.21.15.32:0/1028156843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd61c0049c0 0x7fd61c004e40 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7fd61c0054a0 tx=0x7fd634007b60).stop 2020-12-05T17:41:00.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd63bfff700 1 -- 172.21.15.32:0/1028156843 reap_dead start 2020-12-05T17:41:00.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd640e92700 1 -- 172.21.15.32:0/1028156843 shutdown_connections 2020-12-05T17:41:00.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd640e92700 1 -- 172.21.15.32:0/1028156843 >> 172.21.15.32:0/1028156843 conn(0x7fd63c0b5620 msgr2=0x7fd63c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:00.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd640e92700 1 -- 172.21.15.32:0/1028156843 shutdown_connections 2020-12-05T17:41:00.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.138+0000 7fd640e92700 1 -- 172.21.15.32:0/1028156843 wait complete. 2020-12-05T17:41:00.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd640e92700 1 Processor -- start 2020-12-05T17:41:00.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd640e92700 1 -- start start 2020-12-05T17:41:00.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd640e92700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd63c1424d0 0x7fd63c0667f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:00.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd640e92700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63c064e80 0x7fd63c140fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:00.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd640e92700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd63c1414d0 0x7fd63c159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:00.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd640e92700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c065300 con 0x7fd63c1414d0 2020-12-05T17:41:00.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd640e92700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c148ea0 con 0x7fd63c1424d0 2020-12-05T17:41:00.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd640e92700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c1428b0 con 0x7fd63c064e80 2020-12-05T17:41:00.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd63bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd63c1414d0 0x7fd63c159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:00.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd63bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd63c1414d0 0x7fd63c159160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40092/0 (socket says 172.21.15.32:40092) 2020-12-05T17:41:00.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd63bfff700 1 -- 172.21.15.32:0/2337345392 learned_addr learned my addr 172.21.15.32:0/2337345392 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:00.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd63affd700 1 --2- 172.21.15.32:0/2337345392 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63c064e80 0x7fd63c140fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:00.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd63b7fe700 1 --2- 172.21.15.32:0/2337345392 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd63c1424d0 0x7fd63c0667f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:00.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd63bfff700 1 -- 172.21.15.32:0/2337345392 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63c064e80 msgr2=0x7fd63c140fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:00.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd63bfff700 1 --2- 172.21.15.32:0/2337345392 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63c064e80 0x7fd63c140fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd63bfff700 1 -- 172.21.15.32:0/2337345392 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd63c1424d0 msgr2=0x7fd63c0667f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:00.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd63bfff700 1 --2- 172.21.15.32:0/2337345392 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd63c1424d0 0x7fd63c0667f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd63bfff700 1 -- 172.21.15.32:0/2337345392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c066d90 con 0x7fd63c1414d0 2020-12-05T17:41:00.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd63affd700 1 --2- 172.21.15.32:0/2337345392 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63c064e80 0x7fd63c140fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:00.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd63b7fe700 1 --2- 172.21.15.32:0/2337345392 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd63c1424d0 0x7fd63c0667f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:00.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd63bfff700 1 --2- 172.21.15.32:0/2337345392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd63c1414d0 0x7fd63c159160 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7fd63c148d80 tx=0x7fd634017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:00.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd638ff9700 1 -- 172.21.15.32:0/2337345392 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd634007220 con 0x7fd63c1414d0 2020-12-05T17:41:00.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd640e92700 1 -- 172.21.15.32:0/2337345392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c148760 con 0x7fd63c1414d0 2020-12-05T17:41:00.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd638ff9700 1 -- 172.21.15.32:0/2337345392 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd634007380 con 0x7fd63c1414d0 2020-12-05T17:41:00.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd638ff9700 1 -- 172.21.15.32:0/2337345392 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6340066d0 con 0x7fd63c1414d0 2020-12-05T17:41:00.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.142+0000 7fd640e92700 1 -- 172.21.15.32:0/2337345392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c141950 con 0x7fd63c1414d0 2020-12-05T17:41:00.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.146+0000 7fd640e92700 1 -- 172.21.15.32:0/2337345392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd63c141ba0 con 0x7fd63c1414d0 2020-12-05T17:41:00.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.146+0000 7fd638ff9700 1 -- 172.21.15.32:0/2337345392 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd634010070 con 0x7fd63c1414d0 2020-12-05T17:41:00.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.146+0000 7fd638ff9700 1 --2- 172.21.15.32:0/2337345392 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd620055330 0x7fd6200577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:00.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.146+0000 7fd638ff9700 1 -- 172.21.15.32:0/2337345392 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fd63406e750 con 0x7fd63c1414d0 2020-12-05T17:41:00.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.146+0000 7fd63b7fe700 1 --2- 172.21.15.32:0/2337345392 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd620055330 0x7fd6200577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:00.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.146+0000 7fd63b7fe700 1 --2- 172.21.15.32:0/2337345392 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd620055330 0x7fd6200577f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fd62c00d000 tx=0x7fd62c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:00.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.154+0000 7fd638ff9700 1 -- 172.21.15.32:0/2337345392 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd634044b90 con 0x7fd63c1414d0 2020-12-05T17:41:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.294+0000 7fd640e92700 1 -- 172.21.15.32:0/2337345392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd63c057050 con 0x7fd63c1414d0 2020-12-05T17:41:00.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.294+0000 7fd638ff9700 1 -- 172.21.15.32:0/2337345392 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v125) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd6340481b0 con 0x7fd63c1414d0 2020-12-05T17:41:00.301 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:41:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.298+0000 7fd640e92700 1 -- 172.21.15.32:0/2337345392 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd620055330 msgr2=0x7fd6200577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.298+0000 7fd640e92700 1 --2- 172.21.15.32:0/2337345392 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd620055330 0x7fd6200577f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fd62c00d000 tx=0x7fd62c009040).stop 2020-12-05T17:41:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.298+0000 7fd640e92700 1 -- 172.21.15.32:0/2337345392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd63c1414d0 msgr2=0x7fd63c159160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.298+0000 7fd640e92700 1 --2- 172.21.15.32:0/2337345392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd63c1414d0 0x7fd63c159160 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7fd63c148d80 tx=0x7fd634017000).stop 2020-12-05T17:41:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.298+0000 7fd640e92700 1 -- 172.21.15.32:0/2337345392 shutdown_connections 2020-12-05T17:41:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.298+0000 7fd640e92700 1 --2- 172.21.15.32:0/2337345392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd63c1414d0 0x7fd63c159160 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.298+0000 7fd640e92700 1 --2- 172.21.15.32:0/2337345392 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd63c1424d0 0x7fd63c0667f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.298+0000 7fd640e92700 1 --2- 172.21.15.32:0/2337345392 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd620055330 0x7fd6200577f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.298+0000 7fd640e92700 1 --2- 172.21.15.32:0/2337345392 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63c064e80 0x7fd63c140fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.298+0000 7fd640e92700 1 -- 172.21.15.32:0/2337345392 >> 172.21.15.32:0/2337345392 conn(0x7fd63c0b5620 msgr2=0x7fd63c0ada60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.298+0000 7fd640e92700 1 -- 172.21.15.32:0/2337345392 shutdown_connections 2020-12-05T17:41:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.298+0000 7fd640e92700 1 -- 172.21.15.32:0/2337345392 wait complete. 2020-12-05T17:41:00.315 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:41:00.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5f1cb700 1 Processor -- start 2020-12-05T17:41:00.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5f1cb700 1 -- start start 2020-12-05T17:41:00.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5f1cb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3f58061920 0x7f3f580aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:00.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5f1cb700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f3f58061d00 0x7f3f58142200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:00.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5f1cb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f580b1380 con 0x7f3f58062180 2020-12-05T17:41:00.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5f1cb700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f58062ba0 con 0x7f3f58061d00 2020-12-05T17:41:00.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5f1cb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f580b23d0 con 0x7f3f58061920 2020-12-05T17:41:00.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5e9ca700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3f58062180 0x7f3f580b1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42748/0 (socket says 172.21.15.32:42748) 2020-12-05T17:41:00.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5e9ca700 1 -- 172.21.15.32:0/2053192062 learned_addr learned my addr 172.21.15.32:0/2053192062 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:00.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5e1c9700 1 --2- 172.21.15.32:0/2053192062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3f58061920 0x7f3f580aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:00.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5d9c8700 1 --2- 172.21.15.32:0/2053192062 >> v2:172.21.15.61:3300/0 conn(0x7f3f58061d00 0x7f3f58142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:00.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5d1c7700 1 -- 172.21.15.32:0/2053192062 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568702220 0 0) 0x7f3f580b1380 con 0x7f3f58062180 2020-12-05T17:41:00.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5d1c7700 1 -- 172.21.15.32:0/2053192062 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f40003230 con 0x7f3f58062180 2020-12-05T17:41:00.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5d9c8700 1 -- 172.21.15.32:0/2053192062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3f58061920 msgr2=0x7f3f580aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:00.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5d9c8700 1 --2- 172.21.15.32:0/2053192062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3f58061920 0x7f3f580aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.443 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5d9c8700 1 -- 172.21.15.32:0/2053192062 >> v1:172.21.15.32:6789/0 conn(0x7f3f58062180 legacy=0x7f3f580b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:00.443 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5d9c8700 1 -- 172.21.15.32:0/2053192062 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f580ae620 con 0x7f3f58061d00 2020-12-05T17:41:00.443 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5d9c8700 1 --2- 172.21.15.32:0/2053192062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3f58061d00 0x7f3f58142200 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f3f58062ce0 tx=0x7f3f48006dc0).ready entity=mon.1 client_cookie=6348e52b005a35d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:00.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5d1c7700 1 -- 172.21.15.32:0/2053192062 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f480096a0 con 0x7f3f58061d00 2020-12-05T17:41:00.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5d1c7700 1 -- 172.21.15.32:0/2053192062 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f48009800 con 0x7f3f58061d00 2020-12-05T17:41:00.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5d1c7700 1 -- 172.21.15.32:0/2053192062 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f48009b60 con 0x7f3f58061d00 2020-12-05T17:41:00.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5f1cb700 1 -- 172.21.15.32:0/2053192062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3f58061d00 msgr2=0x7f3f58142200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:00.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5f1cb700 1 --2- 172.21.15.32:0/2053192062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3f58061d00 0x7f3f58142200 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f3f58062ce0 tx=0x7f3f48006dc0).stop 2020-12-05T17:41:00.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5f1cb700 1 -- 172.21.15.32:0/2053192062 shutdown_connections 2020-12-05T17:41:00.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.434+0000 7f3f5f1cb700 1 --2- 172.21.15.32:0/2053192062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3f58061920 0x7f3f580aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5f1cb700 1 --2- 172.21.15.32:0/2053192062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3f58061d00 0x7f3f58142200 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5f1cb700 1 -- 172.21.15.32:0/2053192062 >> 172.21.15.32:0/2053192062 conn(0x7f3f580b5620 msgr2=0x7f3f580b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:00.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5f1cb700 1 -- 172.21.15.32:0/2053192062 shutdown_connections 2020-12-05T17:41:00.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5f1cb700 1 -- 172.21.15.32:0/2053192062 wait complete. 2020-12-05T17:41:00.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5f1cb700 1 Processor -- start 2020-12-05T17:41:00.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5f1cb700 1 -- start start 2020-12-05T17:41:00.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5f1cb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f58061920 0x7f3f5805fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:00.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5f1cb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3f58061d00 0x7f3f5813b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:00.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5f1cb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3f58062180 0x7f3f58141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:00.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5f1cb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f58063d90 con 0x7f3f58061920 2020-12-05T17:41:00.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5f1cb700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f58064030 con 0x7f3f58062180 2020-12-05T17:41:00.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5f1cb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f58064b90 con 0x7f3f58061d00 2020-12-05T17:41:00.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5e1c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f58061920 0x7f3f5805fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:00.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5e1c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f58061920 0x7f3f5805fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40102/0 (socket says 172.21.15.32:40102) 2020-12-05T17:41:00.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5e1c9700 1 -- 172.21.15.32:0/2152128022 learned_addr learned my addr 172.21.15.32:0/2152128022 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:00.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5d9c8700 1 --2- 172.21.15.32:0/2152128022 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3f58061d00 0x7f3f5813b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:00.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5e9ca700 1 --2- 172.21.15.32:0/2152128022 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3f58062180 0x7f3f58141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:00.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5e1c9700 1 -- 172.21.15.32:0/2152128022 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3f58061d00 msgr2=0x7f3f5813b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:00.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5e1c9700 1 --2- 172.21.15.32:0/2152128022 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3f58061d00 0x7f3f5813b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5e1c9700 1 -- 172.21.15.32:0/2152128022 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3f58062180 msgr2=0x7f3f58141720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:00.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5e1c9700 1 --2- 172.21.15.32:0/2152128022 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3f58062180 0x7f3f58141720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5e1c9700 1 -- 172.21.15.32:0/2152128022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5813b590 con 0x7f3f58061920 2020-12-05T17:41:00.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5e1c9700 1 --2- 172.21.15.32:0/2152128022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f58061920 0x7f3f5805fc00 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0x7f3f5400c000 tx=0x7f3f54006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:00.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f477fe700 1 -- 172.21.15.32:0/2152128022 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f5400ca10 con 0x7f3f58061920 2020-12-05T17:41:00.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f477fe700 1 -- 172.21.15.32:0/2152128022 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5400cb70 con 0x7f3f58061920 2020-12-05T17:41:00.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f5f1cb700 1 -- 172.21.15.32:0/2152128022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f58063a80 con 0x7f3f58061920 2020-12-05T17:41:00.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.438+0000 7f3f477fe700 1 -- 172.21.15.32:0/2152128022 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f540123f0 con 0x7f3f58061920 2020-12-05T17:41:00.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.442+0000 7f3f5f1cb700 1 -- 172.21.15.32:0/2152128022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f58141e30 con 0x7f3f58061920 2020-12-05T17:41:00.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.442+0000 7f3f5f1cb700 1 -- 172.21.15.32:0/2152128022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f58057050 con 0x7f3f58061920 2020-12-05T17:41:00.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.442+0000 7f3f477fe700 1 -- 172.21.15.32:0/2152128022 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3f54012550 con 0x7f3f58061920 2020-12-05T17:41:00.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.442+0000 7f3f477fe700 1 --2- 172.21.15.32:0/2152128022 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3f38055330 0x7f3f380577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:00.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.442+0000 7f3f477fe700 1 -- 172.21.15.32:0/2152128022 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f3f5406e060 con 0x7f3f58061920 2020-12-05T17:41:00.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.442+0000 7f3f5d9c8700 1 --2- 172.21.15.32:0/2152128022 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3f38055330 0x7f3f380577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:00.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.442+0000 7f3f5d9c8700 1 --2- 172.21.15.32:0/2152128022 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3f38055330 0x7f3f380577f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f3f4801b000 tx=0x7f3f48015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:00.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.450+0000 7f3f477fe700 1 -- 172.21.15.32:0/2152128022 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3f540444e0 con 0x7f3f58061920 2020-12-05T17:41:00.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.586+0000 7f3f5f1cb700 1 -- 172.21.15.32:0/2152128022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3f5805aab0 con 0x7f3f58061920 2020-12-05T17:41:00.593 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.586+0000 7f3f477fe700 1 -- 172.21.15.32:0/2152128022 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v125) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3f54047b00 con 0x7f3f58061920 2020-12-05T17:41:00.594 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:41:00.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.590+0000 7f3f5f1cb700 1 -- 172.21.15.32:0/2152128022 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3f38055330 msgr2=0x7f3f380577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:00.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.590+0000 7f3f5f1cb700 1 --2- 172.21.15.32:0/2152128022 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3f38055330 0x7f3f380577f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f3f4801b000 tx=0x7f3f48015000).stop 2020-12-05T17:41:00.595 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.590+0000 7f3f5f1cb700 1 -- 172.21.15.32:0/2152128022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f58061920 msgr2=0x7f3f5805fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:00.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.590+0000 7f3f5f1cb700 1 --2- 172.21.15.32:0/2152128022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f58061920 0x7f3f5805fc00 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0x7f3f5400c000 tx=0x7f3f54006dc0).stop 2020-12-05T17:41:00.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.590+0000 7f3f5f1cb700 1 -- 172.21.15.32:0/2152128022 shutdown_connections 2020-12-05T17:41:00.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.590+0000 7f3f5f1cb700 1 --2- 172.21.15.32:0/2152128022 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3f58062180 0x7f3f58141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.596 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.590+0000 7f3f5f1cb700 1 --2- 172.21.15.32:0/2152128022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f58061920 0x7f3f5805fc00 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.590+0000 7f3f5f1cb700 1 --2- 172.21.15.32:0/2152128022 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3f38055330 0x7f3f380577f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.590+0000 7f3f5f1cb700 1 --2- 172.21.15.32:0/2152128022 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3f58061d00 0x7f3f5813b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.590+0000 7f3f5f1cb700 1 -- 172.21.15.32:0/2152128022 >> 172.21.15.32:0/2152128022 conn(0x7f3f580b5620 msgr2=0x7f3f58142920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:00.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.590+0000 7f3f5f1cb700 1 -- 172.21.15.32:0/2152128022 shutdown_connections 2020-12-05T17:41:00.597 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.590+0000 7f3f5f1cb700 1 -- 172.21.15.32:0/2152128022 wait complete. 2020-12-05T17:41:00.607 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-12-05T17:41:00.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.714+0000 7fdb769b7700 1 Processor -- start 2020-12-05T17:41:00.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.714+0000 7fdb769b7700 1 -- start start 2020-12-05T17:41:00.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.714+0000 7fdb769b7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb700b1980 0x7fdb70146580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:00.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.714+0000 7fdb769b7700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fdb70146a70 0x7fdb70066070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:00.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.714+0000 7fdb769b7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb70143000 con 0x7fdb7005f6f0 2020-12-05T17:41:00.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.714+0000 7fdb769b7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb7005fad0 con 0x7fdb700b1980 2020-12-05T17:41:00.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.714+0000 7fdb769b7700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb700b2f80 con 0x7fdb70146a70 2020-12-05T17:41:00.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.714+0000 7fdb759b5700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdb7005f6f0 0x7fdb701428f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42758/0 (socket says 172.21.15.32:42758) 2020-12-05T17:41:00.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.714+0000 7fdb759b5700 1 -- 172.21.15.32:0/575527223 learned_addr learned my addr 172.21.15.32:0/575527223 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:00.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.714+0000 7fdb751b4700 1 --2- 172.21.15.32:0/575527223 >> v2:172.21.15.32:3300/0 conn(0x7fdb700b1980 0x7fdb70146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:00.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.714+0000 7fdb761b6700 1 --2- 172.21.15.32:0/575527223 >> v2:172.21.15.61:3300/0 conn(0x7fdb70146a70 0x7fdb70066070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:00.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.714+0000 7fdb749b3700 1 -- 172.21.15.32:0/575527223 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 500859811 0 0) 0x7fdb70143000 con 0x7fdb7005f6f0 2020-12-05T17:41:00.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.714+0000 7fdb749b3700 1 -- 172.21.15.32:0/575527223 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb58003230 con 0x7fdb7005f6f0 2020-12-05T17:41:00.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb749b3700 1 -- 172.21.15.32:0/575527223 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3834980057 0 0) 0x7fdb58003230 con 0x7fdb7005f6f0 2020-12-05T17:41:00.728 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb749b3700 1 -- 172.21.15.32:0/575527223 >> v2:172.21.15.61:3300/0 conn(0x7fdb70146a70 msgr2=0x7fdb70066070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:00.728 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb749b3700 1 --2- 172.21.15.32:0/575527223 >> v2:172.21.15.61:3300/0 conn(0x7fdb70146a70 0x7fdb70066070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.728 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb749b3700 1 -- 172.21.15.32:0/575527223 >> v2:172.21.15.32:3300/0 conn(0x7fdb700b1980 msgr2=0x7fdb70146580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:00.728 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb749b3700 1 --2- 172.21.15.32:0/575527223 >> v2:172.21.15.32:3300/0 conn(0x7fdb700b1980 0x7fdb70146580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.729 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb749b3700 1 -- 172.21.15.32:0/575527223 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb70145e40 con 0x7fdb7005f6f0 2020-12-05T17:41:00.729 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb749b3700 1 -- 172.21.15.32:0/575527223 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fdb68001a50 con 0x7fdb7005f6f0 2020-12-05T17:41:00.729 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb751b4700 1 --2- 172.21.15.32:0/575527223 >> v2:172.21.15.32:3300/0 conn(0x7fdb700b1980 0x7fdb70146580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:00.729 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb749b3700 1 -- 172.21.15.32:0/575527223 >> v1:172.21.15.32:6789/0 conn(0x7fdb7005f6f0 legacy=0x7fdb701428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:00.729 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb749b3700 1 --2- 172.21.15.32:0/575527223 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb580037f0 0x7fdb58003c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:00.730 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb749b3700 1 --2- 172.21.15.32:0/575527223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb580044d0 0x7fdb58004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:00.730 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb749b3700 1 --2- 172.21.15.32:0/575527223 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb58004e40 0x7fdb5800b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:00.730 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb749b3700 1 -- 172.21.15.32:0/575527223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7005fad0 con 0x7fdb580044d0 2020-12-05T17:41:00.730 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb749b3700 1 -- 172.21.15.32:0/575527223 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb700b2f80 con 0x7fdb580037f0 2020-12-05T17:41:00.731 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb749b3700 1 -- 172.21.15.32:0/575527223 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb58005370 con 0x7fdb58004e40 2020-12-05T17:41:00.731 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb751b4700 1 --2- 172.21.15.32:0/575527223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb580044d0 0x7fdb58004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:00.731 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb761b6700 1 --2- 172.21.15.32:0/575527223 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb58004e40 0x7fdb5800b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:00.731 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb759b5700 1 --2- 172.21.15.32:0/575527223 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb580037f0 0x7fdb58003c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:00.731 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb751b4700 1 -- 172.21.15.32:0/575527223 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb58004e40 msgr2=0x7fdb5800b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:00.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb751b4700 1 --2- 172.21.15.32:0/575527223 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb58004e40 0x7fdb5800b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb751b4700 1 -- 172.21.15.32:0/575527223 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb580037f0 msgr2=0x7fdb58003c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:00.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb751b4700 1 --2- 172.21.15.32:0/575527223 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb580037f0 0x7fdb58003c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb751b4700 1 -- 172.21.15.32:0/575527223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb58003230 con 0x7fdb580044d0 2020-12-05T17:41:00.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb761b6700 1 -- 172.21.15.32:0/575527223 reap_dead start 2020-12-05T17:41:00.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb759b5700 1 --2- 172.21.15.32:0/575527223 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb580037f0 0x7fdb58003c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:41:00.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb751b4700 1 --2- 172.21.15.32:0/575527223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb580044d0 0x7fdb58004950 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 rx=0x7fdb6c00c000 tx=0x7fdb6c008040).ready entity=mon.0 client_cookie=2e8e4533c28a0e70 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:00.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb749b3700 1 -- 172.21.15.32:0/575527223 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb6c00ec30 con 0x7fdb580044d0 2020-12-05T17:41:00.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb749b3700 1 -- 172.21.15.32:0/575527223 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb6c00ed90 con 0x7fdb580044d0 2020-12-05T17:41:00.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb749b3700 1 -- 172.21.15.32:0/575527223 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb6c0125e0 con 0x7fdb580044d0 2020-12-05T17:41:00.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb769b7700 1 -- 172.21.15.32:0/575527223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb580044d0 msgr2=0x7fdb58004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:00.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb769b7700 1 --2- 172.21.15.32:0/575527223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb580044d0 0x7fdb58004950 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 rx=0x7fdb6c00c000 tx=0x7fdb6c008040).stop 2020-12-05T17:41:00.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb769b7700 1 -- 172.21.15.32:0/575527223 shutdown_connections 2020-12-05T17:41:00.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb769b7700 1 --2- 172.21.15.32:0/575527223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb580044d0 0x7fdb58004950 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb769b7700 1 -- 172.21.15.32:0/575527223 >> 172.21.15.32:0/575527223 conn(0x7fdb700604c0 msgr2=0x7fdb7005c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb769b7700 1 -- 172.21.15.32:0/575527223 shutdown_connections 2020-12-05T17:41:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb769b7700 1 -- 172.21.15.32:0/575527223 wait complete. 2020-12-05T17:41:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb769b7700 1 Processor -- start 2020-12-05T17:41:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.718+0000 7fdb769b7700 1 -- start start 2020-12-05T17:41:00.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb769b7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb580044d0 0x7fdb70065ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:00.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb769b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb700651d0 0x7fdb701440f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:00.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb769b7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb70062fa0 0x7fdb70063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:00.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb769b7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb701134a0 con 0x7fdb700651d0 2020-12-05T17:41:00.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb769b7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb70064900 con 0x7fdb70062fa0 2020-12-05T17:41:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb769b7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb701523b0 con 0x7fdb580044d0 2020-12-05T17:41:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb751b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb700651d0 0x7fdb701440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb751b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb700651d0 0x7fdb701440f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40122/0 (socket says 172.21.15.32:40122) 2020-12-05T17:41:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb751b4700 1 -- 172.21.15.32:0/2552183873 learned_addr learned my addr 172.21.15.32:0/2552183873 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb761b6700 1 --2- 172.21.15.32:0/2552183873 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb70062fa0 0x7fdb70063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:00.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb759b5700 1 --2- 172.21.15.32:0/2552183873 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb580044d0 0x7fdb70065ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:00.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb751b4700 1 -- 172.21.15.32:0/2552183873 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb580044d0 msgr2=0x7fdb70065ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:00.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb751b4700 1 --2- 172.21.15.32:0/2552183873 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb580044d0 0x7fdb70065ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb751b4700 1 -- 172.21.15.32:0/2552183873 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb70062fa0 msgr2=0x7fdb70063430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:00.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb751b4700 1 --2- 172.21.15.32:0/2552183873 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb70062fa0 0x7fdb70063430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb751b4700 1 -- 172.21.15.32:0/2552183873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb70065780 con 0x7fdb700651d0 2020-12-05T17:41:00.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb751b4700 1 --2- 172.21.15.32:0/2552183873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb700651d0 0x7fdb701440f0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7fdb6c00c000 tx=0x7fdb6c01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:00.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb62ffd700 1 -- 172.21.15.32:0/2552183873 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb6c01c830 con 0x7fdb700651d0 2020-12-05T17:41:00.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb62ffd700 1 -- 172.21.15.32:0/2552183873 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb6c01c990 con 0x7fdb700651d0 2020-12-05T17:41:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb769b7700 1 -- 172.21.15.32:0/2552183873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb70065a20 con 0x7fdb700651d0 2020-12-05T17:41:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb62ffd700 1 -- 172.21.15.32:0/2552183873 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb6c01ccf0 con 0x7fdb700651d0 2020-12-05T17:41:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb769b7700 1 -- 172.21.15.32:0/2552183873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb70063920 con 0x7fdb700651d0 2020-12-05T17:41:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb769b7700 1 -- 172.21.15.32:0/2552183873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb70065bb0 con 0x7fdb700651d0 2020-12-05T17:41:00.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb62ffd700 1 -- 172.21.15.32:0/2552183873 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fdb6c024420 con 0x7fdb700651d0 2020-12-05T17:41:00.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb62ffd700 1 --2- 172.21.15.32:0/2552183873 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb500554e0 0x7fdb500579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:00.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb62ffd700 1 -- 172.21.15.32:0/2552183873 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fdb6c06fcd0 con 0x7fdb700651d0 2020-12-05T17:41:00.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.722+0000 7fdb759b5700 1 --2- 172.21.15.32:0/2552183873 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb500554e0 0x7fdb500579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:00.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.726+0000 7fdb759b5700 1 --2- 172.21.15.32:0/2552183873 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb500554e0 0x7fdb500579a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7fdb6800c000 tx=0x7fdb68009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:00.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.730+0000 7fdb62ffd700 1 -- 172.21.15.32:0/2552183873 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fdb6c049e60 con 0x7fdb700651d0 2020-12-05T17:41:00.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.874+0000 7fdb769b7700 1 -- 172.21.15.32:0/2552183873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdb70057050 con 0x7fdb700651d0 2020-12-05T17:41:00.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.874+0000 7fdb62ffd700 1 -- 172.21.15.32:0/2552183873 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v125) v1 ==== 84+0+143 (secure 0 0 0) 0x7fdb6c016070 con 0x7fdb700651d0 2020-12-05T17:41:00.880 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:00.880 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2020-12-05T17:41:00.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:00.881 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2020-12-05T17:41:00.881 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2020-12-05T17:41:00.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:00.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:00.882 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2020-12-05T17:41:00.882 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2020-12-05T17:41:00.882 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:00.882 INFO:teuthology.orchestra.run.smithi032.stdout:] 2020-12-05T17:41:00.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.878+0000 7fdb769b7700 1 -- 172.21.15.32:0/2552183873 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb500554e0 msgr2=0x7fdb500579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:00.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.878+0000 7fdb769b7700 1 --2- 172.21.15.32:0/2552183873 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb500554e0 0x7fdb500579a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7fdb6800c000 tx=0x7fdb68009040).stop 2020-12-05T17:41:00.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.878+0000 7fdb769b7700 1 -- 172.21.15.32:0/2552183873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb700651d0 msgr2=0x7fdb701440f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:00.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.878+0000 7fdb769b7700 1 --2- 172.21.15.32:0/2552183873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb700651d0 0x7fdb701440f0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7fdb6c00c000 tx=0x7fdb6c01a000).stop 2020-12-05T17:41:00.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.878+0000 7fdb769b7700 1 -- 172.21.15.32:0/2552183873 shutdown_connections 2020-12-05T17:41:00.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.878+0000 7fdb769b7700 1 --2- 172.21.15.32:0/2552183873 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb70062fa0 0x7fdb70063430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.878+0000 7fdb769b7700 1 --2- 172.21.15.32:0/2552183873 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb500554e0 0x7fdb500579a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.878+0000 7fdb769b7700 1 --2- 172.21.15.32:0/2552183873 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb580044d0 0x7fdb70065ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.878+0000 7fdb769b7700 1 --2- 172.21.15.32:0/2552183873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb700651d0 0x7fdb701440f0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:00.885 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.878+0000 7fdb769b7700 1 -- 172.21.15.32:0/2552183873 >> 172.21.15.32:0/2552183873 conn(0x7fdb700604c0 msgr2=0x7fdb7013f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:00.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.878+0000 7fdb769b7700 1 -- 172.21.15.32:0/2552183873 shutdown_connections 2020-12-05T17:41:00.886 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:00.878+0000 7fdb769b7700 1 -- 172.21.15.32:0/2552183873 wait complete. 2020-12-05T17:41:00.900 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-12-05T17:41:00.900 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-12-05T17:41:01.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.014+0000 7f1edf59e700 1 Processor -- start 2020-12-05T17:41:01.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.014+0000 7f1edf59e700 1 -- start start 2020-12-05T17:41:01.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1edf59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ee005f3b0 0x7f1ee005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:01.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1edf59e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1ee005feb0 0x7f1ee0061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:01.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1edf59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee00b2df0 con 0x7f1ee00b2fb0 2020-12-05T17:41:01.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1edf59e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee00b3c90 con 0x7f1ee005feb0 2020-12-05T17:41:01.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1edf59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee00b2b50 con 0x7f1ee005f3b0 2020-12-05T17:41:01.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1eded9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1ee00b2fb0 0x7f1ee013c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42782/0 (socket says 172.21.15.32:42782) 2020-12-05T17:41:01.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1eded9d700 1 -- 172.21.15.32:0/3343631826 learned_addr learned my addr 172.21.15.32:0/3343631826 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:01.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1eddd9b700 1 --2- 172.21.15.32:0/3343631826 >> v2:172.21.15.61:3300/0 conn(0x7f1ee005feb0 0x7f1ee0061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:01.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1ede59c700 1 --2- 172.21.15.32:0/3343631826 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ee005f3b0 0x7f1ee005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:01.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1edd59a700 1 -- 172.21.15.32:0/3343631826 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2848352039 0 0) 0x7f1ee00b2df0 con 0x7f1ee00b2fb0 2020-12-05T17:41:01.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1edd59a700 1 -- 172.21.15.32:0/3343631826 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ec0003230 con 0x7f1ee00b2fb0 2020-12-05T17:41:01.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1eddd9b700 1 -- 172.21.15.32:0/3343631826 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ee005f3b0 msgr2=0x7f1ee005f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:01.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1eddd9b700 1 --2- 172.21.15.32:0/3343631826 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ee005f3b0 0x7f1ee005f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:01.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1eddd9b700 1 -- 172.21.15.32:0/3343631826 >> v1:172.21.15.32:6789/0 conn(0x7f1ee00b2fb0 legacy=0x7f1ee013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:01.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1eddd9b700 1 -- 172.21.15.32:0/3343631826 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee0142820 con 0x7f1ee005feb0 2020-12-05T17:41:01.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1eddd9b700 1 --2- 172.21.15.32:0/3343631826 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ee005feb0 0x7f1ee0061780 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f1ec8002f30 tx=0x7f1ec8006dc0).ready entity=mon.1 client_cookie=85427567874ecb3 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:01.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1edd59a700 1 -- 172.21.15.32:0/3343631826 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ec80096a0 con 0x7f1ee005feb0 2020-12-05T17:41:01.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1edd59a700 1 -- 172.21.15.32:0/3343631826 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec8009800 con 0x7f1ee005feb0 2020-12-05T17:41:01.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1edd59a700 1 -- 172.21.15.32:0/3343631826 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ec8009b60 con 0x7f1ee005feb0 2020-12-05T17:41:01.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1edf59e700 1 -- 172.21.15.32:0/3343631826 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ee005feb0 msgr2=0x7f1ee0061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:01.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1edf59e700 1 --2- 172.21.15.32:0/3343631826 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ee005feb0 0x7f1ee0061780 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f1ec8002f30 tx=0x7f1ec8006dc0).stop 2020-12-05T17:41:01.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1edf59e700 1 -- 172.21.15.32:0/3343631826 shutdown_connections 2020-12-05T17:41:01.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1edf59e700 1 --2- 172.21.15.32:0/3343631826 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ee005f3b0 0x7f1ee005f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:01.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1edf59e700 1 --2- 172.21.15.32:0/3343631826 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ee005feb0 0x7f1ee0061780 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:01.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1edf59e700 1 -- 172.21.15.32:0/3343631826 >> 172.21.15.32:0/3343631826 conn(0x7f1ee00b8f20 msgr2=0x7f1ee00b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:01.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1edf59e700 1 -- 172.21.15.32:0/3343631826 shutdown_connections 2020-12-05T17:41:01.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.018+0000 7f1edf59e700 1 -- 172.21.15.32:0/3343631826 wait complete. 2020-12-05T17:41:01.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1edf59e700 1 Processor -- start 2020-12-05T17:41:01.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1edf59e700 1 -- start start 2020-12-05T17:41:01.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1edf59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ee005f3b0 0x7f1ee0149280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:01.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1edf59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ee005feb0 0x7f1ee0149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:01.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1edf59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ee00b2fb0 0x7f1ee0140ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:01.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1edf59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee0147d50 con 0x7f1ee005f3b0 2020-12-05T17:41:01.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1edf59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee01551e0 con 0x7f1ee00b2fb0 2020-12-05T17:41:01.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1edf59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee0155480 con 0x7f1ee005feb0 2020-12-05T17:41:01.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1ede59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ee005f3b0 0x7f1ee0149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:01.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1ede59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ee005f3b0 0x7f1ee0149280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40136/0 (socket says 172.21.15.32:40136) 2020-12-05T17:41:01.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1ede59c700 1 -- 172.21.15.32:0/4025756035 learned_addr learned my addr 172.21.15.32:0/4025756035 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:01.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1eddd9b700 1 --2- 172.21.15.32:0/4025756035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ee005feb0 0x7f1ee0149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:01.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1eded9d700 1 --2- 172.21.15.32:0/4025756035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ee00b2fb0 0x7f1ee0140ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:01.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1ede59c700 1 -- 172.21.15.32:0/4025756035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ee005feb0 msgr2=0x7f1ee0149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:01.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1ede59c700 1 --2- 172.21.15.32:0/4025756035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ee005feb0 0x7f1ee0149770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:01.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1ede59c700 1 -- 172.21.15.32:0/4025756035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ee00b2fb0 msgr2=0x7f1ee0140ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:01.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1ede59c700 1 --2- 172.21.15.32:0/4025756035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ee00b2fb0 0x7f1ee0140ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:01.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1ede59c700 1 -- 172.21.15.32:0/4025756035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee01475f0 con 0x7f1ee005f3b0 2020-12-05T17:41:01.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1eded9d700 1 --2- 172.21.15.32:0/4025756035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ee00b2fb0 0x7f1ee0140ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:41:01.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1ede59c700 1 --2- 172.21.15.32:0/4025756035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ee005f3b0 0x7f1ee0149280 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0x7f1ed800c000 tx=0x7f1ed80091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:01.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1ed77fe700 1 -- 172.21.15.32:0/4025756035 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ed800cbb0 con 0x7f1ee005f3b0 2020-12-05T17:41:01.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1ed77fe700 1 -- 172.21.15.32:0/4025756035 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed800cd10 con 0x7f1ee005f3b0 2020-12-05T17:41:01.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1ed77fe700 1 -- 172.21.15.32:0/4025756035 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ed80125e0 con 0x7f1ee005f3b0 2020-12-05T17:41:01.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1edf59e700 1 -- 172.21.15.32:0/4025756035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee01486a0 con 0x7f1ee005f3b0 2020-12-05T17:41:01.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.022+0000 7f1edf59e700 1 -- 172.21.15.32:0/4025756035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee0141090 con 0x7f1ee005f3b0 2020-12-05T17:41:01.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.026+0000 7f1ed77fe700 1 -- 172.21.15.32:0/4025756035 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1ed8022070 con 0x7f1ee005f3b0 2020-12-05T17:41:01.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.026+0000 7f1edf59e700 1 -- 172.21.15.32:0/4025756035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee0155640 con 0x7f1ee005f3b0 2020-12-05T17:41:01.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.026+0000 7f1ed77fe700 1 --2- 172.21.15.32:0/4025756035 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ec40553b0 0x7f1ec4057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:01.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.026+0000 7f1ed77fe700 1 -- 172.21.15.32:0/4025756035 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f1ed806e4f0 con 0x7f1ee005f3b0 2020-12-05T17:41:01.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.026+0000 7f1eddd9b700 1 --2- 172.21.15.32:0/4025756035 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ec40553b0 0x7f1ec4057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:01.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.026+0000 7f1eddd9b700 1 --2- 172.21.15.32:0/4025756035 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ec40553b0 0x7f1ec4057870 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f1ec8019000 tx=0x7f1ec8015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:01.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.034+0000 7f1ed77fe700 1 -- 172.21.15.32:0/4025756035 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1ed80449d0 con 0x7f1ee005f3b0 2020-12-05T17:41:01.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.178+0000 7f1edf59e700 1 -- 172.21.15.32:0/4025756035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f1ee0057050 con 0x7f1ee005f3b0 2020-12-05T17:41:01.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.914+0000 7f1ed77fe700 1 -- 172.21.15.32:0/4025756035 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v126) v1 ==== 133+0+0 (secure 0 0 0) 0x7f1ed8047ff0 con 0x7f1ee005f3b0 2020-12-05T17:41:01.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.918+0000 7f1edf59e700 1 -- 172.21.15.32:0/4025756035 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ec40553b0 msgr2=0x7f1ec4057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:01.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.918+0000 7f1edf59e700 1 --2- 172.21.15.32:0/4025756035 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ec40553b0 0x7f1ec4057870 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f1ec8019000 tx=0x7f1ec8015000).stop 2020-12-05T17:41:01.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.918+0000 7f1edf59e700 1 -- 172.21.15.32:0/4025756035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ee005f3b0 msgr2=0x7f1ee0149280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:01.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.918+0000 7f1edf59e700 1 --2- 172.21.15.32:0/4025756035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ee005f3b0 0x7f1ee0149280 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0x7f1ed800c000 tx=0x7f1ed80091a0).stop 2020-12-05T17:41:01.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.918+0000 7f1edf59e700 1 -- 172.21.15.32:0/4025756035 shutdown_connections 2020-12-05T17:41:01.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.918+0000 7f1edf59e700 1 --2- 172.21.15.32:0/4025756035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ee00b2fb0 0x7f1ee0140ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:01.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.918+0000 7f1edf59e700 1 --2- 172.21.15.32:0/4025756035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ee005f3b0 0x7f1ee0149280 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:01.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.918+0000 7f1edf59e700 1 --2- 172.21.15.32:0/4025756035 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ec40553b0 0x7f1ec4057870 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:01.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.918+0000 7f1edf59e700 1 --2- 172.21.15.32:0/4025756035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ee005feb0 0x7f1ee0149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:01.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.918+0000 7f1edf59e700 1 -- 172.21.15.32:0/4025756035 >> 172.21.15.32:0/4025756035 conn(0x7f1ee00b8f20 msgr2=0x7f1ee0061780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:01.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.918+0000 7f1edf59e700 1 -- 172.21.15.32:0/4025756035 shutdown_connections 2020-12-05T17:41:01.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:01.918+0000 7f1edf59e700 1 -- 172.21.15.32:0/4025756035 wait complete. 2020-12-05T17:41:01.927 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2020-12-05T17:41:01.937 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-12-05T17:41:02.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.050+0000 7f925deb4700 1 Processor -- start 2020-12-05T17:41:02.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.050+0000 7f925deb4700 1 -- start start 2020-12-05T17:41:02.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.050+0000 7f925deb4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f925805f3b0 0x7f925805f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:02.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.050+0000 7f925deb4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92580b2df0 con 0x7f925805feb0 2020-12-05T17:41:02.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.050+0000 7f925deb4700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92580b3c90 con 0x7f92580b2fb0 2020-12-05T17:41:02.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.050+0000 7f925deb4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92580b2b50 con 0x7f925805f3b0 2020-12-05T17:41:02.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.050+0000 7f9257fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f925805feb0 0x7f925813c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42796/0 (socket says 172.21.15.32:42796) 2020-12-05T17:41:02.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.050+0000 7f9257fff700 1 -- 172.21.15.32:0/1604762861 learned_addr learned my addr 172.21.15.32:0/1604762861 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:02.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.050+0000 7f925ceb2700 1 --2- 172.21.15.32:0/1604762861 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f925805f3b0 0x7f925805f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:02.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.050+0000 7f92577fe700 1 -- 172.21.15.32:0/1604762861 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718429714 0 0) 0x7f92580b2df0 con 0x7f925805feb0 2020-12-05T17:41:02.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.050+0000 7f92577fe700 1 -- 172.21.15.32:0/1604762861 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9238003230 con 0x7f925805feb0 2020-12-05T17:41:02.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.054+0000 7f925ceb2700 1 -- 172.21.15.32:0/1604762861 >> v1:172.21.15.61:6789/0 conn(0x7f92580b2fb0 legacy=0x7f9258142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:02.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.054+0000 7f925ceb2700 1 -- 172.21.15.32:0/1604762861 >> v1:172.21.15.32:6789/0 conn(0x7f925805feb0 legacy=0x7f925813c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:02.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.054+0000 7f925ceb2700 1 -- 172.21.15.32:0/1604762861 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9258142820 con 0x7f925805f3b0 2020-12-05T17:41:02.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.054+0000 7f92577fe700 1 -- 172.21.15.32:0/1604762861 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4280066034 0 0) 0x7f92580b3c90 con 0x7f92580b2fb0 2020-12-05T17:41:02.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.054+0000 7f925ceb2700 1 --2- 172.21.15.32:0/1604762861 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f925805f3b0 0x7f925805f790 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f9240002f30 tx=0x7f9240006dc0).ready entity=mon.2 client_cookie=e8c82ccbb8ccd63c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:02.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.054+0000 7f92577fe700 1 -- 172.21.15.32:0/1604762861 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92400097b0 con 0x7f925805f3b0 2020-12-05T17:41:02.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.054+0000 7f92577fe700 1 -- 172.21.15.32:0/1604762861 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9240009910 con 0x7f925805f3b0 2020-12-05T17:41:02.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.054+0000 7f92577fe700 1 -- 172.21.15.32:0/1604762861 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9240009c70 con 0x7f925805f3b0 2020-12-05T17:41:02.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.054+0000 7f925deb4700 1 -- 172.21.15.32:0/1604762861 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f925805f3b0 msgr2=0x7f925805f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:02.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.054+0000 7f925deb4700 1 --2- 172.21.15.32:0/1604762861 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f925805f3b0 0x7f925805f790 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f9240002f30 tx=0x7f9240006dc0).stop 2020-12-05T17:41:02.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.054+0000 7f925deb4700 1 -- 172.21.15.32:0/1604762861 shutdown_connections 2020-12-05T17:41:02.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.054+0000 7f925deb4700 1 --2- 172.21.15.32:0/1604762861 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f925805f3b0 0x7f925805f790 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:02.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.054+0000 7f925deb4700 1 -- 172.21.15.32:0/1604762861 >> 172.21.15.32:0/1604762861 conn(0x7f92580b8f20 msgr2=0x7f92580b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:02.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.054+0000 7f925deb4700 1 -- 172.21.15.32:0/1604762861 shutdown_connections 2020-12-05T17:41:02.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.054+0000 7f925deb4700 1 -- 172.21.15.32:0/1604762861 wait complete. 2020-12-05T17:41:02.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.054+0000 7f925deb4700 1 Processor -- start 2020-12-05T17:41:02.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.054+0000 7f925deb4700 1 -- start start 2020-12-05T17:41:02.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.058+0000 7f925deb4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f925805f3b0 0x7f9258154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:02.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.058+0000 7f925deb4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f925805feb0 0x7f925813e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:02.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.058+0000 7f925deb4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92580b2fb0 0x7f9258148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:02.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.058+0000 7f925deb4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92581419e0 con 0x7f925805feb0 2020-12-05T17:41:02.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.058+0000 7f925deb4700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9258144a00 con 0x7f925805f3b0 2020-12-05T17:41:02.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.058+0000 7f925deb4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92581444c0 con 0x7f92580b2fb0 2020-12-05T17:41:02.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.058+0000 7f9257fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f925805feb0 0x7f925813e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:02.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.058+0000 7f925ceb2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f925805f3b0 0x7f9258154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:02.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.058+0000 7f9257fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f925805feb0 0x7f925813e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40152/0 (socket says 172.21.15.32:40152) 2020-12-05T17:41:02.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.058+0000 7f925d6b3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92580b2fb0 0x7f9258148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:02.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.058+0000 7f9257fff700 1 -- 172.21.15.32:0/2359862027 learned_addr learned my addr 172.21.15.32:0/2359862027 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:02.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.058+0000 7f9257fff700 1 -- 172.21.15.32:0/2359862027 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92580b2fb0 msgr2=0x7f9258148fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:02.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.058+0000 7f9257fff700 1 --2- 172.21.15.32:0/2359862027 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92580b2fb0 0x7f9258148fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:02.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.058+0000 7f9257fff700 1 -- 172.21.15.32:0/2359862027 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f925805f3b0 msgr2=0x7f9258154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:02.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.058+0000 7f9257fff700 1 --2- 172.21.15.32:0/2359862027 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f925805f3b0 0x7f9258154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:02.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.058+0000 7f9257fff700 1 -- 172.21.15.32:0/2359862027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9258144760 con 0x7f925805feb0 2020-12-05T17:41:02.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.058+0000 7f925d6b3700 1 --2- 172.21.15.32:0/2359862027 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92580b2fb0 0x7f9258148fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:02.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.058+0000 7f9257fff700 1 --2- 172.21.15.32:0/2359862027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f925805feb0 0x7f925813e7b0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f92480010d0 tx=0x7f9248007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:02.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.058+0000 7f9255ffb700 1 -- 172.21.15.32:0/2359862027 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92480072e0 con 0x7f925805feb0 2020-12-05T17:41:02.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.058+0000 7f9255ffb700 1 -- 172.21.15.32:0/2359862027 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9248007440 con 0x7f925805feb0 2020-12-05T17:41:02.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.058+0000 7f925deb4700 1 -- 172.21.15.32:0/2359862027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9258143f40 con 0x7f925805feb0 2020-12-05T17:41:02.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.058+0000 7f9255ffb700 1 -- 172.21.15.32:0/2359862027 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92480057d0 con 0x7f925805feb0 2020-12-05T17:41:02.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.058+0000 7f925deb4700 1 -- 172.21.15.32:0/2359862027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9258142310 con 0x7f925805feb0 2020-12-05T17:41:02.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.062+0000 7f925deb4700 1 -- 172.21.15.32:0/2359862027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9258057050 con 0x7f925805feb0 2020-12-05T17:41:02.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.062+0000 7f9255ffb700 1 -- 172.21.15.32:0/2359862027 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f924801a060 con 0x7f925805feb0 2020-12-05T17:41:02.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.062+0000 7f9255ffb700 1 --2- 172.21.15.32:0/2359862027 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f923c055330 0x7f923c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:02.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.062+0000 7f9255ffb700 1 -- 172.21.15.32:0/2359862027 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f924806d3e0 con 0x7f925805feb0 2020-12-05T17:41:02.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.062+0000 7f925ceb2700 1 --2- 172.21.15.32:0/2359862027 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f923c055330 0x7f923c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:02.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.062+0000 7f925ceb2700 1 --2- 172.21.15.32:0/2359862027 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f923c055330 0x7f923c0577f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f924001a000 tx=0x7f9240015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:02.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.070+0000 7f9255ffb700 1 -- 172.21.15.32:0/2359862027 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f92480435e0 con 0x7f925805feb0 2020-12-05T17:41:02.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.214+0000 7f925deb4700 1 -- 172.21.15.32:0/2359862027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f925805aab0 con 0x7f925805feb0 2020-12-05T17:41:02.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.926+0000 7f9255ffb700 1 -- 172.21.15.32:0/2359862027 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v127) v1 ==== 125+0+0 (secure 0 0 0) 0x7f92480180a0 con 0x7f925805feb0 2020-12-05T17:41:02.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.930+0000 7f925deb4700 1 -- 172.21.15.32:0/2359862027 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f923c055330 msgr2=0x7f923c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:02.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.930+0000 7f925deb4700 1 --2- 172.21.15.32:0/2359862027 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f923c055330 0x7f923c0577f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f924001a000 tx=0x7f9240015000).stop 2020-12-05T17:41:02.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.930+0000 7f925deb4700 1 -- 172.21.15.32:0/2359862027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f925805feb0 msgr2=0x7f925813e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:02.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.930+0000 7f925deb4700 1 --2- 172.21.15.32:0/2359862027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f925805feb0 0x7f925813e7b0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f92480010d0 tx=0x7f9248007aa0).stop 2020-12-05T17:41:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.934+0000 7f925deb4700 1 -- 172.21.15.32:0/2359862027 shutdown_connections 2020-12-05T17:41:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.934+0000 7f925deb4700 1 --2- 172.21.15.32:0/2359862027 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f923c055330 0x7f923c0577f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.934+0000 7f925deb4700 1 --2- 172.21.15.32:0/2359862027 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92580b2fb0 0x7f9258148fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.934+0000 7f925deb4700 1 --2- 172.21.15.32:0/2359862027 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f925805f3b0 0x7f9258154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:02.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.934+0000 7f925deb4700 1 --2- 172.21.15.32:0/2359862027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f925805feb0 0x7f925813e7b0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:02.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.934+0000 7f925deb4700 1 -- 172.21.15.32:0/2359862027 >> 172.21.15.32:0/2359862027 conn(0x7f92580b8f20 msgr2=0x7f92580b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:02.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.934+0000 7f925deb4700 1 -- 172.21.15.32:0/2359862027 shutdown_connections 2020-12-05T17:41:02.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:02.934+0000 7f925deb4700 1 -- 172.21.15.32:0/2359862027 wait complete. 2020-12-05T17:41:02.943 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2020-12-05T17:41:02.958 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-12-05T17:41:03.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0efb7700 1 Processor -- start 2020-12-05T17:41:03.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0efb7700 1 -- start start 2020-12-05T17:41:03.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0efb7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e0805f3b0 0x7f0e0805f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:03.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0efb7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0e0805feb0 0x7f0e08061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:03.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0efb7700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0e080b2fb0 0x7f0e0813c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:03.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0efb7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e080b2df0 con 0x7f0e0805feb0 2020-12-05T17:41:03.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0efb7700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e080b3c90 con 0x7f0e080b2fb0 2020-12-05T17:41:03.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0efb7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e080b2b50 con 0x7f0e0805f3b0 2020-12-05T17:41:03.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0d7b4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0e0805feb0 0x7f0e08061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:03.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0d7b4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0e0805feb0 0x7f0e08061780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40162/0 (socket says 172.21.15.32:40162) 2020-12-05T17:41:03.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0d7b4700 1 -- 172.21.15.32:0/131856811 learned_addr learned my addr 172.21.15.32:0/131856811 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:03.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0e7b6700 1 --2- 172.21.15.32:0/131856811 >> v2:172.21.15.61:3300/0 conn(0x7f0e080b2fb0 0x7f0e0813c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:03.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0dfb5700 1 --2- 172.21.15.32:0/131856811 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e0805f3b0 0x7f0e0805f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:03.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0d7b4700 1 -- 172.21.15.32:0/131856811 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e0805f3b0 msgr2=0x7f0e0805f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:03.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0d7b4700 1 --2- 172.21.15.32:0/131856811 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e0805f3b0 0x7f0e0805f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0d7b4700 1 -- 172.21.15.32:0/131856811 >> v2:172.21.15.61:3300/0 conn(0x7f0e080b2fb0 msgr2=0x7f0e0813c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:03.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0d7b4700 1 --2- 172.21.15.32:0/131856811 >> v2:172.21.15.61:3300/0 conn(0x7f0e080b2fb0 0x7f0e0813c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0d7b4700 1 -- 172.21.15.32:0/131856811 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e08142820 con 0x7f0e0805feb0 2020-12-05T17:41:03.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0d7b4700 1 --2- 172.21.15.32:0/131856811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e0805feb0 0x7f0e08061780 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7f0df8002f30 tx=0x7f0df8006e80).ready entity=mon.0 client_cookie=46b0e920f67095a6 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:03.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0cfb3700 1 -- 172.21.15.32:0/131856811 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0df8009640 con 0x7f0e0805feb0 2020-12-05T17:41:03.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0cfb3700 1 -- 172.21.15.32:0/131856811 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df80097a0 con 0x7f0e0805feb0 2020-12-05T17:41:03.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0cfb3700 1 -- 172.21.15.32:0/131856811 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0df8009b00 con 0x7f0e0805feb0 2020-12-05T17:41:03.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0efb7700 1 -- 172.21.15.32:0/131856811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e0805feb0 msgr2=0x7f0e08061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:03.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.070+0000 7f0e0efb7700 1 --2- 172.21.15.32:0/131856811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e0805feb0 0x7f0e08061780 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7f0df8002f30 tx=0x7f0df8006e80).stop 2020-12-05T17:41:03.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0efb7700 1 -- 172.21.15.32:0/131856811 shutdown_connections 2020-12-05T17:41:03.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0efb7700 1 --2- 172.21.15.32:0/131856811 >> v2:172.21.15.61:3300/0 conn(0x7f0e080b2fb0 0x7f0e0813c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0efb7700 1 --2- 172.21.15.32:0/131856811 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e0805f3b0 0x7f0e0805f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0efb7700 1 --2- 172.21.15.32:0/131856811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e0805feb0 0x7f0e08061780 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0efb7700 1 -- 172.21.15.32:0/131856811 >> 172.21.15.32:0/131856811 conn(0x7f0e080b8f20 msgr2=0x7f0e080b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:03.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0efb7700 1 -- 172.21.15.32:0/131856811 shutdown_connections 2020-12-05T17:41:03.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0efb7700 1 -- 172.21.15.32:0/131856811 wait complete. 2020-12-05T17:41:03.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0efb7700 1 Processor -- start 2020-12-05T17:41:03.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0efb7700 1 -- start start 2020-12-05T17:41:03.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0efb7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e0805f3b0 0x7f0e08144e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:03.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0efb7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e0805feb0 0x7f0e081409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:03.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0efb7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e080b2fb0 0x7f0e08140ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:03.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0efb7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e08152660 con 0x7f0e0805feb0 2020-12-05T17:41:03.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0efb7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e08145d00 con 0x7f0e080b2fb0 2020-12-05T17:41:03.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0efb7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e081516b0 con 0x7f0e0805f3b0 2020-12-05T17:41:03.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0d7b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e0805feb0 0x7f0e081409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:03.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0d7b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e0805feb0 0x7f0e081409e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40164/0 (socket says 172.21.15.32:40164) 2020-12-05T17:41:03.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0d7b4700 1 -- 172.21.15.32:0/2966186011 learned_addr learned my addr 172.21.15.32:0/2966186011 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:03.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0e7b6700 1 --2- 172.21.15.32:0/2966186011 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e080b2fb0 0x7f0e08140ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:03.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0dfb5700 1 --2- 172.21.15.32:0/2966186011 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e0805f3b0 0x7f0e08144e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:03.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0d7b4700 1 -- 172.21.15.32:0/2966186011 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e0805f3b0 msgr2=0x7f0e08144e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:03.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0d7b4700 1 --2- 172.21.15.32:0/2966186011 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e0805f3b0 0x7f0e08144e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0d7b4700 1 -- 172.21.15.32:0/2966186011 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e080b2fb0 msgr2=0x7f0e08140ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:03.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0d7b4700 1 --2- 172.21.15.32:0/2966186011 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e080b2fb0 0x7f0e08140ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0d7b4700 1 -- 172.21.15.32:0/2966186011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e081513d0 con 0x7f0e0805feb0 2020-12-05T17:41:03.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0d7b4700 1 --2- 172.21.15.32:0/2966186011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e0805feb0 0x7f0e081409e0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7f0df8002f30 tx=0x7f0df8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:03.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0df6ffd700 1 -- 172.21.15.32:0/2966186011 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0df8011bf0 con 0x7f0e0805feb0 2020-12-05T17:41:03.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0df6ffd700 1 -- 172.21.15.32:0/2966186011 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df8011d50 con 0x7f0e0805feb0 2020-12-05T17:41:03.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0efb7700 1 -- 172.21.15.32:0/2966186011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e08147e90 con 0x7f0e0805feb0 2020-12-05T17:41:03.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0df6ffd700 1 -- 172.21.15.32:0/2966186011 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0df801d640 con 0x7f0e0805feb0 2020-12-05T17:41:03.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.074+0000 7f0e0efb7700 1 -- 172.21.15.32:0/2966186011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e08147580 con 0x7f0e0805feb0 2020-12-05T17:41:03.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.078+0000 7f0e0efb7700 1 -- 172.21.15.32:0/2966186011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e08057050 con 0x7f0e0805feb0 2020-12-05T17:41:03.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.078+0000 7f0df6ffd700 1 -- 172.21.15.32:0/2966186011 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0df800f070 con 0x7f0e0805feb0 2020-12-05T17:41:03.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.078+0000 7f0df6ffd700 1 --2- 172.21.15.32:0/2966186011 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0de80553b0 0x7f0de8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:03.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.078+0000 7f0df6ffd700 1 -- 172.21.15.32:0/2966186011 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f0df8070560 con 0x7f0e0805feb0 2020-12-05T17:41:03.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.078+0000 7f0e0dfb5700 1 --2- 172.21.15.32:0/2966186011 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0de80553b0 0x7f0de8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:03.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.082+0000 7f0e0dfb5700 1 --2- 172.21.15.32:0/2966186011 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0de80553b0 0x7f0de8057870 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f0e0400b000 tx=0x7f0e04006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:03.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.086+0000 7f0df6ffd700 1 -- 172.21.15.32:0/2966186011 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0df8046830 con 0x7f0e0805feb0 2020-12-05T17:41:03.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.274+0000 7f0e0efb7700 1 -- 172.21.15.32:0/2966186011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f0e080b9cb0 con 0x7f0e0805feb0 2020-12-05T17:41:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.286+0000 7f0df6ffd700 1 -- 172.21.15.32:0/2966186011 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 25 and data pool 26 v61) v1 ==== 189+0+0 (secure 0 0 0) 0x7f0df8049e50 con 0x7f0e0805feb0 2020-12-05T17:41:03.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.290+0000 7f0e0efb7700 1 -- 172.21.15.32:0/2966186011 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0de80553b0 msgr2=0x7f0de8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:03.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.290+0000 7f0e0efb7700 1 --2- 172.21.15.32:0/2966186011 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0de80553b0 0x7f0de8057870 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f0e0400b000 tx=0x7f0e04006dc0).stop 2020-12-05T17:41:03.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.290+0000 7f0e0efb7700 1 -- 172.21.15.32:0/2966186011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e0805feb0 msgr2=0x7f0e081409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:03.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.290+0000 7f0e0efb7700 1 --2- 172.21.15.32:0/2966186011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e0805feb0 0x7f0e081409e0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7f0df8002f30 tx=0x7f0df8015000).stop 2020-12-05T17:41:03.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.290+0000 7f0e0efb7700 1 -- 172.21.15.32:0/2966186011 shutdown_connections 2020-12-05T17:41:03.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.290+0000 7f0e0efb7700 1 --2- 172.21.15.32:0/2966186011 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e080b2fb0 0x7f0e08140ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.290+0000 7f0e0efb7700 1 --2- 172.21.15.32:0/2966186011 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0de80553b0 0x7f0de8057870 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.290+0000 7f0e0efb7700 1 --2- 172.21.15.32:0/2966186011 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e0805f3b0 0x7f0e08144e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.290+0000 7f0e0efb7700 1 --2- 172.21.15.32:0/2966186011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e0805feb0 0x7f0e081409e0 secure :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 rx=0x7f0df8002f30 tx=0x7f0df8015000).stop 2020-12-05T17:41:03.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.290+0000 7f0e0efb7700 1 -- 172.21.15.32:0/2966186011 >> 172.21.15.32:0/2966186011 conn(0x7f0e080b8f20 msgr2=0x7f0e080b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:03.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.290+0000 7f0e0efb7700 1 -- 172.21.15.32:0/2966186011 shutdown_connections 2020-12-05T17:41:03.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.290+0000 7f0e0efb7700 1 -- 172.21.15.32:0/2966186011 wait complete. 2020-12-05T17:41:03.305 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 25 and data pool 26 2020-12-05T17:41:03.316 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:41:03.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.410+0000 7ff20759e700 1 Processor -- start 2020-12-05T17:41:03.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.410+0000 7ff20759e700 1 -- start start 2020-12-05T17:41:03.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.410+0000 7ff20759e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff2080b5f00 0x7ff2080b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:03.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff20759e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff2080af480 0x7ff208142ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:03.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff20759e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff208142ab0 con 0x7ff2080ad510 2020-12-05T17:41:03.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff20759e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2080af8f0 con 0x7ff2080af480 2020-12-05T17:41:03.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff20759e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2080b62e0 con 0x7ff2080b5f00 2020-12-05T17:41:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff20659c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff2080b5f00 0x7ff2080b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff205d9b700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7ff2080ad510 0x7ff208066370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:55474/0 (socket says 172.21.15.32:55474) 2020-12-05T17:41:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff205d9b700 1 -- 172.21.15.32:0/636906609 learned_addr learned my addr 172.21.15.32:0/636906609 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff206d9d700 1 --2- 172.21.15.32:0/636906609 >> v2:172.21.15.61:3300/0 conn(0x7ff2080af480 0x7ff208142ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff20559a700 1 -- 172.21.15.32:0/636906609 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3998618577 0 0) 0x7ff208142ab0 con 0x7ff2080ad510 2020-12-05T17:41:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff20559a700 1 -- 172.21.15.32:0/636906609 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1e8003230 con 0x7ff2080ad510 2020-12-05T17:41:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff20659c700 1 -- 172.21.15.32:0/636906609 >> v2:172.21.15.61:3300/0 conn(0x7ff2080af480 msgr2=0x7ff208142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff20659c700 1 --2- 172.21.15.32:0/636906609 >> v2:172.21.15.61:3300/0 conn(0x7ff2080af480 0x7ff208142ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff20659c700 1 -- 172.21.15.32:0/636906609 >> v1:172.21.15.61:6789/0 conn(0x7ff2080ad510 legacy=0x7ff208066370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:03.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff20659c700 1 -- 172.21.15.32:0/636906609 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2080afbf0 con 0x7ff2080b5f00 2020-12-05T17:41:03.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff20659c700 1 --2- 172.21.15.32:0/636906609 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff2080b5f00 0x7ff2080b14e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7ff1f80032b0 tx=0x7ff1f80091a0).ready entity=mon.2 client_cookie=dee17685ccd649d8 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:03.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff20559a700 1 -- 172.21.15.32:0/636906609 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1f800b940 con 0x7ff2080b5f00 2020-12-05T17:41:03.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff20559a700 1 -- 172.21.15.32:0/636906609 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f800baa0 con 0x7ff2080b5f00 2020-12-05T17:41:03.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff20559a700 1 -- 172.21.15.32:0/636906609 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1f800bc10 con 0x7ff2080b5f00 2020-12-05T17:41:03.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff20759e700 1 -- 172.21.15.32:0/636906609 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff2080b5f00 msgr2=0x7ff2080b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:03.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff20759e700 1 --2- 172.21.15.32:0/636906609 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff2080b5f00 0x7ff2080b14e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7ff1f80032b0 tx=0x7ff1f80091a0).stop 2020-12-05T17:41:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff20759e700 1 -- 172.21.15.32:0/636906609 shutdown_connections 2020-12-05T17:41:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff20759e700 1 --2- 172.21.15.32:0/636906609 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff2080b5f00 0x7ff2080b14e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff20759e700 1 --2- 172.21.15.32:0/636906609 >> v2:172.21.15.61:3300/0 conn(0x7ff2080af480 0x7ff208142ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff20759e700 1 -- 172.21.15.32:0/636906609 >> 172.21.15.32:0/636906609 conn(0x7ff2080604c0 msgr2=0x7ff20805c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff20759e700 1 -- 172.21.15.32:0/636906609 shutdown_connections 2020-12-05T17:41:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.414+0000 7ff20759e700 1 -- 172.21.15.32:0/636906609 wait complete. 2020-12-05T17:41:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff20759e700 1 Processor -- start 2020-12-05T17:41:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff20759e700 1 -- start start 2020-12-05T17:41:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff20759e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff2080ad510 0x7ff20813e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff20759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2080af480 0x7ff208064c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:03.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff20759e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff2080b5f00 0x7ff208065130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:03.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff20759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2080b5010 con 0x7ff2080af480 2020-12-05T17:41:03.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff20759e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2080b52b0 con 0x7ff2080ad510 2020-12-05T17:41:03.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff20759e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2080b5550 con 0x7ff2080b5f00 2020-12-05T17:41:03.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff205d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2080af480 0x7ff208064c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:03.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff205d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2080af480 0x7ff208064c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40188/0 (socket says 172.21.15.32:40188) 2020-12-05T17:41:03.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff205d9b700 1 -- 172.21.15.32:0/2848100511 learned_addr learned my addr 172.21.15.32:0/2848100511 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff20659c700 1 --2- 172.21.15.32:0/2848100511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff2080ad510 0x7ff20813e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff206d9d700 1 --2- 172.21.15.32:0/2848100511 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff2080b5f00 0x7ff208065130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff205d9b700 1 -- 172.21.15.32:0/2848100511 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff2080b5f00 msgr2=0x7ff208065130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff205d9b700 1 --2- 172.21.15.32:0/2848100511 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff2080b5f00 0x7ff208065130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff205d9b700 1 -- 172.21.15.32:0/2848100511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff2080ad510 msgr2=0x7ff20813e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:03.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff205d9b700 1 --2- 172.21.15.32:0/2848100511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff2080ad510 0x7ff20813e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff205d9b700 1 -- 172.21.15.32:0/2848100511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2080b4c90 con 0x7ff2080af480 2020-12-05T17:41:03.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff205d9b700 1 --2- 172.21.15.32:0/2848100511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2080af480 0x7ff208064c40 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 rx=0x7ff1f00010d0 tx=0x7ff1f0007af0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:03.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff1ff7fe700 1 -- 172.21.15.32:0/2848100511 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1f00072e0 con 0x7ff2080af480 2020-12-05T17:41:03.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff20759e700 1 -- 172.21.15.32:0/2848100511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff208065cc0 con 0x7ff2080af480 2020-12-05T17:41:03.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff1ff7fe700 1 -- 172.21.15.32:0/2848100511 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f0007440 con 0x7ff2080af480 2020-12-05T17:41:03.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff1ff7fe700 1 -- 172.21.15.32:0/2848100511 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1f00057d0 con 0x7ff2080af480 2020-12-05T17:41:03.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.418+0000 7ff20759e700 1 -- 172.21.15.32:0/2848100511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2080b4390 con 0x7ff2080af480 2020-12-05T17:41:03.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.422+0000 7ff20759e700 1 -- 172.21.15.32:0/2848100511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2080b5710 con 0x7ff2080af480 2020-12-05T17:41:03.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.422+0000 7ff1ff7fe700 1 -- 172.21.15.32:0/2848100511 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff1f0010070 con 0x7ff2080af480 2020-12-05T17:41:03.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.422+0000 7ff1ff7fe700 1 --2- 172.21.15.32:0/2848100511 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff1ec055440 0x7ff1ec057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:03.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.422+0000 7ff1ff7fe700 1 -- 172.21.15.32:0/2848100511 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff1f006d7b0 con 0x7ff2080af480 2020-12-05T17:41:03.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.422+0000 7ff20659c700 1 --2- 172.21.15.32:0/2848100511 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff1ec055440 0x7ff1ec057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:03.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.422+0000 7ff20659c700 1 --2- 172.21.15.32:0/2848100511 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff1ec055440 0x7ff1ec057900 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7ff1f801c000 tx=0x7ff1f8016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:03.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.430+0000 7ff1ff7fe700 1 -- 172.21.15.32:0/2848100511 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff1f00438b0 con 0x7ff2080af480 2020-12-05T17:41:03.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.574+0000 7ff20759e700 1 -- 172.21.15.32:0/2848100511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff208057050 con 0x7ff2080af480 2020-12-05T17:41:03.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.578+0000 7ff1ff7fe700 1 -- 172.21.15.32:0/2848100511 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+16298 (secure 0 0 0) 0x7ff1f0018070 con 0x7ff2080af480 2020-12-05T17:41:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:03.583 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":128,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:41:03.280605+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:41:01.240356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2020-12-05T17:41:02.269469+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:41:03.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.578+0000 7ff20759e700 1 -- 172.21.15.32:0/2848100511 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff1ec055440 msgr2=0x7ff1ec057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:03.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.578+0000 7ff20759e700 1 --2- 172.21.15.32:0/2848100511 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff1ec055440 0x7ff1ec057900 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7ff1f801c000 tx=0x7ff1f8016000).stop 2020-12-05T17:41:03.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.578+0000 7ff20759e700 1 -- 172.21.15.32:0/2848100511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2080af480 msgr2=0x7ff208064c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:03.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.578+0000 7ff20759e700 1 --2- 172.21.15.32:0/2848100511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2080af480 0x7ff208064c40 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 rx=0x7ff1f00010d0 tx=0x7ff1f0007af0).stop 2020-12-05T17:41:03.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.578+0000 7ff20759e700 1 -- 172.21.15.32:0/2848100511 shutdown_connections 2020-12-05T17:41:03.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.578+0000 7ff20759e700 1 --2- 172.21.15.32:0/2848100511 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff1ec055440 0x7ff1ec057900 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.578+0000 7ff20759e700 1 --2- 172.21.15.32:0/2848100511 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff2080b5f00 0x7ff208065130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.578+0000 7ff20759e700 1 --2- 172.21.15.32:0/2848100511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff2080ad510 0x7ff20813e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.585 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.578+0000 7ff20759e700 1 --2- 172.21.15.32:0/2848100511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2080af480 0x7ff208064c40 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.578+0000 7ff20759e700 1 -- 172.21.15.32:0/2848100511 >> 172.21.15.32:0/2848100511 conn(0x7ff2080604c0 msgr2=0x7ff208142e00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:03.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.578+0000 7ff20759e700 1 -- 172.21.15.32:0/2848100511 shutdown_connections 2020-12-05T17:41:03.586 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.578+0000 7ff20759e700 1 -- 172.21.15.32:0/2848100511 wait complete. 2020-12-05T17:41:03.596 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:41:03.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.702+0000 7f7852041700 1 Processor -- start 2020-12-05T17:41:03.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.702+0000 7f7852041700 1 -- start start 2020-12-05T17:41:03.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f7852041700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f784c0ab710 0x7f784c0af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:03.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f7852041700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f784c147400 0x7f784c145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:03.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f7852041700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f784c146a90 con 0x7f784c0b1fc0 2020-12-05T17:41:03.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f7852041700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f784c061e90 con 0x7f784c0ab710 2020-12-05T17:41:03.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f7852041700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784c0b0110 con 0x7f784c147400 2020-12-05T17:41:03.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f7851840700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f784c147400 0x7f784c145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:03.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f785083e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f784c0ab710 0x7f784c0af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:03.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f785083e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f784c0ab710 0x7f784c0af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:60088/0 (socket says 172.21.15.32:60088) 2020-12-05T17:41:03.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f785103f700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f784c0b1fc0 0x7f784c061780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:55486/0 (socket says 172.21.15.32:55486) 2020-12-05T17:41:03.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f7851840700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f784c147400 0x7f784c145d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:36990/0 (socket says 172.21.15.32:36990) 2020-12-05T17:41:03.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f785083e700 1 -- 172.21.15.32:0/2272024380 learned_addr learned my addr 172.21.15.32:0/2272024380 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:03.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f7851840700 1 -- 172.21.15.32:0/2272024380 >> v2:172.21.15.61:3300/0 conn(0x7f784c0ab710 msgr2=0x7f784c0af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:03.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f7851840700 1 --2- 172.21.15.32:0/2272024380 >> v2:172.21.15.61:3300/0 conn(0x7f784c0ab710 0x7f784c0af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f7851840700 1 -- 172.21.15.32:0/2272024380 >> v1:172.21.15.61:6789/0 conn(0x7f784c0b1fc0 legacy=0x7f784c061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:03.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f7851840700 1 -- 172.21.15.32:0/2272024380 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784c0b23a0 con 0x7f784c147400 2020-12-05T17:41:03.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f7851840700 1 --2- 172.21.15.32:0/2272024380 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f784c147400 0x7f784c145d10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f78440032b0 tx=0x7f78440091a0).ready entity=mon.2 client_cookie=855f2b7340c0bc97 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:03.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f7843fff700 1 -- 172.21.15.32:0/2272024380 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f784400b9a0 con 0x7f784c147400 2020-12-05T17:41:03.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f7843fff700 1 -- 172.21.15.32:0/2272024380 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784400bb00 con 0x7f784c147400 2020-12-05T17:41:03.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f7852041700 1 -- 172.21.15.32:0/2272024380 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f784c147400 msgr2=0x7f784c145d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:03.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f7843fff700 1 -- 172.21.15.32:0/2272024380 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f784400bc70 con 0x7f784c147400 2020-12-05T17:41:03.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f7852041700 1 --2- 172.21.15.32:0/2272024380 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f784c147400 0x7f784c145d10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f78440032b0 tx=0x7f78440091a0).stop 2020-12-05T17:41:03.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f7852041700 1 -- 172.21.15.32:0/2272024380 shutdown_connections 2020-12-05T17:41:03.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f7852041700 1 --2- 172.21.15.32:0/2272024380 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f784c147400 0x7f784c145d10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f7852041700 1 --2- 172.21.15.32:0/2272024380 >> v2:172.21.15.61:3300/0 conn(0x7f784c0ab710 0x7f784c0af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.719 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f7852041700 1 -- 172.21.15.32:0/2272024380 >> 172.21.15.32:0/2272024380 conn(0x7f784c0b5530 msgr2=0x7f784c0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:03.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f7852041700 1 -- 172.21.15.32:0/2272024380 shutdown_connections 2020-12-05T17:41:03.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.706+0000 7f7852041700 1 -- 172.21.15.32:0/2272024380 wait complete. 2020-12-05T17:41:03.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f7852041700 1 Processor -- start 2020-12-05T17:41:03.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f7852041700 1 -- start start 2020-12-05T17:41:03.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f7852041700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f784c0ab710 0x7f784c148600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:03.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f7852041700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f784c0b1fc0 0x7f784c0abb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:03.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f7852041700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f784c147400 0x7f784c0ac040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:03.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f7852041700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c0aeaa0 con 0x7f784c0b1fc0 2020-12-05T17:41:03.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f7852041700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c0ad990 con 0x7f784c0ab710 2020-12-05T17:41:03.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f7852041700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784c0ae800 con 0x7f784c147400 2020-12-05T17:41:03.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f785083e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f784c0b1fc0 0x7f784c0abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:03.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f785083e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f784c0b1fc0 0x7f784c0abb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40204/0 (socket says 172.21.15.32:40204) 2020-12-05T17:41:03.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f785083e700 1 -- 172.21.15.32:0/849616163 learned_addr learned my addr 172.21.15.32:0/849616163 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:03.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f7851840700 1 --2- 172.21.15.32:0/849616163 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f784c147400 0x7f784c0ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:03.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f785103f700 1 --2- 172.21.15.32:0/849616163 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f784c0ab710 0x7f784c148600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:03.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f785083e700 1 -- 172.21.15.32:0/849616163 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f784c147400 msgr2=0x7f784c0ac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:03.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f785083e700 1 --2- 172.21.15.32:0/849616163 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f784c147400 0x7f784c0ac040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f785083e700 1 -- 172.21.15.32:0/849616163 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f784c0ab710 msgr2=0x7f784c148600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:03.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f785083e700 1 --2- 172.21.15.32:0/849616163 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f784c0ab710 0x7f784c148600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f785083e700 1 -- 172.21.15.32:0/849616163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784c0ad560 con 0x7f784c0b1fc0 2020-12-05T17:41:03.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f785083e700 1 --2- 172.21.15.32:0/849616163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f784c0b1fc0 0x7f784c0abb50 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0x7f784800c000 tx=0x7f7848008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:03.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f78427fc700 1 -- 172.21.15.32:0/849616163 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f784800ee80 con 0x7f784c0b1fc0 2020-12-05T17:41:03.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f78427fc700 1 -- 172.21.15.32:0/849616163 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784800c820 con 0x7f784c0b1fc0 2020-12-05T17:41:03.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f7852041700 1 -- 172.21.15.32:0/849616163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f784c147e50 con 0x7f784c0b1fc0 2020-12-05T17:41:03.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f78427fc700 1 -- 172.21.15.32:0/849616163 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f784800cb80 con 0x7f784c0b1fc0 2020-12-05T17:41:03.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.710+0000 7f7852041700 1 -- 172.21.15.32:0/849616163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f784c14ad20 con 0x7f784c0b1fc0 2020-12-05T17:41:03.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.714+0000 7f7852041700 1 -- 172.21.15.32:0/849616163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f784c0add10 con 0x7f784c0b1fc0 2020-12-05T17:41:03.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.714+0000 7f78427fc700 1 -- 172.21.15.32:0/849616163 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7848022070 con 0x7f784c0b1fc0 2020-12-05T17:41:03.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.714+0000 7f78427fc700 1 --2- 172.21.15.32:0/849616163 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7834055440 0x7f7834057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:03.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.714+0000 7f78427fc700 1 -- 172.21.15.32:0/849616163 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f784806e470 con 0x7f784c0b1fc0 2020-12-05T17:41:03.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.714+0000 7f785103f700 1 --2- 172.21.15.32:0/849616163 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7834055440 0x7f7834057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:03.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.714+0000 7f785103f700 1 --2- 172.21.15.32:0/849616163 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7834055440 0x7f7834057900 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f783c00d000 tx=0x7f783c007b50).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:03.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.722+0000 7f78427fc700 1 -- 172.21.15.32:0/849616163 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7848044500 con 0x7f784c0b1fc0 2020-12-05T17:41:03.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.862+0000 7f7852041700 1 -- 172.21.15.32:0/849616163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f784c057050 con 0x7f784c0b1fc0 2020-12-05T17:41:03.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.866+0000 7f78427fc700 1 -- 172.21.15.32:0/849616163 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f7848047b20 con 0x7f784c0b1fc0 2020-12-05T17:41:03.871 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:03.871 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":128,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:41:03.280605+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:41:01.240356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2020-12-05T17:41:02.269469+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:41:03.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.866+0000 7f7852041700 1 -- 172.21.15.32:0/849616163 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7834055440 msgr2=0x7f7834057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:03.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.866+0000 7f7852041700 1 --2- 172.21.15.32:0/849616163 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7834055440 0x7f7834057900 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f783c00d000 tx=0x7f783c007b50).stop 2020-12-05T17:41:03.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.866+0000 7f7852041700 1 -- 172.21.15.32:0/849616163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f784c0b1fc0 msgr2=0x7f784c0abb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:03.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.866+0000 7f7852041700 1 --2- 172.21.15.32:0/849616163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f784c0b1fc0 0x7f784c0abb50 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0x7f784800c000 tx=0x7f7848008040).stop 2020-12-05T17:41:03.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.866+0000 7f7852041700 1 -- 172.21.15.32:0/849616163 shutdown_connections 2020-12-05T17:41:03.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.866+0000 7f7852041700 1 --2- 172.21.15.32:0/849616163 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7834055440 0x7f7834057900 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.866+0000 7f7852041700 1 --2- 172.21.15.32:0/849616163 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f784c147400 0x7f784c0ac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.874 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.866+0000 7f7852041700 1 --2- 172.21.15.32:0/849616163 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f784c0ab710 0x7f784c148600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.866+0000 7f7852041700 1 --2- 172.21.15.32:0/849616163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f784c0b1fc0 0x7f784c0abb50 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:03.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.866+0000 7f7852041700 1 -- 172.21.15.32:0/849616163 >> 172.21.15.32:0/849616163 conn(0x7f784c0b5530 msgr2=0x7f784c144c30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:03.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.866+0000 7f7852041700 1 -- 172.21.15.32:0/849616163 shutdown_connections 2020-12-05T17:41:03.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:03.866+0000 7f7852041700 1 -- 172.21.15.32:0/849616163 wait complete. 2020-12-05T17:41:03.886 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-12-05T17:41:04.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.006+0000 7fc52f59e700 1 Processor -- start 2020-12-05T17:41:04.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.006+0000 7fc52f59e700 1 -- start start 2020-12-05T17:41:04.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.006+0000 7fc52f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc53005f550 0x7fc5300acb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:04.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.006+0000 7fc52f59e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc530063040 0x7fc53013fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:04.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.006+0000 7fc52f59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc530067110 con 0x7fc5300af090 2020-12-05T17:41:04.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.006+0000 7fc52f59e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc53013f760 con 0x7fc530063040 2020-12-05T17:41:04.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.006+0000 7fc52f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc53013fa00 con 0x7fc53005f550 2020-12-05T17:41:04.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.006+0000 7fc52ed9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc5300af090 0x7fc5300af4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42862/0 (socket says 172.21.15.32:42862) 2020-12-05T17:41:04.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.006+0000 7fc52ed9d700 1 -- 172.21.15.32:0/1982476509 learned_addr learned my addr 172.21.15.32:0/1982476509 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:04.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.006+0000 7fc52dd9b700 1 --2- 172.21.15.32:0/1982476509 >> v2:172.21.15.61:3300/0 conn(0x7fc530063040 0x7fc53013fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:04.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.006+0000 7fc52e59c700 1 --2- 172.21.15.32:0/1982476509 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc53005f550 0x7fc5300acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:04.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.010+0000 7fc52d59a700 1 -- 172.21.15.32:0/1982476509 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616604904 0 0) 0x7fc530067110 con 0x7fc5300af090 2020-12-05T17:41:04.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.010+0000 7fc52d59a700 1 -- 172.21.15.32:0/1982476509 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc514003230 con 0x7fc5300af090 2020-12-05T17:41:04.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.010+0000 7fc52e59c700 1 -- 172.21.15.32:0/1982476509 >> v2:172.21.15.61:3300/0 conn(0x7fc530063040 msgr2=0x7fc53013fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:04.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.010+0000 7fc52e59c700 1 --2- 172.21.15.32:0/1982476509 >> v2:172.21.15.61:3300/0 conn(0x7fc530063040 0x7fc53013fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:04.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.010+0000 7fc52e59c700 1 -- 172.21.15.32:0/1982476509 >> v1:172.21.15.32:6789/0 conn(0x7fc5300af090 legacy=0x7fc5300af4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:04.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.010+0000 7fc52e59c700 1 -- 172.21.15.32:0/1982476509 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc530066d90 con 0x7fc53005f550 2020-12-05T17:41:04.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.010+0000 7fc52dd9b700 1 --2- 172.21.15.32:0/1982476509 >> v2:172.21.15.61:3300/0 conn(0x7fc530063040 0x7fc53013fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:04.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.010+0000 7fc52e59c700 1 --2- 172.21.15.32:0/1982476509 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc53005f550 0x7fc5300acb80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fc520002f30 tx=0x7fc520006dc0).ready entity=mon.2 client_cookie=9b4d4ba972120657 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:04.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.010+0000 7fc52d59a700 1 -- 172.21.15.32:0/1982476509 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5200096a0 con 0x7fc53005f550 2020-12-05T17:41:04.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.010+0000 7fc52d59a700 1 -- 172.21.15.32:0/1982476509 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc520009800 con 0x7fc53005f550 2020-12-05T17:41:04.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.010+0000 7fc52d59a700 1 -- 172.21.15.32:0/1982476509 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc520009b60 con 0x7fc53005f550 2020-12-05T17:41:04.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.010+0000 7fc52f59e700 1 -- 172.21.15.32:0/1982476509 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc53005f550 msgr2=0x7fc5300acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:04.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.010+0000 7fc52f59e700 1 --2- 172.21.15.32:0/1982476509 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc53005f550 0x7fc5300acb80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fc520002f30 tx=0x7fc520006dc0).stop 2020-12-05T17:41:04.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.010+0000 7fc52f59e700 1 -- 172.21.15.32:0/1982476509 shutdown_connections 2020-12-05T17:41:04.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.010+0000 7fc52f59e700 1 --2- 172.21.15.32:0/1982476509 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc53005f550 0x7fc5300acb80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:04.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.010+0000 7fc52f59e700 1 --2- 172.21.15.32:0/1982476509 >> v2:172.21.15.61:3300/0 conn(0x7fc530063040 0x7fc53013fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:04.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.010+0000 7fc52f59e700 1 -- 172.21.15.32:0/1982476509 >> 172.21.15.32:0/1982476509 conn(0x7fc5300620b0 msgr2=0x7fc530062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:04.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.010+0000 7fc52f59e700 1 -- 172.21.15.32:0/1982476509 shutdown_connections 2020-12-05T17:41:04.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.010+0000 7fc52f59e700 1 -- 172.21.15.32:0/1982476509 wait complete. 2020-12-05T17:41:04.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.014+0000 7fc52f59e700 1 Processor -- start 2020-12-05T17:41:04.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.014+0000 7fc52f59e700 1 -- start start 2020-12-05T17:41:04.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.014+0000 7fc52f59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc53005f550 0x7fc5300b2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.014+0000 7fc52f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc530063040 0x7fc53013d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.014+0000 7fc52f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5300af090 0x7fc53013ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.014+0000 7fc52f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc530150870 con 0x7fc530063040 2020-12-05T17:41:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.014+0000 7fc52f59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53013ded0 con 0x7fc53005f550 2020-12-05T17:41:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.014+0000 7fc52f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc53013a040 con 0x7fc5300af090 2020-12-05T17:41:04.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.014+0000 7fc52dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc530063040 0x7fc53013d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:04.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.014+0000 7fc52dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc530063040 0x7fc53013d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40218/0 (socket says 172.21.15.32:40218) 2020-12-05T17:41:04.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.014+0000 7fc52dd9b700 1 -- 172.21.15.32:0/158939185 learned_addr learned my addr 172.21.15.32:0/158939185 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:04.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.014+0000 7fc52e59c700 1 --2- 172.21.15.32:0/158939185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc53005f550 0x7fc5300b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:04.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.014+0000 7fc52ed9d700 1 --2- 172.21.15.32:0/158939185 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5300af090 0x7fc53013ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.014+0000 7fc52dd9b700 1 -- 172.21.15.32:0/158939185 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5300af090 msgr2=0x7fc53013ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.014+0000 7fc52dd9b700 1 --2- 172.21.15.32:0/158939185 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5300af090 0x7fc53013ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.014+0000 7fc52dd9b700 1 -- 172.21.15.32:0/158939185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc53005f550 msgr2=0x7fc5300b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.014+0000 7fc52dd9b700 1 --2- 172.21.15.32:0/158939185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc53005f550 0x7fc5300b2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.014+0000 7fc52dd9b700 1 -- 172.21.15.32:0/158939185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53013a2e0 con 0x7fc530063040 2020-12-05T17:41:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.014+0000 7fc52dd9b700 1 --2- 172.21.15.32:0/158939185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc530063040 0x7fc53013d9e0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 rx=0x7fc52400c000 tx=0x7fc5240091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.014+0000 7fc51f7fe700 1 -- 172.21.15.32:0/158939185 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc52400cc60 con 0x7fc530063040 2020-12-05T17:41:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.014+0000 7fc51f7fe700 1 -- 172.21.15.32:0/158939185 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52400cdc0 con 0x7fc530063040 2020-12-05T17:41:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.014+0000 7fc52f59e700 1 -- 172.21.15.32:0/158939185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc53013a580 con 0x7fc530063040 2020-12-05T17:41:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.014+0000 7fc51f7fe700 1 -- 172.21.15.32:0/158939185 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5240125e0 con 0x7fc530063040 2020-12-05T17:41:04.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.018+0000 7fc52f59e700 1 -- 172.21.15.32:0/158939185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc53013aa60 con 0x7fc530063040 2020-12-05T17:41:04.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.018+0000 7fc52f59e700 1 -- 172.21.15.32:0/158939185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc530057050 con 0x7fc530063040 2020-12-05T17:41:04.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.018+0000 7fc51f7fe700 1 -- 172.21.15.32:0/158939185 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc524022070 con 0x7fc530063040 2020-12-05T17:41:04.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.018+0000 7fc51f7fe700 1 --2- 172.21.15.32:0/158939185 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc50c0553b0 0x7fc50c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:04.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.018+0000 7fc51f7fe700 1 -- 172.21.15.32:0/158939185 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc52406ea60 con 0x7fc530063040 2020-12-05T17:41:04.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.018+0000 7fc52e59c700 1 --2- 172.21.15.32:0/158939185 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc50c0553b0 0x7fc50c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:04.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.018+0000 7fc52e59c700 1 --2- 172.21.15.32:0/158939185 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc50c0553b0 0x7fc50c057870 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7fc52001b000 tx=0x7fc520015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:04.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.022+0000 7fc51f7fe700 1 -- 172.21.15.32:0/158939185 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc524044c00 con 0x7fc530063040 2020-12-05T17:41:04.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.210+0000 7fc52f59e700 1 -- 172.21.15.32:0/158939185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc5300b1b90 con 0x7fc530063040 2020-12-05T17:41:04.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.286+0000 7fc51f7fe700 1 -- 172.21.15.32:0/158939185 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v62) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc524048220 con 0x7fc530063040 2020-12-05T17:41:04.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.290+0000 7fc52f59e700 1 -- 172.21.15.32:0/158939185 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc50c0553b0 msgr2=0x7fc50c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:04.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.290+0000 7fc52f59e700 1 --2- 172.21.15.32:0/158939185 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc50c0553b0 0x7fc50c057870 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7fc52001b000 tx=0x7fc520015000).stop 2020-12-05T17:41:04.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.290+0000 7fc52f59e700 1 -- 172.21.15.32:0/158939185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc530063040 msgr2=0x7fc53013d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:04.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.290+0000 7fc52f59e700 1 --2- 172.21.15.32:0/158939185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc530063040 0x7fc53013d9e0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 rx=0x7fc52400c000 tx=0x7fc5240091a0).stop 2020-12-05T17:41:04.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.290+0000 7fc52f59e700 1 -- 172.21.15.32:0/158939185 shutdown_connections 2020-12-05T17:41:04.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.290+0000 7fc52f59e700 1 --2- 172.21.15.32:0/158939185 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc50c0553b0 0x7fc50c057870 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:04.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.290+0000 7fc52f59e700 1 --2- 172.21.15.32:0/158939185 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5300af090 0x7fc53013ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:04.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.290+0000 7fc52f59e700 1 --2- 172.21.15.32:0/158939185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc53005f550 0x7fc5300b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:04.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.290+0000 7fc52f59e700 1 --2- 172.21.15.32:0/158939185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc530063040 0x7fc53013d9e0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:04.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.290+0000 7fc52f59e700 1 -- 172.21.15.32:0/158939185 >> 172.21.15.32:0/158939185 conn(0x7fc5300620b0 msgr2=0x7fc530067460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:04.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.290+0000 7fc52f59e700 1 -- 172.21.15.32:0/158939185 shutdown_connections 2020-12-05T17:41:04.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.290+0000 7fc52f59e700 1 -- 172.21.15.32:0/158939185 wait complete. 2020-12-05T17:41:04.311 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:41:04.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.418+0000 7fba6956b700 1 Processor -- start 2020-12-05T17:41:04.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.418+0000 7fba6956b700 1 -- start start 2020-12-05T17:41:04.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.418+0000 7fba6956b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fba6413de30 0x7fba64143b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:04.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.418+0000 7fba6956b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba64063040 con 0x7fba6405f6f0 2020-12-05T17:41:04.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.418+0000 7fba6956b700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba640b24a0 con 0x7fba640af810 2020-12-05T17:41:04.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.418+0000 7fba6956b700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba6405fad0 con 0x7fba6413de30 2020-12-05T17:41:04.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.418+0000 7fba63fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fba6405f6f0 0x7fba640b1d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42872/0 (socket says 172.21.15.32:42872) 2020-12-05T17:41:04.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.418+0000 7fba63fff700 1 -- 172.21.15.32:0/798170951 learned_addr learned my addr 172.21.15.32:0/798170951 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:04.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.418+0000 7fba68d6a700 1 --2- 172.21.15.32:0/798170951 >> v2:172.21.15.61:3300/0 conn(0x7fba6413de30 0x7fba64143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.422+0000 7fba62ffd700 1 -- 172.21.15.32:0/798170951 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1334178488 0 0) 0x7fba64063040 con 0x7fba6405f6f0 2020-12-05T17:41:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.422+0000 7fba62ffd700 1 -- 172.21.15.32:0/798170951 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba44003230 con 0x7fba6405f6f0 2020-12-05T17:41:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.422+0000 7fba68d6a700 1 -- 172.21.15.32:0/798170951 >> v1:172.21.15.61:6789/0 conn(0x7fba640af810 legacy=0x7fba6413d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.422+0000 7fba68d6a700 1 -- 172.21.15.32:0/798170951 >> v1:172.21.15.32:6789/0 conn(0x7fba6405f6f0 legacy=0x7fba640b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:04.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.422+0000 7fba68d6a700 1 -- 172.21.15.32:0/798170951 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba640b3030 con 0x7fba6413de30 2020-12-05T17:41:04.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.422+0000 7fba68d6a700 1 --2- 172.21.15.32:0/798170951 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba6413de30 0x7fba64143b90 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fba5c002f30 tx=0x7fba5c006dc0).ready entity=mon.1 client_cookie=b314dc72f0d95720 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:04.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.422+0000 7fba62ffd700 1 -- 172.21.15.32:0/798170951 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba5c009800 con 0x7fba6413de30 2020-12-05T17:41:04.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.422+0000 7fba62ffd700 1 -- 172.21.15.32:0/798170951 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5c009960 con 0x7fba6413de30 2020-12-05T17:41:04.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.422+0000 7fba62ffd700 1 -- 172.21.15.32:0/798170951 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba5c009cc0 con 0x7fba6413de30 2020-12-05T17:41:04.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.422+0000 7fba6956b700 1 -- 172.21.15.32:0/798170951 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba6413de30 msgr2=0x7fba64143b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:04.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.422+0000 7fba6956b700 1 --2- 172.21.15.32:0/798170951 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba6413de30 0x7fba64143b90 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fba5c002f30 tx=0x7fba5c006dc0).stop 2020-12-05T17:41:04.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.422+0000 7fba6956b700 1 -- 172.21.15.32:0/798170951 shutdown_connections 2020-12-05T17:41:04.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.422+0000 7fba6956b700 1 --2- 172.21.15.32:0/798170951 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba6413de30 0x7fba64143b90 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:04.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.422+0000 7fba6956b700 1 -- 172.21.15.32:0/798170951 >> 172.21.15.32:0/798170951 conn(0x7fba640604c0 msgr2=0x7fba6405c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:04.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.422+0000 7fba6956b700 1 -- 172.21.15.32:0/798170951 shutdown_connections 2020-12-05T17:41:04.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.422+0000 7fba6956b700 1 -- 172.21.15.32:0/798170951 wait complete. 2020-12-05T17:41:04.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba6956b700 1 Processor -- start 2020-12-05T17:41:04.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba6956b700 1 -- start start 2020-12-05T17:41:04.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba6956b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fba6405f6f0 0x7fba64147240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:04.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba6956b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba640af810 0x7fba640b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:04.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba6956b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba6413de30 0x7fba640b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:04.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba6956b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6413a290 con 0x7fba640af810 2020-12-05T17:41:04.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba6956b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba640b4290 con 0x7fba6413de30 2020-12-05T17:41:04.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba6956b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba641528f0 con 0x7fba6405f6f0 2020-12-05T17:41:04.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba637fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba640af810 0x7fba640b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:04.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba637fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba640af810 0x7fba640b4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40232/0 (socket says 172.21.15.32:40232) 2020-12-05T17:41:04.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba637fe700 1 -- 172.21.15.32:0/88085190 learned_addr learned my addr 172.21.15.32:0/88085190 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:04.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba63fff700 1 --2- 172.21.15.32:0/88085190 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fba6405f6f0 0x7fba64147240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:04.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba68d6a700 1 --2- 172.21.15.32:0/88085190 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba6413de30 0x7fba640b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:04.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba637fe700 1 -- 172.21.15.32:0/88085190 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fba6405f6f0 msgr2=0x7fba64147240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:04.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba637fe700 1 --2- 172.21.15.32:0/88085190 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fba6405f6f0 0x7fba64147240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:04.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba637fe700 1 -- 172.21.15.32:0/88085190 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba6413de30 msgr2=0x7fba640b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:04.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba637fe700 1 --2- 172.21.15.32:0/88085190 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba6413de30 0x7fba640b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:04.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba637fe700 1 -- 172.21.15.32:0/88085190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba640b5310 con 0x7fba640af810 2020-12-05T17:41:04.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba68d6a700 1 --2- 172.21.15.32:0/88085190 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba6413de30 0x7fba640b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:04.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba637fe700 1 --2- 172.21.15.32:0/88085190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba640af810 0x7fba640b4e20 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7fba640afcb0 tx=0x7fba4c007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:04.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba617fa700 1 -- 172.21.15.32:0/88085190 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba4c00e4a0 con 0x7fba640af810 2020-12-05T17:41:04.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba617fa700 1 -- 172.21.15.32:0/88085190 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4c00e600 con 0x7fba640af810 2020-12-05T17:41:04.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba6956b700 1 -- 172.21.15.32:0/88085190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba640b3f50 con 0x7fba640af810 2020-12-05T17:41:04.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba617fa700 1 -- 172.21.15.32:0/88085190 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba4c00e960 con 0x7fba640af810 2020-12-05T17:41:04.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.426+0000 7fba6956b700 1 -- 172.21.15.32:0/88085190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba64064830 con 0x7fba640af810 2020-12-05T17:41:04.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.430+0000 7fba6956b700 1 -- 172.21.15.32:0/88085190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba640b4450 con 0x7fba640af810 2020-12-05T17:41:04.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.430+0000 7fba617fa700 1 -- 172.21.15.32:0/88085190 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fba4c019030 con 0x7fba640af810 2020-12-05T17:41:04.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.430+0000 7fba617fa700 1 --2- 172.21.15.32:0/88085190 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fba48055440 0x7fba48057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:04.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.430+0000 7fba617fa700 1 -- 172.21.15.32:0/88085190 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fba4c06d710 con 0x7fba640af810 2020-12-05T17:41:04.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.430+0000 7fba63fff700 1 --2- 172.21.15.32:0/88085190 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fba48055440 0x7fba48057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:04.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.430+0000 7fba63fff700 1 --2- 172.21.15.32:0/88085190 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fba48055440 0x7fba48057900 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fba5400b000 tx=0x7fba54009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:04.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.434+0000 7fba617fa700 1 -- 172.21.15.32:0/88085190 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fba4c0438b0 con 0x7fba640af810 2020-12-05T17:41:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.618+0000 7fba6956b700 1 -- 172.21.15.32:0/88085190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba64152ab0 con 0x7fba640af810 2020-12-05T17:41:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.618+0000 7fba617fa700 1 -- 172.21.15.32:0/88085190 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+2142 (secure 0 0 0) 0x7fba4c046ed0 con 0x7fba640af810 2020-12-05T17:41:04.624 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:04.624 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":62,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6830,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/3985915039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3985915039},{"type":"v1","addr":"172.21.15.61:6833","nonce":3985915039}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":60}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:41:03.280787+0000","modified":"2020-12-05T17:41:04.283186+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7938":{"gid":7938,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1585653076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1585653076},{"type":"v1","addr":"172.21.15.32:6835","nonce":1585653076}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2020-12-05T17:41:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.618+0000 7fba6956b700 1 -- 172.21.15.32:0/88085190 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fba48055440 msgr2=0x7fba48057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.618+0000 7fba6956b700 1 --2- 172.21.15.32:0/88085190 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fba48055440 0x7fba48057900 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fba5400b000 tx=0x7fba54009040).stop 2020-12-05T17:41:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.618+0000 7fba6956b700 1 -- 172.21.15.32:0/88085190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba640af810 msgr2=0x7fba640b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.618+0000 7fba6956b700 1 --2- 172.21.15.32:0/88085190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba640af810 0x7fba640b4e20 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7fba640afcb0 tx=0x7fba4c007c20).stop 2020-12-05T17:41:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.618+0000 7fba6956b700 1 -- 172.21.15.32:0/88085190 shutdown_connections 2020-12-05T17:41:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.618+0000 7fba6956b700 1 --2- 172.21.15.32:0/88085190 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fba6413de30 0x7fba640b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.618+0000 7fba6956b700 1 --2- 172.21.15.32:0/88085190 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fba48055440 0x7fba48057900 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.618+0000 7fba6956b700 1 --2- 172.21.15.32:0/88085190 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fba6405f6f0 0x7fba64147240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.618+0000 7fba6956b700 1 --2- 172.21.15.32:0/88085190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba640af810 0x7fba640b4e20 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.618+0000 7fba6956b700 1 -- 172.21.15.32:0/88085190 >> 172.21.15.32:0/88085190 conn(0x7fba640604c0 msgr2=0x7fba640b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.622+0000 7fba6956b700 1 -- 172.21.15.32:0/88085190 shutdown_connections 2020-12-05T17:41:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.622+0000 7fba6956b700 1 -- 172.21.15.32:0/88085190 wait complete. 2020-12-05T17:41:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 62 2020-12-05T17:41:04.638 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:41:04.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.742+0000 7f45073a2700 1 Processor -- start 2020-12-05T17:41:04.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.742+0000 7f45073a2700 1 -- start start 2020-12-05T17:41:04.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.742+0000 7f45073a2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45000b1980 0x7f4500146580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:04.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.742+0000 7f45073a2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4500143000 con 0x7f4500146a70 2020-12-05T17:41:04.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.742+0000 7f45073a2700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f450005fad0 con 0x7f450005f6f0 2020-12-05T17:41:04.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.742+0000 7f45073a2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45000b2f80 con 0x7f45000b1980 2020-12-05T17:41:04.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.742+0000 7f4505b9f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45000b1980 0x7f4500146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:04.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.742+0000 7f4506ba1700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4500146a70 0x7f4500066070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42890/0 (socket says 172.21.15.32:42890) 2020-12-05T17:41:04.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.742+0000 7f4506ba1700 1 -- 172.21.15.32:0/414085005 learned_addr learned my addr 172.21.15.32:0/414085005 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:04.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.742+0000 7f4505b9f700 1 -- 172.21.15.32:0/414085005 >> v1:172.21.15.61:6789/0 conn(0x7f450005f6f0 legacy=0x7f45001428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:04.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.742+0000 7f4505b9f700 1 -- 172.21.15.32:0/414085005 >> v1:172.21.15.32:6789/0 conn(0x7f4500146a70 legacy=0x7f4500066070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:04.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.742+0000 7f450539e700 1 -- 172.21.15.32:0/414085005 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 820992374 0 0) 0x7f450005fad0 con 0x7f450005f6f0 2020-12-05T17:41:04.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.742+0000 7f4505b9f700 1 -- 172.21.15.32:0/414085005 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4500145e40 con 0x7f45000b1980 2020-12-05T17:41:04.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.742+0000 7f4505b9f700 1 --2- 172.21.15.32:0/414085005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45000b1980 0x7f4500146580 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7f44f8001450 tx=0x7f44f0006dc0).ready entity=mon.0 client_cookie=56aa8e7254390b server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:04.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.742+0000 7f450539e700 1 -- 172.21.15.32:0/414085005 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44f00096b0 con 0x7f45000b1980 2020-12-05T17:41:04.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.742+0000 7f450539e700 1 -- 172.21.15.32:0/414085005 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f0009810 con 0x7f45000b1980 2020-12-05T17:41:04.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.742+0000 7f450539e700 1 -- 172.21.15.32:0/414085005 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44f0009b70 con 0x7f45000b1980 2020-12-05T17:41:04.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.742+0000 7f45073a2700 1 -- 172.21.15.32:0/414085005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45000b1980 msgr2=0x7f4500146580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:04.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.742+0000 7f45073a2700 1 --2- 172.21.15.32:0/414085005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45000b1980 0x7f4500146580 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7f44f8001450 tx=0x7f44f0006dc0).stop 2020-12-05T17:41:04.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.746+0000 7f45073a2700 1 -- 172.21.15.32:0/414085005 shutdown_connections 2020-12-05T17:41:04.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.746+0000 7f45073a2700 1 --2- 172.21.15.32:0/414085005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45000b1980 0x7f4500146580 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:04.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.746+0000 7f45073a2700 1 -- 172.21.15.32:0/414085005 >> 172.21.15.32:0/414085005 conn(0x7f45000604c0 msgr2=0x7f450005c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:04.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.746+0000 7f45073a2700 1 -- 172.21.15.32:0/414085005 shutdown_connections 2020-12-05T17:41:04.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.746+0000 7f45073a2700 1 -- 172.21.15.32:0/414085005 wait complete. 2020-12-05T17:41:04.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.746+0000 7f45073a2700 1 Processor -- start 2020-12-05T17:41:04.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.746+0000 7f45073a2700 1 -- start start 2020-12-05T17:41:04.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.746+0000 7f45073a2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f450005f6f0 0x7f4500113570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:04.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.746+0000 7f45073a2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45000b1980 0x7f4500140370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:04.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.746+0000 7f45073a2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4500146a70 0x7f4500140860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:04.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.746+0000 7f45073a2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45000b6520 con 0x7f45000b1980 2020-12-05T17:41:04.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.746+0000 7f45073a2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4500063fb0 con 0x7f4500146a70 2020-12-05T17:41:04.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.746+0000 7f45073a2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4500140d50 con 0x7f450005f6f0 2020-12-05T17:41:04.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.750+0000 7f4505b9f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45000b1980 0x7f4500140370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:04.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.750+0000 7f4505b9f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45000b1980 0x7f4500140370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40246/0 (socket says 172.21.15.32:40246) 2020-12-05T17:41:04.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.750+0000 7f4505b9f700 1 -- 172.21.15.32:0/135514227 learned_addr learned my addr 172.21.15.32:0/135514227 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:04.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.750+0000 7f45063a0700 1 --2- 172.21.15.32:0/135514227 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f450005f6f0 0x7f4500113570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:04.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.750+0000 7f4506ba1700 1 --2- 172.21.15.32:0/135514227 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4500146a70 0x7f4500140860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:04.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.750+0000 7f4505b9f700 1 -- 172.21.15.32:0/135514227 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f450005f6f0 msgr2=0x7f4500113570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:04.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.750+0000 7f4505b9f700 1 --2- 172.21.15.32:0/135514227 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f450005f6f0 0x7f4500113570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:04.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.750+0000 7f4505b9f700 1 -- 172.21.15.32:0/135514227 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4500146a70 msgr2=0x7f4500140860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:04.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.750+0000 7f4505b9f700 1 --2- 172.21.15.32:0/135514227 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4500146a70 0x7f4500140860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:04.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.750+0000 7f4505b9f700 1 -- 172.21.15.32:0/135514227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4500064450 con 0x7f45000b1980 2020-12-05T17:41:04.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.750+0000 7f4505b9f700 1 --2- 172.21.15.32:0/135514227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45000b1980 0x7f4500140370 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7f44f8001450 tx=0x7f44f0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:04.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.750+0000 7f44ef7fe700 1 -- 172.21.15.32:0/135514227 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44f0011b80 con 0x7f45000b1980 2020-12-05T17:41:04.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.750+0000 7f44ef7fe700 1 -- 172.21.15.32:0/135514227 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f0011ce0 con 0x7f45000b1980 2020-12-05T17:41:04.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.750+0000 7f45073a2700 1 -- 172.21.15.32:0/135514227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45000b6160 con 0x7f45000b1980 2020-12-05T17:41:04.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.750+0000 7f44ef7fe700 1 -- 172.21.15.32:0/135514227 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44f001d640 con 0x7f45000b1980 2020-12-05T17:41:04.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.750+0000 7f45073a2700 1 -- 172.21.15.32:0/135514227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4500065b50 con 0x7f45000b1980 2020-12-05T17:41:04.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.750+0000 7f45073a2700 1 -- 172.21.15.32:0/135514227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4500057050 con 0x7f45000b1980 2020-12-05T17:41:04.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.750+0000 7f44ef7fe700 1 -- 172.21.15.32:0/135514227 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f44f0024030 con 0x7f45000b1980 2020-12-05T17:41:04.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.750+0000 7f44ef7fe700 1 --2- 172.21.15.32:0/135514227 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f44e00553b0 0x7f44e0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:04.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.754+0000 7f44ef7fe700 1 -- 172.21.15.32:0/135514227 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f44f006f2e0 con 0x7f45000b1980 2020-12-05T17:41:04.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.754+0000 7f45063a0700 1 --2- 172.21.15.32:0/135514227 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f44e00553b0 0x7f44e0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:04.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.754+0000 7f45063a0700 1 --2- 172.21.15.32:0/135514227 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f44e00553b0 0x7f44e0057870 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f44fc00d000 tx=0x7f44fc007a60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:04.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.758+0000 7f44ef7fe700 1 -- 172.21.15.32:0/135514227 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f44f0045480 con 0x7f45000b1980 2020-12-05T17:41:04.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.902+0000 7f45073a2700 1 -- 172.21.15.32:0/135514227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4500064120 con 0x7f45000b1980 2020-12-05T17:41:04.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.906+0000 7f44ef7fe700 1 -- 172.21.15.32:0/135514227 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f44f0022020 con 0x7f45000b1980 2020-12-05T17:41:04.911 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:04.911 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":129,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:41:04.284002+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:41:01.240356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2020-12-05T17:41:02.269469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:41:04.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.906+0000 7f45073a2700 1 -- 172.21.15.32:0/135514227 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f44e00553b0 msgr2=0x7f44e0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:04.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.906+0000 7f45073a2700 1 --2- 172.21.15.32:0/135514227 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f44e00553b0 0x7f44e0057870 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f44fc00d000 tx=0x7f44fc007a60).stop 2020-12-05T17:41:04.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.906+0000 7f45073a2700 1 -- 172.21.15.32:0/135514227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45000b1980 msgr2=0x7f4500140370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:04.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.906+0000 7f45073a2700 1 --2- 172.21.15.32:0/135514227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45000b1980 0x7f4500140370 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7f44f8001450 tx=0x7f44f0015000).stop 2020-12-05T17:41:04.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.906+0000 7f45073a2700 1 -- 172.21.15.32:0/135514227 shutdown_connections 2020-12-05T17:41:04.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.906+0000 7f45073a2700 1 --2- 172.21.15.32:0/135514227 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4500146a70 0x7f4500140860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:04.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.906+0000 7f45073a2700 1 --2- 172.21.15.32:0/135514227 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f44e00553b0 0x7f44e0057870 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:04.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.906+0000 7f45073a2700 1 --2- 172.21.15.32:0/135514227 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f450005f6f0 0x7f4500113570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:04.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.906+0000 7f45073a2700 1 --2- 172.21.15.32:0/135514227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45000b1980 0x7f4500140370 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:04.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.906+0000 7f45073a2700 1 -- 172.21.15.32:0/135514227 >> 172.21.15.32:0/135514227 conn(0x7f45000604c0 msgr2=0x7f4500143d40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:04.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.906+0000 7f45073a2700 1 -- 172.21.15.32:0/135514227 shutdown_connections 2020-12-05T17:41:04.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:04.906+0000 7f45073a2700 1 -- 172.21.15.32:0/135514227 wait complete. 2020-12-05T17:41:04.931 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:41:05.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.046+0000 7ff68759e700 1 Processor -- start 2020-12-05T17:41:05.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.046+0000 7ff68759e700 1 -- start start 2020-12-05T17:41:05.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.050+0000 7ff68759e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff6880611b0 0x7ff688153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:05.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.050+0000 7ff68759e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6880b0240 0x7ff6880ae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:05.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.050+0000 7ff68759e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff688080a90 con 0x7ff68814c340 2020-12-05T17:41:05.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.050+0000 7ff68759e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff688152860 con 0x7ff6880611b0 2020-12-05T17:41:05.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.050+0000 7ff68759e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff688152b00 con 0x7ff6880b0240 2020-12-05T17:41:05.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.050+0000 7ff686d9d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff6880611b0 0x7ff688153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:05.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.050+0000 7ff68659c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6880b0240 0x7ff6880ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:05.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.050+0000 7ff686d9d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff6880611b0 0x7ff688153240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:60144/0 (socket says 172.21.15.32:60144) 2020-12-05T17:41:05.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.050+0000 7ff686d9d700 1 -- 172.21.15.32:0/2778239672 learned_addr learned my addr 172.21.15.32:0/2778239672 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:05.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.050+0000 7ff68cecc700 1 --1- 172.21.15.32:0/2778239672 >> v1:172.21.15.61:6789/0 conn(0x7ff68814c340 0x7ff68814f0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:55542/0 (socket says 172.21.15.32:55542) 2020-12-05T17:41:05.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.050+0000 7ff68659c700 1 -- 172.21.15.32:0/2778239672 >> v2:172.21.15.61:3300/0 conn(0x7ff6880611b0 msgr2=0x7ff688153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:05.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.050+0000 7ff68659c700 1 --2- 172.21.15.32:0/2778239672 >> v2:172.21.15.61:3300/0 conn(0x7ff6880611b0 0x7ff688153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.050+0000 7ff68659c700 1 -- 172.21.15.32:0/2778239672 >> v1:172.21.15.61:6789/0 conn(0x7ff68814c340 legacy=0x7ff68814f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:05.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.050+0000 7ff68659c700 1 -- 172.21.15.32:0/2778239672 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff688152da0 con 0x7ff6880b0240 2020-12-05T17:41:05.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.050+0000 7ff68659c700 1 --2- 172.21.15.32:0/2778239672 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6880b0240 0x7ff6880ae860 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7ff67c002f30 tx=0x7ff67c006dc0).ready entity=mon.2 client_cookie=a8f0d4c0927ebba0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:05.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.050+0000 7ff685d9b700 1 -- 172.21.15.32:0/2778239672 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff67c009760 con 0x7ff6880b0240 2020-12-05T17:41:05.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.050+0000 7ff685d9b700 1 -- 172.21.15.32:0/2778239672 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67c0098c0 con 0x7ff6880b0240 2020-12-05T17:41:05.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.050+0000 7ff685d9b700 1 -- 172.21.15.32:0/2778239672 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff67c009c20 con 0x7ff6880b0240 2020-12-05T17:41:05.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.050+0000 7ff68759e700 1 -- 172.21.15.32:0/2778239672 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6880b0240 msgr2=0x7ff6880ae860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:05.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.050+0000 7ff68759e700 1 --2- 172.21.15.32:0/2778239672 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6880b0240 0x7ff6880ae860 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7ff67c002f30 tx=0x7ff67c006dc0).stop 2020-12-05T17:41:05.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.050+0000 7ff68759e700 1 -- 172.21.15.32:0/2778239672 shutdown_connections 2020-12-05T17:41:05.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.050+0000 7ff68759e700 1 --2- 172.21.15.32:0/2778239672 >> v2:172.21.15.61:3300/0 conn(0x7ff6880611b0 0x7ff688153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff68759e700 1 --2- 172.21.15.32:0/2778239672 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6880b0240 0x7ff6880ae860 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff68759e700 1 -- 172.21.15.32:0/2778239672 >> 172.21.15.32:0/2778239672 conn(0x7ff6880b5f90 msgr2=0x7ff6880b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:05.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff68759e700 1 -- 172.21.15.32:0/2778239672 shutdown_connections 2020-12-05T17:41:05.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff68759e700 1 -- 172.21.15.32:0/2778239672 wait complete. 2020-12-05T17:41:05.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff68759e700 1 Processor -- start 2020-12-05T17:41:05.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff68759e700 1 -- start start 2020-12-05T17:41:05.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff68759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6880611b0 0x7ff6880b0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:05.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff68759e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6880b0240 0x7ff68814ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:05.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff68759e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff68814c340 0x7ff688065910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:05.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff68759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff688065490 con 0x7ff6880611b0 2020-12-05T17:41:05.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff68759e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6880651f0 con 0x7ff68814c340 2020-12-05T17:41:05.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff68759e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff688159870 con 0x7ff6880b0240 2020-12-05T17:41:05.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff686d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6880611b0 0x7ff6880b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:05.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff686d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6880611b0 0x7ff6880b0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40260/0 (socket says 172.21.15.32:40260) 2020-12-05T17:41:05.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff686d9d700 1 -- 172.21.15.32:0/991256604 learned_addr learned my addr 172.21.15.32:0/991256604 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:05.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff68659c700 1 --2- 172.21.15.32:0/991256604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6880b0240 0x7ff68814ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:05.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff68cecc700 1 --2- 172.21.15.32:0/991256604 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff68814c340 0x7ff688065910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:05.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff686d9d700 1 -- 172.21.15.32:0/991256604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6880b0240 msgr2=0x7ff68814ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:05.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff686d9d700 1 --2- 172.21.15.32:0/991256604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6880b0240 0x7ff68814ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff686d9d700 1 -- 172.21.15.32:0/991256604 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff68814c340 msgr2=0x7ff688065910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:05.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff686d9d700 1 --2- 172.21.15.32:0/991256604 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff68814c340 0x7ff688065910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff686d9d700 1 -- 172.21.15.32:0/991256604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff688157c30 con 0x7ff6880611b0 2020-12-05T17:41:05.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff68659c700 1 --2- 172.21.15.32:0/991256604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6880b0240 0x7ff68814ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:05.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff68cecc700 1 --2- 172.21.15.32:0/991256604 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff68814c340 0x7ff688065910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:05.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff686d9d700 1 --2- 172.21.15.32:0/991256604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6880611b0 0x7ff6880b0930 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 rx=0x7ff67800c000 tx=0x7ff678008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:05.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.054+0000 7ff6777fe700 1 -- 172.21.15.32:0/991256604 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff67800c820 con 0x7ff6880611b0 2020-12-05T17:41:05.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.058+0000 7ff6777fe700 1 -- 172.21.15.32:0/991256604 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67800c980 con 0x7ff6880611b0 2020-12-05T17:41:05.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.058+0000 7ff68759e700 1 -- 172.21.15.32:0/991256604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68814f1c0 con 0x7ff6880611b0 2020-12-05T17:41:05.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.058+0000 7ff6777fe700 1 -- 172.21.15.32:0/991256604 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff67800cce0 con 0x7ff6880611b0 2020-12-05T17:41:05.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.058+0000 7ff68759e700 1 -- 172.21.15.32:0/991256604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff688064f30 con 0x7ff6880611b0 2020-12-05T17:41:05.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.058+0000 7ff68759e700 1 -- 172.21.15.32:0/991256604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff688066d20 con 0x7ff6880611b0 2020-12-05T17:41:05.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.058+0000 7ff6777fe700 1 -- 172.21.15.32:0/991256604 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff678022070 con 0x7ff6880611b0 2020-12-05T17:41:05.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.058+0000 7ff6777fe700 1 --2- 172.21.15.32:0/991256604 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff66c0553b0 0x7ff66c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:05.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.058+0000 7ff6777fe700 1 -- 172.21.15.32:0/991256604 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff67806eae0 con 0x7ff6880611b0 2020-12-05T17:41:05.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.058+0000 7ff68659c700 1 --2- 172.21.15.32:0/991256604 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff66c0553b0 0x7ff66c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:05.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.058+0000 7ff68659c700 1 --2- 172.21.15.32:0/991256604 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff66c0553b0 0x7ff66c057870 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7ff67c018000 tx=0x7ff67c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:05.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.066+0000 7ff6777fe700 1 -- 172.21.15.32:0/991256604 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff678044c80 con 0x7ff6880611b0 2020-12-05T17:41:05.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.262+0000 7ff68759e700 1 -- 172.21.15.32:0/991256604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7ff688057050 con 0x7ff6880611b0 2020-12-05T17:41:05.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.266+0000 7ff6777fe700 1 -- 172.21.15.32:0/991256604 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v38) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff6780290e0 con 0x7ff6880611b0 2020-12-05T17:41:05.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.270+0000 7ff68759e700 1 -- 172.21.15.32:0/991256604 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff66c0553b0 msgr2=0x7ff66c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:05.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.270+0000 7ff68759e700 1 --2- 172.21.15.32:0/991256604 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff66c0553b0 0x7ff66c057870 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7ff67c018000 tx=0x7ff67c015000).stop 2020-12-05T17:41:05.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.270+0000 7ff68759e700 1 -- 172.21.15.32:0/991256604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6880611b0 msgr2=0x7ff6880b0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:05.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.270+0000 7ff68759e700 1 --2- 172.21.15.32:0/991256604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6880611b0 0x7ff6880b0930 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 rx=0x7ff67800c000 tx=0x7ff678008040).stop 2020-12-05T17:41:05.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.270+0000 7ff68759e700 1 -- 172.21.15.32:0/991256604 shutdown_connections 2020-12-05T17:41:05.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.270+0000 7ff68759e700 1 --2- 172.21.15.32:0/991256604 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff68814c340 0x7ff688065910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.270+0000 7ff68759e700 1 --2- 172.21.15.32:0/991256604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6880611b0 0x7ff6880b0930 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.270+0000 7ff68759e700 1 --2- 172.21.15.32:0/991256604 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff66c0553b0 0x7ff66c057870 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.270+0000 7ff68759e700 1 --2- 172.21.15.32:0/991256604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6880b0240 0x7ff68814ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.270+0000 7ff68759e700 1 -- 172.21.15.32:0/991256604 >> 172.21.15.32:0/991256604 conn(0x7ff6880b5f90 msgr2=0x7ff6880b4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:05.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.270+0000 7ff68759e700 1 -- 172.21.15.32:0/991256604 shutdown_connections 2020-12-05T17:41:05.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.270+0000 7ff68759e700 1 -- 172.21.15.32:0/991256604 wait complete. 2020-12-05T17:41:05.279 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2020-12-05T17:41:05.292 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:41:05.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6b59e700 1 Processor -- start 2020-12-05T17:41:05.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6b59e700 1 -- start start 2020-12-05T17:41:05.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6b59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a6c0611b0 0x7f9a6c153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:05.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6b59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a6c0b0240 0x7f9a6c0ae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:05.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6b59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a6c080a90 con 0x7f9a6c14c340 2020-12-05T17:41:05.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6b59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a6c152860 con 0x7f9a6c0b0240 2020-12-05T17:41:05.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6b59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c152b00 con 0x7f9a6c0611b0 2020-12-05T17:41:05.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a6c0b0240 0x7f9a6c0ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:05.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a6c0b0240 0x7f9a6c0ae860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40266/0 (socket says 172.21.15.32:40266) 2020-12-05T17:41:05.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6259c700 1 -- 172.21.15.32:0/2157729341 learned_addr learned my addr 172.21.15.32:0/2157729341 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:05.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6ad9d700 1 --2- 172.21.15.32:0/2157729341 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a6c0611b0 0x7f9a6c153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:05.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6259c700 1 -- 172.21.15.32:0/2157729341 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a6c0611b0 msgr2=0x7f9a6c153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:05.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6259c700 1 --2- 172.21.15.32:0/2157729341 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a6c0611b0 0x7f9a6c153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6259c700 1 -- 172.21.15.32:0/2157729341 >> v1:172.21.15.32:6789/0 conn(0x7f9a6c14c340 legacy=0x7f9a6c14f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:05.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6259c700 1 -- 172.21.15.32:0/2157729341 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6c152da0 con 0x7f9a6c0b0240 2020-12-05T17:41:05.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6259c700 1 --2- 172.21.15.32:0/2157729341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6c0b0240 0x7f9a6c0ae860 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7f9a54002f30 tx=0x7f9a54006e80).ready entity=mon.0 client_cookie=ec38108778ade21b server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:05.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6a59c700 1 -- 172.21.15.32:0/2157729341 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a540097f0 con 0x7f9a6c0b0240 2020-12-05T17:41:05.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6a59c700 1 -- 172.21.15.32:0/2157729341 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a54009950 con 0x7f9a6c0b0240 2020-12-05T17:41:05.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6a59c700 1 -- 172.21.15.32:0/2157729341 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a54009cb0 con 0x7f9a6c0b0240 2020-12-05T17:41:05.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6b59e700 1 -- 172.21.15.32:0/2157729341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6c0b0240 msgr2=0x7f9a6c0ae860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:05.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6b59e700 1 --2- 172.21.15.32:0/2157729341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6c0b0240 0x7f9a6c0ae860 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7f9a54002f30 tx=0x7f9a54006e80).stop 2020-12-05T17:41:05.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6b59e700 1 -- 172.21.15.32:0/2157729341 shutdown_connections 2020-12-05T17:41:05.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6b59e700 1 --2- 172.21.15.32:0/2157729341 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a6c0611b0 0x7f9a6c153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6b59e700 1 --2- 172.21.15.32:0/2157729341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6c0b0240 0x7f9a6c0ae860 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6b59e700 1 -- 172.21.15.32:0/2157729341 >> 172.21.15.32:0/2157729341 conn(0x7f9a6c0b5f90 msgr2=0x7f9a6c0b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:05.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6b59e700 1 -- 172.21.15.32:0/2157729341 shutdown_connections 2020-12-05T17:41:05.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.410+0000 7f9a6b59e700 1 -- 172.21.15.32:0/2157729341 wait complete. 2020-12-05T17:41:05.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.414+0000 7f9a6b59e700 1 Processor -- start 2020-12-05T17:41:05.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.414+0000 7f9a6b59e700 1 -- start start 2020-12-05T17:41:05.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.414+0000 7f9a6b59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a6c0611b0 0x7f9a6c066780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:05.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.414+0000 7f9a6b59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a6c0b0240 0x7f9a6c0af260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:05.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.414+0000 7f9a6b59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6c14c340 0x7f9a6c07f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:05.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.414+0000 7f9a6b59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c07fb10 con 0x7f9a6c14c340 2020-12-05T17:41:05.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.414+0000 7f9a6b59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c065e10 con 0x7f9a6c0b0240 2020-12-05T17:41:05.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.414+0000 7f9a6b59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c066040 con 0x7f9a6c0611b0 2020-12-05T17:41:05.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.414+0000 7f9a70bbe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6c14c340 0x7f9a6c07f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:05.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.414+0000 7f9a70bbe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6c14c340 0x7f9a6c07f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40276/0 (socket says 172.21.15.32:40276) 2020-12-05T17:41:05.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.414+0000 7f9a70bbe700 1 -- 172.21.15.32:0/821089926 learned_addr learned my addr 172.21.15.32:0/821089926 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:05.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.414+0000 7f9a6ad9d700 1 --2- 172.21.15.32:0/821089926 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a6c0611b0 0x7f9a6c066780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:05.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.418+0000 7f9a6259c700 1 --2- 172.21.15.32:0/821089926 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a6c0b0240 0x7f9a6c0af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:05.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.418+0000 7f9a70bbe700 1 -- 172.21.15.32:0/821089926 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a6c0611b0 msgr2=0x7f9a6c066780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:05.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.418+0000 7f9a70bbe700 1 --2- 172.21.15.32:0/821089926 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a6c0611b0 0x7f9a6c066780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.418+0000 7f9a70bbe700 1 -- 172.21.15.32:0/821089926 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a6c0b0240 msgr2=0x7f9a6c0af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:05.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.418+0000 7f9a70bbe700 1 --2- 172.21.15.32:0/821089926 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a6c0b0240 0x7f9a6c0af260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.418+0000 7f9a70bbe700 1 -- 172.21.15.32:0/821089926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6c0662e0 con 0x7f9a6c14c340 2020-12-05T17:41:05.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.418+0000 7f9a6ad9d700 1 --2- 172.21.15.32:0/821089926 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a6c0611b0 0x7f9a6c066780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:05.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.418+0000 7f9a6259c700 1 --2- 172.21.15.32:0/821089926 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a6c0b0240 0x7f9a6c0af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:41:05.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.418+0000 7f9a70bbe700 1 --2- 172.21.15.32:0/821089926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6c14c340 0x7f9a6c07f620 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0x7f9a6c060810 tx=0x7f9a64007f20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:05.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.418+0000 7f9a63fff700 1 -- 172.21.15.32:0/821089926 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a6400a580 con 0x7f9a6c14c340 2020-12-05T17:41:05.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.418+0000 7f9a63fff700 1 -- 172.21.15.32:0/821089926 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6400a6e0 con 0x7f9a6c14c340 2020-12-05T17:41:05.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.418+0000 7f9a63fff700 1 -- 172.21.15.32:0/821089926 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a6400a850 con 0x7f9a6c14c340 2020-12-05T17:41:05.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.418+0000 7f9a6b59e700 1 -- 172.21.15.32:0/821089926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a6c065250 con 0x7f9a6c14c340 2020-12-05T17:41:05.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.418+0000 7f9a6b59e700 1 -- 172.21.15.32:0/821089926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a6c14de50 con 0x7f9a6c14c340 2020-12-05T17:41:05.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.418+0000 7f9a6b59e700 1 -- 172.21.15.32:0/821089926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a6c066470 con 0x7f9a6c14c340 2020-12-05T17:41:05.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.418+0000 7f9a63fff700 1 -- 172.21.15.32:0/821089926 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9a64010070 con 0x7f9a6c14c340 2020-12-05T17:41:05.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.418+0000 7f9a63fff700 1 --2- 172.21.15.32:0/821089926 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9a50055440 0x7f9a50057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:05.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.418+0000 7f9a6ad9d700 1 --2- 172.21.15.32:0/821089926 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9a50055440 0x7f9a50057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:05.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.418+0000 7f9a63fff700 1 -- 172.21.15.32:0/821089926 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9a6406dbe0 con 0x7f9a6c14c340 2020-12-05T17:41:05.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.422+0000 7f9a6ad9d700 1 --2- 172.21.15.32:0/821089926 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9a50055440 0x7f9a50057900 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f9a5c00c000 tx=0x7f9a5c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:05.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.426+0000 7f9a63fff700 1 -- 172.21.15.32:0/821089926 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9a64043d80 con 0x7f9a6c14c340 2020-12-05T17:41:05.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.614+0000 7f9a6b59e700 1 -- 172.21.15.32:0/821089926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f9a6c057050 con 0x7f9a6c14c340 2020-12-05T17:41:05.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.622+0000 7f9a63fff700 1 -- 172.21.15.32:0/821089926 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v39) v1 ==== 184+0+0 (secure 0 0 0) 0x7f9a640473a0 con 0x7f9a6c14c340 2020-12-05T17:41:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.622+0000 7f9a6b59e700 1 -- 172.21.15.32:0/821089926 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9a50055440 msgr2=0x7f9a50057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:05.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.622+0000 7f9a6b59e700 1 --2- 172.21.15.32:0/821089926 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9a50055440 0x7f9a50057900 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f9a5c00c000 tx=0x7f9a5c008040).stop 2020-12-05T17:41:05.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.622+0000 7f9a6b59e700 1 -- 172.21.15.32:0/821089926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6c14c340 msgr2=0x7f9a6c07f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:05.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.622+0000 7f9a6b59e700 1 --2- 172.21.15.32:0/821089926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6c14c340 0x7f9a6c07f620 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0x7f9a6c060810 tx=0x7f9a64007f20).stop 2020-12-05T17:41:05.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.622+0000 7f9a6b59e700 1 -- 172.21.15.32:0/821089926 shutdown_connections 2020-12-05T17:41:05.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.622+0000 7f9a6b59e700 1 --2- 172.21.15.32:0/821089926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6c14c340 0x7f9a6c07f620 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.626+0000 7f9a6b59e700 1 --2- 172.21.15.32:0/821089926 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9a50055440 0x7f9a50057900 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.626+0000 7f9a6b59e700 1 --2- 172.21.15.32:0/821089926 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a6c0611b0 0x7f9a6c066780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.626+0000 7f9a6b59e700 1 --2- 172.21.15.32:0/821089926 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a6c0b0240 0x7f9a6c0af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.626+0000 7f9a6b59e700 1 -- 172.21.15.32:0/821089926 >> 172.21.15.32:0/821089926 conn(0x7f9a6c0b5f90 msgr2=0x7f9a6c12d570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.626+0000 7f9a6b59e700 1 -- 172.21.15.32:0/821089926 shutdown_connections 2020-12-05T17:41:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.626+0000 7f9a6b59e700 1 -- 172.21.15.32:0/821089926 wait complete. 2020-12-05T17:41:05.633 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2020-12-05T17:41:05.644 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:41:05.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.762+0000 7faaff59e700 1 Processor -- start 2020-12-05T17:41:05.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.762+0000 7faaff59e700 1 -- start start 2020-12-05T17:41:05.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.762+0000 7faaff59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab000b0240 0x7fab00153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:05.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.762+0000 7faaff59e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fab000ae860 0x7fab0014f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.762+0000 7faaff59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab00080a90 con 0x7fab000611b0 2020-12-05T17:41:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.762+0000 7faaff59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab000aec40 con 0x7fab000b0240 2020-12-05T17:41:05.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.762+0000 7faaff59e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab00152860 con 0x7fab000ae860 2020-12-05T17:41:05.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.762+0000 7faafe59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab000b0240 0x7fab00153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:05.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.762+0000 7faafe59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab000b0240 0x7fab00153240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40282/0 (socket says 172.21.15.32:40282) 2020-12-05T17:41:05.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.762+0000 7faafe59c700 1 -- 172.21.15.32:0/2918539070 learned_addr learned my addr 172.21.15.32:0/2918539070 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:05.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.766+0000 7fab04f97700 1 --2- 172.21.15.32:0/2918539070 >> v2:172.21.15.61:3300/0 conn(0x7fab000ae860 0x7fab0014f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:05.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.766+0000 7faafe59c700 1 -- 172.21.15.32:0/2918539070 >> v2:172.21.15.61:3300/0 conn(0x7fab000ae860 msgr2=0x7fab0014f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:05.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.766+0000 7faafe59c700 1 --2- 172.21.15.32:0/2918539070 >> v2:172.21.15.61:3300/0 conn(0x7fab000ae860 0x7fab0014f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.766+0000 7faafe59c700 1 -- 172.21.15.32:0/2918539070 >> v1:172.21.15.61:6789/0 conn(0x7fab000611b0 legacy=0x7fab0014c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:05.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.766+0000 7faafe59c700 1 -- 172.21.15.32:0/2918539070 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab00152b00 con 0x7fab000b0240 2020-12-05T17:41:05.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.766+0000 7faafe59c700 1 --2- 172.21.15.32:0/2918539070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab000b0240 0x7fab00153240 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7faae8002f30 tx=0x7faae8006e80).ready entity=mon.0 client_cookie=340415c52053d94 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:05.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.766+0000 7faafdd9b700 1 -- 172.21.15.32:0/2918539070 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faae80097f0 con 0x7fab000b0240 2020-12-05T17:41:05.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.766+0000 7faafdd9b700 1 -- 172.21.15.32:0/2918539070 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae8009950 con 0x7fab000b0240 2020-12-05T17:41:05.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.766+0000 7faafdd9b700 1 -- 172.21.15.32:0/2918539070 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faae8009cb0 con 0x7fab000b0240 2020-12-05T17:41:05.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.766+0000 7faaff59e700 1 -- 172.21.15.32:0/2918539070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab000b0240 msgr2=0x7fab00153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:05.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.766+0000 7faaff59e700 1 --2- 172.21.15.32:0/2918539070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab000b0240 0x7fab00153240 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7faae8002f30 tx=0x7faae8006e80).stop 2020-12-05T17:41:05.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.766+0000 7faaff59e700 1 -- 172.21.15.32:0/2918539070 shutdown_connections 2020-12-05T17:41:05.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.766+0000 7faaff59e700 1 --2- 172.21.15.32:0/2918539070 >> v2:172.21.15.61:3300/0 conn(0x7fab000ae860 0x7fab0014f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.766+0000 7faaff59e700 1 --2- 172.21.15.32:0/2918539070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab000b0240 0x7fab00153240 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.766+0000 7faaff59e700 1 -- 172.21.15.32:0/2918539070 >> 172.21.15.32:0/2918539070 conn(0x7fab000b5f90 msgr2=0x7fab000b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:05.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.766+0000 7faaff59e700 1 -- 172.21.15.32:0/2918539070 shutdown_connections 2020-12-05T17:41:05.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.766+0000 7faaff59e700 1 -- 172.21.15.32:0/2918539070 wait complete. 2020-12-05T17:41:05.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.766+0000 7faaff59e700 1 Processor -- start 2020-12-05T17:41:05.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.766+0000 7faaff59e700 1 -- start start 2020-12-05T17:41:05.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.770+0000 7faaff59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab000611b0 0x7fab00064e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:05.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.770+0000 7faaff59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab000ae860 0x7fab000b5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:05.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.770+0000 7faaff59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab000b0240 0x7fab0013d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:05.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.770+0000 7faaff59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0013cff0 con 0x7fab000611b0 2020-12-05T17:41:05.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.770+0000 7faaff59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0013d290 con 0x7fab000b0240 2020-12-05T17:41:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.770+0000 7faaff59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab000b5730 con 0x7fab000ae860 2020-12-05T17:41:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.770+0000 7faafed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab000611b0 0x7fab00064e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.770+0000 7faafed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab000611b0 0x7fab00064e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40286/0 (socket says 172.21.15.32:40286) 2020-12-05T17:41:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.770+0000 7faafed9d700 1 -- 172.21.15.32:0/732815199 learned_addr learned my addr 172.21.15.32:0/732815199 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.770+0000 7fab04f97700 1 --2- 172.21.15.32:0/732815199 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab000b0240 0x7fab0013d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:05.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.770+0000 7faafe59c700 1 --2- 172.21.15.32:0/732815199 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab000ae860 0x7fab000b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:05.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.770+0000 7faafed9d700 1 -- 172.21.15.32:0/732815199 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab000ae860 msgr2=0x7fab000b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:05.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.770+0000 7faafed9d700 1 --2- 172.21.15.32:0/732815199 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab000ae860 0x7fab000b5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.770+0000 7faafed9d700 1 -- 172.21.15.32:0/732815199 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab000b0240 msgr2=0x7fab0013d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:05.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.770+0000 7faafed9d700 1 --2- 172.21.15.32:0/732815199 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab000b0240 0x7fab0013d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.770+0000 7faafed9d700 1 -- 172.21.15.32:0/732815199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0012ecd0 con 0x7fab000611b0 2020-12-05T17:41:05.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.770+0000 7fab04f97700 1 --2- 172.21.15.32:0/732815199 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab000b0240 0x7fab0013d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:05.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.770+0000 7faafed9d700 1 --2- 172.21.15.32:0/732815199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab000611b0 0x7fab00064e00 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0x7faaf0007c10 tx=0x7faaf0001460).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:05.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.770+0000 7faaf77fe700 1 -- 172.21.15.32:0/732815199 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faaf000cea0 con 0x7fab000611b0 2020-12-05T17:41:05.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.770+0000 7faaf77fe700 1 -- 172.21.15.32:0/732815199 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf000a9a0 con 0x7fab000611b0 2020-12-05T17:41:05.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.770+0000 7faaff59e700 1 -- 172.21.15.32:0/732815199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab0013dc20 con 0x7fab000611b0 2020-12-05T17:41:05.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.770+0000 7faaf77fe700 1 -- 172.21.15.32:0/732815199 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faaf000ad00 con 0x7fab000611b0 2020-12-05T17:41:05.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.770+0000 7faaff59e700 1 -- 172.21.15.32:0/732815199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab000b0b50 con 0x7fab000611b0 2020-12-05T17:41:05.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.774+0000 7faaff59e700 1 -- 172.21.15.32:0/732815199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab000652f0 con 0x7fab000611b0 2020-12-05T17:41:05.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.774+0000 7faaf77fe700 1 -- 172.21.15.32:0/732815199 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7faaf000f040 con 0x7fab000611b0 2020-12-05T17:41:05.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.774+0000 7faaf77fe700 1 --2- 172.21.15.32:0/732815199 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faae4055320 0x7faae40577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:05.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.774+0000 7faafe59c700 1 --2- 172.21.15.32:0/732815199 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faae4055320 0x7faae40577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:05.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.774+0000 7faaf77fe700 1 -- 172.21.15.32:0/732815199 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7faaf006fb00 con 0x7fab000611b0 2020-12-05T17:41:05.786 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.774+0000 7faafe59c700 1 --2- 172.21.15.32:0/732815199 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faae4055320 0x7faae40577e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7faae801c000 tx=0x7faae8015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:05.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.778+0000 7faaf77fe700 1 -- 172.21.15.32:0/732815199 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7faaf0045b90 con 0x7fab000611b0 2020-12-05T17:41:05.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.974+0000 7faaff59e700 1 -- 172.21.15.32:0/732815199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fab0013d450 con 0x7fab000611b0 2020-12-05T17:41:05.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.978+0000 7faaf77fe700 1 -- 172.21.15.32:0/732815199 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v40) v1 ==== 184+0+0 (secure 0 0 0) 0x7faaf0015070 con 0x7fab000611b0 2020-12-05T17:41:05.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.982+0000 7faaff59e700 1 -- 172.21.15.32:0/732815199 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faae4055320 msgr2=0x7faae40577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:05.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.982+0000 7faaff59e700 1 --2- 172.21.15.32:0/732815199 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faae4055320 0x7faae40577e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7faae801c000 tx=0x7faae8015000).stop 2020-12-05T17:41:05.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.982+0000 7faaff59e700 1 -- 172.21.15.32:0/732815199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab000611b0 msgr2=0x7fab00064e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:05.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.982+0000 7faaff59e700 1 --2- 172.21.15.32:0/732815199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab000611b0 0x7fab00064e00 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0x7faaf0007c10 tx=0x7faaf0001460).stop 2020-12-05T17:41:05.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.982+0000 7faaff59e700 1 -- 172.21.15.32:0/732815199 shutdown_connections 2020-12-05T17:41:05.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.982+0000 7faaff59e700 1 --2- 172.21.15.32:0/732815199 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab000b0240 0x7fab0013d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.982+0000 7faaff59e700 1 --2- 172.21.15.32:0/732815199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab000611b0 0x7fab00064e00 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.982+0000 7faaff59e700 1 --2- 172.21.15.32:0/732815199 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7faae4055320 0x7faae40577e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.982+0000 7faaff59e700 1 --2- 172.21.15.32:0/732815199 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab000ae860 0x7fab000b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:05.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.982+0000 7faaff59e700 1 -- 172.21.15.32:0/732815199 >> 172.21.15.32:0/732815199 conn(0x7fab000b5f90 msgr2=0x7fab001505e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:05.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.982+0000 7faaff59e700 1 -- 172.21.15.32:0/732815199 shutdown_connections 2020-12-05T17:41:05.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:05.982+0000 7faaff59e700 1 -- 172.21.15.32:0/732815199 wait complete. 2020-12-05T17:41:05.990 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.2 2020-12-05T17:41:06.000 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:41:06.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.114+0000 7fea75a60700 1 Processor -- start 2020-12-05T17:41:06.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.114+0000 7fea75a60700 1 -- start start 2020-12-05T17:41:06.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.114+0000 7fea75a60700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea7005f3b0 0x7fea7005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:06.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.114+0000 7fea75a60700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fea70062400 0x7fea700b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:06.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.114+0000 7fea75a60700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fea7005feb0 0x7fea700b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:06.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.114+0000 7fea75a60700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea70142450 con 0x7fea70062400 2020-12-05T17:41:06.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.114+0000 7fea75a60700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea70061780 con 0x7fea7005feb0 2020-12-05T17:41:06.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.114+0000 7fea75a60700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea70061a20 con 0x7fea7005f3b0 2020-12-05T17:41:06.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.114+0000 7fea6ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fea70062400 0x7fea700b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:06.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.114+0000 7fea74a5e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea7005f3b0 0x7fea7005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:06.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.114+0000 7fea6ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fea70062400 0x7fea700b36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40298/0 (socket says 172.21.15.32:40298) 2020-12-05T17:41:06.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.114+0000 7fea6ffff700 1 -- 172.21.15.32:0/3171277507 learned_addr learned my addr 172.21.15.32:0/3171277507 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:06.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.114+0000 7fea7525f700 1 --2- 172.21.15.32:0/3171277507 >> v2:172.21.15.61:3300/0 conn(0x7fea7005feb0 0x7fea700b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:06.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.114+0000 7fea6ffff700 1 -- 172.21.15.32:0/3171277507 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea7005f3b0 msgr2=0x7fea7005f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:06.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.118+0000 7fea6ffff700 1 --2- 172.21.15.32:0/3171277507 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea7005f3b0 0x7fea7005f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:06.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.118+0000 7fea6ffff700 1 -- 172.21.15.32:0/3171277507 >> v2:172.21.15.61:3300/0 conn(0x7fea7005feb0 msgr2=0x7fea700b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:06.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.118+0000 7fea6ffff700 1 --2- 172.21.15.32:0/3171277507 >> v2:172.21.15.61:3300/0 conn(0x7fea7005feb0 0x7fea700b3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:06.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.118+0000 7fea6ffff700 1 -- 172.21.15.32:0/3171277507 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea70061cc0 con 0x7fea70062400 2020-12-05T17:41:06.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.118+0000 7fea6ffff700 1 --2- 172.21.15.32:0/3171277507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea70062400 0x7fea700b36f0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7fea58002f30 tx=0x7fea58006e80).ready entity=mon.0 client_cookie=225d6e313e6593b6 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:06.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.118+0000 7fea6f7fe700 1 -- 172.21.15.32:0/3171277507 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea58009640 con 0x7fea70062400 2020-12-05T17:41:06.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.118+0000 7fea6f7fe700 1 -- 172.21.15.32:0/3171277507 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea580097a0 con 0x7fea70062400 2020-12-05T17:41:06.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.118+0000 7fea6f7fe700 1 -- 172.21.15.32:0/3171277507 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea58009b00 con 0x7fea70062400 2020-12-05T17:41:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.118+0000 7fea75a60700 1 -- 172.21.15.32:0/3171277507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea70062400 msgr2=0x7fea700b36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.118+0000 7fea75a60700 1 --2- 172.21.15.32:0/3171277507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea70062400 0x7fea700b36f0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7fea58002f30 tx=0x7fea58006e80).stop 2020-12-05T17:41:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.118+0000 7fea75a60700 1 -- 172.21.15.32:0/3171277507 shutdown_connections 2020-12-05T17:41:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.118+0000 7fea75a60700 1 --2- 172.21.15.32:0/3171277507 >> v2:172.21.15.61:3300/0 conn(0x7fea7005feb0 0x7fea700b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.118+0000 7fea75a60700 1 --2- 172.21.15.32:0/3171277507 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea7005f3b0 0x7fea7005f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.118+0000 7fea75a60700 1 --2- 172.21.15.32:0/3171277507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea70062400 0x7fea700b36f0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.118+0000 7fea75a60700 1 -- 172.21.15.32:0/3171277507 >> 172.21.15.32:0/3171277507 conn(0x7fea700b8ee0 msgr2=0x7fea700b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.118+0000 7fea75a60700 1 -- 172.21.15.32:0/3171277507 shutdown_connections 2020-12-05T17:41:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.118+0000 7fea75a60700 1 -- 172.21.15.32:0/3171277507 wait complete. 2020-12-05T17:41:06.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.122+0000 7fea75a60700 1 Processor -- start 2020-12-05T17:41:06.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.122+0000 7fea75a60700 1 -- start start 2020-12-05T17:41:06.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.122+0000 7fea75a60700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea7005f3b0 0x7fea70064910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:06.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.122+0000 7fea75a60700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea7005feb0 0x7fea70151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:06.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.122+0000 7fea75a60700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea70062400 0x7fea70066750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:06.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.122+0000 7fea75a60700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea701517f0 con 0x7fea70062400 2020-12-05T17:41:06.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.122+0000 7fea75a60700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea70144700 con 0x7fea7005feb0 2020-12-05T17:41:06.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.122+0000 7fea75a60700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea70144980 con 0x7fea7005f3b0 2020-12-05T17:41:06.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.122+0000 7fea7525f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea70062400 0x7fea70066750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:06.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.122+0000 7fea7525f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea70062400 0x7fea70066750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40304/0 (socket says 172.21.15.32:40304) 2020-12-05T17:41:06.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.122+0000 7fea7525f700 1 -- 172.21.15.32:0/2962094799 learned_addr learned my addr 172.21.15.32:0/2962094799 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:06.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.122+0000 7fea74a5e700 1 --2- 172.21.15.32:0/2962094799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea7005f3b0 0x7fea70064910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:06.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.122+0000 7fea6ffff700 1 --2- 172.21.15.32:0/2962094799 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea7005feb0 0x7fea70151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:06.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.126+0000 7fea7525f700 1 -- 172.21.15.32:0/2962094799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea7005f3b0 msgr2=0x7fea70064910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:06.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.126+0000 7fea7525f700 1 --2- 172.21.15.32:0/2962094799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea7005f3b0 0x7fea70064910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:06.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.126+0000 7fea7525f700 1 -- 172.21.15.32:0/2962094799 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea7005feb0 msgr2=0x7fea70151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:06.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.126+0000 7fea7525f700 1 --2- 172.21.15.32:0/2962094799 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea7005feb0 0x7fea70151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:06.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.126+0000 7fea7525f700 1 -- 172.21.15.32:0/2962094799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea70145020 con 0x7fea70062400 2020-12-05T17:41:06.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.126+0000 7fea7525f700 1 --2- 172.21.15.32:0/2962094799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea70062400 0x7fea70066750 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7fea68006dd0 tx=0x7fea680091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:06.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.126+0000 7fea6dffb700 1 -- 172.21.15.32:0/2962094799 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea6800ca00 con 0x7fea70062400 2020-12-05T17:41:06.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.126+0000 7fea6dffb700 1 -- 172.21.15.32:0/2962094799 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6800cb60 con 0x7fea70062400 2020-12-05T17:41:06.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.126+0000 7fea75a60700 1 -- 172.21.15.32:0/2962094799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea70066c40 con 0x7fea70062400 2020-12-05T17:41:06.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.126+0000 7fea6dffb700 1 -- 172.21.15.32:0/2962094799 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea680123f0 con 0x7fea70062400 2020-12-05T17:41:06.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.126+0000 7fea75a60700 1 -- 172.21.15.32:0/2962094799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea701466c0 con 0x7fea70062400 2020-12-05T17:41:06.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.126+0000 7fea75a60700 1 -- 172.21.15.32:0/2962094799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea70057050 con 0x7fea70062400 2020-12-05T17:41:06.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.126+0000 7fea6dffb700 1 -- 172.21.15.32:0/2962094799 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fea68022070 con 0x7fea70062400 2020-12-05T17:41:06.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.126+0000 7fea6dffb700 1 --2- 172.21.15.32:0/2962094799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fea54055330 0x7fea540577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:06.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.126+0000 7fea6dffb700 1 -- 172.21.15.32:0/2962094799 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fea6806e7a0 con 0x7fea70062400 2020-12-05T17:41:06.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.126+0000 7fea74a5e700 1 --2- 172.21.15.32:0/2962094799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fea54055330 0x7fea540577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:06.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.130+0000 7fea74a5e700 1 --2- 172.21.15.32:0/2962094799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fea54055330 0x7fea540577f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7fea6000b000 tx=0x7fea60006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:06.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.134+0000 7fea6dffb700 1 -- 172.21.15.32:0/2962094799 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fea68044940 con 0x7fea70062400 2020-12-05T17:41:06.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.326+0000 7fea75a60700 1 -- 172.21.15.32:0/2962094799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea700b8c30 con 0x7fea70062400 2020-12-05T17:41:06.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.326+0000 7fea6dffb700 1 -- 172.21.15.32:0/2962094799 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+2142 (secure 0 0 0) 0x7fea68016070 con 0x7fea70062400 2020-12-05T17:41:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:06.332 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":62,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6830,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/3985915039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3985915039},{"type":"v1","addr":"172.21.15.61:6833","nonce":3985915039}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":60}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:41:03.280787+0000","modified":"2020-12-05T17:41:04.283186+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7938":{"gid":7938,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1585653076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1585653076},{"type":"v1","addr":"172.21.15.32:6835","nonce":1585653076}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2020-12-05T17:41:06.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.326+0000 7fea75a60700 1 -- 172.21.15.32:0/2962094799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fea54055330 msgr2=0x7fea540577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:06.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.326+0000 7fea75a60700 1 --2- 172.21.15.32:0/2962094799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fea54055330 0x7fea540577f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7fea6000b000 tx=0x7fea60006dc0).stop 2020-12-05T17:41:06.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.326+0000 7fea75a60700 1 -- 172.21.15.32:0/2962094799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea70062400 msgr2=0x7fea70066750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:06.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.326+0000 7fea75a60700 1 --2- 172.21.15.32:0/2962094799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea70062400 0x7fea70066750 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7fea68006dd0 tx=0x7fea680091a0).stop 2020-12-05T17:41:06.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.330+0000 7fea75a60700 1 -- 172.21.15.32:0/2962094799 shutdown_connections 2020-12-05T17:41:06.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.330+0000 7fea75a60700 1 --2- 172.21.15.32:0/2962094799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea70062400 0x7fea70066750 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:06.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.330+0000 7fea75a60700 1 --2- 172.21.15.32:0/2962094799 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fea54055330 0x7fea540577f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:06.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.330+0000 7fea75a60700 1 --2- 172.21.15.32:0/2962094799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea7005f3b0 0x7fea70064910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:06.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.330+0000 7fea75a60700 1 --2- 172.21.15.32:0/2962094799 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea7005feb0 0x7fea70151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:06.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.330+0000 7fea75a60700 1 -- 172.21.15.32:0/2962094799 >> 172.21.15.32:0/2962094799 conn(0x7fea700b8ee0 msgr2=0x7fea7013f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:06.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.330+0000 7fea75a60700 1 -- 172.21.15.32:0/2962094799 shutdown_connections 2020-12-05T17:41:06.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.330+0000 7fea75a60700 1 -- 172.21.15.32:0/2962094799 wait complete. 2020-12-05T17:41:06.337 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 62 2020-12-05T17:41:06.547 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:41:03.280787+0000', 'modified': '2020-12-05T17:41:04.283186+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7938}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7938': {'gid': 7938, 'name': 'a', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/1585653076', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 1585653076}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 1585653076}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:41:06.548 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:41:06.549 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:41:06.679 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":7938,"want_state":"up:active","state":"up:active","rank_uptime":3.3851052190000002,"mdsmap_epoch":62,"osdmap_epoch":129,"osdmap_epoch_barrier":128,"uptime":8.3493065059999996} 2020-12-05T17:41:06.679 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 7938, "mdsmap_epoch": 62, "osdmap_epoch": 129, "osdmap_epoch_barrier": 128, "rank_uptime": 3.385105219, "state": "up:active", "uptime": 8.349306506, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:41:06.680 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:41:06.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.786+0000 7f9a0b07c700 1 Processor -- start 2020-12-05T17:41:06.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.786+0000 7f9a0b07c700 1 -- start start 2020-12-05T17:41:06.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a0b07c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a04062400 0x7f9a0405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:06.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a0b07c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9a0405feb0 0x7f9a040b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:06.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a0b07c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a04142450 con 0x7f9a0405f3b0 2020-12-05T17:41:06.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a0b07c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a040b42f0 con 0x7f9a04062400 2020-12-05T17:41:06.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a0b07c700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a04061780 con 0x7f9a0405feb0 2020-12-05T17:41:06.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a09879700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a04062400 0x7f9a0405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:06.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a09879700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a04062400 0x7f9a0405f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40310/0 (socket says 172.21.15.32:40310) 2020-12-05T17:41:06.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a09879700 1 -- 172.21.15.32:0/736845592 learned_addr learned my addr 172.21.15.32:0/736845592 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:06.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a0a87b700 1 --2- 172.21.15.32:0/736845592 >> v2:172.21.15.61:3300/0 conn(0x7f9a0405feb0 0x7f9a040b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:06.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a0a87b700 1 -- 172.21.15.32:0/736845592 >> v2:172.21.15.32:3300/0 conn(0x7f9a04062400 msgr2=0x7f9a0405f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:06.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a0a87b700 1 --2- 172.21.15.32:0/736845592 >> v2:172.21.15.32:3300/0 conn(0x7f9a04062400 0x7f9a0405f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:06.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a0a87b700 1 -- 172.21.15.32:0/736845592 >> v1:172.21.15.61:6789/0 conn(0x7f9a0405f3b0 legacy=0x7f9a040b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:06.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a0a87b700 1 -- 172.21.15.32:0/736845592 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a04061a20 con 0x7f9a0405feb0 2020-12-05T17:41:06.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a09879700 1 --2- 172.21.15.32:0/736845592 >> v2:172.21.15.32:3300/0 conn(0x7f9a04062400 0x7f9a0405f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:06.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a0a87b700 1 --2- 172.21.15.32:0/736845592 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a0405feb0 0x7f9a040b3e00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f99fc0032b0 tx=0x7f99fc0091a0).ready entity=mon.1 client_cookie=1eaa9078253bcd71 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:06.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a09078700 1 -- 172.21.15.32:0/736845592 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99fc00b9a0 con 0x7f9a0405feb0 2020-12-05T17:41:06.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a09078700 1 -- 172.21.15.32:0/736845592 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99fc00bb00 con 0x7f9a0405feb0 2020-12-05T17:41:06.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a09078700 1 -- 172.21.15.32:0/736845592 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99fc00bc70 con 0x7f9a0405feb0 2020-12-05T17:41:06.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a0b07c700 1 -- 172.21.15.32:0/736845592 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a0405feb0 msgr2=0x7f9a040b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:06.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a0b07c700 1 --2- 172.21.15.32:0/736845592 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a0405feb0 0x7f9a040b3e00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f99fc0032b0 tx=0x7f99fc0091a0).stop 2020-12-05T17:41:06.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a0b07c700 1 -- 172.21.15.32:0/736845592 shutdown_connections 2020-12-05T17:41:06.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a0b07c700 1 --2- 172.21.15.32:0/736845592 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a0405feb0 0x7f9a040b3e00 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:06.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a0b07c700 1 --2- 172.21.15.32:0/736845592 >> v2:172.21.15.32:3300/0 conn(0x7f9a04062400 0x7f9a0405f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:06.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a0b07c700 1 -- 172.21.15.32:0/736845592 >> 172.21.15.32:0/736845592 conn(0x7f9a040b8ee0 msgr2=0x7f9a040b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:06.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a0b07c700 1 -- 172.21.15.32:0/736845592 shutdown_connections 2020-12-05T17:41:06.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.790+0000 7f9a0b07c700 1 -- 172.21.15.32:0/736845592 wait complete. 2020-12-05T17:41:06.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f9a0b07c700 1 Processor -- start 2020-12-05T17:41:06.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f9a0b07c700 1 -- start start 2020-12-05T17:41:06.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f9a0b07c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a0405f3b0 0x7f9a040b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:06.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f9a0b07c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a0405feb0 0x7f9a0413d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:06.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f9a0b07c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a04062400 0x7f9a0413d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:06.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f9a0b07c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a040b2500 con 0x7f9a04062400 2020-12-05T17:41:06.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f9a0b07c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a04144000 con 0x7f9a0405feb0 2020-12-05T17:41:06.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f9a0b07c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a041443d0 con 0x7f9a0405f3b0 2020-12-05T17:41:06.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f9a0a87b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a04062400 0x7f9a0413d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:06.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f9a0a87b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a04062400 0x7f9a0413d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40318/0 (socket says 172.21.15.32:40318) 2020-12-05T17:41:06.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f9a0a87b700 1 -- 172.21.15.32:0/896613221 learned_addr learned my addr 172.21.15.32:0/896613221 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:06.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f9a09879700 1 --2- 172.21.15.32:0/896613221 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a0405feb0 0x7f9a0413d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:06.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f9a0a07a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a0405f3b0 0x7f9a040b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:06.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f9a0a87b700 1 -- 172.21.15.32:0/896613221 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a0405f3b0 msgr2=0x7f9a040b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:06.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f9a0a87b700 1 --2- 172.21.15.32:0/896613221 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a0405f3b0 0x7f9a040b80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:06.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f9a0a87b700 1 -- 172.21.15.32:0/896613221 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a0405feb0 msgr2=0x7f9a0413d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:06.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f9a0a87b700 1 --2- 172.21.15.32:0/896613221 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a0405feb0 0x7f9a0413d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:06.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f9a0a87b700 1 -- 172.21.15.32:0/896613221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a04144670 con 0x7f9a04062400 2020-12-05T17:41:06.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f9a0a87b700 1 --2- 172.21.15.32:0/896613221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a04062400 0x7f9a0413d710 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 rx=0x7f99fc0032b0 tx=0x7f99fc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:06.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f99f37fe700 1 -- 172.21.15.32:0/896613221 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99fc01f3f0 con 0x7f9a04062400 2020-12-05T17:41:06.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f99f37fe700 1 -- 172.21.15.32:0/896613221 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99fc01f550 con 0x7f9a04062400 2020-12-05T17:41:06.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f9a0b07c700 1 -- 172.21.15.32:0/896613221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0413dc00 con 0x7f9a04062400 2020-12-05T17:41:06.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f99f37fe700 1 -- 172.21.15.32:0/896613221 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99fc01f8b0 con 0x7f9a04062400 2020-12-05T17:41:06.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.794+0000 7f9a0b07c700 1 -- 172.21.15.32:0/896613221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a04144cb0 con 0x7f9a04062400 2020-12-05T17:41:06.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.798+0000 7f9a0b07c700 1 -- 172.21.15.32:0/896613221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a04144800 con 0x7f9a04062400 2020-12-05T17:41:06.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.798+0000 7f99f37fe700 1 -- 172.21.15.32:0/896613221 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f99fc026030 con 0x7f9a04062400 2020-12-05T17:41:06.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.798+0000 7f99f37fe700 1 --2- 172.21.15.32:0/896613221 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f99e40554e0 0x7f99e40579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:06.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.798+0000 7f99f37fe700 1 -- 172.21.15.32:0/896613221 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f99fc06e990 con 0x7f9a04062400 2020-12-05T17:41:06.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.798+0000 7f9a0a07a700 1 --2- 172.21.15.32:0/896613221 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f99e40554e0 0x7f99e40579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:06.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.798+0000 7f9a0a07a700 1 --2- 172.21.15.32:0/896613221 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f99e40554e0 0x7f99e40579a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f9a0000d000 tx=0x7f9a00000b80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:06.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.806+0000 7f99f37fe700 1 -- 172.21.15.32:0/896613221 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f99fc044ab0 con 0x7f9a04062400 2020-12-05T17:41:06.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.946+0000 7f9a0b07c700 1 -- 172.21.15.32:0/896613221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a04057050 con 0x7f9a04062400 2020-12-05T17:41:06.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.950+0000 7f99f37fe700 1 -- 172.21.15.32:0/896613221 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f99fc015090 con 0x7f9a04062400 2020-12-05T17:41:06.956 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:06.956 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":129,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:41:04.284002+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:41:01.240356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2020-12-05T17:41:02.269469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:41:06.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.954+0000 7f9a0b07c700 1 -- 172.21.15.32:0/896613221 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f99e40554e0 msgr2=0x7f99e40579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:06.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.954+0000 7f9a0b07c700 1 --2- 172.21.15.32:0/896613221 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f99e40554e0 0x7f99e40579a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f9a0000d000 tx=0x7f9a00000b80).stop 2020-12-05T17:41:06.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.954+0000 7f9a0b07c700 1 -- 172.21.15.32:0/896613221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a04062400 msgr2=0x7f9a0413d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:06.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.954+0000 7f9a0b07c700 1 --2- 172.21.15.32:0/896613221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a04062400 0x7f9a0413d710 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 rx=0x7f99fc0032b0 tx=0x7f99fc016000).stop 2020-12-05T17:41:06.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.954+0000 7f9a0b07c700 1 -- 172.21.15.32:0/896613221 shutdown_connections 2020-12-05T17:41:06.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.954+0000 7f9a0b07c700 1 --2- 172.21.15.32:0/896613221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a04062400 0x7f9a0413d710 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:06.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.954+0000 7f9a0b07c700 1 --2- 172.21.15.32:0/896613221 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f99e40554e0 0x7f99e40579a0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.954+0000 7f9a0b07c700 1 --2- 172.21.15.32:0/896613221 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9a0405f3b0 0x7f9a040b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.954+0000 7f9a0b07c700 1 --2- 172.21.15.32:0/896613221 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a0405feb0 0x7f9a0413d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.954+0000 7f9a0b07c700 1 -- 172.21.15.32:0/896613221 >> 172.21.15.32:0/896613221 conn(0x7f9a040b8ee0 msgr2=0x7f9a04140770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.954+0000 7f9a0b07c700 1 -- 172.21.15.32:0/896613221 shutdown_connections 2020-12-05T17:41:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:06.954+0000 7f9a0b07c700 1 -- 172.21.15.32:0/896613221 wait complete. 2020-12-05T17:41:06.974 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:41:01.240356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '128', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:41:02.269469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:41:06.974 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:41:07.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.086+0000 7fb4b8cd0700 1 Processor -- start 2020-12-05T17:41:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.086+0000 7fb4b8cd0700 1 -- start start 2020-12-05T17:41:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.086+0000 7fb4b8cd0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4b405feb0 0x7fb4b40b2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.086+0000 7fb4b8cd0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b4061780 con 0x7fb4b405f550 2020-12-05T17:41:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.086+0000 7fb4b8cd0700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b40b38c0 con 0x7fb4b4145880 2020-12-05T17:41:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.086+0000 7fb4b8cd0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b40b3b60 con 0x7fb4b405feb0 2020-12-05T17:41:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.086+0000 7fb4b2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4b405feb0 0x7fb4b40b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.086+0000 7fb4b2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4b405feb0 0x7fb4b40b2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40324/0 (socket says 172.21.15.32:40324) 2020-12-05T17:41:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.086+0000 7fb4b37fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb4b405f550 0x7fb4b40b2030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42970/0 (socket says 172.21.15.32:42970) 2020-12-05T17:41:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.086+0000 7fb4b2ffd700 1 -- 172.21.15.32:0/1047794171 learned_addr learned my addr 172.21.15.32:0/1047794171 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.090+0000 7fb4b2ffd700 1 -- 172.21.15.32:0/1047794171 >> v1:172.21.15.61:6789/0 conn(0x7fb4b4145880 legacy=0x7fb4b4145d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.090+0000 7fb4b2ffd700 1 -- 172.21.15.32:0/1047794171 >> v1:172.21.15.32:6789/0 conn(0x7fb4b405f550 legacy=0x7fb4b40b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.090+0000 7fb4b2ffd700 1 -- 172.21.15.32:0/1047794171 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b40b4200 con 0x7fb4b405feb0 2020-12-05T17:41:07.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.090+0000 7fb4b27fc700 1 -- 172.21.15.32:0/1047794171 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3295778528 0 0) 0x7fb4b4061780 con 0x7fb4b405f550 2020-12-05T17:41:07.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.090+0000 7fb4b2ffd700 1 --2- 172.21.15.32:0/1047794171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b405feb0 0x7fb4b40b2740 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 rx=0x7fb4b40b1e60 tx=0x7fb4a8006e80).ready entity=mon.0 client_cookie=b1350fa0605868db server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:07.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.090+0000 7fb4b27fc700 1 -- 172.21.15.32:0/1047794171 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4a8009800 con 0x7fb4b405feb0 2020-12-05T17:41:07.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.090+0000 7fb4b27fc700 1 -- 172.21.15.32:0/1047794171 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a8009960 con 0x7fb4b405feb0 2020-12-05T17:41:07.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.090+0000 7fb4b27fc700 1 -- 172.21.15.32:0/1047794171 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4a8009cc0 con 0x7fb4b405feb0 2020-12-05T17:41:07.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.090+0000 7fb4b8cd0700 1 -- 172.21.15.32:0/1047794171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b405feb0 msgr2=0x7fb4b40b2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:07.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.090+0000 7fb4b8cd0700 1 --2- 172.21.15.32:0/1047794171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b405feb0 0x7fb4b40b2740 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 rx=0x7fb4b40b1e60 tx=0x7fb4a8006e80).stop 2020-12-05T17:41:07.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.090+0000 7fb4b8cd0700 1 -- 172.21.15.32:0/1047794171 shutdown_connections 2020-12-05T17:41:07.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.090+0000 7fb4b8cd0700 1 --2- 172.21.15.32:0/1047794171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b405feb0 0x7fb4b40b2740 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.090+0000 7fb4b8cd0700 1 -- 172.21.15.32:0/1047794171 >> 172.21.15.32:0/1047794171 conn(0x7fb4b40b8f20 msgr2=0x7fb4b40b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:07.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.090+0000 7fb4b8cd0700 1 -- 172.21.15.32:0/1047794171 shutdown_connections 2020-12-05T17:41:07.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.090+0000 7fb4b8cd0700 1 -- 172.21.15.32:0/1047794171 wait complete. 2020-12-05T17:41:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.094+0000 7fb4b8cd0700 1 Processor -- start 2020-12-05T17:41:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.094+0000 7fb4b8cd0700 1 -- start start 2020-12-05T17:41:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.094+0000 7fb4b8cd0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb4b405f550 0x7fb4b40b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.094+0000 7fb4b8cd0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4b405feb0 0x7fb4b413df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.094+0000 7fb4b8cd0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b4145880 0x7fb4b413e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.094+0000 7fb4b8cd0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b4144ff0 con 0x7fb4b4145880 2020-12-05T17:41:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.094+0000 7fb4b8cd0700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b4145220 con 0x7fb4b405feb0 2020-12-05T17:41:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.094+0000 7fb4b8cd0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b41449c0 con 0x7fb4b405f550 2020-12-05T17:41:07.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.094+0000 7fb4b3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b4145880 0x7fb4b413e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:07.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.094+0000 7fb4b3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b4145880 0x7fb4b413e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40332/0 (socket says 172.21.15.32:40332) 2020-12-05T17:41:07.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.094+0000 7fb4b3fff700 1 -- 172.21.15.32:0/1848713261 learned_addr learned my addr 172.21.15.32:0/1848713261 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:07.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.094+0000 7fb4b37fe700 1 --2- 172.21.15.32:0/1848713261 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb4b405f550 0x7fb4b40b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:07.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.094+0000 7fb4b2ffd700 1 --2- 172.21.15.32:0/1848713261 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4b405feb0 0x7fb4b413df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.094+0000 7fb4b3fff700 1 -- 172.21.15.32:0/1848713261 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb4b405f550 msgr2=0x7fb4b40b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.094+0000 7fb4b3fff700 1 --2- 172.21.15.32:0/1848713261 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb4b405f550 0x7fb4b40b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.094+0000 7fb4b3fff700 1 -- 172.21.15.32:0/1848713261 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4b405feb0 msgr2=0x7fb4b413df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.094+0000 7fb4b3fff700 1 --2- 172.21.15.32:0/1848713261 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4b405feb0 0x7fb4b413df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.094+0000 7fb4b3fff700 1 -- 172.21.15.32:0/1848713261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b4144700 con 0x7fb4b4145880 2020-12-05T17:41:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.094+0000 7fb4b3fff700 1 --2- 172.21.15.32:0/1848713261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b4145880 0x7fb4b413e410 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7fb4b40614b0 tx=0x7fb4ac007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.094+0000 7fb4b0ff9700 1 -- 172.21.15.32:0/1848713261 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4ac013410 con 0x7fb4b4145880 2020-12-05T17:41:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.094+0000 7fb4b0ff9700 1 -- 172.21.15.32:0/1848713261 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ac013570 con 0x7fb4b4145880 2020-12-05T17:41:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.094+0000 7fb4b8cd0700 1 -- 172.21.15.32:0/1848713261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b413e900 con 0x7fb4b4145880 2020-12-05T17:41:07.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.094+0000 7fb4b0ff9700 1 -- 172.21.15.32:0/1848713261 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4ac0138d0 con 0x7fb4b4145880 2020-12-05T17:41:07.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.098+0000 7fb4b8cd0700 1 -- 172.21.15.32:0/1848713261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b41439e0 con 0x7fb4b4145880 2020-12-05T17:41:07.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.098+0000 7fb4b0ff9700 1 -- 172.21.15.32:0/1848713261 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb4ac010030 con 0x7fb4b4145880 2020-12-05T17:41:07.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.098+0000 7fb4b8cd0700 1 -- 172.21.15.32:0/1848713261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b4144b80 con 0x7fb4b4145880 2020-12-05T17:41:07.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.098+0000 7fb4b0ff9700 1 --2- 172.21.15.32:0/1848713261 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb494055440 0x7fb494057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:07.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.098+0000 7fb4b37fe700 1 --2- 172.21.15.32:0/1848713261 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb494055440 0x7fb494057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:07.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.098+0000 7fb4b0ff9700 1 -- 172.21.15.32:0/1848713261 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb4ac06de60 con 0x7fb4b4145880 2020-12-05T17:41:07.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.098+0000 7fb4b37fe700 1 --2- 172.21.15.32:0/1848713261 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb494055440 0x7fb494057900 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fb4a400d000 tx=0x7fb4a4007dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.106+0000 7fb4b0ff9700 1 -- 172.21.15.32:0/1848713261 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb4ac044000 con 0x7fb4b4145880 2020-12-05T17:41:07.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.246+0000 7fb4b8cd0700 1 -- 172.21.15.32:0/1848713261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb4b4144d80 con 0x7fb4b4145880 2020-12-05T17:41:07.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.246+0000 7fb4b0ff9700 1 -- 172.21.15.32:0/1848713261 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v129) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb4ac047620 con 0x7fb4b4145880 2020-12-05T17:41:07.253 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:41:07.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.250+0000 7fb4b8cd0700 1 -- 172.21.15.32:0/1848713261 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb494055440 msgr2=0x7fb494057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:07.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.250+0000 7fb4b8cd0700 1 --2- 172.21.15.32:0/1848713261 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb494055440 0x7fb494057900 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fb4a400d000 tx=0x7fb4a4007dc0).stop 2020-12-05T17:41:07.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.250+0000 7fb4b8cd0700 1 -- 172.21.15.32:0/1848713261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b4145880 msgr2=0x7fb4b413e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:07.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.250+0000 7fb4b8cd0700 1 --2- 172.21.15.32:0/1848713261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b4145880 0x7fb4b413e410 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7fb4b40614b0 tx=0x7fb4ac007bf0).stop 2020-12-05T17:41:07.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.250+0000 7fb4b8cd0700 1 -- 172.21.15.32:0/1848713261 shutdown_connections 2020-12-05T17:41:07.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.250+0000 7fb4b8cd0700 1 --2- 172.21.15.32:0/1848713261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b4145880 0x7fb4b413e410 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.250+0000 7fb4b8cd0700 1 --2- 172.21.15.32:0/1848713261 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb494055440 0x7fb494057900 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.250+0000 7fb4b8cd0700 1 --2- 172.21.15.32:0/1848713261 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb4b405f550 0x7fb4b40b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.250+0000 7fb4b8cd0700 1 --2- 172.21.15.32:0/1848713261 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4b405feb0 0x7fb4b413df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.250+0000 7fb4b8cd0700 1 -- 172.21.15.32:0/1848713261 >> 172.21.15.32:0/1848713261 conn(0x7fb4b40b8f20 msgr2=0x7fb4b40b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:07.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.250+0000 7fb4b8cd0700 1 -- 172.21.15.32:0/1848713261 shutdown_connections 2020-12-05T17:41:07.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.250+0000 7fb4b8cd0700 1 -- 172.21.15.32:0/1848713261 wait complete. 2020-12-05T17:41:07.267 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:41:07.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc96a6cc700 1 Processor -- start 2020-12-05T17:41:07.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc96a6cc700 1 -- start start 2020-12-05T17:41:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc96a6cc700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc9640af4b0 0x7fc964146510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:07.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc96a6cc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc964061780 0x7fc9640afe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:07.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc96a6cc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9640b23e0 con 0x7fc9640b2000 2020-12-05T17:41:07.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc96a6cc700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc964146a00 con 0x7fc9640af4b0 2020-12-05T17:41:07.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc96a6cc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9640ae7b0 con 0x7fc964061780 2020-12-05T17:41:07.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc9696ca700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc9640b2000 0x7fc964145e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42984/0 (socket says 172.21.15.32:42984) 2020-12-05T17:41:07.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc9696ca700 1 -- 172.21.15.32:0/3948119212 learned_addr learned my addr 172.21.15.32:0/3948119212 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:07.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc969ecb700 1 --2- 172.21.15.32:0/3948119212 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc964061780 0x7fc9640afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc968ec9700 1 --2- 172.21.15.32:0/3948119212 >> v2:172.21.15.61:3300/0 conn(0x7fc9640af4b0 0x7fc964146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc95bfff700 1 -- 172.21.15.32:0/3948119212 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307387246 0 0) 0x7fc9640b23e0 con 0x7fc9640b2000 2020-12-05T17:41:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc95bfff700 1 -- 172.21.15.32:0/3948119212 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc948003230 con 0x7fc9640b2000 2020-12-05T17:41:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc95bfff700 1 -- 172.21.15.32:0/3948119212 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2850954370 0 0) 0x7fc948003230 con 0x7fc9640b2000 2020-12-05T17:41:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc95bfff700 1 -- 172.21.15.32:0/3948119212 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc964061780 msgr2=0x7fc9640afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc95bfff700 1 --2- 172.21.15.32:0/3948119212 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc964061780 0x7fc9640afe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc95bfff700 1 -- 172.21.15.32:0/3948119212 >> v2:172.21.15.61:3300/0 conn(0x7fc9640af4b0 msgr2=0x7fc964146510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc95bfff700 1 --2- 172.21.15.32:0/3948119212 >> v2:172.21.15.61:3300/0 conn(0x7fc9640af4b0 0x7fc964146510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc95bfff700 1 -- 172.21.15.32:0/3948119212 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9640aea50 con 0x7fc9640b2000 2020-12-05T17:41:07.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc95bfff700 1 -- 172.21.15.32:0/3948119212 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fc9600016d0 con 0x7fc9640b2000 2020-12-05T17:41:07.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc95bfff700 1 -- 172.21.15.32:0/3948119212 >> v1:172.21.15.32:6789/0 conn(0x7fc9640b2000 legacy=0x7fc964145e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:07.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc95bfff700 1 --2- 172.21.15.32:0/3948119212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc948003720 0x7fc948003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:07.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc95bfff700 1 --2- 172.21.15.32:0/3948119212 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9480044d0 0x7fc948004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:07.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc95bfff700 1 --2- 172.21.15.32:0/3948119212 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc964061780 0x7fc948004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:07.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc95bfff700 1 -- 172.21.15.32:0/3948119212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc964146a00 con 0x7fc948003720 2020-12-05T17:41:07.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc95bfff700 1 -- 172.21.15.32:0/3948119212 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9640ae7b0 con 0x7fc9480044d0 2020-12-05T17:41:07.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc95bfff700 1 -- 172.21.15.32:0/3948119212 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc948004260 con 0x7fc964061780 2020-12-05T17:41:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc9696ca700 1 --2- 172.21.15.32:0/3948119212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc948003720 0x7fc948003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc969ecb700 1 --2- 172.21.15.32:0/3948119212 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc964061780 0x7fc948004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc968ec9700 1 --2- 172.21.15.32:0/3948119212 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9480044d0 0x7fc948004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc9696ca700 1 -- 172.21.15.32:0/3948119212 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc964061780 msgr2=0x7fc948004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc9696ca700 1 --2- 172.21.15.32:0/3948119212 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc964061780 0x7fc948004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc9696ca700 1 -- 172.21.15.32:0/3948119212 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9480044d0 msgr2=0x7fc948004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc9696ca700 1 --2- 172.21.15.32:0/3948119212 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc9480044d0 0x7fc948004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc9696ca700 1 -- 172.21.15.32:0/3948119212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc948003230 con 0x7fc948003720 2020-12-05T17:41:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc9696ca700 1 --2- 172.21.15.32:0/3948119212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc948003720 0x7fc948003b40 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 rx=0x7fc9600013e0 tx=0x7fc9640aebe0).ready entity=mon.0 client_cookie=e6913c0977b7f5e3 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc95bfff700 1 -- 172.21.15.32:0/3948119212 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc96000ce00 con 0x7fc948003720 2020-12-05T17:41:07.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc95bfff700 1 -- 172.21.15.32:0/3948119212 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9600072e0 con 0x7fc948003720 2020-12-05T17:41:07.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc95bfff700 1 -- 172.21.15.32:0/3948119212 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9600057d0 con 0x7fc948003720 2020-12-05T17:41:07.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc96a6cc700 1 -- 172.21.15.32:0/3948119212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc948003720 msgr2=0x7fc948003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc96a6cc700 1 --2- 172.21.15.32:0/3948119212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc948003720 0x7fc948003b40 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 rx=0x7fc9600013e0 tx=0x7fc9640aebe0).stop 2020-12-05T17:41:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc969ecb700 1 -- 172.21.15.32:0/3948119212 reap_dead start 2020-12-05T17:41:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc96a6cc700 1 -- 172.21.15.32:0/3948119212 shutdown_connections 2020-12-05T17:41:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc96a6cc700 1 -- 172.21.15.32:0/3948119212 >> 172.21.15.32:0/3948119212 conn(0x7fc9640b5620 msgr2=0x7fc9640b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:07.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc96a6cc700 1 -- 172.21.15.32:0/3948119212 shutdown_connections 2020-12-05T17:41:07.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.382+0000 7fc96a6cc700 1 -- 172.21.15.32:0/3948119212 wait complete. 2020-12-05T17:41:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.386+0000 7fc96a6cc700 1 Processor -- start 2020-12-05T17:41:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.386+0000 7fc96a6cc700 1 -- start start 2020-12-05T17:41:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc96a6cc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc96414ab50 0x7fc964064f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc96a6cc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc964065480 0x7fc964065900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc96a6cc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc964065fa0 0x7fc964066420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc96a6cc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc964149930 con 0x7fc96414ab50 2020-12-05T17:41:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc96a6cc700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc964143fb0 con 0x7fc964065fa0 2020-12-05T17:41:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc96a6cc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96414af30 con 0x7fc964065480 2020-12-05T17:41:07.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc9696ca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc96414ab50 0x7fc964064f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:07.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc9696ca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc96414ab50 0x7fc964064f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40348/0 (socket says 172.21.15.32:40348) 2020-12-05T17:41:07.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc9696ca700 1 -- 172.21.15.32:0/4253831665 learned_addr learned my addr 172.21.15.32:0/4253831665 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:07.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc968ec9700 1 --2- 172.21.15.32:0/4253831665 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc964065480 0x7fc964065900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:07.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc969ecb700 1 --2- 172.21.15.32:0/4253831665 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc964065fa0 0x7fc964066420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc9696ca700 1 -- 172.21.15.32:0/4253831665 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc964065480 msgr2=0x7fc964065900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc9696ca700 1 --2- 172.21.15.32:0/4253831665 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc964065480 0x7fc964065900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc9696ca700 1 -- 172.21.15.32:0/4253831665 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc964065fa0 msgr2=0x7fc964066420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc9696ca700 1 --2- 172.21.15.32:0/4253831665 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc964065fa0 0x7fc964066420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc9696ca700 1 -- 172.21.15.32:0/4253831665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9641446b0 con 0x7fc96414ab50 2020-12-05T17:41:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc9696ca700 1 --2- 172.21.15.32:0/4253831665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc96414ab50 0x7fc964064f90 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7fc9600013e0 tx=0x7fc960015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc95a7fc700 1 -- 172.21.15.32:0/4253831665 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc960004920 con 0x7fc96414ab50 2020-12-05T17:41:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc95a7fc700 1 -- 172.21.15.32:0/4253831665 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc960004a80 con 0x7fc96414ab50 2020-12-05T17:41:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc96a6cc700 1 -- 172.21.15.32:0/4253831665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc964149b60 con 0x7fc96414ab50 2020-12-05T17:41:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc95a7fc700 1 -- 172.21.15.32:0/4253831665 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc960004de0 con 0x7fc96414ab50 2020-12-05T17:41:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc96a6cc700 1 -- 172.21.15.32:0/4253831665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9641495f0 con 0x7fc96414ab50 2020-12-05T17:41:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc96a6cc700 1 -- 172.21.15.32:0/4253831665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc964057050 con 0x7fc96414ab50 2020-12-05T17:41:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.390+0000 7fc95a7fc700 1 -- 172.21.15.32:0/4253831665 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc960026070 con 0x7fc96414ab50 2020-12-05T17:41:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.394+0000 7fc95a7fc700 1 --2- 172.21.15.32:0/4253831665 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc94c055330 0x7fc94c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.394+0000 7fc95a7fc700 1 -- 172.21.15.32:0/4253831665 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc96006e760 con 0x7fc96414ab50 2020-12-05T17:41:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.394+0000 7fc968ec9700 1 --2- 172.21.15.32:0/4253831665 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc94c055330 0x7fc94c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.394+0000 7fc968ec9700 1 --2- 172.21.15.32:0/4253831665 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc94c055330 0x7fc94c0577f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fc954008000 tx=0x7fc954008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:07.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.402+0000 7fc95a7fc700 1 -- 172.21.15.32:0/4253831665 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc960044900 con 0x7fc96414ab50 2020-12-05T17:41:07.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.546+0000 7fc96a6cc700 1 -- 172.21.15.32:0/4253831665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc96405aab0 con 0x7fc96414ab50 2020-12-05T17:41:07.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.546+0000 7fc95a7fc700 1 -- 172.21.15.32:0/4253831665 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v129) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc960047f20 con 0x7fc96414ab50 2020-12-05T17:41:07.551 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:41:07.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.546+0000 7fc96a6cc700 1 -- 172.21.15.32:0/4253831665 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc94c055330 msgr2=0x7fc94c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:07.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.546+0000 7fc96a6cc700 1 --2- 172.21.15.32:0/4253831665 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc94c055330 0x7fc94c0577f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fc954008000 tx=0x7fc954008040).stop 2020-12-05T17:41:07.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.546+0000 7fc96a6cc700 1 -- 172.21.15.32:0/4253831665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc96414ab50 msgr2=0x7fc964064f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:07.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.546+0000 7fc96a6cc700 1 --2- 172.21.15.32:0/4253831665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc96414ab50 0x7fc964064f90 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7fc9600013e0 tx=0x7fc960015000).stop 2020-12-05T17:41:07.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.546+0000 7fc96a6cc700 1 -- 172.21.15.32:0/4253831665 shutdown_connections 2020-12-05T17:41:07.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.546+0000 7fc96a6cc700 1 --2- 172.21.15.32:0/4253831665 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc964065fa0 0x7fc964066420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.546+0000 7fc96a6cc700 1 --2- 172.21.15.32:0/4253831665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc96414ab50 0x7fc964064f90 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.546+0000 7fc96a6cc700 1 --2- 172.21.15.32:0/4253831665 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fc94c055330 0x7fc94c0577f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.546+0000 7fc96a6cc700 1 --2- 172.21.15.32:0/4253831665 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc964065480 0x7fc964065900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.546+0000 7fc96a6cc700 1 -- 172.21.15.32:0/4253831665 >> 172.21.15.32:0/4253831665 conn(0x7fc9640b5620 msgr2=0x7fc9640b2940 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.546+0000 7fc96a6cc700 1 -- 172.21.15.32:0/4253831665 shutdown_connections 2020-12-05T17:41:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.546+0000 7fc96a6cc700 1 -- 172.21.15.32:0/4253831665 wait complete. 2020-12-05T17:41:07.565 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:41:07.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.678+0000 7ffb2cd41700 1 Processor -- start 2020-12-05T17:41:07.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.678+0000 7ffb2cd41700 1 -- start start 2020-12-05T17:41:07.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb2cd41700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ffb2805feb0 0x7ffb2805f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:07.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb2cd41700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb280b2fb0 0x7ffb28061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:07.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb2cd41700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb280b2df0 con 0x7ffb2805f3b0 2020-12-05T17:41:07.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb2cd41700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb280b3c90 con 0x7ffb2805feb0 2020-12-05T17:41:07.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb2cd41700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb280b2b50 con 0x7ffb280b2fb0 2020-12-05T17:41:07.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb26ffd700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ffb2805feb0 0x7ffb2805f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:07.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb27fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb280b2fb0 0x7ffb28061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:07.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb26ffd700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ffb2805feb0 0x7ffb2805f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:60250/0 (socket says 172.21.15.32:60250) 2020-12-05T17:41:07.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb26ffd700 1 -- 172.21.15.32:0/713183900 learned_addr learned my addr 172.21.15.32:0/713183900 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:07.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb27fff700 1 -- 172.21.15.32:0/713183900 >> v2:172.21.15.61:3300/0 conn(0x7ffb2805feb0 msgr2=0x7ffb2805f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:07.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb27fff700 1 --2- 172.21.15.32:0/713183900 >> v2:172.21.15.61:3300/0 conn(0x7ffb2805feb0 0x7ffb2805f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb27fff700 1 -- 172.21.15.32:0/713183900 >> v1:172.21.15.61:6789/0 conn(0x7ffb2805f3b0 legacy=0x7ffb2813c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:07.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb27fff700 1 -- 172.21.15.32:0/713183900 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb28142820 con 0x7ffb280b2fb0 2020-12-05T17:41:07.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb26ffd700 1 --2- 172.21.15.32:0/713183900 >> v2:172.21.15.61:3300/0 conn(0x7ffb2805feb0 0x7ffb2805f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:07.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb27fff700 1 --2- 172.21.15.32:0/713183900 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb280b2fb0 0x7ffb28061780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7ffb200032b0 tx=0x7ffb200091a0).ready entity=mon.2 client_cookie=5f554294afb8e675 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:07.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb267fc700 1 -- 172.21.15.32:0/713183900 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb2000b9a0 con 0x7ffb280b2fb0 2020-12-05T17:41:07.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb267fc700 1 -- 172.21.15.32:0/713183900 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2000bb00 con 0x7ffb280b2fb0 2020-12-05T17:41:07.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb267fc700 1 -- 172.21.15.32:0/713183900 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb2000bc70 con 0x7ffb280b2fb0 2020-12-05T17:41:07.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb2cd41700 1 -- 172.21.15.32:0/713183900 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb280b2fb0 msgr2=0x7ffb28061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:07.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb2cd41700 1 --2- 172.21.15.32:0/713183900 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb280b2fb0 0x7ffb28061780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7ffb200032b0 tx=0x7ffb200091a0).stop 2020-12-05T17:41:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb2cd41700 1 -- 172.21.15.32:0/713183900 shutdown_connections 2020-12-05T17:41:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb2cd41700 1 --2- 172.21.15.32:0/713183900 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb280b2fb0 0x7ffb28061780 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb2cd41700 1 --2- 172.21.15.32:0/713183900 >> v2:172.21.15.61:3300/0 conn(0x7ffb2805feb0 0x7ffb2805f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb2cd41700 1 -- 172.21.15.32:0/713183900 >> 172.21.15.32:0/713183900 conn(0x7ffb280b8f20 msgr2=0x7ffb280b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb2cd41700 1 -- 172.21.15.32:0/713183900 shutdown_connections 2020-12-05T17:41:07.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.682+0000 7ffb2cd41700 1 -- 172.21.15.32:0/713183900 wait complete. 2020-12-05T17:41:07.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb2cd41700 1 Processor -- start 2020-12-05T17:41:07.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb2cd41700 1 -- start start 2020-12-05T17:41:07.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb2cd41700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb2805f3b0 0x7ffb28145b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:07.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb2cd41700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb2805feb0 0x7ffb28154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb2cd41700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffb280b2fb0 0x7ffb28066350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb2cd41700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb28065d20 con 0x7ffb2805f3b0 2020-12-05T17:41:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb2cd41700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb28064f50 con 0x7ffb280b2fb0 2020-12-05T17:41:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb2cd41700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb28065f50 con 0x7ffb2805feb0 2020-12-05T17:41:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb277fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb2805f3b0 0x7ffb28145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb277fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb2805f3b0 0x7ffb28145b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40362/0 (socket says 172.21.15.32:40362) 2020-12-05T17:41:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb277fe700 1 -- 172.21.15.32:0/2073042959 learned_addr learned my addr 172.21.15.32:0/2073042959 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb27fff700 1 --2- 172.21.15.32:0/2073042959 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffb280b2fb0 0x7ffb28066350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb26ffd700 1 --2- 172.21.15.32:0/2073042959 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb2805feb0 0x7ffb28154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb277fe700 1 -- 172.21.15.32:0/2073042959 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb2805feb0 msgr2=0x7ffb28154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb277fe700 1 --2- 172.21.15.32:0/2073042959 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb2805feb0 0x7ffb28154a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb277fe700 1 -- 172.21.15.32:0/2073042959 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffb280b2fb0 msgr2=0x7ffb28066350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:07.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb277fe700 1 --2- 172.21.15.32:0/2073042959 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffb280b2fb0 0x7ffb28066350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb277fe700 1 -- 172.21.15.32:0/2073042959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb28066840 con 0x7ffb2805f3b0 2020-12-05T17:41:07.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb27fff700 1 --2- 172.21.15.32:0/2073042959 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffb280b2fb0 0x7ffb28066350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:07.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb277fe700 1 --2- 172.21.15.32:0/2073042959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb2805f3b0 0x7ffb28145b00 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7ffb28061550 tx=0x7ffb10007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:07.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb24ff9700 1 -- 172.21.15.32:0/2073042959 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb10007240 con 0x7ffb2805f3b0 2020-12-05T17:41:07.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb24ff9700 1 -- 172.21.15.32:0/2073042959 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb100073a0 con 0x7ffb2805f3b0 2020-12-05T17:41:07.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb2cd41700 1 -- 172.21.15.32:0/2073042959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb281466b0 con 0x7ffb2805f3b0 2020-12-05T17:41:07.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb24ff9700 1 -- 172.21.15.32:0/2073042959 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb100064e0 con 0x7ffb2805f3b0 2020-12-05T17:41:07.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.686+0000 7ffb2cd41700 1 -- 172.21.15.32:0/2073042959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb28066af0 con 0x7ffb2805f3b0 2020-12-05T17:41:07.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.690+0000 7ffb2cd41700 1 -- 172.21.15.32:0/2073042959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb28113ff0 con 0x7ffb2805f3b0 2020-12-05T17:41:07.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.690+0000 7ffb24ff9700 1 -- 172.21.15.32:0/2073042959 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ffb10010070 con 0x7ffb2805f3b0 2020-12-05T17:41:07.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.690+0000 7ffb24ff9700 1 --2- 172.21.15.32:0/2073042959 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffb0c055440 0x7ffb0c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:07.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.690+0000 7ffb24ff9700 1 -- 172.21.15.32:0/2073042959 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ffb1006d730 con 0x7ffb2805f3b0 2020-12-05T17:41:07.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.690+0000 7ffb26ffd700 1 --2- 172.21.15.32:0/2073042959 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffb0c055440 0x7ffb0c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:07.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.690+0000 7ffb26ffd700 1 --2- 172.21.15.32:0/2073042959 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffb0c055440 0x7ffb0c057900 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7ffb18008000 tx=0x7ffb18008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:07.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.698+0000 7ffb24ff9700 1 -- 172.21.15.32:0/2073042959 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ffb10043740 con 0x7ffb2805f3b0 2020-12-05T17:41:07.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.842+0000 7ffb2cd41700 1 -- 172.21.15.32:0/2073042959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffb28057050 con 0x7ffb2805f3b0 2020-12-05T17:41:07.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.842+0000 7ffb24ff9700 1 -- 172.21.15.32:0/2073042959 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v129) v1 ==== 104+0+10 (secure 0 0 0) 0x7ffb10018070 con 0x7ffb2805f3b0 2020-12-05T17:41:07.849 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:41:07.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.846+0000 7ffb2cd41700 1 -- 172.21.15.32:0/2073042959 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffb0c055440 msgr2=0x7ffb0c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:07.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.846+0000 7ffb2cd41700 1 --2- 172.21.15.32:0/2073042959 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffb0c055440 0x7ffb0c057900 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7ffb18008000 tx=0x7ffb18008040).stop 2020-12-05T17:41:07.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.846+0000 7ffb2cd41700 1 -- 172.21.15.32:0/2073042959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb2805f3b0 msgr2=0x7ffb28145b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:07.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.846+0000 7ffb2cd41700 1 --2- 172.21.15.32:0/2073042959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb2805f3b0 0x7ffb28145b00 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7ffb28061550 tx=0x7ffb10007be0).stop 2020-12-05T17:41:07.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.846+0000 7ffb2cd41700 1 -- 172.21.15.32:0/2073042959 shutdown_connections 2020-12-05T17:41:07.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.846+0000 7ffb2cd41700 1 --2- 172.21.15.32:0/2073042959 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffb280b2fb0 0x7ffb28066350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.846+0000 7ffb2cd41700 1 --2- 172.21.15.32:0/2073042959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb2805f3b0 0x7ffb28145b00 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.846+0000 7ffb2cd41700 1 --2- 172.21.15.32:0/2073042959 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffb0c055440 0x7ffb0c057900 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.846+0000 7ffb2cd41700 1 --2- 172.21.15.32:0/2073042959 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb2805feb0 0x7ffb28154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:07.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.846+0000 7ffb2cd41700 1 -- 172.21.15.32:0/2073042959 >> 172.21.15.32:0/2073042959 conn(0x7ffb280b8f20 msgr2=0x7ffb28061780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:07.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.846+0000 7ffb2cd41700 1 -- 172.21.15.32:0/2073042959 shutdown_connections 2020-12-05T17:41:07.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.846+0000 7ffb2cd41700 1 -- 172.21.15.32:0/2073042959 wait complete. 2020-12-05T17:41:07.869 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:41:08.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.994+0000 7fce7daab700 1 Processor -- start 2020-12-05T17:41:08.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.994+0000 7fce7daab700 1 -- start start 2020-12-05T17:41:08.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.994+0000 7fce7daab700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fce780b2000 0x7fce78142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.994+0000 7fce7daab700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fce78143410 0x7fce7813f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.994+0000 7fce7daab700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce780b23e0 con 0x7fce7813fb00 2020-12-05T17:41:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.994+0000 7fce7daab700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce780aeba0 con 0x7fce780b2000 2020-12-05T17:41:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.994+0000 7fce7daab700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce780af190 con 0x7fce78143410 2020-12-05T17:41:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.994+0000 7fce7caa9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fce780b2000 0x7fce78142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.994+0000 7fce7d2aa700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fce7813fb00 0x7fce780af4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43022/0 (socket says 172.21.15.32:43022) 2020-12-05T17:41:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.994+0000 7fce7d2aa700 1 -- 172.21.15.32:0/591954494 learned_addr learned my addr 172.21.15.32:0/591954494 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.994+0000 7fce77fff700 1 --2- 172.21.15.32:0/591954494 >> v2:172.21.15.61:3300/0 conn(0x7fce78143410 0x7fce7813f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.998+0000 7fce7caa9700 1 -- 172.21.15.32:0/591954494 >> v2:172.21.15.61:3300/0 conn(0x7fce78143410 msgr2=0x7fce7813f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:08.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.998+0000 7fce777fe700 1 -- 172.21.15.32:0/591954494 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3433296400 0 0) 0x7fce780b23e0 con 0x7fce7813fb00 2020-12-05T17:41:08.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.998+0000 7fce7caa9700 1 --2- 172.21.15.32:0/591954494 >> v2:172.21.15.61:3300/0 conn(0x7fce78143410 0x7fce7813f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:08.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.998+0000 7fce7caa9700 1 -- 172.21.15.32:0/591954494 >> v1:172.21.15.32:6789/0 conn(0x7fce7813fb00 legacy=0x7fce780af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:08.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.998+0000 7fce7caa9700 1 -- 172.21.15.32:0/591954494 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce780ae010 con 0x7fce780b2000 2020-12-05T17:41:08.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.998+0000 7fce7caa9700 1 --2- 172.21.15.32:0/591954494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce780b2000 0x7fce78142f20 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0x7fce780acda0 tx=0x7fce68006dc0).ready entity=mon.0 client_cookie=c1a0bc66b8e2403b server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:08.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.998+0000 7fce777fe700 1 -- 172.21.15.32:0/591954494 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce68009770 con 0x7fce780b2000 2020-12-05T17:41:08.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.998+0000 7fce777fe700 1 -- 172.21.15.32:0/591954494 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce680098d0 con 0x7fce780b2000 2020-12-05T17:41:08.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.998+0000 7fce777fe700 1 -- 172.21.15.32:0/591954494 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce68009c30 con 0x7fce780b2000 2020-12-05T17:41:08.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.998+0000 7fce7daab700 1 -- 172.21.15.32:0/591954494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce780b2000 msgr2=0x7fce78142f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:08.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.998+0000 7fce7daab700 1 --2- 172.21.15.32:0/591954494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce780b2000 0x7fce78142f20 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0x7fce780acda0 tx=0x7fce68006dc0).stop 2020-12-05T17:41:08.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.998+0000 7fce7daab700 1 -- 172.21.15.32:0/591954494 shutdown_connections 2020-12-05T17:41:08.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.998+0000 7fce7daab700 1 --2- 172.21.15.32:0/591954494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce780b2000 0x7fce78142f20 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:08.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.998+0000 7fce7daab700 1 --2- 172.21.15.32:0/591954494 >> v2:172.21.15.61:3300/0 conn(0x7fce78143410 0x7fce7813f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:08.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.998+0000 7fce7daab700 1 -- 172.21.15.32:0/591954494 >> 172.21.15.32:0/591954494 conn(0x7fce780b5620 msgr2=0x7fce780b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:08.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.998+0000 7fce7daab700 1 -- 172.21.15.32:0/591954494 shutdown_connections 2020-12-05T17:41:08.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.998+0000 7fce7daab700 1 -- 172.21.15.32:0/591954494 wait complete. 2020-12-05T17:41:08.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.998+0000 7fce7daab700 1 Processor -- start 2020-12-05T17:41:08.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:07.998+0000 7fce7daab700 1 -- start start 2020-12-05T17:41:08.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce7daab700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce780b2000 0x7fce78140fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:08.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce7daab700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce7813fb00 0x7fce78143ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:08.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce7daab700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce78143410 0x7fce78065380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:08.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce7daab700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce78064f50 con 0x7fce7813fb00 2020-12-05T17:41:08.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce7daab700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce78145450 con 0x7fce78143410 2020-12-05T17:41:08.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce7daab700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce781456f0 con 0x7fce780b2000 2020-12-05T17:41:08.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce77fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce7813fb00 0x7fce78143ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:08.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce77fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce7813fb00 0x7fce78143ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40376/0 (socket says 172.21.15.32:40376) 2020-12-05T17:41:08.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce77fff700 1 -- 172.21.15.32:0/1304842513 learned_addr learned my addr 172.21.15.32:0/1304842513 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:08.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce7d2aa700 1 --2- 172.21.15.32:0/1304842513 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce78143410 0x7fce78065380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:08.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce7caa9700 1 --2- 172.21.15.32:0/1304842513 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce780b2000 0x7fce78140fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:08.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce77fff700 1 -- 172.21.15.32:0/1304842513 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce780b2000 msgr2=0x7fce78140fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:08.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce77fff700 1 --2- 172.21.15.32:0/1304842513 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce780b2000 0x7fce78140fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:08.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce77fff700 1 -- 172.21.15.32:0/1304842513 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce78143410 msgr2=0x7fce78065380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:08.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce77fff700 1 --2- 172.21.15.32:0/1304842513 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce78143410 0x7fce78065380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:08.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce77fff700 1 -- 172.21.15.32:0/1304842513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce781450d0 con 0x7fce7813fb00 2020-12-05T17:41:08.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce77fff700 1 --2- 172.21.15.32:0/1304842513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce7813fb00 0x7fce78143ba0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7fce6000c000 tx=0x7fce60006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:08.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce75ffb700 1 -- 172.21.15.32:0/1304842513 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce6000ca10 con 0x7fce7813fb00 2020-12-05T17:41:08.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce75ffb700 1 -- 172.21.15.32:0/1304842513 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6000cb70 con 0x7fce7813fb00 2020-12-05T17:41:08.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce75ffb700 1 -- 172.21.15.32:0/1304842513 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce6000be30 con 0x7fce7813fb00 2020-12-05T17:41:08.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce7daab700 1 -- 172.21.15.32:0/1304842513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce78146980 con 0x7fce7813fb00 2020-12-05T17:41:08.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce7daab700 1 -- 172.21.15.32:0/1304842513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce78146440 con 0x7fce7813fb00 2020-12-05T17:41:08.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce75ffb700 1 -- 172.21.15.32:0/1304842513 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fce6001c460 con 0x7fce7813fb00 2020-12-05T17:41:08.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce7daab700 1 -- 172.21.15.32:0/1304842513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce78057050 con 0x7fce7813fb00 2020-12-05T17:41:08.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce75ffb700 1 --2- 172.21.15.32:0/1304842513 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fce5c055330 0x7fce5c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:08.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.002+0000 7fce75ffb700 1 -- 172.21.15.32:0/1304842513 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fce6006e470 con 0x7fce7813fb00 2020-12-05T17:41:08.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.006+0000 7fce7caa9700 1 --2- 172.21.15.32:0/1304842513 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fce5c055330 0x7fce5c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:08.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.006+0000 7fce7caa9700 1 --2- 172.21.15.32:0/1304842513 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fce5c055330 0x7fce5c0577f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7fce6801b000 tx=0x7fce68015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:08.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.010+0000 7fce75ffb700 1 -- 172.21.15.32:0/1304842513 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fce60044510 con 0x7fce7813fb00 2020-12-05T17:41:08.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.158+0000 7fce7daab700 1 -- 172.21.15.32:0/1304842513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fce7805aab0 con 0x7fce7813fb00 2020-12-05T17:41:08.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.158+0000 7fce75ffb700 1 -- 172.21.15.32:0/1304842513 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v129) v1 ==== 100+0+10 (secure 0 0 0) 0x7fce60047b30 con 0x7fce7813fb00 2020-12-05T17:41:08.163 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:41:08.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.158+0000 7fce7daab700 1 -- 172.21.15.32:0/1304842513 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fce5c055330 msgr2=0x7fce5c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:08.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.158+0000 7fce7daab700 1 --2- 172.21.15.32:0/1304842513 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fce5c055330 0x7fce5c0577f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7fce6801b000 tx=0x7fce68015000).stop 2020-12-05T17:41:08.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.158+0000 7fce7daab700 1 -- 172.21.15.32:0/1304842513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce7813fb00 msgr2=0x7fce78143ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:08.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.158+0000 7fce7daab700 1 --2- 172.21.15.32:0/1304842513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce7813fb00 0x7fce78143ba0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7fce6000c000 tx=0x7fce60006dc0).stop 2020-12-05T17:41:08.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.158+0000 7fce7daab700 1 -- 172.21.15.32:0/1304842513 shutdown_connections 2020-12-05T17:41:08.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.158+0000 7fce7daab700 1 --2- 172.21.15.32:0/1304842513 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce78143410 0x7fce78065380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:08.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.158+0000 7fce7daab700 1 --2- 172.21.15.32:0/1304842513 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fce5c055330 0x7fce5c0577f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:08.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.158+0000 7fce7daab700 1 --2- 172.21.15.32:0/1304842513 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce780b2000 0x7fce78140fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:08.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.158+0000 7fce7daab700 1 --2- 172.21.15.32:0/1304842513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce7813fb00 0x7fce78143ba0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:08.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.158+0000 7fce7daab700 1 -- 172.21.15.32:0/1304842513 >> 172.21.15.32:0/1304842513 conn(0x7fce780b5620 msgr2=0x7fce780b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:08.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.158+0000 7fce7daab700 1 -- 172.21.15.32:0/1304842513 shutdown_connections 2020-12-05T17:41:08.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.158+0000 7fce7daab700 1 -- 172.21.15.32:0/1304842513 wait complete. 2020-12-05T17:41:08.176 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:41:08.176 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:41:08.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d05eba700 1 Processor -- start 2020-12-05T17:41:08.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d05eba700 1 -- start start 2020-12-05T17:41:08.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d05eba700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1d0005f3b0 0x7f1d0005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:08.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d05eba700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1d00062400 0x7f1d000b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:08.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d05eba700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1d0005feb0 0x7f1d000b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:08.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d05eba700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d00142450 con 0x7f1d0005feb0 2020-12-05T17:41:08.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d05eba700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d00061780 con 0x7f1d0005f3b0 2020-12-05T17:41:08.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d05eba700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d00061a20 con 0x7f1d00062400 2020-12-05T17:41:08.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d056b9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1d0005feb0 0x7f1d000b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:08.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d056b9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1d0005feb0 0x7f1d000b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40388/0 (socket says 172.21.15.32:40388) 2020-12-05T17:41:08.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d056b9700 1 -- 172.21.15.32:0/3212124357 learned_addr learned my addr 172.21.15.32:0/3212124357 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:08.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d04eb8700 1 --2- 172.21.15.32:0/3212124357 >> v2:172.21.15.61:3300/0 conn(0x7f1d0005f3b0 0x7f1d0005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:08.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1cfffff700 1 --2- 172.21.15.32:0/3212124357 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1d00062400 0x7f1d000b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:08.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d056b9700 1 -- 172.21.15.32:0/3212124357 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1d00062400 msgr2=0x7f1d000b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:08.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d056b9700 1 --2- 172.21.15.32:0/3212124357 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1d00062400 0x7f1d000b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d056b9700 1 -- 172.21.15.32:0/3212124357 >> v2:172.21.15.61:3300/0 conn(0x7f1d0005f3b0 msgr2=0x7f1d0005f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d056b9700 1 --2- 172.21.15.32:0/3212124357 >> v2:172.21.15.61:3300/0 conn(0x7f1d0005f3b0 0x7f1d0005f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d056b9700 1 -- 172.21.15.32:0/3212124357 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d00061cc0 con 0x7f1d0005feb0 2020-12-05T17:41:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d056b9700 1 --2- 172.21.15.32:0/3212124357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d0005feb0 0x7f1d000b3be0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7f1cf80032b0 tx=0x7f1cf8009270).ready entity=mon.0 client_cookie=9eafb1e0c6d1fc5b server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1cff7fe700 1 -- 172.21.15.32:0/3212124357 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cf800bc00 con 0x7f1d0005feb0 2020-12-05T17:41:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1cff7fe700 1 -- 172.21.15.32:0/3212124357 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf800bd60 con 0x7f1d0005feb0 2020-12-05T17:41:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1cff7fe700 1 -- 172.21.15.32:0/3212124357 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cf8011400 con 0x7f1d0005feb0 2020-12-05T17:41:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d05eba700 1 -- 172.21.15.32:0/3212124357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d0005feb0 msgr2=0x7f1d000b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d05eba700 1 --2- 172.21.15.32:0/3212124357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d0005feb0 0x7f1d000b3be0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7f1cf80032b0 tx=0x7f1cf8009270).stop 2020-12-05T17:41:08.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d05eba700 1 -- 172.21.15.32:0/3212124357 shutdown_connections 2020-12-05T17:41:08.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d05eba700 1 --2- 172.21.15.32:0/3212124357 >> v2:172.21.15.61:3300/0 conn(0x7f1d0005f3b0 0x7f1d0005f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:08.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d05eba700 1 --2- 172.21.15.32:0/3212124357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d0005feb0 0x7f1d000b3be0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:08.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d05eba700 1 --2- 172.21.15.32:0/3212124357 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1d00062400 0x7f1d000b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:08.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d05eba700 1 -- 172.21.15.32:0/3212124357 >> 172.21.15.32:0/3212124357 conn(0x7f1d000b8ee0 msgr2=0x7f1d000b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:08.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d05eba700 1 -- 172.21.15.32:0/3212124357 shutdown_connections 2020-12-05T17:41:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.282+0000 7f1d05eba700 1 -- 172.21.15.32:0/3212124357 wait complete. 2020-12-05T17:41:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1d05eba700 1 Processor -- start 2020-12-05T17:41:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1d05eba700 1 -- start start 2020-12-05T17:41:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1d05eba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d0005f3b0 0x7f1d00066290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1d05eba700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d0005feb0 0x7f1d000b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:08.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1d05eba700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1d00062400 0x7f1d000b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:08.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1d05eba700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d001511d0 con 0x7f1d0005f3b0 2020-12-05T17:41:08.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1d05eba700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d001513c0 con 0x7f1d0005feb0 2020-12-05T17:41:08.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1d05eba700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d00152300 con 0x7f1d00062400 2020-12-05T17:41:08.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1d04eb8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d0005f3b0 0x7f1d00066290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:08.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1d04eb8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d0005f3b0 0x7f1d00066290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40390/0 (socket says 172.21.15.32:40390) 2020-12-05T17:41:08.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1d04eb8700 1 -- 172.21.15.32:0/2936510167 learned_addr learned my addr 172.21.15.32:0/2936510167 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:08.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1cfffff700 1 --2- 172.21.15.32:0/2936510167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d0005feb0 0x7f1d000b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:08.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1d056b9700 1 --2- 172.21.15.32:0/2936510167 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1d00062400 0x7f1d000b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:08.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1d04eb8700 1 -- 172.21.15.32:0/2936510167 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1d00062400 msgr2=0x7f1d000b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:08.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1d04eb8700 1 --2- 172.21.15.32:0/2936510167 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1d00062400 0x7f1d000b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:08.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1d04eb8700 1 -- 172.21.15.32:0/2936510167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d0005feb0 msgr2=0x7f1d000b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:08.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1d04eb8700 1 --2- 172.21.15.32:0/2936510167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d0005feb0 0x7f1d000b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:08.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1d04eb8700 1 -- 172.21.15.32:0/2936510167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d000b4260 con 0x7f1d0005f3b0 2020-12-05T17:41:08.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1d056b9700 1 --2- 172.21.15.32:0/2936510167 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1d00062400 0x7f1d000b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:41:08.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1cfffff700 1 --2- 172.21.15.32:0/2936510167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d0005feb0 0x7f1d000b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:41:08.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1d04eb8700 1 --2- 172.21.15.32:0/2936510167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d0005f3b0 0x7f1d00066290 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 rx=0x7f1cf4003070 tx=0x7f1cf4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:08.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1cfdffb700 1 -- 172.21.15.32:0/2936510167 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cf4009980 con 0x7f1d0005f3b0 2020-12-05T17:41:08.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1cfdffb700 1 -- 172.21.15.32:0/2936510167 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf4009ae0 con 0x7f1d0005f3b0 2020-12-05T17:41:08.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1d05eba700 1 -- 172.21.15.32:0/2936510167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d000668d0 con 0x7f1d0005f3b0 2020-12-05T17:41:08.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1cfdffb700 1 -- 172.21.15.32:0/2936510167 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cf4009e40 con 0x7f1d0005f3b0 2020-12-05T17:41:08.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1d05eba700 1 -- 172.21.15.32:0/2936510167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d000b1ec0 con 0x7f1d0005f3b0 2020-12-05T17:41:08.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.286+0000 7f1d05eba700 1 -- 172.21.15.32:0/2936510167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d00057050 con 0x7f1d0005f3b0 2020-12-05T17:41:08.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.290+0000 7f1cfdffb700 1 -- 172.21.15.32:0/2936510167 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1cf4013480 con 0x7f1d0005f3b0 2020-12-05T17:41:08.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.290+0000 7f1cfdffb700 1 --2- 172.21.15.32:0/2936510167 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ce8055440 0x7f1ce8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:08.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.290+0000 7f1cfffff700 1 --2- 172.21.15.32:0/2936510167 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ce8055440 0x7f1ce8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:08.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.290+0000 7f1cfdffb700 1 -- 172.21.15.32:0/2936510167 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1cf406e8c0 con 0x7f1d0005f3b0 2020-12-05T17:41:08.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.290+0000 7f1cfffff700 1 --2- 172.21.15.32:0/2936510167 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ce8055440 0x7f1ce8057900 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f1cec008000 tx=0x7f1cec008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:08.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.294+0000 7f1cfdffb700 1 -- 172.21.15.32:0/2936510167 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1cf4044a00 con 0x7f1d0005f3b0 2020-12-05T17:41:08.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.482+0000 7f1d05eba700 1 -- 172.21.15.32:0/2936510167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d000b0b20 con 0x7f1d0005f3b0 2020-12-05T17:41:08.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.486+0000 7f1cfdffb700 1 -- 172.21.15.32:0/2936510167 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f1cf4048020 con 0x7f1d0005f3b0 2020-12-05T17:41:08.490 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:08.490 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":62,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6830,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/3985915039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3985915039},{"type":"v1","addr":"172.21.15.61:6833","nonce":3985915039}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":60}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:41:03.280787+0000","modified":"2020-12-05T17:41:04.283186+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7938":{"gid":7938,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1585653076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1585653076},{"type":"v1","addr":"172.21.15.32:6835","nonce":1585653076}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2020-12-05T17:41:08.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.486+0000 7f1d05eba700 1 -- 172.21.15.32:0/2936510167 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ce8055440 msgr2=0x7f1ce8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:08.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.486+0000 7f1d05eba700 1 --2- 172.21.15.32:0/2936510167 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ce8055440 0x7f1ce8057900 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f1cec008000 tx=0x7f1cec008040).stop 2020-12-05T17:41:08.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.486+0000 7f1d05eba700 1 -- 172.21.15.32:0/2936510167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d0005f3b0 msgr2=0x7f1d00066290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:08.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.486+0000 7f1d05eba700 1 --2- 172.21.15.32:0/2936510167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d0005f3b0 0x7f1d00066290 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 rx=0x7f1cf4003070 tx=0x7f1cf4008040).stop 2020-12-05T17:41:08.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.486+0000 7f1d05eba700 1 -- 172.21.15.32:0/2936510167 shutdown_connections 2020-12-05T17:41:08.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.486+0000 7f1d05eba700 1 --2- 172.21.15.32:0/2936510167 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ce8055440 0x7f1ce8057900 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:08.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.486+0000 7f1d05eba700 1 --2- 172.21.15.32:0/2936510167 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1d00062400 0x7f1d000b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:08.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.486+0000 7f1d05eba700 1 --2- 172.21.15.32:0/2936510167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d0005f3b0 0x7f1d00066290 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:08.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.486+0000 7f1d05eba700 1 --2- 172.21.15.32:0/2936510167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d0005feb0 0x7f1d000b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.486+0000 7f1d05eba700 1 -- 172.21.15.32:0/2936510167 >> 172.21.15.32:0/2936510167 conn(0x7f1d000b8ee0 msgr2=0x7f1d00142620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.486+0000 7f1d05eba700 1 -- 172.21.15.32:0/2936510167 shutdown_connections 2020-12-05T17:41:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:08.490+0000 7f1d05eba700 1 -- 172.21.15.32:0/2936510167 wait complete. 2020-12-05T17:41:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 62 2020-12-05T17:41:08.712 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:41:03.280787+0000', 'modified': '2020-12-05T17:41:04.283186+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7938}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7938': {'gid': 7938, 'name': 'a', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/1585653076', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 1585653076}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 1585653076}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:41:08.712 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:41:08.713 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:41:08.840 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":7938,"want_state":"up:active","state":"up:active","rank_uptime":5.5471893909999999,"mdsmap_epoch":62,"osdmap_epoch":129,"osdmap_epoch_barrier":128,"uptime":10.511390311} 2020-12-05T17:41:08.841 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 7938, "mdsmap_epoch": 62, "osdmap_epoch": 129, "osdmap_epoch_barrier": 128, "rank_uptime": 5.547189391, "state": "up:active", "uptime": 10.511390311, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:41:08.841 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:41:08.842 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-12-05T17:41:08.842 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-12-05T17:41:08.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:08.843 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:41:08.851 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:41:08.906 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:41:08.907 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:41:08.962 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:41:08.963 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:41:08.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:08.964 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:41:09.009 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:41:09.010 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:41:09.053 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:41:09.070 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:41:09.071 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:41:09.129 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:41:09.130 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:41:09.130 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:09.131 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:41:09.132 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:2020-12-05T17:41:09.126+0000 7f47546d80c0 -1 init, newargv = 0x55b377e223e0 newargc=9ceph-fuse[ 2020-12-05T17:41:09.132 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:11567]: starting ceph client 2020-12-05T17:41:09.142 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[11567]: starting fuse 2020-12-05T17:41:10.239 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:41:10.252 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:41:10.252 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:41:10.310 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:41:10.312 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:41:10.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:10.313 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:41:10.361 INFO:teuthology.orchestra.run.smithi036.stdout:49 2020-12-05T17:41:10.361 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-12-05T17:41:10.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:10.362 INFO:teuthology.orchestra.run.smithi036:> sudo python3 -c ' 2020-12-05T17:41:10.363 INFO:teuthology.orchestra.run.smithi036:> import glob 2020-12-05T17:41:10.363 INFO:teuthology.orchestra.run.smithi036:> import re 2020-12-05T17:41:10.363 INFO:teuthology.orchestra.run.smithi036:> import os 2020-12-05T17:41:10.364 INFO:teuthology.orchestra.run.smithi036:> import subprocess 2020-12-05T17:41:10.364 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:41:10.365 INFO:teuthology.orchestra.run.smithi036:> def find_socket(client_name): 2020-12-05T17:41:10.365 INFO:teuthology.orchestra.run.smithi036:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-12-05T17:41:10.365 INFO:teuthology.orchestra.run.smithi036:> files = glob.glob(asok_path) 2020-12-05T17:41:10.366 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:41:10.366 INFO:teuthology.orchestra.run.smithi036:> # Given a non-glob path, it better be there 2020-12-05T17:41:10.366 INFO:teuthology.orchestra.run.smithi036:> if "*" not in asok_path: 2020-12-05T17:41:10.367 INFO:teuthology.orchestra.run.smithi036:> assert(len(files) == 1) 2020-12-05T17:41:10.367 INFO:teuthology.orchestra.run.smithi036:> return files[0] 2020-12-05T17:41:10.368 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:41:10.368 INFO:teuthology.orchestra.run.smithi036:> for f in files: 2020-12-05T17:41:10.368 INFO:teuthology.orchestra.run.smithi036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:41:10.369 INFO:teuthology.orchestra.run.smithi036:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:41:10.369 INFO:teuthology.orchestra.run.smithi036:> return f 2020-12-05T17:41:10.370 INFO:teuthology.orchestra.run.smithi036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:41:10.370 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:41:10.370 INFO:teuthology.orchestra.run.smithi036:> print(find_socket("client.0")) 2020-12-05T17:41:10.371 INFO:teuthology.orchestra.run.smithi036:> ' 2020-12-05T17:41:10.454 INFO:teuthology.orchestra.run.smithi036.stdout:/var/run/ceph/ceph-client.0.11567.asok 2020-12-05T17:41:10.459 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.11567.asok 2020-12-05T17:41:10.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:10.460 INFO:teuthology.orchestra.run.smithi036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.11567.asok status 2020-12-05T17:41:10.566 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2020-12-05T17:41:10.566 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": { 2020-12-05T17:41:10.566 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:41:10.566 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:41:10.567 INFO:teuthology.orchestra.run.smithi036.stdout: "entity_id": "0", 2020-12-05T17:41:10.567 INFO:teuthology.orchestra.run.smithi036.stdout: "hostname": "smithi036", 2020-12-05T17:41:10.567 INFO:teuthology.orchestra.run.smithi036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-12-05T17:41:10.567 INFO:teuthology.orchestra.run.smithi036.stdout: "pid": "11567", 2020-12-05T17:41:10.568 INFO:teuthology.orchestra.run.smithi036.stdout: "root": "/" 2020-12-05T17:41:10.568 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:41:10.568 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_count": 0, 2020-12-05T17:41:10.568 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_pinned_count": 0, 2020-12-05T17:41:10.568 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 7996, 2020-12-05T17:41:10.569 INFO:teuthology.orchestra.run.smithi036.stdout: "inst": { 2020-12-05T17:41:10.569 INFO:teuthology.orchestra.run.smithi036.stdout: "name": { 2020-12-05T17:41:10.569 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "client", 2020-12-05T17:41:10.569 INFO:teuthology.orchestra.run.smithi036.stdout: "num": 7996 2020-12-05T17:41:10.570 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:41:10.570 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:41:10.570 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:41:10.570 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:41:10.570 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 17074197 2020-12-05T17:41:10.571 INFO:teuthology.orchestra.run.smithi036.stdout: } 2020-12-05T17:41:10.571 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:41:10.571 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:41:10.571 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:41:10.571 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:41:10.572 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 17074197 2020-12-05T17:41:10.572 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:41:10.572 INFO:teuthology.orchestra.run.smithi036.stdout: "inst_str": "client.7996 172.21.15.36:0/17074197", 2020-12-05T17:41:10.573 INFO:teuthology.orchestra.run.smithi036.stdout: "addr_str": "172.21.15.36:0/17074197", 2020-12-05T17:41:10.573 INFO:teuthology.orchestra.run.smithi036.stdout: "inode_count": 1, 2020-12-05T17:41:10.574 INFO:teuthology.orchestra.run.smithi036.stdout: "mds_epoch": 62, 2020-12-05T17:41:10.574 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch": 129, 2020-12-05T17:41:10.575 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:41:10.575 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklisted": false 2020-12-05T17:41:10.576 INFO:teuthology.orchestra.run.smithi036.stdout:} 2020-12-05T17:41:10.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:10.577 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:10.578 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:41:10.584 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:41:10.585 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:41:10.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:10.586 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:41:10.643 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:41:10.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.758+0000 7fbef0887700 1 Processor -- start 2020-12-05T17:41:10.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.758+0000 7fbef0887700 1 -- start start 2020-12-05T17:41:10.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.762+0000 7fbef0887700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fbeec0b5f00 0x7fbeec0b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:10.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.762+0000 7fbef0887700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbeec0ad510 0x7fbeec0af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:10.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.762+0000 7fbef0887700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbeec066370 0x7fbeec0667e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:10.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.762+0000 7fbef0887700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeec142ab0 con 0x7fbeec066370 2020-12-05T17:41:10.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.762+0000 7fbef0887700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeec0b62e0 con 0x7fbeec0b5f00 2020-12-05T17:41:10.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.762+0000 7fbef0887700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeec0afbf0 con 0x7fbeec0ad510 2020-12-05T17:41:10.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.762+0000 7fbeebfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbeec066370 0x7fbeec0667e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:10.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.762+0000 7fbeebfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbeec066370 0x7fbeec0667e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40402/0 (socket says 172.21.15.32:40402) 2020-12-05T17:41:10.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.762+0000 7fbeebfff700 1 -- 172.21.15.32:0/2096672429 learned_addr learned my addr 172.21.15.32:0/2096672429 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:10.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.762+0000 7fbee2ffd700 1 --2- 172.21.15.32:0/2096672429 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbeec0ad510 0x7fbeec0af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:10.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.762+0000 7fbeeb7fe700 1 --2- 172.21.15.32:0/2096672429 >> v2:172.21.15.61:3300/0 conn(0x7fbeec0b5f00 0x7fbeec0b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:10.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.762+0000 7fbeebfff700 1 -- 172.21.15.32:0/2096672429 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbeec0ad510 msgr2=0x7fbeec0af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:10.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.762+0000 7fbeebfff700 1 --2- 172.21.15.32:0/2096672429 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbeec0ad510 0x7fbeec0af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:10.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.762+0000 7fbeebfff700 1 -- 172.21.15.32:0/2096672429 >> v2:172.21.15.61:3300/0 conn(0x7fbeec0b5f00 msgr2=0x7fbeec0b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:10.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.762+0000 7fbeebfff700 1 --2- 172.21.15.32:0/2096672429 >> v2:172.21.15.61:3300/0 conn(0x7fbeec0b5f00 0x7fbeec0b14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:10.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.762+0000 7fbeebfff700 1 -- 172.21.15.32:0/2096672429 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeec0afe90 con 0x7fbeec066370 2020-12-05T17:41:10.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.762+0000 7fbee2ffd700 1 --2- 172.21.15.32:0/2096672429 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbeec0ad510 0x7fbeec0af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:10.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.762+0000 7fbeebfff700 1 --2- 172.21.15.32:0/2096672429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeec066370 0x7fbeec0667e0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7fbee40032b0 tx=0x7fbee4009270).ready entity=mon.0 client_cookie=20f5ea495df9b935 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:10.772 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.762+0000 7fbeeaffd700 1 -- 172.21.15.32:0/2096672429 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbee4009e60 con 0x7fbeec066370 2020-12-05T17:41:10.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.762+0000 7fbeeaffd700 1 -- 172.21.15.32:0/2096672429 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee400bc00 con 0x7fbeec066370 2020-12-05T17:41:10.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.762+0000 7fbeeaffd700 1 -- 172.21.15.32:0/2096672429 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbee400bd70 con 0x7fbeec066370 2020-12-05T17:41:10.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.762+0000 7fbef0887700 1 -- 172.21.15.32:0/2096672429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeec066370 msgr2=0x7fbeec0667e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:10.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.762+0000 7fbef0887700 1 --2- 172.21.15.32:0/2096672429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeec066370 0x7fbeec0667e0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7fbee40032b0 tx=0x7fbee4009270).stop 2020-12-05T17:41:10.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.766+0000 7fbef0887700 1 -- 172.21.15.32:0/2096672429 shutdown_connections 2020-12-05T17:41:10.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.766+0000 7fbef0887700 1 --2- 172.21.15.32:0/2096672429 >> v2:172.21.15.61:3300/0 conn(0x7fbeec0b5f00 0x7fbeec0b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:10.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.766+0000 7fbef0887700 1 --2- 172.21.15.32:0/2096672429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeec066370 0x7fbeec0667e0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:10.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.766+0000 7fbef0887700 1 --2- 172.21.15.32:0/2096672429 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbeec0ad510 0x7fbeec0af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:10.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.766+0000 7fbef0887700 1 -- 172.21.15.32:0/2096672429 >> 172.21.15.32:0/2096672429 conn(0x7fbeec0604c0 msgr2=0x7fbeec05c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:10.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.766+0000 7fbef0887700 1 -- 172.21.15.32:0/2096672429 shutdown_connections 2020-12-05T17:41:10.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.766+0000 7fbef0887700 1 -- 172.21.15.32:0/2096672429 wait complete. 2020-12-05T17:41:10.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.766+0000 7fbef0887700 1 Processor -- start 2020-12-05T17:41:10.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.766+0000 7fbef0887700 1 -- start start 2020-12-05T17:41:10.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbef0887700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbeec066370 0x7fbeec0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:10.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbef0887700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbeec0ad510 0x7fbeec145e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:10.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbef0887700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeec0b5f00 0x7fbeec146350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:10.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbef0887700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeec141740 con 0x7fbeec0b5f00 2020-12-05T17:41:10.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbef0887700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeec065710 con 0x7fbeec066370 2020-12-05T17:41:10.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbef0887700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeec065990 con 0x7fbeec0ad510 2020-12-05T17:41:10.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbeebfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeec0b5f00 0x7fbeec146350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:10.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbeebfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeec0b5f00 0x7fbeec146350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40408/0 (socket says 172.21.15.32:40408) 2020-12-05T17:41:10.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbeebfff700 1 -- 172.21.15.32:0/4118676283 learned_addr learned my addr 172.21.15.32:0/4118676283 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:10.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbee2ffd700 1 --2- 172.21.15.32:0/4118676283 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbeec0ad510 0x7fbeec145e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:10.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbeeb7fe700 1 --2- 172.21.15.32:0/4118676283 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbeec066370 0x7fbeec0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:10.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbeebfff700 1 -- 172.21.15.32:0/4118676283 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbeec0ad510 msgr2=0x7fbeec145e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:10.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbeebfff700 1 --2- 172.21.15.32:0/4118676283 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbeec0ad510 0x7fbeec145e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:10.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbeebfff700 1 -- 172.21.15.32:0/4118676283 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbeec066370 msgr2=0x7fbeec0b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:10.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbeebfff700 1 --2- 172.21.15.32:0/4118676283 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbeec066370 0x7fbeec0b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:10.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbeebfff700 1 -- 172.21.15.32:0/4118676283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeec065420 con 0x7fbeec0b5f00 2020-12-05T17:41:10.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbee2ffd700 1 --2- 172.21.15.32:0/4118676283 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbeec0ad510 0x7fbeec145e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:10.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbeeb7fe700 1 --2- 172.21.15.32:0/4118676283 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbeec066370 0x7fbeec0b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:10.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbeebfff700 1 --2- 172.21.15.32:0/4118676283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeec0b5f00 0x7fbeec146350 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0x7fbeec0658a0 tx=0x7fbee4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:10.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbee97fa700 1 -- 172.21.15.32:0/4118676283 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbee40107a0 con 0x7fbeec0b5f00 2020-12-05T17:41:10.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbee97fa700 1 -- 172.21.15.32:0/4118676283 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee400bed0 con 0x7fbeec0b5f00 2020-12-05T17:41:10.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbef0887700 1 -- 172.21.15.32:0/4118676283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeec13e7e0 con 0x7fbeec0b5f00 2020-12-05T17:41:10.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbee97fa700 1 -- 172.21.15.32:0/4118676283 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbee4010af0 con 0x7fbeec0b5f00 2020-12-05T17:41:10.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbef0887700 1 -- 172.21.15.32:0/4118676283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeec143bd0 con 0x7fbeec0b5f00 2020-12-05T17:41:10.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.770+0000 7fbef0887700 1 -- 172.21.15.32:0/4118676283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeec1415a0 con 0x7fbeec0b5f00 2020-12-05T17:41:10.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.774+0000 7fbee97fa700 1 -- 172.21.15.32:0/4118676283 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbee400f070 con 0x7fbeec0b5f00 2020-12-05T17:41:10.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.774+0000 7fbee97fa700 1 --2- 172.21.15.32:0/4118676283 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbed0055440 0x7fbed0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:10.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.774+0000 7fbee97fa700 1 -- 172.21.15.32:0/4118676283 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbee406f800 con 0x7fbeec0b5f00 2020-12-05T17:41:10.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.774+0000 7fbeeb7fe700 1 --2- 172.21.15.32:0/4118676283 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbed0055440 0x7fbed0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:10.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.774+0000 7fbeeb7fe700 1 --2- 172.21.15.32:0/4118676283 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbed0055440 0x7fbed0057900 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7fbedc00b000 tx=0x7fbedc008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:10.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.778+0000 7fbee97fa700 1 -- 172.21.15.32:0/4118676283 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbeec1415a0 con 0x7fbeec0b5f00 2020-12-05T17:41:10.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.918+0000 7fbef0887700 1 -- 172.21.15.32:0/4118676283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbeec1415a0 con 0x7fbeec0b5f00 2020-12-05T17:41:10.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.922+0000 7fbee97fa700 1 -- 172.21.15.32:0/4118676283 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fbeec1415a0 con 0x7fbeec0b5f00 2020-12-05T17:41:10.927 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:10.927 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":129,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:41:04.284002+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:41:01.240356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2020-12-05T17:41:02.269469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:41:10.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.922+0000 7fbef0887700 1 -- 172.21.15.32:0/4118676283 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbed0055440 msgr2=0x7fbed0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:10.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.922+0000 7fbef0887700 1 --2- 172.21.15.32:0/4118676283 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbed0055440 0x7fbed0057900 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7fbedc00b000 tx=0x7fbedc008040).stop 2020-12-05T17:41:10.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.922+0000 7fbef0887700 1 -- 172.21.15.32:0/4118676283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeec0b5f00 msgr2=0x7fbeec146350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:10.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.922+0000 7fbef0887700 1 --2- 172.21.15.32:0/4118676283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeec0b5f00 0x7fbeec146350 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0x7fbeec0658a0 tx=0x7fbee4016000).stop 2020-12-05T17:41:10.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.922+0000 7fbef0887700 1 -- 172.21.15.32:0/4118676283 shutdown_connections 2020-12-05T17:41:10.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.922+0000 7fbef0887700 1 --2- 172.21.15.32:0/4118676283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeec0b5f00 0x7fbeec146350 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:10.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.922+0000 7fbef0887700 1 --2- 172.21.15.32:0/4118676283 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbeec066370 0x7fbeec0b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:10.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.922+0000 7fbef0887700 1 --2- 172.21.15.32:0/4118676283 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbed0055440 0x7fbed0057900 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:10.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.922+0000 7fbef0887700 1 --2- 172.21.15.32:0/4118676283 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbeec0ad510 0x7fbeec145e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:10.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.922+0000 7fbef0887700 1 -- 172.21.15.32:0/4118676283 >> 172.21.15.32:0/4118676283 conn(0x7fbeec0604c0 msgr2=0x7fbeec142e00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:10.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.922+0000 7fbef0887700 1 -- 172.21.15.32:0/4118676283 shutdown_connections 2020-12-05T17:41:10.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:10.922+0000 7fbef0887700 1 -- 172.21.15.32:0/4118676283 wait complete. 2020-12-05T17:41:10.941 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:41:01.240356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '128', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:41:02.269469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:41:10.942 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:41:11.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.046+0000 7f493958b700 1 Processor -- start 2020-12-05T17:41:11.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.046+0000 7f493958b700 1 -- start start 2020-12-05T17:41:11.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.046+0000 7f493958b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4934061b30 0x7f4934141e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:11.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.046+0000 7f493958b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49341439c0 con 0x7f493405f6f0 2020-12-05T17:41:11.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.046+0000 7f493958b700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f493405fe20 con 0x7f4934142300 2020-12-05T17:41:11.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.046+0000 7f493958b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4934146910 con 0x7f4934061b30 2020-12-05T17:41:11.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.046+0000 7f4933fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f493405f6f0 0x7f4934141700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43060/0 (socket says 172.21.15.32:43060) 2020-12-05T17:41:11.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.046+0000 7f4933fff700 1 -- 172.21.15.32:0/1696603589 learned_addr learned my addr 172.21.15.32:0/1696603589 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:11.053 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.046+0000 7f49337fe700 1 --2- 172.21.15.32:0/1696603589 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4934061b30 0x7f4934141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:11.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.046+0000 7f4932ffd700 1 -- 172.21.15.32:0/1696603589 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4046136635 0 0) 0x7f49341439c0 con 0x7f493405f6f0 2020-12-05T17:41:11.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.046+0000 7f4932ffd700 1 -- 172.21.15.32:0/1696603589 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4920003230 con 0x7f493405f6f0 2020-12-05T17:41:11.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.046+0000 7f4932ffd700 1 -- 172.21.15.32:0/1696603589 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2796515287 0 0) 0x7f493405fe20 con 0x7f4934142300 2020-12-05T17:41:11.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.046+0000 7f4932ffd700 1 -- 172.21.15.32:0/1696603589 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49341439c0 con 0x7f4934142300 2020-12-05T17:41:11.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.046+0000 7f49337fe700 1 -- 172.21.15.32:0/1696603589 >> v1:172.21.15.61:6789/0 conn(0x7f4934142300 legacy=0x7f4934142790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.046+0000 7f49337fe700 1 -- 172.21.15.32:0/1696603589 >> v1:172.21.15.32:6789/0 conn(0x7f493405f6f0 legacy=0x7f4934141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.046+0000 7f49337fe700 1 -- 172.21.15.32:0/1696603589 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493405fad0 con 0x7f4934061b30 2020-12-05T17:41:11.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.046+0000 7f49337fe700 1 --2- 172.21.15.32:0/1696603589 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4934061b30 0x7f4934141e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f4928002f30 tx=0x7f4928006dc0).ready entity=mon.2 client_cookie=aac96752ee1a135e server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:11.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.050+0000 7f4932ffd700 1 -- 172.21.15.32:0/1696603589 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4928009730 con 0x7f4934061b30 2020-12-05T17:41:11.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.050+0000 7f4932ffd700 1 -- 172.21.15.32:0/1696603589 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4928009890 con 0x7f4934061b30 2020-12-05T17:41:11.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.050+0000 7f4932ffd700 1 -- 172.21.15.32:0/1696603589 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4928009bf0 con 0x7f4934061b30 2020-12-05T17:41:11.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.050+0000 7f493958b700 1 -- 172.21.15.32:0/1696603589 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4934061b30 msgr2=0x7f4934141e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.050+0000 7f493958b700 1 --2- 172.21.15.32:0/1696603589 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4934061b30 0x7f4934141e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f4928002f30 tx=0x7f4928006dc0).stop 2020-12-05T17:41:11.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.050+0000 7f493958b700 1 -- 172.21.15.32:0/1696603589 shutdown_connections 2020-12-05T17:41:11.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.050+0000 7f493958b700 1 --2- 172.21.15.32:0/1696603589 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4934061b30 0x7f4934141e10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.050+0000 7f493958b700 1 -- 172.21.15.32:0/1696603589 >> 172.21.15.32:0/1696603589 conn(0x7f4934061010 msgr2=0x7f4934061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:11.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.050+0000 7f493958b700 1 -- 172.21.15.32:0/1696603589 shutdown_connections 2020-12-05T17:41:11.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.050+0000 7f493958b700 1 -- 172.21.15.32:0/1696603589 wait complete. 2020-12-05T17:41:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.050+0000 7f493958b700 1 Processor -- start 2020-12-05T17:41:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.050+0000 7f493958b700 1 -- start start 2020-12-05T17:41:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.050+0000 7f493958b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f493405f6f0 0x7f4934152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.050+0000 7f493958b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4934061b30 0x7f493413d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:11.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.050+0000 7f493958b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4934142300 0x7f493413d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:11.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.054+0000 7f493958b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4934148a00 con 0x7f4934142300 2020-12-05T17:41:11.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.054+0000 7f493958b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4934149950 con 0x7f493405f6f0 2020-12-05T17:41:11.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.054+0000 7f493958b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49341517d0 con 0x7f4934061b30 2020-12-05T17:41:11.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.054+0000 7f4938d8a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4934142300 0x7f493413d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:11.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.054+0000 7f4938d8a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4934142300 0x7f493413d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40422/0 (socket says 172.21.15.32:40422) 2020-12-05T17:41:11.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.054+0000 7f4938d8a700 1 -- 172.21.15.32:0/2082979250 learned_addr learned my addr 172.21.15.32:0/2082979250 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:11.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.054+0000 7f4933fff700 1 --2- 172.21.15.32:0/2082979250 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f493405f6f0 0x7f4934152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:11.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.054+0000 7f49337fe700 1 --2- 172.21.15.32:0/2082979250 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4934061b30 0x7f493413d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:11.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.054+0000 7f4938d8a700 1 -- 172.21.15.32:0/2082979250 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4934061b30 msgr2=0x7f493413d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.054+0000 7f4938d8a700 1 --2- 172.21.15.32:0/2082979250 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4934061b30 0x7f493413d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.054+0000 7f4938d8a700 1 -- 172.21.15.32:0/2082979250 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f493405f6f0 msgr2=0x7f4934152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.054+0000 7f4938d8a700 1 --2- 172.21.15.32:0/2082979250 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f493405f6f0 0x7f4934152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.054+0000 7f4938d8a700 1 -- 172.21.15.32:0/2082979250 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49340b3a60 con 0x7f4934142300 2020-12-05T17:41:11.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.054+0000 7f49337fe700 1 --2- 172.21.15.32:0/2082979250 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4934061b30 0x7f493413d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:11.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.054+0000 7f4938d8a700 1 --2- 172.21.15.32:0/2082979250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4934142300 0x7f493413d9c0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7f492c0010d0 tx=0x7f492c007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:11.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.054+0000 7f49317fa700 1 -- 172.21.15.32:0/2082979250 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f492c00f470 con 0x7f4934142300 2020-12-05T17:41:11.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.054+0000 7f49317fa700 1 -- 172.21.15.32:0/2082979250 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c00f5d0 con 0x7f4934142300 2020-12-05T17:41:11.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.054+0000 7f493958b700 1 -- 172.21.15.32:0/2082979250 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4934063040 con 0x7f4934142300 2020-12-05T17:41:11.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.054+0000 7f49317fa700 1 -- 172.21.15.32:0/2082979250 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f492c00f930 con 0x7f4934142300 2020-12-05T17:41:11.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.054+0000 7f493958b700 1 -- 172.21.15.32:0/2082979250 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4934063510 con 0x7f4934142300 2020-12-05T17:41:11.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.054+0000 7f493958b700 1 -- 172.21.15.32:0/2082979250 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4934151b90 con 0x7f4934142300 2020-12-05T17:41:11.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.054+0000 7f49317fa700 1 -- 172.21.15.32:0/2082979250 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f492c019030 con 0x7f4934142300 2020-12-05T17:41:11.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.058+0000 7f49317fa700 1 --2- 172.21.15.32:0/2082979250 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4914055330 0x7f49140577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:11.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.058+0000 7f49317fa700 1 -- 172.21.15.32:0/2082979250 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f492c06d450 con 0x7f4934142300 2020-12-05T17:41:11.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.058+0000 7f4933fff700 1 --2- 172.21.15.32:0/2082979250 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4914055330 0x7f49140577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:11.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.058+0000 7f4933fff700 1 --2- 172.21.15.32:0/2082979250 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4914055330 0x7f49140577f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f492400d000 tx=0x7f4924009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:11.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.062+0000 7f49317fa700 1 -- 172.21.15.32:0/2082979250 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f492c0435f0 con 0x7f4934142300 2020-12-05T17:41:11.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.206+0000 7f493958b700 1 -- 172.21.15.32:0/2082979250 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f49340b3bf0 con 0x7f4934142300 2020-12-05T17:41:11.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.210+0000 7f49317fa700 1 -- 172.21.15.32:0/2082979250 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v129) v1 ==== 110+0+10 (secure 0 0 0) 0x7f492c046c10 con 0x7f4934142300 2020-12-05T17:41:11.214 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:41:11.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.210+0000 7f493958b700 1 -- 172.21.15.32:0/2082979250 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4914055330 msgr2=0x7f49140577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.210+0000 7f493958b700 1 --2- 172.21.15.32:0/2082979250 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4914055330 0x7f49140577f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f492400d000 tx=0x7f4924009040).stop 2020-12-05T17:41:11.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.210+0000 7f493958b700 1 -- 172.21.15.32:0/2082979250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4934142300 msgr2=0x7f493413d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.210+0000 7f493958b700 1 --2- 172.21.15.32:0/2082979250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4934142300 0x7f493413d9c0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7f492c0010d0 tx=0x7f492c007a80).stop 2020-12-05T17:41:11.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.210+0000 7f493958b700 1 -- 172.21.15.32:0/2082979250 shutdown_connections 2020-12-05T17:41:11.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.210+0000 7f493958b700 1 --2- 172.21.15.32:0/2082979250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4934142300 0x7f493413d9c0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.210+0000 7f493958b700 1 --2- 172.21.15.32:0/2082979250 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f493405f6f0 0x7f4934152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.210+0000 7f493958b700 1 --2- 172.21.15.32:0/2082979250 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4914055330 0x7f49140577f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.210+0000 7f493958b700 1 --2- 172.21.15.32:0/2082979250 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4934061b30 0x7f493413d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.210+0000 7f493958b700 1 -- 172.21.15.32:0/2082979250 >> 172.21.15.32:0/2082979250 conn(0x7f4934061010 msgr2=0x7f4934142710 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:11.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.210+0000 7f493958b700 1 -- 172.21.15.32:0/2082979250 shutdown_connections 2020-12-05T17:41:11.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.210+0000 7f493958b700 1 -- 172.21.15.32:0/2082979250 wait complete. 2020-12-05T17:41:11.228 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:41:11.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.346+0000 7f3139a33700 1 Processor -- start 2020-12-05T17:41:11.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.346+0000 7f3139a33700 1 -- start start 2020-12-05T17:41:11.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.346+0000 7f3139a33700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f313405f550 0x7f31340acb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:11.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.346+0000 7f3139a33700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f31340af090 0x7f31340af500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:11.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.346+0000 7f3139a33700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3134067110 con 0x7f3134063040 2020-12-05T17:41:11.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.346+0000 7f3139a33700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31341405b0 con 0x7f313405f550 2020-12-05T17:41:11.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.346+0000 7f3139a33700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f313413f760 con 0x7f31340af090 2020-12-05T17:41:11.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.346+0000 7f3138a31700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f313405f550 0x7f31340acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:11.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.346+0000 7f3138a31700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f313405f550 0x7f31340acb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40426/0 (socket says 172.21.15.32:40426) 2020-12-05T17:41:11.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.346+0000 7f3138a31700 1 -- 172.21.15.32:0/77312420 learned_addr learned my addr 172.21.15.32:0/77312420 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:11.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.346+0000 7f3139232700 1 --2- 172.21.15.32:0/77312420 >> v2:172.21.15.61:3300/0 conn(0x7f31340af090 0x7f31340af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:11.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.346+0000 7f3138a31700 1 -- 172.21.15.32:0/77312420 >> v2:172.21.15.61:3300/0 conn(0x7f31340af090 msgr2=0x7f31340af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:11.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.346+0000 7f3138a31700 1 --2- 172.21.15.32:0/77312420 >> v2:172.21.15.61:3300/0 conn(0x7f31340af090 0x7f31340af500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.346+0000 7f3138a31700 1 -- 172.21.15.32:0/77312420 >> v1:172.21.15.61:6789/0 conn(0x7f3134063040 legacy=0x7f313413fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.346+0000 7f3138a31700 1 -- 172.21.15.32:0/77312420 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313413fa00 con 0x7f313405f550 2020-12-05T17:41:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.350+0000 7f3138a31700 1 --2- 172.21.15.32:0/77312420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f313405f550 0x7f31340acb80 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0x7f3124002f30 tx=0x7f3124006e80).ready entity=mon.0 client_cookie=e5e27aa1d83bd51f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.350+0000 7f31337fe700 1 -- 172.21.15.32:0/77312420 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31240097f0 con 0x7f313405f550 2020-12-05T17:41:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.350+0000 7f31337fe700 1 -- 172.21.15.32:0/77312420 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3124009950 con 0x7f313405f550 2020-12-05T17:41:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.350+0000 7f31337fe700 1 -- 172.21.15.32:0/77312420 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3124009cb0 con 0x7f313405f550 2020-12-05T17:41:11.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.350+0000 7f3139a33700 1 -- 172.21.15.32:0/77312420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f313405f550 msgr2=0x7f31340acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.350+0000 7f3139a33700 1 --2- 172.21.15.32:0/77312420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f313405f550 0x7f31340acb80 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0x7f3124002f30 tx=0x7f3124006e80).stop 2020-12-05T17:41:11.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.350+0000 7f3139a33700 1 -- 172.21.15.32:0/77312420 shutdown_connections 2020-12-05T17:41:11.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.350+0000 7f3139a33700 1 --2- 172.21.15.32:0/77312420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f313405f550 0x7f31340acb80 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.350+0000 7f3139a33700 1 --2- 172.21.15.32:0/77312420 >> v2:172.21.15.61:3300/0 conn(0x7f31340af090 0x7f31340af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.350+0000 7f3139a33700 1 -- 172.21.15.32:0/77312420 >> 172.21.15.32:0/77312420 conn(0x7f31340620b0 msgr2=0x7f3134062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:11.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.350+0000 7f3139a33700 1 -- 172.21.15.32:0/77312420 shutdown_connections 2020-12-05T17:41:11.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.350+0000 7f3139a33700 1 -- 172.21.15.32:0/77312420 wait complete. 2020-12-05T17:41:11.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3139a33700 1 Processor -- start 2020-12-05T17:41:11.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3139a33700 1 -- start start 2020-12-05T17:41:11.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3139a33700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f313405f550 0x7f31341517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:11.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3139a33700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3134063040 0x7f31340b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:11.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3139a33700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f31340af090 0x7f3134138d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:11.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3139a33700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313410ea40 con 0x7f313405f550 2020-12-05T17:41:11.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3139a33700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31340b24c0 con 0x7f31340af090 2020-12-05T17:41:11.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3139a33700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31341427e0 con 0x7f3134063040 2020-12-05T17:41:11.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3138a31700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f313405f550 0x7f31341517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:11.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3138a31700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f313405f550 0x7f31341517f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40432/0 (socket says 172.21.15.32:40432) 2020-12-05T17:41:11.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3138a31700 1 -- 172.21.15.32:0/2629477590 learned_addr learned my addr 172.21.15.32:0/2629477590 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:11.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3133fff700 1 --2- 172.21.15.32:0/2629477590 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3134063040 0x7f31340b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:11.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3139232700 1 --2- 172.21.15.32:0/2629477590 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f31340af090 0x7f3134138d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:11.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3138a31700 1 -- 172.21.15.32:0/2629477590 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3134063040 msgr2=0x7f31340b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3138a31700 1 --2- 172.21.15.32:0/2629477590 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3134063040 0x7f31340b1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3138a31700 1 -- 172.21.15.32:0/2629477590 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f31340af090 msgr2=0x7f3134138d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3138a31700 1 --2- 172.21.15.32:0/2629477590 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f31340af090 0x7f3134138d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3138a31700 1 -- 172.21.15.32:0/2629477590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31341423e0 con 0x7f313405f550 2020-12-05T17:41:11.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3133fff700 1 --2- 172.21.15.32:0/2629477590 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3134063040 0x7f31340b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:11.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3138a31700 1 --2- 172.21.15.32:0/2629477590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f313405f550 0x7f31341517f0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0x7f3124002f30 tx=0x7f3124015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:11.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3131ffb700 1 -- 172.21.15.32:0/2629477590 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3124011ce0 con 0x7f313405f550 2020-12-05T17:41:11.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3131ffb700 1 -- 172.21.15.32:0/2629477590 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3124011e40 con 0x7f313405f550 2020-12-05T17:41:11.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3139a33700 1 -- 172.21.15.32:0/2629477590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3134141f30 con 0x7f313405f550 2020-12-05T17:41:11.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3131ffb700 1 -- 172.21.15.32:0/2629477590 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f312401e650 con 0x7f313405f550 2020-12-05T17:41:11.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.354+0000 7f3139a33700 1 -- 172.21.15.32:0/2629477590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3134139280 con 0x7f313405f550 2020-12-05T17:41:11.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.358+0000 7f3139a33700 1 -- 172.21.15.32:0/2629477590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3134142570 con 0x7f313405f550 2020-12-05T17:41:11.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.358+0000 7f3131ffb700 1 -- 172.21.15.32:0/2629477590 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f312401a040 con 0x7f313405f550 2020-12-05T17:41:11.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.358+0000 7f3131ffb700 1 --2- 172.21.15.32:0/2629477590 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3114055330 0x7f31140577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:11.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.358+0000 7f3131ffb700 1 -- 172.21.15.32:0/2629477590 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f312406f1b0 con 0x7f313405f550 2020-12-05T17:41:11.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.358+0000 7f3133fff700 1 --2- 172.21.15.32:0/2629477590 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3114055330 0x7f31140577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:11.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.358+0000 7f3133fff700 1 --2- 172.21.15.32:0/2629477590 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3114055330 0x7f31140577f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f312800b000 tx=0x7f3128009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:11.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.362+0000 7f3131ffb700 1 -- 172.21.15.32:0/2629477590 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3124045390 con 0x7f313405f550 2020-12-05T17:41:11.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.506+0000 7f3139a33700 1 -- 172.21.15.32:0/2629477590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f313410e870 con 0x7f313405f550 2020-12-05T17:41:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.510+0000 7f3131ffb700 1 -- 172.21.15.32:0/2629477590 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v129) v1 ==== 92+0+10 (secure 0 0 0) 0x7f31240230a0 con 0x7f313405f550 2020-12-05T17:41:11.514 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:41:11.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.510+0000 7f3139a33700 1 -- 172.21.15.32:0/2629477590 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3114055330 msgr2=0x7f31140577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.510+0000 7f3139a33700 1 --2- 172.21.15.32:0/2629477590 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3114055330 0x7f31140577f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f312800b000 tx=0x7f3128009040).stop 2020-12-05T17:41:11.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.510+0000 7f3139a33700 1 -- 172.21.15.32:0/2629477590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f313405f550 msgr2=0x7f31341517f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.510+0000 7f3139a33700 1 --2- 172.21.15.32:0/2629477590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f313405f550 0x7f31341517f0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0x7f3124002f30 tx=0x7f3124015000).stop 2020-12-05T17:41:11.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.510+0000 7f3139a33700 1 -- 172.21.15.32:0/2629477590 shutdown_connections 2020-12-05T17:41:11.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.510+0000 7f3139a33700 1 --2- 172.21.15.32:0/2629477590 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f31340af090 0x7f3134138d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.510+0000 7f3139a33700 1 --2- 172.21.15.32:0/2629477590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f313405f550 0x7f31341517f0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.510+0000 7f3139a33700 1 --2- 172.21.15.32:0/2629477590 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3114055330 0x7f31140577f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.510+0000 7f3139a33700 1 --2- 172.21.15.32:0/2629477590 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3134063040 0x7f31340b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.510+0000 7f3139a33700 1 -- 172.21.15.32:0/2629477590 >> 172.21.15.32:0/2629477590 conn(0x7f31340620b0 msgr2=0x7f3134067460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.510+0000 7f3139a33700 1 -- 172.21.15.32:0/2629477590 shutdown_connections 2020-12-05T17:41:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.510+0000 7f3139a33700 1 -- 172.21.15.32:0/2629477590 wait complete. 2020-12-05T17:41:11.533 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:41:11.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.646+0000 7fe2dbfff700 1 Processor -- start 2020-12-05T17:41:11.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.646+0000 7fe2dbfff700 1 -- start start 2020-12-05T17:41:11.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.646+0000 7fe2dbfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2dc0b2000 0x7fe2dc145e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:11.655 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.646+0000 7fe2dbfff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2dc0b23e0 con 0x7fe2dc061780 2020-12-05T17:41:11.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.646+0000 7fe2dbfff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2dc0ae7b0 con 0x7fe2dc0af4b0 2020-12-05T17:41:11.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.646+0000 7fe2dbfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc146a00 con 0x7fe2dc0b2000 2020-12-05T17:41:11.656 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.646+0000 7fe2db7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe2dc061780 0x7fe2dc0afe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43092/0 (socket says 172.21.15.32:43092) 2020-12-05T17:41:11.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.646+0000 7fe2db7fe700 1 -- 172.21.15.32:0/229317283 learned_addr learned my addr 172.21.15.32:0/229317283 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:11.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.646+0000 7fe2daffd700 1 --2- 172.21.15.32:0/229317283 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2dc0b2000 0x7fe2dc145e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:11.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2da7fc700 1 -- 172.21.15.32:0/229317283 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 547024678 0 0) 0x7fe2dc0b23e0 con 0x7fe2dc061780 2020-12-05T17:41:11.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2da7fc700 1 -- 172.21.15.32:0/229317283 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2bc003230 con 0x7fe2dc061780 2020-12-05T17:41:11.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2da7fc700 1 -- 172.21.15.32:0/229317283 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 344560272 0 0) 0x7fe2dc0ae7b0 con 0x7fe2dc0af4b0 2020-12-05T17:41:11.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2da7fc700 1 -- 172.21.15.32:0/229317283 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2dc0b23e0 con 0x7fe2dc0af4b0 2020-12-05T17:41:11.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2daffd700 1 -- 172.21.15.32:0/229317283 >> v1:172.21.15.61:6789/0 conn(0x7fe2dc0af4b0 legacy=0x7fe2dc1462f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2daffd700 1 -- 172.21.15.32:0/229317283 >> v1:172.21.15.32:6789/0 conn(0x7fe2dc061780 legacy=0x7fe2dc0afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2daffd700 1 -- 172.21.15.32:0/229317283 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc0aea50 con 0x7fe2dc0b2000 2020-12-05T17:41:11.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2daffd700 1 --2- 172.21.15.32:0/229317283 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2dc0b2000 0x7fe2dc145e00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fe2cc002f30 tx=0x7fe2cc006dc0).ready entity=mon.2 client_cookie=a412f37ca015ccfa server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:11.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2da7fc700 1 -- 172.21.15.32:0/229317283 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2cc009730 con 0x7fe2dc0b2000 2020-12-05T17:41:11.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2da7fc700 1 -- 172.21.15.32:0/229317283 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2cc009890 con 0x7fe2dc0b2000 2020-12-05T17:41:11.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2da7fc700 1 -- 172.21.15.32:0/229317283 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2cc009bf0 con 0x7fe2dc0b2000 2020-12-05T17:41:11.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2dbfff700 1 -- 172.21.15.32:0/229317283 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2dc0b2000 msgr2=0x7fe2dc145e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2dbfff700 1 --2- 172.21.15.32:0/229317283 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2dc0b2000 0x7fe2dc145e00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fe2cc002f30 tx=0x7fe2cc006dc0).stop 2020-12-05T17:41:11.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2dbfff700 1 -- 172.21.15.32:0/229317283 shutdown_connections 2020-12-05T17:41:11.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2dbfff700 1 --2- 172.21.15.32:0/229317283 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2dc0b2000 0x7fe2dc145e00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2dbfff700 1 -- 172.21.15.32:0/229317283 >> 172.21.15.32:0/229317283 conn(0x7fe2dc0b5620 msgr2=0x7fe2dc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:11.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2dbfff700 1 -- 172.21.15.32:0/229317283 shutdown_connections 2020-12-05T17:41:11.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2dbfff700 1 -- 172.21.15.32:0/229317283 wait complete. 2020-12-05T17:41:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2dbfff700 1 Processor -- start 2020-12-05T17:41:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2dbfff700 1 -- start start 2020-12-05T17:41:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2dbfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2dc061780 0x7fe2dc065110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2dbfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2dc0af4b0 0x7fe2dc1485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2dbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2dc0b2000 0x7fe2dc10de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2dbfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc0630e0 con 0x7fe2dc0b2000 2020-12-05T17:41:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2dbfff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc064170 con 0x7fe2dc0af4b0 2020-12-05T17:41:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.650+0000 7fe2dbfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc0636e0 con 0x7fe2dc061780 2020-12-05T17:41:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.654+0000 7fe2db7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2dc0b2000 0x7fe2dc10de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.654+0000 7fe2db7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2dc0b2000 0x7fe2dc10de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40450/0 (socket says 172.21.15.32:40450) 2020-12-05T17:41:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.654+0000 7fe2db7fe700 1 -- 172.21.15.32:0/2136518548 learned_addr learned my addr 172.21.15.32:0/2136518548 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.654+0000 7fe2daffd700 1 --2- 172.21.15.32:0/2136518548 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2dc061780 0x7fe2dc065110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:11.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.654+0000 7fe2d27fc700 1 --2- 172.21.15.32:0/2136518548 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2dc0af4b0 0x7fe2dc1485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:11.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.654+0000 7fe2db7fe700 1 -- 172.21.15.32:0/2136518548 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2dc061780 msgr2=0x7fe2dc065110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.654+0000 7fe2db7fe700 1 --2- 172.21.15.32:0/2136518548 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2dc061780 0x7fe2dc065110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.654+0000 7fe2db7fe700 1 -- 172.21.15.32:0/2136518548 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2dc0af4b0 msgr2=0x7fe2dc1485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.654+0000 7fe2db7fe700 1 --2- 172.21.15.32:0/2136518548 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2dc0af4b0 0x7fe2dc1485c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.654+0000 7fe2db7fe700 1 -- 172.21.15.32:0/2136518548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc064410 con 0x7fe2dc0b2000 2020-12-05T17:41:11.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.654+0000 7fe2d27fc700 1 --2- 172.21.15.32:0/2136518548 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2dc0af4b0 0x7fe2dc1485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:11.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.654+0000 7fe2db7fe700 1 --2- 172.21.15.32:0/2136518548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2dc0b2000 0x7fe2dc10de50 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 rx=0x7fe2d4001450 tx=0x7fe2d4007db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:11.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.654+0000 7fe2d8ff9700 1 -- 172.21.15.32:0/2136518548 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2d4005b70 con 0x7fe2dc0b2000 2020-12-05T17:41:11.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.654+0000 7fe2d8ff9700 1 -- 172.21.15.32:0/2136518548 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d4005cd0 con 0x7fe2dc0b2000 2020-12-05T17:41:11.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.654+0000 7fe2dbfff700 1 -- 172.21.15.32:0/2136518548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2dc149260 con 0x7fe2dc0b2000 2020-12-05T17:41:11.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.654+0000 7fe2d8ff9700 1 -- 172.21.15.32:0/2136518548 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2d4007300 con 0x7fe2dc0b2000 2020-12-05T17:41:11.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.654+0000 7fe2dbfff700 1 -- 172.21.15.32:0/2136518548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2dc066830 con 0x7fe2dc0b2000 2020-12-05T17:41:11.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.654+0000 7fe2dbfff700 1 -- 172.21.15.32:0/2136518548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2dc066e80 con 0x7fe2dc0b2000 2020-12-05T17:41:11.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.654+0000 7fe2d8ff9700 1 -- 172.21.15.32:0/2136518548 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe2d4010030 con 0x7fe2dc0b2000 2020-12-05T17:41:11.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.658+0000 7fe2d8ff9700 1 --2- 172.21.15.32:0/2136518548 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe2c00554e0 0x7fe2c00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:11.675 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.658+0000 7fe2daffd700 1 --2- 172.21.15.32:0/2136518548 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe2c00554e0 0x7fe2c00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:11.675 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.658+0000 7fe2d8ff9700 1 -- 172.21.15.32:0/2136518548 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe2d406e070 con 0x7fe2dc0b2000 2020-12-05T17:41:11.675 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.658+0000 7fe2daffd700 1 --2- 172.21.15.32:0/2136518548 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe2c00554e0 0x7fe2c00579a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7fe2cc019000 tx=0x7fe2cc015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:11.676 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.666+0000 7fe2d8ff9700 1 -- 172.21.15.32:0/2136518548 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe2d40442c0 con 0x7fe2dc0b2000 2020-12-05T17:41:11.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.806+0000 7fe2dbfff700 1 -- 172.21.15.32:0/2136518548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe2dc057050 con 0x7fe2dc0b2000 2020-12-05T17:41:11.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.806+0000 7fe2d8ff9700 1 -- 172.21.15.32:0/2136518548 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v129) v1 ==== 104+0+10 (secure 0 0 0) 0x7fe2d40478e0 con 0x7fe2dc0b2000 2020-12-05T17:41:11.812 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:41:11.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.810+0000 7fe2dbfff700 1 -- 172.21.15.32:0/2136518548 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe2c00554e0 msgr2=0x7fe2c00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.810+0000 7fe2dbfff700 1 --2- 172.21.15.32:0/2136518548 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe2c00554e0 0x7fe2c00579a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7fe2cc019000 tx=0x7fe2cc015000).stop 2020-12-05T17:41:11.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.810+0000 7fe2dbfff700 1 -- 172.21.15.32:0/2136518548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2dc0b2000 msgr2=0x7fe2dc10de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.810+0000 7fe2dbfff700 1 --2- 172.21.15.32:0/2136518548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2dc0b2000 0x7fe2dc10de50 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 rx=0x7fe2d4001450 tx=0x7fe2d4007db0).stop 2020-12-05T17:41:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.810+0000 7fe2dbfff700 1 -- 172.21.15.32:0/2136518548 shutdown_connections 2020-12-05T17:41:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.810+0000 7fe2dbfff700 1 --2- 172.21.15.32:0/2136518548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2dc0b2000 0x7fe2dc10de50 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.810+0000 7fe2dbfff700 1 --2- 172.21.15.32:0/2136518548 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe2c00554e0 0x7fe2c00579a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.810+0000 7fe2dbfff700 1 --2- 172.21.15.32:0/2136518548 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe2dc061780 0x7fe2dc065110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.810+0000 7fe2dbfff700 1 --2- 172.21.15.32:0/2136518548 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2dc0af4b0 0x7fe2dc1485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.810+0000 7fe2dbfff700 1 -- 172.21.15.32:0/2136518548 >> 172.21.15.32:0/2136518548 conn(0x7fe2dc0b5620 msgr2=0x7fe2dc1470b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.810+0000 7fe2dbfff700 1 -- 172.21.15.32:0/2136518548 shutdown_connections 2020-12-05T17:41:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.810+0000 7fe2dbfff700 1 -- 172.21.15.32:0/2136518548 wait complete. 2020-12-05T17:41:11.832 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:41:11.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.950+0000 7fbb93a6a700 1 Processor -- start 2020-12-05T17:41:11.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.950+0000 7fbb93a6a700 1 -- start start 2020-12-05T17:41:11.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.950+0000 7fbb93a6a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbb8c0b5b20 0x7fbb8c0b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:11.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.950+0000 7fbb93a6a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fbb8c0b6690 0x7fbb8c0b5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:11.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.950+0000 7fbb93a6a700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb8c0b1920 con 0x7fbb8c0b2060 2020-12-05T17:41:11.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.950+0000 7fbb93a6a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb8c0b5730 con 0x7fbb8c0b5b20 2020-12-05T17:41:11.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.950+0000 7fbb93a6a700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb8c0b1bc0 con 0x7fbb8c0b6690 2020-12-05T17:41:11.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.950+0000 7fbb91806700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbb8c0b5b20 0x7fbb8c0b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:11.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.950+0000 7fbb91806700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbb8c0b5b20 0x7fbb8c0b5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40454/0 (socket says 172.21.15.32:40454) 2020-12-05T17:41:11.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.950+0000 7fbb91806700 1 -- 172.21.15.32:0/1100655660 learned_addr learned my addr 172.21.15.32:0/1100655660 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:11.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.950+0000 7fbb91005700 1 --2- 172.21.15.32:0/1100655660 >> v2:172.21.15.61:3300/0 conn(0x7fbb8c0b6690 0x7fbb8c0b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:11.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.950+0000 7fbb91806700 1 -- 172.21.15.32:0/1100655660 >> v2:172.21.15.61:3300/0 conn(0x7fbb8c0b6690 msgr2=0x7fbb8c0b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:11.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.950+0000 7fbb91806700 1 --2- 172.21.15.32:0/1100655660 >> v2:172.21.15.61:3300/0 conn(0x7fbb8c0b6690 0x7fbb8c0b5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.950+0000 7fbb91806700 1 -- 172.21.15.32:0/1100655660 >> v1:172.21.15.61:6789/0 conn(0x7fbb8c0b2060 legacy=0x7fbb8c145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.950+0000 7fbb91806700 1 -- 172.21.15.32:0/1100655660 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c0b24d0 con 0x7fbb8c0b5b20 2020-12-05T17:41:11.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.950+0000 7fbb91806700 1 --2- 172.21.15.32:0/1100655660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb8c0b5b20 0x7fbb8c0b5f00 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7fbb88002f30 tx=0x7fbb88006e80).ready entity=mon.0 client_cookie=661b20df65ab4e70 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:11.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.950+0000 7fbb90804700 1 -- 172.21.15.32:0/1100655660 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb880097f0 con 0x7fbb8c0b5b20 2020-12-05T17:41:11.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.950+0000 7fbb90804700 1 -- 172.21.15.32:0/1100655660 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb88009950 con 0x7fbb8c0b5b20 2020-12-05T17:41:11.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.950+0000 7fbb90804700 1 -- 172.21.15.32:0/1100655660 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb88009cb0 con 0x7fbb8c0b5b20 2020-12-05T17:41:11.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.950+0000 7fbb93a6a700 1 -- 172.21.15.32:0/1100655660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb8c0b5b20 msgr2=0x7fbb8c0b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.950+0000 7fbb93a6a700 1 --2- 172.21.15.32:0/1100655660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb8c0b5b20 0x7fbb8c0b5f00 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7fbb88002f30 tx=0x7fbb88006e80).stop 2020-12-05T17:41:11.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.954+0000 7fbb93a6a700 1 -- 172.21.15.32:0/1100655660 shutdown_connections 2020-12-05T17:41:11.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.954+0000 7fbb93a6a700 1 --2- 172.21.15.32:0/1100655660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb8c0b5b20 0x7fbb8c0b5f00 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.954+0000 7fbb93a6a700 1 --2- 172.21.15.32:0/1100655660 >> v2:172.21.15.61:3300/0 conn(0x7fbb8c0b6690 0x7fbb8c0b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.954+0000 7fbb93a6a700 1 -- 172.21.15.32:0/1100655660 >> 172.21.15.32:0/1100655660 conn(0x7fbb8c05f3b0 msgr2=0x7fbb8c05f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:11.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.954+0000 7fbb93a6a700 1 -- 172.21.15.32:0/1100655660 shutdown_connections 2020-12-05T17:41:11.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.954+0000 7fbb93a6a700 1 -- 172.21.15.32:0/1100655660 wait complete. 2020-12-05T17:41:11.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.954+0000 7fbb93a6a700 1 Processor -- start 2020-12-05T17:41:11.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.954+0000 7fbb93a6a700 1 -- start start 2020-12-05T17:41:11.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb93a6a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb8c0b2060 0x7fbb8c141a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:11.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb93a6a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb8c0b5b20 0x7fbb8c147670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:11.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb93a6a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb8c0b6690 0x7fbb8c145480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:11.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb93a6a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c144120 con 0x7fbb8c0b2060 2020-12-05T17:41:11.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb93a6a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c0a25d0 con 0x7fbb8c0b5b20 2020-12-05T17:41:11.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb93a6a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c0a2870 con 0x7fbb8c0b6690 2020-12-05T17:41:11.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb91806700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb8c0b2060 0x7fbb8c141a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:11.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb91806700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb8c0b2060 0x7fbb8c141a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40460/0 (socket says 172.21.15.32:40460) 2020-12-05T17:41:11.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb91806700 1 -- 172.21.15.32:0/1669062428 learned_addr learned my addr 172.21.15.32:0/1669062428 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:11.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb91005700 1 --2- 172.21.15.32:0/1669062428 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb8c0b5b20 0x7fbb8c147670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:11.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb92007700 1 --2- 172.21.15.32:0/1669062428 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb8c0b6690 0x7fbb8c145480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:11.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb91806700 1 -- 172.21.15.32:0/1669062428 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb8c0b6690 msgr2=0x7fbb8c145480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb91806700 1 --2- 172.21.15.32:0/1669062428 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb8c0b6690 0x7fbb8c145480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb91806700 1 -- 172.21.15.32:0/1669062428 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb8c0b5b20 msgr2=0x7fbb8c147670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:11.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb91806700 1 --2- 172.21.15.32:0/1669062428 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb8c0b5b20 0x7fbb8c147670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:11.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb91806700 1 -- 172.21.15.32:0/1669062428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c144520 con 0x7fbb8c0b2060 2020-12-05T17:41:11.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb92007700 1 --2- 172.21.15.32:0/1669062428 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb8c0b6690 0x7fbb8c145480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:11.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb91806700 1 --2- 172.21.15.32:0/1669062428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb8c0b2060 0x7fbb8c141a40 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7fbb88002f30 tx=0x7fbb88015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:11.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb7affd700 1 -- 172.21.15.32:0/1669062428 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb88011c60 con 0x7fbb8c0b2060 2020-12-05T17:41:11.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb7affd700 1 -- 172.21.15.32:0/1669062428 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb88011dc0 con 0x7fbb8c0b2060 2020-12-05T17:41:11.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb93a6a700 1 -- 172.21.15.32:0/1669062428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8c143e10 con 0x7fbb8c0b2060 2020-12-05T17:41:11.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb7affd700 1 -- 172.21.15.32:0/1669062428 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb8801e650 con 0x7fbb8c0b2060 2020-12-05T17:41:11.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb93a6a700 1 -- 172.21.15.32:0/1669062428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8c147b60 con 0x7fbb8c0b2060 2020-12-05T17:41:11.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.958+0000 7fbb7affd700 1 -- 172.21.15.32:0/1669062428 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbb88025030 con 0x7fbb8c0b2060 2020-12-05T17:41:11.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.962+0000 7fbb7affd700 1 --2- 172.21.15.32:0/1669062428 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbb6c055330 0x7fbb6c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:11.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.962+0000 7fbb93a6a700 1 -- 172.21.15.32:0/1669062428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb8c057050 con 0x7fbb8c0b2060 2020-12-05T17:41:11.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.962+0000 7fbb7affd700 1 -- 172.21.15.32:0/1669062428 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbb8806f420 con 0x7fbb8c0b2060 2020-12-05T17:41:11.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.962+0000 7fbb91005700 1 --2- 172.21.15.32:0/1669062428 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbb6c055330 0x7fbb6c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:11.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.962+0000 7fbb91005700 1 --2- 172.21.15.32:0/1669062428 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbb6c055330 0x7fbb6c0577f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7fbb7c00b000 tx=0x7fbb7c006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:11.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:11.966+0000 7fbb7affd700 1 -- 172.21.15.32:0/1669062428 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbb880455c0 con 0x7fbb8c0b2060 2020-12-05T17:41:12.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.110+0000 7fbb93a6a700 1 -- 172.21.15.32:0/1669062428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbb8c05bae0 con 0x7fbb8c0b2060 2020-12-05T17:41:12.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.110+0000 7fbb7affd700 1 -- 172.21.15.32:0/1669062428 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v129) v1 ==== 100+0+10 (secure 0 0 0) 0x7fbb88023020 con 0x7fbb8c0b2060 2020-12-05T17:41:12.116 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:41:12.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.110+0000 7fbb93a6a700 1 -- 172.21.15.32:0/1669062428 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbb6c055330 msgr2=0x7fbb6c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:12.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.110+0000 7fbb93a6a700 1 --2- 172.21.15.32:0/1669062428 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbb6c055330 0x7fbb6c0577f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7fbb7c00b000 tx=0x7fbb7c006dc0).stop 2020-12-05T17:41:12.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.110+0000 7fbb93a6a700 1 -- 172.21.15.32:0/1669062428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb8c0b2060 msgr2=0x7fbb8c141a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:12.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.110+0000 7fbb93a6a700 1 --2- 172.21.15.32:0/1669062428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb8c0b2060 0x7fbb8c141a40 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7fbb88002f30 tx=0x7fbb88015000).stop 2020-12-05T17:41:12.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.110+0000 7fbb93a6a700 1 -- 172.21.15.32:0/1669062428 shutdown_connections 2020-12-05T17:41:12.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.110+0000 7fbb93a6a700 1 --2- 172.21.15.32:0/1669062428 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbb6c055330 0x7fbb6c0577f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:12.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.110+0000 7fbb93a6a700 1 --2- 172.21.15.32:0/1669062428 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb8c0b6690 0x7fbb8c145480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:12.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.110+0000 7fbb93a6a700 1 --2- 172.21.15.32:0/1669062428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb8c0b2060 0x7fbb8c141a40 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:12.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.110+0000 7fbb93a6a700 1 --2- 172.21.15.32:0/1669062428 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb8c0b5b20 0x7fbb8c147670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:12.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.110+0000 7fbb93a6a700 1 -- 172.21.15.32:0/1669062428 >> 172.21.15.32:0/1669062428 conn(0x7fbb8c05f3b0 msgr2=0x7fbb8c142b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:12.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.110+0000 7fbb93a6a700 1 -- 172.21.15.32:0/1669062428 shutdown_connections 2020-12-05T17:41:12.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.110+0000 7fbb93a6a700 1 -- 172.21.15.32:0/1669062428 wait complete. 2020-12-05T17:41:12.129 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:41:12.130 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:41:12.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.238+0000 7f0a337ef700 1 Processor -- start 2020-12-05T17:41:12.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.238+0000 7f0a337ef700 1 -- start start 2020-12-05T17:41:12.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.238+0000 7f0a337ef700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0a2c05fe00 0x7f0a2c061910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:12.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.238+0000 7f0a337ef700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a2c0601e0 con 0x7f0a2c0acca0 2020-12-05T17:41:12.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.238+0000 7f0a337ef700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a2c061fa0 con 0x7f0a2c0b00a0 2020-12-05T17:41:12.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.238+0000 7f0a337ef700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a2c0ac1a0 con 0x7f0a2c05fe00 2020-12-05T17:41:12.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.238+0000 7f0a31d8c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0a2c0acca0 0x7f0a2c145d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43120/0 (socket says 172.21.15.32:43120) 2020-12-05T17:41:12.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.238+0000 7f0a31d8c700 1 -- 172.21.15.32:0/3821412867 learned_addr learned my addr 172.21.15.32:0/3821412867 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:12.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.238+0000 7f0a3158b700 1 --2- 172.21.15.32:0/3821412867 >> v2:172.21.15.61:3300/0 conn(0x7f0a2c05fe00 0x7f0a2c061910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:12.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.238+0000 7f0a2bfff700 1 -- 172.21.15.32:0/3821412867 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 33581097 0 0) 0x7f0a2c0601e0 con 0x7f0a2c0acca0 2020-12-05T17:41:12.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.238+0000 7f0a2bfff700 1 -- 172.21.15.32:0/3821412867 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a18003230 con 0x7f0a2c0acca0 2020-12-05T17:41:12.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.238+0000 7f0a3158b700 1 -- 172.21.15.32:0/3821412867 >> v1:172.21.15.61:6789/0 conn(0x7f0a2c0b00a0 legacy=0x7f0a2c0ac590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:12.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.238+0000 7f0a2bfff700 1 -- 172.21.15.32:0/3821412867 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3259353873 0 0) 0x7f0a2c061fa0 con 0x7f0a2c0b00a0 2020-12-05T17:41:12.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.238+0000 7f0a3158b700 1 -- 172.21.15.32:0/3821412867 >> v1:172.21.15.32:6789/0 conn(0x7f0a2c0acca0 legacy=0x7f0a2c145d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:12.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.238+0000 7f0a3158b700 1 -- 172.21.15.32:0/3821412867 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a2c062240 con 0x7f0a2c05fe00 2020-12-05T17:41:12.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.238+0000 7f0a3158b700 1 --2- 172.21.15.32:0/3821412867 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a2c05fe00 0x7f0a2c061910 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f0a1c0032b0 tx=0x7f0a1c0091a0).ready entity=mon.1 client_cookie=bbebcc6a7bc6643c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:12.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.238+0000 7f0a2bfff700 1 -- 172.21.15.32:0/3821412867 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a1c00b9a0 con 0x7f0a2c05fe00 2020-12-05T17:41:12.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.238+0000 7f0a2bfff700 1 -- 172.21.15.32:0/3821412867 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1c00bb00 con 0x7f0a2c05fe00 2020-12-05T17:41:12.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.238+0000 7f0a2bfff700 1 -- 172.21.15.32:0/3821412867 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a1c00bc70 con 0x7f0a2c05fe00 2020-12-05T17:41:12.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.238+0000 7f0a337ef700 1 -- 172.21.15.32:0/3821412867 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a2c05fe00 msgr2=0x7f0a2c061910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:12.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.238+0000 7f0a337ef700 1 --2- 172.21.15.32:0/3821412867 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a2c05fe00 0x7f0a2c061910 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f0a1c0032b0 tx=0x7f0a1c0091a0).stop 2020-12-05T17:41:12.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a337ef700 1 -- 172.21.15.32:0/3821412867 shutdown_connections 2020-12-05T17:41:12.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a337ef700 1 --2- 172.21.15.32:0/3821412867 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a2c05fe00 0x7f0a2c061910 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:12.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a337ef700 1 -- 172.21.15.32:0/3821412867 >> 172.21.15.32:0/3821412867 conn(0x7f0a2c0b57c0 msgr2=0x7f0a2c05f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:12.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a337ef700 1 -- 172.21.15.32:0/3821412867 shutdown_connections 2020-12-05T17:41:12.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a337ef700 1 -- 172.21.15.32:0/3821412867 wait complete. 2020-12-05T17:41:12.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a337ef700 1 Processor -- start 2020-12-05T17:41:12.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a337ef700 1 -- start start 2020-12-05T17:41:12.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a337ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2c05fe00 0x7f0a2c063f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:12.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a337ef700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0a2c0acca0 0x7f0a2c0a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:12.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a337ef700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a2c0b00a0 0x7f0a2c0a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:12.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a337ef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2c09e120 con 0x7f0a2c05fe00 2020-12-05T17:41:12.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a337ef700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2c156be0 con 0x7f0a2c0b00a0 2020-12-05T17:41:12.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a337ef700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2c09e310 con 0x7f0a2c0acca0 2020-12-05T17:41:12.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a3158b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2c05fe00 0x7f0a2c063f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:12.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a3158b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2c05fe00 0x7f0a2c063f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40474/0 (socket says 172.21.15.32:40474) 2020-12-05T17:41:12.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a3158b700 1 -- 172.21.15.32:0/1261344645 learned_addr learned my addr 172.21.15.32:0/1261344645 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:12.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a31d8c700 1 --2- 172.21.15.32:0/1261344645 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a2c0b00a0 0x7f0a2c0a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:12.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a30d8a700 1 --2- 172.21.15.32:0/1261344645 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0a2c0acca0 0x7f0a2c0a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:12.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a3158b700 1 -- 172.21.15.32:0/1261344645 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0a2c0acca0 msgr2=0x7f0a2c0a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:12.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a3158b700 1 --2- 172.21.15.32:0/1261344645 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0a2c0acca0 0x7f0a2c0a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:12.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a3158b700 1 -- 172.21.15.32:0/1261344645 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a2c0b00a0 msgr2=0x7f0a2c0a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:12.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a3158b700 1 --2- 172.21.15.32:0/1261344645 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a2c0b00a0 0x7f0a2c0a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:12.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a3158b700 1 -- 172.21.15.32:0/1261344645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a2c09fc90 con 0x7f0a2c05fe00 2020-12-05T17:41:12.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a3158b700 1 --2- 172.21.15.32:0/1261344645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2c05fe00 0x7f0a2c063f50 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7f0a1c0032b0 tx=0x7f0a1c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:12.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a2a7fc700 1 -- 172.21.15.32:0/1261344645 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a1c00bc70 con 0x7f0a2c05fe00 2020-12-05T17:41:12.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a2a7fc700 1 -- 172.21.15.32:0/1261344645 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1c01f5e0 con 0x7f0a2c05fe00 2020-12-05T17:41:12.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a337ef700 1 -- 172.21.15.32:0/1261344645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a2c158f70 con 0x7f0a2c05fe00 2020-12-05T17:41:12.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a2a7fc700 1 -- 172.21.15.32:0/1261344645 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a1c01f940 con 0x7f0a2c05fe00 2020-12-05T17:41:12.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.242+0000 7f0a337ef700 1 -- 172.21.15.32:0/1261344645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a2c09e630 con 0x7f0a2c05fe00 2020-12-05T17:41:12.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.246+0000 7f0a337ef700 1 -- 172.21.15.32:0/1261344645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a2c05ab80 con 0x7f0a2c05fe00 2020-12-05T17:41:12.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.246+0000 7f0a2a7fc700 1 -- 172.21.15.32:0/1261344645 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0a1c026030 con 0x7f0a2c05fe00 2020-12-05T17:41:12.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.246+0000 7f0a2a7fc700 1 --2- 172.21.15.32:0/1261344645 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0a0c055440 0x7f0a0c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:12.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.246+0000 7f0a30d8a700 1 --2- 172.21.15.32:0/1261344645 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0a0c055440 0x7f0a0c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:12.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.246+0000 7f0a2a7fc700 1 -- 172.21.15.32:0/1261344645 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0a1c06eda0 con 0x7f0a2c05fe00 2020-12-05T17:41:12.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.246+0000 7f0a30d8a700 1 --2- 172.21.15.32:0/1261344645 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0a0c055440 0x7f0a0c057900 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f0a2000b000 tx=0x7f0a20009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:12.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.250+0000 7f0a2a7fc700 1 -- 172.21.15.32:0/1261344645 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0a2c05ab80 con 0x7f0a2c05fe00 2020-12-05T17:41:12.443 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.438+0000 7f0a337ef700 1 -- 172.21.15.32:0/1261344645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a2c05ab80 con 0x7f0a2c05fe00 2020-12-05T17:41:12.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.438+0000 7f0a2a7fc700 1 -- 172.21.15.32:0/1261344645 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f0a2c05ab80 con 0x7f0a2c05fe00 2020-12-05T17:41:12.444 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:12.444 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":62,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6830,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/3985915039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3985915039},{"type":"v1","addr":"172.21.15.61:6833","nonce":3985915039}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":60}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:41:03.280787+0000","modified":"2020-12-05T17:41:04.283186+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7938":{"gid":7938,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1585653076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1585653076},{"type":"v1","addr":"172.21.15.32:6835","nonce":1585653076}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2020-12-05T17:41:12.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.442+0000 7f0a337ef700 1 -- 172.21.15.32:0/1261344645 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0a0c055440 msgr2=0x7f0a0c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:12.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.442+0000 7f0a337ef700 1 --2- 172.21.15.32:0/1261344645 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0a0c055440 0x7f0a0c057900 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f0a2000b000 tx=0x7f0a20009040).stop 2020-12-05T17:41:12.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.442+0000 7f0a337ef700 1 -- 172.21.15.32:0/1261344645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2c05fe00 msgr2=0x7f0a2c063f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:12.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.442+0000 7f0a337ef700 1 --2- 172.21.15.32:0/1261344645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2c05fe00 0x7f0a2c063f50 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7f0a1c0032b0 tx=0x7f0a1c016000).stop 2020-12-05T17:41:12.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.442+0000 7f0a337ef700 1 -- 172.21.15.32:0/1261344645 shutdown_connections 2020-12-05T17:41:12.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.442+0000 7f0a337ef700 1 --2- 172.21.15.32:0/1261344645 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a2c0b00a0 0x7f0a2c0a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.442+0000 7f0a337ef700 1 --2- 172.21.15.32:0/1261344645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2c05fe00 0x7f0a2c063f50 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.442+0000 7f0a337ef700 1 --2- 172.21.15.32:0/1261344645 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0a0c055440 0x7f0a0c057900 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.442+0000 7f0a337ef700 1 --2- 172.21.15.32:0/1261344645 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0a2c0acca0 0x7f0a2c0a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.442+0000 7f0a337ef700 1 -- 172.21.15.32:0/1261344645 >> 172.21.15.32:0/1261344645 conn(0x7f0a2c0b57c0 msgr2=0x7f0a2c0aad40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.442+0000 7f0a337ef700 1 -- 172.21.15.32:0/1261344645 shutdown_connections 2020-12-05T17:41:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:12.442+0000 7f0a337ef700 1 -- 172.21.15.32:0/1261344645 wait complete. 2020-12-05T17:41:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 62 2020-12-05T17:41:12.665 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:41:03.280787+0000', 'modified': '2020-12-05T17:41:04.283186+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7938}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7938': {'gid': 7938, 'name': 'a', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/1585653076', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 1585653076}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 1585653076}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:41:12.666 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:41:12.666 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:41:12.797 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":7938,"want_state":"up:active","state":"up:active","rank_uptime":9.5034136680000003,"mdsmap_epoch":62,"osdmap_epoch":129,"osdmap_epoch_barrier":128,"uptime":14.467615002000001} 2020-12-05T17:41:12.797 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 7938, "mdsmap_epoch": 62, "osdmap_epoch": 129, "osdmap_epoch_barrier": 128, "rank_uptime": 9.503413668, "state": "up:active", "uptime": 14.467615002, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:41:12.798 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:41:12.798 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-12-05T17:41:12.799 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi002.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-12-05T17:41:12.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:12.800 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:12.807 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:41:12.864 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:41:12.865 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:41:12.921 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:41:12.922 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:41:12.922 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:12.923 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:41:12.967 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:41:12.968 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:13.012 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:41:13.026 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:41:13.026 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:41:13.080 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:41:13.081 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:41:13.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:13.082 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:41:13.089 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:2020-12-05T17:41:13.085+0000 7faa6b2670c0 -1 init, newargv = 0x55b17fab80b0 newargc=9ceph-fuse[ 2020-12-05T17:41:13.089 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:11014]: starting ceph client 2020-12-05T17:41:13.100 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[11014]: starting fuse 2020-12-05T17:41:14.195 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:41:14.211 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:41:14.212 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:41:14.273 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:41:14.274 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:41:14.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:14.275 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:41:14.319 INFO:teuthology.orchestra.run.smithi002.stdout:50 2020-12-05T17:41:14.320 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [50] 2020-12-05T17:41:14.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:14.321 INFO:teuthology.orchestra.run.smithi002:> sudo python3 -c ' 2020-12-05T17:41:14.321 INFO:teuthology.orchestra.run.smithi002:> import glob 2020-12-05T17:41:14.321 INFO:teuthology.orchestra.run.smithi002:> import re 2020-12-05T17:41:14.322 INFO:teuthology.orchestra.run.smithi002:> import os 2020-12-05T17:41:14.322 INFO:teuthology.orchestra.run.smithi002:> import subprocess 2020-12-05T17:41:14.323 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:41:14.323 INFO:teuthology.orchestra.run.smithi002:> def find_socket(client_name): 2020-12-05T17:41:14.323 INFO:teuthology.orchestra.run.smithi002:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-12-05T17:41:14.324 INFO:teuthology.orchestra.run.smithi002:> files = glob.glob(asok_path) 2020-12-05T17:41:14.324 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:41:14.325 INFO:teuthology.orchestra.run.smithi002:> # Given a non-glob path, it better be there 2020-12-05T17:41:14.325 INFO:teuthology.orchestra.run.smithi002:> if "*" not in asok_path: 2020-12-05T17:41:14.325 INFO:teuthology.orchestra.run.smithi002:> assert(len(files) == 1) 2020-12-05T17:41:14.326 INFO:teuthology.orchestra.run.smithi002:> return files[0] 2020-12-05T17:41:14.326 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:41:14.326 INFO:teuthology.orchestra.run.smithi002:> for f in files: 2020-12-05T17:41:14.327 INFO:teuthology.orchestra.run.smithi002:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:41:14.327 INFO:teuthology.orchestra.run.smithi002:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:41:14.328 INFO:teuthology.orchestra.run.smithi002:> return f 2020-12-05T17:41:14.328 INFO:teuthology.orchestra.run.smithi002:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:41:14.328 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:41:14.329 INFO:teuthology.orchestra.run.smithi002:> print(find_socket("client.1")) 2020-12-05T17:41:14.329 INFO:teuthology.orchestra.run.smithi002:> ' 2020-12-05T17:41:14.412 INFO:teuthology.orchestra.run.smithi002.stdout:/var/run/ceph/ceph-client.1.11014.asok 2020-12-05T17:41:14.417 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.11014.asok 2020-12-05T17:41:14.418 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:14.418 INFO:teuthology.orchestra.run.smithi002:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.11014.asok status 2020-12-05T17:41:14.516 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2020-12-05T17:41:14.516 INFO:teuthology.orchestra.run.smithi002.stdout: "metadata": { 2020-12-05T17:41:14.517 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:41:14.517 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:41:14.517 INFO:teuthology.orchestra.run.smithi002.stdout: "entity_id": "1", 2020-12-05T17:41:14.517 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": "smithi002", 2020-12-05T17:41:14.517 INFO:teuthology.orchestra.run.smithi002.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-12-05T17:41:14.518 INFO:teuthology.orchestra.run.smithi002.stdout: "pid": "11014", 2020-12-05T17:41:14.518 INFO:teuthology.orchestra.run.smithi002.stdout: "root": "/" 2020-12-05T17:41:14.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:41:14.518 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_count": 0, 2020-12-05T17:41:14.518 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_pinned_count": 0, 2020-12-05T17:41:14.519 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 8130, 2020-12-05T17:41:14.519 INFO:teuthology.orchestra.run.smithi002.stdout: "inst": { 2020-12-05T17:41:14.519 INFO:teuthology.orchestra.run.smithi002.stdout: "name": { 2020-12-05T17:41:14.519 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "client", 2020-12-05T17:41:14.519 INFO:teuthology.orchestra.run.smithi002.stdout: "num": 8130 2020-12-05T17:41:14.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:41:14.520 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2020-12-05T17:41:14.520 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2020-12-05T17:41:14.520 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2020-12-05T17:41:14.520 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3846082508 2020-12-05T17:41:14.521 INFO:teuthology.orchestra.run.smithi002.stdout: } 2020-12-05T17:41:14.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:41:14.521 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2020-12-05T17:41:14.521 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2020-12-05T17:41:14.522 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2020-12-05T17:41:14.522 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3846082508 2020-12-05T17:41:14.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:41:14.522 INFO:teuthology.orchestra.run.smithi002.stdout: "inst_str": "client.8130 172.21.15.2:0/3846082508", 2020-12-05T17:41:14.522 INFO:teuthology.orchestra.run.smithi002.stdout: "addr_str": "172.21.15.2:0/3846082508", 2020-12-05T17:41:14.523 INFO:teuthology.orchestra.run.smithi002.stdout: "inode_count": 1, 2020-12-05T17:41:14.523 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_epoch": 62, 2020-12-05T17:41:14.523 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch": 129, 2020-12-05T17:41:14.523 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:41:14.523 INFO:teuthology.orchestra.run.smithi002.stdout: "blacklisted": false 2020-12-05T17:41:14.524 INFO:teuthology.orchestra.run.smithi002.stdout:} 2020-12-05T17:41:14.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:14.526 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:14.526 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:14.535 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:41:14.535 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:41:14.535 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:14.536 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:14.595 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:14.596 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:14.596 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:14.645 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:41:14.646 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:41:14.646 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:14.647 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:14.647 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:14.693 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:41:14.694 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:41:14.695 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi002.front.sepia.ceph.com... 2020-12-05T17:41:14.695 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:41:14.695 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:14.778 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[11014]: fuse finished with error 0 and tester_r 0 2020-12-05T17:41:14.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:14.781 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:14.782 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:14.828 INFO:teuthology.orchestra.run.smithi002.stdout:ext2/ext3 2020-12-05T17:41:14.829 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:41:14.830 INFO:teuthology.orchestra.run:waiting for 900 2020-12-05T17:41:14.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:41:14.884 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:14.892 INFO:teuthology.orchestra.run.smithi002:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-12-05T17:41:14.936 INFO:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-12-05T17:41:14.938 INFO:teuthology.orchestra.run.smithi036:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-12-05T17:41:14.940 INFO:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-12-05T17:41:14.943 INFO:teuthology.orchestra.run.smithi061:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-12-05T17:41:14.966 INFO:teuthology.orchestra.run.smithi002.stdout:[global] 2020-12-05T17:41:14.967 INFO:teuthology.orchestra.run.smithi002.stdout: chdir = "" 2020-12-05T17:41:14.967 INFO:teuthology.orchestra.run.smithi002.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-12-05T17:41:14.967 INFO:teuthology.orchestra.run.smithi002.stdout: auth supported = cephx 2020-12-05T17:41:14.967 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.967 INFO:teuthology.orchestra.run.smithi002.stdout: filestore xattr use omap = true 2020-12-05T17:41:14.968 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.968 INFO:teuthology.orchestra.run.smithi002.stdout: mon clock drift allowed = 1.000 2020-12-05T17:41:14.968 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.968 INFO:teuthology.orchestra.run.smithi002.stdout: osd crush chooseleaf type = 0 2020-12-05T17:41:14.968 INFO:teuthology.orchestra.run.smithi002.stdout: auth debug = true 2020-12-05T17:41:14.969 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.969 INFO:teuthology.orchestra.run.smithi002.stdout: ms die on old message = true 2020-12-05T17:41:14.969 INFO:teuthology.orchestra.run.smithi002.stdout: ms die on bug = true 2020-12-05T17:41:14.969 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.969 INFO:teuthology.orchestra.run.smithi002.stdout: mon max pg per osd = 10000 # >= luminous 2020-12-05T17:41:14.970 INFO:teuthology.orchestra.run.smithi002.stdout: mon pg warn max object skew = 0 2020-12-05T17:41:14.970 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.970 INFO:teuthology.orchestra.run.smithi002.stdout: # disable pg_autoscaler by default for new pools 2020-12-05T17:41:14.971 INFO:teuthology.orchestra.run.smithi002.stdout: osd_pool_default_pg_autoscale_mode = off 2020-12-05T17:41:14.971 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.971 INFO:teuthology.orchestra.run.smithi002.stdout: osd pool default size = 2 2020-12-05T17:41:14.971 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.971 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd allow primary affinity = true 2020-12-05T17:41:14.972 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd allow pg remap = true 2020-12-05T17:41:14.972 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on legacy crush tunables = false 2020-12-05T17:41:14.972 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on crush straw calc version zero = false 2020-12-05T17:41:14.972 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on no sortbitwise = false 2020-12-05T17:41:14.972 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on osd down out interval zero = false 2020-12-05T17:41:14.973 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on too few osds = false 2020-12-05T17:41:14.973 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-12-05T17:41:14.973 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_pool_no_redundancy = false 2020-12-05T17:41:14.973 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.973 INFO:teuthology.orchestra.run.smithi002.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-12-05T17:41:14.974 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.974 INFO:teuthology.orchestra.run.smithi002.stdout: osd default data pool replay window = 5 2020-12-05T17:41:14.974 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.974 INFO:teuthology.orchestra.run.smithi002.stdout: mon allow pool delete = true 2020-12-05T17:41:14.974 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.975 INFO:teuthology.orchestra.run.smithi002.stdout: mon cluster log file level = debug 2020-12-05T17:41:14.975 INFO:teuthology.orchestra.run.smithi002.stdout: debug asserts on shutdown = true 2020-12-05T17:41:14.975 INFO:teuthology.orchestra.run.smithi002.stdout: mon host = "172.21.15.32,172.21.15.61,[v2:172.21.15.61:3301,v1:172.21.15.61:6790]" 2020-12-05T17:41:14.975 INFO:teuthology.orchestra.run.smithi002.stdout: fsid = efd7e861-a502-461d-b20d-3b2c0a25fbd9 2020-12-05T17:41:14.975 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.976 INFO:teuthology.orchestra.run.smithi002.stdout:[osd] 2020-12-05T17:41:14.976 INFO:teuthology.orchestra.run.smithi002.stdout: osd journal size = 100 2020-12-05T17:41:14.976 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.976 INFO:teuthology.orchestra.run.smithi002.stdout: osd scrub load threshold = 5.0 2020-12-05T17:41:14.976 INFO:teuthology.orchestra.run.smithi002.stdout: osd scrub max interval = 600 2020-12-05T17:41:14.977 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.977 INFO:teuthology.orchestra.run.smithi002.stdout: osd recover clone overlap = true 2020-12-05T17:41:14.977 INFO:teuthology.orchestra.run.smithi002.stdout: osd recovery max chunk = 1048576 2020-12-05T17:41:14.977 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.977 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug shutdown = true 2020-12-05T17:41:14.977 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug op order = true 2020-12-05T17:41:14.978 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug verify stray on activate = true 2020-12-05T17:41:14.978 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.978 INFO:teuthology.orchestra.run.smithi002.stdout: osd open classes on start = true 2020-12-05T17:41:14.978 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug pg log writeout = true 2020-12-05T17:41:14.979 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.979 INFO:teuthology.orchestra.run.smithi002.stdout: osd deep scrub update digest min age = 30 2020-12-05T17:41:14.979 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.979 INFO:teuthology.orchestra.run.smithi002.stdout: osd map max advance = 10 2020-12-05T17:41:14.979 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.979 INFO:teuthology.orchestra.run.smithi002.stdout: journal zero on create = true 2020-12-05T17:41:14.980 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.980 INFO:teuthology.orchestra.run.smithi002.stdout: filestore ondisk finisher threads = 3 2020-12-05T17:41:14.980 INFO:teuthology.orchestra.run.smithi002.stdout: filestore apply finisher threads = 3 2020-12-05T17:41:14.980 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.980 INFO:teuthology.orchestra.run.smithi002.stdout: bdev debug aio = true 2020-12-05T17:41:14.981 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug misdirected ops = true 2020-12-05T17:41:14.981 INFO:teuthology.orchestra.run.smithi002.stdout: bluestore block size = 96636764160 2020-12-05T17:41:14.981 INFO:teuthology.orchestra.run.smithi002.stdout: bluestore fsck on mount = True 2020-12-05T17:41:14.981 INFO:teuthology.orchestra.run.smithi002.stdout: debug bluefs = 20 2020-12-05T17:41:14.981 INFO:teuthology.orchestra.run.smithi002.stdout: debug bluestore = 20 2020-12-05T17:41:14.982 INFO:teuthology.orchestra.run.smithi002.stdout: debug filestore = 20 2020-12-05T17:41:14.982 INFO:teuthology.orchestra.run.smithi002.stdout: debug journal = 20 2020-12-05T17:41:14.982 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 20 2020-12-05T17:41:14.982 INFO:teuthology.orchestra.run.smithi002.stdout: debug osd = 25 2020-12-05T17:41:14.982 INFO:teuthology.orchestra.run.smithi002.stdout: debug rocksdb = 10 2020-12-05T17:41:14.983 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd backfillfull_ratio = 0.85 2020-12-05T17:41:14.983 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd full ratio = 0.9 2020-12-05T17:41:14.983 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd nearfull ratio = 0.8 2020-12-05T17:41:14.983 INFO:teuthology.orchestra.run.smithi002.stdout: osd failsafe full ratio = 0.95 2020-12-05T17:41:14.983 INFO:teuthology.orchestra.run.smithi002.stdout: osd objectstore = bluestore 2020-12-05T17:41:14.984 INFO:teuthology.orchestra.run.smithi002.stdout: osd op complaint time = 180 2020-12-05T17:41:14.984 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.984 INFO:teuthology.orchestra.run.smithi002.stdout:[mgr] 2020-12-05T17:41:14.984 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2020-12-05T17:41:14.984 INFO:teuthology.orchestra.run.smithi002.stdout: debug mgr = 20 2020-12-05T17:41:14.985 INFO:teuthology.orchestra.run.smithi002.stdout: debug mon = 20 2020-12-05T17:41:14.985 INFO:teuthology.orchestra.run.smithi002.stdout: debug auth = 20 2020-12-05T17:41:14.985 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:41:14.985 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:41:14.985 INFO:teuthology.orchestra.run.smithi002.stdout: mgr/telemetry/nag = false 2020-12-05T17:41:14.986 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.986 INFO:teuthology.orchestra.run.smithi002.stdout:[mon] 2020-12-05T17:41:14.986 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2020-12-05T17:41:14.986 INFO:teuthology.orchestra.run.smithi002.stdout: debug mon = 20 2020-12-05T17:41:14.986 INFO:teuthology.orchestra.run.smithi002.stdout: debug paxos = 20 2020-12-05T17:41:14.987 INFO:teuthology.orchestra.run.smithi002.stdout: debug auth = 20 2020-12-05T17:41:14.987 INFO:teuthology.orchestra.run.smithi002.stdout: mon data avail warn = 5 2020-12-05T17:41:14.987 INFO:teuthology.orchestra.run.smithi002.stdout: mon mgr mkfs grace = 240 2020-12-05T17:41:14.987 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:41:14.987 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd reporter subtree level = osd 2020-12-05T17:41:14.988 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd prime pg temp = true 2020-12-05T17:41:14.988 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:41:14.988 INFO:teuthology.orchestra.run.smithi002.stdout: mon op complaint time = 120 2020-12-05T17:41:14.988 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:14.988 INFO:teuthology.orchestra.run.smithi002.stdout:[client] 2020-12-05T17:41:14.989 INFO:teuthology.orchestra.run.smithi002.stdout: rgw cache enabled = true 2020-12-05T17:41:14.989 INFO:teuthology.orchestra.run.smithi002.stdout: rgw enable ops log = true 2020-12-05T17:41:14.989 INFO:teuthology.orchestra.run.smithi002.stdout: rgw enable usage log = true 2020-12-05T17:41:14.989 INFO:teuthology.orchestra.run.smithi002.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-12-05T17:41:14.989 INFO:teuthology.orchestra.run.smithi002.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-12-05T17:41:14.990 INFO:teuthology.orchestra.run.smithi002.stdout: client mount timeout = 600 2020-12-05T17:41:14.990 INFO:teuthology.orchestra.run.smithi002.stdout: debug client = 20 2020-12-05T17:41:14.990 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2020-12-05T17:41:14.990 INFO:teuthology.orchestra.run.smithi002.stdout: rados mon op timeout = 15m 2020-12-05T17:41:14.991 INFO:teuthology.orchestra.run.smithi002.stdout: rados osd op timeout = 15m 2020-12-05T17:41:14.991 INFO:teuthology.orchestra.run.smithi002.stdout: client debug getattr caps = true 2020-12-05T17:41:14.991 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.a] 2020-12-05T17:41:14.991 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.b] 2020-12-05T17:41:14.991 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.c] 2020-12-05T17:41:14.992 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.a] 2020-12-05T17:41:14.992 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.b] 2020-12-05T17:41:14.992 INFO:teuthology.orchestra.run.smithi002.stdout:[mds] 2020-12-05T17:41:14.992 INFO:teuthology.orchestra.run.smithi002.stdout: debug mds = 20 2020-12-05T17:41:14.992 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2020-12-05T17:41:14.992 INFO:teuthology.orchestra.run.smithi002.stdout: mds bal frag = True 2020-12-05T17:41:14.993 INFO:teuthology.orchestra.run.smithi002.stdout: mds bal fragment size max = 10000 2020-12-05T17:41:14.993 INFO:teuthology.orchestra.run.smithi002.stdout: mds bal merge size = 5 2020-12-05T17:41:14.993 INFO:teuthology.orchestra.run.smithi002.stdout: mds bal split bits = 3 2020-12-05T17:41:14.993 INFO:teuthology.orchestra.run.smithi002.stdout: mds bal split size = 100 2020-12-05T17:41:14.993 INFO:teuthology.orchestra.run.smithi002.stdout: mds debug frag = True 2020-12-05T17:41:14.994 INFO:teuthology.orchestra.run.smithi002.stdout: mds debug scatterstat = True 2020-12-05T17:41:14.994 INFO:teuthology.orchestra.run.smithi002.stdout: mds op complaint time = 180 2020-12-05T17:41:14.994 INFO:teuthology.orchestra.run.smithi002.stdout: mds verify scatter = True 2020-12-05T17:41:14.994 INFO:teuthology.orchestra.run.smithi002.stdout: osd op complaint time = 180 2020-12-05T17:41:14.994 INFO:teuthology.orchestra.run.smithi002.stdout: rados mon op timeout = 15m 2020-12-05T17:41:14.995 INFO:teuthology.orchestra.run.smithi002.stdout: rados osd op timeout = 15m 2020-12-05T17:41:14.997 INFO:teuthology.orchestra.run.smithi053.stdout:[global] 2020-12-05T17:41:14.998 INFO:teuthology.orchestra.run.smithi053.stdout: chdir = "" 2020-12-05T17:41:14.998 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-12-05T17:41:14.998 INFO:teuthology.orchestra.run.smithi053.stdout: auth supported = cephx 2020-12-05T17:41:14.998 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:14.999 INFO:teuthology.orchestra.run.smithi053.stdout: filestore xattr use omap = true 2020-12-05T17:41:14.999 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:14.999 INFO:teuthology.orchestra.run.smithi053.stdout: mon clock drift allowed = 1.000 2020-12-05T17:41:14.999 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:14.999 INFO:teuthology.orchestra.run.smithi053.stdout: osd crush chooseleaf type = 0 2020-12-05T17:41:15.000 INFO:teuthology.orchestra.run.smithi053.stdout: auth debug = true 2020-12-05T17:41:15.000 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:15.000 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on old message = true 2020-12-05T17:41:15.000 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on bug = true 2020-12-05T17:41:15.000 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:15.001 INFO:teuthology.orchestra.run.smithi053.stdout: mon max pg per osd = 10000 # >= luminous 2020-12-05T17:41:15.001 INFO:teuthology.orchestra.run.smithi053.stdout: mon pg warn max object skew = 0 2020-12-05T17:41:15.001 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:15.001 INFO:teuthology.orchestra.run.smithi053.stdout: # disable pg_autoscaler by default for new pools 2020-12-05T17:41:15.001 INFO:teuthology.orchestra.run.smithi053.stdout: osd_pool_default_pg_autoscale_mode = off 2020-12-05T17:41:15.002 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:15.002 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default size = 2 2020-12-05T17:41:15.002 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:15.002 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow primary affinity = true 2020-12-05T17:41:15.002 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow pg remap = true 2020-12-05T17:41:15.003 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on legacy crush tunables = false 2020-12-05T17:41:15.003 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on crush straw calc version zero = false 2020-12-05T17:41:15.003 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on no sortbitwise = false 2020-12-05T17:41:15.003 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on osd down out interval zero = false 2020-12-05T17:41:15.003 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on too few osds = false 2020-12-05T17:41:15.004 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-12-05T17:41:15.004 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_no_redundancy = false 2020-12-05T17:41:15.004 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:15.004 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-12-05T17:41:15.005 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:15.005 INFO:teuthology.orchestra.run.smithi053.stdout: osd default data pool replay window = 5 2020-12-05T17:41:15.005 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:15.005 INFO:teuthology.orchestra.run.smithi053.stdout: mon allow pool delete = true 2020-12-05T17:41:15.005 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:15.005 INFO:teuthology.orchestra.run.smithi053.stdout: mon cluster log file level = debug 2020-12-05T17:41:15.006 INFO:teuthology.orchestra.run.smithi053.stdout: debug asserts on shutdown = true 2020-12-05T17:41:15.006 INFO:teuthology.orchestra.run.smithi053.stdout: mon host = "172.21.15.32,172.21.15.61,[v2:172.21.15.61:3301,v1:172.21.15.61:6790]" 2020-12-05T17:41:15.006 INFO:teuthology.orchestra.run.smithi053.stdout: fsid = efd7e861-a502-461d-b20d-3b2c0a25fbd9 2020-12-05T17:41:15.006 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:15.007 INFO:teuthology.orchestra.run.smithi053.stdout:[osd] 2020-12-05T17:41:15.007 INFO:teuthology.orchestra.run.smithi053.stdout: osd journal size = 100 2020-12-05T17:41:15.007 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:15.007 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub load threshold = 5.0 2020-12-05T17:41:15.007 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub max interval = 600 2020-12-05T17:41:15.008 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:15.008 INFO:teuthology.orchestra.run.smithi053.stdout: osd recover clone overlap = true 2020-12-05T17:41:15.008 INFO:teuthology.orchestra.run.smithi053.stdout: osd recovery max chunk = 1048576 2020-12-05T17:41:15.008 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:15.008 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug shutdown = true 2020-12-05T17:41:15.009 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug op order = true 2020-12-05T17:41:15.009 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug verify stray on activate = true 2020-12-05T17:41:15.009 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:15.009 INFO:teuthology.orchestra.run.smithi053.stdout: osd open classes on start = true 2020-12-05T17:41:15.009 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug pg log writeout = true 2020-12-05T17:41:15.010 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:15.010 INFO:teuthology.orchestra.run.smithi053.stdout: osd deep scrub update digest min age = 30 2020-12-05T17:41:15.010 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:15.010 INFO:teuthology.orchestra.run.smithi053.stdout: osd map max advance = 10 2020-12-05T17:41:15.010 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:15.011 INFO:teuthology.orchestra.run.smithi053.stdout: journal zero on create = true 2020-12-05T17:41:15.011 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:15.011 INFO:teuthology.orchestra.run.smithi053.stdout: filestore ondisk finisher threads = 3 2020-12-05T17:41:15.011 INFO:teuthology.orchestra.run.smithi053.stdout: filestore apply finisher threads = 3 2020-12-05T17:41:15.011 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:15.012 INFO:teuthology.orchestra.run.smithi053.stdout: bdev debug aio = true 2020-12-05T17:41:15.012 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug misdirected ops = true 2020-12-05T17:41:15.012 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore block size = 96636764160 2020-12-05T17:41:15.012 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore fsck on mount = True 2020-12-05T17:41:15.012 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluefs = 20 2020-12-05T17:41:15.013 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluestore = 20 2020-12-05T17:41:15.013 INFO:teuthology.orchestra.run.smithi053.stdout: debug filestore = 20 2020-12-05T17:41:15.013 INFO:teuthology.orchestra.run.smithi053.stdout: debug journal = 20 2020-12-05T17:41:15.013 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 20 2020-12-05T17:41:15.013 INFO:teuthology.orchestra.run.smithi053.stdout: debug osd = 25 2020-12-05T17:41:15.014 INFO:teuthology.orchestra.run.smithi053.stdout: debug rocksdb = 10 2020-12-05T17:41:15.014 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd backfillfull_ratio = 0.85 2020-12-05T17:41:15.014 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd full ratio = 0.9 2020-12-05T17:41:15.014 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd nearfull ratio = 0.8 2020-12-05T17:41:15.014 INFO:teuthology.orchestra.run.smithi053.stdout: osd failsafe full ratio = 0.95 2020-12-05T17:41:15.015 INFO:teuthology.orchestra.run.smithi053.stdout: osd objectstore = bluestore 2020-12-05T17:41:15.015 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2020-12-05T17:41:15.015 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:15.015 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr] 2020-12-05T17:41:15.015 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2020-12-05T17:41:15.016 INFO:teuthology.orchestra.run.smithi053.stdout: debug mgr = 20 2020-12-05T17:41:15.016 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2020-12-05T17:41:15.016 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2020-12-05T17:41:15.016 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:41:15.016 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:41:15.017 INFO:teuthology.orchestra.run.smithi053.stdout: mgr/telemetry/nag = false 2020-12-05T17:41:15.017 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:15.017 INFO:teuthology.orchestra.run.smithi053.stdout:[mon] 2020-12-05T17:41:15.017 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2020-12-05T17:41:15.017 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2020-12-05T17:41:15.018 INFO:teuthology.orchestra.run.smithi053.stdout: debug paxos = 20 2020-12-05T17:41:15.018 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2020-12-05T17:41:15.018 INFO:teuthology.orchestra.run.smithi053.stdout: mon data avail warn = 5 2020-12-05T17:41:15.018 INFO:teuthology.orchestra.run.smithi053.stdout: mon mgr mkfs grace = 240 2020-12-05T17:41:15.018 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:41:15.019 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd reporter subtree level = osd 2020-12-05T17:41:15.019 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd prime pg temp = true 2020-12-05T17:41:15.019 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:41:15.019 INFO:teuthology.orchestra.run.smithi053.stdout: mon op complaint time = 120 2020-12-05T17:41:15.019 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:15.020 INFO:teuthology.orchestra.run.smithi053.stdout:[client] 2020-12-05T17:41:15.020 INFO:teuthology.orchestra.run.smithi053.stdout: rgw cache enabled = true 2020-12-05T17:41:15.020 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable ops log = true 2020-12-05T17:41:15.020 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable usage log = true 2020-12-05T17:41:15.020 INFO:teuthology.orchestra.run.smithi053.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-12-05T17:41:15.021 INFO:teuthology.orchestra.run.smithi053.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-12-05T17:41:15.021 INFO:teuthology.orchestra.run.smithi053.stdout: client mount timeout = 600 2020-12-05T17:41:15.021 INFO:teuthology.orchestra.run.smithi053.stdout: debug client = 20 2020-12-05T17:41:15.021 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2020-12-05T17:41:15.021 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 15m 2020-12-05T17:41:15.022 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 15m 2020-12-05T17:41:15.022 INFO:teuthology.orchestra.run.smithi053.stdout: client debug getattr caps = true 2020-12-05T17:41:15.022 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.a] 2020-12-05T17:41:15.022 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.b] 2020-12-05T17:41:15.022 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.c] 2020-12-05T17:41:15.023 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.a] 2020-12-05T17:41:15.023 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.b] 2020-12-05T17:41:15.023 INFO:teuthology.orchestra.run.smithi053.stdout:[mds] 2020-12-05T17:41:15.023 INFO:teuthology.orchestra.run.smithi053.stdout: debug mds = 20 2020-12-05T17:41:15.023 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2020-12-05T17:41:15.024 INFO:teuthology.orchestra.run.smithi053.stdout: mds bal frag = True 2020-12-05T17:41:15.024 INFO:teuthology.orchestra.run.smithi053.stdout: mds bal fragment size max = 10000 2020-12-05T17:41:15.024 INFO:teuthology.orchestra.run.smithi053.stdout: mds bal merge size = 5 2020-12-05T17:41:15.024 INFO:teuthology.orchestra.run.smithi053.stdout: mds bal split bits = 3 2020-12-05T17:41:15.024 INFO:teuthology.orchestra.run.smithi053.stdout: mds bal split size = 100 2020-12-05T17:41:15.024 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug frag = True 2020-12-05T17:41:15.025 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug scatterstat = True 2020-12-05T17:41:15.025 INFO:teuthology.orchestra.run.smithi053.stdout: mds op complaint time = 180 2020-12-05T17:41:15.025 INFO:teuthology.orchestra.run.smithi053.stdout: mds verify scatter = True 2020-12-05T17:41:15.025 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2020-12-05T17:41:15.025 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 15m 2020-12-05T17:41:15.026 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 15m 2020-12-05T17:41:15.026 INFO:teuthology.orchestra.run.smithi061.stdout:[global] 2020-12-05T17:41:15.026 INFO:teuthology.orchestra.run.smithi061.stdout: chdir = "" 2020-12-05T17:41:15.026 INFO:teuthology.orchestra.run.smithi061.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-12-05T17:41:15.027 INFO:teuthology.orchestra.run.smithi061.stdout: auth supported = cephx 2020-12-05T17:41:15.027 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.027 INFO:teuthology.orchestra.run.smithi061.stdout: filestore xattr use omap = true 2020-12-05T17:41:15.027 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.027 INFO:teuthology.orchestra.run.smithi061.stdout: mon clock drift allowed = 1.000 2020-12-05T17:41:15.028 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.028 INFO:teuthology.orchestra.run.smithi061.stdout: osd crush chooseleaf type = 0 2020-12-05T17:41:15.028 INFO:teuthology.orchestra.run.smithi061.stdout: auth debug = true 2020-12-05T17:41:15.028 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.028 INFO:teuthology.orchestra.run.smithi061.stdout: ms die on old message = true 2020-12-05T17:41:15.029 INFO:teuthology.orchestra.run.smithi061.stdout: ms die on bug = true 2020-12-05T17:41:15.029 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.029 INFO:teuthology.orchestra.run.smithi061.stdout: mon max pg per osd = 10000 # >= luminous 2020-12-05T17:41:15.029 INFO:teuthology.orchestra.run.smithi061.stdout: mon pg warn max object skew = 0 2020-12-05T17:41:15.029 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.030 INFO:teuthology.orchestra.run.smithi061.stdout: # disable pg_autoscaler by default for new pools 2020-12-05T17:41:15.030 INFO:teuthology.orchestra.run.smithi061.stdout: osd_pool_default_pg_autoscale_mode = off 2020-12-05T17:41:15.030 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.030 INFO:teuthology.orchestra.run.smithi061.stdout: osd pool default size = 2 2020-12-05T17:41:15.030 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.031 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd allow primary affinity = true 2020-12-05T17:41:15.031 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd allow pg remap = true 2020-12-05T17:41:15.031 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on legacy crush tunables = false 2020-12-05T17:41:15.031 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on crush straw calc version zero = false 2020-12-05T17:41:15.031 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on no sortbitwise = false 2020-12-05T17:41:15.032 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on osd down out interval zero = false 2020-12-05T17:41:15.032 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on too few osds = false 2020-12-05T17:41:15.032 INFO:teuthology.orchestra.run.smithi061.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-12-05T17:41:15.032 INFO:teuthology.orchestra.run.smithi061.stdout: mon_warn_on_pool_no_redundancy = false 2020-12-05T17:41:15.032 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.033 INFO:teuthology.orchestra.run.smithi061.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-12-05T17:41:15.033 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.033 INFO:teuthology.orchestra.run.smithi061.stdout: osd default data pool replay window = 5 2020-12-05T17:41:15.033 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.033 INFO:teuthology.orchestra.run.smithi061.stdout: mon allow pool delete = true 2020-12-05T17:41:15.034 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.034 INFO:teuthology.orchestra.run.smithi061.stdout: mon cluster log file level = debug 2020-12-05T17:41:15.034 INFO:teuthology.orchestra.run.smithi061.stdout: debug asserts on shutdown = true 2020-12-05T17:41:15.034 INFO:teuthology.orchestra.run.smithi061.stdout: mon host = "172.21.15.32,172.21.15.61,[v2:172.21.15.61:3301,v1:172.21.15.61:6790]" 2020-12-05T17:41:15.034 INFO:teuthology.orchestra.run.smithi061.stdout: fsid = efd7e861-a502-461d-b20d-3b2c0a25fbd9 2020-12-05T17:41:15.035 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.035 INFO:teuthology.orchestra.run.smithi061.stdout:[osd] 2020-12-05T17:41:15.035 INFO:teuthology.orchestra.run.smithi061.stdout: osd journal size = 100 2020-12-05T17:41:15.035 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.035 INFO:teuthology.orchestra.run.smithi061.stdout: osd scrub load threshold = 5.0 2020-12-05T17:41:15.036 INFO:teuthology.orchestra.run.smithi061.stdout: osd scrub max interval = 600 2020-12-05T17:41:15.036 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.036 INFO:teuthology.orchestra.run.smithi061.stdout: osd recover clone overlap = true 2020-12-05T17:41:15.036 INFO:teuthology.orchestra.run.smithi061.stdout: osd recovery max chunk = 1048576 2020-12-05T17:41:15.036 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.037 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug shutdown = true 2020-12-05T17:41:15.037 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug op order = true 2020-12-05T17:41:15.037 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug verify stray on activate = true 2020-12-05T17:41:15.037 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.037 INFO:teuthology.orchestra.run.smithi061.stdout: osd open classes on start = true 2020-12-05T17:41:15.038 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug pg log writeout = true 2020-12-05T17:41:15.038 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.038 INFO:teuthology.orchestra.run.smithi061.stdout: osd deep scrub update digest min age = 30 2020-12-05T17:41:15.038 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.038 INFO:teuthology.orchestra.run.smithi061.stdout: osd map max advance = 10 2020-12-05T17:41:15.039 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.039 INFO:teuthology.orchestra.run.smithi061.stdout: journal zero on create = true 2020-12-05T17:41:15.039 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.039 INFO:teuthology.orchestra.run.smithi061.stdout: filestore ondisk finisher threads = 3 2020-12-05T17:41:15.039 INFO:teuthology.orchestra.run.smithi061.stdout: filestore apply finisher threads = 3 2020-12-05T17:41:15.040 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.040 INFO:teuthology.orchestra.run.smithi061.stdout: bdev debug aio = true 2020-12-05T17:41:15.040 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug misdirected ops = true 2020-12-05T17:41:15.040 INFO:teuthology.orchestra.run.smithi061.stdout: bluestore block size = 96636764160 2020-12-05T17:41:15.040 INFO:teuthology.orchestra.run.smithi061.stdout: bluestore fsck on mount = True 2020-12-05T17:41:15.040 INFO:teuthology.orchestra.run.smithi061.stdout: debug bluefs = 20 2020-12-05T17:41:15.041 INFO:teuthology.orchestra.run.smithi061.stdout: debug bluestore = 20 2020-12-05T17:41:15.041 INFO:teuthology.orchestra.run.smithi061.stdout: debug filestore = 20 2020-12-05T17:41:15.041 INFO:teuthology.orchestra.run.smithi061.stdout: debug journal = 20 2020-12-05T17:41:15.041 INFO:teuthology.orchestra.run.smithi061.stdout: debug ms = 20 2020-12-05T17:41:15.041 INFO:teuthology.orchestra.run.smithi061.stdout: debug osd = 25 2020-12-05T17:41:15.042 INFO:teuthology.orchestra.run.smithi061.stdout: debug rocksdb = 10 2020-12-05T17:41:15.042 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd backfillfull_ratio = 0.85 2020-12-05T17:41:15.042 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd full ratio = 0.9 2020-12-05T17:41:15.042 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd nearfull ratio = 0.8 2020-12-05T17:41:15.042 INFO:teuthology.orchestra.run.smithi061.stdout: osd failsafe full ratio = 0.95 2020-12-05T17:41:15.043 INFO:teuthology.orchestra.run.smithi061.stdout: osd objectstore = bluestore 2020-12-05T17:41:15.043 INFO:teuthology.orchestra.run.smithi061.stdout: osd op complaint time = 180 2020-12-05T17:41:15.043 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.043 INFO:teuthology.orchestra.run.smithi061.stdout:[mgr] 2020-12-05T17:41:15.043 INFO:teuthology.orchestra.run.smithi061.stdout: debug ms = 1 2020-12-05T17:41:15.044 INFO:teuthology.orchestra.run.smithi061.stdout: debug mgr = 20 2020-12-05T17:41:15.044 INFO:teuthology.orchestra.run.smithi061.stdout: debug mon = 20 2020-12-05T17:41:15.044 INFO:teuthology.orchestra.run.smithi061.stdout: debug auth = 20 2020-12-05T17:41:15.044 INFO:teuthology.orchestra.run.smithi061.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:41:15.044 INFO:teuthology.orchestra.run.smithi061.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:41:15.045 INFO:teuthology.orchestra.run.smithi061.stdout: mgr/telemetry/nag = false 2020-12-05T17:41:15.045 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.045 INFO:teuthology.orchestra.run.smithi061.stdout:[mon] 2020-12-05T17:41:15.045 INFO:teuthology.orchestra.run.smithi061.stdout: debug ms = 1 2020-12-05T17:41:15.045 INFO:teuthology.orchestra.run.smithi061.stdout: debug mon = 20 2020-12-05T17:41:15.046 INFO:teuthology.orchestra.run.smithi061.stdout: debug paxos = 20 2020-12-05T17:41:15.046 INFO:teuthology.orchestra.run.smithi061.stdout: debug auth = 20 2020-12-05T17:41:15.046 INFO:teuthology.orchestra.run.smithi061.stdout: mon data avail warn = 5 2020-12-05T17:41:15.046 INFO:teuthology.orchestra.run.smithi061.stdout: mon mgr mkfs grace = 240 2020-12-05T17:41:15.046 INFO:teuthology.orchestra.run.smithi061.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:41:15.047 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd reporter subtree level = osd 2020-12-05T17:41:15.047 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd prime pg temp = true 2020-12-05T17:41:15.047 INFO:teuthology.orchestra.run.smithi061.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:41:15.047 INFO:teuthology.orchestra.run.smithi061.stdout: mon op complaint time = 120 2020-12-05T17:41:15.047 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:15.048 INFO:teuthology.orchestra.run.smithi061.stdout:[client] 2020-12-05T17:41:15.048 INFO:teuthology.orchestra.run.smithi061.stdout: rgw cache enabled = true 2020-12-05T17:41:15.048 INFO:teuthology.orchestra.run.smithi061.stdout: rgw enable ops log = true 2020-12-05T17:41:15.048 INFO:teuthology.orchestra.run.smithi061.stdout: rgw enable usage log = true 2020-12-05T17:41:15.048 INFO:teuthology.orchestra.run.smithi061.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-12-05T17:41:15.049 INFO:teuthology.orchestra.run.smithi061.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-12-05T17:41:15.049 INFO:teuthology.orchestra.run.smithi061.stdout: client mount timeout = 600 2020-12-05T17:41:15.049 INFO:teuthology.orchestra.run.smithi061.stdout: debug client = 20 2020-12-05T17:41:15.049 INFO:teuthology.orchestra.run.smithi061.stdout: debug ms = 1 2020-12-05T17:41:15.049 INFO:teuthology.orchestra.run.smithi061.stdout: rados mon op timeout = 15m 2020-12-05T17:41:15.050 INFO:teuthology.orchestra.run.smithi061.stdout: rados osd op timeout = 15m 2020-12-05T17:41:15.050 INFO:teuthology.orchestra.run.smithi061.stdout: client debug getattr caps = true 2020-12-05T17:41:15.050 INFO:teuthology.orchestra.run.smithi061.stdout:[mon.a] 2020-12-05T17:41:15.050 INFO:teuthology.orchestra.run.smithi061.stdout:[mon.b] 2020-12-05T17:41:15.051 INFO:teuthology.orchestra.run.smithi061.stdout:[mon.c] 2020-12-05T17:41:15.051 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.a] 2020-12-05T17:41:15.051 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.b] 2020-12-05T17:41:15.051 INFO:teuthology.orchestra.run.smithi061.stdout:[mds] 2020-12-05T17:41:15.051 INFO:teuthology.orchestra.run.smithi061.stdout: debug mds = 20 2020-12-05T17:41:15.051 INFO:teuthology.orchestra.run.smithi061.stdout: debug ms = 1 2020-12-05T17:41:15.052 INFO:teuthology.orchestra.run.smithi061.stdout: mds bal frag = True 2020-12-05T17:41:15.052 INFO:teuthology.orchestra.run.smithi061.stdout: mds bal fragment size max = 10000 2020-12-05T17:41:15.052 INFO:teuthology.orchestra.run.smithi061.stdout: mds bal merge size = 5 2020-12-05T17:41:15.052 INFO:teuthology.orchestra.run.smithi061.stdout: mds bal split bits = 3 2020-12-05T17:41:15.053 INFO:teuthology.orchestra.run.smithi061.stdout: mds bal split size = 100 2020-12-05T17:41:15.053 INFO:teuthology.orchestra.run.smithi061.stdout: mds debug frag = True 2020-12-05T17:41:15.053 INFO:teuthology.orchestra.run.smithi061.stdout: mds debug scatterstat = True 2020-12-05T17:41:15.053 INFO:teuthology.orchestra.run.smithi061.stdout: mds op complaint time = 180 2020-12-05T17:41:15.053 INFO:teuthology.orchestra.run.smithi061.stdout: mds verify scatter = True 2020-12-05T17:41:15.054 INFO:teuthology.orchestra.run.smithi061.stdout: osd op complaint time = 180 2020-12-05T17:41:15.054 INFO:teuthology.orchestra.run.smithi061.stdout: rados mon op timeout = 15m 2020-12-05T17:41:15.054 INFO:teuthology.orchestra.run.smithi061.stdout: rados osd op timeout = 15m 2020-12-05T17:41:15.054 INFO:teuthology.orchestra.run.smithi036.stdout:[global] 2020-12-05T17:41:15.055 INFO:teuthology.orchestra.run.smithi036.stdout: chdir = "" 2020-12-05T17:41:15.055 INFO:teuthology.orchestra.run.smithi036.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-12-05T17:41:15.055 INFO:teuthology.orchestra.run.smithi036.stdout: auth supported = cephx 2020-12-05T17:41:15.055 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.055 INFO:teuthology.orchestra.run.smithi036.stdout: filestore xattr use omap = true 2020-12-05T17:41:15.056 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.056 INFO:teuthology.orchestra.run.smithi036.stdout: mon clock drift allowed = 1.000 2020-12-05T17:41:15.056 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.056 INFO:teuthology.orchestra.run.smithi036.stdout: osd crush chooseleaf type = 0 2020-12-05T17:41:15.056 INFO:teuthology.orchestra.run.smithi036.stdout: auth debug = true 2020-12-05T17:41:15.057 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.057 INFO:teuthology.orchestra.run.smithi036.stdout: ms die on old message = true 2020-12-05T17:41:15.057 INFO:teuthology.orchestra.run.smithi036.stdout: ms die on bug = true 2020-12-05T17:41:15.057 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.057 INFO:teuthology.orchestra.run.smithi036.stdout: mon max pg per osd = 10000 # >= luminous 2020-12-05T17:41:15.058 INFO:teuthology.orchestra.run.smithi036.stdout: mon pg warn max object skew = 0 2020-12-05T17:41:15.058 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.058 INFO:teuthology.orchestra.run.smithi036.stdout: # disable pg_autoscaler by default for new pools 2020-12-05T17:41:15.058 INFO:teuthology.orchestra.run.smithi036.stdout: osd_pool_default_pg_autoscale_mode = off 2020-12-05T17:41:15.058 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.059 INFO:teuthology.orchestra.run.smithi036.stdout: osd pool default size = 2 2020-12-05T17:41:15.059 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.059 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd allow primary affinity = true 2020-12-05T17:41:15.059 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd allow pg remap = true 2020-12-05T17:41:15.059 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on legacy crush tunables = false 2020-12-05T17:41:15.060 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on crush straw calc version zero = false 2020-12-05T17:41:15.060 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on no sortbitwise = false 2020-12-05T17:41:15.060 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on osd down out interval zero = false 2020-12-05T17:41:15.060 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on too few osds = false 2020-12-05T17:41:15.060 INFO:teuthology.orchestra.run.smithi036.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-12-05T17:41:15.061 INFO:teuthology.orchestra.run.smithi036.stdout: mon_warn_on_pool_no_redundancy = false 2020-12-05T17:41:15.061 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.061 INFO:teuthology.orchestra.run.smithi036.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-12-05T17:41:15.061 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.062 INFO:teuthology.orchestra.run.smithi036.stdout: osd default data pool replay window = 5 2020-12-05T17:41:15.062 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.062 INFO:teuthology.orchestra.run.smithi036.stdout: mon allow pool delete = true 2020-12-05T17:41:15.062 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.062 INFO:teuthology.orchestra.run.smithi036.stdout: mon cluster log file level = debug 2020-12-05T17:41:15.063 INFO:teuthology.orchestra.run.smithi036.stdout: debug asserts on shutdown = true 2020-12-05T17:41:15.063 INFO:teuthology.orchestra.run.smithi036.stdout: mon host = "172.21.15.32,172.21.15.61,[v2:172.21.15.61:3301,v1:172.21.15.61:6790]" 2020-12-05T17:41:15.063 INFO:teuthology.orchestra.run.smithi036.stdout: fsid = efd7e861-a502-461d-b20d-3b2c0a25fbd9 2020-12-05T17:41:15.063 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.064 INFO:teuthology.orchestra.run.smithi036.stdout:[osd] 2020-12-05T17:41:15.064 INFO:teuthology.orchestra.run.smithi036.stdout: osd journal size = 100 2020-12-05T17:41:15.064 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.064 INFO:teuthology.orchestra.run.smithi036.stdout: osd scrub load threshold = 5.0 2020-12-05T17:41:15.064 INFO:teuthology.orchestra.run.smithi036.stdout: osd scrub max interval = 600 2020-12-05T17:41:15.065 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.065 INFO:teuthology.orchestra.run.smithi036.stdout: osd recover clone overlap = true 2020-12-05T17:41:15.065 INFO:teuthology.orchestra.run.smithi036.stdout: osd recovery max chunk = 1048576 2020-12-05T17:41:15.065 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.066 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug shutdown = true 2020-12-05T17:41:15.066 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug op order = true 2020-12-05T17:41:15.066 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug verify stray on activate = true 2020-12-05T17:41:15.066 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.066 INFO:teuthology.orchestra.run.smithi036.stdout: osd open classes on start = true 2020-12-05T17:41:15.067 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug pg log writeout = true 2020-12-05T17:41:15.067 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.067 INFO:teuthology.orchestra.run.smithi036.stdout: osd deep scrub update digest min age = 30 2020-12-05T17:41:15.067 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.067 INFO:teuthology.orchestra.run.smithi036.stdout: osd map max advance = 10 2020-12-05T17:41:15.068 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.068 INFO:teuthology.orchestra.run.smithi036.stdout: journal zero on create = true 2020-12-05T17:41:15.068 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.068 INFO:teuthology.orchestra.run.smithi036.stdout: filestore ondisk finisher threads = 3 2020-12-05T17:41:15.068 INFO:teuthology.orchestra.run.smithi036.stdout: filestore apply finisher threads = 3 2020-12-05T17:41:15.069 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.069 INFO:teuthology.orchestra.run.smithi036.stdout: bdev debug aio = true 2020-12-05T17:41:15.069 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug misdirected ops = true 2020-12-05T17:41:15.069 INFO:teuthology.orchestra.run.smithi036.stdout: bluestore block size = 96636764160 2020-12-05T17:41:15.069 INFO:teuthology.orchestra.run.smithi036.stdout: bluestore fsck on mount = True 2020-12-05T17:41:15.070 INFO:teuthology.orchestra.run.smithi036.stdout: debug bluefs = 20 2020-12-05T17:41:15.070 INFO:teuthology.orchestra.run.smithi036.stdout: debug bluestore = 20 2020-12-05T17:41:15.070 INFO:teuthology.orchestra.run.smithi036.stdout: debug filestore = 20 2020-12-05T17:41:15.070 INFO:teuthology.orchestra.run.smithi036.stdout: debug journal = 20 2020-12-05T17:41:15.070 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 20 2020-12-05T17:41:15.071 INFO:teuthology.orchestra.run.smithi036.stdout: debug osd = 25 2020-12-05T17:41:15.071 INFO:teuthology.orchestra.run.smithi036.stdout: debug rocksdb = 10 2020-12-05T17:41:15.071 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd backfillfull_ratio = 0.85 2020-12-05T17:41:15.071 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd full ratio = 0.9 2020-12-05T17:41:15.071 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd nearfull ratio = 0.8 2020-12-05T17:41:15.072 INFO:teuthology.orchestra.run.smithi036.stdout: osd failsafe full ratio = 0.95 2020-12-05T17:41:15.072 INFO:teuthology.orchestra.run.smithi036.stdout: osd objectstore = bluestore 2020-12-05T17:41:15.072 INFO:teuthology.orchestra.run.smithi036.stdout: osd op complaint time = 180 2020-12-05T17:41:15.072 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.072 INFO:teuthology.orchestra.run.smithi036.stdout:[mgr] 2020-12-05T17:41:15.073 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2020-12-05T17:41:15.073 INFO:teuthology.orchestra.run.smithi036.stdout: debug mgr = 20 2020-12-05T17:41:15.073 INFO:teuthology.orchestra.run.smithi036.stdout: debug mon = 20 2020-12-05T17:41:15.073 INFO:teuthology.orchestra.run.smithi036.stdout: debug auth = 20 2020-12-05T17:41:15.073 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:41:15.074 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:41:15.074 INFO:teuthology.orchestra.run.smithi036.stdout: mgr/telemetry/nag = false 2020-12-05T17:41:15.074 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.074 INFO:teuthology.orchestra.run.smithi036.stdout:[mon] 2020-12-05T17:41:15.075 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2020-12-05T17:41:15.075 INFO:teuthology.orchestra.run.smithi036.stdout: debug mon = 20 2020-12-05T17:41:15.075 INFO:teuthology.orchestra.run.smithi036.stdout: debug paxos = 20 2020-12-05T17:41:15.075 INFO:teuthology.orchestra.run.smithi036.stdout: debug auth = 20 2020-12-05T17:41:15.075 INFO:teuthology.orchestra.run.smithi036.stdout: mon data avail warn = 5 2020-12-05T17:41:15.075 INFO:teuthology.orchestra.run.smithi036.stdout: mon mgr mkfs grace = 240 2020-12-05T17:41:15.076 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:41:15.076 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd reporter subtree level = osd 2020-12-05T17:41:15.076 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd prime pg temp = true 2020-12-05T17:41:15.076 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:41:15.076 INFO:teuthology.orchestra.run.smithi036.stdout: mon op complaint time = 120 2020-12-05T17:41:15.077 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:15.077 INFO:teuthology.orchestra.run.smithi036.stdout:[client] 2020-12-05T17:41:15.077 INFO:teuthology.orchestra.run.smithi036.stdout: rgw cache enabled = true 2020-12-05T17:41:15.077 INFO:teuthology.orchestra.run.smithi036.stdout: rgw enable ops log = true 2020-12-05T17:41:15.077 INFO:teuthology.orchestra.run.smithi036.stdout: rgw enable usage log = true 2020-12-05T17:41:15.078 INFO:teuthology.orchestra.run.smithi036.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-12-05T17:41:15.078 INFO:teuthology.orchestra.run.smithi036.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-12-05T17:41:15.078 INFO:teuthology.orchestra.run.smithi036.stdout: client mount timeout = 600 2020-12-05T17:41:15.078 INFO:teuthology.orchestra.run.smithi036.stdout: debug client = 20 2020-12-05T17:41:15.078 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2020-12-05T17:41:15.079 INFO:teuthology.orchestra.run.smithi036.stdout: rados mon op timeout = 15m 2020-12-05T17:41:15.079 INFO:teuthology.orchestra.run.smithi036.stdout: rados osd op timeout = 15m 2020-12-05T17:41:15.079 INFO:teuthology.orchestra.run.smithi036.stdout: client debug getattr caps = true 2020-12-05T17:41:15.079 INFO:teuthology.orchestra.run.smithi036.stdout:[mon.a] 2020-12-05T17:41:15.079 INFO:teuthology.orchestra.run.smithi036.stdout:[mon.b] 2020-12-05T17:41:15.080 INFO:teuthology.orchestra.run.smithi036.stdout:[mon.c] 2020-12-05T17:41:15.080 INFO:teuthology.orchestra.run.smithi036.stdout:[mds.a] 2020-12-05T17:41:15.080 INFO:teuthology.orchestra.run.smithi036.stdout:[mds.b] 2020-12-05T17:41:15.080 INFO:teuthology.orchestra.run.smithi036.stdout:[mds] 2020-12-05T17:41:15.080 INFO:teuthology.orchestra.run.smithi036.stdout: debug mds = 20 2020-12-05T17:41:15.081 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2020-12-05T17:41:15.081 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal frag = True 2020-12-05T17:41:15.081 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal fragment size max = 10000 2020-12-05T17:41:15.081 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal merge size = 5 2020-12-05T17:41:15.081 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal split bits = 3 2020-12-05T17:41:15.082 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal split size = 100 2020-12-05T17:41:15.082 INFO:teuthology.orchestra.run.smithi036.stdout: mds debug frag = True 2020-12-05T17:41:15.082 INFO:teuthology.orchestra.run.smithi036.stdout: mds debug scatterstat = True 2020-12-05T17:41:15.082 INFO:teuthology.orchestra.run.smithi036.stdout: mds op complaint time = 180 2020-12-05T17:41:15.082 INFO:teuthology.orchestra.run.smithi036.stdout: mds verify scatter = True 2020-12-05T17:41:15.083 INFO:teuthology.orchestra.run.smithi036.stdout: osd op complaint time = 180 2020-12-05T17:41:15.083 INFO:teuthology.orchestra.run.smithi036.stdout: rados mon op timeout = 15m 2020-12-05T17:41:15.083 INFO:teuthology.orchestra.run.smithi036.stdout: rados osd op timeout = 15m 2020-12-05T17:41:15.083 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2020-12-05T17:41:15.084 INFO:teuthology.orchestra.run.smithi032.stdout: chdir = "" 2020-12-05T17:41:15.084 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-12-05T17:41:15.084 INFO:teuthology.orchestra.run.smithi032.stdout: auth supported = cephx 2020-12-05T17:41:15.084 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.084 INFO:teuthology.orchestra.run.smithi032.stdout: filestore xattr use omap = true 2020-12-05T17:41:15.085 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.085 INFO:teuthology.orchestra.run.smithi032.stdout: mon clock drift allowed = 1.000 2020-12-05T17:41:15.085 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.085 INFO:teuthology.orchestra.run.smithi032.stdout: osd crush chooseleaf type = 0 2020-12-05T17:41:15.085 INFO:teuthology.orchestra.run.smithi032.stdout: auth debug = true 2020-12-05T17:41:15.086 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.086 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on old message = true 2020-12-05T17:41:15.086 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on bug = true 2020-12-05T17:41:15.086 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.086 INFO:teuthology.orchestra.run.smithi032.stdout: mon max pg per osd = 10000 # >= luminous 2020-12-05T17:41:15.087 INFO:teuthology.orchestra.run.smithi032.stdout: mon pg warn max object skew = 0 2020-12-05T17:41:15.087 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.087 INFO:teuthology.orchestra.run.smithi032.stdout: # disable pg_autoscaler by default for new pools 2020-12-05T17:41:15.087 INFO:teuthology.orchestra.run.smithi032.stdout: osd_pool_default_pg_autoscale_mode = off 2020-12-05T17:41:15.087 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.088 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default size = 2 2020-12-05T17:41:15.088 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.088 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow primary affinity = true 2020-12-05T17:41:15.088 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow pg remap = true 2020-12-05T17:41:15.088 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on legacy crush tunables = false 2020-12-05T17:41:15.089 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on crush straw calc version zero = false 2020-12-05T17:41:15.089 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on no sortbitwise = false 2020-12-05T17:41:15.089 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on osd down out interval zero = false 2020-12-05T17:41:15.089 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on too few osds = false 2020-12-05T17:41:15.089 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-12-05T17:41:15.090 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_no_redundancy = false 2020-12-05T17:41:15.090 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.090 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-12-05T17:41:15.090 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.090 INFO:teuthology.orchestra.run.smithi032.stdout: osd default data pool replay window = 5 2020-12-05T17:41:15.091 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.091 INFO:teuthology.orchestra.run.smithi032.stdout: mon allow pool delete = true 2020-12-05T17:41:15.091 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.091 INFO:teuthology.orchestra.run.smithi032.stdout: mon cluster log file level = debug 2020-12-05T17:41:15.091 INFO:teuthology.orchestra.run.smithi032.stdout: debug asserts on shutdown = true 2020-12-05T17:41:15.092 INFO:teuthology.orchestra.run.smithi032.stdout: mon host = "172.21.15.32,172.21.15.61,[v2:172.21.15.61:3301,v1:172.21.15.61:6790]" 2020-12-05T17:41:15.092 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = efd7e861-a502-461d-b20d-3b2c0a25fbd9 2020-12-05T17:41:15.092 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.092 INFO:teuthology.orchestra.run.smithi032.stdout:[osd] 2020-12-05T17:41:15.092 INFO:teuthology.orchestra.run.smithi032.stdout: osd journal size = 100 2020-12-05T17:41:15.093 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.093 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub load threshold = 5.0 2020-12-05T17:41:15.093 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub max interval = 600 2020-12-05T17:41:15.093 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.093 INFO:teuthology.orchestra.run.smithi032.stdout: osd recover clone overlap = true 2020-12-05T17:41:15.094 INFO:teuthology.orchestra.run.smithi032.stdout: osd recovery max chunk = 1048576 2020-12-05T17:41:15.094 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.094 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug shutdown = true 2020-12-05T17:41:15.094 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug op order = true 2020-12-05T17:41:15.095 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug verify stray on activate = true 2020-12-05T17:41:15.095 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.095 INFO:teuthology.orchestra.run.smithi032.stdout: osd open classes on start = true 2020-12-05T17:41:15.095 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug pg log writeout = true 2020-12-05T17:41:15.095 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.096 INFO:teuthology.orchestra.run.smithi032.stdout: osd deep scrub update digest min age = 30 2020-12-05T17:41:15.096 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.096 INFO:teuthology.orchestra.run.smithi032.stdout: osd map max advance = 10 2020-12-05T17:41:15.096 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.096 INFO:teuthology.orchestra.run.smithi032.stdout: journal zero on create = true 2020-12-05T17:41:15.097 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.097 INFO:teuthology.orchestra.run.smithi032.stdout: filestore ondisk finisher threads = 3 2020-12-05T17:41:15.097 INFO:teuthology.orchestra.run.smithi032.stdout: filestore apply finisher threads = 3 2020-12-05T17:41:15.097 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.097 INFO:teuthology.orchestra.run.smithi032.stdout: bdev debug aio = true 2020-12-05T17:41:15.098 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug misdirected ops = true 2020-12-05T17:41:15.098 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore block size = 96636764160 2020-12-05T17:41:15.098 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore fsck on mount = True 2020-12-05T17:41:15.098 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluefs = 20 2020-12-05T17:41:15.098 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluestore = 20 2020-12-05T17:41:15.099 INFO:teuthology.orchestra.run.smithi032.stdout: debug filestore = 20 2020-12-05T17:41:15.099 INFO:teuthology.orchestra.run.smithi032.stdout: debug journal = 20 2020-12-05T17:41:15.099 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 20 2020-12-05T17:41:15.099 INFO:teuthology.orchestra.run.smithi032.stdout: debug osd = 25 2020-12-05T17:41:15.099 INFO:teuthology.orchestra.run.smithi032.stdout: debug rocksdb = 10 2020-12-05T17:41:15.100 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd backfillfull_ratio = 0.85 2020-12-05T17:41:15.100 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd full ratio = 0.9 2020-12-05T17:41:15.100 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd nearfull ratio = 0.8 2020-12-05T17:41:15.100 INFO:teuthology.orchestra.run.smithi032.stdout: osd failsafe full ratio = 0.95 2020-12-05T17:41:15.101 INFO:teuthology.orchestra.run.smithi032.stdout: osd objectstore = bluestore 2020-12-05T17:41:15.101 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2020-12-05T17:41:15.101 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.101 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr] 2020-12-05T17:41:15.101 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2020-12-05T17:41:15.102 INFO:teuthology.orchestra.run.smithi032.stdout: debug mgr = 20 2020-12-05T17:41:15.102 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2020-12-05T17:41:15.102 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2020-12-05T17:41:15.102 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:41:15.102 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:41:15.103 INFO:teuthology.orchestra.run.smithi032.stdout: mgr/telemetry/nag = false 2020-12-05T17:41:15.103 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.103 INFO:teuthology.orchestra.run.smithi032.stdout:[mon] 2020-12-05T17:41:15.103 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2020-12-05T17:41:15.103 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2020-12-05T17:41:15.104 INFO:teuthology.orchestra.run.smithi032.stdout: debug paxos = 20 2020-12-05T17:41:15.104 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2020-12-05T17:41:15.104 INFO:teuthology.orchestra.run.smithi032.stdout: mon data avail warn = 5 2020-12-05T17:41:15.104 INFO:teuthology.orchestra.run.smithi032.stdout: mon mgr mkfs grace = 240 2020-12-05T17:41:15.104 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:41:15.105 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd reporter subtree level = osd 2020-12-05T17:41:15.105 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd prime pg temp = true 2020-12-05T17:41:15.105 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:41:15.105 INFO:teuthology.orchestra.run.smithi032.stdout: mon op complaint time = 120 2020-12-05T17:41:15.105 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.106 INFO:teuthology.orchestra.run.smithi032.stdout:[client] 2020-12-05T17:41:15.106 INFO:teuthology.orchestra.run.smithi032.stdout: rgw cache enabled = true 2020-12-05T17:41:15.106 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable ops log = true 2020-12-05T17:41:15.106 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable usage log = true 2020-12-05T17:41:15.106 INFO:teuthology.orchestra.run.smithi032.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-12-05T17:41:15.107 INFO:teuthology.orchestra.run.smithi032.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-12-05T17:41:15.107 INFO:teuthology.orchestra.run.smithi032.stdout: client mount timeout = 600 2020-12-05T17:41:15.107 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2020-12-05T17:41:15.107 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2020-12-05T17:41:15.107 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 15m 2020-12-05T17:41:15.108 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 15m 2020-12-05T17:41:15.108 INFO:teuthology.orchestra.run.smithi032.stdout: client debug getattr caps = true 2020-12-05T17:41:15.108 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.a] 2020-12-05T17:41:15.108 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.b] 2020-12-05T17:41:15.108 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.c] 2020-12-05T17:41:15.109 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2020-12-05T17:41:15.109 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2020-12-05T17:41:15.109 INFO:teuthology.orchestra.run.smithi032.stdout:[mds] 2020-12-05T17:41:15.113 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds = 20 2020-12-05T17:41:15.113 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2020-12-05T17:41:15.113 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal frag = True 2020-12-05T17:41:15.113 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal fragment size max = 10000 2020-12-05T17:41:15.113 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal merge size = 5 2020-12-05T17:41:15.114 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal split bits = 3 2020-12-05T17:41:15.114 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal split size = 100 2020-12-05T17:41:15.114 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug frag = True 2020-12-05T17:41:15.114 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug scatterstat = True 2020-12-05T17:41:15.115 INFO:teuthology.orchestra.run.smithi032.stdout: mds op complaint time = 180 2020-12-05T17:41:15.115 INFO:teuthology.orchestra.run.smithi032.stdout: mds verify scatter = True 2020-12-05T17:41:15.115 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2020-12-05T17:41:15.115 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 15m 2020-12-05T17:41:15.115 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 15m 2020-12-05T17:41:15.116 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:41:15.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c8a01700 1 Processor -- start 2020-12-05T17:41:15.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c8a01700 1 -- start start 2020-12-05T17:41:15.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c8a01700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59c40b29f0 0x7f59c40677b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:15.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c8a01700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f59c413ed00 0x7f59c413f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:15.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c8a01700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59c4067ca0 con 0x7f59c40b1a10 2020-12-05T17:41:15.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c8a01700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59c41432a0 con 0x7f59c40b29f0 2020-12-05T17:41:15.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c8a01700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59c40b2dd0 con 0x7f59c413ed00 2020-12-05T17:41:15.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59c40b29f0 0x7f59c40677b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:15.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59baffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f59c40b1a10 0x7f59c4142b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43132/0 (socket says 172.21.15.32:43132) 2020-12-05T17:41:15.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59baffd700 1 -- 172.21.15.32:0/2663222379 learned_addr learned my addr 172.21.15.32:0/2663222379 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:15.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c3fff700 1 --2- 172.21.15.32:0/2663222379 >> v2:172.21.15.61:3300/0 conn(0x7f59c413ed00 0x7f59c413f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:15.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c37fe700 1 -- 172.21.15.32:0/2663222379 >> v2:172.21.15.61:3300/0 conn(0x7f59c413ed00 msgr2=0x7f59c413f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:15.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c37fe700 1 --2- 172.21.15.32:0/2663222379 >> v2:172.21.15.61:3300/0 conn(0x7f59c413ed00 0x7f59c413f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c37fe700 1 -- 172.21.15.32:0/2663222379 >> v1:172.21.15.32:6789/0 conn(0x7f59c40b1a10 legacy=0x7f59c4142b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:15.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c37fe700 1 -- 172.21.15.32:0/2663222379 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c4142450 con 0x7f59c40b29f0 2020-12-05T17:41:15.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c2ffd700 1 -- 172.21.15.32:0/2663222379 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918697489 0 0) 0x7f59c4067ca0 con 0x7f59c40b1a10 2020-12-05T17:41:15.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c37fe700 1 --2- 172.21.15.32:0/2663222379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59c40b29f0 0x7f59c40677b0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7f59b4002f30 tx=0x7f59b4006dc0).ready entity=mon.0 client_cookie=b3b475304a24770b server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:15.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c2ffd700 1 -- 172.21.15.32:0/2663222379 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59b4014070 con 0x7f59c40b29f0 2020-12-05T17:41:15.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c2ffd700 1 -- 172.21.15.32:0/2663222379 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b40099e0 con 0x7f59c40b29f0 2020-12-05T17:41:15.244 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c2ffd700 1 -- 172.21.15.32:0/2663222379 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59b4009d40 con 0x7f59c40b29f0 2020-12-05T17:41:15.244 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c8a01700 1 -- 172.21.15.32:0/2663222379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59c40b29f0 msgr2=0x7f59c40677b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:15.244 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c8a01700 1 --2- 172.21.15.32:0/2663222379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59c40b29f0 0x7f59c40677b0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7f59b4002f30 tx=0x7f59b4006dc0).stop 2020-12-05T17:41:15.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c8a01700 1 -- 172.21.15.32:0/2663222379 shutdown_connections 2020-12-05T17:41:15.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c8a01700 1 --2- 172.21.15.32:0/2663222379 >> v2:172.21.15.61:3300/0 conn(0x7f59c413ed00 0x7f59c413f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c8a01700 1 --2- 172.21.15.32:0/2663222379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59c40b29f0 0x7f59c40677b0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c8a01700 1 -- 172.21.15.32:0/2663222379 >> 172.21.15.32:0/2663222379 conn(0x7f59c40b5fd0 msgr2=0x7f59c40b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:15.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c8a01700 1 -- 172.21.15.32:0/2663222379 shutdown_connections 2020-12-05T17:41:15.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.234+0000 7f59c8a01700 1 -- 172.21.15.32:0/2663222379 wait complete. 2020-12-05T17:41:15.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59c8a01700 1 Processor -- start 2020-12-05T17:41:15.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59c8a01700 1 -- start start 2020-12-05T17:41:15.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59c8a01700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59c40b1a10 0x7f59c41414a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:15.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59c8a01700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59c40b29f0 0x7f59c40631a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:15.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59c8a01700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f59c413ed00 0x7f59c4151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59c8a01700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c4063690 con 0x7f59c40b29f0 2020-12-05T17:41:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59c8a01700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c409ed10 con 0x7f59c40b1a10 2020-12-05T17:41:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59c8a01700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c4151490 con 0x7f59c413ed00 2020-12-05T17:41:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59baffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59c40b29f0 0x7f59c40631a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59baffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59c40b29f0 0x7f59c40631a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40492/0 (socket says 172.21.15.32:40492) 2020-12-05T17:41:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59baffd700 1 -- 172.21.15.32:0/4028235666 learned_addr learned my addr 172.21.15.32:0/4028235666 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59c37fe700 1 --2- 172.21.15.32:0/4028235666 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59c40b1a10 0x7f59c41414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59c3fff700 1 --2- 172.21.15.32:0/4028235666 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f59c413ed00 0x7f59c4151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59baffd700 1 -- 172.21.15.32:0/4028235666 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f59c413ed00 msgr2=0x7f59c4151af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59baffd700 1 --2- 172.21.15.32:0/4028235666 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f59c413ed00 0x7f59c4151af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59baffd700 1 -- 172.21.15.32:0/4028235666 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59c40b1a10 msgr2=0x7f59c41414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59baffd700 1 --2- 172.21.15.32:0/4028235666 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59c40b1a10 0x7f59c41414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59baffd700 1 -- 172.21.15.32:0/4028235666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c4064610 con 0x7f59c40b29f0 2020-12-05T17:41:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59c37fe700 1 --2- 172.21.15.32:0/4028235666 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59c40b1a10 0x7f59c41414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59baffd700 1 --2- 172.21.15.32:0/4028235666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59c40b29f0 0x7f59c40631a0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7f59ac000b80 tx=0x7f59ac007a30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59c17fa700 1 -- 172.21.15.32:0/4028235666 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59ac0072e0 con 0x7f59c40b29f0 2020-12-05T17:41:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59c17fa700 1 -- 172.21.15.32:0/4028235666 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ac007440 con 0x7f59c40b29f0 2020-12-05T17:41:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59c8a01700 1 -- 172.21.15.32:0/4028235666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c40648b0 con 0x7f59c40b29f0 2020-12-05T17:41:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59c17fa700 1 -- 172.21.15.32:0/4028235666 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59ac0057d0 con 0x7f59c40b29f0 2020-12-05T17:41:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.238+0000 7f59c8a01700 1 -- 172.21.15.32:0/4028235666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c4151fe0 con 0x7f59c40b29f0 2020-12-05T17:41:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.242+0000 7f59c8a01700 1 -- 172.21.15.32:0/4028235666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59c4064450 con 0x7f59c40b29f0 2020-12-05T17:41:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.242+0000 7f59c17fa700 1 -- 172.21.15.32:0/4028235666 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f59ac01a060 con 0x7f59c40b29f0 2020-12-05T17:41:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.242+0000 7f59c17fa700 1 --2- 172.21.15.32:0/4028235666 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f59a8055330 0x7f59a80577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.242+0000 7f59c17fa700 1 -- 172.21.15.32:0/4028235666 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f59ac06dce0 con 0x7f59c40b29f0 2020-12-05T17:41:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.242+0000 7f59c37fe700 1 --2- 172.21.15.32:0/4028235666 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f59a8055330 0x7f59a80577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.242+0000 7f59c37fe700 1 --2- 172.21.15.32:0/4028235666 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f59a8055330 0x7f59a80577f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7f59b401b000 tx=0x7f59b4016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.250+0000 7f59c17fa700 1 -- 172.21.15.32:0/4028235666 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f59ac043d70 con 0x7f59c40b29f0 2020-12-05T17:41:15.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.394+0000 7f59c8a01700 1 -- 172.21.15.32:0/4028235666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59c4057050 con 0x7f59c40b29f0 2020-12-05T17:41:15.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.394+0000 7f59c17fa700 1 -- 172.21.15.32:0/4028235666 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f59ac018070 con 0x7f59c40b29f0 2020-12-05T17:41:15.402 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:15.402 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":129,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:41:04.284002+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:41:01.240356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2020-12-05T17:41:02.269469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:41:15.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.398+0000 7f59c8a01700 1 -- 172.21.15.32:0/4028235666 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f59a8055330 msgr2=0x7f59a80577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:15.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.398+0000 7f59c8a01700 1 --2- 172.21.15.32:0/4028235666 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f59a8055330 0x7f59a80577f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7f59b401b000 tx=0x7f59b4016000).stop 2020-12-05T17:41:15.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.398+0000 7f59c8a01700 1 -- 172.21.15.32:0/4028235666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59c40b29f0 msgr2=0x7f59c40631a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:15.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.398+0000 7f59c8a01700 1 --2- 172.21.15.32:0/4028235666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59c40b29f0 0x7f59c40631a0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7f59ac000b80 tx=0x7f59ac007a30).stop 2020-12-05T17:41:15.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.398+0000 7f59c8a01700 1 -- 172.21.15.32:0/4028235666 shutdown_connections 2020-12-05T17:41:15.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.398+0000 7f59c8a01700 1 --2- 172.21.15.32:0/4028235666 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f59a8055330 0x7f59a80577f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.398+0000 7f59c8a01700 1 --2- 172.21.15.32:0/4028235666 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f59c413ed00 0x7f59c4151af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.398+0000 7f59c8a01700 1 --2- 172.21.15.32:0/4028235666 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59c40b1a10 0x7f59c41414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.398+0000 7f59c8a01700 1 --2- 172.21.15.32:0/4028235666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59c40b29f0 0x7f59c40631a0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.398+0000 7f59c8a01700 1 -- 172.21.15.32:0/4028235666 >> 172.21.15.32:0/4028235666 conn(0x7f59c40b5fd0 msgr2=0x7f59c40afea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:15.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.398+0000 7f59c8a01700 1 -- 172.21.15.32:0/4028235666 shutdown_connections 2020-12-05T17:41:15.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.398+0000 7f59c8a01700 1 -- 172.21.15.32:0/4028235666 wait complete. 2020-12-05T17:41:15.416 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:41:01.240356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '128', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:41:02.269469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:41:15.417 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:41:15.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.530+0000 7f09d532a700 1 Processor -- start 2020-12-05T17:41:15.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.530+0000 7f09d532a700 1 -- start start 2020-12-05T17:41:15.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09d532a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09d00b5640 0x7f09d00b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:15.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09d532a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09d00a9180 0x7f09d00a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:15.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09d532a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09d00b5a20 con 0x7f09d013f830 2020-12-05T17:41:15.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09d532a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d00abc80 con 0x7f09d00a9180 2020-12-05T17:41:15.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09d532a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d00608f0 con 0x7f09d00b5640 2020-12-05T17:41:15.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09cf7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09d00a9180 0x7f09d00a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:15.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09cf7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09d00a9180 0x7f09d00a95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40496/0 (socket says 172.21.15.32:40496) 2020-12-05T17:41:15.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09cf7fe700 1 -- 172.21.15.32:0/4001349320 learned_addr learned my addr 172.21.15.32:0/4001349320 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09d4b29700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f09d013f830 0x7f09d013fca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43148/0 (socket says 172.21.15.32:43148) 2020-12-05T17:41:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09cffff700 1 --2- 172.21.15.32:0/4001349320 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09d00b5640 0x7f09d00b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09cf7fe700 1 -- 172.21.15.32:0/4001349320 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09d00b5640 msgr2=0x7f09d00b10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09cf7fe700 1 --2- 172.21.15.32:0/4001349320 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09d00b5640 0x7f09d00b10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09cf7fe700 1 -- 172.21.15.32:0/4001349320 >> v1:172.21.15.32:6789/0 conn(0x7f09d013f830 legacy=0x7f09d013fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:15.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09cf7fe700 1 -- 172.21.15.32:0/4001349320 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d0060b90 con 0x7f09d00a9180 2020-12-05T17:41:15.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09ceffd700 1 -- 172.21.15.32:0/4001349320 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3966905950 0 0) 0x7f09d00b5a20 con 0x7f09d013f830 2020-12-05T17:41:15.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09cf7fe700 1 --2- 172.21.15.32:0/4001349320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d00a9180 0x7f09d00a95e0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 rx=0x7f09b8002f30 tx=0x7f09b8006f10).ready entity=mon.0 client_cookie=a8986e753c9886f2 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:15.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09ceffd700 1 -- 172.21.15.32:0/4001349320 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09b800de50 con 0x7f09d00a9180 2020-12-05T17:41:15.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09ceffd700 1 -- 172.21.15.32:0/4001349320 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b8010470 con 0x7f09d00a9180 2020-12-05T17:41:15.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09ceffd700 1 -- 172.21.15.32:0/4001349320 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09b80107d0 con 0x7f09d00a9180 2020-12-05T17:41:15.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09d532a700 1 -- 172.21.15.32:0/4001349320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d00a9180 msgr2=0x7f09d00a95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:15.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09d532a700 1 --2- 172.21.15.32:0/4001349320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d00a9180 0x7f09d00a95e0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 rx=0x7f09b8002f30 tx=0x7f09b8006f10).stop 2020-12-05T17:41:15.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09d532a700 1 -- 172.21.15.32:0/4001349320 shutdown_connections 2020-12-05T17:41:15.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09d532a700 1 --2- 172.21.15.32:0/4001349320 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09d00b5640 0x7f09d00b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09d532a700 1 --2- 172.21.15.32:0/4001349320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d00a9180 0x7f09d00a95e0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09d532a700 1 -- 172.21.15.32:0/4001349320 >> 172.21.15.32:0/4001349320 conn(0x7f09d0062db0 msgr2=0x7f09d00b4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09d532a700 1 -- 172.21.15.32:0/4001349320 shutdown_connections 2020-12-05T17:41:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.534+0000 7f09d532a700 1 -- 172.21.15.32:0/4001349320 wait complete. 2020-12-05T17:41:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09d532a700 1 Processor -- start 2020-12-05T17:41:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09d532a700 1 -- start start 2020-12-05T17:41:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09d532a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f09d00a9180 0x7f09d00ae580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09d532a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d00b5640 0x7f09d0138df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09d532a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09d013f830 0x7f09d01392e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09d532a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d0065bf0 con 0x7f09d00b5640 2020-12-05T17:41:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09d532a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d0065e90 con 0x7f09d00a9180 2020-12-05T17:41:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09d532a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d0066130 con 0x7f09d013f830 2020-12-05T17:41:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09cf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d00b5640 0x7f09d0138df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09cf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d00b5640 0x7f09d0138df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40504/0 (socket says 172.21.15.32:40504) 2020-12-05T17:41:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09cf7fe700 1 -- 172.21.15.32:0/4003424620 learned_addr learned my addr 172.21.15.32:0/4003424620 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09cffff700 1 --2- 172.21.15.32:0/4003424620 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f09d00a9180 0x7f09d00ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09d4b29700 1 --2- 172.21.15.32:0/4003424620 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09d013f830 0x7f09d01392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:15.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09cf7fe700 1 -- 172.21.15.32:0/4003424620 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09d013f830 msgr2=0x7f09d01392e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:15.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09cf7fe700 1 --2- 172.21.15.32:0/4003424620 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09d013f830 0x7f09d01392e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09cf7fe700 1 -- 172.21.15.32:0/4003424620 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f09d00a9180 msgr2=0x7f09d00ae580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:15.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09cf7fe700 1 --2- 172.21.15.32:0/4003424620 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f09d00a9180 0x7f09d00ae580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09cf7fe700 1 -- 172.21.15.32:0/4003424620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d0143260 con 0x7f09d00b5640 2020-12-05T17:41:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09cf7fe700 1 --2- 172.21.15.32:0/4003424620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d00b5640 0x7f09d0138df0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7f09b8002f30 tx=0x7f09b8006f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09cd7fa700 1 -- 172.21.15.32:0/4003424620 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09b8010e30 con 0x7f09d00b5640 2020-12-05T17:41:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09cd7fa700 1 -- 172.21.15.32:0/4003424620 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b8018610 con 0x7f09d00b5640 2020-12-05T17:41:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09d532a700 1 -- 172.21.15.32:0/4003424620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d01397d0 con 0x7f09d00b5640 2020-12-05T17:41:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09cd7fa700 1 -- 172.21.15.32:0/4003424620 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09b8018970 con 0x7f09d00b5640 2020-12-05T17:41:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.538+0000 7f09d532a700 1 -- 172.21.15.32:0/4003424620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d0150be0 con 0x7f09d00b5640 2020-12-05T17:41:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.542+0000 7f09cd7fa700 1 -- 172.21.15.32:0/4003424620 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f09b8021070 con 0x7f09d00b5640 2020-12-05T17:41:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.542+0000 7f09d532a700 1 -- 172.21.15.32:0/4003424620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d0068540 con 0x7f09d00b5640 2020-12-05T17:41:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.542+0000 7f09cd7fa700 1 --2- 172.21.15.32:0/4003424620 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f09b4055330 0x7f09b40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.542+0000 7f09cd7fa700 1 -- 172.21.15.32:0/4003424620 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f09b8071c00 con 0x7f09d00b5640 2020-12-05T17:41:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.542+0000 7f09cffff700 1 --2- 172.21.15.32:0/4003424620 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f09b4055330 0x7f09b40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.542+0000 7f09cffff700 1 --2- 172.21.15.32:0/4003424620 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f09b4055330 0x7f09b40577f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f09c000c000 tx=0x7f09c0008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.546+0000 7f09cd7fa700 1 -- 172.21.15.32:0/4003424620 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f09d0068540 con 0x7f09d00b5640 2020-12-05T17:41:15.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.690+0000 7f09d532a700 1 -- 172.21.15.32:0/4003424620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f09d0068540 con 0x7f09d00b5640 2020-12-05T17:41:15.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.690+0000 7f09cd7fa700 1 -- 172.21.15.32:0/4003424620 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v129) v1 ==== 110+0+10 (secure 0 0 0) 0x7f09d0068540 con 0x7f09d00b5640 2020-12-05T17:41:15.696 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:41:15.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.694+0000 7f09d532a700 1 -- 172.21.15.32:0/4003424620 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f09b4055330 msgr2=0x7f09b40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:15.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.694+0000 7f09d532a700 1 --2- 172.21.15.32:0/4003424620 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f09b4055330 0x7f09b40577f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f09c000c000 tx=0x7f09c0008040).stop 2020-12-05T17:41:15.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.694+0000 7f09d532a700 1 -- 172.21.15.32:0/4003424620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d00b5640 msgr2=0x7f09d0138df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:15.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.694+0000 7f09d532a700 1 --2- 172.21.15.32:0/4003424620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d00b5640 0x7f09d0138df0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7f09b8002f30 tx=0x7f09b8006f10).stop 2020-12-05T17:41:15.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.694+0000 7f09d532a700 1 -- 172.21.15.32:0/4003424620 shutdown_connections 2020-12-05T17:41:15.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.694+0000 7f09d532a700 1 --2- 172.21.15.32:0/4003424620 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f09b4055330 0x7f09b40577f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.694+0000 7f09d532a700 1 --2- 172.21.15.32:0/4003424620 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09d013f830 0x7f09d01392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.694+0000 7f09d532a700 1 --2- 172.21.15.32:0/4003424620 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f09d00a9180 0x7f09d00ae580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.694+0000 7f09d532a700 1 --2- 172.21.15.32:0/4003424620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d00b5640 0x7f09d0138df0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.694+0000 7f09d532a700 1 -- 172.21.15.32:0/4003424620 >> 172.21.15.32:0/4003424620 conn(0x7f09d0062db0 msgr2=0x7f09d00b3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:15.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.694+0000 7f09d532a700 1 -- 172.21.15.32:0/4003424620 shutdown_connections 2020-12-05T17:41:15.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.694+0000 7f09d532a700 1 -- 172.21.15.32:0/4003424620 wait complete. 2020-12-05T17:41:15.710 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:41:15.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.822+0000 7f967a5ec700 1 Processor -- start 2020-12-05T17:41:15.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.822+0000 7f967a5ec700 1 -- start start 2020-12-05T17:41:15.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.822+0000 7f967a5ec700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9674061d60 0x7f96740b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:15.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.822+0000 7f967a5ec700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96740b5fa0 0x7f96740b9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:15.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.822+0000 7f967a5ec700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9674062140 con 0x7f9674145ee0 2020-12-05T17:41:15.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.822+0000 7f967a5ec700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96740b66b0 con 0x7f96740b5fa0 2020-12-05T17:41:15.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.822+0000 7f967a5ec700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96740b2000 con 0x7f9674061d60 2020-12-05T17:41:15.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.822+0000 7f9678de9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96740b5fa0 0x7f96740b9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:15.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.822+0000 7f9678de9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96740b5fa0 0x7f96740b9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40510/0 (socket says 172.21.15.32:40510) 2020-12-05T17:41:15.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.826+0000 7f9678de9700 1 -- 172.21.15.32:0/2631907379 learned_addr learned my addr 172.21.15.32:0/2631907379 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:15.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.826+0000 7f96795ea700 1 --2- 172.21.15.32:0/2631907379 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9674061d60 0x7f96740b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:15.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.826+0000 7f9678de9700 1 -- 172.21.15.32:0/2631907379 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9674061d60 msgr2=0x7f96740b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:15.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.826+0000 7f9678de9700 1 --2- 172.21.15.32:0/2631907379 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9674061d60 0x7f96740b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.832 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.826+0000 7f9678de9700 1 -- 172.21.15.32:0/2631907379 >> v1:172.21.15.61:6789/0 conn(0x7f9674145ee0 legacy=0x7f967413c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:15.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.826+0000 7f9678de9700 1 -- 172.21.15.32:0/2631907379 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96740b9e20 con 0x7f96740b5fa0 2020-12-05T17:41:15.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.826+0000 7f9678de9700 1 --2- 172.21.15.32:0/2631907379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96740b5fa0 0x7f96740b9930 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 rx=0x7f9668002f30 tx=0x7f9668006e80).ready entity=mon.0 client_cookie=e9c009aad678ccb0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:15.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.826+0000 7f9667fff700 1 -- 172.21.15.32:0/2631907379 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96680097f0 con 0x7f96740b5fa0 2020-12-05T17:41:15.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.826+0000 7f9667fff700 1 -- 172.21.15.32:0/2631907379 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9668009950 con 0x7f96740b5fa0 2020-12-05T17:41:15.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.826+0000 7f9667fff700 1 -- 172.21.15.32:0/2631907379 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9668009cb0 con 0x7f96740b5fa0 2020-12-05T17:41:15.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.826+0000 7f967a5ec700 1 -- 172.21.15.32:0/2631907379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96740b5fa0 msgr2=0x7f96740b9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:15.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.826+0000 7f967a5ec700 1 --2- 172.21.15.32:0/2631907379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96740b5fa0 0x7f96740b9930 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 rx=0x7f9668002f30 tx=0x7f9668006e80).stop 2020-12-05T17:41:15.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.826+0000 7f967a5ec700 1 -- 172.21.15.32:0/2631907379 shutdown_connections 2020-12-05T17:41:15.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.826+0000 7f967a5ec700 1 --2- 172.21.15.32:0/2631907379 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9674061d60 0x7f96740b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.826+0000 7f967a5ec700 1 --2- 172.21.15.32:0/2631907379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96740b5fa0 0x7f96740b9930 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.826+0000 7f967a5ec700 1 -- 172.21.15.32:0/2631907379 >> 172.21.15.32:0/2631907379 conn(0x7f96740676a0 msgr2=0x7f967405fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:15.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.826+0000 7f967a5ec700 1 -- 172.21.15.32:0/2631907379 shutdown_connections 2020-12-05T17:41:15.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.826+0000 7f967a5ec700 1 -- 172.21.15.32:0/2631907379 wait complete. 2020-12-05T17:41:15.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f967a5ec700 1 Processor -- start 2020-12-05T17:41:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f967a5ec700 1 -- start start 2020-12-05T17:41:15.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f967a5ec700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9674061d60 0x7f967413d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:15.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f967a5ec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96740b5fa0 0x7f96740accc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:15.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f967a5ec700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9674145ee0 0x7f96741521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:15.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f967a5ec700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96740ae540 con 0x7f96740b5fa0 2020-12-05T17:41:15.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f967a5ec700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967413f860 con 0x7f9674061d60 2020-12-05T17:41:15.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f967a5ec700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9674151a70 con 0x7f9674145ee0 2020-12-05T17:41:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f9678de9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96740b5fa0 0x7f96740accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f9678de9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96740b5fa0 0x7f96740accc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40518/0 (socket says 172.21.15.32:40518) 2020-12-05T17:41:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f9678de9700 1 -- 172.21.15.32:0/3308686489 learned_addr learned my addr 172.21.15.32:0/3308686489 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f96795ea700 1 --2- 172.21.15.32:0/3308686489 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9674061d60 0x7f967413d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f9679deb700 1 --2- 172.21.15.32:0/3308686489 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9674145ee0 0x7f96741521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:15.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f9678de9700 1 -- 172.21.15.32:0/3308686489 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9674145ee0 msgr2=0x7f96741521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:15.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f9678de9700 1 --2- 172.21.15.32:0/3308686489 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9674145ee0 0x7f96741521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f9678de9700 1 -- 172.21.15.32:0/3308686489 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9674061d60 msgr2=0x7f967413d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:15.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f9678de9700 1 --2- 172.21.15.32:0/3308686489 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9674061d60 0x7f967413d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f9678de9700 1 -- 172.21.15.32:0/3308686489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9674151d10 con 0x7f96740b5fa0 2020-12-05T17:41:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f9679deb700 1 --2- 172.21.15.32:0/3308686489 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9674145ee0 0x7f96741521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f9678de9700 1 --2- 172.21.15.32:0/3308686489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96740b5fa0 0x7f96740accc0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 rx=0x7f9668002f30 tx=0x7f9668015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f96667fc700 1 -- 172.21.15.32:0/3308686489 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9668009c50 con 0x7f96740b5fa0 2020-12-05T17:41:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f96667fc700 1 -- 172.21.15.32:0/3308686489 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9668011cd0 con 0x7f96740b5fa0 2020-12-05T17:41:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f96667fc700 1 -- 172.21.15.32:0/3308686489 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f966801e650 con 0x7f96740b5fa0 2020-12-05T17:41:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f967a5ec700 1 -- 172.21.15.32:0/3308686489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f967413d7e0 con 0x7f96740b5fa0 2020-12-05T17:41:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.830+0000 7f967a5ec700 1 -- 172.21.15.32:0/3308686489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96741497c0 con 0x7f96740b5fa0 2020-12-05T17:41:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.834+0000 7f967a5ec700 1 -- 172.21.15.32:0/3308686489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f967413f6d0 con 0x7f96740b5fa0 2020-12-05T17:41:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.834+0000 7f96667fc700 1 -- 172.21.15.32:0/3308686489 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f966801a040 con 0x7f96740b5fa0 2020-12-05T17:41:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.834+0000 7f96667fc700 1 --2- 172.21.15.32:0/3308686489 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f965c0554e0 0x7f965c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.834+0000 7f96795ea700 1 --2- 172.21.15.32:0/3308686489 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f965c0554e0 0x7f965c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.834+0000 7f96667fc700 1 -- 172.21.15.32:0/3308686489 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f966806f160 con 0x7f96740b5fa0 2020-12-05T17:41:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.834+0000 7f96795ea700 1 --2- 172.21.15.32:0/3308686489 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f965c0554e0 0x7f965c0579a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f967000c000 tx=0x7f9670008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.838+0000 7f96667fc700 1 -- 172.21.15.32:0/3308686489 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f96680451f0 con 0x7f96740b5fa0 2020-12-05T17:41:15.952 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:41:15.955 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:41:15.957 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:41:15.959 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:41:15.961 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:41:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.986+0000 7f967a5ec700 1 -- 172.21.15.32:0/3308686489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9674151ea0 con 0x7f96740b5fa0 2020-12-05T17:41:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.986+0000 7f96667fc700 1 -- 172.21.15.32:0/3308686489 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v129) v1 ==== 92+0+10 (secure 0 0 0) 0x7f96680230a0 con 0x7f96740b5fa0 2020-12-05T17:41:15.990 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:41:15.992 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.986+0000 7f967a5ec700 1 -- 172.21.15.32:0/3308686489 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f965c0554e0 msgr2=0x7f965c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:15.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.986+0000 7f967a5ec700 1 --2- 172.21.15.32:0/3308686489 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f965c0554e0 0x7f965c0579a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f967000c000 tx=0x7f9670008040).stop 2020-12-05T17:41:15.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.986+0000 7f967a5ec700 1 -- 172.21.15.32:0/3308686489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96740b5fa0 msgr2=0x7f96740accc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:15.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.986+0000 7f967a5ec700 1 --2- 172.21.15.32:0/3308686489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96740b5fa0 0x7f96740accc0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 rx=0x7f9668002f30 tx=0x7f9668015000).stop 2020-12-05T17:41:15.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.986+0000 7f967a5ec700 1 -- 172.21.15.32:0/3308686489 shutdown_connections 2020-12-05T17:41:15.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.986+0000 7f967a5ec700 1 --2- 172.21.15.32:0/3308686489 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f965c0554e0 0x7f965c0579a0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.986+0000 7f967a5ec700 1 --2- 172.21.15.32:0/3308686489 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9674145ee0 0x7f96741521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.986+0000 7f967a5ec700 1 --2- 172.21.15.32:0/3308686489 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9674061d60 0x7f967413d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.986+0000 7f967a5ec700 1 --2- 172.21.15.32:0/3308686489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96740b5fa0 0x7f96740accc0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:15.994 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.986+0000 7f967a5ec700 1 -- 172.21.15.32:0/3308686489 >> 172.21.15.32:0/3308686489 conn(0x7f96740676a0 msgr2=0x7f96740b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:15.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.986+0000 7f967a5ec700 1 -- 172.21.15.32:0/3308686489 shutdown_connections 2020-12-05T17:41:15.995 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:15.986+0000 7f967a5ec700 1 -- 172.21.15.32:0/3308686489 wait complete. 2020-12-05T17:41:16.004 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:41:16.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.162+0000 7fd368a3d700 1 Processor -- start 2020-12-05T17:41:16.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.162+0000 7fd368a3d700 1 -- start start 2020-12-05T17:41:16.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd368a3d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd3640ab960 0x7fd36413e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:16.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd368a3d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3640af590 0x7fd364142410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:16.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd368a3d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3640ab490 con 0x7fd3640aa610 2020-12-05T17:41:16.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd368a3d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3640abd40 con 0x7fd3640af590 2020-12-05T17:41:16.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd368a3d700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3640aaab0 con 0x7fd3640ab960 2020-12-05T17:41:16.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd362ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd3640aa610 0x7fd36413e6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43172/0 (socket says 172.21.15.32:43172) 2020-12-05T17:41:16.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd362ffd700 1 -- 172.21.15.32:0/3870915595 learned_addr learned my addr 172.21.15.32:0/3870915595 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:16.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd363fff700 1 --2- 172.21.15.32:0/3870915595 >> v2:172.21.15.32:3300/0 conn(0x7fd3640af590 0x7fd364142410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:16.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd3637fe700 1 --2- 172.21.15.32:0/3870915595 >> v2:172.21.15.61:3300/0 conn(0x7fd3640ab960 0x7fd36413e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:16.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd3627fc700 1 -- 172.21.15.32:0/3870915595 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 639351055 0 0) 0x7fd3640ab490 con 0x7fd3640aa610 2020-12-05T17:41:16.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd3627fc700 1 -- 172.21.15.32:0/3870915595 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd344003230 con 0x7fd3640aa610 2020-12-05T17:41:16.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd363fff700 1 -- 172.21.15.32:0/3870915595 >> v2:172.21.15.61:3300/0 conn(0x7fd3640ab960 msgr2=0x7fd36413e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:16.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd363fff700 1 --2- 172.21.15.32:0/3870915595 >> v2:172.21.15.61:3300/0 conn(0x7fd3640ab960 0x7fd36413e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd363fff700 1 -- 172.21.15.32:0/3870915595 >> v1:172.21.15.32:6789/0 conn(0x7fd3640aa610 legacy=0x7fd36413e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:16.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd363fff700 1 -- 172.21.15.32:0/3870915595 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3640aad50 con 0x7fd3640af590 2020-12-05T17:41:16.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd3637fe700 1 --2- 172.21.15.32:0/3870915595 >> v2:172.21.15.61:3300/0 conn(0x7fd3640ab960 0x7fd36413e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:16.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd363fff700 1 --2- 172.21.15.32:0/3870915595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3640af590 0x7fd364142410 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7fd35c0032b0 tx=0x7fd35c0091a0).ready entity=mon.0 client_cookie=d934d52ed2601be0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:16.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd3627fc700 1 -- 172.21.15.32:0/3870915595 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd35c00b940 con 0x7fd3640af590 2020-12-05T17:41:16.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd3627fc700 1 -- 172.21.15.32:0/3870915595 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35c00baa0 con 0x7fd3640af590 2020-12-05T17:41:16.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd3627fc700 1 -- 172.21.15.32:0/3870915595 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd35c00bc10 con 0x7fd3640af590 2020-12-05T17:41:16.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd368a3d700 1 -- 172.21.15.32:0/3870915595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3640af590 msgr2=0x7fd364142410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:16.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd368a3d700 1 --2- 172.21.15.32:0/3870915595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3640af590 0x7fd364142410 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7fd35c0032b0 tx=0x7fd35c0091a0).stop 2020-12-05T17:41:16.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd368a3d700 1 -- 172.21.15.32:0/3870915595 shutdown_connections 2020-12-05T17:41:16.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd368a3d700 1 --2- 172.21.15.32:0/3870915595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3640af590 0x7fd364142410 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd368a3d700 1 --2- 172.21.15.32:0/3870915595 >> v2:172.21.15.61:3300/0 conn(0x7fd3640ab960 0x7fd36413e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd368a3d700 1 -- 172.21.15.32:0/3870915595 >> 172.21.15.32:0/3870915595 conn(0x7fd364062db0 msgr2=0x7fd3640b4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:16.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd368a3d700 1 -- 172.21.15.32:0/3870915595 shutdown_connections 2020-12-05T17:41:16.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.166+0000 7fd368a3d700 1 -- 172.21.15.32:0/3870915595 wait complete. 2020-12-05T17:41:16.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd368a3d700 1 Processor -- start 2020-12-05T17:41:16.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd368a3d700 1 -- start start 2020-12-05T17:41:16.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd368a3d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3640aa610 0x7fd364066d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:16.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd368a3d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd3640ab960 0x7fd36413d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:16.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd368a3d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd3640af590 0x7fd364063860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:16.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd368a3d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640aed70 con 0x7fd3640aa610 2020-12-05T17:41:16.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd368a3d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640af010 con 0x7fd3640af590 2020-12-05T17:41:16.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd368a3d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36413db80 con 0x7fd3640ab960 2020-12-05T17:41:16.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd3637fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3640aa610 0x7fd364066d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:16.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd3637fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3640aa610 0x7fd364066d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40530/0 (socket says 172.21.15.32:40530) 2020-12-05T17:41:16.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd3637fe700 1 -- 172.21.15.32:0/129958186 learned_addr learned my addr 172.21.15.32:0/129958186 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:16.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd363fff700 1 --2- 172.21.15.32:0/129958186 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd3640af590 0x7fd364063860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:16.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd362ffd700 1 --2- 172.21.15.32:0/129958186 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd3640ab960 0x7fd36413d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:16.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd3637fe700 1 -- 172.21.15.32:0/129958186 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd3640ab960 msgr2=0x7fd36413d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:16.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd3637fe700 1 --2- 172.21.15.32:0/129958186 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd3640ab960 0x7fd36413d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd3637fe700 1 -- 172.21.15.32:0/129958186 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd3640af590 msgr2=0x7fd364063860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:16.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd3637fe700 1 --2- 172.21.15.32:0/129958186 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd3640af590 0x7fd364063860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd3637fe700 1 -- 172.21.15.32:0/129958186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd364131f10 con 0x7fd3640aa610 2020-12-05T17:41:16.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd362ffd700 1 --2- 172.21.15.32:0/129958186 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd3640ab960 0x7fd36413d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:16.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd3637fe700 1 --2- 172.21.15.32:0/129958186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3640aa610 0x7fd364066d70 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7fd34c00c000 tx=0x7fd34c006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:16.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd360ff9700 1 -- 172.21.15.32:0/129958186 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd34c00cbf0 con 0x7fd3640aa610 2020-12-05T17:41:16.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd360ff9700 1 -- 172.21.15.32:0/129958186 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34c00cd50 con 0x7fd3640aa610 2020-12-05T17:41:16.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd368a3d700 1 -- 172.21.15.32:0/129958186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3640ae1e0 con 0x7fd3640aa610 2020-12-05T17:41:16.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd360ff9700 1 -- 172.21.15.32:0/129958186 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd34c0125e0 con 0x7fd3640aa610 2020-12-05T17:41:16.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd368a3d700 1 -- 172.21.15.32:0/129958186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3640ae7c0 con 0x7fd3640aa610 2020-12-05T17:41:16.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.170+0000 7fd368a3d700 1 -- 172.21.15.32:0/129958186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd364064090 con 0x7fd3640aa610 2020-12-05T17:41:16.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.174+0000 7fd360ff9700 1 -- 172.21.15.32:0/129958186 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd34c012740 con 0x7fd3640aa610 2020-12-05T17:41:16.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.174+0000 7fd360ff9700 1 --2- 172.21.15.32:0/129958186 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd3480554e0 0x7fd3480579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:16.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.174+0000 7fd360ff9700 1 -- 172.21.15.32:0/129958186 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd34c06e800 con 0x7fd3640aa610 2020-12-05T17:41:16.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.174+0000 7fd362ffd700 1 --2- 172.21.15.32:0/129958186 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd3480554e0 0x7fd3480579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:16.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.174+0000 7fd362ffd700 1 --2- 172.21.15.32:0/129958186 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd3480554e0 0x7fd3480579a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7fd35400b000 tx=0x7fd354009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:16.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.178+0000 7fd360ff9700 1 -- 172.21.15.32:0/129958186 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd34c044900 con 0x7fd3640aa610 2020-12-05T17:41:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.322+0000 7fd368a3d700 1 -- 172.21.15.32:0/129958186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd3640ae3d0 con 0x7fd3640aa610 2020-12-05T17:41:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.322+0000 7fd360ff9700 1 -- 172.21.15.32:0/129958186 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v129) v1 ==== 104+0+10 (secure 0 0 0) 0x7fd34c047f20 con 0x7fd3640aa610 2020-12-05T17:41:16.329 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:41:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.326+0000 7fd368a3d700 1 -- 172.21.15.32:0/129958186 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd3480554e0 msgr2=0x7fd3480579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.326+0000 7fd368a3d700 1 --2- 172.21.15.32:0/129958186 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd3480554e0 0x7fd3480579a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7fd35400b000 tx=0x7fd354009040).stop 2020-12-05T17:41:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.326+0000 7fd368a3d700 1 -- 172.21.15.32:0/129958186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3640aa610 msgr2=0x7fd364066d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.326+0000 7fd368a3d700 1 --2- 172.21.15.32:0/129958186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3640aa610 0x7fd364066d70 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7fd34c00c000 tx=0x7fd34c006dc0).stop 2020-12-05T17:41:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.326+0000 7fd368a3d700 1 -- 172.21.15.32:0/129958186 shutdown_connections 2020-12-05T17:41:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.326+0000 7fd368a3d700 1 --2- 172.21.15.32:0/129958186 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd3640af590 0x7fd364063860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.326+0000 7fd368a3d700 1 --2- 172.21.15.32:0/129958186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3640aa610 0x7fd364066d70 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.326+0000 7fd368a3d700 1 --2- 172.21.15.32:0/129958186 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd3480554e0 0x7fd3480579a0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.326+0000 7fd368a3d700 1 --2- 172.21.15.32:0/129958186 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd3640ab960 0x7fd36413d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.326+0000 7fd368a3d700 1 -- 172.21.15.32:0/129958186 >> 172.21.15.32:0/129958186 conn(0x7fd364062db0 msgr2=0x7fd3640afac0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.326+0000 7fd368a3d700 1 -- 172.21.15.32:0/129958186 shutdown_connections 2020-12-05T17:41:16.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.326+0000 7fd368a3d700 1 -- 172.21.15.32:0/129958186 wait complete. 2020-12-05T17:41:16.346 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:41:16.463 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.458+0000 7f85cdee3700 1 Processor -- start 2020-12-05T17:41:16.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.458+0000 7f85cdee3700 1 -- start start 2020-12-05T17:41:16.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.458+0000 7f85cdee3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85c80b9330 0x7f85c813c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:16.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.458+0000 7f85cdee3700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f85c80b17c0 0x7f85c813cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:16.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.458+0000 7f85cdee3700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c81463d0 con 0x7f85c80b3c10 2020-12-05T17:41:16.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.458+0000 7f85cdee3700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c80b5ab0 con 0x7f85c80b17c0 2020-12-05T17:41:16.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.458+0000 7f85cdee3700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8146670 con 0x7f85c80b9330 2020-12-05T17:41:16.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.458+0000 7f85c7fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f85c80b17c0 0x7f85c813cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:16.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.458+0000 7f85c77fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85c80b9330 0x7f85c813c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:16.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.458+0000 7f85beffd700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f85c80b3c10 0x7f85c8142a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:55826/0 (socket says 172.21.15.32:55826) 2020-12-05T17:41:16.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.458+0000 7f85beffd700 1 -- 172.21.15.32:0/3488970157 learned_addr learned my addr 172.21.15.32:0/3488970157 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:16.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.462+0000 7f85c7fff700 1 -- 172.21.15.32:0/3488970157 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85c80b9330 msgr2=0x7f85c813c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:16.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.462+0000 7f85c7fff700 1 --2- 172.21.15.32:0/3488970157 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85c80b9330 0x7f85c813c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.462+0000 7f85c7fff700 1 -- 172.21.15.32:0/3488970157 >> v1:172.21.15.61:6789/0 conn(0x7f85c80b3c10 legacy=0x7f85c8142a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:16.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.462+0000 7f85c7fff700 1 -- 172.21.15.32:0/3488970157 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c8146910 con 0x7f85c80b17c0 2020-12-05T17:41:16.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.462+0000 7f85c6ffd700 1 -- 172.21.15.32:0/3488970157 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 671180396 0 0) 0x7f85c81463d0 con 0x7f85c80b3c10 2020-12-05T17:41:16.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.462+0000 7f85c7fff700 1 --2- 172.21.15.32:0/3488970157 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85c80b17c0 0x7f85c813cb80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f85c00032b0 tx=0x7f85c00091a0).ready entity=mon.1 client_cookie=9c4633379882e208 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:16.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.462+0000 7f85c6ffd700 1 -- 172.21.15.32:0/3488970157 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85c000b9a0 con 0x7f85c80b17c0 2020-12-05T17:41:16.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.462+0000 7f85c6ffd700 1 -- 172.21.15.32:0/3488970157 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c000bb00 con 0x7f85c80b17c0 2020-12-05T17:41:16.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.462+0000 7f85c6ffd700 1 -- 172.21.15.32:0/3488970157 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85c000bc70 con 0x7f85c80b17c0 2020-12-05T17:41:16.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.462+0000 7f85cdee3700 1 -- 172.21.15.32:0/3488970157 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85c80b17c0 msgr2=0x7f85c813cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:16.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.462+0000 7f85cdee3700 1 --2- 172.21.15.32:0/3488970157 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85c80b17c0 0x7f85c813cb80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f85c00032b0 tx=0x7f85c00091a0).stop 2020-12-05T17:41:16.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.462+0000 7f85cdee3700 1 -- 172.21.15.32:0/3488970157 shutdown_connections 2020-12-05T17:41:16.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.462+0000 7f85cdee3700 1 --2- 172.21.15.32:0/3488970157 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85c80b9330 0x7f85c813c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.462+0000 7f85cdee3700 1 --2- 172.21.15.32:0/3488970157 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85c80b17c0 0x7f85c813cb80 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.462+0000 7f85cdee3700 1 -- 172.21.15.32:0/3488970157 >> 172.21.15.32:0/3488970157 conn(0x7f85c805fea0 msgr2=0x7f85c8061be0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:16.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.462+0000 7f85cdee3700 1 -- 172.21.15.32:0/3488970157 shutdown_connections 2020-12-05T17:41:16.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.462+0000 7f85cdee3700 1 -- 172.21.15.32:0/3488970157 wait complete. 2020-12-05T17:41:16.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.462+0000 7f85cdee3700 1 Processor -- start 2020-12-05T17:41:16.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.462+0000 7f85cdee3700 1 -- start start 2020-12-05T17:41:16.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.466+0000 7f85cdee3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85c80b17c0 0x7f85c80af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:16.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.466+0000 7f85cdee3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85c80b3c10 0x7f85c80b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:16.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.466+0000 7f85cdee3700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85c80b9330 0x7f85c8137fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:16.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.466+0000 7f85cdee3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c80ad720 con 0x7f85c80b17c0 2020-12-05T17:41:16.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.466+0000 7f85cdee3700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c81529d0 con 0x7f85c80b9330 2020-12-05T17:41:16.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.466+0000 7f85cdee3700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8152730 con 0x7f85c80b3c10 2020-12-05T17:41:16.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.466+0000 7f85c77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85c80b17c0 0x7f85c80af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:16.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.466+0000 7f85c77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85c80b17c0 0x7f85c80af7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40544/0 (socket says 172.21.15.32:40544) 2020-12-05T17:41:16.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.466+0000 7f85c77fe700 1 -- 172.21.15.32:0/734492454 learned_addr learned my addr 172.21.15.32:0/734492454 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:16.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.466+0000 7f85beffd700 1 --2- 172.21.15.32:0/734492454 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85c80b3c10 0x7f85c80b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:16.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.466+0000 7f85c7fff700 1 --2- 172.21.15.32:0/734492454 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85c80b9330 0x7f85c8137fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:16.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.466+0000 7f85c77fe700 1 -- 172.21.15.32:0/734492454 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85c80b3c10 msgr2=0x7f85c80b7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:16.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.466+0000 7f85c77fe700 1 --2- 172.21.15.32:0/734492454 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85c80b3c10 0x7f85c80b7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.466+0000 7f85c77fe700 1 -- 172.21.15.32:0/734492454 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85c80b9330 msgr2=0x7f85c8137fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:16.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.466+0000 7f85c77fe700 1 --2- 172.21.15.32:0/734492454 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85c80b9330 0x7f85c8137fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.466+0000 7f85c77fe700 1 -- 172.21.15.32:0/734492454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c8140760 con 0x7f85c80b17c0 2020-12-05T17:41:16.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.466+0000 7f85c77fe700 1 --2- 172.21.15.32:0/734492454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85c80b17c0 0x7f85c80af7a0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0x7f85b800c000 tx=0x7f85b8006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:16.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.466+0000 7f85c57fa700 1 -- 172.21.15.32:0/734492454 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85b800ca10 con 0x7f85c80b17c0 2020-12-05T17:41:16.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.466+0000 7f85c57fa700 1 -- 172.21.15.32:0/734492454 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b800cb70 con 0x7f85c80b17c0 2020-12-05T17:41:16.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.466+0000 7f85cdee3700 1 -- 172.21.15.32:0/734492454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c813c5a0 con 0x7f85c80b17c0 2020-12-05T17:41:16.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.466+0000 7f85c57fa700 1 -- 172.21.15.32:0/734492454 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85b800be50 con 0x7f85c80b17c0 2020-12-05T17:41:16.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.466+0000 7f85cdee3700 1 -- 172.21.15.32:0/734492454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c8152290 con 0x7f85c80b17c0 2020-12-05T17:41:16.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.466+0000 7f85cdee3700 1 -- 172.21.15.32:0/734492454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c8152540 con 0x7f85c80b17c0 2020-12-05T17:41:16.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.470+0000 7f85c57fa700 1 -- 172.21.15.32:0/734492454 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f85b801b400 con 0x7f85c80b17c0 2020-12-05T17:41:16.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.470+0000 7f85c57fa700 1 --2- 172.21.15.32:0/734492454 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f85ac0553b0 0x7f85ac057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:16.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.470+0000 7f85c57fa700 1 -- 172.21.15.32:0/734492454 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f85b806e150 con 0x7f85c80b17c0 2020-12-05T17:41:16.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.470+0000 7f85beffd700 1 --2- 172.21.15.32:0/734492454 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f85ac0553b0 0x7f85ac057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:16.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.470+0000 7f85beffd700 1 --2- 172.21.15.32:0/734492454 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f85ac0553b0 0x7f85ac057870 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f85b000d000 tx=0x7f85b00079d0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:16.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.474+0000 7f85c57fa700 1 -- 172.21.15.32:0/734492454 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f85b8044290 con 0x7f85c80b17c0 2020-12-05T17:41:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.618+0000 7f85cdee3700 1 -- 172.21.15.32:0/734492454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f85c81408f0 con 0x7f85c80b17c0 2020-12-05T17:41:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.618+0000 7f85c57fa700 1 -- 172.21.15.32:0/734492454 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v129) v1 ==== 100+0+10 (secure 0 0 0) 0x7f85b80478b0 con 0x7f85c80b17c0 2020-12-05T17:41:16.622 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:41:16.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.618+0000 7f85cdee3700 1 -- 172.21.15.32:0/734492454 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f85ac0553b0 msgr2=0x7f85ac057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:16.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.618+0000 7f85cdee3700 1 --2- 172.21.15.32:0/734492454 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f85ac0553b0 0x7f85ac057870 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f85b000d000 tx=0x7f85b00079d0).stop 2020-12-05T17:41:16.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.618+0000 7f85cdee3700 1 -- 172.21.15.32:0/734492454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85c80b17c0 msgr2=0x7f85c80af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:16.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.618+0000 7f85cdee3700 1 --2- 172.21.15.32:0/734492454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85c80b17c0 0x7f85c80af7a0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0x7f85b800c000 tx=0x7f85b8006dc0).stop 2020-12-05T17:41:16.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.618+0000 7f85cdee3700 1 -- 172.21.15.32:0/734492454 shutdown_connections 2020-12-05T17:41:16.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.618+0000 7f85cdee3700 1 --2- 172.21.15.32:0/734492454 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85c80b9330 0x7f85c8137fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.618+0000 7f85cdee3700 1 --2- 172.21.15.32:0/734492454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85c80b17c0 0x7f85c80af7a0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.618+0000 7f85cdee3700 1 --2- 172.21.15.32:0/734492454 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f85ac0553b0 0x7f85ac057870 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.618+0000 7f85cdee3700 1 --2- 172.21.15.32:0/734492454 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85c80b3c10 0x7f85c80b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.618+0000 7f85cdee3700 1 -- 172.21.15.32:0/734492454 >> 172.21.15.32:0/734492454 conn(0x7f85c805fea0 msgr2=0x7f85c813b830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:16.626 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.618+0000 7f85cdee3700 1 -- 172.21.15.32:0/734492454 shutdown_connections 2020-12-05T17:41:16.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.618+0000 7f85cdee3700 1 -- 172.21.15.32:0/734492454 wait complete. 2020-12-05T17:41:16.636 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:41:16.637 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:41:16.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff48f8ea700 1 Processor -- start 2020-12-05T17:41:16.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff48f8ea700 1 -- start start 2020-12-05T17:41:16.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff48f8ea700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff48805f5f0 0x7ff48814ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff48f8ea700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff4880b0250 0x7ff488146280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff48f8ea700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48814b370 con 0x7ff4880b5860 2020-12-05T17:41:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff48f8ea700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff48805ffe0 con 0x7ff4880b0250 2020-12-05T17:41:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff48f8ea700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4880b5c40 con 0x7ff48805f5f0 2020-12-05T17:41:16.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff48de87700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff4880b0250 0x7ff488146280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:16.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff48d686700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7ff4880b5860 0x7ff48814a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:55840/0 (socket says 172.21.15.32:55840) 2020-12-05T17:41:16.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff48d686700 1 -- 172.21.15.32:0/2098641150 learned_addr learned my addr 172.21.15.32:0/2098641150 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:16.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff48ce85700 1 --2- 172.21.15.32:0/2098641150 >> v2:172.21.15.61:3300/0 conn(0x7ff48805f5f0 0x7ff48814ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:16.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff47ffff700 1 -- 172.21.15.32:0/2098641150 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 242999533 0 0) 0x7ff48814b370 con 0x7ff4880b5860 2020-12-05T17:41:16.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff47ffff700 1 -- 172.21.15.32:0/2098641150 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff46c003230 con 0x7ff4880b5860 2020-12-05T17:41:16.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff48ce85700 1 -- 172.21.15.32:0/2098641150 >> v2:172.21.15.32:3300/0 conn(0x7ff4880b0250 msgr2=0x7ff488146280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:16.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff48ce85700 1 --2- 172.21.15.32:0/2098641150 >> v2:172.21.15.32:3300/0 conn(0x7ff4880b0250 0x7ff488146280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff48ce85700 1 -- 172.21.15.32:0/2098641150 >> v1:172.21.15.61:6789/0 conn(0x7ff4880b5860 legacy=0x7ff48814a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:16.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff48ce85700 1 -- 172.21.15.32:0/2098641150 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48805fc90 con 0x7ff48805f5f0 2020-12-05T17:41:16.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff48de87700 1 --2- 172.21.15.32:0/2098641150 >> v2:172.21.15.32:3300/0 conn(0x7ff4880b0250 0x7ff488146280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:16.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff48ce85700 1 --2- 172.21.15.32:0/2098641150 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff48805f5f0 0x7ff48814ae80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7ff478002f30 tx=0x7ff478006dc0).ready entity=mon.1 client_cookie=c580e7b48dd7b2c1 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:16.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff47ffff700 1 -- 172.21.15.32:0/2098641150 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4780096f0 con 0x7ff48805f5f0 2020-12-05T17:41:16.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff47ffff700 1 -- 172.21.15.32:0/2098641150 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff478009850 con 0x7ff48805f5f0 2020-12-05T17:41:16.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff48f8ea700 1 -- 172.21.15.32:0/2098641150 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff48805f5f0 msgr2=0x7ff48814ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:16.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff47ffff700 1 -- 172.21.15.32:0/2098641150 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff478009bb0 con 0x7ff48805f5f0 2020-12-05T17:41:16.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.746+0000 7ff48f8ea700 1 --2- 172.21.15.32:0/2098641150 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff48805f5f0 0x7ff48814ae80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7ff478002f30 tx=0x7ff478006dc0).stop 2020-12-05T17:41:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.750+0000 7ff48f8ea700 1 -- 172.21.15.32:0/2098641150 shutdown_connections 2020-12-05T17:41:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.750+0000 7ff48f8ea700 1 --2- 172.21.15.32:0/2098641150 >> v2:172.21.15.32:3300/0 conn(0x7ff4880b0250 0x7ff488146280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.750+0000 7ff48f8ea700 1 --2- 172.21.15.32:0/2098641150 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff48805f5f0 0x7ff48814ae80 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.750+0000 7ff48f8ea700 1 -- 172.21.15.32:0/2098641150 >> 172.21.15.32:0/2098641150 conn(0x7ff4880630c0 msgr2=0x7ff4880b4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.750+0000 7ff48f8ea700 1 -- 172.21.15.32:0/2098641150 shutdown_connections 2020-12-05T17:41:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.750+0000 7ff48f8ea700 1 -- 172.21.15.32:0/2098641150 wait complete. 2020-12-05T17:41:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.750+0000 7ff48f8ea700 1 Processor -- start 2020-12-05T17:41:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.750+0000 7ff48f8ea700 1 -- start start 2020-12-05T17:41:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.750+0000 7ff48f8ea700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff48805f5f0 0x7ff48814df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.750+0000 7ff48f8ea700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff4880b0250 0x7ff488140ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:16.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.750+0000 7ff48f8ea700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4880b5860 0x7ff4881413c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:16.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.750+0000 7ff48f8ea700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff488067480 con 0x7ff4880b5860 2020-12-05T17:41:16.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.750+0000 7ff48f8ea700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4880ae130 con 0x7ff48805f5f0 2020-12-05T17:41:16.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.750+0000 7ff48f8ea700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4880639c0 con 0x7ff4880b0250 2020-12-05T17:41:16.762 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.750+0000 7ff48de87700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4880b5860 0x7ff4881413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:16.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.750+0000 7ff48de87700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4880b5860 0x7ff4881413c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40562/0 (socket says 172.21.15.32:40562) 2020-12-05T17:41:16.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.750+0000 7ff48de87700 1 -- 172.21.15.32:0/2208188294 learned_addr learned my addr 172.21.15.32:0/2208188294 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:16.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.750+0000 7ff48ce85700 1 --2- 172.21.15.32:0/2208188294 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff4880b0250 0x7ff488140ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:16.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.750+0000 7ff48d686700 1 --2- 172.21.15.32:0/2208188294 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff48805f5f0 0x7ff48814df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:16.763 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.754+0000 7ff48de87700 1 -- 172.21.15.32:0/2208188294 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff4880b0250 msgr2=0x7ff488140ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:16.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.754+0000 7ff48de87700 1 --2- 172.21.15.32:0/2208188294 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff4880b0250 0x7ff488140ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.754+0000 7ff48de87700 1 -- 172.21.15.32:0/2208188294 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff48805f5f0 msgr2=0x7ff48814df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:16.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.754+0000 7ff48de87700 1 --2- 172.21.15.32:0/2208188294 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff48805f5f0 0x7ff48814df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.754+0000 7ff48de87700 1 -- 172.21.15.32:0/2208188294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff488063c60 con 0x7ff4880b5860 2020-12-05T17:41:16.764 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.754+0000 7ff48de87700 1 --2- 172.21.15.32:0/2208188294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4880b5860 0x7ff4881413c0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7ff48000c000 tx=0x7ff480006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.754+0000 7ff47e7fc700 1 -- 172.21.15.32:0/2208188294 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff48000ca80 con 0x7ff4880b5860 2020-12-05T17:41:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.754+0000 7ff48f8ea700 1 -- 172.21.15.32:0/2208188294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff488063f00 con 0x7ff4880b5860 2020-12-05T17:41:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.754+0000 7ff47e7fc700 1 -- 172.21.15.32:0/2208188294 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48000cbe0 con 0x7ff4880b5860 2020-12-05T17:41:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.754+0000 7ff47e7fc700 1 -- 172.21.15.32:0/2208188294 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4800123f0 con 0x7ff4880b5860 2020-12-05T17:41:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.754+0000 7ff48f8ea700 1 -- 172.21.15.32:0/2208188294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff488064350 con 0x7ff4880b5860 2020-12-05T17:41:16.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.754+0000 7ff48f8ea700 1 -- 172.21.15.32:0/2208188294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff488057050 con 0x7ff4880b5860 2020-12-05T17:41:16.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.754+0000 7ff47e7fc700 1 -- 172.21.15.32:0/2208188294 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff4800125a0 con 0x7ff4880b5860 2020-12-05T17:41:16.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.758+0000 7ff47e7fc700 1 --2- 172.21.15.32:0/2208188294 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff470055330 0x7ff4700577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:16.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.758+0000 7ff47e7fc700 1 -- 172.21.15.32:0/2208188294 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff48006e720 con 0x7ff4880b5860 2020-12-05T17:41:16.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.758+0000 7ff48d686700 1 --2- 172.21.15.32:0/2208188294 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff470055330 0x7ff4700577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:16.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.758+0000 7ff48d686700 1 --2- 172.21.15.32:0/2208188294 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff470055330 0x7ff4700577f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7ff48400d000 tx=0x7ff484009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:16.768 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.762+0000 7ff47e7fc700 1 -- 172.21.15.32:0/2208188294 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff4800447c0 con 0x7ff4880b5860 2020-12-05T17:41:16.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.946+0000 7ff48f8ea700 1 -- 172.21.15.32:0/2208188294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff488064090 con 0x7ff4880b5860 2020-12-05T17:41:16.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.946+0000 7ff47e7fc700 1 -- 172.21.15.32:0/2208188294 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+2142 (secure 0 0 0) 0x7ff480047de0 con 0x7ff4880b5860 2020-12-05T17:41:16.953 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:16.953 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":62,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6830,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/3985915039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3985915039},{"type":"v1","addr":"172.21.15.61:6833","nonce":3985915039}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":60}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:41:03.280787+0000","modified":"2020-12-05T17:41:04.283186+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7938":{"gid":7938,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1585653076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1585653076},{"type":"v1","addr":"172.21.15.32:6835","nonce":1585653076}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2020-12-05T17:41:16.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.950+0000 7ff48f8ea700 1 -- 172.21.15.32:0/2208188294 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff470055330 msgr2=0x7ff4700577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:16.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.950+0000 7ff48f8ea700 1 --2- 172.21.15.32:0/2208188294 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff470055330 0x7ff4700577f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7ff48400d000 tx=0x7ff484009040).stop 2020-12-05T17:41:16.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.950+0000 7ff48f8ea700 1 -- 172.21.15.32:0/2208188294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4880b5860 msgr2=0x7ff4881413c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.950+0000 7ff48f8ea700 1 --2- 172.21.15.32:0/2208188294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4880b5860 0x7ff4881413c0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7ff48000c000 tx=0x7ff480006dc0).stop 2020-12-05T17:41:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.950+0000 7ff48f8ea700 1 -- 172.21.15.32:0/2208188294 shutdown_connections 2020-12-05T17:41:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.950+0000 7ff48f8ea700 1 --2- 172.21.15.32:0/2208188294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4880b5860 0x7ff4881413c0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.950+0000 7ff48f8ea700 1 --2- 172.21.15.32:0/2208188294 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff48805f5f0 0x7ff48814df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.950+0000 7ff48f8ea700 1 --2- 172.21.15.32:0/2208188294 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff470055330 0x7ff4700577f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.950+0000 7ff48f8ea700 1 --2- 172.21.15.32:0/2208188294 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff4880b0250 0x7ff488140ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.950+0000 7ff48f8ea700 1 -- 172.21.15.32:0/2208188294 >> 172.21.15.32:0/2208188294 conn(0x7ff4880630c0 msgr2=0x7ff48814a690 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.950+0000 7ff48f8ea700 1 -- 172.21.15.32:0/2208188294 shutdown_connections 2020-12-05T17:41:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:16.950+0000 7ff48f8ea700 1 -- 172.21.15.32:0/2208188294 wait complete. 2020-12-05T17:41:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 62 2020-12-05T17:41:17.169 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:41:03.280787+0000', 'modified': '2020-12-05T17:41:04.283186+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7938}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7938': {'gid': 7938, 'name': 'a', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/1585653076', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 1585653076}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 1585653076}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:41:17.169 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:41:17.170 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:41:17.300 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":7938,"want_state":"up:active","state":"up:active","rank_uptime":14.006222600999999,"mdsmap_epoch":62,"osdmap_epoch":129,"osdmap_epoch_barrier":128,"uptime":18.970435989999999} 2020-12-05T17:41:17.300 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 7938, "mdsmap_epoch": 62, "osdmap_epoch": 129, "osdmap_epoch_barrier": 128, "rank_uptime": 14.006222601, "state": "up:active", "uptime": 18.97043599, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:41:17.301 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:41:17.301 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-12-05T17:41:17.301 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi002.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-12-05T17:41:17.302 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:17.302 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:17.310 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:41:17.369 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:41:17.370 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:41:17.423 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:41:17.424 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:41:17.425 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:17.425 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:41:17.472 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:41:17.473 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:17.516 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:41:17.526 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:41:17.526 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:41:17.582 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:41:17.583 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:41:17.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:17.584 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:41:17.591 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:2020-12-05T17:41:17.589+0000 7f3330dc80c0 -1 init, newargv = 0x562dfbf81cf0 newargc=9ceph-fuse[11099]: starting ceph client 2020-12-05T17:41:17.591 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr: 2020-12-05T17:41:17.602 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[11099]: starting fuse 2020-12-05T17:41:18.696 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:41:18.712 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:41:18.713 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:41:18.768 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:41:18.770 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:41:18.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:18.771 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:41:18.824 INFO:teuthology.orchestra.run.smithi002.stdout:50 2020-12-05T17:41:18.824 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [50] 2020-12-05T17:41:18.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:18.825 INFO:teuthology.orchestra.run.smithi002:> sudo python3 -c ' 2020-12-05T17:41:18.826 INFO:teuthology.orchestra.run.smithi002:> import glob 2020-12-05T17:41:18.826 INFO:teuthology.orchestra.run.smithi002:> import re 2020-12-05T17:41:18.826 INFO:teuthology.orchestra.run.smithi002:> import os 2020-12-05T17:41:18.827 INFO:teuthology.orchestra.run.smithi002:> import subprocess 2020-12-05T17:41:18.827 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:41:18.828 INFO:teuthology.orchestra.run.smithi002:> def find_socket(client_name): 2020-12-05T17:41:18.828 INFO:teuthology.orchestra.run.smithi002:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-12-05T17:41:18.828 INFO:teuthology.orchestra.run.smithi002:> files = glob.glob(asok_path) 2020-12-05T17:41:18.829 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:41:18.829 INFO:teuthology.orchestra.run.smithi002:> # Given a non-glob path, it better be there 2020-12-05T17:41:18.830 INFO:teuthology.orchestra.run.smithi002:> if "*" not in asok_path: 2020-12-05T17:41:18.830 INFO:teuthology.orchestra.run.smithi002:> assert(len(files) == 1) 2020-12-05T17:41:18.831 INFO:teuthology.orchestra.run.smithi002:> return files[0] 2020-12-05T17:41:18.831 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:41:18.831 INFO:teuthology.orchestra.run.smithi002:> for f in files: 2020-12-05T17:41:18.832 INFO:teuthology.orchestra.run.smithi002:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:41:18.832 INFO:teuthology.orchestra.run.smithi002:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:41:18.833 INFO:teuthology.orchestra.run.smithi002:> return f 2020-12-05T17:41:18.833 INFO:teuthology.orchestra.run.smithi002:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:41:18.833 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:41:18.834 INFO:teuthology.orchestra.run.smithi002:> print(find_socket("client.1")) 2020-12-05T17:41:18.834 INFO:teuthology.orchestra.run.smithi002:> ' 2020-12-05T17:41:18.914 INFO:teuthology.orchestra.run.smithi002.stdout:/var/run/ceph/ceph-client.1.11099.asok 2020-12-05T17:41:18.919 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.11099.asok 2020-12-05T17:41:18.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:18.920 INFO:teuthology.orchestra.run.smithi002:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.11099.asok status 2020-12-05T17:41:19.068 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2020-12-05T17:41:19.068 INFO:teuthology.orchestra.run.smithi002.stdout: "metadata": { 2020-12-05T17:41:19.068 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:41:19.069 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:41:19.069 INFO:teuthology.orchestra.run.smithi002.stdout: "entity_id": "1", 2020-12-05T17:41:19.069 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": "smithi002", 2020-12-05T17:41:19.069 INFO:teuthology.orchestra.run.smithi002.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-12-05T17:41:19.069 INFO:teuthology.orchestra.run.smithi002.stdout: "pid": "11099", 2020-12-05T17:41:19.070 INFO:teuthology.orchestra.run.smithi002.stdout: "root": "/" 2020-12-05T17:41:19.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:41:19.070 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_count": 0, 2020-12-05T17:41:19.070 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_pinned_count": 0, 2020-12-05T17:41:19.071 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 8092, 2020-12-05T17:41:19.071 INFO:teuthology.orchestra.run.smithi002.stdout: "inst": { 2020-12-05T17:41:19.071 INFO:teuthology.orchestra.run.smithi002.stdout: "name": { 2020-12-05T17:41:19.071 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "client", 2020-12-05T17:41:19.071 INFO:teuthology.orchestra.run.smithi002.stdout: "num": 8092 2020-12-05T17:41:19.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:41:19.072 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2020-12-05T17:41:19.072 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2020-12-05T17:41:19.072 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2020-12-05T17:41:19.073 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1275506218 2020-12-05T17:41:19.073 INFO:teuthology.orchestra.run.smithi002.stdout: } 2020-12-05T17:41:19.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:41:19.073 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2020-12-05T17:41:19.074 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2020-12-05T17:41:19.074 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2020-12-05T17:41:19.074 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1275506218 2020-12-05T17:41:19.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:41:19.074 INFO:teuthology.orchestra.run.smithi002.stdout: "inst_str": "client.8092 172.21.15.2:0/1275506218", 2020-12-05T17:41:19.075 INFO:teuthology.orchestra.run.smithi002.stdout: "addr_str": "172.21.15.2:0/1275506218", 2020-12-05T17:41:19.075 INFO:teuthology.orchestra.run.smithi002.stdout: "inode_count": 1, 2020-12-05T17:41:19.075 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_epoch": 62, 2020-12-05T17:41:19.075 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch": 129, 2020-12-05T17:41:19.076 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:41:19.076 INFO:teuthology.orchestra.run.smithi002.stdout: "blacklisted": false 2020-12-05T17:41:19.076 INFO:teuthology.orchestra.run.smithi002.stdout:} 2020-12-05T17:41:19.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:19.078 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:19.079 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:19.085 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:41:19.086 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:41:19.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:19.087 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:19.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:19.145 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:19.145 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:41:19.153 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:41:19.154 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:41:19.154 INFO:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-12-05T17:41:19.155 INFO:teuthology.orchestra.run.smithi036:> import time 2020-12-05T17:41:19.155 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:41:19.155 INFO:teuthology.orchestra.run.smithi036:> with open("/home/ubuntu/cephtest/mnt.0/testfile", '"'"'w'"'"') as f: 2020-12-05T17:41:19.156 INFO:teuthology.orchestra.run.smithi036:> f.write('"'"'content'"'"') 2020-12-05T17:41:19.156 INFO:teuthology.orchestra.run.smithi036:> f.flush() 2020-12-05T17:41:19.157 INFO:teuthology.orchestra.run.smithi036:> f.write('"'"'content2'"'"') 2020-12-05T17:41:19.157 INFO:teuthology.orchestra.run.smithi036:> while True: 2020-12-05T17:41:19.157 INFO:teuthology.orchestra.run.smithi036:> time.sleep(1) 2020-12-05T17:41:19.158 INFO:teuthology.orchestra.run.smithi036:> ' 2020-12-05T17:41:19.197 INFO:teuthology.orchestra.run.smithi036:> sudo ls /home/ubuntu/cephtest/mnt.0/testfile 2020-12-05T17:41:19.219 INFO:teuthology.orchestra.run.smithi036.stderr:ls: cannot access '/home/ubuntu/cephtest/mnt.0/testfile': No such file or directory 2020-12-05T17:41:19.220 DEBUG:teuthology.orchestra.run:got remote process result: 2 2020-12-05T17:41:20.221 INFO:teuthology.orchestra.run.smithi036:> sudo ls /home/ubuntu/cephtest/mnt.0/testfile 2020-12-05T17:41:20.238 INFO:teuthology.orchestra.run.smithi036.stdout:/home/ubuntu/cephtest/mnt.0/testfile 2020-12-05T17:41:20.239 DEBUG:tasks.cephfs.mount:File testfile became visible from 0 after 1s 2020-12-05T17:41:20.240 INFO:teuthology.orchestra.run.smithi002:> sudo ls /home/ubuntu/cephtest/mnt.1/testfile 2020-12-05T17:41:20.277 INFO:teuthology.orchestra.run.smithi002.stdout:/home/ubuntu/cephtest/mnt.1/testfile 2020-12-05T17:41:20.278 DEBUG:tasks.cephfs.mount:File testfile became visible from 1 after 0s 2020-12-05T17:41:20.279 INFO:teuthology.orchestra.run.smithi002:> cd /home/ubuntu/cephtest/mnt.1 && sudo cat testfile 2020-12-05T17:41:20.371 INFO:teuthology.orchestra.run.smithi002.stdout:content 2020-12-05T17:41:20.387 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:41:20.387 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:41:20.403 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:41:20.421 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:41:20.444 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:41:20.461 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:41:20.479 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:41:20.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:20.480 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:20.481 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:41:20.494 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:41:20.495 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:41:20.496 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi036.front.sepia.ceph.com... 2020-12-05T17:41:20.496 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:41:20.497 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:41:20.580 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[11567]: fuse finished with error 0 and tester_r 0 2020-12-05T17:41:20.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:20.583 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:20.583 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:41:20.629 INFO:teuthology.orchestra.run.smithi036.stdout:ext2/ext3 2020-12-05T17:41:20.630 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:41:20.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:41:20.742 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:41:20.790 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:20.791 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:20.791 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:20.798 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:41:20.799 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:41:20.800 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi002.front.sepia.ceph.com... 2020-12-05T17:41:20.800 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:41:20.801 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:20.891 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[11099]: fuse finished with error 0 and tester_r 0 2020-12-05T17:41:20.892 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:20.893 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:20.893 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:20.900 INFO:teuthology.orchestra.run.smithi002.stdout:ext2/ext3 2020-12-05T17:41:20.901 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:41:20.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:41:20.989 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:20.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:20.997 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:20.997 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:41:21.004 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:41:21.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:41:21.005 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2020-12-05T17:41:21.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:41:21.006 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:41:21.052 INFO:teuthology.orchestra.run.smithi053.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:41:21.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:41:21.054 INFO:teuthology.orchestra.run.smithi002:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-12-05T17:41:21.056 INFO:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-12-05T17:41:21.059 INFO:teuthology.orchestra.run.smithi036:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-12-05T17:41:21.061 INFO:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-12-05T17:41:21.096 INFO:teuthology.orchestra.run.smithi061:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-12-05T17:41:21.100 INFO:teuthology.orchestra.run.smithi036.stdout:[global] 2020-12-05T17:41:21.100 INFO:teuthology.orchestra.run.smithi036.stdout: chdir = "" 2020-12-05T17:41:21.100 INFO:teuthology.orchestra.run.smithi036.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-12-05T17:41:21.100 INFO:teuthology.orchestra.run.smithi036.stdout: auth supported = cephx 2020-12-05T17:41:21.101 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.101 INFO:teuthology.orchestra.run.smithi036.stdout: filestore xattr use omap = true 2020-12-05T17:41:21.101 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.101 INFO:teuthology.orchestra.run.smithi036.stdout: mon clock drift allowed = 1.000 2020-12-05T17:41:21.101 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.102 INFO:teuthology.orchestra.run.smithi036.stdout: osd crush chooseleaf type = 0 2020-12-05T17:41:21.102 INFO:teuthology.orchestra.run.smithi036.stdout: auth debug = true 2020-12-05T17:41:21.102 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.103 INFO:teuthology.orchestra.run.smithi036.stdout: ms die on old message = true 2020-12-05T17:41:21.103 INFO:teuthology.orchestra.run.smithi036.stdout: ms die on bug = true 2020-12-05T17:41:21.103 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.103 INFO:teuthology.orchestra.run.smithi036.stdout: mon max pg per osd = 10000 # >= luminous 2020-12-05T17:41:21.104 INFO:teuthology.orchestra.run.smithi036.stdout: mon pg warn max object skew = 0 2020-12-05T17:41:21.104 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.104 INFO:teuthology.orchestra.run.smithi036.stdout: # disable pg_autoscaler by default for new pools 2020-12-05T17:41:21.104 INFO:teuthology.orchestra.run.smithi036.stdout: osd_pool_default_pg_autoscale_mode = off 2020-12-05T17:41:21.104 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.105 INFO:teuthology.orchestra.run.smithi036.stdout: osd pool default size = 2 2020-12-05T17:41:21.105 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.105 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd allow primary affinity = true 2020-12-05T17:41:21.105 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd allow pg remap = true 2020-12-05T17:41:21.105 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on legacy crush tunables = false 2020-12-05T17:41:21.106 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on crush straw calc version zero = false 2020-12-05T17:41:21.106 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on no sortbitwise = false 2020-12-05T17:41:21.106 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on osd down out interval zero = false 2020-12-05T17:41:21.106 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on too few osds = false 2020-12-05T17:41:21.107 INFO:teuthology.orchestra.run.smithi036.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-12-05T17:41:21.107 INFO:teuthology.orchestra.run.smithi036.stdout: mon_warn_on_pool_no_redundancy = false 2020-12-05T17:41:21.107 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.107 INFO:teuthology.orchestra.run.smithi036.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-12-05T17:41:21.107 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.108 INFO:teuthology.orchestra.run.smithi036.stdout: osd default data pool replay window = 5 2020-12-05T17:41:21.108 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.108 INFO:teuthology.orchestra.run.smithi036.stdout: mon allow pool delete = true 2020-12-05T17:41:21.108 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.108 INFO:teuthology.orchestra.run.smithi036.stdout: mon cluster log file level = debug 2020-12-05T17:41:21.109 INFO:teuthology.orchestra.run.smithi036.stdout: debug asserts on shutdown = true 2020-12-05T17:41:21.109 INFO:teuthology.orchestra.run.smithi036.stdout: mon host = "172.21.15.32,172.21.15.61,[v2:172.21.15.61:3301,v1:172.21.15.61:6790]" 2020-12-05T17:41:21.109 INFO:teuthology.orchestra.run.smithi036.stdout: fsid = efd7e861-a502-461d-b20d-3b2c0a25fbd9 2020-12-05T17:41:21.109 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.110 INFO:teuthology.orchestra.run.smithi036.stdout:[osd] 2020-12-05T17:41:21.110 INFO:teuthology.orchestra.run.smithi036.stdout: osd journal size = 100 2020-12-05T17:41:21.110 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.110 INFO:teuthology.orchestra.run.smithi036.stdout: osd scrub load threshold = 5.0 2020-12-05T17:41:21.110 INFO:teuthology.orchestra.run.smithi036.stdout: osd scrub max interval = 600 2020-12-05T17:41:21.111 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.111 INFO:teuthology.orchestra.run.smithi036.stdout: osd recover clone overlap = true 2020-12-05T17:41:21.111 INFO:teuthology.orchestra.run.smithi036.stdout: osd recovery max chunk = 1048576 2020-12-05T17:41:21.111 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.111 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug shutdown = true 2020-12-05T17:41:21.112 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug op order = true 2020-12-05T17:41:21.112 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug verify stray on activate = true 2020-12-05T17:41:21.112 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.112 INFO:teuthology.orchestra.run.smithi036.stdout: osd open classes on start = true 2020-12-05T17:41:21.112 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug pg log writeout = true 2020-12-05T17:41:21.113 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.113 INFO:teuthology.orchestra.run.smithi036.stdout: osd deep scrub update digest min age = 30 2020-12-05T17:41:21.113 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.113 INFO:teuthology.orchestra.run.smithi036.stdout: osd map max advance = 10 2020-12-05T17:41:21.113 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.114 INFO:teuthology.orchestra.run.smithi036.stdout: journal zero on create = true 2020-12-05T17:41:21.114 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.114 INFO:teuthology.orchestra.run.smithi036.stdout: filestore ondisk finisher threads = 3 2020-12-05T17:41:21.114 INFO:teuthology.orchestra.run.smithi036.stdout: filestore apply finisher threads = 3 2020-12-05T17:41:21.114 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.115 INFO:teuthology.orchestra.run.smithi036.stdout: bdev debug aio = true 2020-12-05T17:41:21.115 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug misdirected ops = true 2020-12-05T17:41:21.115 INFO:teuthology.orchestra.run.smithi036.stdout: bluestore block size = 96636764160 2020-12-05T17:41:21.115 INFO:teuthology.orchestra.run.smithi036.stdout: bluestore fsck on mount = True 2020-12-05T17:41:21.115 INFO:teuthology.orchestra.run.smithi036.stdout: debug bluefs = 20 2020-12-05T17:41:21.116 INFO:teuthology.orchestra.run.smithi036.stdout: debug bluestore = 20 2020-12-05T17:41:21.116 INFO:teuthology.orchestra.run.smithi036.stdout: debug filestore = 20 2020-12-05T17:41:21.116 INFO:teuthology.orchestra.run.smithi036.stdout: debug journal = 20 2020-12-05T17:41:21.116 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 20 2020-12-05T17:41:21.116 INFO:teuthology.orchestra.run.smithi036.stdout: debug osd = 25 2020-12-05T17:41:21.117 INFO:teuthology.orchestra.run.smithi036.stdout: debug rocksdb = 10 2020-12-05T17:41:21.117 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd backfillfull_ratio = 0.85 2020-12-05T17:41:21.117 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd full ratio = 0.9 2020-12-05T17:41:21.117 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd nearfull ratio = 0.8 2020-12-05T17:41:21.118 INFO:teuthology.orchestra.run.smithi036.stdout: osd failsafe full ratio = 0.95 2020-12-05T17:41:21.118 INFO:teuthology.orchestra.run.smithi036.stdout: osd objectstore = bluestore 2020-12-05T17:41:21.118 INFO:teuthology.orchestra.run.smithi036.stdout: osd op complaint time = 180 2020-12-05T17:41:21.118 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.119 INFO:teuthology.orchestra.run.smithi036.stdout:[mgr] 2020-12-05T17:41:21.119 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2020-12-05T17:41:21.119 INFO:teuthology.orchestra.run.smithi036.stdout: debug mgr = 20 2020-12-05T17:41:21.119 INFO:teuthology.orchestra.run.smithi036.stdout: debug mon = 20 2020-12-05T17:41:21.119 INFO:teuthology.orchestra.run.smithi036.stdout: debug auth = 20 2020-12-05T17:41:21.120 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:41:21.120 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:41:21.120 INFO:teuthology.orchestra.run.smithi036.stdout: mgr/telemetry/nag = false 2020-12-05T17:41:21.120 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.120 INFO:teuthology.orchestra.run.smithi036.stdout:[mon] 2020-12-05T17:41:21.121 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2020-12-05T17:41:21.121 INFO:teuthology.orchestra.run.smithi036.stdout: debug mon = 20 2020-12-05T17:41:21.121 INFO:teuthology.orchestra.run.smithi036.stdout: debug paxos = 20 2020-12-05T17:41:21.121 INFO:teuthology.orchestra.run.smithi036.stdout: debug auth = 20 2020-12-05T17:41:21.121 INFO:teuthology.orchestra.run.smithi036.stdout: mon data avail warn = 5 2020-12-05T17:41:21.122 INFO:teuthology.orchestra.run.smithi036.stdout: mon mgr mkfs grace = 240 2020-12-05T17:41:21.122 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:41:21.122 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd reporter subtree level = osd 2020-12-05T17:41:21.122 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd prime pg temp = true 2020-12-05T17:41:21.122 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:41:21.123 INFO:teuthology.orchestra.run.smithi036.stdout: mon op complaint time = 120 2020-12-05T17:41:21.123 INFO:teuthology.orchestra.run.smithi036.stdout: 2020-12-05T17:41:21.123 INFO:teuthology.orchestra.run.smithi036.stdout:[client] 2020-12-05T17:41:21.123 INFO:teuthology.orchestra.run.smithi036.stdout: rgw cache enabled = true 2020-12-05T17:41:21.123 INFO:teuthology.orchestra.run.smithi036.stdout: rgw enable ops log = true 2020-12-05T17:41:21.124 INFO:teuthology.orchestra.run.smithi036.stdout: rgw enable usage log = true 2020-12-05T17:41:21.124 INFO:teuthology.orchestra.run.smithi036.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-12-05T17:41:21.124 INFO:teuthology.orchestra.run.smithi036.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-12-05T17:41:21.124 INFO:teuthology.orchestra.run.smithi036.stdout: client mount timeout = 600 2020-12-05T17:41:21.124 INFO:teuthology.orchestra.run.smithi036.stdout: debug client = 20 2020-12-05T17:41:21.125 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2020-12-05T17:41:21.125 INFO:teuthology.orchestra.run.smithi036.stdout: rados mon op timeout = 15m 2020-12-05T17:41:21.125 INFO:teuthology.orchestra.run.smithi036.stdout: rados osd op timeout = 15m 2020-12-05T17:41:21.125 INFO:teuthology.orchestra.run.smithi036.stdout:[mon.a] 2020-12-05T17:41:21.126 INFO:teuthology.orchestra.run.smithi036.stdout:[mon.b] 2020-12-05T17:41:21.126 INFO:teuthology.orchestra.run.smithi036.stdout:[mon.c] 2020-12-05T17:41:21.126 INFO:teuthology.orchestra.run.smithi036.stdout:[mds.a] 2020-12-05T17:41:21.126 INFO:teuthology.orchestra.run.smithi036.stdout:[mds.b] 2020-12-05T17:41:21.126 INFO:teuthology.orchestra.run.smithi036.stdout:[mds] 2020-12-05T17:41:21.127 INFO:teuthology.orchestra.run.smithi036.stdout: debug mds = 20 2020-12-05T17:41:21.127 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2020-12-05T17:41:21.127 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal frag = True 2020-12-05T17:41:21.127 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal fragment size max = 10000 2020-12-05T17:41:21.128 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal merge size = 5 2020-12-05T17:41:21.128 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal split bits = 3 2020-12-05T17:41:21.128 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal split size = 100 2020-12-05T17:41:21.128 INFO:teuthology.orchestra.run.smithi036.stdout: mds debug frag = True 2020-12-05T17:41:21.128 INFO:teuthology.orchestra.run.smithi036.stdout: mds debug scatterstat = True 2020-12-05T17:41:21.129 INFO:teuthology.orchestra.run.smithi036.stdout: mds op complaint time = 180 2020-12-05T17:41:21.129 INFO:teuthology.orchestra.run.smithi036.stdout: mds verify scatter = True 2020-12-05T17:41:21.129 INFO:teuthology.orchestra.run.smithi036.stdout: osd op complaint time = 180 2020-12-05T17:41:21.129 INFO:teuthology.orchestra.run.smithi036.stdout: rados mon op timeout = 15m 2020-12-05T17:41:21.129 INFO:teuthology.orchestra.run.smithi036.stdout: rados osd op timeout = 15m 2020-12-05T17:41:21.130 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2020-12-05T17:41:21.130 INFO:teuthology.orchestra.run.smithi032.stdout: chdir = "" 2020-12-05T17:41:21.130 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-12-05T17:41:21.130 INFO:teuthology.orchestra.run.smithi032.stdout: auth supported = cephx 2020-12-05T17:41:21.131 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.131 INFO:teuthology.orchestra.run.smithi032.stdout: filestore xattr use omap = true 2020-12-05T17:41:21.131 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.131 INFO:teuthology.orchestra.run.smithi032.stdout: mon clock drift allowed = 1.000 2020-12-05T17:41:21.131 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.132 INFO:teuthology.orchestra.run.smithi032.stdout: osd crush chooseleaf type = 0 2020-12-05T17:41:21.132 INFO:teuthology.orchestra.run.smithi032.stdout: auth debug = true 2020-12-05T17:41:21.132 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.132 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on old message = true 2020-12-05T17:41:21.132 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on bug = true 2020-12-05T17:41:21.132 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.133 INFO:teuthology.orchestra.run.smithi032.stdout: mon max pg per osd = 10000 # >= luminous 2020-12-05T17:41:21.133 INFO:teuthology.orchestra.run.smithi032.stdout: mon pg warn max object skew = 0 2020-12-05T17:41:21.133 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.133 INFO:teuthology.orchestra.run.smithi032.stdout: # disable pg_autoscaler by default for new pools 2020-12-05T17:41:21.134 INFO:teuthology.orchestra.run.smithi032.stdout: osd_pool_default_pg_autoscale_mode = off 2020-12-05T17:41:21.134 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.134 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default size = 2 2020-12-05T17:41:21.134 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.134 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow primary affinity = true 2020-12-05T17:41:21.135 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow pg remap = true 2020-12-05T17:41:21.135 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on legacy crush tunables = false 2020-12-05T17:41:21.135 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on crush straw calc version zero = false 2020-12-05T17:41:21.135 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on no sortbitwise = false 2020-12-05T17:41:21.135 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on osd down out interval zero = false 2020-12-05T17:41:21.136 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on too few osds = false 2020-12-05T17:41:21.136 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-12-05T17:41:21.136 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_no_redundancy = false 2020-12-05T17:41:21.136 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.136 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-12-05T17:41:21.137 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.137 INFO:teuthology.orchestra.run.smithi032.stdout: osd default data pool replay window = 5 2020-12-05T17:41:21.137 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.137 INFO:teuthology.orchestra.run.smithi032.stdout: mon allow pool delete = true 2020-12-05T17:41:21.137 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.138 INFO:teuthology.orchestra.run.smithi032.stdout: mon cluster log file level = debug 2020-12-05T17:41:21.138 INFO:teuthology.orchestra.run.smithi032.stdout: debug asserts on shutdown = true 2020-12-05T17:41:21.138 INFO:teuthology.orchestra.run.smithi032.stdout: mon host = "172.21.15.32,172.21.15.61,[v2:172.21.15.61:3301,v1:172.21.15.61:6790]" 2020-12-05T17:41:21.138 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = efd7e861-a502-461d-b20d-3b2c0a25fbd9 2020-12-05T17:41:21.138 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.139 INFO:teuthology.orchestra.run.smithi032.stdout:[osd] 2020-12-05T17:41:21.139 INFO:teuthology.orchestra.run.smithi032.stdout: osd journal size = 100 2020-12-05T17:41:21.139 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.139 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub load threshold = 5.0 2020-12-05T17:41:21.139 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub max interval = 600 2020-12-05T17:41:21.139 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.140 INFO:teuthology.orchestra.run.smithi032.stdout: osd recover clone overlap = true 2020-12-05T17:41:21.140 INFO:teuthology.orchestra.run.smithi032.stdout: osd recovery max chunk = 1048576 2020-12-05T17:41:21.140 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.140 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug shutdown = true 2020-12-05T17:41:21.140 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug op order = true 2020-12-05T17:41:21.141 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug verify stray on activate = true 2020-12-05T17:41:21.141 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.141 INFO:teuthology.orchestra.run.smithi032.stdout: osd open classes on start = true 2020-12-05T17:41:21.141 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug pg log writeout = true 2020-12-05T17:41:21.141 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.142 INFO:teuthology.orchestra.run.smithi032.stdout: osd deep scrub update digest min age = 30 2020-12-05T17:41:21.142 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.142 INFO:teuthology.orchestra.run.smithi032.stdout: osd map max advance = 10 2020-12-05T17:41:21.142 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.143 INFO:teuthology.orchestra.run.smithi032.stdout: journal zero on create = true 2020-12-05T17:41:21.143 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.143 INFO:teuthology.orchestra.run.smithi032.stdout: filestore ondisk finisher threads = 3 2020-12-05T17:41:21.143 INFO:teuthology.orchestra.run.smithi032.stdout: filestore apply finisher threads = 3 2020-12-05T17:41:21.144 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.144 INFO:teuthology.orchestra.run.smithi032.stdout: bdev debug aio = true 2020-12-05T17:41:21.144 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug misdirected ops = true 2020-12-05T17:41:21.144 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore block size = 96636764160 2020-12-05T17:41:21.144 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore fsck on mount = True 2020-12-05T17:41:21.144 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluefs = 20 2020-12-05T17:41:21.145 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluestore = 20 2020-12-05T17:41:21.145 INFO:teuthology.orchestra.run.smithi032.stdout: debug filestore = 20 2020-12-05T17:41:21.145 INFO:teuthology.orchestra.run.smithi032.stdout: debug journal = 20 2020-12-05T17:41:21.145 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 20 2020-12-05T17:41:21.145 INFO:teuthology.orchestra.run.smithi032.stdout: debug osd = 25 2020-12-05T17:41:21.146 INFO:teuthology.orchestra.run.smithi032.stdout: debug rocksdb = 10 2020-12-05T17:41:21.146 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd backfillfull_ratio = 0.85 2020-12-05T17:41:21.146 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd full ratio = 0.9 2020-12-05T17:41:21.146 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd nearfull ratio = 0.8 2020-12-05T17:41:21.146 INFO:teuthology.orchestra.run.smithi032.stdout: osd failsafe full ratio = 0.95 2020-12-05T17:41:21.147 INFO:teuthology.orchestra.run.smithi032.stdout: osd objectstore = bluestore 2020-12-05T17:41:21.147 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2020-12-05T17:41:21.147 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.147 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr] 2020-12-05T17:41:21.147 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2020-12-05T17:41:21.148 INFO:teuthology.orchestra.run.smithi032.stdout: debug mgr = 20 2020-12-05T17:41:21.148 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2020-12-05T17:41:21.148 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2020-12-05T17:41:21.148 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:41:21.148 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:41:21.149 INFO:teuthology.orchestra.run.smithi032.stdout: mgr/telemetry/nag = false 2020-12-05T17:41:21.149 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.149 INFO:teuthology.orchestra.run.smithi032.stdout:[mon] 2020-12-05T17:41:21.149 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2020-12-05T17:41:21.149 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2020-12-05T17:41:21.150 INFO:teuthology.orchestra.run.smithi032.stdout: debug paxos = 20 2020-12-05T17:41:21.150 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2020-12-05T17:41:21.150 INFO:teuthology.orchestra.run.smithi032.stdout: mon data avail warn = 5 2020-12-05T17:41:21.150 INFO:teuthology.orchestra.run.smithi032.stdout: mon mgr mkfs grace = 240 2020-12-05T17:41:21.150 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:41:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd reporter subtree level = osd 2020-12-05T17:41:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd prime pg temp = true 2020-12-05T17:41:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:41:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: mon op complaint time = 120 2020-12-05T17:41:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:21.152 INFO:teuthology.orchestra.run.smithi032.stdout:[client] 2020-12-05T17:41:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: rgw cache enabled = true 2020-12-05T17:41:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable ops log = true 2020-12-05T17:41:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable usage log = true 2020-12-05T17:41:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-12-05T17:41:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-12-05T17:41:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: client mount timeout = 600 2020-12-05T17:41:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2020-12-05T17:41:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2020-12-05T17:41:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 15m 2020-12-05T17:41:21.154 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 15m 2020-12-05T17:41:21.154 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.a] 2020-12-05T17:41:21.154 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.b] 2020-12-05T17:41:21.154 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.c] 2020-12-05T17:41:21.154 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2020-12-05T17:41:21.155 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2020-12-05T17:41:21.155 INFO:teuthology.orchestra.run.smithi032.stdout:[mds] 2020-12-05T17:41:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds = 20 2020-12-05T17:41:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2020-12-05T17:41:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal frag = True 2020-12-05T17:41:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal fragment size max = 10000 2020-12-05T17:41:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal merge size = 5 2020-12-05T17:41:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal split bits = 3 2020-12-05T17:41:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal split size = 100 2020-12-05T17:41:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug frag = True 2020-12-05T17:41:21.157 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug scatterstat = True 2020-12-05T17:41:21.157 INFO:teuthology.orchestra.run.smithi032.stdout: mds op complaint time = 180 2020-12-05T17:41:21.157 INFO:teuthology.orchestra.run.smithi032.stdout: mds verify scatter = True 2020-12-05T17:41:21.157 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2020-12-05T17:41:21.157 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 15m 2020-12-05T17:41:21.158 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 15m 2020-12-05T17:41:21.158 INFO:teuthology.orchestra.run.smithi002.stdout:[global] 2020-12-05T17:41:21.158 INFO:teuthology.orchestra.run.smithi002.stdout: chdir = "" 2020-12-05T17:41:21.158 INFO:teuthology.orchestra.run.smithi002.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-12-05T17:41:21.158 INFO:teuthology.orchestra.run.smithi002.stdout: auth supported = cephx 2020-12-05T17:41:21.159 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.159 INFO:teuthology.orchestra.run.smithi002.stdout: filestore xattr use omap = true 2020-12-05T17:41:21.159 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.159 INFO:teuthology.orchestra.run.smithi002.stdout: mon clock drift allowed = 1.000 2020-12-05T17:41:21.159 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.160 INFO:teuthology.orchestra.run.smithi002.stdout: osd crush chooseleaf type = 0 2020-12-05T17:41:21.160 INFO:teuthology.orchestra.run.smithi002.stdout: auth debug = true 2020-12-05T17:41:21.160 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.160 INFO:teuthology.orchestra.run.smithi002.stdout: ms die on old message = true 2020-12-05T17:41:21.160 INFO:teuthology.orchestra.run.smithi002.stdout: ms die on bug = true 2020-12-05T17:41:21.161 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.161 INFO:teuthology.orchestra.run.smithi002.stdout: mon max pg per osd = 10000 # >= luminous 2020-12-05T17:41:21.161 INFO:teuthology.orchestra.run.smithi002.stdout: mon pg warn max object skew = 0 2020-12-05T17:41:21.161 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.161 INFO:teuthology.orchestra.run.smithi002.stdout: # disable pg_autoscaler by default for new pools 2020-12-05T17:41:21.162 INFO:teuthology.orchestra.run.smithi002.stdout: osd_pool_default_pg_autoscale_mode = off 2020-12-05T17:41:21.162 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.162 INFO:teuthology.orchestra.run.smithi002.stdout: osd pool default size = 2 2020-12-05T17:41:21.162 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.162 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd allow primary affinity = true 2020-12-05T17:41:21.163 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd allow pg remap = true 2020-12-05T17:41:21.163 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on legacy crush tunables = false 2020-12-05T17:41:21.163 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on crush straw calc version zero = false 2020-12-05T17:41:21.163 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on no sortbitwise = false 2020-12-05T17:41:21.163 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on osd down out interval zero = false 2020-12-05T17:41:21.164 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on too few osds = false 2020-12-05T17:41:21.164 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-12-05T17:41:21.164 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_pool_no_redundancy = false 2020-12-05T17:41:21.164 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.164 INFO:teuthology.orchestra.run.smithi002.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-12-05T17:41:21.165 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.165 INFO:teuthology.orchestra.run.smithi002.stdout: osd default data pool replay window = 5 2020-12-05T17:41:21.165 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.165 INFO:teuthology.orchestra.run.smithi002.stdout: mon allow pool delete = true 2020-12-05T17:41:21.165 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.166 INFO:teuthology.orchestra.run.smithi002.stdout: mon cluster log file level = debug 2020-12-05T17:41:21.166 INFO:teuthology.orchestra.run.smithi002.stdout: debug asserts on shutdown = true 2020-12-05T17:41:21.166 INFO:teuthology.orchestra.run.smithi002.stdout: mon host = "172.21.15.32,172.21.15.61,[v2:172.21.15.61:3301,v1:172.21.15.61:6790]" 2020-12-05T17:41:21.166 INFO:teuthology.orchestra.run.smithi002.stdout: fsid = efd7e861-a502-461d-b20d-3b2c0a25fbd9 2020-12-05T17:41:21.166 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.167 INFO:teuthology.orchestra.run.smithi002.stdout:[osd] 2020-12-05T17:41:21.167 INFO:teuthology.orchestra.run.smithi002.stdout: osd journal size = 100 2020-12-05T17:41:21.167 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.167 INFO:teuthology.orchestra.run.smithi002.stdout: osd scrub load threshold = 5.0 2020-12-05T17:41:21.167 INFO:teuthology.orchestra.run.smithi002.stdout: osd scrub max interval = 600 2020-12-05T17:41:21.168 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.168 INFO:teuthology.orchestra.run.smithi002.stdout: osd recover clone overlap = true 2020-12-05T17:41:21.168 INFO:teuthology.orchestra.run.smithi002.stdout: osd recovery max chunk = 1048576 2020-12-05T17:41:21.168 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.168 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug shutdown = true 2020-12-05T17:41:21.169 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug op order = true 2020-12-05T17:41:21.169 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug verify stray on activate = true 2020-12-05T17:41:21.169 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.169 INFO:teuthology.orchestra.run.smithi002.stdout: osd open classes on start = true 2020-12-05T17:41:21.169 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug pg log writeout = true 2020-12-05T17:41:21.170 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.170 INFO:teuthology.orchestra.run.smithi002.stdout: osd deep scrub update digest min age = 30 2020-12-05T17:41:21.170 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.170 INFO:teuthology.orchestra.run.smithi002.stdout: osd map max advance = 10 2020-12-05T17:41:21.170 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.171 INFO:teuthology.orchestra.run.smithi002.stdout: journal zero on create = true 2020-12-05T17:41:21.171 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.171 INFO:teuthology.orchestra.run.smithi002.stdout: filestore ondisk finisher threads = 3 2020-12-05T17:41:21.171 INFO:teuthology.orchestra.run.smithi002.stdout: filestore apply finisher threads = 3 2020-12-05T17:41:21.171 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.172 INFO:teuthology.orchestra.run.smithi002.stdout: bdev debug aio = true 2020-12-05T17:41:21.172 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug misdirected ops = true 2020-12-05T17:41:21.172 INFO:teuthology.orchestra.run.smithi002.stdout: bluestore block size = 96636764160 2020-12-05T17:41:21.172 INFO:teuthology.orchestra.run.smithi002.stdout: bluestore fsck on mount = True 2020-12-05T17:41:21.172 INFO:teuthology.orchestra.run.smithi002.stdout: debug bluefs = 20 2020-12-05T17:41:21.173 INFO:teuthology.orchestra.run.smithi002.stdout: debug bluestore = 20 2020-12-05T17:41:21.173 INFO:teuthology.orchestra.run.smithi002.stdout: debug filestore = 20 2020-12-05T17:41:21.173 INFO:teuthology.orchestra.run.smithi002.stdout: debug journal = 20 2020-12-05T17:41:21.173 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 20 2020-12-05T17:41:21.173 INFO:teuthology.orchestra.run.smithi002.stdout: debug osd = 25 2020-12-05T17:41:21.174 INFO:teuthology.orchestra.run.smithi002.stdout: debug rocksdb = 10 2020-12-05T17:41:21.174 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd backfillfull_ratio = 0.85 2020-12-05T17:41:21.174 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd full ratio = 0.9 2020-12-05T17:41:21.174 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd nearfull ratio = 0.8 2020-12-05T17:41:21.174 INFO:teuthology.orchestra.run.smithi002.stdout: osd failsafe full ratio = 0.95 2020-12-05T17:41:21.175 INFO:teuthology.orchestra.run.smithi002.stdout: osd objectstore = bluestore 2020-12-05T17:41:21.175 INFO:teuthology.orchestra.run.smithi002.stdout: osd op complaint time = 180 2020-12-05T17:41:21.175 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.175 INFO:teuthology.orchestra.run.smithi002.stdout:[mgr] 2020-12-05T17:41:21.175 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2020-12-05T17:41:21.176 INFO:teuthology.orchestra.run.smithi002.stdout: debug mgr = 20 2020-12-05T17:41:21.176 INFO:teuthology.orchestra.run.smithi002.stdout: debug mon = 20 2020-12-05T17:41:21.176 INFO:teuthology.orchestra.run.smithi002.stdout: debug auth = 20 2020-12-05T17:41:21.176 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:41:21.177 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:41:21.177 INFO:teuthology.orchestra.run.smithi002.stdout: mgr/telemetry/nag = false 2020-12-05T17:41:21.177 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.177 INFO:teuthology.orchestra.run.smithi002.stdout:[mon] 2020-12-05T17:41:21.177 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2020-12-05T17:41:21.178 INFO:teuthology.orchestra.run.smithi002.stdout: debug mon = 20 2020-12-05T17:41:21.178 INFO:teuthology.orchestra.run.smithi002.stdout: debug paxos = 20 2020-12-05T17:41:21.178 INFO:teuthology.orchestra.run.smithi002.stdout: debug auth = 20 2020-12-05T17:41:21.178 INFO:teuthology.orchestra.run.smithi002.stdout: mon data avail warn = 5 2020-12-05T17:41:21.178 INFO:teuthology.orchestra.run.smithi002.stdout: mon mgr mkfs grace = 240 2020-12-05T17:41:21.179 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:41:21.179 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd reporter subtree level = osd 2020-12-05T17:41:21.179 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd prime pg temp = true 2020-12-05T17:41:21.179 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:41:21.179 INFO:teuthology.orchestra.run.smithi002.stdout: mon op complaint time = 120 2020-12-05T17:41:21.180 INFO:teuthology.orchestra.run.smithi002.stdout: 2020-12-05T17:41:21.180 INFO:teuthology.orchestra.run.smithi002.stdout:[client] 2020-12-05T17:41:21.180 INFO:teuthology.orchestra.run.smithi002.stdout: rgw cache enabled = true 2020-12-05T17:41:21.180 INFO:teuthology.orchestra.run.smithi002.stdout: rgw enable ops log = true 2020-12-05T17:41:21.180 INFO:teuthology.orchestra.run.smithi002.stdout: rgw enable usage log = true 2020-12-05T17:41:21.181 INFO:teuthology.orchestra.run.smithi002.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-12-05T17:41:21.181 INFO:teuthology.orchestra.run.smithi002.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-12-05T17:41:21.181 INFO:teuthology.orchestra.run.smithi002.stdout: client mount timeout = 600 2020-12-05T17:41:21.181 INFO:teuthology.orchestra.run.smithi002.stdout: debug client = 20 2020-12-05T17:41:21.181 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2020-12-05T17:41:21.181 INFO:teuthology.orchestra.run.smithi002.stdout: rados mon op timeout = 15m 2020-12-05T17:41:21.182 INFO:teuthology.orchestra.run.smithi002.stdout: rados osd op timeout = 15m 2020-12-05T17:41:21.182 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.a] 2020-12-05T17:41:21.182 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.b] 2020-12-05T17:41:21.182 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.c] 2020-12-05T17:41:21.182 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.a] 2020-12-05T17:41:21.183 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.b] 2020-12-05T17:41:21.183 INFO:teuthology.orchestra.run.smithi002.stdout:[mds] 2020-12-05T17:41:21.183 INFO:teuthology.orchestra.run.smithi002.stdout: debug mds = 20 2020-12-05T17:41:21.183 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2020-12-05T17:41:21.183 INFO:teuthology.orchestra.run.smithi002.stdout: mds bal frag = True 2020-12-05T17:41:21.184 INFO:teuthology.orchestra.run.smithi002.stdout: mds bal fragment size max = 10000 2020-12-05T17:41:21.184 INFO:teuthology.orchestra.run.smithi002.stdout: mds bal merge size = 5 2020-12-05T17:41:21.184 INFO:teuthology.orchestra.run.smithi002.stdout: mds bal split bits = 3 2020-12-05T17:41:21.184 INFO:teuthology.orchestra.run.smithi002.stdout: mds bal split size = 100 2020-12-05T17:41:21.184 INFO:teuthology.orchestra.run.smithi002.stdout: mds debug frag = True 2020-12-05T17:41:21.185 INFO:teuthology.orchestra.run.smithi002.stdout: mds debug scatterstat = True 2020-12-05T17:41:21.185 INFO:teuthology.orchestra.run.smithi002.stdout: mds op complaint time = 180 2020-12-05T17:41:21.185 INFO:teuthology.orchestra.run.smithi002.stdout: mds verify scatter = True 2020-12-05T17:41:21.185 INFO:teuthology.orchestra.run.smithi002.stdout: osd op complaint time = 180 2020-12-05T17:41:21.185 INFO:teuthology.orchestra.run.smithi002.stdout: rados mon op timeout = 15m 2020-12-05T17:41:21.186 INFO:teuthology.orchestra.run.smithi002.stdout: rados osd op timeout = 15m 2020-12-05T17:41:21.188 INFO:teuthology.orchestra.run.smithi061.stdout:[global] 2020-12-05T17:41:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: chdir = "" 2020-12-05T17:41:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-12-05T17:41:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: auth supported = cephx 2020-12-05T17:41:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: filestore xattr use omap = true 2020-12-05T17:41:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: mon clock drift allowed = 1.000 2020-12-05T17:41:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: osd crush chooseleaf type = 0 2020-12-05T17:41:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: auth debug = true 2020-12-05T17:41:21.191 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.191 INFO:teuthology.orchestra.run.smithi061.stdout: ms die on old message = true 2020-12-05T17:41:21.191 INFO:teuthology.orchestra.run.smithi061.stdout: ms die on bug = true 2020-12-05T17:41:21.191 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.191 INFO:teuthology.orchestra.run.smithi061.stdout: mon max pg per osd = 10000 # >= luminous 2020-12-05T17:41:21.192 INFO:teuthology.orchestra.run.smithi061.stdout: mon pg warn max object skew = 0 2020-12-05T17:41:21.192 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.192 INFO:teuthology.orchestra.run.smithi061.stdout: # disable pg_autoscaler by default for new pools 2020-12-05T17:41:21.192 INFO:teuthology.orchestra.run.smithi061.stdout: osd_pool_default_pg_autoscale_mode = off 2020-12-05T17:41:21.192 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.193 INFO:teuthology.orchestra.run.smithi061.stdout: osd pool default size = 2 2020-12-05T17:41:21.193 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.193 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd allow primary affinity = true 2020-12-05T17:41:21.193 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd allow pg remap = true 2020-12-05T17:41:21.193 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on legacy crush tunables = false 2020-12-05T17:41:21.194 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on crush straw calc version zero = false 2020-12-05T17:41:21.194 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on no sortbitwise = false 2020-12-05T17:41:21.194 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on osd down out interval zero = false 2020-12-05T17:41:21.194 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on too few osds = false 2020-12-05T17:41:21.194 INFO:teuthology.orchestra.run.smithi061.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-12-05T17:41:21.195 INFO:teuthology.orchestra.run.smithi061.stdout: mon_warn_on_pool_no_redundancy = false 2020-12-05T17:41:21.195 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.195 INFO:teuthology.orchestra.run.smithi061.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-12-05T17:41:21.195 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.195 INFO:teuthology.orchestra.run.smithi061.stdout: osd default data pool replay window = 5 2020-12-05T17:41:21.196 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.196 INFO:teuthology.orchestra.run.smithi061.stdout: mon allow pool delete = true 2020-12-05T17:41:21.196 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.196 INFO:teuthology.orchestra.run.smithi061.stdout: mon cluster log file level = debug 2020-12-05T17:41:21.196 INFO:teuthology.orchestra.run.smithi061.stdout: debug asserts on shutdown = true 2020-12-05T17:41:21.197 INFO:teuthology.orchestra.run.smithi061.stdout: mon host = "172.21.15.32,172.21.15.61,[v2:172.21.15.61:3301,v1:172.21.15.61:6790]" 2020-12-05T17:41:21.197 INFO:teuthology.orchestra.run.smithi061.stdout: fsid = efd7e861-a502-461d-b20d-3b2c0a25fbd9 2020-12-05T17:41:21.197 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.197 INFO:teuthology.orchestra.run.smithi061.stdout:[osd] 2020-12-05T17:41:21.197 INFO:teuthology.orchestra.run.smithi061.stdout: osd journal size = 100 2020-12-05T17:41:21.198 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.198 INFO:teuthology.orchestra.run.smithi061.stdout: osd scrub load threshold = 5.0 2020-12-05T17:41:21.198 INFO:teuthology.orchestra.run.smithi061.stdout: osd scrub max interval = 600 2020-12-05T17:41:21.198 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.198 INFO:teuthology.orchestra.run.smithi061.stdout: osd recover clone overlap = true 2020-12-05T17:41:21.199 INFO:teuthology.orchestra.run.smithi061.stdout: osd recovery max chunk = 1048576 2020-12-05T17:41:21.199 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.199 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug shutdown = true 2020-12-05T17:41:21.199 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug op order = true 2020-12-05T17:41:21.199 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug verify stray on activate = true 2020-12-05T17:41:21.199 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.200 INFO:teuthology.orchestra.run.smithi061.stdout: osd open classes on start = true 2020-12-05T17:41:21.200 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug pg log writeout = true 2020-12-05T17:41:21.200 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.200 INFO:teuthology.orchestra.run.smithi061.stdout: osd deep scrub update digest min age = 30 2020-12-05T17:41:21.200 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.201 INFO:teuthology.orchestra.run.smithi061.stdout: osd map max advance = 10 2020-12-05T17:41:21.201 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.201 INFO:teuthology.orchestra.run.smithi061.stdout: journal zero on create = true 2020-12-05T17:41:21.201 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.201 INFO:teuthology.orchestra.run.smithi061.stdout: filestore ondisk finisher threads = 3 2020-12-05T17:41:21.202 INFO:teuthology.orchestra.run.smithi061.stdout: filestore apply finisher threads = 3 2020-12-05T17:41:21.202 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.202 INFO:teuthology.orchestra.run.smithi061.stdout: bdev debug aio = true 2020-12-05T17:41:21.202 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug misdirected ops = true 2020-12-05T17:41:21.202 INFO:teuthology.orchestra.run.smithi061.stdout: bluestore block size = 96636764160 2020-12-05T17:41:21.203 INFO:teuthology.orchestra.run.smithi061.stdout: bluestore fsck on mount = True 2020-12-05T17:41:21.203 INFO:teuthology.orchestra.run.smithi061.stdout: debug bluefs = 20 2020-12-05T17:41:21.203 INFO:teuthology.orchestra.run.smithi061.stdout: debug bluestore = 20 2020-12-05T17:41:21.203 INFO:teuthology.orchestra.run.smithi061.stdout: debug filestore = 20 2020-12-05T17:41:21.203 INFO:teuthology.orchestra.run.smithi061.stdout: debug journal = 20 2020-12-05T17:41:21.204 INFO:teuthology.orchestra.run.smithi061.stdout: debug ms = 20 2020-12-05T17:41:21.204 INFO:teuthology.orchestra.run.smithi061.stdout: debug osd = 25 2020-12-05T17:41:21.204 INFO:teuthology.orchestra.run.smithi061.stdout: debug rocksdb = 10 2020-12-05T17:41:21.204 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd backfillfull_ratio = 0.85 2020-12-05T17:41:21.204 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd full ratio = 0.9 2020-12-05T17:41:21.205 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd nearfull ratio = 0.8 2020-12-05T17:41:21.205 INFO:teuthology.orchestra.run.smithi061.stdout: osd failsafe full ratio = 0.95 2020-12-05T17:41:21.205 INFO:teuthology.orchestra.run.smithi061.stdout: osd objectstore = bluestore 2020-12-05T17:41:21.205 INFO:teuthology.orchestra.run.smithi061.stdout: osd op complaint time = 180 2020-12-05T17:41:21.205 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.206 INFO:teuthology.orchestra.run.smithi061.stdout:[mgr] 2020-12-05T17:41:21.206 INFO:teuthology.orchestra.run.smithi061.stdout: debug ms = 1 2020-12-05T17:41:21.206 INFO:teuthology.orchestra.run.smithi061.stdout: debug mgr = 20 2020-12-05T17:41:21.206 INFO:teuthology.orchestra.run.smithi061.stdout: debug mon = 20 2020-12-05T17:41:21.206 INFO:teuthology.orchestra.run.smithi061.stdout: debug auth = 20 2020-12-05T17:41:21.206 INFO:teuthology.orchestra.run.smithi061.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:41:21.207 INFO:teuthology.orchestra.run.smithi061.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:41:21.207 INFO:teuthology.orchestra.run.smithi061.stdout: mgr/telemetry/nag = false 2020-12-05T17:41:21.207 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.207 INFO:teuthology.orchestra.run.smithi061.stdout:[mon] 2020-12-05T17:41:21.207 INFO:teuthology.orchestra.run.smithi061.stdout: debug ms = 1 2020-12-05T17:41:21.208 INFO:teuthology.orchestra.run.smithi061.stdout: debug mon = 20 2020-12-05T17:41:21.208 INFO:teuthology.orchestra.run.smithi061.stdout: debug paxos = 20 2020-12-05T17:41:21.208 INFO:teuthology.orchestra.run.smithi061.stdout: debug auth = 20 2020-12-05T17:41:21.208 INFO:teuthology.orchestra.run.smithi061.stdout: mon data avail warn = 5 2020-12-05T17:41:21.208 INFO:teuthology.orchestra.run.smithi061.stdout: mon mgr mkfs grace = 240 2020-12-05T17:41:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:41:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd reporter subtree level = osd 2020-12-05T17:41:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd prime pg temp = true 2020-12-05T17:41:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:41:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: mon op complaint time = 120 2020-12-05T17:41:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: 2020-12-05T17:41:21.210 INFO:teuthology.orchestra.run.smithi061.stdout:[client] 2020-12-05T17:41:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: rgw cache enabled = true 2020-12-05T17:41:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: rgw enable ops log = true 2020-12-05T17:41:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: rgw enable usage log = true 2020-12-05T17:41:21.211 INFO:teuthology.orchestra.run.smithi061.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-12-05T17:41:21.211 INFO:teuthology.orchestra.run.smithi061.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-12-05T17:41:21.211 INFO:teuthology.orchestra.run.smithi061.stdout: client mount timeout = 600 2020-12-05T17:41:21.211 INFO:teuthology.orchestra.run.smithi061.stdout: debug client = 20 2020-12-05T17:41:21.211 INFO:teuthology.orchestra.run.smithi061.stdout: debug ms = 1 2020-12-05T17:41:21.212 INFO:teuthology.orchestra.run.smithi061.stdout: rados mon op timeout = 15m 2020-12-05T17:41:21.212 INFO:teuthology.orchestra.run.smithi061.stdout: rados osd op timeout = 15m 2020-12-05T17:41:21.212 INFO:teuthology.orchestra.run.smithi061.stdout:[mon.a] 2020-12-05T17:41:21.212 INFO:teuthology.orchestra.run.smithi061.stdout:[mon.b] 2020-12-05T17:41:21.212 INFO:teuthology.orchestra.run.smithi061.stdout:[mon.c] 2020-12-05T17:41:21.213 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.a] 2020-12-05T17:41:21.213 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.b] 2020-12-05T17:41:21.213 INFO:teuthology.orchestra.run.smithi061.stdout:[mds] 2020-12-05T17:41:21.213 INFO:teuthology.orchestra.run.smithi061.stdout: debug mds = 20 2020-12-05T17:41:21.213 INFO:teuthology.orchestra.run.smithi061.stdout: debug ms = 1 2020-12-05T17:41:21.214 INFO:teuthology.orchestra.run.smithi061.stdout: mds bal frag = True 2020-12-05T17:41:21.214 INFO:teuthology.orchestra.run.smithi061.stdout: mds bal fragment size max = 10000 2020-12-05T17:41:21.214 INFO:teuthology.orchestra.run.smithi061.stdout: mds bal merge size = 5 2020-12-05T17:41:21.214 INFO:teuthology.orchestra.run.smithi061.stdout: mds bal split bits = 3 2020-12-05T17:41:21.214 INFO:teuthology.orchestra.run.smithi061.stdout: mds bal split size = 100 2020-12-05T17:41:21.215 INFO:teuthology.orchestra.run.smithi061.stdout: mds debug frag = True 2020-12-05T17:41:21.215 INFO:teuthology.orchestra.run.smithi061.stdout: mds debug scatterstat = True 2020-12-05T17:41:21.215 INFO:teuthology.orchestra.run.smithi061.stdout: mds op complaint time = 180 2020-12-05T17:41:21.215 INFO:teuthology.orchestra.run.smithi061.stdout: mds verify scatter = True 2020-12-05T17:41:21.215 INFO:teuthology.orchestra.run.smithi061.stdout: osd op complaint time = 180 2020-12-05T17:41:21.216 INFO:teuthology.orchestra.run.smithi061.stdout: rados mon op timeout = 15m 2020-12-05T17:41:21.216 INFO:teuthology.orchestra.run.smithi061.stdout: rados osd op timeout = 15m 2020-12-05T17:41:21.216 INFO:teuthology.orchestra.run.smithi053.stdout:[global] 2020-12-05T17:41:21.216 INFO:teuthology.orchestra.run.smithi053.stdout: chdir = "" 2020-12-05T17:41:21.216 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-12-05T17:41:21.217 INFO:teuthology.orchestra.run.smithi053.stdout: auth supported = cephx 2020-12-05T17:41:21.217 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.217 INFO:teuthology.orchestra.run.smithi053.stdout: filestore xattr use omap = true 2020-12-05T17:41:21.217 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.217 INFO:teuthology.orchestra.run.smithi053.stdout: mon clock drift allowed = 1.000 2020-12-05T17:41:21.218 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.218 INFO:teuthology.orchestra.run.smithi053.stdout: osd crush chooseleaf type = 0 2020-12-05T17:41:21.218 INFO:teuthology.orchestra.run.smithi053.stdout: auth debug = true 2020-12-05T17:41:21.218 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.219 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on old message = true 2020-12-05T17:41:21.219 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on bug = true 2020-12-05T17:41:21.219 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.219 INFO:teuthology.orchestra.run.smithi053.stdout: mon max pg per osd = 10000 # >= luminous 2020-12-05T17:41:21.219 INFO:teuthology.orchestra.run.smithi053.stdout: mon pg warn max object skew = 0 2020-12-05T17:41:21.219 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.220 INFO:teuthology.orchestra.run.smithi053.stdout: # disable pg_autoscaler by default for new pools 2020-12-05T17:41:21.220 INFO:teuthology.orchestra.run.smithi053.stdout: osd_pool_default_pg_autoscale_mode = off 2020-12-05T17:41:21.220 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.220 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default size = 2 2020-12-05T17:41:21.220 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.221 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow primary affinity = true 2020-12-05T17:41:21.221 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow pg remap = true 2020-12-05T17:41:21.221 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on legacy crush tunables = false 2020-12-05T17:41:21.221 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on crush straw calc version zero = false 2020-12-05T17:41:21.221 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on no sortbitwise = false 2020-12-05T17:41:21.222 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on osd down out interval zero = false 2020-12-05T17:41:21.222 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on too few osds = false 2020-12-05T17:41:21.222 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-12-05T17:41:21.222 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_no_redundancy = false 2020-12-05T17:41:21.222 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.223 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-12-05T17:41:21.223 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.223 INFO:teuthology.orchestra.run.smithi053.stdout: osd default data pool replay window = 5 2020-12-05T17:41:21.223 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.223 INFO:teuthology.orchestra.run.smithi053.stdout: mon allow pool delete = true 2020-12-05T17:41:21.224 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.224 INFO:teuthology.orchestra.run.smithi053.stdout: mon cluster log file level = debug 2020-12-05T17:41:21.224 INFO:teuthology.orchestra.run.smithi053.stdout: debug asserts on shutdown = true 2020-12-05T17:41:21.224 INFO:teuthology.orchestra.run.smithi053.stdout: mon host = "172.21.15.32,172.21.15.61,[v2:172.21.15.61:3301,v1:172.21.15.61:6790]" 2020-12-05T17:41:21.224 INFO:teuthology.orchestra.run.smithi053.stdout: fsid = efd7e861-a502-461d-b20d-3b2c0a25fbd9 2020-12-05T17:41:21.225 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.225 INFO:teuthology.orchestra.run.smithi053.stdout:[osd] 2020-12-05T17:41:21.225 INFO:teuthology.orchestra.run.smithi053.stdout: osd journal size = 100 2020-12-05T17:41:21.225 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.225 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub load threshold = 5.0 2020-12-05T17:41:21.226 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub max interval = 600 2020-12-05T17:41:21.226 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.226 INFO:teuthology.orchestra.run.smithi053.stdout: osd recover clone overlap = true 2020-12-05T17:41:21.226 INFO:teuthology.orchestra.run.smithi053.stdout: osd recovery max chunk = 1048576 2020-12-05T17:41:21.226 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.227 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug shutdown = true 2020-12-05T17:41:21.227 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug op order = true 2020-12-05T17:41:21.227 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug verify stray on activate = true 2020-12-05T17:41:21.227 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.227 INFO:teuthology.orchestra.run.smithi053.stdout: osd open classes on start = true 2020-12-05T17:41:21.228 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug pg log writeout = true 2020-12-05T17:41:21.228 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.228 INFO:teuthology.orchestra.run.smithi053.stdout: osd deep scrub update digest min age = 30 2020-12-05T17:41:21.228 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.229 INFO:teuthology.orchestra.run.smithi053.stdout: osd map max advance = 10 2020-12-05T17:41:21.229 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.229 INFO:teuthology.orchestra.run.smithi053.stdout: journal zero on create = true 2020-12-05T17:41:21.229 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.229 INFO:teuthology.orchestra.run.smithi053.stdout: filestore ondisk finisher threads = 3 2020-12-05T17:41:21.230 INFO:teuthology.orchestra.run.smithi053.stdout: filestore apply finisher threads = 3 2020-12-05T17:41:21.230 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.230 INFO:teuthology.orchestra.run.smithi053.stdout: bdev debug aio = true 2020-12-05T17:41:21.230 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug misdirected ops = true 2020-12-05T17:41:21.230 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore block size = 96636764160 2020-12-05T17:41:21.231 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore fsck on mount = True 2020-12-05T17:41:21.231 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluefs = 20 2020-12-05T17:41:21.231 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluestore = 20 2020-12-05T17:41:21.231 INFO:teuthology.orchestra.run.smithi053.stdout: debug filestore = 20 2020-12-05T17:41:21.232 INFO:teuthology.orchestra.run.smithi053.stdout: debug journal = 20 2020-12-05T17:41:21.232 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 20 2020-12-05T17:41:21.232 INFO:teuthology.orchestra.run.smithi053.stdout: debug osd = 25 2020-12-05T17:41:21.232 INFO:teuthology.orchestra.run.smithi053.stdout: debug rocksdb = 10 2020-12-05T17:41:21.232 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd backfillfull_ratio = 0.85 2020-12-05T17:41:21.233 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd full ratio = 0.9 2020-12-05T17:41:21.233 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd nearfull ratio = 0.8 2020-12-05T17:41:21.233 INFO:teuthology.orchestra.run.smithi053.stdout: osd failsafe full ratio = 0.95 2020-12-05T17:41:21.233 INFO:teuthology.orchestra.run.smithi053.stdout: osd objectstore = bluestore 2020-12-05T17:41:21.233 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2020-12-05T17:41:21.234 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.234 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr] 2020-12-05T17:41:21.234 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2020-12-05T17:41:21.234 INFO:teuthology.orchestra.run.smithi053.stdout: debug mgr = 20 2020-12-05T17:41:21.234 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2020-12-05T17:41:21.235 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2020-12-05T17:41:21.235 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:41:21.235 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:41:21.235 INFO:teuthology.orchestra.run.smithi053.stdout: mgr/telemetry/nag = false 2020-12-05T17:41:21.235 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.236 INFO:teuthology.orchestra.run.smithi053.stdout:[mon] 2020-12-05T17:41:21.236 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2020-12-05T17:41:21.236 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2020-12-05T17:41:21.236 INFO:teuthology.orchestra.run.smithi053.stdout: debug paxos = 20 2020-12-05T17:41:21.236 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2020-12-05T17:41:21.236 INFO:teuthology.orchestra.run.smithi053.stdout: mon data avail warn = 5 2020-12-05T17:41:21.237 INFO:teuthology.orchestra.run.smithi053.stdout: mon mgr mkfs grace = 240 2020-12-05T17:41:21.237 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2020-12-05T17:41:21.237 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd reporter subtree level = osd 2020-12-05T17:41:21.237 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd prime pg temp = true 2020-12-05T17:41:21.237 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2020-12-05T17:41:21.238 INFO:teuthology.orchestra.run.smithi053.stdout: mon op complaint time = 120 2020-12-05T17:41:21.238 INFO:teuthology.orchestra.run.smithi053.stdout: 2020-12-05T17:41:21.238 INFO:teuthology.orchestra.run.smithi053.stdout:[client] 2020-12-05T17:41:21.238 INFO:teuthology.orchestra.run.smithi053.stdout: rgw cache enabled = true 2020-12-05T17:41:21.238 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable ops log = true 2020-12-05T17:41:21.239 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable usage log = true 2020-12-05T17:41:21.239 INFO:teuthology.orchestra.run.smithi053.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-12-05T17:41:21.239 INFO:teuthology.orchestra.run.smithi053.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-12-05T17:41:21.239 INFO:teuthology.orchestra.run.smithi053.stdout: client mount timeout = 600 2020-12-05T17:41:21.239 INFO:teuthology.orchestra.run.smithi053.stdout: debug client = 20 2020-12-05T17:41:21.240 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2020-12-05T17:41:21.240 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 15m 2020-12-05T17:41:21.240 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 15m 2020-12-05T17:41:21.240 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.a] 2020-12-05T17:41:21.240 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.b] 2020-12-05T17:41:21.241 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.c] 2020-12-05T17:41:21.241 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.a] 2020-12-05T17:41:21.241 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.b] 2020-12-05T17:41:21.241 INFO:teuthology.orchestra.run.smithi053.stdout:[mds] 2020-12-05T17:41:21.241 INFO:teuthology.orchestra.run.smithi053.stdout: debug mds = 20 2020-12-05T17:41:21.242 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2020-12-05T17:41:21.242 INFO:teuthology.orchestra.run.smithi053.stdout: mds bal frag = True 2020-12-05T17:41:21.242 INFO:teuthology.orchestra.run.smithi053.stdout: mds bal fragment size max = 10000 2020-12-05T17:41:21.242 INFO:teuthology.orchestra.run.smithi053.stdout: mds bal merge size = 5 2020-12-05T17:41:21.242 INFO:teuthology.orchestra.run.smithi053.stdout: mds bal split bits = 3 2020-12-05T17:41:21.243 INFO:teuthology.orchestra.run.smithi053.stdout: mds bal split size = 100 2020-12-05T17:41:21.243 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug frag = True 2020-12-05T17:41:21.243 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug scatterstat = True 2020-12-05T17:41:21.243 INFO:teuthology.orchestra.run.smithi053.stdout: mds op complaint time = 180 2020-12-05T17:41:21.243 INFO:teuthology.orchestra.run.smithi053.stdout: mds verify scatter = True 2020-12-05T17:41:21.244 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2020-12-05T17:41:21.244 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 15m 2020-12-05T17:41:21.244 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 15m 2020-12-05T17:41:21.245 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2020-12-05T17:41:21.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.366+0000 7f4833fff700 1 Processor -- start 2020-12-05T17:41:21.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.366+0000 7f4833fff700 1 -- start start 2020-12-05T17:41:21.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.366+0000 7f4833fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f48340ab710 0x7f48340af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:21.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.366+0000 7f4833fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4834146a90 con 0x7f48340b1fc0 2020-12-05T17:41:21.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.366+0000 7f4833fff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48340b23a0 con 0x7f4834147400 2020-12-05T17:41:21.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.366+0000 7f4833fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4834061e90 con 0x7f48340ab710 2020-12-05T17:41:21.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.366+0000 7f48327fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f48340ab710 0x7f48340af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:21.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.366+0000 7f48327fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f48340ab710 0x7f48340af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40568/0 (socket says 172.21.15.32:40568) 2020-12-05T17:41:21.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.366+0000 7f48327fc700 1 -- 172.21.15.32:0/2049618654 learned_addr learned my addr 172.21.15.32:0/2049618654 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:21.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.366+0000 7f4832ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f48340b1fc0 0x7f4834061780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43214/0 (socket says 172.21.15.32:43214) 2020-12-05T17:41:21.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.366+0000 7f48327fc700 1 -- 172.21.15.32:0/2049618654 >> v1:172.21.15.61:6789/0 conn(0x7f4834147400 legacy=0x7f4834145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:21.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.366+0000 7f48327fc700 1 -- 172.21.15.32:0/2049618654 >> v1:172.21.15.32:6789/0 conn(0x7f48340b1fc0 legacy=0x7f4834061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:21.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.366+0000 7f48327fc700 1 -- 172.21.15.32:0/2049618654 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4834146cc0 con 0x7f48340ab710 2020-12-05T17:41:21.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.366+0000 7f48327fc700 1 --2- 172.21.15.32:0/2049618654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48340ab710 0x7f48340af230 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 rx=0x7f4824002f30 tx=0x7f4824006e80).ready entity=mon.0 client_cookie=a2b4708d5dc363d7 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:21.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.370+0000 7f4831ffb700 1 -- 172.21.15.32:0/2049618654 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48240097f0 con 0x7f48340ab710 2020-12-05T17:41:21.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.370+0000 7f4831ffb700 1 -- 172.21.15.32:0/2049618654 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4824009950 con 0x7f48340ab710 2020-12-05T17:41:21.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.370+0000 7f4831ffb700 1 -- 172.21.15.32:0/2049618654 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4824009cb0 con 0x7f48340ab710 2020-12-05T17:41:21.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.370+0000 7f4833fff700 1 -- 172.21.15.32:0/2049618654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48340ab710 msgr2=0x7f48340af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:21.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.370+0000 7f4833fff700 1 --2- 172.21.15.32:0/2049618654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48340ab710 0x7f48340af230 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 rx=0x7f4824002f30 tx=0x7f4824006e80).stop 2020-12-05T17:41:21.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.370+0000 7f4833fff700 1 -- 172.21.15.32:0/2049618654 shutdown_connections 2020-12-05T17:41:21.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.370+0000 7f4833fff700 1 --2- 172.21.15.32:0/2049618654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48340ab710 0x7f48340af230 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:21.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.370+0000 7f4833fff700 1 -- 172.21.15.32:0/2049618654 >> 172.21.15.32:0/2049618654 conn(0x7f48340b5530 msgr2=0x7f48340b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:21.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.370+0000 7f4833fff700 1 -- 172.21.15.32:0/2049618654 shutdown_connections 2020-12-05T17:41:21.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.370+0000 7f4833fff700 1 -- 172.21.15.32:0/2049618654 wait complete. 2020-12-05T17:41:21.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.370+0000 7f4833fff700 1 Processor -- start 2020-12-05T17:41:21.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.370+0000 7f4833fff700 1 -- start start 2020-12-05T17:41:21.381 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.370+0000 7f4833fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48340ab710 0x7f4834066540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:21.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.370+0000 7f4833fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48340b1fc0 0x7f4834066a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:21.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.370+0000 7f4833fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4834147400 0x7f4834148c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:21.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.370+0000 7f4833fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4834066f20 con 0x7f4834147400 2020-12-05T17:41:21.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.370+0000 7f4833fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4834147e50 con 0x7f48340b1fc0 2020-12-05T17:41:21.383 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.370+0000 7f4833fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4834063cf0 con 0x7f48340ab710 2020-12-05T17:41:21.383 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.370+0000 7f48337fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4834147400 0x7f4834148c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:21.383 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.374+0000 7f48337fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4834147400 0x7f4834148c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40576/0 (socket says 172.21.15.32:40576) 2020-12-05T17:41:21.383 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.374+0000 7f48337fe700 1 -- 172.21.15.32:0/211614104 learned_addr learned my addr 172.21.15.32:0/211614104 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:21.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.374+0000 7f4832ffd700 1 --2- 172.21.15.32:0/211614104 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48340ab710 0x7f4834066540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:21.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.374+0000 7f48327fc700 1 --2- 172.21.15.32:0/211614104 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48340b1fc0 0x7f4834066a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:21.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.374+0000 7f48337fe700 1 -- 172.21.15.32:0/211614104 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48340ab710 msgr2=0x7f4834066540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:21.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.374+0000 7f48337fe700 1 --2- 172.21.15.32:0/211614104 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48340ab710 0x7f4834066540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:21.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.374+0000 7f48337fe700 1 -- 172.21.15.32:0/211614104 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48340b1fc0 msgr2=0x7f4834066a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:21.385 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.374+0000 7f48337fe700 1 --2- 172.21.15.32:0/211614104 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48340b1fc0 0x7f4834066a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:21.385 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.374+0000 7f48337fe700 1 -- 172.21.15.32:0/211614104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4834063f90 con 0x7f4834147400 2020-12-05T17:41:21.385 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.374+0000 7f4832ffd700 1 --2- 172.21.15.32:0/211614104 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48340ab710 0x7f4834066540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:21.385 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.374+0000 7f48337fe700 1 --2- 172.21.15.32:0/211614104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4834147400 0x7f4834148c60 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7f482c001210 tx=0x7f482c007f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:21.385 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.374+0000 7f482bfff700 1 -- 172.21.15.32:0/211614104 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f482c006860 con 0x7f4834147400 2020-12-05T17:41:21.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.374+0000 7f482bfff700 1 -- 172.21.15.32:0/211614104 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482c0069c0 con 0x7f4834147400 2020-12-05T17:41:21.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.374+0000 7f482bfff700 1 -- 172.21.15.32:0/211614104 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f482c006d20 con 0x7f4834147400 2020-12-05T17:41:21.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.374+0000 7f4833fff700 1 -- 172.21.15.32:0/211614104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48340ae5e0 con 0x7f4834147400 2020-12-05T17:41:21.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.374+0000 7f4833fff700 1 -- 172.21.15.32:0/211614104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4834149890 con 0x7f4834147400 2020-12-05T17:41:21.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.374+0000 7f4833fff700 1 -- 172.21.15.32:0/211614104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4834057050 con 0x7f4834147400 2020-12-05T17:41:21.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.374+0000 7f482bfff700 1 -- 172.21.15.32:0/211614104 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f482c010030 con 0x7f4834147400 2020-12-05T17:41:21.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.374+0000 7f482bfff700 1 --2- 172.21.15.32:0/211614104 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4818055440 0x7f4818057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:21.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.374+0000 7f482bfff700 1 -- 172.21.15.32:0/211614104 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f482c003070 con 0x7f4834147400 2020-12-05T17:41:21.388 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.374+0000 7f4832ffd700 1 --2- 172.21.15.32:0/211614104 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4818055440 0x7f4818057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:21.388 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.378+0000 7f4832ffd700 1 --2- 172.21.15.32:0/211614104 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4818055440 0x7f4818057900 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f481c00d000 tx=0x7f481c007b10).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:21.388 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.382+0000 7f482bfff700 1 -- 172.21.15.32:0/211614104 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f482c0447e0 con 0x7f4834147400 2020-12-05T17:41:21.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.522+0000 7f4833fff700 1 -- 172.21.15.32:0/211614104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f483405aab0 con 0x7f4834147400 2020-12-05T17:41:21.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.530+0000 7f482bfff700 1 -- 172.21.15.32:0/211614104 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f482c047e00 con 0x7f4834147400 2020-12-05T17:41:21.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.534+0000 7f4833fff700 1 -- 172.21.15.32:0/211614104 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4818055440 msgr2=0x7f4818057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.534+0000 7f4833fff700 1 --2- 172.21.15.32:0/211614104 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4818055440 0x7f4818057900 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f481c00d000 tx=0x7f481c007b10).stop 2020-12-05T17:41:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.534+0000 7f4833fff700 1 -- 172.21.15.32:0/211614104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4834147400 msgr2=0x7f4834148c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.534+0000 7f4833fff700 1 --2- 172.21.15.32:0/211614104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4834147400 0x7f4834148c60 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7f482c001210 tx=0x7f482c007f50).stop 2020-12-05T17:41:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.534+0000 7f4833fff700 1 -- 172.21.15.32:0/211614104 shutdown_connections 2020-12-05T17:41:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.534+0000 7f4833fff700 1 --2- 172.21.15.32:0/211614104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4834147400 0x7f4834148c60 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.534+0000 7f4833fff700 1 --2- 172.21.15.32:0/211614104 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4818055440 0x7f4818057900 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.534+0000 7f4833fff700 1 --2- 172.21.15.32:0/211614104 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48340ab710 0x7f4834066540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.534+0000 7f4833fff700 1 --2- 172.21.15.32:0/211614104 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48340b1fc0 0x7f4834066a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.534+0000 7f4833fff700 1 -- 172.21.15.32:0/211614104 >> 172.21.15.32:0/211614104 conn(0x7f48340b5530 msgr2=0x7f48340af870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:21.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.534+0000 7f4833fff700 1 -- 172.21.15.32:0/211614104 shutdown_connections 2020-12-05T17:41:21.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.534+0000 7f4833fff700 1 -- 172.21.15.32:0/211614104 wait complete. 2020-12-05T17:41:21.551 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps' 2020-12-05T17:41:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.654+0000 7f9508928700 1 Processor -- start 2020-12-05T17:41:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.654+0000 7f9508928700 1 -- start start 2020-12-05T17:41:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.654+0000 7f9508928700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f95040af0d0 0x7f95040b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.654+0000 7f9508928700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f950413f670 0x7f950413fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.654+0000 7f9508928700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95040b23a0 con 0x7f95040b1fc0 2020-12-05T17:41:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.654+0000 7f9508928700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f950413f280 con 0x7f95040af0d0 2020-12-05T17:41:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.654+0000 7f9508928700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95040abfa0 con 0x7f950413f670 2020-12-05T17:41:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f95037fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f95040b1fc0 0x7f950413c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43232/0 (socket says 172.21.15.32:43232) 2020-12-05T17:41:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f95037fe700 1 -- 172.21.15.32:0/2136234364 learned_addr learned my addr 172.21.15.32:0/2136234364 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f9503fff700 1 --2- 172.21.15.32:0/2136234364 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f950413f670 0x7f950413fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f94faffd700 1 --2- 172.21.15.32:0/2136234364 >> v2:172.21.15.61:3300/0 conn(0x7f95040af0d0 0x7f95040b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f9502ffd700 1 -- 172.21.15.32:0/2136234364 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804391814 0 0) 0x7f95040b23a0 con 0x7f95040b1fc0 2020-12-05T17:41:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f9502ffd700 1 -- 172.21.15.32:0/2136234364 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94e4003230 con 0x7f95040b1fc0 2020-12-05T17:41:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f9503fff700 1 -- 172.21.15.32:0/2136234364 >> v2:172.21.15.61:3300/0 conn(0x7f95040af0d0 msgr2=0x7f95040b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f9503fff700 1 --2- 172.21.15.32:0/2136234364 >> v2:172.21.15.61:3300/0 conn(0x7f95040af0d0 0x7f95040b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f9503fff700 1 -- 172.21.15.32:0/2136234364 >> v1:172.21.15.32:6789/0 conn(0x7f95040b1fc0 legacy=0x7f950413c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f9503fff700 1 -- 172.21.15.32:0/2136234364 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95040ac240 con 0x7f950413f670 2020-12-05T17:41:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f9503fff700 1 --2- 172.21.15.32:0/2136234364 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f950413f670 0x7f950413fac0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f94fc0032b0 tx=0x7f94fc0091a0).ready entity=mon.2 client_cookie=9fce38c3b8ab241f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f9502ffd700 1 -- 172.21.15.32:0/2136234364 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94fc00b940 con 0x7f950413f670 2020-12-05T17:41:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f9502ffd700 1 -- 172.21.15.32:0/2136234364 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94fc00baa0 con 0x7f950413f670 2020-12-05T17:41:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f9502ffd700 1 -- 172.21.15.32:0/2136234364 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94fc00bc10 con 0x7f950413f670 2020-12-05T17:41:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f9508928700 1 -- 172.21.15.32:0/2136234364 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f950413f670 msgr2=0x7f950413fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f9508928700 1 --2- 172.21.15.32:0/2136234364 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f950413f670 0x7f950413fac0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f94fc0032b0 tx=0x7f94fc0091a0).stop 2020-12-05T17:41:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f9508928700 1 -- 172.21.15.32:0/2136234364 shutdown_connections 2020-12-05T17:41:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f9508928700 1 --2- 172.21.15.32:0/2136234364 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f950413f670 0x7f950413fac0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f9508928700 1 --2- 172.21.15.32:0/2136234364 >> v2:172.21.15.61:3300/0 conn(0x7f95040af0d0 0x7f95040b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f9508928700 1 -- 172.21.15.32:0/2136234364 >> 172.21.15.32:0/2136234364 conn(0x7f95040b5530 msgr2=0x7f95040b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f9508928700 1 -- 172.21.15.32:0/2136234364 shutdown_connections 2020-12-05T17:41:21.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f9508928700 1 -- 172.21.15.32:0/2136234364 wait complete. 2020-12-05T17:41:21.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f9508928700 1 Processor -- start 2020-12-05T17:41:21.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.658+0000 7f9508928700 1 -- start start 2020-12-05T17:41:21.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f9508928700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f95040af0d0 0x7f95040b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f9508928700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f95040b1fc0 0x7f950412fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f9508928700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950413f670 0x7f95041303d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f9508928700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950413b8c0 con 0x7f950413f670 2020-12-05T17:41:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f9508928700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950413bb60 con 0x7f95040b1fc0 2020-12-05T17:41:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f9508928700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9504135d40 con 0x7f95040af0d0 2020-12-05T17:41:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f9503fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950413f670 0x7f95041303d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f95037fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f95040af0d0 0x7f95040b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f9503fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950413f670 0x7f95041303d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40590/0 (socket says 172.21.15.32:40590) 2020-12-05T17:41:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f95037fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f95040af0d0 0x7f95040b33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:37376/0 (socket says 172.21.15.32:37376) 2020-12-05T17:41:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f9503fff700 1 -- 172.21.15.32:0/2970208789 learned_addr learned my addr 172.21.15.32:0/2970208789 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:21.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f94faffd700 1 --2- 172.21.15.32:0/2970208789 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f95040b1fc0 0x7f950412fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:21.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f9503fff700 1 -- 172.21.15.32:0/2970208789 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f95040af0d0 msgr2=0x7f95040b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:21.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f9503fff700 1 --2- 172.21.15.32:0/2970208789 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f95040af0d0 0x7f95040b33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:21.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f9503fff700 1 -- 172.21.15.32:0/2970208789 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f95040b1fc0 msgr2=0x7f950412fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f9503fff700 1 --2- 172.21.15.32:0/2970208789 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f95040b1fc0 0x7f950412fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f9503fff700 1 -- 172.21.15.32:0/2970208789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9504141990 con 0x7f950413f670 2020-12-05T17:41:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f9503fff700 1 --2- 172.21.15.32:0/2970208789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950413f670 0x7f95041303d0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0x7f94fc0032b0 tx=0x7f94fc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f95017fa700 1 -- 172.21.15.32:0/2970208789 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94fc0205f0 con 0x7f950413f670 2020-12-05T17:41:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f95017fa700 1 -- 172.21.15.32:0/2970208789 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94fc020750 con 0x7f950413f670 2020-12-05T17:41:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f95017fa700 1 -- 172.21.15.32:0/2970208789 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94fc020ab0 con 0x7f950413f670 2020-12-05T17:41:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f9508928700 1 -- 172.21.15.32:0/2970208789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95041308c0 con 0x7f950413f670 2020-12-05T17:41:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f9508928700 1 -- 172.21.15.32:0/2970208789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9504143420 con 0x7f950413f670 2020-12-05T17:41:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.662+0000 7f9508928700 1 -- 172.21.15.32:0/2970208789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9504135f00 con 0x7f950413f670 2020-12-05T17:41:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.666+0000 7f95017fa700 1 -- 172.21.15.32:0/2970208789 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f94fc027030 con 0x7f950413f670 2020-12-05T17:41:21.677 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.666+0000 7f95017fa700 1 --2- 172.21.15.32:0/2970208789 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f94e8055470 0x7f94e8057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:21.677 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.666+0000 7f95037fe700 1 --2- 172.21.15.32:0/2970208789 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f94e8055470 0x7f94e8057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.666+0000 7f95017fa700 1 -- 172.21.15.32:0/2970208789 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f94fc06f310 con 0x7f950413f670 2020-12-05T17:41:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.666+0000 7f95037fe700 1 --2- 172.21.15.32:0/2970208789 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f94e8055470 0x7f94e8057930 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f94f400d000 tx=0x7f94f4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:21.679 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.674+0000 7f95017fa700 1 -- 172.21.15.32:0/2970208789 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f94fc0454f0 con 0x7f950413f670 2020-12-05T17:41:21.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.866+0000 7f9508928700 1 -- 172.21.15.32:0/2970208789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]} v 0) v1 -- 0x7f9504057050 con 0x7f950413f670 2020-12-05T17:41:21.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.914+0000 7f95017fa700 1 -- 172.21.15.32:0/2970208789 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]}]=0 v444) v1 ==== 128+0+0 (secure 0 0 0) 0x7f94fc0150c0 con 0x7f950413f670 2020-12-05T17:41:21.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.918+0000 7f9508928700 1 -- 172.21.15.32:0/2970208789 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f94e8055470 msgr2=0x7f94e8057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:21.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.918+0000 7f9508928700 1 --2- 172.21.15.32:0/2970208789 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f94e8055470 0x7f94e8057930 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f94f400d000 tx=0x7f94f4009040).stop 2020-12-05T17:41:21.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.918+0000 7f9508928700 1 -- 172.21.15.32:0/2970208789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950413f670 msgr2=0x7f95041303d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:21.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.918+0000 7f9508928700 1 --2- 172.21.15.32:0/2970208789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950413f670 0x7f95041303d0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0x7f94fc0032b0 tx=0x7f94fc016000).stop 2020-12-05T17:41:21.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.918+0000 7f9508928700 1 -- 172.21.15.32:0/2970208789 shutdown_connections 2020-12-05T17:41:21.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.918+0000 7f9508928700 1 --2- 172.21.15.32:0/2970208789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950413f670 0x7f95041303d0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:21.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.918+0000 7f9508928700 1 --2- 172.21.15.32:0/2970208789 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f94e8055470 0x7f94e8057930 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:21.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.918+0000 7f9508928700 1 --2- 172.21.15.32:0/2970208789 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f95040af0d0 0x7f95040b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:21.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.918+0000 7f9508928700 1 --2- 172.21.15.32:0/2970208789 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f95040b1fc0 0x7f950412fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:21.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.918+0000 7f9508928700 1 -- 172.21.15.32:0/2970208789 >> 172.21.15.32:0/2970208789 conn(0x7f95040b5530 msgr2=0x7f950405f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:21.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.918+0000 7f9508928700 1 -- 172.21.15.32:0/2970208789 shutdown_connections 2020-12-05T17:41:21.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:21.918+0000 7f9508928700 1 -- 172.21.15.32:0/2970208789 wait complete. 2020-12-05T17:41:21.939 INFO:tasks.cephfs_test_runner:test_getattr_caps (tasks.cephfs.test_misc.TestMisc) ... ok 2020-12-05T17:41:21.939 INFO:tasks.cephfs_test_runner:Starting test: test_root_rctime (tasks.cephfs.test_misc.TestMisc) 2020-12-05T17:41:21.940 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime' 2020-12-05T17:41:22.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.050+0000 7fec9934f700 1 Processor -- start 2020-12-05T17:41:22.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.050+0000 7fec9934f700 1 -- start start 2020-12-05T17:41:22.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.050+0000 7fec9934f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec940b3aa0 0x7fec941430f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:22.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.050+0000 7fec9934f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fec940b1880 0x7fec94146a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:22.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.050+0000 7fec9934f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec940b0230 con 0x7fec940afe50 2020-12-05T17:41:22.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.050+0000 7fec9934f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec940b6a50 con 0x7fec940b1880 2020-12-05T17:41:22.057 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.050+0000 7fec9934f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec94146080 con 0x7fec940b3aa0 2020-12-05T17:41:22.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.050+0000 7fec937fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fec940b1880 0x7fec94146a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:22.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.050+0000 7fec92ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fec940afe50 0x7fec941429e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43242/0 (socket says 172.21.15.32:43242) 2020-12-05T17:41:22.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.050+0000 7fec92ffd700 1 -- 172.21.15.32:0/2602068891 learned_addr learned my addr 172.21.15.32:0/2602068891 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:22.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.050+0000 7fec927fc700 1 --2- 172.21.15.32:0/2602068891 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec940b3aa0 0x7fec941430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:22.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.050+0000 7fec937fe700 1 -- 172.21.15.32:0/2602068891 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec940b3aa0 msgr2=0x7fec941430f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:22.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.050+0000 7fec937fe700 1 --2- 172.21.15.32:0/2602068891 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec940b3aa0 0x7fec941430f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:22.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.050+0000 7fec937fe700 1 -- 172.21.15.32:0/2602068891 >> v1:172.21.15.32:6789/0 conn(0x7fec940afe50 legacy=0x7fec941429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:22.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.050+0000 7fec91ffb700 1 -- 172.21.15.32:0/2602068891 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3782556833 0 0) 0x7fec940b0230 con 0x7fec940afe50 2020-12-05T17:41:22.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.050+0000 7fec937fe700 1 -- 172.21.15.32:0/2602068891 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec94146320 con 0x7fec940b1880 2020-12-05T17:41:22.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.054+0000 7fec937fe700 1 --2- 172.21.15.32:0/2602068891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec940b1880 0x7fec94146a60 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7fec8c002f30 tx=0x7fec8c006dc0).ready entity=mon.0 client_cookie=7e70a71cb215ef53 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:22.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.054+0000 7fec91ffb700 1 -- 172.21.15.32:0/2602068891 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec8c0097f0 con 0x7fec940b1880 2020-12-05T17:41:22.061 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.054+0000 7fec91ffb700 1 -- 172.21.15.32:0/2602068891 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8c009950 con 0x7fec940b1880 2020-12-05T17:41:22.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.054+0000 7fec91ffb700 1 -- 172.21.15.32:0/2602068891 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec8c009cb0 con 0x7fec940b1880 2020-12-05T17:41:22.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.054+0000 7fec9934f700 1 -- 172.21.15.32:0/2602068891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec940b1880 msgr2=0x7fec94146a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:22.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.054+0000 7fec9934f700 1 --2- 172.21.15.32:0/2602068891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec940b1880 0x7fec94146a60 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7fec8c002f30 tx=0x7fec8c006dc0).stop 2020-12-05T17:41:22.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.054+0000 7fec9934f700 1 -- 172.21.15.32:0/2602068891 shutdown_connections 2020-12-05T17:41:22.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.054+0000 7fec9934f700 1 --2- 172.21.15.32:0/2602068891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec940b1880 0x7fec94146a60 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:22.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.054+0000 7fec9934f700 1 --2- 172.21.15.32:0/2602068891 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec940b3aa0 0x7fec941430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:22.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.054+0000 7fec9934f700 1 -- 172.21.15.32:0/2602068891 >> 172.21.15.32:0/2602068891 conn(0x7fec940b9100 msgr2=0x7fec940b9510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:22.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.054+0000 7fec9934f700 1 -- 172.21.15.32:0/2602068891 shutdown_connections 2020-12-05T17:41:22.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.054+0000 7fec9934f700 1 -- 172.21.15.32:0/2602068891 wait complete. 2020-12-05T17:41:22.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.054+0000 7fec9934f700 1 Processor -- start 2020-12-05T17:41:22.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.054+0000 7fec9934f700 1 -- start start 2020-12-05T17:41:22.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec9934f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec940afe50 0x7fec94148820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:22.066 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec9934f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec940b1880 0x7fec94066510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:22.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec9934f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec940b3aa0 0x7fec94066a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:22.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec9934f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec94066ef0 con 0x7fec940afe50 2020-12-05T17:41:22.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec9934f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9413f0f0 con 0x7fec940b1880 2020-12-05T17:41:22.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec9934f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec94064270 con 0x7fec940b3aa0 2020-12-05T17:41:22.067 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec92ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec940afe50 0x7fec94148820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:22.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec92ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec940afe50 0x7fec94148820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40600/0 (socket says 172.21.15.32:40600) 2020-12-05T17:41:22.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec92ffd700 1 -- 172.21.15.32:0/3057260733 learned_addr learned my addr 172.21.15.32:0/3057260733 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:22.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec927fc700 1 --2- 172.21.15.32:0/3057260733 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec940b1880 0x7fec94066510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:22.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec937fe700 1 --2- 172.21.15.32:0/3057260733 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec940b3aa0 0x7fec94066a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec92ffd700 1 -- 172.21.15.32:0/3057260733 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec940b3aa0 msgr2=0x7fec94066a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec92ffd700 1 --2- 172.21.15.32:0/3057260733 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec940b3aa0 0x7fec94066a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec92ffd700 1 -- 172.21.15.32:0/3057260733 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec940b1880 msgr2=0x7fec94066510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec92ffd700 1 --2- 172.21.15.32:0/3057260733 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec940b1880 0x7fec94066510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec92ffd700 1 -- 172.21.15.32:0/3057260733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec940644f0 con 0x7fec940afe50 2020-12-05T17:41:22.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec92ffd700 1 --2- 172.21.15.32:0/3057260733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec940afe50 0x7fec94148820 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7fec7c000fa0 tx=0x7fec7c007b20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:22.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec8bfff700 1 -- 172.21.15.32:0/3057260733 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec7c0072e0 con 0x7fec940afe50 2020-12-05T17:41:22.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec8bfff700 1 -- 172.21.15.32:0/3057260733 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c007440 con 0x7fec940afe50 2020-12-05T17:41:22.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec8bfff700 1 -- 172.21.15.32:0/3057260733 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec7c0057d0 con 0x7fec940afe50 2020-12-05T17:41:22.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec9934f700 1 -- 172.21.15.32:0/3057260733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec94064790 con 0x7fec940afe50 2020-12-05T17:41:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec9934f700 1 -- 172.21.15.32:0/3057260733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec94064ed0 con 0x7fec940afe50 2020-12-05T17:41:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec9934f700 1 -- 172.21.15.32:0/3057260733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec941431a0 con 0x7fec940afe50 2020-12-05T17:41:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.058+0000 7fec8bfff700 1 -- 172.21.15.32:0/3057260733 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fec7c010070 con 0x7fec940afe50 2020-12-05T17:41:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.062+0000 7fec8bfff700 1 --2- 172.21.15.32:0/3057260733 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fec78055440 0x7fec78057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.062+0000 7fec8bfff700 1 -- 172.21.15.32:0/3057260733 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fec7c06d7d0 con 0x7fec940afe50 2020-12-05T17:41:22.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.062+0000 7fec927fc700 1 --2- 172.21.15.32:0/3057260733 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fec78055440 0x7fec78057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:22.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.062+0000 7fec927fc700 1 --2- 172.21.15.32:0/3057260733 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fec78055440 0x7fec78057900 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7fec8400c000 tx=0x7fec84008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.070+0000 7fec8bfff700 1 -- 172.21.15.32:0/3057260733 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fec941431a0 con 0x7fec940afe50 2020-12-05T17:41:22.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.262+0000 7fec9934f700 1 -- 172.21.15.32:0/3057260733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]} v 0) v1 -- 0x7fec941431a0 con 0x7fec940afe50 2020-12-05T17:41:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.918+0000 7fec8bfff700 1 -- 172.21.15.32:0/3057260733 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]}]=0 v445) v1 ==== 130+0+0 (secure 0 0 0) 0x7fec941431a0 con 0x7fec940afe50 2020-12-05T17:41:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.922+0000 7fec9934f700 1 -- 172.21.15.32:0/3057260733 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fec78055440 msgr2=0x7fec78057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.922+0000 7fec9934f700 1 --2- 172.21.15.32:0/3057260733 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fec78055440 0x7fec78057900 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7fec8400c000 tx=0x7fec84008040).stop 2020-12-05T17:41:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.922+0000 7fec9934f700 1 -- 172.21.15.32:0/3057260733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec940afe50 msgr2=0x7fec94148820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.922+0000 7fec9934f700 1 --2- 172.21.15.32:0/3057260733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec940afe50 0x7fec94148820 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7fec7c000fa0 tx=0x7fec7c007b20).stop 2020-12-05T17:41:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.922+0000 7fec9934f700 1 -- 172.21.15.32:0/3057260733 shutdown_connections 2020-12-05T17:41:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.922+0000 7fec9934f700 1 --2- 172.21.15.32:0/3057260733 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fec78055440 0x7fec78057900 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.922+0000 7fec9934f700 1 --2- 172.21.15.32:0/3057260733 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec940b3aa0 0x7fec94066a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.922+0000 7fec9934f700 1 --2- 172.21.15.32:0/3057260733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec940afe50 0x7fec94148820 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.922+0000 7fec9934f700 1 --2- 172.21.15.32:0/3057260733 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec940b1880 0x7fec94066510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.922+0000 7fec9934f700 1 -- 172.21.15.32:0/3057260733 >> 172.21.15.32:0/3057260733 conn(0x7fec940b9100 msgr2=0x7fec940b41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:22.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.922+0000 7fec9934f700 1 -- 172.21.15.32:0/3057260733 shutdown_connections 2020-12-05T17:41:22.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:22.922+0000 7fec9934f700 1 -- 172.21.15.32:0/3057260733 wait complete. 2020-12-05T17:41:22.947 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-12-05T17:41:23.068 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.062+0000 7fb153094700 1 Processor -- start 2020-12-05T17:41:23.069 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.062+0000 7fb153094700 1 -- start start 2020-12-05T17:41:23.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb153094700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb14c0b1720 0x7fb14c14a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:23.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb153094700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb14c14aeb0 0x7fb14c1460b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:23.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb153094700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb14c061f80 con 0x7fb14c061ba0 2020-12-05T17:41:23.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb153094700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb14c0b29a0 con 0x7fb14c14aeb0 2020-12-05T17:41:23.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb153094700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c147ad0 con 0x7fb14c0b1720 2020-12-05T17:41:23.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb152092700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb14c061ba0 0x7fb14c0b2290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43256/0 (socket says 172.21.15.32:43256) 2020-12-05T17:41:23.072 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb152092700 1 -- 172.21.15.32:0/1936241460 learned_addr learned my addr 172.21.15.32:0/1936241460 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:23.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb152893700 1 --2- 172.21.15.32:0/1936241460 >> v2:172.21.15.61:3300/0 conn(0x7fb14c14aeb0 0x7fb14c1460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:23.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb151891700 1 --2- 172.21.15.32:0/1936241460 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb14c0b1720 0x7fb14c14a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:23.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb151090700 1 -- 172.21.15.32:0/1936241460 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1770889731 0 0) 0x7fb14c061f80 con 0x7fb14c061ba0 2020-12-05T17:41:23.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb151090700 1 -- 172.21.15.32:0/1936241460 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb138003230 con 0x7fb14c061ba0 2020-12-05T17:41:23.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb151891700 1 -- 172.21.15.32:0/1936241460 >> v2:172.21.15.61:3300/0 conn(0x7fb14c14aeb0 msgr2=0x7fb14c1460b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:23.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb151891700 1 --2- 172.21.15.32:0/1936241460 >> v2:172.21.15.61:3300/0 conn(0x7fb14c14aeb0 0x7fb14c1460b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:23.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb151891700 1 -- 172.21.15.32:0/1936241460 >> v1:172.21.15.32:6789/0 conn(0x7fb14c061ba0 legacy=0x7fb14c0b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:23.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb151891700 1 -- 172.21.15.32:0/1936241460 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14c14a280 con 0x7fb14c0b1720 2020-12-05T17:41:23.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb152893700 1 --2- 172.21.15.32:0/1936241460 >> v2:172.21.15.61:3300/0 conn(0x7fb14c14aeb0 0x7fb14c1460b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:23.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb151891700 1 --2- 172.21.15.32:0/1936241460 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb14c0b1720 0x7fb14c14a9c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fb1480032b0 tx=0x7fb1480091a0).ready entity=mon.2 client_cookie=f570a3cf9667e05 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:23.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb151090700 1 -- 172.21.15.32:0/1936241460 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb14800b940 con 0x7fb14c0b1720 2020-12-05T17:41:23.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb151090700 1 -- 172.21.15.32:0/1936241460 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14800baa0 con 0x7fb14c0b1720 2020-12-05T17:41:23.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb151090700 1 -- 172.21.15.32:0/1936241460 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb14800bc10 con 0x7fb14c0b1720 2020-12-05T17:41:23.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb153094700 1 -- 172.21.15.32:0/1936241460 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb14c0b1720 msgr2=0x7fb14c14a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:23.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb153094700 1 --2- 172.21.15.32:0/1936241460 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb14c0b1720 0x7fb14c14a9c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fb1480032b0 tx=0x7fb1480091a0).stop 2020-12-05T17:41:23.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb153094700 1 -- 172.21.15.32:0/1936241460 shutdown_connections 2020-12-05T17:41:23.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb153094700 1 --2- 172.21.15.32:0/1936241460 >> v2:172.21.15.61:3300/0 conn(0x7fb14c14aeb0 0x7fb14c1460b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:23.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb153094700 1 --2- 172.21.15.32:0/1936241460 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb14c0b1720 0x7fb14c14a9c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:23.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb153094700 1 -- 172.21.15.32:0/1936241460 >> 172.21.15.32:0/1936241460 conn(0x7fb14c0b6d10 msgr2=0x7fb14c0b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:23.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb153094700 1 -- 172.21.15.32:0/1936241460 shutdown_connections 2020-12-05T17:41:23.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.066+0000 7fb153094700 1 -- 172.21.15.32:0/1936241460 wait complete. 2020-12-05T17:41:23.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.070+0000 7fb153094700 1 Processor -- start 2020-12-05T17:41:23.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.070+0000 7fb153094700 1 -- start start 2020-12-05T17:41:23.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.070+0000 7fb153094700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb14c061ba0 0x7fb14c065c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:23.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.070+0000 7fb153094700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb14c0b1720 0x7fb14c0acec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:23.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.070+0000 7fb153094700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb14c14aeb0 0x7fb14c14b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:23.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.070+0000 7fb153094700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c142a50 con 0x7fb14c0b1720 2020-12-05T17:41:23.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.070+0000 7fb153094700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c0ad7f0 con 0x7fb14c14aeb0 2020-12-05T17:41:23.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.070+0000 7fb153094700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c0657b0 con 0x7fb14c061ba0 2020-12-05T17:41:23.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.070+0000 7fb151891700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb14c0b1720 0x7fb14c0acec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:23.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.070+0000 7fb151891700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb14c0b1720 0x7fb14c0acec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40616/0 (socket says 172.21.15.32:40616) 2020-12-05T17:41:23.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.070+0000 7fb151891700 1 -- 172.21.15.32:0/1448753288 learned_addr learned my addr 172.21.15.32:0/1448753288 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:23.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.070+0000 7fb152092700 1 --2- 172.21.15.32:0/1448753288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb14c061ba0 0x7fb14c065c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:23.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.070+0000 7fb152893700 1 --2- 172.21.15.32:0/1448753288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb14c14aeb0 0x7fb14c14b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:23.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.070+0000 7fb151891700 1 -- 172.21.15.32:0/1448753288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb14c061ba0 msgr2=0x7fb14c065c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:23.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.070+0000 7fb151891700 1 --2- 172.21.15.32:0/1448753288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb14c061ba0 0x7fb14c065c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:23.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.070+0000 7fb151891700 1 -- 172.21.15.32:0/1448753288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb14c14aeb0 msgr2=0x7fb14c14b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:23.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.070+0000 7fb151891700 1 --2- 172.21.15.32:0/1448753288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb14c14aeb0 0x7fb14c14b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:23.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.070+0000 7fb151891700 1 -- 172.21.15.32:0/1448753288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14c14d9d0 con 0x7fb14c0b1720 2020-12-05T17:41:23.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.070+0000 7fb151891700 1 --2- 172.21.15.32:0/1448753288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb14c0b1720 0x7fb14c0acec0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7fb1480032b0 tx=0x7fb148016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:23.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.070+0000 7fb1437fe700 1 -- 172.21.15.32:0/1448753288 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb148011e60 con 0x7fb14c0b1720 2020-12-05T17:41:23.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.070+0000 7fb1437fe700 1 -- 172.21.15.32:0/1448753288 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14801f3f0 con 0x7fb14c0b1720 2020-12-05T17:41:23.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.070+0000 7fb153094700 1 -- 172.21.15.32:0/1448753288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb14c0ad3b0 con 0x7fb14c0b1720 2020-12-05T17:41:23.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.074+0000 7fb1437fe700 1 -- 172.21.15.32:0/1448753288 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb14801f750 con 0x7fb14c0b1720 2020-12-05T17:41:23.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.074+0000 7fb153094700 1 -- 172.21.15.32:0/1448753288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14c14dfc0 con 0x7fb14c0b1720 2020-12-05T17:41:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.074+0000 7fb1437fe700 1 -- 172.21.15.32:0/1448753288 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb14800f070 con 0x7fb14c0b1720 2020-12-05T17:41:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.074+0000 7fb153094700 1 -- 172.21.15.32:0/1448753288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb14c14dbc0 con 0x7fb14c0b1720 2020-12-05T17:41:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.074+0000 7fb1437fe700 1 --2- 172.21.15.32:0/1448753288 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb12c055470 0x7fb12c057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.074+0000 7fb152092700 1 --2- 172.21.15.32:0/1448753288 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb12c055470 0x7fb12c057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.074+0000 7fb1437fe700 1 -- 172.21.15.32:0/1448753288 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb14806e870 con 0x7fb14c0b1720 2020-12-05T17:41:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.074+0000 7fb152092700 1 --2- 172.21.15.32:0/1448753288 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb12c055470 0x7fb12c057930 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fb13c00d000 tx=0x7fb13c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.082+0000 7fb1437fe700 1 -- 172.21.15.32:0/1448753288 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb14c14dbc0 con 0x7fb14c0b1720 2020-12-05T17:41:23.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.222+0000 7fb153094700 1 -- 172.21.15.32:0/1448753288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7fb14c065970 con 0x7fb14c0b1720 2020-12-05T17:41:23.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.230+0000 7fb1437fe700 1 -- 172.21.15.32:0/1448753288 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb1480240c0 con 0x7fb14c0b1720 2020-12-05T17:41:23.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.234+0000 7fb153094700 1 -- 172.21.15.32:0/1448753288 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb12c055470 msgr2=0x7fb12c057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:23.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.234+0000 7fb153094700 1 --2- 172.21.15.32:0/1448753288 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb12c055470 0x7fb12c057930 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fb13c00d000 tx=0x7fb13c009040).stop 2020-12-05T17:41:23.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.234+0000 7fb153094700 1 -- 172.21.15.32:0/1448753288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb14c0b1720 msgr2=0x7fb14c0acec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:23.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.234+0000 7fb153094700 1 --2- 172.21.15.32:0/1448753288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb14c0b1720 0x7fb14c0acec0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7fb1480032b0 tx=0x7fb148016000).stop 2020-12-05T17:41:23.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.234+0000 7fb153094700 1 -- 172.21.15.32:0/1448753288 shutdown_connections 2020-12-05T17:41:23.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.234+0000 7fb153094700 1 --2- 172.21.15.32:0/1448753288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb14c14aeb0 0x7fb14c14b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:23.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.234+0000 7fb153094700 1 --2- 172.21.15.32:0/1448753288 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb12c055470 0x7fb12c057930 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:23.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.234+0000 7fb153094700 1 --2- 172.21.15.32:0/1448753288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb14c061ba0 0x7fb14c065c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:23.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.234+0000 7fb153094700 1 --2- 172.21.15.32:0/1448753288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb14c0b1720 0x7fb14c0acec0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:23.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.234+0000 7fb153094700 1 -- 172.21.15.32:0/1448753288 >> 172.21.15.32:0/1448753288 conn(0x7fb14c0b6d10 msgr2=0x7fb14c146d90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:23.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.234+0000 7fb153094700 1 -- 172.21.15.32:0/1448753288 shutdown_connections 2020-12-05T17:41:23.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.234+0000 7fb153094700 1 -- 172.21.15.32:0/1448753288 wait complete. 2020-12-05T17:41:23.255 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:41:23.256 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:41:23.273 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:41:23.288 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:41:23.305 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:41:23.322 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:41:23.338 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:41:23.338 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:23.339 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:23.339 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:41:23.357 INFO:teuthology.orchestra.run.smithi036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-12-05T17:41:23.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:41:23.359 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-12-05T17:41:23.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:23.360 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:23.360 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:23.367 INFO:teuthology.orchestra.run.smithi002.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-12-05T17:41:23.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:41:23.368 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-12-05T17:41:23.369 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:23.369 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:23.370 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:41:23.377 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:41:23.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:41:23.378 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2020-12-05T17:41:23.378 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-12-05T17:41:23.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.494+0000 7fdb49c03700 1 Processor -- start 2020-12-05T17:41:23.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.494+0000 7fdb49c03700 1 -- start start 2020-12-05T17:41:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb49c03700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb440b1fc0 0x7fdb440af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:23.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb49c03700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fdb440ab710 0x7fdb44061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:23.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb49c03700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb44146a90 con 0x7fdb44147400 2020-12-05T17:41:23.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb49c03700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb440b0110 con 0x7fdb440b1fc0 2020-12-05T17:41:23.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb49c03700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb440b23a0 con 0x7fdb440ab710 2020-12-05T17:41:23.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb48c01700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb440b1fc0 0x7fdb440af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:23.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb48c01700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb440b1fc0 0x7fdb440af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40624/0 (socket says 172.21.15.32:40624) 2020-12-05T17:41:23.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb48c01700 1 -- 172.21.15.32:0/3409880160 learned_addr learned my addr 172.21.15.32:0/3409880160 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:23.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb3bfff700 1 --2- 172.21.15.32:0/3409880160 >> v2:172.21.15.61:3300/0 conn(0x7fdb440ab710 0x7fdb44061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:23.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb48c01700 1 -- 172.21.15.32:0/3409880160 >> v2:172.21.15.61:3300/0 conn(0x7fdb440ab710 msgr2=0x7fdb44061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:23.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb48c01700 1 --2- 172.21.15.32:0/3409880160 >> v2:172.21.15.61:3300/0 conn(0x7fdb440ab710 0x7fdb44061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:23.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb48c01700 1 -- 172.21.15.32:0/3409880160 >> v1:172.21.15.61:6789/0 conn(0x7fdb44147400 legacy=0x7fdb44145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:23.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb48c01700 1 -- 172.21.15.32:0/3409880160 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb44146cc0 con 0x7fdb440b1fc0 2020-12-05T17:41:23.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb3bfff700 1 --2- 172.21.15.32:0/3409880160 >> v2:172.21.15.61:3300/0 conn(0x7fdb440ab710 0x7fdb44061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:23.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb48c01700 1 --2- 172.21.15.32:0/3409880160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb440b1fc0 0x7fdb440af230 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7fdb34002f30 tx=0x7fdb34006e80).ready entity=mon.0 client_cookie=1b93e30f25f0eeec server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:23.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb3b7fe700 1 -- 172.21.15.32:0/3409880160 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb340097f0 con 0x7fdb440b1fc0 2020-12-05T17:41:23.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb3b7fe700 1 -- 172.21.15.32:0/3409880160 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb34009950 con 0x7fdb440b1fc0 2020-12-05T17:41:23.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb3b7fe700 1 -- 172.21.15.32:0/3409880160 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb34009cb0 con 0x7fdb440b1fc0 2020-12-05T17:41:23.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb49c03700 1 -- 172.21.15.32:0/3409880160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb440b1fc0 msgr2=0x7fdb440af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:23.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb49c03700 1 --2- 172.21.15.32:0/3409880160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb440b1fc0 0x7fdb440af230 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7fdb34002f30 tx=0x7fdb34006e80).stop 2020-12-05T17:41:23.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb49c03700 1 -- 172.21.15.32:0/3409880160 shutdown_connections 2020-12-05T17:41:23.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb49c03700 1 --2- 172.21.15.32:0/3409880160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb440b1fc0 0x7fdb440af230 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:23.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb49c03700 1 --2- 172.21.15.32:0/3409880160 >> v2:172.21.15.61:3300/0 conn(0x7fdb440ab710 0x7fdb44061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:23.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb49c03700 1 -- 172.21.15.32:0/3409880160 >> 172.21.15.32:0/3409880160 conn(0x7fdb440b5530 msgr2=0x7fdb440b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:23.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb49c03700 1 -- 172.21.15.32:0/3409880160 shutdown_connections 2020-12-05T17:41:23.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.498+0000 7fdb49c03700 1 -- 172.21.15.32:0/3409880160 wait complete. 2020-12-05T17:41:23.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb49c03700 1 Processor -- start 2020-12-05T17:41:23.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb49c03700 1 -- start start 2020-12-05T17:41:23.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb49c03700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb440ab710 0x7fdb440acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:23.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb49c03700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb440b1fc0 0x7fdb44148620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:23.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb49c03700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb44147400 0x7fdb440ad490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:23.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb49c03700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb440ad980 con 0x7fdb440ab710 2020-12-05T17:41:23.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb49c03700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb440adba0 con 0x7fdb44147400 2020-12-05T17:41:23.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb49c03700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb440ac920 con 0x7fdb440b1fc0 2020-12-05T17:41:23.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb48c01700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb440ab710 0x7fdb440acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:23.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb48c01700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb440ab710 0x7fdb440acdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40630/0 (socket says 172.21.15.32:40630) 2020-12-05T17:41:23.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb49402700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb44147400 0x7fdb440ad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:23.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb48c01700 1 -- 172.21.15.32:0/2772453456 learned_addr learned my addr 172.21.15.32:0/2772453456 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:23.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb3bfff700 1 --2- 172.21.15.32:0/2772453456 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb440b1fc0 0x7fdb44148620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:23.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb48c01700 1 -- 172.21.15.32:0/2772453456 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb440b1fc0 msgr2=0x7fdb44148620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:23.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb48c01700 1 --2- 172.21.15.32:0/2772453456 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb440b1fc0 0x7fdb44148620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:23.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb48c01700 1 -- 172.21.15.32:0/2772453456 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb44147400 msgr2=0x7fdb440ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:23.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb48c01700 1 --2- 172.21.15.32:0/2772453456 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb44147400 0x7fdb440ad490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:23.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb48c01700 1 -- 172.21.15.32:0/2772453456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb440a1c30 con 0x7fdb440ab710 2020-12-05T17:41:23.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb48c01700 1 --2- 172.21.15.32:0/2772453456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb440ab710 0x7fdb440acdc0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7fdb34002f30 tx=0x7fdb34015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:23.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb39ffb700 1 -- 172.21.15.32:0/2772453456 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb34011b80 con 0x7fdb440ab710 2020-12-05T17:41:23.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb39ffb700 1 -- 172.21.15.32:0/2772453456 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb34011ce0 con 0x7fdb440ab710 2020-12-05T17:41:23.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb49c03700 1 -- 172.21.15.32:0/2772453456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb440a1e80 con 0x7fdb440ab710 2020-12-05T17:41:23.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb39ffb700 1 -- 172.21.15.32:0/2772453456 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb3401e650 con 0x7fdb440ab710 2020-12-05T17:41:23.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.502+0000 7fdb49c03700 1 -- 172.21.15.32:0/2772453456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4409e6c0 con 0x7fdb440ab710 2020-12-05T17:41:23.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.506+0000 7fdb49c03700 1 -- 172.21.15.32:0/2772453456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb440add60 con 0x7fdb440ab710 2020-12-05T17:41:23.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.506+0000 7fdb39ffb700 1 -- 172.21.15.32:0/2772453456 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fdb34025030 con 0x7fdb440ab710 2020-12-05T17:41:23.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.506+0000 7fdb39ffb700 1 --2- 172.21.15.32:0/2772453456 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb24055330 0x7fdb240577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:23.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.506+0000 7fdb39ffb700 1 -- 172.21.15.32:0/2772453456 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdb3406f020 con 0x7fdb440ab710 2020-12-05T17:41:23.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.506+0000 7fdb3bfff700 1 --2- 172.21.15.32:0/2772453456 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb24055330 0x7fdb240577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:23.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.506+0000 7fdb3bfff700 1 --2- 172.21.15.32:0/2772453456 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb24055330 0x7fdb240577f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7fdb4000b000 tx=0x7fdb40006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:23.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.514+0000 7fdb39ffb700 1 -- 172.21.15.32:0/2772453456 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fdb340451c0 con 0x7fdb440ab710 2020-12-05T17:41:23.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.658+0000 7fdb49c03700 1 -- 172.21.15.32:0/2772453456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fdb44057050 con 0x7fdb440ab710 2020-12-05T17:41:23.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.662+0000 7fdb39ffb700 1 -- 172.21.15.32:0/2772453456 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v129) v1 ==== 81+0+34051 (secure 0 0 0) 0x7fdb34023020 con 0x7fdb440ab710 2020-12-05T17:41:23.668 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:23.668 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:41:23.668 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 129, 2020-12-05T17:41:23.668 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", 2020-12-05T17:41:23.669 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2020-12-05T17:31:20.975238+0000", 2020-12-05T17:41:23.669 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2020-12-05T17:41:04.284002+0000", 2020-12-05T17:41:23.669 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:41:23.669 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:41:23.670 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-12-05T17:41:23.670 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2020-12-05T17:41:23.670 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2020-12-05T17:41:23.670 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2020-12-05T17:41:23.670 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2020-12-05T17:41:23.671 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2020-12-05T17:41:23.671 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2020-12-05T17:41:23.671 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:41:23.671 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2020-12-05T17:41:23.672 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2020-12-05T17:41:23.672 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-12-05T17:41:23.672 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2020-12-05T17:41:23.672 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2020-12-05T17:41:23.672 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 26, 2020-12-05T17:41:23.673 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2020-12-05T17:41:23.673 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2020-12-05T17:41:23.673 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2020-12-05T17:41:23.673 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2020-12-05T17:41:23.673 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2020-12-05T17:41:23.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.674 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2020-12-05T17:41:23.674 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2020-12-05T17:41:23.674 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:24.858732+0000", 2020-12-05T17:41:23.674 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:41:23.675 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:41:23.675 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:41:23.675 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:41:23.675 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:41:23.676 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:41:23.676 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:41:23.676 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:41:23.676 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2020-12-05T17:41:23.676 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2020-12-05T17:41:23.677 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2020-12-05T17:41:23.677 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2020-12-05T17:41:23.677 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2020-12-05T17:41:23.677 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:41:23.678 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:41:23.678 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:41:23.678 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:41:23.678 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:41:23.678 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:41:23.679 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:41:23.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.679 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "16", 2020-12-05T17:41:23.679 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:41:23.679 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:41:23.680 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:41:23.680 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:41:23.680 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:41:23.680 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:41:23.680 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:41:23.681 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:41:23.681 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:41:23.681 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:41:23.681 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:41:23.681 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:41:23.682 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:41:23.682 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:41:23.682 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:41:23.682 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:41:23.683 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:41:23.683 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:41:23.683 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:41:23.683 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:41:23.683 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:41:23.684 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:41:23.684 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:41:23.684 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:41:23.684 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:41:23.684 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:41:23.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.685 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:41:23.685 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:41:23.685 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:41:23.685 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:41:23.686 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:41:23.686 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:41:23.686 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:41:23.686 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:41:23.687 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:41:23.687 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:41:23.687 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:41:23.687 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:41:23.687 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2020-12-05T17:41:23.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.688 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:41:23.688 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2020-12-05T17:41:23.688 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.689 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2020-12-05T17:41:23.689 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2020-12-05T17:41:23.689 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:33.305595+0000", 2020-12-05T17:41:23.689 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:41:23.690 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:41:23.690 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:41:23.690 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:41:23.690 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:41:23.690 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:41:23.691 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:41:23.691 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:41:23.691 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:41:23.691 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:41:23.691 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:41:23.692 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:41:23.692 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:41:23.692 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:41:23.692 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:41:23.692 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:41:23.693 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:41:23.693 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:41:23.693 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:41:23.693 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:41:23.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.694 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "19", 2020-12-05T17:41:23.694 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:41:23.694 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:41:23.694 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:41:23.694 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:41:23.695 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:41:23.695 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:41:23.695 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:41:23.695 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:41:23.695 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:41:23.696 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:41:23.696 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:41:23.696 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:41:23.696 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:41:23.696 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:41:23.697 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:41:23.697 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:41:23.697 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:41:23.697 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:41:23.698 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:41:23.698 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:41:23.698 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:41:23.698 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:41:23.698 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:41:23.699 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:41:23.699 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:41:23.699 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:41:23.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.700 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:41:23.700 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:41:23.700 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:41:23.700 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:41:23.700 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:41:23.701 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:41:23.701 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:41:23.701 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:41:23.701 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:41:23.701 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:41:23.702 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:41:23.702 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:41:23.702 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:41:23.702 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2020-12-05T17:41:23.703 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.703 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 25, 2020-12-05T17:41:23.703 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2020-12-05T17:41:23.704 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:41:01.240356+0000", 2020-12-05T17:41:23.704 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:41:23.704 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:41:23.704 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:41:23.704 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:41:23.705 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:41:23.705 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:41:23.705 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:41:23.705 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:41:23.705 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:41:23.706 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:41:23.706 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:41:23.706 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:41:23.706 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:41:23.706 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:41:23.707 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:41:23.707 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:41:23.707 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:41:23.707 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:41:23.707 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:41:23.708 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:41:23.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.708 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "128", 2020-12-05T17:41:23.708 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:41:23.708 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:41:23.709 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:41:23.709 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:41:23.709 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:41:23.709 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:41:23.709 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:41:23.710 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:41:23.710 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:41:23.710 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:41:23.710 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:41:23.711 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:41:23.711 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:41:23.711 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:41:23.711 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:41:23.712 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:41:23.712 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:41:23.712 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:41:23.713 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:41:23.713 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:41:23.713 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:41:23.713 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:41:23.713 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:41:23.714 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:41:23.714 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:41:23.714 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:41:23.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.714 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:41:23.715 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:41:23.715 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:41:23.715 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:41:23.715 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:41:23.715 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:41:23.716 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:41:23.716 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:41:23.716 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:41:23.716 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:41:23.716 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:41:23.717 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:41:23.717 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2020-12-05T17:41:23.717 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2020-12-05T17:41:23.717 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2020-12-05T17:41:23.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.718 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:41:23.718 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:41:23.718 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2020-12-05T17:41:23.718 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.718 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.719 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 26, 2020-12-05T17:41:23.719 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2020-12-05T17:41:23.720 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:41:02.269469+0000", 2020-12-05T17:41:23.720 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:41:23.720 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:41:23.720 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:41:23.720 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:41:23.721 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:41:23.721 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:41:23.721 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:41:23.721 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:41:23.721 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:41:23.722 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:41:23.722 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:41:23.722 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:41:23.722 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:41:23.722 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:41:23.723 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:41:23.723 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:41:23.723 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:41:23.723 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:41:23.723 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:41:23.724 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:41:23.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.724 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "129", 2020-12-05T17:41:23.724 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:41:23.724 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:41:23.725 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:41:23.725 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:41:23.725 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:41:23.725 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:41:23.725 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:41:23.726 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:41:23.726 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:41:23.726 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:41:23.726 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:41:23.726 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:41:23.727 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:41:23.727 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:41:23.727 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:41:23.727 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:41:23.727 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:41:23.728 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:41:23.728 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:41:23.728 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:41:23.728 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:41:23.728 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:41:23.729 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:41:23.729 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:41:23.729 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:41:23.729 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:41:23.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.730 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:41:23.730 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:41:23.730 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:41:23.730 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:41:23.730 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:41:23.731 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:41:23.731 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:41:23.731 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:41:23.731 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:41:23.731 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:41:23.731 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:41:23.732 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:41:23.732 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:41:23.732 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:41:23.732 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2020-12-05T17:41:23.733 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.733 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.733 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.733 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:41:23.733 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2020-12-05T17:41:23.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.734 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:41:23.734 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "b941127b-0a49-4e06-8d11-17794ff8b013", 2020-12-05T17:41:23.734 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:41:23.734 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:41:23.735 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:41:23.735 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:41:23.735 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:41:23.735 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:41:23.735 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:41:23.735 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 126, 2020-12-05T17:41:23.736 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:41:23.736 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:41:23.736 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:41:23.736 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.737 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.737 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2020-12-05T17:41:23.737 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:41:23.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.738 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.738 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2020-12-05T17:41:23.738 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:41:23.738 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.738 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.739 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:41:23.739 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.739 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.740 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2020-12-05T17:41:23.740 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:41:23.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.740 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.741 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2020-12-05T17:41:23.741 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:41:23.741 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.741 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.742 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:41:23.742 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.742 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.742 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2020-12-05T17:41:23.743 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:41:23.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.743 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.744 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2020-12-05T17:41:23.744 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:41:23.744 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.744 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.745 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:41:23.745 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.745 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.746 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2020-12-05T17:41:23.746 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:41:23.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.746 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.746 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2020-12-05T17:41:23.747 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:41:23.747 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.747 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.747 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6803/11526", 2020-12-05T17:41:23.748 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6805/11526", 2020-12-05T17:41:23.748 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6809/11526", 2020-12-05T17:41:23.748 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6807/11526", 2020-12-05T17:41:23.748 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:41:23.749 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:41:23.749 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:41:23.749 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.750 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:41:23.750 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "01bd7ff5-a86b-4e11-ba1e-484c309c0fba", 2020-12-05T17:41:23.750 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:41:23.750 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:41:23.750 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:41:23.751 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:41:23.751 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:41:23.751 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:41:23.751 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:41:23.751 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 127, 2020-12-05T17:41:23.752 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:41:23.752 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:41:23.752 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:41:23.752 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.753 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.753 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2020-12-05T17:41:23.753 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:41:23.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.754 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.754 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2020-12-05T17:41:23.754 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:41:23.754 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.754 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.755 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:41:23.755 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.755 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.756 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2020-12-05T17:41:23.756 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:41:23.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.756 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.757 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2020-12-05T17:41:23.757 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:41:23.757 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.757 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.758 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:41:23.758 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.758 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.759 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2020-12-05T17:41:23.759 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:41:23.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.759 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.759 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2020-12-05T17:41:23.760 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:41:23.760 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.760 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.760 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:41:23.761 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.761 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.761 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2020-12-05T17:41:23.761 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:41:23.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.762 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.762 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2020-12-05T17:41:23.762 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:41:23.763 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.763 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.763 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6819/11538", 2020-12-05T17:41:23.763 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6821/11538", 2020-12-05T17:41:23.763 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6825/11538", 2020-12-05T17:41:23.764 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6823/11538", 2020-12-05T17:41:23.764 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:41:23.764 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:41:23.764 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:41:23.764 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.765 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:41:23.765 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "94736ef5-64c2-4a97-b963-bd38cddd9c52", 2020-12-05T17:41:23.765 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:41:23.766 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:41:23.766 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:41:23.766 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:41:23.766 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:41:23.766 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:41:23.767 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:41:23.767 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 127, 2020-12-05T17:41:23.767 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:41:23.767 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:41:23.767 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:41:23.768 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.768 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.768 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2020-12-05T17:41:23.768 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:41:23.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.769 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.769 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2020-12-05T17:41:23.769 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:41:23.769 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.770 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.770 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:41:23.770 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.771 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.771 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2020-12-05T17:41:23.771 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:41:23.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.772 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.772 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2020-12-05T17:41:23.772 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:41:23.772 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.772 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.773 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:41:23.773 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.773 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.774 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2020-12-05T17:41:23.774 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:41:23.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.774 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.775 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2020-12-05T17:41:23.775 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:41:23.775 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.775 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.776 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:41:23.776 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.776 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.776 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2020-12-05T17:41:23.777 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:41:23.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.777 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.777 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2020-12-05T17:41:23.778 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:41:23.778 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.778 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.779 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6811/11521", 2020-12-05T17:41:23.779 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6813/11521", 2020-12-05T17:41:23.779 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6817/11521", 2020-12-05T17:41:23.779 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6815/11521", 2020-12-05T17:41:23.779 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:41:23.780 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:41:23.780 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:41:23.780 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.781 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:41:23.781 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "69a1f3ff-fd25-43df-b60c-e32a89c35234", 2020-12-05T17:41:23.781 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:41:23.781 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:41:23.781 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:41:23.782 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:41:23.782 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:41:23.782 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:41:23.782 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:41:23.782 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 126, 2020-12-05T17:41:23.783 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:41:23.783 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:41:23.783 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:41:23.783 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.784 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.784 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2020-12-05T17:41:23.784 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:41:23.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.785 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.785 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2020-12-05T17:41:23.785 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:41:23.785 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.785 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.786 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:41:23.786 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.786 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.786 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2020-12-05T17:41:23.787 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:41:23.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.787 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2020-12-05T17:41:23.788 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:41:23.788 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.788 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.788 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:41:23.789 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.789 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.789 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6832", 2020-12-05T17:41:23.789 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:41:23.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.790 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.790 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6833", 2020-12-05T17:41:23.790 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:41:23.790 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.791 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.791 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:41:23.791 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.792 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.792 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2020-12-05T17:41:23.792 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:41:23.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.793 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.793 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2020-12-05T17:41:23.793 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:41:23.793 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.793 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.794 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6827/11535", 2020-12-05T17:41:23.794 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6829/11535", 2020-12-05T17:41:23.794 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6833/11535", 2020-12-05T17:41:23.794 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6831/11535", 2020-12-05T17:41:23.794 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:41:23.795 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:41:23.795 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:41:23.795 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.796 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:41:23.796 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c301fe4e-5eab-402c-ae6f-db3e6a7181af", 2020-12-05T17:41:23.796 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:41:23.796 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:41:23.796 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:41:23.797 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:41:23.797 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:41:23.797 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:41:23.797 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:41:23.797 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 127, 2020-12-05T17:41:23.797 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:41:23.798 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:41:23.798 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:41:23.798 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.799 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.799 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6808", 2020-12-05T17:41:23.799 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:41:23.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.800 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.800 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6809", 2020-12-05T17:41:23.800 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:41:23.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.801 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.801 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:41:23.801 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.801 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6810", 2020-12-05T17:41:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:41:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.803 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6811", 2020-12-05T17:41:23.803 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:41:23.803 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.803 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.804 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:41:23.804 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.804 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.804 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6817", 2020-12-05T17:41:23.805 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:41:23.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.805 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.805 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6819", 2020-12-05T17:41:23.806 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:41:23.806 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.806 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.807 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:41:23.807 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.807 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.807 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6812", 2020-12-05T17:41:23.808 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:41:23.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.808 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.809 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6814", 2020-12-05T17:41:23.809 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:41:23.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.809 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.810 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6809/11541", 2020-12-05T17:41:23.810 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6811/11541", 2020-12-05T17:41:23.810 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6819/11541", 2020-12-05T17:41:23.810 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6814/11541", 2020-12-05T17:41:23.811 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:41:23.811 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:41:23.811 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:41:23.811 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.812 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:41:23.812 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "9476eff4-345b-46c5-a473-1d9a89a6d327", 2020-12-05T17:41:23.812 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:41:23.813 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:41:23.813 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:41:23.813 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:41:23.813 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:41:23.814 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:41:23.814 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:41:23.814 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 120, 2020-12-05T17:41:23.814 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:41:23.814 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:41:23.815 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:41:23.815 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.815 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.815 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6800", 2020-12-05T17:41:23.816 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:41:23.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.816 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.816 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6801", 2020-12-05T17:41:23.817 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:41:23.817 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.817 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.817 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:41:23.817 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.818 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.818 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6802", 2020-12-05T17:41:23.818 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:41:23.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.819 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.819 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6803", 2020-12-05T17:41:23.819 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:41:23.820 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.820 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.820 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:41:23.820 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.821 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.821 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6806", 2020-12-05T17:41:23.821 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:41:23.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.822 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.822 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6807", 2020-12-05T17:41:23.822 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:41:23.823 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.823 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.823 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:41:23.823 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.824 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.824 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6804", 2020-12-05T17:41:23.824 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:41:23.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.825 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.825 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6805", 2020-12-05T17:41:23.825 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:41:23.825 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.825 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.826 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6801/11539", 2020-12-05T17:41:23.826 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6803/11539", 2020-12-05T17:41:23.826 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6807/11539", 2020-12-05T17:41:23.826 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6805/11539", 2020-12-05T17:41:23.827 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:41:23.827 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:41:23.827 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:41:23.827 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.828 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:41:23.828 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8dfd8567-9b1c-4d13-9c96-071183e469c2", 2020-12-05T17:41:23.828 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:41:23.828 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:41:23.829 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:41:23.829 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:41:23.829 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:41:23.830 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:41:23.830 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:41:23.831 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 127, 2020-12-05T17:41:23.831 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:41:23.831 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:41:23.832 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:41:23.832 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.833 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.833 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6820", 2020-12-05T17:41:23.833 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:41:23.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.834 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.835 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6822", 2020-12-05T17:41:23.835 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:41:23.835 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.835 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.836 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:41:23.836 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6823", 2020-12-05T17:41:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:41:23.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.838 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.839 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6825", 2020-12-05T17:41:23.839 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:41:23.839 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.839 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.840 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:41:23.840 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.840 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.841 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6829", 2020-12-05T17:41:23.841 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:41:23.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.841 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.842 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6830", 2020-12-05T17:41:23.842 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:41:23.842 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.842 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.843 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:41:23.843 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.843 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.844 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6826", 2020-12-05T17:41:23.844 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:41:23.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.844 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.845 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6827", 2020-12-05T17:41:23.845 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:41:23.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.845 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.846 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6822/11542", 2020-12-05T17:41:23.846 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6825/11542", 2020-12-05T17:41:23.846 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6830/11542", 2020-12-05T17:41:23.846 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6827/11542", 2020-12-05T17:41:23.847 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:41:23.847 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:41:23.847 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:41:23.847 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.848 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:41:23.848 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "24d466b3-01dc-4ba2-8b81-85250d163bd2", 2020-12-05T17:41:23.848 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:41:23.849 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:41:23.849 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:41:23.849 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:41:23.849 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:41:23.849 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:41:23.850 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:41:23.850 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 127, 2020-12-05T17:41:23.850 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:41:23.850 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:41:23.851 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:41:23.851 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.852 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.852 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6813", 2020-12-05T17:41:23.852 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:41:23.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.853 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.853 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6815", 2020-12-05T17:41:23.853 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:41:23.853 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.854 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.854 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:41:23.854 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.855 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.855 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6816", 2020-12-05T17:41:23.855 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:41:23.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.856 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.856 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6818", 2020-12-05T17:41:23.856 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:41:23.856 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.857 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.857 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:41:23.857 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.858 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.858 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6828", 2020-12-05T17:41:23.858 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:41:23.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.859 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.859 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6831", 2020-12-05T17:41:23.859 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:41:23.859 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.860 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.860 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:41:23.860 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:23.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.861 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:23.861 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6821", 2020-12-05T17:41:23.861 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:41:23.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.862 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:23.862 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6824", 2020-12-05T17:41:23.862 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:41:23.862 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.862 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.863 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6815/11540", 2020-12-05T17:41:23.863 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6818/11540", 2020-12-05T17:41:23.863 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6831/11540", 2020-12-05T17:41:23.863 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6824/11540", 2020-12-05T17:41:23.864 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:41:23.864 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:41:23.864 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:41:23.864 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:23.864 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.865 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:41:23.865 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2020-12-05T17:41:23.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.865 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:41:23.865 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:41:23.866 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:41:23.866 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:41:23.866 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:41:23.867 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:41:23.867 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.398868+0000", 2020-12-05T17:41:23.867 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:41:23.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.868 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:41:23.868 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:41:23.869 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:41:23.869 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:41:23.869 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:41:23.869 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:41:23.869 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.675748+0000", 2020-12-05T17:41:23.870 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:41:23.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.870 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:41:23.871 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:41:23.871 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:41:23.871 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:41:23.871 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:41:23.871 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:41:23.871 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.620888+0000", 2020-12-05T17:41:23.872 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:41:23.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.873 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:41:23.873 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:41:23.873 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:41:23.873 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:41:23.874 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:41:23.874 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:41:23.874 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.681600+0000", 2020-12-05T17:41:23.874 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:41:23.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.875 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:41:23.875 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:41:23.875 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:41:23.875 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:41:23.876 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:41:23.876 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:41:23.876 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.658769+0000", 2020-12-05T17:41:23.876 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:41:23.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.877 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:41:23.877 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:41:23.877 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:41:23.878 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:41:23.878 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:41:23.878 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:41:23.878 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.630708+0000", 2020-12-05T17:41:23.879 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:41:23.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.879 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:41:23.879 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:41:23.880 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:41:23.880 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:41:23.880 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:41:23.880 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:41:23.880 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.648352+0000", 2020-12-05T17:41:23.881 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:41:23.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:23.881 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:41:23.882 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:41:23.882 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:41:23.882 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:41:23.882 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:41:23.882 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:41:23.883 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.629399+0000", 2020-12-05T17:41:23.883 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:41:23.883 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.883 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:41:23.883 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2020-12-05T17:41:23.884 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2020-12-05T17:41:23.884 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2020-12-05T17:41:23.884 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2020-12-05T17:41:23.884 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": {}, 2020-12-05T17:41:23.884 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2020-12-05T17:41:23.885 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2020-12-05T17:41:23.885 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2020-12-05T17:41:23.885 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:41:23.885 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2020-12-05T17:41:23.885 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2020-12-05T17:41:23.886 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:41:23.886 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2020-12-05T17:41:23.886 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:41:23.886 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:41:23.887 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2020-12-05T17:41:23.887 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2020-12-05T17:41:23.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.887 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2020-12-05T17:41:23.887 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:41:23.887 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:41:23.888 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2020-12-05T17:41:23.888 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:41:23.888 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:41:23.888 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2020-12-05T17:41:23.888 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:23.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:23.889 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2020-12-05T17:41:23.889 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2020-12-05T17:41:23.889 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2020-12-05T17:41:23.889 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2020-12-05T17:41:23.890 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2020-12-05T17:41:23.890 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:41:23.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.662+0000 7fdb49c03700 1 -- 172.21.15.32:0/2772453456 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb24055330 msgr2=0x7fdb240577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:23.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.662+0000 7fdb49c03700 1 --2- 172.21.15.32:0/2772453456 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb24055330 0x7fdb240577f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7fdb4000b000 tx=0x7fdb40006dc0).stop 2020-12-05T17:41:23.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.662+0000 7fdb49c03700 1 -- 172.21.15.32:0/2772453456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb440ab710 msgr2=0x7fdb440acdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:23.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.662+0000 7fdb49c03700 1 --2- 172.21.15.32:0/2772453456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb440ab710 0x7fdb440acdc0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7fdb34002f30 tx=0x7fdb34015000).stop 2020-12-05T17:41:23.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.662+0000 7fdb49c03700 1 -- 172.21.15.32:0/2772453456 shutdown_connections 2020-12-05T17:41:23.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.662+0000 7fdb49c03700 1 --2- 172.21.15.32:0/2772453456 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb44147400 0x7fdb440ad490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:23.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.662+0000 7fdb49c03700 1 --2- 172.21.15.32:0/2772453456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb440ab710 0x7fdb440acdc0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:23.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.662+0000 7fdb49c03700 1 --2- 172.21.15.32:0/2772453456 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdb24055330 0x7fdb240577f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:23.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.662+0000 7fdb49c03700 1 --2- 172.21.15.32:0/2772453456 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb440b1fc0 0x7fdb44148620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:23.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.662+0000 7fdb49c03700 1 -- 172.21.15.32:0/2772453456 >> 172.21.15.32:0/2772453456 conn(0x7fdb440b5530 msgr2=0x7fdb440b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:23.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.662+0000 7fdb49c03700 1 -- 172.21.15.32:0/2772453456 shutdown_connections 2020-12-05T17:41:23.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:23.662+0000 7fdb49c03700 1 -- 172.21.15.32:0/2772453456 wait complete. 2020-12-05T17:41:23.895 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:41:24.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e95d4700 1 Processor -- start 2020-12-05T17:41:24.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e95d4700 1 -- start start 2020-12-05T17:41:24.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e95d4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8e40b1fc0 0x7ff8e40af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:24.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e95d4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff8e40ab710 0x7ff8e4061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:24.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e95d4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e4146a90 con 0x7ff8e4147400 2020-12-05T17:41:24.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e95d4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e40b0110 con 0x7ff8e40ab710 2020-12-05T17:41:24.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e95d4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e40b23a0 con 0x7ff8e40b1fc0 2020-12-05T17:41:24.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff8e40ab710 0x7ff8e4061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:24.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e8dd3700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff8e4147400 0x7ff8e4145d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43288/0 (socket says 172.21.15.32:43288) 2020-12-05T17:41:24.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e8dd3700 1 -- 172.21.15.32:0/414136565 learned_addr learned my addr 172.21.15.32:0/414136565 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:24.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e3fff700 1 --2- 172.21.15.32:0/414136565 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8e40b1fc0 0x7ff8e40af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:24.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e37fe700 1 -- 172.21.15.32:0/414136565 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8e40b1fc0 msgr2=0x7ff8e40af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:24.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e37fe700 1 --2- 172.21.15.32:0/414136565 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8e40b1fc0 0x7ff8e40af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e37fe700 1 -- 172.21.15.32:0/414136565 >> v1:172.21.15.32:6789/0 conn(0x7ff8e4147400 legacy=0x7ff8e4145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e37fe700 1 -- 172.21.15.32:0/414136565 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e4146cc0 con 0x7ff8e40ab710 2020-12-05T17:41:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e3fff700 1 --2- 172.21.15.32:0/414136565 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8e40b1fc0 0x7ff8e40af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e37fe700 1 --2- 172.21.15.32:0/414136565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8e40ab710 0x7ff8e4061780 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7ff8dc002f30 tx=0x7ff8dc006dc0).ready entity=mon.0 client_cookie=bfc4f04035e1460b server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e2ffd700 1 -- 172.21.15.32:0/414136565 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8dc0097f0 con 0x7ff8e40ab710 2020-12-05T17:41:24.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e2ffd700 1 -- 172.21.15.32:0/414136565 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8dc009950 con 0x7ff8e40ab710 2020-12-05T17:41:24.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e2ffd700 1 -- 172.21.15.32:0/414136565 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8dc009cb0 con 0x7ff8e40ab710 2020-12-05T17:41:24.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e95d4700 1 -- 172.21.15.32:0/414136565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8e40ab710 msgr2=0x7ff8e4061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:24.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e95d4700 1 --2- 172.21.15.32:0/414136565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8e40ab710 0x7ff8e4061780 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7ff8dc002f30 tx=0x7ff8dc006dc0).stop 2020-12-05T17:41:24.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e95d4700 1 -- 172.21.15.32:0/414136565 shutdown_connections 2020-12-05T17:41:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e95d4700 1 --2- 172.21.15.32:0/414136565 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8e40b1fc0 0x7ff8e40af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e95d4700 1 --2- 172.21.15.32:0/414136565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8e40ab710 0x7ff8e4061780 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e95d4700 1 -- 172.21.15.32:0/414136565 >> 172.21.15.32:0/414136565 conn(0x7ff8e40b5530 msgr2=0x7ff8e40b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e95d4700 1 -- 172.21.15.32:0/414136565 shutdown_connections 2020-12-05T17:41:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.014+0000 7ff8e95d4700 1 -- 172.21.15.32:0/414136565 wait complete. 2020-12-05T17:41:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e95d4700 1 Processor -- start 2020-12-05T17:41:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e95d4700 1 -- start start 2020-12-05T17:41:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e95d4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff8e40ab710 0x7ff8e414da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:24.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e95d4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8e40b1fc0 0x7ff8e4149510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:24.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e95d4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8e4147400 0x7ff8e40a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:24.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e95d4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e4149df0 con 0x7ff8e40b1fc0 2020-12-05T17:41:24.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e95d4700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e409ed60 con 0x7ff8e40ab710 2020-12-05T17:41:24.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e95d4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e409f000 con 0x7ff8e4147400 2020-12-05T17:41:24.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8e40b1fc0 0x7ff8e4149510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:24.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8e40b1fc0 0x7ff8e4149510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40646/0 (socket says 172.21.15.32:40646) 2020-12-05T17:41:24.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e37fe700 1 -- 172.21.15.32:0/254173327 learned_addr learned my addr 172.21.15.32:0/254173327 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:24.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e8dd3700 1 --2- 172.21.15.32:0/254173327 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8e4147400 0x7ff8e40a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:24.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e3fff700 1 --2- 172.21.15.32:0/254173327 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff8e40ab710 0x7ff8e414da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:24.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e37fe700 1 -- 172.21.15.32:0/254173327 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8e4147400 msgr2=0x7ff8e40a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:24.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e37fe700 1 --2- 172.21.15.32:0/254173327 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8e4147400 0x7ff8e40a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:24.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e37fe700 1 -- 172.21.15.32:0/254173327 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff8e40ab710 msgr2=0x7ff8e414da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:24.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e37fe700 1 --2- 172.21.15.32:0/254173327 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff8e40ab710 0x7ff8e414da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:24.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e37fe700 1 -- 172.21.15.32:0/254173327 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e4147e50 con 0x7ff8e40b1fc0 2020-12-05T17:41:24.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e8dd3700 1 --2- 172.21.15.32:0/254173327 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8e4147400 0x7ff8e40a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:24.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e37fe700 1 --2- 172.21.15.32:0/254173327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8e40b1fc0 0x7ff8e4149510 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7ff8dc002f30 tx=0x7ff8dc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:24.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e17fa700 1 -- 172.21.15.32:0/254173327 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8dc011b80 con 0x7ff8e40b1fc0 2020-12-05T17:41:24.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e17fa700 1 -- 172.21.15.32:0/254173327 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8dc011ce0 con 0x7ff8e40b1fc0 2020-12-05T17:41:24.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e95d4700 1 -- 172.21.15.32:0/254173327 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e41482f0 con 0x7ff8e40b1fc0 2020-12-05T17:41:24.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e17fa700 1 -- 172.21.15.32:0/254173327 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8dc01d640 con 0x7ff8e40b1fc0 2020-12-05T17:41:24.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.018+0000 7ff8e95d4700 1 -- 172.21.15.32:0/254173327 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e4148e80 con 0x7ff8e40b1fc0 2020-12-05T17:41:24.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.022+0000 7ff8e95d4700 1 -- 172.21.15.32:0/254173327 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e4057050 con 0x7ff8e40b1fc0 2020-12-05T17:41:24.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.022+0000 7ff8e17fa700 1 -- 172.21.15.32:0/254173327 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff8dc024030 con 0x7ff8e40b1fc0 2020-12-05T17:41:24.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.022+0000 7ff8e17fa700 1 --2- 172.21.15.32:0/254173327 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff8c80553b0 0x7ff8c8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:24.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.022+0000 7ff8e17fa700 1 -- 172.21.15.32:0/254173327 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff8dc06f190 con 0x7ff8e40b1fc0 2020-12-05T17:41:24.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.022+0000 7ff8e3fff700 1 --2- 172.21.15.32:0/254173327 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff8c80553b0 0x7ff8c8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:24.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.022+0000 7ff8e3fff700 1 --2- 172.21.15.32:0/254173327 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff8c80553b0 0x7ff8c8057870 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7ff8d400d000 tx=0x7ff8d4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:24.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.026+0000 7ff8e17fa700 1 -- 172.21.15.32:0/254173327 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff8dc045370 con 0x7ff8e40b1fc0 2020-12-05T17:41:24.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.210+0000 7ff8e95d4700 1 -- 172.21.15.32:0/254173327 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8e405aab0 con 0x7ff8e40b1fc0 2020-12-05T17:41:24.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.210+0000 7ff8e17fa700 1 -- 172.21.15.32:0/254173327 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+2142 (secure 0 0 0) 0x7ff8dc022020 con 0x7ff8e40b1fc0 2020-12-05T17:41:24.215 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:24.216 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":62,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6830,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/3985915039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3985915039},{"type":"v1","addr":"172.21.15.61:6833","nonce":3985915039}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":60}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:41:03.280787+0000","modified":"2020-12-05T17:41:04.283186+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7938":{"gid":7938,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1585653076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1585653076},{"type":"v1","addr":"172.21.15.32:6835","nonce":1585653076}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2020-12-05T17:41:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.210+0000 7ff8e95d4700 1 -- 172.21.15.32:0/254173327 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff8c80553b0 msgr2=0x7ff8c8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:24.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.210+0000 7ff8e95d4700 1 --2- 172.21.15.32:0/254173327 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff8c80553b0 0x7ff8c8057870 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7ff8d400d000 tx=0x7ff8d4008040).stop 2020-12-05T17:41:24.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.210+0000 7ff8e95d4700 1 -- 172.21.15.32:0/254173327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8e40b1fc0 msgr2=0x7ff8e4149510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:24.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.210+0000 7ff8e95d4700 1 --2- 172.21.15.32:0/254173327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8e40b1fc0 0x7ff8e4149510 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7ff8dc002f30 tx=0x7ff8dc015000).stop 2020-12-05T17:41:24.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.210+0000 7ff8e95d4700 1 -- 172.21.15.32:0/254173327 shutdown_connections 2020-12-05T17:41:24.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.210+0000 7ff8e95d4700 1 --2- 172.21.15.32:0/254173327 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff8c80553b0 0x7ff8c8057870 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:24.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.210+0000 7ff8e95d4700 1 --2- 172.21.15.32:0/254173327 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8e4147400 0x7ff8e40a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:24.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.210+0000 7ff8e95d4700 1 --2- 172.21.15.32:0/254173327 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff8e40ab710 0x7ff8e414da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:24.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.210+0000 7ff8e95d4700 1 --2- 172.21.15.32:0/254173327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8e40b1fc0 0x7ff8e4149510 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:24.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.210+0000 7ff8e95d4700 1 -- 172.21.15.32:0/254173327 >> 172.21.15.32:0/254173327 conn(0x7ff8e40b5530 msgr2=0x7ff8e40b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:24.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.214+0000 7ff8e95d4700 1 -- 172.21.15.32:0/254173327 shutdown_connections 2020-12-05T17:41:24.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.214+0000 7ff8e95d4700 1 -- 172.21.15.32:0/254173327 wait complete. 2020-12-05T17:41:24.220 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 62 2020-12-05T17:41:24.229 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-12-05T17:41:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.338+0000 7f735759e700 1 Processor -- start 2020-12-05T17:41:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.338+0000 7f735759e700 1 -- start start 2020-12-05T17:41:24.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f735759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f73580ab710 0x7f73580af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:24.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f735759e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f7358147400 0x7f7358145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:24.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f735759e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7358146a90 con 0x7f73580b1fc0 2020-12-05T17:41:24.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f735759e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7358061e90 con 0x7f73580ab710 2020-12-05T17:41:24.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f735759e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73580b0110 con 0x7f7358147400 2020-12-05T17:41:24.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f7355d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f73580ab710 0x7f73580af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:24.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f7355d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f73580ab710 0x7f73580af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40652/0 (socket says 172.21.15.32:40652) 2020-12-05T17:41:24.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f7355d9b700 1 -- 172.21.15.32:0/1152749302 learned_addr learned my addr 172.21.15.32:0/1152749302 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:24.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f7356d9d700 1 --2- 172.21.15.32:0/1152749302 >> v2:172.21.15.61:3300/0 conn(0x7f7358147400 0x7f7358145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:24.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f7355d9b700 1 -- 172.21.15.32:0/1152749302 >> v2:172.21.15.61:3300/0 conn(0x7f7358147400 msgr2=0x7f7358145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:24.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f7355d9b700 1 --2- 172.21.15.32:0/1152749302 >> v2:172.21.15.61:3300/0 conn(0x7f7358147400 0x7f7358145d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:24.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f7355d9b700 1 -- 172.21.15.32:0/1152749302 >> v1:172.21.15.61:6789/0 conn(0x7f73580b1fc0 legacy=0x7f7358061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:24.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f7355d9b700 1 -- 172.21.15.32:0/1152749302 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73580b23a0 con 0x7f73580ab710 2020-12-05T17:41:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f7355d9b700 1 --2- 172.21.15.32:0/1152749302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73580ab710 0x7f73580af230 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7f7340002f30 tx=0x7f7340006e80).ready entity=mon.0 client_cookie=b10e816d1a6ef350 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f735559a700 1 -- 172.21.15.32:0/1152749302 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73400097f0 con 0x7f73580ab710 2020-12-05T17:41:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f735559a700 1 -- 172.21.15.32:0/1152749302 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7340009950 con 0x7f73580ab710 2020-12-05T17:41:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f735559a700 1 -- 172.21.15.32:0/1152749302 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7340009cb0 con 0x7f73580ab710 2020-12-05T17:41:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f735759e700 1 -- 172.21.15.32:0/1152749302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73580ab710 msgr2=0x7f73580af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:24.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f735759e700 1 --2- 172.21.15.32:0/1152749302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73580ab710 0x7f73580af230 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7f7340002f30 tx=0x7f7340006e80).stop 2020-12-05T17:41:24.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f735759e700 1 -- 172.21.15.32:0/1152749302 shutdown_connections 2020-12-05T17:41:24.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f735759e700 1 --2- 172.21.15.32:0/1152749302 >> v2:172.21.15.61:3300/0 conn(0x7f7358147400 0x7f7358145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:24.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f735759e700 1 --2- 172.21.15.32:0/1152749302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73580ab710 0x7f73580af230 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:24.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f735759e700 1 -- 172.21.15.32:0/1152749302 >> 172.21.15.32:0/1152749302 conn(0x7f73580b5530 msgr2=0x7f73580b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f735759e700 1 -- 172.21.15.32:0/1152749302 shutdown_connections 2020-12-05T17:41:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.342+0000 7f735759e700 1 -- 172.21.15.32:0/1152749302 wait complete. 2020-12-05T17:41:24.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f735759e700 1 Processor -- start 2020-12-05T17:41:24.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f735759e700 1 -- start start 2020-12-05T17:41:24.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f735759e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f73580ab710 0x7f7358149e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f735759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73580b1fc0 0x7f73580ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f735759e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7358147400 0x7f73580ae750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f735759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73580b3120 con 0x7f73580b1fc0 2020-12-05T17:41:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f735759e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73580b33c0 con 0x7f7358147400 2020-12-05T17:41:24.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f735759e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73580b3660 con 0x7f73580ab710 2020-12-05T17:41:24.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f7355d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73580b1fc0 0x7f73580ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:24.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f7355d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73580b1fc0 0x7f73580ae260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40658/0 (socket says 172.21.15.32:40658) 2020-12-05T17:41:24.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f7355d9b700 1 -- 172.21.15.32:0/3084111047 learned_addr learned my addr 172.21.15.32:0/3084111047 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:24.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f7356d9d700 1 --2- 172.21.15.32:0/3084111047 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7358147400 0x7f73580ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:24.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f735659c700 1 --2- 172.21.15.32:0/3084111047 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f73580ab710 0x7f7358149e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:24.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f7355d9b700 1 -- 172.21.15.32:0/3084111047 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f73580ab710 msgr2=0x7f7358149e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:24.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f7355d9b700 1 --2- 172.21.15.32:0/3084111047 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f73580ab710 0x7f7358149e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:24.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f7355d9b700 1 -- 172.21.15.32:0/3084111047 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7358147400 msgr2=0x7f73580ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:24.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f7355d9b700 1 --2- 172.21.15.32:0/3084111047 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7358147400 0x7f73580ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f7355d9b700 1 -- 172.21.15.32:0/3084111047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735814a670 con 0x7f73580b1fc0 2020-12-05T17:41:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f7355d9b700 1 --2- 172.21.15.32:0/3084111047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73580b1fc0 0x7f73580ae260 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7f7340002f30 tx=0x7f7340015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f734f7fe700 1 -- 172.21.15.32:0/3084111047 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7340011b80 con 0x7f73580b1fc0 2020-12-05T17:41:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f734f7fe700 1 -- 172.21.15.32:0/3084111047 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7340011ce0 con 0x7f73580b1fc0 2020-12-05T17:41:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f734f7fe700 1 -- 172.21.15.32:0/3084111047 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f734001e650 con 0x7f73580b1fc0 2020-12-05T17:41:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f735759e700 1 -- 172.21.15.32:0/3084111047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73580acb10 con 0x7f73580b1fc0 2020-12-05T17:41:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.346+0000 7f735759e700 1 -- 172.21.15.32:0/3084111047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73580aede0 con 0x7f73580b1fc0 2020-12-05T17:41:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.350+0000 7f735759e700 1 -- 172.21.15.32:0/3084111047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73580abe90 con 0x7f73580b1fc0 2020-12-05T17:41:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.350+0000 7f734f7fe700 1 -- 172.21.15.32:0/3084111047 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f734001a040 con 0x7f73580b1fc0 2020-12-05T17:41:24.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.350+0000 7f734f7fe700 1 --2- 172.21.15.32:0/3084111047 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f733c055440 0x7f733c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:24.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.350+0000 7f734f7fe700 1 -- 172.21.15.32:0/3084111047 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f734006f0b0 con 0x7f73580b1fc0 2020-12-05T17:41:24.361 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.350+0000 7f735659c700 1 --2- 172.21.15.32:0/3084111047 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f733c055440 0x7f733c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:24.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.350+0000 7f735659c700 1 --2- 172.21.15.32:0/3084111047 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f733c055440 0x7f733c057900 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f734800d000 tx=0x7f7348007b60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:24.362 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.354+0000 7f734f7fe700 1 -- 172.21.15.32:0/3084111047 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7340045250 con 0x7f73580b1fc0 2020-12-05T17:41:24.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.542+0000 7f735759e700 1 -- 172.21.15.32:0/3084111047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f73580abe90 con 0x7f73580b1fc0 2020-12-05T17:41:24.930 INFO:tasks.ceph.mon.a.smithi032.stderr:2020-12-05T17:41:24.926+0000 7fdef8136700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-12-05T17:41:24.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.942+0000 7f734f7fe700 1 -- 172.21.15.32:0/3084111047 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) v1 ==== 161+0+0 (secure 0 0 0) 0x7f73580abe90 con 0x7f73580b1fc0 2020-12-05T17:41:24.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.946+0000 7f735759e700 1 -- 172.21.15.32:0/3084111047 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f733c055440 msgr2=0x7f733c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:24.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.946+0000 7f735759e700 1 --2- 172.21.15.32:0/3084111047 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f733c055440 0x7f733c057900 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f734800d000 tx=0x7f7348007b60).stop 2020-12-05T17:41:24.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.946+0000 7f735759e700 1 -- 172.21.15.32:0/3084111047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73580b1fc0 msgr2=0x7f73580ae260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:24.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.946+0000 7f735759e700 1 --2- 172.21.15.32:0/3084111047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73580b1fc0 0x7f73580ae260 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7f7340002f30 tx=0x7f7340015000).stop 2020-12-05T17:41:24.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.946+0000 7f735759e700 1 -- 172.21.15.32:0/3084111047 shutdown_connections 2020-12-05T17:41:24.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.946+0000 7f735759e700 1 --2- 172.21.15.32:0/3084111047 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7358147400 0x7f73580ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:24.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.946+0000 7f735759e700 1 --2- 172.21.15.32:0/3084111047 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f733c055440 0x7f733c057900 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:24.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.946+0000 7f735759e700 1 --2- 172.21.15.32:0/3084111047 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f73580ab710 0x7f7358149e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:24.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.946+0000 7f735759e700 1 --2- 172.21.15.32:0/3084111047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73580b1fc0 0x7f73580ae260 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:24.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.946+0000 7f735759e700 1 -- 172.21.15.32:0/3084111047 >> 172.21.15.32:0/3084111047 conn(0x7f73580b5530 msgr2=0x7f7358144ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:24.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.946+0000 7f735759e700 1 -- 172.21.15.32:0/3084111047 shutdown_connections 2020-12-05T17:41:24.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:24.946+0000 7f735759e700 1 -- 172.21.15.32:0/3084111047 wait complete. 2020-12-05T17:41:24.953 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-12-05T17:41:24.970 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:41:24.973 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:41:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.078+0000 7f88cc9f3700 1 Processor -- start 2020-12-05T17:41:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.078+0000 7f88cc9f3700 1 -- start start 2020-12-05T17:41:25.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.078+0000 7f88cc9f3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88c81466b0 0x7f88c8061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:25.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.078+0000 7f88cc9f3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c814a090 con 0x7f88c80b12d0 2020-12-05T17:41:25.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.078+0000 7f88cc9f3700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c80b7b00 con 0x7f88c80b7720 2020-12-05T17:41:25.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.078+0000 7f88cc9f3700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c80b1740 con 0x7f88c81466b0 2020-12-05T17:41:25.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.078+0000 7f88c759e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f88c80b12d0 0x7f88c814a4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43328/0 (socket says 172.21.15.32:43328) 2020-12-05T17:41:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.078+0000 7f88c759e700 1 -- 172.21.15.32:0/3652807548 learned_addr learned my addr 172.21.15.32:0/3652807548 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.078+0000 7f88c6d9d700 1 --1- 172.21.15.32:0/3652807548 >> v1:172.21.15.61:6789/0 conn(0x7f88c80b7720 0x7f88c8145fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:55964/0 (socket says 172.21.15.32:55964) 2020-12-05T17:41:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.078+0000 7f88c659c700 1 --2- 172.21.15.32:0/3652807548 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88c81466b0 0x7f88c8061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.078+0000 7f88c5d9b700 1 -- 172.21.15.32:0/3652807548 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2404882535 0 0) 0x7f88c814a090 con 0x7f88c80b12d0 2020-12-05T17:41:25.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.078+0000 7f88c5d9b700 1 -- 172.21.15.32:0/3652807548 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88a8003230 con 0x7f88c80b12d0 2020-12-05T17:41:25.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.078+0000 7f88c5d9b700 1 -- 172.21.15.32:0/3652807548 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2193257505 0 0) 0x7f88c80b7b00 con 0x7f88c80b7720 2020-12-05T17:41:25.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.078+0000 7f88c5d9b700 1 -- 172.21.15.32:0/3652807548 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88c814a090 con 0x7f88c80b7720 2020-12-05T17:41:25.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.078+0000 7f88c659c700 1 -- 172.21.15.32:0/3652807548 >> v1:172.21.15.61:6789/0 conn(0x7f88c80b7720 legacy=0x7f88c8145fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.078+0000 7f88c5d9b700 1 -- 172.21.15.32:0/3652807548 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4022806932 0 0) 0x7f88a8003230 con 0x7f88c80b12d0 2020-12-05T17:41:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.078+0000 7f88c659c700 1 -- 172.21.15.32:0/3652807548 >> v1:172.21.15.32:6789/0 conn(0x7f88c80b12d0 legacy=0x7f88c814a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.078+0000 7f88c659c700 1 -- 172.21.15.32:0/3652807548 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c80626f0 con 0x7f88c81466b0 2020-12-05T17:41:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.082+0000 7f88c659c700 1 --2- 172.21.15.32:0/3652807548 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88c81466b0 0x7f88c8061780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f88b00010d0 tx=0x7f88b8006dc0).ready entity=mon.2 client_cookie=4575f0f5ee66d6f5 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.082+0000 7f88c5d9b700 1 -- 172.21.15.32:0/3652807548 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88b00016d0 con 0x7f88c81466b0 2020-12-05T17:41:25.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.082+0000 7f88c5d9b700 1 -- 172.21.15.32:0/3652807548 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b800d5c0 con 0x7f88c81466b0 2020-12-05T17:41:25.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.082+0000 7f88c5d9b700 1 -- 172.21.15.32:0/3652807548 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88b800d730 con 0x7f88c81466b0 2020-12-05T17:41:25.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.082+0000 7f88cc9f3700 1 -- 172.21.15.32:0/3652807548 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88c81466b0 msgr2=0x7f88c8061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:25.090 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.082+0000 7f88cc9f3700 1 --2- 172.21.15.32:0/3652807548 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88c81466b0 0x7f88c8061780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f88b00010d0 tx=0x7f88b8006dc0).stop 2020-12-05T17:41:25.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.082+0000 7f88cc9f3700 1 -- 172.21.15.32:0/3652807548 shutdown_connections 2020-12-05T17:41:25.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.082+0000 7f88cc9f3700 1 --2- 172.21.15.32:0/3652807548 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88c81466b0 0x7f88c8061780 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:25.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.082+0000 7f88cc9f3700 1 -- 172.21.15.32:0/3652807548 >> 172.21.15.32:0/3652807548 conn(0x7f88c805bc20 msgr2=0x7f88c80b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:25.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.082+0000 7f88cc9f3700 1 -- 172.21.15.32:0/3652807548 shutdown_connections 2020-12-05T17:41:25.091 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.082+0000 7f88cc9f3700 1 -- 172.21.15.32:0/3652807548 wait complete. 2020-12-05T17:41:25.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.082+0000 7f88cc9f3700 1 Processor -- start 2020-12-05T17:41:25.092 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.082+0000 7f88cc9f3700 1 -- start start 2020-12-05T17:41:25.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.082+0000 7f88cc9f3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88c80b12d0 0x7f88c80affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:25.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.082+0000 7f88cc9f3700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f88c80b7720 0x7f88c80650f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:25.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.082+0000 7f88cc9f3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88c81466b0 0x7f88c8066690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:25.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.082+0000 7f88cc9f3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c8063b10 con 0x7f88c81466b0 2020-12-05T17:41:25.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.082+0000 7f88cc9f3700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c814cde0 con 0x7f88c80b7720 2020-12-05T17:41:25.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.082+0000 7f88cc9f3700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c814d080 con 0x7f88c80b12d0 2020-12-05T17:41:25.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.082+0000 7f88c759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88c81466b0 0x7f88c8066690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:25.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.082+0000 7f88c759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88c81466b0 0x7f88c8066690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40686/0 (socket says 172.21.15.32:40686) 2020-12-05T17:41:25.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.082+0000 7f88c759e700 1 -- 172.21.15.32:0/3324257464 learned_addr learned my addr 172.21.15.32:0/3324257464 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:25.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.082+0000 7f88c6d9d700 1 --2- 172.21.15.32:0/3324257464 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88c80b12d0 0x7f88c80affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.086+0000 7f88c659c700 1 --2- 172.21.15.32:0/3324257464 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f88c80b7720 0x7f88c80650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.086+0000 7f88c759e700 1 -- 172.21.15.32:0/3324257464 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88c80b12d0 msgr2=0x7f88c80affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.086+0000 7f88c759e700 1 --2- 172.21.15.32:0/3324257464 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88c80b12d0 0x7f88c80affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.086+0000 7f88c759e700 1 -- 172.21.15.32:0/3324257464 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f88c80b7720 msgr2=0x7f88c80650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.086+0000 7f88c759e700 1 --2- 172.21.15.32:0/3324257464 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f88c80b7720 0x7f88c80650f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:25.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.086+0000 7f88c759e700 1 -- 172.21.15.32:0/3324257464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c814d320 con 0x7f88c81466b0 2020-12-05T17:41:25.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.086+0000 7f88c6d9d700 1 --2- 172.21.15.32:0/3324257464 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88c80b12d0 0x7f88c80affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:25.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.086+0000 7f88c759e700 1 --2- 172.21.15.32:0/3324257464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88c81466b0 0x7f88c8066690 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7f88b0000fa0 tx=0x7f88b0001280).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:25.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.086+0000 7f88bffff700 1 -- 172.21.15.32:0/3324257464 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88b0007170 con 0x7f88c81466b0 2020-12-05T17:41:25.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.086+0000 7f88bffff700 1 -- 172.21.15.32:0/3324257464 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b00072d0 con 0x7f88c81466b0 2020-12-05T17:41:25.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.086+0000 7f88cc9f3700 1 -- 172.21.15.32:0/3324257464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c80661f0 con 0x7f88c81466b0 2020-12-05T17:41:25.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.086+0000 7f88bffff700 1 -- 172.21.15.32:0/3324257464 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88b00048f0 con 0x7f88c81466b0 2020-12-05T17:41:25.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.086+0000 7f88cc9f3700 1 -- 172.21.15.32:0/3324257464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c814c9e0 con 0x7f88c81466b0 2020-12-05T17:41:25.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.086+0000 7f88cc9f3700 1 -- 172.21.15.32:0/3324257464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c8066ec0 con 0x7f88c81466b0 2020-12-05T17:41:25.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.086+0000 7f88bffff700 1 -- 172.21.15.32:0/3324257464 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f88b001c030 con 0x7f88c81466b0 2020-12-05T17:41:25.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.086+0000 7f88bffff700 1 --2- 172.21.15.32:0/3324257464 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f88ac055330 0x7f88ac0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:25.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.086+0000 7f88c6d9d700 1 --2- 172.21.15.32:0/3324257464 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f88ac055330 0x7f88ac0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:25.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.086+0000 7f88bffff700 1 -- 172.21.15.32:0/3324257464 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f88b006e2c0 con 0x7f88c81466b0 2020-12-05T17:41:25.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.090+0000 7f88c6d9d700 1 --2- 172.21.15.32:0/3324257464 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f88ac055330 0x7f88ac0577f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f88c000b000 tx=0x7f88c0007d80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:25.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.094+0000 7f88bffff700 1 -- 172.21.15.32:0/3324257464 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f88b0044300 con 0x7f88c81466b0 2020-12-05T17:41:25.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.274+0000 7f88cc9f3700 1 -- 172.21.15.32:0/3324257464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88c8066ec0 con 0x7f88c81466b0 2020-12-05T17:41:25.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.278+0000 7f88bffff700 1 -- 172.21.15.32:0/3324257464 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+1782 (secure 0 0 0) 0x7f88c8066ec0 con 0x7f88c81466b0 2020-12-05T17:41:25.282 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:25.282 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":63,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6830,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/3985915039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3985915039},{"type":"v1","addr":"172.21.15.61:6833","nonce":3985915039}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":60}],"filesystems":[{"mdsmap":{"epoch":63,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:41:03.280787+0000","modified":"2020-12-05T17:41:24.931135+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2020-12-05T17:41:25.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.278+0000 7f88cc9f3700 1 -- 172.21.15.32:0/3324257464 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f88ac055330 msgr2=0x7f88ac0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:25.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.278+0000 7f88cc9f3700 1 --2- 172.21.15.32:0/3324257464 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f88ac055330 0x7f88ac0577f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f88c000b000 tx=0x7f88c0007d80).stop 2020-12-05T17:41:25.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.278+0000 7f88cc9f3700 1 -- 172.21.15.32:0/3324257464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88c81466b0 msgr2=0x7f88c8066690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:25.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.278+0000 7f88cc9f3700 1 --2- 172.21.15.32:0/3324257464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88c81466b0 0x7f88c8066690 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7f88b0000fa0 tx=0x7f88b0001280).stop 2020-12-05T17:41:25.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.278+0000 7f88cc9f3700 1 -- 172.21.15.32:0/3324257464 shutdown_connections 2020-12-05T17:41:25.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.278+0000 7f88cc9f3700 1 --2- 172.21.15.32:0/3324257464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88c81466b0 0x7f88c8066690 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:25.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.278+0000 7f88cc9f3700 1 --2- 172.21.15.32:0/3324257464 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f88ac055330 0x7f88ac0577f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:25.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.278+0000 7f88cc9f3700 1 --2- 172.21.15.32:0/3324257464 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f88c80b12d0 0x7f88c80affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:25.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.278+0000 7f88cc9f3700 1 --2- 172.21.15.32:0/3324257464 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f88c80b7720 0x7f88c80650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:25.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.278+0000 7f88cc9f3700 1 -- 172.21.15.32:0/3324257464 >> 172.21.15.32:0/3324257464 conn(0x7f88c805bc20 msgr2=0x7f88c80adc90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:25.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.278+0000 7f88cc9f3700 1 -- 172.21.15.32:0/3324257464 shutdown_connections 2020-12-05T17:41:25.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.278+0000 7f88cc9f3700 1 -- 172.21.15.32:0/3324257464 wait complete. 2020-12-05T17:41:25.287 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 63 2020-12-05T17:41:25.296 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-12-05T17:41:25.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.410+0000 7fd05ffff700 1 Processor -- start 2020-12-05T17:41:25.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.410+0000 7fd05ffff700 1 -- start start 2020-12-05T17:41:25.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05ffff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd0600abeb0 0x7fd0600ab610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:25.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05ffff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0600ac290 con 0x7fd060062660 2020-12-05T17:41:25.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05ffff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0600b0930 con 0x7fd0600ae6e0 2020-12-05T17:41:25.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05ffff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0600ad8a0 con 0x7fd0600abeb0 2020-12-05T17:41:25.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd0567fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd060062660 0x7fd0600ad190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43338/0 (socket says 172.21.15.32:43338) 2020-12-05T17:41:25.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd0567fc700 1 -- 172.21.15.32:0/237626016 learned_addr learned my addr 172.21.15.32:0/237626016 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:25.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05e7fc700 1 -- 172.21.15.32:0/237626016 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1801809805 0 0) 0x7fd0600ac290 con 0x7fd060062660 2020-12-05T17:41:25.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05e7fc700 1 -- 172.21.15.32:0/237626016 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd040003230 con 0x7fd060062660 2020-12-05T17:41:25.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05effd700 1 --2- 172.21.15.32:0/237626016 >> v2:172.21.15.61:3300/0 conn(0x7fd0600abeb0 0x7fd0600ab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:25.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05e7fc700 1 -- 172.21.15.32:0/237626016 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2594145424 0 0) 0x7fd040003230 con 0x7fd060062660 2020-12-05T17:41:25.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05e7fc700 1 -- 172.21.15.32:0/237626016 >> v2:172.21.15.61:3300/0 conn(0x7fd0600abeb0 msgr2=0x7fd0600ab610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:25.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05e7fc700 1 --2- 172.21.15.32:0/237626016 >> v2:172.21.15.61:3300/0 conn(0x7fd0600abeb0 0x7fd0600ab610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:25.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05e7fc700 1 -- 172.21.15.32:0/237626016 >> v1:172.21.15.61:6789/0 conn(0x7fd0600ae6e0 legacy=0x7fd06013dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:25.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05e7fc700 1 -- 172.21.15.32:0/237626016 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0600b0bd0 con 0x7fd060062660 2020-12-05T17:41:25.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05e7fc700 1 -- 172.21.15.32:0/237626016 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fd0480016d0 con 0x7fd060062660 2020-12-05T17:41:25.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05e7fc700 1 -- 172.21.15.32:0/237626016 >> v1:172.21.15.32:6789/0 conn(0x7fd060062660 legacy=0x7fd0600ad190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:25.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05e7fc700 1 --2- 172.21.15.32:0/237626016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd040003720 0x7fd040003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:25.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05e7fc700 1 --2- 172.21.15.32:0/237626016 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0400044d0 0x7fd040004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:25.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05e7fc700 1 --2- 172.21.15.32:0/237626016 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd040004e40 0x7fd04000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:25.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05e7fc700 1 -- 172.21.15.32:0/237626016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0600ad8a0 con 0x7fd040003720 2020-12-05T17:41:25.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05e7fc700 1 -- 172.21.15.32:0/237626016 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd040004250 con 0x7fd0400044d0 2020-12-05T17:41:25.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05e7fc700 1 -- 172.21.15.32:0/237626016 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04000bc10 con 0x7fd040004e40 2020-12-05T17:41:25.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05effd700 1 --2- 172.21.15.32:0/237626016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd040003720 0x7fd040003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:25.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd0567fc700 1 --2- 172.21.15.32:0/237626016 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0400044d0 0x7fd040004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:25.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05f7fe700 1 --2- 172.21.15.32:0/237626016 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd040004e40 0x7fd04000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:25.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05effd700 1 -- 172.21.15.32:0/237626016 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd040004e40 msgr2=0x7fd04000b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05effd700 1 --2- 172.21.15.32:0/237626016 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd040004e40 0x7fd04000b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05effd700 1 -- 172.21.15.32:0/237626016 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0400044d0 msgr2=0x7fd040004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05effd700 1 --2- 172.21.15.32:0/237626016 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0400044d0 0x7fd040004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05effd700 1 -- 172.21.15.32:0/237626016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd040003230 con 0x7fd040003720 2020-12-05T17:41:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05f7fe700 1 --2- 172.21.15.32:0/237626016 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd040004e40 0x7fd04000b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:25.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05f7fe700 1 -- 172.21.15.32:0/237626016 reap_dead start 2020-12-05T17:41:25.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05effd700 1 --2- 172.21.15.32:0/237626016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd040003720 0x7fd040003b40 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0x7fd05000c000 tx=0x7fd05000a040).ready entity=mon.0 client_cookie=a3910428fd0ee3d2 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:25.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05e7fc700 1 -- 172.21.15.32:0/237626016 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd05000fb90 con 0x7fd040003720 2020-12-05T17:41:25.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05e7fc700 1 -- 172.21.15.32:0/237626016 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05000fcf0 con 0x7fd040003720 2020-12-05T17:41:25.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05e7fc700 1 -- 172.21.15.32:0/237626016 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd050014610 con 0x7fd040003720 2020-12-05T17:41:25.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05ffff700 1 -- 172.21.15.32:0/237626016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd040003720 msgr2=0x7fd040003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:25.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05ffff700 1 --2- 172.21.15.32:0/237626016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd040003720 0x7fd040003b40 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0x7fd05000c000 tx=0x7fd05000a040).stop 2020-12-05T17:41:25.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05ffff700 1 -- 172.21.15.32:0/237626016 shutdown_connections 2020-12-05T17:41:25.430 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05ffff700 1 --2- 172.21.15.32:0/237626016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd040003720 0x7fd040003b40 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:25.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05ffff700 1 -- 172.21.15.32:0/237626016 >> 172.21.15.32:0/237626016 conn(0x7fd0600b5530 msgr2=0x7fd0600b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:25.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05ffff700 1 -- 172.21.15.32:0/237626016 shutdown_connections 2020-12-05T17:41:25.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.414+0000 7fd05ffff700 1 -- 172.21.15.32:0/237626016 wait complete. 2020-12-05T17:41:25.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.418+0000 7fd05ffff700 1 Processor -- start 2020-12-05T17:41:25.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.418+0000 7fd05ffff700 1 -- start start 2020-12-05T17:41:25.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.418+0000 7fd05ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd040003720 0x7fd06014d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:25.435 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.418+0000 7fd05ffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd060064540 0x7fd060063e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:25.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.418+0000 7fd05ffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd06013e8a0 0x7fd06013ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:25.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.418+0000 7fd05ffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06013fa00 con 0x7fd040003720 2020-12-05T17:41:25.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.418+0000 7fd05ffff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06013dc90 con 0x7fd060064540 2020-12-05T17:41:25.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.418+0000 7fd05ffff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06013dec0 con 0x7fd06013e8a0 2020-12-05T17:41:25.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.418+0000 7fd05effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd040003720 0x7fd06014d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:25.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.418+0000 7fd05effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd040003720 0x7fd06014d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40702/0 (socket says 172.21.15.32:40702) 2020-12-05T17:41:25.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.418+0000 7fd05effd700 1 -- 172.21.15.32:0/3968485760 learned_addr learned my addr 172.21.15.32:0/3968485760 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:25.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.418+0000 7fd05f7fe700 1 --2- 172.21.15.32:0/3968485760 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd06013e8a0 0x7fd06013ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:25.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.418+0000 7fd0567fc700 1 --2- 172.21.15.32:0/3968485760 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd060064540 0x7fd060063e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:25.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.422+0000 7fd05effd700 1 -- 172.21.15.32:0/3968485760 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd06013e8a0 msgr2=0x7fd06013ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:25.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.422+0000 7fd05effd700 1 --2- 172.21.15.32:0/3968485760 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd06013e8a0 0x7fd06013ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:25.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.422+0000 7fd05effd700 1 -- 172.21.15.32:0/3968485760 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd060064540 msgr2=0x7fd060063e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:25.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.422+0000 7fd05effd700 1 --2- 172.21.15.32:0/3968485760 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd060064540 0x7fd060063e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:25.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.422+0000 7fd05effd700 1 -- 172.21.15.32:0/3968485760 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06013e160 con 0x7fd040003720 2020-12-05T17:41:25.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.422+0000 7fd05f7fe700 1 --2- 172.21.15.32:0/3968485760 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd06013e8a0 0x7fd06013ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:25.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.422+0000 7fd0567fc700 1 --2- 172.21.15.32:0/3968485760 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd060064540 0x7fd060063e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:25.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.422+0000 7fd05effd700 1 --2- 172.21.15.32:0/3968485760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd040003720 0x7fd06014d160 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 rx=0x7fd05000c000 tx=0x7fd05000a040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:25.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.422+0000 7fd05cff9700 1 -- 172.21.15.32:0/3968485760 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0500086e0 con 0x7fd040003720 2020-12-05T17:41:25.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.422+0000 7fd05cff9700 1 -- 172.21.15.32:0/3968485760 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd050008840 con 0x7fd040003720 2020-12-05T17:41:25.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.422+0000 7fd05ffff700 1 -- 172.21.15.32:0/3968485760 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd06013e400 con 0x7fd040003720 2020-12-05T17:41:25.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.422+0000 7fd05cff9700 1 -- 172.21.15.32:0/3968485760 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd050026610 con 0x7fd040003720 2020-12-05T17:41:25.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.422+0000 7fd05ffff700 1 -- 172.21.15.32:0/3968485760 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0600636f0 con 0x7fd040003720 2020-12-05T17:41:25.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.422+0000 7fd05ffff700 1 -- 172.21.15.32:0/3968485760 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd06013e590 con 0x7fd040003720 2020-12-05T17:41:25.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.422+0000 7fd05cff9700 1 -- 172.21.15.32:0/3968485760 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd050014420 con 0x7fd040003720 2020-12-05T17:41:25.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.426+0000 7fd05cff9700 1 --2- 172.21.15.32:0/3968485760 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd044055500 0x7fd0440579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:25.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.426+0000 7fd05cff9700 1 -- 172.21.15.32:0/3968485760 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fd050071a60 con 0x7fd040003720 2020-12-05T17:41:25.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.426+0000 7fd0567fc700 1 --2- 172.21.15.32:0/3968485760 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd044055500 0x7fd0440579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:25.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.426+0000 7fd0567fc700 1 --2- 172.21.15.32:0/3968485760 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd044055500 0x7fd0440579c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fd048005000 tx=0x7fd048009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:25.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.430+0000 7fd05cff9700 1 -- 172.21.15.32:0/3968485760 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd050047ab0 con 0x7fd040003720 2020-12-05T17:41:25.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.618+0000 7fd05ffff700 1 -- 172.21.15.32:0/3968485760 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd060057050 con 0x7fd040003720 2020-12-05T17:41:25.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.942+0000 7fd05cff9700 1 -- 172.21.15.32:0/3968485760 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v64) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd05004b0d0 con 0x7fd040003720 2020-12-05T17:41:25.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.946+0000 7fd05ffff700 1 -- 172.21.15.32:0/3968485760 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd044055500 msgr2=0x7fd0440579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:25.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.946+0000 7fd05ffff700 1 --2- 172.21.15.32:0/3968485760 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd044055500 0x7fd0440579c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fd048005000 tx=0x7fd048009040).stop 2020-12-05T17:41:25.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.946+0000 7fd05ffff700 1 -- 172.21.15.32:0/3968485760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd040003720 msgr2=0x7fd06014d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:25.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.946+0000 7fd05ffff700 1 --2- 172.21.15.32:0/3968485760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd040003720 0x7fd06014d160 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 rx=0x7fd05000c000 tx=0x7fd05000a040).stop 2020-12-05T17:41:25.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.946+0000 7fd05ffff700 1 -- 172.21.15.32:0/3968485760 shutdown_connections 2020-12-05T17:41:25.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.946+0000 7fd05ffff700 1 --2- 172.21.15.32:0/3968485760 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd044055500 0x7fd0440579c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:25.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.946+0000 7fd05ffff700 1 --2- 172.21.15.32:0/3968485760 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd06013e8a0 0x7fd06013ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:25.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.946+0000 7fd05ffff700 1 --2- 172.21.15.32:0/3968485760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd040003720 0x7fd06014d160 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:25.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.946+0000 7fd05ffff700 1 --2- 172.21.15.32:0/3968485760 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd060064540 0x7fd060063e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:25.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.946+0000 7fd05ffff700 1 -- 172.21.15.32:0/3968485760 >> 172.21.15.32:0/3968485760 conn(0x7fd0600b5530 msgr2=0x7fd060143400 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:25.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.946+0000 7fd05ffff700 1 -- 172.21.15.32:0/3968485760 shutdown_connections 2020-12-05T17:41:25.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:25.946+0000 7fd05ffff700 1 -- 172.21.15.32:0/3968485760 wait complete. 2020-12-05T17:41:25.971 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-12-05T17:41:26.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe6d31700 1 Processor -- start 2020-12-05T17:41:26.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe6d31700 1 -- start start 2020-12-05T17:41:26.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe6d31700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f8fe00b2960 0x7f8fe0140400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:26.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe6d31700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8fe00667f0 0x7f8fe00ae830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:26.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe6d31700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fe00ad9b0 con 0x7f8fe00ad280 2020-12-05T17:41:26.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe6d31700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fe00b32c0 con 0x7f8fe00667f0 2020-12-05T17:41:26.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe6d31700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fe013f4f0 con 0x7f8fe00b2960 2020-12-05T17:41:26.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe6530700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8fe00667f0 0x7f8fe00ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:26.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe6530700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8fe00667f0 0x7f8fe00ae830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40712/0 (socket says 172.21.15.32:40712) 2020-12-05T17:41:26.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe552e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8fe00ad280 0x7f8fe00660e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43362/0 (socket says 172.21.15.32:43362) 2020-12-05T17:41:26.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe6530700 1 -- 172.21.15.32:0/2649555407 learned_addr learned my addr 172.21.15.32:0/2649555407 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:26.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe5d2f700 1 --2- 172.21.15.32:0/2649555407 >> v2:172.21.15.61:3300/0 conn(0x7f8fe00b2960 0x7f8fe0140400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:26.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe4d2d700 1 -- 172.21.15.32:0/2649555407 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3814905817 0 0) 0x7f8fe00ad9b0 con 0x7f8fe00ad280 2020-12-05T17:41:26.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe6530700 1 -- 172.21.15.32:0/2649555407 >> v2:172.21.15.61:3300/0 conn(0x7f8fe00b2960 msgr2=0x7f8fe0140400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:26.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe6530700 1 --2- 172.21.15.32:0/2649555407 >> v2:172.21.15.61:3300/0 conn(0x7f8fe00b2960 0x7f8fe0140400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:26.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe6530700 1 -- 172.21.15.32:0/2649555407 >> v1:172.21.15.32:6789/0 conn(0x7f8fe00ad280 legacy=0x7f8fe00660e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:26.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe6530700 1 -- 172.21.15.32:0/2649555407 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe00b3020 con 0x7f8fe00667f0 2020-12-05T17:41:26.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe5d2f700 1 --2- 172.21.15.32:0/2649555407 >> v2:172.21.15.61:3300/0 conn(0x7f8fe00b2960 0x7f8fe0140400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:26.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe6530700 1 --2- 172.21.15.32:0/2649555407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fe00667f0 0x7f8fe00ae830 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7f8fd80032b0 tx=0x7f8fd8009270).ready entity=mon.0 client_cookie=fb2b8494df7065d8 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:26.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe4d2d700 1 -- 172.21.15.32:0/2649555407 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fd800bdf0 con 0x7f8fe00667f0 2020-12-05T17:41:26.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe4d2d700 1 -- 172.21.15.32:0/2649555407 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd8010470 con 0x7f8fe00667f0 2020-12-05T17:41:26.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe4d2d700 1 -- 172.21.15.32:0/2649555407 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fd80107d0 con 0x7f8fe00667f0 2020-12-05T17:41:26.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe6d31700 1 -- 172.21.15.32:0/2649555407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fe00667f0 msgr2=0x7f8fe00ae830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:26.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.094+0000 7f8fe6d31700 1 --2- 172.21.15.32:0/2649555407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fe00667f0 0x7f8fe00ae830 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7f8fd80032b0 tx=0x7f8fd8009270).stop 2020-12-05T17:41:26.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.098+0000 7f8fe6d31700 1 -- 172.21.15.32:0/2649555407 shutdown_connections 2020-12-05T17:41:26.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.098+0000 7f8fe6d31700 1 --2- 172.21.15.32:0/2649555407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fe00667f0 0x7f8fe00ae830 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:26.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.098+0000 7f8fe6d31700 1 --2- 172.21.15.32:0/2649555407 >> v2:172.21.15.61:3300/0 conn(0x7f8fe00b2960 0x7f8fe0140400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:26.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.098+0000 7f8fe6d31700 1 -- 172.21.15.32:0/2649555407 >> 172.21.15.32:0/2649555407 conn(0x7f8fe0061010 msgr2=0x7f8fe0061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:26.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.098+0000 7f8fe6d31700 1 -- 172.21.15.32:0/2649555407 shutdown_connections 2020-12-05T17:41:26.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.098+0000 7f8fe6d31700 1 -- 172.21.15.32:0/2649555407 wait complete. 2020-12-05T17:41:26.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.098+0000 7f8fe6d31700 1 Processor -- start 2020-12-05T17:41:26.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.098+0000 7f8fe6d31700 1 -- start start 2020-12-05T17:41:26.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.098+0000 7f8fe6d31700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fe00667f0 0x7f8fe0064330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:26.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.098+0000 7f8fe6d31700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8fe00ad280 0x7f8fe0066140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:26.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.098+0000 7f8fe6d31700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8fe00b2960 0x7f8fe013b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:26.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.098+0000 7f8fe6d31700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe0110120 con 0x7f8fe00667f0 2020-12-05T17:41:26.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.098+0000 7f8fe6d31700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe01103c0 con 0x7f8fe00ad280 2020-12-05T17:41:26.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.098+0000 7f8fe6d31700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe0110660 con 0x7f8fe00b2960 2020-12-05T17:41:26.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.098+0000 7f8fe5d2f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fe00667f0 0x7f8fe0064330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:26.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.098+0000 7f8fe5d2f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fe00667f0 0x7f8fe0064330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40718/0 (socket says 172.21.15.32:40718) 2020-12-05T17:41:26.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.098+0000 7f8fe5d2f700 1 -- 172.21.15.32:0/861898758 learned_addr learned my addr 172.21.15.32:0/861898758 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:26.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.098+0000 7f8fe552e700 1 --2- 172.21.15.32:0/861898758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8fe00ad280 0x7f8fe0066140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:26.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.098+0000 7f8fe6530700 1 --2- 172.21.15.32:0/861898758 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8fe00b2960 0x7f8fe013b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:26.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.102+0000 7f8fe5d2f700 1 -- 172.21.15.32:0/861898758 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8fe00b2960 msgr2=0x7f8fe013b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:26.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.102+0000 7f8fe5d2f700 1 --2- 172.21.15.32:0/861898758 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8fe00b2960 0x7f8fe013b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:26.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.102+0000 7f8fe5d2f700 1 -- 172.21.15.32:0/861898758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8fe00ad280 msgr2=0x7f8fe0066140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:26.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.102+0000 7f8fe5d2f700 1 --2- 172.21.15.32:0/861898758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8fe00ad280 0x7f8fe0066140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:26.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.102+0000 7f8fe5d2f700 1 -- 172.21.15.32:0/861898758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe0110b00 con 0x7f8fe00667f0 2020-12-05T17:41:26.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.102+0000 7f8fe5d2f700 1 --2- 172.21.15.32:0/861898758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fe00667f0 0x7f8fe0064330 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7f8fdc0068b0 tx=0x7f8fdc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:26.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.102+0000 7f8fceffd700 1 -- 172.21.15.32:0/861898758 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fdc0127e0 con 0x7f8fe00667f0 2020-12-05T17:41:26.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.102+0000 7f8fceffd700 1 -- 172.21.15.32:0/861898758 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fdc012940 con 0x7f8fe00667f0 2020-12-05T17:41:26.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.102+0000 7f8fe6d31700 1 -- 172.21.15.32:0/861898758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe010fd50 con 0x7f8fe00667f0 2020-12-05T17:41:26.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.102+0000 7f8fceffd700 1 -- 172.21.15.32:0/861898758 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fdc012ca0 con 0x7f8fe00667f0 2020-12-05T17:41:26.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.102+0000 7f8fe6d31700 1 -- 172.21.15.32:0/861898758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe0063f20 con 0x7f8fe00667f0 2020-12-05T17:41:26.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.102+0000 7f8fe6d31700 1 -- 172.21.15.32:0/861898758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fe0057050 con 0x7f8fe00667f0 2020-12-05T17:41:26.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.102+0000 7f8fceffd700 1 -- 172.21.15.32:0/861898758 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8fdc013480 con 0x7f8fe00667f0 2020-12-05T17:41:26.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.102+0000 7f8fceffd700 1 --2- 172.21.15.32:0/861898758 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8fc00554e0 0x7f8fc00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:26.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.102+0000 7f8fceffd700 1 -- 172.21.15.32:0/861898758 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f8fdc06e630 con 0x7f8fe00667f0 2020-12-05T17:41:26.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.102+0000 7f8fe552e700 1 --2- 172.21.15.32:0/861898758 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8fc00554e0 0x7f8fc00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:26.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.102+0000 7f8fe552e700 1 --2- 172.21.15.32:0/861898758 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8fc00554e0 0x7f8fc00579a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f8fd000d000 tx=0x7f8fd0007a40).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:26.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.110+0000 7f8fceffd700 1 -- 172.21.15.32:0/861898758 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8fdc044720 con 0x7f8fe00667f0 2020-12-05T17:41:26.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.250+0000 7f8fe6d31700 1 -- 172.21.15.32:0/861898758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8fe0063f20 con 0x7f8fe00667f0 2020-12-05T17:41:26.993 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.986+0000 7f8fceffd700 1 -- 172.21.15.32:0/861898758 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v131) v1 ==== 185+0+0 (secure 0 0 0) 0x7f8fe0063f20 con 0x7f8fe00667f0 2020-12-05T17:41:26.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.990+0000 7f8fe6d31700 1 -- 172.21.15.32:0/861898758 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8fc00554e0 msgr2=0x7f8fc00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:26.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.990+0000 7f8fe6d31700 1 --2- 172.21.15.32:0/861898758 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8fc00554e0 0x7f8fc00579a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f8fd000d000 tx=0x7f8fd0007a40).stop 2020-12-05T17:41:26.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.990+0000 7f8fe6d31700 1 -- 172.21.15.32:0/861898758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fe00667f0 msgr2=0x7f8fe0064330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:26.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.990+0000 7f8fe6d31700 1 --2- 172.21.15.32:0/861898758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fe00667f0 0x7f8fe0064330 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7f8fdc0068b0 tx=0x7f8fdc008040).stop 2020-12-05T17:41:26.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.990+0000 7f8fe6d31700 1 -- 172.21.15.32:0/861898758 shutdown_connections 2020-12-05T17:41:26.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.990+0000 7f8fe6d31700 1 --2- 172.21.15.32:0/861898758 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8fc00554e0 0x7f8fc00579a0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:26.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.990+0000 7f8fe6d31700 1 --2- 172.21.15.32:0/861898758 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8fe00b2960 0x7f8fe013b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.990+0000 7f8fe6d31700 1 --2- 172.21.15.32:0/861898758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fe00667f0 0x7f8fe0064330 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.990+0000 7f8fe6d31700 1 --2- 172.21.15.32:0/861898758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8fe00ad280 0x7f8fe0066140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.990+0000 7f8fe6d31700 1 -- 172.21.15.32:0/861898758 >> 172.21.15.32:0/861898758 conn(0x7f8fe0061010 msgr2=0x7f8fe005f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.990+0000 7f8fe6d31700 1 -- 172.21.15.32:0/861898758 shutdown_connections 2020-12-05T17:41:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:26.990+0000 7f8fe6d31700 1 -- 172.21.15.32:0/861898758 wait complete. 2020-12-05T17:41:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2020-12-05T17:41:27.012 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-12-05T17:41:27.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7def59e700 1 Processor -- start 2020-12-05T17:41:27.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7def59e700 1 -- start start 2020-12-05T17:41:27.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7def59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7df005f3b0 0x7f7df005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:27.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7def59e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f7df00b2fb0 0x7f7df0061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:27.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7def59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7df00b2df0 con 0x7f7df005feb0 2020-12-05T17:41:27.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7def59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7df00b3c90 con 0x7f7df005f3b0 2020-12-05T17:41:27.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7def59e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7df00b2b50 con 0x7f7df00b2fb0 2020-12-05T17:41:27.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7dee59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7df005f3b0 0x7f7df005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:27.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7dee59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7df005f3b0 0x7f7df005f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40726/0 (socket says 172.21.15.32:40726) 2020-12-05T17:41:27.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7dee59c700 1 -- 172.21.15.32:0/3652901016 learned_addr learned my addr 172.21.15.32:0/3652901016 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:27.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7deed9d700 1 --2- 172.21.15.32:0/3652901016 >> v2:172.21.15.61:3300/0 conn(0x7f7df00b2fb0 0x7f7df0061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:27.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7dee59c700 1 -- 172.21.15.32:0/3652901016 >> v2:172.21.15.61:3300/0 conn(0x7f7df00b2fb0 msgr2=0x7f7df0061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:27.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7dee59c700 1 --2- 172.21.15.32:0/3652901016 >> v2:172.21.15.61:3300/0 conn(0x7f7df00b2fb0 0x7f7df0061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:27.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7dee59c700 1 -- 172.21.15.32:0/3652901016 >> v1:172.21.15.61:6789/0 conn(0x7f7df005feb0 legacy=0x7f7df013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:27.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7dee59c700 1 -- 172.21.15.32:0/3652901016 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df0142820 con 0x7f7df005f3b0 2020-12-05T17:41:27.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7dee59c700 1 --2- 172.21.15.32:0/3652901016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df005f3b0 0x7f7df005f790 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f7de0002f30 tx=0x7f7de0006e80).ready entity=mon.0 client_cookie=a8e5994ccfd3300c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:27.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7ded59a700 1 -- 172.21.15.32:0/3652901016 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7de00097f0 con 0x7f7df005f3b0 2020-12-05T17:41:27.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7ded59a700 1 -- 172.21.15.32:0/3652901016 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de0009950 con 0x7f7df005f3b0 2020-12-05T17:41:27.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7ded59a700 1 -- 172.21.15.32:0/3652901016 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7de0009cb0 con 0x7f7df005f3b0 2020-12-05T17:41:27.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7def59e700 1 -- 172.21.15.32:0/3652901016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df005f3b0 msgr2=0x7f7df005f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:27.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7def59e700 1 --2- 172.21.15.32:0/3652901016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df005f3b0 0x7f7df005f790 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f7de0002f30 tx=0x7f7de0006e80).stop 2020-12-05T17:41:27.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7def59e700 1 -- 172.21.15.32:0/3652901016 shutdown_connections 2020-12-05T17:41:27.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7def59e700 1 --2- 172.21.15.32:0/3652901016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df005f3b0 0x7f7df005f790 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:27.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7def59e700 1 --2- 172.21.15.32:0/3652901016 >> v2:172.21.15.61:3300/0 conn(0x7f7df00b2fb0 0x7f7df0061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:27.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7def59e700 1 -- 172.21.15.32:0/3652901016 >> 172.21.15.32:0/3652901016 conn(0x7f7df00b8f20 msgr2=0x7f7df00b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:27.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7def59e700 1 -- 172.21.15.32:0/3652901016 shutdown_connections 2020-12-05T17:41:27.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.134+0000 7f7def59e700 1 -- 172.21.15.32:0/3652901016 wait complete. 2020-12-05T17:41:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.138+0000 7f7def59e700 1 Processor -- start 2020-12-05T17:41:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.138+0000 7f7def59e700 1 -- start start 2020-12-05T17:41:27.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.138+0000 7f7def59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df005f3b0 0x7f7df01484a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:27.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.138+0000 7f7def59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7df005feb0 0x7f7df013d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:27.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.138+0000 7f7def59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7df00b2fb0 0x7f7df0145e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:27.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.138+0000 7f7def59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df0146350 con 0x7f7df005f3b0 2020-12-05T17:41:27.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.138+0000 7f7def59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df0146610 con 0x7f7df00b2fb0 2020-12-05T17:41:27.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.138+0000 7f7dee59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df005f3b0 0x7f7df01484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:27.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.138+0000 7f7dee59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df005f3b0 0x7f7df01484a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40732/0 (socket says 172.21.15.32:40732) 2020-12-05T17:41:27.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.138+0000 7f7dee59c700 1 -- 172.21.15.32:0/3977154695 learned_addr learned my addr 172.21.15.32:0/3977154695 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:27.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.138+0000 7f7def59e700 1 -- 172.21.15.32:0/3977154695 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df01468b0 con 0x7f7df005feb0 2020-12-05T17:41:27.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.138+0000 7f7deed9d700 1 --2- 172.21.15.32:0/3977154695 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7df00b2fb0 0x7f7df0145e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:27.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.138+0000 7f7dedd9b700 1 --2- 172.21.15.32:0/3977154695 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7df005feb0 0x7f7df013d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:27.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.138+0000 7f7dee59c700 1 -- 172.21.15.32:0/3977154695 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7df005feb0 msgr2=0x7f7df013d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:27.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.138+0000 7f7dee59c700 1 --2- 172.21.15.32:0/3977154695 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7df005feb0 0x7f7df013d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:27.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.138+0000 7f7dee59c700 1 -- 172.21.15.32:0/3977154695 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7df00b2fb0 msgr2=0x7f7df0145e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:27.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.138+0000 7f7dee59c700 1 --2- 172.21.15.32:0/3977154695 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7df00b2fb0 0x7f7df0145e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:27.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.138+0000 7f7dee59c700 1 -- 172.21.15.32:0/3977154695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df0146b50 con 0x7f7df005f3b0 2020-12-05T17:41:27.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.138+0000 7f7dee59c700 1 --2- 172.21.15.32:0/3977154695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df005f3b0 0x7f7df01484a0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7f7de0002f30 tx=0x7f7de0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:27.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.138+0000 7f7ddf7fe700 1 -- 172.21.15.32:0/3977154695 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7de0011c80 con 0x7f7df005f3b0 2020-12-05T17:41:27.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.138+0000 7f7ddf7fe700 1 -- 172.21.15.32:0/3977154695 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de0011de0 con 0x7f7df005f3b0 2020-12-05T17:41:27.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.142+0000 7f7def59e700 1 -- 172.21.15.32:0/3977154695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df0146ff0 con 0x7f7df005f3b0 2020-12-05T17:41:27.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.142+0000 7f7ddf7fe700 1 -- 172.21.15.32:0/3977154695 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7de001e650 con 0x7f7df005f3b0 2020-12-05T17:41:27.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.142+0000 7f7def59e700 1 -- 172.21.15.32:0/3977154695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df0151740 con 0x7f7df005f3b0 2020-12-05T17:41:27.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.142+0000 7f7def59e700 1 -- 172.21.15.32:0/3977154695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df0057050 con 0x7f7df005f3b0 2020-12-05T17:41:27.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.142+0000 7f7ddf7fe700 1 -- 172.21.15.32:0/3977154695 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7de0025030 con 0x7f7df005f3b0 2020-12-05T17:41:27.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.142+0000 7f7ddf7fe700 1 --2- 172.21.15.32:0/3977154695 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7dcc055330 0x7f7dcc0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:27.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.142+0000 7f7ddf7fe700 1 -- 172.21.15.32:0/3977154695 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f7de006f230 con 0x7f7df005f3b0 2020-12-05T17:41:27.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.142+0000 7f7dedd9b700 1 --2- 172.21.15.32:0/3977154695 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7dcc055330 0x7f7dcc0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:27.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.142+0000 7f7dedd9b700 1 --2- 172.21.15.32:0/3977154695 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7dcc055330 0x7f7dcc0577f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f7de400b000 tx=0x7f7de4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:27.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.150+0000 7f7ddf7fe700 1 -- 172.21.15.32:0/3977154695 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7de00455e0 con 0x7f7df005f3b0 2020-12-05T17:41:27.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.298+0000 7f7def59e700 1 -- 172.21.15.32:0/3977154695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7df0146ce0 con 0x7f7df005f3b0 2020-12-05T17:41:28.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:27.998+0000 7f7ddf7fe700 1 -- 172.21.15.32:0/3977154695 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v132) v1 ==== 173+0+0 (secure 0 0 0) 0x7f7de00230d0 con 0x7f7df005f3b0 2020-12-05T17:41:28.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.002+0000 7f7def59e700 1 -- 172.21.15.32:0/3977154695 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7dcc055330 msgr2=0x7f7dcc0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:28.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.002+0000 7f7def59e700 1 --2- 172.21.15.32:0/3977154695 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7dcc055330 0x7f7dcc0577f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f7de400b000 tx=0x7f7de4009040).stop 2020-12-05T17:41:28.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.002+0000 7f7def59e700 1 -- 172.21.15.32:0/3977154695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df005f3b0 msgr2=0x7f7df01484a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:28.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.002+0000 7f7def59e700 1 --2- 172.21.15.32:0/3977154695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df005f3b0 0x7f7df01484a0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7f7de0002f30 tx=0x7f7de0015000).stop 2020-12-05T17:41:28.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.002+0000 7f7def59e700 1 -- 172.21.15.32:0/3977154695 shutdown_connections 2020-12-05T17:41:28.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.002+0000 7f7def59e700 1 --2- 172.21.15.32:0/3977154695 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7df00b2fb0 0x7f7df0145e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:28.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.002+0000 7f7def59e700 1 --2- 172.21.15.32:0/3977154695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df005f3b0 0x7f7df01484a0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:28.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.002+0000 7f7def59e700 1 --2- 172.21.15.32:0/3977154695 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7dcc055330 0x7f7dcc0577f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:28.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.002+0000 7f7def59e700 1 --2- 172.21.15.32:0/3977154695 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7df005feb0 0x7f7df013d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:28.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.002+0000 7f7def59e700 1 -- 172.21.15.32:0/3977154695 >> 172.21.15.32:0/3977154695 conn(0x7f7df00b8f20 msgr2=0x7f7df00b7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:28.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.002+0000 7f7def59e700 1 -- 172.21.15.32:0/3977154695 shutdown_connections 2020-12-05T17:41:28.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.002+0000 7f7def59e700 1 -- 172.21.15.32:0/3977154695 wait complete. 2020-12-05T17:41:28.009 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2020-12-05T17:41:28.021 INFO:tasks.ceph.mds.a:Restarting daemon 2020-12-05T17:41:28.021 INFO:tasks.ceph.mds.a:Stopping old one... 2020-12-05T17:41:28.022 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-12-05T17:41:28.022 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:41:28.022 INFO:tasks.ceph.mds.b:Restarting daemon 2020-12-05T17:41:28.023 INFO:tasks.ceph.mds.b:Stopping old one... 2020-12-05T17:41:28.023 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-12-05T17:41:28.023 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:41:28.032 INFO:tasks.ceph.mds.a:Stopped 2020-12-05T17:41:28.032 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-12-05T17:41:28.033 INFO:tasks.ceph.mds.b:Stopped 2020-12-05T17:41:28.034 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-12-05T17:41:28.075 INFO:tasks.ceph.mds.a:Started 2020-12-05T17:41:28.075 DEBUG:teuthology.parallel:result is None 2020-12-05T17:41:28.078 INFO:tasks.ceph.mds.b:Started 2020-12-05T17:41:28.078 DEBUG:teuthology.parallel:result is None 2020-12-05T17:41:28.079 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-12-05T17:41:28.156 INFO:tasks.ceph.mds.b.smithi061.stdout:starting mds.b at 2020-12-05T17:41:28.164 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:41:28.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.194+0000 7fb3cdb88700 1 Processor -- start 2020-12-05T17:41:28.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.194+0000 7fb3cdb88700 1 -- start start 2020-12-05T17:41:28.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3cdb88700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3c8063250 0x7fb3c80acde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:28.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3cdb88700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3c8138630 0x7fb3c8138ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:28.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3cdb88700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3c80ad2d0 con 0x7fb3c8062900 2020-12-05T17:41:28.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3cdb88700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3c8062ce0 con 0x7fb3c8138630 2020-12-05T17:41:28.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3cdb88700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c8066980 con 0x7fb3c8063250 2020-12-05T17:41:28.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3ccb86700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb3c8062900 0x7fb3c8142870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43400/0 (socket says 172.21.15.32:43400) 2020-12-05T17:41:28.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3ccb86700 1 -- 172.21.15.32:0/1586574285 learned_addr learned my addr 172.21.15.32:0/1586574285 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:28.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3bffff700 1 --2- 172.21.15.32:0/1586574285 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3c8063250 0x7fb3c80acde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:28.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3cd387700 1 --2- 172.21.15.32:0/1586574285 >> v2:172.21.15.61:3300/0 conn(0x7fb3c8138630 0x7fb3c8138ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:28.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3bf7fe700 1 -- 172.21.15.32:0/1586574285 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 551803003 0 0) 0x7fb3c80ad2d0 con 0x7fb3c8062900 2020-12-05T17:41:28.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3bf7fe700 1 -- 172.21.15.32:0/1586574285 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3b4004340 con 0x7fb3c8062900 2020-12-05T17:41:28.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3bffff700 1 -- 172.21.15.32:0/1586574285 >> v2:172.21.15.61:3300/0 conn(0x7fb3c8138630 msgr2=0x7fb3c8138ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:28.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3bffff700 1 --2- 172.21.15.32:0/1586574285 >> v2:172.21.15.61:3300/0 conn(0x7fb3c8138630 0x7fb3c8138ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:28.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3bffff700 1 -- 172.21.15.32:0/1586574285 >> v1:172.21.15.32:6789/0 conn(0x7fb3c8062900 legacy=0x7fb3c8142870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:28.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3bffff700 1 -- 172.21.15.32:0/1586574285 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c8066c20 con 0x7fb3c8063250 2020-12-05T17:41:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3bffff700 1 --2- 172.21.15.32:0/1586574285 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3c8063250 0x7fb3c80acde0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fb3b8002f30 tx=0x7fb3b8006dc0).ready entity=mon.2 client_cookie=46272983d1919911 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3bf7fe700 1 -- 172.21.15.32:0/1586574285 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3b80096a0 con 0x7fb3c8063250 2020-12-05T17:41:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3bf7fe700 1 -- 172.21.15.32:0/1586574285 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b8009800 con 0x7fb3c8063250 2020-12-05T17:41:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3cdb88700 1 -- 172.21.15.32:0/1586574285 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3c8063250 msgr2=0x7fb3c80acde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3bf7fe700 1 -- 172.21.15.32:0/1586574285 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3b8009b60 con 0x7fb3c8063250 2020-12-05T17:41:28.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3cdb88700 1 --2- 172.21.15.32:0/1586574285 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3c8063250 0x7fb3c80acde0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fb3b8002f30 tx=0x7fb3b8006dc0).stop 2020-12-05T17:41:28.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3cdb88700 1 -- 172.21.15.32:0/1586574285 shutdown_connections 2020-12-05T17:41:28.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3cdb88700 1 --2- 172.21.15.32:0/1586574285 >> v2:172.21.15.61:3300/0 conn(0x7fb3c8138630 0x7fb3c8138ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:28.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3cdb88700 1 --2- 172.21.15.32:0/1586574285 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3c8063250 0x7fb3c80acde0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:28.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3cdb88700 1 -- 172.21.15.32:0/1586574285 >> 172.21.15.32:0/1586574285 conn(0x7fb3c80b4f10 msgr2=0x7fb3c80b52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:28.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3cdb88700 1 -- 172.21.15.32:0/1586574285 shutdown_connections 2020-12-05T17:41:28.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3cdb88700 1 -- 172.21.15.32:0/1586574285 wait complete. 2020-12-05T17:41:28.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3cdb88700 1 Processor -- start 2020-12-05T17:41:28.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.198+0000 7fb3cdb88700 1 -- start start 2020-12-05T17:41:28.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3cdb88700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3c8062900 0x7fb3c81255c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:28.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3cdb88700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3c8063250 0x7fb3c814ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:28.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3cdb88700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3c8138630 0x7fb3c8150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:28.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3cdb88700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c814ddf0 con 0x7fb3c8138630 2020-12-05T17:41:28.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3cdb88700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c80b2e90 con 0x7fb3c8063250 2020-12-05T17:41:28.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3cdb88700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c8150400 con 0x7fb3c8062900 2020-12-05T17:41:28.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3cd387700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3c8138630 0x7fb3c8150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:28.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3ccb86700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3c8062900 0x7fb3c81255c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:28.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3cd387700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3c8138630 0x7fb3c8150ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40762/0 (socket says 172.21.15.32:40762) 2020-12-05T17:41:28.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3bffff700 1 --2- 172.21.15.32:0/2775770429 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3c8063250 0x7fb3c814ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:28.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3cd387700 1 -- 172.21.15.32:0/2775770429 learned_addr learned my addr 172.21.15.32:0/2775770429 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:28.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3cd387700 1 -- 172.21.15.32:0/2775770429 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3c8062900 msgr2=0x7fb3c81255c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3cd387700 1 --2- 172.21.15.32:0/2775770429 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3c8062900 0x7fb3c81255c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3cd387700 1 -- 172.21.15.32:0/2775770429 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3c8063250 msgr2=0x7fb3c814ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3cd387700 1 --2- 172.21.15.32:0/2775770429 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3c8063250 0x7fb3c814ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3cd387700 1 -- 172.21.15.32:0/2775770429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c80adb40 con 0x7fb3c8138630 2020-12-05T17:41:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3cd387700 1 --2- 172.21.15.32:0/2775770429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3c8138630 0x7fb3c8150ad0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 rx=0x7fb3c80ade10 tx=0x7fb3c00091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3bdffb700 1 -- 172.21.15.32:0/2775770429 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3c000c950 con 0x7fb3c8138630 2020-12-05T17:41:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3bdffb700 1 -- 172.21.15.32:0/2775770429 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c000cab0 con 0x7fb3c8138630 2020-12-05T17:41:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3cdb88700 1 -- 172.21.15.32:0/2775770429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c8150630 con 0x7fb3c8138630 2020-12-05T17:41:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3bdffb700 1 -- 172.21.15.32:0/2775770429 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3c000ce10 con 0x7fb3c8138630 2020-12-05T17:41:28.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3cdb88700 1 -- 172.21.15.32:0/2775770429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c813e900 con 0x7fb3c8138630 2020-12-05T17:41:28.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3cdb88700 1 -- 172.21.15.32:0/2775770429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c8057050 con 0x7fb3c8138630 2020-12-05T17:41:28.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3bdffb700 1 -- 172.21.15.32:0/2775770429 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb3c0022070 con 0x7fb3c8138630 2020-12-05T17:41:28.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3bdffb700 1 --2- 172.21.15.32:0/2775770429 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3a8055440 0x7fb3a8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:28.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3bdffb700 1 -- 172.21.15.32:0/2775770429 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fb3c006e460 con 0x7fb3c8138630 2020-12-05T17:41:28.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3ccb86700 1 --2- 172.21.15.32:0/2775770429 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3a8055440 0x7fb3a8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:28.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.202+0000 7fb3ccb86700 1 --2- 172.21.15.32:0/2775770429 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3a8055440 0x7fb3a8057900 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fb3c400d000 tx=0x7fb3c4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:28.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.206+0000 7fb3bdffb700 1 -- 172.21.15.32:0/2775770429 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb3c0044840 con 0x7fb3c8138630 2020-12-05T17:41:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:28.362+0000 7fb3cdb88700 1 -- 172.21.15.32:0/2775770429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fb3c805aab0 con 0x7fb3c8138630 2020-12-05T17:41:29.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.014+0000 7fb3bdffb700 1 -- 172.21.15.32:0/2775770429 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v133) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb3c0016070 con 0x7fb3c8138630 2020-12-05T17:41:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.014+0000 7fb3cdb88700 1 -- 172.21.15.32:0/2775770429 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3a8055440 msgr2=0x7fb3a8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.014+0000 7fb3cdb88700 1 --2- 172.21.15.32:0/2775770429 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3a8055440 0x7fb3a8057900 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fb3c400d000 tx=0x7fb3c4009040).stop 2020-12-05T17:41:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.014+0000 7fb3cdb88700 1 -- 172.21.15.32:0/2775770429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3c8138630 msgr2=0x7fb3c8150ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.014+0000 7fb3cdb88700 1 --2- 172.21.15.32:0/2775770429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3c8138630 0x7fb3c8150ad0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 rx=0x7fb3c80ade10 tx=0x7fb3c00091a0).stop 2020-12-05T17:41:29.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.014+0000 7fb3cdb88700 1 -- 172.21.15.32:0/2775770429 shutdown_connections 2020-12-05T17:41:29.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.014+0000 7fb3cdb88700 1 --2- 172.21.15.32:0/2775770429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3c8138630 0x7fb3c8150ad0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.014+0000 7fb3cdb88700 1 --2- 172.21.15.32:0/2775770429 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb3a8055440 0x7fb3a8057900 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.014+0000 7fb3cdb88700 1 --2- 172.21.15.32:0/2775770429 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb3c8062900 0x7fb3c81255c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.014+0000 7fb3cdb88700 1 --2- 172.21.15.32:0/2775770429 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3c8063250 0x7fb3c814ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.014+0000 7fb3cdb88700 1 -- 172.21.15.32:0/2775770429 >> 172.21.15.32:0/2775770429 conn(0x7fb3c80b4f10 msgr2=0x7fb3c8124680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:29.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.014+0000 7fb3cdb88700 1 -- 172.21.15.32:0/2775770429 shutdown_connections 2020-12-05T17:41:29.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.014+0000 7fb3cdb88700 1 -- 172.21.15.32:0/2775770429 wait complete. 2020-12-05T17:41:29.024 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2020-12-05T17:41:29.037 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2020-12-05T17:41:29.038 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-12-05T17:41:29.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.142+0000 7f029293f700 1 Processor -- start 2020-12-05T17:41:29.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.142+0000 7f029293f700 1 -- start start 2020-12-05T17:41:29.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.142+0000 7f029293f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f028c0b1980 0x7f028c146580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:29.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.142+0000 7f029293f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f028c146a70 0x7f028c066070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:29.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029293f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f028c143000 con 0x7f028c05f6f0 2020-12-05T17:41:29.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029293f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f028c05fad0 con 0x7f028c0b1980 2020-12-05T17:41:29.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029293f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0b2f80 con 0x7f028c146a70 2020-12-05T17:41:29.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029193d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f028c05f6f0 0x7f028c1428f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43414/0 (socket says 172.21.15.32:43414) 2020-12-05T17:41:29.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029193d700 1 -- 172.21.15.32:0/2436454241 learned_addr learned my addr 172.21.15.32:0/2436454241 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:29.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029113c700 1 --2- 172.21.15.32:0/2436454241 >> v2:172.21.15.61:3300/0 conn(0x7f028c0b1980 0x7f028c146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:29.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029213e700 1 --2- 172.21.15.32:0/2436454241 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f028c146a70 0x7f028c066070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:29.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029093b700 1 -- 172.21.15.32:0/2436454241 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 179319418 0 0) 0x7f028c143000 con 0x7f028c05f6f0 2020-12-05T17:41:29.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029093b700 1 -- 172.21.15.32:0/2436454241 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0274003230 con 0x7f028c05f6f0 2020-12-05T17:41:29.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029213e700 1 -- 172.21.15.32:0/2436454241 >> v2:172.21.15.61:3300/0 conn(0x7f028c0b1980 msgr2=0x7f028c146580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:29.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029213e700 1 --2- 172.21.15.32:0/2436454241 >> v2:172.21.15.61:3300/0 conn(0x7f028c0b1980 0x7f028c146580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029213e700 1 -- 172.21.15.32:0/2436454241 >> v1:172.21.15.32:6789/0 conn(0x7f028c05f6f0 legacy=0x7f028c1428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:29.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029213e700 1 -- 172.21.15.32:0/2436454241 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028c145e40 con 0x7f028c146a70 2020-12-05T17:41:29.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029213e700 1 --2- 172.21.15.32:0/2436454241 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f028c146a70 0x7f028c066070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f02840032b0 tx=0x7f02840091a0).ready entity=mon.2 client_cookie=1c6fd3dd71bdf559 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:29.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029093b700 1 -- 172.21.15.32:0/2436454241 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f028400b940 con 0x7f028c146a70 2020-12-05T17:41:29.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029093b700 1 -- 172.21.15.32:0/2436454241 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028400baa0 con 0x7f028c146a70 2020-12-05T17:41:29.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029293f700 1 -- 172.21.15.32:0/2436454241 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f028c146a70 msgr2=0x7f028c066070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:29.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029293f700 1 --2- 172.21.15.32:0/2436454241 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f028c146a70 0x7f028c066070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f02840032b0 tx=0x7f02840091a0).stop 2020-12-05T17:41:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029293f700 1 -- 172.21.15.32:0/2436454241 shutdown_connections 2020-12-05T17:41:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029293f700 1 --2- 172.21.15.32:0/2436454241 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f028c146a70 0x7f028c066070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029293f700 1 --2- 172.21.15.32:0/2436454241 >> v2:172.21.15.61:3300/0 conn(0x7f028c0b1980 0x7f028c146580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029293f700 1 -- 172.21.15.32:0/2436454241 >> 172.21.15.32:0/2436454241 conn(0x7f028c0604c0 msgr2=0x7f028c05c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:29.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029293f700 1 -- 172.21.15.32:0/2436454241 shutdown_connections 2020-12-05T17:41:29.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.146+0000 7f029293f700 1 -- 172.21.15.32:0/2436454241 wait complete. 2020-12-05T17:41:29.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f029293f700 1 Processor -- start 2020-12-05T17:41:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f029293f700 1 -- start start 2020-12-05T17:41:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f029293f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f028c05f6f0 0x7f028c1162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f029293f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f028c0b1980 0x7f028c1548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f029293f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f028c146a70 0x7f028c0b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f029293f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c113650 con 0x7f028c146a70 2020-12-05T17:41:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f029293f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0afae0 con 0x7f028c0b1980 2020-12-05T17:41:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f029293f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0afd80 con 0x7f028c05f6f0 2020-12-05T17:41:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f029213e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f028c146a70 0x7f028c0b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f029213e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f028c146a70 0x7f028c0b5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40776/0 (socket says 172.21.15.32:40776) 2020-12-05T17:41:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f029213e700 1 -- 172.21.15.32:0/2291848240 learned_addr learned my addr 172.21.15.32:0/2291848240 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f029113c700 1 --2- 172.21.15.32:0/2291848240 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f028c0b1980 0x7f028c1548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f029193d700 1 --2- 172.21.15.32:0/2291848240 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f028c05f6f0 0x7f028c1162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f029213e700 1 -- 172.21.15.32:0/2291848240 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f028c05f6f0 msgr2=0x7f028c1162e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:29.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f029213e700 1 --2- 172.21.15.32:0/2291848240 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f028c05f6f0 0x7f028c1162e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f029213e700 1 -- 172.21.15.32:0/2291848240 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f028c0b1980 msgr2=0x7f028c1548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:29.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f029213e700 1 --2- 172.21.15.32:0/2291848240 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f028c0b1980 0x7f028c1548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f029213e700 1 -- 172.21.15.32:0/2291848240 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028c0b6500 con 0x7f028c146a70 2020-12-05T17:41:29.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f029213e700 1 --2- 172.21.15.32:0/2291848240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f028c146a70 0x7f028c0b5b60 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x7f02840032b0 tx=0x7f0284016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f027affd700 1 -- 172.21.15.32:0/2291848240 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0284011df0 con 0x7f028c146a70 2020-12-05T17:41:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f027affd700 1 -- 172.21.15.32:0/2291848240 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02840203f0 con 0x7f028c146a70 2020-12-05T17:41:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f027affd700 1 -- 172.21.15.32:0/2291848240 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0284020750 con 0x7f028c146a70 2020-12-05T17:41:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f029293f700 1 -- 172.21.15.32:0/2291848240 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f028c0af6b0 con 0x7f028c146a70 2020-12-05T17:41:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.150+0000 7f029293f700 1 -- 172.21.15.32:0/2291848240 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028c113ca0 con 0x7f028c146a70 2020-12-05T17:41:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.154+0000 7f029293f700 1 -- 172.21.15.32:0/2291848240 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f028c1137c0 con 0x7f028c146a70 2020-12-05T17:41:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.154+0000 7f027affd700 1 -- 172.21.15.32:0/2291848240 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f028401e070 con 0x7f028c146a70 2020-12-05T17:41:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.154+0000 7f027affd700 1 --2- 172.21.15.32:0/2291848240 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f026c055440 0x7f026c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.154+0000 7f029193d700 1 --2- 172.21.15.32:0/2291848240 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f026c055440 0x7f026c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:29.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.154+0000 7f027affd700 1 -- 172.21.15.32:0/2291848240 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f028406eaa0 con 0x7f028c146a70 2020-12-05T17:41:29.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.154+0000 7f029193d700 1 --2- 172.21.15.32:0/2291848240 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f026c055440 0x7f026c057900 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f028800d000 tx=0x7f0288009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:29.167 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.162+0000 7f027affd700 1 -- 172.21.15.32:0/2291848240 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f028c1137c0 con 0x7f028c146a70 2020-12-05T17:41:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.350+0000 7f029293f700 1 -- 172.21.15.32:0/2291848240 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f028c0aff40 con 0x7f028c146a70 2020-12-05T17:41:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.350+0000 7f027affd700 1 -- 172.21.15.32:0/2291848240 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v40) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f028401b070 con 0x7f028c146a70 2020-12-05T17:41:29.356 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:29.357 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:41:29.357 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2020-12-05T17:41:29.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:29.357 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2020-12-05T17:41:29.358 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfpb/aExAAdMKW4yIgC2zhkzyW6l8xug==", 2020-12-05T17:41:29.358 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:29.358 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:41:29.358 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:41:29.359 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:41:29.359 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:29.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:29.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:29.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:29.360 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2020-12-05T17:41:29.360 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfBKTYGRAAaPkezwmTiAvkUfTuDGDNZA==", 2020-12-05T17:41:29.360 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:29.360 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:41:29.361 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:41:29.361 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:41:29.361 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:29.361 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:29.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2020-12-05T17:41:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHw8tflo5EKhAAwV9D64fAoY3uvXCm+Aksag==", 2020-12-05T17:41:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:41:29.363 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:41:29.363 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:29.363 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:29.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:29.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:29.364 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2020-12-05T17:41:29.364 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDKw8tfTjEiERAAkW6mkgApoXwyF49kPjZBGg==", 2020-12-05T17:41:29.364 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:29.364 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:41:29.364 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:41:29.365 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:29.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:29.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:29.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:29.366 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2020-12-05T17:41:29.366 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDMw8tfLEzBMxAAiJ2NifsOLEkt1e3vmhwSTA==", 2020-12-05T17:41:29.366 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:29.366 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:41:29.366 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:41:29.367 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:29.367 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:29.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:29.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:29.367 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2020-12-05T17:41:29.368 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPw8tfV+vnGxAAQHvXJRW5X9Vhe/eYJBJ/uQ==", 2020-12-05T17:41:29.368 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:29.368 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:41:29.368 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:41:29.368 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:29.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:29.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:29.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:29.369 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2020-12-05T17:41:29.370 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUw8tfN8wMJxAApc5LY9HGDeIV/o0eil/g4A==", 2020-12-05T17:41:29.370 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:29.370 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:41:29.370 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:41:29.370 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:29.371 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:29.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:29.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:29.371 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2020-12-05T17:41:29.371 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXw8tfvZ5dDxAAtAfftExkljlBvn7ZSoD/Og==", 2020-12-05T17:41:29.372 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:29.372 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:41:29.372 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:41:29.372 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:29.372 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:29.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:29.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:29.373 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2020-12-05T17:41:29.373 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDZw8tfhXFzMhAAZXNwNGfKv360Tq4N/DX5Mw==", 2020-12-05T17:41:29.373 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:29.374 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:41:29.374 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:41:29.374 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:29.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2020-12-05T17:41:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDcw8tfROfSGhAA/Y2Y1b4k5zQju0MM7LASeQ==", 2020-12-05T17:41:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:29.376 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:41:29.376 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:41:29.376 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:29.376 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:29.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2020-12-05T17:41:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfWDp/HxAAmvLPj7C3WDgj0OC2o7lBJQ==", 2020-12-05T17:41:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:41:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:41:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:41:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2020-12-05T17:41:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfugnmIRAAW6+/QGMLkHw+Z97RHpV4kw==", 2020-12-05T17:41:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:41:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:41:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:41:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2020-12-05T17:41:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfojyqJBAAyta+dSy2yGnC1kKe3YOAYw==", 2020-12-05T17:41:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:41:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:41:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:41:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2020-12-05T17:41:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDFw8tfES0PLBAAA1cUV44snAUDF2TyAmXImw==", 2020-12-05T17:41:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:41:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:41:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:41:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2020-12-05T17:41:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfDr8mERAAS7bpAQXgW78W8GDq1v1zaA==", 2020-12-05T17:41:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:41:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:41:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2020-12-05T17:41:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tf3HmcDhAATNKHhH9zgasedaNEzgG0OQ==", 2020-12-05T17:41:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:41:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:41:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:29.388 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:41:29.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.354+0000 7f029293f700 1 -- 172.21.15.32:0/2291848240 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f026c055440 msgr2=0x7f026c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:29.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.354+0000 7f029293f700 1 --2- 172.21.15.32:0/2291848240 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f026c055440 0x7f026c057900 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f028800d000 tx=0x7f0288009040).stop 2020-12-05T17:41:29.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.354+0000 7f029293f700 1 -- 172.21.15.32:0/2291848240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f028c146a70 msgr2=0x7f028c0b5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:29.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.354+0000 7f029293f700 1 --2- 172.21.15.32:0/2291848240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f028c146a70 0x7f028c0b5b60 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x7f02840032b0 tx=0x7f0284016000).stop 2020-12-05T17:41:29.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.354+0000 7f029293f700 1 -- 172.21.15.32:0/2291848240 shutdown_connections 2020-12-05T17:41:29.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.354+0000 7f029293f700 1 --2- 172.21.15.32:0/2291848240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f028c146a70 0x7f028c0b5b60 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.354+0000 7f029293f700 1 --2- 172.21.15.32:0/2291848240 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f026c055440 0x7f026c057900 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.354+0000 7f029293f700 1 --2- 172.21.15.32:0/2291848240 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f028c05f6f0 0x7f028c1162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.354+0000 7f029293f700 1 --2- 172.21.15.32:0/2291848240 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f028c0b1980 0x7f028c1548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.354+0000 7f029293f700 1 -- 172.21.15.32:0/2291848240 >> 172.21.15.32:0/2291848240 conn(0x7f028c0604c0 msgr2=0x7f028c064fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:29.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.354+0000 7f029293f700 1 -- 172.21.15.32:0/2291848240 shutdown_connections 2020-12-05T17:41:29.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.354+0000 7f029293f700 1 -- 172.21.15.32:0/2291848240 wait complete. 2020-12-05T17:41:29.393 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:41:29.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d54979700 1 Processor -- start 2020-12-05T17:41:29.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d54979700 1 -- start start 2020-12-05T17:41:29.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d54979700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d50062400 0x7f0d5005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:29.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d54979700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0d5005feb0 0x7f0d500b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:29.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d54979700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d50142450 con 0x7f0d5005f3b0 2020-12-05T17:41:29.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d54979700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d500b42f0 con 0x7f0d50062400 2020-12-05T17:41:29.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d54979700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d50061780 con 0x7f0d5005feb0 2020-12-05T17:41:29.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d46ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d50062400 0x7f0d5005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:29.509 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d46ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d50062400 0x7f0d5005f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40784/0 (socket says 172.21.15.32:40784) 2020-12-05T17:41:29.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d4ffff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0d5005feb0 0x7f0d500b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:29.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d46ffd700 1 -- 172.21.15.32:0/639756660 learned_addr learned my addr 172.21.15.32:0/639756660 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:29.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d46ffd700 1 -- 172.21.15.32:0/639756660 >> v2:172.21.15.61:3300/0 conn(0x7f0d5005feb0 msgr2=0x7f0d500b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:29.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d46ffd700 1 --2- 172.21.15.32:0/639756660 >> v2:172.21.15.61:3300/0 conn(0x7f0d5005feb0 0x7f0d500b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d46ffd700 1 -- 172.21.15.32:0/639756660 >> v1:172.21.15.61:6789/0 conn(0x7f0d5005f3b0 legacy=0x7f0d500b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:29.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d46ffd700 1 -- 172.21.15.32:0/639756660 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d50061a20 con 0x7f0d50062400 2020-12-05T17:41:29.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d46ffd700 1 --2- 172.21.15.32:0/639756660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d50062400 0x7f0d5005f790 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 rx=0x7f0d38002f30 tx=0x7f0d38006e80).ready entity=mon.0 client_cookie=60fc3e0a9caafa43 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:29.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d4effd700 1 -- 172.21.15.32:0/639756660 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d380097f0 con 0x7f0d50062400 2020-12-05T17:41:29.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d4effd700 1 -- 172.21.15.32:0/639756660 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d38009950 con 0x7f0d50062400 2020-12-05T17:41:29.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d4effd700 1 -- 172.21.15.32:0/639756660 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d38009cb0 con 0x7f0d50062400 2020-12-05T17:41:29.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d54979700 1 -- 172.21.15.32:0/639756660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d50062400 msgr2=0x7f0d5005f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:29.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d54979700 1 --2- 172.21.15.32:0/639756660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d50062400 0x7f0d5005f790 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 rx=0x7f0d38002f30 tx=0x7f0d38006e80).stop 2020-12-05T17:41:29.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d54979700 1 -- 172.21.15.32:0/639756660 shutdown_connections 2020-12-05T17:41:29.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d54979700 1 --2- 172.21.15.32:0/639756660 >> v2:172.21.15.61:3300/0 conn(0x7f0d5005feb0 0x7f0d500b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d54979700 1 --2- 172.21.15.32:0/639756660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d50062400 0x7f0d5005f790 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d54979700 1 -- 172.21.15.32:0/639756660 >> 172.21.15.32:0/639756660 conn(0x7f0d500b8ee0 msgr2=0x7f0d500b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:29.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d54979700 1 -- 172.21.15.32:0/639756660 shutdown_connections 2020-12-05T17:41:29.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.502+0000 7f0d54979700 1 -- 172.21.15.32:0/639756660 wait complete. 2020-12-05T17:41:29.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d54979700 1 Processor -- start 2020-12-05T17:41:29.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d54979700 1 -- start start 2020-12-05T17:41:29.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d54979700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0d5005f3b0 0x7f0d50145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:29.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d54979700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d5005feb0 0x7f0d50146c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:29.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d54979700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0d50062400 0x7f0d50147130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:29.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d54979700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d50147fa0 con 0x7f0d5005feb0 2020-12-05T17:41:29.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d54979700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d500639e0 con 0x7f0d5005f3b0 2020-12-05T17:41:29.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d54979700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d50063000 con 0x7f0d50062400 2020-12-05T17:41:29.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d46ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d5005feb0 0x7f0d50146c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:29.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d46ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d5005feb0 0x7f0d50146c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40786/0 (socket says 172.21.15.32:40786) 2020-12-05T17:41:29.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d46ffd700 1 -- 172.21.15.32:0/1346995838 learned_addr learned my addr 172.21.15.32:0/1346995838 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:29.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d4ffff700 1 --2- 172.21.15.32:0/1346995838 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0d50062400 0x7f0d50147130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:29.519 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d4f7fe700 1 --2- 172.21.15.32:0/1346995838 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0d5005f3b0 0x7f0d50145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:29.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d46ffd700 1 -- 172.21.15.32:0/1346995838 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0d50062400 msgr2=0x7f0d50147130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:29.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d46ffd700 1 --2- 172.21.15.32:0/1346995838 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0d50062400 0x7f0d50147130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d46ffd700 1 -- 172.21.15.32:0/1346995838 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0d5005f3b0 msgr2=0x7f0d50145750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:29.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d46ffd700 1 --2- 172.21.15.32:0/1346995838 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0d5005f3b0 0x7f0d50145750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.520 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d46ffd700 1 -- 172.21.15.32:0/1346995838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d500632a0 con 0x7f0d5005feb0 2020-12-05T17:41:29.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d46ffd700 1 --2- 172.21.15.32:0/1346995838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d5005feb0 0x7f0d50146c40 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 rx=0x7f0d38002f30 tx=0x7f0d38015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:29.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d4d7fa700 1 -- 172.21.15.32:0/1346995838 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d38011cc0 con 0x7f0d5005feb0 2020-12-05T17:41:29.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d4d7fa700 1 -- 172.21.15.32:0/1346995838 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d38011e20 con 0x7f0d5005feb0 2020-12-05T17:41:29.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d54979700 1 -- 172.21.15.32:0/1346995838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d50063540 con 0x7f0d5005feb0 2020-12-05T17:41:29.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d4d7fa700 1 -- 172.21.15.32:0/1346995838 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d3801e650 con 0x7f0d5005feb0 2020-12-05T17:41:29.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.506+0000 7f0d54979700 1 -- 172.21.15.32:0/1346995838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d500af9b0 con 0x7f0d5005feb0 2020-12-05T17:41:29.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.510+0000 7f0d54979700 1 -- 172.21.15.32:0/1346995838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d500636d0 con 0x7f0d5005feb0 2020-12-05T17:41:29.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.510+0000 7f0d4d7fa700 1 -- 172.21.15.32:0/1346995838 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0d3801a040 con 0x7f0d5005feb0 2020-12-05T17:41:29.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.510+0000 7f0d4d7fa700 1 --2- 172.21.15.32:0/1346995838 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0d34055440 0x7f0d34057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:29.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.510+0000 7f0d4f7fe700 1 --2- 172.21.15.32:0/1346995838 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0d34055440 0x7f0d34057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:29.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.510+0000 7f0d4d7fa700 1 -- 172.21.15.32:0/1346995838 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f0d3806ee60 con 0x7f0d5005feb0 2020-12-05T17:41:29.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.510+0000 7f0d4f7fe700 1 --2- 172.21.15.32:0/1346995838 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0d34055440 0x7f0d34057900 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f0d4000b000 tx=0x7f0d40009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:29.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.518+0000 7f0d4d7fa700 1 -- 172.21.15.32:0/1346995838 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0d380452a0 con 0x7f0d5005feb0 2020-12-05T17:41:29.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.658+0000 7f0d54979700 1 -- 172.21.15.32:0/1346995838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d50057050 con 0x7f0d5005feb0 2020-12-05T17:41:29.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.662+0000 7f0d4d7fa700 1 -- 172.21.15.32:0/1346995838 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+13424 (secure 0 0 0) 0x7f0d38023070 con 0x7f0d5005feb0 2020-12-05T17:41:29.666 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:29.667 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":133,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:41:28.999876+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:41:29.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.662+0000 7f0d54979700 1 -- 172.21.15.32:0/1346995838 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0d34055440 msgr2=0x7f0d34057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:29.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.662+0000 7f0d54979700 1 --2- 172.21.15.32:0/1346995838 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0d34055440 0x7f0d34057900 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f0d4000b000 tx=0x7f0d40009040).stop 2020-12-05T17:41:29.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.662+0000 7f0d54979700 1 -- 172.21.15.32:0/1346995838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d5005feb0 msgr2=0x7f0d50146c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:29.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.662+0000 7f0d54979700 1 --2- 172.21.15.32:0/1346995838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d5005feb0 0x7f0d50146c40 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 rx=0x7f0d38002f30 tx=0x7f0d38015000).stop 2020-12-05T17:41:29.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.662+0000 7f0d54979700 1 -- 172.21.15.32:0/1346995838 shutdown_connections 2020-12-05T17:41:29.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.662+0000 7f0d54979700 1 --2- 172.21.15.32:0/1346995838 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0d34055440 0x7f0d34057900 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.662+0000 7f0d54979700 1 --2- 172.21.15.32:0/1346995838 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0d50062400 0x7f0d50147130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.662+0000 7f0d54979700 1 --2- 172.21.15.32:0/1346995838 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0d5005f3b0 0x7f0d50145750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.662+0000 7f0d54979700 1 --2- 172.21.15.32:0/1346995838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d5005feb0 0x7f0d50146c40 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.662+0000 7f0d54979700 1 -- 172.21.15.32:0/1346995838 >> 172.21.15.32:0/1346995838 conn(0x7f0d500b8ee0 msgr2=0x7f0d50140760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:29.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.662+0000 7f0d54979700 1 -- 172.21.15.32:0/1346995838 shutdown_connections 2020-12-05T17:41:29.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.662+0000 7f0d54979700 1 -- 172.21.15.32:0/1346995838 wait complete. 2020-12-05T17:41:29.680 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-12-05T17:41:29.681 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:41:29.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.794+0000 7f714e314700 1 Processor -- start 2020-12-05T17:41:29.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.794+0000 7f714e314700 1 -- start start 2020-12-05T17:41:29.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f714e314700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71480af4b0 0x7f7148146510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:29.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f714e314700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7148061780 0x7f71480afe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:29.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f714e314700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71480b23e0 con 0x7f71480b2000 2020-12-05T17:41:29.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f714e314700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7148146a00 con 0x7f7148061780 2020-12-05T17:41:29.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f714e314700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71480ae7b0 con 0x7f71480af4b0 2020-12-05T17:41:29.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f714db13700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7148061780 0x7f71480afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:29.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f714d312700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f71480b2000 0x7f7148145e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43442/0 (socket says 172.21.15.32:43442) 2020-12-05T17:41:29.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f714d312700 1 -- 172.21.15.32:0/1299147092 learned_addr learned my addr 172.21.15.32:0/1299147092 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:29.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f714cb11700 1 --2- 172.21.15.32:0/1299147092 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71480af4b0 0x7f7148146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:29.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f714db13700 1 -- 172.21.15.32:0/1299147092 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71480af4b0 msgr2=0x7f7148146510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:29.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f714db13700 1 --2- 172.21.15.32:0/1299147092 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71480af4b0 0x7f7148146510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f714db13700 1 -- 172.21.15.32:0/1299147092 >> v1:172.21.15.32:6789/0 conn(0x7f71480b2000 legacy=0x7f7148145e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:29.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f714db13700 1 -- 172.21.15.32:0/1299147092 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71480aea50 con 0x7f7148061780 2020-12-05T17:41:29.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f714db13700 1 --2- 172.21.15.32:0/1299147092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7148061780 0x7f71480afe20 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0x7f7140002f30 tx=0x7f7140006dc0).ready entity=mon.0 client_cookie=c904d2bf134006e5 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:29.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f713ffff700 1 -- 172.21.15.32:0/1299147092 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71400097f0 con 0x7f7148061780 2020-12-05T17:41:29.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f713ffff700 1 -- 172.21.15.32:0/1299147092 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7140009950 con 0x7f7148061780 2020-12-05T17:41:29.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f713ffff700 1 -- 172.21.15.32:0/1299147092 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7140009cb0 con 0x7f7148061780 2020-12-05T17:41:29.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f714e314700 1 -- 172.21.15.32:0/1299147092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7148061780 msgr2=0x7f71480afe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:29.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f714e314700 1 --2- 172.21.15.32:0/1299147092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7148061780 0x7f71480afe20 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0x7f7140002f30 tx=0x7f7140006dc0).stop 2020-12-05T17:41:29.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f714e314700 1 -- 172.21.15.32:0/1299147092 shutdown_connections 2020-12-05T17:41:29.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f714e314700 1 --2- 172.21.15.32:0/1299147092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7148061780 0x7f71480afe20 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f714e314700 1 --2- 172.21.15.32:0/1299147092 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71480af4b0 0x7f7148146510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f714e314700 1 -- 172.21.15.32:0/1299147092 >> 172.21.15.32:0/1299147092 conn(0x7f71480b5620 msgr2=0x7f71480b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:29.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f714e314700 1 -- 172.21.15.32:0/1299147092 shutdown_connections 2020-12-05T17:41:29.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f714e314700 1 -- 172.21.15.32:0/1299147092 wait complete. 2020-12-05T17:41:29.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.798+0000 7f714e314700 1 Processor -- start 2020-12-05T17:41:29.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f714e314700 1 -- start start 2020-12-05T17:41:29.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f714e314700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7148061780 0x7f7148144550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:29.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f714e314700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71480af4b0 0x7f7148066580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:29.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f714e314700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71480b2000 0x7f7148063a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f714e314700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7148066a70 con 0x7f71480b2000 2020-12-05T17:41:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f714e314700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7148066cc0 con 0x7f7148061780 2020-12-05T17:41:29.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f714e314700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71480635c0 con 0x7f71480af4b0 2020-12-05T17:41:29.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f714db13700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71480b2000 0x7f7148063a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:29.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f714db13700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71480b2000 0x7f7148063a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40804/0 (socket says 172.21.15.32:40804) 2020-12-05T17:41:29.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f714db13700 1 -- 172.21.15.32:0/3736972058 learned_addr learned my addr 172.21.15.32:0/3736972058 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f714d312700 1 --2- 172.21.15.32:0/3736972058 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7148061780 0x7f7148144550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f714cb11700 1 --2- 172.21.15.32:0/3736972058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71480af4b0 0x7f7148066580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f714db13700 1 -- 172.21.15.32:0/3736972058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71480af4b0 msgr2=0x7f7148066580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f714db13700 1 --2- 172.21.15.32:0/3736972058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71480af4b0 0x7f7148066580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f714db13700 1 -- 172.21.15.32:0/3736972058 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7148061780 msgr2=0x7f7148144550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f714db13700 1 --2- 172.21.15.32:0/3736972058 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7148061780 0x7f7148144550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f714db13700 1 -- 172.21.15.32:0/3736972058 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7148063f50 con 0x7f71480b2000 2020-12-05T17:41:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f714db13700 1 --2- 172.21.15.32:0/3736972058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71480b2000 0x7f7148063a60 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 rx=0x7f7140002f30 tx=0x7f7140015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f713e7fc700 1 -- 172.21.15.32:0/3736972058 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7140011bd0 con 0x7f71480b2000 2020-12-05T17:41:29.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f713e7fc700 1 -- 172.21.15.32:0/3736972058 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7140011d30 con 0x7f71480b2000 2020-12-05T17:41:29.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f713e7fc700 1 -- 172.21.15.32:0/3736972058 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7140011ea0 con 0x7f71480b2000 2020-12-05T17:41:29.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f714e314700 1 -- 172.21.15.32:0/3736972058 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f714814a5b0 con 0x7f71480b2000 2020-12-05T17:41:29.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f714e314700 1 -- 172.21.15.32:0/3736972058 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7148065450 con 0x7f71480b2000 2020-12-05T17:41:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.802+0000 7f714e314700 1 -- 172.21.15.32:0/3736972058 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7148063780 con 0x7f71480b2000 2020-12-05T17:41:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.806+0000 7f713e7fc700 1 -- 172.21.15.32:0/3736972058 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f714000f070 con 0x7f71480b2000 2020-12-05T17:41:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.806+0000 7f713e7fc700 1 --2- 172.21.15.32:0/3736972058 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71280554e0 0x7f71280579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.806+0000 7f713e7fc700 1 -- 172.21.15.32:0/3736972058 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f714006f100 con 0x7f71480b2000 2020-12-05T17:41:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.806+0000 7f714d312700 1 --2- 172.21.15.32:0/3736972058 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71280554e0 0x7f71280579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.806+0000 7f714d312700 1 --2- 172.21.15.32:0/3736972058 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71280554e0 0x7f71280579a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f713800d000 tx=0x7f7138000b80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.810+0000 7f713e7fc700 1 -- 172.21.15.32:0/3736972058 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f71400454d0 con 0x7f71480b2000 2020-12-05T17:41:29.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.954+0000 7f714e314700 1 -- 172.21.15.32:0/3736972058 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7148066e80 con 0x7f71480b2000 2020-12-05T17:41:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.954+0000 7f713e7fc700 1 -- 172.21.15.32:0/3736972058 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v133) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7140022070 con 0x7f71480b2000 2020-12-05T17:41:29.959 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:41:29.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.954+0000 7f714e314700 1 -- 172.21.15.32:0/3736972058 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71280554e0 msgr2=0x7f71280579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:29.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.954+0000 7f714e314700 1 --2- 172.21.15.32:0/3736972058 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71280554e0 0x7f71280579a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f713800d000 tx=0x7f7138000b80).stop 2020-12-05T17:41:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.954+0000 7f714e314700 1 -- 172.21.15.32:0/3736972058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71480b2000 msgr2=0x7f7148063a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.954+0000 7f714e314700 1 --2- 172.21.15.32:0/3736972058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71480b2000 0x7f7148063a60 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 rx=0x7f7140002f30 tx=0x7f7140015000).stop 2020-12-05T17:41:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.954+0000 7f714e314700 1 -- 172.21.15.32:0/3736972058 shutdown_connections 2020-12-05T17:41:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.954+0000 7f714e314700 1 --2- 172.21.15.32:0/3736972058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71480b2000 0x7f7148063a60 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.954+0000 7f714e314700 1 --2- 172.21.15.32:0/3736972058 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7148061780 0x7f7148144550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.954+0000 7f714e314700 1 --2- 172.21.15.32:0/3736972058 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f71280554e0 0x7f71280579a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.954+0000 7f714e314700 1 --2- 172.21.15.32:0/3736972058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71480af4b0 0x7f7148066580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:29.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.954+0000 7f714e314700 1 -- 172.21.15.32:0/3736972058 >> 172.21.15.32:0/3736972058 conn(0x7f71480b5620 msgr2=0x7f71480b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:29.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.958+0000 7f714e314700 1 -- 172.21.15.32:0/3736972058 shutdown_connections 2020-12-05T17:41:29.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:29.958+0000 7f714e314700 1 -- 172.21.15.32:0/3736972058 wait complete. 2020-12-05T17:41:29.973 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:41:30.093 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.086+0000 7f687cde6700 1 Processor -- start 2020-12-05T17:41:30.093 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.086+0000 7f687cde6700 1 -- start start 2020-12-05T17:41:30.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f687cde6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f687813ed60 0x7f68780af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:30.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f687cde6700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f68780ad250 0x7f6878142600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:30.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f687cde6700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68780b24e0 con 0x7f68780b2100 2020-12-05T17:41:30.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f687cde6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f687813fd10 con 0x7f687813ed60 2020-12-05T17:41:30.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f687cde6700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68780b0780 con 0x7f68780ad250 2020-12-05T17:41:30.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f6876ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f687813ed60 0x7f68780af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:30.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f6876ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f687813ed60 0x7f68780af360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40810/0 (socket says 172.21.15.32:40810) 2020-12-05T17:41:30.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f6876ffd700 1 -- 172.21.15.32:0/1273694219 learned_addr learned my addr 172.21.15.32:0/1273694219 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:30.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f6877fff700 1 --2- 172.21.15.32:0/1273694219 >> v2:172.21.15.61:3300/0 conn(0x7f68780ad250 0x7f6878142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:30.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f6876ffd700 1 -- 172.21.15.32:0/1273694219 >> v2:172.21.15.61:3300/0 conn(0x7f68780ad250 msgr2=0x7f6878142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:30.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f6876ffd700 1 --2- 172.21.15.32:0/1273694219 >> v2:172.21.15.61:3300/0 conn(0x7f68780ad250 0x7f6878142600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f6876ffd700 1 -- 172.21.15.32:0/1273694219 >> v1:172.21.15.61:6789/0 conn(0x7f68780b2100 legacy=0x7f6878138820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:30.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f6876ffd700 1 -- 172.21.15.32:0/1273694219 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68780b0240 con 0x7f687813ed60 2020-12-05T17:41:30.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f6877fff700 1 --2- 172.21.15.32:0/1273694219 >> v2:172.21.15.61:3300/0 conn(0x7f68780ad250 0x7f6878142600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:30.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f6876ffd700 1 --2- 172.21.15.32:0/1273694219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f687813ed60 0x7f68780af360 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7f6860002f30 tx=0x7f6860006e80).ready entity=mon.0 client_cookie=4ab601efbb9b70f6 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:30.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f68767fc700 1 -- 172.21.15.32:0/1273694219 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68600097f0 con 0x7f687813ed60 2020-12-05T17:41:30.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f68767fc700 1 -- 172.21.15.32:0/1273694219 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6860009950 con 0x7f687813ed60 2020-12-05T17:41:30.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f68767fc700 1 -- 172.21.15.32:0/1273694219 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6860009cb0 con 0x7f687813ed60 2020-12-05T17:41:30.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f687cde6700 1 -- 172.21.15.32:0/1273694219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f687813ed60 msgr2=0x7f68780af360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:30.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f687cde6700 1 --2- 172.21.15.32:0/1273694219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f687813ed60 0x7f68780af360 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7f6860002f30 tx=0x7f6860006e80).stop 2020-12-05T17:41:30.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f687cde6700 1 -- 172.21.15.32:0/1273694219 shutdown_connections 2020-12-05T17:41:30.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f687cde6700 1 --2- 172.21.15.32:0/1273694219 >> v2:172.21.15.61:3300/0 conn(0x7f68780ad250 0x7f6878142600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f687cde6700 1 --2- 172.21.15.32:0/1273694219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f687813ed60 0x7f68780af360 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f687cde6700 1 -- 172.21.15.32:0/1273694219 >> 172.21.15.32:0/1273694219 conn(0x7f68780620b0 msgr2=0x7f6878062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:30.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f687cde6700 1 -- 172.21.15.32:0/1273694219 shutdown_connections 2020-12-05T17:41:30.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.090+0000 7f687cde6700 1 -- 172.21.15.32:0/1273694219 wait complete. 2020-12-05T17:41:30.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f687cde6700 1 Processor -- start 2020-12-05T17:41:30.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f687cde6700 1 -- start start 2020-12-05T17:41:30.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f687cde6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68780ad250 0x7f68780658c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:30.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f687cde6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68780b2100 0x7f6878133d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:30.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f687cde6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6878134210 0x7f6878154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:30.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f687cde6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6878136570 con 0x7f68780b2100 2020-12-05T17:41:30.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f687cde6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6878136760 con 0x7f68780ad250 2020-12-05T17:41:30.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f687cde6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687813c120 con 0x7f6878134210 2020-12-05T17:41:30.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f6876ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68780b2100 0x7f6878133d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:30.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f6876ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68780b2100 0x7f6878133d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40816/0 (socket says 172.21.15.32:40816) 2020-12-05T17:41:30.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f6876ffd700 1 -- 172.21.15.32:0/941685269 learned_addr learned my addr 172.21.15.32:0/941685269 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:30.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f68777fe700 1 --2- 172.21.15.32:0/941685269 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68780ad250 0x7f68780658c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:30.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f6877fff700 1 --2- 172.21.15.32:0/941685269 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6878134210 0x7f6878154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:30.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f6876ffd700 1 -- 172.21.15.32:0/941685269 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6878134210 msgr2=0x7f6878154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:30.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f6876ffd700 1 --2- 172.21.15.32:0/941685269 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6878134210 0x7f6878154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f6876ffd700 1 -- 172.21.15.32:0/941685269 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68780ad250 msgr2=0x7f68780658c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:30.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f6876ffd700 1 --2- 172.21.15.32:0/941685269 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68780ad250 0x7f68780658c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f6876ffd700 1 -- 172.21.15.32:0/941685269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687813c2e0 con 0x7f68780b2100 2020-12-05T17:41:30.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f6877fff700 1 --2- 172.21.15.32:0/941685269 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6878134210 0x7f6878154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:30.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f6876ffd700 1 --2- 172.21.15.32:0/941685269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68780b2100 0x7f6878133d20 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7f6860002f30 tx=0x7f6860015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:30.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f6874ff9700 1 -- 172.21.15.32:0/941685269 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6860011c80 con 0x7f68780b2100 2020-12-05T17:41:30.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f6874ff9700 1 -- 172.21.15.32:0/941685269 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6860011de0 con 0x7f68780b2100 2020-12-05T17:41:30.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f687cde6700 1 -- 172.21.15.32:0/941685269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6878065db0 con 0x7f68780b2100 2020-12-05T17:41:30.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f6874ff9700 1 -- 172.21.15.32:0/941685269 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f686001e650 con 0x7f68780b2100 2020-12-05T17:41:30.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.094+0000 7f687cde6700 1 -- 172.21.15.32:0/941685269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68780631e0 con 0x7f68780b2100 2020-12-05T17:41:30.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.098+0000 7f687cde6700 1 -- 172.21.15.32:0/941685269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6878057050 con 0x7f68780b2100 2020-12-05T17:41:30.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.098+0000 7f6874ff9700 1 -- 172.21.15.32:0/941685269 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6860025030 con 0x7f68780b2100 2020-12-05T17:41:30.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.098+0000 7f6874ff9700 1 --2- 172.21.15.32:0/941685269 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6850055440 0x7f6850057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:30.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.098+0000 7f6874ff9700 1 -- 172.21.15.32:0/941685269 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f686006f0f0 con 0x7f68780b2100 2020-12-05T17:41:30.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.098+0000 7f68777fe700 1 --2- 172.21.15.32:0/941685269 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6850055440 0x7f6850057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:30.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.098+0000 7f68777fe700 1 --2- 172.21.15.32:0/941685269 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6850055440 0x7f6850057900 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f686800d000 tx=0x7f6868007ee0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:30.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.106+0000 7f6874ff9700 1 -- 172.21.15.32:0/941685269 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f68600455d0 con 0x7f68780b2100 2020-12-05T17:41:30.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.246+0000 7f687cde6700 1 -- 172.21.15.32:0/941685269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6878060c10 con 0x7f68780b2100 2020-12-05T17:41:30.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.246+0000 7f6874ff9700 1 -- 172.21.15.32:0/941685269 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v133) v1 ==== 92+0+10 (secure 0 0 0) 0x7f68600230a0 con 0x7f68780b2100 2020-12-05T17:41:30.253 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:41:30.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.250+0000 7f687cde6700 1 -- 172.21.15.32:0/941685269 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6850055440 msgr2=0x7f6850057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:30.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.250+0000 7f687cde6700 1 --2- 172.21.15.32:0/941685269 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6850055440 0x7f6850057900 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f686800d000 tx=0x7f6868007ee0).stop 2020-12-05T17:41:30.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.250+0000 7f687cde6700 1 -- 172.21.15.32:0/941685269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68780b2100 msgr2=0x7f6878133d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:30.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.250+0000 7f687cde6700 1 --2- 172.21.15.32:0/941685269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68780b2100 0x7f6878133d20 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7f6860002f30 tx=0x7f6860015000).stop 2020-12-05T17:41:30.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.250+0000 7f687cde6700 1 -- 172.21.15.32:0/941685269 shutdown_connections 2020-12-05T17:41:30.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.250+0000 7f687cde6700 1 --2- 172.21.15.32:0/941685269 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6850055440 0x7f6850057900 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.250+0000 7f687cde6700 1 --2- 172.21.15.32:0/941685269 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6878134210 0x7f6878154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.250+0000 7f687cde6700 1 --2- 172.21.15.32:0/941685269 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68780ad250 0x7f68780658c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.250+0000 7f687cde6700 1 --2- 172.21.15.32:0/941685269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68780b2100 0x7f6878133d20 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.250+0000 7f687cde6700 1 -- 172.21.15.32:0/941685269 >> 172.21.15.32:0/941685269 conn(0x7f68780620b0 msgr2=0x7f6878067460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:30.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.250+0000 7f687cde6700 1 -- 172.21.15.32:0/941685269 shutdown_connections 2020-12-05T17:41:30.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.250+0000 7f687cde6700 1 -- 172.21.15.32:0/941685269 wait complete. 2020-12-05T17:41:30.273 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-12-05T17:41:30.383 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.378+0000 7efeeffff700 1 Processor -- start 2020-12-05T17:41:30.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.378+0000 7efeeffff700 1 -- start start 2020-12-05T17:41:30.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.378+0000 7efeeffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efef0062400 0x7efef005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:30.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.378+0000 7efeeffff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7efef005feb0 0x7efef00b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:30.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.378+0000 7efeeffff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef0142450 con 0x7efef005f3b0 2020-12-05T17:41:30.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.378+0000 7efeeffff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef00b42f0 con 0x7efef0062400 2020-12-05T17:41:30.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.378+0000 7efeeffff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef0061780 con 0x7efef005feb0 2020-12-05T17:41:30.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.378+0000 7efeee7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efef0062400 0x7efef005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:30.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.378+0000 7efeeeffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efef005f3b0 0x7efef00b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43470/0 (socket says 172.21.15.32:43470) 2020-12-05T17:41:30.387 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.378+0000 7efeeeffd700 1 -- 172.21.15.32:0/40450710 learned_addr learned my addr 172.21.15.32:0/40450710 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:30.388 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.378+0000 7efeef7fe700 1 --2- 172.21.15.32:0/40450710 >> v2:172.21.15.61:3300/0 conn(0x7efef005feb0 0x7efef00b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:30.388 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.378+0000 7efeedffb700 1 -- 172.21.15.32:0/40450710 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 885936879 0 0) 0x7efef0142450 con 0x7efef005f3b0 2020-12-05T17:41:30.388 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.378+0000 7efeedffb700 1 -- 172.21.15.32:0/40450710 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efed4003230 con 0x7efef005f3b0 2020-12-05T17:41:30.388 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.378+0000 7efeee7fc700 1 -- 172.21.15.32:0/40450710 >> v2:172.21.15.61:3300/0 conn(0x7efef005feb0 msgr2=0x7efef00b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:30.388 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.378+0000 7efeee7fc700 1 --2- 172.21.15.32:0/40450710 >> v2:172.21.15.61:3300/0 conn(0x7efef005feb0 0x7efef00b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.389 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.378+0000 7efeee7fc700 1 -- 172.21.15.32:0/40450710 >> v1:172.21.15.32:6789/0 conn(0x7efef005f3b0 legacy=0x7efef00b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:30.389 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.378+0000 7efeee7fc700 1 -- 172.21.15.32:0/40450710 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef0061a20 con 0x7efef0062400 2020-12-05T17:41:30.389 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeee7fc700 1 --2- 172.21.15.32:0/40450710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef0062400 0x7efef005f790 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7efee0002f30 tx=0x7efee0006dc0).ready entity=mon.0 client_cookie=5d4459746f89df71 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:30.389 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeedffb700 1 -- 172.21.15.32:0/40450710 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efee00096f0 con 0x7efef0062400 2020-12-05T17:41:30.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeedffb700 1 -- 172.21.15.32:0/40450710 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee0009850 con 0x7efef0062400 2020-12-05T17:41:30.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeedffb700 1 -- 172.21.15.32:0/40450710 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efee0009bb0 con 0x7efef0062400 2020-12-05T17:41:30.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeeffff700 1 -- 172.21.15.32:0/40450710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef0062400 msgr2=0x7efef005f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:30.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeeffff700 1 --2- 172.21.15.32:0/40450710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef0062400 0x7efef005f790 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7efee0002f30 tx=0x7efee0006dc0).stop 2020-12-05T17:41:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeeffff700 1 -- 172.21.15.32:0/40450710 shutdown_connections 2020-12-05T17:41:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeeffff700 1 --2- 172.21.15.32:0/40450710 >> v2:172.21.15.61:3300/0 conn(0x7efef005feb0 0x7efef00b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeeffff700 1 --2- 172.21.15.32:0/40450710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef0062400 0x7efef005f790 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeeffff700 1 -- 172.21.15.32:0/40450710 >> 172.21.15.32:0/40450710 conn(0x7efef00b8ee0 msgr2=0x7efef00b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeeffff700 1 -- 172.21.15.32:0/40450710 shutdown_connections 2020-12-05T17:41:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeeffff700 1 -- 172.21.15.32:0/40450710 wait complete. 2020-12-05T17:41:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeeffff700 1 Processor -- start 2020-12-05T17:41:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeeffff700 1 -- start start 2020-12-05T17:41:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeeffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efef005f3b0 0x7efef0063340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:30.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeeffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef005feb0 0x7efef01493d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:30.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeeffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efef0062400 0x7efef00af740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:30.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeeffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef0148920 con 0x7efef005feb0 2020-12-05T17:41:30.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeeffff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef0148bc0 con 0x7efef005f3b0 2020-12-05T17:41:30.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeeffff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef0148e60 con 0x7efef0062400 2020-12-05T17:41:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeee7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef005feb0 0x7efef01493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeee7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef005feb0 0x7efef01493d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40830/0 (socket says 172.21.15.32:40830) 2020-12-05T17:41:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeee7fc700 1 -- 172.21.15.32:0/539785499 learned_addr learned my addr 172.21.15.32:0/539785499 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeef7fe700 1 --2- 172.21.15.32:0/539785499 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efef0062400 0x7efef00af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:30.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.382+0000 7efeeeffd700 1 --2- 172.21.15.32:0/539785499 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efef005f3b0 0x7efef0063340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:30.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.386+0000 7efeee7fc700 1 -- 172.21.15.32:0/539785499 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efef0062400 msgr2=0x7efef00af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:30.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.386+0000 7efeee7fc700 1 --2- 172.21.15.32:0/539785499 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efef0062400 0x7efef00af740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.386+0000 7efeee7fc700 1 -- 172.21.15.32:0/539785499 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efef005f3b0 msgr2=0x7efef0063340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:30.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.386+0000 7efeee7fc700 1 --2- 172.21.15.32:0/539785499 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efef005f3b0 0x7efef0063340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.386+0000 7efeee7fc700 1 -- 172.21.15.32:0/539785499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef013e390 con 0x7efef005feb0 2020-12-05T17:41:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.386+0000 7efeee7fc700 1 --2- 172.21.15.32:0/539785499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef005feb0 0x7efef01493d0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7efee0002f30 tx=0x7efee0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.386+0000 7efedffff700 1 -- 172.21.15.32:0/539785499 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efee0011b00 con 0x7efef005feb0 2020-12-05T17:41:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.386+0000 7efedffff700 1 -- 172.21.15.32:0/539785499 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee0011c60 con 0x7efef005feb0 2020-12-05T17:41:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.386+0000 7efeeffff700 1 -- 172.21.15.32:0/539785499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef013e640 con 0x7efef005feb0 2020-12-05T17:41:30.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.386+0000 7efedffff700 1 -- 172.21.15.32:0/539785499 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efee001d450 con 0x7efef005feb0 2020-12-05T17:41:30.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.386+0000 7efeeffff700 1 -- 172.21.15.32:0/539785499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef013eb70 con 0x7efef005feb0 2020-12-05T17:41:30.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.386+0000 7efeeffff700 1 -- 172.21.15.32:0/539785499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef0149220 con 0x7efef005feb0 2020-12-05T17:41:30.400 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.386+0000 7efedffff700 1 -- 172.21.15.32:0/539785499 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7efee0024030 con 0x7efef005feb0 2020-12-05T17:41:30.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.386+0000 7efedffff700 1 --2- 172.21.15.32:0/539785499 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efecc055440 0x7efecc057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.390+0000 7efeeeffd700 1 --2- 172.21.15.32:0/539785499 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efecc055440 0x7efecc057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.390+0000 7efedffff700 1 -- 172.21.15.32:0/539785499 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5600+0+0 (secure 0 0 0) 0x7efee006ee90 con 0x7efef005feb0 2020-12-05T17:41:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.390+0000 7efeeeffd700 1 --2- 172.21.15.32:0/539785499 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efecc055440 0x7efecc057900 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7efee400d000 tx=0x7efee4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.394+0000 7efedffff700 1 -- 172.21.15.32:0/539785499 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7efee0045370 con 0x7efef005feb0 2020-12-05T17:41:30.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.542+0000 7efeeffff700 1 -- 172.21.15.32:0/539785499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7efef0149020 con 0x7efef005feb0 2020-12-05T17:41:30.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.542+0000 7efedffff700 1 -- 172.21.15.32:0/539785499 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v133) v1 ==== 84+0+143 (secure 0 0 0) 0x7efee0022020 con 0x7efef005feb0 2020-12-05T17:41:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:30.547 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2020-12-05T17:41:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2020-12-05T17:41:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2020-12-05T17:41:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2020-12-05T17:41:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2020-12-05T17:41:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:30.549 INFO:teuthology.orchestra.run.smithi032.stdout:] 2020-12-05T17:41:30.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.542+0000 7efeeffff700 1 -- 172.21.15.32:0/539785499 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efecc055440 msgr2=0x7efecc057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:30.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.542+0000 7efeeffff700 1 --2- 172.21.15.32:0/539785499 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efecc055440 0x7efecc057900 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7efee400d000 tx=0x7efee4009040).stop 2020-12-05T17:41:30.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.542+0000 7efeeffff700 1 -- 172.21.15.32:0/539785499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef005feb0 msgr2=0x7efef01493d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:30.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.542+0000 7efeeffff700 1 --2- 172.21.15.32:0/539785499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef005feb0 0x7efef01493d0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7efee0002f30 tx=0x7efee0015000).stop 2020-12-05T17:41:30.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.542+0000 7efeeffff700 1 -- 172.21.15.32:0/539785499 shutdown_connections 2020-12-05T17:41:30.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.542+0000 7efeeffff700 1 --2- 172.21.15.32:0/539785499 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efecc055440 0x7efecc057900 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.542+0000 7efeeffff700 1 --2- 172.21.15.32:0/539785499 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efef0062400 0x7efef00af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.542+0000 7efeeffff700 1 --2- 172.21.15.32:0/539785499 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efef005f3b0 0x7efef0063340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.542+0000 7efeeffff700 1 --2- 172.21.15.32:0/539785499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef005feb0 0x7efef01493d0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.542+0000 7efeeffff700 1 -- 172.21.15.32:0/539785499 >> 172.21.15.32:0/539785499 conn(0x7efef00b8ee0 msgr2=0x7efef0140600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:30.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.542+0000 7efeeffff700 1 -- 172.21.15.32:0/539785499 shutdown_connections 2020-12-05T17:41:30.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.542+0000 7efeeffff700 1 -- 172.21.15.32:0/539785499 wait complete. 2020-12-05T17:41:30.560 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-12-05T17:41:30.561 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-12-05T17:41:30.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e6a6e700 1 Processor -- start 2020-12-05T17:41:30.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e6a6e700 1 -- start start 2020-12-05T17:41:30.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e6a6e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14e00b2000 0x7f14e00af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:30.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e6a6e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f14e00aed90 0x7f14e0140db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:30.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e6a6e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14e0129180 0x7f14e0129610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:30.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e6a6e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14e00649e0 con 0x7f14e0129180 2020-12-05T17:41:30.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e6a6e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14e00b23e0 con 0x7f14e00aed90 2020-12-05T17:41:30.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e6a6e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e0062d80 con 0x7f14e00b2000 2020-12-05T17:41:30.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e626d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14e0129180 0x7f14e0129610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:30.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e626d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14e0129180 0x7f14e0129610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40838/0 (socket says 172.21.15.32:40838) 2020-12-05T17:41:30.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e626d700 1 -- 172.21.15.32:0/1753923181 learned_addr learned my addr 172.21.15.32:0/1753923181 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:30.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e5a6c700 1 --2- 172.21.15.32:0/1753923181 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14e00b2000 0x7f14e00af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:30.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e526b700 1 --2- 172.21.15.32:0/1753923181 >> v2:172.21.15.61:3300/0 conn(0x7f14e00aed90 0x7f14e0140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:30.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e626d700 1 -- 172.21.15.32:0/1753923181 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14e00b2000 msgr2=0x7f14e00af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:30.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e626d700 1 --2- 172.21.15.32:0/1753923181 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14e00b2000 0x7f14e00af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e626d700 1 -- 172.21.15.32:0/1753923181 >> v2:172.21.15.61:3300/0 conn(0x7f14e00aed90 msgr2=0x7f14e0140db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:30.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e626d700 1 --2- 172.21.15.32:0/1753923181 >> v2:172.21.15.61:3300/0 conn(0x7f14e00aed90 0x7f14e0140db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e626d700 1 -- 172.21.15.32:0/1753923181 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e0140670 con 0x7f14e0129180 2020-12-05T17:41:30.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e526b700 1 --2- 172.21.15.32:0/1753923181 >> v2:172.21.15.61:3300/0 conn(0x7f14e00aed90 0x7f14e0140db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:30.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e626d700 1 --2- 172.21.15.32:0/1753923181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e0129180 0x7f14e0129610 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7f14d80032b0 tx=0x7f14d8009270).ready entity=mon.0 client_cookie=98fbcc58125c81e7 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:30.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e4a6a700 1 -- 172.21.15.32:0/1753923181 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14d8009e60 con 0x7f14e0129180 2020-12-05T17:41:30.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e4a6a700 1 -- 172.21.15.32:0/1753923181 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d800bc00 con 0x7f14e0129180 2020-12-05T17:41:30.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e4a6a700 1 -- 172.21.15.32:0/1753923181 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14d800bd70 con 0x7f14e0129180 2020-12-05T17:41:30.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e6a6e700 1 -- 172.21.15.32:0/1753923181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e0129180 msgr2=0x7f14e0129610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:30.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.678+0000 7f14e6a6e700 1 --2- 172.21.15.32:0/1753923181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e0129180 0x7f14e0129610 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7f14d80032b0 tx=0x7f14d8009270).stop 2020-12-05T17:41:30.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.682+0000 7f14e6a6e700 1 -- 172.21.15.32:0/1753923181 shutdown_connections 2020-12-05T17:41:30.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.682+0000 7f14e6a6e700 1 --2- 172.21.15.32:0/1753923181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e0129180 0x7f14e0129610 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.682+0000 7f14e6a6e700 1 --2- 172.21.15.32:0/1753923181 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14e00b2000 0x7f14e00af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.682+0000 7f14e6a6e700 1 --2- 172.21.15.32:0/1753923181 >> v2:172.21.15.61:3300/0 conn(0x7f14e00aed90 0x7f14e0140db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.682+0000 7f14e6a6e700 1 -- 172.21.15.32:0/1753923181 >> 172.21.15.32:0/1753923181 conn(0x7f14e00b5620 msgr2=0x7f14e00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:30.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.682+0000 7f14e6a6e700 1 -- 172.21.15.32:0/1753923181 shutdown_connections 2020-12-05T17:41:30.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.682+0000 7f14e6a6e700 1 -- 172.21.15.32:0/1753923181 wait complete. 2020-12-05T17:41:30.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.682+0000 7f14e6a6e700 1 Processor -- start 2020-12-05T17:41:30.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.682+0000 7f14e6a6e700 1 -- start start 2020-12-05T17:41:30.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.682+0000 7f14e6a6e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e00aed90 0x7f14e00b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.682+0000 7f14e6a6e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14e00b2000 0x7f14e0142d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.682+0000 7f14e6a6e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14e0129180 0x7f14e0143230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.682+0000 7f14e6a6e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e0127640 con 0x7f14e00aed90 2020-12-05T17:41:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.682+0000 7f14e6a6e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e00acc90 con 0x7f14e0129180 2020-12-05T17:41:30.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.682+0000 7f14e6a6e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e00ad210 con 0x7f14e00b2000 2020-12-05T17:41:30.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.682+0000 7f14e5a6c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e00aed90 0x7f14e00b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:30.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.682+0000 7f14e5a6c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e00aed90 0x7f14e00b3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40842/0 (socket says 172.21.15.32:40842) 2020-12-05T17:41:30.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.682+0000 7f14e5a6c700 1 -- 172.21.15.32:0/3670048641 learned_addr learned my addr 172.21.15.32:0/3670048641 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:30.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.682+0000 7f14e626d700 1 --2- 172.21.15.32:0/3670048641 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14e0129180 0x7f14e0143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:30.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.682+0000 7f14e526b700 1 --2- 172.21.15.32:0/3670048641 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14e00b2000 0x7f14e0142d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:30.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.686+0000 7f14e626d700 1 -- 172.21.15.32:0/3670048641 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14e00b2000 msgr2=0x7f14e0142d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:30.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.686+0000 7f14e626d700 1 --2- 172.21.15.32:0/3670048641 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14e00b2000 0x7f14e0142d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.686+0000 7f14e626d700 1 -- 172.21.15.32:0/3670048641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e00aed90 msgr2=0x7f14e00b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:30.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.686+0000 7f14e626d700 1 --2- 172.21.15.32:0/3670048641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e00aed90 0x7f14e00b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.686+0000 7f14e626d700 1 -- 172.21.15.32:0/3670048641 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e01476d0 con 0x7f14e0129180 2020-12-05T17:41:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.686+0000 7f14e5a6c700 1 --2- 172.21.15.32:0/3670048641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e00aed90 0x7f14e00b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.686+0000 7f14e626d700 1 --2- 172.21.15.32:0/3670048641 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14e0129180 0x7f14e0143230 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f14d80032b0 tx=0x7f14d8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.686+0000 7f14ceffd700 1 -- 172.21.15.32:0/3670048641 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14d801f3f0 con 0x7f14e0129180 2020-12-05T17:41:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.686+0000 7f14ceffd700 1 -- 172.21.15.32:0/3670048641 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d801f550 con 0x7f14e0129180 2020-12-05T17:41:30.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.686+0000 7f14e6a6e700 1 -- 172.21.15.32:0/3670048641 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e0143720 con 0x7f14e0129180 2020-12-05T17:41:30.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.686+0000 7f14ceffd700 1 -- 172.21.15.32:0/3670048641 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14d801f8b0 con 0x7f14e0129180 2020-12-05T17:41:30.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.686+0000 7f14e6a6e700 1 -- 172.21.15.32:0/3670048641 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e01444c0 con 0x7f14e0129180 2020-12-05T17:41:30.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.686+0000 7f14ceffd700 1 -- 172.21.15.32:0/3670048641 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f14d800f070 con 0x7f14e0129180 2020-12-05T17:41:30.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.686+0000 7f14e6a6e700 1 -- 172.21.15.32:0/3670048641 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e0057050 con 0x7f14e0129180 2020-12-05T17:41:30.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.686+0000 7f14ceffd700 1 --2- 172.21.15.32:0/3670048641 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f14c0055470 0x7f14c0057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:30.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.686+0000 7f14ceffd700 1 -- 172.21.15.32:0/3670048641 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f14d806e540 con 0x7f14e0129180 2020-12-05T17:41:30.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.686+0000 7f14e5a6c700 1 --2- 172.21.15.32:0/3670048641 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f14c0055470 0x7f14c0057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:30.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.690+0000 7f14e5a6c700 1 --2- 172.21.15.32:0/3670048641 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f14c0055470 0x7f14c0057930 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f14dc00b000 tx=0x7f14dc006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.698+0000 7f14ceffd700 1 -- 172.21.15.32:0/3670048641 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f14d8044980 con 0x7f14e0129180 2020-12-05T17:41:30.845 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:30.838+0000 7f14e6a6e700 1 -- 172.21.15.32:0/3670048641 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f14e00acdd0 con 0x7f14e0129180 2020-12-05T17:41:31.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.022+0000 7f14ceffd700 1 -- 172.21.15.32:0/3670048641 <== mon.1 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v134) v1 ==== 133+0+0 (secure 0 0 0) 0x7f14d8015020 con 0x7f14e0129180 2020-12-05T17:41:31.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.026+0000 7f14e6a6e700 1 -- 172.21.15.32:0/3670048641 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f14c0055470 msgr2=0x7f14c0057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:31.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.026+0000 7f14e6a6e700 1 --2- 172.21.15.32:0/3670048641 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f14c0055470 0x7f14c0057930 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f14dc00b000 tx=0x7f14dc006dc0).stop 2020-12-05T17:41:31.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.026+0000 7f14e6a6e700 1 -- 172.21.15.32:0/3670048641 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14e0129180 msgr2=0x7f14e0143230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:31.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.026+0000 7f14e6a6e700 1 --2- 172.21.15.32:0/3670048641 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14e0129180 0x7f14e0143230 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f14d80032b0 tx=0x7f14d8016000).stop 2020-12-05T17:41:31.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.026+0000 7f14e6a6e700 1 -- 172.21.15.32:0/3670048641 shutdown_connections 2020-12-05T17:41:31.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.026+0000 7f14e6a6e700 1 --2- 172.21.15.32:0/3670048641 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14e0129180 0x7f14e0143230 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:31.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.026+0000 7f14e6a6e700 1 --2- 172.21.15.32:0/3670048641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e00aed90 0x7f14e00b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:31.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.026+0000 7f14e6a6e700 1 --2- 172.21.15.32:0/3670048641 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f14c0055470 0x7f14c0057930 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:31.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.026+0000 7f14e6a6e700 1 --2- 172.21.15.32:0/3670048641 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14e00b2000 0x7f14e0142d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:31.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.026+0000 7f14e6a6e700 1 -- 172.21.15.32:0/3670048641 >> 172.21.15.32:0/3670048641 conn(0x7f14e00b5620 msgr2=0x7f14e0063dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:31.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.026+0000 7f14e6a6e700 1 -- 172.21.15.32:0/3670048641 shutdown_connections 2020-12-05T17:41:31.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.026+0000 7f14e6a6e700 1 -- 172.21.15.32:0/3670048641 wait complete. 2020-12-05T17:41:31.035 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2020-12-05T17:41:31.044 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-12-05T17:41:31.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.158+0000 7efcecfc4700 1 Processor -- start 2020-12-05T17:41:31.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.158+0000 7efcecfc4700 1 -- start start 2020-12-05T17:41:31.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.158+0000 7efcecfc4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efce805f550 0x7efce80acb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:31.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.158+0000 7efcecfc4700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7efce8063040 0x7efce813fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:31.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.158+0000 7efcecfc4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efce8067110 con 0x7efce80af090 2020-12-05T17:41:31.165 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.158+0000 7efcecfc4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce813f760 con 0x7efce805f550 2020-12-05T17:41:31.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.158+0000 7efcecfc4700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce813fa00 con 0x7efce8063040 2020-12-05T17:41:31.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.158+0000 7efce7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efce80af090 0x7efce80af500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43502/0 (socket says 172.21.15.32:43502) 2020-12-05T17:41:31.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.158+0000 7efce7fff700 1 -- 172.21.15.32:0/1138763744 learned_addr learned my addr 172.21.15.32:0/1138763744 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:31.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.158+0000 7efce77fe700 1 --2- 172.21.15.32:0/1138763744 >> v2:172.21.15.32:3300/0 conn(0x7efce805f550 0x7efce80acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:31.166 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.158+0000 7efce6ffd700 1 --2- 172.21.15.32:0/1138763744 >> v2:172.21.15.61:3300/0 conn(0x7efce8063040 0x7efce813fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:31.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.158+0000 7efce67fc700 1 -- 172.21.15.32:0/1138763744 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3156996855 0 0) 0x7efce8067110 con 0x7efce80af090 2020-12-05T17:41:31.168 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.158+0000 7efce67fc700 1 -- 172.21.15.32:0/1138763744 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcd4003230 con 0x7efce80af090 2020-12-05T17:41:31.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.162+0000 7efce77fe700 1 -- 172.21.15.32:0/1138763744 >> v2:172.21.15.61:3300/0 conn(0x7efce8063040 msgr2=0x7efce813fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:31.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.162+0000 7efce77fe700 1 --2- 172.21.15.32:0/1138763744 >> v2:172.21.15.61:3300/0 conn(0x7efce8063040 0x7efce813fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:31.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.162+0000 7efce77fe700 1 -- 172.21.15.32:0/1138763744 >> v1:172.21.15.32:6789/0 conn(0x7efce80af090 legacy=0x7efce80af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:31.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.162+0000 7efce77fe700 1 -- 172.21.15.32:0/1138763744 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce8066d90 con 0x7efce805f550 2020-12-05T17:41:31.169 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.162+0000 7efce77fe700 1 --2- 172.21.15.32:0/1138763744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efce805f550 0x7efce80acb80 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7efcd8002f30 tx=0x7efcd8006dc0).ready entity=mon.0 client_cookie=c02ac4fdef2dff61 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:31.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.162+0000 7efce67fc700 1 -- 172.21.15.32:0/1138763744 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efcd80096f0 con 0x7efce805f550 2020-12-05T17:41:31.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.162+0000 7efce67fc700 1 -- 172.21.15.32:0/1138763744 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd8009850 con 0x7efce805f550 2020-12-05T17:41:31.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.162+0000 7efce67fc700 1 -- 172.21.15.32:0/1138763744 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efcd8009bb0 con 0x7efce805f550 2020-12-05T17:41:31.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.162+0000 7efcecfc4700 1 -- 172.21.15.32:0/1138763744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efce805f550 msgr2=0x7efce80acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:31.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.162+0000 7efcecfc4700 1 --2- 172.21.15.32:0/1138763744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efce805f550 0x7efce80acb80 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7efcd8002f30 tx=0x7efcd8006dc0).stop 2020-12-05T17:41:31.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.162+0000 7efcecfc4700 1 -- 172.21.15.32:0/1138763744 shutdown_connections 2020-12-05T17:41:31.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.162+0000 7efcecfc4700 1 --2- 172.21.15.32:0/1138763744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efce805f550 0x7efce80acb80 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:31.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.162+0000 7efcecfc4700 1 --2- 172.21.15.32:0/1138763744 >> v2:172.21.15.61:3300/0 conn(0x7efce8063040 0x7efce813fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:31.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.162+0000 7efcecfc4700 1 -- 172.21.15.32:0/1138763744 >> 172.21.15.32:0/1138763744 conn(0x7efce80620b0 msgr2=0x7efce8062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:31.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.162+0000 7efcecfc4700 1 -- 172.21.15.32:0/1138763744 shutdown_connections 2020-12-05T17:41:31.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.162+0000 7efcecfc4700 1 -- 172.21.15.32:0/1138763744 wait complete. 2020-12-05T17:41:31.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.162+0000 7efcecfc4700 1 Processor -- start 2020-12-05T17:41:31.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.166+0000 7efcecfc4700 1 -- start start 2020-12-05T17:41:31.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.166+0000 7efcecfc4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efce805f550 0x7efce8142f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:31.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.166+0000 7efcecfc4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efce8063040 0x7efce813a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:31.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.166+0000 7efcecfc4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efce80af090 0x7efce813ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:31.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.166+0000 7efcecfc4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce81456a0 con 0x7efce805f550 2020-12-05T17:41:31.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.166+0000 7efcecfc4700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce8139e70 con 0x7efce8063040 2020-12-05T17:41:31.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.166+0000 7efcecfc4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce8066270 con 0x7efce80af090 2020-12-05T17:41:31.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.166+0000 7efce77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efce805f550 0x7efce8142f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:31.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.166+0000 7efce77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efce805f550 0x7efce8142f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40856/0 (socket says 172.21.15.32:40856) 2020-12-05T17:41:31.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.166+0000 7efce77fe700 1 -- 172.21.15.32:0/2529589047 learned_addr learned my addr 172.21.15.32:0/2529589047 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:31.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.166+0000 7efce7fff700 1 --2- 172.21.15.32:0/2529589047 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efce80af090 0x7efce813ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:31.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.166+0000 7efce6ffd700 1 --2- 172.21.15.32:0/2529589047 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efce8063040 0x7efce813a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:31.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.166+0000 7efce77fe700 1 -- 172.21.15.32:0/2529589047 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efce80af090 msgr2=0x7efce813ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:31.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.166+0000 7efce77fe700 1 --2- 172.21.15.32:0/2529589047 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efce80af090 0x7efce813ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:31.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.166+0000 7efce77fe700 1 -- 172.21.15.32:0/2529589047 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efce8063040 msgr2=0x7efce813a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:31.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.166+0000 7efce77fe700 1 --2- 172.21.15.32:0/2529589047 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efce8063040 0x7efce813a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:31.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.166+0000 7efce77fe700 1 -- 172.21.15.32:0/2529589047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce813a800 con 0x7efce805f550 2020-12-05T17:41:31.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.166+0000 7efce77fe700 1 --2- 172.21.15.32:0/2529589047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efce805f550 0x7efce8142f00 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0x7efcd8002f30 tx=0x7efcd8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:31.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.166+0000 7efce4ff9700 1 -- 172.21.15.32:0/2529589047 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efcd8011c30 con 0x7efce805f550 2020-12-05T17:41:31.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.166+0000 7efcecfc4700 1 -- 172.21.15.32:0/2529589047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce8145200 con 0x7efce805f550 2020-12-05T17:41:31.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.166+0000 7efce4ff9700 1 -- 172.21.15.32:0/2529589047 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd8011d90 con 0x7efce805f550 2020-12-05T17:41:31.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.166+0000 7efce4ff9700 1 -- 172.21.15.32:0/2529589047 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efcd801d640 con 0x7efce805f550 2020-12-05T17:41:31.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.166+0000 7efcecfc4700 1 -- 172.21.15.32:0/2529589047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce813f470 con 0x7efce805f550 2020-12-05T17:41:31.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.170+0000 7efcecfc4700 1 -- 172.21.15.32:0/2529589047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce81458a0 con 0x7efce805f550 2020-12-05T17:41:31.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.170+0000 7efce4ff9700 1 -- 172.21.15.32:0/2529589047 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7efcd800f070 con 0x7efce805f550 2020-12-05T17:41:31.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.170+0000 7efce4ff9700 1 --2- 172.21.15.32:0/2529589047 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efcc8055440 0x7efcc8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:31.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.170+0000 7efce6ffd700 1 --2- 172.21.15.32:0/2529589047 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efcc8055440 0x7efcc8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:31.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.170+0000 7efce4ff9700 1 -- 172.21.15.32:0/2529589047 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5954+0+0 (secure 0 0 0) 0x7efcd806ee10 con 0x7efce805f550 2020-12-05T17:41:31.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.170+0000 7efce6ffd700 1 --2- 172.21.15.32:0/2529589047 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efcc8055440 0x7efcc8057900 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7efcdc008000 tx=0x7efcdc008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:31.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.178+0000 7efce4ff9700 1 -- 172.21.15.32:0/2529589047 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7efcd80450e0 con 0x7efce805f550 2020-12-05T17:41:31.324 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:31.318+0000 7efcecfc4700 1 -- 172.21.15.32:0/2529589047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7efce8057050 con 0x7efce805f550 2020-12-05T17:41:32.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.034+0000 7efce4ff9700 1 -- 172.21.15.32:0/2529589047 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v135) v1 ==== 125+0+0 (secure 0 0 0) 0x7efcd8022030 con 0x7efce805f550 2020-12-05T17:41:32.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.038+0000 7efcecfc4700 1 -- 172.21.15.32:0/2529589047 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efcc8055440 msgr2=0x7efcc8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.038+0000 7efcecfc4700 1 --2- 172.21.15.32:0/2529589047 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efcc8055440 0x7efcc8057900 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7efcdc008000 tx=0x7efcdc008040).stop 2020-12-05T17:41:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.038+0000 7efcecfc4700 1 -- 172.21.15.32:0/2529589047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efce805f550 msgr2=0x7efce8142f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.038+0000 7efcecfc4700 1 --2- 172.21.15.32:0/2529589047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efce805f550 0x7efce8142f00 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0x7efcd8002f30 tx=0x7efcd8015000).stop 2020-12-05T17:41:32.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.038+0000 7efcecfc4700 1 -- 172.21.15.32:0/2529589047 shutdown_connections 2020-12-05T17:41:32.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.038+0000 7efcecfc4700 1 --2- 172.21.15.32:0/2529589047 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efcc8055440 0x7efcc8057900 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.038+0000 7efcecfc4700 1 --2- 172.21.15.32:0/2529589047 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efce80af090 0x7efce813ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.038+0000 7efcecfc4700 1 --2- 172.21.15.32:0/2529589047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efce805f550 0x7efce8142f00 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.038+0000 7efcecfc4700 1 --2- 172.21.15.32:0/2529589047 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efce8063040 0x7efce813a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.038+0000 7efcecfc4700 1 -- 172.21.15.32:0/2529589047 >> 172.21.15.32:0/2529589047 conn(0x7efce80620b0 msgr2=0x7efce80b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:32.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.038+0000 7efcecfc4700 1 -- 172.21.15.32:0/2529589047 shutdown_connections 2020-12-05T17:41:32.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.038+0000 7efcecfc4700 1 -- 172.21.15.32:0/2529589047 wait complete. 2020-12-05T17:41:32.046 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2020-12-05T17:41:32.063 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-12-05T17:41:32.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.174+0000 7f3c17fff700 1 Processor -- start 2020-12-05T17:41:32.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.174+0000 7f3c17fff700 1 -- start start 2020-12-05T17:41:32.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.174+0000 7f3c17fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3c180b1dc0 0x7f3c180b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:32.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.174+0000 7f3c17fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f3c180afd30 0x7f3c180ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:32.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.174+0000 7f3c17fff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c181468e0 con 0x7f3c180af5f0 2020-12-05T17:41:32.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.174+0000 7f3c17fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c18142c90 con 0x7f3c180b1dc0 2020-12-05T17:41:32.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.174+0000 7f3c17fff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c18146b10 con 0x7f3c180afd30 2020-12-05T17:41:32.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.174+0000 7f3c167fc700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f3c180afd30 0x7f3c180ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:32.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.174+0000 7f3c167fc700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f3c180afd30 0x7f3c180ae600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:60760/0 (socket says 172.21.15.32:60760) 2020-12-05T17:41:32.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.174+0000 7f3c167fc700 1 -- 172.21.15.32:0/662489188 learned_addr learned my addr 172.21.15.32:0/662489188 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:32.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.174+0000 7f3c16ffd700 1 --2- 172.21.15.32:0/662489188 >> v2:172.21.15.32:3300/0 conn(0x7f3c180b1dc0 0x7f3c180b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:32.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.174+0000 7f3c167fc700 1 -- 172.21.15.32:0/662489188 >> v2:172.21.15.32:3300/0 conn(0x7f3c180b1dc0 msgr2=0x7f3c180b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:32.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.174+0000 7f3c167fc700 1 --2- 172.21.15.32:0/662489188 >> v2:172.21.15.32:3300/0 conn(0x7f3c180b1dc0 0x7f3c180b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.174+0000 7f3c167fc700 1 -- 172.21.15.32:0/662489188 >> v1:172.21.15.61:6789/0 conn(0x7f3c180af5f0 legacy=0x7f3c180aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:32.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.174+0000 7f3c167fc700 1 -- 172.21.15.32:0/662489188 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c18146db0 con 0x7f3c180afd30 2020-12-05T17:41:32.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.174+0000 7f3c16ffd700 1 --2- 172.21.15.32:0/662489188 >> v2:172.21.15.32:3300/0 conn(0x7f3c180b1dc0 0x7f3c180b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:32.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.174+0000 7f3c167fc700 1 --2- 172.21.15.32:0/662489188 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c180afd30 0x7f3c180ae600 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f3c0c002f30 tx=0x7f3c0c006e80).ready entity=mon.1 client_cookie=c73c62a2fdf148f9 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:32.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.174+0000 7f3c15ffb700 1 -- 172.21.15.32:0/662489188 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c0c0097f0 con 0x7f3c180afd30 2020-12-05T17:41:32.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.174+0000 7f3c15ffb700 1 -- 172.21.15.32:0/662489188 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0c009950 con 0x7f3c180afd30 2020-12-05T17:41:32.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.174+0000 7f3c15ffb700 1 -- 172.21.15.32:0/662489188 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c0c009cb0 con 0x7f3c180afd30 2020-12-05T17:41:32.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.174+0000 7f3c17fff700 1 -- 172.21.15.32:0/662489188 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c180afd30 msgr2=0x7f3c180ae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:32.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.174+0000 7f3c17fff700 1 --2- 172.21.15.32:0/662489188 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c180afd30 0x7f3c180ae600 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f3c0c002f30 tx=0x7f3c0c006e80).stop 2020-12-05T17:41:32.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c17fff700 1 -- 172.21.15.32:0/662489188 shutdown_connections 2020-12-05T17:41:32.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c17fff700 1 --2- 172.21.15.32:0/662489188 >> v2:172.21.15.32:3300/0 conn(0x7f3c180b1dc0 0x7f3c180b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c17fff700 1 --2- 172.21.15.32:0/662489188 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c180afd30 0x7f3c180ae600 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c17fff700 1 -- 172.21.15.32:0/662489188 >> 172.21.15.32:0/662489188 conn(0x7f3c180620b0 msgr2=0x7f3c18062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:32.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c17fff700 1 -- 172.21.15.32:0/662489188 shutdown_connections 2020-12-05T17:41:32.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c17fff700 1 -- 172.21.15.32:0/662489188 wait complete. 2020-12-05T17:41:32.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c17fff700 1 Processor -- start 2020-12-05T17:41:32.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c17fff700 1 -- start start 2020-12-05T17:41:32.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c17fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c180af5f0 0x7f3c18140720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:32.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c17fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3c180afd30 0x7f3c181516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:32.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c17fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c180b1dc0 0x7f3c181552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:32.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c17fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c18151bc0 con 0x7f3c180af5f0 2020-12-05T17:41:32.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c17fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c18063470 con 0x7f3c180b1dc0 2020-12-05T17:41:32.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c17fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c18154e20 con 0x7f3c180afd30 2020-12-05T17:41:32.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c16ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c180af5f0 0x7f3c18140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:32.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c16ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c180af5f0 0x7f3c18140720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40870/0 (socket says 172.21.15.32:40870) 2020-12-05T17:41:32.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c16ffd700 1 -- 172.21.15.32:0/2228674414 learned_addr learned my addr 172.21.15.32:0/2228674414 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:32.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c167fc700 1 --2- 172.21.15.32:0/2228674414 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3c180afd30 0x7f3c181516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:32.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c177fe700 1 --2- 172.21.15.32:0/2228674414 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c180b1dc0 0x7f3c181552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:32.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c16ffd700 1 -- 172.21.15.32:0/2228674414 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3c180afd30 msgr2=0x7f3c181516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:32.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c16ffd700 1 --2- 172.21.15.32:0/2228674414 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3c180afd30 0x7f3c181516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c16ffd700 1 -- 172.21.15.32:0/2228674414 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c180b1dc0 msgr2=0x7f3c181552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:32.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c16ffd700 1 --2- 172.21.15.32:0/2228674414 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c180b1dc0 0x7f3c181552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c16ffd700 1 -- 172.21.15.32:0/2228674414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c18066d70 con 0x7f3c180af5f0 2020-12-05T17:41:32.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c16ffd700 1 --2- 172.21.15.32:0/2228674414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c180af5f0 0x7f3c18140720 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7f3c0800c000 tx=0x7f3c08008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:32.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c07fff700 1 -- 172.21.15.32:0/2228674414 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c0800c8f0 con 0x7f3c180af5f0 2020-12-05T17:41:32.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c07fff700 1 -- 172.21.15.32:0/2228674414 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0800ca50 con 0x7f3c180af5f0 2020-12-05T17:41:32.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c17fff700 1 -- 172.21.15.32:0/2228674414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c18063040 con 0x7f3c180af5f0 2020-12-05T17:41:32.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c07fff700 1 -- 172.21.15.32:0/2228674414 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c0800cdb0 con 0x7f3c180af5f0 2020-12-05T17:41:32.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.178+0000 7f3c17fff700 1 -- 172.21.15.32:0/2228674414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c18149ab0 con 0x7f3c180af5f0 2020-12-05T17:41:32.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.182+0000 7f3c17fff700 1 -- 172.21.15.32:0/2228674414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c18057050 con 0x7f3c180af5f0 2020-12-05T17:41:32.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.182+0000 7f3c07fff700 1 -- 172.21.15.32:0/2228674414 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3c08022070 con 0x7f3c180af5f0 2020-12-05T17:41:32.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.182+0000 7f3c07fff700 1 --2- 172.21.15.32:0/2228674414 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3bf4055440 0x7f3bf4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:32.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.182+0000 7f3c07fff700 1 -- 172.21.15.32:0/2228674414 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f3c0806e4a0 con 0x7f3c180af5f0 2020-12-05T17:41:32.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.182+0000 7f3c167fc700 1 --2- 172.21.15.32:0/2228674414 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3bf4055440 0x7f3bf4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:32.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.182+0000 7f3c167fc700 1 --2- 172.21.15.32:0/2228674414 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3bf4055440 0x7f3bf4057900 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f3c0c01c000 tx=0x7f3c0c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:32.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.186+0000 7f3c07fff700 1 -- 172.21.15.32:0/2228674414 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3c080446c0 con 0x7f3c180af5f0 2020-12-05T17:41:32.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.378+0000 7f3c17fff700 1 -- 172.21.15.32:0/2228674414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f3c18060c10 con 0x7f3c180af5f0 2020-12-05T17:41:32.389 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.386+0000 7f3c07fff700 1 -- 172.21.15.32:0/2228674414 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 27 and data pool 28 v65) v1 ==== 189+0+0 (secure 0 0 0) 0x7f3c08047ce0 con 0x7f3c180af5f0 2020-12-05T17:41:32.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.386+0000 7f3c17fff700 1 -- 172.21.15.32:0/2228674414 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3bf4055440 msgr2=0x7f3bf4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:32.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.386+0000 7f3c17fff700 1 --2- 172.21.15.32:0/2228674414 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3bf4055440 0x7f3bf4057900 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f3c0c01c000 tx=0x7f3c0c015000).stop 2020-12-05T17:41:32.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.386+0000 7f3c17fff700 1 -- 172.21.15.32:0/2228674414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c180af5f0 msgr2=0x7f3c18140720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:32.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.386+0000 7f3c17fff700 1 --2- 172.21.15.32:0/2228674414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c180af5f0 0x7f3c18140720 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7f3c0800c000 tx=0x7f3c08008040).stop 2020-12-05T17:41:32.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.386+0000 7f3c17fff700 1 -- 172.21.15.32:0/2228674414 shutdown_connections 2020-12-05T17:41:32.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.386+0000 7f3c17fff700 1 --2- 172.21.15.32:0/2228674414 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c180b1dc0 0x7f3c181552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.386+0000 7f3c17fff700 1 --2- 172.21.15.32:0/2228674414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c180af5f0 0x7f3c18140720 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.386+0000 7f3c17fff700 1 --2- 172.21.15.32:0/2228674414 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3bf4055440 0x7f3bf4057900 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.386+0000 7f3c17fff700 1 --2- 172.21.15.32:0/2228674414 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3c180afd30 0x7f3c181516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.386+0000 7f3c17fff700 1 -- 172.21.15.32:0/2228674414 >> 172.21.15.32:0/2228674414 conn(0x7f3c180620b0 msgr2=0x7f3c18145de0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:32.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.386+0000 7f3c17fff700 1 -- 172.21.15.32:0/2228674414 shutdown_connections 2020-12-05T17:41:32.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.390+0000 7f3c17fff700 1 -- 172.21.15.32:0/2228674414 wait complete. 2020-12-05T17:41:32.396 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 27 and data pool 28 2020-12-05T17:41:32.412 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:41:32.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f82ce700 1 Processor -- start 2020-12-05T17:41:32.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f82ce700 1 -- start start 2020-12-05T17:41:32.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f82ce700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1f005fe00 0x7fd1f0061910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f82ce700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1f00acca0 0x7fd1f014a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f82ce700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f00601e0 con 0x7fd1f00b00a0 2020-12-05T17:41:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f82ce700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1f00ac1a0 con 0x7fd1f005fe00 2020-12-05T17:41:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f82ce700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f0061fa0 con 0x7fd1f00acca0 2020-12-05T17:41:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f606a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1f005fe00 0x7fd1f0061910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f606a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1f005fe00 0x7fd1f0061910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40890/0 (socket says 172.21.15.32:40890) 2020-12-05T17:41:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f606a700 1 -- 172.21.15.32:0/1249399654 learned_addr learned my addr 172.21.15.32:0/1249399654 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f686b700 1 --2- 172.21.15.32:0/1249399654 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1f00acca0 0x7fd1f014a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:32.521 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f606a700 1 -- 172.21.15.32:0/1249399654 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1f00acca0 msgr2=0x7fd1f014a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f606a700 1 --2- 172.21.15.32:0/1249399654 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1f00acca0 0x7fd1f014a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f606a700 1 -- 172.21.15.32:0/1249399654 >> v1:172.21.15.61:6789/0 conn(0x7fd1f00b00a0 legacy=0x7fd1f00ac590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f606a700 1 -- 172.21.15.32:0/1249399654 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f0062240 con 0x7fd1f005fe00 2020-12-05T17:41:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f606a700 1 --2- 172.21.15.32:0/1249399654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f005fe00 0x7fd1f0061910 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 rx=0x7fd1e0002f30 tx=0x7fd1e0006e80).ready entity=mon.0 client_cookie=67514c237777defc server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f5068700 1 -- 172.21.15.32:0/1249399654 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1e00097f0 con 0x7fd1f005fe00 2020-12-05T17:41:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f5068700 1 -- 172.21.15.32:0/1249399654 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e0009950 con 0x7fd1f005fe00 2020-12-05T17:41:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f5068700 1 -- 172.21.15.32:0/1249399654 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1e0009cb0 con 0x7fd1f005fe00 2020-12-05T17:41:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f82ce700 1 -- 172.21.15.32:0/1249399654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f005fe00 msgr2=0x7fd1f0061910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f82ce700 1 --2- 172.21.15.32:0/1249399654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f005fe00 0x7fd1f0061910 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 rx=0x7fd1e0002f30 tx=0x7fd1e0006e80).stop 2020-12-05T17:41:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f82ce700 1 -- 172.21.15.32:0/1249399654 shutdown_connections 2020-12-05T17:41:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f82ce700 1 --2- 172.21.15.32:0/1249399654 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1f00acca0 0x7fd1f014a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f82ce700 1 --2- 172.21.15.32:0/1249399654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f005fe00 0x7fd1f0061910 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f82ce700 1 -- 172.21.15.32:0/1249399654 >> 172.21.15.32:0/1249399654 conn(0x7fd1f00b57c0 msgr2=0x7fd1f005f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:32.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f82ce700 1 -- 172.21.15.32:0/1249399654 shutdown_connections 2020-12-05T17:41:32.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.510+0000 7fd1f82ce700 1 -- 172.21.15.32:0/1249399654 wait complete. 2020-12-05T17:41:32.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1f82ce700 1 Processor -- start 2020-12-05T17:41:32.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1f82ce700 1 -- start start 2020-12-05T17:41:32.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1f82ce700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1f005fe00 0x7fd1f0066680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:32.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1f82ce700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1f00acca0 0x7fd1f009e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:32.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1f82ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f00b00a0 0x7fd1f0155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:32.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1f82ce700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f0142d40 con 0x7fd1f00b00a0 2020-12-05T17:41:32.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1f82ce700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f0159200 con 0x7fd1f005fe00 2020-12-05T17:41:32.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1f82ce700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f014a9d0 con 0x7fd1f00acca0 2020-12-05T17:41:32.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1f686b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f00b00a0 0x7fd1f0155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:32.527 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1f686b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f00b00a0 0x7fd1f0155bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40900/0 (socket says 172.21.15.32:40900) 2020-12-05T17:41:32.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1f686b700 1 -- 172.21.15.32:0/2630367828 learned_addr learned my addr 172.21.15.32:0/2630367828 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:32.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1f606a700 1 --2- 172.21.15.32:0/2630367828 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1f005fe00 0x7fd1f0066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:32.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1f5869700 1 --2- 172.21.15.32:0/2630367828 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1f00acca0 0x7fd1f009e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:32.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1f686b700 1 -- 172.21.15.32:0/2630367828 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1f00acca0 msgr2=0x7fd1f009e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:32.528 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1f686b700 1 --2- 172.21.15.32:0/2630367828 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1f00acca0 0x7fd1f009e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1f686b700 1 -- 172.21.15.32:0/2630367828 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1f005fe00 msgr2=0x7fd1f0066680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:32.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1f686b700 1 --2- 172.21.15.32:0/2630367828 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1f005fe00 0x7fd1f0066680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1f686b700 1 -- 172.21.15.32:0/2630367828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f014ac70 con 0x7fd1f00b00a0 2020-12-05T17:41:32.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1f686b700 1 --2- 172.21.15.32:0/2630367828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f00b00a0 0x7fd1f0155bf0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7fd1f0142e80 tx=0x7fd1e8006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:32.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1ef7fe700 1 -- 172.21.15.32:0/2630367828 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1e800ca10 con 0x7fd1f00b00a0 2020-12-05T17:41:32.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1ef7fe700 1 -- 172.21.15.32:0/2630367828 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e800cb70 con 0x7fd1f00b00a0 2020-12-05T17:41:32.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1ef7fe700 1 -- 172.21.15.32:0/2630367828 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1e800ced0 con 0x7fd1f00b00a0 2020-12-05T17:41:32.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1f82ce700 1 -- 172.21.15.32:0/2630367828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f0063700 con 0x7fd1f00b00a0 2020-12-05T17:41:32.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.514+0000 7fd1f82ce700 1 -- 172.21.15.32:0/2630367828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f0156440 con 0x7fd1f00b00a0 2020-12-05T17:41:32.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.518+0000 7fd1f82ce700 1 -- 172.21.15.32:0/2630367828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f014a7c0 con 0x7fd1f00b00a0 2020-12-05T17:41:32.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.518+0000 7fd1ef7fe700 1 -- 172.21.15.32:0/2630367828 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd1e801c460 con 0x7fd1f00b00a0 2020-12-05T17:41:32.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.518+0000 7fd1ef7fe700 1 --2- 172.21.15.32:0/2630367828 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1d00553b0 0x7fd1d0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:32.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.518+0000 7fd1ef7fe700 1 -- 172.21.15.32:0/2630367828 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd1e806e450 con 0x7fd1f00b00a0 2020-12-05T17:41:32.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.518+0000 7fd1f606a700 1 --2- 172.21.15.32:0/2630367828 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1d00553b0 0x7fd1d0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:32.533 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.518+0000 7fd1f606a700 1 --2- 172.21.15.32:0/2630367828 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1d00553b0 0x7fd1d0057870 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7fd1e001c000 tx=0x7fd1e0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:32.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.526+0000 7fd1ef7fe700 1 -- 172.21.15.32:0/2630367828 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd1e8044590 con 0x7fd1f00b00a0 2020-12-05T17:41:32.677 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.670+0000 7fd1f82ce700 1 -- 172.21.15.32:0/2630367828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1f0057050 con 0x7fd1f00b00a0 2020-12-05T17:41:32.679 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.674+0000 7fd1ef7fe700 1 -- 172.21.15.32:0/2630367828 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fd1e8047bb0 con 0x7fd1f00b00a0 2020-12-05T17:41:32.679 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:32.680 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":138,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:41:32.383424+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:41:30.899960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2020-12-05T17:41:31.381413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:41:32.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.674+0000 7fd1f82ce700 1 -- 172.21.15.32:0/2630367828 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1d00553b0 msgr2=0x7fd1d0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:32.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.674+0000 7fd1f82ce700 1 --2- 172.21.15.32:0/2630367828 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1d00553b0 0x7fd1d0057870 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7fd1e001c000 tx=0x7fd1e0015000).stop 2020-12-05T17:41:32.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.674+0000 7fd1f82ce700 1 -- 172.21.15.32:0/2630367828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f00b00a0 msgr2=0x7fd1f0155bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:32.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.674+0000 7fd1f82ce700 1 --2- 172.21.15.32:0/2630367828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f00b00a0 0x7fd1f0155bf0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7fd1f0142e80 tx=0x7fd1e8006dc0).stop 2020-12-05T17:41:32.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.674+0000 7fd1f82ce700 1 -- 172.21.15.32:0/2630367828 shutdown_connections 2020-12-05T17:41:32.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.674+0000 7fd1f82ce700 1 --2- 172.21.15.32:0/2630367828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f00b00a0 0x7fd1f0155bf0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.674+0000 7fd1f82ce700 1 --2- 172.21.15.32:0/2630367828 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1f005fe00 0x7fd1f0066680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.674+0000 7fd1f82ce700 1 --2- 172.21.15.32:0/2630367828 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd1d00553b0 0x7fd1d0057870 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.674+0000 7fd1f82ce700 1 --2- 172.21.15.32:0/2630367828 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd1f00acca0 0x7fd1f009e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.674+0000 7fd1f82ce700 1 -- 172.21.15.32:0/2630367828 >> 172.21.15.32:0/2630367828 conn(0x7fd1f00b57c0 msgr2=0x7fd1f00ae0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:32.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.674+0000 7fd1f82ce700 1 -- 172.21.15.32:0/2630367828 shutdown_connections 2020-12-05T17:41:32.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.678+0000 7fd1f82ce700 1 -- 172.21.15.32:0/2630367828 wait complete. 2020-12-05T17:41:32.693 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:41:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.794+0000 7fd09721b700 1 Processor -- start 2020-12-05T17:41:32.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.794+0000 7fd09721b700 1 -- start start 2020-12-05T17:41:32.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.794+0000 7fd09721b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0900b0880 0x7fd090066070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:32.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.794+0000 7fd09721b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd090143000 con 0x7fd0900b1980 2020-12-05T17:41:32.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.794+0000 7fd09721b700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd09005fad0 con 0x7fd09005f6f0 2020-12-05T17:41:32.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.794+0000 7fd09721b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0900b2f80 con 0x7fd0900b0880 2020-12-05T17:41:32.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.794+0000 7fd095a18700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd0900b1980 0x7fd090146580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43554/0 (socket says 172.21.15.32:43554) 2020-12-05T17:41:32.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.794+0000 7fd096219700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fd09005f6f0 0x7fd0901428f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:56192/0 (socket says 172.21.15.32:56192) 2020-12-05T17:41:32.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.794+0000 7fd095a18700 1 -- 172.21.15.32:0/1871623733 learned_addr learned my addr 172.21.15.32:0/1871623733 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:32.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.794+0000 7fd096a1a700 1 --2- 172.21.15.32:0/1871623733 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0900b0880 0x7fd090066070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:32.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.798+0000 7fd095217700 1 -- 172.21.15.32:0/1871623733 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3358183552 0 0) 0x7fd09005fad0 con 0x7fd09005f6f0 2020-12-05T17:41:32.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.798+0000 7fd095217700 1 -- 172.21.15.32:0/1871623733 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd078003230 con 0x7fd09005f6f0 2020-12-05T17:41:32.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.798+0000 7fd095217700 1 -- 172.21.15.32:0/1871623733 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 673858994 0 0) 0x7fd090143000 con 0x7fd0900b1980 2020-12-05T17:41:32.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.798+0000 7fd095217700 1 -- 172.21.15.32:0/1871623733 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd09005fad0 con 0x7fd0900b1980 2020-12-05T17:41:32.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.798+0000 7fd096a1a700 1 -- 172.21.15.32:0/1871623733 >> v1:172.21.15.61:6789/0 conn(0x7fd09005f6f0 legacy=0x7fd0901428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:32.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.798+0000 7fd096a1a700 1 -- 172.21.15.32:0/1871623733 >> v1:172.21.15.32:6789/0 conn(0x7fd0900b1980 legacy=0x7fd090146580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:32.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.798+0000 7fd096a1a700 1 -- 172.21.15.32:0/1871623733 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd090145e40 con 0x7fd0900b0880 2020-12-05T17:41:32.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.798+0000 7fd096a1a700 1 --2- 172.21.15.32:0/1871623733 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0900b0880 0x7fd090066070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fd0880032b0 tx=0x7fd0880091a0).ready entity=mon.2 client_cookie=82f558b367518e9d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:32.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.798+0000 7fd095217700 1 -- 172.21.15.32:0/1871623733 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd08800b940 con 0x7fd0900b0880 2020-12-05T17:41:32.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.798+0000 7fd095217700 1 -- 172.21.15.32:0/1871623733 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08800baa0 con 0x7fd0900b0880 2020-12-05T17:41:32.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.798+0000 7fd095217700 1 -- 172.21.15.32:0/1871623733 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd08800bc10 con 0x7fd0900b0880 2020-12-05T17:41:32.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.798+0000 7fd09721b700 1 -- 172.21.15.32:0/1871623733 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0900b0880 msgr2=0x7fd090066070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:32.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.798+0000 7fd09721b700 1 --2- 172.21.15.32:0/1871623733 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0900b0880 0x7fd090066070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fd0880032b0 tx=0x7fd0880091a0).stop 2020-12-05T17:41:32.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.798+0000 7fd09721b700 1 -- 172.21.15.32:0/1871623733 shutdown_connections 2020-12-05T17:41:32.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.798+0000 7fd09721b700 1 --2- 172.21.15.32:0/1871623733 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0900b0880 0x7fd090066070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.798+0000 7fd09721b700 1 -- 172.21.15.32:0/1871623733 >> 172.21.15.32:0/1871623733 conn(0x7fd0900604c0 msgr2=0x7fd09005c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:32.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.798+0000 7fd09721b700 1 -- 172.21.15.32:0/1871623733 shutdown_connections 2020-12-05T17:41:32.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.798+0000 7fd09721b700 1 -- 172.21.15.32:0/1871623733 wait complete. 2020-12-05T17:41:32.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.802+0000 7fd09721b700 1 Processor -- start 2020-12-05T17:41:32.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.802+0000 7fd09721b700 1 -- start start 2020-12-05T17:41:32.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.802+0000 7fd09721b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd09005f6f0 0x7fd090145220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.802+0000 7fd09721b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0900b0880 0x7fd090113350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.802+0000 7fd09721b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0900b1980 0x7fd0901518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.802+0000 7fd09721b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd090113840 con 0x7fd09005f6f0 2020-12-05T17:41:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.802+0000 7fd09721b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd090113a60 con 0x7fd0900b0880 2020-12-05T17:41:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.802+0000 7fd09721b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd090113d00 con 0x7fd0900b1980 2020-12-05T17:41:32.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.802+0000 7fd096219700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd09005f6f0 0x7fd090145220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:32.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.802+0000 7fd096219700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd09005f6f0 0x7fd090145220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40910/0 (socket says 172.21.15.32:40910) 2020-12-05T17:41:32.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.802+0000 7fd096219700 1 -- 172.21.15.32:0/2770522773 learned_addr learned my addr 172.21.15.32:0/2770522773 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:32.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.802+0000 7fd095a18700 1 --2- 172.21.15.32:0/2770522773 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0900b0880 0x7fd090113350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:32.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.802+0000 7fd096a1a700 1 --2- 172.21.15.32:0/2770522773 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0900b1980 0x7fd0901518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:32.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.802+0000 7fd096219700 1 -- 172.21.15.32:0/2770522773 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0900b1980 msgr2=0x7fd0901518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:32.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.802+0000 7fd096219700 1 --2- 172.21.15.32:0/2770522773 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0900b1980 0x7fd0901518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.802+0000 7fd096219700 1 -- 172.21.15.32:0/2770522773 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0900b0880 msgr2=0x7fd090113350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:32.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.802+0000 7fd096219700 1 --2- 172.21.15.32:0/2770522773 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0900b0880 0x7fd090113350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.802+0000 7fd096219700 1 -- 172.21.15.32:0/2770522773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd090113fa0 con 0x7fd09005f6f0 2020-12-05T17:41:32.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.802+0000 7fd096219700 1 --2- 172.21.15.32:0/2770522773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd09005f6f0 0x7fd090145220 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0x7fd08c0010d0 tx=0x7fd08c007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:32.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.806+0000 7fd07f7fe700 1 -- 172.21.15.32:0/2770522773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd08c0072e0 con 0x7fd09005f6f0 2020-12-05T17:41:32.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.806+0000 7fd07f7fe700 1 -- 172.21.15.32:0/2770522773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08c005520 con 0x7fd09005f6f0 2020-12-05T17:41:32.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.806+0000 7fd09721b700 1 -- 172.21.15.32:0/2770522773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd090151490 con 0x7fd09005f6f0 2020-12-05T17:41:32.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.806+0000 7fd07f7fe700 1 -- 172.21.15.32:0/2770522773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd08c005880 con 0x7fd09005f6f0 2020-12-05T17:41:32.815 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.806+0000 7fd09721b700 1 -- 172.21.15.32:0/2770522773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd090117ad0 con 0x7fd09005f6f0 2020-12-05T17:41:32.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.806+0000 7fd07f7fe700 1 -- 172.21.15.32:0/2770522773 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd08c010070 con 0x7fd09005f6f0 2020-12-05T17:41:32.816 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.806+0000 7fd09721b700 1 -- 172.21.15.32:0/2770522773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd090057050 con 0x7fd09005f6f0 2020-12-05T17:41:32.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.806+0000 7fd07f7fe700 1 --2- 172.21.15.32:0/2770522773 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd0700553b0 0x7fd070057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:32.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.806+0000 7fd07f7fe700 1 -- 172.21.15.32:0/2770522773 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd08c06d860 con 0x7fd09005f6f0 2020-12-05T17:41:32.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.806+0000 7fd095a18700 1 --2- 172.21.15.32:0/2770522773 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd0700553b0 0x7fd070057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:32.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.806+0000 7fd095a18700 1 --2- 172.21.15.32:0/2770522773 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd0700553b0 0x7fd070057870 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7fd08000d000 tx=0x7fd080009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.814+0000 7fd07f7fe700 1 -- 172.21.15.32:0/2770522773 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd08c0438f0 con 0x7fd09005f6f0 2020-12-05T17:41:32.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.954+0000 7fd09721b700 1 -- 172.21.15.32:0/2770522773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd090069690 con 0x7fd09005f6f0 2020-12-05T17:41:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.954+0000 7fd07f7fe700 1 -- 172.21.15.32:0/2770522773 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fd08c046f10 con 0x7fd09005f6f0 2020-12-05T17:41:32.959 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:32.960 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":138,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:41:32.383424+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:41:30.899960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2020-12-05T17:41:31.381413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:41:32.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.954+0000 7fd09721b700 1 -- 172.21.15.32:0/2770522773 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd0700553b0 msgr2=0x7fd070057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:32.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.954+0000 7fd09721b700 1 --2- 172.21.15.32:0/2770522773 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd0700553b0 0x7fd070057870 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7fd08000d000 tx=0x7fd080009040).stop 2020-12-05T17:41:32.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.954+0000 7fd09721b700 1 -- 172.21.15.32:0/2770522773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd09005f6f0 msgr2=0x7fd090145220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:32.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.954+0000 7fd09721b700 1 --2- 172.21.15.32:0/2770522773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd09005f6f0 0x7fd090145220 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0x7fd08c0010d0 tx=0x7fd08c007ac0).stop 2020-12-05T17:41:32.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.954+0000 7fd09721b700 1 -- 172.21.15.32:0/2770522773 shutdown_connections 2020-12-05T17:41:32.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.954+0000 7fd09721b700 1 --2- 172.21.15.32:0/2770522773 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd0700553b0 0x7fd070057870 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.954+0000 7fd09721b700 1 --2- 172.21.15.32:0/2770522773 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0900b1980 0x7fd0901518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.954+0000 7fd09721b700 1 --2- 172.21.15.32:0/2770522773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd09005f6f0 0x7fd090145220 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.954+0000 7fd09721b700 1 --2- 172.21.15.32:0/2770522773 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0900b0880 0x7fd090113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.954+0000 7fd09721b700 1 -- 172.21.15.32:0/2770522773 >> 172.21.15.32:0/2770522773 conn(0x7fd0900604c0 msgr2=0x7fd090141b60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.954+0000 7fd09721b700 1 -- 172.21.15.32:0/2770522773 shutdown_connections 2020-12-05T17:41:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:32.954+0000 7fd09721b700 1 -- 172.21.15.32:0/2770522773 wait complete. 2020-12-05T17:41:32.972 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-12-05T17:41:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.090+0000 7f55b3fff700 1 Processor -- start 2020-12-05T17:41:33.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.090+0000 7f55b3fff700 1 -- start start 2020-12-05T17:41:33.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.090+0000 7f55b3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f55b40af0d0 0x7f55b413daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:33.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.090+0000 7f55b3fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55b413df90 0x7f55b40637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:33.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.090+0000 7f55b3fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f55b40b0d70 0x7f55b4063ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:33.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.090+0000 7f55b3fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b405f5c0 con 0x7f55b40af0d0 2020-12-05T17:41:33.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.090+0000 7f55b3fff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b40af4b0 con 0x7f55b40b0d70 2020-12-05T17:41:33.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.090+0000 7f55b3fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b405f7f0 con 0x7f55b413df90 2020-12-05T17:41:33.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.090+0000 7f55b2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f55b40af0d0 0x7f55b413daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:33.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.090+0000 7f55b2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f55b40af0d0 0x7f55b413daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40918/0 (socket says 172.21.15.32:40918) 2020-12-05T17:41:33.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.090+0000 7f55b2ffd700 1 -- 172.21.15.32:0/96312854 learned_addr learned my addr 172.21.15.32:0/96312854 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:33.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.090+0000 7f55b27fc700 1 --2- 172.21.15.32:0/96312854 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55b413df90 0x7f55b40637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:33.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.090+0000 7f55b37fe700 1 --2- 172.21.15.32:0/96312854 >> v2:172.21.15.61:3300/0 conn(0x7f55b40b0d70 0x7f55b4063ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:33.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.094+0000 7f55b2ffd700 1 -- 172.21.15.32:0/96312854 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55b413df90 msgr2=0x7f55b40637f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:33.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.094+0000 7f55b2ffd700 1 --2- 172.21.15.32:0/96312854 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55b413df90 0x7f55b40637f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.094+0000 7f55b2ffd700 1 -- 172.21.15.32:0/96312854 >> v2:172.21.15.61:3300/0 conn(0x7f55b40b0d70 msgr2=0x7f55b4063ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:33.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.094+0000 7f55b2ffd700 1 --2- 172.21.15.32:0/96312854 >> v2:172.21.15.61:3300/0 conn(0x7f55b40b0d70 0x7f55b4063ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.094+0000 7f55b2ffd700 1 -- 172.21.15.32:0/96312854 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b4066ae0 con 0x7f55b40af0d0 2020-12-05T17:41:33.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.094+0000 7f55b2ffd700 1 --2- 172.21.15.32:0/96312854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55b40af0d0 0x7f55b413daa0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7f55a4002f30 tx=0x7f55a4006e80).ready entity=mon.0 client_cookie=efc3a09669bd3714 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:33.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.094+0000 7f55b1ffb700 1 -- 172.21.15.32:0/96312854 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55a4009640 con 0x7f55b40af0d0 2020-12-05T17:41:33.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.094+0000 7f55b1ffb700 1 -- 172.21.15.32:0/96312854 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a40097a0 con 0x7f55b40af0d0 2020-12-05T17:41:33.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.094+0000 7f55b1ffb700 1 -- 172.21.15.32:0/96312854 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55a4009b00 con 0x7f55b40af0d0 2020-12-05T17:41:33.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.094+0000 7f55b3fff700 1 -- 172.21.15.32:0/96312854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55b40af0d0 msgr2=0x7f55b413daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:33.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.094+0000 7f55b3fff700 1 --2- 172.21.15.32:0/96312854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55b40af0d0 0x7f55b413daa0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7f55a4002f30 tx=0x7f55a4006e80).stop 2020-12-05T17:41:33.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.094+0000 7f55b3fff700 1 -- 172.21.15.32:0/96312854 shutdown_connections 2020-12-05T17:41:33.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.094+0000 7f55b3fff700 1 --2- 172.21.15.32:0/96312854 >> v2:172.21.15.61:3300/0 conn(0x7f55b40b0d70 0x7f55b4063ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.094+0000 7f55b3fff700 1 --2- 172.21.15.32:0/96312854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55b40af0d0 0x7f55b413daa0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.094+0000 7f55b3fff700 1 --2- 172.21.15.32:0/96312854 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55b413df90 0x7f55b40637f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.094+0000 7f55b3fff700 1 -- 172.21.15.32:0/96312854 >> 172.21.15.32:0/96312854 conn(0x7f55b40620b0 msgr2=0x7f55b4062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.094+0000 7f55b3fff700 1 -- 172.21.15.32:0/96312854 shutdown_connections 2020-12-05T17:41:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.094+0000 7f55b3fff700 1 -- 172.21.15.32:0/96312854 wait complete. 2020-12-05T17:41:33.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.094+0000 7f55b3fff700 1 Processor -- start 2020-12-05T17:41:33.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.094+0000 7f55b3fff700 1 -- start start 2020-12-05T17:41:33.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.098+0000 7f55b3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55b40af0d0 0x7f55b4141fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:33.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.098+0000 7f55b3fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55b40b0d70 0x7f55b413e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:33.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.098+0000 7f55b3fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f55b413df90 0x7f55b4142fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:33.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.098+0000 7f55b3fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b413f510 con 0x7f55b40af0d0 2020-12-05T17:41:33.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.098+0000 7f55b3fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b413f740 con 0x7f55b413df90 2020-12-05T17:41:33.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.098+0000 7f55b3fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b413f9e0 con 0x7f55b40b0d70 2020-12-05T17:41:33.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.098+0000 7f55b2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55b40af0d0 0x7f55b4141fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:33.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.098+0000 7f55b2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55b40af0d0 0x7f55b4141fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40924/0 (socket says 172.21.15.32:40924) 2020-12-05T17:41:33.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.098+0000 7f55b2ffd700 1 -- 172.21.15.32:0/3365069408 learned_addr learned my addr 172.21.15.32:0/3365069408 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:33.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.098+0000 7f55b27fc700 1 --2- 172.21.15.32:0/3365069408 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55b40b0d70 0x7f55b413e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:33.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.098+0000 7f55b37fe700 1 --2- 172.21.15.32:0/3365069408 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f55b413df90 0x7f55b4142fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:33.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.098+0000 7f55b2ffd700 1 -- 172.21.15.32:0/3365069408 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55b40b0d70 msgr2=0x7f55b413e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:33.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.098+0000 7f55b2ffd700 1 --2- 172.21.15.32:0/3365069408 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55b40b0d70 0x7f55b413e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.098+0000 7f55b2ffd700 1 -- 172.21.15.32:0/3365069408 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f55b413df90 msgr2=0x7f55b4142fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:33.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.098+0000 7f55b2ffd700 1 --2- 172.21.15.32:0/3365069408 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f55b413df90 0x7f55b4142fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.098+0000 7f55b2ffd700 1 -- 172.21.15.32:0/3365069408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b41473c0 con 0x7f55b40af0d0 2020-12-05T17:41:33.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.098+0000 7f55b2ffd700 1 --2- 172.21.15.32:0/3365069408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55b40af0d0 0x7f55b4141fb0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7f55a4002f30 tx=0x7f55a4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:33.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.098+0000 7f55a3fff700 1 -- 172.21.15.32:0/3365069408 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55a4011ad0 con 0x7f55b40af0d0 2020-12-05T17:41:33.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.098+0000 7f55a3fff700 1 -- 172.21.15.32:0/3365069408 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a4011c30 con 0x7f55b40af0d0 2020-12-05T17:41:33.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.098+0000 7f55b3fff700 1 -- 172.21.15.32:0/3365069408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b4147550 con 0x7f55b40af0d0 2020-12-05T17:41:33.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.098+0000 7f55a3fff700 1 -- 172.21.15.32:0/3365069408 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55a401d450 con 0x7f55b40af0d0 2020-12-05T17:41:33.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.098+0000 7f55b3fff700 1 -- 172.21.15.32:0/3365069408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b413bd90 con 0x7f55b40af0d0 2020-12-05T17:41:33.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.098+0000 7f55b3fff700 1 -- 172.21.15.32:0/3365069408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b4057050 con 0x7f55b40af0d0 2020-12-05T17:41:33.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.102+0000 7f55a3fff700 1 -- 172.21.15.32:0/3365069408 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f55a400f070 con 0x7f55b40af0d0 2020-12-05T17:41:33.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.102+0000 7f55a3fff700 1 --2- 172.21.15.32:0/3365069408 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f55900553b0 0x7f5590057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:33.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.102+0000 7f55b27fc700 1 --2- 172.21.15.32:0/3365069408 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f55900553b0 0x7f5590057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:33.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.102+0000 7f55a3fff700 1 -- 172.21.15.32:0/3365069408 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f55a40705a0 con 0x7f55b40af0d0 2020-12-05T17:41:33.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.102+0000 7f55b27fc700 1 --2- 172.21.15.32:0/3365069408 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f55900553b0 0x7f5590057870 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f55a800b000 tx=0x7f55a8006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:33.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.110+0000 7f55a3fff700 1 -- 172.21.15.32:0/3365069408 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f55a4046630 con 0x7f55b40af0d0 2020-12-05T17:41:33.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.302+0000 7f55b3fff700 1 -- 172.21.15.32:0/3365069408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f55b405fee0 con 0x7f55b40af0d0 2020-12-05T17:41:33.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.394+0000 7f55a3fff700 1 -- 172.21.15.32:0/3365069408 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v66) v1 ==== 118+0+0 (secure 0 0 0) 0x7f55a4049c50 con 0x7f55b40af0d0 2020-12-05T17:41:33.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.398+0000 7f55b3fff700 1 -- 172.21.15.32:0/3365069408 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f55900553b0 msgr2=0x7f5590057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:33.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.398+0000 7f55b3fff700 1 --2- 172.21.15.32:0/3365069408 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f55900553b0 0x7f5590057870 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f55a800b000 tx=0x7f55a8006dc0).stop 2020-12-05T17:41:33.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.398+0000 7f55b3fff700 1 -- 172.21.15.32:0/3365069408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55b40af0d0 msgr2=0x7f55b4141fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:33.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.398+0000 7f55b3fff700 1 --2- 172.21.15.32:0/3365069408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55b40af0d0 0x7f55b4141fb0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7f55a4002f30 tx=0x7f55a4015000).stop 2020-12-05T17:41:33.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.398+0000 7f55b3fff700 1 -- 172.21.15.32:0/3365069408 shutdown_connections 2020-12-05T17:41:33.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.398+0000 7f55b3fff700 1 --2- 172.21.15.32:0/3365069408 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f55b413df90 0x7f55b4142fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.398+0000 7f55b3fff700 1 --2- 172.21.15.32:0/3365069408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55b40af0d0 0x7f55b4141fb0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.398+0000 7f55b3fff700 1 --2- 172.21.15.32:0/3365069408 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f55900553b0 0x7f5590057870 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.398+0000 7f55b3fff700 1 --2- 172.21.15.32:0/3365069408 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55b40b0d70 0x7f55b413e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.398+0000 7f55b3fff700 1 -- 172.21.15.32:0/3365069408 >> 172.21.15.32:0/3365069408 conn(0x7f55b40620b0 msgr2=0x7f55b40b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:33.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.398+0000 7f55b3fff700 1 -- 172.21.15.32:0/3365069408 shutdown_connections 2020-12-05T17:41:33.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.398+0000 7f55b3fff700 1 -- 172.21.15.32:0/3365069408 wait complete. 2020-12-05T17:41:33.421 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:41:33.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.538+0000 7f791bfff700 1 Processor -- start 2020-12-05T17:41:33.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.538+0000 7f791bfff700 1 -- start start 2020-12-05T17:41:33.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.538+0000 7f791bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f791c0b2070 0x7f791c0b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:33.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.538+0000 7f791bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f791c138e30 0x7f791c139270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:33.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.538+0000 7f791bfff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f791c0b2450 con 0x7f791c142980 2020-12-05T17:41:33.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.538+0000 7f791bfff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f791c0acb80 con 0x7f791c0b2070 2020-12-05T17:41:33.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.538+0000 7f791bfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791c066d40 con 0x7f791c138e30 2020-12-05T17:41:33.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.538+0000 7f791affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f791c0b2070 0x7f791c0b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:33.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.538+0000 7f791affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f791c0b2070 0x7f791c0b2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40932/0 (socket says 172.21.15.32:40932) 2020-12-05T17:41:33.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.538+0000 7f791affd700 1 -- 172.21.15.32:0/3625142055 learned_addr learned my addr 172.21.15.32:0/3625142055 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:33.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.538+0000 7f791a7fc700 1 --2- 172.21.15.32:0/3625142055 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f791c138e30 0x7f791c139270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:33.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.538+0000 7f791affd700 1 -- 172.21.15.32:0/3625142055 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f791c138e30 msgr2=0x7f791c139270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:33.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.538+0000 7f791affd700 1 --2- 172.21.15.32:0/3625142055 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f791c138e30 0x7f791c139270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.538+0000 7f791affd700 1 -- 172.21.15.32:0/3625142055 >> v1:172.21.15.61:6789/0 conn(0x7f791c142980 legacy=0x7f791c142df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:33.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.538+0000 7f791affd700 1 -- 172.21.15.32:0/3625142055 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791c066fe0 con 0x7f791c0b2070 2020-12-05T17:41:33.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.538+0000 7f7919ffb700 1 -- 172.21.15.32:0/3625142055 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2145228734 0 0) 0x7f791c0b2450 con 0x7f791c142980 2020-12-05T17:41:33.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.538+0000 7f791a7fc700 1 --2- 172.21.15.32:0/3625142055 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f791c138e30 0x7f791c139270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:33.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.538+0000 7f791affd700 1 --2- 172.21.15.32:0/3625142055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f791c0b2070 0x7f791c0b2ca0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0x7f790c002f30 tx=0x7f790c006e80).ready entity=mon.0 client_cookie=87e981c3c6e9675 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:33.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.538+0000 7f7919ffb700 1 -- 172.21.15.32:0/3625142055 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f790c0097f0 con 0x7f791c0b2070 2020-12-05T17:41:33.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.538+0000 7f7919ffb700 1 -- 172.21.15.32:0/3625142055 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790c009950 con 0x7f791c0b2070 2020-12-05T17:41:33.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.538+0000 7f791bfff700 1 -- 172.21.15.32:0/3625142055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f791c0b2070 msgr2=0x7f791c0b2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:33.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.538+0000 7f7919ffb700 1 -- 172.21.15.32:0/3625142055 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f790c009cb0 con 0x7f791c0b2070 2020-12-05T17:41:33.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.538+0000 7f791bfff700 1 --2- 172.21.15.32:0/3625142055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f791c0b2070 0x7f791c0b2ca0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0x7f790c002f30 tx=0x7f790c006e80).stop 2020-12-05T17:41:33.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791bfff700 1 -- 172.21.15.32:0/3625142055 shutdown_connections 2020-12-05T17:41:33.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791bfff700 1 --2- 172.21.15.32:0/3625142055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f791c0b2070 0x7f791c0b2ca0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791bfff700 1 --2- 172.21.15.32:0/3625142055 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f791c138e30 0x7f791c139270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791bfff700 1 -- 172.21.15.32:0/3625142055 >> 172.21.15.32:0/3625142055 conn(0x7f791c05bbb0 msgr2=0x7f791c062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:33.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791bfff700 1 -- 172.21.15.32:0/3625142055 shutdown_connections 2020-12-05T17:41:33.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791bfff700 1 -- 172.21.15.32:0/3625142055 wait complete. 2020-12-05T17:41:33.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791bfff700 1 Processor -- start 2020-12-05T17:41:33.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791bfff700 1 -- start start 2020-12-05T17:41:33.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f791c0b2070 0x7f791c0b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:33.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f791c138e30 0x7f791c0b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:33.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f791c142980 0x7f791c1398c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:33.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791bfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c0b4c00 con 0x7f791c142980 2020-12-05T17:41:33.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791bfff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c0b4e40 con 0x7f791c138e30 2020-12-05T17:41:33.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791bfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791c13bc80 con 0x7f791c0b2070 2020-12-05T17:41:33.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f791c142980 0x7f791c1398c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:33.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f791c142980 0x7f791c1398c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40942/0 (socket says 172.21.15.32:40942) 2020-12-05T17:41:33.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791b7fe700 1 -- 172.21.15.32:0/390418131 learned_addr learned my addr 172.21.15.32:0/390418131 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:33.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791affd700 1 --2- 172.21.15.32:0/390418131 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f791c0b2070 0x7f791c0b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:33.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791a7fc700 1 --2- 172.21.15.32:0/390418131 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f791c138e30 0x7f791c0b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791b7fe700 1 -- 172.21.15.32:0/390418131 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f791c0b2070 msgr2=0x7f791c0b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791b7fe700 1 --2- 172.21.15.32:0/390418131 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f791c0b2070 0x7f791c0b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791b7fe700 1 -- 172.21.15.32:0/390418131 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f791c138e30 msgr2=0x7f791c0b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791b7fe700 1 --2- 172.21.15.32:0/390418131 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f791c138e30 0x7f791c0b53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791b7fe700 1 -- 172.21.15.32:0/390418131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791c13a7e0 con 0x7f791c142980 2020-12-05T17:41:33.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791affd700 1 --2- 172.21.15.32:0/390418131 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f791c0b2070 0x7f791c0b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:33.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791b7fe700 1 --2- 172.21.15.32:0/390418131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f791c142980 0x7f791c1398c0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7f7914001340 tx=0x7f7914007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:33.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f7913fff700 1 -- 172.21.15.32:0/390418131 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7914006860 con 0x7f791c142980 2020-12-05T17:41:33.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f7913fff700 1 -- 172.21.15.32:0/390418131 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79140069c0 con 0x7f791c142980 2020-12-05T17:41:33.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791bfff700 1 -- 172.21.15.32:0/390418131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f791c0b58b0 con 0x7f791c142980 2020-12-05T17:41:33.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f7913fff700 1 -- 172.21.15.32:0/390418131 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7914006d20 con 0x7f791c142980 2020-12-05T17:41:33.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.542+0000 7f791bfff700 1 -- 172.21.15.32:0/390418131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791c139f50 con 0x7f791c142980 2020-12-05T17:41:33.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.546+0000 7f791bfff700 1 -- 172.21.15.32:0/390418131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f791c057050 con 0x7f791c142980 2020-12-05T17:41:33.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.546+0000 7f7913fff700 1 -- 172.21.15.32:0/390418131 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7914010070 con 0x7f791c142980 2020-12-05T17:41:33.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.546+0000 7f7913fff700 1 --2- 172.21.15.32:0/390418131 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79000554e0 0x7f79000579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:33.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.546+0000 7f791affd700 1 --2- 172.21.15.32:0/390418131 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79000554e0 0x7f79000579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:33.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.546+0000 7f7913fff700 1 -- 172.21.15.32:0/390418131 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f791406d830 con 0x7f791c142980 2020-12-05T17:41:33.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.546+0000 7f791affd700 1 --2- 172.21.15.32:0/390418131 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79000554e0 0x7f79000579a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f790c01a000 tx=0x7f790c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:33.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.550+0000 7f7913fff700 1 -- 172.21.15.32:0/390418131 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7914043860 con 0x7f791c142980 2020-12-05T17:41:33.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.738+0000 7f791bfff700 1 -- 172.21.15.32:0/390418131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f791c05c300 con 0x7f791c142980 2020-12-05T17:41:33.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.738+0000 7f7913fff700 1 -- 172.21.15.32:0/390418131 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f7914046e80 con 0x7f791c142980 2020-12-05T17:41:33.745 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:33.745 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":66,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/3049192140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3049192140},{"type":"v1","addr":"172.21.15.61:6833","nonce":3049192140}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:41:32.383702+0000","modified":"2020-12-05T17:41:33.387236+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8274":{"gid":8274,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/2808222868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2808222868},{"type":"v1","addr":"172.21.15.32:6835","nonce":2808222868}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2020-12-05T17:41:33.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.742+0000 7f791bfff700 1 -- 172.21.15.32:0/390418131 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79000554e0 msgr2=0x7f79000579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:33.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.742+0000 7f791bfff700 1 --2- 172.21.15.32:0/390418131 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79000554e0 0x7f79000579a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f790c01a000 tx=0x7f790c015000).stop 2020-12-05T17:41:33.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.742+0000 7f791bfff700 1 -- 172.21.15.32:0/390418131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f791c142980 msgr2=0x7f791c1398c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:33.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.742+0000 7f791bfff700 1 --2- 172.21.15.32:0/390418131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f791c142980 0x7f791c1398c0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7f7914001340 tx=0x7f7914007f40).stop 2020-12-05T17:41:33.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.742+0000 7f791bfff700 1 -- 172.21.15.32:0/390418131 shutdown_connections 2020-12-05T17:41:33.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.742+0000 7f791bfff700 1 --2- 172.21.15.32:0/390418131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f791c142980 0x7f791c1398c0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.742+0000 7f791bfff700 1 --2- 172.21.15.32:0/390418131 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f79000554e0 0x7f79000579a0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.742+0000 7f791bfff700 1 --2- 172.21.15.32:0/390418131 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f791c0b2070 0x7f791c0b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.742+0000 7f791bfff700 1 --2- 172.21.15.32:0/390418131 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f791c138e30 0x7f791c0b53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.742+0000 7f791bfff700 1 -- 172.21.15.32:0/390418131 >> 172.21.15.32:0/390418131 conn(0x7f791c05bbb0 msgr2=0x7f791c062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:33.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.742+0000 7f791bfff700 1 -- 172.21.15.32:0/390418131 shutdown_connections 2020-12-05T17:41:33.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.742+0000 7f791bfff700 1 -- 172.21.15.32:0/390418131 wait complete. 2020-12-05T17:41:33.750 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 66 2020-12-05T17:41:33.759 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:41:33.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.870+0000 7efc02362700 1 Processor -- start 2020-12-05T17:41:33.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.870+0000 7efc02362700 1 -- start start 2020-12-05T17:41:33.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efc02362700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7efbfc139540 0x7efbfc13f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:33.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efc02362700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efbfc0b2ca0 0x7efbfc13f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:33.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efc02362700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbfc0b2450 con 0x7efbfc0b2070 2020-12-05T17:41:33.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efc02362700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbfc0acb80 con 0x7efbfc0b2ca0 2020-12-05T17:41:33.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efc02362700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbfc066d40 con 0x7efbfc139540 2020-12-05T17:41:33.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efc01b61700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efbfc0b2ca0 0x7efbfc13f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:33.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efc01360700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7efbfc0b2070 0x7efbfc138e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:56234/0 (socket says 172.21.15.32:56234) 2020-12-05T17:41:33.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efc01360700 1 -- 172.21.15.32:0/893846930 learned_addr learned my addr 172.21.15.32:0/893846930 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:33.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efc00b5f700 1 --2- 172.21.15.32:0/893846930 >> v2:172.21.15.61:3300/0 conn(0x7efbfc139540 0x7efbfc13f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:33.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efbebfff700 1 -- 172.21.15.32:0/893846930 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1721769090 0 0) 0x7efbfc0b2450 con 0x7efbfc0b2070 2020-12-05T17:41:33.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efc01b61700 1 -- 172.21.15.32:0/893846930 >> v2:172.21.15.61:3300/0 conn(0x7efbfc139540 msgr2=0x7efbfc13f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:33.881 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efc01b61700 1 --2- 172.21.15.32:0/893846930 >> v2:172.21.15.61:3300/0 conn(0x7efbfc139540 0x7efbfc13f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efc01b61700 1 -- 172.21.15.32:0/893846930 >> v1:172.21.15.61:6789/0 conn(0x7efbfc0b2070 legacy=0x7efbfc138e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:33.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efc01b61700 1 -- 172.21.15.32:0/893846930 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbfc066fe0 con 0x7efbfc0b2ca0 2020-12-05T17:41:33.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efc01b61700 1 --2- 172.21.15.32:0/893846930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbfc0b2ca0 0x7efbfc13f980 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7efbf40032b0 tx=0x7efbf40091a0).ready entity=mon.0 client_cookie=8b884b3713e53d91 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:33.882 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efbebfff700 1 -- 172.21.15.32:0/893846930 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efbf400b9a0 con 0x7efbfc0b2ca0 2020-12-05T17:41:33.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efbebfff700 1 -- 172.21.15.32:0/893846930 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf400bb00 con 0x7efbfc0b2ca0 2020-12-05T17:41:33.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efbebfff700 1 -- 172.21.15.32:0/893846930 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efbf400bc70 con 0x7efbfc0b2ca0 2020-12-05T17:41:33.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efc02362700 1 -- 172.21.15.32:0/893846930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbfc0b2ca0 msgr2=0x7efbfc13f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:33.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efc02362700 1 --2- 172.21.15.32:0/893846930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbfc0b2ca0 0x7efbfc13f980 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7efbf40032b0 tx=0x7efbf40091a0).stop 2020-12-05T17:41:33.883 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efc02362700 1 -- 172.21.15.32:0/893846930 shutdown_connections 2020-12-05T17:41:33.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efc02362700 1 --2- 172.21.15.32:0/893846930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbfc0b2ca0 0x7efbfc13f980 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efc02362700 1 --2- 172.21.15.32:0/893846930 >> v2:172.21.15.61:3300/0 conn(0x7efbfc139540 0x7efbfc13f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efc02362700 1 -- 172.21.15.32:0/893846930 >> 172.21.15.32:0/893846930 conn(0x7efbfc05bbb0 msgr2=0x7efbfc062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:33.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efc02362700 1 -- 172.21.15.32:0/893846930 shutdown_connections 2020-12-05T17:41:33.884 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.874+0000 7efc02362700 1 -- 172.21.15.32:0/893846930 wait complete. 2020-12-05T17:41:33.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efc02362700 1 Processor -- start 2020-12-05T17:41:33.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efc02362700 1 -- start start 2020-12-05T17:41:33.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efc02362700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbfc0b2070 0x7efbfc13c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:33.887 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efc02362700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efbfc0b2ca0 0x7efbfc13c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:33.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efc02362700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efbfc139540 0x7efbfc13aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:33.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efc02362700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc13b770 con 0x7efbfc0b2070 2020-12-05T17:41:33.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efc02362700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc13ba10 con 0x7efbfc0b2ca0 2020-12-05T17:41:33.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efc02362700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc13bcb0 con 0x7efbfc139540 2020-12-05T17:41:33.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efc01360700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbfc0b2070 0x7efbfc13c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:33.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efc01360700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbfc0b2070 0x7efbfc13c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40952/0 (socket says 172.21.15.32:40952) 2020-12-05T17:41:33.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efc01360700 1 -- 172.21.15.32:0/2103756791 learned_addr learned my addr 172.21.15.32:0/2103756791 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:33.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efc01b61700 1 --2- 172.21.15.32:0/2103756791 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efbfc139540 0x7efbfc13aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:33.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efc00b5f700 1 --2- 172.21.15.32:0/2103756791 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efbfc0b2ca0 0x7efbfc13c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:33.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efc01360700 1 -- 172.21.15.32:0/2103756791 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efbfc139540 msgr2=0x7efbfc13aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:33.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efc01360700 1 --2- 172.21.15.32:0/2103756791 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efbfc139540 0x7efbfc13aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efc01360700 1 -- 172.21.15.32:0/2103756791 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efbfc0b2ca0 msgr2=0x7efbfc13c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:33.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efc01360700 1 --2- 172.21.15.32:0/2103756791 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efbfc0b2ca0 0x7efbfc13c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:33.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efc01360700 1 -- 172.21.15.32:0/2103756791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbfc0b5520 con 0x7efbfc0b2070 2020-12-05T17:41:33.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efc01360700 1 --2- 172.21.15.32:0/2103756791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbfc0b2070 0x7efbfc13c150 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7efbf8000b80 tx=0x7efbf8007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:33.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efbea7fc700 1 -- 172.21.15.32:0/2103756791 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efbf80072e0 con 0x7efbfc0b2070 2020-12-05T17:41:33.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efbea7fc700 1 -- 172.21.15.32:0/2103756791 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf8007440 con 0x7efbfc0b2070 2020-12-05T17:41:33.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efc02362700 1 -- 172.21.15.32:0/2103756791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbfc0b57d0 con 0x7efbfc0b2070 2020-12-05T17:41:33.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efbea7fc700 1 -- 172.21.15.32:0/2103756791 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efbf80057d0 con 0x7efbfc0b2070 2020-12-05T17:41:33.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.878+0000 7efc02362700 1 -- 172.21.15.32:0/2103756791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbfc065110 con 0x7efbfc0b2070 2020-12-05T17:41:33.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.882+0000 7efbea7fc700 1 -- 172.21.15.32:0/2103756791 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7efbf801a060 con 0x7efbfc0b2070 2020-12-05T17:41:33.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.882+0000 7efc02362700 1 -- 172.21.15.32:0/2103756791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbfc13be70 con 0x7efbfc0b2070 2020-12-05T17:41:33.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.882+0000 7efbea7fc700 1 --2- 172.21.15.32:0/2103756791 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efbdc0554e0 0x7efbdc0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:33.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.882+0000 7efbea7fc700 1 -- 172.21.15.32:0/2103756791 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7efbf806d950 con 0x7efbfc0b2070 2020-12-05T17:41:33.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.882+0000 7efc00b5f700 1 --2- 172.21.15.32:0/2103756791 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efbdc0554e0 0x7efbdc0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:33.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.882+0000 7efc00b5f700 1 --2- 172.21.15.32:0/2103756791 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efbdc0554e0 0x7efbdc0579a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7efbec00b000 tx=0x7efbec006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:33.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:33.886+0000 7efbea7fc700 1 -- 172.21.15.32:0/2103756791 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7efbf8043af0 con 0x7efbfc0b2070 2020-12-05T17:41:34.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.030+0000 7efc02362700 1 -- 172.21.15.32:0/2103756791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efbfc057050 con 0x7efbfc0b2070 2020-12-05T17:41:34.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.030+0000 7efbea7fc700 1 -- 172.21.15.32:0/2103756791 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16285 (secure 0 0 0) 0x7efbf8018070 con 0x7efbfc0b2070 2020-12-05T17:41:34.037 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:34.037 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":138,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:41:32.383424+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:41:30.899960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2020-12-05T17:41:31.381413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:41:34.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.034+0000 7efc02362700 1 -- 172.21.15.32:0/2103756791 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efbdc0554e0 msgr2=0x7efbdc0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:34.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.034+0000 7efc02362700 1 --2- 172.21.15.32:0/2103756791 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efbdc0554e0 0x7efbdc0579a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7efbec00b000 tx=0x7efbec006dc0).stop 2020-12-05T17:41:34.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.034+0000 7efc02362700 1 -- 172.21.15.32:0/2103756791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbfc0b2070 msgr2=0x7efbfc13c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:34.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.034+0000 7efc02362700 1 --2- 172.21.15.32:0/2103756791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbfc0b2070 0x7efbfc13c150 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7efbf8000b80 tx=0x7efbf8007ac0).stop 2020-12-05T17:41:34.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.034+0000 7efc02362700 1 -- 172.21.15.32:0/2103756791 shutdown_connections 2020-12-05T17:41:34.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.034+0000 7efc02362700 1 --2- 172.21.15.32:0/2103756791 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7efbdc0554e0 0x7efbdc0579a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.034+0000 7efc02362700 1 --2- 172.21.15.32:0/2103756791 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efbfc139540 0x7efbfc13aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.034+0000 7efc02362700 1 --2- 172.21.15.32:0/2103756791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbfc0b2070 0x7efbfc13c150 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.034+0000 7efc02362700 1 --2- 172.21.15.32:0/2103756791 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efbfc0b2ca0 0x7efbfc13c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.034+0000 7efc02362700 1 -- 172.21.15.32:0/2103756791 >> 172.21.15.32:0/2103756791 conn(0x7efbfc05bbb0 msgr2=0x7efbfc0aded0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:34.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.034+0000 7efc02362700 1 -- 172.21.15.32:0/2103756791 shutdown_connections 2020-12-05T17:41:34.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.034+0000 7efc02362700 1 -- 172.21.15.32:0/2103756791 wait complete. 2020-12-05T17:41:34.051 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:41:34.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.170+0000 7f086359e700 1 Processor -- start 2020-12-05T17:41:34.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.170+0000 7f086359e700 1 -- start start 2020-12-05T17:41:34.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.170+0000 7f086359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08640611b0 0x7f0864153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:34.179 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.170+0000 7f086359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0864080a90 con 0x7f08640ae860 2020-12-05T17:41:34.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.170+0000 7f086359e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08640aec40 con 0x7f08640b0240 2020-12-05T17:41:34.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.170+0000 7f086359e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0864152860 con 0x7f08640611b0 2020-12-05T17:41:34.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.170+0000 7f0868c39700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f08640ae860 0x7f086414f0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43610/0 (socket says 172.21.15.32:43610) 2020-12-05T17:41:34.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.170+0000 7f0868c39700 1 -- 172.21.15.32:0/1382195951 learned_addr learned my addr 172.21.15.32:0/1382195951 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:34.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.170+0000 7f0862d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08640611b0 0x7f0864153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:34.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.170+0000 7f0861d9b700 1 -- 172.21.15.32:0/1382195951 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 729291391 0 0) 0x7f0864080a90 con 0x7f08640ae860 2020-12-05T17:41:34.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.170+0000 7f0862d9d700 1 -- 172.21.15.32:0/1382195951 >> v1:172.21.15.61:6789/0 conn(0x7f08640b0240 legacy=0x7f086414c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:34.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.170+0000 7f0862d9d700 1 -- 172.21.15.32:0/1382195951 >> v1:172.21.15.32:6789/0 conn(0x7f08640ae860 legacy=0x7f086414f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:34.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.170+0000 7f0862d9d700 1 -- 172.21.15.32:0/1382195951 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0864152b00 con 0x7f08640611b0 2020-12-05T17:41:34.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.170+0000 7f0862d9d700 1 --2- 172.21.15.32:0/1382195951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08640611b0 0x7f0864153240 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 rx=0x7f0854002f30 tx=0x7f0854006dc0).ready entity=mon.0 client_cookie=cdd8498eae2c9253 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:34.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.170+0000 7f0861d9b700 1 -- 172.21.15.32:0/1382195951 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08540097f0 con 0x7f08640611b0 2020-12-05T17:41:34.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.170+0000 7f0861d9b700 1 -- 172.21.15.32:0/1382195951 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0854009950 con 0x7f08640611b0 2020-12-05T17:41:34.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.170+0000 7f0861d9b700 1 -- 172.21.15.32:0/1382195951 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0854009cb0 con 0x7f08640611b0 2020-12-05T17:41:34.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.170+0000 7f086359e700 1 -- 172.21.15.32:0/1382195951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08640611b0 msgr2=0x7f0864153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:34.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.170+0000 7f086359e700 1 --2- 172.21.15.32:0/1382195951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08640611b0 0x7f0864153240 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 rx=0x7f0854002f30 tx=0x7f0854006dc0).stop 2020-12-05T17:41:34.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.174+0000 7f086359e700 1 -- 172.21.15.32:0/1382195951 shutdown_connections 2020-12-05T17:41:34.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.174+0000 7f086359e700 1 --2- 172.21.15.32:0/1382195951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08640611b0 0x7f0864153240 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.174+0000 7f086359e700 1 -- 172.21.15.32:0/1382195951 >> 172.21.15.32:0/1382195951 conn(0x7f08640b5f90 msgr2=0x7f08640b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:34.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.174+0000 7f086359e700 1 -- 172.21.15.32:0/1382195951 shutdown_connections 2020-12-05T17:41:34.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.174+0000 7f086359e700 1 -- 172.21.15.32:0/1382195951 wait complete. 2020-12-05T17:41:34.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.174+0000 7f086359e700 1 Processor -- start 2020-12-05T17:41:34.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.174+0000 7f086359e700 1 -- start start 2020-12-05T17:41:34.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f086359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08640611b0 0x7f086412c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:34.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f086359e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08640ae860 0x7f08640b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:34.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f086359e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f08640b0240 0x7f08640637e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:34.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f086359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0864129a10 con 0x7f08640611b0 2020-12-05T17:41:34.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f086359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08640b34c0 con 0x7f08640b0240 2020-12-05T17:41:34.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f086359e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08640b21b0 con 0x7f08640ae860 2020-12-05T17:41:34.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f0862d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08640611b0 0x7f086412c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:34.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f0862d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08640611b0 0x7f086412c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40968/0 (socket says 172.21.15.32:40968) 2020-12-05T17:41:34.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f0862d9d700 1 -- 172.21.15.32:0/2917127683 learned_addr learned my addr 172.21.15.32:0/2917127683 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:34.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f086259c700 1 --2- 172.21.15.32:0/2917127683 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08640ae860 0x7f08640b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:34.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f0868c39700 1 --2- 172.21.15.32:0/2917127683 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f08640b0240 0x7f08640637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:34.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f0862d9d700 1 -- 172.21.15.32:0/2917127683 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08640ae860 msgr2=0x7f08640b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:34.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f0862d9d700 1 --2- 172.21.15.32:0/2917127683 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08640ae860 0x7f08640b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f0862d9d700 1 -- 172.21.15.32:0/2917127683 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f08640b0240 msgr2=0x7f08640637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:34.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f0862d9d700 1 --2- 172.21.15.32:0/2917127683 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f08640b0240 0x7f08640637e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f0862d9d700 1 -- 172.21.15.32:0/2917127683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08640b1060 con 0x7f08640611b0 2020-12-05T17:41:34.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f0868c39700 1 --2- 172.21.15.32:0/2917127683 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f08640b0240 0x7f08640637e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:34.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f0862d9d700 1 --2- 172.21.15.32:0/2917127683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08640611b0 0x7f086412c160 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 rx=0x7f086412c6d0 tx=0x7f0854015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:34.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f08537fe700 1 -- 172.21.15.32:0/2917127683 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0854021070 con 0x7f08640611b0 2020-12-05T17:41:34.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f08537fe700 1 -- 172.21.15.32:0/2917127683 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085401d450 con 0x7f08640611b0 2020-12-05T17:41:34.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f086359e700 1 -- 172.21.15.32:0/2917127683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08640b23e0 con 0x7f08640611b0 2020-12-05T17:41:34.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f08537fe700 1 -- 172.21.15.32:0/2917127683 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f085401d7b0 con 0x7f08640611b0 2020-12-05T17:41:34.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f086359e700 1 -- 172.21.15.32:0/2917127683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08640b2d40 con 0x7f08640611b0 2020-12-05T17:41:34.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.178+0000 7f08537fe700 1 -- 172.21.15.32:0/2917127683 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f085400f070 con 0x7f08640611b0 2020-12-05T17:41:34.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.182+0000 7f08537fe700 1 --2- 172.21.15.32:0/2917127683 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0848055350 0x7f0848057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:34.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.182+0000 7f08537fe700 1 -- 172.21.15.32:0/2917127683 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f085406f2e0 con 0x7f08640611b0 2020-12-05T17:41:34.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.182+0000 7f086359e700 1 -- 172.21.15.32:0/2917127683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08640b2860 con 0x7f08640611b0 2020-12-05T17:41:34.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.182+0000 7f086259c700 1 --2- 172.21.15.32:0/2917127683 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0848055350 0x7f0848057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:34.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.182+0000 7f086259c700 1 --2- 172.21.15.32:0/2917127683 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0848055350 0x7f0848057810 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f085800d000 tx=0x7f0858007b60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:34.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.186+0000 7f08537fe700 1 -- 172.21.15.32:0/2917127683 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f08640b2860 con 0x7f08640611b0 2020-12-05T17:41:34.383 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.378+0000 7f086359e700 1 -- 172.21.15.32:0/2917127683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f08640b2860 con 0x7f08640611b0 2020-12-05T17:41:34.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.386+0000 7f08537fe700 1 -- 172.21.15.32:0/2917127683 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v41) v1 ==== 184+0+0 (secure 0 0 0) 0x7f08640b2860 con 0x7f08640611b0 2020-12-05T17:41:34.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.386+0000 7f086359e700 1 -- 172.21.15.32:0/2917127683 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0848055350 msgr2=0x7f0848057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:34.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.386+0000 7f086359e700 1 --2- 172.21.15.32:0/2917127683 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0848055350 0x7f0848057810 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f085800d000 tx=0x7f0858007b60).stop 2020-12-05T17:41:34.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.386+0000 7f086359e700 1 -- 172.21.15.32:0/2917127683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08640611b0 msgr2=0x7f086412c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:34.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.386+0000 7f086359e700 1 --2- 172.21.15.32:0/2917127683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08640611b0 0x7f086412c160 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 rx=0x7f086412c6d0 tx=0x7f0854015000).stop 2020-12-05T17:41:34.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.386+0000 7f086359e700 1 -- 172.21.15.32:0/2917127683 shutdown_connections 2020-12-05T17:41:34.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.386+0000 7f086359e700 1 --2- 172.21.15.32:0/2917127683 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f08640b0240 0x7f08640637e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.386+0000 7f086359e700 1 --2- 172.21.15.32:0/2917127683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08640611b0 0x7f086412c160 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.386+0000 7f086359e700 1 --2- 172.21.15.32:0/2917127683 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0848055350 0x7f0848057810 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.386+0000 7f086359e700 1 --2- 172.21.15.32:0/2917127683 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08640ae860 0x7f08640b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.386+0000 7f086359e700 1 -- 172.21.15.32:0/2917127683 >> 172.21.15.32:0/2917127683 conn(0x7f08640b5f90 msgr2=0x7f086412d630 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:34.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.386+0000 7f086359e700 1 -- 172.21.15.32:0/2917127683 shutdown_connections 2020-12-05T17:41:34.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.386+0000 7f086359e700 1 -- 172.21.15.32:0/2917127683 wait complete. 2020-12-05T17:41:34.396 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2020-12-05T17:41:34.407 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:41:34.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.530+0000 7f1ebf59e700 1 Processor -- start 2020-12-05T17:41:34.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.530+0000 7f1ebf59e700 1 -- start start 2020-12-05T17:41:34.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.530+0000 7f1ebf59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ec00611b0 0x7f1ec0153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:34.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.530+0000 7f1ebf59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ec00b0240 0x7f1ec00ae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:34.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.530+0000 7f1ebf59e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1ec014c340 0x7f1ec014c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:34.540 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.530+0000 7f1ebf59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec0080a90 con 0x7f1ec00611b0 2020-12-05T17:41:34.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.530+0000 7f1ebf59e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec0152860 con 0x7f1ec014c340 2020-12-05T17:41:34.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.530+0000 7f1ebf59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec0152b00 con 0x7f1ec00b0240 2020-12-05T17:41:34.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.530+0000 7f1ebed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ec00611b0 0x7f1ec0153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:34.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.530+0000 7f1ebed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ec00611b0 0x7f1ec0153240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40976/0 (socket says 172.21.15.32:40976) 2020-12-05T17:41:34.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.530+0000 7f1ebed9d700 1 -- 172.21.15.32:0/11498499 learned_addr learned my addr 172.21.15.32:0/11498499 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:34.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.530+0000 7f1eb659c700 1 --2- 172.21.15.32:0/11498499 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ec00b0240 0x7f1ec00ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:34.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.530+0000 7f1ec4d65700 1 --2- 172.21.15.32:0/11498499 >> v2:172.21.15.61:3300/0 conn(0x7f1ec014c340 0x7f1ec014c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:34.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.530+0000 7f1ebed9d700 1 -- 172.21.15.32:0/11498499 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ec00b0240 msgr2=0x7f1ec00ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:34.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.530+0000 7f1ebed9d700 1 --2- 172.21.15.32:0/11498499 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ec00b0240 0x7f1ec00ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.530+0000 7f1ebed9d700 1 -- 172.21.15.32:0/11498499 >> v2:172.21.15.61:3300/0 conn(0x7f1ec014c340 msgr2=0x7f1ec014c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:34.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.530+0000 7f1ebed9d700 1 --2- 172.21.15.32:0/11498499 >> v2:172.21.15.61:3300/0 conn(0x7f1ec014c340 0x7f1ec014c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.530+0000 7f1ebed9d700 1 -- 172.21.15.32:0/11498499 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec0152da0 con 0x7f1ec00611b0 2020-12-05T17:41:34.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.530+0000 7f1eb659c700 1 --2- 172.21.15.32:0/11498499 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ec00b0240 0x7f1ec00ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:34.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.530+0000 7f1ebed9d700 1 --2- 172.21.15.32:0/11498499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ec00611b0 0x7f1ec0153240 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7f1eb0002f30 tx=0x7f1eb0006e80).ready entity=mon.0 client_cookie=51317b764a7cada3 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:34.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.534+0000 7f1ebe59c700 1 -- 172.21.15.32:0/11498499 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1eb00097f0 con 0x7f1ec00611b0 2020-12-05T17:41:34.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.534+0000 7f1ebe59c700 1 -- 172.21.15.32:0/11498499 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb0009950 con 0x7f1ec00611b0 2020-12-05T17:41:34.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.534+0000 7f1ebe59c700 1 -- 172.21.15.32:0/11498499 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1eb0009cb0 con 0x7f1ec00611b0 2020-12-05T17:41:34.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.534+0000 7f1ebf59e700 1 -- 172.21.15.32:0/11498499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ec00611b0 msgr2=0x7f1ec0153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:34.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.534+0000 7f1ebf59e700 1 --2- 172.21.15.32:0/11498499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ec00611b0 0x7f1ec0153240 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7f1eb0002f30 tx=0x7f1eb0006e80).stop 2020-12-05T17:41:34.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.534+0000 7f1ebf59e700 1 -- 172.21.15.32:0/11498499 shutdown_connections 2020-12-05T17:41:34.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.534+0000 7f1ebf59e700 1 --2- 172.21.15.32:0/11498499 >> v2:172.21.15.61:3300/0 conn(0x7f1ec014c340 0x7f1ec014c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.534+0000 7f1ebf59e700 1 --2- 172.21.15.32:0/11498499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ec00611b0 0x7f1ec0153240 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.534+0000 7f1ebf59e700 1 --2- 172.21.15.32:0/11498499 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ec00b0240 0x7f1ec00ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.534+0000 7f1ebf59e700 1 -- 172.21.15.32:0/11498499 >> 172.21.15.32:0/11498499 conn(0x7f1ec00b5f90 msgr2=0x7f1ec00b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:34.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.534+0000 7f1ebf59e700 1 -- 172.21.15.32:0/11498499 shutdown_connections 2020-12-05T17:41:34.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.534+0000 7f1ebf59e700 1 -- 172.21.15.32:0/11498499 wait complete. 2020-12-05T17:41:34.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.534+0000 7f1ebf59e700 1 Processor -- start 2020-12-05T17:41:34.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.538+0000 7f1ebf59e700 1 -- start start 2020-12-05T17:41:34.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.538+0000 7f1ebf59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ec00611b0 0x7f1ec0063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:34.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.538+0000 7f1ebf59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ec00b0240 0x7f1ec00654f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:34.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.538+0000 7f1ebf59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ec014c340 0x7f1ec00659e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:34.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.538+0000 7f1ebf59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec0067250 con 0x7f1ec00b0240 2020-12-05T17:41:34.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.538+0000 7f1ebf59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec00640a0 con 0x7f1ec00611b0 2020-12-05T17:41:34.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.538+0000 7f1ebf59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec0066fb0 con 0x7f1ec014c340 2020-12-05T17:41:34.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.538+0000 7f1eb659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ec00b0240 0x7f1ec00654f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:34.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.538+0000 7f1eb659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ec00b0240 0x7f1ec00654f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40982/0 (socket says 172.21.15.32:40982) 2020-12-05T17:41:34.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.538+0000 7f1eb659c700 1 -- 172.21.15.32:0/1139119016 learned_addr learned my addr 172.21.15.32:0/1139119016 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:34.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.538+0000 7f1ebed9d700 1 --2- 172.21.15.32:0/1139119016 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ec00611b0 0x7f1ec0063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:34.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.538+0000 7f1ec4d65700 1 --2- 172.21.15.32:0/1139119016 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ec014c340 0x7f1ec00659e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:34.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.538+0000 7f1eb659c700 1 -- 172.21.15.32:0/1139119016 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ec014c340 msgr2=0x7f1ec00659e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:34.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.538+0000 7f1eb659c700 1 --2- 172.21.15.32:0/1139119016 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ec014c340 0x7f1ec00659e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.538+0000 7f1eb659c700 1 -- 172.21.15.32:0/1139119016 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ec00611b0 msgr2=0x7f1ec0063800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:34.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.538+0000 7f1eb659c700 1 --2- 172.21.15.32:0/1139119016 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ec00611b0 0x7f1ec0063800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.538+0000 7f1eb659c700 1 -- 172.21.15.32:0/1139119016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec014efa0 con 0x7f1ec00b0240 2020-12-05T17:41:34.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.538+0000 7f1eb659c700 1 --2- 172.21.15.32:0/1139119016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ec00b0240 0x7f1ec00654f0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x7f1ea8006910 tx=0x7f1ea8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:34.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.538+0000 7f1eb7fff700 1 -- 172.21.15.32:0/1139119016 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ea80127e0 con 0x7f1ec00b0240 2020-12-05T17:41:34.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.538+0000 7f1eb7fff700 1 -- 172.21.15.32:0/1139119016 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea8012940 con 0x7f1ec00b0240 2020-12-05T17:41:34.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.538+0000 7f1eb7fff700 1 -- 172.21.15.32:0/1139119016 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ea8012ca0 con 0x7f1ec00b0240 2020-12-05T17:41:34.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.538+0000 7f1ebf59e700 1 -- 172.21.15.32:0/1139119016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec0063cf0 con 0x7f1ec00b0240 2020-12-05T17:41:34.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.538+0000 7f1ebf59e700 1 -- 172.21.15.32:0/1139119016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec0150020 con 0x7f1ec00b0240 2020-12-05T17:41:34.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.542+0000 7f1ebf59e700 1 -- 172.21.15.32:0/1139119016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec0057050 con 0x7f1ec00b0240 2020-12-05T17:41:34.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.542+0000 7f1eb7fff700 1 -- 172.21.15.32:0/1139119016 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1ea8013480 con 0x7f1ec00b0240 2020-12-05T17:41:34.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.542+0000 7f1eb7fff700 1 --2- 172.21.15.32:0/1139119016 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ea40553b0 0x7f1ea4057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:34.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.542+0000 7f1eb7fff700 1 -- 172.21.15.32:0/1139119016 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1ea806e820 con 0x7f1ec00b0240 2020-12-05T17:41:34.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.542+0000 7f1ebed9d700 1 --2- 172.21.15.32:0/1139119016 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ea40553b0 0x7f1ea4057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:34.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.542+0000 7f1ebed9d700 1 --2- 172.21.15.32:0/1139119016 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ea40553b0 0x7f1ea4057870 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f1eb001c000 tx=0x7f1eb0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:34.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.550+0000 7f1eb7fff700 1 -- 172.21.15.32:0/1139119016 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1ea80448c0 con 0x7f1ec00b0240 2020-12-05T17:41:34.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.742+0000 7f1ebf59e700 1 -- 172.21.15.32:0/1139119016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f1ec00b6da0 con 0x7f1ec00b0240 2020-12-05T17:41:34.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.750+0000 7f1eb7fff700 1 -- 172.21.15.32:0/1139119016 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v42) v1 ==== 184+0+0 (secure 0 0 0) 0x7f1ea8012e00 con 0x7f1ec00b0240 2020-12-05T17:41:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.750+0000 7f1ebf59e700 1 -- 172.21.15.32:0/1139119016 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ea40553b0 msgr2=0x7f1ea4057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.750+0000 7f1ebf59e700 1 --2- 172.21.15.32:0/1139119016 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ea40553b0 0x7f1ea4057870 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f1eb001c000 tx=0x7f1eb0015000).stop 2020-12-05T17:41:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.750+0000 7f1ebf59e700 1 -- 172.21.15.32:0/1139119016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ec00b0240 msgr2=0x7f1ec00654f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.750+0000 7f1ebf59e700 1 --2- 172.21.15.32:0/1139119016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ec00b0240 0x7f1ec00654f0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x7f1ea8006910 tx=0x7f1ea8008040).stop 2020-12-05T17:41:34.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.750+0000 7f1ebf59e700 1 -- 172.21.15.32:0/1139119016 shutdown_connections 2020-12-05T17:41:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.750+0000 7f1ebf59e700 1 --2- 172.21.15.32:0/1139119016 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1ea40553b0 0x7f1ea4057870 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.754+0000 7f1ebf59e700 1 --2- 172.21.15.32:0/1139119016 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ec014c340 0x7f1ec00659e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.754+0000 7f1ebf59e700 1 --2- 172.21.15.32:0/1139119016 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ec00611b0 0x7f1ec0063800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.754+0000 7f1ebf59e700 1 --2- 172.21.15.32:0/1139119016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ec00b0240 0x7f1ec00654f0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.754+0000 7f1ebf59e700 1 -- 172.21.15.32:0/1139119016 >> 172.21.15.32:0/1139119016 conn(0x7f1ec00b5f90 msgr2=0x7f1ec0150eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.754+0000 7f1ebf59e700 1 -- 172.21.15.32:0/1139119016 shutdown_connections 2020-12-05T17:41:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.754+0000 7f1ebf59e700 1 -- 172.21.15.32:0/1139119016 wait complete. 2020-12-05T17:41:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2020-12-05T17:41:34.777 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:41:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f07a5c3c700 1 Processor -- start 2020-12-05T17:41:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f07a5c3c700 1 -- start start 2020-12-05T17:41:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f07a5c3c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f07a00611b0 0x7f07a0153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:34.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f07a5c3c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07a00b0240 0x7f07a00ae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:34.908 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f07a5c3c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07a014c340 0x7f07a014c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:34.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f07a5c3c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a0080a90 con 0x7f07a014c340 2020-12-05T17:41:34.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f07a5c3c700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a0152860 con 0x7f07a00611b0 2020-12-05T17:41:34.909 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f07a5c3c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a0152b00 con 0x7f07a00b0240 2020-12-05T17:41:34.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f07a6e9e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07a014c340 0x7f07a014c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:34.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f07a6e9e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07a014c340 0x7f07a014c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40992/0 (socket says 172.21.15.32:40992) 2020-12-05T17:41:34.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f07a6e9e700 1 -- 172.21.15.32:0/2681085917 learned_addr learned my addr 172.21.15.32:0/2681085917 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:34.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f07a543b700 1 --2- 172.21.15.32:0/2681085917 >> v2:172.21.15.61:3300/0 conn(0x7f07a00611b0 0x7f07a0153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:34.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f07a4c3a700 1 --2- 172.21.15.32:0/2681085917 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07a00b0240 0x7f07a00ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:34.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f07a6e9e700 1 -- 172.21.15.32:0/2681085917 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07a00b0240 msgr2=0x7f07a00ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:34.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f07a6e9e700 1 --2- 172.21.15.32:0/2681085917 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07a00b0240 0x7f07a00ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f07a6e9e700 1 -- 172.21.15.32:0/2681085917 >> v2:172.21.15.61:3300/0 conn(0x7f07a00611b0 msgr2=0x7f07a0153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:34.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f07a6e9e700 1 --2- 172.21.15.32:0/2681085917 >> v2:172.21.15.61:3300/0 conn(0x7f07a00611b0 0x7f07a0153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f07a6e9e700 1 -- 172.21.15.32:0/2681085917 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a0152da0 con 0x7f07a014c340 2020-12-05T17:41:34.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f07a6e9e700 1 --2- 172.21.15.32:0/2681085917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07a014c340 0x7f07a014c790 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f07980032b0 tx=0x7f0798009270).ready entity=mon.0 client_cookie=b3901faea7e23951 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:34.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f0797fff700 1 -- 172.21.15.32:0/2681085917 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0798009e60 con 0x7f07a014c340 2020-12-05T17:41:34.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f0797fff700 1 -- 172.21.15.32:0/2681085917 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079800bb50 con 0x7f07a014c340 2020-12-05T17:41:34.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f0797fff700 1 -- 172.21.15.32:0/2681085917 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f079800bcc0 con 0x7f07a014c340 2020-12-05T17:41:34.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f07a5c3c700 1 -- 172.21.15.32:0/2681085917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07a014c340 msgr2=0x7f07a014c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:34.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.898+0000 7f07a5c3c700 1 --2- 172.21.15.32:0/2681085917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07a014c340 0x7f07a014c790 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f07980032b0 tx=0x7f0798009270).stop 2020-12-05T17:41:34.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.902+0000 7f07a5c3c700 1 -- 172.21.15.32:0/2681085917 shutdown_connections 2020-12-05T17:41:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.902+0000 7f07a5c3c700 1 --2- 172.21.15.32:0/2681085917 >> v2:172.21.15.61:3300/0 conn(0x7f07a00611b0 0x7f07a0153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.902+0000 7f07a5c3c700 1 --2- 172.21.15.32:0/2681085917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07a014c340 0x7f07a014c790 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.902+0000 7f07a5c3c700 1 --2- 172.21.15.32:0/2681085917 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07a00b0240 0x7f07a00ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.902+0000 7f07a5c3c700 1 -- 172.21.15.32:0/2681085917 >> 172.21.15.32:0/2681085917 conn(0x7f07a00b5f90 msgr2=0x7f07a00b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:34.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.902+0000 7f07a5c3c700 1 -- 172.21.15.32:0/2681085917 shutdown_connections 2020-12-05T17:41:34.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.902+0000 7f07a5c3c700 1 -- 172.21.15.32:0/2681085917 wait complete. 2020-12-05T17:41:34.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.902+0000 7f07a5c3c700 1 Processor -- start 2020-12-05T17:41:34.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.902+0000 7f07a5c3c700 1 -- start start 2020-12-05T17:41:34.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.906+0000 7f07a5c3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07a00611b0 0x7f07a012d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:34.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.906+0000 7f07a5c3c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07a00b0240 0x7f07a0138b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:34.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.906+0000 7f07a5c3c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07a014c340 0x7f07a014f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:34.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.906+0000 7f07a5c3c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a014e080 con 0x7f07a014c340 2020-12-05T17:41:34.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.906+0000 7f07a5c3c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a014f400 con 0x7f07a00611b0 2020-12-05T17:41:34.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.906+0000 7f07a5c3c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00b2930 con 0x7f07a00b0240 2020-12-05T17:41:34.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.906+0000 7f07a6e9e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07a014c340 0x7f07a014f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.906+0000 7f07a6e9e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07a014c340 0x7f07a014f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40998/0 (socket says 172.21.15.32:40998) 2020-12-05T17:41:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.906+0000 7f07a543b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07a00611b0 0x7f07a012d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.906+0000 7f07a4c3a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07a00b0240 0x7f07a0138b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.906+0000 7f07a6e9e700 1 -- 172.21.15.32:0/4038418484 learned_addr learned my addr 172.21.15.32:0/4038418484 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.906+0000 7f07a6e9e700 1 -- 172.21.15.32:0/4038418484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07a00b0240 msgr2=0x7f07a0138b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:34.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.906+0000 7f07a6e9e700 1 --2- 172.21.15.32:0/4038418484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07a00b0240 0x7f07a0138b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.906+0000 7f07a6e9e700 1 -- 172.21.15.32:0/4038418484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07a00611b0 msgr2=0x7f07a012d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:34.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.906+0000 7f07a6e9e700 1 --2- 172.21.15.32:0/4038418484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07a00611b0 0x7f07a012d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:34.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.906+0000 7f07a6e9e700 1 -- 172.21.15.32:0/4038418484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a00b2bd0 con 0x7f07a014c340 2020-12-05T17:41:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.906+0000 7f07a6e9e700 1 --2- 172.21.15.32:0/4038418484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07a014c340 0x7f07a014f860 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7f07980032b0 tx=0x7f0798016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.906+0000 7f0795ffb700 1 -- 172.21.15.32:0/4038418484 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f079801f3f0 con 0x7f07a014c340 2020-12-05T17:41:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.906+0000 7f0795ffb700 1 -- 172.21.15.32:0/4038418484 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079801f550 con 0x7f07a014c340 2020-12-05T17:41:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.906+0000 7f07a5c3c700 1 -- 172.21.15.32:0/4038418484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a00b2620 con 0x7f07a014c340 2020-12-05T17:41:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.906+0000 7f0795ffb700 1 -- 172.21.15.32:0/4038418484 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f079801f8b0 con 0x7f07a014c340 2020-12-05T17:41:34.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.906+0000 7f07a5c3c700 1 -- 172.21.15.32:0/4038418484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a014d0b0 con 0x7f07a014c340 2020-12-05T17:41:34.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.910+0000 7f07a5c3c700 1 -- 172.21.15.32:0/4038418484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a0057050 con 0x7f07a014c340 2020-12-05T17:41:34.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.910+0000 7f0795ffb700 1 -- 172.21.15.32:0/4038418484 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0798026030 con 0x7f07a014c340 2020-12-05T17:41:34.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.910+0000 7f0795ffb700 1 --2- 172.21.15.32:0/4038418484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f07800553b0 0x7f0780057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:34.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.910+0000 7f0795ffb700 1 -- 172.21.15.32:0/4038418484 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f079806ec90 con 0x7f07a014c340 2020-12-05T17:41:34.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.910+0000 7f07a543b700 1 --2- 172.21.15.32:0/4038418484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f07800553b0 0x7f0780057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:34.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.910+0000 7f07a543b700 1 --2- 172.21.15.32:0/4038418484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f07800553b0 0x7f0780057870 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f079c00b000 tx=0x7f079c006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:34.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:34.922+0000 7f0795ffb700 1 -- 172.21.15.32:0/4038418484 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0798044d90 con 0x7f07a014c340 2020-12-05T17:41:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.110+0000 7f07a5c3c700 1 -- 172.21.15.32:0/4038418484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f07a014f570 con 0x7f07a014c340 2020-12-05T17:41:35.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.118+0000 7f0795ffb700 1 -- 172.21.15.32:0/4038418484 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v43) v1 ==== 184+0+0 (secure 0 0 0) 0x7f07980150e0 con 0x7f07a014c340 2020-12-05T17:41:35.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.118+0000 7f07a5c3c700 1 -- 172.21.15.32:0/4038418484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f07800553b0 msgr2=0x7f0780057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:35.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.118+0000 7f07a5c3c700 1 --2- 172.21.15.32:0/4038418484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f07800553b0 0x7f0780057870 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f079c00b000 tx=0x7f079c006dc0).stop 2020-12-05T17:41:35.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.118+0000 7f07a5c3c700 1 -- 172.21.15.32:0/4038418484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07a014c340 msgr2=0x7f07a014f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:35.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.118+0000 7f07a5c3c700 1 --2- 172.21.15.32:0/4038418484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07a014c340 0x7f07a014f860 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7f07980032b0 tx=0x7f0798016000).stop 2020-12-05T17:41:35.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.118+0000 7f07a5c3c700 1 -- 172.21.15.32:0/4038418484 shutdown_connections 2020-12-05T17:41:35.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.118+0000 7f07a5c3c700 1 --2- 172.21.15.32:0/4038418484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07a014c340 0x7f07a014f860 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.118+0000 7f07a5c3c700 1 --2- 172.21.15.32:0/4038418484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07a00611b0 0x7f07a012d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.118+0000 7f07a5c3c700 1 --2- 172.21.15.32:0/4038418484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f07800553b0 0x7f0780057870 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.118+0000 7f07a5c3c700 1 --2- 172.21.15.32:0/4038418484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07a00b0240 0x7f07a0138b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.118+0000 7f07a5c3c700 1 -- 172.21.15.32:0/4038418484 >> 172.21.15.32:0/4038418484 conn(0x7f07a00b5f90 msgr2=0x7f07a0061590 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.118+0000 7f07a5c3c700 1 -- 172.21.15.32:0/4038418484 shutdown_connections 2020-12-05T17:41:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.118+0000 7f07a5c3c700 1 -- 172.21.15.32:0/4038418484 wait complete. 2020-12-05T17:41:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.2 2020-12-05T17:41:35.137 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:41:35.245 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.238+0000 7f8a0f59e700 1 Processor -- start 2020-12-05T17:41:35.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0f59e700 1 -- start start 2020-12-05T17:41:35.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0f59e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f8a1005f3b0 0x7f8a1005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:35.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a1005feb0 0x7f8a100b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:35.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0f59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a10142450 con 0x7f8a10062400 2020-12-05T17:41:35.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0f59e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a100b42f0 con 0x7f8a1005f3b0 2020-12-05T17:41:35.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a10061780 con 0x7f8a1005feb0 2020-12-05T17:41:35.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0ed9d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a1005feb0 0x7f8a100b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:35.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0e59c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f8a1005f3b0 0x7f8a1005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:35.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0ed9d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a1005feb0 0x7f8a100b3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:37796/0 (socket says 172.21.15.32:37796) 2020-12-05T17:41:35.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0dd9b700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f8a10062400 0x7f8a100b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:56290/0 (socket says 172.21.15.32:56290) 2020-12-05T17:41:35.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0ed9d700 1 -- 172.21.15.32:0/979578114 learned_addr learned my addr 172.21.15.32:0/979578114 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:35.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0ed9d700 1 -- 172.21.15.32:0/979578114 >> v2:172.21.15.61:3300/0 conn(0x7f8a1005f3b0 msgr2=0x7f8a1005f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:35.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0ed9d700 1 --2- 172.21.15.32:0/979578114 >> v2:172.21.15.61:3300/0 conn(0x7f8a1005f3b0 0x7f8a1005f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:35.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0ed9d700 1 -- 172.21.15.32:0/979578114 >> v1:172.21.15.61:6789/0 conn(0x7f8a10062400 legacy=0x7f8a100b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:35.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0ed9d700 1 -- 172.21.15.32:0/979578114 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a10061a20 con 0x7f8a1005feb0 2020-12-05T17:41:35.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0e59c700 1 --2- 172.21.15.32:0/979578114 >> v2:172.21.15.61:3300/0 conn(0x7f8a1005f3b0 0x7f8a1005f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:35.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0ed9d700 1 --2- 172.21.15.32:0/979578114 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a1005feb0 0x7f8a100b3e00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f8a080032b0 tx=0x7f8a08009270).ready entity=mon.2 client_cookie=69a950dbaf536693 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:35.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0d59a700 1 -- 172.21.15.32:0/979578114 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a08009e60 con 0x7f8a1005feb0 2020-12-05T17:41:35.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0d59a700 1 -- 172.21.15.32:0/979578114 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0800bc00 con 0x7f8a1005feb0 2020-12-05T17:41:35.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0d59a700 1 -- 172.21.15.32:0/979578114 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a0800bd70 con 0x7f8a1005feb0 2020-12-05T17:41:35.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0f59e700 1 -- 172.21.15.32:0/979578114 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a1005feb0 msgr2=0x7f8a100b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:35.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0f59e700 1 --2- 172.21.15.32:0/979578114 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a1005feb0 0x7f8a100b3e00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f8a080032b0 tx=0x7f8a08009270).stop 2020-12-05T17:41:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0f59e700 1 -- 172.21.15.32:0/979578114 shutdown_connections 2020-12-05T17:41:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0f59e700 1 --2- 172.21.15.32:0/979578114 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a1005feb0 0x7f8a100b3e00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0f59e700 1 --2- 172.21.15.32:0/979578114 >> v2:172.21.15.61:3300/0 conn(0x7f8a1005f3b0 0x7f8a1005f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:35.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0f59e700 1 -- 172.21.15.32:0/979578114 >> 172.21.15.32:0/979578114 conn(0x7f8a100b8ee0 msgr2=0x7f8a100b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:35.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0f59e700 1 -- 172.21.15.32:0/979578114 shutdown_connections 2020-12-05T17:41:35.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.242+0000 7f8a0f59e700 1 -- 172.21.15.32:0/979578114 wait complete. 2020-12-05T17:41:35.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.246+0000 7f8a0f59e700 1 Processor -- start 2020-12-05T17:41:35.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.246+0000 7f8a0f59e700 1 -- start start 2020-12-05T17:41:35.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.246+0000 7f8a0f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a1005f3b0 0x7f8a10063340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:35.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.246+0000 7f8a0f59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a1005feb0 0x7f8a100648f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:35.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.246+0000 7f8a0f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a10062400 0x7f8a10064de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:35.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.246+0000 7f8a0f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a10149ab0 con 0x7f8a10062400 2020-12-05T17:41:35.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.246+0000 7f8a0f59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a10145920 con 0x7f8a1005feb0 2020-12-05T17:41:35.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.246+0000 7f8a0f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a10145bc0 con 0x7f8a1005f3b0 2020-12-05T17:41:35.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.246+0000 7f8a0ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a10062400 0x7f8a10064de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:35.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.246+0000 7f8a0e59c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a1005f3b0 0x7f8a10063340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:35.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.246+0000 7f8a0dd9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a1005feb0 0x7f8a100648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:35.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.246+0000 7f8a0ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a10062400 0x7f8a10064de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41012/0 (socket says 172.21.15.32:41012) 2020-12-05T17:41:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.246+0000 7f8a0ed9d700 1 -- 172.21.15.32:0/1430538454 learned_addr learned my addr 172.21.15.32:0/1430538454 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.246+0000 7f8a0ed9d700 1 -- 172.21.15.32:0/1430538454 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a1005f3b0 msgr2=0x7f8a10063340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.246+0000 7f8a0ed9d700 1 --2- 172.21.15.32:0/1430538454 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a1005f3b0 0x7f8a10063340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.246+0000 7f8a0ed9d700 1 -- 172.21.15.32:0/1430538454 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a1005feb0 msgr2=0x7f8a100648f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.246+0000 7f8a0ed9d700 1 --2- 172.21.15.32:0/1430538454 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a1005feb0 0x7f8a100648f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:35.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.246+0000 7f8a0ed9d700 1 -- 172.21.15.32:0/1430538454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a10145e60 con 0x7f8a10062400 2020-12-05T17:41:35.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.246+0000 7f8a0e59c700 1 --2- 172.21.15.32:0/1430538454 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a1005f3b0 0x7f8a10063340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:35.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.250+0000 7f8a0ed9d700 1 --2- 172.21.15.32:0/1430538454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a10062400 0x7f8a10064de0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7f8a080032b0 tx=0x7f8a08016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:35.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.250+0000 7f89ff7fe700 1 -- 172.21.15.32:0/1430538454 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a0801e610 con 0x7f8a10062400 2020-12-05T17:41:35.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.250+0000 7f8a0f59e700 1 -- 172.21.15.32:0/1430538454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a10146300 con 0x7f8a10062400 2020-12-05T17:41:35.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.250+0000 7f89ff7fe700 1 -- 172.21.15.32:0/1430538454 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0801e770 con 0x7f8a10062400 2020-12-05T17:41:35.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.250+0000 7f89ff7fe700 1 -- 172.21.15.32:0/1430538454 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a0801ead0 con 0x7f8a10062400 2020-12-05T17:41:35.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.250+0000 7f8a0f59e700 1 -- 172.21.15.32:0/1430538454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a100654f0 con 0x7f8a10062400 2020-12-05T17:41:35.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.250+0000 7f89ff7fe700 1 -- 172.21.15.32:0/1430538454 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8a0800f070 con 0x7f8a10062400 2020-12-05T17:41:35.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.250+0000 7f8a0f59e700 1 -- 172.21.15.32:0/1430538454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a10145120 con 0x7f8a10062400 2020-12-05T17:41:35.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.250+0000 7f89ff7fe700 1 --2- 172.21.15.32:0/1430538454 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f89ec055330 0x7f89ec0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:35.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.250+0000 7f89ff7fe700 1 -- 172.21.15.32:0/1430538454 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8a0806f4f0 con 0x7f8a10062400 2020-12-05T17:41:35.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.250+0000 7f8a0e59c700 1 --2- 172.21.15.32:0/1430538454 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f89ec055330 0x7f89ec0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:35.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.254+0000 7f8a0e59c700 1 --2- 172.21.15.32:0/1430538454 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f89ec055330 0x7f89ec0577f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f8a0000b000 tx=0x7f8a00006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:35.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.258+0000 7f89ff7fe700 1 -- 172.21.15.32:0/1430538454 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8a080455f0 con 0x7f8a10062400 2020-12-05T17:41:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.446+0000 7f8a0f59e700 1 -- 172.21.15.32:0/1430538454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a10057050 con 0x7f8a10062400 2020-12-05T17:41:35.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.446+0000 7f89ff7fe700 1 -- 172.21.15.32:0/1430538454 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f8a08015090 con 0x7f8a10062400 2020-12-05T17:41:35.453 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:35.454 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":66,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/3049192140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3049192140},{"type":"v1","addr":"172.21.15.61:6833","nonce":3049192140}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:41:32.383702+0000","modified":"2020-12-05T17:41:33.387236+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8274":{"gid":8274,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/2808222868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2808222868},{"type":"v1","addr":"172.21.15.32:6835","nonce":2808222868}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2020-12-05T17:41:35.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.450+0000 7f8a0f59e700 1 -- 172.21.15.32:0/1430538454 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f89ec055330 msgr2=0x7f89ec0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:35.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.450+0000 7f8a0f59e700 1 --2- 172.21.15.32:0/1430538454 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f89ec055330 0x7f89ec0577f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f8a0000b000 tx=0x7f8a00006dc0).stop 2020-12-05T17:41:35.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.450+0000 7f8a0f59e700 1 -- 172.21.15.32:0/1430538454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a10062400 msgr2=0x7f8a10064de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:35.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.450+0000 7f8a0f59e700 1 --2- 172.21.15.32:0/1430538454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a10062400 0x7f8a10064de0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7f8a080032b0 tx=0x7f8a08016000).stop 2020-12-05T17:41:35.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.450+0000 7f8a0f59e700 1 -- 172.21.15.32:0/1430538454 shutdown_connections 2020-12-05T17:41:35.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.450+0000 7f8a0f59e700 1 --2- 172.21.15.32:0/1430538454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a10062400 0x7f8a10064de0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:35.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.450+0000 7f8a0f59e700 1 --2- 172.21.15.32:0/1430538454 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f89ec055330 0x7f89ec0577f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:35.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.450+0000 7f8a0f59e700 1 --2- 172.21.15.32:0/1430538454 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a1005f3b0 0x7f8a10063340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:35.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.450+0000 7f8a0f59e700 1 --2- 172.21.15.32:0/1430538454 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a1005feb0 0x7f8a100648f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:35.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.450+0000 7f8a0f59e700 1 -- 172.21.15.32:0/1430538454 >> 172.21.15.32:0/1430538454 conn(0x7f8a100b8ee0 msgr2=0x7f8a10142620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:35.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.450+0000 7f8a0f59e700 1 -- 172.21.15.32:0/1430538454 shutdown_connections 2020-12-05T17:41:35.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.450+0000 7f8a0f59e700 1 -- 172.21.15.32:0/1430538454 wait complete. 2020-12-05T17:41:35.458 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 66 2020-12-05T17:41:35.669 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:41:32.383702+0000', 'modified': '2020-12-05T17:41:33.387236+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8274}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8274': {'gid': 8274, 'name': 'a', 'rank': 0, 'incarnation': 65, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/2808222868', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 2808222868}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 2808222868}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:41:35.669 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:41:35.670 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:41:35.803 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":8274,"want_state":"up:active","state":"up:active","rank_uptime":3.4057907190000001,"mdsmap_epoch":66,"osdmap_epoch":138,"osdmap_epoch_barrier":138,"uptime":7.6255917819999999} 2020-12-05T17:41:35.803 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 8274, "mdsmap_epoch": 66, "osdmap_epoch": 138, "osdmap_epoch_barrier": 138, "rank_uptime": 3.405790719, "state": "up:active", "uptime": 7.625591782, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:41:35.804 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:41:35.914 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff952f8e700 1 Processor -- start 2020-12-05T17:41:35.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff952f8e700 1 -- start start 2020-12-05T17:41:35.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff952f8e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff94c0b2870 0x7ff94c0b0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:35.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff952f8e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff94c05f7f0 0x7ff94c146860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:35.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff952f8e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff94c142f50 0x7ff94c1433e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:35.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff952f8e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff94c1428c0 con 0x7ff94c05f7f0 2020-12-05T17:41:35.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff952f8e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff94c1438d0 con 0x7ff94c0b2870 2020-12-05T17:41:35.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff952f8e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c05fd00 con 0x7ff94c142f50 2020-12-05T17:41:35.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff94bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff94c05f7f0 0x7ff94c146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:35.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff94bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff94c05f7f0 0x7ff94c146860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41016/0 (socket says 172.21.15.32:41016) 2020-12-05T17:41:35.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff94bfff700 1 -- 172.21.15.32:0/948243886 learned_addr learned my addr 172.21.15.32:0/948243886 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:35.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff95152b700 1 --2- 172.21.15.32:0/948243886 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff94c142f50 0x7ff94c1433e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:35.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff950d2a700 1 --2- 172.21.15.32:0/948243886 >> v2:172.21.15.61:3300/0 conn(0x7ff94c0b2870 0x7ff94c0b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:35.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff94bfff700 1 -- 172.21.15.32:0/948243886 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff94c142f50 msgr2=0x7ff94c1433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:35.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff94bfff700 1 --2- 172.21.15.32:0/948243886 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff94c142f50 0x7ff94c1433e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:35.919 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff94bfff700 1 -- 172.21.15.32:0/948243886 >> v2:172.21.15.61:3300/0 conn(0x7ff94c0b2870 msgr2=0x7ff94c0b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:35.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff94bfff700 1 --2- 172.21.15.32:0/948243886 >> v2:172.21.15.61:3300/0 conn(0x7ff94c0b2870 0x7ff94c0b0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:35.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff94bfff700 1 -- 172.21.15.32:0/948243886 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94c142ab0 con 0x7ff94c05f7f0 2020-12-05T17:41:35.920 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff94bfff700 1 --2- 172.21.15.32:0/948243886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94c05f7f0 0x7ff94c146860 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0x7ff938002f30 tx=0x7ff938006e80).ready entity=mon.0 client_cookie=9a026883402fea08 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:35.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff94b7fe700 1 -- 172.21.15.32:0/948243886 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9380097f0 con 0x7ff94c05f7f0 2020-12-05T17:41:35.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff94b7fe700 1 -- 172.21.15.32:0/948243886 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff938009950 con 0x7ff94c05f7f0 2020-12-05T17:41:35.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff94b7fe700 1 -- 172.21.15.32:0/948243886 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff938009cb0 con 0x7ff94c05f7f0 2020-12-05T17:41:35.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff952f8e700 1 -- 172.21.15.32:0/948243886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94c05f7f0 msgr2=0x7ff94c146860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:35.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.910+0000 7ff952f8e700 1 --2- 172.21.15.32:0/948243886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94c05f7f0 0x7ff94c146860 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0x7ff938002f30 tx=0x7ff938006e80).stop 2020-12-05T17:41:35.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.914+0000 7ff952f8e700 1 -- 172.21.15.32:0/948243886 shutdown_connections 2020-12-05T17:41:35.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.914+0000 7ff952f8e700 1 --2- 172.21.15.32:0/948243886 >> v2:172.21.15.61:3300/0 conn(0x7ff94c0b2870 0x7ff94c0b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:35.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.914+0000 7ff952f8e700 1 --2- 172.21.15.32:0/948243886 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff94c142f50 0x7ff94c1433e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:35.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.914+0000 7ff952f8e700 1 --2- 172.21.15.32:0/948243886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94c05f7f0 0x7ff94c146860 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:35.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.914+0000 7ff952f8e700 1 -- 172.21.15.32:0/948243886 >> 172.21.15.32:0/948243886 conn(0x7ff94c0604c0 msgr2=0x7ff94c05c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:35.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.914+0000 7ff952f8e700 1 -- 172.21.15.32:0/948243886 shutdown_connections 2020-12-05T17:41:35.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.914+0000 7ff952f8e700 1 -- 172.21.15.32:0/948243886 wait complete. 2020-12-05T17:41:35.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.914+0000 7ff952f8e700 1 Processor -- start 2020-12-05T17:41:35.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.914+0000 7ff952f8e700 1 -- start start 2020-12-05T17:41:35.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.914+0000 7ff952f8e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff94c05f7f0 0x7ff94c065c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:35.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.914+0000 7ff952f8e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff94c0b2870 0x7ff94c0aec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:35.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.914+0000 7ff952f8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94c142f50 0x7ff94c062ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:35.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.914+0000 7ff952f8e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c065170 con 0x7ff94c142f50 2020-12-05T17:41:35.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.914+0000 7ff952f8e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c064ed0 con 0x7ff94c0b2870 2020-12-05T17:41:35.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.914+0000 7ff952f8e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c0b6020 con 0x7ff94c05f7f0 2020-12-05T17:41:35.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.914+0000 7ff95152b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94c142f50 0x7ff94c062ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:35.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.914+0000 7ff95152b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94c142f50 0x7ff94c062ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41026/0 (socket says 172.21.15.32:41026) 2020-12-05T17:41:35.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.914+0000 7ff95152b700 1 -- 172.21.15.32:0/1002592311 learned_addr learned my addr 172.21.15.32:0/1002592311 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:35.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.914+0000 7ff950d2a700 1 --2- 172.21.15.32:0/1002592311 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff94c05f7f0 0x7ff94c065c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:35.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.914+0000 7ff94bfff700 1 --2- 172.21.15.32:0/1002592311 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff94c0b2870 0x7ff94c0aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:35.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.918+0000 7ff95152b700 1 -- 172.21.15.32:0/1002592311 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff94c05f7f0 msgr2=0x7ff94c065c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:35.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.918+0000 7ff95152b700 1 --2- 172.21.15.32:0/1002592311 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff94c05f7f0 0x7ff94c065c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:35.929 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.918+0000 7ff95152b700 1 -- 172.21.15.32:0/1002592311 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff94c0b2870 msgr2=0x7ff94c0aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:35.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.918+0000 7ff95152b700 1 --2- 172.21.15.32:0/1002592311 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff94c0b2870 0x7ff94c0aec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:35.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.918+0000 7ff95152b700 1 -- 172.21.15.32:0/1002592311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94c0b49f0 con 0x7ff94c142f50 2020-12-05T17:41:35.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.918+0000 7ff94bfff700 1 --2- 172.21.15.32:0/1002592311 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff94c0b2870 0x7ff94c0aec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:35.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.918+0000 7ff95152b700 1 --2- 172.21.15.32:0/1002592311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94c142f50 0x7ff94c062ff0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7ff94400a000 tx=0x7ff944008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:35.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.918+0000 7ff949ffb700 1 -- 172.21.15.32:0/1002592311 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff94400ba70 con 0x7ff94c142f50 2020-12-05T17:41:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.918+0000 7ff949ffb700 1 -- 172.21.15.32:0/1002592311 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94400bbd0 con 0x7ff94c142f50 2020-12-05T17:41:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.918+0000 7ff952f8e700 1 -- 172.21.15.32:0/1002592311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff94c064a70 con 0x7ff94c142f50 2020-12-05T17:41:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.918+0000 7ff949ffb700 1 -- 172.21.15.32:0/1002592311 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff944012420 con 0x7ff94c142f50 2020-12-05T17:41:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.918+0000 7ff952f8e700 1 -- 172.21.15.32:0/1002592311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff94c063700 con 0x7ff94c142f50 2020-12-05T17:41:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.918+0000 7ff949ffb700 1 -- 172.21.15.32:0/1002592311 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff944012580 con 0x7ff94c142f50 2020-12-05T17:41:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.918+0000 7ff949ffb700 1 --2- 172.21.15.32:0/1002592311 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff9340554e0 0x7ff9340579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.918+0000 7ff950d2a700 1 --2- 172.21.15.32:0/1002592311 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff9340554e0 0x7ff9340579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.918+0000 7ff949ffb700 1 -- 172.21.15.32:0/1002592311 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff94406e7b0 con 0x7ff94c142f50 2020-12-05T17:41:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.918+0000 7ff952f8e700 1 -- 172.21.15.32:0/1002592311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff94c0b61e0 con 0x7ff94c142f50 2020-12-05T17:41:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.918+0000 7ff950d2a700 1 --2- 172.21.15.32:0/1002592311 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff9340554e0 0x7ff9340579a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7ff94000c000 tx=0x7ff940008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:35.926+0000 7ff949ffb700 1 -- 172.21.15.32:0/1002592311 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff9440449a0 con 0x7ff94c142f50 2020-12-05T17:41:36.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.070+0000 7ff952f8e700 1 -- 172.21.15.32:0/1002592311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff94c057050 con 0x7ff94c142f50 2020-12-05T17:41:36.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.070+0000 7ff949ffb700 1 -- 172.21.15.32:0/1002592311 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16285 (secure 0 0 0) 0x7ff944047fc0 con 0x7ff94c142f50 2020-12-05T17:41:36.076 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:36.077 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":138,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:41:32.383424+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:41:30.899960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2020-12-05T17:41:31.381413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:41:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.074+0000 7ff952f8e700 1 -- 172.21.15.32:0/1002592311 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff9340554e0 msgr2=0x7ff9340579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.074+0000 7ff952f8e700 1 --2- 172.21.15.32:0/1002592311 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff9340554e0 0x7ff9340579a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7ff94000c000 tx=0x7ff940008040).stop 2020-12-05T17:41:36.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.074+0000 7ff952f8e700 1 -- 172.21.15.32:0/1002592311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94c142f50 msgr2=0x7ff94c062ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:36.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.074+0000 7ff952f8e700 1 --2- 172.21.15.32:0/1002592311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94c142f50 0x7ff94c062ff0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7ff94400a000 tx=0x7ff944008040).stop 2020-12-05T17:41:36.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.074+0000 7ff952f8e700 1 -- 172.21.15.32:0/1002592311 shutdown_connections 2020-12-05T17:41:36.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.074+0000 7ff952f8e700 1 --2- 172.21.15.32:0/1002592311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94c142f50 0x7ff94c062ff0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.074+0000 7ff952f8e700 1 --2- 172.21.15.32:0/1002592311 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff9340554e0 0x7ff9340579a0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.074+0000 7ff952f8e700 1 --2- 172.21.15.32:0/1002592311 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff94c05f7f0 0x7ff94c065c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.074+0000 7ff952f8e700 1 --2- 172.21.15.32:0/1002592311 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff94c0b2870 0x7ff94c0aec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.074+0000 7ff952f8e700 1 -- 172.21.15.32:0/1002592311 >> 172.21.15.32:0/1002592311 conn(0x7ff94c0604c0 msgr2=0x7ff94c05c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:36.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.074+0000 7ff952f8e700 1 -- 172.21.15.32:0/1002592311 shutdown_connections 2020-12-05T17:41:36.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.074+0000 7ff952f8e700 1 -- 172.21.15.32:0/1002592311 wait complete. 2020-12-05T17:41:36.090 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:41:30.899960+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:41:31.381413+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:41:36.091 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:41:36.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.202+0000 7f1edffff700 1 Processor -- start 2020-12-05T17:41:36.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.202+0000 7f1edffff700 1 -- start start 2020-12-05T17:41:36.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.202+0000 7f1edffff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1ee013ed60 0x7f1ee00af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:36.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.202+0000 7f1edffff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee00b24e0 con 0x7f1ee00ad250 2020-12-05T17:41:36.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.202+0000 7f1edffff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee00b0780 con 0x7f1ee00b2100 2020-12-05T17:41:36.209 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.202+0000 7f1edffff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee013fd10 con 0x7f1ee013ed60 2020-12-05T17:41:36.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.202+0000 7f1edf7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1ee00ad250 0x7f1ee0142600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43682/0 (socket says 172.21.15.32:43682) 2020-12-05T17:41:36.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.202+0000 7f1edf7fe700 1 -- 172.21.15.32:0/4150164568 learned_addr learned my addr 172.21.15.32:0/4150164568 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:36.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.202+0000 7f1ede7fc700 1 --2- 172.21.15.32:0/4150164568 >> v2:172.21.15.61:3300/0 conn(0x7f1ee013ed60 0x7f1ee00af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:36.210 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.202+0000 7f1eddffb700 1 -- 172.21.15.32:0/4150164568 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4184508590 0 0) 0x7f1ee00b24e0 con 0x7f1ee00ad250 2020-12-05T17:41:36.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.202+0000 7f1eddffb700 1 -- 172.21.15.32:0/4150164568 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ec4003230 con 0x7f1ee00ad250 2020-12-05T17:41:36.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.202+0000 7f1eddffb700 1 -- 172.21.15.32:0/4150164568 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 533344124 0 0) 0x7f1ee00b0780 con 0x7f1ee00b2100 2020-12-05T17:41:36.211 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.202+0000 7f1eddffb700 1 -- 172.21.15.32:0/4150164568 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ee00b24e0 con 0x7f1ee00b2100 2020-12-05T17:41:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.206+0000 7f1ede7fc700 1 -- 172.21.15.32:0/4150164568 >> v1:172.21.15.61:6789/0 conn(0x7f1ee00b2100 legacy=0x7f1ee0138820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.206+0000 7f1ede7fc700 1 -- 172.21.15.32:0/4150164568 >> v1:172.21.15.32:6789/0 conn(0x7f1ee00ad250 legacy=0x7f1ee0142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.206+0000 7f1eddffb700 1 -- 172.21.15.32:0/4150164568 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4096243609 0 0) 0x7f1ec4003230 con 0x7f1ee00ad250 2020-12-05T17:41:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.206+0000 7f1ede7fc700 1 -- 172.21.15.32:0/4150164568 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee00b0240 con 0x7f1ee013ed60 2020-12-05T17:41:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.206+0000 7f1ede7fc700 1 --2- 172.21.15.32:0/4150164568 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ee013ed60 0x7f1ee00af360 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f1ed80010d0 tx=0x7f1ec8006dc0).ready entity=mon.1 client_cookie=133eefdb44eefd58 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.206+0000 7f1eddffb700 1 -- 172.21.15.32:0/4150164568 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ed80016d0 con 0x7f1ee013ed60 2020-12-05T17:41:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.206+0000 7f1eddffb700 1 -- 172.21.15.32:0/4150164568 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec800d610 con 0x7f1ee013ed60 2020-12-05T17:41:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.206+0000 7f1edffff700 1 -- 172.21.15.32:0/4150164568 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ee013ed60 msgr2=0x7f1ee00af360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.206+0000 7f1eddffb700 1 -- 172.21.15.32:0/4150164568 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ec800d780 con 0x7f1ee013ed60 2020-12-05T17:41:36.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.206+0000 7f1edffff700 1 --2- 172.21.15.32:0/4150164568 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ee013ed60 0x7f1ee00af360 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f1ed80010d0 tx=0x7f1ec8006dc0).stop 2020-12-05T17:41:36.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.206+0000 7f1edffff700 1 -- 172.21.15.32:0/4150164568 shutdown_connections 2020-12-05T17:41:36.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.206+0000 7f1edffff700 1 --2- 172.21.15.32:0/4150164568 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ee013ed60 0x7f1ee00af360 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.206+0000 7f1edffff700 1 -- 172.21.15.32:0/4150164568 >> 172.21.15.32:0/4150164568 conn(0x7f1ee00620b0 msgr2=0x7f1ee0062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:36.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.206+0000 7f1edffff700 1 -- 172.21.15.32:0/4150164568 shutdown_connections 2020-12-05T17:41:36.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.206+0000 7f1edffff700 1 -- 172.21.15.32:0/4150164568 wait complete. 2020-12-05T17:41:36.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.206+0000 7f1edffff700 1 Processor -- start 2020-12-05T17:41:36.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.206+0000 7f1edffff700 1 -- start start 2020-12-05T17:41:36.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.206+0000 7f1edffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ee00ad250 0x7f1ee0066960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:36.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.206+0000 7f1edffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ee00b2100 0x7f1ee013be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1edffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ee013c320 0x7f1ee01327c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1edffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee01539f0 con 0x7f1ee00ad250 2020-12-05T17:41:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1edffff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee01339e0 con 0x7f1ee013c320 2020-12-05T17:41:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1edffff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee014fa20 con 0x7f1ee00b2100 2020-12-05T17:41:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1edeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ee00ad250 0x7f1ee0066960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1edeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ee00ad250 0x7f1ee0066960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41036/0 (socket says 172.21.15.32:41036) 2020-12-05T17:41:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1edeffd700 1 -- 172.21.15.32:0/1236531292 learned_addr learned my addr 172.21.15.32:0/1236531292 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1ede7fc700 1 --2- 172.21.15.32:0/1236531292 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ee00b2100 0x7f1ee013be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1edf7fe700 1 --2- 172.21.15.32:0/1236531292 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ee013c320 0x7f1ee01327c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:36.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1edeffd700 1 -- 172.21.15.32:0/1236531292 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ee00b2100 msgr2=0x7f1ee013be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:36.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1edeffd700 1 --2- 172.21.15.32:0/1236531292 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ee00b2100 0x7f1ee013be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1edeffd700 1 -- 172.21.15.32:0/1236531292 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ee013c320 msgr2=0x7f1ee01327c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:36.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1edeffd700 1 --2- 172.21.15.32:0/1236531292 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ee013c320 0x7f1ee01327c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1edeffd700 1 -- 172.21.15.32:0/1236531292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee0133c10 con 0x7f1ee00ad250 2020-12-05T17:41:36.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1edeffd700 1 --2- 172.21.15.32:0/1236531292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ee00ad250 0x7f1ee0066960 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7f1ed0001450 tx=0x7f1ed0007e70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:36.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1ed7fff700 1 -- 172.21.15.32:0/1236531292 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ed000ce70 con 0x7f1ee00ad250 2020-12-05T17:41:36.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1ed7fff700 1 -- 172.21.15.32:0/1236531292 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed000a770 con 0x7f1ee00ad250 2020-12-05T17:41:36.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1edffff700 1 -- 172.21.15.32:0/1236531292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee0133eb0 con 0x7f1ee00ad250 2020-12-05T17:41:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1ed7fff700 1 -- 172.21.15.32:0/1236531292 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ed000a8e0 con 0x7f1ee00ad250 2020-12-05T17:41:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1edffff700 1 -- 172.21.15.32:0/1236531292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee0065360 con 0x7f1ee00ad250 2020-12-05T17:41:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1ed7fff700 1 -- 172.21.15.32:0/1236531292 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1ed0010070 con 0x7f1ee00ad250 2020-12-05T17:41:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1ed7fff700 1 --2- 172.21.15.32:0/1236531292 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1eb8055350 0x7f1eb8057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1ede7fc700 1 --2- 172.21.15.32:0/1236531292 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1eb8055350 0x7f1eb8057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:36.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.210+0000 7f1ed7fff700 1 -- 172.21.15.32:0/1236531292 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1ed006dd80 con 0x7f1ee00ad250 2020-12-05T17:41:36.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.214+0000 7f1edffff700 1 -- 172.21.15.32:0/1236531292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee0057050 con 0x7f1ee00ad250 2020-12-05T17:41:36.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.214+0000 7f1ede7fc700 1 --2- 172.21.15.32:0/1236531292 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1eb8055350 0x7f1eb8057810 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f1ec801a000 tx=0x7f1ec8015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:36.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.222+0000 7f1ed7fff700 1 -- 172.21.15.32:0/1236531292 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1ed0043e10 con 0x7f1ee00ad250 2020-12-05T17:41:36.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.366+0000 7f1edffff700 1 -- 172.21.15.32:0/1236531292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f1ee0060c10 con 0x7f1ee00ad250 2020-12-05T17:41:36.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.366+0000 7f1ed7fff700 1 -- 172.21.15.32:0/1236531292 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v138) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1ed00290b0 con 0x7f1ee00ad250 2020-12-05T17:41:36.371 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:41:36.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.366+0000 7f1edffff700 1 -- 172.21.15.32:0/1236531292 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1eb8055350 msgr2=0x7f1eb8057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:36.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.366+0000 7f1edffff700 1 --2- 172.21.15.32:0/1236531292 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1eb8055350 0x7f1eb8057810 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f1ec801a000 tx=0x7f1ec8015000).stop 2020-12-05T17:41:36.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.366+0000 7f1edffff700 1 -- 172.21.15.32:0/1236531292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ee00ad250 msgr2=0x7f1ee0066960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:36.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.366+0000 7f1edffff700 1 --2- 172.21.15.32:0/1236531292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ee00ad250 0x7f1ee0066960 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7f1ed0001450 tx=0x7f1ed0007e70).stop 2020-12-05T17:41:36.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.366+0000 7f1edffff700 1 -- 172.21.15.32:0/1236531292 shutdown_connections 2020-12-05T17:41:36.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.366+0000 7f1edffff700 1 --2- 172.21.15.32:0/1236531292 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ee013c320 0x7f1ee01327c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.366+0000 7f1edffff700 1 --2- 172.21.15.32:0/1236531292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ee00ad250 0x7f1ee0066960 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.366+0000 7f1edffff700 1 --2- 172.21.15.32:0/1236531292 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1eb8055350 0x7f1eb8057810 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.366+0000 7f1edffff700 1 --2- 172.21.15.32:0/1236531292 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ee00b2100 0x7f1ee013be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.366+0000 7f1edffff700 1 -- 172.21.15.32:0/1236531292 >> 172.21.15.32:0/1236531292 conn(0x7f1ee00620b0 msgr2=0x7f1ee0062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.366+0000 7f1edffff700 1 -- 172.21.15.32:0/1236531292 shutdown_connections 2020-12-05T17:41:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.366+0000 7f1edffff700 1 -- 172.21.15.32:0/1236531292 wait complete. 2020-12-05T17:41:36.385 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:41:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f053a6b7700 1 Processor -- start 2020-12-05T17:41:36.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f053a6b7700 1 -- start start 2020-12-05T17:41:36.503 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f053a6b7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f053405feb0 0x7f053405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f053a6b7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05340b2df0 con 0x7f05340b2fb0 2020-12-05T17:41:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f053a6b7700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05340b3c90 con 0x7f053405f3b0 2020-12-05T17:41:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f053a6b7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05340b2b50 con 0x7f053405feb0 2020-12-05T17:41:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f0538eb4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f053405feb0 0x7f053405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:36.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f0539eb6700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f05340b2fb0 0x7f0534142cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43696/0 (socket says 172.21.15.32:43696) 2020-12-05T17:41:36.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f0538eb4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f053405feb0 0x7f053405f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41044/0 (socket says 172.21.15.32:41044) 2020-12-05T17:41:36.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f0539eb6700 1 -- 172.21.15.32:0/3319490213 learned_addr learned my addr 172.21.15.32:0/3319490213 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:36.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f0538eb4700 1 -- 172.21.15.32:0/3319490213 >> v1:172.21.15.61:6789/0 conn(0x7f053405f3b0 legacy=0x7f053413c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:36.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f0538eb4700 1 -- 172.21.15.32:0/3319490213 >> v1:172.21.15.32:6789/0 conn(0x7f05340b2fb0 legacy=0x7f0534142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f0523fff700 1 -- 172.21.15.32:0/3319490213 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224984246 0 0) 0x7f05340b2df0 con 0x7f05340b2fb0 2020-12-05T17:41:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f0538eb4700 1 -- 172.21.15.32:0/3319490213 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0534142820 con 0x7f053405feb0 2020-12-05T17:41:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f0538eb4700 1 --2- 172.21.15.32:0/3319490213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f053405feb0 0x7f053405f790 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 rx=0x7f0524002f30 tx=0x7f0524006dc0).ready entity=mon.0 client_cookie=1a991584864362bc server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f0523fff700 1 -- 172.21.15.32:0/3319490213 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05240097f0 con 0x7f053405feb0 2020-12-05T17:41:36.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f0523fff700 1 -- 172.21.15.32:0/3319490213 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0524009950 con 0x7f053405feb0 2020-12-05T17:41:36.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f0523fff700 1 -- 172.21.15.32:0/3319490213 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0524009cb0 con 0x7f053405feb0 2020-12-05T17:41:36.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f053a6b7700 1 -- 172.21.15.32:0/3319490213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f053405feb0 msgr2=0x7f053405f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:36.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f053a6b7700 1 --2- 172.21.15.32:0/3319490213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f053405feb0 0x7f053405f790 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 rx=0x7f0524002f30 tx=0x7f0524006dc0).stop 2020-12-05T17:41:36.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f053a6b7700 1 -- 172.21.15.32:0/3319490213 shutdown_connections 2020-12-05T17:41:36.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f053a6b7700 1 --2- 172.21.15.32:0/3319490213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f053405feb0 0x7f053405f790 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f053a6b7700 1 -- 172.21.15.32:0/3319490213 >> 172.21.15.32:0/3319490213 conn(0x7f05340b8f20 msgr2=0x7f05340b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:36.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f053a6b7700 1 -- 172.21.15.32:0/3319490213 shutdown_connections 2020-12-05T17:41:36.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.498+0000 7f053a6b7700 1 -- 172.21.15.32:0/3319490213 wait complete. 2020-12-05T17:41:36.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.502+0000 7f053a6b7700 1 Processor -- start 2020-12-05T17:41:36.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.502+0000 7f053a6b7700 1 -- start start 2020-12-05T17:41:36.511 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.502+0000 7f053a6b7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f053405f3b0 0x7f0534143d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:36.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.502+0000 7f053a6b7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f053405feb0 0x7f053413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:36.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.502+0000 7f053a6b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05340b2fb0 0x7f053413b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:36.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.502+0000 7f053a6b7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0534145880 con 0x7f05340b2fb0 2020-12-05T17:41:36.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.502+0000 7f053a6b7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0534145b20 con 0x7f053405f3b0 2020-12-05T17:41:36.512 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.502+0000 7f053a6b7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0534145dc0 con 0x7f053405feb0 2020-12-05T17:41:36.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.502+0000 7f0539eb6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05340b2fb0 0x7f053413b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:36.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.502+0000 7f0539eb6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05340b2fb0 0x7f053413b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41054/0 (socket says 172.21.15.32:41054) 2020-12-05T17:41:36.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.502+0000 7f0539eb6700 1 -- 172.21.15.32:0/1776912415 learned_addr learned my addr 172.21.15.32:0/1776912415 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:36.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.502+0000 7f0538eb4700 1 --2- 172.21.15.32:0/1776912415 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f053405feb0 0x7f053413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:36.513 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.502+0000 7f05396b5700 1 --2- 172.21.15.32:0/1776912415 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f053405f3b0 0x7f0534143d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:36.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.502+0000 7f0539eb6700 1 -- 172.21.15.32:0/1776912415 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f053405feb0 msgr2=0x7f053413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:36.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.502+0000 7f0539eb6700 1 --2- 172.21.15.32:0/1776912415 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f053405feb0 0x7f053413b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.502+0000 7f0539eb6700 1 -- 172.21.15.32:0/1776912415 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f053405f3b0 msgr2=0x7f0534143d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:36.514 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.502+0000 7f0539eb6700 1 --2- 172.21.15.32:0/1776912415 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f053405f3b0 0x7f0534143d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.502+0000 7f0539eb6700 1 -- 172.21.15.32:0/1776912415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0534146060 con 0x7f05340b2fb0 2020-12-05T17:41:36.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.502+0000 7f0539eb6700 1 --2- 172.21.15.32:0/1776912415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05340b2fb0 0x7f053413b890 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0x7f052c000f00 tx=0x7f052c007ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:36.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.502+0000 7f05227fc700 1 -- 172.21.15.32:0/1776912415 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f052c005960 con 0x7f05340b2fb0 2020-12-05T17:41:36.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.502+0000 7f05227fc700 1 -- 172.21.15.32:0/1776912415 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052c005ac0 con 0x7f05340b2fb0 2020-12-05T17:41:36.515 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.502+0000 7f053a6b7700 1 -- 172.21.15.32:0/1776912415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0534144200 con 0x7f05340b2fb0 2020-12-05T17:41:36.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.502+0000 7f05227fc700 1 -- 172.21.15.32:0/1776912415 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f052c005e20 con 0x7f05340b2fb0 2020-12-05T17:41:36.516 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.506+0000 7f053a6b7700 1 -- 172.21.15.32:0/1776912415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0534146500 con 0x7f05340b2fb0 2020-12-05T17:41:36.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.506+0000 7f053a6b7700 1 -- 172.21.15.32:0/1776912415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05341461f0 con 0x7f05340b2fb0 2020-12-05T17:41:36.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.506+0000 7f05227fc700 1 -- 172.21.15.32:0/1776912415 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f052c010030 con 0x7f05340b2fb0 2020-12-05T17:41:36.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.506+0000 7f05227fc700 1 --2- 172.21.15.32:0/1776912415 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0514055470 0x7f0514057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:36.517 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.506+0000 7f05396b5700 1 --2- 172.21.15.32:0/1776912415 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0514055470 0x7f0514057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:36.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.506+0000 7f05227fc700 1 -- 172.21.15.32:0/1776912415 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f052c06df60 con 0x7f05340b2fb0 2020-12-05T17:41:36.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.506+0000 7f05396b5700 1 --2- 172.21.15.32:0/1776912415 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0514055470 0x7f0514057930 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f053000d000 tx=0x7f0530000b80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:36.518 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.514+0000 7f05227fc700 1 -- 172.21.15.32:0/1776912415 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f052c044100 con 0x7f05340b2fb0 2020-12-05T17:41:36.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.654+0000 7f053a6b7700 1 -- 172.21.15.32:0/1776912415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0534057050 con 0x7f05340b2fb0 2020-12-05T17:41:36.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.658+0000 7f05227fc700 1 -- 172.21.15.32:0/1776912415 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v138) v1 ==== 92+0+10 (secure 0 0 0) 0x7f052c047720 con 0x7f05340b2fb0 2020-12-05T17:41:36.662 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:41:36.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.658+0000 7f053a6b7700 1 -- 172.21.15.32:0/1776912415 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0514055470 msgr2=0x7f0514057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:36.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.658+0000 7f053a6b7700 1 --2- 172.21.15.32:0/1776912415 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0514055470 0x7f0514057930 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f053000d000 tx=0x7f0530000b80).stop 2020-12-05T17:41:36.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.658+0000 7f053a6b7700 1 -- 172.21.15.32:0/1776912415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05340b2fb0 msgr2=0x7f053413b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:36.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.658+0000 7f053a6b7700 1 --2- 172.21.15.32:0/1776912415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05340b2fb0 0x7f053413b890 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0x7f052c000f00 tx=0x7f052c007ec0).stop 2020-12-05T17:41:36.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.658+0000 7f053a6b7700 1 -- 172.21.15.32:0/1776912415 shutdown_connections 2020-12-05T17:41:36.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.658+0000 7f053a6b7700 1 --2- 172.21.15.32:0/1776912415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05340b2fb0 0x7f053413b890 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.658+0000 7f053a6b7700 1 --2- 172.21.15.32:0/1776912415 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f053405f3b0 0x7f0534143d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.658+0000 7f053a6b7700 1 --2- 172.21.15.32:0/1776912415 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0514055470 0x7f0514057930 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.658+0000 7f053a6b7700 1 --2- 172.21.15.32:0/1776912415 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f053405feb0 0x7f053413b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.658+0000 7f053a6b7700 1 -- 172.21.15.32:0/1776912415 >> 172.21.15.32:0/1776912415 conn(0x7f05340b8f20 msgr2=0x7f0534061780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:36.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.658+0000 7f053a6b7700 1 -- 172.21.15.32:0/1776912415 shutdown_connections 2020-12-05T17:41:36.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.658+0000 7f053a6b7700 1 -- 172.21.15.32:0/1776912415 wait complete. 2020-12-05T17:41:36.679 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:41:36.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.790+0000 7fcf5d2f7700 1 Processor -- start 2020-12-05T17:41:36.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.790+0000 7fcf5d2f7700 1 -- start start 2020-12-05T17:41:36.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.790+0000 7fcf5d2f7700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fcf580b5b20 0x7fcf580b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:36.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.790+0000 7fcf5d2f7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf580b5240 0x7fcf58145f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:36.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.790+0000 7fcf5d2f7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf580b56d0 con 0x7fcf580b6690 2020-12-05T17:41:36.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.790+0000 7fcf5d2f7700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf580b1920 con 0x7fcf580b5b20 2020-12-05T17:41:36.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.790+0000 7fcf5d2f7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf580b1bc0 con 0x7fcf580b5240 2020-12-05T17:41:36.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.790+0000 7fcf567fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcf580b6690 0x7fcf580b2060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43708/0 (socket says 172.21.15.32:43708) 2020-12-05T17:41:36.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.790+0000 7fcf567fc700 1 -- 172.21.15.32:0/4058889779 learned_addr learned my addr 172.21.15.32:0/4058889779 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:36.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.790+0000 7fcf577fe700 1 --2- 172.21.15.32:0/4058889779 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf580b5240 0x7fcf58145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:36.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.790+0000 7fcf56ffd700 1 --2- 172.21.15.32:0/4058889779 >> v2:172.21.15.61:3300/0 conn(0x7fcf580b5b20 0x7fcf580b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:36.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.790+0000 7fcf55ffb700 1 -- 172.21.15.32:0/4058889779 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1731633285 0 0) 0x7fcf580b56d0 con 0x7fcf580b6690 2020-12-05T17:41:36.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.790+0000 7fcf55ffb700 1 -- 172.21.15.32:0/4058889779 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf3c003230 con 0x7fcf580b6690 2020-12-05T17:41:36.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.790+0000 7fcf577fe700 1 -- 172.21.15.32:0/4058889779 >> v2:172.21.15.61:3300/0 conn(0x7fcf580b5b20 msgr2=0x7fcf580b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:36.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.790+0000 7fcf577fe700 1 --2- 172.21.15.32:0/4058889779 >> v2:172.21.15.61:3300/0 conn(0x7fcf580b5b20 0x7fcf580b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.790+0000 7fcf577fe700 1 -- 172.21.15.32:0/4058889779 >> v1:172.21.15.32:6789/0 conn(0x7fcf580b6690 legacy=0x7fcf580b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:36.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.790+0000 7fcf577fe700 1 -- 172.21.15.32:0/4058889779 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf58146400 con 0x7fcf580b5240 2020-12-05T17:41:36.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.790+0000 7fcf577fe700 1 --2- 172.21.15.32:0/4058889779 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf580b5240 0x7fcf58145f10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fcf500032b0 tx=0x7fcf500091a0).ready entity=mon.2 client_cookie=7dd3975fa5a47d4 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:36.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf55ffb700 1 -- 172.21.15.32:0/4058889779 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf5000b940 con 0x7fcf580b5240 2020-12-05T17:41:36.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf55ffb700 1 -- 172.21.15.32:0/4058889779 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5000baa0 con 0x7fcf580b5240 2020-12-05T17:41:36.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf55ffb700 1 -- 172.21.15.32:0/4058889779 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf5000bc10 con 0x7fcf580b5240 2020-12-05T17:41:36.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf5d2f7700 1 -- 172.21.15.32:0/4058889779 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf580b5240 msgr2=0x7fcf58145f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:36.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf5d2f7700 1 --2- 172.21.15.32:0/4058889779 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf580b5240 0x7fcf58145f10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fcf500032b0 tx=0x7fcf500091a0).stop 2020-12-05T17:41:36.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf5d2f7700 1 -- 172.21.15.32:0/4058889779 shutdown_connections 2020-12-05T17:41:36.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf5d2f7700 1 --2- 172.21.15.32:0/4058889779 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf580b5240 0x7fcf58145f10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf5d2f7700 1 --2- 172.21.15.32:0/4058889779 >> v2:172.21.15.61:3300/0 conn(0x7fcf580b5b20 0x7fcf580b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf5d2f7700 1 -- 172.21.15.32:0/4058889779 >> 172.21.15.32:0/4058889779 conn(0x7fcf5805f3b0 msgr2=0x7fcf5805f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:36.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf5d2f7700 1 -- 172.21.15.32:0/4058889779 shutdown_connections 2020-12-05T17:41:36.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf5d2f7700 1 -- 172.21.15.32:0/4058889779 wait complete. 2020-12-05T17:41:36.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf5d2f7700 1 Processor -- start 2020-12-05T17:41:36.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf5d2f7700 1 -- start start 2020-12-05T17:41:36.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf5d2f7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf580b5240 0x7fcf580a3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:36.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf5d2f7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf580b5b20 0x7fcf580aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:36.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf5d2f7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf580b6690 0x7fcf580af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:36.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf5d2f7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf58141100 con 0x7fcf580b5240 2020-12-05T17:41:36.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf5d2f7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf580a35b0 con 0x7fcf580b6690 2020-12-05T17:41:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf5d2f7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf58062dd0 con 0x7fcf580b5b20 2020-12-05T17:41:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf56ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf580b5240 0x7fcf580a3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf56ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf580b5240 0x7fcf580a3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41064/0 (socket says 172.21.15.32:41064) 2020-12-05T17:41:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf56ffd700 1 -- 172.21.15.32:0/2037116399 learned_addr learned my addr 172.21.15.32:0/2037116399 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf577fe700 1 --2- 172.21.15.32:0/2037116399 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf580b6690 0x7fcf580af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:36.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.794+0000 7fcf567fc700 1 --2- 172.21.15.32:0/2037116399 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf580b5b20 0x7fcf580aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:36.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.798+0000 7fcf56ffd700 1 -- 172.21.15.32:0/2037116399 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf580b5b20 msgr2=0x7fcf580aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:36.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.798+0000 7fcf56ffd700 1 --2- 172.21.15.32:0/2037116399 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf580b5b20 0x7fcf580aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.798+0000 7fcf56ffd700 1 -- 172.21.15.32:0/2037116399 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf580b6690 msgr2=0x7fcf580af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:36.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.798+0000 7fcf56ffd700 1 --2- 172.21.15.32:0/2037116399 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf580b6690 0x7fcf580af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.798+0000 7fcf56ffd700 1 -- 172.21.15.32:0/2037116399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf580af670 con 0x7fcf580b5240 2020-12-05T17:41:36.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.798+0000 7fcf56ffd700 1 --2- 172.21.15.32:0/2037116399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf580b5240 0x7fcf580a3cb0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7fcf4800c000 tx=0x7fcf48006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:36.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.798+0000 7fcf47fff700 1 -- 172.21.15.32:0/2037116399 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf4800ca10 con 0x7fcf580b5240 2020-12-05T17:41:36.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.798+0000 7fcf47fff700 1 -- 172.21.15.32:0/2037116399 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4800cb70 con 0x7fcf580b5240 2020-12-05T17:41:36.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.798+0000 7fcf47fff700 1 -- 172.21.15.32:0/2037116399 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf480123f0 con 0x7fcf580b5240 2020-12-05T17:41:36.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.798+0000 7fcf5d2f7700 1 -- 172.21.15.32:0/2037116399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf58066920 con 0x7fcf580b5240 2020-12-05T17:41:36.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.798+0000 7fcf5d2f7700 1 -- 172.21.15.32:0/2037116399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf580a3310 con 0x7fcf580b5240 2020-12-05T17:41:36.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.798+0000 7fcf5d2f7700 1 -- 172.21.15.32:0/2037116399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf580a66e0 con 0x7fcf580b5240 2020-12-05T17:41:36.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.798+0000 7fcf47fff700 1 -- 172.21.15.32:0/2037116399 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fcf48012550 con 0x7fcf580b5240 2020-12-05T17:41:36.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.798+0000 7fcf47fff700 1 --2- 172.21.15.32:0/2037116399 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcf34055500 0x7fcf340579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:36.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.798+0000 7fcf47fff700 1 -- 172.21.15.32:0/2037116399 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcf4806e470 con 0x7fcf580b5240 2020-12-05T17:41:36.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.798+0000 7fcf567fc700 1 --2- 172.21.15.32:0/2037116399 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcf34055500 0x7fcf340579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:36.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.802+0000 7fcf567fc700 1 --2- 172.21.15.32:0/2037116399 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcf34055500 0x7fcf340579c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7fcf4c00d000 tx=0x7fcf4c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:36.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.806+0000 7fcf47fff700 1 -- 172.21.15.32:0/2037116399 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fcf580a66e0 con 0x7fcf580b5240 2020-12-05T17:41:36.956 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.950+0000 7fcf5d2f7700 1 -- 172.21.15.32:0/2037116399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcf580a66e0 con 0x7fcf580b5240 2020-12-05T17:41:36.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.950+0000 7fcf47fff700 1 -- 172.21.15.32:0/2037116399 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v138) v1 ==== 104+0+10 (secure 0 0 0) 0x7fcf580a66e0 con 0x7fcf580b5240 2020-12-05T17:41:36.957 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:41:36.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.954+0000 7fcf5d2f7700 1 -- 172.21.15.32:0/2037116399 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcf34055500 msgr2=0x7fcf340579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:36.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.954+0000 7fcf5d2f7700 1 --2- 172.21.15.32:0/2037116399 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcf34055500 0x7fcf340579c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7fcf4c00d000 tx=0x7fcf4c009040).stop 2020-12-05T17:41:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.954+0000 7fcf5d2f7700 1 -- 172.21.15.32:0/2037116399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf580b5240 msgr2=0x7fcf580a3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.954+0000 7fcf5d2f7700 1 --2- 172.21.15.32:0/2037116399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf580b5240 0x7fcf580a3cb0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7fcf4800c000 tx=0x7fcf48006dc0).stop 2020-12-05T17:41:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.954+0000 7fcf5d2f7700 1 -- 172.21.15.32:0/2037116399 shutdown_connections 2020-12-05T17:41:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.954+0000 7fcf5d2f7700 1 --2- 172.21.15.32:0/2037116399 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf580b6690 0x7fcf580af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.954+0000 7fcf5d2f7700 1 --2- 172.21.15.32:0/2037116399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf580b5240 0x7fcf580a3cb0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.954+0000 7fcf5d2f7700 1 --2- 172.21.15.32:0/2037116399 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcf34055500 0x7fcf340579c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.954+0000 7fcf5d2f7700 1 --2- 172.21.15.32:0/2037116399 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf580b5b20 0x7fcf580aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.954+0000 7fcf5d2f7700 1 -- 172.21.15.32:0/2037116399 >> 172.21.15.32:0/2037116399 conn(0x7fcf5805f3b0 msgr2=0x7fcf581452d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.954+0000 7fcf5d2f7700 1 -- 172.21.15.32:0/2037116399 shutdown_connections 2020-12-05T17:41:36.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:36.954+0000 7fcf5d2f7700 1 -- 172.21.15.32:0/2037116399 wait complete. 2020-12-05T17:41:36.977 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:41:37.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.102+0000 7fcff16c6700 1 Processor -- start 2020-12-05T17:41:37.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.102+0000 7fcff16c6700 1 -- start start 2020-12-05T17:41:37.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.102+0000 7fcff16c6700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fcfec0af5c0 0x7fcfec0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:37.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.102+0000 7fcff16c6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcfec143f00 0x7fcfec144380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:37.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.102+0000 7fcff16c6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfec1407e0 con 0x7fcfec144870 2020-12-05T17:41:37.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.102+0000 7fcff16c6700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfec1405f0 con 0x7fcfec0af5c0 2020-12-05T17:41:37.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.102+0000 7fcff16c6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec061b30 con 0x7fcfec143f00 2020-12-05T17:41:37.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.102+0000 7fcff0ec5700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcfec144870 0x7fcfec05f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43722/0 (socket says 172.21.15.32:43722) 2020-12-05T17:41:37.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.102+0000 7fcff0ec5700 1 -- 172.21.15.32:0/1794133792 learned_addr learned my addr 172.21.15.32:0/1794133792 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:37.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.102+0000 7fcfebfff700 1 --2- 172.21.15.32:0/1794133792 >> v2:172.21.15.61:3300/0 conn(0x7fcfec0af5c0 0x7fcfec0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:37.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.102+0000 7fcfeb7fe700 1 --2- 172.21.15.32:0/1794133792 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcfec143f00 0x7fcfec144380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:37.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.102+0000 7fcfeaffd700 1 -- 172.21.15.32:0/1794133792 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2424478088 0 0) 0x7fcfec1407e0 con 0x7fcfec144870 2020-12-05T17:41:37.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.102+0000 7fcfeaffd700 1 -- 172.21.15.32:0/1794133792 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfcc003230 con 0x7fcfec144870 2020-12-05T17:41:37.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.102+0000 7fcfebfff700 1 -- 172.21.15.32:0/1794133792 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcfec143f00 msgr2=0x7fcfec144380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:37.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.102+0000 7fcfebfff700 1 --2- 172.21.15.32:0/1794133792 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcfec143f00 0x7fcfec144380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:37.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.102+0000 7fcfebfff700 1 -- 172.21.15.32:0/1794133792 >> v1:172.21.15.32:6789/0 conn(0x7fcfec144870 legacy=0x7fcfec05f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:37.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.102+0000 7fcfebfff700 1 -- 172.21.15.32:0/1794133792 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec0b0c70 con 0x7fcfec0af5c0 2020-12-05T17:41:37.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.102+0000 7fcfebfff700 1 --2- 172.21.15.32:0/1794133792 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcfec0af5c0 0x7fcfec0af9a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7fcfe4002f30 tx=0x7fcfe4006d60).ready entity=mon.1 client_cookie=3392951c6aad67f0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:37.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.102+0000 7fcfeaffd700 1 -- 172.21.15.32:0/1794133792 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfe4009580 con 0x7fcfec0af5c0 2020-12-05T17:41:37.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.102+0000 7fcfeaffd700 1 -- 172.21.15.32:0/1794133792 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe40096e0 con 0x7fcfec0af5c0 2020-12-05T17:41:37.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.102+0000 7fcfeaffd700 1 -- 172.21.15.32:0/1794133792 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfe4009850 con 0x7fcfec0af5c0 2020-12-05T17:41:37.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.102+0000 7fcff16c6700 1 -- 172.21.15.32:0/1794133792 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcfec0af5c0 msgr2=0x7fcfec0af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:37.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.102+0000 7fcff16c6700 1 --2- 172.21.15.32:0/1794133792 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcfec0af5c0 0x7fcfec0af9a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7fcfe4002f30 tx=0x7fcfe4006d60).stop 2020-12-05T17:41:37.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.106+0000 7fcff16c6700 1 -- 172.21.15.32:0/1794133792 shutdown_connections 2020-12-05T17:41:37.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.106+0000 7fcff16c6700 1 --2- 172.21.15.32:0/1794133792 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcfec0af5c0 0x7fcfec0af9a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:37.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.106+0000 7fcff16c6700 1 --2- 172.21.15.32:0/1794133792 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcfec143f00 0x7fcfec144380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:37.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.106+0000 7fcff16c6700 1 -- 172.21.15.32:0/1794133792 >> 172.21.15.32:0/1794133792 conn(0x7fcfec061010 msgr2=0x7fcfec061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:37.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.106+0000 7fcff16c6700 1 -- 172.21.15.32:0/1794133792 shutdown_connections 2020-12-05T17:41:37.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.106+0000 7fcff16c6700 1 -- 172.21.15.32:0/1794133792 wait complete. 2020-12-05T17:41:37.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.106+0000 7fcff16c6700 1 Processor -- start 2020-12-05T17:41:37.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.106+0000 7fcff16c6700 1 -- start start 2020-12-05T17:41:37.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.110+0000 7fcff16c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfec0af5c0 0x7fcfec065220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:37.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.110+0000 7fcff16c6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcfec143f00 0x7fcfec0b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:37.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.110+0000 7fcff16c6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcfec144870 0x7fcfec14fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:37.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.110+0000 7fcff16c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec0660e0 con 0x7fcfec0af5c0 2020-12-05T17:41:37.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.110+0000 7fcff16c6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec0b4ee0 con 0x7fcfec144870 2020-12-05T17:41:37.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.110+0000 7fcff16c6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec064cd0 con 0x7fcfec143f00 2020-12-05T17:41:37.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.110+0000 7fcfebfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfec0af5c0 0x7fcfec065220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:37.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.110+0000 7fcfebfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfec0af5c0 0x7fcfec065220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41076/0 (socket says 172.21.15.32:41076) 2020-12-05T17:41:37.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.110+0000 7fcfebfff700 1 -- 172.21.15.32:0/3502710961 learned_addr learned my addr 172.21.15.32:0/3502710961 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:37.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.110+0000 7fcfeb7fe700 1 --2- 172.21.15.32:0/3502710961 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcfec143f00 0x7fcfec0b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:37.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.110+0000 7fcff0ec5700 1 --2- 172.21.15.32:0/3502710961 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcfec144870 0x7fcfec14fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:37.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.110+0000 7fcfebfff700 1 -- 172.21.15.32:0/3502710961 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcfec143f00 msgr2=0x7fcfec0b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:37.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.110+0000 7fcfebfff700 1 --2- 172.21.15.32:0/3502710961 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcfec143f00 0x7fcfec0b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:37.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.110+0000 7fcfebfff700 1 -- 172.21.15.32:0/3502710961 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcfec144870 msgr2=0x7fcfec14fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:37.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.110+0000 7fcfebfff700 1 --2- 172.21.15.32:0/3502710961 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcfec144870 0x7fcfec14fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:37.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.110+0000 7fcfebfff700 1 -- 172.21.15.32:0/3502710961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec14f570 con 0x7fcfec0af5c0 2020-12-05T17:41:37.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.110+0000 7fcfebfff700 1 --2- 172.21.15.32:0/3502710961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfec0af5c0 0x7fcfec065220 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 rx=0x7fcfe4002f30 tx=0x7fcfe4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:37.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.110+0000 7fcfe97fa700 1 -- 172.21.15.32:0/3502710961 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfe4011b80 con 0x7fcfec0af5c0 2020-12-05T17:41:37.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.110+0000 7fcfe97fa700 1 -- 172.21.15.32:0/3502710961 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe4011ce0 con 0x7fcfec0af5c0 2020-12-05T17:41:37.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.110+0000 7fcff16c6700 1 -- 172.21.15.32:0/3502710961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfec14f810 con 0x7fcfec0af5c0 2020-12-05T17:41:37.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.110+0000 7fcfe97fa700 1 -- 172.21.15.32:0/3502710961 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfe4021630 con 0x7fcfec0af5c0 2020-12-05T17:41:37.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.110+0000 7fcff16c6700 1 -- 172.21.15.32:0/3502710961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfec064950 con 0x7fcfec0af5c0 2020-12-05T17:41:37.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.110+0000 7fcff16c6700 1 -- 172.21.15.32:0/3502710961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfec057050 con 0x7fcfec0af5c0 2020-12-05T17:41:37.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.114+0000 7fcfe97fa700 1 -- 172.21.15.32:0/3502710961 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fcfe400f070 con 0x7fcfec0af5c0 2020-12-05T17:41:37.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.114+0000 7fcfe97fa700 1 --2- 172.21.15.32:0/3502710961 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcfd0055470 0x7fcfd0057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:37.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.114+0000 7fcfe97fa700 1 -- 172.21.15.32:0/3502710961 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcfe406f2d0 con 0x7fcfec0af5c0 2020-12-05T17:41:37.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.114+0000 7fcfeb7fe700 1 --2- 172.21.15.32:0/3502710961 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcfd0055470 0x7fcfd0057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:37.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.114+0000 7fcfeb7fe700 1 --2- 172.21.15.32:0/3502710961 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcfd0055470 0x7fcfd0057930 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7fcfd400b000 tx=0x7fcfd40091a0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:37.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.126+0000 7fcfe97fa700 1 -- 172.21.15.32:0/3502710961 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fcfe4045470 con 0x7fcfec0af5c0 2020-12-05T17:41:37.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.266+0000 7fcff16c6700 1 -- 172.21.15.32:0/3502710961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcfec069690 con 0x7fcfec0af5c0 2020-12-05T17:41:37.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.266+0000 7fcfe97fa700 1 -- 172.21.15.32:0/3502710961 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v138) v1 ==== 100+0+10 (secure 0 0 0) 0x7fcfe4015070 con 0x7fcfec0af5c0 2020-12-05T17:41:37.271 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:41:37.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.266+0000 7fcff16c6700 1 -- 172.21.15.32:0/3502710961 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcfd0055470 msgr2=0x7fcfd0057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:37.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.266+0000 7fcff16c6700 1 --2- 172.21.15.32:0/3502710961 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcfd0055470 0x7fcfd0057930 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7fcfd400b000 tx=0x7fcfd40091a0).stop 2020-12-05T17:41:37.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.266+0000 7fcff16c6700 1 -- 172.21.15.32:0/3502710961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfec0af5c0 msgr2=0x7fcfec065220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:37.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.266+0000 7fcff16c6700 1 --2- 172.21.15.32:0/3502710961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfec0af5c0 0x7fcfec065220 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 rx=0x7fcfe4002f30 tx=0x7fcfe4016000).stop 2020-12-05T17:41:37.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.266+0000 7fcff16c6700 1 -- 172.21.15.32:0/3502710961 shutdown_connections 2020-12-05T17:41:37.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.266+0000 7fcff16c6700 1 --2- 172.21.15.32:0/3502710961 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcfec144870 0x7fcfec14fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:37.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.266+0000 7fcff16c6700 1 --2- 172.21.15.32:0/3502710961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfec0af5c0 0x7fcfec065220 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:37.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.266+0000 7fcff16c6700 1 --2- 172.21.15.32:0/3502710961 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcfd0055470 0x7fcfd0057930 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:37.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.266+0000 7fcff16c6700 1 --2- 172.21.15.32:0/3502710961 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcfec143f00 0x7fcfec0b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:37.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.266+0000 7fcff16c6700 1 -- 172.21.15.32:0/3502710961 >> 172.21.15.32:0/3502710961 conn(0x7fcfec061010 msgr2=0x7fcfec140b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:37.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.270+0000 7fcff16c6700 1 -- 172.21.15.32:0/3502710961 shutdown_connections 2020-12-05T17:41:37.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.270+0000 7fcff16c6700 1 -- 172.21.15.32:0/3502710961 wait complete. 2020-12-05T17:41:37.286 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:41:37.286 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:41:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa154afb700 1 Processor -- start 2020-12-05T17:41:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa154afb700 1 -- start start 2020-12-05T17:41:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa154afb700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa15005f3b0 0x7fa15005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa154afb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa150062400 0x7fa1500b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa154afb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa150142450 con 0x7fa15005feb0 2020-12-05T17:41:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa154afb700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1500b42f0 con 0x7fa15005f3b0 2020-12-05T17:41:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa154afb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa150061780 con 0x7fa150062400 2020-12-05T17:41:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa14ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa15005feb0 0x7fa1500b3be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43736/0 (socket says 172.21.15.32:43736) 2020-12-05T17:41:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa14ffff700 1 -- 172.21.15.32:0/4251677898 learned_addr learned my addr 172.21.15.32:0/4251677898 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa14effd700 1 --2- 172.21.15.32:0/4251677898 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa150062400 0x7fa1500b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa14f7fe700 1 --2- 172.21.15.32:0/4251677898 >> v2:172.21.15.61:3300/0 conn(0x7fa15005f3b0 0x7fa15005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:37.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa14e7fc700 1 -- 172.21.15.32:0/4251677898 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 999963819 0 0) 0x7fa150142450 con 0x7fa15005feb0 2020-12-05T17:41:37.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa14e7fc700 1 -- 172.21.15.32:0/4251677898 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa13c003230 con 0x7fa15005feb0 2020-12-05T17:41:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa14f7fe700 1 -- 172.21.15.32:0/4251677898 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa150062400 msgr2=0x7fa1500b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa14e7fc700 1 -- 172.21.15.32:0/4251677898 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2175003091 0 0) 0x7fa13c003230 con 0x7fa15005feb0 2020-12-05T17:41:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa14f7fe700 1 --2- 172.21.15.32:0/4251677898 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa150062400 0x7fa1500b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa14f7fe700 1 -- 172.21.15.32:0/4251677898 >> v1:172.21.15.32:6789/0 conn(0x7fa15005feb0 legacy=0x7fa1500b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa14f7fe700 1 -- 172.21.15.32:0/4251677898 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa150061a20 con 0x7fa15005f3b0 2020-12-05T17:41:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa14effd700 1 --2- 172.21.15.32:0/4251677898 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa150062400 0x7fa1500b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa14f7fe700 1 --2- 172.21.15.32:0/4251677898 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa15005f3b0 0x7fa15005f790 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7fa148001bf0 tx=0x7fa144006dc0).ready entity=mon.1 client_cookie=60286f317c97e20a server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa14e7fc700 1 -- 172.21.15.32:0/4251677898 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa148001a50 con 0x7fa15005f3b0 2020-12-05T17:41:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa14e7fc700 1 -- 172.21.15.32:0/4251677898 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14400d570 con 0x7fa15005f3b0 2020-12-05T17:41:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa14e7fc700 1 -- 172.21.15.32:0/4251677898 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa14400d6e0 con 0x7fa15005f3b0 2020-12-05T17:41:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa154afb700 1 -- 172.21.15.32:0/4251677898 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa15005f3b0 msgr2=0x7fa15005f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa154afb700 1 --2- 172.21.15.32:0/4251677898 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa15005f3b0 0x7fa15005f790 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7fa148001bf0 tx=0x7fa144006dc0).stop 2020-12-05T17:41:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa154afb700 1 -- 172.21.15.32:0/4251677898 shutdown_connections 2020-12-05T17:41:37.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa154afb700 1 --2- 172.21.15.32:0/4251677898 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa15005f3b0 0x7fa15005f790 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:37.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa154afb700 1 --2- 172.21.15.32:0/4251677898 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa150062400 0x7fa1500b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:37.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa154afb700 1 -- 172.21.15.32:0/4251677898 >> 172.21.15.32:0/4251677898 conn(0x7fa1500b8ee0 msgr2=0x7fa1500b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:37.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa154afb700 1 -- 172.21.15.32:0/4251677898 shutdown_connections 2020-12-05T17:41:37.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa154afb700 1 -- 172.21.15.32:0/4251677898 wait complete. 2020-12-05T17:41:37.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa154afb700 1 Processor -- start 2020-12-05T17:41:37.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.394+0000 7fa154afb700 1 -- start start 2020-12-05T17:41:37.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa154afb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa15005f3b0 0x7fa1500b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:37.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa154afb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa15005feb0 0x7fa1501451c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:37.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa154afb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa150062400 0x7fa150064660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:37.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa154afb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa150063190 con 0x7fa150062400 2020-12-05T17:41:37.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa154afb700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa150063400 con 0x7fa15005feb0 2020-12-05T17:41:37.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa154afb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1500636a0 con 0x7fa15005f3b0 2020-12-05T17:41:37.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa14ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa150062400 0x7fa150064660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:37.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa14ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa150062400 0x7fa150064660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41094/0 (socket says 172.21.15.32:41094) 2020-12-05T17:41:37.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa14ffff700 1 -- 172.21.15.32:0/3647617730 learned_addr learned my addr 172.21.15.32:0/3647617730 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:37.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa14effd700 1 --2- 172.21.15.32:0/3647617730 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa15005feb0 0x7fa1501451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:37.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa14f7fe700 1 --2- 172.21.15.32:0/3647617730 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa15005f3b0 0x7fa1500b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:37.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa14ffff700 1 -- 172.21.15.32:0/3647617730 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa15005f3b0 msgr2=0x7fa1500b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:37.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa14ffff700 1 --2- 172.21.15.32:0/3647617730 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa15005f3b0 0x7fa1500b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:37.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa14ffff700 1 -- 172.21.15.32:0/3647617730 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa15005feb0 msgr2=0x7fa1501451c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:37.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa14ffff700 1 --2- 172.21.15.32:0/3647617730 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa15005feb0 0x7fa1501451c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:37.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa14ffff700 1 -- 172.21.15.32:0/3647617730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa150063940 con 0x7fa150062400 2020-12-05T17:41:37.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa14effd700 1 --2- 172.21.15.32:0/3647617730 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa15005feb0 0x7fa1501451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:37.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa14ffff700 1 --2- 172.21.15.32:0/3647617730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa150062400 0x7fa150064660 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7fa148001cb0 tx=0x7fa148001600).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:37.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa14cff9700 1 -- 172.21.15.32:0/3647617730 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa148004a80 con 0x7fa150062400 2020-12-05T17:41:37.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa14cff9700 1 -- 172.21.15.32:0/3647617730 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa148004be0 con 0x7fa150062400 2020-12-05T17:41:37.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa154afb700 1 -- 172.21.15.32:0/3647617730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1501456b0 con 0x7fa150062400 2020-12-05T17:41:37.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa14cff9700 1 -- 172.21.15.32:0/3647617730 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa148007170 con 0x7fa150062400 2020-12-05T17:41:37.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa154afb700 1 -- 172.21.15.32:0/3647617730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1500b2d00 con 0x7fa150062400 2020-12-05T17:41:37.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa154afb700 1 -- 172.21.15.32:0/3647617730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa150064e90 con 0x7fa150062400 2020-12-05T17:41:37.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.398+0000 7fa14cff9700 1 -- 172.21.15.32:0/3647617730 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa148010070 con 0x7fa150062400 2020-12-05T17:41:37.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.402+0000 7fa14cff9700 1 --2- 172.21.15.32:0/3647617730 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa1300553b0 0x7fa130057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:37.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.402+0000 7fa14cff9700 1 -- 172.21.15.32:0/3647617730 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa14806dd70 con 0x7fa150062400 2020-12-05T17:41:37.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.402+0000 7fa14f7fe700 1 --2- 172.21.15.32:0/3647617730 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa1300553b0 0x7fa130057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:37.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.402+0000 7fa14f7fe700 1 --2- 172.21.15.32:0/3647617730 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa1300553b0 0x7fa130057870 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7fa14401a000 tx=0x7fa144015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:37.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.406+0000 7fa14cff9700 1 -- 172.21.15.32:0/3647617730 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa150064e90 con 0x7fa150062400 2020-12-05T17:41:37.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.594+0000 7fa154afb700 1 -- 172.21.15.32:0/3647617730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa150064e90 con 0x7fa150062400 2020-12-05T17:41:37.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.598+0000 7fa14cff9700 1 -- 172.21.15.32:0/3647617730 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+2142 (secure 0 0 0) 0x7fa150064e90 con 0x7fa150062400 2020-12-05T17:41:37.602 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:37.602 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":66,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/3049192140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3049192140},{"type":"v1","addr":"172.21.15.61:6833","nonce":3049192140}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:41:32.383702+0000","modified":"2020-12-05T17:41:33.387236+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8274":{"gid":8274,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/2808222868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2808222868},{"type":"v1","addr":"172.21.15.32:6835","nonce":2808222868}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2020-12-05T17:41:37.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.598+0000 7fa154afb700 1 -- 172.21.15.32:0/3647617730 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa1300553b0 msgr2=0x7fa130057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:37.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.598+0000 7fa154afb700 1 --2- 172.21.15.32:0/3647617730 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa1300553b0 0x7fa130057870 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7fa14401a000 tx=0x7fa144015000).stop 2020-12-05T17:41:37.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.598+0000 7fa154afb700 1 -- 172.21.15.32:0/3647617730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa150062400 msgr2=0x7fa150064660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:37.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.598+0000 7fa154afb700 1 --2- 172.21.15.32:0/3647617730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa150062400 0x7fa150064660 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7fa148001cb0 tx=0x7fa148001600).stop 2020-12-05T17:41:37.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.598+0000 7fa154afb700 1 -- 172.21.15.32:0/3647617730 shutdown_connections 2020-12-05T17:41:37.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.598+0000 7fa154afb700 1 --2- 172.21.15.32:0/3647617730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa150062400 0x7fa150064660 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:37.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.598+0000 7fa154afb700 1 --2- 172.21.15.32:0/3647617730 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa1300553b0 0x7fa130057870 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:37.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.598+0000 7fa154afb700 1 --2- 172.21.15.32:0/3647617730 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa15005f3b0 0x7fa1500b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:37.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.598+0000 7fa154afb700 1 --2- 172.21.15.32:0/3647617730 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa15005feb0 0x7fa1501451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:37.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.598+0000 7fa154afb700 1 -- 172.21.15.32:0/3647617730 >> 172.21.15.32:0/3647617730 conn(0x7fa1500b8ee0 msgr2=0x7fa1500b92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:37.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.598+0000 7fa154afb700 1 -- 172.21.15.32:0/3647617730 shutdown_connections 2020-12-05T17:41:37.608 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:37.598+0000 7fa154afb700 1 -- 172.21.15.32:0/3647617730 wait complete. 2020-12-05T17:41:37.608 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 66 2020-12-05T17:41:37.822 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:41:32.383702+0000', 'modified': '2020-12-05T17:41:33.387236+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8274}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8274': {'gid': 8274, 'name': 'a', 'rank': 0, 'incarnation': 65, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/2808222868', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 2808222868}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 2808222868}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:41:37.823 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:41:37.823 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:41:37.956 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":8274,"want_state":"up:active","state":"up:active","rank_uptime":5.5599910000000001,"mdsmap_epoch":66,"osdmap_epoch":138,"osdmap_epoch_barrier":138,"uptime":9.7798008470000006} 2020-12-05T17:41:37.957 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 8274, "mdsmap_epoch": 66, "osdmap_epoch": 138, "osdmap_epoch_barrier": 138, "rank_uptime": 5.559991, "state": "up:active", "uptime": 9.779800847, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:41:37.957 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:41:37.958 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-12-05T17:41:37.958 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-12-05T17:41:37.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:37.959 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:41:37.967 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:41:38.027 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:41:38.027 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:41:38.082 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:41:38.083 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:41:38.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:38.084 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:41:38.129 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:41:38.130 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:41:38.173 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:41:38.188 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:41:38.188 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:41:38.249 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:2020-12-05T17:41:38.245+0000 7fd23d77c0c0 -1 init, newargv = 0x559a208d38f0 newargc=9ceph-fuse[ 2020-12-05T17:41:38.249 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:11682]: starting ceph client 2020-12-05T17:41:38.249 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:41:38.250 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:41:38.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:38.251 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:41:38.259 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[11682]: starting fuse 2020-12-05T17:41:38.265 INFO:teuthology.orchestra.run.smithi036.stdout:49 2020-12-05T17:41:38.265 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-12-05T17:41:38.266 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:38.266 INFO:teuthology.orchestra.run.smithi036:> sudo python3 -c ' 2020-12-05T17:41:38.267 INFO:teuthology.orchestra.run.smithi036:> import glob 2020-12-05T17:41:38.267 INFO:teuthology.orchestra.run.smithi036:> import re 2020-12-05T17:41:38.268 INFO:teuthology.orchestra.run.smithi036:> import os 2020-12-05T17:41:38.268 INFO:teuthology.orchestra.run.smithi036:> import subprocess 2020-12-05T17:41:38.268 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:41:38.269 INFO:teuthology.orchestra.run.smithi036:> def find_socket(client_name): 2020-12-05T17:41:38.269 INFO:teuthology.orchestra.run.smithi036:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-12-05T17:41:38.270 INFO:teuthology.orchestra.run.smithi036:> files = glob.glob(asok_path) 2020-12-05T17:41:38.270 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:41:38.270 INFO:teuthology.orchestra.run.smithi036:> # Given a non-glob path, it better be there 2020-12-05T17:41:38.271 INFO:teuthology.orchestra.run.smithi036:> if "*" not in asok_path: 2020-12-05T17:41:38.271 INFO:teuthology.orchestra.run.smithi036:> assert(len(files) == 1) 2020-12-05T17:41:38.272 INFO:teuthology.orchestra.run.smithi036:> return files[0] 2020-12-05T17:41:38.272 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:41:38.272 INFO:teuthology.orchestra.run.smithi036:> for f in files: 2020-12-05T17:41:38.273 INFO:teuthology.orchestra.run.smithi036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:41:38.273 INFO:teuthology.orchestra.run.smithi036:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:41:38.274 INFO:teuthology.orchestra.run.smithi036:> return f 2020-12-05T17:41:38.274 INFO:teuthology.orchestra.run.smithi036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:41:38.275 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:41:38.275 INFO:teuthology.orchestra.run.smithi036:> print(find_socket("client.0")) 2020-12-05T17:41:38.275 INFO:teuthology.orchestra.run.smithi036:> ' 2020-12-05T17:41:38.356 INFO:teuthology.orchestra.run.smithi036.stdout:/var/run/ceph/ceph-client.0.11682.asok 2020-12-05T17:41:38.361 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.11682.asok 2020-12-05T17:41:38.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:38.362 INFO:teuthology.orchestra.run.smithi036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.11682.asok status 2020-12-05T17:41:38.459 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2020-12-05T17:41:38.459 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": { 2020-12-05T17:41:38.460 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:41:38.460 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:41:38.460 INFO:teuthology.orchestra.run.smithi036.stdout: "entity_id": "0", 2020-12-05T17:41:38.460 INFO:teuthology.orchestra.run.smithi036.stdout: "hostname": "smithi036", 2020-12-05T17:41:38.461 INFO:teuthology.orchestra.run.smithi036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-12-05T17:41:38.461 INFO:teuthology.orchestra.run.smithi036.stdout: "pid": "11682", 2020-12-05T17:41:38.461 INFO:teuthology.orchestra.run.smithi036.stdout: "root": "/" 2020-12-05T17:41:38.461 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:41:38.462 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_count": 0, 2020-12-05T17:41:38.462 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_pinned_count": 0, 2020-12-05T17:41:38.462 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 8439, 2020-12-05T17:41:38.462 INFO:teuthology.orchestra.run.smithi036.stdout: "inst": { 2020-12-05T17:41:38.462 INFO:teuthology.orchestra.run.smithi036.stdout: "name": { 2020-12-05T17:41:38.463 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "client", 2020-12-05T17:41:38.463 INFO:teuthology.orchestra.run.smithi036.stdout: "num": 8439 2020-12-05T17:41:38.463 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:41:38.463 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:41:38.464 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:41:38.464 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:41:38.464 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 2833124947 2020-12-05T17:41:38.464 INFO:teuthology.orchestra.run.smithi036.stdout: } 2020-12-05T17:41:38.464 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:41:38.465 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:41:38.465 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:41:38.465 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:41:38.465 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 2833124947 2020-12-05T17:41:38.466 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:41:38.466 INFO:teuthology.orchestra.run.smithi036.stdout: "inst_str": "client.8439 172.21.15.36:0/2833124947", 2020-12-05T17:41:38.466 INFO:teuthology.orchestra.run.smithi036.stdout: "addr_str": "172.21.15.36:0/2833124947", 2020-12-05T17:41:38.466 INFO:teuthology.orchestra.run.smithi036.stdout: "inode_count": 1, 2020-12-05T17:41:38.466 INFO:teuthology.orchestra.run.smithi036.stdout: "mds_epoch": 66, 2020-12-05T17:41:38.467 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch": 138, 2020-12-05T17:41:38.467 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:41:38.467 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklisted": false 2020-12-05T17:41:38.467 INFO:teuthology.orchestra.run.smithi036.stdout:} 2020-12-05T17:41:38.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:38.469 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:38.469 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:41:38.477 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:41:38.478 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:41:38.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:38.479 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:41:38.537 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:41:38.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1947faa700 1 Processor -- start 2020-12-05T17:41:38.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1947faa700 1 -- start start 2020-12-05T17:41:38.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1947faa700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f19400afe50 0x7f1940146a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:38.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1947faa700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19400b3aa0 0x7f19401429e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:38.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1947faa700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19400b0230 con 0x7f19400b1880 2020-12-05T17:41:38.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1947faa700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19400b6a50 con 0x7f19400b3aa0 2020-12-05T17:41:38.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1947faa700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1940146080 con 0x7f19400afe50 2020-12-05T17:41:38.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1945545700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19400b3aa0 0x7f19401429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:38.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1945545700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19400b3aa0 0x7f19401429e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41098/0 (socket says 172.21.15.32:41098) 2020-12-05T17:41:38.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1945545700 1 -- 172.21.15.32:0/625439613 learned_addr learned my addr 172.21.15.32:0/625439613 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:38.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1945d46700 1 --2- 172.21.15.32:0/625439613 >> v2:172.21.15.61:3300/0 conn(0x7f19400afe50 0x7f1940146a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:38.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1945d46700 1 -- 172.21.15.32:0/625439613 >> v2:172.21.15.32:3300/0 conn(0x7f19400b3aa0 msgr2=0x7f19401429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:38.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1945d46700 1 --2- 172.21.15.32:0/625439613 >> v2:172.21.15.32:3300/0 conn(0x7f19400b3aa0 0x7f19401429e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:38.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1945d46700 1 -- 172.21.15.32:0/625439613 >> v1:172.21.15.61:6789/0 conn(0x7f19400b1880 legacy=0x7f1940142ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:38.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1945d46700 1 -- 172.21.15.32:0/625439613 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1940146320 con 0x7f19400afe50 2020-12-05T17:41:38.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1945d46700 1 --2- 172.21.15.32:0/625439613 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19400afe50 0x7f1940146a60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f193c002f30 tx=0x7f193c006dc0).ready entity=mon.1 client_cookie=11095e25b7e2ba77 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:38.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1944d44700 1 -- 172.21.15.32:0/625439613 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f193c0097b0 con 0x7f19400afe50 2020-12-05T17:41:38.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1944d44700 1 -- 172.21.15.32:0/625439613 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193c009910 con 0x7f19400afe50 2020-12-05T17:41:38.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1944d44700 1 -- 172.21.15.32:0/625439613 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f193c009c70 con 0x7f19400afe50 2020-12-05T17:41:38.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1947faa700 1 -- 172.21.15.32:0/625439613 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19400afe50 msgr2=0x7f1940146a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:38.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1947faa700 1 --2- 172.21.15.32:0/625439613 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19400afe50 0x7f1940146a60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f193c002f30 tx=0x7f193c006dc0).stop 2020-12-05T17:41:38.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1947faa700 1 -- 172.21.15.32:0/625439613 shutdown_connections 2020-12-05T17:41:38.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1947faa700 1 --2- 172.21.15.32:0/625439613 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19400afe50 0x7f1940146a60 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:38.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1947faa700 1 --2- 172.21.15.32:0/625439613 >> v2:172.21.15.32:3300/0 conn(0x7f19400b3aa0 0x7f19401429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:38.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1947faa700 1 -- 172.21.15.32:0/625439613 >> 172.21.15.32:0/625439613 conn(0x7f19400b9100 msgr2=0x7f19400b9510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:38.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1947faa700 1 -- 172.21.15.32:0/625439613 shutdown_connections 2020-12-05T17:41:38.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.658+0000 7f1947faa700 1 -- 172.21.15.32:0/625439613 wait complete. 2020-12-05T17:41:38.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.662+0000 7f1947faa700 1 Processor -- start 2020-12-05T17:41:38.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.662+0000 7f1947faa700 1 -- start start 2020-12-05T17:41:38.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.662+0000 7f1947faa700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19400afe50 0x7f1940154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:38.668 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.662+0000 7f1947faa700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19400b1880 0x7f1940062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:38.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.662+0000 7f1947faa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19400b3aa0 0x7f19400664d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:38.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.662+0000 7f1947faa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1940064210 con 0x7f19400b3aa0 2020-12-05T17:41:38.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.662+0000 7f1947faa700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1940142f10 con 0x7f19400b1880 2020-12-05T17:41:38.669 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.662+0000 7f1947faa700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19401416a0 con 0x7f19400afe50 2020-12-05T17:41:38.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.662+0000 7f1946547700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19400b3aa0 0x7f19400664d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:38.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.662+0000 7f1945545700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19400b1880 0x7f1940062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:38.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.662+0000 7f1945d46700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19400afe50 0x7f1940154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:38.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.662+0000 7f1946547700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19400b3aa0 0x7f19400664d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41108/0 (socket says 172.21.15.32:41108) 2020-12-05T17:41:38.670 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.662+0000 7f1946547700 1 -- 172.21.15.32:0/1585591702 learned_addr learned my addr 172.21.15.32:0/1585591702 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:38.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.662+0000 7f1946547700 1 -- 172.21.15.32:0/1585591702 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19400afe50 msgr2=0x7f1940154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:38.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.662+0000 7f1946547700 1 --2- 172.21.15.32:0/1585591702 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19400afe50 0x7f1940154f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:38.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.662+0000 7f1946547700 1 -- 172.21.15.32:0/1585591702 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19400b1880 msgr2=0x7f1940062d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:38.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.662+0000 7f1946547700 1 --2- 172.21.15.32:0/1585591702 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19400b1880 0x7f1940062d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:38.671 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.662+0000 7f1946547700 1 -- 172.21.15.32:0/1585591702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1940141940 con 0x7f19400b3aa0 2020-12-05T17:41:38.672 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.666+0000 7f1946547700 1 --2- 172.21.15.32:0/1585591702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19400b3aa0 0x7f19400664d0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7f1940143050 tx=0x7f1938007ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:38.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.666+0000 7f1936ffd700 1 -- 172.21.15.32:0/1585591702 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f193800ce50 con 0x7f19400b3aa0 2020-12-05T17:41:38.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.666+0000 7f1936ffd700 1 -- 172.21.15.32:0/1585591702 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193800a770 con 0x7f19400b3aa0 2020-12-05T17:41:38.673 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.666+0000 7f1947faa700 1 -- 172.21.15.32:0/1585591702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1940141be0 con 0x7f19400b3aa0 2020-12-05T17:41:38.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.666+0000 7f1936ffd700 1 -- 172.21.15.32:0/1585591702 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f193800aad0 con 0x7f19400b3aa0 2020-12-05T17:41:38.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.666+0000 7f1947faa700 1 -- 172.21.15.32:0/1585591702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1940142860 con 0x7f19400b3aa0 2020-12-05T17:41:38.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.666+0000 7f1947faa700 1 -- 172.21.15.32:0/1585591702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1940057050 con 0x7f19400b3aa0 2020-12-05T17:41:38.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.666+0000 7f1936ffd700 1 -- 172.21.15.32:0/1585591702 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1938019030 con 0x7f19400b3aa0 2020-12-05T17:41:38.674 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.670+0000 7f1936ffd700 1 --2- 172.21.15.32:0/1585591702 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1928055440 0x7f1928057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:38.675 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.670+0000 7f1936ffd700 1 -- 172.21.15.32:0/1585591702 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f193806d950 con 0x7f19400b3aa0 2020-12-05T17:41:38.675 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.670+0000 7f1945d46700 1 --2- 172.21.15.32:0/1585591702 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1928055440 0x7f1928057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:38.675 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.670+0000 7f1945d46700 1 --2- 172.21.15.32:0/1585591702 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1928055440 0x7f1928057900 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f193c01a000 tx=0x7f193c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:38.676 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.674+0000 7f1936ffd700 1 -- 172.21.15.32:0/1585591702 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1938043af0 con 0x7f19400b3aa0 2020-12-05T17:41:38.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.822+0000 7f1947faa700 1 -- 172.21.15.32:0/1585591702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1940069690 con 0x7f19400b3aa0 2020-12-05T17:41:38.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.822+0000 7f1936ffd700 1 -- 172.21.15.32:0/1585591702 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f1938047110 con 0x7f19400b3aa0 2020-12-05T17:41:38.825 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:38.825 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":138,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:41:32.383424+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:41:30.899960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2020-12-05T17:41:31.381413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:41:38.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.826+0000 7f1947faa700 1 -- 172.21.15.32:0/1585591702 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1928055440 msgr2=0x7f1928057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:38.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.826+0000 7f1947faa700 1 --2- 172.21.15.32:0/1585591702 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1928055440 0x7f1928057900 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f193c01a000 tx=0x7f193c015000).stop 2020-12-05T17:41:38.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.826+0000 7f1947faa700 1 -- 172.21.15.32:0/1585591702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19400b3aa0 msgr2=0x7f19400664d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:38.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.826+0000 7f1947faa700 1 --2- 172.21.15.32:0/1585591702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19400b3aa0 0x7f19400664d0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7f1940143050 tx=0x7f1938007ef0).stop 2020-12-05T17:41:38.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.826+0000 7f1947faa700 1 -- 172.21.15.32:0/1585591702 shutdown_connections 2020-12-05T17:41:38.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.826+0000 7f1947faa700 1 --2- 172.21.15.32:0/1585591702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19400b3aa0 0x7f19400664d0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:38.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.826+0000 7f1947faa700 1 --2- 172.21.15.32:0/1585591702 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1928055440 0x7f1928057900 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:38.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.826+0000 7f1947faa700 1 --2- 172.21.15.32:0/1585591702 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19400afe50 0x7f1940154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:38.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.826+0000 7f1947faa700 1 --2- 172.21.15.32:0/1585591702 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19400b1880 0x7f1940062d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:38.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.826+0000 7f1947faa700 1 -- 172.21.15.32:0/1585591702 >> 172.21.15.32:0/1585591702 conn(0x7f19400b9100 msgr2=0x7f1940145250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:38.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.826+0000 7f1947faa700 1 -- 172.21.15.32:0/1585591702 shutdown_connections 2020-12-05T17:41:38.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.826+0000 7f1947faa700 1 -- 172.21.15.32:0/1585591702 wait complete. 2020-12-05T17:41:38.839 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:41:30.899960+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:41:31.381413+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:41:38.840 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:41:38.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.958+0000 7f936c0f0700 1 Processor -- start 2020-12-05T17:41:38.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.958+0000 7f936c0f0700 1 -- start start 2020-12-05T17:41:38.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f936c0f0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93640b6690 0x7f93640b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:38.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f936c0f0700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f93640b5240 0x7f9364145f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:38.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f936c0f0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93640b56f0 con 0x7f93640b5b20 2020-12-05T17:41:38.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f936c0f0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93640b1920 con 0x7f93640b6690 2020-12-05T17:41:38.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f936c0f0700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93640b1bc0 con 0x7f93640b5240 2020-12-05T17:41:38.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f936968b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93640b6690 0x7f93640b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:38.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f9369e8c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f93640b5b20 0x7f93640b2060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43760/0 (socket says 172.21.15.32:43760) 2020-12-05T17:41:38.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f9369e8c700 1 -- 172.21.15.32:0/884693340 learned_addr learned my addr 172.21.15.32:0/884693340 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:38.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f936a68d700 1 --2- 172.21.15.32:0/884693340 >> v2:172.21.15.61:3300/0 conn(0x7f93640b5240 0x7f9364145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:38.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f936968b700 1 -- 172.21.15.32:0/884693340 >> v2:172.21.15.61:3300/0 conn(0x7f93640b5240 msgr2=0x7f9364145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:38.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f936968b700 1 --2- 172.21.15.32:0/884693340 >> v2:172.21.15.61:3300/0 conn(0x7f93640b5240 0x7f9364145f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:38.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f936968b700 1 -- 172.21.15.32:0/884693340 >> v1:172.21.15.32:6789/0 conn(0x7f93640b5b20 legacy=0x7f93640b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:38.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f936968b700 1 -- 172.21.15.32:0/884693340 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9364146400 con 0x7f93640b6690 2020-12-05T17:41:38.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f9368e8a700 1 -- 172.21.15.32:0/884693340 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2505775398 0 0) 0x7f93640b56f0 con 0x7f93640b5b20 2020-12-05T17:41:38.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f936968b700 1 --2- 172.21.15.32:0/884693340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93640b6690 0x7f93640b5f00 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7f9358002f30 tx=0x7f9358006dc0).ready entity=mon.0 client_cookie=abd40069278096cb server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:38.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f9368e8a700 1 -- 172.21.15.32:0/884693340 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93580097b0 con 0x7f93640b6690 2020-12-05T17:41:38.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f9368e8a700 1 -- 172.21.15.32:0/884693340 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9358009910 con 0x7f93640b6690 2020-12-05T17:41:38.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f9368e8a700 1 -- 172.21.15.32:0/884693340 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9358009c70 con 0x7f93640b6690 2020-12-05T17:41:38.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f936c0f0700 1 -- 172.21.15.32:0/884693340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93640b6690 msgr2=0x7f93640b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:38.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f936c0f0700 1 --2- 172.21.15.32:0/884693340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93640b6690 0x7f93640b5f00 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7f9358002f30 tx=0x7f9358006dc0).stop 2020-12-05T17:41:38.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f936c0f0700 1 -- 172.21.15.32:0/884693340 shutdown_connections 2020-12-05T17:41:38.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f936c0f0700 1 --2- 172.21.15.32:0/884693340 >> v2:172.21.15.61:3300/0 conn(0x7f93640b5240 0x7f9364145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:38.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f936c0f0700 1 --2- 172.21.15.32:0/884693340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93640b6690 0x7f93640b5f00 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:38.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f936c0f0700 1 -- 172.21.15.32:0/884693340 >> 172.21.15.32:0/884693340 conn(0x7f936405f3b0 msgr2=0x7f936405f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:38.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f936c0f0700 1 -- 172.21.15.32:0/884693340 shutdown_connections 2020-12-05T17:41:38.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f936c0f0700 1 -- 172.21.15.32:0/884693340 wait complete. 2020-12-05T17:41:38.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f936c0f0700 1 Processor -- start 2020-12-05T17:41:38.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.962+0000 7f936c0f0700 1 -- start start 2020-12-05T17:41:38.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.966+0000 7f936c0f0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93640b5240 0x7f93641527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:38.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.966+0000 7f936c0f0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f93640b5b20 0x7f93640a3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:38.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.966+0000 7f936c0f0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93640b6690 0x7f9364063fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:38.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.966+0000 7f936c0f0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9364152cc0 con 0x7f93640b6690 2020-12-05T17:41:38.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.966+0000 7f936c0f0700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93640a3f50 con 0x7f93640b5b20 2020-12-05T17:41:38.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.966+0000 7f936c0f0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93640a4420 con 0x7f93640b5240 2020-12-05T17:41:38.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.966+0000 7f936a68d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93640b6690 0x7f9364063fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:38.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.966+0000 7f936a68d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93640b6690 0x7f9364063fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41122/0 (socket says 172.21.15.32:41122) 2020-12-05T17:41:38.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.966+0000 7f936a68d700 1 -- 172.21.15.32:0/3132952641 learned_addr learned my addr 172.21.15.32:0/3132952641 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:38.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.966+0000 7f936968b700 1 --2- 172.21.15.32:0/3132952641 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f93640b5b20 0x7f93640a3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:38.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.966+0000 7f9369e8c700 1 --2- 172.21.15.32:0/3132952641 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93640b5240 0x7f93641527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:38.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.966+0000 7f936a68d700 1 -- 172.21.15.32:0/3132952641 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93640b5240 msgr2=0x7f93641527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:38.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.966+0000 7f936a68d700 1 --2- 172.21.15.32:0/3132952641 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93640b5240 0x7f93641527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:38.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.966+0000 7f936a68d700 1 -- 172.21.15.32:0/3132952641 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f93640b5b20 msgr2=0x7f93640a3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:38.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.966+0000 7f936a68d700 1 --2- 172.21.15.32:0/3132952641 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f93640b5b20 0x7f93640a3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:38.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.966+0000 7f936a68d700 1 -- 172.21.15.32:0/3132952641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93640a4180 con 0x7f93640b6690 2020-12-05T17:41:38.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.966+0000 7f936a68d700 1 --2- 172.21.15.32:0/3132952641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93640b6690 0x7f9364063fd0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 rx=0x7f935c006dd0 tx=0x7f935c0091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:38.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.966+0000 7f9356ffd700 1 -- 172.21.15.32:0/3132952641 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f935c00c950 con 0x7f93640b6690 2020-12-05T17:41:38.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.966+0000 7f9356ffd700 1 -- 172.21.15.32:0/3132952641 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935c00cab0 con 0x7f93640b6690 2020-12-05T17:41:38.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.966+0000 7f936c0f0700 1 -- 172.21.15.32:0/3132952641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93640a39f0 con 0x7f93640b6690 2020-12-05T17:41:38.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.966+0000 7f9356ffd700 1 -- 172.21.15.32:0/3132952641 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f935c00ce10 con 0x7f93640b6690 2020-12-05T17:41:38.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.966+0000 7f936c0f0700 1 -- 172.21.15.32:0/3132952641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9364066940 con 0x7f93640b6690 2020-12-05T17:41:38.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.970+0000 7f936c0f0700 1 -- 172.21.15.32:0/3132952641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93641524f0 con 0x7f93640b6690 2020-12-05T17:41:38.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.970+0000 7f9356ffd700 1 -- 172.21.15.32:0/3132952641 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f935c022070 con 0x7f93640b6690 2020-12-05T17:41:38.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.970+0000 7f9356ffd700 1 --2- 172.21.15.32:0/3132952641 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9344055500 0x7f93440579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:38.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.970+0000 7f9369e8c700 1 --2- 172.21.15.32:0/3132952641 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9344055500 0x7f93440579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:38.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.970+0000 7f9356ffd700 1 -- 172.21.15.32:0/3132952641 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f935c06e7b0 con 0x7f93640b6690 2020-12-05T17:41:38.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.970+0000 7f9369e8c700 1 --2- 172.21.15.32:0/3132952641 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9344055500 0x7f93440579c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f936000d000 tx=0x7f9360009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:38.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:38.978+0000 7f9356ffd700 1 -- 172.21.15.32:0/3132952641 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f935c044890 con 0x7f93640b6690 2020-12-05T17:41:39.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.122+0000 7f936c0f0700 1 -- 172.21.15.32:0/3132952641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9364066bf0 con 0x7f93640b6690 2020-12-05T17:41:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.122+0000 7f9356ffd700 1 -- 172.21.15.32:0/3132952641 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v138) v1 ==== 110+0+10 (secure 0 0 0) 0x7f935c016070 con 0x7f93640b6690 2020-12-05T17:41:39.124 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:41:39.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.126+0000 7f936c0f0700 1 -- 172.21.15.32:0/3132952641 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9344055500 msgr2=0x7f93440579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:39.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.126+0000 7f936c0f0700 1 --2- 172.21.15.32:0/3132952641 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9344055500 0x7f93440579c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f936000d000 tx=0x7f9360009040).stop 2020-12-05T17:41:39.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.126+0000 7f936c0f0700 1 -- 172.21.15.32:0/3132952641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93640b6690 msgr2=0x7f9364063fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:39.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.126+0000 7f936c0f0700 1 --2- 172.21.15.32:0/3132952641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93640b6690 0x7f9364063fd0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 rx=0x7f935c006dd0 tx=0x7f935c0091a0).stop 2020-12-05T17:41:39.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.126+0000 7f936c0f0700 1 -- 172.21.15.32:0/3132952641 shutdown_connections 2020-12-05T17:41:39.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.126+0000 7f936c0f0700 1 --2- 172.21.15.32:0/3132952641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93640b6690 0x7f9364063fd0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:39.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.126+0000 7f936c0f0700 1 --2- 172.21.15.32:0/3132952641 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9344055500 0x7f93440579c0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:39.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.126+0000 7f936c0f0700 1 --2- 172.21.15.32:0/3132952641 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93640b5240 0x7f93641527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:39.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.126+0000 7f936c0f0700 1 --2- 172.21.15.32:0/3132952641 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f93640b5b20 0x7f93640a3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:39.129 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.126+0000 7f936c0f0700 1 -- 172.21.15.32:0/3132952641 >> 172.21.15.32:0/3132952641 conn(0x7f936405f3b0 msgr2=0x7f93640b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:39.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.126+0000 7f936c0f0700 1 -- 172.21.15.32:0/3132952641 shutdown_connections 2020-12-05T17:41:39.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.126+0000 7f936c0f0700 1 -- 172.21.15.32:0/3132952641 wait complete. 2020-12-05T17:41:39.144 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:41:39.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.258+0000 7f7ee11ec700 1 Processor -- start 2020-12-05T17:41:39.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.258+0000 7f7ee11ec700 1 -- start start 2020-12-05T17:41:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7ee11ec700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7edc0b2000 0x7f7edc145e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7ee11ec700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7edc0af4b0 0x7f7edc1462f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7ee11ec700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7edc0b23e0 con 0x7f7edc061780 2020-12-05T17:41:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7ee11ec700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7edc0ae7b0 con 0x7f7edc0af4b0 2020-12-05T17:41:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7ee11ec700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc0aea50 con 0x7f7edc0b2000 2020-12-05T17:41:39.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7edb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7edc0af4b0 0x7f7edc1462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:39.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7edb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7edc0af4b0 0x7f7edc1462f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41126/0 (socket says 172.21.15.32:41126) 2020-12-05T17:41:39.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7edb7fe700 1 -- 172.21.15.32:0/2522982874 learned_addr learned my addr 172.21.15.32:0/2522982874 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:39.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7edbfff700 1 --2- 172.21.15.32:0/2522982874 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7edc0b2000 0x7f7edc145e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:39.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7edb7fe700 1 -- 172.21.15.32:0/2522982874 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7edc0b2000 msgr2=0x7f7edc145e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:39.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7edb7fe700 1 --2- 172.21.15.32:0/2522982874 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7edc0b2000 0x7f7edc145e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:39.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7edb7fe700 1 -- 172.21.15.32:0/2522982874 >> v1:172.21.15.61:6789/0 conn(0x7f7edc061780 legacy=0x7f7edc0afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:39.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7edb7fe700 1 -- 172.21.15.32:0/2522982874 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc0b0f60 con 0x7f7edc0af4b0 2020-12-05T17:41:39.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7edbfff700 1 --2- 172.21.15.32:0/2522982874 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7edc0b2000 0x7f7edc145e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:39.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7edb7fe700 1 --2- 172.21.15.32:0/2522982874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0af4b0 0x7f7edc1462f0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7f7ec40032b0 tx=0x7f7ec4009270).ready entity=mon.0 client_cookie=86f50894e63b100c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:39.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7edaffd700 1 -- 172.21.15.32:0/2522982874 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ec400bc00 con 0x7f7edc0af4b0 2020-12-05T17:41:39.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7edaffd700 1 -- 172.21.15.32:0/2522982874 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec400bd60 con 0x7f7edc0af4b0 2020-12-05T17:41:39.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7edaffd700 1 -- 172.21.15.32:0/2522982874 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ec4011400 con 0x7f7edc0af4b0 2020-12-05T17:41:39.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7ee11ec700 1 -- 172.21.15.32:0/2522982874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0af4b0 msgr2=0x7f7edc1462f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:39.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7ee11ec700 1 --2- 172.21.15.32:0/2522982874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0af4b0 0x7f7edc1462f0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7f7ec40032b0 tx=0x7f7ec4009270).stop 2020-12-05T17:41:39.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7ee11ec700 1 -- 172.21.15.32:0/2522982874 shutdown_connections 2020-12-05T17:41:39.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7ee11ec700 1 --2- 172.21.15.32:0/2522982874 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7edc0b2000 0x7f7edc145e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:39.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7ee11ec700 1 --2- 172.21.15.32:0/2522982874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0af4b0 0x7f7edc1462f0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:39.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7ee11ec700 1 -- 172.21.15.32:0/2522982874 >> 172.21.15.32:0/2522982874 conn(0x7f7edc0b5620 msgr2=0x7f7edc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:39.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7ee11ec700 1 -- 172.21.15.32:0/2522982874 shutdown_connections 2020-12-05T17:41:39.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.262+0000 7f7ee11ec700 1 -- 172.21.15.32:0/2522982874 wait complete. 2020-12-05T17:41:39.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7ee11ec700 1 Processor -- start 2020-12-05T17:41:39.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7ee11ec700 1 -- start start 2020-12-05T17:41:39.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7ee11ec700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7edc061780 0x7f7edc14a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:39.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7ee11ec700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7edc0af4b0 0x7f7edc13e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:39.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7ee11ec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0b2000 0x7f7edc13e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:39.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7ee11ec700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc141ea0 con 0x7f7edc0b2000 2020-12-05T17:41:39.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7ee11ec700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc13ecb0 con 0x7f7edc0af4b0 2020-12-05T17:41:39.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7ee11ec700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc13fc20 con 0x7f7edc061780 2020-12-05T17:41:39.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7ee09eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0b2000 0x7f7edc13e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:39.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7ee09eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0b2000 0x7f7edc13e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41136/0 (socket says 172.21.15.32:41136) 2020-12-05T17:41:39.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7ee09eb700 1 -- 172.21.15.32:0/2780817869 learned_addr learned my addr 172.21.15.32:0/2780817869 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:39.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7edb7fe700 1 --2- 172.21.15.32:0/2780817869 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7edc0af4b0 0x7f7edc13e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:39.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7edbfff700 1 --2- 172.21.15.32:0/2780817869 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7edc061780 0x7f7edc14a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:39.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7ee09eb700 1 -- 172.21.15.32:0/2780817869 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7edc061780 msgr2=0x7f7edc14a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:39.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7ee09eb700 1 --2- 172.21.15.32:0/2780817869 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7edc061780 0x7f7edc14a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:39.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7ee09eb700 1 -- 172.21.15.32:0/2780817869 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7edc0af4b0 msgr2=0x7f7edc13e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:39.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7ee09eb700 1 --2- 172.21.15.32:0/2780817869 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7edc0af4b0 0x7f7edc13e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:39.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7ee09eb700 1 -- 172.21.15.32:0/2780817869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc142100 con 0x7f7edc0b2000 2020-12-05T17:41:39.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7edbfff700 1 --2- 172.21.15.32:0/2780817869 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7edc061780 0x7f7edc14a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:39.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7ee09eb700 1 --2- 172.21.15.32:0/2780817869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0b2000 0x7f7edc13e7c0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7f7edc0afd00 tx=0x7f7ed4007b20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:39.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7ed97fa700 1 -- 172.21.15.32:0/2780817869 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ed4013410 con 0x7f7edc0b2000 2020-12-05T17:41:39.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7ed97fa700 1 -- 172.21.15.32:0/2780817869 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed4013570 con 0x7f7edc0b2000 2020-12-05T17:41:39.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7ee11ec700 1 -- 172.21.15.32:0/2780817869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7edc1423a0 con 0x7f7edc0b2000 2020-12-05T17:41:39.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7ed97fa700 1 -- 172.21.15.32:0/2780817869 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ed40138d0 con 0x7f7edc0b2000 2020-12-05T17:41:39.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.266+0000 7f7ee11ec700 1 -- 172.21.15.32:0/2780817869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7edc143d60 con 0x7f7edc0b2000 2020-12-05T17:41:39.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.270+0000 7f7ee11ec700 1 -- 172.21.15.32:0/2780817869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7edc057050 con 0x7f7edc0b2000 2020-12-05T17:41:39.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.270+0000 7f7ed97fa700 1 -- 172.21.15.32:0/2780817869 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7ed4010030 con 0x7f7edc0b2000 2020-12-05T17:41:39.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.270+0000 7f7ed97fa700 1 --2- 172.21.15.32:0/2780817869 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7ec0055330 0x7f7ec00577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:39.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.270+0000 7f7ed97fa700 1 -- 172.21.15.32:0/2780817869 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7ed4001e40 con 0x7f7edc0b2000 2020-12-05T17:41:39.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.270+0000 7f7edbfff700 1 --2- 172.21.15.32:0/2780817869 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7ec0055330 0x7f7ec00577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:39.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.270+0000 7f7edbfff700 1 --2- 172.21.15.32:0/2780817869 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7ec0055330 0x7f7ec00577f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f7ecc00c000 tx=0x7f7ecc008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:39.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.278+0000 7f7ed97fa700 1 -- 172.21.15.32:0/2780817869 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7ed4043b30 con 0x7f7edc0b2000 2020-12-05T17:41:39.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.418+0000 7f7ee11ec700 1 -- 172.21.15.32:0/2780817869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7edc05aab0 con 0x7f7edc0b2000 2020-12-05T17:41:39.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.418+0000 7f7ed97fa700 1 -- 172.21.15.32:0/2780817869 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v138) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7ed4047150 con 0x7f7edc0b2000 2020-12-05T17:41:39.420 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:41:39.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.422+0000 7f7ee11ec700 1 -- 172.21.15.32:0/2780817869 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7ec0055330 msgr2=0x7f7ec00577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:39.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.422+0000 7f7ee11ec700 1 --2- 172.21.15.32:0/2780817869 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7ec0055330 0x7f7ec00577f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f7ecc00c000 tx=0x7f7ecc008040).stop 2020-12-05T17:41:39.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.422+0000 7f7ee11ec700 1 -- 172.21.15.32:0/2780817869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0b2000 msgr2=0x7f7edc13e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:39.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.422+0000 7f7ee11ec700 1 --2- 172.21.15.32:0/2780817869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0b2000 0x7f7edc13e7c0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7f7edc0afd00 tx=0x7f7ed4007b20).stop 2020-12-05T17:41:39.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.422+0000 7f7ee11ec700 1 -- 172.21.15.32:0/2780817869 shutdown_connections 2020-12-05T17:41:39.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.422+0000 7f7ee11ec700 1 --2- 172.21.15.32:0/2780817869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0b2000 0x7f7edc13e7c0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:39.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.422+0000 7f7ee11ec700 1 --2- 172.21.15.32:0/2780817869 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7ec0055330 0x7f7ec00577f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:39.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.422+0000 7f7ee11ec700 1 --2- 172.21.15.32:0/2780817869 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7edc061780 0x7f7edc14a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:39.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.422+0000 7f7ee11ec700 1 --2- 172.21.15.32:0/2780817869 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7edc0af4b0 0x7f7edc13e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:39.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.422+0000 7f7ee11ec700 1 -- 172.21.15.32:0/2780817869 >> 172.21.15.32:0/2780817869 conn(0x7f7edc0b5620 msgr2=0x7f7edc14b160 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:39.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.422+0000 7f7ee11ec700 1 -- 172.21.15.32:0/2780817869 shutdown_connections 2020-12-05T17:41:39.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.422+0000 7f7ee11ec700 1 -- 172.21.15.32:0/2780817869 wait complete. 2020-12-05T17:41:39.434 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:41:39.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.550+0000 7fcf7d43c700 1 Processor -- start 2020-12-05T17:41:39.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.550+0000 7fcf7d43c700 1 -- start start 2020-12-05T17:41:39.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.550+0000 7fcf7d43c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf7805feb0 0x7fcf7805f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:39.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.550+0000 7fcf7d43c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf780b2df0 con 0x7fcf7805f3b0 2020-12-05T17:41:39.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.550+0000 7fcf7d43c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf780b3c90 con 0x7fcf780b2fb0 2020-12-05T17:41:39.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.550+0000 7fcf7d43c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf780b2b50 con 0x7fcf7805feb0 2020-12-05T17:41:39.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.550+0000 7fcf77fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcf7805f3b0 0x7fcf7813c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43788/0 (socket says 172.21.15.32:43788) 2020-12-05T17:41:39.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.550+0000 7fcf77fff700 1 -- 172.21.15.32:0/3350381503 learned_addr learned my addr 172.21.15.32:0/3350381503 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:39.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.550+0000 7fcf777fe700 1 --2- 172.21.15.32:0/3350381503 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf7805feb0 0x7fcf7805f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:39.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.550+0000 7fcf76ffd700 1 -- 172.21.15.32:0/3350381503 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945886051 0 0) 0x7fcf780b2df0 con 0x7fcf7805f3b0 2020-12-05T17:41:39.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.550+0000 7fcf76ffd700 1 -- 172.21.15.32:0/3350381503 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf64003230 con 0x7fcf7805f3b0 2020-12-05T17:41:39.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.550+0000 7fcf76ffd700 1 -- 172.21.15.32:0/3350381503 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825744970 0 0) 0x7fcf780b3c90 con 0x7fcf780b2fb0 2020-12-05T17:41:39.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.550+0000 7fcf76ffd700 1 -- 172.21.15.32:0/3350381503 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf780b2df0 con 0x7fcf780b2fb0 2020-12-05T17:41:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.550+0000 7fcf777fe700 1 -- 172.21.15.32:0/3350381503 >> v1:172.21.15.61:6789/0 conn(0x7fcf780b2fb0 legacy=0x7fcf78142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.550+0000 7fcf777fe700 1 -- 172.21.15.32:0/3350381503 >> v1:172.21.15.32:6789/0 conn(0x7fcf7805f3b0 legacy=0x7fcf7813c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.550+0000 7fcf777fe700 1 -- 172.21.15.32:0/3350381503 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf78142820 con 0x7fcf7805feb0 2020-12-05T17:41:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.550+0000 7fcf777fe700 1 --2- 172.21.15.32:0/3350381503 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf7805feb0 0x7fcf7805f790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fcf6c002f30 tx=0x7fcf6c006dc0).ready entity=mon.2 client_cookie=458c6d327679b609 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.550+0000 7fcf76ffd700 1 -- 172.21.15.32:0/3350381503 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf6c009730 con 0x7fcf7805feb0 2020-12-05T17:41:39.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.550+0000 7fcf76ffd700 1 -- 172.21.15.32:0/3350381503 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6c009890 con 0x7fcf7805feb0 2020-12-05T17:41:39.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.550+0000 7fcf76ffd700 1 -- 172.21.15.32:0/3350381503 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf6c009bf0 con 0x7fcf7805feb0 2020-12-05T17:41:39.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.550+0000 7fcf7d43c700 1 -- 172.21.15.32:0/3350381503 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf7805feb0 msgr2=0x7fcf7805f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:39.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.550+0000 7fcf7d43c700 1 --2- 172.21.15.32:0/3350381503 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf7805feb0 0x7fcf7805f790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fcf6c002f30 tx=0x7fcf6c006dc0).stop 2020-12-05T17:41:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.554+0000 7fcf7d43c700 1 -- 172.21.15.32:0/3350381503 shutdown_connections 2020-12-05T17:41:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.554+0000 7fcf7d43c700 1 --2- 172.21.15.32:0/3350381503 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf7805feb0 0x7fcf7805f790 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.554+0000 7fcf7d43c700 1 -- 172.21.15.32:0/3350381503 >> 172.21.15.32:0/3350381503 conn(0x7fcf780b8f20 msgr2=0x7fcf780b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.554+0000 7fcf7d43c700 1 -- 172.21.15.32:0/3350381503 shutdown_connections 2020-12-05T17:41:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.554+0000 7fcf7d43c700 1 -- 172.21.15.32:0/3350381503 wait complete. 2020-12-05T17:41:39.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.554+0000 7fcf7d43c700 1 Processor -- start 2020-12-05T17:41:39.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.554+0000 7fcf7d43c700 1 -- start start 2020-12-05T17:41:39.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.554+0000 7fcf7d43c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf7805f3b0 0x7fcf780651f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:39.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.554+0000 7fcf7d43c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf7805feb0 0x7fcf7813b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:39.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.554+0000 7fcf7d43c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf780b2fb0 0x7fcf781512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:39.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.554+0000 7fcf7d43c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7813dad0 con 0x7fcf780b2fb0 2020-12-05T17:41:39.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.554+0000 7fcf7d43c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7813dd70 con 0x7fcf7805feb0 2020-12-05T17:41:39.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.554+0000 7fcf7d43c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7813e010 con 0x7fcf7805f3b0 2020-12-05T17:41:39.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.554+0000 7fcf7cc3b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf780b2fb0 0x7fcf781512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:39.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.554+0000 7fcf7cc3b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf780b2fb0 0x7fcf781512b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41150/0 (socket says 172.21.15.32:41150) 2020-12-05T17:41:39.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.554+0000 7fcf7cc3b700 1 -- 172.21.15.32:0/12352558 learned_addr learned my addr 172.21.15.32:0/12352558 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:39.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.554+0000 7fcf777fe700 1 --2- 172.21.15.32:0/12352558 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf7805feb0 0x7fcf7813b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:39.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.554+0000 7fcf77fff700 1 --2- 172.21.15.32:0/12352558 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf7805f3b0 0x7fcf780651f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:39.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.558+0000 7fcf7cc3b700 1 -- 172.21.15.32:0/12352558 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf7805f3b0 msgr2=0x7fcf780651f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:39.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.558+0000 7fcf7cc3b700 1 --2- 172.21.15.32:0/12352558 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf7805f3b0 0x7fcf780651f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:39.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.558+0000 7fcf7cc3b700 1 -- 172.21.15.32:0/12352558 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf7805feb0 msgr2=0x7fcf7813b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:39.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.558+0000 7fcf7cc3b700 1 --2- 172.21.15.32:0/12352558 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf7805feb0 0x7fcf7813b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:39.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.558+0000 7fcf7cc3b700 1 -- 172.21.15.32:0/12352558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7813d7e0 con 0x7fcf780b2fb0 2020-12-05T17:41:39.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.558+0000 7fcf7cc3b700 1 --2- 172.21.15.32:0/12352558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf780b2fb0 0x7fcf781512b0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7fcf70001490 tx=0x7fcf70007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:39.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.558+0000 7fcf757fa700 1 -- 172.21.15.32:0/12352558 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf70002240 con 0x7fcf780b2fb0 2020-12-05T17:41:39.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.558+0000 7fcf757fa700 1 -- 172.21.15.32:0/12352558 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf700023a0 con 0x7fcf780b2fb0 2020-12-05T17:41:39.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.558+0000 7fcf757fa700 1 -- 172.21.15.32:0/12352558 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf7000a790 con 0x7fcf780b2fb0 2020-12-05T17:41:39.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.558+0000 7fcf7d43c700 1 -- 172.21.15.32:0/12352558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7813e4b0 con 0x7fcf780b2fb0 2020-12-05T17:41:39.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.558+0000 7fcf7d43c700 1 -- 172.21.15.32:0/12352558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7813bce0 con 0x7fcf780b2fb0 2020-12-05T17:41:39.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.558+0000 7fcf7d43c700 1 -- 172.21.15.32:0/12352558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf78057050 con 0x7fcf780b2fb0 2020-12-05T17:41:39.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.558+0000 7fcf757fa700 1 -- 172.21.15.32:0/12352558 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fcf70019030 con 0x7fcf780b2fb0 2020-12-05T17:41:39.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.562+0000 7fcf757fa700 1 --2- 172.21.15.32:0/12352558 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcf58055330 0x7fcf580577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.562+0000 7fcf77fff700 1 --2- 172.21.15.32:0/12352558 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcf58055330 0x7fcf580577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.562+0000 7fcf757fa700 1 -- 172.21.15.32:0/12352558 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcf7006d930 con 0x7fcf780b2fb0 2020-12-05T17:41:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.562+0000 7fcf77fff700 1 --2- 172.21.15.32:0/12352558 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcf58055330 0x7fcf580577f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7fcf6800b000 tx=0x7fcf68009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.566+0000 7fcf757fa700 1 -- 172.21.15.32:0/12352558 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fcf70043ad0 con 0x7fcf780b2fb0 2020-12-05T17:41:39.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.702+0000 7fcf7d43c700 1 -- 172.21.15.32:0/12352558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcf78064c90 con 0x7fcf780b2fb0 2020-12-05T17:41:39.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.702+0000 7fcf757fa700 1 -- 172.21.15.32:0/12352558 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v138) v1 ==== 104+0+10 (secure 0 0 0) 0x7fcf7006be60 con 0x7fcf780b2fb0 2020-12-05T17:41:39.705 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:41:39.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.706+0000 7fcf7d43c700 1 -- 172.21.15.32:0/12352558 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcf58055330 msgr2=0x7fcf580577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:39.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.706+0000 7fcf7d43c700 1 --2- 172.21.15.32:0/12352558 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcf58055330 0x7fcf580577f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7fcf6800b000 tx=0x7fcf68009040).stop 2020-12-05T17:41:39.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.706+0000 7fcf7d43c700 1 -- 172.21.15.32:0/12352558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf780b2fb0 msgr2=0x7fcf781512b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:39.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.706+0000 7fcf7d43c700 1 --2- 172.21.15.32:0/12352558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf780b2fb0 0x7fcf781512b0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7fcf70001490 tx=0x7fcf70007e00).stop 2020-12-05T17:41:39.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.706+0000 7fcf7d43c700 1 -- 172.21.15.32:0/12352558 shutdown_connections 2020-12-05T17:41:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.706+0000 7fcf7d43c700 1 --2- 172.21.15.32:0/12352558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf780b2fb0 0x7fcf781512b0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.706+0000 7fcf7d43c700 1 --2- 172.21.15.32:0/12352558 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcf58055330 0x7fcf580577f0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.706+0000 7fcf7d43c700 1 --2- 172.21.15.32:0/12352558 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf7805f3b0 0x7fcf780651f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.706+0000 7fcf7d43c700 1 --2- 172.21.15.32:0/12352558 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf7805feb0 0x7fcf7813b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.706+0000 7fcf7d43c700 1 -- 172.21.15.32:0/12352558 >> 172.21.15.32:0/12352558 conn(0x7fcf780b8f20 msgr2=0x7fcf78061780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:39.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.706+0000 7fcf7d43c700 1 -- 172.21.15.32:0/12352558 shutdown_connections 2020-12-05T17:41:39.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.706+0000 7fcf7d43c700 1 -- 172.21.15.32:0/12352558 wait complete. 2020-12-05T17:41:39.723 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:41:39.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.842+0000 7fa3075d7700 1 Processor -- start 2020-12-05T17:41:39.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.842+0000 7fa3075d7700 1 -- start start 2020-12-05T17:41:39.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.842+0000 7fa3075d7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3000b5f00 0x7fa3000b5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:39.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.842+0000 7fa3075d7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3000b1920 con 0x7fa3000b5b20 2020-12-05T17:41:39.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.842+0000 7fa3075d7700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3000b1bc0 con 0x7fa3000b6690 2020-12-05T17:41:39.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.842+0000 7fa3075d7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3000b5730 con 0x7fa3000b5f00 2020-12-05T17:41:39.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.842+0000 7fa305373700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa3000b5b20 0x7fa3000b2060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43802/0 (socket says 172.21.15.32:43802) 2020-12-05T17:41:39.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.842+0000 7fa305373700 1 -- 172.21.15.32:0/4073546206 learned_addr learned my addr 172.21.15.32:0/4073546206 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:39.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.842+0000 7fa305b74700 1 --2- 172.21.15.32:0/4073546206 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3000b5f00 0x7fa3000b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:39.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.842+0000 7fa2f7fff700 1 -- 172.21.15.32:0/4073546206 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1052704965 0 0) 0x7fa3000b1920 con 0x7fa3000b5b20 2020-12-05T17:41:39.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.842+0000 7fa2f7fff700 1 -- 172.21.15.32:0/4073546206 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2e4003230 con 0x7fa3000b5b20 2020-12-05T17:41:39.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.842+0000 7fa2f7fff700 1 -- 172.21.15.32:0/4073546206 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3200937533 0 0) 0x7fa3000b1bc0 con 0x7fa3000b6690 2020-12-05T17:41:39.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.842+0000 7fa305b74700 1 -- 172.21.15.32:0/4073546206 >> v1:172.21.15.61:6789/0 conn(0x7fa3000b6690 legacy=0x7fa300145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:39.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.842+0000 7fa305b74700 1 -- 172.21.15.32:0/4073546206 >> v1:172.21.15.32:6789/0 conn(0x7fa3000b5b20 legacy=0x7fa3000b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:39.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.842+0000 7fa305b74700 1 -- 172.21.15.32:0/4073546206 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa300146620 con 0x7fa3000b5f00 2020-12-05T17:41:39.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.842+0000 7fa305b74700 1 --2- 172.21.15.32:0/4073546206 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3000b5f00 0x7fa3000b5240 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fa2f80032b0 tx=0x7fa2f80091a0).ready entity=mon.2 client_cookie=1413d71f74d1a3da server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:39.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.842+0000 7fa2f7fff700 1 -- 172.21.15.32:0/4073546206 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2f800b9f0 con 0x7fa3000b5f00 2020-12-05T17:41:39.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.842+0000 7fa2f7fff700 1 -- 172.21.15.32:0/4073546206 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f800bb50 con 0x7fa3000b5f00 2020-12-05T17:41:39.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.842+0000 7fa2f7fff700 1 -- 172.21.15.32:0/4073546206 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2f800bcc0 con 0x7fa3000b5f00 2020-12-05T17:41:39.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.842+0000 7fa3075d7700 1 -- 172.21.15.32:0/4073546206 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3000b5f00 msgr2=0x7fa3000b5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:39.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.842+0000 7fa3075d7700 1 --2- 172.21.15.32:0/4073546206 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3000b5f00 0x7fa3000b5240 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fa2f80032b0 tx=0x7fa2f80091a0).stop 2020-12-05T17:41:39.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa3075d7700 1 -- 172.21.15.32:0/4073546206 shutdown_connections 2020-12-05T17:41:39.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa3075d7700 1 --2- 172.21.15.32:0/4073546206 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3000b5f00 0x7fa3000b5240 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:39.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa3075d7700 1 -- 172.21.15.32:0/4073546206 >> 172.21.15.32:0/4073546206 conn(0x7fa30005f3b0 msgr2=0x7fa30005f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:39.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa3075d7700 1 -- 172.21.15.32:0/4073546206 shutdown_connections 2020-12-05T17:41:39.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa3075d7700 1 -- 172.21.15.32:0/4073546206 wait complete. 2020-12-05T17:41:39.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa3075d7700 1 Processor -- start 2020-12-05T17:41:39.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa3075d7700 1 -- start start 2020-12-05T17:41:39.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa3075d7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3000b5b20 0x7fa3000a2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:39.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa3075d7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa3000b5f00 0x7fa300141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:39.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa3075d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3000b6690 0x7fa3000af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:39.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa3075d7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3000659d0 con 0x7fa3000b6690 2020-12-05T17:41:39.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa3075d7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa300065c00 con 0x7fa3000b5f00 2020-12-05T17:41:39.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa3075d7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa300065ea0 con 0x7fa3000b5b20 2020-12-05T17:41:39.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa305b74700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3000b6690 0x7fa3000af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:39.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa305b74700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3000b6690 0x7fa3000af0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41164/0 (socket says 172.21.15.32:41164) 2020-12-05T17:41:39.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa305b74700 1 -- 172.21.15.32:0/416975247 learned_addr learned my addr 172.21.15.32:0/416975247 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:39.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa305373700 1 --2- 172.21.15.32:0/416975247 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3000b5b20 0x7fa3000a2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:39.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa304b72700 1 --2- 172.21.15.32:0/416975247 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa3000b5f00 0x7fa300141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:39.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa305b74700 1 -- 172.21.15.32:0/416975247 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3000b5b20 msgr2=0x7fa3000a2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:39.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa305b74700 1 --2- 172.21.15.32:0/416975247 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3000b5b20 0x7fa3000a2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:39.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa305b74700 1 -- 172.21.15.32:0/416975247 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa3000b5f00 msgr2=0x7fa300141aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:39.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa305b74700 1 --2- 172.21.15.32:0/416975247 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa3000b5f00 0x7fa300141aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:39.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa305b74700 1 -- 172.21.15.32:0/416975247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa300066b90 con 0x7fa3000b6690 2020-12-05T17:41:39.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.846+0000 7fa305b74700 1 --2- 172.21.15.32:0/416975247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3000b6690 0x7fa3000af0c0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7fa2f80032b0 tx=0x7fa2f8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:39.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.850+0000 7fa2f67fc700 1 -- 172.21.15.32:0/416975247 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2f800bcc0 con 0x7fa3000b6690 2020-12-05T17:41:39.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.850+0000 7fa3075d7700 1 -- 172.21.15.32:0/416975247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa300066de0 con 0x7fa3000b6690 2020-12-05T17:41:39.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.850+0000 7fa2f67fc700 1 -- 172.21.15.32:0/416975247 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f80107a0 con 0x7fa3000b6690 2020-12-05T17:41:39.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.850+0000 7fa2f67fc700 1 -- 172.21.15.32:0/416975247 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2f8010b00 con 0x7fa3000b6690 2020-12-05T17:41:39.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.850+0000 7fa3075d7700 1 -- 172.21.15.32:0/416975247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3000a4ad0 con 0x7fa3000b6690 2020-12-05T17:41:39.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.850+0000 7fa3075d7700 1 -- 172.21.15.32:0/416975247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa300066060 con 0x7fa3000b6690 2020-12-05T17:41:39.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.850+0000 7fa2f67fc700 1 -- 172.21.15.32:0/416975247 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa2f8027030 con 0x7fa3000b6690 2020-12-05T17:41:39.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.850+0000 7fa2f67fc700 1 --2- 172.21.15.32:0/416975247 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa2e80554e0 0x7fa2e80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:39.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.850+0000 7fa2f67fc700 1 -- 172.21.15.32:0/416975247 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa2f806f1f0 con 0x7fa3000b6690 2020-12-05T17:41:39.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.850+0000 7fa305373700 1 --2- 172.21.15.32:0/416975247 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa2e80554e0 0x7fa2e80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:39.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.854+0000 7fa305373700 1 --2- 172.21.15.32:0/416975247 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa2e80554e0 0x7fa2e80579a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7fa2fc00b000 tx=0x7fa2fc009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:39.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.858+0000 7fa2f67fc700 1 -- 172.21.15.32:0/416975247 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa2f8045330 con 0x7fa3000b6690 2020-12-05T17:41:40.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.998+0000 7fa3075d7700 1 -- 172.21.15.32:0/416975247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa300057050 con 0x7fa3000b6690 2020-12-05T17:41:40.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:39.998+0000 7fa2f67fc700 1 -- 172.21.15.32:0/416975247 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v138) v1 ==== 100+0+10 (secure 0 0 0) 0x7fa2f80150a0 con 0x7fa3000b6690 2020-12-05T17:41:40.001 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:41:40.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.002+0000 7fa3075d7700 1 -- 172.21.15.32:0/416975247 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa2e80554e0 msgr2=0x7fa2e80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:40.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.002+0000 7fa3075d7700 1 --2- 172.21.15.32:0/416975247 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa2e80554e0 0x7fa2e80579a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7fa2fc00b000 tx=0x7fa2fc009040).stop 2020-12-05T17:41:40.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.002+0000 7fa3075d7700 1 -- 172.21.15.32:0/416975247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3000b6690 msgr2=0x7fa3000af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:40.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.002+0000 7fa3075d7700 1 --2- 172.21.15.32:0/416975247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3000b6690 0x7fa3000af0c0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7fa2f80032b0 tx=0x7fa2f8016000).stop 2020-12-05T17:41:40.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.002+0000 7fa3075d7700 1 -- 172.21.15.32:0/416975247 shutdown_connections 2020-12-05T17:41:40.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.002+0000 7fa3075d7700 1 --2- 172.21.15.32:0/416975247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3000b6690 0x7fa3000af0c0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:40.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.002+0000 7fa3075d7700 1 --2- 172.21.15.32:0/416975247 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa2e80554e0 0x7fa2e80579a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:40.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.002+0000 7fa3075d7700 1 --2- 172.21.15.32:0/416975247 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa3000b5b20 0x7fa3000a2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:40.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.002+0000 7fa3075d7700 1 --2- 172.21.15.32:0/416975247 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa3000b5f00 0x7fa300141aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:40.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.002+0000 7fa3075d7700 1 -- 172.21.15.32:0/416975247 >> 172.21.15.32:0/416975247 conn(0x7fa30005f3b0 msgr2=0x7fa30005f790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:40.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.002+0000 7fa3075d7700 1 -- 172.21.15.32:0/416975247 shutdown_connections 2020-12-05T17:41:40.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.002+0000 7fa3075d7700 1 -- 172.21.15.32:0/416975247 wait complete. 2020-12-05T17:41:40.021 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:41:40.022 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:41:40.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee1299b700 1 Processor -- start 2020-12-05T17:41:40.134 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee1299b700 1 -- start start 2020-12-05T17:41:40.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee1299b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fee0c0b2070 0x7fee0c0b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:40.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee1299b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fee0c138e30 0x7fee0c139270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:40.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee1299b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee0c142980 0x7fee0c142df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:40.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee1299b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee0c0acb80 con 0x7fee0c142980 2020-12-05T17:41:40.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee1299b700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee0c0b2450 con 0x7fee0c138e30 2020-12-05T17:41:40.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee1299b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c066d40 con 0x7fee0c0b2070 2020-12-05T17:41:40.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee1219a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee0c142980 0x7fee0c142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:40.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee1219a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee0c142980 0x7fee0c142df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41172/0 (socket says 172.21.15.32:41172) 2020-12-05T17:41:40.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee11999700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fee0c0b2070 0x7fee0c0b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:40.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee1219a700 1 -- 172.21.15.32:0/2699690969 learned_addr learned my addr 172.21.15.32:0/2699690969 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:40.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee11198700 1 --2- 172.21.15.32:0/2699690969 >> v2:172.21.15.61:3300/0 conn(0x7fee0c138e30 0x7fee0c139270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:40.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee1219a700 1 -- 172.21.15.32:0/2699690969 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fee0c0b2070 msgr2=0x7fee0c0b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:40.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee1219a700 1 --2- 172.21.15.32:0/2699690969 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fee0c0b2070 0x7fee0c0b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee1219a700 1 -- 172.21.15.32:0/2699690969 >> v2:172.21.15.61:3300/0 conn(0x7fee0c138e30 msgr2=0x7fee0c139270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee1219a700 1 --2- 172.21.15.32:0/2699690969 >> v2:172.21.15.61:3300/0 conn(0x7fee0c138e30 0x7fee0c139270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee1219a700 1 -- 172.21.15.32:0/2699690969 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0c066fe0 con 0x7fee0c142980 2020-12-05T17:41:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee1219a700 1 --2- 172.21.15.32:0/2699690969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c142980 0x7fee0c142df0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 rx=0x7fee040032b0 tx=0x7fee04009270).ready entity=mon.0 client_cookie=2e1bdc6795d75f03 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee10997700 1 -- 172.21.15.32:0/2699690969 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee04009e60 con 0x7fee0c142980 2020-12-05T17:41:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee10997700 1 -- 172.21.15.32:0/2699690969 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0400bb50 con 0x7fee0c142980 2020-12-05T17:41:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee1299b700 1 -- 172.21.15.32:0/2699690969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c142980 msgr2=0x7fee0c142df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee10997700 1 -- 172.21.15.32:0/2699690969 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee0400bcc0 con 0x7fee0c142980 2020-12-05T17:41:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.134+0000 7fee1299b700 1 --2- 172.21.15.32:0/2699690969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c142980 0x7fee0c142df0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 rx=0x7fee040032b0 tx=0x7fee04009270).stop 2020-12-05T17:41:40.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee1299b700 1 -- 172.21.15.32:0/2699690969 shutdown_connections 2020-12-05T17:41:40.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee1299b700 1 --2- 172.21.15.32:0/2699690969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c142980 0x7fee0c142df0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee1299b700 1 --2- 172.21.15.32:0/2699690969 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fee0c0b2070 0x7fee0c0b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee1299b700 1 --2- 172.21.15.32:0/2699690969 >> v2:172.21.15.61:3300/0 conn(0x7fee0c138e30 0x7fee0c139270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee1299b700 1 -- 172.21.15.32:0/2699690969 >> 172.21.15.32:0/2699690969 conn(0x7fee0c05bbb0 msgr2=0x7fee0c062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee1299b700 1 -- 172.21.15.32:0/2699690969 shutdown_connections 2020-12-05T17:41:40.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee1299b700 1 -- 172.21.15.32:0/2699690969 wait complete. 2020-12-05T17:41:40.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee1299b700 1 Processor -- start 2020-12-05T17:41:40.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee1299b700 1 -- start start 2020-12-05T17:41:40.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee1299b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee0c0b2070 0x7fee0c140bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:40.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee1299b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fee0c138e30 0x7fee0c145670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:40.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee1299b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c142980 0x7fee0c1529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:40.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee1299b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c145130 con 0x7fee0c142980 2020-12-05T17:41:40.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee1299b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c1453d0 con 0x7fee0c0b2070 2020-12-05T17:41:40.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee1299b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c147900 con 0x7fee0c138e30 2020-12-05T17:41:40.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee1219a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c142980 0x7fee0c1529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:40.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee1219a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c142980 0x7fee0c1529a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41178/0 (socket says 172.21.15.32:41178) 2020-12-05T17:41:40.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee11198700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fee0c138e30 0x7fee0c145670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:40.148 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee1219a700 1 -- 172.21.15.32:0/481269887 learned_addr learned my addr 172.21.15.32:0/481269887 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:40.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee11999700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee0c0b2070 0x7fee0c140bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:40.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee11999700 1 -- 172.21.15.32:0/481269887 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fee0c138e30 msgr2=0x7fee0c145670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:40.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee11999700 1 --2- 172.21.15.32:0/481269887 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fee0c138e30 0x7fee0c145670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:40.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee11999700 1 -- 172.21.15.32:0/481269887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c142980 msgr2=0x7fee0c1529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:40.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee11999700 1 --2- 172.21.15.32:0/481269887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c142980 0x7fee0c1529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:40.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee11999700 1 -- 172.21.15.32:0/481269887 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0c145b60 con 0x7fee0c0b2070 2020-12-05T17:41:40.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee11198700 1 --2- 172.21.15.32:0/481269887 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fee0c138e30 0x7fee0c145670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:40.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee1219a700 1 --2- 172.21.15.32:0/481269887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c142980 0x7fee0c1529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:40.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee11999700 1 --2- 172.21.15.32:0/481269887 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee0c0b2070 0x7fee0c140bc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fedfc00c000 tx=0x7fedfc006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:40.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee0affd700 1 -- 172.21.15.32:0/481269887 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedfc00ca10 con 0x7fee0c0b2070 2020-12-05T17:41:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee0affd700 1 -- 172.21.15.32:0/481269887 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedfc00cb70 con 0x7fee0c0b2070 2020-12-05T17:41:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.138+0000 7fee1299b700 1 -- 172.21.15.32:0/481269887 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee0c145fb0 con 0x7fee0c0b2070 2020-12-05T17:41:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.142+0000 7fee0affd700 1 -- 172.21.15.32:0/481269887 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedfc0123f0 con 0x7fee0c0b2070 2020-12-05T17:41:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.142+0000 7fee1299b700 1 -- 172.21.15.32:0/481269887 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee0c13ae80 con 0x7fee0c0b2070 2020-12-05T17:41:40.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.142+0000 7fee1299b700 1 -- 172.21.15.32:0/481269887 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee0c13b0d0 con 0x7fee0c0b2070 2020-12-05T17:41:40.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.142+0000 7fee0affd700 1 -- 172.21.15.32:0/481269887 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fedfc012550 con 0x7fee0c0b2070 2020-12-05T17:41:40.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.142+0000 7fee0affd700 1 --2- 172.21.15.32:0/481269887 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fedec055440 0x7fedec057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:40.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.142+0000 7fee0affd700 1 -- 172.21.15.32:0/481269887 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fedfc06e3f0 con 0x7fee0c0b2070 2020-12-05T17:41:40.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.142+0000 7fee11198700 1 --2- 172.21.15.32:0/481269887 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fedec055440 0x7fedec057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:40.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.142+0000 7fee11198700 1 --2- 172.21.15.32:0/481269887 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fedec055440 0x7fedec057900 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7fee0000b000 tx=0x7fee00006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:40.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.150+0000 7fee0affd700 1 -- 172.21.15.32:0/481269887 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fedfc044530 con 0x7fee0c0b2070 2020-12-05T17:41:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.334+0000 7fee1299b700 1 -- 172.21.15.32:0/481269887 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee0c057050 con 0x7fee0c0b2070 2020-12-05T17:41:40.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.334+0000 7fee0affd700 1 -- 172.21.15.32:0/481269887 <== mon.1 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+2142 (secure 0 0 0) 0x7fedfc047b50 con 0x7fee0c0b2070 2020-12-05T17:41:40.337 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:40.338 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":66,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/3049192140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3049192140},{"type":"v1","addr":"172.21.15.61:6833","nonce":3049192140}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:41:32.383702+0000","modified":"2020-12-05T17:41:33.387236+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8274":{"gid":8274,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/2808222868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2808222868},{"type":"v1","addr":"172.21.15.32:6835","nonce":2808222868}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2020-12-05T17:41:40.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.338+0000 7fee1299b700 1 -- 172.21.15.32:0/481269887 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fedec055440 msgr2=0x7fedec057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:40.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.338+0000 7fee1299b700 1 --2- 172.21.15.32:0/481269887 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fedec055440 0x7fedec057900 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7fee0000b000 tx=0x7fee00006dc0).stop 2020-12-05T17:41:40.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.338+0000 7fee1299b700 1 -- 172.21.15.32:0/481269887 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee0c0b2070 msgr2=0x7fee0c140bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:40.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.338+0000 7fee1299b700 1 --2- 172.21.15.32:0/481269887 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee0c0b2070 0x7fee0c140bc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fedfc00c000 tx=0x7fedfc006dc0).stop 2020-12-05T17:41:40.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.338+0000 7fee1299b700 1 -- 172.21.15.32:0/481269887 shutdown_connections 2020-12-05T17:41:40.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.338+0000 7fee1299b700 1 --2- 172.21.15.32:0/481269887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c142980 0x7fee0c1529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:40.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.338+0000 7fee1299b700 1 --2- 172.21.15.32:0/481269887 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee0c0b2070 0x7fee0c140bc0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:40.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.338+0000 7fee1299b700 1 --2- 172.21.15.32:0/481269887 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fedec055440 0x7fedec057900 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:40.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.338+0000 7fee1299b700 1 --2- 172.21.15.32:0/481269887 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fee0c138e30 0x7fee0c145670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:40.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.338+0000 7fee1299b700 1 -- 172.21.15.32:0/481269887 >> 172.21.15.32:0/481269887 conn(0x7fee0c05bbb0 msgr2=0x7fee0c0accd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:40.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.338+0000 7fee1299b700 1 -- 172.21.15.32:0/481269887 shutdown_connections 2020-12-05T17:41:40.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:40.338+0000 7fee1299b700 1 -- 172.21.15.32:0/481269887 wait complete. 2020-12-05T17:41:40.344 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 66 2020-12-05T17:41:40.557 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:41:32.383702+0000', 'modified': '2020-12-05T17:41:33.387236+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8274}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8274': {'gid': 8274, 'name': 'a', 'rank': 0, 'incarnation': 65, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/2808222868', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 2808222868}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 2808222868}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:41:40.558 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:41:40.558 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:41:40.688 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":8274,"want_state":"up:active","state":"up:active","rank_uptime":8.2913478690000009,"mdsmap_epoch":66,"osdmap_epoch":138,"osdmap_epoch_barrier":138,"uptime":12.511149142000001} 2020-12-05T17:41:40.689 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 8274, "mdsmap_epoch": 66, "osdmap_epoch": 138, "osdmap_epoch_barrier": 138, "rank_uptime": 8.291347869, "state": "up:active", "uptime": 12.511149142, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:41:40.689 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:41:40.689 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-12-05T17:41:40.690 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi002.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-12-05T17:41:40.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:40.691 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:40.699 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:41:40.757 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:41:40.758 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:41:40.816 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:41:40.817 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:41:40.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:40.818 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:41:40.864 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:41:40.864 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:40.908 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:41:40.923 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:41:40.923 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:41:40.980 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:41:40.982 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:41:40.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:40.983 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:41:40.985 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:2020-12-05T17:41:40.980+0000 7f832617d0c0 -1 init, newargv = 0x56159a2f01f0 newargc=9 2020-12-05T17:41:40.985 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[11197]: starting ceph client 2020-12-05T17:41:40.997 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[11197]: starting fuse 2020-12-05T17:41:42.091 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:41:42.108 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:41:42.108 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:41:42.165 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:41:42.166 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:41:42.167 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:42.167 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:41:42.220 INFO:teuthology.orchestra.run.smithi002.stdout:50 2020-12-05T17:41:42.221 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [50] 2020-12-05T17:41:42.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:42.222 INFO:teuthology.orchestra.run.smithi002:> sudo python3 -c ' 2020-12-05T17:41:42.222 INFO:teuthology.orchestra.run.smithi002:> import glob 2020-12-05T17:41:42.222 INFO:teuthology.orchestra.run.smithi002:> import re 2020-12-05T17:41:42.223 INFO:teuthology.orchestra.run.smithi002:> import os 2020-12-05T17:41:42.223 INFO:teuthology.orchestra.run.smithi002:> import subprocess 2020-12-05T17:41:42.224 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:41:42.224 INFO:teuthology.orchestra.run.smithi002:> def find_socket(client_name): 2020-12-05T17:41:42.224 INFO:teuthology.orchestra.run.smithi002:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-12-05T17:41:42.225 INFO:teuthology.orchestra.run.smithi002:> files = glob.glob(asok_path) 2020-12-05T17:41:42.225 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:41:42.226 INFO:teuthology.orchestra.run.smithi002:> # Given a non-glob path, it better be there 2020-12-05T17:41:42.226 INFO:teuthology.orchestra.run.smithi002:> if "*" not in asok_path: 2020-12-05T17:41:42.226 INFO:teuthology.orchestra.run.smithi002:> assert(len(files) == 1) 2020-12-05T17:41:42.227 INFO:teuthology.orchestra.run.smithi002:> return files[0] 2020-12-05T17:41:42.227 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:41:42.228 INFO:teuthology.orchestra.run.smithi002:> for f in files: 2020-12-05T17:41:42.228 INFO:teuthology.orchestra.run.smithi002:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:41:42.228 INFO:teuthology.orchestra.run.smithi002:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:41:42.229 INFO:teuthology.orchestra.run.smithi002:> return f 2020-12-05T17:41:42.229 INFO:teuthology.orchestra.run.smithi002:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:41:42.230 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:41:42.230 INFO:teuthology.orchestra.run.smithi002:> print(find_socket("client.1")) 2020-12-05T17:41:42.231 INFO:teuthology.orchestra.run.smithi002:> ' 2020-12-05T17:41:42.313 INFO:teuthology.orchestra.run.smithi002.stdout:/var/run/ceph/ceph-client.1.11197.asok 2020-12-05T17:41:42.318 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.11197.asok 2020-12-05T17:41:42.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:42.319 INFO:teuthology.orchestra.run.smithi002:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.11197.asok status 2020-12-05T17:41:42.416 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2020-12-05T17:41:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "metadata": { 2020-12-05T17:41:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:41:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:41:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "entity_id": "1", 2020-12-05T17:41:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": "smithi002", 2020-12-05T17:41:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-12-05T17:41:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: "pid": "11197", 2020-12-05T17:41:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: "root": "/" 2020-12-05T17:41:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:41:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_count": 0, 2020-12-05T17:41:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_pinned_count": 0, 2020-12-05T17:41:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 7244, 2020-12-05T17:41:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "inst": { 2020-12-05T17:41:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "name": { 2020-12-05T17:41:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "client", 2020-12-05T17:41:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "num": 7244 2020-12-05T17:41:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:41:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2020-12-05T17:41:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2020-12-05T17:41:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2020-12-05T17:41:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2635720098 2020-12-05T17:41:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: } 2020-12-05T17:41:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:41:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2020-12-05T17:41:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2020-12-05T17:41:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2020-12-05T17:41:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2635720098 2020-12-05T17:41:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:41:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: "inst_str": "client.7244 172.21.15.2:0/2635720098", 2020-12-05T17:41:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "addr_str": "172.21.15.2:0/2635720098", 2020-12-05T17:41:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "inode_count": 1, 2020-12-05T17:41:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_epoch": 66, 2020-12-05T17:41:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch": 138, 2020-12-05T17:41:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:41:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: "blacklisted": false 2020-12-05T17:41:42.424 INFO:teuthology.orchestra.run.smithi002.stdout:} 2020-12-05T17:41:42.426 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:42.426 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:42.427 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:42.433 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:41:42.434 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:41:42.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:42.435 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:42.495 INFO:teuthology.orchestra.run.smithi036:> cd /home/ubuntu/cephtest/mnt.0 && sudo getfattr --only-values -n ceph.dir.rctime . 2020-12-05T17:41:42.535 INFO:teuthology.orchestra.run.smithi036.stdout:1607190102.492629769 2020-12-05T17:41:42.535 INFO:tasks.cephfs.test_misc:rctime = 1607190102.492629769 2020-12-05T17:41:42.535 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:41:42.536 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:41:42.554 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:41:42.572 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:41:42.591 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:41:42.607 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:41:42.624 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:41:42.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:42.626 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:42.627 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:41:42.638 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:41:42.639 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:41:42.640 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi036.front.sepia.ceph.com... 2020-12-05T17:41:42.640 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:41:42.641 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:41:42.744 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[11682]: fuse finished with error 0 and tester_r 0 2020-12-05T17:41:42.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:42.746 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:42.747 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:41:42.754 INFO:teuthology.orchestra.run.smithi036.stdout:ext2/ext3 2020-12-05T17:41:42.755 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:41:42.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:41:42.846 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:41:42.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:42.854 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:42.854 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:42.862 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:41:42.863 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:41:42.863 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi002.front.sepia.ceph.com... 2020-12-05T17:41:42.864 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:41:42.864 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:42.943 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[11197]: fuse finished with error 0 and tester_r 0 2020-12-05T17:41:42.944 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:42.945 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:42.945 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:43.024 INFO:teuthology.orchestra.run.smithi002.stdout:ext2/ext3 2020-12-05T17:41:43.025 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:41:43.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:41:43.026 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:43.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:43.073 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:43.073 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:41:43.080 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:41:43.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:41:43.081 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2020-12-05T17:41:43.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:41:43.082 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:41:43.128 INFO:teuthology.orchestra.run.smithi053.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:41:43.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:41:43.130 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2020-12-05T17:41:43.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.254+0000 7f75b2413700 1 Processor -- start 2020-12-05T17:41:43.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.254+0000 7f75b2413700 1 -- start start 2020-12-05T17:41:43.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.254+0000 7f75b2413700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f75ac0b5f00 0x7f75ac0b5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:43.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.254+0000 7f75b2413700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75ac0b1920 con 0x7f75ac0b6690 2020-12-05T17:41:43.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.254+0000 7f75b2413700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75ac0b1bc0 con 0x7f75ac0b5b20 2020-12-05T17:41:43.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.254+0000 7f75b2413700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac0b5730 con 0x7f75ac0b5f00 2020-12-05T17:41:43.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.254+0000 7f75ab7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f75ac0b6690 0x7f75ac145f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43832/0 (socket says 172.21.15.32:43832) 2020-12-05T17:41:43.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.254+0000 7f75ab7fe700 1 -- 172.21.15.32:0/538856124 learned_addr learned my addr 172.21.15.32:0/538856124 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:43.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.254+0000 7f75b09b0700 1 --2- 172.21.15.32:0/538856124 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f75ac0b5f00 0x7f75ac0b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:43.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.254+0000 7f75aaffd700 1 -- 172.21.15.32:0/538856124 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2935760336 0 0) 0x7f75ac0b1920 con 0x7f75ac0b6690 2020-12-05T17:41:43.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.254+0000 7f75aaffd700 1 -- 172.21.15.32:0/538856124 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f758c003230 con 0x7f75ac0b6690 2020-12-05T17:41:43.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.254+0000 7f75aaffd700 1 -- 172.21.15.32:0/538856124 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 717370645 0 0) 0x7f75ac0b1bc0 con 0x7f75ac0b5b20 2020-12-05T17:41:43.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.254+0000 7f75aaffd700 1 -- 172.21.15.32:0/538856124 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75ac0b1920 con 0x7f75ac0b5b20 2020-12-05T17:41:43.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.254+0000 7f75b09b0700 1 -- 172.21.15.32:0/538856124 >> v1:172.21.15.61:6789/0 conn(0x7f75ac0b5b20 legacy=0x7f75ac0b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:43.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.254+0000 7f75b09b0700 1 -- 172.21.15.32:0/538856124 >> v1:172.21.15.32:6789/0 conn(0x7f75ac0b6690 legacy=0x7f75ac145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:43.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.254+0000 7f75b09b0700 1 -- 172.21.15.32:0/538856124 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ac146620 con 0x7f75ac0b5f00 2020-12-05T17:41:43.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.258+0000 7f75b09b0700 1 --2- 172.21.15.32:0/538856124 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f75ac0b5f00 0x7f75ac0b5240 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f75a40032b0 tx=0x7f75a40091a0).ready entity=mon.2 client_cookie=65e46f53590157c0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:43.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.258+0000 7f75aaffd700 1 -- 172.21.15.32:0/538856124 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75a400b940 con 0x7f75ac0b5f00 2020-12-05T17:41:43.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.258+0000 7f75aaffd700 1 -- 172.21.15.32:0/538856124 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a400baa0 con 0x7f75ac0b5f00 2020-12-05T17:41:43.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.258+0000 7f75aaffd700 1 -- 172.21.15.32:0/538856124 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75a400bc10 con 0x7f75ac0b5f00 2020-12-05T17:41:43.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.258+0000 7f75b2413700 1 -- 172.21.15.32:0/538856124 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f75ac0b5f00 msgr2=0x7f75ac0b5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:43.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.258+0000 7f75b2413700 1 --2- 172.21.15.32:0/538856124 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f75ac0b5f00 0x7f75ac0b5240 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f75a40032b0 tx=0x7f75a40091a0).stop 2020-12-05T17:41:43.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.258+0000 7f75b2413700 1 -- 172.21.15.32:0/538856124 shutdown_connections 2020-12-05T17:41:43.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.258+0000 7f75b2413700 1 --2- 172.21.15.32:0/538856124 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f75ac0b5f00 0x7f75ac0b5240 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:43.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.258+0000 7f75b2413700 1 -- 172.21.15.32:0/538856124 >> 172.21.15.32:0/538856124 conn(0x7f75ac05f3b0 msgr2=0x7f75ac05f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:43.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.258+0000 7f75b2413700 1 -- 172.21.15.32:0/538856124 shutdown_connections 2020-12-05T17:41:43.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.258+0000 7f75b2413700 1 -- 172.21.15.32:0/538856124 wait complete. 2020-12-05T17:41:43.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.258+0000 7f75b2413700 1 Processor -- start 2020-12-05T17:41:43.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.258+0000 7f75b2413700 1 -- start start 2020-12-05T17:41:43.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.262+0000 7f75b2413700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75ac0b5b20 0x7f75ac0a2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:43.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.262+0000 7f75b2413700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f75ac0b5f00 0x7f75ac064e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:43.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.262+0000 7f75b2413700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f75ac0b6690 0x7f75ac0b7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:43.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.262+0000 7f75b2413700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac0633c0 con 0x7f75ac0b5b20 2020-12-05T17:41:43.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.262+0000 7f75b2413700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac063660 con 0x7f75ac0b5f00 2020-12-05T17:41:43.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.262+0000 7f75b2413700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac064990 con 0x7f75ac0b6690 2020-12-05T17:41:43.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.262+0000 7f75abfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75ac0b5b20 0x7f75ac0a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:43.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.262+0000 7f75abfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75ac0b5b20 0x7f75ac0a2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41188/0 (socket says 172.21.15.32:41188) 2020-12-05T17:41:43.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.262+0000 7f75abfff700 1 -- 172.21.15.32:0/4037544645 learned_addr learned my addr 172.21.15.32:0/4037544645 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:43.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.262+0000 7f75ab7fe700 1 --2- 172.21.15.32:0/4037544645 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f75ac0b5f00 0x7f75ac064e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:43.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.262+0000 7f75b09b0700 1 --2- 172.21.15.32:0/4037544645 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f75ac0b6690 0x7f75ac0b7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:43.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.262+0000 7f75abfff700 1 -- 172.21.15.32:0/4037544645 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f75ac0b6690 msgr2=0x7f75ac0b7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:43.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.262+0000 7f75abfff700 1 --2- 172.21.15.32:0/4037544645 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f75ac0b6690 0x7f75ac0b7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:43.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.262+0000 7f75abfff700 1 -- 172.21.15.32:0/4037544645 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f75ac0b5f00 msgr2=0x7f75ac064e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:43.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.262+0000 7f75abfff700 1 --2- 172.21.15.32:0/4037544645 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f75ac0b5f00 0x7f75ac064e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:43.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.262+0000 7f75abfff700 1 -- 172.21.15.32:0/4037544645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ac063b60 con 0x7f75ac0b5b20 2020-12-05T17:41:43.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.262+0000 7f75abfff700 1 --2- 172.21.15.32:0/4037544645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75ac0b5b20 0x7f75ac0a2870 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f75940010d0 tx=0x7f7594007af0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:43.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.262+0000 7f75a97fa700 1 -- 172.21.15.32:0/4037544645 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f759400f470 con 0x7f75ac0b5b20 2020-12-05T17:41:43.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.262+0000 7f75b2413700 1 -- 172.21.15.32:0/4037544645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ac063040 con 0x7f75ac0b5b20 2020-12-05T17:41:43.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.262+0000 7f75a97fa700 1 -- 172.21.15.32:0/4037544645 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759400f5d0 con 0x7f75ac0b5b20 2020-12-05T17:41:43.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.262+0000 7f75b2413700 1 -- 172.21.15.32:0/4037544645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ac0654c0 con 0x7f75ac0b5b20 2020-12-05T17:41:43.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.262+0000 7f75a97fa700 1 -- 172.21.15.32:0/4037544645 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f759400f930 con 0x7f75ac0b5b20 2020-12-05T17:41:43.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.262+0000 7f75a97fa700 1 -- 172.21.15.32:0/4037544645 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7594019030 con 0x7f75ac0b5b20 2020-12-05T17:41:43.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.266+0000 7f75b2413700 1 -- 172.21.15.32:0/4037544645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75ac057050 con 0x7f75ac0b5b20 2020-12-05T17:41:43.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.266+0000 7f75a97fa700 1 --2- 172.21.15.32:0/4037544645 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f75900554e0 0x7f75900579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:43.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.266+0000 7f75ab7fe700 1 --2- 172.21.15.32:0/4037544645 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f75900554e0 0x7f75900579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:43.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.266+0000 7f75a97fa700 1 -- 172.21.15.32:0/4037544645 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f759406d4d0 con 0x7f75ac0b5b20 2020-12-05T17:41:43.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.266+0000 7f75ab7fe700 1 --2- 172.21.15.32:0/4037544645 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f75900554e0 0x7f75900579a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f759c00d000 tx=0x7f759c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:43.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.274+0000 7f75a97fa700 1 -- 172.21.15.32:0/4037544645 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7594043530 con 0x7f75ac0b5b20 2020-12-05T17:41:43.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.414+0000 7f75b2413700 1 -- 172.21.15.32:0/4037544645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f75ac05bae0 con 0x7f75ac0b5b20 2020-12-05T17:41:43.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.422+0000 7f75a97fa700 1 -- 172.21.15.32:0/4037544645 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7594017020 con 0x7f75ac0b5b20 2020-12-05T17:41:43.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.426+0000 7f75b2413700 1 -- 172.21.15.32:0/4037544645 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f75900554e0 msgr2=0x7f75900579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:43.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.426+0000 7f75b2413700 1 --2- 172.21.15.32:0/4037544645 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f75900554e0 0x7f75900579a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f759c00d000 tx=0x7f759c009040).stop 2020-12-05T17:41:43.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.426+0000 7f75b2413700 1 -- 172.21.15.32:0/4037544645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75ac0b5b20 msgr2=0x7f75ac0a2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:43.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.426+0000 7f75b2413700 1 --2- 172.21.15.32:0/4037544645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75ac0b5b20 0x7f75ac0a2870 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f75940010d0 tx=0x7f7594007af0).stop 2020-12-05T17:41:43.427 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.426+0000 7f75b2413700 1 -- 172.21.15.32:0/4037544645 shutdown_connections 2020-12-05T17:41:43.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.426+0000 7f75b2413700 1 --2- 172.21.15.32:0/4037544645 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f75900554e0 0x7f75900579a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:43.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.426+0000 7f75b2413700 1 --2- 172.21.15.32:0/4037544645 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f75ac0b6690 0x7f75ac0b7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:43.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.426+0000 7f75b2413700 1 --2- 172.21.15.32:0/4037544645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75ac0b5b20 0x7f75ac0a2870 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:43.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.426+0000 7f75b2413700 1 --2- 172.21.15.32:0/4037544645 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f75ac0b5f00 0x7f75ac064e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:43.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.426+0000 7f75b2413700 1 -- 172.21.15.32:0/4037544645 >> 172.21.15.32:0/4037544645 conn(0x7f75ac05f3b0 msgr2=0x7f75ac0b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:43.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.426+0000 7f75b2413700 1 -- 172.21.15.32:0/4037544645 shutdown_connections 2020-12-05T17:41:43.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.426+0000 7f75b2413700 1 -- 172.21.15.32:0/4037544645 wait complete. 2020-12-05T17:41:43.439 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime' 2020-12-05T17:41:43.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.546+0000 7f049ccc0700 1 Processor -- start 2020-12-05T17:41:43.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.546+0000 7f049ccc0700 1 -- start start 2020-12-05T17:41:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.546+0000 7f049ccc0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0498062500 0x7f049805fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:43.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.546+0000 7f049ccc0700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0498061910 0x7f0498144830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:43.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.546+0000 7f049ccc0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04980628e0 con 0x7f0498141e70 2020-12-05T17:41:43.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.546+0000 7f049ccc0700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0498060260 con 0x7f0498061910 2020-12-05T17:41:43.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.546+0000 7f049ccc0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04981440f0 con 0x7f0498062500 2020-12-05T17:41:43.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.546+0000 7f0496d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0498141e70 0x7f04980ae340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43848/0 (socket says 172.21.15.32:43848) 2020-12-05T17:41:43.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.546+0000 7f0496d9d700 1 -- 172.21.15.32:0/4099037365 learned_addr learned my addr 172.21.15.32:0/4099037365 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:43.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.546+0000 7f049659c700 1 --2- 172.21.15.32:0/4099037365 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0498062500 0x7f049805fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:43.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.546+0000 7f0495d9b700 1 --2- 172.21.15.32:0/4099037365 >> v2:172.21.15.61:3300/0 conn(0x7f0498061910 0x7f0498144830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:43.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.546+0000 7f049559a700 1 -- 172.21.15.32:0/4099037365 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3074639988 0 0) 0x7f04980628e0 con 0x7f0498141e70 2020-12-05T17:41:43.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.546+0000 7f049559a700 1 -- 172.21.15.32:0/4099037365 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0478003230 con 0x7f0498141e70 2020-12-05T17:41:43.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.546+0000 7f049559a700 1 -- 172.21.15.32:0/4099037365 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1971765845 0 0) 0x7f0478003230 con 0x7f0498141e70 2020-12-05T17:41:43.555 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.546+0000 7f0495d9b700 1 -- 172.21.15.32:0/4099037365 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0498062500 msgr2=0x7f049805fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:43.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.546+0000 7f0495d9b700 1 --2- 172.21.15.32:0/4099037365 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0498062500 0x7f049805fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:43.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.546+0000 7f0495d9b700 1 -- 172.21.15.32:0/4099037365 >> v1:172.21.15.32:6789/0 conn(0x7f0498141e70 legacy=0x7f04980ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:43.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.546+0000 7f0495d9b700 1 -- 172.21.15.32:0/4099037365 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0498144390 con 0x7f0498061910 2020-12-05T17:41:43.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.546+0000 7f049659c700 1 --2- 172.21.15.32:0/4099037365 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0498062500 0x7f049805fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:43.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.546+0000 7f0495d9b700 1 --2- 172.21.15.32:0/4099037365 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0498061910 0x7f0498144830 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f0480001870 tx=0x7f04900091a0).ready entity=mon.1 client_cookie=620e3a79a00ccdb server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:43.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.550+0000 7f049559a700 1 -- 172.21.15.32:0/4099037365 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04800016d0 con 0x7f0498061910 2020-12-05T17:41:43.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.550+0000 7f049559a700 1 -- 172.21.15.32:0/4099037365 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049000b800 con 0x7f0498061910 2020-12-05T17:41:43.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.550+0000 7f049559a700 1 -- 172.21.15.32:0/4099037365 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f049000b970 con 0x7f0498061910 2020-12-05T17:41:43.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.550+0000 7f049ccc0700 1 -- 172.21.15.32:0/4099037365 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0498061910 msgr2=0x7f0498144830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:43.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.550+0000 7f049ccc0700 1 --2- 172.21.15.32:0/4099037365 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0498061910 0x7f0498144830 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f0480001870 tx=0x7f04900091a0).stop 2020-12-05T17:41:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.550+0000 7f049ccc0700 1 -- 172.21.15.32:0/4099037365 shutdown_connections 2020-12-05T17:41:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.550+0000 7f049ccc0700 1 --2- 172.21.15.32:0/4099037365 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0498062500 0x7f049805fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:43.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.550+0000 7f049ccc0700 1 --2- 172.21.15.32:0/4099037365 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0498061910 0x7f0498144830 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:43.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.550+0000 7f049ccc0700 1 -- 172.21.15.32:0/4099037365 >> 172.21.15.32:0/4099037365 conn(0x7f04980b57c0 msgr2=0x7f049805f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:43.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.550+0000 7f049ccc0700 1 -- 172.21.15.32:0/4099037365 shutdown_connections 2020-12-05T17:41:43.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.550+0000 7f049ccc0700 1 -- 172.21.15.32:0/4099037365 wait complete. 2020-12-05T17:41:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f049ccc0700 1 Processor -- start 2020-12-05T17:41:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f049ccc0700 1 -- start start 2020-12-05T17:41:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f049ccc0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0498061910 0x7f049809ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f049ccc0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0498062500 0x7f049813df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f049ccc0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0498141e70 0x7f049813e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f049ccc0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049814f560 con 0x7f0498061910 2020-12-05T17:41:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f049ccc0700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049814f800 con 0x7f0498062500 2020-12-05T17:41:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f049ccc0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049814faa0 con 0x7f0498141e70 2020-12-05T17:41:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f049659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0498061910 0x7f049809ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f049659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0498061910 0x7f049809ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41202/0 (socket says 172.21.15.32:41202) 2020-12-05T17:41:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f049659c700 1 -- 172.21.15.32:0/2447847745 learned_addr learned my addr 172.21.15.32:0/2447847745 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f0495d9b700 1 --2- 172.21.15.32:0/2447847745 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0498062500 0x7f049813df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f0496d9d700 1 --2- 172.21.15.32:0/2447847745 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0498141e70 0x7f049813e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:43.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f049659c700 1 -- 172.21.15.32:0/2447847745 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0498141e70 msgr2=0x7f049813e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:43.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f049659c700 1 --2- 172.21.15.32:0/2447847745 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0498141e70 0x7f049813e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:43.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f049659c700 1 -- 172.21.15.32:0/2447847745 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0498062500 msgr2=0x7f049813df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:43.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f049659c700 1 --2- 172.21.15.32:0/2447847745 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0498062500 0x7f049813df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:43.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f049659c700 1 -- 172.21.15.32:0/2447847745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049814f250 con 0x7f0498061910 2020-12-05T17:41:43.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f049659c700 1 --2- 172.21.15.32:0/2447847745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0498061910 0x7f049809ecf0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7f048800c000 tx=0x7f0488006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:43.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f048f7fe700 1 -- 172.21.15.32:0/2447847745 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f048800cb40 con 0x7f0498061910 2020-12-05T17:41:43.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f048f7fe700 1 -- 172.21.15.32:0/2447847745 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048800cca0 con 0x7f0498061910 2020-12-05T17:41:43.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f049ccc0700 1 -- 172.21.15.32:0/2447847745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f049813e920 con 0x7f0498061910 2020-12-05T17:41:43.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f048f7fe700 1 -- 172.21.15.32:0/2447847745 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04880123f0 con 0x7f0498061910 2020-12-05T17:41:43.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.554+0000 7f049ccc0700 1 -- 172.21.15.32:0/2447847745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f049814ff40 con 0x7f0498061910 2020-12-05T17:41:43.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.558+0000 7f049ccc0700 1 -- 172.21.15.32:0/2447847745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0498057050 con 0x7f0498061910 2020-12-05T17:41:43.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.558+0000 7f048f7fe700 1 -- 172.21.15.32:0/2447847745 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0488012550 con 0x7f0498061910 2020-12-05T17:41:43.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.558+0000 7f048f7fe700 1 --2- 172.21.15.32:0/2447847745 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f047c055440 0x7f047c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:43.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.558+0000 7f048f7fe700 1 -- 172.21.15.32:0/2447847745 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f048806e830 con 0x7f0498061910 2020-12-05T17:41:43.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.558+0000 7f0495d9b700 1 --2- 172.21.15.32:0/2447847745 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f047c055440 0x7f047c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:43.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.558+0000 7f0495d9b700 1 --2- 172.21.15.32:0/2447847745 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f047c055440 0x7f047c057900 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f049001b000 tx=0x7f0490016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:43.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.566+0000 7f048f7fe700 1 -- 172.21.15.32:0/2447847745 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0488044860 con 0x7f0498061910 2020-12-05T17:41:43.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:43.758+0000 7f049ccc0700 1 -- 172.21.15.32:0/2447847745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]} v 0) v1 -- 0x7f0498069690 con 0x7f0498061910 2020-12-05T17:41:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.098+0000 7f048f7fe700 1 -- 172.21.15.32:0/2447847745 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]}]=0 v465) v1 ==== 127+0+0 (secure 0 0 0) 0x7f0488047e80 con 0x7f0498061910 2020-12-05T17:41:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.102+0000 7f049ccc0700 1 -- 172.21.15.32:0/2447847745 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f047c055440 msgr2=0x7f047c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:44.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.102+0000 7f049ccc0700 1 --2- 172.21.15.32:0/2447847745 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f047c055440 0x7f047c057900 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f049001b000 tx=0x7f0490016000).stop 2020-12-05T17:41:44.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.102+0000 7f049ccc0700 1 -- 172.21.15.32:0/2447847745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0498061910 msgr2=0x7f049809ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:44.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.102+0000 7f049ccc0700 1 --2- 172.21.15.32:0/2447847745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0498061910 0x7f049809ecf0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7f048800c000 tx=0x7f0488006dc0).stop 2020-12-05T17:41:44.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.102+0000 7f049ccc0700 1 -- 172.21.15.32:0/2447847745 shutdown_connections 2020-12-05T17:41:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.102+0000 7f049ccc0700 1 --2- 172.21.15.32:0/2447847745 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f047c055440 0x7f047c057900 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.102+0000 7f049ccc0700 1 --2- 172.21.15.32:0/2447847745 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0498141e70 0x7f049813e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.102+0000 7f049ccc0700 1 --2- 172.21.15.32:0/2447847745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0498061910 0x7f049809ecf0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.102+0000 7f049ccc0700 1 --2- 172.21.15.32:0/2447847745 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0498062500 0x7f049813df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.102+0000 7f049ccc0700 1 -- 172.21.15.32:0/2447847745 >> 172.21.15.32:0/2447847745 conn(0x7f04980b57c0 msgr2=0x7f04980b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.102+0000 7f049ccc0700 1 -- 172.21.15.32:0/2447847745 shutdown_connections 2020-12-05T17:41:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.102+0000 7f049ccc0700 1 -- 172.21.15.32:0/2447847745 wait complete. 2020-12-05T17:41:44.120 INFO:tasks.cephfs_test_runner:test_root_rctime (tasks.cephfs.test_misc.TestMisc) ... ok 2020-12-05T17:41:44.121 INFO:tasks.cephfs_test_runner:Starting test: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2020-12-05T17:41:44.121 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs' 2020-12-05T17:41:44.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.234+0000 7f9dd5d53700 1 Processor -- start 2020-12-05T17:41:44.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.234+0000 7f9dd5d53700 1 -- start start 2020-12-05T17:41:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dd5d53700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9dd00b1fc0 0x7f9dd00b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dd5d53700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9dd00af0d0 0x7f9dd013c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dd5d53700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dd00b23a0 con 0x7f9dd013c860 2020-12-05T17:41:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dd5d53700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dd013f280 con 0x7f9dd00b1fc0 2020-12-05T17:41:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dd5d53700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dd013cc60 con 0x7f9dd00af0d0 2020-12-05T17:41:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dd4d51700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9dd00b1fc0 0x7f9dd00b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dd4d51700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9dd00b1fc0 0x7f9dd00b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41210/0 (socket says 172.21.15.32:41210) 2020-12-05T17:41:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dd4d51700 1 -- 172.21.15.32:0/497627030 learned_addr learned my addr 172.21.15.32:0/497627030 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dcffff700 1 --2- 172.21.15.32:0/497627030 >> v2:172.21.15.61:3300/0 conn(0x7f9dd00af0d0 0x7f9dd013c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dd4d51700 1 -- 172.21.15.32:0/497627030 >> v2:172.21.15.61:3300/0 conn(0x7f9dd00af0d0 msgr2=0x7f9dd013c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dd4d51700 1 --2- 172.21.15.32:0/497627030 >> v2:172.21.15.61:3300/0 conn(0x7f9dd00af0d0 0x7f9dd013c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dd4d51700 1 -- 172.21.15.32:0/497627030 >> v1:172.21.15.61:6789/0 conn(0x7f9dd013c860 legacy=0x7f9dd013f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dd4d51700 1 -- 172.21.15.32:0/497627030 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd00abfa0 con 0x7f9dd00b1fc0 2020-12-05T17:41:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dd4d51700 1 --2- 172.21.15.32:0/497627030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dd00b1fc0 0x7f9dd00b0fa0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7f9dc0002f30 tx=0x7f9dc0006e80).ready entity=mon.0 client_cookie=e88fef34e1070502 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dcf7fe700 1 -- 172.21.15.32:0/497627030 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dc00097f0 con 0x7f9dd00b1fc0 2020-12-05T17:41:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dcf7fe700 1 -- 172.21.15.32:0/497627030 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc0009950 con 0x7f9dd00b1fc0 2020-12-05T17:41:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dcf7fe700 1 -- 172.21.15.32:0/497627030 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dc0009cb0 con 0x7f9dd00b1fc0 2020-12-05T17:41:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dd5d53700 1 -- 172.21.15.32:0/497627030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dd00b1fc0 msgr2=0x7f9dd00b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dd5d53700 1 --2- 172.21.15.32:0/497627030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dd00b1fc0 0x7f9dd00b0fa0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7f9dc0002f30 tx=0x7f9dc0006e80).stop 2020-12-05T17:41:44.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dd5d53700 1 -- 172.21.15.32:0/497627030 shutdown_connections 2020-12-05T17:41:44.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dd5d53700 1 --2- 172.21.15.32:0/497627030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dd00b1fc0 0x7f9dd00b0fa0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:44.246 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dd5d53700 1 --2- 172.21.15.32:0/497627030 >> v2:172.21.15.61:3300/0 conn(0x7f9dd00af0d0 0x7f9dd013c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:44.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dd5d53700 1 -- 172.21.15.32:0/497627030 >> 172.21.15.32:0/497627030 conn(0x7f9dd00b5530 msgr2=0x7f9dd00b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:44.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dd5d53700 1 -- 172.21.15.32:0/497627030 shutdown_connections 2020-12-05T17:41:44.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.238+0000 7f9dd5d53700 1 -- 172.21.15.32:0/497627030 wait complete. 2020-12-05T17:41:44.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dd5d53700 1 Processor -- start 2020-12-05T17:41:44.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dd5d53700 1 -- start start 2020-12-05T17:41:44.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dd5d53700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dd00af0d0 0x7f9dd0063790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:44.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dd5d53700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9dd00b1fc0 0x7f9dd0063e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:44.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dd5d53700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9dd013c860 0x7f9dd0141c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:44.248 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dd5d53700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd0064360 con 0x7f9dd00af0d0 2020-12-05T17:41:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dd5d53700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd00b29d0 con 0x7f9dd013c860 2020-12-05T17:41:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dd5d53700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd0140860 con 0x7f9dd00b1fc0 2020-12-05T17:41:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dd4d51700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dd00af0d0 0x7f9dd0063790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dd4d51700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dd00af0d0 0x7f9dd0063790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41214/0 (socket says 172.21.15.32:41214) 2020-12-05T17:41:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dd4d51700 1 -- 172.21.15.32:0/1627004388 learned_addr learned my addr 172.21.15.32:0/1627004388 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dcffff700 1 --2- 172.21.15.32:0/1627004388 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9dd00b1fc0 0x7f9dd0063e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dd5552700 1 --2- 172.21.15.32:0/1627004388 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9dd013c860 0x7f9dd0141c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dd4d51700 1 -- 172.21.15.32:0/1627004388 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9dd00b1fc0 msgr2=0x7f9dd0063e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dd4d51700 1 --2- 172.21.15.32:0/1627004388 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9dd00b1fc0 0x7f9dd0063e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dd4d51700 1 -- 172.21.15.32:0/1627004388 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9dd013c860 msgr2=0x7f9dd0141c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dd4d51700 1 --2- 172.21.15.32:0/1627004388 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9dd013c860 0x7f9dd0141c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dd4d51700 1 -- 172.21.15.32:0/1627004388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd0141920 con 0x7f9dd00af0d0 2020-12-05T17:41:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dd4d51700 1 --2- 172.21.15.32:0/1627004388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dd00af0d0 0x7f9dd0063790 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7f9dc0002f30 tx=0x7f9dc0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dcdffb700 1 -- 172.21.15.32:0/1627004388 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dc0011ce0 con 0x7f9dd00af0d0 2020-12-05T17:41:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dcdffb700 1 -- 172.21.15.32:0/1627004388 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc0011e40 con 0x7f9dd00af0d0 2020-12-05T17:41:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dcdffb700 1 -- 172.21.15.32:0/1627004388 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dc001e650 con 0x7f9dd00af0d0 2020-12-05T17:41:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dd5d53700 1 -- 172.21.15.32:0/1627004388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd012fca0 con 0x7f9dd00af0d0 2020-12-05T17:41:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.242+0000 7f9dd5d53700 1 -- 172.21.15.32:0/1627004388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd0142290 con 0x7f9dd00af0d0 2020-12-05T17:41:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.246+0000 7f9dd5d53700 1 -- 172.21.15.32:0/1627004388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd0140a20 con 0x7f9dd00af0d0 2020-12-05T17:41:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.246+0000 7f9dcdffb700 1 -- 172.21.15.32:0/1627004388 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9dc001a040 con 0x7f9dd00af0d0 2020-12-05T17:41:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.246+0000 7f9dcdffb700 1 --2- 172.21.15.32:0/1627004388 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9db00554e0 0x7f9db00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.246+0000 7f9dcffff700 1 --2- 172.21.15.32:0/1627004388 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9db00554e0 0x7f9db00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.246+0000 7f9dcdffb700 1 -- 172.21.15.32:0/1627004388 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9dc006f190 con 0x7f9dd00af0d0 2020-12-05T17:41:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.246+0000 7f9dcffff700 1 --2- 172.21.15.32:0/1627004388 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9db00554e0 0x7f9db00579a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f9dc400b000 tx=0x7f9dc4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.250+0000 7f9dcdffb700 1 -- 172.21.15.32:0/1627004388 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9dc0045370 con 0x7f9dd00af0d0 2020-12-05T17:41:44.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:44.446+0000 7f9dd5d53700 1 -- 172.21.15.32:0/1627004388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs"]} v 0) v1 -- 0x7f9dd0063c80 con 0x7f9dd00af0d0 2020-12-05T17:41:45.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.110+0000 7f9dcdffb700 1 -- 172.21.15.32:0/1627004388 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs"]}]=0 v466) v1 ==== 139+0+0 (secure 0 0 0) 0x7f9dc00230d0 con 0x7f9dd00af0d0 2020-12-05T17:41:45.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.114+0000 7f9dd5d53700 1 -- 172.21.15.32:0/1627004388 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9db00554e0 msgr2=0x7f9db00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:45.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.114+0000 7f9dd5d53700 1 --2- 172.21.15.32:0/1627004388 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9db00554e0 0x7f9db00579a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f9dc400b000 tx=0x7f9dc4008040).stop 2020-12-05T17:41:45.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.114+0000 7f9dd5d53700 1 -- 172.21.15.32:0/1627004388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dd00af0d0 msgr2=0x7f9dd0063790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:45.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.114+0000 7f9dd5d53700 1 --2- 172.21.15.32:0/1627004388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dd00af0d0 0x7f9dd0063790 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7f9dc0002f30 tx=0x7f9dc0015000).stop 2020-12-05T17:41:45.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.114+0000 7f9dd5d53700 1 -- 172.21.15.32:0/1627004388 shutdown_connections 2020-12-05T17:41:45.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.114+0000 7f9dd5d53700 1 --2- 172.21.15.32:0/1627004388 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9dd013c860 0x7f9dd0141c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:45.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.114+0000 7f9dd5d53700 1 --2- 172.21.15.32:0/1627004388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dd00af0d0 0x7f9dd0063790 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:45.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.114+0000 7f9dd5d53700 1 --2- 172.21.15.32:0/1627004388 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9db00554e0 0x7f9db00579a0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:45.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.114+0000 7f9dd5d53700 1 --2- 172.21.15.32:0/1627004388 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9dd00b1fc0 0x7f9dd0063e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:45.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.114+0000 7f9dd5d53700 1 -- 172.21.15.32:0/1627004388 >> 172.21.15.32:0/1627004388 conn(0x7f9dd00b5530 msgr2=0x7f9dd00b41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:45.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.114+0000 7f9dd5d53700 1 -- 172.21.15.32:0/1627004388 shutdown_connections 2020-12-05T17:41:45.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.114+0000 7f9dd5d53700 1 -- 172.21.15.32:0/1627004388 wait complete. 2020-12-05T17:41:45.131 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-12-05T17:41:45.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.250+0000 7fe5b70b6700 1 Processor -- start 2020-12-05T17:41:45.251 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.250+0000 7fe5b70b6700 1 -- start start 2020-12-05T17:41:45.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.250+0000 7fe5b70b6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5b005cd70 0x7fe5b013f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:45.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.250+0000 7fe5b70b6700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fe5b00ae140 0x7fe5b013fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:45.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.250+0000 7fe5b70b6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5b005f5c0 con 0x7fe5b00b1990 2020-12-05T17:41:45.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.250+0000 7fe5b70b6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5b005ff30 con 0x7fe5b005cd70 2020-12-05T17:41:45.253 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.250+0000 7fe5b70b6700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5b005d150 con 0x7fe5b00ae140 2020-12-05T17:41:45.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.250+0000 7fe5b4e52700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5b005cd70 0x7fe5b013f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:45.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.250+0000 7fe5b5653700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe5b00b1990 0x7fe5b00b5d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43872/0 (socket says 172.21.15.32:43872) 2020-12-05T17:41:45.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.250+0000 7fe5b5653700 1 -- 172.21.15.32:0/244982017 learned_addr learned my addr 172.21.15.32:0/244982017 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:45.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.250+0000 7fe5affff700 1 --2- 172.21.15.32:0/244982017 >> v2:172.21.15.61:3300/0 conn(0x7fe5b00ae140 0x7fe5b013fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:45.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.250+0000 7fe5b4e52700 1 -- 172.21.15.32:0/244982017 >> v2:172.21.15.61:3300/0 conn(0x7fe5b00ae140 msgr2=0x7fe5b013fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:45.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.250+0000 7fe5b4e52700 1 --2- 172.21.15.32:0/244982017 >> v2:172.21.15.61:3300/0 conn(0x7fe5b00ae140 0x7fe5b013fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:45.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.250+0000 7fe5af7fe700 1 -- 172.21.15.32:0/244982017 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668898530 0 0) 0x7fe5b005f5c0 con 0x7fe5b00b1990 2020-12-05T17:41:45.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.250+0000 7fe5b4e52700 1 -- 172.21.15.32:0/244982017 >> v1:172.21.15.32:6789/0 conn(0x7fe5b00b1990 legacy=0x7fe5b00b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:45.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.250+0000 7fe5b4e52700 1 -- 172.21.15.32:0/244982017 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b0140340 con 0x7fe5b005cd70 2020-12-05T17:41:45.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.254+0000 7fe5b4e52700 1 --2- 172.21.15.32:0/244982017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5b005cd70 0x7fe5b013f960 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7fe5a0002f30 tx=0x7fe5a0006dc0).ready entity=mon.0 client_cookie=a476322805d9bab server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:45.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.254+0000 7fe5af7fe700 1 -- 172.21.15.32:0/244982017 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5a00097b0 con 0x7fe5b005cd70 2020-12-05T17:41:45.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.254+0000 7fe5af7fe700 1 -- 172.21.15.32:0/244982017 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a0009910 con 0x7fe5b005cd70 2020-12-05T17:41:45.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.254+0000 7fe5af7fe700 1 -- 172.21.15.32:0/244982017 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5a0009c70 con 0x7fe5b005cd70 2020-12-05T17:41:45.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.254+0000 7fe5b70b6700 1 -- 172.21.15.32:0/244982017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5b005cd70 msgr2=0x7fe5b013f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:45.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.254+0000 7fe5b70b6700 1 --2- 172.21.15.32:0/244982017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5b005cd70 0x7fe5b013f960 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7fe5a0002f30 tx=0x7fe5a0006dc0).stop 2020-12-05T17:41:45.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.254+0000 7fe5b70b6700 1 -- 172.21.15.32:0/244982017 shutdown_connections 2020-12-05T17:41:45.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.254+0000 7fe5b70b6700 1 --2- 172.21.15.32:0/244982017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5b005cd70 0x7fe5b013f960 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:45.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.254+0000 7fe5b70b6700 1 --2- 172.21.15.32:0/244982017 >> v2:172.21.15.61:3300/0 conn(0x7fe5b00ae140 0x7fe5b013fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.254+0000 7fe5b70b6700 1 -- 172.21.15.32:0/244982017 >> 172.21.15.32:0/244982017 conn(0x7fe5b0069cb0 msgr2=0x7fe5b00604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.254+0000 7fe5b70b6700 1 -- 172.21.15.32:0/244982017 shutdown_connections 2020-12-05T17:41:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.254+0000 7fe5b70b6700 1 -- 172.21.15.32:0/244982017 wait complete. 2020-12-05T17:41:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.254+0000 7fe5b70b6700 1 Processor -- start 2020-12-05T17:41:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.254+0000 7fe5b70b6700 1 -- start start 2020-12-05T17:41:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.258+0000 7fe5b70b6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe5b005cd70 0x7fe5b0143e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.258+0000 7fe5b70b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5b00ae140 0x7fe5b013d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.258+0000 7fe5b70b6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5b00b1990 0x7fe5b0154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.258+0000 7fe5b70b6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b01517b0 con 0x7fe5b00ae140 2020-12-05T17:41:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.258+0000 7fe5b70b6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b0065ee0 con 0x7fe5b005cd70 2020-12-05T17:41:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.258+0000 7fe5b70b6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b0065c80 con 0x7fe5b00b1990 2020-12-05T17:41:45.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.258+0000 7fe5affff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5b00ae140 0x7fe5b013d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:45.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.258+0000 7fe5affff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5b00ae140 0x7fe5b013d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41230/0 (socket says 172.21.15.32:41230) 2020-12-05T17:41:45.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.258+0000 7fe5affff700 1 -- 172.21.15.32:0/807590151 learned_addr learned my addr 172.21.15.32:0/807590151 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:45.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.258+0000 7fe5b4e52700 1 --2- 172.21.15.32:0/807590151 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe5b005cd70 0x7fe5b0143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:45.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.258+0000 7fe5b5653700 1 --2- 172.21.15.32:0/807590151 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5b00b1990 0x7fe5b0154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:45.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.258+0000 7fe5affff700 1 -- 172.21.15.32:0/807590151 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5b00b1990 msgr2=0x7fe5b0154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:45.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.258+0000 7fe5affff700 1 --2- 172.21.15.32:0/807590151 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5b00b1990 0x7fe5b0154d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:45.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.258+0000 7fe5affff700 1 -- 172.21.15.32:0/807590151 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe5b005cd70 msgr2=0x7fe5b0143e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:45.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.258+0000 7fe5affff700 1 --2- 172.21.15.32:0/807590151 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe5b005cd70 0x7fe5b0143e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:45.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.258+0000 7fe5affff700 1 -- 172.21.15.32:0/807590151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b0151990 con 0x7fe5b00ae140 2020-12-05T17:41:45.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.258+0000 7fe5affff700 1 --2- 172.21.15.32:0/807590151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5b00ae140 0x7fe5b013d670 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7fe5a400c000 tx=0x7fe5a4006d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:45.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.258+0000 7fe5adffb700 1 -- 172.21.15.32:0/807590151 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5a4005ea0 con 0x7fe5b00ae140 2020-12-05T17:41:45.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.258+0000 7fe5adffb700 1 -- 172.21.15.32:0/807590151 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a400ba90 con 0x7fe5b00ae140 2020-12-05T17:41:45.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.258+0000 7fe5adffb700 1 -- 172.21.15.32:0/807590151 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5a400bdf0 con 0x7fe5b00ae140 2020-12-05T17:41:45.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.258+0000 7fe5b70b6700 1 -- 172.21.15.32:0/807590151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b0149290 con 0x7fe5b00ae140 2020-12-05T17:41:45.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.258+0000 7fe5b70b6700 1 -- 172.21.15.32:0/807590151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b0144540 con 0x7fe5b00ae140 2020-12-05T17:41:45.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.262+0000 7fe5b70b6700 1 -- 172.21.15.32:0/807590151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b0151b20 con 0x7fe5b00ae140 2020-12-05T17:41:45.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.262+0000 7fe5adffb700 1 -- 172.21.15.32:0/807590151 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe5a401b410 con 0x7fe5b00ae140 2020-12-05T17:41:45.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.262+0000 7fe5adffb700 1 --2- 172.21.15.32:0/807590151 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe5900553b0 0x7fe590057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:45.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.262+0000 7fe5adffb700 1 -- 172.21.15.32:0/807590151 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe5a406e1b0 con 0x7fe5b00ae140 2020-12-05T17:41:45.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.266+0000 7fe5b4e52700 1 --2- 172.21.15.32:0/807590151 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe5900553b0 0x7fe590057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:45.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.266+0000 7fe5b4e52700 1 --2- 172.21.15.32:0/807590151 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe5900553b0 0x7fe590057870 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fe5a001a000 tx=0x7fe5a0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:45.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.266+0000 7fe5adffb700 1 -- 172.21.15.32:0/807590151 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe5a4044290 con 0x7fe5b00ae140 2020-12-05T17:41:45.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.414+0000 7fe5b70b6700 1 -- 172.21.15.32:0/807590151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7fe5b0057050 con 0x7fe5b00ae140 2020-12-05T17:41:45.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.418+0000 7fe5adffb700 1 -- 172.21.15.32:0/807590151 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe5a40478b0 con 0x7fe5b00ae140 2020-12-05T17:41:45.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.418+0000 7fe5b70b6700 1 -- 172.21.15.32:0/807590151 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe5900553b0 msgr2=0x7fe590057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:45.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.418+0000 7fe5b70b6700 1 --2- 172.21.15.32:0/807590151 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe5900553b0 0x7fe590057870 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fe5a001a000 tx=0x7fe5a0015000).stop 2020-12-05T17:41:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.418+0000 7fe5b70b6700 1 -- 172.21.15.32:0/807590151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5b00ae140 msgr2=0x7fe5b013d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.418+0000 7fe5b70b6700 1 --2- 172.21.15.32:0/807590151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5b00ae140 0x7fe5b013d670 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7fe5a400c000 tx=0x7fe5a4006d60).stop 2020-12-05T17:41:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.418+0000 7fe5b70b6700 1 -- 172.21.15.32:0/807590151 shutdown_connections 2020-12-05T17:41:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.418+0000 7fe5b70b6700 1 --2- 172.21.15.32:0/807590151 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe5900553b0 0x7fe590057870 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.418+0000 7fe5b70b6700 1 --2- 172.21.15.32:0/807590151 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe5b00b1990 0x7fe5b0154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.418+0000 7fe5b70b6700 1 --2- 172.21.15.32:0/807590151 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe5b005cd70 0x7fe5b0143e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.418+0000 7fe5b70b6700 1 --2- 172.21.15.32:0/807590151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5b00ae140 0x7fe5b013d670 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.418+0000 7fe5b70b6700 1 -- 172.21.15.32:0/807590151 >> 172.21.15.32:0/807590151 conn(0x7fe5b0069cb0 msgr2=0x7fe5b013f730 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.418+0000 7fe5b70b6700 1 -- 172.21.15.32:0/807590151 shutdown_connections 2020-12-05T17:41:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.418+0000 7fe5b70b6700 1 -- 172.21.15.32:0/807590151 wait complete. 2020-12-05T17:41:45.437 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:41:45.437 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:41:45.454 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:41:45.470 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:41:45.487 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:41:45.505 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:41:45.523 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:41:45.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:45.524 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:45.524 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:41:45.537 INFO:teuthology.orchestra.run.smithi036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-12-05T17:41:45.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:41:45.539 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-12-05T17:41:45.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:45.539 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:45.540 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:41:45.547 INFO:teuthology.orchestra.run.smithi002.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-12-05T17:41:45.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:41:45.548 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-12-05T17:41:45.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:41:45.549 INFO:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:41:45.549 INFO:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-12-05T17:41:45.555 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-12-05T17:41:45.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:41:45.556 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2020-12-05T17:41:45.557 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-12-05T17:41:45.678 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f1574a3b700 1 Processor -- start 2020-12-05T17:41:45.678 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f1574a3b700 1 -- start start 2020-12-05T17:41:45.679 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f1574a3b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f15700b5f00 0x7f15700b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:45.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f1574a3b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1570142ab0 con 0x7f15700af480 2020-12-05T17:41:45.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f1574a3b700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15700b62e0 con 0x7f15700ad510 2020-12-05T17:41:45.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f1574a3b700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15700af890 con 0x7f15700b5f00 2020-12-05T17:41:45.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f156ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f15700af480 0x7f1570142ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43888/0 (socket says 172.21.15.32:43888) 2020-12-05T17:41:45.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f156ffff700 1 -- 172.21.15.32:0/800325352 learned_addr learned my addr 172.21.15.32:0/800325352 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:45.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f156f7fe700 1 --2- 172.21.15.32:0/800325352 >> v2:172.21.15.61:3300/0 conn(0x7f15700b5f00 0x7f15700b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:45.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f156e7fc700 1 -- 172.21.15.32:0/800325352 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1904863122 0 0) 0x7f1570142ab0 con 0x7f15700af480 2020-12-05T17:41:45.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f156e7fc700 1 -- 172.21.15.32:0/800325352 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1550003230 con 0x7f15700af480 2020-12-05T17:41:45.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f156f7fe700 1 -- 172.21.15.32:0/800325352 >> v1:172.21.15.61:6789/0 conn(0x7f15700ad510 legacy=0x7f1570066370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:45.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f156e7fc700 1 -- 172.21.15.32:0/800325352 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1126764893 0 0) 0x7f1550003230 con 0x7f15700af480 2020-12-05T17:41:45.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f156f7fe700 1 -- 172.21.15.32:0/800325352 >> v1:172.21.15.32:6789/0 conn(0x7f15700af480 legacy=0x7f1570142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:45.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f156f7fe700 1 -- 172.21.15.32:0/800325352 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15700afbf0 con 0x7f15700b5f00 2020-12-05T17:41:45.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f156f7fe700 1 --2- 172.21.15.32:0/800325352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15700b5f00 0x7f15700b14e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f1568001870 tx=0x7f1558006dc0).ready entity=mon.1 client_cookie=8d0d5c959c3f9ff1 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:45.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f156e7fc700 1 -- 172.21.15.32:0/800325352 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15680016d0 con 0x7f15700b5f00 2020-12-05T17:41:45.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f156e7fc700 1 -- 172.21.15.32:0/800325352 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1558009620 con 0x7f15700b5f00 2020-12-05T17:41:45.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f1574a3b700 1 -- 172.21.15.32:0/800325352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15700b5f00 msgr2=0x7f15700b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:45.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f156e7fc700 1 -- 172.21.15.32:0/800325352 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1558009790 con 0x7f15700b5f00 2020-12-05T17:41:45.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f1574a3b700 1 --2- 172.21.15.32:0/800325352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15700b5f00 0x7f15700b14e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f1568001870 tx=0x7f1558006dc0).stop 2020-12-05T17:41:45.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f1574a3b700 1 -- 172.21.15.32:0/800325352 shutdown_connections 2020-12-05T17:41:45.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f1574a3b700 1 --2- 172.21.15.32:0/800325352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15700b5f00 0x7f15700b14e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:45.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f1574a3b700 1 -- 172.21.15.32:0/800325352 >> 172.21.15.32:0/800325352 conn(0x7f15700604c0 msgr2=0x7f157005c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.678+0000 7f1574a3b700 1 -- 172.21.15.32:0/800325352 shutdown_connections 2020-12-05T17:41:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f1574a3b700 1 -- 172.21.15.32:0/800325352 wait complete. 2020-12-05T17:41:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f1574a3b700 1 Processor -- start 2020-12-05T17:41:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f1574a3b700 1 -- start start 2020-12-05T17:41:45.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f1574a3b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15700ad510 0x7f157013d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:45.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f1574a3b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15700af480 0x7f1570135630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:45.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f1574a3b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15700b5f00 0x7f15700b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:45.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f1574a3b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1570133100 con 0x7f15700ad510 2020-12-05T17:41:45.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f1574a3b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15700b4c00 con 0x7f15700af480 2020-12-05T17:41:45.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f1574a3b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15700b4ea0 con 0x7f15700b5f00 2020-12-05T17:41:45.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f156f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15700ad510 0x7f157013d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:45.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f156f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15700ad510 0x7f157013d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41242/0 (socket says 172.21.15.32:41242) 2020-12-05T17:41:45.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f156f7fe700 1 -- 172.21.15.32:0/1461978461 learned_addr learned my addr 172.21.15.32:0/1461978461 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:45.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f156ffff700 1 --2- 172.21.15.32:0/1461978461 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15700b5f00 0x7f15700b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:45.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f156effd700 1 --2- 172.21.15.32:0/1461978461 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15700af480 0x7f1570135630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:45.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f156f7fe700 1 -- 172.21.15.32:0/1461978461 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15700b5f00 msgr2=0x7f15700b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:45.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f156f7fe700 1 --2- 172.21.15.32:0/1461978461 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15700b5f00 0x7f15700b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:45.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f156f7fe700 1 -- 172.21.15.32:0/1461978461 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15700af480 msgr2=0x7f1570135630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:45.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f156f7fe700 1 --2- 172.21.15.32:0/1461978461 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15700af480 0x7f1570135630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:45.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f156f7fe700 1 -- 172.21.15.32:0/1461978461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15700b5140 con 0x7f15700ad510 2020-12-05T17:41:45.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f156ffff700 1 --2- 172.21.15.32:0/1461978461 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15700b5f00 0x7f15700b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:41:45.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f156effd700 1 --2- 172.21.15.32:0/1461978461 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15700af480 0x7f1570135630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T17:41:45.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f156f7fe700 1 --2- 172.21.15.32:0/1461978461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15700ad510 0x7f157013d560 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7f1568001870 tx=0x7f15700ae940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:45.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f156cff9700 1 -- 172.21.15.32:0/1461978461 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1558013c90 con 0x7f15700ad510 2020-12-05T17:41:45.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f156cff9700 1 -- 172.21.15.32:0/1461978461 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1558013df0 con 0x7f15700ad510 2020-12-05T17:41:45.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f1574a3b700 1 -- 172.21.15.32:0/1461978461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15701313a0 con 0x7f15700ad510 2020-12-05T17:41:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f156cff9700 1 -- 172.21.15.32:0/1461978461 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f155801e5f0 con 0x7f15700ad510 2020-12-05T17:41:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.682+0000 7f1574a3b700 1 -- 172.21.15.32:0/1461978461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15700b4170 con 0x7f15700ad510 2020-12-05T17:41:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.686+0000 7f1574a3b700 1 -- 172.21.15.32:0/1461978461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1570057050 con 0x7f15700ad510 2020-12-05T17:41:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.686+0000 7f156cff9700 1 -- 172.21.15.32:0/1461978461 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1558025030 con 0x7f15700ad510 2020-12-05T17:41:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.686+0000 7f156cff9700 1 --2- 172.21.15.32:0/1461978461 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15540553b0 0x7f1554057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:45.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.686+0000 7f156cff9700 1 -- 172.21.15.32:0/1461978461 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f155806ee30 con 0x7f15700ad510 2020-12-05T17:41:45.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.686+0000 7f156effd700 1 --2- 172.21.15.32:0/1461978461 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15540553b0 0x7f1554057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:45.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.686+0000 7f156effd700 1 --2- 172.21.15.32:0/1461978461 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15540553b0 0x7f1554057870 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f1560008000 tx=0x7f1560009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:45.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.694+0000 7f156cff9700 1 -- 172.21.15.32:0/1461978461 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1558045010 con 0x7f15700ad510 2020-12-05T17:41:45.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.838+0000 7f1574a3b700 1 -- 172.21.15.32:0/1461978461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f15700b12c0 con 0x7f15700ad510 2020-12-05T17:41:45.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.838+0000 7f156cff9700 1 -- 172.21.15.32:0/1461978461 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v138) v1 ==== 81+0+34051 (secure 0 0 0) 0x7f1558023070 con 0x7f15700ad510 2020-12-05T17:41:45.840 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:45.841 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:41:45.841 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 138, 2020-12-05T17:41:45.841 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", 2020-12-05T17:41:45.841 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2020-12-05T17:31:20.975238+0000", 2020-12-05T17:41:45.842 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2020-12-05T17:41:32.383424+0000", 2020-12-05T17:41:45.842 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:41:45.842 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:41:45.842 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-12-05T17:41:45.842 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2020-12-05T17:41:45.843 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2020-12-05T17:41:45.843 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2020-12-05T17:41:45.843 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2020-12-05T17:41:45.843 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2020-12-05T17:41:45.843 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2020-12-05T17:41:45.844 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:41:45.844 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2020-12-05T17:41:45.844 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2020-12-05T17:41:45.844 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-12-05T17:41:45.844 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2020-12-05T17:41:45.845 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2020-12-05T17:41:45.845 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 28, 2020-12-05T17:41:45.845 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2020-12-05T17:41:45.845 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2020-12-05T17:41:45.846 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2020-12-05T17:41:45.846 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2020-12-05T17:41:45.846 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2020-12-05T17:41:45.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.846 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2020-12-05T17:41:45.847 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2020-12-05T17:41:45.847 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:24.858732+0000", 2020-12-05T17:41:45.847 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:41:45.847 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:41:45.847 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:41:45.848 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:41:45.848 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:41:45.848 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:41:45.848 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:41:45.848 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:41:45.849 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2020-12-05T17:41:45.849 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2020-12-05T17:41:45.849 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2020-12-05T17:41:45.849 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2020-12-05T17:41:45.849 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2020-12-05T17:41:45.850 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:41:45.850 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:41:45.850 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:41:45.850 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:41:45.851 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:41:45.851 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:41:45.851 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:41:45.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.851 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "16", 2020-12-05T17:41:45.852 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:41:45.852 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:41:45.852 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:41:45.852 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:41:45.852 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:41:45.853 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:41:45.853 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:41:45.853 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:41:45.853 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:41:45.853 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:41:45.854 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:41:45.854 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:41:45.854 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:41:45.854 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:41:45.854 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:41:45.855 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:41:45.855 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:41:45.855 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:41:45.855 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:41:45.855 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:41:45.856 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:41:45.856 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:41:45.856 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:41:45.856 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:41:45.856 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:41:45.857 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:41:45.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.857 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:41:45.857 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:41:45.858 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:41:45.858 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:41:45.858 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:41:45.858 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:41:45.858 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:41:45.859 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:41:45.859 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:41:45.859 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:41:45.859 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:41:45.859 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:41:45.860 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2020-12-05T17:41:45.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.860 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:41:45.860 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2020-12-05T17:41:45.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.861 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2020-12-05T17:41:45.861 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2020-12-05T17:41:45.861 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:33.305595+0000", 2020-12-05T17:41:45.862 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:41:45.862 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:41:45.862 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:41:45.862 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:41:45.863 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:41:45.863 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:41:45.863 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:41:45.863 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:41:45.863 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:41:45.864 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:41:45.864 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:41:45.864 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:41:45.864 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:41:45.864 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:41:45.865 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:41:45.865 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:41:45.865 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:41:45.865 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:41:45.865 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:41:45.866 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:41:45.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.866 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "19", 2020-12-05T17:41:45.866 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:41:45.867 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:41:45.867 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:41:45.867 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:41:45.867 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:41:45.867 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:41:45.868 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:41:45.868 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:41:45.868 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:41:45.868 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:41:45.868 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:41:45.869 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:41:45.869 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:41:45.869 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:41:45.869 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:41:45.869 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:41:45.870 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:41:45.870 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:41:45.870 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:41:45.870 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:41:45.870 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:41:45.871 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:41:45.871 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:41:45.871 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:41:45.871 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:41:45.871 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:41:45.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.872 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:41:45.872 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:41:45.872 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:41:45.872 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:41:45.873 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:41:45.873 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:41:45.873 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:41:45.873 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:41:45.874 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:41:45.874 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:41:45.874 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:41:45.874 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:41:45.874 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:41:45.875 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2020-12-05T17:41:45.875 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.875 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 27, 2020-12-05T17:41:45.876 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2020-12-05T17:41:45.876 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:41:30.899960+0000", 2020-12-05T17:41:45.876 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:41:45.876 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:41:45.876 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:41:45.877 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:41:45.877 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:41:45.877 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:41:45.877 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:41:45.877 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:41:45.878 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:41:45.878 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:41:45.878 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:41:45.878 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:41:45.878 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:41:45.879 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:41:45.879 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:41:45.879 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:41:45.879 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:41:45.879 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:41:45.880 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:41:45.880 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:41:45.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.880 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "138", 2020-12-05T17:41:45.880 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:41:45.881 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:41:45.881 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:41:45.881 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:41:45.881 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:41:45.881 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:41:45.882 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:41:45.882 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:41:45.882 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:41:45.882 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:41:45.882 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:41:45.883 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:41:45.883 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:41:45.883 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:41:45.883 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:41:45.883 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:41:45.884 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:41:45.884 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:41:45.884 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:41:45.884 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:41:45.885 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:41:45.885 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:41:45.885 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:41:45.885 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:41:45.885 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:41:45.886 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:41:45.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.886 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:41:45.887 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:41:45.887 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:41:45.887 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:41:45.887 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:41:45.887 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:41:45.888 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:41:45.888 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:41:45.888 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:41:45.888 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:41:45.888 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:41:45.889 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:41:45.889 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2020-12-05T17:41:45.889 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2020-12-05T17:41:45.889 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2020-12-05T17:41:45.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.890 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:41:45.890 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:41:45.890 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2020-12-05T17:41:45.890 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.891 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.891 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 28, 2020-12-05T17:41:45.892 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2020-12-05T17:41:45.892 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:41:31.381413+0000", 2020-12-05T17:41:45.892 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:41:45.892 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:41:45.893 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:41:45.893 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:41:45.893 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:41:45.893 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:41:45.893 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:41:45.894 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:41:45.894 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:41:45.894 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:41:45.894 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:41:45.894 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:41:45.895 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:41:45.895 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:41:45.895 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:41:45.895 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:41:45.895 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:41:45.896 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:41:45.896 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:41:45.896 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:41:45.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.896 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "138", 2020-12-05T17:41:45.897 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:41:45.897 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:41:45.897 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:41:45.897 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:41:45.897 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:41:45.898 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:41:45.898 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:41:45.898 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:41:45.898 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:41:45.899 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:41:45.899 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:41:45.899 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:41:45.899 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:41:45.899 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:41:45.900 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:41:45.900 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:41:45.900 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:41:45.900 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:41:45.901 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:41:45.901 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:41:45.901 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:41:45.901 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:41:45.901 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:41:45.902 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:41:45.902 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:41:45.902 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:41:45.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.902 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:41:45.903 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:41:45.903 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:41:45.903 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:41:45.903 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:41:45.903 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:41:45.904 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:41:45.904 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:41:45.904 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:41:45.904 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:41:45.904 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:41:45.905 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:41:45.905 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:41:45.905 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:41:45.905 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2020-12-05T17:41:45.905 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.906 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.906 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.906 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:41:45.906 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2020-12-05T17:41:45.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.907 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:41:45.907 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "b941127b-0a49-4e06-8d11-17794ff8b013", 2020-12-05T17:41:45.907 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:41:45.907 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:41:45.908 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:41:45.908 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:41:45.908 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:41:45.908 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:41:45.908 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:41:45.909 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 135, 2020-12-05T17:41:45.909 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:41:45.909 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:41:45.909 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:41:45.909 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.910 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.910 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2020-12-05T17:41:45.910 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:41:45.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.911 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:45.911 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2020-12-05T17:41:45.911 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:41:45.912 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.912 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.912 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:41:45.912 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.913 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.913 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2020-12-05T17:41:45.913 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:41:45.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.914 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:45.914 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2020-12-05T17:41:45.914 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:41:45.914 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.915 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.915 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:41:45.915 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.916 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.916 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2020-12-05T17:41:45.916 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:41:45.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.917 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:45.917 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2020-12-05T17:41:45.917 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:41:45.917 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.917 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.918 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:41:45.918 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.918 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.919 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2020-12-05T17:41:45.919 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:41:45.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.920 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:45.920 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2020-12-05T17:41:45.920 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:41:45.920 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.920 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.921 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6803/11526", 2020-12-05T17:41:45.921 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6805/11526", 2020-12-05T17:41:45.921 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6809/11526", 2020-12-05T17:41:45.921 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6807/11526", 2020-12-05T17:41:45.922 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:41:45.922 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:41:45.922 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:41:45.922 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.923 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:41:45.923 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "01bd7ff5-a86b-4e11-ba1e-484c309c0fba", 2020-12-05T17:41:45.923 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:41:45.923 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:41:45.924 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:41:45.924 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:41:45.924 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:41:45.924 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:41:45.924 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:41:45.925 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 135, 2020-12-05T17:41:45.925 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:41:45.925 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:41:45.925 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:41:45.925 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.926 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.926 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2020-12-05T17:41:45.926 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:41:45.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.927 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:45.927 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2020-12-05T17:41:45.927 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:41:45.927 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.928 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.928 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:41:45.928 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.929 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.929 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2020-12-05T17:41:45.929 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:41:45.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.930 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:45.930 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2020-12-05T17:41:45.930 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:41:45.930 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.931 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.931 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:41:45.931 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.932 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.932 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2020-12-05T17:41:45.932 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:41:45.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.933 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:45.933 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2020-12-05T17:41:45.933 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:41:45.933 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.933 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.934 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:41:45.934 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.934 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.935 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2020-12-05T17:41:45.935 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:41:45.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.936 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:45.936 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2020-12-05T17:41:45.936 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:41:45.936 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.936 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.937 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6819/11538", 2020-12-05T17:41:45.937 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6821/11538", 2020-12-05T17:41:45.937 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6825/11538", 2020-12-05T17:41:45.937 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6823/11538", 2020-12-05T17:41:45.938 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:41:45.938 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:41:45.938 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:41:45.938 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.939 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:41:45.939 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "94736ef5-64c2-4a97-b963-bd38cddd9c52", 2020-12-05T17:41:45.939 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:41:45.939 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:41:45.940 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:41:45.940 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:41:45.940 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:41:45.940 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:41:45.940 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:41:45.941 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 134, 2020-12-05T17:41:45.941 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:41:45.941 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:41:45.941 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:41:45.941 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.942 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.942 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2020-12-05T17:41:45.942 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:41:45.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.943 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:45.943 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2020-12-05T17:41:45.943 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:41:45.944 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.944 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.944 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:41:45.944 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.945 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.945 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2020-12-05T17:41:45.945 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:41:45.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.946 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:45.946 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2020-12-05T17:41:45.946 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:41:45.946 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.947 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.947 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:41:45.947 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.948 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.948 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2020-12-05T17:41:45.948 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:41:45.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.949 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:45.949 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2020-12-05T17:41:45.949 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:41:45.949 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.949 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.950 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:41:45.950 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.951 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.951 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2020-12-05T17:41:45.951 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:41:45.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.952 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:45.952 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2020-12-05T17:41:45.952 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:41:45.952 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.952 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.953 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6811/11521", 2020-12-05T17:41:45.953 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6813/11521", 2020-12-05T17:41:45.953 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6817/11521", 2020-12-05T17:41:45.953 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6815/11521", 2020-12-05T17:41:45.954 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:41:45.954 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:41:45.954 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:41:45.954 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.955 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:41:45.955 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "69a1f3ff-fd25-43df-b60c-e32a89c35234", 2020-12-05T17:41:45.955 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:41:45.956 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:41:45.956 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:41:45.956 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:41:45.956 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:41:45.956 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:41:45.957 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:41:45.957 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 126, 2020-12-05T17:41:45.957 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:41:45.957 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:41:45.957 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:41:45.958 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.958 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.958 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2020-12-05T17:41:45.958 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:41:45.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.959 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:45.959 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2020-12-05T17:41:45.959 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:41:45.960 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.960 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.960 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:41:45.961 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.961 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.961 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2020-12-05T17:41:45.961 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:41:45.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.962 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:45.962 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2020-12-05T17:41:45.962 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:41:45.963 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.963 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.963 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:41:45.963 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.964 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.964 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6832", 2020-12-05T17:41:45.964 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:41:45.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.965 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:45.965 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6833", 2020-12-05T17:41:45.965 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:41:45.965 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.966 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.966 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:41:45.966 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.967 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.967 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2020-12-05T17:41:45.967 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:41:45.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.968 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:45.968 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2020-12-05T17:41:45.968 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:41:45.968 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.968 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.969 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6827/11535", 2020-12-05T17:41:45.969 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6829/11535", 2020-12-05T17:41:45.969 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6833/11535", 2020-12-05T17:41:45.969 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6831/11535", 2020-12-05T17:41:45.970 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:41:45.970 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:41:45.970 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:41:45.970 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.971 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:41:45.971 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c301fe4e-5eab-402c-ae6f-db3e6a7181af", 2020-12-05T17:41:45.971 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:41:45.971 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:41:45.972 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:41:45.972 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:41:45.972 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:41:45.972 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:41:45.972 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:41:45.973 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 134, 2020-12-05T17:41:45.973 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:41:45.973 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:41:45.973 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:41:45.973 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.974 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.974 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6808", 2020-12-05T17:41:45.974 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:41:45.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.975 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:45.975 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6809", 2020-12-05T17:41:45.975 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:41:45.976 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.976 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.976 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:41:45.976 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.977 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.977 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6810", 2020-12-05T17:41:45.977 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:41:45.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.978 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:45.978 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6811", 2020-12-05T17:41:45.978 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:41:45.979 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.979 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.979 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:41:45.979 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.980 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.980 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6817", 2020-12-05T17:41:45.980 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:41:45.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.981 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:45.981 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6819", 2020-12-05T17:41:45.981 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:41:45.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.982 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.982 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:41:45.982 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.983 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.983 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6812", 2020-12-05T17:41:45.983 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:41:45.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.984 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:45.984 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6814", 2020-12-05T17:41:45.984 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:41:45.984 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.984 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.985 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6809/11541", 2020-12-05T17:41:45.985 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6811/11541", 2020-12-05T17:41:45.985 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6819/11541", 2020-12-05T17:41:45.986 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6814/11541", 2020-12-05T17:41:45.986 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:41:45.986 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:41:45.986 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:41:45.986 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.987 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:41:45.987 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "9476eff4-345b-46c5-a473-1d9a89a6d327", 2020-12-05T17:41:45.987 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:41:45.988 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:41:45.988 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:41:45.988 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:41:45.988 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:41:45.988 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:41:45.989 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:41:45.989 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 135, 2020-12-05T17:41:45.989 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:41:45.989 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:41:45.990 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:41:45.990 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.990 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.990 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6800", 2020-12-05T17:41:45.991 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:41:45.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.991 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:45.992 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6801", 2020-12-05T17:41:45.992 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:41:45.992 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.992 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.993 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:41:45.993 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.993 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.993 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6802", 2020-12-05T17:41:45.994 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:41:45.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.994 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:45.994 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6803", 2020-12-05T17:41:45.995 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:41:45.995 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.995 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.996 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:41:45.996 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.996 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.996 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6806", 2020-12-05T17:41:45.997 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:41:45.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.997 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:45.998 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6807", 2020-12-05T17:41:45.998 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:41:45.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:45.998 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:45.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:45.999 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:41:45.999 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:45.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:45.999 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:45.999 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6804", 2020-12-05T17:41:46.000 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:41:46.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.000 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:46.001 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6805", 2020-12-05T17:41:46.001 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:41:46.001 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:46.001 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:46.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.002 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6801/11539", 2020-12-05T17:41:46.002 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6803/11539", 2020-12-05T17:41:46.002 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6807/11539", 2020-12-05T17:41:46.002 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6805/11539", 2020-12-05T17:41:46.003 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:41:46.003 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:41:46.003 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:41:46.003 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:46.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.004 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:41:46.004 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8dfd8567-9b1c-4d13-9c96-071183e469c2", 2020-12-05T17:41:46.004 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:41:46.004 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:41:46.005 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:41:46.005 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:41:46.005 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:41:46.005 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:41:46.006 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:41:46.006 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 135, 2020-12-05T17:41:46.006 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:41:46.006 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:41:46.006 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:41:46.007 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:46.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.007 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:46.007 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6820", 2020-12-05T17:41:46.007 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:41:46.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.008 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:46.008 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6822", 2020-12-05T17:41:46.008 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:41:46.009 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:46.009 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:46.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.009 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:41:46.009 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:46.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.010 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:46.010 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6823", 2020-12-05T17:41:46.010 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:41:46.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.011 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:46.011 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6825", 2020-12-05T17:41:46.011 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:41:46.012 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:46.012 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:46.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.012 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:41:46.012 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:46.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.013 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:46.013 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6829", 2020-12-05T17:41:46.013 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:41:46.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.014 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:46.014 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6830", 2020-12-05T17:41:46.014 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:41:46.014 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:46.015 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:46.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.015 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:41:46.015 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:46.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.016 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:46.016 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6826", 2020-12-05T17:41:46.016 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:41:46.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.017 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:46.017 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6827", 2020-12-05T17:41:46.017 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:41:46.017 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:46.018 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:46.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.018 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6822/11542", 2020-12-05T17:41:46.018 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6825/11542", 2020-12-05T17:41:46.018 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6830/11542", 2020-12-05T17:41:46.019 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6827/11542", 2020-12-05T17:41:46.019 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:41:46.019 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:41:46.019 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:41:46.019 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:46.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.020 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:41:46.020 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "24d466b3-01dc-4ba2-8b81-85250d163bd2", 2020-12-05T17:41:46.020 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:41:46.021 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:41:46.021 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:41:46.021 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:41:46.021 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:41:46.021 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:41:46.022 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:41:46.022 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 135, 2020-12-05T17:41:46.022 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:41:46.022 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:41:46.022 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:41:46.023 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:46.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.023 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:46.023 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6813", 2020-12-05T17:41:46.023 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:41:46.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.024 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:46.024 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6815", 2020-12-05T17:41:46.025 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:41:46.025 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:46.025 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:46.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.025 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:41:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6816", 2020-12-05T17:41:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:41:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6818", 2020-12-05T17:41:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:41:46.028 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:46.028 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:46.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.028 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:41:46.029 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:46.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.029 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:46.029 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6828", 2020-12-05T17:41:46.029 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:41:46.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.030 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:46.030 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6831", 2020-12-05T17:41:46.030 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:41:46.031 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:46.031 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:46.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.031 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:41:46.032 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:41:46.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.032 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:41:46.032 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6821", 2020-12-05T17:41:46.032 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:41:46.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.033 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:41:46.033 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6824", 2020-12-05T17:41:46.033 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:41:46.034 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:46.034 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:46.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.034 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6815/11540", 2020-12-05T17:41:46.034 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6818/11540", 2020-12-05T17:41:46.035 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6831/11540", 2020-12-05T17:41:46.035 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6824/11540", 2020-12-05T17:41:46.035 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:41:46.035 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:41:46.036 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:41:46.036 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:46.036 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:46.036 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:41:46.036 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2020-12-05T17:41:46.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.037 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:41:46.037 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:41:46.037 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:41:46.037 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:41:46.038 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:41:46.038 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:41:46.038 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.398868+0000", 2020-12-05T17:41:46.038 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:41:46.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.039 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:41:46.039 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:41:46.039 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:41:46.039 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:41:46.040 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:41:46.040 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:41:46.040 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.675748+0000", 2020-12-05T17:41:46.040 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:41:46.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.041 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:41:46.041 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:41:46.041 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:41:46.041 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:41:46.042 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:41:46.042 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:41:46.042 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.620888+0000", 2020-12-05T17:41:46.042 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:41:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:41:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:41:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:41:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:41:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:41:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:41:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.681600+0000", 2020-12-05T17:41:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:41:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:41:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:41:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:41:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:41:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:41:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:41:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.658769+0000", 2020-12-05T17:41:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:41:46.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.047 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:41:46.047 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:41:46.047 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:41:46.048 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:41:46.048 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:41:46.048 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:41:46.048 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.630708+0000", 2020-12-05T17:41:46.048 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:41:46.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.049 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:41:46.049 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:41:46.049 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:41:46.050 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:41:46.050 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:41:46.050 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:41:46.050 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.648352+0000", 2020-12-05T17:41:46.051 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:41:46.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:46.051 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:41:46.051 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:41:46.052 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:41:46.052 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:41:46.052 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:41:46.052 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:41:46.052 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.629399+0000", 2020-12-05T17:41:46.053 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:41:46.053 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:46.053 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:41:46.053 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2020-12-05T17:41:46.053 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2020-12-05T17:41:46.054 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2020-12-05T17:41:46.054 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2020-12-05T17:41:46.054 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": {}, 2020-12-05T17:41:46.054 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2020-12-05T17:41:46.054 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2020-12-05T17:41:46.055 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2020-12-05T17:41:46.055 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:41:46.055 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2020-12-05T17:41:46.055 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2020-12-05T17:41:46.055 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:41:46.056 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2020-12-05T17:41:46.056 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:41:46.056 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:41:46.056 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2020-12-05T17:41:46.056 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2020-12-05T17:41:46.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.057 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2020-12-05T17:41:46.057 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:41:46.057 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:41:46.057 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2020-12-05T17:41:46.058 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:41:46.058 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:41:46.058 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2020-12-05T17:41:46.058 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:46.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:46.059 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2020-12-05T17:41:46.059 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2020-12-05T17:41:46.059 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2020-12-05T17:41:46.059 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2020-12-05T17:41:46.059 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2020-12-05T17:41:46.060 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:41:46.062 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:41:46.062 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.842+0000 7f1574a3b700 1 -- 172.21.15.32:0/1461978461 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15540553b0 msgr2=0x7f1554057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:46.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.842+0000 7f1574a3b700 1 --2- 172.21.15.32:0/1461978461 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15540553b0 0x7f1554057870 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f1560008000 tx=0x7f1560009040).stop 2020-12-05T17:41:46.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.842+0000 7f1574a3b700 1 -- 172.21.15.32:0/1461978461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15700ad510 msgr2=0x7f157013d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:46.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.842+0000 7f1574a3b700 1 --2- 172.21.15.32:0/1461978461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15700ad510 0x7f157013d560 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7f1568001870 tx=0x7f15700ae940).stop 2020-12-05T17:41:46.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.842+0000 7f1574a3b700 1 -- 172.21.15.32:0/1461978461 shutdown_connections 2020-12-05T17:41:46.063 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.842+0000 7f1574a3b700 1 --2- 172.21.15.32:0/1461978461 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f15540553b0 0x7f1554057870 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:46.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.842+0000 7f1574a3b700 1 --2- 172.21.15.32:0/1461978461 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f15700b5f00 0x7f15700b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:46.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.842+0000 7f1574a3b700 1 --2- 172.21.15.32:0/1461978461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15700ad510 0x7f157013d560 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:46.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.842+0000 7f1574a3b700 1 --2- 172.21.15.32:0/1461978461 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15700af480 0x7f1570135630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:46.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.842+0000 7f1574a3b700 1 -- 172.21.15.32:0/1461978461 >> 172.21.15.32:0/1461978461 conn(0x7f15700604c0 msgr2=0x7f157013fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:46.064 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.842+0000 7f1574a3b700 1 -- 172.21.15.32:0/1461978461 shutdown_connections 2020-12-05T17:41:46.065 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:45.842+0000 7f1574a3b700 1 -- 172.21.15.32:0/1461978461 wait complete. 2020-12-05T17:41:46.065 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:41:46.068 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:41:46.110 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:41:46.112 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:41:46.114 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:41:46.180 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.178+0000 7f56a0a67700 1 Processor -- start 2020-12-05T17:41:46.181 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.178+0000 7f56a0a67700 1 -- start start 2020-12-05T17:41:46.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f56a0a67700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f569c062400 0x7f569c05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:46.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f56a0a67700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f569c05feb0 0x7f569c0b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:46.182 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f56a0a67700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f569c142450 con 0x7f569c05f3b0 2020-12-05T17:41:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f56a0a67700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f569c0b42f0 con 0x7f569c05feb0 2020-12-05T17:41:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f56a0a67700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569c061780 con 0x7f569c062400 2020-12-05T17:41:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f569b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f569c05f3b0 0x7f569c0b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43898/0 (socket says 172.21.15.32:43898) 2020-12-05T17:41:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f569b7fe700 1 -- 172.21.15.32:0/2747213749 learned_addr learned my addr 172.21.15.32:0/2747213749 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f569bfff700 1 --2- 172.21.15.32:0/2747213749 >> v2:172.21.15.61:3300/0 conn(0x7f569c05feb0 0x7f569c0b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:46.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f569affd700 1 --2- 172.21.15.32:0/2747213749 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f569c062400 0x7f569c05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:46.185 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f569a7fc700 1 -- 172.21.15.32:0/2747213749 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2977262137 0 0) 0x7f569c142450 con 0x7f569c05f3b0 2020-12-05T17:41:46.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f569a7fc700 1 -- 172.21.15.32:0/2747213749 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5688003230 con 0x7f569c05f3b0 2020-12-05T17:41:46.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f569affd700 1 -- 172.21.15.32:0/2747213749 >> v2:172.21.15.61:3300/0 conn(0x7f569c05feb0 msgr2=0x7f569c0b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:46.186 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f569affd700 1 --2- 172.21.15.32:0/2747213749 >> v2:172.21.15.61:3300/0 conn(0x7f569c05feb0 0x7f569c0b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:46.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f569affd700 1 -- 172.21.15.32:0/2747213749 >> v1:172.21.15.32:6789/0 conn(0x7f569c05f3b0 legacy=0x7f569c0b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:46.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f569affd700 1 -- 172.21.15.32:0/2747213749 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c061a20 con 0x7f569c062400 2020-12-05T17:41:46.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f569bfff700 1 --2- 172.21.15.32:0/2747213749 >> v2:172.21.15.61:3300/0 conn(0x7f569c05feb0 0x7f569c0b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:46.187 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f569affd700 1 --2- 172.21.15.32:0/2747213749 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f569c062400 0x7f569c05f790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f568c002f30 tx=0x7f568c006dc0).ready entity=mon.2 client_cookie=9d582d6c9abe5573 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:46.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f569a7fc700 1 -- 172.21.15.32:0/2747213749 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f568c0096a0 con 0x7f569c062400 2020-12-05T17:41:46.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f569a7fc700 1 -- 172.21.15.32:0/2747213749 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568c009800 con 0x7f569c062400 2020-12-05T17:41:46.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f56a0a67700 1 -- 172.21.15.32:0/2747213749 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f569c062400 msgr2=0x7f569c05f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:46.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f569a7fc700 1 -- 172.21.15.32:0/2747213749 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f568c009b60 con 0x7f569c062400 2020-12-05T17:41:46.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f56a0a67700 1 --2- 172.21.15.32:0/2747213749 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f569c062400 0x7f569c05f790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f568c002f30 tx=0x7f568c006dc0).stop 2020-12-05T17:41:46.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f56a0a67700 1 -- 172.21.15.32:0/2747213749 shutdown_connections 2020-12-05T17:41:46.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f56a0a67700 1 --2- 172.21.15.32:0/2747213749 >> v2:172.21.15.61:3300/0 conn(0x7f569c05feb0 0x7f569c0b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:46.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f56a0a67700 1 --2- 172.21.15.32:0/2747213749 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f569c062400 0x7f569c05f790 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:46.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f56a0a67700 1 -- 172.21.15.32:0/2747213749 >> 172.21.15.32:0/2747213749 conn(0x7f569c0b8ee0 msgr2=0x7f569c0b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:46.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f56a0a67700 1 -- 172.21.15.32:0/2747213749 shutdown_connections 2020-12-05T17:41:46.190 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.182+0000 7f56a0a67700 1 -- 172.21.15.32:0/2747213749 wait complete. 2020-12-05T17:41:46.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f56a0a67700 1 Processor -- start 2020-12-05T17:41:46.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f56a0a67700 1 -- start start 2020-12-05T17:41:46.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f56a0a67700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c05f3b0 0x7f569c0b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:46.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f56a0a67700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f569c05feb0 0x7f569c064e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:46.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f56a0a67700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f569c062400 0x7f569c0aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:46.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f56a0a67700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c066800 con 0x7f569c05f3b0 2020-12-05T17:41:46.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f56a0a67700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c0a2a00 con 0x7f569c05feb0 2020-12-05T17:41:46.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f56a0a67700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569c0a2c60 con 0x7f569c062400 2020-12-05T17:41:46.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f569b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c05f3b0 0x7f569c0b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:46.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f569b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c05f3b0 0x7f569c0b2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41256/0 (socket says 172.21.15.32:41256) 2020-12-05T17:41:46.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f569b7fe700 1 -- 172.21.15.32:0/1326479845 learned_addr learned my addr 172.21.15.32:0/1326479845 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:46.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f569affd700 1 --2- 172.21.15.32:0/1326479845 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f569c05feb0 0x7f569c064e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:46.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f569bfff700 1 --2- 172.21.15.32:0/1326479845 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f569c062400 0x7f569c0aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:46.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f569b7fe700 1 -- 172.21.15.32:0/1326479845 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f569c062400 msgr2=0x7f569c0aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:46.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f569b7fe700 1 --2- 172.21.15.32:0/1326479845 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f569c062400 0x7f569c0aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:46.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f569b7fe700 1 -- 172.21.15.32:0/1326479845 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f569c05feb0 msgr2=0x7f569c064e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:46.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f569b7fe700 1 --2- 172.21.15.32:0/1326479845 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f569c05feb0 0x7f569c064e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:46.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f569b7fe700 1 -- 172.21.15.32:0/1326479845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c0a2f00 con 0x7f569c05f3b0 2020-12-05T17:41:46.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f569b7fe700 1 --2- 172.21.15.32:0/1326479845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c05f3b0 0x7f569c0b2e70 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0x7f56900010d0 tx=0x7f5690007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:46.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f5698ff9700 1 -- 172.21.15.32:0/1326479845 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56900072e0 con 0x7f569c05f3b0 2020-12-05T17:41:46.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f5698ff9700 1 -- 172.21.15.32:0/1326479845 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5690007440 con 0x7f569c05f3b0 2020-12-05T17:41:46.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f56a0a67700 1 -- 172.21.15.32:0/1326479845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f569c0660f0 con 0x7f569c05f3b0 2020-12-05T17:41:46.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f5698ff9700 1 -- 172.21.15.32:0/1326479845 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56900057d0 con 0x7f569c05f3b0 2020-12-05T17:41:46.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.186+0000 7f56a0a67700 1 -- 172.21.15.32:0/1326479845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f569c0a3860 con 0x7f569c05f3b0 2020-12-05T17:41:46.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.190+0000 7f56a0a67700 1 -- 172.21.15.32:0/1326479845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f569c065570 con 0x7f569c05f3b0 2020-12-05T17:41:46.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.190+0000 7f5698ff9700 1 -- 172.21.15.32:0/1326479845 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5690010070 con 0x7f569c05f3b0 2020-12-05T17:41:46.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.190+0000 7f5698ff9700 1 --2- 172.21.15.32:0/1326479845 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f567c055330 0x7f567c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:46.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.190+0000 7f5698ff9700 1 -- 172.21.15.32:0/1326479845 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f569006d880 con 0x7f569c05f3b0 2020-12-05T17:41:46.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.190+0000 7f569affd700 1 --2- 172.21.15.32:0/1326479845 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f567c055330 0x7f567c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:46.199 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.190+0000 7f569affd700 1 --2- 172.21.15.32:0/1326479845 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f567c055330 0x7f567c0577f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f568c01b000 tx=0x7f568c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:46.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.198+0000 7f5698ff9700 1 -- 172.21.15.32:0/1326479845 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5690043980 con 0x7f569c05f3b0 2020-12-05T17:41:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.386+0000 7f56a0a67700 1 -- 172.21.15.32:0/1326479845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f569c057050 con 0x7f569c05f3b0 2020-12-05T17:41:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.386+0000 7f5698ff9700 1 -- 172.21.15.32:0/1326479845 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f5690046fa0 con 0x7f569c05f3b0 2020-12-05T17:41:46.387 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:46.387 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":66,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/3049192140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3049192140},{"type":"v1","addr":"172.21.15.61:6833","nonce":3049192140}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:41:32.383702+0000","modified":"2020-12-05T17:41:33.387236+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8274":{"gid":8274,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/2808222868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2808222868},{"type":"v1","addr":"172.21.15.32:6835","nonce":2808222868}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2020-12-05T17:41:46.388 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.386+0000 7f56a0a67700 1 -- 172.21.15.32:0/1326479845 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f567c055330 msgr2=0x7f567c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.386+0000 7f56a0a67700 1 --2- 172.21.15.32:0/1326479845 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f567c055330 0x7f567c0577f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f568c01b000 tx=0x7f568c015000).stop 2020-12-05T17:41:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.386+0000 7f56a0a67700 1 -- 172.21.15.32:0/1326479845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c05f3b0 msgr2=0x7f569c0b2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.386+0000 7f56a0a67700 1 --2- 172.21.15.32:0/1326479845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c05f3b0 0x7f569c0b2e70 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0x7f56900010d0 tx=0x7f5690007b50).stop 2020-12-05T17:41:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.386+0000 7f56a0a67700 1 -- 172.21.15.32:0/1326479845 shutdown_connections 2020-12-05T17:41:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.386+0000 7f56a0a67700 1 --2- 172.21.15.32:0/1326479845 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f567c055330 0x7f567c0577f0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.386+0000 7f56a0a67700 1 --2- 172.21.15.32:0/1326479845 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f569c062400 0x7f569c0aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.386+0000 7f56a0a67700 1 --2- 172.21.15.32:0/1326479845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c05f3b0 0x7f569c0b2e70 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.386+0000 7f56a0a67700 1 --2- 172.21.15.32:0/1326479845 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f569c05feb0 0x7f569c064e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.386+0000 7f56a0a67700 1 -- 172.21.15.32:0/1326479845 >> 172.21.15.32:0/1326479845 conn(0x7f569c0b8ee0 msgr2=0x7f569c142620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:46.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.386+0000 7f56a0a67700 1 -- 172.21.15.32:0/1326479845 shutdown_connections 2020-12-05T17:41:46.391 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.386+0000 7f56a0a67700 1 -- 172.21.15.32:0/1326479845 wait complete. 2020-12-05T17:41:46.391 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 66 2020-12-05T17:41:46.401 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-12-05T17:41:46.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f8051483700 1 Processor -- start 2020-12-05T17:41:46.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f8051483700 1 -- start start 2020-12-05T17:41:46.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f8051483700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f804c0ab610 0x7f804c13dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:46.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f8051483700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804c0ac290 con 0x7f804c062660 2020-12-05T17:41:46.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f8051483700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804c0b0930 con 0x7f804c0abeb0 2020-12-05T17:41:46.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f8051483700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804c0ad8a0 con 0x7f804c0ab610 2020-12-05T17:41:46.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f8043fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f804c062660 0x7f804c0ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43912/0 (socket says 172.21.15.32:43912) 2020-12-05T17:41:46.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f8043fff700 1 -- 172.21.15.32:0/2912562366 learned_addr learned my addr 172.21.15.32:0/2912562366 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:46.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f8050c82700 1 --2- 172.21.15.32:0/2912562366 >> v2:172.21.15.32:3300/0 conn(0x7f804c0ab610 0x7f804c13dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:46.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f804b7fe700 1 -- 172.21.15.32:0/2912562366 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 493757602 0 0) 0x7f804c0ac290 con 0x7f804c062660 2020-12-05T17:41:46.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f804b7fe700 1 -- 172.21.15.32:0/2912562366 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f802c004340 con 0x7f804c062660 2020-12-05T17:41:46.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f8050c82700 1 -- 172.21.15.32:0/2912562366 >> v1:172.21.15.61:6789/0 conn(0x7f804c0abeb0 legacy=0x7f804c0ad190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:46.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f804b7fe700 1 -- 172.21.15.32:0/2912562366 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189441011 0 0) 0x7f804c0b0930 con 0x7f804c0abeb0 2020-12-05T17:41:46.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f8050c82700 1 -- 172.21.15.32:0/2912562366 >> v1:172.21.15.32:6789/0 conn(0x7f804c062660 legacy=0x7f804c0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:46.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f8050c82700 1 -- 172.21.15.32:0/2912562366 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804c0b0bd0 con 0x7f804c0ab610 2020-12-05T17:41:46.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f8050c82700 1 --2- 172.21.15.32:0/2912562366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804c0ab610 0x7f804c13dec0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 rx=0x7f80440032b0 tx=0x7f80440091a0).ready entity=mon.0 client_cookie=4ca80eed2838ecbc server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:46.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f804b7fe700 1 -- 172.21.15.32:0/2912562366 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f804400b9a0 con 0x7f804c0ab610 2020-12-05T17:41:46.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f804b7fe700 1 -- 172.21.15.32:0/2912562366 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804400bb00 con 0x7f804c0ab610 2020-12-05T17:41:46.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f804b7fe700 1 -- 172.21.15.32:0/2912562366 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f804400bc70 con 0x7f804c0ab610 2020-12-05T17:41:46.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f8051483700 1 -- 172.21.15.32:0/2912562366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804c0ab610 msgr2=0x7f804c13dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:46.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f8051483700 1 --2- 172.21.15.32:0/2912562366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804c0ab610 0x7f804c13dec0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 rx=0x7f80440032b0 tx=0x7f80440091a0).stop 2020-12-05T17:41:46.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f8051483700 1 -- 172.21.15.32:0/2912562366 shutdown_connections 2020-12-05T17:41:46.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f8051483700 1 --2- 172.21.15.32:0/2912562366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804c0ab610 0x7f804c13dec0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:46.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f8051483700 1 -- 172.21.15.32:0/2912562366 >> 172.21.15.32:0/2912562366 conn(0x7f804c0b5530 msgr2=0x7f804c0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:46.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f8051483700 1 -- 172.21.15.32:0/2912562366 shutdown_connections 2020-12-05T17:41:46.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.562+0000 7f8051483700 1 -- 172.21.15.32:0/2912562366 wait complete. 2020-12-05T17:41:46.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f8051483700 1 Processor -- start 2020-12-05T17:41:46.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f8051483700 1 -- start start 2020-12-05T17:41:46.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f8051483700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804c062660 0x7f804c1403b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:46.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f8051483700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f804c0ab610 0x7f804c151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:46.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f8051483700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f804c0abeb0 0x7f804c151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:46.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f8051483700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804c151470 con 0x7f804c062660 2020-12-05T17:41:46.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f8051483700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804c063aa0 con 0x7f804c0ab610 2020-12-05T17:41:46.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f8051483700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804c0b3140 con 0x7f804c0abeb0 2020-12-05T17:41:46.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f804bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804c062660 0x7f804c1403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:46.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f804bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804c062660 0x7f804c1403b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41270/0 (socket says 172.21.15.32:41270) 2020-12-05T17:41:46.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f804bfff700 1 -- 172.21.15.32:0/3619199367 learned_addr learned my addr 172.21.15.32:0/3619199367 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:46.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f8043fff700 1 --2- 172.21.15.32:0/3619199367 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f804c0ab610 0x7f804c151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:46.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f8050c82700 1 --2- 172.21.15.32:0/3619199367 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f804c0abeb0 0x7f804c151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:46.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f804bfff700 1 -- 172.21.15.32:0/3619199367 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f804c0abeb0 msgr2=0x7f804c151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:46.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f804bfff700 1 --2- 172.21.15.32:0/3619199367 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f804c0abeb0 0x7f804c151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:46.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f804bfff700 1 -- 172.21.15.32:0/3619199367 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f804c0ab610 msgr2=0x7f804c151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:46.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f804bfff700 1 --2- 172.21.15.32:0/3619199367 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f804c0ab610 0x7f804c151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:46.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f804bfff700 1 -- 172.21.15.32:0/3619199367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804c0b33e0 con 0x7f804c062660 2020-12-05T17:41:46.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f804bfff700 1 --2- 172.21.15.32:0/3619199367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804c062660 0x7f804c1403b0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7f803c000b80 tx=0x7f803c007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:46.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f8049ffb700 1 -- 172.21.15.32:0/3619199367 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f803c013410 con 0x7f804c062660 2020-12-05T17:41:46.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f8049ffb700 1 -- 172.21.15.32:0/3619199367 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803c013570 con 0x7f804c062660 2020-12-05T17:41:46.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f8051483700 1 -- 172.21.15.32:0/3619199367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804c13f300 con 0x7f804c062660 2020-12-05T17:41:46.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f8049ffb700 1 -- 172.21.15.32:0/3619199367 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f803c0138d0 con 0x7f804c062660 2020-12-05T17:41:46.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.566+0000 7f8051483700 1 -- 172.21.15.32:0/3619199367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f804c152240 con 0x7f804c062660 2020-12-05T17:41:46.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.570+0000 7f8051483700 1 -- 172.21.15.32:0/3619199367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f804c057050 con 0x7f804c062660 2020-12-05T17:41:46.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.570+0000 7f8049ffb700 1 -- 172.21.15.32:0/3619199367 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f803c010030 con 0x7f804c062660 2020-12-05T17:41:46.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.570+0000 7f8049ffb700 1 --2- 172.21.15.32:0/3619199367 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f80300553b0 0x7f8030057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:46.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.570+0000 7f8043fff700 1 --2- 172.21.15.32:0/3619199367 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f80300553b0 0x7f8030057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:46.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.570+0000 7f8049ffb700 1 -- 172.21.15.32:0/3619199367 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f803c001e40 con 0x7f804c062660 2020-12-05T17:41:46.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.570+0000 7f8043fff700 1 --2- 172.21.15.32:0/3619199367 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f80300553b0 0x7f8030057870 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f803400d000 tx=0x7f8034009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:46.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.578+0000 7f8049ffb700 1 -- 172.21.15.32:0/3619199367 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f803c043b20 con 0x7f804c062660 2020-12-05T17:41:46.766 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:46.766+0000 7f8051483700 1 -- 172.21.15.32:0/3619199367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f804c069ee0 con 0x7f804c062660 2020-12-05T17:41:47.109 INFO:tasks.ceph.mon.a.smithi032.stderr:2020-12-05T17:41:47.110+0000 7fdef8136700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-12-05T17:41:47.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.122+0000 7f8049ffb700 1 -- 172.21.15.32:0/3619199367 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v67) v1 ==== 161+0+0 (secure 0 0 0) 0x7f803c047140 con 0x7f804c062660 2020-12-05T17:41:47.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.122+0000 7f8051483700 1 -- 172.21.15.32:0/3619199367 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f80300553b0 msgr2=0x7f8030057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:47.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.122+0000 7f8051483700 1 --2- 172.21.15.32:0/3619199367 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f80300553b0 0x7f8030057870 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f803400d000 tx=0x7f8034009040).stop 2020-12-05T17:41:47.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.122+0000 7f8051483700 1 -- 172.21.15.32:0/3619199367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804c062660 msgr2=0x7f804c1403b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:47.125 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.122+0000 7f8051483700 1 --2- 172.21.15.32:0/3619199367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804c062660 0x7f804c1403b0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7f803c000b80 tx=0x7f803c007c00).stop 2020-12-05T17:41:47.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.126+0000 7f8051483700 1 -- 172.21.15.32:0/3619199367 shutdown_connections 2020-12-05T17:41:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.126+0000 7f8051483700 1 --2- 172.21.15.32:0/3619199367 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f80300553b0 0x7f8030057870 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.126+0000 7f8051483700 1 --2- 172.21.15.32:0/3619199367 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f804c0abeb0 0x7f804c151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.126+0000 7f8051483700 1 --2- 172.21.15.32:0/3619199367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804c062660 0x7f804c1403b0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.126+0000 7f8051483700 1 --2- 172.21.15.32:0/3619199367 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f804c0ab610 0x7f804c151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.126+0000 7f8051483700 1 -- 172.21.15.32:0/3619199367 >> 172.21.15.32:0/3619199367 conn(0x7f804c0b5530 msgr2=0x7f804c0618a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.126+0000 7f8051483700 1 -- 172.21.15.32:0/3619199367 shutdown_connections 2020-12-05T17:41:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.126+0000 7f8051483700 1 -- 172.21.15.32:0/3619199367 wait complete. 2020-12-05T17:41:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-12-05T17:41:47.142 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:41:47.149 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:41:47.247 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.246+0000 7fed5a87a700 1 Processor -- start 2020-12-05T17:41:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.246+0000 7fed5a87a700 1 -- start start 2020-12-05T17:41:47.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.246+0000 7fed5a87a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fed540ac7e0 0x7fed54061fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:47.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.246+0000 7fed5a87a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed540b6510 con 0x7fed54142a00 2020-12-05T17:41:47.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.246+0000 7fed5a87a700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed540b5550 con 0x7fed54146310 2020-12-05T17:41:47.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.246+0000 7fed5a87a700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed540b6740 con 0x7fed540ac7e0 2020-12-05T17:41:47.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.246+0000 7fed58e17700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fed54142a00 0x7fed54142e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43940/0 (socket says 172.21.15.32:43940) 2020-12-05T17:41:47.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.246+0000 7fed58e17700 1 -- 172.21.15.32:0/4037695659 learned_addr learned my addr 172.21.15.32:0/4037695659 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:47.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.246+0000 7fed537fe700 1 --2- 172.21.15.32:0/4037695659 >> v2:172.21.15.61:3300/0 conn(0x7fed540ac7e0 0x7fed54061fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:47.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.246+0000 7fed52ffd700 1 -- 172.21.15.32:0/4037695659 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4100089638 0 0) 0x7fed540b6510 con 0x7fed54142a00 2020-12-05T17:41:47.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.246+0000 7fed52ffd700 1 -- 172.21.15.32:0/4037695659 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed40003230 con 0x7fed54142a00 2020-12-05T17:41:47.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.246+0000 7fed52ffd700 1 -- 172.21.15.32:0/4037695659 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4134699435 0 0) 0x7fed540b5550 con 0x7fed54146310 2020-12-05T17:41:47.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.246+0000 7fed52ffd700 1 -- 172.21.15.32:0/4037695659 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed540b6510 con 0x7fed54146310 2020-12-05T17:41:47.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.246+0000 7fed537fe700 1 -- 172.21.15.32:0/4037695659 >> v1:172.21.15.61:6789/0 conn(0x7fed54146310 legacy=0x7fed540618a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:47.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.246+0000 7fed537fe700 1 -- 172.21.15.32:0/4037695659 >> v1:172.21.15.32:6789/0 conn(0x7fed54142a00 legacy=0x7fed54142e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:47.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.246+0000 7fed537fe700 1 -- 172.21.15.32:0/4037695659 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed540b69e0 con 0x7fed540ac7e0 2020-12-05T17:41:47.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.246+0000 7fed537fe700 1 --2- 172.21.15.32:0/4037695659 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fed540ac7e0 0x7fed54061fb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fed48002f30 tx=0x7fed48006dc0).ready entity=mon.1 client_cookie=404adfb2661e07ff server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:47.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.250+0000 7fed52ffd700 1 -- 172.21.15.32:0/4037695659 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed48009740 con 0x7fed540ac7e0 2020-12-05T17:41:47.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.250+0000 7fed52ffd700 1 -- 172.21.15.32:0/4037695659 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed480098a0 con 0x7fed540ac7e0 2020-12-05T17:41:47.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.250+0000 7fed52ffd700 1 -- 172.21.15.32:0/4037695659 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed48009c00 con 0x7fed540ac7e0 2020-12-05T17:41:47.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.250+0000 7fed5a87a700 1 -- 172.21.15.32:0/4037695659 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fed540ac7e0 msgr2=0x7fed54061fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:47.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.250+0000 7fed5a87a700 1 --2- 172.21.15.32:0/4037695659 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fed540ac7e0 0x7fed54061fb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fed48002f30 tx=0x7fed48006dc0).stop 2020-12-05T17:41:47.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.250+0000 7fed5a87a700 1 -- 172.21.15.32:0/4037695659 shutdown_connections 2020-12-05T17:41:47.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.250+0000 7fed5a87a700 1 --2- 172.21.15.32:0/4037695659 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fed540ac7e0 0x7fed54061fb0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:47.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.250+0000 7fed5a87a700 1 -- 172.21.15.32:0/4037695659 >> 172.21.15.32:0/4037695659 conn(0x7fed5405f3b0 msgr2=0x7fed5405f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:47.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.250+0000 7fed5a87a700 1 -- 172.21.15.32:0/4037695659 shutdown_connections 2020-12-05T17:41:47.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.250+0000 7fed5a87a700 1 -- 172.21.15.32:0/4037695659 wait complete. 2020-12-05T17:41:47.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.250+0000 7fed5a87a700 1 Processor -- start 2020-12-05T17:41:47.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.250+0000 7fed5a87a700 1 -- start start 2020-12-05T17:41:47.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed5a87a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fed540ac7e0 0x7fed540b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:47.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed5a87a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fed54142a00 0x7fed540ae050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:47.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed5a87a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed54146310 0x7fed54141ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:47.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed5a87a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed540b0b00 con 0x7fed54146310 2020-12-05T17:41:47.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed5a87a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed540b0da0 con 0x7fed540ac7e0 2020-12-05T17:41:47.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed5a87a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed540af7c0 con 0x7fed54142a00 2020-12-05T17:41:47.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed58e17700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed54146310 0x7fed54141ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:47.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed58e17700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed54146310 0x7fed54141ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41300/0 (socket says 172.21.15.32:41300) 2020-12-05T17:41:47.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed58e17700 1 -- 172.21.15.32:0/2838459554 learned_addr learned my addr 172.21.15.32:0/2838459554 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:47.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed537fe700 1 --2- 172.21.15.32:0/2838459554 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fed54142a00 0x7fed540ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed53fff700 1 --2- 172.21.15.32:0/2838459554 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fed540ac7e0 0x7fed540b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed58e17700 1 -- 172.21.15.32:0/2838459554 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fed54142a00 msgr2=0x7fed540ae050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed58e17700 1 --2- 172.21.15.32:0/2838459554 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fed54142a00 0x7fed540ae050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed58e17700 1 -- 172.21.15.32:0/2838459554 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fed540ac7e0 msgr2=0x7fed540b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed58e17700 1 --2- 172.21.15.32:0/2838459554 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fed540ac7e0 0x7fed540b1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed58e17700 1 -- 172.21.15.32:0/2838459554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed540b0440 con 0x7fed54146310 2020-12-05T17:41:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed53fff700 1 --2- 172.21.15.32:0/2838459554 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fed540ac7e0 0x7fed540b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed58e17700 1 --2- 172.21.15.32:0/2838459554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed54146310 0x7fed54141ff0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0x7fed440010d0 tx=0x7fed44007a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed517fa700 1 -- 172.21.15.32:0/2838459554 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed440072e0 con 0x7fed54146310 2020-12-05T17:41:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed517fa700 1 -- 172.21.15.32:0/2838459554 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed44007440 con 0x7fed54146310 2020-12-05T17:41:47.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed5a87a700 1 -- 172.21.15.32:0/2838459554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed541518b0 con 0x7fed54146310 2020-12-05T17:41:47.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed517fa700 1 -- 172.21.15.32:0/2838459554 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed440057d0 con 0x7fed54146310 2020-12-05T17:41:47.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed5a87a700 1 -- 172.21.15.32:0/2838459554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed54151d80 con 0x7fed54146310 2020-12-05T17:41:47.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed5a87a700 1 -- 172.21.15.32:0/2838459554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed540b0960 con 0x7fed54146310 2020-12-05T17:41:47.283 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.254+0000 7fed517fa700 1 -- 172.21.15.32:0/2838459554 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fed44010070 con 0x7fed54146310 2020-12-05T17:41:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.258+0000 7fed517fa700 1 --2- 172.21.15.32:0/2838459554 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fed340554e0 0x7fed340579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.258+0000 7fed517fa700 1 -- 172.21.15.32:0/2838459554 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fed4406d690 con 0x7fed54146310 2020-12-05T17:41:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.258+0000 7fed53fff700 1 --2- 172.21.15.32:0/2838459554 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fed340554e0 0x7fed340579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.258+0000 7fed53fff700 1 --2- 172.21.15.32:0/2838459554 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fed340554e0 0x7fed340579a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7fed4c00b000 tx=0x7fed4c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.266+0000 7fed517fa700 1 -- 172.21.15.32:0/2838459554 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fed540b0960 con 0x7fed54146310 2020-12-05T17:41:47.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.454+0000 7fed5a87a700 1 -- 172.21.15.32:0/2838459554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed540b0960 con 0x7fed54146310 2020-12-05T17:41:47.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.454+0000 7fed517fa700 1 -- 172.21.15.32:0/2838459554 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+1782 (secure 0 0 0) 0x7fed540b0960 con 0x7fed54146310 2020-12-05T17:41:47.456 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:47.457 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":67,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/3049192140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":3049192140},{"type":"v1","addr":"172.21.15.61:6833","nonce":3049192140}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":67,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:41:32.383702+0000","modified":"2020-12-05T17:41:47.110217+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":139,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2020-12-05T17:41:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.458+0000 7fed5a87a700 1 -- 172.21.15.32:0/2838459554 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fed340554e0 msgr2=0x7fed340579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.458+0000 7fed5a87a700 1 --2- 172.21.15.32:0/2838459554 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fed340554e0 0x7fed340579a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7fed4c00b000 tx=0x7fed4c009040).stop 2020-12-05T17:41:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.458+0000 7fed5a87a700 1 -- 172.21.15.32:0/2838459554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed54146310 msgr2=0x7fed54141ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.458+0000 7fed5a87a700 1 --2- 172.21.15.32:0/2838459554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed54146310 0x7fed54141ff0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0x7fed440010d0 tx=0x7fed44007a50).stop 2020-12-05T17:41:47.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.458+0000 7fed5a87a700 1 -- 172.21.15.32:0/2838459554 shutdown_connections 2020-12-05T17:41:47.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.458+0000 7fed5a87a700 1 --2- 172.21.15.32:0/2838459554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed54146310 0x7fed54141ff0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:47.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.458+0000 7fed5a87a700 1 --2- 172.21.15.32:0/2838459554 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fed540ac7e0 0x7fed540b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:47.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.458+0000 7fed5a87a700 1 --2- 172.21.15.32:0/2838459554 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fed340554e0 0x7fed340579a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:47.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.458+0000 7fed5a87a700 1 --2- 172.21.15.32:0/2838459554 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fed54142a00 0x7fed540ae050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:47.461 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.458+0000 7fed5a87a700 1 -- 172.21.15.32:0/2838459554 >> 172.21.15.32:0/2838459554 conn(0x7fed5405f3b0 msgr2=0x7fed540b5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:47.461 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.458+0000 7fed5a87a700 1 -- 172.21.15.32:0/2838459554 shutdown_connections 2020-12-05T17:41:47.461 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.458+0000 7fed5a87a700 1 -- 172.21.15.32:0/2838459554 wait complete. 2020-12-05T17:41:47.461 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 67 2020-12-05T17:41:47.471 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-12-05T17:41:47.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.630+0000 7f62f16ce700 1 Processor -- start 2020-12-05T17:41:47.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.630+0000 7f62f16ce700 1 -- start start 2020-12-05T17:41:47.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62f16ce700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f62ec0b5b20 0x7f62ec0b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:47.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62f16ce700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62ec0b5240 0x7f62ec145f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:47.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62f16ce700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ec0b1920 con 0x7f62ec0b6690 2020-12-05T17:41:47.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62f16ce700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ec0b1bc0 con 0x7f62ec0b5240 2020-12-05T17:41:47.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62f16ce700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ec0b5680 con 0x7f62ec0b5b20 2020-12-05T17:41:47.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62eb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62ec0b5240 0x7f62ec145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:47.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62eb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62ec0b5240 0x7f62ec145f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41304/0 (socket says 172.21.15.32:41304) 2020-12-05T17:41:47.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62eb7fe700 1 -- 172.21.15.32:0/2642246773 learned_addr learned my addr 172.21.15.32:0/2642246773 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:47.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62eaffd700 1 --2- 172.21.15.32:0/2642246773 >> v2:172.21.15.61:3300/0 conn(0x7f62ec0b5b20 0x7f62ec0b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:47.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62eb7fe700 1 -- 172.21.15.32:0/2642246773 >> v2:172.21.15.61:3300/0 conn(0x7f62ec0b5b20 msgr2=0x7f62ec0b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:47.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62eb7fe700 1 --2- 172.21.15.32:0/2642246773 >> v2:172.21.15.61:3300/0 conn(0x7f62ec0b5b20 0x7f62ec0b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:47.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62eb7fe700 1 -- 172.21.15.32:0/2642246773 >> v1:172.21.15.61:6789/0 conn(0x7f62ec0b6690 legacy=0x7f62ec0b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:47.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62eb7fe700 1 -- 172.21.15.32:0/2642246773 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec146400 con 0x7f62ec0b5240 2020-12-05T17:41:47.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62eb7fe700 1 --2- 172.21.15.32:0/2642246773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62ec0b5240 0x7f62ec145f10 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7f62e40032b0 tx=0x7f62e4009270).ready entity=mon.0 client_cookie=b1caee783cb35be5 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:47.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62e9ffb700 1 -- 172.21.15.32:0/2642246773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62e400bc00 con 0x7f62ec0b5240 2020-12-05T17:41:47.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62e9ffb700 1 -- 172.21.15.32:0/2642246773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e400bd60 con 0x7f62ec0b5240 2020-12-05T17:41:47.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62e9ffb700 1 -- 172.21.15.32:0/2642246773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62e4011400 con 0x7f62ec0b5240 2020-12-05T17:41:47.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62f16ce700 1 -- 172.21.15.32:0/2642246773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62ec0b5240 msgr2=0x7f62ec145f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:47.639 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62f16ce700 1 --2- 172.21.15.32:0/2642246773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62ec0b5240 0x7f62ec145f10 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7f62e40032b0 tx=0x7f62e4009270).stop 2020-12-05T17:41:47.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62f16ce700 1 -- 172.21.15.32:0/2642246773 shutdown_connections 2020-12-05T17:41:47.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62f16ce700 1 --2- 172.21.15.32:0/2642246773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62ec0b5240 0x7f62ec145f10 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:47.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62f16ce700 1 --2- 172.21.15.32:0/2642246773 >> v2:172.21.15.61:3300/0 conn(0x7f62ec0b5b20 0x7f62ec0b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:47.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62f16ce700 1 -- 172.21.15.32:0/2642246773 >> 172.21.15.32:0/2642246773 conn(0x7f62ec05f3b0 msgr2=0x7f62ec05f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:47.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62f16ce700 1 -- 172.21.15.32:0/2642246773 shutdown_connections 2020-12-05T17:41:47.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.634+0000 7f62f16ce700 1 -- 172.21.15.32:0/2642246773 wait complete. 2020-12-05T17:41:47.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62f16ce700 1 Processor -- start 2020-12-05T17:41:47.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62f16ce700 1 -- start start 2020-12-05T17:41:47.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62f16ce700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f62ec0b5240 0x7f62ec0a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:47.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62f16ce700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f62ec0b5b20 0x7f62ec152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:47.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62f16ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62ec0b6690 0x7f62ec063fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:47.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62f16ce700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0a4c30 con 0x7f62ec0b6690 2020-12-05T17:41:47.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62f16ce700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0a4010 con 0x7f62ec0b5b20 2020-12-05T17:41:47.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62f16ce700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0a2640 con 0x7f62ec0b5240 2020-12-05T17:41:47.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62eb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62ec0b6690 0x7f62ec063fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:47.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62eb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62ec0b6690 0x7f62ec063fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41314/0 (socket says 172.21.15.32:41314) 2020-12-05T17:41:47.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62eb7fe700 1 -- 172.21.15.32:0/1498556813 learned_addr learned my addr 172.21.15.32:0/1498556813 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:47.644 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62eaffd700 1 --2- 172.21.15.32:0/1498556813 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f62ec0b5240 0x7f62ec0a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:47.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62ea7fc700 1 --2- 172.21.15.32:0/1498556813 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f62ec0b5b20 0x7f62ec152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:47.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62eb7fe700 1 -- 172.21.15.32:0/1498556813 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f62ec0b5240 msgr2=0x7f62ec0a3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:47.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62eb7fe700 1 --2- 172.21.15.32:0/1498556813 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f62ec0b5240 0x7f62ec0a3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:47.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62eb7fe700 1 -- 172.21.15.32:0/1498556813 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f62ec0b5b20 msgr2=0x7f62ec152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:47.645 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62eb7fe700 1 --2- 172.21.15.32:0/1498556813 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f62ec0b5b20 0x7f62ec152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:47.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62eb7fe700 1 -- 172.21.15.32:0/1498556813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec066b00 con 0x7f62ec0b6690 2020-12-05T17:41:47.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62eb7fe700 1 --2- 172.21.15.32:0/1498556813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62ec0b6690 0x7f62ec063fd0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7f62e40032b0 tx=0x7f62e4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:47.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62e3fff700 1 -- 172.21.15.32:0/1498556813 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62e401e610 con 0x7f62ec0b6690 2020-12-05T17:41:47.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62e3fff700 1 -- 172.21.15.32:0/1498556813 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e401e770 con 0x7f62ec0b6690 2020-12-05T17:41:47.646 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62e3fff700 1 -- 172.21.15.32:0/1498556813 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62e401ead0 con 0x7f62ec0b6690 2020-12-05T17:41:47.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62f16ce700 1 -- 172.21.15.32:0/1498556813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ec152cf0 con 0x7f62ec0b6690 2020-12-05T17:41:47.647 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.638+0000 7f62f16ce700 1 -- 172.21.15.32:0/1498556813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ec0a2be0 con 0x7f62ec0b6690 2020-12-05T17:41:47.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.642+0000 7f62f16ce700 1 -- 172.21.15.32:0/1498556813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ec0a2800 con 0x7f62ec0b6690 2020-12-05T17:41:47.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.642+0000 7f62e3fff700 1 -- 172.21.15.32:0/1498556813 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f62e400f070 con 0x7f62ec0b6690 2020-12-05T17:41:47.648 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.642+0000 7f62e3fff700 1 --2- 172.21.15.32:0/1498556813 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f62d0055440 0x7f62d0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:47.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.642+0000 7f62eaffd700 1 --2- 172.21.15.32:0/1498556813 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f62d0055440 0x7f62d0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:47.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.642+0000 7f62e3fff700 1 -- 172.21.15.32:0/1498556813 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f62e406f6a0 con 0x7f62ec0b6690 2020-12-05T17:41:47.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.642+0000 7f62eaffd700 1 --2- 172.21.15.32:0/1498556813 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f62d0055440 0x7f62d0057900 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f62d400c000 tx=0x7f62d4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:47.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.646+0000 7f62e3fff700 1 -- 172.21.15.32:0/1498556813 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f62e4045750 con 0x7f62ec0b6690 2020-12-05T17:41:47.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:47.830+0000 7f62f16ce700 1 -- 172.21.15.32:0/1498556813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f62ec057050 con 0x7f62ec0b6690 2020-12-05T17:41:48.126 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.126+0000 7f62e3fff700 1 -- 172.21.15.32:0/1498556813 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v68) v1 ==== 104+0+0 (secure 0 0 0) 0x7f62e4048d70 con 0x7f62ec0b6690 2020-12-05T17:41:48.130 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.130+0000 7f62f16ce700 1 -- 172.21.15.32:0/1498556813 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f62d0055440 msgr2=0x7f62d0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:48.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.130+0000 7f62f16ce700 1 --2- 172.21.15.32:0/1498556813 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f62d0055440 0x7f62d0057900 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f62d400c000 tx=0x7f62d4008040).stop 2020-12-05T17:41:48.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.130+0000 7f62f16ce700 1 -- 172.21.15.32:0/1498556813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62ec0b6690 msgr2=0x7f62ec063fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:48.131 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.130+0000 7f62f16ce700 1 --2- 172.21.15.32:0/1498556813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62ec0b6690 0x7f62ec063fd0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7f62e40032b0 tx=0x7f62e4016000).stop 2020-12-05T17:41:48.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.130+0000 7f62f16ce700 1 -- 172.21.15.32:0/1498556813 shutdown_connections 2020-12-05T17:41:48.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.130+0000 7f62f16ce700 1 --2- 172.21.15.32:0/1498556813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62ec0b6690 0x7f62ec063fd0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:48.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.130+0000 7f62f16ce700 1 --2- 172.21.15.32:0/1498556813 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f62d0055440 0x7f62d0057900 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:48.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.130+0000 7f62f16ce700 1 --2- 172.21.15.32:0/1498556813 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f62ec0b5240 0x7f62ec0a3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:48.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.130+0000 7f62f16ce700 1 --2- 172.21.15.32:0/1498556813 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f62ec0b5b20 0x7f62ec152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:48.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.130+0000 7f62f16ce700 1 -- 172.21.15.32:0/1498556813 >> 172.21.15.32:0/1498556813 conn(0x7f62ec05f3b0 msgr2=0x7f62ec143e10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:48.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.130+0000 7f62f16ce700 1 -- 172.21.15.32:0/1498556813 shutdown_connections 2020-12-05T17:41:48.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.130+0000 7f62f16ce700 1 -- 172.21.15.32:0/1498556813 wait complete. 2020-12-05T17:41:48.147 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-12-05T17:41:48.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.274+0000 7f0e46fb1700 1 Processor -- start 2020-12-05T17:41:48.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.274+0000 7f0e46fb1700 1 -- start start 2020-12-05T17:41:48.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e46fb1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e4005f3b0 0x7f0e4005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:48.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e46fb1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0e400b2fb0 0x7f0e40061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:48.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e46fb1700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e400b2df0 con 0x7f0e4005feb0 2020-12-05T17:41:48.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e46fb1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e400b3c90 con 0x7f0e400b2fb0 2020-12-05T17:41:48.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e46fb1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e400b2b50 con 0x7f0e4005f3b0 2020-12-05T17:41:48.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e467b0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0e400b2fb0 0x7f0e40061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:48.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e467b0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0e400b2fb0 0x7f0e40061780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41324/0 (socket says 172.21.15.32:41324) 2020-12-05T17:41:48.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e467b0700 1 -- 172.21.15.32:0/2536065136 learned_addr learned my addr 172.21.15.32:0/2536065136 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:48.280 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e45faf700 1 --2- 172.21.15.32:0/2536065136 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e4005f3b0 0x7f0e4005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:48.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e44fad700 1 -- 172.21.15.32:0/2536065136 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1427082690 0 0) 0x7f0e400b2df0 con 0x7f0e4005feb0 2020-12-05T17:41:48.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e44fad700 1 -- 172.21.15.32:0/2536065136 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e28003230 con 0x7f0e4005feb0 2020-12-05T17:41:48.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e467b0700 1 -- 172.21.15.32:0/2536065136 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e4005f3b0 msgr2=0x7f0e4005f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:48.284 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e467b0700 1 --2- 172.21.15.32:0/2536065136 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e4005f3b0 0x7f0e4005f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:48.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e467b0700 1 -- 172.21.15.32:0/2536065136 >> v1:172.21.15.61:6789/0 conn(0x7f0e4005feb0 legacy=0x7f0e4013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:48.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e467b0700 1 -- 172.21.15.32:0/2536065136 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e40142820 con 0x7f0e400b2fb0 2020-12-05T17:41:48.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e467b0700 1 --2- 172.21.15.32:0/2536065136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e400b2fb0 0x7f0e40061780 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7f0e380032b0 tx=0x7f0e38009270).ready entity=mon.0 client_cookie=52f667ba124f8576 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:48.285 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e44fad700 1 -- 172.21.15.32:0/2536065136 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e38009e60 con 0x7f0e400b2fb0 2020-12-05T17:41:48.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e44fad700 1 -- 172.21.15.32:0/2536065136 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3800bb50 con 0x7f0e400b2fb0 2020-12-05T17:41:48.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e44fad700 1 -- 172.21.15.32:0/2536065136 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e3800bcc0 con 0x7f0e400b2fb0 2020-12-05T17:41:48.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e46fb1700 1 -- 172.21.15.32:0/2536065136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e400b2fb0 msgr2=0x7f0e40061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:48.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e46fb1700 1 --2- 172.21.15.32:0/2536065136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e400b2fb0 0x7f0e40061780 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7f0e380032b0 tx=0x7f0e38009270).stop 2020-12-05T17:41:48.286 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e46fb1700 1 -- 172.21.15.32:0/2536065136 shutdown_connections 2020-12-05T17:41:48.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e46fb1700 1 --2- 172.21.15.32:0/2536065136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e400b2fb0 0x7f0e40061780 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:48.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e46fb1700 1 --2- 172.21.15.32:0/2536065136 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e4005f3b0 0x7f0e4005f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:48.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e46fb1700 1 -- 172.21.15.32:0/2536065136 >> 172.21.15.32:0/2536065136 conn(0x7f0e400b8f20 msgr2=0x7f0e400b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:48.287 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e46fb1700 1 -- 172.21.15.32:0/2536065136 shutdown_connections 2020-12-05T17:41:48.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.278+0000 7f0e46fb1700 1 -- 172.21.15.32:0/2536065136 wait complete. 2020-12-05T17:41:48.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e46fb1700 1 Processor -- start 2020-12-05T17:41:48.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e46fb1700 1 -- start start 2020-12-05T17:41:48.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e46fb1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e4005f3b0 0x7f0e40147a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:48.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e46fb1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4005feb0 0x7f0e40149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:48.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e46fb1700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e400b2fb0 0x7f0e40149540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:48.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e46fb1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e40148b30 con 0x7f0e4005feb0 2020-12-05T17:41:48.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e46fb1700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e40148890 con 0x7f0e400b2fb0 2020-12-05T17:41:48.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e46fb1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e40147490 con 0x7f0e4005f3b0 2020-12-05T17:41:48.289 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e457ae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4005feb0 0x7f0e40149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:48.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e457ae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4005feb0 0x7f0e40149050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41328/0 (socket says 172.21.15.32:41328) 2020-12-05T17:41:48.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e457ae700 1 -- 172.21.15.32:0/3996018102 learned_addr learned my addr 172.21.15.32:0/3996018102 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:48.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e45faf700 1 --2- 172.21.15.32:0/3996018102 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e4005f3b0 0x7f0e40147a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:48.290 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e467b0700 1 --2- 172.21.15.32:0/3996018102 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e400b2fb0 0x7f0e40149540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:48.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e457ae700 1 -- 172.21.15.32:0/3996018102 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e4005f3b0 msgr2=0x7f0e40147a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:48.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e457ae700 1 --2- 172.21.15.32:0/3996018102 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e4005f3b0 0x7f0e40147a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:48.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e457ae700 1 -- 172.21.15.32:0/3996018102 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e400b2fb0 msgr2=0x7f0e40149540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:48.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e457ae700 1 --2- 172.21.15.32:0/3996018102 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e400b2fb0 0x7f0e40149540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:48.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e457ae700 1 -- 172.21.15.32:0/3996018102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e40113d80 con 0x7f0e4005feb0 2020-12-05T17:41:48.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e467b0700 1 --2- 172.21.15.32:0/3996018102 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e400b2fb0 0x7f0e40149540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:48.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e457ae700 1 --2- 172.21.15.32:0/3996018102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4005feb0 0x7f0e40149050 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 rx=0x7f0e300010d0 tx=0x7f0e30007b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:48.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e2effd700 1 -- 172.21.15.32:0/3996018102 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e30010510 con 0x7f0e4005feb0 2020-12-05T17:41:48.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e2effd700 1 -- 172.21.15.32:0/3996018102 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e30010670 con 0x7f0e4005feb0 2020-12-05T17:41:48.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e46fb1700 1 -- 172.21.15.32:0/3996018102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e40140ae0 con 0x7f0e4005feb0 2020-12-05T17:41:48.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e2effd700 1 -- 172.21.15.32:0/3996018102 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e300107e0 con 0x7f0e4005feb0 2020-12-05T17:41:48.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.282+0000 7f0e46fb1700 1 -- 172.21.15.32:0/3996018102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e40141010 con 0x7f0e4005feb0 2020-12-05T17:41:48.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.286+0000 7f0e2effd700 1 -- 172.21.15.32:0/3996018102 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0e30010070 con 0x7f0e4005feb0 2020-12-05T17:41:48.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.286+0000 7f0e46fb1700 1 -- 172.21.15.32:0/3996018102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e40149a30 con 0x7f0e4005feb0 2020-12-05T17:41:48.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.286+0000 7f0e2effd700 1 --2- 172.21.15.32:0/3996018102 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0e20055440 0x7f0e20057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:48.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.286+0000 7f0e2effd700 1 -- 172.21.15.32:0/3996018102 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f0e3006db00 con 0x7f0e4005feb0 2020-12-05T17:41:48.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.286+0000 7f0e45faf700 1 --2- 172.21.15.32:0/3996018102 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0e20055440 0x7f0e20057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:48.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.286+0000 7f0e45faf700 1 --2- 172.21.15.32:0/3996018102 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0e20055440 0x7f0e20057900 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f0e3c00b000 tx=0x7f0e3c006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:48.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.290+0000 7f0e2effd700 1 -- 172.21.15.32:0/3996018102 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0e40149a30 con 0x7f0e4005feb0 2020-12-05T17:41:48.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:48.438+0000 7f0e46fb1700 1 -- 172.21.15.32:0/3996018102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0e40149a30 con 0x7f0e4005feb0 2020-12-05T17:41:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.138+0000 7f0e2effd700 1 -- 172.21.15.32:0/3996018102 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v140) v1 ==== 185+0+0 (secure 0 0 0) 0x7f0e40149a30 con 0x7f0e4005feb0 2020-12-05T17:41:49.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.142+0000 7f0e46fb1700 1 -- 172.21.15.32:0/3996018102 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0e20055440 msgr2=0x7f0e20057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:49.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.142+0000 7f0e46fb1700 1 --2- 172.21.15.32:0/3996018102 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0e20055440 0x7f0e20057900 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f0e3c00b000 tx=0x7f0e3c006dc0).stop 2020-12-05T17:41:49.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.142+0000 7f0e46fb1700 1 -- 172.21.15.32:0/3996018102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4005feb0 msgr2=0x7f0e40149050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:49.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.142+0000 7f0e46fb1700 1 --2- 172.21.15.32:0/3996018102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4005feb0 0x7f0e40149050 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 rx=0x7f0e300010d0 tx=0x7f0e30007b30).stop 2020-12-05T17:41:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.142+0000 7f0e46fb1700 1 -- 172.21.15.32:0/3996018102 shutdown_connections 2020-12-05T17:41:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.142+0000 7f0e46fb1700 1 --2- 172.21.15.32:0/3996018102 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e400b2fb0 0x7f0e40149540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.142+0000 7f0e46fb1700 1 --2- 172.21.15.32:0/3996018102 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f0e20055440 0x7f0e20057900 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.142+0000 7f0e46fb1700 1 --2- 172.21.15.32:0/3996018102 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e4005f3b0 0x7f0e40147a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.142+0000 7f0e46fb1700 1 --2- 172.21.15.32:0/3996018102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4005feb0 0x7f0e40149050 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:49.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.142+0000 7f0e46fb1700 1 -- 172.21.15.32:0/3996018102 >> 172.21.15.32:0/3996018102 conn(0x7f0e400b8f20 msgr2=0x7f0e400b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:49.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.142+0000 7f0e46fb1700 1 -- 172.21.15.32:0/3996018102 shutdown_connections 2020-12-05T17:41:49.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.142+0000 7f0e46fb1700 1 -- 172.21.15.32:0/3996018102 wait complete. 2020-12-05T17:41:49.146 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2020-12-05T17:41:49.160 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-12-05T17:41:49.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.286+0000 7f2895901700 1 Processor -- start 2020-12-05T17:41:49.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.286+0000 7f2895901700 1 -- start start 2020-12-05T17:41:49.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f2895901700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f289005feb0 0x7f289005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:49.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f2895901700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28900b2fb0 0x7f2890061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:49.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f2895901700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28900b2df0 con 0x7f289005f3b0 2020-12-05T17:41:49.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f2895901700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28900b3c90 con 0x7f28900b2fb0 2020-12-05T17:41:49.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f2895901700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28900b2b50 con 0x7f289005feb0 2020-12-05T17:41:49.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f2895100700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28900b2fb0 0x7f2890061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:49.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f2895100700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28900b2fb0 0x7f2890061780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41338/0 (socket says 172.21.15.32:41338) 2020-12-05T17:41:49.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f2895100700 1 -- 172.21.15.32:0/536923437 learned_addr learned my addr 172.21.15.32:0/536923437 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:49.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f288ffff700 1 --2- 172.21.15.32:0/536923437 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f289005feb0 0x7f289005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:49.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f2895100700 1 -- 172.21.15.32:0/536923437 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f289005feb0 msgr2=0x7f289005f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:49.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f2895100700 1 --2- 172.21.15.32:0/536923437 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f289005feb0 0x7f289005f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:49.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f2895100700 1 -- 172.21.15.32:0/536923437 >> v1:172.21.15.61:6789/0 conn(0x7f289005f3b0 legacy=0x7f289013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:49.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f2895100700 1 -- 172.21.15.32:0/536923437 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2890142820 con 0x7f28900b2fb0 2020-12-05T17:41:49.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f2895100700 1 --2- 172.21.15.32:0/536923437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28900b2fb0 0x7f2890061780 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7f28880032b0 tx=0x7f2888009270).ready entity=mon.0 client_cookie=19222504bcd09de1 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:49.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f288f7fe700 1 -- 172.21.15.32:0/536923437 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f288800bc00 con 0x7f28900b2fb0 2020-12-05T17:41:49.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f288f7fe700 1 -- 172.21.15.32:0/536923437 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288800bd60 con 0x7f28900b2fb0 2020-12-05T17:41:49.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f288f7fe700 1 -- 172.21.15.32:0/536923437 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2888011400 con 0x7f28900b2fb0 2020-12-05T17:41:49.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f2895901700 1 -- 172.21.15.32:0/536923437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28900b2fb0 msgr2=0x7f2890061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:49.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f2895901700 1 --2- 172.21.15.32:0/536923437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28900b2fb0 0x7f2890061780 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7f28880032b0 tx=0x7f2888009270).stop 2020-12-05T17:41:49.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f2895901700 1 -- 172.21.15.32:0/536923437 shutdown_connections 2020-12-05T17:41:49.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f2895901700 1 --2- 172.21.15.32:0/536923437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28900b2fb0 0x7f2890061780 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:49.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f2895901700 1 --2- 172.21.15.32:0/536923437 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f289005feb0 0x7f289005f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:49.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f2895901700 1 -- 172.21.15.32:0/536923437 >> 172.21.15.32:0/536923437 conn(0x7f28900b8f20 msgr2=0x7f28900b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:49.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f2895901700 1 -- 172.21.15.32:0/536923437 shutdown_connections 2020-12-05T17:41:49.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.290+0000 7f2895901700 1 -- 172.21.15.32:0/536923437 wait complete. 2020-12-05T17:41:49.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f2895901700 1 Processor -- start 2020-12-05T17:41:49.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f2895901700 1 -- start start 2020-12-05T17:41:49.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f2895901700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f289005f3b0 0x7f2890149100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:49.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f2895901700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289005feb0 0x7f28901495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:49.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f2895901700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f28900b2fb0 0x7f2890140a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:49.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f2895901700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2890114330 con 0x7f289005feb0 2020-12-05T17:41:49.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f2895901700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2890113e10 con 0x7f289005f3b0 2020-12-05T17:41:49.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f2895901700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2890114090 con 0x7f28900b2fb0 2020-12-05T17:41:49.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f288ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289005feb0 0x7f28901495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:49.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f288ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289005feb0 0x7f28901495f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41342/0 (socket says 172.21.15.32:41342) 2020-12-05T17:41:49.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f288ffff700 1 -- 172.21.15.32:0/2260004072 learned_addr learned my addr 172.21.15.32:0/2260004072 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:49.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f28948ff700 1 --2- 172.21.15.32:0/2260004072 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f289005f3b0 0x7f2890149100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:49.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f2895100700 1 --2- 172.21.15.32:0/2260004072 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f28900b2fb0 0x7f2890140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:49.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f288ffff700 1 -- 172.21.15.32:0/2260004072 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f28900b2fb0 msgr2=0x7f2890140a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f288ffff700 1 --2- 172.21.15.32:0/2260004072 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f28900b2fb0 0x7f2890140a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f288ffff700 1 -- 172.21.15.32:0/2260004072 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f289005f3b0 msgr2=0x7f2890149100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f288ffff700 1 --2- 172.21.15.32:0/2260004072 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f289005f3b0 0x7f2890149100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f288ffff700 1 -- 172.21.15.32:0/2260004072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2890147600 con 0x7f289005feb0 2020-12-05T17:41:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f288ffff700 1 --2- 172.21.15.32:0/2260004072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289005feb0 0x7f28901495f0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 rx=0x7f2880003070 tx=0x7f2880008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f288dffb700 1 -- 172.21.15.32:0/2260004072 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2880009980 con 0x7f289005feb0 2020-12-05T17:41:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f288dffb700 1 -- 172.21.15.32:0/2260004072 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2880009ae0 con 0x7f289005feb0 2020-12-05T17:41:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f2895901700 1 -- 172.21.15.32:0/2260004072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2890147aa0 con 0x7f289005feb0 2020-12-05T17:41:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f288dffb700 1 -- 172.21.15.32:0/2260004072 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2880009e40 con 0x7f289005feb0 2020-12-05T17:41:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.294+0000 7f2895901700 1 -- 172.21.15.32:0/2260004072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2890148a40 con 0x7f289005feb0 2020-12-05T17:41:49.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.298+0000 7f288dffb700 1 -- 172.21.15.32:0/2260004072 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2880013480 con 0x7f289005feb0 2020-12-05T17:41:49.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.298+0000 7f2895901700 1 -- 172.21.15.32:0/2260004072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2890057050 con 0x7f289005feb0 2020-12-05T17:41:49.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.298+0000 7f288dffb700 1 --2- 172.21.15.32:0/2260004072 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2874055330 0x7f28740577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:49.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.298+0000 7f288dffb700 1 -- 172.21.15.32:0/2260004072 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f288006e920 con 0x7f289005feb0 2020-12-05T17:41:49.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.298+0000 7f28948ff700 1 --2- 172.21.15.32:0/2260004072 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2874055330 0x7f28740577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:49.306 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.302+0000 7f28948ff700 1 --2- 172.21.15.32:0/2260004072 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2874055330 0x7f28740577f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f287800d000 tx=0x7f2878000b80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:49.309 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.306+0000 7f288dffb700 1 -- 172.21.15.32:0/2260004072 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2880044af0 con 0x7f289005feb0 2020-12-05T17:41:49.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:49.446+0000 7f2895901700 1 -- 172.21.15.32:0/2260004072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f289005aab0 con 0x7f289005feb0 2020-12-05T17:41:50.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.142+0000 7f288dffb700 1 -- 172.21.15.32:0/2260004072 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v141) v1 ==== 173+0+0 (secure 0 0 0) 0x7f2880048110 con 0x7f289005feb0 2020-12-05T17:41:50.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.146+0000 7f2895901700 1 -- 172.21.15.32:0/2260004072 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2874055330 msgr2=0x7f28740577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:50.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.146+0000 7f2895901700 1 --2- 172.21.15.32:0/2260004072 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2874055330 0x7f28740577f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f287800d000 tx=0x7f2878000b80).stop 2020-12-05T17:41:50.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.146+0000 7f2895901700 1 -- 172.21.15.32:0/2260004072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289005feb0 msgr2=0x7f28901495f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:50.149 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.146+0000 7f2895901700 1 --2- 172.21.15.32:0/2260004072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289005feb0 0x7f28901495f0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 rx=0x7f2880003070 tx=0x7f2880008040).stop 2020-12-05T17:41:50.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.146+0000 7f2895901700 1 -- 172.21.15.32:0/2260004072 shutdown_connections 2020-12-05T17:41:50.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.146+0000 7f2895901700 1 --2- 172.21.15.32:0/2260004072 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f2874055330 0x7f28740577f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:50.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.146+0000 7f2895901700 1 --2- 172.21.15.32:0/2260004072 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f28900b2fb0 0x7f2890140a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:50.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.146+0000 7f2895901700 1 --2- 172.21.15.32:0/2260004072 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f289005f3b0 0x7f2890149100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:50.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.146+0000 7f2895901700 1 --2- 172.21.15.32:0/2260004072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289005feb0 0x7f28901495f0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:50.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.146+0000 7f2895901700 1 -- 172.21.15.32:0/2260004072 >> 172.21.15.32:0/2260004072 conn(0x7f28900b8f20 msgr2=0x7f28900b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:50.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.146+0000 7f2895901700 1 -- 172.21.15.32:0/2260004072 shutdown_connections 2020-12-05T17:41:50.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.146+0000 7f2895901700 1 -- 172.21.15.32:0/2260004072 wait complete. 2020-12-05T17:41:50.151 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2020-12-05T17:41:50.163 INFO:tasks.ceph.mds.a:Restarting daemon 2020-12-05T17:41:50.163 INFO:tasks.ceph.mds.a:Stopping old one... 2020-12-05T17:41:50.164 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-12-05T17:41:50.164 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:41:50.164 INFO:tasks.ceph.mds.b:Restarting daemon 2020-12-05T17:41:50.164 INFO:tasks.ceph.mds.b:Stopping old one... 2020-12-05T17:41:50.165 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-12-05T17:41:50.165 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T17:41:50.173 INFO:tasks.ceph.mds.a:Stopped 2020-12-05T17:41:50.173 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-12-05T17:41:50.176 INFO:tasks.ceph.mds.b:Stopped 2020-12-05T17:41:50.176 INFO:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-12-05T17:41:50.218 INFO:tasks.ceph.mds.b:Started 2020-12-05T17:41:50.219 DEBUG:teuthology.parallel:result is None 2020-12-05T17:41:50.220 INFO:tasks.ceph.mds.a:Started 2020-12-05T17:41:50.220 DEBUG:teuthology.parallel:result is None 2020-12-05T17:41:50.221 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-12-05T17:41:50.296 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:41:50.297 INFO:tasks.ceph.mds.b.smithi061.stdout:starting mds.b at 2020-12-05T17:41:50.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.338+0000 7f9574a7f700 1 Processor -- start 2020-12-05T17:41:50.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.338+0000 7f9574a7f700 1 -- start start 2020-12-05T17:41:50.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.338+0000 7f9574a7f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f957005f6f0 0x7f95701442e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:50.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.338+0000 7f9574a7f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f95700b2230 0x7f95701447d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:50.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.338+0000 7f9574a7f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f957005fad0 con 0x7f95700acb80 2020-12-05T17:41:50.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.338+0000 7f9574a7f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9570144cc0 con 0x7f95700b2230 2020-12-05T17:41:50.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.338+0000 7f9574a7f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95700ad210 con 0x7f957005f6f0 2020-12-05T17:41:50.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.338+0000 7f956ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f95700b2230 0x7f95701447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:50.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.338+0000 7f956ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f95700b2230 0x7f95701447d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41364/0 (socket says 172.21.15.32:41364) 2020-12-05T17:41:50.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.338+0000 7f956ed9d700 1 -- 172.21.15.32:0/3182415933 learned_addr learned my addr 172.21.15.32:0/3182415933 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:50.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.338+0000 7f956dd9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f95700acb80 0x7f95700b3870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44010/0 (socket says 172.21.15.32:44010) 2020-12-05T17:41:50.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.338+0000 7f956e59c700 1 --2- 172.21.15.32:0/3182415933 >> v2:172.21.15.61:3300/0 conn(0x7f957005f6f0 0x7f95701442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:50.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.338+0000 7f956e59c700 1 -- 172.21.15.32:0/3182415933 >> v2:172.21.15.32:3300/0 conn(0x7f95700b2230 msgr2=0x7f95701447d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:50.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.338+0000 7f956e59c700 1 --2- 172.21.15.32:0/3182415933 >> v2:172.21.15.32:3300/0 conn(0x7f95700b2230 0x7f95701447d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:50.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.338+0000 7f956e59c700 1 -- 172.21.15.32:0/3182415933 >> v1:172.21.15.32:6789/0 conn(0x7f95700acb80 legacy=0x7f95700b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:50.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.338+0000 7f956e59c700 1 -- 172.21.15.32:0/3182415933 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9570143e40 con 0x7f957005f6f0 2020-12-05T17:41:50.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.338+0000 7f956ed9d700 1 --2- 172.21.15.32:0/3182415933 >> v2:172.21.15.32:3300/0 conn(0x7f95700b2230 0x7f95701447d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:50.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.342+0000 7f956e59c700 1 --2- 172.21.15.32:0/3182415933 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f957005f6f0 0x7f95701442e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f9560002f30 tx=0x7f9560006dc0).ready entity=mon.1 client_cookie=2acc233dd50b991f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:50.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.342+0000 7f956d59a700 1 -- 172.21.15.32:0/3182415933 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95600097b0 con 0x7f957005f6f0 2020-12-05T17:41:50.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.342+0000 7f956d59a700 1 -- 172.21.15.32:0/3182415933 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9560009910 con 0x7f957005f6f0 2020-12-05T17:41:50.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.342+0000 7f956d59a700 1 -- 172.21.15.32:0/3182415933 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9560009c70 con 0x7f957005f6f0 2020-12-05T17:41:50.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.342+0000 7f9574a7f700 1 -- 172.21.15.32:0/3182415933 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f957005f6f0 msgr2=0x7f95701442e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:50.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.342+0000 7f9574a7f700 1 --2- 172.21.15.32:0/3182415933 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f957005f6f0 0x7f95701442e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f9560002f30 tx=0x7f9560006dc0).stop 2020-12-05T17:41:50.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.342+0000 7f9574a7f700 1 -- 172.21.15.32:0/3182415933 shutdown_connections 2020-12-05T17:41:50.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.342+0000 7f9574a7f700 1 --2- 172.21.15.32:0/3182415933 >> v2:172.21.15.32:3300/0 conn(0x7f95700b2230 0x7f95701447d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:50.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.342+0000 7f9574a7f700 1 --2- 172.21.15.32:0/3182415933 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f957005f6f0 0x7f95701442e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:50.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.342+0000 7f9574a7f700 1 -- 172.21.15.32:0/3182415933 >> 172.21.15.32:0/3182415933 conn(0x7f95700b6ed0 msgr2=0x7f95700b72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:50.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.342+0000 7f9574a7f700 1 -- 172.21.15.32:0/3182415933 shutdown_connections 2020-12-05T17:41:50.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.342+0000 7f9574a7f700 1 -- 172.21.15.32:0/3182415933 wait complete. 2020-12-05T17:41:50.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.342+0000 7f9574a7f700 1 Processor -- start 2020-12-05T17:41:50.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.342+0000 7f9574a7f700 1 -- start start 2020-12-05T17:41:50.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.342+0000 7f9574a7f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f957005f6f0 0x7f9570065e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:50.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.342+0000 7f9574a7f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95700acb80 0x7f95700af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:50.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f9574a7f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f95700b2230 0x7f95701456a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:50.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f9574a7f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9570065250 con 0x7f95700acb80 2020-12-05T17:41:50.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f9574a7f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9570065480 con 0x7f95700b2230 2020-12-05T17:41:50.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f9574a7f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9570065720 con 0x7f957005f6f0 2020-12-05T17:41:50.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f956dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95700acb80 0x7f95700af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:50.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f956dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95700acb80 0x7f95700af8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41368/0 (socket says 172.21.15.32:41368) 2020-12-05T17:41:50.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f956dd9b700 1 -- 172.21.15.32:0/217784344 learned_addr learned my addr 172.21.15.32:0/217784344 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:50.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f956e59c700 1 --2- 172.21.15.32:0/217784344 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f957005f6f0 0x7f9570065e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:50.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f956ed9d700 1 --2- 172.21.15.32:0/217784344 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f95700b2230 0x7f95701456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:50.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f956dd9b700 1 -- 172.21.15.32:0/217784344 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f957005f6f0 msgr2=0x7f9570065e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:50.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f956dd9b700 1 --2- 172.21.15.32:0/217784344 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f957005f6f0 0x7f9570065e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:50.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f956dd9b700 1 -- 172.21.15.32:0/217784344 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f95700b2230 msgr2=0x7f95701456a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:50.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f956dd9b700 1 --2- 172.21.15.32:0/217784344 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f95700b2230 0x7f95701456a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:50.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f956dd9b700 1 -- 172.21.15.32:0/217784344 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95700659c0 con 0x7f95700acb80 2020-12-05T17:41:50.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f956dd9b700 1 --2- 172.21.15.32:0/217784344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95700acb80 0x7f95700af8c0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7f95700616d0 tx=0x7f9558007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:50.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f95677fe700 1 -- 172.21.15.32:0/217784344 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95580072e0 con 0x7f95700acb80 2020-12-05T17:41:50.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f95677fe700 1 -- 172.21.15.32:0/217784344 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9558005520 con 0x7f95700acb80 2020-12-05T17:41:50.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f9574a7f700 1 -- 172.21.15.32:0/217784344 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9570145b90 con 0x7f95700acb80 2020-12-05T17:41:50.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f95677fe700 1 -- 172.21.15.32:0/217784344 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9558005880 con 0x7f95700acb80 2020-12-05T17:41:50.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f9574a7f700 1 -- 172.21.15.32:0/217784344 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9570066b00 con 0x7f95700acb80 2020-12-05T17:41:50.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f9574a7f700 1 -- 172.21.15.32:0/217784344 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9570065b50 con 0x7f95700acb80 2020-12-05T17:41:50.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f95677fe700 1 -- 172.21.15.32:0/217784344 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f955801a060 con 0x7f95700acb80 2020-12-05T17:41:50.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f95677fe700 1 --2- 172.21.15.32:0/217784344 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9554055470 0x7f9554057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:50.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f95677fe700 1 -- 172.21.15.32:0/217784344 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f955806d430 con 0x7f95700acb80 2020-12-05T17:41:50.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.346+0000 7f956e59c700 1 --2- 172.21.15.32:0/217784344 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9554055470 0x7f9554057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:50.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.350+0000 7f956e59c700 1 --2- 172.21.15.32:0/217784344 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9554055470 0x7f9554057930 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f956001a000 tx=0x7f9560015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:50.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.354+0000 7f95677fe700 1 -- 172.21.15.32:0/217784344 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f95580438b0 con 0x7f95700acb80 2020-12-05T17:41:50.510 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:50.510+0000 7f9574a7f700 1 -- 172.21.15.32:0/217784344 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f9570057050 con 0x7f95700acb80 2020-12-05T17:41:51.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.342+0000 7f95677fe700 1 -- 172.21.15.32:0/217784344 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v143) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9558018020 con 0x7f95700acb80 2020-12-05T17:41:51.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.346+0000 7f9574a7f700 1 -- 172.21.15.32:0/217784344 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9554055470 msgr2=0x7f9554057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:51.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.346+0000 7f9574a7f700 1 --2- 172.21.15.32:0/217784344 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9554055470 0x7f9554057930 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f956001a000 tx=0x7f9560015000).stop 2020-12-05T17:41:51.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.346+0000 7f9574a7f700 1 -- 172.21.15.32:0/217784344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95700acb80 msgr2=0x7f95700af8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:51.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.346+0000 7f9574a7f700 1 --2- 172.21.15.32:0/217784344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95700acb80 0x7f95700af8c0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7f95700616d0 tx=0x7f9558007be0).stop 2020-12-05T17:41:51.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.346+0000 7f9574a7f700 1 -- 172.21.15.32:0/217784344 shutdown_connections 2020-12-05T17:41:51.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.346+0000 7f9574a7f700 1 --2- 172.21.15.32:0/217784344 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f95700b2230 0x7f95701456a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:51.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.346+0000 7f9574a7f700 1 --2- 172.21.15.32:0/217784344 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9554055470 0x7f9554057930 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:51.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.346+0000 7f9574a7f700 1 --2- 172.21.15.32:0/217784344 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f957005f6f0 0x7f9570065e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:51.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.346+0000 7f9574a7f700 1 --2- 172.21.15.32:0/217784344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95700acb80 0x7f95700af8c0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:51.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.346+0000 7f9574a7f700 1 -- 172.21.15.32:0/217784344 >> 172.21.15.32:0/217784344 conn(0x7f95700b6ed0 msgr2=0x7f957013f900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:51.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.346+0000 7f9574a7f700 1 -- 172.21.15.32:0/217784344 shutdown_connections 2020-12-05T17:41:51.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.346+0000 7f9574a7f700 1 -- 172.21.15.32:0/217784344 wait complete. 2020-12-05T17:41:51.350 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2020-12-05T17:41:51.362 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2020-12-05T17:41:51.362 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-12-05T17:41:51.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.478+0000 7fb87759e700 1 Processor -- start 2020-12-05T17:41:51.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.478+0000 7fb87759e700 1 -- start start 2020-12-05T17:41:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.478+0000 7fb87759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8780b5f00 0x7fb8780b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.478+0000 7fb87759e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb8780ad510 0x7fb8780af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:51.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.478+0000 7fb87759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb878142ab0 con 0x7fb878066370 2020-12-05T17:41:51.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.478+0000 7fb87759e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8780b62e0 con 0x7fb8780b5f00 2020-12-05T17:41:51.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.478+0000 7fb87759e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8780afbf0 con 0x7fb8780ad510 2020-12-05T17:41:51.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.478+0000 7fb87659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8780b5f00 0x7fb8780b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:51.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.478+0000 7fb87659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8780b5f00 0x7fb8780b14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41376/0 (socket says 172.21.15.32:41376) 2020-12-05T17:41:51.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.478+0000 7fb87659c700 1 -- 172.21.15.32:0/2843144720 learned_addr learned my addr 172.21.15.32:0/2843144720 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:51.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.478+0000 7fb875d9b700 1 --2- 172.21.15.32:0/2843144720 >> v2:172.21.15.61:3300/0 conn(0x7fb8780ad510 0x7fb8780af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:51.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.478+0000 7fb87659c700 1 -- 172.21.15.32:0/2843144720 >> v2:172.21.15.61:3300/0 conn(0x7fb8780ad510 msgr2=0x7fb8780af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:51.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.478+0000 7fb87659c700 1 --2- 172.21.15.32:0/2843144720 >> v2:172.21.15.61:3300/0 conn(0x7fb8780ad510 0x7fb8780af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:51.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.478+0000 7fb87659c700 1 -- 172.21.15.32:0/2843144720 >> v1:172.21.15.32:6789/0 conn(0x7fb878066370 legacy=0x7fb878142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:51.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.478+0000 7fb87559a700 1 -- 172.21.15.32:0/2843144720 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 805427785 0 0) 0x7fb878142ab0 con 0x7fb878066370 2020-12-05T17:41:51.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.478+0000 7fb87659c700 1 -- 172.21.15.32:0/2843144720 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8780afe90 con 0x7fb8780b5f00 2020-12-05T17:41:51.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.478+0000 7fb87659c700 1 --2- 172.21.15.32:0/2843144720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8780b5f00 0x7fb8780b14e0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7fb868002f30 tx=0x7fb868006e80).ready entity=mon.0 client_cookie=396d98afaed0f115 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.482+0000 7fb87559a700 1 -- 172.21.15.32:0/2843144720 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8680097f0 con 0x7fb8780b5f00 2020-12-05T17:41:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.482+0000 7fb87559a700 1 -- 172.21.15.32:0/2843144720 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb868009950 con 0x7fb8780b5f00 2020-12-05T17:41:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.482+0000 7fb87559a700 1 -- 172.21.15.32:0/2843144720 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb868009cb0 con 0x7fb8780b5f00 2020-12-05T17:41:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.482+0000 7fb87759e700 1 -- 172.21.15.32:0/2843144720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8780b5f00 msgr2=0x7fb8780b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.482+0000 7fb87759e700 1 --2- 172.21.15.32:0/2843144720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8780b5f00 0x7fb8780b14e0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7fb868002f30 tx=0x7fb868006e80).stop 2020-12-05T17:41:51.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.482+0000 7fb87759e700 1 -- 172.21.15.32:0/2843144720 shutdown_connections 2020-12-05T17:41:51.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.482+0000 7fb87759e700 1 --2- 172.21.15.32:0/2843144720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8780b5f00 0x7fb8780b14e0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:51.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.482+0000 7fb87759e700 1 --2- 172.21.15.32:0/2843144720 >> v2:172.21.15.61:3300/0 conn(0x7fb8780ad510 0x7fb8780af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:51.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.482+0000 7fb87759e700 1 -- 172.21.15.32:0/2843144720 >> 172.21.15.32:0/2843144720 conn(0x7fb8780604c0 msgr2=0x7fb87805c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:51.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.482+0000 7fb87759e700 1 -- 172.21.15.32:0/2843144720 shutdown_connections 2020-12-05T17:41:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.482+0000 7fb87759e700 1 -- 172.21.15.32:0/2843144720 wait complete. 2020-12-05T17:41:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.482+0000 7fb87759e700 1 Processor -- start 2020-12-05T17:41:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.482+0000 7fb87759e700 1 -- start start 2020-12-05T17:41:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb87759e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb878066370 0x7fb878144060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb87759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8780ad510 0x7fb878065350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:51.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb87759e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8780b5f00 0x7fb87814fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:51.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb87759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87814f460 con 0x7fb8780ad510 2020-12-05T17:41:51.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb87759e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8781528a0 con 0x7fb878066370 2020-12-05T17:41:51.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb87759e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87814f690 con 0x7fb8780b5f00 2020-12-05T17:41:51.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb875d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8780ad510 0x7fb878065350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:51.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb87659c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb878066370 0x7fb878144060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:51.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb875d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8780ad510 0x7fb878065350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41384/0 (socket says 172.21.15.32:41384) 2020-12-05T17:41:51.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb87659c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb878066370 0x7fb878144060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:33042/0 (socket says 172.21.15.32:33042) 2020-12-05T17:41:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb875d9b700 1 -- 172.21.15.32:0/4109146139 learned_addr learned my addr 172.21.15.32:0/4109146139 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb876d9d700 1 --2- 172.21.15.32:0/4109146139 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8780b5f00 0x7fb87814fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb875d9b700 1 -- 172.21.15.32:0/4109146139 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8780b5f00 msgr2=0x7fb87814fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb875d9b700 1 --2- 172.21.15.32:0/4109146139 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8780b5f00 0x7fb87814fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb875d9b700 1 -- 172.21.15.32:0/4109146139 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb878066370 msgr2=0x7fb878144060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb875d9b700 1 --2- 172.21.15.32:0/4109146139 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb878066370 0x7fb878144060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb875d9b700 1 -- 172.21.15.32:0/4109146139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb878065840 con 0x7fb8780ad510 2020-12-05T17:41:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb876d9d700 1 --2- 172.21.15.32:0/4109146139 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8780b5f00 0x7fb87814fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb87659c700 1 --2- 172.21.15.32:0/4109146139 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb878066370 0x7fb878144060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb875d9b700 1 --2- 172.21.15.32:0/4109146139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8780ad510 0x7fb878065350 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 rx=0x7fb86c003070 tx=0x7fb86c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb8677fe700 1 -- 172.21.15.32:0/4109146139 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb86c009790 con 0x7fb8780ad510 2020-12-05T17:41:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb8677fe700 1 -- 172.21.15.32:0/4109146139 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86c0098f0 con 0x7fb8780ad510 2020-12-05T17:41:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb87759e700 1 -- 172.21.15.32:0/4109146139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb878152b40 con 0x7fb8780ad510 2020-12-05T17:41:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb8677fe700 1 -- 172.21.15.32:0/4109146139 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb86c009c50 con 0x7fb8780ad510 2020-12-05T17:41:51.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.486+0000 7fb87759e700 1 -- 172.21.15.32:0/4109146139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8780b53d0 con 0x7fb8780ad510 2020-12-05T17:41:51.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.490+0000 7fb87759e700 1 -- 172.21.15.32:0/4109146139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb87814f850 con 0x7fb8780ad510 2020-12-05T17:41:51.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.490+0000 7fb8677fe700 1 -- 172.21.15.32:0/4109146139 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb86c013480 con 0x7fb8780ad510 2020-12-05T17:41:51.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.490+0000 7fb8677fe700 1 --2- 172.21.15.32:0/4109146139 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb854055440 0x7fb854057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.490+0000 7fb8677fe700 1 -- 172.21.15.32:0/4109146139 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fb86c06e0e0 con 0x7fb8780ad510 2020-12-05T17:41:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.490+0000 7fb87659c700 1 --2- 172.21.15.32:0/4109146139 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb854055440 0x7fb854057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.490+0000 7fb87659c700 1 --2- 172.21.15.32:0/4109146139 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb854055440 0x7fb854057900 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7fb86801c000 tx=0x7fb868015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.494+0000 7fb8677fe700 1 -- 172.21.15.32:0/4109146139 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb86c044560 con 0x7fb8780ad510 2020-12-05T17:41:51.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.686+0000 7fb87759e700 1 -- 172.21.15.32:0/4109146139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb878057050 con 0x7fb8780ad510 2020-12-05T17:41:51.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.686+0000 7fb8677fe700 1 -- 172.21.15.32:0/4109146139 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v43) v1 ==== 80+0+4242 (secure 0 0 0) 0x7fb86c009db0 con 0x7fb8780ad510 2020-12-05T17:41:51.689 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:51.689 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:41:51.689 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2020-12-05T17:41:51.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:51.690 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2020-12-05T17:41:51.690 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfpb/aExAAdMKW4yIgC2zhkzyW6l8xug==", 2020-12-05T17:41:51.690 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:51.691 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:41:51.691 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:41:51.691 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:41:51.691 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:51.691 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:51.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:51.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:51.692 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2020-12-05T17:41:51.692 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfBKTYGRAAaPkezwmTiAvkUfTuDGDNZA==", 2020-12-05T17:41:51.692 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:51.693 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:41:51.693 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:41:51.693 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:41:51.693 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:51.694 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:51.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:51.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:51.694 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2020-12-05T17:41:51.694 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHw8tflo5EKhAAwV9D64fAoY3uvXCm+Aksag==", 2020-12-05T17:41:51.695 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:51.695 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:41:51.695 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:41:51.695 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:51.695 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:51.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:51.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:51.696 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2020-12-05T17:41:51.696 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDKw8tfTjEiERAAkW6mkgApoXwyF49kPjZBGg==", 2020-12-05T17:41:51.696 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:51.697 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:41:51.697 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:41:51.697 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:51.697 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:51.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:51.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:51.698 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2020-12-05T17:41:51.698 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDMw8tfLEzBMxAAiJ2NifsOLEkt1e3vmhwSTA==", 2020-12-05T17:41:51.698 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:51.699 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:41:51.699 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:41:51.699 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:51.699 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:51.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:51.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:51.700 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2020-12-05T17:41:51.700 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPw8tfV+vnGxAAQHvXJRW5X9Vhe/eYJBJ/uQ==", 2020-12-05T17:41:51.700 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:51.700 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:41:51.701 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:41:51.701 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:51.701 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:51.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:51.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:51.702 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2020-12-05T17:41:51.702 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUw8tfN8wMJxAApc5LY9HGDeIV/o0eil/g4A==", 2020-12-05T17:41:51.702 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:51.702 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:41:51.703 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:41:51.703 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:51.703 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:51.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:51.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:51.704 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2020-12-05T17:41:51.704 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXw8tfvZ5dDxAAtAfftExkljlBvn7ZSoD/Og==", 2020-12-05T17:41:51.704 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:51.704 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:41:51.705 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:41:51.705 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:51.705 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:51.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:51.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:51.706 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2020-12-05T17:41:51.706 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDZw8tfhXFzMhAAZXNwNGfKv360Tq4N/DX5Mw==", 2020-12-05T17:41:51.706 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:51.706 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:41:51.707 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:41:51.707 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:51.707 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:51.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:51.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:51.708 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2020-12-05T17:41:51.708 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDcw8tfROfSGhAA/Y2Y1b4k5zQju0MM7LASeQ==", 2020-12-05T17:41:51.708 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:51.708 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:41:51.708 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:41:51.709 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:51.709 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:51.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:51.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:51.710 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2020-12-05T17:41:51.710 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfWDp/HxAAmvLPj7C3WDgj0OC2o7lBJQ==", 2020-12-05T17:41:51.710 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:51.710 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:41:51.710 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:41:51.711 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:41:51.711 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:51.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:51.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:51.711 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2020-12-05T17:41:51.712 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfugnmIRAAW6+/QGMLkHw+Z97RHpV4kw==", 2020-12-05T17:41:51.712 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:51.712 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:41:51.712 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:41:51.713 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:41:51.713 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:51.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:51.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:51.713 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2020-12-05T17:41:51.714 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfojyqJBAAyta+dSy2yGnC1kKe3YOAYw==", 2020-12-05T17:41:51.714 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:51.714 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2020-12-05T17:41:51.714 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2020-12-05T17:41:51.714 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2020-12-05T17:41:51.715 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:51.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:51.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:51.715 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2020-12-05T17:41:51.715 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDFw8tfES0PLBAAA1cUV44snAUDF2TyAmXImw==", 2020-12-05T17:41:51.716 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:51.716 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:41:51.716 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2020-12-05T17:41:51.716 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2020-12-05T17:41:51.716 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:51.717 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:51.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:51.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:51.717 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2020-12-05T17:41:51.718 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tfDr8mERAAS7bpAQXgW78W8GDq1v1zaA==", 2020-12-05T17:41:51.718 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:51.718 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:41:51.718 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:41:51.718 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:51.719 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:51.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:51.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:51.719 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2020-12-05T17:41:51.719 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDGw8tf3HmcDhAATNKHhH9zgasedaNEzgG0OQ==", 2020-12-05T17:41:51.720 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2020-12-05T17:41:51.720 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2020-12-05T17:41:51.720 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2020-12-05T17:41:51.720 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2020-12-05T17:41:51.720 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:51.721 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:51.721 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:41:51.721 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:41:51.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.690+0000 7fb87759e700 1 -- 172.21.15.32:0/4109146139 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb854055440 msgr2=0x7fb854057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:51.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.690+0000 7fb87759e700 1 --2- 172.21.15.32:0/4109146139 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb854055440 0x7fb854057900 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7fb86801c000 tx=0x7fb868015000).stop 2020-12-05T17:41:51.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.690+0000 7fb87759e700 1 -- 172.21.15.32:0/4109146139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8780ad510 msgr2=0x7fb878065350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:51.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.690+0000 7fb87759e700 1 --2- 172.21.15.32:0/4109146139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8780ad510 0x7fb878065350 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 rx=0x7fb86c003070 tx=0x7fb86c008040).stop 2020-12-05T17:41:51.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.690+0000 7fb87759e700 1 -- 172.21.15.32:0/4109146139 shutdown_connections 2020-12-05T17:41:51.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.690+0000 7fb87759e700 1 --2- 172.21.15.32:0/4109146139 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb854055440 0x7fb854057900 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:51.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.690+0000 7fb87759e700 1 --2- 172.21.15.32:0/4109146139 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8780b5f00 0x7fb87814fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:51.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.690+0000 7fb87759e700 1 --2- 172.21.15.32:0/4109146139 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb878066370 0x7fb878144060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.690+0000 7fb87759e700 1 --2- 172.21.15.32:0/4109146139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8780ad510 0x7fb878065350 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.690+0000 7fb87759e700 1 -- 172.21.15.32:0/4109146139 >> 172.21.15.32:0/4109146139 conn(0x7fb8780604c0 msgr2=0x7fb878143080 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.690+0000 7fb87759e700 1 -- 172.21.15.32:0/4109146139 shutdown_connections 2020-12-05T17:41:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.690+0000 7fb87759e700 1 -- 172.21.15.32:0/4109146139 wait complete. 2020-12-05T17:41:51.726 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:41:51.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.838+0000 7f11b2055700 1 Processor -- start 2020-12-05T17:41:51.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.838+0000 7f11b2055700 1 -- start start 2020-12-05T17:41:51.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f11b2055700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f11ac0b14e0 0x7f11ac0ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:51.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f11b2055700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11ac142ab0 con 0x7f11ac0b5f00 2020-12-05T17:41:51.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f11b2055700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11ac0b62e0 con 0x7f11ac0af480 2020-12-05T17:41:51.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f11b2055700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11ac0af880 con 0x7f11ac0b14e0 2020-12-05T17:41:51.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f11b1854700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f11ac0af480 0x7f11ac142ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:56678/0 (socket says 172.21.15.32:56678) 2020-12-05T17:41:51.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f11b1854700 1 -- 172.21.15.32:0/2347550489 learned_addr learned my addr 172.21.15.32:0/2347550489 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:51.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f11b0852700 1 --2- 172.21.15.32:0/2347550489 >> v2:172.21.15.32:3300/0 conn(0x7f11ac0b14e0 0x7f11ac0ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:51.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f119ffff700 1 -- 172.21.15.32:0/2347550489 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3315400900 0 0) 0x7f11ac142ab0 con 0x7f11ac0b5f00 2020-12-05T17:41:51.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f119ffff700 1 -- 172.21.15.32:0/2347550489 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1190003230 con 0x7f11ac0b5f00 2020-12-05T17:41:51.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f119ffff700 1 -- 172.21.15.32:0/2347550489 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 944110888 0 0) 0x7f11ac0b62e0 con 0x7f11ac0af480 2020-12-05T17:41:51.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f119ffff700 1 -- 172.21.15.32:0/2347550489 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11ac142ab0 con 0x7f11ac0af480 2020-12-05T17:41:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f119ffff700 1 -- 172.21.15.32:0/2347550489 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3266817873 0 0) 0x7f1190003230 con 0x7f11ac0b5f00 2020-12-05T17:41:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f119ffff700 1 -- 172.21.15.32:0/2347550489 >> v2:172.21.15.32:3300/0 conn(0x7f11ac0b14e0 msgr2=0x7f11ac0ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f119ffff700 1 --2- 172.21.15.32:0/2347550489 >> v2:172.21.15.32:3300/0 conn(0x7f11ac0b14e0 0x7f11ac0ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f119ffff700 1 -- 172.21.15.32:0/2347550489 >> v1:172.21.15.61:6789/0 conn(0x7f11ac0af480 legacy=0x7f11ac142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:51.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f119ffff700 1 -- 172.21.15.32:0/2347550489 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ac0afbf0 con 0x7f11ac0b5f00 2020-12-05T17:41:51.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f119ffff700 1 -- 172.21.15.32:0/2347550489 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f11a00016d0 con 0x7f11ac0b5f00 2020-12-05T17:41:51.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f119ffff700 1 -- 172.21.15.32:0/2347550489 >> v1:172.21.15.32:6789/0 conn(0x7f11ac0b5f00 legacy=0x7f11ac066370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:51.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f119ffff700 1 --2- 172.21.15.32:0/2347550489 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11900038b0 0x7f1190003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:51.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f119ffff700 1 --2- 172.21.15.32:0/2347550489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11900044d0 0x7f1190004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:51.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f119ffff700 1 --2- 172.21.15.32:0/2347550489 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1190004e40 0x7f119000d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:51.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f119ffff700 1 -- 172.21.15.32:0/2347550489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac0af880 con 0x7f11900044d0 2020-12-05T17:41:51.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f119ffff700 1 -- 172.21.15.32:0/2347550489 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11900053a0 con 0x7f11900038b0 2020-12-05T17:41:51.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f119ffff700 1 -- 172.21.15.32:0/2347550489 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1190005770 con 0x7f1190004e40 2020-12-05T17:41:51.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f11b0852700 1 --2- 172.21.15.32:0/2347550489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11900044d0 0x7f1190004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:51.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f11b1854700 1 --2- 172.21.15.32:0/2347550489 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1190004e40 0x7f119000d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:51.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f11b0852700 1 -- 172.21.15.32:0/2347550489 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1190004e40 msgr2=0x7f119000d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:51.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f11b0852700 1 --2- 172.21.15.32:0/2347550489 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1190004e40 0x7f119000d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:51.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f11b0852700 1 -- 172.21.15.32:0/2347550489 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11900038b0 msgr2=0x7f1190003cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:51.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f11b0852700 1 --2- 172.21.15.32:0/2347550489 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11900038b0 0x7f1190003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-12-05T17:41:51.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f11b0852700 1 -- 172.21.15.32:0/2347550489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1190003230 con 0x7f11900044d0 2020-12-05T17:41:51.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.842+0000 7f11b1854700 1 -- 172.21.15.32:0/2347550489 reap_dead start 2020-12-05T17:41:51.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.846+0000 7f11b0852700 1 --2- 172.21.15.32:0/2347550489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11900044d0 0x7f1190004950 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7f11a800a000 tx=0x7f11a8008040).ready entity=mon.0 client_cookie=118a3c62cedc7143 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:51.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.846+0000 7f119ffff700 1 -- 172.21.15.32:0/2347550489 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11a800ba60 con 0x7f11900044d0 2020-12-05T17:41:51.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.846+0000 7f119ffff700 1 -- 172.21.15.32:0/2347550489 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a800bbc0 con 0x7f11900044d0 2020-12-05T17:41:51.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.846+0000 7f119ffff700 1 -- 172.21.15.32:0/2347550489 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11a8013480 con 0x7f11900044d0 2020-12-05T17:41:51.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.846+0000 7f11b2055700 1 -- 172.21.15.32:0/2347550489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11900044d0 msgr2=0x7f1190004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:51.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.846+0000 7f11b2055700 1 --2- 172.21.15.32:0/2347550489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11900044d0 0x7f1190004950 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7f11a800a000 tx=0x7f11a8008040).stop 2020-12-05T17:41:51.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.846+0000 7f11b2055700 1 -- 172.21.15.32:0/2347550489 shutdown_connections 2020-12-05T17:41:51.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.846+0000 7f11b2055700 1 --2- 172.21.15.32:0/2347550489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11900044d0 0x7f1190004950 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:51.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.846+0000 7f11b2055700 1 -- 172.21.15.32:0/2347550489 >> 172.21.15.32:0/2347550489 conn(0x7f11ac0604c0 msgr2=0x7f11ac05c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:51.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.846+0000 7f11b2055700 1 -- 172.21.15.32:0/2347550489 shutdown_connections 2020-12-05T17:41:51.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.846+0000 7f11b2055700 1 -- 172.21.15.32:0/2347550489 wait complete. 2020-12-05T17:41:51.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.846+0000 7f11b2055700 1 Processor -- start 2020-12-05T17:41:51.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.846+0000 7f11b2055700 1 -- start start 2020-12-05T17:41:51.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f11b2055700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11900044d0 0x7f11ac14df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:51.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f11b2055700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11ac065060 0x7f11ac10f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:51.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f11b2055700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11ac065590 0x7f11ac13bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:51.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f11b2055700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac063c80 con 0x7f11ac065060 2020-12-05T17:41:51.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f11b2055700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac063eb0 con 0x7f11ac065590 2020-12-05T17:41:51.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f11b2055700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac064150 con 0x7f11900044d0 2020-12-05T17:41:51.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f11b0852700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11ac065060 0x7f11ac10f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:51.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f11b0852700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11ac065060 0x7f11ac10f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41404/0 (socket says 172.21.15.32:41404) 2020-12-05T17:41:51.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f11b0852700 1 -- 172.21.15.32:0/2311381480 learned_addr learned my addr 172.21.15.32:0/2311381480 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:51.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f11b1854700 1 --2- 172.21.15.32:0/2311381480 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11ac065590 0x7f11ac13bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:51.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f11b1053700 1 --2- 172.21.15.32:0/2311381480 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11900044d0 0x7f11ac14df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:51.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f11b0852700 1 -- 172.21.15.32:0/2311381480 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11900044d0 msgr2=0x7f11ac14df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:51.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f11b0852700 1 --2- 172.21.15.32:0/2311381480 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11900044d0 0x7f11ac14df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:51.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f11b0852700 1 -- 172.21.15.32:0/2311381480 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11ac065590 msgr2=0x7f11ac13bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:51.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f11b0852700 1 --2- 172.21.15.32:0/2311381480 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11ac065590 0x7f11ac13bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:51.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f11b0852700 1 -- 172.21.15.32:0/2311381480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ac13c1b0 con 0x7f11ac065060 2020-12-05T17:41:51.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f11b1854700 1 --2- 172.21.15.32:0/2311381480 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11ac065590 0x7f11ac13bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:51.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f11b0852700 1 --2- 172.21.15.32:0/2311381480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11ac065060 0x7f11ac10f160 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7f11a800a000 tx=0x7f11a8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:51.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f119e7fc700 1 -- 172.21.15.32:0/2311381480 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11a8014e00 con 0x7f11ac065060 2020-12-05T17:41:51.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f119e7fc700 1 -- 172.21.15.32:0/2311381480 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a8006750 con 0x7f11ac065060 2020-12-05T17:41:51.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f11b2055700 1 -- 172.21.15.32:0/2311381480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11ac13c340 con 0x7f11ac065060 2020-12-05T17:41:51.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f119e7fc700 1 -- 172.21.15.32:0/2311381480 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11a801d600 con 0x7f11ac065060 2020-12-05T17:41:51.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f11b2055700 1 -- 172.21.15.32:0/2311381480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11ac0b3e40 con 0x7f11ac065060 2020-12-05T17:41:51.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f11b2055700 1 -- 172.21.15.32:0/2311381480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11ac064310 con 0x7f11ac065060 2020-12-05T17:41:51.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f119e7fc700 1 -- 172.21.15.32:0/2311381480 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f11a801d760 con 0x7f11ac065060 2020-12-05T17:41:51.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f119e7fc700 1 --2- 172.21.15.32:0/2311381480 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f11940553b0 0x7f1194057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:51.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f119e7fc700 1 -- 172.21.15.32:0/2311381480 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f11a806f970 con 0x7f11ac065060 2020-12-05T17:41:51.864 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.850+0000 7f11b1053700 1 --2- 172.21.15.32:0/2311381480 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f11940553b0 0x7f1194057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:51.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.854+0000 7f11b1053700 1 --2- 172.21.15.32:0/2311381480 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f11940553b0 0x7f1194057870 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f11a000c000 tx=0x7f11ac0ae940).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:51.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.858+0000 7f119e7fc700 1 -- 172.21.15.32:0/2311381480 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f11a8045e50 con 0x7f11ac065060 2020-12-05T17:41:52.000 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:51.998+0000 7f11b2055700 1 -- 172.21.15.32:0/2311381480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11ac057050 con 0x7f11ac065060 2020-12-05T17:41:52.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.002+0000 7f119e7fc700 1 -- 172.21.15.32:0/2311381480 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+13424 (secure 0 0 0) 0x7f11a801ce60 con 0x7f11ac065060 2020-12-05T17:41:52.002 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:52.003 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":143,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:41:51.333994+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:41:52.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.002+0000 7f11b2055700 1 -- 172.21.15.32:0/2311381480 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f11940553b0 msgr2=0x7f1194057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:52.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.002+0000 7f11b2055700 1 --2- 172.21.15.32:0/2311381480 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f11940553b0 0x7f1194057870 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f11a000c000 tx=0x7f11ac0ae940).stop 2020-12-05T17:41:52.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.002+0000 7f11b2055700 1 -- 172.21.15.32:0/2311381480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11ac065060 msgr2=0x7f11ac10f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:52.004 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.002+0000 7f11b2055700 1 --2- 172.21.15.32:0/2311381480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11ac065060 0x7f11ac10f160 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7f11a800a000 tx=0x7f11a8008040).stop 2020-12-05T17:41:52.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.002+0000 7f11b2055700 1 -- 172.21.15.32:0/2311381480 shutdown_connections 2020-12-05T17:41:52.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.002+0000 7f11b2055700 1 --2- 172.21.15.32:0/2311381480 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11ac065590 0x7f11ac13bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.002+0000 7f11b2055700 1 --2- 172.21.15.32:0/2311381480 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f11940553b0 0x7f1194057870 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.005 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.002+0000 7f11b2055700 1 --2- 172.21.15.32:0/2311381480 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11900044d0 0x7f11ac14df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.002+0000 7f11b2055700 1 --2- 172.21.15.32:0/2311381480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11ac065060 0x7f11ac10f160 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.002+0000 7f11b2055700 1 -- 172.21.15.32:0/2311381480 >> 172.21.15.32:0/2311381480 conn(0x7f11ac0604c0 msgr2=0x7f11ac0b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:52.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.002+0000 7f11b2055700 1 -- 172.21.15.32:0/2311381480 shutdown_connections 2020-12-05T17:41:52.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.002+0000 7f11b2055700 1 -- 172.21.15.32:0/2311381480 wait complete. 2020-12-05T17:41:52.016 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-12-05T17:41:52.017 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:41:52.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.130+0000 7f32759c4700 1 Processor -- start 2020-12-05T17:41:52.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.130+0000 7f32759c4700 1 -- start start 2020-12-05T17:41:52.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f32759c4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32700b2100 0x7f32700af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:52.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f32759c4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f327013ed60 0x7f3270138820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:52.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f32759c4700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f32700ad250 0x7f3270138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:52.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f32759c4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f327013fd10 con 0x7f327013ed60 2020-12-05T17:41:52.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f32759c4700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3270139200 con 0x7f32700ad250 2020-12-05T17:41:52.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f32759c4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32700b24e0 con 0x7f32700b2100 2020-12-05T17:41:52.139 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f3267fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f327013ed60 0x7f3270138820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:52.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f3267fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f327013ed60 0x7f3270138820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41412/0 (socket says 172.21.15.32:41412) 2020-12-05T17:41:52.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f3267fff700 1 -- 172.21.15.32:0/364703373 learned_addr learned my addr 172.21.15.32:0/364703373 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:52.140 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f32751c3700 1 --2- 172.21.15.32:0/364703373 >> v2:172.21.15.61:3300/0 conn(0x7f32700ad250 0x7f3270138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:52.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f32749c2700 1 --2- 172.21.15.32:0/364703373 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32700b2100 0x7f32700af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:52.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f3267fff700 1 -- 172.21.15.32:0/364703373 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32700b2100 msgr2=0x7f32700af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:52.141 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f3267fff700 1 --2- 172.21.15.32:0/364703373 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32700b2100 0x7f32700af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f3267fff700 1 -- 172.21.15.32:0/364703373 >> v2:172.21.15.61:3300/0 conn(0x7f32700ad250 msgr2=0x7f3270138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:52.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f3267fff700 1 --2- 172.21.15.32:0/364703373 >> v2:172.21.15.61:3300/0 conn(0x7f32700ad250 0x7f3270138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.142 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f3267fff700 1 -- 172.21.15.32:0/364703373 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32700b0780 con 0x7f327013ed60 2020-12-05T17:41:52.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f3267fff700 1 --2- 172.21.15.32:0/364703373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f327013ed60 0x7f3270138820 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7f3260002f30 tx=0x7f3260006e80).ready entity=mon.0 client_cookie=8642fdb41732c74b server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:52.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f32677fe700 1 -- 172.21.15.32:0/364703373 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3260014070 con 0x7f327013ed60 2020-12-05T17:41:52.143 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f32677fe700 1 -- 172.21.15.32:0/364703373 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3260009830 con 0x7f327013ed60 2020-12-05T17:41:52.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f32677fe700 1 -- 172.21.15.32:0/364703373 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3260009b90 con 0x7f327013ed60 2020-12-05T17:41:52.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f32759c4700 1 -- 172.21.15.32:0/364703373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f327013ed60 msgr2=0x7f3270138820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:52.144 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f32759c4700 1 --2- 172.21.15.32:0/364703373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f327013ed60 0x7f3270138820 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7f3260002f30 tx=0x7f3260006e80).stop 2020-12-05T17:41:52.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f32759c4700 1 -- 172.21.15.32:0/364703373 shutdown_connections 2020-12-05T17:41:52.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f32759c4700 1 --2- 172.21.15.32:0/364703373 >> v2:172.21.15.61:3300/0 conn(0x7f32700ad250 0x7f3270138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f32759c4700 1 --2- 172.21.15.32:0/364703373 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32700b2100 0x7f32700af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f32759c4700 1 --2- 172.21.15.32:0/364703373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f327013ed60 0x7f3270138820 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f32759c4700 1 -- 172.21.15.32:0/364703373 >> 172.21.15.32:0/364703373 conn(0x7f32700620b0 msgr2=0x7f3270062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:52.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f32759c4700 1 -- 172.21.15.32:0/364703373 shutdown_connections 2020-12-05T17:41:52.147 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.134+0000 7f32759c4700 1 -- 172.21.15.32:0/364703373 wait complete. 2020-12-05T17:41:52.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.138+0000 7f32759c4700 1 Processor -- start 2020-12-05T17:41:52.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.138+0000 7f32759c4700 1 -- start start 2020-12-05T17:41:52.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f32759c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32700ad250 0x7f327013cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:52.150 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f32759c4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32700b2100 0x7f3270154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:52.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f32759c4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f327013bf30 0x7f32700642e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:52.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f32759c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3270139e60 con 0x7f32700ad250 2020-12-05T17:41:52.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f32759c4700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327013a100 con 0x7f32700b2100 2020-12-05T17:41:52.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f32759c4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327013a3a0 con 0x7f327013bf30 2020-12-05T17:41:52.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f32749c2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32700ad250 0x7f327013cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:52.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f32749c2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32700ad250 0x7f327013cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41416/0 (socket says 172.21.15.32:41416) 2020-12-05T17:41:52.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f32749c2700 1 -- 172.21.15.32:0/225973606 learned_addr learned my addr 172.21.15.32:0/225973606 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:52.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f32751c3700 1 --2- 172.21.15.32:0/225973606 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f327013bf30 0x7f32700642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:52.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f3267fff700 1 --2- 172.21.15.32:0/225973606 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32700b2100 0x7f3270154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:52.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f32749c2700 1 -- 172.21.15.32:0/225973606 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f327013bf30 msgr2=0x7f32700642e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:52.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f32749c2700 1 --2- 172.21.15.32:0/225973606 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f327013bf30 0x7f32700642e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f32749c2700 1 -- 172.21.15.32:0/225973606 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32700b2100 msgr2=0x7f3270154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:52.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f32749c2700 1 --2- 172.21.15.32:0/225973606 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32700b2100 0x7f3270154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f32749c2700 1 -- 172.21.15.32:0/225973606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3270063df0 con 0x7f32700ad250 2020-12-05T17:41:52.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f3267fff700 1 --2- 172.21.15.32:0/225973606 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32700b2100 0x7f3270154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:52.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f32749c2700 1 --2- 172.21.15.32:0/225973606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32700ad250 0x7f327013cca0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0x7f326c00c000 tx=0x7f326c006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:52.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f3265ffb700 1 -- 172.21.15.32:0/225973606 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f326c00ca10 con 0x7f32700ad250 2020-12-05T17:41:52.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f3265ffb700 1 -- 172.21.15.32:0/225973606 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326c00cb70 con 0x7f32700ad250 2020-12-05T17:41:52.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f3265ffb700 1 -- 172.21.15.32:0/225973606 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f326c0123f0 con 0x7f32700ad250 2020-12-05T17:41:52.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f32759c4700 1 -- 172.21.15.32:0/225973606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327013ade0 con 0x7f32700ad250 2020-12-05T17:41:52.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f32759c4700 1 -- 172.21.15.32:0/225973606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32700647d0 con 0x7f32700ad250 2020-12-05T17:41:52.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f32759c4700 1 -- 172.21.15.32:0/225973606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3270057050 con 0x7f32700ad250 2020-12-05T17:41:52.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f3265ffb700 1 -- 172.21.15.32:0/225973606 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f326c012550 con 0x7f32700ad250 2020-12-05T17:41:52.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.142+0000 7f3265ffb700 1 --2- 172.21.15.32:0/225973606 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f32500553b0 0x7f3250057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:52.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.146+0000 7f3265ffb700 1 -- 172.21.15.32:0/225973606 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f326c06e120 con 0x7f32700ad250 2020-12-05T17:41:52.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.146+0000 7f3267fff700 1 --2- 172.21.15.32:0/225973606 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f32500553b0 0x7f3250057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:52.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.146+0000 7f3267fff700 1 --2- 172.21.15.32:0/225973606 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f32500553b0 0x7f3250057870 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f3260019000 tx=0x7f3260016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:52.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.150+0000 7f3265ffb700 1 -- 172.21.15.32:0/225973606 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f326c0445a0 con 0x7f32700ad250 2020-12-05T17:41:52.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.298+0000 7f32759c4700 1 -- 172.21.15.32:0/225973606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f3270060c10 con 0x7f32700ad250 2020-12-05T17:41:52.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.298+0000 7f3265ffb700 1 -- 172.21.15.32:0/225973606 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v143) v1 ==== 110+0+10 (secure 0 0 0) 0x7f326c047bc0 con 0x7f32700ad250 2020-12-05T17:41:52.299 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:41:52.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.302+0000 7f32759c4700 1 -- 172.21.15.32:0/225973606 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f32500553b0 msgr2=0x7f3250057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:52.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.302+0000 7f32759c4700 1 --2- 172.21.15.32:0/225973606 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f32500553b0 0x7f3250057870 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f3260019000 tx=0x7f3260016000).stop 2020-12-05T17:41:52.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.302+0000 7f32759c4700 1 -- 172.21.15.32:0/225973606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32700ad250 msgr2=0x7f327013cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:52.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.302+0000 7f32759c4700 1 --2- 172.21.15.32:0/225973606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32700ad250 0x7f327013cca0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0x7f326c00c000 tx=0x7f326c006dc0).stop 2020-12-05T17:41:52.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.302+0000 7f32759c4700 1 -- 172.21.15.32:0/225973606 shutdown_connections 2020-12-05T17:41:52.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.302+0000 7f32759c4700 1 --2- 172.21.15.32:0/225973606 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f32500553b0 0x7f3250057870 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.302+0000 7f32759c4700 1 --2- 172.21.15.32:0/225973606 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f327013bf30 0x7f32700642e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.302+0000 7f32759c4700 1 --2- 172.21.15.32:0/225973606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32700ad250 0x7f327013cca0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.302+0000 7f32759c4700 1 --2- 172.21.15.32:0/225973606 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32700b2100 0x7f3270154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.302+0000 7f32759c4700 1 -- 172.21.15.32:0/225973606 >> 172.21.15.32:0/225973606 conn(0x7f32700620b0 msgr2=0x7f3270138a30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:52.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.302+0000 7f32759c4700 1 -- 172.21.15.32:0/225973606 shutdown_connections 2020-12-05T17:41:52.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.302+0000 7f32759c4700 1 -- 172.21.15.32:0/225973606 wait complete. 2020-12-05T17:41:52.322 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:41:52.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d76bda700 1 Processor -- start 2020-12-05T17:41:52.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d76bda700 1 -- start start 2020-12-05T17:41:52.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d76bda700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d7005f3b0 0x7f4d7005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:52.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d76bda700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4d700b2fb0 0x7f4d70061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:52.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d76bda700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d700b2df0 con 0x7f4d7005feb0 2020-12-05T17:41:52.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d76bda700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d700b3c90 con 0x7f4d700b2fb0 2020-12-05T17:41:52.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d76bda700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d700b2b50 con 0x7f4d7005f3b0 2020-12-05T17:41:52.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d763d9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4d700b2fb0 0x7f4d70061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:52.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d763d9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4d700b2fb0 0x7f4d70061780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41426/0 (socket says 172.21.15.32:41426) 2020-12-05T17:41:52.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d763d9700 1 -- 172.21.15.32:0/4014840760 learned_addr learned my addr 172.21.15.32:0/4014840760 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:52.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d75bd8700 1 --2- 172.21.15.32:0/4014840760 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d7005f3b0 0x7f4d7005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:52.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d763d9700 1 -- 172.21.15.32:0/4014840760 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d7005f3b0 msgr2=0x7f4d7005f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:52.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d763d9700 1 --2- 172.21.15.32:0/4014840760 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d7005f3b0 0x7f4d7005f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d763d9700 1 -- 172.21.15.32:0/4014840760 >> v1:172.21.15.61:6789/0 conn(0x7f4d7005feb0 legacy=0x7f4d7013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:52.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d763d9700 1 -- 172.21.15.32:0/4014840760 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d70142820 con 0x7f4d700b2fb0 2020-12-05T17:41:52.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d763d9700 1 --2- 172.21.15.32:0/4014840760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d700b2fb0 0x7f4d70061780 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7f4d680032b0 tx=0x7f4d68009270).ready entity=mon.0 client_cookie=a15db30717b507bc server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:52.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d74bd6700 1 -- 172.21.15.32:0/4014840760 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d6800bc00 con 0x7f4d700b2fb0 2020-12-05T17:41:52.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d74bd6700 1 -- 172.21.15.32:0/4014840760 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6800bd60 con 0x7f4d700b2fb0 2020-12-05T17:41:52.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d74bd6700 1 -- 172.21.15.32:0/4014840760 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d68011400 con 0x7f4d700b2fb0 2020-12-05T17:41:52.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d76bda700 1 -- 172.21.15.32:0/4014840760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d700b2fb0 msgr2=0x7f4d70061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:52.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d76bda700 1 --2- 172.21.15.32:0/4014840760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d700b2fb0 0x7f4d70061780 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7f4d680032b0 tx=0x7f4d68009270).stop 2020-12-05T17:41:52.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d76bda700 1 -- 172.21.15.32:0/4014840760 shutdown_connections 2020-12-05T17:41:52.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d76bda700 1 --2- 172.21.15.32:0/4014840760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d700b2fb0 0x7f4d70061780 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d76bda700 1 --2- 172.21.15.32:0/4014840760 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d7005f3b0 0x7f4d7005f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d76bda700 1 -- 172.21.15.32:0/4014840760 >> 172.21.15.32:0/4014840760 conn(0x7f4d700b8f20 msgr2=0x7f4d700b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:52.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d76bda700 1 -- 172.21.15.32:0/4014840760 shutdown_connections 2020-12-05T17:41:52.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.442+0000 7f4d76bda700 1 -- 172.21.15.32:0/4014840760 wait complete. 2020-12-05T17:41:52.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.446+0000 7f4d76bda700 1 Processor -- start 2020-12-05T17:41:52.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.446+0000 7f4d76bda700 1 -- start start 2020-12-05T17:41:52.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.446+0000 7f4d76bda700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d7005f3b0 0x7f4d7013e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:52.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.446+0000 7f4d76bda700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d7005feb0 0x7f4d70151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:52.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.446+0000 7f4d76bda700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d700b2fb0 0x7f4d70151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:52.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.446+0000 7f4d76bda700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d70142630 con 0x7f4d700b2fb0 2020-12-05T17:41:52.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.446+0000 7f4d76bda700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d701423d0 con 0x7f4d7005f3b0 2020-12-05T17:41:52.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.446+0000 7f4d76bda700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d70146130 con 0x7f4d7005feb0 2020-12-05T17:41:52.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.446+0000 7f4d763d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d700b2fb0 0x7f4d70151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:52.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.446+0000 7f4d763d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d700b2fb0 0x7f4d70151d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41434/0 (socket says 172.21.15.32:41434) 2020-12-05T17:41:52.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.446+0000 7f4d763d9700 1 -- 172.21.15.32:0/2097467678 learned_addr learned my addr 172.21.15.32:0/2097467678 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:52.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.446+0000 7f4d75bd8700 1 --2- 172.21.15.32:0/2097467678 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d7005f3b0 0x7f4d7013e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:52.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.446+0000 7f4d753d7700 1 --2- 172.21.15.32:0/2097467678 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d7005feb0 0x7f4d70151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:52.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.450+0000 7f4d763d9700 1 -- 172.21.15.32:0/2097467678 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d7005feb0 msgr2=0x7f4d70151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:52.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.450+0000 7f4d763d9700 1 --2- 172.21.15.32:0/2097467678 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d7005feb0 0x7f4d70151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.450+0000 7f4d763d9700 1 -- 172.21.15.32:0/2097467678 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d7005f3b0 msgr2=0x7f4d7013e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:52.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.450+0000 7f4d763d9700 1 --2- 172.21.15.32:0/2097467678 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d7005f3b0 0x7f4d7013e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.450+0000 7f4d763d9700 1 -- 172.21.15.32:0/2097467678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d701513c0 con 0x7f4d700b2fb0 2020-12-05T17:41:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.450+0000 7f4d763d9700 1 --2- 172.21.15.32:0/2097467678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d700b2fb0 0x7f4d70151d10 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 rx=0x7f4d680032b0 tx=0x7f4d68016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.450+0000 7f4d5effd700 1 -- 172.21.15.32:0/2097467678 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d68011e90 con 0x7f4d700b2fb0 2020-12-05T17:41:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.450+0000 7f4d5effd700 1 -- 172.21.15.32:0/2097467678 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6801f3f0 con 0x7f4d700b2fb0 2020-12-05T17:41:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.450+0000 7f4d5effd700 1 -- 172.21.15.32:0/2097467678 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d6801f750 con 0x7f4d700b2fb0 2020-12-05T17:41:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.450+0000 7f4d76bda700 1 -- 172.21.15.32:0/2097467678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d70152200 con 0x7f4d700b2fb0 2020-12-05T17:41:52.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.450+0000 7f4d76bda700 1 -- 172.21.15.32:0/2097467678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d70141210 con 0x7f4d700b2fb0 2020-12-05T17:41:52.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.450+0000 7f4d76bda700 1 -- 172.21.15.32:0/2097467678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d70057050 con 0x7f4d700b2fb0 2020-12-05T17:41:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.450+0000 7f4d5effd700 1 -- 172.21.15.32:0/2097467678 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4d68026030 con 0x7f4d700b2fb0 2020-12-05T17:41:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.454+0000 7f4d5effd700 1 --2- 172.21.15.32:0/2097467678 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4d50055440 0x7f4d50057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.454+0000 7f4d5effd700 1 -- 172.21.15.32:0/2097467678 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f4d6806e940 con 0x7f4d700b2fb0 2020-12-05T17:41:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.454+0000 7f4d75bd8700 1 --2- 172.21.15.32:0/2097467678 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4d50055440 0x7f4d50057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:52.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.454+0000 7f4d75bd8700 1 --2- 172.21.15.32:0/2097467678 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4d50055440 0x7f4d50057900 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f4d6c00c000 tx=0x7f4d6c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:52.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.458+0000 7f4d5effd700 1 -- 172.21.15.32:0/2097467678 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4d68044c90 con 0x7f4d700b2fb0 2020-12-05T17:41:52.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.602+0000 7f4d76bda700 1 -- 172.21.15.32:0/2097467678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4d7005aab0 con 0x7f4d700b2fb0 2020-12-05T17:41:52.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.602+0000 7f4d5effd700 1 -- 172.21.15.32:0/2097467678 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v143) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4d680240a0 con 0x7f4d700b2fb0 2020-12-05T17:41:52.603 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:41:52.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.602+0000 7f4d76bda700 1 -- 172.21.15.32:0/2097467678 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4d50055440 msgr2=0x7f4d50057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:52.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.602+0000 7f4d76bda700 1 --2- 172.21.15.32:0/2097467678 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4d50055440 0x7f4d50057900 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f4d6c00c000 tx=0x7f4d6c008040).stop 2020-12-05T17:41:52.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.602+0000 7f4d76bda700 1 -- 172.21.15.32:0/2097467678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d700b2fb0 msgr2=0x7f4d70151d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:52.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.602+0000 7f4d76bda700 1 --2- 172.21.15.32:0/2097467678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d700b2fb0 0x7f4d70151d10 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 rx=0x7f4d680032b0 tx=0x7f4d68016000).stop 2020-12-05T17:41:52.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.602+0000 7f4d76bda700 1 -- 172.21.15.32:0/2097467678 shutdown_connections 2020-12-05T17:41:52.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.602+0000 7f4d76bda700 1 --2- 172.21.15.32:0/2097467678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d700b2fb0 0x7f4d70151d10 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.602+0000 7f4d76bda700 1 --2- 172.21.15.32:0/2097467678 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d7005f3b0 0x7f4d7013e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.602+0000 7f4d76bda700 1 --2- 172.21.15.32:0/2097467678 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4d50055440 0x7f4d50057900 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.606 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.602+0000 7f4d76bda700 1 --2- 172.21.15.32:0/2097467678 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d7005feb0 0x7f4d70151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.602+0000 7f4d76bda700 1 -- 172.21.15.32:0/2097467678 >> 172.21.15.32:0/2097467678 conn(0x7f4d700b8f20 msgr2=0x7f4d700b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:52.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.602+0000 7f4d76bda700 1 -- 172.21.15.32:0/2097467678 shutdown_connections 2020-12-05T17:41:52.607 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.602+0000 7f4d76bda700 1 -- 172.21.15.32:0/2097467678 wait complete. 2020-12-05T17:41:52.617 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-12-05T17:41:52.730 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.730+0000 7fb34d40f700 1 Processor -- start 2020-12-05T17:41:52.731 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.730+0000 7fb34d40f700 1 -- start start 2020-12-05T17:41:52.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.730+0000 7fb34d40f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3480b5710 0x7fb3480afe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:52.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.730+0000 7fb34d40f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb348069cb0 0x7fb3480b0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:52.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.730+0000 7fb34d40f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3480aa490 con 0x7fb348147070 2020-12-05T17:41:52.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.730+0000 7fb34d40f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3480b5af0 con 0x7fb3480b5710 2020-12-05T17:41:52.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.730+0000 7fb34d40f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3480b0830 con 0x7fb348069cb0 2020-12-05T17:41:52.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.730+0000 7fb3477fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb348147070 0x7fb348147500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44088/0 (socket says 172.21.15.32:44088) 2020-12-05T17:41:52.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.730+0000 7fb3477fe700 1 -- 172.21.15.32:0/381356505 learned_addr learned my addr 172.21.15.32:0/381356505 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:52.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.730+0000 7fb346ffd700 1 --2- 172.21.15.32:0/381356505 >> v2:172.21.15.61:3300/0 conn(0x7fb3480b5710 0x7fb3480afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:52.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.730+0000 7fb3467fc700 1 --2- 172.21.15.32:0/381356505 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb348069cb0 0x7fb3480b0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:52.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.730+0000 7fb345ffb700 1 -- 172.21.15.32:0/381356505 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1062363548 0 0) 0x7fb3480aa490 con 0x7fb348147070 2020-12-05T17:41:52.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.730+0000 7fb345ffb700 1 -- 172.21.15.32:0/381356505 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb328003230 con 0x7fb348147070 2020-12-05T17:41:52.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.730+0000 7fb346ffd700 1 -- 172.21.15.32:0/381356505 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb348069cb0 msgr2=0x7fb3480b0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:52.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.730+0000 7fb346ffd700 1 --2- 172.21.15.32:0/381356505 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb348069cb0 0x7fb3480b0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.730+0000 7fb346ffd700 1 -- 172.21.15.32:0/381356505 >> v1:172.21.15.32:6789/0 conn(0x7fb348147070 legacy=0x7fb348147500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:52.740 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.730+0000 7fb346ffd700 1 -- 172.21.15.32:0/381356505 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3480a83d0 con 0x7fb3480b5710 2020-12-05T17:41:52.740 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.730+0000 7fb346ffd700 1 --2- 172.21.15.32:0/381356505 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3480b5710 0x7fb3480afe50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fb338002f30 tx=0x7fb338006dc0).ready entity=mon.1 client_cookie=4eaf331565ca1c06 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:52.740 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.730+0000 7fb345ffb700 1 -- 172.21.15.32:0/381356505 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3380096a0 con 0x7fb3480b5710 2020-12-05T17:41:52.741 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.730+0000 7fb345ffb700 1 -- 172.21.15.32:0/381356505 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb338009800 con 0x7fb3480b5710 2020-12-05T17:41:52.741 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.730+0000 7fb34d40f700 1 -- 172.21.15.32:0/381356505 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3480b5710 msgr2=0x7fb3480afe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:52.741 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.730+0000 7fb345ffb700 1 -- 172.21.15.32:0/381356505 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb338009b60 con 0x7fb3480b5710 2020-12-05T17:41:52.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.730+0000 7fb34d40f700 1 --2- 172.21.15.32:0/381356505 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3480b5710 0x7fb3480afe50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fb338002f30 tx=0x7fb338006dc0).stop 2020-12-05T17:41:52.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb34d40f700 1 -- 172.21.15.32:0/381356505 shutdown_connections 2020-12-05T17:41:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb34d40f700 1 --2- 172.21.15.32:0/381356505 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3480b5710 0x7fb3480afe50 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb34d40f700 1 --2- 172.21.15.32:0/381356505 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb348069cb0 0x7fb3480b0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb34d40f700 1 -- 172.21.15.32:0/381356505 >> 172.21.15.32:0/381356505 conn(0x7fb34805fe40 msgr2=0x7fb3480b4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:52.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb34d40f700 1 -- 172.21.15.32:0/381356505 shutdown_connections 2020-12-05T17:41:52.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb34d40f700 1 -- 172.21.15.32:0/381356505 wait complete. 2020-12-05T17:41:52.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb34d40f700 1 Processor -- start 2020-12-05T17:41:52.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb34d40f700 1 -- start start 2020-12-05T17:41:52.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb34d40f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb348069cb0 0x7fb348063020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:52.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb34d40f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3480b5710 0x7fb348146510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:52.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb34d40f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb348147070 0x7fb348146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:52.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb34d40f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3480668a0 con 0x7fb348147070 2020-12-05T17:41:52.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb34d40f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3481493a0 con 0x7fb3480b5710 2020-12-05T17:41:52.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb34d40f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb348064a20 con 0x7fb348069cb0 2020-12-05T17:41:52.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb3477fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb348147070 0x7fb348146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:52.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb3477fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb348147070 0x7fb348146a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41448/0 (socket says 172.21.15.32:41448) 2020-12-05T17:41:52.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb3477fe700 1 -- 172.21.15.32:0/2092022555 learned_addr learned my addr 172.21.15.32:0/2092022555 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:52.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb3467fc700 1 --2- 172.21.15.32:0/2092022555 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3480b5710 0x7fb348146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:52.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb346ffd700 1 --2- 172.21.15.32:0/2092022555 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb348069cb0 0x7fb348063020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:52.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb3477fe700 1 -- 172.21.15.32:0/2092022555 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb348069cb0 msgr2=0x7fb348063020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:52.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb3477fe700 1 --2- 172.21.15.32:0/2092022555 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb348069cb0 0x7fb348063020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb3477fe700 1 -- 172.21.15.32:0/2092022555 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3480b5710 msgr2=0x7fb348146510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:52.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb3477fe700 1 --2- 172.21.15.32:0/2092022555 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3480b5710 0x7fb348146510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.734+0000 7fb3477fe700 1 -- 172.21.15.32:0/2092022555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb348149880 con 0x7fb348147070 2020-12-05T17:41:52.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.738+0000 7fb3477fe700 1 --2- 172.21.15.32:0/2092022555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb348147070 0x7fb348146a00 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7fb3400010d0 tx=0x7fb340007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:52.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.738+0000 7fb33ffff700 1 -- 172.21.15.32:0/2092022555 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3400072e0 con 0x7fb348147070 2020-12-05T17:41:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.738+0000 7fb33ffff700 1 -- 172.21.15.32:0/2092022555 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb340007440 con 0x7fb348147070 2020-12-05T17:41:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.738+0000 7fb33ffff700 1 -- 172.21.15.32:0/2092022555 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3400057d0 con 0x7fb348147070 2020-12-05T17:41:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.738+0000 7fb34d40f700 1 -- 172.21.15.32:0/2092022555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb348066ca0 con 0x7fb348147070 2020-12-05T17:41:52.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.738+0000 7fb34d40f700 1 -- 172.21.15.32:0/2092022555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3480636b0 con 0x7fb348147070 2020-12-05T17:41:52.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.738+0000 7fb34d40f700 1 -- 172.21.15.32:0/2092022555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb348064be0 con 0x7fb348147070 2020-12-05T17:41:52.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.738+0000 7fb33ffff700 1 -- 172.21.15.32:0/2092022555 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb340010070 con 0x7fb348147070 2020-12-05T17:41:52.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.738+0000 7fb33ffff700 1 --2- 172.21.15.32:0/2092022555 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb32c0554e0 0x7fb32c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:52.759 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.742+0000 7fb346ffd700 1 --2- 172.21.15.32:0/2092022555 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb32c0554e0 0x7fb32c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:52.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.742+0000 7fb33ffff700 1 -- 172.21.15.32:0/2092022555 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fb34006d490 con 0x7fb348147070 2020-12-05T17:41:52.760 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.742+0000 7fb346ffd700 1 --2- 172.21.15.32:0/2092022555 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb32c0554e0 0x7fb32c0579a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7fb33801b000 tx=0x7fb338015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:52.761 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.746+0000 7fb33ffff700 1 -- 172.21.15.32:0/2092022555 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb340043860 con 0x7fb348147070 2020-12-05T17:41:52.890 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.890+0000 7fb34d40f700 1 -- 172.21.15.32:0/2092022555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb348057050 con 0x7fb348147070 2020-12-05T17:41:52.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.890+0000 7fb33ffff700 1 -- 172.21.15.32:0/2092022555 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v143) v1 ==== 84+0+143 (secure 0 0 0) 0x7fb340046e80 con 0x7fb348147070 2020-12-05T17:41:52.891 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:52.892 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2020-12-05T17:41:52.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:52.892 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2020-12-05T17:41:52.892 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2020-12-05T17:41:52.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:41:52.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:41:52.893 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2020-12-05T17:41:52.893 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2020-12-05T17:41:52.893 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:41:52.894 INFO:teuthology.orchestra.run.smithi032.stdout:] 2020-12-05T17:41:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.890+0000 7fb34d40f700 1 -- 172.21.15.32:0/2092022555 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb32c0554e0 msgr2=0x7fb32c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.890+0000 7fb34d40f700 1 --2- 172.21.15.32:0/2092022555 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb32c0554e0 0x7fb32c0579a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7fb33801b000 tx=0x7fb338015000).stop 2020-12-05T17:41:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.890+0000 7fb34d40f700 1 -- 172.21.15.32:0/2092022555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb348147070 msgr2=0x7fb348146a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.890+0000 7fb34d40f700 1 --2- 172.21.15.32:0/2092022555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb348147070 0x7fb348146a00 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7fb3400010d0 tx=0x7fb340007ac0).stop 2020-12-05T17:41:52.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.890+0000 7fb34d40f700 1 -- 172.21.15.32:0/2092022555 shutdown_connections 2020-12-05T17:41:52.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.890+0000 7fb34d40f700 1 --2- 172.21.15.32:0/2092022555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb348147070 0x7fb348146a00 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.890+0000 7fb34d40f700 1 --2- 172.21.15.32:0/2092022555 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb32c0554e0 0x7fb32c0579a0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.890+0000 7fb34d40f700 1 --2- 172.21.15.32:0/2092022555 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb348069cb0 0x7fb348063020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.896 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.890+0000 7fb34d40f700 1 --2- 172.21.15.32:0/2092022555 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3480b5710 0x7fb348146510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:52.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.890+0000 7fb34d40f700 1 -- 172.21.15.32:0/2092022555 >> 172.21.15.32:0/2092022555 conn(0x7fb34805fe40 msgr2=0x7fb3480ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:52.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.890+0000 7fb34d40f700 1 -- 172.21.15.32:0/2092022555 shutdown_connections 2020-12-05T17:41:52.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:52.890+0000 7fb34d40f700 1 -- 172.21.15.32:0/2092022555 wait complete. 2020-12-05T17:41:52.904 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-12-05T17:41:52.905 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-12-05T17:41:53.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.022+0000 7fb8db1e5700 1 Processor -- start 2020-12-05T17:41:53.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.022+0000 7fb8db1e5700 1 -- start start 2020-12-05T17:41:53.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.022+0000 7fb8db1e5700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb8d405f6f0 0x7fb8d4141700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:53.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.022+0000 7fb8db1e5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8d4061b30 0x7fb8d4141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:53.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.022+0000 7fb8db1e5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8d41420e0 0x7fb8d40b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:53.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.022+0000 7fb8db1e5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d41439c0 con 0x7fb8d41420e0 2020-12-05T17:41:53.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.022+0000 7fb8db1e5700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d4142550 con 0x7fb8d405f6f0 2020-12-05T17:41:53.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.022+0000 7fb8db1e5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d405fe20 con 0x7fb8d4061b30 2020-12-05T17:41:53.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.022+0000 7fb8da9e4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8d41420e0 0x7fb8d40b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:53.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.022+0000 7fb8da9e4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8d41420e0 0x7fb8d40b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41456/0 (socket says 172.21.15.32:41456) 2020-12-05T17:41:53.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.022+0000 7fb8da9e4700 1 -- 172.21.15.32:0/2689534956 learned_addr learned my addr 172.21.15.32:0/2689534956 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:53.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.022+0000 7fb8da1e3700 1 --2- 172.21.15.32:0/2689534956 >> v2:172.21.15.61:3300/0 conn(0x7fb8d405f6f0 0x7fb8d4141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:53.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.022+0000 7fb8d99e2700 1 --2- 172.21.15.32:0/2689534956 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8d4061b30 0x7fb8d4141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:53.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.022+0000 7fb8da9e4700 1 -- 172.21.15.32:0/2689534956 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8d4061b30 msgr2=0x7fb8d4141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:53.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.022+0000 7fb8da9e4700 1 --2- 172.21.15.32:0/2689534956 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8d4061b30 0x7fb8d4141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:53.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.022+0000 7fb8da9e4700 1 -- 172.21.15.32:0/2689534956 >> v2:172.21.15.61:3300/0 conn(0x7fb8d405f6f0 msgr2=0x7fb8d4141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.022+0000 7fb8da9e4700 1 --2- 172.21.15.32:0/2689534956 >> v2:172.21.15.61:3300/0 conn(0x7fb8d405f6f0 0x7fb8d4141700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.022+0000 7fb8da9e4700 1 -- 172.21.15.32:0/2689534956 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d4146910 con 0x7fb8d41420e0 2020-12-05T17:41:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.022+0000 7fb8da1e3700 1 --2- 172.21.15.32:0/2689534956 >> v2:172.21.15.61:3300/0 conn(0x7fb8d405f6f0 0x7fb8d4141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.026+0000 7fb8da9e4700 1 --2- 172.21.15.32:0/2689534956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8d41420e0 0x7fb8d40b2660 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7fb8cc002f30 tx=0x7fb8cc006e80).ready entity=mon.0 client_cookie=88a522f400a14e3d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.026+0000 7fb8d91e1700 1 -- 172.21.15.32:0/2689534956 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8cc0097f0 con 0x7fb8d41420e0 2020-12-05T17:41:53.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.026+0000 7fb8d91e1700 1 -- 172.21.15.32:0/2689534956 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8cc009950 con 0x7fb8d41420e0 2020-12-05T17:41:53.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.026+0000 7fb8d91e1700 1 -- 172.21.15.32:0/2689534956 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8cc009cb0 con 0x7fb8d41420e0 2020-12-05T17:41:53.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.026+0000 7fb8db1e5700 1 -- 172.21.15.32:0/2689534956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8d41420e0 msgr2=0x7fb8d40b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:53.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.026+0000 7fb8db1e5700 1 --2- 172.21.15.32:0/2689534956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8d41420e0 0x7fb8d40b2660 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7fb8cc002f30 tx=0x7fb8cc006e80).stop 2020-12-05T17:41:53.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.026+0000 7fb8db1e5700 1 -- 172.21.15.32:0/2689534956 shutdown_connections 2020-12-05T17:41:53.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.026+0000 7fb8db1e5700 1 --2- 172.21.15.32:0/2689534956 >> v2:172.21.15.61:3300/0 conn(0x7fb8d405f6f0 0x7fb8d4141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:53.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.026+0000 7fb8db1e5700 1 --2- 172.21.15.32:0/2689534956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8d41420e0 0x7fb8d40b2660 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:53.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.026+0000 7fb8db1e5700 1 --2- 172.21.15.32:0/2689534956 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8d4061b30 0x7fb8d4141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:53.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.026+0000 7fb8db1e5700 1 -- 172.21.15.32:0/2689534956 >> 172.21.15.32:0/2689534956 conn(0x7fb8d4061010 msgr2=0x7fb8d4061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:53.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.026+0000 7fb8db1e5700 1 -- 172.21.15.32:0/2689534956 shutdown_connections 2020-12-05T17:41:53.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.026+0000 7fb8db1e5700 1 -- 172.21.15.32:0/2689534956 wait complete. 2020-12-05T17:41:53.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.026+0000 7fb8db1e5700 1 Processor -- start 2020-12-05T17:41:53.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.026+0000 7fb8db1e5700 1 -- start start 2020-12-05T17:41:53.034 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.030+0000 7fb8db1e5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8d405f6f0 0x7fb8d41514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:53.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.030+0000 7fb8db1e5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb8d4061b30 0x7fb8d40b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:53.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.030+0000 7fb8db1e5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8d41420e0 0x7fb8d413f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:53.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.030+0000 7fb8db1e5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d40b5530 con 0x7fb8d41420e0 2020-12-05T17:41:53.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.030+0000 7fb8db1e5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d4064420 con 0x7fb8d4061b30 2020-12-05T17:41:53.035 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.030+0000 7fb8db1e5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d4064180 con 0x7fb8d405f6f0 2020-12-05T17:41:53.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.030+0000 7fb8da9e4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8d41420e0 0x7fb8d413f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:53.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.030+0000 7fb8da1e3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8d405f6f0 0x7fb8d41514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:53.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.030+0000 7fb8da9e4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8d41420e0 0x7fb8d413f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41462/0 (socket says 172.21.15.32:41462) 2020-12-05T17:41:53.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.030+0000 7fb8da9e4700 1 -- 172.21.15.32:0/1918775269 learned_addr learned my addr 172.21.15.32:0/1918775269 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:53.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.030+0000 7fb8d99e2700 1 --2- 172.21.15.32:0/1918775269 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb8d4061b30 0x7fb8d40b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:53.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.030+0000 7fb8da9e4700 1 -- 172.21.15.32:0/1918775269 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8d405f6f0 msgr2=0x7fb8d41514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:53.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.030+0000 7fb8da9e4700 1 --2- 172.21.15.32:0/1918775269 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8d405f6f0 0x7fb8d41514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:53.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.030+0000 7fb8da9e4700 1 -- 172.21.15.32:0/1918775269 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb8d4061b30 msgr2=0x7fb8d40b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:53.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.030+0000 7fb8da9e4700 1 --2- 172.21.15.32:0/1918775269 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb8d4061b30 0x7fb8d40b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:53.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.030+0000 7fb8da9e4700 1 -- 172.21.15.32:0/1918775269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d41519b0 con 0x7fb8d41420e0 2020-12-05T17:41:53.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.030+0000 7fb8da9e4700 1 --2- 172.21.15.32:0/1918775269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8d41420e0 0x7fb8d413f4f0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7fb8cc002f30 tx=0x7fb8cc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:53.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.030+0000 7fb8d37fe700 1 -- 172.21.15.32:0/1918775269 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8cc011b00 con 0x7fb8d41420e0 2020-12-05T17:41:53.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.030+0000 7fb8d37fe700 1 -- 172.21.15.32:0/1918775269 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8cc011c60 con 0x7fb8d41420e0 2020-12-05T17:41:53.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.030+0000 7fb8db1e5700 1 -- 172.21.15.32:0/1918775269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d4065d10 con 0x7fb8d41420e0 2020-12-05T17:41:53.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.030+0000 7fb8d37fe700 1 -- 172.21.15.32:0/1918775269 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8cc01e460 con 0x7fb8d41420e0 2020-12-05T17:41:53.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.030+0000 7fb8db1e5700 1 -- 172.21.15.32:0/1918775269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d40b3f10 con 0x7fb8d41420e0 2020-12-05T17:41:53.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.034+0000 7fb8db1e5700 1 -- 172.21.15.32:0/1918775269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d4057050 con 0x7fb8d41420e0 2020-12-05T17:41:53.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.034+0000 7fb8d37fe700 1 -- 172.21.15.32:0/1918775269 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb8cc025030 con 0x7fb8d41420e0 2020-12-05T17:41:53.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.034+0000 7fb8d37fe700 1 --2- 172.21.15.32:0/1918775269 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8b4055440 0x7fb8b4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:53.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.034+0000 7fb8d37fe700 1 -- 172.21.15.32:0/1918775269 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fb8cc06efb0 con 0x7fb8d41420e0 2020-12-05T17:41:53.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.034+0000 7fb8da1e3700 1 --2- 172.21.15.32:0/1918775269 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8b4055440 0x7fb8b4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:53.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.034+0000 7fb8da1e3700 1 --2- 172.21.15.32:0/1918775269 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8b4055440 0x7fb8b4057900 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7fb8c400d000 tx=0x7fb8c4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:53.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.042+0000 7fb8d37fe700 1 -- 172.21.15.32:0/1918775269 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb8cc045490 con 0x7fb8d41420e0 2020-12-05T17:41:53.184 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:53.182+0000 7fb8db1e5700 1 -- 172.21.15.32:0/1918775269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fb8d4069690 con 0x7fb8d41420e0 2020-12-05T17:41:54.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.189+0000 7fb8d37fe700 1 -- 172.21.15.32:0/1918775269 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v144) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb8cc023020 con 0x7fb8d41420e0 2020-12-05T17:41:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.193+0000 7fb8db1e5700 1 -- 172.21.15.32:0/1918775269 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8b4055440 msgr2=0x7fb8b4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.193+0000 7fb8db1e5700 1 --2- 172.21.15.32:0/1918775269 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8b4055440 0x7fb8b4057900 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7fb8c400d000 tx=0x7fb8c4008040).stop 2020-12-05T17:41:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.193+0000 7fb8db1e5700 1 -- 172.21.15.32:0/1918775269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8d41420e0 msgr2=0x7fb8d413f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.193+0000 7fb8db1e5700 1 --2- 172.21.15.32:0/1918775269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8d41420e0 0x7fb8d413f4f0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7fb8cc002f30 tx=0x7fb8cc015000).stop 2020-12-05T17:41:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.193+0000 7fb8db1e5700 1 -- 172.21.15.32:0/1918775269 shutdown_connections 2020-12-05T17:41:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.193+0000 7fb8db1e5700 1 --2- 172.21.15.32:0/1918775269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8d41420e0 0x7fb8d413f4f0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.193+0000 7fb8db1e5700 1 --2- 172.21.15.32:0/1918775269 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb8b4055440 0x7fb8b4057900 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:54.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.193+0000 7fb8db1e5700 1 --2- 172.21.15.32:0/1918775269 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb8d405f6f0 0x7fb8d41514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:54.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.193+0000 7fb8db1e5700 1 --2- 172.21.15.32:0/1918775269 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb8d4061b30 0x7fb8d40b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:54.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.193+0000 7fb8db1e5700 1 -- 172.21.15.32:0/1918775269 >> 172.21.15.32:0/1918775269 conn(0x7fb8d4061010 msgr2=0x7fb8d40ade50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:54.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.193+0000 7fb8db1e5700 1 -- 172.21.15.32:0/1918775269 shutdown_connections 2020-12-05T17:41:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.193+0000 7fb8db1e5700 1 -- 172.21.15.32:0/1918775269 wait complete. 2020-12-05T17:41:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2020-12-05T17:41:54.211 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-12-05T17:41:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.325+0000 7f403993c700 1 Processor -- start 2020-12-05T17:41:54.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f403993c700 1 -- start start 2020-12-05T17:41:54.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f403993c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f4034063040 0x7f403413fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:54.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f403993c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40340af090 0x7f40340af500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:54.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f403993c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4034067110 con 0x7f403405f550 2020-12-05T17:41:54.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f403993c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40340ad290 con 0x7f40340af090 2020-12-05T17:41:54.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f403993c700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f403413f760 con 0x7f4034063040 2020-12-05T17:41:54.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f403913b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40340af090 0x7f40340af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:54.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f403913b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40340af090 0x7f40340af500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41470/0 (socket says 172.21.15.32:41470) 2020-12-05T17:41:54.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f4033fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f4034063040 0x7f403413fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:54.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f403913b700 1 -- 172.21.15.32:0/2921828757 learned_addr learned my addr 172.21.15.32:0/2921828757 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:54.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f4033fff700 1 -- 172.21.15.32:0/2921828757 >> v2:172.21.15.32:3300/0 conn(0x7f40340af090 msgr2=0x7f40340af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:54.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f4033fff700 1 --2- 172.21.15.32:0/2921828757 >> v2:172.21.15.32:3300/0 conn(0x7f40340af090 0x7f40340af500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:54.339 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f4033fff700 1 -- 172.21.15.32:0/2921828757 >> v1:172.21.15.61:6789/0 conn(0x7f403405f550 legacy=0x7f40340acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:54.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f40337fe700 1 -- 172.21.15.32:0/2921828757 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1591908202 0 0) 0x7f4034067110 con 0x7f403405f550 2020-12-05T17:41:54.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f4033fff700 1 -- 172.21.15.32:0/2921828757 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403413fa00 con 0x7f4034063040 2020-12-05T17:41:54.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f4033fff700 1 --2- 172.21.15.32:0/2921828757 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4034063040 0x7f403413fea0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f4020002f30 tx=0x7f4020006e50).ready entity=mon.1 client_cookie=31eb07273299bcc6 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:54.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f40337fe700 1 -- 172.21.15.32:0/2921828757 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f402000be50 con 0x7f4034063040 2020-12-05T17:41:54.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f40337fe700 1 -- 172.21.15.32:0/2921828757 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4020010470 con 0x7f4034063040 2020-12-05T17:41:54.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f40337fe700 1 -- 172.21.15.32:0/2921828757 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40200107d0 con 0x7f4034063040 2020-12-05T17:41:54.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f403993c700 1 -- 172.21.15.32:0/2921828757 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4034063040 msgr2=0x7f403413fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:54.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f403993c700 1 --2- 172.21.15.32:0/2921828757 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4034063040 0x7f403413fea0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f4020002f30 tx=0x7f4020006e50).stop 2020-12-05T17:41:54.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f403993c700 1 -- 172.21.15.32:0/2921828757 shutdown_connections 2020-12-05T17:41:54.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f403993c700 1 --2- 172.21.15.32:0/2921828757 >> v2:172.21.15.32:3300/0 conn(0x7f40340af090 0x7f40340af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:54.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f403993c700 1 --2- 172.21.15.32:0/2921828757 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4034063040 0x7f403413fea0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:54.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.329+0000 7f403993c700 1 -- 172.21.15.32:0/2921828757 >> 172.21.15.32:0/2921828757 conn(0x7f40340620b0 msgr2=0x7f4034062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:54.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.333+0000 7f403993c700 1 -- 172.21.15.32:0/2921828757 shutdown_connections 2020-12-05T17:41:54.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.333+0000 7f403993c700 1 -- 172.21.15.32:0/2921828757 wait complete. 2020-12-05T17:41:54.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.333+0000 7f403993c700 1 Processor -- start 2020-12-05T17:41:54.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.333+0000 7f403993c700 1 -- start start 2020-12-05T17:41:54.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.333+0000 7f403993c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f403405f550 0x7f4034144160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:54.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.333+0000 7f403993c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4034063040 0x7f403413ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:54.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.333+0000 7f403993c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40340af090 0x7f403413ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:54.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.333+0000 7f403993c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4034138a50 con 0x7f4034063040 2020-12-05T17:41:54.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.333+0000 7f403993c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403413a4e0 con 0x7f403405f550 2020-12-05T17:41:54.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.333+0000 7f403993c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403413c520 con 0x7f40340af090 2020-12-05T17:41:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.333+0000 7f4033fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4034063040 0x7f403413ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.333+0000 7f4033fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4034063040 0x7f403413ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41474/0 (socket says 172.21.15.32:41474) 2020-12-05T17:41:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.333+0000 7f4033fff700 1 -- 172.21.15.32:0/139334076 learned_addr learned my addr 172.21.15.32:0/139334076 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.333+0000 7f403893a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f403405f550 0x7f4034144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:54.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.333+0000 7f403913b700 1 --2- 172.21.15.32:0/139334076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40340af090 0x7f403413ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:54.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.337+0000 7f4033fff700 1 -- 172.21.15.32:0/139334076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40340af090 msgr2=0x7f403413ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:54.353 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.337+0000 7f4033fff700 1 --2- 172.21.15.32:0/139334076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40340af090 0x7f403413ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:54.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.337+0000 7f4033fff700 1 -- 172.21.15.32:0/139334076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f403405f550 msgr2=0x7f4034144160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:54.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.337+0000 7f4033fff700 1 --2- 172.21.15.32:0/139334076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f403405f550 0x7f4034144160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:54.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.337+0000 7f4033fff700 1 -- 172.21.15.32:0/139334076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403413c7c0 con 0x7f4034063040 2020-12-05T17:41:54.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.337+0000 7f4033fff700 1 --2- 172.21.15.32:0/139334076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4034063040 0x7f403413ea30 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0x7f4020002f30 tx=0x7f40340ad3d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:54.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.337+0000 7f4031ffb700 1 -- 172.21.15.32:0/139334076 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4020020070 con 0x7f4034063040 2020-12-05T17:41:54.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.337+0000 7f4031ffb700 1 -- 172.21.15.32:0/139334076 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40200197e0 con 0x7f4034063040 2020-12-05T17:41:54.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.337+0000 7f403993c700 1 -- 172.21.15.32:0/139334076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f403413a160 con 0x7f4034063040 2020-12-05T17:41:54.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.337+0000 7f4031ffb700 1 -- 172.21.15.32:0/139334076 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4020019b40 con 0x7f4034063040 2020-12-05T17:41:54.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.337+0000 7f403993c700 1 -- 172.21.15.32:0/139334076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4034144650 con 0x7f4034063040 2020-12-05T17:41:54.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.337+0000 7f403993c700 1 -- 172.21.15.32:0/139334076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4034057050 con 0x7f4034063040 2020-12-05T17:41:54.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.337+0000 7f4031ffb700 1 -- 172.21.15.32:0/139334076 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4020019ca0 con 0x7f4034063040 2020-12-05T17:41:54.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.337+0000 7f4031ffb700 1 --2- 172.21.15.32:0/139334076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f401c055440 0x7f401c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:54.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.337+0000 7f4031ffb700 1 -- 172.21.15.32:0/139334076 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f4020071960 con 0x7f4034063040 2020-12-05T17:41:54.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.337+0000 7f403893a700 1 --2- 172.21.15.32:0/139334076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f401c055440 0x7f401c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:54.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.341+0000 7f403893a700 1 --2- 172.21.15.32:0/139334076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f401c055440 0x7f401c057900 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f402800d000 tx=0x7f4028007a60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:54.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.345+0000 7f4031ffb700 1 -- 172.21.15.32:0/139334076 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4020047c50 con 0x7f4034063040 2020-12-05T17:41:54.486 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:54.485+0000 7f403993c700 1 -- 172.21.15.32:0/139334076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f4034060c10 con 0x7f4034063040 2020-12-05T17:41:55.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.325+0000 7f4031ffb700 1 -- 172.21.15.32:0/139334076 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v146) v1 ==== 125+0+0 (secure 0 0 0) 0x7f402004b270 con 0x7f4034063040 2020-12-05T17:41:55.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.329+0000 7f403993c700 1 -- 172.21.15.32:0/139334076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f401c055440 msgr2=0x7f401c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:55.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.329+0000 7f403993c700 1 --2- 172.21.15.32:0/139334076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f401c055440 0x7f401c057900 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f402800d000 tx=0x7f4028007a60).stop 2020-12-05T17:41:55.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.329+0000 7f403993c700 1 -- 172.21.15.32:0/139334076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4034063040 msgr2=0x7f403413ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:55.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.329+0000 7f403993c700 1 --2- 172.21.15.32:0/139334076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4034063040 0x7f403413ea30 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0x7f4020002f30 tx=0x7f40340ad3d0).stop 2020-12-05T17:41:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.329+0000 7f403993c700 1 -- 172.21.15.32:0/139334076 shutdown_connections 2020-12-05T17:41:55.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.329+0000 7f403993c700 1 --2- 172.21.15.32:0/139334076 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f401c055440 0x7f401c057900 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:55.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.329+0000 7f403993c700 1 --2- 172.21.15.32:0/139334076 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40340af090 0x7f403413ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:55.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.329+0000 7f403993c700 1 --2- 172.21.15.32:0/139334076 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f403405f550 0x7f4034144160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:55.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.329+0000 7f403993c700 1 --2- 172.21.15.32:0/139334076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4034063040 0x7f403413ea30 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:55.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.329+0000 7f403993c700 1 -- 172.21.15.32:0/139334076 >> 172.21.15.32:0/139334076 conn(0x7f40340620b0 msgr2=0x7f4034067460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:55.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.333+0000 7f403993c700 1 -- 172.21.15.32:0/139334076 shutdown_connections 2020-12-05T17:41:55.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.333+0000 7f403993c700 1 -- 172.21.15.32:0/139334076 wait complete. 2020-12-05T17:41:55.336 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2020-12-05T17:41:55.347 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-12-05T17:41:55.463 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.461+0000 7f17fbfff700 1 Processor -- start 2020-12-05T17:41:55.464 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.461+0000 7f17fbfff700 1 -- start start 2020-12-05T17:41:55.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.461+0000 7f17fbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f17fc0b2960 0x7f17fc140400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:55.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.461+0000 7f17fbfff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f17fc0667f0 0x7f17fc0ae830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:55.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.461+0000 7f17fbfff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17fc0ad9b0 con 0x7f17fc0ad280 2020-12-05T17:41:55.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.461+0000 7f17fbfff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17fc0b32c0 con 0x7f17fc0b2960 2020-12-05T17:41:55.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.461+0000 7f17fbfff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17fc13f4f0 con 0x7f17fc0667f0 2020-12-05T17:41:55.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.461+0000 7f17faffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f17fc0b2960 0x7f17fc140400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:55.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.461+0000 7f17fa7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f17fc0ad280 0x7f17fc0660e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44128/0 (socket says 172.21.15.32:44128) 2020-12-05T17:41:55.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.461+0000 7f17fa7fc700 1 -- 172.21.15.32:0/377572409 learned_addr learned my addr 172.21.15.32:0/377572409 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:55.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.461+0000 7f17fb7fe700 1 --2- 172.21.15.32:0/377572409 >> v2:172.21.15.61:3300/0 conn(0x7f17fc0667f0 0x7f17fc0ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:55.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.461+0000 7f17faffd700 1 -- 172.21.15.32:0/377572409 >> v2:172.21.15.61:3300/0 conn(0x7f17fc0667f0 msgr2=0x7f17fc0ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:55.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.461+0000 7f17faffd700 1 --2- 172.21.15.32:0/377572409 >> v2:172.21.15.61:3300/0 conn(0x7f17fc0667f0 0x7f17fc0ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:55.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.461+0000 7f17faffd700 1 -- 172.21.15.32:0/377572409 >> v1:172.21.15.32:6789/0 conn(0x7f17fc0ad280 legacy=0x7f17fc0660e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:55.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.461+0000 7f17f9ffb700 1 -- 172.21.15.32:0/377572409 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1662811106 0 0) 0x7f17fc0ad9b0 con 0x7f17fc0ad280 2020-12-05T17:41:55.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.461+0000 7f17faffd700 1 -- 172.21.15.32:0/377572409 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17fc0b3020 con 0x7f17fc0b2960 2020-12-05T17:41:55.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.465+0000 7f17faffd700 1 --2- 172.21.15.32:0/377572409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17fc0b2960 0x7f17fc140400 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0x7f17ec0032b0 tx=0x7f17ec0091a0).ready entity=mon.0 client_cookie=cebd8453bd825f30 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:55.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.465+0000 7f17f9ffb700 1 -- 172.21.15.32:0/377572409 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17ec00b9a0 con 0x7f17fc0b2960 2020-12-05T17:41:55.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.465+0000 7f17f9ffb700 1 -- 172.21.15.32:0/377572409 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ec00bb00 con 0x7f17fc0b2960 2020-12-05T17:41:55.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.465+0000 7f17f9ffb700 1 -- 172.21.15.32:0/377572409 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17ec00bc70 con 0x7f17fc0b2960 2020-12-05T17:41:55.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.465+0000 7f17fbfff700 1 -- 172.21.15.32:0/377572409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17fc0b2960 msgr2=0x7f17fc140400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:55.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.465+0000 7f17fbfff700 1 --2- 172.21.15.32:0/377572409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17fc0b2960 0x7f17fc140400 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0x7f17ec0032b0 tx=0x7f17ec0091a0).stop 2020-12-05T17:41:55.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.465+0000 7f17fbfff700 1 -- 172.21.15.32:0/377572409 shutdown_connections 2020-12-05T17:41:55.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.465+0000 7f17fbfff700 1 --2- 172.21.15.32:0/377572409 >> v2:172.21.15.61:3300/0 conn(0x7f17fc0667f0 0x7f17fc0ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:55.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.465+0000 7f17fbfff700 1 --2- 172.21.15.32:0/377572409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17fc0b2960 0x7f17fc140400 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:55.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.465+0000 7f17fbfff700 1 -- 172.21.15.32:0/377572409 >> 172.21.15.32:0/377572409 conn(0x7f17fc061010 msgr2=0x7f17fc061420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:55.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.465+0000 7f17fbfff700 1 -- 172.21.15.32:0/377572409 shutdown_connections 2020-12-05T17:41:55.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.465+0000 7f17fbfff700 1 -- 172.21.15.32:0/377572409 wait complete. 2020-12-05T17:41:55.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.465+0000 7f17fbfff700 1 Processor -- start 2020-12-05T17:41:55.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.465+0000 7f17fbfff700 1 -- start start 2020-12-05T17:41:55.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17fbfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17fc0667f0 0x7f17fc0b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:55.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17fc0ad280 0x7f17fc13b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:55.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17fbfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f17fc0b2960 0x7f17fc13ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:55.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17fbfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc13bf20 con 0x7f17fc0ad280 2020-12-05T17:41:55.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17fbfff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc064e80 con 0x7f17fc0667f0 2020-12-05T17:41:55.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17fbfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc063040 con 0x7f17fc0b2960 2020-12-05T17:41:55.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17fa7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17fc0ad280 0x7f17fc13b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:55.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17fa7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17fc0ad280 0x7f17fc13b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41488/0 (socket says 172.21.15.32:41488) 2020-12-05T17:41:55.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17fa7fc700 1 -- 172.21.15.32:0/2573429826 learned_addr learned my addr 172.21.15.32:0/2573429826 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:55.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17faffd700 1 --2- 172.21.15.32:0/2573429826 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17fc0667f0 0x7f17fc0b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:55.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17fb7fe700 1 --2- 172.21.15.32:0/2573429826 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f17fc0b2960 0x7f17fc13ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:55.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17fa7fc700 1 -- 172.21.15.32:0/2573429826 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f17fc0b2960 msgr2=0x7f17fc13ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:55.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17fa7fc700 1 --2- 172.21.15.32:0/2573429826 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f17fc0b2960 0x7f17fc13ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:55.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17fa7fc700 1 -- 172.21.15.32:0/2573429826 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17fc0667f0 msgr2=0x7f17fc0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:55.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17fa7fc700 1 --2- 172.21.15.32:0/2573429826 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17fc0667f0 0x7f17fc0b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:55.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17fa7fc700 1 -- 172.21.15.32:0/2573429826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17fc063550 con 0x7f17fc0ad280 2020-12-05T17:41:55.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17fa7fc700 1 --2- 172.21.15.32:0/2573429826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17fc0ad280 0x7f17fc13b540 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 rx=0x7f17e4000fa0 tx=0x7f17e4007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:55.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17f3fff700 1 -- 172.21.15.32:0/2573429826 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17e40072e0 con 0x7f17fc0ad280 2020-12-05T17:41:55.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17f3fff700 1 -- 172.21.15.32:0/2573429826 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e4007440 con 0x7f17fc0ad280 2020-12-05T17:41:55.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17fbfff700 1 -- 172.21.15.32:0/2573429826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17fc14f1f0 con 0x7f17fc0ad280 2020-12-05T17:41:55.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17f3fff700 1 -- 172.21.15.32:0/2573429826 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17e40057d0 con 0x7f17fc0ad280 2020-12-05T17:41:55.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17fbfff700 1 -- 172.21.15.32:0/2573429826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17fc143990 con 0x7f17fc0ad280 2020-12-05T17:41:55.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17f3fff700 1 -- 172.21.15.32:0/2573429826 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f17e4010070 con 0x7f17fc0ad280 2020-12-05T17:41:55.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.469+0000 7f17fbfff700 1 -- 172.21.15.32:0/2573429826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17fc064ce0 con 0x7f17fc0ad280 2020-12-05T17:41:55.480 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.473+0000 7f17f3fff700 1 --2- 172.21.15.32:0/2573429826 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f17e0055440 0x7f17e0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:55.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.473+0000 7f17f3fff700 1 -- 172.21.15.32:0/2573429826 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f17e406d7f0 con 0x7f17fc0ad280 2020-12-05T17:41:55.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.473+0000 7f17faffd700 1 --2- 172.21.15.32:0/2573429826 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f17e0055440 0x7f17e0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:55.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.473+0000 7f17faffd700 1 --2- 172.21.15.32:0/2573429826 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f17e0055440 0x7f17e0057900 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f17ec01c000 tx=0x7f17ec016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:55.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.477+0000 7f17f3fff700 1 -- 172.21.15.32:0/2573429826 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f17fc064ce0 con 0x7f17fc0ad280 2020-12-05T17:41:55.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.661+0000 7f17fbfff700 1 -- 172.21.15.32:0/2573429826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f17fc063740 con 0x7f17fc0ad280 2020-12-05T17:41:55.678 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.673+0000 7f17f3fff700 1 -- 172.21.15.32:0/2573429826 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 29 and data pool 30 v71) v1 ==== 189+0+0 (secure 0 0 0) 0x7f17fc063740 con 0x7f17fc0ad280 2020-12-05T17:41:55.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.677+0000 7f17fbfff700 1 -- 172.21.15.32:0/2573429826 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f17e0055440 msgr2=0x7f17e0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:55.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.677+0000 7f17fbfff700 1 --2- 172.21.15.32:0/2573429826 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f17e0055440 0x7f17e0057900 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f17ec01c000 tx=0x7f17ec016000).stop 2020-12-05T17:41:55.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.677+0000 7f17fbfff700 1 -- 172.21.15.32:0/2573429826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17fc0ad280 msgr2=0x7f17fc13b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:55.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.677+0000 7f17fbfff700 1 --2- 172.21.15.32:0/2573429826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17fc0ad280 0x7f17fc13b540 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 rx=0x7f17e4000fa0 tx=0x7f17e4007ac0).stop 2020-12-05T17:41:55.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.677+0000 7f17fbfff700 1 -- 172.21.15.32:0/2573429826 shutdown_connections 2020-12-05T17:41:55.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.677+0000 7f17fbfff700 1 --2- 172.21.15.32:0/2573429826 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f17e0055440 0x7f17e0057900 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:55.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.677+0000 7f17fbfff700 1 --2- 172.21.15.32:0/2573429826 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f17fc0b2960 0x7f17fc13ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:55.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.677+0000 7f17fbfff700 1 --2- 172.21.15.32:0/2573429826 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17fc0667f0 0x7f17fc0b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:55.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.677+0000 7f17fbfff700 1 --2- 172.21.15.32:0/2573429826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17fc0ad280 0x7f17fc13b540 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:55.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.677+0000 7f17fbfff700 1 -- 172.21.15.32:0/2573429826 >> 172.21.15.32:0/2573429826 conn(0x7f17fc061010 msgr2=0x7f17fc0aedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:55.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.677+0000 7f17fbfff700 1 -- 172.21.15.32:0/2573429826 shutdown_connections 2020-12-05T17:41:55.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.677+0000 7f17fbfff700 1 -- 172.21.15.32:0/2573429826 wait complete. 2020-12-05T17:41:55.684 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 29 and data pool 30 2020-12-05T17:41:55.695 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:41:55.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.793+0000 7f9ac6472700 1 Processor -- start 2020-12-05T17:41:55.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.793+0000 7f9ac6472700 1 -- start start 2020-12-05T17:41:55.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ac6472700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9ac005f6f0 0x7f9ac01428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:55.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ac6472700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9ac00b0880 0x7f9ac0066070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:55.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ac6472700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac005fad0 con 0x7f9ac00b1980 2020-12-05T17:41:55.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ac6472700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac00b2f80 con 0x7f9ac005f6f0 2020-12-05T17:41:55.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ac6472700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac0145e40 con 0x7f9ac00b0880 2020-12-05T17:41:55.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ac5470700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9ac005f6f0 0x7f9ac01428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:55.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ac4c6f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9ac00b1980 0x7f9ac0146580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44148/0 (socket says 172.21.15.32:44148) 2020-12-05T17:41:55.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ac4c6f700 1 -- 172.21.15.32:0/117802541 learned_addr learned my addr 172.21.15.32:0/117802541 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:55.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ac5c71700 1 --2- 172.21.15.32:0/117802541 >> v2:172.21.15.61:3300/0 conn(0x7f9ac00b0880 0x7f9ac0066070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:55.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ab7fff700 1 -- 172.21.15.32:0/117802541 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3418214461 0 0) 0x7f9ac005fad0 con 0x7f9ac00b1980 2020-12-05T17:41:55.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ab7fff700 1 -- 172.21.15.32:0/117802541 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9aa4003230 con 0x7f9ac00b1980 2020-12-05T17:41:55.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ac5470700 1 -- 172.21.15.32:0/117802541 >> v2:172.21.15.61:3300/0 conn(0x7f9ac00b0880 msgr2=0x7f9ac0066070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:55.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ac5470700 1 --2- 172.21.15.32:0/117802541 >> v2:172.21.15.61:3300/0 conn(0x7f9ac00b0880 0x7f9ac0066070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:55.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ac5470700 1 -- 172.21.15.32:0/117802541 >> v1:172.21.15.32:6789/0 conn(0x7f9ac00b1980 legacy=0x7f9ac0146580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:55.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ac5470700 1 -- 172.21.15.32:0/117802541 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac01460e0 con 0x7f9ac005f6f0 2020-12-05T17:41:55.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ac5470700 1 --2- 172.21.15.32:0/117802541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ac005f6f0 0x7f9ac01428f0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7f9abc002f30 tx=0x7f9abc006dc0).ready entity=mon.0 client_cookie=8f24eeaa46d09b29 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:55.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ab7fff700 1 -- 172.21.15.32:0/117802541 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9abc0096f0 con 0x7f9ac005f6f0 2020-12-05T17:41:55.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ab7fff700 1 -- 172.21.15.32:0/117802541 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9abc009850 con 0x7f9ac005f6f0 2020-12-05T17:41:55.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ab7fff700 1 -- 172.21.15.32:0/117802541 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9abc009bb0 con 0x7f9ac005f6f0 2020-12-05T17:41:55.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ac6472700 1 -- 172.21.15.32:0/117802541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ac005f6f0 msgr2=0x7f9ac01428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:55.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ac6472700 1 --2- 172.21.15.32:0/117802541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ac005f6f0 0x7f9ac01428f0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7f9abc002f30 tx=0x7f9abc006dc0).stop 2020-12-05T17:41:55.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ac6472700 1 -- 172.21.15.32:0/117802541 shutdown_connections 2020-12-05T17:41:55.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ac6472700 1 --2- 172.21.15.32:0/117802541 >> v2:172.21.15.61:3300/0 conn(0x7f9ac00b0880 0x7f9ac0066070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:55.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ac6472700 1 --2- 172.21.15.32:0/117802541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ac005f6f0 0x7f9ac01428f0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:55.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ac6472700 1 -- 172.21.15.32:0/117802541 >> 172.21.15.32:0/117802541 conn(0x7f9ac00604c0 msgr2=0x7f9ac005c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:55.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ac6472700 1 -- 172.21.15.32:0/117802541 shutdown_connections 2020-12-05T17:41:55.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.797+0000 7f9ac6472700 1 -- 172.21.15.32:0/117802541 wait complete. 2020-12-05T17:41:55.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.801+0000 7f9ac6472700 1 Processor -- start 2020-12-05T17:41:55.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.801+0000 7f9ac6472700 1 -- start start 2020-12-05T17:41:55.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.801+0000 7f9ac6472700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ac005f6f0 0x7f9ac0116360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:55.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.801+0000 7f9ac6472700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ac00b0880 0x7f9ac014e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:55.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.801+0000 7f9ac6472700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ac00b1980 0x7f9ac014ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:55.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.801+0000 7f9ac6472700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac0116c60 con 0x7f9ac005f6f0 2020-12-05T17:41:55.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.801+0000 7f9ac6472700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac0064c80 con 0x7f9ac00b1980 2020-12-05T17:41:55.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.801+0000 7f9ac6472700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac0064df0 con 0x7f9ac00b0880 2020-12-05T17:41:55.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.801+0000 7f9ac5470700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ac005f6f0 0x7f9ac0116360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:55.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.801+0000 7f9ac5470700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ac005f6f0 0x7f9ac0116360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41506/0 (socket says 172.21.15.32:41506) 2020-12-05T17:41:55.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.801+0000 7f9ac5470700 1 -- 172.21.15.32:0/792902730 learned_addr learned my addr 172.21.15.32:0/792902730 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:55.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.801+0000 7f9ac5c71700 1 --2- 172.21.15.32:0/792902730 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ac00b1980 0x7f9ac014ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:55.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.801+0000 7f9ac4c6f700 1 --2- 172.21.15.32:0/792902730 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ac00b0880 0x7f9ac014e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:55.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.805+0000 7f9ac5470700 1 -- 172.21.15.32:0/792902730 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ac00b0880 msgr2=0x7f9ac014e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:55.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.805+0000 7f9ac5470700 1 --2- 172.21.15.32:0/792902730 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ac00b0880 0x7f9ac014e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:55.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.805+0000 7f9ac5470700 1 -- 172.21.15.32:0/792902730 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ac00b1980 msgr2=0x7f9ac014ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:55.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.805+0000 7f9ac5470700 1 --2- 172.21.15.32:0/792902730 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ac00b1980 0x7f9ac014ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:55.810 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.805+0000 7f9ac5470700 1 -- 172.21.15.32:0/792902730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac0065bd0 con 0x7f9ac005f6f0 2020-12-05T17:41:55.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.805+0000 7f9ac5470700 1 --2- 172.21.15.32:0/792902730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ac005f6f0 0x7f9ac0116360 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0x7f9abc002f30 tx=0x7f9abc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:55.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.805+0000 7f9ab67fc700 1 -- 172.21.15.32:0/792902730 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9abc011ac0 con 0x7f9ac005f6f0 2020-12-05T17:41:55.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.805+0000 7f9ab67fc700 1 -- 172.21.15.32:0/792902730 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9abc011c20 con 0x7f9ac005f6f0 2020-12-05T17:41:55.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.805+0000 7f9ab67fc700 1 -- 172.21.15.32:0/792902730 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9abc01d450 con 0x7f9ac005f6f0 2020-12-05T17:41:55.811 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.805+0000 7f9ac6472700 1 -- 172.21.15.32:0/792902730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac01521a0 con 0x7f9ac005f6f0 2020-12-05T17:41:55.812 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.805+0000 7f9ac6472700 1 -- 172.21.15.32:0/792902730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac01169f0 con 0x7f9ac005f6f0 2020-12-05T17:41:55.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.805+0000 7f9ac6472700 1 -- 172.21.15.32:0/792902730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac0057050 con 0x7f9ac005f6f0 2020-12-05T17:41:55.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.805+0000 7f9ab67fc700 1 -- 172.21.15.32:0/792902730 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9abc024030 con 0x7f9ac005f6f0 2020-12-05T17:41:55.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.805+0000 7f9ab67fc700 1 --2- 172.21.15.32:0/792902730 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9aa8055440 0x7f9aa8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:55.813 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.805+0000 7f9ab67fc700 1 -- 172.21.15.32:0/792902730 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9abc06f100 con 0x7f9ac005f6f0 2020-12-05T17:41:55.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.805+0000 7f9ac4c6f700 1 --2- 172.21.15.32:0/792902730 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9aa8055440 0x7f9aa8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:55.814 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.809+0000 7f9ac4c6f700 1 --2- 172.21.15.32:0/792902730 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9aa8055440 0x7f9aa8057900 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f9ab000d000 tx=0x7f9ab0009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:55.817 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.817+0000 7f9ab67fc700 1 -- 172.21.15.32:0/792902730 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9abc0452a0 con 0x7f9ac005f6f0 2020-12-05T17:41:55.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.961+0000 7f9ac6472700 1 -- 172.21.15.32:0/792902730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ac0065dc0 con 0x7f9ac005f6f0 2020-12-05T17:41:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.961+0000 7f9ab67fc700 1 -- 172.21.15.32:0/792902730 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f9abc022020 con 0x7f9ac005f6f0 2020-12-05T17:41:55.965 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:55.965 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":147,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:41:55.663458+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:41:53.238803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2020-12-05T17:41:54.541493+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:41:55.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.965+0000 7f9ac6472700 1 -- 172.21.15.32:0/792902730 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9aa8055440 msgr2=0x7f9aa8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:55.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.965+0000 7f9ac6472700 1 --2- 172.21.15.32:0/792902730 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9aa8055440 0x7f9aa8057900 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f9ab000d000 tx=0x7f9ab0009040).stop 2020-12-05T17:41:55.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.965+0000 7f9ac6472700 1 -- 172.21.15.32:0/792902730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ac005f6f0 msgr2=0x7f9ac0116360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:55.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.965+0000 7f9ac6472700 1 --2- 172.21.15.32:0/792902730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ac005f6f0 0x7f9ac0116360 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0x7f9abc002f30 tx=0x7f9abc015000).stop 2020-12-05T17:41:55.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.965+0000 7f9ac6472700 1 -- 172.21.15.32:0/792902730 shutdown_connections 2020-12-05T17:41:55.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.965+0000 7f9ac6472700 1 --2- 172.21.15.32:0/792902730 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ac00b1980 0x7f9ac014ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:55.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.965+0000 7f9ac6472700 1 --2- 172.21.15.32:0/792902730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ac005f6f0 0x7f9ac0116360 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:55.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.965+0000 7f9ac6472700 1 --2- 172.21.15.32:0/792902730 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9aa8055440 0x7f9aa8057900 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:55.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.965+0000 7f9ac6472700 1 --2- 172.21.15.32:0/792902730 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ac00b0880 0x7f9ac014e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:55.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.965+0000 7f9ac6472700 1 -- 172.21.15.32:0/792902730 >> 172.21.15.32:0/792902730 conn(0x7f9ac00604c0 msgr2=0x7f9ac0061010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:55.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.965+0000 7f9ac6472700 1 -- 172.21.15.32:0/792902730 shutdown_connections 2020-12-05T17:41:55.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:55.965+0000 7f9ac6472700 1 -- 172.21.15.32:0/792902730 wait complete. 2020-12-05T17:41:55.980 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:41:56.095 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.093+0000 7f7e6496a700 1 Processor -- start 2020-12-05T17:41:56.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.093+0000 7f7e6496a700 1 -- start start 2020-12-05T17:41:56.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.093+0000 7f7e6496a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f7e6005f6f0 0x7f7e601428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:56.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.093+0000 7f7e6496a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7e600b1980 0x7f7e60146580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:56.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.093+0000 7f7e6496a700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e6005fad0 con 0x7f7e60146a70 2020-12-05T17:41:56.098 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.093+0000 7f7e6496a700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e600b2f80 con 0x7f7e6005f6f0 2020-12-05T17:41:56.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.093+0000 7f7e6496a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e60145e40 con 0x7f7e600b1980 2020-12-05T17:41:56.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.093+0000 7f7e5effd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7e600b1980 0x7f7e60146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:56.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.093+0000 7f7e5f7fe700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f7e6005f6f0 0x7f7e601428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:56.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.093+0000 7f7e5ffff700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f7e60146a70 0x7f7e60066070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:56804/0 (socket says 172.21.15.32:56804) 2020-12-05T17:41:56.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.093+0000 7f7e5effd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7e600b1980 0x7f7e60146580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:38308/0 (socket says 172.21.15.32:38308) 2020-12-05T17:41:56.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.093+0000 7f7e5f7fe700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f7e6005f6f0 0x7f7e601428f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:33174/0 (socket says 172.21.15.32:33174) 2020-12-05T17:41:56.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.093+0000 7f7e5effd700 1 -- 172.21.15.32:0/1809075823 learned_addr learned my addr 172.21.15.32:0/1809075823 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:56.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e5e7fc700 1 -- 172.21.15.32:0/1809075823 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1397598972 0 0) 0x7f7e6005fad0 con 0x7f7e60146a70 2020-12-05T17:41:56.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e5e7fc700 1 -- 172.21.15.32:0/1809075823 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e40003230 con 0x7f7e60146a70 2020-12-05T17:41:56.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e5effd700 1 -- 172.21.15.32:0/1809075823 >> v2:172.21.15.61:3300/0 conn(0x7f7e6005f6f0 msgr2=0x7f7e601428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:56.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e5effd700 1 --2- 172.21.15.32:0/1809075823 >> v2:172.21.15.61:3300/0 conn(0x7f7e6005f6f0 0x7f7e601428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:56.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e5effd700 1 -- 172.21.15.32:0/1809075823 >> v1:172.21.15.61:6789/0 conn(0x7f7e60146a70 legacy=0x7f7e60066070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:56.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e5effd700 1 -- 172.21.15.32:0/1809075823 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e601460e0 con 0x7f7e600b1980 2020-12-05T17:41:56.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e5f7fe700 1 --2- 172.21.15.32:0/1809075823 >> v2:172.21.15.61:3300/0 conn(0x7f7e6005f6f0 0x7f7e601428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:56.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e5effd700 1 --2- 172.21.15.32:0/1809075823 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7e600b1980 0x7f7e60146580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f7e48002f30 tx=0x7f7e48006e80).ready entity=mon.2 client_cookie=ff82479d865265e6 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:56.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e5e7fc700 1 -- 172.21.15.32:0/1809075823 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e48009640 con 0x7f7e600b1980 2020-12-05T17:41:56.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e5e7fc700 1 -- 172.21.15.32:0/1809075823 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e480097a0 con 0x7f7e600b1980 2020-12-05T17:41:56.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e6496a700 1 -- 172.21.15.32:0/1809075823 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7e600b1980 msgr2=0x7f7e60146580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:56.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e5e7fc700 1 -- 172.21.15.32:0/1809075823 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e48009b00 con 0x7f7e600b1980 2020-12-05T17:41:56.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e6496a700 1 --2- 172.21.15.32:0/1809075823 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7e600b1980 0x7f7e60146580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f7e48002f30 tx=0x7f7e48006e80).stop 2020-12-05T17:41:56.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e6496a700 1 -- 172.21.15.32:0/1809075823 shutdown_connections 2020-12-05T17:41:56.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e6496a700 1 --2- 172.21.15.32:0/1809075823 >> v2:172.21.15.61:3300/0 conn(0x7f7e6005f6f0 0x7f7e601428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:56.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e6496a700 1 --2- 172.21.15.32:0/1809075823 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7e600b1980 0x7f7e60146580 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:56.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e6496a700 1 -- 172.21.15.32:0/1809075823 >> 172.21.15.32:0/1809075823 conn(0x7f7e600604c0 msgr2=0x7f7e6005c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:56.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e6496a700 1 -- 172.21.15.32:0/1809075823 shutdown_connections 2020-12-05T17:41:56.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e6496a700 1 -- 172.21.15.32:0/1809075823 wait complete. 2020-12-05T17:41:56.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e6496a700 1 Processor -- start 2020-12-05T17:41:56.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e6496a700 1 -- start start 2020-12-05T17:41:56.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e6496a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7e6005f6f0 0x7f7e60065bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:56.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e6496a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7e600b1980 0x7f7e600afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:56.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e6496a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e60146a70 0x7f7e60151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:56.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e6496a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e60113540 con 0x7f7e60146a70 2020-12-05T17:41:56.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e6496a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e601554d0 con 0x7f7e600b1980 2020-12-05T17:41:56.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e6496a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e60155770 con 0x7f7e6005f6f0 2020-12-05T17:41:56.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e5ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e60146a70 0x7f7e60151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:56.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e5f7fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7e6005f6f0 0x7f7e60065bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:56.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e5ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e60146a70 0x7f7e60151490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41524/0 (socket says 172.21.15.32:41524) 2020-12-05T17:41:56.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.097+0000 7f7e5ffff700 1 -- 172.21.15.32:0/3343861134 learned_addr learned my addr 172.21.15.32:0/3343861134 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:56.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.101+0000 7f7e5effd700 1 --2- 172.21.15.32:0/3343861134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7e600b1980 0x7f7e600afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:56.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.101+0000 7f7e5ffff700 1 -- 172.21.15.32:0/3343861134 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7e6005f6f0 msgr2=0x7f7e60065bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:56.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.101+0000 7f7e5ffff700 1 --2- 172.21.15.32:0/3343861134 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7e6005f6f0 0x7f7e60065bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:56.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.101+0000 7f7e5ffff700 1 -- 172.21.15.32:0/3343861134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7e600b1980 msgr2=0x7f7e600afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:56.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.101+0000 7f7e5ffff700 1 --2- 172.21.15.32:0/3343861134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7e600b1980 0x7f7e600afa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:56.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.101+0000 7f7e5ffff700 1 -- 172.21.15.32:0/3343861134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e60113230 con 0x7f7e60146a70 2020-12-05T17:41:56.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.101+0000 7f7e5ffff700 1 --2- 172.21.15.32:0/3343861134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e60146a70 0x7f7e60151490 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0x7f7e58001450 tx=0x7f7e58007e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:56.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.101+0000 7f7e5cff9700 1 -- 172.21.15.32:0/3343861134 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e5800ce10 con 0x7f7e60146a70 2020-12-05T17:41:56.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.101+0000 7f7e6496a700 1 -- 172.21.15.32:0/3343861134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e600aff20 con 0x7f7e60146a70 2020-12-05T17:41:56.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.101+0000 7f7e5cff9700 1 -- 172.21.15.32:0/3343861134 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e58005ab0 con 0x7f7e60146a70 2020-12-05T17:41:56.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.101+0000 7f7e6496a700 1 -- 172.21.15.32:0/3343861134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e600b5c50 con 0x7f7e60146a70 2020-12-05T17:41:56.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.101+0000 7f7e5cff9700 1 -- 172.21.15.32:0/3343861134 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e58005e50 con 0x7f7e60146a70 2020-12-05T17:41:56.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.101+0000 7f7e6496a700 1 -- 172.21.15.32:0/3343861134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e600b00b0 con 0x7f7e60146a70 2020-12-05T17:41:56.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.101+0000 7f7e5cff9700 1 -- 172.21.15.32:0/3343861134 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7e58010030 con 0x7f7e60146a70 2020-12-05T17:41:56.113 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.101+0000 7f7e5cff9700 1 --2- 172.21.15.32:0/3343861134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7e44055440 0x7f7e44057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:56.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.105+0000 7f7e5f7fe700 1 --2- 172.21.15.32:0/3343861134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7e44055440 0x7f7e44057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:56.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.105+0000 7f7e5cff9700 1 -- 172.21.15.32:0/3343861134 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7e5806dfc0 con 0x7f7e60146a70 2020-12-05T17:41:56.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.105+0000 7f7e5f7fe700 1 --2- 172.21.15.32:0/3343861134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7e44055440 0x7f7e44057900 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f7e5000b000 tx=0x7f7e50006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:56.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.113+0000 7f7e5cff9700 1 -- 172.21.15.32:0/3343861134 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7e580440c0 con 0x7f7e60146a70 2020-12-05T17:41:56.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.257+0000 7f7e6496a700 1 -- 172.21.15.32:0/3343861134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e60057050 con 0x7f7e60146a70 2020-12-05T17:41:56.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.257+0000 7f7e5cff9700 1 -- 172.21.15.32:0/3343861134 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f7e580476e0 con 0x7f7e60146a70 2020-12-05T17:41:56.259 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:56.260 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":147,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:41:55.663458+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:41:53.238803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2020-12-05T17:41:54.541493+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:41:56.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.257+0000 7f7e6496a700 1 -- 172.21.15.32:0/3343861134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7e44055440 msgr2=0x7f7e44057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:56.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.257+0000 7f7e6496a700 1 --2- 172.21.15.32:0/3343861134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7e44055440 0x7f7e44057900 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f7e5000b000 tx=0x7f7e50006dc0).stop 2020-12-05T17:41:56.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.257+0000 7f7e6496a700 1 -- 172.21.15.32:0/3343861134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e60146a70 msgr2=0x7f7e60151490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:56.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.257+0000 7f7e6496a700 1 --2- 172.21.15.32:0/3343861134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e60146a70 0x7f7e60151490 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0x7f7e58001450 tx=0x7f7e58007e30).stop 2020-12-05T17:41:56.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.261+0000 7f7e6496a700 1 -- 172.21.15.32:0/3343861134 shutdown_connections 2020-12-05T17:41:56.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.261+0000 7f7e6496a700 1 --2- 172.21.15.32:0/3343861134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e60146a70 0x7f7e60151490 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:56.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.261+0000 7f7e6496a700 1 --2- 172.21.15.32:0/3343861134 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7e44055440 0x7f7e44057900 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:56.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.261+0000 7f7e6496a700 1 --2- 172.21.15.32:0/3343861134 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7e6005f6f0 0x7f7e60065bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:56.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.261+0000 7f7e6496a700 1 --2- 172.21.15.32:0/3343861134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7e600b1980 0x7f7e600afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:56.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.261+0000 7f7e6496a700 1 -- 172.21.15.32:0/3343861134 >> 172.21.15.32:0/3343861134 conn(0x7f7e600604c0 msgr2=0x7f7e60143d40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:56.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.261+0000 7f7e6496a700 1 -- 172.21.15.32:0/3343861134 shutdown_connections 2020-12-05T17:41:56.264 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.261+0000 7f7e6496a700 1 -- 172.21.15.32:0/3343861134 wait complete. 2020-12-05T17:41:56.280 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-12-05T17:41:56.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.401+0000 7ff722f62700 1 Processor -- start 2020-12-05T17:41:56.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.401+0000 7ff722f62700 1 -- start start 2020-12-05T17:41:56.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.401+0000 7ff722f62700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff71c05cd70 0x7ff71c143d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:56.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.401+0000 7ff722f62700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff71c063040 0x7ff71c0af470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:56.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.401+0000 7ff722f62700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff71c05d150 con 0x7ff71c05f5c0 2020-12-05T17:41:56.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.401+0000 7ff722f62700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff71c0af080 con 0x7ff71c063040 2020-12-05T17:41:56.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.401+0000 7ff722f62700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c0b1100 con 0x7ff71c05cd70 2020-12-05T17:41:56.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.401+0000 7ff720cfe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff71c05cd70 0x7ff71c143d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:56.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.401+0000 7ff71bfff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff71c063040 0x7ff71c0af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:56.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.401+0000 7ff720cfe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff71c05cd70 0x7ff71c143d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:38318/0 (socket says 172.21.15.32:38318) 2020-12-05T17:41:56.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.401+0000 7ff71bfff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff71c063040 0x7ff71c0af470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.32:33190/0 (socket says 172.21.15.32:33190) 2020-12-05T17:41:56.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.401+0000 7ff720cfe700 1 -- 172.21.15.32:0/829074613 learned_addr learned my addr 172.21.15.32:0/829074613 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:56.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.401+0000 7ff7214ff700 1 --1- 172.21.15.32:0/829074613 >> v1:172.21.15.61:6789/0 conn(0x7ff71c05f5c0 0x7ff71c0b5d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:56820/0 (socket says 172.21.15.32:56820) 2020-12-05T17:41:56.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.401+0000 7ff720cfe700 1 -- 172.21.15.32:0/829074613 >> v2:172.21.15.61:3300/0 conn(0x7ff71c063040 msgr2=0x7ff71c0af470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:56.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.401+0000 7ff720cfe700 1 --2- 172.21.15.32:0/829074613 >> v2:172.21.15.61:3300/0 conn(0x7ff71c063040 0x7ff71c0af470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:56.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.401+0000 7ff720cfe700 1 -- 172.21.15.32:0/829074613 >> v1:172.21.15.61:6789/0 conn(0x7ff71c05f5c0 legacy=0x7ff71c0b5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:56.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.401+0000 7ff720cfe700 1 -- 172.21.15.32:0/829074613 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71c05fb40 con 0x7ff71c05cd70 2020-12-05T17:41:56.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.401+0000 7ff720cfe700 1 --2- 172.21.15.32:0/829074613 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff71c05cd70 0x7ff71c143d50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7ff710002f30 tx=0x7ff710006e80).ready entity=mon.2 client_cookie=f99af248f0c93645 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:56.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.405+0000 7ff71b7fe700 1 -- 172.21.15.32:0/829074613 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7100097f0 con 0x7ff71c05cd70 2020-12-05T17:41:56.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.405+0000 7ff71b7fe700 1 -- 172.21.15.32:0/829074613 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff710009950 con 0x7ff71c05cd70 2020-12-05T17:41:56.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.405+0000 7ff71b7fe700 1 -- 172.21.15.32:0/829074613 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff710009cb0 con 0x7ff71c05cd70 2020-12-05T17:41:56.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.405+0000 7ff722f62700 1 -- 172.21.15.32:0/829074613 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff71c05cd70 msgr2=0x7ff71c143d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:56.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.405+0000 7ff722f62700 1 --2- 172.21.15.32:0/829074613 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff71c05cd70 0x7ff71c143d50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7ff710002f30 tx=0x7ff710006e80).stop 2020-12-05T17:41:56.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.405+0000 7ff722f62700 1 -- 172.21.15.32:0/829074613 shutdown_connections 2020-12-05T17:41:56.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.405+0000 7ff722f62700 1 --2- 172.21.15.32:0/829074613 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff71c05cd70 0x7ff71c143d50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:56.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.405+0000 7ff722f62700 1 --2- 172.21.15.32:0/829074613 >> v2:172.21.15.61:3300/0 conn(0x7ff71c063040 0x7ff71c0af470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:56.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.405+0000 7ff722f62700 1 -- 172.21.15.32:0/829074613 >> 172.21.15.32:0/829074613 conn(0x7ff71c069cb0 msgr2=0x7ff71c0604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:56.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.405+0000 7ff722f62700 1 -- 172.21.15.32:0/829074613 shutdown_connections 2020-12-05T17:41:56.411 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.405+0000 7ff722f62700 1 -- 172.21.15.32:0/829074613 wait complete. 2020-12-05T17:41:56.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.405+0000 7ff722f62700 1 Processor -- start 2020-12-05T17:41:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.405+0000 7ff722f62700 1 -- start start 2020-12-05T17:41:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.409+0000 7ff722f62700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff71c05cd70 0x7ff71c0b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.409+0000 7ff722f62700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff71c05f5c0 0x7ff71c147710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.409+0000 7ff722f62700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff71c063040 0x7ff71c0b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.409+0000 7ff722f62700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c142d80 con 0x7ff71c063040 2020-12-05T17:41:56.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.409+0000 7ff722f62700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c0b2450 con 0x7ff71c05cd70 2020-12-05T17:41:56.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.409+0000 7ff722f62700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c0b6650 con 0x7ff71c05f5c0 2020-12-05T17:41:56.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.409+0000 7ff7214ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff71c063040 0x7ff71c0b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:56.416 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.409+0000 7ff7214ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff71c063040 0x7ff71c0b3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41538/0 (socket says 172.21.15.32:41538) 2020-12-05T17:41:56.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.409+0000 7ff7214ff700 1 -- 172.21.15.32:0/2729731966 learned_addr learned my addr 172.21.15.32:0/2729731966 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:56.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.409+0000 7ff71bfff700 1 --2- 172.21.15.32:0/2729731966 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff71c05f5c0 0x7ff71c147710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:56.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.409+0000 7ff720cfe700 1 --2- 172.21.15.32:0/2729731966 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff71c05cd70 0x7ff71c0b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:56.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.409+0000 7ff7214ff700 1 -- 172.21.15.32:0/2729731966 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff71c05f5c0 msgr2=0x7ff71c147710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:56.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.409+0000 7ff7214ff700 1 --2- 172.21.15.32:0/2729731966 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff71c05f5c0 0x7ff71c147710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:56.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.409+0000 7ff7214ff700 1 -- 172.21.15.32:0/2729731966 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff71c05cd70 msgr2=0x7ff71c0b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:56.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.409+0000 7ff7214ff700 1 --2- 172.21.15.32:0/2729731966 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff71c05cd70 0x7ff71c0b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:56.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.409+0000 7ff7214ff700 1 -- 172.21.15.32:0/2729731966 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71c0b69b0 con 0x7ff71c063040 2020-12-05T17:41:56.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.409+0000 7ff71bfff700 1 --2- 172.21.15.32:0/2729731966 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff71c05f5c0 0x7ff71c147710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:56.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.409+0000 7ff7214ff700 1 --2- 172.21.15.32:0/2729731966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff71c063040 0x7ff71c0b3c00 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0x7ff71c0b36b0 tx=0x7ff714007ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:56.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.409+0000 7ff719ffb700 1 -- 172.21.15.32:0/2729731966 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff71400ce20 con 0x7ff71c063040 2020-12-05T17:41:56.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.409+0000 7ff719ffb700 1 -- 172.21.15.32:0/2729731966 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff714005ab0 con 0x7ff71c063040 2020-12-05T17:41:56.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.409+0000 7ff722f62700 1 -- 172.21.15.32:0/2729731966 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff71c1435b0 con 0x7ff71c063040 2020-12-05T17:41:56.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.409+0000 7ff719ffb700 1 -- 172.21.15.32:0/2729731966 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff714005e10 con 0x7ff71c063040 2020-12-05T17:41:56.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.409+0000 7ff722f62700 1 -- 172.21.15.32:0/2729731966 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff71c0b40f0 con 0x7ff71c063040 2020-12-05T17:41:56.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.413+0000 7ff722f62700 1 -- 172.21.15.32:0/2729731966 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff71c0b64b0 con 0x7ff71c063040 2020-12-05T17:41:56.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.413+0000 7ff719ffb700 1 -- 172.21.15.32:0/2729731966 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff714010030 con 0x7ff71c063040 2020-12-05T17:41:56.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.413+0000 7ff719ffb700 1 --2- 172.21.15.32:0/2729731966 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff7040553b0 0x7ff704057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:56.420 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.413+0000 7ff719ffb700 1 -- 172.21.15.32:0/2729731966 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff71406dfe0 con 0x7ff71c063040 2020-12-05T17:41:56.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.413+0000 7ff720cfe700 1 --2- 172.21.15.32:0/2729731966 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff7040553b0 0x7ff704057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:56.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.413+0000 7ff720cfe700 1 --2- 172.21.15.32:0/2729731966 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff7040553b0 0x7ff704057870 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7ff71001c000 tx=0x7ff710015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:56.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.417+0000 7ff719ffb700 1 -- 172.21.15.32:0/2729731966 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff71c0b64b0 con 0x7ff71c063040 2020-12-05T17:41:56.605 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:56.601+0000 7ff722f62700 1 -- 172.21.15.32:0/2729731966 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff71c0b64b0 con 0x7ff71c063040 2020-12-05T17:41:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.213+0000 7ff719ffb700 1 -- 172.21.15.32:0/2729731966 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v73) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff71c0b64b0 con 0x7ff71c063040 2020-12-05T17:41:57.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.217+0000 7ff722f62700 1 -- 172.21.15.32:0/2729731966 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff7040553b0 msgr2=0x7ff704057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:57.221 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.217+0000 7ff722f62700 1 --2- 172.21.15.32:0/2729731966 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff7040553b0 0x7ff704057870 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7ff71001c000 tx=0x7ff710015000).stop 2020-12-05T17:41:57.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.217+0000 7ff722f62700 1 -- 172.21.15.32:0/2729731966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff71c063040 msgr2=0x7ff71c0b3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:57.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.217+0000 7ff722f62700 1 --2- 172.21.15.32:0/2729731966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff71c063040 0x7ff71c0b3c00 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0x7ff71c0b36b0 tx=0x7ff714007ee0).stop 2020-12-05T17:41:57.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.217+0000 7ff722f62700 1 -- 172.21.15.32:0/2729731966 shutdown_connections 2020-12-05T17:41:57.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.217+0000 7ff722f62700 1 --2- 172.21.15.32:0/2729731966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff71c063040 0x7ff71c0b3c00 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:57.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.217+0000 7ff722f62700 1 --2- 172.21.15.32:0/2729731966 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff71c05cd70 0x7ff71c0b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:57.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.217+0000 7ff722f62700 1 --2- 172.21.15.32:0/2729731966 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ff7040553b0 0x7ff704057870 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:57.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.217+0000 7ff722f62700 1 --2- 172.21.15.32:0/2729731966 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff71c05f5c0 0x7ff71c147710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:57.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.217+0000 7ff722f62700 1 -- 172.21.15.32:0/2729731966 >> 172.21.15.32:0/2729731966 conn(0x7ff71c069cb0 msgr2=0x7ff71c0af390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:57.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.221+0000 7ff722f62700 1 -- 172.21.15.32:0/2729731966 shutdown_connections 2020-12-05T17:41:57.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.221+0000 7ff722f62700 1 -- 172.21.15.32:0/2729731966 wait complete. 2020-12-05T17:41:57.239 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:41:57.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cdb59e700 1 Processor -- start 2020-12-05T17:41:57.354 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cdb59e700 1 -- start start 2020-12-05T17:41:57.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cdb59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5cdc062400 0x7f5cdc05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:57.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cdb59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cdc05feb0 0x7f5cdc0b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:57.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cdb59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cdc142450 con 0x7f5cdc05f3b0 2020-12-05T17:41:57.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cdb59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cdc0b42f0 con 0x7f5cdc062400 2020-12-05T17:41:57.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cdb59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc061780 con 0x7f5cdc05feb0 2020-12-05T17:41:57.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cd9d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5cdc062400 0x7f5cdc05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:57.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cdad9d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cdc05feb0 0x7f5cdc0b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:57.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cd9d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5cdc062400 0x7f5cdc05f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41546/0 (socket says 172.21.15.32:41546) 2020-12-05T17:41:57.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cd9d9b700 1 -- 172.21.15.32:0/2096102175 learned_addr learned my addr 172.21.15.32:0/2096102175 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:57.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cda59c700 1 --1- 172.21.15.32:0/2096102175 >> v1:172.21.15.61:6789/0 conn(0x7f5cdc05f3b0 0x7f5cdc0b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:56830/0 (socket says 172.21.15.32:56830) 2020-12-05T17:41:57.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cd9d9b700 1 -- 172.21.15.32:0/2096102175 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cdc05feb0 msgr2=0x7f5cdc0b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:57.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cd9d9b700 1 --2- 172.21.15.32:0/2096102175 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cdc05feb0 0x7f5cdc0b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:57.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cd9d9b700 1 -- 172.21.15.32:0/2096102175 >> v1:172.21.15.61:6789/0 conn(0x7f5cdc05f3b0 legacy=0x7f5cdc0b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:57.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cd9d9b700 1 -- 172.21.15.32:0/2096102175 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cdc061a20 con 0x7f5cdc062400 2020-12-05T17:41:57.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cd9d9b700 1 --2- 172.21.15.32:0/2096102175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cdc062400 0x7f5cdc05f790 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7f5cd0002f30 tx=0x7f5cd0006e80).ready entity=mon.0 client_cookie=29757419165d080d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:57.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cd959a700 1 -- 172.21.15.32:0/2096102175 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cd00097f0 con 0x7f5cdc062400 2020-12-05T17:41:57.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cd959a700 1 -- 172.21.15.32:0/2096102175 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd0009950 con 0x7f5cdc062400 2020-12-05T17:41:57.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cd959a700 1 -- 172.21.15.32:0/2096102175 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cd0009cb0 con 0x7f5cdc062400 2020-12-05T17:41:57.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cdb59e700 1 -- 172.21.15.32:0/2096102175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cdc062400 msgr2=0x7f5cdc05f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:57.360 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cdb59e700 1 --2- 172.21.15.32:0/2096102175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cdc062400 0x7f5cdc05f790 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7f5cd0002f30 tx=0x7f5cd0006e80).stop 2020-12-05T17:41:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cdb59e700 1 -- 172.21.15.32:0/2096102175 shutdown_connections 2020-12-05T17:41:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cdb59e700 1 --2- 172.21.15.32:0/2096102175 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cdc05feb0 0x7f5cdc0b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cdb59e700 1 --2- 172.21.15.32:0/2096102175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cdc062400 0x7f5cdc05f790 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:57.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cdb59e700 1 -- 172.21.15.32:0/2096102175 >> 172.21.15.32:0/2096102175 conn(0x7f5cdc0b8ee0 msgr2=0x7f5cdc0b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:57.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cdb59e700 1 -- 172.21.15.32:0/2096102175 shutdown_connections 2020-12-05T17:41:57.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.353+0000 7f5cdb59e700 1 -- 172.21.15.32:0/2096102175 wait complete. 2020-12-05T17:41:57.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5cdb59e700 1 Processor -- start 2020-12-05T17:41:57.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5cdb59e700 1 -- start start 2020-12-05T17:41:57.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5cdb59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cdc05f3b0 0x7f5cdc144820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:57.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5cdb59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cdc05feb0 0x7f5cdc13e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:57.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5cdb59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cdc062400 0x7f5cdc063350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:57.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5cdb59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc145740 con 0x7f5cdc05feb0 2020-12-05T17:41:57.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5cdb59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc143c00 con 0x7f5cdc062400 2020-12-05T17:41:57.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5cdb59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc145920 con 0x7f5cdc05f3b0 2020-12-05T17:41:57.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5cd9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cdc05feb0 0x7f5cdc13e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:57.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5cd9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cdc05feb0 0x7f5cdc13e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41548/0 (socket says 172.21.15.32:41548) 2020-12-05T17:41:57.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5cd9d9b700 1 -- 172.21.15.32:0/2062668973 learned_addr learned my addr 172.21.15.32:0/2062668973 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:57.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5cda59c700 1 --2- 172.21.15.32:0/2062668973 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cdc05f3b0 0x7f5cdc144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:57.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5cdad9d700 1 --2- 172.21.15.32:0/2062668973 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cdc062400 0x7f5cdc063350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:57.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5cd9d9b700 1 -- 172.21.15.32:0/2062668973 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cdc05f3b0 msgr2=0x7f5cdc144820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:57.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5cd9d9b700 1 --2- 172.21.15.32:0/2062668973 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cdc05f3b0 0x7f5cdc144820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:57.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5cd9d9b700 1 -- 172.21.15.32:0/2062668973 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cdc062400 msgr2=0x7f5cdc063350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:57.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5cd9d9b700 1 --2- 172.21.15.32:0/2062668973 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cdc062400 0x7f5cdc063350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:57.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5cd9d9b700 1 -- 172.21.15.32:0/2062668973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cdc145bc0 con 0x7f5cdc05feb0 2020-12-05T17:41:57.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5cd9d9b700 1 --2- 172.21.15.32:0/2062668973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cdc05feb0 0x7f5cdc13e640 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7f5cd0002f30 tx=0x7f5cd0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:57.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5ccb7fe700 1 -- 172.21.15.32:0/2062668973 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cd0011b40 con 0x7f5cdc05feb0 2020-12-05T17:41:57.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5ccb7fe700 1 -- 172.21.15.32:0/2062668973 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd0011ca0 con 0x7f5cdc05feb0 2020-12-05T17:41:57.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5cdb59e700 1 -- 172.21.15.32:0/2062668973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cdc144060 con 0x7f5cdc05feb0 2020-12-05T17:41:57.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5ccb7fe700 1 -- 172.21.15.32:0/2062668973 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cd001e460 con 0x7f5cdc05feb0 2020-12-05T17:41:57.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.357+0000 7f5cdb59e700 1 -- 172.21.15.32:0/2062668973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cdc144570 con 0x7f5cdc05feb0 2020-12-05T17:41:57.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.361+0000 7f5cdb59e700 1 -- 172.21.15.32:0/2062668973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cdc057050 con 0x7f5cdc05feb0 2020-12-05T17:41:57.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.361+0000 7f5ccb7fe700 1 -- 172.21.15.32:0/2062668973 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5cd0025030 con 0x7f5cdc05feb0 2020-12-05T17:41:57.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.361+0000 7f5ccb7fe700 1 --2- 172.21.15.32:0/2062668973 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5cb80553f0 0x7f5cb80578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:57.370 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.361+0000 7f5ccb7fe700 1 -- 172.21.15.32:0/2062668973 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5cd006f390 con 0x7f5cdc05feb0 2020-12-05T17:41:57.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.361+0000 7f5cda59c700 1 --2- 172.21.15.32:0/2062668973 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5cb80553f0 0x7f5cb80578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:57.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.361+0000 7f5cda59c700 1 --2- 172.21.15.32:0/2062668973 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5cb80553f0 0x7f5cb80578b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f5ccc00d000 tx=0x7f5ccc007b10).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:57.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.365+0000 7f5ccb7fe700 1 -- 172.21.15.32:0/2062668973 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5cd0045530 con 0x7f5cdc05feb0 2020-12-05T17:41:57.556 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.553+0000 7f5cdb59e700 1 -- 172.21.15.32:0/2062668973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5cdc143d70 con 0x7f5cdc05feb0 2020-12-05T17:41:57.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.553+0000 7f5ccb7fe700 1 -- 172.21.15.32:0/2062668973 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f5cd0023020 con 0x7f5cdc05feb0 2020-12-05T17:41:57.557 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:57.557 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":73,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8473,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/711010965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":711010965},{"type":"v1","addr":"172.21.15.61:6833","nonce":711010965}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":70}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:41:55.663831+0000","modified":"2020-12-05T17:41:57.203619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8580":{"gid":8580,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1575496107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1575496107},{"type":"v1","addr":"172.21.15.32:6835","nonce":1575496107}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2020-12-05T17:41:57.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.557+0000 7f5cdb59e700 1 -- 172.21.15.32:0/2062668973 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5cb80553f0 msgr2=0x7f5cb80578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:57.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.557+0000 7f5cdb59e700 1 --2- 172.21.15.32:0/2062668973 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5cb80553f0 0x7f5cb80578b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f5ccc00d000 tx=0x7f5ccc007b10).stop 2020-12-05T17:41:57.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.557+0000 7f5cdb59e700 1 -- 172.21.15.32:0/2062668973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cdc05feb0 msgr2=0x7f5cdc13e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:57.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.557+0000 7f5cdb59e700 1 --2- 172.21.15.32:0/2062668973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cdc05feb0 0x7f5cdc13e640 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7f5cd0002f30 tx=0x7f5cd0015000).stop 2020-12-05T17:41:57.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.557+0000 7f5cdb59e700 1 -- 172.21.15.32:0/2062668973 shutdown_connections 2020-12-05T17:41:57.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.557+0000 7f5cdb59e700 1 --2- 172.21.15.32:0/2062668973 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cdc062400 0x7f5cdc063350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:57.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.557+0000 7f5cdb59e700 1 --2- 172.21.15.32:0/2062668973 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5cb80553f0 0x7f5cb80578b0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.557+0000 7f5cdb59e700 1 --2- 172.21.15.32:0/2062668973 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cdc05f3b0 0x7f5cdc144820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.557+0000 7f5cdb59e700 1 --2- 172.21.15.32:0/2062668973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cdc05feb0 0x7f5cdc13e640 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.557+0000 7f5cdb59e700 1 -- 172.21.15.32:0/2062668973 >> 172.21.15.32:0/2062668973 conn(0x7f5cdc0b8ee0 msgr2=0x7f5cdc140ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.557+0000 7f5cdb59e700 1 -- 172.21.15.32:0/2062668973 shutdown_connections 2020-12-05T17:41:57.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.557+0000 7f5cdb59e700 1 -- 172.21.15.32:0/2062668973 wait complete. 2020-12-05T17:41:57.562 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 73 2020-12-05T17:41:57.571 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:41:57.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.693+0000 7fcb3f59e700 1 Processor -- start 2020-12-05T17:41:57.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.693+0000 7fcb3f59e700 1 -- start start 2020-12-05T17:41:57.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb400b5650 0x7fcb4014a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:57.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb4014ac60 0x7fcb40146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:57.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3f59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb400b5a30 con 0x7fcb400afd20 2020-12-05T17:41:57.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3f59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb40067990 con 0x7fcb4014ac60 2020-12-05T17:41:57.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb400aea20 con 0x7fcb400b5650 2020-12-05T17:41:57.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb4014ac60 0x7fcb40146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:57.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb4014ac60 0x7fcb40146d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41560/0 (socket says 172.21.15.32:41560) 2020-12-05T17:41:57.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3ed9d700 1 -- 172.21.15.32:0/2133002265 learned_addr learned my addr 172.21.15.32:0/2133002265 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:57.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3e59c700 1 --2- 172.21.15.32:0/2133002265 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb400b5650 0x7fcb4014a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:57.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3ed9d700 1 -- 172.21.15.32:0/2133002265 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb400b5650 msgr2=0x7fcb4014a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:57.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3ed9d700 1 --2- 172.21.15.32:0/2133002265 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb400b5650 0x7fcb4014a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:57.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3ed9d700 1 -- 172.21.15.32:0/2133002265 >> v1:172.21.15.61:6789/0 conn(0x7fcb400afd20 legacy=0x7fcb4014a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:57.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3ed9d700 1 -- 172.21.15.32:0/2133002265 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb400aecc0 con 0x7fcb4014ac60 2020-12-05T17:41:57.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3ed9d700 1 --2- 172.21.15.32:0/2133002265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4014ac60 0x7fcb40146d80 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 rx=0x7fcb380032b0 tx=0x7fcb38009270).ready entity=mon.0 client_cookie=403bba6a21a96920 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:57.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3dd9b700 1 -- 172.21.15.32:0/2133002265 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb3800bc00 con 0x7fcb4014ac60 2020-12-05T17:41:57.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3dd9b700 1 -- 172.21.15.32:0/2133002265 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3800bd60 con 0x7fcb4014ac60 2020-12-05T17:41:57.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3dd9b700 1 -- 172.21.15.32:0/2133002265 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb38011400 con 0x7fcb4014ac60 2020-12-05T17:41:57.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3f59e700 1 -- 172.21.15.32:0/2133002265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4014ac60 msgr2=0x7fcb40146d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:57.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3f59e700 1 --2- 172.21.15.32:0/2133002265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4014ac60 0x7fcb40146d80 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 rx=0x7fcb380032b0 tx=0x7fcb38009270).stop 2020-12-05T17:41:57.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3f59e700 1 -- 172.21.15.32:0/2133002265 shutdown_connections 2020-12-05T17:41:57.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3f59e700 1 --2- 172.21.15.32:0/2133002265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4014ac60 0x7fcb40146d80 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:57.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3f59e700 1 --2- 172.21.15.32:0/2133002265 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb400b5650 0x7fcb4014a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3f59e700 1 -- 172.21.15.32:0/2133002265 >> 172.21.15.32:0/2133002265 conn(0x7fcb4005bbb0 msgr2=0x7fcb40062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3f59e700 1 -- 172.21.15.32:0/2133002265 shutdown_connections 2020-12-05T17:41:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.697+0000 7fcb3f59e700 1 -- 172.21.15.32:0/2133002265 wait complete. 2020-12-05T17:41:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.701+0000 7fcb3f59e700 1 Processor -- start 2020-12-05T17:41:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.701+0000 7fcb3f59e700 1 -- start start 2020-12-05T17:41:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.701+0000 7fcb3f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb400afd20 0x7fcb40064580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.701+0000 7fcb3f59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb400b5650 0x7fcb40066260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.701+0000 7fcb3f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4014ac60 0x7fcb40066750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.701+0000 7fcb3f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb401430f0 con 0x7fcb4014ac60 2020-12-05T17:41:57.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.701+0000 7fcb3f59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb40143320 con 0x7fcb400b5650 2020-12-05T17:41:57.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.701+0000 7fcb3f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb40064dd0 con 0x7fcb400afd20 2020-12-05T17:41:57.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.701+0000 7fcb3ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4014ac60 0x7fcb40066750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:57.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.701+0000 7fcb3ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4014ac60 0x7fcb40066750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41564/0 (socket says 172.21.15.32:41564) 2020-12-05T17:41:57.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.701+0000 7fcb3ed9d700 1 -- 172.21.15.32:0/2955801008 learned_addr learned my addr 172.21.15.32:0/2955801008 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:57.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.701+0000 7fcb3e59c700 1 --2- 172.21.15.32:0/2955801008 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb400afd20 0x7fcb40064580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:57.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.701+0000 7fcb37fff700 1 --2- 172.21.15.32:0/2955801008 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb400b5650 0x7fcb40066260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:57.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.705+0000 7fcb3ed9d700 1 -- 172.21.15.32:0/2955801008 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb400afd20 msgr2=0x7fcb40064580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:57.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.705+0000 7fcb3ed9d700 1 --2- 172.21.15.32:0/2955801008 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb400afd20 0x7fcb40064580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:57.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.705+0000 7fcb3ed9d700 1 -- 172.21.15.32:0/2955801008 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb400b5650 msgr2=0x7fcb40066260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:57.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.705+0000 7fcb3ed9d700 1 --2- 172.21.15.32:0/2955801008 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb400b5650 0x7fcb40066260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:57.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.705+0000 7fcb3ed9d700 1 -- 172.21.15.32:0/2955801008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb401568a0 con 0x7fcb4014ac60 2020-12-05T17:41:57.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.705+0000 7fcb37fff700 1 --2- 172.21.15.32:0/2955801008 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb400b5650 0x7fcb40066260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:57.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.705+0000 7fcb3ed9d700 1 --2- 172.21.15.32:0/2955801008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4014ac60 0x7fcb40066750 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0x7fcb380032b0 tx=0x7fcb38016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:57.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.705+0000 7fcb377fe700 1 -- 172.21.15.32:0/2955801008 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb3801f5e0 con 0x7fcb4014ac60 2020-12-05T17:41:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.705+0000 7fcb377fe700 1 -- 172.21.15.32:0/2955801008 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3801f740 con 0x7fcb4014ac60 2020-12-05T17:41:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.705+0000 7fcb3f59e700 1 -- 172.21.15.32:0/2955801008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb40064a70 con 0x7fcb4014ac60 2020-12-05T17:41:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.705+0000 7fcb377fe700 1 -- 172.21.15.32:0/2955801008 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb3801faa0 con 0x7fcb4014ac60 2020-12-05T17:41:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.705+0000 7fcb3f59e700 1 -- 172.21.15.32:0/2955801008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb40143640 con 0x7fcb4014ac60 2020-12-05T17:41:57.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.705+0000 7fcb3f59e700 1 -- 172.21.15.32:0/2955801008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb40156100 con 0x7fcb4014ac60 2020-12-05T17:41:57.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.705+0000 7fcb377fe700 1 -- 172.21.15.32:0/2955801008 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fcb38026030 con 0x7fcb4014ac60 2020-12-05T17:41:57.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.709+0000 7fcb377fe700 1 --2- 172.21.15.32:0/2955801008 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcb24055330 0x7fcb240577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:57.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.709+0000 7fcb377fe700 1 -- 172.21.15.32:0/2955801008 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcb3806f150 con 0x7fcb4014ac60 2020-12-05T17:41:57.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.709+0000 7fcb3e59c700 1 --2- 172.21.15.32:0/2955801008 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcb24055330 0x7fcb240577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:57.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.709+0000 7fcb3e59c700 1 --2- 172.21.15.32:0/2955801008 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcb24055330 0x7fcb240577f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7fcb3000c000 tx=0x7fcb30008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:57.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.713+0000 7fcb377fe700 1 -- 172.21.15.32:0/2955801008 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fcb380452f0 con 0x7fcb4014ac60 2020-12-05T17:41:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.861+0000 7fcb3f59e700 1 -- 172.21.15.32:0/2955801008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb40057050 con 0x7fcb4014ac60 2020-12-05T17:41:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.861+0000 7fcb377fe700 1 -- 172.21.15.32:0/2955801008 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fcb38015090 con 0x7fcb4014ac60 2020-12-05T17:41:57.865 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:57.866 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":148,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:41:56.666368+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:41:53.238803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2020-12-05T17:41:54.541493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:41:57.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.865+0000 7fcb3f59e700 1 -- 172.21.15.32:0/2955801008 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcb24055330 msgr2=0x7fcb240577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:57.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.865+0000 7fcb3f59e700 1 --2- 172.21.15.32:0/2955801008 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcb24055330 0x7fcb240577f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7fcb3000c000 tx=0x7fcb30008040).stop 2020-12-05T17:41:57.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.865+0000 7fcb3f59e700 1 -- 172.21.15.32:0/2955801008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4014ac60 msgr2=0x7fcb40066750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:57.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.865+0000 7fcb3f59e700 1 --2- 172.21.15.32:0/2955801008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4014ac60 0x7fcb40066750 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0x7fcb380032b0 tx=0x7fcb38016000).stop 2020-12-05T17:41:57.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.865+0000 7fcb3f59e700 1 -- 172.21.15.32:0/2955801008 shutdown_connections 2020-12-05T17:41:57.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.865+0000 7fcb3f59e700 1 --2- 172.21.15.32:0/2955801008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4014ac60 0x7fcb40066750 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:57.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.865+0000 7fcb3f59e700 1 --2- 172.21.15.32:0/2955801008 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcb24055330 0x7fcb240577f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:57.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.865+0000 7fcb3f59e700 1 --2- 172.21.15.32:0/2955801008 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb400afd20 0x7fcb40064580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:57.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.865+0000 7fcb3f59e700 1 --2- 172.21.15.32:0/2955801008 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb400b5650 0x7fcb40066260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:57.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.865+0000 7fcb3f59e700 1 -- 172.21.15.32:0/2955801008 >> 172.21.15.32:0/2955801008 conn(0x7fcb4005bbb0 msgr2=0x7fcb40146f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:57.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.865+0000 7fcb3f59e700 1 -- 172.21.15.32:0/2955801008 shutdown_connections 2020-12-05T17:41:57.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:57.865+0000 7fcb3f59e700 1 -- 172.21.15.32:0/2955801008 wait complete. 2020-12-05T17:41:57.879 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:41:58.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.001+0000 7fe6da3ce700 1 Processor -- start 2020-12-05T17:41:58.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.001+0000 7fe6da3ce700 1 -- start start 2020-12-05T17:41:58.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.001+0000 7fe6da3ce700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fe6d405f6f0 0x7fe6d40adef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:58.006 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.001+0000 7fe6da3ce700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe6d4061780 0x7fe6d414e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:58.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.001+0000 7fe6da3ce700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6d40b0380 con 0x7fe6d405feb0 2020-12-05T17:41:58.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.001+0000 7fe6da3ce700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6d40aeaf0 con 0x7fe6d4061780 2020-12-05T17:41:58.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.001+0000 7fe6da3ce700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6d40aef70 con 0x7fe6d405f6f0 2020-12-05T17:41:58.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.001+0000 7fe6d916c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe6d4061780 0x7fe6d414e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:58.007 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.001+0000 7fe6d916c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe6d4061780 0x7fe6d414e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41572/0 (socket says 172.21.15.32:41572) 2020-12-05T17:41:58.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.001+0000 7fe6d896b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fe6d405f6f0 0x7fe6d40adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:58.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.001+0000 7fe6d916c700 1 -- 172.21.15.32:0/4257859878 learned_addr learned my addr 172.21.15.32:0/4257859878 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:58.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.001+0000 7fe6d916c700 1 -- 172.21.15.32:0/4257859878 >> v2:172.21.15.61:3300/0 conn(0x7fe6d405f6f0 msgr2=0x7fe6d40adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:58.008 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.001+0000 7fe6d916c700 1 --2- 172.21.15.32:0/4257859878 >> v2:172.21.15.61:3300/0 conn(0x7fe6d405f6f0 0x7fe6d40adef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.001+0000 7fe6d916c700 1 -- 172.21.15.32:0/4257859878 >> v1:172.21.15.61:6789/0 conn(0x7fe6d405feb0 legacy=0x7fe6d40ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:58.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.001+0000 7fe6d916c700 1 -- 172.21.15.32:0/4257859878 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d405fad0 con 0x7fe6d4061780 2020-12-05T17:41:58.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.001+0000 7fe6d916c700 1 --2- 172.21.15.32:0/4257859878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6d4061780 0x7fe6d414e240 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7fe6c4002f30 tx=0x7fe6c4006e80).ready entity=mon.0 client_cookie=76ebc260049f11fa server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:58.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.001+0000 7fe6d37fe700 1 -- 172.21.15.32:0/4257859878 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6c40097f0 con 0x7fe6d4061780 2020-12-05T17:41:58.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.001+0000 7fe6d37fe700 1 -- 172.21.15.32:0/4257859878 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c4009950 con 0x7fe6d4061780 2020-12-05T17:41:58.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.001+0000 7fe6d37fe700 1 -- 172.21.15.32:0/4257859878 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6c4009cb0 con 0x7fe6d4061780 2020-12-05T17:41:58.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.001+0000 7fe6da3ce700 1 -- 172.21.15.32:0/4257859878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6d4061780 msgr2=0x7fe6d414e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:58.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.001+0000 7fe6da3ce700 1 --2- 172.21.15.32:0/4257859878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6d4061780 0x7fe6d414e240 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7fe6c4002f30 tx=0x7fe6c4006e80).stop 2020-12-05T17:41:58.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.005+0000 7fe6da3ce700 1 -- 172.21.15.32:0/4257859878 shutdown_connections 2020-12-05T17:41:58.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.005+0000 7fe6da3ce700 1 --2- 172.21.15.32:0/4257859878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6d4061780 0x7fe6d414e240 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.005+0000 7fe6da3ce700 1 --2- 172.21.15.32:0/4257859878 >> v2:172.21.15.61:3300/0 conn(0x7fe6d405f6f0 0x7fe6d40adef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.005+0000 7fe6da3ce700 1 -- 172.21.15.32:0/4257859878 >> 172.21.15.32:0/4257859878 conn(0x7fe6d40b4b10 msgr2=0x7fe6d40b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:58.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.005+0000 7fe6da3ce700 1 -- 172.21.15.32:0/4257859878 shutdown_connections 2020-12-05T17:41:58.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.005+0000 7fe6da3ce700 1 -- 172.21.15.32:0/4257859878 wait complete. 2020-12-05T17:41:58.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.005+0000 7fe6da3ce700 1 Processor -- start 2020-12-05T17:41:58.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.005+0000 7fe6da3ce700 1 -- start start 2020-12-05T17:41:58.014 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.005+0000 7fe6da3ce700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe6d405f6f0 0x7fe6d412da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:58.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.005+0000 7fe6da3ce700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe6d405feb0 0x7fe6d4063fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:58.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.005+0000 7fe6da3ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6d4061780 0x7fe6d4070080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:58.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.005+0000 7fe6da3ce700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d40650d0 con 0x7fe6d4061780 2020-12-05T17:41:58.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.005+0000 7fe6da3ce700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d4070570 con 0x7fe6d405feb0 2020-12-05T17:41:58.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.005+0000 7fe6da3ce700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d4070790 con 0x7fe6d405f6f0 2020-12-05T17:41:58.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.005+0000 7fe6d916c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6d4061780 0x7fe6d4070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:58.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.005+0000 7fe6d916c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6d4061780 0x7fe6d4070080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41576/0 (socket says 172.21.15.32:41576) 2020-12-05T17:41:58.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.005+0000 7fe6d916c700 1 -- 172.21.15.32:0/3529206918 learned_addr learned my addr 172.21.15.32:0/3529206918 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:58.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.005+0000 7fe6d3fff700 1 --2- 172.21.15.32:0/3529206918 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe6d405feb0 0x7fe6d4063fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:58.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.005+0000 7fe6d896b700 1 --2- 172.21.15.32:0/3529206918 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe6d405f6f0 0x7fe6d412da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:58.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.005+0000 7fe6d916c700 1 -- 172.21.15.32:0/3529206918 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe6d405f6f0 msgr2=0x7fe6d412da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:58.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.009+0000 7fe6d916c700 1 --2- 172.21.15.32:0/3529206918 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe6d405f6f0 0x7fe6d412da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.009+0000 7fe6d916c700 1 -- 172.21.15.32:0/3529206918 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe6d405feb0 msgr2=0x7fe6d4063fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:58.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.009+0000 7fe6d916c700 1 --2- 172.21.15.32:0/3529206918 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe6d405feb0 0x7fe6d4063fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.009+0000 7fe6d916c700 1 -- 172.21.15.32:0/3529206918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d4070a30 con 0x7fe6d4061780 2020-12-05T17:41:58.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.009+0000 7fe6d896b700 1 --2- 172.21.15.32:0/3529206918 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe6d405f6f0 0x7fe6d412da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:58.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.009+0000 7fe6d916c700 1 --2- 172.21.15.32:0/3529206918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6d4061780 0x7fe6d4070080 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 rx=0x7fe6c4002f30 tx=0x7fe6c4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:58.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.009+0000 7fe6d17fa700 1 -- 172.21.15.32:0/3529206918 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6c4011c30 con 0x7fe6d4061780 2020-12-05T17:41:58.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.009+0000 7fe6d17fa700 1 -- 172.21.15.32:0/3529206918 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c4011d90 con 0x7fe6d4061780 2020-12-05T17:41:58.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.009+0000 7fe6da3ce700 1 -- 172.21.15.32:0/3529206918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d4071f40 con 0x7fe6d4061780 2020-12-05T17:41:58.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.009+0000 7fe6d17fa700 1 -- 172.21.15.32:0/3529206918 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6c401e650 con 0x7fe6d4061780 2020-12-05T17:41:58.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.009+0000 7fe6da3ce700 1 -- 172.21.15.32:0/3529206918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d4066cf0 con 0x7fe6d4061780 2020-12-05T17:41:58.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.009+0000 7fe6da3ce700 1 -- 172.21.15.32:0/3529206918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6d40720d0 con 0x7fe6d4061780 2020-12-05T17:41:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.009+0000 7fe6d17fa700 1 -- 172.21.15.32:0/3529206918 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe6c4025030 con 0x7fe6d4061780 2020-12-05T17:41:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.009+0000 7fe6d17fa700 1 --2- 172.21.15.32:0/3529206918 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe6b80553b0 0x7fe6b8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.009+0000 7fe6d17fa700 1 -- 172.21.15.32:0/3529206918 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe6c406f400 con 0x7fe6d4061780 2020-12-05T17:41:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.009+0000 7fe6d896b700 1 --2- 172.21.15.32:0/3529206918 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe6b80553b0 0x7fe6b8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.009+0000 7fe6d896b700 1 --2- 172.21.15.32:0/3529206918 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe6b80553b0 0x7fe6b8057870 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7fe6cc00b000 tx=0x7fe6cc0091a0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:58.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.017+0000 7fe6d17fa700 1 -- 172.21.15.32:0/3529206918 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe6c4045500 con 0x7fe6d4061780 2020-12-05T17:41:58.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.205+0000 7fe6da3ce700 1 -- 172.21.15.32:0/3529206918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fe6d4057050 con 0x7fe6d4061780 2020-12-05T17:41:58.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.213+0000 7fe6d17fa700 1 -- 172.21.15.32:0/3529206918 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v44) v1 ==== 184+0+0 (secure 0 0 0) 0x7fe6c4048b20 con 0x7fe6d4061780 2020-12-05T17:41:58.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.213+0000 7fe6da3ce700 1 -- 172.21.15.32:0/3529206918 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe6b80553b0 msgr2=0x7fe6b8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:58.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.213+0000 7fe6da3ce700 1 --2- 172.21.15.32:0/3529206918 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe6b80553b0 0x7fe6b8057870 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7fe6cc00b000 tx=0x7fe6cc0091a0).stop 2020-12-05T17:41:58.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.213+0000 7fe6da3ce700 1 -- 172.21.15.32:0/3529206918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6d4061780 msgr2=0x7fe6d4070080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:58.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.213+0000 7fe6da3ce700 1 --2- 172.21.15.32:0/3529206918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6d4061780 0x7fe6d4070080 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 rx=0x7fe6c4002f30 tx=0x7fe6c4015000).stop 2020-12-05T17:41:58.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.217+0000 7fe6da3ce700 1 -- 172.21.15.32:0/3529206918 shutdown_connections 2020-12-05T17:41:58.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.217+0000 7fe6da3ce700 1 --2- 172.21.15.32:0/3529206918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6d4061780 0x7fe6d4070080 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.217+0000 7fe6da3ce700 1 --2- 172.21.15.32:0/3529206918 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fe6b80553b0 0x7fe6b8057870 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.217+0000 7fe6da3ce700 1 --2- 172.21.15.32:0/3529206918 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe6d405f6f0 0x7fe6d412da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.217+0000 7fe6da3ce700 1 --2- 172.21.15.32:0/3529206918 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe6d405feb0 0x7fe6d4063fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.217+0000 7fe6da3ce700 1 -- 172.21.15.32:0/3529206918 >> 172.21.15.32:0/3529206918 conn(0x7fe6d40b4b10 msgr2=0x7fe6d414f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:58.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.217+0000 7fe6da3ce700 1 -- 172.21.15.32:0/3529206918 shutdown_connections 2020-12-05T17:41:58.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.217+0000 7fe6da3ce700 1 -- 172.21.15.32:0/3529206918 wait complete. 2020-12-05T17:41:58.220 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2020-12-05T17:41:58.233 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:41:58.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.353+0000 7f55a5ee2700 1 Processor -- start 2020-12-05T17:41:58.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.353+0000 7f55a5ee2700 1 -- start start 2020-12-05T17:41:58.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.357+0000 7f55a5ee2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55a0153240 0x7f55a00ae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:58.363 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.357+0000 7f55a5ee2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a0080a90 con 0x7f55a00611b0 2020-12-05T17:41:58.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.357+0000 7f55a5ee2700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a0152860 con 0x7f55a00b0240 2020-12-05T17:41:58.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.357+0000 7f55a5ee2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a0152b00 con 0x7f55a0153240 2020-12-05T17:41:58.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.357+0000 7f55a56e1700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f55a00611b0 0x7f55a014c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44230/0 (socket says 172.21.15.32:44230) 2020-12-05T17:41:58.364 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.357+0000 7f55a56e1700 1 -- 172.21.15.32:0/1879806711 learned_addr learned my addr 172.21.15.32:0/1879806711 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:58.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.357+0000 7f55a7144700 1 --2- 172.21.15.32:0/1879806711 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55a0153240 0x7f55a00ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:58.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.357+0000 7f5597fff700 1 -- 172.21.15.32:0/1879806711 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819202893 0 0) 0x7f55a0152860 con 0x7f55a00b0240 2020-12-05T17:41:58.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.357+0000 7f5597fff700 1 -- 172.21.15.32:0/1879806711 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f558c003230 con 0x7f55a00b0240 2020-12-05T17:41:58.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.357+0000 7f5597fff700 1 -- 172.21.15.32:0/1879806711 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3779693223 0 0) 0x7f55a0080a90 con 0x7f55a00611b0 2020-12-05T17:41:58.365 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.357+0000 7f5597fff700 1 -- 172.21.15.32:0/1879806711 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55a0152860 con 0x7f55a00611b0 2020-12-05T17:41:58.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.357+0000 7f55a7144700 1 -- 172.21.15.32:0/1879806711 >> v1:172.21.15.61:6789/0 conn(0x7f55a00b0240 legacy=0x7f55a014f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:58.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.357+0000 7f5597fff700 1 -- 172.21.15.32:0/1879806711 <== mon.1 v1:172.21.15.61:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3354886017 0 0) 0x7f558c003230 con 0x7f55a00b0240 2020-12-05T17:41:58.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.357+0000 7f55a7144700 1 -- 172.21.15.32:0/1879806711 >> v1:172.21.15.32:6789/0 conn(0x7f55a00611b0 legacy=0x7f55a014c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:58.366 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.357+0000 7f55a7144700 1 -- 172.21.15.32:0/1879806711 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a0152da0 con 0x7f55a0153240 2020-12-05T17:41:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.357+0000 7f55a7144700 1 --2- 172.21.15.32:0/1879806711 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55a0153240 0x7f55a00ae860 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f5590001870 tx=0x7f55980091a0).ready entity=mon.2 client_cookie=f87b82540768794c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.357+0000 7f5597fff700 1 -- 172.21.15.32:0/1879806711 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55900016d0 con 0x7f55a0153240 2020-12-05T17:41:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.357+0000 7f5597fff700 1 -- 172.21.15.32:0/1879806711 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f559800b880 con 0x7f55a0153240 2020-12-05T17:41:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.357+0000 7f5597fff700 1 -- 172.21.15.32:0/1879806711 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f559800b9f0 con 0x7f55a0153240 2020-12-05T17:41:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.357+0000 7f55a5ee2700 1 -- 172.21.15.32:0/1879806711 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55a0153240 msgr2=0x7f55a00ae860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:58.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.357+0000 7f55a5ee2700 1 --2- 172.21.15.32:0/1879806711 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55a0153240 0x7f55a00ae860 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f5590001870 tx=0x7f55980091a0).stop 2020-12-05T17:41:58.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a5ee2700 1 -- 172.21.15.32:0/1879806711 shutdown_connections 2020-12-05T17:41:58.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a5ee2700 1 --2- 172.21.15.32:0/1879806711 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55a0153240 0x7f55a00ae860 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.368 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a5ee2700 1 -- 172.21.15.32:0/1879806711 >> 172.21.15.32:0/1879806711 conn(0x7f55a00b5f90 msgr2=0x7f55a00b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:58.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a5ee2700 1 -- 172.21.15.32:0/1879806711 shutdown_connections 2020-12-05T17:41:58.369 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a5ee2700 1 -- 172.21.15.32:0/1879806711 wait complete. 2020-12-05T17:41:58.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a5ee2700 1 Processor -- start 2020-12-05T17:41:58.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a5ee2700 1 -- start start 2020-12-05T17:41:58.371 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a5ee2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55a00611b0 0x7f55a013cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:58.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a5ee2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f55a00b0240 0x7f55a013dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:58.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a5ee2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55a0153240 0x7f55a01203c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:58.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a5ee2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a013d290 con 0x7f55a0153240 2020-12-05T17:41:58.372 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a5ee2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a013d530 con 0x7f55a00b0240 2020-12-05T17:41:58.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a5ee2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a013d7d0 con 0x7f55a00611b0 2020-12-05T17:41:58.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a7144700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55a0153240 0x7f55a01203c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:58.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a7144700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55a0153240 0x7f55a01203c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41590/0 (socket says 172.21.15.32:41590) 2020-12-05T17:41:58.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a7144700 1 -- 172.21.15.32:0/1622550034 learned_addr learned my addr 172.21.15.32:0/1622550034 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:58.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a56e1700 1 --2- 172.21.15.32:0/1622550034 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55a00611b0 0x7f55a013cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:58.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a4ee0700 1 --2- 172.21.15.32:0/1622550034 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f55a00b0240 0x7f55a013dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:58.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a7144700 1 -- 172.21.15.32:0/1622550034 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55a00611b0 msgr2=0x7f55a013cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:58.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a7144700 1 --2- 172.21.15.32:0/1622550034 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55a00611b0 0x7f55a013cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a7144700 1 -- 172.21.15.32:0/1622550034 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f55a00b0240 msgr2=0x7f55a013dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:58.374 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a7144700 1 --2- 172.21.15.32:0/1622550034 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f55a00b0240 0x7f55a013dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a7144700 1 -- 172.21.15.32:0/1622550034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a01208b0 con 0x7f55a0153240 2020-12-05T17:41:58.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.361+0000 7f55a4ee0700 1 --2- 172.21.15.32:0/1622550034 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f55a00b0240 0x7f55a013dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:41:58.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.365+0000 7f55a7144700 1 --2- 172.21.15.32:0/1622550034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55a0153240 0x7f55a01203c0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7f5590001870 tx=0x7f5598016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:58.375 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.365+0000 7f5595ffb700 1 -- 172.21.15.32:0/1622550034 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5598011d80 con 0x7f55a0153240 2020-12-05T17:41:58.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.365+0000 7f5595ffb700 1 -- 172.21.15.32:0/1622550034 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5598020400 con 0x7f55a0153240 2020-12-05T17:41:58.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.365+0000 7f55a5ee2700 1 -- 172.21.15.32:0/1622550034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a0120b60 con 0x7f55a0153240 2020-12-05T17:41:58.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.365+0000 7f5595ffb700 1 -- 172.21.15.32:0/1622550034 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5598020760 con 0x7f55a0153240 2020-12-05T17:41:58.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.365+0000 7f55a5ee2700 1 -- 172.21.15.32:0/1622550034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a0121700 con 0x7f55a0153240 2020-12-05T17:41:58.376 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.365+0000 7f55a5ee2700 1 -- 172.21.15.32:0/1622550034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a013d990 con 0x7f55a0153240 2020-12-05T17:41:58.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.365+0000 7f5595ffb700 1 -- 172.21.15.32:0/1622550034 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5598027030 con 0x7f55a0153240 2020-12-05T17:41:58.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.365+0000 7f5595ffb700 1 --2- 172.21.15.32:0/1622550034 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5580055320 0x7f55800577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:58.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.365+0000 7f5595ffb700 1 -- 172.21.15.32:0/1622550034 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f559806ea90 con 0x7f55a0153240 2020-12-05T17:41:58.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.365+0000 7f55a56e1700 1 --2- 172.21.15.32:0/1622550034 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5580055320 0x7f55800577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:58.377 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.365+0000 7f55a56e1700 1 --2- 172.21.15.32:0/1622550034 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5580055320 0x7f55800577e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7f559c00d000 tx=0x7f559c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:58.378 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.373+0000 7f5595ffb700 1 -- 172.21.15.32:0/1622550034 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5598044c30 con 0x7f55a0153240 2020-12-05T17:41:58.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.565+0000 7f55a5ee2700 1 -- 172.21.15.32:0/1622550034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f55a0057050 con 0x7f55a0153240 2020-12-05T17:41:58.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.569+0000 7f5595ffb700 1 -- 172.21.15.32:0/1622550034 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v45) v1 ==== 184+0+0 (secure 0 0 0) 0x7f55980150e0 con 0x7f55a0153240 2020-12-05T17:41:58.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.573+0000 7f55a5ee2700 1 -- 172.21.15.32:0/1622550034 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5580055320 msgr2=0x7f55800577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:58.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.573+0000 7f55a5ee2700 1 --2- 172.21.15.32:0/1622550034 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5580055320 0x7f55800577e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7f559c00d000 tx=0x7f559c009040).stop 2020-12-05T17:41:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.573+0000 7f55a5ee2700 1 -- 172.21.15.32:0/1622550034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55a0153240 msgr2=0x7f55a01203c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.573+0000 7f55a5ee2700 1 --2- 172.21.15.32:0/1622550034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55a0153240 0x7f55a01203c0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7f5590001870 tx=0x7f5598016000).stop 2020-12-05T17:41:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.573+0000 7f55a5ee2700 1 -- 172.21.15.32:0/1622550034 shutdown_connections 2020-12-05T17:41:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.573+0000 7f55a5ee2700 1 --2- 172.21.15.32:0/1622550034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55a0153240 0x7f55a01203c0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.573+0000 7f55a5ee2700 1 --2- 172.21.15.32:0/1622550034 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5580055320 0x7f55800577e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.573+0000 7f55a5ee2700 1 --2- 172.21.15.32:0/1622550034 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f55a00611b0 0x7f55a013cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.573+0000 7f55a5ee2700 1 --2- 172.21.15.32:0/1622550034 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f55a00b0240 0x7f55a013dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.573+0000 7f55a5ee2700 1 -- 172.21.15.32:0/1622550034 >> 172.21.15.32:0/1622550034 conn(0x7f55a00b5f90 msgr2=0x7f55a00b5770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:58.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.573+0000 7f55a5ee2700 1 -- 172.21.15.32:0/1622550034 shutdown_connections 2020-12-05T17:41:58.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.573+0000 7f55a5ee2700 1 -- 172.21.15.32:0/1622550034 wait complete. 2020-12-05T17:41:58.578 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2020-12-05T17:41:58.591 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-12-05T17:41:58.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.713+0000 7f5486925700 1 Processor -- start 2020-12-05T17:41:58.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.713+0000 7f5486925700 1 -- start start 2020-12-05T17:41:58.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.713+0000 7f5486925700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f548005f6f0 0x7f54800adef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:58.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.713+0000 7f5486925700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f548005feb0 0x7f54800ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:58.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.713+0000 7f5486925700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54800b0380 con 0x7f5480061780 2020-12-05T17:41:58.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.713+0000 7f5486925700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54800aef70 con 0x7f548005feb0 2020-12-05T17:41:58.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.713+0000 7f5486925700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548005fad0 con 0x7f548005f6f0 2020-12-05T17:41:58.723 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.717+0000 7f5484ec2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f548005f6f0 0x7f54800adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:58.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.717+0000 7f547ffff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f548005feb0 0x7f54800ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:58.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.717+0000 7f54856c3700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f5480061780 0x7f5480152890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:56888/0 (socket says 172.21.15.32:56888) 2020-12-05T17:41:58.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.717+0000 7f54856c3700 1 -- 172.21.15.32:0/1647213304 learned_addr learned my addr 172.21.15.32:0/1647213304 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:58.724 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.717+0000 7f547ffff700 1 -- 172.21.15.32:0/1647213304 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f548005f6f0 msgr2=0x7f54800adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.717+0000 7f547ffff700 1 --2- 172.21.15.32:0/1647213304 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f548005f6f0 0x7f54800adef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.717+0000 7f547ffff700 1 -- 172.21.15.32:0/1647213304 >> v1:172.21.15.61:6789/0 conn(0x7f5480061780 legacy=0x7f5480152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.717+0000 7f547ffff700 1 -- 172.21.15.32:0/1647213304 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54800aecd0 con 0x7f548005feb0 2020-12-05T17:41:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.717+0000 7f5484ec2700 1 --2- 172.21.15.32:0/1647213304 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f548005f6f0 0x7f54800adef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.717+0000 7f547ffff700 1 --2- 172.21.15.32:0/1647213304 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f548005feb0 0x7f54800ae3e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f5474002f30 tx=0x7f5474006dc0).ready entity=mon.1 client_cookie=401b65538cc64cfe server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:58.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.717+0000 7f547f7fe700 1 -- 172.21.15.32:0/1647213304 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5474009760 con 0x7f548005feb0 2020-12-05T17:41:58.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.717+0000 7f547f7fe700 1 -- 172.21.15.32:0/1647213304 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54740098c0 con 0x7f548005feb0 2020-12-05T17:41:58.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.717+0000 7f547f7fe700 1 -- 172.21.15.32:0/1647213304 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5474009c20 con 0x7f548005feb0 2020-12-05T17:41:58.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.717+0000 7f5486925700 1 -- 172.21.15.32:0/1647213304 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f548005feb0 msgr2=0x7f54800ae3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:58.726 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.717+0000 7f5486925700 1 --2- 172.21.15.32:0/1647213304 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f548005feb0 0x7f54800ae3e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f5474002f30 tx=0x7f5474006dc0).stop 2020-12-05T17:41:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.717+0000 7f5486925700 1 -- 172.21.15.32:0/1647213304 shutdown_connections 2020-12-05T17:41:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.717+0000 7f5486925700 1 --2- 172.21.15.32:0/1647213304 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f548005f6f0 0x7f54800adef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.717+0000 7f5486925700 1 --2- 172.21.15.32:0/1647213304 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f548005feb0 0x7f54800ae3e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.717+0000 7f5486925700 1 -- 172.21.15.32:0/1647213304 >> 172.21.15.32:0/1647213304 conn(0x7f54800b4b10 msgr2=0x7f54800b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:58.728 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.717+0000 7f5486925700 1 -- 172.21.15.32:0/1647213304 shutdown_connections 2020-12-05T17:41:58.728 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.717+0000 7f5486925700 1 -- 172.21.15.32:0/1647213304 wait complete. 2020-12-05T17:41:58.730 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.721+0000 7f5486925700 1 Processor -- start 2020-12-05T17:41:58.730 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.721+0000 7f5486925700 1 -- start start 2020-12-05T17:41:58.731 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.721+0000 7f5486925700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f548005f6f0 0x7f5480066250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:58.731 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.721+0000 7f5486925700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f548005feb0 0x7f54800713d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:58.731 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.721+0000 7f5486925700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5480061780 0x7f54800718c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:58.731 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.721+0000 7f5486925700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5480062200 con 0x7f5480061780 2020-12-05T17:41:58.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.721+0000 7f5486925700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5480065100 con 0x7f548005feb0 2020-12-05T17:41:58.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.721+0000 7f5486925700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5480070590 con 0x7f548005f6f0 2020-12-05T17:41:58.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.721+0000 7f54856c3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5480061780 0x7f54800718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:58.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.721+0000 7f5484ec2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f548005f6f0 0x7f5480066250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:58.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.721+0000 7f54856c3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5480061780 0x7f54800718c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41606/0 (socket says 172.21.15.32:41606) 2020-12-05T17:41:58.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.721+0000 7f5484ec2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f548005f6f0 0x7f5480066250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:38392/0 (socket says 172.21.15.32:38392) 2020-12-05T17:41:58.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.725+0000 7f54856c3700 1 -- 172.21.15.32:0/1630441062 learned_addr learned my addr 172.21.15.32:0/1630441062 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:58.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.725+0000 7f547ffff700 1 --2- 172.21.15.32:0/1630441062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f548005feb0 0x7f54800713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:58.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.725+0000 7f54856c3700 1 -- 172.21.15.32:0/1630441062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f548005f6f0 msgr2=0x7f5480066250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:58.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.725+0000 7f54856c3700 1 --2- 172.21.15.32:0/1630441062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f548005f6f0 0x7f5480066250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.725+0000 7f54856c3700 1 -- 172.21.15.32:0/1630441062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f548005feb0 msgr2=0x7f54800713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:58.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.725+0000 7f54856c3700 1 --2- 172.21.15.32:0/1630441062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f548005feb0 0x7f54800713d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.725+0000 7f54856c3700 1 -- 172.21.15.32:0/1630441062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54800653a0 con 0x7f5480061780 2020-12-05T17:41:58.734 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.725+0000 7f54856c3700 1 --2- 172.21.15.32:0/1630441062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5480061780 0x7f54800718c0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7f5480061550 tx=0x7f547800cb40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:58.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.725+0000 7f547d7fa700 1 -- 172.21.15.32:0/1630441062 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5478005b70 con 0x7f5480061780 2020-12-05T17:41:58.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.725+0000 7f547d7fa700 1 -- 172.21.15.32:0/1630441062 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5478005cd0 con 0x7f5480061780 2020-12-05T17:41:58.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.725+0000 7f5486925700 1 -- 172.21.15.32:0/1630441062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5480071db0 con 0x7f5480061780 2020-12-05T17:41:58.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.725+0000 7f547d7fa700 1 -- 172.21.15.32:0/1630441062 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5478007300 con 0x7f5480061780 2020-12-05T17:41:58.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.725+0000 7f5486925700 1 -- 172.21.15.32:0/1630441062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548007fbb0 con 0x7f5480061780 2020-12-05T17:41:58.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.725+0000 7f547d7fa700 1 -- 172.21.15.32:0/1630441062 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5478010030 con 0x7f5480061780 2020-12-05T17:41:58.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.725+0000 7f5486925700 1 -- 172.21.15.32:0/1630441062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5480062370 con 0x7f5480061780 2020-12-05T17:41:58.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.729+0000 7f547d7fa700 1 --2- 172.21.15.32:0/1630441062 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5460055320 0x7f54600577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:58.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.729+0000 7f547d7fa700 1 -- 172.21.15.32:0/1630441062 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f547806df50 con 0x7f5480061780 2020-12-05T17:41:58.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.729+0000 7f5484ec2700 1 --2- 172.21.15.32:0/1630441062 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5460055320 0x7f54600577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:58.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.729+0000 7f5484ec2700 1 --2- 172.21.15.32:0/1630441062 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5460055320 0x7f54600577e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f547000b000 tx=0x7f5470006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:58.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.733+0000 7f547d7fa700 1 -- 172.21.15.32:0/1630441062 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5478043fe0 con 0x7f5480061780 2020-12-05T17:41:58.930 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.929+0000 7f5486925700 1 -- 172.21.15.32:0/1630441062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f5480064f20 con 0x7f5480061780 2020-12-05T17:41:58.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.937+0000 7f547d7fa700 1 -- 172.21.15.32:0/1630441062 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v46) v1 ==== 184+0+0 (secure 0 0 0) 0x7f5478047600 con 0x7f5480061780 2020-12-05T17:41:58.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.937+0000 7f5486925700 1 -- 172.21.15.32:0/1630441062 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5460055320 msgr2=0x7f54600577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:58.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.937+0000 7f5486925700 1 --2- 172.21.15.32:0/1630441062 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5460055320 0x7f54600577e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f547000b000 tx=0x7f5470006dc0).stop 2020-12-05T17:41:58.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.937+0000 7f5486925700 1 -- 172.21.15.32:0/1630441062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5480061780 msgr2=0x7f54800718c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:58.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.937+0000 7f5486925700 1 --2- 172.21.15.32:0/1630441062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5480061780 0x7f54800718c0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7f5480061550 tx=0x7f547800cb40).stop 2020-12-05T17:41:58.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.937+0000 7f5486925700 1 -- 172.21.15.32:0/1630441062 shutdown_connections 2020-12-05T17:41:58.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.937+0000 7f5486925700 1 --2- 172.21.15.32:0/1630441062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5480061780 0x7f54800718c0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.937+0000 7f5486925700 1 --2- 172.21.15.32:0/1630441062 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5460055320 0x7f54600577e0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.937+0000 7f5486925700 1 --2- 172.21.15.32:0/1630441062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f548005f6f0 0x7f5480066250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.937+0000 7f5486925700 1 --2- 172.21.15.32:0/1630441062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f548005feb0 0x7f54800713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:58.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.937+0000 7f5486925700 1 -- 172.21.15.32:0/1630441062 >> 172.21.15.32:0/1630441062 conn(0x7f54800b4b10 msgr2=0x7f54800ae3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:58.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.941+0000 7f5486925700 1 -- 172.21.15.32:0/1630441062 shutdown_connections 2020-12-05T17:41:58.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:58.941+0000 7f5486925700 1 -- 172.21.15.32:0/1630441062 wait complete. 2020-12-05T17:41:58.944 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.2 2020-12-05T17:41:58.956 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:41:59.070 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f41241b7700 1 Processor -- start 2020-12-05T17:41:59.071 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f41241b7700 1 -- start start 2020-12-05T17:41:59.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f41241b7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f411c0afe50 0x7f411c146a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:59.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f41241b7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f411c0b3aa0 0x7f411c1429e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:59.073 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f41241b7700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f411c0b1880 0x7f411c142ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:59.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f41241b7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411c0b0230 con 0x7f411c0afe50 2020-12-05T17:41:59.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f41241b7700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411c0b6a50 con 0x7f411c0b1880 2020-12-05T17:41:59.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f41241b7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c146080 con 0x7f411c0b3aa0 2020-12-05T17:41:59.074 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f4121f53700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f411c0afe50 0x7f411c146a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:59.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f4121f53700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f411c0afe50 0x7f411c146a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41614/0 (socket says 172.21.15.32:41614) 2020-12-05T17:41:59.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f4122754700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f411c0b1880 0x7f411c142ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:59.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f4121f53700 1 -- 172.21.15.32:0/2914560023 learned_addr learned my addr 172.21.15.32:0/2914560023 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:59.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f4121752700 1 --2- 172.21.15.32:0/2914560023 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f411c0b3aa0 0x7f411c1429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:59.075 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f4121f53700 1 -- 172.21.15.32:0/2914560023 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f411c0b3aa0 msgr2=0x7f411c1429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:59.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f4121f53700 1 --2- 172.21.15.32:0/2914560023 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f411c0b3aa0 0x7f411c1429e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:59.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f4121f53700 1 -- 172.21.15.32:0/2914560023 >> v2:172.21.15.61:3300/0 conn(0x7f411c0b1880 msgr2=0x7f411c142ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:59.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f4121f53700 1 --2- 172.21.15.32:0/2914560023 >> v2:172.21.15.61:3300/0 conn(0x7f411c0b1880 0x7f411c142ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:59.076 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f4121f53700 1 -- 172.21.15.32:0/2914560023 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c146320 con 0x7f411c0afe50 2020-12-05T17:41:59.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f4121752700 1 --2- 172.21.15.32:0/2914560023 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f411c0b3aa0 0x7f411c1429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:41:59.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f4121f53700 1 --2- 172.21.15.32:0/2914560023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f411c0afe50 0x7f411c146a60 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7f410c002f30 tx=0x7f410c006e80).ready entity=mon.0 client_cookie=b4b3670909982a25 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:59.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f4120f51700 1 -- 172.21.15.32:0/2914560023 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f410c009640 con 0x7f411c0afe50 2020-12-05T17:41:59.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f4120f51700 1 -- 172.21.15.32:0/2914560023 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410c0097a0 con 0x7f411c0afe50 2020-12-05T17:41:59.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f41241b7700 1 -- 172.21.15.32:0/2914560023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f411c0afe50 msgr2=0x7f411c146a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:59.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f4120f51700 1 -- 172.21.15.32:0/2914560023 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f410c009b00 con 0x7f411c0afe50 2020-12-05T17:41:59.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.069+0000 7f41241b7700 1 --2- 172.21.15.32:0/2914560023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f411c0afe50 0x7f411c146a60 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7f410c002f30 tx=0x7f410c006e80).stop 2020-12-05T17:41:59.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.073+0000 7f41241b7700 1 -- 172.21.15.32:0/2914560023 shutdown_connections 2020-12-05T17:41:59.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.073+0000 7f41241b7700 1 --2- 172.21.15.32:0/2914560023 >> v2:172.21.15.61:3300/0 conn(0x7f411c0b1880 0x7f411c142ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:59.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.073+0000 7f41241b7700 1 --2- 172.21.15.32:0/2914560023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f411c0afe50 0x7f411c146a60 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:59.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.073+0000 7f41241b7700 1 --2- 172.21.15.32:0/2914560023 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f411c0b3aa0 0x7f411c1429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:59.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.073+0000 7f41241b7700 1 -- 172.21.15.32:0/2914560023 >> 172.21.15.32:0/2914560023 conn(0x7f411c0b9100 msgr2=0x7f411c0b9510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:59.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.073+0000 7f41241b7700 1 -- 172.21.15.32:0/2914560023 shutdown_connections 2020-12-05T17:41:59.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.073+0000 7f41241b7700 1 -- 172.21.15.32:0/2914560023 wait complete. 2020-12-05T17:41:59.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.073+0000 7f41241b7700 1 Processor -- start 2020-12-05T17:41:59.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.073+0000 7f41241b7700 1 -- start start 2020-12-05T17:41:59.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.073+0000 7f41241b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f411c0afe50 0x7f411c140630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:59.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.073+0000 7f41241b7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f411c0b1880 0x7f411c140b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:59.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.073+0000 7f41241b7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f411c0b3aa0 0x7f411c1382b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:59.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.073+0000 7f41241b7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c1415e0 con 0x7f411c0afe50 2020-12-05T17:41:59.083 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.073+0000 7f41241b7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c13eed0 con 0x7f411c0b3aa0 2020-12-05T17:41:59.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.073+0000 7f41241b7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c13ec30 con 0x7f411c0b1880 2020-12-05T17:41:59.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.073+0000 7f4121f53700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f411c0afe50 0x7f411c140630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:59.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.073+0000 7f4121f53700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f411c0afe50 0x7f411c140630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41618/0 (socket says 172.21.15.32:41618) 2020-12-05T17:41:59.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.073+0000 7f4121f53700 1 -- 172.21.15.32:0/4228100004 learned_addr learned my addr 172.21.15.32:0/4228100004 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:59.084 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.073+0000 7f4121752700 1 --2- 172.21.15.32:0/4228100004 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f411c0b1880 0x7f411c140b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.073+0000 7f4122754700 1 --2- 172.21.15.32:0/4228100004 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f411c0b3aa0 0x7f411c1382b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.077+0000 7f4121f53700 1 -- 172.21.15.32:0/4228100004 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f411c0b1880 msgr2=0x7f411c140b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.077+0000 7f4121f53700 1 --2- 172.21.15.32:0/4228100004 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f411c0b1880 0x7f411c140b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.077+0000 7f4121f53700 1 -- 172.21.15.32:0/4228100004 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f411c0b3aa0 msgr2=0x7f411c1382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.077+0000 7f4121f53700 1 --2- 172.21.15.32:0/4228100004 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f411c0b3aa0 0x7f411c1382b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:59.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.077+0000 7f4121f53700 1 -- 172.21.15.32:0/4228100004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c143460 con 0x7f411c0afe50 2020-12-05T17:41:59.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.077+0000 7f4121f53700 1 --2- 172.21.15.32:0/4228100004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f411c0afe50 0x7f411c140630 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7f410c002f30 tx=0x7f410c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:59.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.077+0000 7f4112ffd700 1 -- 172.21.15.32:0/4228100004 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f410c011b80 con 0x7f411c0afe50 2020-12-05T17:41:59.086 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.077+0000 7f41241b7700 1 -- 172.21.15.32:0/4228100004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411c13f1f0 con 0x7f411c0afe50 2020-12-05T17:41:59.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.077+0000 7f4112ffd700 1 -- 172.21.15.32:0/4228100004 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410c011ce0 con 0x7f411c0afe50 2020-12-05T17:41:59.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.077+0000 7f4112ffd700 1 -- 172.21.15.32:0/4228100004 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f410c01d640 con 0x7f411c0afe50 2020-12-05T17:41:59.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.077+0000 7f41241b7700 1 -- 172.21.15.32:0/4228100004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411c13feb0 con 0x7f411c0afe50 2020-12-05T17:41:59.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.077+0000 7f41241b7700 1 -- 172.21.15.32:0/4228100004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f411c1435f0 con 0x7f411c0afe50 2020-12-05T17:41:59.087 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.077+0000 7f4112ffd700 1 -- 172.21.15.32:0/4228100004 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f410c00f070 con 0x7f411c0afe50 2020-12-05T17:41:59.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.077+0000 7f4112ffd700 1 --2- 172.21.15.32:0/4228100004 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f40fc0553b0 0x7f40fc057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:59.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.077+0000 7f4112ffd700 1 -- 172.21.15.32:0/4228100004 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f410c06f830 con 0x7f411c0afe50 2020-12-05T17:41:59.088 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.077+0000 7f4121752700 1 --2- 172.21.15.32:0/4228100004 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f40fc0553b0 0x7f40fc057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:59.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.081+0000 7f4121752700 1 --2- 172.21.15.32:0/4228100004 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f40fc0553b0 0x7f40fc057870 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f411800b000 tx=0x7f4118006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:59.089 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.085+0000 7f4112ffd700 1 -- 172.21.15.32:0/4228100004 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f410c045930 con 0x7f411c0afe50 2020-12-05T17:41:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.273+0000 7f41241b7700 1 -- 172.21.15.32:0/4228100004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f411c057050 con 0x7f411c0afe50 2020-12-05T17:41:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.273+0000 7f4112ffd700 1 -- 172.21.15.32:0/4228100004 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f410c006a10 con 0x7f411c0afe50 2020-12-05T17:41:59.275 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:59.275 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":73,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8473,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/711010965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":711010965},{"type":"v1","addr":"172.21.15.61:6833","nonce":711010965}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":70}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:41:55.663831+0000","modified":"2020-12-05T17:41:57.203619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8580":{"gid":8580,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1575496107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1575496107},{"type":"v1","addr":"172.21.15.32:6835","nonce":1575496107}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2020-12-05T17:41:59.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.273+0000 7f41241b7700 1 -- 172.21.15.32:0/4228100004 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f40fc0553b0 msgr2=0x7f40fc057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:59.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.273+0000 7f41241b7700 1 --2- 172.21.15.32:0/4228100004 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f40fc0553b0 0x7f40fc057870 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f411800b000 tx=0x7f4118006dc0).stop 2020-12-05T17:41:59.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.273+0000 7f41241b7700 1 -- 172.21.15.32:0/4228100004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f411c0afe50 msgr2=0x7f411c140630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:59.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.273+0000 7f41241b7700 1 --2- 172.21.15.32:0/4228100004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f411c0afe50 0x7f411c140630 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7f410c002f30 tx=0x7f410c015000).stop 2020-12-05T17:41:59.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.273+0000 7f41241b7700 1 -- 172.21.15.32:0/4228100004 shutdown_connections 2020-12-05T17:41:59.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.273+0000 7f41241b7700 1 --2- 172.21.15.32:0/4228100004 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f411c0b3aa0 0x7f411c1382b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:59.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.273+0000 7f41241b7700 1 --2- 172.21.15.32:0/4228100004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f411c0afe50 0x7f411c140630 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:59.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.273+0000 7f41241b7700 1 --2- 172.21.15.32:0/4228100004 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f40fc0553b0 0x7f40fc057870 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:59.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.273+0000 7f41241b7700 1 --2- 172.21.15.32:0/4228100004 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f411c0b1880 0x7f411c140b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:59.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.273+0000 7f41241b7700 1 -- 172.21.15.32:0/4228100004 >> 172.21.15.32:0/4228100004 conn(0x7f411c0b9100 msgr2=0x7f411c13afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:59.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.273+0000 7f41241b7700 1 -- 172.21.15.32:0/4228100004 shutdown_connections 2020-12-05T17:41:59.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.273+0000 7f41241b7700 1 -- 172.21.15.32:0/4228100004 wait complete. 2020-12-05T17:41:59.280 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 73 2020-12-05T17:41:59.489 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:41:55.663831+0000', 'modified': '2020-12-05T17:41:57.203619+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8580}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8580': {'gid': 8580, 'name': 'a', 'rank': 0, 'incarnation': 71, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/1575496107', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 1575496107}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 1575496107}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:41:59.489 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:41:59.490 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:41:59.622 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":8580,"want_state":"up:active","state":"up:active","rank_uptime":3.9458786730000002,"mdsmap_epoch":73,"osdmap_epoch":148,"osdmap_epoch_barrier":148,"uptime":9.3142609079999996} 2020-12-05T17:41:59.623 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 8580, "mdsmap_epoch": 73, "osdmap_epoch": 148, "osdmap_epoch_barrier": 148, "rank_uptime": 3.945878673, "state": "up:active", "uptime": 9.314260908, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:41:59.624 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:41:59.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.737+0000 7fb9eb59e700 1 Processor -- start 2020-12-05T17:41:59.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.737+0000 7fb9eb59e700 1 -- start start 2020-12-05T17:41:59.741 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.737+0000 7fb9eb59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb9ec05f6f0 0x7fb9ec1428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:59.741 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.737+0000 7fb9eb59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9ec0b0880 0x7fb9ec066070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:59.741 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.737+0000 7fb9eb59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ec05fad0 con 0x7fb9ec0b1980 2020-12-05T17:41:59.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.737+0000 7fb9eb59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9ec0b2f80 con 0x7fb9ec05f6f0 2020-12-05T17:41:59.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.737+0000 7fb9eb59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec145e40 con 0x7fb9ec0b0880 2020-12-05T17:41:59.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.737+0000 7fb9ea59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb9ec05f6f0 0x7fb9ec1428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:59.742 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.737+0000 7fb9e1d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb9ec0b1980 0x7fb9ec146580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44272/0 (socket says 172.21.15.32:44272) 2020-12-05T17:41:59.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.737+0000 7fb9e1d9b700 1 -- 172.21.15.32:0/2748375138 learned_addr learned my addr 172.21.15.32:0/2748375138 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:41:59.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.737+0000 7fb9ead9d700 1 --2- 172.21.15.32:0/2748375138 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9ec0b0880 0x7fb9ec066070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:59.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.737+0000 7fb9ea59c700 1 -- 172.21.15.32:0/2748375138 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9ec0b0880 msgr2=0x7fb9ec066070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:41:59.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.737+0000 7fb9ea59c700 1 --2- 172.21.15.32:0/2748375138 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9ec0b0880 0x7fb9ec066070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:59.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.737+0000 7fb9ea59c700 1 -- 172.21.15.32:0/2748375138 >> v1:172.21.15.32:6789/0 conn(0x7fb9ec0b1980 legacy=0x7fb9ec146580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:59.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.737+0000 7fb9e9d9b700 1 -- 172.21.15.32:0/2748375138 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4152017986 0 0) 0x7fb9ec05fad0 con 0x7fb9ec0b1980 2020-12-05T17:41:59.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.737+0000 7fb9ea59c700 1 -- 172.21.15.32:0/2748375138 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ec1460e0 con 0x7fb9ec05f6f0 2020-12-05T17:41:59.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.737+0000 7fb9ea59c700 1 --2- 172.21.15.32:0/2748375138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9ec05f6f0 0x7fb9ec1428f0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7fb9dc002f30 tx=0x7fb9dc006dc0).ready entity=mon.0 client_cookie=8f4a16c9d51652ea server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:59.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.737+0000 7fb9e9d9b700 1 -- 172.21.15.32:0/2748375138 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9dc009760 con 0x7fb9ec05f6f0 2020-12-05T17:41:59.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.737+0000 7fb9e9d9b700 1 -- 172.21.15.32:0/2748375138 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9dc0098c0 con 0x7fb9ec05f6f0 2020-12-05T17:41:59.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.741+0000 7fb9e9d9b700 1 -- 172.21.15.32:0/2748375138 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9dc009c20 con 0x7fb9ec05f6f0 2020-12-05T17:41:59.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.741+0000 7fb9eb59e700 1 -- 172.21.15.32:0/2748375138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9ec05f6f0 msgr2=0x7fb9ec1428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:59.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.741+0000 7fb9eb59e700 1 --2- 172.21.15.32:0/2748375138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9ec05f6f0 0x7fb9ec1428f0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7fb9dc002f30 tx=0x7fb9dc006dc0).stop 2020-12-05T17:41:59.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.741+0000 7fb9eb59e700 1 -- 172.21.15.32:0/2748375138 shutdown_connections 2020-12-05T17:41:59.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.741+0000 7fb9eb59e700 1 --2- 172.21.15.32:0/2748375138 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9ec0b0880 0x7fb9ec066070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:59.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.741+0000 7fb9eb59e700 1 --2- 172.21.15.32:0/2748375138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9ec05f6f0 0x7fb9ec1428f0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:59.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.741+0000 7fb9eb59e700 1 -- 172.21.15.32:0/2748375138 >> 172.21.15.32:0/2748375138 conn(0x7fb9ec0604c0 msgr2=0x7fb9ec05c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:59.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.741+0000 7fb9eb59e700 1 -- 172.21.15.32:0/2748375138 shutdown_connections 2020-12-05T17:41:59.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.741+0000 7fb9eb59e700 1 -- 172.21.15.32:0/2748375138 wait complete. 2020-12-05T17:41:59.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.741+0000 7fb9eb59e700 1 Processor -- start 2020-12-05T17:41:59.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.741+0000 7fb9eb59e700 1 -- start start 2020-12-05T17:41:59.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.741+0000 7fb9eb59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9ec05f6f0 0x7fb9ec0637d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:59.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.741+0000 7fb9eb59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9ec0b0880 0x7fb9ec154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:59.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.741+0000 7fb9eb59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9ec0b1980 0x7fb9ec116700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:59.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.741+0000 7fb9eb59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec115550 con 0x7fb9ec0b1980 2020-12-05T17:41:59.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.741+0000 7fb9eb59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec0644c0 con 0x7fb9ec05f6f0 2020-12-05T17:41:59.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.741+0000 7fb9eb59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec1162a0 con 0x7fb9ec0b0880 2020-12-05T17:41:59.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.741+0000 7fb9ead9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9ec0b1980 0x7fb9ec116700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:59.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.741+0000 7fb9ead9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9ec0b1980 0x7fb9ec116700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41634/0 (socket says 172.21.15.32:41634) 2020-12-05T17:41:59.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.741+0000 7fb9ead9d700 1 -- 172.21.15.32:0/274184227 learned_addr learned my addr 172.21.15.32:0/274184227 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:41:59.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.741+0000 7fb9e1d9b700 1 --2- 172.21.15.32:0/274184227 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9ec0b0880 0x7fb9ec154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:59.752 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.741+0000 7fb9ea59c700 1 --2- 172.21.15.32:0/274184227 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9ec05f6f0 0x7fb9ec0637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:59.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.745+0000 7fb9ead9d700 1 -- 172.21.15.32:0/274184227 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9ec0b0880 msgr2=0x7fb9ec154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:59.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.745+0000 7fb9ead9d700 1 --2- 172.21.15.32:0/274184227 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9ec0b0880 0x7fb9ec154d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:59.753 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.745+0000 7fb9ead9d700 1 -- 172.21.15.32:0/274184227 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9ec05f6f0 msgr2=0x7fb9ec0637d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.745+0000 7fb9ead9d700 1 --2- 172.21.15.32:0/274184227 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9ec05f6f0 0x7fb9ec0637d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.745+0000 7fb9ead9d700 1 -- 172.21.15.32:0/274184227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ec066360 con 0x7fb9ec0b1980 2020-12-05T17:41:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.745+0000 7fb9ead9d700 1 --2- 172.21.15.32:0/274184227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9ec0b1980 0x7fb9ec116700 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0x7fb9ec151860 tx=0x7fb9e40091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:59.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.745+0000 7fb9e3fff700 1 -- 172.21.15.32:0/274184227 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9e400ee60 con 0x7fb9ec0b1980 2020-12-05T17:41:59.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.745+0000 7fb9e3fff700 1 -- 172.21.15.32:0/274184227 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e400cb00 con 0x7fb9ec0b1980 2020-12-05T17:41:59.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.745+0000 7fb9eb59e700 1 -- 172.21.15.32:0/274184227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ec066600 con 0x7fb9ec0b1980 2020-12-05T17:41:59.755 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.745+0000 7fb9e3fff700 1 -- 172.21.15.32:0/274184227 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9e400ce60 con 0x7fb9ec0b1980 2020-12-05T17:41:59.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.745+0000 7fb9eb59e700 1 -- 172.21.15.32:0/274184227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ec066050 con 0x7fb9ec0b1980 2020-12-05T17:41:59.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.745+0000 7fb9eb59e700 1 -- 172.21.15.32:0/274184227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9ec057050 con 0x7fb9ec0b1980 2020-12-05T17:41:59.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.745+0000 7fb9e3fff700 1 -- 172.21.15.32:0/274184227 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb9e4022070 con 0x7fb9ec0b1980 2020-12-05T17:41:59.756 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.745+0000 7fb9e3fff700 1 --2- 172.21.15.32:0/274184227 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb9d00553b0 0x7fb9d0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:41:59.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.745+0000 7fb9e3fff700 1 -- 172.21.15.32:0/274184227 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb9e406e6e0 con 0x7fb9ec0b1980 2020-12-05T17:41:59.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.745+0000 7fb9ea59c700 1 --2- 172.21.15.32:0/274184227 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb9d00553b0 0x7fb9d0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:41:59.757 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.745+0000 7fb9ea59c700 1 --2- 172.21.15.32:0/274184227 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb9d00553b0 0x7fb9d0057870 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7fb9dc01a000 tx=0x7fb9dc015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:41:59.758 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.753+0000 7fb9e3fff700 1 -- 172.21.15.32:0/274184227 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb9e4044880 con 0x7fb9ec0b1980 2020-12-05T17:41:59.895 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.893+0000 7fb9eb59e700 1 -- 172.21.15.32:0/274184227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9ec0b1700 con 0x7fb9ec0b1980 2020-12-05T17:41:59.897 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.897+0000 7fb9e3fff700 1 -- 172.21.15.32:0/274184227 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fb9e4016070 con 0x7fb9ec0b1980 2020-12-05T17:41:59.898 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:41:59.898 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":148,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:41:56.666368+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:41:53.238803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2020-12-05T17:41:54.541493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:41:59.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.897+0000 7fb9eb59e700 1 -- 172.21.15.32:0/274184227 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb9d00553b0 msgr2=0x7fb9d0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:59.899 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.897+0000 7fb9eb59e700 1 --2- 172.21.15.32:0/274184227 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb9d00553b0 0x7fb9d0057870 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7fb9dc01a000 tx=0x7fb9dc015000).stop 2020-12-05T17:41:59.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.897+0000 7fb9eb59e700 1 -- 172.21.15.32:0/274184227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9ec0b1980 msgr2=0x7fb9ec116700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:41:59.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.897+0000 7fb9eb59e700 1 --2- 172.21.15.32:0/274184227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9ec0b1980 0x7fb9ec116700 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0x7fb9ec151860 tx=0x7fb9e40091a0).stop 2020-12-05T17:41:59.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.897+0000 7fb9eb59e700 1 -- 172.21.15.32:0/274184227 shutdown_connections 2020-12-05T17:41:59.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.897+0000 7fb9eb59e700 1 --2- 172.21.15.32:0/274184227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9ec0b1980 0x7fb9ec116700 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:59.900 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.897+0000 7fb9eb59e700 1 --2- 172.21.15.32:0/274184227 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9ec05f6f0 0x7fb9ec0637d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:59.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.897+0000 7fb9eb59e700 1 --2- 172.21.15.32:0/274184227 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb9d00553b0 0x7fb9d0057870 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:59.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.897+0000 7fb9eb59e700 1 --2- 172.21.15.32:0/274184227 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9ec0b0880 0x7fb9ec154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:41:59.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.897+0000 7fb9eb59e700 1 -- 172.21.15.32:0/274184227 >> 172.21.15.32:0/274184227 conn(0x7fb9ec0604c0 msgr2=0x7fb9ec1410a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:41:59.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.897+0000 7fb9eb59e700 1 -- 172.21.15.32:0/274184227 shutdown_connections 2020-12-05T17:41:59.901 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:41:59.897+0000 7fb9eb59e700 1 -- 172.21.15.32:0/274184227 wait complete. 2020-12-05T17:41:59.911 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:41:53.238803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:41:54.541493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:41:59.912 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:42:00.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.033+0000 7fce4bfff700 1 Processor -- start 2020-12-05T17:42:00.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.033+0000 7fce4bfff700 1 -- start start 2020-12-05T17:42:00.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce4bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce4c05feb0 0x7fce4c05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:00.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce4bfff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4c0b2df0 con 0x7fce4c0b2fb0 2020-12-05T17:42:00.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce4bfff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4c0b3c90 con 0x7fce4c05f3b0 2020-12-05T17:42:00.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce4bfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c0b2b50 con 0x7fce4c05feb0 2020-12-05T17:42:00.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce4b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fce4c0b2fb0 0x7fce4c142cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44290/0 (socket says 172.21.15.32:44290) 2020-12-05T17:42:00.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce4b7fe700 1 -- 172.21.15.32:0/1338554990 learned_addr learned my addr 172.21.15.32:0/1338554990 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:42:00.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce427fc700 1 --2- 172.21.15.32:0/1338554990 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce4c05feb0 0x7fce4c05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce4a7fc700 1 -- 172.21.15.32:0/1338554990 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2159972942 0 0) 0x7fce4c0b2df0 con 0x7fce4c0b2fb0 2020-12-05T17:42:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce4a7fc700 1 -- 172.21.15.32:0/1338554990 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce2c003230 con 0x7fce4c0b2fb0 2020-12-05T17:42:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce4a7fc700 1 -- 172.21.15.32:0/1338554990 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2335366192 0 0) 0x7fce4c0b3c90 con 0x7fce4c05f3b0 2020-12-05T17:42:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce427fc700 1 -- 172.21.15.32:0/1338554990 >> v1:172.21.15.61:6789/0 conn(0x7fce4c05f3b0 legacy=0x7fce4c13c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce427fc700 1 -- 172.21.15.32:0/1338554990 >> v1:172.21.15.32:6789/0 conn(0x7fce4c0b2fb0 legacy=0x7fce4c142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce427fc700 1 -- 172.21.15.32:0/1338554990 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c142820 con 0x7fce4c05feb0 2020-12-05T17:42:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce427fc700 1 --2- 172.21.15.32:0/1338554990 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce4c05feb0 0x7fce4c05f790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fce34002f30 tx=0x7fce34006dc0).ready entity=mon.2 client_cookie=496862b1a2aeadb4 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce4a7fc700 1 -- 172.21.15.32:0/1338554990 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce340097b0 con 0x7fce4c05feb0 2020-12-05T17:42:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce4a7fc700 1 -- 172.21.15.32:0/1338554990 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce34009910 con 0x7fce4c05feb0 2020-12-05T17:42:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce4a7fc700 1 -- 172.21.15.32:0/1338554990 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce34009c70 con 0x7fce4c05feb0 2020-12-05T17:42:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce4bfff700 1 -- 172.21.15.32:0/1338554990 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce4c05feb0 msgr2=0x7fce4c05f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce4bfff700 1 --2- 172.21.15.32:0/1338554990 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce4c05feb0 0x7fce4c05f790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fce34002f30 tx=0x7fce34006dc0).stop 2020-12-05T17:42:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce4bfff700 1 -- 172.21.15.32:0/1338554990 shutdown_connections 2020-12-05T17:42:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce4bfff700 1 --2- 172.21.15.32:0/1338554990 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce4c05feb0 0x7fce4c05f790 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce4bfff700 1 -- 172.21.15.32:0/1338554990 >> 172.21.15.32:0/1338554990 conn(0x7fce4c0b8f20 msgr2=0x7fce4c0b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce4bfff700 1 -- 172.21.15.32:0/1338554990 shutdown_connections 2020-12-05T17:42:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.037+0000 7fce4bfff700 1 -- 172.21.15.32:0/1338554990 wait complete. 2020-12-05T17:42:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce4bfff700 1 Processor -- start 2020-12-05T17:42:00.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce4bfff700 1 -- start start 2020-12-05T17:42:00.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce4bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce4c05f3b0 0x7fce4c154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:00.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce4bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce4c05feb0 0x7fce4c155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:00.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce4bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce4c0b2fb0 0x7fce4c13e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce4bfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c1425b0 con 0x7fce4c05feb0 2020-12-05T17:42:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce4bfff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c151db0 con 0x7fce4c0b2fb0 2020-12-05T17:42:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce4bfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c152050 con 0x7fce4c05f3b0 2020-12-05T17:42:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce427fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce4c05feb0 0x7fce4c155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce427fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce4c05feb0 0x7fce4c155250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41646/0 (socket says 172.21.15.32:41646) 2020-12-05T17:42:00.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce427fc700 1 -- 172.21.15.32:0/3043420560 learned_addr learned my addr 172.21.15.32:0/3043420560 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:00.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce4affd700 1 --2- 172.21.15.32:0/3043420560 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce4c05f3b0 0x7fce4c154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:00.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce4b7fe700 1 --2- 172.21.15.32:0/3043420560 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce4c0b2fb0 0x7fce4c13e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:00.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce427fc700 1 -- 172.21.15.32:0/3043420560 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce4c05f3b0 msgr2=0x7fce4c154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce427fc700 1 --2- 172.21.15.32:0/3043420560 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce4c05f3b0 0x7fce4c154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce427fc700 1 -- 172.21.15.32:0/3043420560 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce4c0b2fb0 msgr2=0x7fce4c13e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce427fc700 1 --2- 172.21.15.32:0/3043420560 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce4c0b2fb0 0x7fce4c13e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce427fc700 1 -- 172.21.15.32:0/3043420560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c140530 con 0x7fce4c05feb0 2020-12-05T17:42:00.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce427fc700 1 --2- 172.21.15.32:0/3043420560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce4c05feb0 0x7fce4c155250 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7fce34002f30 tx=0x7fce34015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:00.049 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce48ff9700 1 -- 172.21.15.32:0/3043420560 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce34009e20 con 0x7fce4c05feb0 2020-12-05T17:42:00.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce4bfff700 1 -- 172.21.15.32:0/3043420560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4c142230 con 0x7fce4c05feb0 2020-12-05T17:42:00.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce48ff9700 1 -- 172.21.15.32:0/3043420560 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce34011d90 con 0x7fce4c05feb0 2020-12-05T17:42:00.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce48ff9700 1 -- 172.21.15.32:0/3043420560 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce3401d640 con 0x7fce4c05feb0 2020-12-05T17:42:00.050 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.041+0000 7fce4bfff700 1 -- 172.21.15.32:0/3043420560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4c141620 con 0x7fce4c05feb0 2020-12-05T17:42:00.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.045+0000 7fce4bfff700 1 -- 172.21.15.32:0/3043420560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce4c057050 con 0x7fce4c05feb0 2020-12-05T17:42:00.051 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.045+0000 7fce48ff9700 1 -- 172.21.15.32:0/3043420560 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fce34024030 con 0x7fce4c05feb0 2020-12-05T17:42:00.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.045+0000 7fce48ff9700 1 --2- 172.21.15.32:0/3043420560 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fce300553b0 0x7fce30057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:00.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.045+0000 7fce48ff9700 1 -- 172.21.15.32:0/3043420560 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fce3406f2c0 con 0x7fce4c05feb0 2020-12-05T17:42:00.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.045+0000 7fce4affd700 1 --2- 172.21.15.32:0/3043420560 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fce300553b0 0x7fce30057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:00.052 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.045+0000 7fce4affd700 1 --2- 172.21.15.32:0/3043420560 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fce300553b0 0x7fce30057870 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7fce3c00d000 tx=0x7fce3c007a30).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:00.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.053+0000 7fce48ff9700 1 -- 172.21.15.32:0/3043420560 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fce34045400 con 0x7fce4c05feb0 2020-12-05T17:42:00.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.197+0000 7fce4bfff700 1 -- 172.21.15.32:0/3043420560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fce4c142420 con 0x7fce4c05feb0 2020-12-05T17:42:00.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.197+0000 7fce48ff9700 1 -- 172.21.15.32:0/3043420560 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v148) v1 ==== 110+0+10 (secure 0 0 0) 0x7fce34022020 con 0x7fce4c05feb0 2020-12-05T17:42:00.201 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:42:00.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.201+0000 7fce4bfff700 1 -- 172.21.15.32:0/3043420560 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fce300553b0 msgr2=0x7fce30057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.201+0000 7fce4bfff700 1 --2- 172.21.15.32:0/3043420560 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fce300553b0 0x7fce30057870 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7fce3c00d000 tx=0x7fce3c007a30).stop 2020-12-05T17:42:00.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.201+0000 7fce4bfff700 1 -- 172.21.15.32:0/3043420560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce4c05feb0 msgr2=0x7fce4c155250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.201+0000 7fce4bfff700 1 --2- 172.21.15.32:0/3043420560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce4c05feb0 0x7fce4c155250 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7fce34002f30 tx=0x7fce34015000).stop 2020-12-05T17:42:00.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.201+0000 7fce4bfff700 1 -- 172.21.15.32:0/3043420560 shutdown_connections 2020-12-05T17:42:00.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.201+0000 7fce4bfff700 1 --2- 172.21.15.32:0/3043420560 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce4c0b2fb0 0x7fce4c13e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.201+0000 7fce4bfff700 1 --2- 172.21.15.32:0/3043420560 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fce300553b0 0x7fce30057870 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.201+0000 7fce4bfff700 1 --2- 172.21.15.32:0/3043420560 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce4c05f3b0 0x7fce4c154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.201+0000 7fce4bfff700 1 --2- 172.21.15.32:0/3043420560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce4c05feb0 0x7fce4c155250 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.201+0000 7fce4bfff700 1 -- 172.21.15.32:0/3043420560 >> 172.21.15.32:0/3043420560 conn(0x7fce4c0b8f20 msgr2=0x7fce4c066930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:00.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.201+0000 7fce4bfff700 1 -- 172.21.15.32:0/3043420560 shutdown_connections 2020-12-05T17:42:00.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.201+0000 7fce4bfff700 1 -- 172.21.15.32:0/3043420560 wait complete. 2020-12-05T17:42:00.215 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:42:00.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.325+0000 7fb8119fb700 1 Processor -- start 2020-12-05T17:42:00.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.325+0000 7fb8119fb700 1 -- start start 2020-12-05T17:42:00.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb8119fb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb80c0b6690 0x7fb80c0b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:00.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb8119fb700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb80c0b5240 0x7fb80c145f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:00.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb8119fb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb80c0b56f0 con 0x7fb80c0b5b20 2020-12-05T17:42:00.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb8119fb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb80c0b1920 con 0x7fb80c0b6690 2020-12-05T17:42:00.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb8119fb700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb80c0b1bc0 con 0x7fb80c0b5240 2020-12-05T17:42:00.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb80a7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb80c0b6690 0x7fb80c0b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:00.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb80affd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb80c0b5b20 0x7fb80c0b2060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44300/0 (socket says 172.21.15.32:44300) 2020-12-05T17:42:00.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb80affd700 1 -- 172.21.15.32:0/3736283416 learned_addr learned my addr 172.21.15.32:0/3736283416 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:42:00.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb80b7fe700 1 --2- 172.21.15.32:0/3736283416 >> v2:172.21.15.61:3300/0 conn(0x7fb80c0b5240 0x7fb80c145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:00.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb80a7fc700 1 -- 172.21.15.32:0/3736283416 >> v2:172.21.15.61:3300/0 conn(0x7fb80c0b5240 msgr2=0x7fb80c145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:42:00.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb80a7fc700 1 --2- 172.21.15.32:0/3736283416 >> v2:172.21.15.61:3300/0 conn(0x7fb80c0b5240 0x7fb80c145f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb80a7fc700 1 -- 172.21.15.32:0/3736283416 >> v1:172.21.15.32:6789/0 conn(0x7fb80c0b5b20 legacy=0x7fb80c0b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb809ffb700 1 -- 172.21.15.32:0/3736283416 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407183599 0 0) 0x7fb80c0b56f0 con 0x7fb80c0b5b20 2020-12-05T17:42:00.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb80a7fc700 1 -- 172.21.15.32:0/3736283416 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c146400 con 0x7fb80c0b6690 2020-12-05T17:42:00.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb80a7fc700 1 --2- 172.21.15.32:0/3736283416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb80c0b6690 0x7fb80c0b5f00 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7fb7f40032b0 tx=0x7fb7f40091a0).ready entity=mon.0 client_cookie=43b75adde680e895 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:00.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb809ffb700 1 -- 172.21.15.32:0/3736283416 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7f400b9a0 con 0x7fb80c0b6690 2020-12-05T17:42:00.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb809ffb700 1 -- 172.21.15.32:0/3736283416 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f400bb00 con 0x7fb80c0b6690 2020-12-05T17:42:00.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb809ffb700 1 -- 172.21.15.32:0/3736283416 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7f400bc70 con 0x7fb80c0b6690 2020-12-05T17:42:00.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb8119fb700 1 -- 172.21.15.32:0/3736283416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb80c0b6690 msgr2=0x7fb80c0b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb8119fb700 1 --2- 172.21.15.32:0/3736283416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb80c0b6690 0x7fb80c0b5f00 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7fb7f40032b0 tx=0x7fb7f40091a0).stop 2020-12-05T17:42:00.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb8119fb700 1 -- 172.21.15.32:0/3736283416 shutdown_connections 2020-12-05T17:42:00.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb8119fb700 1 --2- 172.21.15.32:0/3736283416 >> v2:172.21.15.61:3300/0 conn(0x7fb80c0b5240 0x7fb80c145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb8119fb700 1 --2- 172.21.15.32:0/3736283416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb80c0b6690 0x7fb80c0b5f00 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb8119fb700 1 -- 172.21.15.32:0/3736283416 >> 172.21.15.32:0/3736283416 conn(0x7fb80c05f3b0 msgr2=0x7fb80c05f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:00.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb8119fb700 1 -- 172.21.15.32:0/3736283416 shutdown_connections 2020-12-05T17:42:00.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.329+0000 7fb8119fb700 1 -- 172.21.15.32:0/3736283416 wait complete. 2020-12-05T17:42:00.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.333+0000 7fb8119fb700 1 Processor -- start 2020-12-05T17:42:00.338 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.333+0000 7fb8119fb700 1 -- start start 2020-12-05T17:42:00.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.333+0000 7fb8119fb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb80c0b5240 0x7fb80c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:00.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.333+0000 7fb8119fb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb80c0b5b20 0x7fb80c0a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:00.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.333+0000 7fb8119fb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb80c0b6690 0x7fb80c140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:00.340 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.333+0000 7fb8119fb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c065c50 con 0x7fb80c0b5b20 2020-12-05T17:42:00.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.333+0000 7fb8119fb700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c0668b0 con 0x7fb80c0b6690 2020-12-05T17:42:00.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.333+0000 7fb8119fb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c066b50 con 0x7fb80c0b5240 2020-12-05T17:42:00.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.333+0000 7fb80a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb80c0b5b20 0x7fb80c0a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:00.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.333+0000 7fb80a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb80c0b5b20 0x7fb80c0a5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41660/0 (socket says 172.21.15.32:41660) 2020-12-05T17:42:00.341 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.333+0000 7fb80a7fc700 1 -- 172.21.15.32:0/2725058415 learned_addr learned my addr 172.21.15.32:0/2725058415 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:00.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.333+0000 7fb80affd700 1 --2- 172.21.15.32:0/2725058415 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb80c0b5240 0x7fb80c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:00.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.333+0000 7fb80b7fe700 1 --2- 172.21.15.32:0/2725058415 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb80c0b6690 0x7fb80c140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:00.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.337+0000 7fb80a7fc700 1 -- 172.21.15.32:0/2725058415 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb80c0b5240 msgr2=0x7fb80c0a5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.337+0000 7fb80a7fc700 1 --2- 172.21.15.32:0/2725058415 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb80c0b5240 0x7fb80c0a5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.342 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.337+0000 7fb80a7fc700 1 -- 172.21.15.32:0/2725058415 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb80c0b6690 msgr2=0x7fb80c140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.337+0000 7fb80a7fc700 1 --2- 172.21.15.32:0/2725058415 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb80c0b6690 0x7fb80c140530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.337+0000 7fb80a7fc700 1 -- 172.21.15.32:0/2725058415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c066df0 con 0x7fb80c0b5b20 2020-12-05T17:42:00.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.337+0000 7fb80a7fc700 1 --2- 172.21.15.32:0/2725058415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb80c0b5b20 0x7fb80c0a5d00 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0x7fb7f40032b0 tx=0x7fb7f4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:00.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.337+0000 7fb803fff700 1 -- 172.21.15.32:0/2725058415 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7f40205f0 con 0x7fb80c0b5b20 2020-12-05T17:42:00.343 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.337+0000 7fb803fff700 1 -- 172.21.15.32:0/2725058415 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f4020750 con 0x7fb80c0b5b20 2020-12-05T17:42:00.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.337+0000 7fb8119fb700 1 -- 172.21.15.32:0/2725058415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80c0a3890 con 0x7fb80c0b5b20 2020-12-05T17:42:00.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.337+0000 7fb803fff700 1 -- 172.21.15.32:0/2725058415 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7f4020ab0 con 0x7fb80c0b5b20 2020-12-05T17:42:00.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.337+0000 7fb8119fb700 1 -- 172.21.15.32:0/2725058415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80c0a3d30 con 0x7fb80c0b5b20 2020-12-05T17:42:00.344 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.337+0000 7fb8119fb700 1 -- 172.21.15.32:0/2725058415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb80c0a3a20 con 0x7fb80c0b5b20 2020-12-05T17:42:00.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.337+0000 7fb803fff700 1 -- 172.21.15.32:0/2725058415 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb7f4027030 con 0x7fb80c0b5b20 2020-12-05T17:42:00.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.337+0000 7fb803fff700 1 --2- 172.21.15.32:0/2725058415 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb7f0055470 0x7fb7f0057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:00.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.341+0000 7fb803fff700 1 -- 172.21.15.32:0/2725058415 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb7f406fa00 con 0x7fb80c0b5b20 2020-12-05T17:42:00.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.341+0000 7fb80affd700 1 --2- 172.21.15.32:0/2725058415 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb7f0055470 0x7fb7f0057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:00.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.341+0000 7fb80affd700 1 --2- 172.21.15.32:0/2725058415 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb7f0055470 0x7fb7f0057930 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7fb7fc00d000 tx=0x7fb7fc009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:00.347 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.345+0000 7fb803fff700 1 -- 172.21.15.32:0/2725058415 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb7f40495d0 con 0x7fb80c0b5b20 2020-12-05T17:42:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.485+0000 7fb8119fb700 1 -- 172.21.15.32:0/2725058415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb80c057050 con 0x7fb80c0b5b20 2020-12-05T17:42:00.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.489+0000 7fb803fff700 1 -- 172.21.15.32:0/2725058415 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v148) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb7f4025020 con 0x7fb80c0b5b20 2020-12-05T17:42:00.490 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:42:00.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.489+0000 7fb8119fb700 1 -- 172.21.15.32:0/2725058415 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb7f0055470 msgr2=0x7fb7f0057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.489+0000 7fb8119fb700 1 --2- 172.21.15.32:0/2725058415 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb7f0055470 0x7fb7f0057930 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7fb7fc00d000 tx=0x7fb7fc009040).stop 2020-12-05T17:42:00.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.489+0000 7fb8119fb700 1 -- 172.21.15.32:0/2725058415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb80c0b5b20 msgr2=0x7fb80c0a5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.489+0000 7fb8119fb700 1 --2- 172.21.15.32:0/2725058415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb80c0b5b20 0x7fb80c0a5d00 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0x7fb7f40032b0 tx=0x7fb7f4016000).stop 2020-12-05T17:42:00.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.489+0000 7fb8119fb700 1 -- 172.21.15.32:0/2725058415 shutdown_connections 2020-12-05T17:42:00.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.489+0000 7fb8119fb700 1 --2- 172.21.15.32:0/2725058415 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb80c0b6690 0x7fb80c140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.489+0000 7fb8119fb700 1 --2- 172.21.15.32:0/2725058415 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb7f0055470 0x7fb7f0057930 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.489+0000 7fb8119fb700 1 --2- 172.21.15.32:0/2725058415 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb80c0b5240 0x7fb80c0a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.489+0000 7fb8119fb700 1 --2- 172.21.15.32:0/2725058415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb80c0b5b20 0x7fb80c0a5d00 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.489+0000 7fb8119fb700 1 -- 172.21.15.32:0/2725058415 >> 172.21.15.32:0/2725058415 conn(0x7fb80c05f3b0 msgr2=0x7fb80c0b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:00.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.489+0000 7fb8119fb700 1 -- 172.21.15.32:0/2725058415 shutdown_connections 2020-12-05T17:42:00.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.489+0000 7fb8119fb700 1 -- 172.21.15.32:0/2725058415 wait complete. 2020-12-05T17:42:00.503 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:42:00.619 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.617+0000 7f81ab575700 1 Processor -- start 2020-12-05T17:42:00.620 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.617+0000 7f81ab575700 1 -- start start 2020-12-05T17:42:00.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.617+0000 7f81ab575700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81a41433d0 0x7f81a405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:00.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.617+0000 7f81ab575700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a40b2df0 con 0x7f81a405f3b0 2020-12-05T17:42:00.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.617+0000 7f81ab575700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a40b3c90 con 0x7f81a405feb0 2020-12-05T17:42:00.622 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.617+0000 7f81ab575700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a40b2b50 con 0x7f81a41433d0 2020-12-05T17:42:00.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.617+0000 7f81aad74700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81a41433d0 0x7f81a405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:00.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.617+0000 7f81aa573700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f81a405f3b0 0x7f81a413c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44314/0 (socket says 172.21.15.32:44314) 2020-12-05T17:42:00.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.617+0000 7f81aad74700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81a41433d0 0x7f81a405f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41670/0 (socket says 172.21.15.32:41670) 2020-12-05T17:42:00.623 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.617+0000 7f81aad74700 1 -- 172.21.15.32:0/4002818041 learned_addr learned my addr 172.21.15.32:0/4002818041 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:00.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.621+0000 7f81aad74700 1 -- 172.21.15.32:0/4002818041 >> v1:172.21.15.61:6789/0 conn(0x7f81a405feb0 legacy=0x7f81a4142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:42:00.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.621+0000 7f81aad74700 1 -- 172.21.15.32:0/4002818041 >> v1:172.21.15.32:6789/0 conn(0x7f81a405f3b0 legacy=0x7f81a413c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.621+0000 7f81aad74700 1 -- 172.21.15.32:0/4002818041 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a4142820 con 0x7f81a41433d0 2020-12-05T17:42:00.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.621+0000 7f81aad74700 1 --2- 172.21.15.32:0/4002818041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a41433d0 0x7f81a405f790 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f81a40b1000 tx=0x7f819c009270).ready entity=mon.0 client_cookie=d9031b99d209379b server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:00.624 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.621+0000 7f81a9571700 1 -- 172.21.15.32:0/4002818041 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f819c00bbc0 con 0x7f81a41433d0 2020-12-05T17:42:00.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.621+0000 7f81a9571700 1 -- 172.21.15.32:0/4002818041 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819c00bd20 con 0x7f81a41433d0 2020-12-05T17:42:00.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.621+0000 7f81ab575700 1 -- 172.21.15.32:0/4002818041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a41433d0 msgr2=0x7f81a405f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.621+0000 7f81a9571700 1 -- 172.21.15.32:0/4002818041 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f819c00be90 con 0x7f81a41433d0 2020-12-05T17:42:00.625 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.621+0000 7f81ab575700 1 --2- 172.21.15.32:0/4002818041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a41433d0 0x7f81a405f790 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f81a40b1000 tx=0x7f819c009270).stop 2020-12-05T17:42:00.627 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.621+0000 7f81ab575700 1 -- 172.21.15.32:0/4002818041 shutdown_connections 2020-12-05T17:42:00.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.621+0000 7f81ab575700 1 --2- 172.21.15.32:0/4002818041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a41433d0 0x7f81a405f790 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.621+0000 7f81ab575700 1 -- 172.21.15.32:0/4002818041 >> 172.21.15.32:0/4002818041 conn(0x7f81a40b8f20 msgr2=0x7f81a40b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:00.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.621+0000 7f81ab575700 1 -- 172.21.15.32:0/4002818041 shutdown_connections 2020-12-05T17:42:00.628 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.621+0000 7f81ab575700 1 -- 172.21.15.32:0/4002818041 wait complete. 2020-12-05T17:42:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.621+0000 7f81ab575700 1 Processor -- start 2020-12-05T17:42:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.621+0000 7f81ab575700 1 -- start start 2020-12-05T17:42:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81ab575700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f81a405f3b0 0x7f81a40657d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81ab575700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a405feb0 0x7f81a4154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81ab575700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f81a41433d0 0x7f81a4154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81ab575700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a4147930 con 0x7f81a405feb0 2020-12-05T17:42:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81ab575700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a4147010 con 0x7f81a41433d0 2020-12-05T17:42:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81ab575700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a41483e0 con 0x7f81a405f3b0 2020-12-05T17:42:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81a9d72700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a405feb0 0x7f81a4154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:00.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81a9d72700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a405feb0 0x7f81a4154ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41674/0 (socket says 172.21.15.32:41674) 2020-12-05T17:42:00.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81a9d72700 1 -- 172.21.15.32:0/625091177 learned_addr learned my addr 172.21.15.32:0/625091177 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:00.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81aad74700 1 --2- 172.21.15.32:0/625091177 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f81a41433d0 0x7f81a4154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:00.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81aa573700 1 --2- 172.21.15.32:0/625091177 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f81a405f3b0 0x7f81a40657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:00.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81a9d72700 1 -- 172.21.15.32:0/625091177 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f81a405f3b0 msgr2=0x7f81a40657d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81a9d72700 1 --2- 172.21.15.32:0/625091177 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f81a405f3b0 0x7f81a40657d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81a9d72700 1 -- 172.21.15.32:0/625091177 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f81a41433d0 msgr2=0x7f81a4154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81a9d72700 1 --2- 172.21.15.32:0/625091177 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f81a41433d0 0x7f81a4154fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81a9d72700 1 -- 172.21.15.32:0/625091177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a4148610 con 0x7f81a405feb0 2020-12-05T17:42:00.632 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81a9d72700 1 --2- 172.21.15.32:0/625091177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a405feb0 0x7f81a4154ae0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7f8194001f60 tx=0x7f8194007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:00.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81937fe700 1 -- 172.21.15.32:0/625091177 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f819400e4a0 con 0x7f81a405feb0 2020-12-05T17:42:00.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81937fe700 1 -- 172.21.15.32:0/625091177 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819400e600 con 0x7f81a405feb0 2020-12-05T17:42:00.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81ab575700 1 -- 172.21.15.32:0/625091177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a41488b0 con 0x7f81a405feb0 2020-12-05T17:42:00.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81937fe700 1 -- 172.21.15.32:0/625091177 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f819400e960 con 0x7f81a405feb0 2020-12-05T17:42:00.633 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81ab575700 1 -- 172.21.15.32:0/625091177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a4145290 con 0x7f81a405feb0 2020-12-05T17:42:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81937fe700 1 -- 172.21.15.32:0/625091177 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8194010030 con 0x7f81a405feb0 2020-12-05T17:42:00.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.625+0000 7f81ab575700 1 -- 172.21.15.32:0/625091177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a4057050 con 0x7f81a405feb0 2020-12-05T17:42:00.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.629+0000 7f81937fe700 1 --2- 172.21.15.32:0/625091177 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8184055440 0x7f8184057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:00.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.629+0000 7f81aa573700 1 --2- 172.21.15.32:0/625091177 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8184055440 0x7f8184057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:00.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.629+0000 7f81937fe700 1 -- 172.21.15.32:0/625091177 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f819406dbd0 con 0x7f81a405feb0 2020-12-05T17:42:00.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.629+0000 7f81aa573700 1 --2- 172.21.15.32:0/625091177 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8184055440 0x7f8184057900 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f81a000d000 tx=0x7f81a0007b10).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:00.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.629+0000 7f81937fe700 1 -- 172.21.15.32:0/625091177 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8194043c60 con 0x7f81a405feb0 2020-12-05T17:42:00.773 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.773+0000 7f81ab575700 1 -- 172.21.15.32:0/625091177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f81a405aab0 con 0x7f81a405feb0 2020-12-05T17:42:00.774 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.773+0000 7f81937fe700 1 -- 172.21.15.32:0/625091177 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v148) v1 ==== 104+0+10 (secure 0 0 0) 0x7f8194047280 con 0x7f81a405feb0 2020-12-05T17:42:00.774 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:42:00.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.773+0000 7f81ab575700 1 -- 172.21.15.32:0/625091177 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8184055440 msgr2=0x7f8184057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.773+0000 7f81ab575700 1 --2- 172.21.15.32:0/625091177 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8184055440 0x7f8184057900 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f81a000d000 tx=0x7f81a0007b10).stop 2020-12-05T17:42:00.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.773+0000 7f81ab575700 1 -- 172.21.15.32:0/625091177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a405feb0 msgr2=0x7f81a4154ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.773+0000 7f81ab575700 1 --2- 172.21.15.32:0/625091177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a405feb0 0x7f81a4154ae0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7f8194001f60 tx=0x7f8194007c00).stop 2020-12-05T17:42:00.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.773+0000 7f81ab575700 1 -- 172.21.15.32:0/625091177 shutdown_connections 2020-12-05T17:42:00.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.773+0000 7f81ab575700 1 --2- 172.21.15.32:0/625091177 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f81a41433d0 0x7f81a4154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.773+0000 7f81ab575700 1 --2- 172.21.15.32:0/625091177 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f8184055440 0x7f8184057900 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.773+0000 7f81ab575700 1 --2- 172.21.15.32:0/625091177 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f81a405f3b0 0x7f81a40657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.773+0000 7f81ab575700 1 --2- 172.21.15.32:0/625091177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a405feb0 0x7f81a4154ae0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.773+0000 7f81ab575700 1 -- 172.21.15.32:0/625091177 >> 172.21.15.32:0/625091177 conn(0x7f81a40b8f20 msgr2=0x7f81a40b6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.773+0000 7f81ab575700 1 -- 172.21.15.32:0/625091177 shutdown_connections 2020-12-05T17:42:00.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.773+0000 7f81ab575700 1 -- 172.21.15.32:0/625091177 wait complete. 2020-12-05T17:42:00.788 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:42:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.909+0000 7f26189da700 1 Processor -- start 2020-12-05T17:42:00.911 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.909+0000 7f26189da700 1 -- start start 2020-12-05T17:42:00.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.909+0000 7f26189da700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26140b1720 0x7f261414a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:00.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.909+0000 7f26189da700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2614061f80 con 0x7f261414aeb0 2020-12-05T17:42:00.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.909+0000 7f26189da700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2614147ad0 con 0x7f2614061ba0 2020-12-05T17:42:00.912 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.909+0000 7f26189da700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26140b29a0 con 0x7f26140b1720 2020-12-05T17:42:00.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.909+0000 7f2613fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f261414aeb0 0x7f26141460b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44332/0 (socket says 172.21.15.32:44332) 2020-12-05T17:42:00.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.909+0000 7f2613fff700 1 -- 172.21.15.32:0/1536681630 learned_addr learned my addr 172.21.15.32:0/1536681630 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:42:00.913 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.909+0000 7f2612ffd700 1 --2- 172.21.15.32:0/1536681630 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26140b1720 0x7f261414a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:00.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.909+0000 7f26127fc700 1 -- 172.21.15.32:0/1536681630 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437045600 0 0) 0x7f2614061f80 con 0x7f261414aeb0 2020-12-05T17:42:00.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.909+0000 7f26127fc700 1 -- 172.21.15.32:0/1536681630 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2600003230 con 0x7f261414aeb0 2020-12-05T17:42:00.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.909+0000 7f26127fc700 1 -- 172.21.15.32:0/1536681630 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3461409314 0 0) 0x7f2614147ad0 con 0x7f2614061ba0 2020-12-05T17:42:00.915 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.909+0000 7f26127fc700 1 -- 172.21.15.32:0/1536681630 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2614061f80 con 0x7f2614061ba0 2020-12-05T17:42:00.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.909+0000 7f2612ffd700 1 -- 172.21.15.32:0/1536681630 >> v1:172.21.15.61:6789/0 conn(0x7f2614061ba0 legacy=0x7f26140b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.909+0000 7f2612ffd700 1 -- 172.21.15.32:0/1536681630 >> v1:172.21.15.32:6789/0 conn(0x7f261414aeb0 legacy=0x7f26141460b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.909+0000 7f2612ffd700 1 -- 172.21.15.32:0/1536681630 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261414a280 con 0x7f26140b1720 2020-12-05T17:42:00.916 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.909+0000 7f2612ffd700 1 --2- 172.21.15.32:0/1536681630 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26140b1720 0x7f261414a9c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f2608002f30 tx=0x7f2608006dc0).ready entity=mon.2 client_cookie=e7e8a91497ca1d71 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:00.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.909+0000 7f26127fc700 1 -- 172.21.15.32:0/1536681630 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2608009730 con 0x7f26140b1720 2020-12-05T17:42:00.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.913+0000 7f26127fc700 1 -- 172.21.15.32:0/1536681630 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2608009890 con 0x7f26140b1720 2020-12-05T17:42:00.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.913+0000 7f26127fc700 1 -- 172.21.15.32:0/1536681630 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2608009bf0 con 0x7f26140b1720 2020-12-05T17:42:00.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.913+0000 7f26189da700 1 -- 172.21.15.32:0/1536681630 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26140b1720 msgr2=0x7f261414a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.917 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.913+0000 7f26189da700 1 --2- 172.21.15.32:0/1536681630 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26140b1720 0x7f261414a9c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f2608002f30 tx=0x7f2608006dc0).stop 2020-12-05T17:42:00.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.913+0000 7f26189da700 1 -- 172.21.15.32:0/1536681630 shutdown_connections 2020-12-05T17:42:00.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.913+0000 7f26189da700 1 --2- 172.21.15.32:0/1536681630 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26140b1720 0x7f261414a9c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.913+0000 7f26189da700 1 -- 172.21.15.32:0/1536681630 >> 172.21.15.32:0/1536681630 conn(0x7f26140b6d10 msgr2=0x7f26140b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:00.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.913+0000 7f26189da700 1 -- 172.21.15.32:0/1536681630 shutdown_connections 2020-12-05T17:42:00.918 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.913+0000 7f26189da700 1 -- 172.21.15.32:0/1536681630 wait complete. 2020-12-05T17:42:00.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.913+0000 7f26189da700 1 Processor -- start 2020-12-05T17:42:00.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.913+0000 7f26189da700 1 -- start start 2020-12-05T17:42:00.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f26189da700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2614061ba0 0x7f2614142210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:00.921 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f26189da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26140b1720 0x7f2614140df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:00.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f26189da700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f261414aeb0 0x7f26141412e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:00.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f26189da700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2614149ea0 con 0x7f26140b1720 2020-12-05T17:42:00.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f26189da700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2614148b00 con 0x7f2614061ba0 2020-12-05T17:42:00.922 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f26189da700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2614148fa0 con 0x7f261414aeb0 2020-12-05T17:42:00.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f2612ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26140b1720 0x7f2614140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:00.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f2612ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26140b1720 0x7f2614140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41688/0 (socket says 172.21.15.32:41688) 2020-12-05T17:42:00.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f2612ffd700 1 -- 172.21.15.32:0/2269363675 learned_addr learned my addr 172.21.15.32:0/2269363675 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:00.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f26137fe700 1 --2- 172.21.15.32:0/2269363675 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2614061ba0 0x7f2614142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:00.923 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f2613fff700 1 --2- 172.21.15.32:0/2269363675 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f261414aeb0 0x7f26141412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:00.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f2612ffd700 1 -- 172.21.15.32:0/2269363675 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f261414aeb0 msgr2=0x7f26141412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f2612ffd700 1 --2- 172.21.15.32:0/2269363675 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f261414aeb0 0x7f26141412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f2612ffd700 1 -- 172.21.15.32:0/2269363675 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2614061ba0 msgr2=0x7f2614142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:00.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f2612ffd700 1 --2- 172.21.15.32:0/2269363675 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2614061ba0 0x7f2614142210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:00.924 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f2612ffd700 1 -- 172.21.15.32:0/2269363675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2614149370 con 0x7f26140b1720 2020-12-05T17:42:00.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f2613fff700 1 --2- 172.21.15.32:0/2269363675 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f261414aeb0 0x7f26141412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:42:00.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f26137fe700 1 --2- 172.21.15.32:0/2269363675 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2614061ba0 0x7f2614142210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:42:00.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f2612ffd700 1 --2- 172.21.15.32:0/2269363675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26140b1720 0x7f2614140df0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7f2608002f30 tx=0x7f2608015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:00.925 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f2610ff9700 1 -- 172.21.15.32:0/2269363675 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2608011b00 con 0x7f26140b1720 2020-12-05T17:42:00.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f2610ff9700 1 -- 172.21.15.32:0/2269363675 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2608011c60 con 0x7f26140b1720 2020-12-05T17:42:00.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f26189da700 1 -- 172.21.15.32:0/2269363675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26141417d0 con 0x7f26140b1720 2020-12-05T17:42:00.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f2610ff9700 1 -- 172.21.15.32:0/2269363675 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f260801d450 con 0x7f26140b1720 2020-12-05T17:42:00.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f26189da700 1 -- 172.21.15.32:0/2269363675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26140b0990 con 0x7f26140b1720 2020-12-05T17:42:00.926 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.917+0000 7f26189da700 1 -- 172.21.15.32:0/2269363675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2614148c90 con 0x7f26140b1720 2020-12-05T17:42:00.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.921+0000 7f2610ff9700 1 -- 172.21.15.32:0/2269363675 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2608024030 con 0x7f26140b1720 2020-12-05T17:42:00.927 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.921+0000 7f2610ff9700 1 --2- 172.21.15.32:0/2269363675 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f25f4055330 0x7f25f40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:00.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.921+0000 7f26137fe700 1 --2- 172.21.15.32:0/2269363675 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f25f4055330 0x7f25f40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:00.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.921+0000 7f2610ff9700 1 -- 172.21.15.32:0/2269363675 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f260806f290 con 0x7f26140b1720 2020-12-05T17:42:00.928 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.921+0000 7f26137fe700 1 --2- 172.21.15.32:0/2269363675 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f25f4055330 0x7f25f40577f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f260400b000 tx=0x7f2604009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:00.931 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:00.929+0000 7f2610ff9700 1 -- 172.21.15.32:0/2269363675 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2608045320 con 0x7f26140b1720 2020-12-05T17:42:01.077 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.073+0000 7f26189da700 1 -- 172.21.15.32:0/2269363675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2614057050 con 0x7f26140b1720 2020-12-05T17:42:01.078 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.077+0000 7f2610ff9700 1 -- 172.21.15.32:0/2269363675 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v148) v1 ==== 100+0+10 (secure 0 0 0) 0x7f2608022020 con 0x7f26140b1720 2020-12-05T17:42:01.078 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:42:01.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.077+0000 7f26189da700 1 -- 172.21.15.32:0/2269363675 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f25f4055330 msgr2=0x7f25f40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:01.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.077+0000 7f26189da700 1 --2- 172.21.15.32:0/2269363675 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f25f4055330 0x7f25f40577f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f260400b000 tx=0x7f2604009040).stop 2020-12-05T17:42:01.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.077+0000 7f26189da700 1 -- 172.21.15.32:0/2269363675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26140b1720 msgr2=0x7f2614140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:01.080 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.077+0000 7f26189da700 1 --2- 172.21.15.32:0/2269363675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26140b1720 0x7f2614140df0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7f2608002f30 tx=0x7f2608015000).stop 2020-12-05T17:42:01.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.077+0000 7f26189da700 1 -- 172.21.15.32:0/2269363675 shutdown_connections 2020-12-05T17:42:01.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.077+0000 7f26189da700 1 --2- 172.21.15.32:0/2269363675 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f25f4055330 0x7f25f40577f0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:01.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.077+0000 7f26189da700 1 --2- 172.21.15.32:0/2269363675 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f261414aeb0 0x7f26141412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:01.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.077+0000 7f26189da700 1 --2- 172.21.15.32:0/2269363675 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2614061ba0 0x7f2614142210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:01.081 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.077+0000 7f26189da700 1 --2- 172.21.15.32:0/2269363675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26140b1720 0x7f2614140df0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:01.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.077+0000 7f26189da700 1 -- 172.21.15.32:0/2269363675 >> 172.21.15.32:0/2269363675 conn(0x7f26140b6d10 msgr2=0x7f2614146a20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:01.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.077+0000 7f26189da700 1 -- 172.21.15.32:0/2269363675 shutdown_connections 2020-12-05T17:42:01.082 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.077+0000 7f26189da700 1 -- 172.21.15.32:0/2269363675 wait complete. 2020-12-05T17:42:01.092 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:42:01.092 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:42:01.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.205+0000 7ffa2f2c7700 1 Processor -- start 2020-12-05T17:42:01.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.205+0000 7ffa2f2c7700 1 -- start start 2020-12-05T17:42:01.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.205+0000 7ffa2f2c7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa280b1fc0 0x7ffa280af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:01.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.205+0000 7ffa2f2c7700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ffa28147400 0x7ffa28145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:01.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.205+0000 7ffa2f2c7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa28146a90 con 0x7ffa280ab710 2020-12-05T17:42:01.212 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.205+0000 7ffa2f2c7700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa28061e90 con 0x7ffa28147400 2020-12-05T17:42:01.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.205+0000 7ffa2f2c7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa280b0110 con 0x7ffa280b1fc0 2020-12-05T17:42:01.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.205+0000 7ffa2dac4700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ffa280ab710 0x7ffa28061780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44342/0 (socket says 172.21.15.32:44342) 2020-12-05T17:42:01.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2dac4700 1 -- 172.21.15.32:0/1344590529 learned_addr learned my addr 172.21.15.32:0/1344590529 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:42:01.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2e2c5700 1 --2- 172.21.15.32:0/1344590529 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa280b1fc0 0x7ffa280af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:01.213 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2eac6700 1 --2- 172.21.15.32:0/1344590529 >> v2:172.21.15.61:3300/0 conn(0x7ffa28147400 0x7ffa28145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:01.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2d2c3700 1 -- 172.21.15.32:0/1344590529 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198150456 0 0) 0x7ffa28146a90 con 0x7ffa280ab710 2020-12-05T17:42:01.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2d2c3700 1 -- 172.21.15.32:0/1344590529 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa14003230 con 0x7ffa280ab710 2020-12-05T17:42:01.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2d2c3700 1 -- 172.21.15.32:0/1344590529 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1287393992 0 0) 0x7ffa14003230 con 0x7ffa280ab710 2020-12-05T17:42:01.214 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2d2c3700 1 -- 172.21.15.32:0/1344590529 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa280b1fc0 msgr2=0x7ffa280af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:42:01.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2d2c3700 1 --2- 172.21.15.32:0/1344590529 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa280b1fc0 0x7ffa280af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:01.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2d2c3700 1 -- 172.21.15.32:0/1344590529 >> v2:172.21.15.61:3300/0 conn(0x7ffa28147400 msgr2=0x7ffa28145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:42:01.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2d2c3700 1 --2- 172.21.15.32:0/1344590529 >> v2:172.21.15.61:3300/0 conn(0x7ffa28147400 0x7ffa28145d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:01.215 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2d2c3700 1 -- 172.21.15.32:0/1344590529 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa280b23a0 con 0x7ffa280ab710 2020-12-05T17:42:01.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2d2c3700 1 -- 172.21.15.32:0/1344590529 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7ffa18001a50 con 0x7ffa280ab710 2020-12-05T17:42:01.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2d2c3700 1 -- 172.21.15.32:0/1344590529 >> v1:172.21.15.32:6789/0 conn(0x7ffa280ab710 legacy=0x7ffa28061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:01.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2d2c3700 1 --2- 172.21.15.32:0/1344590529 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa280b1fc0 0x7ffa14003720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:01.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2d2c3700 1 --2- 172.21.15.32:0/1344590529 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa14003c30 0x7ffa140044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:01.216 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2d2c3700 1 --2- 172.21.15.32:0/1344590529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa140049c0 0x7ffa14004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:01.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2d2c3700 1 -- 172.21.15.32:0/1344590529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa28061e90 con 0x7ffa140049c0 2020-12-05T17:42:01.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2d2c3700 1 -- 172.21.15.32:0/1344590529 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa140041c0 con 0x7ffa14003c30 2020-12-05T17:42:01.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2d2c3700 1 -- 172.21.15.32:0/1344590529 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1400b720 con 0x7ffa280b1fc0 2020-12-05T17:42:01.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2eac6700 1 --2- 172.21.15.32:0/1344590529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa140049c0 0x7ffa14004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:01.217 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2e2c5700 1 --2- 172.21.15.32:0/1344590529 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa280b1fc0 0x7ffa14003720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2dac4700 1 --2- 172.21.15.32:0/1344590529 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa14003c30 0x7ffa140044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2eac6700 1 -- 172.21.15.32:0/1344590529 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa280b1fc0 msgr2=0x7ffa14003720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:42:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2eac6700 1 --2- 172.21.15.32:0/1344590529 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa280b1fc0 0x7ffa14003720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2eac6700 1 -- 172.21.15.32:0/1344590529 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa14003c30 msgr2=0x7ffa140044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:42:01.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2eac6700 1 --2- 172.21.15.32:0/1344590529 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa14003c30 0x7ffa140044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:01.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2eac6700 1 -- 172.21.15.32:0/1344590529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa14003230 con 0x7ffa140049c0 2020-12-05T17:42:01.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2eac6700 1 --2- 172.21.15.32:0/1344590529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa140049c0 0x7ffa14004e40 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7ffa2000c000 tx=0x7ffa20008040).ready entity=mon.0 client_cookie=ea48588fbe3eca6e server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:01.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2d2c3700 1 -- 172.21.15.32:0/1344590529 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa2000e910 con 0x7ffa140049c0 2020-12-05T17:42:01.219 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2d2c3700 1 -- 172.21.15.32:0/1344590529 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2000ea70 con 0x7ffa140049c0 2020-12-05T17:42:01.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2d2c3700 1 -- 172.21.15.32:0/1344590529 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa2000edd0 con 0x7ffa140049c0 2020-12-05T17:42:01.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2f2c7700 1 -- 172.21.15.32:0/1344590529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa140049c0 msgr2=0x7ffa14004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:01.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2f2c7700 1 --2- 172.21.15.32:0/1344590529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa140049c0 0x7ffa14004e40 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7ffa2000c000 tx=0x7ffa20008040).stop 2020-12-05T17:42:01.220 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2eac6700 1 -- 172.21.15.32:0/1344590529 reap_dead start 2020-12-05T17:42:01.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2f2c7700 1 -- 172.21.15.32:0/1344590529 shutdown_connections 2020-12-05T17:42:01.223 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2f2c7700 1 -- 172.21.15.32:0/1344590529 >> 172.21.15.32:0/1344590529 conn(0x7ffa280b5530 msgr2=0x7ffa280b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:01.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2f2c7700 1 -- 172.21.15.32:0/1344590529 shutdown_connections 2020-12-05T17:42:01.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.209+0000 7ffa2f2c7700 1 -- 172.21.15.32:0/1344590529 wait complete. 2020-12-05T17:42:01.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa2f2c7700 1 Processor -- start 2020-12-05T17:42:01.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa2f2c7700 1 -- start start 2020-12-05T17:42:01.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa2f2c7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa280a0e80 0x7ffa28141740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:01.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa2f2c7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa28066c70 0x7ffa28141c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:01.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa2f2c7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa281557c0 0x7ffa28155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:01.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa2f2c7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa281553d0 con 0x7ffa28066c70 2020-12-05T17:42:01.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa2f2c7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2809f820 con 0x7ffa281557c0 2020-12-05T17:42:01.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa2f2c7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa280a1260 con 0x7ffa280a0e80 2020-12-05T17:42:01.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa2dac4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa28066c70 0x7ffa28141c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:01.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa2dac4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa28066c70 0x7ffa28141c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41706/0 (socket says 172.21.15.32:41706) 2020-12-05T17:42:01.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa2dac4700 1 -- 172.21.15.32:0/707342356 learned_addr learned my addr 172.21.15.32:0/707342356 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:01.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa2e2c5700 1 --2- 172.21.15.32:0/707342356 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa280a0e80 0x7ffa28141740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:01.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa2eac6700 1 --2- 172.21.15.32:0/707342356 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa281557c0 0x7ffa28155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:01.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa2dac4700 1 -- 172.21.15.32:0/707342356 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa280a0e80 msgr2=0x7ffa28141740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:01.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa2dac4700 1 --2- 172.21.15.32:0/707342356 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa280a0e80 0x7ffa28141740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:01.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa2dac4700 1 -- 172.21.15.32:0/707342356 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa281557c0 msgr2=0x7ffa28155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:01.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa2dac4700 1 --2- 172.21.15.32:0/707342356 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa281557c0 0x7ffa28155c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:01.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa2dac4700 1 -- 172.21.15.32:0/707342356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa28147e50 con 0x7ffa28066c70 2020-12-05T17:42:01.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa2dac4700 1 --2- 172.21.15.32:0/707342356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa28066c70 0x7ffa28141c30 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7ffa280a13f0 tx=0x7ffa280612b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:01.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa277fe700 1 -- 172.21.15.32:0/707342356 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa1800fd70 con 0x7ffa28066c70 2020-12-05T17:42:01.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa277fe700 1 -- 172.21.15.32:0/707342356 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa1800fed0 con 0x7ffa28066c70 2020-12-05T17:42:01.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa2f2c7700 1 -- 172.21.15.32:0/707342356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa2809fd10 con 0x7ffa28066c70 2020-12-05T17:42:01.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa277fe700 1 -- 172.21.15.32:0/707342356 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa18012540 con 0x7ffa28066c70 2020-12-05T17:42:01.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa2f2c7700 1 -- 172.21.15.32:0/707342356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa2809f120 con 0x7ffa28066c70 2020-12-05T17:42:01.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.213+0000 7ffa2f2c7700 1 -- 172.21.15.32:0/707342356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa28057050 con 0x7ffa28066c70 2020-12-05T17:42:01.229 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.217+0000 7ffa277fe700 1 -- 172.21.15.32:0/707342356 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ffa1801b030 con 0x7ffa28066c70 2020-12-05T17:42:01.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.217+0000 7ffa277fe700 1 --2- 172.21.15.32:0/707342356 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffa080553b0 0x7ffa08057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:01.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.217+0000 7ffa2e2c5700 1 --2- 172.21.15.32:0/707342356 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffa080553b0 0x7ffa08057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:01.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.217+0000 7ffa277fe700 1 -- 172.21.15.32:0/707342356 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ffa1806dd30 con 0x7ffa28066c70 2020-12-05T17:42:01.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.217+0000 7ffa2e2c5700 1 --2- 172.21.15.32:0/707342356 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffa080553b0 0x7ffa08057870 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7ffa1c008000 tx=0x7ffa1c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:01.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.225+0000 7ffa277fe700 1 -- 172.21.15.32:0/707342356 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ffa18043ed0 con 0x7ffa28066c70 2020-12-05T17:42:01.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.409+0000 7ffa2f2c7700 1 -- 172.21.15.32:0/707342356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa2805aab0 con 0x7ffa28066c70 2020-12-05T17:42:01.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.409+0000 7ffa277fe700 1 -- 172.21.15.32:0/707342356 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+2139 (secure 0 0 0) 0x7ffa18019030 con 0x7ffa28066c70 2020-12-05T17:42:01.411 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:42:01.411 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":73,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8473,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/711010965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":711010965},{"type":"v1","addr":"172.21.15.61:6833","nonce":711010965}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":70}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:41:55.663831+0000","modified":"2020-12-05T17:41:57.203619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8580":{"gid":8580,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1575496107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1575496107},{"type":"v1","addr":"172.21.15.32:6835","nonce":1575496107}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2020-12-05T17:42:01.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.409+0000 7ffa2f2c7700 1 -- 172.21.15.32:0/707342356 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffa080553b0 msgr2=0x7ffa08057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:01.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.409+0000 7ffa2f2c7700 1 --2- 172.21.15.32:0/707342356 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffa080553b0 0x7ffa08057870 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7ffa1c008000 tx=0x7ffa1c008040).stop 2020-12-05T17:42:01.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.409+0000 7ffa2f2c7700 1 -- 172.21.15.32:0/707342356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa28066c70 msgr2=0x7ffa28141c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:01.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.409+0000 7ffa2f2c7700 1 --2- 172.21.15.32:0/707342356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa28066c70 0x7ffa28141c30 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7ffa280a13f0 tx=0x7ffa280612b0).stop 2020-12-05T17:42:01.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.409+0000 7ffa2f2c7700 1 -- 172.21.15.32:0/707342356 shutdown_connections 2020-12-05T17:42:01.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.409+0000 7ffa2f2c7700 1 --2- 172.21.15.32:0/707342356 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa281557c0 0x7ffa28155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:01.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.409+0000 7ffa2f2c7700 1 --2- 172.21.15.32:0/707342356 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7ffa080553b0 0x7ffa08057870 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:01.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.409+0000 7ffa2f2c7700 1 --2- 172.21.15.32:0/707342356 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa280a0e80 0x7ffa28141740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:01.414 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.409+0000 7ffa2f2c7700 1 --2- 172.21.15.32:0/707342356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa28066c70 0x7ffa28141c30 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:01.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.409+0000 7ffa2f2c7700 1 -- 172.21.15.32:0/707342356 >> 172.21.15.32:0/707342356 conn(0x7ffa280b5530 msgr2=0x7ffa280b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:01.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.409+0000 7ffa2f2c7700 1 -- 172.21.15.32:0/707342356 shutdown_connections 2020-12-05T17:42:01.415 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:01.409+0000 7ffa2f2c7700 1 -- 172.21.15.32:0/707342356 wait complete. 2020-12-05T17:42:01.416 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 73 2020-12-05T17:42:01.627 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:41:55.663831+0000', 'modified': '2020-12-05T17:41:57.203619+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8580}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8580': {'gid': 8580, 'name': 'a', 'rank': 0, 'incarnation': 71, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/1575496107', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 1575496107}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 1575496107}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:42:01.627 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:42:01.628 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:42:01.757 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":8580,"want_state":"up:active","state":"up:active","rank_uptime":6.0804627470000003,"mdsmap_epoch":73,"osdmap_epoch":148,"osdmap_epoch_barrier":148,"uptime":11.448836638} 2020-12-05T17:42:01.757 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 8580, "mdsmap_epoch": 73, "osdmap_epoch": 148, "osdmap_epoch_barrier": 148, "rank_uptime": 6.080462747, "state": "up:active", "uptime": 11.448836638, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:42:01.758 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:42:01.758 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-12-05T17:42:01.759 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-12-05T17:42:01.759 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:42:01.759 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:42:01.768 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:42:01.827 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:42:01.827 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:42:01.882 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:42:01.883 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:42:01.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:42:01.884 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:42:01.934 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:42:01.935 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:42:01.977 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:42:01.994 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:42:01.994 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:42:02.050 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:42:02.051 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:42:02.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:42:02.052 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:42:02.061 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[11766]: starting ceph client2020-12-05T17:42:02.057+0000 7f2ffc2250c0 -1 init, newargv = 0x562e9a1a78c0 newargc=9 2020-12-05T17:42:02.061 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr: 2020-12-05T17:42:02.071 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi036.stderr:ceph-fuse[11766]: starting fuse 2020-12-05T17:42:03.167 INFO:teuthology.orchestra.run.smithi036:> sudo modprobe fuse 2020-12-05T17:42:03.183 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:42:03.183 INFO:teuthology.orchestra.run.smithi036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:42:03.238 INFO:teuthology.orchestra.run.smithi036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:42:03.239 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:42:03.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:42:03.240 INFO:teuthology.orchestra.run.smithi036:> ls /sys/fs/fuse/connections 2020-12-05T17:42:03.289 INFO:teuthology.orchestra.run.smithi036.stdout:49 2020-12-05T17:42:03.289 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-12-05T17:42:03.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:42:03.290 INFO:teuthology.orchestra.run.smithi036:> sudo python3 -c ' 2020-12-05T17:42:03.291 INFO:teuthology.orchestra.run.smithi036:> import glob 2020-12-05T17:42:03.291 INFO:teuthology.orchestra.run.smithi036:> import re 2020-12-05T17:42:03.291 INFO:teuthology.orchestra.run.smithi036:> import os 2020-12-05T17:42:03.292 INFO:teuthology.orchestra.run.smithi036:> import subprocess 2020-12-05T17:42:03.292 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:42:03.292 INFO:teuthology.orchestra.run.smithi036:> def find_socket(client_name): 2020-12-05T17:42:03.293 INFO:teuthology.orchestra.run.smithi036:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-12-05T17:42:03.293 INFO:teuthology.orchestra.run.smithi036:> files = glob.glob(asok_path) 2020-12-05T17:42:03.294 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:42:03.294 INFO:teuthology.orchestra.run.smithi036:> # Given a non-glob path, it better be there 2020-12-05T17:42:03.294 INFO:teuthology.orchestra.run.smithi036:> if "*" not in asok_path: 2020-12-05T17:42:03.295 INFO:teuthology.orchestra.run.smithi036:> assert(len(files) == 1) 2020-12-05T17:42:03.295 INFO:teuthology.orchestra.run.smithi036:> return files[0] 2020-12-05T17:42:03.295 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:42:03.296 INFO:teuthology.orchestra.run.smithi036:> for f in files: 2020-12-05T17:42:03.296 INFO:teuthology.orchestra.run.smithi036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:42:03.296 INFO:teuthology.orchestra.run.smithi036:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:42:03.297 INFO:teuthology.orchestra.run.smithi036:> return f 2020-12-05T17:42:03.297 INFO:teuthology.orchestra.run.smithi036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:42:03.297 INFO:teuthology.orchestra.run.smithi036:> 2020-12-05T17:42:03.298 INFO:teuthology.orchestra.run.smithi036:> print(find_socket("client.0")) 2020-12-05T17:42:03.298 INFO:teuthology.orchestra.run.smithi036:> ' 2020-12-05T17:42:03.380 INFO:teuthology.orchestra.run.smithi036.stdout:/var/run/ceph/ceph-client.0.11766.asok 2020-12-05T17:42:03.385 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.11766.asok 2020-12-05T17:42:03.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:42:03.386 INFO:teuthology.orchestra.run.smithi036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.11766.asok status 2020-12-05T17:42:03.481 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2020-12-05T17:42:03.482 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": { 2020-12-05T17:42:03.482 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:42:03.482 INFO:teuthology.orchestra.run.smithi036.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:42:03.483 INFO:teuthology.orchestra.run.smithi036.stdout: "entity_id": "0", 2020-12-05T17:42:03.483 INFO:teuthology.orchestra.run.smithi036.stdout: "hostname": "smithi036", 2020-12-05T17:42:03.483 INFO:teuthology.orchestra.run.smithi036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-12-05T17:42:03.483 INFO:teuthology.orchestra.run.smithi036.stdout: "pid": "11766", 2020-12-05T17:42:03.484 INFO:teuthology.orchestra.run.smithi036.stdout: "root": "/" 2020-12-05T17:42:03.484 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:42:03.484 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_count": 0, 2020-12-05T17:42:03.484 INFO:teuthology.orchestra.run.smithi036.stdout: "dentry_pinned_count": 0, 2020-12-05T17:42:03.484 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 8748, 2020-12-05T17:42:03.485 INFO:teuthology.orchestra.run.smithi036.stdout: "inst": { 2020-12-05T17:42:03.485 INFO:teuthology.orchestra.run.smithi036.stdout: "name": { 2020-12-05T17:42:03.485 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "client", 2020-12-05T17:42:03.485 INFO:teuthology.orchestra.run.smithi036.stdout: "num": 8748 2020-12-05T17:42:03.485 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:42:03.486 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:42:03.486 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:42:03.486 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:42:03.486 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 3398456363 2020-12-05T17:42:03.487 INFO:teuthology.orchestra.run.smithi036.stdout: } 2020-12-05T17:42:03.487 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:42:03.487 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": { 2020-12-05T17:42:03.487 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2020-12-05T17:42:03.487 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:0", 2020-12-05T17:42:03.488 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 3398456363 2020-12-05T17:42:03.488 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2020-12-05T17:42:03.488 INFO:teuthology.orchestra.run.smithi036.stdout: "inst_str": "client.8748 172.21.15.36:0/3398456363", 2020-12-05T17:42:03.488 INFO:teuthology.orchestra.run.smithi036.stdout: "addr_str": "172.21.15.36:0/3398456363", 2020-12-05T17:42:03.489 INFO:teuthology.orchestra.run.smithi036.stdout: "inode_count": 1, 2020-12-05T17:42:03.489 INFO:teuthology.orchestra.run.smithi036.stdout: "mds_epoch": 73, 2020-12-05T17:42:03.489 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch": 148, 2020-12-05T17:42:03.489 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:42:03.489 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklisted": false 2020-12-05T17:42:03.490 INFO:teuthology.orchestra.run.smithi036.stdout:} 2020-12-05T17:42:03.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:42:03.492 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:42:03.492 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:42:03.499 INFO:teuthology.orchestra.run.smithi036.stdout:fuseblk 2020-12-05T17:42:03.500 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-12-05T17:42:03.501 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:42:03.501 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:42:03.561 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T17:42:03.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.677+0000 7f5183fff700 1 Processor -- start 2020-12-05T17:42:03.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f5183fff700 1 -- start start 2020-12-05T17:42:03.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f5183fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f518405f3b0 0x7f518405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:03.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f5183fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5184142450 con 0x7f518405feb0 2020-12-05T17:42:03.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f5183fff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5184061780 con 0x7f5184062400 2020-12-05T17:42:03.684 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f5183fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5184061a20 con 0x7f518405f3b0 2020-12-05T17:42:03.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51837fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f518405feb0 0x7f51840b0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44364/0 (socket says 172.21.15.32:44364) 2020-12-05T17:42:03.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51837fe700 1 -- 172.21.15.32:0/3785657204 learned_addr learned my addr 172.21.15.32:0/3785657204 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:42:03.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f5182ffd700 1 --2- 172.21.15.32:0/3785657204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f518405f3b0 0x7f518405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:03.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51827fc700 1 -- 172.21.15.32:0/3785657204 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139223399 0 0) 0x7f5184142450 con 0x7f518405feb0 2020-12-05T17:42:03.685 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51827fc700 1 -- 172.21.15.32:0/3785657204 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5168003230 con 0x7f518405feb0 2020-12-05T17:42:03.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51827fc700 1 -- 172.21.15.32:0/3785657204 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 494283894 0 0) 0x7f5184061780 con 0x7f5184062400 2020-12-05T17:42:03.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51827fc700 1 -- 172.21.15.32:0/3785657204 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5184142450 con 0x7f5184062400 2020-12-05T17:42:03.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51827fc700 1 -- 172.21.15.32:0/3785657204 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2881690440 0 0) 0x7f5168003230 con 0x7f518405feb0 2020-12-05T17:42:03.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51827fc700 1 -- 172.21.15.32:0/3785657204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f518405f3b0 msgr2=0x7f518405f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:42:03.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51827fc700 1 --2- 172.21.15.32:0/3785657204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f518405f3b0 0x7f518405f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51827fc700 1 -- 172.21.15.32:0/3785657204 >> v1:172.21.15.61:6789/0 conn(0x7f5184062400 legacy=0x7f51840b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51827fc700 1 -- 172.21.15.32:0/3785657204 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5184061cc0 con 0x7f518405feb0 2020-12-05T17:42:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51827fc700 1 -- 172.21.15.32:0/3785657204 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f517c0019d0 con 0x7f518405feb0 2020-12-05T17:42:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51827fc700 1 -- 172.21.15.32:0/3785657204 >> v1:172.21.15.32:6789/0 conn(0x7f518405feb0 legacy=0x7f51840b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51827fc700 1 --2- 172.21.15.32:0/3785657204 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f51680038e0 0x7f5168003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:03.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51827fc700 1 --2- 172.21.15.32:0/3785657204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51680044d0 0x7f5168004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:03.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51827fc700 1 --2- 172.21.15.32:0/3785657204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f518405f3b0 0x7f5168004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:03.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51827fc700 1 -- 172.21.15.32:0/3785657204 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5184061a20 con 0x7f51680044d0 2020-12-05T17:42:03.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51827fc700 1 -- 172.21.15.32:0/3785657204 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5168005360 con 0x7f51680038e0 2020-12-05T17:42:03.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51827fc700 1 -- 172.21.15.32:0/3785657204 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516800b770 con 0x7f518405f3b0 2020-12-05T17:42:03.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f517bfff700 1 --2- 172.21.15.32:0/3785657204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51680044d0 0x7f5168004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:03.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51837fe700 1 --2- 172.21.15.32:0/3785657204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f518405f3b0 0x7f5168004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:03.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f5182ffd700 1 --2- 172.21.15.32:0/3785657204 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f51680038e0 0x7f5168003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:03.699 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f517bfff700 1 -- 172.21.15.32:0/3785657204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f518405f3b0 msgr2=0x7f5168004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:42:03.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f517bfff700 1 --2- 172.21.15.32:0/3785657204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f518405f3b0 0x7f5168004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:03.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f517bfff700 1 -- 172.21.15.32:0/3785657204 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f51680038e0 msgr2=0x7f5168003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:42:03.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f517bfff700 1 --2- 172.21.15.32:0/3785657204 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f51680038e0 0x7f5168003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:03.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f517bfff700 1 -- 172.21.15.32:0/3785657204 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5168003230 con 0x7f51680044d0 2020-12-05T17:42:03.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51837fe700 1 --2- 172.21.15.32:0/3785657204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f518405f3b0 0x7f5168004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:42:03.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f517bfff700 1 --2- 172.21.15.32:0/3785657204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51680044d0 0x7f5168004950 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7f51680054a0 tx=0x7f516c007b60).ready entity=mon.0 client_cookie=c8475f28ea56dac7 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:03.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51827fc700 1 -- 172.21.15.32:0/3785657204 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f516c00c7d0 con 0x7f51680044d0 2020-12-05T17:42:03.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51827fc700 1 -- 172.21.15.32:0/3785657204 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c00c930 con 0x7f51680044d0 2020-12-05T17:42:03.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51827fc700 1 -- 172.21.15.32:0/3785657204 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f516c00cc90 con 0x7f51680044d0 2020-12-05T17:42:03.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f5183fff700 1 -- 172.21.15.32:0/3785657204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51680044d0 msgr2=0x7f5168004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:03.703 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f5183fff700 1 --2- 172.21.15.32:0/3785657204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51680044d0 0x7f5168004950 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7f51680054a0 tx=0x7f516c007b60).stop 2020-12-05T17:42:03.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.681+0000 7f51837fe700 1 -- 172.21.15.32:0/3785657204 reap_dead start 2020-12-05T17:42:03.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f5183fff700 1 -- 172.21.15.32:0/3785657204 shutdown_connections 2020-12-05T17:42:03.704 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f5183fff700 1 -- 172.21.15.32:0/3785657204 >> 172.21.15.32:0/3785657204 conn(0x7f51840b8ee0 msgr2=0x7f51840b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:03.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f5183fff700 1 -- 172.21.15.32:0/3785657204 shutdown_connections 2020-12-05T17:42:03.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f5183fff700 1 -- 172.21.15.32:0/3785657204 wait complete. 2020-12-05T17:42:03.705 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f5183fff700 1 Processor -- start 2020-12-05T17:42:03.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f5183fff700 1 -- start start 2020-12-05T17:42:03.706 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f5183fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5184063150 0x7f51841459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:03.707 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f5183fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5184144bf0 0x7f5184145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:03.707 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f5183fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5184154970 0x7f5184154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:03.707 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f5183fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51841463c0 con 0x7f5184144bf0 2020-12-05T17:42:03.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f5183fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51840651c0 con 0x7f5184063150 2020-12-05T17:42:03.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f5183fff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5184065460 con 0x7f5184154970 2020-12-05T17:42:03.708 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f517bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5184144bf0 0x7f5184145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:03.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f517bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5184144bf0 0x7f5184145ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41728/0 (socket says 172.21.15.32:41728) 2020-12-05T17:42:03.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f517bfff700 1 -- 172.21.15.32:0/3511535342 learned_addr learned my addr 172.21.15.32:0/3511535342 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:03.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f51837fe700 1 --2- 172.21.15.32:0/3511535342 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5184154970 0x7f5184154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:03.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f5182ffd700 1 --2- 172.21.15.32:0/3511535342 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5184063150 0x7f51841459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:03.710 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f517bfff700 1 -- 172.21.15.32:0/3511535342 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5184154970 msgr2=0x7f5184154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:03.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f517bfff700 1 --2- 172.21.15.32:0/3511535342 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5184154970 0x7f5184154df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:03.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f517bfff700 1 -- 172.21.15.32:0/3511535342 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5184063150 msgr2=0x7f51841459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:03.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f517bfff700 1 --2- 172.21.15.32:0/3511535342 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5184063150 0x7f51841459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:03.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f517bfff700 1 -- 172.21.15.32:0/3511535342 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5184145540 con 0x7f5184144bf0 2020-12-05T17:42:03.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f517bfff700 1 --2- 172.21.15.32:0/3511535342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5184144bf0 0x7f5184145ed0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7f51680054a0 tx=0x7f516c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:03.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f5180ff9700 1 -- 172.21.15.32:0/3511535342 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f516c001ea0 con 0x7f5184144bf0 2020-12-05T17:42:03.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f5180ff9700 1 -- 172.21.15.32:0/3511535342 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c012da0 con 0x7f5184144bf0 2020-12-05T17:42:03.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.685+0000 7f5183fff700 1 -- 172.21.15.32:0/3511535342 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5184144860 con 0x7f5184144bf0 2020-12-05T17:42:03.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.689+0000 7f5180ff9700 1 -- 172.21.15.32:0/3511535342 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f516c01d610 con 0x7f5184144bf0 2020-12-05T17:42:03.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.689+0000 7f5183fff700 1 -- 172.21.15.32:0/3511535342 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51840636d0 con 0x7f5184144bf0 2020-12-05T17:42:03.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.689+0000 7f5183fff700 1 -- 172.21.15.32:0/3511535342 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5184057050 con 0x7f5184144bf0 2020-12-05T17:42:03.714 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.689+0000 7f5180ff9700 1 -- 172.21.15.32:0/3511535342 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f516c024030 con 0x7f5184144bf0 2020-12-05T17:42:03.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.689+0000 7f5180ff9700 1 --2- 172.21.15.32:0/3511535342 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f515c055470 0x7f515c057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:03.715 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.689+0000 7f5182ffd700 1 --2- 172.21.15.32:0/3511535342 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f515c055470 0x7f515c057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:03.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.689+0000 7f5180ff9700 1 -- 172.21.15.32:0/3511535342 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f516c06eac0 con 0x7f5184144bf0 2020-12-05T17:42:03.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.693+0000 7f5182ffd700 1 --2- 172.21.15.32:0/3511535342 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f515c055470 0x7f515c057930 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f5174008000 tx=0x7f5174008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:03.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.697+0000 7f5180ff9700 1 -- 172.21.15.32:0/3511535342 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f516c044b50 con 0x7f5184144bf0 2020-12-05T17:42:03.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.833+0000 7f5183fff700 1 -- 172.21.15.32:0/3511535342 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51841449f0 con 0x7f5184144bf0 2020-12-05T17:42:03.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.837+0000 7f5180ff9700 1 -- 172.21.15.32:0/3511535342 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f516c022070 con 0x7f5184144bf0 2020-12-05T17:42:03.838 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:42:03.838 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":148,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:41:56.666368+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2020-12-05T17:41:53.238803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2020-12-05T17:41:54.541493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T17:42:03.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.837+0000 7f5183fff700 1 -- 172.21.15.32:0/3511535342 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f515c055470 msgr2=0x7f515c057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:03.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.837+0000 7f5183fff700 1 --2- 172.21.15.32:0/3511535342 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f515c055470 0x7f515c057930 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f5174008000 tx=0x7f5174008040).stop 2020-12-05T17:42:03.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.837+0000 7f5183fff700 1 -- 172.21.15.32:0/3511535342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5184144bf0 msgr2=0x7f5184145ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:03.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.837+0000 7f5183fff700 1 --2- 172.21.15.32:0/3511535342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5184144bf0 0x7f5184145ed0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7f51680054a0 tx=0x7f516c017000).stop 2020-12-05T17:42:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.837+0000 7f5183fff700 1 -- 172.21.15.32:0/3511535342 shutdown_connections 2020-12-05T17:42:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.837+0000 7f5183fff700 1 --2- 172.21.15.32:0/3511535342 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f515c055470 0x7f515c057930 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.837+0000 7f5183fff700 1 --2- 172.21.15.32:0/3511535342 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5184154970 0x7f5184154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.837+0000 7f5183fff700 1 --2- 172.21.15.32:0/3511535342 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5184063150 0x7f51841459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.837+0000 7f5183fff700 1 --2- 172.21.15.32:0/3511535342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5184144bf0 0x7f5184145ed0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:03.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.837+0000 7f5183fff700 1 -- 172.21.15.32:0/3511535342 >> 172.21.15.32:0/3511535342 conn(0x7f51840b8ee0 msgr2=0x7f51840b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:03.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.837+0000 7f5183fff700 1 -- 172.21.15.32:0/3511535342 shutdown_connections 2020-12-05T17:42:03.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.837+0000 7f5183fff700 1 -- 172.21.15.32:0/3511535342 wait complete. 2020-12-05T17:42:03.852 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-12-05T17:31:24.858732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-12-05T17:31:33.305595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2020-12-05T17:41:53.238803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2020-12-05T17:41:54.541493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-12-05T17:42:03.853 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-12-05T17:42:03.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.969+0000 7fcb05ccf700 1 Processor -- start 2020-12-05T17:42:03.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.969+0000 7fcb05ccf700 1 -- start start 2020-12-05T17:42:03.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb05ccf700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fcb0005f3b0 0x7fcb0005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:03.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb05ccf700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb0005feb0 0x7fcb00061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:03.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb05ccf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb000b2fb0 0x7fcb0013c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:03.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb05ccf700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb000b2df0 con 0x7fcb000b2fb0 2020-12-05T17:42:03.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb05ccf700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb000b3c90 con 0x7fcb0005f3b0 2020-12-05T17:42:03.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb05ccf700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb000b2b50 con 0x7fcb0005feb0 2020-12-05T17:42:03.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb054ce700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb000b2fb0 0x7fcb0013c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:03.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb054ce700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb000b2fb0 0x7fcb0013c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41736/0 (socket says 172.21.15.32:41736) 2020-12-05T17:42:03.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb054ce700 1 -- 172.21.15.32:0/4133343666 learned_addr learned my addr 172.21.15.32:0/4133343666 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:03.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb04ccd700 1 --2- 172.21.15.32:0/4133343666 >> v2:172.21.15.61:3300/0 conn(0x7fcb0005f3b0 0x7fcb0005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:03.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcafffff700 1 --2- 172.21.15.32:0/4133343666 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb0005feb0 0x7fcb00061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:03.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb054ce700 1 -- 172.21.15.32:0/4133343666 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb0005feb0 msgr2=0x7fcb00061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:42:03.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb054ce700 1 --2- 172.21.15.32:0/4133343666 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb0005feb0 0x7fcb00061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:03.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb054ce700 1 -- 172.21.15.32:0/4133343666 >> v2:172.21.15.61:3300/0 conn(0x7fcb0005f3b0 msgr2=0x7fcb0005f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:42:03.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb054ce700 1 --2- 172.21.15.32:0/4133343666 >> v2:172.21.15.61:3300/0 conn(0x7fcb0005f3b0 0x7fcb0005f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:03.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb054ce700 1 -- 172.21.15.32:0/4133343666 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb00142820 con 0x7fcb000b2fb0 2020-12-05T17:42:03.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb04ccd700 1 --2- 172.21.15.32:0/4133343666 >> v2:172.21.15.61:3300/0 conn(0x7fcb0005f3b0 0x7fcb0005f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:42:03.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb054ce700 1 --2- 172.21.15.32:0/4133343666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb000b2fb0 0x7fcb0013c3b0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 rx=0x7fcaf80032b0 tx=0x7fcaf8009270).ready entity=mon.0 client_cookie=d4ebeeda4770874 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:03.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcaff7fe700 1 -- 172.21.15.32:0/4133343666 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcaf8009e60 con 0x7fcb000b2fb0 2020-12-05T17:42:03.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcaff7fe700 1 -- 172.21.15.32:0/4133343666 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf800bb50 con 0x7fcb000b2fb0 2020-12-05T17:42:03.981 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcaff7fe700 1 -- 172.21.15.32:0/4133343666 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcaf800bcc0 con 0x7fcb000b2fb0 2020-12-05T17:42:03.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb05ccf700 1 -- 172.21.15.32:0/4133343666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb000b2fb0 msgr2=0x7fcb0013c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:03.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb05ccf700 1 --2- 172.21.15.32:0/4133343666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb000b2fb0 0x7fcb0013c3b0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 rx=0x7fcaf80032b0 tx=0x7fcaf8009270).stop 2020-12-05T17:42:03.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb05ccf700 1 -- 172.21.15.32:0/4133343666 shutdown_connections 2020-12-05T17:42:03.982 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb05ccf700 1 --2- 172.21.15.32:0/4133343666 >> v2:172.21.15.61:3300/0 conn(0x7fcb0005f3b0 0x7fcb0005f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:03.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb05ccf700 1 --2- 172.21.15.32:0/4133343666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb000b2fb0 0x7fcb0013c3b0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:03.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb05ccf700 1 --2- 172.21.15.32:0/4133343666 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb0005feb0 0x7fcb00061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:03.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb05ccf700 1 -- 172.21.15.32:0/4133343666 >> 172.21.15.32:0/4133343666 conn(0x7fcb000b8f20 msgr2=0x7fcb000b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:03.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb05ccf700 1 -- 172.21.15.32:0/4133343666 shutdown_connections 2020-12-05T17:42:03.983 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.973+0000 7fcb05ccf700 1 -- 172.21.15.32:0/4133343666 wait complete. 2020-12-05T17:42:03.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcb05ccf700 1 Processor -- start 2020-12-05T17:42:03.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcb05ccf700 1 -- start start 2020-12-05T17:42:03.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcb05ccf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb0005f3b0 0x7fcb0013e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:03.984 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcb05ccf700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb0005feb0 0x7fcb00154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:03.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcb05ccf700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb000b2fb0 0x7fcb00146390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:03.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcb05ccf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb00145f60 con 0x7fcb0005f3b0 2020-12-05T17:42:03.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcb05ccf700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb00144540 con 0x7fcb000b2fb0 2020-12-05T17:42:03.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcb05ccf700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb001447b0 con 0x7fcb0005feb0 2020-12-05T17:42:03.985 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcb04ccd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb0005f3b0 0x7fcb0013e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:03.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcb04ccd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb0005f3b0 0x7fcb0013e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41740/0 (socket says 172.21.15.32:41740) 2020-12-05T17:42:03.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcb04ccd700 1 -- 172.21.15.32:0/432701968 learned_addr learned my addr 172.21.15.32:0/432701968 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:03.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcafffff700 1 --2- 172.21.15.32:0/432701968 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb0005feb0 0x7fcb00154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:03.986 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcb054ce700 1 --2- 172.21.15.32:0/432701968 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb000b2fb0 0x7fcb00146390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:03.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcb04ccd700 1 -- 172.21.15.32:0/432701968 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb0005feb0 msgr2=0x7fcb00154940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:03.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcb04ccd700 1 --2- 172.21.15.32:0/432701968 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb0005feb0 0x7fcb00154940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:03.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcb04ccd700 1 -- 172.21.15.32:0/432701968 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb000b2fb0 msgr2=0x7fcb00146390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:03.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcb04ccd700 1 --2- 172.21.15.32:0/432701968 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb000b2fb0 0x7fcb00146390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:03.987 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcb04ccd700 1 -- 172.21.15.32:0/432701968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb00141830 con 0x7fcb0005f3b0 2020-12-05T17:42:03.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcb04ccd700 1 --2- 172.21.15.32:0/432701968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb0005f3b0 0x7fcb0013e970 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 rx=0x7fcb001446d0 tx=0x7fcaf0006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:03.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcafdffb700 1 -- 172.21.15.32:0/432701968 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcaf000cda0 con 0x7fcb0005f3b0 2020-12-05T17:42:03.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcafdffb700 1 -- 172.21.15.32:0/432701968 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf000ee80 con 0x7fcb0005f3b0 2020-12-05T17:42:03.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcafdffb700 1 -- 172.21.15.32:0/432701968 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcaf00125e0 con 0x7fcb0005f3b0 2020-12-05T17:42:03.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcb05ccf700 1 -- 172.21.15.32:0/432701968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb00141180 con 0x7fcb0005f3b0 2020-12-05T17:42:03.989 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.977+0000 7fcb05ccf700 1 -- 172.21.15.32:0/432701968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb00142560 con 0x7fcb0005f3b0 2020-12-05T17:42:03.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.981+0000 7fcb05ccf700 1 -- 172.21.15.32:0/432701968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb001460d0 con 0x7fcb0005f3b0 2020-12-05T17:42:03.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.981+0000 7fcafdffb700 1 -- 172.21.15.32:0/432701968 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fcaf0012740 con 0x7fcb0005f3b0 2020-12-05T17:42:03.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.981+0000 7fcafdffb700 1 --2- 172.21.15.32:0/432701968 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcae40553f0 0x7fcae40578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:03.990 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.981+0000 7fcafffff700 1 --2- 172.21.15.32:0/432701968 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcae40553f0 0x7fcae40578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:03.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.981+0000 7fcafdffb700 1 -- 172.21.15.32:0/432701968 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcaf006e750 con 0x7fcb0005f3b0 2020-12-05T17:42:03.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.981+0000 7fcafffff700 1 --2- 172.21.15.32:0/432701968 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcae40553f0 0x7fcae40578b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fcae800b000 tx=0x7fcae8006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:03.991 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:03.985+0000 7fcafdffb700 1 -- 172.21.15.32:0/432701968 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fcb001460d0 con 0x7fcb0005f3b0 2020-12-05T17:42:04.132 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.129+0000 7fcb05ccf700 1 -- 172.21.15.32:0/432701968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fcb00141460 con 0x7fcb0005f3b0 2020-12-05T17:42:04.133 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.129+0000 7fcafdffb700 1 -- 172.21.15.32:0/432701968 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v148) v1 ==== 110+0+10 (secure 0 0 0) 0x7fcaf00447e0 con 0x7fcb0005f3b0 2020-12-05T17:42:04.134 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2020-12-05T17:42:04.135 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.133+0000 7fcb05ccf700 1 -- 172.21.15.32:0/432701968 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcae40553f0 msgr2=0x7fcae40578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:04.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.133+0000 7fcb05ccf700 1 --2- 172.21.15.32:0/432701968 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcae40553f0 0x7fcae40578b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fcae800b000 tx=0x7fcae8006dc0).stop 2020-12-05T17:42:04.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.133+0000 7fcb05ccf700 1 -- 172.21.15.32:0/432701968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb0005f3b0 msgr2=0x7fcb0013e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:04.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.133+0000 7fcb05ccf700 1 --2- 172.21.15.32:0/432701968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb0005f3b0 0x7fcb0013e970 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 rx=0x7fcb001446d0 tx=0x7fcaf0006dc0).stop 2020-12-05T17:42:04.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.133+0000 7fcb05ccf700 1 -- 172.21.15.32:0/432701968 shutdown_connections 2020-12-05T17:42:04.136 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.133+0000 7fcb05ccf700 1 --2- 172.21.15.32:0/432701968 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb000b2fb0 0x7fcb00146390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:04.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.133+0000 7fcb05ccf700 1 --2- 172.21.15.32:0/432701968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb0005f3b0 0x7fcb0013e970 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:04.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.133+0000 7fcb05ccf700 1 --2- 172.21.15.32:0/432701968 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fcae40553f0 0x7fcae40578b0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:04.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.133+0000 7fcb05ccf700 1 --2- 172.21.15.32:0/432701968 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb0005feb0 0x7fcb00154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:04.137 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.133+0000 7fcb05ccf700 1 -- 172.21.15.32:0/432701968 >> 172.21.15.32:0/432701968 conn(0x7fcb000b8f20 msgr2=0x7fcb000b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:04.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.133+0000 7fcb05ccf700 1 -- 172.21.15.32:0/432701968 shutdown_connections 2020-12-05T17:42:04.138 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.133+0000 7fcb05ccf700 1 -- 172.21.15.32:0/432701968 wait complete. 2020-12-05T17:42:04.147 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-12-05T17:42:04.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.261+0000 7fd04ffff700 1 Processor -- start 2020-12-05T17:42:04.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.261+0000 7fd04ffff700 1 -- start start 2020-12-05T17:42:04.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.261+0000 7fd04ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd0500b2100 0x7fd0500af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:04.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.261+0000 7fd04ffff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0500b24e0 con 0x7fd0500ad250 2020-12-05T17:42:04.265 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.261+0000 7fd04ffff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0500b0780 con 0x7fd05013ed60 2020-12-05T17:42:04.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.261+0000 7fd04ffff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd05013fd10 con 0x7fd0500b2100 2020-12-05T17:42:04.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.261+0000 7fd04effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd0500b2100 0x7fd0500af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:04.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.261+0000 7fd04f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd0500ad250 0x7fd050142600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44400/0 (socket says 172.21.15.32:44400) 2020-12-05T17:42:04.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.261+0000 7fd04f7fe700 1 -- 172.21.15.32:0/122118326 learned_addr learned my addr 172.21.15.32:0/122118326 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:42:04.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.265+0000 7fd04effd700 1 -- 172.21.15.32:0/122118326 >> v1:172.21.15.61:6789/0 conn(0x7fd05013ed60 legacy=0x7fd050138820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:04.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.265+0000 7fd04dffb700 1 -- 172.21.15.32:0/122118326 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3790179014 0 0) 0x7fd0500b24e0 con 0x7fd0500ad250 2020-12-05T17:42:04.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.265+0000 7fd04effd700 1 -- 172.21.15.32:0/122118326 >> v1:172.21.15.32:6789/0 conn(0x7fd0500ad250 legacy=0x7fd050142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:04.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.265+0000 7fd04effd700 1 -- 172.21.15.32:0/122118326 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0500b0240 con 0x7fd0500b2100 2020-12-05T17:42:04.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.265+0000 7fd04effd700 1 --2- 172.21.15.32:0/122118326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0500b2100 0x7fd0500af360 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 rx=0x7fd0380010d0 tx=0x7fd040006dc0).ready entity=mon.0 client_cookie=5597a7505875ed2a server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:04.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.265+0000 7fd04dffb700 1 -- 172.21.15.32:0/122118326 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0400096f0 con 0x7fd0500b2100 2020-12-05T17:42:04.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.265+0000 7fd04dffb700 1 -- 172.21.15.32:0/122118326 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd040009850 con 0x7fd0500b2100 2020-12-05T17:42:04.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.265+0000 7fd04dffb700 1 -- 172.21.15.32:0/122118326 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd040009bb0 con 0x7fd0500b2100 2020-12-05T17:42:04.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.265+0000 7fd04ffff700 1 -- 172.21.15.32:0/122118326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0500b2100 msgr2=0x7fd0500af360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:04.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.265+0000 7fd04ffff700 1 --2- 172.21.15.32:0/122118326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0500b2100 0x7fd0500af360 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 rx=0x7fd0380010d0 tx=0x7fd040006dc0).stop 2020-12-05T17:42:04.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.265+0000 7fd04ffff700 1 -- 172.21.15.32:0/122118326 shutdown_connections 2020-12-05T17:42:04.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.265+0000 7fd04ffff700 1 --2- 172.21.15.32:0/122118326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0500b2100 0x7fd0500af360 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:04.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.265+0000 7fd04ffff700 1 -- 172.21.15.32:0/122118326 >> 172.21.15.32:0/122118326 conn(0x7fd0500620b0 msgr2=0x7fd050062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:04.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.265+0000 7fd04ffff700 1 -- 172.21.15.32:0/122118326 shutdown_connections 2020-12-05T17:42:04.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.265+0000 7fd04ffff700 1 -- 172.21.15.32:0/122118326 wait complete. 2020-12-05T17:42:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.265+0000 7fd04ffff700 1 Processor -- start 2020-12-05T17:42:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.265+0000 7fd04ffff700 1 -- start start 2020-12-05T17:42:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd04ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0500ad250 0x7fd05013bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd04ffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0500b2100 0x7fd0501391e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd04ffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0500667b0 0x7fd050133b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd04ffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd050066f60 con 0x7fd0500ad250 2020-12-05T17:42:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd04ffff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd050063200 con 0x7fd0500667b0 2020-12-05T17:42:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd04ffff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0500634a0 con 0x7fd0500b2100 2020-12-05T17:42:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd04effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0500ad250 0x7fd05013bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd04effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0500ad250 0x7fd05013bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41754/0 (socket says 172.21.15.32:41754) 2020-12-05T17:42:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd04effd700 1 -- 172.21.15.32:0/3188312268 learned_addr learned my addr 172.21.15.32:0/3188312268 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd04f7fe700 1 --2- 172.21.15.32:0/3188312268 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0500667b0 0x7fd050133b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:04.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd04e7fc700 1 --2- 172.21.15.32:0/3188312268 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0500b2100 0x7fd0501391e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:04.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd04effd700 1 -- 172.21.15.32:0/3188312268 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0500b2100 msgr2=0x7fd0501391e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:04.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd04effd700 1 --2- 172.21.15.32:0/3188312268 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0500b2100 0x7fd0501391e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:04.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd04effd700 1 -- 172.21.15.32:0/3188312268 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0500667b0 msgr2=0x7fd050133b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:04.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd04effd700 1 --2- 172.21.15.32:0/3188312268 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0500667b0 0x7fd050133b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:04.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd04effd700 1 -- 172.21.15.32:0/3188312268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd050154bc0 con 0x7fd0500ad250 2020-12-05T17:42:04.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd04e7fc700 1 --2- 172.21.15.32:0/3188312268 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0500b2100 0x7fd0501391e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:42:04.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd04effd700 1 --2- 172.21.15.32:0/3188312268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0500ad250 0x7fd05013bd00 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7fd0380010d0 tx=0x7fd040015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:04.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd047fff700 1 -- 172.21.15.32:0/3188312268 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd040009bd0 con 0x7fd0500ad250 2020-12-05T17:42:04.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd047fff700 1 -- 172.21.15.32:0/3188312268 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd040011c30 con 0x7fd0500ad250 2020-12-05T17:42:04.276 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd047fff700 1 -- 172.21.15.32:0/3188312268 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd04001d450 con 0x7fd0500ad250 2020-12-05T17:42:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd04ffff700 1 -- 172.21.15.32:0/3188312268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd050134020 con 0x7fd0500ad250 2020-12-05T17:42:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd04ffff700 1 -- 172.21.15.32:0/3188312268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0501344c0 con 0x7fd0500ad250 2020-12-05T17:42:04.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.269+0000 7fd04ffff700 1 -- 172.21.15.32:0/3188312268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd05013a0a0 con 0x7fd0500ad250 2020-12-05T17:42:04.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.273+0000 7fd047fff700 1 -- 172.21.15.32:0/3188312268 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd040024030 con 0x7fd0500ad250 2020-12-05T17:42:04.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.273+0000 7fd047fff700 1 --2- 172.21.15.32:0/3188312268 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd034055350 0x7fd034057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:04.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.273+0000 7fd047fff700 1 -- 172.21.15.32:0/3188312268 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd04006f2c0 con 0x7fd0500ad250 2020-12-05T17:42:04.278 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.273+0000 7fd04e7fc700 1 --2- 172.21.15.32:0/3188312268 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd034055350 0x7fd034057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:04.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.273+0000 7fd04e7fc700 1 --2- 172.21.15.32:0/3188312268 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd034055350 0x7fd034057810 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7fd03800b000 tx=0x7fd038009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:04.279 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.277+0000 7fd047fff700 1 -- 172.21.15.32:0/3188312268 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd040045460 con 0x7fd0500ad250 2020-12-05T17:42:04.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.421+0000 7fd04ffff700 1 -- 172.21.15.32:0/3188312268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd050063020 con 0x7fd0500ad250 2020-12-05T17:42:04.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.421+0000 7fd047fff700 1 -- 172.21.15.32:0/3188312268 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v148) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd040022020 con 0x7fd0500ad250 2020-12-05T17:42:04.422 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:42:04.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.421+0000 7fd04ffff700 1 -- 172.21.15.32:0/3188312268 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd034055350 msgr2=0x7fd034057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:04.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.421+0000 7fd04ffff700 1 --2- 172.21.15.32:0/3188312268 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd034055350 0x7fd034057810 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7fd03800b000 tx=0x7fd038009040).stop 2020-12-05T17:42:04.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.421+0000 7fd04ffff700 1 -- 172.21.15.32:0/3188312268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0500ad250 msgr2=0x7fd05013bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:04.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.421+0000 7fd04ffff700 1 --2- 172.21.15.32:0/3188312268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0500ad250 0x7fd05013bd00 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7fd0380010d0 tx=0x7fd040015000).stop 2020-12-05T17:42:04.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.421+0000 7fd04ffff700 1 -- 172.21.15.32:0/3188312268 shutdown_connections 2020-12-05T17:42:04.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.421+0000 7fd04ffff700 1 --2- 172.21.15.32:0/3188312268 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0500667b0 0x7fd050133b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:04.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.421+0000 7fd04ffff700 1 --2- 172.21.15.32:0/3188312268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0500ad250 0x7fd05013bd00 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:04.425 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.421+0000 7fd04ffff700 1 --2- 172.21.15.32:0/3188312268 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fd034055350 0x7fd034057810 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:04.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.421+0000 7fd04ffff700 1 --2- 172.21.15.32:0/3188312268 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd0500b2100 0x7fd0501391e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:04.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.421+0000 7fd04ffff700 1 -- 172.21.15.32:0/3188312268 >> 172.21.15.32:0/3188312268 conn(0x7fd0500620b0 msgr2=0x7fd0500b3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:04.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.421+0000 7fd04ffff700 1 -- 172.21.15.32:0/3188312268 shutdown_connections 2020-12-05T17:42:04.426 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.421+0000 7fd04ffff700 1 -- 172.21.15.32:0/3188312268 wait complete. 2020-12-05T17:42:04.436 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-12-05T17:42:04.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef3d128700 1 Processor -- start 2020-12-05T17:42:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef3d128700 1 -- start start 2020-12-05T17:42:04.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef3d128700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef380b15e0 0x7fef380b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:04.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef3d128700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef380b06b0 con 0x7fef380b2690 2020-12-05T17:42:04.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef3d128700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef3814a280 con 0x7fef380b0010 2020-12-05T17:42:04.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef3d128700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef380b04c0 con 0x7fef380b15e0 2020-12-05T17:42:04.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef36d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fef380b2690 0x7fef3814a9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44410/0 (socket says 172.21.15.32:44410) 2020-12-05T17:42:04.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef36d9d700 1 -- 172.21.15.32:0/3504869891 learned_addr learned my addr 172.21.15.32:0/3504869891 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:42:04.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef3659c700 1 --2- 172.21.15.32:0/3504869891 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef380b15e0 0x7fef380b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:04.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef35d9b700 1 -- 172.21.15.32:0/3504869891 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1081546190 0 0) 0x7fef380b06b0 con 0x7fef380b2690 2020-12-05T17:42:04.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef35d9b700 1 -- 172.21.15.32:0/3504869891 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef18003230 con 0x7fef380b2690 2020-12-05T17:42:04.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef35d9b700 1 -- 172.21.15.32:0/3504869891 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2759222940 0 0) 0x7fef3814a280 con 0x7fef380b0010 2020-12-05T17:42:04.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef35d9b700 1 -- 172.21.15.32:0/3504869891 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef380b06b0 con 0x7fef380b0010 2020-12-05T17:42:04.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef35d9b700 1 -- 172.21.15.32:0/3504869891 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1716780015 0 0) 0x7fef18003230 con 0x7fef380b2690 2020-12-05T17:42:04.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef35d9b700 1 -- 172.21.15.32:0/3504869891 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef380b15e0 msgr2=0x7fef380b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:42:04.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef35d9b700 1 --2- 172.21.15.32:0/3504869891 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef380b15e0 0x7fef380b2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef35d9b700 1 -- 172.21.15.32:0/3504869891 >> v1:172.21.15.61:6789/0 conn(0x7fef380b0010 legacy=0x7fef38146280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef35d9b700 1 -- 172.21.15.32:0/3504869891 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3814a520 con 0x7fef380b2690 2020-12-05T17:42:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef35d9b700 1 -- 172.21.15.32:0/3504869891 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7fef280019d0 con 0x7fef380b2690 2020-12-05T17:42:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef35d9b700 1 -- 172.21.15.32:0/3504869891 >> v1:172.21.15.32:6789/0 conn(0x7fef380b2690 legacy=0x7fef3814a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef35d9b700 1 --2- 172.21.15.32:0/3504869891 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef180038e0 0x7fef18003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:04.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef35d9b700 1 --2- 172.21.15.32:0/3504869891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef180044d0 0x7fef18004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:04.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef35d9b700 1 --2- 172.21.15.32:0/3504869891 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef380b15e0 0x7fef18004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:04.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef35d9b700 1 -- 172.21.15.32:0/3504869891 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef18005350 con 0x7fef180044d0 2020-12-05T17:42:04.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef35d9b700 1 -- 172.21.15.32:0/3504869891 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1800b720 con 0x7fef180038e0 2020-12-05T17:42:04.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef35d9b700 1 -- 172.21.15.32:0/3504869891 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1800b8e0 con 0x7fef380b15e0 2020-12-05T17:42:04.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef3659c700 1 --2- 172.21.15.32:0/3504869891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef180044d0 0x7fef18004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:04.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef36d9d700 1 --2- 172.21.15.32:0/3504869891 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef180038e0 0x7fef18003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:04.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.557+0000 7fef3759e700 1 --2- 172.21.15.32:0/3504869891 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef380b15e0 0x7fef18004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:04.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef3659c700 1 -- 172.21.15.32:0/3504869891 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef380b15e0 msgr2=0x7fef18004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:42:04.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef3659c700 1 --2- 172.21.15.32:0/3504869891 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef380b15e0 0x7fef18004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:04.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef3659c700 1 -- 172.21.15.32:0/3504869891 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef180038e0 msgr2=0x7fef18003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:42:04.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef3659c700 1 --2- 172.21.15.32:0/3504869891 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef180038e0 0x7fef18003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:04.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef3659c700 1 -- 172.21.15.32:0/3504869891 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef18003230 con 0x7fef180044d0 2020-12-05T17:42:04.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef3759e700 1 --2- 172.21.15.32:0/3504869891 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef380b15e0 0x7fef18004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:42:04.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef3659c700 1 --2- 172.21.15.32:0/3504869891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef180044d0 0x7fef18004950 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 rx=0x7fef2000c000 tx=0x7fef20008040).ready entity=mon.0 client_cookie=8fbdcd1dacd5c203 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:04.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef35d9b700 1 -- 172.21.15.32:0/3504869891 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef2000e970 con 0x7fef180044d0 2020-12-05T17:42:04.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef35d9b700 1 -- 172.21.15.32:0/3504869891 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2000ead0 con 0x7fef180044d0 2020-12-05T17:42:04.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef35d9b700 1 -- 172.21.15.32:0/3504869891 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef2000ee30 con 0x7fef180044d0 2020-12-05T17:42:04.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef3d128700 1 -- 172.21.15.32:0/3504869891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef180044d0 msgr2=0x7fef18004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:04.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef3d128700 1 --2- 172.21.15.32:0/3504869891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef180044d0 0x7fef18004950 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 rx=0x7fef2000c000 tx=0x7fef20008040).stop 2020-12-05T17:42:04.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef3759e700 1 -- 172.21.15.32:0/3504869891 reap_dead start 2020-12-05T17:42:04.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef3d128700 1 -- 172.21.15.32:0/3504869891 shutdown_connections 2020-12-05T17:42:04.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef3d128700 1 -- 172.21.15.32:0/3504869891 >> 172.21.15.32:0/3504869891 conn(0x7fef38062310 msgr2=0x7fef380626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:04.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef3d128700 1 -- 172.21.15.32:0/3504869891 shutdown_connections 2020-12-05T17:42:04.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef3d128700 1 -- 172.21.15.32:0/3504869891 wait complete. 2020-12-05T17:42:04.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef3d128700 1 Processor -- start 2020-12-05T17:42:04.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef3d128700 1 -- start start 2020-12-05T17:42:04.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef3d128700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef38063380 0x7fef38064240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:04.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef3d128700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef38066480 0x7fef38117590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:04.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef3d128700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef38117a80 0x7fef381428f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:04.573 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef3d128700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef38065d20 con 0x7fef38066480 2020-12-05T17:42:04.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef3d128700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef38117f00 con 0x7fef38063380 2020-12-05T17:42:04.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef3d128700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef38063760 con 0x7fef38117a80 2020-12-05T17:42:04.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.561+0000 7fef3659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef38066480 0x7fef38117590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:04.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.565+0000 7fef3659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef38066480 0x7fef38117590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41774/0 (socket says 172.21.15.32:41774) 2020-12-05T17:42:04.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.565+0000 7fef3659c700 1 -- 172.21.15.32:0/2568140070 learned_addr learned my addr 172.21.15.32:0/2568140070 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:04.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.565+0000 7fef3759e700 1 --2- 172.21.15.32:0/2568140070 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef38117a80 0x7fef381428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:04.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.565+0000 7fef36d9d700 1 --2- 172.21.15.32:0/2568140070 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef38063380 0x7fef38064240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:04.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.565+0000 7fef3659c700 1 -- 172.21.15.32:0/2568140070 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef38117a80 msgr2=0x7fef381428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:04.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.565+0000 7fef3659c700 1 --2- 172.21.15.32:0/2568140070 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef38117a80 0x7fef381428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:04.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.565+0000 7fef3659c700 1 -- 172.21.15.32:0/2568140070 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef38063380 msgr2=0x7fef38064240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:04.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.565+0000 7fef3659c700 1 --2- 172.21.15.32:0/2568140070 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef38063380 0x7fef38064240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:04.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.565+0000 7fef3659c700 1 -- 172.21.15.32:0/2568140070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef38065f30 con 0x7fef38066480 2020-12-05T17:42:04.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.565+0000 7fef36d9d700 1 --2- 172.21.15.32:0/2568140070 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef38063380 0x7fef38064240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:42:04.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.565+0000 7fef3759e700 1 --2- 172.21.15.32:0/2568140070 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef38117a80 0x7fef381428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:42:04.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.565+0000 7fef3659c700 1 --2- 172.21.15.32:0/2568140070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef38066480 0x7fef38117590 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7fef380638f0 tx=0x7fef2001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.565+0000 7fef2ffff700 1 -- 172.21.15.32:0/2568140070 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef2001c430 con 0x7fef38066480 2020-12-05T17:42:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.565+0000 7fef2ffff700 1 -- 172.21.15.32:0/2568140070 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2001c590 con 0x7fef38066480 2020-12-05T17:42:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.565+0000 7fef2ffff700 1 -- 172.21.15.32:0/2568140070 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef2001c8f0 con 0x7fef38066480 2020-12-05T17:42:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.565+0000 7fef3d128700 1 -- 172.21.15.32:0/2568140070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef380661d0 con 0x7fef38066480 2020-12-05T17:42:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.565+0000 7fef3d128700 1 -- 172.21.15.32:0/2568140070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef38119fc0 con 0x7fef38066480 2020-12-05T17:42:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.565+0000 7fef3d128700 1 -- 172.21.15.32:0/2568140070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef38117130 con 0x7fef38066480 2020-12-05T17:42:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.565+0000 7fef2ffff700 1 -- 172.21.15.32:0/2568140070 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fef2001ca50 con 0x7fef38066480 2020-12-05T17:42:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.565+0000 7fef2ffff700 1 --2- 172.21.15.32:0/2568140070 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fef1c055470 0x7fef1c057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.565+0000 7fef2ffff700 1 -- 172.21.15.32:0/2568140070 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fef2006fbd0 con 0x7fef38066480 2020-12-05T17:42:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.565+0000 7fef36d9d700 1 --2- 172.21.15.32:0/2568140070 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fef1c055470 0x7fef1c057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:04.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.569+0000 7fef36d9d700 1 --2- 172.21.15.32:0/2568140070 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fef1c055470 0x7fef1c057930 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7fef28005000 tx=0x7fef28009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:04.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.573+0000 7fef2ffff700 1 -- 172.21.15.32:0/2568140070 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fef20045d70 con 0x7fef38066480 2020-12-05T17:42:04.716 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.713+0000 7fef3d128700 1 -- 172.21.15.32:0/2568140070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fef38057050 con 0x7fef38066480 2020-12-05T17:42:04.717 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.713+0000 7fef2ffff700 1 -- 172.21.15.32:0/2568140070 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v148) v1 ==== 104+0+10 (secure 0 0 0) 0x7fef200160a0 con 0x7fef38066480 2020-12-05T17:42:04.717 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:42:04.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.717+0000 7fef3d128700 1 -- 172.21.15.32:0/2568140070 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fef1c055470 msgr2=0x7fef1c057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:04.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.717+0000 7fef3d128700 1 --2- 172.21.15.32:0/2568140070 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fef1c055470 0x7fef1c057930 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7fef28005000 tx=0x7fef28009040).stop 2020-12-05T17:42:04.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.717+0000 7fef3d128700 1 -- 172.21.15.32:0/2568140070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef38066480 msgr2=0x7fef38117590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:04.720 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.717+0000 7fef3d128700 1 --2- 172.21.15.32:0/2568140070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef38066480 0x7fef38117590 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7fef380638f0 tx=0x7fef2001a000).stop 2020-12-05T17:42:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.717+0000 7fef3d128700 1 -- 172.21.15.32:0/2568140070 shutdown_connections 2020-12-05T17:42:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.717+0000 7fef3d128700 1 --2- 172.21.15.32:0/2568140070 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fef1c055470 0x7fef1c057930 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.717+0000 7fef3d128700 1 --2- 172.21.15.32:0/2568140070 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef38117a80 0x7fef381428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.717+0000 7fef3d128700 1 --2- 172.21.15.32:0/2568140070 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef38063380 0x7fef38064240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.717+0000 7fef3d128700 1 --2- 172.21.15.32:0/2568140070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef38066480 0x7fef38117590 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.717+0000 7fef3d128700 1 -- 172.21.15.32:0/2568140070 >> 172.21.15.32:0/2568140070 conn(0x7fef38062310 msgr2=0x7fef3805f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.717+0000 7fef3d128700 1 -- 172.21.15.32:0/2568140070 shutdown_connections 2020-12-05T17:42:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.717+0000 7fef3d128700 1 -- 172.21.15.32:0/2568140070 wait complete. 2020-12-05T17:42:04.736 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-12-05T17:42:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.857+0000 7fa142fe0700 1 Processor -- start 2020-12-05T17:42:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.857+0000 7fa142fe0700 1 -- start start 2020-12-05T17:42:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.857+0000 7fa142fe0700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa13c05feb0 0x7fa13c05f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.857+0000 7fa142fe0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa13c0b2df0 con 0x7fa13c05f3b0 2020-12-05T17:42:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.857+0000 7fa142fe0700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa13c0b3c90 con 0x7fa13c0b2fb0 2020-12-05T17:42:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.857+0000 7fa142fe0700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa13c0b2b50 con 0x7fa13c05feb0 2020-12-05T17:42:04.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.857+0000 7fa141fde700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa13c05f3b0 0x7fa13c13c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44430/0 (socket says 172.21.15.32:44430) 2020-12-05T17:42:04.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.857+0000 7fa141fde700 1 -- 172.21.15.32:0/2212832713 learned_addr learned my addr 172.21.15.32:0/2212832713 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:42:04.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.857+0000 7fa1417dd700 1 --2- 172.21.15.32:0/2212832713 >> v2:172.21.15.61:3300/0 conn(0x7fa13c05feb0 0x7fa13c05f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:04.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.857+0000 7fa140fdc700 1 -- 172.21.15.32:0/2212832713 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 720145209 0 0) 0x7fa13c0b2df0 con 0x7fa13c05f3b0 2020-12-05T17:42:04.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.857+0000 7fa140fdc700 1 -- 172.21.15.32:0/2212832713 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa124003230 con 0x7fa13c05f3b0 2020-12-05T17:42:04.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.857+0000 7fa1417dd700 1 -- 172.21.15.32:0/2212832713 >> v1:172.21.15.61:6789/0 conn(0x7fa13c0b2fb0 legacy=0x7fa13c142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:04.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.857+0000 7fa140fdc700 1 -- 172.21.15.32:0/2212832713 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3408759292 0 0) 0x7fa13c0b3c90 con 0x7fa13c0b2fb0 2020-12-05T17:42:04.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.857+0000 7fa1417dd700 1 -- 172.21.15.32:0/2212832713 >> v1:172.21.15.32:6789/0 conn(0x7fa13c05f3b0 legacy=0x7fa13c13c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:04.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.857+0000 7fa1417dd700 1 -- 172.21.15.32:0/2212832713 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c142820 con 0x7fa13c05feb0 2020-12-05T17:42:04.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.857+0000 7fa1417dd700 1 --2- 172.21.15.32:0/2212832713 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa13c05feb0 0x7fa13c05f790 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7fa12c002f30 tx=0x7fa12c006dc0).ready entity=mon.1 client_cookie=29289406a97acae5 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:04.862 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.857+0000 7fa140fdc700 1 -- 172.21.15.32:0/2212832713 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa12c009800 con 0x7fa13c05feb0 2020-12-05T17:42:04.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.857+0000 7fa140fdc700 1 -- 172.21.15.32:0/2212832713 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12c009960 con 0x7fa13c05feb0 2020-12-05T17:42:04.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.857+0000 7fa140fdc700 1 -- 172.21.15.32:0/2212832713 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa12c009cc0 con 0x7fa13c05feb0 2020-12-05T17:42:04.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.857+0000 7fa142fe0700 1 -- 172.21.15.32:0/2212832713 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa13c05feb0 msgr2=0x7fa13c05f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:04.863 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.857+0000 7fa142fe0700 1 --2- 172.21.15.32:0/2212832713 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa13c05feb0 0x7fa13c05f790 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7fa12c002f30 tx=0x7fa12c006dc0).stop 2020-12-05T17:42:04.865 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.857+0000 7fa142fe0700 1 -- 172.21.15.32:0/2212832713 shutdown_connections 2020-12-05T17:42:04.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa142fe0700 1 --2- 172.21.15.32:0/2212832713 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa13c05feb0 0x7fa13c05f790 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:04.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa142fe0700 1 -- 172.21.15.32:0/2212832713 >> 172.21.15.32:0/2212832713 conn(0x7fa13c0b8f20 msgr2=0x7fa13c0b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:04.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa142fe0700 1 -- 172.21.15.32:0/2212832713 shutdown_connections 2020-12-05T17:42:04.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa142fe0700 1 -- 172.21.15.32:0/2212832713 wait complete. 2020-12-05T17:42:04.866 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa142fe0700 1 Processor -- start 2020-12-05T17:42:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa142fe0700 1 -- start start 2020-12-05T17:42:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa142fe0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa13c05f3b0 0x7fa13c1485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa142fe0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa13c05feb0 0x7fa13c13d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa142fe0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13c0b2fb0 0x7fa13c151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa142fe0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c152270 con 0x7fa13c0b2fb0 2020-12-05T17:42:04.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa142fe0700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c152490 con 0x7fa13c05f3b0 2020-12-05T17:42:04.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa142fe0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c147e80 con 0x7fa13c05feb0 2020-12-05T17:42:04.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa1427df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13c0b2fb0 0x7fa13c151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:04.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa1427df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13c0b2fb0 0x7fa13c151d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41792/0 (socket says 172.21.15.32:41792) 2020-12-05T17:42:04.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa1427df700 1 -- 172.21.15.32:0/1152001632 learned_addr learned my addr 172.21.15.32:0/1152001632 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:04.868 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa1417dd700 1 --2- 172.21.15.32:0/1152001632 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa13c05feb0 0x7fa13c13d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa141fde700 1 --2- 172.21.15.32:0/1152001632 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa13c05f3b0 0x7fa13c1485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa1427df700 1 -- 172.21.15.32:0/1152001632 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa13c05feb0 msgr2=0x7fa13c13d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa1427df700 1 --2- 172.21.15.32:0/1152001632 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa13c05feb0 0x7fa13c13d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa1427df700 1 -- 172.21.15.32:0/1152001632 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa13c05f3b0 msgr2=0x7fa13c1485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa1427df700 1 --2- 172.21.15.32:0/1152001632 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa13c05f3b0 0x7fa13c1485c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa1427df700 1 -- 172.21.15.32:0/1152001632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c148120 con 0x7fa13c0b2fb0 2020-12-05T17:42:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa141fde700 1 --2- 172.21.15.32:0/1152001632 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa13c05f3b0 0x7fa13c1485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:42:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa1427df700 1 --2- 172.21.15.32:0/1152001632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13c0b2fb0 0x7fa13c151d80 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 rx=0x7fa134000f00 tx=0x7fa134007ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa12affd700 1 -- 172.21.15.32:0/1152001632 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1340030c0 con 0x7fa13c0b2fb0 2020-12-05T17:42:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa12affd700 1 -- 172.21.15.32:0/1152001632 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13400a7b0 con 0x7fa13c0b2fb0 2020-12-05T17:42:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa142fe0700 1 -- 172.21.15.32:0/1152001632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13c147030 con 0x7fa13c0b2fb0 2020-12-05T17:42:04.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.861+0000 7fa12affd700 1 -- 172.21.15.32:0/1152001632 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa13400ab10 con 0x7fa13c0b2fb0 2020-12-05T17:42:04.871 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.865+0000 7fa142fe0700 1 -- 172.21.15.32:0/1152001632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13c141e20 con 0x7fa13c0b2fb0 2020-12-05T17:42:04.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.865+0000 7fa12affd700 1 -- 172.21.15.32:0/1152001632 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa134019030 con 0x7fa13c0b2fb0 2020-12-05T17:42:04.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.865+0000 7fa142fe0700 1 -- 172.21.15.32:0/1152001632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa13c1482b0 con 0x7fa13c0b2fb0 2020-12-05T17:42:04.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.865+0000 7fa12affd700 1 --2- 172.21.15.32:0/1152001632 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa11c055330 0x7fa11c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:04.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.865+0000 7fa141fde700 1 --2- 172.21.15.32:0/1152001632 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa11c055330 0x7fa11c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:04.872 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.865+0000 7fa12affd700 1 -- 172.21.15.32:0/1152001632 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa13406d790 con 0x7fa13c0b2fb0 2020-12-05T17:42:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.865+0000 7fa141fde700 1 --2- 172.21.15.32:0/1152001632 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa11c055330 0x7fa11c0577f0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7fa13800b000 tx=0x7fa138009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:04.869+0000 7fa12affd700 1 -- 172.21.15.32:0/1152001632 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa134041df0 con 0x7fa13c0b2fb0 2020-12-05T17:42:05.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.017+0000 7fa142fe0700 1 -- 172.21.15.32:0/1152001632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa13c1471c0 con 0x7fa13c0b2fb0 2020-12-05T17:42:05.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.017+0000 7fa12affd700 1 -- 172.21.15.32:0/1152001632 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v148) v1 ==== 100+0+10 (secure 0 0 0) 0x7fa1340439e0 con 0x7fa13c0b2fb0 2020-12-05T17:42:05.018 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2020-12-05T17:42:05.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.017+0000 7fa142fe0700 1 -- 172.21.15.32:0/1152001632 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa11c055330 msgr2=0x7fa11c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:05.021 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.017+0000 7fa142fe0700 1 --2- 172.21.15.32:0/1152001632 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa11c055330 0x7fa11c0577f0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7fa13800b000 tx=0x7fa138009040).stop 2020-12-05T17:42:05.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.017+0000 7fa142fe0700 1 -- 172.21.15.32:0/1152001632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13c0b2fb0 msgr2=0x7fa13c151d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:05.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.017+0000 7fa142fe0700 1 --2- 172.21.15.32:0/1152001632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13c0b2fb0 0x7fa13c151d80 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 rx=0x7fa134000f00 tx=0x7fa134007ec0).stop 2020-12-05T17:42:05.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.017+0000 7fa142fe0700 1 -- 172.21.15.32:0/1152001632 shutdown_connections 2020-12-05T17:42:05.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.017+0000 7fa142fe0700 1 --2- 172.21.15.32:0/1152001632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13c0b2fb0 0x7fa13c151d80 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:05.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.017+0000 7fa142fe0700 1 --2- 172.21.15.32:0/1152001632 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa13c05f3b0 0x7fa13c1485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:05.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.017+0000 7fa142fe0700 1 --2- 172.21.15.32:0/1152001632 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa11c055330 0x7fa11c0577f0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:05.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.017+0000 7fa142fe0700 1 --2- 172.21.15.32:0/1152001632 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa13c05feb0 0x7fa13c13d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:05.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.017+0000 7fa142fe0700 1 -- 172.21.15.32:0/1152001632 >> 172.21.15.32:0/1152001632 conn(0x7fa13c0b8f20 msgr2=0x7fa13c0b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:05.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.017+0000 7fa142fe0700 1 -- 172.21.15.32:0/1152001632 shutdown_connections 2020-12-05T17:42:05.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.017+0000 7fa142fe0700 1 -- 172.21.15.32:0/1152001632 wait complete. 2020-12-05T17:42:05.037 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-12-05T17:42:05.038 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:42:05.145 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b278a7700 1 Processor -- start 2020-12-05T17:42:05.146 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b278a7700 1 -- start start 2020-12-05T17:42:05.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b278a7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b200afe50 0x7f5b20146a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:05.151 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b278a7700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f5b200b1880 0x7f5b201430f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:05.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b278a7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b200b0230 con 0x7f5b200b3aa0 2020-12-05T17:42:05.152 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b278a7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b200b6a50 con 0x7f5b200afe50 2020-12-05T17:42:05.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b278a7700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b20146080 con 0x7f5b200b1880 2020-12-05T17:42:05.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b24e42700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5b200b3aa0 0x7f5b201429e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44444/0 (socket says 172.21.15.32:44444) 2020-12-05T17:42:05.153 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b25643700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b200afe50 0x7f5b20146a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:05.154 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b24e42700 1 -- 172.21.15.32:0/2356261686 learned_addr learned my addr 172.21.15.32:0/2356261686 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:42:05.155 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b25e44700 1 --2- 172.21.15.32:0/2356261686 >> v2:172.21.15.61:3300/0 conn(0x7f5b200b1880 0x7f5b201430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:05.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b17fff700 1 -- 172.21.15.32:0/2356261686 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 55373552 0 0) 0x7f5b200b0230 con 0x7f5b200b3aa0 2020-12-05T17:42:05.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b17fff700 1 -- 172.21.15.32:0/2356261686 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b04003230 con 0x7f5b200b3aa0 2020-12-05T17:42:05.156 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b25643700 1 -- 172.21.15.32:0/2356261686 >> v2:172.21.15.61:3300/0 conn(0x7f5b200b1880 msgr2=0x7f5b201430f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:42:05.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b25643700 1 --2- 172.21.15.32:0/2356261686 >> v2:172.21.15.61:3300/0 conn(0x7f5b200b1880 0x7f5b201430f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:05.157 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b25643700 1 -- 172.21.15.32:0/2356261686 >> v1:172.21.15.32:6789/0 conn(0x7f5b200b3aa0 legacy=0x7f5b201429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:05.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b17fff700 1 -- 172.21.15.32:0/2356261686 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 51439647 0 0) 0x7f5b04003230 con 0x7f5b200b3aa0 2020-12-05T17:42:05.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b25643700 1 -- 172.21.15.32:0/2356261686 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b20146320 con 0x7f5b200afe50 2020-12-05T17:42:05.158 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b25643700 1 --2- 172.21.15.32:0/2356261686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b200afe50 0x7f5b20146a60 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7f5b10001870 tx=0x7f5b1c006dc0).ready entity=mon.0 client_cookie=143e1cd722b6b190 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:05.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b17fff700 1 -- 172.21.15.32:0/2356261686 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b100016d0 con 0x7f5b200afe50 2020-12-05T17:42:05.159 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b17fff700 1 -- 172.21.15.32:0/2356261686 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1c00d5c0 con 0x7f5b200afe50 2020-12-05T17:42:05.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b17fff700 1 -- 172.21.15.32:0/2356261686 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b1c00d730 con 0x7f5b200afe50 2020-12-05T17:42:05.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b278a7700 1 -- 172.21.15.32:0/2356261686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b200afe50 msgr2=0x7f5b20146a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:05.160 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b278a7700 1 --2- 172.21.15.32:0/2356261686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b200afe50 0x7f5b20146a60 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7f5b10001870 tx=0x7f5b1c006dc0).stop 2020-12-05T17:42:05.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b278a7700 1 -- 172.21.15.32:0/2356261686 shutdown_connections 2020-12-05T17:42:05.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b278a7700 1 --2- 172.21.15.32:0/2356261686 >> v2:172.21.15.61:3300/0 conn(0x7f5b200b1880 0x7f5b201430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:05.161 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b278a7700 1 --2- 172.21.15.32:0/2356261686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b200afe50 0x7f5b20146a60 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:05.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b278a7700 1 -- 172.21.15.32:0/2356261686 >> 172.21.15.32:0/2356261686 conn(0x7f5b200b9100 msgr2=0x7f5b200b9510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:05.162 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b278a7700 1 -- 172.21.15.32:0/2356261686 shutdown_connections 2020-12-05T17:42:05.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.145+0000 7f5b278a7700 1 -- 172.21.15.32:0/2356261686 wait complete. 2020-12-05T17:42:05.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.149+0000 7f5b278a7700 1 Processor -- start 2020-12-05T17:42:05.163 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.149+0000 7f5b278a7700 1 -- start start 2020-12-05T17:42:05.170 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.149+0000 7f5b278a7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5b200afe50 0x7f5b20062dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:05.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.149+0000 7f5b278a7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b200b1880 0x7f5b201556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:05.171 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.149+0000 7f5b278a7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b200b3aa0 0x7f5b20066930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:05.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.149+0000 7f5b278a7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2013bf10 con 0x7f5b200b1880 2020-12-05T17:42:05.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.149+0000 7f5b278a7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2013c320 con 0x7f5b200afe50 2020-12-05T17:42:05.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.149+0000 7f5b278a7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2013c5c0 con 0x7f5b200b3aa0 2020-12-05T17:42:05.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.149+0000 7f5b24e42700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b200b1880 0x7f5b201556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:05.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.149+0000 7f5b24e42700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b200b1880 0x7f5b201556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41804/0 (socket says 172.21.15.32:41804) 2020-12-05T17:42:05.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.149+0000 7f5b24e42700 1 -- 172.21.15.32:0/4063490634 learned_addr learned my addr 172.21.15.32:0/4063490634 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:05.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.149+0000 7f5b25643700 1 --2- 172.21.15.32:0/4063490634 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5b200afe50 0x7f5b20062dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:05.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.149+0000 7f5b25e44700 1 --2- 172.21.15.32:0/4063490634 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b200b3aa0 0x7f5b20066930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:05.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.149+0000 7f5b24e42700 1 -- 172.21.15.32:0/4063490634 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b200b3aa0 msgr2=0x7f5b20066930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:05.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.149+0000 7f5b24e42700 1 --2- 172.21.15.32:0/4063490634 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b200b3aa0 0x7f5b20066930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:05.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.149+0000 7f5b24e42700 1 -- 172.21.15.32:0/4063490634 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5b200afe50 msgr2=0x7f5b20062dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:05.174 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.149+0000 7f5b24e42700 1 --2- 172.21.15.32:0/4063490634 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5b200afe50 0x7f5b20062dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:05.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.149+0000 7f5b24e42700 1 -- 172.21.15.32:0/4063490634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b20066e20 con 0x7f5b200b1880 2020-12-05T17:42:05.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.149+0000 7f5b25643700 1 --2- 172.21.15.32:0/4063490634 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5b200afe50 0x7f5b20062dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T17:42:05.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.149+0000 7f5b24e42700 1 --2- 172.21.15.32:0/4063490634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b200b1880 0x7f5b201556e0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f5b10001930 tx=0x7f5b10001280).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:05.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.149+0000 7f5b167fc700 1 -- 172.21.15.32:0/4063490634 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b10007170 con 0x7f5b200b1880 2020-12-05T17:42:05.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.149+0000 7f5b167fc700 1 -- 172.21.15.32:0/4063490634 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b100072d0 con 0x7f5b200b1880 2020-12-05T17:42:05.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.153+0000 7f5b167fc700 1 -- 172.21.15.32:0/4063490634 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b100048f0 con 0x7f5b200b1880 2020-12-05T17:42:05.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.153+0000 7f5b278a7700 1 -- 172.21.15.32:0/4063490634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2013bc00 con 0x7f5b200b1880 2020-12-05T17:42:05.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.153+0000 7f5b278a7700 1 -- 172.21.15.32:0/4063490634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b201431b0 con 0x7f5b200b1880 2020-12-05T17:42:05.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.153+0000 7f5b167fc700 1 -- 172.21.15.32:0/4063490634 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5b10010070 con 0x7f5b200b1880 2020-12-05T17:42:05.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.153+0000 7f5b278a7700 1 -- 172.21.15.32:0/4063490634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b2013c780 con 0x7f5b200b1880 2020-12-05T17:42:05.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.153+0000 7f5b167fc700 1 --2- 172.21.15.32:0/4063490634 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b080553b0 0x7f5b08057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:05.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.153+0000 7f5b167fc700 1 -- 172.21.15.32:0/4063490634 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5b1006da20 con 0x7f5b200b1880 2020-12-05T17:42:05.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.153+0000 7f5b25643700 1 --2- 172.21.15.32:0/4063490634 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b080553b0 0x7f5b08057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:05.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.153+0000 7f5b25643700 1 --2- 172.21.15.32:0/4063490634 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b080553b0 0x7f5b08057870 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f5b1c01a000 tx=0x7f5b1c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:05.178 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.161+0000 7f5b167fc700 1 -- 172.21.15.32:0/4063490634 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5b10043ab0 con 0x7f5b200b1880 2020-12-05T17:42:05.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.349+0000 7f5b278a7700 1 -- 172.21.15.32:0/4063490634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b20057050 con 0x7f5b200b1880 2020-12-05T17:42:05.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.349+0000 7f5b167fc700 1 -- 172.21.15.32:0/4063490634 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f5b10019030 con 0x7f5b200b1880 2020-12-05T17:42:05.351 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:42:05.352 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":73,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8473,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/711010965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":711010965},{"type":"v1","addr":"172.21.15.61:6833","nonce":711010965}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":70}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:41:55.663831+0000","modified":"2020-12-05T17:41:57.203619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8580":{"gid":8580,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1575496107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1575496107},{"type":"v1","addr":"172.21.15.32:6835","nonce":1575496107}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2020-12-05T17:42:05.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.353+0000 7f5b278a7700 1 -- 172.21.15.32:0/4063490634 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b080553b0 msgr2=0x7f5b08057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:05.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.353+0000 7f5b278a7700 1 --2- 172.21.15.32:0/4063490634 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b080553b0 0x7f5b08057870 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f5b1c01a000 tx=0x7f5b1c015000).stop 2020-12-05T17:42:05.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.353+0000 7f5b278a7700 1 -- 172.21.15.32:0/4063490634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b200b1880 msgr2=0x7f5b201556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:05.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.353+0000 7f5b278a7700 1 --2- 172.21.15.32:0/4063490634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b200b1880 0x7f5b201556e0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f5b10001930 tx=0x7f5b10001280).stop 2020-12-05T17:42:05.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.353+0000 7f5b278a7700 1 -- 172.21.15.32:0/4063490634 shutdown_connections 2020-12-05T17:42:05.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.353+0000 7f5b278a7700 1 --2- 172.21.15.32:0/4063490634 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5b080553b0 0x7f5b08057870 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:05.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.353+0000 7f5b278a7700 1 --2- 172.21.15.32:0/4063490634 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5b200b3aa0 0x7f5b20066930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:05.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.353+0000 7f5b278a7700 1 --2- 172.21.15.32:0/4063490634 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5b200afe50 0x7f5b20062dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:05.358 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.353+0000 7f5b278a7700 1 --2- 172.21.15.32:0/4063490634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b200b1880 0x7f5b201556e0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:05.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.353+0000 7f5b278a7700 1 -- 172.21.15.32:0/4063490634 >> 172.21.15.32:0/4063490634 conn(0x7f5b200b9100 msgr2=0x7f5b20139350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:05.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.353+0000 7f5b278a7700 1 -- 172.21.15.32:0/4063490634 shutdown_connections 2020-12-05T17:42:05.359 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:05.353+0000 7f5b278a7700 1 -- 172.21.15.32:0/4063490634 wait complete. 2020-12-05T17:42:05.360 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 73 2020-12-05T17:42:05.570 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-12-05T17:41:55.663831+0000', 'modified': '2020-12-05T17:41:57.203619+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8580}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8580': {'gid': 8580, 'name': 'a', 'rank': 0, 'incarnation': 71, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/1575496107', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 1575496107}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 1575496107}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-12-05T17:42:05.571 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-12-05T17:42:05.571 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-12-05T17:42:05.701 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","whoami":0,"id":8580,"want_state":"up:active","state":"up:active","rank_uptime":10.024957876,"mdsmap_epoch":73,"osdmap_epoch":148,"osdmap_epoch_barrier":148,"uptime":15.393339437} 2020-12-05T17:42:05.702 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", "id": 8580, "mdsmap_epoch": 73, "osdmap_epoch": 148, "osdmap_epoch_barrier": 148, "rank_uptime": 10.024957876, "state": "up:active", "uptime": 15.393339437, "want_state": "up:active", "whoami": 0 } 2020-12-05T17:42:05.702 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-12-05T17:42:05.703 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-12-05T17:42:05.703 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi002.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-12-05T17:42:05.704 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:42:05.704 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:42:05.711 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:42:05.769 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:42:05.770 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:42:05.825 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:42:05.826 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:42:05.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:42:05.827 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:42:05.872 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-12-05T17:42:05.872 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:42:05.916 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:42:05.933 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:42:05.933 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:42:05.988 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:42:05.989 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:42:05.990 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:42:05.990 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:42:05.994 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:2020-12-05T17:42:05.988+0000 7fea82a130c0 -1 init, newargv = 0x55c7bb469fc0 newargc=9ceph-fuse[11283]: starting ceph client 2020-12-05T17:42:05.995 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr: 2020-12-05T17:42:06.005 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[11283]: starting fuse 2020-12-05T17:42:07.099 INFO:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2020-12-05T17:42:07.115 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-12-05T17:42:07.116 INFO:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-12-05T17:42:07.173 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-12-05T17:42:07.174 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-12-05T17:42:07.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:42:07.175 INFO:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2020-12-05T17:42:07.224 INFO:teuthology.orchestra.run.smithi002.stdout:50 2020-12-05T17:42:07.225 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [50] 2020-12-05T17:42:07.225 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:42:07.226 INFO:teuthology.orchestra.run.smithi002:> sudo python3 -c ' 2020-12-05T17:42:07.226 INFO:teuthology.orchestra.run.smithi002:> import glob 2020-12-05T17:42:07.226 INFO:teuthology.orchestra.run.smithi002:> import re 2020-12-05T17:42:07.227 INFO:teuthology.orchestra.run.smithi002:> import os 2020-12-05T17:42:07.227 INFO:teuthology.orchestra.run.smithi002:> import subprocess 2020-12-05T17:42:07.228 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:42:07.228 INFO:teuthology.orchestra.run.smithi002:> def find_socket(client_name): 2020-12-05T17:42:07.228 INFO:teuthology.orchestra.run.smithi002:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-12-05T17:42:07.229 INFO:teuthology.orchestra.run.smithi002:> files = glob.glob(asok_path) 2020-12-05T17:42:07.229 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:42:07.230 INFO:teuthology.orchestra.run.smithi002:> # Given a non-glob path, it better be there 2020-12-05T17:42:07.230 INFO:teuthology.orchestra.run.smithi002:> if "*" not in asok_path: 2020-12-05T17:42:07.230 INFO:teuthology.orchestra.run.smithi002:> assert(len(files) == 1) 2020-12-05T17:42:07.231 INFO:teuthology.orchestra.run.smithi002:> return files[0] 2020-12-05T17:42:07.231 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:42:07.232 INFO:teuthology.orchestra.run.smithi002:> for f in files: 2020-12-05T17:42:07.232 INFO:teuthology.orchestra.run.smithi002:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-12-05T17:42:07.232 INFO:teuthology.orchestra.run.smithi002:> if os.path.exists("/proc/{0}".format(pid)): 2020-12-05T17:42:07.233 INFO:teuthology.orchestra.run.smithi002:> return f 2020-12-05T17:42:07.233 INFO:teuthology.orchestra.run.smithi002:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-12-05T17:42:07.233 INFO:teuthology.orchestra.run.smithi002:> 2020-12-05T17:42:07.234 INFO:teuthology.orchestra.run.smithi002:> print(find_socket("client.1")) 2020-12-05T17:42:07.234 INFO:teuthology.orchestra.run.smithi002:> ' 2020-12-05T17:42:07.316 INFO:teuthology.orchestra.run.smithi002.stdout:/var/run/ceph/ceph-client.1.11283.asok 2020-12-05T17:42:07.322 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.11283.asok 2020-12-05T17:42:07.322 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:42:07.323 INFO:teuthology.orchestra.run.smithi002:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.11283.asok status 2020-12-05T17:42:07.423 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2020-12-05T17:42:07.423 INFO:teuthology.orchestra.run.smithi002.stdout: "metadata": { 2020-12-05T17:42:07.423 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_sha1": "3b1861acb8137d863184795ccadbd5d0cb026674", 2020-12-05T17:42:07.424 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 15.2.6-607-g3b1861ac (3b1861acb8137d863184795ccadbd5d0cb026674) octopus (stable)", 2020-12-05T17:42:07.424 INFO:teuthology.orchestra.run.smithi002.stdout: "entity_id": "1", 2020-12-05T17:42:07.424 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": "smithi002", 2020-12-05T17:42:07.424 INFO:teuthology.orchestra.run.smithi002.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-12-05T17:42:07.425 INFO:teuthology.orchestra.run.smithi002.stdout: "pid": "11283", 2020-12-05T17:42:07.425 INFO:teuthology.orchestra.run.smithi002.stdout: "root": "/" 2020-12-05T17:42:07.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:42:07.425 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_count": 0, 2020-12-05T17:42:07.425 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_pinned_count": 0, 2020-12-05T17:42:07.426 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 8796, 2020-12-05T17:42:07.426 INFO:teuthology.orchestra.run.smithi002.stdout: "inst": { 2020-12-05T17:42:07.426 INFO:teuthology.orchestra.run.smithi002.stdout: "name": { 2020-12-05T17:42:07.426 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "client", 2020-12-05T17:42:07.427 INFO:teuthology.orchestra.run.smithi002.stdout: "num": 8796 2020-12-05T17:42:07.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:42:07.427 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2020-12-05T17:42:07.427 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2020-12-05T17:42:07.428 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2020-12-05T17:42:07.428 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2860065831 2020-12-05T17:42:07.428 INFO:teuthology.orchestra.run.smithi002.stdout: } 2020-12-05T17:42:07.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:42:07.429 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2020-12-05T17:42:07.429 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2020-12-05T17:42:07.429 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2020-12-05T17:42:07.429 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2860065831 2020-12-05T17:42:07.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2020-12-05T17:42:07.430 INFO:teuthology.orchestra.run.smithi002.stdout: "inst_str": "client.8796 172.21.15.2:0/2860065831", 2020-12-05T17:42:07.430 INFO:teuthology.orchestra.run.smithi002.stdout: "addr_str": "172.21.15.2:0/2860065831", 2020-12-05T17:42:07.430 INFO:teuthology.orchestra.run.smithi002.stdout: "inode_count": 1, 2020-12-05T17:42:07.430 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_epoch": 73, 2020-12-05T17:42:07.431 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch": 148, 2020-12-05T17:42:07.431 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch_barrier": 0, 2020-12-05T17:42:07.431 INFO:teuthology.orchestra.run.smithi002.stdout: "blacklisted": false 2020-12-05T17:42:07.431 INFO:teuthology.orchestra.run.smithi002.stdout:} 2020-12-05T17:42:07.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:42:07.433 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:42:07.433 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:42:07.439 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:42:07.440 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:42:07.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:42:07.441 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:42:07.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:42:07.501 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:42:07.502 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:42:07.549 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:42:07.550 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:42:07.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:42:07.551 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:42:07.551 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:42:07.601 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2020-12-05T17:42:07.602 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-12-05T17:42:07.602 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi002.front.sepia.ceph.com... 2020-12-05T17:42:07.603 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-12-05T17:42:07.603 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:42:07.675 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi002.stderr:ceph-fuse[11283]: fuse finished with error 0 and tester_r 0 2020-12-05T17:42:07.676 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:42:07.677 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:42:07.677 INFO:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:42:07.725 INFO:teuthology.orchestra.run.smithi002.stdout:ext2/ext3 2020-12-05T17:42:07.726 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-12-05T17:42:07.727 INFO:teuthology.orchestra.run:waiting for 900 2020-12-05T17:42:07.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-12-05T17:42:07.790 INFO:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-12-05T17:42:07.802 INFO:teuthology.orchestra.run.smithi036:> cd /home/ubuntu/cephtest/mnt.0 && sudo bash -c 'stat -f .' 2020-12-05T17:42:07.824 INFO:teuthology.orchestra.run.smithi036.stdout: File: "." 2020-12-05T17:42:07.825 INFO:teuthology.orchestra.run.smithi036.stdout: ID: 0 Namelen: 255 Type: fuseblk 2020-12-05T17:42:07.825 INFO:teuthology.orchestra.run.smithi036.stdout:Block size: 4194304 Fundamental block size: 4194304 2020-12-05T17:42:07.825 INFO:teuthology.orchestra.run.smithi036.stdout:Blocks: Total: 86526 Free: 86526 Available: 86526 2020-12-05T17:42:07.825 INFO:teuthology.orchestra.run.smithi036.stdout:Inodes: Total: 1 Free: 0 2020-12-05T17:42:07.826 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-12-05T17:42:07.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.945+0000 7f6e5c99c700 1 Processor -- start 2020-12-05T17:42:07.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.945+0000 7f6e5c99c700 1 -- start start 2020-12-05T17:42:07.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.945+0000 7f6e5c99c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f6e580b52f0 0x7f6e580b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:07.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.945+0000 7f6e5c99c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6e580b1870 0x7f6e58138810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:07.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.945+0000 7f6e5c99c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e581420f0 con 0x7f6e58138d00 2020-12-05T17:42:07.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.945+0000 7f6e5c99c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e580ad4f0 con 0x7f6e580b1870 2020-12-05T17:42:07.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.945+0000 7f6e5c99c700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e58067b90 con 0x7f6e580b52f0 2020-12-05T17:42:07.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.949+0000 7f6e56ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6e580b1870 0x7f6e58138810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:07.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.949+0000 7f6e56ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6e580b1870 0x7f6e58138810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41810/0 (socket says 172.21.15.32:41810) 2020-12-05T17:42:07.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.949+0000 7f6e56ffd700 1 -- 172.21.15.32:0/1275166202 learned_addr learned my addr 172.21.15.32:0/1275166202 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:07.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.949+0000 7f6e577fe700 1 --2- 172.21.15.32:0/1275166202 >> v2:172.21.15.61:3300/0 conn(0x7f6e580b52f0 0x7f6e580b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:07.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.949+0000 7f6e56ffd700 1 -- 172.21.15.32:0/1275166202 >> v2:172.21.15.61:3300/0 conn(0x7f6e580b52f0 msgr2=0x7f6e580b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:42:07.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.949+0000 7f6e56ffd700 1 --2- 172.21.15.32:0/1275166202 >> v2:172.21.15.61:3300/0 conn(0x7f6e580b52f0 0x7f6e580b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:07.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.949+0000 7f6e56ffd700 1 -- 172.21.15.32:0/1275166202 >> v1:172.21.15.61:6789/0 conn(0x7f6e58138d00 legacy=0x7f6e58067480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:07.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.949+0000 7f6e56ffd700 1 -- 172.21.15.32:0/1275166202 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e580b0380 con 0x7f6e580b1870 2020-12-05T17:42:07.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.949+0000 7f6e56ffd700 1 --2- 172.21.15.32:0/1275166202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e580b1870 0x7f6e58138810 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7f6e58067cd0 tx=0x7f6e4c006e80).ready entity=mon.0 client_cookie=f65a665778842aa0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:07.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.949+0000 7f6e567fc700 1 -- 172.21.15.32:0/1275166202 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e4c0099a0 con 0x7f6e580b1870 2020-12-05T17:42:07.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.949+0000 7f6e567fc700 1 -- 172.21.15.32:0/1275166202 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4c009b00 con 0x7f6e580b1870 2020-12-05T17:42:07.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.949+0000 7f6e567fc700 1 -- 172.21.15.32:0/1275166202 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e4c009e60 con 0x7f6e580b1870 2020-12-05T17:42:07.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.949+0000 7f6e5c99c700 1 -- 172.21.15.32:0/1275166202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e580b1870 msgr2=0x7f6e58138810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:07.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.949+0000 7f6e5c99c700 1 --2- 172.21.15.32:0/1275166202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e580b1870 0x7f6e58138810 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7f6e58067cd0 tx=0x7f6e4c006e80).stop 2020-12-05T17:42:07.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.949+0000 7f6e5c99c700 1 -- 172.21.15.32:0/1275166202 shutdown_connections 2020-12-05T17:42:07.957 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.949+0000 7f6e5c99c700 1 --2- 172.21.15.32:0/1275166202 >> v2:172.21.15.61:3300/0 conn(0x7f6e580b52f0 0x7f6e580b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:07.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.949+0000 7f6e5c99c700 1 --2- 172.21.15.32:0/1275166202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e580b1870 0x7f6e58138810 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:07.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.949+0000 7f6e5c99c700 1 -- 172.21.15.32:0/1275166202 >> 172.21.15.32:0/1275166202 conn(0x7f6e5805bbb0 msgr2=0x7f6e58062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:07.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.949+0000 7f6e5c99c700 1 -- 172.21.15.32:0/1275166202 shutdown_connections 2020-12-05T17:42:07.958 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.949+0000 7f6e5c99c700 1 -- 172.21.15.32:0/1275166202 wait complete. 2020-12-05T17:42:07.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.949+0000 7f6e5c99c700 1 Processor -- start 2020-12-05T17:42:07.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.949+0000 7f6e5c99c700 1 -- start start 2020-12-05T17:42:07.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.953+0000 7f6e5c99c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e580b1870 0x7f6e581401b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:07.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.953+0000 7f6e5c99c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e580b52f0 0x7f6e58064950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:07.959 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.953+0000 7f6e5c99c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6e58138d00 0x7f6e5814d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:07.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.953+0000 7f6e5c99c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e58141e30 con 0x7f6e580b1870 2020-12-05T17:42:07.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.953+0000 7f6e5c99c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5813b580 con 0x7f6e580b52f0 2020-12-05T17:42:07.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.953+0000 7f6e5c99c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e58064210 con 0x7f6e58138d00 2020-12-05T17:42:07.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.953+0000 7f6e577fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e580b1870 0x7f6e581401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:07.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.953+0000 7f6e577fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e580b1870 0x7f6e581401b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41816/0 (socket says 172.21.15.32:41816) 2020-12-05T17:42:07.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.953+0000 7f6e577fe700 1 -- 172.21.15.32:0/1342584387 learned_addr learned my addr 172.21.15.32:0/1342584387 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:07.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.953+0000 7f6e56ffd700 1 --2- 172.21.15.32:0/1342584387 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e580b52f0 0x7f6e58064950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:07.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.953+0000 7f6e57fff700 1 --2- 172.21.15.32:0/1342584387 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6e58138d00 0x7f6e5814d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:07.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.953+0000 7f6e577fe700 1 -- 172.21.15.32:0/1342584387 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6e58138d00 msgr2=0x7f6e5814d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:07.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.953+0000 7f6e577fe700 1 --2- 172.21.15.32:0/1342584387 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6e58138d00 0x7f6e5814d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:07.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.953+0000 7f6e577fe700 1 -- 172.21.15.32:0/1342584387 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e580b52f0 msgr2=0x7f6e58064950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:07.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.953+0000 7f6e577fe700 1 --2- 172.21.15.32:0/1342584387 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e580b52f0 0x7f6e58064950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:07.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.953+0000 7f6e577fe700 1 -- 172.21.15.32:0/1342584387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e580644b0 con 0x7f6e580b1870 2020-12-05T17:42:07.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.953+0000 7f6e577fe700 1 --2- 172.21.15.32:0/1342584387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e580b1870 0x7f6e581401b0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 rx=0x7f6e4800c000 tx=0x7f6e48006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:07.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.953+0000 7f6e54ff9700 1 -- 172.21.15.32:0/1342584387 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e4800ca10 con 0x7f6e580b1870 2020-12-05T17:42:07.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.953+0000 7f6e54ff9700 1 -- 172.21.15.32:0/1342584387 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4800cb70 con 0x7f6e580b1870 2020-12-05T17:42:07.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.953+0000 7f6e54ff9700 1 -- 172.21.15.32:0/1342584387 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e480123f0 con 0x7f6e580b1870 2020-12-05T17:42:07.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.953+0000 7f6e5c99c700 1 -- 172.21.15.32:0/1342584387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e581406a0 con 0x7f6e580b1870 2020-12-05T17:42:07.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.953+0000 7f6e5c99c700 1 -- 172.21.15.32:0/1342584387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e58064e40 con 0x7f6e580b1870 2020-12-05T17:42:07.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.957+0000 7f6e5c99c700 1 -- 172.21.15.32:0/1342584387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e58057050 con 0x7f6e580b1870 2020-12-05T17:42:07.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.957+0000 7f6e54ff9700 1 -- 172.21.15.32:0/1342584387 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6e48012550 con 0x7f6e580b1870 2020-12-05T17:42:07.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.957+0000 7f6e54ff9700 1 --2- 172.21.15.32:0/1342584387 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6e380553b0 0x7f6e38057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:07.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.957+0000 7f6e54ff9700 1 -- 172.21.15.32:0/1342584387 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6e4806e510 con 0x7f6e580b1870 2020-12-05T17:42:07.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.957+0000 7f6e56ffd700 1 --2- 172.21.15.32:0/1342584387 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6e380553b0 0x7f6e38057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:07.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.957+0000 7f6e56ffd700 1 --2- 172.21.15.32:0/1342584387 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6e380553b0 0x7f6e38057870 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f6e4c01e000 tx=0x7f6e4c017000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:07.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:07.965+0000 7f6e54ff9700 1 -- 172.21.15.32:0/1342584387 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6e48044650 con 0x7f6e580b1870 2020-12-05T17:42:08.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.105+0000 7f6e5c99c700 1 -- 172.21.15.32:0/1342584387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f6e5805c300 con 0x7f6e580b1870 2020-12-05T17:42:08.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.105+0000 7f6e54ff9700 1 -- 172.21.15.32:0/1342584387 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v148) v1 ==== 81+0+34051 (secure 0 0 0) 0x7f6e48047c70 con 0x7f6e580b1870 2020-12-05T17:42:08.108 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:42:08.108 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T17:42:08.108 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 148, 2020-12-05T17:42:08.108 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "efd7e861-a502-461d-b20d-3b2c0a25fbd9", 2020-12-05T17:42:08.109 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2020-12-05T17:31:20.975238+0000", 2020-12-05T17:42:08.109 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2020-12-05T17:41:56.666368+0000", 2020-12-05T17:42:08.109 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:42:08.109 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2020-12-05T17:31:28.836431+0000", 2020-12-05T17:42:08.109 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-12-05T17:42:08.110 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2020-12-05T17:42:08.110 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2020-12-05T17:42:08.110 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2020-12-05T17:42:08.110 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2020-12-05T17:42:08.111 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2020-12-05T17:42:08.111 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2020-12-05T17:42:08.111 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:42:08.111 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2020-12-05T17:42:08.111 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2020-12-05T17:42:08.112 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-12-05T17:42:08.112 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2020-12-05T17:42:08.112 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2020-12-05T17:42:08.112 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 30, 2020-12-05T17:42:08.112 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2020-12-05T17:42:08.113 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2020-12-05T17:42:08.113 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2020-12-05T17:42:08.113 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2020-12-05T17:42:08.113 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2020-12-05T17:42:08.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.114 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2020-12-05T17:42:08.114 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2020-12-05T17:42:08.114 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:24.858732+0000", 2020-12-05T17:42:08.114 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:42:08.115 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:42:08.115 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:42:08.115 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:42:08.116 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:42:08.116 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:42:08.116 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:42:08.116 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:42:08.116 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2020-12-05T17:42:08.117 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2020-12-05T17:42:08.117 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2020-12-05T17:42:08.117 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2020-12-05T17:42:08.117 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2020-12-05T17:42:08.118 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:42:08.118 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:42:08.118 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:42:08.118 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:42:08.118 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:42:08.119 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:42:08.119 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:42:08.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.119 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "16", 2020-12-05T17:42:08.120 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:42:08.120 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:42:08.120 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:42:08.120 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:42:08.120 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:42:08.121 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:42:08.121 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:42:08.121 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:42:08.121 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:42:08.121 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:42:08.122 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:42:08.122 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:42:08.122 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:42:08.122 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:42:08.123 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:42:08.123 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:42:08.123 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:42:08.123 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:42:08.124 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:42:08.124 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:42:08.124 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:42:08.124 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:42:08.125 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:42:08.125 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:42:08.125 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:42:08.125 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:42:08.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.126 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:42:08.126 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:42:08.126 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:42:08.126 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:42:08.126 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:42:08.127 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:42:08.127 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:42:08.127 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:42:08.127 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:42:08.127 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:42:08.128 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:42:08.128 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:42:08.128 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2020-12-05T17:42:08.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.129 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:42:08.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2020-12-05T17:42:08.129 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.130 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2020-12-05T17:42:08.130 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2020-12-05T17:42:08.130 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:31:33.305595+0000", 2020-12-05T17:42:08.130 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:42:08.130 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:42:08.131 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:42:08.131 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:42:08.131 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:42:08.131 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:42:08.131 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:42:08.132 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:42:08.132 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:42:08.132 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:42:08.132 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:42:08.132 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:42:08.133 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:42:08.133 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:42:08.133 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:42:08.133 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:42:08.133 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:42:08.134 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:42:08.134 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:42:08.134 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:42:08.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.135 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "19", 2020-12-05T17:42:08.135 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:42:08.135 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:42:08.135 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:42:08.135 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:42:08.136 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:42:08.136 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:42:08.136 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:42:08.136 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:42:08.136 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:42:08.137 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:42:08.137 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:42:08.137 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:42:08.137 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:42:08.138 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:42:08.138 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:42:08.138 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:42:08.138 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:42:08.138 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:42:08.139 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:42:08.139 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:42:08.139 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:42:08.139 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:42:08.139 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:42:08.140 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:42:08.140 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:42:08.140 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:42:08.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.140 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:42:08.141 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:42:08.141 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:42:08.141 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:42:08.141 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:42:08.142 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:42:08.142 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:42:08.142 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:42:08.142 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:42:08.142 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:42:08.143 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:42:08.143 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:42:08.143 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:42:08.143 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2020-12-05T17:42:08.143 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.144 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 29, 2020-12-05T17:42:08.144 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2020-12-05T17:42:08.145 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:41:53.238803+0000", 2020-12-05T17:42:08.145 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:42:08.145 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:42:08.145 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:42:08.145 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:42:08.146 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:42:08.146 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:42:08.146 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:42:08.146 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:42:08.147 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:42:08.147 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:42:08.147 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:42:08.147 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:42:08.147 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:42:08.148 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:42:08.148 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:42:08.148 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:42:08.148 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:42:08.148 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:42:08.149 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:42:08.149 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:42:08.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.149 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "147", 2020-12-05T17:42:08.149 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:42:08.150 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:42:08.150 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:42:08.150 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:42:08.150 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:42:08.151 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:42:08.151 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:42:08.151 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:42:08.151 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:42:08.151 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:42:08.152 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:42:08.152 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:42:08.152 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:42:08.152 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:42:08.152 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:42:08.153 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:42:08.153 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:42:08.153 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:42:08.153 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:42:08.153 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:42:08.154 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:42:08.154 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:42:08.154 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:42:08.154 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:42:08.155 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:42:08.155 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:42:08.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.155 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:42:08.155 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:42:08.156 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:42:08.156 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:42:08.156 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:42:08.156 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:42:08.156 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:42:08.157 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:42:08.157 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:42:08.157 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:42:08.157 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:42:08.157 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2020-12-05T17:42:08.158 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2020-12-05T17:42:08.158 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2020-12-05T17:42:08.158 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2020-12-05T17:42:08.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.158 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:42:08.159 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:42:08.159 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2020-12-05T17:42:08.159 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.159 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.160 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 30, 2020-12-05T17:42:08.160 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2020-12-05T17:42:08.160 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2020-12-05T17:41:54.541493+0000", 2020-12-05T17:42:08.161 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2020-12-05T17:42:08.161 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2020-12-05T17:42:08.161 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2020-12-05T17:42:08.161 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2020-12-05T17:42:08.161 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2020-12-05T17:42:08.161 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2020-12-05T17:42:08.162 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2020-12-05T17:42:08.162 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2020-12-05T17:42:08.162 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2020-12-05T17:42:08.162 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2020-12-05T17:42:08.163 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2020-12-05T17:42:08.163 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2020-12-05T17:42:08.163 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2020-12-05T17:42:08.163 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2020-12-05T17:42:08.163 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2020-12-05T17:42:08.164 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2020-12-05T17:42:08.164 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2020-12-05T17:42:08.164 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2020-12-05T17:42:08.164 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2020-12-05T17:42:08.164 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2020-12-05T17:42:08.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.165 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "148", 2020-12-05T17:42:08.165 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2020-12-05T17:42:08.165 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2020-12-05T17:42:08.166 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2020-12-05T17:42:08.166 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2020-12-05T17:42:08.166 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2020-12-05T17:42:08.166 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2020-12-05T17:42:08.166 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2020-12-05T17:42:08.167 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2020-12-05T17:42:08.167 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2020-12-05T17:42:08.167 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2020-12-05T17:42:08.167 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2020-12-05T17:42:08.168 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2020-12-05T17:42:08.168 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2020-12-05T17:42:08.168 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2020-12-05T17:42:08.168 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2020-12-05T17:42:08.168 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2020-12-05T17:42:08.169 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2020-12-05T17:42:08.169 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2020-12-05T17:42:08.169 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-12-05T17:42:08.169 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-12-05T17:42:08.169 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2020-12-05T17:42:08.170 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2020-12-05T17:42:08.170 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2020-12-05T17:42:08.170 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2020-12-05T17:42:08.170 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2020-12-05T17:42:08.170 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2020-12-05T17:42:08.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.171 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2020-12-05T17:42:08.171 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2020-12-05T17:42:08.171 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2020-12-05T17:42:08.171 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2020-12-05T17:42:08.171 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2020-12-05T17:42:08.172 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2020-12-05T17:42:08.172 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2020-12-05T17:42:08.172 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2020-12-05T17:42:08.172 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2020-12-05T17:42:08.173 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2020-12-05T17:42:08.173 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2020-12-05T17:42:08.173 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2020-12-05T17:42:08.173 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2020-12-05T17:42:08.173 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2020-12-05T17:42:08.174 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2020-12-05T17:42:08.174 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.174 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.174 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.174 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:42:08.175 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2020-12-05T17:42:08.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.175 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:42:08.175 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "b941127b-0a49-4e06-8d11-17794ff8b013", 2020-12-05T17:42:08.175 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:42:08.176 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:42:08.176 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:42:08.176 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:42:08.176 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:42:08.176 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:42:08.177 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:42:08.177 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 146, 2020-12-05T17:42:08.177 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:42:08.177 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:42:08.177 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:42:08.178 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.178 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.178 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2020-12-05T17:42:08.178 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:42:08.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.179 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.179 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2020-12-05T17:42:08.180 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:42:08.180 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.180 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.180 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:42:08.181 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.181 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.181 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2020-12-05T17:42:08.181 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:42:08.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.182 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.182 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2020-12-05T17:42:08.182 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:42:08.183 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.183 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.183 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:42:08.183 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.184 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.184 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2020-12-05T17:42:08.184 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:42:08.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.185 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.185 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2020-12-05T17:42:08.186 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:42:08.186 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.186 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.187 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:42:08.187 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.187 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.187 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2020-12-05T17:42:08.188 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:42:08.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.188 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.188 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2020-12-05T17:42:08.189 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11526 2020-12-05T17:42:08.189 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.189 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.190 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6803/11526", 2020-12-05T17:42:08.190 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6805/11526", 2020-12-05T17:42:08.190 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6809/11526", 2020-12-05T17:42:08.190 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6807/11526", 2020-12-05T17:42:08.191 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:42:08.191 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:42:08.191 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:42:08.191 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.192 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:42:08.192 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "01bd7ff5-a86b-4e11-ba1e-484c309c0fba", 2020-12-05T17:42:08.192 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:42:08.192 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:42:08.193 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:42:08.193 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:42:08.193 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:42:08.193 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:42:08.194 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:42:08.194 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 145, 2020-12-05T17:42:08.194 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:42:08.194 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:42:08.194 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:42:08.195 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.195 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.195 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2020-12-05T17:42:08.195 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:42:08.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.196 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.196 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2020-12-05T17:42:08.196 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:42:08.197 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.197 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.197 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:42:08.198 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.198 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.198 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2020-12-05T17:42:08.198 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:42:08.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.199 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.199 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2020-12-05T17:42:08.199 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:42:08.200 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.200 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.200 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:42:08.201 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.201 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.201 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2020-12-05T17:42:08.201 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:42:08.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.202 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.202 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2020-12-05T17:42:08.202 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:42:08.203 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.203 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.203 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:42:08.203 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.204 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.204 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2020-12-05T17:42:08.204 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:42:08.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.205 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.205 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2020-12-05T17:42:08.205 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11538 2020-12-05T17:42:08.206 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.206 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.206 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6819/11538", 2020-12-05T17:42:08.206 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6821/11538", 2020-12-05T17:42:08.207 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6825/11538", 2020-12-05T17:42:08.207 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6823/11538", 2020-12-05T17:42:08.207 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:42:08.207 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:42:08.207 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:42:08.208 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.208 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:42:08.209 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "94736ef5-64c2-4a97-b963-bd38cddd9c52", 2020-12-05T17:42:08.209 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:42:08.209 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:42:08.209 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:42:08.209 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:42:08.210 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:42:08.210 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:42:08.210 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:42:08.210 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 146, 2020-12-05T17:42:08.210 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:42:08.211 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:42:08.211 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:42:08.211 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.211 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.212 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2020-12-05T17:42:08.212 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:42:08.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.212 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.213 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2020-12-05T17:42:08.213 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:42:08.213 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.213 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.214 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:42:08.214 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.214 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.215 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2020-12-05T17:42:08.215 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:42:08.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.215 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.216 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2020-12-05T17:42:08.216 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:42:08.216 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.216 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.217 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:42:08.217 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.217 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.218 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2020-12-05T17:42:08.218 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:42:08.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.218 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.219 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2020-12-05T17:42:08.219 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:42:08.219 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.219 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.220 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:42:08.220 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.220 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.221 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2020-12-05T17:42:08.221 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:42:08.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.221 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.222 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2020-12-05T17:42:08.222 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11521 2020-12-05T17:42:08.222 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.222 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.223 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6811/11521", 2020-12-05T17:42:08.223 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6813/11521", 2020-12-05T17:42:08.223 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6817/11521", 2020-12-05T17:42:08.223 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6815/11521", 2020-12-05T17:42:08.224 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:42:08.224 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:42:08.224 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:42:08.224 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.225 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:42:08.225 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "69a1f3ff-fd25-43df-b60c-e32a89c35234", 2020-12-05T17:42:08.225 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:42:08.225 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:42:08.226 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:42:08.226 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:42:08.226 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:42:08.226 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:42:08.226 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:42:08.227 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 146, 2020-12-05T17:42:08.227 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:42:08.227 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:42:08.227 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:42:08.228 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.228 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.228 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2020-12-05T17:42:08.228 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:42:08.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.229 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.229 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2020-12-05T17:42:08.229 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:42:08.230 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.230 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.230 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:42:08.230 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.231 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.231 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2020-12-05T17:42:08.231 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:42:08.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.232 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.232 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2020-12-05T17:42:08.232 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:42:08.232 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.233 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.233 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:42:08.233 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.234 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.234 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6832", 2020-12-05T17:42:08.234 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:42:08.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.235 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.235 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6833", 2020-12-05T17:42:08.235 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:42:08.236 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.236 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.236 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:42:08.236 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.237 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.237 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2020-12-05T17:42:08.237 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:42:08.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.238 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.238 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2020-12-05T17:42:08.238 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11535 2020-12-05T17:42:08.238 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.239 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.239 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6827/11535", 2020-12-05T17:42:08.239 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6829/11535", 2020-12-05T17:42:08.240 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6833/11535", 2020-12-05T17:42:08.240 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6831/11535", 2020-12-05T17:42:08.240 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:42:08.240 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:42:08.240 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:42:08.241 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.241 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:42:08.241 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c301fe4e-5eab-402c-ae6f-db3e6a7181af", 2020-12-05T17:42:08.242 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:42:08.242 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:42:08.242 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:42:08.242 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:42:08.242 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:42:08.243 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:42:08.243 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:42:08.243 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 144, 2020-12-05T17:42:08.243 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:42:08.243 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:42:08.244 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:42:08.244 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.244 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.244 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6808", 2020-12-05T17:42:08.245 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:42:08.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.245 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.246 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6809", 2020-12-05T17:42:08.246 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:42:08.246 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.246 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.247 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:42:08.247 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.247 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.247 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6810", 2020-12-05T17:42:08.248 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:42:08.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.248 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.248 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6811", 2020-12-05T17:42:08.249 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:42:08.249 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.249 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.250 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:42:08.250 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.250 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.250 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6817", 2020-12-05T17:42:08.251 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:42:08.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.251 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.251 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6819", 2020-12-05T17:42:08.252 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:42:08.252 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.252 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.252 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:42:08.253 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.253 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.253 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6812", 2020-12-05T17:42:08.254 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:42:08.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.254 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.254 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6814", 2020-12-05T17:42:08.255 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11541 2020-12-05T17:42:08.255 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.255 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.255 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6809/11541", 2020-12-05T17:42:08.256 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6811/11541", 2020-12-05T17:42:08.256 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6819/11541", 2020-12-05T17:42:08.256 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6814/11541", 2020-12-05T17:42:08.256 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:42:08.257 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:42:08.257 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:42:08.257 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.258 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:42:08.258 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "9476eff4-345b-46c5-a473-1d9a89a6d327", 2020-12-05T17:42:08.258 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:42:08.258 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:42:08.258 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:42:08.259 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:42:08.259 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:42:08.259 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:42:08.259 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:42:08.259 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 146, 2020-12-05T17:42:08.260 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:42:08.260 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:42:08.260 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:42:08.260 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.261 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.261 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6800", 2020-12-05T17:42:08.261 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:42:08.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.262 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.262 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6801", 2020-12-05T17:42:08.262 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:42:08.262 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.263 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.263 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:42:08.263 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.264 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.264 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6802", 2020-12-05T17:42:08.264 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:42:08.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.265 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.265 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6803", 2020-12-05T17:42:08.265 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:42:08.265 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.265 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.266 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:42:08.266 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.266 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.267 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6806", 2020-12-05T17:42:08.267 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:42:08.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.268 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.268 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6807", 2020-12-05T17:42:08.268 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:42:08.268 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.268 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.269 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:42:08.269 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.270 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.270 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6804", 2020-12-05T17:42:08.270 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:42:08.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.271 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.271 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6805", 2020-12-05T17:42:08.271 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11539 2020-12-05T17:42:08.271 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.272 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.272 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6801/11539", 2020-12-05T17:42:08.272 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6803/11539", 2020-12-05T17:42:08.272 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6807/11539", 2020-12-05T17:42:08.273 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6805/11539", 2020-12-05T17:42:08.273 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:42:08.273 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:42:08.273 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:42:08.273 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.274 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:42:08.274 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8dfd8567-9b1c-4d13-9c96-071183e469c2", 2020-12-05T17:42:08.274 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:42:08.275 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:42:08.275 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:42:08.275 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:42:08.275 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:42:08.276 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:42:08.276 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:42:08.276 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 146, 2020-12-05T17:42:08.276 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:42:08.276 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:42:08.277 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:42:08.277 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.277 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.277 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6820", 2020-12-05T17:42:08.278 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:42:08.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.278 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.278 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6822", 2020-12-05T17:42:08.279 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:42:08.279 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.279 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.279 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:42:08.280 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.280 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.280 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6823", 2020-12-05T17:42:08.281 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:42:08.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.281 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.281 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6825", 2020-12-05T17:42:08.282 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:42:08.282 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.282 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.282 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:42:08.283 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.283 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.283 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6829", 2020-12-05T17:42:08.283 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:42:08.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.284 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.284 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6830", 2020-12-05T17:42:08.285 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:42:08.285 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.285 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.285 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:42:08.286 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.286 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.286 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6826", 2020-12-05T17:42:08.286 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:42:08.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.287 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.287 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6827", 2020-12-05T17:42:08.288 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11542 2020-12-05T17:42:08.288 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.288 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.288 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6822/11542", 2020-12-05T17:42:08.289 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6825/11542", 2020-12-05T17:42:08.289 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6830/11542", 2020-12-05T17:42:08.289 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6827/11542", 2020-12-05T17:42:08.289 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:42:08.289 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:42:08.290 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:42:08.290 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.291 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:42:08.291 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "24d466b3-01dc-4ba2-8b81-85250d163bd2", 2020-12-05T17:42:08.291 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2020-12-05T17:42:08.291 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2020-12-05T17:42:08.292 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2020-12-05T17:42:08.292 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2020-12-05T17:42:08.292 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2020-12-05T17:42:08.292 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2020-12-05T17:42:08.292 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2020-12-05T17:42:08.293 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 146, 2020-12-05T17:42:08.293 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2020-12-05T17:42:08.293 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2020-12-05T17:42:08.293 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2020-12-05T17:42:08.294 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.294 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.294 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6813", 2020-12-05T17:42:08.294 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:42:08.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.295 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.295 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6815", 2020-12-05T17:42:08.296 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:42:08.296 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.296 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.296 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2020-12-05T17:42:08.297 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.297 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.297 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6816", 2020-12-05T17:42:08.298 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:42:08.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.298 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.298 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6818", 2020-12-05T17:42:08.299 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:42:08.299 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.299 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.299 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2020-12-05T17:42:08.300 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.300 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.300 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6828", 2020-12-05T17:42:08.300 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:42:08.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.301 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.301 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6831", 2020-12-05T17:42:08.301 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:42:08.302 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.302 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.302 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2020-12-05T17:42:08.303 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2020-12-05T17:42:08.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.303 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2020-12-05T17:42:08.303 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6821", 2020-12-05T17:42:08.304 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:42:08.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.304 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2020-12-05T17:42:08.304 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.61:6824", 2020-12-05T17:42:08.305 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 11540 2020-12-05T17:42:08.305 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.305 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.305 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.61:6815/11540", 2020-12-05T17:42:08.306 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.61:6818/11540", 2020-12-05T17:42:08.306 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.61:6831/11540", 2020-12-05T17:42:08.306 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.61:6824/11540", 2020-12-05T17:42:08.306 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2020-12-05T17:42:08.306 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2020-12-05T17:42:08.307 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2020-12-05T17:42:08.307 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2020-12-05T17:42:08.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.307 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:42:08.307 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2020-12-05T17:42:08.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.308 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2020-12-05T17:42:08.308 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:42:08.308 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:42:08.309 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:42:08.309 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:42:08.309 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:42:08.309 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.398868+0000", 2020-12-05T17:42:08.309 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:42:08.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.310 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2020-12-05T17:42:08.310 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:42:08.310 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:42:08.311 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:42:08.311 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:42:08.311 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:42:08.311 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.675748+0000", 2020-12-05T17:42:08.312 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:42:08.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.312 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2020-12-05T17:42:08.312 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:42:08.313 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:42:08.313 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:42:08.313 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:42:08.313 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:42:08.313 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.620888+0000", 2020-12-05T17:42:08.314 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:42:08.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.314 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2020-12-05T17:42:08.315 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:42:08.315 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:42:08.315 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:42:08.315 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:42:08.315 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:42:08.316 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.681600+0000", 2020-12-05T17:42:08.316 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:42:08.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.317 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2020-12-05T17:42:08.317 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:42:08.317 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:42:08.317 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:42:08.317 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:42:08.318 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:42:08.318 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.658769+0000", 2020-12-05T17:42:08.318 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:42:08.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.319 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2020-12-05T17:42:08.319 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:42:08.319 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:42:08.319 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:42:08.320 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:42:08.320 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:42:08.320 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.630708+0000", 2020-12-05T17:42:08.320 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:42:08.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.321 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2020-12-05T17:42:08.321 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:42:08.321 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:42:08.322 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:42:08.322 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:42:08.322 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:42:08.322 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.648352+0000", 2020-12-05T17:42:08.322 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:42:08.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2020-12-05T17:42:08.323 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2020-12-05T17:42:08.323 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2020-12-05T17:42:08.324 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2020-12-05T17:42:08.324 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2020-12-05T17:42:08.324 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2020-12-05T17:42:08.324 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2020-12-05T17:42:08.324 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2020-12-05T17:31:27.629399+0000", 2020-12-05T17:42:08.325 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2020-12-05T17:42:08.325 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.325 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2020-12-05T17:42:08.325 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2020-12-05T17:42:08.325 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2020-12-05T17:42:08.326 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2020-12-05T17:42:08.326 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2020-12-05T17:42:08.326 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": {}, 2020-12-05T17:42:08.326 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2020-12-05T17:42:08.327 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2020-12-05T17:42:08.327 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2020-12-05T17:42:08.327 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:42:08.327 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2020-12-05T17:42:08.327 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2020-12-05T17:42:08.328 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:42:08.328 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2020-12-05T17:42:08.328 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:42:08.328 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:42:08.328 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2020-12-05T17:42:08.329 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2020-12-05T17:42:08.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.329 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2020-12-05T17:42:08.329 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2020-12-05T17:42:08.330 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2020-12-05T17:42:08.330 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2020-12-05T17:42:08.330 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2020-12-05T17:42:08.330 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2020-12-05T17:42:08.330 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2020-12-05T17:42:08.331 INFO:teuthology.orchestra.run.smithi032.stdout: } 2020-12-05T17:42:08.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2020-12-05T17:42:08.331 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2020-12-05T17:42:08.331 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2020-12-05T17:42:08.332 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2020-12-05T17:42:08.332 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2020-12-05T17:42:08.332 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2020-12-05T17:42:08.332 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T17:42:08.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.109+0000 7f6e5c99c700 1 -- 172.21.15.32:0/1342584387 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6e380553b0 msgr2=0x7f6e38057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:08.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.109+0000 7f6e5c99c700 1 --2- 172.21.15.32:0/1342584387 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6e380553b0 0x7f6e38057870 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f6e4c01e000 tx=0x7f6e4c017000).stop 2020-12-05T17:42:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.109+0000 7f6e5c99c700 1 -- 172.21.15.32:0/1342584387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e580b1870 msgr2=0x7f6e581401b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.109+0000 7f6e5c99c700 1 --2- 172.21.15.32:0/1342584387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e580b1870 0x7f6e581401b0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 rx=0x7f6e4800c000 tx=0x7f6e48006dc0).stop 2020-12-05T17:42:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.109+0000 7f6e5c99c700 1 -- 172.21.15.32:0/1342584387 shutdown_connections 2020-12-05T17:42:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.109+0000 7f6e5c99c700 1 --2- 172.21.15.32:0/1342584387 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6e380553b0 0x7f6e38057870 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.109+0000 7f6e5c99c700 1 --2- 172.21.15.32:0/1342584387 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6e58138d00 0x7f6e5814d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:08.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.109+0000 7f6e5c99c700 1 --2- 172.21.15.32:0/1342584387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e580b1870 0x7f6e581401b0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:08.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.109+0000 7f6e5c99c700 1 --2- 172.21.15.32:0/1342584387 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e580b52f0 0x7f6e58064950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:08.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.109+0000 7f6e5c99c700 1 -- 172.21.15.32:0/1342584387 >> 172.21.15.32:0/1342584387 conn(0x7f6e5805bbb0 msgr2=0x7f6e58062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:08.336 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.109+0000 7f6e5c99c700 1 -- 172.21.15.32:0/1342584387 shutdown_connections 2020-12-05T17:42:08.337 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.109+0000 7f6e5c99c700 1 -- 172.21.15.32:0/1342584387 wait complete. 2020-12-05T17:42:08.337 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:42:08.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.453+0000 7f1df82c4700 1 Processor -- start 2020-12-05T17:42:08.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.453+0000 7f1df82c4700 1 -- start start 2020-12-05T17:42:08.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df82c4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1df00aead0 0x7f1df00aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:08.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df82c4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1df00ad1b0 0x7f1df01389f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:08.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df82c4700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1df0138ee0 0x7f1df0142290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:08.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df82c4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df013fb70 con 0x7f1df00ad1b0 2020-12-05T17:42:08.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df82c4700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df00af460 con 0x7f1df0138ee0 2020-12-05T17:42:08.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df82c4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df00af700 con 0x7f1df00aead0 2020-12-05T17:42:08.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df585f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1df00ad1b0 0x7f1df01389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:08.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df585f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1df00ad1b0 0x7f1df01389f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41824/0 (socket says 172.21.15.32:41824) 2020-12-05T17:42:08.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df585f700 1 -- 172.21.15.32:0/441432991 learned_addr learned my addr 172.21.15.32:0/441432991 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df6861700 1 --2- 172.21.15.32:0/441432991 >> v2:172.21.15.61:3300/0 conn(0x7f1df0138ee0 0x7f1df0142290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df6060700 1 --2- 172.21.15.32:0/441432991 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1df00aead0 0x7f1df00aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df585f700 1 -- 172.21.15.32:0/441432991 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1df00aead0 msgr2=0x7f1df00aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:42:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df585f700 1 --2- 172.21.15.32:0/441432991 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1df00aead0 0x7f1df00aeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df585f700 1 -- 172.21.15.32:0/441432991 >> v2:172.21.15.61:3300/0 conn(0x7f1df0138ee0 msgr2=0x7f1df0142290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:42:08.462 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df585f700 1 --2- 172.21.15.32:0/441432991 >> v2:172.21.15.61:3300/0 conn(0x7f1df0138ee0 0x7f1df0142290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:08.462 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df585f700 1 -- 172.21.15.32:0/441432991 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df013fdf0 con 0x7f1df00ad1b0 2020-12-05T17:42:08.462 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df585f700 1 --2- 172.21.15.32:0/441432991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df00ad1b0 0x7f1df01389f0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 rx=0x7f1de0002f30 tx=0x7f1de0006e80).ready entity=mon.0 client_cookie=d0a714194f984263 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:08.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df505e700 1 -- 172.21.15.32:0/441432991 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1de00098c0 con 0x7f1df00ad1b0 2020-12-05T17:42:08.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df505e700 1 -- 172.21.15.32:0/441432991 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de0009a20 con 0x7f1df00ad1b0 2020-12-05T17:42:08.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df82c4700 1 -- 172.21.15.32:0/441432991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df00ad1b0 msgr2=0x7f1df01389f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:08.465 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df82c4700 1 --2- 172.21.15.32:0/441432991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df00ad1b0 0x7f1df01389f0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 rx=0x7f1de0002f30 tx=0x7f1de0006e80).stop 2020-12-05T17:42:08.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df505e700 1 -- 172.21.15.32:0/441432991 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1de0009d80 con 0x7f1df00ad1b0 2020-12-05T17:42:08.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df82c4700 1 -- 172.21.15.32:0/441432991 shutdown_connections 2020-12-05T17:42:08.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df82c4700 1 --2- 172.21.15.32:0/441432991 >> v2:172.21.15.61:3300/0 conn(0x7f1df0138ee0 0x7f1df0142290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:08.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df82c4700 1 --2- 172.21.15.32:0/441432991 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1df00aead0 0x7f1df00aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:08.466 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df82c4700 1 --2- 172.21.15.32:0/441432991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df00ad1b0 0x7f1df01389f0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:08.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df82c4700 1 -- 172.21.15.32:0/441432991 >> 172.21.15.32:0/441432991 conn(0x7f1df00b5580 msgr2=0x7f1df00b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:08.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df82c4700 1 -- 172.21.15.32:0/441432991 shutdown_connections 2020-12-05T17:42:08.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.457+0000 7f1df82c4700 1 -- 172.21.15.32:0/441432991 wait complete. 2020-12-05T17:42:08.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1df82c4700 1 Processor -- start 2020-12-05T17:42:08.467 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1df82c4700 1 -- start start 2020-12-05T17:42:08.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1df82c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df00ad1b0 0x7f1df00674c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:08.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1df82c4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1df00aead0 0x7f1df0155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:08.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1df82c4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1df0138ee0 0x7f1df0139310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:08.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1df82c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df00b1f50 con 0x7f1df00ad1b0 2020-12-05T17:42:08.468 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1df82c4700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df0155100 con 0x7f1df0138ee0 2020-12-05T17:42:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1df82c4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df01553a0 con 0x7f1df00aead0 2020-12-05T17:42:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1df6060700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df00ad1b0 0x7f1df00674c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1df6060700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df00ad1b0 0x7f1df00674c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41830/0 (socket says 172.21.15.32:41830) 2020-12-05T17:42:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1df6060700 1 -- 172.21.15.32:0/1366802225 learned_addr learned my addr 172.21.15.32:0/1366802225 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:08.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1df6861700 1 --2- 172.21.15.32:0/1366802225 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1df0138ee0 0x7f1df0139310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:08.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1df585f700 1 --2- 172.21.15.32:0/1366802225 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1df00aead0 0x7f1df0155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:08.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1df6060700 1 -- 172.21.15.32:0/1366802225 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1df00aead0 msgr2=0x7f1df0155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:08.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1df6060700 1 --2- 172.21.15.32:0/1366802225 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1df00aead0 0x7f1df0155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:08.470 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1df6060700 1 -- 172.21.15.32:0/1366802225 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1df0138ee0 msgr2=0x7f1df0139310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:08.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1df6060700 1 --2- 172.21.15.32:0/1366802225 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1df0138ee0 0x7f1df0139310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:08.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1df6060700 1 -- 172.21.15.32:0/1366802225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df00b1c40 con 0x7f1df00ad1b0 2020-12-05T17:42:08.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1df6060700 1 --2- 172.21.15.32:0/1366802225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df00ad1b0 0x7f1df00674c0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f1dec003070 tx=0x7f1dec008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:08.471 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1ddf7fe700 1 -- 172.21.15.32:0/1366802225 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1dec0127e0 con 0x7f1df00ad1b0 2020-12-05T17:42:08.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1ddf7fe700 1 -- 172.21.15.32:0/1366802225 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dec012940 con 0x7f1df00ad1b0 2020-12-05T17:42:08.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1df82c4700 1 -- 172.21.15.32:0/1366802225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df00b18d0 con 0x7f1df00ad1b0 2020-12-05T17:42:08.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1ddf7fe700 1 -- 172.21.15.32:0/1366802225 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1dec012ca0 con 0x7f1df00ad1b0 2020-12-05T17:42:08.472 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.461+0000 7f1df82c4700 1 -- 172.21.15.32:0/1366802225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df0139800 con 0x7f1df00ad1b0 2020-12-05T17:42:08.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.465+0000 7f1df82c4700 1 -- 172.21.15.32:0/1366802225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df0155560 con 0x7f1df00ad1b0 2020-12-05T17:42:08.473 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.465+0000 7f1ddf7fe700 1 -- 172.21.15.32:0/1366802225 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1dec013480 con 0x7f1df00ad1b0 2020-12-05T17:42:08.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.465+0000 7f1ddf7fe700 1 --2- 172.21.15.32:0/1366802225 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1dd0055440 0x7f1dd0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:08.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.465+0000 7f1ddf7fe700 1 -- 172.21.15.32:0/1366802225 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1dec06e6c0 con 0x7f1df00ad1b0 2020-12-05T17:42:08.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.465+0000 7f1df585f700 1 --2- 172.21.15.32:0/1366802225 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1dd0055440 0x7f1dd0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:08.474 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.465+0000 7f1df585f700 1 --2- 172.21.15.32:0/1366802225 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1dd0055440 0x7f1dd0057900 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f1de001b000 tx=0x7f1de0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:08.475 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.473+0000 7f1ddf7fe700 1 -- 172.21.15.32:0/1366802225 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1dec044760 con 0x7f1df00ad1b0 2020-12-05T17:42:08.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.657+0000 7f1df82c4700 1 -- 172.21.15.32:0/1366802225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1df0057050 con 0x7f1df00ad1b0 2020-12-05T17:42:08.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.657+0000 7f1ddf7fe700 1 -- 172.21.15.32:0/1366802225 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f1dec047d80 con 0x7f1df00ad1b0 2020-12-05T17:42:08.659 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:42:08.659 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":73,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8473,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/711010965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":711010965},{"type":"v1","addr":"172.21.15.61:6833","nonce":711010965}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":70}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:41:55.663831+0000","modified":"2020-12-05T17:41:57.203619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8580":{"gid":8580,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1575496107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1575496107},{"type":"v1","addr":"172.21.15.32:6835","nonce":1575496107}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2020-12-05T17:42:08.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.657+0000 7f1df82c4700 1 -- 172.21.15.32:0/1366802225 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1dd0055440 msgr2=0x7f1dd0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:08.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.657+0000 7f1df82c4700 1 --2- 172.21.15.32:0/1366802225 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1dd0055440 0x7f1dd0057900 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f1de001b000 tx=0x7f1de0015000).stop 2020-12-05T17:42:08.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.657+0000 7f1df82c4700 1 -- 172.21.15.32:0/1366802225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df00ad1b0 msgr2=0x7f1df00674c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:08.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.657+0000 7f1df82c4700 1 --2- 172.21.15.32:0/1366802225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df00ad1b0 0x7f1df00674c0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f1dec003070 tx=0x7f1dec008040).stop 2020-12-05T17:42:08.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.657+0000 7f1df82c4700 1 -- 172.21.15.32:0/1366802225 shutdown_connections 2020-12-05T17:42:08.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.657+0000 7f1df82c4700 1 --2- 172.21.15.32:0/1366802225 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1df0138ee0 0x7f1df0139310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:08.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.657+0000 7f1df82c4700 1 --2- 172.21.15.32:0/1366802225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df00ad1b0 0x7f1df00674c0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:08.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.657+0000 7f1df82c4700 1 --2- 172.21.15.32:0/1366802225 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1dd0055440 0x7f1dd0057900 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:08.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.657+0000 7f1df82c4700 1 --2- 172.21.15.32:0/1366802225 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1df00aead0 0x7f1df0155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:08.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.657+0000 7f1df82c4700 1 -- 172.21.15.32:0/1366802225 >> 172.21.15.32:0/1366802225 conn(0x7f1df00b5580 msgr2=0x7f1df0138660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:08.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.657+0000 7f1df82c4700 1 -- 172.21.15.32:0/1366802225 shutdown_connections 2020-12-05T17:42:08.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.657+0000 7f1df82c4700 1 -- 172.21.15.32:0/1366802225 wait complete. 2020-12-05T17:42:08.664 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 73 2020-12-05T17:42:08.673 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-12-05T17:42:08.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f023980f700 1 Processor -- start 2020-12-05T17:42:08.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f023980f700 1 -- start start 2020-12-05T17:42:08.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f023980f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0234061cf0 0x7f0234142570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:08.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f023980f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f02340af0d0 0x7f02340af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:08.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f023980f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02341433d0 con 0x7f0234142f20 2020-12-05T17:42:08.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f023980f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f023413ca50 con 0x7f0234061cf0 2020-12-05T17:42:08.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f023980f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02340b2390 con 0x7f02340af0d0 2020-12-05T17:42:08.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f0233fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0234142f20 0x7f02340b1c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44488/0 (socket says 172.21.15.32:44488) 2020-12-05T17:42:08.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f023880d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0234061cf0 0x7f0234142570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:08.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f0233fff700 1 -- 172.21.15.32:0/3159486654 learned_addr learned my addr 172.21.15.32:0/3159486654 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:42:08.792 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f023900e700 1 --2- 172.21.15.32:0/3159486654 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f02340af0d0 0x7f02340af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:08.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f02337fe700 1 -- 172.21.15.32:0/3159486654 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3857685117 0 0) 0x7f02341433d0 con 0x7f0234142f20 2020-12-05T17:42:08.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f02337fe700 1 -- 172.21.15.32:0/3159486654 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0218003230 con 0x7f0234142f20 2020-12-05T17:42:08.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f023880d700 1 -- 172.21.15.32:0/3159486654 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f02340af0d0 msgr2=0x7f02340af4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:42:08.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f023880d700 1 --2- 172.21.15.32:0/3159486654 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f02340af0d0 0x7f02340af4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:08.793 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f023880d700 1 -- 172.21.15.32:0/3159486654 >> v1:172.21.15.32:6789/0 conn(0x7f0234142f20 legacy=0x7f02340b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:08.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f02337fe700 1 -- 172.21.15.32:0/3159486654 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3230936855 0 0) 0x7f0218003230 con 0x7f0234142f20 2020-12-05T17:42:08.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f023880d700 1 -- 172.21.15.32:0/3159486654 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0234062430 con 0x7f0234061cf0 2020-12-05T17:42:08.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f023900e700 1 --2- 172.21.15.32:0/3159486654 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f02340af0d0 0x7f02340af4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:42:08.794 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f023880d700 1 --2- 172.21.15.32:0/3159486654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0234061cf0 0x7f0234142570 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 rx=0x7f0220001870 tx=0x7f0228006dc0).ready entity=mon.0 client_cookie=74d735de66c82de9 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:08.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f02337fe700 1 -- 172.21.15.32:0/3159486654 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02200016d0 con 0x7f0234061cf0 2020-12-05T17:42:08.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f02337fe700 1 -- 172.21.15.32:0/3159486654 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02280095d0 con 0x7f0234061cf0 2020-12-05T17:42:08.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f02337fe700 1 -- 172.21.15.32:0/3159486654 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0228009740 con 0x7f0234061cf0 2020-12-05T17:42:08.795 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f023980f700 1 -- 172.21.15.32:0/3159486654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0234061cf0 msgr2=0x7f0234142570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:08.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f023980f700 1 --2- 172.21.15.32:0/3159486654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0234061cf0 0x7f0234142570 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 rx=0x7f0220001870 tx=0x7f0228006dc0).stop 2020-12-05T17:42:08.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f023980f700 1 -- 172.21.15.32:0/3159486654 shutdown_connections 2020-12-05T17:42:08.796 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f023980f700 1 --2- 172.21.15.32:0/3159486654 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f02340af0d0 0x7f02340af4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:08.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f023980f700 1 --2- 172.21.15.32:0/3159486654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0234061cf0 0x7f0234142570 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:08.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f023980f700 1 -- 172.21.15.32:0/3159486654 >> 172.21.15.32:0/3159486654 conn(0x7f02340b5530 msgr2=0x7f02340b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:08.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f023980f700 1 -- 172.21.15.32:0/3159486654 shutdown_connections 2020-12-05T17:42:08.797 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.785+0000 7f023980f700 1 -- 172.21.15.32:0/3159486654 wait complete. 2020-12-05T17:42:08.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f023980f700 1 Processor -- start 2020-12-05T17:42:08.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f023980f700 1 -- start start 2020-12-05T17:42:08.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f023980f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0234061cf0 0x7f0234066b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:08.798 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f023980f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02340af0d0 0x7f023413d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:08.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f023980f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0234142f20 0x7f02340634a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:08.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f023980f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02340aedf0 con 0x7f02340af0d0 2020-12-05T17:42:08.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f023980f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02341559f0 con 0x7f0234061cf0 2020-12-05T17:42:08.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f023980f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f023413cd40 con 0x7f0234142f20 2020-12-05T17:42:08.799 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f0233fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02340af0d0 0x7f023413d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:08.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f0233fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02340af0d0 0x7f023413d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41846/0 (socket says 172.21.15.32:41846) 2020-12-05T17:42:08.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f0233fff700 1 -- 172.21.15.32:0/1884073758 learned_addr learned my addr 172.21.15.32:0/1884073758 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:08.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f023880d700 1 --2- 172.21.15.32:0/1884073758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0234061cf0 0x7f0234066b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:08.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f023900e700 1 --2- 172.21.15.32:0/1884073758 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0234142f20 0x7f02340634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:08.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f0233fff700 1 -- 172.21.15.32:0/1884073758 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0234142f20 msgr2=0x7f02340634a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:08.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f0233fff700 1 --2- 172.21.15.32:0/1884073758 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0234142f20 0x7f02340634a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:08.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f0233fff700 1 -- 172.21.15.32:0/1884073758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0234061cf0 msgr2=0x7f0234066b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:08.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f0233fff700 1 --2- 172.21.15.32:0/1884073758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0234061cf0 0x7f0234066b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:08.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f0233fff700 1 -- 172.21.15.32:0/1884073758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0234141b80 con 0x7f02340af0d0 2020-12-05T17:42:08.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f0233fff700 1 --2- 172.21.15.32:0/1884073758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02340af0d0 0x7f023413d1e0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7f0220000e90 tx=0x7f02200017d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:08.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f0231ffb700 1 -- 172.21.15.32:0/1884073758 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0220007170 con 0x7f02340af0d0 2020-12-05T17:42:08.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f0231ffb700 1 -- 172.21.15.32:0/1884073758 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02200072d0 con 0x7f02340af0d0 2020-12-05T17:42:08.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f023980f700 1 -- 172.21.15.32:0/1884073758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02341555f0 con 0x7f02340af0d0 2020-12-05T17:42:08.802 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f0231ffb700 1 -- 172.21.15.32:0/1884073758 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02200048f0 con 0x7f02340af0d0 2020-12-05T17:42:08.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.789+0000 7f023980f700 1 -- 172.21.15.32:0/1884073758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0234064e80 con 0x7f02340af0d0 2020-12-05T17:42:08.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.793+0000 7f023980f700 1 -- 172.21.15.32:0/1884073758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f023413cf00 con 0x7f02340af0d0 2020-12-05T17:42:08.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.793+0000 7f0231ffb700 1 -- 172.21.15.32:0/1884073758 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0220004a50 con 0x7f02340af0d0 2020-12-05T17:42:08.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.793+0000 7f0231ffb700 1 --2- 172.21.15.32:0/1884073758 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f021c0554e0 0x7f021c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:08.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.793+0000 7f0231ffb700 1 -- 172.21.15.32:0/1884073758 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f022006da20 con 0x7f02340af0d0 2020-12-05T17:42:08.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.793+0000 7f023880d700 1 --2- 172.21.15.32:0/1884073758 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f021c0554e0 0x7f021c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:08.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.793+0000 7f023880d700 1 --2- 172.21.15.32:0/1884073758 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f021c0554e0 0x7f021c0579a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f022801a000 tx=0x7f0228015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:08.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.801+0000 7f0231ffb700 1 -- 172.21.15.32:0/1884073758 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0220043b60 con 0x7f02340af0d0 2020-12-05T17:42:08.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:08.985+0000 7f023980f700 1 -- 172.21.15.32:0/1884073758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0234155bb0 con 0x7f02340af0d0 2020-12-05T17:42:09.274 INFO:tasks.ceph.mon.a.smithi032.stderr:2020-12-05T17:42:09.273+0000 7fdef8136700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-12-05T17:42:09.288 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.285+0000 7f0231ffb700 1 -- 172.21.15.32:0/1884073758 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v74) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0220047180 con 0x7f02340af0d0 2020-12-05T17:42:09.292 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.289+0000 7f023980f700 1 -- 172.21.15.32:0/1884073758 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f021c0554e0 msgr2=0x7f021c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:09.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.289+0000 7f023980f700 1 --2- 172.21.15.32:0/1884073758 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f021c0554e0 0x7f021c0579a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f022801a000 tx=0x7f0228015000).stop 2020-12-05T17:42:09.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.289+0000 7f023980f700 1 -- 172.21.15.32:0/1884073758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02340af0d0 msgr2=0x7f023413d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:09.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.289+0000 7f023980f700 1 --2- 172.21.15.32:0/1884073758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02340af0d0 0x7f023413d1e0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7f0220000e90 tx=0x7f02200017d0).stop 2020-12-05T17:42:09.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.289+0000 7f023980f700 1 -- 172.21.15.32:0/1884073758 shutdown_connections 2020-12-05T17:42:09.293 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.289+0000 7f023980f700 1 --2- 172.21.15.32:0/1884073758 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f021c0554e0 0x7f021c0579a0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:09.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.289+0000 7f023980f700 1 --2- 172.21.15.32:0/1884073758 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0234142f20 0x7f02340634a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:09.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.289+0000 7f023980f700 1 --2- 172.21.15.32:0/1884073758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0234061cf0 0x7f0234066b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:09.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.289+0000 7f023980f700 1 --2- 172.21.15.32:0/1884073758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02340af0d0 0x7f023413d1e0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:09.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.289+0000 7f023980f700 1 -- 172.21.15.32:0/1884073758 >> 172.21.15.32:0/1884073758 conn(0x7f02340b5530 msgr2=0x7f02340afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:09.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.289+0000 7f023980f700 1 -- 172.21.15.32:0/1884073758 shutdown_connections 2020-12-05T17:42:09.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.289+0000 7f023980f700 1 -- 172.21.15.32:0/1884073758 wait complete. 2020-12-05T17:42:09.295 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-12-05T17:42:09.314 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-12-05T17:42:09.319 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2020-12-05T17:42:09.428 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.425+0000 7f4f76230700 1 Processor -- start 2020-12-05T17:42:09.429 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.425+0000 7f4f76230700 1 -- start start 2020-12-05T17:42:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f76230700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4f7005f6f0 0x7f4f701428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f76230700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f700b1980 0x7f4f70146580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f76230700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f7005fad0 con 0x7f4f70146a70 2020-12-05T17:42:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f76230700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f700b2f80 con 0x7f4f7005f6f0 2020-12-05T17:42:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f76230700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f70145e40 con 0x7f4f700b1980 2020-12-05T17:42:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f7522e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4f7005f6f0 0x7f4f701428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f7522e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4f7005f6f0 0x7f4f701428f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41864/0 (socket says 172.21.15.32:41864) 2020-12-05T17:42:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f7522e700 1 -- 172.21.15.32:0/1289119450 learned_addr learned my addr 172.21.15.32:0/1289119450 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f74a2d700 1 --2- 172.21.15.32:0/1289119450 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f700b1980 0x7f4f70146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f7522e700 1 -- 172.21.15.32:0/1289119450 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f700b1980 msgr2=0x7f4f70146580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:42:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f7522e700 1 --2- 172.21.15.32:0/1289119450 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f700b1980 0x7f4f70146580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f7522e700 1 -- 172.21.15.32:0/1289119450 >> v1:172.21.15.61:6789/0 conn(0x7f4f70146a70 legacy=0x7f4f70066070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f7522e700 1 -- 172.21.15.32:0/1289119450 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f701460e0 con 0x7f4f7005f6f0 2020-12-05T17:42:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f7522e700 1 --2- 172.21.15.32:0/1289119450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f7005f6f0 0x7f4f701428f0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 rx=0x7f4f6c002f30 tx=0x7f4f6c006e80).ready entity=mon.0 client_cookie=46c29c238a289bf1 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f5ffff700 1 -- 172.21.15.32:0/1289119450 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f6c0097f0 con 0x7f4f7005f6f0 2020-12-05T17:42:09.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f5ffff700 1 -- 172.21.15.32:0/1289119450 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6c009950 con 0x7f4f7005f6f0 2020-12-05T17:42:09.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f5ffff700 1 -- 172.21.15.32:0/1289119450 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f6c009cb0 con 0x7f4f7005f6f0 2020-12-05T17:42:09.437 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f76230700 1 -- 172.21.15.32:0/1289119450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f7005f6f0 msgr2=0x7f4f701428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:09.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f76230700 1 --2- 172.21.15.32:0/1289119450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f7005f6f0 0x7f4f701428f0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 rx=0x7f4f6c002f30 tx=0x7f4f6c006e80).stop 2020-12-05T17:42:09.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f76230700 1 -- 172.21.15.32:0/1289119450 shutdown_connections 2020-12-05T17:42:09.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f76230700 1 --2- 172.21.15.32:0/1289119450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f7005f6f0 0x7f4f701428f0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:09.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f76230700 1 --2- 172.21.15.32:0/1289119450 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f700b1980 0x7f4f70146580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:09.438 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f76230700 1 -- 172.21.15.32:0/1289119450 >> 172.21.15.32:0/1289119450 conn(0x7f4f700604c0 msgr2=0x7f4f7005c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:09.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f76230700 1 -- 172.21.15.32:0/1289119450 shutdown_connections 2020-12-05T17:42:09.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.429+0000 7f4f76230700 1 -- 172.21.15.32:0/1289119450 wait complete. 2020-12-05T17:42:09.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.433+0000 7f4f76230700 1 Processor -- start 2020-12-05T17:42:09.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.433+0000 7f4f76230700 1 -- start start 2020-12-05T17:42:09.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.433+0000 7f4f76230700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f7005f6f0 0x7f4f700b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:09.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.433+0000 7f4f76230700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4f700b1980 0x7f4f700b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:09.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.433+0000 7f4f76230700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f70146a70 0x7f4f70114d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:09.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.433+0000 7f4f76230700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f700b5a50 con 0x7f4f70146a70 2020-12-05T17:42:09.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.433+0000 7f4f76230700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f700b5cf0 con 0x7f4f700b1980 2020-12-05T17:42:09.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.433+0000 7f4f76230700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f700b4310 con 0x7f4f7005f6f0 2020-12-05T17:42:09.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.433+0000 7f4f75a2f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f70146a70 0x7f4f70114d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:09.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.433+0000 7f4f75a2f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f70146a70 0x7f4f70114d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41874/0 (socket says 172.21.15.32:41874) 2020-12-05T17:42:09.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.433+0000 7f4f75a2f700 1 -- 172.21.15.32:0/1308952934 learned_addr learned my addr 172.21.15.32:0/1308952934 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:09.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.433+0000 7f4f74a2d700 1 --2- 172.21.15.32:0/1308952934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4f700b1980 0x7f4f700b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:09.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.433+0000 7f4f7522e700 1 --2- 172.21.15.32:0/1308952934 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f7005f6f0 0x7f4f700b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:09.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.433+0000 7f4f75a2f700 1 -- 172.21.15.32:0/1308952934 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f7005f6f0 msgr2=0x7f4f700b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:09.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.433+0000 7f4f75a2f700 1 --2- 172.21.15.32:0/1308952934 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f7005f6f0 0x7f4f700b4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:09.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.433+0000 7f4f75a2f700 1 -- 172.21.15.32:0/1308952934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4f700b1980 msgr2=0x7f4f700b6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:09.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.433+0000 7f4f75a2f700 1 --2- 172.21.15.32:0/1308952934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4f700b1980 0x7f4f700b6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:09.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.433+0000 7f4f75a2f700 1 -- 172.21.15.32:0/1308952934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f700b47b0 con 0x7f4f70146a70 2020-12-05T17:42:09.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.437+0000 7f4f75a2f700 1 --2- 172.21.15.32:0/1308952934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f70146a70 0x7f4f70114d50 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7f4f7006a370 tx=0x7f4f68007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:09.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.437+0000 7f4f5e7fc700 1 -- 172.21.15.32:0/1308952934 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f68003080 con 0x7f4f70146a70 2020-12-05T17:42:09.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.437+0000 7f4f5e7fc700 1 -- 172.21.15.32:0/1308952934 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6800a7b0 con 0x7f4f70146a70 2020-12-05T17:42:09.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.437+0000 7f4f76230700 1 -- 172.21.15.32:0/1308952934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f7013e8b0 con 0x7f4f70146a70 2020-12-05T17:42:09.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.437+0000 7f4f5e7fc700 1 -- 172.21.15.32:0/1308952934 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f6800ab10 con 0x7f4f70146a70 2020-12-05T17:42:09.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.437+0000 7f4f76230700 1 -- 172.21.15.32:0/1308952934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f7013ed60 con 0x7f4f70146a70 2020-12-05T17:42:09.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.437+0000 7f4f76230700 1 -- 172.21.15.32:0/1308952934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f7013f350 con 0x7f4f70146a70 2020-12-05T17:42:09.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.437+0000 7f4f5e7fc700 1 -- 172.21.15.32:0/1308952934 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4f68019030 con 0x7f4f70146a70 2020-12-05T17:42:09.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.437+0000 7f4f5e7fc700 1 --2- 172.21.15.32:0/1308952934 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4f500553b0 0x7f4f50057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:09.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.437+0000 7f4f5e7fc700 1 -- 172.21.15.32:0/1308952934 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f4f6806d7e0 con 0x7f4f70146a70 2020-12-05T17:42:09.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.437+0000 7f4f7522e700 1 --2- 172.21.15.32:0/1308952934 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4f500553b0 0x7f4f50057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:09.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.437+0000 7f4f7522e700 1 --2- 172.21.15.32:0/1308952934 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4f500553b0 0x7f4f50057870 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f4f6c01c000 tx=0x7f4f6c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:09.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.445+0000 7f4f5e7fc700 1 -- 172.21.15.32:0/1308952934 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4f68043890 con 0x7f4f70146a70 2020-12-05T17:42:09.637 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.633+0000 7f4f76230700 1 -- 172.21.15.32:0/1308952934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f70057050 con 0x7f4f70146a70 2020-12-05T17:42:09.638 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.637+0000 7f4f5e7fc700 1 -- 172.21.15.32:0/1308952934 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+1779 (secure 0 0 0) 0x7f4f68046eb0 con 0x7f4f70146a70 2020-12-05T17:42:09.638 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T17:42:09.638 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":74,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8473,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.61:6833/711010965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":711010965},{"type":"v1","addr":"172.21.15.61:6833","nonce":711010965}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-05T17:41:55.663831+0000","modified":"2020-12-05T17:42:09.274802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":149,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2020-12-05T17:42:09.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.637+0000 7f4f76230700 1 -- 172.21.15.32:0/1308952934 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4f500553b0 msgr2=0x7f4f50057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:09.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.637+0000 7f4f76230700 1 --2- 172.21.15.32:0/1308952934 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4f500553b0 0x7f4f50057870 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f4f6c01c000 tx=0x7f4f6c015000).stop 2020-12-05T17:42:09.641 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.637+0000 7f4f76230700 1 -- 172.21.15.32:0/1308952934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f70146a70 msgr2=0x7f4f70114d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:09.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.637+0000 7f4f76230700 1 --2- 172.21.15.32:0/1308952934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f70146a70 0x7f4f70114d50 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7f4f7006a370 tx=0x7f4f68007f70).stop 2020-12-05T17:42:09.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.637+0000 7f4f76230700 1 -- 172.21.15.32:0/1308952934 shutdown_connections 2020-12-05T17:42:09.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.637+0000 7f4f76230700 1 --2- 172.21.15.32:0/1308952934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f70146a70 0x7f4f70114d50 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:09.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.637+0000 7f4f76230700 1 --2- 172.21.15.32:0/1308952934 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4f500553b0 0x7f4f50057870 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:09.642 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.637+0000 7f4f76230700 1 --2- 172.21.15.32:0/1308952934 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f7005f6f0 0x7f4f700b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:09.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.637+0000 7f4f76230700 1 --2- 172.21.15.32:0/1308952934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4f700b1980 0x7f4f700b6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:09.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.637+0000 7f4f76230700 1 -- 172.21.15.32:0/1308952934 >> 172.21.15.32:0/1308952934 conn(0x7f4f700604c0 msgr2=0x7f4f7013f900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:09.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.637+0000 7f4f76230700 1 -- 172.21.15.32:0/1308952934 shutdown_connections 2020-12-05T17:42:09.643 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.637+0000 7f4f76230700 1 -- 172.21.15.32:0/1308952934 wait complete. 2020-12-05T17:42:09.644 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 74 2020-12-05T17:42:09.659 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-12-05T17:42:09.821 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8bbe1700 1 Processor -- start 2020-12-05T17:42:09.822 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8bbe1700 1 -- start start 2020-12-05T17:42:09.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8bbe1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5d840b5b20 0x7f5d840b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:09.823 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8bbe1700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f5d840b5240 0x7f5d84145f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:09.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8bbe1700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d840b1920 con 0x7f5d840b6690 2020-12-05T17:42:09.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8bbe1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d840b1bc0 con 0x7f5d840b5b20 2020-12-05T17:42:09.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8bbe1700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d840b5680 con 0x7f5d840b5240 2020-12-05T17:42:09.824 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8997d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5d840b5b20 0x7f5d840b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:09.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8997d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5d840b5b20 0x7f5d840b5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41878/0 (socket says 172.21.15.32:41878) 2020-12-05T17:42:09.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8a17e700 1 --2- 172.21.15.32:0/1322851 >> v2:172.21.15.61:3300/0 conn(0x7f5d840b5240 0x7f5d84145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:09.825 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8997d700 1 -- 172.21.15.32:0/1322851 learned_addr learned my addr 172.21.15.32:0/1322851 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:09.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8997d700 1 -- 172.21.15.32:0/1322851 >> v2:172.21.15.61:3300/0 conn(0x7f5d840b5240 msgr2=0x7f5d84145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:42:09.827 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8997d700 1 --2- 172.21.15.32:0/1322851 >> v2:172.21.15.61:3300/0 conn(0x7f5d840b5240 0x7f5d84145f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:09.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8997d700 1 -- 172.21.15.32:0/1322851 >> v1:172.21.15.61:6789/0 conn(0x7f5d840b6690 legacy=0x7f5d840b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:09.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8997d700 1 -- 172.21.15.32:0/1322851 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d84146400 con 0x7f5d840b5b20 2020-12-05T17:42:09.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8997d700 1 --2- 172.21.15.32:0/1322851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d840b5b20 0x7f5d840b5f00 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 rx=0x7f5d80002f30 tx=0x7f5d80006e80).ready entity=mon.0 client_cookie=aecacdce4df2e9ad server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:09.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8897b700 1 -- 172.21.15.32:0/1322851 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d800097f0 con 0x7f5d840b5b20 2020-12-05T17:42:09.828 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8897b700 1 -- 172.21.15.32:0/1322851 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d80009950 con 0x7f5d840b5b20 2020-12-05T17:42:09.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8897b700 1 -- 172.21.15.32:0/1322851 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d80009cb0 con 0x7f5d840b5b20 2020-12-05T17:42:09.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8bbe1700 1 -- 172.21.15.32:0/1322851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d840b5b20 msgr2=0x7f5d840b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:09.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8bbe1700 1 --2- 172.21.15.32:0/1322851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d840b5b20 0x7f5d840b5f00 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 rx=0x7f5d80002f30 tx=0x7f5d80006e80).stop 2020-12-05T17:42:09.829 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8bbe1700 1 -- 172.21.15.32:0/1322851 shutdown_connections 2020-12-05T17:42:09.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8bbe1700 1 --2- 172.21.15.32:0/1322851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d840b5b20 0x7f5d840b5f00 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:09.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8bbe1700 1 --2- 172.21.15.32:0/1322851 >> v2:172.21.15.61:3300/0 conn(0x7f5d840b5240 0x7f5d84145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:09.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.821+0000 7f5d8bbe1700 1 -- 172.21.15.32:0/1322851 >> 172.21.15.32:0/1322851 conn(0x7f5d8405f3b0 msgr2=0x7f5d8405f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:09.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.825+0000 7f5d8bbe1700 1 -- 172.21.15.32:0/1322851 shutdown_connections 2020-12-05T17:42:09.830 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.825+0000 7f5d8bbe1700 1 -- 172.21.15.32:0/1322851 wait complete. 2020-12-05T17:42:09.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.825+0000 7f5d8bbe1700 1 Processor -- start 2020-12-05T17:42:09.831 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.825+0000 7f5d8bbe1700 1 -- start start 2020-12-05T17:42:09.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.825+0000 7f5d8bbe1700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d840b5240 0x7f5d84065b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:09.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.825+0000 7f5d8bbe1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d840b5b20 0x7f5d84152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:09.833 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.825+0000 7f5d8bbe1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5d840b6690 0x7f5d840a6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:09.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.825+0000 7f5d8bbe1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d84151c40 con 0x7f5d840b5b20 2020-12-05T17:42:09.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.825+0000 7f5d8bbe1700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d84152e70 con 0x7f5d840b5240 2020-12-05T17:42:09.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.825+0000 7f5d8bbe1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d840a4f10 con 0x7f5d840b6690 2020-12-05T17:42:09.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.825+0000 7f5d8917c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d840b5b20 0x7f5d84152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:09.834 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.825+0000 7f5d8a17e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5d840b6690 0x7f5d840a6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:09.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.825+0000 7f5d8997d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d840b5240 0x7f5d84065b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:09.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.825+0000 7f5d8917c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d840b5b20 0x7f5d84152450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41888/0 (socket says 172.21.15.32:41888) 2020-12-05T17:42:09.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.825+0000 7f5d8917c700 1 -- 172.21.15.32:0/1599575483 learned_addr learned my addr 172.21.15.32:0/1599575483 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:09.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.825+0000 7f5d8917c700 1 -- 172.21.15.32:0/1599575483 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5d840b6690 msgr2=0x7f5d840a6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:09.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.825+0000 7f5d8917c700 1 --2- 172.21.15.32:0/1599575483 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5d840b6690 0x7f5d840a6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:09.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.825+0000 7f5d8917c700 1 -- 172.21.15.32:0/1599575483 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d840b5240 msgr2=0x7f5d84065b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:09.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.825+0000 7f5d8917c700 1 --2- 172.21.15.32:0/1599575483 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d840b5240 0x7f5d84065b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:09.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.825+0000 7f5d8917c700 1 -- 172.21.15.32:0/1599575483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d840a51b0 con 0x7f5d840b5b20 2020-12-05T17:42:09.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.829+0000 7f5d8917c700 1 --2- 172.21.15.32:0/1599575483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d840b5b20 0x7f5d84152450 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7f5d84061150 tx=0x7f5d74007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:09.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.829+0000 7f5d72ffd700 1 -- 172.21.15.32:0/1599575483 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d74013410 con 0x7f5d840b5b20 2020-12-05T17:42:09.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.829+0000 7f5d8bbe1700 1 -- 172.21.15.32:0/1599575483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d840a5450 con 0x7f5d840b5b20 2020-12-05T17:42:09.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.829+0000 7f5d8bbe1700 1 -- 172.21.15.32:0/1599575483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d840664f0 con 0x7f5d840b5b20 2020-12-05T17:42:09.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.829+0000 7f5d72ffd700 1 -- 172.21.15.32:0/1599575483 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d74013570 con 0x7f5d840b5b20 2020-12-05T17:42:09.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.829+0000 7f5d72ffd700 1 -- 172.21.15.32:0/1599575483 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d74013720 con 0x7f5d840b5b20 2020-12-05T17:42:09.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.829+0000 7f5d8bbe1700 1 -- 172.21.15.32:0/1599575483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d600033e0 con 0x7f5d840b5b20 2020-12-05T17:42:09.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.829+0000 7f5d72ffd700 1 -- 172.21.15.32:0/1599575483 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5d74019020 con 0x7f5d840b5b20 2020-12-05T17:42:09.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.829+0000 7f5d72ffd700 1 --2- 172.21.15.32:0/1599575483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5d640553b0 0x7f5d64057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:09.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.829+0000 7f5d72ffd700 1 -- 172.21.15.32:0/1599575483 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f5d7406dea0 con 0x7f5d840b5b20 2020-12-05T17:42:09.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.829+0000 7f5d8997d700 1 --2- 172.21.15.32:0/1599575483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5d640553b0 0x7f5d64057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:09.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.829+0000 7f5d8997d700 1 --2- 172.21.15.32:0/1599575483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5d640553b0 0x7f5d64057870 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f5d8001c000 tx=0x7f5d80015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:09.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:09.841+0000 7f5d72ffd700 1 -- 172.21.15.32:0/1599575483 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5d74043ff0 con 0x7f5d840b5b20 2020-12-05T17:42:10.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.025+0000 7f5d8bbe1700 1 -- 172.21.15.32:0/1599575483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5d600022a0 con 0x7f5d840b5b20 2020-12-05T17:42:10.291 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.289+0000 7f5d72ffd700 1 -- 172.21.15.32:0/1599575483 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v75) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5d74047610 con 0x7f5d840b5b20 2020-12-05T17:42:10.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.293+0000 7f5d8bbe1700 1 -- 172.21.15.32:0/1599575483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5d640553b0 msgr2=0x7f5d64057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:10.295 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.293+0000 7f5d8bbe1700 1 --2- 172.21.15.32:0/1599575483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5d640553b0 0x7f5d64057870 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f5d8001c000 tx=0x7f5d80015000).stop 2020-12-05T17:42:10.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.293+0000 7f5d8bbe1700 1 -- 172.21.15.32:0/1599575483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d840b5b20 msgr2=0x7f5d84152450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:10.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.293+0000 7f5d8bbe1700 1 --2- 172.21.15.32:0/1599575483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d840b5b20 0x7f5d84152450 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7f5d84061150 tx=0x7f5d74007c20).stop 2020-12-05T17:42:10.296 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.293+0000 7f5d8bbe1700 1 -- 172.21.15.32:0/1599575483 shutdown_connections 2020-12-05T17:42:10.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.293+0000 7f5d8bbe1700 1 --2- 172.21.15.32:0/1599575483 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f5d640553b0 0x7f5d64057870 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:10.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.293+0000 7f5d8bbe1700 1 --2- 172.21.15.32:0/1599575483 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5d840b6690 0x7f5d840a6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:10.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.293+0000 7f5d8bbe1700 1 --2- 172.21.15.32:0/1599575483 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d840b5240 0x7f5d84065b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:10.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.293+0000 7f5d8bbe1700 1 --2- 172.21.15.32:0/1599575483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d840b5b20 0x7f5d84152450 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:10.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.293+0000 7f5d8bbe1700 1 -- 172.21.15.32:0/1599575483 >> 172.21.15.32:0/1599575483 conn(0x7f5d8405f3b0 msgr2=0x7f5d8405f790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:10.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.293+0000 7f5d8bbe1700 1 -- 172.21.15.32:0/1599575483 shutdown_connections 2020-12-05T17:42:10.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.293+0000 7f5d8bbe1700 1 -- 172.21.15.32:0/1599575483 wait complete. 2020-12-05T17:42:10.312 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-12-05T17:42:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.437+0000 7f9b1cb9a700 1 Processor -- start 2020-12-05T17:42:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.437+0000 7f9b1cb9a700 1 -- start start 2020-12-05T17:42:10.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.437+0000 7f9b1cb9a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b180b5f00 0x7f9b180b5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:10.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.437+0000 7f9b1cb9a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b180b1920 con 0x7f9b180b5b20 2020-12-05T17:42:10.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.437+0000 7f9b1cb9a700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b180b1bc0 con 0x7f9b180b6690 2020-12-05T17:42:10.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.437+0000 7f9b1cb9a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b180b5730 con 0x7f9b180b5f00 2020-12-05T17:42:10.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.437+0000 7f9b1659c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9b180b5b20 0x7f9b180b2060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44542/0 (socket says 172.21.15.32:44542) 2020-12-05T17:42:10.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.437+0000 7f9b1659c700 1 -- 172.21.15.32:0/519126726 learned_addr learned my addr 172.21.15.32:0/519126726 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T17:42:10.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.437+0000 7f9b16d9d700 1 --2- 172.21.15.32:0/519126726 >> v2:172.21.15.32:3300/0 conn(0x7f9b180b5f00 0x7f9b180b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:10.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.441+0000 7f9b15d9b700 1 -- 172.21.15.32:0/519126726 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2136679530 0 0) 0x7f9b180b1920 con 0x7f9b180b5b20 2020-12-05T17:42:10.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.441+0000 7f9b15d9b700 1 -- 172.21.15.32:0/519126726 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9af8003230 con 0x7f9b180b5b20 2020-12-05T17:42:10.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.441+0000 7f9b15d9b700 1 -- 172.21.15.32:0/519126726 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704323263 0 0) 0x7f9b180b1bc0 con 0x7f9b180b6690 2020-12-05T17:42:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.441+0000 7f9b15d9b700 1 -- 172.21.15.32:0/519126726 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b180b1920 con 0x7f9b180b6690 2020-12-05T17:42:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.441+0000 7f9b16d9d700 1 -- 172.21.15.32:0/519126726 >> v1:172.21.15.61:6789/0 conn(0x7f9b180b6690 legacy=0x7f9b18145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.441+0000 7f9b16d9d700 1 -- 172.21.15.32:0/519126726 >> v1:172.21.15.32:6789/0 conn(0x7f9b180b5b20 legacy=0x7f9b180b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.441+0000 7f9b16d9d700 1 -- 172.21.15.32:0/519126726 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b18146620 con 0x7f9b180b5f00 2020-12-05T17:42:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.441+0000 7f9b16d9d700 1 --2- 172.21.15.32:0/519126726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b180b5f00 0x7f9b180b5240 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 rx=0x7f9b100032b0 tx=0x7f9b100091a0).ready entity=mon.0 client_cookie=852bd68ded790331 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:10.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.441+0000 7f9b15d9b700 1 -- 172.21.15.32:0/519126726 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b1000b940 con 0x7f9b180b5f00 2020-12-05T17:42:10.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.441+0000 7f9b15d9b700 1 -- 172.21.15.32:0/519126726 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1000baa0 con 0x7f9b180b5f00 2020-12-05T17:42:10.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.441+0000 7f9b15d9b700 1 -- 172.21.15.32:0/519126726 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b1000bc10 con 0x7f9b180b5f00 2020-12-05T17:42:10.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.441+0000 7f9b1cb9a700 1 -- 172.21.15.32:0/519126726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b180b5f00 msgr2=0x7f9b180b5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:10.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.441+0000 7f9b1cb9a700 1 --2- 172.21.15.32:0/519126726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b180b5f00 0x7f9b180b5240 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 rx=0x7f9b100032b0 tx=0x7f9b100091a0).stop 2020-12-05T17:42:10.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.441+0000 7f9b1cb9a700 1 -- 172.21.15.32:0/519126726 shutdown_connections 2020-12-05T17:42:10.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.441+0000 7f9b1cb9a700 1 --2- 172.21.15.32:0/519126726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b180b5f00 0x7f9b180b5240 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:10.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.441+0000 7f9b1cb9a700 1 -- 172.21.15.32:0/519126726 >> 172.21.15.32:0/519126726 conn(0x7f9b1805f3b0 msgr2=0x7f9b1805f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:10.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.441+0000 7f9b1cb9a700 1 -- 172.21.15.32:0/519126726 shutdown_connections 2020-12-05T17:42:10.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.441+0000 7f9b1cb9a700 1 -- 172.21.15.32:0/519126726 wait complete. 2020-12-05T17:42:10.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.441+0000 7f9b1cb9a700 1 Processor -- start 2020-12-05T17:42:10.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.441+0000 7f9b1cb9a700 1 -- start start 2020-12-05T17:42:10.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b1cb9a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b180b5b20 0x7f9b18152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:10.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b1cb9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b180b5f00 0x7f9b18066880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:10.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b1cb9a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b180b6690 0x7f9b18065210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:10.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b1cb9a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b180a5b80 con 0x7f9b180b6690 2020-12-05T17:42:10.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b1cb9a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b180a5e20 con 0x7f9b180b5f00 2020-12-05T17:42:10.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b1cb9a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b180a60c0 con 0x7f9b180b5b20 2020-12-05T17:42:10.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b16d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b180b6690 0x7f9b18065210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b16d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b180b6690 0x7f9b18065210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41902/0 (socket says 172.21.15.32:41902) 2020-12-05T17:42:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b16d9d700 1 -- 172.21.15.32:0/1128826501 learned_addr learned my addr 172.21.15.32:0/1128826501 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b1659c700 1 --2- 172.21.15.32:0/1128826501 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b180b5b20 0x7f9b18152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b0ffff700 1 --2- 172.21.15.32:0/1128826501 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b180b5f00 0x7f9b18066880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:10.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b16d9d700 1 -- 172.21.15.32:0/1128826501 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b180b5b20 msgr2=0x7f9b18152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:10.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b16d9d700 1 --2- 172.21.15.32:0/1128826501 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b180b5b20 0x7f9b18152480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:10.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b16d9d700 1 -- 172.21.15.32:0/1128826501 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b180b5f00 msgr2=0x7f9b18066880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:10.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b16d9d700 1 --2- 172.21.15.32:0/1128826501 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b180b5f00 0x7f9b18066880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:10.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b16d9d700 1 -- 172.21.15.32:0/1128826501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b18066d70 con 0x7f9b180b6690 2020-12-05T17:42:10.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b1659c700 1 --2- 172.21.15.32:0/1128826501 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b180b5b20 0x7f9b18152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T17:42:10.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b16d9d700 1 --2- 172.21.15.32:0/1128826501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b180b6690 0x7f9b18065210 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 rx=0x7f9b100032b0 tx=0x7f9b10016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:10.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b0f7fe700 1 -- 172.21.15.32:0/1128826501 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b100105b0 con 0x7f9b180b6690 2020-12-05T17:42:10.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b0f7fe700 1 -- 172.21.15.32:0/1128826501 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b10010710 con 0x7f9b180b6690 2020-12-05T17:42:10.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b1cb9a700 1 -- 172.21.15.32:0/1128826501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b180a5320 con 0x7f9b180b6690 2020-12-05T17:42:10.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b0f7fe700 1 -- 172.21.15.32:0/1128826501 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b10010a70 con 0x7f9b180b6690 2020-12-05T17:42:10.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b1cb9a700 1 -- 172.21.15.32:0/1128826501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b18065700 con 0x7f9b180b6690 2020-12-05T17:42:10.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.445+0000 7f9b1cb9a700 1 -- 172.21.15.32:0/1128826501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b180a29b0 con 0x7f9b180b6690 2020-12-05T17:42:10.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.449+0000 7f9b0f7fe700 1 -- 172.21.15.32:0/1128826501 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9b1001e070 con 0x7f9b180b6690 2020-12-05T17:42:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.449+0000 7f9b0f7fe700 1 --2- 172.21.15.32:0/1128826501 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9afc0554e0 0x7f9afc0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.449+0000 7f9b0f7fe700 1 -- 172.21.15.32:0/1128826501 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f9b1006ee80 con 0x7f9b180b6690 2020-12-05T17:42:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.449+0000 7f9b1659c700 1 --2- 172.21.15.32:0/1128826501 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9afc0554e0 0x7f9afc0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:10.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.449+0000 7f9b1659c700 1 --2- 172.21.15.32:0/1128826501 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9afc0554e0 0x7f9afc0579a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f9b0800b000 tx=0x7f9b08009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:10.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.457+0000 7f9b0f7fe700 1 -- 172.21.15.32:0/1128826501 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9b180a29b0 con 0x7f9b180b6690 2020-12-05T17:42:10.598 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:10.597+0000 7f9b1cb9a700 1 -- 172.21.15.32:0/1128826501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9b180a29b0 con 0x7f9b180b6690 2020-12-05T17:42:11.294 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.293+0000 7f9b0f7fe700 1 -- 172.21.15.32:0/1128826501 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v150) v1 ==== 185+0+0 (secure 0 0 0) 0x7f9b180a29b0 con 0x7f9b180b6690 2020-12-05T17:42:11.297 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.293+0000 7f9b1cb9a700 1 -- 172.21.15.32:0/1128826501 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9afc0554e0 msgr2=0x7f9afc0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:11.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.293+0000 7f9b1cb9a700 1 --2- 172.21.15.32:0/1128826501 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9afc0554e0 0x7f9afc0579a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f9b0800b000 tx=0x7f9b08009040).stop 2020-12-05T17:42:11.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.293+0000 7f9b1cb9a700 1 -- 172.21.15.32:0/1128826501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b180b6690 msgr2=0x7f9b18065210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:11.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.293+0000 7f9b1cb9a700 1 --2- 172.21.15.32:0/1128826501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b180b6690 0x7f9b18065210 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 rx=0x7f9b100032b0 tx=0x7f9b10016000).stop 2020-12-05T17:42:11.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.293+0000 7f9b1cb9a700 1 -- 172.21.15.32:0/1128826501 shutdown_connections 2020-12-05T17:42:11.299 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.293+0000 7f9b1cb9a700 1 --2- 172.21.15.32:0/1128826501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b180b6690 0x7f9b18065210 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:11.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.293+0000 7f9b1cb9a700 1 --2- 172.21.15.32:0/1128826501 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9afc0554e0 0x7f9afc0579a0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:11.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.293+0000 7f9b1cb9a700 1 --2- 172.21.15.32:0/1128826501 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9b180b5b20 0x7f9b18152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:11.300 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.293+0000 7f9b1cb9a700 1 --2- 172.21.15.32:0/1128826501 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9b180b5f00 0x7f9b18066880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:11.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.293+0000 7f9b1cb9a700 1 -- 172.21.15.32:0/1128826501 >> 172.21.15.32:0/1128826501 conn(0x7f9b1805f3b0 msgr2=0x7f9b180b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:11.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.293+0000 7f9b1cb9a700 1 -- 172.21.15.32:0/1128826501 shutdown_connections 2020-12-05T17:42:11.301 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.293+0000 7f9b1cb9a700 1 -- 172.21.15.32:0/1128826501 wait complete. 2020-12-05T17:42:11.302 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2020-12-05T17:42:11.312 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-12-05T17:42:11.441 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.441+0000 7f74ddf23700 1 Processor -- start 2020-12-05T17:42:11.442 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.441+0000 7f74ddf23700 1 -- start start 2020-12-05T17:42:11.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.441+0000 7f74ddf23700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74d8063040 0x7f74d813fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:11.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.441+0000 7f74ddf23700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f74d80af090 0x7f74d80af500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:11.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.441+0000 7f74ddf23700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74d8067110 con 0x7f74d805f550 2020-12-05T17:42:11.444 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.441+0000 7f74ddf23700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74d80ad290 con 0x7f74d8063040 2020-12-05T17:42:11.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.441+0000 7f74ddf23700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74d813f760 con 0x7f74d80af090 2020-12-05T17:42:11.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.441+0000 7f74cffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74d8063040 0x7f74d813fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:11.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.441+0000 7f74cffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74d8063040 0x7f74d813fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41910/0 (socket says 172.21.15.32:41910) 2020-12-05T17:42:11.445 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.441+0000 7f74cffff700 1 -- 172.21.15.32:0/1098090307 learned_addr learned my addr 172.21.15.32:0/1098090307 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:11.446 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.441+0000 7f74dd722700 1 --2- 172.21.15.32:0/1098090307 >> v2:172.21.15.61:3300/0 conn(0x7f74d80af090 0x7f74d80af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:11.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.441+0000 7f74cffff700 1 -- 172.21.15.32:0/1098090307 >> v2:172.21.15.61:3300/0 conn(0x7f74d80af090 msgr2=0x7f74d80af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T17:42:11.447 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.441+0000 7f74cffff700 1 --2- 172.21.15.32:0/1098090307 >> v2:172.21.15.61:3300/0 conn(0x7f74d80af090 0x7f74d80af500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:11.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.441+0000 7f74cffff700 1 -- 172.21.15.32:0/1098090307 >> v1:172.21.15.61:6789/0 conn(0x7f74d805f550 legacy=0x7f74d80acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:11.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.441+0000 7f74cf7fe700 1 -- 172.21.15.32:0/1098090307 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1408687393 0 0) 0x7f74d8067110 con 0x7f74d805f550 2020-12-05T17:42:11.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.441+0000 7f74cffff700 1 -- 172.21.15.32:0/1098090307 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d813fa00 con 0x7f74d8063040 2020-12-05T17:42:11.448 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.441+0000 7f74cffff700 1 --2- 172.21.15.32:0/1098090307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d8063040 0x7f74d813fea0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 rx=0x7f74c8002f30 tx=0x7f74c8006e80).ready entity=mon.0 client_cookie=c62b88ad0f8964c5 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:11.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.441+0000 7f74cf7fe700 1 -- 172.21.15.32:0/1098090307 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74c80097f0 con 0x7f74d8063040 2020-12-05T17:42:11.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.441+0000 7f74cf7fe700 1 -- 172.21.15.32:0/1098090307 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c8009950 con 0x7f74d8063040 2020-12-05T17:42:11.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.441+0000 7f74cf7fe700 1 -- 172.21.15.32:0/1098090307 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74c8009cb0 con 0x7f74d8063040 2020-12-05T17:42:11.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.441+0000 7f74ddf23700 1 -- 172.21.15.32:0/1098090307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d8063040 msgr2=0x7f74d813fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:11.449 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.445+0000 7f74ddf23700 1 --2- 172.21.15.32:0/1098090307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d8063040 0x7f74d813fea0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 rx=0x7f74c8002f30 tx=0x7f74c8006e80).stop 2020-12-05T17:42:11.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.445+0000 7f74ddf23700 1 -- 172.21.15.32:0/1098090307 shutdown_connections 2020-12-05T17:42:11.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.445+0000 7f74ddf23700 1 --2- 172.21.15.32:0/1098090307 >> v2:172.21.15.61:3300/0 conn(0x7f74d80af090 0x7f74d80af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:11.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.445+0000 7f74ddf23700 1 --2- 172.21.15.32:0/1098090307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d8063040 0x7f74d813fea0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:11.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.445+0000 7f74ddf23700 1 -- 172.21.15.32:0/1098090307 >> 172.21.15.32:0/1098090307 conn(0x7f74d80620b0 msgr2=0x7f74d8062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:11.450 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.445+0000 7f74ddf23700 1 -- 172.21.15.32:0/1098090307 shutdown_connections 2020-12-05T17:42:11.451 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.445+0000 7f74ddf23700 1 -- 172.21.15.32:0/1098090307 wait complete. 2020-12-05T17:42:11.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.445+0000 7f74ddf23700 1 Processor -- start 2020-12-05T17:42:11.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.445+0000 7f74ddf23700 1 -- start start 2020-12-05T17:42:11.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.449+0000 7f74ddf23700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74d805f550 0x7f74d813c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:11.453 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.449+0000 7f74ddf23700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74d8063040 0x7f74d80b2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:11.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.449+0000 7f74ddf23700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d80af090 0x7f74d8138eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:11.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.449+0000 7f74ddf23700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d813b9a0 con 0x7f74d80af090 2020-12-05T17:42:11.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.449+0000 7f74ddf23700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d813bc40 con 0x7f74d8063040 2020-12-05T17:42:11.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.449+0000 7f74ddf23700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74d8139f30 con 0x7f74d805f550 2020-12-05T17:42:11.454 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.449+0000 7f74dd722700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d80af090 0x7f74d8138eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:11.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.449+0000 7f74dd722700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d80af090 0x7f74d8138eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41918/0 (socket says 172.21.15.32:41918) 2020-12-05T17:42:11.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.449+0000 7f74dd722700 1 -- 172.21.15.32:0/997232634 learned_addr learned my addr 172.21.15.32:0/997232634 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T17:42:11.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.449+0000 7f74dcf21700 1 --2- 172.21.15.32:0/997232634 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74d805f550 0x7f74d813c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:11.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.449+0000 7f74cffff700 1 --2- 172.21.15.32:0/997232634 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74d8063040 0x7f74d80b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:11.455 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.449+0000 7f74dd722700 1 -- 172.21.15.32:0/997232634 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74d805f550 msgr2=0x7f74d813c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:11.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.449+0000 7f74dd722700 1 --2- 172.21.15.32:0/997232634 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74d805f550 0x7f74d813c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:11.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.449+0000 7f74dd722700 1 -- 172.21.15.32:0/997232634 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74d8063040 msgr2=0x7f74d80b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:11.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.449+0000 7f74dd722700 1 --2- 172.21.15.32:0/997232634 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74d8063040 0x7f74d80b2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:11.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.449+0000 7f74dd722700 1 -- 172.21.15.32:0/997232634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d813a210 con 0x7f74d80af090 2020-12-05T17:42:11.456 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.449+0000 7f74dd722700 1 --2- 172.21.15.32:0/997232634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d80af090 0x7f74d8138eb0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7f74d0006dd0 tx=0x7f74d00091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:11.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.449+0000 7f74cdffb700 1 -- 172.21.15.32:0/997232634 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74d000c8f0 con 0x7f74d80af090 2020-12-05T17:42:11.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.449+0000 7f74cdffb700 1 -- 172.21.15.32:0/997232634 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d000ca50 con 0x7f74d80af090 2020-12-05T17:42:11.457 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.449+0000 7f74ddf23700 1 -- 172.21.15.32:0/997232634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d813b1c0 con 0x7f74d80af090 2020-12-05T17:42:11.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.449+0000 7f74cdffb700 1 -- 172.21.15.32:0/997232634 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74d000cdb0 con 0x7f74d80af090 2020-12-05T17:42:11.458 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.449+0000 7f74ddf23700 1 -- 172.21.15.32:0/997232634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d81393a0 con 0x7f74d80af090 2020-12-05T17:42:11.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.453+0000 7f74ddf23700 1 -- 172.21.15.32:0/997232634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74d813b800 con 0x7f74d80af090 2020-12-05T17:42:11.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.453+0000 7f74cdffb700 1 -- 172.21.15.32:0/997232634 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f74d0022070 con 0x7f74d80af090 2020-12-05T17:42:11.459 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.453+0000 7f74cdffb700 1 --2- 172.21.15.32:0/997232634 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f74b8055440 0x7f74b8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T17:42:11.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.453+0000 7f74cdffb700 1 -- 172.21.15.32:0/997232634 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f74d006e410 con 0x7f74d80af090 2020-12-05T17:42:11.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.453+0000 7f74dcf21700 1 --2- 172.21.15.32:0/997232634 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f74b8055440 0x7f74b8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T17:42:11.460 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.453+0000 7f74dcf21700 1 --2- 172.21.15.32:0/997232634 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f74b8055440 0x7f74b8057900 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f74d400d000 tx=0x7f74d4007a60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T17:42:11.462 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.461+0000 7f74cdffb700 1 -- 172.21.15.32:0/997232634 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f74d813b800 con 0x7f74d80af090 2020-12-05T17:42:11.601 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:11.597+0000 7f74ddf23700 1 -- 172.21.15.32:0/997232634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f74d813b800 con 0x7f74d80af090 2020-12-05T17:42:12.298 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:12.297+0000 7f74cdffb700 1 -- 172.21.15.32:0/997232634 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v151) v1 ==== 173+0+0 (secure 0 0 0) 0x7f74d813b800 con 0x7f74d80af090 2020-12-05T17:42:12.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:12.297+0000 7f74ddf23700 1 -- 172.21.15.32:0/997232634 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f74b8055440 msgr2=0x7f74b8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:12.302 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:12.297+0000 7f74ddf23700 1 --2- 172.21.15.32:0/997232634 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f74b8055440 0x7f74b8057900 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f74d400d000 tx=0x7f74d4007a60).stop 2020-12-05T17:42:12.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:12.297+0000 7f74ddf23700 1 -- 172.21.15.32:0/997232634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d80af090 msgr2=0x7f74d8138eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T17:42:12.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:12.297+0000 7f74ddf23700 1 --2- 172.21.15.32:0/997232634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d80af090 0x7f74d8138eb0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7f74d0006dd0 tx=0x7f74d00091a0).stop 2020-12-05T17:42:12.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:12.301+0000 7f74ddf23700 1 -- 172.21.15.32:0/997232634 shutdown_connections 2020-12-05T17:42:12.303 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:12.301+0000 7f74ddf23700 1 --2- 172.21.15.32:0/997232634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d80af090 0x7f74d8138eb0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:12.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:12.301+0000 7f74ddf23700 1 --2- 172.21.15.32:0/997232634 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f74b8055440 0x7f74b8057900 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:12.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:12.301+0000 7f74ddf23700 1 --2- 172.21.15.32:0/997232634 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74d805f550 0x7f74d813c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:12.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:12.301+0000 7f74ddf23700 1 --2- 172.21.15.32:0/997232634 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74d8063040 0x7f74d80b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T17:42:12.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:12.301+0000 7f74ddf23700 1 -- 172.21.15.32:0/997232634 >> 172.21.15.32:0/997232634 conn(0x7f74d80620b0 msgr2=0x7f74d8062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T17:42:12.304 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:12.301+0000 7f74ddf23700 1 -- 172.21.15.32:0/997232634 shutdown_connections 2020-12-05T17:42:12.305 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T17:42:12.301+0000 7f74ddf23700 1 -- 172.21.15.32:0/997232634 wait complete. 2020-12-05T17:42:12.305 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2020-12-05T17:42:12.319 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-12-05T17:42:12.319 INFO:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:42:12.337 INFO:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:42:12.354 INFO:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:42:12.372 INFO:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:42:12.390 INFO:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-12-05T17:42:12.407 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-12-05T17:42:12.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:42:12.408 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:42:12.408 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:42:16.162 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:42:16.165 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:42:16.167 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:42:16.169 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:42:16.172 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:42:46.210 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:42:46.213 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:42:46.216 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:42:46.218 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:42:46.220 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:43:16.275 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:43:16.277 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:43:16.280 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:43:16.282 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:43:16.284 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:43:46.325 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:43:46.328 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:43:46.330 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:43:46.333 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:43:46.335 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:44:16.383 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:44:16.386 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:44:16.389 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:44:16.392 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:44:16.394 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:44:46.568 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:44:46.571 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:44:46.574 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:44:46.576 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:44:46.579 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:45:16.638 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:45:16.641 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:45:16.643 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:45:16.646 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:45:16.649 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:45:46.731 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:45:46.734 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:45:46.736 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:45:46.739 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:45:46.741 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:46:16.832 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:46:16.834 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:46:16.837 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:46:16.839 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:46:16.842 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:46:46.894 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:46:46.897 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:46:46.899 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:46:46.902 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:46:46.904 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:47:16.941 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:47:16.945 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:47:16.947 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:47:16.950 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:47:16.952 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:47:46.989 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:47:46.994 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:47:46.998 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:47:47.002 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:47:47.005 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:48:17.060 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:48:17.063 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:48:17.066 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:48:17.069 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:48:17.072 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:48:47.127 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:48:47.129 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:48:47.132 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:48:47.134 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:48:47.136 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:49:17.177 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:49:17.180 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:49:17.183 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:49:17.186 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:49:17.189 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:49:47.241 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:49:47.244 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:49:47.246 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:49:47.248 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:49:47.251 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:50:17.296 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:50:17.299 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:50:17.301 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:50:17.303 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:50:17.306 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:50:47.366 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:50:47.369 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:50:47.371 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:50:47.374 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:50:47.376 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:51:17.431 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:51:17.435 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:51:17.438 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:51:17.441 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:51:17.444 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:51:47.485 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:51:47.488 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:51:47.490 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:51:47.492 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:51:47.495 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:52:17.541 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:52:17.544 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:52:17.546 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:52:17.549 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:52:17.551 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:52:47.605 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:52:47.607 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:52:47.610 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:52:47.612 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:52:47.617 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:53:17.660 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:53:17.663 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:53:17.665 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:53:17.668 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:53:17.670 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:53:47.715 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:53:47.718 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:53:47.720 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:53:47.722 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:53:47.725 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:54:17.767 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:54:17.769 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:54:17.773 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:54:17.775 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:54:17.777 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:54:47.831 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:54:47.834 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:54:47.836 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:54:47.839 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:54:47.841 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:55:17.879 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:55:17.882 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:55:17.884 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:55:17.886 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:55:17.889 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:55:47.945 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:55:47.947 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:55:47.950 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:55:47.953 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:55:47.955 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:56:18.016 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:56:18.019 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:56:18.022 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:56:18.024 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:56:18.026 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:56:48.081 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:56:48.084 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:56:48.086 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:56:48.088 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:56:48.090 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:57:12.419 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 316, in copy_file_to copy_to_log(src, logger, capture=stream) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 273, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2020-12-05T17:57:12.426 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 316, in copy_file_to copy_to_log(src, logger, capture=stream) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 273, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2020-12-05T17:57:12.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T17:57:12.428 INFO:tasks.cephfs_test_runner:test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) ... ERROR 2020-12-05T17:57:12.430 INFO:tasks.cephfs_test_runner:ERROR 2020-12-05T17:57:12.430 INFO:tasks.cephfs_test_runner: 2020-12-05T17:57:12.431 INFO:tasks.cephfs_test_runner:====================================================================== 2020-12-05T17:57:12.431 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2020-12-05T17:57:12.431 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2020-12-05T17:57:12.431 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2020-12-05T17:57:12.432 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/test_misc.py", line 24, in test_statfs_on_deleted_fs 2020-12-05T17:57:12.432 INFO:tasks.cephfs_test_runner: p = self.mount_a.run_shell_payload("stat -f .", wait=False, timeout=60, check_status=False) 2020-12-05T17:57:12.432 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/mount.py", line 201, in run_shell_payload 2020-12-05T17:57:12.433 INFO:tasks.cephfs_test_runner: return self.run_shell(["bash", "-c", Raw(f"'{payload}'")], **kwargs) 2020-12-05T17:57:12.433 INFO:tasks.cephfs_test_runner:TypeError: run_shell() got an unexpected keyword argument 'timeout' 2020-12-05T17:57:12.433 INFO:tasks.cephfs_test_runner: 2020-12-05T17:57:12.433 INFO:tasks.cephfs_test_runner:====================================================================== 2020-12-05T17:57:12.434 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2020-12-05T17:57:12.434 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2020-12-05T17:57:12.434 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2020-12-05T17:57:12.435 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2020-12-05T17:57:12.435 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2020-12-05T17:57:12.435 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2020-12-05T17:57:12.435 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2020-12-05T17:57:12.436 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2020-12-05T17:57:12.436 INFO:tasks.cephfs_test_runner: 2020-12-05T17:57:12.436 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2020-12-05T17:57:12.437 INFO:tasks.cephfs_test_runner: 2020-12-05T17:57:12.437 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2020-12-05T17:57:12.437 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/cephfs_test_case.py", line 175, in tearDown 2020-12-05T17:57:12.437 INFO:tasks.cephfs_test_runner: m.teardown() 2020-12-05T17:57:12.438 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/fuse_mount.py", line 408, in teardown 2020-12-05T17:57:12.438 INFO:tasks.cephfs_test_runner: self.umount() 2020-12-05T17:57:12.438 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/fuse_mount.py", line 261, in umount 2020-12-05T17:57:12.438 INFO:tasks.cephfs_test_runner: if not self.is_mounted(): 2020-12-05T17:57:12.439 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2020-12-05T17:57:12.439 INFO:tasks.cephfs_test_runner: proc.wait() 2020-12-05T17:57:12.439 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 147, in wait 2020-12-05T17:57:12.439 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2020-12-05T17:57:12.440 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 683, in gevent._greenlet.Greenlet.get 2020-12-05T17:57:12.440 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 317, in gevent._greenlet.Greenlet._raise_exception 2020-12-05T17:57:12.440 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 47, in reraise 2020-12-05T17:57:12.440 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2020-12-05T17:57:12.441 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run 2020-12-05T17:57:12.441 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 316, in copy_file_to 2020-12-05T17:57:12.441 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream) 2020-12-05T17:57:12.442 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 273, in copy_to_log 2020-12-05T17:57:12.442 INFO:tasks.cephfs_test_runner: for line in f: 2020-12-05T17:57:12.442 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2020-12-05T17:57:12.442 INFO:tasks.cephfs_test_runner: line = self.readline() 2020-12-05T17:57:12.443 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2020-12-05T17:57:12.443 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2020-12-05T17:57:12.443 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2020-12-05T17:57:12.443 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2020-12-05T17:57:12.444 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2020-12-05T17:57:12.444 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2020-12-05T17:57:12.444 INFO:tasks.cephfs_test_runner:socket.timeout 2020-12-05T17:57:12.444 INFO:tasks.cephfs_test_runner: 2020-12-05T17:57:12.445 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2020-12-05T17:57:12.445 INFO:tasks.cephfs_test_runner:Ran 12 tests in 1500.551s 2020-12-05T17:57:12.445 INFO:tasks.cephfs_test_runner: 2020-12-05T17:57:12.446 INFO:tasks.cephfs_test_runner:FAILED (errors=2) 2020-12-05T17:57:12.446 INFO:tasks.cephfs_test_runner: 2020-12-05T17:57:12.446 INFO:tasks.cephfs_test_runner:====================================================================== 2020-12-05T17:57:12.446 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2020-12-05T17:57:12.447 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2020-12-05T17:57:12.447 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2020-12-05T17:57:12.447 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/test_misc.py", line 24, in test_statfs_on_deleted_fs 2020-12-05T17:57:12.448 INFO:tasks.cephfs_test_runner: p = self.mount_a.run_shell_payload("stat -f .", wait=False, timeout=60, check_status=False) 2020-12-05T17:57:12.448 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/mount.py", line 201, in run_shell_payload 2020-12-05T17:57:12.448 INFO:tasks.cephfs_test_runner: return self.run_shell(["bash", "-c", Raw(f"'{payload}'")], **kwargs) 2020-12-05T17:57:12.448 INFO:tasks.cephfs_test_runner:TypeError: run_shell() got an unexpected keyword argument 'timeout' 2020-12-05T17:57:12.449 INFO:tasks.cephfs_test_runner: 2020-12-05T17:57:12.449 INFO:tasks.cephfs_test_runner:====================================================================== 2020-12-05T17:57:12.449 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2020-12-05T17:57:12.449 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2020-12-05T17:57:12.450 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2020-12-05T17:57:12.450 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2020-12-05T17:57:12.450 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2020-12-05T17:57:12.451 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2020-12-05T17:57:12.451 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2020-12-05T17:57:12.451 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2020-12-05T17:57:12.451 INFO:tasks.cephfs_test_runner: 2020-12-05T17:57:12.452 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2020-12-05T17:57:12.452 INFO:tasks.cephfs_test_runner: 2020-12-05T17:57:12.452 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2020-12-05T17:57:12.452 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/cephfs_test_case.py", line 175, in tearDown 2020-12-05T17:57:12.453 INFO:tasks.cephfs_test_runner: m.teardown() 2020-12-05T17:57:12.453 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/fuse_mount.py", line 408, in teardown 2020-12-05T17:57:12.453 INFO:tasks.cephfs_test_runner: self.umount() 2020-12-05T17:57:12.453 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/fuse_mount.py", line 261, in umount 2020-12-05T17:57:12.454 INFO:tasks.cephfs_test_runner: if not self.is_mounted(): 2020-12-05T17:57:12.454 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2020-12-05T17:57:12.454 INFO:tasks.cephfs_test_runner: proc.wait() 2020-12-05T17:57:12.454 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 147, in wait 2020-12-05T17:57:12.455 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2020-12-05T17:57:12.455 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 683, in gevent._greenlet.Greenlet.get 2020-12-05T17:57:12.455 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 317, in gevent._greenlet.Greenlet._raise_exception 2020-12-05T17:57:12.455 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 47, in reraise 2020-12-05T17:57:12.456 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2020-12-05T17:57:12.456 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run 2020-12-05T17:57:12.456 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 316, in copy_file_to 2020-12-05T17:57:12.456 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream) 2020-12-05T17:57:12.457 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 273, in copy_to_log 2020-12-05T17:57:12.457 INFO:tasks.cephfs_test_runner: for line in f: 2020-12-05T17:57:12.457 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2020-12-05T17:57:12.457 INFO:tasks.cephfs_test_runner: line = self.readline() 2020-12-05T17:57:12.458 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2020-12-05T17:57:12.458 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2020-12-05T17:57:12.458 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2020-12-05T17:57:12.459 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2020-12-05T17:57:12.459 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2020-12-05T17:57:12.459 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2020-12-05T17:57:12.459 INFO:tasks.cephfs_test_runner:socket.timeout 2020-12-05T17:57:12.460 INFO:tasks.cephfs_test_runner: 2020-12-05T17:57:12.460 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 93, in run_tasks manager.__enter__() File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc), test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2020-12-05T17:57:12.486 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=a6fbc796a88a4ab893f8a0ba66fa62de Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 93, in run_tasks manager.__enter__() File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc), test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2020-12-05T17:57:12.486 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2020-12-05T17:57:12.495 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2020-12-05T17:57:12.517 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2020-12-05T17:57:12.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-12-05T17:57:12.518 INFO:teuthology.orchestra.run.smithi036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-12-05T17:57:12.519 INFO:teuthology.orchestra.run.smithi036:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-12-05T17:57:18.130 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:57:18.132 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:57:18.135 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:57:18.137 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:57:18.140 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:57:48.184 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:57:48.187 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:57:48.190 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:57:48.192 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:57:48.194 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:58:18.248 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:58:18.252 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:58:18.254 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:58:18.257 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:58:18.260 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:58:48.309 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:58:48.311 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:58:48.314 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:58:48.316 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:58:48.318 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:59:18.364 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:59:18.367 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:59:18.369 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:59:18.372 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:59:18.374 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:59:48.417 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:59:48.420 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:59:48.423 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:59:48.425 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T17:59:48.428 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:00:18.482 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:00:18.484 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:00:18.487 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:00:18.489 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:00:18.492 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:00:48.554 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:00:48.556 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:00:48.559 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:00:48.561 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:00:48.564 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:01:18.627 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:01:18.630 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:01:18.633 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:01:18.637 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:01:18.640 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:01:48.731 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:01:48.734 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:01:48.736 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:01:48.739 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:01:48.741 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:02:18.788 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:02:18.791 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:02:18.794 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:02:18.796 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:02:18.799 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:02:48.842 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:02:48.845 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:02:48.847 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:02:48.849 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:02:48.852 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:03:18.896 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:03:18.899 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:03:18.902 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:03:18.904 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:03:18.907 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:03:48.950 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:03:48.952 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:03:48.955 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:03:48.957 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:03:48.960 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:04:18.999 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:04:19.002 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:04:19.004 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:04:19.007 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:04:19.009 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:04:49.062 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:04:49.066 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:04:49.068 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:04:49.070 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:04:49.073 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:05:19.118 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:05:19.121 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:05:19.123 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:05:19.126 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:05:19.128 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:05:49.191 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:05:49.194 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:05:49.196 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:05:49.198 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:05:49.201 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:06:19.264 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:06:19.267 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:06:19.269 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:06:19.272 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:06:19.274 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:06:49.328 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:06:49.331 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:06:49.334 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:06:49.337 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:06:49.339 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:07:19.375 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:07:19.378 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:07:19.380 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:07:19.383 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:07:19.385 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:07:49.424 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:07:49.427 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:07:49.430 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:07:49.432 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:07:49.434 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:08:19.483 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:08:19.486 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:08:19.488 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:08:19.491 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:08:19.493 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:08:49.553 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:08:49.556 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:08:49.558 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:08:49.560 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:08:49.563 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:09:19.599 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:09:19.602 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:09:19.605 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:09:19.607 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:09:19.610 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:09:49.661 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:09:49.664 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:09:49.667 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:09:49.670 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:09:49.672 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:10:19.718 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:10:19.721 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:10:19.723 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:10:19.726 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:10:19.728 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:10:49.934 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:10:49.937 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:10:49.941 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:10:49.943 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:10:49.946 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:11:20.004 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:11:20.007 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:11:20.009 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:11:20.012 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:11:20.014 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:11:50.072 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:11:50.075 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:11:50.078 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:11:50.080 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:11:50.083 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:12:12.525 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 316, in copy_file_to copy_to_log(src, logger, capture=stream) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 273, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2020-12-05T18:12:12.526 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 316, in copy_file_to copy_to_log(src, logger, capture=stream) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 273, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2020-12-05T18:12:12.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T18:12:12.530 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 172, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/ceph_fuse.py", line 159, in task if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 147, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 683, in gevent._greenlet.Greenlet.get File "src/gevent/greenlet.py", line 317, in gevent._greenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 47, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 316, in copy_file_to copy_to_log(src, logger, capture=stream) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 273, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2020-12-05T18:12:12.531 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2020-12-05T18:12:12.548 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2020-12-05T18:12:12.549 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T18:12:12.677 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.672+0000 7eff6d32e700 1 Processor -- start 2020-12-05T18:12:12.677 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.672+0000 7eff6d32e700 1 -- start start 2020-12-05T18:12:12.679 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff6d32e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff680b14e0 0x7eff680ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:12.679 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff6d32e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7eff680af480 0x7eff68142ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:12.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff6d32e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff68142ab0 con 0x7eff680b5f00 2020-12-05T18:12:12.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff6d32e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff680af8f0 con 0x7eff680af480 2020-12-05T18:12:12.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff6d32e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff680b62e0 con 0x7eff680b14e0 2020-12-05T18:12:12.680 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff67fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7eff680b5f00 0x7eff68066370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44612/0 (socket says 172.21.15.32:44612) 2020-12-05T18:12:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff67fff700 1 -- 172.21.15.32:0/2222919695 learned_addr learned my addr 172.21.15.32:0/2222919695 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T18:12:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff6cb2d700 1 --2- 172.21.15.32:0/2222919695 >> v2:172.21.15.61:3300/0 conn(0x7eff680af480 0x7eff68142ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff677fe700 1 --2- 172.21.15.32:0/2222919695 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff680b14e0 0x7eff680ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff66ffd700 1 -- 172.21.15.32:0/2222919695 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2521212155 0 0) 0x7eff68142ab0 con 0x7eff680b5f00 2020-12-05T18:12:12.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff66ffd700 1 -- 172.21.15.32:0/2222919695 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff48003230 con 0x7eff680b5f00 2020-12-05T18:12:12.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff6cb2d700 1 -- 172.21.15.32:0/2222919695 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff680b14e0 msgr2=0x7eff680ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T18:12:12.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff6cb2d700 1 --2- 172.21.15.32:0/2222919695 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff680b14e0 0x7eff680ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:12.682 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff6cb2d700 1 -- 172.21.15.32:0/2222919695 >> v1:172.21.15.32:6789/0 conn(0x7eff680b5f00 legacy=0x7eff68066370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:12.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff6cb2d700 1 -- 172.21.15.32:0/2222919695 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff680afbf0 con 0x7eff680af480 2020-12-05T18:12:12.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff677fe700 1 --2- 172.21.15.32:0/2222919695 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff680b14e0 0x7eff680ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T18:12:12.683 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff6cb2d700 1 --2- 172.21.15.32:0/2222919695 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff680af480 0x7eff68142ee0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7eff600032b0 tx=0x7eff600091a0).ready entity=mon.1 client_cookie=f6138c98a29c7c21 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:12.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff66ffd700 1 -- 172.21.15.32:0/2222919695 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff6000b940 con 0x7eff680af480 2020-12-05T18:12:12.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff66ffd700 1 -- 172.21.15.32:0/2222919695 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6000baa0 con 0x7eff680af480 2020-12-05T18:12:12.686 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff66ffd700 1 -- 172.21.15.32:0/2222919695 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff6000bc10 con 0x7eff680af480 2020-12-05T18:12:12.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff6d32e700 1 -- 172.21.15.32:0/2222919695 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff680af480 msgr2=0x7eff68142ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:12.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff6d32e700 1 --2- 172.21.15.32:0/2222919695 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff680af480 0x7eff68142ee0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7eff600032b0 tx=0x7eff600091a0).stop 2020-12-05T18:12:12.687 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff6d32e700 1 -- 172.21.15.32:0/2222919695 shutdown_connections 2020-12-05T18:12:12.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff6d32e700 1 --2- 172.21.15.32:0/2222919695 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff680af480 0x7eff68142ee0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:12.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff6d32e700 1 --2- 172.21.15.32:0/2222919695 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff680b14e0 0x7eff680ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:12.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff6d32e700 1 -- 172.21.15.32:0/2222919695 >> 172.21.15.32:0/2222919695 conn(0x7eff680604c0 msgr2=0x7eff6805c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:12.688 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff6d32e700 1 -- 172.21.15.32:0/2222919695 shutdown_connections 2020-12-05T18:12:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.676+0000 7eff6d32e700 1 -- 172.21.15.32:0/2222919695 wait complete. 2020-12-05T18:12:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff6d32e700 1 Processor -- start 2020-12-05T18:12:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff6d32e700 1 -- start start 2020-12-05T18:12:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff6d32e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff680af480 0x7eff6810f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:12.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff6d32e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff680b14e0 0x7eff6814f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:12.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff6d32e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff680b5f00 0x7eff6813bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:12.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff6d32e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6814f8e0 con 0x7eff680b14e0 2020-12-05T18:12:12.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff6d32e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6814fb40 con 0x7eff680af480 2020-12-05T18:12:12.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff6d32e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6814fde0 con 0x7eff680b5f00 2020-12-05T18:12:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff680b14e0 0x7eff6814f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff680b14e0 0x7eff6814f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41968/0 (socket says 172.21.15.32:41968) 2020-12-05T18:12:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff677fe700 1 -- 172.21.15.32:0/1810219950 learned_addr learned my addr 172.21.15.32:0/1810219950 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:12.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff6cb2d700 1 --2- 172.21.15.32:0/1810219950 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff680b5f00 0x7eff6813bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:12.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff67fff700 1 --2- 172.21.15.32:0/1810219950 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff680af480 0x7eff6810f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:12.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff677fe700 1 -- 172.21.15.32:0/1810219950 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff680b5f00 msgr2=0x7eff6813bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:12.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff677fe700 1 --2- 172.21.15.32:0/1810219950 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff680b5f00 0x7eff6813bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:12.692 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff677fe700 1 -- 172.21.15.32:0/1810219950 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff680af480 msgr2=0x7eff6810f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:12.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff677fe700 1 --2- 172.21.15.32:0/1810219950 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff680af480 0x7eff6810f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:12.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff677fe700 1 -- 172.21.15.32:0/1810219950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff68150080 con 0x7eff680b14e0 2020-12-05T18:12:12.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff677fe700 1 --2- 172.21.15.32:0/1810219950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff680b14e0 0x7eff6814f3f0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 rx=0x7eff5800c000 tx=0x7eff58006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:12.693 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff657fa700 1 -- 172.21.15.32:0/1810219950 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff5800cb40 con 0x7eff680b14e0 2020-12-05T18:12:12.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff6d32e700 1 -- 172.21.15.32:0/1810219950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff68147580 con 0x7eff680b14e0 2020-12-05T18:12:12.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff657fa700 1 -- 172.21.15.32:0/1810219950 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5800cca0 con 0x7eff680b14e0 2020-12-05T18:12:12.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff657fa700 1 -- 172.21.15.32:0/1810219950 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff580123f0 con 0x7eff680b14e0 2020-12-05T18:12:12.694 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff6d32e700 1 -- 172.21.15.32:0/1810219950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff6813c180 con 0x7eff680b14e0 2020-12-05T18:12:12.695 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.680+0000 7eff6d32e700 1 -- 172.21.15.32:0/1810219950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff6813c7d0 con 0x7eff680b14e0 2020-12-05T18:12:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.684+0000 7eff657fa700 1 -- 172.21.15.32:0/1810219950 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7eff58012550 con 0x7eff680b14e0 2020-12-05T18:12:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.684+0000 7eff657fa700 1 --2- 172.21.15.32:0/1810219950 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7eff4c055440 0x7eff4c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.684+0000 7eff657fa700 1 -- 172.21.15.32:0/1810219950 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7eff5806e4f0 con 0x7eff680b14e0 2020-12-05T18:12:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.684+0000 7eff67fff700 1 --2- 172.21.15.32:0/1810219950 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7eff4c055440 0x7eff4c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.684+0000 7eff67fff700 1 --2- 172.21.15.32:0/1810219950 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7eff4c055440 0x7eff4c057900 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7eff5000d000 tx=0x7eff50009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.692+0000 7eff657fa700 1 -- 172.21.15.32:0/1810219950 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7eff58044870 con 0x7eff680b14e0 2020-12-05T18:12:12.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.832+0000 7eff6d32e700 1 -- 172.21.15.32:0/1810219950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff68057050 con 0x7eff680b14e0 2020-12-05T18:12:12.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.832+0000 7eff657fa700 1 -- 172.21.15.32:0/1810219950 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+13553 (secure 0 0 0) 0x7eff58047e90 con 0x7eff680b14e0 2020-12-05T18:12:12.836 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T18:12:12.837 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":151,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:42:12.290182+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.32:6835/1575496107":"2020-12-06T17:42:08.989610+0000","172.21.15.32:6834/1575496107":"2020-12-06T17:42:08.989610+0000"},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T18:12:12.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.832+0000 7eff6d32e700 1 -- 172.21.15.32:0/1810219950 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7eff4c055440 msgr2=0x7eff4c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:12.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.832+0000 7eff6d32e700 1 --2- 172.21.15.32:0/1810219950 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7eff4c055440 0x7eff4c057900 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7eff5000d000 tx=0x7eff50009040).stop 2020-12-05T18:12:12.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.832+0000 7eff6d32e700 1 -- 172.21.15.32:0/1810219950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff680b14e0 msgr2=0x7eff6814f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:12.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.832+0000 7eff6d32e700 1 --2- 172.21.15.32:0/1810219950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff680b14e0 0x7eff6814f3f0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 rx=0x7eff5800c000 tx=0x7eff58006dc0).stop 2020-12-05T18:12:12.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.836+0000 7eff6d32e700 1 -- 172.21.15.32:0/1810219950 shutdown_connections 2020-12-05T18:12:12.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.836+0000 7eff6d32e700 1 --2- 172.21.15.32:0/1810219950 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7eff4c055440 0x7eff4c057900 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:12.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.836+0000 7eff6d32e700 1 --2- 172.21.15.32:0/1810219950 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff680b5f00 0x7eff6813bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:12.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.836+0000 7eff6d32e700 1 --2- 172.21.15.32:0/1810219950 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff680af480 0x7eff6810f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:12.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.836+0000 7eff6d32e700 1 --2- 172.21.15.32:0/1810219950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff680b14e0 0x7eff6814f3f0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:12.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.836+0000 7eff6d32e700 1 -- 172.21.15.32:0/1810219950 >> 172.21.15.32:0/1810219950 conn(0x7eff680604c0 msgr2=0x7eff6813f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:12.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.836+0000 7eff6d32e700 1 -- 172.21.15.32:0/1810219950 shutdown_connections 2020-12-05T18:12:12.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.836+0000 7eff6d32e700 1 -- 172.21.15.32:0/1810219950 wait complete. 2020-12-05T18:12:12.850 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-12-05T18:12:12.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.960+0000 7f92d8d56700 1 Processor -- start 2020-12-05T18:12:12.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.960+0000 7f92d8d56700 1 -- start start 2020-12-05T18:12:12.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.960+0000 7f92d8d56700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92d40b5f00 0x7f92d40b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:12.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.960+0000 7f92d8d56700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92d4142ab0 con 0x7f92d40af480 2020-12-05T18:12:12.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.960+0000 7f92d8d56700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92d40b62e0 con 0x7f92d40ad510 2020-12-05T18:12:12.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.960+0000 7f92d8d56700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92d40af890 con 0x7f92d40b5f00 2020-12-05T18:12:12.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.960+0000 7f92d37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92d40b5f00 0x7f92d40b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:12.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.960+0000 7f92d3fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f92d40af480 0x7f92d4142ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44622/0 (socket says 172.21.15.32:44622) 2020-12-05T18:12:12.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.960+0000 7f92d3fff700 1 -- 172.21.15.32:0/644984891 learned_addr learned my addr 172.21.15.32:0/644984891 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T18:12:12.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.960+0000 7f92d37fe700 1 -- 172.21.15.32:0/644984891 >> v1:172.21.15.61:6789/0 conn(0x7f92d40ad510 legacy=0x7f92d4066370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:12.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.960+0000 7f92d37fe700 1 -- 172.21.15.32:0/644984891 >> v1:172.21.15.32:6789/0 conn(0x7f92d40af480 legacy=0x7f92d4142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:12.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.960+0000 7f92d27fc700 1 -- 172.21.15.32:0/644984891 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4097339781 0 0) 0x7f92d4142ab0 con 0x7f92d40af480 2020-12-05T18:12:12.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.960+0000 7f92d37fe700 1 -- 172.21.15.32:0/644984891 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d40afbf0 con 0x7f92d40b5f00 2020-12-05T18:12:12.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.960+0000 7f92d37fe700 1 --2- 172.21.15.32:0/644984891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92d40b5f00 0x7f92d40b14e0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7f92c40032b0 tx=0x7f92c40091a0).ready entity=mon.0 client_cookie=e5825d3d0d68e830 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:12.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.960+0000 7f92d27fc700 1 -- 172.21.15.32:0/644984891 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92c400ba50 con 0x7f92d40b5f00 2020-12-05T18:12:12.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.960+0000 7f92d27fc700 1 -- 172.21.15.32:0/644984891 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c400bbb0 con 0x7f92d40b5f00 2020-12-05T18:12:12.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.960+0000 7f92d27fc700 1 -- 172.21.15.32:0/644984891 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92c400bd20 con 0x7f92d40b5f00 2020-12-05T18:12:12.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.960+0000 7f92d8d56700 1 -- 172.21.15.32:0/644984891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92d40b5f00 msgr2=0x7f92d40b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:12.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.960+0000 7f92d8d56700 1 --2- 172.21.15.32:0/644984891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92d40b5f00 0x7f92d40b14e0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7f92c40032b0 tx=0x7f92c40091a0).stop 2020-12-05T18:12:12.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.960+0000 7f92d8d56700 1 -- 172.21.15.32:0/644984891 shutdown_connections 2020-12-05T18:12:12.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.960+0000 7f92d8d56700 1 --2- 172.21.15.32:0/644984891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92d40b5f00 0x7f92d40b14e0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:12.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.960+0000 7f92d8d56700 1 -- 172.21.15.32:0/644984891 >> 172.21.15.32:0/644984891 conn(0x7f92d40604c0 msgr2=0x7f92d405c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:12.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d8d56700 1 -- 172.21.15.32:0/644984891 shutdown_connections 2020-12-05T18:12:12.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d8d56700 1 -- 172.21.15.32:0/644984891 wait complete. 2020-12-05T18:12:12.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d8d56700 1 Processor -- start 2020-12-05T18:12:12.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d8d56700 1 -- start start 2020-12-05T18:12:12.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d8d56700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92d40ad510 0x7f92d4063f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:12.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d8d56700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92d40af480 0x7f92d4140f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:12.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d8d56700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92d40b5f00 0x7f92d4063320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:12.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d8d56700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d413e320 con 0x7f92d40b5f00 2020-12-05T18:12:12.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d8d56700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d413d790 con 0x7f92d40af480 2020-12-05T18:12:12.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d8d56700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d413da30 con 0x7f92d40ad510 2020-12-05T18:12:12.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92d40b5f00 0x7f92d4063320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:12.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92d40b5f00 0x7f92d4063320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41984/0 (socket says 172.21.15.32:41984) 2020-12-05T18:12:12.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d3fff700 1 -- 172.21.15.32:0/3477587927 learned_addr learned my addr 172.21.15.32:0/3477587927 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:12.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d37fe700 1 --2- 172.21.15.32:0/3477587927 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92d40ad510 0x7f92d4063f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:12.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d2ffd700 1 --2- 172.21.15.32:0/3477587927 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92d40af480 0x7f92d4140f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:12.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d3fff700 1 -- 172.21.15.32:0/3477587927 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92d40ad510 msgr2=0x7f92d4063f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:12.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d3fff700 1 --2- 172.21.15.32:0/3477587927 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92d40ad510 0x7f92d4063f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:12.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d3fff700 1 -- 172.21.15.32:0/3477587927 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92d40af480 msgr2=0x7f92d4140f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:12.975 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d3fff700 1 --2- 172.21.15.32:0/3477587927 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92d40af480 0x7f92d4140f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:12.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d3fff700 1 -- 172.21.15.32:0/3477587927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d413cee0 con 0x7f92d40b5f00 2020-12-05T18:12:12.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d3fff700 1 --2- 172.21.15.32:0/3477587927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92d40b5f00 0x7f92d4063320 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 rx=0x7f92cc000b80 tx=0x7f92cc007a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:12.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d0ff9700 1 -- 172.21.15.32:0/3477587927 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92cc0072e0 con 0x7f92d40b5f00 2020-12-05T18:12:12.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d8d56700 1 -- 172.21.15.32:0/3477587927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d413cab0 con 0x7f92d40b5f00 2020-12-05T18:12:12.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d0ff9700 1 -- 172.21.15.32:0/3477587927 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92cc007440 con 0x7f92d40b5f00 2020-12-05T18:12:12.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d8d56700 1 -- 172.21.15.32:0/3477587927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d413d500 con 0x7f92d40b5f00 2020-12-05T18:12:12.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.964+0000 7f92d0ff9700 1 -- 172.21.15.32:0/3477587927 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92cc0055e0 con 0x7f92d40b5f00 2020-12-05T18:12:12.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.968+0000 7f92d8d56700 1 -- 172.21.15.32:0/3477587927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d413dbf0 con 0x7f92d40b5f00 2020-12-05T18:12:12.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.968+0000 7f92d0ff9700 1 -- 172.21.15.32:0/3477587927 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f92cc010070 con 0x7f92d40b5f00 2020-12-05T18:12:12.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.968+0000 7f92d0ff9700 1 --2- 172.21.15.32:0/3477587927 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f92ac0553b0 0x7f92ac057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:12.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.968+0000 7f92d0ff9700 1 -- 172.21.15.32:0/3477587927 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f92cc06d450 con 0x7f92d40b5f00 2020-12-05T18:12:12.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.968+0000 7f92d37fe700 1 --2- 172.21.15.32:0/3477587927 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f92ac0553b0 0x7f92ac057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:12.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.968+0000 7f92d37fe700 1 --2- 172.21.15.32:0/3477587927 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f92ac0553b0 0x7f92ac057870 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f92c4019000 tx=0x7f92c4016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:12.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:12.972+0000 7f92d0ff9700 1 -- 172.21.15.32:0/3477587927 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f92cc043740 con 0x7f92d40b5f00 2020-12-05T18:12:13.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.108+0000 7f92d8d56700 1 -- 172.21.15.32:0/3477587927 --> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f92d4057050 con 0x7f92ac0553b0 2020-12-05T18:12:13.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.112+0000 7f92d0ff9700 1 -- 172.21.15.32:0/3477587927 <== mgr.4101 v2:172.21.15.32:6800/11355 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+66177 (secure 0 0 0) 0x7f92d4057050 con 0x7f92ac0553b0 2020-12-05T18:12:13.118 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T18:12:13.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.116+0000 7f92d8d56700 1 -- 172.21.15.32:0/3477587927 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f92ac0553b0 msgr2=0x7f92ac057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:13.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.116+0000 7f92d8d56700 1 --2- 172.21.15.32:0/3477587927 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f92ac0553b0 0x7f92ac057870 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f92c4019000 tx=0x7f92c4016000).stop 2020-12-05T18:12:13.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.116+0000 7f92d8d56700 1 -- 172.21.15.32:0/3477587927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92d40b5f00 msgr2=0x7f92d4063320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:13.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.116+0000 7f92d8d56700 1 --2- 172.21.15.32:0/3477587927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92d40b5f00 0x7f92d4063320 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 rx=0x7f92cc000b80 tx=0x7f92cc007a50).stop 2020-12-05T18:12:13.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.116+0000 7f92d8d56700 1 -- 172.21.15.32:0/3477587927 shutdown_connections 2020-12-05T18:12:13.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.116+0000 7f92d8d56700 1 --2- 172.21.15.32:0/3477587927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92d40b5f00 0x7f92d4063320 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:13.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.116+0000 7f92d8d56700 1 --2- 172.21.15.32:0/3477587927 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f92ac0553b0 0x7f92ac057870 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:13.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.116+0000 7f92d8d56700 1 --2- 172.21.15.32:0/3477587927 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92d40ad510 0x7f92d4063f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:13.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.116+0000 7f92d8d56700 1 --2- 172.21.15.32:0/3477587927 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92d40af480 0x7f92d4140f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:13.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.116+0000 7f92d8d56700 1 -- 172.21.15.32:0/3477587927 >> 172.21.15.32:0/3477587927 conn(0x7f92d40604c0 msgr2=0x7f92d41432a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:13.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.116+0000 7f92d8d56700 1 -- 172.21.15.32:0/3477587927 shutdown_connections 2020-12-05T18:12:13.124 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.116+0000 7f92d8d56700 1 -- 172.21.15.32:0/3477587927 wait complete. 2020-12-05T18:12:13.124 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2020-12-05T18:12:13.136 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":1366,"stamp":"2020-12-05T18:12:11.518801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":8403712,"kb_used_data":15040,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746571008,"statfs":{"total":773094113280,"available":764488712192,"internally_reserved":8589934592,"allocated":15400960,"data_stored":7505640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003356"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":"129","reported_epoch":"150","state":"active+clean","last_fresh":"2020-12-05T17:42:11.669103+0000","last_change":"2020-12-05T17:31:35.374114+0000","last_active":"2020-12-05T17:42:11.669103+0000","last_peered":"2020-12-05T17:42:11.669103+0000","last_clean":"2020-12-05T17:42:11.669103+0000","last_became_active":"2020-12-05T17:31:35.373231+0000","last_became_peered":"2020-12-05T17:31:35.373231+0000","last_unstale":"2020-12-05T17:42:11.669103+0000","last_undegraded":"2020-12-05T17:42:11.669103+0000","last_fullsized":"2020-12-05T17:42:11.669103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"134","reported_epoch":"150","state":"active+clean","last_fresh":"2020-12-05T17:42:11.294377+0000","last_change":"2020-12-05T17:31:35.123847+0000","last_active":"2020-12-05T17:42:11.294377+0000","last_peered":"2020-12-05T17:42:11.294377+0000","last_clean":"2020-12-05T17:42:11.294377+0000","last_became_active":"2020-12-05T17:31:35.123169+0000","last_became_peered":"2020-12-05T17:31:35.123169+0000","last_unstale":"2020-12-05T17:42:11.294377+0000","last_undegraded":"2020-12-05T17:42:11.294377+0000","last_fullsized":"2020-12-05T17:42:11.294377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"128","reported_epoch":"150","state":"active+clean","last_fresh":"2020-12-05T17:42:11.668246+0000","last_change":"2020-12-05T17:31:35.617802+0000","last_active":"2020-12-05T17:42:11.668246+0000","last_peered":"2020-12-05T17:42:11.668246+0000","last_clean":"2020-12-05T17:42:11.668246+0000","last_became_active":"2020-12-05T17:31:35.617427+0000","last_became_peered":"2020-12-05T17:31:35.617427+0000","last_unstale":"2020-12-05T17:42:11.668246+0000","last_undegraded":"2020-12-05T17:42:11.668246+0000","last_fullsized":"2020-12-05T17:42:11.668246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"134","reported_epoch":"150","state":"active+clean","last_fresh":"2020-12-05T17:42:11.294441+0000","last_change":"2020-12-05T17:31:35.124252+0000","last_active":"2020-12-05T17:42:11.294441+0000","last_peered":"2020-12-05T17:42:11.294441+0000","last_clean":"2020-12-05T17:42:11.294441+0000","last_became_active":"2020-12-05T17:31:35.123671+0000","last_became_peered":"2020-12-05T17:31:35.123671+0000","last_unstale":"2020-12-05T17:42:11.294441+0000","last_undegraded":"2020-12-05T17:42:11.294441+0000","last_fullsized":"2020-12-05T17:42:11.294441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"139","reported_epoch":"150","state":"active+clean","last_fresh":"2020-12-05T17:42:11.294592+0000","last_change":"2020-12-05T17:31:30.406003+0000","last_active":"2020-12-05T17:42:11.294592+0000","last_peered":"2020-12-05T17:42:11.294592+0000","last_clean":"2020-12-05T17:42:11.294592+0000","last_became_active":"2020-12-05T17:31:30.405245+0000","last_became_peered":"2020-12-05T17:31:30.405245+0000","last_unstale":"2020-12-05T17:42:11.294592+0000","last_undegraded":"2020-12-05T17:42:11.294592+0000","last_fullsized":"2020-12-05T17:42:11.294592+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:25.806577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:25.806577+0000","last_clean_scrub_stamp":"2020-12-05T17:31:25.806577+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"134","reported_epoch":"150","state":"active+clean","last_fresh":"2020-12-05T17:42:11.294687+0000","last_change":"2020-12-05T17:31:35.118985+0000","last_active":"2020-12-05T17:42:11.294687+0000","last_peered":"2020-12-05T17:42:11.294687+0000","last_clean":"2020-12-05T17:42:11.294687+0000","last_became_active":"2020-12-05T17:31:35.118441+0000","last_became_peered":"2020-12-05T17:31:35.118441+0000","last_unstale":"2020-12-05T17:42:11.294687+0000","last_undegraded":"2020-12-05T17:42:11.294687+0000","last_fullsized":"2020-12-05T17:42:11.294687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"134","reported_epoch":"150","state":"active+clean","last_fresh":"2020-12-05T17:42:11.296171+0000","last_change":"2020-12-05T17:31:35.375163+0000","last_active":"2020-12-05T17:42:11.296171+0000","last_peered":"2020-12-05T17:42:11.296171+0000","last_clean":"2020-12-05T17:42:11.296171+0000","last_became_active":"2020-12-05T17:31:35.374461+0000","last_became_peered":"2020-12-05T17:31:35.374461+0000","last_unstale":"2020-12-05T17:42:11.296171+0000","last_undegraded":"2020-12-05T17:42:11.296171+0000","last_fullsized":"2020-12-05T17:42:11.296171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"129","reported_epoch":"150","state":"active+clean","last_fresh":"2020-12-05T17:42:11.670153+0000","last_change":"2020-12-05T17:31:35.123497+0000","last_active":"2020-12-05T17:42:11.670153+0000","last_peered":"2020-12-05T17:42:11.670153+0000","last_clean":"2020-12-05T17:42:11.670153+0000","last_became_active":"2020-12-05T17:31:35.122972+0000","last_became_peered":"2020-12-05T17:31:35.122972+0000","last_unstale":"2020-12-05T17:42:11.670153+0000","last_undegraded":"2020-12-05T17:42:11.670153+0000","last_fullsized":"2020-12-05T17:42:11.670153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"130","reported_epoch":"150","state":"active+clean","last_fresh":"2020-12-05T17:42:11.667390+0000","last_change":"2020-12-05T17:31:35.127416+0000","last_active":"2020-12-05T17:42:11.667390+0000","last_peered":"2020-12-05T17:42:11.667390+0000","last_clean":"2020-12-05T17:42:11.667390+0000","last_became_active":"2020-12-05T17:31:35.126829+0000","last_became_peered":"2020-12-05T17:31:35.126829+0000","last_unstale":"2020-12-05T17:42:11.667390+0000","last_undegraded":"2020-12-05T17:42:11.667390+0000","last_fullsized":"2020-12-05T17:42:11.667390+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542634,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":1.7090000000000001,"5min":1.869,"15min":1.8340000000000001},"min":{"1min":1.242,"5min":1.1910000000000001,"15min":1.163},"max":{"1min":2.6299999999999999,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":1.6759999999999999},{"interface":"front","average":{"1min":1.847,"5min":1.8939999999999999,"15min":1.8440000000000001},"min":{"1min":1.4159999999999999,"5min":1.264,"15min":1.0920000000000001},"max":{"1min":3.0579999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":1.512}]},{"osd":1,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":1.9299999999999999,"5min":1.8959999999999999,"15min":1.8700000000000001},"min":{"1min":1.452,"5min":1.3779999999999999,"15min":1.24},"max":{"1min":2.8100000000000001,"5min":3.1019999999999999,"15min":3.444},"last":1.6120000000000001},{"interface":"front","average":{"1min":2.5369999999999999,"5min":2.3290000000000002,"15min":2.3140000000000001},"min":{"1min":1.669,"5min":1.3979999999999999,"15min":1.3979999999999999},"max":{"1min":3.472,"5min":3.472,"15min":3.6629999999999998},"last":2.766}]},{"osd":2,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":2.2789999999999999,"5min":2.3159999999999998,"15min":2.2530000000000001},"min":{"1min":1.597,"5min":1.5249999999999999,"15min":1.296},"max":{"1min":3.2669999999999999,"5min":3.6469999999999998,"15min":3.6469999999999998},"last":2.246},{"interface":"front","average":{"1min":2.2410000000000001,"5min":2.3220000000000001,"15min":2.2949999999999999},"min":{"1min":1.7250000000000001,"5min":1.5289999999999999,"15min":1.161},"max":{"1min":3.161,"5min":3.6850000000000001,"15min":3.968},"last":2.105}]},{"osd":3,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":2.524,"5min":2.6040000000000001,"15min":2.6110000000000002},"min":{"1min":1.9419999999999999,"5min":1.6100000000000001,"15min":1.6100000000000001},"max":{"1min":3.6139999999999999,"5min":3.6139999999999999,"15min":3.7709999999999999},"last":2.645},{"interface":"front","average":{"1min":2.3690000000000002,"5min":2.5249999999999999,"15min":2.4830000000000001},"min":{"1min":1.673,"5min":1.6220000000000001,"15min":1.6220000000000001},"max":{"1min":3.556,"5min":4.6909999999999998,"15min":4.6909999999999998},"last":2.4990000000000001}]},{"osd":4,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":2.665,"5min":2.718,"15min":2.7090000000000001},"min":{"1min":1.8720000000000001,"5min":1.706,"15min":1.6859999999999999},"max":{"1min":3.98,"5min":3.98,"15min":4.0739999999999998},"last":3.2170000000000001},{"interface":"front","average":{"1min":2.8620000000000001,"5min":2.9300000000000002,"15min":2.9169999999999998},"min":{"1min":1.861,"5min":1.5189999999999999,"15min":1.5189999999999999},"max":{"1min":3.9020000000000001,"5min":3.9020000000000001,"15min":4.0490000000000004},"last":3.048}]},{"osd":5,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":2.7650000000000001,"5min":2.8359999999999999,"15min":2.8380000000000001},"min":{"1min":1.929,"5min":1.7929999999999999,"15min":1.581},"max":{"1min":3.806,"5min":5.1050000000000004,"15min":5.1050000000000004},"last":3.1429999999999998},{"interface":"front","average":{"1min":3.109,"5min":3.117,"15min":3.113},"min":{"1min":2.4780000000000002,"5min":1.823,"15min":1.823},"max":{"1min":4.2380000000000004,"5min":4.335,"15min":4.375},"last":3.4550000000000001}]},{"osd":6,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":3.226,"5min":3.383,"15min":3.3220000000000001},"min":{"1min":2.7200000000000002,"5min":2.234,"15min":2.194},"max":{"1min":4.2160000000000002,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":3.6520000000000001},{"interface":"front","average":{"1min":3.2040000000000002,"5min":3.4079999999999999,"15min":3.3300000000000001},"min":{"1min":2.7509999999999999,"5min":2.395,"15min":2.395},"max":{"1min":4.1150000000000002,"5min":5.8940000000000001,"15min":5.8940000000000001},"last":3.3570000000000002}]}]},{"osd":6,"up_from":14,"seq":60129542635,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 5 18:11:53 2020","interfaces":[{"interface":"back","average":{"1min":1.744,"5min":1.772,"15min":1.7090000000000001},"min":{"1min":1.3959999999999999,"5min":1.1499999999999999,"15min":1.022},"max":{"1min":3.036,"5min":3.794,"15min":3.794},"last":2.2330000000000001},{"interface":"front","average":{"1min":1.8120000000000001,"5min":1.921,"15min":1.8480000000000001},"min":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.0760000000000001},"max":{"1min":2.3559999999999999,"5min":5.1349999999999998,"15min":5.1349999999999998},"last":1.4079999999999999}]},{"osd":1,"last update":"Sat Dec 5 18:11:53 2020","interfaces":[{"interface":"back","average":{"1min":1.774,"5min":1.885,"15min":1.855},"min":{"1min":1.403,"5min":1.403,"15min":1.224},"max":{"1min":2.1629999999999998,"5min":4.3029999999999999,"15min":4.3029999999999999},"last":1.8740000000000001},{"interface":"front","average":{"1min":2.2480000000000002,"5min":2.234,"15min":2.1930000000000001},"min":{"1min":1.667,"5min":1.3420000000000001,"15min":1.125},"max":{"1min":2.891,"5min":3.258,"15min":3.4620000000000002},"last":2.8759999999999999}]},{"osd":2,"last update":"Sat Dec 5 18:11:53 2020","interfaces":[{"interface":"back","average":{"1min":2.2040000000000002,"5min":2.331,"15min":2.266},"min":{"1min":1.512,"5min":1.456,"15min":1.393},"max":{"1min":3.4319999999999999,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":2.5110000000000001},{"interface":"front","average":{"1min":2.544,"5min":2.6000000000000001,"15min":2.581},"min":{"1min":1.9490000000000001,"5min":1.7849999999999999,"15min":1.5580000000000001},"max":{"1min":3.3599999999999999,"5min":4.6989999999999998,"15min":4.6989999999999998},"last":2.7559999999999998}]},{"osd":3,"last update":"Sat Dec 5 18:11:53 2020","interfaces":[{"interface":"back","average":{"1min":2.6640000000000001,"5min":2.6389999999999998,"15min":2.544},"min":{"1min":1.8919999999999999,"5min":1.8320000000000001,"15min":1.643},"max":{"1min":3.702,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":3.2000000000000002},{"interface":"front","average":{"1min":2.8260000000000001,"5min":2.71,"15min":2.661},"min":{"1min":1.9870000000000001,"5min":1.5129999999999999,"15min":1.298},"max":{"1min":3.8929999999999998,"5min":4.4729999999999999,"15min":4.4729999999999999},"last":2.1429999999999998}]},{"osd":4,"last update":"Sat Dec 5 18:11:53 2020","interfaces":[{"interface":"back","average":{"1min":2.9740000000000002,"5min":2.8380000000000001,"15min":2.786},"min":{"1min":2.1909999999999998,"5min":1.117,"15min":1.117},"max":{"1min":4.0919999999999996,"5min":4.1779999999999999,"15min":4.1779999999999999},"last":3.448},{"interface":"front","average":{"1min":3.0350000000000001,"5min":2.9319999999999999,"15min":2.8969999999999998},"min":{"1min":2.5430000000000001,"5min":1.7210000000000001,"15min":1.264},"max":{"1min":4.2670000000000003,"5min":5.5330000000000004,"15min":5.5330000000000004},"last":2.976}]},{"osd":5,"last update":"Sat Dec 5 18:11:53 2020","interfaces":[{"interface":"back","average":{"1min":2.6110000000000002,"5min":2.6389999999999998,"15min":2.6230000000000002},"min":{"1min":1.96,"5min":1.784,"15min":1.165},"max":{"1min":3.653,"5min":4.7699999999999996,"15min":4.7699999999999996},"last":2.3260000000000001},{"interface":"front","average":{"1min":3.2490000000000001,"5min":3.1850000000000001,"15min":3.1459999999999999},"min":{"1min":2.5830000000000002,"5min":1.429,"15min":1.429},"max":{"1min":4.4589999999999996,"5min":4.5,"15min":4.5},"last":3.6469999999999998}]},{"osd":7,"last update":"Sat Dec 5 18:11:53 2020","interfaces":[{"interface":"back","average":{"1min":3.4740000000000002,"5min":3.3380000000000001,"15min":3.2919999999999998},"min":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.077},"max":{"1min":4.6459999999999999,"5min":5.9379999999999997,"15min":5.9379999999999997},"last":3.3340000000000001},{"interface":"front","average":{"1min":3.23,"5min":3.222,"15min":3.202},"min":{"1min":2.5179999999999998,"5min":2.1659999999999999,"15min":2.0169999999999999},"max":{"1min":4.1219999999999999,"5min":4.9909999999999997,"15min":4.9909999999999997},"last":3.27}]}]},{"osd":1,"up_from":14,"seq":60129542634,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":1.766,"5min":1.714,"15min":1.7809999999999999},"min":{"1min":1.3919999999999999,"5min":1.383,"15min":1.093},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":3.3180000000000001},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.671,"5min":1.8160000000000001,"15min":1.8120000000000001},"min":{"1min":1.502,"5min":1.393,"15min":1.1830000000000001},"max":{"1min":2.242,"5min":2.6040000000000001,"15min":2.9500000000000002},"last":1.7609999999999999}]},{"osd":2,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":1.6839999999999999,"5min":1.825,"15min":1.8009999999999999},"min":{"1min":1.4339999999999999,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.4159999999999999,"5min":3.3679999999999999,"15min":3.3679999999999999},"last":1.9670000000000001},{"interface":"front","average":{"1min":2.4809999999999999,"5min":2.3210000000000002,"15min":2.2149999999999999},"min":{"1min":1.9430000000000001,"5min":1.546,"15min":1.0840000000000001},"max":{"1min":2.9769999999999999,"5min":3.2469999999999999,"15min":3.4849999999999999},"last":1.8720000000000001}]},{"osd":3,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":2.335,"5min":2.395,"15min":2.3900000000000001},"min":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.353},"max":{"1min":3.0600000000000001,"5min":3.379,"15min":3.3849999999999998},"last":2.1320000000000001},{"interface":"front","average":{"1min":2.3650000000000002,"5min":2.379,"15min":2.4039999999999999},"min":{"1min":1.8169999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"max":{"1min":2.843,"5min":3.3140000000000001,"15min":3.4020000000000001},"last":2.8370000000000002}]},{"osd":4,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":2.4140000000000001,"5min":2.3839999999999999,"15min":2.3610000000000002},"min":{"1min":2.0019999999999998,"5min":1.627,"15min":1.587},"max":{"1min":3.7879999999999998,"5min":4.0899999999999999,"15min":4.1429999999999998},"last":2.524},{"interface":"front","average":{"1min":3.113,"5min":2.98,"15min":2.9380000000000002},"min":{"1min":2.7959999999999998,"5min":2.0800000000000001,"15min":1.5649999999999999},"max":{"1min":3.5270000000000001,"5min":3.9580000000000002,"15min":4.0119999999999996},"last":2.3359999999999999}]},{"osd":5,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":2.7519999999999998,"5min":2.8690000000000002,"15min":2.8359999999999999},"min":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":1.75},"max":{"1min":3.4710000000000001,"5min":3.9860000000000002,"15min":4.1020000000000003},"last":2.4369999999999998},{"interface":"front","average":{"1min":2.7709999999999999,"5min":2.8530000000000002,"15min":2.847},"min":{"1min":2.024,"5min":2.024,"15min":2.008},"max":{"1min":3.702,"5min":3.75,"15min":3.919},"last":2.2589999999999999}]},{"osd":6,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":3.395,"5min":3.2679999999999998,"15min":3.2370000000000001},"min":{"1min":3.113,"5min":2.5880000000000001,"15min":2.1800000000000002},"max":{"1min":3.944,"5min":4.282,"15min":4.3879999999999999},"last":2.5880000000000001},{"interface":"front","average":{"1min":3.1720000000000002,"5min":3.222,"15min":3.1899999999999999},"min":{"1min":2.7090000000000001,"5min":2.5190000000000001,"15min":2.2879999999999998},"max":{"1min":3.9180000000000001,"5min":4.399,"15min":4.399},"last":3.2370000000000001}]},{"osd":7,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":3.3759999999999999,"5min":3.3999999999999999,"15min":3.3570000000000002},"min":{"1min":2.552,"5min":2.552,"15min":2.5059999999999998},"max":{"1min":4.0919999999999996,"5min":4.5279999999999996,"15min":4.5279999999999996},"last":3.6030000000000002},{"interface":"front","average":{"1min":3.1459999999999999,"5min":3.1850000000000001,"15min":3.153},"min":{"1min":2.5880000000000001,"5min":2.4470000000000001,"15min":2.2669999999999999},"max":{"1min":3.5990000000000002,"5min":4.2329999999999997,"15min":4.2329999999999997},"last":2.746}]}]},{"osd":0,"up_from":14,"seq":60129542635,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Dec 5 18:12:06 2020","interfaces":[{"interface":"back","average":{"1min":1.8140000000000001,"5min":1.7589999999999999,"15min":1.7709999999999999},"min":{"1min":1.4019999999999999,"5min":1.3149999999999999,"15min":1.0349999999999999},"max":{"1min":2.9239999999999999,"5min":2.9369999999999998,"15min":3.0510000000000002},"last":1.71},{"interface":"front","average":{"1min":1.8149999999999999,"5min":1.8180000000000001,"15min":1.831},"min":{"1min":1.53,"5min":1.2929999999999999,"15min":1.2929999999999999},"max":{"1min":3.0070000000000001,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":1.855}]},{"osd":2,"last update":"Sat Dec 5 18:12:06 2020","interfaces":[{"interface":"back","average":{"1min":2.2410000000000001,"5min":2.2789999999999999,"15min":2.2160000000000002},"min":{"1min":1.488,"5min":1.3819999999999999,"15min":1.3480000000000001},"max":{"1min":2.7709999999999999,"5min":3.1890000000000001,"15min":3.2120000000000002},"last":2.6190000000000002},{"interface":"front","average":{"1min":2.6150000000000002,"5min":2.496,"15min":2.4830000000000001},"min":{"1min":2.0339999999999998,"5min":1.498,"15min":1.407},"max":{"1min":3.2080000000000002,"5min":3.3839999999999999,"15min":3.6600000000000001},"last":2.9529999999999998}]},{"osd":3,"last update":"Sat Dec 5 18:12:06 2020","interfaces":[{"interface":"back","average":{"1min":1.98,"5min":1.8859999999999999,"15min":1.8600000000000001},"min":{"1min":1.4930000000000001,"5min":1.385,"15min":1.2130000000000001},"max":{"1min":2.9039999999999999,"5min":2.9039999999999999,"15min":3.984},"last":1.798},{"interface":"front","average":{"1min":2.395,"5min":2.476,"15min":2.496},"min":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"max":{"1min":2.895,"5min":3.7429999999999999,"15min":4.1790000000000003},"last":2.3959999999999999}]},{"osd":4,"last update":"Sat Dec 5 18:12:06 2020","interfaces":[{"interface":"back","average":{"1min":2.6880000000000002,"5min":2.5630000000000002,"15min":2.452},"min":{"1min":2.0600000000000001,"5min":1.669,"15min":1.4970000000000001},"max":{"1min":3.8540000000000001,"5min":3.9660000000000002,"15min":3.9660000000000002},"last":2.528},{"interface":"front","average":{"1min":2.4420000000000002,"5min":2.536,"15min":2.5070000000000001},"min":{"1min":1.802,"5min":1.397,"15min":1.397},"max":{"1min":3.7330000000000001,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":2.7080000000000002}]},{"osd":5,"last update":"Sat Dec 5 18:12:06 2020","interfaces":[{"interface":"back","average":{"1min":2.3849999999999998,"5min":2.5449999999999999,"15min":2.5129999999999999},"min":{"1min":1.917,"5min":1.917,"15min":1.4850000000000001},"max":{"1min":3.0449999999999999,"5min":3.6259999999999999,"15min":3.8260000000000001},"last":2.3340000000000001},{"interface":"front","average":{"1min":2.9529999999999998,"5min":3.0289999999999999,"15min":3.0350000000000001},"min":{"1min":2.5150000000000001,"5min":2.1829999999999998,"15min":1.9610000000000001},"max":{"1min":3.2509999999999999,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":2.8809999999999998}]},{"osd":6,"last update":"Sat Dec 5 18:12:06 2020","interfaces":[{"interface":"back","average":{"1min":3.25,"5min":3.2589999999999999,"15min":3.2490000000000001},"min":{"1min":2.8769999999999998,"5min":2.5369999999999999,"15min":2.2879999999999998},"max":{"1min":3.681,"5min":4.5709999999999997,"15min":5.3280000000000003},"last":3.1190000000000002},{"interface":"front","average":{"1min":3.3300000000000001,"5min":3.0830000000000002,"15min":3.0630000000000002},"min":{"1min":2.758,"5min":2.2069999999999999,"15min":2.1709999999999998},"max":{"1min":3.77,"5min":3.7730000000000001,"15min":4.2519999999999998},"last":3.4630000000000001}]},{"osd":7,"last update":"Sat Dec 5 18:12:06 2020","interfaces":[{"interface":"back","average":{"1min":3.4399999999999999,"5min":3.3050000000000002,"15min":3.2639999999999998},"min":{"1min":2.75,"5min":2.5760000000000001,"15min":2.456},"max":{"1min":4,"5min":4.0599999999999996,"15min":4.3300000000000001},"last":3.2639999999999998},{"interface":"front","average":{"1min":3.0209999999999999,"5min":3.0579999999999998,"15min":3.028},"min":{"1min":2.6360000000000001,"5min":1.849,"15min":1.849},"max":{"1min":3.3170000000000002,"5min":3.9990000000000001,"15min":3.9990000000000001},"last":3.0430000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542634,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 5 18:11:49 2020","interfaces":[{"interface":"back","average":{"1min":1.714,"5min":1.849,"15min":1.8149999999999999},"min":{"1min":1.3440000000000001,"5min":1.171,"15min":1.141},"max":{"1min":2.3839999999999999,"5min":3.052,"15min":3.6659999999999999},"last":2.0190000000000001},{"interface":"front","average":{"1min":1.823,"5min":1.8069999999999999,"15min":1.806},"min":{"1min":1.452,"5min":1.3240000000000001,"15min":0.96499999999999997},"max":{"1min":2.246,"5min":3.202,"15min":3.5310000000000001},"last":1.3580000000000001}]},{"osd":1,"last update":"Sat Dec 5 18:11:49 2020","interfaces":[{"interface":"back","average":{"1min":2.1560000000000001,"5min":2.1259999999999999,"15min":2.1219999999999999},"min":{"1min":1.097,"5min":1.097,"15min":1.097},"max":{"1min":2.9049999999999998,"5min":3.0800000000000001,"15min":3.3300000000000001},"last":2.6000000000000001},{"interface":"front","average":{"1min":2.4830000000000001,"5min":2.4849999999999999,"15min":2.52},"min":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.3009999999999999},"max":{"1min":3.1419999999999999,"5min":3.5230000000000001,"15min":3.5510000000000002},"last":2.419}]},{"osd":3,"last update":"Sat Dec 5 18:11:49 2020","interfaces":[{"interface":"back","average":{"1min":1.899,"5min":1.857,"15min":1.8959999999999999},"min":{"1min":1.149,"5min":1.149,"15min":1.149},"max":{"1min":2.6930000000000001,"5min":2.6930000000000001,"15min":3.0470000000000002},"last":2.1930000000000001},{"interface":"front","average":{"1min":2.3849999999999998,"5min":2.403,"15min":2.4390000000000001},"min":{"1min":1.581,"5min":1.54,"15min":1.4530000000000001},"max":{"1min":3.0049999999999999,"5min":3.153,"15min":3.536},"last":1.7749999999999999}]},{"osd":4,"last update":"Sat Dec 5 18:11:49 2020","interfaces":[{"interface":"back","average":{"1min":2.5230000000000001,"5min":2.4489999999999998,"15min":2.3639999999999999},"min":{"1min":1.6990000000000001,"5min":1.502,"15min":1.431},"max":{"1min":3.8170000000000002,"5min":4.6159999999999997,"15min":4.6159999999999997},"last":1.923},{"interface":"front","average":{"1min":2.5430000000000001,"5min":2.4630000000000001,"15min":2.3889999999999998},"min":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"max":{"1min":3.6309999999999998,"5min":3.8410000000000002,"15min":3.9009999999999998},"last":2.6459999999999999}]},{"osd":5,"last update":"Sat Dec 5 18:11:49 2020","interfaces":[{"interface":"back","average":{"1min":2.875,"5min":2.9380000000000002,"15min":2.899},"min":{"1min":1.726,"5min":1.726,"15min":1.498},"max":{"1min":3.4079999999999999,"5min":4.1929999999999996,"15min":4.1929999999999996},"last":2.3380000000000001},{"interface":"front","average":{"1min":3.1549999999999998,"5min":3.0419999999999998,"15min":2.9670000000000001},"min":{"1min":2.4900000000000002,"5min":2.198,"15min":1.7509999999999999},"max":{"1min":3.9529999999999998,"5min":4.3289999999999997,"15min":4.3289999999999997},"last":2.1360000000000001}]},{"osd":6,"last update":"Sat Dec 5 18:11:49 2020","interfaces":[{"interface":"back","average":{"1min":2.9700000000000002,"5min":2.8940000000000001,"15min":2.8519999999999999},"min":{"1min":2.371,"5min":1.9810000000000001,"15min":1.476},"max":{"1min":3.895,"5min":4.476,"15min":4.476},"last":3.0369999999999999},{"interface":"front","average":{"1min":3.173,"5min":3.23,"15min":3.1909999999999998},"min":{"1min":2.331,"5min":2.2810000000000001,"15min":1.6870000000000001},"max":{"1min":3.782,"5min":4.1399999999999997,"15min":4.2439999999999998},"last":2.7989999999999999}]},{"osd":7,"last update":"Sat Dec 5 18:11:49 2020","interfaces":[{"interface":"back","average":{"1min":3.2949999999999999,"5min":3.27,"15min":3.246},"min":{"1min":2.4900000000000002,"5min":2.4900000000000002,"15min":2.177},"max":{"1min":4.0049999999999999,"5min":4.1820000000000004,"15min":4.2709999999999999},"last":2.7229999999999999},{"interface":"front","average":{"1min":3.2629999999999999,"5min":3.1669999999999998,"15min":3.1360000000000001},"min":{"1min":2.7869999999999999,"5min":2.4950000000000001,"15min":1.885},"max":{"1min":3.6669999999999998,"5min":4.0780000000000003,"15min":4.0780000000000003},"last":3.403}]}]},{"osd":3,"up_from":14,"seq":60129542634,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.6319999999999999,"15min":1.722},"min":{"1min":1.3899999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":2.5579999999999998,"5min":2.5579999999999998,"15min":2.7349999999999999},"last":1.909},{"interface":"front","average":{"1min":1.7350000000000001,"5min":1.748,"15min":1.766},"min":{"1min":1.486,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":2.2010000000000001,"5min":2.6920000000000002,"15min":2.8399999999999999},"last":1.8100000000000001}]},{"osd":1,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":1.8959999999999999,"5min":1.75,"15min":1.768},"min":{"1min":1.3839999999999999,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":2.7400000000000002,"5min":2.7400000000000002,"15min":3.5539999999999998},"last":1.704},{"interface":"front","average":{"1min":2.1469999999999998,"5min":2.2549999999999999,"15min":2.242},"min":{"1min":1.7729999999999999,"5min":1.228,"15min":1.228},"max":{"1min":2.5299999999999998,"5min":3.1269999999999998,"15min":3.2370000000000001},"last":2.617}]},{"osd":2,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.4260000000000002,"5min":2.2770000000000001,"15min":2.3170000000000002},"min":{"1min":1.9730000000000001,"5min":1.4179999999999999,"15min":1.236},"max":{"1min":3.1659999999999999,"5min":3.2229999999999999,"15min":3.4470000000000001},"last":2.552},{"interface":"front","average":{"1min":2.4079999999999999,"5min":2.4529999999999998,"15min":2.4620000000000002},"min":{"1min":1.954,"5min":1.7689999999999999,"15min":1.595},"max":{"1min":2.9449999999999998,"5min":3.323,"15min":3.3849999999999998},"last":2.391}]},{"osd":4,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.5089999999999999,"5min":2.5369999999999999,"15min":2.6190000000000002},"min":{"1min":1.829,"5min":1.552,"15min":1.552},"max":{"1min":3.4409999999999998,"5min":3.4409999999999998,"15min":3.8300000000000001},"last":2.3210000000000002},{"interface":"front","average":{"1min":2.2890000000000001,"5min":2.3530000000000002,"15min":2.4159999999999999},"min":{"1min":1.7909999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"max":{"1min":3.0009999999999999,"5min":3.8860000000000001,"15min":4.4039999999999999},"last":2.194}]},{"osd":5,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.9649999999999999,"5min":2.9289999999999998,"15min":2.9590000000000001},"min":{"1min":2.411,"5min":1.9419999999999999,"15min":1.556},"max":{"1min":3.6469999999999998,"5min":4.1420000000000003,"15min":4.1420000000000003},"last":2.9380000000000002},{"interface":"front","average":{"1min":2.964,"5min":2.9729999999999999,"15min":3.0470000000000002},"min":{"1min":2.4079999999999999,"5min":1.716,"15min":1.716},"max":{"1min":3.6469999999999998,"5min":3.9950000000000001,"15min":4.3300000000000001},"last":3.3530000000000002}]},{"osd":6,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.8769999999999998,"5min":2.9039999999999999,"15min":2.919},"min":{"1min":2.2269999999999999,"5min":1.9950000000000001,"15min":1.972},"max":{"1min":3.6589999999999998,"5min":3.8199999999999998,"15min":5.3570000000000002},"last":2.8599999999999999},{"interface":"front","average":{"1min":3.149,"5min":3.181,"15min":3.2120000000000002},"min":{"1min":2.5739999999999998,"5min":2.1400000000000001,"15min":2.1400000000000001},"max":{"1min":3.7839999999999998,"5min":3.8900000000000001,"15min":4.0179999999999998},"last":3.194}]},{"osd":7,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":3.1989999999999998,"5min":3.2389999999999999,"15min":3.2799999999999998},"min":{"1min":2.7570000000000001,"5min":2.1840000000000002,"15min":2.1840000000000002},"max":{"1min":3.9020000000000001,"5min":3.9510000000000001,"15min":4.665},"last":3.153},{"interface":"front","average":{"1min":3.048,"5min":3.1259999999999999,"15min":3.1640000000000001},"min":{"1min":2.4359999999999999,"5min":2.2530000000000001,"15min":2.0209999999999999},"max":{"1min":3.669,"5min":4.0599999999999996,"15min":5.742},"last":3.4470000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542635,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 5 18:11:54 2020","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.5920000000000001,"15min":1.6879999999999999},"min":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"max":{"1min":2.6800000000000002,"5min":3.032,"15min":3.032},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.6970000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"min":{"1min":1.3049999999999999,"5min":1.2490000000000001,"15min":1.0529999999999999},"max":{"1min":2.718,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":1.649}]},{"osd":1,"last update":"Sat Dec 5 18:11:54 2020","interfaces":[{"interface":"back","average":{"1min":2.081,"5min":2.2109999999999999,"15min":2.1589999999999998},"min":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.337},"max":{"1min":2.722,"5min":3.2290000000000001,"15min":3.3919999999999999},"last":2.0670000000000002},{"interface":"front","average":{"1min":2.5329999999999999,"5min":2.6560000000000001,"15min":2.6299999999999999},"min":{"1min":1.694,"5min":1.5049999999999999,"15min":1.5049999999999999},"max":{"1min":3.1859999999999999,"5min":3.6709999999999998,"15min":3.6709999999999998},"last":2.6019999999999999}]},{"osd":2,"last update":"Sat Dec 5 18:11:54 2020","interfaces":[{"interface":"back","average":{"1min":2.0960000000000001,"5min":2.044,"15min":2.0150000000000001},"min":{"1min":1.631,"5min":1.5409999999999999,"15min":1.3380000000000001},"max":{"1min":3.153,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.881},{"interface":"front","average":{"1min":2.4590000000000001,"5min":2.464,"15min":2.419},"min":{"1min":1.667,"5min":1.667,"15min":1.2849999999999999},"max":{"1min":3.129,"5min":3.452,"15min":3.681},"last":2.5270000000000001}]},{"osd":3,"last update":"Sat Dec 5 18:11:54 2020","interfaces":[{"interface":"back","average":{"1min":2.2160000000000002,"5min":2.1389999999999998,"15min":2.1680000000000001},"min":{"1min":1.6659999999999999,"5min":1.212,"15min":1.212},"max":{"1min":2.9870000000000001,"5min":3.085,"15min":3.677},"last":2.198},{"interface":"front","average":{"1min":2.472,"5min":2.4660000000000002,"15min":2.4489999999999998},"min":{"1min":1.71,"5min":1.3380000000000001,"15min":1.3380000000000001},"max":{"1min":3.1110000000000002,"5min":3.4289999999999998,"15min":3.4740000000000002},"last":2.2610000000000001}]},{"osd":5,"last update":"Sat Dec 5 18:11:54 2020","interfaces":[{"interface":"back","average":{"1min":2.7309999999999999,"5min":2.7829999999999999,"15min":2.722},"min":{"1min":1.8879999999999999,"5min":1.8400000000000001,"15min":1.51},"max":{"1min":3.6549999999999998,"5min":3.9729999999999999,"15min":3.9910000000000001},"last":2.891},{"interface":"front","average":{"1min":2.6230000000000002,"5min":2.621,"15min":2.617},"min":{"1min":1.764,"5min":1.764,"15min":1.5660000000000001},"max":{"1min":3.3570000000000002,"5min":3.8130000000000002,"15min":3.8130000000000002},"last":2.6499999999999999}]},{"osd":6,"last update":"Sat Dec 5 18:11:54 2020","interfaces":[{"interface":"back","average":{"1min":2.8290000000000002,"5min":2.8519999999999999,"15min":2.8220000000000001},"min":{"1min":2.0569999999999999,"5min":1.7889999999999999,"15min":1.139},"max":{"1min":3.3439999999999999,"5min":3.7989999999999999,"15min":3.8740000000000001},"last":2.8149999999999999},{"interface":"front","average":{"1min":2.9929999999999999,"5min":3.069,"15min":3.0449999999999999},"min":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"max":{"1min":3.8969999999999998,"5min":4.0430000000000001,"15min":4.3760000000000003},"last":3.1070000000000002}]},{"osd":7,"last update":"Sat Dec 5 18:11:54 2020","interfaces":[{"interface":"back","average":{"1min":3.0329999999999999,"5min":3.044,"15min":3.0259999999999998},"min":{"1min":2.2999999999999998,"5min":2.1640000000000001,"15min":1.0580000000000001},"max":{"1min":3.6659999999999999,"5min":4.1340000000000003,"15min":4.2290000000000001},"last":2.9990000000000001},{"interface":"front","average":{"1min":3.2730000000000001,"5min":3.3079999999999998,"15min":3.2789999999999999},"min":{"1min":2.73,"5min":2.1699999999999999,"15min":1.611},"max":{"1min":3.7570000000000001,"5min":4.1779999999999999,"15min":4.3339999999999996},"last":3.0430000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542634,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.615},"min":{"1min":1.222,"5min":1.1200000000000001,"15min":1.1200000000000001},"max":{"1min":2.3490000000000002,"5min":2.7120000000000002,"15min":2.7570000000000001},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.647,"5min":1.78,"15min":1.7549999999999999},"min":{"1min":1.286,"5min":1.2509999999999999,"15min":1.1639999999999999},"max":{"1min":2.5910000000000002,"5min":3.1859999999999999,"15min":3.1859999999999999},"last":1.7969999999999999}]},{"osd":1,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.3559999999999999,"5min":2.4359999999999999,"15min":2.444},"min":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":0.997},"max":{"1min":3.3250000000000002,"5min":3.4870000000000001,"15min":3.4870000000000001},"last":2.5350000000000001},{"interface":"front","average":{"1min":2.6219999999999999,"5min":2.5249999999999999,"15min":2.625},"min":{"1min":1.528,"5min":1.4159999999999999,"15min":1.292},"max":{"1min":3.3809999999999998,"5min":3.6459999999999999,"15min":4.1139999999999999},"last":3.2120000000000002}]},{"osd":2,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.008,"5min":2.0619999999999998,"15min":2.0419999999999998},"min":{"1min":1.5980000000000001,"5min":1.3109999999999999,"15min":1.3109999999999999},"max":{"1min":3.2189999999999999,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.9379999999999999},{"interface":"front","average":{"1min":2.2730000000000001,"5min":2.2389999999999999,"15min":2.246},"min":{"1min":1.7410000000000001,"5min":1.52,"15min":1.478},"max":{"1min":3.3519999999999999,"5min":3.3519999999999999,"15min":3.468},"last":2.4140000000000001}]},{"osd":3,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.4649999999999999,"5min":2.3919999999999999,"15min":2.3730000000000002},"min":{"1min":1.8899999999999999,"5min":1.4330000000000001,"15min":1.4330000000000001},"max":{"1min":3.1179999999999999,"5min":3.6480000000000001,"15min":3.7810000000000001},"last":2.931},{"interface":"front","average":{"1min":2.5350000000000001,"5min":2.4689999999999999,"15min":2.4500000000000002},"min":{"1min":2.0190000000000001,"5min":1.5669999999999999,"15min":1.5669999999999999},"max":{"1min":3.4239999999999999,"5min":3.6629999999999998,"15min":3.7930000000000001},"last":3.536}]},{"osd":4,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.629,"5min":2.544,"15min":2.4990000000000001},"min":{"1min":2.0179999999999998,"5min":1.6240000000000001,"15min":1.6240000000000001},"max":{"1min":3.431,"5min":3.6520000000000001,"15min":3.8340000000000001},"last":2.996},{"interface":"front","average":{"1min":2.8929999999999998,"5min":2.7930000000000001,"15min":2.7989999999999999},"min":{"1min":2.1549999999999998,"5min":1.8839999999999999,"15min":1.647},"max":{"1min":3.4209999999999998,"5min":4.0270000000000001,"15min":4.2789999999999999},"last":3.7970000000000002}]},{"osd":6,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.9390000000000001,"5min":2.9590000000000001,"15min":2.879},"min":{"1min":2.2269999999999999,"5min":1.806,"15min":1.635},"max":{"1min":3.4209999999999998,"5min":3.992,"15min":4.093},"last":3.75},{"interface":"front","average":{"1min":2.9870000000000001,"5min":2.911,"15min":2.867},"min":{"1min":2.3100000000000001,"5min":1.274,"15min":1.274},"max":{"1min":3.609,"5min":3.7770000000000001,"15min":4.2030000000000003},"last":3.3109999999999999}]},{"osd":7,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":3.29,"5min":3.1789999999999998,"15min":3.1469999999999998},"min":{"1min":2.556,"5min":1.754,"15min":1.754},"max":{"1min":3.7850000000000001,"5min":4.3959999999999999,"15min":4.625},"last":3.375},{"interface":"front","average":{"1min":3.4289999999999998,"5min":3.3100000000000001,"15min":3.2959999999999998},"min":{"1min":2.859,"5min":2.226,"15min":2.153},"max":{"1min":4.0949999999999998,"5min":4.1970000000000001,"15min":4.556},"last":3.9510000000000001}]}]}]}} 2020-12-05T18:12:13.139 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-12-05T18:12:13.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.252+0000 7f3040997700 1 Processor -- start 2020-12-05T18:12:13.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.252+0000 7f3040997700 1 -- start start 2020-12-05T18:12:13.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.252+0000 7f3040997700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f303c0b5f00 0x7f303c0b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:13.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.252+0000 7f3040997700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f303c0ad510 0x7f303c0af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:13.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.252+0000 7f3040997700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f303c142ab0 con 0x7f303c066370 2020-12-05T18:12:13.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.252+0000 7f3040997700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f303c0b62e0 con 0x7f303c0ad510 2020-12-05T18:12:13.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.252+0000 7f3040997700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f303c0afbf0 con 0x7f303c0b5f00 2020-12-05T18:12:13.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.252+0000 7f303259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f303c0ad510 0x7f303c0af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:13.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.252+0000 7f303259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f303c0ad510 0x7f303c0af480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41988/0 (socket says 172.21.15.32:41988) 2020-12-05T18:12:13.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.252+0000 7f303259c700 1 -- 172.21.15.32:0/718077471 learned_addr learned my addr 172.21.15.32:0/718077471 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:13.260 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.252+0000 7f303ad9d700 1 --2- 172.21.15.32:0/718077471 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f303c0b5f00 0x7f303c0b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:13.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f303259c700 1 -- 172.21.15.32:0/718077471 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f303c0b5f00 msgr2=0x7f303c0b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T18:12:13.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f303259c700 1 --2- 172.21.15.32:0/718077471 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f303c0b5f00 0x7f303c0b14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:13.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f303259c700 1 -- 172.21.15.32:0/718077471 >> v1:172.21.15.32:6789/0 conn(0x7f303c066370 legacy=0x7f303c142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:13.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f303259c700 1 -- 172.21.15.32:0/718077471 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303c0afe90 con 0x7f303c0ad510 2020-12-05T18:12:13.261 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f303259c700 1 --2- 172.21.15.32:0/718077471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f303c0ad510 0x7f303c0af480 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 rx=0x7f3024002f30 tx=0x7f3024006e80).ready entity=mon.0 client_cookie=49e3cbf06d365769 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:13.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f303a59c700 1 -- 172.21.15.32:0/718077471 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30240097f0 con 0x7f303c0ad510 2020-12-05T18:12:13.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f303a59c700 1 -- 172.21.15.32:0/718077471 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3024009950 con 0x7f303c0ad510 2020-12-05T18:12:13.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f303a59c700 1 -- 172.21.15.32:0/718077471 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3024009ac0 con 0x7f303c0ad510 2020-12-05T18:12:13.262 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f3040997700 1 -- 172.21.15.32:0/718077471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f303c0ad510 msgr2=0x7f303c0af480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:13.263 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f3040997700 1 --2- 172.21.15.32:0/718077471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f303c0ad510 0x7f303c0af480 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 rx=0x7f3024002f30 tx=0x7f3024006e80).stop 2020-12-05T18:12:13.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f3040997700 1 -- 172.21.15.32:0/718077471 shutdown_connections 2020-12-05T18:12:13.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f3040997700 1 --2- 172.21.15.32:0/718077471 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f303c0b5f00 0x7f303c0b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:13.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f3040997700 1 --2- 172.21.15.32:0/718077471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f303c0ad510 0x7f303c0af480 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:13.266 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f3040997700 1 -- 172.21.15.32:0/718077471 >> 172.21.15.32:0/718077471 conn(0x7f303c0604c0 msgr2=0x7f303c05c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:13.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f3040997700 1 -- 172.21.15.32:0/718077471 shutdown_connections 2020-12-05T18:12:13.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f3040997700 1 -- 172.21.15.32:0/718077471 wait complete. 2020-12-05T18:12:13.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f3040997700 1 Processor -- start 2020-12-05T18:12:13.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f3040997700 1 -- start start 2020-12-05T18:12:13.267 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f3040997700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f303c066370 0x7f303c1431e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:13.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f3040997700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f303c0ad510 0x7f303c13b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:13.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f3040997700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f303c0b5f00 0x7f303c13bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:13.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f3040997700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303c13d6f0 con 0x7f303c066370 2020-12-05T18:12:13.268 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f3040997700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303c10f640 con 0x7f303c0b5f00 2020-12-05T18:12:13.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f3040997700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f303c13c010 con 0x7f303c0ad510 2020-12-05T18:12:13.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f303ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f303c066370 0x7f303c1431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:13.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f303ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f303c066370 0x7f303c1431e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41994/0 (socket says 172.21.15.32:41994) 2020-12-05T18:12:13.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.256+0000 7f303ad9d700 1 -- 172.21.15.32:0/1222205238 learned_addr learned my addr 172.21.15.32:0/1222205238 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:13.269 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.260+0000 7f303259c700 1 --2- 172.21.15.32:0/1222205238 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f303c0ad510 0x7f303c13b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:13.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.260+0000 7f303b59e700 1 --2- 172.21.15.32:0/1222205238 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f303c0b5f00 0x7f303c13bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:13.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.260+0000 7f303ad9d700 1 -- 172.21.15.32:0/1222205238 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f303c0ad510 msgr2=0x7f303c13b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:13.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.260+0000 7f303ad9d700 1 --2- 172.21.15.32:0/1222205238 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f303c0ad510 0x7f303c13b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:13.270 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.260+0000 7f303ad9d700 1 -- 172.21.15.32:0/1222205238 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f303c0b5f00 msgr2=0x7f303c13bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:13.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.260+0000 7f303ad9d700 1 --2- 172.21.15.32:0/1222205238 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f303c0b5f00 0x7f303c13bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:13.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.260+0000 7f303ad9d700 1 -- 172.21.15.32:0/1222205238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303c110b00 con 0x7f303c066370 2020-12-05T18:12:13.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.260+0000 7f303b59e700 1 --2- 172.21.15.32:0/1222205238 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f303c0b5f00 0x7f303c13bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T18:12:13.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.260+0000 7f303ad9d700 1 --2- 172.21.15.32:0/1222205238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f303c066370 0x7f303c1431e0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 rx=0x7f302c00c000 tx=0x7f302c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:13.271 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.260+0000 7f3038d99700 1 -- 172.21.15.32:0/1222205238 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f302c00c820 con 0x7f303c066370 2020-12-05T18:12:13.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.260+0000 7f3038d99700 1 -- 172.21.15.32:0/1222205238 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302c00c980 con 0x7f303c066370 2020-12-05T18:12:13.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.260+0000 7f3040997700 1 -- 172.21.15.32:0/1222205238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303c110da0 con 0x7f303c066370 2020-12-05T18:12:13.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.260+0000 7f3038d99700 1 -- 172.21.15.32:0/1222205238 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f302c00cce0 con 0x7f303c066370 2020-12-05T18:12:13.272 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.260+0000 7f3040997700 1 -- 172.21.15.32:0/1222205238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f303c10f330 con 0x7f303c066370 2020-12-05T18:12:13.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.260+0000 7f3040997700 1 -- 172.21.15.32:0/1222205238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f303c110870 con 0x7f303c066370 2020-12-05T18:12:13.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.260+0000 7f3038d99700 1 -- 172.21.15.32:0/1222205238 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f302c022070 con 0x7f303c066370 2020-12-05T18:12:13.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.260+0000 7f3038d99700 1 --2- 172.21.15.32:0/1222205238 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3014055470 0x7f3014057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:13.273 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.260+0000 7f3038d99700 1 -- 172.21.15.32:0/1222205238 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f302c06e730 con 0x7f303c066370 2020-12-05T18:12:13.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.264+0000 7f303259c700 1 --2- 172.21.15.32:0/1222205238 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3014055470 0x7f3014057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:13.274 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.264+0000 7f303259c700 1 --2- 172.21.15.32:0/1222205238 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3014055470 0x7f3014057930 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f302401b000 tx=0x7f3024016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:13.275 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.268+0000 7f3038d99700 1 -- 172.21.15.32:0/1222205238 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f302c044bb0 con 0x7f303c066370 2020-12-05T18:12:13.412 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.408+0000 7f3040997700 1 -- 172.21.15.32:0/1222205238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f303c110f30 con 0x7f303c066370 2020-12-05T18:12:13.413 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.408+0000 7f3038d99700 1 -- 172.21.15.32:0/1222205238 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+13553 (secure 0 0 0) 0x7f302c02c510 con 0x7f303c066370 2020-12-05T18:12:13.414 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T18:12:13.414 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":151,"fsid":"efd7e861-a502-461d-b20d-3b2c0a25fbd9","created":"2020-12-05T17:31:20.975238+0000","modified":"2020-12-05T17:42:12.290182+0000","last_up_change":"2020-12-05T17:31:28.836431+0000","last_in_change":"2020-12-05T17:31:28.836431+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-05T17:31:24.858732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-05T17:31:33.305595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b941127b-0a49-4e06-8d11-17794ff8b013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":11526},{"type":"v1","addr":"172.21.15.32:6803","nonce":11526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":11526},{"type":"v1","addr":"172.21.15.32:6805","nonce":11526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":11526},{"type":"v1","addr":"172.21.15.32:6809","nonce":11526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":11526},{"type":"v1","addr":"172.21.15.32:6807","nonce":11526}]},"public_addr":"172.21.15.32:6803/11526","cluster_addr":"172.21.15.32:6805/11526","heartbeat_back_addr":"172.21.15.32:6809/11526","heartbeat_front_addr":"172.21.15.32:6807/11526","state":["exists","up"]},{"osd":1,"uuid":"01bd7ff5-a86b-4e11-ba1e-484c309c0fba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":11538},{"type":"v1","addr":"172.21.15.32:6819","nonce":11538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":11538},{"type":"v1","addr":"172.21.15.32:6821","nonce":11538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":11538},{"type":"v1","addr":"172.21.15.32:6825","nonce":11538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":11538},{"type":"v1","addr":"172.21.15.32:6823","nonce":11538}]},"public_addr":"172.21.15.32:6819/11538","cluster_addr":"172.21.15.32:6821/11538","heartbeat_back_addr":"172.21.15.32:6825/11538","heartbeat_front_addr":"172.21.15.32:6823/11538","state":["exists","up"]},{"osd":2,"uuid":"94736ef5-64c2-4a97-b963-bd38cddd9c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":11521},{"type":"v1","addr":"172.21.15.32:6811","nonce":11521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":11521},{"type":"v1","addr":"172.21.15.32:6813","nonce":11521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":11521},{"type":"v1","addr":"172.21.15.32:6817","nonce":11521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":11521},{"type":"v1","addr":"172.21.15.32:6815","nonce":11521}]},"public_addr":"172.21.15.32:6811/11521","cluster_addr":"172.21.15.32:6813/11521","heartbeat_back_addr":"172.21.15.32:6817/11521","heartbeat_front_addr":"172.21.15.32:6815/11521","state":["exists","up"]},{"osd":3,"uuid":"69a1f3ff-fd25-43df-b60c-e32a89c35234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":11535},{"type":"v1","addr":"172.21.15.32:6827","nonce":11535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":11535},{"type":"v1","addr":"172.21.15.32:6829","nonce":11535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":11535},{"type":"v1","addr":"172.21.15.32:6833","nonce":11535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":11535},{"type":"v1","addr":"172.21.15.32:6831","nonce":11535}]},"public_addr":"172.21.15.32:6827/11535","cluster_addr":"172.21.15.32:6829/11535","heartbeat_back_addr":"172.21.15.32:6833/11535","heartbeat_front_addr":"172.21.15.32:6831/11535","state":["exists","up"]},{"osd":4,"uuid":"c301fe4e-5eab-402c-ae6f-db3e6a7181af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":11541},{"type":"v1","addr":"172.21.15.61:6809","nonce":11541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":11541},{"type":"v1","addr":"172.21.15.61:6811","nonce":11541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6817","nonce":11541},{"type":"v1","addr":"172.21.15.61:6819","nonce":11541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":11541},{"type":"v1","addr":"172.21.15.61:6814","nonce":11541}]},"public_addr":"172.21.15.61:6809/11541","cluster_addr":"172.21.15.61:6811/11541","heartbeat_back_addr":"172.21.15.61:6819/11541","heartbeat_front_addr":"172.21.15.61:6814/11541","state":["exists","up"]},{"osd":5,"uuid":"9476eff4-345b-46c5-a473-1d9a89a6d327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":11539},{"type":"v1","addr":"172.21.15.61:6801","nonce":11539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":11539},{"type":"v1","addr":"172.21.15.61:6803","nonce":11539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":11539},{"type":"v1","addr":"172.21.15.61:6807","nonce":11539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":11539},{"type":"v1","addr":"172.21.15.61:6805","nonce":11539}]},"public_addr":"172.21.15.61:6801/11539","cluster_addr":"172.21.15.61:6803/11539","heartbeat_back_addr":"172.21.15.61:6807/11539","heartbeat_front_addr":"172.21.15.61:6805/11539","state":["exists","up"]},{"osd":6,"uuid":"8dfd8567-9b1c-4d13-9c96-071183e469c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":11542},{"type":"v1","addr":"172.21.15.61:6822","nonce":11542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6823","nonce":11542},{"type":"v1","addr":"172.21.15.61:6825","nonce":11542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6829","nonce":11542},{"type":"v1","addr":"172.21.15.61:6830","nonce":11542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":11542},{"type":"v1","addr":"172.21.15.61:6827","nonce":11542}]},"public_addr":"172.21.15.61:6822/11542","cluster_addr":"172.21.15.61:6825/11542","heartbeat_back_addr":"172.21.15.61:6830/11542","heartbeat_front_addr":"172.21.15.61:6827/11542","state":["exists","up"]},{"osd":7,"uuid":"24d466b3-01dc-4ba2-8b81-85250d163bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":11540},{"type":"v1","addr":"172.21.15.61:6815","nonce":11540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":11540},{"type":"v1","addr":"172.21.15.61:6818","nonce":11540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":11540},{"type":"v1","addr":"172.21.15.61:6831","nonce":11540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6821","nonce":11540},{"type":"v1","addr":"172.21.15.61:6824","nonce":11540}]},"public_addr":"172.21.15.61:6815/11540","cluster_addr":"172.21.15.61:6818/11540","heartbeat_back_addr":"172.21.15.61:6831/11540","heartbeat_front_addr":"172.21.15.61:6824/11540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.398868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.675748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.620888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.681600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.658769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.630708+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.648352+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-12-05T17:31:27.629399+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.32:6835/1575496107":"2020-12-06T17:42:08.989610+0000","172.21.15.32:6834/1575496107":"2020-12-06T17:42:08.989610+0000"},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-12-05T18:12:13.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.412+0000 7f3040997700 1 -- 172.21.15.32:0/1222205238 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3014055470 msgr2=0x7f3014057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:13.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.412+0000 7f3040997700 1 --2- 172.21.15.32:0/1222205238 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3014055470 0x7f3014057930 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f302401b000 tx=0x7f3024016000).stop 2020-12-05T18:12:13.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.412+0000 7f3040997700 1 -- 172.21.15.32:0/1222205238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f303c066370 msgr2=0x7f303c1431e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:13.417 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.412+0000 7f3040997700 1 --2- 172.21.15.32:0/1222205238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f303c066370 0x7f303c1431e0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 rx=0x7f302c00c000 tx=0x7f302c008040).stop 2020-12-05T18:12:13.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.412+0000 7f3040997700 1 -- 172.21.15.32:0/1222205238 shutdown_connections 2020-12-05T18:12:13.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.412+0000 7f3040997700 1 --2- 172.21.15.32:0/1222205238 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f303c0b5f00 0x7f303c13bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:13.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.412+0000 7f3040997700 1 --2- 172.21.15.32:0/1222205238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f303c066370 0x7f303c1431e0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:13.418 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.412+0000 7f3040997700 1 --2- 172.21.15.32:0/1222205238 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3014055470 0x7f3014057930 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:13.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.412+0000 7f3040997700 1 --2- 172.21.15.32:0/1222205238 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f303c0ad510 0x7f303c13b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:13.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.412+0000 7f3040997700 1 -- 172.21.15.32:0/1222205238 >> 172.21.15.32:0/1222205238 conn(0x7f303c0604c0 msgr2=0x7f303c13e210 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:13.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.412+0000 7f3040997700 1 -- 172.21.15.32:0/1222205238 shutdown_connections 2020-12-05T18:12:13.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:13.412+0000 7f3040997700 1 -- 172.21.15.32:0/1222205238 wait complete. 2020-12-05T18:12:14.434 INFO:tasks.ceph:Scrubbing osd.0 2020-12-05T18:12:14.435 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.0 config set osd_debug_deep_scrub_sleep 0 2020-12-05T18:12:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.560+0000 7fa98f59e700 1 Processor -- start 2020-12-05T18:12:14.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.560+0000 7fa98f59e700 1 -- start start 2020-12-05T18:12:14.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.560+0000 7fa98f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa99005f3b0 0x7fa99005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:14.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.560+0000 7fa98f59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9900b2df0 con 0x7fa9900b2fb0 2020-12-05T18:12:14.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.560+0000 7fa98f59e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9900b3c90 con 0x7fa99005feb0 2020-12-05T18:12:14.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.560+0000 7fa98f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9900b2b50 con 0x7fa99005f3b0 2020-12-05T18:12:14.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.560+0000 7fa98ed9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa9900b2fb0 0x7fa990142cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44654/0 (socket says 172.21.15.32:44654) 2020-12-05T18:12:14.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.560+0000 7fa98ed9d700 1 -- 172.21.15.32:0/44443105 learned_addr learned my addr 172.21.15.32:0/44443105 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T18:12:14.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.560+0000 7fa98e59c700 1 --2- 172.21.15.32:0/44443105 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa99005f3b0 0x7fa99005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:14.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.564+0000 7fa98dd9b700 1 -- 172.21.15.32:0/44443105 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4036770804 0 0) 0x7fa9900b2df0 con 0x7fa9900b2fb0 2020-12-05T18:12:14.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.564+0000 7fa98dd9b700 1 -- 172.21.15.32:0/44443105 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa970003230 con 0x7fa9900b2fb0 2020-12-05T18:12:14.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.564+0000 7fa98dd9b700 1 -- 172.21.15.32:0/44443105 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3442698561 0 0) 0x7fa9900b3c90 con 0x7fa99005feb0 2020-12-05T18:12:14.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.564+0000 7fa98dd9b700 1 -- 172.21.15.32:0/44443105 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9900b2df0 con 0x7fa99005feb0 2020-12-05T18:12:14.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.564+0000 7fa98e59c700 1 -- 172.21.15.32:0/44443105 >> v1:172.21.15.61:6789/0 conn(0x7fa99005feb0 legacy=0x7fa99013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:14.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.564+0000 7fa98e59c700 1 -- 172.21.15.32:0/44443105 >> v1:172.21.15.32:6789/0 conn(0x7fa9900b2fb0 legacy=0x7fa990142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:14.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.564+0000 7fa98e59c700 1 -- 172.21.15.32:0/44443105 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa990142820 con 0x7fa99005f3b0 2020-12-05T18:12:14.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.564+0000 7fa98e59c700 1 --2- 172.21.15.32:0/44443105 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa99005f3b0 0x7fa99005f790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fa980002f30 tx=0x7fa980006dc0).ready entity=mon.2 client_cookie=87c96d18530f27c3 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:14.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.564+0000 7fa98dd9b700 1 -- 172.21.15.32:0/44443105 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa980009730 con 0x7fa99005f3b0 2020-12-05T18:12:14.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.564+0000 7fa98dd9b700 1 -- 172.21.15.32:0/44443105 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa980009890 con 0x7fa99005f3b0 2020-12-05T18:12:14.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.564+0000 7fa98dd9b700 1 -- 172.21.15.32:0/44443105 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa980009bf0 con 0x7fa99005f3b0 2020-12-05T18:12:14.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.564+0000 7fa98f59e700 1 -- 172.21.15.32:0/44443105 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa99005f3b0 msgr2=0x7fa99005f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:14.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.564+0000 7fa98f59e700 1 --2- 172.21.15.32:0/44443105 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa99005f3b0 0x7fa99005f790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fa980002f30 tx=0x7fa980006dc0).stop 2020-12-05T18:12:14.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.564+0000 7fa98f59e700 1 -- 172.21.15.32:0/44443105 shutdown_connections 2020-12-05T18:12:14.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.564+0000 7fa98f59e700 1 --2- 172.21.15.32:0/44443105 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa99005f3b0 0x7fa99005f790 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:14.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.564+0000 7fa98f59e700 1 -- 172.21.15.32:0/44443105 >> 172.21.15.32:0/44443105 conn(0x7fa9900b8f20 msgr2=0x7fa9900b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:14.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.564+0000 7fa98f59e700 1 -- 172.21.15.32:0/44443105 shutdown_connections 2020-12-05T18:12:14.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.564+0000 7fa98f59e700 1 -- 172.21.15.32:0/44443105 wait complete. 2020-12-05T18:12:14.574 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.568+0000 7fa98f59e700 1 Processor -- start 2020-12-05T18:12:14.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.568+0000 7fa98f59e700 1 -- start start 2020-12-05T18:12:14.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.568+0000 7fa98f59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa99005f3b0 0x7fa990144260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:14.575 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.568+0000 7fa98f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa99005feb0 0x7fa9901453b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:14.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.568+0000 7fa98f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9900b2fb0 0x7fa9901458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:14.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.568+0000 7fa98f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99013cdb0 con 0x7fa9900b2fb0 2020-12-05T18:12:14.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.568+0000 7fa98f59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99013cfe0 con 0x7fa99005f3b0 2020-12-05T18:12:14.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.568+0000 7fa98f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99013d280 con 0x7fa99005feb0 2020-12-05T18:12:14.576 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.568+0000 7fa98ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9900b2fb0 0x7fa9901458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:14.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.568+0000 7fa98ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9900b2fb0 0x7fa9901458a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42012/0 (socket says 172.21.15.32:42012) 2020-12-05T18:12:14.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.568+0000 7fa98e59c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa99005f3b0 0x7fa990144260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:14.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.568+0000 7fa98ed9d700 1 -- 172.21.15.32:0/299017256 learned_addr learned my addr 172.21.15.32:0/299017256 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:14.577 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.568+0000 7fa985d9b700 1 --2- 172.21.15.32:0/299017256 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa99005feb0 0x7fa9901453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:14.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.572+0000 7fa98ed9d700 1 -- 172.21.15.32:0/299017256 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa99005feb0 msgr2=0x7fa9901453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:14.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.572+0000 7fa98ed9d700 1 --2- 172.21.15.32:0/299017256 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa99005feb0 0x7fa9901453b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:14.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.572+0000 7fa98ed9d700 1 -- 172.21.15.32:0/299017256 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa99005f3b0 msgr2=0x7fa990144260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:14.578 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.572+0000 7fa98ed9d700 1 --2- 172.21.15.32:0/299017256 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa99005f3b0 0x7fa990144260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:14.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.572+0000 7fa98ed9d700 1 -- 172.21.15.32:0/299017256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99013d520 con 0x7fa9900b2fb0 2020-12-05T18:12:14.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.572+0000 7fa98ed9d700 1 --2- 172.21.15.32:0/299017256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9900b2fb0 0x7fa9901458a0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 rx=0x7fa988001450 tx=0x7fa988007db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:14.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.572+0000 7fa987fff700 1 -- 172.21.15.32:0/299017256 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa988005960 con 0x7fa9900b2fb0 2020-12-05T18:12:14.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.572+0000 7fa987fff700 1 -- 172.21.15.32:0/299017256 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa988005ac0 con 0x7fa9900b2fb0 2020-12-05T18:12:14.579 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.572+0000 7fa98f59e700 1 -- 172.21.15.32:0/299017256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa990151380 con 0x7fa9900b2fb0 2020-12-05T18:12:14.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.572+0000 7fa987fff700 1 -- 172.21.15.32:0/299017256 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa988005e20 con 0x7fa9900b2fb0 2020-12-05T18:12:14.580 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.572+0000 7fa98f59e700 1 -- 172.21.15.32:0/299017256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa990145f30 con 0x7fa9900b2fb0 2020-12-05T18:12:14.581 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.572+0000 7fa98f59e700 1 -- 172.21.15.32:0/299017256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa990146ce0 con 0x7fa9900b2fb0 2020-12-05T18:12:14.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.572+0000 7fa987fff700 1 -- 172.21.15.32:0/299017256 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa988010030 con 0x7fa9900b2fb0 2020-12-05T18:12:14.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.572+0000 7fa987fff700 1 --2- 172.21.15.32:0/299017256 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa9740553b0 0x7fa974057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:14.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.572+0000 7fa987fff700 1 -- 172.21.15.32:0/299017256 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fa98806dc80 con 0x7fa9900b2fb0 2020-12-05T18:12:14.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.572+0000 7fa98e59c700 1 --2- 172.21.15.32:0/299017256 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa9740553b0 0x7fa974057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:14.582 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.572+0000 7fa987fff700 1 --2- 172.21.15.32:0/299017256 >> [v2:172.21.15.32:6802/11526,v1:172.21.15.32:6803/11526] conn(0x7fa97405c0d0 0x7fa97405e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:14.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.572+0000 7fa987fff700 1 -- 172.21.15.32:0/299017256 --> [v2:172.21.15.32:6802/11526,v1:172.21.15.32:6803/11526] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa97405eb70 con 0x7fa97405c0d0 2020-12-05T18:12:14.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.572+0000 7fa987fff700 1 -- 172.21.15.32:0/299017256 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa988070310 con 0x7fa9900b2fb0 2020-12-05T18:12:14.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.572+0000 7fa985d9b700 1 --2- 172.21.15.32:0/299017256 >> [v2:172.21.15.32:6802/11526,v1:172.21.15.32:6803/11526] conn(0x7fa97405c0d0 0x7fa97405e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:14.583 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.576+0000 7fa98e59c700 1 --2- 172.21.15.32:0/299017256 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa9740553b0 0x7fa974057870 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7fa980019000 tx=0x7fa980015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:14.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.576+0000 7fa985d9b700 1 --2- 172.21.15.32:0/299017256 >> [v2:172.21.15.32:6802/11526,v1:172.21.15.32:6803/11526] conn(0x7fa97405c0d0 0x7fa97405e4f0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:14.584 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.576+0000 7fa987fff700 1 -- 172.21.15.32:0/299017256 <== osd.0 v2:172.21.15.32:6802/11526 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7fa97405eb70 con 0x7fa97405c0d0 2020-12-05T18:12:14.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.596+0000 7fa98f59e700 1 -- 172.21.15.32:0/299017256 --> [v2:172.21.15.32:6802/11526,v1:172.21.15.32:6803/11526] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa9901474a0 con 0x7fa97405c0d0 2020-12-05T18:12:14.600 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.596+0000 7fa987fff700 1 -- 172.21.15.32:0/299017256 <== osd.0 v2:172.21.15.32:6802/11526 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa9780038a0 con 0x7fa97405c0d0 2020-12-05T18:12:14.600 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T18:12:14.601 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-12-05T18:12:14.601 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T18:12:14.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.596+0000 7fa98f59e700 1 -- 172.21.15.32:0/299017256 >> [v2:172.21.15.32:6802/11526,v1:172.21.15.32:6803/11526] conn(0x7fa97405c0d0 msgr2=0x7fa97405e4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:14.602 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.596+0000 7fa98f59e700 1 --2- 172.21.15.32:0/299017256 >> [v2:172.21.15.32:6802/11526,v1:172.21.15.32:6803/11526] conn(0x7fa97405c0d0 0x7fa97405e4f0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:14.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.596+0000 7fa98f59e700 1 -- 172.21.15.32:0/299017256 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa9740553b0 msgr2=0x7fa974057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:14.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.596+0000 7fa98f59e700 1 --2- 172.21.15.32:0/299017256 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa9740553b0 0x7fa974057870 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7fa980019000 tx=0x7fa980015000).stop 2020-12-05T18:12:14.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.596+0000 7fa98f59e700 1 -- 172.21.15.32:0/299017256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9900b2fb0 msgr2=0x7fa9901458a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:14.603 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.596+0000 7fa98f59e700 1 --2- 172.21.15.32:0/299017256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9900b2fb0 0x7fa9901458a0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 rx=0x7fa988001450 tx=0x7fa988007db0).stop 2020-12-05T18:12:14.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.596+0000 7fa98ed9d700 1 -- 172.21.15.32:0/299017256 reap_dead start 2020-12-05T18:12:14.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.596+0000 7fa98f59e700 1 -- 172.21.15.32:0/299017256 shutdown_connections 2020-12-05T18:12:14.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.596+0000 7fa98f59e700 1 -- 172.21.15.32:0/299017256 >> 172.21.15.32:0/299017256 conn(0x7fa9900b8f20 msgr2=0x7fa9900b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:14.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.596+0000 7fa98f59e700 1 -- 172.21.15.32:0/299017256 shutdown_connections 2020-12-05T18:12:14.604 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.596+0000 7fa98f59e700 1 -- 172.21.15.32:0/299017256 wait complete. 2020-12-05T18:12:14.614 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 0 2020-12-05T18:12:14.729 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.724+0000 7f42a5c1c700 1 Processor -- start 2020-12-05T18:12:14.730 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.724+0000 7f42a5c1c700 1 -- start start 2020-12-05T18:12:14.731 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f42a5c1c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f42a00ae750 0x7f42a0145b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:14.731 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f42a5c1c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f42a00b1880 0x7f42a00b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:14.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f42a5c1c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a0061b30 con 0x7f42a00b7580 2020-12-05T18:12:14.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f42a5c1c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a00b21b0 con 0x7f42a00b1880 2020-12-05T18:12:14.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f42a5c1c700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a0146050 con 0x7f42a00ae750 2020-12-05T18:12:14.732 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f42a541b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f42a00b1880 0x7f42a00b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:14.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f42a4c1a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f42a00b7580 0x7f42a013f7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44666/0 (socket says 172.21.15.32:44666) 2020-12-05T18:12:14.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f42a541b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f42a00b1880 0x7f42a00b1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42022/0 (socket says 172.21.15.32:42022) 2020-12-05T18:12:14.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f42a541b700 1 -- 172.21.15.32:0/1297001852 learned_addr learned my addr 172.21.15.32:0/1297001852 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:14.733 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f429ffff700 1 --2- 172.21.15.32:0/1297001852 >> v2:172.21.15.61:3300/0 conn(0x7f42a00ae750 0x7f42a0145b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:14.735 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f42a541b700 1 -- 172.21.15.32:0/1297001852 >> v2:172.21.15.61:3300/0 conn(0x7f42a00ae750 msgr2=0x7f42a0145b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T18:12:14.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f42a541b700 1 --2- 172.21.15.32:0/1297001852 >> v2:172.21.15.61:3300/0 conn(0x7f42a00ae750 0x7f42a0145b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:14.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f42a541b700 1 -- 172.21.15.32:0/1297001852 >> v1:172.21.15.32:6789/0 conn(0x7f42a00b7580 legacy=0x7f42a013f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:14.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f429f7fe700 1 -- 172.21.15.32:0/1297001852 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1941813582 0 0) 0x7f42a0061b30 con 0x7f42a00b7580 2020-12-05T18:12:14.736 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f42a541b700 1 -- 172.21.15.32:0/1297001852 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a0061d70 con 0x7f42a00b1880 2020-12-05T18:12:14.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f42a541b700 1 --2- 172.21.15.32:0/1297001852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a00b1880 0x7f42a00b1cc0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 rx=0x7f42980032b0 tx=0x7f4298009270).ready entity=mon.0 client_cookie=fc1d704d70fdf3a5 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:14.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f429f7fe700 1 -- 172.21.15.32:0/1297001852 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4298009e60 con 0x7f42a00b1880 2020-12-05T18:12:14.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f429f7fe700 1 -- 172.21.15.32:0/1297001852 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429800bc00 con 0x7f42a00b1880 2020-12-05T18:12:14.737 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f429f7fe700 1 -- 172.21.15.32:0/1297001852 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f429800bd70 con 0x7f42a00b1880 2020-12-05T18:12:14.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f42a5c1c700 1 -- 172.21.15.32:0/1297001852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a00b1880 msgr2=0x7f42a00b1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:14.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f42a5c1c700 1 --2- 172.21.15.32:0/1297001852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a00b1880 0x7f42a00b1cc0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 rx=0x7f42980032b0 tx=0x7f4298009270).stop 2020-12-05T18:12:14.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f42a5c1c700 1 -- 172.21.15.32:0/1297001852 shutdown_connections 2020-12-05T18:12:14.738 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f42a5c1c700 1 --2- 172.21.15.32:0/1297001852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a00b1880 0x7f42a00b1cc0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:14.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f42a5c1c700 1 --2- 172.21.15.32:0/1297001852 >> v2:172.21.15.61:3300/0 conn(0x7f42a00ae750 0x7f42a0145b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:14.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f42a5c1c700 1 -- 172.21.15.32:0/1297001852 >> 172.21.15.32:0/1297001852 conn(0x7f42a005bc20 msgr2=0x7f42a00b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:14.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.728+0000 7f42a5c1c700 1 -- 172.21.15.32:0/1297001852 shutdown_connections 2020-12-05T18:12:14.739 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.732+0000 7f42a5c1c700 1 -- 172.21.15.32:0/1297001852 wait complete. 2020-12-05T18:12:14.740 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.732+0000 7f42a5c1c700 1 Processor -- start 2020-12-05T18:12:14.740 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.732+0000 7f42a5c1c700 1 -- start start 2020-12-05T18:12:14.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.732+0000 7f42a5c1c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42a00ae750 0x7f42a0066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:14.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.732+0000 7f42a5c1c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42a00b1880 0x7f42a01512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:14.743 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.732+0000 7f42a5c1c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a00b7580 0x7f42a0151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:14.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.732+0000 7f42a5c1c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a0063040 con 0x7f42a00b7580 2020-12-05T18:12:14.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.732+0000 7f42a5c1c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a0149a30 con 0x7f42a00ae750 2020-12-05T18:12:14.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.732+0000 7f42a5c1c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a0064540 con 0x7f42a00b1880 2020-12-05T18:12:14.744 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.732+0000 7f42a541b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a00b7580 0x7f42a0151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:14.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.732+0000 7f42a541b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a00b7580 0x7f42a0151790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42026/0 (socket says 172.21.15.32:42026) 2020-12-05T18:12:14.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.732+0000 7f42a541b700 1 -- 172.21.15.32:0/2711688812 learned_addr learned my addr 172.21.15.32:0/2711688812 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:14.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.732+0000 7f429ffff700 1 --2- 172.21.15.32:0/2711688812 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42a00b1880 0x7f42a01512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:14.745 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.732+0000 7f42a4c1a700 1 --2- 172.21.15.32:0/2711688812 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42a00ae750 0x7f42a0066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:14.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.732+0000 7f42a541b700 1 -- 172.21.15.32:0/2711688812 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42a00b1880 msgr2=0x7f42a01512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:14.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.732+0000 7f42a541b700 1 --2- 172.21.15.32:0/2711688812 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42a00b1880 0x7f42a01512a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:14.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.732+0000 7f42a541b700 1 -- 172.21.15.32:0/2711688812 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42a00ae750 msgr2=0x7f42a0066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:14.746 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.732+0000 7f42a541b700 1 --2- 172.21.15.32:0/2711688812 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42a00ae750 0x7f42a0066ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:14.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.732+0000 7f42a541b700 1 -- 172.21.15.32:0/2711688812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a0063230 con 0x7f42a00b7580 2020-12-05T18:12:14.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.732+0000 7f42a4c1a700 1 --2- 172.21.15.32:0/2711688812 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42a00ae750 0x7f42a0066ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T18:12:14.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.736+0000 7f42a541b700 1 --2- 172.21.15.32:0/2711688812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a00b7580 0x7f42a0151790 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 rx=0x7f42980032b0 tx=0x7f4298016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:14.747 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.736+0000 7f429dffb700 1 -- 172.21.15.32:0/2711688812 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42980107a0 con 0x7f42a00b7580 2020-12-05T18:12:14.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.736+0000 7f429dffb700 1 -- 172.21.15.32:0/2711688812 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4298010900 con 0x7f42a00b7580 2020-12-05T18:12:14.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.736+0000 7f42a5c1c700 1 -- 172.21.15.32:0/2711688812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a00636d0 con 0x7f42a00b7580 2020-12-05T18:12:14.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.736+0000 7f429dffb700 1 -- 172.21.15.32:0/2711688812 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4298010c60 con 0x7f42a00b7580 2020-12-05T18:12:14.748 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.736+0000 7f42a5c1c700 1 -- 172.21.15.32:0/2711688812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a00642b0 con 0x7f42a00b7580 2020-12-05T18:12:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.736+0000 7f42a5c1c700 1 -- 172.21.15.32:0/2711688812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a0064700 con 0x7f42a00b7580 2020-12-05T18:12:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.736+0000 7f429dffb700 1 -- 172.21.15.32:0/2711688812 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f429800f070 con 0x7f42a00b7580 2020-12-05T18:12:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.736+0000 7f429dffb700 1 --2- 172.21.15.32:0/2711688812 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4288055440 0x7f4288057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.736+0000 7f429dffb700 1 -- 172.21.15.32:0/2711688812 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f429806f520 con 0x7f42a00b7580 2020-12-05T18:12:14.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.736+0000 7f42a4c1a700 1 --2- 172.21.15.32:0/2711688812 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4288055440 0x7f4288057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:14.750 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.740+0000 7f42a4c1a700 1 --2- 172.21.15.32:0/2711688812 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4288055440 0x7f4288057900 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7f428c00b000 tx=0x7f428c0070e0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:14.751 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.744+0000 7f429dffb700 1 -- 172.21.15.32:0/2711688812 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f42980459a0 con 0x7f42a00b7580 2020-12-05T18:12:14.888 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.884+0000 7f42a5c1c700 1 -- 172.21.15.32:0/2711688812 --> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f42a0114610 con 0x7f4288055440 2020-12-05T18:12:14.889 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.884+0000 7f429dffb700 1 -- 172.21.15.32:0/2711688812 <== mgr.4101 v2:172.21.15.32:6800/11355 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f42a0114610 con 0x7f4288055440 2020-12-05T18:12:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.888+0000 7f42a5c1c700 1 -- 172.21.15.32:0/2711688812 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4288055440 msgr2=0x7f4288057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.888+0000 7f42a5c1c700 1 --2- 172.21.15.32:0/2711688812 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4288055440 0x7f4288057900 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7f428c00b000 tx=0x7f428c0070e0).stop 2020-12-05T18:12:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.888+0000 7f42a5c1c700 1 -- 172.21.15.32:0/2711688812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a00b7580 msgr2=0x7f42a0151790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.888+0000 7f42a5c1c700 1 --2- 172.21.15.32:0/2711688812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a00b7580 0x7f42a0151790 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 rx=0x7f42980032b0 tx=0x7f4298016000).stop 2020-12-05T18:12:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.888+0000 7f42a5c1c700 1 -- 172.21.15.32:0/2711688812 shutdown_connections 2020-12-05T18:12:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.888+0000 7f42a5c1c700 1 --2- 172.21.15.32:0/2711688812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a00b7580 0x7f42a0151790 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.888+0000 7f42a5c1c700 1 --2- 172.21.15.32:0/2711688812 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42a00ae750 0x7f42a0066ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:14.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.888+0000 7f42a5c1c700 1 --2- 172.21.15.32:0/2711688812 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f4288055440 0x7f4288057900 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:14.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.888+0000 7f42a5c1c700 1 --2- 172.21.15.32:0/2711688812 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42a00b1880 0x7f42a01512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:14.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.888+0000 7f42a5c1c700 1 -- 172.21.15.32:0/2711688812 >> 172.21.15.32:0/2711688812 conn(0x7f42a005bc20 msgr2=0x7f42a01458b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:14.893 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.888+0000 7f42a5c1c700 1 -- 172.21.15.32:0/2711688812 shutdown_connections 2020-12-05T18:12:14.894 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:14.888+0000 7f42a5c1c700 1 -- 172.21.15.32:0/2711688812 wait complete. 2020-12-05T18:12:14.894 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 0 to deep-scrub 2020-12-05T18:12:14.903 INFO:tasks.ceph:Scrubbing osd.1 2020-12-05T18:12:14.903 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.1 config set osd_debug_deep_scrub_sleep 0 2020-12-05T18:12:15.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.024+0000 7fb066d59700 1 Processor -- start 2020-12-05T18:12:15.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.024+0000 7fb066d59700 1 -- start start 2020-12-05T18:12:15.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.024+0000 7fb066d59700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb0600b5b20 0x7fb0600b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.024+0000 7fb066d59700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0600b6690 0x7fb0600b5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.024+0000 7fb066d59700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0600b1920 con 0x7fb0600b2060 2020-12-05T18:12:15.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.024+0000 7fb066d59700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0600b5730 con 0x7fb0600b6690 2020-12-05T18:12:15.031 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.024+0000 7fb066d59700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0600b1bc0 con 0x7fb0600b5b20 2020-12-05T18:12:15.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.028+0000 7fb05ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0600b6690 0x7fb0600b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.028+0000 7fb05ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0600b6690 0x7fb0600b5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42032/0 (socket says 172.21.15.32:42032) 2020-12-05T18:12:15.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.028+0000 7fb05ffff700 1 -- 172.21.15.32:0/1470327643 learned_addr learned my addr 172.21.15.32:0/1470327643 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:15.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.028+0000 7fb064af5700 1 --2- 172.21.15.32:0/1470327643 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb0600b5b20 0x7fb0600b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.032 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.028+0000 7fb05ffff700 1 -- 172.21.15.32:0/1470327643 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb0600b5b20 msgr2=0x7fb0600b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T18:12:15.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.028+0000 7fb05ffff700 1 --2- 172.21.15.32:0/1470327643 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb0600b5b20 0x7fb0600b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.028+0000 7fb05ffff700 1 -- 172.21.15.32:0/1470327643 >> v1:172.21.15.61:6789/0 conn(0x7fb0600b2060 legacy=0x7fb060145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.028+0000 7fb05ffff700 1 -- 172.21.15.32:0/1470327643 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0600b24c0 con 0x7fb0600b6690 2020-12-05T18:12:15.033 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.028+0000 7fb05ffff700 1 --2- 172.21.15.32:0/1470327643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0600b6690 0x7fb0600b5240 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 rx=0x7fb050002f30 tx=0x7fb050006e80).ready entity=mon.0 client_cookie=8ef923c75155cc5e server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:15.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.028+0000 7fb05f7fe700 1 -- 172.21.15.32:0/1470327643 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0500097f0 con 0x7fb0600b6690 2020-12-05T18:12:15.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.028+0000 7fb05f7fe700 1 -- 172.21.15.32:0/1470327643 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb050009950 con 0x7fb0600b6690 2020-12-05T18:12:15.036 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.028+0000 7fb05f7fe700 1 -- 172.21.15.32:0/1470327643 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb050009cb0 con 0x7fb0600b6690 2020-12-05T18:12:15.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.028+0000 7fb066d59700 1 -- 172.21.15.32:0/1470327643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0600b6690 msgr2=0x7fb0600b5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.028+0000 7fb066d59700 1 --2- 172.21.15.32:0/1470327643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0600b6690 0x7fb0600b5240 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 rx=0x7fb050002f30 tx=0x7fb050006e80).stop 2020-12-05T18:12:15.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.028+0000 7fb066d59700 1 -- 172.21.15.32:0/1470327643 shutdown_connections 2020-12-05T18:12:15.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.028+0000 7fb066d59700 1 --2- 172.21.15.32:0/1470327643 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb0600b5b20 0x7fb0600b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.037 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.028+0000 7fb066d59700 1 --2- 172.21.15.32:0/1470327643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0600b6690 0x7fb0600b5240 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.028+0000 7fb066d59700 1 -- 172.21.15.32:0/1470327643 >> 172.21.15.32:0/1470327643 conn(0x7fb06005f3b0 msgr2=0x7fb06005f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:15.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.028+0000 7fb066d59700 1 -- 172.21.15.32:0/1470327643 shutdown_connections 2020-12-05T18:12:15.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.028+0000 7fb066d59700 1 -- 172.21.15.32:0/1470327643 wait complete. 2020-12-05T18:12:15.038 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.028+0000 7fb066d59700 1 Processor -- start 2020-12-05T18:12:15.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb066d59700 1 -- start start 2020-12-05T18:12:15.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb066d59700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb0600b2060 0x7fb0600a2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb066d59700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb0600b5b20 0x7fb060152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb066d59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0600b6690 0x7fb060145dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.039 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb066d59700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0601518e0 con 0x7fb0600b6690 2020-12-05T18:12:15.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb066d59700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb060151b20 con 0x7fb0600b5b20 2020-12-05T18:12:15.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb066d59700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb060151dc0 con 0x7fb0600b2060 2020-12-05T18:12:15.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb0652f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0600b6690 0x7fb060145dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.040 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb0652f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0600b6690 0x7fb060145dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42042/0 (socket says 172.21.15.32:42042) 2020-12-05T18:12:15.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb0652f6700 1 -- 172.21.15.32:0/2334741999 learned_addr learned my addr 172.21.15.32:0/2334741999 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:15.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb064af5700 1 --2- 172.21.15.32:0/2334741999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb0600b2060 0x7fb0600a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb05ffff700 1 --2- 172.21.15.32:0/2334741999 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb0600b5b20 0x7fb060152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.041 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb0652f6700 1 -- 172.21.15.32:0/2334741999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb0600b2060 msgr2=0x7fb0600a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb0652f6700 1 --2- 172.21.15.32:0/2334741999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb0600b2060 0x7fb0600a2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb0652f6700 1 -- 172.21.15.32:0/2334741999 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb0600b5b20 msgr2=0x7fb060152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb0652f6700 1 --2- 172.21.15.32:0/2334741999 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb0600b5b20 0x7fb060152260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb0652f6700 1 -- 172.21.15.32:0/2334741999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb060144410 con 0x7fb0600b6690 2020-12-05T18:12:15.042 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb05ffff700 1 --2- 172.21.15.32:0/2334741999 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb0600b5b20 0x7fb060152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T18:12:15.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb0652f6700 1 --2- 172.21.15.32:0/2334741999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0600b6690 0x7fb060145dd0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7fb0600b1f40 tx=0x7fb058007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:15.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb05dffb700 1 -- 172.21.15.32:0/2334741999 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb05800a7b0 con 0x7fb0600b6690 2020-12-05T18:12:15.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb05dffb700 1 -- 172.21.15.32:0/2334741999 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05800a910 con 0x7fb0600b6690 2020-12-05T18:12:15.043 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb066d59700 1 -- 172.21.15.32:0/2334741999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0601489d0 con 0x7fb0600b6690 2020-12-05T18:12:15.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb05dffb700 1 -- 172.21.15.32:0/2334741999 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb05800ac70 con 0x7fb0600b6690 2020-12-05T18:12:15.044 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb066d59700 1 -- 172.21.15.32:0/2334741999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0601528f0 con 0x7fb0600b6690 2020-12-05T18:12:15.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.032+0000 7fb066d59700 1 -- 172.21.15.32:0/2334741999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb060151f80 con 0x7fb0600b6690 2020-12-05T18:12:15.045 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.036+0000 7fb05dffb700 1 -- 172.21.15.32:0/2334741999 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb058019030 con 0x7fb0600b6690 2020-12-05T18:12:15.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.036+0000 7fb05dffb700 1 --2- 172.21.15.32:0/2334741999 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb044055400 0x7fb0440578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.036+0000 7fb05dffb700 1 -- 172.21.15.32:0/2334741999 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fb05806d6b0 con 0x7fb0600b6690 2020-12-05T18:12:15.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.036+0000 7fb064af5700 1 --2- 172.21.15.32:0/2334741999 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb044055400 0x7fb0440578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.046 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.036+0000 7fb05dffb700 1 --2- 172.21.15.32:0/2334741999 >> [v2:172.21.15.32:6818/11538,v1:172.21.15.32:6819/11538] conn(0x7fb04405c1a0 0x7fb04405e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.036+0000 7fb05dffb700 1 -- 172.21.15.32:0/2334741999 --> [v2:172.21.15.32:6818/11538,v1:172.21.15.32:6819/11538] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb04405ec40 con 0x7fb04405c1a0 2020-12-05T18:12:15.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.036+0000 7fb05dffb700 1 -- 172.21.15.32:0/2334741999 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb05806fcc0 con 0x7fb0600b6690 2020-12-05T18:12:15.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.036+0000 7fb064af5700 1 --2- 172.21.15.32:0/2334741999 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb044055400 0x7fb0440578c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fb04800b000 tx=0x7fb048006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:15.047 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.036+0000 7fb05ffff700 1 --2- 172.21.15.32:0/2334741999 >> [v2:172.21.15.32:6818/11538,v1:172.21.15.32:6819/11538] conn(0x7fb04405c1a0 0x7fb04405e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.036+0000 7fb05ffff700 1 --2- 172.21.15.32:0/2334741999 >> [v2:172.21.15.32:6818/11538,v1:172.21.15.32:6819/11538] conn(0x7fb04405c1a0 0x7fb04405e5c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:15.048 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.036+0000 7fb05dffb700 1 -- 172.21.15.32:0/2334741999 <== osd.1 v2:172.21.15.32:6818/11538 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7fb04405ec40 con 0x7fb04405c1a0 2020-12-05T18:12:15.055 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.052+0000 7fb066d59700 1 -- 172.21.15.32:0/2334741999 --> [v2:172.21.15.32:6818/11538,v1:172.21.15.32:6819/11538] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb060146580 con 0x7fb04405c1a0 2020-12-05T18:12:15.056 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.052+0000 7fb05dffb700 1 -- 172.21.15.32:0/2334741999 <== osd.1 v2:172.21.15.32:6818/11538 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb050006910 con 0x7fb04405c1a0 2020-12-05T18:12:15.056 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T18:12:15.056 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-12-05T18:12:15.057 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T18:12:15.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.052+0000 7fb066d59700 1 -- 172.21.15.32:0/2334741999 >> [v2:172.21.15.32:6818/11538,v1:172.21.15.32:6819/11538] conn(0x7fb04405c1a0 msgr2=0x7fb04405e5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.052+0000 7fb066d59700 1 --2- 172.21.15.32:0/2334741999 >> [v2:172.21.15.32:6818/11538,v1:172.21.15.32:6819/11538] conn(0x7fb04405c1a0 0x7fb04405e5c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.052+0000 7fb066d59700 1 -- 172.21.15.32:0/2334741999 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb044055400 msgr2=0x7fb0440578c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.058 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.052+0000 7fb066d59700 1 --2- 172.21.15.32:0/2334741999 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fb044055400 0x7fb0440578c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fb04800b000 tx=0x7fb048006dc0).stop 2020-12-05T18:12:15.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.052+0000 7fb066d59700 1 -- 172.21.15.32:0/2334741999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0600b6690 msgr2=0x7fb060145dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.052+0000 7fb066d59700 1 --2- 172.21.15.32:0/2334741999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0600b6690 0x7fb060145dd0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7fb0600b1f40 tx=0x7fb058007f10).stop 2020-12-05T18:12:15.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.052+0000 7fb0652f6700 1 -- 172.21.15.32:0/2334741999 reap_dead start 2020-12-05T18:12:15.059 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.052+0000 7fb066d59700 1 -- 172.21.15.32:0/2334741999 shutdown_connections 2020-12-05T18:12:15.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.052+0000 7fb066d59700 1 -- 172.21.15.32:0/2334741999 >> 172.21.15.32:0/2334741999 conn(0x7fb06005f3b0 msgr2=0x7fb0600b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:15.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.052+0000 7fb066d59700 1 -- 172.21.15.32:0/2334741999 shutdown_connections 2020-12-05T18:12:15.060 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.052+0000 7fb066d59700 1 -- 172.21.15.32:0/2334741999 wait complete. 2020-12-05T18:12:15.068 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 1 2020-12-05T18:12:15.188 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.184+0000 7fbf92814700 1 Processor -- start 2020-12-05T18:12:15.189 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.184+0000 7fbf92814700 1 -- start start 2020-12-05T18:12:15.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf92814700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbf8c0b52f0 0x7fbf8c0b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf92814700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf8c0b1870 0x7fbf8c138810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf92814700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf8c1420f0 con 0x7fbf8c138d00 2020-12-05T18:12:15.191 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf92814700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf8c0ad4f0 con 0x7fbf8c0b1870 2020-12-05T18:12:15.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf92814700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c0b0380 con 0x7fbf8c0b52f0 2020-12-05T18:12:15.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf91011700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf8c0b1870 0x7fbf8c138810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf91011700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf8c0b1870 0x7fbf8c138810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42050/0 (socket says 172.21.15.32:42050) 2020-12-05T18:12:15.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf91011700 1 -- 172.21.15.32:0/2330767883 learned_addr learned my addr 172.21.15.32:0/2330767883 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:15.192 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf91812700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbf8c0b52f0 0x7fbf8c0b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.193 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf92013700 1 --1- 172.21.15.32:0/2330767883 >> v1:172.21.15.61:6789/0 conn(0x7fbf8c138d00 0x7fbf8c0ae860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.32:57340/0 (socket says 172.21.15.32:57340) 2020-12-05T18:12:15.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf91011700 1 -- 172.21.15.32:0/2330767883 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbf8c0b52f0 msgr2=0x7fbf8c0b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T18:12:15.194 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf91011700 1 --2- 172.21.15.32:0/2330767883 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbf8c0b52f0 0x7fbf8c0b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf91011700 1 -- 172.21.15.32:0/2330767883 >> v1:172.21.15.61:6789/0 conn(0x7fbf8c138d00 legacy=0x7fbf8c0ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf91011700 1 -- 172.21.15.32:0/2330767883 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8c0b0620 con 0x7fbf8c0b1870 2020-12-05T18:12:15.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf90810700 1 -- 172.21.15.32:0/2330767883 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1783559702 0 0) 0x7fbf8c1420f0 con 0x7fbf8c138d00 2020-12-05T18:12:15.195 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf91011700 1 --2- 172.21.15.32:0/2330767883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8c0b1870 0x7fbf8c138810 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 rx=0x7fbf8c0adc80 tx=0x7fbf80006e80).ready entity=mon.0 client_cookie=8fa76f11b43e6e74 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:15.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf90810700 1 -- 172.21.15.32:0/2330767883 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf800097b0 con 0x7fbf8c0b1870 2020-12-05T18:12:15.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf90810700 1 -- 172.21.15.32:0/2330767883 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf80009910 con 0x7fbf8c0b1870 2020-12-05T18:12:15.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf90810700 1 -- 172.21.15.32:0/2330767883 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf80009c70 con 0x7fbf8c0b1870 2020-12-05T18:12:15.196 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf92814700 1 -- 172.21.15.32:0/2330767883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8c0b1870 msgr2=0x7fbf8c138810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf92814700 1 --2- 172.21.15.32:0/2330767883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8c0b1870 0x7fbf8c138810 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 rx=0x7fbf8c0adc80 tx=0x7fbf80006e80).stop 2020-12-05T18:12:15.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf92814700 1 -- 172.21.15.32:0/2330767883 shutdown_connections 2020-12-05T18:12:15.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf92814700 1 --2- 172.21.15.32:0/2330767883 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbf8c0b52f0 0x7fbf8c0b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.197 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf92814700 1 --2- 172.21.15.32:0/2330767883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8c0b1870 0x7fbf8c138810 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf92814700 1 -- 172.21.15.32:0/2330767883 >> 172.21.15.32:0/2330767883 conn(0x7fbf8c05bbb0 msgr2=0x7fbf8c062070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:15.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf92814700 1 -- 172.21.15.32:0/2330767883 shutdown_connections 2020-12-05T18:12:15.198 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.188+0000 7fbf92814700 1 -- 172.21.15.32:0/2330767883 wait complete. 2020-12-05T18:12:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.192+0000 7fbf92814700 1 Processor -- start 2020-12-05T18:12:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.192+0000 7fbf92814700 1 -- start start 2020-12-05T18:12:15.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.196+0000 7fbf92814700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbf8c0b1870 0x7fbf8c1395a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.196+0000 7fbf92814700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8c0b52f0 0x7fbf8c140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.196+0000 7fbf92814700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbf8c138d00 0x7fbf8c13c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.201 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.196+0000 7fbf92814700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c13b850 con 0x7fbf8c0b52f0 2020-12-05T18:12:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.196+0000 7fbf92814700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c141490 con 0x7fbf8c0b1870 2020-12-05T18:12:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.196+0000 7fbf92814700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c13ba40 con 0x7fbf8c138d00 2020-12-05T18:12:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.196+0000 7fbf91011700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8c0b52f0 0x7fbf8c140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.196+0000 7fbf91011700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8c0b52f0 0x7fbf8c140fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42056/0 (socket says 172.21.15.32:42056) 2020-12-05T18:12:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.196+0000 7fbf91812700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbf8c0b1870 0x7fbf8c1395a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.196+0000 7fbf91011700 1 -- 172.21.15.32:0/2375901514 learned_addr learned my addr 172.21.15.32:0/2375901514 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.196+0000 7fbf92013700 1 --2- 172.21.15.32:0/2375901514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbf8c138d00 0x7fbf8c13c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.196+0000 7fbf91011700 1 -- 172.21.15.32:0/2375901514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbf8c138d00 msgr2=0x7fbf8c13c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.196+0000 7fbf91011700 1 --2- 172.21.15.32:0/2375901514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbf8c138d00 0x7fbf8c13c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.196+0000 7fbf91011700 1 -- 172.21.15.32:0/2375901514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbf8c0b1870 msgr2=0x7fbf8c1395a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.196+0000 7fbf91011700 1 --2- 172.21.15.32:0/2375901514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbf8c0b1870 0x7fbf8c1395a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.196+0000 7fbf91011700 1 -- 172.21.15.32:0/2375901514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8c13db70 con 0x7fbf8c0b52f0 2020-12-05T18:12:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.196+0000 7fbf91011700 1 --2- 172.21.15.32:0/2375901514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8c0b52f0 0x7fbf8c140fa0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 rx=0x7fbf8c0adc80 tx=0x7fbf80015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.196+0000 7fbf8affd700 1 -- 172.21.15.32:0/2375901514 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf8001ddd0 con 0x7fbf8c0b52f0 2020-12-05T18:12:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.196+0000 7fbf8affd700 1 -- 172.21.15.32:0/2375901514 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf8001d030 con 0x7fbf8c0b52f0 2020-12-05T18:12:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.196+0000 7fbf92814700 1 -- 172.21.15.32:0/2375901514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf8c13df00 con 0x7fbf8c0b52f0 2020-12-05T18:12:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.196+0000 7fbf8affd700 1 -- 172.21.15.32:0/2375901514 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf8001d390 con 0x7fbf8c0b52f0 2020-12-05T18:12:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.196+0000 7fbf92814700 1 -- 172.21.15.32:0/2375901514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf8c13c460 con 0x7fbf8c0b52f0 2020-12-05T18:12:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.196+0000 7fbf92814700 1 -- 172.21.15.32:0/2375901514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf8c13bc00 con 0x7fbf8c0b52f0 2020-12-05T18:12:15.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.200+0000 7fbf8affd700 1 -- 172.21.15.32:0/2375901514 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbf8002d030 con 0x7fbf8c0b52f0 2020-12-05T18:12:15.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.200+0000 7fbf8affd700 1 --2- 172.21.15.32:0/2375901514 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbf6c0554e0 0x7fbf6c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.200+0000 7fbf8affd700 1 -- 172.21.15.32:0/2375901514 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fbf80077400 con 0x7fbf8c0b52f0 2020-12-05T18:12:15.207 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.200+0000 7fbf91812700 1 --2- 172.21.15.32:0/2375901514 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbf6c0554e0 0x7fbf6c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.200+0000 7fbf91812700 1 --2- 172.21.15.32:0/2375901514 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbf6c0554e0 0x7fbf6c0579a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fbf8400b000 tx=0x7fbf840091a0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:15.208 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.204+0000 7fbf8affd700 1 -- 172.21.15.32:0/2375901514 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbf8004d6f0 con 0x7fbf8c0b52f0 2020-12-05T18:12:15.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.348+0000 7fbf92814700 1 -- 172.21.15.32:0/2375901514 --> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf8c13dd60 con 0x7fbf6c0554e0 2020-12-05T18:12:15.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.348+0000 7fbf8affd700 1 -- 172.21.15.32:0/2375901514 <== mgr.4101 v2:172.21.15.32:6800/11355 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbf84011ae0 con 0x7fbf6c0554e0 2020-12-05T18:12:15.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.352+0000 7fbf92814700 1 -- 172.21.15.32:0/2375901514 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbf6c0554e0 msgr2=0x7fbf6c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.352+0000 7fbf92814700 1 --2- 172.21.15.32:0/2375901514 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbf6c0554e0 0x7fbf6c0579a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fbf8400b000 tx=0x7fbf840091a0).stop 2020-12-05T18:12:15.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.352+0000 7fbf92814700 1 -- 172.21.15.32:0/2375901514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8c0b52f0 msgr2=0x7fbf8c140fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.355 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.352+0000 7fbf92814700 1 --2- 172.21.15.32:0/2375901514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8c0b52f0 0x7fbf8c140fa0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 rx=0x7fbf8c0adc80 tx=0x7fbf80015000).stop 2020-12-05T18:12:15.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.352+0000 7fbf92814700 1 -- 172.21.15.32:0/2375901514 shutdown_connections 2020-12-05T18:12:15.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.352+0000 7fbf92814700 1 --2- 172.21.15.32:0/2375901514 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fbf6c0554e0 0x7fbf6c0579a0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.352+0000 7fbf92814700 1 --2- 172.21.15.32:0/2375901514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbf8c138d00 0x7fbf8c13c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.356 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.352+0000 7fbf92814700 1 --2- 172.21.15.32:0/2375901514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbf8c0b1870 0x7fbf8c1395a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.352+0000 7fbf92814700 1 --2- 172.21.15.32:0/2375901514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8c0b52f0 0x7fbf8c140fa0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.352+0000 7fbf92814700 1 -- 172.21.15.32:0/2375901514 >> 172.21.15.32:0/2375901514 conn(0x7fbf8c05bbb0 msgr2=0x7fbf8c0672e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:15.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.352+0000 7fbf92814700 1 -- 172.21.15.32:0/2375901514 shutdown_connections 2020-12-05T18:12:15.357 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.352+0000 7fbf92814700 1 -- 172.21.15.32:0/2375901514 wait complete. 2020-12-05T18:12:15.358 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 1 to deep-scrub 2020-12-05T18:12:15.372 INFO:tasks.ceph:Scrubbing osd.2 2020-12-05T18:12:15.373 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.2 config set osd_debug_deep_scrub_sleep 0 2020-12-05T18:12:15.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.484+0000 7f714359e700 1 Processor -- start 2020-12-05T18:12:15.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.484+0000 7f714359e700 1 -- start start 2020-12-05T18:12:15.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.484+0000 7f714359e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f714405f3b0 0x7f714405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.484+0000 7f714359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71440b2fb0 0x7f7144061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.484+0000 7f714359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71440b2df0 con 0x7f714405feb0 2020-12-05T18:12:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.484+0000 7f714359e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71440b3c90 con 0x7f71440b2fb0 2020-12-05T18:12:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.484+0000 7f714359e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71440b2b50 con 0x7f714405f3b0 2020-12-05T18:12:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.484+0000 7f7142d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71440b2fb0 0x7f7144061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.484+0000 7f7141d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f714405feb0 0x7f714413c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44712/0 (socket says 172.21.15.32:44712) 2020-12-05T18:12:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.484+0000 7f7142d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71440b2fb0 0x7f7144061780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42064/0 (socket says 172.21.15.32:42064) 2020-12-05T18:12:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.484+0000 7f7141d9b700 1 -- 172.21.15.32:0/1928444910 learned_addr learned my addr 172.21.15.32:0/1928444910 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T18:12:15.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.488+0000 7f714259c700 1 --2- 172.21.15.32:0/1928444910 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f714405f3b0 0x7f714405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.488+0000 7f7142d9d700 1 -- 172.21.15.32:0/1928444910 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f714405f3b0 msgr2=0x7f714405f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T18:12:15.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.488+0000 7f7142d9d700 1 --2- 172.21.15.32:0/1928444910 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f714405f3b0 0x7f714405f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.488+0000 7f7142d9d700 1 -- 172.21.15.32:0/1928444910 >> v1:172.21.15.32:6789/0 conn(0x7f714405feb0 legacy=0x7f714413c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.488+0000 7f714159a700 1 -- 172.21.15.32:0/1928444910 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4255422317 0 0) 0x7f71440b2df0 con 0x7f714405feb0 2020-12-05T18:12:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.488+0000 7f7142d9d700 1 -- 172.21.15.32:0/1928444910 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7144142820 con 0x7f71440b2fb0 2020-12-05T18:12:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.488+0000 7f7142d9d700 1 --2- 172.21.15.32:0/1928444910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71440b2fb0 0x7f7144061780 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 rx=0x7f713c0032b0 tx=0x7f713c0091a0).ready entity=mon.0 client_cookie=8c3830f7c94f296a server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.488+0000 7f714159a700 1 -- 172.21.15.32:0/1928444910 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f713c00b9a0 con 0x7f71440b2fb0 2020-12-05T18:12:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.488+0000 7f714159a700 1 -- 172.21.15.32:0/1928444910 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713c00bb00 con 0x7f71440b2fb0 2020-12-05T18:12:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.488+0000 7f714159a700 1 -- 172.21.15.32:0/1928444910 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f713c00bc70 con 0x7f71440b2fb0 2020-12-05T18:12:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.488+0000 7f714359e700 1 -- 172.21.15.32:0/1928444910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71440b2fb0 msgr2=0x7f7144061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.488+0000 7f714359e700 1 --2- 172.21.15.32:0/1928444910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71440b2fb0 0x7f7144061780 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 rx=0x7f713c0032b0 tx=0x7f713c0091a0).stop 2020-12-05T18:12:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.488+0000 7f714359e700 1 -- 172.21.15.32:0/1928444910 shutdown_connections 2020-12-05T18:12:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.488+0000 7f714359e700 1 --2- 172.21.15.32:0/1928444910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71440b2fb0 0x7f7144061780 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.488+0000 7f714359e700 1 --2- 172.21.15.32:0/1928444910 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f714405f3b0 0x7f714405f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.488+0000 7f714359e700 1 -- 172.21.15.32:0/1928444910 >> 172.21.15.32:0/1928444910 conn(0x7f71440b8f20 msgr2=0x7f71440b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:15.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.488+0000 7f714359e700 1 -- 172.21.15.32:0/1928444910 shutdown_connections 2020-12-05T18:12:15.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.488+0000 7f714359e700 1 -- 172.21.15.32:0/1928444910 wait complete. 2020-12-05T18:12:15.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.488+0000 7f714359e700 1 Processor -- start 2020-12-05T18:12:15.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.488+0000 7f714359e700 1 -- start start 2020-12-05T18:12:15.499 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.492+0000 7f714359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f714405f3b0 0x7f7144146700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.492+0000 7f714359e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f714405feb0 0x7f71441415c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.492+0000 7f714359e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71440b2fb0 0x7f7144149020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.492+0000 7f714359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7144144890 con 0x7f714405f3b0 2020-12-05T18:12:15.500 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.492+0000 7f714359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7144142590 con 0x7f71440b2fb0 2020-12-05T18:12:15.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.492+0000 7f714359e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7144143eb0 con 0x7f714405feb0 2020-12-05T18:12:15.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.492+0000 7f714259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f714405f3b0 0x7f7144146700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.492+0000 7f714259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f714405f3b0 0x7f7144146700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42068/0 (socket says 172.21.15.32:42068) 2020-12-05T18:12:15.501 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.492+0000 7f714259c700 1 -- 172.21.15.32:0/2623885774 learned_addr learned my addr 172.21.15.32:0/2623885774 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:15.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.492+0000 7f7141d9b700 1 --2- 172.21.15.32:0/2623885774 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f714405feb0 0x7f71441415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.492+0000 7f7142d9d700 1 --2- 172.21.15.32:0/2623885774 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71440b2fb0 0x7f7144149020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.492+0000 7f714259c700 1 -- 172.21.15.32:0/2623885774 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f714405feb0 msgr2=0x7f71441415c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.492+0000 7f714259c700 1 --2- 172.21.15.32:0/2623885774 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f714405feb0 0x7f71441415c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.502 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.492+0000 7f714259c700 1 -- 172.21.15.32:0/2623885774 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71440b2fb0 msgr2=0x7f7144149020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.503 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.492+0000 7f714259c700 1 --2- 172.21.15.32:0/2623885774 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71440b2fb0 0x7f7144149020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.503 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.492+0000 7f714259c700 1 -- 172.21.15.32:0/2623885774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7144144150 con 0x7f714405f3b0 2020-12-05T18:12:15.503 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.492+0000 7f714259c700 1 --2- 172.21.15.32:0/2623885774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f714405f3b0 0x7f7144146700 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 rx=0x7f71441449d0 tx=0x7f7138006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:15.503 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.492+0000 7f71337fe700 1 -- 172.21.15.32:0/2623885774 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f713800ca10 con 0x7f714405f3b0 2020-12-05T18:12:15.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.492+0000 7f71337fe700 1 -- 172.21.15.32:0/2623885774 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713800cb70 con 0x7f714405f3b0 2020-12-05T18:12:15.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.492+0000 7f714359e700 1 -- 172.21.15.32:0/2623885774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71441443f0 con 0x7f714405f3b0 2020-12-05T18:12:15.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.492+0000 7f71337fe700 1 -- 172.21.15.32:0/2623885774 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71380123f0 con 0x7f714405f3b0 2020-12-05T18:12:15.504 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.492+0000 7f714359e700 1 -- 172.21.15.32:0/2623885774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7144141020 con 0x7f714405f3b0 2020-12-05T18:12:15.505 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.492+0000 7f714359e700 1 -- 172.21.15.32:0/2623885774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7144144580 con 0x7f714405f3b0 2020-12-05T18:12:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.496+0000 7f71337fe700 1 -- 172.21.15.32:0/2623885774 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7138012550 con 0x7f714405f3b0 2020-12-05T18:12:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.496+0000 7f71337fe700 1 --2- 172.21.15.32:0/2623885774 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7120055440 0x7f7120057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.496+0000 7f71337fe700 1 -- 172.21.15.32:0/2623885774 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f713806e1b0 con 0x7f714405f3b0 2020-12-05T18:12:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.496+0000 7f7141d9b700 1 --2- 172.21.15.32:0/2623885774 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7120055440 0x7f7120057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.496+0000 7f71337fe700 1 --2- 172.21.15.32:0/2623885774 >> [v2:172.21.15.32:6810/11521,v1:172.21.15.32:6811/11521] conn(0x7f712005c160 0x7f712005e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.496+0000 7f71337fe700 1 -- 172.21.15.32:0/2623885774 --> [v2:172.21.15.32:6810/11521,v1:172.21.15.32:6811/11521] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f712005eb60 con 0x7f712005c160 2020-12-05T18:12:15.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.496+0000 7f71337fe700 1 -- 172.21.15.32:0/2623885774 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71380707e0 con 0x7f714405f3b0 2020-12-05T18:12:15.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.496+0000 7f7142d9d700 1 --2- 172.21.15.32:0/2623885774 >> [v2:172.21.15.32:6810/11521,v1:172.21.15.32:6811/11521] conn(0x7f712005c160 0x7f712005e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.496+0000 7f7141d9b700 1 --2- 172.21.15.32:0/2623885774 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7120055440 0x7f7120057900 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7f713400d000 tx=0x7f7134009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:15.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.496+0000 7f7142d9d700 1 --2- 172.21.15.32:0/2623885774 >> [v2:172.21.15.32:6810/11521,v1:172.21.15.32:6811/11521] conn(0x7f712005c160 0x7f712005e580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:15.508 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.500+0000 7f71337fe700 1 -- 172.21.15.32:0/2623885774 <== osd.2 v2:172.21.15.32:6810/11521 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f712005eb60 con 0x7f712005c160 2020-12-05T18:12:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.520+0000 7f714359e700 1 -- 172.21.15.32:0/2623885774 --> [v2:172.21.15.32:6810/11521,v1:172.21.15.32:6811/11521] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7144057050 con 0x7f712005c160 2020-12-05T18:12:15.523 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.520+0000 7f71337fe700 1 -- 172.21.15.32:0/2623885774 <== osd.2 v2:172.21.15.32:6810/11521 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7144057050 con 0x7f712005c160 2020-12-05T18:12:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.520+0000 7f714359e700 1 -- 172.21.15.32:0/2623885774 >> [v2:172.21.15.32:6810/11521,v1:172.21.15.32:6811/11521] conn(0x7f712005c160 msgr2=0x7f712005e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.520+0000 7f714359e700 1 --2- 172.21.15.32:0/2623885774 >> [v2:172.21.15.32:6810/11521,v1:172.21.15.32:6811/11521] conn(0x7f712005c160 0x7f712005e580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.520+0000 7f714359e700 1 -- 172.21.15.32:0/2623885774 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7120055440 msgr2=0x7f7120057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.520+0000 7f714359e700 1 --2- 172.21.15.32:0/2623885774 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f7120055440 0x7f7120057900 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7f713400d000 tx=0x7f7134009040).stop 2020-12-05T18:12:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.520+0000 7f714359e700 1 -- 172.21.15.32:0/2623885774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f714405f3b0 msgr2=0x7f7144146700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.520+0000 7f714359e700 1 --2- 172.21.15.32:0/2623885774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f714405f3b0 0x7f7144146700 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 rx=0x7f71441449d0 tx=0x7f7138006dc0).stop 2020-12-05T18:12:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.520+0000 7f7142d9d700 1 -- 172.21.15.32:0/2623885774 reap_dead start 2020-12-05T18:12:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.520+0000 7f714359e700 1 -- 172.21.15.32:0/2623885774 shutdown_connections 2020-12-05T18:12:15.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.520+0000 7f714359e700 1 -- 172.21.15.32:0/2623885774 >> 172.21.15.32:0/2623885774 conn(0x7f71440b8f20 msgr2=0x7f71440b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:15.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.520+0000 7f714359e700 1 -- 172.21.15.32:0/2623885774 shutdown_connections 2020-12-05T18:12:15.526 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.520+0000 7f714359e700 1 -- 172.21.15.32:0/2623885774 wait complete. 2020-12-05T18:12:15.526 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T18:12:15.527 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-12-05T18:12:15.527 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T18:12:15.535 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 2 2020-12-05T18:12:15.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.644+0000 7f15935f8700 1 Processor -- start 2020-12-05T18:12:15.649 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.644+0000 7f15935f8700 1 -- start start 2020-12-05T18:12:15.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.644+0000 7f15935f8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f158c0afc50 0x7f158c1444d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f15935f8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158c0b25a0 con 0x7f158c14a3e0 2020-12-05T18:12:15.651 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f15935f8700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158c061a90 con 0x7f158c0b21c0 2020-12-05T18:12:15.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f15935f8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158c061d30 con 0x7f158c0afc50 2020-12-05T18:12:15.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f1591b95700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f158c14a3e0 0x7f158c14a820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44728/0 (socket says 172.21.15.32:44728) 2020-12-05T18:12:15.652 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f1591b95700 1 -- 172.21.15.32:0/2386174815 learned_addr learned my addr 172.21.15.32:0/2386174815 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T18:12:15.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f1590b93700 1 --2- 172.21.15.32:0/2386174815 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f158c0afc50 0x7f158c1444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f158bfff700 1 -- 172.21.15.32:0/2386174815 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940868603 0 0) 0x7f158c0b25a0 con 0x7f158c14a3e0 2020-12-05T18:12:15.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f158bfff700 1 -- 172.21.15.32:0/2386174815 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1578003230 con 0x7f158c14a3e0 2020-12-05T18:12:15.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f158bfff700 1 -- 172.21.15.32:0/2386174815 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519336945 0 0) 0x7f158c061a90 con 0x7f158c0b21c0 2020-12-05T18:12:15.653 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f158bfff700 1 -- 172.21.15.32:0/2386174815 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f158c0b25a0 con 0x7f158c0b21c0 2020-12-05T18:12:15.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f1590b93700 1 -- 172.21.15.32:0/2386174815 >> v1:172.21.15.61:6789/0 conn(0x7f158c0b21c0 legacy=0x7f158c143dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f1590b93700 1 -- 172.21.15.32:0/2386174815 >> v1:172.21.15.32:6789/0 conn(0x7f158c14a3e0 legacy=0x7f158c14a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f1590b93700 1 -- 172.21.15.32:0/2386174815 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c062a90 con 0x7f158c0afc50 2020-12-05T18:12:15.654 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f1590b93700 1 --2- 172.21.15.32:0/2386174815 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f158c0afc50 0x7f158c1444d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f1580002f30 tx=0x7f1580006dc0).ready entity=mon.2 client_cookie=37f54ec2b439575 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:15.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f158bfff700 1 -- 172.21.15.32:0/2386174815 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1580009730 con 0x7f158c0afc50 2020-12-05T18:12:15.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f158bfff700 1 -- 172.21.15.32:0/2386174815 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1580009890 con 0x7f158c0afc50 2020-12-05T18:12:15.657 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f15935f8700 1 -- 172.21.15.32:0/2386174815 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f158c0afc50 msgr2=0x7f158c1444d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f158bfff700 1 -- 172.21.15.32:0/2386174815 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1580009bf0 con 0x7f158c0afc50 2020-12-05T18:12:15.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f15935f8700 1 --2- 172.21.15.32:0/2386174815 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f158c0afc50 0x7f158c1444d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f1580002f30 tx=0x7f1580006dc0).stop 2020-12-05T18:12:15.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f15935f8700 1 -- 172.21.15.32:0/2386174815 shutdown_connections 2020-12-05T18:12:15.658 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f15935f8700 1 --2- 172.21.15.32:0/2386174815 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f158c0afc50 0x7f158c1444d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f15935f8700 1 -- 172.21.15.32:0/2386174815 >> 172.21.15.32:0/2386174815 conn(0x7f158c0b57a0 msgr2=0x7f158c0b1770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:15.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f15935f8700 1 -- 172.21.15.32:0/2386174815 shutdown_connections 2020-12-05T18:12:15.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.648+0000 7f15935f8700 1 -- 172.21.15.32:0/2386174815 wait complete. 2020-12-05T18:12:15.659 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f15935f8700 1 Processor -- start 2020-12-05T18:12:15.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f15935f8700 1 -- start start 2020-12-05T18:12:15.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f15935f8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f158c0afc50 0x7f158c09fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f15935f8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f158c0b21c0 0x7f158c146ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f15935f8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f158c14a3e0 0x7f158c1567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.660 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f15935f8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c09e940 con 0x7f158c0afc50 2020-12-05T18:12:15.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f15935f8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c0a0250 con 0x7f158c14a3e0 2020-12-05T18:12:15.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f15935f8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158c155e00 con 0x7f158c0b21c0 2020-12-05T18:12:15.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f1591394700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f158c0afc50 0x7f158c09fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.661 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f1591394700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f158c0afc50 0x7f158c09fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42086/0 (socket says 172.21.15.32:42086) 2020-12-05T18:12:15.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f1591394700 1 -- 172.21.15.32:0/2841709915 learned_addr learned my addr 172.21.15.32:0/2841709915 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:15.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f1591b95700 1 --2- 172.21.15.32:0/2841709915 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f158c14a3e0 0x7f158c1567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f1590b93700 1 --2- 172.21.15.32:0/2841709915 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f158c0b21c0 0x7f158c146ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.662 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f1591394700 1 -- 172.21.15.32:0/2841709915 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f158c0b21c0 msgr2=0x7f158c146ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f1591394700 1 --2- 172.21.15.32:0/2841709915 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f158c0b21c0 0x7f158c146ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f1591394700 1 -- 172.21.15.32:0/2841709915 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f158c14a3e0 msgr2=0x7f158c1567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f1591394700 1 --2- 172.21.15.32:0/2841709915 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f158c14a3e0 0x7f158c1567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.663 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f1591394700 1 -- 172.21.15.32:0/2841709915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c1560a0 con 0x7f158c0afc50 2020-12-05T18:12:15.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f1591394700 1 --2- 172.21.15.32:0/2841709915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f158c0afc50 0x7f158c09fd60 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 rx=0x7f157c001450 tx=0x7f157c007eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:15.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f158a7fc700 1 -- 172.21.15.32:0/2841709915 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f157c00ce70 con 0x7f158c0afc50 2020-12-05T18:12:15.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f158a7fc700 1 -- 172.21.15.32:0/2841709915 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c00a7b0 con 0x7f158c0afc50 2020-12-05T18:12:15.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f15935f8700 1 -- 172.21.15.32:0/2841709915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f158c156340 con 0x7f158c0afc50 2020-12-05T18:12:15.664 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f158a7fc700 1 -- 172.21.15.32:0/2841709915 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f157c00a920 con 0x7f158c0afc50 2020-12-05T18:12:15.665 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.652+0000 7f15935f8700 1 -- 172.21.15.32:0/2841709915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158c156cd0 con 0x7f158c0afc50 2020-12-05T18:12:15.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.656+0000 7f158a7fc700 1 -- 172.21.15.32:0/2841709915 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f157c010070 con 0x7f158c0afc50 2020-12-05T18:12:15.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.656+0000 7f15935f8700 1 -- 172.21.15.32:0/2841709915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158c0a1f90 con 0x7f158c0afc50 2020-12-05T18:12:15.666 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.656+0000 7f158a7fc700 1 --2- 172.21.15.32:0/2841709915 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f156c0553b0 0x7f156c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.656+0000 7f158a7fc700 1 -- 172.21.15.32:0/2841709915 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f157c06db00 con 0x7f158c0afc50 2020-12-05T18:12:15.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.656+0000 7f1590b93700 1 --2- 172.21.15.32:0/2841709915 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f156c0553b0 0x7f156c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.656+0000 7f1590b93700 1 --2- 172.21.15.32:0/2841709915 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f156c0553b0 0x7f156c057870 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f1580019000 tx=0x7f1580015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:15.667 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.660+0000 7f158a7fc700 1 -- 172.21.15.32:0/2841709915 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f158c0a1f90 con 0x7f158c0afc50 2020-12-05T18:12:15.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.800+0000 7f15935f8700 1 -- 172.21.15.32:0/2841709915 --> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f158c0a1f90 con 0x7f156c0553b0 2020-12-05T18:12:15.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.800+0000 7f158a7fc700 1 -- 172.21.15.32:0/2841709915 <== mgr.4101 v2:172.21.15.32:6800/11355 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f158c0a1f90 con 0x7f156c0553b0 2020-12-05T18:12:15.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.804+0000 7f15935f8700 1 -- 172.21.15.32:0/2841709915 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f156c0553b0 msgr2=0x7f156c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.804+0000 7f15935f8700 1 --2- 172.21.15.32:0/2841709915 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f156c0553b0 0x7f156c057870 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f1580019000 tx=0x7f1580015000).stop 2020-12-05T18:12:15.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.804+0000 7f15935f8700 1 -- 172.21.15.32:0/2841709915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f158c0afc50 msgr2=0x7f158c09fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.804+0000 7f15935f8700 1 --2- 172.21.15.32:0/2841709915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f158c0afc50 0x7f158c09fd60 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 rx=0x7f157c001450 tx=0x7f157c007eb0).stop 2020-12-05T18:12:15.807 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.804+0000 7f15935f8700 1 -- 172.21.15.32:0/2841709915 shutdown_connections 2020-12-05T18:12:15.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.804+0000 7f15935f8700 1 --2- 172.21.15.32:0/2841709915 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f158c14a3e0 0x7f158c1567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.804+0000 7f15935f8700 1 --2- 172.21.15.32:0/2841709915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f158c0afc50 0x7f158c09fd60 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.804+0000 7f15935f8700 1 --2- 172.21.15.32:0/2841709915 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f156c0553b0 0x7f156c057870 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.808 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.804+0000 7f15935f8700 1 --2- 172.21.15.32:0/2841709915 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f158c0b21c0 0x7f158c146ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.804+0000 7f15935f8700 1 -- 172.21.15.32:0/2841709915 >> 172.21.15.32:0/2841709915 conn(0x7f158c0b57a0 msgr2=0x7f158c0ae120 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:15.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.804+0000 7f15935f8700 1 -- 172.21.15.32:0/2841709915 shutdown_connections 2020-12-05T18:12:15.809 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.804+0000 7f15935f8700 1 -- 172.21.15.32:0/2841709915 wait complete. 2020-12-05T18:12:15.810 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 2 to deep-scrub 2020-12-05T18:12:15.823 INFO:tasks.ceph:Scrubbing osd.3 2020-12-05T18:12:15.823 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.3 config set osd_debug_deep_scrub_sleep 0 2020-12-05T18:12:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.936+0000 7fedf5f10700 1 Processor -- start 2020-12-05T18:12:15.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.936+0000 7fedf5f10700 1 -- start start 2020-12-05T18:12:15.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedf5f10700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fedf005f3b0 0x7fedf005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedf5f10700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fedf005feb0 0x7fedf0061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedf5f10700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf00b2df0 con 0x7fedf00b2fb0 2020-12-05T18:12:15.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedf5f10700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf00b3c90 con 0x7fedf005feb0 2020-12-05T18:12:15.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedf5f10700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf00b2b50 con 0x7fedf005f3b0 2020-12-05T18:12:15.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedeffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fedf005feb0 0x7fedf0061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedf570f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fedf00b2fb0 0x7fedf013c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44742/0 (socket says 172.21.15.32:44742) 2020-12-05T18:12:15.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedf570f700 1 -- 172.21.15.32:0/2294682821 learned_addr learned my addr 172.21.15.32:0/2294682821 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T18:12:15.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedf4f0e700 1 --2- 172.21.15.32:0/2294682821 >> v2:172.21.15.61:3300/0 conn(0x7fedf005f3b0 0x7fedf005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedeffff700 1 -- 172.21.15.32:0/2294682821 >> v2:172.21.15.61:3300/0 conn(0x7fedf005f3b0 msgr2=0x7fedf005f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T18:12:15.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedeffff700 1 --2- 172.21.15.32:0/2294682821 >> v2:172.21.15.61:3300/0 conn(0x7fedf005f3b0 0x7fedf005f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedeffff700 1 -- 172.21.15.32:0/2294682821 >> v1:172.21.15.32:6789/0 conn(0x7fedf00b2fb0 legacy=0x7fedf013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedeffff700 1 -- 172.21.15.32:0/2294682821 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf0142820 con 0x7fedf005feb0 2020-12-05T18:12:15.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedef7fe700 1 -- 172.21.15.32:0/2294682821 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2050699486 0 0) 0x7fedf00b2df0 con 0x7fedf00b2fb0 2020-12-05T18:12:15.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedeffff700 1 --2- 172.21.15.32:0/2294682821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedf005feb0 0x7fedf0061780 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 rx=0x7fedd8002f30 tx=0x7fedd8006dc0).ready entity=mon.0 client_cookie=20af48b924e2554d server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:15.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedef7fe700 1 -- 172.21.15.32:0/2294682821 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedd80097b0 con 0x7fedf005feb0 2020-12-05T18:12:15.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedef7fe700 1 -- 172.21.15.32:0/2294682821 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd8009910 con 0x7fedf005feb0 2020-12-05T18:12:15.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedef7fe700 1 -- 172.21.15.32:0/2294682821 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedd8009c70 con 0x7fedf005feb0 2020-12-05T18:12:15.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedf5f10700 1 -- 172.21.15.32:0/2294682821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedf005feb0 msgr2=0x7fedf0061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedf5f10700 1 --2- 172.21.15.32:0/2294682821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedf005feb0 0x7fedf0061780 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 rx=0x7fedd8002f30 tx=0x7fedd8006dc0).stop 2020-12-05T18:12:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedf5f10700 1 -- 172.21.15.32:0/2294682821 shutdown_connections 2020-12-05T18:12:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedf5f10700 1 --2- 172.21.15.32:0/2294682821 >> v2:172.21.15.61:3300/0 conn(0x7fedf005f3b0 0x7fedf005f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedf5f10700 1 --2- 172.21.15.32:0/2294682821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedf005feb0 0x7fedf0061780 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedf5f10700 1 -- 172.21.15.32:0/2294682821 >> 172.21.15.32:0/2294682821 conn(0x7fedf00b8f20 msgr2=0x7fedf00b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedf5f10700 1 -- 172.21.15.32:0/2294682821 shutdown_connections 2020-12-05T18:12:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.940+0000 7fedf5f10700 1 -- 172.21.15.32:0/2294682821 wait complete. 2020-12-05T18:12:15.960 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fedf5f10700 1 Processor -- start 2020-12-05T18:12:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fedf5f10700 1 -- start start 2020-12-05T18:12:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fedf5f10700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fedf005f3b0 0x7fedf0141320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fedf5f10700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedf005feb0 0x7fedf0141810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fedf5f10700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fedf00b2fb0 0x7fedf0147ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.962 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fedf5f10700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf013e8a0 con 0x7fedf005feb0 2020-12-05T18:12:15.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fedf5f10700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf013ead0 con 0x7fedf00b2fb0 2020-12-05T18:12:15.963 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fedf5f10700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf0145760 con 0x7fedf005f3b0 2020-12-05T18:12:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fedeffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedf005feb0 0x7fedf0141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fedeffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedf005feb0 0x7fedf0141810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42098/0 (socket says 172.21.15.32:42098) 2020-12-05T18:12:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fedeffff700 1 -- 172.21.15.32:0/65061873 learned_addr learned my addr 172.21.15.32:0/65061873 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fedf4f0e700 1 --2- 172.21.15.32:0/65061873 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fedf005f3b0 0x7fedf0141320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fedf570f700 1 --2- 172.21.15.32:0/65061873 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fedf00b2fb0 0x7fedf0147ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fedeffff700 1 -- 172.21.15.32:0/65061873 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fedf005f3b0 msgr2=0x7fedf0141320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fedeffff700 1 --2- 172.21.15.32:0/65061873 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fedf005f3b0 0x7fedf0141320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fedeffff700 1 -- 172.21.15.32:0/65061873 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fedf00b2fb0 msgr2=0x7fedf0147ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.966 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fedeffff700 1 --2- 172.21.15.32:0/65061873 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fedf00b2fb0 0x7fedf0147ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fedeffff700 1 -- 172.21.15.32:0/65061873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf0146550 con 0x7fedf005feb0 2020-12-05T18:12:15.967 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fedf4f0e700 1 --2- 172.21.15.32:0/65061873 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fedf005f3b0 0x7fedf0141320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T18:12:15.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fedeffff700 1 --2- 172.21.15.32:0/65061873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedf005feb0 0x7fedf0141810 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 rx=0x7fedd8002f30 tx=0x7fedd8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:15.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fededffb700 1 -- 172.21.15.32:0/65061873 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedd8011b40 con 0x7fedf005feb0 2020-12-05T18:12:15.968 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fededffb700 1 -- 172.21.15.32:0/65061873 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd8011ca0 con 0x7fedf005feb0 2020-12-05T18:12:15.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fededffb700 1 -- 172.21.15.32:0/65061873 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedd801d440 con 0x7fedf005feb0 2020-12-05T18:12:15.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fedf5f10700 1 -- 172.21.15.32:0/65061873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf0147580 con 0x7fedf005feb0 2020-12-05T18:12:15.969 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.944+0000 7fedf5f10700 1 -- 172.21.15.32:0/65061873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf0148390 con 0x7fedf005feb0 2020-12-05T18:12:15.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.948+0000 7fedf5f10700 1 -- 172.21.15.32:0/65061873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fedf013ec90 con 0x7fedf005feb0 2020-12-05T18:12:15.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.948+0000 7fededffb700 1 -- 172.21.15.32:0/65061873 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fedd8017070 con 0x7fedf005feb0 2020-12-05T18:12:15.970 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.948+0000 7fededffb700 1 --2- 172.21.15.32:0/65061873 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fedd4055440 0x7fedd4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.948+0000 7fededffb700 1 -- 172.21.15.32:0/65061873 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fedd806e9d0 con 0x7fedf005feb0 2020-12-05T18:12:15.971 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.948+0000 7fedf4f0e700 1 --2- 172.21.15.32:0/65061873 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fedd4055440 0x7fedd4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.948+0000 7fededffb700 1 --2- 172.21.15.32:0/65061873 >> [v2:172.21.15.32:6826/11535,v1:172.21.15.32:6827/11535] conn(0x7fedd405c160 0x7fedd405e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:15.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.948+0000 7fededffb700 1 -- 172.21.15.32:0/65061873 --> [v2:172.21.15.32:6826/11535,v1:172.21.15.32:6827/11535] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fedd405eb60 con 0x7fedd405c160 2020-12-05T18:12:15.972 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.948+0000 7fededffb700 1 -- 172.21.15.32:0/65061873 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedd8073240 con 0x7fedf005feb0 2020-12-05T18:12:15.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.948+0000 7fedf570f700 1 --2- 172.21.15.32:0/65061873 >> [v2:172.21.15.32:6826/11535,v1:172.21.15.32:6827/11535] conn(0x7fedd405c160 0x7fedd405e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:15.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.948+0000 7fedf4f0e700 1 --2- 172.21.15.32:0/65061873 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fedd4055440 0x7fedd4057900 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7fede000b000 tx=0x7fede0008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:15.973 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.948+0000 7fedf570f700 1 --2- 172.21.15.32:0/65061873 >> [v2:172.21.15.32:6826/11535,v1:172.21.15.32:6827/11535] conn(0x7fedd405c160 0x7fedd405e580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:15.974 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.952+0000 7fededffb700 1 -- 172.21.15.32:0/65061873 <== osd.3 v2:172.21.15.32:6826/11535 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7fedd405eb60 con 0x7fedd405c160 2020-12-05T18:12:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.968+0000 7fedf5f10700 1 -- 172.21.15.32:0/65061873 --> [v2:172.21.15.32:6826/11535,v1:172.21.15.32:6827/11535] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fedf0057050 con 0x7fedd405c160 2020-12-05T18:12:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.972+0000 7fededffb700 1 -- 172.21.15.32:0/65061873 <== osd.3 v2:172.21.15.32:6826/11535 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fedf0057050 con 0x7fedd405c160 2020-12-05T18:12:15.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.972+0000 7fedf5f10700 1 -- 172.21.15.32:0/65061873 >> [v2:172.21.15.32:6826/11535,v1:172.21.15.32:6827/11535] conn(0x7fedd405c160 msgr2=0x7fedd405e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.972+0000 7fedf5f10700 1 --2- 172.21.15.32:0/65061873 >> [v2:172.21.15.32:6826/11535,v1:172.21.15.32:6827/11535] conn(0x7fedd405c160 0x7fedd405e580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:15.977 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.972+0000 7fedf5f10700 1 -- 172.21.15.32:0/65061873 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fedd4055440 msgr2=0x7fedd4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.972+0000 7fedf5f10700 1 --2- 172.21.15.32:0/65061873 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fedd4055440 0x7fedd4057900 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7fede000b000 tx=0x7fede0008040).stop 2020-12-05T18:12:15.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.972+0000 7fedf5f10700 1 -- 172.21.15.32:0/65061873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedf005feb0 msgr2=0x7fedf0141810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:15.978 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.972+0000 7fedf5f10700 1 --2- 172.21.15.32:0/65061873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedf005feb0 0x7fedf0141810 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 rx=0x7fedd8002f30 tx=0x7fedd8015000).stop 2020-12-05T18:12:15.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.972+0000 7fedf570f700 1 -- 172.21.15.32:0/65061873 reap_dead start 2020-12-05T18:12:15.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.972+0000 7fedf5f10700 1 -- 172.21.15.32:0/65061873 shutdown_connections 2020-12-05T18:12:15.979 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.972+0000 7fedf5f10700 1 -- 172.21.15.32:0/65061873 >> 172.21.15.32:0/65061873 conn(0x7fedf00b8f20 msgr2=0x7fedf00b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.972+0000 7fedf5f10700 1 -- 172.21.15.32:0/65061873 shutdown_connections 2020-12-05T18:12:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:15.972+0000 7fedf5f10700 1 -- 172.21.15.32:0/65061873 wait complete. 2020-12-05T18:12:15.980 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T18:12:15.981 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-12-05T18:12:15.981 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T18:12:15.987 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 3 2020-12-05T18:12:16.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64d518d700 1 Processor -- start 2020-12-05T18:12:16.099 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64d518d700 1 -- start start 2020-12-05T18:12:16.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64d518d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64d00b1fc0 0x7f64d00af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64d518d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f64d0147400 0x7f64d0145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64d518d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d0146a90 con 0x7f64d00ab710 2020-12-05T18:12:16.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64d518d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d0061e90 con 0x7f64d00b1fc0 2020-12-05T18:12:16.103 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64d518d700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d00b0110 con 0x7f64d0147400 2020-12-05T18:12:16.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64cffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64d00b1fc0 0x7f64d00af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.104 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64cffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64d00b1fc0 0x7f64d00af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42106/0 (socket says 172.21.15.32:42106) 2020-12-05T18:12:16.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64cf7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f64d00ab710 0x7f64d0061780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44758/0 (socket says 172.21.15.32:44758) 2020-12-05T18:12:16.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64cffff700 1 -- 172.21.15.32:0/2645063479 learned_addr learned my addr 172.21.15.32:0/2645063479 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:16.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64d498c700 1 --2- 172.21.15.32:0/2645063479 >> v2:172.21.15.61:3300/0 conn(0x7f64d0147400 0x7f64d0145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.105 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64cffff700 1 -- 172.21.15.32:0/2645063479 >> v2:172.21.15.61:3300/0 conn(0x7f64d0147400 msgr2=0x7f64d0145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T18:12:16.106 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64cffff700 1 --2- 172.21.15.32:0/2645063479 >> v2:172.21.15.61:3300/0 conn(0x7f64d0147400 0x7f64d0145d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64cffff700 1 -- 172.21.15.32:0/2645063479 >> v1:172.21.15.32:6789/0 conn(0x7f64d00ab710 legacy=0x7f64d0061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.107 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64ceffd700 1 -- 172.21.15.32:0/2645063479 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2908367527 0 0) 0x7f64d0146a90 con 0x7f64d00ab710 2020-12-05T18:12:16.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64cffff700 1 -- 172.21.15.32:0/2645063479 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d00b23a0 con 0x7f64d00b1fc0 2020-12-05T18:12:16.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64cffff700 1 --2- 172.21.15.32:0/2645063479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d00b1fc0 0x7f64d00af230 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7f64c80032b0 tx=0x7f64c8009270).ready entity=mon.0 client_cookie=a87fe1a42570df6f server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:16.108 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64ceffd700 1 -- 172.21.15.32:0/2645063479 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64c8009e60 con 0x7f64d00b1fc0 2020-12-05T18:12:16.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64ceffd700 1 -- 172.21.15.32:0/2645063479 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c800bc00 con 0x7f64d00b1fc0 2020-12-05T18:12:16.109 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64ceffd700 1 -- 172.21.15.32:0/2645063479 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64c800bd70 con 0x7f64d00b1fc0 2020-12-05T18:12:16.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64d518d700 1 -- 172.21.15.32:0/2645063479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d00b1fc0 msgr2=0x7f64d00af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64d518d700 1 --2- 172.21.15.32:0/2645063479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d00b1fc0 0x7f64d00af230 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7f64c80032b0 tx=0x7f64c8009270).stop 2020-12-05T18:12:16.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64d518d700 1 -- 172.21.15.32:0/2645063479 shutdown_connections 2020-12-05T18:12:16.110 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64d518d700 1 --2- 172.21.15.32:0/2645063479 >> v2:172.21.15.61:3300/0 conn(0x7f64d0147400 0x7f64d0145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64d518d700 1 --2- 172.21.15.32:0/2645063479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d00b1fc0 0x7f64d00af230 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.096+0000 7f64d518d700 1 -- 172.21.15.32:0/2645063479 >> 172.21.15.32:0/2645063479 conn(0x7f64d00b5530 msgr2=0x7f64d00b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:16.111 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.100+0000 7f64d518d700 1 -- 172.21.15.32:0/2645063479 shutdown_connections 2020-12-05T18:12:16.112 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.100+0000 7f64d518d700 1 -- 172.21.15.32:0/2645063479 wait complete. 2020-12-05T18:12:16.114 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.100+0000 7f64d518d700 1 Processor -- start 2020-12-05T18:12:16.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.100+0000 7f64d518d700 1 -- start start 2020-12-05T18:12:16.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.100+0000 7f64d518d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d00ab710 0x7f64d014b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.100+0000 7f64d518d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f64d00b1fc0 0x7f64d0158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.115 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.100+0000 7f64d518d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f64d0147400 0x7f64d00638a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.100+0000 7f64d518d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d0063470 con 0x7f64d00ab710 2020-12-05T18:12:16.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.100+0000 7f64d518d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d00acd00 con 0x7f64d0147400 2020-12-05T18:12:16.116 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.100+0000 7f64d518d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d00acfa0 con 0x7f64d00b1fc0 2020-12-05T18:12:16.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.100+0000 7f64cffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d00ab710 0x7f64d014b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.100+0000 7f64cffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d00ab710 0x7f64d014b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42116/0 (socket says 172.21.15.32:42116) 2020-12-05T18:12:16.117 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.100+0000 7f64cffff700 1 -- 172.21.15.32:0/2022995465 learned_addr learned my addr 172.21.15.32:0/2022995465 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:16.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.100+0000 7f64d498c700 1 --2- 172.21.15.32:0/2022995465 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f64d0147400 0x7f64d00638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.100+0000 7f64cf7fe700 1 --2- 172.21.15.32:0/2022995465 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f64d00b1fc0 0x7f64d0158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.118 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.100+0000 7f64cffff700 1 -- 172.21.15.32:0/2022995465 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f64d00b1fc0 msgr2=0x7f64d0158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.100+0000 7f64cffff700 1 --2- 172.21.15.32:0/2022995465 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f64d00b1fc0 0x7f64d0158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.100+0000 7f64cffff700 1 -- 172.21.15.32:0/2022995465 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f64d0147400 msgr2=0x7f64d00638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.100+0000 7f64cffff700 1 --2- 172.21.15.32:0/2022995465 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f64d0147400 0x7f64d00638a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.100+0000 7f64cffff700 1 -- 172.21.15.32:0/2022995465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d014de40 con 0x7f64d00ab710 2020-12-05T18:12:16.119 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.100+0000 7f64cf7fe700 1 --2- 172.21.15.32:0/2022995465 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f64d00b1fc0 0x7f64d0158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T18:12:16.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.100+0000 7f64cffff700 1 --2- 172.21.15.32:0/2022995465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d00ab710 0x7f64d014b550 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 rx=0x7f64c80032b0 tx=0x7f64c8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:16.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.104+0000 7f64cd7fa700 1 -- 172.21.15.32:0/2022995465 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64c80107a0 con 0x7f64d00ab710 2020-12-05T18:12:16.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.104+0000 7f64cd7fa700 1 -- 172.21.15.32:0/2022995465 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c8010900 con 0x7f64d00ab710 2020-12-05T18:12:16.120 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.104+0000 7f64d518d700 1 -- 172.21.15.32:0/2022995465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d00add00 con 0x7f64d00ab710 2020-12-05T18:12:16.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.104+0000 7f64cd7fa700 1 -- 172.21.15.32:0/2022995465 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64c8010c60 con 0x7f64d00ab710 2020-12-05T18:12:16.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.104+0000 7f64d518d700 1 -- 172.21.15.32:0/2022995465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d00abd70 con 0x7f64d00ab710 2020-12-05T18:12:16.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.104+0000 7f64cd7fa700 1 -- 172.21.15.32:0/2022995465 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f64c800f070 con 0x7f64d00ab710 2020-12-05T18:12:16.121 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.104+0000 7f64d518d700 1 -- 172.21.15.32:0/2022995465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d00ad110 con 0x7f64d00ab710 2020-12-05T18:12:16.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.104+0000 7f64cd7fa700 1 --2- 172.21.15.32:0/2022995465 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f64b4055440 0x7f64b4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.104+0000 7f64cd7fa700 1 -- 172.21.15.32:0/2022995465 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f64c806f5a0 con 0x7f64d00ab710 2020-12-05T18:12:16.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.104+0000 7f64cf7fe700 1 --2- 172.21.15.32:0/2022995465 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f64b4055440 0x7f64b4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.122 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.108+0000 7f64cf7fe700 1 --2- 172.21.15.32:0/2022995465 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f64b4055440 0x7f64b4057900 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f64b800b000 tx=0x7f64b8007100).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:16.123 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.112+0000 7f64cd7fa700 1 -- 172.21.15.32:0/2022995465 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f64d00ad110 con 0x7f64d00ab710 2020-12-05T18:12:16.254 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.252+0000 7f64d518d700 1 -- 172.21.15.32:0/2022995465 --> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f64d00acb80 con 0x7f64b4055440 2020-12-05T18:12:16.255 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.252+0000 7f64cd7fa700 1 -- 172.21.15.32:0/2022995465 <== mgr.4101 v2:172.21.15.32:6800/11355 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f64b8007af0 con 0x7f64b4055440 2020-12-05T18:12:16.256 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.252+0000 7f64d518d700 1 -- 172.21.15.32:0/2022995465 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f64b4055440 msgr2=0x7f64b4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.252+0000 7f64d518d700 1 --2- 172.21.15.32:0/2022995465 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f64b4055440 0x7f64b4057900 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f64b800b000 tx=0x7f64b8007100).stop 2020-12-05T18:12:16.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.252+0000 7f64d518d700 1 -- 172.21.15.32:0/2022995465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d00ab710 msgr2=0x7f64d014b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.252+0000 7f64d518d700 1 --2- 172.21.15.32:0/2022995465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d00ab710 0x7f64d014b550 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 rx=0x7f64c80032b0 tx=0x7f64c8016000).stop 2020-12-05T18:12:16.257 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.252+0000 7f64d518d700 1 -- 172.21.15.32:0/2022995465 shutdown_connections 2020-12-05T18:12:16.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.252+0000 7f64d518d700 1 --2- 172.21.15.32:0/2022995465 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f64d0147400 0x7f64d00638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.252+0000 7f64d518d700 1 --2- 172.21.15.32:0/2022995465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d00ab710 0x7f64d014b550 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.252+0000 7f64d518d700 1 --2- 172.21.15.32:0/2022995465 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f64b4055440 0x7f64b4057900 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.258 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.252+0000 7f64d518d700 1 --2- 172.21.15.32:0/2022995465 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f64d00b1fc0 0x7f64d0158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.252+0000 7f64d518d700 1 -- 172.21.15.32:0/2022995465 >> 172.21.15.32:0/2022995465 conn(0x7f64d00b5530 msgr2=0x7f64d00af740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:16.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.252+0000 7f64d518d700 1 -- 172.21.15.32:0/2022995465 shutdown_connections 2020-12-05T18:12:16.259 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.252+0000 7f64d518d700 1 -- 172.21.15.32:0/2022995465 wait complete. 2020-12-05T18:12:16.260 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 3 to deep-scrub 2020-12-05T18:12:16.269 INFO:tasks.ceph:Scrubbing osd.4 2020-12-05T18:12:16.270 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.4 config set osd_debug_deep_scrub_sleep 0 2020-12-05T18:12:16.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.384+0000 7fac8759e700 1 Processor -- start 2020-12-05T18:12:16.390 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8759e700 1 -- start start 2020-12-05T18:12:16.392 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fac880b1e60 0x7fac880af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8759e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fac88062ac0 0x7fac88138fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8759e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac880aedf0 0x7fac881394a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8759e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac880ae0a0 con 0x7fac880b1e60 2020-12-05T18:12:16.393 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8759e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac88139990 con 0x7fac88062ac0 2020-12-05T18:12:16.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8759e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac88143390 con 0x7fac880aedf0 2020-12-05T18:12:16.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fac880b1e60 0x7fac880af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fac880b1e60 0x7fac880af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42122/0 (socket says 172.21.15.32:42122) 2020-12-05T18:12:16.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8659c700 1 -- 172.21.15.32:0/1380654751 learned_addr learned my addr 172.21.15.32:0/1380654751 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:16.394 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac7dd9b700 1 --2- 172.21.15.32:0/1380654751 >> v2:172.21.15.61:3300/0 conn(0x7fac88062ac0 0x7fac88138fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac86d9d700 1 --2- 172.21.15.32:0/1380654751 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac880aedf0 0x7fac881394a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8659c700 1 -- 172.21.15.32:0/1380654751 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac880aedf0 msgr2=0x7fac881394a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T18:12:16.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8659c700 1 --2- 172.21.15.32:0/1380654751 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac880aedf0 0x7fac881394a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.395 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8659c700 1 -- 172.21.15.32:0/1380654751 >> v2:172.21.15.61:3300/0 conn(0x7fac88062ac0 msgr2=0x7fac88138fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T18:12:16.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8659c700 1 --2- 172.21.15.32:0/1380654751 >> v2:172.21.15.61:3300/0 conn(0x7fac88062ac0 0x7fac88138fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8659c700 1 -- 172.21.15.32:0/1380654751 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac88143610 con 0x7fac880b1e60 2020-12-05T18:12:16.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8659c700 1 --2- 172.21.15.32:0/1380654751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880b1e60 0x7fac880af4b0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7fac78002f30 tx=0x7fac78006e80).ready entity=mon.0 client_cookie=2aeccc0e94df1705 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:16.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac85d9b700 1 -- 172.21.15.32:0/1380654751 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac78009640 con 0x7fac880b1e60 2020-12-05T18:12:16.396 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac85d9b700 1 -- 172.21.15.32:0/1380654751 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac780097a0 con 0x7fac880b1e60 2020-12-05T18:12:16.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac85d9b700 1 -- 172.21.15.32:0/1380654751 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac78009b00 con 0x7fac880b1e60 2020-12-05T18:12:16.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8759e700 1 -- 172.21.15.32:0/1380654751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880b1e60 msgr2=0x7fac880af4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8759e700 1 --2- 172.21.15.32:0/1380654751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880b1e60 0x7fac880af4b0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7fac78002f30 tx=0x7fac78006e80).stop 2020-12-05T18:12:16.397 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8759e700 1 -- 172.21.15.32:0/1380654751 shutdown_connections 2020-12-05T18:12:16.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8759e700 1 --2- 172.21.15.32:0/1380654751 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac880aedf0 0x7fac881394a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8759e700 1 --2- 172.21.15.32:0/1380654751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880b1e60 0x7fac880af4b0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8759e700 1 --2- 172.21.15.32:0/1380654751 >> v2:172.21.15.61:3300/0 conn(0x7fac88062ac0 0x7fac88138fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.398 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8759e700 1 -- 172.21.15.32:0/1380654751 >> 172.21.15.32:0/1380654751 conn(0x7fac880b5620 msgr2=0x7fac880b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:16.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8759e700 1 -- 172.21.15.32:0/1380654751 shutdown_connections 2020-12-05T18:12:16.399 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8759e700 1 -- 172.21.15.32:0/1380654751 wait complete. 2020-12-05T18:12:16.401 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8759e700 1 Processor -- start 2020-12-05T18:12:16.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.388+0000 7fac8759e700 1 -- start start 2020-12-05T18:12:16.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac8759e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac88062ac0 0x7fac88066960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac8759e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac880aedf0 0x7fac8814f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.402 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac8759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880b1e60 0x7fac8814f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac8759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac88131c10 con 0x7fac880b1e60 2020-12-05T18:12:16.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac8759e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8814fe10 con 0x7fac88062ac0 2020-12-05T18:12:16.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac8759e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac88066e50 con 0x7fac880aedf0 2020-12-05T18:12:16.403 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac86d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880b1e60 0x7fac8814f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac86d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880b1e60 0x7fac8814f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42128/0 (socket says 172.21.15.32:42128) 2020-12-05T18:12:16.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac86d9d700 1 -- 172.21.15.32:0/2987115115 learned_addr learned my addr 172.21.15.32:0/2987115115 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:16.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac8659c700 1 --2- 172.21.15.32:0/2987115115 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac88062ac0 0x7fac88066960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac7dd9b700 1 --2- 172.21.15.32:0/2987115115 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac880aedf0 0x7fac8814f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.404 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac86d9d700 1 -- 172.21.15.32:0/2987115115 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac880aedf0 msgr2=0x7fac8814f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac86d9d700 1 --2- 172.21.15.32:0/2987115115 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac880aedf0 0x7fac8814f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac86d9d700 1 -- 172.21.15.32:0/2987115115 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac88062ac0 msgr2=0x7fac88066960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac86d9d700 1 --2- 172.21.15.32:0/2987115115 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac88062ac0 0x7fac88066960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.405 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac86d9d700 1 -- 172.21.15.32:0/2987115115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac88063ef0 con 0x7fac880b1e60 2020-12-05T18:12:16.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac8659c700 1 --2- 172.21.15.32:0/2987115115 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac88062ac0 0x7fac88066960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T18:12:16.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac86d9d700 1 --2- 172.21.15.32:0/2987115115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880b1e60 0x7fac8814f920 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x7fac880653e0 tx=0x7fac800091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:16.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac7ffff700 1 -- 172.21.15.32:0/2987115115 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac8000cbb0 con 0x7fac880b1e60 2020-12-05T18:12:16.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac7ffff700 1 -- 172.21.15.32:0/2987115115 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8000cd10 con 0x7fac880b1e60 2020-12-05T18:12:16.406 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac8759e700 1 -- 172.21.15.32:0/2987115115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac88066220 con 0x7fac880b1e60 2020-12-05T18:12:16.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac7ffff700 1 -- 172.21.15.32:0/2987115115 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac800125e0 con 0x7fac880b1e60 2020-12-05T18:12:16.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac8759e700 1 -- 172.21.15.32:0/2987115115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac8813f260 con 0x7fac880b1e60 2020-12-05T18:12:16.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac8759e700 1 -- 172.21.15.32:0/2987115115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fac880666c0 con 0x7fac880b1e60 2020-12-05T18:12:16.407 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.392+0000 7fac7ffff700 1 -- 172.21.15.32:0/2987115115 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fac80022070 con 0x7fac880b1e60 2020-12-05T18:12:16.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.396+0000 7fac7ffff700 1 --2- 172.21.15.32:0/2987115115 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fac6c0553b0 0x7fac6c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.396+0000 7fac7ffff700 1 -- 172.21.15.32:0/2987115115 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fac8006e500 con 0x7fac880b1e60 2020-12-05T18:12:16.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.396+0000 7fac7ffff700 1 --2- 172.21.15.32:0/2987115115 >> [v2:172.21.15.61:6808/11541,v1:172.21.15.61:6809/11541] conn(0x7fac6c05c0d0 0x7fac6c05e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.396+0000 7fac7ffff700 1 -- 172.21.15.32:0/2987115115 --> [v2:172.21.15.61:6808/11541,v1:172.21.15.61:6809/11541] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fac6c05eb70 con 0x7fac6c05c0d0 2020-12-05T18:12:16.408 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.396+0000 7fac7ffff700 1 -- 172.21.15.32:0/2987115115 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac80070b90 con 0x7fac880b1e60 2020-12-05T18:12:16.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.396+0000 7fac8659c700 1 --2- 172.21.15.32:0/2987115115 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fac6c0553b0 0x7fac6c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.396+0000 7fac7dd9b700 1 --2- 172.21.15.32:0/2987115115 >> [v2:172.21.15.61:6808/11541,v1:172.21.15.61:6809/11541] conn(0x7fac6c05c0d0 0x7fac6c05e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.396+0000 7fac8659c700 1 --2- 172.21.15.32:0/2987115115 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fac6c0553b0 0x7fac6c057870 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7fac78019000 tx=0x7fac78015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:16.409 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.396+0000 7fac7dd9b700 1 --2- 172.21.15.32:0/2987115115 >> [v2:172.21.15.61:6808/11541,v1:172.21.15.61:6809/11541] conn(0x7fac6c05c0d0 0x7fac6c05e4f0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:16.410 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.396+0000 7fac7ffff700 1 -- 172.21.15.32:0/2987115115 <== osd.4 v2:172.21.15.61:6808/11541 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7fac6c05eb70 con 0x7fac6c05c0d0 2020-12-05T18:12:16.419 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.416+0000 7fac8759e700 1 -- 172.21.15.32:0/2987115115 --> [v2:172.21.15.61:6808/11541,v1:172.21.15.61:6809/11541] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fac88057050 con 0x7fac6c05c0d0 2020-12-05T18:12:16.421 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.416+0000 7fac7ffff700 1 -- 172.21.15.32:0/2987115115 <== osd.4 v2:172.21.15.61:6808/11541 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fac88057050 con 0x7fac6c05c0d0 2020-12-05T18:12:16.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.416+0000 7fac8759e700 1 -- 172.21.15.32:0/2987115115 >> [v2:172.21.15.61:6808/11541,v1:172.21.15.61:6809/11541] conn(0x7fac6c05c0d0 msgr2=0x7fac6c05e4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.416+0000 7fac8759e700 1 --2- 172.21.15.32:0/2987115115 >> [v2:172.21.15.61:6808/11541,v1:172.21.15.61:6809/11541] conn(0x7fac6c05c0d0 0x7fac6c05e4f0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.416+0000 7fac8759e700 1 -- 172.21.15.32:0/2987115115 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fac6c0553b0 msgr2=0x7fac6c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.416+0000 7fac8759e700 1 --2- 172.21.15.32:0/2987115115 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fac6c0553b0 0x7fac6c057870 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7fac78019000 tx=0x7fac78015000).stop 2020-12-05T18:12:16.422 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.416+0000 7fac8759e700 1 -- 172.21.15.32:0/2987115115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880b1e60 msgr2=0x7fac8814f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.416+0000 7fac8759e700 1 --2- 172.21.15.32:0/2987115115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880b1e60 0x7fac8814f920 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x7fac880653e0 tx=0x7fac800091a0).stop 2020-12-05T18:12:16.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.416+0000 7fac86d9d700 1 -- 172.21.15.32:0/2987115115 reap_dead start 2020-12-05T18:12:16.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.416+0000 7fac8759e700 1 -- 172.21.15.32:0/2987115115 shutdown_connections 2020-12-05T18:12:16.423 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.416+0000 7fac8759e700 1 -- 172.21.15.32:0/2987115115 >> 172.21.15.32:0/2987115115 conn(0x7fac880b5620 msgr2=0x7fac880b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:16.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.416+0000 7fac8759e700 1 -- 172.21.15.32:0/2987115115 shutdown_connections 2020-12-05T18:12:16.424 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.416+0000 7fac8759e700 1 -- 172.21.15.32:0/2987115115 wait complete. 2020-12-05T18:12:16.424 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T18:12:16.424 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-12-05T18:12:16.424 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T18:12:16.433 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 4 2020-12-05T18:12:16.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.544+0000 7f3ecbfff700 1 Processor -- start 2020-12-05T18:12:16.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.544+0000 7f3ecbfff700 1 -- start start 2020-12-05T18:12:16.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.544+0000 7f3ecbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ecc142f20 0x7f3ecc0af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.544+0000 7f3ecbfff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ecc0b2390 con 0x7f3ecc142570 2020-12-05T18:12:16.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.544+0000 7f3ecbfff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ecc062430 con 0x7f3ecc061cf0 2020-12-05T18:12:16.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.544+0000 7f3ecbfff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ecc13ca50 con 0x7f3ecc142f20 2020-12-05T18:12:16.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.544+0000 7f3ecb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ecc142f20 0x7f3ecc0af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.544+0000 7f3eca7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3ecc142570 0x7f3ecc0af0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44786/0 (socket says 172.21.15.32:44786) 2020-12-05T18:12:16.551 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.544+0000 7f3eca7fc700 1 -- 172.21.15.32:0/544940872 learned_addr learned my addr 172.21.15.32:0/544940872 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T18:12:16.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.544+0000 7f3ec9ffb700 1 -- 172.21.15.32:0/544940872 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3377347311 0 0) 0x7f3ecc062430 con 0x7f3ecc061cf0 2020-12-05T18:12:16.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.544+0000 7f3ecb7fe700 1 -- 172.21.15.32:0/544940872 >> v1:172.21.15.61:6789/0 conn(0x7f3ecc061cf0 legacy=0x7f3ecc0b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.544+0000 7f3ecb7fe700 1 -- 172.21.15.32:0/544940872 >> v1:172.21.15.32:6789/0 conn(0x7f3ecc142570 legacy=0x7f3ecc0af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.552 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.544+0000 7f3ecb7fe700 1 -- 172.21.15.32:0/544940872 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ecc0626d0 con 0x7f3ecc142f20 2020-12-05T18:12:16.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.544+0000 7f3ecb7fe700 1 --2- 172.21.15.32:0/544940872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ecc142f20 0x7f3ecc0af7e0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 rx=0x7f3eb40010d0 tx=0x7f3ec4006dc0).ready entity=mon.0 client_cookie=89b1632e13098e14 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:16.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.544+0000 7f3ec9ffb700 1 -- 172.21.15.32:0/544940872 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ec40096b0 con 0x7f3ecc142f20 2020-12-05T18:12:16.553 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.544+0000 7f3ec9ffb700 1 -- 172.21.15.32:0/544940872 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec4009810 con 0x7f3ecc142f20 2020-12-05T18:12:16.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.544+0000 7f3ec9ffb700 1 -- 172.21.15.32:0/544940872 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ec4009b70 con 0x7f3ecc142f20 2020-12-05T18:12:16.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.544+0000 7f3ecbfff700 1 -- 172.21.15.32:0/544940872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ecc142f20 msgr2=0x7f3ecc0af7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.554 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.544+0000 7f3ecbfff700 1 --2- 172.21.15.32:0/544940872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ecc142f20 0x7f3ecc0af7e0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 rx=0x7f3eb40010d0 tx=0x7f3ec4006dc0).stop 2020-12-05T18:12:16.557 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.548+0000 7f3ecbfff700 1 -- 172.21.15.32:0/544940872 shutdown_connections 2020-12-05T18:12:16.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.548+0000 7f3ecbfff700 1 --2- 172.21.15.32:0/544940872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ecc142f20 0x7f3ecc0af7e0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.548+0000 7f3ecbfff700 1 -- 172.21.15.32:0/544940872 >> 172.21.15.32:0/544940872 conn(0x7f3ecc0b5530 msgr2=0x7f3ecc0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:16.558 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.548+0000 7f3ecbfff700 1 -- 172.21.15.32:0/544940872 shutdown_connections 2020-12-05T18:12:16.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.548+0000 7f3ecbfff700 1 -- 172.21.15.32:0/544940872 wait complete. 2020-12-05T18:12:16.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.548+0000 7f3ecbfff700 1 Processor -- start 2020-12-05T18:12:16.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.548+0000 7f3ecbfff700 1 -- start start 2020-12-05T18:12:16.559 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.548+0000 7f3ecbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ecc061cf0 0x7f3ecc0631e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.548+0000 7f3ecbfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3ecc142570 0x7f3ecc151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.548+0000 7f3ecbfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ecc142f20 0x7f3ecc151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.560 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.548+0000 7f3ecbfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc13e640 con 0x7f3ecc061cf0 2020-12-05T18:12:16.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.548+0000 7f3ecbfff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc152460 con 0x7f3ecc142570 2020-12-05T18:12:16.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.548+0000 7f3ecbfff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc1462e0 con 0x7f3ecc142f20 2020-12-05T18:12:16.561 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.548+0000 7f3ecaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ecc061cf0 0x7f3ecc0631e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.548+0000 7f3ecaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ecc061cf0 0x7f3ecc0631e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42142/0 (socket says 172.21.15.32:42142) 2020-12-05T18:12:16.562 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.548+0000 7f3ecaffd700 1 -- 172.21.15.32:0/1259129484 learned_addr learned my addr 172.21.15.32:0/1259129484 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:16.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.548+0000 7f3eca7fc700 1 --2- 172.21.15.32:0/1259129484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3ecc142570 0x7f3ecc151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.548+0000 7f3ecb7fe700 1 --2- 172.21.15.32:0/1259129484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ecc142f20 0x7f3ecc151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.563 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.552+0000 7f3ecaffd700 1 -- 172.21.15.32:0/1259129484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ecc142f20 msgr2=0x7f3ecc151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.552+0000 7f3ecaffd700 1 --2- 172.21.15.32:0/1259129484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ecc142f20 0x7f3ecc151b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.564 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.552+0000 7f3ecaffd700 1 -- 172.21.15.32:0/1259129484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3ecc142570 msgr2=0x7f3ecc151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.552+0000 7f3ecaffd700 1 --2- 172.21.15.32:0/1259129484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3ecc142570 0x7f3ecc151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.565 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.552+0000 7f3ecaffd700 1 -- 172.21.15.32:0/1259129484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ecc0636d0 con 0x7f3ecc061cf0 2020-12-05T18:12:16.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.552+0000 7f3eca7fc700 1 --2- 172.21.15.32:0/1259129484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3ecc142570 0x7f3ecc151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T18:12:16.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.552+0000 7f3ecaffd700 1 --2- 172.21.15.32:0/1259129484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ecc061cf0 0x7f3ecc0631e0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 rx=0x7f3ebc001210 tx=0x7f3ebc007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:16.566 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.552+0000 7f3ec3fff700 1 -- 172.21.15.32:0/1259129484 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ebc00a7b0 con 0x7f3ecc061cf0 2020-12-05T18:12:16.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.552+0000 7f3ec3fff700 1 -- 172.21.15.32:0/1259129484 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ebc00a910 con 0x7f3ecc061cf0 2020-12-05T18:12:16.567 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.552+0000 7f3ecbfff700 1 -- 172.21.15.32:0/1259129484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ecc146780 con 0x7f3ecc061cf0 2020-12-05T18:12:16.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.552+0000 7f3ec3fff700 1 -- 172.21.15.32:0/1259129484 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ebc00aa80 con 0x7f3ecc061cf0 2020-12-05T18:12:16.568 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.552+0000 7f3ecbfff700 1 -- 172.21.15.32:0/1259129484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ecc152060 con 0x7f3ecc061cf0 2020-12-05T18:12:16.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.552+0000 7f3ecbfff700 1 -- 172.21.15.32:0/1259129484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ecc152620 con 0x7f3ecc061cf0 2020-12-05T18:12:16.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.552+0000 7f3ec3fff700 1 -- 172.21.15.32:0/1259129484 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3ebc010070 con 0x7f3ecc061cf0 2020-12-05T18:12:16.569 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.552+0000 7f3ec3fff700 1 --2- 172.21.15.32:0/1259129484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3eb00554e0 0x7f3eb00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.552+0000 7f3ec3fff700 1 -- 172.21.15.32:0/1259129484 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f3ebc06da80 con 0x7f3ecc061cf0 2020-12-05T18:12:16.570 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.552+0000 7f3eca7fc700 1 --2- 172.21.15.32:0/1259129484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3eb00554e0 0x7f3eb00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.571 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.552+0000 7f3eca7fc700 1 --2- 172.21.15.32:0/1259129484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3eb00554e0 0x7f3eb00579a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f3eb4008000 tx=0x7f3eb4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:16.572 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.560+0000 7f3ec3fff700 1 -- 172.21.15.32:0/1259129484 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3ebc043f00 con 0x7f3ecc061cf0 2020-12-05T18:12:16.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.704+0000 7f3ecbfff700 1 -- 172.21.15.32:0/1259129484 --> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ecc142db0 con 0x7f3eb00554e0 2020-12-05T18:12:16.709 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.704+0000 7f3ec3fff700 1 -- 172.21.15.32:0/1259129484 <== mgr.4101 v2:172.21.15.32:6800/11355 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3ecc142db0 con 0x7f3eb00554e0 2020-12-05T18:12:16.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.708+0000 7f3ecbfff700 1 -- 172.21.15.32:0/1259129484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3eb00554e0 msgr2=0x7f3eb00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.708+0000 7f3ecbfff700 1 --2- 172.21.15.32:0/1259129484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3eb00554e0 0x7f3eb00579a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f3eb4008000 tx=0x7f3eb4009040).stop 2020-12-05T18:12:16.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.708+0000 7f3ecbfff700 1 -- 172.21.15.32:0/1259129484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ecc061cf0 msgr2=0x7f3ecc0631e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.711 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.708+0000 7f3ecbfff700 1 --2- 172.21.15.32:0/1259129484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ecc061cf0 0x7f3ecc0631e0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 rx=0x7f3ebc001210 tx=0x7f3ebc007e10).stop 2020-12-05T18:12:16.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.708+0000 7f3ecbfff700 1 -- 172.21.15.32:0/1259129484 shutdown_connections 2020-12-05T18:12:16.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.708+0000 7f3ecbfff700 1 --2- 172.21.15.32:0/1259129484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f3eb00554e0 0x7f3eb00579a0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.708+0000 7f3ecbfff700 1 --2- 172.21.15.32:0/1259129484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ecc142f20 0x7f3ecc151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.708+0000 7f3ecbfff700 1 --2- 172.21.15.32:0/1259129484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ecc061cf0 0x7f3ecc0631e0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.712 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.708+0000 7f3ecbfff700 1 --2- 172.21.15.32:0/1259129484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3ecc142570 0x7f3ecc151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.708+0000 7f3ecbfff700 1 -- 172.21.15.32:0/1259129484 >> 172.21.15.32:0/1259129484 conn(0x7f3ecc0b5530 msgr2=0x7f3ecc140b20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:16.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.708+0000 7f3ecbfff700 1 -- 172.21.15.32:0/1259129484 shutdown_connections 2020-12-05T18:12:16.713 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.708+0000 7f3ecbfff700 1 -- 172.21.15.32:0/1259129484 wait complete. 2020-12-05T18:12:16.713 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 4 to deep-scrub 2020-12-05T18:12:16.722 INFO:tasks.ceph:Scrubbing osd.5 2020-12-05T18:12:16.723 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.5 config set osd_debug_deep_scrub_sleep 0 2020-12-05T18:12:16.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.836+0000 7f688f59e700 1 Processor -- start 2020-12-05T18:12:16.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.836+0000 7f688f59e700 1 -- start start 2020-12-05T18:12:16.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.836+0000 7f688f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68900b0d70 0x7f68900637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.836+0000 7f688f59e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f6890063ce0 0x7f6890064190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.836+0000 7f688f59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689005f5c0 con 0x7f68900af0d0 2020-12-05T18:12:16.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.836+0000 7f688f59e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68900af4b0 con 0x7f6890063ce0 2020-12-05T18:12:16.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.836+0000 7f688f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f689013e1b0 con 0x7f68900b0d70 2020-12-05T18:12:16.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.836+0000 7f688e59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f68900af0d0 0x7f689013daa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44798/0 (socket says 172.21.15.32:44798) 2020-12-05T18:12:16.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.836+0000 7f688e59c700 1 -- 172.21.15.32:0/1096858192 learned_addr learned my addr 172.21.15.32:0/1096858192 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T18:12:16.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.836+0000 7f688dd9b700 1 --2- 172.21.15.32:0/1096858192 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68900b0d70 0x7f68900637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.836+0000 7f688ed9d700 1 --2- 172.21.15.32:0/1096858192 >> v2:172.21.15.61:3300/0 conn(0x7f6890063ce0 0x7f6890064190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.836+0000 7f688d59a700 1 -- 172.21.15.32:0/1096858192 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3192096361 0 0) 0x7f689005f5c0 con 0x7f68900af0d0 2020-12-05T18:12:16.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.836+0000 7f688d59a700 1 -- 172.21.15.32:0/1096858192 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6874003230 con 0x7f68900af0d0 2020-12-05T18:12:16.845 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.840+0000 7f688dd9b700 1 -- 172.21.15.32:0/1096858192 >> v2:172.21.15.61:3300/0 conn(0x7f6890063ce0 msgr2=0x7f6890064190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T18:12:16.845 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.840+0000 7f688dd9b700 1 --2- 172.21.15.32:0/1096858192 >> v2:172.21.15.61:3300/0 conn(0x7f6890063ce0 0x7f6890064190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.840+0000 7f688dd9b700 1 -- 172.21.15.32:0/1096858192 >> v1:172.21.15.32:6789/0 conn(0x7f68900af0d0 legacy=0x7f689013daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.840+0000 7f688dd9b700 1 -- 172.21.15.32:0/1096858192 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689005f7f0 con 0x7f68900b0d70 2020-12-05T18:12:16.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.840+0000 7f688ed9d700 1 --2- 172.21.15.32:0/1096858192 >> v2:172.21.15.61:3300/0 conn(0x7f6890063ce0 0x7f6890064190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T18:12:16.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.840+0000 7f688dd9b700 1 --2- 172.21.15.32:0/1096858192 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68900b0d70 0x7f68900637f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f68780032b0 tx=0x7f68780091a0).ready entity=mon.2 client_cookie=9133c4af76a19a4 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:16.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.840+0000 7f688d59a700 1 -- 172.21.15.32:0/1096858192 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f687800b940 con 0x7f68900b0d70 2020-12-05T18:12:16.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.840+0000 7f688d59a700 1 -- 172.21.15.32:0/1096858192 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687800baa0 con 0x7f68900b0d70 2020-12-05T18:12:16.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.840+0000 7f688d59a700 1 -- 172.21.15.32:0/1096858192 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f687800bc10 con 0x7f68900b0d70 2020-12-05T18:12:16.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.840+0000 7f688f59e700 1 -- 172.21.15.32:0/1096858192 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68900b0d70 msgr2=0x7f68900637f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.840+0000 7f688f59e700 1 --2- 172.21.15.32:0/1096858192 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68900b0d70 0x7f68900637f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f68780032b0 tx=0x7f68780091a0).stop 2020-12-05T18:12:16.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.840+0000 7f688f59e700 1 -- 172.21.15.32:0/1096858192 shutdown_connections 2020-12-05T18:12:16.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.840+0000 7f688f59e700 1 --2- 172.21.15.32:0/1096858192 >> v2:172.21.15.61:3300/0 conn(0x7f6890063ce0 0x7f6890064190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.848 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.840+0000 7f688f59e700 1 --2- 172.21.15.32:0/1096858192 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68900b0d70 0x7f68900637f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.840+0000 7f688f59e700 1 -- 172.21.15.32:0/1096858192 >> 172.21.15.32:0/1096858192 conn(0x7f68900620b0 msgr2=0x7f6890062490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:16.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.840+0000 7f688f59e700 1 -- 172.21.15.32:0/1096858192 shutdown_connections 2020-12-05T18:12:16.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.840+0000 7f688f59e700 1 -- 172.21.15.32:0/1096858192 wait complete. 2020-12-05T18:12:16.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.840+0000 7f688f59e700 1 Processor -- start 2020-12-05T18:12:16.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.840+0000 7f688f59e700 1 -- start start 2020-12-05T18:12:16.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.844+0000 7f688f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6890063ce0 0x7f6890143000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.844+0000 7f688f59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68900af0d0 0x7f6890065d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.844+0000 7f688f59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68900b0d70 0x7f6890066280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.844+0000 7f688f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68900656a0 con 0x7f6890063ce0 2020-12-05T18:12:16.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.844+0000 7f688f59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6890143df0 con 0x7f68900b0d70 2020-12-05T18:12:16.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.844+0000 7f688f59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6890144090 con 0x7f68900af0d0 2020-12-05T18:12:16.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.844+0000 7f688e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6890063ce0 0x7f6890143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.844+0000 7f688e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6890063ce0 0x7f6890143000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42156/0 (socket says 172.21.15.32:42156) 2020-12-05T18:12:16.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.844+0000 7f688e59c700 1 -- 172.21.15.32:0/1544496481 learned_addr learned my addr 172.21.15.32:0/1544496481 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:16.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.844+0000 7f688dd9b700 1 --2- 172.21.15.32:0/1544496481 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68900af0d0 0x7f6890065d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.844+0000 7f688ed9d700 1 --2- 172.21.15.32:0/1544496481 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68900b0d70 0x7f6890066280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.844+0000 7f688e59c700 1 -- 172.21.15.32:0/1544496481 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68900af0d0 msgr2=0x7f6890065d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.844+0000 7f688e59c700 1 --2- 172.21.15.32:0/1544496481 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68900af0d0 0x7f6890065d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.844+0000 7f688e59c700 1 -- 172.21.15.32:0/1544496481 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68900b0d70 msgr2=0x7f6890066280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.844+0000 7f688e59c700 1 --2- 172.21.15.32:0/1544496481 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68900b0d70 0x7f6890066280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.844+0000 7f688e59c700 1 -- 172.21.15.32:0/1544496481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6890144330 con 0x7f6890063ce0 2020-12-05T18:12:16.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.844+0000 7f688ed9d700 1 --2- 172.21.15.32:0/1544496481 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68900b0d70 0x7f6890066280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T18:12:16.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.844+0000 7f688e59c700 1 --2- 172.21.15.32:0/1544496481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6890063ce0 0x7f6890143000 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7f68800010d0 tx=0x7f6880007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:16.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.844+0000 7f68877fe700 1 -- 172.21.15.32:0/1544496481 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6880007320 con 0x7f6890063ce0 2020-12-05T18:12:16.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.844+0000 7f68877fe700 1 -- 172.21.15.32:0/1544496481 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6880007480 con 0x7f6890063ce0 2020-12-05T18:12:16.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.844+0000 7f68877fe700 1 -- 172.21.15.32:0/1544496481 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68800057d0 con 0x7f6890063ce0 2020-12-05T18:12:16.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.844+0000 7f688f59e700 1 -- 172.21.15.32:0/1544496481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6890064720 con 0x7f6890063ce0 2020-12-05T18:12:16.857 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.844+0000 7f688f59e700 1 -- 172.21.15.32:0/1544496481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f689013ce50 con 0x7f6890063ce0 2020-12-05T18:12:16.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.848+0000 7f688f59e700 1 -- 172.21.15.32:0/1544496481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f68900648b0 con 0x7f6890063ce0 2020-12-05T18:12:16.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.848+0000 7f68877fe700 1 -- 172.21.15.32:0/1544496481 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6880010030 con 0x7f6890063ce0 2020-12-05T18:12:16.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.848+0000 7f68877fe700 1 --2- 172.21.15.32:0/1544496481 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6868055440 0x7f6868057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.858 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.848+0000 7f68877fe700 1 -- 172.21.15.32:0/1544496481 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f688006db30 con 0x7f6890063ce0 2020-12-05T18:12:16.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.848+0000 7f688dd9b700 1 --2- 172.21.15.32:0/1544496481 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6868055440 0x7f6868057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.848+0000 7f68877fe700 1 --2- 172.21.15.32:0/1544496481 >> [v2:172.21.15.61:6800/11539,v1:172.21.15.61:6801/11539] conn(0x7f686805c1b0 0x7f686805e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:16.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.848+0000 7f68877fe700 1 -- 172.21.15.32:0/1544496481 --> [v2:172.21.15.61:6800/11539,v1:172.21.15.61:6801/11539] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f686805ebb0 con 0x7f686805c1b0 2020-12-05T18:12:16.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.848+0000 7f68877fe700 1 -- 172.21.15.32:0/1544496481 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68800721e0 con 0x7f6890063ce0 2020-12-05T18:12:16.859 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.848+0000 7f688dd9b700 1 --2- 172.21.15.32:0/1544496481 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6868055440 0x7f6868057900 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f687801c000 tx=0x7f6878016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:16.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.848+0000 7f688ed9d700 1 --2- 172.21.15.32:0/1544496481 >> [v2:172.21.15.61:6800/11539,v1:172.21.15.61:6801/11539] conn(0x7f686805c1b0 0x7f686805e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:16.860 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.848+0000 7f688ed9d700 1 --2- 172.21.15.32:0/1544496481 >> [v2:172.21.15.61:6800/11539,v1:172.21.15.61:6801/11539] conn(0x7f686805c1b0 0x7f686805e5d0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:16.861 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.852+0000 7f68877fe700 1 -- 172.21.15.32:0/1544496481 <== osd.5 v2:172.21.15.61:6800/11539 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f686805ebb0 con 0x7f686805c1b0 2020-12-05T18:12:16.875 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.872+0000 7f688f59e700 1 -- 172.21.15.32:0/1544496481 --> [v2:172.21.15.61:6800/11539,v1:172.21.15.61:6801/11539] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6890057050 con 0x7f686805c1b0 2020-12-05T18:12:16.876 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.872+0000 7f68877fe700 1 -- 172.21.15.32:0/1544496481 <== osd.5 v2:172.21.15.61:6800/11539 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f6890057050 con 0x7f686805c1b0 2020-12-05T18:12:16.876 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T18:12:16.877 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-12-05T18:12:16.877 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T18:12:16.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.872+0000 7f688f59e700 1 -- 172.21.15.32:0/1544496481 >> [v2:172.21.15.61:6800/11539,v1:172.21.15.61:6801/11539] conn(0x7f686805c1b0 msgr2=0x7f686805e5d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.872+0000 7f688f59e700 1 --2- 172.21.15.32:0/1544496481 >> [v2:172.21.15.61:6800/11539,v1:172.21.15.61:6801/11539] conn(0x7f686805c1b0 0x7f686805e5d0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:16.878 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.872+0000 7f688f59e700 1 -- 172.21.15.32:0/1544496481 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6868055440 msgr2=0x7f6868057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.872+0000 7f688f59e700 1 --2- 172.21.15.32:0/1544496481 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f6868055440 0x7f6868057900 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f687801c000 tx=0x7f6878016000).stop 2020-12-05T18:12:16.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.872+0000 7f688f59e700 1 -- 172.21.15.32:0/1544496481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6890063ce0 msgr2=0x7f6890143000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:16.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.872+0000 7f688f59e700 1 --2- 172.21.15.32:0/1544496481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6890063ce0 0x7f6890143000 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7f68800010d0 tx=0x7f6880007b50).stop 2020-12-05T18:12:16.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.872+0000 7f688ed9d700 1 -- 172.21.15.32:0/1544496481 reap_dead start 2020-12-05T18:12:16.879 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.872+0000 7f688f59e700 1 -- 172.21.15.32:0/1544496481 shutdown_connections 2020-12-05T18:12:16.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.872+0000 7f688f59e700 1 -- 172.21.15.32:0/1544496481 >> 172.21.15.32:0/1544496481 conn(0x7f68900620b0 msgr2=0x7f68900b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:16.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.872+0000 7f688f59e700 1 -- 172.21.15.32:0/1544496481 shutdown_connections 2020-12-05T18:12:16.880 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:16.872+0000 7f688f59e700 1 -- 172.21.15.32:0/1544496481 wait complete. 2020-12-05T18:12:16.889 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 5 2020-12-05T18:12:17.009 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.004+0000 7f939ed85700 1 Processor -- start 2020-12-05T18:12:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.004+0000 7f939ed85700 1 -- start start 2020-12-05T18:12:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.008+0000 7f939ed85700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93981435f0 0x7f93980ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.008+0000 7f939ed85700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9398142ab0 con 0x7f93980b14e0 2020-12-05T18:12:17.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.008+0000 7f939ed85700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93980b62e0 con 0x7f93980b5f00 2020-12-05T18:12:17.012 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.008+0000 7f939ed85700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93980afbf0 con 0x7f93981435f0 2020-12-05T18:12:17.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.008+0000 7f939d582700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f93980b14e0 0x7f9398142ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44816/0 (socket says 172.21.15.32:44816) 2020-12-05T18:12:17.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.008+0000 7f939d582700 1 -- 172.21.15.32:0/3089444851 learned_addr learned my addr 172.21.15.32:0/3089444851 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T18:12:17.013 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.008+0000 7f939e584700 1 --2- 172.21.15.32:0/3089444851 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93981435f0 0x7f93980ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.008+0000 7f939cd81700 1 -- 172.21.15.32:0/3089444851 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3411655239 0 0) 0x7f9398142ab0 con 0x7f93980b14e0 2020-12-05T18:12:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.008+0000 7f939cd81700 1 -- 172.21.15.32:0/3089444851 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9384003230 con 0x7f93980b14e0 2020-12-05T18:12:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.008+0000 7f939cd81700 1 -- 172.21.15.32:0/3089444851 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 975857158 0 0) 0x7f93980b62e0 con 0x7f93980b5f00 2020-12-05T18:12:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.008+0000 7f939cd81700 1 -- 172.21.15.32:0/3089444851 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9398142ab0 con 0x7f93980b5f00 2020-12-05T18:12:17.016 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.008+0000 7f939e584700 1 -- 172.21.15.32:0/3089444851 >> v1:172.21.15.61:6789/0 conn(0x7f93980b5f00 legacy=0x7f9398066370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.008+0000 7f939e584700 1 -- 172.21.15.32:0/3089444851 >> v1:172.21.15.32:6789/0 conn(0x7f93980b14e0 legacy=0x7f9398142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.008+0000 7f939e584700 1 -- 172.21.15.32:0/3089444851 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93980afe90 con 0x7f93981435f0 2020-12-05T18:12:17.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.008+0000 7f939e584700 1 --2- 172.21.15.32:0/3089444851 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93981435f0 0x7f93980ad510 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f9390002f30 tx=0x7f9390006dc0).ready entity=mon.2 client_cookie=24fb183d2dfce1dd server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:17.017 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.008+0000 7f939cd81700 1 -- 172.21.15.32:0/3089444851 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9390009730 con 0x7f93981435f0 2020-12-05T18:12:17.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.008+0000 7f939cd81700 1 -- 172.21.15.32:0/3089444851 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9390009890 con 0x7f93981435f0 2020-12-05T18:12:17.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.008+0000 7f939cd81700 1 -- 172.21.15.32:0/3089444851 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9390009bf0 con 0x7f93981435f0 2020-12-05T18:12:17.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.008+0000 7f939ed85700 1 -- 172.21.15.32:0/3089444851 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93981435f0 msgr2=0x7f93980ad510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.018 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.008+0000 7f939ed85700 1 --2- 172.21.15.32:0/3089444851 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93981435f0 0x7f93980ad510 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f9390002f30 tx=0x7f9390006dc0).stop 2020-12-05T18:12:17.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.012+0000 7f939ed85700 1 -- 172.21.15.32:0/3089444851 shutdown_connections 2020-12-05T18:12:17.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.012+0000 7f939ed85700 1 --2- 172.21.15.32:0/3089444851 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93981435f0 0x7f93980ad510 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.012+0000 7f939ed85700 1 -- 172.21.15.32:0/3089444851 >> 172.21.15.32:0/3089444851 conn(0x7f93980604c0 msgr2=0x7f939805c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:17.019 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.012+0000 7f939ed85700 1 -- 172.21.15.32:0/3089444851 shutdown_connections 2020-12-05T18:12:17.020 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.012+0000 7f939ed85700 1 -- 172.21.15.32:0/3089444851 wait complete. 2020-12-05T18:12:17.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.012+0000 7f939ed85700 1 Processor -- start 2020-12-05T18:12:17.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.012+0000 7f939ed85700 1 -- start start 2020-12-05T18:12:17.022 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.016+0000 7f939ed85700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f93980b14e0 0x7f93981497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.016+0000 7f939ed85700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93980b5f00 0x7f9398140bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.016+0000 7f939ed85700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93981435f0 0x7f93981460e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.016+0000 7f939ed85700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9398144e40 con 0x7f93980b5f00 2020-12-05T18:12:17.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.016+0000 7f939ed85700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9398144ba0 con 0x7f93980b14e0 2020-12-05T18:12:17.023 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.016+0000 7f939ed85700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9398144820 con 0x7f93981435f0 2020-12-05T18:12:17.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.016+0000 7f939d582700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93980b5f00 0x7f9398140bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.024 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.016+0000 7f939d582700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93980b5f00 0x7f9398140bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42174/0 (socket says 172.21.15.32:42174) 2020-12-05T18:12:17.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.016+0000 7f939d582700 1 -- 172.21.15.32:0/2171780100 learned_addr learned my addr 172.21.15.32:0/2171780100 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:17.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.016+0000 7f939dd83700 1 --2- 172.21.15.32:0/2171780100 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f93980b14e0 0x7f93981497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.016+0000 7f939e584700 1 --2- 172.21.15.32:0/2171780100 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93981435f0 0x7f93981460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.016+0000 7f939d582700 1 -- 172.21.15.32:0/2171780100 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93981435f0 msgr2=0x7f93981460e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.025 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.016+0000 7f939d582700 1 --2- 172.21.15.32:0/2171780100 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93981435f0 0x7f93981460e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.016+0000 7f939d582700 1 -- 172.21.15.32:0/2171780100 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f93980b14e0 msgr2=0x7f93981497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.016+0000 7f939d582700 1 --2- 172.21.15.32:0/2171780100 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f93980b14e0 0x7f93981497d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.016+0000 7f939d582700 1 -- 172.21.15.32:0/2171780100 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9398145450 con 0x7f93980b5f00 2020-12-05T18:12:17.026 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.016+0000 7f939d582700 1 --2- 172.21.15.32:0/2171780100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93980b5f00 0x7f9398140bc0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7f938c001190 tx=0x7f938c007b20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:17.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.016+0000 7f9396ffd700 1 -- 172.21.15.32:0/2171780100 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f938c0072e0 con 0x7f93980b5f00 2020-12-05T18:12:17.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.016+0000 7f939ed85700 1 -- 172.21.15.32:0/2171780100 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9398145700 con 0x7f93980b5f00 2020-12-05T18:12:17.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.016+0000 7f9396ffd700 1 -- 172.21.15.32:0/2171780100 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938c007440 con 0x7f93980b5f00 2020-12-05T18:12:17.027 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.016+0000 7f9396ffd700 1 -- 172.21.15.32:0/2171780100 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f938c0057d0 con 0x7f93980b5f00 2020-12-05T18:12:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.016+0000 7f939ed85700 1 -- 172.21.15.32:0/2171780100 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9398146770 con 0x7f93980b5f00 2020-12-05T18:12:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.020+0000 7f9396ffd700 1 -- 172.21.15.32:0/2171780100 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f938c010070 con 0x7f93980b5f00 2020-12-05T18:12:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.020+0000 7f939ed85700 1 -- 172.21.15.32:0/2171780100 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93981449e0 con 0x7f93980b5f00 2020-12-05T18:12:17.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.020+0000 7f9396ffd700 1 --2- 172.21.15.32:0/2171780100 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9378055440 0x7f9378057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.020+0000 7f9396ffd700 1 -- 172.21.15.32:0/2171780100 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f938c06d4e0 con 0x7f93980b5f00 2020-12-05T18:12:17.029 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.020+0000 7f939dd83700 1 --2- 172.21.15.32:0/2171780100 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9378055440 0x7f9378057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.020+0000 7f939dd83700 1 --2- 172.21.15.32:0/2171780100 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9378055440 0x7f9378057900 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7f938800d000 tx=0x7f9388007d70).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:17.030 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.024+0000 7f9396ffd700 1 -- 172.21.15.32:0/2171780100 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f938c0438c0 con 0x7f93980b5f00 2020-12-05T18:12:17.172 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.168+0000 7f939ed85700 1 -- 172.21.15.32:0/2171780100 --> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f9398057050 con 0x7f9378055440 2020-12-05T18:12:17.173 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.168+0000 7f9396ffd700 1 -- 172.21.15.32:0/2171780100 <== mgr.4101 v2:172.21.15.32:6800/11355 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9398057050 con 0x7f9378055440 2020-12-05T18:12:17.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.172+0000 7f939ed85700 1 -- 172.21.15.32:0/2171780100 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9378055440 msgr2=0x7f9378057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.172+0000 7f939ed85700 1 --2- 172.21.15.32:0/2171780100 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9378055440 0x7f9378057900 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7f938800d000 tx=0x7f9388007d70).stop 2020-12-05T18:12:17.175 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.172+0000 7f939ed85700 1 -- 172.21.15.32:0/2171780100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93980b5f00 msgr2=0x7f9398140bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.172+0000 7f939ed85700 1 --2- 172.21.15.32:0/2171780100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93980b5f00 0x7f9398140bc0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7f938c001190 tx=0x7f938c007b20).stop 2020-12-05T18:12:17.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.172+0000 7f939ed85700 1 -- 172.21.15.32:0/2171780100 shutdown_connections 2020-12-05T18:12:17.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.172+0000 7f939ed85700 1 --2- 172.21.15.32:0/2171780100 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f9378055440 0x7f9378057900 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.172+0000 7f939ed85700 1 --2- 172.21.15.32:0/2171780100 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93981435f0 0x7f93981460e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.176 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.172+0000 7f939ed85700 1 --2- 172.21.15.32:0/2171780100 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f93980b14e0 0x7f93981497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.172+0000 7f939ed85700 1 --2- 172.21.15.32:0/2171780100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93980b5f00 0x7f9398140bc0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.172+0000 7f939ed85700 1 -- 172.21.15.32:0/2171780100 >> 172.21.15.32:0/2171780100 conn(0x7f93980604c0 msgr2=0x7f939813fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:17.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.172+0000 7f939ed85700 1 -- 172.21.15.32:0/2171780100 shutdown_connections 2020-12-05T18:12:17.177 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.172+0000 7f939ed85700 1 -- 172.21.15.32:0/2171780100 wait complete. 2020-12-05T18:12:17.178 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 5 to deep-scrub 2020-12-05T18:12:17.187 INFO:tasks.ceph:Scrubbing osd.6 2020-12-05T18:12:17.187 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.6 config set osd_debug_deep_scrub_sleep 0 2020-12-05T18:12:17.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.308+0000 7f108697c700 1 Processor -- start 2020-12-05T18:12:17.313 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.308+0000 7f108697c700 1 -- start start 2020-12-05T18:12:17.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f108697c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f108005f3b0 0x7f108005f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f108697c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f10800b2fb0 0x7f1080061780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.315 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f108697c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10800b2df0 con 0x7f108005feb0 2020-12-05T18:12:17.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f108697c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10800b3c90 con 0x7f108005f3b0 2020-12-05T18:12:17.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f108697c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10800b2b50 con 0x7f10800b2fb0 2020-12-05T18:12:17.316 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f108597a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f108005f3b0 0x7f108005f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f1085179700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f108005feb0 0x7f108013c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44830/0 (socket says 172.21.15.32:44830) 2020-12-05T18:12:17.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f1085179700 1 -- 172.21.15.32:0/1495789633 learned_addr learned my addr 172.21.15.32:0/1495789633 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T18:12:17.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f108617b700 1 --2- 172.21.15.32:0/1495789633 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f10800b2fb0 0x7f1080061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.317 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f108597a700 1 -- 172.21.15.32:0/1495789633 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f10800b2fb0 msgr2=0x7f1080061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T18:12:17.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f108597a700 1 --2- 172.21.15.32:0/1495789633 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f10800b2fb0 0x7f1080061780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f1084978700 1 -- 172.21.15.32:0/1495789633 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 431417104 0 0) 0x7f10800b2df0 con 0x7f108005feb0 2020-12-05T18:12:17.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f108597a700 1 -- 172.21.15.32:0/1495789633 >> v1:172.21.15.32:6789/0 conn(0x7f108005feb0 legacy=0x7f108013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.318 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f108597a700 1 -- 172.21.15.32:0/1495789633 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1080142820 con 0x7f108005f3b0 2020-12-05T18:12:17.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f108617b700 1 --2- 172.21.15.32:0/1495789633 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f10800b2fb0 0x7f1080061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T18:12:17.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f108597a700 1 --2- 172.21.15.32:0/1495789633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f108005f3b0 0x7f108005f790 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 rx=0x7f107c002f30 tx=0x7f107c006dc0).ready entity=mon.0 client_cookie=df926a210ca54801 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:17.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f1084978700 1 -- 172.21.15.32:0/1495789633 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f107c0097f0 con 0x7f108005f3b0 2020-12-05T18:12:17.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f1084978700 1 -- 172.21.15.32:0/1495789633 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c009950 con 0x7f108005f3b0 2020-12-05T18:12:17.319 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f108697c700 1 -- 172.21.15.32:0/1495789633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f108005f3b0 msgr2=0x7f108005f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f1084978700 1 -- 172.21.15.32:0/1495789633 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f107c009cb0 con 0x7f108005f3b0 2020-12-05T18:12:17.320 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f108697c700 1 --2- 172.21.15.32:0/1495789633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f108005f3b0 0x7f108005f790 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 rx=0x7f107c002f30 tx=0x7f107c006dc0).stop 2020-12-05T18:12:17.323 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f108697c700 1 -- 172.21.15.32:0/1495789633 shutdown_connections 2020-12-05T18:12:17.324 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f108697c700 1 --2- 172.21.15.32:0/1495789633 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f10800b2fb0 0x7f1080061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.324 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f108697c700 1 --2- 172.21.15.32:0/1495789633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f108005f3b0 0x7f108005f790 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.324 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f108697c700 1 -- 172.21.15.32:0/1495789633 >> 172.21.15.32:0/1495789633 conn(0x7f10800b8f20 msgr2=0x7f10800b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:17.324 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f108697c700 1 -- 172.21.15.32:0/1495789633 shutdown_connections 2020-12-05T18:12:17.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f108697c700 1 -- 172.21.15.32:0/1495789633 wait complete. 2020-12-05T18:12:17.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f108697c700 1 Processor -- start 2020-12-05T18:12:17.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.312+0000 7f108697c700 1 -- start start 2020-12-05T18:12:17.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f108697c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f108005f3b0 0x7f108013e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.325 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f108697c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f108005feb0 0x7f10801460d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f108697c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f10800b2fb0 0x7f10801465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f108697c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1080143f20 con 0x7f108005f3b0 2020-12-05T18:12:17.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f108697c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1080142640 con 0x7f10800b2fb0 2020-12-05T18:12:17.326 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f108697c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1080143c80 con 0x7f108005feb0 2020-12-05T18:12:17.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f108597a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f108005f3b0 0x7f108013e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f108597a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f108005f3b0 0x7f108013e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42186/0 (socket says 172.21.15.32:42186) 2020-12-05T18:12:17.327 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f108597a700 1 -- 172.21.15.32:0/2196665929 learned_addr learned my addr 172.21.15.32:0/2196665929 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:17.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f108617b700 1 --2- 172.21.15.32:0/2196665929 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f10800b2fb0 0x7f10801465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f108597a700 1 -- 172.21.15.32:0/2196665929 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f108005feb0 msgr2=0x7f10801460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f1085179700 1 --2- 172.21.15.32:0/2196665929 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f108005feb0 0x7f10801460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.328 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f108597a700 1 --2- 172.21.15.32:0/2196665929 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f108005feb0 0x7f10801460d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f108597a700 1 -- 172.21.15.32:0/2196665929 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f10800b2fb0 msgr2=0x7f10801465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f108597a700 1 --2- 172.21.15.32:0/2196665929 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f10800b2fb0 0x7f10801465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f108597a700 1 -- 172.21.15.32:0/2196665929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1080142330 con 0x7f108005f3b0 2020-12-05T18:12:17.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f108617b700 1 --2- 172.21.15.32:0/2196665929 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f10800b2fb0 0x7f10801465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T18:12:17.329 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f1085179700 1 --2- 172.21.15.32:0/2196665929 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f108005feb0 0x7f10801460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-12-05T18:12:17.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f108597a700 1 --2- 172.21.15.32:0/2196665929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f108005f3b0 0x7f108013e700 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 rx=0x7f1080145c30 tx=0x7f107c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:17.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f106effd700 1 -- 172.21.15.32:0/2196665929 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f107c011c40 con 0x7f108005f3b0 2020-12-05T18:12:17.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f106effd700 1 -- 172.21.15.32:0/2196665929 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c011da0 con 0x7f108005f3b0 2020-12-05T18:12:17.330 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f108697c700 1 -- 172.21.15.32:0/2196665929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1080146ab0 con 0x7f108005f3b0 2020-12-05T18:12:17.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f106effd700 1 -- 172.21.15.32:0/2196665929 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f107c01d640 con 0x7f108005f3b0 2020-12-05T18:12:17.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f108697c700 1 -- 172.21.15.32:0/2196665929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1080141180 con 0x7f108005f3b0 2020-12-05T18:12:17.331 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.316+0000 7f108697c700 1 -- 172.21.15.32:0/2196665929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f10801424c0 con 0x7f108005f3b0 2020-12-05T18:12:17.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.320+0000 7f106effd700 1 -- 172.21.15.32:0/2196665929 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f107c024030 con 0x7f108005f3b0 2020-12-05T18:12:17.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.320+0000 7f106effd700 1 --2- 172.21.15.32:0/2196665929 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1060055440 0x7f1060057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.320+0000 7f106effd700 1 -- 172.21.15.32:0/2196665929 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f107c06eff0 con 0x7f108005f3b0 2020-12-05T18:12:17.332 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.320+0000 7f1085179700 1 --2- 172.21.15.32:0/2196665929 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1060055440 0x7f1060057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.320+0000 7f106effd700 1 --2- 172.21.15.32:0/2196665929 >> [v2:172.21.15.61:6820/11542,v1:172.21.15.61:6822/11542] conn(0x7f106005c160 0x7f106005e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.320+0000 7f106effd700 1 -- 172.21.15.32:0/2196665929 --> [v2:172.21.15.61:6820/11542,v1:172.21.15.61:6822/11542] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f106005eb60 con 0x7f106005c160 2020-12-05T18:12:17.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.320+0000 7f106effd700 1 -- 172.21.15.32:0/2196665929 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) v2 ==== 24+0+0 (secure 0 0 0) 0x7f107c071660 con 0x7f108005f3b0 2020-12-05T18:12:17.333 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.320+0000 7f1085179700 1 --2- 172.21.15.32:0/2196665929 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1060055440 0x7f1060057900 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f1070008000 tx=0x7f1070009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:17.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.320+0000 7f108617b700 1 --2- 172.21.15.32:0/2196665929 >> [v2:172.21.15.61:6820/11542,v1:172.21.15.61:6822/11542] conn(0x7f106005c160 0x7f106005e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.334 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.320+0000 7f108617b700 1 --2- 172.21.15.32:0/2196665929 >> [v2:172.21.15.61:6820/11542,v1:172.21.15.61:6822/11542] conn(0x7f106005c160 0x7f106005e580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:17.335 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.324+0000 7f106effd700 1 -- 172.21.15.32:0/2196665929 <== osd.6 v2:172.21.15.61:6820/11542 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f106005eb60 con 0x7f106005c160 2020-12-05T18:12:17.345 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.340+0000 7f108697c700 1 -- 172.21.15.32:0/2196665929 --> [v2:172.21.15.61:6820/11542,v1:172.21.15.61:6822/11542] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1080057050 con 0x7f106005c160 2020-12-05T18:12:17.346 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.344+0000 7f106effd700 1 -- 172.21.15.32:0/2196665929 <== osd.6 v2:172.21.15.61:6820/11542 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f1080057050 con 0x7f106005c160 2020-12-05T18:12:17.347 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T18:12:17.347 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-12-05T18:12:17.347 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T18:12:17.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.344+0000 7f108697c700 1 -- 172.21.15.32:0/2196665929 >> [v2:172.21.15.61:6820/11542,v1:172.21.15.61:6822/11542] conn(0x7f106005c160 msgr2=0x7f106005e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.348 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.344+0000 7f108697c700 1 --2- 172.21.15.32:0/2196665929 >> [v2:172.21.15.61:6820/11542,v1:172.21.15.61:6822/11542] conn(0x7f106005c160 0x7f106005e580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.344+0000 7f108697c700 1 -- 172.21.15.32:0/2196665929 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1060055440 msgr2=0x7f1060057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.344+0000 7f108697c700 1 --2- 172.21.15.32:0/2196665929 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f1060055440 0x7f1060057900 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f1070008000 tx=0x7f1070009040).stop 2020-12-05T18:12:17.349 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.344+0000 7f108697c700 1 -- 172.21.15.32:0/2196665929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f108005f3b0 msgr2=0x7f108013e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.344+0000 7f108697c700 1 --2- 172.21.15.32:0/2196665929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f108005f3b0 0x7f108013e700 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 rx=0x7f1080145c30 tx=0x7f107c015000).stop 2020-12-05T18:12:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.344+0000 7f108617b700 1 -- 172.21.15.32:0/2196665929 reap_dead start 2020-12-05T18:12:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.344+0000 7f108697c700 1 -- 172.21.15.32:0/2196665929 shutdown_connections 2020-12-05T18:12:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.344+0000 7f108697c700 1 -- 172.21.15.32:0/2196665929 >> 172.21.15.32:0/2196665929 conn(0x7f10800b8f20 msgr2=0x7f10800b52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:17.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.344+0000 7f108697c700 1 -- 172.21.15.32:0/2196665929 shutdown_connections 2020-12-05T18:12:17.351 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.344+0000 7f108697c700 1 -- 172.21.15.32:0/2196665929 wait complete. 2020-12-05T18:12:17.360 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 6 2020-12-05T18:12:17.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.472+0000 7fa5bb2f5700 1 Processor -- start 2020-12-05T18:12:17.476 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.472+0000 7fa5bb2f5700 1 -- start start 2020-12-05T18:12:17.477 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.472+0000 7fa5bb2f5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5b405feb0 0x7fa5b405f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.472+0000 7fa5bb2f5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b4142450 con 0x7fa5b405f3b0 2020-12-05T18:12:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.472+0000 7fa5bb2f5700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b4061780 con 0x7fa5b4062400 2020-12-05T18:12:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.472+0000 7fa5bb2f5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b4061a20 con 0x7fa5b405feb0 2020-12-05T18:12:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.472+0000 7fa5baaf4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5b405feb0 0x7fa5b405f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.472+0000 7fa5baaf4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5b405feb0 0x7fa5b405f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42200/0 (socket says 172.21.15.32:42200) 2020-12-05T18:12:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.472+0000 7fa5baaf4700 1 -- 172.21.15.32:0/2487973162 learned_addr learned my addr 172.21.15.32:0/2487973162 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:17.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.476+0000 7fa5baaf4700 1 -- 172.21.15.32:0/2487973162 >> v1:172.21.15.61:6789/0 conn(0x7fa5b4062400 legacy=0x7fa5b40b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.476+0000 7fa5baaf4700 1 -- 172.21.15.32:0/2487973162 >> v1:172.21.15.32:6789/0 conn(0x7fa5b405f3b0 legacy=0x7fa5b40b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.476+0000 7fa5b92f1700 1 -- 172.21.15.32:0/2487973162 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059424888 0 0) 0x7fa5b4061780 con 0x7fa5b4062400 2020-12-05T18:12:17.481 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.476+0000 7fa5baaf4700 1 -- 172.21.15.32:0/2487973162 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b4061cc0 con 0x7fa5b405feb0 2020-12-05T18:12:17.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.476+0000 7fa5baaf4700 1 --2- 172.21.15.32:0/2487973162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5b405feb0 0x7fa5b405f790 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 rx=0x7fa5b00010d0 tx=0x7fa5ac009270).ready entity=mon.0 client_cookie=df65c543a760474c server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:17.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.476+0000 7fa5b92f1700 1 -- 172.21.15.32:0/2487973162 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5ac00bac0 con 0x7fa5b405feb0 2020-12-05T18:12:17.482 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.476+0000 7fa5b92f1700 1 -- 172.21.15.32:0/2487973162 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ac00bc20 con 0x7fa5b405feb0 2020-12-05T18:12:17.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.476+0000 7fa5b92f1700 1 -- 172.21.15.32:0/2487973162 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5ac00bd90 con 0x7fa5b405feb0 2020-12-05T18:12:17.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.476+0000 7fa5bb2f5700 1 -- 172.21.15.32:0/2487973162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5b405feb0 msgr2=0x7fa5b405f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.476+0000 7fa5bb2f5700 1 --2- 172.21.15.32:0/2487973162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5b405feb0 0x7fa5b405f790 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 rx=0x7fa5b00010d0 tx=0x7fa5ac009270).stop 2020-12-05T18:12:17.483 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.476+0000 7fa5bb2f5700 1 -- 172.21.15.32:0/2487973162 shutdown_connections 2020-12-05T18:12:17.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.476+0000 7fa5bb2f5700 1 --2- 172.21.15.32:0/2487973162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5b405feb0 0x7fa5b405f790 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.476+0000 7fa5bb2f5700 1 -- 172.21.15.32:0/2487973162 >> 172.21.15.32:0/2487973162 conn(0x7fa5b40b8ee0 msgr2=0x7fa5b40b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:17.484 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.476+0000 7fa5bb2f5700 1 -- 172.21.15.32:0/2487973162 shutdown_connections 2020-12-05T18:12:17.485 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.476+0000 7fa5bb2f5700 1 -- 172.21.15.32:0/2487973162 wait complete. 2020-12-05T18:12:17.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5bb2f5700 1 Processor -- start 2020-12-05T18:12:17.487 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5bb2f5700 1 -- start start 2020-12-05T18:12:17.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5bb2f5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5b405f3b0 0x7fa5b4066160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5bb2f5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5b405feb0 0x7fa5b40646b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5bb2f5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5b4062400 0x7fa5b4065500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.488 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5bb2f5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b41448d0 con 0x7fa5b405f3b0 2020-12-05T18:12:17.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5bb2f5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b41435d0 con 0x7fa5b4062400 2020-12-05T18:12:17.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5bb2f5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b4144ac0 con 0x7fa5b405feb0 2020-12-05T18:12:17.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5ba2f3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5b405f3b0 0x7fa5b4066160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5ba2f3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5b405f3b0 0x7fa5b4066160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42202/0 (socket says 172.21.15.32:42202) 2020-12-05T18:12:17.489 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5ba2f3700 1 -- 172.21.15.32:0/2902379484 learned_addr learned my addr 172.21.15.32:0/2902379484 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:17.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5baaf4700 1 --2- 172.21.15.32:0/2902379484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5b4062400 0x7fa5b4065500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5b9af2700 1 --2- 172.21.15.32:0/2902379484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5b405feb0 0x7fa5b40646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5ba2f3700 1 -- 172.21.15.32:0/2902379484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5b405feb0 msgr2=0x7fa5b40646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.490 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5ba2f3700 1 --2- 172.21.15.32:0/2902379484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5b405feb0 0x7fa5b40646b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5ba2f3700 1 -- 172.21.15.32:0/2902379484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5b4062400 msgr2=0x7fa5b4065500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5ba2f3700 1 --2- 172.21.15.32:0/2902379484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5b4062400 0x7fa5b4065500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5ba2f3700 1 -- 172.21.15.32:0/2902379484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b40650a0 con 0x7fa5b405f3b0 2020-12-05T18:12:17.491 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5ba2f3700 1 --2- 172.21.15.32:0/2902379484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5b405f3b0 0x7fa5b4066160 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 rx=0x7fa5b0001190 tx=0x7fa5b0007b80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:17.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5a37fe700 1 -- 172.21.15.32:0/2902379484 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5b000e4a0 con 0x7fa5b405f3b0 2020-12-05T18:12:17.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5a37fe700 1 -- 172.21.15.32:0/2902379484 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b000e600 con 0x7fa5b405f3b0 2020-12-05T18:12:17.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5bb2f5700 1 -- 172.21.15.32:0/2902379484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b41444f0 con 0x7fa5b405f3b0 2020-12-05T18:12:17.492 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5a37fe700 1 -- 172.21.15.32:0/2902379484 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5b000e960 con 0x7fa5b405f3b0 2020-12-05T18:12:17.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.480+0000 7fa5bb2f5700 1 -- 172.21.15.32:0/2902379484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b4066a70 con 0x7fa5b405f3b0 2020-12-05T18:12:17.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.484+0000 7fa5bb2f5700 1 -- 172.21.15.32:0/2902379484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b4064ee0 con 0x7fa5b405f3b0 2020-12-05T18:12:17.493 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.484+0000 7fa5a37fe700 1 -- 172.21.15.32:0/2902379484 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa5b0019030 con 0x7fa5b405f3b0 2020-12-05T18:12:17.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.484+0000 7fa5a37fe700 1 --2- 172.21.15.32:0/2902379484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa5940554e0 0x7fa5940579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.484+0000 7fa5a37fe700 1 -- 172.21.15.32:0/2902379484 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fa5b006d1a0 con 0x7fa5b405f3b0 2020-12-05T18:12:17.494 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.484+0000 7fa5b9af2700 1 --2- 172.21.15.32:0/2902379484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa5940554e0 0x7fa5940579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.484+0000 7fa5b9af2700 1 --2- 172.21.15.32:0/2902379484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa5940554e0 0x7fa5940579a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7fa5a400d000 tx=0x7fa5a4003040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:17.495 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.488+0000 7fa5a37fe700 1 -- 172.21.15.32:0/2902379484 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa5b0041df0 con 0x7fa5b405f3b0 2020-12-05T18:12:17.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.628+0000 7fa5bb2f5700 1 -- 172.21.15.32:0/2902379484 --> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5b4144680 con 0x7fa5940554e0 2020-12-05T18:12:17.631 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.628+0000 7fa5a37fe700 1 -- 172.21.15.32:0/2902379484 <== mgr.4101 v2:172.21.15.32:6800/11355 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa5b4144680 con 0x7fa5940554e0 2020-12-05T18:12:17.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.628+0000 7fa5bb2f5700 1 -- 172.21.15.32:0/2902379484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa5940554e0 msgr2=0x7fa5940579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.628+0000 7fa5bb2f5700 1 --2- 172.21.15.32:0/2902379484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa5940554e0 0x7fa5940579a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7fa5a400d000 tx=0x7fa5a4003040).stop 2020-12-05T18:12:17.634 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.628+0000 7fa5bb2f5700 1 -- 172.21.15.32:0/2902379484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5b405f3b0 msgr2=0x7fa5b4066160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.628+0000 7fa5bb2f5700 1 --2- 172.21.15.32:0/2902379484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5b405f3b0 0x7fa5b4066160 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 rx=0x7fa5b0001190 tx=0x7fa5b0007b80).stop 2020-12-05T18:12:17.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.628+0000 7fa5bb2f5700 1 -- 172.21.15.32:0/2902379484 shutdown_connections 2020-12-05T18:12:17.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.628+0000 7fa5bb2f5700 1 --2- 172.21.15.32:0/2902379484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5b4062400 0x7fa5b4065500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.635 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.628+0000 7fa5bb2f5700 1 --2- 172.21.15.32:0/2902379484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5b405f3b0 0x7fa5b4066160 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.628+0000 7fa5bb2f5700 1 --2- 172.21.15.32:0/2902379484 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fa5940554e0 0x7fa5940579a0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.628+0000 7fa5bb2f5700 1 --2- 172.21.15.32:0/2902379484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5b405feb0 0x7fa5b40646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.628+0000 7fa5bb2f5700 1 -- 172.21.15.32:0/2902379484 >> 172.21.15.32:0/2902379484 conn(0x7fa5b40b8ee0 msgr2=0x7fa5b40b7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:17.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.632+0000 7fa5bb2f5700 1 -- 172.21.15.32:0/2902379484 shutdown_connections 2020-12-05T18:12:17.636 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.632+0000 7fa5bb2f5700 1 -- 172.21.15.32:0/2902379484 wait complete. 2020-12-05T18:12:17.637 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 6 to deep-scrub 2020-12-05T18:12:17.650 INFO:tasks.ceph:Scrubbing osd.7 2020-12-05T18:12:17.651 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.7 config set osd_debug_deep_scrub_sleep 0 2020-12-05T18:12:17.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.764+0000 7f044b0b8700 1 Processor -- start 2020-12-05T18:12:17.767 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.764+0000 7f044b0b8700 1 -- start start 2020-12-05T18:12:17.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.764+0000 7f044b0b8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0444061ba0 0x7f04440b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.764+0000 7f044b0b8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04440b1720 0x7f044414a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.769 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.764+0000 7f044b0b8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0444061f80 con 0x7f044414aeb0 2020-12-05T18:12:17.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.764+0000 7f044b0b8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0444147ad0 con 0x7f04440b1720 2020-12-05T18:12:17.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.764+0000 7f044b0b8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f044414a280 con 0x7f0444061ba0 2020-12-05T18:12:17.770 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.764+0000 7f044a8b7700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f044414aeb0 0x7f04441460b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44860/0 (socket says 172.21.15.32:44860) 2020-12-05T18:12:17.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.764+0000 7f044a8b7700 1 -- 172.21.15.32:0/1522335158 learned_addr learned my addr 172.21.15.32:0/1522335158 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T18:12:17.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.764+0000 7f04498b5700 1 --2- 172.21.15.32:0/1522335158 >> v2:172.21.15.32:3300/0 conn(0x7f04440b1720 0x7f044414a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.771 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.764+0000 7f044a0b6700 1 --2- 172.21.15.32:0/1522335158 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0444061ba0 0x7f04440b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.764+0000 7f04498b5700 1 -- 172.21.15.32:0/1522335158 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0444061ba0 msgr2=0x7f04440b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T18:12:17.775 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.764+0000 7f04498b5700 1 --2- 172.21.15.32:0/1522335158 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0444061ba0 0x7f04440b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.764+0000 7f04498b5700 1 -- 172.21.15.32:0/1522335158 >> v1:172.21.15.32:6789/0 conn(0x7f044414aeb0 legacy=0x7f04441460b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.764+0000 7f04498b5700 1 -- 172.21.15.32:0/1522335158 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044414a520 con 0x7f04440b1720 2020-12-05T18:12:17.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.764+0000 7f04498b5700 1 --2- 172.21.15.32:0/1522335158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04440b1720 0x7f044414a9c0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 rx=0x7f0440002f30 tx=0x7f0440006dc0).ready entity=mon.0 client_cookie=14cd724524dcadef server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:17.776 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.764+0000 7f04490b4700 1 -- 172.21.15.32:0/1522335158 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0440009760 con 0x7f04440b1720 2020-12-05T18:12:17.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.764+0000 7f04490b4700 1 -- 172.21.15.32:0/1522335158 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04400098c0 con 0x7f04440b1720 2020-12-05T18:12:17.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.764+0000 7f044b0b8700 1 -- 172.21.15.32:0/1522335158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04440b1720 msgr2=0x7f044414a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.764+0000 7f044b0b8700 1 --2- 172.21.15.32:0/1522335158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04440b1720 0x7f044414a9c0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 rx=0x7f0440002f30 tx=0x7f0440006dc0).stop 2020-12-05T18:12:17.777 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f044b0b8700 1 -- 172.21.15.32:0/1522335158 shutdown_connections 2020-12-05T18:12:17.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f044b0b8700 1 --2- 172.21.15.32:0/1522335158 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0444061ba0 0x7f04440b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f044b0b8700 1 --2- 172.21.15.32:0/1522335158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04440b1720 0x7f044414a9c0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.778 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f044b0b8700 1 -- 172.21.15.32:0/1522335158 >> 172.21.15.32:0/1522335158 conn(0x7f04440b6d10 msgr2=0x7f04440b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:17.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f044b0b8700 1 -- 172.21.15.32:0/1522335158 shutdown_connections 2020-12-05T18:12:17.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f044b0b8700 1 -- 172.21.15.32:0/1522335158 wait complete. 2020-12-05T18:12:17.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f044b0b8700 1 Processor -- start 2020-12-05T18:12:17.779 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f044b0b8700 1 -- start start 2020-12-05T18:12:17.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f044b0b8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0444061ba0 0x7f0444159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f044b0b8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04440b1720 0x7f04440b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f044b0b8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f044414aeb0 0x7f0444066650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f044b0b8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04440657f0 con 0x7f044414aeb0 2020-12-05T18:12:17.780 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f044b0b8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0444064ea0 con 0x7f04440b1720 2020-12-05T18:12:17.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f044b0b8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f044414cea0 con 0x7f0444061ba0 2020-12-05T18:12:17.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f044a8b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f044414aeb0 0x7f0444066650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f044a8b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f044414aeb0 0x7f0444066650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42218/0 (socket says 172.21.15.32:42218) 2020-12-05T18:12:17.781 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f044a8b7700 1 -- 172.21.15.32:0/2706214999 learned_addr learned my addr 172.21.15.32:0/2706214999 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:17.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f04498b5700 1 --2- 172.21.15.32:0/2706214999 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04440b1720 0x7f04440b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f044a0b6700 1 --2- 172.21.15.32:0/2706214999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0444061ba0 0x7f0444159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f044a8b7700 1 -- 172.21.15.32:0/2706214999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0444061ba0 msgr2=0x7f0444159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.782 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f044a8b7700 1 --2- 172.21.15.32:0/2706214999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0444061ba0 0x7f0444159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f044a8b7700 1 -- 172.21.15.32:0/2706214999 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04440b1720 msgr2=0x7f04440b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f044a8b7700 1 --2- 172.21.15.32:0/2706214999 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04440b1720 0x7f04440b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f044a8b7700 1 -- 172.21.15.32:0/2706214999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044414d140 con 0x7f044414aeb0 2020-12-05T18:12:17.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.768+0000 7f04498b5700 1 --2- 172.21.15.32:0/2706214999 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04440b1720 0x7f04440b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-12-05T18:12:17.783 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.772+0000 7f044a8b7700 1 --2- 172.21.15.32:0/2706214999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f044414aeb0 0x7f0444066650 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 rx=0x7f044405f270 tx=0x7f043c00cbf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:17.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.772+0000 7f04377fe700 1 -- 172.21.15.32:0/2706214999 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f043c005b70 con 0x7f044414aeb0 2020-12-05T18:12:17.784 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.772+0000 7f04377fe700 1 -- 172.21.15.32:0/2706214999 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043c005cd0 con 0x7f044414aeb0 2020-12-05T18:12:17.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.772+0000 7f04377fe700 1 -- 172.21.15.32:0/2706214999 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f043c007300 con 0x7f044414aeb0 2020-12-05T18:12:17.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.772+0000 7f044b0b8700 1 -- 172.21.15.32:0/2706214999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f044414c920 con 0x7f044414aeb0 2020-12-05T18:12:17.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.772+0000 7f044b0b8700 1 -- 172.21.15.32:0/2706214999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f044414dcd0 con 0x7f044414aeb0 2020-12-05T18:12:17.785 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.772+0000 7f044b0b8700 1 -- 172.21.15.32:0/2706214999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f044414d2d0 con 0x7f044414aeb0 2020-12-05T18:12:17.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.772+0000 7f04377fe700 1 -- 172.21.15.32:0/2706214999 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f043c010030 con 0x7f044414aeb0 2020-12-05T18:12:17.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.772+0000 7f04377fe700 1 --2- 172.21.15.32:0/2706214999 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f04240553b0 0x7f0424057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.787 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.772+0000 7f044a0b6700 1 --2- 172.21.15.32:0/2706214999 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f04240553b0 0x7f0424057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.772+0000 7f04377fe700 1 -- 172.21.15.32:0/2706214999 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f043c06dd50 con 0x7f044414aeb0 2020-12-05T18:12:17.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.772+0000 7f04377fe700 1 --2- 172.21.15.32:0/2706214999 >> [v2:172.21.15.61:6813/11540,v1:172.21.15.61:6815/11540] conn(0x7f042405c0d0 0x7f042405e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.772+0000 7f04377fe700 1 -- 172.21.15.32:0/2706214999 --> [v2:172.21.15.61:6813/11540,v1:172.21.15.61:6815/11540] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f042405eb70 con 0x7f042405c0d0 2020-12-05T18:12:17.788 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.772+0000 7f04377fe700 1 -- 172.21.15.32:0/2706214999 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) v2 ==== 24+0+0 (secure 0 0 0) 0x7f043c0703e0 con 0x7f044414aeb0 2020-12-05T18:12:17.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.772+0000 7f044a0b6700 1 --2- 172.21.15.32:0/2706214999 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f04240553b0 0x7f0424057870 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f043800b000 tx=0x7f0438008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:17.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.772+0000 7f04498b5700 1 --2- 172.21.15.32:0/2706214999 >> [v2:172.21.15.61:6813/11540,v1:172.21.15.61:6815/11540] conn(0x7f042405c0d0 0x7f042405e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.776+0000 7f04498b5700 1 --2- 172.21.15.32:0/2706214999 >> [v2:172.21.15.61:6813/11540,v1:172.21.15.61:6815/11540] conn(0x7f042405c0d0 0x7f042405e4f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:17.789 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.776+0000 7f04377fe700 1 -- 172.21.15.32:0/2706214999 <== osd.7 v2:172.21.15.61:6813/11540 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f042405eb70 con 0x7f042405c0d0 2020-12-05T18:12:17.800 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.796+0000 7f044b0b8700 1 -- 172.21.15.32:0/2706214999 --> [v2:172.21.15.61:6813/11540,v1:172.21.15.61:6815/11540] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0444057050 con 0x7f042405c0d0 2020-12-05T18:12:17.801 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.796+0000 7f04377fe700 1 -- 172.21.15.32:0/2706214999 <== osd.7 v2:172.21.15.61:6813/11540 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f0444057050 con 0x7f042405c0d0 2020-12-05T18:12:17.802 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2020-12-05T18:12:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-12-05T18:12:17.802 INFO:teuthology.orchestra.run.smithi032.stdout:} 2020-12-05T18:12:17.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.800+0000 7f044b0b8700 1 -- 172.21.15.32:0/2706214999 >> [v2:172.21.15.61:6813/11540,v1:172.21.15.61:6815/11540] conn(0x7f042405c0d0 msgr2=0x7f042405e4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.803 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.800+0000 7f044b0b8700 1 --2- 172.21.15.32:0/2706214999 >> [v2:172.21.15.61:6813/11540,v1:172.21.15.61:6815/11540] conn(0x7f042405c0d0 0x7f042405e4f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.800+0000 7f044b0b8700 1 -- 172.21.15.32:0/2706214999 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f04240553b0 msgr2=0x7f0424057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.800+0000 7f044b0b8700 1 --2- 172.21.15.32:0/2706214999 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f04240553b0 0x7f0424057870 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f043800b000 tx=0x7f0438008040).stop 2020-12-05T18:12:17.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.800+0000 7f044b0b8700 1 -- 172.21.15.32:0/2706214999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f044414aeb0 msgr2=0x7f0444066650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.804 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.800+0000 7f044b0b8700 1 --2- 172.21.15.32:0/2706214999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f044414aeb0 0x7f0444066650 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 rx=0x7f044405f270 tx=0x7f043c00cbf0).stop 2020-12-05T18:12:17.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.800+0000 7f044a8b7700 1 -- 172.21.15.32:0/2706214999 reap_dead start 2020-12-05T18:12:17.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.800+0000 7f044b0b8700 1 -- 172.21.15.32:0/2706214999 shutdown_connections 2020-12-05T18:12:17.805 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.800+0000 7f044b0b8700 1 -- 172.21.15.32:0/2706214999 >> 172.21.15.32:0/2706214999 conn(0x7f04440b6d10 msgr2=0x7f04440b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:17.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.800+0000 7f044b0b8700 1 -- 172.21.15.32:0/2706214999 shutdown_connections 2020-12-05T18:12:17.806 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.800+0000 7f044b0b8700 1 -- 172.21.15.32:0/2706214999 wait complete. 2020-12-05T18:12:17.818 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 7 2020-12-05T18:12:17.934 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.932+0000 7fee9963b700 1 Processor -- start 2020-12-05T18:12:17.935 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.932+0000 7fee9963b700 1 -- start start 2020-12-05T18:12:17.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.932+0000 7fee9963b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fee940ab710 0x7fee940af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.936 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.932+0000 7fee9963b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee94146a90 con 0x7fee940b1fc0 2020-12-05T18:12:17.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.932+0000 7fee9963b700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee940b23a0 con 0x7fee94147400 2020-12-05T18:12:17.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.932+0000 7fee9963b700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee94061e90 con 0x7fee940ab710 2020-12-05T18:12:17.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.932+0000 7fee93fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fee940b1fc0 0x7fee94061780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44874/0 (socket says 172.21.15.32:44874) 2020-12-05T18:12:17.937 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.932+0000 7fee93fff700 1 -- 172.21.15.32:0/2546467935 learned_addr learned my addr 172.21.15.32:0/2546467935 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T18:12:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.932+0000 7fee937fe700 1 --2- 172.21.15.32:0/2546467935 >> v2:172.21.15.61:3300/0 conn(0x7fee940ab710 0x7fee940af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.932+0000 7fee92ffd700 1 -- 172.21.15.32:0/2546467935 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3177393047 0 0) 0x7fee94146a90 con 0x7fee940b1fc0 2020-12-05T18:12:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.932+0000 7fee92ffd700 1 -- 172.21.15.32:0/2546467935 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee74003230 con 0x7fee940b1fc0 2020-12-05T18:12:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.932+0000 7fee92ffd700 1 -- 172.21.15.32:0/2546467935 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3682113328 0 0) 0x7fee940b23a0 con 0x7fee94147400 2020-12-05T18:12:17.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.932+0000 7fee92ffd700 1 -- 172.21.15.32:0/2546467935 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee94146a90 con 0x7fee94147400 2020-12-05T18:12:17.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.932+0000 7fee937fe700 1 -- 172.21.15.32:0/2546467935 >> v1:172.21.15.61:6789/0 conn(0x7fee94147400 legacy=0x7fee94145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.932+0000 7fee937fe700 1 -- 172.21.15.32:0/2546467935 >> v1:172.21.15.32:6789/0 conn(0x7fee940b1fc0 legacy=0x7fee94061780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.932+0000 7fee92ffd700 1 -- 172.21.15.32:0/2546467935 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 714349807 0 0) 0x7fee74003230 con 0x7fee940b1fc0 2020-12-05T18:12:17.939 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.932+0000 7fee937fe700 1 -- 172.21.15.32:0/2546467935 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee94146cc0 con 0x7fee940ab710 2020-12-05T18:12:17.940 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.932+0000 7fee937fe700 1 --2- 172.21.15.32:0/2546467935 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee940ab710 0x7fee940af230 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7fee7c001870 tx=0x7fee840091a0).ready entity=mon.1 client_cookie=1eb68a77c99fcfd8 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:17.942 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.932+0000 7fee92ffd700 1 -- 172.21.15.32:0/2546467935 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee7c0016d0 con 0x7fee940ab710 2020-12-05T18:12:17.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.932+0000 7fee92ffd700 1 -- 172.21.15.32:0/2546467935 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8400b800 con 0x7fee940ab710 2020-12-05T18:12:17.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.932+0000 7fee92ffd700 1 -- 172.21.15.32:0/2546467935 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee8400b970 con 0x7fee940ab710 2020-12-05T18:12:17.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.932+0000 7fee9963b700 1 -- 172.21.15.32:0/2546467935 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee940ab710 msgr2=0x7fee940af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.932+0000 7fee9963b700 1 --2- 172.21.15.32:0/2546467935 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee940ab710 0x7fee940af230 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7fee7c001870 tx=0x7fee840091a0).stop 2020-12-05T18:12:17.943 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee9963b700 1 -- 172.21.15.32:0/2546467935 shutdown_connections 2020-12-05T18:12:17.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee9963b700 1 --2- 172.21.15.32:0/2546467935 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee940ab710 0x7fee940af230 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee9963b700 1 -- 172.21.15.32:0/2546467935 >> 172.21.15.32:0/2546467935 conn(0x7fee940b5530 msgr2=0x7fee940b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:17.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee9963b700 1 -- 172.21.15.32:0/2546467935 shutdown_connections 2020-12-05T18:12:17.944 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee9963b700 1 -- 172.21.15.32:0/2546467935 wait complete. 2020-12-05T18:12:17.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee9963b700 1 Processor -- start 2020-12-05T18:12:17.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee9963b700 1 -- start start 2020-12-05T18:12:17.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee9963b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee940ab710 0x7fee9414d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.945 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee9963b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee940b1fc0 0x7fee941557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee9963b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fee94147400 0x7fee941593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee9963b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee940abcd0 con 0x7fee940ab710 2020-12-05T18:12:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee9963b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee94158f00 con 0x7fee940b1fc0 2020-12-05T18:12:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee9963b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9414d760 con 0x7fee94147400 2020-12-05T18:12:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee93fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee940ab710 0x7fee9414d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee93fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee940ab710 0x7fee9414d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42234/0 (socket says 172.21.15.32:42234) 2020-12-05T18:12:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee93fff700 1 -- 172.21.15.32:0/2208003370 learned_addr learned my addr 172.21.15.32:0/2208003370 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee937fe700 1 --2- 172.21.15.32:0/2208003370 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee940b1fc0 0x7fee941557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee98e3a700 1 --2- 172.21.15.32:0/2208003370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fee94147400 0x7fee941593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee93fff700 1 -- 172.21.15.32:0/2208003370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fee94147400 msgr2=0x7fee941593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee93fff700 1 --2- 172.21.15.32:0/2208003370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fee94147400 0x7fee941593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee93fff700 1 -- 172.21.15.32:0/2208003370 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee940b1fc0 msgr2=0x7fee941557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:17.948 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee93fff700 1 --2- 172.21.15.32:0/2208003370 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee940b1fc0 0x7fee941557c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:17.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee93fff700 1 -- 172.21.15.32:0/2208003370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee94147e50 con 0x7fee940ab710 2020-12-05T18:12:17.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee93fff700 1 --2- 172.21.15.32:0/2208003370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee940ab710 0x7fee9414d270 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 rx=0x7fee7c000fa0 tx=0x7fee7c001b00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:17.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.936+0000 7fee917fa700 1 -- 172.21.15.32:0/2208003370 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee7c007170 con 0x7fee940ab710 2020-12-05T18:12:17.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.940+0000 7fee917fa700 1 -- 172.21.15.32:0/2208003370 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7c0072d0 con 0x7fee940ab710 2020-12-05T18:12:17.949 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.940+0000 7fee9963b700 1 -- 172.21.15.32:0/2208003370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee94155cb0 con 0x7fee940ab710 2020-12-05T18:12:17.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.940+0000 7fee917fa700 1 -- 172.21.15.32:0/2208003370 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee7c0048f0 con 0x7fee940ab710 2020-12-05T18:12:17.950 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.940+0000 7fee9963b700 1 -- 172.21.15.32:0/2208003370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee941424f0 con 0x7fee940ab710 2020-12-05T18:12:17.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.940+0000 7fee9963b700 1 -- 172.21.15.32:0/2208003370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee94057050 con 0x7fee940ab710 2020-12-05T18:12:17.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.940+0000 7fee917fa700 1 -- 172.21.15.32:0/2208003370 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fee7c010070 con 0x7fee940ab710 2020-12-05T18:12:17.951 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.940+0000 7fee917fa700 1 --2- 172.21.15.32:0/2208003370 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fee78055440 0x7fee78057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.940+0000 7fee917fa700 1 -- 172.21.15.32:0/2208003370 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fee7c06da60 con 0x7fee940ab710 2020-12-05T18:12:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.940+0000 7fee937fe700 1 --2- 172.21.15.32:0/2208003370 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fee78055440 0x7fee78057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.940+0000 7fee937fe700 1 --2- 172.21.15.32:0/2208003370 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fee78055440 0x7fee78057900 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7fee8401b000 tx=0x7fee84016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:17.953 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:17.948+0000 7fee917fa700 1 -- 172.21.15.32:0/2208003370 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fee7c043d50 con 0x7fee940ab710 2020-12-05T18:12:18.096 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.092+0000 7fee9963b700 1 -- 172.21.15.32:0/2208003370 --> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fee940abb50 con 0x7fee78055440 2020-12-05T18:12:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.092+0000 7fee917fa700 1 -- 172.21.15.32:0/2208003370 <== mgr.4101 v2:172.21.15.32:6800/11355 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fee840099f0 con 0x7fee78055440 2020-12-05T18:12:18.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.096+0000 7fee9963b700 1 -- 172.21.15.32:0/2208003370 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fee78055440 msgr2=0x7fee78057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:18.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.096+0000 7fee9963b700 1 --2- 172.21.15.32:0/2208003370 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fee78055440 0x7fee78057900 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7fee8401b000 tx=0x7fee84016000).stop 2020-12-05T18:12:18.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.096+0000 7fee9963b700 1 -- 172.21.15.32:0/2208003370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee940ab710 msgr2=0x7fee9414d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:18.100 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.096+0000 7fee9963b700 1 --2- 172.21.15.32:0/2208003370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee940ab710 0x7fee9414d270 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 rx=0x7fee7c000fa0 tx=0x7fee7c001b00).stop 2020-12-05T18:12:18.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.096+0000 7fee9963b700 1 -- 172.21.15.32:0/2208003370 shutdown_connections 2020-12-05T18:12:18.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.096+0000 7fee9963b700 1 --2- 172.21.15.32:0/2208003370 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fee78055440 0x7fee78057900 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:18.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.096+0000 7fee9963b700 1 --2- 172.21.15.32:0/2208003370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fee94147400 0x7fee941593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:18.101 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.096+0000 7fee9963b700 1 --2- 172.21.15.32:0/2208003370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee940ab710 0x7fee9414d270 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:18.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.096+0000 7fee9963b700 1 --2- 172.21.15.32:0/2208003370 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee940b1fc0 0x7fee941557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:18.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.096+0000 7fee9963b700 1 -- 172.21.15.32:0/2208003370 >> 172.21.15.32:0/2208003370 conn(0x7fee940b5530 msgr2=0x7fee940b12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:18.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.096+0000 7fee9963b700 1 -- 172.21.15.32:0/2208003370 shutdown_connections 2020-12-05T18:12:18.102 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.096+0000 7fee9963b700 1 -- 172.21.15.32:0/2208003370 wait complete. 2020-12-05T18:12:18.103 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 7 to deep-scrub 2020-12-05T18:12:18.111 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-12-05T18:12:18.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.216+0000 7fdceb483700 1 Processor -- start 2020-12-05T18:12:18.222 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdceb483700 1 -- start start 2020-12-05T18:12:18.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdceb483700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fdce40af0d0 0x7fdce40b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:18.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdceb483700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdce413f670 0x7fdce413fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:18.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdceb483700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce40b23a0 con 0x7fdce40b1fc0 2020-12-05T18:12:18.224 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdceb483700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce413f280 con 0x7fdce413f670 2020-12-05T18:12:18.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdceb483700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce40abfa0 con 0x7fdce40af0d0 2020-12-05T18:12:18.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdceac82700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdce413f670 0x7fdce413fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:18.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdceac82700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdce413f670 0x7fdce413fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42242/0 (socket says 172.21.15.32:42242) 2020-12-05T18:12:18.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdceac82700 1 -- 172.21.15.32:0/4056228906 learned_addr learned my addr 172.21.15.32:0/4056228906 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:18.225 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdce9c80700 1 --2- 172.21.15.32:0/4056228906 >> v2:172.21.15.61:3300/0 conn(0x7fdce40af0d0 0x7fdce40b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:18.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdceac82700 1 -- 172.21.15.32:0/4056228906 >> v2:172.21.15.61:3300/0 conn(0x7fdce40af0d0 msgr2=0x7fdce40b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T18:12:18.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdceac82700 1 --2- 172.21.15.32:0/4056228906 >> v2:172.21.15.61:3300/0 conn(0x7fdce40af0d0 0x7fdce40b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:18.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdceac82700 1 -- 172.21.15.32:0/4056228906 >> v1:172.21.15.32:6789/0 conn(0x7fdce40b1fc0 legacy=0x7fdce413c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:18.226 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdceac82700 1 -- 172.21.15.32:0/4056228906 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce40ac240 con 0x7fdce413f670 2020-12-05T18:12:18.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdceac82700 1 --2- 172.21.15.32:0/4056228906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce413f670 0x7fdce413fa70 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 rx=0x7fdcdc0032b0 tx=0x7fdcdc009270).ready entity=mon.0 client_cookie=61c1f6d0035f74a5 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:18.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdce947f700 1 -- 172.21.15.32:0/4056228906 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcdc00bc00 con 0x7fdce413f670 2020-12-05T18:12:18.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdce947f700 1 -- 172.21.15.32:0/4056228906 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcdc00bd60 con 0x7fdce413f670 2020-12-05T18:12:18.227 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdceb483700 1 -- 172.21.15.32:0/4056228906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce413f670 msgr2=0x7fdce413fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:18.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdce947f700 1 -- 172.21.15.32:0/4056228906 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcdc011400 con 0x7fdce413f670 2020-12-05T18:12:18.228 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdceb483700 1 --2- 172.21.15.32:0/4056228906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce413f670 0x7fdce413fa70 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 rx=0x7fdcdc0032b0 tx=0x7fdcdc009270).stop 2020-12-05T18:12:18.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdceb483700 1 -- 172.21.15.32:0/4056228906 shutdown_connections 2020-12-05T18:12:18.230 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdceb483700 1 --2- 172.21.15.32:0/4056228906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce413f670 0x7fdce413fa70 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:18.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdceb483700 1 --2- 172.21.15.32:0/4056228906 >> v2:172.21.15.61:3300/0 conn(0x7fdce40af0d0 0x7fdce40b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:18.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdceb483700 1 -- 172.21.15.32:0/4056228906 >> 172.21.15.32:0/4056228906 conn(0x7fdce40b5530 msgr2=0x7fdce40b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:18.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdceb483700 1 -- 172.21.15.32:0/4056228906 shutdown_connections 2020-12-05T18:12:18.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.220+0000 7fdceb483700 1 -- 172.21.15.32:0/4056228906 wait complete. 2020-12-05T18:12:18.231 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdceb483700 1 Processor -- start 2020-12-05T18:12:18.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdceb483700 1 -- start start 2020-12-05T18:12:18.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdceb483700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce40af0d0 0x7fdce412f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:18.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdceb483700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdce40b1fc0 0x7fdce41416f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:18.232 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdceb483700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdce413f670 0x7fdce4141be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:18.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdceb483700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce41409c0 con 0x7fdce40af0d0 2020-12-05T18:12:18.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdceb483700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce412ffe0 con 0x7fdce413f670 2020-12-05T18:12:18.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdceb483700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce413e9a0 con 0x7fdce40b1fc0 2020-12-05T18:12:18.233 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdcea481700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce40af0d0 0x7fdce412f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:18.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdcea481700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce40af0d0 0x7fdce412f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42246/0 (socket says 172.21.15.32:42246) 2020-12-05T18:12:18.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdcea481700 1 -- 172.21.15.32:0/2465571464 learned_addr learned my addr 172.21.15.32:0/2465571464 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:18.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdceac82700 1 --2- 172.21.15.32:0/2465571464 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdce413f670 0x7fdce4141be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:18.234 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdce9c80700 1 --2- 172.21.15.32:0/2465571464 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdce40b1fc0 0x7fdce41416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdcea481700 1 -- 172.21.15.32:0/2465571464 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdce40b1fc0 msgr2=0x7fdce41416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdcea481700 1 --2- 172.21.15.32:0/2465571464 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdce40b1fc0 0x7fdce41416f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdcea481700 1 -- 172.21.15.32:0/2465571464 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdce413f670 msgr2=0x7fdce4141be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdcea481700 1 --2- 172.21.15.32:0/2465571464 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdce413f670 0x7fdce4141be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdcea481700 1 -- 172.21.15.32:0/2465571464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce412fa90 con 0x7fdce40af0d0 2020-12-05T18:12:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdcea481700 1 --2- 172.21.15.32:0/2465571464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce40af0d0 0x7fdce412f3d0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 rx=0x7fdce40608f0 tx=0x7fdcd8007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdcd77fe700 1 -- 172.21.15.32:0/2465571464 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcd80072a0 con 0x7fdce40af0d0 2020-12-05T18:12:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdcd77fe700 1 -- 172.21.15.32:0/2465571464 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd80064e0 con 0x7fdce40af0d0 2020-12-05T18:12:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdceb483700 1 -- 172.21.15.32:0/2465571464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce405f9e0 con 0x7fdce40af0d0 2020-12-05T18:12:18.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdcd77fe700 1 -- 172.21.15.32:0/2465571464 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcd8006840 con 0x7fdce40af0d0 2020-12-05T18:12:18.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdceb483700 1 -- 172.21.15.32:0/2465571464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce413d8f0 con 0x7fdce40af0d0 2020-12-05T18:12:18.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.224+0000 7fdceb483700 1 -- 172.21.15.32:0/2465571464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce412fc20 con 0x7fdce40af0d0 2020-12-05T18:12:18.237 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.228+0000 7fdcd77fe700 1 -- 172.21.15.32:0/2465571464 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fdcd8019020 con 0x7fdce40af0d0 2020-12-05T18:12:18.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.228+0000 7fdcd77fe700 1 --2- 172.21.15.32:0/2465571464 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdcc4055330 0x7fdcc40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:18.238 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.228+0000 7fdcd77fe700 1 -- 172.21.15.32:0/2465571464 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fdcd806d660 con 0x7fdce40af0d0 2020-12-05T18:12:18.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.228+0000 7fdce9c80700 1 --2- 172.21.15.32:0/2465571464 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdcc4055330 0x7fdcc40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:18.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.228+0000 7fdce9c80700 1 --2- 172.21.15.32:0/2465571464 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdcc4055330 0x7fdcc40577f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7fdce000c000 tx=0x7fdce0008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:18.239 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.232+0000 7fdcd77fe700 1 -- 172.21.15.32:0/2465571464 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fdcd8043a40 con 0x7fdce40af0d0 2020-12-05T18:12:18.373 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.368+0000 7fdceb483700 1 -- 172.21.15.32:0/2465571464 --> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdce413eba0 con 0x7fdcc4055330 2020-12-05T18:12:18.379 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.376+0000 7fdcd77fe700 1 -- 172.21.15.32:0/2465571464 <== mgr.4101 v2:172.21.15.32:6800/11355 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+66178 (secure 0 0 0) 0x7fdce000ddf0 con 0x7fdcc4055330 2020-12-05T18:12:18.381 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T18:12:18.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.376+0000 7fdceb483700 1 -- 172.21.15.32:0/2465571464 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdcc4055330 msgr2=0x7fdcc40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:18.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.376+0000 7fdceb483700 1 --2- 172.21.15.32:0/2465571464 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdcc4055330 0x7fdcc40577f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7fdce000c000 tx=0x7fdce0008040).stop 2020-12-05T18:12:18.382 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.376+0000 7fdceb483700 1 -- 172.21.15.32:0/2465571464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce40af0d0 msgr2=0x7fdce412f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:18.383 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.376+0000 7fdceb483700 1 --2- 172.21.15.32:0/2465571464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce40af0d0 0x7fdce412f3d0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 rx=0x7fdce40608f0 tx=0x7fdcd8007be0).stop 2020-12-05T18:12:18.383 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.376+0000 7fdceb483700 1 -- 172.21.15.32:0/2465571464 shutdown_connections 2020-12-05T18:12:18.383 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.376+0000 7fdceb483700 1 --2- 172.21.15.32:0/2465571464 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdce413f670 0x7fdce4141be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:18.383 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.376+0000 7fdceb483700 1 --2- 172.21.15.32:0/2465571464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce40af0d0 0x7fdce412f3d0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:18.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.376+0000 7fdceb483700 1 --2- 172.21.15.32:0/2465571464 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7fdcc4055330 0x7fdcc40577f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:18.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.376+0000 7fdceb483700 1 --2- 172.21.15.32:0/2465571464 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdce40b1fc0 0x7fdce41416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:18.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.376+0000 7fdceb483700 1 -- 172.21.15.32:0/2465571464 >> 172.21.15.32:0/2465571464 conn(0x7fdce40b5530 msgr2=0x7fdce40ad7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:18.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.380+0000 7fdceb483700 1 -- 172.21.15.32:0/2465571464 shutdown_connections 2020-12-05T18:12:18.384 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:18.380+0000 7fdceb483700 1 -- 172.21.15.32:0/2465571464 wait complete. 2020-12-05T18:12:18.385 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2020-12-05T18:12:18.396 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":1369,"stamp":"2020-12-05T18:12:17.520642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":8403712,"kb_used_data":15040,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746571008,"statfs":{"total":773094113280,"available":764488712192,"internally_reserved":8589934592,"allocated":15400960,"data_stored":7505640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003565"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":"129","reported_epoch":"150","state":"active+clean","last_fresh":"2020-12-05T17:42:11.669103+0000","last_change":"2020-12-05T17:31:35.374114+0000","last_active":"2020-12-05T17:42:11.669103+0000","last_peered":"2020-12-05T17:42:11.669103+0000","last_clean":"2020-12-05T17:42:11.669103+0000","last_became_active":"2020-12-05T17:31:35.373231+0000","last_became_peered":"2020-12-05T17:31:35.373231+0000","last_unstale":"2020-12-05T17:42:11.669103+0000","last_undegraded":"2020-12-05T17:42:11.669103+0000","last_fullsized":"2020-12-05T17:42:11.669103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"134","reported_epoch":"150","state":"active+clean","last_fresh":"2020-12-05T17:42:11.294377+0000","last_change":"2020-12-05T17:31:35.123847+0000","last_active":"2020-12-05T17:42:11.294377+0000","last_peered":"2020-12-05T17:42:11.294377+0000","last_clean":"2020-12-05T17:42:11.294377+0000","last_became_active":"2020-12-05T17:31:35.123169+0000","last_became_peered":"2020-12-05T17:31:35.123169+0000","last_unstale":"2020-12-05T17:42:11.294377+0000","last_undegraded":"2020-12-05T17:42:11.294377+0000","last_fullsized":"2020-12-05T17:42:11.294377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"128","reported_epoch":"150","state":"active+clean","last_fresh":"2020-12-05T17:42:11.668246+0000","last_change":"2020-12-05T17:31:35.617802+0000","last_active":"2020-12-05T17:42:11.668246+0000","last_peered":"2020-12-05T17:42:11.668246+0000","last_clean":"2020-12-05T17:42:11.668246+0000","last_became_active":"2020-12-05T17:31:35.617427+0000","last_became_peered":"2020-12-05T17:31:35.617427+0000","last_unstale":"2020-12-05T17:42:11.668246+0000","last_undegraded":"2020-12-05T17:42:11.668246+0000","last_fullsized":"2020-12-05T17:42:11.668246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"134","reported_epoch":"150","state":"active+clean","last_fresh":"2020-12-05T17:42:11.294441+0000","last_change":"2020-12-05T17:31:35.124252+0000","last_active":"2020-12-05T17:42:11.294441+0000","last_peered":"2020-12-05T17:42:11.294441+0000","last_clean":"2020-12-05T17:42:11.294441+0000","last_became_active":"2020-12-05T17:31:35.123671+0000","last_became_peered":"2020-12-05T17:31:35.123671+0000","last_unstale":"2020-12-05T17:42:11.294441+0000","last_undegraded":"2020-12-05T17:42:11.294441+0000","last_fullsized":"2020-12-05T17:42:11.294441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"139","reported_epoch":"150","state":"active+clean","last_fresh":"2020-12-05T17:42:11.294592+0000","last_change":"2020-12-05T17:31:30.406003+0000","last_active":"2020-12-05T17:42:11.294592+0000","last_peered":"2020-12-05T17:42:11.294592+0000","last_clean":"2020-12-05T17:42:11.294592+0000","last_became_active":"2020-12-05T17:31:30.405245+0000","last_became_peered":"2020-12-05T17:31:30.405245+0000","last_unstale":"2020-12-05T17:42:11.294592+0000","last_undegraded":"2020-12-05T17:42:11.294592+0000","last_fullsized":"2020-12-05T17:42:11.294592+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:25.806577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:25.806577+0000","last_clean_scrub_stamp":"2020-12-05T17:31:25.806577+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"134","reported_epoch":"150","state":"active+clean","last_fresh":"2020-12-05T17:42:11.294687+0000","last_change":"2020-12-05T17:31:35.118985+0000","last_active":"2020-12-05T17:42:11.294687+0000","last_peered":"2020-12-05T17:42:11.294687+0000","last_clean":"2020-12-05T17:42:11.294687+0000","last_became_active":"2020-12-05T17:31:35.118441+0000","last_became_peered":"2020-12-05T17:31:35.118441+0000","last_unstale":"2020-12-05T17:42:11.294687+0000","last_undegraded":"2020-12-05T17:42:11.294687+0000","last_fullsized":"2020-12-05T17:42:11.294687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"134","reported_epoch":"150","state":"active+clean","last_fresh":"2020-12-05T17:42:11.296171+0000","last_change":"2020-12-05T17:31:35.375163+0000","last_active":"2020-12-05T17:42:11.296171+0000","last_peered":"2020-12-05T17:42:11.296171+0000","last_clean":"2020-12-05T17:42:11.296171+0000","last_became_active":"2020-12-05T17:31:35.374461+0000","last_became_peered":"2020-12-05T17:31:35.374461+0000","last_unstale":"2020-12-05T17:42:11.296171+0000","last_undegraded":"2020-12-05T17:42:11.296171+0000","last_fullsized":"2020-12-05T17:42:11.296171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"129","reported_epoch":"150","state":"active+clean","last_fresh":"2020-12-05T17:42:11.670153+0000","last_change":"2020-12-05T17:31:35.123497+0000","last_active":"2020-12-05T17:42:11.670153+0000","last_peered":"2020-12-05T17:42:11.670153+0000","last_clean":"2020-12-05T17:42:11.670153+0000","last_became_active":"2020-12-05T17:31:35.122972+0000","last_became_peered":"2020-12-05T17:31:35.122972+0000","last_unstale":"2020-12-05T17:42:11.670153+0000","last_undegraded":"2020-12-05T17:42:11.670153+0000","last_fullsized":"2020-12-05T17:42:11.670153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"130","reported_epoch":"150","state":"active+clean","last_fresh":"2020-12-05T17:42:11.667390+0000","last_change":"2020-12-05T17:31:35.127416+0000","last_active":"2020-12-05T17:42:11.667390+0000","last_peered":"2020-12-05T17:42:11.667390+0000","last_clean":"2020-12-05T17:42:11.667390+0000","last_became_active":"2020-12-05T17:31:35.126829+0000","last_became_peered":"2020-12-05T17:31:35.126829+0000","last_unstale":"2020-12-05T17:42:11.667390+0000","last_undegraded":"2020-12-05T17:42:11.667390+0000","last_fullsized":"2020-12-05T17:42:11.667390+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T17:31:34.092187+0000","last_clean_scrub_stamp":"2020-12-05T17:31:34.092187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542635,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":1.7090000000000001,"5min":1.869,"15min":1.8340000000000001},"min":{"1min":1.242,"5min":1.1910000000000001,"15min":1.163},"max":{"1min":2.6299999999999999,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":2.8919999999999999},{"interface":"front","average":{"1min":1.847,"5min":1.8939999999999999,"15min":1.8440000000000001},"min":{"1min":1.4159999999999999,"5min":1.264,"15min":1.0920000000000001},"max":{"1min":3.0579999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":1.2569999999999999}]},{"osd":1,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":1.9299999999999999,"5min":1.8959999999999999,"15min":1.8700000000000001},"min":{"1min":1.452,"5min":1.3779999999999999,"15min":1.24},"max":{"1min":2.8100000000000001,"5min":3.1019999999999999,"15min":3.444},"last":1.107},{"interface":"front","average":{"1min":2.5369999999999999,"5min":2.3290000000000002,"15min":2.3140000000000001},"min":{"1min":1.669,"5min":1.3979999999999999,"15min":1.3979999999999999},"max":{"1min":3.472,"5min":3.472,"15min":3.6629999999999998},"last":1.964}]},{"osd":2,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":2.2789999999999999,"5min":2.3159999999999998,"15min":2.2530000000000001},"min":{"1min":1.597,"5min":1.5249999999999999,"15min":1.296},"max":{"1min":3.2669999999999999,"5min":3.6469999999999998,"15min":3.6469999999999998},"last":2.1019999999999999},{"interface":"front","average":{"1min":2.2410000000000001,"5min":2.3220000000000001,"15min":2.2949999999999999},"min":{"1min":1.7250000000000001,"5min":1.5289999999999999,"15min":1.161},"max":{"1min":3.161,"5min":3.6850000000000001,"15min":3.968},"last":1.51}]},{"osd":3,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":2.524,"5min":2.6040000000000001,"15min":2.6110000000000002},"min":{"1min":1.9419999999999999,"5min":1.6100000000000001,"15min":1.6100000000000001},"max":{"1min":3.6139999999999999,"5min":3.6139999999999999,"15min":3.7709999999999999},"last":2.5009999999999999},{"interface":"front","average":{"1min":2.3690000000000002,"5min":2.5249999999999999,"15min":2.4830000000000001},"min":{"1min":1.673,"5min":1.6220000000000001,"15min":1.6220000000000001},"max":{"1min":3.556,"5min":4.6909999999999998,"15min":4.6909999999999998},"last":2.4239999999999999}]},{"osd":4,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":2.665,"5min":2.718,"15min":2.7090000000000001},"min":{"1min":1.8720000000000001,"5min":1.706,"15min":1.6859999999999999},"max":{"1min":3.98,"5min":3.98,"15min":4.0739999999999998},"last":2.3450000000000002},{"interface":"front","average":{"1min":2.8620000000000001,"5min":2.9300000000000002,"15min":2.9169999999999998},"min":{"1min":1.861,"5min":1.5189999999999999,"15min":1.5189999999999999},"max":{"1min":3.9020000000000001,"5min":3.9020000000000001,"15min":4.0490000000000004},"last":3.294}]},{"osd":5,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":2.7650000000000001,"5min":2.8359999999999999,"15min":2.8380000000000001},"min":{"1min":1.929,"5min":1.7929999999999999,"15min":1.581},"max":{"1min":3.806,"5min":5.1050000000000004,"15min":5.1050000000000004},"last":2.032},{"interface":"front","average":{"1min":3.109,"5min":3.117,"15min":3.113},"min":{"1min":2.4780000000000002,"5min":1.823,"15min":1.823},"max":{"1min":4.2380000000000004,"5min":4.335,"15min":4.375},"last":2.6640000000000001}]},{"osd":6,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":3.226,"5min":3.383,"15min":3.3220000000000001},"min":{"1min":2.7200000000000002,"5min":2.234,"15min":2.194},"max":{"1min":4.2160000000000002,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":2.9910000000000001},{"interface":"front","average":{"1min":3.2040000000000002,"5min":3.4079999999999999,"15min":3.3300000000000001},"min":{"1min":2.7509999999999999,"5min":2.395,"15min":2.395},"max":{"1min":4.1150000000000002,"5min":5.8940000000000001,"15min":5.8940000000000001},"last":3.6539999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542636,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 5 18:11:53 2020","interfaces":[{"interface":"back","average":{"1min":1.744,"5min":1.772,"15min":1.7090000000000001},"min":{"1min":1.3959999999999999,"5min":1.1499999999999999,"15min":1.022},"max":{"1min":3.036,"5min":3.794,"15min":3.794},"last":3.0110000000000001},{"interface":"front","average":{"1min":1.8120000000000001,"5min":1.921,"15min":1.8480000000000001},"min":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.0760000000000001},"max":{"1min":2.3559999999999999,"5min":5.1349999999999998,"15min":5.1349999999999998},"last":1.6819999999999999}]},{"osd":1,"last update":"Sat Dec 5 18:11:53 2020","interfaces":[{"interface":"back","average":{"1min":1.774,"5min":1.885,"15min":1.855},"min":{"1min":1.403,"5min":1.403,"15min":1.224},"max":{"1min":2.1629999999999998,"5min":4.3029999999999999,"15min":4.3029999999999999},"last":1.8100000000000001},{"interface":"front","average":{"1min":2.2480000000000002,"5min":2.234,"15min":2.1930000000000001},"min":{"1min":1.667,"5min":1.3420000000000001,"15min":1.125},"max":{"1min":2.891,"5min":3.258,"15min":3.4620000000000002},"last":1.6060000000000001}]},{"osd":2,"last update":"Sat Dec 5 18:11:53 2020","interfaces":[{"interface":"back","average":{"1min":2.2040000000000002,"5min":2.331,"15min":2.266},"min":{"1min":1.512,"5min":1.456,"15min":1.393},"max":{"1min":3.4319999999999999,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":2.4039999999999999},{"interface":"front","average":{"1min":2.544,"5min":2.6000000000000001,"15min":2.581},"min":{"1min":1.9490000000000001,"5min":1.7849999999999999,"15min":1.5580000000000001},"max":{"1min":3.3599999999999999,"5min":4.6989999999999998,"15min":4.6989999999999998},"last":2.6869999999999998}]},{"osd":3,"last update":"Sat Dec 5 18:11:53 2020","interfaces":[{"interface":"back","average":{"1min":2.6640000000000001,"5min":2.6389999999999998,"15min":2.544},"min":{"1min":1.8919999999999999,"5min":1.8320000000000001,"15min":1.643},"max":{"1min":3.702,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":2.2250000000000001},{"interface":"front","average":{"1min":2.8260000000000001,"5min":2.71,"15min":2.661},"min":{"1min":1.9870000000000001,"5min":1.5129999999999999,"15min":1.298},"max":{"1min":3.8929999999999998,"5min":4.4729999999999999,"15min":4.4729999999999999},"last":2.7349999999999999}]},{"osd":4,"last update":"Sat Dec 5 18:11:53 2020","interfaces":[{"interface":"back","average":{"1min":2.9740000000000002,"5min":2.8380000000000001,"15min":2.786},"min":{"1min":2.1909999999999998,"5min":1.117,"15min":1.117},"max":{"1min":4.0919999999999996,"5min":4.1779999999999999,"15min":4.1779999999999999},"last":2.6030000000000002},{"interface":"front","average":{"1min":3.0350000000000001,"5min":2.9319999999999999,"15min":2.8969999999999998},"min":{"1min":2.5430000000000001,"5min":1.7210000000000001,"15min":1.264},"max":{"1min":4.2670000000000003,"5min":5.5330000000000004,"15min":5.5330000000000004},"last":2.948}]},{"osd":5,"last update":"Sat Dec 5 18:11:53 2020","interfaces":[{"interface":"back","average":{"1min":2.6110000000000002,"5min":2.6389999999999998,"15min":2.6230000000000002},"min":{"1min":1.96,"5min":1.784,"15min":1.165},"max":{"1min":3.653,"5min":4.7699999999999996,"15min":4.7699999999999996},"last":2.3069999999999999},{"interface":"front","average":{"1min":3.2490000000000001,"5min":3.1850000000000001,"15min":3.1459999999999999},"min":{"1min":2.5830000000000002,"5min":1.429,"15min":1.429},"max":{"1min":4.4589999999999996,"5min":4.5,"15min":4.5},"last":3.4510000000000001}]},{"osd":7,"last update":"Sat Dec 5 18:11:53 2020","interfaces":[{"interface":"back","average":{"1min":3.4740000000000002,"5min":3.3380000000000001,"15min":3.2919999999999998},"min":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.077},"max":{"1min":4.6459999999999999,"5min":5.9379999999999997,"15min":5.9379999999999997},"last":4.0970000000000004},{"interface":"front","average":{"1min":3.23,"5min":3.222,"15min":3.202},"min":{"1min":2.5179999999999998,"5min":2.1659999999999999,"15min":2.0169999999999999},"max":{"1min":4.1219999999999999,"5min":4.9909999999999997,"15min":4.9909999999999997},"last":4.032}]}]},{"osd":1,"up_from":14,"seq":60129542635,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":1.766,"5min":1.714,"15min":1.7809999999999999},"min":{"1min":1.3919999999999999,"5min":1.383,"15min":1.093},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":3.3180000000000001},"last":2.742},{"interface":"front","average":{"1min":1.671,"5min":1.8160000000000001,"15min":1.8120000000000001},"min":{"1min":1.502,"5min":1.393,"15min":1.1830000000000001},"max":{"1min":2.242,"5min":2.6040000000000001,"15min":2.9500000000000002},"last":2.2200000000000002}]},{"osd":2,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":1.6839999999999999,"5min":1.825,"15min":1.8009999999999999},"min":{"1min":1.4339999999999999,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.4159999999999999,"5min":3.3679999999999999,"15min":3.3679999999999999},"last":1.756},{"interface":"front","average":{"1min":2.4809999999999999,"5min":2.3210000000000002,"15min":2.2149999999999999},"min":{"1min":1.9430000000000001,"5min":1.546,"15min":1.0840000000000001},"max":{"1min":2.9769999999999999,"5min":3.2469999999999999,"15min":3.4849999999999999},"last":1.827}]},{"osd":3,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":2.335,"5min":2.395,"15min":2.3900000000000001},"min":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.353},"max":{"1min":3.0600000000000001,"5min":3.379,"15min":3.3849999999999998},"last":2.266},{"interface":"front","average":{"1min":2.3650000000000002,"5min":2.379,"15min":2.4039999999999999},"min":{"1min":1.8169999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"max":{"1min":2.843,"5min":3.3140000000000001,"15min":3.4020000000000001},"last":2.4609999999999999}]},{"osd":4,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":2.4140000000000001,"5min":2.3839999999999999,"15min":2.3610000000000002},"min":{"1min":2.0019999999999998,"5min":1.627,"15min":1.587},"max":{"1min":3.7879999999999998,"5min":4.0899999999999999,"15min":4.1429999999999998},"last":2.3399999999999999},{"interface":"front","average":{"1min":3.113,"5min":2.98,"15min":2.9380000000000002},"min":{"1min":2.7959999999999998,"5min":2.0800000000000001,"15min":1.5649999999999999},"max":{"1min":3.5270000000000001,"5min":3.9580000000000002,"15min":4.0119999999999996},"last":2.5110000000000001}]},{"osd":5,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":2.7519999999999998,"5min":2.8690000000000002,"15min":2.8359999999999999},"min":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":1.75},"max":{"1min":3.4710000000000001,"5min":3.9860000000000002,"15min":4.1020000000000003},"last":2.6699999999999999},{"interface":"front","average":{"1min":2.7709999999999999,"5min":2.8530000000000002,"15min":2.847},"min":{"1min":2.024,"5min":2.024,"15min":2.008},"max":{"1min":3.702,"5min":3.75,"15min":3.919},"last":3.0369999999999999}]},{"osd":6,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":3.395,"5min":3.2679999999999998,"15min":3.2370000000000001},"min":{"1min":3.113,"5min":2.5880000000000001,"15min":2.1800000000000002},"max":{"1min":3.944,"5min":4.282,"15min":4.3879999999999999},"last":2.9670000000000001},{"interface":"front","average":{"1min":3.1720000000000002,"5min":3.222,"15min":3.1899999999999999},"min":{"1min":2.7090000000000001,"5min":2.5190000000000001,"15min":2.2879999999999998},"max":{"1min":3.9180000000000001,"5min":4.399,"15min":4.399},"last":3.1280000000000001}]},{"osd":7,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":3.3759999999999999,"5min":3.3999999999999999,"15min":3.3570000000000002},"min":{"1min":2.552,"5min":2.552,"15min":2.5059999999999998},"max":{"1min":4.0919999999999996,"5min":4.5279999999999996,"15min":4.5279999999999996},"last":3.5470000000000002},{"interface":"front","average":{"1min":3.1459999999999999,"5min":3.1850000000000001,"15min":3.153},"min":{"1min":2.5880000000000001,"5min":2.4470000000000001,"15min":2.2669999999999999},"max":{"1min":3.5990000000000002,"5min":4.2329999999999997,"15min":4.2329999999999997},"last":3.4420000000000002}]}]},{"osd":0,"up_from":14,"seq":60129542636,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Dec 5 18:12:06 2020","interfaces":[{"interface":"back","average":{"1min":1.8140000000000001,"5min":1.7589999999999999,"15min":1.7709999999999999},"min":{"1min":1.4019999999999999,"5min":1.3149999999999999,"15min":1.0349999999999999},"max":{"1min":2.9239999999999999,"5min":2.9369999999999998,"15min":3.0510000000000002},"last":2.2890000000000001},{"interface":"front","average":{"1min":1.8149999999999999,"5min":1.8180000000000001,"15min":1.831},"min":{"1min":1.53,"5min":1.2929999999999999,"15min":1.2929999999999999},"max":{"1min":3.0070000000000001,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":1.554}]},{"osd":2,"last update":"Sat Dec 5 18:12:06 2020","interfaces":[{"interface":"back","average":{"1min":2.2410000000000001,"5min":2.2789999999999999,"15min":2.2160000000000002},"min":{"1min":1.488,"5min":1.3819999999999999,"15min":1.3480000000000001},"max":{"1min":2.7709999999999999,"5min":3.1890000000000001,"15min":3.2120000000000002},"last":1.865},{"interface":"front","average":{"1min":2.6150000000000002,"5min":2.496,"15min":2.4830000000000001},"min":{"1min":2.0339999999999998,"5min":1.498,"15min":1.407},"max":{"1min":3.2080000000000002,"5min":3.3839999999999999,"15min":3.6600000000000001},"last":1.8180000000000001}]},{"osd":3,"last update":"Sat Dec 5 18:12:06 2020","interfaces":[{"interface":"back","average":{"1min":1.98,"5min":1.8859999999999999,"15min":1.8600000000000001},"min":{"1min":1.4930000000000001,"5min":1.385,"15min":1.2130000000000001},"max":{"1min":2.9039999999999999,"5min":2.9039999999999999,"15min":3.984},"last":1.974},{"interface":"front","average":{"1min":2.395,"5min":2.476,"15min":2.496},"min":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"max":{"1min":2.895,"5min":3.7429999999999999,"15min":4.1790000000000003},"last":3.077}]},{"osd":4,"last update":"Sat Dec 5 18:12:06 2020","interfaces":[{"interface":"back","average":{"1min":2.6880000000000002,"5min":2.5630000000000002,"15min":2.452},"min":{"1min":2.0600000000000001,"5min":1.669,"15min":1.4970000000000001},"max":{"1min":3.8540000000000001,"5min":3.9660000000000002,"15min":3.9660000000000002},"last":2.4620000000000002},{"interface":"front","average":{"1min":2.4420000000000002,"5min":2.536,"15min":2.5070000000000001},"min":{"1min":1.802,"5min":1.397,"15min":1.397},"max":{"1min":3.7330000000000001,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":3.2879999999999998}]},{"osd":5,"last update":"Sat Dec 5 18:12:06 2020","interfaces":[{"interface":"back","average":{"1min":2.3849999999999998,"5min":2.5449999999999999,"15min":2.5129999999999999},"min":{"1min":1.917,"5min":1.917,"15min":1.4850000000000001},"max":{"1min":3.0449999999999999,"5min":3.6259999999999999,"15min":3.8260000000000001},"last":3.6989999999999998},{"interface":"front","average":{"1min":2.9529999999999998,"5min":3.0289999999999999,"15min":3.0350000000000001},"min":{"1min":2.5150000000000001,"5min":2.1829999999999998,"15min":1.9610000000000001},"max":{"1min":3.2509999999999999,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":3.548}]},{"osd":6,"last update":"Sat Dec 5 18:12:06 2020","interfaces":[{"interface":"back","average":{"1min":3.25,"5min":3.2589999999999999,"15min":3.2490000000000001},"min":{"1min":2.8769999999999998,"5min":2.5369999999999999,"15min":2.2879999999999998},"max":{"1min":3.681,"5min":4.5709999999999997,"15min":5.3280000000000003},"last":3.7970000000000002},{"interface":"front","average":{"1min":3.3300000000000001,"5min":3.0830000000000002,"15min":3.0630000000000002},"min":{"1min":2.758,"5min":2.2069999999999999,"15min":2.1709999999999998},"max":{"1min":3.77,"5min":3.7730000000000001,"15min":4.2519999999999998},"last":2.3650000000000002}]},{"osd":7,"last update":"Sat Dec 5 18:12:06 2020","interfaces":[{"interface":"back","average":{"1min":3.4399999999999999,"5min":3.3050000000000002,"15min":3.2639999999999998},"min":{"1min":2.75,"5min":2.5760000000000001,"15min":2.456},"max":{"1min":4,"5min":4.0599999999999996,"15min":4.3300000000000001},"last":2.2240000000000002},{"interface":"front","average":{"1min":3.0209999999999999,"5min":3.0579999999999998,"15min":3.028},"min":{"1min":2.6360000000000001,"5min":1.849,"15min":1.849},"max":{"1min":3.3170000000000002,"5min":3.9990000000000001,"15min":3.9990000000000001},"last":4.1139999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542635,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 5 18:11:49 2020","interfaces":[{"interface":"back","average":{"1min":1.714,"5min":1.849,"15min":1.8149999999999999},"min":{"1min":1.3440000000000001,"5min":1.171,"15min":1.141},"max":{"1min":2.3839999999999999,"5min":3.052,"15min":3.6659999999999999},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.823,"5min":1.8069999999999999,"15min":1.806},"min":{"1min":1.452,"5min":1.3240000000000001,"15min":0.96499999999999997},"max":{"1min":2.246,"5min":3.202,"15min":3.5310000000000001},"last":2.379}]},{"osd":1,"last update":"Sat Dec 5 18:11:49 2020","interfaces":[{"interface":"back","average":{"1min":2.1560000000000001,"5min":2.1259999999999999,"15min":2.1219999999999999},"min":{"1min":1.097,"5min":1.097,"15min":1.097},"max":{"1min":2.9049999999999998,"5min":3.0800000000000001,"15min":3.3300000000000001},"last":2.073},{"interface":"front","average":{"1min":2.4830000000000001,"5min":2.4849999999999999,"15min":2.52},"min":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.3009999999999999},"max":{"1min":3.1419999999999999,"5min":3.5230000000000001,"15min":3.5510000000000002},"last":2.9390000000000001}]},{"osd":3,"last update":"Sat Dec 5 18:11:49 2020","interfaces":[{"interface":"back","average":{"1min":1.899,"5min":1.857,"15min":1.8959999999999999},"min":{"1min":1.149,"5min":1.149,"15min":1.149},"max":{"1min":2.6930000000000001,"5min":2.6930000000000001,"15min":3.0470000000000002},"last":2.6190000000000002},{"interface":"front","average":{"1min":2.3849999999999998,"5min":2.403,"15min":2.4390000000000001},"min":{"1min":1.581,"5min":1.54,"15min":1.4530000000000001},"max":{"1min":3.0049999999999999,"5min":3.153,"15min":3.536},"last":2.262}]},{"osd":4,"last update":"Sat Dec 5 18:11:49 2020","interfaces":[{"interface":"back","average":{"1min":2.5230000000000001,"5min":2.4489999999999998,"15min":2.3639999999999999},"min":{"1min":1.6990000000000001,"5min":1.502,"15min":1.431},"max":{"1min":3.8170000000000002,"5min":4.6159999999999997,"15min":4.6159999999999997},"last":1.7849999999999999},{"interface":"front","average":{"1min":2.5430000000000001,"5min":2.4630000000000001,"15min":2.3889999999999998},"min":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"max":{"1min":3.6309999999999998,"5min":3.8410000000000002,"15min":3.9009999999999998},"last":2.8559999999999999}]},{"osd":5,"last update":"Sat Dec 5 18:11:49 2020","interfaces":[{"interface":"back","average":{"1min":2.875,"5min":2.9380000000000002,"15min":2.899},"min":{"1min":1.726,"5min":1.726,"15min":1.498},"max":{"1min":3.4079999999999999,"5min":4.1929999999999996,"15min":4.1929999999999996},"last":2.8119999999999998},{"interface":"front","average":{"1min":3.1549999999999998,"5min":3.0419999999999998,"15min":2.9670000000000001},"min":{"1min":2.4900000000000002,"5min":2.198,"15min":1.7509999999999999},"max":{"1min":3.9529999999999998,"5min":4.3289999999999997,"15min":4.3289999999999997},"last":3.335}]},{"osd":6,"last update":"Sat Dec 5 18:11:49 2020","interfaces":[{"interface":"back","average":{"1min":2.9700000000000002,"5min":2.8940000000000001,"15min":2.8519999999999999},"min":{"1min":2.371,"5min":1.9810000000000001,"15min":1.476},"max":{"1min":3.895,"5min":4.476,"15min":4.476},"last":3.1120000000000001},{"interface":"front","average":{"1min":3.173,"5min":3.23,"15min":3.1909999999999998},"min":{"1min":2.331,"5min":2.2810000000000001,"15min":1.6870000000000001},"max":{"1min":3.782,"5min":4.1399999999999997,"15min":4.2439999999999998},"last":2.5150000000000001}]},{"osd":7,"last update":"Sat Dec 5 18:11:49 2020","interfaces":[{"interface":"back","average":{"1min":3.2949999999999999,"5min":3.27,"15min":3.246},"min":{"1min":2.4900000000000002,"5min":2.4900000000000002,"15min":2.177},"max":{"1min":4.0049999999999999,"5min":4.1820000000000004,"15min":4.2709999999999999},"last":3.7210000000000001},{"interface":"front","average":{"1min":3.2629999999999999,"5min":3.1669999999999998,"15min":3.1360000000000001},"min":{"1min":2.7869999999999999,"5min":2.4950000000000001,"15min":1.885},"max":{"1min":3.6669999999999998,"5min":4.0780000000000003,"15min":4.0780000000000003},"last":3.9889999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542635,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.6319999999999999,"15min":1.722},"min":{"1min":1.3899999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":2.5579999999999998,"5min":2.5579999999999998,"15min":2.7349999999999999},"last":1.8600000000000001},{"interface":"front","average":{"1min":1.7350000000000001,"5min":1.748,"15min":1.766},"min":{"1min":1.486,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":2.2010000000000001,"5min":2.6920000000000002,"15min":2.8399999999999999},"last":2.0619999999999998}]},{"osd":1,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":1.8959999999999999,"5min":1.75,"15min":1.768},"min":{"1min":1.3839999999999999,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":2.7400000000000002,"5min":2.7400000000000002,"15min":3.5539999999999998},"last":1.9450000000000001},{"interface":"front","average":{"1min":2.1469999999999998,"5min":2.2549999999999999,"15min":2.242},"min":{"1min":1.7729999999999999,"5min":1.228,"15min":1.228},"max":{"1min":2.5299999999999998,"5min":3.1269999999999998,"15min":3.2370000000000001},"last":2.5920000000000001}]},{"osd":2,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.4260000000000002,"5min":2.2770000000000001,"15min":2.3170000000000002},"min":{"1min":1.9730000000000001,"5min":1.4179999999999999,"15min":1.236},"max":{"1min":3.1659999999999999,"5min":3.2229999999999999,"15min":3.4470000000000001},"last":2.6890000000000001},{"interface":"front","average":{"1min":2.4079999999999999,"5min":2.4529999999999998,"15min":2.4620000000000002},"min":{"1min":1.954,"5min":1.7689999999999999,"15min":1.595},"max":{"1min":2.9449999999999998,"5min":3.323,"15min":3.3849999999999998},"last":2.9670000000000001}]},{"osd":4,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.5089999999999999,"5min":2.5369999999999999,"15min":2.6190000000000002},"min":{"1min":1.829,"5min":1.552,"15min":1.552},"max":{"1min":3.4409999999999998,"5min":3.4409999999999998,"15min":3.8300000000000001},"last":3.105},{"interface":"front","average":{"1min":2.2890000000000001,"5min":2.3530000000000002,"15min":2.4159999999999999},"min":{"1min":1.7909999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"max":{"1min":3.0009999999999999,"5min":3.8860000000000001,"15min":4.4039999999999999},"last":2.758}]},{"osd":5,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.9649999999999999,"5min":2.9289999999999998,"15min":2.9590000000000001},"min":{"1min":2.411,"5min":1.9419999999999999,"15min":1.556},"max":{"1min":3.6469999999999998,"5min":4.1420000000000003,"15min":4.1420000000000003},"last":3.4380000000000002},{"interface":"front","average":{"1min":2.964,"5min":2.9729999999999999,"15min":3.0470000000000002},"min":{"1min":2.4079999999999999,"5min":1.716,"15min":1.716},"max":{"1min":3.6469999999999998,"5min":3.9950000000000001,"15min":4.3300000000000001},"last":3.359}]},{"osd":6,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.8769999999999998,"5min":2.9039999999999999,"15min":2.919},"min":{"1min":2.2269999999999999,"5min":1.9950000000000001,"15min":1.972},"max":{"1min":3.6589999999999998,"5min":3.8199999999999998,"15min":5.3570000000000002},"last":3.198},{"interface":"front","average":{"1min":3.149,"5min":3.181,"15min":3.2120000000000002},"min":{"1min":2.5739999999999998,"5min":2.1400000000000001,"15min":2.1400000000000001},"max":{"1min":3.7839999999999998,"5min":3.8900000000000001,"15min":4.0179999999999998},"last":3.8319999999999999}]},{"osd":7,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":3.1989999999999998,"5min":3.2389999999999999,"15min":3.2799999999999998},"min":{"1min":2.7570000000000001,"5min":2.1840000000000002,"15min":2.1840000000000002},"max":{"1min":3.9020000000000001,"5min":3.9510000000000001,"15min":4.665},"last":3.593},{"interface":"front","average":{"1min":3.048,"5min":3.1259999999999999,"15min":3.1640000000000001},"min":{"1min":2.4359999999999999,"5min":2.2530000000000001,"15min":2.0209999999999999},"max":{"1min":3.669,"5min":4.0599999999999996,"15min":5.742},"last":3.5339999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542636,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 5 18:11:54 2020","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.5920000000000001,"15min":1.6879999999999999},"min":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"max":{"1min":2.6800000000000002,"5min":3.032,"15min":3.032},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.6970000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"min":{"1min":1.3049999999999999,"5min":1.2490000000000001,"15min":1.0529999999999999},"max":{"1min":2.718,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":1.5620000000000001}]},{"osd":1,"last update":"Sat Dec 5 18:11:54 2020","interfaces":[{"interface":"back","average":{"1min":2.081,"5min":2.2109999999999999,"15min":2.1589999999999998},"min":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.337},"max":{"1min":2.722,"5min":3.2290000000000001,"15min":3.3919999999999999},"last":2.7090000000000001},{"interface":"front","average":{"1min":2.5329999999999999,"5min":2.6560000000000001,"15min":2.6299999999999999},"min":{"1min":1.694,"5min":1.5049999999999999,"15min":1.5049999999999999},"max":{"1min":3.1859999999999999,"5min":3.6709999999999998,"15min":3.6709999999999998},"last":2.1389999999999998}]},{"osd":2,"last update":"Sat Dec 5 18:11:54 2020","interfaces":[{"interface":"back","average":{"1min":2.0960000000000001,"5min":2.044,"15min":2.0150000000000001},"min":{"1min":1.631,"5min":1.5409999999999999,"15min":1.3380000000000001},"max":{"1min":3.153,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.653},{"interface":"front","average":{"1min":2.4590000000000001,"5min":2.464,"15min":2.419},"min":{"1min":1.667,"5min":1.667,"15min":1.2849999999999999},"max":{"1min":3.129,"5min":3.452,"15min":3.681},"last":2.23}]},{"osd":3,"last update":"Sat Dec 5 18:11:54 2020","interfaces":[{"interface":"back","average":{"1min":2.2160000000000002,"5min":2.1389999999999998,"15min":2.1680000000000001},"min":{"1min":1.6659999999999999,"5min":1.212,"15min":1.212},"max":{"1min":2.9870000000000001,"5min":3.085,"15min":3.677},"last":2.0009999999999999},{"interface":"front","average":{"1min":2.472,"5min":2.4660000000000002,"15min":2.4489999999999998},"min":{"1min":1.71,"5min":1.3380000000000001,"15min":1.3380000000000001},"max":{"1min":3.1110000000000002,"5min":3.4289999999999998,"15min":3.4740000000000002},"last":2.6200000000000001}]},{"osd":5,"last update":"Sat Dec 5 18:11:54 2020","interfaces":[{"interface":"back","average":{"1min":2.7309999999999999,"5min":2.7829999999999999,"15min":2.722},"min":{"1min":1.8879999999999999,"5min":1.8400000000000001,"15min":1.51},"max":{"1min":3.6549999999999998,"5min":3.9729999999999999,"15min":3.9910000000000001},"last":1.7709999999999999},{"interface":"front","average":{"1min":2.6230000000000002,"5min":2.621,"15min":2.617},"min":{"1min":1.764,"5min":1.764,"15min":1.5660000000000001},"max":{"1min":3.3570000000000002,"5min":3.8130000000000002,"15min":3.8130000000000002},"last":2.536}]},{"osd":6,"last update":"Sat Dec 5 18:11:54 2020","interfaces":[{"interface":"back","average":{"1min":2.8290000000000002,"5min":2.8519999999999999,"15min":2.8220000000000001},"min":{"1min":2.0569999999999999,"5min":1.7889999999999999,"15min":1.139},"max":{"1min":3.3439999999999999,"5min":3.7989999999999999,"15min":3.8740000000000001},"last":3.0129999999999999},{"interface":"front","average":{"1min":2.9929999999999999,"5min":3.069,"15min":3.0449999999999999},"min":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"max":{"1min":3.8969999999999998,"5min":4.0430000000000001,"15min":4.3760000000000003},"last":3.097}]},{"osd":7,"last update":"Sat Dec 5 18:11:54 2020","interfaces":[{"interface":"back","average":{"1min":3.0329999999999999,"5min":3.044,"15min":3.0259999999999998},"min":{"1min":2.2999999999999998,"5min":2.1640000000000001,"15min":1.0580000000000001},"max":{"1min":3.6659999999999999,"5min":4.1340000000000003,"15min":4.2290000000000001},"last":3.1859999999999999},{"interface":"front","average":{"1min":3.2730000000000001,"5min":3.3079999999999998,"15min":3.2789999999999999},"min":{"1min":2.73,"5min":2.1699999999999999,"15min":1.611},"max":{"1min":3.7570000000000001,"5min":4.1779999999999999,"15min":4.3339999999999996},"last":3.2280000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542635,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.615},"min":{"1min":1.222,"5min":1.1200000000000001,"15min":1.1200000000000001},"max":{"1min":2.3490000000000002,"5min":2.7120000000000002,"15min":2.7570000000000001},"last":1.532},{"interface":"front","average":{"1min":1.647,"5min":1.78,"15min":1.7549999999999999},"min":{"1min":1.286,"5min":1.2509999999999999,"15min":1.1639999999999999},"max":{"1min":2.5910000000000002,"5min":3.1859999999999999,"15min":3.1859999999999999},"last":1.8009999999999999}]},{"osd":1,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.3559999999999999,"5min":2.4359999999999999,"15min":2.444},"min":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":0.997},"max":{"1min":3.3250000000000002,"5min":3.4870000000000001,"15min":3.4870000000000001},"last":3.665},{"interface":"front","average":{"1min":2.6219999999999999,"5min":2.5249999999999999,"15min":2.625},"min":{"1min":1.528,"5min":1.4159999999999999,"15min":1.292},"max":{"1min":3.3809999999999998,"5min":3.6459999999999999,"15min":4.1139999999999999},"last":2.6429999999999998}]},{"osd":2,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.008,"5min":2.0619999999999998,"15min":2.0419999999999998},"min":{"1min":1.5980000000000001,"5min":1.3109999999999999,"15min":1.3109999999999999},"max":{"1min":3.2189999999999999,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.653},{"interface":"front","average":{"1min":2.2730000000000001,"5min":2.2389999999999999,"15min":2.246},"min":{"1min":1.7410000000000001,"5min":1.52,"15min":1.478},"max":{"1min":3.3519999999999999,"5min":3.3519999999999999,"15min":3.468},"last":2.1360000000000001}]},{"osd":3,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.4649999999999999,"5min":2.3919999999999999,"15min":2.3730000000000002},"min":{"1min":1.8899999999999999,"5min":1.4330000000000001,"15min":1.4330000000000001},"max":{"1min":3.1179999999999999,"5min":3.6480000000000001,"15min":3.7810000000000001},"last":2.5449999999999999},{"interface":"front","average":{"1min":2.5350000000000001,"5min":2.4689999999999999,"15min":2.4500000000000002},"min":{"1min":2.0190000000000001,"5min":1.5669999999999999,"15min":1.5669999999999999},"max":{"1min":3.4239999999999999,"5min":3.6629999999999998,"15min":3.7930000000000001},"last":2.262}]},{"osd":4,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.629,"5min":2.544,"15min":2.4990000000000001},"min":{"1min":2.0179999999999998,"5min":1.6240000000000001,"15min":1.6240000000000001},"max":{"1min":3.431,"5min":3.6520000000000001,"15min":3.8340000000000001},"last":1.9419999999999999},{"interface":"front","average":{"1min":2.8929999999999998,"5min":2.7930000000000001,"15min":2.7989999999999999},"min":{"1min":2.1549999999999998,"5min":1.8839999999999999,"15min":1.647},"max":{"1min":3.4209999999999998,"5min":4.0270000000000001,"15min":4.2789999999999999},"last":3.3039999999999998}]},{"osd":6,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.9390000000000001,"5min":2.9590000000000001,"15min":2.879},"min":{"1min":2.2269999999999999,"5min":1.806,"15min":1.635},"max":{"1min":3.4209999999999998,"5min":3.992,"15min":4.093},"last":2.8519999999999999},{"interface":"front","average":{"1min":2.9870000000000001,"5min":2.911,"15min":2.867},"min":{"1min":2.3100000000000001,"5min":1.274,"15min":1.274},"max":{"1min":3.609,"5min":3.7770000000000001,"15min":4.2030000000000003},"last":2.387}]},{"osd":7,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":3.29,"5min":3.1789999999999998,"15min":3.1469999999999998},"min":{"1min":2.556,"5min":1.754,"15min":1.754},"max":{"1min":3.7850000000000001,"5min":4.3959999999999999,"15min":4.625},"last":2.952},{"interface":"front","average":{"1min":3.4289999999999998,"5min":3.3100000000000001,"15min":3.2959999999999998},"min":{"1min":2.859,"5min":2.226,"15min":2.153},"max":{"1min":4.0949999999999998,"5min":4.1970000000000001,"15min":4.556},"last":3.0590000000000002}]}]}]}} 2020-12-05T18:12:18.401 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2020-12-05T17:31:34.092187+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=5, tm_hour=17, tm_min=31, tm_sec=34, tm_wday=5, tm_yday=340, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=5, tm_hour=18, tm_min=12, tm_sec=13, tm_wday=5, tm_yday=340, tm_isdst=0) 2020-12-05T18:12:18.401 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2020-12-05T17:31:34.092187+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=5, tm_hour=17, tm_min=31, tm_sec=34, tm_wday=5, tm_yday=340, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=5, tm_hour=18, tm_min=12, tm_sec=13, tm_wday=5, tm_yday=340, tm_isdst=0) 2020-12-05T18:12:18.402 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2020-12-05T17:31:34.092187+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=5, tm_hour=17, tm_min=31, tm_sec=34, tm_wday=5, tm_yday=340, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=5, tm_hour=18, tm_min=12, tm_sec=13, tm_wday=5, tm_yday=340, tm_isdst=0) 2020-12-05T18:12:18.402 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2020-12-05T17:31:34.092187+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=5, tm_hour=17, tm_min=31, tm_sec=34, tm_wday=5, tm_yday=340, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=5, tm_hour=18, tm_min=12, tm_sec=13, tm_wday=5, tm_yday=340, tm_isdst=0) 2020-12-05T18:12:18.403 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2020-12-05T17:31:25.806577+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=5, tm_hour=17, tm_min=31, tm_sec=25, tm_wday=5, tm_yday=340, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=5, tm_hour=18, tm_min=12, tm_sec=13, tm_wday=5, tm_yday=340, tm_isdst=0) 2020-12-05T18:12:18.403 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2020-12-05T17:31:34.092187+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=5, tm_hour=17, tm_min=31, tm_sec=34, tm_wday=5, tm_yday=340, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=5, tm_hour=18, tm_min=12, tm_sec=13, tm_wday=5, tm_yday=340, tm_isdst=0) 2020-12-05T18:12:18.404 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2020-12-05T17:31:34.092187+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=5, tm_hour=17, tm_min=31, tm_sec=34, tm_wday=5, tm_yday=340, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=5, tm_hour=18, tm_min=12, tm_sec=13, tm_wday=5, tm_yday=340, tm_isdst=0) 2020-12-05T18:12:18.405 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2020-12-05T17:31:34.092187+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=5, tm_hour=17, tm_min=31, tm_sec=34, tm_wday=5, tm_yday=340, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=5, tm_hour=18, tm_min=12, tm_sec=13, tm_wday=5, tm_yday=340, tm_isdst=0) 2020-12-05T18:12:18.405 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2020-12-05T17:31:34.092187+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=5, tm_hour=17, tm_min=31, tm_sec=34, tm_wday=5, tm_yday=340, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=5, tm_hour=18, tm_min=12, tm_sec=13, tm_wday=5, tm_yday=340, tm_isdst=0) 2020-12-05T18:12:18.406 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2020-12-05T18:12:20.130 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:12:20.133 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:12:20.135 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:12:20.137 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:12:20.140 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:12:38.407 INFO:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-12-05T18:12:38.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.524+0000 7f91effff700 1 Processor -- start 2020-12-05T18:12:38.529 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.524+0000 7f91effff700 1 -- start start 2020-12-05T18:12:38.530 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.524+0000 7f91effff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f91f005f6f0 0x7f91f01428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:38.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91effff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91f00b1980 0x7f91f0146580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:38.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91effff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f005fad0 con 0x7f91f0146a70 2020-12-05T18:12:38.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91effff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f00b2f80 con 0x7f91f005f6f0 2020-12-05T18:12:38.531 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91effff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f0145e40 con 0x7f91f00b1980 2020-12-05T18:12:38.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91ee7fc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91f00b1980 0x7f91f0146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:38.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91eeffd700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f91f005f6f0 0x7f91f01428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:38.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91ee7fc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91f00b1980 0x7f91f0146580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.32:39048/0 (socket says 172.21.15.32:39048) 2020-12-05T18:12:38.532 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91ee7fc700 1 -- 172.21.15.32:0/4215463642 learned_addr learned my addr 172.21.15.32:0/4215463642 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:38.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91ee7fc700 1 -- 172.21.15.32:0/4215463642 >> v2:172.21.15.61:3300/0 conn(0x7f91f005f6f0 msgr2=0x7f91f01428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T18:12:38.534 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91ee7fc700 1 --2- 172.21.15.32:0/4215463642 >> v2:172.21.15.61:3300/0 conn(0x7f91f005f6f0 0x7f91f01428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:38.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91ee7fc700 1 -- 172.21.15.32:0/4215463642 >> v1:172.21.15.61:6789/0 conn(0x7f91f0146a70 legacy=0x7f91f0066070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:38.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91ee7fc700 1 -- 172.21.15.32:0/4215463642 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f01460e0 con 0x7f91f00b1980 2020-12-05T18:12:38.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91ee7fc700 1 --2- 172.21.15.32:0/4215463642 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91f00b1980 0x7f91f0146580 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f91d8002f30 tx=0x7f91d8006e80).ready entity=mon.2 client_cookie=1d754c3609e8097a server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:38.535 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91edffb700 1 -- 172.21.15.32:0/4215463642 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91d80097f0 con 0x7f91f00b1980 2020-12-05T18:12:38.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91edffb700 1 -- 172.21.15.32:0/4215463642 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d8009950 con 0x7f91f00b1980 2020-12-05T18:12:38.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91edffb700 1 -- 172.21.15.32:0/4215463642 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91d8009cb0 con 0x7f91f00b1980 2020-12-05T18:12:38.536 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91effff700 1 -- 172.21.15.32:0/4215463642 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91f00b1980 msgr2=0x7f91f0146580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:38.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91effff700 1 --2- 172.21.15.32:0/4215463642 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91f00b1980 0x7f91f0146580 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f91d8002f30 tx=0x7f91d8006e80).stop 2020-12-05T18:12:38.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91effff700 1 -- 172.21.15.32:0/4215463642 shutdown_connections 2020-12-05T18:12:38.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91effff700 1 --2- 172.21.15.32:0/4215463642 >> v2:172.21.15.61:3300/0 conn(0x7f91f005f6f0 0x7f91f01428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:38.537 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91effff700 1 --2- 172.21.15.32:0/4215463642 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91f00b1980 0x7f91f0146580 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91effff700 1 -- 172.21.15.32:0/4215463642 >> 172.21.15.32:0/4215463642 conn(0x7f91f00604c0 msgr2=0x7f91f005c830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91effff700 1 -- 172.21.15.32:0/4215463642 shutdown_connections 2020-12-05T18:12:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.528+0000 7f91effff700 1 -- 172.21.15.32:0/4215463642 wait complete. 2020-12-05T18:12:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91effff700 1 Processor -- start 2020-12-05T18:12:38.539 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91effff700 1 -- start start 2020-12-05T18:12:38.541 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91effff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91f005f6f0 0x7f91f0064480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:38.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91effff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91f00b1980 0x7f91f00b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:38.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91effff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91f0146a70 0x7f91f0062fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:38.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91effff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f0065920 con 0x7f91f005f6f0 2020-12-05T18:12:38.542 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91effff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f0114ca0 con 0x7f91f00b1980 2020-12-05T18:12:38.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91effff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f0114f40 con 0x7f91f0146a70 2020-12-05T18:12:38.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91eeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91f005f6f0 0x7f91f0064480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:38.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91ee7fc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91f00b1980 0x7f91f00b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:38.543 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91eeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91f005f6f0 0x7f91f0064480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42260/0 (socket says 172.21.15.32:42260) 2020-12-05T18:12:38.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91ef7fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91f0146a70 0x7f91f0062fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:38.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91eeffd700 1 -- 172.21.15.32:0/2942086161 learned_addr learned my addr 172.21.15.32:0/2942086161 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:38.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91ef7fe700 1 -- 172.21.15.32:0/2942086161 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91f00b1980 msgr2=0x7f91f00b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:38.544 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91ef7fe700 1 --2- 172.21.15.32:0/2942086161 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91f00b1980 0x7f91f00b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:38.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91ef7fe700 1 -- 172.21.15.32:0/2942086161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91f005f6f0 msgr2=0x7f91f0064480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:38.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91ef7fe700 1 --2- 172.21.15.32:0/2942086161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91f005f6f0 0x7f91f0064480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:38.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91ef7fe700 1 -- 172.21.15.32:0/2942086161 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f0063490 con 0x7f91f0146a70 2020-12-05T18:12:38.545 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91eeffd700 1 --2- 172.21.15.32:0/2942086161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91f005f6f0 0x7f91f0064480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-12-05T18:12:38.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91ef7fe700 1 --2- 172.21.15.32:0/2942086161 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91f0146a70 0x7f91f0062fa0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f91f006a370 tx=0x7f91e8007e60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:38.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91e7fff700 1 -- 172.21.15.32:0/2942086161 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91e800e4a0 con 0x7f91f0146a70 2020-12-05T18:12:38.546 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91e7fff700 1 -- 172.21.15.32:0/2942086161 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e800e600 con 0x7f91f0146a70 2020-12-05T18:12:38.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91e7fff700 1 -- 172.21.15.32:0/2942086161 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91e800e960 con 0x7f91f0146a70 2020-12-05T18:12:38.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91effff700 1 -- 172.21.15.32:0/2942086161 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f00641d0 con 0x7f91f0146a70 2020-12-05T18:12:38.547 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.532+0000 7f91effff700 1 -- 172.21.15.32:0/2942086161 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f0065330 con 0x7f91f0146a70 2020-12-05T18:12:38.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.536+0000 7f91effff700 1 -- 172.21.15.32:0/2942086161 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f0065a90 con 0x7f91f0146a70 2020-12-05T18:12:38.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.536+0000 7f91e7fff700 1 -- 172.21.15.32:0/2942086161 <== mon.2 v2:172.21.15.61:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f91e8019030 con 0x7f91f0146a70 2020-12-05T18:12:38.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.536+0000 7f91e7fff700 1 --2- 172.21.15.32:0/2942086161 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f91d4055440 0x7f91d4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:38.548 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.536+0000 7f91eeffd700 1 --2- 172.21.15.32:0/2942086161 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f91d4055440 0x7f91d4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:38.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.536+0000 7f91e7fff700 1 -- 172.21.15.32:0/2942086161 <== mon.2 v2:172.21.15.61:3301/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f91e806d330 con 0x7f91f0146a70 2020-12-05T18:12:38.549 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.536+0000 7f91eeffd700 1 --2- 172.21.15.32:0/2942086161 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f91d4055440 0x7f91d4057900 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7f91e000b000 tx=0x7f91e0006dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:38.550 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.548+0000 7f91e7fff700 1 -- 172.21.15.32:0/2942086161 <== mon.2 v2:172.21.15.61:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f91e8043640 con 0x7f91f0146a70 2020-12-05T18:12:38.690 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.688+0000 7f91effff700 1 -- 172.21.15.32:0/2942086161 --> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f91f0115080 con 0x7f91d4055440 2020-12-05T18:12:38.698 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.692+0000 7f91e7fff700 1 -- 172.21.15.32:0/2942086161 <== mgr.4101 v2:172.21.15.32:6800/11355 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+66266 (secure 0 0 0) 0x7f91f0115080 con 0x7f91d4055440 2020-12-05T18:12:38.698 INFO:teuthology.orchestra.run.smithi032.stdout: 2020-12-05T18:12:38.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.696+0000 7f91effff700 1 -- 172.21.15.32:0/2942086161 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f91d4055440 msgr2=0x7f91d4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:38.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.696+0000 7f91effff700 1 --2- 172.21.15.32:0/2942086161 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f91d4055440 0x7f91d4057900 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7f91e000b000 tx=0x7f91e0006dc0).stop 2020-12-05T18:12:38.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.696+0000 7f91effff700 1 -- 172.21.15.32:0/2942086161 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91f0146a70 msgr2=0x7f91f0062fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:38.700 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.696+0000 7f91effff700 1 --2- 172.21.15.32:0/2942086161 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91f0146a70 0x7f91f0062fa0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f91f006a370 tx=0x7f91e8007e60).stop 2020-12-05T18:12:38.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.696+0000 7f91effff700 1 -- 172.21.15.32:0/2942086161 shutdown_connections 2020-12-05T18:12:38.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.696+0000 7f91effff700 1 --2- 172.21.15.32:0/2942086161 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f91d4055440 0x7f91d4057900 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:38.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.696+0000 7f91effff700 1 --2- 172.21.15.32:0/2942086161 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91f0146a70 0x7f91f0062fa0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:38.701 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.696+0000 7f91effff700 1 --2- 172.21.15.32:0/2942086161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91f005f6f0 0x7f91f0064480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:38.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.696+0000 7f91effff700 1 --2- 172.21.15.32:0/2942086161 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91f00b1980 0x7f91f00b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:38.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.696+0000 7f91effff700 1 -- 172.21.15.32:0/2942086161 >> 172.21.15.32:0/2942086161 conn(0x7f91f00604c0 msgr2=0x7f91f0144970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:38.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.696+0000 7f91effff700 1 -- 172.21.15.32:0/2942086161 shutdown_connections 2020-12-05T18:12:38.702 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.696+0000 7f91effff700 1 -- 172.21.15.32:0/2942086161 wait complete. 2020-12-05T18:12:38.703 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2020-12-05T18:12:38.712 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":1379,"stamp":"2020-12-05T18:12:37.526338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1180,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":8403712,"kb_used_data":15040,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746571008,"statfs":{"total":773094113280,"available":764488712192,"internally_reserved":8589934592,"allocated":15400960,"data_stored":7505640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003547"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":"132","reported_epoch":"151","state":"active+clean","last_fresh":"2020-12-05T18:12:16.668618+0000","last_change":"2020-12-05T18:12:16.668796+0000","last_active":"2020-12-05T18:12:16.668618+0000","last_peered":"2020-12-05T18:12:16.668618+0000","last_clean":"2020-12-05T18:12:16.668618+0000","last_became_active":"2020-12-05T17:31:35.373231+0000","last_became_peered":"2020-12-05T17:31:35.373231+0000","last_unstale":"2020-12-05T18:12:16.668618+0000","last_undegraded":"2020-12-05T18:12:16.668618+0000","last_fullsized":"2020-12-05T18:12:16.668618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T18:12:16.668608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T18:12:16.668608+0000","last_clean_scrub_stamp":"2020-12-05T18:12:16.668608+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"137","reported_epoch":"151","state":"active+clean","last_fresh":"2020-12-05T18:12:22.738495+0000","last_change":"2020-12-05T18:12:22.738617+0000","last_active":"2020-12-05T18:12:22.738495+0000","last_peered":"2020-12-05T18:12:22.738495+0000","last_clean":"2020-12-05T18:12:22.738495+0000","last_became_active":"2020-12-05T17:31:35.123169+0000","last_became_peered":"2020-12-05T17:31:35.123169+0000","last_unstale":"2020-12-05T18:12:22.738495+0000","last_undegraded":"2020-12-05T18:12:22.738495+0000","last_fullsized":"2020-12-05T18:12:22.738495+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T18:12:22.738478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T18:12:22.738478+0000","last_clean_scrub_stamp":"2020-12-05T18:12:22.738478+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"131","reported_epoch":"151","state":"active+clean","last_fresh":"2020-12-05T18:12:17.711248+0000","last_change":"2020-12-05T18:12:17.711445+0000","last_active":"2020-12-05T18:12:17.711248+0000","last_peered":"2020-12-05T18:12:17.711248+0000","last_clean":"2020-12-05T18:12:17.711248+0000","last_became_active":"2020-12-05T17:31:35.617427+0000","last_became_peered":"2020-12-05T17:31:35.617427+0000","last_unstale":"2020-12-05T18:12:17.711248+0000","last_undegraded":"2020-12-05T18:12:17.711248+0000","last_fullsized":"2020-12-05T18:12:17.711248+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T18:12:17.711235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T18:12:17.711235+0000","last_clean_scrub_stamp":"2020-12-05T18:12:17.711235+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"137","reported_epoch":"151","state":"active+clean","last_fresh":"2020-12-05T18:12:25.656988+0000","last_change":"2020-12-05T18:12:25.657097+0000","last_active":"2020-12-05T18:12:25.656988+0000","last_peered":"2020-12-05T18:12:25.656988+0000","last_clean":"2020-12-05T18:12:25.656988+0000","last_became_active":"2020-12-05T17:31:35.123671+0000","last_became_peered":"2020-12-05T17:31:35.123671+0000","last_unstale":"2020-12-05T18:12:25.656988+0000","last_undegraded":"2020-12-05T18:12:25.656988+0000","last_fullsized":"2020-12-05T18:12:25.656988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T18:12:25.656983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T18:12:25.656983+0000","last_clean_scrub_stamp":"2020-12-05T18:12:25.656983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"151'4","reported_seq":"144","reported_epoch":"151","state":"active+clean","last_fresh":"2020-12-05T18:12:21.678013+0000","last_change":"2020-12-05T18:12:21.678172+0000","last_active":"2020-12-05T18:12:21.678013+0000","last_peered":"2020-12-05T18:12:21.678013+0000","last_clean":"2020-12-05T18:12:21.678013+0000","last_became_active":"2020-12-05T17:31:30.405245+0000","last_became_peered":"2020-12-05T17:31:30.405245+0000","last_unstale":"2020-12-05T18:12:21.678013+0000","last_undegraded":"2020-12-05T18:12:21.678013+0000","last_fullsized":"2020-12-05T18:12:21.678013+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'4","last_scrub_stamp":"2020-12-05T18:12:21.678001+0000","last_deep_scrub":"151'4","last_deep_scrub_stamp":"2020-12-05T18:12:21.678001+0000","last_clean_scrub_stamp":"2020-12-05T18:12:21.678001+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1180,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"137","reported_epoch":"151","state":"active+clean","last_fresh":"2020-12-05T18:12:25.723070+0000","last_change":"2020-12-05T18:12:25.723267+0000","last_active":"2020-12-05T18:12:25.723070+0000","last_peered":"2020-12-05T18:12:25.723070+0000","last_clean":"2020-12-05T18:12:25.723070+0000","last_became_active":"2020-12-05T17:31:35.118441+0000","last_became_peered":"2020-12-05T17:31:35.118441+0000","last_unstale":"2020-12-05T18:12:25.723070+0000","last_undegraded":"2020-12-05T18:12:25.723070+0000","last_fullsized":"2020-12-05T18:12:25.723070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T18:12:25.723059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T18:12:25.723059+0000","last_clean_scrub_stamp":"2020-12-05T18:12:25.723059+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"137","reported_epoch":"151","state":"active+clean","last_fresh":"2020-12-05T18:12:27.699131+0000","last_change":"2020-12-05T18:12:27.699322+0000","last_active":"2020-12-05T18:12:27.699131+0000","last_peered":"2020-12-05T18:12:27.699131+0000","last_clean":"2020-12-05T18:12:27.699131+0000","last_became_active":"2020-12-05T17:31:35.374461+0000","last_became_peered":"2020-12-05T17:31:35.374461+0000","last_unstale":"2020-12-05T18:12:27.699131+0000","last_undegraded":"2020-12-05T18:12:27.699131+0000","last_fullsized":"2020-12-05T18:12:27.699131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T18:12:27.699118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T18:12:27.699118+0000","last_clean_scrub_stamp":"2020-12-05T18:12:27.699118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"132","reported_epoch":"151","state":"active+clean","last_fresh":"2020-12-05T18:12:20.636196+0000","last_change":"2020-12-05T18:12:20.636315+0000","last_active":"2020-12-05T18:12:20.636196+0000","last_peered":"2020-12-05T18:12:20.636196+0000","last_clean":"2020-12-05T18:12:20.636196+0000","last_became_active":"2020-12-05T17:31:35.122972+0000","last_became_peered":"2020-12-05T17:31:35.122972+0000","last_unstale":"2020-12-05T18:12:20.636196+0000","last_undegraded":"2020-12-05T18:12:20.636196+0000","last_fullsized":"2020-12-05T18:12:20.636196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T18:12:20.636190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T18:12:20.636190+0000","last_clean_scrub_stamp":"2020-12-05T18:12:20.636190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"133","reported_epoch":"151","state":"active+clean","last_fresh":"2020-12-05T18:12:21.864923+0000","last_change":"2020-12-05T18:12:21.865007+0000","last_active":"2020-12-05T18:12:21.864923+0000","last_peered":"2020-12-05T18:12:21.864923+0000","last_clean":"2020-12-05T18:12:21.864923+0000","last_became_active":"2020-12-05T17:31:35.126829+0000","last_became_peered":"2020-12-05T17:31:35.126829+0000","last_unstale":"2020-12-05T18:12:21.864923+0000","last_undegraded":"2020-12-05T18:12:21.864923+0000","last_fullsized":"2020-12-05T18:12:21.864923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-05T18:12:21.864919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-05T18:12:21.864919+0000","last_clean_scrub_stamp":"2020-12-05T18:12:21.864919+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1180,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542639,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":1.7090000000000001,"5min":1.869,"15min":1.8340000000000001},"min":{"1min":1.242,"5min":1.1910000000000001,"15min":1.163},"max":{"1min":2.6299999999999999,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":2.8730000000000002},{"interface":"front","average":{"1min":1.847,"5min":1.8939999999999999,"15min":1.8440000000000001},"min":{"1min":1.4159999999999999,"5min":1.264,"15min":1.0920000000000001},"max":{"1min":3.0579999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":1.5840000000000001}]},{"osd":1,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":1.9299999999999999,"5min":1.8959999999999999,"15min":1.8700000000000001},"min":{"1min":1.452,"5min":1.3779999999999999,"15min":1.24},"max":{"1min":2.8100000000000001,"5min":3.1019999999999999,"15min":3.444},"last":2.383},{"interface":"front","average":{"1min":2.5369999999999999,"5min":2.3290000000000002,"15min":2.3140000000000001},"min":{"1min":1.669,"5min":1.3979999999999999,"15min":1.3979999999999999},"max":{"1min":3.472,"5min":3.472,"15min":3.6629999999999998},"last":1.9630000000000001}]},{"osd":2,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":2.2789999999999999,"5min":2.3159999999999998,"15min":2.2530000000000001},"min":{"1min":1.597,"5min":1.5249999999999999,"15min":1.296},"max":{"1min":3.2669999999999999,"5min":3.6469999999999998,"15min":3.6469999999999998},"last":2.0369999999999999},{"interface":"front","average":{"1min":2.2410000000000001,"5min":2.3220000000000001,"15min":2.2949999999999999},"min":{"1min":1.7250000000000001,"5min":1.5289999999999999,"15min":1.161},"max":{"1min":3.161,"5min":3.6850000000000001,"15min":3.968},"last":1.827}]},{"osd":3,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":2.524,"5min":2.6040000000000001,"15min":2.6110000000000002},"min":{"1min":1.9419999999999999,"5min":1.6100000000000001,"15min":1.6100000000000001},"max":{"1min":3.6139999999999999,"5min":3.6139999999999999,"15min":3.7709999999999999},"last":2.4940000000000002},{"interface":"front","average":{"1min":2.3690000000000002,"5min":2.5249999999999999,"15min":2.4830000000000001},"min":{"1min":1.673,"5min":1.6220000000000001,"15min":1.6220000000000001},"max":{"1min":3.556,"5min":4.6909999999999998,"15min":4.6909999999999998},"last":2.5979999999999999}]},{"osd":4,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":2.665,"5min":2.718,"15min":2.7090000000000001},"min":{"1min":1.8720000000000001,"5min":1.706,"15min":1.6859999999999999},"max":{"1min":3.98,"5min":3.98,"15min":4.0739999999999998},"last":2.335},{"interface":"front","average":{"1min":2.8620000000000001,"5min":2.9300000000000002,"15min":2.9169999999999998},"min":{"1min":1.861,"5min":1.5189999999999999,"15min":1.5189999999999999},"max":{"1min":3.9020000000000001,"5min":3.9020000000000001,"15min":4.0490000000000004},"last":3.2850000000000001}]},{"osd":5,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":2.7650000000000001,"5min":2.8359999999999999,"15min":2.8380000000000001},"min":{"1min":1.929,"5min":1.7929999999999999,"15min":1.581},"max":{"1min":3.806,"5min":5.1050000000000004,"15min":5.1050000000000004},"last":2.1699999999999999},{"interface":"front","average":{"1min":3.109,"5min":3.117,"15min":3.113},"min":{"1min":2.4780000000000002,"5min":1.823,"15min":1.823},"max":{"1min":4.2380000000000004,"5min":4.335,"15min":4.375},"last":2.7679999999999998}]},{"osd":6,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":3.226,"5min":3.383,"15min":3.3220000000000001},"min":{"1min":2.7200000000000002,"5min":2.234,"15min":2.194},"max":{"1min":4.2160000000000002,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":2.5590000000000002},{"interface":"front","average":{"1min":3.2040000000000002,"5min":3.4079999999999999,"15min":3.3300000000000001},"min":{"1min":2.7509999999999999,"5min":2.395,"15min":2.395},"max":{"1min":4.1150000000000002,"5min":5.8940000000000001,"15min":5.8940000000000001},"last":3.6800000000000002}]}]},{"osd":6,"up_from":14,"seq":60129542640,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 5 18:11:53 2020","interfaces":[{"interface":"back","average":{"1min":1.744,"5min":1.772,"15min":1.7090000000000001},"min":{"1min":1.3959999999999999,"5min":1.1499999999999999,"15min":1.022},"max":{"1min":3.036,"5min":3.794,"15min":3.794},"last":2.8319999999999999},{"interface":"front","average":{"1min":1.8120000000000001,"5min":1.921,"15min":1.8480000000000001},"min":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.0760000000000001},"max":{"1min":2.3559999999999999,"5min":5.1349999999999998,"15min":5.1349999999999998},"last":1.494}]},{"osd":1,"last update":"Sat Dec 5 18:11:53 2020","interfaces":[{"interface":"back","average":{"1min":1.774,"5min":1.885,"15min":1.855},"min":{"1min":1.403,"5min":1.403,"15min":1.224},"max":{"1min":2.1629999999999998,"5min":4.3029999999999999,"15min":4.3029999999999999},"last":2.2200000000000002},{"interface":"front","average":{"1min":2.2480000000000002,"5min":2.234,"15min":2.1930000000000001},"min":{"1min":1.667,"5min":1.3420000000000001,"15min":1.125},"max":{"1min":2.891,"5min":3.258,"15min":3.4620000000000002},"last":2.0470000000000002}]},{"osd":2,"last update":"Sat Dec 5 18:11:53 2020","interfaces":[{"interface":"back","average":{"1min":2.2040000000000002,"5min":2.331,"15min":2.266},"min":{"1min":1.512,"5min":1.456,"15min":1.393},"max":{"1min":3.4319999999999999,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":1.873},{"interface":"front","average":{"1min":2.544,"5min":2.6000000000000001,"15min":2.581},"min":{"1min":1.9490000000000001,"5min":1.7849999999999999,"15min":1.5580000000000001},"max":{"1min":3.3599999999999999,"5min":4.6989999999999998,"15min":4.6989999999999998},"last":2.9990000000000001}]},{"osd":3,"last update":"Sat Dec 5 18:11:53 2020","interfaces":[{"interface":"back","average":{"1min":2.6640000000000001,"5min":2.6389999999999998,"15min":2.544},"min":{"1min":1.8919999999999999,"5min":1.8320000000000001,"15min":1.643},"max":{"1min":3.702,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":2.415},{"interface":"front","average":{"1min":2.8260000000000001,"5min":2.71,"15min":2.661},"min":{"1min":1.9870000000000001,"5min":1.5129999999999999,"15min":1.298},"max":{"1min":3.8929999999999998,"5min":4.4729999999999999,"15min":4.4729999999999999},"last":2.694}]},{"osd":4,"last update":"Sat Dec 5 18:11:53 2020","interfaces":[{"interface":"back","average":{"1min":2.9740000000000002,"5min":2.8380000000000001,"15min":2.786},"min":{"1min":2.1909999999999998,"5min":1.117,"15min":1.117},"max":{"1min":4.0919999999999996,"5min":4.1779999999999999,"15min":4.1779999999999999},"last":1.6619999999999999},{"interface":"front","average":{"1min":3.0350000000000001,"5min":2.9319999999999999,"15min":2.8969999999999998},"min":{"1min":2.5430000000000001,"5min":1.7210000000000001,"15min":1.264},"max":{"1min":4.2670000000000003,"5min":5.5330000000000004,"15min":5.5330000000000004},"last":2.3039999999999998}]},{"osd":5,"last update":"Sat Dec 5 18:11:53 2020","interfaces":[{"interface":"back","average":{"1min":2.6110000000000002,"5min":2.6389999999999998,"15min":2.6230000000000002},"min":{"1min":1.96,"5min":1.784,"15min":1.165},"max":{"1min":3.653,"5min":4.7699999999999996,"15min":4.7699999999999996},"last":2.6160000000000001},{"interface":"front","average":{"1min":3.2490000000000001,"5min":3.1850000000000001,"15min":3.1459999999999999},"min":{"1min":2.5830000000000002,"5min":1.429,"15min":1.429},"max":{"1min":4.4589999999999996,"5min":4.5,"15min":4.5},"last":3.2469999999999999}]},{"osd":7,"last update":"Sat Dec 5 18:11:53 2020","interfaces":[{"interface":"back","average":{"1min":3.4740000000000002,"5min":3.3380000000000001,"15min":3.2919999999999998},"min":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.077},"max":{"1min":4.6459999999999999,"5min":5.9379999999999997,"15min":5.9379999999999997},"last":3.0739999999999998},{"interface":"front","average":{"1min":3.23,"5min":3.222,"15min":3.202},"min":{"1min":2.5179999999999998,"5min":2.1659999999999999,"15min":2.0169999999999999},"max":{"1min":4.1219999999999999,"5min":4.9909999999999997,"15min":4.9909999999999997},"last":3.8799999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542639,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":1.766,"5min":1.714,"15min":1.7809999999999999},"min":{"1min":1.3919999999999999,"5min":1.383,"15min":1.093},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":3.3180000000000001},"last":1.8160000000000001},{"interface":"front","average":{"1min":1.671,"5min":1.8160000000000001,"15min":1.8120000000000001},"min":{"1min":1.502,"5min":1.393,"15min":1.1830000000000001},"max":{"1min":2.242,"5min":2.6040000000000001,"15min":2.9500000000000002},"last":1.6679999999999999}]},{"osd":2,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":1.6839999999999999,"5min":1.825,"15min":1.8009999999999999},"min":{"1min":1.4339999999999999,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.4159999999999999,"5min":3.3679999999999999,"15min":3.3679999999999999},"last":1.74},{"interface":"front","average":{"1min":2.4809999999999999,"5min":2.3210000000000002,"15min":2.2149999999999999},"min":{"1min":1.9430000000000001,"5min":1.546,"15min":1.0840000000000001},"max":{"1min":2.9769999999999999,"5min":3.2469999999999999,"15min":3.4849999999999999},"last":2.3119999999999998}]},{"osd":3,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":2.335,"5min":2.395,"15min":2.3900000000000001},"min":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.353},"max":{"1min":3.0600000000000001,"5min":3.379,"15min":3.3849999999999998},"last":2.1120000000000001},{"interface":"front","average":{"1min":2.3650000000000002,"5min":2.379,"15min":2.4039999999999999},"min":{"1min":1.8169999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"max":{"1min":2.843,"5min":3.3140000000000001,"15min":3.4020000000000001},"last":2.2010000000000001}]},{"osd":4,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":2.4140000000000001,"5min":2.3839999999999999,"15min":2.3610000000000002},"min":{"1min":2.0019999999999998,"5min":1.627,"15min":1.587},"max":{"1min":3.7879999999999998,"5min":4.0899999999999999,"15min":4.1429999999999998},"last":2.1600000000000001},{"interface":"front","average":{"1min":3.113,"5min":2.98,"15min":2.9380000000000002},"min":{"1min":2.7959999999999998,"5min":2.0800000000000001,"15min":1.5649999999999999},"max":{"1min":3.5270000000000001,"5min":3.9580000000000002,"15min":4.0119999999999996},"last":3.391}]},{"osd":5,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":2.7519999999999998,"5min":2.8690000000000002,"15min":2.8359999999999999},"min":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":1.75},"max":{"1min":3.4710000000000001,"5min":3.9860000000000002,"15min":4.1020000000000003},"last":2.8679999999999999},{"interface":"front","average":{"1min":2.7709999999999999,"5min":2.8530000000000002,"15min":2.847},"min":{"1min":2.024,"5min":2.024,"15min":2.008},"max":{"1min":3.702,"5min":3.75,"15min":3.919},"last":2.3650000000000002}]},{"osd":6,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":3.395,"5min":3.2679999999999998,"15min":3.2370000000000001},"min":{"1min":3.113,"5min":2.5880000000000001,"15min":2.1800000000000002},"max":{"1min":3.944,"5min":4.282,"15min":4.3879999999999999},"last":3.6139999999999999},{"interface":"front","average":{"1min":3.1720000000000002,"5min":3.222,"15min":3.1899999999999999},"min":{"1min":2.7090000000000001,"5min":2.5190000000000001,"15min":2.2879999999999998},"max":{"1min":3.9180000000000001,"5min":4.399,"15min":4.399},"last":3.2999999999999998}]},{"osd":7,"last update":"Sat Dec 5 18:11:46 2020","interfaces":[{"interface":"back","average":{"1min":3.3759999999999999,"5min":3.3999999999999999,"15min":3.3570000000000002},"min":{"1min":2.552,"5min":2.552,"15min":2.5059999999999998},"max":{"1min":4.0919999999999996,"5min":4.5279999999999996,"15min":4.5279999999999996},"last":3.6989999999999998},{"interface":"front","average":{"1min":3.1459999999999999,"5min":3.1850000000000001,"15min":3.153},"min":{"1min":2.5880000000000001,"5min":2.4470000000000001,"15min":2.2669999999999999},"max":{"1min":3.5990000000000002,"5min":4.2329999999999997,"15min":4.2329999999999997},"last":2.9809999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542640,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Dec 5 18:12:06 2020","interfaces":[{"interface":"back","average":{"1min":1.8140000000000001,"5min":1.7589999999999999,"15min":1.7709999999999999},"min":{"1min":1.4019999999999999,"5min":1.3149999999999999,"15min":1.0349999999999999},"max":{"1min":2.9239999999999999,"5min":2.9369999999999998,"15min":3.0510000000000002},"last":1.798},{"interface":"front","average":{"1min":1.8149999999999999,"5min":1.8180000000000001,"15min":1.831},"min":{"1min":1.53,"5min":1.2929999999999999,"15min":1.2929999999999999},"max":{"1min":3.0070000000000001,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":1.7090000000000001}]},{"osd":2,"last update":"Sat Dec 5 18:12:06 2020","interfaces":[{"interface":"back","average":{"1min":2.2410000000000001,"5min":2.2789999999999999,"15min":2.2160000000000002},"min":{"1min":1.488,"5min":1.3819999999999999,"15min":1.3480000000000001},"max":{"1min":2.7709999999999999,"5min":3.1890000000000001,"15min":3.2120000000000002},"last":2.0430000000000001},{"interface":"front","average":{"1min":2.6150000000000002,"5min":2.496,"15min":2.4830000000000001},"min":{"1min":2.0339999999999998,"5min":1.498,"15min":1.407},"max":{"1min":3.2080000000000002,"5min":3.3839999999999999,"15min":3.6600000000000001},"last":2.7509999999999999}]},{"osd":3,"last update":"Sat Dec 5 18:12:06 2020","interfaces":[{"interface":"back","average":{"1min":1.98,"5min":1.8859999999999999,"15min":1.8600000000000001},"min":{"1min":1.4930000000000001,"5min":1.385,"15min":1.2130000000000001},"max":{"1min":2.9039999999999999,"5min":2.9039999999999999,"15min":3.984},"last":1.8680000000000001},{"interface":"front","average":{"1min":2.395,"5min":2.476,"15min":2.496},"min":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"max":{"1min":2.895,"5min":3.7429999999999999,"15min":4.1790000000000003},"last":2.4420000000000002}]},{"osd":4,"last update":"Sat Dec 5 18:12:06 2020","interfaces":[{"interface":"back","average":{"1min":2.6880000000000002,"5min":2.5630000000000002,"15min":2.452},"min":{"1min":2.0600000000000001,"5min":1.669,"15min":1.4970000000000001},"max":{"1min":3.8540000000000001,"5min":3.9660000000000002,"15min":3.9660000000000002},"last":2.375},{"interface":"front","average":{"1min":2.4420000000000002,"5min":2.536,"15min":2.5070000000000001},"min":{"1min":1.802,"5min":1.397,"15min":1.397},"max":{"1min":3.7330000000000001,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":2.5209999999999999}]},{"osd":5,"last update":"Sat Dec 5 18:12:06 2020","interfaces":[{"interface":"back","average":{"1min":2.3849999999999998,"5min":2.5449999999999999,"15min":2.5129999999999999},"min":{"1min":1.917,"5min":1.917,"15min":1.4850000000000001},"max":{"1min":3.0449999999999999,"5min":3.6259999999999999,"15min":3.8260000000000001},"last":2.2360000000000002},{"interface":"front","average":{"1min":2.9529999999999998,"5min":3.0289999999999999,"15min":3.0350000000000001},"min":{"1min":2.5150000000000001,"5min":2.1829999999999998,"15min":1.9610000000000001},"max":{"1min":3.2509999999999999,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":3.2719999999999998}]},{"osd":6,"last update":"Sat Dec 5 18:12:06 2020","interfaces":[{"interface":"back","average":{"1min":3.25,"5min":3.2589999999999999,"15min":3.2490000000000001},"min":{"1min":2.8769999999999998,"5min":2.5369999999999999,"15min":2.2879999999999998},"max":{"1min":3.681,"5min":4.5709999999999997,"15min":5.3280000000000003},"last":3.48},{"interface":"front","average":{"1min":3.3300000000000001,"5min":3.0830000000000002,"15min":3.0630000000000002},"min":{"1min":2.758,"5min":2.2069999999999999,"15min":2.1709999999999998},"max":{"1min":3.77,"5min":3.7730000000000001,"15min":4.2519999999999998},"last":3.1779999999999999}]},{"osd":7,"last update":"Sat Dec 5 18:12:06 2020","interfaces":[{"interface":"back","average":{"1min":3.4399999999999999,"5min":3.3050000000000002,"15min":3.2639999999999998},"min":{"1min":2.75,"5min":2.5760000000000001,"15min":2.456},"max":{"1min":4,"5min":4.0599999999999996,"15min":4.3300000000000001},"last":3.5720000000000001},{"interface":"front","average":{"1min":3.0209999999999999,"5min":3.0579999999999998,"15min":3.028},"min":{"1min":2.6360000000000001,"5min":1.849,"15min":1.849},"max":{"1min":3.3170000000000002,"5min":3.9990000000000001,"15min":3.9990000000000001},"last":3.1190000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542639,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 5 18:11:49 2020","interfaces":[{"interface":"back","average":{"1min":1.714,"5min":1.849,"15min":1.8149999999999999},"min":{"1min":1.3440000000000001,"5min":1.171,"15min":1.141},"max":{"1min":2.3839999999999999,"5min":3.052,"15min":3.6659999999999999},"last":1.7150000000000001},{"interface":"front","average":{"1min":1.823,"5min":1.8069999999999999,"15min":1.806},"min":{"1min":1.452,"5min":1.3240000000000001,"15min":0.96499999999999997},"max":{"1min":2.246,"5min":3.202,"15min":3.5310000000000001},"last":1.762}]},{"osd":1,"last update":"Sat Dec 5 18:11:49 2020","interfaces":[{"interface":"back","average":{"1min":2.1560000000000001,"5min":2.1259999999999999,"15min":2.1219999999999999},"min":{"1min":1.097,"5min":1.097,"15min":1.097},"max":{"1min":2.9049999999999998,"5min":3.0800000000000001,"15min":3.3300000000000001},"last":2.0840000000000001},{"interface":"front","average":{"1min":2.4830000000000001,"5min":2.4849999999999999,"15min":2.52},"min":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.3009999999999999},"max":{"1min":3.1419999999999999,"5min":3.5230000000000001,"15min":3.5510000000000002},"last":2.8759999999999999}]},{"osd":3,"last update":"Sat Dec 5 18:11:49 2020","interfaces":[{"interface":"back","average":{"1min":1.899,"5min":1.857,"15min":1.8959999999999999},"min":{"1min":1.149,"5min":1.149,"15min":1.149},"max":{"1min":2.6930000000000001,"5min":2.6930000000000001,"15min":3.0470000000000002},"last":1.8700000000000001},{"interface":"front","average":{"1min":2.3849999999999998,"5min":2.403,"15min":2.4390000000000001},"min":{"1min":1.581,"5min":1.54,"15min":1.4530000000000001},"max":{"1min":3.0049999999999999,"5min":3.153,"15min":3.536},"last":2.3050000000000002}]},{"osd":4,"last update":"Sat Dec 5 18:11:49 2020","interfaces":[{"interface":"back","average":{"1min":2.5230000000000001,"5min":2.4489999999999998,"15min":2.3639999999999999},"min":{"1min":1.6990000000000001,"5min":1.502,"15min":1.431},"max":{"1min":3.8170000000000002,"5min":4.6159999999999997,"15min":4.6159999999999997},"last":2.4860000000000002},{"interface":"front","average":{"1min":2.5430000000000001,"5min":2.4630000000000001,"15min":2.3889999999999998},"min":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"max":{"1min":3.6309999999999998,"5min":3.8410000000000002,"15min":3.9009999999999998},"last":2.149}]},{"osd":5,"last update":"Sat Dec 5 18:11:49 2020","interfaces":[{"interface":"back","average":{"1min":2.875,"5min":2.9380000000000002,"15min":2.899},"min":{"1min":1.726,"5min":1.726,"15min":1.498},"max":{"1min":3.4079999999999999,"5min":4.1929999999999996,"15min":4.1929999999999996},"last":2.5760000000000001},{"interface":"front","average":{"1min":3.1549999999999998,"5min":3.0419999999999998,"15min":2.9670000000000001},"min":{"1min":2.4900000000000002,"5min":2.198,"15min":1.7509999999999999},"max":{"1min":3.9529999999999998,"5min":4.3289999999999997,"15min":4.3289999999999997},"last":3.3199999999999998}]},{"osd":6,"last update":"Sat Dec 5 18:11:49 2020","interfaces":[{"interface":"back","average":{"1min":2.9700000000000002,"5min":2.8940000000000001,"15min":2.8519999999999999},"min":{"1min":2.371,"5min":1.9810000000000001,"15min":1.476},"max":{"1min":3.895,"5min":4.476,"15min":4.476},"last":3},{"interface":"front","average":{"1min":3.173,"5min":3.23,"15min":3.1909999999999998},"min":{"1min":2.331,"5min":2.2810000000000001,"15min":1.6870000000000001},"max":{"1min":3.782,"5min":4.1399999999999997,"15min":4.2439999999999998},"last":2.7599999999999998}]},{"osd":7,"last update":"Sat Dec 5 18:11:49 2020","interfaces":[{"interface":"back","average":{"1min":3.2949999999999999,"5min":3.27,"15min":3.246},"min":{"1min":2.4900000000000002,"5min":2.4900000000000002,"15min":2.177},"max":{"1min":4.0049999999999999,"5min":4.1820000000000004,"15min":4.2709999999999999},"last":3.6890000000000001},{"interface":"front","average":{"1min":3.2629999999999999,"5min":3.1669999999999998,"15min":3.1360000000000001},"min":{"1min":2.7869999999999999,"5min":2.4950000000000001,"15min":1.885},"max":{"1min":3.6669999999999998,"5min":4.0780000000000003,"15min":4.0780000000000003},"last":3.2410000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542639,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.6319999999999999,"15min":1.722},"min":{"1min":1.3899999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":2.5579999999999998,"5min":2.5579999999999998,"15min":2.7349999999999999},"last":1.448},{"interface":"front","average":{"1min":1.7350000000000001,"5min":1.748,"15min":1.766},"min":{"1min":1.486,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":2.2010000000000001,"5min":2.6920000000000002,"15min":2.8399999999999999},"last":2.1970000000000001}]},{"osd":1,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":1.8959999999999999,"5min":1.75,"15min":1.768},"min":{"1min":1.3839999999999999,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":2.7400000000000002,"5min":2.7400000000000002,"15min":3.5539999999999998},"last":1.5009999999999999},{"interface":"front","average":{"1min":2.1469999999999998,"5min":2.2549999999999999,"15min":2.242},"min":{"1min":1.7729999999999999,"5min":1.228,"15min":1.228},"max":{"1min":2.5299999999999998,"5min":3.1269999999999998,"15min":3.2370000000000001},"last":2.1320000000000001}]},{"osd":2,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.4260000000000002,"5min":2.2770000000000001,"15min":2.3170000000000002},"min":{"1min":1.9730000000000001,"5min":1.4179999999999999,"15min":1.236},"max":{"1min":3.1659999999999999,"5min":3.2229999999999999,"15min":3.4470000000000001},"last":2.9689999999999999},{"interface":"front","average":{"1min":2.4079999999999999,"5min":2.4529999999999998,"15min":2.4620000000000002},"min":{"1min":1.954,"5min":1.7689999999999999,"15min":1.595},"max":{"1min":2.9449999999999998,"5min":3.323,"15min":3.3849999999999998},"last":1.8500000000000001}]},{"osd":4,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.5089999999999999,"5min":2.5369999999999999,"15min":2.6190000000000002},"min":{"1min":1.829,"5min":1.552,"15min":1.552},"max":{"1min":3.4409999999999998,"5min":3.4409999999999998,"15min":3.8300000000000001},"last":1.631},{"interface":"front","average":{"1min":2.2890000000000001,"5min":2.3530000000000002,"15min":2.4159999999999999},"min":{"1min":1.7909999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"max":{"1min":3.0009999999999999,"5min":3.8860000000000001,"15min":4.4039999999999999},"last":2.2730000000000001}]},{"osd":5,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.9649999999999999,"5min":2.9289999999999998,"15min":2.9590000000000001},"min":{"1min":2.411,"5min":1.9419999999999999,"15min":1.556},"max":{"1min":3.6469999999999998,"5min":4.1420000000000003,"15min":4.1420000000000003},"last":2.383},{"interface":"front","average":{"1min":2.964,"5min":2.9729999999999999,"15min":3.0470000000000002},"min":{"1min":2.4079999999999999,"5min":1.716,"15min":1.716},"max":{"1min":3.6469999999999998,"5min":3.9950000000000001,"15min":4.3300000000000001},"last":2.573}]},{"osd":6,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.8769999999999998,"5min":2.9039999999999999,"15min":2.919},"min":{"1min":2.2269999999999999,"5min":1.9950000000000001,"15min":1.972},"max":{"1min":3.6589999999999998,"5min":3.8199999999999998,"15min":5.3570000000000002},"last":2.7690000000000001},{"interface":"front","average":{"1min":3.149,"5min":3.181,"15min":3.2120000000000002},"min":{"1min":2.5739999999999998,"5min":2.1400000000000001,"15min":2.1400000000000001},"max":{"1min":3.7839999999999998,"5min":3.8900000000000001,"15min":4.0179999999999998},"last":2.6869999999999998}]},{"osd":7,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":3.1989999999999998,"5min":3.2389999999999999,"15min":3.2799999999999998},"min":{"1min":2.7570000000000001,"5min":2.1840000000000002,"15min":2.1840000000000002},"max":{"1min":3.9020000000000001,"5min":3.9510000000000001,"15min":4.665},"last":3.3159999999999998},{"interface":"front","average":{"1min":3.048,"5min":3.1259999999999999,"15min":3.1640000000000001},"min":{"1min":2.4359999999999999,"5min":2.2530000000000001,"15min":2.0209999999999999},"max":{"1min":3.669,"5min":4.0599999999999996,"15min":5.742},"last":3.1549999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542640,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 5 18:11:54 2020","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.5920000000000001,"15min":1.6879999999999999},"min":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"max":{"1min":2.6800000000000002,"5min":3.032,"15min":3.032},"last":1.448},{"interface":"front","average":{"1min":1.6970000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"min":{"1min":1.3049999999999999,"5min":1.2490000000000001,"15min":1.0529999999999999},"max":{"1min":2.718,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":1.5840000000000001}]},{"osd":1,"last update":"Sat Dec 5 18:11:54 2020","interfaces":[{"interface":"back","average":{"1min":2.081,"5min":2.2109999999999999,"15min":2.1589999999999998},"min":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.337},"max":{"1min":2.722,"5min":3.2290000000000001,"15min":3.3919999999999999},"last":2.2690000000000001},{"interface":"front","average":{"1min":2.5329999999999999,"5min":2.6560000000000001,"15min":2.6299999999999999},"min":{"1min":1.694,"5min":1.5049999999999999,"15min":1.5049999999999999},"max":{"1min":3.1859999999999999,"5min":3.6709999999999998,"15min":3.6709999999999998},"last":2.399}]},{"osd":2,"last update":"Sat Dec 5 18:11:54 2020","interfaces":[{"interface":"back","average":{"1min":2.0960000000000001,"5min":2.044,"15min":2.0150000000000001},"min":{"1min":1.631,"5min":1.5409999999999999,"15min":1.3380000000000001},"max":{"1min":3.153,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.8080000000000001},{"interface":"front","average":{"1min":2.4590000000000001,"5min":2.464,"15min":2.419},"min":{"1min":1.667,"5min":1.667,"15min":1.2849999999999999},"max":{"1min":3.129,"5min":3.452,"15min":3.681},"last":2.7200000000000002}]},{"osd":3,"last update":"Sat Dec 5 18:11:54 2020","interfaces":[{"interface":"back","average":{"1min":2.2160000000000002,"5min":2.1389999999999998,"15min":2.1680000000000001},"min":{"1min":1.6659999999999999,"5min":1.212,"15min":1.212},"max":{"1min":2.9870000000000001,"5min":3.085,"15min":3.677},"last":2.1909999999999998},{"interface":"front","average":{"1min":2.472,"5min":2.4660000000000002,"15min":2.4489999999999998},"min":{"1min":1.71,"5min":1.3380000000000001,"15min":1.3380000000000001},"max":{"1min":3.1110000000000002,"5min":3.4289999999999998,"15min":3.4740000000000002},"last":2.8050000000000002}]},{"osd":5,"last update":"Sat Dec 5 18:11:54 2020","interfaces":[{"interface":"back","average":{"1min":2.7309999999999999,"5min":2.7829999999999999,"15min":2.722},"min":{"1min":1.8879999999999999,"5min":1.8400000000000001,"15min":1.51},"max":{"1min":3.6549999999999998,"5min":3.9729999999999999,"15min":3.9910000000000001},"last":3.0110000000000001},{"interface":"front","average":{"1min":2.6230000000000002,"5min":2.621,"15min":2.617},"min":{"1min":1.764,"5min":1.764,"15min":1.5660000000000001},"max":{"1min":3.3570000000000002,"5min":3.8130000000000002,"15min":3.8130000000000002},"last":2.577}]},{"osd":6,"last update":"Sat Dec 5 18:11:54 2020","interfaces":[{"interface":"back","average":{"1min":2.8290000000000002,"5min":2.8519999999999999,"15min":2.8220000000000001},"min":{"1min":2.0569999999999999,"5min":1.7889999999999999,"15min":1.139},"max":{"1min":3.3439999999999999,"5min":3.7989999999999999,"15min":3.8740000000000001},"last":3.1190000000000002},{"interface":"front","average":{"1min":2.9929999999999999,"5min":3.069,"15min":3.0449999999999999},"min":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"max":{"1min":3.8969999999999998,"5min":4.0430000000000001,"15min":4.3760000000000003},"last":3.2410000000000001}]},{"osd":7,"last update":"Sat Dec 5 18:11:54 2020","interfaces":[{"interface":"back","average":{"1min":3.0329999999999999,"5min":3.044,"15min":3.0259999999999998},"min":{"1min":2.2999999999999998,"5min":2.1640000000000001,"15min":1.0580000000000001},"max":{"1min":3.6659999999999999,"5min":4.1340000000000003,"15min":4.2290000000000001},"last":2.9369999999999998},{"interface":"front","average":{"1min":3.2730000000000001,"5min":3.3079999999999998,"15min":3.2789999999999999},"min":{"1min":2.73,"5min":2.1699999999999999,"15min":1.611},"max":{"1min":3.7570000000000001,"5min":4.1779999999999999,"15min":4.3339999999999996},"last":3.3149999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542639,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050464,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321376,"statfs":{"total":96636764160,"available":95561089024,"internally_reserved":1073741824,"allocated":1925120,"data_stored":938205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.615},"min":{"1min":1.222,"5min":1.1200000000000001,"15min":1.1200000000000001},"max":{"1min":2.3490000000000002,"5min":2.7120000000000002,"15min":2.7570000000000001},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.647,"5min":1.78,"15min":1.7549999999999999},"min":{"1min":1.286,"5min":1.2509999999999999,"15min":1.1639999999999999},"max":{"1min":2.5910000000000002,"5min":3.1859999999999999,"15min":3.1859999999999999},"last":1.9179999999999999}]},{"osd":1,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.3559999999999999,"5min":2.4359999999999999,"15min":2.444},"min":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":0.997},"max":{"1min":3.3250000000000002,"5min":3.4870000000000001,"15min":3.4870000000000001},"last":3.145},{"interface":"front","average":{"1min":2.6219999999999999,"5min":2.5249999999999999,"15min":2.625},"min":{"1min":1.528,"5min":1.4159999999999999,"15min":1.292},"max":{"1min":3.3809999999999998,"5min":3.6459999999999999,"15min":4.1139999999999999},"last":2.778}]},{"osd":2,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.008,"5min":2.0619999999999998,"15min":2.0419999999999998},"min":{"1min":1.5980000000000001,"5min":1.3109999999999999,"15min":1.3109999999999999},"max":{"1min":3.2189999999999999,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.6140000000000001},{"interface":"front","average":{"1min":2.2730000000000001,"5min":2.2389999999999999,"15min":2.246},"min":{"1min":1.7410000000000001,"5min":1.52,"15min":1.478},"max":{"1min":3.3519999999999999,"5min":3.3519999999999999,"15min":3.468},"last":1.8440000000000001}]},{"osd":3,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.4649999999999999,"5min":2.3919999999999999,"15min":2.3730000000000002},"min":{"1min":1.8899999999999999,"5min":1.4330000000000001,"15min":1.4330000000000001},"max":{"1min":3.1179999999999999,"5min":3.6480000000000001,"15min":3.7810000000000001},"last":2.444},{"interface":"front","average":{"1min":2.5350000000000001,"5min":2.4689999999999999,"15min":2.4500000000000002},"min":{"1min":2.0190000000000001,"5min":1.5669999999999999,"15min":1.5669999999999999},"max":{"1min":3.4239999999999999,"5min":3.6629999999999998,"15min":3.7930000000000001},"last":2.383}]},{"osd":4,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.629,"5min":2.544,"15min":2.4990000000000001},"min":{"1min":2.0179999999999998,"5min":1.6240000000000001,"15min":1.6240000000000001},"max":{"1min":3.431,"5min":3.6520000000000001,"15min":3.8340000000000001},"last":2.7000000000000002},{"interface":"front","average":{"1min":2.8929999999999998,"5min":2.7930000000000001,"15min":2.7989999999999999},"min":{"1min":2.1549999999999998,"5min":1.8839999999999999,"15min":1.647},"max":{"1min":3.4209999999999998,"5min":4.0270000000000001,"15min":4.2789999999999999},"last":2.956}]},{"osd":6,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":2.9390000000000001,"5min":2.9590000000000001,"15min":2.879},"min":{"1min":2.2269999999999999,"5min":1.806,"15min":1.635},"max":{"1min":3.4209999999999998,"5min":3.992,"15min":4.093},"last":2.085},{"interface":"front","average":{"1min":2.9870000000000001,"5min":2.911,"15min":2.867},"min":{"1min":2.3100000000000001,"5min":1.274,"15min":1.274},"max":{"1min":3.609,"5min":3.7770000000000001,"15min":4.2030000000000003},"last":2.282}]},{"osd":7,"last update":"Sat Dec 5 18:11:37 2020","interfaces":[{"interface":"back","average":{"1min":3.29,"5min":3.1789999999999998,"15min":3.1469999999999998},"min":{"1min":2.556,"5min":1.754,"15min":1.754},"max":{"1min":3.7850000000000001,"5min":4.3959999999999999,"15min":4.625},"last":3.6019999999999999},{"interface":"front","average":{"1min":3.4289999999999998,"5min":3.3100000000000001,"15min":3.2959999999999998},"min":{"1min":2.859,"5min":2.226,"15min":2.153},"max":{"1min":4.0949999999999998,"5min":4.1970000000000001,"15min":4.556},"last":2.6040000000000001}]}]}]}} 2020-12-05T18:12:38.715 INFO:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2020-12-05T18:12:38.835 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.824+0000 7f11bb59e700 1 Processor -- start 2020-12-05T18:12:38.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.824+0000 7f11bb59e700 1 -- start start 2020-12-05T18:12:38.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11bb59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11bc061a60 0x7f11bc025a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:38.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11bb59e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f11bc0802c0 0x7f11bc153480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:38.836 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11bb59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11bc025800 con 0x7f11bc07fe40 2020-12-05T18:12:38.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11bb59e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11bc066610 con 0x7f11bc0802c0 2020-12-05T18:12:38.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11bb59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc061e40 con 0x7f11bc061a60 2020-12-05T18:12:38.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11ba59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f11bc07fe40 0x7f11bc152d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44916/0 (socket says 172.21.15.32:44916) 2020-12-05T18:12:38.837 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11ba59c700 1 -- 172.21.15.32:0/77856020 learned_addr learned my addr 172.21.15.32:0/77856020 (peer_addr_for_me v1:172.21.15.32:0/0) 2020-12-05T18:12:38.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11c0c73700 1 --2- 172.21.15.32:0/77856020 >> v2:172.21.15.61:3300/0 conn(0x7f11bc0802c0 0x7f11bc153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:38.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11bad9d700 1 --2- 172.21.15.32:0/77856020 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11bc061a60 0x7f11bc025a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:38.838 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11b9d9b700 1 -- 172.21.15.32:0/77856020 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695555115 0 0) 0x7f11bc025800 con 0x7f11bc07fe40 2020-12-05T18:12:38.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11b9d9b700 1 -- 172.21.15.32:0/77856020 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11a4003230 con 0x7f11bc07fe40 2020-12-05T18:12:38.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11b9d9b700 1 -- 172.21.15.32:0/77856020 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1774955766 0 0) 0x7f11a4003230 con 0x7f11bc07fe40 2020-12-05T18:12:38.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11b9d9b700 1 -- 172.21.15.32:0/77856020 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11bc061a60 msgr2=0x7f11bc025a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T18:12:38.839 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11b9d9b700 1 --2- 172.21.15.32:0/77856020 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11bc061a60 0x7f11bc025a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:38.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11b9d9b700 1 -- 172.21.15.32:0/77856020 >> v2:172.21.15.61:3300/0 conn(0x7f11bc0802c0 msgr2=0x7f11bc153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T18:12:38.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11b9d9b700 1 --2- 172.21.15.32:0/77856020 >> v2:172.21.15.61:3300/0 conn(0x7f11bc0802c0 0x7f11bc153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:38.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11b9d9b700 1 -- 172.21.15.32:0/77856020 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11bc066870 con 0x7f11bc07fe40 2020-12-05T18:12:38.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11b9d9b700 1 -- 172.21.15.32:0/77856020 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2063789944 0 0) 0x7f11b00016d0 con 0x7f11bc07fe40 2020-12-05T18:12:38.840 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11b9d9b700 1 -- 172.21.15.32:0/77856020 >> v1:172.21.15.32:6789/0 conn(0x7f11bc07fe40 legacy=0x7f11bc152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:38.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11b9d9b700 1 --2- 172.21.15.32:0/77856020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11a4003720 0x7f11a4003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:38.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11b9d9b700 1 --2- 172.21.15.32:0/77856020 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11a40044d0 0x7f11a4004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:38.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11b9d9b700 1 --2- 172.21.15.32:0/77856020 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11bc061a60 0x7f11a4004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:38.841 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11b9d9b700 1 -- 172.21.15.32:0/77856020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc066610 con 0x7f11a4003720 2020-12-05T18:12:38.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11b9d9b700 1 -- 172.21.15.32:0/77856020 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc061e40 con 0x7f11a40044d0 2020-12-05T18:12:38.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11b9d9b700 1 -- 172.21.15.32:0/77856020 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a4004260 con 0x7f11bc061a60 2020-12-05T18:12:38.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11bad9d700 1 --2- 172.21.15.32:0/77856020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11a4003720 0x7f11a4003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:38.842 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11ba59c700 1 --2- 172.21.15.32:0/77856020 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11a40044d0 0x7f11a4004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:38.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11c0c73700 1 --2- 172.21.15.32:0/77856020 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11bc061a60 0x7f11a4004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:38.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11bad9d700 1 -- 172.21.15.32:0/77856020 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11bc061a60 msgr2=0x7f11a4004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T18:12:38.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11bad9d700 1 --2- 172.21.15.32:0/77856020 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11bc061a60 0x7f11a4004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:38.843 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11bad9d700 1 -- 172.21.15.32:0/77856020 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11a40044d0 msgr2=0x7f11a4004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-12-05T18:12:38.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11bad9d700 1 --2- 172.21.15.32:0/77856020 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11a40044d0 0x7f11a4004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:38.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11bad9d700 1 -- 172.21.15.32:0/77856020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a4003230 con 0x7f11a4003720 2020-12-05T18:12:38.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11bad9d700 1 --2- 172.21.15.32:0/77856020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11a4003720 0x7f11a4003b40 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 rx=0x7f11ac00c000 tx=0x7f11ac0091a0).ready entity=mon.0 client_cookie=813891e6cd057038 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:38.844 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11b9d9b700 1 -- 172.21.15.32:0/77856020 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11ac00ec30 con 0x7f11a4003720 2020-12-05T18:12:38.845 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11b9d9b700 1 -- 172.21.15.32:0/77856020 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ac00ed90 con 0x7f11a4003720 2020-12-05T18:12:38.845 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11b9d9b700 1 -- 172.21.15.32:0/77856020 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11ac0125e0 con 0x7f11a4003720 2020-12-05T18:12:38.845 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11bb59e700 1 -- 172.21.15.32:0/77856020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11a4003720 msgr2=0x7f11a4003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:38.845 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11bb59e700 1 --2- 172.21.15.32:0/77856020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11a4003720 0x7f11a4003b40 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 rx=0x7f11ac00c000 tx=0x7f11ac0091a0).stop 2020-12-05T18:12:38.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.828+0000 7f11c0c73700 1 -- 172.21.15.32:0/77856020 reap_dead start 2020-12-05T18:12:38.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.832+0000 7f11bb59e700 1 -- 172.21.15.32:0/77856020 shutdown_connections 2020-12-05T18:12:38.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.832+0000 7f11bb59e700 1 -- 172.21.15.32:0/77856020 >> 172.21.15.32:0/77856020 conn(0x7f11bc0b6ab0 msgr2=0x7f11bc061010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:38.846 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.832+0000 7f11bb59e700 1 -- 172.21.15.32:0/77856020 shutdown_connections 2020-12-05T18:12:38.847 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.832+0000 7f11bb59e700 1 -- 172.21.15.32:0/77856020 wait complete. 2020-12-05T18:12:38.849 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.832+0000 7f11bb59e700 1 Processor -- start 2020-12-05T18:12:38.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.832+0000 7f11bb59e700 1 -- start start 2020-12-05T18:12:38.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.832+0000 7f11bb59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11bc0ae050 0x7f11bc09e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:38.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.832+0000 7f11bb59e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11bc09e120 0x7f11bc0649a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:38.850 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.836+0000 7f11bb59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11bc0a5f60 0x7f11bc0a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:38.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.836+0000 7f11bb59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc0aecf0 con 0x7f11bc0ae050 2020-12-05T18:12:38.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.836+0000 7f11bb59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc0a68d0 con 0x7f11bc0a5f60 2020-12-05T18:12:38.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.836+0000 7f11bb59e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc09f2f0 con 0x7f11bc09e120 2020-12-05T18:12:38.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.836+0000 7f11bad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11bc0ae050 0x7f11bc09e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:38.851 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.836+0000 7f11bad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11bc0ae050 0x7f11bc09e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42280/0 (socket says 172.21.15.32:42280) 2020-12-05T18:12:38.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.836+0000 7f11bad9d700 1 -- 172.21.15.32:0/1166708578 learned_addr learned my addr 172.21.15.32:0/1166708578 (peer_addr_for_me v2:172.21.15.32:0/0) 2020-12-05T18:12:38.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.836+0000 7f11ba59c700 1 --2- 172.21.15.32:0/1166708578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11bc09e120 0x7f11bc0649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:38.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.836+0000 7f11c0c73700 1 --2- 172.21.15.32:0/1166708578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11bc0a5f60 0x7f11bc0a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:38.852 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.836+0000 7f11bad9d700 1 -- 172.21.15.32:0/1166708578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11bc09e120 msgr2=0x7f11bc0649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:38.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.836+0000 7f11bad9d700 1 --2- 172.21.15.32:0/1166708578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11bc09e120 0x7f11bc0649a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:38.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.836+0000 7f11bad9d700 1 -- 172.21.15.32:0/1166708578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11bc0a5f60 msgr2=0x7f11bc0a63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:38.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.836+0000 7f11bad9d700 1 --2- 172.21.15.32:0/1166708578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11bc0a5f60 0x7f11bc0a63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:38.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.836+0000 7f11bad9d700 1 -- 172.21.15.32:0/1166708578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11bc09f590 con 0x7f11bc0ae050 2020-12-05T18:12:38.853 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.836+0000 7f11bad9d700 1 --2- 172.21.15.32:0/1166708578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11bc0ae050 0x7f11bc09e630 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 rx=0x7f11ac00c000 tx=0x7f11ac01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:38.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.836+0000 7f11ab7fe700 1 -- 172.21.15.32:0/1166708578 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11ac0127f0 con 0x7f11bc0ae050 2020-12-05T18:12:38.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.836+0000 7f11ab7fe700 1 -- 172.21.15.32:0/1166708578 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ac012950 con 0x7f11bc0ae050 2020-12-05T18:12:38.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.836+0000 7f11bb59e700 1 -- 172.21.15.32:0/1166708578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11bc0a5ac0 con 0x7f11bc0ae050 2020-12-05T18:12:38.854 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.836+0000 7f11ab7fe700 1 -- 172.21.15.32:0/1166708578 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11ac012cb0 con 0x7f11bc0ae050 2020-12-05T18:12:38.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.836+0000 7f11bb59e700 1 -- 172.21.15.32:0/1166708578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11bc0a8d20 con 0x7f11bc0ae050 2020-12-05T18:12:38.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.836+0000 7f11bb59e700 1 -- 172.21.15.32:0/1166708578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11bc0a5c50 con 0x7f11bc0ae050 2020-12-05T18:12:38.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.836+0000 7f11ab7fe700 1 -- 172.21.15.32:0/1166708578 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f11ac0243f0 con 0x7f11bc0ae050 2020-12-05T18:12:38.855 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.840+0000 7f11ab7fe700 1 --2- 172.21.15.32:0/1166708578 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f11a0055320 0x7f11a00577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-12-05T18:12:38.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.840+0000 7f11ab7fe700 1 -- 172.21.15.32:0/1166708578 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f11ac06f810 con 0x7f11bc0ae050 2020-12-05T18:12:38.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.840+0000 7f11ba59c700 1 --2- 172.21.15.32:0/1166708578 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f11a0055320 0x7f11a00577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-12-05T18:12:38.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.840+0000 7f11ba59c700 1 --2- 172.21.15.32:0/1166708578 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f11a0055320 0x7f11a00577e0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f11b0005000 tx=0x7f11b00077c0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-12-05T18:12:38.856 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.844+0000 7f11ab7fe700 1 -- 172.21.15.32:0/1166708578 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f11ac049660 con 0x7f11bc0ae050 2020-12-05T18:12:38.988 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.984+0000 7f11bb59e700 1 -- 172.21.15.32:0/1166708578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"} v 0) v1 -- 0x7f11bc057050 con 0x7f11bc0ae050 2020-12-05T18:12:38.996 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.992+0000 7f11ab7fe700 1 -- 172.21.15.32:0/1166708578 <== mon.0 v2:172.21.15.32:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f11ac049660 con 0x7f11bc0ae050 2020-12-05T18:12:38.997 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.992+0000 7f11ab7fe700 1 -- 172.21.15.32:0/1166708578 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}]=0 v25) v1 ==== 123+0+0 (secure 0 0 0) 0x7f11ac016070 con 0x7f11bc0ae050 2020-12-05T18:12:39.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.996+0000 7f11bb59e700 1 -- 172.21.15.32:0/1166708578 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f11a0055320 msgr2=0x7f11a00577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:39.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.996+0000 7f11bb59e700 1 --2- 172.21.15.32:0/1166708578 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f11a0055320 0x7f11a00577e0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f11b0005000 tx=0x7f11b00077c0).stop 2020-12-05T18:12:39.001 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.996+0000 7f11bb59e700 1 -- 172.21.15.32:0/1166708578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11bc0ae050 msgr2=0x7f11bc09e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-12-05T18:12:39.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.996+0000 7f11bb59e700 1 --2- 172.21.15.32:0/1166708578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11bc0ae050 0x7f11bc09e630 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 rx=0x7f11ac00c000 tx=0x7f11ac01a000).stop 2020-12-05T18:12:39.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.996+0000 7f11bb59e700 1 -- 172.21.15.32:0/1166708578 shutdown_connections 2020-12-05T18:12:39.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.996+0000 7f11bb59e700 1 --2- 172.21.15.32:0/1166708578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11bc0a5f60 0x7f11bc0a63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:39.002 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.996+0000 7f11bb59e700 1 --2- 172.21.15.32:0/1166708578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11bc0ae050 0x7f11bc09e630 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:39.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.996+0000 7f11bb59e700 1 --2- 172.21.15.32:0/1166708578 >> [v2:172.21.15.32:6800/11355,v1:172.21.15.32:6801/11355] conn(0x7f11a0055320 0x7f11a00577e0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:39.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.996+0000 7f11bb59e700 1 --2- 172.21.15.32:0/1166708578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11bc09e120 0x7f11bc0649a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-12-05T18:12:39.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.996+0000 7f11bb59e700 1 -- 172.21.15.32:0/1166708578 >> 172.21.15.32:0/1166708578 conn(0x7f11bc0b6ab0 msgr2=0x7f11bc0ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-12-05T18:12:39.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.996+0000 7f11bb59e700 1 -- 172.21.15.32:0/1166708578 shutdown_connections 2020-12-05T18:12:39.003 INFO:teuthology.orchestra.run.smithi032.stderr:2020-12-05T18:12:38.996+0000 7f11bb59e700 1 -- 172.21.15.32:0/1166708578 wait complete. 2020-12-05T18:12:39.015 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/ceph.py", line 1842, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 172, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/ceph_fuse.py", line 159, in task if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 147, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 683, in gevent._greenlet.Greenlet.get File "src/gevent/greenlet.py", line 317, in gevent._greenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 47, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 316, in copy_file_to copy_to_log(src, logger, capture=stream) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 273, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2020-12-05T18:12:39.016 INFO:teuthology.misc:Shutting down mds daemons... 2020-12-05T18:12:39.016 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-12-05T18:12:39.017 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T18:12:39.025 INFO:tasks.ceph.mds.a:Stopped 2020-12-05T18:12:39.026 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-12-05T18:12:39.027 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T18:12:39.042 INFO:tasks.ceph.mds.b:Stopped 2020-12-05T18:12:39.043 INFO:teuthology.misc:Shutting down osd daemons... 2020-12-05T18:12:39.044 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2020-12-05T18:12:39.044 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T18:12:45.146 INFO:tasks.ceph.osd.0:Stopped 2020-12-05T18:12:45.148 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2020-12-05T18:12:45.148 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T18:12:45.171 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2020-12-05T18:12:45.225 INFO:tasks.ceph.osd.1:Stopped 2020-12-05T18:12:45.225 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2020-12-05T18:12:45.226 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T18:12:45.294 INFO:tasks.ceph.osd.2:Stopped 2020-12-05T18:12:45.294 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2020-12-05T18:12:45.295 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T18:12:45.359 INFO:tasks.ceph.osd.3:Stopped 2020-12-05T18:12:45.360 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2020-12-05T18:12:45.361 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T18:12:45.425 INFO:tasks.ceph.osd.4:Stopped 2020-12-05T18:12:45.426 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2020-12-05T18:12:45.427 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T18:12:45.494 INFO:tasks.ceph.osd.5:Stopped 2020-12-05T18:12:45.495 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2020-12-05T18:12:45.496 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T18:12:45.575 INFO:tasks.ceph.osd.6:Stopped 2020-12-05T18:12:45.576 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2020-12-05T18:12:45.577 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T18:12:45.654 INFO:tasks.ceph.osd.7:Stopped 2020-12-05T18:12:45.655 INFO:teuthology.misc:Shutting down mgr daemons... 2020-12-05T18:12:45.655 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2020-12-05T18:12:45.656 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T18:12:45.679 INFO:tasks.ceph.mgr.y:Stopped 2020-12-05T18:12:45.680 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2020-12-05T18:12:45.681 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T18:12:45.706 INFO:tasks.ceph.mgr.x:Stopped 2020-12-05T18:12:45.707 INFO:teuthology.misc:Shutting down mon daemons... 2020-12-05T18:12:45.708 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2020-12-05T18:12:45.709 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T18:12:45.746 INFO:tasks.ceph.mon.a:Stopped 2020-12-05T18:12:45.747 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2020-12-05T18:12:45.747 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T18:12:45.771 INFO:tasks.ceph.mon.b:Stopped 2020-12-05T18:12:45.772 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2020-12-05T18:12:45.772 INFO:teuthology.orchestra.run:waiting for 300 2020-12-05T18:12:45.834 INFO:tasks.ceph.mon.c:Stopped 2020-12-05T18:12:45.835 INFO:tasks.ceph:Checking cluster log for badness... 2020-12-05T18:12:45.836 INFO:teuthology.orchestra.run.smithi032:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'evicting unresponsive client' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'has not responded to cap revoke by MDS for over' | egrep -v MDS_CLIENT_LATE_RELEASE | egrep -v 'responding to mclientcaps' | head -n 1 2020-12-05T18:12:45.857 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi032.front.sepia.ceph.com 2020-12-05T18:12:45.858 INFO:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2020-12-05T18:12:46.052 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi032.front.sepia.ceph.com 2020-12-05T18:12:46.053 INFO:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2020-12-05T18:12:46.200 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi032.front.sepia.ceph.com 2020-12-05T18:12:46.201 INFO:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2020-12-05T18:12:46.320 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi032.front.sepia.ceph.com 2020-12-05T18:12:46.321 INFO:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2020-12-05T18:12:46.544 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi061.front.sepia.ceph.com 2020-12-05T18:12:46.545 INFO:teuthology.orchestra.run.smithi061:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2020-12-05T18:12:46.737 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi061.front.sepia.ceph.com 2020-12-05T18:12:46.738 INFO:teuthology.orchestra.run.smithi061:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2020-12-05T18:12:46.877 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi061.front.sepia.ceph.com 2020-12-05T18:12:46.878 INFO:teuthology.orchestra.run.smithi061:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2020-12-05T18:12:46.985 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi061.front.sepia.ceph.com 2020-12-05T18:12:46.986 INFO:teuthology.orchestra.run.smithi061:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2020-12-05T18:12:47.121 INFO:tasks.ceph:Archiving mon data... 2020-12-05T18:12:47.123 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342/data/mon.a.tgz 2020-12-05T18:12:47.124 INFO:teuthology.orchestra.run.smithi032:> mktemp 2020-12-05T18:12:47.130 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.QIj04m71c8 2020-12-05T18:12:47.131 INFO:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.QIj04m71c8 2020-12-05T18:12:47.399 INFO:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.QIj04m71c8 2020-12-05T18:12:47.465 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.QIj04m71c8 is 594KB 2020-12-05T18:12:47.528 INFO:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.QIj04m71c8 2020-12-05T18:12:47.536 DEBUG:teuthology.misc:Transferring archived files from smithi061:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342/data/mon.b.tgz 2020-12-05T18:12:47.536 INFO:teuthology.orchestra.run.smithi061:> mktemp 2020-12-05T18:12:47.543 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/tmp.wlhFq7JMDo 2020-12-05T18:12:47.544 INFO:teuthology.orchestra.run.smithi061:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.wlhFq7JMDo 2020-12-05T18:12:47.847 INFO:teuthology.orchestra.run.smithi061:> sudo chmod 0666 /tmp/tmp.wlhFq7JMDo 2020-12-05T18:12:47.907 DEBUG:teuthology.orchestra.remote:smithi061:/tmp/tmp.wlhFq7JMDo is 779KB 2020-12-05T18:12:48.016 INFO:teuthology.orchestra.run.smithi061:> rm -fr /tmp/tmp.wlhFq7JMDo 2020-12-05T18:12:48.023 DEBUG:teuthology.misc:Transferring archived files from smithi061:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342/data/mon.c.tgz 2020-12-05T18:12:48.024 INFO:teuthology.orchestra.run.smithi061:> mktemp 2020-12-05T18:12:48.072 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/tmp.YvW4jD8erL 2020-12-05T18:12:48.072 INFO:teuthology.orchestra.run.smithi061:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.YvW4jD8erL 2020-12-05T18:12:48.375 INFO:teuthology.orchestra.run.smithi061:> sudo chmod 0666 /tmp/tmp.YvW4jD8erL 2020-12-05T18:12:48.446 DEBUG:teuthology.orchestra.remote:smithi061:/tmp/tmp.YvW4jD8erL is 779KB 2020-12-05T18:12:48.513 INFO:teuthology.orchestra.run.smithi061:> rm -fr /tmp/tmp.YvW4jD8erL 2020-12-05T18:12:48.520 INFO:tasks.ceph:Cleaning ceph cluster... 2020-12-05T18:12:48.520 INFO:teuthology.orchestra.run.smithi002:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2020-12-05T18:12:48.523 INFO:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2020-12-05T18:12:48.526 INFO:teuthology.orchestra.run.smithi036:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2020-12-05T18:12:48.528 INFO:teuthology.orchestra.run.smithi053:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2020-12-05T18:12:48.531 INFO:teuthology.orchestra.run.smithi061:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2020-12-05T18:12:48.576 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2020-12-05T18:12:48.577 INFO:teuthology.orchestra.run.smithi032:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2020-12-05T18:12:48.583 INFO:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2020-12-05T18:12:48.606 INFO:teuthology.orchestra.run.smithi032.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2020-12-05T18:12:48.619 INFO:teuthology.orchestra.run.smithi036:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2020-12-05T18:12:48.622 INFO:teuthology.orchestra.run.smithi002:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2020-12-05T18:12:48.625 INFO:teuthology.orchestra.run.smithi053:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2020-12-05T18:12:48.641 INFO:teuthology.orchestra.run.smithi036.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2020-12-05T18:12:48.644 INFO:teuthology.orchestra.run.smithi061.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2020-12-05T18:12:48.648 INFO:teuthology.orchestra.run.smithi053.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2020-12-05T18:12:48.648 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2020-12-05T18:12:48.656 INFO:tasks.ceph:Archiving crash dumps... 2020-12-05T18:12:48.659 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342/remote/smithi032/crash 2020-12-05T18:12:48.661 INFO:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2020-12-05T18:12:48.681 DEBUG:teuthology.misc:Transferring archived files from smithi061:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342/remote/smithi061/crash 2020-12-05T18:12:48.683 INFO:teuthology.orchestra.run.smithi061:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2020-12-05T18:12:48.711 DEBUG:teuthology.misc:Transferring archived files from smithi036:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342/remote/smithi036/crash 2020-12-05T18:12:48.713 INFO:teuthology.orchestra.run.smithi036:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2020-12-05T18:12:48.731 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342/remote/smithi002/crash 2020-12-05T18:12:48.733 INFO:teuthology.orchestra.run.smithi002:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2020-12-05T18:12:48.753 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342/remote/smithi053/crash 2020-12-05T18:12:48.755 INFO:teuthology.orchestra.run.smithi053:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2020-12-05T18:12:48.776 INFO:tasks.ceph:Shutting down logrotate 2020-12-05T18:12:48.776 INFO:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:12:48.797 INFO:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:12:48.800 INFO:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:12:48.802 INFO:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:12:48.819 INFO:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-05T18:12:48.856 INFO:teuthology.orchestra.run.smithi002:> sudo rm /etc/logrotate.d/ceph-test.conf 2020-12-05T18:12:48.890 INFO:teuthology.orchestra.run.smithi032:> sudo rm /etc/logrotate.d/ceph-test.conf 2020-12-05T18:12:48.905 INFO:teuthology.orchestra.run.smithi036:> sudo rm /etc/logrotate.d/ceph-test.conf 2020-12-05T18:12:48.921 INFO:teuthology.orchestra.run.smithi053:> sudo rm /etc/logrotate.d/ceph-test.conf 2020-12-05T18:12:48.936 INFO:teuthology.orchestra.run.smithi061:> sudo rm /etc/logrotate.d/ceph-test.conf 2020-12-05T18:12:48.951 INFO:tasks.ceph:Compressing logs... 2020-12-05T18:12:48.952 INFO:teuthology.orchestra.run.smithi002:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-12-05T18:12:48.954 INFO:teuthology.orchestra.run.smithi032:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-12-05T18:12:48.957 INFO:teuthology.orchestra.run.smithi036:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-12-05T18:12:48.965 INFO:teuthology.orchestra.run.smithi053:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-12-05T18:12:48.979 INFO:teuthology.orchestra.run.smithi061:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-12-05T18:12:50.172 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2020-12-05T18:13:13.602 INFO:tasks.ceph:Archiving logs... 2020-12-05T18:13:13.603 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/log/ceph to /home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342/remote/smithi032/log 2020-12-05T18:13:13.604 INFO:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /var/log/ceph -- . 2020-12-05T18:13:18.372 DEBUG:teuthology.misc:Transferring archived files from smithi061:/var/log/ceph to /home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342/remote/smithi061/log 2020-12-05T18:13:18.373 INFO:teuthology.orchestra.run.smithi061:> sudo tar cz -f - -C /var/log/ceph -- . 2020-12-05T18:13:21.767 DEBUG:teuthology.misc:Transferring archived files from smithi036:/var/log/ceph to /home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342/remote/smithi036/log 2020-12-05T18:13:21.769 INFO:teuthology.orchestra.run.smithi036:> sudo tar cz -f - -C /var/log/ceph -- . 2020-12-05T18:13:22.122 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/log/ceph to /home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342/remote/smithi002/log 2020-12-05T18:13:22.123 INFO:teuthology.orchestra.run.smithi002:> sudo tar cz -f - -C /var/log/ceph -- . 2020-12-05T18:13:22.153 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/log/ceph to /home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342/remote/smithi053/log 2020-12-05T18:13:22.154 INFO:teuthology.orchestra.run.smithi053:> sudo tar cz -f - -C /var/log/ceph -- . 2020-12-05T18:13:22.174 DEBUG:teuthology.run_tasks:Unwinding manager install 2020-12-05T18:13:22.185 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/install/__init__.py", line 617, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 172, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/ceph_fuse.py", line 159, in task if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 147, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 683, in gevent._greenlet.Greenlet.get File "src/gevent/greenlet.py", line 317, in gevent._greenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 47, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 316, in copy_file_to copy_to_log(src, logger, capture=stream) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 273, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2020-12-05T18:13:22.186 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits... 2020-12-05T18:13:22.187 INFO:teuthology.orchestra.run.smithi002:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2020-12-05T18:13:22.193 INFO:teuthology.orchestra.run.smithi032:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2020-12-05T18:13:22.196 INFO:teuthology.orchestra.run.smithi036:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2020-12-05T18:13:22.199 INFO:teuthology.orchestra.run.smithi053:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2020-12-05T18:13:22.215 INFO:teuthology.orchestra.run.smithi061:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2020-12-05T18:13:22.314 DEBUG:teuthology.run_tasks:Unwinding manager clock 2020-12-05T18:13:22.325 INFO:teuthology.task.clock:Checking final clock skew... 2020-12-05T18:13:22.326 INFO:teuthology.orchestra.run.smithi032:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-12-05T18:13:22.344 INFO:teuthology.orchestra.run.smithi032.stdout: remote refid st t when poll reach delay offset jitter 2020-12-05T18:13:22.345 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================== 2020-12-05T18:13:22.345 INFO:teuthology.orchestra.run.smithi032.stdout:+hv01.front.sepi 216.240.36.24 3 u 106 128 377 0.142 -0.550 0.675 2020-12-05T18:13:22.345 INFO:teuthology.orchestra.run.smithi032.stdout:*hv02.front.sepi 163.237.218.18 2 u 43 128 377 0.131 -0.577 0.402 2020-12-05T18:13:22.345 INFO:teuthology.orchestra.run.smithi032.stdout:-hv03.front.sepi 74.6.168.72 3 u 49 128 377 0.127 1.554 0.438 2020-12-05T18:13:22.346 INFO:teuthology.orchestra.run.smithi032.stdout:+hv04.front.sepi 144.172.126.201 3 u 36 128 377 0.132 -2.373 0.517 2020-12-05T18:13:22.347 INFO:teuthology.orchestra.run.smithi061:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-12-05T18:13:22.368 INFO:teuthology.orchestra.run.smithi061.stdout: remote refid st t when poll reach delay offset jitter 2020-12-05T18:13:22.369 INFO:teuthology.orchestra.run.smithi061.stdout:============================================================================== 2020-12-05T18:13:22.369 INFO:teuthology.orchestra.run.smithi061.stdout:+hv01.front.sepi 216.240.36.24 3 u 43 128 377 0.092 1.068 0.499 2020-12-05T18:13:22.369 INFO:teuthology.orchestra.run.smithi061.stdout:*hv02.front.sepi 163.237.218.18 2 u 105 128 377 0.071 2.449 0.732 2020-12-05T18:13:22.370 INFO:teuthology.orchestra.run.smithi061.stdout:-hv03.front.sepi 74.6.168.72 3 u 47 128 377 0.070 4.019 0.675 2020-12-05T18:13:22.370 INFO:teuthology.orchestra.run.smithi061.stdout:+hv04.front.sepi 144.172.126.201 3 u 49 128 377 0.084 1.238 1.084 2020-12-05T18:13:22.371 INFO:teuthology.orchestra.run.smithi036:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-12-05T18:13:22.393 INFO:teuthology.orchestra.run.smithi036.stdout: remote refid st t when poll reach delay offset jitter 2020-12-05T18:13:22.393 INFO:teuthology.orchestra.run.smithi036.stdout:============================================================================== 2020-12-05T18:13:22.393 INFO:teuthology.orchestra.run.smithi036.stdout:+hv01.front.sepi 216.240.36.24 3 u 98 128 377 0.100 -1.733 0.207 2020-12-05T18:13:22.394 INFO:teuthology.orchestra.run.smithi036.stdout:*hv02.front.sepi 163.237.218.18 2 u 105 128 377 0.097 -0.897 0.212 2020-12-05T18:13:22.394 INFO:teuthology.orchestra.run.smithi036.stdout:-hv03.front.sepi 74.6.168.72 3 u 109 128 377 0.070 0.704 0.220 2020-12-05T18:13:22.394 INFO:teuthology.orchestra.run.smithi036.stdout:+hv04.front.sepi 144.172.126.201 3 u 99 128 377 0.073 -2.533 0.205 2020-12-05T18:13:22.395 INFO:teuthology.orchestra.run.smithi002:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-12-05T18:13:22.418 INFO:teuthology.orchestra.run.smithi002.stdout: remote refid st t when poll reach delay offset jitter 2020-12-05T18:13:22.418 INFO:teuthology.orchestra.run.smithi002.stdout:============================================================================== 2020-12-05T18:13:22.418 INFO:teuthology.orchestra.run.smithi002.stdout:+hv01.front.sepi 216.240.36.24 3 u 20 128 377 0.113 0.480 0.579 2020-12-05T18:13:22.419 INFO:teuthology.orchestra.run.smithi002.stdout:*hv02.front.sepi 163.237.218.18 2 u 75 128 377 0.071 1.216 0.468 2020-12-05T18:13:22.419 INFO:teuthology.orchestra.run.smithi002.stdout:-hv03.front.sepi 74.6.168.72 3 u 79 128 377 0.066 2.957 0.596 2020-12-05T18:13:22.419 INFO:teuthology.orchestra.run.smithi002.stdout:+hv04.front.sepi 144.172.126.201 3 u 81 128 377 0.076 -0.584 0.357 2020-12-05T18:13:22.420 INFO:teuthology.orchestra.run.smithi053:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-12-05T18:13:22.442 INFO:teuthology.orchestra.run.smithi053.stdout: remote refid st t when poll reach delay offset jitter 2020-12-05T18:13:22.442 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================== 2020-12-05T18:13:22.443 INFO:teuthology.orchestra.run.smithi053.stdout:+hv01.front.sepi 216.240.36.24 3 u 78 128 377 0.110 -1.332 0.314 2020-12-05T18:13:22.443 INFO:teuthology.orchestra.run.smithi053.stdout:*hv02.front.sepi 163.237.218.18 2 u 87 128 377 0.058 -0.566 0.393 2020-12-05T18:13:22.443 INFO:teuthology.orchestra.run.smithi053.stdout:-hv03.front.sepi 74.6.168.72 3 u 22 128 377 0.081 1.188 0.174 2020-12-05T18:13:22.443 INFO:teuthology.orchestra.run.smithi053.stdout:+hv04.front.sepi 144.172.126.201 3 u 83 128 377 0.092 -2.053 0.232 2020-12-05T18:13:22.444 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2020-12-05T18:13:22.455 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2020-12-05T18:13:22.456 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2020-12-05T18:13:22.467 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2020-12-05T18:13:22.479 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2020-12-05T18:13:22.489 INFO:teuthology.task.internal:Duration was 2868.490571 seconds 2020-12-05T18:13:22.491 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2020-12-05T18:13:22.501 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2020-12-05T18:13:22.502 INFO:teuthology.orchestra.run.smithi002:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2020-12-05T18:13:22.505 INFO:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2020-12-05T18:13:22.507 INFO:teuthology.orchestra.run.smithi036:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2020-12-05T18:13:22.510 INFO:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2020-12-05T18:13:22.513 INFO:teuthology.orchestra.run.smithi061:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2020-12-05T18:13:22.564 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2020-12-05T18:13:22.565 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi032.front.sepia.ceph.com 2020-12-05T18:13:22.565 INFO:teuthology.orchestra.run.smithi032:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/*.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2020-12-05T18:13:22.578 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi061.front.sepia.ceph.com 2020-12-05T18:13:22.579 INFO:teuthology.orchestra.run.smithi061:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/*.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2020-12-05T18:13:22.592 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi036.front.sepia.ceph.com 2020-12-05T18:13:22.593 INFO:teuthology.orchestra.run.smithi036:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/*.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2020-12-05T18:13:22.608 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi002.front.sepia.ceph.com 2020-12-05T18:13:22.609 INFO:teuthology.orchestra.run.smithi002:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/*.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2020-12-05T18:13:22.624 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi053.front.sepia.ceph.com 2020-12-05T18:13:22.625 INFO:teuthology.orchestra.run.smithi053:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/*.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2020-12-05T18:13:22.638 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2020-12-05T18:13:22.638 INFO:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-12-05T18:13:22.670 INFO:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-12-05T18:13:22.672 INFO:teuthology.orchestra.run.smithi036:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-12-05T18:13:22.675 INFO:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-12-05T18:13:22.683 INFO:teuthology.orchestra.run.smithi061:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-12-05T18:13:22.711 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2020-12-05T18:13:22.722 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2020-12-05T18:13:22.723 INFO:teuthology.orchestra.run.smithi002:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2020-12-05T18:13:22.751 INFO:teuthology.orchestra.run.smithi032:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2020-12-05T18:13:22.767 INFO:teuthology.orchestra.run.smithi036:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2020-12-05T18:13:22.783 INFO:teuthology.orchestra.run.smithi053:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2020-12-05T18:13:22.799 INFO:teuthology.orchestra.run.smithi061:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2020-12-05T18:13:22.816 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2020-12-05T18:13:22.827 INFO:teuthology.orchestra.run.smithi002:> sudo sysctl -w kernel.core_pattern=core && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2020-12-05T18:13:22.830 INFO:teuthology.orchestra.run.smithi032:> sudo sysctl -w kernel.core_pattern=core && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2020-12-05T18:13:22.833 INFO:teuthology.orchestra.run.smithi036:> sudo sysctl -w kernel.core_pattern=core && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2020-12-05T18:13:22.835 INFO:teuthology.orchestra.run.smithi053:> sudo sysctl -w kernel.core_pattern=core && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2020-12-05T18:13:22.842 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = core 2020-12-05T18:13:22.844 INFO:teuthology.orchestra.run.smithi061:> sudo sysctl -w kernel.core_pattern=core && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2020-12-05T18:13:22.846 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = core 2020-12-05T18:13:22.847 INFO:teuthology.orchestra.run.smithi036.stdout:kernel.core_pattern = core 2020-12-05T18:13:22.856 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = core 2020-12-05T18:13:22.871 INFO:teuthology.orchestra.run.smithi061.stdout:kernel.core_pattern = core 2020-12-05T18:13:22.887 INFO:teuthology.orchestra.run.smithi032:> test -e /home/ubuntu/cephtest/archive/coredump 2020-12-05T18:13:22.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T18:13:22.911 INFO:teuthology.orchestra.run.smithi061:> test -e /home/ubuntu/cephtest/archive/coredump 2020-12-05T18:13:22.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T18:13:22.936 INFO:teuthology.orchestra.run.smithi036:> test -e /home/ubuntu/cephtest/archive/coredump 2020-12-05T18:13:22.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T18:13:22.943 INFO:teuthology.orchestra.run.smithi002:> test -e /home/ubuntu/cephtest/archive/coredump 2020-12-05T18:13:22.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T18:13:22.949 INFO:teuthology.orchestra.run.smithi053:> test -e /home/ubuntu/cephtest/archive/coredump 2020-12-05T18:13:22.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-05T18:13:22.955 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2020-12-05T18:13:22.967 INFO:teuthology.task.internal:Transferring archived files... 2020-12-05T18:13:22.968 DEBUG:teuthology.misc:Transferring archived files from smithi032:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342/remote/smithi032 2020-12-05T18:13:22.969 INFO:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2020-12-05T18:13:22.993 DEBUG:teuthology.misc:Transferring archived files from smithi061:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342/remote/smithi061 2020-12-05T18:13:22.994 INFO:teuthology.orchestra.run.smithi061:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2020-12-05T18:13:23.017 DEBUG:teuthology.misc:Transferring archived files from smithi036:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342/remote/smithi036 2020-12-05T18:13:23.018 INFO:teuthology.orchestra.run.smithi036:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2020-12-05T18:13:23.041 DEBUG:teuthology.misc:Transferring archived files from smithi002:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342/remote/smithi002 2020-12-05T18:13:23.042 INFO:teuthology.orchestra.run.smithi002:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2020-12-05T18:13:23.067 DEBUG:teuthology.misc:Transferring archived files from smithi053:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2020-11-30_04:10:01-fs-octopus-distro-basic-smithi/5667342/remote/smithi053 2020-12-05T18:13:23.068 INFO:teuthology.orchestra.run.smithi053:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2020-12-05T18:13:23.090 INFO:teuthology.task.internal:Removing archive directory... 2020-12-05T18:13:23.091 INFO:teuthology.orchestra.run.smithi002:> rm -rf -- /home/ubuntu/cephtest/archive 2020-12-05T18:13:23.110 INFO:teuthology.orchestra.run.smithi032:> rm -rf -- /home/ubuntu/cephtest/archive 2020-12-05T18:13:23.113 INFO:teuthology.orchestra.run.smithi036:> rm -rf -- /home/ubuntu/cephtest/archive 2020-12-05T18:13:23.116 INFO:teuthology.orchestra.run.smithi053:> rm -rf -- /home/ubuntu/cephtest/archive 2020-12-05T18:13:23.131 INFO:teuthology.orchestra.run.smithi061:> rm -rf -- /home/ubuntu/cephtest/archive 2020-12-05T18:13:23.139 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2020-12-05T18:13:23.152 INFO:teuthology.task.internal:Not uploading archives. 2020-12-05T18:13:23.153 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2020-12-05T18:13:23.165 INFO:teuthology.task.internal:Tidying up after the test... 2020-12-05T18:13:23.166 INFO:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2020-12-05T18:13:23.169 INFO:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2020-12-05T18:13:23.172 INFO:teuthology.orchestra.run.smithi036:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2020-12-05T18:13:23.174 INFO:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2020-12-05T18:13:23.176 INFO:teuthology.orchestra.run.smithi002.stdout: 524339 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 5 18:13 /home/ubuntu/cephtest 2020-12-05T18:13:23.177 INFO:teuthology.orchestra.run.smithi032.stdout: 524339 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 5 18:13 /home/ubuntu/cephtest 2020-12-05T18:13:23.180 INFO:teuthology.orchestra.run.smithi061:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2020-12-05T18:13:23.184 INFO:teuthology.orchestra.run.smithi053.stdout: 524339 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 5 18:13 /home/ubuntu/cephtest 2020-12-05T18:13:23.189 INFO:teuthology.orchestra.run.smithi061.stdout: 524339 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 5 18:13 /home/ubuntu/cephtest 2020-12-06T04:42:58.675 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2020-12-06T04:42:58.699 DEBUG:teuthology.task.console_log:Killing console logger for smithi032 2020-12-06T04:42:58.699 DEBUG:teuthology.task.console_log:Killing console logger for smithi061 2020-12-06T04:42:58.700 DEBUG:teuthology.task.console_log:Killing console logger for smithi036 2020-12-06T04:42:58.700 DEBUG:teuthology.task.console_log:Killing console logger for smithi002 2020-12-06T04:42:58.700 DEBUG:teuthology.task.console_log:Killing console logger for smithi053 2020-12-06T04:42:58.701 DEBUG:teuthology.task.console_log:Killing console logger for smithi032 2020-12-06T04:42:58.701 DEBUG:teuthology.task.console_log:Killing console logger for smithi061 2020-12-06T04:42:58.702 DEBUG:teuthology.task.console_log:Killing console logger for smithi036 2020-12-06T04:42:58.702 DEBUG:teuthology.task.console_log:Killing console logger for smithi002 2020-12-06T04:42:58.702 DEBUG:teuthology.task.console_log:Killing console logger for smithi053 2020-12-06T04:42:58.703 DEBUG:teuthology.exit:Finished running handlers